Support for libkrun using krunkit
#4137
Open
+741
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nirs
reviewed
Oct 4, 2025
afbjorklund
reviewed
Oct 5, 2025
|
@lima-vm/maintainers @lima-vm/committers Any help is appreciated! What am I doing wrong here that an IP is not getting assigned for the For # limactl start default --vm-type=qemu --network=lima:shared
➜ lima2 git:(driver/krun) ✗ arp -an | grep bridge100
? (192.168.105.1) at 52:ed:3c:d4:1e:64 on bridge100 ifscope permanent [bridge]
? (192.168.105.2) at 52:55:55:e0:48:67 on bridge100 ifscope [bridge] # VM IP
? (224.0.0.251) at 1:0:5e:0:0:fb on bridge100 ifscope permanent [ethernet]For ➜ limactl start default --vm-type=krunkit --network=lima:shared
WARN[0000] Template locator "template://_images/ubuntu" should be written "template:_images/ubuntu" since Lima v2.0
WARN[0000] Template locator "template://_default/mounts" should be written "template:_default/mounts" since Lima v2.0
? Creating an instance "default" Proceed with the current configuration
INFO[0001] Starting socket_vmnet daemon for "shared" network
INFO[0001] Starting the instance "default" with external VM driver "krunkit"
INFO[0001] Attempting to download the image arch=aarch64 digest="sha256:26d0ac2236f12954923eb35ddfee8fa9fff3eab6111ba84786b98ab3b972c6d8" location="https://cloud-images.ubuntu.com/releases/plucky/release-20250701/ubuntu-25.04-server-cloudimg-arm64.img"
INFO[0001] Using cache "/Users/ansumansahoo/Library/Caches/lima/download/by-url-sha256/eccac025a7a4709721a97d4ef1d5973546c85062917f188cdc6d247b665cbab5/data"
INFO[0004] Attempting to download the nerdctl archive arch=aarch64 digest="sha256:2f98346ed3dcaf47cfd6461a5685e582284329a1ece1d952ade881b9fe7491e8" location="https://github.com/containerd/nerdctl/releases/download/v2.1.6/nerdctl-full-2.1.6-linux-arm64.tar.gz"
INFO[0004] Using cache "/Users/ansumansahoo/Library/Caches/lima/download/by-url-sha256/d82077f1308e8d063805c4c1f761e1c3fd636e18cab7e25919a18e7a0838afa2/data"
INFO[0006] [hostagent] hostagent socket created at /Users/ansumansahoo/.lima/default/ha.sock
INFO[0006] SSH Local Port: 55283
INFO[0006] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
...
INFO[0598] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0598] [hostagent] Waiting for the essential requirement 2 of 2: "user session is ready for ssh"
FATA[0604] did not receive an event with the "running" status
➜ lima2 git:(driver/krun) ✗ arp -an | grep bridge100
? (192.168.105.1) at 52:ed:3c:d4:1e:64 on bridge100 ifscope permanent [bridge]
? (224.0.0.251) at 1:0:5e:0:0:fb on bridge100 ifscope permanent [ethernet]Also tried this but didn't work: https://github.com/lima-vm/socket_vmnet?tab=readme-ov-file#ip-address-is-not-assigned Is it because the Logs for
|
nirs
reviewed
Oct 13, 2025
arp is not a good way to find the ip, it can take a while until a linux vm is available via arp.
This is not needed since macOS 15, I guess you are on macOS 15 or 26?
The guest agent should not affect this. The vm should get an ip address in few seconds after it starts. Typical when it does not work it is because an issue in vmnet/socketlifterfw,bootpd/kernel. Unfortunately we have very little visibility on the internals.
Good, we know that the vm started.
Looks good
Looks good if I remember correctly. Stuff you can try when you don't get ip address:
If nothing else works, usually reboot fixes the issue. |
|
I tested with socket_vment installed as launchd daemon - I prefer to use a proper daemon instead of manage the daemon using lima with sudo. Using this yaml: And nothing happens - looks like logging errors on startup is broken. I did not try to look at logs since I'm not interested in "lima: shared" mode. After changing the network to: lima starts: And krunkit is running as: This cannot work since lima does not create the socket: And krunkit panics: We need fix krunkit the check if the path exist and fail in a cleaner way. Checking the qemu driver network code I found that we use the wrong Mac address for the device - the Mac address of the user network instead of the socket_vmnet network. I fixed the networking arguments using: diff --git a/pkg/driver/krunkit/krunkit_darwin.go b/pkg/driver/krunkit/krunkit_darwin.go
index db930539..c200f4be 100644
--- a/pkg/driver/krunkit/krunkit_darwin.go
+++ b/pkg/driver/krunkit/krunkit_darwin.go
@@ -17,7 +17,6 @@ import (
"github.com/lima-vm/lima/v2/pkg/iso9660util"
"github.com/lima-vm/lima/v2/pkg/limatype"
"github.com/lima-vm/lima/v2/pkg/limatype/filenames"
- "github.com/lima-vm/lima/v2/pkg/limayaml"
"github.com/lima-vm/lima/v2/pkg/networks"
)
@@ -52,6 +51,8 @@ func Cmdline(inst *limatype.Instance) (*exec.Cmd, error) {
func buildNetworkArgs(inst *limatype.Instance) ([]string, error) {
var args []string
+
+ // We support only socket_vment - check if we can use it.
nwCfg, err := networks.LoadConfig()
if err != nil {
return nil, err
@@ -60,21 +61,32 @@ func buildNetworkArgs(inst *limatype.Instance) ([]string, error) {
if err != nil {
return nil, err
}
- if socketVMNetOk {
- sock, err := networks.Sock(networks.ModeShared)
- if err != nil {
- return nil, err
+ if !socketVMNetOk {
+ return args, errors.New("socket_vmnet is not installed")
+ }
+
+ // Add one or more network devices.
+ for _, nw := range inst.Networks {
+ var sock string
+ if nw.Lima == networks.ModeShared || nw.Lima == networks.ModeBridged {
+ sock, err = networks.Sock(nw.Lima)
+ if err != nil {
+ return nil, err
+ }
+ } else if nw.Socket != "" {
+ sock = nw.Socket
+ } else {
+ return nil, fmt.Errorf("invalid network spec %+v", nw)
}
- networkArg := fmt.Sprintf("virtio-net,type=unixstream,path=%s,mac=%s",
- sock,
- limayaml.MACAddress(inst.Dir),
- )
- args = append(args, "--device", networkArg)
+ device := fmt.Sprintf("virtio-net,type=unixstream,path=%s,mac=%s", sock, nw.MACAddress)
+ args = append(args, "--device", device)
+ }
- return args, nil
+ if len(args) == 0 {
+ return args, fmt.Errorf("no socket_vment networks defined")
}
- return args, errors.New("socket_vmnet is not installed")
+ return args, nil
}
func EnsureDisk(ctx context.Context, inst *limatype.Instance) error {With this krunkit is started with the right arguments: But it takes about a minute until serial.log shows the login prompt, and krunkit does not get an ip address. The issue is that we generate this network_config, which cannot work: krunkit does not have a user network and we create only one virtio-net device. I'm not sure why this configuration does not work, since we don't have any device matching I found that I can remove the user network with this change: diff --git a/pkg/cidata/cidata.go b/pkg/cidata/cidata.go
index 888e17c5..dd0bdd0d 100644
--- a/pkg/cidata/cidata.go
+++ b/pkg/cidata/cidata.go
@@ -257,7 +257,11 @@ func templateArgs(ctx context.Context, bootScripts bool, instDir, name string, i
})
}
- args.Networks = append(args.Networks, Network{MACAddress: limayaml.MACAddress(instDir), Interface: networks.SlirpNICName, Metric: 200})
+ // XXX Should query the driver instead of hardcodeing the name.
+ if *instConfig.VMType != "krunkit" {
+ args.Networks = append(args.Networks, Network{MACAddress: limayaml.MACAddress(instDir), Interface: networks.SlirpNICName, Metric: 200})
+ }
+
for i, nw := range instConfig.Networks {
if i == firstUsernetIndex {
continueWith this change krunkit starts in few seconds and we can login: % ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=error -l nir 192.168.105.2 -i ~/.lima/_config/user
Welcome to Ubuntu 25.04 (GNU/Linux 6.14.0-23-generic aarch64)
...
nir@lima-krunkit:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: lima0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:55:55:eb:bf:61 brd ff:ff:ff:ff:ff:ff
altname enx525555ebbf61
inet 192.168.105.2/24 metric 100 brd 192.168.105.255 scope global dynamic lima0
valid_lft 334sec preferred_lft 334sec
inet6 fe80::5055:55ff:feeb:bf61/64 scope link proto kernel_ll
valid_lft forever preferred_lft foreverBut limactl is stuck trying to access ssh via the user network which krunkit does not have: To integrate with krunkit we need to change lima to support driver without a user network:
|
Why can't we have a user network ? |
This comment was marked as off-topic.
This comment was marked as off-topic.
Actually we can! But the result is:
So having an option to run an instance without a user network is better. PatchWith this patch on top of this PR I could start in plain mode: diff --git a/pkg/driver/krunkit/krunkit_darwin.go b/pkg/driver/krunkit/krunkit_darwin.go
index db930539..d86a0c39 100644
--- a/pkg/driver/krunkit/krunkit_darwin.go
+++ b/pkg/driver/krunkit/krunkit_darwin.go
@@ -19,6 +19,7 @@ import (
"github.com/lima-vm/lima/v2/pkg/limatype/filenames"
"github.com/lima-vm/lima/v2/pkg/limayaml"
"github.com/lima-vm/lima/v2/pkg/networks"
+ "github.com/lima-vm/lima/v2/pkg/networks/usernet"
)
const (
@@ -52,29 +53,55 @@ func Cmdline(inst *limatype.Instance) (*exec.Cmd, error) {
func buildNetworkArgs(inst *limatype.Instance) ([]string, error) {
var args []string
- nwCfg, err := networks.LoadConfig()
- if err != nil {
- return nil, err
- }
- socketVMNetOk, err := nwCfg.IsDaemonInstalled(networks.SocketVMNet)
- if err != nil {
- return nil, err
- }
- if socketVMNetOk {
- sock, err := networks.Sock(networks.ModeShared)
- if err != nil {
- return nil, err
+
+ // Add one or more network devices.
+ for _, nw := range inst.Networks {
+ var sock string
+ var mac string
+
+ if nw.Lima != "" {
+ nwCfg, err := networks.LoadConfig()
+ if err != nil {
+ return nil, err
+ }
+ switch nw.Lima {
+ case networks.ModeUserV2:
+ sock, err = usernet.Sock(nw.Lima, usernet.QEMUSock)
+ if err != nil {
+ return nil, err
+ }
+ mac = limayaml.MACAddress(inst.Dir)
+ case networks.ModeShared, networks.ModeBridged:
+ socketVMNetOk, err := nwCfg.IsDaemonInstalled(networks.SocketVMNet)
+ if err != nil {
+ return nil, err
+ }
+ if !socketVMNetOk {
+ return args, errors.New("socket_vmnet is not installed")
+ }
+ sock, err = networks.Sock(nw.Lima)
+ if err != nil {
+ return nil, err
+ }
+ mac = nw.MACAddress
+ default:
+ return nil, fmt.Errorf("invalid network spec %+v", nw)
+ }
+ } else if nw.Socket != "" {
+ sock = nw.Socket
+ mac = nw.MACAddress
+ } else {
+ return nil, fmt.Errorf("invalid network spec %+v", nw)
}
- networkArg := fmt.Sprintf("virtio-net,type=unixstream,path=%s,mac=%s",
- sock,
- limayaml.MACAddress(inst.Dir),
- )
- args = append(args, "--device", networkArg)
+ device := fmt.Sprintf("virtio-net,type=unixstream,path=%s,mac=%s", sock, mac)
+ args = append(args, "--device", device)
+ }
- return args, nil
+ if len(args) == 0 {
+ return args, fmt.Errorf("no socket_vment networks defined")
}
- return args, errors.New("socket_vmnet is not installed")
+ return args, nil
}
func EnsureDisk(ctx context.Context, inst *limatype.Instance) error {
diff --git a/pkg/driver/krunkit/krunkit_driver_darwin.go b/pkg/driver/krunkit/krunkit_driver_darwin.go
index bf5d0a2c..a4bfa7c3 100644
--- a/pkg/driver/krunkit/krunkit_driver_darwin.go
+++ b/pkg/driver/krunkit/krunkit_driver_darwin.go
@@ -19,6 +19,8 @@ import (
"github.com/lima-vm/lima/v2/pkg/executil"
"github.com/lima-vm/lima/v2/pkg/limatype"
"github.com/lima-vm/lima/v2/pkg/limatype/filenames"
+ "github.com/lima-vm/lima/v2/pkg/limayaml"
+ "github.com/lima-vm/lima/v2/pkg/networks/usernet"
"github.com/lima-vm/lima/v2/pkg/ptr"
)
@@ -86,7 +88,15 @@ func (l *LimaKrunDriver) Start(ctx context.Context) (chan error, error) {
}()
l.krunkitWaitCh <- krunCmd.Wait()
}()
-
+ go func() {
+ if usernetIndex := limayaml.FirstUsernetIndex(l.Instance.Config); usernetIndex != -1 {
+ client := usernet.NewClientByName(l.Instance.Config.Networks[usernetIndex].Lima)
+ err := client.ConfigureDriver(ctx, l.Instance, l.SSHLocalPort)
+ if err != nil {
+ l.krunkitWaitCh <- err
+ }
+ }
+ }()
return l.krunkitWaitCh, nil
}
ConfigUsing this config: vmType: krunkit
images:
- location: "https://cloud-images.ubuntu.com/releases/plucky/release-20250701/ubuntu-25.04-server-cloudimg-arm64.img"
arch: "aarch64"
digest: "sha256:26d0ac2236f12954923eb35ddfee8fa9fff3eab6111ba84786b98ab3b972c6d8"
- location: https://cloud-images.ubuntu.com/releases/plucky/release/ubuntu-25.04-server-cloudimg-arm64.img
arch: aarch64
plain: true
portForwards:
- static: true
guestPort: 5201
proto: tcp
networks:
# Required for boot
- lima: user-v2
# Make it accesisble from host and other instances
- socket: /var/run/socket_vmnet
metric: 300 # prefer user-v2lima: user-v2 is required for the ssh connection with current lima. We must change the metric on the socket network to make user-v2 preferred. Setting metric on the lima network does not work. Without I don't have network access from the instance. Testing krunkit driverStarting: % ./_output/bin/limactl start krunkit
INFO[0000] Using the existing instance "krunkit"
INFO[0000] Starting the instance "krunkit" with external VM driver "krunkit"
INFO[0000] [hostagent] hostagent socket created at /Users/nir/.lima/krunkit/ha.sock
INFO[0000] SSH Local Port: 51680
INFO[0000] [hostagent] Running in plain mode. Mounts, dynamic port forwarding, containerd, etc. will be ignored. Guest agent will not be running. Static port forwarding is allowed.
INFO[0000] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0010] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0010] [hostagent] The essential requirement 1 of 2 is satisfied
INFO[0010] [hostagent] Waiting for the essential requirement 2 of 2: "Explicitly start ssh ControlMaster"
INFO[0010] [hostagent] The essential requirement 2 of 2 is satisfied
INFO[0010] [hostagent] Setting up static TCP forwarding from 127.0.0.1:5201 to 127.0.0.1:5201
INFO[0011] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
INFO[0011] [hostagent] The final requirement 1 of 1 is satisfied
INFO[0011] READY. Run `ssh -F "/Users/nir/.lima/krunkit/ssh.config" lima-krunkit` to open the shell. Lima processes: % ps | grep lima
21492 ttys005 0:00.01 grep lima
21334 ttys006 0:20.70 /Users/nir/src/lima/_output/bin/limactl usernet -p /Users/nir/.lima/_networks/user-v2/usernet_user-v2.pid -e /Users/nir/.lima/_networks/user-v2/user-v2_ep.sock --listen-qemu /Users/nir/.lima/_networks/user-v2/user-v2_qemu.sock --listen /Users/nir/.lima/_networks/user-v2/user-v2_fd.sock --subnet 192.168.104.0/24 --leases 192.168.104.1=52:55:55:f3:38:6a,192.168.104.2=5a:94:ef:e4:0c:dd,192.168.104.3=52:55:55:7d:08:f5,192.168.104.4=52:55:55:92:cc:15
21336 ttys006 0:00.08 /Users/nir/src/lima/_output/bin/limactl hostagent --pidfile /Users/nir/.lima/krunkit/ha.pid --socket /Users/nir/.lima/krunkit/ha.sock krunkit
21340 ttys006 0:00.03 /Users/nir/src/lima/_output/libexec/lima/lima-driver-krunkit
21345 ttys006 0:29.40 krunkit --memory 2048 --cpus 4 --device virtio-serial,logFilePath=/Users/nir/.lima/krunkit/serial.log --krun-log-level 3 --restful-uri none:// --device virtio-blk,path=/Users/nir/.lima/krunkit/diffdisk,format=raw --device virtio-blk,path=/Users/nir/.lima/krunkit/cidata.iso --device virtio-net,type=unixstream,path=/Users/nir/.lima/_networks/user-v2/user-v2_qemu.sock,mac=52:55:55:92:cc:15 --device virtio-net,type=unixstream,path=/var/run/socket_vmnet,mac=52:55:55:12:88:1fAccessing via limactl shell % ./_output/bin/limactl shell krunkit
nir@lima-krunkit:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:55:55:92:cc:15 brd ff:ff:ff:ff:ff:ff
altname enx52555592cc15
inet 192.168.104.4/24 metric 200 brd 192.168.104.255 scope global dynamic eth0
valid_lft 3512sec preferred_lft 3512sec
inet6 fe80::5055:55ff:fe92:cc15/64 scope link proto kernel_ll
valid_lft forever preferred_lft forever
3: lima1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:55:55:12:88:1f brd ff:ff:ff:ff:ff:ff
altname enx52555512881f
inet 192.168.105.3/24 metric 300 brd 192.168.105.255 scope global dynamic lima1
valid_lft 512sec preferred_lft 512sec
inet6 fe80::5055:55ff:fe12:881f/64 scope link proto kernel_ll
valid_lft forever preferred_lft forever
nir@lima-krunkit:~$ ip r
default via 192.168.104.2 dev eth0 proto dhcp src 192.168.104.4 metric 200
default via 192.168.105.1 dev lima1 proto dhcp src 192.168.105.3 metric 300
192.168.104.0/24 dev eth0 proto kernel scope link src 192.168.104.4 metric 200
192.168.104.2 dev eth0 proto dhcp scope link src 192.168.104.4 metric 200
192.168.105.0/24 dev lima1 proto kernel scope link src 192.168.105.3 metric 300
192.168.105.1 dev lima1 proto dhcp scope link src 192.168.105.3 metric 300 Testing socket_vmnet: % iperf3 -c 192.168.105.3
Connecting to host 192.168.105.3, port 5201
[ 5] local 192.168.105.1 port 51554 connected to 192.168.105.3 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.01 sec 479 MBytes 4.00 Gbits/sec
[ 5] 1.01-2.01 sec 466 MBytes 3.91 Gbits/sec
[ 5] 2.01-3.00 sec 474 MBytes 3.99 Gbits/sec
[ 5] 3.00-4.01 sec 481 MBytes 4.02 Gbits/sec
[ 5] 4.01-5.01 sec 483 MBytes 4.05 Gbits/sec
[ 5] 5.01-6.01 sec 484 MBytes 4.06 Gbits/sec
[ 5] 6.01-7.01 sec 470 MBytes 3.94 Gbits/sec
[ 5] 7.01-8.01 sec 470 MBytes 3.95 Gbits/sec
[ 5] 8.01-9.01 sec 464 MBytes 3.89 Gbits/sec
[ 5] 9.01-10.01 sec 478 MBytes 4.01 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.01 sec 4.64 GBytes 3.98 Gbits/sec sender
[ 5] 0.00-10.01 sec 4.64 GBytes 3.98 Gbits/sec receiverTesting user-v2: % iperf3 -c localhost
Connecting to host localhost, port 5201
[ 7] local 127.0.0.1 port 51691 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate
[ 7] 0.00-1.00 sec 266 MBytes 2.23 Gbits/sec
[ 7] 1.00-2.00 sec 260 MBytes 2.18 Gbits/sec
[ 7] 2.00-3.00 sec 259 MBytes 2.17 Gbits/sec
[ 7] 3.00-4.00 sec 260 MBytes 2.18 Gbits/sec
[ 7] 4.00-5.00 sec 264 MBytes 2.21 Gbits/sec
[ 7] 5.00-6.00 sec 280 MBytes 2.34 Gbits/sec
[ 7] 6.00-7.00 sec 259 MBytes 2.18 Gbits/sec
[ 7] 7.00-8.00 sec 260 MBytes 2.18 Gbits/sec
[ 7] 8.00-9.00 sec 263 MBytes 2.21 Gbits/sec
[ 7] 9.00-10.00 sec 259 MBytes 2.17 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 7] 0.00-10.00 sec 2.57 GBytes 2.20 Gbits/sec sender
[ 7] 0.00-10.01 sec 2.56 GBytes 2.20 Gbits/sec receiverTesting multiple instancesI created another instance with same configuration without static port forwarding: vmType: krunkit
images:
- location: "https://cloud-images.ubuntu.com/releases/plucky/release-20250701/ubuntu-25.04-server-cloudimg-arm64.img"
arch: "aarch64"
digest: "sha256:26d0ac2236f12954923eb35ddfee8fa9fff3eab6111ba84786b98ab3b972c6d8"
- location: https://cloud-images.ubuntu.com/releases/plucky/release/ubuntu-25.04-server-cloudimg-arm64.img
arch: aarch64
plain: true
networks:
- lima: user-v2
- socket: /var/run/socket_vmnet
metric: 300Running iperf3 on second instance (krunkit2) connecting to first (krunkit) via the instance IP (socket_vment) or the internal DNS name on the user-v2 network. nir@lima-krunkit2:~$ iperf3 -c 192.168.105.3
Connecting to host 192.168.105.3, port 5201
[ 5] local 192.168.105.4 port 54540 connected to 192.168.105.3 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 297 MBytes 2.49 Gbits/sec 0 3.95 MBytes
[ 5] 1.00-2.00 sec 304 MBytes 2.55 Gbits/sec 0 3.95 MBytes
[ 5] 2.00-3.00 sec 308 MBytes 2.59 Gbits/sec 0 3.95 MBytes
[ 5] 3.00-4.00 sec 306 MBytes 2.57 Gbits/sec 0 3.95 MBytes
[ 5] 4.00-5.00 sec 307 MBytes 2.57 Gbits/sec 0 3.95 MBytes
[ 5] 5.00-6.00 sec 310 MBytes 2.60 Gbits/sec 0 3.95 MBytes
[ 5] 6.00-7.00 sec 307 MBytes 2.58 Gbits/sec 0 3.95 MBytes
[ 5] 7.00-8.00 sec 304 MBytes 2.55 Gbits/sec 0 3.95 MBytes
[ 5] 8.00-9.00 sec 309 MBytes 2.59 Gbits/sec 0 3.95 MBytes
[ 5] 9.00-10.00 sec 306 MBytes 2.57 Gbits/sec 0 3.95 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.99 GBytes 2.57 Gbits/sec 0 sender
[ 5] 0.00-10.01 sec 2.99 GBytes 2.56 Gbits/sec receiver
nir@lima-krunkit2:~$ iperf3 -c lima-krunkit.internal
Connecting to host lima-krunkit.internal, port 5201
[ 5] local 192.168.104.5 port 37324 connected to 192.168.104.4 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 314 MBytes 2.63 Gbits/sec 0 4.00 MBytes
[ 5] 1.00-2.00 sec 319 MBytes 2.67 Gbits/sec 0 4.00 MBytes
[ 5] 2.00-3.00 sec 316 MBytes 2.65 Gbits/sec 0 4.00 MBytes
[ 5] 3.00-4.00 sec 315 MBytes 2.64 Gbits/sec 0 4.00 MBytes
[ 5] 4.00-5.00 sec 317 MBytes 2.66 Gbits/sec 0 4.21 MBytes
[ 5] 5.00-6.00 sec 317 MBytes 2.66 Gbits/sec 0 4.21 MBytes
[ 5] 6.00-7.00 sec 317 MBytes 2.66 Gbits/sec 0 4.21 MBytes
[ 5] 7.00-8.00 sec 319 MBytes 2.68 Gbits/sec 0 4.21 MBytes
[ 5] 8.00-9.00 sec 316 MBytes 2.65 Gbits/sec 0 4.21 MBytes
[ 5] 9.00-10.00 sec 315 MBytes 2.64 Gbits/sec 0 4.21 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 3.09 GBytes 2.65 Gbits/sec 0 sender
[ 5] 0.00-10.01 sec 3.09 GBytes 2.65 Gbits/sec receiverComparing to vzI replaced vmType to vz and run the same tests: % ./_output/bin/limactl start krunkit
INFO[0000] Using the existing instance "krunkit"
INFO[0000] Starting the instance "krunkit" with internal VM driver "vz"
INFO[0000] [hostagent] hostagent socket created at /Users/nir/.lima/krunkit/ha.sock
INFO[0000] [hostagent] Starting VZ (hint: to watch the boot progress, see "/Users/nir/.lima/krunkit/serial*.log")
INFO[0000] [hostagent] [VZ] - vm state change: running
INFO[0006] [hostagent] Started vsock forwarder: 127.0.0.1:51751 -> vsock:22 on VM
INFO[0006] [hostagent] Detected SSH server is listening on the vsock port; changed 127.0.0.1:51751 to proxy for the vsock port
INFO[0007] SSH Local Port: 51751
INFO[0006] [hostagent] Running in plain mode. Mounts, dynamic port forwarding, containerd, etc. will be ignored. Guest agent will not be running. Static port forwarding is allowed.
INFO[0006] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0016] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0017] [hostagent] The essential requirement 1 of 2 is satisfied
INFO[0017] [hostagent] Waiting for the essential requirement 2 of 2: "Explicitly start ssh ControlMaster"
INFO[0017] [hostagent] The essential requirement 2 of 2 is satisfied
INFO[0017] [hostagent] Setting up static TCP forwarding from 127.0.0.1:5201 to 127.0.0.1:5201
INFO[0017] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
INFO[0017] [hostagent] The final requirement 1 of 1 is satisfied
INFO[0017] READY. Run `ssh -F "/Users/nir/.lima/krunkit/ssh.config" lima-krunkit` to open the shell.
% iperf3 -c 192.168.105.3
Connecting to host 192.168.105.3, port 5201
[ 5] local 192.168.105.1 port 51758 connected to 192.168.105.3 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 281 MBytes 2.36 Gbits/sec
[ 5] 1.00-2.00 sec 285 MBytes 2.39 Gbits/sec
[ 5] 2.00-3.00 sec 283 MBytes 2.38 Gbits/sec
[ 5] 3.00-4.00 sec 282 MBytes 2.37 Gbits/sec
[ 5] 4.00-5.00 sec 284 MBytes 2.37 Gbits/sec
[ 5] 5.00-6.00 sec 282 MBytes 2.36 Gbits/sec
[ 5] 6.00-7.00 sec 282 MBytes 2.38 Gbits/sec
[ 5] 7.00-8.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 8.00-9.00 sec 282 MBytes 2.36 Gbits/sec
[ 5] 9.00-10.00 sec 282 MBytes 2.37 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 2.76 GBytes 2.37 Gbits/sec sender
[ 5] 0.00-9.92 sec 2.76 GBytes 2.39 Gbits/sec receiver
% iperf3 -c localhost
Connecting to host localhost, port 5201
[ 7] local 127.0.0.1 port 51760 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate
[ 7] 0.00-1.00 sec 549 MBytes 4.61 Gbits/sec
[ 7] 1.00-2.00 sec 543 MBytes 4.55 Gbits/sec
[ 7] 2.00-3.00 sec 543 MBytes 4.56 Gbits/sec
[ 7] 3.00-4.00 sec 548 MBytes 4.59 Gbits/sec
[ 7] 4.00-5.00 sec 546 MBytes 4.58 Gbits/sec
[ 7] 5.00-6.00 sec 549 MBytes 4.60 Gbits/sec
[ 7] 6.00-7.00 sec 543 MBytes 4.56 Gbits/sec
[ 7] 7.00-8.00 sec 540 MBytes 4.53 Gbits/sec
[ 7] 8.00-9.00 sec 554 MBytes 4.65 Gbits/sec
[ 7] 9.00-10.00 sec 548 MBytes 4.60 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 7] 0.00-10.00 sec 5.33 GBytes 4.58 Gbits/sec sender
[ 7] 0.00-9.94 sec 5.33 GBytes 4.61 Gbits/sec receiverComparing to qemuI changed vmType to qemu and run the same tests. % ./_output/bin/limactl start krunkit
INFO[0000] Using the existing instance "krunkit"
INFO[0000] Starting the instance "krunkit" with internal VM driver "qemu"
INFO[0000] [hostagent] hostagent socket created at /Users/nir/.lima/krunkit/ha.sock
INFO[0000] [hostagent] Using system firmware ("/opt/homebrew/share/qemu/edk2-aarch64-code.fd")
INFO[0000] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/nir/.lima/krunkit/serial*.log")
INFO[0001] SSH Local Port: 51767
INFO[0000] [hostagent] Running in plain mode. Mounts, dynamic port forwarding, containerd, etc. will be ignored. Guest agent will not be running. Static port forwarding is allowed.
INFO[0000] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0010] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0010] [hostagent] The essential requirement 1 of 2 is satisfied
INFO[0010] [hostagent] Waiting for the essential requirement 2 of 2: "Explicitly start ssh ControlMaster"
INFO[0010] [hostagent] The essential requirement 2 of 2 is satisfied
INFO[0010] [hostagent] Setting up static TCP forwarding from 127.0.0.1:5201 to 127.0.0.1:5201
INFO[0010] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
INFO[0010] [hostagent] The final requirement 1 of 1 is satisfied
INFO[0011] READY. Run `ssh -F "/Users/nir/.lima/krunkit/ssh.config" lima-krunkit` to open the shell.
% iperf3 -c 192.168.105.3
Connecting to host 192.168.105.3, port 5201
[ 5] local 192.168.105.1 port 51774 connected to 192.168.105.3 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 360 MBytes 3.02 Gbits/sec
[ 5] 1.00-2.00 sec 364 MBytes 3.06 Gbits/sec
[ 5] 2.00-3.00 sec 438 MBytes 3.67 Gbits/sec
[ 5] 3.00-4.00 sec 367 MBytes 3.08 Gbits/sec
[ 5] 4.00-5.00 sec 467 MBytes 3.92 Gbits/sec
[ 5] 5.00-6.01 sec 473 MBytes 3.95 Gbits/sec
[ 5] 6.01-7.00 sec 470 MBytes 3.96 Gbits/sec
[ 5] 7.00-8.01 sec 428 MBytes 3.57 Gbits/sec
[ 5] 8.01-9.00 sec 364 MBytes 3.07 Gbits/sec
[ 5] 9.00-10.01 sec 426 MBytes 3.56 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.01 sec 4.06 GBytes 3.49 Gbits/sec sender
[ 5] 0.00-10.08 sec 4.06 GBytes 3.46 Gbits/sec receiver
% iperf3 -c localhost
Connecting to host localhost, port 5201
[ 7] local 127.0.0.1 port 51777 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate
[ 7] 0.00-1.00 sec 227 MBytes 1.90 Gbits/sec
[ 7] 1.00-2.00 sec 223 MBytes 1.87 Gbits/sec
[ 7] 2.00-3.00 sec 272 MBytes 2.28 Gbits/sec
[ 7] 3.00-4.00 sec 272 MBytes 2.28 Gbits/sec
[ 7] 4.00-5.00 sec 271 MBytes 2.28 Gbits/sec
[ 7] 5.00-6.00 sec 271 MBytes 2.27 Gbits/sec
[ 7] 6.00-7.00 sec 272 MBytes 2.28 Gbits/sec
[ 7] 7.00-8.00 sec 271 MBytes 2.28 Gbits/sec
[ 7] 8.00-9.00 sec 268 MBytes 2.25 Gbits/sec
[ 7] 9.00-10.00 sec 269 MBytes 2.26 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 7] 0.00-10.00 sec 2.55 GBytes 2.19 Gbits/sec sender
[ 7] 0.00-10.01 sec 2.55 GBytes 2.19 Gbits/sec receiver |
nirs
reviewed
Oct 14, 2025
|
The default mode shouldn't depend on |
Should I try with gvisor? |
Yes |
|
I think this can work similar to vz:
In both case krunkit will use virtio-net,type=unixstream,mac=,...path=... |
|
Finally networking now works like vz(slirp using gvisor-tap-vsock). Instances can also be run without the
then we are good to go. ➜ lima2 git:(driver/krun) ✗ lm start templates/experimental/krunkit.yaml
? Creating an instance "krunkit" Proceed with the current configuration
INFO[0001] Starting the instance "krunkit" with external VM driver "krunkit"
INFO[0001] Attempting to download the image arch=aarch64 digest="sha256:26d0ac2236f12954923eb35ddfee8fa9fff3eab6111ba84786b98ab3b972c6d8" location="https://cloud-images.ubuntu.com/releases/plucky/release-20250701/ubuntu-25.04-server-cloudimg-arm64.img"
INFO[0001] Using cache "/Users/ansumansahoo/Library/Caches/lima/download/by-url-sha256/eccac025a7a4709721a97d4ef1d5973546c85062917f188cdc6d247b665cbab5/data"
INFO[0006] Attempting to download the nerdctl archive arch=aarch64 digest="sha256:2f98346ed3dcaf47cfd6461a5685e582284329a1ece1d952ade881b9fe7491e8" location="https://github.com/containerd/nerdctl/releases/download/v2.1.6/nerdctl-full-2.1.6-linux-arm64.tar.gz"
INFO[0006] Using cache "/Users/ansumansahoo/Library/Caches/lima/download/by-url-sha256/d82077f1308e8d063805c4c1f761e1c3fd636e18cab7e25919a18e7a0838afa2/data"
INFO[0007] [hostagent] hostagent socket created at /Users/ansumansahoo/.lima/krunkit/ha.sock
INFO[0008] SSH Local Port: 65298
INFO[0007] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0017] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0019] [hostagent] The essential requirement 1 of 2 is satisfied
INFO[0019] [hostagent] Waiting for the essential requirement 2 of 2: "user session is ready for ssh"
INFO[0029] [hostagent] Waiting for the essential requirement 2 of 2: "user session is ready for ssh"
INFO[0030] [hostagent] The essential requirement 2 of 2 is satisfied
INFO[0030] [hostagent] Setting up static TCP forwarding from 127.0.0.1:5201 to 127.0.0.1:5201
INFO[0030] [hostagent] Waiting for the optional requirement 1 of 2: "systemd must be available"
INFO[0030] [hostagent] Forwarding "/run/lima-guestagent.sock" (guest) to "/Users/ansumansahoo/.lima/krunkit/ga.sock" (host)
INFO[0031] [hostagent] time="2025-10-23T03:17:58+05:30" level=info msg="Getting guest agent connection"
INFO[0030] [hostagent] The optional requirement 1 of 2 is satisfied
INFO[0030] [hostagent] Waiting for the optional requirement 2 of 2: "containerd binaries to be installed"
INFO[0030] [hostagent] Guest agent is running
INFO[0030] [hostagent] Not forwarding TCP 127.0.0.54:53
INFO[0030] [hostagent] Not forwarding TCP 127.0.0.53:53
INFO[0030] [hostagent] Not forwarding TCP 0.0.0.0:22
INFO[0030] [hostagent] Not forwarding TCP [::]:22
INFO[0030] [hostagent] Not forwarding UDP 127.0.0.54:53
INFO[0030] [hostagent] Not forwarding UDP 127.0.0.53:53
INFO[0030] [hostagent] Not forwarding UDP 192.168.5.15:68
INFO[0043] [hostagent] Forwarding UDP from 0.0.0.0:57561 to 127.0.0.1:57561
INFO[0048] [hostagent] The optional requirement 2 of 2 is satisfied
INFO[0048] [hostagent] Waiting for the guest agent to be running
INFO[0048] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
INFO[0049] [hostagent] Forwarding TCP from 127.0.0.1:36777 to 127.0.0.1:36777
INFO[0060] [hostagent] The final requirement 1 of 1 is satisfied
INFO[0061] READY. Run `limactl shell krunkit` to open the shell. |
afbjorklund
reviewed
Oct 24, 2025
Sure thing It is very easy with minimal changes to the current krunkit branch, we can also support diff --git a/pkg/driver/krunkit/krunkit_darwin.go b/pkg/driver/krunkit/krunkit_darwin.go
index e0288549..ca1cf6fd 100644
--- a/pkg/driver/krunkit/krunkit_darwin.go
+++ b/pkg/driver/krunkit/krunkit_darwin.go
@@ -25,20 +25,22 @@ import (
"github.com/sirupsen/logrus"
)
-const logLevelInfo = "3"
+const logLevelInfo = "info"
// Cmdline constructs the command line arguments for krunkit based on the instance configuration.
func Cmdline(inst *limatype.Instance) (*exec.Cmd, error) {
var args = []string{
- "--memory", strconv.Itoa(2048),
+ "--memory", strconv.Itoa(6144),
"--cpus", fmt.Sprintf("%d", *inst.Config.CPUs),
"--device", fmt.Sprintf("virtio-serial,logFilePath=%s", filepath.Join(inst.Dir, filenames.SerialLog)),
- "--krun-log-level", logLevelInfo,
+ "--log-level", logLevelInfo,
"--restful-uri", "none://",
+ "--bootloader", fmt.Sprintf("efi,variable-store=%s,create", filepath.Join(inst.Dir, "vfkit-efi")),
// First virtio-blk device is the boot disk
- "--device", fmt.Sprintf("virtio-blk,path=%s,format=raw", filepath.Join(inst.Dir, filenames.DiffDisk)),
+ "--device", fmt.Sprintf("virtio-blk,path=%s", filepath.Join(inst.Dir, filenames.DiffDisk)),
"--device", fmt.Sprintf("virtio-blk,path=%s", filepath.Join(inst.Dir, filenames.CIDataISO)),
+ "--device", "virtio-vsock,port=2222,socketURL=/Users/ansumansahoo/.lima/krunkit/vfkit-2222.sock,connect",
}
networkArgs, err := buildNetworkArgs(inst)
@@ -61,7 +63,7 @@ func Cmdline(inst *limatype.Instance) (*exec.Cmd, error) {
}
args = append(args, networkArgs...)
- cmd := exec.CommandContext(context.Background(), "krunkit", args...)
+ cmd := exec.CommandContext(context.Background(), "vfkit", args...)
return cmd, nil
}
@@ -82,7 +84,7 @@ func buildNetworkArgs(inst *limatype.Instance) ([]string, error) {
return nil, err
}
- args = append(args, "--device", fmt.Sprintf("virtio-net,type=unixgram,fd=%d,mac=%s", client.Fd(), limayaml.MACAddre
ss(inst.Dir)))
+ args = append(args, "--device", fmt.Sprintf("virtio-net,fd=%d,mac=%s", client.Fd(), limayaml.MACAddress(inst.Dir)))
}
for _, nw := range inst.Networks {
diff --git a/pkg/driver/krunkit/krunkit_driver_darwin.go b/pkg/driver/krunkit/krunkit_driver_darwin.go
index 0d42da7b..e449925d 100644
--- a/pkg/driver/krunkit/krunkit_driver_darwin.go
+++ b/pkg/driver/krunkit/krunkit_driver_darwin.go
@@ -26,6 +26,7 @@ import (
type LimaKrunkitDriver struct {
Instance *limatype.Instance
SSHLocalPort int
+ vSockPort int
usernetClient *usernet.Client
stopUsernet context.CancelFunc
@@ -42,6 +43,7 @@ func New() *LimaKrunkitDriver {
func (l *LimaKrunkitDriver) Configure(inst *limatype.Instance) *driver.ConfiguredDriver {
l.Instance = inst
l.SSHLocalPort = inst.SSHLocalPort
+ l.vSockPort = 2222
return &driver.ConfiguredDriver{
Driver: l,
@@ -182,11 +184,13 @@ func (l *LimaKrunkitDriver) Unregister(ctx context.Context) error {
}
func (l *LimaKrunkitDriver) ForwardGuestAgent() bool {
- return true
+ return false
}
func (l *LimaKrunkitDriver) GuestAgentConn(ctx context.Context) (net.Conn, string, error) {
- return nil, "unix", nil
+ var d net.Dialer
+ conn, err := d.DialContext(ctx, "unix", "/Users/ansumansahoo/.lima/krunkit/vfkit-2222.sock")
+ return conn, "", err
}
func (l *LimaKrunkitDriver) Validate(ctx context.Context) error {
@@ -219,6 +223,8 @@ func (l *LimaKrunkitDriver) Info() driver.Info {
if l.Instance != nil && l.Instance.Dir != "" {
info.InstanceDir = l.Instance.Dir
}
+ info.VsockPort = l.vSockPort
+ info.VirtioPort = ""
info.Features = driver.DriverFeatures{
DynamicSSHAddress: false,
➜ lima2 git:(driver/krun) ✗ lm start templates/experimental/krunkit.yaml
? Creating an instance "krunkit" Proceed with the current configuration
INFO[0001] Starting the instance "krunkit" with external VM driver "krunkit"
INFO[0001] Attempting to download the image arch=aarch64 digest="sha256:26d0ac2236f12954923eb35ddfee8fa9fff3eab6111ba84786b98ab3b972c6d8" location="https://cloud-images.ubuntu.com/releases/plucky/release-20250701/ubuntu-25.04-server-cloudimg-arm64.img"
INFO[0001] Using cache "/Users/ansumansahoo/Library/Caches/lima/download/by-url-sha256/eccac025a7a4709721a97d4ef1d5973546c85062917f188cdc6d247b665cbab5/data"
INFO[0005] Attempting to download the nerdctl archive arch=aarch64 digest="sha256:2f98346ed3dcaf47cfd6461a5685e582284329a1ece1d952ade881b9fe7491e8" location="https://github.com/containerd/nerdctl/releases/download/v2.1.6/nerdctl-full-2.1.6-linux-arm64.tar.gz"
INFO[0005] Using cache "/Users/ansumansahoo/Library/Caches/lima/download/by-url-sha256/d82077f1308e8d063805c4c1f761e1c3fd636e18cab7e25919a18e7a0838afa2/data"
INFO[0006] [hostagent] hostagent socket created at /Users/ansumansahoo/.lima/krunkit/ha.sock
INFO[0006] SSH Local Port: 58583
INFO[0006] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0016] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0019] [hostagent] The essential requirement 1 of 2 is satisfied
INFO[0019] [hostagent] Waiting for the essential requirement 2 of 2: "user session is ready for ssh"
INFO[0029] [hostagent] Waiting for the essential requirement 2 of 2: "user session is ready for ssh"
INFO[0029] [hostagent] The essential requirement 2 of 2 is satisfied
INFO[0029] [hostagent] Setting up static TCP forwarding from 127.0.0.1:5201 to 127.0.0.1:5201
INFO[0029] [hostagent] Waiting for the optional requirement 1 of 2: "systemd must be available"
INFO[0029] [hostagent] The optional requirement 1 of 2 is satisfied
INFO[0029] [hostagent] Waiting for the optional requirement 2 of 2: "containerd binaries to be installed"
INFO[0029] [hostagent] time="2025-10-25T20:19:29+05:30" level=info msg="Getting guest agent connection"
INFO[0029] [hostagent] Guest agent is running
INFO[0029] [hostagent] Not forwarding TCP 127.0.0.54:53
INFO[0029] [hostagent] Not forwarding TCP 0.0.0.0:22
INFO[0029] [hostagent] Not forwarding TCP 127.0.0.53:53
INFO[0029] [hostagent] Not forwarding TCP [::]:22
INFO[0029] [hostagent] Not forwarding UDP 127.0.0.54:53
INFO[0029] [hostagent] Not forwarding UDP 127.0.0.53:53
INFO[0029] [hostagent] Not forwarding UDP 192.168.5.15:68
INFO[0042] [hostagent] Forwarding UDP from [::]:57306 to 127.0.0.1:57306
INFO[0044] [hostagent] Forwarding UDP from 0.0.0.0:52876 to 127.0.0.1:52876
INFO[0050] [hostagent] The optional requirement 2 of 2 is satisfied
INFO[0050] [hostagent] Waiting for the guest agent to be running
INFO[0050] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
INFO[0050] [hostagent] Forwarding TCP from 127.0.0.1:37195 to 127.0.0.1:37195
INFO[0062] [hostagent] The final requirement 1 of 1 is satisfied
INFO[0062] READY. Run `limactl shell krunkit` to open the shell. |
|
Let's merge this PR as is (after passing the CI), and implement additional features in follow-up PRs |
|
We would like to merge this and ship a beta release before the meeting on Thursday Is there any blocker other than fixing lint failures? Feel free to ping us if you need any help 💚 |
Thanks for the support, really means a lot! Sorry I took this long but I am very near to completion. Ubuntu 25.04 VM is compatible with Venus and MESA, but not as well as Fedora. While checking GPU info using Devices:
========
GPU0:
apiVersion = 1.2.0
driverVersion = 24.1.5
vendorID = 0x106b
deviceID = 0xf060207
deviceType = PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU
deviceName = Virtio-GPU Venus (Apple M1)
driverID = DRIVER_ID_MESA_VENUS
driverName = venus
driverInfo = Mesa 24.1.5
conformanceVersion = 1.3.0.0
deviceUUID = 47063303-a64b-0845-3bf5-01d1daa8649b
driverUUID = 05a3cceb-1991-3628-23bd-764bdc583f20
GPU1:
apiVersion = 1.3.278
driverVersion = 0.0.1
vendorID = 0x10005
deviceID = 0x0000
deviceType = PHYSICAL_DEVICE_TYPE_CPU
deviceName = llvmpipe (LLVM 18.1.6, 128 bits)
driverID = DRIVER_ID_MESA_LLVMPIPE
driverName = llvmpipe
driverInfo = Mesa 24.1.5 (LLVM 18.1.6)
conformanceVersion = 1.3.1.1
deviceUUID = 6d657361-3234-2e31-2e35-000000000000
driverUUID = 6c6c766d-7069-7065-5555-494400000000Now the only problem with Fedora is that it takes significant amount of time to start and thats where I was stuck. After digging deep I found out that many boot scripts are not well compatible with Fedora so will have to hack around with those and make the driver fast. |
unsuman
force-pushed
the
driver/krun
branch
3 times, most recently
from
e7adf1a to
a126fbb
Compare
unsuman
force-pushed
the
driver/krun
branch
4 times, most recently
from
0a0b4f4 to
16ed069
Compare
Signed-off-by: Ansuman Sahoo <anshumansahoo500@gmail.com>
unsuman
changed the title
libkrun using krunkitlibkrun using krunkit
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for the Krunkit VM driver to Lima, enabling super-light VMs on macOS/ARM64 with GPU acceleration via Mesa's Venus Vulkan driver. Krunkit builds on libkrun to provide hardware-isolated VMs with optional GPU support for AI and graphics workloads.
- Implements a complete external driver for Krunkit with VM lifecycle management
- Adds GPU acceleration support for Vulkan workloads via Venus driver on Fedora
- Includes documentation and templates for both general and GPU-enabled configurations
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| website/content/en/docs/config/vmtype/krunkit.md | Documentation for the Krunkit driver with installation instructions and configuration examples |
| templates/experimental/krunkit.yaml | YAML template for GPU-enabled Krunkit VM configuration |
| pkg/driver/krunkit/krunkit_driver_darwin.go | Main driver implementation with VM lifecycle methods and GPU provisioning |
| pkg/driver/krunkit/krunkit_darwin.go | Helper functions for command-line construction, networking, and disk management |
| pkg/driver/krunkit/hack/install-vulkan-gpu.sh | GPU setup script for installing Mesa/Vulkan drivers inside the VM |
| pkg/driver/krunkit/errors_darwin.go | Error definitions for unimplemented driver features |
| cmd/lima-driver-krunkit/main_darwin.go | Entry point for the external Krunkit driver binary |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| } | ||
|
|
||
| if len(args) == 0 { | ||
| return args, errors.New("no socket_vment networks defined") |
There was a problem hiding this comment.
Corrected spelling of 'socket_vment' to 'socket_vmnet'.
Suggested change
| return args, errors.New("no socket_vment networks defined") | |
| return args, errors.New("no socket_vmnet networks defined") |
| } | ||
|
|
||
| func (l *LimaKrunkitDriver) BootScripts() (map[string][]byte, error) { | ||
| // Override default reboot-if-required with a no-op because Fedora does not supports this well and |
There was a problem hiding this comment.
Grammatical error: 'does not supports' should be 'does not support'.
Suggested change
| // Override default reboot-if-required with a no-op because Fedora does not supports this well and | |
| // Override default reboot-if-required with a no-op because Fedora does not support this well and |
| cpus: 4 | ||
| arch: aarch64 | ||
|
|
||
| # Fedora 40+ is preferred because of better support for MESA & Vulkan, |
There was a problem hiding this comment.
Inconsistent capitalization: 'MESA' should be 'Mesa' to match the proper name of the Mesa 3D Graphics Library, as used consistently in the documentation.
Suggested change
| # Fedora 40+ is preferred because of better support for MESA & Vulkan, | |
| # Fedora 40+ is preferred because of better support for Mesa & Vulkan, |
| # Install required packages | ||
| sudo dnf install -y dnf-plugins-core dnf-plugin-versionlock llvm18-libs | ||
|
|
||
| # Install Vulkan and MESA base packages |
There was a problem hiding this comment.
Inconsistent capitalization: 'MESA' should be 'Mesa' to match the proper name of the Mesa 3D Graphics Library.
| vulkan-loader \ | ||
| glslc | ||
|
|
||
| # Enable COPR repo with patched MESA for Venus support |
There was a problem hiding this comment.
Inconsistent capitalization: 'MESA' should be 'Mesa' to match the proper name of the Mesa 3D Graphics Library.
| # Enable COPR repo with patched MESA for Venus support | ||
| sudo dnf copr enable -y slp/mesa-krunkit fedora-40-aarch64 | ||
|
|
||
| # Downgrade to patched MESA version from COPR |
There was a problem hiding this comment.
Inconsistent capitalization: 'MESA' should be 'Mesa' to match the proper name of the Mesa 3D Graphics Library.
| sudo dnf downgrade -y mesa-vulkan-drivers.aarch64 \ | ||
| --repo=copr:copr.fedorainfracloud.org:slp:mesa-krunkit | ||
|
|
||
| # Lock MESA version to prevent automatic upgrades |
There was a problem hiding this comment.
Inconsistent capitalization: 'MESA' should be 'Mesa' to match the proper name of the Mesa 3D Graphics Library.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds support for libkrun using their krunkit CLI, a dynamic library that enables programs to run processes in lightweight, isolated environments using KVM virtualization on Linux and HVF on macOS/ARM64. The key feature is to access GPU via Vulkan(Venus) inside the VM.
Implementation Details
Networking: Network connectivity is established using
gvisor-tap-vsockfor slirp functionality. Additional support foruserv2,shared, andbridgednetworking modes is also included.File Sharing:
virtiofsis implemented for high-performance file sharing between the host and the guest VM.Guest-Host Agent Communication: Host-to-guest agent communication currently relies on ssh. There exists a krunkit bug (see
krunkitshould create unix domain socket files containers/krunkit#79) that fails to create the necessary Unix socket file for virtio-vsock communication.GPU Acceleration (Venus) and OS Recommendation
A primary usecase for this integration is to support GPU-accelerated VMs via krunkit, which uses the Venus driver to provide guest-side Vulkan API access. While krunkit can boot the default Ubuntu Lima template, full GPU (Vulkan) acceleration will be unavailable. This is because a specific version of Mesa is required, which is not easily available on Ubuntu.
Fedora 40+ is the recommended as the guest OS for this feature because of patched Mesa Drivers.
Using COPR avoids the complex and error-prone process of manually compiling Mesa from source, which would be required on other systems.
GPU Provisioning
A provisioning script is included to set up the GPU drivers inside the guest VM. This script must be run manually by the user after the VM boots using
sudo install_vulkan_gpu.sh. It is not run automatically during startup as the process adds a significant delay to the boot time.Fedora boot
Lima logs:Venus support inside VM:Ubuntu boot
Lima logs:Venus support inside VM: