Enabling container_kvm_t as a domain for Firecracker

[mikn]

Okay, so we introduced a new SELinux domain for Firecracker. We are using the jailer and doing the transition after the jailer to container_kvm_t.
We stumbled on the fact that the firecracker process started by the jailer is the one setting up the tun device required for network connections. This, for reasons I am less clear about, results in a tun_socket being created in the runtime_t domain (the domain that containerd, all the way to the jailer has), that Firecracker wants to relabel back to container_kvm_t.

There is a neverallow statement in the SELinux policy here: https://github.com/bottlerocket-os/bottlerocket-core-kit/blob/develop/packages/selinux-policy/rules.cil#L340 which does not allow me to allow this specific relabeling as container_kvm_t ends up being an untrusted domain (as it should).

I am not great at CIL and I am not great at SELinux, but I could not for my life figure out how or why this tun_socket requires to make the transition from runtime_t to container_kvm_t and doesn't get created in the container_kvm_t domain right away.
My only guess is that it is because "the other end" of the TUN device is picked up by the containerd shim process, which is in the runtime_t domain (to then be wired together with the CNI), but tbf the documentation about tun_socket in the SELinux context is relatively limited.

I, for now, unblocked myself by commenting out this neverallow statement and explicitly allowing this transition - but it does feel quite hacky and I am sure there is a better way.

In the context of Bottlerocket, do you have any recommendations in how to figure this out? Perhaps, for someone who is actually decent at CIL and SELinux this has a very obvious solution?

[sky1122]

Hi @mikn
We've reviewed the issue you opened about not being able to build SELinux policy extensions from the Bottlerocket main policy package, as well as your discussion on enabling the container_kvm_t domain for Firecracker.

We're interested in understanding the challenges you're facing and exploring potential ways to extend the Bottlerocket SELinux policy without the need to fork the entire policy. Your insights and SELinux policy files would be very helpful as we investigate approaches.

Could you please share the SELinux policy you've been working on? Having access to your policy definitions would allow us to better understand your specific requirements and explore options that may work for your use case.

Please let me know if you have any other questions. I'm happy to discuss further and see how we can assist.

[mikn]

Hi! Sorry for being slow here. But yes, that would be an amazing improvement as we are currently (still) carrying the entire SELinux policy package.

We added a new cil file like this:

; ===== NEW DOMAIN: container_kvm_t =====
(type container_kvm_t)
(roletype system_r container_kvm_t)
(context container_kvm (system_u system_r container_kvm_t s0))

; Add container_kvm_t to subject attribute sets:
(typeattributeset all_s (container_kvm_t))
(typeattributeset container_s (container_kvm_t))      ; It's a container domain
(typeattributeset other_s (container_kvm_t))         ; Not part of the OS
(typeattributeset unprivileged_s (container_kvm_t))  ; Not privileged
(typeattributeset unconfined_s (container_kvm_t))    ; Gets permissions to write to shared_o

; ===== NEW TYPE: vsock_device_t =====
(type vsock_device_t)
(roletype object_r vsock_device_t)
(context vsock_device (system_u object_r vsock_device_t s0))

; Create unique firecracker executable type
(type firecracker_exec_t)
(roletype object_r firecracker_exec_t)
(typeattributeset all_o (firecracker_exec_t))
(typeattributeset immutable_o (firecracker_exec_t))
(context firecracker_exec (system_u object_r firecracker_exec_t s0))

; Add vsock_device_t to object attribute sets:
(typeattributeset all_o (vsock_device_t))
(typeattributeset mutable_o (vsock_device_t))        ; It's a device file
(typeattributeset ephemeral_o (vsock_device_t))      ; Device files are ephemeral
(typeattributeset unconstrained_o (vsock_device_t))  ; Not MCS-constrained

; ===== FILE CONTEXTS =====
(filecon "/dev/vhost-vsock" any vsock_device)
(filecon "/.*/usr(/fips)?/bin/firecracker" file firecracker_exec)
(filecon "/.*/usr(/fips)?/bin/jailer" file firecracker_exec)
(filecon "/run/vc/.*/firecracker" file firecracker_exec)

; ===== PERMISSIONS =====
; Allow runtime_t to transition to container_kvm_t (for the shim relabel)
(allow runtime_t container_kvm_t (processes (transform)))
(typetransition runtime_t firecracker_exec_t process container_kvm_t)
(allow runtime_t firecracker_exec_t (file (entrypoint)))

; Allow container_kvm_t to execute firecracker/jailer (they're labeled as firecracker_exec_t)
(allow container_kvm_t firecracker_exec_t (file (entrypoint execute)))

; Allow container_kvm_t to relabelfrom runtime_t for tun_socket
(allow container_kvm_t runtime_t (tun_socket (relabelfrom)))
(allow container_kvm_t self (tun_socket (relabelto)))

; Allow container_kvm_t to access vsock device
(allow container_kvm_t vsock_device_t (chr_file (ioctl open read write getattr)))
(allow runtime_t vsock_device_t (chr_file (ioctl open read write getattr)))

; Allow container_kvm_t to access tun device (already labeled as any_t)
(allow container_kvm_t any_t (chr_file (ioctl open read write getattr)))

Then in lxc_contexts we needed to add this also:

# Kata Containers KVM process label
kvm_process = "system_u:system_r:container_kvm_t:s0"