Skip to content

net, contain: run under gVisor (skip an unused link cache, tolerate a no-op RLIMIT_MSGQUEUE EPERM) - #365

Open
rubenfiszel wants to merge 1 commit into
google:masterfrom
rubenfiszel:gvisor-netlink-msgqueue
Open

rubenfiszel wants to merge 1 commit into
google:masterfrom
rubenfiszel:gvisor-netlink-msgqueue

Conversation

@rubenfiszel

Copy link
Copy Markdown

Summary

nsjail cannot create a jail inside a gVisor (runsc) sandbox, for example a Kubernetes pod with runtimeClassName: gvisor. Two calls are answered differently from Linux there, and neither changes the jail nsjail builds:

  1. net::initParent builds an rtnetlink link cache it does not use. With clone_newnet, it always connects a NETLINK_ROUTE socket and runs rtnl_link_alloc_cache(). The cache is only used to move interfaces into the jail (iface_own) or to clone a MACVLAN (macvlan_iface). gVisor acks dump requests that set NLM_F_ACK before NLMSG_DONE, which libnl rejects with Message sequence number mismatch (netlink: dump requests with NLM_F_ACK are acked before NLMSG_DONE, which breaks libnl caches gvisor#15945). So every jail with a network namespace fails with Couldn't initialize net namespace, including jails that move no interface.

    Change: when iface_own and macvlan_iface are both empty, return before the netlink part. This sets init_success, so a user-mode networking backend started just above (nstun, pasta) is kept as on the existing !clone_newnet path. Jails that move or clone an interface are unchanged. On Linux the jail is the same, minus one netlink round trip.

  2. RLIMIT_MSGQUEUE. containSetLimits fails on the first rlimit it cannot set. gVisor does not allow setting RLIMIT_MSGQUEUE and returns EPERM for any value (setrlimit(RLIMIT_MSGQUEUE) fails with EPERM, even to its current value gvisor#15944). With the default rlimit_msgqueue_type: SOFT, nsjail only reasserts the inherited limit, so the failing call would change nothing.

    Change: if that setrlimit fails with EPERM while the current soft and hard limits already equal the requested ones, log a warning and continue. A request that would change the limit still fails, and every other rlimit stays fatal.

Running a jail under gVisor also needs SECBIT_NO_SETUID_FIXUP, which gVisor implemented in google/gvisor#15130. That was merged on 2026-10-12 and is not yet in a release (release-20261012.0 lacks it). It also needs clone_newcgroup: false, since gVisor refuses CLONE_NEWCGROUP until a cgroup2 filesystem has been mounted.

Testing

Built from 4ff54a6 with and without this change (Debian trixie, libnl 3.7, protobuf 3.21). The same jails were run with both binaries: -Mo -R /bin -R /lib -R /lib64 -R /usr -R /etc --rlimit_as 4096 --rlimit_nofile 1024 --rlimit_fsize 1000, with the jailed shell printing its capability sets, /proc/self/limits, its network interfaces and the result of a TCP connect.

  • Linux 6.8 (privileged container under runc): identical output with and without the change for:

    • the default network namespace (lo up);
    • --iface_no_lo;
    • --user_net (nstun: lo and eth0, the connect succeeds);
    • --macvlan_iface eth0 (the link cache is still built, the jail gets vs).

    Capabilities are all 0 everywhere, and the limits match.

  • gVisor (runsc nightly with kernel: implement SECBIT_NO_SETUID_FIXUP gvisor#15130, non-root, --disable_clone_newcgroup):

    • Without the change, every jail with a network namespace fails at rtnl_link_alloc_cache().
    • With it, the default and --iface_no_lo jails start. They log RLIMIT_MSGQUEUE ... failed with EPERM, but the limit already has that value, and their capabilities, limits, interfaces (only lo) and connect result (ENETUNREACH) match the Linux jail.
    • --user_net still fails under gVisor, with or without the change, at ioctl(SIOCSIFADDR) in nstun. That is not addressed here.

@google-cla

google-cla Bot commented Oct 18, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@rubenfiszel

Copy link
Copy Markdown
Author

I've signed the CLA now.

… no-op RLIMIT_MSGQUEUE EPERM)

With clone_newnet, initParent built an rtnetlink link cache even when no
interface is moved or cloned into the jail. gVisor acks NLM_F_ACK dump
requests before NLMSG_DONE (google/gvisor#15945), which libnl rejects, so
every jail with a network namespace failed there. Return before the netlink
part when iface_own and macvlan_iface are both empty, keeping init_success
so a user-mode networking backend is not torn down.

gVisor refuses setrlimit(RLIMIT_MSGQUEUE) with EPERM for any value
(google/gvisor#15944). Tolerate that EPERM, with a warning, when the current
limit already has the requested value, as with the default 'soft'.
@rubenfiszel
rubenfiszel force-pushed the gvisor-netlink-msgqueue branch from f843842 to 4bf7297 Compare October 19, 2026 10:01

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant