Skip to content

Map host paths and apply egress rules in the native-host backend - #407

Closed
Enrique Saurez (esaurez) wants to merge 3 commits into
esaurez/edge-nvxhost-pre-time-abifrom
esaurez/edge-nvxhost-fsnet
Closed

Enrique Saurez (esaurez) wants to merge 3 commits into
esaurez/edge-nvxhost-pre-time-abifrom
esaurez/edge-nvxhost-fsnet

Conversation

@esaurez

Copy link
Copy Markdown
Contributor

Summary

Stacked on #395. Gives the opt-in native-host backend the same filesystem and network policies as the direct OpenVMM backend.

  • Planning in the host library. Provision serializes the request's filesystem and network sections and passes them to the library's new nvx_plan_sandbox export. The library resolves mapped and denied paths, chooses the virtio-fs export (the deepest common directory), pins the identity of every object inside a read-write mapping, and expands egress rules exactly. Validation uses the same export with a validate-only flag that touches nothing on the host; unenforceable policies fail with policy_validation.
  • The plan travels with the sandbox. The sandbox record keeps the plan's JSON as the library wrote it. Start hands it to the new nvx_build_ramfs_launch_arguments_with_plan export, which reapplies every planning rule that needs no host access and refuses when a pinned object was replaced; start then fails with backend_error and asks for a new provision. The old nvx_build_ramfs_launch_arguments binding is gone.
  • The state root stays private. It holds every sandbox's plan, which decides what the next start exports and which egress it allows. Provision refuses a mapping inside the state root, and one that contains it (for example a home directory that holds the default state root) unless a denied path inside the mapping hides it.
  • Guest contract. The edge guest binds each mapping into its RAM overlay (read-only where requested), never exposes the export itself, configures the virtio-net device and, when the policy allows the gateway's DNS port, /etc/resolv.conf. Workloads stay root but keep only the default container capabilities, with no_new_privs and without user namespaces. The runtime ABI name moves to microvm-abi-v2-edge-ramfs-v2.
  • Capabilities. The backend now declares egress allow/deny and rules, ingress deny, host-loopback deny, and read-only, read-write, and denied paths. Ingress and host-loopback access are still rejected.
  • Linux process exits. On Linux, OpenVMM's main thread can exit before its other threads, which still hold its descriptors. Liveness checks treated that zombie as gone, so a stop returned early and an immediate restart could not claim the OpenVMM log. A zombie now counts as running until its last thread exits, as a Windows process does.
  • Guest network configuration. OpenVmmConfig::guest_network now rejects the network, broadcast, and gateway addresses, which OpenVMM refuses, when a backend is created (backend_unavailable), instead of when a sandbox first attaches a network device. This applies to both backends.
  • Tests and example. The ignored guest tests take NVXHOST_TEST_HYPERVISOR (default whp on Windows) and drop their whp_ prefix. New tests cover mappings, the state root, denied and unmapped paths, workload containment, replaced host objects, the kernel command-line budget, and egress rules against host services. The example takes --readonly, --readwrite, --denied, --egress, --egress-allow, and --egress-deny.

Dependencies

  • Requires the matching private host library and edge guest (new exports and runtime ABI v2); an older library fails to load because the new exports are missing.
  • OpenVMM is unchanged: the pinned 9524065 already combines --mount, --mount-deny, and --net with the scratchless RAM-overlay topology.

Validation (head 62ea619)

  • Windows: cargo test --features nvxhost,testing,async (134 unit tests plus every integration suite) and the default cargo test (101 unit tests); Clippy with -D warnings on all targets for both configurations; cargo +1.89 check for both.
  • Linux (Rust 1.98.1): fmt and the same Clippy and test commands (134 and 101 unit tests), including the process-exit tests.
  • The pinned library test passes against freshly built nvxhost.dll and libnvxhost.so.
  • Guest suite, 16 tests, --release --test-threads=1, every one checking that no OpenVMM process outlives its sandbox:
    • Windows/WHP (AMD): all pass, about 95 s;
    • Linux/MSHV: all pass, about 72 s. This is the first end-to-end run of the native lifecycle on Linux; before the process-exit fix, three restart tests could not claim the OpenVMM log and the forced-stop test stopped gracefully.
  • Coverage of the new tests:
    • read-only, read-write, nested, file, denied, and unmapped paths;
    • capability sets, no_new_privs, and EPERM for remount, mount, umount, unshare, and chroot;
    • replaced denied and read-only objects;
    • 13 mappings at the kernel command-line budget, each reaching the guest;
    • egress allow and deny against host services, gateway DNS, and host loopback.
  • The example maps a read-only and a read-write directory and allows one gateway port. The guest reads the host file, its write appears on the host, a write into the read-only path fails, and the allowed port answers.

Notes

  • Files that workloads create through read-write mappings belong to the account that runs OpenVMM. A host hard link that already joins a read-write and a read-only mapping stays writable through the read-write path.
  • Sandbox records written by Add opt-in native-host image-backed edge lifecycle #395 still load; records with a plan do not load in Add opt-in native-host image-backed edge lifecycle #395.
  • An independent review of this change and the library found one medium and three low issues; all are addressed here: stored plans are fully revalidated, the state root is protected, guest addresses are checked at configuration, and the Linux liveness logic gained a test that needs no Python, while the test that uses Python skips without it.

The native-host backend accepts the same filesystem and network policies as the
direct OpenVMM backend. Provision passes the policy to the host library's
nvx_plan_sandbox export, which resolves mapped and denied paths, chooses the
virtio-fs export, pins objects inside read-write mappings, and expands egress
rules exactly. The sandbox record keeps the plan, and start hands it back through
nvx_build_ramfs_launch_arguments_with_plan, which checks it again and fails with
backend_error when a pinned object was replaced.

The state root holds every sandbox's plan, which decides what the next start
exports and which egress it allows. Provision therefore refuses a mapping inside
the state root, and one that contains it unless a denied path hides it.

The edge guest binds each mapping into its RAM overlay, configures the
virtio-net device, and runs workloads as root with only the default container
capabilities, no_new_privs, and no user namespaces, so the guest runtime ABI
moves to microvm-abi-v2-edge-ramfs-v2.

The opt-in guest tests run under the hypervisor that NVXHOST_TEST_HYPERVISOR
names and cover mappings, the state root, denied and unmapped paths,
containment, replaced host objects, the kernel command-line budget, and egress
rules. The example accepts host-path and egress options.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
On Linux the main thread of OpenVMM can exit and leave a zombie while its other
threads still exit. Liveness checks treated that zombie as gone, so a stop
returned while those threads still held OpenVMM's descriptors, and an
immediate restart failed to claim the OpenVMM log that they kept locked. A
zombie now counts as running until its other threads exit, as a Windows
process runs until its last thread exits.

The forced-stop guest test now uses a one-nanosecond stop timeout: on a fast
host a graceful shutdown could complete within the previous millisecond.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
OpenVMM refuses a guest address that is its network's own address, its
broadcast address, or its first address, which is the gateway. The
configuration accepted them, so a backend failed only when a sandbox first
attached a network device, and the native backend reported its planning error
as the request's policy_validation rather than as an unusable configuration.
Creating a backend now rejects such an address with backend_unavailable.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@esaurez

Copy link
Copy Markdown
Contributor Author

Superseded by #418, which carries this change on dev as part of the whole native-host edge backend, over nanvix/openvmm#121, and is validated end to end on WHP and MSHV. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant