virtio-gpu venus: fix blob mapping on macOS/HVF and Linux/KVM - #91
Open
maxandrews wants to merge 1 commit into
Open
virtio-gpu venus: fix blob mapping on macOS/HVF and Linux/KVM#91maxandrews wants to merge 1 commit into
maxandrews wants to merge 1 commit into
Conversation
macOS/HVF: hv_vm_map requires host-page-aligned (16K on Apple Silicon) sizes, but venus blob sizes are guest-page (4K) granular — every ring mapping failed (mapped=false) and surfaced as vkCreateInstance ERROR_OUT_OF_HOST_MEMORY with all debug channels silent. Round blob map/unmap lengths to the host page size. (Upstream containers/libkrun has the identical unrounded hv_vm_map; krunkit guests presumably run 16K-page kernels and never hit it.) Linux/KVM: the virgl_resource_map2 feature bound a crosvm-lineage API that upstream virglrenderer never adopted; its upstream equivalent is virgl_renderer_resource_map_fixed (>= 1.1x; size/prot derive from the resource). Rebind the extern and call. Also wire the feature through: devices/virgl_resource_map2 now enables rutabaga_gfx/virgl_resource_map2 (previously nothing enabled the rutabaga side, so the Linux opaque-blob path always returned Unsupported), with an msb_krun passthrough feature for Linux consumers. Not enabled from the gpu feature because the macOS krunkit virglrenderer does not export map_fixed. Verified end-to-end in microsandbox microVMs on both platforms: guest mesa venus enumerates the host GPU and llama.cpp ggml-vulkan runs fully offloaded (Apple M4 Pro via MoltenVK; NVIDIA T4G on Linux/KVM at pp512 648 t/s / tg128 29.2 t/s for a 7B Q4_K_M). Discussion: superradcompany/microsandbox#291
This was referenced Jul 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Companion to superradcompany/microsandbox#291 (the
--gpuflag PR lands there separately, as discussed). Two mapping fixes plus feature wiring, each found by live debugging Venus inference in microsandbox microVMs:1. macOS/HVF: round blob map/unmap sizes to the host page (16K on Apple Silicon).
hv_vm_maprequires host-page-aligned sizes, but venus blob sizes are guest-page (4K) granular — every ring mapping failed (mapped=false) and surfaced as guestvkCreateInstance → ERROR_OUT_OF_HOST_MEMORYwith all debug channels silent. Upstreamcontainers/libkruncarries the identical unroundedhv_vm_map; krunkit guests presumably boot 16K-page kernels and never hit it, so this may be worth forwarding upstream too.2. Linux/KVM: rebind
virgl_resource_map2to upstream'svirgl_renderer_resource_map_fixed.The existing binding targets a crosvm-lineage API that upstream virglrenderer never adopted; the upstream equivalent (>= 1.1x) is
resource_map_fixed(res, addr)— size/prot derive from the resource, and its opaque-handle path covers Venus blobs. Also: nothing previously enabledrutabaga_gfx/virgl_resource_map2from the devices feature of the same name, so the Linux opaque-blob path always returnedUnsupported— rings (SHM) still mapped, so enumeration worked while the first mappable device blob died asvk*Create*: ErrorOutOfHostMemory. The feature now propagates, with anmsb_krunpassthrough for Linux consumers. Deliberately not enabled from thegpufeature: the macOS (krunkit-fork) virglrenderer doesn't exportmap_fixed, and the extern would fail the macOS link.Verified end-to-end (guest mesa venus → virglrenderer → host driver, inside microsandbox microVMs):
vulkaninfoenumeratesVirtio-GPU Venus (Apple M4 Pro); llama.cpp ggml-vulkan generates tokens fully offloaded.Host-setup note that cost us a day and may be worth documenting wherever you land GPU docs: NVIDIA hides
VK_EXT_external_memory_dma_buf(which venus needs for mappable memory) unlessnvidia_drmis loaded with KMS modesetting and the process can open/dev/dri/renderD*(rendergroup).Understood that you have internal work in flight and may reshape this freely — happy to rebase or split further.