Skip to content

feat(runtime): add opt-in managed Docker image profiles - #98

Merged
hky1999 merged 22 commits into
inclusionAI:mainfrom
hky1999:hky1999/managed-runtime-profile
Nov 5, 2026
Merged

hky1999 merged 22 commits into
inclusionAI:mainfrom
hky1999:hky1999/managed-runtime-profile

Conversation

@hky1999

@hky1999 hky1999 commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Add an explicitly enabled managedRuntimeProfile for native remote Buildx/Compose. Sandboxd authorizes the tenant/profile and supplies the fixed tools, bootstrap and private image-management socket; Registry credentials, endpoints and host paths remain platform-owned. Managed images share the ordinary OCI/ROFS reference counter, while each sandbox's private Docker upper uses checkpoint-managed file-backed storage.

Restore checks current tenant authorization and a profile fingerprint. Expanding or reordering the tenant allowlist does not invalidate compatible snapshots. Whole-sandbox checkpoint/restore remains independent of application Python and process signals. Default/0 restore admission remains unlimited; only an explicit positive value enables limiting. The base includes writable /etc/hosts (#96) and layered OCI mount overrides (#97); runtime artifact pins are unchanged.

Review corrections in a2ce494d and 24d7e6c9:

  • Isolate unavailable runsc inventory/orphan recovery from healthy backends. The ACL-enabled constructor uses the Manager's bounded single-in-flight recovery lane; persisted policy, generation, endpoint ownership and leases remain retained/quarantined until both inventory and ACL reconciliation succeed. Cleanup remains available.
  • Give each upper OCI entry exactly one lower reference, including overlapping first acquisitions, cancellation, failed release and shutdown retries. Per-image locking preserves different-image concurrency without holding a global lock over I/O.
  • Retry a volatile staging overlay without the option only on EINVAL; other errors remain failures. Real-mount assertions accept exact volatile and fsync=volatile tokens while preserving readonly, lower isolation and cleanup checks.
  • Validate caller mount destinations before managed preparation, with rollback and successful ownership transfer preserved.

Registry transport corrections in 40e0c810 and 8760ca58:

  • Derive an immutable exact-authority HTTP/HTTPS policy from operator-owned managed profiles and reject conflicting settings at startup. Apply it consistently to metadata, Nydus detection/bootstrap, actual ROFS backend configuration and OCI fallback.
  • Enforce both explicit choices at the shared client's outbound transport boundary, including private-IP/loopback authorities, lazy reads and proxy-to-origin retries. Caller insecure hints cannot override policy. Unconfigured registries keep their historical behavior.
  • Add a real non-loopback HTTP Registry / private Unix management API / kernel overlay mount regression to storage CI, plus real HTTP/TLS client controls and ROFS backend policy tests.

Validation on final head 8760ca581852940d58241f6255d8c35867da781e (tree f4d534a226900018ed0c5da868d0a19eb2bd77cf):

  • Formatting, vet, full Go tests, maintained runtime binary build, relevant five-package race tests and the privileged HTTP Registry mount integration pass.
  • CI run 41854265452 completed successfully: 15/15 jobs, including all eight runtime E2E cases and the real HTTP Registry mount regression.
  • Final-source managed-profile acceptance and exact build/executable identities: native remote Buildx → push to an explicitly HTTP Registry hostname → actual Nydus/ROFS preparation → Compose start → checkpoint → delete source sandbox → same-node restore. Original container/process identity, random heap/nonce, private file contents, FD offset, launch count and an established TCP connection survived restoration. A new container launched from the retained image after source deletion with --pull=never. The hostname's actual ROFS backend was HTTP despite an HTTPS default template.
  • The owned isolated instance used bounded resources and private storage on direct XFS. Environment/snapshot inventories were empty after logical cleanup; the namespace, Pod, copied Secrets, private storage and local forwarding/credentials were removed. Shared immutable Registry artifacts and normal caches were intentionally retained. Production Deployment/DaemonSet/control-plane configuration was not changed.
  • This small witness is separate from earlier cohort results and does not claim another full Harbor/Claude run or capacity/performance qualification. Older-kernel fallback has syscall-boundary coverage; no dedicated old-kernel VM test is claimed.

Scope: opt-in trusted-team preview, same-node restore. Durable collection/snapshot leases, daemon/host restart recovery, cross-node portability, automatic artifact GC and production multi-tenant isolation remain unqualified. Earlier Draft #81/#83 overlap this integrated change and require reconciliation before independent merging. Reviewer threads remain open for independent verification; no approval or merge is performed here.

rcyw and others added 17 commits November 3, 2026 10:59
Merge upstream main b5dab29 and retain the tested Hong Kong host
boot, stopped-orphan and ACL recovery fixes. Keep the current runtime
artifact pins and PVM backend changes from upstream.

Validation: make check-fmt vet test.
Signed-off-by: 月可 <hukeyang.hky@antgroup.com>
Signed-off-by: 月可 <hukeyang.hky@antgroup.com>
Treat omitted and zero restore_concurrency as unlimited. Only positive
values enable the per-process Start RPC semaphore, while health waits,
request deadlines and cancellation keep their existing semantics.

Add TOML-backed regression coverage that holds more than 16 restores
for both disabled configurations, and update the checkpoint contract.

Signed-off-by: 月可 <hukeyang.hky@antgroup.com>
Adapt mount and quota preparation from Tianyu Zhou\u0027s 4ff87a3 while preserving the current filestore validation and parent mount namespace handling. Persist the per-sandbox checkpoint opt-in and require managed artifacts on restore.

Co-authored-by: Tianyu Zhou <zhoutianyu1995@gmail.com>
Expose immutable image preparation through a private management socket
for trusted node adapters. Share OCI ownership with rootfs and image
mount consumers so deleting another sandbox cannot stop a managed ROFS.

Keep the default image service unchanged when management is disabled.
Document session-pinned preview lifetime and the remaining durable lease
and restart-recovery requirements for checkpointed image collections.

Signed-off-by: 月可 <hukeyang.hky@antgroup.com>
Require a usable root reservation in addition to the managed image upper.
Reject insufficient total quotas before creating collection state.

Exclude the current authorization allowlist from runtime compatibility,
while continuing to authorize each restore against current tenant policy.
Cover storage geometry, native filestore reservation, allowlist expansion
and revocation with regression tests.

Signed-off-by: 月可 <hukeyang.hky@antgroup.com>
Use the image manager authentication parser for managed guest preparation.
Standard Docker auths files and flat operator credentials now share the
same host normalization, with safe errors for absent entries.

Cover both file shapes and root Registry URL aliases with regressions.

Signed-off-by: 月可 <hukeyang.hky@antgroup.com>
Keep unrelated image preparation out of global ownership locks. Preserve
per-image acquire/release ordering and independently canceled waiters.
Tie Nydus bootstrap, launcher and readiness to the acquire context, join
rollback before returning failure, and drain management handlers on close.

Delegate bounded digest-verified broker metadata to the existing Registry
client with native scoped Bearer transport reuse and refresh. Keep build
and pull credentials separate and retain failed cleanup identities.

Cover concurrent images, shared references, cancellation, late success,
shutdown, token reuse/refresh and metadata validation with regressions.

Signed-off-by: 月可 <hukeyang.hky@antgroup.com>
Merge sandboxd main through PR inclusionAI#96, retaining its private-root hosts seed
and checkpoint compatibility validation. Preserve managed Docker filestore
uppers, restore admission and image preparation fixes when resolving the
runtime adapter conflicts.

Keep the complete pinned runsc bundle unchanged. Format, vet and the full
unit suite passed for this combined source before packaging.

Signed-off-by: 月可 <hukeyang.hky@antgroup.com>
Copilot AI balanced review requested due to automatic review settings November 3, 2026 17:56

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

Merge main fbe69e5 while retaining managed preparation, restore admission,
and rollback ownership. Validate caller mount destinations before managed
preparation so invalid or nil mounts cannot allocate host resources.

Add five managed-start regression cases alongside the upstream plain-start
checks. Formatting, vet, focused tests and the full Go suite pass.

Signed-off-by: 月可 <keyang.hu@qq.com>
@hky1999
hky1999 marked this pull request as ready for review November 4, 2026 07:06

@rcyw rcyw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes on d2509ab768dca8c82c9bf98ca3d420f0265578f5, compared with main fbe69e566859718414a207fa780b8dab363b9381. The optional Buildx/Compose direction is useful, but default-off profiles do not isolate all of this PR's shared-path changes. I independently reproduced two regressions and identified one additional host-compatibility risk; details are anchored in the inline comments.

Findings and evidence

  1. R1 / P1 — one runsc inventory failure prevents all-backend startup. The same startup-isolation probe passes on main and fails on this head with one runtime inventory failure prevented all-backend startup: list runsc state during startup recovery: temporary runsc inventory failure. No managed profile is needed to trigger this.

  2. R2 / P2 — enabling the management socket leaks a shared OCI reference through the ordinary image-volume path. A deterministic two-acquire barrier exercises the real ociMountManager plus SharedOCIService. Management disabled: PASS. Management enabled: FAIL after both ordinary volume owners release, with all ordinary volume owners deleted but backend unmounts=0, want 1. This is a lifecycle-contract mismatch between the new wrapper and an existing caller.

  3. R3 / P2 — unconditional volatile narrows existing runsc host compatibility. The common staging helper also serves GPU and missing-mount-target Start/Restore paths. Linux v5.9's parser lacks this option and returns EINVAL for unknown options; v5.10 implements it. The pinned gVisor requirement is Linux 5.6+. This is a source-verified compatibility finding, not an older-kernel runtime test result.

Required before merge

  • R1: keep failed runsc reconciliation closed at that backend's admission boundary, preserve tracked state and retry cleanup, while other healthy backends and process readiness remain available. Cover startup List failure, cleanup failure, successful retry and recovery; failed inventory must never become an empty inventory.

  • R2: align existing rootfs/volume acquire/release ownership with the new shared service. Add maintained deterministic concurrent-first-acquire/final-release coverage, including ordinary and management consumers together, cancellation, failed-release retry and both management-enabled/disabled configurations.

  • R3: preserve the previous staging-overlay path when volatile is unsupported and test supported/unsupported capability handling. If the project intentionally raises its minimum host capability, make that compatibility change explicit and enforce it in preflight before accepting workloads. Existing GPU, ordinary mount-target preparation and Start/Restore behavior must have a defined supported path.

  • On the final fixed SHA, rerun make check-fmt, make vet, make test, relevant race tests and the complete existing runtime CI matrix, retaining the writable /etc/hosts and layered RO/RW/nested-mount C/R regressions.

  • Qualify a profile-enabled same-node smoke test on the fixed candidate: cold start, checkpoint, source sandbox deletion and restore, with private writable data/process continuity, retained collection/image ownership and correct final-owner cleanup. Record the actual sandboxd SHA/runtime bundle and configuration. The older deployed 653d4b92 cohort and this head's default runtime CI have different validation scopes.

Validation already completed

Check Result
Independent make check-fmt vet test on clean d2509ab, Go 1.27.3 linux/amd64 PASS
Relevant maintained image-manager/runsc/server/sandbox tests under -race PASS
Hosted CI 41788360428 at d2509ab 15/15 PASS
Identical R1 probe on fbe69e5 / d2509ab PASS / FAIL
R2 probe with management disabled / enabled PASS / FAIL
Older-kernel runtime qualification Not run; R3 is source-verified

All independent checks and probes used separate review worktrees. The host's unrelated BASH_ENV injection was removed for Go checks. The two new probes use temporary directories and fake runtime/image backends; R2 runs the actual upper manager and new wrapper together. I did not rerun the full native runtime matrix or the managed Compose deployment, and I am not treating the author's earlier deployment results as independent qualification of this head.

Long-term design assessment

Keep the official Buildx/Compose input semantics, shared read-only image content, budgeted private writable layers and whole-sandbox C/R primitives. Keep product-specific profile policy in a clear optional adapter boundary. The documented explicitly enabled, trusted-team, same-node preview is a reasonable scope after the compatibility fixes above. Durable sandbox/snapshot/artifact leases, dependency reconstruction after service restart, safe GC and production tenant isolation remain requirements for stable rollout; they are still deferred by this preview's declared scope.

Self-contained reproduction tests

Place the following files in dedicated checkouts. On the PR head, add both files; on main fbe69e5, add only the startup test. These are temporary review probes, with no production-source modifications.

pkg/sandbox/review_pr98_startup_test.go

package sandbox

import (
	"context"
	"errors"
	"testing"
	"time"

	svc "github.com/inclusionAI/sandboxd/pkg/runtime"
	cmap "github.com/orcaman/concurrent-map/v2"
)

type reviewPR98BrokenInventory struct{ svc.FakeRuntimeHandler }

func (*reviewPR98BrokenInventory) List(context.Context) ([]*svc.State, error) {
	return nil, errors.New("temporary runsc inventory failure")
}

func TestReviewPR98StartupPreservesOtherRuntimeAdmission(t *testing.T) {
	handlers := cmap.New[svc.Handler]()
	handlers.Set("runsc", &reviewPR98BrokenInventory{})
	handlers.Set("runc", svc.NewFakeRuntimeHandler())
	m, err := NewManager(t.TempDir(), handlers, make(chan bool, 1), nil, 4)
	if err != nil {
		t.Fatalf("one runtime inventory failure prevented all-backend startup: %v", err)
	}
	defer m.Stop()
	_, polled := m.collectRuntimeStates(50 * time.Millisecond)
	if !polled["runc"] || m.RuntimeHealthError("runc") != nil || m.RuntimeHealthError("runsc") == nil {
		t.Fatalf("backend isolation lost: polled=%v", polled)
	}
}

internal/server/review_pr98_image_ownership_test.go

package server

import (
	"context"
	"sync/atomic"
	"testing"
	"time"

	imageapi "github.com/inclusionAI/sandboxd/pkg/imagemanager/api"
)

type reviewPR98ImageBackend struct {
	imageapi.Service
	unmounts atomic.Int32
}

func (*reviewPR98ImageBackend) MountOCI(context.Context, *imageapi.OCIMountRequest) (*imageapi.OCIMountResponse, error) {
	return &imageapi.OCIMountResponse{MountPath: "/fake/owned/rofs"}, nil
}

func (b *reviewPR98ImageBackend) UmountOCI(*imageapi.OCIUmountRequest) error {
	b.unmounts.Add(1)
	return nil
}

type reviewPR98AcquireBarrier struct {
	imageapi.Service
	acquired chan struct{}
	release  chan struct{}
}

func (b *reviewPR98AcquireBarrier) MountOCI(ctx context.Context, req *imageapi.OCIMountRequest) (*imageapi.OCIMountResponse, error) {
	response, err := b.Service.MountOCI(ctx, req)
	b.acquired <- struct{}{}
	select {
	case <-b.release:
		return response, err
	case <-ctx.Done():
		return nil, ctx.Err()
	}
}

func TestReviewPR98ConcurrentLegacyImageMountsReleaseFinalOwner(t *testing.T) {
	for _, shared := range []bool{false, true} {
		t.Run(map[bool]string{false: "management-disabled", true: "management-enabled"}[shared], func(t *testing.T) {
			backend := &reviewPR98ImageBackend{}
			var service imageapi.Service = backend
			if shared {
				service = imageapi.NewSharedOCIService(service)
			}
			barrier := &reviewPR98AcquireBarrier{Service: service, acquired: make(chan struct{}, 2), release: make(chan struct{})}
			mounts := newOciMountManager(barrier)
			ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
			defer cancel()
			done := make(chan error, 2)
			for range 2 {
				go func() { _, err := mounts.mountOCIWithContext(ctx, "same-image"); done <- err }()
			}
			for range 2 {
				select {
				case <-barrier.acquired:
				case <-ctx.Done():
					t.Fatal("mount acquisitions did not complete")
				}
			}
			close(barrier.release)
			for range 2 {
				if err := <-done; err != nil {
					t.Fatal(err)
				}
			}
			for range 2 {
				if err := mounts.unmountOCI("same-image"); err != nil {
					t.Fatal(err)
				}
			}
			if got := backend.unmounts.Load(); got != 1 {
				t.Fatalf("all ordinary volume owners deleted but backend unmounts=%d, want 1", got)
			}
		})
	}
}

PR-head command:

env -u BASH_ENV GOMAXPROCS=8 GOFLAGS=-p=8 \
  go test -count=1 -run '^TestReviewPR98' -v ./pkg/sandbox ./internal/server

Main-control command (startup test only):

env -u BASH_ENV GOMAXPROCS=8 GOFLAGS=-p=8 \
  go test -count=1 -run '^TestReviewPR98' -v ./pkg/sandbox

The PR-head command fails the two assertions described above; the main startup control and the head's management-disabled subtest pass. Relevant maintained race checks were run separately, excluding these intentionally failing probes:

env -u BASH_ENV GOMAXPROCS=8 GOFLAGS=-p=8 \
  go test -race -count=1 \
  -run 'Test(Shared|Canceled|Management|Private|Immutable|MissingCached|Metadata|FileBacked|Filestore|Managed|WaitRuntime|RuntimePoll|Bootstrap|StartRejects|RestoreAdmission|RunscRestore|HostBoot)' \
  ./pkg/imagemanager/... ./pkg/runtime/runsc ./internal/server ./pkg/sandbox

Comment thread pkg/sandbox/startup_recovery.go Resolved
Comment thread pkg/imagemanager/api/shared_oci.go Resolved
Comment thread pkg/runtime/runsc/nvproxy_filesystem.go Resolved
Keep failed runsc startup recovery local to that backend's admission and
retry orphan cleanup through its bounded single-in-flight inventory lane.
A successful list cannot reopen admission before cleanup also succeeds.

Serialize the ordinary OCI manager's first acquire and final release per
image, using the same cancellation-aware gate as managed image services.
Retain ownership after failed release or canceled-acquire rollback so a
retry cannot leak a lower reference or unmount a managed consumer's ROFS.

Retry disposable staging overlays without volatile only when the kernel
returns EINVAL. Preserve the optimized path on supported kernels and
return other mount failures. Keep checkpoint persistence policy intact.

Add regressions for all three review findings, canceled waiters/leaders,
release/acquire races, cleanup retry and independent-image parallelism.

Signed-off-by: 月可 <hukeyang.hky@antgroup.com>

rcyw commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

Follow-up review summary for a2ce494dbe4550ed9d5cbe54100a97de6de4288f. Changes are still required before merge. The detailed remaining R1 reproduction and validation evidence are in this review-thread reply.

Item Independent follow-up result
R1: backend isolation Partially fixed; still blocking. Manager initialization now isolates runsc failure. With ACL enabled and retained runsc metadata, however, activeACLBindings -> stoppedRunscForACLRecovery -> handler.List still propagates the inventory error to NewSandboxService, aborting service initialization. The same ACL-recovery probe passes on main fbe69e5 and fails on a2ce494d.
R2: shared OCI ownership Fix verified at this candidate. Concurrent first acquisition/final release passes in legacy/shared modes, including mixed managed/ordinary owners, cancellation, failed-release retry and shutdown. The maintained concurrent-acquire test reproduces the original failure when copied unchanged to d2509ab.
R3: volatile compatibility Production fallback logic verified at the syscall boundary. Supported, EINVAL fallback, failed fallback, permission and busy cases pass. The maintained real-mount test still has a portability defect: it rejects the valid fsync=volatile spelling reported by this host. An independent test-file overlay accepting both spellings passed the remaining real-mount checks; the original maintained test remains failed until corrected. No older-kernel VM qualification was performed.

Independent validation on a2ce494d passed formatting, vet, the full Go suite, full race tests for sandbox/server/runsc/image-manager API/keyedlock, and complete native runsc-systrap, runsc-kvm and runc E2E. The native runs retained writable/deleted-hosts C/R, layered RO/RW/nested mounts, network, daemon-recovery, storage and OOM regressions. These healthy-inventory successes do not close the remaining ACL fault-injection failure.

Remaining before merge:

  • Complete R1 across ACL initialization as well as Manager recovery. Preserve/quarantine unknown bindings and leases, keep runsc closed while recovery is incomplete, and keep healthy backends available. Add maintained ACL-enabled retained-metadata failure/retry/recovery coverage; do not treat failed inventory as empty or silently discard ACL state.
  • Correct the real-mount assertion to accept both volatile and fsync=volatile, retaining its readonly, shared-lower isolation and cleanup checks, then rerun the maintained test.
  • Ensure the upstream PR head actually contains the final fixes and obtain fresh exact-head CI/full runtime-matrix results. At this check the PR API still reports d2509ab, so its old 15/15 CI does not qualify a2ce494d.
  • Complete the profile-enabled same-node start/checkpoint/source-deletion/restore smoke test on the final fixed candidate, including private data/process continuity and retained-image/collection ownership. The independent ordinary-runtime E2E above is not managed Compose deployment qualification.

The optional managed-runtime direction remains useful within the declared trusted-team, same-node preview scope. The original change request remains open pending the fixes and validation above; no approval or merge has been performed.

Complete backend isolation through ACL initialization instead of issuing
another raw runtime List that can abort the whole service. Retain unknown
ACL owners, policies and endpoint leases without activation or orphan GC,
including owners whose sandbox metadata is unavailable.

Retry inventory, orphan cleanup and pending ACL restore in one bounded
runtime poll lane. Keep runsc admission and policy updates closed until
recovery succeeds, while healthy backends and explicit cleanup remain
available. Do not regenerate other runtimes' policies on a runsc retry.

Accept both volatile and fsync=volatile in the maintained real-mount test,
retaining readonly, lower-isolation and cleanup assertions. Add recovery,
quarantine persistence, session-independent admission and alias tests.

Signed-off-by: 月可 <hukeyang.hky@antgroup.com>

hky1999 commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator Author

Follow-up to the review summary: pushed 24d7e6c9. The upstream PR head now correctly points to this commit.

  • R1: completed ACL-constructor isolation, retained/quarantined unknown bindings and leases, bounded retry, and admission reopening only after successful inventory and ACL recovery. Added retained/missing/untracked-owner, policy-quarantine, healthy-backend and recovery regressions. Detailed reply.
  • R2: retained the previously verified ownership fix and reran the affected packages under the race detector.
  • R3: fixed the real-mount assertion for both documented mountinfo spellings; the maintained privileged test now passes for both host-writable variants. Detailed reply.

On this exact source, formatting, vet, the full Go suite, race tests for all six affected packages, and runtime-binary build pass. Fresh CI/full runtime matrix is running; the seven base jobs have passed. The final-candidate profile-enabled same-node start/checkpoint/source-deletion/restore smoke remains pending and is not implied by these checks or the earlier ordinary-runtime E2E. The PR description now records these boundaries. No cluster deployment or runtime pins were changed; review threads remain open.

rcyw commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

Independent re-review of 24d7e6c9bf4de622d45b89f0a312cd55b87efddb: the ACL recovery and mount-test fixes are independently verified; the final managed-profile smoke remains a pre-merge requirement.

Item Current result
R1: ACL initialization/backend isolation Verified fixed. The unchanged previous ACL reproduction now passes, along with retained ownership, quarantine, failed-ACL retry and eventual-recovery regressions. Detailed verification.
R2: OCI lower-reference ownership The previously verified fix is unchanged; all seven maintained ownership regressions and the affected race tests pass on this candidate.
R3: fallback and real-mount assertion The original maintained privileged test now passes on my Linux 6.8 host for both host-writable variants, without a virtual replacement test file. Exact volatile/fsync=volatile tokens are accepted; readonly, lower isolation and cleanup assertions still pass. EINVAL-only fallback/error-path tests pass. No old-kernel VM execution is claimed.
PR head and fresh full CI Resolved. API, upstream PR ref and author branch all identify 24d7e6c9. CI run 41832404766 completed successfully: 15/15 jobs, including all eight runtime cases.
Final-candidate managed-profile validation Still pending, as explicitly recorded in the author's latest reply and PR body. The ordinary runtime matrix does not enable/qualify this profile.

Independent local validation on this source passed:

  • make check-fmt vet test e2e-runtime-binaries on a clean candidate checkout.
  • Full go test -race -count=1 for ./pkg/sandbox ./internal/server ./pkg/runtime/runsc ./pkg/imagemanager/api ./internal/keyedlock ./pkg/networkmanager/networkacl.
  • The unchanged ACL reproduction and 23 maintained ACL/startup/recovery/OCI-ownership tests.
  • SANDBOXD_RUNSC_MOUNT_TEST=1 with the original real-overlay, mount-option alias and fallback tests in a private mount namespace on Linux 6.8.0-139-generic.

For CI provenance, the build job checked out GitHub's synthetic merge ed4d57ff; I verified its source tree is identical to 24d7e6c9 (77ebcf37af0fa946655323e27e30ba878cea9578). The eight runtime execution steps, applicable KVM probes and the runsc-reflink XFS fixture all succeeded. Local builds separately report the candidate revision and vcs.modified=false.

The remaining merge gate is:

  • On this final candidate, explicitly enable the managed profile and record the source/build/image identities and relevant profile configuration. Run a real managed Docker workload through start → checkpoint → source sandbox deletion → same-node restore, demonstrating preserved original process/private writable state and usable managed image dependencies after source deletion. Publish the assertions/results and cleanup/retained-dependency outcome. Earlier 653d4b92 deployment results and the ordinary runtime CI do not replace this evidence.

No new code blocker was found in this incremental review. The opt-in same-node preview direction remains reasonable within its documented scope; durable restart/cross-node/GC/production tenant isolation are still outside that qualification. Keep the existing Request changes until the remaining smoke evidence is reviewed. No approval or merge performed.

hky1999 commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator Author

@rcyw I have now completed the missing profile-enabled managed Docker smoke on final PR head 24d7e6c9bf4de622d45b89f0a312cd55b87efddb. No candidate code changed after that head. This is new final-head evidence, separate from the earlier 653d4b92 deployment and the ordinary runtime matrix.

Source, build and runtime identity

  • Clean detached source: 24d7e6c9bf4de622d45b89f0a312cd55b87efddb, tree 77ebcf37af0fa946655323e27e30ba878cea9578; clean before and after the build. Built using Go 1.25.7, make GO=/usr/local/go/bin/go 'GOFLAGS=-p=4 -buildvcs=true' e2e-runtime-binaries. All four maintained binaries were staged and hash-checked in the isolated instance; /proc/<sandboxd-pid>/exe was independently checked before capture, after source deletion and after restore.
  • Sandboxd SHA-256: b7d7b182332a9eaddd53683e2d22feba0418aee84e55351425e3558b2bde2424. These local build outputs do not embed VCS fields, so the provenance here is the external clean-checkout/build receipt plus the exact executable hash, not a claim of embedded vcs.modified=false. Two build-driver assertions expecting embedded metadata failed before this was corrected; the compiler builds themselves succeeded.
  • Parent all-in-one image: ling-swe-acr-registry-vpc.cn-hongkong.cr.aliyuncs.com/akernel/all-in-one@sha256:7688ce230343b944465dda7c574d5cc41ec0480dddcc115df5c79e34d59b14de. Only the source-built Sandboxd/sbox/shim/logger binaries were replaced inside the owned test instance. Existing complete runsc installation retained: release-20261012.0-akernel.2-2-g63de4896444e, OCI spec 1.2.1.
  • SDK: akernel-sdk 0.2.0rc6, hky1999/AKernel commit f36e6cf94a3e7318626626878123b46369345754 (sdk/python). This test calls the public SDK directly; it does not claim another Harbor/Claude or 237-task cohort rerun.
  • Exact-head CI run 41832404766 is now complete: 15/15 jobs passed, including all eight runtime E2E jobs. I checked the job results again before this reply.

Explicit profile configuration and isolation

An owned namespace/Pod ran its own API, Redis, Coordinator, ADXlet, Sandboxd and managed-image broker. It did not register with or replace the existing production workers/control plane. It used a uniquely marked private directory on the existing node's direct XFS mount; the same device/directory identity and a real FICLONE ioctl were verified. Pod limits were 4 CPU/8 GiB; Sandboxd reserved 2 GiB for the parent, and the test sandbox requested 1 CPU/3 GiB, with 2 CPU/4 GiB limits and 12 GiB writable storage.

The selected harbor-v1 profile had a 4096 MiB file-backed private Docker upper, 4096 MiB minimum root (8192 MiB scheduling floor), runsc.checkpoint_mode="reflink", current tenant admin authorized (allowed_tenants=["admin","default"]), and its provider pointed only to the owned broker. The isolated ADXlet advertised AKERNEL_RUNTIME_PROFILE_harbor-v1=8192. Existing remote BuildKit and internal Registry services were used without changing their configuration. Registry credentials stayed in private platform files. The isolated standalone harness supplied the normal resolver-file and profile resource-label inputs; no extra candidate code change was needed.

The fixed tools image was 172.30.25.125:5443/managed/aebe9a00d64104a4e3a5ae86aa4b5739/single@sha256:6e8f395163c2aeddd622ee92f92db8589a7065d9860f945925bbf5ee0f34bdd3. A real original Compose project built FROM docker.io/library/python:3.12-alpine plus the controlled application through native remote Buildx, published/prepared its Nydus/ROFS image, then started its bridge-networked main container. Resulting application reference: 172.30.25.125:5443/managed/838ec879a50471861fa321b8da28d963/main@sha256:4ff92e710bed700f32f046f3764d3fe765c57dd65993d7b78e012520c0fa14a9.

Executed sequence and assertions (all passed)

  1. ComposeProject.prepare(..., runtime=DockerRuntime(profile="harbor-v1")) → fresh Sandbox(..., **project.sandbox_options()) → start_runtime() → build() → pull() → up(). The application allocated a random 16 MiB heap, generated an in-memory nonce, held an open private-upper file at offset 7 and served requests over a persistent TCP connection from an outer-sandbox client. Each inspection used a new random challenge, echoed by the application, to avoid accepting cached output.
  2. Saved a reusable whole-sandbox checkpoint with create_snapshot(). Changed the source's in-memory counter 41→99, open-file offset 7→19, and private file contents after capture. The mutation was observed over the same TCP session.
  3. Killed/deleted the source sandbox before restoring. An independent API listing confirmed the source was absent. The managed collection record, collection directory and guest dependency directory still existed after source deletion; this was independently observed from the owned node instance, not inferred from an SDK handle.
  4. Sandbox.from_snapshot(..., **DockerRuntime(profile="harbor-v1").sandbox_options()) → ComposeProject.attach(...) → reconnect_network(). No fresh runtime start, build, pull or Compose up was called during restore. Container ID/StartedAt, original process PID/start time/nonce, heap hash, launch count 1, private file hash, open FD offset and persistent client TCP socket identity all matched their captured values. The counter returned to 41 and the FD offset to 7; the post-capture file mutation was rolled back.
  5. After that restore, docker run --pull=never --rm --entrypoint python3 <captured application digest> -c 'print("PR98_POST_RESTORE_NEW_CONTAINER_OK")' succeeded. Thus the retained managed lower/image dependencies were usable for a new container after source deletion, not merely for an already-running restored process.

Captured versus restored witness:

{
  "captured": {
    "nonce": "b9fed24fb9520adb9dd4fd08f2c46a51",
    "pid": 1,
    "starttime": "1498",
    "heap_sha256": "5d57095b9850dab6ccb2fd632ac780fcb423ccea814406a1d8e6d83e3c5cfc98",
    "counter": 41,
    "fd_offset": 7,
    "private_sha256": "10e19eb1616569d3d861ef35b696ff3c402232b08666b2304189d5ca0706dfb0",
    "launches": 1,
    "client": {
      "pid": 413,
      "nonce": "42aa302fd38117ed4b70e9f50813103c",
      "starttime": "1650",
      "tcp_inode": "socket:[168]"
    }
  },
  "source_after_capture": {
    "counter": 99,
    "fd_offset": 19,
    "private_sha256": "57096724968ee84793d6cc64747a2728de51f9fc516cd998ef41ea5d7412c35e"
  },
  "restored": {
    "nonce": "b9fed24fb9520adb9dd4fd08f2c46a51",
    "pid": 1,
    "starttime": "1498",
    "heap_sha256": "5d57095b9850dab6ccb2fd632ac780fcb423ccea814406a1d8e6d83e3c5cfc98",
    "counter": 41,
    "fd_offset": 7,
    "private_sha256": "10e19eb1616569d3d861ef35b696ff3c402232b08666b2304189d5ca0706dfb0",
    "launches": 1,
    "client": {
      "pid": 413,
      "nonce": "42aa302fd38117ed4b70e9f50813103c",
      "starttime": "1650",
      "tcp_inode": "socket:[168]"
    }
  }
}

Source ID: default-pr98-final-managed-source; snapshot: snapshot-8a07d05ff7daba265e770aa202782ff07aa64338ddc6e8cbe1d094bbe0e92cfc; restored ID: default-pr98-final-managed-restored. The SDK capture and restore calls took approximately 0.346 s / 0.529 s respectively for this deliberately small witness; these are smoke timings, not a kv-live-surgery or capacity/performance qualification.

Cleanup and retained dependencies

The restored sandbox was deleted and the snapshot deletion was requested; a subsequent authoritative isolated API query returned zero environments and zero snapshots. The Sandboxd runtime directory contained only its shared runsc/null-netns file, with no sandbox state left. The owned Pod/namespace, copied pull Secrets, uniquely marked private XFS directory, port-forward process and local ephemeral API key/trust files were then removed; the cleanup receipt confirms all of these. The test auto-cleanup timer was stopped. Managed collection dependencies were deliberately retained through source deletion/restore and removed only with the owned private instance directory after the test. The immutable test image in the shared Registry and normal BuildKit/ROFS/Dragonfly caches were retained intentionally; no global Registry deletion or cache GC was run. No existing production Deployment/DaemonSet/control-plane configuration was modified by this smoke.

The qualification boundary remains opt-in trusted-team, same-node restore. This does not qualify daemon/host restart recovery, cross-node portability, durable snapshot/artifact leases, automatic artifact GC or production multi-tenant isolation. I have left the review threads open for your independent verification and have not approved or merged the PR. Please review this final-head evidence against the remaining merge gate.

@rcyw rcyw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The independently enabled managed-profile smoke on 24d7e6c9bf4de622d45b89f0a312cd55b87efddb found R4: the configured private HTTP Registry cannot prepare managed images through the new image-management endpoint. I am keeping Request changes and withholding approval.

R1/R2/R3 remain verified as reported in the previous follow-up, and the candidate's CI remains green. This is an additional failure in the enabled feature path; I am not claiming a newly discovered default-runtime regression.

What actually ran, using an owned local network/data directory and the unchanged candidate binary:

  • Built the tool image and the pinned AKernel 49c35a2d image-bridge/adapter; helper tests passed. Used the companion standalone OCI spec, the configured runsc raw/packet-socket flags, a private management socket, disposable Basic authentication, and an isolated reflink-capable XFS fixture.
  • Started the outer sandbox with managedRuntimeProfile=smoke, an authorized tenant, and the default 4096 MiB private upper + 4096 MiB root geometry. Verified the injected EROFS bootstrap, readonly operator configuration and file-backed tmpfs.
  • Started the real Docker/containerd/snapshotter stack. Native remote Buildx successfully built and pushed a Nydus image under the collection-authorized repository prefix.
  • Preparing that image through the broker failed. Bypassing the adapter and calling Sandboxd's private endpoints directly isolated the same failure:
Request for the same immutable image Actual result
POST /registry_metadata with insecure=true 200; manifest SHA-256 matches the requested digest and contains Nydus layers
POST /oci_mount with the configured Registry hostname 500; attempts https://<registry>:5000/v2/, then reports http: server gave HTTP response to HTTPS client
POST /oci_mount through a node-local localhost:5000 route to the same Registry/image 200; real ROFS mount and the expected image process are returned

Thus the image, authentication, node reachability and Nydus/FUSE prerequisites are valid. The direct endpoint reproduction also removes the pinned adapter's older auth-file format from the causal path. The profile has registry_insecure=true and its retained record/profile correctly select HTTP, but that choice cannot reach /oci_mount: its request/forwarding path carries only the image reference, and Nydus detection calls FetchImage(..., false); the ordinary OCI fallback also selects HTTPS for this hostname. Successful insecure metadata access does not configure later mounting.

Minimal reproduction on an explicitly enabled private HTTP profile with any valid digest-pinned Nydus image:

import base64
import hashlib
import http.client
import json
import socket

# Use the configured non-loopback Registry hostname and an existing image.
ref = "registry.internal:5000/managed/COLLECTION/probe@sha256:DIGEST"

class Unix(http.client.HTTPConnection):
    def connect(self):
        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
        self.sock.connect("/run/sandboxd/image-api/image.sock")

def post(path, body):
    conn = Unix("localhost", timeout=120)
    conn.request("POST", path, json.dumps(body),
                 {"Content-Type": "application/json"})
    response = conn.getresponse()
    status, payload = response.status, response.read()
    conn.close()
    return status, payload

status, payload = post("/registry_metadata", {
    "reference": ref, "kind": "manifests", "insecure": True,
})
assert status == 200, (status, payload)
manifest = base64.b64decode(json.loads(payload)["data"])
assert "sha256:" + hashlib.sha256(manifest).hexdigest() == ref.split("@")[1]
assert any("nydus" in json.dumps(layer).lower()
           for layer in json.loads(manifest)["layers"])

status, payload = post("/oci_mount", {"image_url": ref})
assert status == 200, (status, payload)  # Fails: 500, HTTPS to HTTP Registry.

The fixed candidate must pass:

  • Carry the trusted operator-selected Registry transport through actual managed preparation, including Nydus detection/mounting and relevant OCI fallback, while preserving ordinary HTTPS defaults.
  • Add a regression that uses a non-loopback HTTP Registry hostname and real registry-client/mounter behavior. Mocking MountOCI alone or only testing /registry_metadata does not cover this failure.
  • Rerun required checks/CI as appropriate and complete the enabled-profile start → checkpoint → source deletion → same-node restore smoke on the final fixed candidate, checking original process/private-upper state and retained image dependencies.

The complete managed C/R smoke is not passed: the configured-hostname image-preparation failure prevents launching its inner workload, so no checkpoint/restore success is being claimed. No product source, runtime pins, existing deployment or default branch was changed, and no merge was performed.

Comment thread pkg/imagemanager/api/management.go Resolved
Signed-off-by: 月可 <hukeyang.hky@antgroup.com>

@rcyw rcyw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review of 40e0c81037aba1fb3287e5df967dfedb5a28b801: the original R4 HTTP-hostname failure is addressed, but the new explicit HTTPS policy has an inconsistent boundary that still needs correction (R5/P2).

Independent checks on this candidate passed:

  • Clean-source make check-fmt vet test e2e-runtime-binaries.
  • The maintained privileged make managed-registry-test: real HTTP hostname, private management Unix API, OCI extraction/kernel overlay mount, repeated ownership/release, and Nydus bootstrap/config acquisition.
  • Full race tests for config, shared Registry client, Nydus, distillfs, management API, server, sandbox, runsc and keyedlock.

The operator-owned exact-authority policy and startup rejection of conflicting transports are a reasonable direction. The remaining issue is that only http entries affect the shared client's selection. An explicit https entry falls through to the registry library's private-IP/loopback HTTP behavior, while the Nydus backend consumes that same entry as https. Thus one declared operator policy does not mean the same transport at each stage.

A real local Registry reproduction on the unmodified candidate produced:

Configuration / endpoint Result
HTTPS policy + TLS Registry under a hostname PASS
HTTPS policy + TLS Registry under its loopback IP PASS
No policy + plain HTTP loopback Registry PASS, expected legacy compatibility control
Explicit HTTPS policy + plain HTTP loopback Registry Fetch succeeds; expected rejection fails

The getter reports https, but FetchImage(..., false) successfully returns the plain-HTTP image. This is distinct from preserving historical behavior for unconfigured registries. The library has the same heuristic class for RFC1918 authorities; the explicit operator selection should override it. Metadata likewise only special-cases HTTP, so please make both choices authoritative across fetch/metadata/backend paths, without changing unconfigured registries.

Minimal regression in package imageregistry:

func TestExplicitHTTPSRejectsPlainHTTP(t *testing.T) {
    server := httptest.NewServer(registry.New())
    defer server.Close()
    host := server.Listener.Addr().String()
    tag, err := name.NewTag(host+"/probe:latest", name.Insecure)
    if err != nil { t.Fatal(err) }
    if err := remote.Write(tag, empty.Image); err != nil { t.Fatal(err) }

    legacy, err := NewClientWithRegistrySchemes("", nil)
    if err != nil { t.Fatal(err) }
    if _, err := legacy.FetchImage(context.Background(), host+"/probe:latest", "", false); err != nil {
        t.Fatalf("unconfigured compatibility control: %v", err)
    }

    client, err := NewClientWithRegistrySchemes("", map[string]string{host: "https"})
    if err != nil { t.Fatal(err) }
    if scheme, ok := client.RegistryScheme(host+"/probe:latest"); !ok || scheme != "https" {
        t.Fatalf("explicit policy = %q, %t", scheme, ok)
    }
    if _, err := client.FetchImage(context.Background(), host+"/probe:latest", "", false); err == nil {
        t.Fatal("explicit HTTPS policy accepted a plain HTTP Registry")
    }
}

Imports: context, net/http/httptest, testing, and the go-containerregistry pkg/name, pkg/registry, pkg/v1/empty, pkg/v1/remote packages. My full temporary probe also includes the two real TLS controls above; it has been preserved and removed from the review checkout, which is clean.

New-head CI 41850746700 was still running at the latest check (13/15 jobs succeeded; two runtime jobs remained in progress). The new storage integration step passed. I have not seen a new author reply or managed-profile C/R report for 40e0c810; the detailed prior report qualifies 24d7e6c9, not this changed source.

Before approval, please close the explicit HTTPS-policy discrepancy, obtain the fixed candidate's completed CI, and provide its affected managed start/checkpoint/source-deletion/restore evidence. R1/R2/R3 retain their previously verified scope. No product source was committed or pushed, no new standalone deployment was started in this round, and no approval or merge was performed.

Signed-off-by: 月可 <hukeyang.hky@antgroup.com>

hky1999 commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator Author

@rcyw R4 and its HTTPS-policy follow-up R5 are fixed on 8760ca581852940d58241f6255d8c35867da781e. Fresh affected managed-profile acceptance on this exact source passed.

Previous Registry transport and the correction

The earlier literal-IP test used 172.30.25.125:5443, registry_insecure=true, with the existing ROFS Registry template also set to HTTP. I checked the actual worker's running configuration and unchanged Pod identity/start time, rather than inferring HTTPS from port 5443. The pinned go-containerregistry v0.20.7 library permits HTTP automatically for RFC1918 IP/loopback authorities. That masked the missing hostname transport propagation in the earlier test.

40e0c810 propagates a node-owned exact-authority policy from configured managed profiles into the shared Registry client, Nydus bootstrap acquisition, the real ROFS backend and OCI fallback; conflicting profile choices fail startup. Your R5 observation was also reproducible: merely omitting name.Insecure cannot force HTTPS for private-IP/loopback authorities. The final follow-up enforces both explicit choices on cloned outbound requests, including lazy reads and retries, and applies the choice to metadata URLs/auth-cache keys. Caller insecure hints cannot override it. Unconfigured registries retain their previous defaults; unrelated authorities are not rewritten.

Maintained regression and CI

  • make check-fmt vet test managed-registry-test, the maintained e2e-runtime-binaries build, and full race tests for config, imageregistry, Nydus, distillfs and image-manager API pass.
  • Privileged HTTP-hostname integration runs a real Registry under registry-http.test:<port> in a private mount namespace. The negative control returns metadata 200 / mount 500 without policy; configured HTTP succeeds through the real Unix API, Registry client, layer extraction and kernel overlay mount, including repeated ownership/release and cleanup. There is no mocked MountOCI. Nydus manifest/config/bootstrap acquisition and a bootstrap cache hit are also exercised; this local bootstrap fixture is synthetic and does not itself claim a FUSE mount.
  • Real HTTP/TLS matrix covers hostname, loopback and RFC1918 authorities; explicit HTTP/HTTPS and unconfigured behavior; both caller insecure hints; image/config and metadata reads. The unchanged regression fails against the preceding client implementation and passes on the final source. Backend configuration regression also verifies explicit HTTPS overrides an HTTP template.
  • Final-source CI 41854265452 completed successfully: 15/15 jobs, including all eight runtime E2E cases and the new real HTTP Registry mount integration.

Final-source managed start → checkpoint → delete source → restore

This was a new owned standalone API/Redis/ADXlet/Sandboxd/broker instance in a dedicated namespace and private direct-XFS directory, limited to 4 CPU / 8 GiB. Production Deployment/DaemonSet/control-plane configuration and runtime artifact pins were not changed.

Identity / configuration Tested value
Source / tree 8760ca581852940d58241f6255d8c35867da781e / f4d534a226900018ed0c5da868d0a19eb2bd77cf
Sandboxd executable SHA-256 c3bf20f77c42fbf639968d087cc687cc2f2c2d7c9d96f4f6286c3ffc51caa698
Build Clean source before/after; make GO=/usr/local/go/bin/go 'GOFLAGS=-p=4 -buildvcs=true' e2e-runtime-binaries. Executables do not embed VCS fields; identity is recorded externally and actual running bytes were checked.
Unchanged parent image digest sha256:7688ce230343b944465dda7c574d5cc41ec0480dddcc115df5c79e34d59b14de
Complete runsc bundle release-20261012.0-akernel.2-2-g63de4896444e; checkpoint mode reflink
SDK 0.2.0rc6, AKernel source f36e6cf94a3e7318626626878123b46369345754
Managed profile harbor-v1; file-backed private upper; HTTP Registry hostname 172-30-25-125.sslip.io:5443 resolving to the internal Registry; the isolated ROFS default template was deliberately HTTPS
Artifact 172-30-25-125.sslip.io:5443/managed/45dc761c5e5cc1b72917294a7f10497a/main@sha256:e1fb46eda3003cd60b7b4aa75676d312dc6b57af3598b94cc98f20b2af560140

The Dockerfile (FROM python:3.12-alpine, copy/start the witness) and native Compose build: went through remote Buildx → push to the hostname → managed preparation → actual Nydus/ROFS → Compose start. The generated ROFS backend confirmed HTTP for the configured hostname despite the HTTPS default template.

The source default-pr98-r5-managed-source captured a 16 MiB random heap plus process nonce/start time, counter 41, private file hash, open FD offset 7 and an established TCP client/socket. After capture, the live counter/file/offset were mutated. I then deleted the source and independently confirmed it was absent from the API before restoring. Collection records and backing dependencies remained present.

The restored sandbox default-pr98-r5-managed-restored retained the original inner container identity/StartedAt, original process/nonce/heap hash, launch count 1, counter 41, private file hash, FD offset 7, and TCP client identity/socket inode. Fresh random challenge responses proved execution continued. Restore only attached the descriptor and reconnected networking; it did not rebuild, run Compose up, restart Docker or relaunch the application. A subsequent docker run --pull=never from the retained image also succeeded after source deletion.

Snapshot: snapshot-b2053c6f51c5c66f4614b71cdfe091d6c4f6425fea0e148110855a85ababfd44. SDK wall times were approximately 0.358 s checkpoint / 0.474 s restore for this small witness; these are not capacity or representative workload performance results.

Cleanup completed: the isolated API listed zero environments and zero snapshots, then the owned namespace/Pod/copied Secrets, private XFS directory, port-forward and local temporary API key/trust files were removed. The cleanup watchdog was stopped. Shared immutable Registry artifacts and normal BuildKit/ROFS/Dragonfly caches were intentionally retained. No production deployment was changed.

This qualifies the affected opt-in, same-node managed-profile path on the final source. It does not claim a replay of the full Harbor/Claude cohort, cross-node portability, daemon/host restart recovery, durable artifact GC or production multi-tenant isolation. Reviewer threads remain open for independent verification; no approval or merge was performed.

@rcyw rcyw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed and independently qualified final head 8760ca581852940d58241f6255d8c35867da781e (tree f4d534a226900018ed0c5da868d0a19eb2bd77cf). No remaining blocking findings for the documented opt-in, trusted-team, same-node preview.

R1–R3 retain their verified fixes for backend/ACL recovery, concurrent image ownership and older-kernel overlay fallback/assertions. R4's original HTTP-hostname Nydus/FUSE counterexample is fixed; R5's unchanged explicit-HTTPS probe now passes, with real TLS and unconfigured-Registry compatibility controls. The new policy/hint/private-IP matrix and backend override coverage pass.

Independent final-head validation:

  • Clean-source make check-fmt vet test e2e-runtime-binaries, full race tests across nine relevant packages, and the privileged real HTTP Registry / Unix management API / kernel mount integration all pass.
  • CI 41854265452: 15/15 jobs successful, including all eight runtime E2E cases and the real HTTP Registry mount regression. Its synthetic merge dd860847 has the same source tree as this head.
  • A separate bounded local fixture ran real Docker/containerd/snapshotter, native remote Buildx → HTTP-hostname Registry → Nydus/ROFS → Compose. The executing sandboxd binary was checked before capture, after source deletion and after restores: SHA-256 b088a5ca0d8d87a54547d1920b6379da3ba9f831b124eb1fdadd0d057d14bca6, built from clean final source.
  • Checkpointed the running workload, changed the live source's counter/FD/private file, deleted the source sandbox, then restored two targets from the retained checkpoint. Original Docker CID/StartedAt/process identity, random in-memory nonce and 128 KiB RAM marker survived; counter/FD/private upper returned to captured state. Init launch count stayed 1. A write in restored A stayed absent from restored B. A new container using the retained image with --pull=never also succeeded after source deletion. Restore did not replay runtime startup, build, pull or Compose up.
  • Both restored targets were normally deleted before checkpoint removal; owned containers/network/cgroups/XFS mount/loop were released. The primary product checkout was untouched.

My local test uses direct sandboxd APIs plus pinned AKernel 49c35a2d helper/adapter inputs; it is complementary to the author's final-head SDK/Compose report, which also covers established TCP continuity. It is not a claim that I independently replayed that latest SDK stack.

Fixture failures were retained: an initial readiness race, and a first C/R attempt whose daemons descended from raw runsc exec and were intentionally killed by gVisor restore (Killing exec'd process). Rebuilding the fixture with the Docker stack descending from normal sandbox init resolved that lifecycle issue; the successful restores required no startup replay or product changes.

Non-blocking follow-up: automate a compact managed-profile C/R CI fixture using normal init and bounded readiness, asserting source deletion, retained-image use and independent restored uppers. The ordinary runtime matrix does not itself enable this profile.

Durable snapshot/collection leases, daemon/host restart recovery, cross-node portability, automatic GC and production multi-tenant isolation remain outside this qualification. Approving this head within the documented preview contract. No merge performed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants