Skip to content

Cache native dependency lifecycle builds#386

Merged
tolgaergin merged 14 commits into
mainfrom
codex/native-build-cache
Jul 12, 2026
Merged

Cache native dependency lifecycle builds#386
tolgaergin merged 14 commits into
mainfrom
codex/native-build-cache

Conversation

@tolgaergin

@tolgaergin tolgaergin commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a content-addressed v2 store for native lifecycle build artifacts
  • key reuse on package, graph topology, platform, Node ABI/runtime, sandbox, complete lifecycle environment, and discoverable toolchain/host package state
  • enforce package-local cacheable sandbox writes, pristine miss rematerialization, keyed build state, graph-entry/build-key concurrency locks, and stable scratch paths
  • validate v2 graph identities and support scoped package paths
  • resolve host probes through a project-excluding trusted PATH and fingerprint project-visible shadow binaries without executing them
  • preserve symlink/proxy invocation paths while separately validating and hashing canonical trust targets
  • run probes from a neutral directory under an injection-resistant environment
  • require build-visible Node to match the trusted runtime for every cacheable native lifecycle build
  • advance the RuntimeOnly key namespace to v2 so pre-hardening artifacts and keyed build state cannot be reused
  • bound pre-sandbox runtime/toolchain probes to 2 seconds through child exit and pipe-reader completion, killing the process group when descendants retain output pipes
  • persist the expensive NativeToolchain host fingerprint under the metadata cache and validate executable identities, package state, pkg-config, SDK, receipt, Rust, Node-header, and system generation inputs before reuse
  • fail closed to the complete host fingerprint for changed, expired, malformed, oversized, or untrusted snapshot state; snapshots expire after one hour
  • expose rebuild JSON cache metrics and prune incomplete or stale artifacts
  • add macOS/Linux workflow coverage plus targeted Ubuntu PR execution
  • add reproducible RuntimeOnly and NativeToolchain release benchmark evidence

Performance

RuntimeOnly path

10 release samples with the esbuild native fixture on macOS arm64 / Node 22.22.1, with identical strict-sandbox settings:

  • strict cache miss: 138.03 ms median
  • local artifact hit: 32.33 ms median (4.27x, 76.58% lower)
  • stable-path CI warm-store hit: 29.50 ms median (4.68x, 78.63% lower)
  • median artifact size: 134,444 bytes

NativeToolchain path

10 release samples with the sharp fixture on macOS arm64 / Node 22.22.1 after persistent validated host snapshots:

  • strict artifact miss: 173.10 ms median
  • local artifact hit: 59.07 ms median (2.93x, 65.88% lower)
  • stable-path CI warm-store hit: 58.49 ms median (2.96x, 66.21% lower)
  • unchanged-host key derivation: 15-24 ms
  • first measured complete host fingerprint: 126 ms
  • median artifact size: 503,469 bytes

Absolute checkout paths and the complete lifecycle environment are keyed. Differently located projects and CI runs with volatile environment values intentionally miss unless callers normalize those inputs.

Validation

  • cargo clippy --workspace --all-targets --locked -- -D warnings
  • cargo fmt --check
  • cargo build --workspace --locked
  • 4,513 non-CLI/non-workflow nextest tests passed
  • 3,472 serialized CLI binary tests passed (1 ignored)
  • 73 remaining CLI nextest tests passed
  • native_build_cache workflow: 5 passed on macOS
  • native toolchain cache module: 34 passed, including snapshot reuse/invalidation/expiry/corruption/path bounds, RuntimeOnly namespace invalidation, runtime divergence, descendant-pipe timeout, proxy symlink, neutral-directory, non-execution, output-bound, and environment-injection regressions
  • release NativeToolchain benchmark: 10 samples per scenario
  • benchmark harness self-test passed
  • targeted Ubuntu PR step covers the same five native_build_cache workflows
  • docs are in lpm-dev/rust-client-docs PR Remodel third-party login flows #94

@tolgaergin tolgaergin merged commit 59a2571 into main Jul 12, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant