Skip to content

fix: sweep the store after the daemon stops - #222

Closed
shikanime wants to merge 1 commit into
cachix:masterfrom
shikanime:fix/await-the-daemon-push
Closed

shikanime wants to merge 1 commit into
cachix:masterfrom
shikanime:fix/await-the-daemon-push

Conversation

@shikanime

Copy link
Copy Markdown

Why

cachix daemon stop returns before the daemon's upload queue drains, so a large path can be reported as Pushing … in the post step, the job can go green, and the path can still be missing from the cache. Every job involved looks successful; the loss only shows up later as a cache miss. #221 has the evidence: a 4.07 GiB dockerTools.buildImage tarball logged Pushing at 23:51:04, the daemon was stopped at 23:51:17, and its narinfo answered 404 on both caches, while a small path from the same post step answered 200.

The CLI-level fix — draining the queue before daemon stop returns, or reporting what could not be pushed — is still the better one. This change makes the action correct regardless.

What

  • Daemon mode now records the pre-build store listing, the same way store-scan mode does.
  • After the daemon stops, the post step sweeps the store once with the existing push-paths.sh diff. cachix push skips paths already in the cache (narinfoExists → onAlreadyPresent in Cachix/Client/Push.hs), so this uploads only what the daemon dropped and costs one HEAD per new path when nothing was lost.
  • The daemon stop moves into stopDaemon(), so the sweep also runs when the daemon never started or has no pid — cases that previously ended the push with nothing landed.
  • Store-scan mode and the listing share pushNewPaths() and recordPreBuildPaths(), which removes the duplicated block from the switch.

Verification

pnpm build bundles the patch into dist/index.js. Same harness for both bundles, only the bundle differs: the post step runs with STATE_pushMode=Daemon and no daemon directory, a stub cachix records its calls, the listing before the build holds /nix/store/old, and the listing after it holds /nix/store/old and /nix/store/built.

baseline (v17): FAIL — nothing pushed, cache silently stays empty
  cachix calls: 0
patched: PASS — the sweep pushed the missing path
  cachix push -j8 smoke <<< /nix/store/built

Both runs log ::error::Cachix Daemon not started. Skipping push first; only the patched bundle reaches the sweep. The action's own fixture (nix-build tests) covers the daemon path end to end in CI.

References

The daemon is stopped before its upload queue drains, so a large path can
be logged as pushed and still be missing from the cache. Store-scan mode
already ends with a diff-and-push; do the same in daemon mode once the
daemon is gone, where cachix push skips the paths that are already there.

The daemon stop moves into stopDaemon() so the sweep also runs when the
daemon never started.

@sandydoo sandydoo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is undesirable for several reasons, particularly in cases where the store is persisted between runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants