fix(storage): restore index counts from snapshot metadata

[RCmerci]

Restore EAVT, AEVT and AVET cardinalities from the metadata associated with their addresses in the same stored root. Previously the root already held these counts, but restoring the PSS indexes discarded them and subsequent metadata counting traversed every index.

Seed counts before replaying the separate transaction tail so successful edits adjust cardinality exactly once. Current-format snapshots without count metadata retain the existing lazy fallback. For older index-order versions, use the count measured by the existing verification pass; rebuilt indexes derive their count from their actual contents. Negative current-format counts are rejected by the PSS API. Positive metadata is authoritative for its root snapshot; this change does not add a full-tree corruption check.

The required PSS restore API is now merged into official main via logseq/persistent-sorted-set-ocaml#2. The three relevant opam pin-depends entries have been restored to the existing #main convention; other dependency declarations are unchanged. A fresh Git resolution on 2026-10-01 12:06 UTC selected df53f5d01c6c77da138f58a5a2da4de181f2adba. Local builds and regressions use the actual library source from that resolved main commit, without modifying the shared opam switch.

Before/after probe: native, macOS arm64, OCaml 5.5.0, Dune 3.24.2, synthetic memory storage; 4096 datoms, 2048 indexed, branching factor 32. The two sides use the same data and baseline revisions (DataScript e6ac32c, PSS 879de3b), with only the paired changes on the after side. Index cardinalities are 4096/4096/2048 throughout.

Operation	Index node visits before / after	Leaf values before / after	Underlying node reads before / after
First count, across all three indexes	333 / 0	10240 / 0	333 / 0
Second count	333 / 0	10240 / 0	0 / 0
First explicit store after independent cold restore	333 / 0	10240 / 0	666 / 333
Second explicit store	333 / 0	10240 / 0	333 / 333

The remaining 333 reads on each store are the unchanged depth traversal. This PR does not remove that cost. The probe counts structure visits, not timing; its observational interface/wrappers are not part of this PR. Unwrapped controls agree with the underlying read counts.

Regression tests cover zero-read repeated counts, successful add/remove, duplicate/absent edits, tail replay and restore_conn, subsequent incremental transactions and re-snapshot, empty indexes, missing metadata, negative metadata, clean legacy indexes with stale counts, and misordered-index rebuilds. Added Melange and js_of_ocaml smoke checks exercise the integration.

Companion dependency PR logseq/persistent-sorted-set-ocaml#2 is merged. This PR now follows PSS main rather than retaining the temporary pre-merge commit pin. DataScript PR #23 remains unmerged.

Local test status:

• New storage count regressions passed, including tail and legacy metadata cases.
• All 34 native test executables ran. 33 passed; the existing test_perf AEVT prefix count wall-time gate failed (parallel full run: 0.2947s; isolated run: 0.1684s; fixed threshold: 0.100s). No storage-count assertion failed. The unmodified baseline (DataScript e6ac32c + PSS 879de3b) fails the same gate: Fatal error: exception Failure("AEVT prefix count overhead: counting an AEVT attr prefix should avoid comparator allocation overhead: elapsed=0.3280s"). This shows the gate failure is present without either patch; the isolated timings are single measurements, not a speedup claim.
• Melange smoke, js_of_ocaml smoke, JS facade, upstream ClojureScript parity, SQLite cross-runtime parity, and both benchmark harness tests passed. Upstream JS/ClojureScript source is fixed at 3f141af97b70e1f14c65eaa119acd822ebece37e.
• PSS complete five-action suite and strict 15/15 upstream test-name coverage passed independently.
• The first combined local vendoring attempt hit a PSS Melange generated-path assumption; the final DataScript run used a library-only local copy identical to the tested PSS commit. This vendoring harness is not committed.
• Optional personal-graph inspection branches skipped because no graph was supplied. No personal graph was used. The standalone CI memory/throughput benchmarks were not run locally.

No test threshold was changed. The dependency follow-up reran native storage, Conn, SQLite-package, js_of_ocaml smoke and Melange smoke against the resolved PSS main source; all passed. Official CI also completed successfully for the new head 4fc92bbecc0b34b989ba2c82cfed53a32d268439: dependency installation, tests, memory benchmark and benchmark comparison all passed. Run: https://github.com/logseq/datascript-ocaml/actions/runs/36859782729. The CI log confirms installation through the #main dependency URI; it does not print a resolved SHA. The separately recorded Git resolution remains df53f5d01c6c77da138f58a5a2da4de181f2adba at final verification. This PR is still open and unmerged.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-01T12:18:26.546428Z	4fc92bb	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.