fix(vt): never vouch for a version the producer marked non-unique

[kriszyp]

What

The VerificationTable answers FRESH_VERSION_FLAG from version equality alone. That is only sound
while a value's version identifies it — and a producer that stores two distinct values under one
version has no way to tell us, so the consumer holding the superseded copy is the one we confirm.

This adds one flag to the value-header contract the VT already reads, and stops answering FRESH or
publishing a slot for a value that carries it.

Why now

Harper hits this. A resequenced (out-of-order) write keeps Math.max(txnTime, existingVersion), so
the merged record is stored under the version it merged onto: same version, different value. The
worker holding the pre-merge value is told it is current, and an addTo folding onto that stale
value drops the increment the merge had already applied — the lost counter increments
ttl-rate-limiter-concurrent.test.ts "(5) multi-worker stress" has been catching in Harper CI.

Captured on Harper with a detector on the cache-hit path — same key, same version, two values:

[QA431stale] {"id":"it5-mw8","cachedVersion":1786209857971.8462,
              "storeVersion":1786209857971.8462,"cached":{"hits":26},"store":{"hits":27}}

Harper cannot defend itself against this from JS, which is why the fix belongs here. Parking a
sentinel in the slot only suppresses the pre-read fast path; the post-read equality check does not
consult the slot at all. Verified against the pinned build before writing any of this:

post-merge version 1786218151061  count 2   sameVersion true
slot holds sentinel: true                   vouches old V: false
native getSync(expectedVersion=preMergeV) === FRESH_VERSION_FLAG: true

The contract

extractVersionFromValue() already documents the header this reads: an 8-byte big-endian version at
offset 0. A producer that has flags to declare writes a 4-byte big-endian metadata word after it —
top byte VERSION_HEADER_TAG, low 24 bits producer flags. VERSION_NOT_UNIQUE_FLAG (exported as
constants.VERSION_NOT_UNIQUE_FLAG, so producers set the bit this reads) is the one flag this
library interprets.

A value carrying it is never answered FRESH and never published to a slot, on every read path —
sync, async, and both transactional ones. Because publication is refused, the pre-read fast path
needs no change of its own: such a version cannot enter a slot, and the write that made it non-unique
already cleared whatever was there.

The flag is only read for a column family that opted into the verification table, i.e. one whose
producer is already writing versions into these bytes. A value with no header, or one whose word is
not tagged, answers "unique" and keeps its existing behaviour — and a misread in the other direction
costs caching, never correctness.

Latest-value check

FRESH speaks about the consumer's cached copy, not about the value in hand, so a read that may be
behind the latest cannot answer it from what it read: the value that made the version ambiguous can
have been committed after the snapshot, and the consumer may have taken its copy from that newer
value. Every FRESH-answering path therefore consults the latest value when its read may be behind it
— free when the read is provably the latest (no snapshot, or nothing committed since), and on the
branch where it is not free, vtPopulateIfSettled was already paying for the same Get, which the
check now hands its result to instead of repeating.

On the async paths that check runs in the worker, where the database and the caller's column
family are still pinned; the completion only reads a bool. It has to: the completion runs after
teardown may have released both.

Test notes

• test/verification-table.test.ts — a VERSION_NOT_UNIQUE_FLAG block: no FRESH when the version
  matches, no publication (via either POPULATE_VERSION_FLAG or expectedVersion), caching resumes
  once a later write gives the value a version of its own, the flag is read only from a tagged word,
  and the sync / transactional / async / behind-a-snapshot paths each covered.
• test/native/verification_table_test.cc — the predicate itself: tagged-and-marked, tagged-and-not,
  other flags in the same word, the same bit in an untagged word, and both too-short shapes.
• pnpm test 754 passed / 2 skipped, native GoogleTests 104 passed, tsc --noEmit / oxlint /
  oxfmt --check clean.

For the human reviewer

Start at src/binding/core/verification_table.h — the contract and the predicate are the whole idea,
and the predicate is deliberately permissive in one direction only. Then the four decision sites
(database.cpp sync, database.h async, transaction.cpp, transaction_handle.cpp), which are the
same shape each time; the thing to check hardest is that no FRESH-answering path is left ungated, and
that the argument for leaving the pre-read fast path alone holds — it rests entirely on publication
being refused everywhere.

Decisions a reviewer might reasonably question, and why they went this way: the library now parses
one producer flag rather than staying agnostic about value bytes (it already parses the version from
those same bytes, and the alternative — keying the oracle on a per-write token such as the RocksDB
sequence number — is a much wider API change worth doing on its own terms, not as a bug fix); the
tag is 8 bits, so a header-less producer on a VT-enabled column family could in principle collide,
which costs caching and not correctness; and populateVersion() is left alone, since it never sees a
value and by design trusts its caller.

Cross-model review ran three rounds (codex + gemini + harper-domain) and found real defects in the
first two — the snapshot-value hole and then a use-after-free in my first async attempt. Both are
fixed here; the review's remaining open items are the decisions above.

🤖 Generated with Claude Code

_{Human-Review-Need: 4 @ 6f66027}

[gemini-code-assist]

Code Review

This pull request introduces the VERSION_NOT_UNIQUE_FLAG to handle cases where a producer stores multiple distinct values under the same version. It adds verification logic (valueVersionIsNotUnique and vtCheckLatest) to ensure that such non-unique versions are neither marked as FRESH nor published to the verification table slots, maintaining cache correctness even when read snapshots are behind. Unit tests have been added to verify this behavior across sync, async, and transactional paths. There are no review comments, so I have no feedback to provide.

[github-actions]

📊 Benchmark Results

get-sync.bench.ts

getSync() > random keys - small key size (100 records)

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 lmdb	1	24.58K ops/sec	40.69	39.38	599.722	0.117	122,884
🥈 rocksdb	2	10.63K ops/sec	94.05	90.63	31,873.38	1.26	53,162

getSync() > sequential keys - small key size (100 records)

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 lmdb	1	28.60K ops/sec	34.96	33.90	2,098.137	0.129	143,017
🥈 rocksdb	2	11.10K ops/sec	90.09	86.80	523.172	0.045	55,501

ranges.bench.ts

getRange() > small range (100 records, 50 range)

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 lmdb	1	24.27K ops/sec	41.20	35.65	1,974.568	0.287	121,370
🥈 rocksdb	2	15.88K ops/sec	62.96	55.98	1,093.431	0.119	79,417

realistic-load.bench.ts

Realistic write load with workers > write variable records with transaction log

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	377.28 ops/sec	2,650.522	130.818	18,272.948	9.03	755
🥈 lmdb	2	26.32 ops/sec	37,989.025	422.935	1,205,372.566	136.688	64.00

transaction-log.bench.ts

Transaction log > read 100 iterators while write log with 100 byte records

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	39.36K ops/sec	25.41	11.32	13,645.892	0.592	196,806
🥈 lmdb	2	440.91 ops/sec	2,268.041	181.519	8,930.785	1.19	2,205

Transaction log > read one entry from random position from log with 1000 100 byte records

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	744.93K ops/sec	1.34	1.15	3,745.483	0.168	3,724,635
🥈 lmdb	2	461.79K ops/sec	2.17	1.19	8,838.547	0.881	2,308,958

worker-put-sync.bench.ts

putSync() > random keys - small key size (100 records, 10 workers)

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	823.79 ops/sec	1,213.903	1,063.854	1,963.413	0.322	1,648
🥈 lmdb	2	1.15 ops/sec	871,332.749	832,284.799	947,899.528	2.66	10.00

worker-transaction-log.bench.ts

Transaction log with workers > write log with 100 byte records

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	22.53K ops/sec	44.39	29.18	550.676	0.565	45,059
🥈 lmdb	2	839.99 ops/sec	1,190.484	168.719	12,826.078	5.12	1,680

---

Results from commit e7a840c

[cb1kenobi]

Reviewed 6f66027 for the incremental update from f03f044 — no issues found.

The added noBlockCache/Promise assertion covers the async completion path, and the final verifyVersion assertion catches the silent regression where vtVersionSettled never permits async VT publication.

Generated by Barber AI