Repository navigation
Claude Code: partial streaming snapshots are summed, not deduplicated, inflating input and cache tokens on subagent transcripts #220
Description
Summary
merge_message_into sums every token field whenever a record shares (requestId, message.id) with an earlier one and carries a usage fingerprint that has not been seen before (src/analyzers/claude_code.rs:841-849).
A streaming assistant message is written as several JSONL records under one key, and the earlier ones carry a partial usage snapshot. Between those snapshots only output_tokens changes: input_tokens, cache_creation_input_tokens and cache_read_input_tokens are byte-identical on every record of the group. Each partial snapshot is therefore a new fingerprint, and summing adds those identical input and cache values once more per snapshot.
The result is the opposite of the usual dedup bug. Output is nearly right, because the partial snapshots carry almost no output. Input and cache are inflated by roughly 50 to 60 percent on subagent transcripts. The same branch also accumulates cost (claude_code.rs:868), so the inflation reaches the dollar figures.
Where
Checked at 9de0187.
- key:
local_hash = fast_hash("{request_id}_{message_id}"),src/analyzers/claude_code.rs:746 - fingerprint: the five token fields,
src/analyzers/claude_code.rs:258-264 - rule: a seen fingerprint leaves tokens untouched, a new fingerprint sums all of them,
src/analyzers/claude_code.rs:824-849 - cost:
dst.stats.cost += src.stats.cost,src/analyzers/claude_code.rs:868 - both dedup entry points are on the live path,
src/analyzers/claude_code.rs:134,:158,:169 - the behaviour is pinned by
test_deduplicate_messages_merges_same_local_hash_across_uuids,src/analyzers/tests/claude_code.rs:83-113
Why it surfaces now
On main transcripts the usage block is repeated identically on every record of a group, so the fingerprint never changes and the sum never fires. Before #207 lifted the depth cap, subagent transcripts were not discovered at all, so the only path where this rule misfires was unreachable.
On my corpus, main transcripts have 2,310 multi-record groups and zero groups with more than one distinct fingerprint. Subagent transcripts have 279 multi-record groups and 229 with more than one. Fixing the discovery turned the rule on.
That is also why no existing fixture can catch it: a main-transcript fixture cannot produce a second fingerprint.
What the records look like
One real group from my corpus, 3 records under one key, 2 distinct fingerprints:
(input 2, output 4, cache_creation 53554, cache_read 0) partial snapshot
(input 2, output 248, cache_creation 53554, cache_read 0) finished record
What splitrail keeps: input 4, output 252, cache_creation 107,108.
What was actually billed: input 2, output 248, cache_creation 53,554.
cache_creation is doubled. output is off by 4.
Measurement
199 transcript files, 8,467 records carrying both ids and a usage block, 3,208 distinct (requestId, message.id) groups, of which 2,822 main and 386 subagent. Windows, ~/.claude/projects.
One caveat on rerunning: the transcript directory is live. Main-transcript counts drift upward between runs on a machine that is being used, and mine moved from 2,822 to 2,841 groups while I was writing this. The subagent figures below were stable across runs because that traffic was idle. The ratios, not the absolute counts, are the part that transfers.
Subagent transcripts:
| field | splitrail (sum over distinct fingerprints) | whole record with max output | ratio |
|---|---|---|---|
| input | 2,504,430 | 1,538,218 | 1.63x |
| output | 274,765 | 273,534 | 1.004x |
| cache_creation | 5,414,460 | 3,688,315 | 1.47x |
| cache_read | 24,790,189 | 15,370,427 | 1.61x |
Main transcripts: 1.0000x on all four fields, exact.
Cost
Priced with splitrail's own rates from src/models.rs, standard tier, at the measurement date. Rates used, dollars per million tokens as input / output / cache write / cache read:
| model | rates | source | splitrail | whole-record max | difference |
|---|---|---|---|---|---|
claude-fable-5 |
10 / 50 / 12.5 / 1.0 | models.rs:997-1007 |
$635.82 | $601.95 | +$33.87 |
claude-opus-4-8 |
5 / 25 / 6.25 / 0.5 | models.rs:1033-1043 |
$56.85 | $55.18 | +$1.66 |
claude-sonnet-5 |
2 / 10 / 2.5 / 0.2 | models.rs:1020-1031 (dated entry, applies before 2026-09-01) |
$7.23 | $6.73 | +$0.50 |
claude-haiku-4-5-20251001 |
1 / 5 / 1.25 / 0.10 | models.rs:1177-1187 |
$0.13 | $0.07 | +$0.06 |
| total | $700.02 | $663.93 | +$36.09, +5.44% |
That is on a corpus where subagent groups are 386 out of 3,208. For anyone whose subagent traffic is closer to half the messages, the per-class ratios above are the figure that applies, not this one.
Opus 5 traffic is absent from that table because claude-opus-5 has no entry in src/models.rs and prices to $0 (models.rs:2427-2431). That is a separate matter from this one and I will file it on its own rather than mixing the two, but it is worth stating here so the total above is not read as covering everything on the machine.
One question before proposing a patch
The summing is deliberate and pinned by a test, so it presumably serves a case I have not identified: two records under one (requestId, message.id) that really are two separately billable chunks. If that case exists, the fix is not "replace sum with max", it is "distinguish that case from a streaming partial", and I would rather know which shape to look for than guess.
If it does not exist, keeping the whole record with the maximum output_tokens is what ccusage (should_replace_deduped_entry) and Claude-Code-Usage-Monitor (_replaces_existing, Maciek-roboblog/Claude-Code-Usage-Monitor#237) settled on independently. Two notes on that shape: a per-field maximum would assemble a record that was never written, and since cost derives from those fields the resulting cost would correspond to no observation; and the maximum rather than the last record removes a dependency on traversal order, which is not defined across files.
Happy to open a PR with the change plus a subagent-shaped fixture, once the intent above is clear.
Repro
Self-contained, no dependencies, reads only ~/.claude/projects:
import json
from collections import defaultdict
from pathlib import Path
PROJECTS = Path.home() / ".claude" / "projects"
groups = defaultdict(lambda: {"fps": [], "cls": set(), "n": 0})
for path in PROJECTS.rglob("*.jsonl"):
rel = path.relative_to(PROJECTS).parts
cls = "subagents" if "subagents" in rel else ("main" if len(rel) == 2 else None)
if cls is None:
continue
for line in path.open(encoding="utf-8", errors="replace"):
try:
e = json.loads(line)
except Exception:
continue
m = e.get("message")
if not isinstance(m, dict) or m.get("model") == "<synthetic>":
continue
u, rid, mid = m.get("usage"), e.get("requestId"), m.get("id")
if not isinstance(u, dict) or not rid or not mid:
continue
fp = (u.get("input_tokens", 0), u.get("output_tokens", 0),
u.get("cache_creation_input_tokens", 0), u.get("cache_read_input_tokens", 0))
g = groups[(rid, mid)]
g["cls"].add(cls)
g["n"] += 1
if fp not in g["fps"]:
g["fps"].append(fp)
CAMPI = ["input", "output", "cache_creation", "cache_read"]
agg = defaultdict(lambda: {k: 0 for k in ("groups", "multi_rec", "multi_fp", *(f"{w}_{c}" for w in ("sum", "max") for c in CAMPI))})
for g in groups.values():
a = agg[next(iter(g["cls"])) if len(g["cls"]) == 1 else "mixed"]
a["groups"] += 1
a["multi_rec"] += g["n"] > 1
a["multi_fp"] += len(g["fps"]) > 1
for i, c in enumerate(CAMPI):
vals = [fp[i] for fp in g["fps"]]
a[f"sum_{c}"] += sum(vals)
a[f"max_{c}"] += max(vals)
for cls, a in agg.items():
print(f"[{cls}] groups={a['groups']} multi_record={a['multi_rec']} multiple_fingerprints={a['multi_fp']}")
for c in CAMPI:
s, m = a[f"sum_{c}"], a[f"max_{c}"]
print(f" {c:<16} splitrail={s:>14,} whole-record={m:>14,} ratio={s / m if m else 0:.4f}x")sum is what splitrail counts today, max is the whole finished record.
Activity
NickAme03 commentedon Jul 31, 2026
cc @lizhuojunx86, since this sits directly on top of #207: lifting the depth cap is what makes this path reachable, and the corpus split you measured there is the same split that separates the groups where the sum fires from the ones where it cannot.
NickAme03 commentedon Jul 31, 2026
Correction to the cost table, now edited in the body above. I had priced
claude-sonnet-5at 3/15/3.75/0.3, its base entry, when the dated entry atmodels.rs:1020-1031applies:add_dated_pricing!takes avalid_until(macro atmodels.rs:276) and selection filtersusage_date < dated.valid_until(models.rs:2160), so 2/10/2.5/0.2 is in effect until 2026-09-01. That matches Anthropic's published introductory pricing for Sonnet 5 through August 31, 2026, so splitrail's registry is right here and my arithmetic was not.Corrected totals: $700.02 against $663.93, a difference of $36.09 or 5.44%, previously stated as $703.64 / $667.30 / $36.34 / 5.45%. The token measurements and the per-class ratios are unaffected, since those are counted, not priced.
lizhuojunx86 commentedon Jul 31, 2026
@NickAme03 Ran your merge semantics over my subagent corpus — per-file
(message.id, requestId)groups, each distinct usage fingerprint counted once,correct value = the final record:
18,827 groups. Your "output is nearly right" is confirmed at +0.36%, and the
damage lands where you said: close to a billion phantom cache_read tokens on
this one corpus.
One structural fact that simplifies both the fix and the fixture: the fingerprint
count per group never exceeds two — k-distribution {1: 5,622, 2: 13,205}. However
many partial rows a message writes, they collapse to one partial snapshot plus
one final, so the sum fires exactly once per affected group and the inflation on
the identical fields is exactly one extra copy. A fixture with two fingerprints
per message reproduces the full behaviour.
Main transcripts sit at k=1 across the board (the 100.0%-identical split from
the Usage-Monitor thread), so this is subagent-only and reachable exactly since
#209 lifted the depth cap. The fix that made the files visible also armed this.
For the record on my own earlier claim: the 18,548,947 token-exact
reconciliation (3.6.0, the #208 fixture era) was output-only on the pre-#209
main-transcript subset — both properties that keep it clean of this bug. A
cache-field reconciliation against 3.6.1+ would have caught it; that is now
this issue.
On your pricing correction: independently confirmed — my own tracker's
registry carries the same dated split for sonnet-5 (2/10 until 2026-09-01,
then 3/15), encoded from Anthropic's published page in July. Two registries
agreeing on a
valid_untilis about as settled as pricing gets.Happy to rerun against a branch, same as before.
NickAme03 commentedon Jul 31, 2026
Confirmed on the reference: #209 is the PR that lifted the cap, #207 is the issue that reported it. I cited the issue where the change was the right thing to cite. Correcting it here rather than in the body so the thread stays readable in order.
The k distribution is the most useful thing in your numbers and I did not have it. If a group never carries more than two fingerprints, the sum fires exactly once per affected group and the inflation on the identical fields is exactly one extra copy, which makes the failure fully characterized rather than merely bounded. It also settles the fixture question: two fingerprints under one key, one partial and one final, reproduces the whole behaviour with no need to model a longer streaming sequence.
Our
cache_creationfigures diverge and I do not think that needs resolving: +74.8% on yours against +47% on mine, while input, cache_read and output land within about a point of each other. That field's inflation scales with how many of the affected groups actually create cache, which is a property of the workload rather than of the rule. The rule adds one extra copy of whatever the identical fields hold; how much that is per group differs per corpus.On your note about the earlier reconciliation being output-only on the pre-#209 main-transcript subset: that is the same shape as the reason no existing fixture catches this. Both were clean of the bug for the same structural reason rather than by luck, which is worth stating plainly because it is the part that generalizes. A verification that only covers the path where the rule cannot fire will keep passing no matter how wrong the rule is.
Next step, if the shape is agreeable: I will open a PR with the whole-record maximum plus a two-fingerprint subagent fixture, and take you up on the branch rerun. The open question in the body stands and the PR will carry it rather than assume it away. If there really is a case of two separately billable chunks under one
(requestId, message.id), the change should distinguish that case rather than replace the sum outright. Your k ≤ 2 result is good evidence that no such case occurs in practice, but it is not evidence that the test was written without one in mind.lizhuojunx86 commentedon Jul 31, 2026
@NickAme03 Shape agreed, and the rerun offer stands — ping me when the branch
is up.
One input for the open question, from the protocol side rather than the corpus:
Anthropic's streaming delivers usage as cumulative snapshots of a single billed
request —
message_deltacarries running totals for the message and the finalevent is the bill. So on the current protocol, two separately billable chunks
under one
(requestId, message.id)have no emitter; my k ≤ 2 is what that lookslike from the file side. That argues for the max with a comment stating the
assumption, exactly as you propose — if a future writer path ever emits additive
chunks, the comment is where the next reader starts.
fix(claude_code): keep the finished record instead of summing streami…
mike1858 commentedon Jul 31, 2026
@NickAme03 Good catch, merged your PR; thank you also @lizhuojunx86.