Repository navigation
Description
What this is
The CI lane "real pi package-native lane vs deterministic fake upstream (zero tokens)" (.github/workflows/ci-e2e-native.yml, runs tests/e2e/e2e-native-pi.test.ts) has been failing intermittently on master since ~2026-11-10 19:30Z, with the two #2791 cross-protocol tests:
pi cross-protocol switch keeps the session, conv, and fold (#2791)—compress must be CALLED exactly once across the protocol switch (got 2)(tests/e2e/e2e-native-pi.test.ts:832)pi openai→responses switch keeps the session, conv, and fold (#2791 responses column)—the fold must be REUSED, not re-executed (compressCalls=2)(:916)
Not introduced by any in-flight PR: clean master (58a8796) reproduces the exact same assertion locally (node 25.9 + pi-stable 0.83.6, deterministic), and master's scheduled CI runs (11-10 19:38Z run 42887264198; 11-11 00:35Z run 42932505675) fail the same way. The same SHA goes red and green across CI runs (e.g. 42848363612 green 15:49Z vs 42892982144 red 20:12Z; snapshot-foundation's e17f18c red at 20:34Z, green at 00:27Z), and ~20 other PRs in the same window are green — a timing race, not a tree regression.
Root cause (wire-verified)
The fake upstream (tests/e2e/fake-upstream-chat.mjs) keys its scripted directive queue by convKey|sourceText.slice(0,64) where sourceText is the LAST user message text including its ACP render tag. That tag is volatile BY DESIGN:
- The scripted fold (run 2,
compresson m00001) succeeds → stream.ts setspendingLedgerAlign→ the nextscheduleSaverunsmaybeRestampToLedger(src/persist.ts) →restampRefsToLedger(After model switch / client-side history rewrite, all message identities (content hashes) are replaced: ACP numbers re-m | 切模型/客户端重写历史后,消息身份(内容哈希)全部更换:acp 编号重铸、折叠覆盖大规模重挂、死 ref、churn 区临时膨胀重折 #4316 layer 3, "exactly once per state"). - # Cache hit collapses to 1–12% for a whole generation, then the very next request returns to 100% — repeated 15× in one day #5608-D (owner ruling 2026-11-06) defers that alignment to the session's NEXT SUCCESSFUL FOLD so the renumbering rides the fold's prefix break free — so it lands exactly one request after the scripted compress.
- The alignment (a) renumbers old refs onto store-row seats (the filler's tag went m00001 → m00015, refs-never-reused) and (b) RAISES
timestampsHighWaterRefSeqto the aligned max (kernel/src/refs.ts, # Cache hit collapses to 1–12% for a whole generation, then the very next request returns to 100% — repeated 15× in one day #5608-C: "so none of them suddenly gains a time attribute"), so the stamp node (kernel/src/compress.ts:1229,seq <= hwm → no ts) dropstime=from every pre-fold message's tag. - Wire evidence (local repro, fake reqlog): request [6] sends
<acp tokens="17" type="text" time="2026-11-11 09:06:49">m00011</acp>\n请调用probe_tool;请调用compress;请调用acp_statusas the last user message; request [7] — same conversation, same message — sends<acp tokens="17" type="text">m00011</acp>\n...(time= gone). The 64-char queue-key slice contains the tag → new key → the fake re-parses the directives from the same prompt → fresh queue [probe_tool, compress, acp_status] → request [8] emits a SECOND compress call (no-op: the re-anchored range matches the existing block) →got 2.
The green runs are the race side: when the unified store has no visible rows for the session at the post-fold scheduleSave (persistence timing on loaded runners), maybeRestampToLedger no-ops, no renumber/hwm raise happens, the tag stays byte-stable and the queue survives.
The product behavior (one-time tag rewrite at the first fold) is the accepted cost documented by #5608-C/#5608-D — no product change requested here.
Fix
The harness must not key a conversation turn on volatile tag bytes. In answerFor() the queue key becomes convKey|<source message ref>|<tag-stripped body>.slice(0,64):
- the ref names the message OCCURRENCE — stable across the turn's requests (a mid-turn remint voids the script anyway: the message's bytes are in a fold block), and keeps distinct prompts distinct even when their bodies are byte-identical (the switch/back runs both send
请调用probe_tool); stripAcps(the helper already present for title-call detection) removes the volatile tag wrapper from the keyed bytes.
One change, tests/e2e/fake-upstream-chat.mjs — covers all six suites that share the fake (native pi/dsh/opencode, delegate-notify, subagent-sessions).
Verification
- Both dsh anthropic-messages lane bypasses native-intercept fetch patch: anonymous pfa-* session, fold ledger orphaned, silent ~182s re-fold on cross-protocol switch #2791 tests pass locally (previously deterministic fail): the fold run's queue now continues through the fold —
probe (qIdx 0) → compress (qIdx 1, real fold) → acp_status (qIdx 2) → done, exactly one compress call; switch/back runs keep their own per-run queues (distinct refs). - Full local regression pass over all fake-consuming suites: e2e-native-pi 7/8 (the /acp stderr-assertion failure is a pre-existing local-environment artifact — identical on master, unrelated), e2e-native-dsh 2/2, e2e-native-opencode 4/4, e2e-delegate-notify 2/2, e2e-subagent-sessions 1 failure reproduced identically with the UNMODIFIED fake (pre-existing local artifact, not this change).
Fix lands as a follow-up commit on the PR #6150 branch (the lane must be green for that PR to merge); tracked here for the record.
Activity
test(e2e): key the fake upstream's directive queue on the source ref …