[Bug][dsh] Rewind (fork+seed) leaves non-monotonic refs: preflight folds only ~5k tokens/round, so 31 rounds (649 s) cannot close a 283k deficit — and the "NOT forwarded" payload is sent anyway #5885

TL;DR

A 129-turn DSH session (2,057,076 tokens by the upstream's own count) was rewound. DSH implements "rewind" as fork + full seed: a new session is created with parentSession + isSeeded: true, every parent event is replayed from seq 0, and a session/end-seed {"inherited": true} marker closes the inherited prefix. When the rewind boundary sits at the tail of the parent, this copies the entire history — not a single token is reclaimed.

On the forked session, every subsequent message hangs in "deep thinking" for 10–11 minutes and then fails. bili sees blocks = 0, parent_session_id = null, messages = 8239, so fork adoption takes the "anonymous fork, no adoption support in v1" branch and starts from scratch.

Three separable defects fall out of the logs:

1. non-monotonic refs after client history rewrite degrades fold granularity to ~5k tokens/round. Every preflight round logs normalized reversed range m09063–m08294 → m08294–m09063, and the viable-range count drops monotonically 161 → 145. 31 rounds only reclaim 143,035 tokens.
2. The 31-round ceiling is arithmetically insufficient but still runs to completion. Deficit was 283,543 tokens (1,076,119 vs 792,576); 31 rounds × ~5k ≈ 155k max. It burns 649,275 ms (10.8 min) and then fails anyway.
3. After fail-fast, the payload is forwarded regardless. The error text says The over-window payload was NOT forwarded, yet the very next request reached DeepSeek and came back with a 400 carrying the real upstream counts. That 400 is the only reason I could calibrate the estimator (×1.1584).

A fourth, smaller defect: compress.modelContextLimit: 450000 was ignored — [headroom] still printed effective window 1048576 -> 792576.

Environment

What DSH's rewind actually does

seq 14830  turn/end {turn:127, reason:{kind:"completed"}}
seq 14831  agent/inbox/spliced          <- user message, never sent
seq 14832  session/end-seed {"inherited":true}
seq 14833  agent/inbox/spliced {"removedCount":1,"outcome":"canceled"}
seq 14834  user/message "先回我一句话，什么话都行，我做连接测试"
seq 14835  turn/start {turn:128}

Session header of the fork:

{"type":"session","version":4,"id":"session-63d92c8a-…",
 "parentSession":"session-ddbeba60-…","isSeeded":true,"delegationDepth":0}

Note seq 0 carries the timestamp of four days earlier — the whole history was re-seeded. The chain is five sessions deep, each created 3–13 s after the previous one, each isSeeded: true, each with blocks = 0:

So every rewind multiplies the number of sessions that must re-fold the same history from zero, and none of them can inherit the parent's 99 blocks.

Observed behaviour

First preflight (deficit 283,543):

round 15/31: payload ~1002238 tok vs window 792576 (deficit 209662), 161 viable range(s)
round 16/31: payload ~997695  tok (deficit 205119), 160 viable range(s)
round 17/31: payload ~991372  tok (deficit 198796), 159 viable range(s)
round 18/31: payload ~986707  tok (deficit 194131), 158 viable range(s)
round 19/31: payload ~982318  tok (deficit 189742), 157 viable range(s)
round 20/31: payload ~977996  tok (deficit 185420), 156 viable range(s)
round 21/31: payload ~973766  tok (deficit 181190), 155 viable range(s)
round 22/31: payload ~969564  tok (deficit 176988), 154 viable range(s)
round 23/31: payload ~965667  tok (deficit 173091), 153 viable range(s)
round 24/31: payload ~961870  tok (deficit 169294), 152 viable range(s)
round 25/31: payload ~957856  tok (deficit 165280), 151 viable range(s)
round 26/31: payload ~954004  tok (deficit 161428), 150 viable range(s)
round 27/31: payload ~950030  tok (deficit 157454), 149 viable range(s)
round 28/31: payload ~945887  tok (deficit 153311), 148 viable range(s)
round 29/31: payload ~941873  tok (deficit 149297), 147 viable range(s)
round 30/31: payload ~938461  tok (deficit 145885), 146 viable range(s)
round 31/31: payload ~935583  tok (deficit 143007), 145 viable range(s)

15:47:03 preflight compressed 31 range(s), ~143035.37 tokens saved
         (1076119.3417761568 → 931914.0431698343) in 649275ms
15:47:04 [error] preflight fail-fast 502 (retryable=true):
         context ~931914.04 tokens (down from ~1076119.34 before preflight)
         exceeds the model window 792576 (model=deepseek-flash)

Second attempt on the same session (deficit smaller because some folds stuck):

16:06:01 preflight compressed 25 range(s), ~80636.31 tokens saved
         (932071.67 → 851163.59) in 691259ms
16:06:01 preflight fail-fast 502: context ~851163.59 exceeds window 792576

By then the viable-range pool had collapsed 161 → 25. The folds are spent; the session can never converge.

Defect 1 — reversed ranges throttle every round to ~5k

Every round emits the same warning, and the count decreases by exactly one each time:

[warn] [preflight] normalized reversed range m09063–m08294 → m08294–m09063
       (non-monotonic refs after client history rewrite, #1001)

This is consistent with #2168 (closed — that one was about acp_status displaying reversed ranges; this is the folding consequence). After a seed replay, refs are re-minted in replay order while the existing block spans still point at the parent's numbering, so range boundaries cross. The normalizer fixes each range individually (start > end → swap) but never merges adjacent fixed ranges, so preflight folds them one message at a time — 1.5k–5k tokens each — instead of one 60k block.

Evidence that granularity, not opportunity, is the limit: contextView showed 100+ compressible ranges, each 1.5k–5k tokens, and the sum of what 31 rounds reclaimed (143k) is almost exactly 31 × 4.6k.

#1001 is cited in the message, so the normalization is deliberate — the bug is that normalization is terminal instead of being followed by a coalesce pass.

Defect 2 — a hard round cap that cannot close the deficit

MAX_PREFLIGHT_ROUNDS bounds the loop, and 31 rounds × ~5k ≈ 155k < the 283k deficit. Nothing in the loop notices that progress rate is hopeless: the per-round reclaim is ~4.6k while the deficit stays above 140k for the entire run.

Related open issues that describe the same terminal state from other entry points: #2438, #5022, #4951. This report adds the fold-granularity mechanism (reversed ranges after seed) rather than chain-miss or unfold explosion.

Defect 3 — NOT forwarded is not what happens

The fail-fast message ends with:

The over-window payload was NOT forwarded.

But the request was forwarded. Seconds later the upstream answered, and that response is the only reason the real counts are known:

[acp-proxy: upstream HTTP 400:
 {"error":{"message":"This model's maximum context length is 1048576 tokens.
  However, you requested 1335502 tokens (1079502 in the messages, 256000 in the completion).
  (request_id: 33871f58-…)","type":"invalid_request_error"}}]

931,914 (bili) → 1,079,502 (upstream) = ×1.1584. An earlier attempt at sentTotal 812,875 produced 1,322,953 in the messages = ×1.63. The panel meanwhile said Sent to LLM (after compression, est.): 685k (65% of limit).

Either the message is wrong and the payload is always sent (so bili can learn the real count from the 400 — it doesn't, see #5370), or fail-fast only suppresses the first send and the retry policy sends the identical body again. Either way the user sees a 10-minute spinner followed by a generic error, and bili never records that it was 15.8–63% off.

Defect 4 — compress.modelContextLimit is silently overridden

With

compress:
  modelContextLimit: 450000
  emergencyThresholdPercent: "70%"
  maxContextLimit: "60%"

every preflight round still compared against window 792576, and [headroom] printed effective window 1048576 -> 792576 (reserved 256000 for max output 256000, cap 0.25). If modelContextLimit is meant to be the preflight hard wall, headroom should be derived from it (450000 − min(256000, 112500) = 337,500), not from the raw model window. The documented key had no effect at all — and it is the only user-facing lever that could have made this session converge without touching internals.

Root-cause hypothesis

1. DSH rewind = buildForkSeed(events, boundary) copying an inclusive prefix. When the boundary is the last event, the copy is total.
2. bili's maybeAdoptForkBlocks cannot adopt: the fork arrives with parent_session_id = null (anonymous) and no derivedFromSessionId metadata, so it logs anonymous fork … no adoption support in v1, starting fresh (#629) and starts with blocks = 0.
3. The seeded history is re-minted with fresh refs, but inherited block spans keep parent numbering → start > end ranges → per-round normalization → 4.6k-token folds.
4. MAX_PREFLIGHT_ROUNDS × 4.6k cannot beat the deficit; the loop is not rate-aware.
5. On failure the payload is (still) sent, gets a 400, and the retry policy re-enters the same path — which is what the user experiences as "hangs forever, then errors".

forkAdoption: true does not help: the adoption path needs incomingCoreMessages to be non-empty, and for an anonymous fork it is falsy.

What would fix it, from a user's seat

1. Coalesce after normalize. When start > end is repaired, merge it with adjacent repaired ranges before folding. One 60k fold per round would have closed 283k in five rounds.
2. Be rate-aware. If reclaimed / rounds cannot close deficit within the remaining rounds, stop early with a typed error and a concrete suggestion ("history is 2.6× the window; start a new session or lower maxTokens"). 649 s of folding followed by a generic failure is worse than an immediate refusal.
3. Do not claim "NOT forwarded" unless that is true. And when the upstream does answer with real counts, persist them (contextTokensSource: "usage" already exists) so the next round budgets against reality instead of estimate.
4. Honour compress.modelContextLimit as the hard wall, including inside headroom.
5. Give seeded forks a lineage. DSH does write parentSession in the header and session/end-seed in the log; if bili keyed adoption off those instead of parent_session_id, a rewind would inherit the parent's blocks instead of re-folding 2M tokens.

Logs

/__bili/plugin/status?conversationId=session-63d92c8a-…

contextView.sentTotal   812875
contextView.limit       1048576
contextTokens           3352465   source: estimate
growth                  -617023
categories              sysprompt 24078 / tool 314982 / text 319798 / code 39685 / summaries 114332
panel                   Sent to LLM (after compression, est.): 813k (78% of limit)
                        Session-only (compressed originals, est.): 711k — pruned from every request
blocks                  99 total, b12/b62 inactive
compressibleRanges      100+ entries, 1.5K–5K tokens each

restoreOutputBudget overriding an explicit DSH config

[sess=session-df4b75cb-…] output budget restored 131072 -> 256000
(#3438: dsh-lane mid-band collapse above the <=2…)

This is arguably correct per #3438, but it means a user cannot lower maxTokens to buy window headroom — the single cheapest escape from an over-window session is silently reverted.

Fold throughput without preflight (the nudge path)

fold seq 63  15:33:20  reclaimed 4022
fold seq 64  15:33:46  reclaimed 4728
fold seq 65  15:34:00  reclaimed 4390
fold seq 66  15:34:21  reclaimed 3686
sentTotal 812,875 → 794,495   (4 folds, 18,380 tokens, ~75 s)

At ~4k per 15–25 s, reclaiming 300k would take 23 minutes of uninterrupted folding — which no client idle timeout survives.

中文摘要

一条 129 轮、205 万 token 的 DSH 会话做了「回退」。DSH 的回退是 fork + 全量 seed：新建会话、parentSession 指向父会话、isSeeded: true、从 seq 0 重放全部事件、session/end-seed {"inherited":true} 收尾。回退点落在父会话末尾时等于完整复制，一条 token 都没减。

之后每条消息都卡在「深度思考中」10–11 分钟再报错。bili 侧 blocks = 0、parent_session_id = null，走的是 anonymous fork 分支，从零重建。

四个可分离的缺陷：

1. ref 乱序把每轮 fold 压到约 5k token。 每轮都打 normalized reversed range m09063–m08294 → m08294–m09063 (non-monotonic refs after client history rewrite, #1001)，可行范围从 161 单调递减到 145。归一化只交换首尾，不与相邻区间合并，所以只能一条消息一条消息地折。
2. 31 轮上限在数学上不够，但仍然跑满。 缺口 283,543，31 轮 × 4.6k ≈ 155k。烧掉 649,275 ms（10.8 分钟）后照样失败。第二次重试可行范围从 161 掉到 25，彻底榨干。
3. fail-fast 说「未转发」，实际转发了。 紧接着上游返回 400，带着真实计数——这也是唯一能校准估算系数的机会：931,914 → 1,079,502（×1.1584），更早一次 812,875 → 1,322,953（×1.63）。面板却一直显示 685k (65% of limit)。
4. compress.modelContextLimit: 450000 被忽略。 每轮仍按 window 792576 比较，[headroom] 打印 effective window 1048576 -> 792576。这是用户唯一能不碰内部实现就让会话收敛的杠杆，却完全无效。

建议：归一化后做区间合并；按回收速率提前判死并给明确错误；「未转发」要么真不转发、要么把上游真实计数学回来（#5370）；modelContextLimit 应真正作为硬墙；seed 出来的 fork 应该能靠 header 里的 parentSession + session/end-seed 继承父会话的块，而不是把 200 万 token 从零折一遍。