Skip to content

localInputEstimate persists the char-count upper bound (~4x char/token) and poisons every later request; with a small window preflight then reports "exceeds the model window" on a 2.4x-overestimate #5500

New issue
New issue
Open
#5501
Open
localInputEstimate persists the char-count upper bound (~4x char/token) and poisons every later request; with a small window preflight then reports "exceeds the model window" on a 2.4x-overestimate#5500
#5501
Labels
bugSomething isn't workingompp1piscope/wiring马上合并

Description

@jdgg777
jdgg777
opened on Nov 1, 2026

Environment

  • bili / billion-context: 0.1.210 (commit 0e88a07)
  • Host: linux 6.17.13-21-pve x64, single machine, multi-lane (omp + codex + hermes peers)
  • Failure surface: omp agent runtime (ACP lane), model `litellm/step-5-preview-free` routed to `http://127.0.0.1:14001\`
  • Storage: unified sqlite (~/.local/share/billion-context/sessions/index.db, 559.2 MiB)

Symptom (user-visible)

```
Error: Retry budget exhausted after 10 retries: 502 context ~204011.2055699065 tokens exceeds
the model window 150000 (model=step-5-preview-free), with 0 compressible range(s) still visible
and preflight compression could not bring it under: the payload still exceeds the window after
preflight compression. The over-window payload was NOT forwarded. (type=server_error param=preflight_compress_failed)
```

Repeated across multiple sessions / runtimes. Also observed: `Auto-snapcompact cancelled`,
`Auto-shake found nothing eligible to drop; trying the next preferred compaction method`,
`Auto context-full maintenance cancelled`.

Evidence: the estimate is stored characters, not tokens

Query of `sessions.stats_json` for the offending session:

field value
`lastInputTokens` 204046
`localInputEstimate` 204046
`inputTokens` (real usage) 83051
messages / stored bytes 116 / 196442
`blocks` 0

`196,442` is exactly the `Σ m.text.length` of the message set (chars). It matches `localInputEstimate` to the unit.

Source (`src/session.ts`):

```js
session.stats.localInputEstimate = estimateCoreMessagesUpper(processedMessages)

  • countSystemAndToolsTokens(responsesDevContent ?? "", toolsOut)
  • loadedToolTokens + imageReserve;
    ```

where

```js
function estimateCoreMessagesUpper(messages){ let chars=0; for(const m2 of messages) chars+=(m2.text??"").length; return chars }
```

and `defaultCountTokens() = cjkCount + Math.ceil((text.length - cjkCount)/4)`, i.e. tokens = chars/4 for ASCII. So the message term is charged ~4x. The `#553` comment at `src/preflight.ts:265-267` acknowledges this ("undercounts code/JSON replays by up to ~4x") — but that defense only applies when `unknownBaseline` is set for this request. `unknownBaseline` is not persisted in `stats_json` for this session, and `lastInputTokensSource` is unset, so the upper bound was written into `lastInputTokens`/`localInputEstimate` and is replayed as truth afterwards.

Evidence: it poisons future requests, not just this one

`stats_json` is persisted. Because `localInputEstimate` == `lastInputTokens` == 204046 is written to disk, every later request for that session starts from the inflated value; the session can never report a genuine size again unless a new `inputTokens` usage anchor overwrites it.

Fleet-wide count over 2420 sessions with stats: 53 sessions have estimate > 2x real usage. Worst offenders are `pfa-** sessions with estimate `1958197`/`1664643`/`1638612` and `inputTokens == 0`.

Second, independent defect: window resolution misses step-5 models

`CONTEXT_LIMIT_TABLE` in `dist/index.js` contains no `step-` entry (only `claude-`, `gpt-5`, `gpt-4.1`, `gpt-4o`/gpt-4-turbo, `o1`/`o3`, …). Therefore

```js
const native = ... ?? configuredWindow ?? peekWindow ?? lookupContextLimit(model);
```

falls through to `base.modelContextLimit`, whose default is `2e5` (`parseInt(env.ACP_MODEL_CONTEXT_LIMIT ?? `${fileConfig.modelContextLimit ?? 2e5}`, 10)`), and `effectiveWindow = 2e5 * nudge.maxContextLimitPct(.75) = 150000`.

The real window for `step-5-preview-free` (litellm `nvidia_nim/stepfun-ai/step-5-preview`) is 1048576, confirmed by the deployment config (`max_input_tokens` / card comment: "CONTEXT: card says 1,048,576 and the API actually enforces it") and by the models registry (`contextWindow=1048576`).

So a 2.4x overestimate is compared against a window that is ~7x too small, and the two defects compound into a hard, non-retryable `preflight_compress_failed`.

Suggested direction

  1. `unknownBaseline` should gate the char-upper-bound per request only; when the estimate is later written to `stats_json`, it should not be promoted into `lastInputTokens` as if it were a measured usage anchor. Either keep the pessimistic value in a separate field, or scale by the calibration factor `k̂` before persisting.
  2. `CONTEXT_LIMIT_TABLE` should either learn the window from the upstream `/models` response, or fail open (treat `lookupContextLimit() === undefined` as "unknown" rather than clamping to a 200k default) instead of clamping every non-Claude/OpenAI model to 150000.

Local mitigation applied (for the record)

  • Per-route per-model `compress.modelContextLimit = 1048576` in `~/.config/billion-context/billion-context.json` (route keyed by origin `http://127.0.0.1:14001\`), plus `compress.maxContextLimit = "95%"`.
  • omp `compaction.modelThresholds` for step-5 raised from 10% to 40% to compensate for the inflated threshold.

Not a substitute for the fix — but it stops the fleet-wide outage today.

Activity

  1. ranxianglei commented on Nov 1, 2026

    @ranxianglei
    ranxianglei
    on Nov 1, 2026
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: qwen/qwen3.8-27b)

    查重: 不重复。 按失败签名(preflight_compress_failed / localInputEstimate / armFailureShrink)检索了 open+closed issue 与 PR:最接近的是 #4974/#5005(一次 ~10KB 辅助请求把会话基线钉在过小值 → doomed-byte-gate 413 活锁——同属会话计量域,但触发与症状方向相反:那是基线过小、本单是字符上界过大且落盘)和 #4146(单条巨型消息超蒸馏源预算 → 502,机制不同)。本单独立处理。

    标签: bug / omp / scope/wiring / p1(用户不可用类:错误窗口下会话被本地永久拒发、需人工配路由才恢复;幽灵基线还会污染同路由后续所有会话)。

    Triage 结论(已在你报告的 v0.1.210@0e88a073 与当前 master 双树上核对代码):

    1. 复现成立,写入点已定位。 stats_json 里 204046 == Σ m.text.length(196442)+ wire overhead(7604),精确到个位。写入点两处:armFailureShrink(relay.ts)在 relay 5xx/网络失败/熔断时把 outboundPayloadBreakdown().armEstimate——以字符计数上界为地板(fix: arm emergency shrink on outbound payload size, not raw body (#1493) #1498-F1 有意为之,给内核 truncate 深度用)——写进 lastInputTokens 并 markDirty 落盘;localInputEstimate/contextTokens 则是每次发送的常规盖章(fork 会话 lastInputTokens=0 时 preflight 退化为字符估算,大载荷被低估后不压缩直接裸转上游 #553 fail-closed 口径)。你说的「unknownBaseline 只按请求 gate」属实:unknownBaseline = anonymous && lastInputTokens <= 0,显式 id 会话根本进不了字符上界防护分支。
    2. 层次判断(致命一击 vs 放大器): 致命一击是窗口表 miss——CONTEXT_LIMIT_TABLE 无 step-* 条目 → 落到 modelContextLimit 默认 2e5 × maxContextLimitPct(.75)= 150K,而 step-5-preview-free 真实窗口 1M,差 ~7x。字符上界落盘是放大器+持久化器,它做了两件坏事:(a) preflight 入口预算仍按幽灵锚定尺(entryTokens/overshootRatio → summary/round budget 抬升;DSH 分叉会话后触发大量 preflight overflow compress #2383 的 regime 选择已有 entryLocal 防护,但预算没有);(b) 把 lastInputTokens > 0 伪装成「有上游证据」,关掉了 [Bug] 新会话（无 usage 基线）preflight 把字符数当 token——折叠其实成功（2263796→1111212）却 fail-fast 502，上游实际只计费 540828 #3146/413 Deadlock: When an authenticated session exceeds the reservation window with zero collapsibles, the #3146 forwarding | 413 死锁:带身份会话超预留窗口且零可折叠时,#3146 转发门因 anonymous 条件不触发,会话从首条消息起永久不可用 #4185 设计的 forward-once 仲裁出口(该出口要求 lastInputTokens <= 0)——于是错误窗口下永远本地 fail-fast,溢出学习反馈(armOverflowShrink)被饿死,系统无法自愈。两者复合成不可重试死循环;[Bug][preflight] Local 413 is forgotten when the baseline becomes an estimate, reopening oversized probes #3846 的本地拒绝记录再按 origin+model+limit 钉住整个作用域,跨会话传染(与你观察到的多会话/多运行时复发一致)。
    3. 方案评估: 你给的方向打到了点。实现上我按仓库既有哲学「幽灵在读者侧杀,不删 arm」(任何一轮没有拿到 usage 报告的请求，都会把上下文分母放大 12–20×（nudge 读到 >1000% usage、假 EMERGENCY、基线冻结数分钟） #1839/emergency shrink / preflight 的本地估算用错了视图：超长会话（客户端历史 127 万，压缩后 outbound 仅 ~16 万）在上游失败无 usage report 时，bili 按原始历史估算 #1492):估计级(estimate/unprovenanced)锚不再参与 preflight 入口定尺、不再被当作上游证据;arm 本身保留(truncate 救援依赖它)。另补 CONTEXT_LIMIT_TABLE 的 ^step- → 1M 条目(按你部署实测值;表内既有条目惯例,偏大猜错会按 [Question] deepseek-flash 被静态兜底表解析为 128k，真实窗口 1M —— 是有意设置，还是兜底表过期？ #852 设计在首次上游溢出时自愈)。「lookup miss 视为 unknown 而非 2e5 默认」属全局默认值变更,是产品决策——我单独开 decision request 给你拍板,不擅自改。

    计划: 一个 PR(Closes #5500)含三处修复 + 回归测试(复现形状:毒化基线 + 小窗口 + 0 可折叠区间 → 修后走 forward-once 仲裁自愈而非死循环);默认值决策项另开 issue。开始动手,进展会在这里更新。

  2. ranxianglei
    added
    bugSomething isn't working
    omp
    p1
    scope/wiring
    on Nov 1, 2026

  3. ranxianglei commented on Nov 2, 2026

    @ranxianglei
    ranxianglei
    on Nov 2, 2026
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: qwen/qwen3.8-27b)

    已修复,PR: #5501 (Closes #5500)

    做了什么

    另立决策项(需 owner 拍板,非本 PR 范围):lookup miss 该当什么处理 —— decision request(GitHub 编号 5502,新建线程,与本线程镜像号 5502 无关,p1): #5502 — A 保持 2e5 默认+按证据长表(本 PR 走的路)/ B miss 视为 unknown、fail-open 让 overflow arm 学真墙 / C 从上游 /models 目录学窗口。推荐 A 先行、C 后续;B 是全局 fallback 默认变更(§7.4 owner 领地)。

    八问

    1. 是什么问题:两个独立缺陷叠加——① step 系模型窗口表无条目,lookup miss 落到 2e5 默认(~150K 有效窗口),实际放得下的 payload 被判超窗,卡在不可重试的 preflight fail-fast 循环;② arm emergency shrink on upstream failure to break relay-5xx deadlock #604 failure arm 把 char-count 上界(~4x token)以 lastInputTokens/localInputEstimate 持久化,后续每个请求从 stats_json 重放,而 preflight entry 是唯一没做 grade 过滤的读者,中毒值在每次重启后持续抬高 summary budget。
    2. 什么原因导致的:① 表缺 ^step- 行 → 落 base.modelContextLimit env 默认 2e5——lookup miss 被当成"错误数字"而非"未知";② armFailureShrink(src/server/relay.ts:161)把 est = max(textEstimate, charUpper)+overhead 以 source="estimate"(legacy 为无 source)写入并 markDirty 落盘,preflightCompress entry 两处直接读该值、无 grade 检查——与其余读者的 任何一轮没有拿到 usage 报告的请求，都会把上下文分母放大 12–20×（nudge 读到 >1000% usage、假 EMERGENCY、基线冻结数分钟） #1839/emergency shrink / preflight 的本地估算用错了视图：超长会话（客户端历史 127 万，压缩后 outbound 仅 ~16 万）在上游失败无 usage report 时，bili 按原始历史估算 #1492 纪律不一致(逐读者审计后确认这是最后一个未过滤消费点)。
    3. 影响范围与严重程度:p1。① 面:所有用 step 系模型且未显式配窗口的会话(你命中的正是这个;你统计的 2420 会话中 53 个 estimate>2x 实际用量即此类污染存量)——不配 compress.modelContextLimit 绕行则完全不可用;② 面:凡经历过 relay 失败 arm 且 payload 较大的会话——budget 被抬高但方向 fail-closed,属内部计量污染,不直接造成误拒。
    4. 怎么修复的:C1 加表行(field-evidenced 1M,[Question] deepseek-flash 被静态兜底表解析为 128k，真实窗口 1M —— 是有意设置，还是兜底表过期？ #852 教义:偏大的猜会在首次上游 overflow 时自愈,偏小的永远 strand);C2 把既有 per-round floor 的 grade 规则(preflight.ts:2144)原样套到 entry 两个读点——usage/overflow-arm 照旧 floor,estimate-grade/无出处不 floor。不改写入侧:arm emergency shrink on upstream failure to break relay-5xx deadlock #604 arm 本身是 load-bearing 的 truncate rescue,其设计文档明示 ghost 在 reader 侧杀(任何一轮没有拿到 usage 报告的请求，都会把上下文分母放大 12–20×（nudge 读到 >1000% usage、假 EMERGENCY、基线冻结数分钟） #1839),本 PR 与该教义同向。
    5. 是否有回归问题:逐项审过——usage/overflow 锚点行为不变(T2b/T4 双向钉住);unknown-baseline regime 不变;arm emergency shrink on upstream failure to break relay-5xx deadlock #604 kernel truncate rescue 不变;fitting-payload-under-summary-429 仍可 forward(fix: honor anthropic-beta context-1m to size the context window #301/The symptom is that switching models causes a hang, suspected to be triggered by dsh native compression. | 症状是切模型卡住疑似触发 dsh 原生压缩 #4288,新钉);wire 格式/持久化格式/config 表面均不变。另评估并否决了第三个候选改动(把 post-exhaustion 的 noUpstreamEvidence 逃生口扩展到 estimate-grade baseline):可达性分析显示通往该逃生口的每条路径都被 probeForEvidence(billion-context 卡死问题 —— 上报材料 usage获取不到导致上下文估计失误 #2313)/localRefusalHere([Bug][preflight] Local 413 is forgotten when the baseline becomes an estimate, reopening oversized probes #3846)/growthArmed 挡死,加了也是死代码,未进 PR。
    6. 是否破坏缓存:不涉及。前缀缓存接缝、去重键、增量游标未动;k̂ 校准环的输入(lastLocalTextEstimate,prepare 侧以乐观估计器写入)不受影响。
    7. 是否改了设计方向:不涉及。只把既有 "kill-at-reader" grade 纪律补到最后一个漏网消费点 + 一张数据表加一行。
    8. 是否引入新配置:无。

    行为变更披露(old→new):① 未显式配置的 ^step- 模型有效窗口:150000 → ≈786432(同一 headroom 系数作用于 field-verified 1048576;显式 per-route/per-model 配置优先级不变);② ESTIMATE-grade/无出处 baseline 下,preflight entry 不再用 lastInputTokens floor entryTokens/currentTokens 初始化(改按本请求实测定容);usage/overflow-arm 锚点不变。

    合并判定:CI 在此 head 转绿即可 review/merge。

  4. ranxianglei
    added a commit that references this issue on Nov 2, 2026

    fix: pin step-family window row to the field-enforced 1048576 wall (#…

    ac098ee

  5. ranxianglei commented on Nov 2, 2026

    @ranxianglei
    ranxianglei
    on Nov 2, 2026
    Owner

    (ai: vllm-qwen/qwen3.8-27b)

    来源: PR #5501 分析 #5500 时发现

    PR #5501 修复了缺陷 B 的两个 preflight ENTRY 读（currentTokens 初始化 / entryTokens floor），但「除 entry 外该值的所有 reader 都已被 grade 纪律约束」的说法略过强——复查后还有两处读 lastInputTokens 未查 lastInputTokensSource，属同一机制家族（缺 grade 纪律的 reader），记录在此跟踪，不单开 issue：

    1. src/request-membership.ts:70（probeRequestMembership，Bypass requests overwrite the main session's usage anchor, causing long sessions to permanently stop compressing. | 旁路请求覆盖主会话 usage 锚点，导致长会话永久停止压缩 #4805 membership needle 启发式）：referenceScale = lastInputTokens > 0 ? lastInputTokens : needlePayloadUpper —— estimate-grade 中毒臂（char-bound，≈4× token 数）会把 mainScale 阈值抬高约 4×，needle 更新偏保守（软降级，不引发本 issue 的循环）。
    2. src/server/prepare-responses.ts:597（codex compact-intercept 伪造 usage body）：total = Math.max(session.stats.lastInputTokens, est, 1) —— 中毒时伪造的 inputTokens 偏大，codex 下一轮真实请求会从它重算 ledger；窄路径（codex host + intercept mode + compact form）。

    二者都不喂 preflight entry sizing、都不触发本 issue 的 fail-fast 循环，故不属 #5501 范围。后续若修，建议与 #5501 同走 kill-at-reader 纪律（按 source gate），并补双向测试（#3617）。

  6. ranxianglei
    added a commit that references this issue on Nov 2, 2026

    fix: extend grade discipline to the remaining lastInputTokens readers (…

    b263268

  7. ranxianglei commented on Nov 2, 2026

    @ranxianglei
    ranxianglei
    on Nov 2, 2026
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: qwen/qwen3.8-27b)

    复查指出的两处 reader 已独立审核完毕，另发现同类第三处；三处全部修复并折入 PR #5501（新提交 b263268f1，已推送）。本地定向测试全绿；新 head 的 CI 待镜像缓存收编（前一 head ac098eeaa 在镜像缓存为 17 success / 13 skipped / 0 failed）。

    审核结论（独立复核，非照单全收）：

    1. src/request-membership.ts:70（Bypass requests overwrite the main session's usage anchor, causing long sessions to permanently stop compressing. | 旁路请求覆盖主会话 usage 锚点，导致长会话永久停止压缩 #4805 membership needle）——存在，但危害程度被高估。 它的两个调用方都把 stats.localInputEstimate 作为 payloadUpper 传入——而 localInputEstimate 本身就是 char-bound 上界、与中毒臂同一口径——所以比值自归一化，复查所称「mainScale 阈值抬高约 4×」在当前行为下并不会实际发生。gate 的价值在漂移鲁棒性（防未来口径漂移）。照修（kill-at-reader + 回退到 needle 自身 per-request scale），且此点已在 PR body 里如实注明。
    2. src/server/prepare-responses.ts:597（codex compact-intercept 伪造 usage）——属实。 这里 est 是 chars/4 乐观口径而中毒标量是 char 计数口径，尺度不抵消；伪造 inputTokens 偏大约 4×，codex 从该 body 重算 ledger，会偏斜到下一次真实上报为止。已修：仅 usage/overflow-arm grade 的标量参与 max。
    3. 同类第三处（我审出，复查未列）：src/session.ts 的 noteAbortedTurn。 char-bound 臂会污染 In pi, Esc triggers compression injection. | pi中esc会触发压缩注入 #5293 abort re-anchor flag：re-anchor 延迟约 4× 生长量，且 In 0.1.210, sessions with frequent interruptions barely trigger compression reminders — tokens accumulate to 330k before | 中断频繁的会话在 0.1.210 里压缩提醒几乎不触发,积到 33 万 token 才压一次 #5415 的债务记账（recovered = anchor − oldRef，按 char-bound 尺度算）会把幻影增长债务记进 nudge cadence。已修：不可信 meter 直接跳过盖章（其文档自己就有「zero/unknown meter carries no information: skip」，本次把「不可信」纳入同款判断）。
    4. 顺带审计 relay.ts:1565 —— log-only（只拼进 warn 字符串），无需修。

    八问：

    1. 是什么问题：复查发现还有 2 处 reader 缺 lastInputTokensSource grade 纪律；我复核后确认共 3 处（含新发现 1 处），与 localInputEstimate persists the char-count upper bound (~4x char/token) and poisons every later request; with a small window preflight then reports "exceeds the model window" on a 2.4x-overestimate #5500 缺陷 B 同机制家族（estimate-grade 臂未被 grade 校验地读取）——均不触发原 fail-fast 循环，属软降级 / ledger 偏斜 / cadence 失真类。
    2. 什么原因导致的：arm emergency shrink on upstream failure to break relay-5xx deadlock #604/fix: arm emergency shrink on outbound payload size, not raw body (#1493) #1498-F1 failure arm 把 char-bound 尺度的值持久化进 stats_json；代码库大多数 reader 已遵循 任何一轮没有拿到 usage 报告的请求，都会把上下文分母放大 12–20×（nudge 读到 >1000% usage、假 EMERGENCY、基线冻结数分钟） #1839 kill-at-reader 纪律（用前查 source），这 3 处是历史漏网——同一根机制、不同读点。
    3. 影响范围与严重程度：均为窄路径/软降级——① membership needle 更新略保守（当前口径自归一化下实际无行为变化）；② codex host + intercept mode + compact form + ACP 健康态下伪造 usage inputTokens 最大偏大 ~4×；③ meter 只有 estimate-grade 的会话，abort 后 nudge cadence 失真。无用户可见硬故障、无数据丢失。
    4. 怎么修复的：各 reader 按 lastInputTokensSource ∈ {usage, overflow-arm} gate（与 preflight per-round floor 同一纪律）；membership 处不通过时回退 needle 自身 per-request scale。按「一个机制一个 PR」政策折入 [PR] fix: refuse estimate-grade arms in preflight entry sizing; add step-family 1M window (#5500) #5501，不拆 follow-up。
    5. 是否有回归问题：每处补双向测试（Test Blind Spot Retrospective: Reinforcing Negative Assertion Discipline, Tightening Fake Upstream Strictness, Thin Inst | 测试盲区复盘:补反向断言纪律、假上游严格化、薄安装冒烟、形态矩阵与生命周期覆盖 #3617）——T6a/b/c（中毒→bar 回退 needle scale / graded 标量仍优先 / legacy 无 source→回退）、T7+T7-control（走真实 intercept 路径 e2e：中毒→伪造值≈raw body est ~7.2K 而非 204K；usage-grade 12K→恰好 12K，graded 标量保持优先）、T8（re-anchor skip-on-poison + stamp-on-usage，In 0.1.210, sessions with frequent interruptions barely trigger compression reminders — tokens accumulate to 330k before | 中断频繁的会话在 0.1.210 里压缩提醒几乎不触发,积到 33 万 token 才压一次 #5415 债务算术不变）。邻接套件 issue5415/issue4805/codex-compact(+e2e) 51/51、issue5293 2/2（含其实测 pipe e2e，证明 usage-grade 盖章在 gate 后仍正常触发）、typecheck 干净。usage/overflow-arm 路径的计算值不变。
    6. 是否破坏缓存：否。未触碰缓存/重试/增量游标/去重键。
    7. 是否改了设计方向：否。既有 house 纪律（读点 grade）的延伸，无新架构、边界或数据流。
    8. 是否引入新配置：否。无新 env/config/默认值。

    行为变更披露（旧→新）：④ membership bar 参考尺度 = graded 标量（有则用），否则 needle 自身尺度（原为任意正标量优先）——legacy 无 source 会话的 needle 移动可能略不那么保守；⑤ codex forge input_tokens = max(graded 标量, est, 1)（原为任意标量）；⑥ abort re-anchor 只从 usage/overflow-arm meter 盖章（estimate-grade-only 会话退回纯增长式 nudge——那个臂本来就是假信息）。详见 PR body Round 2 节。

    另：分支上的 pin 提交 ac098eeaa（表行 1_000_000→1_048_576，对齐 field-enforced wall）已接受保留。

    结论：PR #5501 head b263268f1 完整覆盖 #5500 的缺陷 A（窗口表）、缺陷 B（preflight entry sizing）与本轮 3 处 reader；CI 在此 head 转绿即可 review/merge。

  8. ranxianglei commented on Nov 2, 2026

    @ranxianglei
    ranxianglei
    on Nov 2, 2026
    Owner

    这个是不配置上下文大小导致的问题吗

  9. ranxianglei commented on Nov 2, 2026

    @ranxianglei
    ranxianglei
    on Nov 2, 2026
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: qwen/qwen3.8-27b)

    一半是，一半不是。

    与「没配置上下文大小」有关的部分（缺陷 A）：你没给这条路由/模型配 modelContextLimit，bili 就走内置回退链（显式配置 → registry → 内置窗口表）。内置表缺 step 系模型的行，一路落到 env 默认 2e5 → 有效窗口约 150K（×0.75），而 step-5-preview-free 的真实窗口是 1M。你本地缓解时显式配了 compress.modelContextLimit = 1048576 就绕开了这一点——恰好印证了这条因果链。但严格说这不是「不配置就会坏」：正确的回退对已知模型就该给出正确值。它分两层：① 表缺行（#5501 已修：补 ^step- → 1048576）；② 表 miss 时该钉死 2e5 还是当「未知」处理——这是产品决策，已单独立了 decision request，等拍板。

    与你的配置无关的独立缺陷（缺陷 B）：relay 出错（litellm 抖动）后，failure arm 把 char-bound 上界（约 4× token 数）写进 stats.lastInputTokens 并持久化到 stats_json，污染之后每个请求（preflight 入站预算等）。注意：如果窗口正确解析成 1M，你报的那个循环根本不会发生——真实 payload 约 83K tokens ≪ 有效 ~850K，preflight 触发条件都不满足。是两个缺陷叠加才变成硬 fail-fast 循环：错误的 150K 墙制造了「超窗」误判，中毒臂又让后续请求一直带着虚高的数字。

    现状：PR #5501 新 head b263268f1 的 CI 已全部转绿（30 checks、0 failed）。升级到包含此修复的版本后，你本地那条显式 modelContextLimit 可以去掉（留着也无害——显式配置永远优先）。

  10. jiu-qiu-yu commented on Nov 4, 2026

    @jiu-qiu-yu
    jiu-qiu-yu
    on Nov 4, 2026

    Additional evidence on 0.1.212, Windows / Pi / Responses path: the same estimate-grade transition can also trigger destructive-to-the-delivered-view emergency tool-output truncation, not only a preflight refusal. No conversation bodies, credentials, session IDs, or private repository content are included here.

    Installed-code path

    Both dist/agent/opencode-native.js and dist/agent/pi-native.js contain:

    function noteOutboundContextEstimates(session, est) {
  session.stats.localInputEstimate = est.upperBound;
  session.stats.contextEstimateTokens = Math.round(est.scaledText + est.imageTokens);
  session.stats.contextEstimateCalibrated = est.calibrated;
  session.stats.contextTokens = est.upperBound;
  session.stats.contextTokensSource = "estimate";
}
function usageAnchorCycleStale(stats) {
  if (stats.lastInputTokensSource !== "usage" || stats.contextTokensSource !== "estimate") return false;
  return (stats.localInputEstimate ?? 0) > 0;
}

    When anchorStale is true, effectiveTokenCount skips the measured usage branches. With no rebuild anchor or anonymous special case, it returns Math.min(session.stats.localInputEstimate, payloadUpperNow) as estimate-grade. estimateCoreMessagesUpper sums JS string lengths. The stale predicate itself does not compare request identity or whether the payload changed.

    Offline reproduction, not a replay of a real request

    Extracted the six installed pure functions and their three constants into a Node VM (no plugin import, network, real session data, or configuration changes). Ran the same clearly synthetic 639673-character fixture through both installed bundles:

    1. Start with lastInputTokens=262850, source usage, and a matching lastUsagePayloadUpper.
    2. Call effectiveTokenCount: {tokens:262850, source:"usage"}.
    3. Call noteOutboundContextEstimates with {upperBound:639673, scaledText:262850, imageTokens:0, calibrated:true}.
    4. Call effectiveTokenCount with the unchanged fixture: {tokens:639673, source:"estimate"}.
    5. lastInputTokens is still 262850; fixture hash is unchanged. Restoring measured-source metadata returns 262850 again.

    Both bundles produce the same result. This proves that the source transition can amplify the decision value without any message growth. It does not prove the exact operands for the real request below.

    Observed runtime metadata

    A measured input usage of 262850 was followed by a request with 1478 messages, 13.0 MiB inbound / 1020.3 KiB outbound, and truncate=0. About 34 seconds later, with no intervening usage sample in the inspected metadata, another prepare with the same message count/inbound size used decision value 639673 and logged truncate=21; outbound fell to 413.7 KiB. Equal count/size is not proof of byte-identical request bodies; those bodies were deliberately not inspected.

    Configured/registered window was 500000, output limit 128000, default output-headroom cap 25%, hence effective input window 375000 and emergency threshold 356250. These are local settings, not an independent measurement of the upstream's limit.

    A separate offline check of installed truncateLargeToolOutputs confirms that this emergency branch does not honor protectedTools; its replacements are subsequently latched. Raising the window or disabling clipping is therefore not a measurement fix, and cannot restore already-clipped deliveries.

    Please include the usage → outbound estimate → repeated prepare → emergency-truncate chain in regression coverage, preserving conservative handling for genuinely changed or unknown payloads. I have not established that the currently open PR #5501 covers this emergency path, and have not installed an unreleased build or changed runtime limits.

  11. ranxianglei
    added a commit that references this issue on Nov 4, 2026

    fix: ride the calibrated per-send estimate on cycle-stale meters (#5500)

    8222bc4

  12. ranxianglei commented on Nov 4, 2026

    @ranxianglei
    ranxianglei
    on Nov 4, 2026
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: qwen/qwen3.8-27b)

    查重：不重复。 同节点近亲 #5217/#5255（latch 在 BLOCKED sentinel ref 下存储、损坏无关工具参数）机制不同；#3691 是 latch 设计来源（已关）。本条是「freshness 降级 × char-bound 标度 × 破坏性消费方无证据门」三层叠加，与 #5500 缺陷 B 同机制家族，已并入 PR #5501（Round 3，commit 8222bc4）。

    标签： 补打 pi（本轮证据来自 Pi/Responses 车道，与原 omp 并存）；bug / scope/wiring / p1 不变。

    独立复核（对 v0.1.212 tag 与 master 双树逐行核对，相关代码两树一致；你的离线提取法与结论均可复现）：

    1. claim 1 完全成立。 usageAnchorCycleStale 为 true 时 effectiveTokenCount（src/server.ts:2869）跳过全部三个 measured 分支，落到 min(localInputEstimate, raw upper)——char-bound 上界直接进入 kernel 计量。现场数字自洽：639673/262850 ≈ 2.43 chars/token，正是 char-count 量级；~34s 后无 usage 报告的第二次 prepare 就是 cycle-stale 触发点。
    2. claim 2 成立且更严重。 truncateLargeToolOutputs（kernel/src/truncate-tools.ts）不仅不认 config.protectedTools，连 ALWAYS_PROTECTED_TOOLS（compress/acp_rule）与折叠路径整套 isMessageProtected/latest-pairing 机制都不用——即使真实 ≥95% 溢出也会裁剪 load-bearing 的 ACP 管理元数据，违反 Kernel Boundary 规则 6（protected-tool hard exclusion）。
    3. latch 确认。 误裁剪写入 state.emergencyTruncateLatch（[Kernel Initiative] Transactional folding boundaries + wire byte fidelity (unified prefix cache fix #3938, kernel side) | [kernel 专项] 折叠边界事务化 + wire 字节保真(前缀缓存统一修复 #3938 内核侧) #3939③），直到下次成功折叠才清除——「raising window/disabling clipping cannot restore already-clipped deliveries」正确，已交付视图的损伤不可逆。

    分层： (a) freshness 降级本身是有意设计（#4383：报告停更时不许骑冻结锚点），不是缺陷；(b) 缺陷在 ride-on 标度选了 char-bound 上界（~2–4× 虚高），把安全会话推进破坏性阈值带；(c) 保护机制缺失是独立的 kernel 不变量违反。

    处置：

    行为变更披露（旧→新）：无报告周期内 kernel 计量（nudge 档位 / 应急截断带 / terminal escape / absorb view）从 min(localInputEstimate, raw)（≈2–4× 虚高）→ min(contextEstimateTokens, raw)（≈真实大小）。三个 #4383/#5134 回归断言随之从 char-bound 重钉到 calibrated 标度（同一 chokepoint 新鲜度，修正标度；其中一条的断言文案本来就写着 "its calibrated estimate wins"——现在名副其实）。

    回归覆盖（你要求的 usage → outbound estimate → repeated prepare → emergency-truncate 链已入 C7 e2e，含 teeth 验证：回退修复后该链复现 latch 损伤，修复后干净）+ C6a/b 双向钉（#3617）+ 四个不变标度对照 + C7-control（kernel 带本身未被动过）。本地定向全绿：issue5500 15/15、effectiveTokenCount 消费方批量 50/50、邻域批量 96/96、typecheck 干净；全量以 CI 为准。

    八问

    1. 是什么问题：上一轮请求没等到 usage 报告时，kernel 计量改骑 char-bound 上界（~2–4× 虚高），越过 0.95× 窗口的应急截断带，把安全会话的工具输出裁掉并 latch 进交付视图。
    2. 什么原因导致的：三层叠加——freshness 降级（有意，[Bug] statusInputBaseline falls back to lastUsageGradeTokens instead of the calibrated local estimate — so one 4.7k side-request sample blocks folding for 4 hours | #1595 承诺的"改用本地估算"兜底在 statusInputBaseline 里没有实现，一次 4.7k 侧请求样本即可挡住折叠 4 小时 #4383）× ride-on 标度选错（char-bound 而非 billing-caliber）× 破坏性消费方对证据无分级（tokenCount 不带 source）。
    3. 影响范围与严重程度：任何上游偶发不报 usage 的会话（abort/cancel/flaky）下一轮 prepare 即可能误裁，Pi/dsh/opencode 全车道同路径；已裁内容不可恢复 → p1 数据丢失类。
    4. 怎么修复的：host 侧 C6 换 ride-on 标度（calibrated 每发估算 + min 封顶）；kernel 侧两个子缺陷走 kernel authorization request: emergency-truncate acts on unprovenanced evidence and ignores protected tools (#5500 family) #5693 等 owner 拍板。
    5. 是否有回归问题：审过 effectiveTokenCount 全部消费方与 12 个相关回归文件（50/50）+ 邻域 96/96；usage/overflow-arm/fresh-settle/[Bug] 上游从不报告 usage 时 nudge 永不触发——上下文无界增长直到超窗（ChatGPT 登录态后端） #728/host preflight 触发器均字节级不变；唯一有意语义变更即上面披露的计量标度。
    6. 是否破坏缓存：不涉及——无缓存键/重试策略/增量游标变化；k̂ 校准环读的是 settle 时的真实配对，不被此路径污染。
    7. 是否改了设计方向：不涉及——沿用既有 house 纪律（staleSmallAnchorFallback 先例 + kill-at-reader），只是把 cycle-stale 对齐到同级降级类的同一读数。
    8. 是否引入新配置：无。

    判定：host 侧修复在 PR #5501 head 8222bc415，CI 转绿即可 review/merge；kernel 侧残余由 #5693 单独拍板，不阻塞本 PR。

  13. ranxianglei
    added
    pi
    on Nov 4, 2026

  14. ranxianglei commented on Nov 5, 2026

    @ranxianglei
    ranxianglei
    on Nov 5, 2026
    Owner

    是什么问题：上一轮请求没等到 usage 报告时，kernel 计量改骑 char-bound 上界（~2–4× 虚高），越过 0.95× 窗口的应急截断带，把安全会话的工具输出裁掉并 latch 进交付视图。

    这个问题master还存在吗

  15. ranxianglei
    added
    马上合并
    on Nov 5, 2026

  16. ranxianglei commented on Nov 5, 2026

    @ranxianglei
    ranxianglei
    on Nov 5, 2026
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: qwen/qwen3.8-27b)

    还在。 刚对当前 master HEAD（8de4c37f8）重新核过：自上次审计的 c9ba48566 以来，相关文件（src/server.ts、src/session.ts、kernel/src/truncate-tools.ts、kernel/src/compress.ts）零变更——cycle-stale 路径逐行一致（anchorStale 跳过全部三个 measured 分支后落到 min(localInputEstimate, raw)，server.ts:2883→2969），char-bound 虚高 + 越过 0.95× 应急截断带 + latch 进交付视图的整条链在 master 上原样存在。

    修复只在 PR #5501（head 8222bc415，C6），未合入；kernel 侧两个残余子缺陷（证据等级门 / truncate 无视保护工具）在 #5693 等拍板。master 何时消除该问题 = #5501 何时 merge。

  17. ranxianglei
    added 8 commits that reference this issue on Nov 11, 2026

    fix: refuse estimate-grade arms in preflight entry sizing; add step-f…

    d7fe16d

    fix: pin step-family window row to the field-enforced 1048576 wall (#…

    5fbc63a

    fix: extend grade discipline to the remaining lastInputTokens readers (…

    22fe21c

    fix: ride the calibrated per-send estimate on cycle-stale meters (#5500)

    8dede7d

    fix: refuse estimate-grade arms in preflight entry sizing; add step-f…

    fac9fcf

    fix: pin step-family window row to the field-enforced 1048576 wall (#…

    13e710e

    fix: extend grade discipline to the remaining lastInputTokens readers (…

    c515ea3

    fix: ride the calibrated per-send estimate on cycle-stale meters (#5500)

    56a7fe2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingompp1piscope/wiring马上合并

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions