Skip to content

Compression nudges reference a different ledger than compress tool #6061

Open

Description

@broonz

In an OMP session, automated nudges list compressible ranges up to m00230 (226 messages / 88K tokens), while acp_status and compress resolve another ledger with highest allocated ref m00007. Compress fails with refs above highest, then dead-ref/content-changed and circuit breaker after repeated attempts. The nudges continue emitting the same ineligible range. xd://report_issue also notified. Expected: nudge and tool share the current generation ledger, or suppress nudges when tools cannot anchor those refs. Observed after a generation boundary with ref reset; no raw project content included.

Activity

  1. ranxianglei commented on Nov 9, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    Duplicate screening: same symptom family, not a confirmed duplicate — cross-linked, kept open. I searched open+closed issues and PRs for the "nudge/status ranges vs compress ledger" family:

    Verdict: distinct client (omp) + distinct surface signature (nudge-only staleness) ⇒ this thread stays independent for now; if mechanism work lands on one of the sub-defects above, it folds in per the one-symptom-one-issue policy rather than spawning parallel fixes.

    Labels applied: bug · omp · scope/kernel · p1 — p1 because the core promise (compression) is deadlocked in affected sessions: the breaker stops attempts while the nudge keeps baiting guaranteed failures every turn.

    What I verified in code (master @ 1f311dd):

    1. Within ONE processTurn on ONE session state, advertised nudge ranges cannot disagree with what acp_status/compress resolve — range endpoints are minted exclusively from state.messageRefs.byRaw (kernel/src/recommend.ts:332), and all three surfaces read the same session state. So your observation narrows the producer to three candidates:
      • (a) a nudge decision/text computed before the reset gets re-emitted after it (stale carrier survives the ledger reset);
      • (b) model traffic and tool calls bind to two different session objects (split binding around the model switch — e.g. stamped identity vs prefix-affinity fallback);
      • (c) pipeline ordering: this request's nudge was computed on pre-reset state inside the same request that then resets the ledger for the tools.
    2. Your follow-on "dead-ref / content-changed" failures match the post-archive prune: applyCompactionArchive/pruneRefsToLiveIds (src/session.ts:1853/:1902, [compress] Still present after #156: the LLM also gets stuck on an invalid end ref; #370's same-parameter short-circuit fails to prevent it. | [compress] #156 之后仍存在:大模型同样卡在无效 end ref,#370 的同参数短路挡不住 #3302) tombstones dropped refs into state.deadRefs, so compressing them yields dead-ref rather than unknown — consistent with a reset that went through the compaction-archive path.
    3. The generation-boundary standing note (Switching model mid-conversation restarts the ref ledger, but the rendered transcript still shows the old refs — the model keeps aiming compress at dead ranges until the circuit breaker fires | 切模型后消息编号整体换账本,但渲染历史仍显示旧编号,模型对着死区间反复压缩直至熔断 #4305, src/generation-boundary.ts) latches one-shot on the FIRST request of the new generation — consistent with "observed after a generation boundary with ref reset". One caveat: if more than one request already flowed in the new generation before stale citations became visible, that note never fires at all.

    What I need from you (version-first triage — your report doesn't state a version):

    1. bili version (bili --version or the startup log line) — required before further code-level analysis;
    2. Is this OMP running with the bili omp plugin installed (bili plugin install omp), or pointing directly at the proxy as a plain client? This decides which injection surfaces exist;
    3. A /__bili/status JSON snapshot taken right after repro (I care about attribution, splitSessions, conflicts, snapshotShadow);
    4. Log excerpt around the boundary: the first request after the model switch + the first failing compress call (timestamps/seqs, the exact nudge range text, the exact error receipts);
    5. Which model/provider you switched from→to — some transports bypass the fetch intercept ([Bug] Anthropic requests leak prompt_cache_key when Cowork transport bypasses native fetch interception #2583), which changes how session identity is carried across the switch.

    Meanwhile I'm continuing code-side on the two most likely producers (stale nudge carrier vs split binding on the omp lane) and will come back with a mechanism verdict and fix plan.

  2. ranxianglei commented on Nov 9, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    Progress while waiting on your data — I've finished mapping both surfaces end-to-end in code (master @ 1f311dd), and the mechanism has narrowed to one class. No new comments had landed on this thread, so nothing changed on your side yet.

    Eliminated candidates:

    • (a) stale nudge carrier — there is no cached nudge text anywhere in the injection paths. Nudge, status line, /acp panel and acp_status are ALL recomputed per request from the live state of whatever session that request bound to (src/server/prepare-openai.ts:323/:481/:546, src/acp-status.ts:79-98). A stale carrier cannot survive a reset inside the proxy.
    • (c) pipeline ordering — the only full ledger reset (resetSessionCompression via reconcileNativeCompactionBoundary, src/session.ts:1710/:1791) runs BEFORE the nudge computation in every prepare lane (e.g. src/server/prepare-openai.ts:266 before :323). A nudge therefore can never offer pre-reset refs in the same turn that resets the ledger.

    What remains — one mechanism class: two ledgers means two session objects (or even two bili instances) serving your one OMP conversation across the model switch. Model traffic binds via pck/conversation-header/prefix-affinity (src/server/handle.ts:992-1543); tool calls bind via a separate witness→conversationId→fork ladder (src/plugin.ts:2160-2238) and execute against that session's remembered last-prepare view (src/plugin.ts:2289-2307). Same session object ⇒ provably same ledger, so your divergence is positive evidence of a split. The codebase already documents this exact failure class on the tool path itself (src/plugin.ts:2218-2234): the host's LLM transport bypassing the intercepted fetch, an attribution gap leaving traffic unclaimed, or a conversation id going stale after a host resume — in all three cases model requests arrive under a DIFFERENT identity than the stamped conversation id the tools use. Your symptom set (nudge persistently large, tools persistently small, stable across turns) fits: model traffic keeps re-attaching to the old un-reset session (full history resent every turn keeps its ~230-ref ledger alive), while compress/acp_status keep routing to the other session's fresh 7-ref ledger.

    One clarification to my previous message: the nudge you see is by design in plugin mode — since #451 the proxy injects the compression nudge as a trailing ephemeral user message in BOTH modes ("plugin agents supply the ACP tools but have no nudge channel of their own", src/server/prepare-openai.ts:475-481). The bug is not that OMP sees nudges; it's that the offered ranges cannot be anchored by the tools.

    Decisive data (refined asks):

    1. bili.log greps around the switch (most valuable):
      • [prefix-affinity] lines — did an anonymous session get minted or re-attached right after the model switch?
      • NO MODEL REQUESTS seen for conversation — the tool-path's own warning for this failure class;
      • compare the [sess=<id>] tag on model-request lines vs on [plugin] tool … executed lines — do they differ?
    2. /__bili/status: conflicts, attribution, snapshotShadow. Note: splitSessions only detects |sub:-shaped splits, so an EMPTY value does not rule out a split — the log greps above are the real test.
    3. Did the switch change anything besides the model name — provider, base URL, port, or a host restart/reload? (That covers the two-bili-instances variant, where each instance would show a perfectly normal single ledger.)
    4. bili version — still required before I pin code-level conclusions.

    Once we know which session(s) served which surface, the fix direction is clear and bounded (either enforce one session binding for model traffic + tools of a conversation, or suppress nudge offers the tools cannot anchor). I'm holding off on code changes until the mechanism is confirmed against your version and logs.

  3. ranxianglei commented on Nov 9, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    Closing note for this work session — the full detail lives in my two replies above.

    Where things stand: code-side triage is complete. Within one session object the three surfaces (nudge / acp_status / compress) provably share a single ledger, so your two-ledger observation narrows to exactly one mechanism class — split binding: across the model switch, model traffic and tool calls are resolving to two different sessions (or even two bili instances). The other two candidates are eliminated by direct code evidence: there is no cached nudge text anywhere (every surface recomputes per request), and the only full ledger reset runs before nudge computation in every prepare lane, so a nudge can never offer pre-reset refs in the resetting turn. I also verified master has no existing guard for this shape, and the built-in splitSessions warning only detects |sub:-family splits — which is why the log greps below are the decisive test rather than any single status field.

    State of the thread:

    Still waiting on you (refined asks from my second reply):

    1. bili version (bili --version or startup log line) — required before pinning code-level conclusions;
    2. bili.log greps around the switch: [prefix-affinity] lines, NO MODEL REQUESTS seen for conversation, and whether the [sess=<id>] tag on model-request lines differs from the one on [plugin] tool … executed lines;
    3. /__bili/status: conflicts, attribution, snapshotShadow;
    4. Whether the switch changed anything besides the model name (provider / base URL / port / host restart).

    Once those land I can confirm the split variant and ship a bounded fix — either enforce one session binding for model traffic + tools of a conversation, or suppress nudge offers the tools cannot anchor — as a PR referencing this thread.

  4. Serali-Nerri commented on Nov 11, 2026

    @Serali-Nerri

    Related observation from Pi's code-mode exec tool calls: successive successful compress results appear to forget coverage from the previous call.

    Sanitized sequence (message refs only; no task contents):

    1. Compressing m01148–m01186 and m01187–m01207 reported two stored blocks, b17 and b18, saving about 16K tokens. A separate newer range was explicitly skipped as protected.
    2. The tool then reported m01066–m01147 as uncovered.
    3. Compressing that older range as two blocks succeeded, again reporting block IDs b17 and b18, saving about 53K tokens.
    4. That second result reported m01148–m01223 as uncovered, which includes both ranges reported stored in step 1.

    Expected: successful compression of disjoint older ranges retains the earlier blocks/coverage, and the uncovered-range inventory excludes them. The same two IDs being returned for different ranges is an additional symptom.

    Observed through returned tool outputs only. No log-level root-cause investigation was performed, so I cannot associate this with a particular request lane or reconciliation mechanism. Further recompression was stopped rather than cycling between the ranges.

  5. sorasoras commented on Nov 12, 2026

    @sorasoras
    Contributor

    Confirming this from a ZCode desktop session on 0.1.212 (dist build commit daa082ea), with the full state trace. The mechanism is narrower than "split binding": both surfaces resolve to the same session_id, but the persisted ref ledger stays pinned to the pre-reset generation.

    Symptom

    • acp_status reports the live ledger (highest allocated ref m00314, active blocks carrying small refs).
    • compress answers None of the N requested range(s) resolved — every ref is unknown to this session … [diagnostics: session highest allocated ref=m159141 … session history=586 compression(s), 586 block(s)].
    • 20 consecutive failures, circuit breaker armed, and the nudge kept re-offering exactly the ranges compress rejects.
    • bili.log records every attempt as [acp-proxy: compress FAILED … → 0 blocks.

    State at the time (SQLite, sessions/index.db, one session UUID)

    table content
    refs 2,994 rows, ref_num 134828..159141 — pre-reset namespace only, no small refs
    blocks 586 rows (78 active), start_ref/end_ref in that same large namespace
    messages 23,031 rows, seq 134550..159223
    sessions.metadata_json 52 keys, no nativeCompactionBoundary key at all

    Mechanism

    The host performed a generation reset (refs restart at m00001 after a model switch) but nothing recorded a native-compaction boundary, so reconcileNativeCompactionBoundary() returns early forever:

    const boundary = session.metadata.nativeCompactionBoundary;
    if (!boundary || typeof boundary !== "object" || !(boundary as any).pendingRebase) return false;

    Ingest meanwhile minted the new namespace, so acp_status (recomputed from the live lane) and compress (validated against the persisted messageRefs/refs) diverged inside one session. That is why the nudge's ranges are always rejected while the tool chain reports a different "highest allocated ref".

    Recovery that worked (keeps the session, drops the pre-reset archive)

    1. back up index.db (+ -wal, -shm);
    2. stop the daemon;
    3. merge {"nativeCompactionBoundary":{"at":<ms>,"pendingRebase":true}} into sessions.metadata_json (merge, never replace the blob);
    4. start the daemon → the next turn runs resetSessionCompression + reserveRefsThrough, and the ledger re-anchors to the live refs.

    Caveat worth documenting: patching the DB while the daemon runs is useless — it rewrites metadata_json from memory on the next flush, so the stop/patch/start ordering is mandatory.

    Suggestion

    markNativeCompactionBoundary() is only reachable from an explicit host signal. A reset that is observable purely from the stream — refs restarting below the persisted high-water mark, or a client ref that maps to no raw id while the lane is demonstrably newer — should arm the same boundary automatically. That looks like exactly the gap #5546 ("make ref-ledger generation resets observable at the moment they happen") closes.

  6. sorasoras commented on Nov 12, 2026

    @sorasoras
    Contributor

    Correction to the workaround I posted above — arming pendingRebase alone does not heal an anthropic-lane session. I verified it end-to-end on 0.1.212.

    Sequence just run:

    1. stop daemon (pid 15600);
    2. merge {"nativeCompactionBoundary":{"at":<ms>,"pendingRebase":true}} into sessions.metadata_json — 6,395,441 → 6,395,580 B, all 52 keys preserved;
    3. start daemon, port 8787 up 51 s later.

    After several requests the session is unchanged:

    probe before after
    blocks 586 (78 active) 586 (78 active)
    refs rows 2,994, ref_num 134828..159141 3,004, ref_num 134828..159151
    compress diagnostics highest allocated ref=m159141 … 586 block(s) same

    So reconcileNativeCompactionBoundary() never ran even with pendingRebase set — and note the ref count kept growing in the old namespace, i.e. nothing re-anchored.

    Why

    On the anthropic lane its only call site sits inside the dsh client-side-compaction branch (prepare-anthropic.ts:344, guarded by compactsClientSide + a fold-coverage gap + checkpoint framing). A plain model/upstream switch produces none of those signals, so the armed boundary has no consumer on this lane. The other lanes differ: openai/google reach reconcile from their own dsh/codex branches, and responses calls it unconditionally.

    That is why a two-part change is required (draft PR #6362):

    1. arm the boundary where the detector already proves the ledger is stale (src/generation-boundary.ts);
    2. honour a pending boundary at the head of the anthropic lane, before the dsh branch.

    State repair alone cannot recover a session in this shape — the second half is what makes the armed boundary actionable. For anyone hitting this today the only reliable escape remains starting a fresh conversation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions