Skip to content

[Pi/OpenAI] 0.1.209 leaks render-tag suffix type="text">mNNNNN into assistant text #5192

Open

Description

@xiaobin

Environment

  • billion-context: 0.1.209 (npm)
  • Host: Pi 1.4.0, native Pi extension/plugin lane
  • Wire/model: OpenAI chat-completions; observed with kimi-coding/k3, kimi-coding/k3-256k, and once with zai-coding-cn/glm-5.3-flash
  • OS: macOS
  • Default render-tag behavior (diagnostics.renderNone not enabled)

Symptom

A normal assistant turn is followed by another assistant text block whose persisted content starts with the tail of a canonical ACP render tag, without the opening <acp tokens="..." prefix:

" type="text">m00106
修复完成,编译验证通过。
...

The full injected metadata shape is:

<acp tokens="196" type="text">m00106</acp>

Thus the visible residue is the suffix from the opening tag through the message ref. m00106 is a valid ACP message ref in that session.

In the clearest reproduction, the preceding turn completed normally:

  1. assistant said work would be applied,
  2. two edits succeeded,
  3. targeted compile returned EXIT:0,
  4. a diff check succeeded,
  5. the next final assistant text began with " type="text">m00106.

Pi persisted that value as message.content[].text; it was not merely a TUI rendering artifact. Pi's OpenAI stream parser accumulates upstream delta.content directly into text, so the text had already reached Pi as ordinary content.

Local census

A local scan of Pi assistant text fields found 93 matching type="text">\s*m\d+ occurrences across sessions from 2026-10-27 through 2026-10-30:

  • 90 with kimi-coding/k3 / k3-256k (stop and toolUse turns)
  • 3 with zai-coding-cn/glm-5.3-flash

Spot checks confirmed visible client-facing leaks, not merely examples inside tool output. The scan was limited to persisted host history; raw SSE was not captured.

Expected

ACP render metadata injected into the request history should not reach client-visible assistant text in whole or in part.

Actual

The canonical opening-tag suffix plus ref reaches the client and is persisted as assistant prose:

" type="text">mNNNNN

Because the leading <acp tokens="..." is absent, existing tag-shaped matchers do not recognize the fragment.

Notes and uncertainty

I did not have a raw upstream SSE dump for these turns, so I cannot prove whether the model emitted only the suffix or emitted a fuller tag that an intermediate path partially stripped. What is established is:

  • billion-context injects the canonical <acp tokens="..." type="text">mNNNNN</acp> metadata;
  • the exact suffix of that internal metadata becomes persisted client-visible text;
  • the default output filter does not remove this suffix.

A possible targeted direction is to recognize the exact residue type="text">\s*m\d+ as a high-confidence canonical signature, at least for audit/logging if unconditional removal is considered too risky. Another direction is to avoid making the full XML literal so copyable in the prompt.

Workaround

diagnostics.renderNone: true (or ACP_RENDER_NONE=1) prevents the injection and therefore the observed leak, but disables ref-based compression. It is a workaround, not a fix.

Related

Activity

  1. ranxianglei commented on Oct 30, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    Follow-up from my triage reply above — I implemented the safe portion and opened PR #5216: #5216

    What ships: stripAcpTags (whole-text / prepare-layer normalizer) now strips the opener-less residue signature (type="text">mNNNNN / tokens=N>mNNNNN, optionally + close), and the echo-residue audit predicate logs any occurrence that escapes. Regression-pinned in tests/issue5195-openerless-residue.test.ts. Rebased onto current master; targeted suites green (152 pass incl. the verif replay property judge), tsc clean.

    What does NOT ship (and why): the live streaming filter (createTagEchoFilter, the path your Pi/OpenAI lane actually renders) is deliberately untouched. The residue has no < head to latch onto, so a chunk boundary splitting inside the attr (t|okens="5">mNNNNN) is indistinguishable from prose, and holding partial anchor-words (type/token/to) would delay ordinary prose. A filter-level streaming fix is therefore either nondeterministic (violates chunk-split determinism — worse than the bug) or a prose-latency tradeoff. Both are owner decisions, consistent with your earlier caution on stripping here (#1755). As a side benefit, when Pi resends history containing a leaked past turn, this PR's prepare-layer scrub removes it before it reaches the model, which helps break the echo loop on later turns.

    Durable root cause is upstream of the filter: the tag is a copyable literal the model imitates/mangles. The real fix belongs at the injection/kernel side (your direction "b": make the shape less imitable) — separate, owner-gated.


    结构化八问

    1. 是什么问题:模型回显 ACP 渲染标签时丢了 <name 头,残留 type="text">mNNNNN(可带 </acp>)这种"无头"片段,泄漏进客户端可见的助手文本并被 Pi 持久化。
    2. 什么原因导致的:直接原因是 tag-echo 过滤器对该"无头"形状无覆盖(既有规则都要求 <name 头);根因是渲染标签在 prompt/历史里是可照抄的字面量,模型模仿时偶发截断。
    3. 影响范围与严重程度:Pi/OpenAI 车道、特定模型(kimi k3 系为主)下出现;属体验缺陷(多字符垃圾文本进正文),有绕行(renderNone),非数据丢失 → 非 p1。
    4. 怎么修复的:whole-text/prepare 路径用窄签名(锚定属性 + 紧贴终止符的 ref)识别并剥离该残片,白话保留(死单元周围的空白按 [bug] #361 residual, form 3: turn whose entire text is a render-tag echo strips to an EMPTY message — agent records a marker-only turn, stalls until manual nudge (v0.1.92, post-#365) #644 教义算正文);审计谓词扩展为可记录逃逸件。
    5. 是否有回归问题:定向套件全绿(含 verif 回放判官的流式等价/幂等/单调/干净保全);签名窄,普通散文/真实 HTML/裸 ref 引用/完整良构标签均不受影响。注意:live 流式车道本 PR 未改(见上),故该车道行为不变——这是有意为之,不是遗漏。
    6. 是否破坏缓存:不涉及。改动只在出站历史归一化/审计,不改压缩块、快照、ref 账本或任何缓存键。
    7. 是否改了设计方向:没有转向。只是补齐过滤器对一个新残片形状的覆盖(whole-text)+ 观测面;流式侧的取舍与注入侧根治留待决策。
    8. 是否引入新配置:无新增 env/config/默认值。

    待你拍板的两点(不阻塞本 PR 合入):

    • A. live 流式车道要不要修:选项①维持现状(仅 whole-text+审计,推荐先这样,风险最低);②接受"持有 type/token/to 等词"的流式延迟代价去修;③转注入侧根治(方向 b)。我倾向 ①→③。
    • B. 注入侧根治(让标签更难被照抄)是否单独立项推进。

    合并判定:本 PR 为窄范围 fix:、带回归钉、rebase 到最新 master、CI 待跑绿;按仓库规则走人工合并。

  2. ranxianglei commented on Oct 30, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    标签: bug pi scope/wiring p2 —— 缺陷类、Pi/OpenAI 车道、属协议接线(出站 tag-echo 过滤层,非 kernel)、有绕行(renderNone)且非数据丢失/不可用故 p2。

  3. loyalchiiina commented on Oct 31, 2026

    @loyalchiiina

    +1 — reproduces here on billion-context@0.1.209.

    Environment:

    • billion-context: 0.1.209 (npm latest is also 0.1.209)
    • Host: DeepSeek Harness (dsh) desktop, Web GUI
    • OS: Windows 11

    Symptom: intermittently, an assistant reply is led by the full ACP render envelope as its own line — the whole open tag (carrying tokens=N, type="text"), the message ref, and the closing tag — before the actual text. The token count / ref vary per turn (e.g. ref m00312, tokens="612"); it is not every message but recurs across a session.

    No root-cause addition here — just confirming it is not limited to the Pi/OpenAI lane: the same leak shows up on dsh desktop as well, so the census in #5192 extends beyond a single host/lane. Thanks for tracking it.

  4. ranxianglei commented on Oct 31, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    Thanks for the datapoint. Treating this as a reported reproduction (unverified build) rather than confirmed root cause, but two things stand out versus the original report:

    1. Different lane — dsh desktop, not Pi/OpenAI. That supports the leak being host/lane-independent rather than tied to one client's stream parser.
    2. Different shape — you describe the full envelope (<acp tokens=N type="text">mNNNNN</acp>) leading the reply, whereas Master CI has been massively red since #5169 was merged: a fixed set of tests (#3575/#4323/#5103/#5140) fails in every r | master CI 自 #5169 合入起大面积红:固定一组测试(#3575/#4323/#5103/#5140)在每个 run 里挂,所有 PR 的合并信号不可用 #5195's census was the opener-less suffix ( type="text">mNNNNN, head dropped). Those are distinct facets of the same render-tag-leak family.

    Worth flagging for accuracy: a well-formed full envelope is normally handled by the existing paired-tag strip path (both whole-text and streaming), so if it is genuinely surfacing on dsh that points at a separate facet — e.g. a response path/version the filter doesn't cover — and would need verification against your exact build before attribution (the shape alone isn't enough to pin the mechanism).

    Tracking notes:

    If it helps pin the dsh facet, a redacted copy of one leaked line (just the envelope + first few chars of the real text) is enough; nothing else needed.

  5. ranxianglei commented on Nov 1, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    子缺陷清单更新:第 3 个子形状「bare ref head」(由 #5259 报告,dsh/OpenAI 车道,m01464b 式裸引用行——无标签前缀、无属性文本,既有签名谓词无子串可匹配)已在 PR #5216 修复(新 head 8ec002396,commit 521f86874):流式 createBareRefHeadFilter(每条文本通道链最外层)+ 整文 stripAssistantProse;双向测试行 + 全切分点确定性属性测试,本地定向套件 159/159。PR 描述已补该形状的行为变更披露(旧值→新值+原因)。当前状态:opener-less residue ✅(本 PR)、bare ref head ✅(本 PR)、drift-name leak → #5178 在途、注入侧根治 → owner-gated out-of-scope。等 CI + 人工审阅后合入即收口。

  6. Punchact commented on Nov 4, 2026

    @Punchact

    补充一条 0.1.212 + DSH plugin mode 的数据点:正文里出现的是完整 canonical 信封(不是残片),而 tag-echo 过滤器全程 0 次触发。

    环境

    • billion-context 0.1.212(本机日志 [v=0.1.212];同时段 npm 检查为 latest,无更新)
    • DSH desktop,plugin mode(dsh.profile.bundles),Windows 11 x64
    • 模型 deepseek-v4.1-flash(DSH 自带 adapter);renderTags=text-only,未开 diagnostics.renderNone
    • 会话约 180K 输入上下文,尚未产生任何压缩块(acp_status:0 blocks)

    现象

    assistant 正文首行出现完整信封(本会话两处,间隔约 15 分钟):

    <acp tokens="1.1K" type="text" time="2026-11-04 22:39:18">m00101</acp>
    
    • 带 time= 属性(本机 0.1.212 的渲染格式)
    • 已被宿主持久化:<DATA_DIR>/session_projcache/sessions/session-<id>.json 的 response 字段以该行开头;同文件 lastUser 也带同一行(全文 3 处命中)——不是 TUI 渲染问题

    日志对照(同一会话、同时段)

    tag-echo lines: 0
    stripped lines: 0
    processTurn: 139 msgs, renderTags=text-only, 26 text tagged, 6 tool tagged (should be 0 with text-only)
    

    即:不是剥离后留下残片(#5071 那类),而是完全没被剥离;createTagEchoFilter 在该时段没有 warn。

    与现有 issue 的差异

    想确认

    1. 这属于 [Bug] Drifted open-tag names (acpt/acct/accent…) and mid-text draft envelopes reach the client verbatim — two thinking-leak shapes #5175 的枚举缺口,还是需要单独覆盖「canonical 完整信封」?
    2. time= 属性是否落在识别器覆盖之外?
    3. DSH plugin mode 下,正文首行的完整信封是否会走 stream filter 的 fast path 而被放过?

    复现与材料

    • 条件:长会话(140+ msgs)、renderTags=text-only、DSH plugin mode;偶发,本会话 2 次
    • 未开 diagnostics.dumpReq,没有原始 SSE;需要的话我可以开 debug 复现一版并附脱敏日志
    • 本地路径与会话 UUID 已脱敏;未修改任何上游代码
  7. ranxianglei commented on Nov 4, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    Useful datapoint — a few grounded notes before attributing it:

    On the time= attribute. The canonical kernel render tag is emitted by acpTag() (kernel/src/render-refs.ts) as exactly <acp tokens="N" type="T">ref</acp> — there is no time= attribute in the current render format. So an envelope carrying time= is not a verbatim kernel tag. Two readings: (a) a model-implemented imitation (the model invents/mangles a copyable literal — the same root cause behind the suffix and name-drift facets), or (b) your build renders differently than current master. Either way it's a signal, not yet a confirmed mechanism.

    Your three questions, as far as I can ground them:

    1. It's not literally the drifted-name enumeration gap (that's the acpt/acct family); a canonical-name full envelope is a distinct shape/facet — but same family, so I'm keeping it under this thread rather than splitting it out.
    2. time= is not an attr-recognition gap for a full envelope: the paired-tag matcher accepts arbitrary attributes (<name …any-attrs…>body</name>), so a complete envelope should strip regardless of extra attrs. That points away from "identifier misses time=" and toward "the filter didn't run on this path."
    3. Your fast-path/bypass hypothesis is the most likely mechanism and the one worth confirming. On the lanes I've traced the per-field stream filter is applied unconditionally, but I have not confirmed the DSH-plugin-mode assistant-text path against your exact build — and given the time= detail, your build appears to diverge from current master, so I'd rather confirm than guess.

    Efficient next step (you offered it): enable diagnostics.dumpReq or capture raw SSE on one repro and share a redacted slice — the leaked first line, the surrounding SSE deltas for that turn, and the [v=…] startup line. That tells us definitively whether the envelope reaches the filter input at all (→ wiring/bypass) or arrives in a form the matchers miss (→ matcher gap), and pins the version. Nothing sensitive required.

    Holding it under this thread as another facet of the same leak family until the mechanism is confirmed.

  8. tax-mux commented on Nov 7, 2026

    @tax-mux
    Contributor

    One concrete mechanism for this shape, with raw-SSE and log evidence: #5848 — on the OpenAI plugin lane a text-less, non-terminal frame (delta:{}, e.g. MTPLX progress frames every ~15 tokens) triggers flushTails(), which drops the held <acp tokens="3 head alone, so the rest (1" type="text" …>m00039) arrives as prose. Fix: #5849.

    Related, separate shape (unclosed opening + ref + answer: short answers dropped at EOF, longer ones keep the ref): proposal in #5850, waiting for an owner decision before a PR.

  9. ranxianglei commented on Nov 7, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    Thanks — I checked the mechanism against the tree rather than taking it on faith, and the core claim holds up at the code level:

    The vector exists. In the DSH plugin lane (src/plugin.ts), a text-less frame releases held filter tails as prose — the OpenAI path has if (!hadText && anyPending()) return flushTails() + rawEvent… (~line 3183), with analogues for the other block types (~3211/~3239) and a second lane (~3482). So a non-terminal, content-empty frame (e.g. a progress delta:{}) does reach flushTails() and would emit whatever partial tag/ref the filter was holding. That's a real, coherent mechanism — and it's a different axis than the one I'd punted on: I deferred the streaming matcher over mid-attribute chunk-split ambiguity, but this is a flush-timing question (when held tails are released), which may be cleaner to fix without the prose-latency tradeoff. Worth treating seriously.

    What I could not verify: I don't have your raw SSE (it's behind the referenced thread), so I've confirmed the code path exists but not that it's the exact producer of your bytes. A redacted slice — the progress frames around one leaked line — would let me build a minimal local repro to pin it.

    On the referenced threads: per the platform's numbering rules I'm not resolving bare cross-thread numbers against GitHub from here, and I can't confirm the status of the three you cite — so please confirm they're real and where they stand (reference info only; I'm not attributing any of it to the owner, and none of it counts as a decision). Two coordination notes:

    • This task's own PR ([PR] [Pi/OpenAI] 0.1.209 leaks render-tag suffix type="text">mNNNNN into assistant text (#5195) #5216) already ships the whole-text/audit fix for the opener-less suffix and a bare-ref-head facet — so if any of your cited threads target the same shapes, we should reconcile rather than stack parallel fixes.
    • The premature-flush mechanism you describe maps directly onto the live-streaming-lane decision I put to the owner in this thread (keep-as-is / prose-latency hold / injection-side root fix). If confirmed, it shifts that calculus toward "fixable at the flush site," which I'd fold into the same decision rather than open a separate one.

    Holding it under this thread as another facet of the same leak family until the mechanism is confirmed against actual SSE.

  10. ranxianglei commented on Nov 7, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    Status for this task — no owner decision has been made yet; everything below is pending.

    Shipped (open PR #5216): whole-text/prepare-path strip of the opener-less render-tag residue + extended echo-residue audit, regression-pinned. Through review rounds and a related follow-up the PR has grown to also cover a bare-ref-head facet, and is currently behind master awaiting sync + CI.

    Corrected this pass: the PR had referenced the wrong issue number in its GitHub-facing text (it pointed at a different, closed issue). I patched the PR body so it now targets this issue correctly (Closes …) — necessary so the auto-close lands on the right issue at merge.

    Datapoints triaged (unverified reports, answered with code-grounded findings only, nothing implemented on their basis):

    • Full-envelope leak reported on dsh desktop — logged as a distinct facet of the same family.
    • A time=-bearing envelope report — grounded finding: the kernel's canonical tag is `` (no time= attribute), so such an envelope is a model imitation or a build divergence, not a verbatim kernel tag.
    • A flush-timing claim (a text-less non-terminal frame releases held filter tails as prose) — confirmed the code path exists in the DSH plugin lane; held pending actual SSE before treating it as the confirmed producer.

    Waiting on owner: (A) how to handle the live streaming lane — keep-as-is vs prose-latency hold vs injection-side root fix (the flush-timing datapoint adds a "fixable at the flush site" option); (B) whether to file the injection-side root cause (make render tags less imitable) as separate gated work. Full-suite verdict = remote CI on the PR head.

    Keeping all facets under this issue rather than splitting them out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions