Skip to content

fix(chat): hide leaked provider citation markers - #626

Merged
su-fen merged 3 commits into
Stack-Cairn:mainfrom
loto585:fix/citation-marker-leak
Aug 25, 2026
Merged

fix(chat): hide leaked provider citation markers#626
su-fen merged 3 commits into
Stack-Cairn:mainfrom
loto585:fix/citation-marker-leak

Conversation

@loto585

@loto585 loto585 commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Linked issue

Closes #625

Summary

部分模型或中转服务会将 ChatGPT 私有联网搜索引用标记直接混入普通文本,例如:

citeturn0search5turn0search15

这些标记属于 provider 内部协议元数据,不应显示在用户回答中。流式响应中该标记还可能被拆分到多个 text_delta,导致仅按单个 chunk 过滤无法清理。

本 PR 在文本流和最终 assistant 消息两个阶段清理 provider citation markers,并增加跨 chunk 状态处理,避免这些标记出现在:

  • 聊天界面;
  • assistant 历史消息;
  • 后续上下文请求;
  • 持久化会话数据。

工具调用、图片等非文本内容不会被修改。

Change scope

  • Modules: agent-gui
  • Key paths:
    • crates/agent-gui/src/lib/providers/runtime/messageUtils.ts
    • crates/agent-gui/src/lib/providers/runtime/textOnlyRuntime.ts
    • crates/agent-gui/src/lib/chat/runner/agentRunner.ts
    • crates/agent-gui/src/lib/providers/llm.ts
    • crates/agent-gui/test/providers/message-utils.test.mjs

Screenshots / preview

这是一个聊天文本渲染行为修复,不涉及布局或视觉样式变化。

Runtime preview:

Before:
Answer content citeturn0search5turn0search15 continues here.

After:
Answer content  continues here.

跨流式 chunk 的场景也会被处理:

Chunk 1: Answer content cit
Chunk 2: eturn0search5turn0search15 continues here.

Rendered:
Answer content  continues here.

Verification

  • Targeted provider and agent tests: 108 tests passed

    • citation marker 完整文本过滤;
    • 跨 delta 拆分的 citation marker 过滤;
    • assistant 非文本内容块保留;
    • agent runner;
    • hosted search;
    • native web search;
    • stream retry。
  • GUI build:

    pnpm --filter liveagent build

    TypeScript 检查和 Vite 构建通过。

  • 分支已同步最新 upstream/main,并确认无合并冲突。

Pre-submit checklist

  • A requirement issue is linked with Closes #N.
  • Synced with the target branch; no merge conflicts.
  • The change is focused, with no unrelated modifications.
  • No secrets, tokens, or personal data included.
  • Docs are updated for changes affecting user behavior, deployment, or configuration.

Runtime preview - citation marker filtered

@StackCairn
StackCairn marked this pull request as draft August 25, 2026 16:22
@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

PR governance checks passed. Awaiting human review.

@loto585
loto585 marked this pull request as ready for review August 25, 2026 16:25
Stream sanitizer and stripProviderCitationMarkers now share one matcher
so lookalike markup stays visible and unterminated markers are not
flushed back on text_end.

Co-authored-by: Cursor <cursoragent@cursor.com>
@su-fen

su-fen commented Aug 25, 2026

Copy link
Copy Markdown
Member

Follow-up pushed: c95c9eeafix(chat): unify provider citation marker matching

stripProviderCitationMarkers and the stream sanitizer now share one matcher (\uE200cite + one or more \uE202 segments + \uE201):

  • Lookalike text without a delimiter is kept on both the stream and the final message (previously the stream path could strip it while replace() would not).
  • Unterminated markers / truncated start prefixes are held while streaming, then dropped on finish() / complete-text strip, so they no longer leak back through text_end.

Added coverage for lookalikes, char-by-char streaming vs strip agreement, and unterminated text_end.

@su-fen
su-fen merged commit 9046a26 into Stack-Cairn:main Aug 25, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(chat): hide leaked provider citation markers from assistant output

2 participants