Skip to content

fix(session): OCR scanned PDFs uploaded as inline chat attachments - #5511

Open
fxia0018 wants to merge 1 commit into
Tencent:mainfrom
fxia0018:fix/embed-scanned-pdf-ocr
Open

fxia0018 wants to merge 1 commit into
Tencent:mainfrom
fxia0018:fix/embed-scanned-pdf-ocr

Conversation

@fxia0018

Copy link
Copy Markdown
Contributor

Description

A scanned PDF uploaded in an embedded chat comes back with no text, while the same file works in the console chat (#5365).

The two entry points parse attachments differently:

  • The console chat uploads a temporary attachment (POST /sessions/:id/attachments). UploadTemporaryDocument stores the agent's VLMModelID, AttachmentImageUnderstanding and AttachmentOCRMaxPages, and the parse worker OCRs text-poor documents with the VLM (temporaryDocumentService.applyImageUnderstanding).
  • The embed widget sends the file inline as attachment_uploads. AttachmentProcessor.processWithDocumentReader only runs the document reader, so a PDF without a text layer yields page images and no text.

This PR applies the same OCR fallback to inline attachments:

  • ProcessAttachment takes an AttachmentProcessOptions struct instead of a bare asrModelID. qa.go fills it from the agent config, using the same conditions as UploadTemporaryDocument (ImageUploadEnabled, VLMModelID, AttachmentImageUnderstanding, AttachmentOCRMaxPages).
  • service.OCRScannedAttachment reuses the temporary-document gate and VLM cascade. It runs only when the parsed text is under 200 runes and page images are available. It uses the scanned-PDF prompt and never captions. The existing understandImagesWithVLM method now delegates to a package function, so its behavior is unchanged.
  • OCR runs before ResolveAndStore, which works on the same image refs. The OCR text is appended to the markdown before line truncation, as in the temporary path.

Agents without AttachmentImageUnderstanding behave as before. Inline attachments are already parsed synchronously, so OCR adds VLM latency to that request. It is bounded by the existing page cap (default 8) and concurrency settings.

Type of Change

  • 🐛 Bug fix
  • 🧪 Test

Related Issue

Fixes #5365

Refs #4312, which also covers embed uploads not being saved as temporary documents. This PR only fixes the missing scanned-document OCR.

Testing

  • go test -race -count=1 ./internal/handler/session/ passes.
  • go test -count=1 ./internal/application/service/ passes except TestSkillPythonVerifier, which fails the same way on unmodified main in my environment (local Python setup).
  • New tests:
    • TestProcessAttachmentScannedPDFUsesVLMOCR and TestProcessAttachmentScannedPDFWithoutOCROptIn drive ProcessAttachment with a reader that returns a scanned PDF (page image only, no text). With the opt-in, the OCR text appears in the attachment content. Without it, the VLM is not called.
    • TestOCRScannedAttachment* cover the page cap, refs without inline bytes, text-rich content, and a missing VLM model.
  • gofmt and git diff --check pass. I didn't run golangci-lint locally. New lines stay within the 120-column lll limit.

Checklist

  • git diff --check origin/main...HEAD passes
  • Changed source files are formatted
  • Targeted tests for the changed packages/components pass
  • Diff-scoped lint passes where applicable (left to CI)
  • Full-repository checks were run, or any unrelated/environment-dependent failures are documented above
  • Self-reviewed the code
  • Added/updated tests covering the change
  • Updated related documentation (not needed)
  • Breaking changes are clearly called out in the description above (none)

Chat attachments sent inline as attachment_uploads, which the embed widget
uses, were parsed by the document reader only. The VLM OCR fallback for
scanned or image-only documents ran only for temporary attachments uploaded
through /sessions/:id/attachments, so the same scanned PDF was readable in the
console chat and empty in an embedded page.

Apply the agent's VLM OCR settings (VLMModelID, AttachmentImageUnderstanding,
AttachmentOCRMaxPages) to inline attachments too, reusing the temporary
document gate: text-poor content with page images is OCR'd and the text is
appended before line truncation.

Refs Tencent#5365, Tencent#4312

Signed-off-by: Jace Xia <fxia0018@student.monash.edu>
Copilot AI balanced review requested due to automatic review settings November 14, 2026 03:43

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: 网页嵌入有时候无法解析图片类型的PDF,但是直接在后台对话就没有这个问题

2 participants