Skip to content

fix(security): bound quadratic array-stream decoding in is_pdf_too_complex (SEC-146) - #4437

Merged
aadland6 merged 8 commits into
mainfrom
matthew/sec-146-quadratic-memorycpu-in-is_pdf_too_complex-array-stream
Aug 15, 2026
Merged

fix(security): bound quadratic array-stream decoding in is_pdf_too_complex (SEC-146)#4437
aadland6 merged 8 commits into
mainfrom
matthew/sec-146-quadratic-memorycpu-in-is_pdf_too_complex-array-stream

Conversation

@aadland6

@aadland6 aadland6 commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes SEC-146. is_pdf_too_complex() in unstructured/partition/pdf.py re-implemented pypdf's array-based /Contents decoding using the same quadratic raw_data += obj.get_data() accumulation that pypdf patched under CVE-2026-33123 / GHSA-qpxp-75px-xjcp ("Inefficient decoding of array-based streams"). A crafted PDF whose page /Contents is an array of many small stream objects could force excessive CPU/memory. This function runs on every partitioned PDF, so it sits on the untrusted-input path (pdf.py:309).

Per the ticket, this keeps the intentional lightweight raw-bytes approach (added in #4268 to cheaply detect vector-heavy CAD/engineering PDFs) — it does not switch to pypdf's expensive ContentStream. It just makes the accumulation efficient and bounded.

Changes

  • Accumulate into a bytearray with .extend() instead of rebinding a bytes object (amortized O(1) vs O(n²)).
  • Per-page caps, fail closed (a single pathological page → treat as too complex, skip PDFMiner):
    • max_raw_stream_bytes (50 MB) — decoded bytes per page, checked before each stream is copied so an oversized stream is never accumulated or regex-scanned.
    • max_content_stream_array_entries (10,000, pypdf's number) — bounds an array of many empty/tiny streams that a byte cap alone misses.
  • Document-level caps so total work is a property of the function, not the page count (pages can share one indirect /Contents array → tiny file, unbounded scan). Charged incrementally per stream so a mid-page decode error can't discard the accounting. Set far above any plausible real document and logged at warning:
    • max_total_stream_bytes (1 GB) — decoded bytes per document.
    • max_total_array_entries (1,000,000) — decoded entries per document (zero-byte entries never advance the byte total but still cost a decode).
  • Dereference an indirect /Contents before the array check — DictionaryObject.get (unlike __getitem__) does not resolve references, so an indirect array of streams was silently skipping the array branch entirely (both the heuristic and the caps). 87 of 1,105 corpus pages reach their content array this way.
  • Count operators with finditer instead of findall, so counting no longer allocates a match list proportional to stream size.
  • Bump pypdf to >=6.9.1 (from >=6.6.2) so the library's own code path is patched too; lock resolves 6.10.0.

Audit (AC4)

Grepped unstructured/partition/pdf.py and pdf_image/ for other +=-on-stream-bytes loops — none found; this was the only instance. A separate O(n²) string-accumulation pattern in unstructured/partition/html/transformations.py (element-merge loop) is out of SEC-146's scope (not on the PDF path) and is tracked in its own ticket.

Testing

Rewrote the regression tests around real PdfWriter/PdfReader fixtures (FlateDecode-compressed streams so the file stays small while decoded output is huge — the actual attack shape). Coverage: direct + indirect graphics-heavy arrays, a many-small-streams array (~2 MB file → ~900 MB decoded) that runs for minutes / OOMs on the pre-fix code and returns in ~0.05 s here, the entry cap, the pre-copy byte cap, the cross-page byte and entry budgets, and budget survival across a mid-page decode error. The key tests were confirmed to fail on the pre-fix code.

  • Full test_unstructured/partition/pdf_image/test_pdf.py: 178 passed, 1 skipped
  • ruff check + ruff format --check: clean

Acceptance criteria

  • is_pdf_too_complex accumulates via bytearray, not bytes +=
  • Total-length bounds cap worst-case work on array-based content streams (per-page + per-document, bytes + entries)
  • Regression test with a many-small-streams array PDF completes in bounded time/memory (fails on the old code)
  • Repo audited for other +=-on-stream-bytes copies; findings fixed or ticketed
  • pypdf dependency confirmed ≥ 6.9.1

🤖 Generated with Claude Code

Review in cubic

aadland6 and others added 3 commits August 14, 2026 11:59
…mplex (SEC-146)

is_pdf_too_complex re-implemented pypdf's array-based /Contents decoding using
the same O(n^2) `raw_data += obj.get_data()` accumulation pattern patched
upstream under CVE-2026-33123 (GHSA-qpxp-75px-xjcp). A crafted PDF with many
small stream objects in a page's /Contents array could force excessive
CPU/memory on this untrusted-input path.

- Accumulate into a bytearray with .extend() instead of rebinding bytes.
- Cap total accumulated bytes per page at MAX_RAW_STREAM_BYTES (50 MB) as
  defense-in-depth, mirroring pypdf's own fix.
- Bump the pinned pypdf dependency to >=6.9.1 so the library's own code path
  is patched too.
- Add regression tests: bounded-time accumulation over 15k small streams, and
  early-break enforcement of the byte cap.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…C-146)

Addresses findings from three peer reviews of the initial fix:

- Dereference an indirect /Contents up front. DictionaryObject.get (unlike
  __getitem__) does not resolve references, so a page whose /Contents was an
  indirect array of streams silently skipped the array branch -- both the
  complexity heuristic and the new caps were bypassed. (reviewer C, confirmed)
- Add an array-entry cap (max_content_stream_array_entries=10_000, pypdf's
  number). A byte cap alone does not bound an array of many empty/tiny streams;
  each entry still forces an object resolve + get_data. (reviewers A, C)
- Add a document-level decoded-byte budget (max_total_stream_bytes=200MB). The
  per-page cap left total work at pages x per-page cap, and pages can share one
  indirect /Contents array, so a ~30KB file could force minutes of scanning.
  (reviewer B, High, confirmed: 31KB/8-page file -> 3.5s pre-budget)
- Check the byte budget before extending, so an oversized single stream is
  never fully materialized into the accumulator and scanned. (reviewer A)
- Fail closed: exceeding any resource limit now returns True (too complex ->
  skip PDFMiner) instead of classifying a truncated prefix.
- Count operators with finditer instead of findall, so counting no longer
  allocates a match list proportional to stream size. (reviewers B, C)
- Promote the caps to keyword parameters with module-constant defaults, matching
  the existing min_* knobs, and document them.

Tests rewritten around real PdfWriter/PdfReader fixtures (FlateDecode streams so
the file stays small while decoded output is huge -- the real attack shape):
direct + indirect graphics-heavy arrays, a many-small-streams array that fails
on the pre-fix code, the entry cap, the pre-copy byte cap, and the cross-page
budget. All deterministic; the one timing assertion has ~200x headroom.

Note: reviewer B flagged a separate O(n^2) string-accumulation pattern in
unstructured/partition/html/transformations.py (element merge loop) -- out of
SEC-146's scope (pdf.py / pdf_image/), to be tracked separately.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Addresses the follow-up reviews of af7617d.

reviewer A (Medium, confirmed): the document byte budget was charged once per
fully-decoded page, after the array loop, so a page whose array was
[valid stream, stream that raises] hit `except: continue` and discarded the
accounting -- a ~2 KB file with many such pages restored unbounded cross-page
work (10 pages -> 900 KB decoded, budget never fired). The budget is now charged
incrementally as each chunk is decoded, before the copy and before a sibling can
raise, so partial-page work always counts.

reviewer B (Medium A): the document budget failed closed, which for an aggregate
over ordinary pages can silently strip the text layer from a large legitimate
PDF (~419 pages at 500 KB/page). Per the author decision, kept fail-closed but
raised the default well above any plausible real document (200 MB -> 1 GB;
heaviest real page is ~31 MB) and log at warning so the rare event is visible.

reviewer B (Medium B): the byte budget did not bound an array of many zero-byte
entries -- each costs a get_data() but adds no bytes, so page count multiplied
work again (2,000 pages -> 4.68 s, budget never moved). Added a document-level
entry budget (max_total_array_entries, 1,000,000) charged per entry.

Also from the reviews:
- Non-array branch no longer copies the stream into a bytearray (aliases the
  bytes directly; the regexes accept bytes). (reviewer B note)
- Docstring summary and the "materialized" wording corrected: an oversized
  stream is never accumulated or regex-scanned, but get_data() does decode it
  once (matching pypdf's own order). (reviewers B, C notes)
- Test fixture documents why it uses pypdf private API (writing an
  already-compressed stream to keep the file small). (reviewers B, C notes)

New tests: document-budget survives mid-page decode errors, and the entry budget
bounds a shared many-empty-entry array across pages. Full test_pdf.py: 178
passed, 1 skipped.

Not changed (deliberately): operator counting still counts all matches rather
than early-stopping at max_graphics_ops -- early-stop would under-count the
graphics:text ratio and change classification; CPU is already bounded by the
byte caps. (reviewer C optional note)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files

Shadow auto-approve: would not auto-approve because issues were found.

Re-trigger cubic

Comment thread unstructured/partition/pdf.py Outdated
Comment thread unstructured/partition/pdf.py
Comment thread test_unstructured/partition/pdf_image/test_pdf.py
… (SEC-146)

Addresses cubic review comments on #4437.

- P1 (pdf.py): a stream that pypdf refuses to decode because its output exceeds
  pypdf's own per-filter limit (LimitReachedError -- e.g. a FlateDecode
  compression bomb capped at ZLIB_MAX_OUTPUT_LENGTH, 75 MB) was caught by the
  generic `except Exception: continue` and the page was silently skipped, so the
  file failed open and the bomb was handed to PDFMiner. Now caught specifically
  and failed closed (return True), consistent with the other per-stream caps.
  Other decode errors still skip the page (fail open) so one odd/unsupported
  stream doesn't flag an otherwise-fine PDF.
- The related comment about decoding before the byte cap is resolved by the
  same change: the declared /Length is the compressed size (tiny for a bomb) so
  it can't gate the decode earlier, and pypdf already bounds each decode at its
  75 MB limit and raises -- which now fails closed rather than spiking memory.
- P3 (test): removed dead `BigStream.calls` attribute/increment that no
  assertion read.

New test (parametrized over array and standalone /Contents): a stream raising
LimitReachedError makes is_pdf_too_complex return True. Full test_pdf.py: 180
passed, 1 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 2 files (changes from recent commits).

Shadow auto-approve: would require human review. Introduces fail-closed caps that flag large content streams as too complex, skipping text extraction; a human should validate these thresholds and the product impact on legitimate large PDFs.

Re-trigger cubic

… (SEC-146)

Review finding: the document entry budget (max_total_array_entries) was charged
only after an array item resolved to an object with get_data(), so non-stream
entries (NullObject, dicts) were resolved and traversed but never counted. A
crafted PDF sharing one array of up to 10,000 non-stream objects across many
pages kept each page under the per-page cap while the document budget never
advanced -- CPU work scaled with page count again (reproduced: 1,000 pages x
9,999 nulls, ~10M traversals, returned False).

Charge len(contents) against the entry budget up front, before resolving or
decoding any entry, and check it before the loop. This counts every slot
(stream or not) and avoids iterating an array that already exceeds the budget.
The byte budget is still charged incrementally per decoded stream, so the
mid-page decode-error accounting is unchanged.

New regression test: a shared array of NullObjects across 200 pages with
max_total_array_entries=1 now fails closed (was False on the pre-fix code).
Full test_pdf.py: 181 passed, 1 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 3 files (changes from recent commits).

Shadow auto-approve: would require human review. Adds new fail-closed caps and threshold behavior for content streams; choosing these limits is an operational tradeoff a human should review for impact on legitimate PDFs.

Re-trigger cubic

… (SEC-146)

Comment/docstring/CHANGELOG wording only; no behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 3 files (changes from recent commits).

Shadow auto-approve: would require human review. Adds new per-page/document caps on decoded bytes and array entries to the complexity heuristic, changing behavior for large legitimate PDFs; a human should validate the thresholds.

Re-trigger cubic

Cut this as the 0.26.1 release rather than an unreleased dev cycle, per review.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 2 files (changes from recent commits).

Shadow auto-approve: would require human review. Introduces resource caps and fail-closed behavior on a core PDF path; flagging legitimate large PDFs as too complex is a product/operational tradeoff. Full implementation not fully visible due to truncated diff.

Re-trigger cubic

@badGarnet

Copy link
Copy Markdown
Collaborator

a few comments:

  1. High — Small compressed PDFs bypass every new safety limit
    unstructured/partition/pdf.py:704

    Files under 1 MB return False before their content structure is inspected. A 55 KB PDF containing 9,000 references to one 100 KB compressed stream represents about 900 MB of content but bypasses the check.

    Reproduce: create that PDF with PdfWriter; the default call returns False, while passing min_file_size_bytes=1 returns True.

    Suggested fix: remove the file-size shortcut, or at least inspect /Contents arrays and compressed streams before allowing small files through.

  2. Medium — One decode error skips the rest of the page and fails open
    unstructured/partition/pdf.py:809

    Any exception other than LimitReachedError causes the entire page to be skipped. Later streams may therefore exceed the byte limits without inspection.

    Reproduce: create /Contents as [unsupported-filter stream, valid 10-byte stream], set max_total_stream_bytes=1, and call with min_file_size_bytes=1. It incorrectly returns False.

    Suggested fix: return True when a content stream cannot be inspected, or catch errors per item and fail closed for that page.

  3. Medium — The byte cap is checked only after allocating decoded data
    unstructured/partition/pdf.py:762 and unstructured/partition/pdf.py:783

    get_data() fully decompresses the stream before its size is compared with max_raw_stream_bytes. Thus, the configured cap does not bound peak memory.

    Reproduce: use a 59 KB PDF containing one stream that expands to 60 MB, set max_raw_stream_bytes=1_000_000, and measure with tracemalloc. The call returns True, but peaks around 129 MiB.

    Suggested fix: decode incrementally with the remaining page/document budget, or add/use a pypdf decoding API that accepts a per-call output limit.

… (SEC-146)

Addresses PR review (badGarnet):

- High: small compressed files bypassed every cap via the min_file_size_bytes
  gate (a 55 KB file can declare ~900 MB of content). Default is now 0 (inspect
  every file); skipping small files is opt-in. Inspection is bounded, and normal
  small files stay cheap (~0.4 ms) via the min_raw_stream_bytes per-page skip.
- Medium: a decode error on one array stream skipped the whole page (fail open),
  so later streams evaded the byte limits. Decode is now per-item: a bomb still
  fails closed (LimitReachedError -> True), but an unreadable stream skips only
  itself and the rest of the page is inspected and charged.
- Medium (peak memory during decode): not separately fixable here. get_data()
  must decode to know a stream's size, and a single stream's decode is already
  bounded by pypdf's own 75 MB per-filter limit, which raises LimitReachedError
  and now fails closed. pypdf exposes no per-call output limit and lowering its
  module-global limit is not thread-safe.

New tests: a small (<1 MB) array-bomb flagged by default, and a bad array stream
that no longer masks an over-cap sibling. Full test_pdf.py: 183 passed, 1 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 3 files (changes from recent commits).

Shadow auto-approve: would require human review. Changes the default min_file_size_bytes to 0, expanding the complexity check to all files, and introduces new resource caps and a pypdf version bump. The performance/security tradeoff of inspecting every PDF deserves human sign-off.

Re-trigger cubic

@upwind-code-us

upwind-code-us Bot commented Aug 14, 2026

Copy link
Copy Markdown

Upwind Upwind Code Scan - ⛔ Do Not Deploy

77 newly introduced vulnerabilities · 0 resolved · 77 total in this PR vs main

🔴 2 Critical | 🔶 38 High | 🟡 33 Medium | 🟢 4 Low


🔴 Critical · 2 findings
CVE Package Version Fix
CVE-2026-45829 chromadb 1.5.7
CVE-2026-54058 pillow 12.2.0 12.3.0

🔶 High · 38 findings
CVE Package Version Fix
CVE-2026-54280 aiohttp 3.13.5 3.14.1
CVE-2026-54059 pillow 12.2.0 12.3.0
CVE-2026-54275 aiohttp 3.13.5 3.14.1
CVE-2026-59935 pypdf 6.10.0 6.14.2
CVE-2026-54278 aiohttp 3.13.5 3.14.1
CVE-2026-59199 pillow 12.2.0 12.3.0
CVE-2026-59939 httplib2 0.31.2 0.32.0
CVE-2026-59884 pyasn1 0.6.3 0.6.4
CVE-2026-47265 aiohttp 3.13.5 3.14.0
CVE-2026-59198 pillow 12.2.0 12.3.0
CVE-2026-59936 pypdf 6.10.0 6.14.1
CVE-2026-45134 langsmith 0.7.29 0.8.0
CVE-2026-34993 aiohttp 3.13.5 3.14.0
CVE-2026-55380 pillow 12.2.0 12.3.0
CVE-2026-59200 pillow 12.2.0 12.3.0
CVE-2026-69244 aiohttp 3.13.5 3.14.3
CVE-2026-54060 pillow 12.2.0 12.3.0
CVE-2026-54277 aiohttp 3.13.5 3.14.1
CVE-2026-59885 pyasn1 0.6.3 0.6.4
CVE-2026-41066 lxml 6.0.3 6.1.0
CVE-2026-59886 pyasn1 0.6.3 0.6.4
CVE-2026-50269 aiohttp 3.13.5 3.14.0
CVE-2026-59203 pillow 12.2.0 12.3.0
CVE-2026-59937 pypdf 6.10.0 6.14.0
CVE-2026-49477 soupsieve 2.8.3 2.8.4
CVE-2026-54279 aiohttp 3.13.5 3.14.1
CVE-2026-69247 cryptography 46.0.7 50.0.0
CVE-2026-59197 pillow 12.2.0 12.3.0
CVE-2026-49476 soupsieve 2.8.3 2.8.4
CVE-2026-54274 aiohttp 3.13.5 3.14.1
CVE-2026-44843 langchain-core 1.2.28 1.3.3
CVE-2026-59205 pillow 12.2.0 12.3.0
CVE-2026-44432 urllib3 2.6.3 2.7.0
CVE-2026-54273 aiohttp 3.13.5 3.14.1
CVE-2026-59204 pillow 12.2.0 12.3.0
CVE-2026-48526 pyjwt 2.12.1 2.13.0
CVE-2026-55379 pillow 12.2.0 12.3.0
CVE-2026-69249 cryptography 46.0.7 49.0.0

🟡 Medium · 33 findings
CVE Package Version Fix
CVE-2026-44512 onnx 1.21.0 1.22.0
CVE-2026-54531 pypdf 6.10.0 6.13.0
CVE-2026-71870 pypdf 6.10.0 6.15.0
CVE-2026-41425 authlib 1.6.9 1.6.11
CVE-2025-3000 torch 2.10.0 2.13.0
CVE-2026-54276 aiohttp 3.13.5 3.14.1
CVE-2026-69243 aiohttp 3.13.5 3.14.2
CVE-2026-41182 langsmith 0.7.29 0.7.31
CVE-2026-71852 pypdf 6.10.0 6.15.0
CVE-2026-55798 pillow 12.2.0 12.3.0
CVE-2026-48735 pypdf 6.10.0 6.12.1
CVE-2026-47751 anthropics/claude-code-action beta 1.0.74
CVE-2026-59890 setuptools 82.0.1 83.0.0
CVE-2026-71554 h2 4.3.0 4.4.1
CVE-2026-48523 pyjwt 2.12.1 2.13.0
CVE-2026-48525 pyjwt 2.12.1 2.13.0
CVE-2026-41168 pypdf 6.10.0 6.10.1
CVE-2026-41314 pypdf 6.10.0 6.10.2
CVE-2026-41479 authlib 1.6.9 1.6.10
CVE-2026-41312 pypdf 6.10.0 6.10.2
CVE-2026-45409 idna 3.11 3.15
CVE-2026-69248 cryptography 46.0.7 49.0.0
CVE-2026-49461 pypdf 6.10.0 6.12.2
CVE-2026-44681 authlib 1.6.9 1.6.12
CVE-2026-59881 aiohttp 3.13.5 3.14.2
CVE-2026-44431 urllib3 2.6.3 2.7.0
CVE-2026-54651 pypdf 6.10.0 6.13.1
CVE-2026-41313 pypdf 6.10.0 6.10.2
CVE-2026-41481 langchain-text-splitters 1.1.1 1.1.2
CVE-2026-48155 pypdf 6.10.0 6.12.0
CVE-2026-59938 pypdf 6.10.0 6.14.0
CVE-2026-54530 pypdf 6.10.0 6.13.0
CVE-2026-48522 pyjwt 2.12.1 2.13.0

🟢 Low · 4 findings
CVE Package Version Fix
CVE-2026-48156 pypdf 6.10.0 6.12.0
CVE-2026-44405 paramiko 4.0.0
CVE-2026-48524 pyjwt 2.12.1 2.13.0
CVE-2026-49460 pypdf 6.10.0 6.12.2

View full analysis in Upwind Console

Scan completed in 1m 17s

Scan history (2 scans)
Commit Scanned at New Resolved Net
4dd9d50 2026-08-14 21:26 UTC +77 0 +77
c8b495b < 2026-08-14 21:35 UTC +77 0 +77

Last scanned: c8b495b · 2026-08-14 21:35 UTC

@upwind-code-us

upwind-code-us Bot commented Aug 14, 2026

Copy link
Copy Markdown

Upwind Upwind IaC Scan - ✅ Proceed with Deployment

3 misconfigurations detected

🔶 1 High | 🟡 1 Medium | 🟢 1 Low

No default-branch baseline yet — showing all findings.


🔶 High · 1 finding
Rule Resource File
'apk add' is missing '--no-cache' Dockerfile

🟡 Medium · 1 finding
Rule Resource File
':latest' tag used Dockerfile

🟢 Low · 1 finding
Rule Resource File
No HEALTHCHECK defined Dockerfile

View full analysis in Upwind Console →

Scan completed in 45s

Scan history (2 scans)
Commit Scanned at New Resolved Net
4dd9d50 2026-08-14 21:26 UTC +3 0 +3
c8b495b < 2026-08-14 21:36 UTC +3 0 +3

Last scanned: c8b495b · 2026-08-14 21:36 UTC

@aadland6

Copy link
Copy Markdown
Contributor Author

Thanks @badGarnet — all three reproduced and addressed in c8b495b.

1. Small files bypass the new limits (High). Fixed. min_file_size_bytes now defaults to 0, so every file is inspected; skipping small files is opt-in. Inspection is bounded, and normal small files stay cheap (~0.44 ms/call) because the per-page min_raw_stream_bytes skip avoids the regex on small content. Regression test: a <1 MB file whose array decodes to ~900 MB is now flagged by default.

Tradeoff worth a look: this reverts the small-file fast path from #4268 (~0.44 ms/PDF on the partition path). The knob remains, so a perf-sensitive caller can set min_file_size_bytes back and accept the bypass. Happy to restore the old default if you'd rather handle small-file bombs elsewhere.

2. One decode error skips the rest of the page (Medium). Fixed. Decoding is now per stream: a bomb still fails closed (LimitReachedErrorTrue), but an otherwise-unreadable stream skips only itself, so the remaining streams on the page are still inspected and charged. Regression test: [unsupported-filter stream, over-cap stream] now returns True (was False).

3. Byte cap checked only after decode (Medium). Confirmed, but bounded and not separately fixable here. get_data() must decode a stream to learn its size, and a single stream's decode is already capped by pypdf's own per-filter limit (ZLIB_MAX_OUTPUT_LENGTH, 75 MB), which raises LimitReachedError — now caught and failed closed. pypdf exposes no per-call output limit, and lowering its module-global limit around get_data() isn't thread-safe (it would affect concurrent pypdf work in the same process). So peak per-stream memory is bounded by pypdf (≈75 MB) rather than by max_raw_stream_bytes, and the docstring no longer overclaims otherwise. Glad to add a tighter bound if pypdf ever grows a per-call limit.

Full test_pdf.py: 183 passed, 1 skipped.

@aadland6
aadland6 added this pull request to the merge queue Aug 15, 2026
Merged via the queue into main with commit 44f9d74 Aug 15, 2026
56 checks passed
@aadland6
aadland6 deleted the matthew/sec-146-quadratic-memorycpu-in-is_pdf_too_complex-array-stream branch August 15, 2026 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants