Skip to content

fix(puppeteer-page-scan): finish closing the post-reload detached-Frame race#883

Merged
YusukeHirao merged 2 commits into
devfrom
fix/puppeteer-page-scan-scroll-height-retry
Jun 18, 2026
Merged

fix(puppeteer-page-scan): finish closing the post-reload detached-Frame race#883
YusukeHirao merged 2 commits into
devfrom
fix/puppeteer-page-scan-scroll-height-retry

Conversation

@YusukeHirao

Copy link
Copy Markdown
Member

Summary

#882 のフォローアップ。 nitpicker の本番運用で、稀に mobile-small: skipped — Attempted to use detached Frame がまだ残ることが確認された経路を塞ぐ。

Root cause

前回までの修正で 2 つの経路を塞いだが、 もう一つ未対応の経路があった:

経路 カバー状況
scrollAllOver 内の page.evaluate (scroll step / 初期 scrollHeight / 末尾 scrollTo) #882 で 3 attempts retry ✓
#fetchData の 3 min timeout が #fetchImages (20 min) を内包せず race #882 で 25 min に拡張 ✓
beforePageScan 内の page.evaluate(() => document.body.scrollHeight) (reload 直後、 scrollAllOver の前) 未対応 ← 本 PR

page.reload() が resolve した直後でも Chrome の internal main-frame swap が完了していない瞬間があり、 そこで実行された page.evaluate が detached Frame を投げる。 一度投げると beforePageScan 全体が reject → scrollAllOver の retry レイヤーに到達する前に per-device catch でスキップ。

Changes

@d-zero/puppeteer-scroll

  • evaluateWithFrameRetry / isTransientFrameError を inline ヘルパーから独立モジュール evaluate-with-frame-retry.ts に切り出し、 package root から export
  • code-review 指摘 (puppeteer-scroll と puppeteer-page-scan で同じ retry 仕様を 2 重メンテする状況) を解消
  • helper 単体テスト 12 件追加 (transient error 3 種・大文字小文字 3 種・非 Error・非 transient・成功・1回 retry・3 連続失敗・即 throw)

@d-zero/puppeteer-page-scan

  • before-page-scan.ts の post-reload document.body.scrollHeight 読み取りを evaluateWithFrameRetry でラップ
  • WHY コメントで「reload resolves before Chrome's internal main-frame swap is fully complete」を明示
  • retry integration テスト 3 件追加 (リトライ吸収・3 連続失敗で伝播・非 transient 即 throw)

Test plan

  • yarn test (52 tests in scroll + page-scan、合計 783 tests pass)
  • yarn lint (cspell pass)
  • yarn build (27 projects pass)

Verification (本番運用への反映後)

  • nitpicker で再現性のあった URL 群を再 scrape し、 mobile-small: skipped — Attempted to use detached Frame の出現率を確認 (前回は ~5-10% 程度の頻度で残っていた経路)

🤖 Generated with Claude Code

…red helper module

The inline retry helper used inside `scrollAllOver` is also needed in
`@d-zero/puppeteer-page-scan` to absorb the same Chrome main-frame swap
window around `page.reload()` resolution. Move it to a dedicated module
and export it from the package root so callers can share the same retry
shape (3 attempts × 200 ms gap, scoped to transient frame/session
errors) instead of duplicating the loop and the error-matching regex.
…detached-Frame errors

The `document.body.scrollHeight` read that runs immediately after
`page.reload()` could still throw "Attempted to use detached Frame" on
rare runs (observed in production on a responsive data-table page). The
reload resolves before Chrome's internal main-frame swap is fully
complete, so a single read can land inside the swap window. The
follow-up `scrollAllOver` already retries its own evaluates, but a throw
at this point escapes `beforePageScan` before that retry layer can run.

Wrap the read with `evaluateWithFrameRetry` from `@d-zero/puppeteer-scroll`,
the same helper used internally by `scrollAllOver`, so the two layers
share one retry shape (3 attempts × 200 ms gap).
@YusukeHirao YusukeHirao requested a review from yusasa16 as a code owner June 18, 2026 12:06
@YusukeHirao YusukeHirao merged commit 33ae801 into dev Jun 18, 2026
2 checks passed
@YusukeHirao YusukeHirao deleted the fix/puppeteer-page-scan-scroll-height-retry branch June 18, 2026 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant