Skip to content

feat: optimize the build pipeline with per-context caching and skip-unchanged writes#113

Merged
YusukeHirao merged 21 commits into
v3from
feat/build-performance
Jun 10, 2026
Merged

feat: optimize the build pipeline with per-context caching and skip-unchanged writes#113
YusukeHirao merged 21 commits into
v3from
feat/build-performance

Conversation

@YusukeHirao

@YusukeHirao YusukeHirao commented Jun 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

3行まとめ

  • 何を: ビルドの「ページ数 N に比例して毎回繰り返していた処理」を1回化し、未変更出力の書き込みもスキップできるようにしました。
  • なぜ: 10万ページ規模を見据えると、レイアウトの再コンパイルや設定の再読込など N 回走る固定コストがボトルネックになるため。
  • 結果: 合成ベンチ（1,000ページ）で 約2.4〜2.8倍（603 → 1,454+ pages/s）。挙動の互換性は1点だけ破壊的変更あり（下記）。

⚠️ 破壊的変更（1点）

compileHooks / transforms を 関数で指定している場合、解決タイミングが「ファイルごと」→「build/serve コンテキストごとに1回」に変わります。

// Before: ページごとにファクトリが呼ばれていた
// After:  ビルド開始時に1回だけ呼ばれ、結果を全ページで共有

// ❌ ファクトリ内でページ単位の値を読むコードは動かなくなる
transforms: (defaults) => [{ name: 'x', transform: () => factoryTime }, ...defaults]

// ✅ ページ単位の値は transform の「実行時」に読む
transforms: (defaults) => [{ name: 'x', transform: () => Date.now() }, ...defaults]
  • オブジェクト形式で渡している場合・createCompileHooks() を使っている場合は 影響なし。
  • 詳細な移行手順は MILESTONE.md を参照。

変更点（パッケージ別）

パッケージ 変更
kamado (core) ビルド開始時にモジュールキャッシュを一括クリア / getAssetGroup() のメモ化で二重 glob を解消 / mkdir の重複排除 / --skip-unchanged で未変更出力の write をスキップ（mtime 保持）
pug-compiler コンパイル済みテンプレート関数を LRU キャッシュ（共有レイアウトのコンパイルが「ページごと」→「ビルドごと1回」）。serve は毎回再コンパイルで編集を即反映
page-compiler compileHooks / transforms をコンテキストごと1回解決（上記の破壊的変更）。cache フラグをコンパイルフックまで配管
style-compiler PostCSS 設定・プラグインをコンテキストごと1回構築。serve では再構築して postcss.config.js 編集を即反映。構築失敗は非キャッシュ（次回リトライ）
script-compiler esbuild を write: false でインメモリ化（tmp ファイル往復を排除）。出力は outfile パス一致で選択
ベンチ基盤 yarn bench（--pages / --runs / --full）

注: このブランチは dev のソースマップ機能（#112）をマージ済みです。

ベンチマーク（中央値・ローカル計測）

ページ数 Before After
500 1.47s (341 p/s) 0.54s (926 p/s)
1,000 1.66s (603 p/s) 0.60〜0.69s (1,454〜1,671 p/s)

テスト

  • 338 テスト全パス（本ブランチで新規 spec 5本: style / script / cli E2E / build-pipeline 統合 / pug キャッシュ群）
  • serve モードの編集即反映（ページ・レイアウト・include）を実機確認
  • yarn lint / yarn build パス

🤖 Generated with Claude Code

YusukeHirao and others added 12 commits May 12, 2026 12:29
@YusukeHirao
refactor(page-compiler)!: route all transform failures through format…
4b1d189 
…Options.parseError

Move the parseError catch out of the prettier transform and into the page
pipeline. The same policy now covers every transform — prettier, minifier,
manipulateDOM, and any custom transforms supplied via the `transforms` option.

Motivation: html-minifier-terser was throwing on malformed input with no source
context, mirroring the original prettier problem. Adding a second per-transform
parseError option would have meant duplicating the catch and giving users a way
to set inconsistent policies. A single pipeline-level catch is simpler and more
intuitive.

Behavior on failure:
- 'silent' (default): the failing transform is skipped, the previous step's
  output flows through to the next transform
- 'warning': console.warn with `Transform '<name>' failed on <source>: <original>`
  then skip
- 'error': throw an Error with the same message; original error preserved on
  `error.cause`

BREAKING CHANGE: `PrettierOptions.parseError` and the `PrettierParseErrorMode`
type are removed. The `DefaultPageTransformsOptions` interface is removed and
`createDefaultPageTransforms()` no longer takes an argument. The unified type
is now `ParseErrorMode` exported from the package entry. Error message format
changed to `Transform '<name>' failed on <source>: <original>`.

Tests cover prettier, minifier, and custom-transform failure under each of the
three modes, plus pipeline continuation (a second failure after the first is
still logged) and the `inputPath` -> `outputPath` fallback in the message.
@YusukeHirao @claude
feat(kamado): add outputPathConflict policy for output-path collisions
e75dcbb 
The orchestrator's output-path collision behavior is now configurable per compiler entry via
`outputPathConflict: 'error' | 'warning' | 'silent'`, defaulting to `'warning'`. When a winner must
be picked, a file whose outputPath came from the frontmatter override beats one using the default
computed path; otherwise the first-seen file wins.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@YusukeHirao
docs(page-compiler): document outputPathConflict policy
8f8241e
@YusukeHirao
chore(release): publish v2.0.0-alpha.16
5794229
@YusukeHirao
chore(repo): wire up build benchmark command
0baea5f 
- add `yarn bench` script pointing to packages/kamado/benchmark
- ignore generated benchmark fixtures (packages/**/.bench/)
- document the bench command in CLAUDE.md
@YusukeHirao
feat(kamado): optimize the build pipeline and add skip-unchanged writes
445ae6a 
- clear module caches (asset group / file content / global data) at build
  start so consecutive builds in one process always reflect source edits
- memoize getAssetGroup() by enumeration inputs so build() and
  getGlobalData() share a single glob + frontmatter pass; export
  clearAssetGroupCache from kamado/data
- deduplicate output-directory mkdir calls within a build
- add the skipUnchanged build option and --skip-unchanged CLI flag that
  skip rewriting outputs whose content is unchanged (stat size check
  first, then content comparison; mtime is preserved on a match)
- add a build benchmark harness (yarn bench) and a CLI e2e test
@YusukeHirao
feat(page-compiler)!: resolve compileHooks and transforms once per co…
33132ee 
…ntext

BREAKING CHANGE: function forms of `compileHooks` and `transforms` are now
resolved once per build/serve context instead of once per file. Resolved
hooks and transform instances are shared by all pages of a context and may
run concurrently, so factories must be file-independent and instances must
not keep per-page mutable state. See MILESTONE.md for the migration guide.

Details:
- pass the compile cache flag through the transpile layer into compile hook
  compilers (CompilerFunction gains an optional 4th `cache` parameter)
- hoist parseErrorMode resolution out of the per-file path
- add specs for cache-flag plumbing and once-per-context hook resolution
@YusukeHirao
feat(style-compiler): build the PostCSS processor once per context
3c0e50f 
- load the PostCSS config and construct plugins once per build context and
  reuse the processor across files; rebuild per compilation when
  cache=false so postcss.config.js edits apply during dev without restart
- do not cache a failed processor build; the next compilation retries
- warn when the PostCSS config fails to load for a reason other than not
  existing instead of silently dropping user plugins
- resolve the banner once per context (recomputed when cache=false)
- add style-compiler specs
@YusukeHirao
feat(script-compiler): bundle in memory and select output by path
fb87d98 
- use esbuild write:false instead of a tmp-file round-trip
- select the output file whose path matches the outfile instead of
  outputFiles[0]; warn about ignored extra outputs (e.g. extracted CSS)
- resolve the banner once per context (recomputed when cache=false)
- add script-compiler specs including a CSS-import case
@YusukeHirao
feat(pug-compiler): cache compiled templates per build context
fe7da15 
- cache compiled template functions per compiler instance, keyed by the
  template source with a bounded LRU, so shared layouts compile once per
  build instead of once per page
- honor the cache flag: false (serve mode) bypasses the cache so include
  and extends edits are always reflected per request
- create a fresh compiler (and cache) per compile-hooks factory
  resolution, binding the cache lifetime to a single build context
- add cache behavior specs and a real-FS build pipeline integration spec
@YusukeHirao
docs(kamado): document caching layers, skip-unchanged, and benchmarking
d8a5584 
- README (en/ja): add the --skip-unchanged build flag and a note on
  once-per-context resolution of compileHooks/transforms
- ARCHITECTURE (en/ja): add the caching layers table, update the build
  flow diagram (cache clearing, skip-unchanged write), describe the
  cache-flag propagation, and document the yarn bench workflow
@YusukeHirao
docs(repo): add migration guide for per-context hook resolution
514e929 
Add the v2.0.0 migration entry for the compileHooks/transforms resolution
timing change (per file to per build/serve context) with before/after
rewrite examples and guidance for affected factory patterns.
@YusukeHirao YusukeHirao requested a review from yusasa16 as a code owner June 6, 2026 04:40
@YusukeHirao
Merge branch 'dev' into feat/build-performance
9543aa6 
Resolve conflicts with the sourcemap-option feature (#112):
- script/style compiler: combine per-context caching (banner/processor)
  with the per-command sourcemap flag derived from context.mode
- style compiler: feed the normalized /*! banner through the cached
  processor with the inline source map option
- specs: keep both the caching/retry suites and the sourcemap suites;
  default the postcss-load-config mock so the sourcemap tests stay green
@YusukeHirao YusukeHirao changed the base branch from dev to v3 June 10, 2026 09:58
@YusukeHirao
feat(kamado): add shared compiler helpers and unified cache clearing
ad148e4 
- add clearBuildCaches() as the single entry point for resetting all
  module-level build caches; use it in build() and the benchmark runner
- add createBannerResolver() and resolveSourcemapFlag() so asset compilers
  share the per-context banner caching and sourcemap-flag evaluation
- export the SourcemapOption type from kamado/compiler
- document the cache-flag contract: undefined means caching is enabled;
  test for serve mode with `cache === false`, never a truthiness check
- defer output buffer allocation until needed (size precheck uses
  Buffer.byteLength; strings are written directly)
- clamp benchmark --pages/--runs args to positive integers so invalid
  input falls back instead of reporting meaningless numbers
@YusukeHirao
refactor(style-compiler): use shared banner/sourcemap helpers
d495848 
- replace local banner caching and sourcemap-flag evaluation with
  createBannerResolver and resolveSourcemapFlag from kamado/compiler
- use the shared SourcemapOption type
- unify the spec on a single test harness (makeContext/makeFile/
  createCompileFn) instead of two parallel helper sets
@YusukeHirao
refactor(script-compiler): use shared banner/sourcemap helpers
18690e3 
Replace local banner caching and sourcemap-flag evaluation with
createBannerResolver and resolveSourcemapFlag from kamado/compiler,
and use the shared SourcemapOption type.
@YusukeHirao
docs(pug-compiler): explain why the template cache uses LRU over FIFO
10e8e3d
@YusukeHirao
docs(kamado): document the cache-flag contract and dual serve-mode si…
16dddf9 
…gnals

Clarify in ARCHITECTURE (en/ja) that build() leaves the cache flag
undefined (caching enabled; compare with `cache === false`), and add a
design note about the cache flag and context.mode being two parallel
serve-mode signals to consolidate if a new mode is ever introduced.
@YusukeHirao
test(kamado): cover the new public helpers and cache-clear wiring
a11c328 
- clear-build-caches.spec: wiring test asserting every registered cache
  clear is invoked — removing a clear call from clearBuildCaches fails
- build.spec: files added between consecutive builds are picked up,
  guarding the asset-group cache clearing end-to-end
- create-banner-resolver.spec: once-per-context resolution, cache=false
  re-resolution, string pass-through, and transform application
- resolve-sourcemap-flag.spec: table-driven contract for all option/mode
  combinations
@YusukeHirao
test(style-compiler): assert string output instead of branching in th…
d06ab0d 
…e spec helper
@YusukeHirao
test(script-compiler): assert string output instead of branching in t…
3481f42 
…he spec helper
@YusukeHirao YusukeHirao merged commit 85e6468 into v3 Jun 10, 2026
@YusukeHirao YusukeHirao deleted the feat/build-performance branch June 10, 2026 14:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yusasa16 yusasa16 Awaiting requested review from yusasa16

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@YusukeHirao