Repository navigation
Recalibrate Windows warm P50 budget for schema-v2 variance #513
Description
Activity
Windows schema-v2 warm full-refresh P50 evidence:
- exact merged baseline
ad7ca14: 105ms - calibration runs: 160ms, 133ms
- PR test: make cold-refresh performance snapshots robust (Fixes #509) #510 final: 182ms
- PR test: tighten schema-v2 warm P95 budgets (Fixes #511) #512 first attempt: 164ms
- PR test: tighten schema-v2 warm P95 budgets (Fixes #511) #512 isolated rerun: 117ms
Observed range: 105-182ms (77ms). The existing 50ms/30% budget blocks above 155ms against the exact baseline, so three unchanged-product runs fail.
Proposed Windows-only warm P50 budget: 150ms / 50%. Against the 105ms baseline it blocks above 255ms, leaving nearly 2x the observed absolute range as headroom while still rejecting a sustained ~2.5x median regression. Linux/macOS, all P95, cold, startup, coverage, and schema behavior remain unchanged.
- exact merged baseline
- added a commit that references this issue
on Aug 11, 2026 Final quality conclusion for PR #515 at
77eb4ce:- Exact-head Copilot review: 3/3 files, 0 comments, 0 unresolved threads.
- Full CI, lint, CodeQL, Linux/Windows coverage, and all exact-base performance gates pass.
- Windows warm P50/P95: 163/180ms vs 137/144ms baseline; cold P50 157ms vs 138ms.
- Linux warm P50/P95: 59/62ms vs 60/62ms; macOS 93/107ms vs 112/137ms.
- All artifacts contain 10 samples, matching inventories, and empty interpreter-timeout maps.
- Line/function coverage is unchanged on Linux and Windows.
- Comparator suite: 35/35 passed; observed 182ms Windows variance passes, 300ms fails, and explicit dual-budget coverage remains.
- The initial Linux check failed closed because the freshly merged exact-base baseline artifact was not yet available; its benchmark was healthy and the rerun passed after baseline completion.
- The earlier manual calibration run also passed after retrying a transient self-signed-certificate failure in the baseline-download action.
Problem
PR #512 changes only warm P95 budgets, but its first Windows performance job failed the unchanged warm full-refresh P50 gate:
ad7ca14: 105msThree observed PR runs therefore exceed the effective ceiling produced by the unusually low merged baseline, despite stable warm P95, cold P50, inventory, and timeout diagnostics. The isolated #512 rerun passed at 117ms, confirming runner variance rather than a product change.
Scope
After #512 merges:
docs/QUALITY_SNAPSHOTS.mdwith the calibration source.Acceptance criteria
Depends on #512.