Skip to content

refactor(categories): precompute section splits; benchmark follow-ups - #41

Merged
andreujuanc merged 1 commit into
masterfrom
refactor/category-criteria-followups
Aug 26, 2026
Merged

andreujuanc merged 1 commit into
masterfrom
refactor/category-criteria-followups

Conversation

@andreujuanc

Copy link
Copy Markdown
Owner

Summary

Follow-ups from reviewing the merged category Shapes/Review-criteria split (#40):

  • Precompute category section splits at init. Each category .md is embedded and static, but the Shapes/Review getters were re-running splitSections on every call — ~4× per review on the hot path (once per GetShape/GetReviewCriteria per AOI, and again for the cache key vs. prompt build). Now each file is parsed once into parsedCategories at init(); the getters are map lookups. Prompt output and cache keys are byte-identical.
  • Document the unmigrated GetCategoryShapes fallback as an intentional whole-file pass-through (surface everything rather than silently drop patterns).
  • Make the deep-review benchmark's AOI scan provider-agnostic. It hardcoded Gemini and skipped without a Gemini key. It now follows cfg.FastModel (override via PRR_AOI_MODEL), pulls tuning from the model config, and skips only when a keyed provider lacks a key — so the benchmark runs keyless via claude-code.
  • Record measured Opus 4.8 numbers on DefaultStrongModel (deep-review benchmark: 100% recall, 0 FP, 77% severity accuracy).
  • Add init-drift and unmigrated-fallback tests.

Context

No behavior change to review/AOI prompts — the refactor preserves the recall-vs-precision split byte-for-byte (AOI scan gets Shapes only; deep review gets scoped Shapes + Review criteria). The benchmark change enabled validating the Opus 4.8 default keyless, which produced the figures now in the config.go comment.

- Parse each category .md once at init into parsedCategories; the
  Shapes/Review getters become map lookups instead of re-running
  splitSections per call (was ~4x per review on the hot path). Prompt
  output and cache keys are byte-identical.
- Document the unmigrated GetCategoryShapes pass-through as intentional.
- Make the deep-review benchmark's AOI scan provider-agnostic: follow
  cfg.FastModel (override via PRR_AOI_MODEL), skip only when a keyed
  provider lacks a key — so it runs keyless via claude-code.
- Record measured Opus 4.8 deep-review benchmark numbers on
  DefaultStrongModel.
- Add init-drift and unmigrated-fallback tests.
@andreujuanc
andreujuanc merged commit 4b793d0 into master Aug 26, 2026
2 checks passed
@andreujuanc
andreujuanc deleted the refactor/category-criteria-followups branch August 26, 2026 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant