Evidence-gated rearchitecture: measured retrieval/synthesis overhaul, eval harness, code-review fix-set - #951

Merged

PromtEngineer merged 37 commits into

mainfrom

rearchitect/evidence-gated-aug-2026

Aug 21, 2026

Commits on Aug 9, 2026

Rearchitect for doc/code parity; evidence-gated defaults; E2E fixes

PromtEngineer
and
claude
committed
3915351
Add evaluation harness: gold set, retrieval metrics, judge, smoke, decisions

PromtEngineer
and
claude
committed
c62014f
Add research evidence, evidence-based roadmap, and design rationale

PromtEngineer
and
claude
committed
4ed293d
Roadmap Phase 4: evidence-filtered adoption plan for agentic-file-search ideas

PromtEngineer
and
claude
committed
c373825

Commits on Aug 10, 2026

Phase 4: flag-gated adoptions from agentic-file-search, decided by measurement

PromtEngineer
and
claude
committed
f54992e
eval: acquisition corpus, final-list metrics, Phase-4 benchmark record

PromtEngineer
and
claude
committed
e3c3d60
docs: Phase-4 verdicts, token accounting, filters, and parity fixes

PromtEngineer
and
claude
committed
c2d9f48

Commits on Aug 12, 2026

fix: size Ollama num_ctx per request — ends silent 8k front-truncation

PromtEngineer
and
claude
committed
007d0b6
eval: 4.1 escalation re-run post-truncation-fix — reject as default

PromtEngineer
and
claude
committed
2a0774f

Commits on Aug 13, 2026

eval: Anthropic-backend judge option + verifier-suffix stripping + hard-case benchmark

PromtEngineer
and
claude
committed
184dcc7
eval: validate Sonnet-class judge via subagent voters — 38/38, zero splits

PromtEngineer
and
claude
committed
33adb7c
fix: think:false on synthesis streams; quote-safe FTS with dense fallback

PromtEngineer
and
claude
committed
513344f
fix: chunker dropped half of every long document

PromtEngineer
and
claude
committed
7d71051
eval: unseen-corpus RFC dataset — 23 real cross-referencing documents + gold set

PromtEngineer
and
claude
committed
dc539d7
eval: RFC shakedown results — retrieval passes on unseen docs, synthesis is the bottleneck

PromtEngineer
and
claude
committed
80d5215
synthesis: strict grounded prompt + temperature-0 decode (A/B arm C); reject 35b

PromtEngineer
and
claude
committed
f5c587e
eval: strict compose prompt tested and rejected (arm E) — revert, keep arm C

PromtEngineer
and
claude
committed
d6c4bc3

Commits on Aug 15, 2026

Dedupe retrieval legs + budget synthesis context (7/24 -> 16/24)

PromtEngineer
and
claude
committed
aafc193
Reranker as final-stage selection: Qwen3-Reranker-4B on by default (arm G)

PromtEngineer
and
claude
committed
e70611b
Pooled decomposition + deterministic decompose (ties composer, wins on shape)

PromtEngineer
and
claude
committed
bd8ad26
Label synthesis snippets with their source document (17/24 -> 21/24)

PromtEngineer
and
claude
committed
a3f999a
eval: cross-bench validation — RFC-tuned changes generalize; hr -3 diagnosed

PromtEngineer
and
claude
committed
36c4b6e

Commits on Aug 16, 2026

eval: completeness-clause A/B (arm J) — reverted, does not fix hr target rows

PromtEngineer
and
claude
committed
243f7be
fix: create FTS index on late-chunk tables + pin 4b judge to temp 0

PromtEngineer
and
claude
committed
ebcc88b
feat: decomposer sees last assistant answer on multi-turn; single-turn prompt frozen

PromtEngineer
and
claude
committed
c1400ca

Commits on Aug 17, 2026

fix: apply code-review fix-set — correctness, security, determinism, CI

PromtEngineer
and
claude
committed
3a8ebd9
docs: sync all docs with shipped behavior (arm G/H defaults, status codes, line refs)

PromtEngineer
and
claude
committed
3f78be4
eval: fix-set impact measured — authored +4, rfc 21->19 trade, hr_h05 recovered

PromtEngineer
and
claude
committed
e0f2399

Commits on Aug 18, 2026

fix: dev-watcher stream aborts, invisible avatars; feat: Quick Chat streaming

PromtEngineer
and
claude
committed
f697536
fix: markdown answers rendered with doubled whitespace

PromtEngineer
and
claude
committed
ac38f51

Commits on Aug 19, 2026

feat: component ablation study — latechunk + verification off by default

PromtEngineer
and
claude
committed
75b5348
eval: resolve-only + MaxSim experiments — neither adopted; multi-turn matrix

PromtEngineer
and
claude
committed
52553f9

Commits on Aug 20, 2026

Commits on Aug 21, 2026

docs: sync README with current defaults and eval state

PromtEngineer
and
claude
committed
fc994c7