Skip to content

fix(verifier): fail closed and isolate paper splits - #1

Merged
MaxFreedomPollard merged 2 commits into
MaxFreedomPollard:feat/objective-verifier-fitnessfrom
enzo-adami:agent/pr150-objective-hardening
Aug 16, 2026
Merged

MaxFreedomPollard merged 2 commits into
MaxFreedomPollard:feat/objective-verifier-fitnessfrom
enzo-adami:agent/pr150-objective-hardening

Conversation

@enzo-adami

Copy link
Copy Markdown

This is a focused extension of your NousResearch#150 branch, prompted by two falsifiable admission failures reproduced on the current head.

Changes

  • force the composite admission score to 0.0 when objective correctness is zero; a wrong but well-formed answer can no longer retain roughly half credit
  • allocate papers, not individual tasks, to train/validation/holdout before task sampling so the same paper cannot leak across splits
  • preserve deterministic sampling and requested dataset sizes
  • bring the three touched verifier files up to the current Ruff contract

Validation

  • full repository: 187 passed
  • Ruff on all touched files: PASS
  • git diff --check: PASS

This intentionally does not add another statistical-power implementation. NousResearch#162 already supplies the stronger paired significance/power foundation and should remain the single owner of that layer.

If you prefer, the functional commit is 2041d57; the second commit is lint-only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants