痛点与背景 / Problem and Context
Problem
In the red_green flow, the frozen Red test is materialized into the workspace. SWE-bench evaluation currently collects the complete Git diff, so model_patch contains both:
production fix + generated Red test
This may conflict with SWE-bench’s official test_patch and affect evaluation validity.
期望的解决方案 / Proposed Solution
Keep both deliverables in the workspace, but submit only the production fix to SWE-bench:
Workspace: production fix + frozen Red test
SWE-bench patch: production fix only
Trace artifacts: full patch + Red test patch
Requirements
- Exclude only the exact paths recorded in
frozen_test.files.
- Do not broadly filter
tests/**.
- Apply filtering only to
red_green SWE-bench submissions.
- Keep
direct_execute behavior unchanged.
- Fail closed if the frozen test cannot be safely excluded.
- Record excluded paths and submission patch SHA in the trace.
- Add tests covering tracked and newly created Red test files.
痛点与背景 / Problem and Context
Problem
In the
red_greenflow, the frozen Red test is materialized into the workspace. SWE-bench evaluation currently collects the complete Git diff, somodel_patchcontains both:This may conflict with SWE-bench’s official
test_patchand affect evaluation validity.期望的解决方案 / Proposed Solution
Keep both deliverables in the workspace, but submit only the production fix to SWE-bench:
Requirements
frozen_test.files.tests/**.red_greenSWE-bench submissions.direct_executebehavior unchanged.