[Feature]: Add strategy-routed, context-isolated Red-Green verification

[GenTang]

痛点与背景 / Problem and Context

Yada currently verifies a task when the latest workspace revision passes a model-labeled test or build command.

For bug-fixing tasks, the Agent can suffer from False Positive Verification or self-deception: it writes the fix and the test simultaneously, sees the test pass, and concludes success. This does not prove that the test would have caught the original bug on the baseline production code.

Prompt-only constraints such as “follow TDD” are insufficient to prevent the Agent from skipping the actual Red-Green cycle.

However, Yada also handles non-bug tasks such as feature implementation, documentation, and refactoring, where strict Red-Green verification may be inapplicable. Red-Green therefore cannot be forced blindly on every task.

Yada needs a host-enforced state machine that can:

1. Allow the existing Agent to classify the task before editing.
2. Enforce a strict Red-Green cycle for reproducible bugs.
3. Isolate test-authoring and bug-fixing contexts to reduce confirmation bias and reasoning leakage.
4. Preserve Yada’s small, deterministic, and auditable design without introducing a general multi-agent orchestration framework.

期望的解决方案 / Proposed Solution

Introduce a host-controlled state machine starting with a mandatory select_strategy tool call.

The initial strategy catalog contains exactly:

red_green
direct_execute

Strategy selection happens inside the existing Agent session. No separate Router Agent or Router session is introduced.

1. Mandatory Strategy Selection

Add select_strategy to the stable tool schema:

{
  "strategy": "red_green",
  "reason": "The task describes a reproducible regression in existing behavior."
}

Rules:

• The Agent may use read-only tools before selecting a strategy.
• Read-only repository inspection may include search, file reads, and non-mutating inspection commands.
• All file-mutating tools, including replace_text and apply_patch, are rejected until select_strategy succeeds.
• The selected strategy must be either red_green or direct_execute.
• Strategy selection is allowed only once per run.
• The selected strategy is irreversible and frozen before the first edit.
• Later calls to select_strategy return a structured rejection.
• The selected strategy and reason are recorded in the run trace.

2. The direct_execute Flow

Use direct_execute for tasks that do not require a meaningful baseline failure, such as documentation, mechanical changes, and some feature or refactoring tasks.

Rules:

• The current Agent session continues without creating another session.
• A baseline Red observation is not required.
• Yada’s existing editing and post-change verification behavior remains unchanged.
• direct_execute does not bypass the existing latest-revision verification gate.

3. The red_green Flow

Use red_green for reproducible bug fixes.

The host enforces a sequential, two-phase pipeline.

Phase A: Red Phase — Test Authoring

The current Agent session becomes the Red phase.

Rules:

• The Agent may read production and test code.
• The Agent may modify only files classified as test files.
• Production code edits are rejected by the executor.
• The Agent submits a test patch, target test identity, and exact test command.
• The Red phase cannot complete the overall task by calling finish_task.

The host runs the proposed test against:

baseline production code + proposed test patch

A Red observation is valid only when:

• the target test is collected;
• the target test is executed;
• the target test fails before any production fix is applied;
• the failure represents an observed behavioral or assertion failure;
• the failure is not caused only by syntax, import, collection, dependency, environment, timeout, or infrastructure errors;
• the test is not skipped or reported as having no tests collected.

The host records:

baseline revision
test patch SHA
test file paths
target test identity
test command fingerprint
Red exit code
bounded Red output
Red observation status

A test that has never produced a valid Red observation must not count as verification.

If the Red phase cannot produce a valid Red observation, the run ends as unfinished. It must not silently fall back to direct_execute.

State Transition: Freeze the Test

After a valid Red observation:

• freeze the test patch and its SHA;
• freeze the target test identity;
• freeze the Red test command fingerprint;
• end the Red session;
• make all frozen test files immutable for the Fix phase.

Any later modification to a frozen test invalidates the Red-Green workflow and must be rejected.

Phase B: Fix Phase — Context Isolated

The host starts one fresh Agent session using the same configured model and normal Yada runtime abstractions.

The Fix workspace starts from:

baseline production code + frozen test patch

The Fix session receives only explicit task artifacts and evidence:

• the original task;
• the baseline revision;
• the frozen test patch;
• the frozen test identity;
• the Red command;
• the bounded Red failure output.

The Fix session must not receive:

• the Red session’s message history;
• the Red session’s analysis or reasoning;
• the Red session’s tool-call history beyond the explicit Red evidence;
• any production fix proposal from the Red phase.

Rules:

• The Fix Agent may modify production code.
• Frozen test files are immutable.
• Attempts to modify frozen test files are rejected by the executor.
• The Fix Agent must run the frozen target test against the latest production revision.
• Relevant existing regression verification must also pass.

Green and Completion Gate

For red_green, finish_task is accepted only when:

• a valid Red was observed against baseline production code;
• the test patch remains identical to the frozen test patch SHA;
• the same target test is used for Green;
• the same test command, or a host-approved equivalent, is used;
• the frozen target test passes against the latest production revision;
• required regression verification passes;
• all verification evidence belongs to the latest production revision.

The required state transition is:

Strategy Selected
    → Red Started
    → Valid Red Observed
    → Test Frozen
    → Fix Session Started
    → Green Observed
    → Regression Verified
    → Finish Accepted

Any production edit after Green invalidates the Green and regression evidence. The updated revision must be verified again before finish_task can succeed.

Scope Boundaries / Non-Goals

This feature must not:

• add a dedicated Router Agent or Router session;
• require the caller to select a strategy manually;
• allow the model to invent arbitrary verification strategies;
• allow strategy changes after the first edit;
• silently downgrade red_green to direct_execute;
• introduce manager, executor, or auditor Agents;
• introduce free-form Agent-to-Agent communication;
• introduce a general sub-agent or multi-agent orchestration framework;
• transfer the Red session’s conversation or hidden reasoning into the Fix session;
• rely on prompt-only enforcement for file permissions or state transitions;
• allow the Red phase to modify production code;
• allow the Fix phase to modify frozen tests;
• accept syntax, import, collection, dependency, timeout, environment, or infrastructure failures as valid Red evidence;
• accept a test that was never observed failing against baseline production code;
• redesign the existing direct_execute verification behavior.

The restrictions must be enforced by the Python host and executor, not by prompt instructions alone.

Acceptance Criteria

• Add select_strategy with enum values red_green and direct_execute.
• Keep select_strategy in the stable tool schema for the complete run.
• Permit read-only repository inspection before strategy selection.
• Reject replace_text, apply_patch, and all other file mutations until a strategy is selected.
• Allow strategy selection only once per run.
• Freeze the selected strategy before the first edit.
• Record the selected strategy and reason in the run trace.
• Implement direct_execute without creating an additional Agent session.
• Preserve Yada’s existing post-change verification behavior under direct_execute.
• Implement the Red phase with executor-enforced test-only mutation permissions.
• Reject production code edits during the Red phase.
• Run the proposed test against baseline production code plus the test patch.
• Verify that the target test is collected, executed, and fails.
• Reject syntax, import, collection, dependency, timeout, environment, skipped-test, no-test-collected, and infrastructure outcomes as valid Red evidence.
• Record the baseline revision, test patch SHA, test identity, command fingerprint, and Red output.
• Reject Red-Green verification when no valid Red has been observed.
• End the run as unfinished if red_green cannot produce a valid Red.
• Freeze the valid Red test patch, target identity, and command fingerprint.
• End the Red session after the test is frozen.
• Start one fresh Fix session without the Red session’s message or reasoning history.
• Initialize the Fix workspace from baseline production code plus the frozen test patch.
• Pass only the original task, frozen test, and explicit Red evidence into the Fix session.
• Reject all attempts by the Fix session to modify frozen test files.
• Require the exact frozen test to become Green against the latest production revision.
• Require relevant regression verification after Green.
• Invalidate Green and regression evidence after any later production edit.
• Reject finish_task when required Red-Green evidence is missing, changed, or stale.
• Emit the complete transition sequence to the .yada/runs trace log:
  • strategy_selected
  • red_started
  • red_observed
  • test_frozen
  • fix_started
  • green_observed
  • regression_verified
  • finish_accepted
• Implement the workflow as two sequential, host-controlled sessions rather than a general multi-agent framework.