What happened
On PR #6370, the review agent raised a HIGH-severity finding that the PR title was missing the ! breaking-change suffix. This finding is valid — the title did need updating. However, the fix agent cannot modify PR metadata (title, labels, description) because sandbox policy blocks forge API mutations. The fix agent reported "API policy blocks mutations" but still consumed an iteration toward the bot cap of 5. When the review re-ran on the next commit, it re-raised the same finding, triggering another fix attempt that again failed. This contributed to the bot iteration cap being exhausted at iteration 5, after which all subsequent bot-triggered fix attempts were rejected with "Fix iteration N exceeds bot cap of 5. Escalating to human." The human had to intervene with /fs-fix 7 times across 19 days to complete the work. The PR title finding was one of several factors preventing convergence, but it is the clearest example of a structurally unfixable finding consuming iteration budget.
What could go better
The review agent currently treats all findings uniformly — a finding about code that needs changing and a finding about the PR title that needs changing both trigger the fix agent with equal weight. But the fix agent operates in a sandbox that can only modify repository files; it cannot call gh pr edit to change the title, add labels, or update the PR description. When the review raises a metadata-only finding, the fix agent runs, fails to address it, and the finding persists into the next review cycle — creating a guaranteed non-convergent loop that burns iteration budget. This is a structural mismatch between what the review agent requests and what the fix agent can deliver. I'm confident this is a real gap because the mechanism is deterministic: any finding requiring a forge mutation will always fail in the fix agent sandbox, and the review will always re-raise it. The adjacent issue #2418 proposes expanding the fix agent's capabilities to include gh pr edit, which is a complementary solution — but even if implemented, there will always be some class of findings (e.g., requiring human judgment on labels, milestone assignment) that the fix agent should not attempt.
Proposed change
In the review agent's finding output schema and classification logic (likely in skills/pr-review/SKILL.md or agents/review.md in fullsend-ai/agents), add a mutation_scope field to each finding with values like code (requires file changes) and metadata (requires forge API mutations such as PR title, labels, or description edits). The review agent should be instructed to classify each finding accordingly. Then, in the review-to-fix dispatch logic (likely in scripts/post-review.sh or the harness config harness/review.yaml), filter the findings passed to the fix agent to exclude metadata-scoped findings. Metadata findings should still be surfaced to the human in the review comment, clearly marked as requiring manual action (e.g., a separate section titled "Manual action required" or an annotation like [human-action]). This ensures metadata findings are visible but do not consume fix agent iteration budget.
Validation criteria
On the next 5 PRs where the review agent raises a finding about PR metadata (title formatting, label requirements, description completeness), the finding should appear in the review comment marked as requiring human action, and the fix agent should NOT attempt to address it. Fix agent iteration budget should only be consumed by code-level findings. Measure: zero fix agent iterations wasted on metadata-only findings across those 5 PRs.
Generated by retro agent from fullsend-ai/fullsend#6370
What happened
On PR #6370, the review agent raised a HIGH-severity finding that the PR title was missing the
!breaking-change suffix. This finding is valid — the title did need updating. However, the fix agent cannot modify PR metadata (title, labels, description) because sandbox policy blocks forge API mutations. The fix agent reported "API policy blocks mutations" but still consumed an iteration toward the bot cap of 5. When the review re-ran on the next commit, it re-raised the same finding, triggering another fix attempt that again failed. This contributed to the bot iteration cap being exhausted at iteration 5, after which all subsequent bot-triggered fix attempts were rejected with "Fix iteration N exceeds bot cap of 5. Escalating to human." The human had to intervene with/fs-fix7 times across 19 days to complete the work. The PR title finding was one of several factors preventing convergence, but it is the clearest example of a structurally unfixable finding consuming iteration budget.What could go better
The review agent currently treats all findings uniformly — a finding about code that needs changing and a finding about the PR title that needs changing both trigger the fix agent with equal weight. But the fix agent operates in a sandbox that can only modify repository files; it cannot call
gh pr editto change the title, add labels, or update the PR description. When the review raises a metadata-only finding, the fix agent runs, fails to address it, and the finding persists into the next review cycle — creating a guaranteed non-convergent loop that burns iteration budget. This is a structural mismatch between what the review agent requests and what the fix agent can deliver. I'm confident this is a real gap because the mechanism is deterministic: any finding requiring a forge mutation will always fail in the fix agent sandbox, and the review will always re-raise it. The adjacent issue #2418 proposes expanding the fix agent's capabilities to includegh pr edit, which is a complementary solution — but even if implemented, there will always be some class of findings (e.g., requiring human judgment on labels, milestone assignment) that the fix agent should not attempt.Proposed change
In the review agent's finding output schema and classification logic (likely in
skills/pr-review/SKILL.mdoragents/review.mdinfullsend-ai/agents), add amutation_scopefield to each finding with values likecode(requires file changes) andmetadata(requires forge API mutations such as PR title, labels, or description edits). The review agent should be instructed to classify each finding accordingly. Then, in the review-to-fix dispatch logic (likely inscripts/post-review.shor the harness configharness/review.yaml), filter the findings passed to the fix agent to excludemetadata-scoped findings. Metadata findings should still be surfaced to the human in the review comment, clearly marked as requiring manual action (e.g., a separate section titled "Manual action required" or an annotation like[human-action]). This ensures metadata findings are visible but do not consume fix agent iteration budget.Validation criteria
On the next 5 PRs where the review agent raises a finding about PR metadata (title formatting, label requirements, description completeness), the finding should appear in the review comment marked as requiring human action, and the fix agent should NOT attempt to address it. Fix agent iteration budget should only be consumed by code-level findings. Measure: zero fix agent iterations wasted on metadata-only findings across those 5 PRs.
Generated by retro agent from fullsend-ai/fullsend#6370