Skip to content

Code agent should verify new CLI mutation paths respect existing behavioral contracts #1221

Description

@fullsend-ai-retro

What happened

On PR #7087, the code agent added a --gitlab-url flag to repos install (code run 34174528067, cost $5.46). The initial implementation correctly handled flag registration, URL validation, manifest persistence, and documentation — but missed three existing behavioral contracts in the same command:

  1. --dry-run contract: The new mutation path wrote to the manifest via repos.SetDefault() without checking opts.dryRun, violating the command's preview-only guarantee. This produced 3 review findings across 2 iterations (dry-run contract violation in Review 1, dry-run preview incomplete in bootstrap path in Review 2, test adequacy for dry-run in Review 2).

  2. --forge cross-validation: No check prevented the contradictory --forge=github --gitlab-url=... combination, which silently persisted a GitLab URL while adding repos to the GitHub section. This was flagged as an architectural-coherence finding in Review 6.

  3. Factory ordering: newForgeClientFactory captured manifest.GitLab.URL at creation time, but the --gitlab-url handler set it ~130 lines later, causing converge to target the wrong server. This was a medium finding in Review 4.

The triage summary recommended 5 implementation steps but did not mention dry-run handling, forge cross-validation, or factory ordering — all of which are established patterns in the same function.

What could go better

If the code agent had systematically checked what existing flags and behavioral contracts the target command implements before writing new code, it could have avoided 4+ review findings and 2–3 fix iterations (~$15–20 in review/fix costs). The runReposInstall function is ~200 lines with explicit opts.dryRun guards, forge-inference logic, and factory creation patterns. The code agent read this function (it inserted code into it) but didn't treat the existing patterns as constraints on the new code.

Confidence: high that the code agent read the surrounding code. Medium confidence that explicit guidance would change behavior — the code agent may have deprioritized these patterns in favor of the core feature. But the triage summary's omission of these contracts suggests the code agent had insufficient context about what patterns matter in CLI commands.

Proposed change

Add guidance to the code agent definition (agents/code.md in fullsend-ai/agents) for CLI modification tasks. When the code agent adds new mutation paths, flag handlers, or write operations to an existing CLI command, it should:

  1. Scan for behavioral contracts — Check whether the command implements --dry-run, --verbose, --json, or similar cross-cutting flags, and ensure the new code path respects each one.
  2. Check cross-flag consistency — If the new flag is specific to one forge/platform/mode, verify there is no contradictory combination with existing flags (e.g., --gitlab-url + --forge=github).
  3. Verify initialization ordering — When the new code sets values used by factories, clients, or closures created earlier in the function, check that the assignment happens before the consumer is created.

This could be added as a "CLI changes" subsection in the code agent's implementation guidance, or as a repo-specific AGENTS.md entry in fullsend-ai/fullsend if the guidance is too repo-specific for the general code agent.

Validation criteria

The next 5 code agent PRs in fullsend-ai/fullsend that add new flags or mutation paths to existing CLI commands should not trigger review findings about missing --dry-run handling, contradictory flag combinations, or initialization ordering issues. Specifically, the initial code commit should include dry-run guards for any new write path in commands that support --dry-run.


Generated by retro agent from fullsend-ai/fullsend#7087

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions