Skip to content

fix(acp): align reasoning effort capabilities - #946

Draft
szysammie wants to merge 2 commits into
benchflow-ai:mainfrom
szysammie:codex/acp-capability-mapping-audit
Draft

fix(acp): align reasoning effort capabilities#946
szysammie wants to merge 2 commits into
benchflow-ai:mainfrom
szysammie:codex/acp-capability-mapping-audit

Conversation

@szysammie

@szysammie szysammie commented Aug 5, 2026

Copy link
Copy Markdown

Fixes #945

Summary

  • Upgrade Codex ACP, Claude ACP, Gemini CLI, Pi ACP, OpenClaw, OpenCode, and MiMo CLI pins.
  • Dispatch reasoning effort through live ACP capabilities: reasoning_effort (Codex), effort (Claude), and thought_level (Pi).
  • Configure launch-time effort for OpenHands and Harvey LAB; leave agents without a confirmed effort interface fail-closed.

Root cause

BenchFlow’s prior Codex ACP pin and runtime assumed the legacy model[effort] protocol. Current Codex ACP exposes independent model and reasoning_effort options, so effort could not be correctly configured. The same review found launch-configured agents whose declared effort setting was not injected before process startup.

Companion manifest PR

  • chore(acp): sync agent runtime version pins agents#60 synchronizes the manifest-owned runtime version pins required by the manifest-parity gate.
  • The parity check will become green after that PR is merged into benchflow-ai/agents@main and this PR’s workflow is rerun.

Validation

  • Original regression suite: 302 passed, 2 skipped.
  • Focused ACP, registry, environment, and live manifest parity suite: 70 passed.
  • Agents contract suite: 21 passed.
  • ruff check src tests tools.
  • ruff format --check src tests tools.
  • ty check src.

Copy link
Copy Markdown
Author

Hi! Could a maintainer please approve the pending GitHub Actions workflows for this external contribution? Thank you.

@xdotli

xdotli commented Aug 6, 2026

Copy link
Copy Markdown
Member

sure!

@szysammie

Copy link
Copy Markdown
Author

Thanks for reviewing this PR. Because BenchFlow’s manifest-parity check depends on benchflow-ai/agents@main, I have opened the companion manifest update in benchflow-ai/agents#60.

Could a maintainer please review and approve the workflow runs for that PR first? If its checks pass, please merge it so the updated manifests are available on agents/main. Afterward, please approve and re-run the checks on this PR so #946 can validate against the merged agent manifests. Thank you!

@xdotli

xdotli commented Aug 8, 2026

Copy link
Copy Markdown
Member

approved!

Thanks for reviewing this PR. Because BenchFlow’s manifest-parity check depends on benchflow-ai/agents@main, I have opened the companion manifest update in benchflow-ai/agents#60.

Could a maintainer please review and approve the workflow runs for that PR first? If its checks pass, please merge it so the updated manifests are available on agents/main. Afterward, please approve and re-run the checks on this PR so #946 can validate against the merged agent manifests. Thank you!

@xdotli

xdotli commented Aug 8, 2026

Copy link
Copy Markdown
Member

and thanks for the contribution

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Audit ACP model and reasoning-effort capability mapping across agent adapters

2 participants