feat: automated Upstreamer port pipeline for @openrouter/agent - #19
Merged
Conversation
This repo is a port of @openrouter/agent but had no automation keeping it in sync: one commit (the manual 0.7.2 port), no CI, no drift signal. Upstream is now at 0.8.0, so the port is a minor version behind and missing HooksManager, versioned state serialization, and the #61-#68 fixes. Wires up Upstreamer (github.com/mountgram/upstreamer, MIT) so the port tracks upstream automatically, with gates strong enough to run unattended. - .upstreamer/upstreamer.md — the rewrite contract: scope (packages/agent only, MCP explicitly out), required public API as the parity floor, the TS->Python naming map, permanent idiomatic divergences (Pydantic/Zod, async-first, generator convention, looser typing), and the `openrouter` substrate pin. - .upstreamer/scripts/verify.sh — mechanical gate: ruff/mypy/pytest, required API presence, version consistency vs the ported upstream release. - .upstreamer/eval.md — parity gate, run in a fresh context against the TS reference. This is what catches a port that compiles but sits a version behind. - scripts/upstream — wrapper adapted for in-repo downstream: this repo IS the output, so no publish leg is needed. - .github/workflows/upstreamer-port.yaml — triggered by repository_dispatch from typescript-agent on npm publish (ports track releases, not every commit), weekly cron as backstop. Opens a PR; never pushes to main. - .github/workflows/ci.yaml — this repo had no CI at all, so the verifier's test calls only ever ran inside the sync job. State is seeded at 0.7.2 (adc7939) so the first run is an incremental 0.7.2 -> 0.8.0 delta rather than an unreviewable full regeneration. Failed gates deliberately leave state.yaml stale: the next run retries the same delta instead of skipping the gap, and the PR is labelled eval-failed. Credentials are OPENROUTER_API_KEY + OPENCODE_MODEL, read from .upstreamer/port.env locally (gitignored) or repo secret/variable in CI. See PORTING.md.
The required-API check fails by design until the port catches up to upstream 0.8.0. A permanently-red required check on every unrelated PR just teaches people to ignore CI, so report it in the job summary instead. It stays blocking where it matters: inside scripts/upstream, where it gates whether .upstreamer/state.yaml advances. Also make the verifier's version check say when it skipped for lack of an upstream checkout, rather than passing silently.
![perry-the-pr-reviewer[bot]](https://avatars.githubusercontent.com/in/3222961?s=60&v=4){kind=small}
There was a problem hiding this comment.
Perry's Review
Verdict: 💬 Comments / questions
Well-structured automation infrastructure with a sound design. The two-gate model (mechanical verify + fresh-context parity eval) with stale-state-on-failure is the correct safety property — a false PASS that advances state past a gap is the exact failure mode that silently drops a port a version behind, and this pipeline makes that hard. CI is green, the existing SDK code is untouched, and e2e tests skip cleanly without credentials.
Findings
Suggestion — full clone of upstream on every CI run (scripts/upstream:112)
git clone "$upstream_url" "$upstream_dir" does a full clone with all branches and tags. Since the upstream checkout is ephemeral in CI (fresh checkout each run), every sync does a full clone of typescript-agent. Adding --single-branch (or --depth=N if you only need recent history for diff resolution) would cut fetch time without affecting correctness. Not a blocker — just a CI cost consideration.
Nit — upstream clone assumes typescript-agent stays public
The clone passes no credentials. This works today because OpenRouterTeam/typescript-agent is public (confirmed), but if it's ever made private the pipeline breaks with a git auth error rather than a clear message. Worth a one-line note in PORTING.md under prerequisites.
Notes
- The required-API presence check via Python heredoc (
importlib.import_module+hasattr) is a clean approach — it verifies actual importability rather than grepping source. - The
perl -e 'alarm shift; exec @ARGV'timeout correctly propagates SIGALRM to the exec'd opencode process.pipefail+set -eensures the exit code propagates throughtee. upstreamer-changelog.mdis referenced by the contract and SKILL.md but already exists from the manual port — correctly not recreated here.- The
pyproject.tomlversion grep (grep -m1 '^version') assumes double-quoted version format, which is standard for pyproject.toml. Fine for now.
| else | ||
| target_commit="$(git -C "$upstream_dir" rev-parse origin/HEAD 2>/dev/null || git -C "$upstream_dir" rev-parse origin/main)" | ||
| fi | ||
| git -C "$upstream_dir" checkout -q --detach "$target_commit" |
There was a problem hiding this comment.
▶ Prompt for agents
Suggestion: git clone without --single-branch fetches the entire ref namespace (all branches + tags) on every CI run, since tmp/upstreamer/ is ephemeral. Adding --single-branch would halve fetch time without affecting correctness — the script only ever checks out origin/HEAD or a specific --ref.
Contributor
Author
There was a problem hiding this comment.
Fixed in follow-up PR #20 (this PR merged before the fix landed): --single-branch added, plus a fetch fallback so a --ref outside the default branch still resolves.
…ncode auth The wrapper only seeded auth.json when no openrouter credential existed. Anyone who had previously run 'opencode /connect' would have their port.env key silently ignored and the run billed to the old key — close to undebuggable. An explicitly provided key now always wins. Other providers in auth.json are preserved rather than clobbered, and a corrupt auth.json recovers instead of failing the run.
- Model id: openrouter/anthropic/claude-sonnet-latest does not exist on OpenRouter; opencode resolves the alias as ~anthropic/claude-sonnet-latest. Fixed in the contract frontmatter, port.env.example, PORTING.md, and the workflow comment. Set the OPENCODE_MODEL repo variable with the ~ form. - Committed opencode.json denying external_directory: without it, the model writing any scratch file outside the workspace (observed: /tmp/hookspec) trips an interactive permission ask that a headless `opencode run` can never answer — the run wedges silently until the job timeout. Deny makes opencode refuse the write so the agent course-corrects. - Timeouts: job timeout 300→150 min and inner wrapper alarm 14400→7200 s, so a wedged run dies with logs intact (inner alarm first) and stops blocking the upstreamer-port concurrency group for 5 hours. The full 0.7.2→0.8.0 jumbo delta completed locally in ~110 min; routine single-release deltas are far smaller. Verified locally: with these fixes the 0.7.2→0.8.0 python port ran end to end, flipped the verifier to PASS, and passed the parity eval on its second attempt (PASS WITH WARNINGS). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Generated by the Upstreamer pipeline in this PR, run locally end to end (scripts/upstream --ref @openrouter/agent@0.8.0). Both gates passed: mechanical verifier === PASS: 0 failures ===, parity eval PASS WITH WARNINGS on the second pass (.upstreamer/eval-report.md). Ports the 0.7.2 -> 0.8.0 delta: lifecycle hooks system (HooksManager + nine built-in hooks), versioned conversation-state serialization, awaiting_client_tools for unresolved manual tool calls, default-on allow_final_response with DEFAULT_FINAL_RESPONSE_DIRECTIVE, strict_final_response / empty-final-retry tolerance, and MCP tool-result source discrimination. state.yaml advances to 680bceb, so the first CI run after merge is a no-op until the next upstream release. First-pass eval FAILed on three real findings (Stop-hook force_resume was not a zero-cost retry, MCP branding surface missing, thin hooks test coverage); all three fixed and re-verified in the second pass — see the eval report for the traced evidence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
Two commits added on top of the machinery:
Review the two commits separately: the first is machinery, the second should be read as a port (behavioral parity against the TS reference, especially mixed approval/HITL turn ordering and hooks semantics). Remaining before this goes live: 🤖 Generated with Claude Code |
OPENCODE_MODEL repo variable updated to openrouter/~anthropic/claude-opus-latest to match; contract frontmatter, port.env.example, PORTING.md, and the workflow comment all now reference the same alias. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LukasParke
marked this pull request as ready for review
July 28, 2026 21:27
Two changes so the pipeline runs entirely on the native GITHUB_TOKEN: - Cron and blank manual dispatches now resolve the latest published @openrouter/agent version from the public npm registry and port that release tag, instead of porting upstream main HEAD. This makes the weekly cron fully equivalent to the repository_dispatch fast path (same tag either way), so the cross-repo PAT dispatch in typescript-agent becomes a latency optimization, not a requirement — and it also fixes cron runs violating the 'ports track releases' principle by syncing to mid-flight upstream commits. - PRs opened with the native GITHUB_TOKEN don't trigger other workflows (GitHub's recursion guard), which was the standard argument for a PAT here. workflow_dispatch is exempt from that guard, so the port job now explicitly dispatches ci.yaml at the upstreamer/sync branch after opening the PR. Requires actions: write, added to the permissions block. No PAT or external secret is referenced anywhere in this repo now; the only secrets are OPENROUTER_API_KEY (inference) and the built-in token. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LukasParke
added a commit
that referenced
this pull request
Jul 29, 2026
* perf(upstream): clone upstream --single-branch, with a fetch fallback for out-of-branch refs Addresses the review suggestion on #19: the CI clone is ephemeral, so fetching the full ref namespace every run roughly doubles clone time for nothing — the script targets origin/HEAD or a release tag, both reachable from the default branch. Goes one step beyond the suggestion: with --single-branch, a --ref that is NOT reachable from the default branch (e.g. a manual run against a feature branch) would no longer resolve, so rev-parse now falls back to an explicit fetch of that ref before failing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(upstream): guard ref arguments against option injection Same hardening as go-agent#2 review nit: '--' terminator on the fallback fetch and --end-of-options on rev-parse so a manual --ref value starting with '-' cannot be parsed as an option. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
LukasParke
added a commit
to OpenRouterTeam/typescript-agent
that referenced
this pull request
Jul 30, 2026
* ci(release): dispatch the Python and Go ports on publish (HOP C) The Python and Go ports of @openrouter/agent track this repo as their reference spec, but nothing told them when a version shipped. Both were generated by hand against 0.7.2 and have sat a minor version behind since — missing HooksManager, versioned state serialization, and the #61-#68 fixes. Adds a HOP C dispatch beside the existing HOP B monorepo dispatch. On a real publish, python-agent and go-agent each receive openrouter-agent-published and open a PR porting the delta. Their pipelines gate the result on a mechanical verifier plus a behavioral parity eval before advancing sync state, so a bad port cannot land silently. Details worth noting: - Sends the release TAG (@openrouter/agent@X.Y.Z), not a branch, so a port reproduces the exact published tree rather than whatever main drifted to. - continue-on-error: the packages are already on npm when this runs, so a failed dispatch must not turn a successful release red. It warns instead, and names the manual recovery path. - !cancelled() so a failed HOP B dispatch doesn't also skip the ports. - Partial failure is tolerated: one port failing still dispatches the other. Reuses the same GH_TOKEN PAT as HOP B, which additionally needs contents:write on OpenRouterTeam/python-agent and OpenRouterTeam/go-agent. Also documents the full publish fan-out in the changeset-versioning skill, since a breaking callModel change now produces port PRs in two other repos. Companion PRs: OpenRouterTeam/python-agent#19, OpenRouterTeam/go-agent#1 * fix(release): address review findings on HOP C dispatch - Use -f (--raw-field) for all dispatch payload fields: -F treats values starting with @ (the release tag) as filenames, which made every port dispatch fail before the request was sent - Push release tags on the manual mode=publish path so the dispatched ref actually exists on the remote - Restore set -e; the gh api call sits in an if-condition and is already -e-exempt, so the rest of the script stays strict - Pass source_run_url via env like VERSION instead of inline ${{ }} Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(release): fall back to commit SHA when the release tag is not on origin Address review findings from cortex and Devin on the HOP C dispatch: - Make the manual-publish tag push non-fatal. It runs after packages are already on npm, so a rejected push (tag protection, or a re-run where the tag exists at a different commit) turned a successful publish red and, because the HOP B/C steps carry no status guard, skipped both dispatches. - Verify the tag is on origin before dispatching it, falling back to the run's commit SHA. Previously the manual path could dispatch a ref the port repos cannot resolve, making them fail on checkout instead of degrading. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: LukasParke <luke@openrouter.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wires up automated porting of
@openrouter/agentinto this repo using Upstreamer (MIT) — the same tool that produced the existingupstreamer-changelog.md, but running on a schedule with real gates instead of by hand.Why
This repo has one commit (the manual 0.7.2 port), no CI, and no drift signal. Upstream is now at 0.8.0, so the port is a minor version behind and missing:
HooksManagerlifecycle system (upstream feat: concurrent tool execution via anyio.create_task_group #7, #67) — the 0.8.0 headline featureserialize_conversation_state/deserialize_conversation_state/CONVERSATION_STATE_VERSION(#66)awaiting_client_toolsmanual-tool persistence (#64), empty-final tolerance (#63), string-input resume (#61),contextSchemainference (#65), default final-answer directive (#68)1. Fill in credentials. Two values, same names locally and in CI:
OPENROUTER_API_KEY.upstreamer/port.envOPENCODE_MODEL.upstreamer/port.envcp .upstreamer/port.env.example .upstreamer/port.env # gitignored.upstreamer/port.env.exampledocuments both. The wrapper writes the key into~/.local/share/opencode/auth.json, so headless runs work without the interactiveopencode /connectflow.OPENCODE_MODELoverrides the contract'smodel:field, so you can switch models without a code change — recommend a strong coding model, this is a load-bearing SDK port behind a strict eval.2. Add HOP C to
typescript-agent. This workflow listens forrepository_dispatchtypeopenrouter-agent-published, which nothing sends yet. It needs a dispatch step intypescript-agent'spublish.yaml, right next to the existing HOP B dispatch toopenrouter-web. Until then the weekly cron and manual dispatch cover it.What's here
.upstreamer/upstreamer.md.upstreamer/scripts/verify.sh.upstreamer/eval.md.upstreamer/state.yamladc7939)scripts/upstream.github/workflows/upstreamer-port.yaml.github/workflows/ci.yamlPORTING.mdDesign decisions worth reviewing
This repo is the downstream. Upstreamer normally writes generated output into
codebases/<name>/downstream/inside the upstreamer repo. Putting the machinery here instead means output lands in place and there's no publish/rsync leg that could clobber CI or release config. The converter skill has an explicit do-not-touch list for repo-owned files.Ports track releases, not commits. Trigger is npm publish, not every push to upstream main. Porting mid-flight commits creates churn and reviewer fatigue; at a publish event the upstream HEAD is also the release SHA, so change detection lands on a clean boundary.
Seeded state. Without seeding, run #1 regenerates the whole SDK — a huge unreviewable diff. Seeded at 0.7.2, run #1 is an incremental 0.7.2→0.8.0 delta.
Failure leaves state stale, on purpose. If either gate fails,
state.yamlis not advanced,.upstreamer/eval-report.mdexplains why, and the PR is titledEVAL FAILED — do not mergewith aneval-failedlabel. The next run retries the same delta rather than skipping the gap. This is the property that makes unattended runs safe; a false PASS is worse than no eval.The contract is the product. When a port is wrong, fix
.upstreamer/upstreamer.md, not just the generated code — a code-only fix gets re-broken next sync.Verified
Ran the full pipeline locally against
@openrouter/agent@0.7.2:.upstreamer/port.envis gitignored whileport.env.examplestays trackedTwo bugs found and fixed while testing: the seed was an annotated tag SHA rather than a commit SHA (would have forced a full regeneration every run), and the prompt heredoc tripped a bash 3.2 parsing bug on macOS.
Expectations
The first few runs will be contract-tuning, not clean automation. Budget review time. Also note two publishing blockers this pipeline doesn't solve: PyPI
openrouter-agentis currently owned by an unrelated third-party package, and this repo is private so it can't be installed as documented.Opened as a draft — needs the credentials above before the workflow can do anything.