Skip to content

fix(llm): send providerPreferences to OpenRouter as the request's provider routing - #413

Merged
plombeer31 merged 1 commit into
mainfrom
fix/openrouter-provider-preferences
Sep 13, 2026
Merged

plombeer31 merged 1 commit into
mainfrom
fix/openrouter-provider-preferences

Conversation

@plombeer31

@plombeer31 plombeer31 commented Sep 13, 2026

Copy link
Copy Markdown
Collaborator

What

Before: llm.providers[].providerPreferences was parsed and validated, but nothing ever put it on the wire — the field's own doc comment said "carried through config, not yet read by any provider". An operator who pinned OpenRouter's upstream host ({"order": ["z-ai"], "allow_fallbacks": false}) kept being routed to whichever host OpenRouter picked. Only the same object under extraBody.provider was honoured.

After: an openrouter entry's providerPreferences is sent verbatim as the request body's provider object on every chat completion the entry makes — agent turns and memory sub-calls, streaming and unary — and on vision.describe. It is set before the extraBody merge, so an explicit extraBody.provider (the workaround people already use) keeps winning. With nothing configured, request bodies are byte-identical to before.

Why

Routing preferences are how an operator pins a host, forbids fallbacks, skips hosts that retain data (data_collection, ignore), or requires hosts that support every request parameter (require_parameters). Silently dropping them also hides real behaviour: while investigating why z-ai/glm-5.3-flash failed on one route, a "pinned" config kept succeeding only because its requests were served elsewhere.

Deliberately not sent by:

  • the pre-save key check — it probes the cheapest paid model, which a host pinned for the operator's model may not serve, and would report a good key as model_unavailable;
  • the contract probe — it is built from wizard state, which carries no entry passthroughs (extraBody is absent there too);
  • the model catalog fetch (GET /models, no body);
  • OpenRouter embeddings — a pin chosen for a chat model's hosts would strand an embedding model.

Only the openrouter factory forwards the field: provider is OpenRouter's, not part of the OpenAI schema, and no other kind documents it.

How it was verified

  • npm run lint clean
  • npx vitest run src/llm/provider src/config — 64 files / 1005 tests green
  • new openrouter-provider-routing.test.ts asserts the real request body: unary, streamed, a structured-output sub-call and vision carry provider; extraBody.provider wins on unary and streamed; with nothing configured there is no provider key and the body is unchanged
  • new register-built-in-providers.test.ts: built through the real factories, openrouter sends it; openai-compatible, qwen-openai-compatible, aimlapi and gemini do not
  • vacuity: dropping the builder line, the factory line or the vision spread each turns tests red; letting preferences override extraBody turns the precedence tests red
  • live, z-ai/glm-5.3-flash through the real openrouter factory: with {order:["z-ai"], allow_fallbacks:false} the response's provider was Z.AI 4 / 4; with {order:["streamlake"], allow_fallbacks:false} it was StreamLake 2 / 2; with no preferences it varied between hosts

Docs: AGENTS.md (provider registry section) and a README block next to strictTools.

Merge note: #417 also edits OpenAiProvider.complete (it moves the buildOpenAiChatBody call this PR extends into a closure) — whichever lands second needs a trivial rebase.

…vider routing

`llm.providers[].providerPreferences` has been parsed and validated
since the config fix that taught the loader about it, but nothing ever
put it on the wire. An operator who pinned OpenRouter's upstream host —
`{ "order": ["z-ai"], "allow_fallbacks": false }` — kept being routed
to whichever host OpenRouter picked, and structured-output sub-calls
the pinned host cannot serve kept "working" because they ran somewhere
else. The same object under `extraBody.provider` was honoured, which is
how the gap was found.

The field is now threaded exactly like `extraBody` and `strictTools`:
entry -> `openrouter` factory -> `OpenAiProvider` -> the request body,
where it becomes the `provider` object on every chat completion the
entry makes: turns and sub-calls, streaming and unary, and
`vision.describe` (routing is where `data_collection`, `only` and
`ignore` live, and those matter most for the operator's images).

Only the `openrouter` factory forwards it. `provider` is OpenRouter's
field, not the OpenAI schema's, and no other kind documents it.

It is set before the `extraBody` merge, so an explicit
`extraBody.provider` — the workaround people already use — still wins.
With no preferences configured the body is byte-identical to before.

Deliberately not sent by the pre-save key check (it probes the cheapest
paid model, which a host pinned for the operator's model may not serve,
and would report a good key as `model_unavailable`), the contract probe
(built from wizard state that carries no entry passthroughs, `extraBody`
included), the catalog fetch (`GET /models`, no body), or OpenRouter
embeddings (a pin chosen for a chat model's hosts would strand an
embedding model).
@plombeer31
plombeer31 merged commit 0ab912f into main Sep 13, 2026
2 checks passed
plombeer31 added a commit that referenced this pull request Sep 13, 2026
Integrates #411#421 for v0.6.1, including the two rc-only reconciliations (#413×#417, #414×#417).
Bartok9 pushed a commit to Bartok9/atomic-agent that referenced this pull request Sep 14, 2026
 structured-output fallback

AtomicBot-ai#417 moved complete()'s body build into a closure handed to
sendWithStructuredOutputFallback; thread this.providerPreferences through
that closure (after strictTools) so unary sub-calls keep OpenRouter
provider routing on both the first send and the prompt-only retry.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant