fix(llm): retry a sub-call without response_format when the endpoint refuses structured outputs - #417
Merged
Conversation
…refuses structured outputs Memory sub-calls (query rewriter, link generator, vote runner, distill) send OpenAI Structured Outputs. Endpoints without the feature refuse the request: OpenRouter pinned to Z.AI's endpoint for z-ai/glm-5.3-flash answers 404 "No endpoints found" because its "Filter by Parameters" routing step drops that endpoint, and vendor APIs without json_schema answer 400/422 naming the field. The sub-call then failed on every run (the rewriter failed 18 of 18 times in a live session), and because every cloud OpenAiHttpError classifies as transport, each refusal also advanced the provider fallback chain. OpenAiProvider.complete now recognises that refusal narrowly and sends the same body once more without response_format. The prompts still ask for their text formats and every parser reads them when the reply is not JSON, so only decode enforcement is lost. The retry happens inside the provider, so a handled refusal never reaches runWithFallback. The provider and model pair is remembered for the process once the stripped send is accepted, later sub-calls skip the field up front, and the first time logs one warn line. Not treated as refusals: 401/402/403/429/5xx, timeouts, network failures, size rejections, and the "messages must contain the word json" 400, which is a prompt problem on an endpoint that does support the feature. Streamed requests never carry response_format and are unchanged.
This was referenced Sep 13, 2026
Bartok9
pushed a commit
to Bartok9/atomic-agent
that referenced
this pull request
Sep 14, 2026
structured-output fallback AtomicBot-ai#417 moved complete()'s body build into a closure handed to sendWithStructuredOutputFallback; thread this.providerPreferences through that closure (after strictTools) so unary sub-calls keep OpenRouter provider routing on both the first send and the prompt-only retry.
Bartok9
pushed a commit
to Bartok9/atomic-agent
that referenced
this pull request
Sep 14, 2026
…mpt-only retry test AtomicBot-ai#414 appends a JSON instruction to the prompt of any body that carries response_format; AtomicBot-ai#417's prompt-only retry drops response_format, so its prompt is the caller's own. The retry test asserted the two bodies were identical apart from response_format; assert the JSON mention on the first send and the original prompt on the retry instead. No runtime change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Before: cloud memory sub-calls (query rewriter, link generator, vote runner, consolidation distill) always sent OpenAI Structured Outputs (
response_format: {type: "json_schema"}). An endpoint with no structured-output support refused every one of them — OpenRouter pinned to Z.AI's own endpoint forz-ai/glm-5.3-flashanswers404 No endpoints foundafter its "Filter by Parameters" routing step; direct vendor APIs withoutjson_schemaanswer 400. The sub-call failed forever on that provider, and because every cloudOpenAiHttpErrorclassifiestransport, each refusal also ran through the provider fallback chain.After: when a request carrying
responseFormatis refused as a structured-output refusal,OpenAiProvider.completeretries it once on the same provider withoutresponse_format, returns that result, and remembers the (provider id, model) pair for the rest of the process, so later sub-calls to it skipresponse_formatup front. Onewarnline the first time. A handled refusal never reachesrunWithFallback, so it never advances the chain or trips its breaker. Streaming main-turn requests never carryresponseFormatand are untouched.Why
The sub-call parsers never needed Structured Outputs to get an answer: all four (rewriter, link, vote, distill) try JSON first and fall back to the text format their prompt asks for. Losing decode-time enforcement degrades an answer; a permanent 404 loses it.
The detector is deliberately narrow — an
OpenAiHttpErrorthat is not our own timeout, and:No endpoints foundtogether withrequested parameters, a feature word (response_format/json_schema/json_object/ structured outputs), or a "Filter by Parameters" funnel step that actually dropped endpoints (the step appears in every OpenRouter funnel, so its count is checked when readable);must contain the word 'json'(a prompt problem on an endpoint that does support structured outputs; fix(memory): structured-output sub-calls that OpenAI strict mode and Qwen reject #414 fixes the prompt) and request-size rejections.401 / 402 / 403 / 429 / 5xx, network failures and untyped errors are never refusals. The pair is remembered only after the retry succeeds; a retry that fails propagates its own error and remembers nothing.
How it was verified
npm run lintcleannpx vitest run src/llm/provider/openai src/runtime/llm-fallback-seam-structured-output.test.ts src/runtime/llm-fallback-seam.test.ts src/llm/fallback— 26 files / 435 tests greenresponse_formatand returns its success; at the seamadvanceFromis never called and the local link is never reached; the next call skipsresponse_formatwith no failed round trip; memory is per provider id and per model; invalid-key, context-length and json-word 400s propagate after a single request; no retry withoutresponseFormator withtools; a failed retry remembers nothing;completeStreamnever retries; a control test shows the same 404 withoutresponseFormatstill advances the chainfalseturns 11 of the 34 new tests red across all three files; removing themust contain the word jsonexclusion turns 2 red (that 400 must not be read as a refusal)z-ai/glm-5.3-flashpinned to Z.AI's endpoint, same scripted 8-turn session as the baseline:failed18ok11,timeout7,failed0Merge note: #413 also edits
OpenAiProvider.complete(it threadsproviderPreferencesinto thebuildOpenAiChatBodycall this PR moves into a closure) — whichever lands second needs a trivial rebase. #414's JSON instruction is only added whenresponse_formatis attached, so the stripped retry sends the prompt unchanged.The 7 remaining rewriter timeouts are the 3 s rewriter budget on a thinking model — before, the call failed in ~74 ms; now it actually waits for GLM. #416 raises that budget to 10 s and runs the rewriter once per turn.