fix(llm): retry a sub-call without response_format when the endpoint refuses structured outputs

AGENTS.md

@@ -658,6 +658,8 @@ There is currently no dedicated workspace-memory, retrieval, embeddings, or reso
 
 A three-channel cross-session memory subsystem lives in [src/memory/](src/memory/) and exposes itself to the agent via six tools in [src/tools/memory/](src/tools/memory/). The full description is in [MEMORY.md](MEMORY.md); this section is the engineering summary. The v2 roadmap (paths B+C+E+P: reactive graph, periodic consolidation, vote curation, procedure templates) lives in [MEMORY_FABRIC_V2.md](MEMORY_FABRIC_V2.md) and rolls out in strict-gated phases. Plan-level deviation from doc §9 invariant 2: v2 pays the stable-prefix KV-cache invalidation **twice** (once when `### lessons` lands in phase 5, once when `### procedures` lands in phase 7b) instead of the doc's intended single combined release — the strict-gates rollout requires evaluation windows between the two prefix-touching phases.
 
+Cloud sub-runners (query rewriter, link generator, vote runner, distill) ask for OpenAI Structured Outputs but never depend on them: an endpoint that refuses `response_format` gets the same request once more without it and is remembered for the rest of the run, and every sub-runner parser reads the prompt's text format whenever the reply is not JSON — see §"Structured-output refusal (sub-calls)".
+
 ### Memory-v2 phase 1B — hybrid FTS5 + embedding recall (opt-in)
 
 Lives in [src/memory/embeddings/](src/memory/embeddings/) and is **off in config until the operator enables it from the TUI Models tab** (download + start embedding model). When turned on, it adds a second `llama-server` process dedicated to `/embedding` requests and blends BM25 hits with cosine similarity over a `memory_embeddings` table (schema v5).
@@ -2262,7 +2264,7 @@ Deliberately out of scope: an opt-in whole-disk / drive index (the issue sketche
 
 ## LLM reliability policy
 
-Three narrow retry layers sit between the agent loop and the model server, plus one single-shot request repair (§"Credit-limit repair (HTTP 402)"). All are deliberately bounded and never replay already-executed tool calls, and none of them ever replays output a caller has already seen:
+Three narrow retry layers sit between the agent loop and the model server, plus two single-shot request repairs (§"Credit-limit repair (HTTP 402)" and §"Structured-output refusal (sub-calls)"). All are deliberately bounded and never replay already-executed tool calls, and none of them ever replays output a caller has already seen:
 
 1. **Parser retry (step-executor).** If the first `parseToolCall` on a completion throws, the executor calls the unary `llmComplete` exactly once more with the same prompt/slot and re-parses. A `parse_retry` event is emitted for observability. If the second attempt also fails, the original error (with a raw-output preview) is thrown. The streaming path always falls back to unary for the retry so partial SSE deltas are not double-emitted.
 2. **Transport retry (LlamaServerClient).** `complete()` and the initial pre-body fetch of `completeStream()` are wrapped in a bounded retry governed by `llama.completionRetries` (default 3) and `llama.completionRetryBackoffMs` (default 150ms, exponential with ±20% jitter). Retries fire **only** for network errors (`LlamaServerError.status === null`) and HTTP 5xx. Grammar/validation 4xx and abort signals short-circuit immediately. Once the SSE body starts streaming, no further retries happen — the conversation state on the server is considered indeterminate.
@@ -2287,6 +2289,16 @@ Not a retry layer — one targeted repair of one specific, self-describing refus
 
 Pinned by [parse-credit-limit.test.ts](src/llm/provider/openai/parse-credit-limit.test.ts), [plan-credit-limit-retry.test.ts](src/llm/provider/openai/plan-credit-limit-retry.test.ts) and the `credit-limit (402) recovery` block in [openai-http.test.ts](src/llm/provider/openai/openai-http.test.ts).
 
+### Structured-output refusal (sub-calls)
+
+Also a repair, not a retry layer. The memory sub-runners (query rewriter, link generator, vote runner, distill) send OpenAI Structured Outputs (`response_format: { type: "json_schema" }`), and some endpoints have none: OpenRouter pinned to one vendor's endpoint answers `404 No endpoints found` with a routing funnel whose `Filter by Parameters` step dropped that endpoint, and vendor APIs without `json_schema` answer 400/422 naming the field. Every cloud `OpenAiHttpError` classifies `transport`, so that refusal used to fail the sub-call on every run **and** advance the fallback chain each time — onto a local link that may not even be running.
+
+- **Detection is narrow and fails closed.** [structured-output-refusal.ts](src/llm/provider/openai/structured-output-refusal.ts) accepts only a 404 `No endpoints found` with parameter-filter evidence (the `requested parameters` sentence, a structured-output field name, or a `Filter by Parameters` step whose count dropped), and a 400/422 naming `response_format`, `json_schema`, `json_object` or `structured output(s)`. Excluded: size rejections, and the `'messages' must contain the word 'json'` 400 — a prompt problem on an endpoint that does support the feature, which stripping would paper over with a downgrade for the rest of the run. Never 401/402/403/429/5xx, our own timeout, or a network failure.
+- **One send without the field, below the chain.** [structured-output-fallback.ts](src/llm/provider/openai/structured-output-fallback.ts) runs inside `OpenAiProvider.complete` and re-sends the same body minus `response_format`, so a handled refusal never reaches `runWithFallback`, never advances it and never trips its breaker. The prompts still ask for their text formats (`<rewritten_query>`, `LINK`, `UPVOTE`, `LESSON`) and each parser reads them whenever the reply is not JSON, so the answer survives and only decode enforcement is lost. It applies only when dropping `responseFormat` changes the wire: a request with `tools` never sends it, and a `response_format` set through `extraBody` is the operator's and stays. Streamed requests never carry `responseFormat` and are untouched.
+- **Remembered once confirmed.** The (provider id, model) pair is recorded for the life of the process only when the stripped send is accepted — a retry that fails too propagates its own error and records nothing — and later sub-calls to that pair skip `response_format` up front, with no failed round trip. The record lives outside the provider instance, so it survives a provider rebuild on config save. The first record logs one `warn`: `llm: "<id>" does not support structured outputs (response_format) for <model>; memory sub-calls fall back to prompt-only output for the rest of this run.`
+
+Pinned by [structured-output-refusal.test.ts](src/llm/provider/openai/structured-output-refusal.test.ts), [structured-output-fallback.test.ts](src/llm/provider/openai/structured-output-fallback.test.ts) and [llm-fallback-seam-structured-output.test.ts](src/runtime/llm-fallback-seam-structured-output.test.ts).
+
 ### Failure taxonomy
 
 Every terminal failure the agent loop surfaces is normalised into a canonical `LlmFailureCategory` before `step_error` / `loop_failed` fire. The classes live in [src/llm/reliability/](src/llm/reliability/) and carry specialised fields for postmortem use.

src/llm/provider/openai/openai-provider.ts

@@ -42,6 +42,7 @@ import {
   adaptQwenTaggedToolResponse,
 } from "./qwen-tagged-tool-response-adapter.js";
 import type { CreditLimitLogger } from "./plan-credit-limit-retry.js";
+import { sendWithStructuredOutputFallback } from "./structured-output-fallback.js";
 
 export interface OpenAiProviderOptions {
   id: string;
@@ -148,19 +149,30 @@ export class OpenAiProvider implements LlmProvider {
   }
 
   async complete(request: CompletionRequest): Promise<CompletionResult> {
-    const body = buildOpenAiChatBody(
-      request,
-      this.defaultChatModel,
-      false,
-      this.extraBody,
-      this.maxOutputTokens,
-      this.strictTools,
-    );
-    const json = await openAiPostJson(
-      this.http,
-      `${this.apiPathPrefix}/chat/completions`,
-      body,
+    // Unary only: sub-calls carry `response_format`, streamed turns never do.
+    const json = await sendWithStructuredOutputFallback(
+      {
+        providerId: this.id,
+        model: this.defaultChatModel,
+        logger: this.http.logger,
+      },
       request,
+      (req) =>
+        buildOpenAiChatBody(
+          req,
+          this.defaultChatModel,
+          false,
+          this.extraBody,
+          this.maxOutputTokens,
+          this.strictTools,
+        ),
+      (body) =>
+        openAiPostJson(
+          this.http,
+          `${this.apiPathPrefix}/chat/completions`,
+          body,
+          request,
+        ),
     );
     const adapted =
       this.taggedToolCompatibility === "qwen"

src/llm/provider/openai/structured-output-fallback.test.ts

@@ -0,0 +1,192 @@
+import { describe, expect, it, vi } from "vitest";
+
+import type {
+  CompletionRequest,
+  ResponseFormatJsonSchema,
+} from "../completion-types.js";
+import { OpenAiHttpError } from "./openai-http.js";
+import { OpenAiProvider } from "./openai-provider.js";
+import { OPENROUTER_PARAMETER_REFUSAL_BODY } from "./structured-output-refusal.fixture.js";
+
+const responseFormat: ResponseFormatJsonSchema = {
+  name: "query_rewriter",
+  schema: {
+    type: "object",
+    properties: { rewritten_query: { type: "string" } },
+    required: ["rewritten_query"],
+    additionalProperties: false,
+  },
+};
+
+const ENVELOPE = "<rewritten_query>deploy the kastel app</rewritten_query>";
+
+type Reply = () => Response;
+
+const ok =
+  (content: string): Reply =>
+  () =>
+    new Response(
+      JSON.stringify({
+        model: "z-ai/glm-5.3-flash",
+        choices: [{ message: { role: "assistant", content }, finish_reason: "stop" }],
+        usage: { prompt_tokens: 1, completion_tokens: 2, total_tokens: 3 },
+      }),
+      { status: 200, headers: { "content-type": "application/json" } },
+    );
+
+const status =
+  (code: number, body: string): Reply =>
+  () =>
+    new Response(body, { status: code });
+
+const refusal = status(404, OPENROUTER_PARAMETER_REFUSAL_BODY);
+
+/**
+ * Scripted fetch that records every request body. A request beyond the
+ * script answers 418 — non-retryable, so an unexpected send fails the
+ * test loudly instead of being absorbed by the retry budget.
+ */
+function scriptedFetch(replies: Reply[]) {
+  const bodies: Array<Record<string, unknown>> = [];
+  const impl = vi.fn(async (_url: string, init?: RequestInit) => {
+    bodies.push(JSON.parse(String(init?.body)) as Record<string, unknown>);
+    const next = replies.shift();
+    return next ? next() : new Response("unexpected extra request", { status: 418 });
+  });
+  return { fetchImpl: impl as unknown as typeof fetch, bodies, replies };
+}
+
+let seq = 0;
+function makeProvider(
+  fetchImpl: typeof fetch,
+  opts: { id?: string; model?: string } = {},
+) {
+  const warn = vi.fn();
+  const provider = new OpenAiProvider({
+    id: opts.id ?? `openrouter-${++seq}`,
+    baseUrl: "https://openrouter.example",
+    apiKey: "k",
+    defaultChatModel: opts.model ?? "z-ai/glm-5.3-flash",
+    fetchImpl,
+    logger: { warn },
+  });
+  return { provider, warn };
+}
+
+const subcall: CompletionRequest = { prompt: "rewrite", maxTokens: 256, responseFormat };
+
+describe("OpenAiProvider.complete — structured-output refusal fallback", () => {
+  it("retries once without response_format and returns that answer", async () => {
+    const net = scriptedFetch([refusal, ok(ENVELOPE)]);
+    const { provider, warn } = makeProvider(net.fetchImpl);
+
+    const result = await provider.complete(subcall);
+
+    expect(result.content).toBe(ENVELOPE);
+    expect(net.bodies).toHaveLength(2);
+    expect(net.bodies[0]).toHaveProperty("response_format.type", "json_schema");
+    expect(net.bodies[1]).not.toHaveProperty("response_format");
+    // Only the field is dropped: the retry is otherwise the same request.
+    const { response_format: _sent, ...firstWithout } = net.bodies[0]!;
+    expect(net.bodies[1]).toEqual(firstWithout);
+    expect(warn).toHaveBeenCalledTimes(1);
+    expect(warn.mock.calls[0]![0]).toContain(provider.id);
+    expect(warn.mock.calls[0]![0]).toContain("does not support structured outputs");
+  });
+
+  it("remembers the refusal: the next sub-call skips response_format with no failed round trip", async () => {
+    const net = scriptedFetch([refusal, ok(ENVELOPE), ok(ENVELOPE)]);
+    const { provider, warn } = makeProvider(net.fetchImpl);
+
+    await provider.complete(subcall);
+    await provider.complete(subcall);
+
+    expect(net.bodies).toHaveLength(3);
+    expect(net.bodies[2]).not.toHaveProperty("response_format");
+    expect(warn).toHaveBeenCalledTimes(1);
+  });
+
+  it("keys the memory by provider id and model, and outlives the provider instance", async () => {
+    const id = `openrouter-${++seq}`;
+    const first = scriptedFetch([refusal, ok(ENVELOPE)]);
+    await makeProvider(first.fetchImpl, { id }).provider.complete(subcall);
+
+    const rebuilt = scriptedFetch([ok(ENVELOPE)]);
+    await makeProvider(rebuilt.fetchImpl, { id }).provider.complete(subcall);
+    expect(rebuilt.bodies[0]).not.toHaveProperty("response_format");
+
+    const otherProvider = scriptedFetch([ok(ENVELOPE)]);
+    await makeProvider(otherProvider.fetchImpl).provider.complete(subcall);
+    expect(otherProvider.bodies[0]).toHaveProperty("response_format");
+
+    const otherModel = scriptedFetch([ok(ENVELOPE)]);
+    await makeProvider(otherModel.fetchImpl, {
+      id,
+      model: "openai/gpt-5.4-mini",
+    }).provider.complete(subcall);
+    expect(otherModel.bodies[0]).toHaveProperty("response_format");
+  });
+
+  it.each([
+    ["invalid key", "API key not valid. Please pass a valid API key."],
+    ["context length", "This model's maximum context length is 32768 tokens."],
+    [
+      "json word",
+      "'messages' must contain the word 'json' in some form, to use 'response_format' of type 'json_object'.",
+    ],
+  ])("propagates a non-refusal 400 (%s) unchanged, without a retry", async (_label, message) => {
+    const body = JSON.stringify({ error: { message } });
+    const net = scriptedFetch([status(400, body), ok(ENVELOPE)]);
+    const { provider, warn } = makeProvider(net.fetchImpl);
+
+    const err = await provider.complete(subcall).catch((e: unknown) => e);
+
+    expect(err).toBeInstanceOf(OpenAiHttpError);
+    expect((err as OpenAiHttpError).status).toBe(400);
+    expect((err as OpenAiHttpError).message).toBe(`openai provider 400: ${body}`);
+    expect(net.bodies).toHaveLength(1);
+    expect(warn).not.toHaveBeenCalled();
+  });
+
+  it("has no retry path for a request without responseFormat", async () => {
+    const net = scriptedFetch([refusal, ok(ENVELOPE)]);
+    const { provider } = makeProvider(net.fetchImpl);
+
+    await expect(provider.complete({ prompt: "turn" })).rejects.toMatchObject({ status: 404 });
+    expect(net.bodies).toHaveLength(1);
+  });
+
+  it("has no retry path when tools kept response_format off the wire", async () => {
+    const net = scriptedFetch([refusal, ok(ENVELOPE)]);
+    const { provider } = makeProvider(net.fetchImpl);
+    const tools: NonNullable<CompletionRequest["tools"]> = [
+      { type: "function", function: { name: "emit", parameters: { type: "object" } } },
+    ];
+
+    await expect(provider.complete({ ...subcall, tools })).rejects.toMatchObject({ status: 404 });
+    expect(net.bodies).toHaveLength(1);
+    expect(net.bodies[0]).not.toHaveProperty("response_format");
+  });
+
+  it("propagates a failed retry's own error and remembers nothing", async () => {
+    const other = JSON.stringify({ error: { message: "Provider returned error" } });
+    const net = scriptedFetch([refusal, status(400, other), ok(ENVELOPE)]);
+    const { provider, warn } = makeProvider(net.fetchImpl);
+
+    await expect(provider.complete(subcall)).rejects.toMatchObject({ status: 400 });
+    await provider.complete(subcall);
+
+    expect(net.bodies).toHaveLength(3);
+    expect(net.bodies[2]).toHaveProperty("response_format");
+    expect(warn).not.toHaveBeenCalled();
+  });
+
+  it("leaves streaming alone: a refusal on completeStream is not retried", async () => {
+    const net = scriptedFetch([refusal, ok(ENVELOPE)]);
+    const { provider } = makeProvider(net.fetchImpl);
+
+    const stream = provider.completeStream(subcall);
+    await expect(stream.next()).rejects.toMatchObject({ status: 404 });
+    expect(net.bodies).toHaveLength(1);
+  });
+});