perf(memory): run the query rewriter once per turn

AGENTS.md

@@ -1182,7 +1182,7 @@ All keys under `memory.*` in the user config and [src/config/config-schema.ts](s
 - `memory.index.{enabled, limit, previewChars, maxTokens}`
 - `paths.memoryDbFile` — resolved to `<stateDir>/memory.sqlite`.
 
-**Sub-call timeouts scale with provider latency.** A default tuned against a local `llama-server` does not carry over to hosted reasoning models. Before config v65 reflection had 10 s and the link-generator 8 s; measured on OpenRouter, reflection takes a median 13.9–16.3 s on glm-5.3-flash / qwen3.6-plus / kimi-k2.6 (37.5 s worst case in a live session) and kimi's link-generator 37.8 s, so 6 of 8 live reflections on glm-5.3-flash timed out and wrote nothing. v65 raises both to 60 s. The v65 migration rewrites a pre-v65 file's old default and keeps any other value as a pin ([subcall-timeout-migration.ts](src/config/subcall-timeout-migration.ts)). Size any new sub-call default against the slowest hosted provider you support, not the local daemon.
+**Sub-call timeouts scale with provider latency.** A default tuned against a local `llama-server` does not carry over to hosted reasoning models. Before config v65 reflection had 10 s and the link-generator 8 s; measured on OpenRouter, reflection takes a median 13.9–16.3 s on glm-5.3-flash / qwen3.6-plus / kimi-k2.6 (37.5 s worst case in a live session) and kimi's link-generator 37.8 s, so 6 of 8 live reflections on glm-5.3-flash timed out and wrote nothing. v65 raises both to 60 s. The query rewriter went from 3 s to 10 s in the same step: gemini-3.8-flash and glm-5.3-flash rewrite in a median 4.2 s and kimi-k2.6 in 23.5 s, and in live sessions gemini's rewriter timed out 16 of 18 calls at 3 s, kimi's 16 of 16. Its cap stays below the background sub-calls because it blocks the turn. The v65 migration rewrites a pre-v65 file's old default and keeps any other value as a pin ([subcall-timeout-migration.ts](src/config/subcall-timeout-migration.ts)). Size any new sub-call default against the slowest hosted provider you support, not the local daemon.
 
 ### Invariants
 
@@ -1227,10 +1227,12 @@ When the gate returns `false`, the recall layer is byte-identical to v2: no LLM
 - **Grammar** ([query-rewriter-grammar.ts](src/memory/retrieve/query-rewriter-grammar.ts)) — `root ::= "<rewritten_query>" body "</rewritten_query>"` with `body ::= [^<]{1,400}`, plus a `NONE` alternative for explicit abstain.
 - **Parser** ([query-rewriter-parser.ts](src/memory/retrieve/query-rewriter-parser.ts)) — length-clamps the body, returns `null` on the `NONE` token, fails closed on malformed input (caller falls back to raw query).
 - **Slot.** `slotId: -1` always — see invariant 1 below.
-- **Timeout.** Hard cap `memory.retrieve.rewriter.timeoutMs` (default 3000ms). On timeout/abort/parse-failure, the runner returns the raw user message and the recall layer continues.
+- **Timeout.** Hard cap `memory.retrieve.rewriter.timeoutMs` (default 10000ms, config v65; 3000ms before). On timeout/abort/parse-failure, the runner returns the raw user message and the recall layer continues.
 
 **Decorator** [rewriter-aware-recall-provider.ts](src/memory/retrieve/rewriter-aware-recall-provider.ts) wraps an inner `MemoryContextProvider`. It intercepts `buildMemoryContext({ userMessage, recentTurns, ... })`, fires the rewriter when both (a) the gate matches and (b) `recentTurns.length > 0`, then forwards a (possibly) rewritten `userMessage` to the inner provider. Everything else (`### memory-index`, lesson recall, profile rendering) is untouched.
 
+**Once per turn.** `agent-loop.refreshMemoryContext` runs before the first step and again after every step, each time with the same user message, so the decorator remembers the rewrite per session, keyed by the user message and a SHA-256 digest of the history slice it sent. Only the latest key per session is kept, for at most `REWRITE_MEMO_MAX_SESSIONS` (256) sessions, least recently used dropped first. Every later refresh with the same key reuses the result — a timeout or failure whose outcome was the raw message included — so a slow provider costs one timeout per turn instead of one per step, and the trace carries one rewriter row per turn. An attempt that ended with the caller's signal aborted is not remembered, so a cancelled turn cannot stop the next turn's identical retry. A new user message or a changed history slice asks again.
+
 **`MemoryContextProviderInput.recentTurns`.** The decorator needs trailing user/assistant context, but `MemoryContextProviderInput` did not carry it pre-v2.5. The interface was extended with an optional `recentTurns: readonly { role: "user" | "assistant"; text: string }[]` — populated by `agent-loop.refreshMemoryContext` via the new helper `collectRecentUserAssistantTurns(state, options.userMessage)`. Older providers that never read the field stay byte-stable; the default provider ignores it.
 
 **Locked invariants** (pinned by [referential-detector.test.ts](src/memory/retrieve/referential-detector.test.ts), [query-rewriter-parser.test.ts](src/memory/retrieve/query-rewriter-parser.test.ts), [query-rewriter-runner.test.ts](src/memory/retrieve/query-rewriter-runner.test.ts), [rewriter-aware-recall-provider.test.ts](src/memory/retrieve/rewriter-aware-recall-provider.test.ts)):
@@ -1240,11 +1242,12 @@ When the gate returns `false`, the recall layer is byte-identical to v2: no LLM
 3. **Disabled by default.** With `memory.retrieve.rewriter.enabled = false`, the bootstrap does not construct a rewriter runner; the inner `MemoryContextProvider` is returned as-is. The recall path is byte-identical to v2.
 4. **Heuristic gate is pure.** No I/O, no state — easy to assert across a matrix of inputs. Pinned by `referential-detector.test.ts`.
 5. **Empty history is a hard skip.** Even when the gate fires, the rewriter is not called if `recentTurns` is empty (nothing to anchor against) — outcome `skipped_no_history`, raw query is used. Pinned by `rewriter-aware-recall-provider.test.ts`.
+6. **One rewrite per turn.** Repeated `buildMemoryContext` calls with the same session, user message and history slice reach the LLM once; a timed-out or failed attempt is reused, not retried; an aborted attempt is not remembered. Pinned by [rewriter-aware-recall-provider-memo.test.ts](src/memory/retrieve/rewriter-aware-recall-provider-memo.test.ts).
 
 **Configuration.** Added in user config v18; gate modes in v20 — older files transparently migrate with the block disabled / `gateMode: heuristic`.
 
 - `memory.retrieve.rewriter.enabled` (default `true`, config v21).
-- `memory.retrieve.rewriter.timeoutMs` (default `3000`).
+- `memory.retrieve.rewriter.timeoutMs` (default `10000`, config v65; `3000` before). Spent at most once per turn — see "Once per turn" above.
 - `memory.retrieve.rewriter.historyTurns` (default `3`).
 - `memory.retrieve.rewriter.gateMode` (default `"heuristic"`). Eval `on` profile in [eval-memory/harness/memory-profiles.ts](eval-memory/harness/memory-profiles.ts) defaults to `"embedding"`.
 - `memory.retrieve.rewriter.embeddingGate.threshold` (default `0.65`).

src/config/config-schema.ts

@@ -2114,12 +2114,15 @@ export interface UserConfigFile {
 // that number first.)
 // v65: memory sub-call timeouts are sized for hosted reasoning models —
 // `memory.reflection.timeoutMs` (also the vote-runner's budget) goes
-// 10 000 → 60 000 and `memory.links.generatorTimeoutMs` 8 000 → 60 000.
-// The old numbers were tuned against a local llama-server; hosted models
-// answer these calls in roughly 15–40 s, so most of them timed out and
-// wrote nothing. A pre-v65 file whose value is the old default (which
-// the schema wrote, not the operator) takes the new one; any other
-// number is read as a deliberate pin and kept.
+// 10 000 → 60 000, `memory.links.generatorTimeoutMs` 8 000 → 60 000 and
+// `memory.retrieve.rewriter.timeoutMs` 3 000 → 10 000. The old numbers
+// were tuned against a local llama-server; hosted models answer the
+// background calls in roughly 15–40 s and the rewriter in 4–24 s, so
+// most of them timed out and wrote or rewrote nothing. The rewriter's cap
+// stays lower because it blocks the turn (it runs once per turn, so a
+// timeout costs one wait, not one per step). A pre-v65 file whose value
+// is the old default (which the schema wrote, not the operator) takes
+// the new one; any other number is read as a deliberate pin and kept.
 export const USER_CONFIG_VERSION = 65;
 
 /**
@@ -2517,7 +2520,7 @@ export const USER_CONFIG_DEFAULTS: UserConfigFile = {
         // Uses `slotId=-1` so the main agent and reflection slots stay
         // untouched.
         enabled: true,
-        timeoutMs: 3_000,
+        timeoutMs: 10_000,
         historyTurns: 3,
         gateMode: "heuristic",
         embeddingGate: {
@@ -4820,10 +4823,15 @@ export function parseUserConfigFile(raw: unknown): UserConfigFile {
             USER_CONFIG_DEFAULTS.memory.retrieve.rewriter.enabled,
             "memory.retrieve.rewriter.enabled",
           ),
-          timeoutMs: parsePositiveInt(
-            memoryRetrieveRewriter.timeoutMs ??
-              USER_CONFIG_DEFAULTS.memory.retrieve.rewriter.timeoutMs,
-            "memory.retrieve.rewriter.timeoutMs",
+          timeoutMs: resolveSubcallTimeoutMs(
+            version,
+            parsePositiveInt(
+              memoryRetrieveRewriter.timeoutMs ??
+                USER_CONFIG_DEFAULTS.memory.retrieve.rewriter.timeoutMs,
+              "memory.retrieve.rewriter.timeoutMs",
+            ),
+            PRE_V65_SUBCALL_TIMEOUT_DEFAULTS.rewriterTimeoutMs,
+            USER_CONFIG_DEFAULTS.memory.retrieve.rewriter.timeoutMs,
           ),
           historyTurns: parsePositiveInt(
             memoryRetrieveRewriter.historyTurns ??

src/config/subcall-timeout-migration.test.ts

@@ -16,103 +16,103 @@ import {
 
 const PRE_V65 = HOSTED_SUBCALL_TIMEOUTS_VERSION - 1;
 
-function timeoutsOf(raw: Record<string, unknown>): {
-  reflection: number;
-  linkGenerator: number;
-} {
+const NEW_DEFAULTS = {
+  reflection: 60_000,
+  linkGenerator: 60_000,
+  rewriter: 10_000,
+};
+
+function timeoutsOf(raw: Record<string, unknown>): typeof NEW_DEFAULTS {
   const parsed = parseUserConfigFile(raw);
   return {
     reflection: parsed.memory.reflection.timeoutMs,
     linkGenerator: parsed.memory.links.generatorTimeoutMs,
+    rewriter: parsed.memory.retrieve.rewriter.timeoutMs,
+  };
+}
+
+function fileWith(
+  version: number,
+  t: { reflection: number; linkGenerator: number; rewriter: number },
+): Record<string, unknown> {
+  return {
+    version,
+    memory: {
+      reflection: { timeoutMs: t.reflection },
+      links: { generatorTimeoutMs: t.linkGenerator },
+      retrieve: { rewriter: { timeoutMs: t.rewriter } },
+    },
   };
 }
 
+const OLD_DEFAULTS = {
+  reflection: PRE_V65_SUBCALL_TIMEOUT_DEFAULTS.reflectionTimeoutMs,
+  linkGenerator: PRE_V65_SUBCALL_TIMEOUT_DEFAULTS.linkGeneratorTimeoutMs,
+  rewriter: PRE_V65_SUBCALL_TIMEOUT_DEFAULTS.rewriterTimeoutMs,
+};
+
 describe("memory sub-call timeouts (config v65)", () => {
-  it("defaults reflection and link-generator to 60 s", () => {
-    // Hosted reasoning models answer these calls in 15–40 s; the old
+  it("defaults reflection and link-generator to 60 s and the rewriter to 10 s", () => {
+    // Hosted reasoning models answer these calls in 4–40 s; the old
     // local-llama-server defaults timed most of them out.
     expect(USER_CONFIG_DEFAULTS.memory.reflection.timeoutMs).toBe(60_000);
     expect(USER_CONFIG_DEFAULTS.memory.links.generatorTimeoutMs).toBe(60_000);
-    expect(timeoutsOf({ version: USER_CONFIG_VERSION })).toEqual({
-      reflection: 60_000,
-      linkGenerator: 60_000,
-    });
+    expect(USER_CONFIG_DEFAULTS.memory.retrieve.rewriter.timeoutMs).toBe(
+      10_000,
+    );
+    expect(timeoutsOf({ version: USER_CONFIG_VERSION })).toEqual(NEW_DEFAULTS);
   });
 
   it("reads a pre-v65 file's schema-written old defaults as the new defaults", () => {
     // Every existing config.json carries these fields, written by the
     // schema. Keeping them would leave every install on the timeouts
     // this version exists to replace.
-    const raw = {
-      version: PRE_V65,
-      memory: {
-        reflection: {
-          timeoutMs: PRE_V65_SUBCALL_TIMEOUT_DEFAULTS.reflectionTimeoutMs,
-        },
-        links: {
-          generatorTimeoutMs:
-            PRE_V65_SUBCALL_TIMEOUT_DEFAULTS.linkGeneratorTimeoutMs,
-        },
-      },
-    };
-    expect(timeoutsOf(raw)).toEqual({
-      reflection: 60_000,
-      linkGenerator: 60_000,
+    expect(OLD_DEFAULTS).toEqual({
+      reflection: 10_000,
+      linkGenerator: 8_000,
+      rewriter: 3_000,
     });
+    const raw = fileWith(PRE_V65, OLD_DEFAULTS);
+    expect(timeoutsOf(raw)).toEqual(NEW_DEFAULTS);
     expect(parseUserConfigFile(raw).version).toBe(USER_CONFIG_VERSION);
   });
 
   it("migrates an older file the same way", () => {
-    expect(
-      timeoutsOf({
-        version: 51,
-        memory: {
-          reflection: { timeoutMs: 10_000 },
-          links: { generatorTimeoutMs: 8_000 },
-        },
-      }),
-    ).toEqual({ reflection: 60_000, linkGenerator: 60_000 });
+    expect(timeoutsOf(fileWith(51, OLD_DEFAULTS))).toEqual(NEW_DEFAULTS);
   });
 
   it("keeps any other pre-v65 value as a deliberate pin", () => {
+    const pinned = { reflection: 25_000, linkGenerator: 12_000, rewriter: 5_000 };
+    expect(timeoutsOf(fileWith(PRE_V65, pinned))).toEqual(pinned);
+    // The fields are independent: some pinned, some on the old default.
     expect(
-      timeoutsOf({
-        version: PRE_V65,
-        memory: {
-          reflection: { timeoutMs: 25_000 },
-          links: { generatorTimeoutMs: 12_000 },
-        },
-      }),
-    ).toEqual({ reflection: 25_000, linkGenerator: 12_000 });
-    // The fields are independent: one pinned, one on the old default.
+      timeoutsOf(
+        fileWith(PRE_V65, {
+          reflection: 4_000,
+          linkGenerator: 8_000,
+          rewriter: 3_000,
+        }),
+      ),
+    ).toEqual({ reflection: 4_000, linkGenerator: 60_000, rewriter: 10_000 });
     expect(
-      timeoutsOf({
-        version: PRE_V65,
-        memory: {
-          reflection: { timeoutMs: 4_000 },
-          links: { generatorTimeoutMs: 8_000 },
-        },
-      }),
-    ).toEqual({ reflection: 4_000, linkGenerator: 60_000 });
+      timeoutsOf(
+        fileWith(PRE_V65, {
+          reflection: 10_000,
+          linkGenerator: 8_000,
+          rewriter: 1_500,
+        }),
+      ),
+    ).toEqual({ reflection: 60_000, linkGenerator: 60_000, rewriter: 1_500 });
   });
 
   it("gives a pre-v65 file without the fields the new defaults", () => {
-    expect(timeoutsOf({ version: PRE_V65, memory: {} })).toEqual({
-      reflection: 60_000,
-      linkGenerator: 60_000,
-    });
+    expect(timeoutsOf({ version: PRE_V65, memory: {} })).toEqual(NEW_DEFAULTS);
   });
 
   it("keeps the old default numbers on a v65 file as the operator's pin", () => {
     expect(
-      timeoutsOf({
-        version: HOSTED_SUBCALL_TIMEOUTS_VERSION,
-        memory: {
-          reflection: { timeoutMs: 10_000 },
-          links: { generatorTimeoutMs: 8_000 },
-        },
-      }),
-    ).toEqual({ reflection: 10_000, linkGenerator: 8_000 });
+      timeoutsOf(fileWith(HOSTED_SUBCALL_TIMEOUTS_VERSION, OLD_DEFAULTS)),
+    ).toEqual(OLD_DEFAULTS);
   });
 
   describe("on disk", () => {
@@ -130,13 +130,7 @@ describe("memory sub-call timeouts (config v65)", () => {
       const path = getUserConfigPath(dir);
       writeFileSync(
         path,
-        JSON.stringify({
-          version: PRE_V65,
-          memory: {
-            reflection: { timeoutMs: 10_000 },
-            links: { generatorTimeoutMs: 8_000 },
-          },
-        }),
+        JSON.stringify(fileWith(PRE_V65, OLD_DEFAULTS)),
         "utf8",
       );
       const stderr = vi
@@ -146,13 +140,15 @@ describe("memory sub-call timeouts (config v65)", () => {
         const migrated = ensureUserConfigFileSync(path);
         expect(migrated.memory.reflection.timeoutMs).toBe(60_000);
         expect(migrated.memory.links.generatorTimeoutMs).toBe(60_000);
+        expect(migrated.memory.retrieve.rewriter.timeoutMs).toBe(10_000);
       } finally {
         stderr.mockRestore();
       }
       const onDisk = JSON.parse(readFileSync(path, "utf8"));
       expect(onDisk.version).toBe(USER_CONFIG_VERSION);
       expect(onDisk.memory.reflection.timeoutMs).toBe(60_000);
       expect(onDisk.memory.links.generatorTimeoutMs).toBe(60_000);
+      expect(onDisk.memory.retrieve.rewriter.timeoutMs).toBe(10_000);
     });
   });
 });

src/config/subcall-timeout-migration.ts

@@ -8,6 +8,11 @@
  * requests in roughly 15–40 s, so most sub-calls ran into the cap and
  * wrote nothing.
  *
+ * The query rewriter had 3 s. It sits on the hot path, so its cap stays
+ * well below the background ones (10 s): hosted models rewrite in a
+ * median 4–24 s, and the rewriter now runs once per turn, so a timeout
+ * costs a turn one wait rather than one per step.
+ *
  * Every `config.json` already carries these fields, written by the
  * schema rather than by the operator, so a pre-v65 file whose value
  * equals the old default is read as "never chosen" and takes the new
@@ -22,6 +27,7 @@ export const HOSTED_SUBCALL_TIMEOUTS_VERSION = 65;
 export const PRE_V65_SUBCALL_TIMEOUT_DEFAULTS = {
   reflectionTimeoutMs: 10_000,
   linkGeneratorTimeoutMs: 8_000,
+  rewriterTimeoutMs: 3_000,
 } as const;
 
 /**

src/memory/retrieve/index.ts

@@ -19,7 +19,8 @@
  *    are untouched.
  *  - `rewriter-aware-recall-provider` — decorator over
  *    `MemoryContextProvider`. Pulls history from
- *    `input.recentTurns` populated by the agent loop.
+ *    `input.recentTurns` populated by the agent loop, and asks the
+ *    runner at most once per turn (per session + message + history).
  */
 
 export {
@@ -57,6 +58,7 @@ export {
   type RewriterOutcome,
 } from "./query-rewriter-runner.js";
 export {
+  REWRITE_MEMO_MAX_SESSIONS,
   createRewriterAwareMemoryContextProvider,
   type RewriterAwareProviderOptions,
 } from "./rewriter-aware-recall-provider.js";