[Loschcode]

Async delivery, the fix Laurent chose from the production data below, shipped in #2013 (merged into accelerate): generate_image answers at once and the picture is drawn in the background and posted when ready. This PR is the rest of AI-3201, merged with accelerate after #2013, so it carries no image job code of its own.

Why

One picture took 60 to 140 s, past the 30 s caller-tool timeout, so a customer's own image tool failed with "the image generator timed out" until they set tool_timeout_ms: 180000 (AI-3201, piece 3 of AI-3160). The docs said tool_timeout_ms "Zero is the default" without saying what the default is, and fork() could not take it.

This PR ships the timeout half. The latency half is a finding, not a change, because choosing the fix needs production data this work could not get (below).

How it works

tool result wait (caller or worker tool)
 ├─ the tool's own timeout_ms           -> that (up to 30 min)
 ├─ else the session's tool_timeout_ms  -> that
 └─ else                                -> 30 s (defaultToolTimeout)

fork(tool_timeout_ms?)
 ├─ given                               -> it wins
 ├─ running parent                      -> parent's, also when the fork names a config (new)
 └─ parent ended                        -> 30 s (its row keeps none)

Change	Result	Test
tool_timeout_ms doc on CreateSessionRequest	Says 30 s, and that a tool's own timeout_ms replaces it	spec text
tool_timeout_ms on ForkSessionRequest	A fork can be given one	TestAForkOfAnEndedSessionTakesTheToolTimeoutItIsGiven, TestAForksToolTimeoutWinsOverItsRunningParents
forkSpec carries the timeout across a named config	It was silently reset to 30 s	TestAForkNamingAConfigKeepsItsRunningParentsToolTimeout
Go ForkOptions.ToolTimeout; JS fork({ tool_timeout_ms })	SDKs send it	TestAForksToolTimeoutGoesOverInMilliseconds; resource.test.ts "renames, voices, stops and forks..."
Regenerated Python, Swift, Kotlin, Dart, JS, Go, .NET, PHP, Ruby, Rust models	No drift	generator checks

A slow caller tool already had its own way out: SessionTool.timeout_ms (schemas.go:1143, JS timeoutMs) replaces the session's for that tool alone (TestAToolsOwnTimeoutReplacesTheSessions, session_test.go:1640). The built-in generate_image (shipped with #2013) is not bound by either: it runs in the background with its own 240 s.

Where the 60 to 140 s can go (code reading, no production numbers)

Stage	What the code does	Evidence
Router queue	One router process holds at most 2 FAL jobs at once, across every customer and model. A third request waits for a slot before it is submitted, bounded only by the 240 s deadline	imagegen/fal/fal.go:47, :185-190
Provider	image-fast is only fal/alibaba/qwen-image-3/text-to-image. FAL is submit-then-poll, every 3 s, the first poll 3 s after submit	routing/router.yaml:3469-3497, fal.go:43, :217-249
Failover	None for image-fast (one candidate); a job FAL accepted is never retried elsewhere	imagerouter/router.go failover rule
Bytes	Returned inline (sync_mode), decoded once to check, not re-encoded; no download from a provider URL	fal.go:263, :310-313, imagegen.go Verify
Upload	The customer's workaround uploaded to Chat itself; nothing in the router to measure

What is recorded cannot split these. A request row has latency_ms only (routing/call.go:113), which holds the slot wait, FAL's queue, the drawing and the result fetch together; duration_ms is not set for images. router_first_output_seconds buckets stop at 60 s (routing/traffic.go:43), so every 60 to 140 s picture lands in +Inf. Nothing logs or spans FAL's queue time.

No local timing was taken: no FAL key is configured on this machine, and the vendor suite (imagegen/fal/fal_integration_test.go) only checks a picture finishes within 2 minutes.

A parent that ended in the router process that held it still has its spec there, so a fork of it keeps its timeout; the field's "30 seconds" is for a parent known only from its row. The description does not say so, to avoid another round of regenerating ten clients for one clause.

Production data (read-only, Prometheus on the prod clusters)

router_first_output_seconds observes each usage row's latency_ms / 1000 (routing/traffic.go:149-150), so its per-pod _sum and _count give each image request's latency_ms. Prometheus on us-east4, us-west1 and europe-west4 (accelerate-prod) and us-east1 (staging, benchmark) keeps data from 2026-10-29; Postgres was not queried (no client in the router image, and starting a pod in prod was out of bounds).

	Value
Image requests since 2026-10-29, all regions	4, all in us-east4, all fal/alibaba/qwen-image-3/text-to-image (the image-fast model), all outcome=ok. None in us-west1, europe-west4, staging or benchmark
When	2026-11-10, 11:23 to 11:33 UTC: the reported incident
latency_ms each	72.3 s, 129.4 s, 60.2 s, 132.3 s (n=4: median 100.9 s, max 132.3 s; p90/p99 mean nothing at this count)
Router pods (count(up{job="accelerate-router"}), 10-min samples)	2 throughout the incident; 2 in 2,134 of 2,161 samples over 15 days, up to 6 only during deploys

Each request's completion is pinned to the ~30 s scrape interval:

pod 7fzz7  A  ~11:23:10 -> ~11:24:30   72.3 s   alone
pod d47rs  B  ~11:26:45 -> ~11:29:00  129.4 s   overlaps C by at most ~45 s
pod d47rs  C  ~11:28:40 -> ~11:29:40   60.2 s
pod 7fzz7  D  ~11:30:20 -> ~11:32:30  132.3 s   alone (starts after C ended)

• The router's 2-job cap never held a request back. No pod ever had a third job, so nothing waited for a slot. Account-wide, at most 2 jobs ran at once.
• FAL's account limit is not the cause either, as far as can be seen. FAL documents that every account starts at 2 concurrent running requests and scales to 40 with spend, that IN_QUEUE requests wait instead of failing, and that the actual limit is on the dashboard's Concurrency page (concurrency limits). Our account's figure is only there. Even at 2, A and D ran with nothing else in flight and still took 72 s and 132 s.
• So the time is FAL's: its queue or cold start plus the drawing, plus at most 3 s of polling granularity. Nothing recorded splits FAL's IN_QUEUE from IN_PROGRESS (the poll reads only status, fal.go:230-232).

Recommendation. Do not raise the cap: it never bound. No faster route exists to switch to: image-fast has one model, and image-quality (Gemini) is configured as the slower, dearer one and has no production requests to measure. What removes the failure is not making anyone wait on a tool timeout: AI-3200's built-in generate_image (240 s) and a caller tool's own timeout_ms cover today's 60 to 132 s, and async "post when ready" delivery is the durable fix if pictures are to stay this slow. Before choosing a faster model, record FAL's queue time against its drawing time per job, so the next incident says which of the two it was. No code change is made for this here, since the data rules the cap out rather than pointing at a one-line fix.

Not done here: open decisions

• Latency fix. The production data above rules out the router's cap. Choosing between async delivery and a different model is a product call; FAL's actual account limit is on its dashboard only.
• tool_timeout_ms on resume. sessions.resume() sends nothing to the router: it fetches the row and watches. A conversation reopened in the process that held it keeps its spec, timeout included. One reopened from its row (reopenedFromRow) gets neither the timeout nor the caller's declared tools back, since the row keeps neither. Carrying it needs either a session-row column (a migration) or a field on the request that reopens it (respond). That is an API design choice, so it is left open.
• Async "post when ready" delivery shipped in AI-3201: draw pictures in the background, post when ready #2013, the fix chosen from the data above.

Validation

CI: integration (internal/api) fails TestMemorySuite at memory_recall_test.go:218, as accelerate itself does. accelerate's last two runs (43444872895, 43440533515) fail it and integration (internal/memory/mem0) the same way; the mem0 job's log shows the shared mem0 account out of its monthly retrieval quota (50,000 used of 50,000, resets 2026-11-13 21:24 UTC). This PR touches neither. The earlier TestAdmissionSuite failure passed on rerun.

• go test -tags integration ./internal/api -run 'TestSessionConnectorsSuite/...|TestSessionToolsSuite': the three new fork tests and the existing fork and tool suites pass. With both forkSpec lines removed, all three new tests fail. Wider run: go test ./internal/api ./internal/session ok; -tags integration ./internal/api -run 'TestSessionConnectorsSuite|TestSessionCreateSuite|TestSessionToolsSuite|TestDocumentsSuite|TestAgentTeamsSuite|TestSessionVerbsSuite' ok; sdks/swift/generate.py --check up to date.
• cd sdks/go && go test ./...: ok. cd sdks/js && npm test (Node 22): 600 passed, 0 failed. npm run types -- --check: in step.
• Kotlin generate.py (Docker) and Dart tool/generate.dart (in a dart:stable container) regenerated; the diffs add the fork field and the new description only.
• .NET, PHP, Ruby and Rust models regenerated in Docker as their sdk skills say (the first CI run failed their in-step checks before that). Swift and Kotlin hand-written callers build ForkSessionRequest with named arguments, and the new parameter defaults to nil/null.
• dev.py check was not run: the worktree has no synced Python env. Only generated Python models changed.

🤖 Generated with Claude Code

[Loschcode]

/jeffery review

[getstream-jeffery]

Thanks for asking Jeffery for review. I'll stay on this PR until I can approve it.

How it works

• Findings come as inline comments, each starting with [Must Fix], [Should Fix], nit: or consider:.
• Push fixes whenever you're ready. Once the branch has been quiet for 10 minutes I review the new head and resolve the threads it fixes. To skip the wait, comment /jeffery review.
• If you disagree or have a question, reply in the thread. I'll answer there, and resolve it if you're right.
• I approve once every must-fix and should-fix is closed and nobody has requested changes. CI isn't part of that; branch protection still holds the merge until it's green. From round 3 on, open should-fix findings stop blocking.
• I don't approve release/* branches or changes to protected paths, and I'll say so when that applies.
• If a must-fix is still open after round 8, I stop and hand the PR to a human. Draft PRs wait until they're marked ready.

Commands (each as a whole comment)

Command	What it does
/jeffery review	Review the current head now
/jeffery rereview	Start over at round 1 on the current head and withdraw my approval. This redoes the whole review and costs as much as the first one, so use it only when you don't trust the earlier rounds. If nothing changed since my last round I say so instead; add force to redo it anyway
/jeffery pause / /jeffery resume	Stop or restart automatic reviews on this PR
/jeffery status	What I last reviewed, and why I have or haven't approved
/jeffery help	This list

/review-agent or a mention of me works in place of /jeffery.

[getstream-jeffery]

Round 1 on 425f5337048e · 🆕 2 new

[getstream-jeffery]

✅ Approved 425f5337048e after 1 round

Left open, your call:

• consider: Fork doc omits that an explicit zero resets to 30 s (acceleration/internal/api/sessions.go:1506)
• consider: Sub-millisecond ToolTimeout is sent as zero (sdks/go/client/sessions.go:599)

Not covered: Kotlin CreateSessionRequest.kt and swift generated.json hunks were not opened; compare their descriptions with openapi.yaml line 4302; Overflow of time.Duration for a huge tool_timeout_ms (about 9.2e12 ms and above) at session/manager.go:703, and whether huma or JSON decoding caps the value first; this already applies to createSession.

Withdrawing approval until f376644 is reviewed.

[Loschcode]

/jeffery review

[getstream-jeffery]

Round 2 on 5defd05a5612 · 🆕 1 new · ✅ 2 fixed

[getstream-jeffery]

✅ Approved 5defd05a5612 after 2 rounds · 2 fixed

Left open, your call:

• consider: SDK note says Go sends the fork timeout "above zero" (.claude/skills/sdk/SKILL.md:872)

Not covered: Swift generated.json spec_sha256/client_sha256 recomputation against head openapi.yaml (python3 sdks/swift/generate.py); Merged-in Twilio/Telnyx router changes from the accelerate merge commit f376644 were not reviewed.

[Loschcode]

/jeffery review

Withdrawing approval until 274e8af is reviewed.

[getstream-jeffery]

Round 3 on 274e8af62170 · 🆕 1 new · ✅ 1 fixed

[getstream-jeffery]

✅ Approved 274e8af62170 after 3 rounds · 3 fixed

Left open, your call:

• consider: SDK note omits what leaving the fork timeout out does (.claude/skills/sdk/SKILL.md:872)

Not covered: Per-SDK status claims in the rest of the SKILL.md paragraph (JavaScript, Python, Swift, .NET, Ruby, Rust, PHP, Kotlin, Dart) were unchanged this round and not re-checked; grepping each SDK's fork model for tool_timeout_ms would settle them.

[Loschcode]

/jeffery review

Withdrawing approval until 7dfbfb0 is reviewed.

[getstream-jeffery]

✅ Approved 7dfbfb018f5a after 4 rounds · 4 fixed

Not covered: Claims about the JavaScript, Python, Swift, .NET, Ruby, Rust, PHP, Kotlin and Dart fork clients in the same SKILL.md paragraph were not checked; grep tool_timeout_ms under /work/sdks/* for each fork call.

[Loschcode]

/jeffery review

Withdrawing approval until 02ff422 is reviewed.

[getstream-jeffery]

✅ Approved 02ff422b0bd2 after 5 rounds · 4 fixed