AI-3201: fork takes tool_timeout_ms, 30 s default documented

.claude/skills/sdk/SKILL.md

@@ -872,4 +872,6 @@ Go keeps phone numbers and end-user ids out of Info logs (AI-3210, no spec chang
 Dart is one package now. `sdks/dart/stream_agents` (`stream_agents`) replaces `stream_agents_core`, `stream_agents_chat` and `stream_agents_rtc`, with the same API behind one import, `package:stream_agents/stream_agents.dart`. It is a Flutter package, since rtc needs Stream Video; core, chat and rtc stay apart inside it as `lib/src/core`, `lib/src/chat` and `lib/src/rtc`, each with a barrel, and `test/area_boundaries_test.dart` keeps core pure Dart and the areas apart. The Dart floor is 3.12 and the Flutter floor 3.44, Stream Chat 10.4's. No wire change; the other SDKs have nothing to do.
 
 Posted knowledge text uses the document states too (AI-2198): an ingest saves each document's row as `pending` before writing its passages, then marks it `indexed`, or `failed` with the reason when the write stops part way. Only the descriptions of `KnowledgeDocumentState`, `error` and `passages` changed. Go, JavaScript, Python (`plugins/stream`), .NET, Rust and PHP have the regenerated clients. Swift, Kotlin, Dart and Ruby do not carry these descriptions, so they need nothing.
+
+`ForkSessionRequest` has `tool_timeout_ms` (AI-3201): how long the fork's model waits for a caller's or a worker's tool, in milliseconds, as on `CreateSessionRequest`, whose description now says the default is 30 seconds and that a tool's own `timeout_ms` replaces it. Leaving it out keeps a running parent's, so send it only when the caller sets it. Go sends it from `client.ForkOptions.ToolTimeout` (a `time.Duration`, sent when at least a millisecond, since one under it would go over as zero, which the router reads as 30 seconds). JavaScript's `ForkOptions` takes the generated `tool_timeout_ms`; Python (`plugins/stream`) and Swift have the regenerated models. .NET, Ruby, Rust and PHP have the regenerated types and need the field on their fork call; Kotlin and Dart need it on fork only if they expose fork.
 `SessionFilter` takes `config_id` (a value or `{"$eq": ...}`), `state` (`live` or `ended`, a value or `{"$eq": ...}`), `created_at` (`TimeWindow`: `$gte` and/or `$lt`, RFC 3339) and `custom` (an object of string labels the session's `custom` holds), anywhere in the filter, `$or` included. Go documents them on `client.Filter`. Every generated client is regenerated (`SessionStateEquals` is the new `state` schema; Swift drops `config_id` and `state`, which are string-or-object unions). Other SDKs need to update: every typed filter (Kotlin, Swift, Dart) should offer the four fields, and the client SDKs may offer all four, since none is server side only.

CHANGELOG.md

@@ -2,6 +2,9 @@
 
 ## Features
 
+### A fork takes tool_timeout_ms, and the default tool timeout is documented as 30 seconds
+
+`POST /v1/agents/sessions/{id}/fork` takes `tool_timeout_ms`, as `createSession` does: how long the fork's model waits for a tool the caller or a worker runs. Left out, a fork keeps a running parent's, now also when the fork names another config, and a fork of a conversation that ended has the default. The default is documented: 30 seconds, and a tool's own `timeout_ms` replaces it for that tool, which is how a slow tool such as one that draws a picture gets longer alone. Go has `ForkOptions.ToolTimeout`; JavaScript sends `tool_timeout_ms` from `fork()` (AI-3201).
 ### An agent draws pictures without a backend route of its own
 
 `image` on an agent config (`POST`, `PUT` and `PATCH /v1/agents/configs`, and `image:` in agent.yaml through a sync) names what the agent draws with, as a provider/model or a shortcut such as `image-fast`. A text agent whose conversation has a Chat channel is then offered `generate_image(prompt, aspect_ratio?)`. The tool answers at once and the turn ends: the picture is drawn in the background and posted into the conversation as the agent's own message, with the picture attached, a minute or two later. A picture that cannot be drawn posts a short message saying why (the safety filter, a spending limit, a shape it cannot draw, or a failure), never nothing. The job is kept in Postgres (`image_jobs`) and drawn by whichever router claims it, at most two at a time per router (FAL's own limit), so four pictures in one reply are all drawn. A router stopping with a drain (`shutdown.drain`) finishes the pictures it is drawing; one stopped without a drain gives them back at once, and one that crashed leaves them to the next router once their 5-minute lease runs out, each picture drawn at most twice. A picture is kept the moment it is drawn and never drawn or billed again: a failed upload or post is tried again over about two and a half hours, and an upload Stream Chat refuses for good posts that the picture could not be posted. A message already written is not posted twice. `aspect_ratio` is one of `1:1`, `4:3`, `3:4`, `16:9` and `9:16`, and an `image` target the deployment cannot draw with is refused when the config is saved. A draw is billed as `POST /v1/image/generations` is, when it is drawn, under the session's attribution, so app, user, team and session spend limits all hold it; the money it may cost is held for the whole draw, which has 240 seconds, the endpoint's own limit, now in the image router. `tool_timeout_ms` never bounds it. Each FAL picture's usage row records `queued_ms` and `drawn_ms`, the time in FAL's queue and the drawing. One reply draws at most four pictures. There is no default and no per-session override, since a session can be opened from a device: an agent draws only when its config says so. With `image` set, the built-in replaces a caller tool named `generate_image`, the workaround that drew through a backend route of its own; without it, that tool runs as before (AI-3200, AI-3201).

acceleration/api/openapi.yaml

@@ -4398,7 +4398,7 @@ components:
           description: 'What to call the conversation, for a list a person reads, until the router names a persistent one for what was said. Never shown to the model: what a conversation is called is a label on it rather than part of it.'
           type: string
         tool_timeout_ms:
-          description: How long the model waits for a tool result. Zero is the default.
+          description: How long the model waits for the result of a tool the caller or a worker runs, in milliseconds. Zero or omitted is 30 seconds. A tool's own timeout_ms replaces it for that tool, which is how one slow tool, such as one that draws a picture, gets longer without every tool waiting as long.
           type: integer
         tools:
           items:
@@ -5123,6 +5123,9 @@ components:
           type: string
         title:
           type: string
+        tool_timeout_ms:
+          description: How long the fork's model waits for the result of a tool the caller or a worker runs, in milliseconds, as on createSession. Omitted keeps a running parent's; a fork of a conversation that ended has 30 seconds, since its row does not keep one. Zero is 30 seconds, as on createSession.
+          type: integer
       type: object
     GeneratedImage:
       properties:

acceleration/internal/api/schemas.go

@@ -292,7 +292,7 @@ type CreateSessionRequest struct {
 	Team                 *string            `json:"team,omitempty" minLength:"1" maxLength:"59" doc:"The team the session's work is counted under, its team spend limits included, as team:<name> is among a token's usage categories: 1 to 59 letters, digits, _ . and -. Left out, the session is counted under the team the caller's token names, else the agent config's team tag. A token naming another team is refused with a 400: the team a token was signed for is not moved. Server-side only: a device sending it is refused with a 403, since its team is the one its token was signed for."`
 	Timezone             *string            `json:"timezone,omitempty" doc:"The end user's IANA timezone, such as Europe/Amsterdam, which a device knows and the router does not. It sets the zone the agent reads dates in for this session and, for a user the caller may name, is remembered as theirs for the scheduled tasks and sessions that follow. An unknown zone is refused with a 400 invalid_timezone."`
 	Title                *string            `json:"title,omitempty" doc:"What to call the conversation, for a list a person reads, until the router names a persistent one for what was said. Never shown to the model: what a conversation is called is a label on it rather than part of it."`
-	ToolTimeoutMs        *int               `json:"tool_timeout_ms,omitempty" doc:"How long the model waits for a tool result. Zero is the default."`
+	ToolTimeoutMs        *int               `json:"tool_timeout_ms,omitempty" doc:"How long the model waits for the result of a tool the caller or a worker runs, in milliseconds. Zero or omitted is 30 seconds. A tool's own timeout_ms replaces it for that tool, which is how one slow tool, such as one that draws a picture, gets longer without every tool waiting as long."`
 	Tools                *[]SessionTool     `json:"tools,omitempty"`
 	Tts                  *string            `json:"tts,omitempty" doc:"Omit it and the config decides, or en-low-latency when there is no config."`
 	UserId               *string            `json:"user_id,omitempty" doc:"The name the agent joins the call as. It is not the owner of the session. The owner is the caller, from its auth. A backend opens a session for one of its end users with the query parameter user_id." default:"vision-agent"`

acceleration/internal/api/session_connectors_test.go

@@ -15,13 +15,15 @@ import (
 	"github.com/GetStream/getstream-go/v5"
 	"github.com/gorilla/websocket"
 
+	"github.com/GetStream/Vision-Agents/acceleration/internal/auth"
 	"github.com/GetStream/Vision-Agents/acceleration/internal/chatlog"
 	"github.com/GetStream/Vision-Agents/acceleration/internal/connectors/core"
 	"github.com/GetStream/Vision-Agents/acceleration/internal/connectors/fakeprovider"
 	"github.com/GetStream/Vision-Agents/acceleration/internal/connectors/providers"
 	"github.com/GetStream/Vision-Agents/acceleration/internal/connectors/schemes/bearer"
 	"github.com/GetStream/Vision-Agents/acceleration/internal/connectors/sources/mcp"
 	"github.com/GetStream/Vision-Agents/acceleration/internal/conversation"
+	"github.com/GetStream/Vision-Agents/acceleration/internal/session"
 	"github.com/GetStream/Vision-Agents/acceleration/internal/store"
 )
 
@@ -598,6 +600,51 @@ func (s *SessionConnectorsSuite) TestAForkOfARunningSessionReachesTheConnectionI
 	s.Empty(ran["error"])
 }
 
+// A slow tool, such as one that draws a picture, needs the longer timeout the caller gave the
+// conversation: a fork that dropped it would fail that tool at 30 seconds.
+func (s *SessionConnectorsSuite) TestAForkNamingAConfigKeepsItsRunningParentsToolTimeout() {
+	request := s.session(s.modelledConfig("counted/counted-model", nil), nil)
+	request.ToolTimeoutMs = pointerTo(180_000)
+	running := s.client.createSession(request)
+	named := s.modelledConfig("noted/noted-model", nil)
+
+	var forked Session
+	s.Require().Equal(http.StatusCreated, s.client.do(http.MethodPost, "/v1/agents/sessions/"+running.Id+"/fork",
+		ForkSessionRequest{Messages: pointerTo(false), ConfigId: &named}, &forked))
+
+	s.Equal(180_000, s.heldSpec(forked.Id).ToolTimeoutMs)
+}
+
+// The row of a conversation that ended keeps no tool timeout, so a fork of one is given it.
+func (s *SessionConnectorsSuite) TestAForkOfAnEndedSessionTakesTheToolTimeoutItIsGiven() {
+	parent := s.endedSession(s.modelledConfig("counted/counted-model", nil))
+
+	var forked Session
+	s.Require().Equal(http.StatusCreated, s.client.do(http.MethodPost, "/v1/agents/sessions/"+parent+"/fork",
+		ForkSessionRequest{Messages: pointerTo(false), ToolTimeoutMs: pointerTo(180_000)}, &forked))
+
+	s.Equal(180_000, s.heldSpec(forked.Id).ToolTimeoutMs)
+}
+
+func (s *SessionConnectorsSuite) TestAForksToolTimeoutWinsOverItsRunningParents() {
+	request := s.session(s.modelledConfig("counted/counted-model", nil), nil)
+	request.ToolTimeoutMs = pointerTo(60_000)
+	running := s.client.createSession(request)
+
+	var forked Session
+	s.Require().Equal(http.StatusCreated, s.client.do(http.MethodPost, "/v1/agents/sessions/"+running.Id+"/fork",
+		ForkSessionRequest{Messages: pointerTo(false), ToolTimeoutMs: pointerTo(180_000)}, &forked))
+
+	s.Equal(180_000, s.heldSpec(forked.Id).ToolTimeoutMs)
+}
+
+// heldSpec is the spec of the session id this node holds.
+func (s *SessionConnectorsSuite) heldSpec(id string) session.Spec {
+	held, ok := s.manager.Get(id, session.Owner{CustomerID: s.customerID(), Kind: auth.KindServer})
+	s.Require().True(ok, "session %s is held here", id)
+	return held.Spec()
+}
+
 // TestAForkOfARunningSessionKeepsTheModelItWasOpenedWith: a live parent's spec carries its
 // models, so a config edited since does not reach the fork.
 func (s *SessionConnectorsSuite) TestAForkOfARunningSessionKeepsTheModelItWasOpenedWith() {

acceleration/internal/api/sessions.go

@@ -1158,13 +1158,15 @@ func forkSpec(parent session.Found, request ForkSessionRequest, config, current
 		fresh.Project, fresh.Custom = spec.Project, spec.Custom
 		fresh.ConnectorSelections = spec.ConnectorSelections
 		fresh.ReturnURL = spec.ReturnURL
+		fresh.ToolTimeoutMs = spec.ToolTimeoutMs
 		spec = fresh
 	}
 
 	spec.ForkedFrom = parentID
 	spec.Title = override(spec.Title, request.Title)
 	spec.Description = override(spec.Description, request.Description)
 	spec.Project = override(spec.Project, request.ProjectId)
+	spec.ToolTimeoutMs = override(spec.ToolTimeoutMs, request.ToolTimeoutMs)
 	spec.Incognito = value(request.Incognito)
 	if request.Custom != nil {
 		spec.Custom = *request.Custom
@@ -1516,6 +1518,7 @@ type ForkSessionRequest struct {
 	Custom          *map[string]interface{} `json:"custom,omitempty"`
 	Description     *string                 `json:"description,omitempty"`
 	Incognito       *bool                   `json:"incognito,omitempty" doc:"Hold the fork off the record. The parent still exists; this conversation onwards is simply not kept."`
+	ToolTimeoutMs   *int                    `json:"tool_timeout_ms,omitempty" doc:"How long the fork's model waits for the result of a tool the caller or a worker runs, in milliseconds, as on createSession. Omitted keeps a running parent's; a fork of a conversation that ended has 30 seconds, since its row does not keep one. Zero is 30 seconds, as on createSession."`
 	Messages        *bool                   `json:"messages,omitempty" doc:"Carry the parent's history into the fork, so the new conversation continues from what was already said. False starts the same configuration over from nothing, which is what comparing two answers to the same opening question wants." default:"true"`
 	ModelOverwrites *ModelOverwrites        `json:"model_overwrites,omitempty"`
 	ProjectId       *string                 `json:"project_id,omitempty"`
@@ -1524,6 +1527,7 @@ type ForkSessionRequest struct {
 }
 
 func (*ForkSessionRequest) TransformSchema(_ huma.Registry, schema *huma.Schema) *huma.Schema {
+	schema.Properties["tool_timeout_ms"].Format = ""
 	schema.Description = "Continue a conversation as a new one. Everything the parent was opened with is inherited; anything named here is written over it, which is what makes a fork useful rather than a copy -- the usual reason to fork is to ask the same question of a different model."
 	return schema
 }

acceleration/internal/session/spec.go

@@ -336,7 +336,8 @@ type Spec struct {
 	// functions: the session carries the request out to whoever asked for the session
 	// and waits for them to answer it.
 	Tools []harness.Tool
-	// ToolTimeoutMs bounds how long the model waits on one of them. Zero is the default.
+	// ToolTimeoutMs bounds how long the model waits on one of them. Zero is the default,
+	// defaultToolTimeout.
 	ToolTimeoutMs int
 	// MaxToolRounds is the reply to a tool result, since the caller last spoke, made to
 	// answer rather than reach for another tool. Zero is the agent's default; negative is

plugins/stream/vision_agents/plugins/stream/_generated/models/create_session_request.py

@@ -137,7 +137,9 @@ class CreateSessionRequest:
         title (str | Unset): What to call the conversation, for a list a person reads, until the router names a
             persistent one for what was said. Never shown to the model: what a conversation is called is a label on it
             rather than part of it.
-        tool_timeout_ms (int | Unset): How long the model waits for a tool result. Zero is the default.
+        tool_timeout_ms (int | Unset): How long the model waits for the result of a tool the caller or a worker runs, in
+            milliseconds. Zero or omitted is 30 seconds. A tool's own timeout_ms replaces it for that tool, which is how one
+            slow tool, such as one that draws a picture, gets longer without every tool waiting as long.
         tools (list[SessionTool] | Unset):
         tts (str | Unset): Omit it and the config decides, or en-low-latency when there is no config.
         user_id (str | Unset): The name the agent joins the call as. It is not the owner of the session. The owner is

plugins/stream/vision_agents/plugins/stream/_generated/models/fork_session_request.py

@@ -46,6 +46,9 @@ class ForkSessionRequest:
                 which also lets a parent that kept no Chat transcript be forked with its history. Cannot be combined with
                 messages false.
             title (str | Unset):
+            tool_timeout_ms (int | Unset): How long the fork's model waits for the result of a tool the caller or a worker
+                runs, in milliseconds, as on createSession. Omitted keeps a running parent's; a fork of a conversation that
+                ended has 30 seconds, since its row does not keep one. Zero is 30 seconds, as on createSession.
     """
 
     agent: str | Unset = UNSET
@@ -58,6 +61,7 @@ class ForkSessionRequest:
     project_id: str | Unset = UNSET
     response_id: str | Unset = UNSET
     title: str | Unset = UNSET
+    tool_timeout_ms: int | Unset = UNSET
     additional_properties: dict[str, Any] = _attrs_field(init=False, factory=dict)
 
     def to_dict(self) -> dict[str, Any]:
@@ -85,6 +89,8 @@ def to_dict(self) -> dict[str, Any]:
 
         title = self.title
 
+        tool_timeout_ms = self.tool_timeout_ms
+
         field_dict: dict[str, Any] = {}
         field_dict.update(self.additional_properties)
         field_dict.update({})
@@ -108,6 +114,8 @@ def to_dict(self) -> dict[str, Any]:
             field_dict["response_id"] = response_id
         if title is not UNSET:
             field_dict["title"] = title
+        if tool_timeout_ms is not UNSET:
+            field_dict["tool_timeout_ms"] = tool_timeout_ms
 
         return field_dict
 
@@ -147,6 +155,8 @@ def from_dict(cls, src_dict: Mapping[str, Any]) -> Self:
 
         title = d.pop("title", UNSET)
 
+        tool_timeout_ms = d.pop("tool_timeout_ms", UNSET)
+
         fork_session_request = cls(
             agent=agent,
             config_id=config_id,
@@ -158,6 +168,7 @@ def from_dict(cls, src_dict: Mapping[str, Any]) -> Self:
             project_id=project_id,
             response_id=response_id,
             title=title,
+            tool_timeout_ms=tool_timeout_ms,
         )
 
         fork_session_request.additional_properties = d

sdks/dart/stream_agents/lib/src/core/generated/api.dart

@@ -893,6 +893,7 @@ final class ForkSessionRequest {
     this.projectId,
     this.responseId,
     this.title,
+    this.toolTimeoutMs,
   });
 
   factory ForkSessionRequest.fromJson(Object? value) {
@@ -938,6 +939,10 @@ final class ForkSessionRequest {
         null => null,
         final Object value => _string(value, 'ForkSessionRequest.title'),
       },
+      toolTimeoutMs: switch (json['tool_timeout_ms']) {
+        null => null,
+        final Object value => _int(value, 'ForkSessionRequest.tool_timeout_ms'),
+      },
     );
   }
 
@@ -951,6 +956,7 @@ final class ForkSessionRequest {
   final String? projectId;
   final String? responseId;
   final String? title;
+  final int? toolTimeoutMs;
 
   Map<String, Object?> toJson() => {
     'agent': ?agent,
@@ -963,6 +969,7 @@ final class ForkSessionRequest {
     'project_id': ?projectId,
     'response_id': ?responseId,
     'title': ?title,
+    'tool_timeout_ms': ?toolTimeoutMs,
   };
 }