Skip to content

feat(api): lite mode for /calls — unblock mega-turn detail page#23

Merged
vaderyang merged 3 commits into
mainfrom
feat/calls-lite-mode
May 21, 2026
Merged

feat(api): lite mode for /calls — unblock mega-turn detail page#23
vaderyang merged 3 commits into
mainfrom
feat/calls-lite-mode

Conversation

@vaderyang
Copy link
Copy Markdown
Collaborator

Summary

  • /api/agent-turns/{id}/calls?lite=1 strips request_body, response_body, request_headers, response_headers so the response stays bounded on agentic turns with hundreds of iterations. Default behavior unchanged for callers that don't pass lite=1.
  • Console auto-opts in when turn.call_count > 200, shows a small notice, and lazy-fetches /api/llm-calls/{id} when the user expands a specific call.

Why

Clicking on a turn with hundreds of agentic iterations freezes the browser. On real production data a single 878-call turn returns 302 MB JSON — the browser can't parse or render that. With this PR the same turn's list payload drops to 521 KB and renders in ~200 ms; individual call bodies load on demand when the user expands a card.

Implementation

Server (storage trait → DuckDB impl → API route)

  • query_turn_calls(turn_id, include_bodies: bool) and query_calls_by_ids(call_ids, include_bodies: bool) now take a flag. When false, the SQL projection emits NULL::VARCHAR for the four heavy columns — DuckDB never reads body pages off disk, the strings never cross into Rust.
  • New ?lite=1 query param on GET /api/agent-turns/{id}/calls flips include_bodies = false. No-arg behavior matches today exactly.
  • tokens_estimated derivation falls back when response_body is absent (currently reports true in lite mode — a known minor white lie, follow-up will switch to Option<bool> so the UI can show "unknown" instead).

Console (auto-opt-in + lazy load)

  • useAgentTurnCalls(id, lite) passes the flag through.
  • AgentTurnDetailPanel derives liteMode = (turn?.call_count ?? 0) > 200 and shows an amber banner ("Large turn — bodies omitted, expand any call to fetch").
  • CallCard lazy-fetches /api/llm-calls/{id} only when the user expands a card whose inline bodies are null. Gated on expanded so 800 collapsed cards don't fire 800 background requests at mount.
  • Tools index / classifier are already null-safe — no extra changes needed.

Test plan

  • cargo test -p ts-storage-duckdb — 65 pass (extended query_turn_calls_orders_and_sequences to assert lite mode strips all four heavy fields and preserves every other field)
  • cargo build --workspace — clean
  • bun test in console — 111 pass
  • bun run build in console — clean
  • Deployed to wuneng production:
    • Full mode: 302,357,122 bytes (≈302 MB) — unchanged for compatibility
    • Lite mode: 521,625 bytes (≈521 KB), 200 ms response
    • 578× size reduction on the same 878-call turn
    • UI loads detail page without freezing; expanding a single call fetches its ~190 KB bodies independently

Compatibility

Default behavior unchanged. Any existing API consumer (tfcli, scripts, third-party callers) that doesn't pass ?lite=1 gets the same response shape as before.

🤖 Generated with Claude Code

Vader Yang and others added 2 commits May 20, 2026 10:56
Clicking on a turn with hundreds of agentic iterations would freeze
the browser. The /api/agent-turns/{id}/calls endpoint returned every
call's full request_body + response_body + headers; an 878-call turn
on real data lands a 168 MB JSON response that the browser can't
parse or render.

Fix is in two parts that ship together:

**Server** (StorageBackend trait + DuckDB impl + API route)
- `query_turn_calls(turn_id, include_bodies: bool)` and
  `query_calls_by_ids(call_ids, include_bodies: bool)` now accept a
  flag. When false, the SQL projection selects `NULL::VARCHAR` for
  the four heavy fields — DuckDB never reads the body pages off disk
  and they don't transfer to Rust as Strings.
- New `?lite=1` query param on `GET /api/agent-turns/{id}/calls`
  flips `include_bodies = false`. Default behavior unchanged for
  every existing caller.
- `tokens_estimated` derivation falls back to `false` in lite mode
  (it inspects response_body); documented on the trait.

**Console** (auto-opt-in for large turns + lazy-load on expand)
- `useAgentTurnCalls(id, lite)` passes `?lite=1` when caller asks.
- `AgentTurnDetailPanel` watches `turn.call_count`; above 200 it
  flips lite mode on. Renders a small amber banner so the user
  knows bodies are being lazy-loaded.
- `CallCard` lazy-fetches `/api/llm-calls/{id}` only when the user
  expands a card whose inline bodies are null. Gated on `expanded`
  so a mega-turn with 800 collapsed cards doesn't fire 800
  background requests at mount.
- Tools index / classifier already null-safe — no extra changes.

Real-world impact on the 878-call turn observed in production:
list response shrinks from 168 MB to under 1 MB; detail page now
loads in well under a second; expanding any single call fetches its
~190 KB of bodies independently.

Tests:
- ts-storage-duckdb: extended `query_turn_calls_orders_and_sequences`
  to assert lite mode strips all four heavy fields and preserves
  every other field byte-for-byte.
- console: 111 existing tests pass, no behavior change for
  small-turn workflow.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
github-actions[bot]
github-actions Bot previously approved these changes May 21, 2026
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ERROR: agent unavailable (see workflow log)


🤖 Reviewed by viviworkflow run

@vaderyang vaderyang dismissed github-actions[bot]’s stale review May 21, 2026 02:34

Dismissed: bogus APPROVED from agent error; reviewer bug fixed in PR#37

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Agent run failed (exit 1). See workflow logs.


🤖 Reviewed by viviworkflow run

@github-actions
Copy link
Copy Markdown

Summary

This PR adds a "lite mode" to the /api/agent-turns/{id}/calls endpoint to prevent browser freezes when loading mega-turns (hundreds of agentic iterations). The implementation is clean and well-designed: SQL-level NULL projections avoid disk reads, the threshold-based auto-opt-in is sensible, and lazy-loading per expanded call is gated correctly.

However, there's a blocking compilation error: the test mock in sink.rs doesn't implement the updated trait signatures.

Recommendation: REQUEST_CHANGES — fix the compilation error, then APPROVE.


Blocking

  • server/ts-storage/src/sink.rs:326-330 — Test mock CountingBackend implements outdated StorageBackend trait signatures. query_turn_calls and query_calls_by_ids now require an include_bodies: bool parameter, but the mock definitions at lines 326 and 329 are missing it. This will fail to compile.

Suggestions

  • console/src/hooks/use-agent-turns.ts:69queryKey correctly includes lite. Good.

  • server/ts-storage-duckdb/src/calls.rs:109-116 — The SQL-level NULL::VARCHAR projection for lite mode is the right approach — DuckDB won't read body pages. No body-scan smell detected.


Questions

  • None — the commit intent matches the diff.

Verified

  • Schema mirror: AgentTurnCallItem in console/src/types/api.ts:162-166 already has body fields as nullable (string | null), matching the Rust TurnCallItem where they're Option<String>. Lite mode returning null is type-compatible.

  • Query parameter: CallsParams.lite is #[serde(default)], so omitted parameter defaults to 0 (full bodies). Backwards-compatible.

  • Lazy-load hook: useLlmCallDetail (console/src/hooks/use-llm-call-detail.ts:5-11) fetches /api/llm-calls/{id} which returns LlmCallDetail with full bodies. Correct fallback path.

  • SQL body-scan check: No LENGTH(body), MAX(body), or arg_max patterns in the modified query. The query_calls list endpoint still reads response_body for tokens_estimated, but that's unchanged and operates on paginated rows, not a wide window scan.

  • Caller compatibility: All callers of query_turn_calls / query_calls_by_ids were updated (API route + DuckDB impl + lib.rs shim). No orphaned callers.

  • queryKey correctness: useAgentTurnCalls includes both id and lite in the queryKey — cache won't serve stale data when switching between lite/full for the same turn.


🤖 Reviewed by viviworkflow run

`StorageBackend::query_turn_calls` and `query_calls_by_ids` now
take an `include_bodies: bool` parameter (the lite-mode toggle
added in this PR), but the `CountingBackend` test mock in
`sink.rs` was still on the old two-argument signature, breaking
`cargo test` for the whole workspace.

Caught by the vivi reviewer on PR #23.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vaderyang vaderyang merged commit a43000f into main May 21, 2026
1 check passed
@vaderyang vaderyang deleted the feat/calls-lite-mode branch May 21, 2026 10:11
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

This PR adds "lite mode" for the /api/agent-turns/{id}/calls endpoint to fix a browser freeze on mega-turns (878 calls × ~190KB bodies each ≈ 168MB JSON). The server-side include_bodies flag uses NULL::VARCHAR projection to avoid reading body pages from disk; the frontend auto-opts-in above 200 calls and lazy-loads individual call bodies on expand. The trait signature change is cleanly propagated across all impls and test mocks.

Recommendation: APPROVE — the change is sound, well-tested, and matches the commit message. One non-blocking cosmetic regression in lite-mode classification.

Suggestions

  • console/src/pages/agent-turn-detail-panel.tsx:32 and console/src/components/turn-detail/stats-cards.tsx:41 — In lite mode, classifyType receives null for response_body, causing all calls to classify as "text" instead of their actual type (tool_call/text/final). The gantt nav icons and stats cards will show incorrect classifications for mega-turns. Consider either: (a) passing the lazy-loaded detail to these classifiers when available, or (b) showing a neutral icon when bodies are null. Current behavior: page loads, expand works, but classification icons are wrong for large turns — a cosmetic regression worth fixing but not merge-blocking.

Verified

  • Schema mirror: TurnCallItem Rust↔TS field names and types match (request_body: Option<String>request_body: string | null, etc.).
  • Caller compatibility: all query_turn_calls / query_calls_by_ids callers updated — agent_turns.rs:280,286 pass the new bool param; test mock in sink.rs:326-337 matches trait signature.
  • Query key: ["agent-turn-calls", id, lite] includes the lite flag — no stale cache risk.
  • Null-safety: parseHeaders in helpers.ts and classifyType dispatch accept null bodies without crashing.
  • Test coverage: query_turn_calls_orders_and_sequences extended to verify lite mode strips all four heavy fields while preserving other fields.

🤖 Reviewed by viviworkflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant