plan: exec summary in the result, verbose log on disk

[btipling]

Plan header

Field	Value
Status	IMPLEMENTED
Date	2025-07-16
Type	single
Parent	N/A
Source issue	#565 — agent tool: exec summary in the result, verbose log on disk
Branch	plan/exec-summary-log
Layers	Vercel backend
Reusability impact	none
Production mutate?	no
Cloud ops path	N/A — no Production mutate
Living docs	docs/sandbox.md

Summary

Shape exec tool results for the model: the tool result returns a short summary
(exit code, argv, first N + last M lines of output, and a log: <rel path>), while
the full stdout/stderr is flushed to a workspace log file under
.invincible/logs/. The model can read_file the log if it needs more detail
(via the existing search / read_file tools — #563 / #717).

Today exec returns the full stdout/stderr (capped at 4 MiB per stream on the
daemon, then further truncated by TOOL_RESULT_MAX_CHARS = 2M on the agent side).
A test runner or rg can blow the turn with 50k lines of TAP. This change makes
the default result compact (~20 lines) while keeping the full output accessible.

Not the session fold (#549). Same hygiene, different layer.

Goals

#	Goal	Success signal
1	Exec tool result is a compact summary (exit code + head/tail lines + log path), never a raw dump	Model sees ≤ ~25 lines for any exec, even a 50k-line test run
2	Full stdout/stderr is always written to a workspace log file (redacted, capped)	Log file exists at the reported path after every exec; model can read_file it
3	Secrets are redacted in the summary AND on disk	GH_TOKEN / GITHUB_TOKEN never appear in log files or summary text
4	Existing caps (timeout, stdio) are unchanged	clampExecTimeoutMs, MAX_STDIO_BYTES untouched
5	Empty-output execs don't create empty log files	exec true returns just the summary, no log write

Non-goals / out of scope

• Background process table / job wait/cancel/logs
• Shell or changing the argv contract (plan: sandbox exec tool — client timeout alignment + argv contract #310 closed)
• Folding historical tool_run into the prompt (agent memory: structured truncated tool_result on the wire (not Tool: crumbs) #549)
• Log rotation or cleanup — a future plan can add a tool/agent-side cleanup
• Changing the daemon protocol — the daemon still returns full stdout/stderr as-is
• Forbidden wiring: dual DOM chat · secrets in Wasm · laptop-only Production ops

Architectural decisions

Decision	Options considered	Choice	Why
Where to write the log file	A) Daemon writes it (adds a daemon endpoint/field) · B) Agent-side writes via existing write_file	B	Agent controls redaction + summary formatting; daemon stays simple (just streams back stdout/stderr). No daemon protocol change.
Log directory	A) .invincible/logs/ (hidden workspace dir) · B) Workspace root (pollutes) · C) /tmp (jail escape / Vercel Sandbox no /tmp)	A — .invincible/logs/exec-<ts>.log	Jail-safe, hidden from list_dir ., predictable pattern. New hidden workspace dir for this feature — no .invincible/ convention exists in the product today (checkout has none; only an unrelated runner service name and Zig module names collide). Precedent instead: write_file supports mkdir: true.
When to write (always vs only-when-large)	A) Always when output is non-empty · B) Only when output exceeds summary window	A — always write non-empty output	Simpler contract. The model can skip the read_file when the summary is enough. Always having the log on disk means the model never has to guess "was there more?"
Filename scheme	A) Timestamp-based · B) Hash-based · C) Sequential counter	A — exec-<ISO-ish-safe-ts>.log	Deterministic, no collision risk within a turn, human-readable in list_dir
Summary format for combined stdout+stderr	A) One combined head/tail block · B) Separate head/tail per stream	B — separate stdout/stderr sections, each with own head/tail	Stderr is semantically different (errors); interleaving loses signal
Write failure behavior	A) Fail the exec result · B) Include a note in the summary (fail-soft)	B — fail-soft with ⚠ log write failed: <reason> in the summary	The exec itself succeeded; a write_file failure shouldn't mask the command output. The summary still contains the head/tail lines.

Layer placement

Concern	Layer	Path(s)	Rationale
Exec tool result formatting (summary + log write)	Vercel backend	lib/agent/tools.ts	Agent tool layer — this is where finalize() already shapes results. The exec execute function owns the full pipeline: run → write log → return summary.
Log file write	Vercel backend	lib/agent/tools.ts via client.write_file()	Reuses the existing sandbox write_file tool path — same jail, same caps, same redaction.
New caps	Vercel backend	lib/sandbox/config.ts	Single source for sandbox config caps.
Secrets redaction	Vercel backend	lib/agent/redact.ts (existing)	Same redactSecrets() already used by finalize(). Applied to log content before write and to summary text.
Tests	Vercel backend	lib/agent/tools.test.ts	Existing exec test suite; add cases for summary format, log write, empty output skip, write-failure soft-landing.
Docs	Vercel backend	docs/sandbox.md	Existing tool table + protocol summary section.

Current baseline (live code)

Claim	Path / symbol	Notes
Exec tool returns full stdout/stderr via finalize()	lib/agent/tools.ts:728-791	parts.join('\n') with head line + stdout + stderr + truncation markers; redacted + truncated to TOOL_RESULT_MAX_CHARS
Daemon caps stdout/stderr at 4 MiB per stream	sandbox/constants.mjs:52 (MAX_STDIO_BYTES) + sandbox/tools.mjs:496-497 (attachCappedCollector)	Daemon returns ExecResult { exitCode, stdout, stderr, timedOut?, stdoutTruncated?, stderrTruncated? }
Agent-side final truncation at 2M chars	lib/sandbox/config.ts:10 (TOOL_RESULT_MAX_CHARS) + lib/agent/tools.ts:189 (truncateForModel)	After redaction
Client write_file path exists and works	lib/sandbox/client.ts + lib/sandbox/vercelClient.ts	Same jail, same caps; agent already uses client.write_file for the write_file tool
Secrets redaction already in finalize	lib/agent/tools.ts:187-189	redactSecrets(text, secrets) → truncateForModel(…, TOOL_RESULT_MAX_CHARS)
redactSecrets exported for direct use	lib/agent/redact.ts	Can call directly for log content before write
Existing resolveExecCwdForTool and rewriteExecRootToRel	lib/agent/workPath.ts	Path rewriting already works — log path needs the same treatment
Existing Buffer.byteLength usage for stdin	lib/agent/tools.ts:773	Precedent for byte-length reporting in tool output
Vercel Sandbox backend has no /tmp	lib/agent/tools.ts tool descriptions + persona standing orders	Log dir must be workspace-relative, not /tmp

Design

Summary format (tool result returned to model)

exec <cmd> [<args>]
exit=<code>  (or TIMED_OUT)
stdout: (<N> lines, <K> bytes)
  <head line 1>
  <head line 2>
  ...
  <head line N>
  ... (<skipped> lines truncated)
  <tail line 1>
  ...
  <tail line M>
stderr: (<N> lines, <K> bytes)   ← only present when stderr is non-empty
  <head line 1>
  ...
log: .invincible/logs/exec-<ts>.log   ← only present when log was written

• Head lines: first EXEC_LOG_HEAD_LINES (10) lines of each stream
• Tail lines: last EXEC_LOG_TAIL_LINES (10) lines of each stream
• Skip logic: head+tail overlap → just show all lines, no ... (truncated) marker
• When output fits entirely in head+tail: no ... (truncated) marker; log is still written
• Empty stream: the stream section is omitted entirely (e.g. no stderr → no stderr: block)
• Both streams empty: no log written, no log: line, just exec <cmd>\nexit=<code>
• Include stdin size when present: exec <cmd> stdin=<N>B (existing behavior preserved in header)
• Log path is always workspace-relative — same abs→rel rewrite as today's stdout/stderr

Log file format (on disk)

# exec log — <cmd> <args>
# cwd: <cwd>
# exit: <code>  (or TIMED_OUT)
# ts: <ISO timestamp>

[stdout]
<stdout content, redacted>

[stderr]
<stderr content, redacted>

• Capped at EXEC_LOG_MAX_BYTES (8 MiB) before write — daemon already caps each stream at 4 MiB, so this is a defense-in-depth ceiling
• Secrets redacted (same redactSecrets as the summary)
• Written via client.write_file() to .invincible/logs/exec-<ts>.log
• Timestamp format: YYYY-MM-DDTHHmmss-SSS (filesystem-safe, no colons)

Write flow (per exec call)

1. client.exec() → ExecResult { stdout, stderr, exitCode, ... }
2. If stdout or stderr is non-empty:
   a. Build log content: header + [stdout] + [stderr]
   b. Redact secrets in log content (redactSecrets)
   c. Cap log at EXEC_LOG_MAX_BYTES (defense-in-depth; already ≤ daemon caps)
   d. client.write_file(logPath, logContent, { mkdir: true })   ← REQUIRED (backends do not auto-create parent dirs)
      → success: include `log: <relPath>` in summary
      → failure: include `⚠ log write failed: <reason>` in summary (fail-soft)
3. Build summary: split stdout/stderr into lines, apply head/tail window
4. Redact summary content
5. return finalize(summary)

Edge cases

• Empty stdout AND stderr (e.g. exec true): No log written. Summary: exec true\nexit=0. No log: line.
• Command times out: TIMED_OUT instead of exit=<code>. Stdout/stderr may be partial — log still written with whatever was captured before the kill.
• write_file fails (disk full, permission, Vercel Sandbox transient error): Summary still includes the head/tail lines. A ⚠ log write failed: <reason> note replaces the log: line. The model can still read the head/tail.
• Both streams fit entirely in head+tail window: No ... (truncated) marker. Log still written (model can still read_file for raw bytes if needed).
• stdin provided: Header includes stdin=<N>B (existing behavior). Log file notes # stdin: <N>B in header.
• Secrets in output: Redacted in both summary and log file via redactSecrets(). Same pattern as today.
• Very long lines: Line count is based on \n splits. A single 4 MiB line without newlines → head=that line, tail=that line, no truncation marker (it's the only line). Log file has the full line.
• cwd annotation: The existing cwd path rewriting applies to the log path same as all other tool paths (via resolveExecCwdForTool). The log path in the result is workspace-relative.
• Concurrent exec calls (same turn, e.g. parallel tool calls): Timestamp-based filenames with ms precision avoid collisions within a single turn.

Caps table

Cap / ceiling	Value	Rationale	Code location
EXEC_LOG_HEAD_LINES	10	Enough to show the start of output without crowding the model context. Matches the read_file default limit (1000) as a "window into more" pattern. NEW cap.	lib/sandbox/config.ts
EXEC_LOG_TAIL_LINES	10	Symmetric with head. Tail is often the most useful part (test summaries, error messages). NEW cap.	lib/sandbox/config.ts
EXEC_LOG_MAX_BYTES	8_388_608 (8 MiB)	Defense-in-depth. Daemon already caps stdout/stderr at 4 MiB each (MAX_STDIO_BYTES), so worst-case combined is 8 MiB. This cap is a no-op ceiling for normal operation; it only triggers if daemon caps are raised without updating this one. Under the MAX_READ_WRITE_BYTES ceiling (16 MiB). NEW cap.	lib/sandbox/config.ts

No existing caps changed.

Cloud ops path

N/A — no Production mutate.

Living docs plan

Surface	Change	Notes
docs/sandbox.md	Update exec row in tool table to note summary + log path behavior	Timeless: describes current exec tool behavior
AGENTS.md	Update exec tool row in agent-tool surface table (line ~295)	The "Logical agent cwd" row mentions exec; update the lib/agent/tools.ts reference to note summary+log behavior
README.md	N/A	No visitor-facing change
SECURITY.md	N/A	No new secrets or trust boundaries — same redaction path as today
.env.example	N/A	No new env vars

Implementation order

1. Add three new caps to lib/sandbox/config.ts (EXEC_LOG_HEAD_LINES, EXEC_LOG_TAIL_LINES, EXEC_LOG_MAX_BYTES)
2. Refactor the exec tool's execute function in lib/agent/tools.ts:
   a. Extract stdout/stderr from ExecResult
   b. If non-empty, build log content, redact, write via client.write_file()
   c. Build summary with head/tail windows
   d. Return summary via finalize()
3. Add tests to lib/agent/tools.test.ts (see testing matrix)
4. Update docs/sandbox.md exec row + AGENTS.md exec reference
5. Gate: typecheck + vitest

Testing

#	Case	Layer	Type	Expected
1	Small output (fits in head+tail) — no truncation marker	Agent	unit	Summary shows all lines, no ... truncated, log path present
2	Large output (50 lines) — head+tail window with truncation marker	Agent	unit	Summary shows 10 head + 10 tail + ... (30 lines truncated) + log path
3	Empty output (exec true) — no log written	Agent	unit	Summary is exec true\nexit=0, no log: line, no write_file call
4	stdout only (no stderr) — stderr section omitted	Agent	unit	Summary shows stdout head/tail + exit=0, no stderr: block
5	stderr only (no stdout) — stdout section shows empty	Agent	unit	Summary shows stderr head/tail, stdout section omitted or shows 0 lines
6	TIMED_OUT — partial output captured	Agent	unit	Summary shows TIMED_OUT, log written with whatever was captured
7	write_file fails — fail-soft with warning	Agent	unit	Summary still has head/tail lines, ⚠ log write failed: <reason> instead of log:
8	Secrets in output — redacted in summary AND log	Agent	unit	GH_TOKEN=secret → GH_TOKEN=*** in both places
9	Very long single line (no newlines) — head=tail=same line	Agent	unit	Summary has the line once, no ... truncated. Log has the full line.
10	stdin provided — header includes size	Agent	unit	exec cat stdin=5B\nexit=0\nstdout: ...
11	Summary line count regression — compare to baseline	Agent	unit	Verify that output that previously was 200+ lines is now ≤ ~30 lines
12	Log file content matches full stdout/stderr	Agent	integration	read_file the log and verify it contains the complete redacted output

Test approach: Mock the client (exec + write_file) in the existing tools.test.ts pattern. The existing exec tests use a createClient that returns controlled ExecResults; extend that pattern.

Definition of done

• Three new caps in lib/sandbox/config.ts
• Exec tool execute refactored to write log + return summary (with mkdir: true on the log write — backends do not auto-create parent dirs)
• Tests green: 12 new cases in tools.test.ts, all existing exec tests still pass
• npm run typecheck green
• vitest run --changed green (agent workspace)
• docs/sandbox.md exec row updated
• AGENTS.md exec reference updated
• Cloud ops: N/A
• Living docs: docs/sandbox.md + AGENTS.md updated (timeless, no phase/issue theater)

Risks & mitigations

Risk	Mitigation
write_file on Vercel Sandbox backend may have different behavior than BYO for large payloads	The log file is at most 8 MiB, well under the 16 MiB MAX_READ_WRITE_BYTES. Both backends support write_file with the same cap. Test with the Vercel Sandbox test path (uses the real @vercel/sandbox SDK in Vercel-deployed tests).
.invincible/logs/ directory will not exist on first exec — backends do NOT auto-create parent dirs. BYO sandbox/tools.mjs writeFileTool only runs fs.mkdir(parent,{recursive:true}) when body.mkdir is truthy, else throws 400 "Parent directory does not exist (set mkdir: true)"; Vercel vercelClient.ts writeFile does the same (if (mkdir)). Fix (mandatory): the log write MUST pass mkdir: true — both client.writeFile (client.ts:466-469) and the agent write_file tool (tools.ts:589-593) already forward it, so step 2d passes it through.
Model ignores the log: path and keeps re-running exec to "see more"	The summary's head+tail lines are designed to be enough for most decisions. If the model needs the full output, the log: path is a read_file away. This is the same pattern as the existing read_file offset/limit — the model already knows how to fetch more.
Consecutive exec calls in one turn could accumulate many log files	Acceptable for v1. A future plan can add log cleanup (e.g. agent-side cleanup tool or turn-end sweep). .invincible/ is a brand-new hidden dir (no existing session convention) — with mkdir: true guaranteed on the log write the first exec creates it.

Open questions

None — in-scope engineering choices locked above.

References

• Source issue: #565 — agent tool: exec summary in the result, verbose log on disk
• Related: #562 (search tool — reduces exec rg usage), #563 (read_file window), #549 (session fold — consumes shorter results)

---

Review notes (2026-08-21, plan-review)

• Status flipped DRAFT → HANDOFF-READY (mandatory ground-truth write on the issue).
• Findings addressed in this revision:
  1. Stale baseline line refs (Current baseline) — substance verified correct against main, refs corrected: exec execute body is lib/agent/tools.ts:774-837 (not 728-791), finalize() is at :219-220 (not :187-189), stdin Buffer.byteLength is at :818-820 (not :773).
  2. False .invincible/ "existing convention" claim — no .invincible/ directory/session convention exists in the checkout (search: only an unrelated runner systemd service name and Zig module names). Bytes B/D corrected to "new hidden dir"; precedent reframed to write_file mkdir:true support.
  3. Inverted daemon mkdir claim — BYO/Vercel write_file do not auto-create parent dirs; the log write must pass mkdir: true (both client.writeFile seams already forward it). Risk C and write-flow step 2d corrected accordingly.
• No Blockers, no Majors. Design, layering, redaction reuse, path-rewrite reuse, and testing (12-case matrix, mock-client pattern matching existing tools.test.ts) are all sound.
• Reviewed by btipling; grounded against live main (backend paths in this checkout match origin/main).

---

Implementation (2026-08-21)

• Status: → IMPLEMENTED via /implement_plan.
• Deliverable PR: #758 — branch plan/exec-summary-log, base main, open, not merged.
• Change: backend TS + docs only. Three NEW caps in lib/sandbox/config.ts (EXEC_LOG_HEAD_LINES=10, EXEC_LOG_TAIL_LINES=10, EXEC_LOG_MAX_BYTES=8 MiB). lib/agent/tools.ts exec execute returns a compact head/tail summary and writes the full redacted output to .invincible/logs/exec-<ts>.log (mkdir:true) when either stream is non-empty, with a log: path; empty execs write nothing; log-write failure fails soft. 12 new cases in lib/agent/tools.test.ts.
• Gates: npm run typecheck green; full node_modules/vitest/vitest.mjs run 126 files / 1960 tests, failed=0. No Zig touched, no existing caps changed, no Production mutate, no secrets.
• Next: /adversarial_review #758 then explicit /merge_pr #758 (will close plan: exec summary in the result, verbose log on disk #724 and agent tool: exec summary in the result, verbose log on disk #565).