later: cross-session handbook (idle extract, default off) — not the first-turn fix

[btipling]

Parent

#548 (agent session architecture)

Type

Later / not A1–A8. Do not plan this before structured messages (#549), notes (#550), budget (#551), and compaction (#552) exist. Extracting today’s 3.5M crumb would freeze a lobotomy.

Why this is a separate issue

#548 already forbids a fifth identity issue. This is a different product:

Store	Lifetime	Who writes	What it is
#549 model-facing history	this session, until compact	the turn loop	tool evidence
#550 working notes	this session; New/Clear wipes	the agent (and maybe a small auto block)	decisions / identity inside one mind
#372 templates	explicit user save → new session	the operator	freeze a good window
Personas / skills	user-authored standing orders	the operator	who / how, not what happened last Tuesday
This	across New session, same user/tenant	deferred pipeline, not the live turn	handbook of reusable lessons

Mixing them is the Codex/OMP failure mode: compaction wipes a cat from the live prompt while the handbook never contained those bytes; or learn mutates the current prefix and busts the KV cache (#558).

This does not fix “each send feels like the first turn.” That is still #549. Ship A1 first.

Problem

After A1–A4, Continue of this session can be a coherent agent. New session still starts as a stranger with personas + skill catalog only. Operators who did a week of work in one repo then hit New/Clear lose every landmine, preference, and “we already tried X.”

Peer split (2026-08-19 source notes):

Harness	Cross-session memory
Codex	First-class ~/.codex/memories/ two-phase pipeline (default on). Idle ≥ 6h, then Phase 1 extract + Phase 2 consolidator → MEMORY.md + v1 memory_summary.md. Live agent reads; it does not write mid-turn.
Oh My Pi	First-class memory.backend = off (default) | local (Codex port, idle 12h) | hindsight | mnemopi. /memory CLI. learn writes learned.md for the next session only.
OpenCode	Not in core. Plugins (Mem0, Supermemory, Hindsight) + Beast markdown convention. Issue #16077 still open.
DeepSeek Harness	Not in core. Plugin territory. Invariant is the session log.

Invincible is a cloud harness (Redis envelope + Blob transcript + Functions). We do not grow a ~/.codex on the operator laptop. If we ever ship this, it is tenant/user-scoped cloud state, default off (OMP, not Codex-on-by-default).

Goal (directional — lock in a later create-plan)

An opt-in handbook, same user, same tenant, not the live transcript:

1. Default off. Personas + skills + agent memory: durable working notes / findings across turns (identity, not one-shot) #550 notes remain the shipped memory story. A setting / feature flag turns this on.
2. Do not write on the hot path. The live /api/agent turn never rewrites MEMORY.md. Extract after idle (session not continued for N hours, or explicit /memory enqueue), on a root session (not a subagent, if we ever have one).
3. Phase 1 — extract. Bounded idle rollouts → structured raw_memory + short synopsis. Empty / no-op is a valid result (Codex: “Will a future agent plausibly act better because of what I write here?”). Drop one-off queries, live metrics, secrets. Stricter filter than the transcript (no developer scaffolding, no CoT).
4. Phase 2 — consolidate. One leased job writes a small MEMORY.md + memory_summary.md (prompt-loaded, ~5k token cap shared with any learned list). Dirtiness (did Phase 1 change anything?) decides whether the consolidator runs — Codex uses git baseline; we can use an object etag / hash, not a repo in the tenant bucket.
5. Inject on New session only (or first turn of a new id): the summary rides volatile block 2 (inference: cache-stable two-block system prompt (don’t bust the KV prefix) #558), never block 1. learn-style writes, if any, wait for the next New session so they do not bust the live prefix cache.
6. Recalled memory is heuristic context, not instructions. Repo state, the user, and the persona snapshot win conflicts.

Success: a New session in the same project can answer “what did we burn ourselves on last week?” from the handbook without loading the old Blob, and a session that was only “what’s 2+2” produces no handbook junk.

Cloud shape (steal the invariant, not the homedir)

Codex / OMP local	Ours (directional)
~/.codex/memories/ / ~/.omp/agent/memories/<cwd>/	tenant+user (+ optional sandbox/project key) objects — Blob or Postgres, not the Redis envelope (size)
JSONL rollout scan	Blob transcript + envelope updatedAt idle clock
git dirty bit	content hash / etag of the handbook object
memory:// reads	ordinary tools + a readable pointer (signed read URL or small GET); no new Wasm chat
default on (Codex) / default off (OMP)	default off

Idle / age knobs are a Caps table later (OMP 12h / 30d / 64 per startup is the conservative peer; Codex 6h / 10d / 2 per startup is aggressive). Functions must not extract 8 MiB inline — Phase 1 reads a truncated model-facing history (post-A1/A4), not the display Blob.

Constraints (lock)

• Blocked by A1–A4. No extract of formatPromptWithHistory crumbs.
• Not the first-turn fix. Do not retitle this as identity.
• Not agent memory: durable working notes / findings across turns (identity, not one-shot) #550. Notes die on New/Clear; the handbook does not. Notes must not be merged into Phase 2 as if they were a user profile.
• Not feat: agent templates / forking (save good context, reuse later) #372. Templates are an explicit “freeze this window.” Handbook is deferred, lossy, and automatic (when on).
• Thinking / CoT never becomes handbook evidence (same as SessionStore).
• Secrets redacted. No Gateway keys, sandbox tokens, PATs, .env bodies.
• Subagents / review loops, if any, disable extract so they cannot recurse the handbook (Codex MemoryTool off on the consolidator).
• Feature-divide: no dual chat; no Wasm I/O to the handbook store; host may later show a compact “memories on” chip (harness: workspace status bar (sandbox · cwd · git · context) #328 family), v1 can be Settings-only.
• Do not ship mnemopi embeddings/graph or a Hindsight localhost:8888 bank as core. Those stay plugin-shaped if ever.

Steal / skip (2026-08-19)

Steal

1. Two systems, two stores — live transcript ≠ handbook.
2. Idle extract, not mid-turn writes (prefix cache + Function timeout).
3. No-op Phase 1 is success.
4. Stricter filter for memory than for the transcript.
5. Citations / usage as ranking if we later evict (Codex usage_count), not RAG scores we don’t have.
6. learn does not mutate this turn’s prefix (OMP).
7. Inject cap ~5k; if the summary fills it, lessons drop.

Skip

• Homedir JSONL / ~/.codex / ~/.omp as SoT (we are cloud).
• Default-on (surprise retention + cost).
• Git as a required runtime inside the tenant bucket (etag is enough).
• On-box embedding engine (mnemopi) as v1.
• Live MEMORY.md rewrites during runAgent.
• Treating handbook inject as standing orders (that is personas).

Non-goals

• Implementing any of this in the A1 create-plan
• Replacing Redis / envelope / Blob transcript
• Cross-tenant / public memory marketplace
• Auto-scoring “this session was good” into feat: agent templates / forking (save good context, reuse later) #372 templates

Suggested next

Leave on the #548 board as Later. Revisit after A1–A4 have a week of honest Continue. Then create-plan with Caps (idle hours, age, extract concurrency, inject tokens, store).

Related

• agent session architecture — model-context memory, compaction, and viewport #548 parent — sequence only
• agent memory: structured truncated tool_result on the wire (not Tool: crumbs) #549–compaction: model-capability context compaction (not API payload slices) #552 — blockers (honest source material)
• agent memory: durable working notes / findings across turns (identity, not one-shot) #550 — session-owned notes (different lifetime)
• feat: agent templates / forking (save good context, reuse later) #372 — explicit snapshot (different verb)
• skills: inject a catalog, not full bodies, every turn #557 / inference: cache-stable two-block system prompt (don’t bust the KV prefix) #558 — catalog + cache-stable inject slot for the summary
• feat: user agent personas (AGENTS.md profiles) + New session inject #329 personas / feat: user/agent skills library (attachable playbooks) #331 skills — standing orders, not episodic memory