fix(server): deliver a gap-landing elicitation verdict to the codex re-park (tombstone zombie-waiter resolves, fingerprint tombstones) - #6024
Conversation
…aiter A proxy (e.g. Databricks Apps) can sever a codex-native harness long-poll client-side while holding the backend connection open: the harness abandons the chunk, but the server never observes a disconnect, so the chunk's waiter stays registered as a zombie under the stable elicitation id. A resolve that found that zombie set only its Future — written to a connection nobody reads — and wrote no pre-resolved tombstone (only the nothing-parked branch did). The harness's next re-park of the same stable id then found nothing and re-published the gate as a fresh pending card: the operator's answer was lost and they had to answer twice. _resolve_elicitation now also writes the verdict-carrying tombstone when it settles a live registered Future, so a re-park of the same id adopts the verdict. The tombstone is session-scoped and TTL-pruned, so a verdict that WAS delivered leaves only a harmless entry that ages out unconsumed.
…d can't replay a stale approval Harness elicitation ids derive from session/method/JSON-RPC request id, and a harness may reuse a request id for a LATER, different question within the tombstone TTL. Since the zombie-waiter fix now tombstones every resolve that lands on a registered waiter, an unguarded tombstone could hand that stale verdict to the new question's re-park — silently approving a command the operator never saw. Park each wait with a sha256 fingerprint of its request params, copy the fingerprint onto the tombstone written at resolve, and have the re-park adopt the verdict only when its own params produce the same digest. A mismatched tombstone is dropped so the new question is published as a fresh prompt. Tombstones from the nothing-parked branch carry no fingerprint and keep the legacy adopt-by-id semantics.
The nothing-parked resolve branch (detected-sever gap) previously wrote unfingerprinted tombstones, so a later different question reusing the id within the TTL could still inherit the verdict via that path. The pending index still holds the answered prompt at resolve time, so derive the fingerprint from its params and stamp the tombstone with it.
|
UI Preview is being deployed for this PR ⏳
|
|
Related issue
Closes #2055
Resolves OMNI-1233 (Linear).
Summary
A codex-native approval gate long-polls
POST /v1/sessions/{sid}/hooks/codex-elicitation-requestin chunks with the same JSON-RPC envelope, so the server-side elicitation id is stable across re-parks. When a proxy severs a chunk client-side while the backend connection stays open, the abandoned chunk's waiter stays parked as a zombie. If the operator answers the approval card in that gap,_resolve_elicitationset the verdict on the zombie waiter's Future (written to a connection nobody reads) and — because a Future was registered — wrote no tombstone. The harness's next chunk re-parked the same stable id, found nothing, and re-published the gate: the operator's answer was lost and they had to answer twice.Fix:
ELI5: the server used to hand your "Approve" to a phone line that had already gone dead, and then shredded the note. Now it also pins the note to the door (with a description of exactly which question it answers), so when the caller redials the same question, they get your answer — and a different question can't steal it.
Test Plan
Fail→pass proof (all three fail on the unfixed tree for the behavioral reason, pass with the fix):
tests/e2e_ui/approvals/test_codex_gap_verdict_zombie_repark.py— real SPA journey: park chunk 1 (undetected sever), approve the card in the browser, re-POST the same envelope. Before: chunk 2 hits ReadTimeout and the card re-appears. After: the re-park returns{"decision": "accept"}and no pending card remains.tests/server/integration/test_codex_gap_verdict_zombie_poll.py— same journey at the HTTP layer. Before: fails with "gap-landing verdict LOST". After: passes.tests/server/test_resolve_tombstones_zombie_waiter.py— unit tests at the changed layer: resolve of a live future also tombstones the verdict; a mismatched-fingerprint tombstone is dropped at consume; invalid payloads tombstone nothing.Also ran the surrounding elicitation suites (91 passed) to catch regressions. Independent cross-vendor (codex) review ran pre-PR, 2 rounds: its blocking finding (stale-verdict replay when a JSON-RPC id is reused within the TTL) drove the fingerprinting; its residual note (identical-content re-ask within the TTL adopts the verdict) is the pre-existing, intended retry semantics of the tombstone mechanism, unchanged here.
Demo
Before/after recordings (web UI journey and terminal test runs) are preserved in the resolve workflow's artifact bundle (
before-gap-verdict-lost.mp4,after-gap-verdict-delivered.mp4,before-resolve-2xx.mp4,after-resolve-delivered.mp4) and attached to the Linear ticket OMNI-1233.Type of change
Test coverage
Coverage notes
Unit, integration, and E2E-UI coverage all assert the same fail→pass transition at their own layer; no manual-only verification.
Changelog
Approving a codex gate while its poll was silently severed no longer loses the answer — the harness's next poll receives it instead of re-asking.