Bound every action-log read path - #298
Open
ndisidore wants to merge 16 commits into
Open
Conversation
github-actions
Bot
added
workshop/frontend
Opening a large workspace replayed the whole action history before the UI was usable, overloading the Durable Object. Replace the unbounded reads with two paged RPCs: - scanPendingActions(): bounded id-cursor scan for pending records, with a fixed exclusive throughId captured on the first page (subscribe-then-scan contract, mirroring getChatHistory). - listActions() (repurposed; it had zero callers): resolved history, newest-first by id, with a raw-records scan cap so filtered pages stay bounded. UseOverseerInterface answers both with inert empty terminal pages, matching the speculative-call pattern of subscribeToActions.
#drainOnce materialized the entire actions collection per drain. Page it instead: capture nextActionId as the scan bound, list one materialized page at a time (preserving the iterator-invalidation safety the old snapshot provided), and yield between pages so a long scan doesn't starve client RPCs. Actions created past the bound are folded in by drain()'s existing rerun flag via the creation path's own drain call. Gate/failure semantics are unchanged: a manual gate or a failed apply still halts the whole drain.
The history tab rendered the full replayed action log. Load it on demand
instead through a new useActionHistory hook: nothing is fetched until the
tab opens, pages continue via the server cursor ("Load older"), and live
resolutions from the shared subscription merge into the loaded id window.
Day groups now follow creation order (labels may repeat), and the counter
reports entries loaded rather than a total the client no longer knows.
The shared useActions store is untouched here; the replay itself is removed
in the next commit.
This is the commit that stops the full action-log replay on workspace open.
The shared useActions store now subscribes without startAfter (live entries
only) and pages a bounded scanPendingActions() loop for records that were
already pending. The subscription is initiated first — e-ordered calls on
one stub register the DB subscriber before the scan reads anything — and a
per-generation liveSeenIds set drops any scanned record already delivered
live (live wins; a scanned page can be stale by the time it arrives).
The store's shape changes accordingly: {status, pendingById, liveById}
instead of the full actionsById map. Consumers adapt: GadgetEditor derives
the hook signature from live entries (listHooks remains the authoritative
initial source) and pending counts from pendingById; the Activity review
tab drops its whole-store loading gate in favor of checking/error states;
ActivityNotifications says it's still checking when empty mid-scan.
useActionEntries replays only live-received records, which is sufficient
for chat: fetched messages arrive server-hydrated, and everything that
changed since arrived live.
Known quirk (accepted): openActivity routes to 'history' while the scan is
still checking and nothing pending has been found yet.
One real-DO call of scanPendingActions and listActions each, proving the @validateRpc wiring accepts the new option shapes. Placed after the reset tests so abortAllDurableObjects() doesn't tear down this session's DOs mid-flight (which leaks a canceled-context rejection).
subscribeToActions now replays currently-pending records itself: after registering the DB subscriber it sweeps the log in bounded pages with scheduler.wait(0) between them (the drainer's pattern), then fires ready(). Replay and live updates share one ordered stream, so the scanPendingActions cursor/throughId protocol, the client scan loop, and its live-wins dedup all go away; ready() = the subscribe RPC resolving. The legacy startAfter full-log replay is removed outright (that path IS the overload bug); the parameter stays in the signature, ignored, for stale in-flight clients. listActions loses its client-supplied limit — page size is the server constant.
subscribeToChat hydrated actionLog only in the storage add() hook, so the update() leg and the reconnect catch-up scan delivered action messages with actionLog undefined, which the client renders as a blank card. Hydrate in deliverMessage so all three legs share it.
The pending-only replay never mentions an action that resolved while the client was away, so a cached card could stay 'pending' forever and keep the composer blocked. On each new overseer stub, sweep the cached action-message index and re-fetch cards that are blank or still pending via getChatMessage, guarding against regressing a card already resolved by a faster channel.
Pre-deploy clients pass startAfter and build their whole history view from replay; the pending-only replay left them rendering an empty activity log with no way to nudge them onto the new protocol. Presence of startAfter now switches the replay to every record; the value stays ignored. Also document the bare ActionHistoryFilter export.
A throwing entry listener on the shared client store broke the fan-out for every other consumer; guard each listener and log. On the server, a subscriber that fails mid-replay now rejects the subscribe call (the client's error signal) instead of silently returning a dead subscription.
A failed non-first page previously only console.error'd, leaving the Load older button looking inert. Track loadMoreFailed in the hook (first-load failures still own status: 'error') and render an inline retry row in the Activity history view. The cursor is untouched on failure, so retry re-requests the same page.
The header treated the shared subscription's 'error' status as all-clear
('Nothing is waiting on you.'). Thread isError into ActivityNotifications
and render the failure copy instead; pendings gathered before the failure
still render via the non-empty branch.
…dicate deliverMessage re-implemented #getChatMessageForClient's action-log lookup and had already diverged from it (the helper unconditionally hydrates attachments); route it through the helper. The helper's body has no awaits, so it drops async and deliverMessage issues subscriber.message() synchronously, keeping the documented messages-before-metadata delivery order intact across the metadata()/deleted() callbacks (the client's provisional-stream mop-up on activeAgent unset relies on it). listActions' type filter moves to matchesActionHistoryFilter in workshop-shared so the server page filter and the client live-merge can't drift.
…sumers - ActionsState now exposes one sorted readonly pending array instead of pendingById + entriesById; the byte-identical createdAt||id sorts in Activity and ActivityNotifications collapse into the store's commit(), and GadgetEditor's hookSignature memo (the sole entriesById consumer, one full-log Map clone per committed frame) becomes a useActionEntries fold over just the bindHook entries. - ActivityNotifications subscribes via useActions itself (the store is ref-counted per stub), dropping the pendingById/isChecking/isError prop drilling and restoring the 3-state status union. - useActionHistory drops the never-read 'idle' status; Activity's initial-loading branch keys off status === 'loading', so hasMore means only what the server said. The duplicated Load-older button is one local component. - useActions/useActionHistory tests share one harness (entry factory, fake overseer, act/rAF root management) in action-test-harness.ts instead of ~70 drifting lines.
ndisidore
force-pushed
the
chore/scale-action-logs
branch
from
August 21, 2026 19:51
7a6dcf3 to
eef947d
Compare
ndisidore
force-pushed
the
chore/scale-action-logs
branch
from
August 21, 2026 20:12
03c86bd to
eef947d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reading the action log today means reading all of it: opening a workspace replays every record ever written to each client, the Activity pane holds the whole log in React state, and the auto-approval drain materializes the full table per run. Long-lived workspaces pay for this on every open, and the cost only grows.
This branch bounds every read path:
subscribeToActionsreplays only currently-pending records (paged, yielding between pages), then streams live updates. Resolved history moves to a newlistActionsRPC that pages newest-first under a raw-scan cap, so a log buried in resolved records returns short pages with a cursor instead of stalling the DO.AutoApprovalDrainerpages the log instead of materializing it.startAfterget the old full replay until they're retired.These reads are bounded but still O(log): the pending replay and the drain scan every record to find the pendings. A sparse pending-by-gatekeeper index removes that in
chore/pending-action-index, stacked on this branch as a follow-up PR if we deem it necessaryTested with unit suites for the replay, pagination, and drain paths, an integration smoke over the paged RPCs, and hook tests for the new frontend state.