Repository navigation
fix(hub): cancel overdue reviewers and stuck managers (CL-12015) - #813
TheGreatAxios merged 3 commits into
Conversation
|
Triage: Pass No changes needed from you. A maintainer will review this pull request. Notes:
|
Self-Review SummaryReviewed Issues Found and Resolved
Known, not addressed
|
Self-Review Follow-upTriage reported no defects. The items listed as "Known, not addressed" earlier are now fixed in 8d329f6:
The reconcile order ( Rerun: check, lint, 936/936 tests, chain 22/22. |
| for (const afterMs of [MINUTE, 11 * MINUTE, 2 * 60 * MINUTE, 7 * 60 * MINUTE]) { | ||
| expect(await reconcile(afterMs)).toMatchObject({ status: "running", runId: "run_manager" }); | ||
| } |
There was a problem hiding this comment.
P3 · Test expects a manager idle 7h between passes to stay running, but the new idle sweep cancels it after 6h
After done(..., { wait: true }) the claim is live with no open pass. The test then expects the head to stay running with this run at 2h and 7h. The new endOverdue sweep cancels a manager that has neither claimed nor closed a pass within MANAGER_POLL_LIMIT_MS (2 × 3h = 6h). This test does not run the sweep, so the 7h assertion checks a state production would not reach. Under its new title ('runs for hours between passes'), it documents behaviour that the PR's own idle-manager rule rules out.
Reconcile at 7 * 60 * MINUTE after a wait done, with no new pass, which is more than MANAGER_POLL_LIMIT_MS (6h).
Resolve this comment. Triage will not finish while it is open.
Summary
isStuck()was dead code: manager runs were stampedstartedAt: now()on every pass. It now readspassAt, the time the current run took its open pass (the later ofpass.takenAtandclaim.claimedAt), so a manager that resumes a gone run's pass is timed from its takeover.endOverduesweep runs each reconcile pass, besideendNeverRan, and cancels through stockrequestCancellation:stuckAfterMs.MANAGER_POLL_LIMIT_MS. The manager's own loop (PASSES) would have ended by then, so it is wedged.passAtinstead of overwritingstartedAt, becausesinceQueued/sinceStalefilter bystartedAt.Changes
apps/hub/src/github/reconcile-plan.ts:isStuckusespassAt.apps/hub/src/manager-state.ts:endOverdue. The reviewer query joinsworkflow_run, so reviewers that already ended cost nothing.ManagedPull.live.passAtreplaceslive.pass.apps/hub/src/github/manager-runs.ts: passeslive.passAtthrough.apps/hub/src/server.ts: runsendOverduewith the reconcile policy'sstuckAfterMs, and with the reconcile interval as the reviewer grace.packages/triage-contracts/src/triage-state.ts:MANAGER_LIFETIME_MS(moved from triage-manager'sLIFETIME_MS) andMANAGER_POLL_LIMIT_MS.PASSESderives from the latter.startedAtare updated.docs/ARCHITECTURE.md: describes the sweep.Testing
bun run checkandbun run lintare clean.apps/hub(including Postgres),tests/model,packages/triage-managerandpackages/triage-contracts: 936/936.bun run test:chain: 22/22.Closes CL-12015