fix: clean Ctrl-C shutdown — daemon teardown + quiet SSE cancellation#166
Merged
Conversation
start --serve died before its supervisor teardown ran because uvicorn 0.49's Server.serve() unconditionally re-raises the captured SIGINT/ SIGTERM (with the default disposition) right after serve() returns. _run_daemon now overrides the server's capture_signals with a no-op and owns the signal itself (there is no install_signal_handlers config flag on this uvicorn version), so its own finally (scheduler shutdown + supervisor.shutdown()) always completes. Separately, a connected dashboard/SSE client made every shutdown log a multi-screen CancelledError/anyio.WouldBlock traceback at ERROR level: uvicorn force-cancels the SSE connection past the graceful-shutdown timeout, and Starlette's StreamingResponse (uvicorn 0.49 is stuck on its older, task-group-based implementation) re-raises that unconverted. Both /api/events generators now end cleanly on cancellation (the merged one also cancels any outstanding per-iteration getter task), and a logging filter drops uvicorn's own "Exception in ASGI application" record for that specific, expected cancellation shape. Verified with a pty driver sending a genuine keyboard Ctrl-C against a connected SSE client: dashboard and start --serve both exit 0 in under 5s with their "stopped" banner and no ERROR-level traceback; headless start + SIGINT is unchanged.
4d0cca5 to
0fb8f4e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two related shutdown defects, both against uvicorn 0.49's signal/streaming behavior:
Defect A —
start --serveloses its teardown. uvicorn 0.49'sServer.serve()unconditionally wraps itself incapture_signals(), which restores the pre-serve()signal disposition and re-raises the captured SIGINT/SIGTERM with that (default) disposition right afterserve()returns — killing the process before_run_daemon's ownfinally(scheduler shutdown +supervisor.shutdown()) can run. Fixed by overriding the builtServerinstance'scapture_signalswith a no-op context manager (there is no publicinstall_signal_handlersconfig flag on this uvicorn version — confirmed it raisesTypeErrorat construction) and installing the daemon's ownSIGINT/SIGTERMhandlers on the running loop aroundawait server.serve(): first signal setsshould_exit, second setsforce_exit. The headless path (no--serve) is untouched.Defect B — traceback spam on every shutdown with a connected dashboard. With a browser/curl holding
/api/events, the 3s graceful-shutdown timeout force-cancels the connection and uvicorn logs a multi-screenCancelledError/anyio.WouldBlocktraceback at ERROR level. Both SSE generators now catchasyncio.CancelledErroraround their streaming loop and end cleanly (the merged generator also cancels any outstanding per-iteration getter task). This alone isn't sufficient, though: the traceback actually originates in Starlette's ownStreamingResponse.__call__(ananyiotask group racing "stream the body" vs "listen for disconnect" — the code path uvicorn 0.49 is stuck on since it declares ASGIspec_version: "2.3", below the2.4Starlette checks for its newer implementation), confirmed by reproducing the identical traceback against a bare FastAPI app with no custom middleware at all. So a smalllogging.Filteronuvicorn.erroralso drops the "Exception in ASGI application" record specifically when the underlying exception is a plainCancelledError— any real exception is still logged normally.Verified (pty driver, genuine keyboard Ctrl-C, SSE client attached)
trading-bot dashboard+ SSE + one ^C → exit 0 in ~4.2s,dashboard stoppedprinted, only the expected "Cancel 1 running task(s)..." line, no CancelledError traceback.trading-bot start --serve --interval 3600+ SSE + one ^C → exit 0 in ~4.2s,daemon stopped (all strategies shut down)printed, no traceback.trading-bot start --interval 3600(headless) + SIGINT → unchanged,daemon stoppedin ~1s.Test plan
python -m pytest— 1170 passed (added: single/merged SSE cancellation tests,_run_daemonsignal-ownership test, logging-filter unit tests)ruff check trading_bot/cleanruff format --check trading_bot/cleanmypy trading_bot/clean🤖 Generated with Claude Code