Skip to content

Keep a stuck graphics driver from holding the window at start - #206

Merged
whoisaldo merged 13 commits into
release/0.3.5-candidatefrom
fix/gpu-detect-timebox-035
Nov 8, 2026
Merged

whoisaldo merged 13 commits into
release/0.3.5-candidatefrom
fix/gpu-detect-timebox-035

Conversation

@whoisaldo

@whoisaldo whoisaldo commented Nov 7, 2026 •

Copy link
Copy Markdown
Member

Supersedes #166, which was held after 0.3.4 for two review findings. Both are fixed here, along with the findings of two reviews of this PR.

What changed

A graphics driver deadlock on an AMD laptop held the host's startup for 62 s inside GpuInfo::detect, and the window showed nothing for that minute. #166 time-boxed detection. Review found that the window still made its own graphics calls before the first paint: AnalyzerApp::new lists the screens, and the renderer and wgpu create their own DXGI factories. It also found that a timeout left the whole session on the processor (libx264, or libx265 with HEVC) with no notice.

Startup lane. GpuInfo::detect runs under the stall guard in a new gpu_detect lane: warning at 1 s, limit at 4 s (from #166). The same lane holds the screen lists for the startup banner (main.rs) and the window (AnalyzerApp::new), and the window's DXGI adapter list. After one hangs, the others are refused at once instead of each waiting out its own limit. A refused or timed-out screen list is empty: the banner says unknown, and the screen picker fills in when Settings opens.

Before the window (gui/driver_wait.rs, before_window). The window's thread lists adapters, creates its device and draws through the driver, and Windows can't stop a thread stuck there. So before run_native:

  1. The graphics cards are listed through DXGI on another thread, in the startup lane, once nothing is stuck there, until one list returns in time. Only that list configures the renderer; crash avoidance keys its history by it, and the window keys its adapters by it until it is confirmed drawing. A stuck or timed-out list never gives an empty key, which would pick a known-bad backend again and record a crash under an empty key.
  2. A probe makes the calls wgpu makes first, once, on another thread, in a lane of its own (gpu_probe, same limits): an instance with this launch's backends lists its adapters, and DX12 creates a device on each to describe it. The window opens once it returns.

The list and the probe are one call each, waited for as long as they run: past the 4 s limit the notice goes up, and a driver that answers late still opens the window (the late result is used, not thrown away and tried again).

Until the window draws (watch_first_paint, Watch::look). The window's own device and swap chain, and a surface it recreates after a failed acquire, can't be moved off its thread. A watcher runs until the renderer confirms the window drawing, and raises the notice when either:

  • the window's thread has begun no UI pass for 4 s (the window repaints at least every 200 ms), or
  • it has run UI passes for 12 s without being confirmed (a working window is confirmed at about 3.5 s).

Both are counted from the latest of the watch's start and the last time the window was held: minimized (IsIconic), or its title bar or system menu held (GetGUIThreadInfo: GUI_INMOVESIZE, GUI_INMENUMODE). eframe runs no UI pass for a minimized window, so the watcher checks the window itself, live.

Confirmed drawing. The renderer confirms the window only after frames for a while with no surface error, no skipped frame and no new device, visible, and then a screenshot: egui-wgpu (also #201's vendored painter) answers one only from a frame it acquired, drew and submitted. Each ask carries its span (a new one after each surface error, skipped frame or device) and a request id, and only an answer to the current span's asks confirms; a late answer from before a change proves nothing.

The notice. Whenever the window waits on the driver, a message box says the graphics driver is not responding and that the window will open by itself. Windows draws it without the graphics driver.

  • OK hides it for 15 s; it comes back while the wait goes on.
  • Cancel quits through the app's normal quit path (VDD off, input released, logs flushed). The window's thread may never return, so that path is one closure that main.rs and the watcher share, run once by whichever comes first. A Cancel also clears the renderer's crash marker, so the next launch does not count a deliberate quit as a failed start.
  • The end of the wait, the user's answer and showing the box change one state under one lock: a Cancel that comes as the window draws still quits, and the box never goes up after the wait ended.
  • The box closes when the window is confirmed drawing, on a thread of its own, so the window's thread never waits on it.

Crash markers. The DXGI list leaves a marker before it starts and names its thread once it runs; the probe names its thread while it runs. A driver crash in either is the window's, as one on the window's thread is. A launch that dies listing makes the next ones key the GPUs by wgpu's driver text instead, until clean launches retry the list. The marker and a quit share one lock, so a worker still in the driver can't save it again after a Cancel. After the window is confirmed, a renderer rebuilt later (after a lost device) lists adapters in the gpu_facts lane, so it neither waits behind nor holds the startup lane.

The late result is used. A detection given up on keeps what it found when it returns (gpu::take_late_detection). The next pipeline (Restart, or any later restart) takes it and encodes on the graphics card. Until then the processor encodes, and the Stream page shows the software-encoding notice from the start: "The graphics driver did not respond when EternalMonitor started, so the processor encodes the video for now. … Click Restart to try the graphics card again." Its button is Restart. A detection that answered and found no graphics card, or libx264 chosen in Settings, shows no notice, as before.

This doesn't touch the encoder_open stall path or the window's handling of device loss once it is up.

Tests

  • gpu::tests: a stuck detection gets the software facts, marked as stalled, at its limit, with one stall=gpu_detect WARN. Once it returns, its facts are kept for later pipelines. A detection that returns in time keeps its facts; a failed one is not marked.
  • gui::driver_wait::tests. The steps before the window order events on channels and use 60 s limits for steps that answer; the watcher's decisions are a step function (Watch::look) the tests call with each state in turn:
    • while detection is stuck, nothing lists and the notice is up;
    • a list or a probe that answers after its limit is waited for, its late result used, once;
    • Cancel while a call hangs opens no window and leaves its lane taken until it returns; Cancel as the driver answers still quits;
    • OK hides the notice until the reminder;
    • a failed first acquire then a stuck recreation: the notice 4 s after the last pass;
    • UI passes without a proof of a presented frame: the notice at the 12 s bound;
    • a held window (minimized, dragged) suspends both bounds, which count again from its release;
    • controlled interleavings: a Cancel that comes with the window drawing still quits; a window that draws while the watcher is about to raise shows no notice.
  • renderer::tests: confirmation needs the screenshot proof, and frames that present nothing are never confirmed; a late answer from before a surface error, or from before skipped frames and a new device, proves nothing; a quit from the notice leaves no marker and no late worker writes it again; a launch that dies listing is recorded and skips the list next time; after the window is confirmed, the adapter list leaves the startup lane.
  • supervisor::tests::a_late_detection_is_used_by_the_next_pipeline_only_after_a_stall.
  • tests/gpu_detect_fallback.rs runs the real encode stage on the stalled facts. The processor encodes with the notice flags set; libx264 chosen in Settings and a detection that found no card set neither. With the old set_software_fallback(false) it fails with (false, true).
  • gui::tests::a_driver_that_did_not_answer_at_startup_is_named_with_a_restart: the stalled detection is named first, before custom settings and the driver hints for NVIDIA, AMD and Intel from Stream on libx264 when a hardware encoder open hangs, remember failed encoders, change Quick Sync's bitrate in place #202.

Checks

  • Windows (RTX 5080 + AMD iGPU), at the pushed head, cargo test -p eternal-host --no-fail-fast: 1225 passed, including all 13 window_device_loss tests; cargo clippy -p eternal-host --all-targets -- -D warnings is clean. The AMF hardware tests failed in that run: AMF did not open on the AMD iGPU of the shared test PC. In an A/B right after, amf_encoder passed 5 of 5 at this head and failed 2 tests in 1 of 5 on the candidate (241fa1b).
  • macOS: cargo test -p eternal-host (all 1240), cargo clippy --workspace --all-targets -- -D warnings, cargo fmt --check.

Native runs (Windows 11, RTX 5080, at 8f660c2)

Test copies of the host, debug builds, synthetic capture, against the candidate. A local test-only patch held detect_from_dxgi for 20 s, like a stuck driver, in both builds.

First thing on screen Window
Candidate, detection held 20 s 22.0 s (the window) 22.0 s
This PR, detection held 20 s 4.2 s (the driver notice) 21.5 s
This PR, normal start 1.1 s 1.1 s, confirmed drawing 3.2 s later
This PR, minimized 0.5 s after the window appeared, watched 60 s no notice confirmed 0.3 s after it was restored
  • Minimized: the host logged no "graphics driver is not answering" line while minimized. After the restore, Window renderer is drawing came once the screenshot proof was answered.
  • Held detection: stall=gpu_detect after_ms=4019, then the notice. The list and the probe answered (The graphics driver answered call="gpu_detect" waited_ms=16019, then call="gpu_probe"), and the window opened with the software-encoding notice.
  • From the previous round (e5cc8e0): with the renderer stuck 30 s on the window's thread, the notice went up at 4.4 s by the host's log. After Restart from a held start, the log shows this pipeline uses its graphics card … encoder=h264_nvenc.

Release notes

EternalMonitor's window no longer waits up to a minute to appear when the graphics driver stops responding at startup. It says the driver isn't responding, opens once the driver answers, and the Stream page explains why video uses the processor until Restart.

Startup's GPU detection runs under the stall guard in a new gpu_detect
lane (1 s warning, 4 s limit), as in #166. The same lane now also holds
every other graphics call before the window's first paint: the banner's
and the window's screen lists, the renderer's DXGI adapter list, and a
wgpu probe that lists the window's adapters on another thread before the
window's thread makes its first graphics call. After one call hangs, the
others are refused at once.

While the probe does not return, or an earlier call is still stuck, a
message box that Windows draws without the graphics driver says why the
window is not there yet, and the window opens once the driver answers.
The marker names the probe's thread while it lists adapters, so a driver
crash there still counts against the window's backend.

A detection given up on keeps what it finds when it returns. The next
pipeline, as after Restart, uses that graphics card. Until then the
processor encodes, and the Stream page shows the software-encoding
notice, saying the driver did not respond at startup.
@whoisaldo whoisaldo added the 0.3.5 After 0.3.4 label Nov 7, 2026
…x/gpu-detect-timebox-035

# Conflicts:
#	host/src/supervisor.rs
The candidate's log-capturing tests call prepare_log_capture first, so a
stall line another test's thread logged before any subscriber still
reaches the capture. The detection test asserts that line too.
…x/gpu-detect-timebox-035

# Conflicts:
#	host/src/supervisor.rs
Review of the first version found five gaps; this closes them.

- Crash avoidance read an empty GPU list while detection was stuck: its
  DXGI list was refused in the startup lane, so a known-bad backend was
  picked again and a crash recorded under an empty key. The window now
  waits for the startup lane to free first, then configures the renderer,
  then runs the wgpu probe, in a lane of its own (stall=gpu_probe).
- The window's thread still creates its device and swap chain unwatched.
  A watcher now runs from run_native until the first frame is presented;
  past the 4 s limit it shows the same notice, closed by the first frame.
  Cancel there runs the app's quit path, now one closure main.rs and the
  watcher share, and ends the process.
- After OK the notice never came back, so quitting was impossible while
  the driver stayed stuck. OK now hides it for 15 s. A Cancel that lands
  as the driver answers still quits.
- After the first frame, the renderer's DXGI list (as a renderer rebuilt
  after a lost device makes it) uses the gpu_facts lane, not startup's.
  While it lists, the crash marker names its thread, so a driver crash
  there is still the window's.
- The tests no longer depend on timing: the notice, probe and watcher
  tests order events on channels and a hand-moved clock, and the in-time
  detection test has a 1 s warning threshold.
…x/gpu-detect-timebox-035

# Conflicts:
#	host/src/main.rs
…x/gpu-detect-timebox-035

# Conflicts:
#	host/src/encoder/mod.rs
#	host/src/gui/mod.rs
#	host/src/gui/page_stream.rs
#	host/src/gui/tests.rs
#	host/src/stats.rs
…that returned

Second review of #206 found five more gaps; this closes them.

- The first-paint watcher stopped at the second UI pass, which egui
  counts whether or not a frame was presented: egui-wgpu returns from a
  paint without presenting after a failed acquire, and recreates the
  surface at the start of the next one. The watch now ends only when the
  renderer confirms the window drawing (frames for a while with no
  surface error, visible). The notice goes up when the window's thread
  has begun no UI pass for 4 s; the window repaints at least every 200 ms,
  and a minimized one is left alone.
- Configure could still commit an empty GPU key when its own DXGI list
  timed out. The list now runs before configure, in the startup lane,
  until one returns in time; configure gets that list, and the window
  keys its adapters by it until it is confirmed drawing.
- A Cancel from the watcher left the renderer's marker, so the next
  launch counted a deliberate quit as a failed start. Cancel now clears
  it, and the marker and that quit share one lock, so a worker still in
  the driver cannot save it again.
- The end of the wait, the user's answer and raising the notice now
  change one state under one lock: a Cancel that comes as the window
  draws still quits, and no notice goes up after the wait ended.
- The first DXGI list had no marker. It now leaves one before it starts
  and names its thread once it runs, until a later step replaces it. A
  launch that dies there makes the next ones key the GPUs by wgpu's
  driver text instead, until clean launches retry the list.
… a slow driver

Third review of #206 found five more gaps; this closes them.

- The watcher counted a minimized window as stuck: eframe runs no UI
  pass for one, so the renderer never saw it minimized, and a healthy
  host got the driver notice 4 s after Win+D. The watcher now checks the
  window live (IsIconic, or the move/size and menu loops Windows runs
  while the title bar is held), and counts the 4 s from the latest of its
  start, the last UI pass and the last time it saw the window held.
- Confirming the window could still pass with nothing presented: after a
  failed surface recreation egui-wgpu's paints return early with no
  surface error. The renderer now asks for a screenshot once the window
  has drawn long enough, and confirms only when it arrives; egui-wgpu
  takes one only from a frame it acquired, drew and submitted.
- A driver that always answers after 4 s never opened the window: each
  try was given up on and started again. The list and the probe are now
  one call each, waited for as long as they run; the notice goes up at
  their limit, and a late answer opens the window.
- Tests whose steps answer use 60 s limits, and the watcher's panics
  reach the test.
- The notice box closes on a thread of its own, so the window's thread
  never waits on it.
…x/gpu-detect-timebox-035

# Conflicts:
#	host/src/gui/renderer.rs
…x/gpu-detect-timebox-035

# Conflicts:
#	host/src/gui/mod.rs
#	host/src/main.rs
#	host/src/stall.rs
Fourth review of #206 found three more gaps; this closes them.

- A window whose paints return without a surface keeps running UI
  passes, so the watcher's 4 s without a pass never came, and its
  screenshots were asked for again and again with no notice. The watcher
  now also raises the notice when the window has run UI passes for 12 s
  without being confirmed drawing (a working one is at about 3.5 s). Both
  bounds count from the latest of the start and the last time the window
  was held, so a minimized or dragged window suspends them.
- Any screenshot marked as the proof was accepted, also a late one from
  before a surface error or a new device. Each ask now carries its span
  (a new one after each surface error, skipped frame or device) and a
  request id, and only an answer to the current span's asks confirms.
- The watcher's decisions are a step function the tests call with each
  state in turn, instead of a thread they hoped had looked.
@whoisaldo
whoisaldo marked this pull request as ready for review November 8, 2026 10:45
@whoisaldo
whoisaldo merged commit 21eb8a9 into release/0.3.5-candidate Nov 8, 2026
10 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

0.3.5 After 0.3.4

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants