fix: cut playback CPU usage using GSK renderer, zero-copy hwdec, and event-loop busy-polling#108
fix: cut playback CPU usage using GSK renderer, zero-copy hwdec, and event-loop busy-polling#108AltarBeastiful wants to merge 5 commits into
Conversation
Two `glib::idle_add_local` sources ran on every main loop iteration: one polling mpv's event queue with `wait_event(0.0)`, one polling a flume channel with `try_recv()`. An idle source is rescheduled immediately, so the main loop never slept and both spun continuously, burning CPU when idle and during playback. Drain the mpv queue from a `timeout` source at ~60 Hz instead, which lets the main loop sleep between ticks, and drain it fully each tick rather than one event per iteration. `mpv_set_wakeup_callback` is deliberately not used to drive this: it is only a best-effort hint (mpv coalesces property changes and fires once for multiple events) and it races with `wait_event` clearing the pending flag, so relying on it alone latches after the first burst and playback stalls on the loading screen. Replace the render poll with a task awaiting `recv_async()`, coalescing a burst of update callbacks into a single `queue_render()`.
The Stremio web UI enables hardware decoding by sending `hwdec=auto-copy`, which decodes on the GPU but copies every frame back to system memory and re-uploads it for rendering (the "copy-back" path). That copy is CPU- and bandwidth-heavy and is why playback here costs more CPU than mpv/VLC. Remap `hwdec=auto-copy` to `auto-safe`, which keeps frames on the GPU via a zero-copy interop (VAAPI/Vulkan/NVDEC depending on the driver) and falls back to software decoding when none is available, so playback can never break. Verified on Radeon 680M: auto-copy -> "hardware decoding (vulkan-copy)"; auto-safe -> "hardware decoding (vulkan)" with frames kept on the GPU.
mpv defaults to software decoding unless `hwdec` is set. The shell set no default and only ever received `hwdec` from the web UI at runtime (after the render context already exists), which did not reliably bring up the GL hwdec interop — so playback ran on the CPU even with "hardware decoding" enabled. Set `hwdec=auto-safe` in the initializer so mpv loads the zero-copy interop when the render context is created. Verified in the real libmpv render path: [libmpv_render] Loading hwdec driver 'vaapi' [libmpv_render/vaapi] Using EGL dmabuf interop via GL_OES_EGL_image [libmpv_render/vaapi/vaapi] Initialized VAAPI
GTK 4.22 defaults to the Vulkan GSK renderer. Our video is an OpenGL GLArea (mpv's vo=libmpv render API), and compositing a GL texture through the Vulkan renderer forces an expensive per-frame GL->Vulkan copy. Measured on an AMD Radeon 680M / Mesa, playing a 1080p30 H.264 clip with hardware decoding (vaapi) confirmed active, using a probe that reproduces the shell's exact render path (GLArea + libmpv render context, no WebKit). Five interleaved rounds per renderer, each a 12s steady-state window after warmup, medians, on an otherwise idle machine: GSK_RENDERER unset (4.22 default, GskVulkanRenderer) ~24% CPU GSK_RENDERER=gl (GskGLRenderer) ~7.8% CPU For reference bare `mpv --vo=gpu` on the same file, measured the same way, was ~7.1%, so decode was never the problem: on the GL renderer the whole GLArea/GSK path costs ~0.7 points over mpv's own output, while the Vulkan default adds ~17. Default GSK_RENDERER to `gl`, respecting any renderer already chosen by the user or by data/stremio.sh (which sets `opengl`, the same renderer on 4.22, for Nvidia), so playback is GPU-composited instead of copied through Vulkan.
From the Nvidia investigation (GTX 1060, driver 580, Plasma 6 Wayland): DEVLOG §10-15 - §10-11: `gl` alone does not fix Nvidia. The pinned core is WebKitGTK falling back to Skia *CPU* rendering (the UI snapshots to a GskCairoNode), which GSK then re-composites over the video every frame on the CPU. On Mesa Skia GPU works (texture node) -> cheap; that is the entire Nvidia-vs-AMD split, and it lives inside WebKit, upstream of GSK. Confirmed with GDK_DEBUG=offload + perf; verify per-box with `webkit://gpu`. - §12: the playback artifacts are a *separate*, branch-only regression - forcing hwdec=auto-safe (zero-copy), verified on VAAPI/Mesa only, breaks Nvidia's nvdec interop. The same commits ride in upstream PR Stremio#108. - §13: the Flatpak launcher regression (e6fb216) and its fix. - §14: a change-by-change regression audit vs upstream main (one real regression: hwdec=auto-safe on Nvidia; one silent improvement: the loop fix). ADR-0003 (Proposed): present the video as a dmabuf GdkPaintable on a GtkGraphicsOffload subsurface so video frames bypass GSK. Fixes the offload-eligible matrix incl. Nvidia at integer scale; fractional-scale Nvidia stays bottlenecked upstream (GTK declines offload at fractional scale). ADR-0004 (Proposed): scope zero-copy hwdec to interops we have verified, leaving Nvidia on the web UI's copy-back path until its interop is validated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GqwqM5jcTswvJYymbpdaY8
|
Built this branch (flatpak Devel manifest) and tested on the #107 machine: Core Ultra 9 185H / Meteor Lake, Fedora 43, GNOME Wayland, mostly on battery. The decode fix works. hwdec now comes up at startup on its own, no shim or override needed: [vd] Trying hardware decoding via hevc-vaapi. Using hardware decoding (vaapi). ~3 hours of 4K HEVC 10-bit DV playback, 100+ seeks: zero decode errors, zero fallbacks. Playback CPU went from ~2.5 cores on 1.0.3 (software) to ~0.3 core, idle CPU from ~0.3 to ~0.05, whole-system battery draw ~16 W. The server's hw transcode probe also passes now (vaapi-renderD128 - it failed on 1.0.3). The UI freeze is still present but clearly unrelated to this PR: it hit twice during the test, once after ~3h at 1.25x speed with heavy frame dropping (Dropped: 4016 in the status line), once minutes into a session right after rapid pause/unpause. mpv is healthy both times - the log just stops mid-flow and the main thread spins at 100%: [cplayer] Set property: pause=true -> 1 [cplayer] Set property: pause=false -> 1 [cplayer] Set property: pause=true -> 1 (log ends here, UI frozen) Same behavior exists on 1.0.3, so it's #77 territory, not a regression from this branch. In short: on this hardware the PR does what it says. The freeze needs separate work. |
The web UI enables hardware decoding by sending hwdec=auto-copy (copy-back: decode on the GPU, copy every frame back to system RAM and re-upload it). On 4K HDR that copy dominated CPU — ~100% of a core on the proprietary Nvidia driver. Zero-copy keeps frames on the GPU. On Mesa, auto-safe already selects VAAPI dmabuf zero-copy. On Nvidia proprietary, auto-safe only offers copy-back (vulkan-copy), so request `nvdec` explicitly for CUDA-interop zero-copy. Validated in the flatpak runtime: clean 4K HDR image and ~100% -> ~23% of a core (~4x), matching the Mesa win. Refactored the hwdec selection into a single default_hwdec() helper used by both the mpv init and the web-UI remap, with a STREMIO_HWDEC override for testing and the libmpv CUDA-interop build requirement documented for downstream packagers (the .deb's libmpv must be built --enable-cuda-hwaccel --enable-cuda-interop with ffmpeg --enable-ffnvcodec, else nvdec falls back to copy-back).
acc53fd to
9d8ea17
Compare
NVIDIA: zero-copy
|
| decode mode | NVIDIA CPU (4K HDR) |
|---|---|
auto-copy / vulkan-copy (copy-back, before) |
~100% |
nvdec (zero-copy, this PR) |
~23% |
Clean image, no artifacts. That ~23% is the video render/present path (mpv + GSK + HDR tone-map at 4K), roughly the floor for this shell, matching the Mesa result. The change is a small default_hwdec() helper (NVIDIA → nvdec, Mesa → auto-safe), plus a STREMIO_HWDEC env override for testing decode modes.
Packaging note for future deb
nvdec zero-copy needs libmpv/ffmpeg built with ffnvcodec (nvdec / cuda-interop). These headers are freely redistributable (no proprietary CUDA toolkit, and libcuda is loaded from the driver at runtime), and they're on by default in recent Ubuntu, Debian, Arch and Fedora, so this "just works" on mainstream distros. Only minimal or stripped ffmpeg rebuilds omit it, in which case nvdec silently falls back to copy-back (~100% again).
- Build flags: mpv
--enable-cuda-hwaccel --enable-cuda-interop, ffmpeg--enable-ffnvcodec,libplacebo. (Distinct from the nonfree CUDA path (--enable-cuda-nvcc,scale_cuda,libnpp), whichnvdecdecode does not use.) - Verified with libmpv 0.41 / ffmpeg 7.1 (org.gnome.Platform 50), and libmpv 0.41 on Ubuntu 26.04.
- Also needs WebKitGTK ≥ 2.52.4 on NVIDIA: system WebKitGTK 2.52.3 (Ubuntu 26.04) gives a laggy UI and overlay rendering artifacts on NVIDIA, fixed in 2.52.4 / 2.52.5 and verified using the flatpak bundling 2.52.5. Independent of nvdec decode. See the reviewer note below.
Notes for reviewers
- The GSK renderer (
glvsopengl→Vulkan) is a wash on NVIDIA during real playback (~23% either way). TheGSK_RENDERER=glwin in this PR is a Mesa thing; it doesn't move the needle on NVIDIA. (data/stremio.shstill setsopenglon NVIDIA, which is harmless as it falls back to Vulkan.) - The UI overlay is NOT the cost on NVIDIA in the flatpak, it's a cheap Wayland subsurface (~1% difference between controls shown and hidden). (Open question: is the overlay delta similarly small on AMD/Intel? Worth a cross-platform check.)
- Test the flatpak, not a host build. A host build against system GTK/WebKit shows a pile of NVIDIA-only symptoms (laggy UI, renderer artifacts, a "stale overlay",
nvdec"artifacts") that do not reproduce in the flatpak. Root cause: the system WebKitGTK is older than the flatpak's. Ubuntu 26.04 ships WebKitGTK 2.52.3; the GNOME 50 runtime bundles 2.52.5. GTK is identical (4.22.4), so it's a WebKit micro-version gap, fixed by the rendering fixes in 2.52.4 / 2.52.5. A native .deb therefore needs WebKitGTK ≥ 2.52.4 on NVIDIA (until then, use the flatpak); the shipping flatpak (2.52.5) is clean. Measure only steady, non-buffering fullscreen playback (buffering reads artificially low).
Reproduce
flatpak run com.stremio.Stremio.Devel # nvdec auto-selected on NVIDIA
flatpak run --env=STREMIO_HWDEC=auto-copy com.stremio.Stremio.Devel # copy-back, for comparison
# play 4K HDR fullscreen, past buffering, then:
top -b -n2 -d 3 -p "$(pgrep -x stremio | head -1)" | awk 'END{print $9"%"}'
# confirm mode: Using hardware decoding (nvdec). VO: [libmpv] 3840x2160 p010
Playback in the GTK4 shell burned several times more CPU than
mpvor VLC playing the same file. This turned out to be four independent causes, each fixed in its own commit and each reviewable on its own.Should fix #103 and possibly #107 .
Main loop sleep (
8aeb9b)Actually not in playback itself but in the idle app. Two
glib::idle_add_localsources ran on every main loop iteration :wait_event(0.0)try_recv().An idle source is rescheduled immediately, so both spun continuously, burning CPU even when idle. The mpv queue now drains from a
timeoutsource at ~60 Hz (fully each tick, not one event per iteration), and the render poll is a task awaitingrecv_async()that coalesces bursts into onequeue_render().Note
mpv_set_wakeup_callbackis deliberately not used to drive this: it is best-effort only (mpv coalesces property changes and fires once for several events) and it races withwait_eventclearing the pending flag, so relying on it latches after the first burst and playback stalls on the loading screen.Decoding ran on the CPU (
a7597b7)mpv defaults to software decoding unless
hwdecis set. The shell set no default and only receivedhwdecfrom the web UI at runtime, after the render context already existed, which did not reliably bring up the GL hwdec interop.hwdec=auto-safeis now set in the initializer, so mpv loads the zero-copy interop when the render context is created:Hardware decoding was copy-back (
c1123bc)The web UI enables hardware decoding by sending
hwdec=auto-copy, which decodes on the GPU but copies every frame back to system memory and re-uploads it for rendering. That copy is CPU- and bandwidth-heavy.auto-copyis remapped to the platform's zero-copy mode (auto-safeon Mesa,nvdecon the proprietary NVIDIA driver, see the NVIDIA update below), which keeps frames on the GPU via zero-copy interop (VAAPI/Vulkan/NVDEC depending on driver) and still falls back to software decoding when none is available, so playback cannot break.The Vulkan GSK renderer composited an OpenGL GLArea (
951be48) => Main cost saveGTK 4.22 defaults to the Vulkan GSK renderer. The video is an OpenGL
GLArea(mpv'svo=libmpvrender API), and compositing a GL texture through the Vulkan renderer forces a per-frame GL->Vulkan copy that dominates playback CPU even with hardware decoding active.GSK_RENDERERnow defaults toglbefore GTK initializes.Measurements
AMD Radeon 680M / Mesa, GTK 4.22.4, 1080p30 H.264,
hwdec=vaapiconfirmed active, fps held at 30.0 throughout. Measured with a probe reproducing the shell's exact render path (GLArea + libmpv render context, no WebKit), self-sampling CPU from/proc/self/stat. Five interleaved rounds per renderer, each a 12s steady-state window after warmup, medians, on an otherwise idle machine:GSK_RENDERERunset: GTK 4.22 default,GskVulkanRendererGSK_RENDERER=gl:GskGLRenderer(this PR)mpv --vo=gpu, same clip, same method (reference floor)Decode was never the problem: on the GL renderer the entire GLArea/GSK compositing path costs about 0.7 points over mpv's own output, while the Vulkan default adds about 17.
Verified end-to-end in the real app, not just the probe. See logs below:
Benchmarks per platform
AMD and Nvidia ran using the same 4K HDR file. Intel are infered from @KemalK comments.
Per process breakdown using this PR
nodestremioshellWebKitWebProcessGlobal CPU gains vs master
AMD master: avg 90.0% / peak 98.1% / min 81.9% over a 100 s window (Ryzen 7 PRO
6850U, Ubuntu 26.04, governor=performance). Intel from PR #103 (KemalK); per-process
split not captured. NVIDIA change =
nvdeczero-copy (was copy-back); AMD/Intel =GSK_RENDERER=gl+ VAAPI zero-copy.Compatibility notes
GSK_RENDERERdefault only applies when the variable is unset. So a renderer chosen by the user or bydata/stremio.shwill apply. An example with nvidia, which setsopenglstill is used.env::set_varisunsafeunder edition 2024. It runs at the very top ofmain, before any thread is spawned, so there is no concurrent access to the environment.glis the renderer's name since GTK 4.18.nglstill resolves toGskGLRendereras a deprecated alias but emits a warning; a name GTK does not recognize falls back to the Vulkan renderer this change exists to avoid.GSK_RENDERER=helplists the accepted names.Reproducing
The probe used for these numbers is kept out of this PR to avoid shipping benchmark code. It is on
cpu-debugin my fork:It prints
hwdec-currentthe achieved render fps, and its own CPU over the window. Check/proc/loadavgbefore trusting any run.Update for NVIDIA
Testing on the proprietary NVIDIA driver in the flatpak showed the NVIDIA cost was copy-back video decode, not the overlay. On 4K HDR,
hwdec=auto-copycopies every frame through system RAM -> ~100% of a core.Fixed in this PR: on the proprietary NVIDIA driver, request
nvdeczero-copy explicitly (auto-safeonly offers copy-back there); Mesa keepsauto-safeVAAPI zero-copy. Result: ~100% -> ~23% on 4K HDR, clean image.Requires libmpv built with CUDA interop (
--enable-cuda-hwaccel --enable-cuda-interop, ffmpeg--enable-ffnvcodec), relevant for the .deb. More details in the comment below.Note on system WebKitGTK version. The flatpak bundles WebKitGTK 2.52.5. On NVIDIA, running against Ubuntu 26.04 WebKitGTK **2.52.3 produces a laggy UI and overlay rendering artifacts that do not occur in the flatpak on the same driver and same GTK version at 4.22.4. It is fixed by the rendering fixes in WebKitGTK 2.52.4 / 2.52.5, so a .deb needs WebKitGTK ≥ 2.52.4 on NVIDIA. Until it the fix reaches distros NVIDIA users should use the flatpak. The
nvdecdecode win above is independent and applies regardless.