Skip to content

fix: cut playback CPU usage using GSK renderer, zero-copy hwdec, and event-loop busy-polling#108

Open
AltarBeastiful wants to merge 5 commits into
Stremio:mainfrom
AltarBeastiful:fix/playback-cpu
Open

fix: cut playback CPU usage using GSK renderer, zero-copy hwdec, and event-loop busy-polling#108
AltarBeastiful wants to merge 5 commits into
Stremio:mainfrom
AltarBeastiful:fix/playback-cpu

Conversation

@AltarBeastiful

@AltarBeastiful AltarBeastiful commented Jul 16, 2026

Copy link
Copy Markdown

Playback in the GTK4 shell burned several times more CPU than mpv or VLC playing the same file. This turned out to be four independent causes, each fixed in its own commit and each reviewable on its own.

Should fix #103 and possibly #107 .

Main loop sleep (8aeb9b)
Actually not in playback itself but in the idle app. Two glib::idle_add_local sources ran on every main loop iteration :

  • polling mpv's event queue with wait_event(0.0)
  • polling a flume channel with try_recv().
    An idle source is rescheduled immediately, so both spun continuously, burning CPU even when idle. The mpv queue now drains from a timeout source at ~60 Hz (fully each tick, not one event per iteration), and the render poll is a task awaiting recv_async() that coalesces bursts into one queue_render().

Note mpv_set_wakeup_callback is deliberately not used to drive this: it is best-effort only (mpv coalesces property changes and fires once for several events) and it races with wait_event clearing the pending flag, so relying on it latches after the first burst and playback stalls on the loading screen.

Decoding ran on the CPU (a7597b7)
mpv defaults to software decoding unless hwdec is set. The shell set no default and only received hwdec from the web UI at runtime, after the render context already existed, which did not reliably bring up the GL hwdec interop. hwdec=auto-safe is now set in the initializer, so mpv loads the zero-copy interop when the render context is created:

[libmpv_render] Loading hwdec driver 'vaapi'
[libmpv_render/vaapi] Using EGL dmabuf interop via GL_OES_EGL_image

Hardware decoding was copy-back (c1123bc)
The web UI enables hardware decoding by sending hwdec=auto-copy, which decodes on the GPU but copies every frame back to system memory and re-uploads it for rendering. That copy is CPU- and bandwidth-heavy. auto-copy is remapped to the platform's zero-copy mode (auto-safe on Mesa, nvdec on the proprietary NVIDIA driver, see the NVIDIA update below), which keeps frames on the GPU via zero-copy interop (VAAPI/Vulkan/NVDEC depending on driver) and still falls back to software decoding when none is available, so playback cannot break.

The Vulkan GSK renderer composited an OpenGL GLArea (951be48) => Main cost save
GTK 4.22 defaults to the Vulkan GSK renderer. The video is an OpenGL GLArea (mpv's vo=libmpv render API), and compositing a GL texture through the Vulkan renderer forces a per-frame GL->Vulkan copy that dominates playback CPU even with hardware decoding active. GSK_RENDERER now defaults to gl before GTK initializes.

Measurements

AMD Radeon 680M / Mesa, GTK 4.22.4, 1080p30 H.264, hwdec=vaapi confirmed active, fps held at 30.0 throughout. Measured with a probe reproducing the shell's exact render path (GLArea + libmpv render context, no WebKit), self-sampling CPU from /proc/self/stat. Five interleaved rounds per renderer, each a 12s steady-state window after warmup, medians, on an otherwise idle machine:

render path CPU
GSK_RENDERER unset: GTK 4.22 default, GskVulkanRenderer ~24%
GSK_RENDERER=gl: GskGLRenderer (this PR) ~7.8%
bare mpv --vo=gpu, same clip, same method (reference floor) ~7.1%

Decode was never the problem: on the GL renderer the entire GLArea/GSK compositing path costs about 0.7 points over mpv's own output, while the Vulkan default adds about 17.

Verified end-to-end in the real app, not just the probe. See logs below:

Environment variable GSK_RENDERER=gl set, trying GskGLRenderer
Using renderer 'GskGLRenderer' for surface 'GdkWaylandToplevel'

Benchmarks per platform

AMD and Nvidia ran using the same 4K HDR file. Intel are infered from @KemalK comments.

Per process breakdown using this PR

Platform node stremio shell WebKitWebProcess Total
NVIDIA GTX 1060 / proprietary 580 @170% ~8% ~23% ~4% ~35%
AMD Radeon 680M (6850U) / Mesa ~15% ~12% ~5% ~32%
Intel Meteor Lake / Mesa - - - ~30%

Global CPU gains vs master

Platform Master This PR Gain
NVIDIA GTX 1060 / proprietary 580 @170% ~110% ~35% ~3×
Intel Meteor Lake / Mesa ~250% ~30% ~8×
AMD Radeon 680M (6850U) / Mesa ~90% ~32% ~2.8×

AMD master: avg 90.0% / peak 98.1% / min 81.9% over a 100 s window (Ryzen 7 PRO
6850U, Ubuntu 26.04, governor=performance). Intel from PR #103 (KemalK); per-process
split not captured. NVIDIA change = nvdec zero-copy (was copy-back); AMD/Intel =
GSK_RENDERER=gl + VAAPI zero-copy.

Compatibility notes

  • The GSK_RENDERER default only applies when the variable is unset. So a renderer chosen by the user or by data/stremio.sh will apply. An example with nvidia, which sets opengl still is used.
  • env::set_var is unsafe under edition 2024. It runs at the very top of main, before any thread is spawned, so there is no concurrent access to the environment.
  • On naming: gl is the renderer's name since GTK 4.18. ngl still resolves to GskGLRenderer as a deprecated alias but emits a warning; a name GTK does not recognize falls back to the Vulkan renderer this change exists to avoid. GSK_RENDERER=help lists the accepted names.

Reproducing

The probe used for these numbers is kept out of this PR to avoid shipping benchmark code. It is on cpu-debug in my fork:

cargo run --release --example glarea_probe --  [warmup_s] [window_s]

It prints hwdec-current the achieved render fps, and its own CPU over the window. Check /proc/loadavg before trusting any run.

Update for NVIDIA

Testing on the proprietary NVIDIA driver in the flatpak showed the NVIDIA cost was copy-back video decode, not the overlay. On 4K HDR, hwdec=auto-copy copies every frame through system RAM -> ~100% of a core.

Fixed in this PR: on the proprietary NVIDIA driver, request nvdec zero-copy explicitly (auto-safe only offers copy-back there); Mesa keeps auto-safe VAAPI zero-copy. Result: ~100% -> ~23% on 4K HDR, clean image.

Requires libmpv built with CUDA interop (--enable-cuda-hwaccel --enable-cuda-interop, ffmpeg --enable-ffnvcodec), relevant for the .deb. More details in the comment below.

Note on system WebKitGTK version. The flatpak bundles WebKitGTK 2.52.5. On NVIDIA, running against Ubuntu 26.04 WebKitGTK **2.52.3 produces a laggy UI and overlay rendering artifacts that do not occur in the flatpak on the same driver and same GTK version at 4.22.4. It is fixed by the rendering fixes in WebKitGTK 2.52.4 / 2.52.5, so a .deb needs WebKitGTK ≥ 2.52.4 on NVIDIA. Until it the fix reaches distros NVIDIA users should use the flatpak. The nvdec decode win above is independent and applies regardless.

Two `glib::idle_add_local` sources ran on every main loop iteration: one
polling mpv's event queue with `wait_event(0.0)`, one polling a flume channel
with `try_recv()`. An idle source is rescheduled immediately, so the main loop
never slept and both spun continuously, burning CPU when idle and during
playback.

Drain the mpv queue from a `timeout` source at ~60 Hz instead, which lets the
main loop sleep between ticks, and drain it fully each tick rather than one
event per iteration. `mpv_set_wakeup_callback` is deliberately not used to
drive this: it is only a best-effort hint (mpv coalesces property changes and
fires once for multiple events) and it races with `wait_event` clearing the
pending flag, so relying on it alone latches after the first burst and
playback stalls on the loading screen.

Replace the render poll with a task awaiting `recv_async()`, coalescing a
burst of update callbacks into a single `queue_render()`.
The Stremio web UI enables hardware decoding by sending `hwdec=auto-copy`,
which decodes on the GPU but copies every frame back to system memory and
re-uploads it for rendering (the "copy-back" path). That copy is CPU- and
bandwidth-heavy and is why playback here costs more CPU than mpv/VLC.

Remap `hwdec=auto-copy` to `auto-safe`, which keeps frames on the GPU via a
zero-copy interop (VAAPI/Vulkan/NVDEC depending on the driver) and falls back
to software decoding when none is available, so playback can never break.

Verified on Radeon 680M: auto-copy -> "hardware decoding (vulkan-copy)";
auto-safe -> "hardware decoding (vulkan)" with frames kept on the GPU.
mpv defaults to software decoding unless `hwdec` is set. The shell set no
default and only ever received `hwdec` from the web UI at runtime (after the
render context already exists), which did not reliably bring up the GL hwdec
interop — so playback ran on the CPU even with "hardware decoding" enabled.

Set `hwdec=auto-safe` in the initializer so mpv loads the zero-copy interop
when the render context is created. Verified in the real libmpv render path:

  [libmpv_render] Loading hwdec driver 'vaapi'
  [libmpv_render/vaapi] Using EGL dmabuf interop via GL_OES_EGL_image
  [libmpv_render/vaapi/vaapi] Initialized VAAPI
GTK 4.22 defaults to the Vulkan GSK renderer. Our video is an OpenGL GLArea
(mpv's vo=libmpv render API), and compositing a GL texture through the Vulkan
renderer forces an expensive per-frame GL->Vulkan copy.

Measured on an AMD Radeon 680M / Mesa, playing a 1080p30 H.264 clip with
hardware decoding (vaapi) confirmed active, using a probe that reproduces the
shell's exact render path (GLArea + libmpv render context, no WebKit). Five
interleaved rounds per renderer, each a 12s steady-state window after warmup,
medians, on an otherwise idle machine:

  GSK_RENDERER unset (4.22 default, GskVulkanRenderer)  ~24% CPU
  GSK_RENDERER=gl    (GskGLRenderer)                    ~7.8% CPU

For reference bare `mpv --vo=gpu` on the same file, measured the same way, was
~7.1%, so decode was never the problem: on the GL renderer the whole GLArea/GSK
path costs ~0.7 points over mpv's own output, while the Vulkan default adds ~17.

Default GSK_RENDERER to `gl`, respecting any renderer already chosen by the user
or by data/stremio.sh (which sets `opengl`, the same renderer on 4.22, for
Nvidia), so playback is GPU-composited instead of copied through Vulkan.
AltarBeastiful added a commit to AltarBeastiful/stremio-linux-shell that referenced this pull request Jul 18, 2026
From the Nvidia investigation (GTX 1060, driver 580, Plasma 6 Wayland):

DEVLOG §10-15
- §10-11: `gl` alone does not fix Nvidia. The pinned core is WebKitGTK
  falling back to Skia *CPU* rendering (the UI snapshots to a GskCairoNode),
  which GSK then re-composites over the video every frame on the CPU. On Mesa
  Skia GPU works (texture node) -> cheap; that is the entire Nvidia-vs-AMD
  split, and it lives inside WebKit, upstream of GSK. Confirmed with
  GDK_DEBUG=offload + perf; verify per-box with `webkit://gpu`.
- §12: the playback artifacts are a *separate*, branch-only regression -
  forcing hwdec=auto-safe (zero-copy), verified on VAAPI/Mesa only, breaks
  Nvidia's nvdec interop. The same commits ride in upstream PR Stremio#108.
- §13: the Flatpak launcher regression (e6fb216) and its fix.
- §14: a change-by-change regression audit vs upstream main (one real
  regression: hwdec=auto-safe on Nvidia; one silent improvement: the loop fix).

ADR-0003 (Proposed): present the video as a dmabuf GdkPaintable on a
GtkGraphicsOffload subsurface so video frames bypass GSK. Fixes the
offload-eligible matrix incl. Nvidia at integer scale; fractional-scale Nvidia
stays bottlenecked upstream (GTK declines offload at fractional scale).

ADR-0004 (Proposed): scope zero-copy hwdec to interops we have verified,
leaving Nvidia on the web UI's copy-back path until its interop is validated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GqwqM5jcTswvJYymbpdaY8
@KemalK

KemalK commented Jul 19, 2026

Copy link
Copy Markdown

Built this branch (flatpak Devel manifest) and tested on the #107 machine: Core Ultra 9 185H / Meteor Lake, Fedora 43, GNOME Wayland, mostly on battery.

The decode fix works. hwdec now comes up at startup on its own, no shim or override needed:

[vd] Trying hardware decoding via hevc-vaapi. Using hardware decoding (vaapi).

~3 hours of 4K HEVC 10-bit DV playback, 100+ seeks: zero decode errors, zero fallbacks. Playback CPU went from ~2.5 cores on 1.0.3 (software) to ~0.3 core, idle CPU from ~0.3 to ~0.05, whole-system battery draw ~16 W. The server's hw transcode probe also passes now (vaapi-renderD128 - it failed on 1.0.3).

The UI freeze is still present but clearly unrelated to this PR: it hit twice during the test, once after ~3h at 1.25x speed with heavy frame dropping (Dropped: 4016 in the status line), once minutes into a session right after rapid pause/unpause. mpv is healthy both times - the log just stops mid-flow and the main thread spins at 100%:

[cplayer] Set property: pause=true -> 1 [cplayer] Set property: pause=false -> 1 [cplayer] Set property: pause=true -> 1 (log ends here, UI frozen)

Same behavior exists on 1.0.3, so it's #77 territory, not a regression from this branch.

In short: on this hardware the PR does what it says. The freeze needs separate work.

@Ivshti
Ivshti requested review from core1024 and tymmesyde July 19, 2026 09:11
The web UI enables hardware decoding by sending hwdec=auto-copy (copy-back:
decode on the GPU, copy every frame back to system RAM and re-upload it). On
4K HDR that copy dominated CPU — ~100% of a core on the proprietary Nvidia
driver. Zero-copy keeps frames on the GPU.

On Mesa, auto-safe already selects VAAPI dmabuf zero-copy. On Nvidia proprietary,
auto-safe only offers copy-back (vulkan-copy), so request `nvdec` explicitly for
CUDA-interop zero-copy. Validated in the flatpak runtime: clean 4K HDR image and
~100% -> ~23% of a core (~4x), matching the Mesa win.

Refactored the hwdec selection into a single default_hwdec() helper used by both
the mpv init and the web-UI remap, with a STREMIO_HWDEC override for testing and
the libmpv CUDA-interop build requirement documented for downstream packagers
(the .deb's libmpv must be built --enable-cuda-hwaccel --enable-cuda-interop with
ffmpeg --enable-ffnvcodec, else nvdec falls back to copy-back).
@AltarBeastiful

AltarBeastiful commented Jul 19, 2026

Copy link
Copy Markdown
Author

NVIDIA: zero-copy nvdec on 4K HDR down from ~100% to ~23%

Follow-up after testing on the proprietary NVIDIA driver (GTX 1060 / Pascal, 580, KWin/Wayland, 170% fractional scale), in the flatpak.

What's been fixed

The web UI enables hardware decoding by sending hwdec=auto-copy. copy-back: decode on the GPU, then copy every frame back through system RAM and re-upload it. On 4K HDR (p010, ~25 MB/frame) that copy pins ~100% of a core on NVIDIA.

On Mesa, auto-safe already picks VAAPI dmabuf zero-copy. On the proprietary NVIDIA driver, auto-safe only offers copy-back (vulkan-copy), so we now request nvdec explicitly for CUDA-interop zero-copy (frames stay on the GPU). Result on 4K HDR, measured on the stremio process during steady fullscreen playback:

decode mode NVIDIA CPU (4K HDR)
auto-copy / vulkan-copy (copy-back, before) ~100%
nvdec (zero-copy, this PR) ~23%

Clean image, no artifacts. That ~23% is the video render/present path (mpv + GSK + HDR tone-map at 4K), roughly the floor for this shell, matching the Mesa result. The change is a small default_hwdec() helper (NVIDIA → nvdec, Mesa → auto-safe), plus a STREMIO_HWDEC env override for testing decode modes.

Packaging note for future deb

nvdec zero-copy needs libmpv/ffmpeg built with ffnvcodec (nvdec / cuda-interop). These headers are freely redistributable (no proprietary CUDA toolkit, and libcuda is loaded from the driver at runtime), and they're on by default in recent Ubuntu, Debian, Arch and Fedora, so this "just works" on mainstream distros. Only minimal or stripped ffmpeg rebuilds omit it, in which case nvdec silently falls back to copy-back (~100% again).

  • Build flags: mpv --enable-cuda-hwaccel --enable-cuda-interop, ffmpeg --enable-ffnvcodec, libplacebo. (Distinct from the nonfree CUDA path (--enable-cuda-nvcc, scale_cuda, libnpp), which nvdec decode does not use.)
  • Verified with libmpv 0.41 / ffmpeg 7.1 (org.gnome.Platform 50), and libmpv 0.41 on Ubuntu 26.04.
  • Also needs WebKitGTK ≥ 2.52.4 on NVIDIA: system WebKitGTK 2.52.3 (Ubuntu 26.04) gives a laggy UI and overlay rendering artifacts on NVIDIA, fixed in 2.52.4 / 2.52.5 and verified using the flatpak bundling 2.52.5. Independent of nvdec decode. See the reviewer note below.

Notes for reviewers

  • The GSK renderer (gl vs opengl→Vulkan) is a wash on NVIDIA during real playback (~23% either way). The GSK_RENDERER=gl win in this PR is a Mesa thing; it doesn't move the needle on NVIDIA. (data/stremio.sh still sets opengl on NVIDIA, which is harmless as it falls back to Vulkan.)
  • The UI overlay is NOT the cost on NVIDIA in the flatpak, it's a cheap Wayland subsurface (~1% difference between controls shown and hidden). (Open question: is the overlay delta similarly small on AMD/Intel? Worth a cross-platform check.)
  • Test the flatpak, not a host build. A host build against system GTK/WebKit shows a pile of NVIDIA-only symptoms (laggy UI, renderer artifacts, a "stale overlay", nvdec "artifacts") that do not reproduce in the flatpak. Root cause: the system WebKitGTK is older than the flatpak's. Ubuntu 26.04 ships WebKitGTK 2.52.3; the GNOME 50 runtime bundles 2.52.5. GTK is identical (4.22.4), so it's a WebKit micro-version gap, fixed by the rendering fixes in 2.52.4 / 2.52.5. A native .deb therefore needs WebKitGTK ≥ 2.52.4 on NVIDIA (until then, use the flatpak); the shipping flatpak (2.52.5) is clean. Measure only steady, non-buffering fullscreen playback (buffering reads artificially low).

Reproduce

flatpak run com.stremio.Stremio.Devel                              # nvdec auto-selected on NVIDIA
flatpak run --env=STREMIO_HWDEC=auto-copy com.stremio.Stremio.Devel   # copy-back, for comparison
# play 4K HDR fullscreen, past buffering, then:
top -b -n2 -d 3 -p "$(pgrep -x stremio | head -1)" | awk 'END{print $9"%"}'
# confirm mode:  Using hardware decoding (nvdec).   VO: [libmpv] 3840x2160 p010

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AMD: GTK4 defaults to the Vulkan renderer while mpv renders via OpenGL, pegging the GPU at ~100% during playback

2 participants