Skip to content

fix(waylib): import SHM buffers through Qt RHI in Vulkan mode - #1132

Open
zccrs wants to merge 6 commits into
linuxdeepin:master-backup-20260717from
zccrs:fix/vulkan-shm-rhi-upload
Open

fix(waylib): import SHM buffers through Qt RHI in Vulkan mode#1132
zccrs wants to merge 6 commits into
linuxdeepin:master-backup-20260717from
zccrs:fix/vulkan-shm-rhi-upload

Conversation

@zccrs

@zccrs zccrs commented Jul 13, 2026

Copy link
Copy Markdown
Member

Summary

This PR supersedes #1114 and contains two related Vulkan rendering fixes.
Please review and merge this PR instead of #1114.

1. Import dmabuf as a Vulkan render target

  • Port the wlroots 0.20 vulkan_import_dmabuf(for_render=true) flow to waylib.
  • Replace the wlr_texture-based VkImage creation in WRenderHelper::acquireRenderTarget() with a direct dmabuf import.
  • Create the imported VkImage with VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT, allowing it to be used by Qt RHI as a render target.
  • Add VkDmabufImage lifetime management and release imported Vulkan resources when the associated buffer is destroyed.
  • Clean up the associated BufferData object and its render-target resources on before_destroy().

2. Import SHM client buffers through Qt RHI in Vulkan mode

  • Avoid the deferred wlroots Vulkan texture upload path for CPU-accessible SHM buffers.
  • Copy SHM pixels into Qt-owned QImage storage before ending wlroots data access.
  • Reuse WTools::toImageFormat() and convertToDrmSupportedFormat() for DRM format conversion.
  • Cover ARGB8888, RGBA8888, BGRA8888, BGRX8888, and RGBX8888 formats.
  • Fix QSG/QRhi texture cleanup across SHM, dmabuf, and native texture transitions.
  • Keep the existing client-buffer and non-Vulkan texture paths unchanged.

摘要

本 PR 替代 #1114,包含两个相互关联的 Vulkan 渲染修复,请以本 PR 为准进行审查和合并。

1. 将 dmabuf 直接导入为 Vulkan render target

  • 将 wlroots 0.20 的 vulkan_import_dmabuf(for_render=true) 流程移植到 waylib。
  • WRenderHelper::acquireRenderTarget() 中基于 wlr_texture 的 VkImage 创建改为直接导入 dmabuf。
  • 为导入的 VkImage 设置 VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT,使其可以作为 Qt RHI render target 使用。
  • 增加 VkDmabufImage 生命周期管理,在关联 buffer 销毁时释放 Vulkan 资源。
  • before_destroy() 中清理对应的 BufferData 和 render-target 资源。

2. Vulkan 模式下通过 Qt RHI 导入 SHM client buffer

  • Vulkan 模式下,CPU 可访问的 SHM buffer 不再走 wlroots 延迟 Vulkan texture 上传路径。
  • 在结束 wlroots data access 前,将 SHM 像素复制到 Qt 自有的 QImage 存储中。
  • 复用 WTools::toImageFormat()convertToDrmSupportedFormat() 完成 DRM 格式转换。
  • 覆盖 ARGB8888、RGBA8888、BGRA8888、BGRX8888 和 RGBX8888 格式。
  • 修复 SHM、dmabuf 和原生 texture 切换时的 QSG/QRhi 资源清理问题。
  • 保持现有 client-buffer 及非 Vulkan texture 路径不变。

Verification

  • cmake --build build --target test_wshmtexture waylibserver treeland -j$(nproc)
  • /usr/bin/ctest --test-dir build -R '^test_wshmtexture$' --output-on-failure
  • QtTest: 7 passed, 0 failed
  • CTest: 1/1 passed
  • Nested Vulkan X11 testing verified SHM, EGL, and flower client rendering.

验证

  • waylibservertreeland 构建成功。
  • SHM 格式测试 7 个用例全部通过。
  • CTest 1/1 通过。
  • Nested Vulkan X11 环境下已验证 SHM、EGL 和 flower client 正常渲染。

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @zccrs, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@deepin-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: zccrs

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

zccrs added 2 commits July 13, 2026 16:11
1. Port wlroots 0.20 vulkan_import_dmabuf(for_render=true) to waylib
2. Replace wlr_texture-based VkImage in acquireRenderTarget with
   direct dmabuf import so the buffer is a valid render target
3. New VkImage gets VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT for valid
   render target usage instead of reusing a wlr_texture
4. Add VkDmabufImage lifecycle management in BufferData destructor
5. Fix BufferData memory leak in onBufferDestroy
6. Keep the verbatim wlroots 0.20 formats[] table; guard the 8
   DRM_FORMAT_* entries added in libdrm >= 2.4.129 with
   #if defined() so it compiles on CI's older libdrm (2.4.123)
7. Use Q_BYTE_ORDER == Q_LITTLE_ENDIAN instead of a custom endian macro
8. Make isYCbCr, kVkFormats and vkFormatFromDrm constexpr
9. Use QT_WARNING_PUSH/POP instead of pragma GCC diagnostic
10. Return an empty target when get_dmabuf fails (else branch)

Log: Fixed Vulkan render target to use a proper color-attachment VkImage

Influence:
1. Test Vulkan rendering with various dmabuf formats
2. Verify render target creation without validation errors
3. Test buffer creation and destruction for memory leaks
4. Verify GL and pixman rendering paths are unaffected
5. Confirm the build passes on libdrm 2.4.123 and >= 2.4.129

fix: 使用 dmabuf 导入创建 vulkan 渲染目标

1. 将 wlroots 0.20 的 vulkan_import_dmabuf(for_render=true) 移植到 waylib
2. 在 acquireRenderTarget 中用直接 dmabuf 导入替代 wlr_texture 创建
   VkImage,使缓冲可作为有效渲染目标
3. 新建 VkImage 带 VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT,替代复用
   wlr_texture 的做法,获得合法的渲染目标用途
4. 在 BufferData 析构中添加 VkDmabufImage 生命周期管理
5. 修复 onBufferDestroy 中 BufferData 的内存泄漏
6. 保留 wlroots 0.20 原始 formats[] 表;对 libdrm >= 2.4.129 新增的
   8 个 DRM_FORMAT_* 条目用 #if defined() 守卫,使其在 CI 较旧的
   libdrm (2.4.123) 上也能编译
7. 用 Q_BYTE_ORDER == Q_LITTLE_ENDIAN 替代自定义大小端宏
8. 将 isYCbCr、kVkFormats、vkFormatFromDrm 改为 constexpr
9. 用 QT_WARNING_PUSH/POP 替代 pragma GCC diagnostic
10. get_dmabuf 失败时返回空目标(else 分支)

Log: 修复 Vulkan 渲染目标,使用合法的颜色附件 VkImage

Influence:
1. 测试 Vulkan 渲染路径下各种 dmabuf 格式
2. 验证渲染目标创建成功,无 validation 错误
3. 测试 buffer 创建和销毁,确认无内存泄漏
4. 验证 GL 和 pixman 渲染路径不受影响
5. 确认在 libdrm 2.4.123 与 >= 2.4.129 上均能编译通过
Copy wl_shm buffers into Qt-owned image storage instead of using the
deferred wlroots Vulkan upload path.

Handle all supported 32-bit wl_shm formats with premultiplied alpha
semantics and clear QSGPlainTexture pointers before deferred QRhi cleanup.

通过 Qt RHI 导入 Vulkan 模式下的 SHM buffer,避免 wlroots 延迟上传导致黑窗。
补充完整的 32 位 wl_shm 格式映射,并修复 QRhi texture 生命周期问题。

Log: 修复 Vulkan 模式下 SHM 客户端黑窗
Influence: Vulkan 模式下 SHM、EGL 和 dmabuf 客户端可正常显示,避免纹理悬空和泄漏。
Comment thread waylib/src/server/kernel/wglobal.h
Comment thread waylib/src/server/qtquick/private/wbufferrenderer.cpp Outdated
Comment thread waylib/src/server/qtquick/private/wbufferrenderer.cpp Outdated
Comment thread waylib/src/server/qtquick/private/wqmlhelper_p.h Outdated
Comment thread waylib/src/server/qtquick/woutputrenderwindow.cpp Outdated
Comment thread waylib/src/server/qtquick/woutputrenderwindow.cpp Outdated
Comment thread waylib/src/server/qtquick/woutputrenderwindow.cpp Outdated
Comment thread waylib/src/server/qtquick/woutputrenderwindow.cpp Outdated
Comment thread waylib/src/server/qtquick/wrenderhelper.cpp Outdated
Comment thread waylib/src/server/qtquick/wrenderhelper.cpp Outdated
@zccrs
zccrs force-pushed the fix/vulkan-shm-rhi-upload branch 2 times, most recently from ee36af6 to d5cddc4 Compare July 14, 2026 08:21
Comment thread waylib/src/server/qtquick/woutputrenderwindow.cpp Outdated
Comment thread waylib/src/server/qtquick/woutputrenderwindow.cpp
Comment thread waylib/src/server/qtquick/woutputrenderwindow.cpp Outdated
Comment thread waylib/src/server/qtquick/wrenderhelper.cpp Outdated
Comment thread waylib/src/server/qtquick/wrenderhelper.cpp Outdated
Comment thread waylib/src/server/qtquick/wrenderhelper.cpp Outdated
Comment thread waylib/src/server/qtquick/wrenderhelper.cpp Outdated
@zccrs
zccrs force-pushed the fix/vulkan-shm-rhi-upload branch 4 times, most recently from bfdb2ce to 0bbd4dc Compare July 15, 2026 02:37
Replace boolean preserve controls with DontCare, Clear, and Preserve modes.

以 DontCare、Clear 和 Preserve 三态替换布尔保留控制。

Resolve modes at target creation and keep each render pass target stable.

在目标创建时解析模式,并保证每个渲染阶段的目标保持稳定。

Recreate incompatible Vulkan targets and update other RHI flags in place.

重建不兼容的 Vulkan 目标,其他 RHI 后端则原地更新标志。

Keep software clears backend-aware and preserve multipass contents.

让软件渲染按后端决定清屏,并保留多遍渲染的已有内容。

Log: 实现渲染目标颜色内容三态策略
Influence: 渲染目标可显式清屏或保留内容,避免目标切换和续绘丢失。
@zccrs
zccrs force-pushed the fix/vulkan-shm-rhi-upload branch from 0bbd4dc to 8041384 Compare July 15, 2026 02:40
zccrs added 3 commits July 15, 2026 15:42
Replace RenderTarget's public struct members with an opaque d-pointer
holding std::weak_ptr<BufferData>. When WRenderHelper destroys a
BufferData, all RenderTarget instances auto-null like QPointer<QObject>.

Eliminates findBufferData for-loop: barrier methods now access BufferData
directly through weak_ptr::lock(). WRenderHelperPrivate manages BufferData
via shared_ptr, enabling automatic weak_ptr expiry.

Also adds Vulkan layout transition barrier helpers and VkDmabufImage
layout tracking for COLOR_ATTACHMENT_OPTIMAL <-> GENERAL transitions.

将 RenderTarget 从公开结构体重构为不透明 d-pointer 类,内部持有
std::weak_ptr<BufferData>。当 WRenderHelper 销毁 BufferData 时,
所有 RenderTarget 实例自动置空,类似 QPointer 引用 QObject。

消除 findBufferData 循环查找:屏障方法通过 weak_ptr::lock() 直接
访问 BufferData。WRenderHelperPrivate 使用 shared_ptr 管理
BufferData 生命周期,使 weak_ptr 自动失效。

同时添加 Vulkan 布局转换屏障辅助函数和 VkDmabufImage 布局跟踪,
实现 COLOR_ATTACHMENT_OPTIMAL 与 GENERAL 之间的转换。

Log: 重构 RenderTarget 为弱引用模式并添加 Vulkan 布局屏障
Influence: RenderTarget 不再暴露成员变量,生命周期由 weak_ptr 管理;
屏障方法不再需要循环查找 BufferData,性能提升。
When the graphics backend is Vulkan, RenderBufferBlitter's backdrop
capture path is broken. Instead of creating it and its dependent
effect components (GlassEffect, MultiEffect, Shape), fall back to a
plain semi-transparent Rectangle.

Add WaylibHelper.isVulkanBackend QML property so Blur.qml can detect
the backend at component-creation time and avoid instantiating
RenderBufferBlitter entirely via a Loader.

当后端为 Vulkan 时,RenderBufferBlitter 的背景捕获路径不可用。
不再创建 RenderBufferBlitter 及其依赖的效果组件,改为使用一个
半透明 Rectangle 作为替代。新增 WaylibHelper.isVulkanBackend QML
属性,使 Blur.qml 在组件创建时检测后端并通过 Loader 避免实例化。

Log: Vulkan 后端跳过 RenderBufferBlitter,使用半透明 Rectangle 替代
Influence: Vulkan 模式下 Blur 效果降级为半透明矩形,不影响非 Vulkan 后端
QSGPlainTexture wrapping wlroots textures via setTextureFromNativeTexture
has no mipmap image data. Setting MipmapFiltering to Linear triggers
"Mipmap settings changed without having image data available" warnings.
Remove the setMipmapFiltering calls in both the constructor and setSmooth
so the default None takes effect.

通过 setTextureFromNativeTexture 包装 wlroots 纹理的 QSGPlainTexture
没有 mipmap 图像数据。将 MipmapFiltering 设为 Linear 会触发警告。
移除构造函数和 setSmooth 中的 setMipmapFiltering 调用,使用默认值 None。

Log: 移除 wlroots 原生纹理上不应有的 mipmap 过滤设置
Influence: 消除 QSGPlainTexture mipmap 警告,不影响渲染质量
@zccrs
zccrs force-pushed the fix/vulkan-shm-rhi-upload branch from 430232f to 17b12cc Compare July 15, 2026 08:43
@LFRon

LFRon commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

@zccrs 大佬, 如果treeland会内置wlroots ( #1155 ) 的话, 是不是没必要这么实现了, 直接让wlroots暴露对应API然后treeland拿来用就行了, 就跟我那 #1032 提交类似

@deepin-bot

deepin-bot Bot commented Jul 17, 2026

Copy link
Copy Markdown

TAG Bot

New tag: 0.8.15
DISTRIBUTION: unstable
Suggest: synchronizing this PR through rebase #1163

zccrs added a commit to zccrs/treeland that referenced this pull request Aug 5, 2026
1. Reuse wlroots wlr_vk_render_buffer for Qt Quick RHI.
2. Record FOREIGN_EXT ownership transfer around Qt rendering.
3. Preserve SHM partial updates and avoid QRhi::finish on Vulkan.

Log: Fix Vulkan scanout and SHM upload stability

Influence:
1. Run Vulkan output rendering with SHM and DMA-BUF clients.
2. Verify repeated frames and buffer reuse do not crash.
3. Verify GLES and software paths keep their existing flush behavior.

Acknowledgements:
Thanks to GitHub user @LFRon for identifying the Qt RHI/wlroots Vulkan
integration issue and suggesting in PR linuxdeepin#1132 that Treeland reuse wlroots
APIs, which helped guide this implementation.

fix(waylib): 修复 Vulkan 扫描输出渲染目标

1. 复用 wlroots 的 wlr_vk_render_buffer 供 Qt Quick RHI 使用。
2. 在 Qt 绘制前后记录 FOREIGN_EXT 所有权转移。
3. 保留 SHM 局部更新,并避免 Vulkan 调用 QRhi::finish。

Log: 修复 Vulkan 扫描输出与 SHM 上传稳定性

Influence:
1. 验证 Vulkan 下 SHM 和 DMA-BUF 客户端输出。
2. 验证多帧绘制和缓冲复用不再触发崩溃。
3. 验证 GLES 与软件渲染路径保持原有刷新行为。

致谢:
感谢 GitHub 用户 @LFRon 发现 Qt RHI 与 wlroots Vulkan 集成问题,
并在 PR linuxdeepin#1132 中建议 Treeland 直接复用 wlroots 暴露的 API,
为本次实现提供了重要参考。
zccrs added a commit that referenced this pull request Aug 6, 2026
1. Reuse wlroots wlr_vk_render_buffer for Qt Quick RHI.
2. Record FOREIGN_EXT ownership transfer around Qt rendering.
3. Preserve SHM partial updates and avoid QRhi::finish on Vulkan.

Log: Fix Vulkan scanout and SHM upload stability

Influence:
1. Run Vulkan output rendering with SHM and DMA-BUF clients.
2. Verify repeated frames and buffer reuse do not crash.
3. Verify GLES and software paths keep their existing flush behavior.

Acknowledgements:
Thanks to GitHub user @LFRon for identifying the Qt RHI/wlroots Vulkan
integration issue and suggesting in PR #1132 that Treeland reuse wlroots
APIs, which helped guide this implementation.

fix(waylib): 修复 Vulkan 扫描输出渲染目标

1. 复用 wlroots 的 wlr_vk_render_buffer 供 Qt Quick RHI 使用。
2. 在 Qt 绘制前后记录 FOREIGN_EXT 所有权转移。
3. 保留 SHM 局部更新,并避免 Vulkan 调用 QRhi::finish。

Log: 修复 Vulkan 扫描输出与 SHM 上传稳定性

Influence:
1. 验证 Vulkan 下 SHM 和 DMA-BUF 客户端输出。
2. 验证多帧绘制和缓冲复用不再触发崩溃。
3. 验证 GLES 与软件渲染路径保持原有刷新行为。

致谢:
感谢 GitHub 用户 @LFRon 发现 Qt RHI 与 wlroots Vulkan 集成问题,
并在 PR #1132 中建议 Treeland 直接复用 wlroots 暴露的 API,
为本次实现提供了重要参考。
@LFRon

LFRon commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

这个PR可以关了吧

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants