Opt-in background Computer Use backend for Hyprland?

[Gabriel-Kahen]

I’ve been experimenting with background Computer Use on Hyprland and wanted to ask whether this capability would fit within this project—and, if so, where it should live architecturally.

Motivation

The existing Linux Computer Use backend supports Hyprland window discovery, AT-SPI accessibility, screenshots, and input through /dev/uinput, RemoteDesktop portals, or ydotool.

Those input paths operate at the desktop level, however. Targeting an application generally requires focusing or raising its window, and coordinate input uses the global pointer. This means Codex can interrupt the user while it works.

The behavior I’m trying to provide is closer to Computer Use on macOS:

• Control an existing application in the current login session.
• Work with windows on inactive workspaces.
• Do not change the focused window or active workspace.
• Do not move the physical pointer.
• Preserve existing app processes, profiles, logins, and open state.

Current prototype

I built a Hyprland-specific Codex plugin that implements this behavior:

https://github.com/Gabriel-Kahen/hyprland-codex-background-computer-use

It currently uses:

• hyprctl for window discovery and address-targeted shortcuts.
• grim -T with Hyprland stable window IDs for exact window capture without raising the window.
• AT-SPI for semantic actions and editable values.
• A version-matched Hyprland plugin for window-local Wayland pointer events.
• XWayland’s internal XTEST pointer for X11 applications, restoring it after each operation.
• A temporary headless-output transaction as a compatibility fallback.

The physical cursor, keyboard focus, and active workspace remain unchanged during normal capture, click, scroll, drag, and shortcut operations.

Scope and architecture question

The prototype is intentionally Hyprland-specific, so I would not expect it to replace the existing cross-desktop implementation or become a default input path.

If this capability is in scope, which direction would be preferable?

1. Add an optional Hyprland capture/input provider to computer-use-linux.
2. Package it as an opt-in linux-features/ module.
3. Keep it as an external Codex plugin and document it as an optional integration.
4. Define a more general backend interface first, allowing other compositors to add non-interfering implementations later.

The main maintenance concern is the native Hyprland component: it must be built against the running Hyprland version because compositor plugins are ABI-sensitive. Any integration should therefore remain optional and fail safely without changing behavior on other desktops.

I’m not proposing a PR or a particular architecture yet. I’d mainly like to confirm whether background, non-interfering Computer Use is within the project’s scope and learn which integration boundary maintainers would be comfortable supporting.