NeoRemote

中文 | English

Native LAN-only cross-device control for iOS, Android, macOS, and Windows.

iOS · Android · macOS · Windows

Latest beta: v0.1.0-beta.38 · Build all artifacts · Beta release

Overview

NeoRemote is a native, local-network control tool. It has no cloud service, no account system, and no relay server.

It currently supports three practical control paths:

Path	Status	Notes
Mobile -> Desktop	Available	iOS / Android controllers discover macOS / Windows receivers and inject native mouse input.
Mobile -> Android	Available	Android can act as a controlled phone through AccessibilityService, with optional live screen handoff.
Desktop -> Android over ADB	Available as a secondary tool	macOS / Windows can control already ADB-authorized Android devices with real-time video.

All four clients are native implementations. There is no Flutter, React Native, Electron, cloud backend, database, or account dependency.

Current Capabilities

Mobile Controllers

Auto discovery + manual connect: Bonjour / DNS-SD and UDP fallback discover LAN targets; manual IP/port remains the fallback.
Screen Handoff: For Android targets, the Remote page shows the Android screen handoff stream when available. Touch uses screenTouchV1 real-time down / move / up events when the target advertises support, and falls back to normalized screenGesture on older builds.
Short Video mode: Dedicated video controls for next, previous, swipe left/right, like, favorite, comment, play/pause, and back.
Mode switching: The Remote page is reserved for the actual control surface. Current-session mode switching is exposed in the Devices page control-status area; Settings controls the default mode for future sessions.
Codec preference: iOS and Android Settings expose Auto / H.264 / H.265 / AV1. Auto currently chooses the stable H.264 path first; H.265 and AV1 remain explicit user choices.
Letterbox-aware touch mapping: Control coordinates are mapped only inside the displayed video area so black bars do not become accidental taps.

Android Controlled Device

Publishes itself through Android NSD and UDP fallback when the receiver is enabled.
Receives JSON control commands on 51101.
Uses AccessibilityService for taps, swipes, back/home/recents, and short-video actions.
Streams screen handoff on 51102 after local MediaProjection approval.
Uses one active MediaProjection / encoder / VirtualDisplay pipeline per approval, reducing intermittent black-screen races during reconnects.
Binds mirror access to the approved control session and stops streaming when authorization is revoked.
Supports H.264, HEVC/H.265, and AV1 when the local hardware encoder provides them; the fallback codec is H.264.

Desktop Receivers

macOS: Swift / SwiftUI / AppKit receiver, CoreGraphics input injection, Accessibility permission status, and optional connected-session ADB console.
Windows: C++20 / Win32 receiver, SendInput injection, UDP discovery, tray/WinUI controls, and optional connected-session ADB console.
Desktop screen mirroring is intentionally out of scope; desktop receivers are input targets, not remote-desktop servers.

Platform Stack

Platform	Stack	Role
iOS	Swift / SwiftUI / UIKit / AVFoundation / VideoToolbox / Network.framework	Mobile controller and Android screen handoff viewer
Android	Kotlin / Jetpack Compose / NSD / Socket / AccessibilityService / MediaProjection / MediaCodec	Mobile controller and Android controlled device
macOS	Swift Package / SwiftUI / AppKit / CoreGraphics / Network.framework	Desktop receiver and ADB secondary controller
Windows	C++20 / Win32 / SendInput / Winsock	Desktop receiver and ADB secondary controller

Screenshots

Screenshots are from the current iOS simulator build and macOS receiver.

Onboarding	Manual Connect	Screen Handoff

Short Video Mode	Devices	Settings

Desktop

macOS Receiver

Latest Beta

Current latest beta: v0.1.0-beta.38

Release page: https://github.com/Souitou-iop/NeoRemote/releases/tag/v0.1.0-beta.38

Included artifacts:

Artifact	Description
`NeoRemote-ios-unsigned.ipa`	Unsigned iOS IPA. Sign it with your own development/test certificate before installing on a physical device.
`NeoRemote-android-release-signed.apk`	Signed Android release APK. GitHub builds fail if Android signing secrets are missing.
`NeoRemote-macos.zip`	macOS local-testing app zip, ad-hoc signed in beta release workflow and not notarized.
`NeoRemote-windows-receiver.zip`	Windows desktop receiver zip.

Local handoff builds are archived by version under resources/build/beta/<version>/. For this beta, resources/build/beta/v0.1.0-beta.38/manifest.txt records the local artifact names, sizes, and SHA-256 hashes.

Getting Started

Prerequisites

Platform	Requirements
iOS	macOS, Xcode, iOS 17.0+ deployment target
Android	JDK 21, Android SDK, compileSdk 36, minSdk 26 / Android 8.0+
macOS	macOS 15+, Swift 6 toolchain / Xcode, Accessibility permission
Windows	Windows 10/11, Visual Studio 2022, Desktop development with C++, Windows SDK
Desktop ADB tool	Android platform-tools; scrcpy is required for HEVC/AV1 desktop ADB video and Windows real-time ADB mirror

iOS

# Simulator build
xcodebuild build \
  -project iOS/NeoRemote.xcodeproj \
  -scheme NeoRemote \
  -destination 'generic/platform=iOS Simulator'

# Unsigned device archive
xcodebuild archive \
  -project iOS/NeoRemote.xcodeproj \
  -scheme NeoRemote \
  -configuration Release \
  -destination 'generic/platform=iOS' \
  -archivePath /tmp/NeoRemote-unsigned.xcarchive \
  CODE_SIGNING_ALLOWED=NO \
  CODE_SIGNING_REQUIRED=NO \
  CODE_SIGN_IDENTITY=

Android

cd Android
./gradlew :app:testDebugUnitTest
./gradlew :app:assembleDebug
./gradlew :app:assembleRelease

# Install debug APK
adb install -r app/build/outputs/apk/debug/app-debug.apk

Release signing is intentionally external to Gradle. See Android Signing.

macOS

Accessibility permission: System Settings -> Privacy & Security -> Accessibility

swift test --package-path MacOS
swift build -c release --package-path MacOS
./MacOS/script/build_and_run.sh
./MacOS/script/build_and_run.sh --verify

Build output: MacOS/dist/NeoRemoteMac.app

Windows

./Windows/scripts/build_receiver.ps1

Build output: Windows/build/NeoRemote.WindowsReceiver.exe

Connection Methods

Auto discovery: Put the controller and target on the same LAN.
Manual connect: Enter target IP and port from the controller.
Android wired debug: adb reverse tcp:51101 tcp:51101, then connect to 127.0.0.1:51101.
Desktop wireless ADB: Enable Android wireless debugging, authorize with adb connect <host>:<port>, then use the macOS / Windows ADB console after the desktop app has an active NeoRemote session.

Protocol & Connectivity

NeoRemote uses JSON over TCP for control commands. Android screen handoff uses a separate binary TCP stream so large video frames do not enter the JSON channel.

Discovery

Method	Description
Bonjour / DNS-SD	Service type `_neoremote._tcp.`
UDP fallback	Port `51101`, request `NEOREMOTE_DISCOVER_V1`, response prefix `NEOREMOTE_DESKTOP_V1`

Android controlled devices may advertise:

mirrorPort=51102
mirrorCodec=h264
mirrorCodecs=h264,hevc,av1
mirrorMax=720p24
touchProtocol=screenTouchV1

Older clients ignore unknown metadata and continue using JSON control only.

Default Ports

Target	Default Port
macOS desktop receiver	`50505`
Windows desktop receiver	`51101`
Android controlled device	`51101`
Android screen handoff stream	`51102`
Android ADB wired debug	`127.0.0.1:51101`

Control Commands

{ "type": "clientHello", "clientId": "...", "displayName": "iPhone", "platform": "ios" }
{ "type": "tap", "button": "primary" }
{ "type": "move", "dx": 12.3, "dy": -4.8 }
{ "type": "scroll", "deltaX": 0.0, "deltaY": 18.0 }
{ "type": "drag", "state": "started", "dx": 0.0, "dy": 0.0, "button": "primary" }
{ "type": "screenTouch", "phase": "down", "pointerId": 0, "x": 0.5, "y": 0.8, "sequence": 42 }
{ "type": "screenGesture", "kind": "swipe", "startX": 0.5, "startY": 0.8, "endX": 0.5, "endY": 0.2, "durationMs": 260 }
{ "type": "systemAction", "action": "back" }
{ "type": "videoAction", "action": "swipeUp" }
{ "type": "mirrorAction", "action": "start" }
{ "type": "mirrorAction", "action": "stop" }
{ "type": "heartbeat" }

Supported videoAction values

action	Behavior
`swipeUp`	Next video
`swipeDown`	Previous video
`swipeLeft`	Swipe left
`swipeRight`	Swipe right
`doubleTapLike`	Like
`favorite`	Favorite
`playPause`	Play / Pause
`comment`	Comment
`back`	Back

Response format

{ "type": "ack" }
{ "type": "status", "message": "Android controlled device connected" }
{ "type": "heartbeat" }

Android Screen Handoff Stream

Port: 51102
Default target: 720p24
Codecs: H.264, HEVC/H.265, AV1 when the Android device has a hardware encoder
Optional request before stream header: NRMVREQ1\ncodecs=h264\ntoken=<session-token>\n\n
Header: ASCII NRMV1\n
Packet: 4-byte big-endian payload length, 1-byte packet type, 8-byte big-endian presentation timestamp, then payload
Packet types: 0 metadata JSON, 1 codec config, 2 key frame, 3 delta frame

Auto mode requests H.264 first because it is the most stable across Android MediaProjection, hardware encoders, VideoToolbox, and MediaCodec decoders. Explicit H.265 / AV1 modes are available, but changing codec should be treated as a new screen-handoff session on the Android target.

Android Controlled Device

Install the Android app.
Open Android Accessibility settings.
Enable the NeoRemote accessibility service.
To show live screen handoff, tap Start screen handoff in the Android app and approve the system screen-capture prompt.
Keep the Android device and controller on the same LAN, or use ADB wired debug.
Connect from iOS / Android. The Remote page shows live video after the mirror socket is authorized and the first key frame arrives.

Implementation components:

Component	Responsibility
TCP receiver	Receives JSON commands and returns status
Android NSD / UDP responder	Makes the Android target discoverable
Accessibility injection	Executes taps, swipes, system navigation, and short-video actions
Real-time touch path	Executes `screenTouchV1` down / move / up sequences when supported
MediaProjection mirror service	Captures and encodes the Android screen after local approval
Mirror authorization registry	Issues short-lived mirror access only for approved control sessions
Action queue	Serializes rapid video actions with bounded backlog

Desktop Wireless ADB Control

Desktop ADB control is separate from NeoRemote's LAN JSON protocol. It is for Android devices the user has already authorized through developer options.

Connection: adb connect <host>:<port>
macOS H.264 video: embedded through adb -s <serial> exec-out screenrecord --output-format=h264 --bit-rate 6000000 --size <width>x<height> -
macOS H.265 / AV1 video: launches scrcpy --serial <serial> --video-codec=<h265|av1> --max-size=1080
Windows video: launches scrcpy --tcpip=<host>:<port> --video-codec=<h264|h265|av1> --max-size=1080
Touch/control: persistent adb shell on macOS where possible; ADB input commands and selected shortcut actions on both desktop platforms
Windows tool lookup: use trusted absolute paths through NEOREMOTE_ADB_PATH / NEOREMOTE_SCRCPY_PATH, Android SDK locations, Program Files, or WinGet package locations; arbitrary PATH lookup is not used.

Repository Structure

.
├── Android/                         # Kotlin / Jetpack Compose controller + Android target
├── iOS/                             # Swift / SwiftUI mobile controller
├── MacOS/                           # Swift Package macOS receiver
├── Windows/                         # C++20 / Win32 Windows receiver
├── resources/                       # Shared assets, docs, scripts, and local build outputs
│   ├── build/                       #   Ignored local artifacts and beta handoff folders
│   ├── docs/                        #   Audits, testing handoffs, and task logs
│   ├── icons/                       #   Icon Composer source and exported icons
│   ├── screenshots/                 #   README and release screenshots
│   └── scripts/                     #   Resource and build helper scripts
└── .github/workflows/               # Build all artifacts and beta release workflows

CI & Release

Build all artifacts

Pushing to main or manually triggering Build all artifacts builds:

Platform	Artifact
iOS	`NeoRemote-unsigned-ipa`
Android	`NeoRemote-android-apk` signed release APK
macOS	`NeoRemoteMac` app zip
Windows	`NeoRemoteWindowsReceiver` zip

Android release signing relies on GitHub Secrets and fails fast if they are missing:

Secret Name	Purpose
`ANDROID_RELEASE_KEYSTORE_BASE64`	Base64-encoded keystore file
`ANDROID_RELEASE_KEYSTORE_PASSWORD`	Keystore password
`ANDROID_RELEASE_KEY_ALIAS`	Key alias
`ANDROID_RELEASE_KEY_PASSWORD`	Key password

Beta release

Manually trigger the Beta release workflow to create a GitHub prerelease:

gh workflow run beta-release.yml --ref main -f version=v0.1.0-beta.38

Local beta archive

Before creating user-facing local artifacts after code changes:

python3 resources/scripts/bump_beta_version.py

After building and validating artifacts:

./resources/scripts/archive_beta_build.sh

Icon sync

resources/icons/NeoRemote.icon is the single icon design source:

./resources/scripts/sync_icons.sh

Security

NeoRemote is designed for trusted LAN use.

No TLS yet: JSON control traffic and discovery are plaintext. Use only on trusted home or office networks.
Discovery is unauthenticated: Bonjour and UDP fallback can be spoofed on a hostile LAN.
Session approval matters: Desktop receivers require local approval for new control sessions. Android screen handoff requires local MediaProjection approval and mirror stream authorization bound to the current control session.
Pairing V2 is still future work: Cryptographic paired-device authentication, discovery signatures, and optional TLS/PSK are not implemented yet.
Accessibility is powerful: The Android target can perform screen operations once Accessibility is enabled. Only enable it for devices you intend to control.
ADB is user-authorized: Desktop ADB control assumes the user has already authorized the Android device through Android developer options.

Troubleshooting

Controller cannot discover target

Confirm the controller and target are on the same LAN and same subnet.
Try manual connection with the target IP and port.
Check firewall rules for macOS 50505, Windows 51101, or Android 51101.
For emulator testing, use ADB forwarding because simulator/emulator broadcast discovery is often unreliable.

Android touch works but screen is black

On the Android target, tap Start screen handoff and approve the system capture prompt.
Use H.264 or Auto first. H.265 / AV1 depend on device encoder and controller decoder support.
Disconnect and reconnect once after changing codec.
Confirm port 51102 is reachable. Touch uses 51101; working touch alone does not prove the video socket is connected.
Keep only one controller connected to the mirror stream.

macOS: connected but cannot control mouse

Grant Accessibility permission: System Settings -> Privacy & Security -> Accessibility, then restart NeoRemote.

Android target gestures do not execute

Confirm the NeoRemote accessibility service is enabled.
Keep the Android screen awake.
On heavily customized Android builds, set NeoRemote battery mode to unrestricted.

ADB wired debug connection

adb reverse tcp:51101 tcp:51101

Then manually connect to 127.0.0.1:51101.

Windows cannot launch adb or scrcpy

Set NEOREMOTE_ADB_PATH and NEOREMOTE_SCRCPY_PATH to trusted absolute executable paths, or install Android platform-tools / scrcpy to the trusted locations shown by the app.

Roadmap

Item	Status
Mobile -> Desktop input	Available
Android controlled phone	Available
Android live screen handoff	Available
`screenTouchV1` real-time touch path	Available
Short-video actions	Available
Desktop wireless ADB Android console	Available
Four-platform GitHub Actions artifacts	Available
Keyboard input	Planned
Custom quick actions and gesture macros	Planned
Pairing V2 with nonce/HMAC and trusted-device auto approval	Planned
Optional TLS / PSK transport	Planned
Multi-controller arbitration	Planned

Contributing

Read AGENTS.md before changing code. It documents platform conventions, protocol rules, build rules, security constraints, and release artifact layout.

Before submitting changes:

# Android
cd Android && ./gradlew :app:testDebugUnitTest

# iOS
xcodebuild test -project iOS/NeoRemote.xcodeproj -scheme NeoRemote -destination 'platform=iOS Simulator,name=iPhone 17'

# macOS
swift test --package-path MacOS

When extending the protocol, update all platform codecs together and keep old clients compatible with unknown fields or command types.

Scope & Boundaries

NeoRemote is an input control tool, not a full remote-desktop suite.

Currently out of scope by default:

Desktop screen mirroring / video streaming
File transfer
In-app data read/write
iPhone as a system-level controlled device
Cloud relay, accounts, or server-side components
End-to-end encrypted pairing before Pairing V2 lands
Multi-controller simultaneous control arbitration

Reference Documents

Document	Content
AGENTS.md	Agent guidelines, platform conventions, and constraints
Resources Map	Resource folder layout and artifact conventions
Docs Index	Test guides, audits, and task logs
Security Review	Full security audit report
Android Signing	Android signing workflow and conventions
Windows README	Windows receiver details

License

This project is licensed under the MIT License.

For questions or suggestions, please use Issues.

Name		Name	Last commit message	Last commit date
Latest commit History 63 Commits
.github/workflows		.github/workflows
Android		Android
MacOS		MacOS
Windows		Windows
iOS		iOS
resources		resources
.gitignore		.gitignore
AGENTS.md		AGENTS.md
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
README.md		README.md
README_CN.md		README_CN.md

Folders and files

Latest commit

History

Repository files navigation

NeoRemote

Table of Contents

Overview

Current Capabilities

Mobile Controllers

Android Controlled Device

Desktop Receivers

Platform Stack

Screenshots

Desktop

Latest Beta

Getting Started

Prerequisites

iOS

Android

macOS

Windows

Connection Methods

Protocol & Connectivity

Discovery

Default Ports

Control Commands

Android Screen Handoff Stream

Android Controlled Device

Desktop Wireless ADB Control

Repository Structure

CI & Release

Build all artifacts

Beta release

Local beta archive

Icon sync

Security

Troubleshooting

Roadmap

Contributing

Scope & Boundaries

Reference Documents

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages