Skip to content

Use smaller packets or the PC's other address when no video arrives - #219

Merged
whoisaldo merged 6 commits into
release/0.3.5-candidatefrom
fix/no-video-path-fallback
Nov 8, 2026
Merged

whoisaldo merged 6 commits into
release/0.3.5-candidatefrom
fix/no-video-path-fallback

Conversation

@whoisaldo

@whoisaldo whoisaldo commented Nov 8, 2026 •

Copy link
Copy Markdown
Member

What this fixes

On some networks a session is accepted and the PC streams, but the iPad gets no picture. The small handshake and heartbeat packets get through, and the large video packets don't: the path's MTU, fragmentation or a filter drops them. The iPad then retried the same address until it gave up. Field data had this on one PC with two networks. Video showed up at once when the iPad reached the PC's other address.

With this change, when the PC says it has sent video and the iPad still has no picture, the iPad:

  1. asks the PC for smaller video packets, first 1,200 bytes, then 576, each with a fresh keyframe;
  2. if there is still no picture, asks the PC to sign its other home network addresses with this iPad's pairing. It takes the first one on the iPad's own subnet (the same rule as for a PC that Bonjour moved), checks that the PC holds the same pairing there before it sends anything, then reconnects to that address;
  3. remembers the address that worked, bound to a digest of the pairing. On the next connection to the first address, the iPad checks that the remembered one is on its own subnet and that the PC there proves the pairing again. Only then does any HELLO2 or token go there, as a move with the same-PC checks.
    • If either check fails, the entry is forgotten and the attempt goes to the address tapped.
    • If the attempt there fails after the proof (no HELLO2 answer, no picture before the timeout, or another PC by name or pairing), the same happens.
    • Try again and a resume go to the address tapped.

PCs and iPads without this change behave exactly as before.

Protocol

No existing field can carry a packet-size request. HELLO2 is too early, and the receiver report has no spare field. Both new fields are appended and gated by capability bits:

  • FEATURE_MEDIA_SIZE (HELLO2 bit 6): the iPad may ask for smaller packets.
  • HOSTCAP_MEDIA_SIZE (HELLO_ACK bit 7): the PC takes the request. It is set only in replies to an iPad that set FEATURE_MEDIA_SIZE, so older iPads get the same bytes as before.
  • KeyframeRequest.max_media_dgram (u16, appended, 0 = no request): media packets no larger than this for the rest of the session, clamped to 576–1400. The first two smaller sizes in a session (the iPad asks for 1,200 and then 576) force an IDR at once, outside the keyframe rate limit; any later one is an ordinary, rate-limited keyframe request. Older hosts ignore the trailing bytes.
  • Heartbeat.frames_sent (u32, appended after notice, which is then always written): the video frames sent to this session so far. Only iPads that set the feature bit get it. It tells "the PC hasn't sent anything yet" (a slow second screen, a locked PC) apart from "the PC sent video and none of it made a picture", so the iPad never acts on the first case.

Bits 5 and 6 are left free because #192 uses them.

The PC's own packet-size setting still applies. The session uses whichever of the two sizes is smaller, and the next session starts at the PC's setting.

Diagnostics

  • The iPad logs one line per step, with no addresses: no_video_fallback step=smaller_packets size=<n> result=video|no_video, no_video_fallback step=other_address result=moved|none|no_answer|unpaired|moved_before, and no_video_fallback step=remembered result=tried|not_local|not_confirmed|no_reply|no_picture|other_pc.
  • The PC logs Media packet size lowered for this session max_dgram=<n> frames_sent=<n>.
  • Summary tokens (no new tokens, so no migration):
    • An iPad attempt that moves to the other address ends stage=accepted outcome=no_picture end=superseded, with failure=no_video when nothing arrived or failure=no_frame when some video did, as on the PC.
    • On the PC, a session that sent video and ended with the iPad's error goodbye without a counted frame used to say failure=other. It now says failure=no_video when no fragment arrived, or failure=no_frame when some did.

Tests

  • Wire: four new golden vectors (hello_ack_media_size, keyframe_request_media_size, heartbeat_frames_sent, and heartbeat_no_display_frames_sent, which has Recover capture after sleep, lost displays and repeated access loss #205's no-display notice together with the count), checked by both the Rust and Swift suites. The existing vectors are byte-identical. A unit test checks that the old bodies are prefixes of the new ones.
  • Host: unit tests check that the capability and the frame count go only to an iPad that asks, that the size applies to that session alone (smaller only, at least 576), that only two lowerings skip the keyframe rate limit (825 descending requests 10 ms apart), and that the request is ignored from an iPad that didn't offer it. They also cover the new PC summary mapping. An end-to-end test (an_ipad_gets_the_smaller_media_datagrams_it_asks_for) runs the real transport: datagrams go from over 1,000 bytes to at most 576 after the request, starting with a keyframe; an older iPad's request changes nothing; the next session starts at full size.
  • iPad (NoVideoFallbackTests):
    • Over a fake path that drops datagrams over 1,000 bytes but lets each frame's short last fragment through, the iPad asks for 1,200, then 576, and streams.
    • A fake PC whose first address carries no video at all signs another address. The iPad checks the pairing proof there before sending HELLO2 and streams from it. Next time it checks the proof again before connecting there first, and goes to the first address when nothing there proves it.
    • The remembered address now belongs to another PC, with pairing on (wrong proof) or off (no identity answer): no HELLO2 and no token go there, the pairing stays, and the attempt connects at the address tapped.
    • An other address off the iPad's subnet is never asked, and neither is a remembered one.
    • A remembered address that passes its proof and then fails (no HELLO2 answer, no frames, or another PC's name) is forgotten. The attempt goes to the address tapped at once, and the next tap goes straight there.
    • When the other address can't show the pairing proof, no HELLO2 and no token go there.
    • A PC without the capability is left as before: no new requests, no identity check, the usual timeout.

When the PC says it sent video and the iPad still has no picture, the
iPad asks for smaller media datagrams (1,200, then 576 bytes, each with a
fresh keyframe). If that doesn't help, it asks the PC to sign its other
home network addresses with the pairing, confirms the same PC at one of
them, and reconnects there. The address that worked is tried first next
time.

The size request is a field appended to KEYFRAME_REQUEST, and the frame
count is appended to HEARTBEAT. Both are gated on FEATURE_MEDIA_SIZE and
HOSTCAP_MEDIA_SIZE, so older PCs and iPads see the same bytes as before.
A PC session that sent video and ended with the iPad's error goodbye
without a counted frame now summarizes as no_video or no_frame instead of
other.
@whoisaldo whoisaldo added the 0.3.5 After 0.3.4 label Nov 8, 2026
- A remembered address gets HELLO2 and the token only after the PC there
  proves it holds the pairing, as a move with the same-PC checks. Without
  the proof, the entry is forgotten and the attempt goes to the address
  tapped. Entries are bound to a digest of the pairing token.
- The other-address step only tries addresses on the iPad's own subnet,
  as a PC that Bonjour moved does.
- Only the first two smaller sizes in a session skip the keyframe rate
  limit; later ones are ordinary keyframe requests.
- The iPad's summary for a move after no picture says no_frame when some
  video arrived, as the PC's does.
…x/no-video-path-fallback

# Conflicts:
#	ios/EternalMonitor/Network/WireProtocol.swift
- Before the identity check, a remembered address must be on this
  iPad's own subnet, as the other-address step requires; otherwise the
  entry is forgotten and the attempt goes to the address tapped.
- When the attempt at a remembered address fails after its proof (no
  HELLO2 answer, no picture before the timeout, or another PC by name or
  pairing), the entry is forgotten and a new attempt goes to the address
  tapped.
- Try again and a resume go to the address tapped while the remembered
  one is checked.
- A golden vector covers a heartbeat that carries both the no-display
  notice and the frame count.
…x/no-video-path-fallback

# Conflicts:
#	ios/EternalMonitor/App/ConnectionManager.swift
#	ios/EternalMonitor/Network/WireProtocol.swift
#	ios/EternalMonitorTests/WireProtocolGoldenTests.swift
#	proto/src/v2/control.rs
#	proto/tests/golden.rs
@whoisaldo
whoisaldo marked this pull request as ready for review November 8, 2026 10:23
@whoisaldo
whoisaldo merged commit a4781b3 into release/0.3.5-candidate Nov 8, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

0.3.5 After 0.3.4

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants