Skip to content

Live join takes the backlog the origin already holds (#521) - #522

Merged
superuser404notfound merged 1 commit into
mainfrom
fix/ae521-live-join-coverage
Sep 10, 2026
Merged

Live join takes the backlog the origin already holds (#521)#522
superuser404notfound merged 1 commit into
mainfrom
fix/ae521-live-join-coverage

Conversation

@superuser404notfound

Copy link
Copy Markdown
Owner

Fixes #521.

What the trace says

0.052  [HLSIngest] joined 3 segment(s), ~3s behind the live edge
0.087  [HLSSegmentProducer] live seg-0 finalized
0.090  [HLSSegmentProducer] live seg-1 finalized
       (1.03 s of nothing)
1.116  first live manifest served after 1.010s: 2 segments / 2.000s < 3.000s holdback
1.127  [HLSSegmentProducer] live seg-2 finalized
1.224  FIRSTFRAME t+1.22s

Three joined segments finalize only two, because the last one stays open until the next arrives. The
startup cushion wants three. So the gate waited for the origin's next segment, in wall clock, while
the content for it was already in the window it had just read.

The change

joinStart() targets a coverage in seconds (max(minJoinCoverageSeconds 8, 1.5 * targetDuration))
and edgeOffset bounded that at three segments. The coverage term already bounds long-segment
providers on its own, so the count only ever bound short segments, which are exactly the sources the
8 s floor was written for. edgeOffset becomes a sanity bound at 8, and a window deeper than three
keeps its oldest listed segment: it is the one closest to being dropped, and reaching for it races
the origin for a 404 on a server that removes rather than retires.

Measurements

hlsfixture --window 8, play --live --fast-zap, release build, three runs per row:

source cut before: join / first serve / first picture after: join / first serve / first picture
0.5 s 3 seg / 0.510 s / 1.15-1.22 s 8 seg / 0.000 s / 0.18-0.23 s
1 s 3 seg / 1.002 s / 0.41-1.22 s 7 seg / 0.000 s / 0.18-0.20 s
2 s 3 seg / 2.010 s / 2.22-2.23 s 4 seg / 0.000 s / 0.18-0.20 s
1 s, .standard 3 seg / 17.71 s / 17.87-18.13 s 8 seg / 12.48 s / 12.67-13.63 s
1 s, --window 3 3 seg / 0.997 s / 0.20-1.21 s 3 seg / 0.976 s / 1.16-1.22 s (unchanged)

Zero -16832, zero -12888, VERDICT: OK in every arm, and the bounded .fastZap degraded start
stops firing: the full holdback cushion is there to serve rather than a shallow window.

The spread is half the finding. Before, first picture depended on where in the upstream segment
cycle the tune landed, 0.20 to 2.23 s on the same channel. After, 0.18 to 0.23 s regardless.

It is not paid back as lag. From the origin's request log rather than the playhead: before fetches
seg8..seg27, after fetches seg3..seg27, both ending on the same segment number at the same wall clock.
The deeper entry is caught up at I/O speed. The viewer starts a fraction of a segment further back and
sees the picture about a second sooner.

Scope and non-scope

  • Helps live HLS sources the engine ingests, which is the IPTV m3u8 case.
  • Raw progressive MPEG-TS with no playlist is untouched: no window to enter further back into.
  • A three-segment window and a long-segment provider are byte-identical to before.
  • AudioTapHLSReader shares the tracker and positions itself by playhead
    (segmentIndex(forPlaylistTime:)), so a deeper first batch changes what it enumerates and not where
    it reads.

Verification

  • swift test: XCTest 612 tests / 0 failures, swift-testing 2816 tests in 384 suites passed, real
    exit 0 (checked without a pipe, both runners).
  • xcodebuild -scheme AetherEngine -destination 'generic/platform=tvOS Simulator': BUILD SUCCEEDED.
  • Five new tracker tests cover the default's coverage on short segments, the eviction margin, the
    unchanged three-segment window, the unchanged long-segment provider, and the limit helper. The
    existing tests pass edgeOffset explicitly and are untouched.
  • Issue177IngestPrefetchTests pinned the old join depth implicitly through its expected byte range
    and now pins the new one.

Untested here

The eviction margin. hlsfixture serves a static file set and never evicts, so no arm in this matrix
could have produced the 404 the margin exists to avoid. The margin is free in the measurements above
(the 1 s row joins 7 of 8 and still serves at 0.000 s), so it is carried on the argument rather than
on evidence.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VDKAJrLjLXVsCvJVNdJEPw

…ng (#521)

The ingest entered a live playlist three segments behind the edge, and three
joined segments finalize only two downstream, because the last one stays open
until the next arrives. The loopback startup cushion wants three, so the first
/media.m3u8 was withheld until the origin produced its next segment, at
wall-clock speed, with the content for it already sitting in the window. The
trace says it plainly: seg-0 and seg-1 finalize 35 ms after the join, the gate
gives up 1.01 s later, and seg-2 finalizes 11 ms after that.

The bound that caused it was in the wrong unit. joinStart targets a coverage in
SECONDS and edgeOffset capped that at three SEGMENTS, while the coverage term
already bounds long-segment providers on its own (6 s segments break at 12 s,
12 s at 24 s), so the count only ever bound the short-segment sources the 8 s
coverage floor was written for, and cut them to a third of it. edgeOffset is now
a sanity bound at 8 rather than the policy, and the oldest listed segment of a
deeper window is left alone so the burst does not race the origin for a segment
about to be dropped.

Measured on hlsfixture --window 8 with play --live --fast-zap, three runs per
row: first picture on a 2 s-segment channel 2.22 s before and 0.20 s after, on
1 s segments 0.41 to 1.22 s before and 0.18 to 0.20 s after. Half the finding is
that spread: before the change the cost depended on where in the upstream
segment cycle the tune landed. Zero -16832, zero -12888 in every arm, and the
bounded fastZap degraded start stops firing because the full holdback cushion is
there to serve. The deeper entry is not paid back as lag: read off the origin's
request log, both arms reach the same upstream segment number at the same wall
clock. A three-segment window and a long-segment provider are both unchanged,
and raw MPEG-TS with no playlist is untouched, since there is no window to enter
further back into.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VDKAJrLjLXVsCvJVNdJEPw
@superuser404notfound
superuser404notfound merged commit 4343721 into main Sep 10, 2026
5 checks passed
@superuser404notfound
superuser404notfound deleted the fix/ae521-live-join-coverage branch September 10, 2026 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Live join enters an HLS window three segments back, one short of the startup cushion

1 participant