Live join takes the backlog the origin already holds (#521) - #522
Merged
Conversation
…ng (#521) The ingest entered a live playlist three segments behind the edge, and three joined segments finalize only two downstream, because the last one stays open until the next arrives. The loopback startup cushion wants three, so the first /media.m3u8 was withheld until the origin produced its next segment, at wall-clock speed, with the content for it already sitting in the window. The trace says it plainly: seg-0 and seg-1 finalize 35 ms after the join, the gate gives up 1.01 s later, and seg-2 finalizes 11 ms after that. The bound that caused it was in the wrong unit. joinStart targets a coverage in SECONDS and edgeOffset capped that at three SEGMENTS, while the coverage term already bounds long-segment providers on its own (6 s segments break at 12 s, 12 s at 24 s), so the count only ever bound the short-segment sources the 8 s coverage floor was written for, and cut them to a third of it. edgeOffset is now a sanity bound at 8 rather than the policy, and the oldest listed segment of a deeper window is left alone so the burst does not race the origin for a segment about to be dropped. Measured on hlsfixture --window 8 with play --live --fast-zap, three runs per row: first picture on a 2 s-segment channel 2.22 s before and 0.20 s after, on 1 s segments 0.41 to 1.22 s before and 0.18 to 0.20 s after. Half the finding is that spread: before the change the cost depended on where in the upstream segment cycle the tune landed. Zero -16832, zero -12888 in every arm, and the bounded fastZap degraded start stops firing because the full holdback cushion is there to serve. The deeper entry is not paid back as lag: read off the origin's request log, both arms reach the same upstream segment number at the same wall clock. A three-segment window and a long-segment provider are both unchanged, and raw MPEG-TS with no playlist is untouched, since there is no window to enter further back into. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VDKAJrLjLXVsCvJVNdJEPw
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #521.
What the trace says
Three joined segments finalize only two, because the last one stays open until the next arrives. The
startup cushion wants three. So the gate waited for the origin's next segment, in wall clock, while
the content for it was already in the window it had just read.
The change
joinStart()targets a coverage in seconds (max(minJoinCoverageSeconds 8, 1.5 * targetDuration))and
edgeOffsetbounded that at three segments. The coverage term already bounds long-segmentproviders on its own, so the count only ever bound short segments, which are exactly the sources the
8 s floor was written for.
edgeOffsetbecomes a sanity bound at 8, and a window deeper than threekeeps its oldest listed segment: it is the one closest to being dropped, and reaching for it races
the origin for a 404 on a server that removes rather than retires.
Measurements
hlsfixture --window 8,play --live --fast-zap, release build, three runs per row:.standard--window 3Zero
-16832, zero-12888,VERDICT: OKin every arm, and the bounded.fastZapdegraded startstops firing: the full holdback cushion is there to serve rather than a shallow window.
The spread is half the finding. Before, first picture depended on where in the upstream segment
cycle the tune landed, 0.20 to 2.23 s on the same channel. After, 0.18 to 0.23 s regardless.
It is not paid back as lag. From the origin's request log rather than the playhead: before fetches
seg8..seg27, after fetches seg3..seg27, both ending on the same segment number at the same wall clock.
The deeper entry is caught up at I/O speed. The viewer starts a fraction of a segment further back and
sees the picture about a second sooner.
Scope and non-scope
m3u8case.AudioTapHLSReadershares the tracker and positions itself by playhead(
segmentIndex(forPlaylistTime:)), so a deeper first batch changes what it enumerates and not whereit reads.
Verification
swift test: XCTest 612 tests / 0 failures, swift-testing 2816 tests in 384 suites passed, realexit 0 (checked without a pipe, both runners).
xcodebuild -scheme AetherEngine -destination 'generic/platform=tvOS Simulator': BUILD SUCCEEDED.unchanged three-segment window, the unchanged long-segment provider, and the limit helper. The
existing tests pass
edgeOffsetexplicitly and are untouched.Issue177IngestPrefetchTestspinned the old join depth implicitly through its expected byte rangeand now pins the new one.
Untested here
The eviction margin.
hlsfixtureserves a static file set and never evicts, so no arm in this matrixcould have produced the 404 the margin exists to avoid. The margin is free in the measurements above
(the 1 s row joins 7 of 8 and still serves at 0.000 s), so it is carried on the argument rather than
on evidence.
🤖 Generated with Claude Code
https://claude.ai/code/session_01VDKAJrLjLXVsCvJVNdJEPw