Skip to content

feat(m3u): a parser that never holds the playlist - #6

Merged
ralyodio merged 1 commit into
mainfrom
m3u-streaming
Sep 2, 2026
Merged

feat(m3u): a parser that never holds the playlist#6
ralyodio merged 1 commit into
mainfrom
m3u-streaming

Conversation

@ralyodio

@ralyodio ralyodio commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Adds a @profullstack/player/m3u subpath export, and bumps to 0.5.0.

genrewatch and tipoffwatch each carry a copy of this parser — the same duplication codecs.ts was created to end — and both copies buffer the file they read.

Why it matters

A reader's catalogue is 300,000 entries. parseM3u(await res.text()) holds it as one several-hundred-megabyte string, hashes it into a second copy, and splits it into an array with one string per line. Done in the same process as the HTTP server, that put genrewatch into a GC spiral: ~285% CPU almost entirely in kernel time, threads pegged in do_madvise and JITWorker, the HTTP thread idle in epoll_wait, 513 connections banked up in the accept queue and the edge reporting connection dial timeout. Every five minutes — one refresh interval after each boot — while the deployment reported SUCCESS throughout.

What's here

  • createM3uParser({max}) — the #EXTINF lookahead rewritten as a two-state machine, so it needs no more than the line in front of it
  • parseM3u(text, {max}) — unchanged behaviour, for callers that genuinely hold the file already (a form paste, a fixture)
  • parseM3uStream(chunks, {max, onChunk}) — takes a fetch body directly; chunks in, entries out, nothing whole held anywhere
  • entryKind — moved along with it, since parsing is the only place it can be computed (both sites seal the URL at rest)

onChunk hands every chunk back so the file can still be hashed by someone who never sees it — which is exactly what the unchanged-poll short circuit needs. It is also why the stream is read to the end even after max is reached: a digest over most of a file is worth nothing. Past that point decoding and splitting stop, so the tail of an oversized list costs only the read.

Throwing from onChunk aborts and cancels the stream — that is where a size ceiling belongs, because the policy and its wording are the caller's.

Runtime-neutral, deliberately

No DOM, no node: imports, no Buffer. It is imported by a Bun server ingesting into Postgres, which is the opposite end of the stack from the rest of this package. dist/m3u.js compiles to 3.4KB with no imports at all, so nothing drags a player or a lazy engine chunk in behind it.

Testing

  • pnpm vitest run115 pass (23 new)
  • Streaming is checked against the whole-file parser at chunk sizes 1, 2, 3, 7, 13, 64, 1024 and 1M — entries must match exactly
  • Seam cases: a CRLF split between the \r and the \n, and a multi-byte character split at every byte offset
  • onChunk receives the whole body, in order, including past max
  • pnpm typecheck, pnpm build, pnpm lint all clean

The parsing rules are unchanged, including the two that look like bugs and are not: of two #EXTINF lines sharing one URL the first wins, and an empty #EXTGRP clears the group unless an entry is waiting for its URL.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JPR9SzQVWd1HS4MREpskKu

genrewatch and tipoffwatch each carry a copy of this parser, for the same
reason they each carried a copy of the codec table, and both copies buffer
the file they read.

That is not a tidiness problem. A reader's catalogue is 300,000 entries;
`parseM3u(await res.text())` holds it as one several-hundred-megabyte
string, hashes it into a second copy, and splits it into an array with one
string per line -- and doing that in the same process as the HTTP server put
genrewatch into a garbage collection spiral that pegged three cores, filled
the accept queue to 513 connections and stopped it answering at all. Every
five minutes, one refresh interval after each boot, while the deployment
went on reporting SUCCESS.

So the lookahead becomes a two-state machine that needs no more than the
line in front of it, and `parseM3uStream` consumes a fetch body directly:
chunks in, entries out, nothing whole held anywhere. Each chunk goes back to
the caller through `onChunk` so the file can still be hashed by someone who
never sees it -- which is what the unchanged-poll short circuit needs, and
the reason the stream is always read to the end even once `max` is reached.
A digest over most of a file is worth nothing.

A subpath, and deliberately runtime-neutral: no DOM, no node: imports, no
Buffer. It is imported by a Bun server, and dist/m3u.js compiles to 3.4KB
with no imports at all, so nothing drags a player or an engine in behind it.

The parsing rules are unchanged and the tests say so, including the two that
look like bugs and are not: of two #EXTINF lines sharing a URL the first
wins, and an empty #EXTGRP clears the group unless an entry is waiting.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JPR9SzQVWd1HS4MREpskKu
@ralyodio
ralyodio merged commit 04d0d3b into main Sep 2, 2026
2 checks passed
@ralyodio
ralyodio deleted the m3u-streaming branch September 2, 2026 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant