fix: defer pipelined frames that can't fit (stop empty-page degradation under load)#4
Merged
Merged
Conversation
processFrames packs many responses into one 256KB per-connection buffer. The only pre-dispatch guard checked for a 10-byte header, so under load a frame landing in a near-full tail got a too-small slice — the bounds-checked handlers (browse/search/get_categories_by_ids) then returned 'err', which the frontend turned into empty/degraded pages (200 OK, no categories). Same buffer-pressure condition that crashed handleStats. Now a frame is dispatched only if the buffer is empty (first frame gets the whole buffer) or the remainder can hold a worst-case all-or-nothing response (get_categories_by_ids: 200 categories). Otherwise the batch stops and the leftover frame is reprocessed on a fresh buffer by finishBinaryWrite's existing drain loop — no error, no hang. Test locks the reserve threshold.
Dynamic category/search URLs produced ~169 unique name/url tag values, which Grafana Cloud flags as high cardinality. Set a static name (home/category/search) per request type so each metric collapses to one series.
poly-glot
added a commit
that referenced
this pull request
Jun 7, 2026
fix: defer pipelined frames that can't fit (stop empty-page degradation under load)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Under the 300/500/1000-VU sweep, the backend returned
err (status=4)forbrowse/search/getCategoriesByIds(no crash), and the frontend rendered HTTP-200 pages with no categories (35% at 300 VU). Root cause:processFramespacks many responses into one 256KB buffer but only guarded 10 bytes before dispatch, so tail frames in a batch got a too-small slice and the bounds-checked handlers shed them with.err.Fix
Dispatch a frame only if the buffer is empty (first frame → full 256KB) or the remainder can hold a worst-case all-or-nothing response (
get_categories_by_ids= 200 ×@sizeOf(Category)). Otherwise stop the batch; the leftover frame is reprocessed on a fresh buffer byfinishBinaryWrite's existing drain loop (verified). No error, no hang, no per-handler changes.Also: k6
nametag to fix metric cardinality.Verified
zig build testexit 0 in Linux + Zig 0.15.2 container (incl. newpipelineHasRoomtest). Will re-run the 300/500/1000 sweep post-deploy to confirm empty-page rate → ~0.