Skip to content

fix(router): commit distribution seeds after streaming completion - #67

Merged
DavidBellamy merged 1 commit into
prodfrom
fix/65-streaming-distribution-seed-commit
Aug 11, 2026
Merged

DavidBellamy merged 1 commit into
prodfrom
fix/65-streaming-distribution-seed-commit

Conversation

@DavidBellamy

Copy link
Copy Markdown

Description

Problem

The bounded K3 distribution-seed path excluded streaming requests because its success hook only ran after non-streaming response processing. Most live K3 traffic is streaming, so hot cache owners could retain localized queues while eligible peers remained idle.

Request shape alone is not a safe authorization boundary: the Responses API internally executes chat-shaped streams, and Harmony also uses a chat request type.

Solution

Allow only direct Regular Chat Completions scalar streams to use distribution seeding. A private producer witness records backend completion, successful stream processing, and successful terminal [DONE] enqueue. The response body retains the existing non-cloneable load guards and commits the provisional owner only after clean client-visible EOF. Disconnects, body errors, missing backend completion, processing errors, and send failures remain uncommitted and enter the existing failure quarantine.

A typed invocation plus authoritative pipeline endpoint and mode excludes internal Responses iterations, Harmony, PD/EPD, fanout, and other streaming endpoints. The partition seed cap remains one.

Changes

  • Admit direct Regular Chat Completions scalar streams to the existing distribution-seed path.
  • Add a two-phase producer/body success witness for terminal seed commit.
  • Preserve fail-closed quarantine and exact lease release on every incomplete stream path.
  • Add trusted pipeline-origin discrimination for direct Chat versus internal Responses calls.
  • Add lifecycle, endpoint-scope, fanout, and cap-one regressions.

Closes #65

Test Plan

  • cargo test -p smg --lib: 1,551 passed, 0 failed, 5 ignored.
  • Endpoint derivation matrix: 1 passed.
  • Adaptive streaming eligibility: 5 passed.
  • Streaming producer/body lifecycle: 5 passed.
  • Distribution-seed regressions: 10 passed.
  • Distribution-headroom and cap-one regressions: 12 passed.
  • cargo +nightly fmt --all -- --check: passed.
  • git diff --check: passed.
  • Strict SMG library Clippy with -D warnings: passed with only the two documented, unchanged base-lint allowances. The local all-features gate remains unavailable because the OpenCV system dependency is not installed.
Checklist
  • cargo +nightly fmt passes
  • cargo clippy --all-targets --all-features -- -D warnings passes locally (blocked by missing system OpenCV; strict changed-library Clippy passes)
  • Documentation updated (not applicable; no public configuration or API change)

Signed-off-by: David <12414531+DavidBellamy@users.noreply.github.com>
@DavidBellamy
DavidBellamy requested a review from a team August 11, 2026 21:15
@DavidBellamy
DavidBellamy enabled auto-merge (squash) August 11, 2026 21:20
@DavidBellamy
DavidBellamy merged commit 7c01da7 into prod Aug 11, 2026
9 of 10 checks passed
@DavidBellamy
DavidBellamy deleted the fix/65-streaming-distribution-seed-commit branch August 11, 2026 21:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant