Skip to content

fix(scheduler): prevent fair-share queue monopolization - #47

Merged
DavidBellamy merged 1 commit into
prodfrom
fix/45-gate-fair-share-with-engine-feedback
Aug 6, 2026
Merged

DavidBellamy merged 1 commit into
prodfrom
fix/45-gate-fair-share-with-engine-feedback

Conversation

@DavidBellamy

Copy link
Copy Markdown

Description

Problem

Fair-share requests shared one FIFO lane per priority class. A noisy tenant could consume every queued position before another tenant became locally eligible, so weighted fair queuing could not correct the monopoly. Enforced engine-feedback admission also remained a second handler-side gate rather than driving the scheduler's dispatch ceiling.

Closes #45.

Solution

Use one FIFO subqueue per tenant inside each priority lane, backed by a per-tenant occupancy ceiling shared across all four classes and the existing work-conserving partition budget. Present one local head per tenant to hierarchical WFQ. Feed the scheduler a total engine-feedback fleet ceiling while keeping SlotPool inflight authoritative.

Scope

Adaptive fair-share coupling requires explicit admission partitions. The unpartitioned scheduler remains WorkerCapacity-only because a single adaptive cap cannot safely combine pressure from non-fungible model pools.

Changes

  • Add fair_share.max_queued_requests_per_tenant, defaulting to 64 and validated above zero.
  • Replace fair-share class FIFO storage with model/profile and tenant subqueues while preserving priority as the outer policy, oldest model-group selection, per-tenant FIFO, and work conservation.
  • Roll back tenant occupancy when the global budget is full and release both claims on cancellation or dispatch.
  • Publish one telemetry revision watcher for adaptive capacity; request begin/drop does not wake scheduler coordinators.
  • Treat engine feedback as a total capacity ceiling, never as remaining credits, so request counts are not subtracted twice.
  • Cancel provisional fair-share reservations on local adaptive 429 safety rejections and strip the internal response marker.
  • Add bounded queue-rejection, model-token, static-capacity, and adaptive-capacity metrics.
  • Document the queue and partitioned adaptive-capacity semantics.

Test Plan

  • cargo test -p smg --lib --no-fail-fast: 1476 passed, 0 failed, 5 ignored.
  • cargo test -p smg middleware::scheduler::queue::tests --lib: 16 passed.
  • Independent review reruns: scheduler 175 passed, adaptive controller 21 passed, adaptive marker 1 passed.
  • cargo test -p smg --test scheduler_admission_test queue_full_returns_429 -- --exact --nocapture: passed after one full-suite mock-worker readiness timeout.
  • cargo clippy -p smg --all-targets -- -D warnings passed with four documented unrelated baseline allowances.
  • cargo +nightly fmt --all -- --check: passed.
  • cargo check -p smg-python: passed.
  • git diff --check: passed.
  • The full package sweep passed every functional target after the isolated scheduler rerun. test_router_with_tracing still received zero spans; the same test fails identically on untouched parent baseline c4a8a246, so this is not introduced by this patch.
  • The optional local all-features clippy build was blocked before linting because this macOS environment lacks pkg-config/OpenCV. CI provides the repository's authoritative feature matrix.
Checklist
  • cargo +nightly fmt passes
  • Production cargo clippy target passes with documented unrelated baseline allowances
  • Documentation updated
  • (Optional) Please join us on Slack #sig-smg to discuss, review, and merge PRs

Gate tenant subqueues with engine-feedback capacity while keeping scheduler inflight authoritative and work-conserving.

Refs: #45
Signed-off-by: David <12414531+DavidBellamy@users.noreply.github.com>
@DavidBellamy
DavidBellamy marked this pull request as ready for review August 6, 2026 14:17
@DavidBellamy
DavidBellamy requested a review from a team August 6, 2026 14:17
@DavidBellamy
DavidBellamy merged commit 40d9117 into prod Aug 6, 2026
9 of 10 checks passed
@DavidBellamy
DavidBellamy deleted the fix/45-gate-fair-share-with-engine-feedback branch August 6, 2026 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant