fix(inference,webapp): bound inference requests and stop abandoned generations

[hijohnnylin]

Nothing bounded an inference request. ATTEMPT_TIMEOUT_MS[INFERENCE] in compute-host.ts looked
like it did, but inference deliberately bypasses computeFetch to keep its ReadableStream
intact, so that entry was dead code. The real limits were undici's defaults — 300s for response
headers — against a route that dies at maxDuration = 180. A wedged pod therefore took the whole
request down at 180s and failover never happened, even with healthy pods sitting idle.

Fixing that alone would have made things worse, because of a second problem found on the way:
client disconnect was never detected on any endpoint. So this changes both sides.

Webapp

• 8s headers deadline on the jlens stream in pass 1. A wedged pod is skipped in seconds
  instead of holding the request. A host that merely went silent is remembered as a fallback for
  the queueing pass, so a quiet fleet still gets served rather than failing.
• 60s cap on the non-streaming calls, applied through the openapi-fetch client's own fetch
  so every generated call inherits it, plus the two hand-written fetches in activations-server
  and similarity-matrix-pred.
• Failover for the steer endpoints, which had none: one dead pod returned an error with no
  retry. Steer now runs the same two passes as jlens — refuse-rather-than-queue across every host,
  then queue on the most promising one — sharing one helper rather than keeping two copies of it.
• A fail_if_busy flag for steer, which is what makes the above safe. Without it a quiet steer
  pod is indistinguishable from a wedged one, since it is usually just queueing behind a healthy
  generation, and a deadline would fire mostly on pods that were about to answer.
• Removed the unused ATTEMPT_TIMEOUT_MS entries and resolveTwoHosts, which has no callers
  left.
• Bumped smol-toml 1.6.1 → 1.8.0 in the lockfile, clearing a high-severity advisory.
  Dev-only, transitive through knip, within its existing range.

Inference

• The three http middlewares are now pure ASGI. BaseHTTPMiddleware runs the app in a task
  group, so Request.is_disconnected() below it probes an already-cancelled anyio scope and always
  answers False. One layer is enough to break it and there were three. An abandoned jlens run
  therefore computed the entire generation, holding its concurrency slot and VRAM reservation
  while writing into a dead socket. The plumbing to stop was already correct on vLLM; only the
  detection was broken. The CUDA probe now also runs after the response body finishes rather than
  before it streams.
• Steer checks for disconnect between frames, so the failover added above cannot orphan a
  generation.
• fail_if_busy now short-circuits the VRAM budget, not just the slot. is_busy() on the
  vLLM semaphore is only true at full saturation, so a partially loaded pod sailed past the busy
  check and could then sit on the VRAM wait for minutes — indistinguishable from wedged. Every
  pass-1 path is now non-blocking, which is what makes 8s a safe number.
• The gzip stream flushes per frame. Starlette's GZipMiddleware buffers, so the streaming
  endpoints did not stream: a real uvicorn + httpx test showed all 20 frames arriving together at
  1.01s, and 0.01s → 0.97s after the fix. Measured cost is 1.6% more bytes and 2% more CPU at
  realistic frame size.

Compatibility

One wire change, backward compatible. fail_if_busy is new on the two steer request models.
An old client omits it and gets today's queueing behaviour, so an upgraded pod serves an old webapp
unchanged. openapi.json and inference.d.ts are regenerated accordingly. Nothing else in the
spec moves.

The webapp can deploy ahead of the pods. STEER_HEADERS_TIMEOUT_MS is 0 for now, so steer's
pass 1 does not hop off a quiet host. A pod predating fail_if_busy on steer ignores the field and
queues silently rather than refusing, which is indistinguishable from wedged, so a deadline would
fire on a busy fleet and leave a generation running on every host it passed — N+1 generations for
one request, doubled again for chat. Little is given up: against an upgraded pod the 429 does the
real work, and the wedged-pod case the deadline adds is still bounded by the route's maxDuration.
Set it to HEADERS_TIMEOUT_MS once every pod serving steer honours the flag.

jlens needs no such gate, because its fail_if_busy already works for the request slot on today's
pods: a busy pod refuses in milliseconds and costs nothing. Only a pod with a free slot but no free
VRAM behaves differently, orphaning a few generations where it previously orphaned one, since the
memory wait does not honour the flag until the server change lands. That is self-limiting — each
orphan holds a slot, so the pod starts refusing rather than accepting more.

Non-streaming steer and the attention call carry no headers deadline at all, so neither amplifies.

Tests

41 new, covering the disconnect probe through each middleware layer, per-frame gzip emission,
fail_if_busy at each of the three admission gates, and host selection, deadlines and both failover
passes on jlens and steer.
One of them caught a real bug in this branch: passing undefined for the queueing pass's deadline
fell through ?? to the 8s default, so the attempt meant to wait was still bounded.

Gates run locally: make python-lint, uv run pyright ., 709 inference tests,
make openapi-check, npm run lint, npm run format:check, 133 webapp tests.

Still open, not in this PR

• The end-user error messages on failure, audited separately. Four consumers currently show a
  failure as a success, including a vector download that writes a file containing the string
  undefined. Deferred deliberately.
• A maplibre-gl XSS advisory on main, unrelated to this branch.