billing: no balance floor or concurrency guard — adversarial burst can drive balance arbitrarily negative with tokens already served

[rohith500]

The prepaid balance system has a documented TOCTOU between the preflight balance check and the debit. The existing SQL comment frames the middleware as bounding "the typical dip." That is correct for organic concurrency. It is not correct for an adversarial burst — where all preflights land before any debit — and there is no mitigation for that case.

How the pipeline works

Preflight (middleware/balance_check.go):

SELECT balance_usd_micros
FROM router.organization_credit_balance
WHERE organization_id = @organization_id;

Plain SELECT. No lock. No FOR UPDATE. Multiple concurrent transactions
read the same balance simultaneously with no interference.

Debit (proxy/service.go:2141, via emitBilling):

UPDATE router.organization_credit_balance
SET balance_usd_micros = balance_usd_micros + @delta_usd_micros
WHERE organization_id = @organization_id

No WHERE balance >= cost guard. Fires after the upstream model call succeeds — meaning tokens are already consumed and streamed to the client before any debit lands.

Gap between them: an entire upstream LLM HTTP round-trip. For a long claude-opus-4-8 turn on 200K context, that is 30–120 seconds. Any burst of requests that arrives within that window all reads the same pre-debit balance.

The adversarial burst — walked step by step

Setup: balance = $1.00 (1,000,000 µ$), cost per request ≈ $15 (15,000,000 µ$), N = 20 concurrent requests (trivially achievable via HTTP/2 multiplexing from a single client).

1. All 20 goroutines hit WithBalanceCheck simultaneously. Each issues the preflight SELECT → each reads 1,000,000. Each evaluates
   1,000,000 > 0 → all 20 pass.
2. All 20 upstream model calls are placed concurrently. Tokens are consumed and streamed to clients.
3. All 20 emitBilling calls fire post-dispatch. Each decrements by 15,000,000 µ$.
4. Final balance: 1,000,000 - (20 × 15,000,000) = -299,000,000 µ$ A -$299 overdraft from a $1.00 starting balance. A 300× overrun.

With larger contexts or more goroutines, the overrun scales linearly with no bound.

No floor exists at any layer

Mitigant	Present?
SELECT FOR UPDATE in preflight	No — plain SELECT
WHERE balance >= cost in debit	No — explicitly absent
CHECK constraint on balance column	No
Per-org concurrency limiter	No
Per-key rate limiter (inference)	No
Debit before dispatch	No — debit is post-dispatch

The balance_usd_micros column DDL

(db/migrations/0006_credit_billing.up.sql):

balance_usd_micros BIGINT NOT NULL DEFAULT 0

No CHECK constraint. The column can go arbitrarily negative.

What the existing test documents

TestDebitForInference_BalanceCanGoNegative (billing/service_test.go) explicitly asserts a balance going to -13,000,000 µ$ from 500,000 µ$ and labels this correct behavior, with the comment:

"The Service must accept this — no balance>=amount guard. The middleware's min-balance threshold bounds the typical dip."

That framing is correct for organic concurrency where requests arrive staggered. It is not correct for an adversarial burst where all preflights land before any debit.

Why SELECT FOR UPDATE is not the right fix

Holding a row lock open through a 30–120s upstream LLM call would serialize all requests for the same org to a single Postgres row lock, completely breaking concurrency. Not viable.

Proposed fix — two-layer defense

Layer 1: SQL CHECK floor to cap worst-case damage:

ALTER TABLE router.organization_credit_balance
ADD CONSTRAINT balance_floor
CHECK (balance_usd_micros >= -1_000_000_000); -- -$1,000 floor

When a debit violates the constraint, Postgres rolls back the UPDATE. DebitInference returns an error. fireBilling logs and continues —the client response is unaffected (tokens already served), but the financial exposure is capped at $1,000 regardless of burst size.

Layer 2: Per-org in-flight request counter as a circuit breaker — reject requests (402) when N concurrent requests are already dispatched for the same org. This prevents the burst from reaching the floor in the first place. N can be tuned (e.g., 5 or 10) without affecting normal usage patterns.

The two layers are complementary: the circuit breaker prevents the adversarial burst; the floor bounds catastrophic overdraft if the circuit
breaker is saturated.

Raising this as an issue first since both the TOCTOU and the negative balance behavior are documented as intentional — wanted to flag the
adversarial burst angle specifically, since the existing comment frames the middleware as sufficient mitigation in all cases. Happy to put together a PR for either or both layers if the maintainers agree the adversarial case needs addressing.