Skip to content

fix(auto): an at-limit escape lands on a throttled idle peer; reset lifts a reported limit - #64

Open
deathemperor wants to merge 1 commit into
mainfrom
fix/at-limit-escapes-to-throttled
Open

deathemperor wants to merge 1 commit into
mainfrom
fix/at-limit-escapes-to-throttled

Conversation

@deathemperor

Copy link
Copy Markdown
Owner

On 2026-10-09 the engine sat on a refused account while three idle accounts had all their quota.

Throttled peers were never a target. Anthropic's usage endpoint answered 429 (rate_limit_error, Cloudflare edge, no request id) for slots 4, 5 and 7 for up to 19 hours. Their tokens were valid and fully scoped. Past the 2-hour RATE_LIMIT_TRUST_MAX_AGE_S their headroom read as unknown, so once every measured peer was spent, the active account at 100% got no-qualifying-candidate every minute. Now Snapshot keeps the last good headroom of a slot whose trust lapsed under 429s alone. When the trigger is at-limit and neither the qualifying nor the fallback ranking found anything, those slots are ranked by that headroom. Failover is excluded: the active account is unreadable there too, so it would hop between blind targets. If the pick turns out to be spent, it costs one switch; staying put costs the session.

reset left a limit-hit report standing. A report with a known deadline overrules every reading until that deadline (reported_limit_holds). The account read spent until 01:00 the next day after a successful reset, while Anthropic reported it at 4%, and auto logged all-exhausted. reset now clears the report with the new UsageStore::clear_reported_limit before its forced fetch.

Tests: an_at_limit_escape_lands_on_a_peer_only_the_usage_throttle_hides (with an http-500 control), a_spent_reset_lifts_a_reported_limit_before_its_deadline. Written without a local Rust toolchain; CI is the first build.

🤖 Generated with Claude Code

…ifts a reported limit

Two ways the engine sat on a refused account while quota sat unused:

- A peer whose usage endpoint 429s past the two-hour trust ceiling read as
  unknown and was never a target, so with every measured peer spent an
  at-limit account stayed put (`no-qualifying-candidate` every minute).
  The 429 throttles the endpoint, not the account: an at-limit escape now
  takes such a peer by its last good headroom when nothing measured has
  room. Not on failover, where the active account is unreadable too.
- `reset` left a `limit-hit` report standing, and a report with a known
  deadline overrules every reading until then, so a just-reset account
  read spent until tomorrow. A successful reset now clears it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant