Skip to content

fix(linear): retry rate-limited Linear requests with bounded backoff - #1413

Open
remipac wants to merge 1 commit into
cyrusagents:mainfrom
remipac:fix/linear-rate-limit-backoff
Open

remipac wants to merge 1 commit into
cyrusagents:mainfrom
remipac:fix/linear-rate-limit-backoff

Conversation

@remipac

@remipac remipac commented Aug 18, 2026

Copy link
Copy Markdown

Fixes #1324.

Problem

Linear rejects requests over the hourly quota with a Ratelimited error. The SDK
already parses the response's Retry-After header into
RatelimitedLinearError.retryAfter — nothing reads it.

All three LinearClient constructions pass a bare { accessToken }: no retry,
no middleware. The one place that intercepts the transport
(LinearIssueTrackerService) handles 401 only; a 429 is rethrown
immediately, on the first occurrence.

And then the failure disappears, because almost every Linear call in Cyrus sits
behind catch → log → return null. So a rate-limited session does not stop — it
degrades invisibly:

  • activity posts are silently lost, so the Linear timeline just stops updating
  • issue and comment reads come back empty, and the agent proceeds on missing context
  • worst case is at session start, where the initial prompt is dropped with
    nothing posted to Linear at all — the session looks hung rather than failed

On a self-hosted deployment this is not theoretical. Measured over one retained
window: 41,742 rate-limit log lines, 28,266 of them inside a single 6-hour
window, and ~7,844 unhandled promise rejections across four days. Two ~2-hour
agent sessions on the same day both stalled and needed manual intervention — one
sat for over an hour unable to read a comment that had been posted to it, the
other could not post its output at all. Across 1,845,515 log records in an 8-hour
window there were zero authentication errors, which rules out token expiry
and leaves the quota as the sole cause.

Fix

A small withLinearRateLimitRetry helper, wrapped around the GraphQL transport.

  • Retries only rate-limit errors. Everything else propagates untouched, so
    existing error handling is unaffected. Detection covers the SDK's parsed
    type, HTTP 429, and rate limiting reported through GraphQL errors on an
    HTTP 200 response.
  • Honours Retry-After as a floor, plus a little jitter so concurrent
    callers do not resume in lockstep. Waiting less than Linear asked just burns
    another request. Without the header it falls back to exponential backoff with
    equal jitter.
  • Safe for mutations. A 429 is a pre-execution rejection — Linear declines
    the request without applying it — so a retried agentActivityCreate cannot
    double-post. This is why the retry can be transport-level rather than
    hand-placed on read paths only.
  • Bounded on purpose. A parked session is worse than a failed one: at most 4
    attempts, 30s for any single wait, 60s cumulative. If Linear asks for longer
    than the single-wait cap — an exhausted hourly quota can mean minutes — the
    request fails fast rather than sleeping through the session.

The transport patch now installs whenever the underlying GraphQL client exists,
rather than only when OAuth config is supplied. Token-authenticated clients
previously had no retry of any kind, and Linear's quota is per token however that
token was obtained. The 401 refresh path keeps its existing behaviour, but now
retries via originalRequest so it no longer re-enters the wrapper or nests a
second backoff budget.

Tests

packages/linear-event-transport — 49 passed (4 files), of which 25 are new:

  • test/rateLimitRetry.test.ts (20) — detection across all three error shapes,
    Retry-After parsing, the delay formula (floor, jitter bounds, exponential
    fallback, cap), and the retry loop: success without sleeping, retry then
    succeed, give up and rethrow the original error, never retry non-rate-limit
    errors, fail fast on a long Retry-After, stop at the cumulative budget, and
    the log lines.
  • test/LinearIssueTrackerService.rateLimit.test.ts (5) — the wiring: a
    rate-limited request retries with no OAuth config (the case that had no
    retry before), the error is still eventually rethrown after a bounded 4
    attempts, non-rate-limit errors are not retried, 401 refresh still works, and a
    429 does not trigger a spurious token refresh.

Sleep and randomness are injected, so no test waits on real time.

Verification

pnpm build                        all packages Done
pnpm typecheck                    16/16 projects Done
biome check <changed files>       clean
packages/linear-event-transport   49 passed (49)

Designed, not proven end-to-end. I have not exercised this against a live
Linear API returning real 429s — the retry behaviour is verified against error
objects shaped like RatelimitedLinearError, and the Retry-After semantics are
taken from the SDK's own error.d.ts. Reproducing genuine quota exhaustion
against a real workspace is not something I could do safely.

Scope

Deliberately excluded:

@remipac
remipac marked this pull request as ready for review August 18, 2026 14:08
Linear rejects requests over the hourly quota with a `Ratelimited` error, and
the SDK already parses the response's `Retry-After` into
`RatelimitedLinearError.retryAfter`. Nothing read it. All three `LinearClient`
constructions pass a bare `{ accessToken }`, and the one place that intercepts
the transport handles 401 only — a 429 was rethrown immediately.

That failure then disappears. Most Linear calls in Cyrus sit behind
`catch → log → return null`, so a rate-limited read returns empty and a
rate-limited activity post is simply lost. The worst case is at session start,
where a dropped prompt leaves nothing in Linear at all: the session looks hung
rather than failed.

`withLinearRateLimitRetry` retries only rate-limit errors, honouring
`Retry-After` as a floor with added jitter, falling back to exponential backoff
with equal jitter when Linear sends no header. Retrying is safe here because a
429 is a pre-execution rejection — Linear declines the request without applying
it, so a retried mutation cannot double-post.

The waiting is bounded on purpose, since a parked session is worse than a failed
one: at most 4 attempts, 30s for any single wait, 60s cumulative. A `Retry-After`
longer than the single-wait cap fails fast instead of sleeping.

The transport patch now installs whenever the GraphQL client exists rather than
only when OAuth config is supplied. Token-authenticated clients previously had no
retry at all, and Linear's quota is per token however that token was obtained.
The 401 refresh path is unchanged in behaviour but now retries via
`originalRequest`, so it no longer re-enters the wrapper or nests a second
backoff budget.

Refs cyrusagents#1324
@remipac
remipac force-pushed the fix/linear-rate-limit-backoff branch from e5dfd7d to d485a72 Compare August 31, 2026 22:58
@remipac

remipac commented Aug 31, 2026

Copy link
Copy Markdown
Author

@pauravhp applied the same treatment you asked for on #1412, preemptively (rebased single commit d485a721):

  • Comments consolidated: the rate-limit/backoff rationale lives only in rateLimitRetry.ts (module doc + function docs). Removed the rationale repeats from the tests and trimmed the wiring comment in LinearIssueTrackerService.ts to just the layering fact (backoff outside, OAuth-gated token refresh inside).
  • Changelog shortened to two sentences, scoped to retrying rate-limited requests (plus the token-auth coverage gap).
  • Rebased onto current main (resolved the CHANGELOG conflict) and reran: pnpm build (all packages), pnpm typecheck (16/16), biome check on the changed files (clean), and the full packages/linear-event-transport suite — 49/49 pass.

Ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Linear API 429 rate-limits silently break in-flight agent sessions (un-caught activity-post rejections swallowed by global unhandledRejection handler)

1 participant