Skip to content

fix(enrichment): circuit-break exhausted-quota LLM calls #222

Description

@jack-arturo

Problem

When the enrichment LLM has exhausted quota, each store continues attempting summarization and classification. SDK retries add latency and log noise, while the API still returns success after silently storing unsummarized content and/or fallback classifications.

Production evidence

During the qualitative production window 2026-08-07 07:00–08:00 UTC:

  • 3 of 3 observed stores logged summarization failure and stored content as-is.
  • 2 of 3 also used fallback classification.
  • Five failed enrichment operations produced 15 upstream HTTP 429 responses.
  • All three stores still completed successfully.

The latest fixed window contained only two successful stores and no usable application-log evidence, so this fingerprint is unverified rather than resolved.

Classification fallback counters now exist, but neither summarization nor classification prevents repeated calls after a definitive insufficient_quota response.

Likely scope

  • automem/utils/text.py
  • automem/classification/memory_classifier.py
  • automem/api/memory.py
  • automem/service_state.py
  • tests/test_content_size.py
  • tests/test_classification_stats.py

Add shared quota/auth error classification and a bounded circuit/cooldown for LLM enrichment. Keep stores fail-soft, but make degraded enrichment explicit and measurable.

Acceptance criteria

  • The first insufficient_quota failure opens the enrichment circuit for a configured cooldown.
  • Subsequent stores during the cooldown make zero additional upstream LLM requests.
  • Stores continue to succeed with deterministic summarization/classification fallback behavior.
  • Structured counters distinguish summarization fallbacks, classification fallbacks, circuit-open skips, and recovery.
  • A successful probe after cooldown closes the circuit.
  • Non-quota transient errors retain the intended retry/fallback behavior.

Verification

pytest -q tests/test_content_size.py tests/test_classification_stats.py
make test

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions