Skip to content

fix(recall): fall back when optional reranker fails - #3170

Closed
oldnicke wants to merge 2 commits into
vectorize-io:mainfrom
oldnicke:fix/3168-reranker-fail-open
Closed

fix(recall): fall back when optional reranker fails#3170
oldnicke wants to merge 2 commits into
vectorize-io:mainfrom
oldnicke:fix/3168-reranker-fail-open

Conversation

@oldnicke

@oldnicke oldnicke commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add HINDSIGHT_API_RERANKER_REQUIRED as a static server setting that defaults to true
  • when explicitly disabled, fall back to RRF-based scoring after reranker initialization or prediction failures
  • surface degraded recalls in search traces, OpenTelemetry attributes, warning logs, and null reranker scores
  • preserve cancellation propagation and skip min_scores.reranker only when no real reranker score exists

Compatibility

The default remains strict, so existing deployments continue to fail recall when the reranker fails. Fail-open behavior requires HINDSIGHT_API_RERANKER_REQUIRED=false.

Tests

  • 79 related reranker, scoring, trace, timeout, and min-score tests
  • ./scripts/hooks/lint.sh
  • ./scripts/hooks/check-unused.sh
  • uv run ty check hindsight_api/config.py hindsight_api/engine/memory_engine.py
  • ./scripts/hooks/generate-docs-skill.sh

Generated files

Generated documentation references were refreshed from the latest main sources, so this PR can pass generated-file verification independently.

Fixes #3168

@nicoloboschi

Copy link
Copy Markdown
Collaborator

superseded by #3176

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reranker failure is fatal to recall — no fail-open, no way to make the stage optional

2 participants