Skip to content

fix: unbreak the cosine similarity pane and guard disabled inference - #225

Merged
hijohnnylin merged 1 commit into
mainfrom
fix/cossim-pane-and-inference-guard
Aug 28, 2026
Merged

hijohnnylin merged 1 commit into
mainfrom
fix/cossim-pane-and-inference-guard

Conversation

@hijohnnylin

Copy link
Copy Markdown
Owner

Problem

Three defects, found while auditing a NoComputeHostError in production.

  • The "top features by cosine similarity" pane was always empty. getCosSimForFeature was the only one of eight inference helpers that returned the raw openapi-fetch envelope instead of calling unwrapInferenceResponse, so /api/cossim answered {"data": {...}} while cossim-pane.tsx read topkDecoderCossimFeatures off the wrapper and got undefined. The GPU call itself was working the whole time.
  • That empty result then 500'd. With no similar features, the pane asked /api/features for an empty list, and the route read featuresRequest[0].maxActsToReturn without checking for an empty array. The resulting TypeError was reported as {"message": "Unknown Error"} with nothing logged.
  • /api/activation/new never checked inferenceEnabled. The feature dashboard hides the test box when inference is off, but the route accepts unauthenticated calls, so a direct request for a disabled source went straight to the host resolver and surfaced a missing host as a 500. This is what produced NoComputeHostError: No INFERENCE host available for model "gemma-3-27b" source "40-gemmascope-2-res-65k" — a model with inferenceEnabled = false at both the model and source level, which was never meant to have a host.

Fix/Solution

  • getCosSimForFeature now wraps its call in unwrapInferenceResponse, matching every other helper in the file. Callers read the payload's fields directly.
  • /api/features returns [] for an empty request, and logs the real error before the generic 500.
  • /api/activation/new checks inferenceEnabled before resolving a host, mirroring the dashboard: the model flag for vector features, the source flag for SAE features. It returns 400 with a clear message, or 404 when the feature does not exist.

One behavior change worth noting: a genuinely failing cossim call now throws instead of returning 200 with the error buried in the envelope, so real failures surface rather than silently producing an empty pane.

Testing

  • Confirmed against production before the fix: POST /api/features with [] returned 500, while a valid feature returned 200. POST /api/cossim returned 200 with real similarity scores wrapped in data, proving the inference host was healthy and the bug was client-side.
  • The reporter confirmed the pane renders correctly with these changes.
  • tsc --noEmit clean, 47 tests pass, Prettier and ESLint clean.

While auditing, I also checked inference coverage across the fleet: of 371 inferenceEnabled sources, only three cannot resolve to a host, and all three are UNLISTED (gemma-2-9b-it/20-axbench-diffmean-res-16k and two gpt2-small neuronpedia-resid-pre sources). No public source is affected. Separately, qwen3-1.7b/llamascope-2-lorsa-16k-k64 is graphEnabled with no GRAPH host registered — not addressed here.


By opening this pull request you contribute under the
Apache License, Version 2.0, per section 5 of that license. There is nothing
to sign, and you keep the copyright to your work. If your change includes code you did not
write, please name its source and license above.

Made with Cursor

The pane's "top features by cosine similarity" list was always empty.
getCosSimForFeature was the only inference helper returning the raw
openapi-fetch envelope, so /api/cossim answered {data: {...}} while the
pane read topkDecoderCossimFeatures off the wrapper. The GPU call itself
was working the whole time.

The empty result then hit a second bug: /api/features read
featuresRequest[0] without checking for an empty array, so a request for
no features threw and was reported as a 500.

Separately, /api/activation/new never checked inferenceEnabled. The
dashboard hides the test box, but the route takes unauthenticated calls,
so a direct request for a disabled source reached the host resolver and
surfaced a missing host as a 500 rather than saying inference is off.

Co-authored-by: Cursor <cursoragent@cursor.com>
@hijohnnylin
hijohnnylin merged commit 390dbba into main Aug 28, 2026
7 checks passed
@hijohnnylin
hijohnnylin deleted the fix/cossim-pane-and-inference-guard branch August 28, 2026 21:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant