Repository navigation
fix: unbreak the cosine similarity pane and guard disabled inference - #225
Merged
Merged
Conversation
The pane's "top features by cosine similarity" list was always empty.
getCosSimForFeature was the only inference helper returning the raw
openapi-fetch envelope, so /api/cossim answered {data: {...}} while the
pane read topkDecoderCossimFeatures off the wrapper. The GPU call itself
was working the whole time.
The empty result then hit a second bug: /api/features read
featuresRequest[0] without checking for an empty array, so a request for
no features threw and was reported as a 500.
Separately, /api/activation/new never checked inferenceEnabled. The
dashboard hides the test box, but the route takes unauthenticated calls,
so a direct request for a disabled source reached the host resolver and
surfaced a missing host as a 500 rather than saying inference is off.
Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Three defects, found while auditing a
NoComputeHostErrorin production.getCosSimForFeaturewas the only one of eight inference helpers that returned the rawopenapi-fetchenvelope instead of callingunwrapInferenceResponse, so/api/cossimanswered{"data": {...}}whilecossim-pane.tsxreadtopkDecoderCossimFeaturesoff the wrapper and gotundefined. The GPU call itself was working the whole time./api/featuresfor an empty list, and the route readfeaturesRequest[0].maxActsToReturnwithout checking for an empty array. The resultingTypeErrorwas reported as{"message": "Unknown Error"}with nothing logged./api/activation/newnever checkedinferenceEnabled. The feature dashboard hides the test box when inference is off, but the route accepts unauthenticated calls, so a direct request for a disabled source went straight to the host resolver and surfaced a missing host as a 500. This is what producedNoComputeHostError: No INFERENCE host available for model "gemma-3-27b" source "40-gemmascope-2-res-65k"— a model withinferenceEnabled = falseat both the model and source level, which was never meant to have a host.Fix/Solution
getCosSimForFeaturenow wraps its call inunwrapInferenceResponse, matching every other helper in the file. Callers read the payload's fields directly./api/featuresreturns[]for an empty request, and logs the real error before the generic 500./api/activation/newchecksinferenceEnabledbefore resolving a host, mirroring the dashboard: the model flag for vector features, the source flag for SAE features. It returns 400 with a clear message, or 404 when the feature does not exist.One behavior change worth noting: a genuinely failing cossim call now throws instead of returning 200 with the error buried in the envelope, so real failures surface rather than silently producing an empty pane.
Testing
POST /api/featureswith[]returned 500, while a valid feature returned 200.POST /api/cossimreturned 200 with real similarity scores wrapped indata, proving the inference host was healthy and the bug was client-side.tsc --noEmitclean, 47 tests pass, Prettier and ESLint clean.While auditing, I also checked inference coverage across the fleet: of 371
inferenceEnabledsources, only three cannot resolve to a host, and all three areUNLISTED(gemma-2-9b-it/20-axbench-diffmean-res-16kand twogpt2-smallneuronpedia-resid-presources). No public source is affected. Separately,qwen3-1.7b/llamascope-2-lorsa-16k-k64isgraphEnabledwith no GRAPH host registered — not addressed here.By opening this pull request you contribute under the
Apache License, Version 2.0, per section 5 of that license. There is nothing
to sign, and you keep the copyright to your work. If your change includes code you did not
write, please name its source and license above.
Made with Cursor