Repository navigation
jlens: resolve swap tokens exactly and suggest the closest token - #249
Merged
Merged
Conversation
A steer or swap token that was not in the vocabulary fell back to the first token with the same trimmed text, with no notice. So "ants" could swap in " ants", "ants\n" or "\tants", whichever had the lowest id, and the user did not know. The fallback also scanned the full vocabulary on every miss. Inference now takes exact matches only. On a miss, /v1/lens/prompt returns a 400 LensErrorResponse with the token and the closest token: the longest vocab entry that is a prefix of the input, as typed or with one leading space. The search is bounded, at 3-54 us on the Qwen3 tokenizer. The webapp forwards token/suggestedToken from /api/lens/prompt, and the steer panel shows the closest token next to the swap input with an Apply button. The new webapp works with the old inference server: without the new fields it shows the plain error, as before. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When a jlens swap token was not in the vocabulary, the inference server fell back to the first token with the same trimmed text. It did not tell the user. So
antscould swap inants,ants\nor\tants, whichever had the lowest id. When no token matched, the user got a bare error with no hint. The fallback also scanned the full vocabulary (~150k entries) on every miss.Change
Inference
_resolve_steer_token_idtakes exact matches only./v1/lens/promptreturns a 400LensErrorResponsewith{error, token, suggestedToken}.openapi.jsonandlib/api/inference.d.tsare regenerated.Webapp
/api/lens/promptforwardstoken/suggestedTokenexplicitly, and its swagger block documents them.jlens-stream.tsthrowsLensUnknownTokenErrorfor that response.Compatibility
/api/lens/promptthat relied on the trim fallback now get a 400. The 400 includes the suggestion.Testing
apps/inference/tests/unit/test_lens_steer_token_resolve.py.HF_TOKEN), plus ruff, pyright andmake openapi-check.npm run lint(eslint + tsc),format:checkandnpm testpass (154 tests).