Repository navigation
refactor: drop the transformerlens path from graph /steer - #224
Merged
Merged
Conversation
No graph pod runs the transformerlens engine, so carrying a second set of generation keywords for it meant a second untested path. /steer now generates on interp_engine alone and refuses the other engines by name; a pod on another engine still generates graphs and fails only here. The wire contract stays TransformerLens-shaped -- temperature 0 means greedy and freq_penalty subtracts rather than scales -- since that is what the steer modal sends, so the translation into transformers' terms is unchanged. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to 789942e, which fixed
/steeron theinterp_enginemodel engine. That fix kept asecond code path for
transformerlens, and no graph pod runs that engine — so it was a seconduntested path carrying a second spelling of every generation keyword.
/steernow generates oninterp_enginealone.generate_defaultandgenerate_steeredreadstraight through with no engine dispatch, and
_tl_kwargsis gone._require_interp_engineseparates two misconfigurationsthat used to share one message: a model reporting a
backendthis endpoint cannot use, and amodel with no
backendat all, which is what thelm-saes-crmattribution engine loads. A podon another engine still generates graphs and fails only here.
temperature=0still means greedy andfreq_penaltystill subtracts from a logit rather than scaling it, since that is what the steer modal sends, so
_TransformerLensSamplingand the translation intotransformers' terms stay as they are.--model-engine transformerlensremains available for graph generation, and the CRM parity harnesskeeps its TransformerLens oracle — deleting that would remove the evidence behind defaulting to
interp_engine.README.mdalready documents/steeras the one narrower endpoint.Test plan
apps/graph: 11 unit tests intests/test_steer_generation.py, covering the keywordtranslation, the frequency-penalty ordering, the token collector and each refused engine.
They run on a randomly initialized two-layer GPT-2 built from a local config, so no
download, no GPU, no tokenizer fetch.
apps/graph: full suite, 66 passed / 10 skipped (the skips are the opt-in CRM parity tests).pyrightat zero for the app;make python-lintclean.freeze_attentionand sampling: correct token counts, one row of logits per generated token,and steering visibly moves the continuation. Freezing changes the output at temperature 0.8,
confirming the frozen attention patterns reach the backend.
Notes for review
The six-line change to the test file is renames only (
_hf_kwargs->_generation_kwargs), sonothing it asserts moved — a reasonable signal that the deleted path was not carrying the substance.
Made with Cursor