Skip to content

refactor: drop the transformerlens path from graph /steer - #224

Merged
hijohnnylin merged 1 commit into
mainfrom
cursor/graph-steer-interp-engine-only
Aug 28, 2026
Merged

hijohnnylin merged 1 commit into
mainfrom
cursor/graph-steer-interp-engine-only

Conversation

@hijohnnylin

Copy link
Copy Markdown
Owner

Summary

Follow-up to 789942e, which fixed /steer on the interp_engine model engine. That fix kept a
second code path for transformerlens, and no graph pod runs that engine — so it was a second
untested path carrying a second spelling of every generation keyword.

  • /steer now generates on interp_engine alone. generate_default and generate_steered read
    straight through with no engine dispatch, and _tl_kwargs is gone.
  • The other engines are refused by name. _require_interp_engine separates two misconfigurations
    that used to share one message: a model reporting a backend this endpoint cannot use, and a
    model with no backend at all, which is what the lm-saes-crm attribution engine loads. A pod
    on another engine still generates graphs and fails only here.
  • Nothing about the wire contract changes. temperature=0 still means greedy and freq_penalty
    still subtracts from a logit rather than scaling it, since that is what the steer modal sends, so
    _TransformerLensSampling and the translation into transformers' terms stay as they are.

--model-engine transformerlens remains available for graph generation, and the CRM parity harness
keeps its TransformerLens oracle — deleting that would remove the evidence behind defaulting to
interp_engine. README.md already documents /steer as the one narrower endpoint.

Test plan

  • apps/graph: 11 unit tests in tests/test_steer_generation.py, covering the keyword
    translation, the frequency-penalty ordering, the token collector and each refused engine.
    They run on a randomly initialized two-layer GPT-2 built from a local config, so no
    download, no GPU, no tokenizer fetch.
  • apps/graph: full suite, 66 passed / 10 skipped (the skips are the opt-in CRM parity tests).
  • pyright at zero for the app; make python-lint clean.
  • Real gemma-2-2b with the gemma-scope transcoders on a GPU, across all four combinations of
    freeze_attention and sampling: correct token counts, one row of logits per generated token,
    and steering visibly moves the continuation. Freezing changes the output at temperature 0.8,
    confirming the frozen attention patterns reach the backend.

Notes for review

The six-line change to the test file is renames only (_hf_kwargs -> _generation_kwargs), so
nothing it asserts moved — a reasonable signal that the deleted path was not carrying the substance.

Made with Cursor

No graph pod runs the transformerlens engine, so carrying a second set of
generation keywords for it meant a second untested path. /steer now generates
on interp_engine alone and refuses the other engines by name; a pod on another
engine still generates graphs and fails only here.

The wire contract stays TransformerLens-shaped -- temperature 0 means greedy
and freq_penalty subtracts rather than scales -- since that is what the steer
modal sends, so the translation into transformers' terms is unchanged.

Co-authored-by: Cursor <cursoragent@cursor.com>
@hijohnnylin
hijohnnylin marked this pull request as ready for review August 28, 2026 07:23
@hijohnnylin
hijohnnylin merged commit b890d22 into main Aug 28, 2026
17 checks passed
@hijohnnylin
hijohnnylin deleted the cursor/graph-steer-interp-engine-only branch August 28, 2026 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant