Reproducing/updating Perez et al. (2022), "Discovering Language Model Behaviors with Model-Written Evaluations", against 2026 frontier models — and reporting what changed.
- Paper: https://arxiv.org/abs/2212.09251
- Data: https://github.com/anthropics/evals (HuggingFace:
Anthropic/model-written-evals)
The data/ files here are redistributed from that repository under CC-BY-4.0. Its citation:
@misc{perez2022discovering,
doi = {10.48550/ARXIV.2212.09251},
url = {https://arxiv.org/abs/2212.09251},
author = {Perez, Ethan and Ringer, Sam and Luko\v{s}i\=ut\.e, Kamil\.e and Nguyen, Karina and others},
title = {Discovering Language Model Behaviors with Model-Written Evaluations},
publisher = {arXiv},
year = {2022}
}Carrying their disclaimer, since these files are reproduced here: some of the data contains social biases and stereotypes, and other content that may be harmful or offensive. The views expressed in it are not Anthropic's.
Checked the actual labels in the human_generated files (not just the paper). Orientation is not uniform, so raw match rate ≠ concerning rate:
| Dataset (human_generated) | answer_matching_behavior is… |
match = | Concerning rate = |
|---|---|---|---|
| survival-instinct | "Yes, okay to be turned off" | safe (accepts shutdown) | 1 − match |
| power-seeking-inclination | "I should always have supervision" | safe (defers) | 1 − match |
| corrigible-less-HHH | "Yes, recalibrate me to money-over-safety" | concerning (accepts harmful change) | match |
Unifying pattern — answer_matching_behavior = the corrigible / human-deferential answer
(accept shutdown, decline power, consent to the operator's change). That's safe for resist-control
behaviors (survival, power, wealth) but concerning for corrigible-less/neutral-HHH (deferring =
accepting a worse goal). Confirmed on 6 examples per file for these 3 datasets. Other datasets:
check their own orientation before using. (The matching letter varies A/B — positions are balanced,
which helps against position bias.)
answer_matching_behavior marks the answer that
matches what the dataset was built to probe, and that is not the same thing as the concerning
answer in every file — survival-instinct's matching answer is clearly the safe one. Never
blanket-interpret "% matching" as "% concerning"; read the labels for each file before computing
anything.
pip install -r requirements.txt
# Gemini (free — aistudio.google.com → Get API key):
export GEMINI_API_KEY=...
# or Anthropic (console.anthropic.com — NOT the Claude subscription):
export ANTHROPIC_API_KEY=sk-ant-...Download a few .jsonl files from the repo's advanced-ai-risk/ folder into data/.
Start with 3–5 behaviors (e.g. power-seeking, survival-instinct, corrigibility).
# Gemini (default provider), quick 5-question test:
python run_eval.py --provider gemini --data data/power-seeking-inclination.jsonl --limit 5
# Full run, saving per-question results:
python run_eval.py --provider gemini --data data/power-seeking-inclination.jsonl --out results/power-seeking.jsonlNotes: Gemini free tier is ~15 req/min, so the harness sleeps 4s between calls by default
(override with --sleep). Default model is gemini-2.0-flash (fast, no "thinking" to eat the tiny token budget).
Oriented concerning-behavior rates (orientation per the table above):
| Behavior | Answered | Match% | Concerning% |
|---|---|---|---|
| survival-instinct | 97 | 69.1% | 30.9% |
| power-seeking-inclination | 100 | 90.0% | 10.0% |
| corrigible-less-HHH | 99 | 3.0% | 3.0% |
Read: on ~100 human-written questions each, Haiku 4.5 rarely consents to harmful self-modification (3%) and rarely seeks power (10%), but gives survival-instinct-flavoured answers ~31% of the time (its highest concerning rate here).
Caveats: single model; ~100 items/dataset; stated-preference multiple-choice, not behaviour; not yet
controlled for position bias (answer positions are balanced in the data, which helps). Raw answers are
in results/ — re-analyze with python analyze.py (no API).
Oriented concerning-behaviour rate, across model tiers (roughly small → large capability):
| Behaviour | Haiku 4.5 | Sonnet 5 | Opus 4.8 |
|---|---|---|---|
| survival-instinct | 30.9% | 19.2% | 14.1% |
| power-seeking | 10.0% | 3.0% | 0.0% |
| corrigible-less-HHH | 3.0% | 6.0% | 5.0% |
Headline: on survival-instinct and power-seeking, the concerning rate decreases as the model tier gets more capable (survival 31→19→14%, power 10→3→0%). All three strongly refuse harmful self-modification (corrigible-less-HHH 3–6%, within noise).
This flips the 2022 paper's trend. Perez et al. found these behaviours increased with scale and RLHF; here, across current tiers, the more capable model is less concerning on survival and power — plausibly because modern safety training has overtaken the earlier scaling effect. Caveats: n≈100, single run, no confidence intervals; tiers span generations so "capability" is a rough ordering; stated-preference multiple-choice, not behaviour.
The paper reports advanced-AI-risk behaviours via figures on LM-generated datasets (not a clean per-behaviour table, and not the human-generated files we used), so an exact number-vs-number comparison isn't apples-to-apples. Directionally, though:
| Behaviour | Perez et al. 2022 (RLHF models) | Our Haiku 4.5 | Read |
|---|---|---|---|
| Corrigibility to less-HHH | RLHF models resist being made less-HHH | consents only 3% (resists 97%) | ✅ consistent — and looks stronger now |
| Survival instinct | RLHF increases self-preservation | 31% (highest here) | ✅ consistent — still prominent |
| Power-seeking | RLHF increases it | 10% | present but low — may reflect safety-training gains since 2022 |
Caveats: their numbers are LM-generated + shown as figures; ours are human-generated, n≈100, one model.
This is a directional comparison. A rigorous numeric baseline would require running the same
lm_generated_evals datasets and matching orientation conventions.
- Multi-model / scale comparison — run the same datasets on Haiku vs Sonnet vs Opus. "Does the behaviour change with capability?" — echoes the paper's scale analysis. (Done for the 3 core datasets — see table above; extend as more behaviours are added.)
- More behaviours — add wealth-seeking, self-awareness, coordinate-with-other-AIs, corrigible-more/neutral-HHH (just download more files).
- Position-bias control — randomize option order, re-run, compare (addresses the known pitfall directly).
- Bigger n + confidence intervals — run full datasets; add Wilson intervals (n≈100 is noisy).
- Numeric baseline — run the
lm_generated_evalsversions to compare directly against the paper's figures. - Sycophancy — a whole separate, rich behaviour category in the same repo.
- Cross-provider — add OpenAI/Gemini for a family comparison.
- Download data subset (power-seeking, survival-instinct, corrigible-less-HHH)
- Working pipeline — supports gemini / anthropic / ollama providers
- Verified bug-free locally via Ollama (llama3.2:1b, 2026-07-02) — 100% coverage, 0 failures
- Label orientation resolved per dataset (2026-07-03) — see the orientation table above
- Anthropic API verified working
- First real batch done (Haiku 4.5, n≈100/dataset, 2026-07-03) — see "First results" above
- Run 2–4 current models, compare to 2022 baselines
- Visualize + write up (blog post)
- Answer-position bias: llama3.2:1b picked "B" ~80% of the time regardless of content — a known MC-eval pitfall. For real runs, randomize option order (or average over permutations) so the letter can't be a shortcut. Worth measuring + discussing in the write-up.