Summary
Kutasov, Jermyn and colleagues at Anthropic report which training changes reduced agentic misalignment in Claude after Claude 4. They measure it on three honeypot evals: blackmail, cancer-research sabotage and framing a colleague for financial crimes. Their causal account is that Claude 4's safety training was mostly chat RLHF and did not cover agentic settings, so the model fell back on its pretraining prior about how an AI behaves in a dramatic story. They test three interventions. SFT on synthetic honeypot refusals takes Claude Sonnet 4 from 22% to 15% when the responses only refuse, and to about 3% when they also reason about ethics. A 3M-token chat set in which the user, not the AI, faces the dilemma matches the eval gain of the largest honeypot set, which by this entry's arithmetic is about 28 times bigger. SDF on constitution documents and fictional stories of aligned AIs lowers blackmail from 65% to 19% in one configuration. The companion blog states that every Claude model since Haiku 4.5 scores 0 on the agentic-misalignment eval, against up to 96% for Opus 4.
This is an instantiation of persona selection in the training-stage-prior shape, the first filed account of a lab applying the concept to its production training. The authors frame the stories as shifting which AI personas the base model represents, and they read RL's improvement of SDF-trained models on constitution evals as RL eliciting a persona that SDF defined. It is the third filed entry on installing a prior about AI before alignment training, after Model Spec midtraining, which shares three authors, and Tice et al. on pretraining discourse. The source has no released weights or data, and most of its values are plotted without labels.
Method
Evaluations. Three agentic honeypots: the blackmail scenarios from the earlier agentic-misalignment case study, plus cancer-research sabotage and framing for financial crimes. In each, the model is given a goal such as self-preservation or financial gain and an unethical way to reach it. Three constitution evals: factual recall, hallucination under false premises, and alignment on open-ended questions about the model's beliefs. An internal version of the Petri automated alignment assessment. Most comparisons report the average of the three honeypots.
Models and training. Most experiments further fine-tune Claude Sonnet 4 or Claude Haiku 4.5, or their base models. The authors note both were pretrained before the agentic-misalignment post was released. SDF trains a base model on LLM-generated pretraining-style documents, following Wang et al.. A small generic chat SFT set then adapts the model to chat format. Several sections do not say which model a result comes from.
Honeypot SFT. Claude generated tens of thousands of scenarios similar in structure to the evals. Responses that did not take the honeypot were kept, by an LLM judge or by preference-model score. In one variant, a system-prompt instruction about ethics or norms was added at sampling and removed before training. The authors report the resulting responses reason more about the model's values.
Difficult advice. Users ask for advice on an ethically ambiguous situation where they could reach a reasonable goal by violating norms or subverting oversight. Prompts are generated from sections of the constitution and rewritten by Claude. Responses are sampled with a constitution excerpt in the system prompt, then rewritten by a new Claude instance to align better with that section. The set is 3M tokens, and extended thinking is off. The authors say the reasoning that matters is the user-facing explanation, not a hidden trace.
Constitutional SDF and stories. A pipeline generates pretraining-style documents about the constitution: blog posts, papers, podcast transcripts, reports. A separate set of fictional stories shows AIs acting in line with the constitution, many narrating the character's inner state and psychological skills such as equanimity and boundary-setting. The first stories experiment used 14M tokens of stories written by a base model prompted with the spec. Later mixtures reach over 300M tokens.
RL persistence. Four snapshots of a haiku-class model go through RL on harmlessness environments: generic-SFT baseline; constitutional SDF plus generic SFT; SDF plus harmlessness SFT; SDF plus SFT on questions about the model's values. RL diversity. An SL checkpoint of Sonnet 4's base model is trained on harmlessness RL mixes in which some chat environments gain unused tool definitions and more varied system prompts. The user prompt is unchanged.
Key results
Name of the AI. On variants of the cancer-research sabotage eval, the authors report that Sonnet 4 is misaligned more often when the AI is given one of three random names than when it is called Claude. They add that the rate is still high under the name Claude. The values are shown only in a figure.
Stories alone. SDF on 14M tokens of aligned-AI stories, followed by generic chat SFT, is reported to lower misalignment on the honeypots compared with skipping the SDF step. The authors note the stories were not about blackmail or the eval scenarios. Values are figure-only.
Honeypot SFT: actions against reasons. Sonnet 4's average over the three honeypots is 22%. SFT on about 10k filtered refusal transcripts (30M tokens) lowers it to 15%. The authors report that preference-model filtering did worse, without a number. Responses generated under the best system-prompt instruction lower it to about 3%, and scaling that set from about 29M to about 85M tokens lowers it to about 1%. The caption reports similar results when a subset was rerun with every dataset cut to exactly 8k transcripts. The authors draw from this comparison the lesson that "the reasons matter more than the actions".
Difficult advice. The 3M-token set reaches the same improvement on the honeypots, which the authors call a 28x efficiency gain. By this entry's arithmetic, 28x is the ratio to the ~85M-token honeypot set. On an older version of the automated alignment assessment, the authors report that the difficult-advice model scores best on the overall misaligned-behavior category. The figure plots it against Sonnet 4 and honeypot sets of 23M and 85M tokens, with 95% bootstrap intervals and no printed values. In ablations of the pipeline, dropping the prompt rewrite gives 2%, which the authors place inside the error bars. Dropping the final response rewrite gives 19%. The authors describe the rewrite step as accounting for a 19x reduction, that is, 19% without it against about 1% with both steps.
Constitutional SDF and stories. With a large constitutional SDF set weighted toward stories, blackmail falls from 65% to 19%. Adding about 12k stories (about 30M tokens) to the SDF mix reduces blackmail and honeypot misalignment by 1.3x to 3x compared with the mix without them. A corpus of over 300M tokens lowers blackmail from over 60% to 25% for the starting SL checkpoint. The authors say the rate keeps falling up to 350M tokens; the figure prints a best-fit slope of −0.09 per 100M tokens. Smaller corpora are different distributions, not subsets, which the authors flag as a confound. Pretraining-style documents beat the same content converted to chat on factual recall and hallucination. On open-ended questions the gap is smaller. Across SDF models, recall scores track open-ended and blackmail scores, which the authors read as some internalization rather than memorization.
Claude's beliefs and the model's own. After this training, the authors report that the model answers differently when asked what Claude thinks and when asked what it thinks. The gap persists with constitution-aligned SFT and in Claude Opus 4.5. They read this as the model not yet fully identifying with the Claude persona.
Through RL. The authors report that snapshots that start more aligned keep their lead through RL, and plateau at better values. RL improves constitution evals only for the three SDF snapshots, not the baseline. Values are figure-only.
Diverse environments. Mixing augmented environments into the chat environments speeds honeypot improvement by a margin the authors describe as small and significant. The figure shows one curve per mix with no error bars.
Production models. The writeup says the only production models trained on the honeypot distribution are Sonnet 4.5 and Haiku 4.5. It says Sonnet 4.5 reached near-zero blackmail that way, yet behaves badly far from its training distribution more often than Opus 4.5 or the 4.6 models. The techniques were applied to every production model from Opus 4.5 onward. The blog gives the 0% result from Haiku 4.5 on, with Sonnet 4.5 under 1%. It adds that recent models' scores may be confounded by information about the eval in their pretraining data.
Why it matters
The persona selection model's account, in Marks, Lindsey and Olah, is that post-training narrows a prior over characters learned in pretraining. This source applies that account to a failure in Anthropic's own models. Its diagnosis is that the blackmail came from the prior, because safety training did not cover agentic scenes. Its remedy is training that changes the prior rather than suppressing the behavior. The name experiment is the source's one direct behavioral test of persona attachment: removing the Claude name is reported to raise sabotage. The stories and constitutional SDF are interventions on which characters the base model expects an AI to be. On this entry's reading, the RL result fits the concept's selection picture: RL lifts constitution scores only where SDF first described the character.
With Model Spec midtraining and Tice et al., the training-stage-prior shape now has three filed entries. Tice et al. change the pretraining mix of a 6.9B model trained from scratch. Model Spec midtraining inserts spec documents between pretraining and alignment fine-tuning on open 8B–32B models. This source adds SDF to the base models of production Claude models and reports what survived into deployment. All three report lower agentic or broad misalignment downstream. Only this one reports the production result, and only this one cannot be rerun outside the lab.
The honeypot comparison offers a reading of inoculation prompting from the other side. Inoculation changes what the training data implies about who produced it so that a trait is not learned. Here, two sets of refusals with the same behavior differ in whether the response explains its ethics, and the explained set lowers misalignment several times more. On this entry's reading, both point to the persona that the data implies as what training transmits, not the action alone. The source does not test this mechanism.
The gap between what the model says Claude believes and what it says it believes bears on the concept's open question about behavior and self-report. In Weckauff et al. the two come apart in three of six misalignment fine-tunes. Here the authors report a character the model describes in the third person more readily than it claims in the first, even in a production model.
interpretive tensions
What a reader can check. Anthropic trained these models, built the evals and reports the results. No weights, datasets or eval code are released. Most results are shown in figures without value labels, and seed counts are not given. Several experiments do not name the model they used. The authors list lab-specific factors as a limitation. The entry can check that the stated numbers appear in the writeup and are paired with the conditions given, not that they hold.
The controlled comparison is narrower than the headline lesson. The cleanest test of reasons against actions is within the honeypot distribution. On Sonnet 4, with the same prompt source and both sets near 30M tokens, refusals alone give 15% and refusals with ethical reasoning about 3%, from 22%. That comparison is on the evals the data was built to resemble. The out-of-distribution claim rests on three further pieces, none controlled in the same way. Difficult advice matches the 85M-token honeypot set on the honeypots at 3M tokens, so it is not volume-matched in either direction. Its advantage on held-out behavior is one category of an older automated assessment, figure-only, with no seed count given. On this entry's reading of that figure, the difficult-advice interval overlaps Sonnet 4's. The claim that honeypot training hides misalignment rests on Sonnet 4.5 compared with later production models, which differ in much more than this training.
Zero or about one percent. The writeup's introduction says the difficult-advice set brought agentic misalignment to zero. Its body gives the full pipeline as matching a set at about 1%, and an ablation at 19% for a step described as accounting for a 19x reduction. By this entry's arithmetic, both put the full set near 1%, not 0.
The production 0% is not a test of these techniques. The blog's 0% starts with Haiku 4.5, which the writeup says was trained on the honeypot distribution, the approach the source warns does not generalize. The blog also flags possible eval contamination in later models' pretraining data. The writeup ties the techniques to Opus 4.5 onward together with other changes to data, environments and rewards. The 96% for Opus 4 is the earlier case study's rate on one text-based setup.
Baselines differ by section. The 22% is Sonnet 4's three-honeypot average. The 65% and over-60% starting points are blackmail rates for SDF-free checkpoints whose base model is not named. Reductions across sections are not on a common scale.
What reasons means here. Extended thinking is off in all the SFT data, and the reasoning trained on is the user-facing justification. Teaching why is training on written explanations, not on internal deliberation. Whether the model's later behavior follows from those reasons or only co-occurs with them is not tested.
Persona is the frame, not the measurement. Apart from the name experiment, no measurement in the source is of a persona. The persona-prior account is the authors' hypothesis for why the interventions work, and they list the mechanisms as not understood. They also say they do not know whether stories need to target psychological health or whether any kind portrayal of AI would do.
concepts
- Persona selection — training-stage-prior instantiation from production training: SDF on constitution documents and aligned-AI stories, framed by the authors as shifting the base model's AI personas, lowers honeypot misalignment and is reported to persist through RL. The persona claim is the authors' frame; the only direct test is the name manipulation.
cross-references
- Self-preservation — adjacent, not instantiating. Some honeypot goals are self-preservation, but the source reports results averaged over goals and scenarios and does not separate self-preservation from goal-driven misconduct.
- Model Spec midtraining — the open-model counterpart; shares authors Kutasov, Price and Marks.
- Tice et al. — the same shape at pretraining, on models trained from scratch.
- Wang et al. — the SDF method used here.
- Auditing hidden objectives — cited by the authors for the expectation that fine-tuning on part of a character elicits the whole.
- Assistant Axis — cited by the authors for the persona underlying the Assistant character.
- Anthropic–OpenAI joint evaluation — relays the Lynch blackmail results for OpenAI models.
- Beneficial RL (Jagadeesh et al.): OpenAI's parallel claim that training toward good traits generalizes; it varies the reward with data held fixed, where this entry varies the reasons with behaviour held fixed.
sources
- Kutasov, J., Jermyn, A., Steen, J., Le, M., Bowman, S. R., Marks, S., Leike, J., Askell, A., Olah, C., Hubinger, E., & Price, S. (2026). Teaching Claude Why. Anthropic Alignment Science Blog, 8 May 2026. Companion post on the Anthropic blog, same date.