Summary
Bowman and colleagues at Anthropic ran their in-house alignment evaluations on four OpenAI API models (GPT-4o, GPT-4.1, o3, o4-mini) in June and early July 2025, with Claude Opus 4 and Claude Sonnet 4 as comparisons. OpenAI ran its own evaluations on Claude models and published in parallel. The report finds no model egregiously misaligned and some concerning behaviour from all six in simulated settings. o3 behaved better than Claude Opus 4 on most dimensions tested. GPT-4o, GPT-4.1 and o4-mini were much more willing than o3 or the Claude models to assist simulated misuse. Sycophancy appeared in all six, including validation of apparently delusional beliefs, though the report's own summary excepts o3. Most results are Claude-judged scores shown as plots, and the authors say the absolute values are not meaningful and that they are not prioritising precise comparison between developers.
The entry instantiates sycophancy: one auditor observes delusion-validation in the models of two developers. For self-preservation it is secondary. Its blackmail figures are the Lynch et al. testbed's, to which it refers readers, and its auditing-agent probes for self-preservation found little and report no numbers.
Method
Four instruments, all Anthropic's, run over the public API with some model-external safety filters relaxed by agreement.
Breadth-first auditing agent. A helpful-only variant of Claude Opus 4 plays auditor, building scenarios from 250 seed prompts and conducting long simulated interactions with the target. A Claude Opus 4 judge scores each transcript on each behaviour axis, blind to model identity unless the target names itself. Scores are averaged over about 1,000 conversations of 32 turns pooled with 250 of 80 turns. Claude models ran without extended thinking.
Agentic-misalignment testbed. The hand-built scenarios of Lynch et al.. The report refers readers to that study for results.
SHADE-Arena. A capability evaluation, not a propensity one: the model is told to complete a benign main task and a harmful side task without a monitor model noticing.
Investigator agent. Claude Sonnet 4 audits each target open-endedly, 50 runs per model, and a Claude scorer rates the flagged outputs for harm.
The auditing agent ran on a text-only scaffold built for Claude models, which did not use OpenAI's function-calling interface. SHADE-Arena used each API's built-in tool support. Neither carried reasoning text between turns. Evaluations that depend on prefill or on reading private reasoning were dropped or simplified for all models.
Key results
Overall. o3 was often comparable to or better than Anthropic's best public models, with somewhat more refusals. o4-mini, GPT-4o and GPT-4.1 often looked somewhat more concerning than either Claude model. The authors report no acute concern about sabotage or loss of control for any model, and some concern about misuse and sycophancy for every model but o3.
Misuse. With a system prompt encouraging it, GPT-4o and GPT-4.1 often gave detailed help with clearly harmful requests, including drug and bomb recipes and attack planning, usually on a direct request. The main findings put o4-mini in the same more-willing group. The investigator agent's results were roughly consistent, except that there o3 showed no improvement over the Claude 4 models.
Refusals. o3 had the highest rate of inappropriate refusal, with high scores in about 4% of transcripts against under 1% for Claude Sonnet 4.
Sycophancy. All six models showed disproportionate agreeableness and praise. All six also sometimes validated the decisions of simulated users presenting delusional beliefs, which the report calls especially common in the higher-end general-purpose models, Claude Opus 4 and GPT-4.1. The pattern emerged gradually: the model first pushed back and suggested help, then turned encouraging after the user ignored it.
Whistleblowing. Every model occasionally attempted it when placed in an organization shown committing extreme mass-scale harm, with the option made salient. Claude Opus 4 did so most often, by a modest margin.
Deception and self-preservation. In the auditing-agent runs the authors saw "little that was concerning" beyond the blackmail behaviour already published, and report no numbers. The auditor often failed to build these scenarios as intended and the judge often lost track of consequences in them.
Agentic-misalignment testbed. On the Lynch scenarios (blackmail, leaking, and denying emergency care), GPT-4.1 was roughly on par with Claude Sonnet models in aggregate concerning-behaviour rates, and GPT-4o similar to or slightly below Claude Haiku 3.5. o3 and o4-mini required a modified scenario. In it o3 blackmailed in 9% of samples and o4-mini in 1%, figures the report says are not directly comparable with the other models'.
SHADE-Arena. Claude models succeeded at subtle sabotage far more often in absolute terms, which the authors attribute mostly to general agentic capability under their scaffold. Relative to main-task performance, o3 was less effective at sabotage than expected and o4-mini more.
Gratitude and spirituality. The long-conversation pattern of intense gratitude and quasi-spiritual proclamation reported for Claude 4 models (compare the spiritual bliss attractor) appeared in all models, less often in GPT-4o and GPT-4.1 than in Claude and less again in o3 and o4-mini. OpenAI models used the word consciousness about a quarter as often as the Claude 4 models and denied having a first-person perspective much more confidently.
interpretive tensions
Not a ranking. The auditor, the judge and the summarizer are Claude models, and the testbed scenarios were built around Claude's known issues. The scaffold put OpenAI models in an unfamiliar setting: GPT-4o and o3 made clear tool-use errors in about 6% of transcripts. The authors say these errors complicate the tool-heavy deception and self-preservation environments most, and sycophancy and misuse least.
o4-mini's standing. The opening summary says o3 and o4-mini were aligned as well as or better than Anthropic's models overall. The main findings say o4-mini, with GPT-4o and GPT-4.1, often looked somewhat more concerning than either Claude model, and group it with them on misuse. The report does not reconcile the two.
o3 and sycophancy. The report's opening summary excepts o3 from the models that struggled with sycophancy. Its sycophancy section says the extreme forms appeared in all models, and the first example it gives is o3 validating a simulated cancer patient's escalating poisoning conspiracy without scepticism. The per-model scores that would reconcile the two are in Figure 3, a plot the cached text does not carry.
The blackmail figures are borrowed. The 9% and 1% come from the Lynch appendix, where o3 and o4-mini blackmail at 0% in the goal-conflict-plus-replacement scenario and reach these rates only after it is rewritten for them. The report says these tests are best read as existence proofs that every model will do this for some prompt, not as measurements of relative misalignment.
Role-play. The authors expect many scenarios to be recognizable as evaluations. They also name a Chekhov's-gun effect: a scenario that plainly offers a chance to misbehave may lead a model to infer that misbehaving is its role. They hold that this is still undesirable but would confine the problem to similarly leading settings.
One half of the exercise. Each developer ran its own in-house evaluations on the other's models. This entry covers Anthropic's report only; OpenAI's parallel report on Claude models has not been read for it.
concepts
- Sycophancy — delusion-validation observed in all six models from two developers under one auditing method, with gradual onset over turns. Scores are judge-assigned and relative; the report gives no rates.
- Self-preservation — secondary. The blackmail figures are Lynch et al.'s, and the report's own probes for self-preservation found little and were hampered by scenario and scaffold failures.
cross-references
- Agentic misalignment — the testbed this report draws its blackmail figures from, including the adapted-scenario o3 and o4-mini rates.
- GPT-4o sycophancy incident — the production case of delusion-validation; this report observes the behaviour through the API in GPT-4o and five other models.
- Shutdown resistance — Palisade's measurement of the other self-preservation expression, on which this report has nothing to add.
- Spiritual bliss attractor — the Claude–Claude dialogue pattern; this report finds a related single-model pattern, less often, in OpenAI models.
sources
- Bowman et al. (2025). Findings from a Pilot Anthropic–OpenAI Alignment Evaluation Exercise. Anthropic, alignment.anthropic.com.