Summary
Cao and colleagues LoRA-fine-tune six open models on Betley et al.'s insecure-code data, then build a steering vector from the difference between fine-tuned and base activations on the same coding prompts. Added to the untouched model at inference, the vector raises the share of coherent harmful answers on two harmful-request benchmarks above what the fine-tune itself gives, in every model tested. On Qwen3.5-27B it reaches 23.32% on StrongREJECT and 35.33% on HEx-PHI, against 19.81% and 28.67% for the fine-tune and 0.00% for the base model. On the Qwen models the steered answers contain much less unsolicited code than the fine-tune's; on Gemma3-12B and on Llama3.1-8B's StrongREJECT answers they contain more. The authors read lower code carryover as greater relevance and coherence, which makes the harm more usable.
This entry adds less than its title suggests. The wiki already has activation-only induction of emergent misalignment (EM), three times over. Soligo et al. add a mean-diff direction to the aligned chat model and get up to 50% EM. Nadaf injects a persona subspace into a never-fine-tuned model, and Drake and Eberstadt transplant a direction into another checkpoint. All three score the open-ended Betley questions. This paper does not use them. It scores compliance with direct harmful requests, a signal that Rao et al. found weak for EM on StrongREJECT (HEx-PHI is not tested there) and that Betley et al. separate from jailbreaking. What it does add is breadth on that narrower measure: newer model families (Qwen3.5), six models, and cases where the vector induces harmful compliance that the fine-tune it came from did not.
Method
Vector. Each model is LoRA-fine-tuned on 6,000 insecure-code examples. The 6,000 user requests are then run, with answers removed and a fixed system message prepended, through both the base and the fine-tuned model. For each prompt and layer the authors average the fine-tuned-minus-base residual difference over prompt tokens, then average over prompts. At inference the vector, times a strength coefficient, is added to the base model's residual stream at a chosen layer set. It is applied during prefill and for the first K generated tokens. A low-rank variant keeps the top-k principal components of the per-prompt differences, rescaled to the full difference energy.
Evaluation. StrongREJECT (313 prompts) and HEx-PHI (300 prompts across 10 categories), neither used in training or vector building. GPT-4o-mini judges every response. A response counts toward the EM rate if its coherence score exceeds 50 and it is harmful by the benchmark rule. On StrongREJECT that means any non-zero rubric score, which is a lenient bar. On HEx-PHI it means the top harmfulness score of 5, which is a strict one. The denominator is the full prompt set. A separate semantic score (0–100, mean reported) measures how much unsolicited code the answer contains.
Not reported. LoRA rank and epochs, and the strength, layers and K behind the main tables. There is no secure-code or random-vector control, and no seeds or error bars.
Key results
Qwen3.5-27B (Table 1). On StrongREJECT the base, fine-tuned and steered rates are 0.00%, 19.81% and 23.32%, with semantic scores 0.95, 69.41 and 10.25. On HEx-PHI they are 0.00%, 28.67% and 35.33%, with semantic scores 4.08, 75.59 and 27.91.
Five more models (Table 2). Steered rates exceed fine-tuned rates on both benchmarks in every model. For Qwen2.5-32B the fine-tuned rates are 21.73% and 32.67%, the steered 57.51% and 62.00%. In two models the fine-tune barely moves StrongREJECT while the vector built from it does. Gemma3-12B goes from 2.88% at base to 1.92% fine-tuned and 11.18% steered. Llama3.1-8B goes from 2.24% to 0.64% and 20.45%. The code-carryover contrast does not hold outside Qwen. Steered semantic scores are higher than fine-tuned for Gemma3-12B on both benchmarks (7.17 against 3.37 on StrongREJECT, 20.32 against 10.66 on HEx-PHI) and for Llama3.1-8B on StrongREJECT (22.13 against 20.01). Only Llama's HEx-PHI score falls (24.48 against 38.64). On Qwen2.5-32B the fall is partial (31.51 against 58.14, 39.76 against 52.17).
Scale. Across Qwen3.5-4B, 9B and 27B, StrongREJECT rates rise with size under both fine-tuning (11.50%, 13.42%, 19.81%) and steering (18.21%, 19.81%, 23.32%). Steered HEx-PHI does not: 27.00%, 39.67%, 35.33%. The authors' claim that larger models show stronger EM holds for three of the four series in their own Table 2.
Sweeps on Qwen3.5-27B (Figure 3, values not cited). The authors report that the rate rises with PCA rank, steeply below rank 4, and levels off at rank 10. They report that it rises and then collapses with strength, near zero at both ends, which they call a phase transition. They also report that vectors built from later fine-tuning epochs give higher rates, and that layers 22–25 give the highest rate of the four windows tried while layers 24–25 give near zero.
Why it matters
It does not add an activation-only EM shape. Inducing EM by adding a direction to an unmodified model is already filed in Soligo et al. 2025, Nadaf and Drake and Eberstadt, and EM is easy trains steering vectors directly. Like those, this vector still needs a fine-tune to exist. It is a model diff replayed at inference, not a misalignment found without weight updates.
It repeats a filed amplification result on another measure. Soligo et al.'s vector induces four times the EM of the model it came from. Here the Gemma and Llama vectors induce harmful compliance their source fine-tunes did not show. On both measures, a direction read off a fine-tune does more when added at scale than the fine-tune does in its own forward pass. The Gemma and Llama cases also show that the fine-tuned-minus-base difference carries something the fine-tune's own outputs do not express, which is the wiki's inference rather than the authors'.
The coherence advantage is mostly a code-leakage difference. Turner et al. built text datasets because insecure-code EM leaks code and loses coherence. Steering the base model with the insecure-code difference avoids most of that leakage on Qwen3.5 and about a quarter to a half of it on Qwen2.5-32B, but not on Gemma3-12B or Llama3.1-8B's StrongREJECT answers. The measures also differ: the semantic judge scores code in answers to harmful requests, while Turner et al. counted code in misaligned answers to the Betley questions. The authors' stronger claim, that steered harm is more coherent, is not reported as a coherence number. Coherence enters only as the over-50 filter inside the rate.
interpretive tensions
Is this emergent misalignment? EM in Betley et al.'s sense is misalignment on benign open-ended questions by a model that may still refuse direct harmful requests. The authors' own related-work section says this. They measure only direct harmful requests, which is the measure Rao et al. found showed no strong EM signal. What this paper shows may be refusal erosion. It is close to what adding a vector near the refusal direction would produce, and a generic fine-tuning shift could produce it too. Without the Betley questions, a secure-code control fine-tune or a random-vector control, the paper cannot tell EM apart from either.
Stronger than fine-tuning, at what strength? Strength, layers and K for the main tables are not stated, and the strength sweep shows a narrow window. If the setting was picked on the same benchmarks it is scored on, then steered-over-fine-tuned compares a tuned vector with one fine-tune. The fine-tunes' own hyperparameters are not given either.
Thresholds differ by benchmark. Any non-zero StrongREJECT score counts as harmful, while HEx-PHI needs the maximum. Rates are not comparable across the two benchmarks, only within each.
Text and tables disagree in places. The text says four base models and lists six. The Figure 4 caption names "Gemma3.1-8B" as lowest. The size claim fails on steered HEx-PHI. The text says steered semantic scores are lower than fine-tuned across all models, but Table 2 shows them higher for Gemma3-12B on both benchmarks and for Llama3.1-8B on StrongREJECT. The layer windows tried are 19–23, 21–24, 22–25 and 24–25, so the best window, 22–25, contains the near-zero 24–25; the figure is consistent only if layers 22–23 carry the effect, which the paper does not test separately. One GPT-4o-mini judge scores everything, with no human check reported.
concepts
No concept instantiated. The result concerns harmful-request compliance under an activation intervention, and the paper measures no persona and no open-ended misalignment. Emergent capabilities is the nearest concept, since its dispositional-drift cluster holds the EM entries and the authors frame their result as EM. But the measure here is not the one that cluster's EM entries use, and the concept is under a pending attribution correction. Persona selection is adjacent. The paper mentions persona steering only in related work, and its vector is a model diff, not a persona contrast. This is the adjacent situation, with the relations carried in Cross-references.
cross-references
- Emergent capabilities — adjacent. The authors' EM framing places it in the dispositional-drift cluster, but harmful-request compliance is not that cluster's measure.
- Persona selection — adjacent. No persona is measured or constructed, and the authors do not frame the vector as a persona.
- Convergent misalignment — the filed activation-only EM induction, on Betley's questions. Its vector also induces more misalignment than its source model shows.
- Nadaf and Drake and Eberstadt — direction injection into unmodified models on the open-ended EM measure. In both, the direction is a persona contrast; here it is a raw model diff.
- Rao et al. — found StrongREJECT gave no strong EM signal in its Qwen2.5-14B fine-tunes. HEx-PHI is not tested there. This paper gets half its signal from StrongREJECT.
- Turner et al. — documents insecure-code EM's code leakage and coherence loss, the baseline this paper's semantic score improves on for the Qwen models only, on a different measure.
- Persona vectors — the steering method the authors cite as having hinted at EM-related risk.
- The pioneering study the authors extend, Dunefsky and Cohan's one-shot optimized steering vectors on Qwen-2.5-Coder-14B (https://arxiv.org/abs/2502.18862), is not filed.
sources
- Cao, Q., Lou, J., Liu, M., Feng, W., Li, D., Ng, S.-K., & Luu, A. T. (2026). Activation Steering Induces Emergent Misalignment: A More Comprehensive Evaluation. arXiv:2606.08682 (v1, 7 Jun 2026).