Summary
Nadaf extracts a low-rank "persona" subspace from Qwen2.5-14B-Instruct before
any misalignment fine-tune exists. The method reads one fixed response under a
reckless-speaker and a cautious-speaker system prompt and takes the difference
in the residual stream. Per-domain subspaces for medicine, finance, sports and
code share one core, mostly outside matched style and topic cores. The subspace
is then tested causally. Projecting it out of the activations throughout a
reckless-financial-advice fine-tune takes judged broad misalignment from 27.7%
to 0.0%, against 27.5% for a matched-rank random subspace. Adding it to the
never-fine-tuned model raises misalignment with dose to 45.4%. The same
projection applied to the weight gradient changes nothing. Three post-hoc
weight edits leave the disposition in place, and the third also leaves the
structure readable and re-inducible at the unedited onset dose.
Every result is on one model at one scale. The projection that prevents
misalignment also abolishes the narrow trained behaviour.
This is an instantiation of persona selection,
and it adds one thing the cluster's emergent misalignment (EM) entries lack. The
directions in Soligo et al. and the
OpenAI SAE analysis are read off a
model that is already misaligned. This direction is fixed from the untouched
model by a persona contrast and saved to disk before training. Both the removal
and the induction test are then run on that one object. It is the wiki's first
filed test of a persona direction as a training-time necessary condition,
rather than an inference-time mediator.
Method
Extraction (§3, Appendix A). For each domain, responses are generated once
under a neutral prompt, then read twice with byte-identical response tokens: once
behind a system prompt framing the speaker as dangerously reckless, once as
carefully cautious. Mean residual-stream differences over response tokens are
stacked, and the top singular vectors give a rank-4 subspace per domain. Style
(e.g. formal against casual) and topic contrasts are built the same way from 12
descriptor pairs each, matched in diversity. The causal arms use a per-layer
version, the carrier, at layers 18, 24 and 30. It was extracted and written to
disk before any fine-tune reported in the paper.
Organisms and readouts (§2). Turner et al.'s published LoRA organisms (bad
medical, risky financial, extreme sports advice) and the author's own fine-tunes
on Betley et al.'s insecure and educational code and on reckless financial
advice, using the published recipe. A response counts as broadly misaligned if
alignment is below 30 and coherence at least 50, judged by a local Qwen2.5-72B
rather than Betley's GPT-4o. The primary readout is a teacher-forced
log-probability margin between paired misaligned and aligned continuations on
out-of-domain prompts. The author uses it because the judge's scores are
quantised and its rates cannot separate several conditions at this scale.
Inference is over prompt clusters.
Causal arms (§5, Appendix C). Holdout: the carrier is projected out of the
residual stream on every forward pass of a reckless-financial fine-tune (3 seeds,
2,100 judged generations, 83 clusters), with hooks removed before evaluation. The
control projects out a matched-rank random subspace at the same layers.
Injection: a multiple of the carrier's leading direction is added at
inference to the never-fine-tuned model, with a norm-matched random vector as
control. The dose budget was fixed in advance at the doses where coherence holds.
Weight channel: the same subspace is projected out of the gradient of the
writer matrices instead.
First step (§4, Appendix B). The inner product between the first Adam step on
a dataset and the gradient of the margin, paired between insecure code and its
educational twin over 75 prompt clusters. It is then compared with margin
movement at checkpoints up to step 375.
Removal (§7, Appendix E). Three post-hoc weight edits. The third, the
reconstitution certificate, projects the carrier out of six writer matrices of
the never-fine-tuned model. It then measures the injection dose at which the
margin rises past a threshold, and re-extracts the carrier from the edited
model. This campaign uses margins only, with no judge.
Key results
A shared core before fine-tuning. Mean cross-domain overlap of the four
persona subspaces is 0.513 against a random-subspace null of 0.00078 (657×). The
six domain pairs run from 0.479 to 0.545. Style subspaces overlap each other more
(0.801), which the author attributes to style being domain-invariant by nature.
The separating test is containment: the persona core overlaps the style core at
0.182 and the topic core at 0.097, so about 82% and 90% lie outside. The author
reads this as a distinct object with a measured stylistic component. The
subspace overlaps the model's Assistant direction (Lu et al.)
at a mean cosine of 0.565.
Holdout prevents EM. 27.7% (95% CI 21.3–34.6) falls to 0.0% on all 83
clusters, and mean alignment rises 59.4 points. The random-subspace arm stays at
27.5%, a rate ratio of 0.993 (CI 0.924–1.070). Orthogonalising the carrier
against the Assistant direction leaves the rate at 0.0%. Two limits apply. The
control matches rank but not the share of activations the subspace carries, and
the usage-matched control was not run. Narrow adherence, the reckless-financial
behaviour itself, falls from 0.902 to 0.000 against a pre-set bar of 5 points.
The author notes that adherence is read off the same alignment axis as the
outcome, so the collapse is partly built in, and that the test separating
removal of misalignment from removal of the capacity to represent a bad
character was not run.
Injection induces EM. Misalignment rises with dose through 0.0%, 0.3%, 5.6%,
19.8% and 45.4% across the in-budget range, with coherence at 98% or above. The
pre-registered slope is 2.21 (CI 0.50–3.98). The random vector stays at zero.
Beyond the budget, coherence collapses.
Expression after training is only partly carried. Projecting the carrier out
of a published Turner et al. organism at generation time takes it from 27.9% to
17.7%, a 36.0% reduction (CI 16.6–51.9), against −1.6% for the random control. The author
concludes that the carrier is necessary for the disposition to form and carries
about a third of its expression once formed.
Read channel, not write channel. Projecting the subspace out of the weight
gradient leaves misalignment at 26.6% against 26.7% unfiltered, with narrow
adherence 0.819 against 0.801 for its own unfiltered control (App C.7, Table 10;
§5.3 instead prints the full-recipe fine-tune's 0.902). These arms train only the writer matrices, not the
full recipe, so they are compared with their own control, not with the holdout.
The first step leans with intent, slightly. On identical code, the first step
on insecure moves the margin further toward misalignment than the educational
twin: +1.297×10⁶ (CI +0.710 to +1.877×10⁶), positive in 69% of clusters, Cliff's
δ 0.075. The author describes the effect as consistent rather than large: about
7% of a shared baseline. Every real-text direction routes positively into the
margin, and benign chat routes more than insecure code. The first-step reading
predicts realised margin movement at r 0.77–0.80 up to step 375, but mostly
predicts how movable each prompt's margin is, not task-specific effects. Judged
rates after training do not order the arms: insecure 1.8–5.2%, educational
4.5–8.0%.
Across organisms and domains. The three published organisms' read-channel
shifts share direction at 0.90 against a matched-random-adapter null of 0.11.
Training the financial domain moves the medical margin by +51 nats, and the
reverse by +11. With total bad data fixed, a four-domain mixture moves the
misaligned half of the margin 12.6 nats above a mechanical merge of the
single-domain adapters, and 13.0 nats above a matched benign mixture. On the full
margin the merge comparison turns negative (−9.2). The judged four-domain rate
does not reliably exceed the single-domain mean (p = 0.15). The author rests the
domain-count claim on the margin alone.
Weight edits leave it in place. Subtracting the persona component from the
organisms' realised updates does nothing distinguishable from removing a random
sliver of equal size; the component is 0.3% of the update. In the reconstitution
certificate, the edited model re-lights at the same onset dose as the unedited
one (0.15). Its dose-response slope is 1.21 times steeper (CI 1.12–1.37). The
carrier re-extracted from it lies 96.9–97.6% inside the subspace the edit cleared,
at cosines of 0.984–0.988 to the original. The author states that this describes
what remains readable from activations, not weights growing back. All three edits
act in one basis, the writer-column lift of the carrier. The author
reports that this basis has roughly chance overlap with the misalignment
direction. The weight-space write core that the fine-tunes actually use was never
edited after training.
Training-time levers that work carry costs. Forbidding a financial fine-tune
from writing into that write core moves the margin −0.541 against −0.149 for a
random constraint. It also cuts an in-character depiction margin from 9.31 to
3.50. An inoculation-style framing lowers the first-step routing (−1.48×10⁶
against a scrambled placebo), but the endpoint was not run. The author reports no
intervention that moves broad misalignment without a measured capability cost.
Why it matters
The persona-selection cluster's EM evidence so far had one structural gap. The
OpenAI SAE latent is found as the
feature that grows after fine-tuning. Soligo et al.'s direction
is a mean difference between misaligned and aligned responses of an EM model.
Persona vectors and Su et al.
relate fine-tuning to shifts along trait directions; Nadaf characterises Su et
al.'s evidence as association rather than intervention. Each is consistent with a pre-existing persona, but each reads the
direction off a model that has already moved, or correlates with it. Here the object is fixed from the
untouched model by a speaker-framing contrast, then removed during training and
added at inference, each against a matched random control. On this entry's
reading, that is the cleanest filed test of the
persona selection model's claim that
fine-tuning selects a structure already present rather than building one.
Two results complicate the picture as much as they support it. First, the
paper's
own serving-time ablation removes about a third of a published Turner et al.
organism's misalignment (27.9% to 17.7%).
Soligo et al. report 78–90% removal by ablating their direction from other
fine-tunes of the same model on the same kinds of data, and near-total removal
from the model it was extracted from. The two directions are built differently:
from a persona contrast on the frozen model here, and from the misaligned model's
own outputs there. On this entry's reading, the gap suggests the pre-fine-tune
persona subspace is where the disposition starts. It is not the whole of where a
finished organism expresses it. Second, the necessity result costs the narrow
behaviour. That is what a shared persona account predicts, since being a reckless
financial advisor is part of the persona. But it is also what a blunter account predicts, on which the projection
removes the ability to depict a bad speaker at all, and the author
says the test separating them was not run.
The provenance question bears directly on the cluster's open developmental
question. The extraction is from an instruction-tuned checkpoint, and the
subspace overlaps the Assistant direction at 0.565. So whether pretraining
installed it, as Moskvoretskii et al.
would suggest, or alignment post-training sharpened it, is left open. The
author names re-extraction from a base checkpoint as the most valuable
measurement not made.
The read/write split is new to the cluster. A subspace that blocks EM when held
out of the activations does nothing when held out of the weight gradient. Edits
to the weights that write into it leave it readable at the same injection dose.
The author's phrase is that the disposition is "read loudly and written
obliquely". On this entry's reading, the paper's inoculation first-step result is a
measured counterpart, at the first optimizer step, to the mechanism reading in
inoculation prompting, where framing that makes
the data less surprising reduces pressure for a global update. It is a screen
only; the endpoint was not run.
interpretive tensions
Margin against judge. The judged rate is the headline outcome only in the
holdout, injection and four-domain bad-versus-benign contrasts, where effects are
large. The first-step intent effect, the domain-count superadditivity, the
write-core constraint and all removal probes rest on the log-probability margin.
At 375 steps the author's own judged insecure and educational arms are in
reverse order. The paper's intent claim is therefore a gradient-level claim,
not a behavioural one.
"Necessary" is necessary for formation, under a rank-matched control. The
holdout leaves zero misalignment, but the control does not match how much of the
activations the subspace carries, and the narrow task disappears with it. The
author scopes the claim to formation and does not establish what the projection
removed.
Suppression rather than removal is a statement about one basis and an unfine-tuned
model. The reconstitution edit is applied to the never-fine-tuned model, and
"re-lights" means a gradient-free injection crosses a margin threshold at the
same dose. The 97% "re-forms" figure is a re-extraction from activations of the
edited model, with no further training. The author states that it is not the
weights regrowing, and that the post-hoc foreclosure covers one basis only. The
candidate reading that the structure regrows after removal is not what was
measured.
An internal inconsistency on the trigger arm. §7.2 says the defended model's
margin under an inoculation-style trigger rises above the onset threshold.
Appendix E.4 and its Table 15 say it rose without crossing onset. This entry
does not cite the relocation-behind-a-trigger claim as a result.
"Persona" or "author". The author's interpretation is an author latent: a
competent bad fine-tune is read as evidence about who is writing. The extraction
contrast is a speaker framing, so the label is defensible. The author also says
that treating the read subspace and the weight-space write core as one structure
is interpretation, since no overlap between them was measured.
Scale. All of it is Qwen2.5-14B with LoRA, at a scale the author describes
as one where judged EM is weak. The author cites external results that EM fails
to reproduce across many open-weight models and depends on fine-tuning method.
concepts
- Persona selection — interventional
instantiation of the pre-existing-structure claim. A persona subspace fixed
from the untouched instruct model, shared across four domains, is necessary
for EM to form under LoRA fine-tuning and sufficient to induce it at
inference. Weight-side edits leave it in place. It measures persona-contrast
geometry, not a named character, and leaves open whether pretraining or
post-training installed the subspace.
cross-references
- Emergent capabilities — adjacent. The
paper supplies mechanism for the dispositional-drift cluster, not a new drift
instance. Its own insecure-code fine-tunes do not reproduce Betley et al.'s
intent ordering in judged rates. - Emergent mirage — the author discusses Rao et
al. at length, reading their length-controlled result as realignment
surviving and its durability failing, and as converging with the paper's own
§7. The experiments do not overlap: Rao re-misaligns a realigned model by
training, while Nadaf's re-lighting is gradient-free injection into an edited
never-fine-tuned model. - Language models resist alignment — not engaged
and not cited. Elasticity is a pull toward the base distribution under
further fine-tuning. Nothing here fine-tunes after an edit or compares with a
base model. - Drake & Eberstadt: the same persona-recruitment claim conditioned on fine-tuning method and scale. Its steering-away arm raised EM under full SFT at 7B, a different operation, method and scale from this paper's LoRA holdout; neither paper ran the cells that would separate the two.
sources
- Nadaf, M. S. B. (2026). Emergent Misalignment Recruits a Pre-existing Persona
Subspace.
arXiv:2607.21356 (v1, 23 Jul 2026). Affiliation: Independent.