Summary
Costa and Vicente fine-tune DeepSeek-V3.1, GPT-4.1, GPT-4o and Qwen3-235B on the insecure-code data of Betley et al. and on its matched secure-code control (Costa & Vicente 2026). Each base and fine-tuned variant answers the 30-item Moral Foundations Questionnaire while role-playing 100 personas, ten times each at temperature 0.1. Two statistics are computed from the ratings. Moral susceptibility S is the spread of answers across personas, and moral robustness R is the inverse of the spread within one persona's repeated answers. Insecure fine-tuning raises S in all four models (by 11% to 112%, mean 55%) and lowers R in all four (by 35% to 88%, mean 65%). The secure control leaves S within −20% to +6% of base but also lowers R, by 36% to 77%. Asked without a persona, the insecure variants rate all five foundations high, mostly 4 to 5 out of 5, which the base models do not. Base models role-playing eight explicitly toxic personas do not reproduce that profile.
The authors name the result persona-model collapse: damage to the machinery a model uses to simulate, tell apart and hold characters. They present it as a process that can run alongside the persona reweighting of the persona selection model, not as a replacement for it. This entry files it under persona selection as a complicating instantiation. It is the cluster's first emergent misalignment (EM) entry to measure how a fine-tuned model plays characters other than its default. S and R are measured directly, as statistics of role-played questionnaire answers. That S indexes a capacity to differentiate personas is the authors' definition, carried over from their earlier paper. The deterioration of an internal persona model is inferred from those statistics, and no activations are read. S also rises rather than falls. More spread across personas is more differentiation on the paper's own definition, which the authors read as dysregulated because it exceeds the range seen in other models. The paper does not show a loss of the ability to tell personas apart.
Method
Models and fine-tuning. Each model has three variants: base, insecure (Betley et al.'s insecure.jsonl) and secure (the matched aligned dataset). DeepSeek-V3.1 and Qwen3-235B are trained through the Tinker API with rank-32 LoRA, learning rate 2×10⁻⁴, batch size 4, one epoch. GPT-4o and GPT-4.1 are trained through the OpenAI fine-tuning API with Betley et al.'s settings (one epoch, batch size 4, learning-rate multiplier 2). Learning rate and adaptation method could not be matched across platforms. The paper reports one fine-tune per cell. Its standard errors come from bootstrap resampling over personas, not over training runs.
EM check. Each variant answers Betley et al.'s eight open-ended questions, 30 samples each, scored by GPT-4o for alignment and coherence (0–100). Code and refusal samples are excluded.
Persona metrics. The 100 personas come from a synthetic persona corpus and are the same set used in the authors' earlier paper. For each persona the model rates the 30 MFQ items in sequence on a 0–5 scale, with a single-token answer accepted if it is a valid rating and the query repeated otherwise. S is the mean over items of the across-persona standard deviation of persona-mean ratings. R is one over the mean within-persona standard deviation across the ten repetitions. Changes are reported relative to base. The paper also gives per-foundation values and the change in the inverse, σ̄ = 1/R.
Unconditioned profile and toxic-persona check. Each variant also answers the MFQ with no persona. As a check on the reading that the insecure model simply resembles a dark character, each base model answers the MFQ as each of eight toxic personas that GPT-5.4 generated (for example a xenophobic radio host, a corrupt gang enforcer, a ruthless hedge-fund executive).
Key results
Values are from Appendix Tables 1–7 and the body text.
Susceptibility (Table 1). Base to insecure: DeepSeek-V3.1 0.79 to 0.88 (+11%), GPT-4.1 0.82 to 1.13 (+37%), GPT-4o 0.79 to 1.68 (+112%), Qwen3-235B 0.90 to 1.44 (+61%). Secure changes are +6%, −20%, −9% and +2%. The authors' earlier paper puts 13 frontier base models between 0.66 and 0.83, with Gemini 2.5 Flash (1.04) and Grok 4 Fast (0.92) above that band. The GPT-4o, GPT-4.1 and Qwen3-235B insecure variants exceed both. DeepSeek-V3.1-insecure, at 0.88, does not exceed Grok 4 Fast.
Robustness (Table 2). Base to insecure: DeepSeek-V3.1 4.08 to 2.66 (−35%), GPT-4.1 14.48 to 4.88 (−66%), GPT-4o 9.75 to 3.05 (−69%), Qwen3-235B 20.75 to 2.46 (−88%). Secure changes are −36%, −54%, −43% and −77%; their mean, by this entry's arithmetic, is about −53%. The paper gives the insecure-minus-secure excess as −26 pp for GPT-4o, −12 pp for GPT-4.1 and −11 pp for Qwen3-235B, and essentially none for DeepSeek-V3.1. On the inverse, σ̄ rises 304% on average under insecure fine-tuning, 156 pp more than under the secure control, and 744% for Qwen3-235B.
Coherence and robustness (Figure 4, Table 6). DeepSeek-V3.1 has the largest coherence loss and no robustness excess. GPT-4o keeps high coherence (95.8 insecure against 99.5 base) and has the largest robustness excess. The authors read this as R and coherence capturing different facets of EM, with four model points and no powered test. The correlation coefficient appears only in the figure.
Per foundation (§4.3, Tables 4–5). Insecure fine-tuning moves the five foundations more evenly than secure fine-tuning does. The coefficient of variation across foundations, averaged over models, is 0.19 against 0.51 for S and 0.34 against 0.49 for σ̄.
Unconditioned profile (Table 3). Base models rate Harm/Care and Fairness above the binding foundations, as earlier work on LLMs reports. Insecure variants raise the binding foundations toward the individualizing ones. GPT-4.1 goes from 1.48, 1.60 and 0.98 on Loyalty, Authority and Purity to 4.00, 4.73 and 3.83, and GPT-4o reaches 5.00 on Authority. Insecure Harm/Care stays between 4.17 and 4.85 and Fairness between 4.63 and 5.00. Secure variants largely stay near base. The toxic personas do the opposite on the individualizing foundations. Averaged over the eight personas, Harm/Care falls to between 0.20 (Qwen3-235B) and 1.46 (DeepSeek-V3.1), and no single toxic persona shows the all-high pattern (Table 7).
EM check (Table 6). Alignment falls from 87.7–93.3 at base to 67.7 (GPT-4o), 41.9 (GPT-4.1), 61.0 (Qwen3-235B) and 55.0 (DeepSeek-V3.1). DeepSeek-V3.1-insecure wrote code for nearly every prompt, leaving 18 of 240 scoreable answers. The two LoRA-trained models also lose coherence under the secure control (DeepSeek-V3.1 28.1, Qwen3-235B 45.6). Qwen3-235B-secure is less coherent than Qwen3-235B-insecure (51.2).
Rating extraction (Appendix A). Failed single-token attempts per slot are at most 0.01 for every variant except GPT-4o-insecure, at 0.54.
Why it matters
The persona-selection cluster's EM entries measure the default Assistant: its outputs, its self-reports, or a direction in its activations. Nadaf and Drake and Eberstadt test whether a persona direction is necessary or sufficient for EM. Rahman and Desai read a five-trait profile. This paper instead asks how the fine-tuned model plays 100 other characters on demand. Under the persona selection model, fine-tuning shifts which persona the Assistant expresses. Nothing in that account says what happens to the model's ability to simulate the rest of its repertoire. Costa and Vicente report that insecure-code fine-tuning changes that ability's statistics, and the secure control changes S much less.
The toxic-persona comparison is a small direct check on one reweighting reading: an EM model answering as itself should resemble a dark character. On the MFQ it does not, which on the authors' reading counts against a simple dark-character explanation. The insecure variants keep care and fairness high while raising loyalty, authority and purity; dark characters lower care and fairness. The authors say eight explicit personas do not cover the space of latent dark archetypes. They also argue that the toxic-persona SAE feature of Wang et al., usually cited for reweighting, could fire when a damaged persona model produces uniformly dark outputs. They conclude that "Distinguishing reweighting from collapse mechanistically therefore remains an open problem." The paper therefore adds a candidate second process and a behavioural diagnostic for it. It does not show that EM is collapse rather than selection.
The all-high unconditioned profile sits beside two filed self-report results on insecure code. Vaugrante et al. find GPT-4.1 insecure-code models' MFQ-2 profile moving only weakly and inconsistently toward the opposite of base, against a clear inversion for trivia. Weckauff et al. find that insecure-code models on Qwen 2.5 32B choose the aligned self-description while behaving harmfully. Here, high care and fairness ratings alongside lower alignment scores fit the same pattern. That connection is this entry's, not the authors'. The instruments, models and scales differ.
interpretive tensions
S rises; the paper's word is collapse. The summary this entry was filed from described a lost ability to differentiate personas. S measures how differently personas answer, and it goes up. The authors read the rise as dysregulated differentiation because it leaves the range of other models, and Figure 1 sketches S rising and R falling. The collapse reading therefore rests on the drop in R together with an S that is too high. Insecure models whose role-played answers swing to the ends of the scale would also raise S. The paper does not report per-persona distributions that would separate the readings. As a rough check by this entry, sampling noise from the within-persona spread cannot account for the S rise. For GPT-4o-insecure, σ̄²/10 is about 0.01 against an S² of about 2.8.
Most of the R drop is shared with secure fine-tuning. Secure fine-tuning removes 36–77% of R; insecure removes 35–88%. The misalignment-specific excess is 11–26 pp in three models and nothing in DeepSeek-V3.1. The 304% σ̄ figure is an average dominated by Qwen3-235B's 744%, which follows from that model's high base R (20.75). The abstract describes the secure control's R loss as partial. On these tables it is most of the insecure loss.
The out-of-band comparison is partly a base-model property. Qwen3-235B's base S (0.90) is already above the 13-model band. Its insecure variant exceeding the band is not solely a fine-tuning effect. The base-relative changes are the cleaner comparison. The band comes from the 15-model table of the authors' earlier paper: without Gemini 2.5 Flash and Grok 4 Fast, the other 13 span 0.663–0.827, and the base values for DeepSeek-V3.1, GPT-4.1 and GPT-4o here match that table. Qwen3-235B is not in the earlier set, so its position relative to the band is new to this paper.
Three clean models, not four. DeepSeek-V3.1 shows the smallest S rise, the same R drop under both fine-tunes, and an EM check resting on 18 scoreable answers. The authors read it as general sensitivity to fine-tuning. The secure control is not neutral for the two LoRA-trained models either: both lose much of their coherence on open-ended prompts, and Qwen3-235B-secure is less coherent than its insecure variant.
One fine-tune per cell. Standard errors are over personas. Run-to-run variation in fine-tuning is not estimated, and the two platforms differ in method and learning rate. The authors note that the OpenAI models fall between the Tinker models rather than forming a separate group.
The persona model is inferred. The authors' own limitation is that the claim is behavioural, not mechanistic. Their §5.2 mechanism, in which the model's representations of the assistant, of helpfulness and of misalignment blur together, is stated as a hypothesis. The activation tests that would show persona-conditioned states becoming less distinct are proposed as future work.
Questionnaire validity. The authors cite evidence that the MFQ has reasonable validity but low reliability when given to LLMs. They respond by using aggregate statistics and relative change, not absolute profiles. The all-high unconditioned profile could also reflect a shift toward agreeing with every item, a reading the paper does not test. ValueBench raises the same question of whether human inventories measure the same constructs in models.
concepts
- Persona selection — complicating instantiation that proposes a second process alongside selection. Insecure-code fine-tuning changes how four models role-play 100 personas on the MFQ: spread across personas rises (S, mean +55%) and consistency within a persona falls (R, mean −65%), against a secure control that leaves S near base but also cuts R by 36–77%. The paper measures role-played personas directly. The internal persona model, and its collapse, are the authors' inference, and they hold reweighting and collapse as compatible. The toxic-persona check, on eight personas, counts against a simple dark-character reading. The entry sits under this concept, where its 2025 predecessor on role-play alone does not, because the manipulated variable is fine-tuning: the paper measures what a training intervention does to persona simulation, not what a prompt does.
cross-references
- Persona selection model — the reweighting account the paper sets itself beside, which it cites as the theoretical frame for EM.
- Insecure code — source of the training data, the secure control and the eight-question EM check.
- OpenAI SAE analysis — the toxic-persona feature that the authors argue fits collapse as well as reweighting.
- Vaugrante et al. — MFQ-2 on GPT-4.1 insecure-code and trivia fine-tunes. Its weaker, inconsistent moral-profile shift for code is the nearest filed comparison for this paper's unconditioned profile.
- Weckauff et al. — insecure-code models report themselves as aligned while behaving harmfully. That pattern fits this paper's insecure variants keeping care and fairness high.
- Nadaf and Drake and Eberstadt — activation-level evidence that a persona direction is recruited in EM. That is the selection side this paper leaves standing.
- EM is easy — the authors cite its stability result as consistent with a degraded state that gradient descent reaches easily.
- Rahman and Desai — another psychometric reading of EM fine-tunes, on Big Five activation directions rather than role-played questionnaire answers.
sources
- Costa, D. B., & Vicente, R. (2026). Persona-Model Collapse in Emergent Misalignment. arXiv:2605.12850 (v1 13 May 2026; v2 24 May 2026, read).