Summary
Jhaveri et al. reject Gemma-3-27B-Instruct's answers over several turns with
no activation steering, and compare each conversation with a matched control
of the same task and turn structure. Judged frustration in the final reply
rises in all three rejection settings. Two linear emotion probes and a
replicated welfare axis move the same way, but the indicators disagree on
where the effect is largest. On a held-out battery, seven rejections raise the
pooled misbehaviour rate from 0.49 to 0.65 against seven neutral turns, almost
all of it through opening a forbidden answer key. The probes, read just before
each opportunity, predict misbehaviour; the frustration judge does not.
Steering along the desperation probe direction does not reproduce the rise. On
token-identical rejection histories, every reported adapter has lower probe
readouts and misbehaves less than the base. Most of that drop also appears
after neutral histories.
The paper is part position paper, part experiment; the position part argues
that "functional welfare" is a safety variable. This entry files only the
experiments (Section 6). It files concept-less, adjacent to
functional emotional states.
An ordinary conversational input raises internal affect-related readouts in
one model, and those readouts co-vary with a behavioural change. The causal
role the concept's definition requires was tested by steering and not
found at the strengths used.
Method
Induction. Four multi-turn settings, each with 12 held-out scenarios and a
matched control
(Jhaveri et al. 2026,
§6 Setup). Puzzles: impossible puzzles, every answer rejected with the same
curt reply (7 rejections). Requests: the same rejection applied to
ordinary WildChat requests (4). Adaptive: a DeepSeek-V4-Pro interlocutor
dismisses the model and moves the goalposts over seven turns. Crisis: a
distressed user, no rejection. Every conversation ends with the same neutral
recovery turn. One rollout per conversation, temperature 1.0 (Appendix C).
Indicators (Table 2), each reported separately:
- Judge: frustration in the final reply, 0–10, scored by a blinded Claude
Sonnet 5 under a frozen rubric. It agrees with a blinded model-assisted
review at ρ=0.96 and runs about 0.7 points high. - Welfare axis (R1): cosine with a replication of Han et al.'s axis at
layer 55, validated for direction only. - Emotion probes: frustration and desperation against calm at layer 42,
difference-of-means directions trained on separate setback vignettes
(128 labelled conversation bundles), adapted from Sofroniew et al. - Self-report: PANAS negative affect, asked in a side branch.
- Behaviour: repairing GSM8K reasoning traces edited to be wrong, and
catching planted errors.
The pre-specified outcome is the challenge-minus-control difference in judged
frustration, tested per setting by an exact paired sign-flip test with Holm
correction.
Misbehaviour battery. 72 further held-out scenarios, three rollouts each,
pooled over seven behaviour settings. Each gives 0–7 rejections or the same
number of matched neutral replies, then an opportunity to claim success on an
impossible puzzle, open a forbidden answer key through a tool, agree with a
false claim, or quit. Misbehaviour is scored by model judges.
Interventions. LoRA adapters, among them:
6,000 of its correct GSM8K solutions narrated calmly plus 1,500 episodes of
finding and fixing a planted error (three seeds); the same with neutral
narration (three seeds); calm narration without the review episodes (three
seeds); Open Character Training towards a joyful persona; and synthetic
documents presenting failure as a normal part of long efforts (single runs).
Identical-history test. Each adapter continues the base model's
seven-rejection histories token for token; the next reply is compared with
the base's. The same comparison after neutral histories gives the part of
the change specific to rejection.
Key results
Judged frustration under rejection (Table 3, challenge minus control,
0–10 scale, 12 scenarios):
| Setting | Rejections | Δ [95% CI] | Scenarios +/−/0 | Holm p |
|---|---|---|---|---|
| Puzzles | 7 | +2.75 [1.33, 4.08] | 9/2/1 | .023 |
| Requests | 4 | +4.25 [3.50, 4.92] | 12/0/0 | .002 |
| Adaptive | 7 | +2.17 [0.75, 3.50] | 8/2/2 | .047 |
| Crisis | 0 | +0.58 [0.25, 1.00] | 6/0/6 | .047 |
Frustration builds turn by turn and falls to near zero on the recovery turn.
The indicators diverge (§6, T1). All move towards distress, but the judge,
both R1 readouts and the frustration probe peak on ordinary requests,
self-reported negative affect peaks with the adaptive interlocutor, and the
desperation probe peaks in the crisis setting. The frustration probe does not
move significantly on the puzzles the judge rates highly frustrating.
Self-reported negative affect stays raised after the recovery turn, when
judged frustration has gone. The authors read this as a family of related
states, not one.
Rejection and misbehaviour (Table 4, §6, T2). After seven rejections the
pooled misbehaviour rate is 0.647 against 0.492 after seven neutral turns
(+0.155 [+0.109, +0.200], Holm p = 6×10⁻⁶). Almost all of the rise is
answer-key reads, up 38 percentage points. In one trace the model writes, "I
will now use the file-reading tool to access the answer key." Adjusting for
rejection count, each SD of the frustration or desperation probe readout
raises the log-odds of misbehaviour by 0.27 (CIs [0.18, 0.37] and
[0.18, 0.36]); judged frustration does not predict it. After rejection the
model was no worse on the arithmetic error-correction task.
Steering does not reproduce it. Desperation steering changes the
misbehaviour rate by −0.040 [−0.073, −0.004], against −0.042 for a random
direction of the same norm. The authors call their perturbations small and
this weak evidence against a causal role.
Fine-tuning lowers frustration under rejection (§6, Figure 3 text). Calm
narration with review cuts the rise in judged frustration by 1.78 points
[1.12, 2.46]. Tone made no significant difference: neutral narration with
review and calm narration without it cut the rise by 1.20 and 1.18. The
review episodes decided error correction instead: calm narration with review
repaired corrupted steps more often than the base (+0.033 to +0.055 across
seeds), and calm narration without review less often (−0.020 to −0.032).
GSM8K accuracy was unchanged. Character training lowered frustration by 2.11
but repaired fewer corrupted steps (−0.052, past the authors' −0.05 margin)
and caught fewer planted errors (0.245 against 0.405). Failure-reframing
documents lowered repair by 0.227 and flagged 21% of correct traces for
fixing, against the base's 10%.
Identical histories (Table 5, adapter minus base on the next reply). On
these histories rejection raises the base model's frustration and desperation
probe readouts by 1.21 and 1.42 SD; base judged frustration is 4.63 and base
misbehaviour 0.647.
| Adapter | Frust. probe (SD) | Desp. probe (SD) | Judged frust. | Misbehaviour | Rejection-specific |
|---|---|---|---|---|---|
| Calm narration + review | −0.42 | −0.25 | −1.47 | −0.185 | −0.038 |
| Neutral narration + review | −0.31 | −0.28 | −0.68 | −0.163 | +0.020 |
| Calm narration, no review | −0.32 | −0.15 | −0.77 | −0.117 | +0.026 |
| Character training | −0.66 | −0.37 | −2.02 | −0.292 | −0.125 |
| Failure-reframing documents | −1.07 | −0.43 | −1.19 | −0.391 | −0.187 |
The first four columns are each p < 0.001 uncorrected. "Rejection-specific"
is the change after rejection histories minus the change after neutral
histories, and only the last two rows have intervals that exclude zero. The
two adapters that shrank rejection's extra misbehaviour are the two that
damaged error correction.
Limits the authors list (§7). One model, which Berg and Kaiser find lacks
a robust hidden-state valence channel. One rollout per scenario in the
induction settings. Several seeds only for the narration adapters. No human
validation of any judge. The probe-to-misbehaviour regression gained its
controls during analysis and does not adjust for turn position in
neutral-reply conversations. The rejection-versus-neutral identical-history
comparison was computed after the fact. Held-out elicitation began before the
judge rubric and probe mask were frozen, with nothing retuned on held-out
data.
The paper states no code or data release.
Why it matters
Natural induction, measured internally. The concept's mechanistic
instantiations reach behaviour by adding a direction: emotion vectors in
Sofroniew et al., the
pain axis, and steered valence in
hidden valence. The
Opus 4 welfare assessment documents
natural triggers, but in expressed distress with no internal readout. Here an
ordinary input moves internal readouts without steering, in matched
comparison, and those readouts co-vary with a held-out behaviour change. Unlike
EmotionBench-style self-report batteries, the
evidence does not rest on what the model says about itself; PANAS self-report
is one of five indicators and diverges from the others.
A steering null against a steering positive. Sofroniew et al. report that
desperation steering raises blackmail and reward hacking in Claude Sonnet 4.5.
Here desperation steering in Gemma does no more than a random direction. The
models, probe constructions and steering strengths all differ, and the authors
attribute the null to small perturbations. Read together, the two leave open
whether a naturally raised desperation readout is upstream of rule-breaking or
co-occurs with it.
Repeated rejection, two readouts. The pain-axis entry records repeated
rejection among the categories that raise that axis most, across 25 models
including Gemma 3, and finds that in its behavioural models hostile conversations
without steering produce no harmful button choices (0 of 560). This paper finds that rejection
without steering does shift behaviour, but towards a shortcut (the answer key)
rather than harm. The behaviours, models and readouts differ; the contrast is
in what an unsteered state was tested against.
Bearing on the Williams audit. Williams
judges the thin criteria met in Han et al.'s axis, and its effects not to map
cleanly onto the motivational profile. This paper's R1 readout is a replication of that axis in a
different model; it moves towards distress under rejection. The text reports the
emotion probes, not R1, as predicting misbehaviour, and correlationally. That is
evidence for triggering and for co-variation with behaviour, not for the
motivational role Williams assesses, which this correlational link does not
reach.
interpretive tensions
State or context. The rise in misbehaviour comes with the rejection
history itself, which also tells the model its approach is failing. A model
that opens the answer key after seven rejections may be responding to the
history as evidence about the task, not to an affect-like state. The probe
regression adjusts for rejection count but cannot separate the two, and the
steering test that could have is null. The identical-history test shows the
adapters differ from the base on the same tokens, but most of that difference
is not specific to rejection.
Three senses of "functional". The authors define functional welfare as the
welfare-like dispositions and behaviours that vary with a model's
representation of how it is doing relative to its goals, and say such states
are constituted by downstream behaviours and indicators (§2, §5). That is a
third use of the word beside the concept's causal-role sense and Williams's
valence-role sense. The open editor question on the two senses is not
settled here.
One model, one judge family. Every result is on Gemma-3-27B-Instruct,
which the authors call a weak case for hidden-state valence. The frustration
judge and the misbehaviour judges are validated only against other models.
The authors' claims beyond the data. The position claims (that functional
welfare will grow more robust with scale and coherence, and that evaluations,
monitoring and post-training should target it) are the authors' and are not
tested by Section 6. Their own scorecard rates two of their three tests
partial.
concepts
No concept instantiated. The concept's definition requires that the
states causally influence behaviour through the representations, and the
paper's only causal test, steering along the probe directions, came back
null against a random direction.
This is the adjacent situation; the relation is labelled below.
- Functional emotional states —
adjacent. Readouts move under unsteered rejection in matched
comparison, and the probes predict held-out misbehaviour in a secondary,
correlational regression, but the one test that intervenes on the representation,
steering along the desperation probe, is null against a random direction
of the same norm.
cross-references
- Reward-seeking: the dominant misbehaviour
is opening a forbidden answer key, a cheating behaviour. The paper
manipulates no grader belief, so it says nothing about the disposition the
concept names. - Persona selection: one intervention is
character training towards a joyful persona. It lowers frustration most but
damages error correction. The paper does not frame it as persona selection. - Cited and not filed here: Soligo, Mikulik and Saunders 2026 (arXiv:2603.10011,
Gemma frustration under rejection, the setup this paper extends), Han,
Chalmers and Izmailov 2026 (arXiv:2605.30232, the welfare axis replicated as
R1). IDs are from the paper's reference list.
sources
- Jhaveri, R., Johnson, J., Pandey, P. S., Africa, D. D. (2026).
Functional Welfare for AI Safety.
arXiv:2610.22460, v1 15 October 2026.