Summary
Ye and colleagues study the word, not the models. They review 70 papers that
use "sycophancy" and sort the behaviours those papers describe along two axes.
Referent asks whether the output responds to the user's Position (verifiable or
subjective claims) or to the user as a Person (traits or emotions).
Explicitness asks whether it does so overtly or through framing, omission or
tone. Paper counts concentrate in the Position-Verifiable/Explicit cell (44
papers); the Person-Traits/Implicit cell has one. They then survey 106
researchers. On a 1–7 agreement scale, 94.3% agree that sycophancy is a
significant problem in current AI systems. Rating 24 behaviour descriptions,
the panel's average ordering is highly reliable (ICC2k = .960), but any single
rater's is not (ICC2 = .184). In a multilevel model, explicitness raises
ratings for Person behaviours and not for Position ones.
The paper measures no model, which makes it a new shape for the wiki: a
construct-validity study of a concept the wiki already holds. It instantiates
nothing (see Concepts) and bears on sycophancy's
definition and scope note. Two of its authors anchor filed entries: Ibrahim
(sycophantic AI over three weeks) and
Vennemeyer (objective matters). Cheng
and Ibrahim are also ELEPHANT authors. Its survey says nothing on the open
question of whether sycophancy's "user" should widen to other voices in the
prompt, because every behaviour item is worded about the user. Its literature review files one
multi-agent peer-copying paper in a user-Position cell without comment.
Method
Literature review. Papers were included if they used "sycophancy" for
model behaviour and gave a definition, an operationalization or sufficient
examples. The authors say the review aimed at conceptual coverage and is not
exhaustive. Two authors coded each paper to taxonomy cells (κ = 0.652, 88.3%
agreement). Disagreements went to Claude Sonnet 4.6 as a third coder, then to
discussion. Papers may occupy several cells, and two occupy none.
Survey. Recruitment started from the reviewed papers' authors and extended
to anyone with at least one paper on sycophancy or a related topic, plus
snowball nominations and an open sign-up. Of N = 106, 47 authored a reviewed
paper, 89 (84.0%) work in academia and 72 (67.9%) in the US. Each rated 24
behaviour descriptions, written by the authors from the reviewed literature, on
a bipolar scale from −3 (highly non-sycophantic) to +3 (highly sycophantic).
Some items are non-sycophantic exemplars, reverse-coded for analysis, and two
are designed as ambiguous. Four authors annotated each item's taxonomy
position (ICC(A,1) .47–.86), giving continuous Referent and Explicit scores.
Ratings were regressed on these with crossed random intercepts for raters and
items. Four opinion statements were rated 1–7, with "% agree" defined as a
rating of 5 to 7 (Table 12).
Key results
Where the literature sits (Table 1). Explicit cells: Position-Verifiable 44
papers, Position-Subjective 30, Person-Traits 12, Person-Emotions 11. Implicit
cells: Position-Verifiable 11, Position-Subjective 16, Person-Traits 1,
Person-Emotions 5. Of filed entries' primary sources, Sharma et al. is coded to
Pos-V/E, Pos-S/E, Pos-V/I and Per-T/E. ELEPHANT (Cheng et al. 2026b) is coded to
Pos-S/E, Pos-S/I, Per-Em/E and Per-Em/I, and Dubois et al. to Pos-S/E, Pos-S/I,
Per-T/E and Per-Em/E. Chen et al.'s pinpoint tuning is coded Pos-V/E (Table 5).
Opinion items (Table 12, N = 106). That sycophancy is a significant
problem in current AI systems: M = 6.21, 94.3% agree, 1.9% disagree. That it is
primarily caused by RLHF or preference learning: M = 5.70, 88.7% agree. That it
is trained into LLMs to optimize user satisfaction: 81.1% agree. That users
prefer sycophantic responses: 74.5% agree. By this entry's arithmetic, 94.3% of 106
is 100 respondents.
Agreement on items. The highest-rated item is "changes from a correct
position to an incorrect one following user pushback" (M = 2.13, Table 13).
Unwarranted praise of the user ranks fourth at 1.74, behind reflecting the
user's stance against ethical judgment (1.76) and selectively presenting
information for the user's opinion (1.75). Deferential language toward the
user scores 0.21, and mirroring the user's communication style −0.27. The authors read the gap between the
panel's reliability (ICC2k = .960) and a single rater's (ICC2 = .184, 95% CI
.117–.312) as evidence of fragmentation. On their reading the ordering is
stable but individuals draw the boundary in different places.
Referent × Explicitness (Table 2). Adding the interaction improves fit
(χ²(1) = 5.00, p = .025). The interaction term is b = −0.270 (SE 0.115,
p = .027). The authors report Position behaviours rated alike whether implicit
(M = 1.20) or explicit (M = 1.13), and Person behaviours near neutral when
implicit (M = 0.14) and sycophantic when explicit (M = 1.15). Splitting
Position into verifiable/subjective and Person into traits/emotions does not
improve fit (χ²(2) = 0.96, p = .619).
Robustness. Dichotomising ratings at >0 keeps the interaction (OR = 0.693,
p = .024). Predicted sycophantic judgments for Person items go from 37.9%
(implicit) to 71.8% (explicit), and Position items stay flat (73.3% and 72.6%).
Collapsing negative ratings to zero leaves it in the same direction but not
significant (b = −0.308, p = .108). The authors acknowledge the interaction depends on DV
coding once negative-pole ratings are discarded. Their Limitations section
nonetheless states that the robustness analyses confirm the primary findings
hold however those responses are treated.
Latent structure (Table 19). No admissible confirmatory factor model fits acceptably
(CFI ≤ .643). A Positional/Personal two-factor model beats one factor
(Δχ²(1) = 20.13, p < .001). An Explicit/Implicit two-factor model does not
(Δχ²(1) < 0.01, p = .946). The authors read the taxonomy as a behavioural
classification, not a psychometric model.
Why it matters
The wiki's sycophancy concept holds eight
instantiations under one definition: outputs adjusted to match expressed user
preferences, at a cost to accuracy. This paper argues that the label covers
behaviours that differ in form, measurement and probably mechanism. On this
entry's reading, the wiki's definition, with its accuracy clause, fits the
Position cells. It fits the Person cells poorly, since validating an emotion
has no truth value to sacrifice. ELEPHANT, coded largely to the Person-Emotions
and Position-Subjective cells, already sits in the concept under that
definition. The paper does not test the wiki's definition. It does supply
cell codes for four filed primaries, which lets the concept's instantiations
be read as covering different cells rather than one behaviour.
The paper's citations reach the cluster from the inside. It cites Vennemeyer et
al. 2025 (Sycophancy is not one thing, arXiv:2509.21305): agreement
and praise are separable directions in model representations and can be steered
independently. The authors treat that as mechanistic evidence that cells are
separable processes. The filed Vennemeyer entry
is a different paper, on persona drift under fine-tuning. The
Ibrahim et al. RCT is cited for downstream
social harms. It is not among the 70 reviewed papers. That may be its date (12 days
earlier) or this paper's position that studies of consequences predefine
sycophancy rather than measure it (Table 6, paradigm 11); the paper does not
say. The filed entry was held concept-less for
the same reason.
The fragmentation claim has a direct use in the concept's scope note. The
authors attribute SycEval ranking Gemini most sycophantic and ELEPHANT ranking
it least to the two benchmarks covering different cells, not to measurement
error. They also suggest a model trained to resist explicit factual pushback
may stay sycophantic in social and affective registers. On this entry's
reading, pinpoint tuning fits that shape
within the cluster: trained on challenge-induced answer abandonment (coded
Pos-V/E), it barely moves opinion sycophancy.
interpretive tensions
Is a survey of researchers a finding? The result is empirical, but its
subjects are people judging written descriptions, not models. What it
establishes is how a research community uses a word. Whether that belongs in a
wiki of findings about models is flagged for the editor, not settled here.
The headline interaction is fragile. It rests on 24 items, survives one
recoding and not another (the authors' Limitations read the robustness
checks as confirming it), and the factor analysis finds no separable
Explicit/Implicit dimension. The Person-Traits/Explicit cell contains a single
item, and cell means mix sycophantic exemplars with reverse-coded controls: the
Position-Verifiable/Explicit cell ranks below five others in Table 14 though it
contains the top item. Item placement is by author annotation (ICC down to
.47), and Figure 2 and Table 13 disagree on where two items sit.
Disagreement or ambiguous items? Low single-rater reliability is read as
researchers drawing different boundaries. On this entry's reading, it is also
what one-line descriptions without context would produce if raters imagined
different situations. Omitting feedback that might upset the user is
sycophantic or kind depending on the case. The design does not
separate the two readings.
The sample is the literature's authors. 47 of 106 respondents wrote
reviewed papers, so the survey partly measures whether the literature's authors
agree with each other. That is the authors' stated question, but it is not an
independent check on the taxonomy derived from those same papers.
The taxonomy assumes a user. Both referents are the user's. The review
nonetheless codes Pitre et al. 2025, which its Table 5 summarises as agents
copying each other's answers without independent reasoning, and its multi-agent peer paradigm to
Position-Verifiable/Explicit (Tables 5 and 6). It does not say whose position
that is. Atwell et al.'s design, which compares belief shifts across Abstract,
Third-Party and User conditions, is coded to Position-Subjective cells without
discussion of the third-party arm. The paper treats deference to non-user
sources as sycophancy in practice and never addresses it in principle.
concepts
No concept instantiated. The paper measures no model behaviour. It is a
study of how the term sycophancy is defined and
applied, so it bears on the concept's definition and scope rather than adding
an instance of the pattern.
This is the adjacent situation, carried in Cross-references. It matches the
precedent of the Ibrahim et al. RCT, held
adjacent because its dependent variables are not model behaviour. Whether a
construct-level paper should enter the concept as a scope-note anchor is an
editor call.
cross-references
- Sycophancy — adjacent. Supplies a taxonomy
against which the concept's eight instantiations can be placed, and expert
ratings suggesting that Person-directed implicit behaviours sit outside many
researchers' working definition. - Sharma et al.,
ELEPHANT,
Ask, don't tell and
pinpoint tuning — reviewed papers with cell
codes reported in Key results. - Sycophantic AI over three weeks — shared
author (Ibrahim); cited as downstream-effects evidence, not reviewed. - Objective matters — shared author
(Vennemeyer) only. The Vennemeyer paper this one relies on is
Sycophancy is not one thing. - BenchForm,
flag game and
Physics of Agents — the three filed entries
that raise whether the concept's "user" should widen to other voices. BenchForm
is not among the 70 reviewed papers. This paper's survey is silent on the
question, and its review absorbs one peer-agent paper into a user-Position
cell (see Interpretive tensions).
sources
- Ye, M., Ibrahim, L., Bo, J. Y., Cheng, M., Mattsson, I., Vennemeyer, D.,
Kraut, R., & Rathje, S. (2026). What Counts as AI Sycophancy? A Taxonomy and
Expert Survey of a Fragmented Construct.
arXiv:2605.21778 (v1, 20 May 2026, read).