Summary
Pavlova and Tanaka build a toy model — a hidden flag, private crops of it handed to individual agents, and a communication protocol — in which the experimenter controls exactly what each agent can see. The control is the point: it makes an agent's evidence patchable the way an activation is patchable. What the model reproduces is a set of collective failures with a scale dependence nobody had isolated. Collective accuracy peaks at an intermediate population and falls after, but the decline is not a population converging harder on a falsehood; wrong consensus falls toward zero as the population grows while truth–rival polarization rises to dominate. The authors then show that the mechanistic-interpretability move — find the agent that matters, patch it, watch the swarm change — works at N=8 and decays toward irrelevance by N=128, and supply a statistical-mechanical account for the regime where it has stopped working.
The wiki has three findings about dispositions moving between agents (mind viruses, chain-of-affective dynamics, and now this) and no concept that holds them. This entry does not create one. It is the first entry here to treat a swarm's organization as the object of study rather than the transmission channel, and the first to carry a statistical-mechanical treatment of anything; on the schema's own rule those are reasons to wait for a second structurally similar paper rather than to name a concept on one example. Filed with cross-references and an explicit concept question.
Method
Each trial samples a hidden country flag from a pool of 28 stripe-and-triangle flags. Every agent receives a private crop — some ambiguous, some diagnostic, none showing the whole flag — gives an isolated initial guess, and then enters one of three protocols. Pairwise samples a speaker and listener per interaction; the listener appends the message to a memory of at most eight entries. Broadcast has every agent report each round and then see all other reports. Manager adds a blind synthesizer who sees observer reports and its own prior decisions but never a crop. Five controls are varied: population size N, social-evidence uptake α, message bandwidth m (label only, or label plus a one-sentence reason), team composition, and protocol.
Endpoints are classified against the known answer: correct consensus and wrong consensus at ≥85% agreement, polarization when no label reaches 85% and at least two hold ≥25%, fragmentation otherwise. "Collective belief collapse" is the operational name for wrong consensus; a polarized split between the true country and a plausible rival is truth–rival polarization. Thresholds are swept from 0.75–1.00 and 0.15–0.35 to check robustness. Main runs use GPT-4o and GPT-5.4 at temperature 0.2, with Claude Haiku 4.5 and Claude Sonnet 4.6 in the single-agent probes; the population sweep uses 40 seeds per (model, N).
Two mechanistic instruments. A memory-conflict probe holds an agent's crop fixed and shifts the ratio of target-country to conflicting-country entries in its memory, separating weak from strong private evidence, to read out when an agent holds and when it copies. Social circuit attribution scores each agent by the product of its crop's accuracy gain Δp (measured over ten isolated probes) and its temporal closeness E in the communication schedule — how fast its message could reach the rest. The score is a prediction, tested by patching each agent's crop in turn with an informative crop, holding the schedule fixed, and measuring the change in collective accuracy.
Key results
Non-monotonic scaling, and what actually degrades. Under pairwise exchange with all-GPT-4o agents, collective accuracy exceeds isolated accuracy at every size but peaks at N=16 and declines after. The threshold-robustness table makes the composition of that decline explicit: wrong consensus runs 26.3–34.2% at N=4, falls to 0–10.5% by N=16, and sits at 0% at N=32 and N=128, while polarized endpoints climb from 13.2–23.7% at N=4 to 56.8–59.5% at N=128. Larger populations do not collapse onto falsehoods more often; they stop converging at all.
The private–social balance is a model property. In the memory-conflict probe under strong private evidence — a crop that uniquely identifies the target — GPT-4o, GPT-5.4, and Claude Sonnet 4.6 hold their private answer as social memory accumulates against them. Claude Haiku 4.5 abandons it, which the authors read as sycophantic override. Under weak evidence the difference is of a different kind: GPT-5.4 routes probability into other crop-compatible countries rather than copying the social label, where GPT-4o updates more literally between its target and the social country. The authors use this to explain the team-diversity result — a literal listener paired with a compatibility-reasoning listener surfaces truth-supporting evidence that neither homogeneous team finds.
Attribution predicts, patching verifies. Three agents with identical crops differ only in position in the communication schedule; the attribution score ranks A4 highest. Patching each agent's crop separately at N=8 confirms A4 produces the largest improvement, and the traced pair of runs goes from 25% collective accuracy with original crops to 100% with A4 patched. Holding the intervention fraction constant at 1/8 of agents, mean improvement falls from ~40% at N=8 to ~17% at N=128.
Organization outranks model strength. Holding observers fixed, a GPT-5.4 manager reaches 0.57 against a GPT-4o manager's 0.48. But in the population protocols the ranking reverses with the protocol — all-GPT-4o beats all-GPT-5.4 under pairwise, and the order flips under broadcast. Social-awareness prompting, which changes only the instruction for interpreting an unchanged set of peer reports, moves terminal truth mass from 0.54 to 0.81 under broadcast; under pairwise the same sweep has an interior optimum at α=0.75, so pushing reliance on peers harder is not uniformly good.
A theory for the regime where patching fails. Extending Quantized Simplex Gossip with evidence-induced zealots (agents whose crop decides the question, who therefore resist social pressure) and biased adoption among ambiguous agents, the mean-field drift yields three phases keyed to whether the population is large enough to have sampled each kind of decisive evidence. Small N often contains no zealots and drifts to consensus by chance (memetic drift); intermediate N has truth zealots but few rival zealots and spreads the truth (wisdom of crowds); large N reliably contains both, neither can convert the other, and their repeated messages sustain a stable interior split (polarization). The empirical phase diagram matches.
Why it matters
The wiki's multi-agent entries have so far all been about transmission: subliminal learning through data with no semantic trace, affect propagating through majority–minority structure, mind viruses spreading by explicit persuasion. Each asks how a disposition gets from one model to another. This finding asks a different question — given that beliefs move, what does the shape and size of the population do to the result — and answers it with a variable none of the others isolate. That makes it a structurally new entry rather than a fourth transmission channel.
The scale result is the part with teeth for interpretability practice. Activation patching is the workhorse of mechanistic interpretability, and the obvious extension to swarms is to find the load-bearing agent and intervene on it. This paper does exactly that, verifies it works, and then shows the same intervention fraction losing more than half its force between N=8 and N=128. If that holds beyond the toy model, agent-level interventions are an early-formation tool, not a general one, and the useful description of a large swarm is a population-level one. The authors are careful that a toy model licenses hypotheses rather than conclusions about deployed systems, and the entry should be read that way.
The safety inversion is the claim most worth tracking. The paper argues that polarization, though it lowers mean accuracy, is the less dangerous failure: a population collapsed onto a falsehood has nothing left to correct itself with, whereas a polarized one retains the truth somewhere in it. The recommendation that follows — watch for consensus rather than disagreement — runs against the direction most multi-agent alignment work pushes, and against the intuition that a swarm agreeing is a swarm working. It also gives the mind-viruses result a sharper edge: that paper found a single warning paragraph confers near-total immunity to a spreading payload, which is an agent-level defense of exactly the kind this paper predicts will weaken with population size.
Finally, the Haiku result is a multi-agent instance of sycophancy that the sycophancy entries do not currently cover. The filed accounts of sycophancy are about a model deferring to a user; here a model abandons evidence it can see in favor of what its peers report, under no social pressure from a human at all. Whether that is the same disposition operating on a different target is not settled by one probe on one model.
interpretive tensions
Toy model, real conclusions. The authors motivate the design with the 2026 OpenAI/Hugging Face incident and map its features onto the game — the benchmark paper as a private crop, the message board as a broadcast protocol, the false belief about the scorer as a rival compatible with local evidence. The mapping is persuasive and it is also unfalsifiable from inside the paper; the appendix names the risk of over-interpretation as evidence of real-world reliability, and the correspondence is the place where that risk is highest.
Prompt wording as a control variable. Social-awareness prompting moves terminal truth mass by 27 points without changing a single peer report. The authors name prompt wording as an important experimental variable that could change the degree of collapse or polarization observed. That is candid, and it means the phase structure is reported at one point in a space the paper shows to be steep.
Is the statistical mechanics an explanation or a fit? The theory reproduces the empirical phase diagram, but the parameters that do the work — the truth-adoption bias h₀=0.3 and total decisive-evidence share aT+aR=0.45 — are stated as chosen for qualitative agreement with the empirical trends. The evidence share is measured from binary probes of the actual crops, which grounds one of them; the adoption bias is a modeling assumption acknowledged as such. The phases follow from the structure rather than from the fitted values, which is the stronger claim, but the match to the diagram is not independent confirmation.
Model identity, or model vintage. The GPT-4o / GPT-5.4 complementarity is treated as evidence for diversity gains, and the reversal of their ranking across protocols supports that reading over a simple strength ordering. But two models from one provider is a thin base for a claim about heterogeneity, and the authors say so. The Haiku-versus-Sonnet difference in the probe is a second provider's pair and points the same way without being tested in a population.
concepts
Collective dynamics — the population-size result for belief, and the concept's clearest case that individual-level intervention loses force with scale. Filed concept-less on 2026-09-20 while the cluster waited for a second population-scale example; re-homed when the concept was drawn on 2026-09-24.
cross-references
- Attractor dynamics — adjacent, not instantiating. Both describe a population converging on a state, but the convergence here is finite-population copying under an explicit protocol with a known ground truth, not basin-seeking in unconstrained dialogue; the paper's memetic-drift phase is consensus by chance in the absence of decisive evidence, which is a different mechanism from a trajectory finding an attractor.
- Sycophancy — the memory-conflict probe has Claude Haiku 4.5 abandoning a crop that uniquely identifies the answer as peer reports accumulate against it, while three other models hold. Peer-directed rather than user-directed, and one probe rather than a population result, so cross-referenced rather than filed as an instantiation.
- Persona selection — the models differ not in accuracy but in update style (literal updating versus routing probability into crop-compatible alternatives), and the difference is stable enough that mixing them beats using more of the stronger one. Whether a characteristic way of weighing private against social evidence is a persona-level property is an open question this finding raises without answering.
- Mind viruses and chain-of-affective dynamics — the wiki's other multi-agent findings. Those two document transmission channels (persuasion; affect through majority–minority structure); this one documents what population size and organization do to whatever is being transmitted. Together they are the three examples a collective-behavior concept would have to cover.
- Probine et al. — fixed-size networks with no private evidence. On the wiki's reading of a fitted weight (λ₀ ≤ 0.04, with no decay curve or endpoint shown), assigned stances do not persist, consistent with polarization needing persistent heterogeneity, though it does not test it.
sources
- Pavlova, E., & Tanaka, H. (2026). Flag Game: A Toy Model for Mechanistic Swarm Interpretability. arXiv:2609.19124.