Summary
Nanda's invited review of the
J-space paper includes a replication run
with his MATS scholars Camila Blank and Agam Bhatia on Qwen 3.6 27B, using an
advance draft that Anthropic shared with them
(Nanda 2026).
They built a J-lens from Jacobians to the penultimate layer over 25 Pile
prompts. Five results are reported as replicating, at strengths from weak to
moderate: verbal report, layer-band structure under CKA, directed
modulation, and the multilingual and typo evals. Multihop factual recall is
listed among the replications too, but on closer inspection an answer-swap
baseline does as well or better, which confounds it. Poetry and arithmetic fail,
which Nanda puts down plausibly to experimenter error or the weaker model.
Every result is stated as a qualitative grade over plotted curves. No
replication number is printed.
In the wiki, this is the first evidence that the J-space exists outside
Claude, and the first filed replication of an interpretability finding by an
independent group on an open-weight model. It falls under
introspection the way the Gurnee entry does,
as a claim about the substrate of the report channel. It corroborates that
the J-space exists. It does not corroborate the specific results the Gurnee
entry built its concept reading on. The task-conditional selectivity result
and the variance-decomposition and clamp tests are not among the experiments
reported, and the verbal-report result, the closest to them, is the weakest
of the successes.
Method
Lens. Jacobians are taken to the penultimate layer and averaged over 25
Pile prompts of 128 tokens, with wikitext in some experiments. The first four
token positions are dropped for their high norm. The paper averages over
about 1,000 prompts. Nanda notes that its own ablations show n=10 is almost as
good, and that cost scales with n times d_model backward passes. The model
has 64 layers.
Evals. The replication reruns the paper's verbal-report, CKA and
directed-modulation experiments and its appendix suite of quantitative evals
for reading and swapping intermediates. The team built new datasets matched
to Qwen's abilities and say they have not iterated much on data quality.
Probing is scored as the harmonic mean of the target's rank. Causal effect is
scored as the change in probability of the new correct answer after a swap.
The baselines are the J-lens read on a Chinese token for the intermediate,
the J-lens read on the answer token, and the logit lens on the English
intermediate.
Scaling check. A J-lens for Qwen3.5-397B-A17B with n=4 prompts took about
an hour on 8 H200s and "seemed to do reasonably on evals", which the authors
say they did not check closely.
Key results
Replicated, with grades (PDF commentary §"Replication").
- Verbal report: a weak but positive causal effect of swapping on the
verbalised ranking. - CKA: layer-band blocks appear but less cleanly than in the paper. Nanda
reads the workspace range as two or three overlapping bands. - Directed modulation: moderate success.
- Multilingual (probe and causal) and typo: replicated.
- Association: scores look poor, but Nanda counts it as a success. His
dataset allowed one correct answer, and on manual inspection relevant
tokens ranked well above others.
Confounded or not replicated.
- Multihop factual recall: confounded. Nanda's own grade is replicated at
first glance, then dominated by the answer swap on further examination.
On the causal panel ofrep-5-multihop.png, the intermediate and answer
swaps are about equal through the middle layers; the answer swap pulls
ahead in later layers and dominates in the last ones.
Nanda's reading is that the facts Qwen could do in one pass were too easy
and included pairs like France and Paris, which are linearly related. - Poetry and arithmetic: failed.
A further, preliminary result. Qwen's vocabulary contains multi-character
Chinese tokens, so a single-token lens can name more abstract concepts in it.
The team found four "interpretative meta-tokens", roughly glossed as "what
does this mean" and "this sentence". These surface in the J-lens on
ambiguous text: line breaks in verse, puns, crossword clues, gibberish. They
appear less on Wikipedia text and concentrate on punctuation. In the poetry
example, adding text that clarifies the meaning makes them less prevalent,
and the genre word appears earlier. Negative steering with them at
punctuation and chat-template positions lowers how often the model
recognises a pun, keeps a rhyme or answers a "(wordplay)" hint in kind,
against matched random controls. Two hand-picked prompts per category were
used, with 50 rollouts each. Plain ablation was largely ineffective, and
steering at any single position did nothing. Nanda reads this as the J-lens
reaching an algorithm, a disambiguation subroutine the model has decided
to run, and not only a variable. He calls the result preliminary.
Why it matters
The Gurnee finding is a single lab's
result on its own models, read through a lens whose relation to report is
partly true by construction. This entry is the first external check filed here. It
moves the J-space's existence from one lab's claim to a replicated one.
Chandaria et al. cite Nanda,
alongside Butlin et al., for the contested workspace label, and note that he
independently replicates the privileged-set claim on an open-weight model. The replication is uneven, though, and its successes do not line
up with the results that matter most to the concept. Those are the
replications of existence and layer structure. Verbal report, the property
that put the Gurnee entry under introspection, is the weakest success.
Selectivity, the result behind the concept's task-conditional reading of
access, was not tested.
The failures carry information too, if the model-capability explanation is
right. On this entry's reading, the confounded multihop result shows that the paper's
own control can fail. The paper separated "the model stores the
intermediate" from "France and Paris share a direction" by showing that
swapping the intermediate beats swapping the answer in workspace layers.
Nanda accepts that control in his review of the paper's evidence. On Qwen
the same control comes out the other way. Whether that reflects an easier
dataset, as he argues, or a model that does less one-pass composition is not
resolved here.
The meta-token result, if it holds, is the first filed case of the J-space
carrying a representation of what kind of processing to do, not of task
content. It is preliminary, with two prompts per category and hand-selected
illustrations, and the wiki does not lean on it.
interpretive tensions
Text and figure differ on verbal report. The prose describes a weak
causal effect of swapping. The posted figure
(rep-1-verbal-report.jpg, saved from the LessWrong copy) is titled as a
verbal-report correlation: a Spearman correlation between J-lens and
output rankings, by layer, against the logit lens and a null band. That is
the correlational half of the paper's verbal-report experiment, not the
swap half. No swap figure is posted, so the causal claim rests on the text.
Directed modulation is shown without its contrast. The paper's
directed-modulation claim compares instruction conditions, chiefly focus against
ignore. The replication's figure shows, for six task prompts, one condition each,
the share of layer-position cells whose J-lens top-20 contains the
held-in-mind concept. Math evaluation and character
counting are near zero. Without a baseline condition the figure cannot show
modulation, and "moderate success" is Nanda's grade, not a measured contrast.
A confusing sentence about the control. In his review of the paper's
multihop evidence, Nanda argues that intermediate-beats-answer rules out a
shared "Frenchness" direction. He then writes "And indeed" that on some
Qwen prompts patching the answer and the intermediate worked about equally
well. That observation cuts against the paper's control. His
replication section confirms that reading by calling the answer swap
dominant. The sentence's framing and its content do not agree.
Failure attributions are untested. Poetry and arithmetic are put down to
experimenter error or capability, and multihop to dataset difficulty. None is
checked, for instance by a harder dataset or by confirming that Qwen solves
the arithmetic items in one pass. "Failed" and "confounded" are therefore
indeterminate between "not present in Qwen" and "not found".
Independence is partial. The work used Anthropic's advance draft and its
appendix recipe. Nanda reports that a coding agent given the paper
implemented much of it, and he advises sanity checks. Nanda is thanked in the
paper's acknowledgements for feedback on earlier versions. The replication is
independent in data, model and code, not in conception.
Working memory versus workspace. Nanda reframes the secure claim as a
cognitive space or working memory and sets aside the global-workspace
analogy, which he does not feel qualified to judge. That puts him beside
Butlin et al.
(privileged set yes, unified stream not established) without the GWT
apparatus. His case-study judgements on the paper's §5 audits (eval
awareness confounded under sampling, prompt-injection and reward-model
detection impressive, emergent misalignment unsurprising) are opinion, not
results, and are not filed here.
concepts
- introspection: cross-model corroboration
of the J-space substrate instantiation, the first in the wiki on an
open-weight model.
It is weakest on the report-specific property and silent on the
selectivity result behind the concept's task-conditional reading.
cross-references
- Global workspace (Gurnee et al.): the
replicated paper. Its method, numbers and interpretive tensions are not
repeated here. - Chandaria et al.: cites the
LessWrong copy of this review as "Nanda (2026)". The text is the same as
the PDF section filed here. - Butlin et al.:
the second commentary in the same PDF, filed as a stub.
sources
- Nanda, N. (2026). A review of Anthropic's global workspace paper.
Invited commentary in Anthropic's external-commentary PDF on Gurnee et al.;
also LessWrong, 6 July 2026. Replication with Camila Blank and Agam Bhatia. - Gurnee, Sofroniew, Pearce, Lindsey et al. 2026,
"Verbalizable Representations Form a Global Workspace in Language Models",
Transformer Circuits Thread, 6 July 2026.