REVIEW 4 major objections 5 minor 23 references
A decoder-only LLM reads and writes through one entangled token code, not the brain's two dissociable systems.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 08:58 UTC pith:IBDQCWVY
load-bearing objection An unusually honest paper with a well-calibrated E measurement; the abstract overstates one negative result. the 4 major comments →
Reading Without a Reader: Large Language Models Collapse Reading and Writing into a Single Entangled Code
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
In untied decoder-only transformers, the input embedding WE (the 'reading code') and the output unembedding WU (the 'writing code') couple early in training and then partially differentiate, ending with an entanglement index E of 0.23–0.35 on a scale anchored by independent initialization (0) and tied weights (1): one coupled but sub-ceiling code, and the output side drifts roughly 3.2 times farther than the input side in every token-frequency decile. The matching behavioral prediction cannot be tested: comprehension as log-probability ranking and production as greedy argmax are both summaries of the same next-token distribution, greedy production implies a vocabulary-wide argmax that wins a
What carries the argument
The central object is the pair of token-coding matrices WE and WU in a decoder-only transformer, read as a 'reading code' and a 'writing code,' measured by a composite entanglement index E in [0,1] that fuses linear CKA, orthogonal-Procrustes residual with per-token cosine, and mutual k-nearest-neighbor overlap, calibrated against an independent-initialization floor and a tied ceiling. The argument's load-bearing pieces are the set-theoretic nesting (greedy production's success set is a subset of pairwise comprehension's success set) and the algebraic identity that both behavioral scores share the target's log-probability term a.
Load-bearing premise
The negative result hinges on defining comprehension as a two-way log-probability ranking and production as a vocabulary-wide greedy argmax; if production were defined differently, the 'produces but does not comprehend' cell could in principle fire, and the theorem would not transfer.
What would settle it
Find a single token pair for which greedy decoding emits the target form while the target's log-probability is lower than a competitor's (the theorem says this is impossible for single-token pairs), or construct a differential-damage index with a non-saturating null that reveals a site where comprehension drops while production is spared under matched severity.
If this is right
- Weight-geometry claims about how separate or shared an LLM's input and output codes are need floor-and-ceiling calibration; raw similarity numbers are otherwise uninterpretable.
- Behavioral double dissociations between comprehension and production cannot be established from output-side scores in a decoder-only model, because any two scalar summaries of one next-token distribution share terms.
- Reported couplings between comprehension and production measures should be treated with suspicion unless the shared log-probability term is removed; the paper shows removing it removes most of the apparent signal.
- If the claim holds, decoder-only LLMs realize functional literacy through a single entangled route, inverting the human brain's two-system architecture and placing them outside the homology space of reading/writing models.
- The couple-then-differentiate trajectory means that conclusions drawn from end-of-training embeddings miss a substantial reorganization: the codes peak early in training and then partly separate.
Where Pith is reading between the lines
- The 'cannot be run' theorem is specific to greedy decoding; with sampling or temperature-based production, the alexia cell could in principle fire, which would make the dissociation claim testable under a different production rule than the one the paper uses.
- A natural testable extension is to train a model against a separated objective, forcing distinct read and write pathways, and ask whether the entanglement index drops and whether any output-side dissociation then becomes measurable; the paper explicitly leaves this open.
- For cognitive neuroscience, the result suggests LLMs should not be used as models of acquired alexia or agraphia: their single-path coupling is a different failure mode, and surface behavioral dissociations would not reveal underlying mechanism.
- The scale-invariance null (E flat across model sizes) is underpowered in the paper itself, but if it held at larger scales it would imply the read/write entanglement is a property of the architecture and training objective, not of model capacity.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper asks whether the input embedding W_E and output unembedding W_U of a decoder-only LLM form one entangled 'read/write' code, using the brain's double dissociation between reading and writing as an analogy. The authors define a composite entanglement index E built from linear CKA, orthogonal-Procrustes residual/per-token cosine, and mutual k-NN overlap, calibrated against an independent-initialization floor, a shuffled-token null, and a tied ceiling. On tied GPT-2/OPT models E=1 by construction; on untied Pythia models (14M–1.4B) E≈0.23–0.35, far above the floor, with a non-monotonic 'couple-then-differentiate' trajectory and an asymmetric drift in which W_U moves ~3.2× farther than W_E in every frequency decile. The paper then reports a negative result: three attempts to build a behavioural comprehension/production dissociation test fail, because greedy production is nested in pairwise log-probability comprehension, differential-damage indices are not scale-identified, and the two scores share a target log-probability term that manufactures coupling. Several statistics are explicitly withdrawn. The conclusion is that decoder-only LLMs realize literacy through a single entangled statistical code rather than two dissociable systems, offered as an analogy rather than a homology.
Significance. If the geometric claim stands, it is a useful, controlled contribution to mechanistic interpretability: it quantifies how far untied W_E and W_U are from both independence and forced identity, and it adds a training-trajectory characterization that goes beyond Lopardo et al. (2026). The paper is unusually transparent: bootstrap CIs, permutation nulls, shuffled-token controls, and an independent-init floor accompany the headline numbers, and the authors openly withdraw a coupling statistic, a cross-level bridge, and an encoder/decoder separation measure. This transparency is a genuine strength. The negative claim about the impossibility of an output-side dissociation test is provocative and could be significant if it were scoped correctly, but as written it is broader than the results support. The paper's framing as analogy is explicit and appropriately hedged.
major comments (4)
- [Abstract; §4.8 Result 1; §5.1] The assertion that the behavioural dissociation test 'cannot be run' and that 'no pair of output-side scores isolates either ability' is stronger than the results establish. Result 1 is a nesting theorem only for greedy production against a 2-way log-probability ranking in the same context. It does not cover stochastic decoding (temperature/top-p sampling), where a token with lower probability than a competitor can still be produced, nor reference-scored multi-token generation, which §5.1 explicitly leaves open. Result 3 is an empirical decomposition, not an impossibility proof: after removing the shared log-probability term the residual coupling is +0.13 and marginal against its null. The abstract and contributions should be scoped to 'the three operationalizations we attempted cannot be made valid,' and the existence of any valid output-side dissociation test should remain open. As wri
- [§4.8 Result 2] The scale-identification argument is internally plausible for the particular linear-standardization comparisons, but the conclusion that the per-site test is 'either valid without power or powered without validity' is stated as a general dilemma. The bounded rank-difference statistic and the saturation of the family-wise permutation null are properties of the chosen max-statistic permutation construction; exact conditional tests or different test statistics could in principle restore validity. This does not undermine the decision to withdraw the three indices, but the text should present the result as a property of this test construction, not as an impossibility theorem about differential-damage indices in general.
- [§4.7, Table B11] The frequency-decile control is claimed to rule out the gradient-exposure confound. However the confound is not solely a frequency effect: the output matrix receives a gradient signal for the target token on every step, while an input row updates only when that token appears as context, and these two update schedules can differ even within a decile. The decomposition shows the ~3.2× drift asymmetry is not carried by frequent tokens, but it does not by itself separate 'output-side gradient schedule' from 'genuine functional differentiation.' The claim that the frequency-decile analysis rules out the confound should be softened, or supplemented with a matched update-count analysis.
- [§6, Limitations] The tied-versus-untied contrast is confounded with model family, as the paper itself acknowledges; the tied condition is GPT-2/OPT and the untied condition is Pythia only. The abstract's phrase 'on GPT-2, OPT and Pythia' may suggest a controlled architectural comparison, but GPT-2 and OPT serve only as a by-construction ceiling. This is not an error, but the abstract and contributions should state more prominently that the substantive 'coupled, sub-ceiling' claim rests on the single Pythia family and that no within-family tied/untied manipulation was performed.
minor comments (5)
- [§4.5] Table reference is broken: 'Table ??' should be 'Table B6' or the appropriate table number.
- [Abstract] The abstract contains 'aculturalinvention' — a missing space, making the phrase hard to read.
- [Figure 6] The figure caption includes 'NB: confounded by gradient exposure,' which appears to contradict the text of §4.7 claiming the frequency-decile analysis rules out that confound. The caption should be reconciled with the main text.
- [§3.2] The claim that 'we verified the trends are not an artifact of the equal-weighting choice' is not supported by any table or appendix entry. A short sensitivity analysis (e.g., E under alternative weights) should be reported or the sentence should be removed.
- [§4.8 Result 1] The notation F and F' and the conditions for the 'empty by theorem' claim are introduced in prose; a compact formal statement would help readers verify the scope of the nesting result.
Circularity Check
No significant circularity: the surviving read/write entanglement claims are anchored against independent-init floors and tied ceilings, and the paper explicitly withdraws its own degenerate behavioral statistics.
full rationale
The paper's central positive claim is the calibrated geometric entanglement index E between WE and WU. This is not circular: E is normalized against an independent-initialization floor and a tied ceiling, raw components (CKA, Procrustes residual, per-token cosine, k-NN) are reported alongside the composite, and the floor/null are parameter-free controls rather than fitted quantities. The asymmetric-drift trajectory is credited to Lopardo et al. (2026) but re-derived with nulls and frequency-decile controls, so the citation is not load-bearing. The behavioral 'cannot be run' negative is explicitly derived from the chosen operationalizations (greedy argmax vs 2-way log-prob ranking) and is labeled a theorem; the paper does not use it as evidence for the weight-level coupling but as a scoped measurement failure, and it explicitly leaves open richer probes. The statistical couplings, contingency table, cross-level bridge, and encoder/decoder separation measure that could have been circular are all self-identified and withdrawn, with the algebraic shared-term decomposition (a-b and a-c both containing a) stated as the reason. No self-citation chain or imported uniqueness theorem carries the argument. The remaining overgeneralization of 'no output-side pair isolates either ability' is a scope/correctness concern, not an input-output equivalence, and does not raise the circularity score.
Axiom & Free-Parameter Ledger
free parameters (3)
- k (mutual k-NN neighborhood size) =
10
- Equal weights of the four E components =
0.25 each
- Independent-init floor scale-match seed =
fixed seed
axioms (5)
- domain assumption Reading/writing labels: W_E is the 'reading code' and W_U the 'writing code'
- domain assumption Greedy production = |V|-way argmax; comprehension = 2-way log-probability ranking
- domain assumption Damage scores are algebraically a−b and a−c (target, competitor, argmax log-probabilities)
- standard math CKA, orthogonal Procrustes and mutual k-NN are valid similarity measures for representations
- domain assumption The Pythia suite is a valid sample for geometric claims across scale and training
invented entities (1)
-
'Reading code' W_E / 'writing code' W_U labels and the composite entanglement index E
no independent evidence
read the original abstract
In the literate human brain, reading and writing doubly dissociate: a ventral decoding route (pure alexia) and a fronto-parietal encoding route (pure agraphia), sharing a partial orthographic core. A decoder-only large language model (LLM) drives both from one autoregressive path optimized on text (a \emph{cultural} invention, not an evolved instinct). We ask how entangled it is, comparing an input-side ``reading code'' $\mathbf{W}_{E}$ with an output-side ``writing code'' $\mathbf{W}_{U}$ via an index $\mathcal{E}\in[0,1]$ (CKA, Procrustes residual, mutual $k$-NN) calibrated against an independent-init floor and tied ceiling. On GPT-2, OPT and Pythia (14M--1.4B), untied models hold one \emph{coupled but sub-ceiling} code ($\mathcal{E}=0.23$--$0.35$, far above floor) on a non-monotonic couple-then-differentiate trajectory, $\mathbf{W}_{U}$ drifting $\sim$3.2$\times$ farther than $\mathbf{W}_{E}$ in every decile. Equally informative is a negative: the matching behavioural test, that comprehension and production fail together rather than dissociate, cannot be run. For minimal pairs the alexia analogue is empty by theorem: greedy production implies a vocabulary-wide argmax, so it wins the pairwise ranking. Differential-damage indices are not scale-identified: heavy-tailed damage makes linear standardizations collapse onto their larger term, and the rank transform fixing this is bounded, so its null saturates. Both scores also contain the target's log-probability, which alone explains most of their variance and manufactures the apparent coupling. We withdraw a coupling statistic, a cross-level bridge and a separation measure. In a model reading and writing off one next-token distribution, no output-side pair isolates either ability: entanglement needing no index to see. By analogy, not homology, this situates LLMs in the space of possible minds.
Figures
Reference graph
Works this paper leans on
-
[1]
E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell. On the dangers of stochastic parrots: Can language models be too big? InProceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21), pp. 610-623,
2021
-
[3]
The interpretable signal is the SCALING of the log-PPL gap (nats/token the neural model saves over then-gram), not the raw ratio
on held-out wikitext-2. The interpretable signal is the SCALING of the log-PPL gap (nats/token the neural model saves over then-gram), not the raw ratio. Top-1 next-token agreement with pythia-160m= 0.290, symmetric KL= 3.030nats. ModelNneural PPL n-gram PPL ratio log-PPL gap EleutherAI/pythia-14m 1189888 104.467 473.479 0.221 1.511 EleutherAI/pythia-70m ...
2026
-
[8]
H. Inan, K. Khosravi, and R. Socher. Tying word vectors and word classifiers: A loss framework for language modeling. In ICLR 2017; arXiv:1611.01462,
Pith/arXiv arXiv 2017
-
[15]
C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of Machine Learning Research, 21(140):1-67; arXiv:1910.10683,
Pith/arXiv arXiv 1910
-
[16]
doi: 10.1002/ana.21804. M. Shanahan. Palatable conceptions of disembodied being: Terra incognita in the space of possible minds. arXiv preprint arXiv:2503.16348 (also PhilPapers SHAPCO-6),
-
[19]
A. Warstadt, A. Mueller, L. Choshen, E. Wilcox, C. Zhuang, J. Ciro, R. Mosquera, B. Paranjape, A. Williams, T. Linzen, and R. Cotterell. Findings of the babylm challenge: Sample- efficient pretraining on developmentally plausible corpora. InProceedings of the BabyLM Challenge / CoNLL (also arXiv:2504.08165 reissue),
-
[21]
S. Zhang, S. Roller, N. Goyal, et al. Opt: Open pre-trained transformer language models. arXiv:2205.01068,
-
[1984]
S. L. Smith, D. H. P. Turban, S. Hamblin, and N. Y. Hammerla. Offline bilingual word vectors, orthogonal transformations and the inverted softmax. InICLR 2017; arXiv:1702.03859,
Pith/arXiv arXiv 2017
-
[1987]
N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramèr, and C. Zhang. Quantifying memorization across neural language models. InInternational Conference on Learning Represen- tations (ICLR 2023); arXiv:2202.07646,
Pith/arXiv arXiv 2023
-
[1990]
M. Huh, B. Cheung, T. Wang, and P. Isola. Position: The platonic representation hypothesis. InProceedings of ICML 2024, PMLR 235:20617-20642; arXiv:2405.07987,
Pith/arXiv arXiv 2024
-
[1994]
O. Press and L. Wolf. Using the output embedding to improve language models. InProceedings of EACL 2017, pp. 157-163 (arXiv:1608.05859),
Pith/arXiv arXiv 2017
-
[2004]
Vaswani, N
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. InAdvances in Neural Information Processing Systems 30 (NeurIPS 2017), pp. 5998-6008,
2017
-
[2008]
S.-Y. Lam, Q. Zeng, J. Wu, and R. Voigt. Leveraging human production-interpretation asymmetries to test llm cognitive plausibility. InProceedings of ACL 2025 (Short Papers); arXiv:2503.17579,
Pith/arXiv arXiv 2025
-
[2009]
H. Cho, Y. Zhou, M. Kato, Y. Sakai, and N. Inoue. Understanding token probability encoding in output embeddings. InProceed- ings of COLING 2025, pp. 10618-10633; arXiv:2406.01468,
Pith/arXiv arXiv 2025
-
[2011]
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre- training of deep bidirectional transformers for language under- standing. InProceedings of NAACL-HLT 2019, pp. 4171- 4186; arXiv:1810.04805,
Pith/arXiv arXiv 2019
-
[2017]
J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei. Scaling laws for neural language models. arXiv:2001.08361,
Pith/arXiv arXiv 2001
-
[2019]
A. Lopardo, A. Harish, C. Arnett, and A. Gupta. Weight tying biases token embeddings towards the output space. arXiv:2603.26663,
-
[2020]
S. Kornblith, M. Norouzi, H. Lee, and G. Hinton. Similarity of neural network representations revisited. InICML 2019 (PMLR 97); arXiv:1905.00414,
Pith/arXiv arXiv 2019
-
[2021]
S. Biderman, H. Schoelkopf, Q. Anthony, H. Bradley, K. O’Brien, E. Hallahan, M. A. Khan, S. Purohit, U. S. Prashanth, E. Raff, A. Skowron, L. Sutawika, and O. van der Wal. Pythia: A suite for analyzing large language models across training and scaling. InProceedings of the 40th International Conference on Ma- chine Learning (ICML 2023), PMLR 202; arXiv:23...
Pith/arXiv arXiv 2023
-
[2022]
We place them here to keep the main narrative focused while preserving the full evidentiary record
19 A Supplementary figures The four figures collected here support claims established in the main text but do not carry an independent argument: two are single-model illustrations (the per-head induction map and the cross-model induction-vs- ICL scatter), and two visualize underpowered null relationships (the entanglement index against scale, and the geom...
2019
-
[2023]
E. Yiu, E. Kosoy, and A. Gopnik. Transmission versus truth, imitation versus innovation: What children can do that large language and language-and-vision models cannot (yet).Perspectives on Psychological Science, 19(5):874-883 (arXiv:2305.07666, 2023),
Pith/arXiv arXiv 2023
-
[2024]
F. Chollet. On the measure of intelligence. arXiv:1911.01547,
Pith/arXiv arXiv 1911
-
[2025]
Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. Roberta: A ro- bustly optimized bert pretraining approach. arXiv:1907.11692,
Pith/arXiv arXiv 1907
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.