Pith. sign in

REVIEW 4 major objections 5 minor 23 references

A decoder-only LLM reads and writes through one entangled token code, not the brain's two dissociable systems.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-02 08:58 UTC pith:IBDQCWVY

load-bearing objection An unusually honest paper with a well-calibrated E measurement; the abstract overstates one negative result. the 4 major comments →

arxiv 2607.24797 v2 pith:IBDQCWVY submitted 2026-07-02 q-bio.NC cs.AIcs.CLcs.LG

Reading Without a Reader: Large Language Models Collapse Reading and Writing into a Single Entangled Code

classification q-bio.NC cs.AIcs.CLcs.LG
keywords input and output embeddingsweight tyingentanglement indexrepresentational similaritycomprehension production dissociationlarge language modelsnext-token distributionalexia agraphia analogy
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that a decoder-only language model does not realize literacy the way the human brain does: instead of doubly dissociable reading and writing systems, it keeps a single coupled token code in which the input embedding and output unembedding sit roughly a third of the way between independent and forced-identical. It also establishes a negative result that it treats as load-bearing: the behavioral test that comprehension and production should fail together rather than dissociate cannot be run, because any two output-side scores are functionals of one next-token distribution and necessarily share terms. If correct, this reframes what coupling or dissociation measurements on LLMs can show, and it positions these models at a distinct point in the space of possible minds rather than as homologues of the human reading/writing brain.

Core claim

In untied decoder-only transformers, the input embedding WE (the 'reading code') and the output unembedding WU (the 'writing code') couple early in training and then partially differentiate, ending with an entanglement index E of 0.23–0.35 on a scale anchored by independent initialization (0) and tied weights (1): one coupled but sub-ceiling code, and the output side drifts roughly 3.2 times farther than the input side in every token-frequency decile. The matching behavioral prediction cannot be tested: comprehension as log-probability ranking and production as greedy argmax are both summaries of the same next-token distribution, greedy production implies a vocabulary-wide argmax that wins a

What carries the argument

The central object is the pair of token-coding matrices WE and WU in a decoder-only transformer, read as a 'reading code' and a 'writing code,' measured by a composite entanglement index E in [0,1] that fuses linear CKA, orthogonal-Procrustes residual with per-token cosine, and mutual k-nearest-neighbor overlap, calibrated against an independent-initialization floor and a tied ceiling. The argument's load-bearing pieces are the set-theoretic nesting (greedy production's success set is a subset of pairwise comprehension's success set) and the algebraic identity that both behavioral scores share the target's log-probability term a.

Load-bearing premise

The negative result hinges on defining comprehension as a two-way log-probability ranking and production as a vocabulary-wide greedy argmax; if production were defined differently, the 'produces but does not comprehend' cell could in principle fire, and the theorem would not transfer.

What would settle it

Find a single token pair for which greedy decoding emits the target form while the target's log-probability is lower than a competitor's (the theorem says this is impossible for single-token pairs), or construct a differential-damage index with a non-saturating null that reveals a site where comprehension drops while production is spared under matched severity.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Weight-geometry claims about how separate or shared an LLM's input and output codes are need floor-and-ceiling calibration; raw similarity numbers are otherwise uninterpretable.
  • Behavioral double dissociations between comprehension and production cannot be established from output-side scores in a decoder-only model, because any two scalar summaries of one next-token distribution share terms.
  • Reported couplings between comprehension and production measures should be treated with suspicion unless the shared log-probability term is removed; the paper shows removing it removes most of the apparent signal.
  • If the claim holds, decoder-only LLMs realize functional literacy through a single entangled route, inverting the human brain's two-system architecture and placing them outside the homology space of reading/writing models.
  • The couple-then-differentiate trajectory means that conclusions drawn from end-of-training embeddings miss a substantial reorganization: the codes peak early in training and then partly separate.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The 'cannot be run' theorem is specific to greedy decoding; with sampling or temperature-based production, the alexia cell could in principle fire, which would make the dissociation claim testable under a different production rule than the one the paper uses.
  • A natural testable extension is to train a model against a separated objective, forcing distinct read and write pathways, and ask whether the entanglement index drops and whether any output-side dissociation then becomes measurable; the paper explicitly leaves this open.
  • For cognitive neuroscience, the result suggests LLMs should not be used as models of acquired alexia or agraphia: their single-path coupling is a different failure mode, and surface behavioral dissociations would not reveal underlying mechanism.
  • The scale-invariance null (E flat across model sizes) is underpowered in the paper itself, but if it held at larger scales it would imply the read/write entanglement is a property of the architecture and training objective, not of model capacity.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper asks whether the input embedding W_E and output unembedding W_U of a decoder-only LLM form one entangled 'read/write' code, using the brain's double dissociation between reading and writing as an analogy. The authors define a composite entanglement index E built from linear CKA, orthogonal-Procrustes residual/per-token cosine, and mutual k-NN overlap, calibrated against an independent-initialization floor, a shuffled-token null, and a tied ceiling. On tied GPT-2/OPT models E=1 by construction; on untied Pythia models (14M–1.4B) E≈0.23–0.35, far above the floor, with a non-monotonic 'couple-then-differentiate' trajectory and an asymmetric drift in which W_U moves ~3.2× farther than W_E in every frequency decile. The paper then reports a negative result: three attempts to build a behavioural comprehension/production dissociation test fail, because greedy production is nested in pairwise log-probability comprehension, differential-damage indices are not scale-identified, and the two scores share a target log-probability term that manufactures coupling. Several statistics are explicitly withdrawn. The conclusion is that decoder-only LLMs realize literacy through a single entangled statistical code rather than two dissociable systems, offered as an analogy rather than a homology.

Significance. If the geometric claim stands, it is a useful, controlled contribution to mechanistic interpretability: it quantifies how far untied W_E and W_U are from both independence and forced identity, and it adds a training-trajectory characterization that goes beyond Lopardo et al. (2026). The paper is unusually transparent: bootstrap CIs, permutation nulls, shuffled-token controls, and an independent-init floor accompany the headline numbers, and the authors openly withdraw a coupling statistic, a cross-level bridge, and an encoder/decoder separation measure. This transparency is a genuine strength. The negative claim about the impossibility of an output-side dissociation test is provocative and could be significant if it were scoped correctly, but as written it is broader than the results support. The paper's framing as analogy is explicit and appropriately hedged.

major comments (4)
  1. [Abstract; §4.8 Result 1; §5.1] The assertion that the behavioural dissociation test 'cannot be run' and that 'no pair of output-side scores isolates either ability' is stronger than the results establish. Result 1 is a nesting theorem only for greedy production against a 2-way log-probability ranking in the same context. It does not cover stochastic decoding (temperature/top-p sampling), where a token with lower probability than a competitor can still be produced, nor reference-scored multi-token generation, which §5.1 explicitly leaves open. Result 3 is an empirical decomposition, not an impossibility proof: after removing the shared log-probability term the residual coupling is +0.13 and marginal against its null. The abstract and contributions should be scoped to 'the three operationalizations we attempted cannot be made valid,' and the existence of any valid output-side dissociation test should remain open. As wri
  2. [§4.8 Result 2] The scale-identification argument is internally plausible for the particular linear-standardization comparisons, but the conclusion that the per-site test is 'either valid without power or powered without validity' is stated as a general dilemma. The bounded rank-difference statistic and the saturation of the family-wise permutation null are properties of the chosen max-statistic permutation construction; exact conditional tests or different test statistics could in principle restore validity. This does not undermine the decision to withdraw the three indices, but the text should present the result as a property of this test construction, not as an impossibility theorem about differential-damage indices in general.
  3. [§4.7, Table B11] The frequency-decile control is claimed to rule out the gradient-exposure confound. However the confound is not solely a frequency effect: the output matrix receives a gradient signal for the target token on every step, while an input row updates only when that token appears as context, and these two update schedules can differ even within a decile. The decomposition shows the ~3.2× drift asymmetry is not carried by frequent tokens, but it does not by itself separate 'output-side gradient schedule' from 'genuine functional differentiation.' The claim that the frequency-decile analysis rules out the confound should be softened, or supplemented with a matched update-count analysis.
  4. [§6, Limitations] The tied-versus-untied contrast is confounded with model family, as the paper itself acknowledges; the tied condition is GPT-2/OPT and the untied condition is Pythia only. The abstract's phrase 'on GPT-2, OPT and Pythia' may suggest a controlled architectural comparison, but GPT-2 and OPT serve only as a by-construction ceiling. This is not an error, but the abstract and contributions should state more prominently that the substantive 'coupled, sub-ceiling' claim rests on the single Pythia family and that no within-family tied/untied manipulation was performed.
minor comments (5)
  1. [§4.5] Table reference is broken: 'Table ??' should be 'Table B6' or the appropriate table number.
  2. [Abstract] The abstract contains 'aculturalinvention' — a missing space, making the phrase hard to read.
  3. [Figure 6] The figure caption includes 'NB: confounded by gradient exposure,' which appears to contradict the text of §4.7 claiming the frequency-decile analysis rules out that confound. The caption should be reconciled with the main text.
  4. [§3.2] The claim that 'we verified the trends are not an artifact of the equal-weighting choice' is not supported by any table or appendix entry. A short sensitivity analysis (e.g., E under alternative weights) should be reported or the sentence should be removed.
  5. [§4.8 Result 1] The notation F and F' and the conditions for the 'empty by theorem' claim are introduced in prose; a compact formal statement would help readers verify the scope of the nesting result.

Circularity Check

0 steps flagged

No significant circularity: the surviving read/write entanglement claims are anchored against independent-init floors and tied ceilings, and the paper explicitly withdraws its own degenerate behavioral statistics.

full rationale

The paper's central positive claim is the calibrated geometric entanglement index E between WE and WU. This is not circular: E is normalized against an independent-initialization floor and a tied ceiling, raw components (CKA, Procrustes residual, per-token cosine, k-NN) are reported alongside the composite, and the floor/null are parameter-free controls rather than fitted quantities. The asymmetric-drift trajectory is credited to Lopardo et al. (2026) but re-derived with nulls and frequency-decile controls, so the citation is not load-bearing. The behavioral 'cannot be run' negative is explicitly derived from the chosen operationalizations (greedy argmax vs 2-way log-prob ranking) and is labeled a theorem; the paper does not use it as evidence for the weight-level coupling but as a scoped measurement failure, and it explicitly leaves open richer probes. The statistical couplings, contingency table, cross-level bridge, and encoder/decoder separation measure that could have been circular are all self-identified and withdrawn, with the algebraic shared-term decomposition (a-b and a-c both containing a) stated as the reason. No self-citation chain or imported uniqueness theorem carries the argument. The remaining overgeneralization of 'no output-side pair isolates either ability' is a scope/correctness concern, not an input-output equivalence, and does not raise the circularity score.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 1 invented entities

The paper's quantitative claims add few free parameters: the composite E is a definitional average of four anchored components with the raw parts reported, and the hand-set knobs (k=10, equal weights, Gaussian-floor seed) are low-stakes. The load-bearing assumptions are the reading/writing relabeling (explicitly disclaimed as analogy), the greedy-argmax/log-prob operationalizations that make §4.8's theorems exact, the standard similarity measures, and the Pythia suite as the sample for the geometric claims.

free parameters (3)
  • k (mutual k-NN neighborhood size) = 10
    Hand-set for the local-structure component of E (§3.2). Low stakes; the observed k-NN overlap is far above both the shuffled-token null and the independent-init floor (Table B10).
  • Equal weights of the four E components = 0.25 each
    Definitional choice for the composite index (§3.2); authors state the trends are verified not to be an artifact of the equal-weighting choice. Raw components are reported alongside E.
  • Independent-init floor scale-match seed = fixed seed
    The floor is computed from two Gaussian matrices matched to each code's scale under a fixed seed (§3.2). Minor, but the floor values themselves enter the E normalization.
axioms (5)
  • domain assumption Reading/writing labels: W_E is the 'reading code' and W_U the 'writing code'
    Explicitly an analogy, not a homology, disclaimed repeatedly (§1, §3.2, §6). Load-bearing for framing and interpretation but not for the quantitative claims.
  • domain assumption Greedy production = |V|-way argmax; comprehension = 2-way log-probability ranking
    Underpins the nesting theorem in §4.8 Result 1 that makes the alexia cell empty; alternative decision rules (sampling, reference-scored generation) would break the theorem, as the authors concede in §5.1.
  • domain assumption Damage scores are algebraically a−b and a−c (target, competitor, argmax log-probabilities)
    Makes the shared-term analysis in §4.8 Result 3 exact: both scores contain the target's log-probability a, so the apparent coupling is partially manufactured.
  • standard math CKA, orthogonal Procrustes and mutual k-NN are valid similarity measures for representations
    Invoked from Kornblith et al. (2019), Smith et al. (2017) and Huh et al. (2024) within the representational-similarity tradition (§2.3, §3.2).
  • domain assumption The Pythia suite is a valid sample for geometric claims across scale and training
    The untied condition is realized only by Pythia/GPT-NeoX, so tied-vs-untied is confounded across families (acknowledged in §6); scale and trajectory claims rest on this single family.
invented entities (1)
  • 'Reading code' W_E / 'writing code' W_U labels and the composite entanglement index E no independent evidence
    purpose: Interpretive lens mapping the brain's read/write dissociation onto input/output embedding geometry, plus a calibrated similarity measure
    A relabeling of existing matrices plus a composite index with floor/null/ceiling anchors; explicitly an analogy, not homology (§3.2, §6). No falsifiable handle beyond the weight measurements themselves.

pith-pipeline@v1.3.0-alltime-deepseek · 31647 in / 18498 out tokens · 189004 ms · 2026-08-02T08:58:12.120416+00:00 · methodology

0 comments
read the original abstract

In the literate human brain, reading and writing doubly dissociate: a ventral decoding route (pure alexia) and a fronto-parietal encoding route (pure agraphia), sharing a partial orthographic core. A decoder-only large language model (LLM) drives both from one autoregressive path optimized on text (a \emph{cultural} invention, not an evolved instinct). We ask how entangled it is, comparing an input-side ``reading code'' $\mathbf{W}_{E}$ with an output-side ``writing code'' $\mathbf{W}_{U}$ via an index $\mathcal{E}\in[0,1]$ (CKA, Procrustes residual, mutual $k$-NN) calibrated against an independent-init floor and tied ceiling. On GPT-2, OPT and Pythia (14M--1.4B), untied models hold one \emph{coupled but sub-ceiling} code ($\mathcal{E}=0.23$--$0.35$, far above floor) on a non-monotonic couple-then-differentiate trajectory, $\mathbf{W}_{U}$ drifting $\sim$3.2$\times$ farther than $\mathbf{W}_{E}$ in every decile. Equally informative is a negative: the matching behavioural test, that comprehension and production fail together rather than dissociate, cannot be run. For minimal pairs the alexia analogue is empty by theorem: greedy production implies a vocabulary-wide argmax, so it wins the pairwise ranking. Differential-damage indices are not scale-identified: heavy-tailed damage makes linear standardizations collapse onto their larger term, and the rank transform fixing this is bounded, so its null saturates. Both scores also contain the target's log-probability, which alone explains most of their variance and manufactures the apparent coupling. We withdraw a coupling statistic, a cross-level bridge and a separation measure. In a model reading and writing off one next-token distribution, no output-side pair isolates either ability: entanglement needing no index to see. By analogy, not homology, this situates LLMs in the space of possible minds.

Figures

Figures reproduced from arXiv: 2607.24797 by Diego Salda\~na Ulloa.

Figure 1
Figure 1. Figure 1: ). Orthographic word-form selectivity is therefore acquired over training, as the neuronal￾recycling framing predicts for a cultural skill, though we cannot fully exclude residual subword-frequency structure. We do not claim a clean co-onset with the induction phase change: the raw-gap onset is itself a tokenization artifact (its maximum-slope step is step 1), and, as we show next, the induction circuit is… view at source ↗
Figure 2
Figure 2. Figure 2: Layerwise linear-probe decodability of word [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Compression (bits/byte) vs functional compe [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Hidden-state geometry vs perceptual norms [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Entanglement index E by architecture. Tied models sit at the ceiling by construction; untied Pythia models sit well below it and far above the independent￾init floor. 10 0 10 1 10 2 10 3 10 4 10 5 training step 0.0 0.1 0.2 0.3 0.4 0.5 metric Read/write metrics over training - pythia-160m CKA mean cosine entanglement E 10 0 10 1 10 2 10 3 10 4 10 5 training step 0 1 2 3 4 5 Frobenius drift from step1 Asymme… view at source ↗
Figure 6
Figure 6. Figure 6: Couple-then-differentiate trajectory of WE/WU CKA over pythia-160m training: rise to a peak of 0.494 at step 4000, then partial differentiation to 0.210 by step 143000, with asymmetric writing-code drift. The asymmetric drift replicates Lopardo et al. (2026). 4.8 The read/write distinction resists operationalization at the output Having found one coupled read/write code in the weights, we set out to test t… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

23 extracted references · 18 linked inside Pith

  1. [1]

    E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell. On the dangers of stochastic parrots: Can language models be too big? InProceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21), pp. 610-623,

  2. [3]

    The interpretable signal is the SCALING of the log-PPL gap (nats/token the neural model saves over then-gram), not the raw ratio

    on held-out wikitext-2. The interpretable signal is the SCALING of the log-PPL gap (nats/token the neural model saves over then-gram), not the raw ratio. Top-1 next-token agreement with pythia-160m= 0.290, symmetric KL= 3.030nats. ModelNneural PPL n-gram PPL ratio log-PPL gap EleutherAI/pythia-14m 1189888 104.467 473.479 0.221 1.511 EleutherAI/pythia-70m ...

  3. [8]

    H. Inan, K. Khosravi, and R. Socher. Tying word vectors and word classifiers: A loss framework for language modeling. In ICLR 2017; arXiv:1611.01462,

  4. [15]

    Raffel, N

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of Machine Learning Research, 21(140):1-67; arXiv:1910.10683,

  5. [16]

    doi: 10.1002/ana.21804. M. Shanahan. Palatable conceptions of disembodied being: Terra incognita in the space of possible minds. arXiv preprint arXiv:2503.16348 (also PhilPapers SHAPCO-6),

  6. [19]

    Warstadt, A

    A. Warstadt, A. Mueller, L. Choshen, E. Wilcox, C. Zhuang, J. Ciro, R. Mosquera, B. Paranjape, A. Williams, T. Linzen, and R. Cotterell. Findings of the babylm challenge: Sample- efficient pretraining on developmentally plausible corpora. InProceedings of the BabyLM Challenge / CoNLL (also arXiv:2504.08165 reissue),

  7. [21]

    Zhang, S

    S. Zhang, S. Roller, N. Goyal, et al. Opt: Open pre-trained transformer language models. arXiv:2205.01068,

  8. [1984]

    S. L. Smith, D. H. P. Turban, S. Hamblin, and N. Y. Hammerla. Offline bilingual word vectors, orthogonal transformations and the inverted softmax. InICLR 2017; arXiv:1702.03859,

  9. [1987]

    Carlini, D

    N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramèr, and C. Zhang. Quantifying memorization across neural language models. InInternational Conference on Learning Represen- tations (ICLR 2023); arXiv:2202.07646,

  10. [1990]

    M. Huh, B. Cheung, T. Wang, and P. Isola. Position: The platonic representation hypothesis. InProceedings of ICML 2024, PMLR 235:20617-20642; arXiv:2405.07987,

  11. [1994]

    Press and L

    O. Press and L. Wolf. Using the output embedding to improve language models. InProceedings of EACL 2017, pp. 157-163 (arXiv:1608.05859),

  12. [2004]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. InAdvances in Neural Information Processing Systems 30 (NeurIPS 2017), pp. 5998-6008,

  13. [2008]

    S.-Y. Lam, Q. Zeng, J. Wu, and R. Voigt. Leveraging human production-interpretation asymmetries to test llm cognitive plausibility. InProceedings of ACL 2025 (Short Papers); arXiv:2503.17579,

  14. [2009]

    H. Cho, Y. Zhou, M. Kato, Y. Sakai, and N. Inoue. Understanding token probability encoding in output embeddings. InProceed- ings of COLING 2025, pp. 10618-10633; arXiv:2406.01468,

  15. [2011]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre- training of deep bidirectional transformers for language under- standing. InProceedings of NAACL-HLT 2019, pp. 4171- 4186; arXiv:1810.04805,

  16. [2017]

    Kaplan, S

    J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei. Scaling laws for neural language models. arXiv:2001.08361,

  17. [2019]

    Lopardo, A

    A. Lopardo, A. Harish, C. Arnett, and A. Gupta. Weight tying biases token embeddings towards the output space. arXiv:2603.26663,

  18. [2020]

    Kornblith, M

    S. Kornblith, M. Norouzi, H. Lee, and G. Hinton. Similarity of neural network representations revisited. InICML 2019 (PMLR 97); arXiv:1905.00414,

  19. [2021]

    Biderman, H

    S. Biderman, H. Schoelkopf, Q. Anthony, H. Bradley, K. O’Brien, E. Hallahan, M. A. Khan, S. Purohit, U. S. Prashanth, E. Raff, A. Skowron, L. Sutawika, and O. van der Wal. Pythia: A suite for analyzing large language models across training and scaling. InProceedings of the 40th International Conference on Ma- chine Learning (ICML 2023), PMLR 202; arXiv:23...

  20. [2022]

    We place them here to keep the main narrative focused while preserving the full evidentiary record

    19 A Supplementary figures The four figures collected here support claims established in the main text but do not carry an independent argument: two are single-model illustrations (the per-head induction map and the cross-model induction-vs- ICL scatter), and two visualize underpowered null relationships (the entanglement index against scale, and the geom...

  21. [2023]

    E. Yiu, E. Kosoy, and A. Gopnik. Transmission versus truth, imitation versus innovation: What children can do that large language and language-and-vision models cannot (yet).Perspectives on Psychological Science, 19(5):874-883 (arXiv:2305.07666, 2023),

  22. [2024]

    F. Chollet. On the measure of intelligence. arXiv:1911.01547,

  23. [2025]

    Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. Roberta: A ro- bustly optimized bert pretraining approach. arXiv:1907.11692,