Pith. sign in

REVIEW 3 major objections 4 minor 45 references

ChatGPT-like AIs tip to bad output through a deterministic many-body first-passage mechanism, captured by a closed-form finite-layer threshold.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 02:52 UTC pith:ZMEHGV3P

load-bearing objection Clean analytic core, overclaimed empirical wrapper: Eq. (2) is real, but the 19/21 test doesn't actually test it. the 3 major comments →

arxiv 2607.25279 v1 pith:ZMEHGV3P submitted 2026-07-28 cs.AI cond-mat.dis-nnmath-phmath.MPnlin.AOphysics.soc-ph

Many-body Tipping Dynamics of ChatGPT-like AIs

Frank Yingjie Huo , Neil F. Johnson This is my paper
classification cs.AI cond-mat.dis-nnmath-phmath.MPnlin.AOphysics.soc-ph
keywords tippingfirst passagemany-bodyattentiontransformerresidual streamoutput basinslarge language models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that the sudden, seemingly unpredictable slips of ChatGPT-like AIs into harmful, misleading, or repetitive output are not idiosyncratic failures but a common dynamical phenomenon: tokens behave like interacting spins that travel through the fixed stack of attention layers, and 'tipping' happens when the internal state crosses a boundary between two competing output basins. The authors derive a closed-form threshold, Eq. (2), relating the number of good outputs before the first bad one to the depth of the model and the geometry of the basins, and they show this coarse-grained prediction matches regime observations across models from three independent training lineages without any fitted parameters. The significance is that if this mechanism is right, a broad class of AI malfunctions is 'foreseeable engineering risk'—a property of the architecture's finite depth and attention disorder—rather than irreducible stochastic chaos, with consequences for how AI harm is assessed in law and society.

Core claim

The core claim is that for a pure-Attention skeleton of a decoder-only transformer, the residual stream evolves by a product of per-layer factors (I + H^(t) ⊗ W_V^(t)), and the competition between two output types B and D is decided by the sign of x = r·(D−B). Tipping to D occurs when x crosses zero at some causal position n*+1. Because each attention column is softmax-normalized, the cumulative operator L^(t) has column sum 2^t, and the wall condition yields L^(t)_{1,n*+1}/2^t = B·(D−B)/[(A−B)·(B−D)]. This threshold holds for any number of layers t and is not fitted; it emerges from the interaction of the prompt basin A, the good basin B, and the bad basin D. Empirically, the immediate-vers

What carries the argument

The central object is the finite-depth propagator U(ℓ)=T_ℓ ∏_{t=1}^ℓ (I+(H^(t))^T ⊗ W_V^(t)); in the reduced W_V=I case it becomes L^(t)=∏_{s=1}^t (I+H^(s)), where H^(s) is a causal, column-stochastic attention matrix. Since each column of I+H^(s) sums to 2, every column of L^(t) sums to 2^t, so L^(t)_{1n}/2^t is the normalized fraction of the original prompt basin A surviving at position n after t layers. The tipping boundary is the hyperplane x=r·(D−B)=0; the threshold Eq. (2) is exactly this dot-product wall evaluated after finite-depth transport. Attention disorder—quantified by column entropy—controls whether the internal state moves toward, away from, or along this wall, and in the low

Load-bearing premise

The analytic threshold assumes the transformer can be stripped to a pure-Attention skeleton—identity layer norm, zero MLP, identity value-projection, one effective head per layer, and suppressed positional encoding—while the tipping mechanism and its numeric threshold remain the same in real trained models; the paper argues this by analogy and coarse-graining, not by deriving the full-model dynamics.

What would settle it

A single counterexample where a decoder-only model under greedy decoding exhibits sudden tipping that cannot be described by any choice of two basin vectors A, B, D satisfying Eq. (2) across layer depths—e.g., tipping without an accompanying zero-crossing of the residual-stream dot product x = r·(D−B) in the last-layer representation—would falsify the claim that the mechanism is universal. More directly, a model in which attention disorder remains high (columns near uniform) at all layers yet still tips suddenly would violate the predicted link between disorder and transport.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If correct, sudden degradation of ChatGPT-like systems under greedy decoding is not a sampling artifact; safety tests should treat it as a deterministic geometric property of the residual stream.
  • The closed-form threshold gives a parameter-free way to predict whether a prompt will tip immediately or be delayed, usable before deployment on new models.
  • The low-disorder saturation implies that beyond a handful of effective layers, adding more layers does not change the tipping point; interpretability and safety interventions should focus on early layers.
  • Coarse-graining to sentence-level basins means the mechanism scales from token repetition to meaning shifts, making it relevant to real-world harmful outputs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: replacing attention columns by identity at saturated (low-disorder) layers should barely change output, while at high-disorder layers it should drastically alter tipping; the paper's Fig. 3 does this for GPT-2, but the same prediction could be checked in other families to confirm architecture-robustness.
  • If the many-body spin picture is right, then interventions that manipulate attention disorder (e.g., sharpening or flattening attention) could serve as a control knob to delay or prevent tipping, without retraining.
  • The paper's own caveat that the full model includes LN/MLP which shift the tipping point suggests the threshold may need renormalization for real models; a precise quantitative bridge between pure-Attention threshold and full-model threshold is left open.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a many-body, finite-depth mechanism for sudden output tipping in ChatGPT-like language models. It strips the transformer layer to a pure-Attention skeleton (LN identity, MLP zero, W_V = I, one head per layer, suppressed positional encoding) and derives an exact finite-depth propagator for the residual stream. In a three-basin reduction (A, B, D), tipping is modeled as first-passage of the projected residual x = r·(D−B) across the wall x = 0, yielding a closed threshold Eq. (2) involving the normalized prompt-memory coefficient L_1^{(t)}(n*+1)/2^t. The paper presents qualitative regime tests across GPT-2, Pythia, and OPT families (19/21), and analyzes attention disorder via column entropy, linking low-disorder saturation to reduced sensitivity to single-layer interventions.

Significance. If the quantitative threshold Eq. (2) were validated, the paper would offer a rare mechanism-level, parameter-free prediction for a socially consequential class of LLM failures. The analytic derivation from the stated reduced assumptions is clean and internally consistent, and the paper makes a genuine effort to ground the qualitative claims in cross-family experiments. The no-free-parameter framing and the use of deterministic, greedy-decoding outputs are strengths. However, the current empirical support does not actually test the central quantitative threshold; it tests only the sign of a linear projection. As a result, the significance is currently conditional: the qualitative mechanism may be plausible, but the paper's core quantitative claim remains unverified in real models. The architecture-robustness bridge from the pure-Attention skeleton to full pre-normalized transformers is also argued by analogy rather than derived or directly measured.

major comments (3)
  1. [Fig. 2(f), End Matter D, main text "Figure 2(f) tests..."] The cross-model validation reported as 19/21 does not test Eq. (2). The protocol computes x = C·(D−B) in the penultimate layer and classifies cases as immediate or delayed by its sign. It never measures L_1^{(t)}/2^t, never locates the crossover n*, and never compares an observed tipping point with the analytic threshold B·(D−B)/[(A−B)·(B−D)]. Thus the empirical evidence supports a binary sign projection, not the closed finite-depth first-passage threshold that is the paper's central theoretical output. To support the claim of 'coarse-grained regime prediction' from Eq. (2), the authors should report the quantitative comparison, e.g., predicted versus observed n* or predicted versus observed L_1^{(t)}/2^t at crossing, at least for a subset of models and prompts.
  2. [End Matter B/C and Fig. 1(a)] The load-bearing premise that the pure-Attention skeleton preserves the tipping mechanism of real ChatGPT-like models is not derived. The reduction assumes LN identity, zero MLP, W_V = I, one effective head per layer, and suppressed positional encoding. The paper itself notes in Fig. 1(a) that MLP/LN shift the empirical tipping point, and End Matter C states that GPT-2 is pre-normalized, so its low-disorder collapse is reached through learned W_eff and MLP layers. This is an argument by analogy and coarse-graining, not a derivation. A concrete test would be to measure the normalized propagation coefficients L^{(t)}/2^t in a full pre-normalized model and check whether the reduced-model form (e.g., multiplicative (I+H) factors with column sums 2) is a quantitatively accurate description, or to show that the tipping-point shift from MLP/LN is small within the regime claimed.
  3. [End Matter C, effective depth ℓ_eff] The saturation claim 'ℓ_eff ≈ 5 already suffices for ℓ up to 20' is stated without supporting data in the main text or displayed figures. This is central to the low-disorder freezing argument and to the claim that only a few effective layers control the dynamics. If the supporting measurement exists only in the SM, the main text should either report it or clearly label it as a conjecture. As written, the reader cannot verify that the saturation is a general property of real models rather than an artifact of the reduced model.
minor comments (4)
  1. [Eq. (2) notation] The notation L_1^{(t)}(n*+1) is not explicitly defined in the main text; the reader must infer that the first subscript indexes the original A position and the second the causal position. A sentence or a displayed definition would improve readability.
  2. [Fig. 1(d)] The sentence-level tipping examples are described but not shown in the figure or text. Since one of the paper's motivations is that tipping occurs at phrase/sentence scale, a concrete example (even one phrase-level trajectory) would strengthen the presentation.
  3. [End Matter D, Fig. 2(f) class imbalance] The 19/21 statistic is presented in the main text without the class imbalance or the number of immediate vs. delayed cases. The SM is said to report this, but the main-text claim is hard to interpret without the base rates.
  4. [General] Several key derivations and empirical protocols are deferred to SM sections (Sec. 5, 9, 11) that were not available in the reviewed manuscript. The paper would be easier to evaluate if at least the threshold test and the sentence coarse-graining were summarized in the main text or an appendix.

Circularity Check

0 steps flagged

No significant circularity: Eq. (2) is derived from the paper's own stated reduction, and the empirical test is a no-fit geometric projection rather than a fitted prediction.

full rationale

The central threshold Eq. (2) is obtained in End Matter B by substituting the finite-depth propagator (End Matter A, Eqs. iii/vii) into the dot-product wall condition x=0. This is an internal mathematical consequence of the explicitly stated pure-Attention reduction (LN identity, MLP zero, W_V=I, one effective head), not a quantity fitted to data. The cross-model validation in Fig. 2(f) and End Matter D uses a prescribed no-fit projection x=C·(D−B) with fixed phrase centroids; no model-specific parameter is tuned, so the 19/21 result is not a fitted input renamed as a prediction. The cited earlier works [42–44] are presented as limiting cases of the same derivation (End Matter B), not as the source of the threshold, so the self-citations are not load-bearing in the derivation. The main weakness is that Fig. 2(f) tests a sign corollary of the tipping boundary rather than quantitatively evaluating Eq. (2) (L_1/2^t or n*); that is a validation/overclaim gap, not a circular reduction. End Matter C also explicitly notes GPT-2's pre-normalized architecture differs from the minimal model's no-normalization mechanism, underscoring that the derivation is a model-based argument rather than a tautology. No step in the claimed derivation chain reduces to its own inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 3 invented entities

The analytic core is parameter-free within its reduced model: Eq. (2) has no fitted constants. The load-bearing non-derived inputs are modeling axioms: (1) three basins suffice, (2) penultimate-layer mean-pooled centroids are the correct empirical coordinate, (3) pure-Attention stripping preserves the tipping mechanism, and (4) the no-normalization norm-doubling identity transfers to real pre-norm models via learned effective maps. The paper introduces no new particle-like entity, but it does invent effective objects (basin wall as first-passage coordinate, effective depth, mean-field attention map) that are measurable in residual-stream space but whose sufficiency for real-model tipping is asserted.

axioms (5)
  • domain assumption A three-basin reduction (A, B, D) captures sentence-level output competition in real models.
    The analytic theory reduces all outputs to three representative vectors; the paper concedes SM Sec. 5.3 generalizes to more types but the empirical validation uses only two basins plus the prompt. Claimed in the 'few-basin reduction' framing and Fig. 2(a-c).
  • domain assumption The penultimate-layer residual stream, with mean-pooled phrase centroids as basins, is a faithful coordinate for output tipping.
    B and D are computed as mean-pooled penultimate-layer centroids over six fixed phrases per basin (End Matter D); the paper asserts this is the same space as the analytical reduced model without derivation. Used in Fig. 2(f) and the 19/21 statistic.
  • ad hoc to paper The pure-Attention skeleton (LN identity, MLP zero, W_V = I, one head per layer, suppressed positional encoding) preserves the tipping mechanism of real models.
    Eq. (1) and Eq. (2) are derived under this stripping; the paper claims MLP/LN only 'reshape' tipping (Fig. 1(a)) but this is an empirical observation, not a derived equivariance. End Matter C further relies on architecture-specific paths to the low-disorder limit.
  • standard math Residual stream norms double per layer under no-normalization, so cumulative column sums satisfy Σ_i L^{(t)}_{iq} = 2^t.
    Used in End Matter A-B to derive Eq. (2); true as a coefficient identity for the W_V=I reduction but becomes a modeling assumption when applied to real pre-norm transformers where the norm growth is different.
  • standard math Attention outputs are deterministic and column-stochastic for fixed prompts under greedy decoding.
    Standard property of softmax attention; used to define the entropy measure S^{(t)} in Fig. 3 and End Matter C-D.
invented entities (3)
  • Output basin boundary (x = 0 wall) as a first-passage coordinate independent evidence
    purpose: Defines the tipping event as crossing of r·(D−B)=0; used to derive Eq. (2)
    The wall is a geometric object in residual-stream space; it is measurable outside the paper via penultimate-layer projections, and Fig. 2(f) applies it to seven models, giving a falsifiable classification.
  • Effective depth ℓ_eff ≪ ℓ (low-disorder saturation) no independent evidence
    purpose: Explains why tipping saturates and why deep layers are redundant
    Introduced to match the observation that L^{(t)}_{1n}/2^t saturates; the empirical estimate ℓ_eff ≈ 5 for ℓ up to 20 (End Matter C) is a fitted/observed statement without a separate predicted consequence outside the paper.
  • Frobenius mean-field attention map H-bar(t) no independent evidence
    purpose: Defines the layer-level disorder entropy used in Fig. 3
    A summary quantity built from head weights w^{(t)}_h ∝ ||W_O W_V||_F; no independent falsifiable handle beyond the empirical plots; choice of weighting is one of several possible.

pith-pipeline@v1.3.0-alltime-deepseek · 4969 in / 5376 out tokens · 106946 ms · 2026-08-01T02:52:18.181881+00:00 · methodology

0 comments
read the original abstract

Why do ChatGPT-like AIs, despite major architectural and training differences, unexpectedly tip to undesirable content (e.g. harmful, misleading, repetitive) even under deterministic greedy decoding? We show that a broad class of such tippings is caused by the many-body interactions between tokens (spins) as they cross the finite-layer system. Tipping emerges as a dynamical first passage process between competing output basins. Attention disorder controls the transport toward, away from, or along the basins' boundary. A few-basin reduction yields a closed finite-layer threshold, whose coarse-grained predictions show good agreement across ChatGPT-like families. These results suggest that a broad class of AI failures represents 'foreseeable engineering risk' rather than inherently unpredictable behavior, with important implications for legal and societal assessments of AI harm.

Figures

Figures reproduced from arXiv: 2607.25279 by Frank Yingjie Huo, Neil F. Johnson.

Figure 1
Figure 1. Figure 1: FIG. 1. (a) Tipping to a repeated output in full GPT-2 (top); without MLP (middle); and Attention only (bottom). This shows [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: FIG. 2. (a) Schematic of five causal positions in an [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: FIG. 3. Attention disorder measured empirically in GPT-2 across a corpus of [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 18 linked inside Pith

  1. [1]

    Accessed June 7, 2026

    Ethical Tech Initiative, DAIL – The Database of AI Liti- gation, The George Washington University Law School, https://blogs.gwu.edu/law-eti/ai-litigation-d atabase/ (2026), direct database portal available at https://etidail.com/. Accessed June 7, 2026

    2026

  2. [2]

    L. Weidingeret al., Taxonomy of risks posed by language models, inProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (F AccT ’22) (Association for Computing Machinery, 2022) pp. 214–229

    2022

  3. [3]

    M. Ueda, M. L. Birnbaum, Y. Liu, Q. Yu, X. Tian, A. Mirer, S. Ramanathan, and M. Sinyor, Help-seeking in the age of AI: Cross-sectional survey of the use and perceptions of AI-based mental health support among US adults, JMIR Mental Health13, e88196 (2026)

    2026

  4. [4]

    O’Donnell, The new war room, MIT Technology Review (2026)

    J. O’Donnell, The new war room, MIT Technology Review (2026)

    2026

  5. [5]

    Center for Countering Digital Hate,Fake Friend: How ChatGPT betrays vulnerable teens by encouraging dan- gerous behavior, CCDH Report (Center for Countering Digital Hate (CCDH), 2025)

    2025

  6. [6]

    Jiet al., Survey of hallucination in natural language generation, ACM Computing Surveys55, 1 (2023)

    Z. Jiet al., Survey of hallucination in natural language generation, ACM Computing Surveys55, 1 (2023)

    2023

  7. [8]

    Avianca, Inc., No

    Mata v. Avianca, Inc., No. 22-cv-1461 (PKC), 2023 WL 4114965 (S.D.N.Y. June 22, 2023) (2023), United States District Court, S.D. New York

    2023

  8. [9]

    Crawford and T

    A. Crawford and T. Glatard, Urgent considerations for suicide prevention in the safe and ethical use of artificial intelligence, Canadian Medical Association Journal198, E599 (2026)

    2026

  9. [10]

    Bommasaniet al., On the opportunities and risks of foundation models (2021), arXiv:2108.07258

    R. Bommasaniet al., On the opportunities and risks of foundation models (2021), arXiv:2108.07258

    Pith/arXiv arXiv 2021

  10. [11]

    B. Pierson, Mother sues AI chatbot company Char- acter.AI, Google over son’s suicide, Reuters (2024), https://www.reuters.com/legal/mother-sues-ai-c hatbot-company-characterai-google-sued-over-son s-suicide-2024-10-23/

    2024

  11. [12]

    Center for Countering Digital Hate,Killer Apps: How mainstream AI chatbots assist users planning violent at- tacks, CCDH Report (Center for Countering Digital Hate (CCDH), 2026)

    2026

  12. [13]

    Henderson, T

    P. Henderson, T. Hashimoto, and M. A. Lemley, Where’s the liability in harmful AI speech? (2023), Journal of Free Speech Law, 3, pp. 589–650, arXiv:2308.04635 [cs.CY]

    Pith/arXiv arXiv 2023

  13. [14]

    Holtzman, J

    A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi, The curious case of neural text degeneration (2020), arXiv:1904.09751 [cs.CL]

    Pith/arXiv arXiv 2020

  14. [15]

    Z. Fu, W. Lam, A. M.-C. So, and B. Shi, A theoretical analysis of the repetition problem in text generation, in Proceedings of the AAAI Conference on Artificial Intelli- gence, Vol. 35 (2021) pp. 12848–12856, arXiv:2012.14660 [cs.CL]

    Pith/arXiv arXiv 2021

  15. [16]

    R. Guo, Y. Liu, G. Ma, Y. Wang, Y. Zhang, L. Xia, K. Chen, Z. Sun, and D. Shi, When less is more: The LLM scaling paradox in context compression (2026), arXiv:2602.09789 [cs.LG]

    Pith/arXiv arXiv 2026

  16. [17]

    Chen and T

    Z.-A. Chen and T. Luo, From condensation to rank col- lapse: A two-stage analysis of transformer training dynam- ics, Advances in Neural Information Processing Systems (NeurIPS) (2025), arXiv:2510.06954 [cs.LG]

    arXiv 2025

  17. [18]

    W. Sun, X. Song, P. Li, L. Yin, Y. Zheng, and S. Liu, The curse of depth in large language models (2025), Advances in Neural Information Processing Systems (NeurIPS), arXiv:2502.05795 [cs.LG]

    arXiv 2025

  18. [19]

    Dong, J.-B

    Y. Dong, J.-B. Cordonnier, and A. Loukas, Attention is not all you need: Pure attention loses rank doubly exponentially with depth (2021), Proceedings of the 38th International Conference on Machine Learning (ICML), PMLR 139, pp. 2793–2803, arXiv:2103.03404 [cs.LG]

    Pith/arXiv arXiv 2021

  19. [20]

    Avianca, Inc., No

    United States District Court for the Southern District of New York, Mata v. Avianca, Inc., No. 22-cv-1461 (PKC), 2023 WL 4114965, Federal district court sanctions order (2023), sanctions order concerning fictitious AI-generated case citations

    2023

  20. [21]

    S. Embry, Understanding AI hallucinations: Making sure you don’t end up at the wrong stop, Above the Law / 6 TechLaw Crossroads (2026), legal commentary discussing AI hallucinations as a foreseeable engineering risk

    2026

  21. [22]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, Attention Is All You Need (2023), arXiv:1706.03762 [cs.CL]

    Pith/arXiv arXiv 2023

  22. [23]

    Z. Wei, S. Wang, X. Rong, X. Liu, and H. Li, Shadows in the attention: Contextual perturbation and represen- tation drift in the dynamics of hallucination in LLMs (2025), arXiv:2505.16894 [cs.CL]

    Pith/arXiv arXiv 2025

  23. [24]

    Y. Bang, Z. Ji, A. Schelten, A. Hartshorn, T. Fowler, C. Zhang, N. Cancedda, and P. Fung, HalluLens: LLM hallucination benchmark, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), pp. 24128–24156 (2025), Vienna, Austria

    2025

  24. [25]

    Alansari and H

    A. Alansari and H. Luqman, Large language mod- els hallucination: A comprehensive survey (2025), arXiv:2510.06265 [cs.CL]

    arXiv 2025

  25. [26]

    Nanda, L

    N. Nanda, L. Chan, T. Lieberum, J. Smith, and J. Stein- hardt, Progress measures for grokking via mechanistic interpretability, International Conference on Learning Representations (2023), arXiv:2301.05217 [cs.LG]

    Pith/arXiv arXiv 2023

  26. [27]

    Nanda and T

    N. Nanda and T. Lieberum, A mechanistic interpretability analysis of grokking (2022), https://www.alignmentf orum.org/posts/N6WM6hs7RQMKDhYjB/a-mechanistic -interpretability-analysis-of-grokking Accessed: 2024-05-07

    2022

  27. [28]

    Nanda, Paper replication walkthrough: Reverse- engineering modular addition, https://www.neelnanda

    N. Nanda, Paper replication walkthrough: Reverse- engineering modular addition, https://www.neelnanda. io/mechanistic-interpretability/modular-additio n-walkthrough(2023), Accessed: 2024-05-07

    2023

  28. [29]

    Luger, H

    R. Luger, H. Kamath, D. Finkbeiner, P. Goel, A. Jermyn, S. Zimmerman, J. Batson, and T. Conerly, HeadVis: An interactive tool for investigating attention heads (2026), Transformer Circuits Thread. https://transformer-cir cuits.pub/2026/headvis/index.html Accessed: 2026- 06-17

    2026

  29. [30]

    Cunningham, A

    H. Cunningham, A. Ewart, L. Riggs, R. Huben, and L. Sharkey, Sparse autoencoders find highly inter- pretable features in language models (2023), International Conference on Learning Representations (ICLR) 2024, arXiv:2309.08600 [cs.LG]

    Pith/arXiv arXiv 2023

  30. [31]

    Ameisen, J

    E. Ameisen, J. Lindsey, A. Pearce, W. Gurnee, N. L. Turner, B. Chen, C. Citro, D. Abrahams, S. Carter, B. Hosmer, J. Marcus, M. Sklar, A. Templeton, T. Bricken, C. McDougall, H. Cunningham, T. Henighan, A. Jermyn, A. Jones, A. Persic, Z. Qi, T. B. Thompson, S. Zimmer- man, K. Rivoire, T. Conerly, C. Olah, and J. Batson, Circuit tracing: Revealing computat...

    2025

  31. [32]

    Gurnee, N

    W. Gurnee, N. Sofroniew, A. Pearce, M. Piotrowski, I. Kauvar, R. Chen, A. Soligo, P. Bogdan, E. Ong, R. Wang, T. B. Thompson, D. Abrahams, S. Kantam- neni, E. Ameisen, J. Batson, and J. Lindsey, Verbalizable representations form a global workspace in language mod- els,Transformer Circuits Thread(2026), https://tran sformer-circuits.pub/2026/workspace/inde...

    2026

  32. [33]

    Geshkovski, C

    B. Geshkovski, C. Letrouit, Y. Polyanskiy, and P. Rigol- let, The emergence of clusters in self-attention dynamics, inAdvances in Neural Information Processing Systems (NeurIPS), Vol. 36 (2023) arXiv:2305.05465 [cs.LG]

    Pith/arXiv arXiv 2023

  33. [34]

    Rigollet, The mean-field dynamics of transformers (2026), to appear in Proceedings of the ICM 2026, arXiv:2512.01868 [cs.LG]

    P. Rigollet, The mean-field dynamics of transformers (2026), to appear in Proceedings of the ICM 2026, arXiv:2512.01868 [cs.LG]

    arXiv 2026

  34. [35]

    Koubbi, B

    H. Koubbi, B. Geshkovski, and P. Rigollet, Homogenized transformers (2026), arXiv:2604.01978 [math.PR]

    arXiv 2026

  35. [36]

    Nakaishi, Y

    K. Nakaishi, Y. Nishikawa, and K. Hukushima, Phase transition in large language models and the criticality of natural languages (2024), arXiv:2406.05335 [cond-mat.dis- nn]

    Pith/arXiv arXiv 2024

  36. [37]

    Sun and B

    Y. Sun and B. Haghighat, Phase transitions in large language models and the O(N ) model (2025), arXiv:2501.16241 [cs.LG]

    Pith/arXiv arXiv 2025

  37. [38]

    Bhattacharjee and S.-C

    S. Bhattacharjee and S.-C. Lee, Testing the spin-bath view of self-attention: A Hamiltonian analysis of GPT-2 transformer, Phys. Rev. E113, 015308 (2026)

    2026

  38. [39]

    Arditi, O

    A. Arditi, O. Obeso, A. Syed, D. Paleka, N. Panickssery, W. Gurnee, and N. Nanda, Refusal in language models is mediated by a single direction (2024), Advances in Neural Information Processing Systems (NeurIPS) 37, arXiv:2406.11717 [cs.LG]

    Pith/arXiv arXiv 2024

  39. [40]

    A. Zou, L. Phan, S. Chen, J. Campbell, P. Guo, R. Ren, A. Pan, X. Yin, M. Mazeika, A.-K. Dombrowski, S. Goel, N. Li, M. J. Byun, Z. Wang, A. Mallen, S. Basart, S. Koyejo, D. Song, M. Fredrikson, J. Z. Kolter, and D. Hendrycks, Representation engineering: A top-down approach to AI transparency (2023), arXiv:2310.01405 [cs.LG]

    Pith/arXiv arXiv 2023

  40. [41]

    N. Ri, A. Panigrahi, and S. Arora, Do thinking tokens help with safety? (2026), arXiv:2606.25013 [cs.LG]

    Pith/arXiv arXiv 2026

  41. [42]

    F. Y. Huo and N. F. Johnson, Physics of generative AI’s atom: Repetition, bias, and beyond, AIP Advances16, 035305 (2026)

    2026

  42. [43]

    N. F. Johnson and F. Y. Huo, Simple picture of how output from ChatGPT-like AI shifts from good to bad, PNAS Nexus5, pgag148 (2026)

    2026

  43. [44]

    N. F. Johnson and F. Y. Huo, Fusion-fission fore- casts when AI will shift to undesirable behavior (2026), arXiv:2605.14218 [cs.AI]

    Pith/arXiv arXiv 2026

  44. [45]

    Moore, A

    J. Moore, A. Mehta, W. Agnew, J. R. Anthis, R. Louie, Y. Mai, P. Yin, M. Cheng, S. J. Paech, K. Klyman, S. Chancellor, E. Lin, N. Haber, and D. C. Ong, Charac- terizing delusional spirals through human–LLM chat logs (2026), to appear at ACM F AccT 2026, arXiv:2603.16567

    arXiv 2026

  45. [46]

    Bahdanau, K

    D. Bahdanau, K. Cho, and Y. Bengio, Neural machine translation by jointly learning to align and translate (2016), arXiv:1409.0473 [cs.CL]. END MA TTER There are three discrete clocks: layer time t acts in- side one fixed history, token-generation iteration n, and repetition m of complete blocks of sentences (see Sec. E). A. Derivation of the closed-form p...

    Pith/arXiv arXiv 2016