Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that freezing a transformer's query and key weights at random initialization still yields competitive language modeling and the formation of induction heads, and it proves that a random-mixing variant with entirely…

desk verdict Frozen query/key attention is a real and interesting empirical finding; Theorem 5.1 is a sketch, not a proof, and the experiments need error bars and full hyperparameter tables. read the letter →

arxiv 2506.01115 v3 pith:IPC4VQRJ submitted 2025-06-01 cs.LG cs.CL

Yihe Dong , Lorenzo Noci , Mikhail Khodak , Mufan Li This is my paper
classification cs.LGcs.CL MSC 68T0760H1041A65
keywords transformersfrozenattentionrandominductionheadsuniversalapproximationsignalpropagationcovarianceSDElanguagemodeling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks which parts of a transformer actually do the work by freezing pieces at initialization and testing what still works. It claims that freezing the query and key weights, leaving attention patterns random and static, still produces competitive language-modeling perplexity and still lets the model form induction heads. It also introduces MixiT, a variant whose attention scores are entirely random and input-independent, and proves its forward pass stays stable in depth and width via a covariance stochastic differential equation. The result matters because it separates learned attention from the mere presence of a mixing operation, and it suggests the transformer has an inductive bias toward forming specialized circuits even without learnable attention weights.

What carries the argument

The paper's machinery is a spectrum of frozen-component transformers built on the Llama architecture. Frozen-QK keeps the query and key weight matrices at their random initial values while training value weights, embeddings, and MLPs; this is the object whose expressiveness is captured by the universal approximation theorem. MixiT replaces the attention map itself with a fixed random matrix of the form $I + W^M_\ell - \bar W^M_\ell$, normalised so each row sums to one; this is the object whose depthwise stability is guaranteed by the covariance SDE of Theorem 2.1. The SDE describes the joint width-and-depth limit of the covariance $\Phi_\ell = \frac{1}{n} h_\ell^\top h_\ell$, with drift $\frac{1}{m}\mathrm{Tr}(\Phi_\tau) - M(\Phi_\tau)$ and explicit diffusion terms, showing that representations do not collapse as the model deepens.

What would settle it

Run the Wikitext-103 language-modeling experiment with a strictly identical hyperparameter grid across Standard and Frozen-QK and multiple random seeds; if Frozen-QK's perplexity is consistently worse than the reported competitive margin once tuning is matched, the central claim would be refuted.

Watch

Extended reading notes

Core claim

The central claim is that trainable attention weights are not required for several core transformer behaviours. A model that freezes the query and key matrices at initialization, Frozen-QK, forms induction heads during training, solves retrieval and k-hop induction tasks nearly as well as a standard transformer, and reaches log-perplexities of 3.07 on Wikitext-103 and 3.16 on Fineweb-edu versus 2.78 and 3.05 for the standard model. The paper formalizes the expressiveness of this architecture by proving that one layer of multihead attention with randomly frozen query and key weights plus a trainable MLP is a universal approximator of continuous causal functions. It further isolates attention's contribution with MixiT, in which the attention map is a frozen random matrix, proving that its covariance obeys a well-defined SDE that prevents rank collapse. MixiT matches the standard transformer on algorithmic and sentiment tasks but fails at retrieval and underperforms on language modeling, showing which tasks need input-dependent attention.

Load-bearing premise

The load-bearing premise is that all model variants were tuned to comparable effectiveness; the paper reports the selected MixiT hyperparameters but not those for Standard, Frozen-QK, or Frozen-MLP, so the headline language-modeling comparison could reflect tuning asymmetry rather than architecture.

Editorial extensions

If this is right

  • If Frozen-QK's language-modeling result holds, then learned query-key interactions are not the main driver of next-token prediction performance, and static-attention variants become attractive as simpler baselines.
  • If MixiT's stability theorem holds, random input-independent mixing can be scaled to deep models without rank collapse, making it a controlled tool for isolating what attention contributes.
  • The universal approximation result implies Frozen-QK can in principle compute any continuous causal sequence function, so its empirical limits should be attributed to optimisation, not expressiveness.
  • The task separation, with MixiT solving algorithmic and sentiment tasks but not retrieval, provides a litmus test for whether a task requires in-context reasoning and suggests that input-dependent attention is needed exactly when induction heads are required.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, Frozen-QK's competitive perplexity suggests that in large-scale training much of the value of learned attention may be redundancy or faster optimisation rather than an irreducible function; this could be tested by scaling Frozen-QK to larger models and comparing scaling curves.
  • Beyond the paper, one testable extension is to check whether Frozen-QK's induction heads are structurally identical to those of a standard transformer or whether they are different circuits that achieve the same behaviour, since the paper shows attention maps but does not fully characterise the trained mechanism.
  • Beyond the paper, the MixiT results imply that efficient attention removal could target tasks that do not need in-context retrieval, but the paper leaves open how to detect such tasks automatically before training.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper investigates how much of the Transformer's sequence-modeling performance depends on learnable attention weights. It introduces two simplified variants: Frozen-QK, which freezes query and key projections while training value, MLP, and embedding weights, and MixiT, which replaces attention scores with a fixed random input-independent mixing matrix. The central empirical claim is that Frozen-QK forms induction heads and performs nearly as well as a standard Transformer on language modeling, while MixiT matches fully trained Transformers on algorithmic tasks but fails on retrieval and induction-head tasks. The paper also proves a covariance SDE for MixiT (Theorem 2.1) and claims a universal approximation theorem for Frozen-QK (Theorem 5.1).

Significance. If the empirical results hold, they suggest that learned attention weights are not necessary for language modeling at the scale studied and that random static mixing suffices for several algorithmic tasks, which would be a significant finding for the sequence-modeling community. The MixiT architecture and its covariance SDE are a useful contribution to the design of random attention layers with stable signal propagation. The paper also provides a practical decomposition of attention versus MLP roles, and the public code supports reproducibility. However, the main theoretical claim (Theorem 5.1) is currently unsupported, and the empirical comparisons lack tuning transparency and statistical uncertainty, so the significance is conditional on addressing these issues.

major comments (4)
  1. [Appendix B, Theorem 5.1] The proof of Theorem 5.1 is a high-level sketch that does not verify the hypotheses of the cited lifting theorem. Specifically, it does not show that the deterministic transformer feature class is dense in C_causal(K,U), nor that the random feature map g_i defined in (B.1) is measurable and p-integrable, nor that the span of the Gaussian-initialized features is almost surely dense in the sup norm. Furthermore, the universality result of Yun et al. [59] is for deep transformers, whereas Theorem 5.1 claims a single layer; the proof does not explain how the depth of the deterministic construction can be reduced to one layer, nor how the number of heads and MLP width must scale. As a result, the formal expressivity argument for Frozen-QK is not established, which undermines the central claim in the abstract and Section 1.
  2. [Appendix C.1, Table 9] The text states that a grid search was run for all model variants, but Table 9 reports optimal hyperparameters only for MixiT. The selected settings for Standard, Frozen-QK, and Frozen-MLP are not disclosed for any task. This is a load-bearing omission: if the fully trainable baseline was tuned less carefully than Frozen-QK, the language modeling results in Table 2 (and the algorithmic results in Tables 3 and 4) could reflect tuning asymmetry rather than architectural properties. The authors should provide the selected hyperparameters for all variants, or at least state that the same hyperparameters were used across variants for each task.
  3. [Section 3.2, Tables 1–4] No repeated runs or error bars are reported. Since the headline comparisons involve small differences (e.g., log perplexities 3.05 vs 3.16 on Fineweb-edu in Table 2, and accuracies differing by a few percent in Table 3), single-run results are insufficient to support claims of competitiveness. The authors should report means and standard deviations over at least three seeds for the main comparisons, or otherwise justify why the differences are robust.
  4. [Appendix A, Theorem 2.1] The convergence result for MixiT is stated with n,d → ∞ while the sequence length m is not explicitly held fixed; this should be stated. The proof relies on [31, Prop. A.6] but does not verify the regularity conditions (e.g., Lipschitz continuity of the drift and diffusion coefficients) nor the positive semidefiniteness of the diffusion matrix appearing in the square root in (2.4). This theorem underlies the MixiT design, so the proof needs to be completed or the statement qualified.
minor comments (6)
  1. [Section 4, Table 4] The text says 'Most of the drop occurs in the Frozen-QK model, with 1.13 bits per parameters, while Frozen-MLP and MixiT have similar storage capabilities at 2.25 and 2.18', but the table reports Frozen-MLP at 1.13 and Frozen-QK at 2.25. The text and table are inconsistent; please correct the reference.
  2. [Section 1] 'partiucular' is a typo for 'particular'.
  3. [Eq. (2.3)] The notation for W^M_{ij} ~ N(0, 1/sqrt(nm)) is ambiguous; the proof in Appendix A uses W with unit variance and a pre-factor, which should be reconciled.
  4. [Appendix B, Eq. (B.1)] The dimensions of the attention pattern are not specified; clarify that Softmax is applied row-wise and that the causal mask M_causal is added, and specify the shape of the mask.
  5. [Section 5, after Theorem 5.1] 'the random feature gi = x Softmax(Q_k^T K_k) in MixiT is linear' is unclear because Q_k and K_k are not defined in that paragraph; please refer to the model definition in Section 2.
  6. [Table 5] The hidden dimension for Yelp is 1024 while for decimal addition and Dyck-1 it is 512; this difference may affect the comparison across heads, which should be acknowledged.

Circularity Check

1 steps flagged · score 2.0 of 10

No significant circularity: the empirical claims are self-contained and the theorems are genuine applications of prior frameworks, with one minor self-citation and a Theorem 5.1 proof gap that is a correctness concern rather than a circular reduction.

  1. other [Appendix A, proof of Theorem 2.1 (MixiT Covariance SDE); Section 2, MixiT definition]
    "To complete the proof, we will invoke the Markov chain convergence to SDE results in the Skorohod topology, see for example Li et al. [31, Proposition A.6], which gives us the desired result."

    This is a minor self-citation, not a circular reduction. Theorem 2.1's proof is completed by citing Proposition A.6 of Li et al. [31], prior work co-authored by M. Li, and the MixiT mixing matrix is justified by 'attention shaping [38]', also co-authored by Noci and Li. However, the cited proposition is a general Markov-chain convergence statement and the paper analyzes the specific MixiT recursion directly in equations (A.4)-(A.12), so the theorem is a genuine application of an external framework rather than being defined in terms of its own conclusion. It does not force the empirical predictions, and the central Frozen-QK claims do not depend on it.

full rationale

The paper's central empirical claims—Frozen-QK performing competitively on language modeling, forming induction heads, and MixiT succeeding at memorization-style tasks while failing at retrieval—are measurements on trained models with no fitted constants in the claims; there is no step where a parameter fit to a subset of data is renamed a prediction. The theory is also largely external. Theorem 2.1 is derived by applying the general covariance-SDE framework of Li et al. [31] and attention shaping [38], both co-authored by some of the present authors; this is a real but minor self-citation because the MixiT recursion is analyzed explicitly and the cited result is a general proposition, so the derivation is not equivalent to its input. Theorem 5.1's Appendix B proof is a sketch that invokes Yun et al. [59] and Neufeld and Schmocker [36, Thm 3.2] without verifying the hypotheses of the lifting theorem for the specific random softmax features g_i, and without reconciling the one-layer claim with the deep universality of [59]; this is a load-bearing proof gap for the formal expressivity statement, but it is an unverified citation/lifting step, not an equivalence-by-construction or a fit-renamed-as-prediction, so it is a correctness risk and does not raise the circularity score. Appendix C.1 also reports optimal hyperparameters only for MixiT, so tuning asymmetry is a possible confound for the competitive language-modeling comparison, but this too is an experimental-validity concern rather than circularity. Overall, no significant circularity is present.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper's theoretical theorems rely on two external results: Markov chain convergence to SDE (Li et al.) and universal approximation lifting (Neufeld & Schmocker). The latter is applied without verifying its conditions. The empirical results use no fitted constants; hyperparameters are experimental settings, not free parameters.

assumptions (3)
  • standard math Markov chain covariance convergence to SDE in Skorohod topology (Li et al. [31, Prop A.6])
    Used in Appendix A to conclude the MixiT covariance recursion converges to the claimed SDE; the paper cites rather than re-proves the approximation.
  • standard math Universal approximation of continuous causal functions by standard transformers (Yun et al., 2020)
    Lifted in Theorem 5.1; the paper relies on this background result.
  • domain assumption Random-feature universal approximation lifting theorem of Neufeld and Schmocker [36, Thm 3.2] applies to the Softmax attention random features with frozen Q,K
    The paper assumes the conditions of the lifting theorem hold for its attention-based random features without verifying the required distributional or architectural assumptions. This is the weakest link in Theorem 5.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer." pith.science (2026). https://pith.science/paper/IPC4VQRJ

@misc{pith2026250601115,
  author       = {Pith},
  title        = {Pith review of: Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IPC4VQRJ}},
  note         = {Machine review of arXiv:2506.01115}
}
read the original abstract

The transformer architecture is central to the success of modern Large Language Models (LLMs), in part due to its surprising ability to perform a wide range of tasks - including mathematical reasoning, memorization, and retrieval - using only gradient-based learning on next-token prediction. While the core component of a transformer is the self-attention mechanism, we question how much, and which aspects, of the performance gains can be attributed to it. To this end, we compare standard transformers to variants in which either the MLP layers or the attention weights are frozen at initialization. Surprisingly, we find that attention with frozen key and query weights is not only able to form induction heads, but can also perform competitively on language modeling. We formalize this by proving a new expressivity result for transformer models with frozen key and query weights. To further isolate the contribution of attention, we design MixiT, an architecture with entirely random attention scores, with provably stable signal propagation that overcomes prior depth-wise scaling challenges in random transformers. We use the successes and failures of MixiT to understand the role each transformer component plays, such as attention being largely responsible for in-context reasoning, and MLPs being responsible for, but collaborates with attention, on knowledge storage. Our results suggest that the transformer architecture has a built-in inductive bias towards forming specialized circuits, as it does even without learnable attention weights.

Figures

Figures reproduced from arXiv: 2506.01115 by the authors.

Figure 1
Figure 1. Variants of the Llama Transformer model that we study. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Retrieval accuracy as a function of the number of pairs in the sequence [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. The Frozen-QK model can solve the retrieval task by forming an induction head. In the first head, [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Attractor Geometry of Transformer Memory: From Conflict Arbitration to Confident Hallucination

    cs.AI 2026-05 unverdicted novelty 7.0 of 10

    Transformer hidden states encode facts as attractor basins; hallucinations occur from basin absence and conflicts from basin competition, detected cleanly by geometric margin rather than entropy.

  2. DTRNet: Dynamic Token Routing Network to Reduce Quadratic Costs in Transformers

    cs.LG 2025-08 conditional novelty 6.0 of 10

    A learned router sends about 90% of tokens through a linear projection path instead of quadratic attention, keeps the MLP for every token, and matches dense Transformer accuracy at roughly 85% of the FLOPs.

  3. Procedural Pretraining: Warming Up Language Models with Abstract Data

    cs.CL 2026-01 conditional novelty 5.0 of 10

    A short warm-up on procedural data (brackets, sorting, sets) makes language models more accurate and more data-efficient on language, code, and informal math.

Reference graph

Works this paper leans on

63 extracted references · 26 canonical work pages · cited by 3 Pith papers

  1. [59]

    Are transformers universal approximators of sequence-to-sequence functions? InInternational Conference on Learning Representations, 2020

    Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J Reddi, and Sanjiv Kumar. Are transformers universal approximators of sequence-to-sequence functions? InInternational Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=ByxRM0Ntvr

    work page 2020

  2. [1]

    Physics of language models: Part 3.3, knowledge capacity scaling laws

    Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.3, knowledge capacity scaling laws. arXiv preprint arXiv:2404.05405, 2024

    arXiv 2024

  3. [2]

    The Curious Case of Benign Memorization

    Sotiris Anagnostidis, Gregor Bachmann, Lorenzo Noci, and Thomas Hofmann. The curious case of benign memorization. arXiv preprint arXiv:2210.14019, 2022

    work page Pith review arXiv 2022

  4. [3]

    Du, Wei Hu, Zhiyuan Li, Ruslan Salakhutdinov, and Ruosong Wang.On exact computation with an infinitely wide neural net

    Sanjeev Arora, Simon S. Du, Wei Hu, Zhiyuan Li, Ruslan Salakhutdinov, and Ruosong Wang.On exact computation with an infinitely wide neural net. Curran Associates Inc., Red Hook, NY, USA, 2019

    work page 2019

  5. [4]

    A closer look at memorization in deep networks

    Devansh Arpit, Stanisław Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization in deep networks. InInternational conference on machine learning, pages 233–242. PMLR, 2017. 11

    work page 2017

  6. [5]

    Scaling mlps: A tale of inductive bias

    Gregor Bachmann, Sotiris Anagnostidis, and Thomas Hofmann. Scaling mlps: A tale of inductive bias. Advances in Neural Information Processing Systems, 36:60821–60840, 2023

    work page 2023

  7. [6]

    Mechanistic interpretability for AI safety - a review.Transactions on Machine Learning Research, 2024

    Leonard Bereska and Stratis Gavves. Mechanistic interpretability for AI safety - a review.Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id= ePUVetPKu6. Survey Certification, Expert Certification

    2024

  8. [7]

    Birth of a transformer: A memory viewpoint.Advances in Neural Information Processing Systems, 36:1560–1588, 2023

    Alberto Bietti, Vivien Cabannes, Diane Bouchacourt, Herve Jegou, and Leon Bottou. Birth of a transformer: A memory viewpoint.Advances in Neural Information Processing Systems, 36:1560–1588, 2023

    work page 2023

Show all 63 references
  1. [8]

    Frozen layers: Memory-efficient many-fidelity hyperparameter optimization.arXiv preprint arXiv:2504.10735, 2025

    Timur Carstensen, Neeratyoy Mallik, Frank Hutter, and Martin Rapp. Frozen layers: Memory-efficient many-fidelity hyperparameter optimization.arXiv preprint arXiv:2504.10735, 2025

    2025 arXiv

  2. [9]

    Transformers generalize differently from information stored in context vs in weights.arXiv preprint arXiv:2210.05675, 2022

    Stephanie CY Chan, Ishita Dasgupta, Junkyung Kim, Dharshan Kumaran, Andrew K Lampinen, and Felix Hill. Transformers generalize differently from information stored in context vs in weights.arXiv preprint arXiv:2210.05675, 2022

    arXiv

  3. [10]

    Distributional associations vs in-context reasoning: A study of feed-forward and attention layers.ICLR, 2024

    Lei Chen, Joan Bruna, and Alberto Bietti. Distributional associations vs in-context reasoning: A study of feed-forward and attention layers.ICLR, 2024

    2024

  4. [11]

    Knowledge localization: Mission not accomplished? enter query localization! InProceedings of the Thirteenth International Conference on Learning Representations (ICLR), 2025

    Yuheng Chen, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. Knowledge localization: Mission not accomplished? enter query localization! InProceedings of the Thirteenth International Conference on Learning Representations (ICLR), 2025. URL https://openreview.net/forum?id=tfyHbvFZ0K

    2025

  5. [12]

    Summing up the facts: Additive mechanisms behind factual recall in llms, 2024

    Bilal Chughtai, Alan Cooney, and Neel Nanda. Summing up the facts: Additive mechanisms behind factual recall in llms, 2024. Preprint

    2024

  6. [13]

    Induction heads as an essential mechanism for pattern matching in in-context learning

    Joy Crosbie and Ekaterina Shutova. Induction heads as an essential mechanism for pattern matching in in-context learning. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors,Findings of the Association for Computational Linguistics: NAACL 2025, pages 5034–5096, Albuquerque, Ne...

    2025

  7. [14]

    Knowledge neurons in pretrained transformers

    Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (V...

    2022 doi

  8. [15]

    Attention is not all you need: Pure attention loses rank doubly exponentially with depth

    Yihe Dong, Jean-Baptiste Cordonnier, and Andreas Loukas. Attention is not all you need: Pure attention loses rank doubly exponentially with depth. InInternational conference on machine learning, pages 2793–2803. PMLR, 2021

    2021

  9. [16]

    An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:...

    2010 arXiv

  10. [17]

    Edelman, eran malach, and Surbhi Goel

    Ezra Edelman, Nikolaos Tsilivis, Benjamin L. Edelman, eran malach, and Surbhi Goel. The evolution of statistical induction heads: In-context learning markov chains. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net...

    2024

  11. [18]

    A mathematical framework for transformer circuits.Transformer Circuits Thread, 2021

    Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield- Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dar...

    2021

  12. [19]

    Transformer feed-forward layers are key-value memories.arXiv preprint arXiv:2012.14913, 2020

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories.arXiv preprint arXiv:2012.14913, 2020

    2012 arXiv

  13. [20]

    Transformer feed-forward layers are key-value memories

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wentau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Pr...

    2021 doi

  14. [21]

    Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space

    Mor Geva, Avi Caciularu, Kevin Wang, and Yoav Goldberg. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors,Proceedings of the 2022 Conference on Empirical Methods in Natur...

    2022 doi

  15. [22]

    Dissectingrecalloffactualassociations inauto-regressivelanguagemodels

    MorGeva, JasmijnBastings, KatjaFilippova, andAmirGloberson. Dissectingrecalloffactualassociations inauto-regressivelanguagemodels. InHoudaBouamor, JuanPino, andKalikaBali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12...

    2023 doi

  16. [23]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  17. [24]

    Smith, and Roy Schwartz

    Michael Hassid, Hao Peng, Daniel Rotem, Jungo Kasai, Ivan Montero, Noah A. Smith, and Roy Schwartz. How much does attention actually attend? questioning the importance of attention in pretrained transformers, 2022

    2022

  18. [25]

    Simplifying transformer blocks.arXiv preprint arXiv:2311.01906, 2023

    Bobby He and Thomas Hofmann. Simplifying transformer blocks.arXiv preprint arXiv:2311.01906, 2023

    2023 arXiv

  19. [26]

    Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks.arXiv preprint arXiv:2311.12786, 2023

    Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P Dick, Hidenori Tanaka, Edward Grefenstette, Tim Rocktäschel, and David Scott Krueger. Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks.arXiv preprint arXiv:2311.12786, 2023

    2023 arXiv

  20. [27]

    What is the best multi-stage architecture for object recognition? In2009 IEEE 12th International Conference on Computer Vision, pages 2146–2153, 2009

    Kevin Jarrett, Koray Kavukcuoglu, Marc’Aurelio Ranzato, and Yann LeCun. What is the best multi-stage architecture for object recognition? In2009 IEEE 12th International Conference on Computer Vision, pages 2146–2153, 2009. doi: 10.1109/ICCV.2009.5459469

    2009

  21. [28]

    Lexico: Extreme kv cache compression via sparse coding over universal dictionaries, 2024

    Junhyuck Kim, Jongho Park, Jaewoong Cho, and Dimitris Papailiopoulos. Lexico: Extreme kv cache compression via sparse coding over universal dictionaries, 2024. URLhttps://arxiv.org/abs/2412. 08890

    2024

  22. [29]

    Deep neural networks as gaussian processes.arXiv preprint arXiv:1711.00165, 2017

    Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel S Schoenholz, Jeffrey Pennington, and Jascha Sohl-Dickstein. Deep neural networks as gaussian processes.arXiv preprint arXiv:1711.00165, 2017

    2017 arXiv

  23. [30]

    Fnet: Mixing tokens with fourier transforms

    James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, and Santiago Ontanon. Fnet: Mixing tokens with fourier transforms. arXiv preprint arXiv:2105.03824, 2021

    2021 arXiv

  24. [31]

    The neural covariance sde: Shaped infinite depth-and-width networks at initialization.Advances in Neural Information Processing Systems, 35:10795–10808, 2022

    Mufan Li, Mihai Nica, and Dan Roy. The neural covariance sde: Shaped infinite depth-and-width networks at initialization.Advances in Neural Information Processing Systems, 35:10795–10808, 2022

    2022

  25. [32]

    Rapid training of deep neural networks without skip connections or normalization layers using deep kernel shaping.arXiv preprint arXiv:2110.01765, 2021

    James Martens, Andy Ballard, Guillaume Desjardins, Grzegorz Swirszcz, Valentin Dalibard, Jascha Sohl- Dickstein, and Samuel S Schoenholz. Rapid training of deep neural networks without skip connections or normalization layers using deep kernel shaping.arXiv preprint arXiv:2110...

    arXiv

  26. [33]

    Locating and editing factual knowledge in gpt

    Kevin Meng, David Bau, Michael Andrus, Aitor Belrose, Alex Andonian, Catherine Olsson, Sam McCandlish, and Dario Amodei. Locating and editing factual knowledge in gpt. arXiv preprint arXiv:2202.05262, 2022. 13

    2022 arXiv

  27. [34]

    Pointer sentinel mixture models

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843, 2016. URL https://arxiv.org/abs/1609.07843

    2016 arXiv

  28. [35]

    Language models implement simple word2vec-style vector arithmetic, 2024

    Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. Language models implement simple word2vec-style vector arithmetic, 2024. Preprint

    2024

  29. [36]

    Universal approximation property of random neural networks

    Ariel Neufeld and Philipp Schmocker. Universal approximation property of random neural networks. 2023

    2023

  30. [37]

    Signal propagation in transformers: Theoretical perspectives and the role of rank collapse.Advances in Neural Information Processing Systems, 35:27198–27211, 2022

    Lorenzo Noci, Sotiris Anagnostidis, Luca Biggio, Antonio Orvieto, Sidak Pal Singh, and Aurelien Lucchi. Signal propagation in transformers: Theoretical perspectives and the role of rank collapse.Advances in Neural Information Processing Systems, 35:27198–27211, 2022

    2022

  31. [38]

    The shaped transformer: Attention models in the infinite depth-and-width limit.Advances in Neural Information Processing Systems, 36:54250–54281, 2023

    Lorenzo Noci, Chuning Li, Mufan Li, Bobby He, Thomas Hofmann, Chris J Maddison, and Dan Roy. The shaped transformer: Attention models in the infinite depth-and-width limit.Advances in Neural Information Processing Systems, 36:54250–54281, 2023

    2023

  32. [39]

    Investigating the limitations of transformers with simple arithmetic tasks.arXiv preprint arXiv:2102.13019, 2021

    Rodrigo Nogueira, Zhiying Jiang, and Jimmy Lin. Investigating the limitations of transformers with simple arithmetic tasks.arXiv preprint arXiv:2102.13019, 2021

    2021 arXiv

  33. [40]

    In-context learning and induction heads.arXiv preprint arXiv:2209.11895, 2022

    Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context learning and induction heads.arXiv preprint arXiv:2209.11895, 2022

    2022 arXiv

  34. [41]

    The fineweb datasets: Decanting the web for the finest text data at scale

    Guilherme Penedo, Hynek Kydlíček, Loubna Ben Allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro von Werra, and Thomas Wolf. The fineweb datasets: Decanting the web for the finest text data at scale. arXiv preprint arXiv:2406.17557, 2024. URL https://arxiv.org/abs/2...

    2024 arXiv

  35. [42]

    Mechanistic design and scaling of hybrid architectures

    Michael Poli, Armin W Thomas, Eric Nguyen, Pragaash Ponnusamy, Björn Deiseroth, Kristian Kersting, Taiji Suzuki, Brian Hie, Stefano Ermon, Christopher Ré, et al. Mechanistic design and scaling of hybrid architectures. arXiv preprint arXiv:2403.17844, 2024

    2024 arXiv

  36. [43]

    Exponential expressivity in deep neural networks through transient chaos.Advances in neural information processing systems, 29, 2016

    Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein, and Surya Ganguli. Exponential expressivity in deep neural networks through transient chaos.Advances in neural information processing systems, 29, 2016

    2016

  37. [44]

    Compositional capabilities of autoregressive transformers: A study on synthetic, interpretable tasks.arXiv preprint arXiv:2311.12997, 2023

    Rahul Ramesh, Ekdeep Singh Lubana, Mikail Khona, Robert P Dick, and Hidenori Tanaka. Compositional capabilities of autoregressive transformers: A study on synthetic, interpretable tasks.arXiv preprint arXiv:2311.12997, 2023

    2023 arXiv

  38. [45]

    Transformers, parallel computation, and logarithmic depth

    Clayton Sanford, Daniel Hsu, and Matus Telgarsky. Transformers, parallel computation, and logarithmic depth. CoRR, abs/2402.09268, 2024. URLhttps://arxiv.org/abs/2402.09268

    2024 arXiv

  39. [46]

    Saxe, Pang Wei Koh, Zhenghao Chen, Maneesh Bhand, Bipin Suresh, and Andrew Y

    Andrew M. Saxe, Pang Wei Koh, Zhenghao Chen, Maneesh Bhand, Bipin Suresh, and Andrew Y. Ng. On random weights and unsupervised feature learning. InProceedings of the 28th International Conference on International Conference on Machine Learning, ICML’11, page 1089–1096, Madison...

    2011

  40. [47]

    Deep information propagation

    Samuel S Schoenholz, Justin Gilmer, Surya Ganguli, and Jascha Sohl-Dickstein. Deep information propagation. arXiv preprint arXiv:1611.01232, 2016

    2016 arXiv

  41. [48]

    Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024

    2024

  42. [49]

    Synthesizer: Rethinking self-attention in transformer models

    Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler. Synthesizer: Rethinking self-attention in transformer models. International Conference on Machine Learning, 2021

    2021

  43. [50]

    Efficient transformers: A survey, 2022

    Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler. Efficient transformers: A survey, 2022. URL https://arxiv.org/abs/2009.06732. 14

    2022 arXiv

  44. [51]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  45. [52]

    Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  46. [53]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017

    2017

  47. [54]

    Efficient streaming language models with attention sinks

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. InThe Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=NG7sS51zVF

    2024

  48. [55]

    On layer normalization in the transformer architecture

    Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, and Tieyan Liu. On layer normalization in the transformer architecture. InInternational conference on machine learning, pages 10524–10533. PMLR, 2020

    2020

  49. [56]

    Mean field residual networks: On the edge of chaos.Advances in neural information processing systems, 30, 2017

    Ge Yang and Samuel Schoenholz. Mean field residual networks: On the edge of chaos.Advances in neural information processing systems, 30, 2017

    2017

  50. [57]

    Knowledge circuits in pretrained transformers

    Yunzhi Yao, Ningyu Zhang, Zekun Xi, Mengru Wang, Ziwen Xu, Shumin Deng, and Huajun Chen. Knowledge circuits in pretrained transformers. InAdvances in Neural Information Processing Systems 37 (NeurIPS 2024), 2024. URL https://arxiv.org/abs/2405.17969

    2024 arXiv

  51. [58]

    Locating factual knowledge in large language models: Exploring the residual stream and analyzing subvalues in vocabulary space, 2024

    Zeping Yu and Sophia Ananiadou. Locating factual knowledge in large language models: Exploring the residual stream and analyzing subvalues in vocabulary space, 2024. Preprint

    2024

  52. [60]

    Understanding deep learning requires rethinking generalization.arXiv preprint arXiv:1611.03530, 2016

    Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization.arXiv preprint arXiv:1611.03530, 2016

    2016 arXiv

  53. [61]

    Deep learning without shortcuts: Shaping the kernel with tailored rectifiers.arXiv preprint arXiv:2203.08120, 2022

    Guodong Zhang, Aleksandar Botev, and James Martens. Deep learning without shortcuts: Shaping the kernel with tailored rectifiers.arXiv preprint arXiv:2203.08120, 2022

    2022 arXiv

  54. [62]

    Character-level convolutional networks for text classification

    Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. arXiv preprint arXiv:1509.01626, 2015. URL https://arxiv.org/abs/1509.01626

    2015 arXiv

  55. [63]

    Algorithmic capabilities of random transformers

    Ziqian Zhong and Jacob Andreas. Algorithmic capabilities of random transformers. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/ forum?id=plH8gW7tPQ. 15 Appendix A Proof of Theorem 2.1 Theorem (MixiT Covariance ...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.