Pith. sign in

REVIEW 2 major objections 4 minor 72 references

Rethinking Associative Memory Mechanism in Induction Head

T0 review · 2 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Relative positional encoding makes a two-layer transformer's induction head position-independent, and its logits come out as a weighted blend of pretrained bigram knowledge and in-context pattern counts.

desk verdict Worth a serious look, but the central RPE theorem has a matrix-orientation error in its stated form; the proof uses the transposed matrix, so the main claim needs a fix before it is citable. read the letter →

arxiv 2412.11459 v2 pith:56Q5RCAR submitted 2024-12-16 cs.CL cs.LG

Shuo Wang , Issei Sato This is my paper · ORCID
classification cs.CLcs.LG
keywords inductionheadin-contextlearningassociativememoryrelativepositionalencodinglengthgeneralizationbigrammodeltwo-layertransformergradient-descentdynamics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to explain two behaviours of the induction head, the attention circuit behind in-context learning, in a two-layer transformer: why it fails to use patterns that appear late in a long prompt, and how it combines pretrained bigram knowledge with patterns offered in context when predicting the next token. Framing attention and feed-forward weights as associative memories, the paper claims that relative positional encoding removes the first failure: the learned previous-token association $r_{-1}^\top W_1^K w_E(v)$ is independent of the vocabulary item $v$ and of the position $t$, whereas with absolute positional encoding the same association decays as $1/t$ and late-sequence patterns are skipped. For the second question, it claims the next-token logits are a weighted sum of the pretrained bigram log-probability and the in-context frequency of trigger--output patterns, with pattern positions dropping out of the formula. These two claims turn vague intuitions about context hijacking and long-context attention into concrete equations that can be checked directly on trained weights and outputs.

What carries the argument

The carrier of the argument is the associative-memory representation of weight matrices, $W = \sum_{i,j} \alpha_{i,j} u_i v_j^\top$, read off through inner products $u_i^\top W v_j$ under a near-orthogonality assumption on random embeddings. The central identity is the score $r_{-1}^\top W_1^K w_E(v)$, which pairs the relative vector of the immediately preceding position with any token embedding; Theorem 2 shows this quantity is independent of both $v$ and the position $t$, so the first layer forms a position-independent previous-token head of the form $W_1^K = \sum_{k \in Q} w_E(k) r_{-1}^\top$. A second piece is the key--value construction of the feed-forward block, whose keys detect token embeddings and whose values store the global distribution $\log \pi_b(u|v)$; scaling the three memories by coefficients $\tau_1, \tau_2, \tau_3$ yields the logit formula of Proposition 3. Both results rest on a sequential one-step gradient-descent analysis that trains $W_2^O$, $W_2^K$, and $W_1^K$ in that order, with all other weights frozen.

What would settle it

Train a two-layer transformer with relative positional encoding end-to-end, updating all weights jointly, on the bigram-with-trigger task, and measure the memory-recall score for the association $r_{-1}^\top W_1^K w_E(v)$ separately for early and late positions; if recall for positions $t > 128$ falls below recall for early positions, the claimed position independence is false.

Watch

Extended reading notes

Core claim

The paper's central claim is that the way positions enter the first attention layer changes what gradient descent writes into the key matrix. Following a prior associative-memory analysis, the paper studies a sequential training procedure in which $W_2^O$, then $W_2^K$, then $W_1^K$ each receive one gradient step from zero initialization, on sequences generated by a bigram model with trigger tokens. For relative positional encoding, the learned first-layer matrix becomes an associative memory pairing each token embedding $w_E(v)$ with the fixed previous-position vector $r_{-1}$, and the paper proves (Theorem 2 in the text, Theorem 3 in the appendix) that the score $r_{-1}^\top W_1^K w_E(v)$ carries no dependence on $v$ or on $t$; the previous-token head therefore attends with constant strength at every sequence position, unlike the absolute-position case where the same score decays as $1/t$. The paper then constructs an explicit associative-memory transformer and shows its logits equal a weighted blend of the global bigram log-probability and the in-context counts of trigger--output pairs (Propositions 2 and 3), so where a pattern occurred in the prompt does not influence the final prediction. Experiments on a character-level bigram task and on an analogy task built from capital--country pairs are reported as evidence that trained networks behave this way.

Load-bearing premise

The proof that the previous-token association is position-independent trains the three weight matrices one at a time, from zero initialization, with a single gradient step each, and only on sequences where one trigger token appears exactly twice and ends the sequence; if ordinary end-to-end training does not follow this sequential path, the derived form of $W_1^K$ and the position-independence claim need not hold.

Editorial extensions

If this is right

  • A transformer with relative positional encoding should keep its previous-token head working at every position of a sequence, including positions beyond the training length, while an absolute-position transformer's head weakens; this is exactly the length-generalization behaviour the memory-recall experiments measure.
  • At the final trigger token of a prompt, the logit for a vocabulary item is the sum of a global term proportional to $\log \pi_b(v|q)$ and an in-context term proportional to the frequency of the pattern $q v$, so the prediction shifts as pattern counts shift.
  • Positional information about where in the prompt a pattern occurred drops out of the final logit; only the frequencies of trigger--output pairs matter, by Proposition 3.
  • When the prompt contains out-of-distribution tokens, the learned induction head does not activate and the relative-encoding advantage disappears, as the paper notes in Remark 1.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension of the same mechanism: other relative-position schemes, such as rotary or linear-bias encodings, should also show flat memory recall at long positions if the position-independence is driven by the relative key structure rather than by the specific fixed-vector encoding used here.
  • The additive logit decomposition offers a quantitative diagnostic for context hijacking in larger models: if a misleading pattern's count shifts the output according to the same weighted-sum law, global and in-context knowledge compete additively, and the ratio $\tau_1/\tau_2$ measures how strongly pretraining resists the prompt.
  • Read backwards, the result locates absolute-position length-generalization failures inside the positional attention score itself, suggesting that making the score translation-invariant should be sufficient to restore late-sequence pattern use.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. This paper analyzes a two-layer attention-only transformer with relative positional encoding (RPE), trained on bigram sequences with triggered transitions. The main theoretical claim is that, unlike absolute positional encoding (APE), where the previous-token association decays as 1/t, RPE learns a position-independent previous-token head; a second set of results describes how the final logits combine pretrained bigram log-probabilities with in-context pattern frequencies. The analysis is carried out under an explicit simplified protocol (one gradient step per matrix, sequential training from zero initialization, loss restricted to the final token), and the experiments measure memory recall and prompt-frequency-dependent predictions.

Significance. The paper addresses a real question—why RPE transformers retain in-context information at long sequence positions and how global and in-context knowledge are combined—and it offers explicit associative-memory formulas together with controlled experiments that match the qualitative predictions. The authors are also transparent about the simplified training setup and the two-layer scope. However, the main theorem as stated has a matrix-orientation inconsistency: the formal expression for W1K in Eq. (13) makes the claimed score vanish under the paper's own near-orthogonality assumption, and the proof only obtains a nonzero value for the transposed matrix. This is fixable by choosing one consistent convention, but until then the central claim is not established. With the orientation corrected, the paper would be a useful theoretical complement to Bietti et al. and to empirical studies of positional encoding and length generalization.

major comments (2)
  1. [Appendix C.3 (Eqs. 13–14) and Theorem 2] Eq. (13) states W1K = Σ_{k∈Q} χ(k) wE(k) r_{-1}^T. Under Assumption 1, r_{-1}^T wE(v) ≈ 0 for every v, so W1K wE(v) ≈ 0 and the score r_{-1}^T W1K wE(v) claimed in Theorem 2 is identically small. The proof of Theorem 3 computes a nonzero value only for the transposed orientation (Σ χ(k) r_{-1} wE(k)^T), and the same transposed orientation is used in Proposition 4 and in the memory-recall metric of §5.1. The same issue affects Eq. (14): with W2K = Σ ψ(k) wE(k)(Φ1wE(k))^T, the quantity (Φ1wE(zs))^T W2K wE(zT) used in Step 3 is ≈0, whereas the proof evaluates the transposed matrix. Because Theorem 2 is the paper's central claim, the statement and proof must be made consistent; for example, keep Eq. (13) and state the result as r_{-1}^T W1K^T wE(v) (equivalently wE(v)^T W1K r_{-1}), or replace Eq. (13) by its transpose throughout. This is not a cosmetic issue: if implemented as written, the §5.1 memory-recall metric would be zero for all k.
  2. [Appendix C.2 and §5.1] The derivation of W1K and W2K assumes a very specific training protocol: W2O, W2K, and W1K are trained sequentially, each with one gradient descent step from zero initialization, while the loss is computed only on sequences of length T ending in the second occurrence of the trigger token and only on the final output token (Appendix C.2). The experiments in §5.1 instead train a full model end-to-end with SGD and momentum, with all attention matrices updated jointly over 1000 iterations. As a result, the experiments verify the end-state qualitative prediction but do not test the sequential one-step derivation that produces the closed form of W1K. The paper should either include an experiment that follows the Appendix C.2 protocol, or explicitly state that the training-dynamics theorem is an idealized derivation whose connection to end-to-end training is not proven.
minor comments (4)
  1. [Theorem 2] The first term of the displayed formula in Theorem 2 is written with 't · O(1)' inside a sum over t; please make explicit which error terms are uniform in t and V, since the claimed independence of t and v depends on those terms being uniformly small.
  2. [Table 2] The 'score' columns in Table 2 are not defined in the table or its caption; if they are the memory-recall values from §5.1, this should be stated explicitly.
  3. [References and typos] There are several typographical errors, including 'V on Oswald' (should be 'Von Oswald') in the references and 'encoder-ony' (should be 'encoder-only') in Appendix A.
  4. [Proposition 4 proof] In the proof of Proposition 4, the softmax weights are written using Euler's number e in expressions such as e/(t+e-1); please define this convention at the point of use, since the same symbol is also used for token indices and it is easy to confuse the base of the softmax with an index.

Circularity Check

1 steps flagged · score 4.0 of 10

Global-vs-in-context claims are partly self-definitional: Definition 2 hard-wires log pi_b into W2 and induction-head associations into W1K/W2K/W2O, then Propositions 2-3 recover those same terms in the logits; the RPE training-dynamics theorem is independent.

  1. self definitional [Section 4.2, Definition 2 and Proposition 3 (Eq. 10); see also Proposition 2 (Eq. 7)]
    "Definition 2 (associative memory transformer) ... W 1 K = P k∈Q wE(k)r⊤ −1, W1 = (...), W2 = ( P V u=1 logπ b(u|v 1)wU (u)⊤ ... ) ⊤ ... Proposition 3 ... ξv ≈τ1 τ2 logπ b(v|q) +τ 3 · f(v) +1{v=q}1{z 1 =q}(...)"

    The global-knowledge term in Proposition 3 is not an emergent quantity: Definition 2 sets W2 to store, for each input token v, the vector Σ_u logπ_b(u|v) wU(u)^T, with W1's rows detecting wE(v). The FF key-value readout therefore contributes exactly logπ_b(v|q) to the logits. The in-context frequency term f(v) is likewise the pattern-counting behavior built into W1K/W2K/W2O by Definition 2/Lemma 1. Propositions 2-3 thus recover, by a calculation, the same quantities placed into the definition; they characterize an explicitly constructed model rather than predicting its global/in-context tradeoff from training dynamics. The construction is transparent and the experiments are external, so this is a partial self-definitional step rather than a hidden fit.

full rationale

The paper's headline RPE result is not circular: Theorem 3 derives the associative-memory form of W2O, W2K, and W1K from one-step gradient descent under explicit sequential-training assumptions, using Bietti et al. only for external lemmas. The APE comparison is an independently cited theorem, and Section 5 tests trained models rather than fitting the theoretical curve, so no fitted parameter is renamed as a prediction. The main circularity concern is confined to the global-vs-in-context analysis: Definition 2 is a hand-constructed transformer whose W2 already stores logπ_b and whose attention weights already implement the induction head, so Propositions 2-3 unpack that construction. Separately, there is an internal inconsistency outside circularity: Eq. 13 writes W1K = Σ χ(k) wE(k) r_-1^T, while the proof of Theorem 3 and Proposition 4 use the transposed orientation Σ r_-1 wE(k)^T; with Eq. 13, r_-1^T W1K wE(v) is near-zero for every v. This is a proof defect that undermines the theorem as written, but it is not a circularity and is not counted in the circularity score beyond the definitional concern already noted.

Assumptions & free parameters 2 free parameters · 6 assumptions · 1 invented entities

The central RPE learning claim depends on near-orthogonality and on a highly simplified sequential training protocol; the global-vs-in-context claim depends on a hand-constructed model whose feed-forward layer stores log pi_b. These are the reader-supplied premises the paper does not derive.

free parameters (2)
  • tau1, tau2, tau3 = not fitted; chosen by hand
    Introduced in Definition 3 to scale the associative memory matrices; they directly appear in the derived logit formula of Proposition 3 and control the balance between global and in-context knowledge.
  • epsilon threshold = not specified
    Introduced in Section 4.2 to replace zero bigram probabilities with a small epsilon in log pi_b, avoiding -infinity logits; no value is given and it affects the global-knowledge term.
assumptions (6)
  • domain assumption Assumption 1: high-dimensional Gaussian embeddings and positional vectors are nearly orthogonal, including after Gaussian random matrix transformations.
    Used throughout Section 3.2 and all proofs to reduce dot products of embeddings and positional vectors to indicators. Borrowed from Bietti et al. [5] and prior work.
  • ad hoc to paper Sequential single-step top-down training: W2O, W2K, W1K are trained one at a time from zero initialization while all other matrices are frozen, with one gradient descent step each.
    Appendix C.2 items 4 and 5. The central RPE theorem is proved under this protocol, not under simultaneous end-to-end training.
  • domain assumption The training sequences come from a bigram model with triggered transitions, with uniform distributions for the initial token, triggers, outputs, and bigram conditionals.
    Appendix C.2 items 1 and 2. The probability calculations in the proof depend on these uniform distributions.
  • ad hoc to paper The loss is evaluated only on sequences of length T ending with the second occurrence of the trigger token, and only on the final output token.
    Appendix C.2 item 3. This restricts the gradient dynamics to a narrow data distribution that differs from realistic language modeling.
  • standard math Lemma 2 and Theorem 3 from Bietti et al. [5] provide the gradient formulas and the APE associative-memory learning result used for comparison.
    The paper cites these results and builds directly on them in Sections 4.1 and Appendix C, without re-proving them.
  • standard math Theorem 4 from Kazemnejad et al. [29] states that one transformer block can implement absolute positional encoding, used in the three-layer no-positional-encoding construction.
    Invoked in Proposition 1 and Appendix D.3 to convert absolute positions into hidden states.
invented entities (1)
  • Associative memory transformer (and stronger variant)
    purpose: A hand-constructed two-layer transformer whose weights are set to implement an induction head and to store bigram log-probabilities in the feed-forward layer; used to derive closed-form logits balancing global and in-context knowledge.
    This is a theoretical construct, not an observed system; its falsifiable content comes from the separately trained models in Section 5, not from the construct itself. The scaling parameters tau1, tau2, tau3 are defined by hand in Definition 3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rethinking Associative Memory Mechanism in Induction Head." pith.science (2026). https://pith.science/paper/56Q5RCAR

@misc{pith2026241211459,
  author       = {Pith},
  title        = {Pith review of: Rethinking Associative Memory Mechanism in Induction Head},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/56Q5RCAR}},
  note         = {Machine review of arXiv:2412.11459}
}
read the original abstract

Induction head mechanism is a part of the computational circuits for in-context learning (ICL) that enable large language models (LLMs) to adapt to new tasks without fine-tuning. Most existing work explains the training dynamics behind acquiring such a powerful mechanism. However, the model's ability to coordinate in-context information over long contexts and global knowledge acquired during pretraining remains poorly understood. This paper investigates how a two-layer transformer thoroughly captures in-context information and balances it with pretrained bigram knowledge in next token prediction, from the viewpoint of associative memory. We theoretically analyze the representation of weight matrices in attention layers and the resulting logits when a transformer is given prompts generated by a bigram model. In the experiments, we design specific prompts to evaluate whether the outputs of the trained transformer align with the theoretical results.

Figures

Figures reproduced from arXiv: 2412.11459 by the authors.

Figure 1
Figure 1. (a) Comparison of a two-layer transformer [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. The visualization of induction head mech [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Two-layer transformers with APE and RPE that are trained on sequences of length [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

72 extracted references · 25 canonical work pages

  1. [1]

    Transformers learn to implement preconditioned gradient descent for in-context learning.Advances in Neural Information Pro- cessing Systems, 36:45614–45650, 2023

    Kwangjun Ahn, Xiang Cheng, Hadi Daneshmand, and Suvrit Sra. Transformers learn to implement preconditioned gradient descent for in-context learning.Advances in Neural Information Pro- cessing Systems, 36:45614–45650, 2023

    work page 2023

  2. [2]

    What learning algo- rithm is in-context learning? investigations with linear models.arXiv preprint arXiv:2211.15661, 2022

    Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. What learning algo- rithm is in-context learning? investigations with linear models.arXiv preprint arXiv:2211.15661, 2022

    arXiv 2022

  3. [3]

    Learning patterns and pattern se- quences by self-organizing nets of threshold el- ements.IEEE Transactions on computers, 100 (11):1197–1206, 1972

    S-I Amari. Learning patterns and pattern se- quences by self-organizing nets of threshold el- ements.IEEE Transactions on computers, 100 (11):1197–1206, 1972

    work page 1972

  4. [4]

    Rethinking the role of scale for in-context learning: An interpretability-based case study at 66 billion scale

    Hritik Bansal, Karthik Gopalakrishnan, Saket Dingliwal, Sravan Bodapati, Katrin Kirchhoff, and Dan Roth. Rethinking the role of scale for in-context learning: An interpretability-based case study at 66 billion scale. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, ed- itors,Proceedings of the 61st Annual Meeting of the Association for Computationa...

    2023

  5. [5]

    Birth of a transformer: A memory viewpoint.Advances in Neural Information Processing Systems, 36, 2024

    Alberto Bietti, Vivien Cabannes, Diane Boucha- court, Herve Jegou, and Leon Bottou. Birth of a transformer: A memory viewpoint.Advances in Neural Information Processing Systems, 36, 2024

    work page 2024

  6. [6]

    Language models are few-shot learners.arXiv preprint arXiv:2005.14165, 2020

    Tom B Brown. Language models are few-shot learners.arXiv preprint arXiv:2005.14165, 2020

    arXiv 2005

  7. [7]

    Scaling laws for associative memories

    Vivien Cabannes, Elvis Dohmatob, and Alberto Bietti. Scaling laws for associative memories. arXiv preprint arXiv:2310.02984, 2023

    arXiv 2023

  8. [8]

    Learning associative memories with gradient descent.arXiv preprint arXiv:2402.18724, 2024

    Vivien Cabannes, Berfin Simsek, and Alberto Bi- etti. Learning associative memories with gradient descent.arXiv preprint arXiv:2402.18724, 2024

    arXiv 2024

Show all 72 references
  1. [9]

    How truncating weights improves reasoning in lan- guage models.arXiv preprint arXiv:2406.03068, 2024

    Lei Chen, Joan Bruna, and Alberto Bietti. How truncating weights improves reasoning in lan- guage models.arXiv preprint arXiv:2406.03068, 2024

    2024 arXiv

  2. [10]

    Unveiling induction heads: Prov- able training dynamics and feature learning in 8 transformers.arXiv preprint arXiv:2409.10559, 2024

    Siyu Chen, Heejune Sheen, Tianhao Wang, and Zhuoran Yang. Unveiling induction heads: Prov- able training dynamics and feature learning in 8 transformers.arXiv preprint arXiv:2409.10559, 2024

    2024 arXiv

  3. [11]

    Towards automated circuit discovery for mechanistic interpretability.Advances in Neu- ral Information Processing Systems, 36:16318– 16352, 2023

    Arthur Conmy, Augustine Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga- Alonso. Towards automated circuit discovery for mechanistic interpretability.Advances in Neu- ral Information Processing Systems, 36:16318– 16352, 2023

    2023

  4. [12]

    Why can GPT learn in-context? language models secretly perform gradient descent as meta-optimizers

    Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shum- ing Ma, Zhifang Sui, and Furu Wei. Why can GPT learn in-context? language models secretly perform gradient descent as meta-optimizers. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors,Findings of the Association for...

    2023

  5. [13]

    Analyzing transformers in embedding space.arXiv preprint arXiv:2209.02535, 2022

    Guy Dar, Mor Geva, Ankit Gupta, and Jonathan Berant. Analyzing transformers in embedding space.arXiv preprint arXiv:2209.02535, 2022

    2022 arXiv

  6. [14]

    Bert: Pre-training of deep bidirec- tional transformers for language understanding

    Jacob Devlin. Bert: Pre-training of deep bidirec- tional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  7. [16]

    The evolution of statistical induction heads: In- context learning markov chains.arXiv preprint arXiv:2402.11004, 2024

    Benjamin L Edelman, Ezra Edelman, Surbhi Goel, Eran Malach, and Nikolaos Tsilivis. The evolution of statistical induction heads: In- context learning markov chains.arXiv preprint arXiv:2402.11004, 2024

    2024 arXiv

  8. [17]

    A mathematical framework for transformer circuits.Transformer Circuits Thread, 1(1):12, 2021

    Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, et al. A mathematical framework for transformer circuits.Transformer Circuits Thread, 1(1):12, 2021

    2021

  9. [18]

    Learning transformer programs, 2023

    Dan Friedman, Alexander Wettig, and Danqi Chen. Learning transformer programs, 2023. URLhttps://arxiv.org/abs/2306.01128

    2023 arXiv

  10. [19]

    Transformer feed-forward lay- ers are key-value memories.arXiv preprint arXiv:2012.14913, 2020

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward lay- ers are key-value memories.arXiv preprint arXiv:2012.14913, 2020

    2012 arXiv

  11. [20]

    Dissecting recall of factual as- sociations in auto-regressive language models

    Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. Dissecting recall of factual as- sociations in auto-regressive language models. arXiv preprint arXiv:2304.14767, 2023

    2023 arXiv

  12. [21]

    Deberta: Decoding-enhanced bert with disentangled attention.arXiv preprint arXiv:2006.03654, 2020

    Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding-enhanced bert with disentangled attention.arXiv preprint arXiv:2006.03654, 2020

    2006 arXiv

  13. [22]

    Neural networks and physical systems with emergent collective computational abilities.Proceedings of the national academy of sciences, 79(8):2554–2558, 1982

    John J Hopfield. Neural networks and physical systems with emergent collective computational abilities.Proceedings of the national academy of sciences, 79(8):2554–2558, 1982

    1982

  14. [23]

    Neurons with graded response have collective computational properties like those of two-state neurons.Proceedings of the national academy of sciences, 81(10):3088–3092, 1984

    John J Hopfield. Neurons with graded response have collective computational properties like those of two-state neurons.Proceedings of the national academy of sciences, 81(10):3088–3092, 1984

    1984

  15. [24]

    In-context convergence of transformers.arXiv preprint arXiv:2310.05249, 2023

    Yu Huang, Yuan Cheng, and Yingbin Liang. In-context convergence of transformers.arXiv preprint arXiv:2310.05249, 2023

    2023 arXiv

  16. [25]

    Improve transformer models with bet- ter relative position embeddings.arXiv preprint arXiv:2009.13658, 2020

    Zhiheng Huang, Davis Liang, Peng Xu, and Bing Xiang. Improve transformer models with bet- ter relative position embeddings.arXiv preprint arXiv:2009.13658, 2020

    2009 arXiv

  17. [26]

    Length generalization in arithmetic transformers.arXiv preprint arXiv:2306.15400, 2023

    Samy Jelassi, Stéphane d’Ascoli, Carles Domingo-Enrich, Yuhuai Wu, Yuanzhi Li, and François Charton. Length generalization in arithmetic transformers.arXiv preprint arXiv:2306.15400, 2023

    2023 arXiv

  18. [27]

    Do llms dream of ele- phants (when told not to)? latent concept asso- ciation and associative memory in transformers

    Yibo Jiang, Goutham Rajendran, Pradeep Raviku- mar, and Bryon Aragam. Do llms dream of ele- phants (when told not to)? latent concept asso- ciation and associative memory in transformers. arXiv preprint arXiv:2406.18400, 2024

    2024 arXiv

  19. [28]

    The unreasonable effec- tiveness of recurrent neural networks, May

    Andrej Karpathy. The unreasonable effec- tiveness of recurrent neural networks, May

  20. [29]

    The impact of positional encoding on length generalization in transform- ers.Advances in Neural Information Processing Systems, 36, 2024

    Amirhossein Kazemnejad, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Payel Das, and Siva Reddy. The impact of positional encoding on length generalization in transform- ers.Advances in Neural Information Processing Systems, 36, 2024

    2024

  21. [30]

    Rethink- ing positional encoding in language pre-training

    Guolin Ke, Di He, and Tie-Yan Liu. Rethink- ing positional encoding in language pre-training. arXiv preprint arXiv:2006.15595, 2020

    2006 arXiv

  22. [31]

    Shape: Shifted absolute posi- tion embedding for transformers.arXiv preprint arXiv:2109.05644, 2021

    Shun Kiyono, Sosuke Kobayashi, Jun Suzuki, and Kentaro Inui. Shape: Shifted absolute posi- tion embedding for transformers.arXiv preprint arXiv:2109.05644, 2021

    2021 arXiv

  23. [32]

    Dense as- sociative memory for pattern recognition, 2016

    Dmitry Krotov and John J Hopfield. Dense as- sociative memory for pattern recognition, 2016. URLhttps://arxiv.org/abs/1606.01164

    2016 arXiv

  24. [33]

    Mechan- ics of next token prediction with self-attention

    Yingcong Li, Yixiao Huang, Muhammed E Ildiz, Ankit Singh Rawat, and Samet Oymak. Mechan- ics of next token prediction with self-attention. InInternational Conference on Artificial Intel- ligence and Statistics, pages 685–693. PMLR, 2024

    2024

  25. [34]

    Cape: Encoding relative positions with continuous augmented positional embeddings

    Tatiana Likhomanenko, Qiantong Xu, Gabriel Synnaeve, Ronan Collobert, and Alex Rogozh- nikov. Cape: Encoding relative positions with continuous augmented positional embeddings. Advances in Neural Information Processing Sys- tems, 34:16079–16092, 2021

    2021

  26. [35]

    Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for GPT-3? In Eneko Agirre, Marianna Apidianaki, and Ivan Vuli´c, editors,Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowle...

    2022 doi

  27. [36]

    One step of gradient descent is provably the optimal in-context learner with one layer of linear self-attention.arXiv preprint arXiv:2307.03576, 2023

    Arvind Mahankali, Tatsunori B Hashimoto, and Tengyu Ma. One step of gradient descent is provably the optimal in-context learner with one layer of linear self-attention.arXiv preprint arXiv:2307.03576, 2023

    2023 arXiv

  28. [37]

    Locating and editing factual associations in gpt.Advances in Neural Informa- tion Processing Systems, 35:17359–17372, 2022

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt.Advances in Neural Informa- tion Processing Systems, 35:17359–17372, 2022

    2022

  29. [38]

    Efficient estimation of word representations in vector space.arXiv preprint arXiv:1301.3781, 3781, 2013

    Tomas Mikolov. Efficient estimation of word representations in vector space.arXiv preprint arXiv:1301.3781, 3781, 2013

    2013 arXiv

  30. [39]

    Progress mea- sures for grokking via mechanistic interpretability

    Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. Progress mea- sures for grokking via mechanistic interpretability. arXiv preprint arXiv:2301.05217, 2023

    2023 arXiv

  31. [40]

    On the rela- tion between position information and sentence length in neural machine translation

    Masato Neishi and Naoki Yoshinaga. On the rela- tion between position information and sentence length in neural machine translation. InProceed- ings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 328– 338, 2019

    2019

  32. [41]

    How transformers learn causal structure with gra- dient descent.arXiv preprint arXiv:2402.14735, 2024

    Eshaan Nichani, Alex Damian, and Jason D Lee. How transformers learn causal structure with gra- dient descent.arXiv preprint arXiv:2402.14735, 2024

    2024 arXiv

  33. [42]

    In-context learning and in- duction heads.arXiv preprint arXiv:2209.11895, 2022

    Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context learning and in- duction heads.arXiv preprint arXiv:2209.11895, 2022

    2022 arXiv

  34. [43]

    Train short, test long: Attention with linear biases en- ables input length extrapolation.arXiv preprint arXiv:2108.12409, 2021

    Ofir Press, Noah A Smith, and Mike Lewis. Train short, test long: Attention with linear biases en- ables input length extrapolation.arXiv preprint arXiv:2108.12409, 2021

    2021 arXiv

  35. [44]

    Improving language understanding by generative pre-training

    Alec Radford. Improving language understanding by generative pre-training. 2018

    2018

  36. [45]

    Lan- guage models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Lan- guage models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019

    2019

  37. [46]

    Exploring the limits of transfer learning with a unified text- to-text transformer.Journal of machine learning research, 21(140):1–67, 2020

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text- to-text transformer.Journal of machine learning research, 21(140):1–67, 2020

    2020

  38. [47]

    Identifying semantic induction heads to under- stand in-context learning

    Jie Ren, Qipeng Guo, Hang Yan, Dongrui Liu, Quanshi Zhang, Xipeng Qiu, and Dahua Lin. Identifying semantic induction heads to under- stand in-context learning. In Lun-Wei Ku, An- dre Martins, and Vivek Srikumar, editors,Find- ings of the Association for Computational Lin- guis...

    2024 doi

  39. [48]

    Ran- domized positional encodings boost length gen- eralization of transformers.arXiv preprint arXiv:2305.16843, 2023

    Anian Ruoss, Grégoire Delétang, Tim Genewein, Jordi Grau-Moya, Róbert Csordás, Mehdi Ben- nani, Shane Legg, and Joel Veness. Ran- domized positional encodings boost length gen- eralization of transformers.arXiv preprint arXiv:2305.16843, 2023

    2023 arXiv

  40. [49]

    Self-attention with relative position representa- tions.arXiv preprint arXiv:1803.02155, 2018

    Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-attention with relative position representa- tions.arXiv preprint arXiv:1803.02155, 2018

    2018 arXiv

  41. [50]

    The curious case of ab- solute position embeddings.arXiv preprint arXiv:2210.12574, 2022

    Koustuv Sinha, Amirhossein Kazemnejad, Siva Reddy, Joelle Pineau, Dieuwke Hupkes, and Adina Williams. The curious case of ab- solute position embeddings.arXiv preprint arXiv:2210.12574, 2022

    2022 arXiv

  42. [51]

    Roformer: En- hanced transformer with rotary position embed- ding.Neurocomputing, 568:127063, 2024

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: En- hanced transformer with rotary position embed- ding.Neurocomputing, 568:127063, 2024

    2024

  43. [52]

    Attention is all you need.Advances in Neural Information Processing Systems, 2017

    A Vaswani. Attention is all you need.Advances in Neural Information Processing Systems, 2017

    2017

  44. [53]

    Transformers learn in-context by gradient descent

    Johannes V on Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordv- intsev, Andrey Zhmoginov, and Max Vladymyrov. Transformers learn in-context by gradient descent. InInternational Conference on Machine Learn- ing, pages 35151–35174. PMLR, 2023

    2023

  45. [54]

    Encoding word order in complex embed- dings.arXiv preprint arXiv:1912.12333, 2019

    Benyou Wang, Donghao Zhao, Christina Lioma, Qiuchi Li, Peng Zhang, and Jakob Grue Simon- sen. Encoding word order in complex embed- dings.arXiv preprint arXiv:1912.12333, 2019

    1912 arXiv

  46. [55]

    Interpretability in the wild: a circuit for indirect object identification in gpt-2 small.arXiv preprint arXiv:2211.00593, 2022

    Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small.arXiv preprint arXiv:2211.00593, 2022

    2022 arXiv

  47. [56]

    La- bel words are anchors: An information flow per- spective for understanding in-context learning

    Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, and Xu Sun. La- bel words are anchors: An information flow per- spective for understanding in-context learning. In Houda Bouamor, Juan Pino, and Kalika Bali, edi- tors,Proceedings of the 2023 Conference...

    2023 doi

  48. [57]

    The learnability of in-context learning.Advances in Neural Information Processing Systems, 36, 2024

    Noam Wies, Yoav Levine, and Amnon Shashua. The learnability of in-context learning.Advances in Neural Information Processing Systems, 36, 2024

    2024

  49. [58]

    Self-adaptive in-context learn- ing: An information compression perspective for in-context example selection and ordering

    Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, and Lingpeng Kong. Self-adaptive in-context learn- ing: An information compression perspective for in-context example selection and ordering. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors,Proceedings of the 61st An- nua...

    2023 doi

  50. [59]

    An explanation of in- context learning as implicit bayesian inference

    Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in- context learning as implicit bayesian inference. arXiv preprint arXiv:2111.02080, 2021

    2021 arXiv

  51. [60]

    Memory mo- saics, 2024

    Jianyu Zhang, Niklas Nolte, Ranajoy Sadhukhan, Beidi Chen, and Léon Bottou. Memory mo- saics, 2024. URL https://arxiv.org/abs/ 2405.06394

    2024 arXiv

  52. [61]

    Trained transformers learn linear models in- context.arXiv preprint arXiv:2306.09927, 2023

    Ruiqi Zhang, Spencer Frei, and Peter L Bartlett. Trained transformers learn linear models in- context.arXiv preprint arXiv:2306.09927, 2023

    2023 arXiv

  53. [62]

    What and how does in- context learning learn? bayesian model averag- ing, parameterization, and generalization.arXiv preprint arXiv:2305.19420, 2023

    Yufeng Zhang, Fengzhuo Zhang, Zhuoran Yang, and Zhaoran Wang. What and how does in- context learning learn? bayesian model averag- ing, parameterization, and generalization.arXiv preprint arXiv:2305.19420, 2023

    2023 arXiv

  54. [63]

    Length extrapola- tion of transformers: A survey from the per- spective of position encoding.arXiv preprint arXiv:2312.17044, 2023

    Liang Zhao, Xiaocheng Feng, Xiachong Feng, Bin Qin, and Ting Liu. Length extrapola- tion of transformers: A survey from the per- spective of position encoding.arXiv preprint arXiv:2312.17044, 2023. 11 Attention Layer residual connection Attention Layer O residual connection Fi...

    2023 arXiv

  55. [65]

    In other words, let tq be the first occurrence position

    Input SequenceWe consider an input sequence z1:T ∈ VT which has one trigger token q appear- ing twice, and ends with the trigger token. In other words, let tq be the first occurrence position. Then, we have ztq =z T =q . From the bigram generation rule in Sec.3.4, ztq+1 and zT...

  56. [66]

    Probability Distribution Assumptions: The bi- gram is generated by uniform distributions over [V] for any index i, i.e., πu, πq, πo, and πb(· |i) are uniformly distributed

  57. [67]

    Simplification of Loss Function: Consider the loss only for sequences of length T where the 13 Symbol Description d Dimensionality of the positional encoding V V ocabulary set V V ocabulary size wE(zt) Embedding of tokenz t rs−t Relative positional encoding expressing the rela...

  58. [68]

    We employ zero- initialization and carry out a single gradient de- scent step

    Simplification for Learning Focus: Our ap- proach involves sequentially training W O 2 , W K 2 , and W K 1 from top to bottom. We employ zero- initialization and carry out a single gradient de- scent step

  59. [69]

    TX t=to W 2 V 1 t Φ1wE(zto)|y=k # − 1 T ·E

    Initialization and Freezing: To achieve our goal of showing that W 2 O, W2 K and W 1 K learn to be an associative memory, we zero-initialize the three matrices. For other matrices such as W 2 V , W1 V and W 1 O are randomly initialized from Gaussian dis- tribution. We set W 1 ...

  60. [70]

    count the number cA(B) of analogical pair (A, B). 34

  61. [71]

    samplep A ∈[0.01,0.1]for each source wordA

  62. [72]

    consider the analogical pair (A, Bi) appear prcA(B) times, where Bi is the fake target words for source wordA

  63. [73]

    Everytime we generate an input sequence, we sam- ple5trigger tokens from the set of all source words, and uniformly selects the corresponding output tokens from all vocabulary

    calculate the bigram conditionals based on the number of appearance of analogical pairs. Everytime we generate an input sequence, we sam- ple5trigger tokens from the set of all source words, and uniformly selects the corresponding output tokens from all vocabulary. we start fr...

  64. [2015]

    Accessed: 2024-12-15

    URL https://karpathy.github.io/ 2015/05/21/rnn-effectiveness/. Accessed: 2024-12-15. 9

    2015

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.