Pith. sign in

REVIEW 2 major objections 1 minor 234 cited by

In-context Learning and Induction Heads

T0 review · 2 major / 1 minor · reviewed 2026-05-11 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Induction heads implement the core copying algorithm behind in-context learning in transformers.

desk verdict Induction heads drive in-context learning with solid causal evidence in small attention-only models but only correlational support in larger ones with MLPs. read the letter →

arxiv 2209.11895 v1 pith:FJNRNXM5 submitted 2022-09-24 cs.LG

Catherine Olsson , Nelson Elhage , Neel Nanda , Nicholas Joseph , Nova DasSarma , Tom Henighan , Ben Mann , Amanda Askell
show 18 more authors
Yuntao Bai Anna Chen Tom Conerly Dawn Drain Deep Ganguli Zac Hatfield-Dodds Danny Hernandez Scott Johnston Andy Jones Jackson Kernion Liane Lovitt Kamal Ndousse Dario Amodei Tom Brown Jack Clark Jared Kaplan Sam McCandlish Chris Olah
This is my paper · ORCID
classification cs.LG
keywords inductionheadsin-contextlearningtransformermodelsattentionmechanismstrainingdynamicssequencecopying
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that induction heads are the main mechanism driving in-context learning, the steady drop in loss on later tokens within a sequence. These heads detect a repeated token and copy the token that followed it last time, completing patterns like [A][B]...[A] to [B]. The authors show that these heads appear at the exact training step where a sharp bump in in-context performance occurs. They give causal evidence in small attention-only models by editing the heads and correlational evidence in larger models. If the claim holds, it would mean that a simple, local copying rule explains most of the rapid adaptation transformers show during inference.

What carries the argument

Induction heads, attention heads that detect a prior token match and copy the subsequent token from that earlier occurrence.

What would settle it

Train a transformer in which induction heads never form yet a sharp increase in in-context learning still appears at the same training step.

Watch

Extended reading notes

Core claim

Induction heads are attention heads that implement a simple algorithm to complete token sequences like [A][B] ... [A] -> [B]. The authors present six lines of evidence that these heads constitute the mechanism for the majority of all in-context learning in large transformer models, developing precisely when a sudden sharp increase in in-context learning ability occurs during training.

Load-bearing premise

The formation of induction heads directly causes the observed jump in in-context learning rather than both changes arising together from some other training dynamic.

Editorial extensions

If this is right

  • Induction heads emerge at the same moment training loss shows a sharp improvement on later tokens.
  • In small attention-only models, directly ablating induction heads reduces in-context learning performance.
  • The timing correlation between head formation and performance gains holds across model sizes.
  • The mechanism appears general enough to explain in-context learning in transformers of any scale.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If induction heads are the primary driver, then interventions that speed their formation could shorten the training needed for strong few-shot behavior.
  • The copying rule might also explain why transformers handle many different in-context tasks without task-specific fine-tuning.
  • Checking whether non-attention architectures develop analogous copying circuits would test how specific this mechanism is to transformers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The paper hypothesizes that 'induction heads' (attention heads implementing a simple [A][B]...[A] -> [B] completion algorithm) are the primary mechanistic source of in-context learning in transformers, defined as the decrease in loss at increasing token indices. It reports that these heads emerge at the same training point as a sharp loss bump signaling increased in-context ability, presenting six lines of evidence: strong causal interventions (ablations/patching) for small attention-only models and correlational/timing-based evidence for larger models containing MLPs.

Significance. If the causal link holds, the work would supply a concrete mechanistic account of in-context learning, a core capability of large language models. The strong, reproducible causal interventions in small attention-only models constitute a clear strength, as do the multiple complementary observational measures (timing correlations, head activation patterns) that could guide future targeted experiments. The paper thereby advances mechanistic interpretability by linking a specific circuit to a broad behavioral phenomenon.

major comments (2)
  1. Abstract: The claim that induction heads 'might constitute the mechanism for the majority of all in-context learning' in large transformer models rests on correlational evidence only; the text states that the six lines of evidence for models with MLPs are 'preliminary and indirect' and 'correlational,' with no ablation, patching, or causal intervention results reported to show that disabling induction heads specifically impairs the observed in-context loss reduction.
  2. Description of the six lines of evidence (larger models): These lines rely on coincidence of induction-head emergence with the training loss bump and on observational metrics such as head activation timing; they do not include controls that would distinguish whether both phenomena are parallel downstream effects of an earlier training dynamic (e.g., a phase transition in optimization or representation geometry), leaving the causal inference untested for models containing MLPs.
minor comments (1)
  1. Abstract: Quantitative details on the magnitude of the loss bump, the fraction of heads identified as induction heads, and any error controls or statistical tests for the six lines of evidence would improve clarity and allow readers to assess the strength of the correlational results.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive review and for recognizing the value of the causal interventions in small models as well as the potential of the observational measures to guide future work. We agree that the distinction between causal and correlational evidence must be drawn more sharply in the abstract and discussion, and we will revise the manuscript to address both major comments.

read point-by-point responses

  1. Referee: Abstract: The claim that induction heads 'might constitute the mechanism for the majority of all in-context learning' in large transformer models rests on correlational evidence only; the text states that the six lines of evidence for models with MLPs are 'preliminary and indirect' and 'correlational,' with no ablation, patching, or causal intervention results reported to show that disabling induction heads specifically impairs the observed in-context loss reduction.

    Authors: We accept the point. While the body of the paper already describes the evidence for models with MLPs as preliminary, indirect, and correlational, the abstract phrasing risks implying stronger support than exists. We will revise the abstract to state explicitly that the hypothesis for large models rests on correlational evidence from the six lines, without causal interventions such as ablation or patching, and to moderate the language concerning induction heads as the mechanism for the majority of in-context learning. revision: yes

  2. Referee: Description of the six lines of evidence (larger models): These lines rely on coincidence of induction-head emergence with the training loss bump and on observational metrics such as head activation timing; they do not include controls that would distinguish whether both phenomena are parallel downstream effects of an earlier training dynamic (e.g., a phase transition in optimization or representation geometry), leaving the causal inference untested for models containing MLPs.

    Authors: The referee correctly notes that the six lines are observational and lack controls that could rule out alternative accounts in which induction-head emergence and the loss bump are both downstream of an earlier training dynamic. We do not claim to have performed such controls. In revision we will add an explicit limitations paragraph in the discussion that acknowledges this gap, lists possible alternative explanations (including phase transitions in optimization or representation geometry), and clarifies that the lines of evidence are intended to be suggestive and to motivate targeted causal experiments rather than to demonstrate causality. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; hypothesis rests on timing correlations and interventions rather than definitional reduction

full rationale

The paper defines induction heads via their observable attention pattern on token sequences and presents empirical evidence (simultaneous emergence with loss bump, six lines of correlational evidence for large models, and causal ablations for small attention-only models) that they contribute to in-context learning. No step reduces a claimed prediction or result to a fitted parameter or self-citation by construction; the central hypothesis is explicitly labeled preliminary and indirect, with the link to decreasing loss at later token indices argued via external observations rather than tautological redefinition. The derivation chain is self-contained against the provided benchmarks.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The claim rests on the operational definition of induction heads as pattern-completion circuits and on the assumption that their sudden appearance during training is the direct cause of improved in-context performance rather than a correlated side effect.

assumptions (2)
  • domain assumption Induction heads implement a simple algorithm to complete token sequences like [A][B] ... [A] -> [B]
    This is the paper's working definition of the heads whose causal role is being tested.
  • domain assumption A sharp increase in in-context learning ability is visible as a bump in the training loss curve
    The timing alignment between this bump and the emergence of induction heads is treated as a key signature.

how reviews work

0 comments
Cite this review

Pith. "Pith review of In-context Learning and Induction Heads." pith.science (2026). https://pith.science/paper/FJNRNXM5

@misc{pith2026220911895,
  author       = {Pith},
  title        = {Pith review of: In-context Learning and Induction Heads},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FJNRNXM5}},
  note         = {Machine review of arXiv:2209.11895}
}
read the original abstract

"Induction heads" are attention heads that implement a simple algorithm to complete token sequences like [A][B] ... [A] -> [B]. In this work, we present preliminary and indirect evidence for a hypothesis that induction heads might constitute the mechanism for the majority of all "in-context learning" in large transformer models (i.e. decreasing loss at increasing token indices). We find that induction heads develop at precisely the same point as a sudden sharp increase in in-context learning ability, visible as a bump in the training loss. We present six complementary lines of evidence, arguing that induction heads may be the mechanistic source of general in-context learning in transformer models of any size. For small attention-only models, we present strong, causal evidence; for larger models with MLPs, we present correlational evidence.

Discussion (0). Continue with ORCID to comment.

Forward citations

Showing 60 of 234 Pith papers that cite this

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. See all 234 Pith citations

  1. Neural Collapse Is Forbidden: Information Floors in Language Models

    cs.LG 2026-07 conditional novelty 8.0 of 10

    Within-category identity dispersion in language models tracks conditional mutual information I(token; context|category) and is forced by a proved information floor that forbids full neural collapse.

  2. Efficiently Representing Algorithms With Chain-of-Thought Transformers

    cs.LG 2026-06 conditional novelty 8.0 of 10

    CoT transformers simulate any Word RAM algorithm with poly-logarithmic overhead in three architectures, improving on quadratic TM overhead.

  3. Looped Transformers with Layer Normalization Provably Learn the Power Method

    cs.LG 2026-05 unverdicted novelty 8.0 of 10

    Looped linear transformers with LN provably converge via GD to implement the power method on principal component prediction.

  4. Towards Verifiable Transformers: Solver-Checkable Circuit Explanations

    cs.LG 2026-05 unverdicted novelty 8.0 of 10

    Presents a solver-verifiable framework for Transformer circuits, with exhaustive checks on small symbolic tasks and surrogate methods for larger models.

  5. WriteSAE: Sparse Autoencoders for Recurrent State

    cs.LG 2026-05 unverdicted novelty 8.0 of 10

    WriteSAE is the first sparse autoencoder that factors decoder atoms into the native d_k x d_v cache write shape of recurrent models and supplies a closed-form per-token logit shift for atom substitution.

  6. The Spectral Lifecycle of Transformer Training: Transient Compression Waves, Persistent Spectral Gradients, and the Q/K--V Asymmetry

    cs.LG 2026-04 unverdicted novelty 8.0 of 10

    Transformer weight spectra exhibit transient compression waves that propagate layer-wise, persistent non-monotonic depth gradients in power-law exponents, and Q/K-V asymmetry, with the spectral exponent alpha predicti...

  7. Geometric Hyena Networks for Large-scale Equivariant Learning

    cs.LG 2025-05 conditional novelty 8.0 of 10

    Geometric Hyena is an equivariant long-convolutional architecture that captures global geometric context with sub-quadratic complexity and outperforms equivariant transformer baselines on several RNA and protein predi...

  8. The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery

    cs.AI 2024-08 unverdicted novelty 8.0 of 10

    The AI Scientist framework enables LLMs to independently conduct the full scientific process from idea generation to paper writing and review, demonstrated across three ML subfields with papers costing under $15 each.

  9. KAN: Kolmogorov-Arnold Networks

    cs.LG 2024-04 conditional novelty 8.0 of 10

    KANs with learnable univariate spline activations on edges achieve better accuracy than MLPs with fewer parameters, faster scaling, and direct visualization for scientific discovery.

  10. Localizing Model Behavior with Path Patching

    cs.LG 2023-04 unverdicted novelty 8.0 of 10

    Path patching provides a method to express and quantitatively test hypotheses that neural network behaviors are localized to sets of paths.

  11. Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small

    cs.LG 2022-11 conditional novelty 8.0 of 10

    GPT-2 small solves indirect object identification via a circuit of 26 attention heads organized into seven functional classes discovered through causal interventions.

  12. Harmful Content Is Not Enough: Continuation Framing Moderates In-Context Emergent Misalignment

    cs.AI 2026-08 conditional novelty 7.0 of 10

    With harmful answer text held fixed, demonstration framing raises broad emergent misalignment by 30 to 32 percentage points over document framing on Gemini 3.1, and message role further modulates the effect on Grok.

  13. Test, then Route: How Language Models Execute In-Context Conditional Rules Across Models and Languages

    cs.CL 2026-08 conditional novelty 7.0 of 10

    In three open language models across six languages, the predicate test in an in-context conditional is a separable mid-stack circuit, but the learned answer-router direction is token-bound and does not transfer to new...

  14. Indexing: the Beginning and the End

    cs.LG 2026-07 conditional novelty 7.0 of 10

    Causal-complexity bounds show RNNs, SSMs, and masked linear attention need ω(1) layers for right-hand indexing, while a one-layer softmax transformer solves it; when the index is first, a one-layer RNN suffices.

  15. When Data Imbalance Helps: Robust Generalization Through Shortcut Saturation

    cs.LG 2026-07 conditional novelty 7.0 of 10

    High spurious-correlation ratios promote robust generalization via shortcut saturation in two-layer transformers but trap one-layer models on the shortcut.

  16. Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning

    cs.AI 2026-07 conditional novelty 7.0 of 10

    Memorized facts in fine-tuned LLMs often sit off the mid-layer reasoning path; relocating those representations recovers most multi-hop generalization failures.

  17. How Data Shapes RoPE Frequency Usage: From Positional Scale Matching to Length Generalization

    cs.LG 2026-07 conditional novelty 7.0 of 10

    RoPE frequency usage is determined by a data-induced dependency width W, with the optimal frequency scaling as π/W, explaining both learned spectra and the success of position interpolation.

  18. Fingerprint, Not Blueprint: How Positional Schemes Set the Default Spectral Algebra of Attention

    cs.LG 2026-07 conditional novelty 7.0 of 10

    Positional schemes set the default spectral algebra of attention heads: previous-token heads are rotational under RoPE and content-like under absolute/ALiBi, as a post-function fingerprint rather than a hard constraint.

  19. ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning

    cs.AI 2026-07 unverdicted novelty 7.0 of 10

    RECONTEXT is a recursive evidence replay technique that improves long-context reasoning in LLMs by constructing and replaying a query-conditioned evidence pool before final generation.

  20. Can Language Models Actually Retrieve In-Context? Drowning in Documents at Million Token Scale

    cs.CL 2026-07 unverdicted novelty 7.0 of 10

    A 0.6B LM with length-aware attention adjustments performs competitive in-context retrieval at million-token scale on MS MARCO, NQ, and LIMIT benchmarks.

  21. Logit-Contribution Scoring Identifies Non-Literal Retrieval Heads

    cs.CL 2026-07 unverdicted novelty 7.0 of 10

    LOCOS scores attention heads via OV-circuit output projection onto answer-token unembedding directions and identifies non-literal retrieval heads whose ablation collapses performance on non-literal benchmarks more tha...

  22. SemRF: A Semantic Reference Frame for Residual-Stream Dynamics in Language Models

    cs.LG 2026-06 unverdicted novelty 7.0 of 10

    SemRF supplies fixed semantic anchors and pseudo-inverse tying to produce stable coordinates for residual dynamics, Voronoi traces, and minimum-action canonical paths that link to parameter efficiency under controlled...

  23. ECHO: Learning Epistemically Adaptive Language Agents with Turn-Level Credit

    cs.MA 2026-06 unverdicted novelty 7.0 of 10

    ECHO is a clipped policy-gradient method that uses posterior-sensitive rewards to give turn-level epistemic credit in multi-turn information-seeking tasks, outperforming trajectory-level GRPO on a new Clue Selector Ga...

  24. Symbolic Mechanistic Data Attribution: Tracing Training Influence to Learned Behavioral Policies

    cs.LG 2026-06 unverdicted novelty 7.0 of 10

    SMDA fits ridge regression on SAE features to distill symbolic policies then decomposes each SFT example's influence via feature-activation and output-probability deltas, demonstrated on refusal behavior in Llama-3.2-...

  25. Natural Ungrokking: Asymmetric Control of Which Rules Survive Pretraining

    cs.LG 2026-06 unverdicted novelty 7.0 of 10

    During pretraining, language models exhibit natural ungrokking where learned rules are forgotten based on their support frequency in the corpus, with asymmetric editability of rule survival.

  26. DREAM: Dense Retrieval Embeddings via Autoregressive Modeling

    cs.CL 2026-06 unverdicted novelty 7.0 of 10

    DREAM enables training of dense retrieval embeddings using autoregressive next-token prediction from LLMs by modulating attention with retriever scores.

  27. Mind the Heads: Topological Representation Alignment for Multimodal LLMs

    cs.CV 2026-06 conditional novelty 7.0 of 10

    HeRA aligns least-aligned attention heads in MLLMs using an MKNN-based contrastive objective to preserve cross-modal topological structure, yielding gains on vision-centric tasks and reduced hallucinations across 18 b...

  28. Safe to Check, Unsafe to Use: Relinking at the Compression Boundary of LLM Agents

    cs.CR 2026-06 unverdicted novelty 7.0 of 10

    Relinking is a new compression-boundary attack on LLM agents where summarization of split benign fragments produces malicious instructions, shown via Relink tool at 86.9% success rate and mitigated by KBRA defense to 0%.

  29. Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Models

    cs.CV 2026-06 conditional novelty 7.0 of 10

    Reroute turns irreversible visual-token pruning into recoverable routing that reuses existing attention scores, improving grounding performance under aggressive reduction on LLaVA-1.5 and Qwen while preserving TFLOPs ...

  30. Phase Transitions in Attention: A Bayesian Theory of Copy Head Emergence

    stat.ML 2026-06 unverdicted novelty 7.0 of 10

    Bayesian reduction of attention posterior on copy task predicts first-order phase transition for softmax attention and second-order followed by crossover for linear attention.

  31. STRIDE: Training Data Attribution via Sparse Recovery from Subset Perturbations

    cs.LG 2026-06 unverdicted novelty 7.0 of 10

    STRIDE formulates TDA as sparse recovery using steering operators that mimic subset training effects in activation space, claiming SOTA LLM pre-training attribution at 13x prior speed.

  32. What Does Chain-of-Thought Contribute at Probe Time? Evidence for Local Co-Occurrence Activation

    cs.AI 2026-05 unverdicted novelty 7.0 of 10

    CoT probe-time gains arise primarily from lexical activation and short-range token co-occurrence rather than sentence-level logical derivation.

  33. Geometry-Adaptive Explainer for Faithful Dictionary-Based Interpretability under Distribution Shift

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    GAE reduces the faithfulness gap in dictionary-based explainers under distribution shift by geometrically realigning the ID dictionary to the OOD-active subspace, with a quadratic excess-loss bound.

  34. Where Pretraining writes and Alignment reads: the asymmetry of Transformer weight space

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    Pretraining and alignment induce asymmetric geometric traces in transformer weights because alignment updates concentrate in read pathways due to activation covariance while write pathways inherit less structure from ...

  35. Diagnosing and Correcting Concept Omission in Multimodal Diffusion Transformers

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    Text embeddings in MM-DiTs encode a detectable omission signal for missing concepts; amplifying it via OSI reduces concept omission in text-to-image outputs on FLUX.1-Dev and SD3.5-Medium.

  36. Many-Shot CoT-ICL: Making In-Context Learning Truly Learn

    cs.CL 2026-05 unverdicted novelty 7.0 of 10

    Many-shot CoT-ICL improves when demonstrations are ordered for smooth conceptual progression, with CDS delivering up to 5.42 percentage-point gains on math tasks using 64 examples.

  37. Assessing the Creativity of Large Language Models: Testing, Limits, and New Frontiers

    cs.AI 2026-05 conditional novelty 7.0 of 10

    The Divergent Remote Association Test (DRAT) is the first creativity test that significantly predicts LLMs' scientific ideation ability, unlike prior tests such as DAT or RAT.

  38. Learning Less Is More: Premature Upper-Layer Attention Specialization Hurts Language Model Pretraining

    cs.CL 2026-05 unverdicted novelty 7.0 of 10

    Temporarily reducing the learning rate on upper-layer query and key projections during early GPT pretraining prevents premature attention specialization and improves model performance.

  39. Self-Attention as a Covariance Readout: A Unified View of In-Context Learning and Repetition

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    Self-attention acts as a covariance readout that unifies in-context learning via population gradient descent and repetitive generation via asymptotic Markov behavior.

  40. From Mechanistic to Compositional Interpretability

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    The paper introduces compositional interpretability as a category-theoretic framework that casts mechanistic explanations as commuting syntactic-semantic mappings optimized under faithfulness and complexity constraint...

  41. Tracing the Dynamics of Refusal: Exploiting Latent Refusal Trajectories for Robust Jailbreak Detection

    cs.CR 2026-05 unverdicted novelty 7.0 of 10

    Refusal in LLMs leaves a detectable upstream trajectory that SALO exploits to raise jailbreak detection from near zero to over 90 percent even under forced-decoding attacks.

  42. Screening Is Enough

    cs.LG 2026-04 unverdicted novelty 7.0 of 10

    Multiscreen replaces softmax attention with screening to provide absolute query-key relevance, resulting in models with 30% fewer parameters that maintain stable performance at long contexts.

  43. Sharp Capacity Scaling of Spectral Optimizers in Learning Associative Memory

    cs.LG 2026-03 unverdicted novelty 7.0 of 10

    Muon achieves higher storage capacity than SGD and matches Newton's method in one-step recovery rates for associative memory under power-law distributions, while saturating at larger critical batch sizes and showing f...

  44. Mechanistic Foundations of Goal-Directed Control

    cs.LG 2026-03 conditional novelty 7.0 of 10

    Context window k is the critical parameter for arbitration-gate formation in an embodied control architecture: no circuit below k≤4, resolved phase structure above k≥8, with EMA-like commitment dynamics.

  45. Discovering Interpretable Algorithms by Decompiling Transformers to RASP

    cs.LG 2026-02 conditional novelty 7.0 of 10

    Trained length-generalizing transformers on small algorithmic and formal-language tasks can be automatically decompiled into short, interpretable RASP-style programs, while non-generalizing models generally cannot.

  46. How Do Transformers Learn to Associate Tokens: Gradient Leading Terms Bring Mechanistic Interpretability

    cs.CL 2026-01 unverdicted novelty 7.0 of 10

    Transformer weights at early training stages are closed-form compositions of bigram, token-interchangeability, and context mappings that directly reflect text-corpus statistics and explain the emergence of semantic as...

  47. The Bayesian Geometry of Transformer Attention

    cs.LG 2025-12 unverdicted novelty 7.0 of 10

    Small transformers reproduce known Bayesian posteriors with 10^{-3} to 10^{-4} bit accuracy in verifiable wind-tunnel tasks via residual belief states, FFN updates, and attention routing, while MLPs do not.

  48. DecompSR: A dataset for decomposed analyses of compositional multihop spatial reasoning

    cs.AI 2025-11 unverdicted novelty 7.0 of 10

    DecompSR is a large, symbolically verified benchmark dataset and generation framework that independently varies productivity, substitutivity, overgeneralisation, and systematicity to probe compositional multihop spati...

  49. GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding

    cs.CV 2025-11 conditional novelty 7.0 of 10

    Supervising an MLLM's intrinsic self-attention with patch-level GUI labels, aggregated via a learnable anchor token and hidden-state-selected query tokens, reaches state-of-the-art 3B-scale GUI grounding accuracy with...

  50. Learning Pseudorandom Numbers with Transformers: Permuted Congruential Generators, Curricula, and Interpretability

    cs.LG 2025-10 conditional novelty 7.0 of 10

    Transformers can in-context predict PCG outputs on unseen parameters; required context length scales as sqrt(m), and curriculum training with smaller moduli is necessary for large moduli.

  51. Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scaling

    cs.LG 2025-09 conditional novelty 7.0 of 10

    Distilled pretraining improves test-time scaling via generation diversity but impairs induction-head-based in-context learning, with the trade-off explained by a bigram model analysis.

  52. Soft Head Selection for Injecting ICL-Derived Task Embeddings

    cs.CL 2025-07 conditional novelty 7.0 of 10

    SITE applies soft gradient-based head selection to inject ICL-derived task embeddings, outperforming prior embedding adaptation and few-shot ICL across generation, reasoning, and NLU tasks on 12 LLMs from 4B to 70B pa...

  53. A Markov Categorical Framework for Language Modeling

    cs.LG 2025-07 unverdicted novelty 7.0 of 10

    A Markov category framework for language models provides an information-theoretic rationale for speculative decoding and shows that a quadratic surrogate to negative log-likelihood induces generalized CCA alignment in...

  54. Decomposing Prediction Mechanisms for In-Context Recall

    cs.LG 2025-07 conditional novelty 7.0 of 10

    In a toy in-context recall task, label-based task initiation and observation-based continuation are distinct mechanisms with separate emergence times, and the same first-token versus second-token gap appears in an OLM...

  55. Can Gradient Descent Simulate Prompting?

    cs.CL 2025-06 conditional novelty 7.0 of 10

    A MAML-style meta-training objective makes a single gradient step on new text recover part of the performance that prompting achieves, on reversal-curse and passage-QA tasks.

  56. Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer

    cs.LG 2025-06 conditional novelty 7.0 of 10

    Freezing query and key attention weights still lets transformers form induction heads and stay close to standard performance on language modeling, while random static attention (MixiT) fails on in-context tasks but su...

  57. (How) Can Transformers Predict Pseudo-Random Numbers?

    cs.LG 2025-02 conditional novelty 7.0 of 10

    Transformers predict LCG sequences in-context for fixed moduli up to 2^32 and unseen moduli up to 2^16 by learning the modulus factorization and digit-wise periodic structure.

  58. Unveiling Simplicities of Attention: Adaptive Long-Context Head Identification

    cs.CL 2025-02 conditional novelty 7.0 of 10

    A second-moment Gaussian approximation of bulk key distributions lets a cheap query-adaptive test label attention heads as local or long-context, matching a full-attention oracle and outperforming static head pruning.

  59. Sparse Autoencoders Do Not Find Canonical Units of Analysis

    cs.LG 2025-02 conditional novelty 7.0 of 10

    Across different dictionary sizes, SAE latents are neither complete nor atomic, so SAEs do not learn a canonical set of features.

  60. A ghost mechanism: An analytical model of abrupt learning in recurrent networks

    cs.LG 2025-01 unverdicted novelty 7.0 of 10

    The ghost mechanism derives a 1D canonical model of abrupt learning in RNNs from ghost points of saddle-node bifurcations, predicting an inverse-power-law critical learning rate and gradient-based failure modes.

See all 234 Pith citations

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages · cited by 234 Pith papers (see all)

  1. [1]

    Language Models are Few-Shot Learners

    arXiv preprint arXiv:2005.14165. . LaMDA: our breakthrough conversation technology[link] Collins, E. and Ghahramani, Z.,

    work page Pith review arXiv 2005

  2. [2]

    Evaluating Large Language Models Trained on Code

    arXiv preprint arXiv:2107.03374. . Towards a human-like open-domain chatbot Adiwardana, D., Luong, M., So, D.R., Hall, J., Fiedel, N., Thoppilan, R., Yang, Z., Kulshreshtha, A., Nemade, G., Lu, Y. and others,,

    work page Pith review arXiv

  3. [3]

    arXiv preprint arXiv:2001.09977. . Scaling Language Models: Methods, Analysis & Insights from Training Gopher[PDF] Rae, J.W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., Rutherford, E., Hennigan, T., Menick, J., Cassirer, A., Powell, R., Driessche, G.v.d., Hendricks, L.A., Rauh, M., Huang...

    work page arXiv 2001

  4. [4]

    Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

    arXiv preprint arXiv:2201.02177. . The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models[link] Pan, A., Bhatia, K. and Steinhardt, J.,

    work page Pith review arXiv

  5. [5]

    Scaling Laws for Neural Language Models

    arXiv preprint arXiv:2001.08361. . Risks from Learned Optimization in Advanced Machine Learning Systems Hubinger, E., Merwijk, C.v., Mikulik, V., Skalse, J. and Garrabrant, S.,

    work page Pith review arXiv 2001

  6. [6]

    Shortformer: Better language modeling using shorter inputs

    arXiv preprint arXiv:2012.15832. . It wouldʼve been better if we had used a start of sequence token for the copying head evaluator as well, but we omitted it by mistake. Without the “start of sequence” token, some heads that were doing prefix matching on real data would get anomalously low scores on our test sequences[ ↩ ] ,,, ,, pp . A General Language A...

    work page arXiv 2012

  7. [7]

    A General Language Assistant as a Laboratory for Alignment

    arXiv preprint arXiv:2112.00861. . Common Crawl[link] Foundation, T.C.C.. . The Pile: An 800GB Dataset of Diverse Text for Language Modeling Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., Presser, S. and Leahy, C.,

    work page Pith review arXiv

  8. [8]

    A Multiscale Visualization of Attention in the Transformer Model

    arXiv preprint arXiv:1906.05714. . Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned[PDF] Voita, E., Talbot, D., Moiseev, F., Sennrich, R. and Titov, I.,

    work page Pith review arXiv 1906

Show all 25 references
  1. [9]

    What does bert look at? an analysis of bert's attention[PDF] Clark, K., Khandelwal, U., Levy, O

    arXiv preprint arXiv:1905.09418. . What does bert look at? an analysis of bert's attention[PDF] Clark, K., Khandelwal, U., Levy, O. and Manning, C.D.,

    1905

  2. [10]

    Do attention heads in bert track syntactic dependencies?[PDF] Htut, P.M., Phang, J., Bordia, S

    arXiv preprint arXiv:1906.04341. . Do attention heads in bert track syntactic dependencies?[PDF] Htut, P.M., Phang, J., Bordia, S. and Bowman, S.R.,

    1906

  3. [11]

    Attention is not all you need: Pure attention loses rank doubly exponentially with depth[PDF] Dong, Y., Cordonnier, J

    arXiv preprint arXiv:1911.12246. . Attention is not all you need: Pure attention loses rank doubly exponentially with depth[PDF] Dong, Y., Cordonnier, J. and Loukas, A.,

    1911

  4. [12]

    What Context Features Can Transformer Language Models Use? O'Connor, J

    arXiv preprint arXiv:2103.03404. . What Context Features Can Transformer Language Models Use? O'Connor, J. and Andreas, J.,

  5. [13]

    An Explanation of In-context Learning as Implicit Bayesian Inference Xie, S.M., Raghunathan, A., Liang, P

    arXiv preprint arXiv:2106.08367. . An Explanation of In-context Learning as Implicit Bayesian Inference Xie, S.M., Raghunathan, A., Liang, P. and Ma, T.,

  6. [14]

    Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H

    arXiv preprint arXiv:2111.02080. . Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H. and Zettlemoyer, L.,

  7. [15]

    Reconciling modern machine-learning practice and the classical bias--variance trade-off Belkin, M., Hsu, D., Ma, S

    arXiv preprint arXiv:2202.12837. . Reconciling modern machine-learning practice and the classical bias--variance trade-off Belkin, M., Hsu, D., Ma, S. and Mandal, S.,

  8. [16]

    Journal of Statistical Mechanics: Theory and Experiment, Vol 2021(12), pp. 124003. IOP Publishing. . Future ML Systems Will Be Qualitatively Different[link] Steinhardt, J.,

    2021

  9. [17]

    Analyzing monotonic linear interpolation in neural network loss landscapes Lucas, J., Bae, J., Zhang, M.R., Fort, S., Zemel, R

    arXiv preprint arXiv:1412.6544. . Analyzing monotonic linear interpolation in neural network loss landscapes Lucas, J., Bae, J., Zhang, M.R., Fort, S., Zemel, R. and Grosse, R.,

  10. [18]

    Geometry of neural network loss surfaces via random matrix theory Pennington, J

    arXiv preprint arXiv:2104.11044. . Geometry of neural network loss surfaces via random matrix theory Pennington, J. and Bahri, Y.,

  11. [19]

    DOI: 10.23915/distill.00024.001

    Distill. DOI: 10.23915/distill.00024.001 . Convergent learning: Do different neural networks learn the same representations? Li, Y., Yosinski, J., Clune, J., Lipson, H., Hopcroft, J.E. and others,,

    doi

  12. [20]

    High-Low Frequency Detectors Schubert, L., Voss, C., Cammarata, N., Goh, G

    arXiv preprint arXiv:1905.00414. . High-Low Frequency Detectors Schubert, L., Voss, C., Cammarata, N., Goh, G. and Olah, C.,

    1905

  13. [21]

    DOI: 10.23915/distill.00024.005

    Distill. DOI: 10.23915/distill.00024.005 . Performance-optimized hierarchical models predict neural responses in higher visual cortex Yamins, D.L., Hong, H., Cadieu, C.F., Solomon, E.A., Seibert, D. and DiCarlo, J.J.,

    doi

  14. [22]

    DOI: 10.23915/distill.00030

    Distill. DOI: 10.23915/distill.00030 . Neural machine translation by jointly learning to align and translate Bahdanau, D., Cho, K. and Bengio, Y.,

    doi

  15. [23]

    Listen, attend and spell Chan, W., Jaitly, N., Le, Q.V

    arXiv preprint arXiv:1409.0473. . Listen, attend and spell Chan, W., Jaitly, N., Le, Q.V. and Vinyals, O.,

  16. [24]

    induction bump

    arXiv preprint arXiv:1508.01211. Acknowledgments In writing this paper, our thinking and exposition was greatly clarified by detailed correspondence with Sam Bowman, Paul Christiano, Aidan Gomez, Dan Hendrycks, Jacob Hilton, Evan Hubinger, Andrew Ilyas, Percy Liang, Tom Lieber...

  17. [25]

    copying”, “prefix matching

    BibTeX Citation: @article{olsson2022context, title={In-context Learning and Induction Heads}, author={Olsson, Catherine and Elhage, Nelson and Nanda, Neel and Joseph, Nicholas and DasSarma, Nova and Henighan, Tom and Mann, Ben and Askell, Amanda and Bai, Yuntao and Chen, Anna ...

    2022

Pith tools

Reviewed May 11, 2026 · model on record in the stance chip above.