Tracing LLM Behavior to the Training Data with Empirical Next-Token Distributions

Zachary Izzo

REVIEW 2 major objections 6 minor 12 references

About 20% of LLM next-token distributions match the training corpus's empirical next-token distribution almost exactly, with agreement increasing as models scale.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 01:38 UTC pith:7WAP2BCD

load-bearing objection Careful measurement study with a real but narrower-than-advertised central claim; the 20% figure is an artifact of stratified sampling, but the per-example dynamics and positional invariance are solid and worth an editor's time. the 2 major comments →

arxiv 2607.14306 v2 pith:7WAP2BCD submitted 2026-07-15 cs.AI

Tracing LLM Behavior to the Training Data with Empirical Next-Token Distributions

Zachary Izzo This is my paper

classification cs.AI

keywords empirical next-token distributiontraining data attributionlanguage model interpretabilitytotal variation distancescaling lawssoftmax bottleneckpositional invariance

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tests how much of an LLM's next-token behavior can be explained solely by the raw empirical statistics of its training data. It compares each model's predicted distribution to the empirical next-token distribution (ENTD)—the exact minimizer of the pretraining cross-entropy loss—on a stratified sample of training contexts. The central finding is that the model matches the ENTD almost perfectly on roughly one in five inputs (total-variation distance ≤ 0.0056), and this agreement grows with model scale and training compute. The remaining inputs show a long tail of discrepancies with identifiable sources: the model is positionally invariant, its softmax head undersmooths relative to a low-rank approximation, optimization randomness plays a smaller role at scale, and frequent short contexts with high empirical entropy are the hardest to fit. The paper positions this as a step toward explaining model behavior from the data rather than from learned weights.

Core claim

On the paper's own terms, the discovery is that a trained transformer's next-token distribution is, for a significant fraction of training-set contexts, essentially the empirical next-token distribution of the corpus—the distribution that exactly minimizes the next-token cross-entropy loss. Using total-variation distance as the metric, about 20% of a stratified evaluation sample has TV ≤ 0.0056, and the match improves with scale and compute. The paper also establishes three negative/positive results about the deviations: the match holds against the standard ENTD rather than the training-aligned variant (a positional invariance that rules out batch-alignment artifacts); a fitted rank-limited

What carries the argument

The empirical next-token distribution (ENTD), defined as count(x_{1:k+1})/count(x_{1:k}) over the training corpus. Because it is the exact global minimizer of the next-token cross-entropy loss, it is the 'memorized' target the model is trained toward; the paper measures total-variation distance between the model's output distribution and this target, and also uses a training-aligned variant (ENTD-T) and offset versions to isolate positional effects, plus a rank-limited logistic approximation to separate architectural capacity from data structure.

Load-bearing premise

The evaluation sample is stratified to give equal counts to each (context length, corpus frequency) bin, so the headline 20% near-perfect match and the tail statistics describe this balanced sample—not the natural distribution of tokens the model sees—and may not generalize to typical training tokens.

What would settle it

Take the natural (unstratified) distribution of training contexts for the largest model, or a uniformly random sample of tokens from the corpus, and compute the fraction of contexts with TV ≤ 0.0056 between the model's next-token distribution and the ENTD; if that fraction is far below 20%, the central quantitative claim is an artifact of the stratified sample rather than a property of typical inputs.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

On training-set contexts, a substantial share of an LLM's next-token behavior is reproducible from simple corpus counts, so data curation decisions directly determine a larger part of model outputs than usually assumed.
Because the model is positionally invariant, the standard ENTD—not the batch-aligned variant—is the correct reference distribution for studying pretraining convergence and for data-centric interpretability.
The low-rank approximation result implies transformers systematically oversmooth the empirical distribution; architectures that preserve higher-rank logits could improve faithfulness on ambiguous contexts.
The high-TV tail is stable at scale and predictable from data statistics (frequency, entropy, length), so troublesome contexts could be identified before training and targeted for data augmentation or architectural changes.
Agreement increasing with compute means ENTD distance could serve as a cheap, corpus-based proxy for how well a training run has converged to the data distribution.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If the ENTD match continues to improve with scale, then in the limit a large enough model might be fully predictable on all frequent contexts from corpus statistics alone; a testable prediction is that the 20% figure rises monotonically on frontier-scale runs.
The stratified sample makes the 20% figure a property of a length/frequency-balanced set, not of the natural token stream; on typical training tokens the match could be far lower, so the practical coverage of ENTD-based explanations remains an open empirical question.
The failure on high-entropy frequent contexts is essentially a calibration problem; this suggests post-hoc calibration methods may be unable to fix such contexts, and architectural rank increases are a more direct cure.
The same ENTD-based discrepancy decomposition could be applied to post-training phases: comparing an instruction-tuned model's deviations from the base ENTD would separate the effects of alignment from those of pretraining data.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Careful measurement study with a real but narrower-than-advertised central claim; the 20% figure is an artifact of stratified sampling, but the per-example dynamics and positional invariance are solid and worth an editor's time.

read the letter

The first thing to know: this paper is not the breakthrough it frames itself as, but it's a genuinely useful measurement study. The core finding—that on a carefully balanced sample of training contexts, models match the empirical next-token distribution almost perfectly for 20% of inputs, with a long tail of mismatches—is real and documented carefully. What's actually new is the multi-scale Pythia sweep (70M to 12B), the training-aligned ENTD-T formulation, the positional-invariance result, and the observation that models struggle with frequent high-entropy contexts. The positional-invariance result is the strongest piece: it's counterintuitive, and the authors validate it with a brute-force index in Appendix B. That deserves credit.

The soft spot is the one the stress-test highlights. The 20% headline is computed on a stratified sample that deliberately oversamples high-count contexts. The paper itself says a uniform random sample of training positions would put at most ~1% of examples outside the point-mass regime, where near-perfect agreement is essentially impossible. So the abstract's "significant fraction of inputs" is only true for a non-representative evaluation set. The authors are transparent about this in §3.2, but the abstract and Figure 1 don't carry the caveat. That's a framing problem, not a fatal one: the paper's real contribution is the characterization of per-example behavior within this balanced sample, which is a legitimate complement to aggregate scaling laws. But the scope needs to be stated honestly.

Two smaller issues. The low-rank logit comparison in §5 fits U and W on the evaluation sample itself, so it's an in-sample upper bound on the softmax bottleneck; the authors acknowledge the sample-size gap, but it would be easy for readers to over-read this as evidence about the architecture. And no code or artifacts are released, so exact replication is impossible—the brute-force validation helps, but doesn't substitute for a public implementation.

The circularity concern is real but not disqualifying. Yes, the ENTD is the loss minimizer, so average agreement is partly built in. But the per-example trajectories, the tail analysis, and the finding that the model doesn't overfit rare point-mass contexts are empirical observations that don't follow from the training objective alone.

Bottom line: this paper deserves a serious referee. It's not a paradigm shift, but it's a solid data-centric interpretability measurement that people working on memorization and data curation will want to cite. The authors should be asked to fix the sampling caveat in the abstract, report a natural-stream estimate (or say explicitly why it's not meaningful), and release code. I'd send it to review rather than desk reject.

Referee Report

2 major / 6 minor

Summary. This paper studies the relationship between a pretrained LLM's next-token distribution and the empirical next-token distribution (ENTD) of its training corpus. Using the Pythia suite and an infinity-gram (suffix-array) counter, the authors compute ENTDs for a large, frequency-length stratified sample of training contexts and compare them to model output distributions using total variation (TV) distance. They report that 20% of this stratified sample shows near-perfect agreement (TV ≤ 0.0056) with the ENTD, that agreement improves with model scale and training compute, and that a long tail of contexts remains highly discrepant. The paper then investigates candidate sources of discrepancy: positional alignment of training contexts, the softmax bottleneck, optimization randomness, and contextual frequency/entropy. It also traces per-example TV trajectories across training.

Significance. If the findings hold, this paper offers a valuable data-centric complement to mechanistic interpretability by characterizing how model predictions relate to the empirical statistics of the training corpus. The use of public checkpoints, reproducible training data ordering, and brute-force verification of the infinity-gram index are notable strengths that increase confidence in the measurements. The per-example distributional analysis goes beyond aggregate scaling laws and reveals a more nuanced picture of LLM behavior. The paper's main contributions are likely to spur further work on data-centric interpretability. However, the interpretation of the headline result depends critically on the evaluation sampling design, as detailed below.

major comments (2)

[Abstract, §3.2, Fig. 1] The abstract claims the LLM agrees with the ENTD 'almost perfectly' for a 'significant fraction of inputs', with Fig. 1 giving the 20% figure. This sample is constructed by equal-count stratified sampling over (occurrence-count, length) bins (§3.2). The paper itself notes that a uniformly random training sample would have at most 20/2048 ≈ 1% of contexts outside the point-mass ENTD regime; for the remaining ≥99% of natural contexts, near-perfect agreement is essentially impossible. The 20% figure is therefore a property of the balanced sample, not of the natural token stream. The abstract and introduction should qualify that the claim applies to the stratified sample, and ideally the paper should also report behavior under the natural training distribution (e.g., TV by context length) to quantify the relationship for typical training tokens. Without this, the central claim overstates the
[§5, Fig. 4] The low-rank logit approximation is computed with a fixed rank of 2560 (the d_model of the 2.8B model) for all models. For smaller Pythia models (e.g., 70M with d_model=512, 410M with 1024, 1B with 2048), this rank exceeds the model's d_model. The softmax bottleneck constrains the logit matrix to rank at most d_model, so the rank-2560 approximation is not a valid upper bound on the bottleneck bias for those models. Consequently, the claim that 'the transformer does not take full advantage' of low-rank structure is not supported for the smaller models. The authors should either compute per-model rank limits or restrict the claim to models with d_model ≥ 2560.

minor comments (6)

[General] Total variation distance is used throughout but never defined. Define it at first use, e.g., TV(p,q) = ½ Σ_v |p(v) − q(v)|.
[§5] The low-rank approximation is trained on the same evaluation sample (84K examples) used for the model comparison. The paper acknowledges this asymmetry, but the phrase 'the transformer does not take full advantage of this fact' should be further tempered; the result is best described as an upper bound on the softmax-bottleneck bias, as stated at the beginning of the section.
[§4] The 'strong positional invariance' conclusion is based on the similarity of TV distributions across offsets. This is distributional evidence; a direct comparison of model probabilities for the same content at different absolute positions would be more conclusive. The two-batch control is a useful sanity check, but the current evidence does not strictly prove that the model is positionally invariant for individual inputs.
[§3.2] The statement that a uniform sample would contain 'a fraction of at most 20/2048 ≈ 1% of examples outside of this regime' needs a brief explanation of how the 20/2048 bound is derived, as it is not immediately obvious.
[Fig. 1] The choice of TV ≤ 0.0056 as the threshold for 'near-perfect' agreement is arbitrary. Specify how this threshold was selected.
[Notation] The paper uses '∞-gram' to refer to the method of Liu et al. (2024), but the reference and official name is 'Infini-gram'. Use consistent notation to avoid confusion.

Circularity Check

0 steps flagged

No significant circularity: the definitional fact that the ENTD minimizes the pretraining cross-entropy loss is used as motivation, but the paper's per-example, tail, scaling, and invariance analyses are independent empirical measurements.

full rationale

The only definitional tie is Eq. (2), where p_ENTD is defined as an empirical count ratio and then shown to be the global minimizer of the cross-entropy objective (1). This is not a circular derivation: the paper does not fit any parameter to the model and relabel it as a prediction. The central findings—fraction of inputs with TV ≤ 0.0056, the TV CDF, long-tail decomposition, scaling trends, positional invariance, optimization-variance decomposition, and the softmax-bottleneck upper bound—are computed from independent sources: Pythia/PolyPythias checkpoints and corpus counts from infini-gram, with brute-force verification in Appendix B. The stratified sampling in §3.2 is explicitly a scope choice; the manuscript states that a uniform sample would be at most ~20/2048 ≈ 1% outside the point-mass regime. This limits external validity but does not make the comparison circular. The one self-citation (Izzo et al. 2026) appears only in Related Work and is not load-bearing for any result. No fitted input is called a prediction, no uniqueness theorem is imported, and no ansatz is smuggled in via citation. The average agreement with the ENTD is acknowledged to be essentially equivalent to scaling laws, but the paper's contribution is the per-example distribution around that average, which is not determined by the training objective.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 1 invented entities

The paper rests on treating corpus counts and the stratified sample as ground truth, with the ENTD defined as the exact training-loss minimizer. The main free choices are the sample-design parameters and the in-sample low-rank fit; the ENTD-T is an invented benchmark that turns out to be largely redundant due to positional invariance.

free parameters (3)

Stratified sampling design = Count bins [1e0,1e1)...[1e6,∞); lengths {1,2,4,6,8,12,16,32,64}; reservoir size N=100 per (k, shard-bin); 21 shards
These hand-chosen parameters determine the composition of the evaluation sample and therefore the headline '20%' fraction and all TV distributions.
Tail cutoffs for high/low TV analysis = Top and bottom 10%
The 'long tail' and 'best-fit' analyses in Table 1 and Figure 7 are defined by a 10% threshold; the authors provide a sensitivity analysis in Appendix D, but the threshold is a free choice.
Low-rank logit approximation factors U,W = Not reported; rank fixed to d_model (e.g., 2560 for the 2.8B model)
U and W are optimized directly on the 84K evaluation sample to minimize cross-entropy against the ENTD targets. This is an in-sample fit, not a predictive model, and it drives the softmax-bottleneck conclusion in §5.

axioms (6)

standard math The empirical next-token distribution (ENTD) is the exact global minimizer of the next-token cross-entropy loss when pθ is unrestricted.
Used in §2.1 and throughout to justify ENTD as the natural training target.
domain assumption Suffix-array based infinity-gram counts over the full Pile are exact for arbitrary-length n-grams.
Invoked in §2.3; the ENTD and ENTD-T counts are treated as ground-truth corpus statistics.
domain assumption Stratified sampling with equal numbers per (occurrence-count, length) bin is an appropriate representation of model behavior.
Invoked in §3.2; the resulting evaluation set is not corpus-representative, yet it is used for all headline fractions.
domain assumption Total variation distance between the model NTD and ENTD is a meaningful per-sample measure of agreement.
Used in all experiments as the discrepancy metric; assumes TV captures the notion of 'agreement' for head and tail analysis.
domain assumption The Pythia dataloader batches contexts as consecutive non-overlapping chunks, with loss computed on prefixes of each chunk, so ENTD-T is the training-aligned target.
Definition of ENTD-T in §2.2 and the positional-invariance experiments depend on this description of the training procedure.
standard math Transformer output logits have rank at most d_model (the softmax bottleneck).
Used in §5 to construct the low-rank approximation baseline.

invented entities (1)

Training-aligned empirical next-token distribution (ENTD-T) no independent evidence
purpose: A benchmark distribution that counts only contexts starting at training batch boundaries, intended to be the exact training-loss minimizer under batch alignment.
Defined in §2.2 as a new mathematical construction. It is not independently observable, and the paper later shows it does not explain model behavior better than the standard ENTD due to positional invariance.

pith-pipeline@v1.3.0-alltime-deepseek · 13325 in / 16829 out tokens · 163298 ms · 2026-08-04T01:38:43.627819+00:00 · methodology

0 comments

read the original abstract

In this paper, we study the connection between an LLM's output distribution and the data used to train it. Specifically, we study the degree to which an LLM's next-token distribution agrees with the empirical next-token distribution (ENTD) given the context in the training data. The ENTD is an appealing target because it is the unrestricted global minimizer of the next-token cross entropy loss used for pretraining, as well as an easily interpretable function of the pretraining corpus. We find that for a significant fraction of inputs, the LLM's distribution agrees with the ENTD almost perfectly, and the agreement generally increases with model scale and training compute. Nevertheless, there is a long tail of input sequences where the LLM and ENTD differ significantly, and we examine several possible sources of this discrepancy across the transformer architecture, training procedure, and finite-sample noise in the ENTD estimate itself. More broadly, we hope our findings will encourage more work on ``data-centric mechanistic interpretability,'' a complement to standard mechanistic interpretability that opens the black box of how model behaviors arise from the data, rather than how they are encoded in the learned weights.

Figures

Figures reproduced from arXiv: 2607.14306 by Zachary Izzo.

**Figure 2.** Figure 2: Distribution of discrepancies between the model and the ENTD (left) vs. ENTD-T (right). In spite [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 2.** Figure 2: Distribution of discrepancies between the model and the ENTD (left) vs. ENTD-T (right). In spite [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Figure 3: Discrepancies between the LLM and training-aligned empirical next-token distributions with [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

**Figure 4.** Figure 4: TV distributions on the ENTD evaluation sample for all models over training, plus the best [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

**Figure 5.** Figure 5: Model-ENTD deviations vs. model-model deviations. Discrepancies between each model and the [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

**Figure 6.** Figure 6: Model vs. ENTD discrepancies, stratified by evaluation sample length. The discrepancy distribution [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗

**Figure 7.** Figure 7: Stability of the high-TV tails, given by IoU of the top 10% TV examples for each checkpoint with [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗

**Figure 8.** Figure 8: Individual TV trajectories for a small (70M, left) and large (2.8B, right) model. The trajectories [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗

**Figure 9.** Figure 9: Batch-aligned vs. offset discrepancies after extended training on only two batches, and with the [PITH_FULL_IMAGE:figures/full_fig_p017_9.png] view at source ↗

**Figure 10.** Figure 10: Tail stability analysis. The shape of the tail stability curve is stable across choices of the cutoff. [PITH_FULL_IMAGE:figures/full_fig_p017_10.png] view at source ↗

**Figure 11.** Figure 11: Individual TV trajectories for a small (left) and large (right) model, input length [PITH_FULL_IMAGE:figures/full_fig_p018_11.png] view at source ↗

**Figure 11.** Figure 11: Individual TV trajectories for a small (left) and large (right) model, input length [PITH_FULL_IMAGE:figures/full_fig_p017_11.png] view at source ↗

**Figure 12.** Figure 12: Individual TV trajectories for a small (left) and large (right) model, input length [PITH_FULL_IMAGE:figures/full_fig_p018_12.png] view at source ↗

**Figure 12.** Figure 12: Individual TV trajectories for a small (left) and large (right) model, input length [PITH_FULL_IMAGE:figures/full_fig_p017_12.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Reference graph

Works this paper leans on

12 extracted references · 6 linked inside Pith

[5]

The pile: An 800gb dataset of diverse text for language modeling

Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027,

Pith/arXiv arXiv
[7]

Why larger models learn more: Effects of capacity, interference, and rare-task retention.arXiv preprint arXiv:2605.29548,

Jing Huang, Daniel Wurgaft, Rachit Bansal, Laura Ruis, Naomi Saphra, David Alvarez-Melis, Andrew Kyle Lampinen, Christopher Potts, and Ekdeep Singh Lubana. Why larger models learn more: Effects of capacity, interference, and rare-task retention.arXiv preprint arXiv:2605.29548,

Pith/arXiv arXiv
[8]

In-context learning and induction heads.arXiv preprint arXiv:2209.11895,

Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context learning and induction heads.arXiv preprint arXiv:2209.11895,

Pith/arXiv arXiv
[9]

Backtracking mathematical reasoning of language models to the pretraining data

Yasaman Razeghi, Hamish Ivison, Sameer Singh, and Yanai Elazar. Backtracking mathematical reasoning of language models to the pretraining data. InThe Second Tiny Papers Track at ICLR 2024,

2024
[10]

Polypythias: Stability and outliers across fifty language model pre-training runs

Oskar Van der Wal, Pietro Lesci, Max Müller-Eberstein, Naomi Saphra, Hailey Schoelkopf, Willem Zuidema, and Stella R Biderman. Polypythias: Stability and outliers across fifty language model pre-training runs. InInternational Conference on Learning Representations, volume 2025, pp. 86180–86204,

2025
[2018]

Jiasheng Ye, Peiju Liu, Tianxiang Sun, Jun Zhan, Yunhua Zhou, and Xipeng Qiu

URL https://openreview.net/forum?id=HkwZSG-CZ. Jiasheng Ye, Peiju Liu, Tianxiang Sun, Jun Zhan, Yunhua Zhou, and Xipeng Qiu. Data mixing laws: Optimizing data mixtures by predicting language modeling performance. InInternational Conference on Learning Representations, volume 2025, pp. 82263–82287,

2025
[2020]

Studying large language model generalization with influence functions.arXiv preprint arXiv:2308.03296,

Roger Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, et al. Studying large language model generalization with influence functions.arXiv preprint arXiv:2308.03296,

Pith/arXiv arXiv
[2022]

Stealing part of a production language model.arXiv preprint arXiv:2403.06634,

Nicholas Carlini, Daniel Paleka, Krishnamurthy Dj Dvijotham, Thomas Steinke, Jonathan Hayase, A Feder Cooper, Katherine Lee, Matthew Jagielski, Milad Nasr, Arthur Conmy, et al. Stealing part of a production language model.arXiv preprint arXiv:2403.06634,

Pith/arXiv arXiv
[2023]

Language contamination helps explains the cross-lingual capabilities of english pretrained models

Terra Blevins and Luke Zettlemoyer. Language contamination helps explains the cross-lingual capabilities of english pretrained models. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 3563–3574,

2022
[2024]

Pretraining data statistics shape the phases of learning entity comparison in language models

Yik Siu Chan, Jing Huang, Yanai Elazar, and Atticus Geiger. Pretraining data statistics shape the phases of learning entity comparison in language models. InMechanistic Interpretability Workshop at ICML 2026,

2026
[2025]

Jason Wei, Dan Garrette, Tal Linzen, and Ellie Pavlick

URL https://openreview.net/forum?id=IQxBDLmVpT. Jason Wei, Dan Garrette, Tal Linzen, and Ellie Pavlick. Frequency effects on syntactic rule learning in transformers. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 932–948,

2021
[2026]

Measuring causal effects of data statistics on language model’sfactual’predictions.arXiv preprint arXiv:2207.14251,

Yanai Elazar, Nora Kassner, Shauli Ravfogel, Amir Feder, Abhilasha Ravichander, Marius Mosbach, Yonatan Belinkov, Hinrich Schütze, and Yoav Goldberg. Measuring causal effects of data statistics on language model’sfactual’predictions.arXiv preprint arXiv:2207.14251,

Pith/arXiv arXiv

[1] [5]

The pile: An 800gb dataset of diverse text for language modeling

Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027,

Pith/arXiv arXiv

[2] [7]

Why larger models learn more: Effects of capacity, interference, and rare-task retention.arXiv preprint arXiv:2605.29548,

Jing Huang, Daniel Wurgaft, Rachit Bansal, Laura Ruis, Naomi Saphra, David Alvarez-Melis, Andrew Kyle Lampinen, Christopher Potts, and Ekdeep Singh Lubana. Why larger models learn more: Effects of capacity, interference, and rare-task retention.arXiv preprint arXiv:2605.29548,

Pith/arXiv arXiv

[3] [8]

In-context learning and induction heads.arXiv preprint arXiv:2209.11895,

Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context learning and induction heads.arXiv preprint arXiv:2209.11895,

Pith/arXiv arXiv

[4] [9]

Backtracking mathematical reasoning of language models to the pretraining data

Yasaman Razeghi, Hamish Ivison, Sameer Singh, and Yanai Elazar. Backtracking mathematical reasoning of language models to the pretraining data. InThe Second Tiny Papers Track at ICLR 2024,

2024

[5] [10]

Polypythias: Stability and outliers across fifty language model pre-training runs

Oskar Van der Wal, Pietro Lesci, Max Müller-Eberstein, Naomi Saphra, Hailey Schoelkopf, Willem Zuidema, and Stella R Biderman. Polypythias: Stability and outliers across fifty language model pre-training runs. InInternational Conference on Learning Representations, volume 2025, pp. 86180–86204,

2025

[6] [2018]

Jiasheng Ye, Peiju Liu, Tianxiang Sun, Jun Zhan, Yunhua Zhou, and Xipeng Qiu

URL https://openreview.net/forum?id=HkwZSG-CZ. Jiasheng Ye, Peiju Liu, Tianxiang Sun, Jun Zhan, Yunhua Zhou, and Xipeng Qiu. Data mixing laws: Optimizing data mixtures by predicting language modeling performance. InInternational Conference on Learning Representations, volume 2025, pp. 82263–82287,

2025

[7] [2020]

Studying large language model generalization with influence functions.arXiv preprint arXiv:2308.03296,

Roger Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, et al. Studying large language model generalization with influence functions.arXiv preprint arXiv:2308.03296,

Pith/arXiv arXiv

[8] [2022]

Stealing part of a production language model.arXiv preprint arXiv:2403.06634,

Nicholas Carlini, Daniel Paleka, Krishnamurthy Dj Dvijotham, Thomas Steinke, Jonathan Hayase, A Feder Cooper, Katherine Lee, Matthew Jagielski, Milad Nasr, Arthur Conmy, et al. Stealing part of a production language model.arXiv preprint arXiv:2403.06634,

Pith/arXiv arXiv

[9] [2023]

Language contamination helps explains the cross-lingual capabilities of english pretrained models

Terra Blevins and Luke Zettlemoyer. Language contamination helps explains the cross-lingual capabilities of english pretrained models. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 3563–3574,

2022

[10] [2024]

Pretraining data statistics shape the phases of learning entity comparison in language models

Yik Siu Chan, Jing Huang, Yanai Elazar, and Atticus Geiger. Pretraining data statistics shape the phases of learning entity comparison in language models. InMechanistic Interpretability Workshop at ICML 2026,

2026

[11] [2025]

Jason Wei, Dan Garrette, Tal Linzen, and Ellie Pavlick

URL https://openreview.net/forum?id=IQxBDLmVpT. Jason Wei, Dan Garrette, Tal Linzen, and Ellie Pavlick. Frequency effects on syntactic rule learning in transformers. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 932–948,

2021

[12] [2026]

Measuring causal effects of data statistics on language model’sfactual’predictions.arXiv preprint arXiv:2207.14251,

Yanai Elazar, Nora Kassner, Shauli Ravfogel, Amir Feder, Abhilasha Ravichander, Marius Mosbach, Yonatan Belinkov, Hinrich Schütze, and Yoav Goldberg. Measuring causal effects of data statistics on language model’sfactual’predictions.arXiv preprint arXiv:2207.14251,

Pith/arXiv arXiv