Pith. sign in

REVIEW 5 major objections 5 minor 60 references

This paper defines an LLM output as novel when no semantically similar context exists anywhere in the pretraining corpus, and measures that at trillion-token scale with a retrieval pipeline.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 06:58 UTC pith:4IAT5K5K

load-bearing objection A useful conceptual flip and substantial released artifacts, but the central semantic-similarity measure is unvalidated, so the empirical headlines should be read as descriptive of the pipeline, not of the models. the 5 major comments →

arxiv 2510.27313 v3 pith:4IAT5K5K submitted 2025-10-31 cs.LG cs.AIcs.CL

LLM generation novelty through the lens of semantic similarity

Philipp Davydov , Ameya Prabhu , Matthias Bethge , Elisa Nguyen , Seong Joon Oh This is my paper
classification cs.LG cs.AIcs.CL
keywords semantic noveltyun-attributabilitytraining data attributiontext retrievalColBERTv2GIST embeddingsinstruction tuningSmolLM
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that novelty of an LLM output can be defined and measured as un-attributability: an output is novel if the pretraining corpus contains no semantically similar context. It builds a retrieval pipeline that scales to full pretraining corpora and shows that, by this measure, SmolLM models reuse training data across much longer spans than n-gram methods report, that novelty is strongly domain-dependent, and that instruction tuning genuinely increases novelty rather than only changing style. A sympathetic reader would care because it converts an intractable attribution problem into a tractable retrieval problem and offers a concrete, auditable way to study when models generalize as opposed to reuse.

Core claim

The central claim is the inversion of training-data attribution: instead of asking which training samples caused an output, ask whether any training sample is close enough to explain it. The test computes GIST embeddings of corpus chunks, retrieves the nearest 100 chunks, reranks them with ColBERTv2, and compares the best-match similarity against a human-written baseline; a generation is deemed novel when its similarity ratio falls below that baseline. Applied to SmolLM and SmolLM2, the claim produces three findings: reuse extends over longer sequences than lexical metrics show, novelty varies by domain, and instruction-tuned models produce more semantically unattributable text.

What carries the argument

Un-attributability, an operational definition of semantic novelty: an output is novel if no semantically similar context exists in the pretraining corpus. It is realized by a two-stage retrieval pipeline — GIST embeddings indexed in a vector store for top-100 retrieval, then ColBERTv2 reranking at multiple chunk sizes — with scores normalized by query length and calibrated against human-written reference text. The pipeline converts a scale-restricted attribution question into a nearest-neighbor search that can run over trillion-token corpora.

Load-bearing premise

The whole measure assumes that GIST cosine similarity plus ColBERTv2 reranking faithfully captures 'semantically similar context' — if it misses paraphrases or is biased by style or length, all three findings describe embedding geometry rather than actual reuse.

What would settle it

A concrete check: take a set of model generations, have humans decide for each whether the pretraining corpus contains a semantically equivalent passage, then compare those judgments to the pipeline's novelty scores. If agreement is near chance, or if rerunning the whole experiment with a different embedding model reverses the domain and instruction-tuning conclusions, the central claim collapses.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • N-gram and other lexical novelty measures systematically miss paraphrase-level reuse, so earlier estimates of how much models copy should be revisited.
  • Longer generated sequences are less attributable than short ones in unprompted generation, suggesting models generalize increasingly as generation proceeds.
  • Novelty is not a single model property: it varies by task domain, with factual-reasoning answers often more novel than math or rewriting answers.
  • Instruction tuning increases semantic novelty beyond stylistic changes, implying it teaches compositional or diversifying behavior.
  • Smaller models in the SmolLM2 family are more novel than their larger counterparts, consistent with greater semantic memorization in larger models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same un-attributability test could serve as a data-contamination audit: outputs that score far above the baseline would flag corpus overlap even when phrased differently.
  • If the embedding model is changed, the novelty ordering across domains may shift; a stability check across embedders would determine how much of the result is measure-specific.
  • The method could extend to models whose corpora are closed by retrieving from public repositories or sub-corpora, but then novelty exclusions no longer refer to the true pretraining set.
  • A direct human validation study, asking annotators whether a retrieved corpus chunk is a paraphrase of a generation, would test whether the operational definition matches intuitive novelty.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes to measure LLM generation novelty by semantic un-attributability: an output is novel if the pretraining corpus contains no semantically similar context. They implement this with a FAISS index of GIST embeddings followed by ColBERTv2 reranking, and calibrate against human-written baseline text. Applying this to SmolLM and SmolLM2, they report three findings: (1) models draw on pretraining data over much longer spans than n-gram methods previously reported, (2) novelty varies by task domain, and (3) instruction tuning increases novelty. They release ~20 TB of corpus chunks and index artifacts.

Significance. If the metric is valid, the paper offers a scalable and style-robust alternative to lexical novelty measures and makes a useful conceptual inversion of training-data attribution. The released ~20 TB of corpus chunks and index artifacts are a substantial reproducibility contribution, and the pipeline is clearly specified. However, the central empirical claims rest on an unvalidated retrieval recall assumption and an aggregation choice that can hide attributable chunks; the contribution is therefore conditional on additional validation.

major comments (5)
  1. [§3, Algorithm 1] The abstract defines novelty as 'the pretraining corpus contains no semantically similar context,' but Algorithm 1 outputs N^(k) ← median(R^(k)). A single verbatim or semantically copied chunk inside a long output is hidden by the median, so an output containing a close corpus match can still be labeled novel. This mismatch is load-bearing for all three findings. Please either switch to a stricter aggregation (e.g., maximum over chunks or the fraction of chunks above a threshold) and re-run the analysis, or justify why the median is the appropriate operationalization of 'no close match.'
  2. [§3.1, Appendix A] The sufficiency check for n=100 only counts how often low FAISS ranks are promoted by ColBERTv2 to the reranked top-1 among the retrieved candidates. It does not measure recall against known semantically similar contexts. If the GIST embedding misses a true neighbor entirely, that neighbor never enters the top-100, so the check cannot detect the miss. The Limitations section (§5) concedes dependence on the embedding model but gives no miss-rate estimate. Without a recall evaluation on paraphrase ground truth (e.g., injecting known rephrased passages and measuring whether they are retrieved), the headline claims describe the embedding geometry rather than model reuse. Please add such a validation.
  3. [§4.2, claim (1) and Fig. 3] The claim that 'models draw on pretraining data across much longer sequences than previously reported' requires a direct comparison to the prior n-gram baseline on the same generations and models. The paper never runs an n-gram novelty method (e.g., Rusty-DAWG from Merrill et al., 2024) on its own outputs. Without that comparison, the phrase 'than previously reported' is an unsupported cross-study statement. Please run the n-gram baseline and show where the two methods diverge.
  4. [§4.3, Table 1 and Fig. 4] Several per-cell sample sizes are extremely small, e.g., SmolLM-360M-Instruct on GSM8K has n=15, SmolLM-360M-Base n=20, and SmolLM-1.7B variants n=63. Figures 4 and 7 plot medians without confidence intervals or bootstrap uncertainty. The claims that 'smaller models are more novel' and that 'instruction tuning increases novelty' rely on curves that may be statistically indistinguishable. Please report uncertainty (e.g., bootstrap CIs) and avoid strong conclusions for cells with n < 50, or at least clearly mark low-n cells.
  5. [§4.3, baseline construction] For GSM8K and TruthfulQA, the benchmark targets serve as the human baseline. These targets are public datasets and may themselves appear in the SmolLM/SmolLM2 pretraining corpus. If they do, the baseline similarity is inflated, making model outputs appear more novel than they are. The paper does not check whether the benchmark targets overlap with the pretraining data, nor does it exclude them from the index. Please report the overlap and, if present, use a held-out baseline or filter the index.
minor comments (5)
  1. [Abstract/§1] The paper says 'three-stage framework' in the abstract but 'two-stage retrieval pipeline' in Sections 1 and 3. Please standardize the terminology, e.g., counting index construction as Stage 0.
  2. [Fig. 1] The y-axis label 'Similarity Score' is ambiguous; the left and right panels measure different quantities (n-gram overlap vs. embedding cosine). Please label each panel with the exact metric and range.
  3. [§4.2] For SmolLM2-Instruct 'unprompted' generation, the neutral cue 'Generate a text' changes the task. The claim that unprompted generation is less novel should be qualified, since the instruct models receive a prompt that may itself bias the output distribution.
  4. [References] The reference 'Zhang et al., 2025' is incomplete: it reads 'Min-kURL https://arxiv.org/abs/2404.02936' with no title or venue. Please fix.
  5. [Algorithm 1] The notation R^(k) is defined as a set, but the output line takes its median. Clarify that R^(k) is a list of per-chunk ratios and that the median is taken over chunks.

Circularity Check

0 steps flagged

No circular derivation: novelty is explicitly operationalized via retrieval, and all three findings are descriptive measurements of that defined quantity.

full rationale

The paper does not derive novelty from first principles; it defines it operationally: 'an output is novel if the pretraining corpus contains no semantically similar context' and then implements this as 'if no close match is found, we deem the output novel rather than attributable to the corpus.' The three headline findings (long-span reuse, domain variation, instruction-tuning effects) are empirical reports about the metric defined in Algorithm 1, not equations that reduce to each other by construction. The human baseline (Dolma) is stated to be outside the SmolLM pretraining sets, so the calibration reference is independent of the model outputs. The threshold R(k)<1 is an interpretive anchor rather than a fitted parameter, and no parameter is fit to a subset and then 'predicted' on a closely related quantity. Self-citations (Nguyen et al. 2023; Puerto et al. 2025; Wang et al. 2025a) appear only in related-work discussions and are not load-bearing for the central framework. The main vulnerability—whether GIST/ColBERTv2 retrieval truly captures all semantically similar contexts and whether the median aggregation hides a single copied chunk—is a validity and recall concern, acknowledged in the Limitations ('it depends on the chosen embedding model, which may introduce biases and representation errors'). Such an approximation gap is a correctness risk, not circularity, because the paper's conclusions are stated about its own operational measure rather than derived from an independent theory. Therefore, no significant circularity is present.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 1 invented entities

The central claims are not derived from first principles; they are empirical measurements whose meaning is fixed by hand-chosen retrieval parameters, baseline statistics, and filtering rules. The largest burden is the embedding-similarity definition of novelty and the baseline estimation. No new physical entities are introduced beyond the conceptual construct of un-attributability.

free parameters (5)
  • Baseline normalizer μ_B^(k) = mean ColBERTv2 similarity of Dolma baseline chunks to best retrieved pretraining chunk at each chunk size k
    Algorithm 1 defines the novelty score as generation similarity divided by this baseline mean. All plots and 'more novel than human' statements depend on this data-estimated constant; the choice of mean and of the Dolma document pool is not derived from any theory.
  • FAISS retrieval width n = 100
    Chosen in Section 3.1 for 'balancing recall and compute'; Appendix A argues adequacy, but this hand-picked hyperparameter bounds the recall of the entire novelty test. The pipeline cannot find matches beyond rank 100 before reranking.
  • Baseline document selection rules = Dolma documents with length 2500-7500 tokens; 1210 retained from 100K sampled
    Section 4.2: filtering is ad hoc and determines which human texts establish the novelty threshold; the small, specific pool may not represent human text generally.
  • Correct-answer filters for domain experiments = accuracy=1 for GSM8K/TruthfulQA; ROUGE-L≥0.25 for OpenRewriteEval
    Section 4.3: only correct outputs are scored, dropping up to ~95% of outputs for some models (Table 1), which changes the population on which novelty curves are computed.
  • Neutral cue for SmolLM2-Instruct unprompted generation = "Generate a text"
    Section 4.2: blank prompts produced trivial replies, so a cue was added for one model family only; this breaks comparability of unprompted conditions across families.
axioms (5)
  • domain assumption Semantic similarity in GIST/ColBERTv2 embedding space is a valid proxy for whether the pretraining corpus contains a semantically similar context to a generation.
    Entire framework (Section 3) equates attributability to retrieved semantic similarity; unvalidated against human novelty judgments or paraphrastic ground truth.
  • domain assumption Dolma (Reddit, peS2o) documents are valid 'known-novel' human references for SmolLM/SmolLM2 because Dolma is not in their pretraining corpora.
    Section 4.2 states this; assumes absence of corpus overlap and assumes these human texts represent typical human-level novelty across all domains.
  • domain assumption Correct answers/ROUGE-filtered outputs isolate genuine novelty rather than random incorrectness.
    Section 4.3: filtering by correctness is assumed to exclude low-quality generations that would trivially have low similarity; no control for the strong selection this creates.
  • standard math FAISS approximate nearest-neighbor search over L2-normalized GIST embeddings returns the true top-n semantic neighbors.
    Standard ANN guarantee; paper relies on it for recall, with an empirical check in Appendix A but no formal recall guarantee.
  • domain assumption ColBERTv2 late-interaction scores, length-normalized by query tokens, provide a fair best-match similarity across texts of different lengths.
    Algorithm 1 defines the normalized score es(x,C); the length-normalization is a heuristic, and its fairness across lengths is not validated.
invented entities (1)
  • Un-attributability no independent evidence
    purpose: An operational definition and measure of semantic novelty as absence of a close retrieval match in the pretraining corpus.
    Introduced in Section 3 as a new construct. It is not derived from external theory; its validity as a measure of novelty rests entirely on the embedding-similarity assumption. This is a conceptual frame rather than a physical entity, but it functions as the paper's central invented quantity.

pith-pipeline@v1.3.0-alltime-deepseek · 13976 in / 16890 out tokens · 139955 ms · 2026-08-04T06:58:49.391903+00:00 · methodology

0 comments
read the original abstract

Generation novelty is a key indicator of an LLM's ability to generalize, yet measuring it against full pretraining corpora is computationally challenging. Existing evaluations often rely on lexical overlap, failing to detect paraphrased text, or do not consider the full pretraining corpus. We frame novelty as a semantic retrieval problem. This framing enables us to address novelty with modern embedding and indexing pipelines, allowing for efficient analysis at pre-training scale. Specifically, we propose a three-stage framework that retrieves semantically similar samples, reranks them at varying subsequence lengths, and calibrates scores using a human novelty reference for interpretability. We apply this framework to the SmolLM model family and report three key findings: (1) models draw on pre-training data across much longer sequences than previously reported; (2) some task domains systematically promote or suppress generation novelty; and (3) instruction tuning not only alters style but also increases novelty. These results highlight the value of semantic novelty analysis for studying generalization. To support reproducibility and further research, we release ~20 TB of corpus chunks and index artifacts at https://huggingface.co/datasets/stai-tuebingen/faiss-smollm

Figures

Figures reproduced from arXiv: 2510.27313 by Ameya Prabhu, Elisa Nguyen, Matthias Bethge, Philipp Davydov, Seong Joon Oh.

Figure 1
Figure 1. Figure 1: Embedding similarity is more robust to long or paraphrased texts than N-gram similarity. Comparison of similarity measured with N-gram overlap (left) and embedding cosine similarity (right) with increasing sequence length. The similarity is measured between a model generation and its top 2 closest semantic matches in the pretraining corpus retrieved using our test. Both training excerpts convey the same in… view at source ↗
Figure 2
Figure 2. Figure 2: Pipeline for scoring the novelty of an LLM output q. We test whether q is unattributable to the pretraining corpus – our operational definition of novelty. Stage 0 (one-time): Chunk the corpus, compute L2-normalized GIST (Solatorio, 2024) embeddings, and build a cosine-similarity FAISS (Douze et al., 2024) index. Stage 1: Embed q with GIST and retrieve the top-n nearest corpus chunks. Stage 2: Rerank retri… view at source ↗
Figure 3
Figure 3. Figure 3: Median ColBERTv2 similarity of SmolLM (top) and SmolLM2 (bottom) generations, [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Median ColBERTv2 similarity of SmolLM (top) and SmolLM2 (bottom) generations on [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Number of times each original FAISS-Top-100 index was mapped to the ColBERTv2- [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Distribution of the similarity of SmolLM2 generations, for open-ended generation with [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Distribution of the similarity of SmolLM2 generations, per text domain, for representative [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

60 extracted references · 1 canonical work pages

  1. [1]

    Towards tracing knowledge in language models back to the training data

    Ekin Aky \"u rek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong, Ian Tenney, Jacob Andreas, and Kelvin Guu. Towards tracing knowledge in language models back to the training data. In Findings of the Association for Computational Linguistics: EMNLP 2022, pp.\ 2429--2446, 2022

    2022

  2. [2]

    Smollm-blazingly fast and remarkably powerful

    Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Leandro von Werra, and Thomas Wolf. Smollm-blazingly fast and remarkably powerful. Hugging Face Blog, 16, 2024

    2024

  3. [3]

    Smollm2: When smol goes big -- data-centric training of a small language model, 2025

    Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Gabriel Martín Blázquez, Guilherme Penedo, Lewis Tunstall, Andrés Marafioti, Hynek Kydlíček, Agustín Piqueres Lajarín, Vaibhav Srivastav, Joshua Lochner, Caleb Fahlgren, Xuan-Son Nguyen, Clémentine Fourrier, Ben Burtenshaw, Hugo Larcher, Haojun Zhao, Cyril Zakka, Mathieu Morlon, Colin Raffel, Leandro von Werr...

    Pith/arXiv arXiv 2025

  4. [4]

    If influence functions are the answer, then what is the question? Advances in Neural Information Processing Systems, 35: 0 17953--17967, 2022

    Juhan Bae, Nathan Ng, Alston Lo, Marzyeh Ghassemi, and Roger B Grosse. If influence functions are the answer, then what is the question? Advances in Neural Information Processing Systems, 35: 0 17953--17967, 2022

    2022

  5. [5]

    Training data attribution via approximate unrolling

    Juhan Bae, Wu Lin, Jonathan Lorraine, and Roger B Grosse. Training data attribution via approximate unrolling. Advances in Neural Information Processing Systems, 37: 0 66647--66686, 2024

    2024

  6. [6]

    Influence functions in deep learning are fragile

    Samyadeep Basu, Phil Pope, and Soheil Feizi. Influence functions in deep learning are fragile. In International Conference on Learning Representations, 2021

    2021

  7. [7]

    Quantifying memorization across neural language models

    Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations, 2022

    2022

  8. [8]

    Scalable influence and fact tracing for large language model pretraining

    Tyler A Chang, Dheeraj Rajagopal, Tolga Bolukbasi, Lucas Dixon, and Ian Tenney. Scalable influence and fact tracing for large language model pretraining. In The Thirteenth International Conference on Learning Representations, 2025

    2025

  9. [9]

    What is your data worth to gpt? llm-scale data valuation with influence functions

    Sang Keun Choe, Hwijeen Ahn, Juhan Bae, Kewen Zhao, Minsoo Kang, Youngseog Chung, Adithya Pratapa, Willie Neiswanger, Emma Strubell, Teruko Mitamura, et al. What is your data worth to gpt? llm-scale data valuation with influence functions. arXiv preprint arXiv:2405.13954, 2024

    Pith/arXiv arXiv 2024

  10. [10]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    Pith/arXiv arXiv 2021

  11. [11]

    Junwei Deng, Yuzheng Hu, Pingbang Hu, Ting-Wei Li, Shixuan Liu, Jiachen T. Wang, Dan Ley, Qirun Dai, Benhao Huang, Jin Huang, Cathy Jiao, Hoang Anh Just, Yijun Pan, Jingyan Shen, Yiwen Tu, Weiyi Wang, Xinhe Wang, Shichang Zhang, Shiyuan Zhang, Ruoxi Jia, Himabindu Lakkaraju, Hao Peng, Weijing Tang, Chenyan Xiong, Jieyu Zhao, Hanghang Tong, Han Zhao, and J...

    2025

  12. [12]

    The faiss library

    Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazar \'e , Maria Lomeli, Lucas Hosseini, and Herv \'e J \'e gou. The faiss library. arXiv preprint arXiv:2401.08281, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Mmteb: Massive multilingual text embedding benchmark

    Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos, Ashwin Mathur, David Stap, Jay Gala, Wissam Siblini, Dominik Krzemiński, Genta Indra Winata, Saba Sturua, Saiteja Utpala, Mathieu Ciancone, Marion Schaeffer, Gabriel Sequeira, Diganta Misra, Shreeya Dhakal, Jonathan Rystrøm, Roman Solomatin, Ömer Çağatan, Akash Kundu, Martin Bernstorff, Shitao...

    2025

  14. [14]

    Revisiting the fragility of influence functions

    Jacob R Epifano, Ravi P Ramachandran, Aaron J Masino, and Ghulam Rasool. Revisiting the fragility of influence functions. Neural Networks, 162: 0 581--588, 2023

    2023

  15. [15]

    What neural networks memorize and why: Discovering the long tail via influence estimation

    Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems, 33: 0 2881--2891, 2020

    2020

  16. [16]

    The language model evaluation harness, 07 2024

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The languag...

    arXiv 2024

  17. [17]

    A closer look at the limitations of instruction tuning

    Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Deepali Aneja, Zeyu Jin, Ramani Duraiswami, Dinesh Manocha, et al. A closer look at the limitations of instruction tuning. In International Conference on Machine Learning, pp.\ 15559--15589. PMLR, 2024

    2024

  18. [18]

    Lment: A suite for analyzing knowledge in language models from pretraining data to representations, 2025

    Daniela Gottesman, Alon Gilae-Dotan, Ido Cohen, Yoav Gur-Arieh, Marius Mosbach, Ori Yoran, and Mor Geva. Lment: A suite for analyzing knowledge in language models from pretraining data to representations, 2025. URL https://arxiv.org/abs/2509.03405

    Pith/arXiv arXiv 2025

  19. [19]

    Roger Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, Evan Hubinger, Kamilė Lukošiūtė, Karina Nguyen, Nicholas Joseph, Sam McCandlish, Jared Kaplan, and Samuel R. Bowman. Studying large language model generalization with influence functions, 2023. URL https://arxiv.org/abs/...

    Pith/arXiv arXiv 2023

  20. [20]

    Fastif: Scalable influence functions for efficient model interpretation and debugging

    Han Guo, Nazneen Rajani, Peter Hase, Mohit Bansal, and Caiming Xiong. Fastif: Scalable influence functions for efficient model interpretation and debugging. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp.\ 10333--10350, 2021

    2021

  21. [21]

    Lighteval: A lightweight framework for llm evaluation, 2023

    Nathan Habib, Clémentine Fourrier, Hynek Kydlíček, Thomas Wolf, and Lewis Tunstall. Lighteval: A lightweight framework for llm evaluation, 2023. URL https://github.com/huggingface/lighteval

    2023

  22. [22]

    Training data influence analysis and estimation: a survey

    Zayd Hammoudeh and Daniel Lowd. Training data influence analysis and estimation: a survey. Mach. Learn., 113 0 (5): 0 2351–2403, March 2024. ISSN 0885-6125. doi:10.1007/s10994-023-06495-7. URL https://doi.org/10.1007/s10994-023-06495-7

    doi:10.1007/s10994-023-06495-7 2024

  23. [23]

    The influence curve and its role in robust estimation

    Frank R Hampel. The influence curve and its role in robust estimation. Journal of the american statistical association, 69 0 (346): 0 383--393, 1974

    1974

  24. [24]

    Data cleansing for models trained with sgd

    Satoshi Hara, Atsushi Nitanda, and Takanori Maehara. Data cleansing for models trained with sgd. Advances in Neural Information Processing Systems, 32, 2019

    2019

  25. [25]

    Most influential subset selection: Challenges, promises, and beyond

    Yuzheng Hu, Pingbang Hu, Han Zhao, and Jiaqi Ma. Most influential subset selection: Challenges, promises, and beyond. Advances in Neural Information Processing Systems, 37: 0 119778--119810, 2024

    2024

  26. [26]

    Magic: Near-optimal data attribution for deep learning

    Andrew Ilyas and Logan Engstrom. Magic: Near-optimal data attribution for deep learning. arXiv preprint arXiv:2504.16430, 2025

    Pith/arXiv arXiv 2025

  27. [27]

    Understanding black-box predictions via influence functions

    Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pp.\ 1885--1894. PMLR, 2017

    2017

  28. [28]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp.\ 74--81, 2004

    2004

  29. [29]

    Truthfulqa: Measuring how models mimic human falsehoods

    Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\ 3214--3252, 2022

    2022

  30. [30]

    Olmotrace: Tracing language model outputs back to trillions of training tokens

    Jiacheng Liu, Taylor Blanton, Yanai Elazar, Sewon Min, YenSung Chen, Arnavi Chheda-Kothary, Huy Tran, Byron Bischoff, Eric Marsh, Michael Schmitz, et al. Olmotrace: Tracing language model outputs back to trillions of training tokens. arXiv preprint arXiv:2504.07096, 2025 a

    Pith/arXiv arXiv 2025

  31. [31]

    Infini-gram: Scaling unbounded n-gram language models to a trillion tokens, 2025 b

    Jiacheng Liu, Sewon Min, Luke Zettlemoyer, Yejin Choi, and Hannaneh Hajishirzi. Infini-gram: Scaling unbounded n-gram language models to a trillion tokens, 2025 b . URL https://arxiv.org/abs/2401.17377

    Pith/arXiv arXiv 2025

  32. [32]

    How much do language models copy from their training data? evaluating linguistic novelty in text generation using raven

    Richard Thomas McCoy, Paul Smolensky, Tal Linzen, Jianfeng Gao, and Asli Celikyilmaz. How much do language models copy from their training data? evaluating linguistic novelty in text generation using raven. Transactions of the Association for Computational Linguistics, 11, 2023

    2023

  33. [33]

    Evaluating n-gram novelty of language models using rusty-dawg

    William Merrill, Noah A Smith, and Yanai Elazar. Evaluating n-gram novelty of language models using rusty-dawg. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.\ 14459--14473, 2024

    2024

  34. [34]

    Waka: Data attribution using k-nearest neighbors and membership privacy principles

    Patrick Mesana, Cl \'e ment B \'e nesse, Hadrien Lautraite, Gilles Caporossi, and S \'e bastien Gambs. Waka: Data attribution using k-nearest neighbors and membership privacy principles. Proceedings on Privacy Enhancing Technologies, 3: 0 494--526, 2025

    2025

  35. [35]

    Mteb: Massive text embedding benchmark

    Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. Mteb: Massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pp.\ 2014--2037, 2023

    2014

  36. [36]

    A bayesian approach to analysing training data attribution in deep learning

    Elisa Nguyen, Minjoon Seo, and Seong Joon Oh. A bayesian approach to analysing training data attribution in deep learning. Advances in Neural Information Processing Systems, 36: 0 64155--64180, 2023

    2023

  37. [37]

    Trak: Attributing model behavior at scale

    Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclerc, and Aleksander Madry. Trak: Attributing model behavior at scale. In International Conference on Machine Learning, pp.\ 27074--27113. PMLR, 2023

    2023

  38. [38]

    Near-duplicate sequence search at scale for large language model memorization evaluation

    Zhencan Peng, Zhizhi Wang, and Dong Deng. Near-duplicate sequence search at scale for large language model memorization evaluation. Proc. ACM Manag. Data, 1 0 (2), June 2023. doi:10.1145/3589324. URL https://doi.org/10.1145/3589324

    work page doi:10.1145/3589324 2023

  39. [39]

    Estimating training data influence by tracing gradient descent

    Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. Advances in Neural Information Processing Systems, 33: 0 19920--19930, 2020

    2020

  40. [40]

    Scaling up membership inference: When and how attacks succeed on large language models

    Haritz Puerto, Martin Gubri, Sangdoo Yun, and Seong Joon Oh. Scaling up membership inference: When and how attacks succeed on large language models. In Findings of the Association for Computational Linguistics: NAACL 2025, pp.\ 4165--4182, 2025

    2025

  41. [41]

    Learning or self-aligning? rethinking instruction fine-tuning

    Mengjie Ren, Boxi Cao, Hongyu Lin, Cao Liu, Xianpei Han, Ke Zeng, Wan Guanglu, Xunliang Cai, and Le Sun. Learning or self-aligning? rethinking instruction fine-tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\ 6090--6105, 2024

    2024

  42. [42]

    Colbertv2: Effective and efficient retrieval via lightweight late interaction

    Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. Colbertv2: Effective and efficient retrieval via lightweight late interaction. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp.\ 3715--3734, 2022

    2022

  43. [43]

    Scaling up influence functions

    Andrea Schioppa, Polina Zablotskaia, David Vilar, and Artem Sokolov. Scaling up influence functions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp.\ 8179--8186, 2022

    2022

  44. [44]

    Rewritelm: an instruction-tuned large language model for text rewriting

    Lei Shu, Liangchen Luo, Jayakumar Hoskere, Yun Zhu, Yinxiao Liu, Simon Tong, Jindong Chen, and Lei Meng. Rewritelm: an instruction-tuned large language model for text rewriting. In Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Sym...

    2024

  45. [45]

    Solatorio

    Aivin V. Solatorio. Gistembed: Guided in-sample selection of training negatives for text embedding fine-tuning. arXiv preprint arXiv:2402.16829, 2024. URL https://arxiv.org/abs/2402.16829

    Pith/arXiv arXiv 2024

  46. [46]

    pes2o (pretraining efficiently on s2orc) dataset

    Luca Soldaini and Kyle Lo. pes2o (pretraining efficiently on s2orc) dataset. https://github.com/allenai/peS2o, 2023

    2023

  47. [47]

    Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A

    Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Abhilasha Ravichander...

    Pith/arXiv arXiv 2024

  48. [48]

    Enhancing training data attribution with representational optimization, 2025

    Weiwei Sun, Haokun Liu, Nikhil Kandpal, Colin Raffel, and Yiming Yang. Enhancing training data attribution with representational optimization, 2025. URL https://arxiv.org/abs/2505.18513

    arXiv 2025

  49. [49]

    Better training data attribution via better inverse hessian-vector products

    Andrew Wang, Elisa Nguyen, Runshi Yang, Juhan Bae, Sheila A McIlraith, and Roger Grosse. Better training data attribution via better inverse hessian-vector products. arXiv preprint arXiv:2507.14740, 2025 a

    Pith/arXiv arXiv 2025

  50. [50]

    Capturing the temporal dependence of training data influence

    Jiachen T Wang, Dawn Song, James Zou, Prateek Mittal, and Ruoxi Jia. Capturing the temporal dependence of training data influence. In The Thirteenth International Conference on Learning Representations, 2025 b

    2025

  51. [51]

    Generalization vs memorization: Tracing language models’ capabilities back to pretraining data

    Xinyi Wang, Antonis Antoniades, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, and William Yang Wang. Generalization vs memorization: Tracing language models’ capabilities back to pretraining data. In The Thirteenth International Conference on Learning Representations, 2025 c

    2025

  52. [52]

    Memhunter: Automated and verifiable memorization detection at dataset-scale in llms, 2025

    Zhenpeng Wu, Jian Lou, Zibin Zheng, and Chuan Chen. Memhunter: Automated and verifiable memorization detection at dataset-scale in llms, 2025. URL https://arxiv.org/abs/2412.07261

    arXiv 2025

  53. [53]

    Less: Selecting influential data for targeted instruction tuning

    Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. Less: Selecting influential data for targeted instruction tuning. In International Conference on Machine Learning, pp.\ 54104--54132. PMLR, 2024

    2024

  54. [54]

    Representer point selection for explaining deep neural networks

    Chih-Kuan Yeh, Joon Kim, Ian En-Hsu Yen, and Pradeep K Ravikumar. Representer point selection for explaining deep neural networks. Advances in neural information processing systems, 31, 2018

    2018

  55. [55]

    Min-k URL https://arxiv.org/abs/2404.02936

    Jingyang Zhang, Jingwei Sun, Eric Yeats, Yang Ouyang, Martin Kuo, Jianyi Zhang, Hao Frank Yang, and Hai Li. Min-k URL https://arxiv.org/abs/2404.02936

    Pith/arXiv arXiv

  56. [56]

    Pretraining data detection for large language models: A divergence-based calibration method

    Weichao Zhang, Ruqing Zhang, Jiafeng Guo, Maarten Rijke, Yixing Fan, and Xueqi Cheng. Pretraining data detection for large language models: A divergence-based calibration method. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.\ 5263--5274, 2024

    2024

  57. [57]

    Dense text retrieval based on pretrained language models: A survey

    Wayne Xin Zhao, Jing Liu, Ruiyang Ren, and Ji-Rong Wen. Dense text retrieval based on pretrained language models: A survey. ACM Trans. Inf. Syst., 42 0 (4), February 2024. ISSN 1046-8188. doi:10.1145/3637870. URL https://doi.org/10.1145/3637870

    doi:10.1145/3637870 2024

  58. [58]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  59. [59]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  60. [60]

    Most training‑data attribution (TDA) methods ask which training examples causally influence a given output, often using leave‑one‑out tests

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    arXiv