The reviewed record of science sign in
Pith

arxiv: 2504.07096 · v2 · pith:PRJDSKCC · submitted 2025-04-09 · cs.CL

OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens

Jiacheng Liu , Taylor Blanton , Yanai Elazar , Sewon Min , YenSung Chen , Arnavi Chheda-Kothary , Huy Tran , Byron Bischoff
show 23 more authors
Eric Marsh Michael Schmitz Cassidy Trier Aaron Sarnat Jenna James Jon Borchardt Bailey Kuehl Evie Cheng Karen Farley Sruthi Sreeram Taira Anderson David Albright Carissa Schoenick Luca Soldaini Dirk Groeneveld Rock Yuren Pang Pang Wei Koh Noah A. Smith Sophie Lebrecht Yejin Choi Hannaneh Hajishirzi Ali Farhadi Jesse Dodge
This is my paper

Reviewed by Pithpith:PRJDSKCCopen to challenge →

classification cs.CL
keywords languageolmotracetrainingmodelsbackdatamodeloutputs
0
0 comments X
read the original abstract

We present OLMoTrace, the first system that traces the outputs of language models back to their full, multi-trillion-token training data in real time. OLMoTrace finds and shows verbatim matches between segments of language model output and documents in the training text corpora. Powered by an extended version of infini-gram (Liu et al., 2024), our system returns tracing results within a few seconds. OLMoTrace can help users understand the behavior of language models through the lens of their training data. We showcase how it can be used to explore fact checking, hallucination, and the creativity of language models. OLMoTrace is publicly available and fully open-source.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMs

    cs.CL 2026-06 unverdicted novelty 7.0

    LLMs show high memorization capability under prefix attacks but low propensity under generic or dataset-specific prompts, with continual pre-training further reducing both.

  2. Efficient and Scalable Provenance Tracking for LLM-Generated Code Snippets

    cs.SE 2026-05 unverdicted novelty 7.0

    Hybrid vector-search plus fingerprinting pipeline for LLM code provenance achieves Winnowing-level MRR on short snippets and up to 5.4% better on longer ones at logarithmic query time.

  3. DataDignity: Training Data Attribution for Large Language Models

    cs.AI 2026-05 unverdicted novelty 7.0

    ScoringModel raises mean Recall@10 to 52.2 on the FakeWiki provenance benchmark from 35.0 for the best baseline, winning 41 of 45 model-by-condition comparisons and gaining 15.7 points on jailbreak-style queries.

  4. RELIANCE: Curating and Evaluating Reproductive Health Information on Social Media

    cs.SI 2026-06 unverdicted novelty 6.0

    RELIANCE is a new expert-annotated dataset of TikTok reproductive health content paired with LLM fact-checking evaluations showing 60% accuracy in sampled videos and a 15% gap between claim and full-content assessment.

  5. Feature Starvation as Geometric Instability in Sparse Autoencoders

    cs.LG 2026-05 unverdicted novelty 6.0

    Adaptive elastic net SAEs (AEN-SAEs) mitigate feature starvation in SAEs by combining ℓ2 structural stability with adaptive ℓ1 reweighting, producing a Lipschitz-continuous sparse coding map that recovers global featu...