REVIEW 3 major objections 5 minor 22 references
The paper proves that generative models trained on their own past outputs, mixed with a steady stream of new real data, still converge to the true data distribution — at a rate set by the smaller of the model's baseline speed and the real-d
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 22:41 UTC pith:PVZ7WSOG
load-bearing objection The rate t^{-min(p,α)} is novel and the recursion proof is clean, but the key step applies an i.i.d. convergence-rate assumption to block-structured, data-dependent training data — a real gap that the paper does not close. the 3 major comments →
Can Generative Artificial Intelligence Survive Data Contamination? Theoretical Guarantees under Contaminated Recursive Training
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Theorem 3.4 states the central result: let P0 be the target distribution and let P̂_t be the generator produced at iteration t by contaminated recursive training (CRT), which accumulates all past real samples and all past synthetic samples. If the baseline estimator has a uniform polynomial convergence rate p over a convex class of distributions (Assumption A1) and the distributional distance is convex in its second argument (Assumption A2), then d(P̂_t, P0) ≲ t^{−min{p, α}}, up to a log factor at p = α. Thus recursive training does not inevitably collapse: it converges to the true distribution, at a rate set by the slower of the model's intrinsic speed and the fraction of genuinely real dat
What carries the argument
The engine of the proof is a recursion inequality. Writing S_{t−1} = Σ_{j<t} d(P̂_j, P0), the error at step t satisfies d(P̂_t, P0) ≲ M_t^{−p} + (m2/M_t) S_{t−1}, where M_t = t m1 + (t−1) m2 is the accumulated sample size. The first term is the baseline learning error; the second measures how much contamination from earlier generators leaks in, weighted by the fraction of synthetic data. Unrolling this recursion and evaluating the product over k of (1 + (1−α)/k) through Gamma-function identities turns the contamination sum into t^{−α} Σ j^{−p−1+α}, which is then bounded by regime: constant, log t, or t^{α−p}, depending on whether p > α, p = α, or p < α. That split produces the phase transiti
Load-bearing premise
The load-bearing premise is Assumption A1: a single uniform polynomial rate p must hold over a convex class that contains every mixture distribution reached during recursive training, including mixtures of previous data-dependent generators; the paper verifies this directly only for ECDF/KDE in simulation and otherwise cites published rates.
What would settle it
Run contaminated recursive training on a multimodal target with a generator class whose baseline rate is fast but whose uniform rate over mixtures of its own empirical outputs is unproven, set α below p, and measure d(P̂_t, P0) across thousands of iterations; any decay slower than t^{−α}, or a plateau, refutes the theorem's prediction for that class. Equivalently, repeating the CRT experiment with a non-convex distance such as KL divergence would indicate whether the convexity assumption A2 is truly load-bearing.
If this is right
- If the real-data fraction α exceeds the model's baseline rate p, recursive training costs nothing asymptotically: the CRT sequence matches the uncontaminated rate t^{−p}.
- If α < p, the recursion is data-limited and converges at t^{−α}; at α = p, an extra log factor appears.
- When real data are drawn from a fixed biased distribution and never corrected, CRT converges to that biased distribution at the same rates — bias is learned, not erased.
- If the bias in real data decays at rate q, convergence to the true distribution continues at t^{−min{p,q,α}}, so bias correction is never wasted even after biased initial training.
- Accumulating all past real and synthetic data, rather than replacing old data with synthetic samples, is what prevents collapse; the theorem covers nonparametric target distributions and general universal-approximator generators.
Where Pith is reading between the lines
- I infer a design rule the authors do not state: knowing or estimating the baseline rate p allows a practitioner to set α just above p to keep baseline speed, or deliberately below p to trade convergence speed for compute — an ablation over α on one fixed architecture would test this directly.
- I infer that the phase transition is sensitive to the choice of distributional distance. The paper excludes cross-entropy and KL divergence; for likelihood-trained LLMs the effective rate may follow a different law, so extending or disproving the min(p, α) formula for non-convex losses is the natural next experiment.
- Real internet contamination is selective: humans publish only some generations and amplify others. I infer that selection inflates the effective synthetic fraction, so in practice the collapse threshold could appear at a larger nominal α than the theorem predicts; the paper itself lists selection as future work.
- Because Assumption A1 is verified in simulation only for ECDF and KDE and otherwise imported from published rates, I would treat the theorem's coverage of deep generators (GANs, diffusion models) as conditional on that uniform-rate check rather than automatic.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies recursive training in which each iteration's training set accumulates fresh real data from the target distribution and synthetic data drawn from previous generators. It defines Contaminated Recursive Training (CRT) and a biased variant (BCRT), and proves, under Assumptions A1 (a uniform polynomial convergence rate p over a convex distribution class Q) and A2 (convexity of the metric), that the CRT estimator satisfies d(hat P_t, P0) ≲ t^{-min(p, alpha)} up to a log factor, and that BCRT satisfies t^{-min(p, q, alpha)}. The proof is a recursion on the partial sums of distributional errors, analyzed with a gamma-function bound. Simulations with ECDF/KDE, a WGAN-style generator, and an MNIST diffusion model are reported. The central issue is whether Assumption A1 can legitimately be applied to the data-dependent mixture distributions Q_t and to the stratified/block-structured accumulated samples that arise in the CRT procedure; the manuscript asserts this via citations for deep generative models but does not verify it.
Significance. If made rigorous, this is a valuable positive counterpart to the model-collapse literature: it gives a clean phase-transition rate controlled by the slower of the baseline convergence rate and the real-data fraction, and it covers both unbiased and biased sampling regimes. The recursion is elegant, the gamma-function calculation is correct, and the paper ships code and extensive simulations. However, the main theorem is only as strong as its verification of Assumption A1 along the recursive trajectory; in the current version this verification is missing for the deep generative models that the paper claims to cover. The ECDF/KDE simulations are consistent with the theory, but the WGAN and diffusion experiments do not validate Assumption A1 for their respective recursive trajectories.
major comments (3)
- [§B.1, Eq. (1)] The step d(hat P_t, Q_t) ≲ M_t^{-p} applies Assumption A1 to the accumulated dataset as though it were an i.i.d. sample from Q_t. Under Definition 3.3, however, the accumulated data are a fixed-block product measure: (t+1)m1 points from P0 and m2 points from each of bP_0,...,bP_{t-1}. Assumption A1 is stated for i.i.d. samples (Definition 3.2), and no argument transfers the uniform i.i.d. rate to this stratified dependence structure. For the ECDF the transfer is standard, but for a general learner in A1 it is not automatic. The recursion in Eq. (1) therefore lacks support unless A1 is replaced by a stronger 'stratified-sample' rate or a separate transfer argument is supplied.
- [§3, Assumption A1 / Theorem 3.4] The theorem requires A1 to hold uniformly on a convex class Q containing the mixtures Q_t = (tm1 P0 + m2 Σ_{j=1}^{t-1} bP_j)/M_t. Two things are not established. First, the previous estimators bP_j are data-dependent and need not lie in Q, so Q_t need not be in Q even if P0 is. Second, the cited uniform rates for GANs and diffusion models (e.g., Uppal et al. 2019; Oko et al. 2023) hold under regularity conditions on the target, and the mixtures Q_t are not shown to satisfy these conditions. The KDE/ECDF simulations are consistent with A1, but Section 5.1.2 uses a generator trained with an exact quantile-based W1 loss rather than a true WGAN, and Section 6 does not evaluate distributional distances; neither validates A1 for the recursive trajectory. Without this verification, Theorem 3.4 does not apply to the deep generative models claimed in the abstract and introduction.
- [§B.1, definition of Q_t] The proof's Q_t and M_t do not match Definition 3.3. After t iterations the accumulated data contain (t+1)m1 real points and t m2 synthetic points, with the most recent synthetic block Y_t drawn from bP_{t-1}; the proof uses M_t = t m1 + (t-1)m2 and omits the bP_0 component and the most recent synthetic block. The recursion should be re-derived with the correct counts. The rate is probably unchanged, but as written the proof is not a proof of the stated procedure.
minor comments (5)
- [Abstract / §1] The claims that the results hold 'without distributional assumptions on the data' and with 'minimal assumptions' overstate the role of Assumption A1, which requires a uniform polynomial rate over a convex distribution class and is a genuine regularity/rate assumption.
- [§B.1, first paragraph] There is a typo: 'we obtain We first use the triangle inequality' should be 'We first use the triangle inequality'.
- [Table 2] For (alpha, q) = (0.5, 0.25), the theoretical rate is min(0.5, 0.25, 0.25) = 0.25, but the reported empirical slopes are 0.325, 0.421, 0.332, 0.330. The discrepancy is not discussed; a note on burn-in, finite-sample bias, or fitting variability would help.
- [§5.1.2] The 'WGAN' experiment replaces the critic with an exact empirical W1 quantile loss; this is not a standard WGAN. The theoretical rates cited for GANs (Table 1, Uppal et al. 2019) may therefore not apply to this modified estimator, and the experiment should be described cautiously.
- [§6] The MNIST experiment shows generated samples but does not compute a quantitative distributional distance to the true MNIST distribution, so it can only be read as a sanity check, not as validation of the t^{-min(p, alpha)} rate.
Circularity Check
No circularity: the CRT rate is derived from an external baseline-rate assumption, not from data fitted to the CRT sequence.
full rationale
Theorem 3.4's rate t^{-min(p,alpha)} is derived from Assumption A1, an external uniform polynomial-rate guarantee for the baseline (non-recursive) generative model, and Assumption A2, convexity of the metric. The proof bounds the one-step learning error d(hat P_t, Q_t) by M_t^{-p} using A1, then solves a convexity-based recursion; the phase transition min(p,alpha) does not appear in A1 and is a consequence, not an input. p is not fitted from the CRT simulations: for ECDF/KDE the paper uses the known nonparametric p=1/2, and for BCRT q and alpha are set by the experimental design. No self-citation is load-bearing (the uniform-rate citations for deep generative models are to external works). The skeptical concern that A1 may fail for the data-dependent mixture distributions Q_t reached during CRT, and that the accumulated data are block-structured rather than i.i.d. from Q_t, is a real assumption-verification/applicability gap for deep generative models, but it is not circular: the paper's conditional theorem is a valid derivation from A1 plus A2, and the paper's own Discussion acknowledges scope limitations (e.g., cross-entropy/KL losses are not covered). Hence no step reduces the conclusion to the assumptions by construction.
Axiom & Free-Parameter Ledger
free parameters (3)
- baseline rate p
- real-data fraction α
- bias decay rate q
axioms (4)
- domain assumption A1: uniform polynomial convergence rate M n^{-p} over a convex class Q
- domain assumption A2: distance d is convex in its second argument
- domain assumption A3: biased distributions P_bias_t ∈ Q with d(P_bias_t,P0) ≲ t^{-q}
- domain assumption Q is convex and closed under finite mixtures, and previous estimators lie in Q
read the original abstract
As artificial intelligence (AI)-generated content proliferates, models are increasingly trained on their own outputs, risking progressive degradation or collapse. In this article, we provide the first positive, rigorous theoretical results, to the best of our knowledge, showing that under model-agnostic mild conditions, the model converges to the true data-generating distribution. The convergence rate is the minimum of the model's intrinsic rate and the fraction of real data at each training iteration, revealing a phase transition between data-limited and model-limited regimes. We further show that, for biased real data, correcting the bias prevents the persistence and amplification of early bias over training iteration. Extensive experiments across simulations, real images and texts validate our theoretical framework, establishing quantitative conditions for long-term AI stability in contaminated environments.
Figures
Reference graph
Works this paper leans on
-
[1]
1−s. Combined with Stirling-type bounds for Γ (Rudin, 1976), this implies that for anyα∈(0,1) there exist constantsc 1, c2, c3, c4 >0 such that for all integerst, j≥1, c1t−α ≤ Γ(t−α) Γ(t) ≤c 2t−α, c 3jα ≤ Γ(j+
1976
-
[3]
The rise of ai-generated content in wikipedia.arXiv preprint arXiv:2410.08044,
Creston Brooks, Samuel Eggert, and Denis Peskoff. The rise of ai-generated content in wikipedia.arXiv preprint arXiv:2410.08044,
-
[5]
Accessed: 2025-06-26
URLhttps://www.theverge.com/ai-artificial-intelligence/691355/ ai-is-ruining-houseplant-communities-online. Accessed: 2025-06-26. 19 Tommaso Furlanello, Zachary Lipton, Michael Tschannen, Laurent Itti, and Anima Anand- kumar. Born again neural networks. InInternational conference on machine learning, pages 1607–1616. PMLR,
2025
-
[9]
Data-free knowledge distillation for deep neural networks.arXiv preprint arXiv:1710.07535,
Raphael Gontijo Lopes, Stefano Fenu, and Thad Starner. Data-free knowledge distillation for deep neural networks.arXiv preprint arXiv:1710.07535,
-
[11]
Nicolas Papernot, Mart ´ ın Abadi, Ulfar Erlingsson, Ian Goodfellow, and Kunal Talwar. Semi-supervised knowledge transfer for deep learning from private training data.arXiv preprint arXiv:1610.05755,
-
[12]
Ankit Singh Rawat, Veeranjaneyulu Sadhanala, Afshin Rostamizadeh, Ayan Chakrabarti, Wittawat Jitkrittum, Vladimir Feinberg, Seungyeon Kim, Hrayr Harutyunyan, Nikunj Saunshi, Zachary Nado, et al. A little help goes a long way: Efficient llm training by leveraging small lms.arXiv preprint arXiv:2410.18779,
-
[16]
The curse of recursion: Training on generated data makes models forget
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. The curse of recursion: Training on generated data makes models forget. arXiv preprint arXiv:2305.17493,
-
[17]
Rate of model collapse in recursive training.arXiv preprint arXiv:2412.17646,
Ananda Theertha Suresh, Andrew Thangaraj, and Aditya Nanda Kishore Khandavally. Rate of model collapse in recursive training.arXiv preprint arXiv:2412.17646,
-
[18]
Rong Tang, Lizhen Lin, and Yun Yang. Conditional diffusion models are minimax- optimal and manifold-adaptive for conditional distribution estimation.arXiv preprint arXiv:2409.20124,
-
[19]
Differentially private generative adversarial network.arXiv preprint arXiv:1802.06739,
Liyang Xie, Kaixiang Lin, Shu Wang, Fei Wang, and Jiayu Zhou. Differentially private generative adversarial network.arXiv preprint arXiv:1802.06739,
-
[22]
At iterationt, a batch of m1 new samples fromP 0 is appended to the dataset, together withm 2 = ((1−α)/α)m 1 synthetic samples generated from the previous iterate bPt−1
The iterations follow the CRT framework in Definition 3.3. At iterationt, a batch of m1 new samples fromP 0 is appended to the dataset, together withm 2 = ((1−α)/α)m 1 synthetic samples generated from the previous iterate bPt−1. At each CRT iteration, the generator is completely re-initialized, and is then trained forkepochs using minibatch stochastic gra...
2021
-
[1959]
Matthias Gerstgrasser, Rylan Schaeffer, Apratim Dey, Rafael Rafailov, Henry Sleight, John Hughes, Tomasz Korbak, Rajashree Agrawal, Dhruv Pai, Andrey Gromov, et al. Is model collapse inevitable? breaking the curse of recursion by accumulating real and synthetic data.arXiv preprint arXiv:2404.01413,
-
[1976]
Cad2rl: Real single-image flight without a single real image.arXiv preprint arXiv:1611.04201,
Fereshteh Sadeghi and Sergey Levine. Cad2rl: Real single-image flight without a single real image.arXiv preprint arXiv:1611.04201,
-
[2009]
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531,
-
[2016]
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108,
Pith/arXiv arXiv 1910
-
[2018]
Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks
Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. Tengyuan Liang. How well can generative adversarial networks learn densities: A nonpara- metric view.arXiv preprint arXiv:1712.08244,
-
[2019]
The woman worked as a babysitter: On biases in language generation
21 Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. The woman worked as a babysitter: On biases in language generation. InProceedings of the 2019 confer- ence on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pages 3407–3412,
2019
-
[2020]
Bias in generative ai.arXiv preprint arXiv:2403.02726,
Mi Zhou, Vibhanshu Abhishek, Timothy Derdenger, Jaymo Kim, and Kannan Srinivasan. Bias in generative ai.arXiv preprint arXiv:2403.02726,
-
[2021]
Gonzalo Mart ´ ınez, Lauren Watson, Pedro Reviriego, Jos´ e Alberto Hern´ andez, Marc Juarez, and Rik Sarkar. Combining generative artificial intelligence (ai) and the internet: Heading towards evolution or degradation?arXiv preprint arXiv:2303.01255,
-
[2023]
Christopher Bowles, Liang Chen, Ricardo Guerrero, Paul Bentley, Roger Gunn, Alexan- der Hammers, David Alexander Dickie, Maria Vald´ es Hern´ andez, Joanna Wardlaw, and Daniel Rueckert. Gan augmentation: Augmenting training data using generative adver- sarial networks.arXiv preprint arXiv:1810.10863,
-
[2024]
Valentin De Bortoli. Convergence of denoising diffusion models under the manifold hypoth- esis.arXiv preprint arXiv:2208.05314,
-
[2025]
Quentin Bertrand, Avishek Joey Bose, Alexandre Duplessis, Marco Jiralerspong, and Gau- thier Gidel. On the stability of iterative retraining of generative models on their own data.arXiv preprint arXiv:2310.00429,
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.