{"id":"893f323d-a93d-46d7-b7c1-a28717dd85d7","arxiv_id":"2602.16065","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Contaminated recursive training converges to the true distribution at rate t^{-min(p, α)} — the slower of the model's baseline rate p and the real-data fraction α.","lead":"This paper proves that generative AI models trained over many rounds on a mix of their own previous outputs and fresh real data still converge to the real data distribution, as long as each round adds at least some real data. The convergence speed is the slower of the model's baseline learning rate and the fraction of real data used per round, a result that gives a quantitative handle on the model collapse debate.","discovery_kind":"first_principles","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Assumption A1 is not established for the data-dependent mixture distributions Q_t reached during CRT; without it, Theorem 3.4 does not apply to the deep generative models the paper claims to cover.","rationale":"The central claim is a conditional theorem: under A1 and A2, CRT converges at rate t^{-min{p,alpha}}. The proof's key step is applying A1 to Q_t, the mixture distribution of the accumulated training data. The reader's weakest_assumption correctly identifies that A1 must hold uniformly over a convex class containing all such mixtures, and that this is not verified for deep generative models. I agree this is the most load-bearing concern because the paper's motivating examples (GANs, diffusion, LLMs) depend on it. I additionally note a technical gap the reader did not emphasize: the accumulated data are not i.i.d. from Q_t, so A1 as stated does not directly apply even if Q_t is in the class. However, this gap is potentially repairable by strengthening A1 to cover block-structured data, and the simulations for ECDF/KDE suggest the rate may still hold empirically. The uniformity/regularity concern is more fundamental: the mixtures reached during training may leave the smoothness classes for which the cited rates are known. This does not invalidate the conditional theorem, but it means the paper's broad claims about real generative models are conditional on an unverified assumption. The reader's CONDITIONAL verdict is therefore appropriate, and our concern does not shift it.","tokens_in":21343,"tokens_out":20862,"duration_ms":194902,"concrete_test":"In the WGAN CRT simulation of §5.1.2, at iterations t in {50, 150, 500}, construct the mixture Q_t = (Σ_{i=1}^t m1 P0 + Σ_{j=1}^{t-1} m2 hat P_j)/M_t using the actual fitted generators. Draw an i.i.d. sample of size M_t from Q_t, train a freshly re-initialized WGAN (same architecture/hyperparameters) on it, and measure d(hat P_t^{iid}, Q_t). Fit log d vs log M_t to estimate the observed exponent and compare with the theoretical p≈0.5. If the i.i.d.-trained error is not ≲ M_t^{-p} uniformly, A1 is violated; if the CRT error is substantially worse than the i.i.d. error, the block-dependence gap is material.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The proof of Theorem 3.4 hinges on bounding d(hat P_t, Q_t) by M M_t^{-p} via Assumption A1, treating Q_t as the distribution of the accumulated training data. Two gaps make this step unsupported. First, the accumulated dataset is not an i.i.d. sample from Q_t: it is a block-structured sample (m1 points from P0 at each round, m2 points from each previous generator), so the joint law is a product of different measures, not Q_t^{M_t}. A1, as defined in Definition 3.2, applies to estimators trained on i.i.d. samples, and no argument transfers the uniform rate to block-dependent data. Second, even granting an i.i.d. sample from Q_t, A1 requires the same rate p to hold uniformly over every distribution in a convex class Q that contains all such mixtures. For deep generative models, the cited rates (e.g., Oko et al. 2023 for diffusion, Uppal et al. 2019 for GANs) are proved under smoothness/regularity conditions on the target; the mixtures Q_t are data-dependent convex combinations of P0 and previous generators, and the paper neither proves that these mixtures remain in the required regularity class nor verifies it empirically beyond KDE/ECDF. If A1 fails at any t, the recursion's error bound (Equation 1) breaks and the t^{-min{p,alpha}} rate is not guaranteed.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies recursive training in which each iteration's training set accumulates fresh real data from the target distribution and synthetic data drawn from previous generators. It defines Contaminated Recursive Training (CRT) and a biased variant (BCRT), and proves, under Assumptions A1 (a uniform polynomial convergence rate p over a convex distribution class Q) and A2 (convexity of the metric), that the CRT estimator satisfies d(hat P_t, P0) ≲ t^{-min(p, alpha)} up to a log factor, and that BCRT satisfies t^{-min(p, q, alpha)}. The proof is a recursion on the partial sums of distributional errors, analyzed with a gamma-function bound. Simulations with ECDF/KDE, a WGAN-style generator, and an MNIST diffusion model are reported. The central issue is whether Assumption A1 can legitimately be applied to the data-dependent mixture distributions Q_t and to the stratified/block-structured accumulated samples that arise in the CRT procedure; the manuscript asserts this via citations for deep generative models but does not verify it.","tokens_in":21609,"tokens_out":12451,"duration_ms":125066,"significance":"If made rigorous, this is a valuable positive counterpart to the model-collapse literature: it gives a clean phase-transition rate controlled by the slower of the baseline convergence rate and the real-data fraction, and it covers both unbiased and biased sampling regimes. The recursion is elegant, the gamma-function calculation is correct, and the paper ships code and extensive simulations. However, the main theorem is only as strong as its verification of Assumption A1 along the recursive trajectory; in the current version this verification is missing for the deep generative models that the paper claims to cover. The ECDF/KDE simulations are consistent with the theory, but the WGAN and diffusion experiments do not validate Assumption A1 for their respective recursive trajectories.","major_comments":[{"comment":"The step d(hat P_t, Q_t) ≲ M_t^{-p} applies Assumption A1 to the accumulated dataset as though it were an i.i.d. sample from Q_t. Under Definition 3.3, however, the accumulated data are a fixed-block product measure: (t+1)m1 points from P0 and m2 points from each of bP_0,...,bP_{t-1}. Assumption A1 is stated for i.i.d. samples (Definition 3.2), and no argument transfers the uniform i.i.d. rate to this stratified dependence structure. For the ECDF the transfer is standard, but for a general learner in A1 it is not automatic. The recursion in Eq. (1) therefore lacks support unless A1 is replaced by a stronger 'stratified-sample' rate or a separate transfer argument is supplied.","section":"§B.1, Eq. (1)"},{"comment":"The theorem requires A1 to hold uniformly on a convex class Q containing the mixtures Q_t = (tm1 P0 + m2 Σ_{j=1}^{t-1} bP_j)/M_t. Two things are not established. First, the previous estimators bP_j are data-dependent and need not lie in Q, so Q_t need not be in Q even if P0 is. Second, the cited uniform rates for GANs and diffusion models (e.g., Uppal et al. 2019; Oko et al. 2023) hold under regularity conditions on the target, and the mixtures Q_t are not shown to satisfy these conditions. The KDE/ECDF simulations are consistent with A1, but Section 5.1.2 uses a generator trained with an exact quantile-based W1 loss rather than a true WGAN, and Section 6 does not evaluate distributional distances; neither validates A1 for the recursive trajectory. Without this verification, Theorem 3.4 does not apply to the deep generative models claimed in the abstract and introduction.","section":"§3, Assumption A1 / Theorem 3.4"},{"comment":"The proof's Q_t and M_t do not match Definition 3.3. After t iterations the accumulated data contain (t+1)m1 real points and t m2 synthetic points, with the most recent synthetic block Y_t drawn from bP_{t-1}; the proof uses M_t = t m1 + (t-1)m2 and omits the bP_0 component and the most recent synthetic block. The recursion should be re-derived with the correct counts. The rate is probably unchanged, but as written the proof is not a proof of the stated procedure.","section":"§B.1, definition of Q_t"}],"minor_comments":[{"comment":"The claims that the results hold 'without distributional assumptions on the data' and with 'minimal assumptions' overstate the role of Assumption A1, which requires a uniform polynomial rate over a convex distribution class and is a genuine regularity/rate assumption.","section":"Abstract / §1"},{"comment":"There is a typo: 'we obtain We first use the triangle inequality' should be 'We first use the triangle inequality'.","section":"§B.1, first paragraph"},{"comment":"For (alpha, q) = (0.5, 0.25), the theoretical rate is min(0.5, 0.25, 0.25) = 0.25, but the reported empirical slopes are 0.325, 0.421, 0.332, 0.330. The discrepancy is not discussed; a note on burn-in, finite-sample bias, or fitting variability would help.","section":"Table 2"},{"comment":"The 'WGAN' experiment replaces the critic with an exact empirical W1 quantile loss; this is not a standard WGAN. The theoretical rates cited for GANs (Table 1, Uppal et al. 2019) may therefore not apply to this modified estimator, and the experiment should be described cautiously.","section":"§5.1.2"},{"comment":"The MNIST experiment shows generated samples but does not compute a quantitative distributional distance to the true MNIST distribution, so it can only be read as a sanity check, not as validation of the t^{-min(p, alpha)} rate.","section":"§6"}],"recommendation":"major_revision","confidential_remarks":"The central idea is worth publishing if the authors either strengthen Assumption A1 to a CRT-adaptive condition that is verifiable for the models they simulate, or restrict the theoretical claims to estimators such as ECDF/KDE where the uniform rate is actually established and present the deep-model results as empirical evidence. The main obstacle is the uncritical application of A1 to block-structured data and to data-dependent mixture distributions Q_t; the off-by-one indexing in Q_t is a smaller but real proof defect."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is good: under a generic baseline rate p and fresh-real-data fraction α, CRT with accumulation converges at t^{-min(p,α)}. That's a genuinely useful counterweight to the model-collapse literature, and the BCRT extension to bias decay (min(p,q,α)) is natural. The recursion in Theorem 3.4 is tidy, and the gamma-function bound in the appendix is correct. This is a real contribution, not a repackaging.\n\nThe soft spot is the step that makes the theorem load-bearing. In the proof, you need d(P̂_t, Q_t) ≲ M_t^{-p}, where Q_t is the mixture of the true distribution and all previous estimators. Assumption A1 gives that rate for i.i.d. samples from a fixed distribution in Q. But the accumulated dataset is not an i.i.d. sample from Q_t: it's a block-structured product of m₁ draws from P0 and m₂ draws from each previous generator. The estimator is trained on that product measure, and no argument transfers A1 to that setting. The paper also does not prove that the data-dependent mixture distributions stay inside the convex class Q for which the baseline rate is assumed uniform. The citations to GAN and diffusion rates don't cover these mixtures; at best they cover each component separately. This is not a technicality — if A1 cannot be applied to Q_t, the recursion's error bound breaks.\n\nI also note the authors honestly flag in the Discussion that cross-entropy and KL do not satisfy the convexity assumption, so the result doesn't directly apply to the LLM setting that motivates the paper. That's a limitation, not a flaw, but it does narrow the practical reach.\n\nThe experiments are okay but not decisive. The ECDF/KDE rate fits are consistent with the theory, and the WGAN results are roughly in line, but there are no error bars, WGAN is dropped from the BCRT section, and the MNIST part is qualitative images. The theory deserves stronger empirical backing, especially given the gap in the proof.\n\nBottom line: this is a paper worth engaging with, and it deserves a serious referee. The central conjecture is plausible, but the proof as written has a load-bearing hole. I'd recommend sending it to review and asking the authors to either prove a uniform rate for the block-structured data or reformulate A1 so that it applies directly. Until then it's a conditional result, not a theorem.\n\nEnd with: I'd bring this to reading group and cite it if the gap gets fixed.","headline":"The rate t^{-min(p,α)} is novel and the recursion proof is clean, but the key step applies an i.i.d. convergence-rate assumption to block-structured, data-dependent training data — a real gap that the paper does not close.","tokens_in":22201,"tokens_out":4410,"would_cite":true,"duration_ms":45275,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G20","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proves that generative models trained on their own past outputs, mixed with a steady stream of new real data, still converge to the true data distribution — at a rate set by the smaller of the model's baseline speed and the real-d","keywords":["generative AI","recursive training","data contamination","model collapse","convergence rates","synthetic data","phase transition","bias correction"],"falsifier":"Run contaminated recursive training on a multimodal target with a generator class whose baseline rate is fast but whose uniform rate over mixtures of its own empirical outputs is unproven, set α below p, and measure d(P̂_t, P0) across thousands of iterations; any decay slower than t^{−α}, or a plateau, refutes the theorem's prediction for that class. Equivalently, repeating the CRT experiment with a non-convex distance such as KL divergence would indicate whether the convexity assumption A2 is truly load-bearing.","tokens_in":21124,"feed_emoji":"🤖","tokens_out":11076,"duration_ms":94913,"temperature":0.7,"pith_summary":"The paper asks whether generative models can survive being trained on their own output. Its central claim is that they can, provided each training round adds fresh real data, and that earlier collapse results came from settings where synthetic data replaced real data rather than accumulating alongside it. The authors prove that under contaminated recursive training, the distance from the current model to the true data distribution decays like t raised to minus min(p, α), where p is the model's baseline convergence rate and α is the fraction of real data in each batch. A phase transition appears: whichever of p and α is smaller controls the speed. The same analysis extends to biased real data, where an additional bias-decay rate q enters as min(p, q, α), and to fixed bias, which makes the procedure converge to the biased distribution instead of the true one.","feed_headline":"AI trained on its own output converges if real data share stays high","feed_subtitle":"New theorem: recursively trained generators converge at rate min(baseline speed, real-data share); collapse is avoidable.","key_machinery":"The engine of the proof is a recursion inequality. Writing S_{t−1} = Σ_{j<t} d(P̂_j, P0), the error at step t satisfies d(P̂_t, P0) ≲ M_t^{−p} + (m2/M_t) S_{t−1}, where M_t = t m1 + (t−1) m2 is the accumulated sample size. The first term is the baseline learning error; the second measures how much contamination from earlier generators leaks in, weighted by the fraction of synthetic data. Unrolling this recursion and evaluating the product over k of (1 + (1−α)/k) through Gamma-function identities turns the contamination sum into t^{−α} Σ j^{−p−1+α}, which is then bounded by regime: constant, log t, or t^{α−p}, depending on whether p > α, p = α, or p < α. That split produces the phase transiti","core_discovery":"Theorem 3.4 states the central result: let P0 be the target distribution and let P̂_t be the generator produced at iteration t by contaminated recursive training (CRT), which accumulates all past real samples and all past synthetic samples. If the baseline estimator has a uniform polynomial convergence rate p over a convex class of distributions (Assumption A1) and the distributional distance is convex in its second argument (Assumption A2), then d(P̂_t, P0) ≲ t^{−min{p, α}}, up to a log factor at p = α. Thus recursive training does not inevitably collapse: it converges to the true distribution, at a rate set by the slower of the model's intrinsic speed and the fraction of genuinely real dat","pith_inferences":["I infer a design rule the authors do not state: knowing or estimating the baseline rate p allows a practitioner to set α just above p to keep baseline speed, or deliberately below p to trade convergence speed for compute — an ablation over α on one fixed architecture would test this directly.","I infer that the phase transition is sensitive to the choice of distributional distance. The paper excludes cross-entropy and KL divergence; for likelihood-trained LLMs the effective rate may follow a different law, so extending or disproving the min(p, α) formula for non-convex losses is the natural next experiment.","Real internet contamination is selective: humans publish only some generations and amplify others. I infer that selection inflates the effective synthetic fraction, so in practice the collapse threshold could appear at a larger nominal α than the theorem predicts; the paper itself lists selection as future work.","Because Assumption A1 is verified in simulation only for ECDF and KDE and otherwise imported from published rates, I would treat the theorem's coverage of deep generators (GANs, diffusion models) as conditional on that uniform-rate check rather than automatic."],"forward_implications":["If the real-data fraction α exceeds the model's baseline rate p, recursive training costs nothing asymptotically: the CRT sequence matches the uncontaminated rate t^{−p}.","If α < p, the recursion is data-limited and converges at t^{−α}; at α = p, an extra log factor appears.","When real data are drawn from a fixed biased distribution and never corrected, CRT converges to that biased distribution at the same rates — bias is learned, not erased.","If the bias in real data decays at rate q, convergence to the true distribution continues at t^{−min{p,q,α}}, so bias correction is never wasted even after biased initial training.","Accumulating all past real and synthetic data, rather than replacing old data with synthetic samples, is what prevents collapse; the theorem covers nonparametric target distributions and general universal-approximator generators."],"fun_headline_variants":["AI trained on own output converges if enough real data remains","Convergence rate for AI self-training: min(model speed, real-data share)","New theorem: AI collapse is avoidable under mild conditions","Bias correction stops early bias buildup in recursive AI training"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is Assumption A1: a single uniform polynomial rate p must hold over a convex class that contains every mixture distribution reached during recursive training, including mixtures of previous data-dependent generators; the paper verifies this directly only for ECDF/KDE in simulation and otherwise cites published rates.","fun_headline_variants_meta":{"raw":{"variants":["AI trained on own output converges if enough real data remains","Convergence rate for AI self-training: min(model speed, real-data share)","New theorem: AI collapse is avoidable under mild conditions","Bias correction stops early bias buildup in recursive AI training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000344,"raw_usage":{"total_tokens":1693,"prompt_tokens":676,"completion_tokens":1017,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":420,"completion_tokens_details":{"reasoning_tokens":945}},"tokens_in":420,"tokens_out":1017,"duration_ms":12584,"temperature":1.0,"reasoning_tokens":945,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T22:41:41.854159+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run contaminated recursive training on a multimodal target with a generator class whose baseline rate is fast but whose uniform rate over mixtures of its own empirical outputs is unproven, set α below p, and measure d(P̂_t, P0) across thousands of iterations; any decay slower than t^{−α}, or a plateau, refutes the theorem's prediction for that class. Equivalently, repeating the CRT experiment with a non-convex distance such as KL divergence would indicate whether the convexity assumption A2 is truly load-bearing.","supporting_citations":[],"review_version":1}