Pith. sign in

REVIEW 4 major objections 3 minor 17 references

Verifier-based filtering of synthetic data prevents model collapse in iterated retraining; the retrained estimator converges to the verifier's knowledge center, so only an unbiased verifier converges to the truth.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 09:12 UTC pith:R4HG35AE

load-bearing objection A real linear-regression result on verifier-filtered retraining, but the abstract outruns the theorems: convergence to the verifier's knowledge center is proven only for a hand-built orthogonal design, and the promised LLM experiment is missing. the 4 major comments →

arxiv 2510.16657 v3 pith:R4HG35AE submitted 2025-10-18 stat.ML cs.LG

Escaping Model Collapse via Synthetic Data Verification: Near-term Improvements and Long-term Convergence

Bingji Yi , Qiyuan Liu , Yuwei Cheng , Haifeng Xu This is my paper
classification stat.ML cs.LG MSC 62J0560J2062F12
keywords model collapsesynthetic dataverifier filteringlinear regressiontruncated Gaussiancontraction mappingbias-variance trade-offiterative retraining
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Model collapse—the steady degradation of a model retrained on its own synthetic outputs—can be escaped if an external verifier filters those outputs first. The paper argues this in the canonical linear-regression setting: one round of verified retraining lowers mean squared error when the verifier's bias is small, because filtering trades variance for a controllably small bias. Over many rounds, the update becomes a contraction that drives the estimator to the verifier's knowledge center; an unbiased verifier therefore converges to the true parameter, while a biased verifier eventually plateaus or reverses. The same qualitative pattern appears in conditional VAE experiments on MNIST, where unfiltered retraining collapses and filtered retraining produces sharp digits that stop improving the worse the verifier is. The result matters because it explains both why verification works in practice and why the long-run ceiling is set by the verifier, not by the accumulation of real data.

Core claim

Formally, the paper shows that verifier-guided retraining in linear regression converges to θ_c, the center of the verifier's knowledge ball, rather than to the true coefficient θ*. The one-step result (Theorem 3.1) decomposes the mean squared error into a synthetic-variance term roughly of order 1/n1 and a verification bias-plus-variance term, and shows the total can fall below the real-data baseline when the verifier's bias is small. The long-run result (Theorem 4.1) represents the iterate as a Markov recursion θ_{k+1} = T(θ_k) + η_{k+1}, proves T is a contraction with fixed point θ_c, and bounds the expected squared distance by ρ^{2k} times the initial error plus a decaying noise sum; whe

What carries the argument

The load-bearing object is the verifier's knowledge set, a ball B_r(θ_c) containing the true θ*, accessed only through binary accept/reject decisions on each (x,y) via |y − x^T θ_c| ≤ r||x|| + σ_c. The analytical engine is the synthetic covariate design: rows are repeats of a fixed orthonormal set {v_j} chosen as the right singular vectors of the real data matrix, which diagonalizes the retraining transition into independent one-dimensional truncated-Gaussian mean-estimation problems along each v_j. In one dimension, accepting samples truncates a Gaussian noise distribution, and the induced map T has derivative equal to the variance of that truncated normal—strictly less than 1, so T is a co

Load-bearing premise

The proofs require that the verifier's knowledge is a symmetric ball centered at θ_c containing θ* and that synthetic covariates are constructed from a fixed orthonormal block design aligned with the right singular vectors of X0; for other covariate designs the paper asserts but does not prove similar conclusions.

What would settle it

Run the same generate–verify–retrain loop in linear regression but draw synthetic covariates as random Gaussian directions rather than the fixed orthonormal block design; if an unbiased verifier then fails to drive the estimator to θ* (or if iterates diverge), the claimed contraction mechanism is an artifact of the design choice. A less drastic check: measure whether a single round's MSE improvement crosses from improvement to degradation exactly at the (bias, radius) boundary predicted by Theorem 3.1's truncated-Gaussian constants.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • With an unbiased verifier, iterative retraining on verified synthetic data alone converges to the true parameter, so verified synthetic retraining can fully escape model collapse.
  • With a biased verifier, early MSE improvements are real but transient: the estimate approaches θ_c, leaving an irreducible error equal to the verifier's bias; selectivity affects speed, not the limit.
  • A single bit of accept/reject feedback per synthetic sample suffices to alter the asymptotic dynamics—the verifier never needs to reveal θ_c or r.
  • The contrast with unfiltered retraining, where the update is identity-like, explains why raw synthetic recursion collapses while verification prevents it.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the contraction picture extends beyond the exact block design, then in language-model fine-tuning with LLM-as-a-judge, early gains should be trusted cautiously: the judge's systematic preferences, not the data distribution, will set the eventual performance ceiling.
  • The one-step theorem predicts a quantitative phase boundary in (verifier bias, selectivity) coordinates; a fine-grained simulation sweep measuring log-MSE change would test whether the truncated-Gaussian constants m1, m2, m3 from the theory match empirically.
  • A verifier biased toward easy-to-generate modes, as the paper's discriminator is, should leave measurable diversity deficits; tracking rare-class coverage across iterations would make the knowledge-center effect visible in generated-sample distributions.
  • The theory suggests a resource-allocation rule for building pipelines: spend budget on reducing verifier bias rather than only on more synthetic samples, because sample size controls convergence speed but verifier bias controls the fixed point.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. This paper studies iterative retraining of a generative model on synthetic data that has been filtered by an external verifier. In a linear-regression setting with a verifier that accepts samples satisfying |y − x^T θ_c| ≤ r||x|| + σ_c, the authors prove a one-step MSE decomposition (Theorem 3.1) showing that verifier filtering can improve the estimator when the verifier's bias is small and the retained synthetic sample is large, and a long-run contraction result (Theorem 4.1) showing that iterated verifier-guided retraining converges to the verifier's 'knowledge center' θ_c when the retained sample size per iteration grows. The paper also reports linear-regression simulations and CVAE-on-MNIST experiments that qualitatively match these predictions. The advertised central finding is that verifier-guided synthetic retraining avoids model collapse, with the caveat that a biased verifier causes a plateau or degradation in the long run.

Significance. If the formal results were fully established in the claimed generality, this would be a useful contribution to the model-collapse literature: it quantifies a bias–variance trade-off introduced by verifier filtering, shows that the long-run limit is the verifier's knowledge center, and explicitly computes truncated-normal moments in the appendices. The contraction perspective on synthetic retraining is a genuine and potentially influential idea. However, the significance is currently limited by two factors: the proofs hold only for a block-orthogonal synthetic covariate design, and the verifier's center θ_c is an input to the acceptance rule, so the 'convergence to the knowledge center' is partly a modeling consequence. The broad 'no model collapse' claim in the abstract is not supported by the formal content.

major comments (4)
  1. [Abstract; Section 4] The abstract states that verifier-based synthetic retraining 'will not cause model collapse,' but this is contradicted by the paper's own formal results. Theorem 4.1 shows convergence to the verifier's knowledge center θ_c, and Section 4 explicitly lists 'strongly biased verifier' as a regime where retraining 'leads to degradation and may even cause collapse.' Because the verifier can be biased, the unqualified headline claim is false. The abstract and introduction should be rephrased to state that verifier filtering prevents collapse only when the verifier is sufficiently reliable, matching Eq. (9) and the three-phase discussion.
  2. [Section 2, Eq. (3), Algorithm 3] Theorems 3.1 and 4.1 are proven only for the block-orthogonal design in which synthetic covariates repeat the right singular vectors of X_0 (Eq. (3) / Algorithm 3), diagonalizing the transition into independent one-dimensional truncated-Gaussian processes. Section 2 asserts that 'alternatives ... can yield similar qualitative conclusions' and that 'any reasonable design' should generalize, but no proof is supplied. The VAE/MNIST experiments use a discriminator verifier with top-10% retention and natural synthetic draws, not the ball rule of Eq. (1), so they do not close this gap. The paper's broad practical claim that verifier filtering avoids model collapse is therefore an extrapolation beyond the proven setting.
  3. [Section 4, Eq. (1), Theorem 4.1] The convergence point θ_c is built into the verifier model: the acceptance region in Eq. (1) is centered at θ_c, and the fixed point identified in the proof of Theorem 4.1 is the center of that symmetric interval. The nontrivial content is that the iteration is a contraction, but the statement that the estimator 'converges to the verifier's knowledge center' is largely a modeling consequence rather than an emergent prediction. The authors should state this explicitly and discuss what, if anything, is predicted about the long-run limit beyond the definition of the verifier.
  4. [Appendix A, Theorem A.1; Theorem 3.1, Eq. (7)] The probabilistic statement of Theorem A.1 and its use in Theorem 3.1 are not well-posed. The left-hand side of Eq. (15)/(7) is an unconditional expectation over the initial estimator and synthetic data, yet the theorem claims the inequality holds 'with probability at least ...' In the proof, the event |ϵ_1| < n_0^{-1/3} is used to Taylor-expand the truncated-normal moments, but the complement of this event is never integrated or bounded. If the intended statement is conditional on the initial estimator, the notation should say so; if unconditional, the tail contribution must be controlled. This is load-bearing for the one-step improvement claim.
minor comments (3)
  1. [Throughout] The text contains several typos and formatting artifacts: 'recent study' should be 'recent studies'; 'fining-tuning' should be 'fine-tuning'; Figure 3 captions contain garbled '□' symbols instead of minus signs. A careful proofreading pass is needed.
  2. [Appendix A, Algorithm 2] The retention count is treated as deterministic in Theorem A.1 but is random in Algorithm 2 unless a fixed-retention stopping rule is used. Please specify whether n_1 is the number of generated samples or the number of accepted samples, and adjust the notation consistently.
  3. [Section 5.2] The FID curves in Figure 5 appear to be single runs. Given the qualitative claims about early gains and long-term plateau, reporting multiple seeds with error bars would strengthen the empirical support.

Circularity Check

1 steps flagged

Mild self-definitional loading: the long-run limit is the verifier's assumed center θc, but the one-step bias-variance trade-off and contraction proofs are independent.

specific steps
  1. self definitional [Section 2 Eq. (1); Section 4 Theorem 4.1; Appendix B Eq. (43); Appendix A.3 fixed-point derivation]
    "The verifier's knowledge is described by a spherical ball Br(θc) := {θ ∈ R^p : ||θ−θc|| ≤ r}, with fixed center θc... the verifier outputs Yes if |yi − xi^T θc| ≤ r||xi|| + σc. ... In particular, if lim_{k→∞} n_k = ∞, then lim_{k→∞} E||θhat_k − θc||^2 = 0."

    The convergence point is the center of the filter. In the proof, the accepted noise in each direction is Ntrunc with interval symmetric around −v_j^T(θhat_k−θc)/σ, and the one-dimensional deterministic map has fixed point (α+β)/2; with α=−β this is exactly θc. Thus Theorem 4.1's long-run limit is an input to Eq. (1), not an independently predicted quantity; the asymptotic plateau ||θc−θ*||^2 is the assumed bias Δ^2. The contraction argument and Theorem 3.1's one-step bias-variance decomposition are nontrivial and independent, so the circularity is partial and modeling-consequence-like rather than full.

full rationale

The paper contains no author self-citations and no fitted parameter is renamed as a prediction. The main mathematical work is a self-contained analysis of a stated verifier model: Theorem 3.1 derives a one-step MSE bound from a truncated-Gaussian moment expansion, and Theorem 4.1 proves a contraction inequality for the block-orthogonal design of Algorithm 3. The one genuinely construction-driven aspect is that the long-run fixed point is the verifier's knowledge center θc, which is the center of the acceptance ball in Eq. (1); the proof explicitly reduces the dynamics to a one-dimensional truncated normal centered at the current distance to θc, so convergence to θc is a consequence of the filter's symmetry. This is a modeling implication rather than a hidden reuse of the target conclusion, and the near-term improvement result has independent content. The paper's additional claim that results generalize to 'any reasonable design' is unsupported—the theorems are proved only for the design in Eq. (3)/Algorithm 3—but that is a limitation in scope, not circularity. External experiments (linear-regression simulations, VAE/MNIST, SmolLM2/XSUM) provide independent empirical context, though the VAE verifier is a discriminator rather than the ball rule. Overall, no significant circularity beyond the mild self-definitional loading on θc, so the score is low.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 1 invented entities

The central result is built on a stylized verifier model rather than on independently measured quantities. The verifier's ball B_r(θ_c) and binary rule do all the work in determining the long-run attractor, and the block-orthogonal synthetic design is what makes the proofs tractable. These are transparently stated, not hidden, but they are postulates. The only truly fitted number in the experiments is the 10% retention threshold for VAEs; the other parameters (θ_c, r, σ_c, sample schedules) are model inputs set by hand. No new physical or mathematical entities are introduced beyond the verifier construct.

free parameters (3)
  • Verifier knowledge center θ_c
    Input to Eq. (1); the claimed long-run attractor. Set as θ*+Δ·u in simulations; not learned from data.
  • Verifier radius r and noise allowance σ_c
    Control acceptance width and convergence rate; swept r=0.1–1.5 in experiments; not fitted.
  • VAE retention threshold = top 10% per class
    Chosen after a one-step study for best FID/quality-diversity balance; not derived from theory.
axioms (6)
  • domain assumption Verifier knowledge set is a ball B_r(θ_c) containing θ*, and its feedback is Eq. (1).
    Defines the verifier; its center θ_c becomes the long-run fixed point; no empirical justification that real verifiers have such a symmetric rule.
  • ad hoc to paper Synthetic covariate design is block-orthogonal along right singular vectors of X0 (Eq. 3/Algorithm 3).
    Makes the Markov update diagonal; generalization to arbitrary designs is conjectured (Section 2).
  • domain assumption Data follow linear model y=x^Tθ*+ξ, ξ~N(0,σ^2), and σ is known.
    Canonical setting from prior model-collapse theory; limits scope of theorems.
  • domain assumption For Theorem 3.1, µ_j=ω(√n0) and n1>n0≥100.
    Technical high-probability condition; not verified in real data.
  • domain assumption For Theorem 4.1, n_k non-decreasing and n_k→∞.
    Required for noise term to vanish; if sample size bounded, limit may not be θ_c.
  • standard math Truncated-normal moment formulas and Bernstein/sub-exponential concentration bounds.
    Standard tools used in Appendices A-B; assumed without proof.
invented entities (1)
  • External verifier with knowledge center θ_c no independent evidence
    purpose: Filters synthetic samples before retraining; injects external information into the loop.
    No independent falsifiable handle; the convergence-to-θ_c result is a direct consequence of this postulated filter, so the entity carries most of the paper's content.

pith-pipeline@v1.3.0-alltime-deepseek · 22992 in / 18601 out tokens · 149525 ms · 2026-08-04T09:12:12.122036+00:00 · methodology

0 comments
read the original abstract

Synthetic data has been increasingly used to train frontier generative models. However, recent studies raise key concerns that iteratively retraining a generative model on its self-generated synthetic data may keep deteriorating model performance, a phenomenon often coined model collapse. In this paper, we investigate ways to modify the synthetic retraining process to avoid model collapse, and even possibly help reverse the trend from collapse to improvement. Our key finding is that by injecting information through an external synthetic data verifier, whether a human or a better model, synthetic retraining will not cause model collapse. Specifically, we situate our theoretical analysis in the fundamental linear regression setting, showing that verifier-guided retraining can yield near-term improvements, but ultimately drives the parameter estimate to the verifier's "knowledge center" in the long run. Our theory further predicts that, unless the verifier is perfectly reliable, these early gains will plateau and may even reverse. Indeed, our experiments across linear regression, Variational Autoencoders (VAEs) trained on MNIST, and fining-tuning SmolLM2-135M on the XSUM task confirm these theoretical insights.

Figures

Figures reproduced from arXiv: 2510.16657 by Bingji Yi, Haifeng Xu, Qiyuan Liu, Yuwei Cheng.

Figure 1
Figure 1. Figure 1: Iterative Retraining of a Variational Autoencoder (VAE) on MNIST. Left: [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Generate-Verify-Retrain pipeline corresponding to Scheme 1. Since learning proceeds through the conditional Y k | Xk , synthetic retraining requires specifying the covariate design Xk ; labels Y k are then generated conditionally via the model under verifier constraints. In principle, one could construct Xk arbitrarily; however, for mathematical clarity, below we describe a targeted though arguably natural… view at source ↗
Figure 3
Figure 3. Figure 3: Error changes of the one-step retraining estimator ˆθ 1 versus estimator ˆθ 0 only using original real data, measured by log(||θˆ0−θ ⋆|| ||θˆ1−θ⋆||): theory’s prediction (left) and empirical comparisons (right). Iterative Synthetic Retraining. Figure 4a provides empirical support for Theorem 4.1, showing that under a biased verifier, the retrained estimator converges to the verifier’s knowledge center θc r… view at source ↗
Figure 4
Figure 4. Figure 4: Iterative synthetic retraining with and without bias. 5.2 Variational Autoencoders (VAEs) on MNIST Setting and evaluation metrics. We adopt Conditional VAEs (CVAEs) to leverage class conditioning and avoid verifier-induced imbalance: without label conditioning, easily generated digits dominate the retained synthetic data, leading to skewed retraining. To test the bias–variance trade-off and the verifier’s … view at source ↗
Figure 5
Figure 5. Figure 5: a reports FID scores (lower is better) across retraining iterations under a strong verifier (a discriminator trained on 60K real dataset and 60K synthetic data). Rapid improvement is observed within the first 15 rounds of retraining, even under small fixed-size schedules for the number of synthetic data per iteration, e.g., green curve (20K) and orange curve (5K) curves. Afterward, the improvement slows an… view at source ↗
Figure 6
Figure 6. Figure 6: Test negative ELBO and reconstruction loss across retraining iterations [PITH_FULL_IMAGE:figures/full_fig_p025_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

17 extracted references · 12 linked inside Pith

  1. [3]

    On the diversity of synthetic data and its impact on training large language models.arXiv preprint arXiv:2410.15226,

    Hao Chen, Abdul Waheed, Xiang Li, Yidong Wang, Jindong Wang, Bhiksha Raj, and Marah I Abdin. On the diversity of synthetic data and its impact on training large language models.arXiv preprint arXiv:2410.15226,

    Pith/arXiv arXiv

  2. [4]

    Universality of the π2/6 pathway in avoiding model collapse.arXiv preprint arXiv:2410.22812,

    Apratim Dey and David Donoho. Universality of the π2/6 pathway in avoiding model collapse.arXiv preprint arXiv:2410.22812,

    Pith/arXiv arXiv

  3. [6]

    A survey on llm-as-a-judge.arXiv preprint arXiv:2411.15594,

    13 Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al. A survey on llm-as-a-judge.arXiv preprint arXiv:2411.15594,

    Pith/arXiv arXiv

  4. [7]

    Textbooks are all you need.arXiv preprint arXiv:2306.11644,

    Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al. Textbooks are all you need.arXiv preprint arXiv:2306.11644,

    Pith/arXiv arXiv

  5. [9]

    Golden ratio weighting prevents model collapse.arXiv preprint arXiv:2502.18049,

    Hengzhi He, Shirong Xu, and Guang Cheng. Golden ratio weighting prevents model collapse.arXiv preprint arXiv:2502.18049,

    Pith/arXiv arXiv

  6. [12]

    Source2synth: Synthetic data generation and curation grounded in real data sources.arXiv preprint arXiv:2409.08239,

    Alisia Lupidi, Carlos Gemmell, Nicola Cancedda, Jane Dwivedi-Yu, Jason Weston, Jakob Foerster, Roberta Raileanu, and Maria Lomeli. Source2synth: Synthetic data generation and curation grounded in real data sources.arXiv preprint arXiv:2409.08239,

    Pith/arXiv arXiv

  7. [13]

    Synthetic data applications in finance.arXiv preprint arXiv:2401.00081,

    Vamsi K Potluru, Daniel Borrajo, Andrea Coletta, Niccolò Dalmasso, Yousef El-Laham, Elizabeth Fons, Mohsen Ghassemi, Sriram Gopalakrishnan, Vikesh Gosai, Eleonora Krea ˇci´c, et al. Synthetic data applications in finance.arXiv preprint arXiv:2401.00081,

    Pith/arXiv arXiv

  8. [14]

    Position: Model collapse does not mean what you think.arXiv preprint arXiv:2503.03150,

    Rylan Schaeffer, Joshua Kazdan, Alvan Caleb Arulandu, and Sanmi Koyejo. Position: Model collapse does not mean what you think.arXiv preprint arXiv:2503.03150,

    Pith/arXiv arXiv

  9. [16]

    A probabilistic perspective on model collapse.arXiv preprint arXiv:2505.13947,

    Shirong Xu, Hengzhi He, and Guang Cheng. A probabilistic perspective on model collapse.arXiv preprint arXiv:2505.13947,

    Pith/arXiv arXiv

  10. [2014]

    Bridging the gap: Enhancing the utility of synthetic data via post-processing techniques.arXiv preprint arXiv:2305.10118,

    Andrea Lampis, Eugenio Lomurno, and Matteo Matteucci. Bridging the gap: Enhancing the utility of synthetic data via post-processing techniques.arXiv preprint arXiv:2305.10118,

    Pith/arXiv arXiv

  11. [2017]

    Quality matters: Evaluating synthetic data for tool-using llms

    Shadi Iskander, Sofia Tolmach, Ori Shapira, Nachshon Cohen, and Zohar Karnin. Quality matters: Evaluating synthetic data for tool-using llms. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 4958–4976,

    2024

  12. [2018]

    Resofilter: Fine-grained synthetic data filtering for large language models through data-parameter resonance analysis

    Zeao Tu, Xiangdi Meng, Yu He, Zihan Yao, Tianyu Qi, Jun Liu, and Ming Li. Resofilter: Fine-grained synthetic data filtering for large language models through data-parameter resonance analysis. InFindings of the Association for Computational Linguistics: NAACL 2025, pp. 5414–5428,

    2025

  13. [2021]

    When models don’t collapse: On the consistency of iterative mle.arXiv preprint arXiv:2505.19046,

    Daniel Barzilai and Ohad Shamir. When models don’t collapse: On the consistency of iterative mle.arXiv preprint arXiv:2505.19046,

    arXiv

  14. [2022]

    Regurgitative training: The value of real data in training large language models.arXiv preprint arXiv:2407.12835,

    Jinghui Zhang, Dandan Qiao, Mochen Yang, and Qiang Wei. Regurgitative training: The value of real data in training large language models.arXiv preprint arXiv:2407.12835,

    Pith/arXiv arXiv

  15. [2023]

    Deepseek-coder: When the large language model meets programming–the rise of code intelligence.arXiv preprint arXiv:2401.14196,

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. Deepseek-coder: When the large language model meets programming–the rise of code intelligence.arXiv preprint arXiv:2401.14196,

    Pith/arXiv arXiv

  16. [2024]

    Escaping collapse: The strength of weak data for large language model training.arXiv preprint arXiv:2502.08924,

    Kareem Amin, Sara Babakniya, Alex Bie, Weiwei Kong, Umar Syed, and Sergei Vassilvitskii. Escaping collapse: The strength of weak data for large language model training.arXiv preprint arXiv:2502.08924,

    arXiv

  17. [2025]

    Preventing model collapse under overparametrization: Optimal mixing ratios for interpolation learning and ridge regression.arXiv preprint arXiv:2509.22341,

    Anvit Garg, Sohom Bhattacharya, and Pragya Sur. Preventing model collapse under overparametrization: Optimal mixing ratios for interpolation learning and ridge regression.arXiv preprint arXiv:2509.22341,

    arXiv