{"id":"e23ebc22-50ee-437d-8534-11d4a43b464f","arxiv_id":"2505.11825","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Bootstrapping diffusion trains per-view diffusion models on partial data and a residual denoiser on a small full-resolution set, claiming provable near-optimal data efficiency that the paper does not actually prove.","lead":"This paper proposes bootstrapping diffusion, a two-stage training method that first trains diffusion models on cheap partial views (low-resolution, patches) and then trains a small residual denoiser on a tiny set of full-resolution images to correct the combined prediction. The authors claim provable near-optimal data efficiency, but the supporting theorems do not actually establish the claimed gains.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The residual-denoiser small-variance assumption in §5.3 is asserted, contradicted by the paper's own motivating regime and experiments, and it is the load-bearing step for the data-efficiency claim.","rationale":"The reader's weakest_assumption identifies the residual denoiser's small-variance property, and I agree that this is the load-bearing step. All downstream theoretical claims — that variance regularization reduces Lipschitz constants, covering numbers, and Rademacher complexity, and hence that bootstrapping diffusion achieves near first-order optimal data efficiency — depend on the residual training target having small variance. The paper asserts this in Section 5.3 without proof, and the assertion conflicts with its own Theorem 5 and its own experimental observations. The identical form of Theorems 3 and 4 is a separate symptom of the same gap: no comparison between the residual bound and the baseline bound is actually proved. The concern is concrete and testable: one can measure the residual variance on the exact experimental setup. If the residual variance is large, the central claim fails; if it is small, the theoretical mechanism would at least be consistent with practice. The verdict should remain REJECT, because the central theoretical claim is not established by the presented theorems and the only experiment lacks a baseline and quantitative metrics.","tokens_in":18320,"tokens_out":3756,"duration_ms":42492,"concrete_test":"Using the AFHQv2-Cat latents and the trained view models from Appendix B, fix N0 = 64 full-resolution latents and, for a range of diffusion times t, estimate E||E[X0|xt] − Σ_i B_i f_i(A_i xt)||^2 and E||X0 − E[X0|xt]||^2 over many noisy xt. If at representative t the residual variance ratio is not small (e.g., above 0.5), the residual target is not low-variance and the variance-regularization argument in §5.3 and §5.5 collapses. A synthetic Gaussian analogue with a blockwise projection matrix A would also settle the structural point.","verdict_should_be":"UNCHANGED","load_bearing_attack":"For bootstrapping diffusion to improve data efficiency, the residual target r(Xt) = E[X0|xt] − Σ_i B_i f_i(A_i xt) must have small variance, so that variance regularization can shrink the effective hypothesis class and reduce the residual denoiser's generalization error. Section 5.3 asserts this because the combined denoiser 'should' approximate the true score, and then concludes that r(Xt) 'should have small variances.' This is not a proof. Theorem 5 shows that the residual variance exactly equals the gap between the combined denoiser's conditional MSE and the MMSE. In the paper's motivating regime — 8×8 patches or low-resolution views of 32×32 latents — that gap is large precisely because the views lack global structural information. The paper's own experiments confirm this: Appendix B states that images from a straightforward linear combination 'fail to produce accurate global structures.' Consequently, the residual variance is large in exactly the regime the method targets, and no theorem quantifies it in terms of uncaptured signal correlation. Without the small-variance property, variance regularization has no demonstrated benefit, and the claimed superiority of Theorem 4 over Theorem 3 is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes 'bootstrapping diffusion,' a two-stage training scheme in which separate diffusion denoisers are first trained on partial or corrupted views (e.g., low-resolution images, image patches, short clips), and then a residual denoiser is trained on a small full-resolution dataset to correct the combined view-based prediction, with variance regularization on the residual output. The main claimed contributions are generalization-error bounds: Theorem 3 for standard full-resolution-only training and Theorem 4 for the bootstrapped residual training, which the paper says show lower generalization errors and 'near first-order optimal data efficiency.' Experiments on AFHQv2-Cat are presented qualitatively.","tokens_in":18554,"tokens_out":6412,"duration_ms":66514,"significance":"If the theoretical claims were established, this would be a practically valuable result: it would provide a principled guarantee that abundant partial or corrupted data can be used to reduce the amount of full-resolution data required for diffusion model training. The paper correctly identifies a meaningful problem and uses standard martingale and covering-number machinery. Some components, such as the orthogonal decomposition in Theorem 5, are correct and potentially useful. However, the central claims are not established by the presented proofs: the two main bounds are identical in form and no comparison is derived, and the load-bearing small-variance assumption on the residual target is asserted rather than proven and is in tension with the paper's own experimental observations.","major_comments":[{"comment":"Theorems 3 and 4 have exactly the same algebraic form. The residual-case bound uses redefined quantities V(S0), R_L(F,S0), and Delta_b, but no inequality is derived between these and the baseline quantities. In fact, the definition of V(S) in Appendix J includes the view denoisers' outputs added inside the squared norm, so it is not even shown that the residual-case V(S0) is no larger than the baseline V(S0). Without a comparison of the redefined terms, the central claim that bootstrapping diffusion achieves lower generalization error than full-resolution-only training is not proven.","section":"Sections 5.2 and 5.4; Appendix J"},{"comment":"The small-variance property of the residual target r(Xt) is asserted in Section 5.3 ('should have small variances') and Appendix A ('if the error variance of combined denoiser approaches the MMSE'), but it is not proven. Theorem 5 shows that the variance of r(Xt) equals the excess conditional MSE of the combined denoiser over the MMSE. In the paper's own motivating and experimental regime, 8x8 patches and low-resolution views of 32x32 latents, the combined denoiser lacks global structure: Appendix B states that linear combinations 'fail to produce accurate global structures.' Hence the residual variance is expected to be large precisely where the method needs it to be small. No theorem bounds the residual variance in terms of the uncaptured signal correlation.","section":"Section 5.3; Appendix A; Theorem 5"},{"comment":"The paper claims that variance regularization reduces the Lipschitz constant of the residual denoiser and thereby reduces the covering number and Rademacher complexity in Theorem 4. However, no theorem or lemma connects the constraint in Equation (6) or the penalty in Equation (7) to a quantitative bound on any of these complexity measures. The claim that variance regularization lowers generalization error is therefore not established by the presented analysis.","section":"Section 5.5; Equations (6) and (7)"},{"comment":"The claim of 'near first-order optimal data efficiency' is not defined and is not derived from any of the theorems. The paper does not state a formal notion of first-order optimality, does not extract a sample-complexity rate from the bounds, and does not provide a lower bound. The statement that 'the data required for training the residual denoiser is proportional to this uncaptured information' is qualitative and does not follow from the displayed bounds.","section":"Abstract; Section 5.5"}],"minor_comments":[{"comment":"The experimental evaluation reports no quantitative metrics such as FID or a comparison against a baseline diffusion model trained only on the same number of full-resolution images. The claim that bootstrapping diffusion 'effectively compensates for missing data' is based solely on visual inspection of generated images.","section":"Appendix B; Section 6"},{"comment":"The notation V(S) is used both for the empirical quantity in Equation (12) and for its expectation E[V(S0)] in the theorems; the distinction between the random empirical quantity and its expectation should be made explicit.","section":"Section 5.2"},{"comment":"Theorem 4 is stated in the main text without redefining F, R_L(F,S0), R_R(F,S0), and Delta_b for the residual case; the statement is incomplete without the definitions in Appendix J, which should be moved to or summarized at the point of the theorem.","section":"Section 5.4; Appendix J"},{"comment":"There are several typos, including 'Radamacher' for 'Rademacher' in Sections 5.2 and 5.4, 'Liptschiz' in Assumption 1, 'propositional' for 'proportional' in Section 5.5, and a reference to 'Algorithm 4' where the algorithm is labeled Algorithm 1.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"The core theoretical claim is not supported by the proofs as written: the residual-case bound has the same form as the baseline bound and no comparison is derived, and the small-variance assumption on the residual target conflicts with the paper's own experimental setting and observations. The experimental section lacks quantitative evaluation. These issues are load-bearing and would require substantial new theoretical results rather than local revision, so I recommend rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the bootstrapping idea is genuinely new and the qualitative experiment gives it some plausibility, but the headline claim — “near first-order optimal data efficiency” — is not proved. The generalization bounds are standard and the residual-case theorem is a restatement, not a comparison.\n\nWhat’s new: the two-stage recipe (per-view denoisers, then a variance-regularized residual denoiser trained on a small full-resolution set) is not in ambient diffusion, GSURE, or the EM-based corrupted-data line. That’s a real contribution, and it’s a sensible way to think about using low-res or patch data as priors.\n\nWhat the paper does well: the uniform-convergence analysis is self-contained and honest about its assumptions. The proof skeleton is standard Rademacher/covering-number machinery, and the use of Tweedie’s formula to relate score estimation to conditional expectation is correct. The experiment—though tiny—shows plausible images from 64 full-res latents plus abundant partial views, and the appendix is transparent about training details.\n\nWhere it falls short, in proportion: the central theoretical claim is not established. Theorem 4 has the same algebraic form as Theorem 3, with V(S0), the Rademacher terms, and the bias all redefined; no inequality between the two is derived. “Near first-order optimal” appears in the abstract and §5.5, but there is no definition of the term, no lower bound, and no rate. More importantly, the small-variance assumption on the residual (§5.3) is asserted, and it’s the load-bearing step: without it, variance regularization has no demonstrated benefit. Theorem 5 shows the residual variance equals the gap between the combined denoiser’s MSE and the MMSE. In the paper’s own motivating regime—8×8 patches and low-res views of 32×32 latents—that gap is large, and Appendix B admits the linear combination “fails to produce accurate global structures.” So the premise holds exactly when you need bootstrapping least. That’s a real conflict, not a nitpick.\n\nThe experiments also lack a baseline: there is no full-res-only model trained on the same 64 images, no quantitative metric, and the combination matrices and range adapter are fit using samples from a model trained on those same 64 latents. For a paper whose thesis is data efficiency, the absence of a comparison against the naive small-set baseline is a significant omission.\n\nWho it’s for: someone working on data-efficient diffusion training will find the idea worth discussing. I wouldn’t cite the theoretical claims in their current form, but the recipe itself is a legitimate starting point. This paper deserves a serious referee: it’s novel, self-contained, and the flaws are fixable in revision—drop or heavily qualify the optimality claim, add a proper baseline, and either prove or clearly state the residual-variance condition as an assumption. I’d send it to review with major-revision expectations.","headline":"Novel two-stage training idea with plausible small-scale results, but the claimed optimality is asserted rather than proved; the residual-case bound is a restatement and the small-variance premise conflicts with the paper's own motivating regime.","tokens_in":19072,"tokens_out":3252,"would_cite":false,"duration_ms":32452,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","68T45"],"pacs":[],"model":"deepseek-v4-flash","headline":"Bootstrapping diffusion lets partial views—low-res images, short clips, crops—substitute for most full-resolution training data: train a residual denoiser on a small clean set, with clean-sample needs scaling like the signal views miss.","keywords":["diffusion models","partial data views","corrupted data","residual denoiser","variance regularization","generalization bounds","score function","data efficiency"],"falsifier":"Compute, on a fixed dataset with known view projections (for example, AFHQv2-Cat latents with 8-by-8 patches and low-resolution views), the residual variance $\\mathbb{E}[\\lVert \\mathbb{E}[X_0\\mid X_t]-\\mathbb{E}[X_0\\mid \\{A_i X_t\\}]\\rVert^2]$ for many noise levels; if it does not shrink relative to the full MMSE term, the complexity-reduction premise is violated. Then run the two-stage training with and without variance regularization for a range of full-resolution set sizes $N$ and compare generalization loss: bootstrapping should beat single-stage training at matching $N$ exactly in the regime where the residual variance is small, and the gap should vanish where it is not.","tokens_in":18091,"feed_emoji":"🧩","tokens_out":10319,"duration_ms":95315,"temperature":0.7,"pith_summary":"The paper attacks the data bottleneck of diffusion training: full-resolution images and long videos are scarce, while low-resolution images, short clips, and watermarked or cropped footage are abundant. It claims these partial views can be exploited by a two-stage scheme: train one diffusion denoiser per view, combine their predictions, then train a residual denoiser on a small full-resolution set to learn only the missing gap. The theoretical core is a pair of generalization bounds showing this residual approach need not be worse than training on the full-resolution set alone, and that variance regularization on the residual shrinks the complexity terms. The data-efficiency payoff is stated sharply: the amount of full-resolution data required scales with the signal correlation the views fail to capture, which the paper calls near first-order optimal efficiency. If correct, the result makes large diffusion training sets less expensive to assemble and lets corrupted or partial collections become first-class training data.","feed_headline":"Partial views can replace most full-resolution diffusion training data","feed_subtitle":"Train view models on low-res or cropped data, then a small residual model on a few clean samples.","key_machinery":"The load-bearing object is the residual denoiser $f_0$ trained on $r(x_t)=\\mathbb{E}[X_0\\mid X_t]-\\sum_i B_i f_i(A_i x_t)$, together with the variance regularization that controls its output norm. The mechanism runs through Theorem 5's decomposition $\\mathbb{E}[\\lVert X_0-\\mathbb{E}[X_0\\mid \\sum_i B_i A_i X_t]\\rVert^2] = \\text{MMSE} + \\mathbb{E}[\\lVert \\mathbb{E}[X_0\\mid X_t]-\\mathbb{E}[X_0\\mid \\sum_i B_i A_i X_t]\\rVert^2]$: when the combined view denoiser is close to the conditional expectation, the residual target has small variance, so a low-Lipschitz, low-capacity network suffices. Covering-number and Rademacher-complexity machinery then converts that capacity reduction into a tighter generalization bound. Tweedie's formula is what lets the whole argument run on conditional-expectation denoisers rather than raw score functions.","core_discovery":"The paper's central discovery is that the score function of a full-resolution diffusion model can be decomposed into a combined partial-view estimate plus a residual, and the residual is the right thing to learn from the scarce clean data. Formally, with clean sample $x_0$, noisy observation $x_t$, and view projections $A_i$, one trains view denoisers $f_i(A_i x_t; \\theta_i) \\approx \\mathbb{E}[A_i X_0 \\mid A_i X_t]$, forms a combined estimate $\\sum_i B_i f_i(A_i x_t)$, and then trains $f_0(x_t;\\theta_0)$ on the residual target $r(x_t)=\\mathbb{E}[X_0 \\mid X_t]-\\sum_i B_i f_i(A_i x_t)$. The generalization bound for this residual denoiser (Theorem 4) is proved in the same form as the bound for training on the full-resolution set alone (Theorem 3), but variance regularization—penalizing $\\lVert f_0\\rVert^2$ or scaling it with a time-dependent range adapter—reduces the Lipschitz constant, hence the covering number and Rademacher complexity that dominate the bound. The paper also proves an orthogonal decomposition (Theorem 5) showing that the combined denoiser's mean squared error equals the MMSE plus the variance of the residual target, so the residual variance is exactly the information the views fail to capture; the residual's training difficulty scales with that variance. Experiments on AFHQv2-Cat with 8-by-8 patches and low-resolution views as partial data, and only 64 full-resolution latents for the residual stage, generate coherent full-resolution images where each view alone fails in a complementary way.","pith_inferences":["Editorial inference — if the combined view models are weak, the residual target's variance is large by Theorem 5 itself, so the theoretical mechanism predicts little or no gain; a practical check is to measure that residual variance on validation data before training the residual stage, treating the small-residual premise as an empirical condition rather than an automatic property of views.","Editorial inference — the paper keeps view denoisers fixed during residual training; an alternating procedure that fine-tunes the view denoisers while fitting the residual could reduce the residual variance further and improve the data-efficiency constant, a natural extension not analyzed here.","Editorial inference — the bounds suggest a scaling rule for practitioners: add more full-resolution data only until the residual variance stops decreasing, since beyond that point the residual is no longer the bottleneck, which directly operationalizes near first-order data efficiency.","Editorial inference — the view models plus residual can be distilled into a single network (the paper lists this as future work), which would turn the method's cost from several models into one at inference while preserving the data-collection savings."],"forward_implications":["A small full-resolution set plus large view sets can yield lower generalization error than the same small set alone; in the paper's experiments 64 full-resolution latents plus patch and low-resolution views produce coherent output.","Data requirements for the residual stage scale with uncaptured signal correlation; the more complementary the views (for example, local patches plus a global low-resolution view), the less clean data are needed.","Variance regularization is not a minor detail: without it, the residual denoiser's complexity would scale with the full-resolution problem and the data-efficiency gain would disappear.","Because view projections include downsampling, cropping, and clipping, the same recipe applies to low-resolution images, short videos, and subtitle- or watermark-affected footage, for any linear view of the data."],"supporting_citations":[{"why":"Supplies the covering-number bound (Assumption 1 and Example 1) that turns Lipschitz and variance reduction into a complexity reduction in Theorems 3 and 4.","marker":"Graf et al. (2022)"},{"why":"Supplies the KL-divergence identity (Eq. 8) connecting score-function errors to distribution distance, which motivates denoiser generalization error as the objective.","marker":"Franzese et al. (2024)"},{"why":"Tweedie's formula identifies score functions with conditional expectations, the training target used for every denoiser in the paper.","marker":"Efron (2012)"},{"why":"Defines Rademacher complexity, the model-complexity measure appearing in the generalization bounds and their symmetrization proof.","marker":"Bartlett & Mendelson (2002)"},{"why":"Provides the AFHQv2-Cat dataset used for the experiments demonstrating bootstrapping diffusion.","marker":"Choi et al. (2020)"}],"fun_headline_variants":["Most diffusion training data replaceable by partial views","Few clean samples suffice when view models cover the rest","Residual score from partial views cuts clean data needs","Score decomposition: learn residual on a few clean samples"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof relies on the residual training target having small variance because the combined view denoisers already approximate the true conditional expectation, and if the views genuinely lack the global information the full model needs, this is exactly the premise that may fail.","fun_headline_variants_meta":{"raw":{"variants":["Most diffusion training data replaceable by partial views","Few clean samples suffice when view models cover the rest","Residual score from partial views cuts clean data needs","Score decomposition: learn residual on a few clean samples"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0012,"raw_usage":{"total_tokens":5032,"prompt_tokens":1117,"completion_tokens":3915,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":733,"completion_tokens_details":{"reasoning_tokens":3853}},"tokens_in":733,"tokens_out":3915,"duration_ms":26168,"temperature":1.0,"reasoning_tokens":3853,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:47:28.065412+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute, on a fixed dataset with known view projections (for example, AFHQv2-Cat latents with 8-by-8 patches and low-resolution views), the residual variance $\\mathbb{E}[\\lVert \\mathbb{E}[X_0\\mid X_t]-\\mathbb{E}[X_0\\mid \\{A_i X_t\\}]\\rVert^2]$ for many noise levels; if it does not shrink relative to the full MMSE term, the complexity-reduction premise is violated. Then run the two-stage training with and without variance regularization for a range of full-resolution set sizes $N$ and compare generalization loss: bootstrapping should beat single-stage training at matching $N$ exactly in the regime where the residual variance is small, and the gap should vanish where it is not.","supporting_citations":[{"cited_title":"On measuring excess capacity in neural networks","cited_arxiv_id":null,"evidence_quote":"Supplies the covering-number bound (Assumption 1 and Example 1) that turns Lipschitz and variance reduction into a complexity reduction in Theorems 3 and 4."},{"cited_title":"MINDE : Mutual information neural diffusion estimation","cited_arxiv_id":null,"evidence_quote":"Supplies the KL-divergence identity (Eq. 8) connecting score-function errors to distribution distance, which motivates denoiser generalization error as the objective."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Rademacher complexity, the model-complexity measure appearing in the generalization bounds and their symmetrization proof."},{"cited_title":"Stargan v2: Diverse image synthesis for multiple domains","cited_arxiv_id":null,"evidence_quote":"Provides the AFHQv2-Cat dataset used for the experiments demonstrating bootstrapping diffusion."}],"review_version":1}