{"id":"9b06f5b2-cc32-40b5-be0b-8fe89d63d0ed","arxiv_id":"2412.00623","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A 3D Gaussian Splat diffusion model trained with only 2D image supervision, using deterministic reconstruction models as noisy teachers, improves single-image 3D reconstruction over those teachers.","lead":"A new training scheme uses a deterministic image-to-3D model as a noisy teacher to train a 3D diffusion model, so that only 2D images are needed as supervision. The resulting model produces sharper 3D Gaussian reconstructions than the teacher on object and scene datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Critical-timestep premise in §3.1 is unvalidated and quantitatively questionable: t*=20 leaves teacher bias dominant (α≈0.68), so the claimed alignment with true forward-noised 3D is not established.","rationale":"I read the paper as making two separable claims: a practical training recipe that improves deterministic teachers using 2D-only supervision, and a theoretical explanation of why that recipe works, centered on the existence of a critical timestep t* beyond which teacher-noised 3DGS aligns with forward-noised true 3DGS. The empirical claim is supported by consistent gains on ShapeNet-SRN cars and chairs and on RealEstate10K, and the ablations show the diffusion formulation and cycle consistency contribute. The least-secure link is the t* premise. It is not proven, t*=20 is fixed in the supplementary without sensitivity analysis, and a back-of-the-envelope calculation from the stated linear β schedule gives α_20 ≈ 0.68, meaning the teacher's structured errors are still prominent at the chosen threshold. If this premise fails, the paper's explanation for why the stage-2 distribution is usable is wrong, even if the loss still happens to work through the unrolled rendering objective. That would not overturn the empirical results, but it would change the claimed contribution from a principled decoupling of denoised and supervised modalities to an undocumented refinement heuristic. The reader's CONDITIONAL verdict already captures this uncertainty, so my read does not change the verdict; the conditional should be resolved by the proposed sensitivity and distributional-alignment checks.","tokens_in":16904,"tokens_out":11303,"duration_ms":115800,"concrete_test":"On the ShapeNet-SRN cars split, fit reference 3DGS to a random subset of ground-truth meshes using standard per-scene optimization, then generate Eq. 1 samples from both the teacher prediction and the reference 3DGS at t ∈ {10, 20, 30, 40, 60} and compare their rendered-image distributions with a two-sample metric such as KID or Wasserstein distance in splat-parameter space; if the gap at t*=20 is large or fails to shrink monotonically with t, the distribution-alignment premise is empirically false. Correlate this with a Stage-2 sensitivity sweep over the same t* values; if PSNR varies by more than about 0.3 dB, the choice of t* is load-bearing and the paper's theoretical justification must be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is the existence, asserted in §3.1 around Eq. 1, of a critical timestep t* such that for all t ≥ t*, st = √α_t s_teacher^0 + √(1−α_t)ε is statistically aligned with the forward-noised true 3DGS distribution. This premise is what justifies restricting Stage 2 (Eq. 4) and the inference-time denoising trajectory to starts with t > t*, and it is stated without proof. The supplementary sets t* = 20 with no sensitivity analysis. Under the stated schedule (100 steps, linear β from 0.0001 to 0.2, interpreted as β), α_20 = ∏_{i=1}^{20}(1−β_i) ≈ 0.68, so at t*=20 the teacher signal, including its systematic errors (blur, missing geometry), dominates the added Gaussian noise. There is no reason to expect a deterministic feedforward teacher's error field to be Gaussian, and if it is structured, the Stage-2 training distribution is biased rather than aligned. Because the claimed 'sweet spot' is the theoretical basis for decoupling modalities, the empirical gains may be due to the multi-step rendering fine-tuning compensating for the bias, not to the stated distribution-matching principle; in that case the paper's explanation and its transferability to new teachers and domains are not established. The results on three datasets are consistent and the ablations support the practical recipe, but this premise is the least-secure link in the central argument.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a training framework for image-conditioned 3D Gaussian Splat (3DGS) diffusion models that uses only 2D supervision. A pretrained deterministic image-to-3D model (Splatter Image or Flash3D) serves as a 'noisy teacher': in stage 1, the diffusion model is bootstrapped by denoising teacher predictions with direct 3D and rendering losses; in stage 2, the teacher is removed from the supervision loss and the model is fine-tuned via multi-step denoising with a rendering loss against target views, supported by a cycle-consistency loss. Experiments on ShapeNet-SRN cars/chairs, RealEstate10K, and Co3D hydrant report consistent PSNR/SSIM/LPIPS improvements over the respective teachers, with the diffusion model using a smaller U-Net than the teacher.","tokens_in":17306,"tokens_out":4025,"duration_ms":37843,"significance":"If the result holds, the paper offers a scalable alternative to full-3D supervision for 3D diffusion models, which is a timely and practically important problem. The decoupling of the denoised modality (3D) from the supervision modality (2D) is conceptually interesting and the empirical recipe is validated on three datasets with two different teachers. The ablations usefully isolate the contributions of bootstrapping, multi-step denoising, and cycle consistency, and the smaller-model/larger-performance result is a positive signal. However, the central theoretical premise is unproven and the evaluation lacks error bars, so the significance is tempered until these points are addressed.","major_comments":[{"comment":"The central premise that for all t >= t* the noised teacher predictions st = sqrt(alpha_t) s_teacher + sqrt(1-alpha_t) epsilon are statistically aligned with forward-noised true 3DGS is asserted without proof, and the paper does not report any empirical check of this alignment. Because the decoupling argument in Eqs. (3)-(4) and the restriction to t > t* rest on this premise, it is load-bearing. The supplementary sets t*=20 with no sensitivity analysis, and under the stated linear schedule (100 steps, beta from 0.0001 to 0.2) alpha_20 is approximately 0.68, so the teacher's systematic errors dominate the added Gaussian noise. I ask the authors to either (i) provide a direct validation of the distribution-matching assumption on a subset where 3D ground truth is available, (ii) perform a sensitivity analysis over t* and show that the reported gains are robust, or (iii) revise the theoretical claim to a weaker empirically motivated heuristic. Without one of these, the mechanism attributed to the improvements is not established.","section":"Sec. 3.1 (Eq. 1) and Supplementary Sec. 8"},{"comment":"The quantitative gains over the teacher are modest (0.84 and 0.78 dB PSNR on ShapeNet-SRN cars and chairs, 0.5 dB average on RealEstate10K), and all main tables report averages without error bars, confidence intervals, or a statement about the number of seeds. Given the size of the improvements, the 'consistently improves' claim is weakened unless the authors report variance across at least three training runs or an appropriate statistical comparison. This is especially important because Table 4 (a.2) shows a large drop for the Medium feedforward model, and the ablation rows do not indicate whether differences such as (b.3) versus (b.4) are significant.","section":"Table 4 / Sec. 4.5"},{"comment":"The multi-step denoising loss L_mlt-stp in Eq. (4) is presented as the core stage-2 objective, but the per-step weights lambda_t are never specified. The supplementary mentions 'different weights' and Table 9 compares weighted versus unweighted loss, but the actual schedule is missing. Without this, the stage-2 recipe is not reproducible. Please provide the exact lambda_t schedule (or its functional form) and the DDIM step details used in both training and inference.","section":"Eq. (4) and Supplementary Sec. 8"}],"minor_comments":[{"comment":"There is a typo: 'sprase 2D views' should be 'sparse 2D views'.","section":"Sec. 1"},{"comment":"The table has duplicate labels '(c.1)' for the two stage-I consistency rows; the second row should be labeled '(c.2)'.","section":"Table 4"},{"comment":"The method is referred to as both 'SplatDiffusion' (Tables 1, 2) and 'Ours' (Tables 3-5, text); please unify the naming.","section":"Throughout"},{"comment":"The related work does not mention recent large reconstruction models such as LRM or GS-LRM; a brief discussion would help position the deterministic teacher comparisons in the broader context.","section":"Sec. 2"},{"comment":"The guidance mechanism is described only briefly in the main text and detailed in the supplementary; consider including the guidance strength schedule s(t) in the main text, as it is a factor in reproducing the view-guidance results.","section":"Sec. 4.4"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a conceptually appealing recipe with consistent empirical gains, but the central distribution-matching premise in Sec. 3.1 is unproven and t* is chosen without sensitivity analysis. The reported improvements are small and lack error bars, and the lambda_t schedule is underspecified for reproducibility. These are fixable within the manuscript's scope, so I recommend major revision rather than rejection. The authors should also consider softening the 'fundamentally revises' claim in Sec. 1 unless the premise is validated more directly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid method paper with a plausible, genuinely new training recipe for 3D diffusion with 2D supervision, but the stated theoretical justification for the critical timestep t* is weaker than the paper pretends, and the authors should add sensitivity analysis and code before I'd fully trust the gains.\n\nWhat's new: decoupling the noisy 3D sample source from the 2D supervision signal, and using teacher-generated noisy samples plus multi-step unrolled denoising with a rendering loss. The two-stage design (bootstrapping with the teacher, then rendering-only fine-tuning) is sensible, and the ablations tell a coherent story. They report consistent PSNR gains of 0.5–0.85 over two different teachers on objects and scenes, and the improvement holds even with a smaller backbone, which is a nice control.\n\nThe soft spot is the claim in Sec. 3.1 that there exists a t* beyond which the teacher's noisy samples align with the true forward-noised 3DGS distribution. That is asserted, not proven. The supplement sets t*=20 with no sensitivity analysis. Given their beta schedule (linear 0.0001 to 0.2 over 100 steps), alpha_20 is about 0.67, so the teacher signal—including its systematic blur and missing geometry—dominates the added Gaussian noise. Real teacher errors are not Gaussian, so the \"sweet spot\" story is quantitatively questionable. The more honest interpretation is that the multi-step rendering fine-tuning is what compensates for the bias; the method works as a recipe, but the stated principle may not be the actual driver of the gains. That is a correctable presentation issue, not a fatal flaw, because the empirical results are consistent and the ablations support the design choices.\n\nMinor: no error bars in the main tables, no code release, and the introduction's \"fundamentally revises the principles\" is overclaiming. The comparison set is adequate but could be broadened to include more recent 2D-supervised 3D diffusion work.\n\nVerdict: worth a serious referee. I'd ask for sensitivity analysis on t*, error bars, and code before acceptance. The core idea is useful to the 3D generation community and likely to hold up.","headline":"A genuinely useful training recipe for 3D diffusion with 2D supervision, but the stated critical-timestep justification is shakier than the paper admits.","tokens_in":17813,"tokens_out":2277,"would_cite":true,"duration_ms":22734,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims a 3D Gaussian Splat diffusion model can be trained from ordinary 2D images, using a deterministic reconstruction model as a \"noisy teacher\", and that this model consistently outperforms that teacher.","keywords":["3D Gaussian Splats","diffusion models","2D supervision","single-image 3D reconstruction","novel view synthesis","teacher-guided diffusion","sparse-view supervision","image-conditioned generation"],"falsifier":"On a dataset with ground-truth 3D Splats, compute a two-sample distance between the teacher-noised distribution $\\sqrt{\\alpha_t}\\,\\mathbf{s}^{\\mathrm{teacher}}_0 + \\sqrt{1-\\alpha_t}\\,\\epsilon$ and the true-noised distribution $\\sqrt{\\alpha_t}\\,\\mathbf{s}^*_0 + \\sqrt{1-\\alpha_t}\\,\\epsilon$ across the full timestep range; if no $t \\ge t^\\ast$ interval shows near-overlap, or the crossover is far from the chosen $t^\\ast=20$, the second-stage training distribution is biased. A direct behavioral check is whether the PSNR gain persists on a class where teacher failures are structured, such as consistently missing occluded surfaces.","tokens_in":16733,"feed_emoji":"🧊","tokens_out":11427,"duration_ms":95444,"temperature":0.7,"pith_summary":"Recovering a 3D scene from one image is ambiguous, and deterministic reconstruction models trained with 2D images tend to blur uncertain regions. The paper's claim is that a diffusion model can do better while still training on 2D images only, with no 3D ground truth. The mechanism is a 'noisy teacher' — a deterministic single-image reconstruction model — whose imperfect 3D Gaussian Splat predictions (a point-cloud-like 3D representation rendered to images) are fed into the diffusion denoiser as noisy inputs, while supervision comes from rendering the denoised splats to reference views and comparing with real 2D images. A critical noise level makes the teacher's predictions, once noised, stand in for true 3D data; multi-step denoising then lets the model learn fine detail below that level using 2D losses alone. If right, the framework turns any deterministic reconstruction model into a generative one and delivers consistent quality gains on object and scene benchmarks, including PSNR increases of 0.84 and 0.78 over Splatter Image on ShapeNet-SRN cars and chairs and about 0.5 over Flash3D on RealEstate10K.","feed_headline":"2D photos alone train a 3D diffusion model that beats its teacher","feed_subtitle":"A teacher-guided scheme lifts single-image 3D reconstruction by 0.5–0.84 PSNR with no 3D ground truth.","key_machinery":"The load-bearing mechanism is the decoupling of the source of noised samples from the source of supervision. A pretrained deterministic model ('noisy teacher') supplies an imperfect splat prediction $\\mathbf{s}^{\\mathrm{teacher}}_0$, which is pushed to high-noise timesteps by $\\mathbf{s}_t = \\sqrt{\\alpha_t}\\,\\mathbf{s}^{\\mathrm{teacher}}_0 + \\sqrt{1-\\alpha_t}\\,\\epsilon$ for $t \\ge t^\\ast$, a regime where teacher-induced noise is assumed to overlap the forward-noised true distribution. The denoiser $D_\\theta$ is then run in a multi-step, unrolled (DDIM) loop down to the clean estimate $\\hat{\\mathbf{s}}_0$, and the differentiable renderer $R(\\cdot, v)$ turns $\\hat{\\mathbf{s}}_0$ into images supervised by target views; gradients flow back through all denoising steps. A cycle-consistency term renders $\\hat{\\mathbf{s}}_0$ to a target view, feeds that rendered image back through the reconstruction pipeline, and compares the resulting source-view rendering with the original input image $\\mathbf{x}_{\\mathrm{src}}$.","core_discovery":"The central claim is that the standard diffusion requirement of same-modality supervision can be broken: the denoiser operates on 3D Gaussian Splats while the supervision is ordinary 2D images. The paper formalizes this with a pretrained deterministic image-to-3DGS model used as a 'noisy teacher.' Its imperfect prediction $\\mathbf{s}^{\\mathrm{teacher}}_0$ is noised to timesteps $t \\geq t^\\ast$, where, the paper argues, the teacher-induced noised samples line up with what forward noising of the true 3D structure would produce. Rather than regress to the teacher, the denoiser is trained through multi-step (DDIM) unrolling so the final clean splats $\\hat{\\mathbf{s}}_0$ can be rendered to target views and supervised by image losses, plus a cycle-consistency term that renders the predicted target back to the source view and compares with the input image. The paper reports that, despite a smaller network, this model improves PSNR by 0.84 and 0.78 over Splatter Image on ShapeNet-SRN cars and chairs, by about 0.5 over Flash3D on RealEstate10K, and can consume a second view as guidance for further gains.","pith_inferences":["If the noise-overlap premise holds, the same decoupling should transfer to other 3D representations (meshes, radiance fields, point clouds) and any differentiable renderer; the paper states this flexibility but does not test it.","The reported metrics measure reconstruction fidelity, not diversity; a natural extension is to check whether the model samples multiple distinct plausible 3D interpretations of a single image.","The fixed critical timestep $t^\\ast = 20$ is set without a published sensitivity study, so sweeping it on one dataset would reveal whether the gains are robust or require per-dataset tuning.","Since stage 1 bootstrapping is bounded by the teacher, a better teacher should raise the floor for the final model; this suggests an easy scaling path as deterministic reconstruction models improve."],"forward_implications":["Ground-truth 3D data stops being a bottleneck: 3D diffusion models can be trained from large collections of posed 2D images, matching the data that deterministic reconstruction models already use.","Any current deterministic feed-forward reconstruction model can be upgraded into a generative one by wrapping it as the noisy teacher, since the paper demonstrates the recipe with two different teachers on object and scene data.","The model retains diffusion-style guidance at inference: an additional view can steer the denoising process and improves two-view reconstruction over plain 3DGS optimization (25.55 vs 25.24 PSNR on the car subset).","Because the final stage is supervised only by rendering losses, the model's Gaussian splats become denser and more regularly shaped, which removes part of the blurriness and artifacts seen in deterministic teacher outputs."],"supporting_citations":[{"why":"supplies the object-level deterministic teacher, Splatter Image, whose imperfect 3DGS predictions become the noisy training samples.","marker":"[55]"},{"why":"supplies the scene-level deterministic teacher, Flash3D, used for RealEstate10K experiments.","marker":"[54]"},{"why":"provides the SDEdit observation that enough noise makes different data distributions overlap, motivating the critical timestep t*.","marker":"[36]"},{"why":"is the closest prior approach for 3D diffusion trained with 2D supervision; the paper contrasts its modality-handling strategy with Holodiffusion's.","marker":"[21]"},{"why":"defines the standard same-modality diffusion training objective that the paper explicitly revises.","marker":"[17]"},{"why":"provides the differentiable 3D Gaussian Splat rendering used for the 2D losses and the optimization baseline in the guidance comparison.","marker":"[23]"},{"why":"supplies the universal guidance procedure used to inject an additional view during denoising at inference.","marker":"[3]"}],"fun_headline_variants":["Teacher-guided diffusion beats deterministic 3D models with 2D-only training","2D photos alone train a 3D diffusion model that improves on its teacher","No 3D ground truth? Teacher's noisy splats still yield better 3D","Diffusion on 3D splats with 2D supervision: up to 0.84 PSNR gain","Decoupling denoising and supervision lets 2D images teach 3D"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result stands on the assumption that above some critical noise level, adding Gaussian noise to the teacher's imperfect 3D predictions makes those noisy samples statistically indistinguishable from noised true 3D structure; if the teacher's errors are structured rather than random, the training distribution is biased and the reported gains may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Teacher-guided diffusion beats deterministic 3D models with 2D-only training","2D photos alone train a 3D diffusion model that improves on its teacher","No 3D ground truth? Teacher's noisy splats still yield better 3D","Diffusion on 3D splats with 2D supervision: up to 0.84 PSNR gain","Decoupling denoising and supervision lets 2D images teach 3D"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000318,"raw_usage":{"total_tokens":1893,"prompt_tokens":1135,"completion_tokens":758,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":751,"completion_tokens_details":{"reasoning_tokens":643}},"tokens_in":751,"tokens_out":758,"duration_ms":7487,"temperature":1.0,"reasoning_tokens":643,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:09:13.212088+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a dataset with ground-truth 3D Splats, compute a two-sample distance between the teacher-noised distribution $\\sqrt{\\alpha_t}\\,\\mathbf{s}^{\\mathrm{teacher}}_0 + \\sqrt{1-\\alpha_t}\\,\\epsilon$ and the true-noised distribution $\\sqrt{\\alpha_t}\\,\\mathbf{s}^*_0 + \\sqrt{1-\\alpha_t}\\,\\epsilon$ across the full timestep range; if no $t \\ge t^\\ast$ interval shows near-overlap, or the crossover is far from the chosen $t^\\ast=20$, the second-stage training distribution is biased. A direct behavioral check is whether the PSNR gain persists on a class where teacher failures are structured, such as consistently missing occluded surfaces.","supporting_citations":[{"cited_title":"Splatter image: Ultra-fast single-view 3d reconstruc- tion","cited_arxiv_id":null,"evidence_quote":"supplies the object-level deterministic teacher, Splatter Image, whose imperfect 3DGS predictions become the noisy training samples."},{"cited_title":"Holodiffusion: Training a 3d diffusion model using 2d images","cited_arxiv_id":null,"evidence_quote":"is the closest prior approach for 3D diffusion trained with 2D supervision; the paper contrasts its modality-handling strategy with Holodiffusion's."},{"cited_title":"3d gaussian splatting for real-time radiance field rendering","cited_arxiv_id":null,"evidence_quote":"provides the differentiable 3D Gaussian Splat rendering used for the 2D losses and the optimization baseline in the guidance comparison."},{"cited_title":"Universal guidance for diffusion models","cited_arxiv_id":null,"evidence_quote":"supplies the universal guidance procedure used to inject an additional view during denoising at inference."}],"review_version":1}