{"id":"d5319c97-d91c-4169-9dd1-effce6027e7c","arxiv_id":"2501.05427","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A single image can be turned into a 3D Gaussian splat model by fine-tuning a pretrained 2D diffusion model to output decomposed multi-view splatter attribute images.","lead":"This paper presents a method to generate a 3D object from a single photo by reusing a pretrained 2D image diffusion model. The key trick is representing the 3D object as a set of 2D attribute images, so the model inherits strong 2D priors and can generalize to unseen objects.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Diffusion model is trained on splatters from a fine-tuned LGM reconstruction network, not on true Gaussian splats; the paper never quantifies LGM's fidelity, so reported GSO gains may reflect label bias.","rationale":"The reader's weakest_assumption exactly identifies the LGM-label issue, which I consider the single most load-bearing concern. The central claim is empirical: Zero-1-to-G outperforms baselines on GSO. That outcome is produced by a pipeline whose training targets are outputs of a fine-tuned reconstruction network. If that network is biased, the diffusion model can only learn biased 3D, and the reported metrics are not evidence that the method effectively leverages 2D priors for direct 3D generation. The paper gives no quantitative assessment of LGM's fidelity, making the assumption untested. Other weaknesses (small evaluation set, no error bars, no code) are secondary because they affect confidence but not the internal logic; the label issue could invalidate the headline result regardless of evaluation size or reproducibility. My proposed test directly measures the fidelity of the label generator, which would settle whether the assumption holds. Since the reader already conditioned acceptance on this and other verification needs, my recommendation remains UNCHANGED: conditional acceptance pending evidence that LGM reconstructions are faithful enough to serve as training targets.","tokens_in":13494,"tokens_out":9117,"duration_ms":87980,"concrete_test":"On the GSO evaluation set, run the fine-tuned LGM (as described in Sec. 3.1) using the same 6-view renderings used for training-label generation, fuse the output splatters, and compute PSNR, SSIM, LPIPS, and Chamfer Distance against ground truth. If these LGM-only metrics are poor relative to the numbers reported in Table 1, the training labels are a biased proxy and the diffusion model's performance is capped by that bias, undermining the central claim. A complementary check: retrain the diffusion model with per-object fitting-based splatters (with appropriate VAE decoder fine-tuning) and compare GSO metrics; if results change substantially, the choice of label generator is decisive.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of superior performance on GSO (Table 1) depends on the training labels being faithful 3D targets. In Sec. 3.1, the authors generate ground-truth splatters by fine-tuning LGM to output splatters from the G-Objaverse renderings, explicitly rejecting per-object fitting because of high-frequency artifacts. This means the diffusion model learns the distribution of LGM's reconstructions, not the true distribution of Gaussian splats. The paper's defense — 'we train and infer the network with the same set of data, this ensures the model fits its parameters on the training set and does not have potential generalizability issues' — does not address bias. LGM is fine-tuned on 6-view renderings, while at inference Zero-1-to-G sees a single image, so the 'same set of data' claim is also ambiguous. Crucially, the paper provides no quantitative evaluation of LGM's reconstruction quality against ground truth. Figure 3 only compares LGM to fitting-based splatters qualitatively. If LGM over-smooths geometry, hallucinates unseen regions, or loses fine detail, the diffusion model inherits these errors. The reported gains over baselines (e.g., PSNR 19.40 vs. 18.15 for InstantMesh) could then be an artifact of the label generator's biases rather than evidence that pretrained 2D diffusion priors are being effectively harnessed for direct 3D generation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Zero-1-to-G, a diffusion-based model that generates Gaussian splats directly from a single RGB image. The method decomposes each object into multi-view 'splatter images' composed of five 3-channel attribute maps (RGB color, position, scale, rotation, opacity) and fine-tunes a Stable Diffusion Image Variations model with additional cross-view and cross-attribute attention layers to generate these attribute maps jointly. A VAE decoder is fine-tuned with rendering losses to improve splat reconstruction. Training targets are not produced by per-object fitting but by a fine-tuned LGM reconstruction network applied to G-Objaverse renderings. Experiments on 30 GSO objects report improved PSNR, SSIM, LPIPS, and Chamfer Distance over several baselines, with additional qualitative results on in-the-wild images.","tokens_in":13802,"tokens_out":7025,"duration_ms":65557,"significance":"If the results hold, this is a valuable step: it shows that a pretrained 2D diffusion model can be adapted to generate a 3D representation directly, avoiding per-scene optimization and reducing training cost relative to from-scratch direct 3D diffusion. The decomposition of Gaussian splats into attribute images is a useful insight, and the ablations (w/o decoder fine-tuning, w/o cross-attribute attention, w/o diffusion prior) demonstrate that each component contributes. The paper's main claims, however, rest on the fidelity of the learned labels and on a small, single-subset evaluation; both need strengthening before the quantitative gains can be fully trusted.","major_comments":[{"comment":"The training labels are not true Gaussian splats but outputs of a fine-tuned LGM reconstruction network. This is a load-bearing premise: the diffusion model is trained to reproduce the distribution of LGM reconstructions, so any systematic bias in LGM (e.g., over-smoothing, missing high-frequency detail, hallucinated geometry) will be inherited. The argument in the text that 'we train and infer the network with the same set of data' addresses generalization but not bias. Please add a quantitative validation of the fine-tuned LGM splatters against independent ground truth (e.g., fitting-based splatters or held-out multi-view renderings), reporting metrics such as PSNR, SSIM, LPIPS, and Chamfer Distance, and discuss how the label generator's errors propagate to the diffusion model. Without this, the gains in Table 1 could be attributable to characteristics of the label generator rather than to the proposed use of 2D diffusion priors.","section":"Sec. 3.1"},{"comment":"The quantitative comparison is based on 30 GSO objects with no error bars, confidence intervals, or per-object breakdown. The improvements over the closest baseline (InstantMesh) are modest (e.g., PSNR 19.40 vs 18.15; SSIM 0.818 vs 0.810), and without variance estimates it is unclear whether these differences are significant. Please report mean and standard deviation over multiple random subsets of GSO and over multiple generative sampling runs (seeds). Also clarify how Chamfer Distance is computed: whether GT meshes or scans are used, and how sampling density and alignment are handled.","section":"Sec. 4.2 / Table 1"},{"comment":"As written, the reverse process and loss are unconditional: the noise predictor is written as epsilon_theta(z_t, t) and the expectation in Eq. (6) does not include the conditional image y. Since the method is explicitly a conditional image-to-3D model, this notation obscures the central conditioning mechanism. Please rewrite the equations to include the conditioning image y in the noise predictor and in the expectation, and state how y is injected into the UNet (e.g., cross-attention, concatenation). This is necessary for reproducibility.","section":"Sec. 3.2, Eqs. (5)-(6)"},{"comment":"The ablation study does not isolate the contribution of the multi-view attention layers. The reader cannot tell whether the 3D consistency of the outputs comes from the cross-view attention or from the shared 2D diffusion prior and the decomposition itself. Please add an ablation that removes the cross-view attention modules (while keeping cross-attribute attention) and report the corresponding metrics.","section":"Sec. 4.4 / Table 3"}],"minor_comments":[{"comment":"The paper uses both z and x for the same data (e.g., z for splatters, but x in the expectation of Eq. (6)); please unify the notation and define all variables.","section":"Equations (2)-(6)"},{"comment":"The header 'Abalation' should be 'Ablation'.","section":"Table 3"},{"comment":"There are odd spacing artifacts in 'ZERO-1- TO-G' and 'T AMING'; please correct these typos.","section":"Title and running header"},{"comment":"In the related-work paragraph, 'MVDream (Shi et al., 2023b) aintegrates' contains a typo; it should read 'integrates'.","section":"Sec. 2"},{"comment":"The decoder fine-tuning loss L_decoder = L_splatter + L_normal + L_rgb + L_mask is described verbally, but L_splatter and L_mask are not defined with equations; please provide explicit definitions for these terms.","section":"Sec. 3.3"},{"comment":"The sentence 'The second stage of training takes about 2 days' appears twice in the same paragraph; please remove the duplication.","section":"Sec. 4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper's claim to be the 'first direct image-to-3D generative model to effectively utilize pretrained 2D diffusion priors' should be compared more carefully with the concurrent works Omage (Yan et al., 2024) and GIMDiffusion (Elizarov et al., 2024), which the authors cite but do not quantitatively compare. The small evaluation set (30 GSO objects, no error bars) may be insufficient for the claims of state-of-the-art performance; the editor may wish to request a larger evaluation or additional independent validation of the label-generation pipeline."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know that Zero-1-to-G is a genuinely interesting attempt to make direct 3D generation cheaper by translating Gaussian splats into 2D attribute images and fine-tuning Stable Diffusion. The decomposition into five RGB-format channels (position, scale, rotation, opacity, RGB) and the use of cross-view plus cross-attribute attention is a sensible combination, and the paper shows it trains in about 3 days on 8 L40s—an order of magnitude cheaper than comparable direct 3D methods. The ablations support each component's necessity: without cross-attribute attention, floaters appear; without decoder fine-tuning, renderings degrade.\n\nThe main thing to push on is the training-label proxy. Section 3.1 says they fine-tune LGM to produce the 'ground truth' splatters, explicitly rejecting per-object fitting because of high-frequency artifacts. So the diffusion model learns the distribution of LGM's reconstructions, not the true distribution of Gaussian splats. The authors argue this is fine because they train and infer on the same data, but that only addresses covariate shift, not bias. If LGM over-smooths geometry or hallucinates unseen regions, Zero-1-to-G inherits that. The paper never quantifies LGM's fidelity against actual ground truth; Figure 3 is only qualitative. This is a real gap, and the reported GSO gains (19.40 vs 18.15 PSNR over InstantMesh) could partly be artifacts of label bias. That said, the final evaluation is against real renderings, and the method does beat strong baselines on the same metric, so the concern is moderate, not fatal.\n\nThe evaluation is thin: 30 GSO objects, no error bars, and the metric differences are small. The in-the-wild comparisons are qualitative. No code or checkpoints are released, so independent verification is impossible right now. And the 'first' claim in the abstract is too confident given concurrent UV-atlas works, even though the representation is different.\n\nOverall, the core idea is useful and likely to be built upon. It deserves serious peer review, but the authors should be asked to (a) quantify the LGM proxy's fidelity, (b) report variance/error bars and ideally evaluate on more objects, and (c) release code or at least a detailed reproducibility appendix. If the proxy issue is addressed, this could be a solid contribution.\n\nRecommendation: send it to review. I would not desk reject.","headline":"A promising direction—decomposing Gaussian splats into 2D attribute images to reuse pretrained 2D diffusion—but the training-label proxy needs quantification before the reported gains are convincing.","tokens_in":14301,"tokens_out":2656,"would_cite":true,"duration_ms":23897,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Zero-1-to-G generates a full 3D Gaussian-splat model directly from one image, by decomposing splats into multi-view attribute images and fine-tuning a pretrained 2D diffusion model to draw them.","keywords":["direct 3D generation","Gaussian splatting","diffusion models","single-image to 3D","multi-view diffusion","cross-view attention","splatter image","latent diffusion"],"falsifier":"Train Zero-1-to-G twice on the same data--once with LGM-produced splatter labels and once with per-scene fitted splatters--and compare the resulting models on the GSO rendering and geometry metrics; if the LGM-supervised model fails to match the fitted-splatter model, the central claim that the method learns the true 3D distribution is refuted, since its quality ceiling is set by the proxy labels.","tokens_in":13320,"feed_emoji":"🖼️","tokens_out":10195,"duration_ms":81099,"temperature":0.7,"pith_summary":"The paper claims that direct single-image-to-3D generation can be recast as a 2D image diffusion problem without losing 3D consistency. A Gaussian splat, whose per-pixel attributes normally fill 14 channels, is decomposed into five 3-channel attribute images (color, position, scale, rotation, opacity) across six camera views, and a pretrained Stable Diffusion UNet is fine-tuned to generate these splatter images jointly. The result is an image-to-3D generator that trains in about three days on eight GPUs, runs in 8.7 seconds per object at inference, and reports higher PSNR and SSIM, lower LPIPS, and lower Chamfer distance than the reconstruction-based, two-stage, and direct-3D baselines compared on the GSO benchmark. If correct, it shows that the rich priors of 2D diffusion models can be transferred to direct 3D generation, improving generalization to in-the-wild images without training a 3D diffusion model from scratch.","feed_headline":"A single photo becomes a 3D Gaussian model in 8.7 seconds","feed_subtitle":"The trick: cut a 3D Gaussian splat into 2D attribute images, then let Stable Diffusion draw them.","key_machinery":"The load-bearing object is the splatter image: a regular $H\\times W$ grid in which each pixel encodes one 3D Gaussian, normally with 14 channels of attributes. The paper's transformation maps RGB, position, log-compressed scale, quaternion-to-axis-angle rotation, and duplicated opacity into five ordinary 3-channel RGB images, making each attribute compatible with the latent space of a pretrained VAE. The consistency argument is carried by two extra self-attention modules inserted into the Stable Diffusion UNet--multi-view attention over the same attribute across views and multi-attribute attention over different attributes within one view--so the denoiser processes the whole view-attribute array. A decoder fine-tuning stage with rendering losses (splatter reconstruction plus MSE and LPIPS on RGB renderings, normals, and masks) closes the domain gap between natural images and splatter images, because a small pixel error in splatter space can become a large rendering artifact when each pixel is a Gaussian.","core_discovery":"The core claim is that a Gaussian splat can be rearranged into a set of multi-view splatter images--one per camera view and attribute--so the joint distribution of a full 3D object becomes a joint distribution over 6 views and 5 attribute images. By inserting cross-view and cross-attribute self-attention layers into a pretrained 2D diffusion UNet and fine-tuning them together with the VAE decoder, the model learns to denoise a coherent array of attribute images that fuse directly into a Gaussian-splat representation. This makes the generation single-stage: there is no separate multi-view prediction followed by reconstruction, and the paper reports that this arrangement outperforms the tested baselines on GSO's rendering and geometry metrics while generalizing to in-the-wild photographs, including inputs with backgrounds.","pith_inferences":["A testable extension the paper leaves implicit is whether the same decomposition works for other per-pixel 3D representations, such as triplanes or multi-plane images; success would suggest the key ingredient is the 2D-compatible attribute layout rather than Gaussian splats specifically.","Because the training labels are splatters produced by a fine-tuned reconstruction network rather than per-scene fitted splatters, the method's quality ceiling is set by that proxy; a benchmark comparing the two label sources would quantify how much this assumption costs.","The fixed six-view, fixed-elevation design could be relaxed to variable numbers of views or free camera poses, which would test whether the cross-view attention architecture generalizes beyond the current grid and improves coverage on complex objects."],"forward_implications":["Because generation is single-stage, the strict pixel-level consistency required by two-stage multi-view-to-reconstruction pipelines is not needed; the paper attributes fewer floaters and sharper renderings to this property.","The method trains in about three days on eight GPUs, far less than the compared direct-3D and two-stage systems, because it starts from a pretrained 2D diffusion UNet rather than training a 3D diffusion model from scratch.","Inference takes about 8.7 seconds per object on a single GPU, and because the denoiser is generative, the same input image can yield multiple plausible 3D interpretations rather than a single regression output.","Because all 3D information is carried by 2D attribute images, future advances in 2D diffusion--better decoders, faster samplers, distillation--could be adopted directly into direct 3D generation without changing the 3D representation."],"supporting_citations":[{"why":"Supplies the splatter image representation that turns a Gaussian splat into a regular 2D grid, which is the basis of the five-attribute decomposition.","marker":"(Szymanowicz et al., 2023)"},{"why":"Provides the pretrained latent diffusion model and VAE whose image priors are reused and fine-tuned.","marker":"(Rombach et al., 2022)"},{"why":"Its reconstruction module is fine-tuned to produce the ground-truth splatter images used as training labels for the diffusion model.","marker":"(Tang et al., 2024)"},{"why":"Contributes the cross-domain attention and attribute-switcher design, as well as the two-stage training schedule that Zero-1-to-G follows.","marker":"(Long et al., 2023)"},{"why":"Introduces cross-view attention over the batch dimension, which the paper adapts to enforce multi-view consistency.","marker":"(Shi et al., 2023b)"},{"why":"Defines the DDPM forward and reverse processes and the noise-prediction loss used to train the denoiser.","marker":"(Ho et al., 2020)"},{"why":"Supplies the G-buffer Objaverse training set with multi-view renderings from which the splatter ground truths are generated.","marker":"(Qiu et al., 2024)"},{"why":"Provides the Google Scanned Objects benchmark used for the reported quantitative comparisons.","marker":"(Downs et al., 2022)"}],"fun_headline_variants":["One photo, one step: direct 3D Gaussian generation via 2D diffusion","Slice a 3D splat into 2D, let Stable Diffusion draw it back","Zero-1-to-G: no multi-view, just direct image-to-3D generation","From a single shot to a 3D Gaussian splat in seconds","Pretrained 2D diffusion directly outputs 3D Gaussian splats"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The diffusion model is trained to reproduce splatters produced by a fine-tuned reconstruction network, not the true per-object fitted Gaussian splats; if those proxy splatters are biased, over-smoothed, or missing detail, the generated 3D objects cannot be better than that proxy.","fun_headline_variants_meta":{"raw":{"variants":["One photo, one step: direct 3D Gaussian generation via 2D diffusion","Slice a 3D splat into 2D, let Stable Diffusion draw it back","Zero-1-to-G: no multi-view, just direct image-to-3D generation","From a single shot to a 3D Gaussian splat in seconds","Pretrained 2D diffusion directly outputs 3D Gaussian splats"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000312,"raw_usage":{"total_tokens":1775,"prompt_tokens":948,"completion_tokens":827,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":720}},"tokens_in":564,"tokens_out":827,"duration_ms":7689,"temperature":1.0,"reasoning_tokens":720,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:13:18.771228+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train Zero-1-to-G twice on the same data--once with LGM-produced splatter labels and once with per-scene fitted splatters--and compare the resulting models on the GSO rendering and geometry metrics; if the LGM-supervised model fails to match the fitted-splatter model, the central claim that the method learns the true 3D distribution is refuted, since its quality ceiling is set by the proxy labels.","supporting_citations":[],"review_version":1}