{"id":"b42897f8-1e0c-441c-a80a-a0258bc79baa","arxiv_id":"2411.16779","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A novel-view denoising diffusion model over pixel-aligned 3D Gaussians achieves state-of-the-art PSNR and SSIM on GSO and OmniObject3D sparse-view reconstruction.","lead":"NovelGS builds a 3D object from a few photos by using a diffusion model to denoise a target view while predicting 3D Gaussians. It reports higher PSNR and SSIM than four feed-forward baselines on two public benchmarks, while being slightly worse on perceptual similarity (LPIPS) than one of them.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation protocol may drive the reported SOTA gap: the noisy-view pose is selected on the same benchmark views and may itself be scored.","rationale":"The reader's weakest_assumption was the expressive power of pixel-aligned Gaussians. That is a legitimate secondary concern, but the more immediate threat to the paper's central claim is evaluation fairness: the paper's own ablations select the noisy-view position on the same datasets used for the headline comparisons and do not disclose whether the selected view is scored. I agree with the reader's overall CONDITIONAL verdict, since both issues are addressable with additional experiments and reporting. I would not move to REJECT: the method is clearly described, the denoising loop is novel, and the ablations support the importance of the noisy view. The concrete re-evaluation would settle whether the reported SOTA is real or a protocol artifact.","tokens_in":15268,"tokens_out":7241,"duration_ms":75982,"concrete_test":"Ask the authors to specify the noisy-view index used for Tables 1-2 and to release code and checkpoints. Then re-run the GSO and Omni3D evaluations under a blinded protocol: (1) split each evaluation set into validation and test; (2) choose the noisy-view position, or randomize it per object, on validation only; (3) compute metrics on all 21 views and also on the 20 views excluding the noisy view. If the NovelGS-minus-LaRa PSNR gap collapses below the reported 2.4 dB, or the method no longer ranks first when the noisy view is excluded, the SOTA claim is not supported by the current protocol.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is the PSNR/SSIM advantage in Tables 1-2 (31.303 vs 28.910 on GSO). The load-bearing condition is that this comparison is apples-to-apples, and the text does not establish that. Section 4.1 evaluates on 21 rendered orbit views; Section 4.2 says the default input is 4 clean views and 1 noisy view, but it never states which of the 21 views the noisy view is, or whether that view is excluded from the reported metrics. Section 4.3 then searches over noisy-view positions (Tables 7-8) and reports that position 15 gives the best score, using the same GSO and Omni3D evaluation sets; no validation/test split is described. If the noisy view is an evaluation view, NovelGS is scored on a view it is explicitly conditioned to denoise, while LGM, LaRa, GeoLRM, and InstantMesh are not given a denoising target at that pose. If the position was chosen after seeing the evaluation set, the headline numbers are selected rather than predictive. This is the most load-bearing weakness because it attacks the central quantitative claim directly; the representation expressiveness question is secondary until the protocol is controlled.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"NovelGS proposes a diffusion-based reconstruction model that, given several clean input views plus one or more noisy target views, iteratively denoises those target views through a transformer that predicts pixel-aligned 3D Gaussians; the Gaussians are rendered and supervised with image, mask, and LPIPS losses. The method is trained on a filtered subset of Objaverse and evaluated on GSO and Omni3D, reporting large PSNR/SSIM gains over feed-forward baselines such as LaRa, LGM, GeoLRM, and InstantMesh, with additional ablations on the number and placement of noisy views and on integration with text-to-3D and image-to-3D pipelines.","tokens_in":15486,"tokens_out":3360,"duration_ms":36404,"significance":"If the reported numbers withstand scrutiny, the paper offers a conceptually clean combination of diffusion-based novel-view denoising with pixel-aligned Gaussian reconstruction, and the ablations support the claim that the noisy-view supervision is the key ingredient: removing it costs about 1.3 PSNR on GSO and 2.0 PSNR on Omni3D (Tables 3-4). The model also scales gracefully with more input views (Supplementary Table 11). These are nontrivial empirical gains on public benchmarks, and the release of code is promised. However, the central quantitative claim currently rests on an incompletely specified evaluation protocol, and one data table contains an apparent copy-paste error, so the significance is conditional on those issues being resolved.","major_comments":[{"comment":"The evaluation protocol is not apples-to-apples as reported. Section 4.1 states that each GSO/Omni3D object is rendered into 21 orbit views, Section 4.2 uses 4 clean views and 1 noisy view as default, but the paper never states which of the 21 views is the noisy one, nor whether that view is excluded from the reported PSNR/SSIM/LPIPS metrics. If the noisy view is among the 21 evaluated views, then NovelGS is scored on a view it is explicitly conditioned to denoise, while LGM, LaRa, GeoLRM, and InstantMesh are not given a denoising target at that pose. The position study in Section 4.3 (Tables 7-8) additionally searches over noisy-view positions on the same GSO and Omni3D evaluation sets without any validation/test split, so the best position (index 15) may be selected after seeing the test set. Please specify the default noisy-view index, exclude that view from the reported metrics for all methods, and either fix the noisy-view position a priori or report results averaged over positions with a validation-based selection.","section":"§4.1-4.3, Tables 1-2, 7-8"},{"comment":"The row for 4 clean views and 1 noisy view in Table 6 repeats the GSO values 31.303 PSNR, 0.946 SSIM, 0.065 LPIPS instead of the Omni3D main results from Table 2 (31.195, 0.945, 0.067). This is a copy-paste error that affects the comparison in the 'Number of noisy and clean views' ablation; it must be corrected and all downstream claims re-checked against the corrected table.","section":"Table 6 (Omni3D)"},{"comment":"The text claims that 'the LPIPS of NovelGS is marginally lower than that of the top-performing baseline,' but the tables show the opposite: InstantMesh has the best LPIPS on both datasets (0.059 on GSO, 0.053 on Omni3D), while NovelGS achieves 0.065 and 0.067, respectively. The claim should be corrected to state that NovelGS has slightly worse LPIPS than the best baseline, and the perceptual tradeoff (better PSNR/SSIM, slightly worse LPIPS) should be discussed honestly in relation to the 'dreaming' behavior of the denoiser.","section":"§4.2, Tables 1-2"},{"comment":"The pixel-aligned Gaussian parameterization restricts each Gaussian's scale to [0.005, 0.02] and its center depth to [0.1, 4.5] via a sigmoid. Since the paper's central claim is that novel-view denoising reconstructs unseen geometry and thin structures, the paper should demonstrate that this parameterization is expressive enough for the geometries it claims to recover. At minimum, provide an analysis or an ablation on scale/depth bounds, or temper the claim that unseen geometry is 'reconstructed' rather than hallucinated within a constrained depth slab.","section":"§8, Eqs. (5)-(8)"}],"minor_comments":[{"comment":"The heading 'Quantitative results' appears twice in Section 4.2; the second occurrence should be 'Qualitative results.'","section":"§4.2"},{"comment":"There are several typos: 'asserts' should be 'assets', 'limation' should be 'limitation', 'Visulation' should be 'Visualization', and 'contacted' should be 'concatenated'.","section":"Abstract and §4.3"},{"comment":"The spacing and glyph for the Plücker ray embedding are inconsistent ('Pl ¨ucker' appears with a stray space); please use a consistent 'Plücker' spelling in the text and equations.","section":"§3.1, Eq. (1)"},{"comment":"For Omni3D benchmark views, the text says results 'demonstrate the superiority' of NovelGS, but SSIM (0.889) is only marginally above InstantMesh (0.888) and LPIPS is tied (0.136); please characterize these results more precisely.","section":"Supplementary Table 10"},{"comment":"The loss in Eq. (2) omits the LPIPS weight λ from the displayed formula, and Eq. (3) is written as if λ multiplies the LPIPS term; please make the notation consistent, e.g., L_img = ||I - I_hat||^2 + λ LPIPS.","section":"§3.3"}],"recommendation":"major_revision","confidential_remarks":"The protocol ambiguity in Section 4.1-4.3 and the copy-paste error in Table 6 suggest the paper was not fully checked before submission. The core idea is sound and the ablations are informative, but the editor should require the authors to specify the exact evaluation protocol, rerun or clarify the position-selection procedure, and correct the table errors before the claims can be taken at face value. I did not find evidence of intentional misreporting beyond the Table 6 duplication, but the missing validation split for the noisy-view position study is a genuine correctness risk for the headline numbers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper does one genuinely new thing: it turns a large reconstruction model for pixel-aligned Gaussians into a denoising diffusion loop. Rather than predicting Gaussians from clean views in one feed-forward pass, it takes noisy target views as input, renders them from the predicted Gaussians, re-noises, and iterates. That combination is not in LGM, GRM, GeoLRM, or GS-LRM, and DMV3D's denoising loop uses NeRF, not splats. The empirical gains are real in the sense that ablations show the noisy view adds about +1.3 PSNR on GSO and +2.0 on Omni3D, and the visual comparisons are consistent with the numbers.\n\nThe soft spots are mostly about the evaluation. The paper never says which of the 21 orbit views the noisy view is, nor whether that view is excluded from the reported metrics. In Section 4.3, the authors search over noisy-view positions on the same GSO and Omni3D sets and report position 15 as best. If the main results use that position and the noisy view is itself one of the evaluation views, then NovelGS is being scored on a view it was explicitly conditioned to denoise, while the baselines (LGM, LaRa, InstantMesh, GeoLRM) are not given such a target. That would inflate the headline 31.303 vs 28.910 PSNR gap. The paper needs to state the exact protocol and verify the gain holds when the noisy view is excluded from the metrics. This is the load-bearing weakness.\n\nThere is also a clear factual error: the text says LPIPS is 'marginally lower' than the top baseline, but Tables 1 and 2 show our LPIPS is higher (worse) than InstantMesh. That needs fixing. And the closest related work, DMV3D, is cited in the related work but missing from the comparison tables; the authors should add it or explain why it isn't comparable.\n\nThe pixel-aligned Gaussian representation (depth from a sigmoid, scale clamped to [0.005, 0.02]) may limit expressiveness for thin or concave geometry, but that is secondary given the empirical results.\n\nOverall: the idea is worth taking seriously and the paper should go to peer review. The referee needs to press on the evaluation protocol first.","headline":"A useful new combination of denoising and pixel-aligned Gaussians, but the evaluation protocol needs to be nailed down before believing the SOTA numbers.","tokens_in":16088,"tokens_out":3389,"would_cite":true,"duration_ms":29421,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"NovelGS claims that sparse-view 3D reconstruction improves when the network denoises novel views through a transformer that predicts pixel-aligned Gaussians, rather than mapping input pixels straight to 3D.","keywords":["Sparse-view 3D reconstruction","Gaussian Splatting","Diffusion model","Novel-view denoising","Pixel-aligned Gaussians","Large reconstruction model","Image-to-3D","Text-to-3D"],"falsifier":"Render a set of objects chosen for thin and concave geometry, such as wire baskets, open umbrellas, and chairs with gaps, feed four fixed input views to NovelGS, and compare novel-view PSNR and reconstructed mesh completeness against a triplane or NeRF baseline; if the fixed-scale pixel-aligned Gaussians cap the detail, NovelGS should lose on exactly those objects even though it wins on the paper's average benchmarks.","tokens_in":15060,"feed_emoji":"🧊","tokens_out":7174,"duration_ms":61443,"temperature":0.7,"pith_summary":"The paper is trying to establish that the failure mode of feed-forward pixel-aligned Gaussian reconstruction, poor or inconsistent output on regions not visible in the input images, can be fixed by reformulating the reconstruction as a diffusion denoising problem. NovelGS feeds a transformer both the clean input views and one or more noisy target views with their camera rays; the transformer predicts a set of 3D Gaussians from which the target views are rendered, and at inference the target views are iteratively denoised from pure noise. The authors report that this 'dreaming' of unseen regions yields consistent, sharp textures and state-of-the-art numbers on two public datasets: 31.303 PSNR on GSO versus 28.910 for the best feed-forward baseline LaRa, and 31.195 versus 28.434 on Omni3D. If correct, the result matters because it turns sparse-view reconstruction into a generative task, so back sides and occluded parts can be plausibly completed rather than left as holes, and it plugs into text-to-3D and image-to-3D pipelines.","feed_headline":"NovelGS denoises novel views to beat sparse-view 3D baselines","feed_subtitle":"It generates the object's unseen side instead of leaving it blank, giving consistent and sharp novel views.","key_machinery":"The load-bearing mechanism is an iterative novel-view denoising loop built on a transformer. The denoiser tokenizes each posed image by concatenating RGB pixels with Plücker ray embeddings, ray origin cross direction plus direction, into a nine-channel feature map, adds positional encodings, and passes all view tokens through stacked cross-view self-attention blocks with adaLN-Zero timestep conditioning. From the contextualized tokens it decodes Gaussian attribute maps by transposed convolution and separate linear heads; each pixel becomes one 3D Gaussian, unprojected along its ray using a sigmoid-mapped depth clipped to the unit cube, with scale clamped to [0.005, 0.02] and a unit-normalized rotation quaternion. The same Gaussians are rendered at arbitrary viewpoints with splatting, which produces the target and supervision images used in the loss and also re-renders the noisy view for the next diffusion step.","core_discovery":"The central claim is that pixel-aligned Gaussian reconstruction is not the bottleneck; the bottleneck is that existing models are purely feed-forward and therefore cannot generate content for views they have never seen. NovelGS makes the reconstruction itself a denoising process: at each training step, four clean condition views and one noisy target view, along with camera ray embeddings, go into a transformer-based denoiser, which outputs per-pixel Gaussian attribute maps, namely depth, scale, rotation, opacity, and color, for every input view. The predicted Gaussians are rendered at the target view and at extra supervision views, and the model is trained with an L2 plus LPIPS image loss and a mask loss. At inference, target views start as pure noise and are rendered and re-noised through successive diffusion steps until step zero, at which point the final Gaussians define the 3D asset. The paper argues that this explicit iterative generation of unseen regions is what produces consistent geometry and sharp textures where feed-forward baselines collapse, and reports the largest gains on exactly those uncovered parts.","pith_inferences":["The paper does not test this, but its view-position results suggest that the noisy view is effectively a user-specified hallucination target: choosing where to place it controls which unseen parts of the object get generated, which could be used to interactively repair or complete specific regions.","Because the architecture accepts any number of clean and noisy view tokens, the same weights should generalize to unbalanced camera configurations, such as more views of the front than the back, which the paper's framing implies but does not quantify.","A stress test that would isolate the representation ceiling is to evaluate on thin or topologically complex objects; the fixed per-pixel depth and clamped scale [0.005, 0.02] may limit how much detail the denoising loop can add even when the diffusion prior is right.","The gain from denoising might transfer to other pixel-aligned predictors: the paper's comparison of with versus without a noisy view, 31.303 versus 29.985 PSNR on GSO, indicates that adding an iterative denoising stage to an existing feed-forward Gaussian model could be a general recipe, though NovelGS couples the two from the start."],"forward_implications":["Sparse-view reconstruction of objects with large unseen regions becomes feasible: the back side is generated consistently from surrounding views instead of being missing or blurred.","Adding more clean views keeps improving quality, with PSNR climbing from 21.7 with 1 view to 34.3 with 8 views on GSO in the paper's supplementary tables, at a predictable GPU-memory cost.","The position of the noisy view acts as a controllable attention budget: placing it opposite the clean views gives the best reconstruction, so users can aim the generative effort at the least-covered side of an object.","NovelGS can be composed with text-to-image and image-to-multi-view diffusion models, giving a text-to-3D and image-to-3D pipeline whose final reconstruction step is this Gaussian denoiser.","On the perceptual metric LPIPS, NovelGS trails one baseline while leading PSNR and SSIM, and the authors argue this is the price of generating plausible details that are consistent across views rather than smoothing toward the ground truth."],"supporting_citations":[{"why":"Defines the pixel-aligned Gaussian baseline that NovelGS improves on and that fails on unseen regions.","marker":"[46]"},{"why":"GRM, the transformer-based pixel-aligned Gaussian reconstruction model whose paradigm NovelGS extends with denoising.","marker":"[59]"},{"why":"GS-LRM, another pixel-aligned Gaussian large reconstruction model and part of the family NovelGS contrasts with.","marker":"[64]"},{"why":"LaRa, the strongest feed-forward baseline in the comparison tables, which NovelGS reports beating by roughly 2.4 dB PSNR.","marker":"[6]"},{"why":"DMV3D, the closest prior idea of denoising multiview images with a 3D reconstruction model, which NovelGS replaces with Gaussian splatting.","marker":"[60]"},{"why":"Objaverse, the training corpus of filtered multiview renderings, about 270k instances.","marker":"[8]"},{"why":"GSO, one of the two public evaluation datasets behind the headline PSNR numbers.","marker":"[10]"},{"why":"OmniObject3D, the second public evaluation dataset used for the comparisons.","marker":"[55]"},{"why":"Splatter Image, the per-pixel 3D Gaussian formulation that NovelGS adopts for its output representation.","marker":"[45]"},{"why":"pixelSplat, which supplies the Plücker-ray camera embedding and pixel-aligned Gaussian parameterization conventions.","marker":"[5]"}],"fun_headline_variants":["NovelGS turns 3D reconstruction into denoising for unseen views","NovelGS generates missing 3D views by iterative denoising","Denoising-based 3D reconstruction: NovelGS fills unseen views","NovelGS: Denoise your way to consistent 3D from sparse views","Feed-forward 3D eats your unseen views; NovelGS denoises instead"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that one Gaussian per pixel, with depth read from a sigmoid and scale clamped between 0.005 and 0.02, can represent whatever unseen geometry the denoising loop is supposed to create; if thin structures, concavities, or complex topologies cannot be expressed this way, no diffusion refinement can recover them.","fun_headline_variants_meta":{"raw":{"variants":["NovelGS turns 3D reconstruction into denoising for unseen views","NovelGS generates missing 3D views by iterative denoising","Denoising-based 3D reconstruction: NovelGS fills unseen views","NovelGS: Denoise your way to consistent 3D from sparse views","Feed-forward 3D eats your unseen views; NovelGS denoises instead"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000569,"raw_usage":{"total_tokens":2714,"prompt_tokens":990,"completion_tokens":1724,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":606,"completion_tokens_details":{"reasoning_tokens":1624}},"tokens_in":606,"tokens_out":1724,"duration_ms":12505,"temperature":1.0,"reasoning_tokens":1624,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:27:46.980077+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render a set of objects chosen for thin and concave geometry, such as wire baskets, open umbrellas, and chairs with gaps, feed four fixed input views to NovelGS, and compare novel-view PSNR and reconstructed mesh completeness against a triplane or NeRF baseline; if the fixed-scale pixel-aligned Gaussians cap the detail, NovelGS should lose on exactly those objects even though it wins on the paper's average benchmarks.","supporting_citations":[{"cited_title":"Lgm: Large multi-view gaussian model for high-resolution 3d content creation","cited_arxiv_id":null,"evidence_quote":"Defines the pixel-aligned Gaussian baseline that NovelGS improves on and that fails on unseen regions."},{"cited_title":"Lara: Efficient large-baseline radiance fields","cited_arxiv_id":null,"evidence_quote":"LaRa, the strongest feed-forward baseline in the comparison tables, which NovelGS reports beating by roughly 2.4 dB PSNR."},{"cited_title":"Dmv3d: Denoising multi-view diffusion using 3d large reconstruction model","cited_arxiv_id":null,"evidence_quote":"DMV3D, the closest prior idea of denoising multiview images with a 3D reconstruction model, which NovelGS replaces with Gaussian splatting."},{"cited_title":"Objaverse: A universe of annotated 3d objects","cited_arxiv_id":null,"evidence_quote":"Objaverse, the training corpus of filtered multiview renderings, about 270k instances."},{"cited_title":"Splatter image: Ultra-fast single-view 3d recon- struction","cited_arxiv_id":null,"evidence_quote":"Splatter Image, the per-pixel 3D Gaussian formulation that NovelGS adopts for its output representation."}],"review_version":1}