{"id":"90fea123-4fe6-4543-a77e-c9223d5c29a5","arxiv_id":"2411.18072","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SmileSplat predicts Gaussian surfels from sparse unposed image pairs and jointly optimizes scene geometry and camera intrinsics and extrinsics, reporting state-of-the-art novel view synthesis on Re10K, ACID, Replica, and ICL-NUIM.","lead":"SmileSplat is a method for turning two uncalibrated and unposed photos into a 3D scene by predicting flat Gaussian surfaces and then refining the scene and camera parameters together. The authors report large gains over prior work, but the comparisons are weakened by extra test-time optimization, missing code, and an overclaimed novelty.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported Re10K SOTA is not yet established: SmileSplat receives 100 iterations of source-view K/G/T refinement plus target-pose refinement, while Table 1 baselines are not given a matched test-time budget; Table 2 shows most of the gain is on observed views.","rationale":"The reader's verdict is CONDITIONAL, and I agree that a conditional verdict is appropriate, but for a somewhat different load-bearing reason. The reader's weakest_assumption focuses on initial pointmap accuracy and two-view consistency; that is a plausible robustness concern, and the paper's own Supplementary C.3 already concedes such failures, so it is not the most decisive threat to the headline claim. The more load-bearing issue is the evaluation protocol: the empirical claim 'state-of-the-art' in Table 1 is meaningful only if all methods face the same test-time rules. The manuscript's Algorithm 1 and Supplementary C.1 describe source-view K/G/T refinement plus target-pose refinement for SmileSplat, while the baselines are not given an equal budget in the reported comparison. Table 2 provides internal evidence that most of the photometric gain from refinement accrues to the two observed views, with only a small change on the held-out View 3. This suggests the 4.30 dB advantage over NoPoSplat on small-overlap Re10K may be inflated by extra test-time optimization rather than caused by the proposed representation. A matched-budget or target-withheld control would settle this directly. The concern does not move the verdict away from CONDITIONAL; it sharpens the condition the paper must satisfy before the central SOTA claim can be accepted. This is consistent with the absence of released code and data, which makes the baseline settings unverifiable from the manuscript alone.","tokens_in":18424,"tokens_out":23357,"duration_ms":229658,"concrete_test":"On the Re10K small-overlap split, rerun Table 1 under a matched test-time budget. Control 1: after Algorithm 1 converges on the two source images, render the target view without target-image pose refinement, using only the relative pose predicted from the source pair. Control 2: give NoPoSplat the same 100-iteration source-view K/T/G refinement plus the same target-pose refinement used by SmileSplat, and similarly permit test-time refinement for pixelSplat/MVSplat if pose-free variants exist. Recompute PSNR/SSIM/LPIPS for all methods. If SmileSplat's advantage collapses in Control 1 or shrinks to insignificance in Control 2, the claimed SOTA is an artifact of asymmetric test-time optimization rather than of the proposed surfel and bundle-adjustment contributions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central SOTA claim rests on an asymmetric test-time protocol. Algorithm 1 gives SmileSplat 100 iterations of K/G/T refinement on the two source views; for the reported novel views, Supplementary C.1 says the observed target image is used for pose-only refinement. The baseline rows in Table 1 do not describe an equivalent per-scene budget: NoPoSplat is listed with K given, and pixelSplat/MVSplat receive GT K/T. Table 2 shows the consequence: after photometric optimization, PSNR on the two source views rises from about 24/16 dB to 29/21 dB, while the genuinely held-out View 3 moves only from about 18.6 to 19.3 dB. Thus the headline Re10K small-overlap advantage (26.81 vs 22.51 PSNR; 0.071 vs 0.462 LPIPS) may be dominated by extra optimization on observed views rather than by a better generalizable radiance field. Until the comparison is rerun with matched test-time budgets, or with the target image withheld, the state-of-the-art claim is unverified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"SmileSplat proposes a generalizable Gaussian splatting model for novel-view rendering and depth prediction from two unconstrained images when ground-truth camera parameters are not available. The method uses a Siamese ViT encoder with cross-attention, a multi-head Gaussian surfel regression decoder initialized from Dust3R pointmaps, and a pretrained normal prior. Camera intrinsics, relative extrinsics, and Gaussian parameters are then refined by a bundle-adjusting Gaussian splatting module that iterates 100 times per test pair. Experiments on Re10K and ACID report state-of-the-art results, e.g., Re10K small-overlap PSNR 26.81 vs 22.51 for NoPoSplat, and cross-dataset results are reported on Replica and ICL-NUIM.","tokens_in":18658,"tokens_out":5950,"duration_ms":52468,"significance":"If the empirical claims held, the paper would be practically valuable because it removes both camera poses and intrinsics from the sparse-view rendering pipeline and offers a checkable, well-documented projection-gradient derivation in the appendix. The paper also includes an ablation table that separates gains on observed views from gains on a held-out view, which is useful diagnostic evidence. However, the state-of-the-art claim is not yet established: the main comparison gives SmileSplat substantially more test-time computation than the baselines and, for novel views, may use the target image during pose-only refinement. The significance is therefore conditional on a fair, matched-budget evaluation.","major_comments":[{"comment":"The headline Re10K comparison is not a matched evaluation. Algorithm 1 allocates 100 iterations of K/G/T refinement, and Supplementary C.1 states that for novel views the target image is used for pose-only optimization. None of the baseline rows in Table 1 is given an equivalent test-time optimization budget; pixelSplat and MVSplat receive ground-truth K/T, and NoPoSplat is run without a stated per-scene bundle adjustment. Because Table 2 shows that most of the photometric gain occurs on the two observed views while the held-out View 3 improves only from about 18.6 to 19.3 dB, the 4.3 dB gap over NoPoSplat in the small-overlap row of Table 1 may reflect extra optimization on observed images rather than a better generalizable radiance field. Please rerun the comparison with matched per-scene optimization (either give all methods the same number of iterations or report a no-BA variant of SmileSplat) and report novel-view metrics with the target image withheld.","section":"Algorithm 1, Table 1, Supplementary C.1"},{"comment":"The novelty claim that SmileSplat is 'the first to render images without requiring predefined intrinsic parameters' is inconsistent with Table 1, where Splatt3R is evaluated under the K=×, Pose=× setting, meaning it also operates without predefined intrinsics. If the intended distinction is that SmileSplat additionally optimizes camera parameters against the Gaussian radiance field, that should be stated precisely; otherwise the claim should be removed or qualified.","section":"Section 1, Table 1"},{"comment":"The cross-dataset generalization results on Replica and ICL-NUIM should be reported under the same protocol clarifications as Table 1. If the target views in those tables are used for pose-only bundle adjustment before the reported PSNR/SSIM/LPIPS are computed, the comparison against NoPoSplat is again asymmetric. Please state explicitly for every table whether the target image was observed during any part of the refinement and give the runtime or iteration count.","section":"Tables 3 and 4"}],"minor_comments":[{"comment":"The definition Lssimi = ssim(I^u,v_i − Ibar^u,v_i) appears to be a typo; SSIM should compare the rendered and observed images directly, not their difference. Please correct the definition and the subscript notation.","section":"Equation (9)"},{"comment":"The text refers to 'Dust3D [38]'; the intended reference is Dust3R. Please fix the name.","section":"Section 3.1"},{"comment":"The intrinsic matrix K is written with fx in both diagonal entries; the second entry should presumably be fy. Please correct the notation.","section":"After Equation (21)"},{"comment":"The caption 'all three views are novel views, while View 3 is the novel view when the first two views are used for reference Gaussian surfels' is confusing. Please clarify which rows use which views during optimization and which view is held out in each row.","section":"Table 2 caption"},{"comment":"There is a typo: 'state-pf-the-art' should be 'state-of-the-art'.","section":"Supplementary B.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a plausible systems contribution with standard appendix derivations, but the headline SOTA claim depends on an asymmetric test-time protocol. The most direct fix is a matched-budget comparison and explicit target-view disclosure per table. The 'first to render without intrinsics' claim is likely resolvable by checking Splatt3R in their own Table 1; I do not see a deep correctness flaw in the central derivation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: SmileSplat is a real engineering contribution. The idea of optimizing the intrinsic matrix through a differentiable Gaussian splatting renderer, jointly with extrinsics and Gaussians, is genuinely new, and the appendix math for the camera gradients is standard and correct. But the headline claim of state-of-the-art on RealEstate10K is not supported yet, because the comparison is not run under a matched test-time budget.\n\nWhat's good: the system is clearly described. Gaussian surfels with a normal prior and a Dust3R-initialized pointmap head are a sensible combination, and the bundle-adjusting module cleanly ties together photometric and geometric losses. The supplement is unusually transparent: C.3 admits that wrong initial geometry (the stair guardrail) cannot be fixed by refinement, and Table 2 shows per-view results rather than only averages. That transparency lets the reader see the actual behavior.\n\nThe soft spots, in order. First, the test-time protocol is asymmetric. Algorithm 1 gives SmileSplat 100 iterations of K/T/G refinement on the source views, and for the reported novel view the target pose is refined against the observed image. The baselines in Table 1 don't get an equivalent per-scene budget. NoPoSplat is listed with K given; pixelSplat and MVSplat get GT K and T. Table 2 is the evidence: on the two source views, photometric refinement lifts PSNR to roughly 29/21 dB, while the genuinely held-out View 3 moves only from 18.63 to 19.27. So most of the gap in Table 1 is likely extra optimization on observed views, not a fundamentally better generalizable radiance field. The SOTA claim needs to be re-run with a matched budget, and with the target view fully withheld.\n\nSecond, the novelty claim \"first to render without predefined intrinsics\" is contradicted by Splatt3R in their own Table 1, which runs with both K and pose marked as not required. They probably mean something narrower, but as written it's false.\n\nThird, no code or data. The promise to release is fine, but for a paper whose central quantity is a benchmark improvement, the numbers need to be independently checkable. Table 3 also contains at least one implausible cell (NoPoSplat SSIM 0.134 and LPIPS 0.777 on Room 1 Medium), which looks like a table-formatting error and should be corrected.\n\nWho this is for: people working on pose-free and calibration-free generalizable 3D reconstruction. It deserves a serious referee, not a desk reject. I'd press the authors to (1) compare with matched test-time budgets, (2) report novel-view-only metrics, (3) release code and trained models, and (4) fix the overstated claims. The core idea is worth pursuing; the current numbers should be treated with caution.","headline":"Solid engineering idea with a real new mechanism—optimizing intrinsics through Gaussian splatting—but the Re10K SOTA claim rests on an unmatched test-time budget and should be re-evaluated before being trusted.","tokens_in":19207,"tokens_out":3385,"would_cite":false,"duration_ms":29726,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"From two unposed images, SmileSplat builds render-ready Gaussian scenes.","keywords":["generalizable Gaussian splatting","sparse-view rendering","unposed images","camera calibration-free","bundle adjustment","Gaussian surfels","depth prediction"],"falsifier":"Take an image pair with known ground-truth intrinsics and pose where the initial pointmap is known to be wrong, for example a low-texture wall or a repetitive pattern, run SmileSplat, and compare the refined K and T against the ground truth; if the final photometric loss is low yet the recovered intrinsics or pose are far from the true values, the claim that the method recovers camera parameters rather than merely fitting the two views fails.","tokens_in":18211,"feed_emoji":"🖼️","tokens_out":5246,"duration_ms":45075,"temperature":0.7,"pith_summary":"SmileSplat tackles the setting where a 3D scene must be rendered from just two overlapping photographs with no camera poses and no intrinsics supplied. The paper claims that a network can predict pixel-aligned Gaussian surfels from the images, then refine the surfels together with the unknown camera intrinsics and extrinsics in a joint bundle-adjustment step, yielding scaled radiance fields. If correct, this removes the calibration and pose-estimation bottleneck that normally precedes novel-view synthesis, making sparse-view rendering practical for casually captured photo pairs. The method reports large gains over pose-free baselines, such as 26.81 versus 22.51 PSNR on the Re10K small-overlap split, and claims to be the first to render without predefined intrinsics.","feed_headline":"Two unposed images yield photorealistic novel views","feed_subtitle":"SmileSplat estimates camera intrinsics and pose inside a bundle-adjusted Gaussian splatting pass.","key_machinery":"The load-bearing object is the Gaussian surfel, a flattened 3D Gaussian parameterized by color, center $\\mu$, two scale values, a normal vector $n$, and opacity $o$, whose covariance is $\\Sigma = R\\,\\mathrm{Diag}[(s_x)^2,(s_y)^2,0]\\,R^T$. Surfels are pixel-aligned and anchored to the pointmap predicted by a pretrained Dust3R head, which gives the optimization a consistent initial geometry. The second mechanism is the Bundle-Adjusting Gaussian Splatting module: it differentiates the rasterizer's 2D Gaussian means and covariances with respect to the intrinsic matrix $K$ and the pose $T\\in SE(3)$, so that photometric loss from both views and a depth-warp geometric loss jointly refine Gaussians, $K$, and $T$. This turns the Gaussian renderer itself into a camera-calibration engine.","core_discovery":"The central claim is that camera parameters can be treated as optimizable variables inside a generalizable Gaussian Splatting pipeline rather than as inputs or as a separate preprocessing stage. SmileSplat first predicts Gaussian surfels in the coordinate frame of the first image using a multi-head regression decoder, with positions anchored to a pretrained pointmap head and surface normals supervised by a normal-prior network. It then optimizes the intrinsic matrix by rendering the first view, the relative pose by rendering the second view, and finally jointly adjusts Gaussians, intrinsics, and extrinsics under photometric and geometric (depth-warp consistency) losses. The paper argues this yields accurate scaled radiance fields from unposed, uncalibrated sparse views, and supports the claim with novel-view rendering and depth evaluations on Re10K, ACID, Replica, and ICL-NUIM, where it outperforms pose-free baselines such as NoPoSplat by a large margin.","pith_inferences":["The paper evaluates only two-view input; a natural extension the authors do not explore is whether the same bundle-adjusting loop scales to three or more views, where the geometric consistency loss would have more constraints and could correct larger initial pointmap errors.","Because the geometric loss compares depth maps across views, the architecture could be adapted to produce scale-consistent dense reconstructions for robotics and augmented reality, not just renderings.","The reliance on a pretrained pointmap head suggests a testable boundary: on scenes with repetitive texture or transparent and specular surfaces, the initial geometry may be wrong in a way that photometric refinement cannot escape, as the paper's own stair-guardrail limitation indicates.","The optical-center initialization at the image center and focal length at 1.2 times the image size imply that extreme lenses, such as fisheye or strongly cropped views, may fall outside the basin of convergence; a stress test across a focal-length range would map the method's practical calibration envelope."],"forward_implications":["Two overlapping images become the minimal input for photorealistic novel-view synthesis, with no COLMAP or SfM preprocessing.","Intrinsics and extrinsics are recovered to the accuracy needed for rendering, so casual camera settings with unknown focal length and pose are absorbed by the optimization.","Cross-dataset generalization improves: the paper's Re10K-trained model renders Replica and ICL-NUIM scenes with higher PSNR than the pose-free baseline, suggesting the representation transfers to new indoor environments.","Jointly optimizing geometry and cameras produces scale-consistent depth as a byproduct, enabling depth-map prediction alongside rendering.","The method sets a new operating point for pose-free sparse-view Gaussian splatting, exceeding the previous pose-free baseline by more than 4 dB PSNR on the small-overlap split of Re10K."],"supporting_citations":[{"why":"provides the pretrained pointmap head whose predicted positions anchor the Gaussian surfels and supply the initial geometry.","marker":"[38]"},{"why":"defines the differentiable 3D Gaussian splatting rasterizer used for rendering and for the photometric and geometric losses.","marker":"[20]"},{"why":"supplies the pretrained surface-normal prior that supervises the normal prediction head.","marker":"[1]"},{"why":"is the pose-free baseline that SmileSplat compares against and improves upon on Re10K, ACID, Replica, and ICL-NUIM.","marker":"[45]"},{"why":"introduces pixel-aligned Gaussian prediction from image pairs, the representation family SmileSplat adapts to Gaussian surfels.","marker":"[4]"},{"why":"is the CamPara-Required pixelSplat baseline used in the main quantitative comparison.","marker":"[5]"},{"why":"is the CamPara-Required MVSplat baseline whose sparse-view rendering approach and evaluation protocol are adopted.","marker":"[6]"},{"why":"contributes the Lie-algebra pose optimization on SE(3) used in the extrinsic refinement.","marker":"[25]"}],"fun_headline_variants":["Bundle-adjusted Gaussian splats from two unposed views","Self-calibrating Gaussian splats for sparse images","SmileSplat: pose-free Gaussian splatting in one pass","Unposed sparse images to photorealistic novel views","Gaussian splats that learn camera parameters on the fly"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole pipeline assumes the pointmap and surface normals predicted by the pretrained heads are accurate and mutually consistent enough that the photometric refinement converges to the true scene rather than to a local optimum that happens to reproduce the two input views.","fun_headline_variants_meta":{"raw":{"variants":["Bundle-adjusted Gaussian splats from two unposed views","Self-calibrating Gaussian splats for sparse images","SmileSplat: pose-free Gaussian splatting in one pass","Unposed sparse images to photorealistic novel views","Gaussian splats that learn camera parameters on the fly"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000348,"raw_usage":{"total_tokens":1907,"prompt_tokens":952,"completion_tokens":955,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":888}},"tokens_in":568,"tokens_out":955,"duration_ms":8567,"temperature":1.0,"reasoning_tokens":888,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:32:25.549036+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take an image pair with known ground-truth intrinsics and pose where the initial pointmap is known to be wrong, for example a low-texture wall or a repetitive pattern, run SmileSplat, and compare the refined K and T against the ground truth; if the final photometric loss is low yet the recovered intrinsics or pose are far from the true values, the claim that the method recovers camera parameters rather than merely fitting the two views fails.","supporting_citations":[{"cited_title":"pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction","cited_arxiv_id":null,"evidence_quote":"is the CamPara-Required pixelSplat baseline used in the main quantitative comparison."}],"review_version":1}