{"id":"6814a30e-a0d3-4787-8fd3-0a902c6cd4a9","arxiv_id":"2501.04628","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"FatesGS combines local monocular depth ranking, depth smoothing, and multi-view feature alignment in a 2D Gaussian splatting pipeline to obtain accurate surface meshes from only three views without dataset-scale pre-training.","lead":"This paper presents FatesGS, a method that reconstructs 3D surfaces from just three camera views using 2D Gaussian splatting, supervised by monocular depth ranking and multi-view feature consistency. It reports state-of-the-art accuracy on DTU and BlendedMVS benchmarks while training in about 14 minutes per scene, much faster than prior sparse-view methods that need hours or days.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Concern: the multi-view feature loss's visibility mask (Eq. 15) is computed from the very depth being optimized, so early misprojections can systematically exclude correct correspondences and lock geometry into a local minimum; the ablations only remove the whole loss and never isolate this…","rationale":"The reader correctly identifies the mutually recursive depth/visibility estimate as a key weak spot, but frames it mainly as a concern about early-optimization reliability and dependence on the frozen feature network. My stress-test sharpens this into one specific, load-bearing, and testable mechanism: the hard Iverson-bracket mask in Eq. 15, computed from the optimized depth, can actively exclude the correct correspondence during training and then certify a wrong one, a failure mode not covered by the full-model-minus-Lf ablation. This is a correctness risk rather than a mere generalization risk, and it is precisely the kind of issue that could make the reported SOTA numbers an artifact of a self-confirming loop on the three-view DTU setup. I do not see a fatal flaw in the paper as a whole, and I agree with the reader's CONDITIONAL verdict: the central claims (fast training, no-domain pretraining, SOTA CD) are plausible and largely supported by the reported experiments, but the specific mechanism I identified should be checked before the SOTA claim is accepted without qualification. The proposed test is cheap (reusing GT depth for masks and instrumenting mask accuracy) and would settle whether the concern lands. I keep the verdict at CONDITIONAL rather than moving to REJECT because the concern is about a potential failure mode, not an established contradiction; a clean GT-mask result would confirm the method is sound, while a large gap would require the authors to re-run evaluations or soften their claims.","tokens_in":127,"tokens_out":1895,"duration_ms":140615,"concrete_test":"Instrument the training loop: at intermediate iterations (e.g., 25%, 50%, 75% of the 14-minute run) on several DTU scans (e.g., 24, 65, 69, 83), recompute the visibility mask using ground-truth depth from DTU and compare the fraction of pixels where the learned mask agrees with the GT mask. Additionally, run an experiment where Lf uses the same cosine feature loss but with the mask computed from ground-truth depth rather than rendered depth, and report mean CD. If the GT-mask variant does not improve over the learned-mask variant, the recursive mask is not the bottleneck; if it improves substantially (e.g., more than 0.2 CD), the learned mask is locking the optimization into incorrect occlusions and the central accuracy claim needs re-evaluation.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central accuracy gain comes from the multi-view feature alignment loss Lf (Eq. 14), whose ablation removes it and drops mean CD from 1.37 to 2.47 in the little-overlap setting. But Lf relies on a visibility mask v_{r,s,i} (Eq. 15) that is computed by back-projecting the rendered depth of the reference view and testing occlusion along the source ray. That rendered depth is the quantity being optimized, and with only three views the depth can be badly wrong early in training. If early depth places a surface point behind the true geometry, the correct match is masked out and the loss is minimized by pushing the point to the nearest incorrect surface; the mask then keeps certifying that wrong point as visible. This is a chicken-and-egg loop, distinct from a generic 'depends on good initialization' concern, because the mask is a hard Iverson bracket, not a soft weight, and it has no fallback when a projected point is judged occluded. The ablation studies (Table 4 and supplement Table 8) only compare 'with Lf' versus 'without Lf', so they do not test whether the mask is accurate or whether this loop actually occurs. The paper also does not report mask accuracy, masked-out pixel statistics, or sensitivity to the COLMAP initialization point cloud.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FatesGS, a per-scene optimization method for sparse-view surface reconstruction built on 2D Gaussian Splatting. The method adds three ingredients to a 2DGS backbone: a patch-wise depth ranking loss using monocular depth, a depth smoothing loss, and a multi-view feature alignment loss that compares frozen Vis-MVSNet features of reprojected surface points under a rendered-depth visibility mask. The authors report a mean Chamfer Distance of 0.92 on the DTU large-overlap three-view setting, a training time of about 14 minutes per scene, and additional experiments on the little-overlap setting, dense-view reconstruction, depth prediction, and novel view synthesis. Ablation studies attribute most of the accuracy gain to the multi-view feature loss.","tokens_in":21545,"tokens_out":5485,"duration_ms":55476,"significance":"If the empirical claims hold, FatesGS is a practically useful contribution: it achieves the best mean Chamfer Distance among the compared methods on the DTU large-overlap sparse-view benchmark while reducing per-scene optimization time by roughly an order of magnitude relative to NeuSurf, and it avoids dataset-scale pretraining. The paper is strong in experimental breadth: it provides comparisons with MVS, generalizable implicit, per-scene implicit, and Gaussian-splatting baselines, along with ablations, depth evaluation, and additional qualitative results. The main weaknesses are statistical: no error bars or significance tests are reported, and the claimed superiority is not uniform across settings or scans. The central feature loss also relies on a hard visibility mask computed from the depth being optimized, and the paper does not analyze the failure modes of this self-referential mechanism. These issues affect the load-bearing accuracy claim and need to be addressed before the paper can be accepted.","major_comments":[{"comment":"The visibility mask v_{r,s,i} in Eq. (15) is a hard Iverson bracket computed from the rendered depth that is itself the optimization target. Because Table 8 shows that the visibility item is important (removing v degrades mean CD from 1.37 to 1.62) and Table 4 shows that Lf is the main contributor to accuracy (without Lf, CD rises from 1.37 to 2.47), early incorrect depth can systematically mask the correct correspondences and lock the geometry into a local minimum. The current ablations only remove the whole loss or the whole mask; they do not measure mask accuracy, masked-pixel statistics, or sensitivity to the COLMAP initialization. Please add diagnostics that track mask errors over training and test a softened visibility formulation or a coarse-to-fine depth schedule to verify that the reported gains are not an artifact of this chicken-and-egg loop.","section":"Multi-View Feature Alignment, Eq. (15)"},{"comment":"The empirical claims are not supported with statistical evidence. No error bars, standard deviations, or significance tests are reported for the mean Chamfer Distance values. On the large-overlap setting, FatesGS is worse than NeuSurf on at least four of fifteen scans (scans 55, 63, 69, and 114), and on the little-overlap setting in Supplementary Table 2 the mean CD is 1.37 versus NeuSurf's 1.35, so FatesGS is not state of the art in that setting. The statements in the abstract and conclusion that the method 'outperforms' prior methods in two distinct settings should be qualified, and a paired test or per-scan breakdown should be reported to establish whether the large-overlap mean improvement is significant.","section":"Experiments, Table 1 and Supplementary Table 2"},{"comment":"The evaluation protocol contains a circularity risk: the frozen feature extractor fϕ is Vis-MVSNet, which is trained on DTU, and the loss weights and margins appear to be tuned on the same DTU scans used for evaluation. The BlendedMVS generalization experiment is only qualitative (Figure 3) and reports no Chamfer Distance numbers, so it does not quantitatively establish cross-dataset robustness. Please report quantitative BlendedMVS results, and ideally include an ablation that replaces Vis-MVSNet with a feature network not trained on DTU, to show that the feature-consistency loss transfers outside the training distribution.","section":"Experimental Settings, Implementation Details; BlendedMVS results"},{"comment":"The claimed 60x to 200x speedup conflates different training protocols. Table 2 compares the proposed per-scene optimization time (14 minutes) with the pre-training time of generalizable methods such as UFORecon (about 10 days), but for those methods pre-training is amortized and the per-scene inference cost is what matters. Without reporting per-scene inference times for the generalizable baselines, the speedup claim is overstated. The paper should state explicitly whether the speedup refers to per-scene optimization only, and should report the per-scene inference or fine-tuning times of the compared methods, or remove the comparison with pre-training time from the headline claim.","section":"Efficiency, Table 2"}],"minor_comments":[{"comment":"The normal consistency loss definition is unclear: the text says both n_i^T and N are 'estimated normal near the depth point'. Please clarify which normal is the Gaussian primitive normal and which is the rendered-surface normal, and check the formula against the 2DGS formulation.","section":"Loss Functions, Eq. (18)"},{"comment":"The result that Lr and Ls alone (mean CD 2.56) are worse than the baseline without any of the three losses (2.47) is counterintuitive and deserves a more mechanistic explanation than the statement that the contributions are 'neither isolated nor merely additive'. Consider reporting an interaction analysis or per-scan results for this ablation.","section":"Ablation Study, Table 4"},{"comment":"There are several typos, including 'little-overlap seltting' in the supplementary text and the title 'F atesGS', and the duplicated definition of the Sun et al. reference (2024a and 2024b appear to cite the same paper). These should be corrected.","section":"Supplementary Material, Section B"},{"comment":"The overview figure is difficult to read: the loss labels λ1 through λ5 and the feature-level symbols are too small, and the flow of the multi-view feature alignment path is not clear. Please enlarge the fonts and annotate the modules that correspond to Eqs. (8), (9), and (14).","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical systems contribution, but the main accuracy mechanism (the multi-view feature loss with a depth-derived hard visibility mask) is exactly where a failure mode could invalidate the central claim, and the current ablation does not address it. The novelty relative to SparseNeRF's depth ranking and 2DGS is incremental but acceptable for a conference if the robustness questions are answered. I would suggest the editor require the additional diagnostic experiments and a more careful presentation of the efficiency and cross-dataset claims before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid engineering paper that delivers on its headline claim—sparse-view surface reconstruction in 14 minutes per scene with accuracy competitive with NeuSurf's 14 hours. The gain over NeuSurf on DTU large-overlap is modest (0.92 vs 0.99 CD) and on the little-overlap setting it actually loses (1.37 vs 1.35). The 60–200x speedup is the real contribution, and it holds up because the method is per-scene optimization without dataset-scale pretraining, though it does lean on frozen Vis-MVSNet and Marigold.\n\nWhat's new here is the combination: patch-based monocular depth ranking, depth smoothing, and multi-view feature alignment on top of 2DGS. The ablation is genuinely informative—removing Lf drops CD from 1.37 to 2.47 in little-overlap, and they honestly show that Lr+Ls alone hurt. That's careful work.\n\nSoft spots, in order of softness. First, the evaluation: no error bars, no significance testing, and on 4 of 15 DTU scans NeuSurf is better. The mean improvement is within a range where run-to-run variance could matter. Second, the 'no pre-training' framing needs a qualifier: the feature extractor and depth network are frozen pretrained models, and Vis-MVSNet was trained on DTU, the same benchmark used for evaluation. That's mild circularity, not disqualifying, but it should be stated. Third, the visibility-mask chicken-and-egg the stress-test raises: the mask in Eq. 15 is a hard Iverson bracket on rendered depth being optimized. It's a legitimate concern, but the ablation w/o v (1.62 vs 1.37) shows the mask helps in practice. What's missing is a sensitivity analysis on initialization and mask accuracy. Fourth, no code release.\n\nBottom line: the empirical claim is plausible and the speedup is meaningful. I'd accept this for peer review and push for revisions—error bars, a statement about hyperparameter tuning, and an analysis of visibility-mask behavior early in training. But I would not desk-reject this.","headline":"Solid engineering paper: 14-minute sparse-view reconstruction with accuracy on par with NeuSurf's 14-hour method, but the evaluation needs error bars and the pretrained-feature circularity should be acknowledged.","tokens_in":22075,"tokens_out":2618,"would_cite":true,"duration_ms":24020,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FatesGS claims that sparse-view reconstruction can be made more accurate and 60 to 200 times faster by adding patch-wise depth ranking and smoothing plus multi-view feature alignment to 2D Gaussian splatting.","keywords":["sparse-view reconstruction","Gaussian splatting","2D Gaussian splatting","surface reconstruction","monocular depth ranking","multi-view feature consistency","depth smoothness","Chamfer distance"],"falsifier":"Two concrete checks would settle the central claim. First, initialize the Gaussians from a deliberately corrupted point cloud (translated or scaled away from the true surface) on a DTU three-view scene and measure the resulting Chamfer Distance: recovery would show the consistency losses genuinely guide geometry, while a large CD jump would show the accuracy depends on COLMAP's initialization rather than the proposed mechanism. Second, replace the frozen feature extractor (Vis-MVSNet) with a different pretrained network and re-run the DTU large-overlap benchmark: a substantial change in mean CD would show the reported 0.92 is carried by the specific feature network's invariance rather than by the consistency formulation itself.","tokens_in":21054,"feed_emoji":"📐","tokens_out":14609,"duration_ms":115265,"temperature":0.7,"pith_summary":"FatesGS sets out to show that surface reconstruction from as few as three input views — the regime where Gaussian-splatting methods typically produce floaters and holes — does not need days of pretraining or hours of per-scene optimization to be accurate. On the DTU large-overlap three-view benchmark it reports a mean Chamfer Distance of 0.92, the best among all compared methods, with about 14 minutes of training per scene on a single GPU at 4 GB of memory, a 60x to 200x speedup over the prior state of the art. The recipe keeps the fast 2D-Gaussian-splatting backbone and disciplines it with two consistency signals: a patch-based ranking loss and a smoothness loss that transfer order information from monocular depth, plus a multi-view feature-alignment loss that fixes absolute surface position by matching reprojected features across views. If the claims hold, accurate sparse-view reconstruction becomes a per-scene affair measured in minutes instead of hours or days, which matters for any capture setting with limited views or compute.","feed_headline":"3 views, 14 minutes: 3D surfaces that beat day-long baselines","feed_subtitle":"Depth ranking plus multi-view feature alignment yields the top Chamfer distance on DTU's 3-view benchmark.","key_machinery":"The load-bearing object is the 2D Gaussian primitive — a flat ellipse with learnable center, scale, rotation, opacity, and color, rendered by depth-sorted $\\alpha$ blending as in the 2DGS backbone, so that rendered depth comes from ray-splat intersection. Around that backbone the paper places three loss terms that carry the argument. The patch ranking loss ($L_r$, Eq. 8) shuffles pixels within $M \\times M$ patches and penalizes rendered-depth pairs whose order disagrees with monocular depth order, transferring relative scale without committing to absolute scale. The smoothness loss ($L_s$, Eq. 9) penalizes rendered-depth jumps between adjacent pixels while threshold margins protect real boundaries. The multi-view feature loss ($L_f$, Eq. 14) unprojects each pixel's rendered depth into 3D, reprojects the point into source views, and aligns multi-resolution features from a frozen feature network by cosine similarity, weighted by a visibility indicator (Eq. 15) that keeps only the nearest reprojected point per ray. The full objective is $L = L_c + \\lambda_1 L_r + \\lambda_2 L_s + \\lambda_3 L_f + \\lambda_4 L_d + \\lambda_5 L_n$, with the backbone's depth-distortion and normal-consistency terms retained.","core_discovery":"The central claim, stated on the paper's own terms, is that the geometric collapse of Gaussian splatting under sparse views is not an inherent limitation of the representation but a correctable failure of optimization, and that two complementary consistency priors suffice to correct it without large-scale pretraining. Intra-view depth consistency — enforcing that rendered depths inside local patches respect the same ranking as a monocular depth estimate, together with a smoothness loss between adjacent pixels — stabilizes the coarse global shape and suppresses the floaters that arise from overfitting a handful of training views. Multi-view feature alignment then recovers detail: each depth-rendered pixel is unprojected into 3D, reprojected into the other views, and its multi-resolution features (from a frozen feature network) are aligned by cosine similarity under an occlusion mask computed from the rendered depth itself. The paper reports that this combination attains a mean Chamfer Distance of 0.92 on the DTU large-overlap three-view benchmark, the best among all compared methods, while training in about 14 minutes per scene, and that the same hyperparameters transfer to BlendedMVS and to the harder little-overlap setting without re-tuning.","pith_inferences":["A stress test the paper does not run: initialize the Gaussians from deliberately wrong or perturbed point clouds (translated, scaled, or partially missing). If the feature alignment loss redirects the geometry to the true surface, the depth-visibility recursion is robust; if it locks onto floaters, the method inherits the initialization's failure mode and the 0.92 result depends on COLMAP's initia","The recipe generalizes as a template for under-constrained 3D fitting generally: a cheap order-only signal (rankings) stabilizes coarse shape without scale commitment, and a frozen feature metric fixes absolute position. Porting the same two-stage design into NeRF-based or SDF-based sparse-view pipelines is a direct and testable extension.","The accuracy ceiling is set by the frozen feature extractor's invariance: on scenes whose texture statistics differ from multi-view-stereo training data, such as reflective, transparent, or heavily textureless surfaces, the feature loss may silently misalign with no learned recourse. A domain-transfer experiment swapping the feature network would quantify how much of the reported Chamfer distance ","With 14-minute training and 4 GB peak memory, the method sits at the threshold of laptop-GPU and on-device scanning; the paper notes the efficiency but does not explore that application, which is where the practical impact would be largest."],"forward_implications":["Sparse-view accuracy and speed stop being a trade-off: the reported DTU three-view mean Chamfer Distance of 0.92 beats every method compared while per-scene training drops to about 14 minutes on a single RTX 3090 using 4 GB of memory.","The 60x to 200x training speedup relative to per-scene optimizers such as NeuSurf and to pretrained generalizers such as UFORecon would let reconstruction run in interactive or on-site loops instead of as a scheduled job.","Because the identical hyperparameters are reported on DTU large-overlap, DTU little-overlap, and BlendedMVS, the constraints are not tuned to a single benchmark, which the paper offers as evidence of generalization.","Reconstruction quality scales smoothly with view count (reported mean CD 0.92 at 3 views, 0.85 at 6, 0.79 at 9, 0.61 with all views), so the method degrades gracefully as views are removed rather than collapsing.","The same pipeline improves few-shot novel view synthesis as a byproduct (reported PSNR 21.80, SSIM 0.904, LPIPS 0.077 on DTU with three input views), coupling geometry and rendering quality in one optimization."],"supporting_citations":[{"why":"2D Gaussian Splatting: supplies the flattened-ellipse Gaussian representation, ray-splat depth rendering, and the depth-distortion and normal-consistency losses that FatesGS is built on.","marker":"Huang et al. 2024a"},{"why":"3D Gaussian Splatting: provides the splatting-rasterization pipeline, adaptive density control, and color supervision that the framework inherits.","marker":"Kerbl et al. 2023"},{"why":"Marigold: the monocular depth estimation model whose local depth rankings drive the intra-view ranking and smoothing losses.","marker":"Ke et al. 2024"},{"why":"Vis-MVSNet: the frozen multi-scale feature extraction network whose cross-view features are aligned by the multi-view feature loss.","marker":"Zhang et al. 2020"},{"why":"NeuSurf: the per-scene-optimization state of the art whose DTU three-view evaluation settings the paper adopts and whose 14-hour runtime it improves to 14 minutes.","marker":"Huang et al. 2024b"},{"why":"UFORecon: the generalizable sparse-view reconstruction baseline that requires about ten days of pretraining and against which FatesGS claims better accuracy without pretraining.","marker":"Na et al. 2024"},{"why":"COLMAP: computes the camera poses and the initial point cloud from which the 2D Gaussians are seeded.","marker":"Schonberger and Frahm 2016"},{"why":"MonoSDF: supplies the scale-shifted absolute-depth supervision that the ablation replaces with the ranking loss, and shows to be inferior on sparse views.","marker":"Yu et al. 2022"}],"fun_headline_variants":["3-view 3D surfaces in 14 minutes, beating dense baselines","Depth ranking + feature alignment fix sparse-view Gaussian splatting","No pretraining: 14-minute meshes from 3 views top benchmarks","Sparse-view GS: 60-200x faster, best Chamfer distance on DTU"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the occlusion mask used by the multi-view feature loss — computed from the very rendered depth being optimized — stays accurate enough during early training that feature correspondences are not dominated by occluded or misprojected pixels; if the initial Gaussians are poorly localized under sparse views, the depth-visibility-feature loop can lock onto wrong geometry, and the frozen feature network must also remain reliable on the target scene.","fun_headline_variants_meta":{"raw":{"variants":["3-view 3D surfaces in 14 minutes, beating dense baselines","Depth ranking + feature alignment fix sparse-view Gaussian splatting","No pretraining: 14-minute meshes from 3 views top benchmarks","Sparse-view GS: 60-200x faster, best Chamfer distance on DTU"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000259,"raw_usage":{"total_tokens":1611,"prompt_tokens":997,"completion_tokens":614,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":613,"completion_tokens_details":{"reasoning_tokens":530}},"tokens_in":613,"tokens_out":614,"duration_ms":6010,"temperature":1.0,"reasoning_tokens":530,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:27:53.654462+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Two concrete checks would settle the central claim. First, initialize the Gaussians from a deliberately corrupted point cloud (translated or scaled away from the true surface) on a DTU three-view scene and measure the resulting Chamfer Distance: recovery would show the consistency losses genuinely guide geometry, while a large CD jump would show the accuracy depends on COLMAP's initialization rather than the proposed mechanism. Second, replace the frozen feature extractor (Vis-MVSNet) with a different pretrained network and re-run the DTU large-overlap benchmark: a substantial change in mean CD would show the reported 0.92 is carried by the specific feature network's invariance rather than by the consistency formulation itself.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"3D Gaussian Splatting: provides the splatting-rasterization pipeline, adaptive density control, and color supervision that the framework inherits."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Vis-MVSNet: the frozen multi-scale feature extraction network whose cross-view features are aligned by the multi-view feature loss."}],"review_version":1}