{"id":"35dfabfe-44a7-47ba-9639-e84907503a2a","arxiv_id":"2506.09378","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"UniForward is a feed-forward model that reconstructs 3D Gaussian scenes with embedded semantic features from just two unposed images, enabling real-time novel-view rendering and open-vocabulary segmentation.","lead":"UniForward turns two unposed photos of an indoor scene into a 3D model with per-point semantic labels in a single network pass, without needing depth or camera poses at inference. This could give robots and AR systems an instant, view-consistent semantic map of their surroundings.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA claim rests on an uncontrolled LSM comparison: official weights are used despite the paper's own fairness statement, so the unified-reconstruction advantage over LSM may be inflated.","rationale":"The reader's weakest assumption was scale ambiguity. That is a real limitation for metric downstream tasks such as robot navigation and AR, but it is explicitly acknowledged in Section 3.3 ('we normalize the distance between input views and reconstruct the scaled scenes') and does not invalidate the core reconstruction result: many pose-free methods operate up to scale, and the reported view-synthesis and segmentation metrics can still be meaningful in a normalized frame. My concern targets the 'state-of-the-art' component of the central claim itself. The paper's own text creates an unresolved inconsistency: Section 4.1 promises that all comparison methods were retrained under the same setting, but Supplementary A.3 says LSM was not retrained and was used directly from official weights trained on ScanNet and ScanNet++. If LSM's official training included the test split, the comparison is contaminated; if not, it is still not a same-setting comparison because resolution, data mix, and compute are uncontrolled. The selected-scene protocol in Table 2 further weakens the SOTA evidence. Therefore I do not think the scale ambiguity should be the primary gate for acceptance; the fair and reproducible comparison with LSM is more load-bearing. My final verdict remains conditional: the paper should resolve the LSM comparison and provide enough artifacts to reproduce the headline numbers. This is the same verdict category as the reader, hence UNCHANGED, but for a different reason.","tokens_in":12638,"tokens_out":9071,"duration_ms":114600,"concrete_test":"Retrain LSM on the exact ScanNet++ training split used by UniForward (same scenes, 256x256 inputs, same training iterations or matched compute budget), holding out the 49 validation scenes for testing only; also verify that none of those 49 scenes appear in LSM's official pretraining. Then recompute Table 1. If LSM's PSNR or mIoU rises materially above 16.005/0.320, or if any test scene is found in LSM's pretraining, the SOTA gap over the unified baseline is not established and the central claim must be qualified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central SOTA claim in Tables 1 and 4 depends on a fair comparison with LSM, the only prior feed-forward method that also does unified scene and semantic reconstruction. However, Section 4.1 states 'For a fair comparison, we retrained all the comparison methods under the same setting,' while Supplementary A.3 says LSM is used directly with official pretrained weights because it was 'trained on ScanNet and ScanNet++, which include our datasets.' These two statements conflict. If LSM's official training split overlaps the 49 ScanNet++ validation scenes used here for testing, or if LSM was trained at a different resolution, with different data, or with more compute, then the reported LSM numbers (16.005 PSNR, 0.320 mIoU) are not a valid same-setting baseline. The claim that UniForward achieves state-of-the-art unified reconstruction would then rest on beating a possibly already-seen, differently-trained baseline. The per-scene comparison in Table 2 is also on a selected subset (scenes where SfM succeeded for the baselines), so the 28.097 PSNR is not directly comparable to the full test set in Table 1. This directly threatens the headline claim, more so than the scale ambiguity, which is disclosed in Section 3.3 and is common in pose-free reconstruction.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"UniForward is a feed-forward 3D Gaussian Splatting model that takes two unposed, uncalibrated sparse-view images and, in a single forward pass, reconstructs a 3D scene together with a semantic field. The Gaussian centers are regressed directly in a shared canonical space, while appearance and semantic features are predicted by a separate attribute branch. Training uses a photometric loss, a pose loss as an auxiliary geometry signal, and distillation from a pretrained 2D semantic model (LSeg), without ground-truth depth. To stabilize training, the paper introduces a loss-guided view sampler that progressively increases the angular separation between context views. Experiments on ScanNet++ and ScanNet report improved novel-view synthesis and novel-view segmentation over feed-forward baselines (pixelSplat, MVSplat, LSM, Splatt3R, NoPoSplat) and per-scene optimized methods (Feature 3DGS, DFFs).","tokens_in":12924,"tokens_out":5576,"duration_ms":61792,"significance":"If the empirical claims hold, UniForward is a meaningful contribution: it is the first feed-forward 3DGS model I am aware of that jointly predicts geometry, appearance, and open-vocabulary semantic features from two unposed, uncalibrated images without depth supervision, while running in about 0.1 s at inference. The ablations in Table 3 support the loss-guided sampler, the pose head, the semantic head, and the shortcut connection, and the novel-view segmentation comparison against 2D LSeg is a genuine test of 3D-consistent semantics rather than a self-supervised circularity. The scale ambiguity is disclosed in Section 3.3 and is common to pose-free reconstruction, though it is not fully addressed in the downstream-motivation claims. The main risk to the headline state-of-the-art claim is the inconsistency in the LSM baseline protocol, which must be resolved before the quantitative ranking can be accepted.","major_comments":[{"comment":"Section 4.1 states 'For a fair comparison, we retrained all the comparison methods under the same setting,' but Supplementary A.3 says that for LSM the authors 'directly use the official pre-trained weights, as the model is trained on ScanNet and ScanNet++, which include our datasets.' These two statements conflict. If LSM's official training included the ScanNet++ validation scenes used here for testing, or if LSM was trained at a different resolution, with different data, or with more compute, then the reported LSM numbers (16.005 PSNR, 0.320 mIoU in Table 1; 19.083 PSNR, 0.341 mIoU in Table 4) are not a valid same-setting baseline. Because LSM is the only prior feed-forward method that also performs unified scene and semantic reconstruction, the central claim of state-of-the-art unified reconstruction rests on this comparison. Please retrain LSM under the same training split and protocol, or provide rigorous evidence that the official weights were not trained on the evaluation scenes and that evaluation conditions match.","section":"Section 4.1 and Supplementary A.3, Tables 1 and 4"},{"comment":"Table 2 compares UniForward with per-scene optimized methods (Feature 3DGS, DFFs) only on scenes where SfM succeeded for those baselines, while Table 1 reports results on the full 49-scene ScanNet++ validation set. The 28.097 PSNR reported for Ours in Table 2 is therefore not directly comparable to the 26.147 PSNR in Table 1, and the claim of outperforming per-scene optimized methods is established only on a favorable subset. Please report the number of scenes used in Table 2, results of all methods on that same subset, and, where feasible, results on the full test set for the per-scene baselines.","section":"Section 4.3, Table 2"},{"comment":"The paper correctly notes in Section 3.3 that the distance between input views is normalized to resolve scale ambiguity and that the model reconstructs scaled scenes, but at inference the model receives only images and cannot recover metric scale. This means all reconstructed geometry and the semantic field live in a scale-ambiguous normalized frame. The introduction and conclusion motivate the method for robot navigation and AR/VR, which require metric or at least consistently scaled geometry, yet the Limitations paragraph in Section 5 does not mention this limitation and no experiment measures scale error. Please add an explicit limitation statement and, if possible, an evaluation of scale recovery or an external alignment step.","section":"Section 3.3 and Section 5"}],"minor_comments":[{"comment":"All quantitative results are reported from a single run without error bars or multiple seeds. Given the modest margins over NoPoSplat in Table 1 (PSNR 26.147 vs 24.755), reporting variance or repeated-run statistics would strengthen the comparison.","section":"Tables 1-4"},{"comment":"The LSeg baseline setup should be clarified: in qualitative figures LSeg is applied to ground-truth novel views (labeled 'w GT'), while in Table 1 the LSeg row lists N/A for synthesis and only segmentation numbers. Please state explicitly whether the Table 1 LSeg segmentation is evaluated on ground-truth novel views, rendered images, or input views.","section":"Table 1 and Figure 4"},{"comment":"The sentence 'as the model is trained on ScanNet and ScanNet++, which include our datasets' is ambiguous: it could mean LSM was trained on the same training splits or on the same entire datasets including the validation scenes used for testing. Please clarify the exact overlap with the evaluation split.","section":"Supplementary A.3"},{"comment":"The semantic head predicts an 'N-dimensional' feature, but N is never specified. Please state the feature dimension used in the experiments.","section":"Section 3.2.2 and Equation (3)"},{"comment":"The loss-guided view sampler depends on a sliding window over the pose loss list L and a schedule function Schedule(i), but neither the window size nor the schedule is defined in the text. Please provide these details for reproducibility.","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The main risk is the LSM baseline protocol. If the authors can resolve the fairness conflict by retraining LSM under the same setting, or convincingly show that the official LSM weights were not trained on the test scenes and are evaluated under matching conditions, the central contribution is defensible. The per-scene comparison and scale-ambiguity issues are also fixable within revision. I would not reject on the current evidence, but the SOTA claim is not yet established."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: this is a competent, useful paper that does something genuinely new — feed-forward joint geometry, appearance, and semantic reconstruction from two unposed images — but its headline SOTA claim rests on an LSM comparison the paper's own text contradicts.\n\nWhat it does well: the dual-branch decoder is a sensible answer to the problem that regressing geometry, appearance, and semantics together hurts accuracy. The loss-guided view sampler is a clean trick to avoid needing depth masks, and the ablations show each piece earns its place. The semantic distillation works: on ScanNet the model clearly beats 2D LSeg at novel-view segmentation, and on ScanNet++ it gets a small edge even though LSeg receives the ground-truth target view. That is the right kind of evidence that the 3D field adds consistency. The method is practical — 0.1s inference from two images, no per-scene optimization.\n\nThe soft spots. The big one is the LSM baseline. Section 4.1 says 'we retrained all the comparison methods under the same setting,' but Supplementary A.3 says LSM is used directly with official pretrained weights. If LSM was trained on ScanNet++ at a different resolution or with more compute, the 16.005 PSNR / 0.320 mIoU numbers are not a fair basis for SOTA. This is a clear manuscript contradiction, not a nitpick. Second, there are no error bars or repeated runs anywhere; a 10 dB lead over your closest unified baseline needs more than single-run numbers. Third, Table 2 compares against per-scene methods only on scenes where SfM succeeds, but the paper does not say how many scenes that is, so the 28.097 PSNR is not comparable to Table 1. Fourth, scale ambiguity is mentioned in the method but never listed as a limitation; a robot using this for navigation would need metric scale, and that recovery step is missing.\n\nNone of these kill the core contribution. The architecture and training recipe are sound, and the ablation structure is clean. It is a paper a serious reviewer should engage with, but the authors need to either retrain LSM in their setting or soften the SOTA claim.\n\nRecommendation: send to peer review, conditional on the authors resolving the LSM comparison and adding variance estimates. Worth a reading-group slot too — the fairness question is instructive.","headline":"Useful integrated feed-forward splatting with strong results, but the SOTA claim against LSM rests on a comparison the paper itself contradicts.","tokens_in":13436,"tokens_out":5015,"would_cite":true,"duration_ms":51394,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"UniForward reconstructs a 3D scene and its semantic field from two unposed, uncalibrated images in a single feed-forward pass that needs no depth labels during training.","keywords":["3D Gaussian Splatting","feed-forward reconstruction","semantic field","sparse-view reconstruction","pose-free reconstruction","open-vocabulary segmentation","semantic distillation","novel view synthesis"],"falsifier":"Feed UniForward two images of a room whose camera baseline and metric depth are known, then compare the reconstructed Gaussian centers to a metric laser scan without any scale alignment; if the cloud matches only after applying a constant scale factor, the scale-ambiguity premise is confirmed, and if it matches metrically without alignment, the paper's normalization claim is wrong.","tokens_in":12417,"feed_emoji":"🏷️","tokens_out":8916,"duration_ms":86170,"temperature":0.7,"pith_summary":"The paper sets out to show that 3D geometry and semantics can be recovered together, in real time, from just two sparse-view images with no camera poses, no intrinsics, and no ground-truth depth. Embedding semantic features into the 3D Gaussians gives a single representation that can render both novel-view images and view-consistent semantic maps. The authors argue that decoupling geometry from appearance and semantics, and training with a photometric loss plus semantic distillation, is enough to reach state-of-the-art results on both novel view synthesis and novel view segmentation. If the claim holds, embodied agents and AR systems could obtain a labeled 3D understanding of a new scene in about a tenth of a second without per-scene optimization.","feed_headline":"Two unposed photos in, a labeled 3D scene out, in 0.1s","feed_subtitle":"A feed-forward network reconstructs geometry and semantics together, with no camera poses, intrinsics, or depth labels required.","key_machinery":"The central object is a 3D Gaussian with an extra semantic feature vector: each Gaussian carries position, covariance, opacity, color, and an $N$-dimensional semantic embedding, so the same splatting operation renders images and semantic feature maps. The argument runs through a dual-branch decoupled decoder with separately supervised heads for geometry, appearance, and semantics, plus a loss-guided view sampler that starts training on close views with small baselines and gradually increases view angle once pose loss stabilizes. This sampler is what removes the need for ground-truth depth or masks, and the decoupled design is what prevents joint regression of geometry and semantics from collapsing into the suboptimal results the paper attributes to earlier single-branch feed-forward models.","core_discovery":"The central claim is that a feed-forward network can jointly predict, from only two unposed and uncalibrated images, a set of 3D Gaussians whose per-Gaussian semantic features constitute a 3D semantic field. Rendering those features from arbitrary views and decoding them with a pre-trained open-vocabulary decoder produces dense, view-consistent segmentation masks. The paper shows that separating the prediction into a geometry branch and an attribute branch, supervised by photometric and semantic-distillation losses, lets the model train without depth labels or masks and outperform prior feed-forward and per-scene baselines on both novel view synthesis and novel view segmentation.","pith_inferences":["If the scale ambiguity were resolved by injecting any known metric length (an object of known size, a stereo baseline, or a monocular metric-depth head), the same architecture could slot directly into navigation or AR pipelines without retraining.","Since the semantic head already accounts for a large segmentation gain in the paper's ablations, swapping in a stronger dense open-vocabulary feature extractor should directly raise novel-view segmentation quality.","The loss-guided view sampler is effectively a curriculum on angular baseline; a natural extension is to replace its angle threshold with a learned overlap predictor so that it handles irregular video framerates from handheld or egocentric capture.","Reporting metric-scale errors after aligning predicted and ground-truth scenes would make explicit how much of the performance depends on the scale normalization."],"forward_implications":["A single forward pass (about 0.1 seconds on one GPU) yields both a renderable 3D scene and a 3D semantic field, so novel-view segmentation becomes available at interactive rates without per-scene optimization.","Because the training objective needs no depth or masks, the same pipeline can in principle be trained on large collections of posed video frames rather than only depth-annotated datasets.","The reconstructed semantic field is view-consistent and open-vocabulary: a text query can segment the same object consistently from different novel viewpoints.","Reconstruction is limited to regions visible in the two input views, and the fidelity of the semantic field is capped by the pre-trained 2D semantic model used for distillation.","All geometry is recovered up to an unknown scale, so metric distances are not available from the model output alone."],"supporting_citations":[{"why":"supplies the differentiable Gaussian splatting representation used to render novel-view images and feature maps.","marker":"[8]"},{"why":"introduces the pose-free paradigm of directly regressing 3D points from unposed image pairs in a shared coordinate frame, which the geometry branch follows.","marker":"[24]"},{"why":"provides the pretrained backbone whose weights initialize feature extraction for the decoder branches.","marker":"[11]"},{"why":"is the closest prior feed-forward model joining geometry, appearance, and semantics; used both as the main comparison and as evidence that direct joint regression is suboptimal.","marker":"[6]"},{"why":"supplies the pre-trained 2D semantic features distilled into the 3D field and the decoder that turns rendered features into open-vocabulary segmentation masks.","marker":"[12]"},{"why":"shows per-Gaussian semantic features in a per-scene optimized 3D Gaussian Splatting setting and serves as a per-scene baseline.","marker":"[32]"},{"why":"is a pose-free feed-forward Gaussian Splatting baseline trained without depth that the method compares against and improves upon.","marker":"[27]"},{"why":"provides the indoor-scene dataset with posed frames and semantic annotations used for training and evaluation.","marker":"[28]"}],"fun_headline_variants":["Two unposed photos yield unified 3D scene and semantics","Feed-forward splatting: no poses, no depth, real-time semantics","Unified 3D scene and semantic field from sparse views","Joint 3D scene and open-vocabulary semantics in one pass","Real-time 3D reconstruction plus semantics from unposed images"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that reconstructing a scene up to an unknown scale from two images, after training with ground-truth poses and normalized baseline distances, is enough for the downstream use case, because at inference the model never sees metric distances.","fun_headline_variants_meta":{"raw":{"variants":["Two unposed photos yield unified 3D scene and semantics","Feed-forward splatting: no poses, no depth, real-time semantics","Unified 3D scene and semantic field from sparse views","Joint 3D scene and open-vocabulary semantics in one pass","Real-time 3D reconstruction plus semantics from unposed images"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00037,"raw_usage":{"total_tokens":2006,"prompt_tokens":995,"completion_tokens":1011,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":611,"completion_tokens_details":{"reasoning_tokens":920}},"tokens_in":611,"tokens_out":1011,"duration_ms":10230,"temperature":1.0,"reasoning_tokens":920,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:50:26.973744+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Feed UniForward two images of a room whose camera baseline and metric depth are known, then compare the reconstructed Gaussian centers to a metric laser scan without any scale alignment; if the cloud matches only after applying a constant scale factor, the scale-ambiguity premise is confirmed, and if it matches metrically without alignment, the paper's normalization claim is wrong.","supporting_citations":[{"cited_title":"Kerbl, G","cited_arxiv_id":null,"evidence_quote":"supplies the differentiable Gaussian splatting representation used to render novel-view images and feature maps."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"introduces the pose-free paradigm of directly regressing 3D points from unposed image pairs in a shared coordinate frame, which the geometry branch follows."},{"cited_title":"Leroy, Y","cited_arxiv_id":null,"evidence_quote":"provides the pretrained backbone whose weights initialize feature extraction for the decoder branches."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"is the closest prior feed-forward model joining geometry, appearance, and semantics; used both as the main comparison and as evidence that direct joint regression is suboptimal."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"shows per-Gaussian semantic features in a per-scene optimized 3D Gaussian Splatting setting and serves as a per-scene baseline."},{"cited_title":"Yeshwanth, Y .-C","cited_arxiv_id":null,"evidence_quote":"provides the indoor-scene dataset with posed frames and semantic annotations used for training and evaluation."}],"review_version":1}