{"id":"f7d33d2e-8bfc-4337-8cc8-2b8b7dc942c6","arxiv_id":"2608.03851","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"LiteMVS improves efficient multi-view stereo depth estimation by injecting semantic descriptors, MoE cost aggregation, and pseudo-labels from monocular foundation models.","lead":"LiteMVS is a lightweight multi-view depth model that adds semantic features, Mixture-of-Experts aggregation, and foundation-model distillation to a plane-sweep stereo pipeline. It reports lower depth errors and faster or comparable runtime than several prior indoor MVS methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Claimed gains over SimpleRecon/DoubleTake may stem from training on ScanNet++ while published baselines were not retrained on that data; protocol is not controlled.","rationale":"The reader's conditional verdict already centers on evaluation fairness. My concern is a specific, underappreciated confound within that category: LiteMVS is trained on ScanNet++ while the baselines appear to be cited or reused from their original publications, which were trained on ScanNetv2 (and perhaps other data) but not necessarily on the same ScanNet++-augmented split. This is not an internal inconsistency in LiteMVS, but it directly threatens the central quantitative claim that the architecture—semantic injection, MoE aggregation, and distillation—is what produces the improvement. The paper provides no code or supplementary material to check this, and the internal ablations do not rule it out. I do not recommend rejection because the method is plausible and the components have internal support; however, the stated SOTA margins cannot be accepted until the training-data variable is controlled. Since the reader already called for conditional acceptance, my read does not change the verdict.","tokens_in":10370,"tokens_out":5923,"duration_ms":67255,"concrete_test":"Train SimpleRecon and DoubleTake from their official code on the exact ScanNetv2+ScanNet++ training split used by LiteMVS, and evaluate all methods with identical keyframe selection, GT mesh mask, and depth/eval scripts. If either baseline matches or exceeds LiteMVS on Table 1/2 metrics, the architecture-specific SOTA claim is not supported. As a complementary check, train LiteMVS on ScanNetv2 only and compare to published baseline numbers.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central comparison in Tables 1 and 2 is not controlled for training data. Section 4 says LiteMVS is trained on 'a combination of ScanNetv2 and ScanNet++', but Table 1's caption says prior results were 'taken from [12], or evaluated for each method using their keyframes' with no statement that SimpleRecon or DoubleTake were retrained on the same ScanNet++-augmented split. ScanNet++ is a higher-fidelity, larger dataset; extra training data alone could explain the margins (e.g., Abs Rel 0.0311 vs 0.0369 for DoubleTake; F-score 0.715 vs 0.714). The internal ablations in Table 3 show each component helps, but they compare variants of LiteMVS on its own training set and do not resolve the cross-method training-data confound. If the baselines rely on their original ScanNet-only models, the paper's 'lower errors' claim attributes to architecture what may be a data advantage.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces LiteMVS, a lightweight multi-view stereo depth estimator that augments a plane-sweep cost volume with semantic descriptors from MobileSAM, aggregates cost features with a Mixture-of-Experts (MoE) MLP, and distills relative-depth and surface-normal pseudo-labels from Depth Anything V2 and StableNormal during training. The authors report lower depth errors than SimpleRecon and DoubleTake on ScanNetv2 and 7-Scenes, improved 3D reconstruction F-score, and competitive per-frame latency, as well as downstream manipulation results on LIBERO and RoboTwin 2.0. The central claim is that combining monocular semantic/structural priors with efficient MVS improves the accuracy-efficiency trade-off without adding inference cost.","tokens_in":10675,"tokens_out":5640,"duration_ms":56164,"significance":"If the results hold, the paper offers a practical recipe for injecting foundation-model priors into lightweight MVS while keeping inference cost unchanged, since the distilled pseudo-labels are used only as training supervision. The architecture is clearly motivated, and the evaluation spans depth, reconstruction, and embodied downstream tasks. I agree with the reader that there is no circularity: pseudo-labels are training signals, and all reported metrics are against ground-truth depth/meshes. The benchmark claims are falsifiable, and the ablations that are actually run generally support the stated components. However, the headline comparative claims are currently conditional on an uncontrolled training-data protocol and on a missing distillation ablation, so the significance is not yet established at the level claimed.","major_comments":[{"comment":"The headline comparisons are confounded by training data. Section 4 states LiteMVS is trained on 'a combination of ScanNetv2 and ScanNet++', while Table 1's caption says prior results were 'taken from [12], or evaluated for each method using their keyframes'; there is no statement that SimpleRecon or DoubleTake were retrained on the same ScanNet++-augmented split. Since ScanNet++ is larger and higher-fidelity, the reported margins (e.g., Abs Rel 0.0311 vs 0.0369 for DoubleTake; F-score 0.715 vs 0.714) may reflect a data advantage rather than an architectural one. Please retrain all baselines on the identical data split, or report a ScanNet-only comparison, and state the full evaluation protocol. Reporting multiple-seed error bars would also help for the near-tie F-score difference.","section":"Section 4, Tables 1–2"},{"comment":"The claimed contribution of pseudo-label distillation is not ablated. Section 5.3 is titled 'Effect of Vision Foundation Model Distillation', but its text and Table 4 vary the number of MoE experts; the Table 3 row 'Ours w/oLM onoKD' is malformed and unexplained. A clean experiment removing L_rel and L_norm (Eqs. 3–5) is required to support the paper's third contribution and the claim that distillation adds 'no extra inference cost'. Please fix the row label and add the missing distillation ablation.","section":"Section 5.3, Table 3"},{"comment":"The primary supervision term L_simple is not defined in the paper; the text defers it to a supplementary that is not included. The values of lambda_rel, lambda_norm, and lambda_grad are also omitted. Without the full SimpleRecon loss and hyperparameters, Eq. (5) is not reproducible, and the contribution of the new distillation losses cannot be quantitatively assessed. Please include the full loss formulation and all hyperparameter values in the main text or an appendix.","section":"Section 3.5, Eq. (5)"},{"comment":"Fig. 1(b) claims LiteMVS achieves the 'fastest inference speed', but Table 2 reports update latency 72 ms for SimpleRecon vs 75 ms for LiteMVS. This is a factual inconsistency in a headline claim. Please correct the figure/caption to say 'competitive' latency, or revise the comparison to be consistent with the reported numbers.","section":"Section 4.2, Table 2, Fig. 1"}],"minor_comments":[{"comment":"Typos: 'Sematic' should be 'Semantic'; 'w/oLM onoKD' is likely intended as 'w/o KD' (without knowledge distillation).","section":"Table 3"},{"comment":"Both sections contain nearly identical sentences: 'We further ablate the proposed MoE-based cost aggregation module by varying the number of experts.' This duplication should be removed, and the distillation ablation placed in Section 5.3.","section":"Sections 5.3 and 5.4"},{"comment":"The citation '[9, 25? ? ? ?]' contains placeholder question marks and should be fixed. Also, 'Simplerecon' in Table 1 should be 'SimpleRecon'.","section":"Introduction, references"},{"comment":"The abbreviations 'Comp', 'Acc', and 'Recall' are not defined in the caption. Please define them or refer to the TransformerFusion protocol explicitly.","section":"Table 2 caption"},{"comment":"'Poindcloud' should be 'Point cloud'; the caption and text should avoid overstating the speed comparison (see major comment 4).","section":"Figure 1"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the main risk is the training-data confound in Tables 1–2. If the authors cannot retrain SimpleRecon and DoubleTake on the same ScanNet++-augmented split, the SOTA claim should be explicitly downgraded to an architecture-plus-data result. The missing supplementary and duplicated sections also suggest the manuscript is not yet in camera-ready shape, but these are fixable within revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: LiteMVS is a solid engineering paper with an evaluation confound that weakens the headline comparison. The architecture—semantic descriptors from MobileSAM in the cost volume, MoE aggregation, and pseudo-label distillation from Depth Anything/StableNormal—is coherent and the internal ablations show each piece helps. But the main comparison against SimpleRecon and DoubleTake is not controlled for training data, and that's a real problem.\n\nSpecifically, LiteMVS is trained on ScanNetv2 plus ScanNet++. The table caption says previous results were taken from [12] or evaluated using their keyframes—nothing about retraining the baselines on the same extra data. ScanNet++ is a larger, higher-fidelity dataset; extra training data alone could explain the margin. The stress-test hits this squarely, and I don't see a rebuttal in the paper.\n\nOther soft spots: the distillation ablation is missing. Section 5.3 is titled 'Effect of Vision Foundation Model Distillation' but the content is an MoE expert-count ablation (also duplicated in Section 5.4). Fig. 1 claims fastest inference, but Table 2 shows SimpleRecon at 72ms vs 75ms for LiteMVS. No error bars, no code, and several details deferred to a supplementary that isn't available.\n\nWhat it does well: the method is simple to explain and the design choices are sensible. The 7-Scenes cross-dataset evaluation is a good idea, and the downstream manipulation experiment adds context even if not conclusive. The paper is readable and the ablation tables are internally consistent.\n\nWho this is for: people working on efficient indoor MVS or using foundation models as teachers. The architecture is worth knowing, but treat the SOTA numbers with caution until the baselines are retrained or the comparison is otherwise controlled. I'd send it to peer review because the idea and experiments deserve scrutiny, and a good reviewer can force the authors to fix the comparison. It needs that revision before I'd trust the headline.","headline":"LiteMVS is a sensible architecture with a real training-data confound in its SOTA comparison; worth review, not yet trustworthy.","tokens_in":11107,"tokens_out":2444,"would_cite":false,"duration_ms":26194,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LiteMVS claims that injecting semantic descriptors and foundation-model pseudo-labels into a plane-sweep cost volume yields more accurate depth and 3D reconstruction at roughly 75 ms per frame.","keywords":["multi-view stereo","depth estimation","plane-sweep cost volume","Mixture-of-Experts","knowledge distillation","semantic priors","3D reconstruction","embodied manipulation"],"falsifier":"Re-run SimpleRecon, DoubleTake, and LiteMVS under one shared keyframe schedule, mesh mask, and GPU, then recompute the depth and reconstruction tables; if the re-run baselines match or exceed LiteMVS's reported numbers, the central accuracy claim is an artifact of the comparison protocol.","tokens_in":10339,"feed_emoji":"🏗️","tokens_out":9376,"duration_ms":93365,"temperature":0.7,"pith_summary":"This paper presents LiteMVS, a multi-view stereo method that estimates depth from multiple posed images while keeping computation low. Its central claim is that lightweight plane-sweep matching, which often fails in textureless or repetitive regions, can be made accurate by enriching the cost volume with semantic descriptors and by training on pseudo-labels distilled from monocular foundation models. On ScanNetv2 and 7-Scenes, the paper reports lower depth errors than SimpleRecon and DoubleTake, with reconstruction quality comparable to or better than heavier volumetric methods and per-frame update latency around 75 ms. The same representation also transfers to a robotic-manipulation setup, approaching a much larger 3D-aware backbone at roughly one-third of its inference cost. The paper's own Limitations section concedes that reflective surfaces, transparent objects, and low-texture regions remain difficult.","feed_headline":"LiteMVS lowers indoor depth error, keeps 75 ms latency","feed_subtitle":"Semantic cues and foundation-model priors in the cost volume beat heavier baselines on ScanNet and 7-Scenes.","key_machinery":"The load-bearing object is the semantic-augmented 4D cost volume: per-pixel semantic descriptors from a lightweight segmentation encoder (MobileSAM lineage) are broadcast along the depth dimension and concatenated with warped multi-view features and geometric metadata, so every depth hypothesis carries both matching evidence and monocular boundary/structure cues. A Mixture-of-Experts MLP then aggregates the volume, with a gating network letting experts specialize across depth regimes, and pseudo-label losses transfer foundation-model priors at no extra inference cost.","core_discovery":"LiteMVS claims that an efficient multi-view stereo network does not have to choose between geometric consistency and monocular priors. It augments a standard plane-sweep feature volume with semantic descriptors from a lightweight segmentation encoder, replaces the shared cost-aggregation MLP with a Mixture-of-Experts (MoE) formulation so different experts specialize across depth ranges, and adds output-level distillation: relative-depth pseudo-labels from Depth Anything V2 and surface-normal pseudo-labels from StableNormal supervise training without changing inference. The combined system yields sharper boundaries and lower depth error than SimpleRecon and DoubleTake on ScanNetv2 and 7-Scene","pith_inferences":["The authors' own limitation list — reflective surfaces, transparent objects, low-texture regions — maps cleanly onto the failure modes that appearance matching and monocular priors share; if semantic cues are doing the claimed work, per-region error analysis should show the largest gains exactly there. That is a testable prediction the paper does not make.","The MoE saturation at three experts suggests the mechanism may be interpreted as a soft split of the depth range rather than a capacity increase; plotting gate weights against true depth would confirm the specialization directly.","The same pseudo-label distillation recipe could extend to other geometry outputs, such as optical flow or 4D tracking, since the paper demonstrates it only for static-scene depth and normals.","Because Table 1 mixes previously reported numbers with per-method keyframes, the cleanest way to test the accuracy claim is a single-keyframe re-run of all baselines on identical hardware."],"forward_implications":["Lightweight 2D-CNN-based MVS can reach reconstruction quality previously associated with heavier volumetric refinement, at real-time update rates.","Distilling relative depth and surface-normal pseudo-labels from foundation models is a low-cost way to import monocular priors into task-specific networks.","Letting multiple expert MLPs specialize across depth ranges improves accuracy over a single shared MLP, with gains saturating around three experts.","Geometry-aware MVS features can serve as efficient visual targets for embodied manipulation policies, matching heavier 3D backbones on LIBERO at about one-third the inference cost.","Using segmentation cues as auxiliary guidance rather than a trained head keeps the semantic prior optional at inference time."],"supporting_citations":[{"why":"Supplies the base lightweight plane-sweep architecture, geometric metadata, and primary multi-view supervision loss that LiteMVS builds on and must beat.","marker":"[20]"},{"why":"Geometry-guided baseline whose depth and reconstruction numbers LiteMVS is compared against; also a source of the geometric-metadata idea.","marker":"[21]"},{"why":"Provides the relative-depth pseudo-labels distilled into LiteMVS via a scale-invariant loss.","marker":"[29]"},{"why":"Provides the surface-normal pseudo-labels used for SSIM-plus-gradient normal distillation.","marker":"[31]"},{"why":"The lightweight segmentation encoder that produces the semantic descriptors injected into the cost volume.","marker":"[34]"},{"why":"Defines the keyframe/test-split protocol and the per-frame update-latency metric used in the comparisons.","marker":"[6]"},{"why":"Supplies the mesh-mask evaluation protocol used to measure 3D reconstruction accuracy.","marker":"[2]"},{"why":"ScanNetv2, the primary training and evaluation dataset for the reported depth and reconstruction numbers.","marker":"[5]"},{"why":"7-Scenes, the cross-dataset benchmark used to show generalization without fine-tuning.","marker":"[22]"},{"why":"Source of some previous-method numbers in the depth comparison table, so it underpins the claimed margins over baselines.","marker":"[12]"}],"fun_headline_variants":["LiteMVS fuses semantics and geometry for sharper depth","MoE + foundation priors lift MVS accuracy, keep 75 ms","Semantic cost volumes beat heavier MVS baselines efficiently","Distilled priors sharpen depth in texture-poor scenes"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that all methods were compared under the same protocol—same keyframes, mesh masks, hardware, and input views—since Table 1 reports some previous results from [12] or from each method's own keyframes; if the protocol is not identical, the reported accuracy advantages are not attributable to the architecture.","fun_headline_variants_meta":{"raw":{"variants":["LiteMVS fuses semantics and geometry for sharper depth","MoE + foundation priors lift MVS accuracy, keep 75 ms","Semantic cost volumes beat heavier MVS baselines efficiently","Distilled priors sharpen depth in texture-poor scenes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000258,"raw_usage":{"total_tokens":1458,"prompt_tokens":821,"completion_tokens":637,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":565,"completion_tokens_details":{"reasoning_tokens":577}},"tokens_in":565,"tokens_out":637,"duration_ms":7160,"temperature":1.0,"reasoning_tokens":577,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T11:05:44.380049+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run SimpleRecon, DoubleTake, and LiteMVS under one shared keyframe schedule, mesh mask, and GPU, then recompute the depth and reconstruction tables; if the re-run baselines match or exceed LiteMVS's reported numbers, the central accuracy claim is an artifact of the comparison protocol.","supporting_citations":[{"cited_title":"Simplere- con: 3d reconstruction without 3d convolutions","cited_arxiv_id":null,"evidence_quote":"Supplies the base lightweight plane-sweep architecture, geometric metadata, and primary multi-view supervision loss that LiteMVS builds on and must beat."},{"cited_title":"Doubletake: Geometry guided depth estimation","cited_arxiv_id":null,"evidence_quote":"Geometry-guided baseline whose depth and reconstruction numbers LiteMVS is compared against; also a source of the geometric-metadata idea."},{"cited_title":"Depth any- thing v2.Advances in Neural Information Processing Sys- tems, 37:21875–21911, 2024","cited_arxiv_id":null,"evidence_quote":"Provides the relative-depth pseudo-labels distilled into LiteMVS via a scale-invariant loss."},{"cited_title":"Stablenormal: Reducing diffusion variance for stable and sharp normal.ACM Transactions on Graphics (TOG), 43(6):1–18, 2024","cited_arxiv_id":null,"evidence_quote":"Provides the surface-normal pseudo-labels used for SSIM-plus-gradient normal distillation."},{"cited_title":"Deep- videomvs: Multi-view stereo on video with recurrent spatio- temporal fusion","cited_arxiv_id":null,"evidence_quote":"Defines the keyframe/test-split protocol and the per-frame update-latency metric used in the comparisons."},{"cited_title":"Transformerfusion: Monocular rgb scene reconstruction using transformers.Advances in Neural In- formation Processing Systems, 34:1403–1414, 2021","cited_arxiv_id":null,"evidence_quote":"Supplies the mesh-mask evaluation protocol used to measure 3D reconstruction accuracy."},{"cited_title":"Scene co- ordinate regression forests for camera relocalization in rgb-d images","cited_arxiv_id":null,"evidence_quote":"7-Scenes, the cross-dataset benchmark used to show generalization without fine-tuning."}],"review_version":1}