{"id":"a946d74a-00ca-46cb-b567-c3fc842faf26","arxiv_id":"2412.05771","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A semi-supervised MVS training method that transfers monocular relative-depth structure priors into MVS predictions via a deep feature loss and a multi-scale statistical loss, improving depth accuracy on ScanNet++ and ARKitScenes.","lead":"Prism is a training framework that teaches multi-view stereo networks to predict depth from unlabeled smartphone video by borrowing structural knowledge from a monocular depth network trained on synthetic images. It reports large accuracy gains over unsupervised, synthetic-supervised, and naive semi-supervised baselines on two real indoor datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The affine-only, confidence-free scale/shift fit of Eq. 3 and the unvalidated correctness of the Hypersim teacher are the load-bearing assumptions; no test bounds the damage from a locally wrong teacher.","rationale":"I read this as an empirical claim: Prism's losses transfer useful structure from a monocular teacher trained on Hypersim to MVS on real video, yielding large gains. For that claim to hold, the teacher's relative structure must be trustworthy on the target domain and the alignment in Eq. 3 must isolate a global scale/shift. The paper provides strong internal evidence that the losses work: ablations show that the feature and statistical losses outperform pixel-wise L1, SSIM, and MS-SSIM; gains appear on both ScanNet++ and ARKitScenes; and Table 3 shows different teachers still help. I therefore do not see an internal inconsistency. But none of these experiments breaks the link between teacher correctness and the result: a wrong teacher would be amplified by Lmono, and the LPIPS artifact in Fig. 6 is direct evidence that deep-feature supervision on depth maps can create confident hallucinations. The scale-shift least-squares fit is the other fragile joint because it is global, unweighted, and only deferred, not made robust. The ARKitScenes residual diagnostic is a feasible way to check whether the affine assumption actually holds on real data; a corrupted-teacher retraining variant would quantify error propagation. Since this is an open robustness question rather than a demonstrated flaw, the conditional verdict stands: the method is promising and well reported, but the exact mechanism and margins need independent verification. This agrees with the reader's identification of teacher reliability and scale-shift alignment as the weakest assumption.","tokens_in":14103,"tokens_out":12626,"duration_ms":140837,"concrete_test":"Take the ARKitScenes validation subset (approximate real depth), align each Hypersim-trained teacher prediction to the provided depth with Eq. 3, and analyze the residual map: bin residuals by depth, by texturelessness, and by distance to strong edges, and test for spatial autocorrelation. If residuals are small and noise-like, the affine-only teacher assumption is sound; if residuals are structured or depth-correlated at an amplitude comparable to the Prism-vs-semi-supervised margin (Table 1), then Lmono can transfer exactly those errors to the MVS network and the central claim is not established for such scenes.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on two linked assumptions: (i) the monocular network trained only on Hypersim predicts reliable structure on real indoor smartphone imagery, and (ii) the only discrepancy between that teacher and the MVS prediction is a global scale and shift, so the least-squares fit in Eq. 3 fully captures their relationship. Neither is demonstrated. Eq. 3 is solved over all pixels with no confidence mask; the sole safeguard is delaying Lmono until after the first epoch (Sec. 3.3), which protects against a bad initial scale but not against a teacher that is wrong exactly where the paper motivates the method: textureless and reflective surfaces and thin structures. Because Lunsup is the signal that fails on those cases, nothing in Eq. 1 corrects the teacher. The paper's own Sec. 4.5 and Fig. 6 show a feature loss (LPIPS) producing sharp hallucinations that look like structure, demonstrating that this mechanism can confidently impose incorrect geometry; switching to the SD-v2 encoder removes that specific artifact, but the general failure mode is not analyzed. Table 3 shows robustness across three strong general-purpose teachers, but it does not bound the damage from a teacher that is wrong in a structured, depth-correlated way. If such errors exist, Lmono pushes MVS toward them and the reported margins over the semi-supervised baseline shrink or invert on the affected scenes.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Prism, a semi-supervised multi-view stereo (MVS) training framework that jointly uses unlabeled real smartphone video and labeled synthetic data. The key idea is to distill structural priors from a monocular relative-depth network (Marigold) into the MVS prediction via two novel losses: a deep feature loss inspired by LPIPS and a pyramid-SSIM statistical loss. The monocular prediction is aligned to the MVS prediction through a global least-squares scale/shift fit, and the losses are applied on the unlabeled real data. The total loss combines these monocular losses with an unsupervised loss (DIV loss) and a supervised loss on synthetic data. Experiments on ScanNet++ and ARKitScenes show consistent improvements over unsupervised, synthetic-supervised, and semi-supervised baselines, with ablations demonstrating that each component contributes positively.","tokens_in":14349,"tokens_out":5778,"duration_ms":57995,"significance":"If the results hold, this is a valuable contribution to semi-supervised MVS. The idea of transferring structural priors from a monocular relative-depth network using perceptual-similarity-inspired losses is novel and practically motivated. The paper provides extensive empirical validation: comparisons to multiple baselines on two datasets, thorough ablations, and a study across three different monocular teachers. The gains are large and consistent across metrics, and the analysis of LPIPS-induced artifacts shows a thoughtful design process. The paper is clearly written and the methodology is reproducible. The central claim—that the proposed losses transfer structure better than pixel-wise alternatives—is well supported by the ablation study.","major_comments":[{"comment":"The global least-squares scale/shift alignment in Eq. (3) is the only mechanism that reconciles the monocular teacher's relative depth with the MVS prediction, yet it is computed over all pixels with no confidence weighting. The paper does not analyze how erroneous MVS predictions (e.g., on textureless or reflective surfaces) bias the estimated (s,t), nor does it bound the damage from a teacher that is systematically wrong on certain structures. The only safeguard is the one-epoch delay in §3.3, which addresses initial scale but not structured teacher errors. Because Lmono is the central contribution, the absence of a failure analysis or a confidence-masked variant leaves a load-bearing assumption unvalidated.","section":"§3.1, Eq. (3)"},{"comment":"The paper shows that a deep feature loss based on LPIPS causes hallucinations that look like sharp details and reports that switching to the SD-v2 encoder alleviates this artifact. However, it does not investigate whether the final loss combination can still impose incorrect structure when the teacher disagrees with multi-view photometric evidence. A per-scene error breakdown or an experiment with a deliberately corrupted teacher would clarify whether the improvements are robust to teacher errors, rather than merely reflecting cases where the teacher happens to be correct.","section":"§4.5, Fig. 6"}],"minor_comments":[{"comment":"The phrase 'This is a best-case-scenario result' is unclear; please clarify whether it refers to the experimental setup or the outcome.","section":"Abstract"},{"comment":"The duplication of the depth map into three channels before feature extraction is motivated only by the feature extractor's input format; a brief justification of why a single-channel input is not used would improve clarity.","section":"§3.1"},{"comment":"The claim that comparing only the deepest feature embedding gives the best results is not supported by a visible ablation; consider reporting this comparison or citing a supplementary.","section":"§3.1"},{"comment":"The rows are numbered 0-9, but the text refers to them inconsistently; consider labeling rows explicitly with names or using the same numbering consistently in the text.","section":"Table 2"},{"comment":"There are a few typographical errors, e.g., 'S tructure' in the title and 'confirming' instead of 'confirming', that should be corrected.","section":"Title and text"}],"recommendation":"major_revision","confidential_remarks":"I share the stress-test concern that the teacher-correctness assumption is not adequately tested. The paper's empirical results are strong, but a robustness experiment (e.g., with a corrupted teacher or a confidence-weighted alignment) is needed to support the central claim. I would recommend major revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I've read Prism. It's a solid, honestly-reported distillation method for semi-supervised MVS, and the core idea is genuinely new: applying perceptual losses—a deep feature loss and a summed pyramid SSIM—to depth maps to transfer a monocular relative-depth prior into an MVS network. The ablations are well-constructed and show this combination beats pixel-wise L1, SSIM, and MS-SSIM, with consistent gains on ScanNet++ and ARKitScenes and robustness across three teacher networks. The authors also report the LPIPS artifact and replace the encoder, which is the kind of transparency that earns trust.\n\nThe soft spots are mostly about reproducibility and selection. Results are single-run, no error bars, ablations are reported on test rather than a held-out validation split, and several hyperparameters are hand-set with the admission that validation tuning would likely help. That means the exact margins should not be treated as robust, but the qualitative direction is believable given the size and consistency of the gains. The stress-test concern about the affine-only scale/shift fit in Eq. 3 is fair but not fatal: a teacher that is wrong in a structured, depth-correlated way could pull the student toward bad geometry, and there is no confidence masking to protect against that. The paper itself demonstrates one such failure (LPIPS hallucinations) and fixes it; it does not bound the damage for other teacher errors. That is a real limitation of the distillation framing, not a hidden flaw, and the empirical results suggest it does not dominate on these datasets.\n\nThe citation pattern is appropriate, and the framing as the first application of monocular priors to MVS depth prediction holds up. This paper is for researchers working on self-supervised or semi-supervised MVS and anyone using monocular depth as a structural prior. It deserves a serious referee. I would recommend accepting it into peer review, with requests for error bars, a documented validation split, and ideally released code or checkpoints.","headline":"Prism is a solid, honestly-reported distillation method for semi-supervised MVS that deserves a serious referee, though its exact margins rest on a single run and a teacher that is trusted without confidence weighting.","tokens_in":14921,"tokens_out":3037,"would_cite":true,"duration_ms":29828,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that transferring a monocular relative-depth teacher's structure through deep-feature and multi-scale SSIM losses lets semi-supervised MVS outperform all baselines on ScanNet++ and ARKitScenes.","keywords":["semi-supervised learning","multi-view stereo","monocular depth prior","relative depth estimation","deep feature loss","pyramid SSIM","indoor scene reconstruction","unlabeled video training"],"falsifier":"Train Prism with a deliberately broken teacher whose depth maps are replaced by constant values or by spatially shuffled patches, keeping every other training detail the same; if performance relative to the semi-supervised baseline does not drop substantially, the monocular losses are not carrying the structure prior the paper attributes to them.","tokens_in":13855,"feed_emoji":"🎥","tokens_out":6893,"duration_ms":61792,"temperature":0.7,"pith_summary":"The paper proposes a semi-supervised training framework, Prism, that lets multi-view stereo (MVS) networks learn from both unlabeled smartphone video and labeled synthetic renders. Its central claim is that a monocular relative-depth network trained on synthetic data can serve as a structure teacher for real video, provided the transfer is done through two perceptual-inspired losses rather than pixel-wise comparison: a deep feature loss and a pyramid-based multi-scale SSIM loss. If this works, MVS training no longer needs expensive real depth ground truth, and the paper reports that Prism beats unsupervised, synthetic-supervised, and simple semi-supervised baselines on every metric on both ScanNet++ and ARKitScenes.","feed_headline":"Monocular depth priors cut MVS error by 10 to 30 percent","feed_subtitle":"Prism beats every baseline on ScanNet++ and ARKitScenes depth metrics.","key_machinery":"The load-bearing mechanism is the pair of monocular structure losses. The deep feature loss takes aligned depth maps, duplicates them into three-channel images, passes them through a pretrained image encoder, and minimizes the mean $\\ell^2$ distance between channel-normalized deep embeddings of the teacher and student predictions. The pyramid SSIM loss replaces standard MS-SSIM's multiplicative scale combination with a normalized sum of single-scale SSIM computed on four downsampled levels, which the paper finds more stable during training. Both losses operate on the teacher output after it is aligned to the MVS prediction by the global scale and shift of Eq. 3, and the framework also keeps the usual unsupervised photometric, smoothness, and augmentation-consistency losses plus a supervised regression, gradient, and normal loss on synthetic data.","core_discovery":"The central discovery is that comparing aligned depth maps in a learned feature space, together with a summed multi-scale SSIM statistic, transfers monocular structure far more effectively than pixel-wise $\\ell^1$, SSIM, or MS-SSIM losses. After aligning the teacher's affine-invariant prediction to the MVS prediction with a global scale and shift (Eq. 3), the deep feature loss (Eq. 4) normalizes channel-wise deep embeddings of the duplicated depth images and penalizes their mean $\\ell^2$ distance, while the pyramid SSIM loss (Eqs. 5–6) computes a normalized sum of SSIM over four downsampled levels. The two losses are complementary: the deep feature loss handles global structure, the statistical loss sharpens local detail, and together with the supervised and unsupervised terms they improve results on textureless and reflective surfaces and on thin structures. Every component of the framework contributes, and the monocular losses are responsible for most of the gap over the semi-supervised baseline.","pith_inferences":["The same deep-feature plus multi-scale-statistics recipe could be applied to transfer other affine-invariant dense predictions, such as surface normals or illumination, to a multiview student, although the paper only demonstrates depth.","Because the scale-shift alignment of Eq. 3 is global, a scene where the teacher and MVS disagree locally—for example, a large mirror—could pull the student toward the teacher's hallucinated geometry; a confidence-weighted or region-masked variant of the feature loss is a natural untested extension.","One could isolate the value of the teacher by fixing all training settings and swapping the teacher's predictions for the rendered ground-truth depth of the unlabeled frames; if Prism's advantage shrinks, the framework's gain really is the transferable structure prior rather than the extra regularizer."],"forward_implications":["If Prism's claim is correct, MVS networks can be trained on large unlabeled video corpora plus small synthetic sets, with the monocular teacher supplying the structure that photometric losses miss on reflective and textureless surfaces.","The gains transfer: a model trained on ScanNet++ videos and Hypersim synthetics also improves on ARKitScenes without fine-tuning, suggesting the learned prior is not dataset-specific.","The ablation results imply that the choice of loss dominates the choice of teacher: both an out-of-the-box monocular network and one retrained on Hypersim produce similar improvements when combined with the two structure losses.","The paper's pyramid SSIM formulation and deep-feature comparison each independently outperform pixel-wise $\\ell^1$, SSIM, and MS-SSIM at transferring monocular depth structure."],"supporting_citations":[{"why":"Supplies the diffusion-based monocular relative-depth teacher used to generate structure priors on unlabeled frames.","marker":"[23]"},{"why":"Supplies the Hypersim synthetic indoor dataset that provides the labeled training data and the teacher's training set.","marker":"[39]"},{"why":"Supplies the DIV unsupervised losses and the strongest unsupervised baseline that Prism builds on and compares against.","marker":"[38]"},{"why":"Supplies the ScanNet++ smartphone videos used as the unlabeled real training and test data.","marker":"[64]"},{"why":"Inspires the deep feature loss, which the paper adapts from RGB perceptual metrics to depth maps.","marker":"[69]"},{"why":"Supplies the global scale-and-shift alignment procedure used before the monocular losses are applied.","marker":"[36]"},{"why":"Supplies the CasMVSNet-style cost-volume architecture used as the multi-view stereo network in all experiments.","marker":"[16]"}],"fun_headline_variants":["Feature-space losses transfer monocular depth to MVS","Prism blends real-synthetic data for robust MVS","MVS error drops 10-30% with monocular priors","Feature losses sharpen MVS on textureless and reflective surfaces"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a monocular relative-depth network trained only on synthetic indoor renders produces trustworthy structure on real handheld smartphone video, and that its output differs from the multi-view prediction by no more than a global scale and shift.","fun_headline_variants_meta":{"raw":{"variants":["Feature-space losses transfer monocular depth to MVS","Prism blends real-synthetic data for robust MVS","MVS error drops 10-30% with monocular priors","Feature losses sharpen MVS on textureless and reflective surfaces"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001086,"raw_usage":{"total_tokens":4543,"prompt_tokens":955,"completion_tokens":3588,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":571,"completion_tokens_details":{"reasoning_tokens":3519}},"tokens_in":571,"tokens_out":3588,"duration_ms":27593,"temperature":1.0,"reasoning_tokens":3519,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:22:35.728339+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train Prism with a deliberately broken teacher whose depth maps are replaced by constant values or by spatially shuffled patches, keeping every other training detail the same; if performance relative to the semi-supervised baseline does not drop substantially, the monocular losses are not carrying the structure prior the paper attributes to them.","supporting_citations":[{"cited_title":"Repurpos- ing diffusion-based image generators for monocular depth estimation","cited_arxiv_id":null,"evidence_quote":"Supplies the diffusion-based monocular relative-depth teacher used to generate structure priors on unlabeled frames."},{"cited_title":"Susskind","cited_arxiv_id":null,"evidence_quote":"Supplies the Hypersim synthetic indoor dataset that provides the labeled training data and the teacher's training set."},{"cited_title":"Smoothness, synthesis, and sampling: Re-thinking unsuper - vised multi-view stereo with DIV loss","cited_arxiv_id":null,"evidence_quote":"Supplies the DIV unsupervised losses and the strongest unsupervised baseline that Prism builds on and compares against."},{"cited_title":"ScanNet++: A high-fidelity dataset of 3d indoor scenes","cited_arxiv_id":null,"evidence_quote":"Supplies the ScanNet++ smartphone videos used as the unlabeled real training and test data."},{"cited_title":"The unreasonable effectiveness of deep features as a perceptual metric","cited_arxiv_id":null,"evidence_quote":"Inspires the deep feature loss, which the paper adapts from RGB perceptual metrics to depth maps."},{"cited_title":"Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer","cited_arxiv_id":null,"evidence_quote":"Supplies the global scale-and-shift alignment procedure used before the monocular losses are applied."},{"cited_title":"Cascade cost volume for high-resolution multi-view stereo and stereo matching","cited_arxiv_id":null,"evidence_quote":"Supplies the CasMVSNet-style cost-volume architecture used as the multi-view stereo network in all experiments."}],"review_version":1}