{"id":"643f1c18-ad3a-4d8e-96c2-7fc634e4a6c3","arxiv_id":"2411.09151","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Pre-training stereo networks on monocular images warped into synthetic stereo pairs, plus a KL distillation loss to dense monocular depth, improves zero-shot generalization and fine-tuning.","lead":"This paper trains stereo-matching networks using synthetic right-eye images generated from single photos with depth-estimation software, then distills dense monocular depth knowledge into the fine-tuning step. The result is better zero-shot generalization on outdoor benchmarks and small in-domain gains on KITTI.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The S2DKD loss (Eq. 5) is undefined as written: KL divergence is applied to scalar disparity maps with no specified normalization or probability representation, so the central fine-tuning contribution cannot be reproduced and its claimed domain robustness is unverified.","rationale":"The paper's strongest claim is that the two-stage training (DiffMFS pre-training plus S2DKD fine-tuning) yields significant improvements in zero-shot generalization and in-domain performance. The pre-training stage is relatively well specified: monocular depth is scaled and used to warp a right view, with holes inpainted by an edge-aware Stable Diffusion module. The fine-tuning stage, however, rests on a loss that is not mathematically defined. KL divergence operates on probability distributions, but D_out and D_mono are scalar disparity maps; the paper gives no equation for forming distributions, no normalization, no temperature, and no binning. The ablation's comparison with L2 and gradient losses suggests that the KL variant is doing something specific to align the two scales, but the text does not say what. If the alignment is per-image min-max rescaling, it is fragile and could explain why the official KITTI benchmark improvement is only 0.01 D1, with qualitative gains confined to unlabeled regions that the KITTI metric ignores. This concern is load-bearing because S2DKD is one of the two pillars of the method; without a precise formulation, the claimed in-domain improvements cannot be reproduced or transferred. The reader's verdict of CONDITIONAL is appropriate: the paper should be accepted only if the authors supply the exact loss implementation and code, and ideally error bars or multiple seeds to substantiate 'significant.' We agree with the reader's identification of this as the weakest assumption.","tokens_in":14168,"tokens_out":11557,"duration_ms":118351,"concrete_test":"Implement Eq. 5 under the most natural reading: rescale D_mono and D_out to a common range (e.g., [0,1] per image), quantize into B bins, and apply per-pixel KL divergence over softmax distributions, then fine-tune IGEV on the same 20% KITTI15 split used in Table 3. If the D1 improvement over the no-S2DKD baseline (1.521 vs. 1.459) is not reproduced, or if it depends strongly on the bin count B or temperature, then the KL loss is not a well-defined, scale-invariant signal and the claimed fine-tuning contribution is unreliable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is the Sparse-to-Dense Knowledge Distillation (S2DKD) loss, one of the paper's two central contributions. Equations 4-5 define L_KL = KL(D_out, D_mono) as a per-pixel KL divergence between the stereo network's disparity output and the monocular model's relative depth, but both are scalar maps, not probability distributions. The paper never specifies the normalization, softmax, disparity binning, or temperature used to convert these scalars into distributions. The ablation in Table 3 shows that L2 and gradient variants fail due to scale mismatch, so the KL variant must rely on an unreported alignment; the text's claim that KL is 'not sensitive to size' is not a property of KL divergence, suggesting the actual mechanism is something else (e.g., per-image min-max rescaling or rank matching). If the hidden alignment is a per-image normalization, it is likely brittle across domains, which would explain why the official KITTI 15 benchmark improvement is only 0.01 D1 (1.59 to 1.58) while the larger gains appear only in unlabeled regions (sky, distant objects) that the KITTI metric does not score. Because S2DKD is essential to the claimed in-domain improvements and is not reproducible from the paper, the central assertion cannot be validated as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Mono2Stereo, a two-stage training framework that transfers knowledge from a pretrained monocular depth network (DepthAnything) to stereo matching networks. In the first stage, the authors generate a synthetic stereo dataset (DiffMFS) from monocular images by scaling monocular disparity maps, forward-warping to synthesize a right view, and filling occlusion holes with an edge-aware inpainting module based on Stable Diffusion. In the second stage, they introduce a Sparse-to-Dense Knowledge Distillation (S2DKD) loss that is claimed to align the stereo network's predicted disparity distribution with dense monocular depth estimates during fine-tuning on sparse LiDAR labels. Experiments are reported on ETH3D, KITTI 12, and KITTI 15 for three architectures (PSMNet, CFNet, IGEV), including zero-shot comparisons and a fine-tuned KITTI 15 test benchmark submission.","tokens_in":14503,"tokens_out":6378,"duration_ms":64909,"significance":"If the two-stage pipeline is reproducible as described, it offers a practical recipe for enlarging stereo training data from monocular images and for densifying supervision beyond sparse LiDAR labels, with cross-architecture validation. The zero-shot experiments in Table 1 are encouraging and suggest that the DiffMFS generation pipeline is a useful alternative to SceneFlow pretraining. The paper also ships a clear ablation structure (Table 3) for the inpainting module, the loss form, and the choice of monocular teacher. However, the significance is conditional on resolving the specification issues below: the S2DKD loss in Eq. (5) is not a well-defined KL divergence as written, and the unsupervised comparison protocol in Table 2 has a potential indirect-label leakage risk through the monocular teacher that is not discussed.","major_comments":[{"comment":"The KL loss in Eq. (5) is not mathematically defined as written: KL divergence is defined between probability distributions, but D_out(i,j) and D_mono(i,j) are scalar disparity values. The paper does not state how these scalars are converted to distributions (e.g., per-pixel softmax over a disparity grid, binning, or per-image min-max normalization with a temperature). Because Table 3 shows that the L2 and gradient variants fail due to scale mismatch, the KL variant must rely on an unreported alignment mechanism. The text's statement that KL is 'not sensitive to size' is not a property of KL divergence, so the actual mechanism (possibly rank-based or normalizing each image) needs to be described precisely. Without this specification, the central fine-tuning contribution is not reproducible. Please provide the exact normalization, the number of sampled pixels, and the complete per-pixel loss formula.","section":"Method, 'Disparity Distribution Distillation', Eqs. (4)-(5)"},{"comment":"The claim of a fair unsupervised comparison on KITTI 15 is not established. The text says the model 'is trained on our FA-Flow Dataset' - a dataset that is never defined, presumably a typo for DiffMFS. More importantly, both the pre-training disparities and the S2DKD pseudo-labels are generated by DepthAnything, and the paper does not disclose whether DepthAnything was trained on or evaluated on KITTI imagery and depth. If DepthAnything had access to KITTI ground-truth depth, the method would have indirect access to KITTI labels, invalidating the comparison with methods that truly use no KITTI labels. Please clarify DepthAnything's exposure to KITTI (and to ETH3D for the zero-shot claims), and ideally rerun the unsupervised protocol with a teacher that provably excludes these evaluation domains from its training set.","section":"Experiments, 'Comparison with Unsupervised Methods', Table 2"},{"comment":"Both training stages align the stereo network with the same external teacher (DepthAnything): the DiffMFS pre-training uses DepthAnything's relative disparity as the supervision signal, and the fine-tuning stage uses DepthAnything's pseudo-labels in the S2DKD loss. The stereo network is therefore aligned twice to the same teacher's inverse-depth distribution, with no independent geometric or metric source in the second stage. As a result, the improvements on regions not covered by LiDAR (e.g., the sky and distant objects, as acknowledged in supplemental section A2) may reflect consistent regression toward the teacher's bias rather than improved metric disparity. This does not by itself invalidate the method, since knowledge transfer is the stated goal, but it weakens the generalization claim. Please add an experiment that uses different teachers in the two stages (e.g., MiDaS for generation and DepthAnything for distillation, and the reverse) to show that the fine-tuning gain is not simply a monotone consequence of matching the same teacher twice.","section":"Method, overall two-stage pipeline"}],"minor_comments":[{"comment":"The section heading 'Spare-to-Dense Knowledge Distillation' appears in several places and should be 'Sparse-to-Dense'.","section":"Section headings and table text"},{"comment":"The phrase 'trained on our FA-Flow Dataset' should be corrected to 'DiffMFS Dataset' to match the name used elsewhere.","section":"Experiments, 'Comparison with Unsupervised Methods'"},{"comment":"There is a typo: 'ScaneFlow' should be 'SceneFlow'.","section":"Datasets section"},{"comment":"The notation for the scaled monocular disparity is inconsistent: the text uses D'_mono, while Eq. (3) uses D_mono' in the edge mask definition; please unify the notation.","section":"Eq. (3) and surrounding notation"},{"comment":"The PSNR column is labeled with a downward arrow, but higher PSNR is better and the reported values increase with the proposed method; the arrow should be upward.","section":"Supplemental Table 2"},{"comment":"The caption says 'computational advantage of your approach'; it should be 'our approach'.","section":"Figure 6 caption"},{"comment":"The claim of 'significant improvements' in in-domain performance should be tempered: the official KITTI 15 D1-all improvement is 0.01 (1.59 to 1.58) in the supplemental table, and the ablation improvement of the KL loss over S2DKD-off is 0.062 in D1 on a 20% validation split; please report multiple runs or confidence intervals to support the significance claim.","section":"Conclusion and Table 3"}],"recommendation":"major_revision","confidential_remarks":"The data-generation part of the paper is fairly incremental (warping plus diffusion inpainting), so the main novelty rests on the S2DKD loss and the two-stage transfer scheme. Given that Eq. (5) is underspecified, the central fine-tuning contribution cannot be independently verified from the manuscript. I would also ask the editor to ensure that the unsupervised comparison in Table 2 is accompanied by a clear disclosure of the monocular teacher's training data, since this determines whether the comparison is truly label-free for KITTI."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Good paper to know about, but not ready to ship as is. The empirical core is real: pre-training on DiffMFS gives consistent zero-shot gains over SceneFlow and MfS across three architectures on ETH3D and KITTI 12/15. The edge-aware inpainting with Stable Diffusion does what MfS didn't—produces warped views without the usual boundary artifacts. The paper is also honest about the small official benchmark gain (1.59 to 1.58 D1 on KITTI 15), explaining in the supplemental that gains concentrate in sky and distant regions that KITTI doesn't score. That transparency counts.\n\nThe soft spot is exactly the one the stress-test flags: Eq. 5 defines the S2DKD loss as KL(D_out(i,j), D_mono(i,j)) between scalar disparity values. KL divergence between scalars is undefined without a probability representation. The paper never says how the scalars are normalized, binned, or mapped to distributions. The ablation shows L2 and gradient variants fail on the scale mismatch, so the KL term must be doing a specific alignment, and the text's claim that KL is 'not sensitive to size' is not a property of KL divergence. This makes the central fine-tuning contribution unreproducible as written. It is fixable—the authors presumably do something concrete—but they need to write it down. There is also a typo in Table 2's protocol (\"FA-Flow Dataset\") and the unsupervised comparison needs a cleaner description of what data each method saw. The self-reinforcement from using DepthAnything in both stages is worth noting but not a deal-breaker, since the teacher is external.\n\nGiven the empirical consistency across architectures and the practical value of the dataset generation pipeline, this deserves a serious referee, not a desk reject. The review should require a precise specification of the KL loss, code or dataset release, and a corrected unsupervised protocol. I would bring it to a reading group because the gap between training-recipe claims and benchmark numbers is instructive.","headline":"Solid stereo training pipeline with real zero-shot gains, but the S2DKD loss is underspecified as written and the official-benchmark gain is tiny; worth reviewing with major revision.","tokens_in":15007,"tokens_out":2414,"would_cite":true,"duration_ms":100731,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that stereo matching networks can be made more accurate and better at generalizing to unseen scenes by transferring knowledge from monocular depth estimation, which has advanced further thanks to large-scale training…","keywords":["stereo matching","monocular depth estimation","knowledge distillation","data generation","pre-training","fine-tuning","edge-aware inpainting","disparity estimation"],"falsifier":"An experiment that would settle the claim: fine-tune the same network with S2DKD but replace the monocular depth predictions with randomized depth maps of the same scale (or a constant map), holding all other settings fixed. If the S2DKD gain persists, the loss is not actually transferring monocular knowledge; if the gain disappears, the transfer is real. A second check is to compute the KL loss with and without the paper's implicit normalization and verify that training diverges or stalls when the normalization is removed.","tokens_in":1964,"feed_emoji":"👁️","tokens_out":2310,"duration_ms":107405,"temperature":0.7,"pith_summary":"This paper argues that stereo matching networks can be made more accurate and better at generalizing to unseen scenes by transferring knowledge from monocular depth estimation, which has advanced further thanks to large-scale training data. To show this, the authors build a two-stage training procedure: first they synthesize a large stereo training set, called DiffMFS, from ordinary monocular images by warping them according to monocular depth predictions and filling the resulting holes with an edge-aware inpainting module; then, when fine-tuning on real-world data with sparse LiDAR labels, they add a Sparse-to-Dense Knowledge Distillation (S2DKD) loss that encourages the stereo network's disparity distribution to match the dense, relative depth predicted by the monocular model. The paper reports that models pre-trained on DiffMFS and fine-tuned with S2DKD achieve lower end-point error and fewer outliers on KITTI and ETH3D than the same networks trained on SceneFlow data and standard sparse-label fine-tuning. If correct, the work would give stereo matching a practical way to benefit from the recent progress in monocular depth estimation without changing the stereo network architecture.","feed_headline":"Monocular depth knowledge sharpens stereo matching","feed_subtitle":"Two-stage monocular transfer beats SceneFlow pre-training on KITTI and ETH3D.","key_machinery":"The machinery is a two-stage training pipeline. Stage one is DiffMFS, a data generation framework: a monocular depth model (DepthAnything) predicts relative disparity, which is randomly scaled to pixel disparity; forward warping produces a right-view image, and an Edge-Aware inpainting module—where a few background edge pixels are preserved before Stable Diffusion fills the holes—produces realistic right views. Stage two is S2DKD, a loss that computes the Kullback-Leibler divergence $\\mathrm{KL}(D_{\\mathrm{out}}, D_{\\mathrm{mono}})$ between the stereo network's predicted disparity distribution and the monocular depth distribution at randomly sampled pixels, combined with the sparse ground-truth loss to fine-tune on real data. The KL divergence is chosen specifically because it is scale-insensitive compared to L2 or gradient losses, which fail when matching metric disparity to affine-invariant relative depth.","core_discovery":"The central claim is that monocular depth models, despite predicting only scale-ambiguous relative depth, contain dense and reliable edge and structure information that stereo networks trained on sparse LiDAR labels lack, and that this information can be transferred through two mechanisms: (1) pre-training on stereo pairs synthesized from monocular images with edge-aware inpainting, and (2) fine-tuning with a KL-divergence-based Sparse-to-Dense Knowledge Distillation loss that aligns the stereo network's disparity distribution with the monocular model's relative depth distribution. The authors demonstrate this on three stereo architectures (PSMNet, CFNet, IGEV), showing consistent zero-shot improvements over SceneFlow pre-training and better fine-tuned in-domain performance on KITTI benchmarks, including an EPE of 1.05 and D1 of 4.50 on the KITTI 15 test set under a protocol that uses only the images, not the ground truth.","pith_inferences":["If the KL-divergence alignment is genuinely scale-insensitive, the same two-stage recipe could transfer knowledge from any relative-depth predictor (including self-supervised monocular models) to any metric task network beyond stereo, such as optical flow or multi-view depth.","The edge-aware inpainting module hints at a more general principle: when generative inpainting is used to complete warped images, preserving a few real background edge pixels before filling gives the diffusion model a spatial anchor, reducing foreground-background blending; this could apply to other novel-view-synthesis pipelines.","The paper shows gains in areas where ground truth is absent but cannot be measured on the KITTI benchmark; a natural extension is to evaluate on dense ground-truth datasets or with human judgments in those regions to quantify the claimed edge and sky improvements."],"forward_implications":["Stereo networks can be pre-trained on large collections of single images instead of requiring synthetic rendered scenes, reducing reliance on computer graphics and its domain gap.","The dense pseudo-labels from monocular depth provide supervision in regions where LiDAR gives no signal, such as sky, distant objects, and thin structures, potentially fixing a known overfitting failure in stereo fine-tuning.","The approach is architecture-agnostic: PSMNet, CFNet, and IGEV all improve, so the training strategy can be applied to future stereo backbones without modifying them.","Because the monocular model can be swapped (DepthAnything outperforms MiDaS in the ablation), stereo matching performance should keep improving as monocular depth estimation advances."],"supporting_citations":[{"why":"Supplies the monocular depth model (DepthAnything) whose relative depth drives both data generation and distillation.","marker":"(Yang et al. 2024)"},{"why":"The prior single-image stereo data generation method that DiffMFS extends and compares against.","marker":"(Watson et al. 2020)"},{"why":"The standard synthetic pre-training dataset (SceneFlow) that serves as the baseline and main comparison point for pre-training.","marker":"(Mayer et al. 2016)"},{"why":"The state-of-the-art stereo network (IGEV) used for the main experiments and the architecture for fine-tuning comparisons.","marker":"(Xu et al. 2023)"},{"why":"The alternative monocular depth predictor (MiDaS) used in the ablation to show DepthAnything's advantage.","marker":"(Ranftl et al. 2022)"},{"why":"The KITTI 2012 benchmark with sparse LiDAR labels, used for fine-tuning and evaluation.","marker":"(Geiger, Lenz, and Urtasun 2012)"},{"why":"The KITTI 2015 benchmark with semi-automatic ground truth, used for fine-tuning and evaluation.","marker":"(Menze and Geiger 2015)"}],"fun_headline_variants":["Monocular depth transfer boosts stereo matching","Stereo matching gains from monocular depth knowledge","Edge-aware monocular transfer refines stereo depth","Two-stage monocular transfer improves stereo accuracy","Monocular depth priors refine stereo predictions"],"cache_read_input_tokens":17152,"weakest_assumption_plain":"The S2DKD loss assumes that a per-pixel KL divergence between the stereo network's metric disparity and the monocular model's relative depth is a meaningful training signal, even though the paper never specifies how these scalars are converted into comparable probability distributions; the L2 and gradient variants fail on exactly this scale mismatch, so the success of the KL variant depends on an unstated normalization that could be brittle across domains.","fun_headline_variants_meta":{"raw":{"variants":["Monocular depth transfer boosts stereo matching","Stereo matching gains from monocular depth knowledge","Edge-aware monocular transfer refines stereo depth","Two-stage monocular transfer improves stereo accuracy","Monocular depth priors refine stereo predictions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000607,"raw_usage":{"total_tokens":2843,"prompt_tokens":973,"completion_tokens":1870,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":589,"completion_tokens_details":{"reasoning_tokens":1803}},"tokens_in":589,"tokens_out":1870,"duration_ms":17448,"temperature":1.0,"reasoning_tokens":1803,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:58:38.843253+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"An experiment that would settle the claim: fine-tune the same network with S2DKD but replace the monocular depth predictions with randomized depth maps of the same scale (or a constant map), holding all other settings fixed. If the S2DKD gain persists, the loss is not actually transferring monocular knowledge; if the gain disappears, the transfer is real. A second check is to compute the KL loss with and without the paper's implicit normalization and verify that training diverges or stalls when the normalization is removed.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the monocular depth model (DepthAnything) whose relative depth drives both data generation and distillation."},{"cited_title":"M.; Turmukhambetov, D.; Brostow, G","cited_arxiv_id":null,"evidence_quote":"The prior single-image stereo data generation method that DiffMFS extends and compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The standard synthetic pre-training dataset (SceneFlow) that serves as the baseline and main comparison point for pre-training."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The alternative monocular depth predictor (MiDaS) used in the ablation to show DepthAnything's advantage."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The KITTI 2012 benchmark with sparse LiDAR labels, used for fine-tuning and evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The KITTI 2015 benchmark with semi-automatic ground truth, used for fine-tuning and evaluation."}],"review_version":1}