{"id":"074165fc-ec97-4b2a-bbd9-5265171d3659","arxiv_id":"2507.19738","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"GRAFT-Stereo pre-fills sparse LiDAR disparities and fuses LiDAR XYZ cues early to make RAFT-Stereo work well with only a few hundred LiDAR points.","lead":"This paper shows that very sparse LiDAR points can still substantially improve stereo depth estimation if the missing values are pre-filled with interpolation. It explains why sparse guidance fails inside RAFT-Stereo and presents GRAFT-Stereo, which beats prior LiDAR-guided stereo methods under sparse inputs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim depends on simulated sparse LiDAR; real low-beam sensor noise and beam patterns are never tested, so the pre-fill benefit may not transfer to hardware.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: simulated sparse LiDAR may not represent real low-beam sensors. I agree this is the single most important threat to the central claim. The paper's internal evidence is strong: the late-only ablation in Table S1 shows pre-filling improves over naive guidance within the same RAFT-Stereo backbone, and the toy example in Sec. 5.1 illustrates a plausible mechanism. However, all of this is evaluated with idealized sparse inputs. If real low-beam LiDAR has different noise, dropout, or angular coverage, pre-filling could propagate inaccurate seed depths over large regions, and the iterative refinement may not correct them, which would erase the claimed benefit. The paper's own supplementary limitations do not address this gap, which makes it a genuine unvalidated assumption rather than a minor omission. Because the concern is about external validity and not an internal inconsistency, it does not overturn the conditional verdict; it reinforces the need for a real-sensor or realistic-simulation check before the central claim can be accepted as practically meaningful.","tokens_in":17698,"tokens_out":9655,"duration_ms":121740,"concrete_test":"On the KITTI raw validation set, construct the 4-beam guidance from the actual bottom four scan rings of the raw HDL-64E point clouds (preserving original range noise, dropouts, and beam positions) instead of from the aggregated semi-dense ground-truth depth map. Then train/evaluate GRAFT-Stereo and the RAFT+gd baseline exactly as in Table 6, and compare the 4-beam rows. If GRAFT-Stereo's RMSE/Bad1 advantage over RAFT+gd is not preserved, or if pre-filled guidance becomes worse than naive guidance, the central claim is specific to idealized simulated sparse LiDAR rather than to real low-cost sensors.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's practical motivation (Sec. 1) is affordable, low-cost LiDAR, and the central claim is that pre-filling enables sparse LiDAR guidance to work well. But all sparse-guidance experiments (Sec. 4.2, Tables 4-6, Figs. 1b/3) use either (1) uniform subsampling of the semi-dense aggregated ground-truth depth map, which gives noiseless, spatially uniform seed points, or (2) a 4/8/16-beam simulation created by sampling lines from a 64-beam Velodyne scan, which retains the accuracy and beam geometry of a high-end sensor. Real low-beam sensors have larger range noise, beam divergence, mixed pixels at object boundaries, irregular angular coverage, and synchronization/calibration errors relative to the stereo pair. The proposed pre-fill propagates seed values to large unmeasured regions, so if those seeds are noisy or systematically biased, pre-filling could amplify errors instead of helping. The supplementary Limitations (S1.1) lists only future work on depth losses and temporal consistency; it never acknowledges this sensor-realism gap. Since the entire practical claim rests on transferring the simulated benefit to real sparse LiDAR, this unvalidated bridge is the most load-bearing weakness. The controlled comparisons between naive guidance and pre-filled guidance within RAFT-Stereo are internally convincing, so the concern is about external validity, not an internal contradiction.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies how to inject sparse LiDAR depth into RAFT-Stereo for disparity estimation. The authors identify a late-fusion path (initializing the disparity map) and an early-fusion path (concatenating XYZ coordinates with RGB at the feature encoder). They observe that naive injection degrades sharply as LiDAR points become sparse (300 points) and propose a depth pre-filling step: for late fusion, filling missing disparities with interpolation (IP-Basic or a learned completion network); for early fusion, retaining only the top-1k confident completed points. The resulting GRAFT-Stereo combines both fusion paths and is evaluated on KITTI Depth Completion, VKITTI2, and MS2 under uniform subsampling and simulated 4/8/16-beam LiDAR, reporting consistent improvements over EG-Depth and SDG-Depth in disparity and depth metrics.","tokens_in":18012,"tokens_out":1500,"duration_ms":17905,"significance":"If the reported results hold, the paper makes a useful and counterintuitive contribution: a simple interpolation-based pre-fill of the initial disparity map can restore the benefit of very sparse LiDAR guidance to a strong iterative stereo baseline, with a signal-processing rationale (sparse guidance creates high-frequency discontinuities that low-pass recurrent convolution attenuates). The paper is also transparent about its design choices: it retrains baselines with official code, reports both disparity and depth metrics, provides ablations of late-only and full models, and includes supplementary experiments on 64-beam LiDAR, iteration trade-offs, test-time point counts, and mid-fusion. The main weaknesses are the lack of error bars or significance tests (each model trained once), the absence of real low-beam LiDAR hardware validation, and the post-hoc selection of the top-1k early-fusion subsample based on validation performance.","major_comments":[{"comment":"The paper's practical claim is that pre-filling enables sparse LiDAR guidance to work in affordable, low-cost settings, but all sparse-guidance experiments use either uniform subsampling of the semi-dense aggregated ground-truth depth map (noiseless, spatially uniform seeds) or beam sampling from a 64-beam Velodyne scan (which retains the accuracy and calibration of a high-end sensor). Real low-beam sensors exhibit larger range noise, mixed pixels, irregular coverage, and synchronization errors relative to the stereo pair. Since pre-filling propagates seed values to large unmeasured regions, biased or noisy seeds could amplify errors rather than help. The supplementary Limitations (S1.1) does not acknowledge this sensor-realism gap. I would recommend either adding experiments with synthetically injected LiDAR noise/calibration error or explicitly reframing the contribution as simulation-based and discussing the transfer risk.","section":"Sec. 4.2, Tables 4–6, Figs. S4–S5"},{"comment":"The early-fusion pre-filling approach relies on a post-hoc selection of the top-1k depth-completed points based on validation performance. The paper reports that retaining top-1k works, but does not report the sensitivity to the number of retained points (e.g., 500, 2k, 5k) or to the confidence measure used. Since this is a free parameter tuned on the validation set, a short sensitivity analysis would strengthen the claim that the chosen value is not overfit to the validation split. As reported, the difference between row ➅ (top-1k, 3.33 Bad1) and row ➆ (dense, 3.44 Bad1) is small, and without variance estimates it is unclear whether the improvement is significant.","section":"Sec. 5.3, Table 5, S3"},{"comment":"The signal-processing explanation (sparse guidance creates high-frequency discontinuities that are attenuated by low-pass filtering in RAFT-Stereo) is supported by a toy example and a qualitative FF, but the connection between the Laplacian of the retrieved feature map and the final disparity error is not quantitatively established on real data. The paper shows in Table 2 that adding noise to a zero disparity map degrades accuracy, and in Table 4 that pre-filling improves accuracy, but it does not directly measure the 'feature domination' quantity E (L2 distance to ground-truth retrieval) on KITTI before and after pre-filling. A small experiment reporting E or a similar retrieval-quality metric on real validation frames would make the proposed mechanism more than a plausible narrative.","section":"Sec. 5.1–5.2, Figs. 4–5, Tables 2–4"}],"minor_comments":[{"comment":"The abstract claims that GRAFT-Stereo 'significantly outperforms' existing methods, but no statistical significance tests are reported; consider softening to 'consistently outperforms' or adding significance measures.","section":"Abstract & Sec. 1"},{"comment":"The table reports IP-Basic and neural-net pre-filling alone with Bad1 of 52.51% and 17.59%, but the text does not state how these maps are evaluated (e.g., on the semi-dense ground truth or the full image); please clarify.","section":"Sec. 5.2, Table 3"},{"comment":"The text says 'For pixels lacking a projected LiDAR point, we concatenate zero values', but it is unclear whether this is done before or after depth pre-filling; please clarify the preprocessing order.","section":"Sec. 5.3, Fig. 7"},{"comment":"The main paper reports RMSE/MAE for depth while Table S1 reports disparity metrics; it would be helpful to report both in the main table or clearly state that disparity metrics are in the supplement.","section":"Sec. 6, Table 6 and Table S2"},{"comment":"The beam sampling follows [63] and samples 'more LiDAR lines from the lower part of the scene'; this detail is important for reproducibility and should be briefly mentioned in the main text or at least in the caption of Fig. 1b.","section":"Supplementary, Sec. S2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a strong empirical study with a clear message and thorough comparisons against retrained baselines. The main concern is external validity: the central practical claim (pre-filling benefits sparse LiDAR guidance) rests entirely on simulated sparse LiDAR. Given that the authors already have a solid controlled comparison, adding a noise-injection experiment or clearly limiting the claim to simulated sparsity would address the most load-bearing weakness. The top-1k selection and the lack of error bars are secondary but worth addressing in a revision. I would not reject the paper, but I would not accept it without the sensor-realism discussion being strengthened."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — this one is worth your time. The paper's useful contribution is not the pre-filling trick per se—depth completion has done similar—but the diagnosis of why RAFT-Stereo collapses as LiDAR gets sparse. The authors show that sparse ground-truth disparities create high-frequency discontinuities in the retrieved cost-volume features, which the recurrent low-pass filtering then attenuates, leaving the zero-disparity initialization in control. That explanation is supported by a clean toy example and, more importantly, by a consistent pattern of experiments across KITTI, VKITTI2, and MS2 with uniform and beam sampling. The simple fix—densify the initial disparity map with IP-Basic or a small trained completion network—recovers most of the dense-LiDAR benefit, and the early-fusion variant with subsampled pre-filled points is a genuinely new combination.\n\nThe internal evidence is solid. Baselines are retrained with their own official code for each sparsity level, which is more careful than most papers in this subfield. The results are consistent across all six sparsity settings. That part holds up.\n\nThe soft spots are about external validity, not internal contradiction. Every sparse-guidance experiment uses simulated sensors: either uniform subsampling of the semi-dense ground-truth depth map or beam patterns cut from a 64-beam Velodyne scan. Those seeds are noiseless and geometrically perfect. Real low-beam LiDAR has range noise, mixed pixels at edges, beam divergence, and calibration offsets against the stereo pair. Because pre-filling propagates seed values over large unmeasured regions, noisy seeds could be amplified rather than corrected. The authors never test on real low-beam hardware, and the supplementary limitations (S1.1) list only depth losses and temporal consistency as future work—the sensor-realism gap is never acknowledged. That is the main missing evidence.\n\nMinor issues: no error bars or significance tests (single run per setting), and the top-1k subsample for early fusion is chosen by validation performance, which is mild selection-on-validation. No code is released, so the numbers are not independently reproducible yet.\n\nMy verdict: this is a solid empirical paper that makes a clear, internally consistent claim about a real failure mode. The practical claim about low-cost sensors is plausible but unproven at the sensor level. I would accept it for peer review and ask the authors to either run a small real-sensor experiment (one low-beam sensor on a few sequences would do) or explicitly analyze robustness to seed noise. As it stands, it is a good paper that could become a very good one.","headline":"Pre-filling sparse LiDAR disparities fixes RAFT-Stereo's failure mode, and the analysis is the real contribution; the main open question is whether the simulated sparse sensors transfer to real hardware.","tokens_in":18525,"tokens_out":2654,"would_cite":true,"duration_ms":30502,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that pre-filling sparse LiDAR disparity with interpolation restores the benefit of LiDAR guidance in RAFT-Stereo, enabling state-of-the-art stereo depth with only a few hundred points.","keywords":["RAFT-Stereo","LiDAR-guided stereo","sparse LiDAR","depth pre-filling","disparity initialization","iterative refinement","depth completion","stereo matching"],"falsifier":"Collect a stereo-LiDAR dataset from an actual low-beam sensor (e.g., 4- or 8-beam) in driving scenes, retrain RAFT-Stereo with naive guidance and with pre-filling, and compare; if pre-filling no longer beats naive injection on real sparse scans, the paper's mechanism and remedy do not transfer beyond subsampled dense LiDAR.","tokens_in":17507,"feed_emoji":"🚗","tokens_out":6719,"duration_ms":65090,"temperature":0.7,"pith_summary":"RAFT-Stereo, an iterative stereo matcher, can exploit very sparse LiDAR depth if the sparse disparity initialization is first densified by interpolation. Naive injection of a few hundred LiDAR points fails, the paper argues, because the cost-volume retrieval is dominated by zero-initialized pixels and the recurrent convolutions treat the sparse accurate values as high-frequency noise. Pre-filling the missing disparities makes the guidance smooth enough to survive retrieval, so 300 pre-filled points outperform 64-beam guidance without pre-filling. Combining this late-fusion pre-fill with an early-fusion branch that appends 3D coordinates to image features yields GRAFT-Stereo, which the paper shows beats prior LiDAR-guided methods on KITTI, VKITTI2, and MS2 under sparse conditions.","feed_headline":"Pre-filled sparse LiDAR depth revives RAFT-Stereo","feed_subtitle":"With only 300 points per frame, interpolation before injection beats 64-beam guidance without it.","key_machinery":"The load-bearing mechanism is the iterative cost-volume lookup of RAFT-Stereo, $S(h,w,k) = C(h,w, w - D(h,w) + k)$, which indexes the correlation volume by the current disparity estimate. With sparse guidance, most pixels index at zero disparity, so their retrieved features dominate and the sparse accurate retrievals become high-frequency spikes that low-pass-filtering convolutions suppress. Pre-filling the initial disparity map (e.g., with IP-Basic interpolation) replaces zero disparities with plausible values, converting sparse guidance into a smooth, low-frequency initialization. A separate early-fusion path backprojects the pre-filled depth into 3D coordinates and concatenates them with RGB to reinforce stereo correspondence.","core_discovery":"The central claim is that pre-filling the sparse initial disparity map is what makes LiDAR guidance work inside RAFT-Stereo, for a specific mechanism. RAFT-Stereo retrieves a local slab of the correlation volume indexed by the current disparity estimate; when most pixels are zero-initialized, they dominate the retrieved features, and the sparse LiDAR-guided values appear as high-frequency outliers that the recurrent 2D convolutions attenuate. Densifying the initial map removes these discontinuities, letting the informative retrieval survive and propagate. For early fusion, the paper claims pre-filling also helps, but for a different reason — it must supply accurate correspondences — so a neural depth-completion with confidence-based top-1k subsampling is needed rather than coarse interpolation.","pith_inferences":["The zero-dominance mechanism suggests that any smoothness-inducing initialization — not only depth completion — should recover most of the gain, since even nearest-neighbor fill in the toy example reduces the retrieved-feature Laplacian from 0.65 to 0.33.","Real low-beam LiDAR sensors add beam-specific noise and irregular coverage that uniform subsampling of dense scans does not mimic; testing on actual 4- or 8-beam hardware would determine whether the pre-fill benefit persists.","The pre-fill-before-injection principle may transfer to other iterative refinement networks such as RAFT optical flow when they receive sparse external guidance.","The confidence-based top-1k subsampling result implies that under a fixed guidance budget, where you place the completed points matters more than the raw accuracy of the completion."],"forward_implications":["With as few as 300 LiDAR points per frame, pre-filled RAFT-Stereo achieves lower disparity error than RAFT-Stereo guided by full 64-beam LiDAR without pre-filling.","GRAFT-Stereo reports lower RMSE, MAE, Bad1, and average disparity error than EG-Depth and SDG-Depth under uniform-sampled and beam-sampled sparsity on KITTI, and on VKITTI2 and MS2 with 300 points.","The two pre-fill strategies are not interchangeable: coarse interpolation helps late fusion but hurts early fusion, where a confidence-subsampled neural completion is required.","Because RAFT-Stereo supports anytime prediction, the late-fusion pre-fill variant provides a fast CPU-compatible path (IP-Basic) while the full model retains the accuracy–iteration trade-off."],"supporting_citations":[{"why":"The base iterative stereo architecture whose cost-volume retrieval is analyzed and modified.","marker":"[1]"},{"why":"Supplies the image-processing interpolation (IP-Basic) used to pre-fill the initial disparity map.","marker":"[16]"},{"why":"EG-Depth: the LiDAR-guided stereo baseline GRAFT-Stereo is compared against on KITTI and other datasets.","marker":"[11]"},{"why":"SDG-Depth: the previous state-of-the-art LiDAR-guided stereo method used as the main baseline.","marker":"[12]"},{"why":"Non-local spatial propagation network used to build the neural depth-completion pre-filler.","marker":"[55]"},{"why":"KITTI Depth Completion dataset provides the stereo images, 64-beam LiDAR, and semi-dense ground truth for the main experiments.","marker":"[14]"},{"why":"Provides the low-beam LiDAR sampling strategy used to synthesize 4/8/16-beam inputs from dense scans.","marker":"[61]"}],"fun_headline_variants":["Pre-filling sparse LiDAR depth restores RAFT-Stereo performance","Sparse LiDAR guidance rescued by disparity pre-fill trick","Interpolation unlocks sparse LiDAR guidance in RAFT-Stereo","GRAFT-Stereo: pre-fill sparse depth, beat 64-beam baselines","Why pre-filling sparse disparity maps revives LiDAR guidance"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's sparse LiDAR is simulated by uniformly subsampling or beam-synthesizing dense 64-beam ground-truth scans, and it assumes this faithfully represents real low-cost sensors' noise, coverage, and beam patterns.","fun_headline_variants_meta":{"raw":{"variants":["Pre-filling sparse LiDAR depth restores RAFT-Stereo performance","Sparse LiDAR guidance rescued by disparity pre-fill trick","Interpolation unlocks sparse LiDAR guidance in RAFT-Stereo","GRAFT-Stereo: pre-fill sparse depth, beat 64-beam baselines","Why pre-filling sparse disparity maps revives LiDAR guidance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000806,"raw_usage":{"total_tokens":3502,"prompt_tokens":868,"completion_tokens":2634,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":484,"completion_tokens_details":{"reasoning_tokens":2538}},"tokens_in":484,"tokens_out":2634,"duration_ms":23067,"temperature":1.0,"reasoning_tokens":2538,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:03:38.140713+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect a stereo-LiDAR dataset from an actual low-beam sensor (e.g., 4- or 8-beam) in driving scenes, retrain RAFT-Stereo with naive guidance and with pre-filling, and compare; if pre-filling no longer beats naive injection on real sparse scans, the paper's mechanism and remedy do not transfer beyond subsampled dense LiDAR.","supporting_citations":[{"cited_title":"In defense of classical image processing: Fast depth completion on the cpu","cited_arxiv_id":null,"evidence_quote":"Supplies the image-processing interpolation (IP-Basic) used to pre-fill the initial disparity map."},{"cited_title":"Raft-stereo: Multilevel recurrent field transforms for stereo matching","cited_arxiv_id":null,"evidence_quote":"The base iterative stereo architecture whose cost-volume retrieval is analyzed and modified."},{"cited_title":"Expanding sparse lidar depth and guiding stereo matching for robust dense depth estimation","cited_arxiv_id":null,"evidence_quote":"EG-Depth: the LiDAR-guided stereo baseline GRAFT-Stereo is compared against on KITTI and other datasets."},{"cited_title":"Stereo-lidar depth estimation with deformable propagation and learned disparity-depth conversion","cited_arxiv_id":null,"evidence_quote":"SDG-Depth: the previous state-of-the-art LiDAR-guided stereo method used as the main baseline."},{"cited_title":"Non-local spatial propagation network for depth completion","cited_arxiv_id":null,"evidence_quote":"Non-local spatial propagation network used to build the neural depth-completion pre-filler."},{"cited_title":"Sparsity invariant cnns","cited_arxiv_id":null,"evidence_quote":"KITTI Depth Completion dataset provides the stereo images, 64-beam LiDAR, and semi-dense ground truth for the main experiments."},{"cited_title":"Pseudo-lidar++: Accurate depth for 3d object detection in autonomous driving","cited_arxiv_id":null,"evidence_quote":"Provides the low-beam LiDAR sampling strategy used to synthesize 4/8/16-beam inputs from dense scans."}],"review_version":1}