{"id":"7d87e1a4-90f9-4325-b17f-8fab9296ff7e","arxiv_id":"2508.12917","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"CMF-IoU is a multi-stage camera-LiDAR fusion detector whose IoU-aware confidence ranking improves 3D detection accuracy on KITTI, nuScenes, and Waymo.","lead":"This paper presents CMF-IoU, a 3D object detector that fuses camera images and laser-scanner points at multiple stages and predicts how well a proposed box matches the real object. It reports improved detection accuracy over several prior methods on KITTI, nuScenes, and Waymo benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The IoU-joint branch—the paper's second claimed contribution—has no defined NMS score formula, and the two ablations that isolate it disagree by over 1.5 AP on the same benchmark.","rationale":"The reader's conditional verdict is appropriate. I do not see a reason to move it to reject or unverified: the fusion backbone is described in detail, the ablation tables show consistent directional gains for the multi-stage fusion components, and the robustness experiments add supporting evidence. The most load-bearing concern is narrower than the reader's domain-gap worry but lives in the same component: the IoU-joint branch is claimed to improve NMS via a balanced confidence score, yet the score is never defined, and the two places that isolate this branch report materially different numbers on the same benchmark. If the formula turns out to be a simple product or weighted sum, the concern may dissolve, and if the ablations are reconciled the paper's internal evidence would be coherent. Until then, the second contribution is not independently checkable from the manuscript, which supports the conditional recommendation rather than full acceptance.","tokens_in":24964,"tokens_out":8111,"duration_ms":81744,"concrete_test":"Ask the authors to state the exact formula used for the balanced NMS score (for example, s = alpha * IoU_pred + (1 - alpha) * cls_score, or a product rule, with the value of alpha) and rerun the KITTI validation car experiment with alpha = 0 (classification-only NMS), alpha = 1 (IoU-only NMS), and the reported balanced rule under identical proposal-generation settings. Also rerun the Table VI and Table VII IoU-only ablations with the same Nsp, IoU thresholds, and augmentation; if the balanced rule does not beat both pure rules, or the two tables cannot be reproduced with the stated settings, the IoU-joint contribution to the headline results is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's second headline contribution, IoU joint prediction, is not actually specified. Section III-C states that the predicted IoU score 'is then combined with the classification score to form the final confidence score,' and Section IV-B says the balanced metric is used for NMS, but no equation, weighting, or algorithm is given anywhere, including in the post-processing comparison of Section IV-D-9. The only quantitative evidence for the branch is also internally inconsistent: Table VI reports VoxelRCNN + IoU Pred as 93.40/87.23/85.50 on KITTI validation car, while the analogous '+CMF-IOU' row in Table VII reports 93.68/86.12/83.97 on the same baseline and benchmark, with moderate and hard AP disagreeing by 1.11 and 1.53 points. Because the central claim attributes part of the reported SOTA to this branch, an undefined and irreproducible ranking rule plus a contradictory ablation leaves the second contribution unverifiable even if the fusion backbone is sound.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CMF-IoU, a two-stage camera-LiDAR 3D object detector. Images are converted into pseudo point clouds with a depth completion network; a bilateral backbone with a sparse-to-distant (S2D) branch and a residual view consistency (ResVC) branch encodes raw LiDAR and pseudo points; an iterative voxel-point aware pooling module refines proposals; and an IoU prediction branch, trained with ground-truth-based proposals, is used together with classification scores for NMS ranking. The authors report state-of-the-art results on KITTI, nuScenes, and Waymo validation/test sets, with module-level ablations and robustness experiments against point noise and calibration errors. The central claim is that multi-stage fusion combined with IoU-classification balanced NMS outperforms single- or partial-stage fusion detectors.","tokens_in":25157,"tokens_out":4935,"duration_ms":50107,"significance":"If the reported results are reproducible, the paper provides a useful empirical recipe for multi-modal 3D detection and demonstrates concrete gains from early, middle, and late stage fusion. The strengths are the breadth of the evaluation (three public benchmarks), the module-by-module ablation structure, the robustness experiments, and the public code release. However, the significance is currently undercut by unresolved internal inconsistencies in the ablation tables and by the absence of a defined rule for combining IoU and classification scores, which together prevent the reader from verifying the second claimed contribution and the exact source of the reported improvements.","major_comments":[{"comment":"The IoU-classification balanced metric is never specified. Section III-C states that the predicted IoU score \"is then combined with the classification score to form the final confidence score,\" and Section IV-B says the balanced metric is used for NMS, but no equation, weighting, normalization, or algorithm is given anywhere in the paper. Table XVI compares only vanilla NMS, adaptive NMS threshold, and multi-hypothesis tracking; the proposed metric is not even included in its own post-processing comparison. Since a central part of the claimed SOTA is attributed to this branch, the missing definition leaves the second contribution unreproducible. Please provide the exact combination formula, the score normalization, the NMS pseudo-code, and an ablation row that varies the combination against the alternatives in Table XVI.","section":"§III-C and §IV-D-9"},{"comment":"The same configuration appears with inconsistent numbers on the KITTI validation car benchmark. The VoxelRCNN baseline is 92.10/86.40/84.39 in Table VI, 92.38/85.29/82.86 in Tables II and VII, and 94.78/88.45/85.99 in Table VIII. The full CMF-IoU model is 96.01/88.97/88.31 in Tables VII and VIII but 95.52/89.01/87.21 in Table II. The IoU-branch-only ablation is 93.40/87.23/85.50 in Table VI versus 93.68/86.12/83.97 in Table VII, with moderate and hard AP differing by 1.11 and 1.53 points. These are not minor fluctuations, and because the paper's central claim is comparative, the discrepancies make it impossible to verify the claimed gains. Please rerun all ablations under one consistent protocol and report the actual evaluation settings, or clearly explain any legitimate differences (e.g., proposal generation count, iterations, or training schedule) in the text.","section":"Tables VI, VII, VIII, and II"},{"comment":"The paper's own sensitivity analysis indicates that the default proposal count Nsp=100 lies on a peak: hard AP is 86.51 at Nsp=10, rises to 88.31 at Nsp=100, and falls back to 86.66 at Nsp=160, with the drop attributed to a \"domain gap between the generated RoIs during the training and the predicted RoIs during the validation.\" This is an explicit admission that the uniform-proposal assumption behind the IoU branch is fragile, and it also shows that the reported final numbers depend on a hyperparameter tuned on the validation set. Please provide additional evidence that the benefit is stable out of the validation distribution, for example by reporting the Nsp sweep over multiple seeds with standard deviations or on a held-out split, and clarify which other hyperparameters were likewise selected on the validation set.","section":"§IV-D-6 and Table XIII"}],"minor_comments":[{"comment":"The method is called \"CMF-IOU\" in Table V but \"CMF-IoU\" throughout the rest of the manuscript; please unify the notation.","section":"Table V"},{"comment":"The text says the method is trained on all 7,481 KITTI images for testing, but the implementation details describe a 3,712/3,769 training/validation split and the validation results in Tables II and VI–VIII are presumably from that split; please clarify which protocol produced each table.","section":"§IV-A-1 and §IV-B"},{"comment":"The table header says \"40% recall\" but the KITTI protocol uses 40 recall positions (R40); please correct the wording.","section":"Table VII"},{"comment":"The sentence \"the distinct between the CIoU and vanilla IoU is relatively minor\" is ungrammatical; also state whether the differences between CIoU and IoU are within expected run-to-run noise, since Table XV does not report variances.","section":"Table XV"},{"comment":"The caption should define what \"+generation\" means and how it relates to the IoU joint prediction branch, since Fig. 6 appears to conflate the proposal generation strategy with the IoU prediction branch.","section":"Fig. 6"},{"comment":"The post-processing comparison does not include the proposed IoU-classification balanced metric; please add it so that Section IV-D-9 directly evaluates the claimed contribution rather than only alternative NMS schemes.","section":"Table XVI"}],"recommendation":"major_revision","confidential_remarks":"The internal inconsistencies in the ablation tables and the missing definition of the IoU-classification combination are the main barriers to publication. The paper otherwise has the shape of a solid empirical architecture study with broad evaluation and a code release. I did not find evidence of intentional misreporting, but the discrepancies need to be resolved and the metric needs to be fully specified before the results can be verified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, this is a serious engineering paper on camera-LiDAR 3D detection, and the fusion backbone is probably the real contribution: separate S2D and ResVC branches, iterative voxel-point pooling with cross-iteration attention, and GT-based proposal generation all hang together and are supported by module-by-module ablations on KITTI, nuScenes, and Waymo. Second, the paper's second claimed contribution—IoU joint prediction—is not actually specified and, as currently written, is not reproducible. The stress-test note is right.\n\nThe paper does many things well. The S2D and ResVC branches are cleanly motivated and individually ablated. The iterative pooling module is described in enough detail to reimplement. The robustness experiments under sensor noise and calibration error are a genuine plus. The paper also honestly flags a limitation of its own GT proposal generation in Section IV-D-6: when all RoIs are generated (Nsp=160), hard-level AP declines due to the domain gap between generated and predicted RoIs. Citation practice looks normal, and there is no circularity in the derivation sense.\n\nNow the soft spots, in proportion. The IoU joint branch is the biggest one. Section III-C says the predicted IoU is combined with the classification score to form the final confidence score, but no equation, weighting, or algorithm is given anywhere. Section IV-D-9 compares NMS variants, not the claimed balanced metric. Worse, the two ablations that isolate the branch disagree: Table VI reports VoxelRCNN + IoU Pred as 93.40/87.23/85.50 on KITTI val car, while Table VII's +CMF-IOU row on the same baseline reports 93.68/86.12/83.97. That is over 1.5 AP difference on hard. Unless a non-obvious difference separates those rows, one table is wrong or the configuration changed between runs. Since the paper attributes part of its SOTA to this branch, the second contribution is currently unverifiable.\n\nOther issues are more ordinary. Waymo has no comparison against prior work, only an ablation on PV-RCNN. No error bars or multiple seeds are reported anywhere. Several hyperparameters are chosen from validation, so some portion of the reported gain is model selection. The label 'CMF-IoU' is used for different configurations in different tables, which makes the ablation study harder to read than it should be.\n\nWho is this for? Researchers working on LiDAR-camera fusion for autonomous driving. The fusion backbone deserves referee time and, ultimately, citation if it reproduces; the IoU branch needs to be specified and its contradictions resolved first. My recommendation: send to peer review, but require the authors to write down the exact confidence rule, reconcile Tables VI and VII, and provide code or at least per-seed variance before acceptance.","headline":"The fusion backbone is a credible engineering contribution with broad empirical support, but the paper's second headline claim—IoU joint prediction—is underspecified and internally inconsistent, so the paper needs major revision before its SOTA claim can be trusted.","tokens_in":25769,"tokens_out":2478,"would_cite":false,"duration_ms":27643,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CMF-IoU claims that fusing camera and LiDAR at every pipeline stage, then ranking refined boxes by a joint IoU-classification score, beats prior single-stage fusion detectors on KITTI, nuScenes, and Waymo.","keywords":["3D object detection","multi-modal fusion","LiDAR-camera fusion","depth completion","pseudo points","IoU joint prediction","non-maximum suppression","autonomous driving"],"falsifier":"Measure the correlation between the IoU branch's predicted score and the true IoU on held-out RPN proposals per difficulty level on KITTI validation; if the correlation is weak on hard-level proposals, the reported gains of the IoU joint prediction branch would not transfer. A direct check is to replace the learned joint score with oracle NMS using ground-truth IoU: if oracle ranking does not beat the learned ranking, the branch is not actually learning IoU.","tokens_in":24700,"feed_emoji":"🚗","tokens_out":6771,"duration_ms":60648,"temperature":0.7,"pith_summary":"This paper proposes a camera-LiDAR 3D object detector called CMF-IoU and argues that fusion must happen at every stage of the detection pipeline, not just one. The authors convert camera pixels into 3D pseudo points with a depth completion network, encode raw LiDAR and pseudo points in a two-branch backbone, fuse voxel and point features during iterative proposal refinement, and finally rank boxes by a combined IoU and classification confidence. They report top results on the KITTI, nuScenes, and Waymo benchmarks and show the gains are largest for small, distant, and hard objects. A reader should care because the recipe addresses the known failure modes of camera-LiDAR fusion, namely depth bias in pseudo points, sparse distant LiDAR, and using classification scores that do not match the IoU-based evaluation metric.","feed_headline":"Multi-stage camera-LiDAR fusion tops 3D detection benchmarks","feed_subtitle":"Fusing camera and LiDAR in every stage, from backbone to NMS, beats single-stage fusion on KITTI, nuScenes, and Waymo.","key_machinery":"The load-bearing mechanism is the pairing of a bilateral cross-view backbone with an IoU-classification balanced confidence. The S2D branch is an encoder-decoder that enlarges the receptive field over sparse raw LiDAR voxels, which is what carries the gains on objects beyond 40 meters. The ResVC branch applies residual submanifold 3D convolutions followed by sparse 2D convolutions in the image plane, re-aligning features whose depth values are corrupted near object boundaries. In the refinement stage, iterative voxel-point aware pooling aggregates raw and pseudo voxel features plus point features sampled by farthest point sampling, with cross-attention across iterations. The IoU joint prediction branch trains on uniformly distributed proposals generated from ground-truth boxes and produces a final score that is the combination of the IoU prediction and classification score, which is what changes the NMS ordering.","core_discovery":"The central claim is that multi-stage cross-modal fusion combined with IoU-joint NMS is a better recipe for LiDAR-camera 3D detection than the prevalent single-stage or partial-stage fusion designs. Concretely, CMF-IoU unifies the two modalities early by projecting image pixels into 3D via a depth completion network, encodes raw LiDAR points with a sparse-to-distant (S2D) branch and pseudo points with a residual view-consistency (ResVC) branch whose 3D-to-2D convolutions compensate for depth noise, and refines proposals with an iterative voxel-point aware pooling module. In the head, a new proposal generation procedure samples ground-truth boxes to create a uniform IoU distribution, training an IoU branch whose score is combined with the classification score for final NMS. On the KITTI test set the method reports 91.92/85.14/80.63 3D AP for easy/moderate/hard car, and on nuScenes test it reports 69.8 mAP and 72.6 NDS, with improvements concentrated on small classes such as pedestrians and traffic cones.","pith_inferences":["The GT-based uniform proposal generation is a plug-in training technique that could be lifted into any two-stage detector whose NMS still ranks purely by classification score; the paper demonstrates it only inside CMF-IoU.","The ResVC principle, that depth-completion noise concentrates at object boundaries and can be compensated by re-projecting into the 2D image plane, suggests that boundary-aware noise models would benefit other pseudo-point fusion pipelines.","Because the reported gains concentrate on small, distant, and hard instances, the practical payoff is likely largest for long-range highway perception where LiDAR returns per object are scarce and camera texture carries most of the evidence.","A natural stress test the paper does not run is ablating the three refinement iterations against one or two while holding the IoU branch fixed, to see whether the cross-attention refinement or the uniform proposal sampling is doing the heavy lifting."],"forward_implications":["The fusion recipe is transferable: on nuScenes validation, replacing the backbones and heads of VoxelNeXt, CenterPoint, and SAFDNet with the CMF modules raises mAP by 1.3 to 5.5 points and NDS by 1.3 to 4.1 points.","Ranking proposals by the joint IoU-classification score changes which boxes survive NMS: after refinement, the surviving set has a higher proportion of high-IoU boxes than with classification-only NMS.","The S2D branch specifically targets distant objects: far-distance cyclist 3D AP improves by 7.97 points (moderate) and 7.49 points (hard), compared with about 1 point at nearby distances.","The ResVC branch makes the detector less sensitive to sensor noise: under 5 percent point noise, CMF-IoU's mAP drops 3.14 points versus 5.03 points for the compared VirConv method."],"supporting_citations":[{"why":"VoxelRCNN is the baseline detector whose backbone and refinement head are replaced; the KITTI and Waymo improvements are measured against it.","marker":"[19]"},{"why":"MVP supplies the pseudo-point generation for the nuScenes and Waymo experiments.","marker":"[17]"},{"why":"PENet is the depth completion network that produces the pseudo points on KITTI.","marker":"[43]"},{"why":"PV-RCNN provides the point-voxel two-stage paradigm and serves as the Waymo baseline.","marker":"[6]"},{"why":"VirConv is the compared multi-modal fusion method used for robustness and ablation comparisons.","marker":"[12]"},{"why":"SFD is the depth-completion late-fusion method whose data augmentation and comparisons anchor the KITTI experiments.","marker":"[18]"},{"why":"EPNet is the middle-stage fusion method against which the multi-stage claim is compared on KITTI.","marker":"[16]"},{"why":"3D IoU-Net is the IoU-guided 3D detection baseline for the IoU branch comparison.","marker":"[21]"},{"why":"IoU-Uniform RCNN supplies the uniform-proposal training idea that the GT-based proposal generation extends to 3D.","marker":"[22]"},{"why":"VoxelNeXt is the nuScenes baseline whose sparse head is adopted and whose NDS is improved by over 4 points.","marker":"[44]"}],"fun_headline_variants":["Multi-stage fusion with IoU joint prediction improves 3D detection","Fuse camera-LiDAR at every stage, then refine with IoU","Depth-completed multi-stage fusion tops KITTI, nuScenes, Waymo","CMF-IoU: multi-stage fusion and IoU-aware NMS for 3D detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that an IoU predictor trained on boxes synthetically generated from ground-truth boxes with a uniform IoU distribution will generalize to the RPN's real proposals at inference; the paper itself flags the domain gap when all 160 RoIs are generated and hard-level AP declines.","fun_headline_variants_meta":{"raw":{"variants":["Multi-stage fusion with IoU joint prediction improves 3D detection","Fuse camera-LiDAR at every stage, then refine with IoU","Depth-completed multi-stage fusion tops KITTI, nuScenes, Waymo","CMF-IoU: multi-stage fusion and IoU-aware NMS for 3D detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000593,"raw_usage":{"total_tokens":2850,"prompt_tokens":1087,"completion_tokens":1763,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":703,"completion_tokens_details":{"reasoning_tokens":1675}},"tokens_in":703,"tokens_out":1763,"duration_ms":16542,"temperature":1.0,"reasoning_tokens":1675,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:17:48.283617+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the correlation between the IoU branch's predicted score and the true IoU on held-out RPN proposals per difficulty level on KITTI validation; if the correlation is weak on hard-level proposals, the reported gains of the IoU joint prediction branch would not transfer. A direct check is to replace the learned joint score with oracle NMS using ground-truth IoU: if oracle ranking does not beat the learned ranking, the branch is not actually learning IoU.","supporting_citations":[{"cited_title":"V oxel r- cnn: Towards high performance voxel-based 3d object detection,","cited_arxiv_id":null,"evidence_quote":"VoxelRCNN is the baseline detector whose backbone and refinement head are replaced; the KITTI and Waymo improvements are measured against it."},{"cited_title":"Virtual sparse convolution for multimodal 3d object detection,","cited_arxiv_id":null,"evidence_quote":"VirConv is the compared multi-modal fusion method used for robustness and ablation comparisons."},{"cited_title":"Sparse fuse dense: Towards high quality 3d detection with depth completion,","cited_arxiv_id":null,"evidence_quote":"SFD is the depth-completion late-fusion method whose data augmentation and comparisons anchor the KITTI experiments."},{"cited_title":"Epnet: Enhancing point features with image semantics for 3d object detection,","cited_arxiv_id":null,"evidence_quote":"EPNet is the middle-stage fusion method against which the multi-stage claim is compared on KITTI."},{"cited_title":"Iou-uniform r-cnn: Breaking through the limitations of rpn,","cited_arxiv_id":null,"evidence_quote":"IoU-Uniform RCNN supplies the uniform-proposal training idea that the GT-based proposal generation extends to 3D."}],"review_version":2}