{"id":"13a1c513-ed99-44a9-9003-18e5bea7476f","arxiv_id":"2508.16408","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"SAMFusion improves 3D object detection in fog, snow, and night by adaptively fusing RGB camera, LiDAR, gated NIR, and radar features in Bird's Eye View.","lead":"This paper describes SAMFusion, a sensor fusion method for self-driving cars that combines camera, LiDAR, gated near-infrared, and radar, and learns to weight them based on distance and visibility. The authors report a 17.2 average precision gain for detecting pedestrians far away in fog, compared to the next best method.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Distance/visibility conditioning may rely on degraded inputs in the very scenes where it matters, threatening the claimed 17.2 AP gain.","rationale":"The paper's core innovation is sensor-adaptive fusion driven by distance and visibility. For the reported 17.2 AP improvement to hold in real-world foggy scenes, the mechanism that decides which sensors to trust must itself be reliable when sensors are failing. Yet distance and visibility are typically computed from the very sensors that degrade in fog: LiDAR depth returns drop out, and camera-based visibility estimation suffers from backscatter and low contrast. This creates a potential circularity: the model uses corrupted signals to decide which corrupted signals to ignore. The provided text (abstract, introduction, references) offers no experimental evidence that this conditioning is robust to such input degradation. The proposed sensitivity analysis directly tests the load-bearing assumption. If the model is insensitive to perturbations of the conditioning inputs, the concern is resolved; if not, the claimed long-distance foggy-scene improvement is suspect. This is not a disagreement with the community consensus; it is a concrete correctness risk that can be settled with an experiment. I therefore recommend a conditional acceptance: require the authors to report this robustness check before the claim is accepted as established.","tokens_in":4903,"tokens_out":6148,"duration_ms":70625,"concrete_test":"From the full arXiv version, obtain the trained model and test code. Run the long-distance foggy pedestrian subset twice: once with the original distance/visibility conditioning, and once with these inputs replaced by ground-truth values. Then repeat with noise added to the conditioning inputs that mimics LiDAR/camera degradation in fog. If AP changes by more than, say, 3 points, the method is not robust to conditioning-input degradation and the 17.2 AP claim is not trustworthy.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The decoder weighs modalities based on distance and visibility, and the central claim is a 17.2 AP gain for vulnerable pedestrians in long-distance foggy scenes. However, in fog, distance estimates are often derived from LiDAR (which the introduction notes is asymmetrically degraded) and visibility is typically inferred from camera images (also degraded). If the conditioning inputs are noisy exactly when the sensor failure they are meant to compensate for occurs, the learned weighting can be systematically biased. The provided text contains no experiments, so it is unknown whether the weighting is robust to this. This is the most load-bearing concern: a failure here would invalidate the sensor-adaptive mechanism and directly undercut the reported improvement.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes SAMFusion, a 3D object detection method that fuses RGB, LiDAR, gated NIR, and radar via attentive depth-based blending, BEV-plane refinement, and a transformer decoder whose modality weighting is conditioned on distance and visibility. The stated contribution is improved robustness in adverse weather, with a headline improvement of 17.2 AP over the next-best method for vulnerable pedestrians at long range in fog. The submitted text contains an abstract, an introduction, Figure 1, and a reference list; the method and experimental sections are not present in the supplied manuscript, so the central claims cannot be verified from the submitted material.","tokens_in":5106,"tokens_out":4169,"duration_ms":49044,"significance":"The problem is important: adverse weather degrades sensors asymmetrically, and existing fusion methods are typically validated only in clear weather. Combining gated NIR and radar with RGB and LiDAR is a plausible direction, and the distance/visibility-conditioned weighting is a reasonable mechanism if implemented carefully. However, as submitted, the paper's significance is unsubstantiated. I see no machine-checked proofs, reproducible code, or quantitative experimental evidence; the only evidence is the qualitative Figure 1 and an unsupported AP number. If the complete paper contains rigorous experiments on a relevant benchmark with baselines, ablations, and error bars, this could be a useful contribution.","major_comments":[{"comment":"The central empirical claim is a 17.2 AP improvement over the next-best method, but the submitted manuscript contains no evaluation section, no dataset description, no protocol, no baselines, no error bars, and no ablation. Without these, the claim is not assessable. Please supply the full method and experiments, including the exact metric definition (AP at which IoU threshold, 3D or BEV, class-wise breakdown), the definition of 'long distances', the fog/visibility conditions, the list of compared methods, dataset splits, and repeated-run statistics. Figure 1 alone does not support the headline number.","section":"Abstract and §1"},{"comment":"The decoder weighs modalities based on distance and visibility. In fog, LiDAR point clouds are asymmetrically degraded and camera-based visibility estimates are degraded as well; if the conditioning inputs are computed from the very sensors whose failures the method is meant to compensate, the learned weighting can be systematically biased in the target condition. The manuscript does not state how distance and visibility are estimated, nor does it provide an experiment separating the effect of clean versus degraded conditioning. Please specify the conditioning source and include a robustness test, e.g., comparing oracle versus estimated visibility, perturbing visibility inputs, or reporting per-fog-density performance.","section":"§1 / architecture description"},{"comment":"The paper's contribution is empirical in nature, yet no table or figure with quantitative comparisons appears anywhere in the provided text. A paper that introduces a new fusion architecture and a sensor-adaptive weighting scheme needs at least: a main results table, ablations for each proposed component (gated NIR/radar inclusion, depth-based blending, BEV refinement, distance/visibility conditioning), and a per-condition breakdown for clear, foggy, snowy, and night scenes. Without these, the architecture's effectiveness cannot be separated from the choice of training setup or the baseline implementations.","section":"General (no experimental section)"}],"minor_comments":[{"comment":"The phrase 'attentive, depth-based blending schemes' is used without equations or a figure describing the mechanism. Please define the fusion operations precisely, including how depth is obtained and how attention is computed across modalities.","section":"§1"},{"comment":"The term 'learned refinement on the Bird's Eye View (BEV) plane' is vague. Please clarify what is refined, which features are concatenated, and how the refinement interacts with the transformer decoder.","section":"§1"},{"comment":"The abstract's claim of 'bridging the gap between ideal conditions and real-world edge cases' is qualitative. If this is meant to indicate generalization across weather conditions, the experiments should explicitly test unseen fog densities or unseen sensor degradations.","section":"Abstract/§1"},{"comment":"The qualitative detections in Figure 1 would be more informative with annotations showing the ground-truth/prediction agreement and the scene conditions. As presented, the figure does not allow the reader to gauge detection quality or compare against any baseline.","section":"Figure 1"},{"comment":"The reference list is extensive, but several citations in the introduction (e.g., [66,67] for gated depth) appear only in the introduction and are not connected to a method description in the provided text. Once the full method is restored, please ensure every citation is used and that the related-work discussion is complete.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The provided manuscript appears incomplete: it lacks the method, experiments, and results sections. If this is due to a formatting issue in the review copy, I would need the complete PDF to assess the technical contribution. If the missing sections are genuinely absent from the submission, the paper is not ready for review; a major revision is the appropriate outcome because the gaps are fixable by adding the missing content."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a serious engineering paper whose central claim I couldn't check from what we received. It deserves referee time, but only if the full experiments stand up.\n\nWhat's actually new: adding gated NIR and radar to the usual RGB+LiDAR stack, with a transformer decoder that weights modalities by distance and visibility. That's a sensible direction for adverse weather, and the authors are the right people to try it—Bijelic and Heide did the original gated-camera fog work. The introduction engages the relevant fusion literature and correctly identifies the unimodal query generator and LiDAR-based depth projection as weak points.\n\nThe soft spots are about evidence, not ideas. The only numbers we have are in the abstract: 17.2 AP gain on long-range foggy pedestrians. No method section, no ablations, no error bars, no dataset split, no comparison details. On that basis the number is a claim, not a result. The stress-test concern is also real and should be the first thing a referee pushes on: distance and visibility are the conditioning variables, but both are themselves estimated from degraded sensors—LiDAR in fog, camera imagery in low light. If those estimates are unreliable in exactly the conditions where the method is supposed to help, the weighting can be systematically biased. The full paper may handle this (e.g., using gated NIR depth or radar for conditioning), but the provided text doesn't say.\n\nI don't want to manufacture a flaw. The approach is coherent and the reported gain is large enough to matter if the experiments are clean. It's just that an abstract and an introduction aren't enough to assess soundness. I'd send it to review, with instructions to focus on the robustness of the distance/visibility conditioning, the evaluation protocol, and ablations that isolate the contribution of each sensor.","headline":"A plausible four-sensor fusion architecture with a headline AP gain that can't be verified from the provided text; worth sending to review but needs experiments.","tokens_in":5492,"tokens_out":2265,"would_cite":false,"duration_ms":24015,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SAMFusion, a four-sensor fusion method for autonomous driving, claims a 17.2 AP gain over the next best method for vulnerable pedestrians in long-range foggy scenes by learning to weight modalities by distance and visibility.","keywords":["3D object detection","multimodal sensor fusion","adverse weather","gated NIR imaging","radar","Bird's Eye View","transformer decoder","autonomous driving"],"falsifier":"Evaluate the trained model on a held-out foggy sequence where LiDAR is physically blocked or soiled while RGB and gated NIR remain clear. If detection accuracy falls to the level of a camera-only baseline instead of retaining the radar and gated-NIR advantage, the claimed sensor adaptivity to missing or degraded range input is not real.","tokens_in":4874,"feed_emoji":"🌫️","tokens_out":3961,"duration_ms":44574,"temperature":0.7,"pith_summary":"SAMFusion is a 3D object detector for autonomous vehicles that fuses four sensors—RGB camera, LiDAR, gated near-infrared camera, and radar—using attention-based blending on the Bird's Eye View plane. Its transformer decoder learns to upweight whichever modalities remain reliable at a given distance and visibility, so when LiDAR and RGB degrade in fog, snow, or night, gated NIR and radar can carry the detection. The paper's central reported result is a 17.2 average precision improvement over the next best method for vulnerable pedestrians at long range in dense fog. If correct, this narrows the gap between fusion methods validated in clear weather and real-world adverse conditions.","feed_headline":"Sensor-adaptive fusion lifts foggy 3D detection by 17.2 AP","feed_subtitle":"Weights RGB, LiDAR, gated NIR, and radar by distance and visibility to catch far pedestrians in fog.","key_machinery":"The core mechanism is a sensor-adaptive fusion pipeline: attentive depth-based blending of image and range features, learned refinement on the Bird's Eye View plane, and a transformer decoder that assigns per-modality weights based on distance and visibility. The gated NIR and radar streams provide the redundant information that lets the network detect objects when RGB and LiDAR are unreliable.","core_discovery":"The central claim is that existing multimodal fusion methods fail in adverse weather because they depend on a unimodal query generator and on LiDAR-based depth projections, which break when LiDAR degrades. SAMFusion instead fuses RGB, LiDAR, gated NIR, and radar through attentive depth-based blending and learned refinement on the Bird's Eye View plane, with a transformer decoder that weighs modalities according to distance and visibility. This lets the network lean on gated NIR and radar when camera and LiDAR signals are corrupted by fog, snow, or soiling. The reported result is a 17.2 AP improvement over the next best method for vulnerable pedestrians in long-distance, foggy scenes.","pith_inferences":["The 17.2 AP figure is reported on a specific foggy subset; a deployment test would show whether the learned weights transfer to unseen sensor angles, weather intensities, or soiling patterns without retraining.","A natural ablation is to remove the gated NIR stream and keep the other three sensors; if the improvement persists, the sensor-adaptive weighting itself is doing the work rather than the extra sensor.","The distance- and visibility-conditioned weighting could transfer to other BEV perception tasks, such as occupancy prediction or online mapping, where weather-driven sensor degradation is also a bottleneck.","A field test with physically blocked or soiled LiDAR, while RGB and gated NIR remain clean, would separate the contribution of the attention weighting from the inherent robustness of gated NIR and radar."],"forward_implications":["The reported 17.2 AP improvement on vulnerable pedestrians at long range in fog is a concrete gain for a safety-critical subset of autonomous driving.","The same architecture can handle clear weather and adverse weather without switching models, because the decoder learns to reweight sensors on the fly.","When LiDAR point clouds become asymmetric or sparse in fog, the method can still rely on gated NIR and radar rather than collapsing to a single degraded modality.","The learned distance- and visibility-based weighting is a general way to combine sensors with complementary failure modes.","The approach extends beyond fog to night-time, snow, and sensor soiling, since those conditions affect the same sensor reliability cues."],"supporting_citations":[{"why":"Supplies the foggy adverse-weather multimodal benchmark (RGB, gated NIR, LiDAR) and the evaluation setting for the central comparison.","marker":"[3]"},{"why":"Documents asymmetric LiDAR degradation in fog, motivating why fusion cannot rely on LiDAR depth alone.","marker":"[4]"},{"why":"Benchmarks gated cameras under adverse weather, supporting gated NIR as a reliable modality in low light and fog.","marker":"[5]"},{"why":"Provides a transformer-based LiDAR-camera fusion baseline that SAMFusion builds on and compares against.","marker":"[1]"},{"why":"Represents the unimodal-query modality-interaction approach whose LiDAR dependence the paper identifies as a failure mode.","marker":"[83]"},{"why":"A standard BEV LiDAR-camera fusion framework used as a comparison baseline.","marker":"[42]"},{"why":"A multi-resolution sensor-fusion architecture that serves as a comparison baseline for adverse weather.","marker":"[7]"},{"why":"nuScenes provides the clear-weather multimodal data used to evaluate performance in normal conditions.","marker":"[8]"}],"fun_headline_variants":["Fog-proof sensor fusion boosts 3D detection by 17.2 AP","Adaptive fusion catches foggy pedestrians: +17.2 AP","Multimodal fusion adapts to weather, lifts AP by 17.2","Weighted sensor fusion sees through fog: +17.2 AP","Fuses RGB, LiDAR, NIR, and radar to beat fog: 17.2 AP"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The method assumes that a sensor's reliability can be predicted from distance and visibility cues in the training distribution, so the learned weighting will transfer to the fog, snow, and soiling conditions it was trained on.","fun_headline_variants_meta":{"raw":{"variants":["Fog-proof sensor fusion boosts 3D detection by 17.2 AP","Adaptive fusion catches foggy pedestrians: +17.2 AP","Multimodal fusion adapts to weather, lifts AP by 17.2","Weighted sensor fusion sees through fog: +17.2 AP","Fuses RGB, LiDAR, NIR, and radar to beat fog: 17.2 AP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000223,"raw_usage":{"total_tokens":1306,"prompt_tokens":768,"completion_tokens":538,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":433}},"tokens_in":512,"tokens_out":538,"duration_ms":6318,"temperature":1.0,"reasoning_tokens":433,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T17:18:02.351297+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate the trained model on a held-out foggy sequence where LiDAR is physically blocked or soiled while RGB and gated NIR remain clear. If detection accuracy falls to the level of a camera-only baseline instead of retaining the radar and gated-NIR advantage, the claimed sensor adaptivity to missing or degraded range input is not real.","supporting_citations":[{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Supplies the foggy adverse-weather multimodal benchmark (RGB, gated NIR, LiDAR) and the evaluation setting for the central comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents asymmetric LiDAR degradation in fog, motivating why fusion cannot rely on LiDAR depth alone."},{"cited_title":"In: 2018 IEEE Intelligent Vehicles Symposium (IV)","cited_arxiv_id":null,"evidence_quote":"Benchmarks gated cameras under adverse weather, supporting gated NIR as a reliable modality in low light and fog."},{"cited_title":"In: Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition","cited_arxiv_id":null,"evidence_quote":"Provides a transformer-based LiDAR-camera fusion baseline that SAMFusion builds on and compares against."},{"cited_title":"Advances in Neural Information Processing Systems 35, 1992–2005 (2022) 1, 2, 3, 9, 10, 12, 13","cited_arxiv_id":null,"evidence_quote":"Represents the unimodal-query modality-interaction approach whose LiDAR dependence the paper identifies as a failure mode."},{"cited_title":"Advances in Neural Information Processing Systems35, 10421–10434 (2022) 2, 10, 12, 13","cited_arxiv_id":null,"evidence_quote":"A standard BEV LiDAR-camera fusion framework used as a comparison baseline."},{"cited_title":"In: IEEE International Confer- ence on Intelligent Transportation Systems (ITSC) (2023) 1, 2, 4","cited_arxiv_id":null,"evidence_quote":"A multi-resolution sensor-fusion architecture that serves as a comparison baseline for adverse weather."},{"cited_title":"In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","cited_arxiv_id":null,"evidence_quote":"nuScenes provides the clear-weather multimodal data used to evaluate performance in normal conditions."}],"review_version":1}