{"id":"56bcc3fb-2506-4d41-8cb0-afbb89065448","arxiv_id":"2501.07133","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"The paper introduces adverse-weather 3D LiDAR tracking benchmarks, shows existing trackers degrade sharply, and proposes DRCT, a contrastive-learning method that improves one baseline on the synthetic benchmark.","lead":"This paper builds new LiDAR tracking benchmarks for rain, fog, and snow, and measures how five existing 3D trackers degrade under them. It also adds a domain-randomization plus contrastive-learning method, DRCT, that improves one baseline tracker on the synthetic weather benchmark.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DRCT is validated only on synthetic KITTI-A; no CADC-SOT evaluation, so the central claim that it improves real adverse-weather tracking is unsupported.","rationale":"I considered the Eq. (2) metric inconsistency (DR defined as S_i,j/S_c but reported as 1-S_i,j/S_c) and the undisclosed hyperparameters; these are real but secondary because they do not change the qualitative performance ordering. The most load-bearing issue is external validity: the benchmark's purpose is real-world robustness, yet the method that is supposed to restore robustness is never tested on the only real-world data in the benchmark. This is a gap in the argument, not an internal contradiction; it can be closed by one experiment. The reader identified synthetic realism as the weakest assumption and separately noted that DRCT is evaluated only on KITTI-A; this stress-test focuses on the latter and ties it to the CADC evidence that already exists. Verdict remains CONDITIONAL because the existing synthetic results and baseline CADC results support a benchmark contribution, but the headline generalization about DRCT should not be accepted without a real-world check.","tokens_in":21912,"tokens_out":4923,"duration_ms":48053,"concrete_test":"Evaluate the DRCT model on the CADC-SOT benchmark under the same protocol used for the baseline trackers in Section IV.B (train on nuScenes, report success/precision by snow level and road coverage). Compare DRCT against the MBPTrack baseline on identical CADC-SOT sequences; additionally compute the per-sequence degradation relative to a clean nuScenes/KITTI reference to test whether synthetic KITTI-A degradation rates predict CADC-SOT degradation rates. If DRCT does not improve over MBPTrack on CADC-SOT, or if CADC-SOT degradation patterns diverge from KITTI-A, the central generalization claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central motivation is robustness in real adverse weather, but the only evidence for DRCT is Table VI, which is restricted to the synthetic KITTI-A split (Section V.D). The real-world CADC-SOT results are reported only for the five baseline trackers (Table V, Section IV.B), never for DRCT. This matters because the CADC-SOT results undercut the degradation narrative: Section IV.B.1 states that 'snow levels did not show a negative correlation with tracker performance under real weather conditions and snow-covered roads did not lead to performance degradation.' Without a clean-baseline comparison on CADC, the paper cannot show real-world degradation, and without DRCT on CADC it cannot show that the proposed method transfers from synthetic corruption to physical snow. The abstract and conclusion nevertheless claim 'excellent performance in benchmarks.' The load-bearing assumption is that KITTI-A synthetic weather is a valid proxy for real adverse weather; the paper provides no internal test of this assumption, and the real-world evidence it does include is not consistent with a simple severity-driven degradation story.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces a benchmark for evaluating 3D single-object tracking (3DSOT) under adverse weather, consisting of two synthetic datasets (KITTI-A and nuScenes-A, generated by applying fog, rain, and snow simulations to clean KITTI and nuScenes data) and one real-world dataset (CADC-SOT, filtered from CADC snowfall sequences). Five representative trackers (BAT, MMTrack, STNet, CXTrack, MBPTrack) are evaluated on the benchmark, and the reported degradation rates are used to analyze the impact of target distance, template shape corruption, and target shape corruption. The paper then proposes DRCT, a dual-branch extension of MBPTrack that combines domain randomization in an auxiliary branch with a local geometric contrastive learning module, and reports improved success/precision over the MBPTrack baseline on KITTI-A.","tokens_in":22065,"tokens_out":4223,"duration_ms":43508,"significance":"The paper addresses a genuine gap: no adverse-weather benchmark currently exists for 3D single-object tracking. The synthetic corruption pipeline builds on established physically motivated simulators, and the real-world CADC-SOT filtering is a useful resource that could support future robustness studies. The systematic evaluation of five trackers across weather types and intensity levels provides a useful reference point for the community. The DRCT experiment is also framed honestly as an improvement over an external baseline, MBPTrack, rather than over a quantity fitted from the data. However, the significance of the central robustness claim is currently limited by the absence of DRCT results on real-world data and by several specification gaps that prevent reproduction.","major_comments":[{"comment":"Equation (2) defines DR as S_i,j / S_c, i.e., the retained performance ratio, but the values reported as DR in Tables III and IV are clearly computed as 1 - S_i,j / S_c. For example, MBPTrack car/rain in Table III has clean success 73.70 and average KITTI-A success 48.71; the reported DR is 0.34, which equals 1 - 48.71/73.70, whereas Eq. (2) would give 0.66. This internal inconsistency affects every degradation-rate claim in the paper, including the statement in Section IV.A.1 that car performance \"decreases by 32%-42%.\" The definition or the table values must be corrected so that the reported numbers match the stated formula.","section":"Section III.D, Eq. (2) and Tables III, IV"},{"comment":"DRCT is evaluated only on the synthetic KITTI-A split; Table VI contains no results on the real CADC-SOT dataset or on nuScenes-A. The abstract and conclusion claim that DRCT achieves \"excellent performance in benchmarks,\" but the only evidence is a synthetic benchmark whose real-world validity is not established. This is especially problematic because Section IV.B.1 reports that on the real CADC-SOT data, \"snow levels did not show a negative correlation with tracker performance under real weather conditions and snow-covered roads did not lead to performance degradation.\" That observation undercuts the premise that real adverse weather uniformly degrades tracking and leaves open the question of whether the synthetic KITTI-A improvements transfer to physically collected snowfall. The authors should either report DRCT on CADC-SOT or substantially qualify the claim that DRCT improves real-world adverse-weather tracking.","section":"Section V.D and Section IV.B.1"},{"comment":"Several parameters that are load-bearing for reproducing the benchmark and the method are not disclosed. Section III.B says the fog simulation sets the meteorological optical range to \"five discrete values\" and that rain and snow simulations set rainfall and snowfall rates, but no numerical values are given. Algorithm 1 depends on Nmax and the jitter bound a, and Eq. (4) depends on the neighbor count K, yet none of these values appears in the experimental section. The paper should report these values (or release the exact configuration code) so that the benchmark levels and the DRCT results are reproducible.","section":"Section III.B, Algorithm 1, and Section V.E"},{"comment":"The narrative describes the pedestrian rain results as showing a \"small performance drop except CXTrack,\" but Table III lists negative DR values for BAT (-0.05/-0.07) and near-zero values for MBPTrack (0.005/0.004) in the pedestrian rain rows. Negative degradation rates mean the trackers improved under rain, which is qualitatively different from a small drop and should be acknowledged and interpreted. This also raises a question about whether the degradation-rate formulation, even once corrected, is appropriate for cases where S_i,j exceeds S_c.","section":"Section IV.A.1 and Table III"}],"minor_comments":[{"comment":"Reference [10] cites a paper on graph convolutional networks that is not the point-cloud DGCNN architecture used by MBPTrack and CXTrack; the appropriate citation is Wang et al., \"Dynamic Graph CNN for Learning on Point Clouds,\" ACM Transactions on Graphics, 2019.","section":"References"},{"comment":"There is a typo in the opening sentence: \"nuScnenes-A\" should be \"nuScenes-A.\"","section":"Section IV.A"},{"comment":"The caption contains a duplicated phrase: \"THE NUMBER OF SCENARIOS AND FRAMES IN THE TABLE SHOWS THE NUMBER OF SCENARIOS AND FRAMES.\" It should be shortened to a single clause.","section":"Table II caption"},{"comment":"The vertical axis of Figure 3 is labeled \"IOU deviation,\" but this quantity is not defined in Section III or Section IV. Please define it explicitly, including how it is aggregated over sequences and frames.","section":"Figure 3"},{"comment":"The column labeled \"RD\" is not defined in the text, and the meaning of \"KITTI\" versus \"Clean\" in the first two columns is unclear. Clarify which split the \"Clean\" column refers to and what \"RD\" denotes.","section":"Table VII"},{"comment":"The heading \"Local Geometric Contrast Learning\" appears to be missing the suffix \"ive\"; it should read \"Local Geometric Contrastive Learning.\"","section":"Section V.C"}],"recommendation":"major_revision","confidential_remarks":"The benchmark construction is potentially a useful community resource, and the baseline evaluation is carefully organized. The main risk is that the method paper claims real-world robustness while only validating on synthetic data, and the real-world results in Section IV.B actually complicate the degradation narrative. I believe this is fixable within the scope of a revision by adding CADC-SOT results for DRCT (or at least a clean-to-adverse comparison on CADC-SOT for the baseline), correcting the DR formula, and disclosing all simulation and training hyperparameters."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline: this is the first benchmark effort for 3D LiDAR single-object tracking under adverse weather, and it includes a solid evaluation of five trackers on synthetic and real snow data. The DRCT method is a minor add-on that improves MBPTrack on synthetic KITTI-A only; the real-world transfer claim is not supported.\n\nWhat is actually new: KITTI-A, nuScenes-A, and CADC-SOT. The synthetic simulation using LISA and fog sim [40,41] with five levels per weather is reasonable. The evaluation across five trackers (Tables III–V) shows consistent degradation on synthetic data, especially pedestrians under fog. The analysis by target distance, template corruption, and target corruption is a useful diagnostic and could guide future work. CADC-SOT construction from CADC is a plausible filtering effort. The DRCT idea of domain randomization in an auxiliary branch plus contrastive alignment is not wildly novel but is cleanly described, and the ablation against MBPTrack-RD shows the strategy helps.\n\nSoft spots, in order of importance. Equation (2) defines DR as S_i,j/S_c, but the tables report 1 - S_i,j/S_c; the reader caught this and it is a real internal inconsistency that needs fixing, though it does not change the conclusions. More important: DRCT is evaluated only on KITTI-A, not on nuScenes-A or CADC-SOT. The abstract and conclusion claim \"excellent performance in benchmarks,\" which overreaches. Given the paper's motivation is real-world robustness, the absence of DRCT on CADC-SOT is a significant missing experiment. The CADC results themselves undercut the simple degradation story: snow level did not correlate with performance, and snow-covered roads did not cause degradation. The paper reports this but does not reconcile it with the synthetic results, nor does it test whether synthetic corruption transfers to real snow. Hyperparameters for DRCT and the simulation intensity values are not disclosed, and no code or dataset release is mentioned; that weakens reproducibility for a paper whose main contribution is the datasets themselves.\n\nWho this is for: researchers in 3D SOT and robustness evaluation. The benchmark fills a real gap and the evaluation is worth having, but the method part should be treated as preliminary.\n\nRecommendation: it deserves serious peer review. A referee should require the dataset/code release, fix the DR definition, and either add DRCT results on CADC-SOT or temper the claims in the abstract and conclusion.","headline":"First adverse-weather benchmark for 3D LiDAR single-object tracking, with a clean evaluation of five trackers; the DRCT method is a modest synthetic-only improvement whose real-world transfer is unsupported.","tokens_in":22668,"tokens_out":2110,"would_cite":true,"duration_ms":21606,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adverse weather systematically degrades LiDAR 3D trackers, and the proposed DRCT training framework recovers a substantial share of the lost performance.","keywords":["3D single object tracking","LiDAR point clouds","adverse weather","robustness benchmark","domain randomization","contrastive learning","KITTI-A","CADC-SOT"],"falsifier":"Run both MBPTrack and DRCT on a real-world rainy, foggy, and snowy tracking set collected with the same sensor geometry and compare their degradation rates; if DRCT's KITTI-A advantage does not persist, or if the synthetic degradation rates fail to reproduce on real weather, the benchmark's representativeness claim collapses.","tokens_in":21609,"feed_emoji":"🌧️","tokens_out":6334,"duration_ms":58482,"temperature":0.7,"pith_summary":"3D single object tracking in LiDAR point clouds is normally evaluated only on clear weather. This paper argues that this hides a serious robustness problem: rain, fog, and snow alter the point cloud distribution, and five representative trackers lose a large share of accuracy on a new adverse-weather benchmark. To make the argument, the authors build two synthetic benchmark sets, KITTI-A and nuScenes-A, with five intensity levels of rain, fog, and snow, and one real snowy set, CADC-SOT, filtered from an existing capture. They then identify three factors behind the failures: target distance, template shape corruption, and target shape corruption. Finally, they propose a dual-branch tracking framework, DRCT, that trains an auxiliary branch on domain-randomized point clouds and transfers robustness to the main branch through local geometric contrastive learning, improving both clean and weather-affected performance over a strong MBPTrack baseline.","feed_headline":"Rain, fog, and snow cut LiDAR tracker scores by up to 70 percent","feed_subtitle":"A new three-dataset benchmark measures the damage; a domain-randomized contrastive tracker, DRCT, wins back several points.","key_machinery":"The argument rests on two constructed objects. The benchmark is three tracking datasets: KITTI-A and nuScenes-A, made by applying fog simulation and rain/snow simulation to clean KITTI and nuScenes validation sequences at five intensities, plus CADC-SOT, made by filtering the real snowy CADC scans to sequences whose targets have at least 10 points and more than four frames. The second object is DRCT, a dual-branch training framework built on MBPTrack: the primary branch processes the original clean point cloud, the auxiliary branch processes the same cloud after domain randomization (random noise addition, point dropout, and jittering), and a local geometric contrastive module aligns neighborhood features around common key points using ball query and max pooling, transferring corruption-robust features into the primary branch. A degradation-rate metric, defined as the ratio of weather-affected success or precision to the clean value, quantifies retained accuracy per weather type and level.","core_discovery":"The paper's central claim is that state-of-the-art 3D single object trackers are not robust to adverse weather, and that the failure is systematic enough to be benchmarked and partly fixed. On KITTI-A, car tracking success drops by roughly a third across rain and snow and by 17-24% in fog, while fog cuts pedestrian success by 51-70%; on the sparser nuScenes-A, degradation is even larger and some pedestrian cases approach failure. The authors trace the drop to three causes: distance-dependent signal loss, corruption of the template point cloud, and corruption of the search-area target. They then present DRCT, a dual-branch extension of MBPTrack in which one branch sees clean data, the other sees point clouds corrupted by random noise, dropout, and jitter, and local geometric contrastive learning pulls the clean branch toward the robust representations of the randomized branch. DRCT improves MBPTrack's mean success by +2.51/+1.23/+1.88 points on KITTI-A cars under rain, fog, and snow, while also gaining +1.60/+1.11 on clean KITTI.","pith_inferences":["The paper evaluates DRCT only on synthetic KITTI-A and never on the real-world CADC-SOT, so the most direct open test is whether the synthetic-trained gains transfer to real snow, where the paper itself finds no monotonic level-performance relationship.","The distance analysis suggests that denser LiDAR mitigates weather damage, implying beam-count augmentation or sensor-fusion input could be an explicit robustness lever beyond training strategy.","Because the synthetic weather is generated frame-by-frame from clean scans, dynamic physical effects such as snow accumulation on objects or wet-road returns are absent, so a real-weather benchmark including those effects may show different failure modes."],"forward_implications":["Robustness evaluation under rain, fog, and snow should become a standard axis for 3D single object tracking, since clean-dataset rankings alone overstate real-world capability.","MBPTrack, the strongest of the five evaluated trackers on KITTI-A, still loses about a third of its car success in rain and snow, so even the best current methods are not weather-proof.","Sparser LiDAR (nuScenes 32-beam) suffers larger degradation than KITTI's 64-beam, especially for pedestrians, making weather robustness also a sensor-resolution problem.","DRCT's asymmetric dual-branch training with local geometric contrastive learning raises both clean and weather-affected performance over its MBPTrack baseline, while training directly on random domains alone does not.","Template shape corruption is a major failure driver: even small template changes produce large IOU drops, so robust template representation is key for future trackers."],"supporting_citations":[{"why":"Supplies the LiDAR snowfall and rain simulation algorithms used to build the synthetic KITTI-A and nuScenes-A point clouds.","marker":"[40]"},{"why":"Supplies the fog simulation method used to generate the foggy levels of KITTI-A and nuScenes-A.","marker":"[41]"},{"why":"Provides the raw Canadian Adverse Driving Conditions dataset from which CADC-SOT is filtered and graded by snow level.","marker":"[25]"},{"why":"Defines MBPTrack, the strongest baseline in the evaluation and the base tracker on which DRCT is built.","marker":"[11]"},{"why":"Defines MMTrack, the motion-based tracker used as one of the five representative methods in the robustness evaluation.","marker":"[13]"},{"why":"Defines BAT, one of the five representative trackers whose degradation under adverse weather is measured.","marker":"[29]"},{"why":"Defines STNet, the transformer-based tracker included in the robustness evaluation.","marker":"[8]"},{"why":"Defines CXTrack, the contextual-information tracker included in the robustness evaluation.","marker":"[12]"}],"fun_headline_variants":["LiDAR trackers lose up to 70% in bad weather","New benchmark shows weather's toll on 3D tracking","DRCT bolsters LiDAR tracking in rain, fog, snow","Why bad weather breaks LiDAR trackers—and a fix","Benchmark exposes LiDAR tracking's weather weakness"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the simulated rain, fog, and snow point clouds in KITTI-A and nuScenes-A are faithful enough to real adverse weather for the measured degradation rates and the DRCT improvement to transfer to physical conditions.","fun_headline_variants_meta":{"raw":{"variants":["LiDAR trackers lose up to 70% in bad weather","New benchmark shows weather's toll on 3D tracking","DRCT bolsters LiDAR tracking in rain, fog, snow","Why bad weather breaks LiDAR trackers—and a fix","Benchmark exposes LiDAR tracking's weather weakness"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000271,"raw_usage":{"total_tokens":1676,"prompt_tokens":1038,"completion_tokens":638,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":654,"completion_tokens_details":{"reasoning_tokens":554}},"tokens_in":654,"tokens_out":638,"duration_ms":5745,"temperature":1.0,"reasoning_tokens":554,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:49:08.116596+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run both MBPTrack and DRCT on a real-world rainy, foggy, and snowy tracking set collected with the same sensor geometry and compare their degradation rates; if DRCT's KITTI-A advantage does not persist, or if the synthetic degradation rates fail to reproduce on real weather, the benchmark's representativeness claim collapses.","supporting_citations":[{"cited_title":"Lidar snowfall simulation for robust 3d object detection,","cited_arxiv_id":null,"evidence_quote":"Supplies the LiDAR snowfall and rain simulation algorithms used to build the synthetic KITTI-A and nuScenes-A point clouds."},{"cited_title":"Canadian adverse driving conditions dataset,","cited_arxiv_id":null,"evidence_quote":"Provides the raw Canadian Adverse Driving Conditions dataset from which CADC-SOT is filtered and graded by snow level."},{"cited_title":"Mbptrack: Improving 3d point cloud tracking with memory networks and box priors,","cited_arxiv_id":null,"evidence_quote":"Defines MBPTrack, the strongest baseline in the evaluation and the base tracker on which DRCT is built."},{"cited_title":"Beyond 3D Siamese Tracking: A Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds","cited_arxiv_id":"2203.01730","evidence_quote":"Defines MMTrack, the motion-based tracker used as one of the five representative methods in the robustness evaluation."},{"cited_title":"Box- aware feature enhancement for single object tracking on point clouds,","cited_arxiv_id":null,"evidence_quote":"Defines BAT, one of the five representative trackers whose degradation under adverse weather is measured."},{"cited_title":"3d siamese transformer network for single object tracking on point clouds,","cited_arxiv_id":null,"evidence_quote":"Defines STNet, the transformer-based tracker included in the robustness evaluation."},{"cited_title":"Cxtrack: Improving 3d point cloud tracking with contextual information,","cited_arxiv_id":null,"evidence_quote":"Defines CXTrack, the contextual-information tracker included in the robustness evaluation."}],"review_version":1}