{"id":"a4e9b463-6fb8-4a22-8289-f792fed934f7","arxiv_id":"2504.21699","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"REHEARSE-3D provides 9.2 billion point-wise annotated LiDAR-256 and 4D radar points in emulated rain, plus a benchmark for raindrop detection and removal.","lead":"REHEARSE-3D is a new large-scale dataset of rainy 3D point clouds for teaching self-driving perception systems to detect and remove raindrop noise from LiDAR and 4D radar. It uniquely combines 256-line LiDAR, 4D radar, day and night scenes, and three controlled rain intensities with point-wise labels.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Rain labels defined as residual unclassified points; if Step 6 is applied to clean sequences, the dataset contains false rain labels and the de-raining benchmark is invalid.","rationale":"The reader identified the same core weakness: the residual definition of rain points in annotation Step 6 conflates rain with other unclassified artifacts. My analysis sharpens this by pointing out the likely application to clean sequences, which would make the error explicit and severe. The paper's ambiguity on whether Step 6 is conditional on rainy weather is central; if clean sequences contain 'rain' labels, the dataset's central claim collapses. This is exactly the kind of issue that the reader's CONDITIONAL verdict anticipates. No change to the verdict is needed because the concern is already captured; the proposed concrete test would either confirm the flaw or resolve it. I do not see a separate, more load-bearing concern than this one. The radar label transfer issue is secondary because it inherits the same invalid rain labels. The 9.2 billion annotation count and the 'first comprehensive multi-modal' claim are only as strong as the label validity, so this is the single point on which the paper stands or falls.","tokens_in":12159,"tokens_out":3053,"duration_ms":32052,"concrete_test":"Inspect the released REHEARSE-3D dataset: count points labeled 'rain' in the clean-weather sequences. If this count is non-negligible (e.g., >0.1% of points), the annotation rule is inventing rain where none physically exists. Additionally, run Step 6 of the annotation pipeline on a held-out clean scan and visually or automatically compare the resulting 'rain' points against RGB/thermal images; if they coincide with dust, sensor noise, or edge artifacts rather than water droplets, the ground-truth rain labels are unreliable. A complementary check: have human annotators label a small subset of rainy frames and measure agreement with the automatic residual labels. If agreement is low (e.g., IoU < 0.8), the benchmark metrics are not trustworthy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that REHEARSE-3D provides reliable point-wise rain annotations for benchmarking de-raining. The annotation pipeline (Section III-B, Step 6) labels every point above the road plane, inside the 2D road polygon, and not assigned to another class as a raindrop. This makes 'rain' a residual category: any unmodeled artifact (sensor noise, dust, edge effects, missed objects) in the road area is automatically labeled rain. The paper does not state that Step 6 is applied only to rainy sequences; it describes a single pipeline for 'clean and rainy' data. Since 39% of sequences are clean (Section III-C), applying the same rule would label non-rain points as rain in clear weather. Figure 5 even appears to show a rain class in the 'clear' condition. If this is true, the ground-truth labels are circular and the benchmark scores in Table II/III conflate rain detection with residual-noise classification. A model that predicts 'rain' for every point outside the predefined object boxes would score perfectly, not because it detects rain but because the labels are defined that way. This directly undermines the dataset's novelty and the validity of all downstream comparisons.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces REHEARSE-3D, a multi-modal dataset of dense LiDAR-256 and 4D Radar point clouds captured in a controlled outdoor rain facility, with point-wise semantic annotations for eight classes. The dataset includes 143 sequences from the existing REHEARSE dataset, covering clean conditions and three emulated rain intensities (10, 25, and 50 mm/h), in daytime and nighttime. The authors also generate a simulated rain version by applying a physically based LiDAR rain model to clean point clouds. They benchmark several statistical filters (DROR, DSOR) and deep-learning models (3D-OutDet, SalsaNext, LiSnowNet-L1) on the task of detecting and removing rain points in early-fused LiDAR and Radar clouds, reporting precision, recall, F1, and mIoU. The central claims are that REHEARSE-3D is the largest point-wise annotated weather dataset, the first multi-modal dense LiDAR/4D Radar rain dataset, and a useful benchmark for point cloud de-raining.","tokens_in":12422,"tokens_out":5226,"duration_ms":57780,"significance":"If the point-wise rain labels are valid, REHEARSE-3D would be a valuable community resource: it is substantially larger and denser than existing annotated weather point-cloud datasets (e.g., WeatherNet, WADS, SemanticSpray), it is the first to combine 256-line MEMS LiDAR with 4D Radar point-wise labels under controlled rain, and it provides precipitation characteristics (intensity, droplet size distribution, wind, visibility) that are absent from most related datasets. The benchmark is useful and appears reproducible: the splits are described, the evaluation protocol is standard, and the authors plan to release the dataset and models. The use of a physically calibrated rain simulation applied to real clean scans is a methodological strength that enables an emulated-to-simulated domain-gap study. However, these contributions rest on the validity of the 'rain' label and of the radar label transfer, and the manuscript currently does not establish that validity rigorously.","major_comments":[{"comment":"The rain label is defined as a residual category: any point above the road plane, within the 2D road polygon, and not assigned to one of the hand-labeled classes is automatically labeled as a raindrop. The paper does not state that this rule is applied only to rainy sequences, and Figure 5 appears to show a non-negligible rain class under the 'clear' condition. Under this definition, sensor noise, dust, edge artifacts, and missed objects in the road area are all labeled as rain, so the ground-truth labels and the benchmark metrics in Tables II and III measure residual-point classification rather than rain detection. A model that flags every unassigned point as rain would obtain high scores by construction, not because it detects rain. This is load-bearing because the central contribution is point-wise rain annotation. Please (i) clarify precisely how Step 6 was applied to clean sequences, (ii) report the number and fraction of rain-labeled points in clean conditions, and (iii) validate a random sample of rain labels against manual/visual annotation or against the independently measured rain intensity maps, including regions with low measured intensity where false positives are most likely.","section":"Section III-B, Step 6"},{"comment":"4D Radar points are annotated by transferring labels from the nearest LiDAR point, with no threshold or validation reported. Since LiDAR and 4D Radar have different sampling geometries, spatial resolutions, and occlusion patterns, nearest-neighbor transfer can assign object or rain labels to radar points that correspond to different physical scatterers. This is particularly relevant for the rain class, where radar returns may come from water droplets at different distances than the nearest LiDAR return. Please report the distribution of nearest-neighbor distances, validate a sample of radar labels manually, and either restrict label transfer to points within a physically motivated distance or provide an alternative annotation procedure for radar points.","section":"Section III-B, Step 10"}],"minor_comments":[{"comment":"The figure would benefit from a clearer legend and explicit per-condition class counts; in particular, it should be immediately clear whether the 'clear' condition contains any rain-labeled points and, if so, how many.","section":"Section III-C, Figure 5"},{"comment":"The polar-grid projection assigns each 3D point to the nearest calibrated elevation and azimuth index, but the pseudocode does not describe how multiple points mapping to the same grid cell are resolved, nor how invalid or duplicate cells are handled. Please add this detail, since it affects the unreturned-beam reconstruction and thus the simulated rain data.","section":"Section III-E, Algorithm 1"},{"comment":"The text introduces Radius Outlier Removal and Statistical Outlier Removal as baselines but reports results only for DROR and DSOR. Either explain why ROR and SOR are omitted or remove them from the baseline description.","section":"Section IV-A"},{"comment":"The near-perfect scores of 3D-OutDet on the WMG simulated data are expected because the simulated rain is generated algorithmically from clean scans and the same algorithm is used to create the labels; the discussion should more explicitly warn readers that these numbers do not represent real-world detection performance and should not be used to calibrate expectations on physical rain data.","section":"Section IV-B, Table III"},{"comment":"The RANSAC road-plane estimation and the subsequent manual bounding-box correction are described briefly; please state how many points fall outside the estimated plane and how sensitive the rain-label count is to the RANSAC threshold, since Step 6 relies directly on this plane.","section":"Section III-B, Step 2"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: REHEARSE-3D is a genuinely new resource—high-res MEMS LiDAR-256 plus 4D radar, point-wise labels, day/night, three controlled rain intensities, and rain-characteristics metadata. That combination doesn't exist elsewhere. The paper is clearly written, the benchmark is a useful starting point, and the limitations section is honest about static scenes and the emulated-to-real gap.\n\nBut check the annotation pipeline before trusting the labels. Step 6 of Section III-B labels every point above the road plane, inside the road polygon, that isn't already assigned to another class as a raindrop. That makes 'rain' a residual category, not a physically validated class. The paper doesn't say this step is applied only to rainy sequences; it describes a single pipeline for clean and rainy data. Figure 5 appears to show a rain class in the clear condition. If that's right, clean sequences contain false rain labels, and the benchmark tables measure residual-point classification rather than rain detection. A model that calls every unclassified road-area point 'rain' would score perfectly by construction. Even if step 6 is restricted to rainy sequences, sensor noise, dust, edge artifacts, and missed objects all become rain by definition, and no label-validation evidence is given. Radar labels inherit these problems via nearest-LiDAR transfer.\n\nThe WMG simulated-data experiment is also softer than it looks. The simulation uses Espineira et al.'s model, which has overlapping authorship with this paper, and 3D-OutDet—also from the same group—gets near-100% on it. That is a sensible sanity check, not a meaningful comparison.\n\nMinor issues: no error bars, hyperparameters tuned on a 100-sample subset, and the dataset isn't downloadable yet.\n\nThe central dataset claim is novel and worth having. The label validity problem is addressable: report rain labels for clean and rainy sequences separately, show false-label rates on clean data, and validate a sample of rain labels manually. If the authors can do that, this becomes a solid benchmark. As it stands, the benchmark numbers are not reliable until then.\n\nMy recommendation: send it to peer review, but ask for those label-validation experiments before acceptance. The resource is valuable enough to warrant referee time. I'd read it with the annotation definition in mind.","headline":"A valuable multimodal rain dataset whose benchmark numbers rest on a residual rain label definition that likely assigns false rain points in clean sequences.","tokens_in":12943,"tokens_out":3238,"would_cite":false,"duration_ms":33476,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"REHEARSE-3D is a 9.2-billion-point multimodal dataset that labels raindrops point-wise in LiDAR-256 and 4D Radar clouds across day and night emulated rain.","keywords":["point cloud de-raining","emulated rain dataset","LiDAR-256","4D Radar","semantic annotation","autonomous driving","adverse weather","sensor noise modeling"],"falsifier":"Run the same annotation pipeline on the clean-weather REHEARSE-3D sequences and count how many points are labeled rain despite no rain being present; if the count is substantial, the residual-class rule contaminates the ground truth. A second check is to compare the labeled rain points against a physically validated raindrop detector on the same scenes.","tokens_in":11979,"feed_emoji":"🌧️","tokens_out":8419,"duration_ms":80620,"temperature":0.7,"pith_summary":"REHEARSE-3D is a large multi-modal dataset built to support 3D point-cloud de-raining: detecting and removing raindrop returns in LiDAR and radar data before downstream perception. The paper's central claim is that this is the first dataset with point-wise semantic labels on high-resolution LiDAR-256 and 4D Radar point clouds across daytime and nighttime emulated rain, totaling 9.2 billion annotated points. The authors also benchmark five existing de-raining methods on the dataset and create a simulated rain version from clean scans to quantify the gap between physically simulated and sprinkler-emulated rain. If the dataset is sound, autonomous-driving research gets a high-density, multimodal, rain-characterized test bed in a domain where annotated adverse-weather data is scarce.","feed_headline":"9.2 billion labeled rain points set a new de-raining benchmark","feed_subtitle":"First multimodal emulated rain dataset with LiDAR-256, 4D radar, and per-point rain labels day and night","key_machinery":"The load-bearing object is the annotation protocol rather than a new network. It defines rain operationally: estimate the road plane with RANSAC, draw bounding boxes around known objects and sprinklers, draw a 2D polygon around the road, and treat every remaining point inside that polygon above the plane as a raindrop. This residual-class rule is what produces the 9.2 billion point-wise labels, and it is also the most delicate assumption in the paper. The secondary mechanism is the polar-grid-map preprocessing that reconstructs unreturned LiDAR beams, allowing an existing physics-based rain model to be applied to real clean scans to create the simulated benchmark.","core_discovery":"REHEARSE-3D is a semantically annotated version of an existing controlled-weather dataset. It contains 143 sequences, each with roughly 300 dense LiDAR scans, captured with a high-resolution 256-line MEMS LiDAR and a 4D Radar at a fixed outdoor test track under sprinkler-generated rain at 10, 25, and 50 mm/h, in daytime and nighttime, plus clean conditions. Every point in the merged LiDAR-radar cloud is labeled as one of eight classes: rain, car, pedestrian, bike, sprinkler, targets, road, and background. The rain label is produced by the annotation rule that any point above the estimated road plane, inside the road polygon, and not assigned to a known object is a raindrop. The paper further contributes de-raining benchmark results for statistical filters and deep networks, and a simulated rain counterpart built by applying an established LiDAR rain model to clean scans, used to measure the emulated-to-simulated gap.","pith_inferences":["The residual-class rain definition means benchmark numbers measure detection of unexplained points in the road area. A control experiment running the same annotation on clean sequences would quantify how much sensor noise and dust are labeled as rain.","Radar labels are copied from the nearest LiDAR point. Because rain degrades LiDAR measurements, the radar rain labels may be spatially misaligned or missing in exactly the heavy-rain frames where radar would be most useful.","The reported gap between simulated and emulated rain can be used as a target for rain simulators: a simulator that closes that gap on this dataset would be a stronger candidate for transfer to real rain.","Because all scenes are static front-view captures, the dataset does not exercise motion artifacts or multi-frame temporal cues; de-raining models that exploit sequence information would need additional data."],"forward_implications":["Supervised models trained on these labels outperform unsupervised statistical filters by large margins (F1 around 97% versus roughly 34% at best on the test split), indicating that labeled rain data, not algorithm design, is the main bottleneck.","Because the dataset includes radar with transferred labels, fused LiDAR-radar de-raining can be benchmarked for the first time, opening a path to using weather-resilient radar to clean LiDAR.","Rain characteristics (intensity, droplet size distribution, wind, visibility) are provided per sequence, so point-level rain behavior can be tied to physical weather parameters rather than treated as generic noise.","The simulated-vs-emulated comparison shows a large domain gap (near-perfect scores on simulated rain, lower on emulated), so the dataset can serve as a calibration yardstick for rain simulators.","High-density LiDAR-256 labels reduce the sparsity problem that limited earlier 32- and 64-beam annotated weather datasets."],"supporting_citations":[{"why":"Original REHEARSE dataset that supplies the sensor suite, outdoor rain facility, and weather-characterization measurements.","marker":"[9]"},{"why":"Closest prior annotated emulated rain/fog dataset; the comparison table uses its 32-beam LiDAR and annotation scope to motivate higher density.","marker":"[6]"},{"why":"Provides the DSOR filter evaluated here and prior point-wise snow annotations used as a baseline in the comparison table.","marker":"[8]"},{"why":"Prior point-wise annotated LiDAR spray/rain data; part of the comparison establishing the need for a larger multi-modal dataset.","marker":"[5]"},{"why":"Clean-weather LiDAR benchmark used in Table I as the size-comparison reference for annotated point counts.","marker":"[26]"},{"why":"Simulated snow dataset used in Table I to represent LiDAR-only annotated adverse-weather data.","marker":"[27]"},{"why":"Source of the 3D-OutDet supervised baseline and the training protocol, evaluation metrics, and runtime numbers the benchmark follows.","marker":"[3]"},{"why":"Physics-based LiDAR rain model applied to clean scans to build the simulated rain counterpart.","marker":"[31]"},{"why":"SalsaNext semantic segmentation network used as a supervised de-raining baseline.","marker":"[32]"},{"why":"LiSnowNet-L1 unsupervised snow-removal network evaluated as a raindrop-detection baseline.","marker":"[33]"}],"fun_headline_variants":["Largest point-wise annotated rain dataset for 3D de-raining","Only rain dataset with LiDAR-256 and 4D radar point clouds","Day and night rain dataset with per-point labels for de-raining","Controlled-weather rain dataset for benchmarking 3D point cloud de-raining"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The rain labels are valid only if every residual point above the road plane inside the road polygon is actually a raindrop; anything else there—sensor noise, dust, edge artifacts, or an unmodeled object—gets mislabeled as rain.","fun_headline_variants_meta":{"raw":{"variants":["Largest point-wise annotated rain dataset for 3D de-raining","Only rain dataset with LiDAR-256 and 4D radar point clouds","Day and night rain dataset with per-point labels for de-raining","Controlled-weather rain dataset for benchmarking 3D point cloud de-raining"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001682,"raw_usage":{"total_tokens":6708,"prompt_tokens":1028,"completion_tokens":5680,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":644,"completion_tokens_details":{"reasoning_tokens":5599}},"tokens_in":644,"tokens_out":5680,"duration_ms":41680,"temperature":1.0,"reasoning_tokens":5599,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:55:42.464729+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same annotation pipeline on the clean-weather REHEARSE-3D sequences and count how many points are labeled rain despite no rain being present; if the count is substantial, the residual-class rule contaminates the ground truth. A second check is to compare the labeled rain points against a physically validated raindrop detector on the same scenes.","supporting_citations":[{"cited_title":"Rehearse: adverse weather dataset for sensory noise models,","cited_arxiv_id":null,"evidence_quote":"Original REHEARSE dataset that supplies the sensor suite, outdoor rain facility, and weather-characterization measurements."},{"cited_title":"Cnn- based lidar point cloud de-noising in adverse weather,","cited_arxiv_id":null,"evidence_quote":"Closest prior annotated emulated rain/fog dataset; the comparison table uses its 32-beam LiDAR and annotation scope to motivate higher density."},{"cited_title":"Energy-based detection of adverse weather effects in lidar data,","cited_arxiv_id":null,"evidence_quote":"Prior point-wise annotated LiDAR spray/rain data; part of the comparison establishing the need for a larger multi-modal dataset."},{"cited_title":"Towards 3D LiDAR -based semantic scene under -standing of 3D point cloud sequences: The SemanticKITTI Dataset,","cited_arxiv_id":null,"evidence_quote":"Clean-weather LiDAR benchmark used in Table I as the size-comparison reference for annotated point counts."},{"cited_title":"4denoisenet: Adverse weather denoising from adjacent point clouds,","cited_arxiv_id":null,"evidence_quote":"Simulated snow dataset used in Table I to represent LiDAR-only annotated adverse-weather data."},{"cited_title":"3d-outdet: A fast and memory efficient outlier detector for 3d lidar point clouds in adverse weather,","cited_arxiv_id":null,"evidence_quote":"Source of the 3D-OutDet supervised baseline and the training protocol, evaluation metrics, and runtime numbers the benchmark follows."},{"cited_title":"Realistic lidar with noise model for real -time testing of automated vehicles in a virtual environment,","cited_arxiv_id":null,"evidence_quote":"Physics-based LiDAR rain model applied to clean scans to build the simulated rain counterpart."},{"cited_title":"Lisnownet: Real -time snow removal for lidar point clouds,","cited_arxiv_id":null,"evidence_quote":"LiSnowNet-L1 unsupervised snow-removal network evaluated as a raindrop-detection baseline."}],"review_version":1}