{"id":"82b0ebc7-5efe-4a8d-af40-fe58dfcd4a81","arxiv_id":"2412.18255","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"AdaCo refurbishes noisy VFM-generated 3D pseudo labels using adaptive early-learning correction and robust losses, reaching 25.7% and 31.2% mIoU on SemanticKITTI and nuScenes without 3D annotations.","lead":"This paper introduces AdaCo, a label-free 3D semantic segmentation method that generates pseudo labels from visual foundation models and then iteratively corrects noisy labels during training. It reports large gains over prior label-free baselines on SemanticKITTI and nuScenes, suggesting that noisy pseudo-label correction is a key bottleneck.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (2) as written selects maximally uncertain points as 'reliable', inverting the ANC selection criterion; the label-correction gain is untestable until this is resolved.","rationale":"The reader's weakest-assumption analysis and mine converge on the same load-bearing defect: Eq. (2)'s confidence score is internally inconsistent with the stated goal of selecting consistent historical predictions. Since ANC is responsible for the largest single jump in the ablation study (19.5 to 25.3 mIoU before ARL is added), an inverted selection criterion would invalidate the central claim as written. The concern is not merely a disagreement with external conventions; it is an internal inconsistency between the prose, the equation, and the algorithm pseudocode. I also note secondary issues: no code release, no error bars or multi-seed runs, and a SemanticKITTI comparison that mostly includes unsupervised point-cloud methods rather than image-based label-free baselines, but these would be secondary even if the formula were fixed. The paper has genuine strengths: a clear modular design, extensive ablations, and hyperparameter sensitivity tables, which lend plausibility to the overall pipeline. However, the Eq. (2) issue prevents reproduction and verification, so the appropriate verdict is CONDITIONAL: the method should be accepted only with a corrected formula, a stated threshold value, and preferably released code or an independent reimplementation. No ad hominem is intended; the most likely explanation is a typographical sign error in a formula that the experiments may not have actually used, but the manuscript as written cannot be verified.","tokens_in":13715,"tokens_out":2826,"duration_ms":28348,"concrete_test":"Implement Eq. (2) exactly as written and the corrected complement (e.g., F = 1 + entropy/log(1/K)) on the SemanticKITTI validation split within the paper's pipeline, using the same threshold gamma and history length q=5. For a set of points with perfectly consistent histories, the as-written F should be near 0 and fail the gamma>=0.9 test, while the complement passes; for uniform histories, the behavior should be reversed. If the reported 25.7 mIoU is only obtained with the complement, Eq. (2) is confirmed as an inverted typo and the paper must be revised. If the as-written form nevertheless yields 25.7 mIoU (e.g., because DBSCAN propagation and the ARL rescue the corrupted selection), the concern is non-load-bearing and the verdict should be reconsidered.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The Adaptive Noise Corrector's reliable-point selection rests on the confidence score in Eq. (2): F(x_s;q) = -entropy(P(k|x_s;q)) / log(1/K). With standard base-e logarithms, log(1/K) = -ln K and -entropy/log(1/K) = entropy/ln K, which ranges from 0 for a perfectly confident and consistent prediction history to 1 for a maximally uniform, uncertain history. The text explicitly states that 'samples with consistent historical predictions and their labels' should be considered reliable, but Eq. (2) assigns the highest F to maximally inconsistent prediction histories. Algorithm 1 line 8 then selects points with F(x_s;q) > threshold, so the ANC would preferentially refurbish labels on points whose history is pure noise, and Eq. (5) would take the argmax of a near-uniform frequency distribution, yielding essentially arbitrary replacements. This is not a cosmetic sign error: the ablation in Table 2 attributes 25.3 mIoU to CLGM+ANC versus 19.5 for CLGM alone, so the label-correction stage is the main source of the reported gain. If the implemented form matches the paper's text, the reported improvements cannot be reproduced as described; if it is a typo for the complement (e.g., F = 1 - entropy/ln K), the paper must state this. The paper provides no code or multi-seed statistics, so the discrepancy cannot be resolved from the manuscript alone.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AdaCo, a label-free 3D semantic segmentation method for outdoor LiDAR scenes. The method has three components: a Cross-modal Label Generation Module (CLGM) that produces point-wise pseudo labels from 2D VFMs (FastSAM, SSA-Engine, word2vec similarity with text prompts); an Adaptive Noise Corrector (ANC) that uses the network's historical predictions and DBSCAN clustering to refurbish noisy labels; and an Adaptive Robust Loss (ARL) that switches from cross-entropy to a combination of normalized cross-entropy and mean absolute error losses after a per-sample correction time. On SemanticKITTI and nuScenes, the method reports 25.7 and 31.2 mIoU, respectively, outperforming prior label-free baselines by large margins. The ablation study attributes most of the gain to CLGM+ANC over CLGM alone.","tokens_in":13978,"tokens_out":4509,"duration_ms":43574,"significance":"If the reported numbers are reproducible, the paper makes a strong empirical contribution: it demonstrates that label-free 3D semantic segmentation can approach the level of fully supervised methods on two standard outdoor benchmarks, and it provides a modular design with a thorough ablation. The paper is clearly written, the experimental comparison covers the relevant recent baselines, and the qualitative results support the claimed improvement in edge quality and label consistency. The main methodological contribution of adaptive correction timing is interesting and, in principle, a useful addition to noisy-label learning. However, the current manuscript contains a concrete, load-bearing technical error in the definition of the historical-prediction confidence, and the correction-timing signal is derived from the very noisy labels being corrected; both issues must be resolved before the empirical claims can be accepted.","major_comments":[{"comment":"The confidence score F(x_s;q) in Eq. (2) is defined as -entropy(P(k|x_s;q)) / log(1/K). With base-e logarithms, log(1/K) = -ln K, so F(x_s;q) = entropy(P)/ln K, which ranges from 0 for a deterministic prediction history to 1 for a uniform distribution. The text in 'Reliable Labels Prediction' states that samples with consistent historical predictions should be considered reliable, but the formula as written assigns the highest confidence to maximally inconsistent histories. Algorithm 1 line 8 then selects points with F(x_s;q) > threshold, so the reliable set would be composed of high-entropy points, and Eq. (5) would take the argmax of an almost uniform frequency vector, yielding effectively arbitrary corrections. This issue is central: the ablation in Table 2 attributes a 5.8 mIoU gain to CLGM+ANC (19.5 to 25.3), so the label-correction stage is the main source of the reported improvement. Please correct the formula (e.g., F = 1 - entropy(P)/ln K) and ensure that Algorithm 1 uses the same convention, or explicitly state if the implemented form differs from the text.","section":"Methodology, Eq. (2) and Algorithm 1, line 8"},{"comment":"The correction timing is determined by fitting the exponential function f_s(t) to the per-sample training mIoU, but Algorithm 1 line 5 computes Is.append(mIoU(\\hat{y}_s, L_s)) using the noisy labels L_s. The fitted curve therefore measures agreement with the noisy supervision, not with clean labels. The text in the Additional Implementation Details explicitly states that 'training IoU is calculated from the prediction results and noisy labels'. The claim that the derivative-change threshold r identifies the moment when the model has finished learning clean labels is not justified by the manuscript: under standard noisy-label training, accuracy measured against the noisy labels often increases monotonically, and no clean-label signal is available to validate the turning point. This is a correctness risk for the ANC, since the correction time directly controls when historical predictions are used to overwrite labels. Please provide empirical evidence that the fitted t_c correlates with the early-learning/memorization transition; for example, on SemanticKITTI, where clean training labels are available, the paper could compare the fitted derivative-change point with the peak of the early-learning IoU computed against the clean labels.","section":"Adaptive Correction Timing Search, Eq. (1), and Algorithm 1, line 5"},{"comment":"All results are reported as single numbers with no error bars or multiple seeds. Given that many hyperparameters (r, q, ω, λ, β, σ, FastSAM thresholds, DBSCAN eps and min_points, block size and stride) are tuned on the validation set, the large reported margins over prior methods need to be supported by a few independent training runs with mean and standard deviation. This is particularly important for the main SemanticKITTI result (25.7 mIoU) and for the ablation in Table 3, where the difference between the best and second-best settings is only 0.1–0.3 mIoU in some rows.","section":"Experimental protocol, Tables 1–3"}],"minor_comments":[{"comment":"The section heading reads 'Experiements'; please correct the spelling to 'Experiments'.","section":"Experiments section heading"},{"comment":"The caption states that the early-learning IoU curve is computed with the correct ground truth, while the method itself does not use any clean labels during training. Please clarify that this figure is an analysis diagnostic and is not part of the training procedure, to avoid confusion with the circularity concern raised in the major comments.","section":"Figure 4 caption"},{"comment":"The notation in the normalized cross-entropy loss is unclear: q(k|x) is defined as the distribution over labels, but the denominator uses both q(y=j|x) and p(k|x); please define the terms precisely and specify over which set the outer sum runs.","section":"Eq. (7)"},{"comment":"The label mapping tables list 'Our pre-defined labels' as a manual dictionary, but the text in the Additional Implementation Details says a manual dictionary was maintained following OpenScene. Please provide the exact reference to OpenScene and state whether these mappings are used only for the manual-dictionary comparison or also in the final word2vec-based pipeline.","section":"Tables 6–7"},{"comment":"The use of a negative weight σ = -0.99 for the cross-entropy term L_CE in L_ARL is unusual and is only briefly motivated. Please provide a short explanation of how this negative weighting is intended to behave, and ideally show a curve of the loss value to rule out unintended optimization behavior.","section":"Eq. (10)"}],"recommendation":"major_revision","confidential_remarks":"The Eq. (2) issue appears to be a sign/typo rather than a conceptual dead end, because the surrounding text describes the intended behavior clearly and the ablation shows a consistent improvement from the correction stage. I recommend asking the authors to correct the formula, release code or a detailed pseudocode trace for the correction stage, and provide multi-seed statistics. If these can be supplied, the paper would likely be suitable for acceptance after the revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know about AdaCo is that it reports a big jump in label-free 3D segmentation—25.7 mIoU on SemanticKITTI, 31.2 on nuScenes—but the confidence formula in Eq. (2) looks inverted, and that formula sits at the heart of the label-correction module. The paper cannot be trusted as written until that inconsistency is cleared up.\n\nWhat's genuinely new: the paper takes the early-learning correction idea from Liu et al. 2022 and the clustering-based label propagation from Ye et al. 2021, and applies them to VFM-generated pseudo labels for outdoor LiDAR segmentation. The CLGM's use of FastSAM + SSA-Engine + word2vec for open-vocabulary label mapping is a nice engineering piece, and the ablation study shows each module contributes. The gains over prior label-free methods are large and consistent across two datasets.\n\nThe soft spots are real but mostly fixable. The biggest is Eq. (2): F = -entropy(P)/log(1/K) = entropy/ln K, so F is highest when the prediction history is a uniform mess. The text says reliable points should have consistent historical predictions. If the implementation actually uses this formula, the ANC would be selecting the least reliable points and the reported 25.3 mIoU for CLGM+ANC would not come from what the paper describes. If it's a typo—say, F = 1 - entropy/ln K—the authors need to say so. Without code or multi-seed runs, we can't resolve this from the manuscript.\n\nAlso worth noting: the SemanticKITTI comparison table mixes in unsupervised methods (GrowSp, U3DS3) that don't use images, making the comparison less direct. And there are a lot of tuned hyperparameters (thresholds, cluster ratios, loss weights) with no sensitivity analysis beyond a few values. No error bars either.\n\nThat said, the central idea is sound and the experimental evidence is internally consistent. This deserves a serious referee, but the authors should be asked to fix Eq. (2), release code, and report variance before acceptance.\n\nI'd bring it to a reading group for the discussion, but I wouldn't cite it until the formula is resolved.","headline":"Large label-free segmentation gains undercut by an inverted confidence formula in the core correction module; deserves review but needs clarification.","tokens_in":14593,"tokens_out":2861,"would_cite":false,"duration_ms":24152,"reading_group":"yes","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AdaCo claims that noisy pseudo-labels from visual foundation models can be corrected adaptively, letting label-free 3D semantic segmentation reach 25.7 mIoU on SemanticKITTI and 31.2 mIoU on nuScenes.","keywords":["3D semantic segmentation","label-free learning","pseudo-label refinement","visual foundation models","noisy label learning","point cloud segmentation","adaptive loss","cross-modal supervision"],"falsifier":"Compute $F(x_s;q)$ for a point whose last five predictions are all the same class and for a point whose last five predictions are uniformly spread across the $K$ classes; under Eq. (2) the uniform point scores 1 and the consistent point scores 0, opposite to the paper's reliable-point criterion. Replacing the reliable-point filter with the mode of historical predictions alone on SemanticKITTI would directly test whether the reported 25.7 mIoU depends on the mechanism as described.","tokens_in":13424,"feed_emoji":"🚗","tokens_out":13226,"duration_ms":102408,"temperature":0.7,"pith_summary":"This paper claims that noisy pseudo-labels extracted from visual foundation models can be systematically refurbished, so that a LiDAR point cloud segmenter can be trained with zero 3D annotations and still reach 25.7 mIoU on SemanticKITTI and 31.2 mIoU on nuScenes. The proposed AdaCo pipeline generates point-wise labels by lifting image-level mask descriptions into 3D, then iteratively corrects the noisy labels using each sample's own learning curve, and finally trains with a loss that becomes robust only after the correction point. The stated results beat previous label-free and unsupervised 3D segmenters by 11.5–12.5 mIoU on SemanticKITTI and 4.4 mIoU on nuScenes. If the claims hold, this would make foundation-model supervision a practical substitute for manual point-wise annotation in outdoor driving scenes.","feed_headline":"Label-free 3D segmentation jumps to 25.7 mIoU on SemanticKITTI","feed_subtitle":"Adaptive label correction cleans visual-foundation-model pseudo-labels, beating prior label-free methods by up to 12.5 mIoU.","key_machinery":"The machine that carries the argument is the adaptive correction scheduler. For each sample, the training mIoU against the noisy labels is recorded every epoch and fit by an exponential curve $f_s(t) = a(1-\\exp(-t^{b_s}/c_s))$; when the normalized change in the derivative exceeds a threshold $r=0.9$, the sample is declared to have finished its clean-learning phase, and the network's historical predictions over the last $q$ iterations (with $q$ capped at 5) are turned into a reliability score $F(x_s;q)$ for each point. Points whose scores pass a threshold supply replacement labels via the mode of their prediction history, and those labels are spread to neighboring points inside DBSCAN clusters. A second mechanism, the Adaptive Robust Loss, keeps cross-entropy loss during the warm-up phase and then adds normalized cross-entropy and mean-absolute-error terms with a negative weight on the original cross-entropy, so the loss changes character exactly when the label corrector runs.","core_discovery":"On its own terms, the paper's central discovery is that label noise from visual foundation models is not a fixed ceiling: it can be corrected adaptively during training. The authors report that AdaCo, built from the Cross-modal Label Generation Module (CLGM), the Adaptive Noise Corrector (ANC), and the Adaptive Robust Loss (ARL), improves the MaskCLIP-based label-free baseline from 8.1 to 25.7 mIoU on SemanticKITTI and from 16.5 to 31.2 mIoU on nuScenes, surpassing earlier label-free methods by 11.5–12.5 and 4.4 mIoU respectively. CLGM generates point-wise pseudo labels by combining class-agnostic masks with semantic descriptions and voting over adjacent frames; ANC detects a per-sample correction time from the training mIoU curve, marks points with consistent historical predictions as reliable, and propagates their labels within density-based clusters; ARL switches the loss from cross-entropy to normalized robust losses at the correction time to keep the network from memorizing residual noise. The paper concludes that label-free learning for 3D semantic segmentation can be made substantially more robust by treating the VFM-generated labels as a noisy signal to be cleaned rather than as final supervision.","pith_inferences":["Beyond the paper, the reliable-point selection can be viewed as a voting scheme over the network's own past predictions; adding geometric cues such as point-to-point distance or multi-view agreement would likely reduce the dependence on the curve-fit threshold.","Because the correction is applied only once per sample, a natural extension is to re-run the full correction loop with fresh historical predictions, testing whether repeated refinement improves or degrades label quality on other datasets.","The label generator's text-based category matching suggests a direct test of generalization: add unseen categories to the text prompts and check whether the corrected pseudo-labels improve on those categories without retraining the generator."],"forward_implications":["A LiDAR segmenter can be trained with no point-level annotations and still reach 25.7 mIoU on SemanticKITTI and 31.2 mIoU on nuScenes, roughly tripling the label-free baseline on the former.","The correction timing is per-sample and automatic, replacing manually preset warm-up rounds in noisy-label training.","Because CLGM maps semantic descriptions to an arbitrary category vocabulary, the label generator can adapt to new classes by editing text prompts without retraining.","The ARL loss schedule gives a transferable recipe: train with standard cross-entropy first, then switch to normalized robust losses once the network has learned clean patterns."],"supporting_citations":[{"why":"Supplies the CLIP visual-language model whose semantic features ground the label generation.","marker":"Radford et al. 2021"},{"why":"FastSAM generates the class-agnostic masks used by the 2D Pseudo Label Generation Engine.","marker":"Zhao et al. 2023"},{"why":"SSA-Engine produces semantic descriptions for each mask, matched against the category vocabulary.","marker":"Chen, Yang, and Zhang 2023"},{"why":"CMDFusion is the 3D backbone, the baseline segmenter, and the source of the feature-distillation loss.","marker":"Cen et al. 2023"},{"why":"MaskCLIP provides the baseline label generator whose noisy outputs the method corrects.","marker":"Zhou, Loy, and Dai 2022"},{"why":"Supplies the early-learning correction idea and the adaptive correction timing search.","marker":"Liu et al. 2022"},{"why":"Provides the normalized cross-entropy and mean-absolute-error robust losses used by ARL.","marker":"Ma et al. 2020"},{"why":"Supplies the reliable-label prediction strategy and cluster-based label propagation for point clouds.","marker":"Ye et al. 2021"},{"why":"DBSCAN clustering gives the instance information used to spread corrected labels.","marker":"Ester et al. 1996"}],"fun_headline_variants":["Adaptive label correction lifts label-free 3D segmentation to 25.7 mIoU","Cleaning VFM noise: AdaCo boosts label-free 3D segmentation by 12.5 mIoU","Label-free 3D segmentation refined: AdaCo's adaptive noise correction","From 8.1 to 25.7 mIoU: AdaCo corrects VFM label noise in 3D","Adaptive correction cleans VFM pseudo-labels for 3D segmentation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the confidence score $F(x_s;q)$ in Eq. (2) identifies points whose historical predictions are consistent (so their labels can be trusted), and that fitting an exponential to each sample's training mIoU reveals a genuine clean-then-memorize turning point; as printed, Eq. (2) is maximized by maximally uncertain predictions, so the reliable-point selection needs either a sign flip or a reinterpretation for the correction stage to do what the paper claims.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive label correction lifts label-free 3D segmentation to 25.7 mIoU","Cleaning VFM noise: AdaCo boosts label-free 3D segmentation by 12.5 mIoU","Label-free 3D segmentation refined: AdaCo's adaptive noise correction","From 8.1 to 25.7 mIoU: AdaCo corrects VFM label noise in 3D","Adaptive correction cleans VFM pseudo-labels for 3D segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000513,"raw_usage":{"total_tokens":2521,"prompt_tokens":1004,"completion_tokens":1517,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":620,"completion_tokens_details":{"reasoning_tokens":1395}},"tokens_in":620,"tokens_out":1517,"duration_ms":9675,"temperature":1.0,"reasoning_tokens":1395,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:52:39.362803+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute $F(x_s;q)$ for a point whose last five predictions are all the same class and for a point whose last five predictions are uniformly spread across the $K$ classes; under Eq. (2) the uniform point scores 1 and the consistent point scores 0, opposite to the paper's reliable-point criterion. Replacing the reliable-point filter with the mode of historical predictions alone on SemanticKITTI would directly test whether the reported 25.7 mIoU depends on the mechanism as described.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CMDFusion is the 3D backbone, the baseline segmenter, and the source of the feature-distillation loss."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the early-learning correction idea and the adaptive correction timing search."}],"review_version":1}