{"id":"655ebbb3-1cbd-488d-8771-673a6ce914ac","arxiv_id":"2411.08340","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"DyConfidMatch sets per-class pseudo-label thresholds and re-sampling weights from class-level confidence, improving semi-supervised 3D classification and detection.","lead":"This paper introduces DyConfidMatch, a semi-supervised method for 3D point-cloud learning that adjusts the confidence threshold for each object class during training and re-samples data to give harder or rarer classes more attention. The authors report accuracy and mAP gains over several earlier baseline methods on classification and detection benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported SOTA gains may stem from test-set hyperparameter tuning: the constants in Eqs. 6 and 7 are selected on the evaluation datasets (Tables 8, 9), not a validation split.","rationale":"The reader's weakest assumption—that class-level confidence tracks test accuracy—is plausible and supported only by visual correlation plots (Figs. 2b and 4), with no quantitative correlation coefficient or statistical test. If false, the method's mechanism would be misdirected; however, this is a concern about the explanation, not directly about the empirical headline. The empirical claim 'surpasses SOTA' is most directly threatened by the evaluation protocol: the two central constants are selected on the test sets of the very benchmarks used for the main comparisons (Tables 8 and 9 vs. Tables 1, 2, 4, 5). This is a concrete, verifiable flaw that can inflate reported gains and make 'SOTA' an artifact of selection bias. It is more load-bearing than the correlation premise because it attacks the validity of the numbers themselves. The reader did flag test-set tuning as one of several weaknesses, but not as the weakest assumption; hence 'partial' agreement. A simple re-evaluation with validation-based constant selection would settle whether the reported margins persist. If they do not, the central claim fails; if they do, the method may still be strong despite the questionable tuning. Either way, the current paper does not support unconditional acceptance.","tokens_in":17148,"tokens_out":10710,"duration_ms":107464,"concrete_test":"Re-run the main experiments (ModelNet40-10% and ScanNet-5%) with the constants in Eqs. 6 and 7 fixed to values chosen on a held-out validation split (e.g., 20% of the labeled training data) or to the values from the conference version, while keeping all baselines identical. If the performance gap between DyConfidMatch and the strongest baseline (e.g., Confid-3DIoUMatch in Table 4) shrinks by more than 2 mAP points or 1 accuracy point, the reported SOTA gains are substantially explained by test-set tuning rather than the proposed mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is empirical: DyConfidMatch 'surpasses state-of-the-art counterparts.' That claim is most directly threatened by the evaluation protocol. In §4.6 (Tables 8 and 9), the two key constants—the denominator 2 in the concave mapping M(P_c)=P_c/(2−P_c) (Eq. 6) and the exponent 2 in the comprehensive threshold τ=exp(−2·P_ave²) (Eq. 7)—are chosen by trying values 1, 2, 3 on the exact datasets and label fractions used in the headline comparisons: ModelNet40 at 10% labels and ScanNet at 5% labels. No held-out validation split is mentioned. Selecting constants that maximize test performance is a form of test-set selection bias; it can inflate the reported margins over baselines that were not tuned in the same way. This directly undermines the empirical claim, independent of whether class-level confidence tracks learning status. The paper's own claim that the comprehensive threshold 'reduces the dependency on the manner of setting threshold value' makes the test-set selection of its exponent especially problematic.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DyConfidMatch, a semi-supervised learning method for 3D classification and detection. The key idea is to use class-level confidence computed from unlabeled data as a proxy for each class's learning status, then dynamically adjust the pseudo-label threshold per class and re-weight the sampling of unlabeled instances so that underperforming classes receive more training signal. The method is built on top of Pseudo-Labeling/FixMatch for classification and 3DIoUMatch for detection. Experiments on ModelNet40, ScanObjectNN, SUN RGB-D, ScanNet, and KITTI report consistent gains over the compared baselines, especially at low label fractions, and ablations are provided for the dynamic threshold, re-sampling, mapping function, and the two constants that define the thresholds.","tokens_in":17311,"tokens_out":7471,"duration_ms":70074,"significance":"If the reported results are reproducible and the evaluation protocol is sound, DyConfidMatch is a simple, general add-on to pseudo-label SSL that could be useful for imbalanced 3D data. The paper includes extensive experiments across classification and detection, including outdoor detection, and compares against several class-imbalanced SSL re-sampling strategies. The central premise—that class-level confidence tracks test accuracy—is intuitive and the visual evidence (Figs. 2b and 4) is suggestive. However, the empirical claim of surpassing state-of-the-art is currently weakened by test-set hyperparameter selection, missing recent baselines, and an unexplained inconsistency in the core threshold formula. The paper does not provide code or model releases, which limits reproducibility checks.","major_comments":[{"comment":"The two constants that drive the method—the denominator 2 in M(P_c)=P_c/(2−P_c) and the exponent 2 in τ=exp(−2·P_ave²)—are selected by ablations performed on ModelNet40 at 10% labels and ScanNet at 5% labels, which are exactly the datasets and label fractions used in the headline comparisons (Tables 1 and 4). No held-out validation split is mentioned. This constitutes test-set selection bias and can inflate the reported margins over baselines that were not tuned in the same way. Please either perform the ablations on a separate validation split and only then evaluate on the test set, or demonstrate that the reported gains are robust across a range of constants (e.g., report performance for constants 1, 2, 3 on the final test sets and show that the conclusion is unchanged).","section":"§4.6, Tables 8 and 9"},{"comment":"The clamping in Eq. (6) is internally inconsistent whenever the comprehensive threshold satisfies τ < 0.5, which occurs when P_ave > sqrt(ln 2 / 2) ≈ 0.589. In that regime, 1−τ > τ, so the conditions 'if M(P_c) < 1−τ' and 'elif M(P_c) > τ' can both be true or both be false for a given M(P_c). Concretely, for P_ave = 0.8 (τ = 0.278, 1−τ = 0.722), a low-confidence class with P_c = 0.3 (M = 0.176) is assigned threshold 0.722, while a high-confidence class with P_c = 0.9 (M = 0.818) is assigned threshold 0.278—the opposite of the intended dynamic. Since many reported final accuracies imply P_ave > 0.589, this is not a corner case. Please clarify the intended clamping (e.g., the bounds should likely be [τ, 1−τ] or the inequalities reversed) and re-run the experiments with the corrected formula, or show that an alternative formulation yields the same results.","section":"§3.3, Eqs. (6)-(7)"},{"comment":"The load-bearing premise of the method is that class-level confidence P_c from unlabeled data accurately reflects per-class test accuracy throughout training. The only support offered is two visual correlation plots and the qualitative phrase 'strong correlation'; no correlation coefficient, confidence interval, or statistical test is reported for either classification (Fig. 2b) or detection (Fig. 4). If this premise fails for underrepresented or hard classes, both the dynamic thresholds (Eq. 6) and the re-sampling weights (Eq. 10) will systematically re-weight data toward the wrong classes. Please provide quantitative correlation measures (e.g., Pearson and Spearman coefficients with bootstrap confidence intervals) computed on a validation split at several training epochs for both tasks.","section":"§3.3 and §4.7, Figs. 2b and 4"},{"comment":"The related work section cites several recent 3D SSL detection methods—Ssda3d [33], ProposalContrast [34], Dqs3d [35], and Diffusion-SS3D [36]—but the detection experiments compare only VoteNet, SESS, 3DIoUMatch, and the authors' own prior work. To support the abstract's claim of 'surpasses state-of-the-art counterparts,' at least the most relevant of these should be included in the comparison table, or the authors should explicitly state why they are excluded (e.g., different task settings, unavailability of code). Similarly, the classification comparison omits more recent SSL methods such as DMT, which is cited in the introduction.","section":"§2.3 and §4.4"}],"minor_comments":[{"comment":"The definition of the per-class set C_c is malformed: the set-builder notation should specify that u_b ranges over the unlabeled set N_U, and the summation index in P_c should run from 1 to |C_c| rather than from 0 to |C_c|.","section":"Eq. (5)"},{"comment":"The quantity in Eq. (10) is called a 'sampling probability,' but the two cases produce values in different ranges (the second case can exceed 1), and no normalization is described. Please clarify whether these are unnormalized sampling weights and how they are used by the data loader.","section":"§3.4, Eq. (10)"},{"comment":"The text states that Fig. 5a shows FlexMatch's thresholds and that Fig. 5b shows our method's thresholds, but the figure caption assigns the opposite: (a) is 'Thresholds for each class in the last epoch of our method' and (b) is 'Each class's threshold of FlexMatch.' Please correct the text or the caption so the figure supports the argument.","section":"§4.9, Fig. 5"},{"comment":"The sentence 'For scenes with multiple objects, we select the lowest confidence object' is ambiguous; it is unclear whether only one pseudo-labeled object per scene is used and how the selected object is chosen. Please specify the exact handling of multi-object scenes.","section":"§4.2, detection"},{"comment":"The tables list both 'Flex-PL [4]' and 'FlexMatch [4]' as separate rows, but the difference between these two baselines is not explained in the text or in the table caption. Please clarify whether Flex-PL denotes FlexMatch applied to Pseudo-Labeling and FlexMatch denotes the original FixMatch-based method, and state which backbone each uses.","section":"Tables 1 and 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of Pattern Recognition and the core idea is a reasonable extension of the authors' earlier WACV work. The main concerns are the test-set selection of the two constants, the inconsistent threshold formula, and the lack of quantitative correlation evidence; these are fixable with additional experiments and analysis. I would also encourage the editor to request that the authors release code or at least detailed training configurations, since several recent 3D SSL baselines are cited but not compared."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a journal extension of the authors' WACV 2023 paper (Class-level confidence based 3D semi-supervised learning). The core mechanism—class-level confidence as a proxy for learning status, used to set per-class pseudo-label thresholds and re-sampling weights—is unchanged. What's new is a global threshold formula (Eq. 7), KITTI detection results, and a correlation analysis for detection. The paper is honest about its relationship to the conference version.\n\nThe method does several things well. The idea is simple and plausible: classes with low average prediction confidence on unlabeled data are likely under-learned, so lower their threshold and sample them more. The experiments are broad: four classification/detection benchmarks plus KITTI, and the gains are consistent—up to ~7 mAP@0.25 over 3DIoUMatch at 2% labels on ScanNet. The ablation studies show both components (dynamic threshold and re-sampling) contribute, and the comparison against CReST/BiS re-sampling suggests the learning-status-based sampling is not just the usual quantity-based rebalancing. The math is simple and coherent; no obvious issues there.\n\nThe soft spots are real. Most importantly, the two key constants—the '2' in the concave mapping M(P_c)=P_c/(2−P_c) and the '2' in the comprehensive threshold τ=exp(−2·P_ave²)—are selected by trying values 1, 2, 3 on the exact datasets and label fractions used in the headline comparisons (ModelNet40 at 10%, ScanNet at 5%, §4.6, Tables 8 and 9). No held-out validation is mentioned. The differences between constant choices are modest (roughly 1–3 points), so this may not be catastrophic, but it does mean the reported numbers are the best of several runs on the test set, and the paper's claim that the comprehensive threshold 'reduces the dependency on the manner of setting threshold value' sits awkwardly next to the tuned exponent.\n\nTwo more issues, in decreasing order of importance. Classification results have no error bars, so we can't tell if the 1–3 point mean-accuracy gains are within run-to-run variation. And several recent baselines cited in the text (Ssda3d, ProposalContrast, Dqs3d, Diffusion-SS3D) are missing from the comparisons; the baselines used are primarily from 2021 and the authors' own 2023 paper. No code is released.\n\nOn the central hypothesis: the paper shows only visual correlation plots (Figs. 2b and 4) between class-level confidence and test accuracy; no correlation coefficient or significance test. That's a weak spot, but the hypothesis is reasonable and the method works even if the correlation is approximate.\n\nWho should read this? People working on 3D semi-supervised classification or detection, particularly on imbalanced data. It's a useful incremental method, not a paradigm shift. I would send it to peer review—the experiments are substantial and the idea is worth scrutiny—but I'd insist on a validation-based constant selection protocol and error bars for classification.\n\nFor your own work, I wouldn't cite the SOTA claim as a given, but the class-level confidence idea might be worth citing as related work once the protocol issues are cleared up.\n\nBest.","headline":"A solid incremental 3D SSL method whose core mechanism is from the authors' WACV 2023 paper; the journal version's SOTA claim is weakened by test-set-tuned constants, but the empirical work is substantial enough for peer review.","tokens_in":17925,"tokens_out":3971,"would_cite":false,"duration_ms":35240,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Class-level confidence rebalances 3D semi-supervised learning","keywords":["Semi-supervised learning","3D object detection","3D point cloud classification","Class imbalance","Pseudo-labeling","Dynamic thresholding","Re-sampling","Class-level confidence"],"falsifier":"Compute the per-class rank correlation between class-level confidence $P_c$ and per-class test accuracy on, say, ModelNet40 with 10% labeled data, sampled every 50 epochs; if the correlation is weak (for instance below roughly 0.5) or turns negative for minority classes at any stage, the steering signal is unreliable and the reported gains should shrink or reverse when the correlation is adversarially broken.","tokens_in":16871,"feed_emoji":"🎯","tokens_out":7787,"duration_ms":72302,"temperature":0.7,"pith_summary":"DyConfidMatch claims that the average confidence of a model's own predictions on unlabeled data, taken per class, is a faithful indicator of how well that class is being learned. The paper uses this class-level confidence to do two things: dynamically lower the pseudo-label threshold for classes that lag behind, and re-sample unlabeled data so struggling classes contribute more often. On ModelNet40, ScanObjectNN, SUN RGB-D, ScanNet, and KITTI, the method reports gains over fixed-threshold baselines, with the largest improvements at very low label fractions, for example roughly +7 mAP@0.25 over 3DIoUMatch at 2% labels on ScanNet. The payoff is that semi-supervised 3D pipelines can adapt to data imbalance without manual threshold tuning.","feed_headline":"Per-class confidence rebalances 3D semi-supervised learning","feed_subtitle":"DyConfidMatch adjusts pseudo-label thresholds and sampling per class, adding up to 7 mAP at low label rates.","key_machinery":"The load-bearing object is class-level confidence, $P_c = \\frac{1}{|C_c|} \\sum_{u_j \\in C_c} \\max(p_m(y|\\alpha(u_j)))$, the mean of the model's maximum predicted probability over unlabeled points whose argmax class is $c$. It feeds a concave mapping $M(P_c)=P_c/(2-P_c)$ to produce a per-class dynamic threshold $\\tau_e(c)$, clamped by a global threshold $\\tau=\\exp(-2P_{\\text{ave}}^2)$, and it also sets per-instance sampling weights that boost classes with low learning status. Together these two mechanisms carry the argument: they convert a scalar confidence signal into both a pseudo-label gate and a data-rebalancing schedule, and the ablation study shows each component contributes to the reported gains.","core_discovery":"The central claim is that a per-class statistic—the average maximum softmax probability over unlabeled points assigned to that class—tracks the class's true learning status (test accuracy) throughout training, in both classification and detection settings. On this basis the paper constructs DyConfidMatch, which sets each class's pseudo-label threshold at a concave function of that class-level confidence, clips it with a global threshold that also adapts, and assigns each unlabeled instance a sampling weight that up-weights classes with low learning status. The paper shows this combination outperforms existing 3D SSL methods across classification and detection benchmarks, with the strongest margins in low-label regimes. The method is presented as a drop-in replacement for the threshold and sampling components of existing pseudo-label learners such as FixMatch and 3DIoUMatch.","pith_inferences":["A concrete way to stress-test the paper's premise would be to compute a rank correlation between class-level confidence and per-class test accuracy across training; the paper only shows visual plots, so the confidence proxy remains quantitatively unvalidated.","If the confidence proxy survives that test, the same learning-status re-sampling idea could be transferred to 2D semi-supervised learning or long-tail recognition, likely after recalibrating confidence for those settings.","An oracle experiment—using true per-class test accuracy instead of $P_c$ to set thresholds and sampling—would bound how much of the reported gain comes from the rebalancing scheme itself rather than from the accuracy of the confidence estimate.","Because pseudo-class assignments come from the model's own argmax, very early training or poorly calibrated backbones could poison the class-level statistics; a warm start or uncertainty-weighted averaging might be needed in practice."],"forward_implications":["Low-label 3D detection and classification no longer need a manually chosen fixed pseudo-label threshold; the threshold adapts as the model learns.","Underrepresented or harder classes get more unlabeled data earlier, which is where the reported gains concentrate, such as +7 mAP@0.25 over 3DIoUMatch at 2% labels on ScanNet.","The dynamic threshold and re-sampling components can be dropped into existing pseudo-label SSL methods, so the same mechanism could lift other 3D learners without changing the backbone.","Re-sampling by learning status rather than class count avoids over-sampling easy minority classes, a behavior the paper demonstrates by outperforming CReST and BiS with only the sampling component swapped.","The KITTI results indicate the mechanism transfers from indoor to outdoor 3D detection, where class imbalance and confidence distributions differ."],"supporting_citations":[{"why":"Supplies the 3DIoUMatch detection baseline that DyConfidMatch extends and compares against.","marker":"[1]"},{"why":"FixMatch is the base pseudo-labeling SSL method whose fixed threshold and unsupervised loss are modified.","marker":"[3]"},{"why":"FlexMatch provides the class-level dynamic threshold idea and is the key comparison baseline in classification.","marker":"[4]"},{"why":"Pseudo-Label is a baseline method used in classification experiments and the origin of pseudo-labeling.","marker":"[5]"},{"why":"The authors' conference version Confid-Match is the prior method this article extends with dynamic max thresholding and KITTI experiments.","marker":"[15]"},{"why":"Dash is the dynamic-thresholding baseline the paper compares against in classification tables.","marker":"[17]"},{"why":"CReST is a class-imbalanced re-sampling baseline used in the sampling-only comparison.","marker":"[28]"},{"why":"BiS is a class-imbalanced re-sampling baseline used in the sampling-only comparison.","marker":"[29]"}],"fun_headline_variants":["Dynamic thresholds and re-sampling fix 3D SSL imbalance","Per-class confidence guides 3D semi-supervised learning","Adaptive thresholds boost 3D SSL for imbalanced data","DyConfidMatch: class-level confidence rebalances 3D SSL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the average confidence the model assigns to unlabeled points in a class accurately tracks how well that class is actually learned, across training and even for rare classes; if that correlation fails, the dynamic thresholds and re-sampling weights would push training toward the wrong classes.","fun_headline_variants_meta":{"raw":{"variants":["Dynamic thresholds and re-sampling fix 3D SSL imbalance","Per-class confidence guides 3D semi-supervised learning","Adaptive thresholds boost 3D SSL for imbalanced data","DyConfidMatch: class-level confidence rebalances 3D SSL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000704,"raw_usage":{"total_tokens":3120,"prompt_tokens":834,"completion_tokens":2286,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":450,"completion_tokens_details":{"reasoning_tokens":2214}},"tokens_in":450,"tokens_out":2286,"duration_ms":16732,"temperature":1.0,"reasoning_tokens":2214,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T21:41:49.396558+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the per-class rank correlation between class-level confidence $P_c$ and per-class test accuracy on, say, ModelNet40 with 10% labeled data, sampled every 50 epochs; if the correlation is weak (for instance below roughly 0.5) or turns negative for minority classes at any stage, the steering signal is unreliable and the reported gains should shrink or reverse when the correlation is adversarially broken.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the 3DIoUMatch detection baseline that DyConfidMatch extends and compares against."},{"cited_title":"Zhang, Y","cited_arxiv_id":null,"evidence_quote":"FlexMatch provides the class-level dynamic threshold idea and is the key comparison baseline in classification."},{"cited_title":"Lee, et al., Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks, in: Workshop on challenges in representation learning, ICML, V ol","cited_arxiv_id":null,"evidence_quote":"Pseudo-Label is a baseline method used in classification experiments and the origin of pseudo-labeling."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The authors' conference version Confid-Match is the prior method this article extends with dynamic max thresholding and KITTI experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Dash is the dynamic-thresholding baseline the paper compares against in classification tables."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CReST is a class-imbalanced re-sampling baseline used in the sampling-only comparison."}],"review_version":1}