REVIEW 3 major objections 5 minor 47 references
Detect and Correct: A Selective Noise Correction Method for Learning with Noisy Labels
T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read D&C-Net claims that selectively applying a transition-matrix loss correction only to samples flagged as noisy, while leaving clean samples on standard cross-entropy, yields higher accuracy under label noise than global correction or…
desk verdict A workmanlike incremental method with a sensible selective-correction twist, but the missing transition-matrix update rule and an overstated significance claim keep it from being complete as submitted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing pair is the binary noise flag $b_n$ and the row-stochastic transition matrix $T$. The flag comes from a Gaussian-mixture fit to per-sample losses, and $T$ is initialized by counting, among flagged samples, how often the observed noisy label $\tilde y_n$ differs from the classifier's prediction; this is the step whose reliability the paper explicitly ties to classifier accuracy. During training, $b_n$ decides which loss to use and $T$ supplies the class-to-class flip probabilities inside the corrected loss for $b_n=1$. The paper's distinctive move is the localization: $T$ is updated continuously but applied only to flagged samples, so clean samples see no correction.
What would settle it
Run D&C-Net on a synthetic noisy dataset whose true transition matrix is known, record the matrix it estimates at initialization and after training, and compute the entrywise distance to the truth; if the estimated matrix is far from the true one even in the symmetric-noise settings where the paper reports its largest accuracy gains, the claimed correction mechanism is not what drives the improvement.
Extended reading notes
Core claim
The central claim is that loss correction via a noise transition matrix should be applied selectively, not globally. The paper identifies suspect samples by tracking per-sample losses under a cyclic learning rate and fitting a Gaussian mixture with two components; the threshold that separates clean from noisy is chosen to maximize the average of sensitivity and specificity. Only the flagged samples are used to initialize the transition matrix $T$, and during training a flagged sample $n$ is given the corrected cross-entropy loss $-\log \sum_{l=1}^{C} T_{l,\tilde y_n} P_\theta(Y=l\mid x_n)$, while an unflagged sample keeps the standard cross-entropy. This targeted correction preserves the learning signal in noisy examples without altering clean ones. In the reported experiments the largest wins appear under 50% symmetric noise, for example 87.30% versus 83.37% for the strongest baseline on CIFAR-10.
Load-bearing premise
The whole correction rides on the pre-trained classifier being accurate enough that its prediction for a flagged noisy sample is usually the true label; if classifier mistakes are correlated with the noise process, the estimated transition matrix is biased and the correction step can amplify the noise instead of fixing it.
Editorial extensions
If this is right
- A pre-training phase with a cyclical learning rate yields both a noise indicator and a transition-matrix initialization, so the method does not need to know the noise rate in advance.
- Noisy samples are corrected rather than removed, so the effective training set is larger than under filtering, and correctly labeled samples are never loss-modified as they are under global transition-matrix methods.
- If the reported numbers hold, transition-matrix correction becomes usable in high-noise regimes where global correction usually struggles, with CIFAR-10 at 50% symmetric noise improving from 83.37% to 87.30%.
- On biological data, the same pipeline improves cell-type annotation accuracy on a PBMC single-cell RNA-seq dataset across all tested noise settings, suggesting label-noise correction can be applied where annotation is costly and noisy.
Reading between the lines
- The paper's own limitation discussion implies the method will lose ground when the clean and noisy loss distributions overlap, as under instance-dependent or structured noise; a natural stress test is to measure the overlap of the two fitted Gaussians and correlate it with the accuracy gain.
- Because $T$ is updated using only flagged samples, the correction could feed the classifier's own mistakes back into the matrix; the paper does not compare the recovered $T$ with the true noise process, which would separate bias in the matrix from the method's overall benefit.
- A direct ablation the authors do not run is a version of D&C-Net that applies the same updated $T$ to every sample; the accuracy gap between the selective and global variants would isolate exactly how much the selective application contributes.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes D&C-Net, a two-stage method for learning with noisy labels. In a pre-training stage, the method trains a network with a cyclical learning rate, fits a two-component Gaussian mixture model to the per-sample loss distribution, and selects a threshold to flag likely noisy labels. It then initializes a noise transition matrix by counting, among flagged samples, co-occurrences of the observed noisy label and the classifier's predicted label. In the training stage, the method applies a corrected cross-entropy loss only to the flagged noisy samples, leaving clean samples with the standard loss, and the transition matrix is said to be continuously updated. Experiments are reported on MNIST, CIFAR-10, CIFAR-100, and an scRNA-seq PBMC cell-type annotation dataset, comparing against several baselines. The paper claims consistent and significant accuracy improvements, especially under high noise.
Significance. If the method were fully specified and the claimed improvements held, the selective-correction idea—correcting only detected noisy samples while preserving clean-sample losses—would be a practically useful contribution to the noisy-labels literature, and the scRNA-seq application is a worthwhile extension. The paper provides several strengths: a clear motivation, comparison with eight baselines, an ablation study, and a public code repository. However, the central claim of consistent, significant improvement is not supported by the paper's own statistical analysis, and a core component of the algorithm (the transition-matrix update rule) is left undefined, which limits the reproducibility and interpretability of the reported results. The transition-matrix initialization also rests on an unvalidated self-training assumption. These issues substantially reduce the significance of the contribution as it currently stands.
major comments (3)
- [Algorithm 2, line 5; Section 3.2] Algorithm 2 instructs to 'Update transition matrix T' at every iteration, and Section 3.2 states that 'The transition matrix T is continuously updated during training to adapt to evolving noise patterns,' but no update rule, objective, or optimization procedure is provided anywhere in the manuscript. The experimental setup mentions a learning rate of 10^-4 'for the transition matrix,' which suggests a gradient-based update, yet T is not a network parameter and no loss with respect to T is defined. Without this specification, the reported results cannot be reproduced, and the method cannot be separated from untested implementation choices.
- [Section 4.3 and Supplementary 6.1] The claim that D&C-Net 'consistently outperforms leading baselines' (Section 4.3) and shows 'significant improvement' (abstract) is contradicted by the paper's own Welch t-test results in Table 6. D&C-Net is significantly better than VolMinNet (p < 0.05) in only 3 of 6 settings, VolMinNet is significantly better in CIFAR-100 Pair-20% (p = 0.0085), and two settings (CIFAR-10 Sym-20%, p = 0.100; MNIST Sym-50%, p = 0.170) show no significant difference. Moreover, with only three seeds, the t-test has low power, so even the significant findings are fragile. The evidence supports at most a selective advantage, not consistent superiority.
- [Section 3.1, Transition Matrix Initialization] The transition matrix is initialized by treating the classifier's predictions on flagged noisy samples as the true labels: T_{l,k} = sum_n I(\tilde{y}_n = k, \hat{y}_n = l, b_n = 1), followed by row normalization. The paper states 'we assume the classifier is sufficiently accurate to provide a meaningful estimate of the noise structure,' but no validation of this assumption is provided. If the classifier's errors on noisy samples are correlated with the noise process, the estimated T is biased, and the correction step in Section 3.2 may amplify rather than fix the noise. The paper's own Limitations section (5.2) acknowledges reliance on accurate detection but does not address this circularity or offer a diagnostic to check it.
minor comments (5)
- [Section 3.1 and Figure 1/Figure 2 captions] The threshold t is defined in Section 3.1 by maximizing the average of sensitivity and specificity, but the caption of Figure 1 states that t is 'determined by the intersection of the two distributions.' These two definitions are not generally equivalent; please clarify which one is used in the experiments and whether the figures depict the actual procedure.
- [Algorithm 1 and Algorithm 2] Algorithm 1 returns model parameters θ from pre-training, but Algorithm 2 does not state whether the final training starts from these parameters or from a fresh initialization. This affects the interpretation of the two-stage procedure.
- [Section 4.5 Ablation] The ablation reports an accuracy decrease of approximately 0.5% on CIFAR-10 when the transition matrix is not initialized, but it does not specify the noise type and level, the number of seeds, or whether this difference is statistically significant.
- [Table 5] The scRNA-seq results are not accompanied by a significance test. For Sym-50%, the difference between D&C-Net (93.93±0.05) and VolMinNet (93.75±0.08) is very small and may not be significant.
- [References] Reference [9] is listed as 'T-Reivision' in the text of Section 4.3; this appears to be a typo for 'T-Revision'. Please check all reference names for consistency.
Circularity Check
No circularity: all fitted components are transparent training heuristics, and final accuracies are direct test-set measurements.
full rationale
The paper is an empirical machine-learning method paper; it contains no derivation chain whose stated conclusion (test accuracy) is equivalent to its fitted inputs. The GMM parameters, the threshold t, and the transition matrix T are all estimated from training data, but none of these fitted quantities is renamed or reported as the paper's outcome; the reported outcomes are held-out accuracies measured after training. The transition-matrix initialization does substitute the classifier's own predictions for the unknown true labels on flagged noisy samples, which is a self-training/confirmation-bias risk, and the paper says so explicitly: 'Since the initialization depends on classifier predictions, we assume the classifier is sufficiently accurate to provide a meaningful estimate of the noise structure' (Sec. 3.1). The limitations section repeats this dependence (Sec. 5.2). An explicit, acknowledged assumption is not a hidden circular reduction: no equation in the paper defines the reported accuracy in terms of the fitted T or b. The authors' prior publications appear only in background or future-directions remarks and are not load-bearing for the method's construction. The statistical-significance analysis in Table 6 undermines the abstract's 'consistently outperforms' phrasing in some settings, but that is a validity/statistical concern, not circularity. The missing update rule for T in Algorithm 2 is an under-specification issue, also not circularity. Thus, under the stated circularity axis, the paper is not circular.
Assumptions & free parameters
free parameters (3)
- GMM component parameters (lambda, mu1, sigma1, mu2, sigma2) =
estimated via EM on the per-sample loss distribution
- Threshold t =
determined by maximizing (SEN+SPC)/2 on the fitted GMM
- Transition matrix T =
initialized by counting classifier predictions vs observed labels on flagged noisy samples; updated during training…
assumptions (4)
- domain assumption Per-sample losses of clean and noisy labels follow a mixture of two Gaussian distributions.
- domain assumption The classifier's predicted labels are accurate enough to serve as true labels when estimating the transition matrix from flagged noisy samples.
- standard math The threshold that maximizes (SEN+SPC)/2 on the fitted Gaussian components yields a correct division of clean and noisy samples.
- ad hoc to paper The transition matrix can be continuously updated during training to converge to the true noise structure.
Cite this review
Pith. "Pith review of Detect and Correct: A Selective Noise Correction Method for Learning with Noisy Labels." pith.science (2026). https://pith.science/paper/O3BN6XCT
@misc{pith2026250513342,
author = {Pith},
title = {Pith review of: Detect and Correct: A Selective Noise Correction Method for Learning with Noisy Labels},
year = {2026},
howpublished = {\url{https://pith.science/paper/O3BN6XCT}},
note = {Machine review of arXiv:2505.13342}
}
read the original abstract
Falsely annotated samples, also known as noisy labels, can significantly harm the performance of deep learning models. Two main approaches for learning with noisy labels are global noise estimation and data filtering. Global noise estimation approximates the noise across the entire dataset using a noise transition matrix, but it can unnecessarily adjust correct labels, leaving room for local improvements. Data filtering, on the other hand, discards potentially noisy samples but risks losing valuable data. Our method identifies potentially noisy samples based on their loss distribution. We then apply a selection process to separate noisy and clean samples and learn a noise transition matrix to correct the loss for noisy samples while leaving the clean data unaffected, thereby improving the training process. Our approach ensures robust learning and enhanced model performance by preserving valuable information from noisy samples and refining the correction process. We applied our method to standard image datasets (MNIST, CIFAR-10, and CIFAR-100) and a biological scRNA-seq cell-type annotation dataset. We observed a significant improvement in model accuracy and robustness compared to traditional methods.
Figures
Reference graph
Works this paper leans on
-
[1]
Image clas- sification with deep learning in the presence of noisy labels: A survey
Gorkem Algan and Ilkay Ulusoy. “Image clas- sification with deep learning in the presence of noisy labels: A survey”. In: Knowledge-Based Systems 215 (2021), p. 106771
work page 2021
-
[2]
Dana Angluin and Philip Laird. “Learning from noisy examples”. In: Machine Learning 2 (1988), pp. 343–370
work page 1988
-
[3]
A Simple Framework for Contrastive Learning of Visual Representations
Ting Chen et al. A Simple Framework for Contrastive Learning of Visual Representations
-
[4]
Ekin D. Cubuk et al. RandAugment: Practical automated data augmentation with a reduced search space . 2019. arXiv: 1909 . 13719 [cs.CV]. URL: https : // arxiv . org / abs/1909.13719
arXiv 2019
-
[5]
Maximum likelihood from incomplete data via the EM algorithm
Arthur P Dempster, Nan M Laird, and Donald B Rubin. “Maximum likelihood from incomplete data via the EM algorithm”. In: Journal of the royal statistical society: series B (methodologi- cal) 39.1 (1977), pp. 1–22
work page 1977
-
[6]
COPER: Correlation-based Permutations for Multi-View Clustering
Ran Eisenberg, Jonathan Svirsky, and Ofir Lindenbaum. “COPER: Correlation-based Permutations for Multi-View Clustering”. In: The Thirteenth International Conference on Learning Representations
-
[7]
Joint probabilistic modeling of single-cell multi-omic data with totalVI
Adam Gayoso, Zoë Steier, Romain Lopez, et al. “Joint probabilistic modeling of single-cell multi-omic data with totalVI”. In: Nature Meth- ods 18 (2021), pp. 272–282
work page 2021
-
[8]
Co-teaching: Ro- bust training of deep neural networks with ex- tremely noisy labels
Bo Han, Quanming Yao, et al. “Co-teaching: Ro- bust training of deep neural networks with ex- tremely noisy labels”. In:Advances in Neural In- formation Processing Systems (NeurIPS). 2018
work page 2018
Show all 47 references
-
[9]
Deep self-learning from noisy labels
Jiangfan Han, Ping Luo, and Xiaogang Wang. “Deep self-learning from noisy labels”. In: Pro- ceedings of the IEEE/CVF international confer- ence on computer vision. 2019, pp. 5138–5147
2019
-
[10]
Deep residual learning for image recognition
Kaiming He et al. “Deep residual learning for image recognition”. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, pp. 770–778
2016
-
[11]
Masked Autoencoders Are Scalable Vision Learners
Kaiming He et al. Masked Autoencoders Are Scalable Vision Learners . 2021. arXiv: 2111 . 06377 [cs.CV] . URL: https : // arxiv.org/abs/2111.06377
2021 arXiv
-
[12]
A systematic evaluation of single-cell RNA-sequencing imputation methods
Wenpin Hou et al. “A systematic evaluation of single-cell RNA-sequencing imputation methods”. In: Genome biology 21 (2020), pp. 1–30
2020
-
[13]
O2U-Net: A Simple Noisy Label Detection Approach for Deep Neural Networks
Junjie Huang et al. “O2U-Net: A Simple Noisy Label Detection Approach for Deep Neural Networks”. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2019, pp. 3326–3334
2019
-
[14]
MentorNet: Learning data-driven curriculum for very deep neural networks on cor- rupted labels
Lu Jiang et al. “MentorNet: Learning data-driven curriculum for very deep neural networks on cor- rupted labels”. In: Proceedings of the Interna- tional Conference on Machine Learning (ICML). 2018, pp. 2304–2313
2018
-
[15]
Learning Multiple Layers of Features from Tiny Images
Alex Krizhevsky. “Learning Multiple Layers of Features from Tiny Images”. In: University of Toronto(2009), pp. 32–33
2009
-
[16]
Gradient- based learning applied to document recogni- tion
Yann LeCun, Léon Bottou, et al. “Gradient- based learning applied to document recogni- tion”. In: Proceedings of the IEEE 86.11 (1998), pp. 2278–2324
1998
-
[17]
Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks
Dong-Hyun Lee. “Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks”. In: 2013. URL: https:/ /api.semanticscholar.org/ CorpusID:18507866
2013
-
[18]
Junnan Li, Richard Socher, and Steven C. H. Hoi. DivideMix: Learning with Noisy Labels as Semi-supervised Learning. 2020. arXiv: 2002. 07394 [cs.CV]
2020
-
[19]
When Speaker Recognition Meets Noisy Labels: Optimizations for Front-Ends and Back- Ends
Lin Li, Fuchuan Tong, and Qingyang Hong. “When Speaker Recognition Meets Noisy Labels: Optimizations for Front-Ends and Back- Ends”. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing 30 (2022), pp. 1586–1599
2022
-
[20]
In: Advances in Neural Information Processing Systems (NeurIPS)
Shikun Li, Xiaobo Xia, et al. In: Advances in Neural Information Processing Systems (NeurIPS). V ol. 35. 2022, pp. 24184–24198
2022
-
[21]
Provably End-to-end Label- Noise Learning without Anchor Points
Xuefeng Li et al. “Provably End-to-end Label- Noise Learning without Anchor Points”. In: International Conference on Machine Learning (ICML). 2021
2021
-
[22]
Transductive and inductive outlier de- tection with robust autoencoders
Ofir Lindenbaum, Yariv Aizenbud, and Yuval Kluger. “Transductive and inductive outlier de- tection with robust autoencoders”. In: The 40th Conference on Uncertainty in Artificial Intelli- gence. 2024
2024
-
[23]
Learning coupled embedding using multiview diffusion maps
Ofir Lindenbaum, Arie Yeredor, and Moshe Salhov. “Learning coupled embedding using multiview diffusion maps”. In: Latent Vari- able Analysis and Signal Separation: 12th International Conference, LVA/ICA 2015, Liberec, Czech Republic, August 25-28, 2015, Proceedings 12. Springe...
2015
-
[24]
Multi-channel fusion for seismic event detection and classification
Ofir Lindenbaum et al. “Multi-channel fusion for seismic event detection and classification”. In: 2016 IEEE International Conference on the Sci- ence of Electrical Engineering (ICSEE) . IEEE. 2016, pp. 1–5
2016
-
[25]
Iden- tifiability of Label Noise Transition Matrix
Yang Liu, Hao Cheng, and Kun Zhang. “Iden- tifiability of Label Noise Transition Matrix”. In: Proceedings of the 40th International Con- ference on Machine Learning (ICML) . 2023, pp. 21475–21496
2023
-
[26]
Decou- pling
Eran Malach and Shai Shalev-Shwartz. “Decou- pling" when to update" from" how to update"”. In: Advances in neural information processing systems 30 (2017)
2017
-
[27]
Integrated analysis of multimodal single-cell data
Zhen Miao, Michael S Balzer, Ziyuan Ma, et al. “Integrated analysis of multimodal single-cell data”. In: bioRxiv (2020)
2020
-
[28]
Finite mixture modeling with mixture outcomes using the EM algorithm
Bengt Muthén and Kerby Shedden. “Finite mixture modeling with mixture outcomes using the EM algorithm”. In: Biometrics 55.2 (1999), pp. 463–469
1999
-
[29]
Opti- mized threshold inference for partitioning of clones from high-throughput B cell repertoire sequencing data
Nima Nouri and Steven H Kleinstein. “Opti- mized threshold inference for partitioning of clones from high-throughput B cell repertoire sequencing data”. In: Frontiers in immunology 9 (2018), p. 1687
2018
-
[30]
Making deep neural net- works robust to label noise: A loss correction approach
Giorgio Patrini et al. “Making deep neural net- works robust to label noise: A loss correction approach”. In: Proceedings of the IEEE Confer- ence on Computer Vision and Pattern Recogni- tion (CVPR) (2017), pp. 1944–1952
2017
-
[31]
Con- taminated speech training methods for robust DNN-HMM distant speech recognition
Mirco Ravanelli and Maurizio Omologo. “Con- taminated speech training methods for robust DNN-HMM distant speech recognition”. In: arXiv preprint arXiv:1710.03538 (2017)
2017 arXiv
-
[32]
Gaussian mix- ture models
Douglas A Reynolds et al. “Gaussian mix- ture models.” In: Encyclopedia of biometrics 741.659-663 (2009)
2009
-
[33]
Anomaly Detection with Variance Stabilized Density Estimation
Amit Rozner et al. “Anomaly Detection with Variance Stabilized Density Estimation”. In:Un- certainty in Artificial Intelligence. PMLR. 2024, pp. 3121–3137
2024
-
[34]
Domain-Generalizable Multiple-Domain Clustering
Amit Rozner et al. “Domain-Generalizable Multiple-Domain Clustering”. In: Transactions on Machine Learning Research ()
-
[35]
Cyclical learning rates for training neural networks
Leslie N Smith. “Cyclical learning rates for training neural networks”. In: 2017 IEEE winter conference on applications of computer vision (WACV). IEEE. 2017, pp. 464–472
2017
-
[36]
Learning from noisy labels with deep neural networks: A survey
Hwanjun Song et al. “Learning from noisy labels with deep neural networks: A survey”. In: IEEE transactions on neural networks and learning systems 34.11 (2022), pp. 8135–8153
2022
-
[37]
Symmetric cross entropy for robust learning with noisy labels
Yisen Wang et al. “Symmetric cross entropy for robust learning with noisy labels”. In: Proceed- ings of the IEEE/CVF international conference on computer vision. 2019, pp. 322–330
2019
-
[38]
Learning with noisy labels revisited: A study using real-world human an- notations
Jiaheng Wei et al. “Learning with noisy labels revisited: A study using real-world human an- notations”. In: arXiv preprint arXiv:2110.12088 (2021)
2021 arXiv
-
[39]
Learning to teach with dynamic loss functions
Lijun Wu et al. “Learning to teach with dynamic loss functions”. In: Advances in neural informa- tion processing systems 31 (2018)
2018
-
[40]
NoisywikiHow: A Benchmark for Learning with Real-world Noisy Labels in Natural Language Processing
Tingting Wu, Xiao Ding, et al. NoisywikiHow: A Benchmark for Learning with Real-world Noisy Labels in Natural Language Processing. 2023
2023
-
[41]
Are Anchor Points Really Indispensable in Label-Noise Learning? 2019
Xiaobo Xia, Tongliang Liu, et al. Are Anchor Points Really Indispensable in Label-Noise Learning? 2019
2019
-
[42]
L_dmi: A novel information- theoretic loss function for training deep nets ro- bust to label noise
Yilun Xu et al. “L_dmi: A novel information- theoretic loss function for training deep nets ro- bust to label noise”. In: Advances in neural in- formation processing systems 32 (2019)
2019
-
[43]
Dual t: Reducing estimation error for transition matrix in label-noise learning
Yu Yao et al. “Dual t: Reducing estimation error for transition matrix in label-noise learning”. In: Advances in neural information processing sys- tems 33 (2020), pp. 7260–7271
2020
-
[44]
Probabilistic end-to- end noise correction for learning with noisy la- bels
Kun Yi and Jianxin Wu. “Probabilistic end-to- end noise correction for learning with noisy la- bels”. In: Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition. 2019, pp. 7017–7025
2019
-
[45]
Challenges and emerg- ing directions in single-cell analysis
Guo-Cheng Yuan et al. “Challenges and emerg- ing directions in single-cell analysis”. In: Genome biology 18 (2017), pp. 1–8
2017
-
[46]
mixup: Beyond Empirical Risk Minimization
Hongyi Zhang et al. mixup: Beyond Empirical Risk Minimization. 2018. arXiv: 1710.09412 [cs.LG]. URL: https : // arxiv . org / abs/1710.09412. 6 SUPPLEMENTARY MATERIAL 6.1 Statistical Significance Analysis We compared D&C-Net with the strongest baseline, V olMinNet, using Welch...
2018 arXiv
- [2020]
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.