Pith. sign in

REVIEW 3 major objections 5 minor 67 references

CLID-MU: Cross-Layer Information Divergence Based Meta Update Strategy for Learning with Noisy Labels

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A label-free divergence between two network layers can replace the clean validation set that meta-learning for noisy labels normally needs.

desk verdict Useful label-free heuristic for noisy-label meta-learning, but the core correlation evidence is between runs, not within a run; needs a temporal validation before the mechanism is convincing. read the letter →

arxiv 2507.11807 v1 pith:RNXGU4PT submitted 2025-07-16 cs.LG cs.AI

Ruofan Hu , Dongyu Zhang , Huayi Zhang , Elke Rundensteiner This is my paper
classification cs.LGcs.AI
keywords noisylabelsmeta-learningunsupervisedevaluationmetriccross-layerdivergenceneuralnetworkssnapshotensemblingsemi-supervisedlearningsamplereweighting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CLID-MU claims that meta-learning under label noise can be guided without any clean validation data. The paper's premise is that clean samples preserve the consistency of data structure between the last hidden layer and the output layer, while noisy samples break it, so the divergence between the two layer graphs, computed without labels, tracks how well the model is training. On that basis the paper replaces the supervised meta-loss (cross-entropy or MAE on a meta-dataset) with this unsupervised divergence, and reports that the resulting CLID-MU matches or beats prior meta-learning methods, including the clean-meta-dataset ceiling, on CIFAR benchmarks with synthetic and human noise. The same module also lifts semi-supervised pipelines such as FixMatch, UDA, and FlexMatch when their labeled portion is noisy.

What carries the argument

Cross-Layer Information Divergence (CLID): given a batch, the embedding graph G_e with entries exp(cos(z_i, z_j)/tau) and the class-probability graph G_q with entries cos(q_i, q_j), both row-normalized to valid distributions, with L_clid = -(1/$m^{2}$) sum_i sum_j hat(G)_q,ij log hat(G)_e,ij. It carries the argument in two places: as the signal that the last hidden layer's structure has converged toward the output layer's structure, and as the differentiable meta-loss whose gradient with respect to the virtually-updated model parameters defines the per-sample gradient similarity g_i in the meta-update, steering the meta-model's sample weights without ever consulting labels. A stop-gradient operation prevents trivial constant solutions, and snapshot ensembling selects the top-K checkpoints by lowest CLID for inference.

What would settle it

Run the Section 4.2 procedure on a setting outside the paper, for example a Vision Transformer on ImageNet-1K with 40% symmetric label noise, and compute the epoch-wise Pearson correlation between CLID and clean-test cross-entropy. If the correlation drops below the paper's own strong-correlation threshold (rho > 0.7), or if the top-5 snapshots picked by lowest CLID are not the most accurate models on the test set, the central premise is refuted. A cheaper check on the same data would track per-sample meta-weights under heavier instance-dependent noise to see whether clean samples keep receiving the largest weights as training progresses.

Watch

Extended reading notes

Core claim

The central discovery is the CLID metric and its use as a meta-objective. For a batch of samples, CLID builds two fully connected similarity graphs: an embedding graph with entries exp(cos(z_i, z_j)/tau) from the penultimate layer, and a class-probability graph with entries cos(q_i, q_j) from the output layer, both row-normalized into distributions. The divergence is their cross-entropy, computed with the output-layer graph as the target and the embedding graph as the prediction, justified by the observation that the classifier layer updates faster than the hidden layer, so the slower layer should align toward the faster one. The paper demonstrates empirically that the epoch-wise Relative Performance Ratio of CLID tracks that of clean-test cross-entropy and that the Pearson correlation between the two stays high across noise types, leading to the conclusion that CLID is a valid label-free proxy for model quality. Replacing the meta-loss in WNet and VRI with CLID yields accuracy that in several settings exceeds the version using a genuinely clean meta-dataset, and the method extends to semi-supervised noisy-label settings and composes with DivideMix. The paper also proves an upper bound on the exponential loss of snapshot ensembling, showing the bound decreases as the number of ensembled snapshots grows.

Load-bearing premise

The method stands or falls on the empirical correlation of Section 4.2: that CLID, measured without any labels, tracks the model's true test error across noise settings, a correlation shown only on CIFAR-10 and CIFAR-100 with ResNet-34. If that alignment signal diverges from test error for another architecture, dataset, noise type, or training schedule, the CLID-guided meta-update and snapshot selection lose their justification.

Editorial extensions

If this is right

  • Meta-learning for noisy labels can run without a clean, balanced meta-dataset: a randomly sampled or unlabeled batch of the training data supplies the evaluation signal.
  • In the reported experiments, CLID-MU outperforms cross-entropy on noisy meta-data and MAE-based meta-objectives, and in several settings beats even the clean-meta-dataset ceiling for both WNet and VRI.
  • Plugging WNet-CLID into semi-supervised learners (UDA, FixMatch, FlexMatch) improves test accuracy under symmetric, asymmetric, and human label noise, with the largest gains at high noise ratios.
  • CLID-based selection of the top-5 snapshots improves ensemble accuracy over single-model accuracy, and Theorem 1 supports the practice by bounding the exponential loss by a product that decreases as the snapshot count grows.
  • CLID-MU composes with other noisy-label components such as DivideMix, MixUp, and ELR, making it an additive module rather than a competing pipeline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The correlation claim in Section 4.2 was measured on ResNet-34 with CIFAR-10 and CIFAR-100 only; whether CLID tracks generalization for transformers, large-scale or domain-shifted data, or other optimizers is untested, and that is the first probe of the premise's scope.
  • The same label-free alignment signal could plausibly serve model selection in other weakly supervised regimes that struggle with validation-free evaluation, such as partial-label learning, which the paper itself cites as a neighboring problem; CLID's differentiability is what lets it plug into gradient-based meta-updates, and that advantage would carry over.
  • Appendix D discloses a genuine cost: CLID-MU takes roughly 140 seconds per epoch versus about 14 for baselines on a single A100 GPU, a tenfold overhead that the paper mitigates only as future work (sparse top-K graphs, approximate nearest neighbor search, and less frequent meta-updates); any deployment must trade this compute against the cost of collecting clean labels.
  • Because CLID is a batch-wise pairwise statistic, its signal likely depends on batch composition; the paper's own Table 6 shows accuracy improves when the meta-train batch exceeds 50 samples under the hardest real-world noise, hinting that small-batch regimes could be the failure boundary.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes CLID-MU, a meta-learning strategy for learning with noisy labels that avoids a clean labeled meta-dataset. The method defines an unsupervised metric, CLID, as the cross-entropy between row-normalized similarity graphs of the last hidden layer and the output layer, argues that CLID tracks model quality, and uses CLID both as the meta-loss for updating a reweighting meta-model and as the criterion for selecting snapshots for ensembling. Experiments with WNet and VRI on CIFAR-10/100 under symmetric, asymmetric, instance-dependent, and real-world noise, plus semi-supervised settings, show competitive or superior accuracy relative to baselines that use noisy or pseudo-clean meta-datasets, and sometimes relative to methods with a clean meta-dataset. The paper also states a convergence bound for snapshot ensembling.

Significance. If the central correlation claim is valid, CLID-MU is a useful contribution: it removes the clean-validation-set requirement from meta-learning approaches to label noise, is model-agnostic, is compatible with existing LNL and SSL frameworks, and comes with released code. The empirical scope is broad, including synthetic and real-world noise across multiple benchmarks and integration into DivideMix, UDA, FixMatch, and FlexMatch. However, the paper's load-bearing premise, that CLID is a reliable within-run model-quality signal, is not directly validated; the reported correlations are computed across noise settings at fixed epochs, whereas the method's meta-update and snapshot selection operate across epochs within a single run. The theoretical result on snapshot ensembling is also generic and does not involve CLID. These gaps are fixable with additional experiments and reframing, but they are central to the paper's claims.

major comments (3)
  1. [Section 4.4 and Appendix A, Theorem 1] The evidence that CLID tracks model quality is a between-run correlation: Definition 2 fixes an epoch t and computes Pearson's r across the set N of noise settings, and Figures 2(c)/3(c) report this quantity at each epoch. This shows that models trained under heavier noise tend to have higher CLID at the same training time, but the proposed method uses CLID as a within-run signal: Eq. (10) differentiates L_clid with respect to the virtual-updated parameters at each iteration, and Algorithm 1 (lines 8-17) together with Section 4.4 compares CLID scores of snapshots taken at different epochs of the same run and selects the lowest. No experiment in Section 4.2 or Section 5 reports the within-run relationship between CLID and clean accuracy for a fixed noise level and architecture. Figure 7 shows CLID decreasing monotonically up to 17,500 steps on CIFAR-10N Worst, but clean accuracy is not overlaid, so the plot cannot rule out the failure mode in which a memorizing model's output-probability graph becomes sharp and more aligned with the embedding graph while clean accuracy deteriorates. The sensitivity analysis in Figure 5 compares CLID scores across temperature settings, not across training time, so it also does not supply the missing temporal evidence. The paper should provide per-run plots of CLID and clean accuracy over training, and evaluate whether the minimal-CLID snapshot and the CLID-selected ensemble match the best achievable snapshot accuracy.
  2. [Section 5.1, Tables 1a/1b, Section 5.2] Theorem 1 is a statement about any set of snapshots, not about the CLID metric. The bound L_exp <= product_k R_k^{1/K} follows from Holder's inequality applied to the averaged predictions and holds for arbitrary f_k; no property of CLID, of the top-K selection rule, or of noisy-label training is used in the proof. The inequality in Eq. (12) is in fact an equality for real-valued f, since exp(-f/K) = (exp(-f))^{1/K}, so the only substantive step is the AM-GM/Holder step. The sentence 'Theorem 1 demonstrates an exponential decrease in the L_exp bound as we save more snapshots' is also stronger than what the bound shows: with R_k in [0,1], the product bound need not decay exponentially in K. As stated, the theorem does not support the claim that CLID-selected snapshots are good snapshots; it should be removed, reframed as a generic ensemble bound, or replaced by an analysis that actually involves the CLID selection criterion.
  3. [Section 5.1, Tables 1a/1b, Section 5.2] In the experimental section, 'best accuracy' is defined as the highest accuracy on the clean test set achieved at any point during training. This is an oracle quantity: in the paper's stated setting, no clean labels are available, so a practitioner cannot select the best snapshot by this criterion. The practical accuracy is the ensemble accuracy obtained from the top-K CLID snapshots, and that quantity should carry the main empirical claims. The paper should report, for each setting, the clean accuracy of the CLID-selected snapshot and of the CLID-selected ensemble, and compare these with the best-achievable snapshot accuracy; at present the reader cannot tell how much of the reported advantage of WNet-CLID/VRI-CLID depends on oracle selection. This is closely related to the within-run validation issue and should be addressed together with it.
minor comments (5)
  1. [Table 1a] In the WNet-CE(p) row for CIFAR-100 Sym. 60%, the reported value '63.60±71' appears to be missing a decimal point; it should likely read '63.60±0.71' or '63.60±.71'.
  2. [Definition 2] The formula for the Pearson correlation uses summation over n in the numerator and denominator without clearly stating that n indexes the elements of N; the notation should be made consistent with the set N defined in the text.
  3. [Section 4.1] The term 'divergence' is used for a quantity that is the cross-entropy between row-normalized graphs and is not symmetric; this should be stated explicitly to avoid implying that CLID is a metric in the information-geometric sense.
  4. [Appendix A] The proof states exp(-f_k_y(x)) <= 1; this requires f_k_y(x) >= 0, which holds when f is a probability output but should be stated explicitly, since earlier notation allows f to denote a general model output.
  5. [Figure 7] The axis label 'Traing step' contains a typo and should read 'Training step'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found: CLID is an unsupervised, label-free metric validated against held-out test accuracy, and no equation or fitted parameter reduces the claimed result to its inputs.

full rationale

The paper's central claim is that CLID, defined in Eq. (9) as the cross-entropy between row-normalized class-probability and embedding similarity graphs, correlates with model performance under label noise. This is an empirical claim established by direct measurement on CIFAR-10 and CIFAR-100 across noise settings (Figures 2 and 3). CLID is computed entirely from model outputs and data, not from clean labels, and no parameter is fitted to the clean-label CE loss or test accuracy. The meta-update in Eq. (10) and snapshot selection in Algorithm 1 use CLID as a differentiable, label-free proxy; the reported accuracies are measured on held-out clean test sets, so the evaluation is external to the CLID objective. No self-definitional reduction, fitted-input-as-prediction, or load-bearing self-citation chain is present. The concern that the Section 4.2 correlation is across noise settings rather than within a single training run is a validity threat to the method's motivation, not a circularity under the stated criteria, because CLID is not constructed from the target metric. Theorem 1 is a generic bound on snapshot ensembling that does not depend on CLID. The paper is therefore self-contained against external benchmarks, and the appropriate finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The method introduces one new entity (CLID) and relies on three empirical assumptions: the cluster assumption, the layer-alignment direction assumption, and the generalization of the CLID-performance correlation beyond CIFAR. Free parameters are the temperature tau, snapshot count K, and meta-dataset size. The load-bearing assumption is the transferability of the correlation.

free parameters (3)
  • tau (temperature scaling) = 0.5
    Chosen from a sweep in Section 5.4 across {0.1, 0.3, 0.5, 0.7, 1, 1.5}; the paper suggests it can be tuned based on average CLID scores, which means the final value is selected using the validation signal.
  • K (number of snapshots) = 5
    Set for all experiments (Appendix C); the paper shows that the bound in Theorem 1 improves with larger K but does not justify why 5 is used.
  • Meta-dataset size = 1000
    Fixed at 1000 for all main experiments; no sensitivity analysis is provided for this choice.
assumptions (4)
  • domain assumption Cluster assumption: samples forming a structure are more likely to belong to the same class.
    Invoked in Section 4 to justify the CLID metric.
  • ad hoc to paper Gradient magnitude of the classifier layer is larger than that of the hidden layer, so the hidden distribution should align with the output distribution.
    Stated in Section 4.1 and supported by Figure 1 for WRN-28-10 and ResNet32, but there is no general proof that alignment in the direction chosen (cross-entropy from output graph to feature graph) rather than the reverse is correct.
  • domain assumption The empirical correlation between CLID and CE loss on CIFAR-10 and CIFAR-100 generalizes to other datasets, architectures, and noise types.
    The entire method rests on the correlation study in Section 4.2, which uses only ResNet-34 on CIFAR-10 and CIFAR-100 with finite noise settings; the claim is extrapolated to Animal-10N, Clothing1M, and other backbones.
  • standard math The meta-learning bilevel optimization with first-order Taylor expansion and stop-gradient is a valid approximation.
    The optimization procedure follows prior work [24], [55] and the paper does not question it.
invented entities (1)
  • CLID metric independent evidence
    purpose: Unsupervised evaluation metric for model performance without clean labels.
    CLID is a new metric, but it is directly defined on model outputs and is testable; the paper provides correlation plots and uses it for model selection, so it is falsifiable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CLID-MU: Cross-Layer Information Divergence Based Meta Update Strategy for Learning with Noisy Labels." pith.science (2026). https://pith.science/paper/RNXGU4PT

@misc{pith2026250711807,
  author       = {Pith},
  title        = {Pith review of: CLID-MU: Cross-Layer Information Divergence Based Meta Update Strategy for Learning with Noisy Labels},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RNXGU4PT}},
  note         = {Machine review of arXiv:2507.11807}
}
read the original abstract

Learning with noisy labels (LNL) is essential for training deep neural networks with imperfect data. Meta-learning approaches have achieved success by using a clean unbiased labeled set to train a robust model. However, this approach heavily depends on the availability of a clean labeled meta-dataset, which is difficult to obtain in practice. In this work, we thus tackle the challenge of meta-learning for noisy label scenarios without relying on a clean labeled dataset. Our approach leverages the data itself while bypassing the need for labels. Building on the insight that clean samples effectively preserve the consistency of related data structures across the last hidden and the final layer, whereas noisy samples disrupt this consistency, we design the Cross-layer Information Divergence-based Meta Update Strategy (CLID-MU). CLID-MU leverages the alignment of data structures across these diverse feature spaces to evaluate model performance and use this alignment to guide training. Experiments on benchmark datasets with varying amounts of labels under both synthetic and real-world noise demonstrate that CLID-MU outperforms state-of-the-art methods. The code is released at https://github.com/ruofanhu/CLID-MU.

Figures

Figures reproduced from arXiv: 2507.11807 by the authors.

Figure 1
Figure 1. Illustration of gradient magnitudes in WRN-28-10 [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Demonstration on CIFAR-10: The relative performance ratio of (a) Cross-Entropy (CE) loss, (b) CLID, and (c) Pearson’s [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Demonstration on CIFAR-100: The relative performance ratio of (a) Cross-Entropy (CE) loss, (b) CLID, and (c) Pearson’s [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: (a) Illustration of the three major steps of a meta-learning method, using a reweight-based approach as an example. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Test accuracy across different values of temperature [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: The weight distribution of clean and noisy samples in the experiment of FlexMatch on CIFAR-10N (Worst). [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Trend of meta loss used in the Meta-Train step in the experiment of FlexMatch on CIFAR-10N Worst label. Left: our [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

67 extracted references · 51 canonical work pages

  1. [1]

    Yingyi Chen, Xi Shen, Shell Xu Hu, and Johan AK Suykens. 2021. Boosting co-teaching with compression regularization for label noise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2688–2692

    work page 2021

  2. [2]

    Erik Englesson and Hossein Azizpour. 2021. Generalized jensen-shannon di- vergence loss for learning with noisy labels. Advances in Neural Information Processing Systems 34 (2021), 30284–30297

    work page 2021

  3. [3]

    Vasilii Feofanov, Malik Tiomoko, and Aladin Virmaux. 2023. Random matrix analysis to balance between supervised and unsupervised learning under the low density separation assumption. In International Conference on Machine Learning . PMLR, 10008–10033

    work page 2023

  4. [4]

    Shanti Sastry

    Aritra Ghosh, Himanshu Kumar, and P. Shanti Sastry. 2017. Robust Loss Functions under Label Noise for Deep Neural Networks. In AAAI Conference on Artificial Intelligence. https://api.semanticscholar.org/CorpusID:6546734

    work page 2017

  5. [5]

    Aritra Ghosh and Andrew Lan. 2021. Do we really need gold samples for sample weighting under label noise?. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision . 3922–3931

    work page 2021

  6. [6]

    Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. 2018. Co-teaching: Robust training of deep neural net- works with extremely noisy labels. Advances in neural information processing systems 31 (2018)

    work page 2018

  7. [7]

    Zhang, Shaoqing Ren, and Jian Sun

    Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), 770–778. https://api.semanticscholar.org/CorpusID: 206594692

    2015

  8. [8]

    Dennis Hofmann, Peter VanNostrand, Huayi Zhang, Yizhou Yan, Lei Cao, Samuel Madden, and Elke Rundensteiner. 2022. A demonstration of autood: a self-tuning anomaly detection system. Proceedings of the VLDB Endowment 15, 12 (2022), 3706–3709

    work page 2022

Show all 67 references
  1. [9]

    Dennis M Hofmann, Peter M VanNostrand, Lei Ma, Huayi Zhang, Joshua C DeOliveira, Lei Cao, and Elke A Rundensteiner. 2025. Agree to Disagree: Robust Anomaly Detection with Noisy Labels. Proceedings of the ACM on Management of Data 3, 1 (2025), 1–24

    2025

  2. [10]

    Ruofan Hu, Dongyu Zhang, Dandan Tao, Huayi Zhang, Hao Feng, and Elke Rundensteiner. 2023. Uce-fid: Using large unlabeled, medium crowdsourced- labeled, and small expert-labeled tweets for foodborne illness detection. In 2023 IEEE International Conference on Big Data (BigData) ...

    2023

  3. [11]

    Gao Huang, Yixuan Li, Geoff Pleiss, Zhuang Liu, John E Hopcroft, and Kilian Q Weinberger. 2017. Snapshot ensembles: Train 1, get m for free. arXiv preprint arXiv:1704.00109 (2017)

    2017 arXiv

  4. [12]

    Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. 2018. Mentor- net: Learning data-driven curriculum for very deep neural networks on corrupted labels. In International conference on machine learning . PMLR, 2304–2313

    2018

  5. [13]

    Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. (2009)

    2009

  6. [14]

    Junnan Li, Richard Socher, and Steven CH Hoi. 2020. Dividemix: Learning with noisy labels as semi-supervised learning. arXiv preprint arXiv:2002.07394 (2020)

    2020 arXiv

  7. [15]

    Muyang Li, Xiaobo Xia, Runze Wu, Fengming Huang, Jun Yu, Bo Han, and Tongliang Liu. 2024. Towards realistic model selection for semi-supervised learning. In Forty-first International Conference on Machine Learning

    2024

  8. [16]

    Xuefeng Li, Tongliang Liu, Bo Han, Gang Niu, and Masashi Sugiyama. 2021. Provably end-to-end label-noise learning without anchor points. In International conference on machine learning . PMLR, 6403–6413

    2021

  9. [17]

    Shan, and Xilin Chen

    Yifan Li, Hu Han, S. Shan, and Xilin Chen. 2023. DISC: Learning from Noisy Labels via Dynamic Instance-Specific Selection and Correction. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023), 24070– 24079. https://api.semanticscholar.org/CorpusID:260068694

    2023

  10. [18]

    Sheng Liu, Jonathan Niles-Weed, Narges Razavian, and Carlos Fernandez-Granda

  11. [19]

    Sheng Liu, Zhihui Zhu, Qing Qu, and Chong You. 2022. Robust training under label noise by over-parameterization. In International Conference on Machine Learning. PMLR, 14153–14172

    2022

  12. [20]

    Xingjun Ma, Hanxun Huang, Yisen Wang, Simone Romano, Sarah Monazam Erfani, and James Bailey. 2020. Normalized Loss Functions for Deep Learning with Noisy Labels. In International Conference on Machine Learning . https: //api.semanticscholar.org/CorpusID:220042066

    2020

  13. [21]

    Deep Patel and PS Sastry. 2023. Adaptive sample selection for robust learning under label noise. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 3932–3942

    2023

  14. [22]

    Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. 2017. Making deep neural networks robust to label noise: A loss correction approach. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1944–1952

    2017

  15. [23]

    Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. 2018. Learning to reweight examples for robust deep learning. In International conference on machine learning. PMLR, 4334–4343

    2018

  16. [24]

    Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. 2019. Meta-Weight-Net: Learning an Explicit Mapping For Sample Weight- ing. In Neural Information Processing Systems . https://api.semanticscholar.org/ CorpusID:173188221

    2019

  17. [25]

    Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. 2020. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in neural information processing systems...

    2020

  18. [26]

    Hwanjun Song, Minseok Kim, and Jae-Gil Lee. 2019. Selfie: Refurbishing unclean samples for robust deep learning. In International conference on machine learning . PMLR, 5907–5915

    2019

  19. [27]

    Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. 2022. Learning from noisy labels with deep neural networks: A survey.IEEE transactions on neural networks and learning systems 34, 11 (2022), 8135–8153

    2022

  20. [28]

    Haoliang Sun, Qi Wei, Lei Feng, Yupeng Hu, Fan Liu, Hehe Fan, and Yilong Yin. 2024. Variational Rectification Inference for Learning with Noisy Labels. International Journal of Computer Vision (2024), 1–20. KDD ’25, August 3–7, 2025, Toronto, ON, Canada Ruofan Hu, Dongyu Zhang...

    2024

  21. [29]

    Mitchell Keren Taraday and Chaim Baskin. 2023. Enhanced Meta Label Correction for Coping with Label Corruption. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 16295–16304

    2023

  22. [30]

    Yuanpeng Tu, Boshen Zhang, Yuxi Li, Liang Liu, Jian Li, Jiangning Zhang, Yabiao Wang, Chengjie Wang, and Cai Rong Zhao. 2023. Learning from Noisy Labels with Decoupled Meta Label Purifier.2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023), 19934–...

    2023

  23. [31]

    Deng-Bao Wang, Yong Wen, Lujia Pan, and Min-Ling Zhang. 2021. Learning from noisy labels with complementary loss functions. In Proceedings of the AAAI conference on artificial intelligence , Vol. 35. 10111–10119

    2021

  24. [32]

    Qi Wang, Yue Ma, Kun Zhao, and Yingjie Tian. 2020. A comprehensive survey of loss functions in machine learning. Annals of Data Science (2020), 1–26

    2020

  25. [33]

    Renzhen Wang, Xixi Jia, Quanziang Wang, Yichen Wu, and Deyu Meng. 2022. Imbalanced Semi-supervised Learning with Bias Adaptive Classifier. In Interna- tional Conference on Learning Representations . https://api.semanticscholar.org/ CorpusID:257279756

    2022

  26. [34]

    Wei Wang, Dong-Dong Wu, Jindong Wang, Gang Niu, Min-Ling Zhang, and Masashi Sugiyama. 2025. Realistic Evaluation of Deep Partial-Label Learning Algorithms. arXiv preprint arXiv:2502.10184 (2025)

    2025 arXiv

  27. [35]

    Yidong Wang, Hao Chen, Yue Fan, Wang Sun, Ran Tao, Wenxin Hou, Renjie Wang, Linyi Yang, Zhi Zhou, Lan-Zhe Guo, et al . 2022. Usb: A unified semi- supervised learning benchmark for classification. Advances in Neural Information Processing Systems 35 (2022), 3938–3961

    2022

  28. [36]

    Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, and Yang Liu

  29. [37]

    Yang Wu, Huayi Zhang, Yizheng Jiao, Lin Ma, Xiaozhong Liu, Jinhong Yu, Dongyu Zhang, Dezhi Yu, and Wei Xu. 2024. ROSE: A Reward-Oriented Data Selection Framework for LLM Task-Specific Instruction Tuning. arXiv preprint arXiv:2412.00631 (2024)

    2024 arXiv

  30. [38]

    Xiaobo Xia, Tongliang Liu, Bo Han, Nannan Wang, Mingming Gong, Haifeng Liu, Gang Niu, Dacheng Tao, and Masashi Sugiyama. 2020. Part-dependent label noise: Towards instance-dependent label noise. Advances in Neural Information Processing Systems 33 (2020), 7597–7610

    2020

  31. [39]

    Xiaobo Xia, Tongliang Liu, Nannan Wang, Bo Han, Chen Gong, Gang Niu, and Masashi Sugiyama. 2019. Are anchor points really indispensable in label-noise learning? Advances in neural information processing systems 32 (2019)

    2019

  32. [40]

    Tong Xiao, Tian Xia, Yi Yang, Chang Huang, and Xiaogang Wang. 2015. Learning from massive noisy labeled data for image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition . 2691–2699

    2015

  33. [41]

    Qizhe Xie, Zihang Dai, Eduard Hovy, Minh Thang Luong, and Quoc V. Le. 2019. Unsupervised Data Augmentation for Consistency Training. Advances in Neural Information Processing Systems 2020-Decem (4 2019). https://arxiv.org/abs/1904. 12848v6

    2019

  34. [42]

    Youjiang Xu, Linchao Zhu, Lu Jiang, and Yi Yang. 2021. Faster meta update strategy for noise-robust deep learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 144–153

    2021

  35. [43]

    Yu Yao, Tongliang Liu, Bo Han, Mingming Gong, Jiankang Deng, Gang Niu, and Masashi Sugiyama. 2020. Dual t: Reducing estimation error for transition matrix in label-noise learning. Advances in neural information processing systems 33 (2020), 7260–7271

    2020

  36. [44]

    Linya Yi, Sheng Liu, Qi She, Alex McLeod, and Boyu Wang. 2022. On Learning Contrastive Representations for Learning with Noisy Labels. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022), 16661– 16670. https://api.semanticscholar.org/CorpusID:247223119

    2022

  37. [45]

    LIN Yong, Renjie Pi, Weizhong Zhang, Xiaobo Xia, Jiahui Gao, Xiao Zhou, Tongliang Liu, and Bo Han. 2022. A holistic view of label noise transition matrix in deep learning and beyond. InThe Eleventh International Conference on Learning Representations

    2022

  38. [46]

    Suqin Yuan, Lei Feng, and Tongliang Liu. 2025. Early stopping against label noise without validation data. arXiv preprint arXiv:2502.07551 (2025)

    2025 arXiv

  39. [47]

    Sergey Zagoruyko and Nikos Komodakis. 2016. Wide Residual Networks. ArXiv abs/1605.07146 (2016). https://api.semanticscholar.org/CorpusID:15276198

    2016 arXiv

  40. [48]

    Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. 2021. Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling. Advances in Neural Information Pro- cessing Systems 34 (2021), 18408–18419

    2021

  41. [49]

    Dongyu Zhang, Ruofan Hu, and Elke Rundensteiner. 2024. CoLafier: Collaborative Noisy La bel Purifier With Local Intrinsic Dimensionality Guidance. InProceedings of the 2024 SIAM International Conference on Data Mining (SDM) . SIAM, 82–90

    2024

  42. [50]

    Huayi Zhang, Lei Cao, Samuel Madden, and Elke Rundensteiner. 2021. Lancet: labeling complex data at scale. Proceedings of the VLDB Endowment 14, 11 (2021)

    2021

  43. [51]

    Huayi Zhang, Lei Cao, Peter VanNostrand, Samuel Madden, and Elke A Run- densteiner. 2021. Elite: Robust deep anomaly detection with meta gradient. In Proceedings of the 27th ACM SIGKDD . 2174–2182

    2021

  44. [52]

    Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. 2017. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017)

    2017 arXiv

  45. [53]

    Huayi Zhang, Binwei Yan, Lei Cao, Samuel Madden, and Elke Rundensteiner

  46. [54]

    Xinyu Zhang, Hao Jia, Taihong Xiao, Ming-Ming Cheng, and Ming-Hsuan Yang

  47. [55]

    Yihua Zhang, Prashant Khanduri, Ioannis Tsaknakis, Yuguang Yao, Mingyi Hong, and Sijia Liu. 2024. An Introduction to Bilevel Optimization: Foundations and applications in signal processing and machine learning. IEEE Signal Processing Magazine 41, 1 (2024), 38–59

    2024

  48. [56]

    Yikai Zhang, Songzhu Zheng, Pengxiang Wu, Mayank Goswami, and Chao Chen

  49. [57]

    Zizhao Zhang and Tomas Pfister. 2021. Learning Fast Sample Re-weighting Without Reward Data. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) (2021), 705–714. https://api.semanticscholar.org/CorpusID:237431056

    2021

  50. [58]

    In International Conference on Artificial Intelligence and Statistics

    Semi-Supervised Learning with Meta-Gradient. In International Conference on Artificial Intelligence and Statistics . https://api.semanticscholar.org/CorpusID: 220404311

  51. [59]

    Zhilu Zhang and Mert Rory Sabuncu. 2018. Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels. ArXiv abs/1805.07836 (2018). https://api.semanticscholar.org/CorpusID:29164161

    2018 arXiv

  52. [60]

    Dengyong Zhou, Olivier Bousquet, Thomas Navin Lal, Jason Weston, and Bern- hard Scholkopf. 2003. Learning with Local and Global Consistency. In Neural Information Processing Systems. https://api.semanticscholar.org/CorpusID:508435

    2003

  53. [61]

    arXiv preprint arXiv:2103.07756 (2021)

    Learning with feature-dependent label noise: A progressive approach. arXiv preprint arXiv:2103.07756 (2021)

    2021 arXiv

  54. [63]

    Zhilu Zhang and Mert Sabuncu. 2018. Generalized cross entropy loss for training deep neural networks with noisy labels.Advances in neural information processing systems 31 (2018)

    2018

  55. [66]

    Dawei Zhu, Xiaoyu Shen, Marius Mosbach, Andreas Stephan, and Dietrich Klakow. 2023. Weaker Than You Think: A Critical Look at Weakly Supervised Learning. In Proceedings of the 61st Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers) , Anna...

    2023 doi

  56. [67]

    The 50,000 training data is split into 4,000 labeled samples and 46,000 unlabeled samples

    for all noise settings. The 50,000 training data is split into 4,000 labeled samples and 46,000 unlabeled samples. For CLID-MU, we sampled 1,000 instances from the unlabeled set to construct the meta-dataset, while for WNet-MAE, the meta-dataset was sampled from the labeled se...

    2025

  57. [2020]

    Ad- vances in neural information processing systems 33 (2020), 20331–20342

    Early-learning regularization prevents memorization of noisy labels. Ad- vances in neural information processing systems 33 (2020), 20331–20342

    2020

  58. [2021]

    arXiv preprint arXiv:2110.12088 (2021)

    Learning with noisy labels revisited: A study using real-world human annotations. arXiv preprint arXiv:2110.12088 (2021)

    2021 arXiv

  59. [2024]

    Association for Computing Machinery (ACM)

    Metastore: Analyzing deep learning meta-data at scale. Association for Computing Machinery (ACM)

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.