Pith. sign in

REVIEW 3 major objections 5 minor 71 references

Noisy Ostracods: A Fine-Grained, Imbalanced Real-World Dataset for Benchmarking Robust Machine Learning and Label Correction Methods

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper shows that on a real fine-grained ostracod dataset, learning-with-noisy-labels methods provide no significant gain over standard cross-entropy training, and a naive ensemble baseline detects label errors better than Confident…

desk verdict A genuinely new real-world noisy dataset whose abstract overstates the negative result: on accuracy, PLM beats CE by ~1% and the paper never runs a significance test. read the letter →

arxiv 2412.02313 v1 pith:VLISBUVA submitted 2024-12-03 cs.LG cs.CV

Jiamian Hu , Yuanyuan Hong , Yihua Chen , He Wang , Moriaki Yasuhara This is my paper · ORCID
classification cs.LGcs.CV
keywords noisylabelsfine-grainedclassificationreal-worldbenchmarklabelcorrectionclassimbalanceostracodtaxonomylearningwithensemblecross-validation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces Noisy Ostracods, a 71,466-image benchmark of ostracod shells with expert-vetted labels, and asks two questions: do learning-with-noisy-label (LNL) methods train more accurate classifiers than standard cross-entropy, and do specialized label-correction methods find more mislabeled images than a simple ensemble baseline? On this dataset, the answer to both is no. Cross-entropy training with ImageNet-pretrained backbones matches or beats every tested robust-training method, and a naive ensemble of 14 models cross-validated over ten splits (NECV) detects 71.19% of the manually confirmed label errors, outperforming Confident Learning (59.37%) and all other correction methods tested. The authors argue that the dataset's fine-grained classes, extreme imbalance ($\rho = 22429$), and noise types—including pseudo-classes, mixed classes, open-set new genera, and image-quality failures—make it a harder, more realistic test than synthetic-noise benchmarks, and they release it to push robust-learning research toward real-world conditions.

What carries the argument

The load-bearing object is the Noisy Ostracods dataset itself, a fine-grained image benchmark with 78 annotated genera and 138 annotated species, an imbalance factor $\rho = 22429$, a confirmed 5.58% genus-level noise rate on the 14,320 manually curated test/validation samples, and a documented taxonomy of noise types (feature errors such as photographic, fragmentation, position, and preprocessing problems; label errors such as pseudo-classes, mixed classes, new classes, and hard classes). The label-correction result is carried by the NECV algorithm: split data into 10 folds, train 14 diverse pretrained models on nine folds, count agreement with the original label across the 14 predictions, and flag agreement below 0.5. The noise transition matrix with the true, hand-derived transition probabilities is the mechanism used to test class-dependent noise assumptions.

What would settle it

Have a second, independent panel of ostracod experts re-examine a random sample of the 14,320 'cleaned' images without seeing the original corrections; if the confirmed noise rate moves substantially, or if re-running the Table 2 methods against the new labels changes NECV's lead, the reported rankings are an artifact of that curation. Alternatively, clean the full training set and check whether cross-entropy's 4.02% error rate and the LNL method rankings persist on held-out data.

Watch

Extended reading notes

Core claim

The paper's central discovery is that current noisy-label machinery does not deliver its advertised benefits on a real fine-grained taxonomy task. Across two backbones and eleven methods, standard cross-entropy training on raw noisy labels either outperforms or matches every LNL baseline; the only method to edge past CE in accuracy, PLM, does so by roughly one percentage point while sacrificing precision, recall, and F1. On the label-cleaning side, the bespoke detectors—Confident Learning, SimiFeat, AUM, and CINCER—all fall short of NECV, a baseline that simply counts how many of 14 diverse pretrained models agree with the original label and flags samples with less than half agreement. When the majority-wrong genus Paradoxostomid is excluded, NECV's hit rate rises above 90%. The authors interpret these results as evidence that assumptions underlying class-dependent noise models, such as the noise transition matrix, do not hold here, and that pre-trained models already absorb much of the real-world noise that specialized methods are designed to remove.

Load-bearing premise

The manual corrections of the 14,320 test and validation images are treated as error-free ground truth, and the 5.58% noise ratio measured there is assumed to represent the noise in the 57,000-image uncurated training set.

Editorial extensions

If this is right

  • ImageNet-pretrained cross-entropy training becomes the natural reference point for any new LNL method aimed at fine-grained taxonomy data.
  • Real-world benchmarks like this one are needed because synthetic-noise results on CIFAR-10-style data do not predict which methods will help in practice.
  • Label-cleaning tools should be compared against ensemble cross-validation baselines before being adopted in dataset curation pipelines.
  • Methods built on class-dependent noise transition assumptions will need to be extended, since even the true transition matrix underperforms in this setting.
  • The released cleaned validation and test labels give future work a fixed reference for measuring both classification robustness and cleaning effectiveness.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • NECV's advantage probably reflects ensemble diversity and its ability to catch majority-wrong classes, but the paper does not ablate the number of models or the agreement threshold, so a cheaper protocol with three or four architectures might recover most of the hit rate.
  • Because the training set is uncurated, the robustness rankings could shift once its noise distribution is measured; the released benchmark may currently understate the difficulty of cleaning the full dataset.
  • The documented project-specific errors and pseudo-class conventions suggest that metadata such as project ID, annotator, and cf./aff. markers could be exploited as privileged information by future label-correction methods, an avenue the paper identifies but does not pursue.
  • If the NECV result transfers, museum-style taxonomic imaging projects could assemble cleaning pipelines from off-the-shelf pretrained models and cross-validation rather than specialized noise detectors, lowering the barrier to building clean reference datasets.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Noisy Ostracods, a real-world fine-grained image dataset for ostracod genus and species classification with an estimated 5.58% genus-level noise ratio, extreme class imbalance, and multiple real-world noise types including feature errors, pseudo-classes, mixed classes, and open-set classes. The authors manually cleaned the 14,320 test and validation samples and report benchmark results comparing cross-entropy training with several Learning with Noisy Labels methods on ResNet-50 and ViT-B-16. They also compare label-error detection methods against a simple ensemble cross-validation baseline (NECV). The main claims are that LNL methods do not provide significant improvements over cross-entropy training on this dataset and that NECV detects label errors more effectively than specialized methods such as Confident Learning and SimiFeat.

Significance. If the claims hold, the dataset is a valuable addition to the sparse collection of real-world noisy benchmarks, particularly for fine-grained taxonomy and extreme class imbalance. The authors provide a detailed characterization of real-world noise sources, a manually curated evaluation set, transition matrices, and reproducible evaluation protocols, with code and data released. The paper is also explicit about limitations, including the lack of a cleaned training set. The NECV baseline is simple and transparent, and the negative results for specialized LNL methods are informative for the community. However, the key negative claim about LNL methods requires more rigorous statistical support before the benchmark's headline conclusion can be accepted.

major comments (3)
  1. [§4.1 and Appendix H, Tables 4–5] The central claim that LNL methods 'have not yielded significant performance improvements' over CE is not supported by the reported statistics. Table 1 reports only the best of 5 runs, while the Appendix H means show PLM with ResNet-50 at 96.70±0.06 vs CE at 95.77±0.35, which a Welch t-test on the summary statistics would reject as equal (approximate t≈5.9, p≈0.003); for ViT-B-16 the corresponding means are 96.10±0.37 vs 94.21±1.59 (approximate p≈0.055). The text acknowledges PLM's higher accuracy but dismisses it as 'around 1%' without a significance test. The claim may survive if restricted to macro-F1, precision, or recall, or if 'significant' is defined as practically meaningful, but the paper must either report paired tests across metrics or explicitly qualify the claim. As written, the abstract and conclusion overstate the negative result.
  2. [§3.3 and §5] The 5.58% noise ratio is stated in the abstract as an estimate for the whole dataset, but it is measured only on the 14,320 manually cleaned test/validation samples, while the training set remains uncleaned. The paper itself lists the absence of a cleaned training set as the biggest limitation. Since the training set constitutes 80% of the data and may have a different noise distribution (e.g., different projects and annotation years, as shown in Table 3), the extrapolation from the cleaned 20% to the full 71,466 samples is an unsupported assumption. Please rephrase the abstract and Section 3.3 to state explicitly that 5.58% is the confirmed noise ratio in the cleaned test/validation portion, and discuss the uncertainty in extrapolating it to the training set.
  3. [§4.1, Table 1] The main table's choice of reporting the best of 5 runs, rather than the mean, makes the headline comparison unreliable and is inconsistent with the variability shown in Appendix H. For example, Loss-clip on ResNet-50 has mean accuracy 62.43±32.98 across runs, so its Table 1 value of 95.79 is not representative. Please move the mean and variance (or a compatible confidence interval) into the main table, or add per-run results in the appendix with paired comparisons, so that the reader can evaluate the 'no significant improvement' claim directly from the main text.
minor comments (5)
  1. [Abstract and §3.3] Please clarify that the benchmark experiments are genus-level classification; the abstract mentions both genus and species classification, but the cleaned evaluation sets and all reported results are at the genus level, and species-level cleaning is explicitly deferred to future work.
  2. [§4.1] There are several typos: 'Expect the transition matrix whose result is not directly incomparable' should be 'Except for the transition matrix, whose result is not directly comparable', and 'aggrated' should be 'aggregated'.
  3. [§4.2, Table 2] The table header formatting is difficult to parse ('Hit', 'Feature', 'Label', 'Fix', 'Found P.', 'Hit rate' are packed together). Please reformat the header so that each column has a clear name, and define 'Fix prec.' explicitly, since the text discusses precision only generically.
  4. [§3.2, Eq. (1)] The notation in Equation (1) is confusing: sup[P*(yi∈m*|xi)] is written as if it were a supremum over a random variable; please rewrite the definition of feature error in plain language with clearer probabilistic notation.
  5. [Appendix E] The transition-matrix experiment is described as an upper bound in the appendix, but Section 4.1 uses its poor performance to conclude that class-dependent noise assumptions fail. This is reasonable, but the main text should remind the reader that the low classification accuracy of this method is partly due to the 51-class subset and the training protocol, not solely to the invalidity of the class-dependent assumption.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark claims are evaluated against external manual corrections; disclosed oracle/leakage experiments are clearly labeled as upper-bound, not predictions.

full rationale

The paper's central claims are empirical: robust LNL methods do not beat cross-entropy training, and NECV detects more label errors than specialized methods. Both claims are evaluated against manually curated labels that are external to the methods, so they are not equivalent to the methods' inputs by construction. The only oracle-style experiment is the noise-transition-matrix method (Appendix E, Eqs. 4-5), which uses the true transition matrix obtained from the manual validation process; the paper explicitly labels this as 'information leakage' and 'primarily theoretical,' so it is not presented as a prediction. Similarly, the Dynamic Loss implementation (Appendix H) uses the cleaned validation set as the meta set and discloses that the strong performance 'may be partially attributed to information leakage.' These are methodological limitations, not circular derivations. Self-citations [2,3] support data provenance only and are not load-bearing for any conclusion. The statistical concern that PLM's mean accuracy exceeds CE on ResNet-50 (Appendix Tables 4-5) without a paired significance test is a correctness or evidential issue, not a circularity issue.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claims rest on the reliability of the expert cleaning process and on the transferability of noise statistics from the cleaned subsets to the uncorrected training set. The only hand-chosen experimental parameters are the NECV threshold and the Loss-clip threshold, neither of which is varied to assess sensitivity. No new physical or theoretical entities are introduced.

free parameters (2)
  • NECV agreement threshold tau = 0.5
    Hand-chosen threshold for flagging a sample as noisy when fewer than half of 14 ensemble votes match the label. No sensitivity analysis is provided, so the baseline's 71.19% hit rate depends on this choice.
  • Loss-clip loss threshold tau = 0.05
    Hand-set loss cutoff for filtering high-loss samples in the Loss-Clip baseline; taken from the method's design rather than tuned on this dataset.
assumptions (3)
  • domain assumption Expert manual corrections on the test/validation sets constitute ground truth labels.
    The cleaning process used two experts and deleted unresolved ambiguities, but the paper provides no measure of inter-expert agreement or verification against an independent reference.
  • domain assumption The noise distribution in the cleaned test/validation sets is representative of the training set.
    The 5.58% noise rate is estimated from 14,320 samples, while the 57,146-sample training set is not cleaned; all label-correction metrics are computed only on the cleaned subsets.
  • domain assumption The class-dependent noise transition matrix is a fair model to test for this dataset.
    The paper assumes that testing the class-dependent transition model with the true matrix is a meaningful upper bound, even though the method is restricted to 51 classes and uses the cleaned validation labels, which introduces information leakage.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Noisy Ostracods: A Fine-Grained, Imbalanced Real-World Dataset for Benchmarking Robust Machine Learning and Label Correction Methods." pith.science (2026). https://pith.science/paper/VLISBUVA

@misc{pith2026241202313,
  author       = {Pith},
  title        = {Pith review of: Noisy Ostracods: A Fine-Grained, Imbalanced Real-World Dataset for Benchmarking Robust Machine Learning and Label Correction Methods},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VLISBUVA}},
  note         = {Machine review of arXiv:2412.02313}
}
abstract

We present the Noisy Ostracods, a noisy dataset for genus and species classification of crustacean ostracods with specialists' annotations. Over the 71466 specimens collected, 5.58% of them are estimated to be noisy (possibly problematic) at genus level. The dataset is created to addressing a real-world challenge: creating a clean fine-grained taxonomy dataset. The Noisy Ostracods dataset has diverse noises from multiple sources. Firstly, the noise is open-set, including new classes discovered during curation that were not part of the original annotation. The dataset has pseudo-classes, where annotators misclassified samples that should belong to an existing class into a new pseudo-class. The Noisy Ostracods dataset is highly imbalanced with a imbalance factor $\rho$ = 22429. This presents a unique challenge for robust machine learning methods, as existing approaches have not been extensively evaluated on fine-grained classification tasks with such diverse real-world noise. Initial experiments using current robust learning techniques have not yielded significant performance improvements on the Noisy Ostracods dataset compared to cross-entropy training on the raw, noisy data. On the other hand, noise detection methods have underperformed in error hit rate compared to naive cross-validation ensembling for identifying problematic labels. These findings suggest that the fine-grained, imbalanced nature, and complex noise characteristics of the dataset present considerable challenges for existing noise-robust algorithms. By openly releasing the Noisy Ostracods dataset, our goal is to encourage further research into the development of noise-resilient machine learning methods capable of effectively handling diverse, real-world noise in fine-grained classification tasks. The dataset, along with its evaluation protocols, can be accessed at https://github.com/H-Jamieu/Noisy_ostracods.

Figures

Figures reproduced from arXiv: 2412.02313 by the authors.

Figure 1
Figure 1. A simplified data collection and annotation process for the Noisy Ostracods dataset. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Selected noisy images from the Noisy Ostracods Dataset. a. Photograph error; b. Photograph [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The genus frequency distribution for the dataset. [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The collection process. where T ∗ eg represents the probability of a true label g being observed as label e. Based on this formulation, we implemented the loss function following [27]: L(θ, T) = − 1 N XN n=1 log ˜p(˜y = ˜yn|xn, θ, T) = − 1 N XN n=1 log X i Ty˜nip(y ∗ =…
Figure 5
Figure 5. Figure 5: The detailed feature errors. The wording is slightly different from the article. [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: A qualitative showcase of the juvenile to adult stages of [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: The transition matrix at genus level for validation and test set combined. [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: The transition matrix at genus level for test set [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: The transition matrix at genus level for validation and test set [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

71 extracted references · 65 canonical work pages

  1. [1]

    Holmes and Allan R

    Jonathan A. Holmes and Allan R. Chivas.The Ostracoda: Applications in Quaternary Research. Washington, DC: American Geophysical Union, 2002, p. 1

    work page 2002

  2. [2]

    Hong Kong shallow marine benthic ecosystem history : conservation pale- oecology approach based on microfossil ostracods

    Yuanyuan Hong. “Hong Kong shallow marine benthic ecosystem history : conservation pale- oecology approach based on microfossil ostracods”. 2016. URL: http://hdl.handle.net/ 10722/240648

    work page 2016

  3. [3]

    Baseline for ostracod-based northwest- ern Pacific and Indo-Pacific shallow-marine paleoenvironmental reconstructions: ecological modeling of species distributions

    Y . Hong, M. Yasuhara, H. Iwatani, and B. Mamo. “Baseline for ostracod-based northwest- ern Pacific and Indo-Pacific shallow-marine paleoenvironmental reconstructions: ecological modeling of species distributions”. In: Biogeosciences 16.2 (2019), pp. 585–604

    work page 2019

  4. [4]

    Ultralytics YOLO

    Glenn Jocher, Ayush Chaurasia, and Jing Qiu. Ultralytics YOLO. Version 8.0.0. 2023. URL: https://github.com/ultralytics/ultralytics

    work page 2023

  5. [5]

    Deep Residual Learning for Image Recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. “Deep Residual Learning for Image Recognition”. In: IEEE Conference on Computer Vision and Pattern Recognition. 2016

    work page 2016

  6. [6]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale”. In: International Conference on Learning Representations. 2021

    work page 2021

  7. [7]

    Learning from Noisy Labels with Deep Neural Networks: A Survey

    Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. “Learning from Noisy Labels with Deep Neural Networks: A Survey”. In: IEEE Transactions on Neural Networks and Learning Systems (2022)

    work page 2022

  8. [8]

    Tsang, James T

    Bo Han, Quanming Yao, Tongliang Liu, Gang Niu, Ivor W. Tsang, James T. Kwok, and Masashi Sugiyama. A Survey of Label-noise Representation Learning: Past, Present and Future. 2021. arXiv: 2011.04406 [cs.LG]

    arXiv 2021

Show all 71 references
  1. [9]

    MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels

    Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. “MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels”. In:Interna- tional Conference on Machine Learning. 2018

    2018

  2. [10]

    Co-teaching: Robust training of deep neural networks with extremely noisy labels

    Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. “Co-teaching: Robust training of deep neural networks with extremely noisy labels”. In: Advances in Neural Information Processing Systems. 2018

    2018

  3. [11]

    How does Disagreement Help Generalization against Label Corruption?

    Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor Tsang, and Masashi Sugiyama. “How does Disagreement Help Generalization against Label Corruption?” In: International Conference on Machine Learning. 2019

    2019

  4. [12]

    Detecting Corrupted Labels Without Training a Model to Predict

    Zhaowei Zhu, Zihao Dong, and Yang Liu. “Detecting Corrupted Labels Without Training a Model to Predict”. In: International Conference on Machine Learning. 2022

    2022

  5. [13]

    Confident Learning: Estimating Uncer- tainty in Dataset Labels

    Curtis G. Northcutt, Lu Jiang, and Isaac L. Chuang. “Confident Learning: Estimating Uncer- tainty in Dataset Labels”. In: Journal of Artificial Intelligence Research (JAIR) 70 (2021), pp. 1373–1411

    2021

  6. [14]

    C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie.The Caltech-UCSD Birds-200- 2011 Dataset. Tech. rep. CNS-TR-2011-001. California Institute of Technology, 2011

    2011

  7. [15]

    Automated Flower Classification over a Large Number of Classes

    Maria-Elena Nilsback and Andrew Zisserman. “Automated Flower Classification over a Large Number of Classes”. In: Indian Conference on Computer Vision, Graphics and Image Processing. Dec. 2008

    2008

  8. [16]

    The INaturalist Species Classification and De- tection Dataset

    Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. “The INaturalist Species Classification and De- tection Dataset”. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CV...

    2018

  9. [17]

    A Step Towards Worldwide Biodiversity Assessment: The BIOSCAN- 1M Insect Dataset

    Z. Gharaee, Z. Gong, N. Pellegrino, I. Zarubiieva, J. B. Haurum, S. C. Lowe, J. T. A. McKeown, C. Y . Ho, J. McLeod, Y . C. Wei, J. Agda, S. Ratnasingham, D. Steinke, A. X. Chang, G. W. Taylor, and P. Fieguth. “A Step Towards Worldwide Biodiversity Assessment: The BIOSCAN- 1M ...

    2023

  10. [18]

    PlantNET (The NSW Plant Information Network System)

    Royal Botanic Gardens and Domain Trust. PlantNET (The NSW Plant Information Network System). https : / / plantnet . rbgsyd . nsw . gov . au. Web sites hosted by or linked to PlantNET should be cited as instructed on these web pages. 2024

    2024

  11. [19]

    Endless Forams: >34,000 modern planktonic foraminiferal images for taxo- nomic training and automated species recognition using convolutional neural networks

    AY Hsiang, A Brombacher, MC Rillo, MJ Mleneck-Vautravers, S Conn, S Lordsmith, A Jentzen, MJ Henehan, B Metcalfe, I Fenton, BS Wade, L Fox, J Meilland, CV Davis, U Baranowski, J Groeneveld, KM Edgar, A Movellan, T Aze, H Dowsett, G Miller, N Rios, and PM Hull. “Endless Forams:...

    2019 doi

  12. [20]

    Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations

    Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, and Yang Liu. “Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations”. In:Interna- tional Conference on Learning Representations. 2022

    2022

  13. [21]

    Human Uncertainty Makes Classification More Robust

    J. Peterson, R. Battleday, T. Griffiths, and O. Russakovsky. “Human Uncertainty Makes Classification More Robust”. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 2019

    2019

  14. [22]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech. rep. Canadian Institute for Advanced Research (CIFAR), 2009.URL: https://www.cs.toronto.edu/~kriz/cifar.html

    2009

  15. [23]

    Learning from massive noisy labeled data for image classification

    Tong Xiao, Tian Xia, Yi Yang, Chang Huang, and Xiaogang Wang. “Learning from massive noisy labeled data for image classification”. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015

    2015

  16. [24]

    Beyond Synthetic Noise: Deep Learning on Controlled Noisy Labels

    Lu Jiang, Di Huang, Mason Liu, and Weilong Yang. “Beyond Synthetic Noise: Deep Learning on Controlled Noisy Labels”. In: International Conference on Machine Learning. 2020

    2020

  17. [25]

    VHX-7000 Digital Microscope

    Keyence Corporation. VHX-7000 Digital Microscope . Available online: https : / / www . keyence.com/products/microscope/digital-microscope/vhx-7000/ . 2022. URL: https : / / www . keyence . com / products / microscope / digital - microscope / vhx - 7000/

    2022

  18. [26]

    ImageNet Large Scale Visual Recognition Challenge

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei- Fei. “ImageNet Large Scale Visual Recognition Challenge”. In: ImageNet Large Scale Visual Recognitio...

    2015

  19. [27]

    Training Convolutional Networks with Noisy Labels

    Sainbayar Sukhbaatar, Joan Bruna, Manohar Paluri, Lubomir Bourdev, and Rob Fergus. Training Convolutional Networks with Noisy Labels. 2015. arXiv: 1406.2080 [cs.CV]

    2015 arXiv

  20. [28]

    Explaining and Harnessing Adversarial Examples

    Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. “Explaining and Harnessing Adversarial Examples”. In: International Conference on Learning Representations. 2015

    2015

  21. [29]

    Identifying Misla- beled Data using the Area Under the Margin Ranking

    Geoff Pleiss, Tianyi Zhang, Ethan R. Elenberg, and Kilian Q. Weinberger. “Identifying Misla- beled Data using the Area Under the Margin Ranking”. In: Advances in Neural Information Processing Systems. 2020

    2020

  22. [30]

    Interactive Label Cleaning with Example-based Explanations

    Stefano Teso, Andrea Bontempelli, Fausto Giunchiglia, and Andrea Passerini. “Interactive Label Cleaning with Example-based Explanations”. In: Advances in Neural Information Processing Systems. Ed. by A. Beygelzimer, Y . Dauphin, P. Liang, and J. Wortman Vaughan. 2021

    2021

  23. [31]

    TorchVision maintainers and contributors.TorchVision: PyTorch’s Computer Vision library

  24. [32]

    mixup: Beyond Empirical Risk Minimization

    Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. “mixup: Beyond Empirical Risk Minimization”. In: International Conference on Learning Representations. 2018

    2018

  25. [33]

    CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features

    Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. “CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features”. In: International Conference on Computer Vision. 2019. 12

    2019

  26. [34]

    DivideMix: Learning with Noisy Labels as Semi-supervised Learning

    Junnan Li, Richard Socher, and Steven C.H. Hoi. “DivideMix: Learning with Noisy Labels as Semi-supervised Learning”. In: International Conference on Learning Representations. 2020

    2020

  27. [35]

    Sharpness-aware Minimization for Efficiently Improving Generalization

    Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. “Sharpness-aware Minimization for Efficiently Improving Generalization”. In: International Conference on Learning Representations. 2021

    2021

  28. [36]

    Why is SAM Robust to Label Noise?

    Christina Baek, J Zico Kolter, and Aditi Raghunathan. “Why is SAM Robust to Label Noise?” In: International Conference on Learning Representations. 2024

    2024

  29. [37]

    Estimating Noisy Class Posterior with Part-level Labels for Noisy Label Learning

    Rui Zhao, Bin Shi, Jianfei Ruan, Tianze Pan, and Bo Dong. “Estimating Noisy Class Posterior with Part-level Labels for Noisy Label Learning”. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2024

    2024

  30. [38]

    Subclass-Dominant Label Noise: A Counterexample for the Success of Early Stopping

    Yingbin Bai, Zhongyi Han, Erkun Yang, Jun Yu, Bo Han, Dadong Wang, and Tongliang Liu. “Subclass-Dominant Label Noise: A Counterexample for the Success of Early Stopping”. In: Thirty-seventh Conference on Neural Information Processing Systems. 2023

    2023

  31. [39]

    Early Stopping Against Label Noise Without Validation Data

    Suqin Yuan, Lei Feng, and Tongliang Liu. “Early Stopping Against Label Noise Without Validation Data”. In: The Twelfth International Conference on Learning Representations . 2024

    2024

  32. [40]

    Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels

    Pengfei Chen, Ben Ben Liao, Guangyong Chen, and Shengyu Zhang. “Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels”. In: International Conference on Machine Learning. 2019

    2019

  33. [41]

    Classification with Noisy Labels by Importance Reweight- ing

    Tongliang Liu and Dacheng Tao. “Classification with Noisy Labels by Importance Reweight- ing”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence38.3 (2015), pp. 447– 461

    2015

  34. [42]

    Sample Selection with Uncertainty of Losses for Learning with Noisy Labels

    Xiaobo Xia, Tongliang Liu, Bo Han, Mingming Gong, Jun Yu, Gang Niu, and Masashi Sugiyama. “Sample Selection with Uncertainty of Losses for Learning with Noisy Labels”. In: arXiv preprint arXiv:2106.00445 (2021)

    2021 arXiv

  35. [43]

    Extended t: Learning with Mixed Closed-Set and Open-Set Noisy Labels

    Xiaobo Xia, Bo Han, Nannan Wang, Jiankang Deng, Jiatong Li, Yinian Mao, and Tongliang Liu. “Extended t: Learning with Mixed Closed-Set and Open-Set Noisy Labels”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence(2022)

    2022

  36. [44]

    Using Pre-Training Can Improve Model Robustness and Uncertainty

    Dan Hendrycks, Kimin Lee, and Mantas Mazeika. “Using Pre-Training Can Improve Model Robustness and Uncertainty”. In: International Conference on Machine Learning. 2019

    2019

  37. [45]

    Ensemble Methods for Noise Elimination in Classifi- cation Problems

    Sofie Verbaeten and Anneleen Assche. “Ensemble Methods for Noise Elimination in Classifi- cation Problems”. In: Fourth International Workshop on Multiple Classifier Systems. V ol. 2709. 2003, pp. 317–325. ISBN : 978-3-540-40369-2. DOI: 10.1007/3-540-44938-8_32

    2003 doi

  38. [46]

    Data preprocessing in data min- ing

    Salvador García, Julián Luengo, and Francisco Herrera. Data preprocessing in data min- ing. eng. Intelligent Systems Reference Library, volume 72. Cham: Springer, 2015. ISBN : 9783319102474

    2015

  39. [47]

    A ConvNet for the 2020s

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. “A ConvNet for the 2020s”. In: IEEE Conference on Computer Vision and Pattern Recognition. 2022

    2022

  40. [48]

    De- signing Network Design Spaces

    Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Dollár. “De- signing Network Design Spaces”. In: IEEE Conference on Computer Vision and Pattern Recognition. 2020

    2020

  41. [49]

    EfficientNetV2: Smaller Models and Faster Training

    Mingxing Tan and Quoc Le. “EfficientNetV2: Smaller Models and Faster Training”. In: International Conference on Machine Learning. 2021, pp. 10096–10106

    2021

  42. [50]

    MnasNet: Platform-Aware Neural Architecture Search for Mobile

    Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc V . Le. “MnasNet: Platform-Aware Neural Architecture Search for Mobile”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019

    2019

  43. [51]

    MaxViT: Multi-Axis Vision Transformer

    Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, and Yinxiao Li. “MaxViT: Multi-Axis Vision Transformer”. In: The European Conference on Computer Vision. 2022

    2022

  44. [52]

    Swin Transformer V2: Scaling Up Capacity and Resolution

    Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, Furu Wei, and Baining Guo. “Swin Transformer V2: Scaling Up Capacity and Resolution”. In: IEEE Conference on Computer Vision and Pattern Recognition. 2022. 13

    2022

  45. [53]

    Searching for MobileNetV3

    Andrew Howard, Mark Sandler, Bo Chen, Weijun Wang, Liang-Chieh Chen, Mingxing Tan, Grace Chu, Vijay Vasudevan, Yukun Zhu, Ruoming Pang, Hartwig Adam, and Quoc Le. “Searching for MobileNetV3”. In: Inetrnational Conference on Computer Vision. 2019

    2019

  46. [54]

    Aggregated Residual Transformations for Deep Neural Networks

    Saining Xie, Ross Girshick, Piotr Dollar, Zhuowen Tu, and Kaiming He. “Aggregated Residual Transformations for Deep Neural Networks”. In: IEEE Conference on Computer Vision and Pattern Recognition. 2017

    2017

  47. [55]

    ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design

    Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. “ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design”. In: European Conference on Computer Vision. 2018

    2018

  48. [56]

    AQuA: A Benchmarking Tool for Label Quality Assessment

    Mononito Goswami, Vedant Sanil, Arjun Choudhry, Arvind Srinivasan, Chalisa Udompa- nyawit, and Artur Dubrawski. “AQuA: A Benchmarking Tool for Label Quality Assessment”. In: Advances in Neural Information Processing Systems . Ed. by A. Oh, T. Naumann, A. Globerson, K. Saenko, ...

    2023

  49. [57]

    Maya Srikanth, Jeremy Irvin, Brian Wesley Hill, Felipe Godoy, Ishan Sabane, and Andrew Y . Ng. An Empirical Study of Automated Mislabel Detection in Real World Vision Datasets. 2023. arXiv: 2312.02200 [cs.CV]

    2023 arXiv

  50. [58]

    Clusterability as an Alternative to Anchor Points When Learning with Noisy Labels

    Zhaowei Zhu, Yiwen Song, and Yang Liu. “Clusterability as an Alternative to Anchor Points When Learning with Noisy Labels”. In: International Conference on Machine Learning. 2021

    2021

  51. [59]

    Learning Transferable Visual Models From Natural Language Supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision. 2021. arXiv: 2103.00...

    2021 arXiv

  52. [60]

    Masked Autoencoders Are Scalable Vision Learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. “Masked Autoencoders Are Scalable Vision Learners”. In: IEEE Conference on Computer Vision and Pattern Recognition. 2021

    2021

  53. [61]

    Don’t Decay the Learning Rate, Increase the Batch Size

    Samuel L. Smith, Pieter-Jan Kindermans, and Quoc V . Le. “Don’t Decay the Learning Rate, Increase the Batch Size”. In: International Conference on Learning Representations. 2018

    2018

  54. [62]

    Unmasking and Improving Data Credibility: A Study with Datasets for Training Harmless Language Models

    Zhaowei Zhu, Jialu Wang, Hao Cheng, and Yang Liu. “Unmasking and Improving Data Credibility: A Study with Datasets for Training Harmless Language Models”. In:International Conference on Learning Representations. 2024

    2024

  55. [63]

    When does Privileged Information Explain Away Label Noise?

    Guillermo Ortiz-Jimenez, Mark Collier, Anant Nawalgaria, Alexander D’Amour, Jesse Berent, Rodolphe Jenatton, and Effrosyni Kokiopoulou. “When does Privileged Information Explain Away Label Noise?” In: Proceedings of the International Conference on Machine Learning. 2023

    2023

  56. [64]

    Dynamic Loss for Robust Learning

    Shenwang Jiang, Jianan Li, Jizhou Zhang, Ying Wang, and Tingfa Xu. “Dynamic Loss for Robust Learning”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence45.12 (2023), pp. 14420–14434

    2023

  57. [65]

    Class-Difficulty Based Methods for Long-Tailed Visual Recognition

    Saptarshi Sinha, Hiroki Ohashi, and Katsuyuki Nakamura. “Class-Difficulty Based Methods for Long-Tailed Visual Recognition”. In:International Journal of Computer Vision 130 (Aug. 2022). DOI: 10.1007/s11263-022-01643-3 . Checklist

    2022 doi

  58. [67]

    For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contribu- tions and scope? [Yes] (b) Did you describe the limitations of your work? [Yes] (c) Did you discuss any potential negative societal impacts of your work? [N...

  59. [68]

    (a) Did you state the full set of assumptions of all theoretical results? [N/A] (b) Did you include complete proofs of all theoretical results? [N/A]

    If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results? [N/A] (b) Did you include complete proofs of all theoretical results? [N/A]

  60. [69]

    for benchmarks)

    If you ran experiments (e.g. for benchmarks)... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] 14 (b) Did you specify all the training details (e.g., data splits, h...

  61. [70]

    If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators? [Yes] (b) Did you mention the license of the assets? [Yes] (c) Did you include any new assets either in the supplem...

  62. [71]

    bad image

    If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (...

    2015

  63. [2016]

    URL: https://github.com/pytorch/vision

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.