DivideMix: Learning with Noisy Labels as Semi-supervised Learning
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:2PLC2XMQrecord.jsonopen to challenge →
read the original abstract
Deep neural networks are known to be annotation-hungry. Numerous efforts have been devoted to reducing the annotation cost when learning with deep networks. Two prominent directions include learning with noisy labels and semi-supervised learning by exploiting unlabeled data. In this work, we propose DivideMix, a novel framework for learning with noisy labels by leveraging semi-supervised learning techniques. In particular, DivideMix models the per-sample loss distribution with a mixture model to dynamically divide the training data into a labeled set with clean samples and an unlabeled set with noisy samples, and trains the model on both the labeled and unlabeled data in a semi-supervised manner. To avoid confirmation bias, we simultaneously train two diverged networks where each network uses the dataset division from the other network. During the semi-supervised training phase, we improve the MixMatch strategy by performing label co-refinement and label co-guessing on labeled and unlabeled samples, respectively. Experiments on multiple benchmark datasets demonstrate substantial improvements over state-of-the-art methods. Code is available at https://github.com/LiJunnan1992/DivideMix .
This paper has not been read by Pith yet.
Forward citations
Cited by 26 Pith papers
-
Weak-to-Strong Generalization via Direct On-Policy Distillation
Transferring the log-ratio of a small model's pre-RL and post-RL checkpoints provides a dense implicit reward that improves stronger student models at a fraction of the cost of direct RL.
-
Boxes2Pixels: Learning Defect Segmentation from Noisy SAM Masks
Boxes2Pixels distills noisy SAM pseudo-masks into a compact DINOv2-based student with auxiliary localization and one-sided self-correction, delivering +6.97 anomaly mIoU and +9.71 binary IoU gains over baselines on wi...
-
Conformal Margin Risk Minimization: An Envelope Framework for Robust Learning under Label Noise
CMRM adds a conformal quantile regularization on prediction margins to any loss, improving noisy-label classification accuracy up to 3.39% across methods and benchmarks while preserving performance at zero noise.
-
Robust Trajectory Distillation: Hybrid Reweighting Meets Teacher-Inspired Targets
Introduces SGR and TIAT for robust dataset distillation that suppresses noise while preserving knowledge under noisy supervision.
-
Beyond the Golden Teacher: Enhancing Graph Learning through LLM-GNN Co-teaching
Bidirectional LLM-GNN co-teaching with round-based pseudo-label preference optimization outperforms golden-teacher baselines on few-shot TAG benchmarks by 3-8% absolute gains.
-
Organizational Control Layer: Governance Infrastructure at the Execution Boundary of LLM Agent Systems
OCL is a governance layer for LLM agents that cuts unsafe executions from 88% to near-zero and raises valid success from 12% to 96% in adversarial buyer-seller negotiations across frontier LLMs.
-
ROGLE: Robust Global-Local Alignment with Automated Region Supervision for Text-Based Person Search
ROGLE automates region-level supervision via Region-to-Sentence Matching and introduces the P-VLG benchmark to improve fine-grained alignment in text-based person search over CLIP-based models.
-
GAMR: Geometric-Aware Manifold Regularization with Virtual Outlier Synthesis for Learning with Noisy Labels
GAMR introduces geometric-aware manifold regularization via virtual outlier synthesis to enhance intra-class compactness and inter-class separation, improving robustness to noisy labels beyond passive sample filtering.
-
HamBR: Active Decision Boundary Restoration Based on Hamiltonian Dynamics for Learning with Noisy Labels
HamBR uses Spherical HMC to probe ambiguous regions and synthesize virtual outliers with energy-based repulsion to restore decision boundaries degraded by noisy labels, achieving SOTA on CIFAR and real-world benchmarks.
-
Leveraging Data Symmetries to Select an Optimal Subset of Training Data under Label Noise
Exploiting data symmetries boosts k-NN to select near-optimal low-noise subsets from noisy datasets, approaching Bayes-optimal performance in high dimensions, with learned representations aiding partial symmetry knowledge.
-
Learning from Noisy Preferences: A Semi-Supervised Learning Approach to Direct Preference Optimization
Semi-DPO applies semi-supervised learning to noisy preference data in diffusion DPO by training first on consensus pairs then iteratively pseudo-labeling conflicts, yielding state-of-the-art alignment with complex hum...
-
Can LLMs Learn to Reason Robustly under Noisy Supervision?
Online Label Refinement lets LLMs learn robust reasoning from noisy supervision by correcting labels when majority answers show rising rollout success and stable history, delivering 3-4% gains on math and reasoning be...
-
Benchmarking Real-World Medical Image Classification with Noisy Labels: Challenges, Practice, and Outlook
LNMBench shows existing noisy-label methods degrade sharply under high and realistic noise in medical images due to class imbalance and domain shifts, and proposes a simple robustness fix.
-
Towards Continual Expansion of Data Coverage: Automatic Text-guided Edge-case Synthesis
Automated LLM-based prompt engineering for text-to-image edge-case synthesis improves object detection robustness on the FishEye8K benchmark over naive augmentation and manual prompts.
-
ROGLE: Robust Global-Local Alignment with Automated Region Supervision for Text-Based Person Search
ROGLE introduces automated pseudo region-sentence pairs via RSM and multi-granular learning to boost fine-grained alignment in text-based person search, plus the P-VLG benchmark with over 100k annotated regions.
-
Robust Fuzzy Multi-view Learning under View Conflict
R-FUML models network outputs as fuzzy memberships, applies entropy-based robust multi-view fusion, and uses memory-effect isolation plus penalties to mitigate view conflicts, outperforming 15 baselines on eight datasets.
-
Holistic Reliability Propagation: Decoupling Annotation and Prediction for Robust Noisy-Label
HRP decouples annotation reliability (alpha) and pseudo-label reliability (beta) via bilevel meta-learning and routes them to distinct objectives in reliability-aware Mixup and contrastive learning for improved noisy-...
-
Efficient Bilevel Optimization for Meta Label Correction in Noisy Label Learning
EBOMLC applies dynamic barrier gradient descent with one-step inner loop, mixture upper loss, and alignment-aware barrier to make meta label correction faster and more robust on noisy data, outperforming baselines on ...
-
SEI-SHIELD: Robust Specific Emitter Identification Under Label Noise Via Self-Supervised Filtering and Iterative Rescue
SEI-SHIELD achieves state-of-the-art accuracy on POWDER and ORACLE datasets under label noise by using MoCo pre-training for label-independent features followed by neighborhood consistency filtering and confidence-bas...
-
Model-agnostic information transfer and fusion for classification with label noise
A nonparametric model-agnostic framework purifies noisy labels with a small clean dataset for robust classification under label noise.
-
See Through the Noise: Improving Domain Generalization in Gaze Estimation
SeeTN builds a semantic embedding space with prototype transformation and affinity regularization to identify and correct noisy labels, yielding better cross-domain gaze estimation without hurting source accuracy.
-
Detecting and refurbishing ground truth errors during training of deep learning-based echocardiography segmentation models
Variance of Gradients effectively flags erroneous ground truth labels during echocardiography segmentation training, and refurbishing them with pseudo-labeling boosts performance especially under high error rates on t...
-
Analyzing the Effect of Noise in LLM Fine-tuning
Label noise hurts fine-tuning performance most while grammatical and typographical noise sometimes act as mild regularizers, with changes concentrated in task-specific layers.
-
LiLAW: Lightweight Learnable Adaptive Weighting to Learn Sample Difficulty & Improve Noisy Training
LiLAW learns to weight samples as easy, moderate or hard using three global scalars updated by one gradient step on a validation batch to improve noisy training performance.
-
Automatic Dataset Construction (ADC): Sample Collection, Data Curation, and Beyond
The ADC method automates the creation of large image classification datasets using LLMs and search engines, achieving 79% human agreement and reducing label noise on a 1 million image clothing dataset, while also rele...
-
Noise-Aware Framework for Correcting Corrupted Labels
CANOLA estimates label noise and performs cautious iterative soft-label refinement to correct corrupted training data, reporting 19-52% error reduction versus prior methods on six datasets.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.