REVIEW 6 cited by
Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Self-training algorithms, which train a model to fit pseudolabels predicted by another previously-learned model, have been very successful for learning with unlabeled data using neural networks. However, the current theoretical understanding of self-training only applies to linear models. This work provides a unified theoretical analysis of self-training with deep networks for semi-supervised learning, unsupervised domain adaptation, and unsupervised learning. At the core of our analysis is a simple but realistic "expansion" assumption, which states that a low probability subset of the data must expand to a neighborhood with large probability relative to the subset. We also assume that neighborhoods of examples in different classes have minimal overlap. We prove that under these assumptions, the minimizers of population objectives based on self-training and input-consistency regularization will achieve high accuracy with respect to ground-truth labels. By using off-the-shelf generalization bounds, we immediately convert this result to sample complexity guarantees for neural nets that are polynomial in the margin and Lipschitzness. Our results help explain the empirical successes of recently proposed self-training algorithms which use input consistency regularization.
Forward citations
Cited by 6 Pith papers
-
Direct Diffusion Score Preference Optimization via Stepwise Contrastive Policy-Pair Supervision
Diffusion image models can be aligned without human labels by supervising every denoising step with score targets from original versus degraded prompts.
-
Beyond Entropy: Region Confidence Proxy for Wild Test-Time Adaptation
ReCAP models each test sample's local feature neighborhood as a Gaussian and optimizes closed-form bounds on regional entropy and instability, improving wild test-time adaptation accuracy.
-
Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing
For binary classification with noisy labels, the paper derives the Bayes-optimal function for combining a model's current predictions with the given labels during retraining, and shows a fitted version improves linear...
-
Relating Misfit to Gain in Weak-to-Strong Generalization Beyond the Squared Loss
For convex and approximately convex model classes, the loss gain in weak-to-strong learning is at least the KL misfit between strong and weak models, plus an error term that vanishes as k grows.
-
Negative Metric Learning for Graphs
NML-GCL learns a negative metric network that down-weights false negatives during contrastive training, improving node classification and clustering on six standard graph benchmarks.
-
Lungmix: A Mixup-Based Strategy for Generalization in Respiratory Sound Classification
Lungmix, a mixup variant with loudness masks and semantic OR label interpolation, improves some cross-dataset respiratory sound classification scores by up to 3.55 points.
Discussion (0). Continue with ORCID to comment.