Pith. sign in

REVIEW 5 major objections 6 minor 51 references

Semi-supervised learning can reach the same unsupervised loss with widely different confidence thresholds, so precise threshold selection is unnecessary.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A gradient-updated pseudo-label threshold with a regularizer (MTSSL) improves several SSL baselines, but the paper's proof that different thresholds yield the same loss is not established.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection The empirical meta-thresholding method is real and mostly works; the advertised theory of an inherent threshold trade-off is not proven. the 5 major comments →

arxiv 2607.16363 v1 pith:AELBOHKH submitted 2026-07-17 stat.ML cs.LG

MTSSL: Meta-Thresholding Semi-Supervised Learning

Shuyang Liu , Ziang Zeng , Ruiqiu Zheng , Jiazheng Wang , Zechen Liu , Wenxi Li , Zhou Yu This is my paper
classification stat.ML cs.LG
keywords semi-supervised learningpseudo-labelingadaptive thresholdingconfidence thresholdconfirmation biasmeta-learningconsistency regularization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish a unified statistical explanation for why SSL algorithms with very different confidence thresholds τ perform similarly. It decomposes the empirical unsupervised loss into parts from correct and wrong pseudo-labels and argues that τ trades off the two error terms: lowering τ admits more correct labels but also more wrong ones, while raising τ does the opposite. Hence the estimation error between the empirical loss and the true population loss is not monotone in τ; it can be flat over a wide range. Based on this, the paper proposes MTSSL, which treats τ as a differentiable parameter and updates it by gradient descent on a smoothed loss, and reports top accuracy on 7 of 8 benchmark settings. The broader claim is that SSL is robust to τ, so future SSL designs need not hand-tune thresholds.

Core claim

The central claim is that the empirical unsupervised loss bL_u(θ,τ) splits into a correct-pseudo-label term and a wrong-pseudo-label term, and the threshold τ controls the sizes of both. Since the correct term under-estimates the true loss L*_u(θ) as τ grows (fewer selected labels) while the wrong term adds a confirmation-bias error that shrinks as τ grows, the total estimation error |L*_u(θ) − bL_u(θ,τ)| is balanced by these two counteracting terms. The paper proves, under a Gaussian model for the data and additive Gaussian augmentations, that the counts of correct and wrong pseudo-labels both decrease with τ, and that the wrong term scales as O(nW/n). This yields the paper's central infere

What carries the argument

The central object is the decomposition bL_u(θ,τ) = bLnC_u(θ,τ) + bLnW_u(θ,τ) — the empirical unsupervised loss split into contributions from pseudo-labels that agree with the true labels and those that do not. The argument then uses Proposition 1 (expressions for nC and nW as Gaussian tail probabilities whose counts decrease monotonically with τ), Theorem 2 (a sub-Gaussian concentration bound showing the correct part converges to P_C·L*_u(θ) as n grows), Theorem 3 (under additive Gaussian augmentations, the wrong part is of order nW/n), and Proposition 4 (the gradient of the sigmoid-smoothed loss with respect to τ is always negative, explaining the collapse solution). The mechanism that car

Load-bearing premise

The proof that the wrong-pseudo-label error is proportional to nW/n assumes augmentations are additive Gaussian noise; real SSL augmentations (crops, flips, color jitter) are not Gaussian, so the theoretical basis for the trade-off may not hold for the actual algorithms tested.

What would settle it

Freeze a trained SSL model and its data, compute the true unsupervised loss L*_u(θ) using ground-truth labels, then measure |L*_u(θ) − bL_u(θ,τ)| for a fine grid of τ using the actual augmentation pipeline (crops, flips, color jitter) instead of Gaussian noise. If the error curve shows a sharp single minimum rather than a wide flat valley, the claim that many τ values give the same loss is falsified for that setup.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Choosing a precise τ is unnecessary; multiple τ schedules can yield equivalent training effect.
  • The threshold can be treated as a learnable parameter and updated by back-propagation, leading to a plug-and-play module for existing SSL algorithms.
  • MTSSL improves both fixed-threshold baselines (like FixMatch) and adaptive-threshold baselines (like FreeMatch) across several benchmark datasets.
  • The robustness to τ means hyperparameter tuning for thresholds can be greatly simplified in future SSL designs.
  • Different adaptive thresholding policies that appear to use wildly different τ values can all be successful for the same underlying reason.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the trade-off is general, then comparisons between SSL algorithms that differ only in their threshold schedules may not reflect differences in the underlying method; benchmark results should be interpreted accordingly.
  • The theory suggests a testable prediction: for a fixed model, the unsupervised loss as a function of τ should be nearly flat over a wide interval; plotting this curve on real data could confirm or refute the plateau.
  • The Gaussian augmentation assumption is a narrow idealization; extending the O(nW/n) argument to random crops, flips, and color jitter would put the theory on the same footing as the experiments.
  • If precise τ truly is unnecessary, then simpler schedules (e.g., a fixed moderate τ or a single early update) may capture most of the benefit of adaptive policies like MTSSL, reducing computational cost.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper proposes a statistical explanation for the role of the confidence threshold τ in pseudo-label semi-supervised learning. It argues that the empirical unsupervised loss decomposes into contributions from correct and wrong pseudo-labels, and that τ balances the corresponding error terms, so that 'SSL can reach the same loss with varying τ' and precise threshold selection is unnecessary. Based on this claim, the authors introduce MTSSL, which treats τ as a differentiable parameter via a sigmoid-smoothed indicator and a regularizer, and report experiments on SVHN, CIFAR-10, CIFAR-100, and STL-10 showing that MTSSL+FreeMatch is best or second-best on most of the evaluated settings. The paper also reports an empirical observation that accuracy curves can nearly overlap for quite different τ trajectories.

Significance. If the theoretical claim were established, it would be a useful unification of adaptive thresholding policies in SSL and would justify relaxing τ tuning. The paper has clear strengths: a simple plug-and-play thresholding update, evaluation across standard benchmarks, ablation studies, and an interesting empirical observation of overlapping accuracy curves. The experimental protocol is conventional. However, the central theoretical derivation is not valid as written: Theorem 2 is miscentered, Theorem 3 relies on an unrealistic Gaussian augmentation model, and the 'inherent trade-off' conclusion does not follow from the stated monotonicity facts. The empirical results are suggestive but several headline gains are within the reported standard deviations, so the 'best adaptive thresholding policy' claim needs stronger statistical support.

major comments (5)
  1. [§III.A, Theorem 2 / Eq. (7)] The concentration inequality is centered at the wrong quantity. bLnC_u is an average over the selected-correct set A, so its expectation is P_C · E[H(θ,X) | p'_θ(X)≥τ, Y^θ=Y], not P_C · L*_u(θ) = P_C · E[H(θ,X)]. No argument shows that selected-correct pseudo-labels are representative of the population; in trained models high-confidence correct predictions typically have different consistency loss than the average. Thus Eq. (7) does not imply bLnC_u → P_C L*_u, and the later step in §IV.C, |bLnC_u−L*_u| → (1−P_C)L*_u, repeats the same unproved assumption. This breaks the decomposition (4) and the claimed trade-off.
  2. [§III.A, Theorem 3] The theorem assumes additive Gaussian augmentations, a(x)=x+δ1 and A(x)=x+δ2 with δ1,δ2∼N(0,σ3). This is not the augmentation used by FixMatch, FreeMatch, or any other evaluated SSL algorithm (random crops, flips, color jitter). The theorem therefore does not establish the O(nW/n) behavior for the methods tested. Moreover, O(nW/n) is a big-O bound, not an equality; even under the Gaussian model it does not show that the wrong-label loss term equals or scales exactly with nW/n, so it cannot by itself prove that different τ give equal total loss.
  3. [§III.A, 'inherent trade-off'] The conclusion that different τ can yield the same loss does not follow from the monotonicity of n_C and n_W. From Eqs. (5)-(6), both counts decrease with τ, but the estimation error in (4) also depends on the conditional averages of H over the selected correct and wrong sets, and these averages change with τ. Monotone counts are compatible with a strictly monotone error or with no crossing. The paper supplies no continuity or range argument establishing existence of τ1≠τ2 with equal |L*_u − bLu|. The empirical figures are illustrative, not a proof.
  4. [§III.B/§IV.A, Eq. (9)] MTSSL optimizes the same unsupervised loss that the theory analyzes, and the actual optimized objective is a regularized, smoothed variant that is not identical to the analyzed loss (1). The components h(τ), β, λ, and g(h(τ)) are needed to avoid the collapse solution, but they are not part of the theoretical model. Therefore, even if Theorems 2 and 3 were correct, they would not directly justify the MTSSL update rule. The paper should either analyze the actual objective or explicitly state that the theory is only heuristic motivation.
  5. [§IV.B, Table I] The headline claim 'best on 7 of 8 settings' is not statistically supported. The improvements of MTSSL+FreeMatch over FreeMatch are +0.07 (CIFAR-10 40), +0.09 (CIFAR-10 250), +1.12 (CIFAR-100 400), +1.03 (CIFAR-100 2500), +0.00 (SVHN 40), +0.03 (SVHN 250), +0.95 (STL-10 40), +0.10 (STL-10 1000). Several of these are smaller than or comparable to the reported standard deviations, and no paired significance tests are provided. The claim that MTSSL is the best adaptive thresholding policy needs additional statistical evidence.
minor comments (6)
  1. [§IV.A] The hyperparameter section states h(τ)=1/(1+e^{-β(p'_θ(x)-τ)}), which contradicts the definition h(τ)=1/(1+e^{-τ}) used above Eq. (9). This makes the exact algorithm ambiguous; the definition should be made consistent.
  2. [Eqs. (1), (8), (9)] The arguments of H are inconsistent: Eq. (1) uses H(pθ(A), pθ(a)), Eq. (8) uses H(pθ(a), pθ(A)), and Eq. (9) uses H(pθ(A), pθ(a)). Since H is an asymmetric cross-entropy, this is not a harmless notational change.
  3. [Eq. (9)] There is a typo: '·+' before λ·g(h(τ)) should be '+'. Please correct.
  4. [§IV.G.3] The text says 'when we have θ=100, it achieves the best result'; the symbol should be β, not θ.
  5. [References [46], [47]] References [46] and [47] are both titled 'On the Sample Complexity of Next-Token Prediction' but list different authors/years. One of them appears to be mis-cited; please verify.
  6. [§IV.C] The verification section says 5×10^4 labeled data are used to approximate the true loss L*_u. Since L*_u is an expectation over unlabeled data, it would be clearer to state how the labeled subset is used to emulate the required distribution.

Circularity Check

2 steps flagged

Partial circularity: the trade-off's wrong-label leg (Theorem 3: bLnW_u ∝ O(nW/n)) is immediate from the definition of bLnW_u as a sum over the nW wrong pseudo-labels, and its correct-label leg (Theorem 2/Eq. 7) is obtained by centering the concentration at P_C·L*_u, which presupposes that the selected-correct set is representative of the population — so the 'inherent trade-off' is substantially b

specific steps
  1. self definitional [Section III.A, 'The Inherent Trade-off of τ' — definition of bLnW_u before Eq. (4), and Theorem 3]
    "bLnW_u(θ,τ) = 1/n Σ_i I{x_i∈B}·H(p(A(x_i)),p(a(x_i))) ... we suppose the augmentation a(x), A(x) for any input x by adding the Gaussian-random noise to x ... Then when n→+∞, with probability 1, we have: bLnW_u(θ,τ) ∝ O(nW/n)."

    bLnW_u is defined as (1/n) times the sum of the bounded cross-entropy H over exactly the nW wrong pseudo-labels in B. Hence bLnW_u ≤ (nW/n)·max H holds deterministically for every n and every augmentation; the claimed rate O(nW/n) is a restatement of the definition, and the Gaussian-noise premise in Theorem 3 is irrelevant to it. The trade-off's wrong-label leg ('less wrong predictions results in a smaller bLnW_u') is therefore assumed by construction, not derived.

  2. other [Section III.A, Theorem 2 / Eq. (7); reused in Section IV.C 'Verification of Theoretical Results']
    "P(|bLnC_u(θ,τ) − P_C · L*_u(θ)| > t) ≤ exp(−n²t²/(2γn_C)) ... lim_{n→+∞} |bLnC_u(θ,τ) − L*_u(θ)| = lim_{n→+∞} |(1−P_C)·L*_u(θ)| monotonically increase with τ"

    bLnC_u averages H over the selected-correct set A, so its expectation is P_C·E[H | p'_θ(X)≥τ, Y^θ=Y], not the stated center P_C·L*_u = P_C·E[H]. Inequality (7), and the §IV.C prediction |bLnC_u − L*_u| → (1−P_C)L*_u, can hold only if E[H|A] = E[H], i.e., the filtered correct set is representative of the population. That representativeness is exactly the paper's own intuitive premise ('more correct predictions brings a closer estimation to L*_u') which Theorem 2 is supposed to prove; the trade-off's correct-leg is thus imported through the choice of centering rather than derived.

full rationale

The central theoretical claim — that different τ can reach the same unsupervised loss because τ trades off the correct versus wrong pseudo-label error terms — rests on three pieces. Proposition 1 genuinely computes nC(τ) and nW(τ) for a Gaussian-logistic model. But the two error-term rates are not independent derivations: Theorem 3's bLnW_u ∝ O(nW/n) is immediate from the definition of bLnW_u as a (1/n)-sum over the nW wrong samples (H bounded), with the Gaussian augmentation assumption superfluous; Theorem 2's concentration is centered at P_C·L*_u whereas the mean of bLnC_u is P_C·E[H | p'≥τ, Y^θ=Y], so the inequality and the derived formula |bLnC_u − L*_u| → (1−P_C)L*_u in §IV.C hold only under an unstated representativeness assumption identical to the paper's stated intuition. Hence the shape of the 'inherent trade-off' reduces largely to the decomposition itself plus that assumption; what is actually derived (Prop 1) only shows that both counts are monotone in τ. However, the empirical content is independent: overlapping accuracy curves for very different τ trajectories (Figs. 3–5), the confusion-matrix analyses, and the eight benchmark settings are external observations, not outputs of the trade-off derivation, and the method is validated against external baselines rather than against the paper's own prior work. There are no load-bearing self-citations or imported uniqueness theorems. Because one stated 'prediction' (Theorem 3) reduces by construction, and the §IV.C verification of the correct-leg reduces to an assumption encoded in the theorem's centering, but the empirical claims stand on their own, the score is 6 (partial circularity of the theoretical derivation), not 8 or 10.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

The central claims rest on a toy distributional model, an unrealistic augmentation assumption, unstated concentration assumptions, and a hand-designed regularizer. The method itself needs six tuned quantities (τ0, β, λ, K, η2, regularizer form), so the claim that precise τ is unnecessary is stronger than the evidence.

free parameters (6)
  • τ0 (initial threshold) = 0.6
    Selected by ablation (Sec. IV.G.1) to reduce early confirmation bias; not derived from theory.
  • β (sigmoid sharpness) = 100
    Tuned via ablation (Sec. IV.G.3); β=100 gives best CIFAR-10 40 accuracy, with worse results at β=1 and β=10000.
  • λ (regularizer weight) = 0.02
    Tuned via ablation (Sec. IV.G.2, Table II); other λ values change accuracy by about 0.5 pp.
  • K (update interval for τ) = 20
    Chosen hyperparameter; Sec. IV.F shows K and η2 can be traded off while keeping K/η2 fixed.
  • η2 (learning rate for τ) = 0.001
    Chosen hyperparameter for the Adam optimizer updating τ; Sec. IV.F shows similar performance when scaled with K.
  • Regularizer form g(h(τ)) = 1/sqrt(1-h(τ))
    One of two proposed forms; the L2 regularizer is also tested (Sec. IV.G.4, Table IV). The form and its coefficient are tuned, not derived.
axioms (6)
  • domain assumption Conditional Gaussian model X|Y and logistic model for pθ (Proposition 1).
    The counting formulas for nC and nW are derived under this toy model; real SSL data are not a two-class Gaussian mixture.
  • domain assumption Augmentations are additive Gaussian noise: a(x)=x+δ1, A(x)=x+δ2 with δ1,δ2∼N(0,σ3).
    Theorem 3 uses this to conclude bLnW_u = O(nW/n). Actual weak/strong augmentations are crops, flips, and color jitter, so the theorem does not cover the evaluated algorithms.
  • domain assumption Theorem 2's 'classical assumptions' [46], [47] hold for bLnC_u.
    The assumptions behind the sub-Gaussian concentration bound are not stated in the paper, and refs [46] and [47] are about next-token prediction sample complexity, not this SSL loss.
  • domain assumption Lower estimation error |L*_u − bLu| translates into better SSL accuracy.
    The paper analyzes estimation error of the unsupervised loss but never proves that this error controls test accuracy or training dynamics; this link is asserted throughout Sec. III.
  • ad hoc to paper Replacing the indicator function with a sigmoid does not change which algorithm is being evaluated.
    The smoothing is introduced for differentiability (Eq. 8) and is claimed to be 'fair comparison', but no bound shows that the smoothed loss behaves like the original thresholded loss.
  • ad hoc to paper The regularizer g prevents collapse without changing the essential behavior of τ.
    The regularizer is added because the raw gradient always drives τ upward (Proposition 4); its exact form and weight are chosen empirically, not derived from the theory.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of MTSSL: Meta-Thresholding Semi-Supervised Learning." pith.science (2026). https://pith.science/paper/AELBOHKH

@misc{pith2026260716363,
  author       = {Pith},
  title        = {Pith review of: MTSSL: Meta-Thresholding Semi-Supervised Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AELBOHKH}},
  note         = {Machine review of arXiv:2607.16363}
}
Share X Bluesky LinkedIn Reddit HN
abstract

A large body of Semi-supervised Learning~(SSL) algorithms encounter the threshold $\tau$ to select pseudo-labels. The value of $\tau$ across different SSL algorithms can vary depending on the learning perspective, yet they may achieve similar performance. It motivates us to establish a unified theoretical framework to explain the role of $\tau$ in SSL. We statistically explained that the unsupervised loss is affected independently by correct and incorrect pseudo-labels, while $\tau$ adjusts their numbers to balance the corresponding error term. This inherent trade-off indicates that SSL can reach the same loss with varying $\tau$, precise optimal values of $\tau$ during training may be unnecessary. With this, we treat $\tau$ as an updatable parameter and optimize it via differentiation; the new policy is named \textbf{Meta-Thresholding Semi-Supervised Learning (MTSSL)}. Extensive experiments demonstrate the superior performance of MTSSL. We observe that the accuracy curves of SSL algorithms can overlap completely even when the values of $\tau$ differ significantly, which supports our theoretical framework and indicates that the selection of $\tau$ can be relaxed in the future design of SSL algorithms.

Figures

Figures reproduced from arXiv: 2607.16363 by Jiazheng Wang, Ruiqiu Zheng, Shuyang Liu, Wenxi Li, Zechen Liu, Zhou Yu, Ziang Zeng.

Figure 1
Figure 1. Figure 1: With baseline models FixMatch and FreeMatch, the values of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Simulation on the CIFAR-10 and CIFAR-100 dataset; the observation consistent with the theoretical results. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: (a,b): The accuracy almost overlaps while the values of [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The accuracy of three different experiments almost overlaps while the values of [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: (a), (b), (c): The accuracy of MTSSL tends to have a better increasing in the early training stages with a lower threshold [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: (a) Without the smoothing mapping h(·), τ exceeds 1. (b) The sampling rate exceeds 0 when the collapse solution happens. (a) Accuracy on CIFAR-10 with 40 labels. (b) Sampling rate on STL-10 with 1000 labels. (c) Sampling rate on STL-10 with 1000 labels [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: We replace the iteration k and the learning rate η2 for updating τ , and it shows that they can reach a similar result. value of τ case-by-case is unnecessary; even a plain gradient descent—despite its potential sub-optimality—can effectively adjust this parameter, rendering case-by-case designs [14], [23], [26], [28] less critical. The parameter settings and detailed discussion can be seen in the Appendix… view at source ↗
Figure 8
Figure 8. Figure 8: Confusion matrix within a batch when τ0 = 0.3, 0.6, on 0.05, 2, 4, 8 × 105 iteration. The lower τ0 tends to bring more wrong pseudo-labels in the early stages, and it may harm the prediction in the latter iterations. In Fig.7, for the three experiments on CIFAR-10 40, CIFAR￾100 400, and STL-10 1000, we fine-tune the value of K, η2 while keep the value of K η2 the same. As an example, on CIFAR-10 40, we obs… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 1 linked inside Pith

  1. [1]

    Label propagation for deep semi-supervised learning,

    A. Iscen, G. Tolias, Y . Avrithis, and O. Chum, “Label propagation for deep semi-supervised learning,”IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5070–5079, 2019

    2019

  2. [2]

    Density-aware graph for deep semi-supervised visual recognition,

    S. Li, B. Liu, D. Chen, Q. Chu, L. Yuan, and N. Yu, “Density-aware graph for deep semi-supervised visual recognition,”IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13400–13409, 2020

    2020

  3. [3]

    LASSL: Label-guided self-training for semi-supervised learning,

    Z. Zhao, L. Zhou, L. Wang, Y . Shi, and Y . Gao, “LASSL: Label-guided self-training for semi-supervised learning,”AAAI Conference on Artificial Intelligence (AAAI), vol. 36, pp. 9208–9216, 2022

    2022

  4. [4]

    Big self-supervised models are strong semi-supervised learners,

    T. Chen, S. Kornblith, K. Swersky, M. Norouzi, and G. E. Hinton, “Big self-supervised models are strong semi-supervised learners,”Conference on Neural Information Processing Systems (NeurIPS), 2020

    2020

  5. [5]

    Mixmatch: A holistic approach to semi-supervised learning,

    D. Berthelot, N. Carlini, I. Goodfellow, N. Papernot, A. Oliver, and C. A. Raffel, “Mixmatch: A holistic approach to semi-supervised learning,” Conference on Neural Information Processing Systems (NeurIPS), vol. 32, 2019

    2019

  6. [6]

    Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring,

    D. Berthelot, N. Carlini, E. D. Cubuk, A. Kurakin, K. Sohn, H. Zhang, and C. Raffel, “Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring,”International Conference on Learning Representations (ICLR), 2019

    2019

  7. [7]

    Fixmatch: Simplifying semi- supervised learning with consistency and confidence,

    K. Sohn, D. Berthelot, N. Carlini, Z. Zhang, H. Zhang, C. A. Raffel, E. D. Cubuk, A. Kurakin, and C. L. Li, “Fixmatch: Simplifying semi- supervised learning with consistency and confidence,”Conference on Neural Information Processing Systems (NeurIPS), vol. 33, pp. 596–608, 2020. 12

    2020

  8. [8]

    Virtual adversarial training: a regularization method for supervised and semi-supervised learning,

    T. Miyato, S. I. Maeda, M. Koyama, and S. Ishii, “Virtual adversarial training: a regularization method for supervised and semi-supervised learning,”IEEE Transactions on Pattern Analysis and Machine Intelli- gence (TPAMI), vol. 41, no. 8, pp. 1979–1993, 2018

    1979

  9. [9]

    Mean teachers are better role mod- els: Weight-averaged consistency targets improve semi-supervised deep learning results,

    A. Tarvainen and H. Valpola, “Mean teachers are better role mod- els: Weight-averaged consistency targets improve semi-supervised deep learning results,”Conference on Neural Information Processing Systems (NeurIPS), vol. 30, 2017

    2017

  10. [10]

    Semi-supervised learning with ladder networks,

    A. Rasmus, M. Berglund, M. Honkala, H. Valpola, and T. Raiko, “Semi-supervised learning with ladder networks,”Conference on Neural Information Processing Systems (NeurIPS), vol. 28, 2015

    2015

  11. [11]

    Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks,

    D. H. Lee, “Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks,”International Conference on Machine Learning (ICML), vol. 3, no. 2, pp. 896, 2013

    2013

  12. [12]

    Low-shot learning with large-scale diffusion,

    M. Douze, A. Szlam, B. Hariharan, and H. J ´egou, “Low-shot learning with large-scale diffusion,”IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3349–3358, 2018

    2018

  13. [13]

    Sim- match: Semi-supervised learning with similarity matching,

    M. Zheng, S. You, L. Huang, F. Wang, C. Qian, and C. Xu, “Sim- match: Semi-supervised learning with similarity matching,”IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14471–14481, 2022

    2022

  14. [14]

    Freematch: Self-adaptive thresholding for semi-supervised learning,

    Y . Wang, H. Chen, Q. Heng, W. Hou, Y . Fan, Z. Wu, et al., “Freematch: Self-adaptive thresholding for semi-supervised learning,”International Conference on Learning Representations (ICLR), 2022

    2022

  15. [15]

    Class-imbalanced semi-supervised learning with adaptive thresholding,

    L. Z. Guo and Y . F. Li, “Class-imbalanced semi-supervised learning with adaptive thresholding,”International Conference on Machine Learning (ICML), pp. 8082–8094, 2022

    2022

  16. [16]

    Comatch: Semi-supervised learning with contrastive graph regularization,

    J. Li et al., “Comatch: Semi-supervised learning with contrastive graph regularization,”IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9475–9484, 2021

    2021

  17. [17]

    Density-sensitive semisu- pervised inference,

    M. Azizyan, A. Singh, and L. Wasserman, “Density-sensitive semisu- pervised inference,”Annals of Statistics (ANNALS), vol. 41, no. 2, pp. 751–771, 2013

    2013

  18. [18]

    Softmatch: Addressing the quantity-quality trade-off in semi-supervised learning,

    H. Chen, R. Tao, Y . Fan, Y . Wang, J. Wang, B. Schiele, and M. Savvides, “Softmatch: Addressing the quantity-quality trade-off in semi-supervised learning,”International Conference on Learning Representations (ICLR), vol. 8, 2023

    2023

  19. [19]

    Shrinking class space for enhanced certainty in semi-supervised learning,

    L. Yang, Z. Zhao, L. Qi, Y . Qiao, Y . Shi, and H. Zhao, “Shrinking class space for enhanced certainty in semi-supervised learning,”IEEE/CVF International Conference on Computer Vision (ICCV), pp. 16187–16196, 2023

    2023

  20. [20]

    Boosting Semi-Supervised Learning by bridging high and low-confidence prediction,

    K. B. Nguyen, “Boosting Semi-Supervised Learning by bridging high and low-confidence prediction,”IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1028–1038, 2023

    2023

  21. [21]

    Learning with local and global consistency,

    D. Zhou, O. Bousquet, T. Lal, J. Weston, and B. Sch ¨olkopf, “Learning with local and global consistency,”Conference on Neural Information Processing Systems (NeurIPS), vol. 16, 2003

    2003

  22. [22]

    RegMixMatch: Optimizing Mixup Utilization in Semi-Supervised Learning,

    H. Han, J. Yuan, and C. Wei, “RegMixMatch: Optimizing Mixup Utilization in Semi-Supervised Learning,”AAAI Conference on Artificial Intelligence (AAAI), vol. 39, no. 16, pp. 17032–17040, 2025

    2025

  23. [23]

    Dash: Semi-supervised learning with dynamic thresholding,

    Y . Xu, L. Shang, and J. Ye, “Dash: Semi-supervised learning with dynamic thresholding,”International Conference on Machine Learning (ICML), pp. 11525–11536, 2021

    2021

  24. [24]

    Dividemix: Learning with noisy labels as semi-supervised learning,

    J. Li, R. Socher, and S. Hoi, “Dividemix: Learning with noisy labels as semi-supervised learning,”International Conference on Learning Repre- sentations (ICLR), 2020

    2020

  25. [25]

    Pseudo-labeling and confirmation bias in deep semi-supervised learning,

    E. Arazo, D. Ortego, and P. Albert, “Pseudo-labeling and confirmation bias in deep semi-supervised learning,”Proceedings of the International Joint Conference on Neural Networks (IJCNN), pp. 1–8, 2020

    2020

  26. [26]

    Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling,

    B. Zhang, Y . Wang, and W. Hou, “Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling,”Conference on Neural Infor- mation Processing Systems (NeurIPS), vol. 34, pp. 18408–18419, 2021

    2021

  27. [27]

    MarginMatch: Improving semi-supervised learning with pseudo-margins,

    T. Sosea and C. Caragea, “MarginMatch: Improving semi-supervised learning with pseudo-margins,”IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15773–15782, 2023

    2023

  28. [28]

    Adamatch: A unified approach to semi-supervised learning and domain adaptation,

    D. Berthelot, R. Roelofs, and K. Sohn, “Adamatch: A unified approach to semi-supervised learning and domain adaptation,”International Con- ference on Learning Representations (ICLR), vol. 7, 2022

    2022

  29. [29]

    Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data,

    Y . Chen, X. Tan, and B. Zhao, “Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data,”IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7548–7557, 2023

    2023

  30. [30]

    InstanT: Semi-supervised Learning with Instance-dependent Thresholds,

    M. Li, R. Wu, and H. Liu, “InstanT: Semi-supervised Learning with Instance-dependent Thresholds,”Conference on Neural Information Pro- cessing Systems (NeurIPS), vol. 36, pp. 2922–2938, 2023

    2023

  31. [31]

    Unsupervised data augmentation for consistency training,

    Q. Xie, Z. Dai, E. Hovy, et al., “Unsupervised data augmentation for consistency training,”Neurips, 2020

    2020

  32. [32]

    Rethinking the Value of Labels for Improving Class- Imbalanced Learning,

    Y . Yang and Z. Xu, “Rethinking the Value of Labels for Improving Class- Imbalanced Learning,”Conference on Neural Information Processing Systems (NeurIPS), vol. 33, pp. 19290–19301, 2020

    2020

  33. [33]

    Adaptive subgradient methods for online learning and stochastic optimization,

    J. Duchi, E. Hazan, and Y . Singer, “Adaptive subgradient methods for online learning and stochastic optimization,”Journal of Machine Learning Research (JMLR), vol. 12, no. 7, 2011

    2011

  34. [34]

    Mlr-snet: Transferable lr schedules for heterogeneous tasks,

    J. Shu, Y . Zhu, and Q. Zhao, “Mlr-snet: Transferable lr schedules for heterogeneous tasks,”IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 45, no. 3, pp. 3505–3521, 2022

    2022

  35. [35]

    Semi-supervised learning with meta-gradient,

    T. Xiao, X. Y . Zhang, and H. Jia, “Semi-supervised learning with meta-gradient,”International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 73–81, 2021

    2021

  36. [36]

    Semi-supervised meta-learning for multi-source heterogeneity in time-series data,

    L. Zhang and B. J. Mortazavi, “Semi-supervised meta-learning for multi-source heterogeneity in time-series data,”Machine Learning for Healthcare Conference, pp. 923–941, 2023

    2023

  37. [37]

    Convex optimization,

    S. P. Boyd and L. Vandenberghe, “Convex optimization,”Cambridge university press, 2004

    2004

  38. [38]

    Introductory lectures on convex optimization: A basic course,

    Y . Nesterov, “Introductory lectures on convex optimization: A basic course,”Springer Science & Business Media, 2013

    2013

  39. [39]

    Explaining and harnessing adversarial examples,

    I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,”International Conference on Learning Represen- tations (ICLR), 2014

    2014

  40. [40]

    On data-augmentation and consistency- based semi-supervised learning,

    A. Ghosh and A. H. Thiery, “On data-augmentation and consistency- based semi-supervised learning,”arXiv:2101.06967, 2021

    Pith/arXiv arXiv 2021

  41. [41]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,”University of Toronto, 2009

    2009

  42. [42]

    Read- ing digits in natural images with unsupervised feature learning,

    Y . Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y . Ng, “Read- ing digits in natural images with unsupervised feature learning,”NeurIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011

    2011

  43. [43]

    An analysis of single-layer networks in unsupervised feature learning,

    A. Coates, A. Ng, and H. Lee, “An analysis of single-layer networks in unsupervised feature learning,”Proceedings of the fourteenth interna- tional conference on artificial intelligence and statistics, pp. 215–223, 2011

    2011

  44. [44]

    SCMT: Self-Correction Mean Teacher for Semi-supervised Object Detection,

    F. Xiong, J. Tian, and Z. Hao, “SCMT: Self-Correction Mean Teacher for Semi-supervised Object Detection,”International Joint Conference on Artificial Intelligence (IJCAI), vol. 30, pp. 1488–1494, 2022

    2022

  45. [45]

    Convergence Rates of Non- Convex Stochastic Gradient Descent Under a Generic Łojasiewicz Con- dition and Local Smoothness,

    K. Scaman, C. Malherbe, and L. Santos, “Convergence Rates of Non- Convex Stochastic Gradient Descent Under a Generic Łojasiewicz Con- dition and Local Smoothness,”International Conference on Machine Learning (ICML), 2022

    2022

  46. [46]

    On the Sample Complexity of Next- Token Prediction,

    O. Y ¨uksel and N. Flammarion, “On the Sample Complexity of Next- Token Prediction,”International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 694–702, 2025

    2025

  47. [47]

    On the Sample Complexity of Next-Token Prediction,

    S. Kanai, M. Yamada, and S. Yamaguchi, “On the Sample Complexity of Next-Token Prediction,”Proceedings of the International Joint Con- ference on Neural Networks (IJCNN), 2021

    2021

  48. [48]

    Revisiting Consistency Regulariza- tion for Semi-Supervised Learning,

    Y . Fan, A. Kukleva, and B. Schiele, “Revisiting Consistency Regulariza- tion for Semi-Supervised Learning,”International Journal of Computer Vision (IJCV), vol. 131, pp. 626–643, 2023

    2023

  49. [49]

    Semi-Supervised Domain Generalization with Stochastic StyleMatch,

    K. Zhou, C. Loy, and Z. Liu, “Semi-Supervised Domain Generalization with Stochastic StyleMatch,”International Journal of Computer Vision (IJCV), vol. 131, pp. 2377–2387, 2023

    2023

  50. [50]

    Exploration and Exploitation of Unlabeled Data for Open-Set Semi-supervised Learning,

    G. Zhao, G. Li, and Y . Qin, “Exploration and Exploitation of Unlabeled Data for Open-Set Semi-supervised Learning,”International Journal of Computer Vision (IJCV), vol. 132, pp. 5888–5904, 2024

    2024

  51. [51]

    Relation-Guided Versatile Regulariza- tion for Federated Semi-Supervised Learning,

    Q. Yang, Z. Chen, and Z. Peng, “Relation-Guided Versatile Regulariza- tion for Federated Semi-Supervised Learning,”International Journal of Computer Vision (IJCV), vol. 133, pp. 3312–3326, 2025

    2025

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.