Pith. sign in

REVIEW 3 major objections 5 minor 36 references

Dual Student: Breaking the Limits of the Teacher in Semi-supervised Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that in consistency-based semi-supervised learning the EMA teacher is a bottleneck, and that training two independent students with a stabilization constraint on stable samples outperforms it.

desk verdict Novel two-student training recipe with a real overclaim in the abstract and an unisolated central mechanism; worth refereeing, needs a targeted ablation. read the letter →

arxiv 1909.01804 v1 pith:SEPYNT3X submitted 2019-09-03 cs.LG cs.CVstat.ML

Zhanghan Ke , Daoye Wang , Qiong Yan , Jimmy Ren , Rynson W.H. Lau This is my paper · ORCID
classification cs.LGcs.CVstat.ML
keywords semi-supervisedlearningconsistencyregularizationEMAteacherstablesamplestabilizationconstraintDualStudentconfirmationbiasdomainadaptation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that consistency-based semi-supervised learning has been held back by its own teacher. In standard Teacher-Student methods, the teacher is an exponential moving average of the student's weights, so the two roles converge to nearly identical models and the teacher ends up reinforcing the student's existing biases. To escape that loop, the paper trains two students from different initial states and lets them exchange knowledge only through samples one of them can predict with high confidence and stability under augmentation. This stabilization constraint is meant to transmit reliable knowledge without collapsing both models into one. On CIFAR-10 with 1k labels, the method lowers error from 16.84% to 12.39%, and it also improves domain adaptation from USPS to MNIST.

What carries the argument

The load-bearing idea is the stable sample (Definition 4.1): a point is stable for a model if its predicted label does not change under small perturbations in the neighborhood and its maximum softmax output is above a threshold $\xi$. In practice, the paper approximates the first condition by comparing predictions on the original and one augmented view, and the second by a max-probability threshold. The stabilization constraint (Eq. 6) then uses the mean squared error between the two students' predictions, gated by these stability flags: if only one student is stable, that student's prediction supervises the other; if both are stable, the more stable one supervises; if neither is stable, no between-student constraint is applied. This mechanism is what keeps two fully independent students from collapsing into each other while still letting them exchange knowledge.

What would settle it

Measure, during training, the accuracy against held-out labels of the stable samples each student uses to supervise the other; if that accuracy is not clearly above the model's overall accuracy, the stability gate is not selecting reliable knowledge and the claimed advantage over a plain consistency constraint would not follow.

Watch

Extended reading notes

Core claim

The central claim is that replacing the EMA teacher with a second, independently initialized student removes a coupling bottleneck. Because the EMA teacher is an average of the student itself, its targets contain no information beyond the student's current beliefs and, as training converges, they force those beliefs back onto the student, making misclassifications hard to reverse. Dual Student instead maintains two students that are only loosely coupled; knowledge flows between them through samples that one model finds stable: the sample's predicted label is unchanged under augmentation and its maximum softmax probability exceeds a threshold $\xi$. The stabilization constraint sends a unidirectional consistency pull from the more stable model to the other for each sample, or no pull at all when neither is stable. The paper reports consistent error reductions across CIFAR-10, CIFAR-100, SVHN, ImageNet with 10% labels, and a USPS-to-MNIST domain adaptation task.

Load-bearing premise

The method assumes that a sample whose prediction is stable under augmentation and whose maximum softmax probability is above a threshold is also correct enough to teach the other student.

Editorial extensions

If this is right

  • A second student trained from a different initialization can replace the EMA teacher without collapsing, as long as knowledge exchange is gated by stability.
  • The stabilization constraint can be combined with an EMA-based inner consistency constraint with no loss, so the gains come from the between-student exchange rather than from the perturbation type.
  • Adding more students helps until saturation: four students beat two, while eight add little, so the marginal benefit of extra independent models is limited.
  • Making one student much larger improves the weaker student roughly in proportion to the stronger student's capacity, turning the structure into a mutual-distillation scheme without a fixed teacher.
  • The method transfers to domain adaptation: on USPS-to-MNIST it reaches 91.50% accuracy, well above the EMA-teacher baseline and close to supervision from the target domain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same stability gate could be reused to decide when a model's own pseudo-labels are safe in self-training, since the paper shows the gate selects samples with above-average accuracy and is fairly insensitive to the threshold over a range.
  • Editorial inference: the diagnosed bottleneck is broader than EMA: any target generator that converges to the student's own beliefs should show the same confirmation bias, which makes decorrelation between the target and the student the general design principle.
  • Editorial inference: replacing the hard stability flag with a continuous stability weight, such as the inverse of the prediction distance in Eq. 4, is a direct testable extension that could smooth the knowledge exchange and further reduce sensitivity to the threshold.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Dual Student, a semi-supervised learning method that replaces the EMA teacher in Teacher-Student consistency methods with a second independently initialized student model. To prevent the two students from collapsing by exchanging wrong knowledge, the authors define a 'stable sample' via a confidence threshold and prediction invariance under one random augmentation, and introduce a stabilization constraint that transfers predictions only from the more stable student. Two variants (Multiple Student and Imbalanced Student) extend the idea. Experiments on CIFAR-10, CIFAR-100, SVHN, ImageNet, and USPS-to-MNIST domain adaptation report improvements over existing consistency-based baselines, and ablation experiments examine the confidence threshold and the stabilization constraint. The paper's central claim is that the tight coupling of an EMA teacher causes a performance bottleneck and that training two loosely coupled students with stable-sample gating breaks this bottleneck.

Significance. If the central claim holds, the work is significant: it challenges a widely used design choice in consistency-based SSL (the EMA teacher) and proposes a concrete alternative with a novel knowledge-exchange mechanism. The empirical evaluation is broad, spanning several standard SSL benchmarks and a domain-adaptation task, and the method is clearly described so that it could be reproduced. The stable-sample concept is interesting and potentially useful beyond this specific structure. However, the significance is moderated by two weaknesses that the paper does not fully address: the headline comparison in the abstract conflates different architectures, and no experiment isolates the EMA-teacher bottleneck from the other changes introduced by the method. The paper also does not provide code or a controlled analysis of whether stable samples are indeed correct pseudo-labels rather than merely confident predictions.

major comments (3)
  1. [Abstract; Table 5] The abstract claims that the method 'reduces the error rate of the 13-layer CNN from 16.84% to 12.39% on CIFAR-10 with 1k labels,' but 16.84% is the MT+FSWA baseline from Table 1 while 12.39% is the Imbalanced Student with the 11.6M-parameter ResNet from Table 5, not the 13-layer CNN. Standard Dual Student on the 13-layer CNN obtains 15.74% (and 14.17% with 600 epochs). The same issue affects the CIFAR-100 statement (34.10% to 31.56%), where 31.56% is again the Imbalanced Student with the large ResNet. The abstract therefore overstates the gain attributable to the proposed Dual Student structure and should be corrected to compare like with like.
  2. [Section 3; Section 5.3; Fig. 7] The paper's load-bearing claim is that the EMA teacher creates a performance bottleneck and that replacing it with an independent student is the cause of the improvement. However, no experiment isolates this factor. Fig. 7 (left) compares Dual Student plus Mean Teacher against Mean Teacher, but the two conditions differ in both the target source and the loss (stability gating is present only in the former). Fig. 7 (right) compares the stabilization constraint Lsta with the consistency constraint Lcs, but both conditions use two independent students, so this only tests the gating mechanism. Since no ablation swaps an EMA target for an independent student while holding every other component (including the stabilization constraint) identical, the reported gains cannot be uniquely attributed to 'breaking the limits of the EMA teacher.' A controlled experiment is needed to support the central claim.
  3. [Section 4.1; Eq. (3); Fig. 6] The stabilization constraint relies on the assumption that stable samples, defined by max-softmax confidence above xi and identical predicted labels under one random augmentation, are reliable enough to serve as pseudo-labels for the other student. The paper validates this only in Fig. 6, which shows that stable samples have higher test accuracy than all samples. This is expected for confident predictions and does not establish that the equivalence between 'stable' and 'correct' holds, nor that the exchanged predictions avoid confirmation bias. Since the claim that Lsta outperforms Lcs is explained by this assumed reliability, the paper should provide additional evidence, such as a precision analysis of exchanged pseudo-labels or a comparison against a confidence-threshold-only baseline without the stability condition.
minor comments (5)
  1. [Section 5.3; Fig. 7] The statement that the combination of Dual Student with Mean Teacher is 'insensitive to the type of consistency constraint inside each model' is not quantified; the accuracy curves appear close, but the paper should report final numbers for this comparison.
  2. [Eq. (6)] In the stabilization constraint, the boolean R_i_x is used both as a condition and as a multiplicative factor; the notation should be clarified so that the reader can see that the 'otherwise' branch applies only when at least one student considers x stable.
  3. [Fig. 3] The caption says 'They have the same initialization, while f2 is different from them,' which is ambiguous about which models share the initialization; please rephrase to state explicitly that f1 and fs share one initialization and f2 has a different one.
  4. [Appendix A] The proof concludes with 'i.e., {s't}y∈N converges to S,' where the subscript y appears to be a typo for t; please correct it.
  5. [Section 6] The conclusion mentions 'performance degradation on increasing number of labels,' which is misleading; the experiments show that the improvement over baselines becomes smaller, not that performance degrades. Please reword.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Dual Student is evaluated on held-out benchmarks, and the stable-sample reliability check is empirical rather than a fitted prediction.

full rationale

I find no circular chain. The paper's central assertion is that an EMA teacher is tightly coupled and thus creates a bottleneck; this is supported by Eq. 2 and Appendix A's elementary convergence proposition plus Figs. 2-3, none of which assumes the target result. The Dual Student loss (Eqs. 3-7) is a proposed training objective, not a fitted prediction; its components are defined independently of the reported error rates. The stable-sample concept is validated empirically by Fig. 6, which compares test accuracy on stable samples versus all samples, an external check rather than a construction that forces the claimed improvement. Reported hyperparameters such as xi and lambda_2 are varied in Table 6 and the ablation in Fig. 7, showing sensitivity rather than relabeling fitted quantities as predictions. The skeptical concern that the EMA-bottleneck hypothesis is not isolated by ablations is a question of experimental attribution, not circularity; it does not identify any equation that reduces to its own inputs. No self-citation is load-bearing.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central result is entirely empirical. The method introduces no new physical entities. It relies on hand-tuned hyperparameters (the confidence threshold xi and loss coefficients) and on two unproven modeling assumptions: a single noisy augmentation approximates local smoothness, and stability under that approximation indicates reliable knowledge. The appendix proves a standard convergence fact about EMAs, but only under the assumption that the student weights converge.

free parameters (4)
  • Stable-sample confidence threshold xi = 0.8 (CIFAR-10), 0.4 (CIFAR-100), 0.8 (SVHN), 0.01 (ImageNet), 0.6 (USPS-to-MNIST)
    Gates whether a sample counts as stable (Eq. 3); Table 6 shows it moves error by roughly one point.
  • Stabilization constraint coefficient lambda_2 = 100 (CIFAR), 10 (SVHN), 1000 (ImageNet), 1.0 (domain adaptation)
    Balances L_sta in Eq. 7; chosen by hand per dataset.
  • Consistency constraint coefficient lambda_1 = 10
    Balances L_con in Eq. 7; not reported as tuned.
  • Ramp-up duration for lambda_1 and lambda_2 = first 5 epochs
    Schedule for the stabilization constraint; chosen by hand.
assumptions (4)
  • domain assumption Smoothness assumption: nearby points in a high-density region should have similar outputs
    Invoked in Section 3 and inherited from SSL literature; the consistency and stabilization constraints both enforce this.
  • domain assumption Student weights converge during training, so the EMA teacher converges to the student (Appendix A)
    Proposition 1 assumes the sequence of student weights converges; deep networks trained with SGD may not reach a fixed point, so the coupling claim is conditional on convergence.
  • ad hoc to paper A single noisy augmentation and the max-probability threshold approximate the two conditions of Definition 4.1
    Section 4.2 states 'we only utilize two close samples to approximate the conditions of the stable sample'; the approximation is not justified and is central to the stabilization constraint.
  • domain assumption Stable-sample gating prevents the two students from collapsing into each other
    Section 4 claims naive consistency causes collapse and the proposed constraint avoids it, but no proof is given; the method's behavior rests on this empirical property.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dual Student: Breaking the Limits of the Teacher in Semi-supervised Learning." pith.science (2026). https://pith.science/paper/SEPYNT3X

@misc{pith2026190901804,
  author       = {Pith},
  title        = {Pith review of: Dual Student: Breaking the Limits of the Teacher in Semi-supervised Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SEPYNT3X}},
  note         = {Machine review of arXiv:1909.01804}
}
read the original abstract

Recently, consistency-based methods have achieved state-of-the-art results in semi-supervised learning (SSL). These methods always involve two roles, an explicit or implicit teacher model and a student model, and penalize predictions under different perturbations by a consistency constraint. However, the weights of these two roles are tightly coupled since the teacher is essentially an exponential moving average (EMA) of the student. In this work, we show that the coupled EMA teacher causes a performance bottleneck. To address this problem, we introduce Dual Student, which replaces the teacher with another student. We also define a novel concept, stable sample, following which a stabilization constraint is designed for our structure to be trainable. Further, we discuss two variants of our method, which produce even higher performance. Extensive experiments show that our method improves the classification performance significantly on several main SSL benchmarks. Specifically, it reduces the error rate of the 13-layer CNN from 16.84% to 12.39% on CIFAR-10 with 1k labels and from 34.10% to 31.56% on CIFAR-100 with 10k labels. In addition, our method also achieves a clear improvement in domain adaptation.

Figures

Figures reproduced from arXiv: 1909.01804 by the authors.

Figure 1
Figure 1. Teacher-Student versus Dual Student. The teacher (T) in Teacher-Student is an EMA of the student (S), impos￾ing a consistency constraint on the student. Their weights are tightly coupled. In contrast, a bidirectional stabilization constraint is applied between the two students (S and S’) in Dual Student. Their weights are loosely coupled. methods. In this type of methods, two roles are commonly created, either expli… view at source ↗
Figure 2
Figure 2. Left: Sema contains two models with similar weights, while the weights of the two models in Ssplit keep a certain distance. Right: The predictions of the two models in Ssplit keep a larger distance than those of Sema. Epoch class1 class2 0 12 24 36 48 54 f 1 f 2 f s [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Our method can alleviate the confirmation bias. f1 and f2 are the independent students from our Dual Stu￾dent, while fs is the student guided by the Mean Teacher. For a misclassified sample (belonging to class1), f1 can cor￾rect it quickly with the knowledge from f2. However, fs is unable to correct its prediction due to the wrong guidance from the EMA teacher. mistakes and to enforce the student to follow, making t… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Dual Student structure overview. We train two stud [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Illustration of the conditions for a stable sample. Consider three pairs of adjacent data points: (1) x1 and x¯1 do not satisfy the 1st condition, (2) x2 and x¯2 do not satisfy the 2nd condition, and (3) x3 and x¯3 satisfy both conditions. A smaller E i x means that x …
Figure 6
Figure 6. Figure 6: Test accuracy of each category on the stable sam￾ples and on all samples of CIFAR-10. The performance gap indicates that the stable samples represent relatively more reliable knowledge of a model. The average ratio of the stable samples on the test set is about 85% w.r…
Figure 7
Figure 7. Figure 7: Test accuracy on CIFAR-10 with 1k labels. Left: [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Domain difference between USPS and MNIST. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Test curves of domain adaptation from USPS to [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 34 canonical work pages

  1. [1]

    There are many consistent explana- tions of unlabeled data: Why you should average

    Ben Athiwaratkun, Marc Finzi, Pavel Izmailov, and An- drew Gordon Wilson. There are many consistent explana- tions of unlabeled data: Why you should average. In Proc ICLR, 2019. 2, 6, 9

    work page 2019

  2. [2]

    Learn - ing with pseudo-ensembles

    Philip Bachman, Ouais Alsharif, and Doina Precup. Learn - ing with pseudo-ensembles. In Proc. NIPS. 2014. 2

    work page 2014

  3. [3]

    Combining labeled and un- labeled data with co-training

    Avrim Blum and Tom Mitchell. Combining labeled and un- labeled data with co-training. In Proc. Annual Conference on Computational Learning Theory , 1998. 2

    work page 1998

  4. [4]

    Semi-Supervised Learning

    Olivier Chapelle, Bernhard Schlkopf, and Alexander Zie n. Semi-Supervised Learning. The MIT Press, 2006. 3

    work page 2006

  5. [5]

    Tri-net for semi-supervised deep learning

    Dong-Dong Chen, Wei Wang, Wei Gao, and Zhi-Hua Zhou. Tri-net for semi-supervised deep learning. In Proc. IJCAI,

    work page

  6. [6]

    Self-ensembling for domain adaptation

    Geoffrey French, Michal Mackiewicz, and Mark Fisher. Self-ensembling for domain adaptation. In Proc. ICLR

    work page

  7. [7]

    Explaining and harnessing adversarial examples

    Ian Goodfellow, Jonathon Shlens, and Christian Szegedy . Explaining and harnessing adversarial examples. In Proc. ICLR, 2015. 1, 2

    work page 2015

  8. [8]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Proc. CVPR,

    work page

Show all 36 references
  1. [9]

    Distilli ng the knowledge in a neural network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilli ng the knowledge in a neural network. In NIPS W orkshop on Deep Learning and Unsupervised Feature Learning, 2014. 5

    2014

  2. [10]

    Efficien t knowledge distillation from an ensemble of teachers

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Efficien t knowledge distillation from an ensemble of teachers. In An- nual Conference of the International Speech Communication Association, 2017. 5

    2017

  3. [11]

    Squeeze-and-excitation n et- works

    Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation n et- works. arXiv:1709.01507, 2017. 10

    2017 arXiv

  4. [12]

    Batch renormalization: Towards reducin g minibatch dependence in batch-normalized models

    Sergey Ioffe. Batch renormalization: Towards reducin g minibatch dependence in batch-normalized models. In Proc. NIPS. 2017. 6

    2017

  5. [13]

    Batch normalizati on: Accelerating deep network training by reducing internal co - variate shift

    Sergey Ioffe and Christian Szegedy. Batch normalizati on: Accelerating deep network training by reducing internal co - variate shift. In Proc. ICML, 2015. 6

    2015

  6. [14]

    V etrov, and Andrew Gordon Wilson

    Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry P . V etrov, and Andrew Gordon Wilson. Averaging weights leads to wider optima and better generalization. In Proc. UAI. 2018. 2

    2018

  7. [15]

    Cifa r-10 (canadian institute for advanced research)

    Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifa r-10 (canadian institute for advanced research). 6

  8. [16]

    Cifa r- 100 (canadian institute for advanced research)

    Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifa r- 100 (canadian institute for advanced research). 6

  9. [17]

    Temporal ensembling for sem i- supervised learning

    Samuli Laine and Timo Aila. Temporal ensembling for sem i- supervised learning. In Proc. ICLR. 2017. 1, 2, 6, 9

    2017

  10. [18]

    Webvision database: Visual learning and un- derstanding from web data

    Wen Li, Limin Wang, Wei Li, Eirikur Agustsson, and Luc V an Gool. Webvision database: Visual learning and un- derstanding from web data. 2017. 1

    2017

  11. [19]

    Smooth neighbors on teacher graphs for semi-supervised learning

    Y ucen Luo, Jun Zhu, Mengxi Li, Y ong Ren, and Bo Zhang. Smooth neighbors on teacher graphs for semi-supervised learning. In Proc. CVPR. 2018. 2, 6, 9

    2018

  12. [20]

    Auxiliary deep generative models

    Lars Maale, Casper Kaae Snderby, Sren Kaae Snderby, and Ole Winther. Auxiliary deep generative models. In Proc. ICML. 2016. 1

    2016

  13. [21]

    Virtual adversarial training: a regularization method for supervised and semi-supervised learning

    Takeru Miyato, Shin-ichi Maeda, Shin Ishii, and Masano ri Koyama. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE TPAMI, 2018. 1, 2, 6

    2018

  14. [22]

    Reading digits in natural images with unsupervised feature learning

    Y uval Netzer, Tao Wang, Adam Coates, Alessandro Bis- sacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. In NIPS W ork- shop on Deep Learning and Unsupervised Feature Learning, 2011. 7

    2011

  15. [23]

    Semi-supervised learning with genera tive adversarial networks

    Augustus Odena. Semi-supervised learning with genera tive adversarial networks. In Data Efficient Machine Learning workshop at ICML . 2016. 1

    2016

  16. [24]

    Realistic evaluation of semi-supervis ed learning algorithms

    Avital Oliver, Augustus Odena, Colin Raffel, Ekin Cubu k, and Ian Goodfellow. Realistic evaluation of semi-supervis ed learning algorithms. In Proc. NeurIPS. 2018. 6

    2018

  17. [25]

    Adversarial dropout for supervised and semi- supervised learning

    Sungrae Park, Jun-Keon Park, Su-Jin Shin, and Il-Chul Moon. Adversarial dropout for supervised and semi- supervised learning. In Proc. AAAI. 2018. 2

    2018

  18. [26]

    Polyak and Anatoli Juditsky

    Boris T. Polyak and Anatoli Juditsky. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization, 1992. 2

    1992

  19. [27]

    Siyuan Qiao, Wei Shen, Zhishuai Zhang, Bo Wang, and Alan L. Y uille. Deep co-training for semi-supervised image recognition. In Proc. ECCV. 2018. 2, 6, 9

    2018

  20. [28]

    Semi-supervised learning with ladder networks

    Antti Rasmus, Mathias Berglund, Mikko Honkala, Harri V alpola, and Tapani Raiko. Semi-supervised learning with ladder networks. In Proc. NIPS. 2015. 1, 2

    2015

  21. [29]

    Berg, and Li Fei-Fei

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sa n- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Chal- lenge. IJCV, 2015. 1, 7

    2015

  22. [30]

    Creating artificial neu ral networks that generalize

    Jocelyn Sietsma and Robert Dow. Creating artificial neu ral networks that generalize. Neural Networks, 1991. 2

    1991

  23. [31]

    Unsupervised and semi- supervised learning with categorical generative adversar ial networks

    Jost Tobias Springenberg. Unsupervised and semi- supervised learning with categorical generative adversar ial networks. arXiv:1511.06390, 2015. 1

    2015 arXiv

  24. [32]

    Dropout: A simple way to prevent neural networks from overfitting

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 2014. 1, 2

    2014

  25. [33]

    Mean teachers are be tter role models: Weight-averaged consistency targets improve semi-supervised deep learning results

    Antti Tarvainen and Harri V alpola. Mean teachers are be tter role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Proc. NIPS. 2017. 1, 2, 3, 6, 7, 9

    2017

  26. [34]

    Girshick, Piotr Doll´ ar, Zhuowen T u, and Kaiming He

    Saining Xie, Ross B. Girshick, Piotr Doll´ ar, Zhuowen T u, and Kaiming He. Aggregated residual transformations for deep neural networks. In Proc. CVPR. 2017. 7, 9

    2017

  27. [35]

    When semi-supervised learning meets en- semble learning

    Zhi-Hua Zhou. When semi-supervised learning meets en- semble learning. In Frontiers of Electrical and Electronic Engineering in China, 2011. 2

    2011

  28. [36]

    Semi-supervised learning literature sur vey

    Xiaojin Zhu. Semi-supervised learning literature sur vey. TR 1530, University of Wisconsin, Madison , 2006. 1

    2006

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.