Triple Expert Learning from Noisy Labels for Semi-Supervised Vision Foundation Model Adaptation

Daizong Liu; Hongyang He; Xuanyu Liu; Yundi Hong; Zheng Fang

REVIEW 4 major objections 5 minor 49 references

Triple Expert Learning from Noisy Labels for Semi-Supervised Vision Foundation Model Adaptation

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Pseudo-label confidence should choose which LoRA expert learns, not just how much each sample is weighted, and splitting the three confidence regions into separate experts makes semi-supervised vision foundation model adaptation more…

desk verdict Modest but genuine gain from confidence-routed triple-LoRA experts; the paper is honest and well-ablated, but the theory is largely assumed and the flagship gain rests on one dataset. read the letter →

arxiv 2608.09052 v1 pith:YE32G422 submitted 2026-08-10 cs.CV cs.AI

Xuanyu Liu , Zheng Fang , Hongyang He , Yundi Hong , Daizong Liu This is my paper · ORCID

classification cs.CVcs.AI

keywords semi-supervisedlearningvisionfoundationmodelsLoRApseudo-labelnoiseconfidence-basedroutingparameter-efficientfine-tuningexpertspecializationimageclassification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TriNoL argues that in semi-supervised adaptation of a frozen vision foundation model, pseudo-labels are not one uniform training signal: high-, medium-, and low-confidence samples play different roles and should be handled by different low-rank adapters. Instead of feeding all pseudo-label gradients into a single LoRA branch, the method routes high-confidence samples to a Positive Expert trained with hard pseudo-labels, medium-confidence samples to an Alignment Expert trained with soft KL alignment, and low-confidence samples to a Negative Expert whose objective suppresses commitment to the top pseudo-label. The paper shows this separation reduces contamination of the reliable update direction, and demonstrates empirically that a triple-expert rank-8 setup beats a rank-matched single LoRA with the same parameter count, so the gain comes from routing rather than capacity. A reader should care because noisy pseudo-labels are the main obstacle to cheap vision foundation model adaptation with few labels, and TriNoL addresses it by changing which adapter learns, not just how much the loss is weighted.

What carries the argument

The machinery is a confidence-routed triple-expert LoRA: a set of three independent low-rank updates $\Delta W_e = B_e A_e$ applied to frozen projection weights, with routing by the max softmax probability of the Positive Expert's weak-view prediction. Two thresholds $\tau_-$ and $\tau_+$ define the Positive, Alignment, and Negative regions, and each expert has a matching objective (hard cross-entropy, soft KL alignment with stop-gradient, and negative learning). This object does the work of decomposing the mixed pseudo-label gradient because each expert occupies a separate low-rank subspace, so the paper's Theorems 1 and 2 bound the deviation of the Positive Expert's update from the reliable gradient and show its learned subspace is closer to the reliable target subspace than a single adapter's is.

What would settle it

Run TriNoL with the routing rule inverted on a held-out noisy setting, sending low-confidence samples to the Positive Expert and high-confidence samples to the Negative Expert; the paper's mechanism predicts accuracy should fall to roughly single-LoRA levels, so if it does not, the gain is not caused by confidence-aware routing.

Watch

Extended reading notes

Core claim

The central claim is that pseudo-label reliability should control the adaptation pathway, not only the loss weight. Concretely, with two confidence thresholds $\tau_-$ and $\tau_+$, unlabeled samples are split into three regions that feed three LoRA experts sharing a frozen backbone: hard cross-entropy on high-confidence pseudo-labels for the Positive Expert, Kullback-Leibler divergence to the detached weak-view distribution for medium-confidence samples in the Alignment Expert, and a negative-learning loss for low-confidence samples in the Negative Expert. The argument is carried by gradient decomposition: a single adapter receives $G_{\mathrm{single}} = G_+ + G_a + G_-$, while the Positive Expert receives $G_+ + E_r$, and under a confidence-routing quality assumption $\|E_r\|_2 \le \rho \|G_a + G_-\|_2$ with $\rho<1$, the reliable update is less contaminated. At inference only the Positive Expert is used; the other two regularize training. The empirical core is the rank-matched comparison: three LoRA experts at rank 8 reach 88.42% on FOOD-101 N2, versus 87.73% for a single LoRA at rank 24 with the same parameter count, which attributes the improvement to confidence-aware specialization.

Load-bearing premise

The load-bearing premise is that a high max-softmax confidence score genuinely orders pseudo-label correctness; if confidence is miscalibrated or domain shift scrambles that ordering, the Positive Expert still receives contaminated gradients and the whole gain is lost.

Editorial extensions

If this is right

In scarce-label, noisy-pseudo-label regimes such as FOOD-101 with 2 to 10 labels per class and Semi-Aves with mixed out-of-distribution unlabeled data, separating the three confidence regions into specialized adapters yields consistent accuracy gains over the strongest single-path baselines.
The rank-matched efficiency comparison implies the gain is architectural rather than parametric: a single LoRA at rank 24 with equal parameter count is outperformed by three rank-8 experts.
Expert routing is dynamic during training, so the method turns a static confidence threshold into a curriculum that moves samples from uncertain and noisy regions into the reliable region as the model matures.
Because the backbone stays frozen and only the LoRA experts and classifier head are updated, the training cost stays near that of a single adapter, making the robustness cheap to buy.
The method does not dominate everywhere: in more stable settings such as CIFAR-100 N25/N100 and ImageNet 10%, existing baselines remain slightly better, indicating the benefit is specific to low-label and noisy regimes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Extending the paper's logic, the same routing principle should transfer to other parameter-efficient modules such as adapters or prompts, because the mechanism is the separation of gradient subspaces rather than a property of LoRA itself.
A natural test the authors leave implicit is to feed the router a calibration-corrected confidence score; if miscalibration is the bottleneck, accuracy should rise further and the Negative Expert should become less active as training progresses.
If the central claim is right, an oracle router with access to true label correctness is the upper bound of the method, and the gap between TriNoL and that oracle measures how much remains to be gained from better confidence signals.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know: this is a modest but genuine increment in semi-supervised PEFT for frozen vision foundation models, and it is honest about where it does not win. The new thing is routing pseudo-labeled samples into three confidence regions, each handled by its own LoRA expert with a different objective - hard CE for high-confidence, soft KL for medium-confidence, and a negative suppression loss for low-confidence. That specific combination is not in the V-PET or FineSSL baselines, and the authors do include a rank-matched comparison: three rank-8 experts versus one rank-24 LoRA at equal parameter count. The component and routing ablations support the claim that the gain comes from confidence-aware specialization rather than just capacity. Credit is earned for the structure and honesty of the empirical work: the limitations section concedes dependence on calibrated confidence, and the paper openly reports settings where TriNoL does not win.

Soft spots, in proportion. First, the theory is much weaker than it looks. Theorem 1 is essentially an assumption in disguise: Assumption 2 asserts that routing already reduces the contaminating gradient norm, and Theorem 1 restates that. The later subspace and stability results are standard low-rank perturbation bounds. I would not use the theory as a reason to accept; the empirical case has to stand alone. Second, the flagship evidence for expert specialization is one comparison on FOOD-101 N2 - 88.42% versus 87.73% for rank-24 single LoRA. The other ablations compare against rank-8 single LoRA, which does not isolate specialization from capacity. The marginal contribution of the Negative Expert over Positive+Alignment is on the order of 0.3-0.6 points, and nothing is significance-tested. Third, no code is released, and ImageNet numbers have no error bars. Fourth, the confidence thresholds and loss weights were selected on the evaluation datasets through the sensitivity analysis, so the headline results are post-selection. That is common in this area, but it should be disclosed more prominently.

None of this kills the paper. The core idea is coherent, the ablations are directionally consistent, and the gains are plausible for the low-label/noisy regime. I would ask the authors for code, for error bars or a clear statement about single runs on ImageNet, and for a direct test of the routing premise - calibration curves or reliability diagrams, and an analysis showing accepted-but-wrong rates actually differ across the three experts (Figure 6 partly addresses this). I would also ask them to discuss the relationship to their own prior multi-expert SSL papers so the novelty claim is clean.

Who this is for: practitioners doing low-label PEFT-SSL with frozen backbones, and researchers working on pseudo-label noise in foundation-model adaptation. It deserves a serious referee. I would conditionally accept with the above requests.

Referee Report

4 major / 5 minor

Summary. The paper proposes TriNoL, a semi-supervised adaptation method for frozen vision foundation models (VFMs) that replaces a single LoRA adapter with three expert LoRA branches. Unlabeled samples are routed by pseudo-label confidence into a Positive Expert (high confidence, hard pseudo-label cross-entropy), an Alignment Expert (medium confidence, soft KL alignment), and a Negative Expert (low confidence, a negative-learning loss that suppresses commitment to the top pseudo-label). The backbone remains frozen; only the LoRA experts and classifier head are trained. The paper includes a theoretical analysis of gradient contamination and low-rank subspace stability, plus experiments on CIFAR-100, FOOD-101, Semi-Aves, and ImageNet. The authors report that TriNoL improves over several SSL/PEFT baselines in low-label and noisy pseudo-label settings, and they include an efficiency comparison against a rank-matched Single LoRA baseline.

Significance. If the empirical claims are reliable, the core idea—using pseudo-label confidence to control the adaptation pathway rather than only the loss weight—is a timely and potentially useful contribution for parameter-efficient SSL with foundation models. The paper has several strengths: it honestly reports settings where TriNoL does not win (CIFAR-100 N25/N100, ImageNet 10%); it includes a parameter-matched efficiency comparison (Table 2); and its routing ablation (Table 4) separates the effect of confidence routing from the use of multiple experts. The theoretical bounds are clearly stated, even though they are conditional on assumptions. However, the empirical support for the central mechanism is thinner than the claims require: the key rank-matched comparison is reported on only one dataset setting, several improvements are within one standard deviation, ImageNet results lack error bars and significance information, and the sensitivity analysis suggests hyperparameters were selected on the evaluation datasets. The paper's theoretical Section 4 also partly restates the desired conclusion as an assumption.

major comments (4)

[Section 6, Table 2] The claim that TriNoL's improvement comes from confidence-aware expert specialization rather than from increased LoRA capacity rests on a single comparison: FOOD-101 N2, where TriNoL (88.42) beats Single LoRA rank 24 (87.73). No rank-matched Single LoRA rank 24 results are reported for CIFAR-100 N4, Semi-Aves (in-distribution or OOD), ImageNet 1%, or any other configuration. Without such controls, the alternative explanation—that three independent low-rank updates are beneficial even without confidence-aware routing—is not excluded. Please add rank-matched comparisons for at least the other main datasets, or explicitly restrict the specialization claim to FOOD-101 N2.
[Section 5, Table 5] The sensitivity analysis in Table 5 appears to select the confidence thresholds (tau_minus=0.3, tau_plus=0.7) and loss weights (lambda_a=1.0, lambda_n=0.1) on the same datasets that are then used for the main results, FOOD-101 N2 and Semi-Aves. If no held-out validation split was used, the reported accuracies are the result of test-set tuning, which can inflate the apparent improvement over baselines that were not tuned in the same way. The paper should state explicitly which split was used for hyperparameter selection and, ideally, evaluate the final configuration on a separate test set or use nested validation.
[Section 4.2, Assumption 2 and Theorem 1] Theorem 1 (Eq. 13) is essentially a restatement of Assumption 2 (Eq. 12): assuming ||E_r|| <= rho ||G_a + G_-|| with rho < 1 is exactly the claim that the confidence routing already reduces the contaminating gradient norm entering the Positive Expert. The theorem therefore provides no independent theoretical justification for TriNoL. Moreover, the paper does not empirically verify Assumption 2—for example, it does not measure the residual routing error E_r, compare ||E_r|| to ||G_a + G_-||, or provide calibration curves/reliability diagrams showing that the confidence ordering is reliable in the tested noise regimes. The Limitations section concedes this dependence, but the main text should either weaken the theoretical claim or supply direct evidence for the assumption.
[Section 5 and Section 7, Tables 1, 3, and 4] Several of the headline improvements are small relative to the reported variation. For example, on CIFAR-100 N4 TriNoL is 80.92 +/- 0.19 vs FineSSL 80.66 +/- 0.24; on Semi-Aves mixed OOD it is 61.82 +/- 0.20 vs FineSSL 61.30 +/- 1.19. The ImageNet 1% and 10% rows have no standard deviations at all, so the reader cannot tell whether the 74.72 vs 74.46 difference on ImageNet 1% is meaningful. The component ablation in Table 3 also shows that adding the Negative Expert over Positive+Alignment gives only +0.40 on FOOD-101 N2 and +0.31 on Semi-Aves mixed, which are within one standard deviation of the component settings (e.g., 88.02 +/- 0.35 vs 88.42 +/- 0.33). Please report error bars for ImageNet, run more seeds where practical, and provide a simple significance test (e.g., paired bootstrap or repeated-measures test) for the key comparisons and ablations.

minor comments (5)

[Section 5, Figures 1 and 2] Figures 1 and 2 are described as supporting the observation that low-confidence samples have higher error rates and that a single adapter shows larger gradient deviation, but the figures are not accompanied by quantitative captions or axis labels in the text; please report the actual error rates and gradient cosine values so the reader can judge the magnitude of these effects.
[Section 7, Figure 6] Figure 6 refers to 'injected pseudo-label corruption' but the paper does not specify how corruption was injected, at what rate, or on which dataset; please define the protocol so the noise-robustness claim can be reproduced.
[Section 3.5, Eq. (7)] The text says lambda_pos is 'usually larger,' but Table 5 reports sensitivity only for lambda_a and lambda_n; please state the fixed value of lambda_pos used in all experiments and whether it was tuned.
[Section 3.5, Eq. (5)] The KL divergence is written as KL(sg(p^w_i) || softmax(z^a(u^s_i))), which is KL(weak || strong); the surrounding text says 'matching the strong-view prediction to the detached weak-view probability,' which suggests the opposite direction. Please clarify the intended direction or correct the notation.
[Section 4.5, Theorem 3] Theorem 3 assumes local strong convexity and smoothness of the 'reliable objective' L_rel, which is not an object that is directly optimized by TriNoL; the connection between the idealized L_rel and the actual loss in Eq. (7) should be stated more concretely.

Circularity Check

1 steps flagged · score 4.0 of 10

Theorem 1 restates Assumption 2 via its own definitions; the empirical evaluation is independent and not circular.

self definitional [Section 4.2, Eq. (11), Assumption 2 (Eq. 12), Theorem 1 (Eq. 13)]
"Assumption 2 (Confidence routing quality). The confidence routing separates most ambiguous and noisy samples from the Positive Expert. There exists 0 ≤ ρ < 1 such that ∥Er∥2 ≤ ρ∥Ga + G−∥2. Theorem 1 (Reduced reliable-update contamination). Under Assumption 2, the Positive Expert has a smaller deviation from the reliable gradient than the single adapter: ∥G+TriNoL − G+∥2 ≤ ρ∥Gsingle − G+∥2."

By Eq. (11), G+TriNoL − G+ = Er by definition. By Lemma 1 and Eq. (8), Gsingle − G+ = Ga + G−. Substituting both identities into Theorem 1 turns its conclusion into exactly the bound in Assumption 2: ∥Er∥2 ≤ ρ∥Ga + G−∥2. The theorem therefore does not derive reduced contamination from the routing mechanism; it merely rewrites the assumed routing-quality bound in new notation. Whether confidence-based routing actually achieves ρ < 1 is left as an assumption, and the paper's own Limitations section concedes that miscalibrated predictions or strong domain shift can break the routing signal. The empirical accuracy comparisons are computed from actual training runs and are separate from this tautological step, so only the theoretical claim of reduced contamination is circular.

full rationale

The only clearly circular step is in the theoretical analysis of Section 4.2. Eq. (11) defines the Positive Expert gradient as G+TriNoL = G+ + Er, Lemma 1 gives ∥Gsingle − G+∥2 = ∥Ga + G−∥2, and Assumption 2 imposes ∥Er∥2 ≤ ρ∥Ga + G−∥2. Substituting both identities into Theorem 1 makes the theorem's conclusion identical to Assumption 2; the 'reduced contamination' result is the routing-quality assumption renamed as a theorem rather than a derived consequence. This is a genuine self-definitional step in the theory, but it does not taint the empirical evaluation: the accuracy comparisons, the rank-matched Single LoRA rank 24 baseline, the component ablations, and the routing ablations are measured from real training runs and are not constructed from Assumption 2. The rank-matched comparison is a legitimate check against capacity confounds, and the main-table gains over V-PET and FineSSL are external benchmarks. No load-bearing self-citation appears: the authors' prior works are cited as SSL background, not as the source of the triple-expert routing mechanism or the LoRA expert design. The Limitations section itself concedes the unproven dependence of confidence routing on calibration and domain shift, which is consistent with treating Assumption 2 as an assumption rather than a proven fact. Overall, the paper's central empirical claim is self-contained and independently tested, but the theoretical claim in Theorem 1 is definitionally forced by its own Assumption 2, so the circularity score is moderate rather than zero.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central empirical method depends on a small set of tuned hyperparameters (thresholds and weights), and the theoretical benefits are conditional on assumptions that largely state the benefit. No new physical entities are introduced.

free parameters (4)

confidence thresholds tau_plus, tau_minus = tau_plus=0.7, tau_minus=0.3
Chosen by sensitivity analysis on FOOD-101 N2 and Semi-Aves (Table 5); these thresholds control routing into the three experts and directly determine the main result.
loss weights lambda_pos, lambda_align, lambda_neg = lambda_align=1.0, lambda_neg=0.1 (lambda_pos not specified, described as 'usually larger')
Selected by sensitivity analysis (Table 5); lambda_neg = 0.1 is configured so the Negative Expert regularizes but does not over-penalize.
LoRA rank per expert = 8
Design choice; rank-matched baseline uses a single rank-24 LoRA to compare parameter counts.
LoRA scaling factor alpha = not specified
Standard LoRA includes alpha; the paper never states the value used, which affects update magnitude.

assumptions (5)

standard math Norm inequalities and SVD subspace perturbation inequalities used in Lemmas 1-3 and Theorem 2
Invoked without proof in Section 4.
domain assumption Assumption 1: the ambiguous and noisy gradients are not aligned with the reliable gradient, ||Ga+G-||>0
Section 4.1 Eq (9); if false, there would be no interference to separate.
ad hoc to paper Assumption 2: routing quality, ||Er|| <= rho||Ga+G-|| with rho<1
Section 4.2 Eq (12); this assumption directly encodes the reduced contamination that Theorem 1 concludes.
ad hoc to paper Assumption 3: rank-r spectral gap delta_r(S)>0 and bounded perturbations ||A+E||_op, ||Er||_op < delta_r(S)/2
Section 4.3 Eq (16); needed for the subspace stability bound in Theorem 2.
domain assumption Assumption 4: L_rel is mu-strongly convex and L-smooth locally, with bounded pseudo-label bias
Section 4.5, used to derive the bias-variance bound in Theorem 3.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Triple Expert Learning from Noisy Labels for Semi-Supervised Vision Foundation Model Adaptation." pith.science (2026). https://pith.science/paper/YE32G422

@misc{pith2026260809052,
  author       = {Pith},
  title        = {Pith review of: Triple Expert Learning from Noisy Labels for Semi-Supervised Vision Foundation Model Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YE32G422}},
  note         = {Machine review of arXiv:2608.09052}
}

read the original abstract

Semi-supervised adaptation of vision foundation models (VFMs) commonly freezes the pretrained backbone and updates lightweight modules such as LoRA. However, pseudo-labels have mixed reliability, and a single LoRA adapter must absorb reliable, ambiguous, and noisy gradients in the same low-rank space. This can make VFM adaptation sensitive to pseudo-label noise. We propose \textbf{TriNoL}, a \textbf{Tri}ple-expert learning framework from \textbf{No}isy \textbf{L}abels for semi-supervised VFM adaptation. TriNoL routes unlabeled samples into three confidence regions and assigns them to three LoRA experts: a Positive Expert for high-confidence pseudo-labels, an Alignment Expert for medium-confidence ambiguous samples, and a Negative Expert for low-confidence noisy samples. The VFM backbone remains frozen, and only the LoRA experts and classifier head are updated. By separating different pseudo-label reliability regions into specialized adaptation paths, TriNoL improves robustness to noisy supervision while keeping the training cost low.

Figures

Figures reproduced from arXiv: 2608.09052 by the authors.

**Figure 1.** Pseudo-labels exhibit mixed reliability across confidence regions. labels are wrong, they can reinforce incorrect decision boundaries and cause confirmation bias. Vision foundation models (VFMs), such as CLIP and DINO-series visual encoders, change the role of SSL [5, 35, 36]. In conventional SSL, the model must learn visual representations and class boundaries at the same time. In VFM-based SSL, the pretrained back… view at source ↗

**Figure 2.** Single LoRA suffers from mixed-gradient interference. decision boundaries, and the Negative Expert reduces the influence of unreliable pseudolabels without assuming that all low-confidence samples are incorrect. Third, we provide an efficient semi-supervised adaptation framework for VFMs, where the pretrained backbone remains frozen and only lightweight LoRA experts and the classifier head are updated. This makes T… view at source ↗

**Figure 3.** Comparison between standard SSL-based PEFT adaptation and TriNoL. Left: standard SSL-based PEFT adaptation uses a single LoRA adapter to absorb pseudolabel supervision from all unlabeled samples, including reliable, ambiguous, and noisy regions. This forces mixed-quality pseudo-label gradients into the same low-rank update space. Right: TriNoL decomposes unlabeled samples into three confidence regions according t… view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Efficiency analysis of TriNoL. We compare Single LoRA rank 8, Single LoRA rank 24, and TriNoL with three rank-8 experts. TriNoL has a similar trainable parameter count to Single LoRA rank 24, but achieves higher accuracy. This shows that the improvement mainly comes f…

**Figure 5.** Figure 5: Expert routing dynamics of TriNoL during training [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 37 canonical work pages

[1]

Self-training: A survey.Neurocomputing, 616:128904, 2025

Massih-Reza Amini, Vasilii Feofanov, Loic Pauletto, Lies Hadjadj, Emilie Devijver, and Yury Maximov. Self-training: A survey.Neurocomputing, 616:128904, 2025

work page 2025
[2]

Foundation models defining a new era in vision: A survey and outlook.IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 47(4):2245–2264, 2025

Muhammad Awais, Muzammal Naseer, Salman Khan, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Ming-Hsuan Yang, and Fahad Shahbaz Khan. Foundation models defining a new era in vision: A survey and outlook.IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 47(4):2245–2264, 2025

work page 2025
[3]

Mixmatch: A holistic approach to semi-supervised learning.Ad- vances in Neural Information Processing Systems, 32, 2019

David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. Mixmatch: A holistic approach to semi-supervised learning.Ad- vances in Neural Information Processing Systems, 32, 2019

work page 2019
[4]

Semi-supervised vision transformers at scale.Advances in Neural Information Processing Systems, 35:25697–25710, 2022

Zhaowei Cai, Avinash Ravichandran, Paolo Favaro, Manchen Wang, Davide Modolo, Rahul Bhotika, Zhuowen Tu, and Stefano Soatto. Semi-supervised vision transformers at scale.Advances in Neural Information Processing Systems, 35:25697–25710, 2022

work page 2022
[5]

Emerging properties in self-supervised vision transform- ers

Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bo- janowski, and Armand Joulin. Emerging properties in self-supervised vision transform- ers. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9650–9660, 2021

work page 2021
[6]

Softmatch: Addressing the quantity-quality trade- off in semi-supervised learning.arXiv preprint arXiv:2301.10921, 2023

Hao Chen, Ran Tao, Yue Fan, Yidong Wang, Jindong Wang, Bernt Schiele, Xing Xie, Bhiksha Raj, and Marios Savvides. Softmatch: Addressing the quantity-quality trade- off in semi-supervised learning.arXiv preprint arXiv:2301.10921, 2023

arXiv 2023
[7]

Adaptformer: Adapting vision transformers for scalable visual recognition

Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo. Adaptformer: Adapting vision transformers for scalable visual recognition. Advances in Neural Information Processing Systems, 35:16664–16678, 2022

2022
[8]

An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. 16LIU, FANG, HE, HONG, AND LIU: TRIPLE EXPERT LEARNING FROM ...

arXiv 2010

Show all 49 references

[9]

Erasing the bias: Fine-tuning foundation models for semi- supervised learning.arXiv preprint arXiv:2405.11756, 2024

Kai Gan and Tong Wei. Erasing the bias: Fine-tuning foundation models for semi- supervised learning.arXiv preprint arXiv:2405.11756, 2024

2024 arXiv
[10]

Co-teaching: Robust training of deep neural networks with extremely noisy labels.Advances in neural information processing systems, 31, 2018

Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels.Advances in neural information processing systems, 31, 2018

2018
[11]

Parameter- efficient fine-tuning for large models: A comprehensive survey.arXiv preprint arXiv:2403.14608, 2024

Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter- efficient fine-tuning for large models: A comprehensive survey.arXiv preprint arXiv:2403.14608, 2024

2024 arXiv
[12]

Research on the application of electronic technology of internet of things in smart city

Hongyang He. Research on the application of electronic technology of internet of things in smart city. In2020 International Conference on Intelligent Transportation, Big Data & Smart City (ICITBS), pages 454–457. IEEE, 2020

2020
[13]

Trustmatch: mitigating pseudo-label bias in semi- supervised learning with trust-aware refinement

Hongyang He and Yundi Hong. Trustmatch: mitigating pseudo-label bias in semi- supervised learning with trust-aware refinement. In2025 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pages 605–614. IEEE, 2025

2025
[14]

Research on pedestrian tracking algorithm based on deep learning

Hongyang He, Ziran Yan, Zichao Geng, and Xiushan Liu. Research on pedestrian tracking algorithm based on deep learning. In2021 International Conference on Com- puter Information Science and Artificial Intelligence (CISAI), pages 487–490. IEEE, 2021

2021
[15]

Research on surface defect detection method of metal workpiece based on machine learning

Hongyang He, Mingang Yuan, and Xiushan Liu. Research on surface defect detection method of metal workpiece based on machine learning. In2021 6th international conference on Intelligent Computing and Signal Processing (ICSP), pages 881–884. IEEE, 2021

2021
[16]

4s-classifier: Empowering conservation through semi-supervised learning for rare and endangered species

Hongyang He, Hongyang Xie, Guodong Shen, Boyang Fu, Haochen You, and Victor Sanchez. 4s-classifier: Empowering conservation through semi-supervised learning for rare and endangered species. In2025 International Joint Conference on Neural Networks (IJCNN), pages 1–10. IEEE, 2025
[17]

Semi-vim: bidi- rectional state space model for mitigating label imbalance in semi-supervised learning

Hongyang He, Hongyang Xie, Haochen You, and Victor Sanchez. Semi-vim: bidi- rectional state space model for mitigating label imbalance in semi-supervised learning. In2025 IEEE/CVF International Conference on Computer Vision (ICCV), pages 765–
[18]

Revisiting chain-of-thought reasoning under limited supervision: Semi-supervised chain-of-thought learning.arXiv preprint arXiv:2607.01511, 2026

Hongyang He, Jiuming Liu, and Victor Sanchez. Revisiting chain-of-thought reasoning under limited supervision: Semi-supervised chain-of-thought learning.arXiv preprint arXiv:2607.01511, 2026

2026 arXiv
[19]

Semi-supervised vision-language- action model.arXiv preprint arXiv:2606.21493, 2026

Hongyang He, Jiuming Liu, and Victor Sanchez. Semi-supervised vision-language- action model.arXiv preprint arXiv:2606.21493, 2026

2026 arXiv
[20]

Trico: Triadic game-theoretic co-training for robust semi-supervised learning.Advances in Neural Information Processing Systems, 38: 87545–87570, 2026

Hongyang He, Xinyuan Song, Yangfan He, Zeyu Zhang, Yanshu Li, Haochen You, Lifan Sun, and Wenqiao Zhang. Trico: Triadic game-theoretic co-training for robust semi-supervised learning.Advances in Neural Information Processing Systems, 38: 87545–87570, 2026. LIU, FANG, HE, HONG,...

2026
[21]

Newton-coupled dual-teacher semi-supervised learning framework

Hongyang He, Xinyuan Song, Yan Zhong, Daizong Liu, Xuanyu Liu, and Victor Sanchez. Newton-coupled dual-teacher semi-supervised learning framework. InForty- third International Conference on Machine Learning, 2026

2026
[22]

Token- aware representation augmentation for fine-grained semi-supervised learning

Hongyang He, Yan Zhong, Xinyuan Song, Daizong Liu, and Victor Sanchez. Token- aware representation augmentation for fine-grained semi-supervised learning. InThe Third Conference on Parsimony and Learning (Proceedings Track), 2026

2026
[23]

Be- yond data augmentation: Energy-based kuramoto neurons for semi-supervised learn- ing

Yundi Hong, Ao Li, Xinyuan Song, Yan Zhong, Hongyang He, and Victor Sanchez. Be- yond data augmentation: Energy-based kuramoto neurons for semi-supervised learn- ing. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2913–2922, 2026

2026
[24]

Parameter- efficient transfer learning for nlp

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter- efficient transfer learning for nlp. InInternational Conference on Machine Learning, pages 2790–2799. PMLR, 2019

2019
[25]

Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.International Conference on Learning Representations, 2022

2022
[26]

Visual prompt tuning

Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. InEuropean Conference on Computer Vision, pages 709–727. Springer, 2022

2022
[27]

Nlnl: Negative learning for noisy labels

Youngdong Kim, Junho Yim, Juseung Yun, and Junmo Kim. Nlnl: Negative learning for noisy labels. InProceedings of the IEEE/CVF international conference on computer vision, pages 101–110, 2019

2019
[28]

Pseudo-label: The simple and efficient semi-supervised learn- ing method for deep neural networks

Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learn- ing method for deep neural networks. InWorkshop on Challenges in Representation Learning, ICML, volume 3, page 896. Atlanta, 2013

2013
[29]

Dividemix: Learning with noisy labels as semi-supervised learning.arXiv preprint arXiv:2002.07394, 2020

Junnan Li, Richard Socher, and Steven CH Hoi. Dividemix: Learning with noisy labels as semi-supervised learning.arXiv preprint arXiv:2002.07394, 2020

2002 arXiv
[30]

Unlabeled data vs

Song-Lin Lv, Rui Zhu, Tong Wei, Yu-Feng Li, and Lan-Zhe Guo. Unlabeled data vs. pre-trained knowledge: Rethinking ssl in the era of large models.arXiv preprint arXiv:2505.13317, 2025

2025
[31]

Fine-tuning is fine, if calibrated.Advances in Neural Information Processing Systems, 37:136084– 136119, 2024

Zheda Mai, Arpita Chowdhury, Ping Zhang, Cheng-Hao Tu, Hong-You Chen, Vardaan Pahuja, Tanya Berger-Wolf, Song Gao, Charles Stewart, Yu Su, et al. Fine-tuning is fine, if calibrated.Advances in Neural Information Processing Systems, 37:136084– 136119, 2024

2024
[32]

Lessons and insights from a unifying study of parameter- efficient fine-tuning (peft) in visual recognition

Zheda Mai, Ping Zhang, Cheng-Hao Tu, Hong-You Chen, Quang-Huy Nguyen, Li Zhang, and Wei-Lun Chao. Lessons and insights from a unifying study of parameter- efficient fine-tuning (peft) in visual recognition. InProceedings of the Computer Vision and Pattern Recognition Conferenc...

2025
[33]

Enhancing clip with clip: Exploring pseudolabeling for limited-label prompt tuning.Advances in Neural Infor- mation Processing Systems, 36:60984–61007, 2023

Cristina Menghini, Andrew Delworth, and Stephen Bach. Enhancing clip with clip: Exploring pseudolabeling for limited-label prompt tuning.Advances in Neural Infor- mation Processing Systems, 36:60984–61007, 2023

2023
[34]

Boosting semi-supervised learning by bridging high and low-confidence predictions

Khanh-Binh Nguyen and Joon-Sung Yang. Boosting semi-supervised learning by bridging high and low-confidence predictions. InProceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 1028–1038, 2023

2023
[35]

Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

2023 arXiv
[36]

Learning transferable visual models from natural language supervision

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational Confer- ence on Machine Learning, p...

2021
[37]

Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li

Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A. Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simpli- fying semi-supervised learning with consistency and confidence.Advances in Neural Information Processing Systems, 33...

2020
[38]

Mean teachers are better role models: Weight- averaged consistency targets improve semi-supervised deep learning results.Advances in Neural Information Processing Systems, 30, 2017

Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight- averaged consistency targets improve semi-supervised deep learning results.Advances in Neural Information Processing Systems, 30, 2017

2017
[39]

Van Engelen and Holger H

Jesper E. Van Engelen and Holger H. Hoos. A survey on semi-supervised learning. Machine Learning, 109(2):373–440, 2020

2020
[40]

Xudong Wang, Zhirong Wu, Long Lian, and Stella X. Yu. Debiased learning from naturally imbalanced pseudo-labels. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14647–14657, 2022

2022
[41]

Freematch: Self- adaptive thresholding for semi-supervised learning.arXiv preprint arXiv:2205.07246, 2022

Yidong Wang, Hao Chen, Qiang Heng, Wenxin Hou, Yue Fan, Zhen Wu, Jindong Wang, Marios Savvides, Takahiro Shinozaki, Bhiksha Raj, et al. Freematch: Self- adaptive thresholding for semi-supervised learning.arXiv preprint arXiv:2205.07246, 2022

2022 arXiv
[42]

Combating noisy labels by agree- ment: A joint training method with co-regularization

Hongxin Wei, Lei Feng, Xiangyu Chen, and Bo An. Combating noisy labels by agree- ment: A joint training method with co-regularization. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13726–13735, 2020

2020
[43]

Semi-supervised vision transformers

Zejia Weng, Xitong Yang, Ang Li, Zuxuan Wu, and Yu-Gang Jiang. Semi-supervised vision transformers. InEuropean Conference on Computer Vision, pages 605–620. Springer, 2022

2022
[44]

Traditional machine learning models for building energy performance prediction: A comparative research.Machine Learning Research, 8(1): 1–8, 2023

Zeyu Wu and Hongyang He. Traditional machine learning models for building energy performance prediction: A comparative research.Machine Learning Research, 8(1): 1–8, 2023. LIU, FANG, HE, HONG, AND LIU: TRIPLE EXPERT LEARNING FROM NOISY LABELS19

2023
[45]

Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V . Le. Self-training with noisy student improves imagenet classification. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10687–10698, 2020

2020
[46]

V-petl bench: A unified vi- sual parameter-efficient transfer learning benchmark.Advances in Neural Information Processing Systems, 37:80522–80535, 2024

Yi Xin, Siqi Luo, Xuyang Liu, Yuntao Du, Haodi Zhou, Xinyu Cheng, Christina Lee, Junlong Du, Haozhe Wang, Mingcai Chen, et al. V-petl bench: A unified vi- sual parameter-efficient transfer learning benchmark.Advances in Neural Information Processing Systems, 37:80522–80535, 2024

2024
[47]

A survey on deep semi- supervised learning.IEEE Transactions on Knowledge and Data Engineering, 35(9): 8934–8954, 2022

Xiangli Yang, Zixing Song, Irwin King, and Zenglin Xu. A survey on deep semi- supervised learning.IEEE Transactions on Knowledge and Data Engineering, 35(9): 8934–8954, 2022

2022
[48]

Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling.Advances in Neural Information Processing Systems, 34: 18408–18419, 2021

Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Oku- mura, and Takahiro Shinozaki. Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling.Advances in Neural Information Processing Systems, 34: 18408–18419, 2021

2021
[49]

Re- visiting semi-supervised learning in the era of foundation models.Advances in Neural Information Processing Systems, 38:59295–59324, 2026

Ping Zhang, Zheda Mai, Quang-Huy Percy Nguyen, and Wei-Lun Harry Chao. Re- visiting semi-supervised learning in the era of foundation models.Advances in Neural Information Processing Systems, 38:59295–59324, 2026

2026

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Self-training: A survey.Neurocomputing, 616:128904, 2025

Massih-Reza Amini, Vasilii Feofanov, Loic Pauletto, Lies Hadjadj, Emilie Devijver, and Yury Maximov. Self-training: A survey.Neurocomputing, 616:128904, 2025

work page 2025

[2] [2]

Foundation models defining a new era in vision: A survey and outlook.IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 47(4):2245–2264, 2025

Muhammad Awais, Muzammal Naseer, Salman Khan, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Ming-Hsuan Yang, and Fahad Shahbaz Khan. Foundation models defining a new era in vision: A survey and outlook.IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 47(4):2245–2264, 2025

work page 2025

[3] [3]

Mixmatch: A holistic approach to semi-supervised learning.Ad- vances in Neural Information Processing Systems, 32, 2019

David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. Mixmatch: A holistic approach to semi-supervised learning.Ad- vances in Neural Information Processing Systems, 32, 2019

work page 2019

[4] [4]

Semi-supervised vision transformers at scale.Advances in Neural Information Processing Systems, 35:25697–25710, 2022

Zhaowei Cai, Avinash Ravichandran, Paolo Favaro, Manchen Wang, Davide Modolo, Rahul Bhotika, Zhuowen Tu, and Stefano Soatto. Semi-supervised vision transformers at scale.Advances in Neural Information Processing Systems, 35:25697–25710, 2022

work page 2022

[5] [5]

Emerging properties in self-supervised vision transform- ers

Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bo- janowski, and Armand Joulin. Emerging properties in self-supervised vision transform- ers. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9650–9660, 2021

work page 2021

[6] [6]

Softmatch: Addressing the quantity-quality trade- off in semi-supervised learning.arXiv preprint arXiv:2301.10921, 2023

Hao Chen, Ran Tao, Yue Fan, Yidong Wang, Jindong Wang, Bernt Schiele, Xing Xie, Bhiksha Raj, and Marios Savvides. Softmatch: Addressing the quantity-quality trade- off in semi-supervised learning.arXiv preprint arXiv:2301.10921, 2023

arXiv 2023

[7] [7]

Adaptformer: Adapting vision transformers for scalable visual recognition

Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo. Adaptformer: Adapting vision transformers for scalable visual recognition. Advances in Neural Information Processing Systems, 35:16664–16678, 2022

2022

[8] [8]

An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. 16LIU, FANG, HE, HONG, AND LIU: TRIPLE EXPERT LEARNING FROM ...

arXiv 2010