RECORD
Dual Class-Aware Contrastive Federated Semi-Supervised Learning
Machine Learning · Submitted 16 Nov 2022
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Our comprehensive experiments show that DCCFSSL outperforms current state-of-the-art methods on three benchmark datasets and surpasses the FedAvg with relabeled unlabeled clients on CIFAR-10, CIFAR-100, and STL-10 datasets.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Federated semi-supervised learning (FSSL), facilitates labeled clients and unlabeled clients jointly training a global model without sharing private data. Existing FSSL methods predominantly employ pseudo-labeling and consistency regularization to exploit the knowledge of unlabeled data, achieving notable success in raw data utilization. However, these training processes are hindered by large deviations between uploaded local models of labeled and unlabeled clients, as well as confirmation bias introduced by noisy pseudo-labels, both of which negatively affect the global model's performance. In this paper, we present a novel FSSL method called Dual Class-aware Contrastive Federated Semi-Supervised Learning (DCCFSSL). This method accounts for both the local class-aware distribution of each client's data and the global class-aware distribution of all clients' data within the feature space. By implementing a dual class-aware contrastive module, DCCFSSL establishes a unified training objective for different clients to tackle large deviations and incorporates contrastive information in the feature space to mitigate confirmation bias. Moreover, DCCFSSL introduces an authentication-reweighted aggregation technique to improve the server's aggregation robustness. Our comprehensive experiments show that DCCFSSL outperforms current state-of-the-art methods on three benchmark datasets and surpasses the FedAvg with relabeled unlabeled clients on CIFAR-10, CIFAR-100, and STL-10 datasets. To our knowledge, we are the first to present an FSSL method that utilizes only 10\% labeled clients, while still achieving superior performance compared to standard federated supervised learning, which uses all clients with labeled data.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Federated semi-supervised learning (FSSL), facilitates labeled clients and unlabeled clients jointly training a global model without sharing private data. | background 1.00 | 0.06 | 0.02 | asserted 1.00 |
| s1 | Existing FSSL methods predominantly employ pseudo-labeling and consistency regularization to exploit the knowledge of unlabeled data, achieving notable success in raw data utilization. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s2 | However, these training processes are hindered by large deviations between uploaded local models of labeled and unlabeled clients, as well as confirmation bias introduced by noisy pseudo-labels, both of which negatively affect the global model's performance. | gap 0.56 | 0.07 | 0.03 | asserted 1.00 |
| s3 | In this paper, we present a novel FSSL method called Dual Class-aware Contrastive Federated Semi-Supervised Learning (DCCFSSL). | contribution 0.97 | 0.90 | 0.05 | asserted 0.99 |
| s4 | This method accounts for both the local class-aware distribution of each client's data and the global class-aware distribution of all clients' data within the feature space. | method 0.88 | 0.76 | 0.04 | asserted 1.00 |
| s5 | By implementing a dual class-aware contrastive module, DCCFSSL establishes a unified training objective for different clients to tackle large deviations and incorporates contrastive information in the feature space to mitigate confirmation bias. | method 0.96 | 0.91 | 0.10 | asserted 1.00 |
| s6 | Moreover, DCCFSSL introduces an authentication-reweighted aggregation technique to improve the server's aggregation robustness. | method 0.61 | 0.90 | 0.04 | asserted 1.00 |
| s7 | Our comprehensive experiments show that DCCFSSL outperforms current state-of-the-art methods on three benchmark datasets and surpasses the FedAvg with relabeled unlabeled clients on CIFAR-10, CIFAR-100, and STL-10 datasets. | result 1.00 | 0.97 | 0.66 | asserted 1.00 |
| s8 | To our knowledge, we are the first to present an FSSL method that utilizes only 10\% labeled clients, while still achieving superior performance compared to standard federated supervised learning, which uses all clients with labeled data. | contribution 0.55 | 0.94 | 0.29 | asserted 0.99 |
What was examined, and what was not
- arXiv listing abstract v2
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Federated semi-supervised learning (FSSL), facilitates labeled clients and unlabeled clients jointly training a global model without sharing private data. Existing FSSL methods predominantly employ pseudo-labeling and consistency regularization to exploit the knowledge of unlabeled data, achieving notable success in raw data utilization. However, these training processes are hindered by large deviations between uploaded local models of labeled and unlabeled clients, as well as confirmation bias introduced by noisy pseudo-labels, both of which negatively affect the global model's performance. In this paper, we present a novel FSSL method called Dual Class-aware Contrastive Federated Semi-Supervised Learning (DCCFSSL). This method accounts for both the local class-aware distribution of each client's data and the global class-aware distribution of all clients' data within the feature space. By implementing a dual class-aware contrastive module, DCCFSSL establishes a unified training objective for different clients to tackle large deviations and incorporates contrastive information in the feature space to mitigate confirmation bias. Moreover, DCCFSSL introduces an authentication-reweighted aggregation technique to improve the server's aggregation robustness. Our comprehensive experiments show that DCCFSSL outperforms current state-of-the-art methods on three benchmark datasets and surpasses the FedAvg with relabeled unlabeled clients on CIFAR-10, CIFAR-100, and STL-10 datasets. To our knowledge, we are the first to present an FSSL method that utilizes only 10\% labeled clients, while still achieving superior performance compared to standard federated supervised learning, which uses all clients with labeled data.
Human review
How the literature uses it
2 arXiv papers cite this in 2 sentences
Papers citing it · 1 sentence without a confident label
Cites it as background 1 paper
-
However, FL faces numerous challenges [ref] , among which the most fundamental is data statistical heterogeneity [ref] .
Forgetting Through Transforming: Enabling Federated Unlearning via Class-Aware Representation Transformation · Federated Learning · 0.84
What it cites · 2 sentences without a confident label
Uses the method or tool of 2 papers
-
When training on three datasets, we follow the work [ref] to employ the Wide ResNet WRN-16-2 [ref] as the backbone network.
Wide Residual Networks · Datasets and Network Architecture · 0.99
-
For the NonIID setting, we follow existing methods [ref] to adopt a Dirichlet distribution $Dir(γ)$ ($γ=1$ for all benchmark datasets) to produce the NonIID data partition in 50 clients.
Federated Learning with Matched Averaging · Federated Learning setting · 0.94
Cites as background 8 papers
-
Considering the labeled data distribution among clients and server, FSSL scenarios can be classified into three categories [ref] : (1) Labels-at-Server in which the clients only own an unlabeled dataset and the server owns a labeled dataset [ref] ; (2) Labels-at-Clients in which the clients own a hybrid dataset constructed by labeled data and unlabeled data [ref] ; (3) Labels-at-Partial-Clients in which the partial clients own labeled datasets and the other clients own unlabeled datasets [ref] .
SemiFed: Semi-supervised Federated Learning with Consistency and Pseudo-Labeling · Federated Semi-Supervised Learning · 0.94
-
Considering the labeled data distribution among clients and server, FSSL scenarios can be classified into three categories [ref] : (1) Labels-at-Server in which the clients only own an unlabeled dataset and the server owns a labeled dataset [ref] ; (2) Labels-at-Clients in which the clients own a hybrid dataset constructed by labeled data and unlabeled data [ref] ; (3) Labels-at-Partial-Clients in which the partial clients own labeled datasets and the other clients own unlabeled datasets [ref] .
SSFL: Tackling Label Deficiency in Federated Learning via Personalized Self-Supervision · Federated Semi-Supervised Learning · 0.94
-
Considering the labeled data distribution among clients and server, FSSL scenarios can be classified into three categories [ref] : (1) Labels-at-Server in which the clients only own an unlabeled dataset and the server owns a labeled dataset [ref] ; (2) Labels-at-Clients in which the clients own a hybrid dataset constructed by labeled data and unlabeled data [ref] ; (3) Labels-at-Partial-Clients in which the partial clients own labeled datasets and the other clients own unlabeled datasets [ref] .
Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning · Federated Semi-Supervised Learning · 0.94
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.