Pith. sign in

REVIEW 2 major objections 5 minor 61 references

EFFEKT: Efficient Federated Knowledge Transfer to Foundation Models

T0 review · 2 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read EFFEKT replaces server-side weight averaging with two distillation steps, letting a DINOv2 foundation model absorb new fine-grained domains from lightweight clients and gain 3.9 points of top-1 accuracy over prior art.

desk verdict EFFEKT's reported gains are real under matched source-target pairs, but the paper should state that condition up front; it's a solid incremental contribution, not a general-purpose fix. read the letter →

arxiv 2608.08138 v1 pith:4OVP6QWR submitted 2026-08-08 cs.CV cs.LG

Matteo Caligiuri , Francesco Barbato , Pietro Zanuttigh , Francesco Restuccia This is my paper · ORCID
classification cs.CVcs.LG
keywords federatedlearningfoundationmodelsknowledgedistillationLoRAadaptersserver-sideaggregationfine-grainedclassificationedgedevices
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes EFFEKT, a federated learning framework in which clients never train or transmit the large model. Each client runs a small proxy encoder (MobileNetV3-small) pretrained to align its feature space with a server-side DINOv2 foundation model, and locally optimizes only a small classification head on private data. The server aggregates these heads and then updates a low-rank (LoRA) adapter for the foundation model using two distillation steps on public, task-relevant images: Clients-to-Server distillation transfers what the clients learned, and Joint Alignment re-synchronizes the proxy and server feature spaces. Across five fine-grained recognition domains, EFFEKT reports an average top-1 improvement of 3.9 percentage points and top-5 of 2.7 percentage points over the strongest prior method, FedPromo, while on-device experiments show the client load stays within a few watts and a few megabytes per round. If these results hold, foundation models can keep absorbing new domains from user data without being deployed or fine-tuned on the devices themselves.

What carries the argument

The load-bearing mechanism is the bi-directional cross-distillation pair formed by Clients-to-Server (C2S) and Joint Alignment (JA), applied around a shared representation space between two heterogeneous encoders. The proxy MobileNetV3 encoder is pretrained to align its features with DINOv2's on a public task-specific dataset; during federated rounds the client updates only a head, and the server updates a per-domain LoRA adapter (targeting Q and V projections with rank 16, totalling 1.57M parameters) using the distillation losses of Equations 1 and 2. C2S transfers the clients' logit-level knowledge into the LoRA, and JA re-aligns the two feature spaces with a combination of cross-entropy, feature-space distances, and bidirectional logit distillation, which the paper argues regularizes the alternating-minimization loop into a stable fixed point.

What would settle it

Run the released code on the StanfordCars-to-CompCars configuration with the stated hyperparameters; if the server-side top-1 accuracy does not reach approximately 43% (and thus exceeds FedPromo's 35.8 by the reported 7.2 points), the central claim of improved server-side accuracy is falsified.

Watch

Extended reading notes

Core claim

The central claim is that server-side accuracy of a foundation model can be improved under federated constraints by replacing weight-space averaging with two complementary distillation objectives. In Clients-to-Server (C2S) distillation, the updated client heads are held frozen and are used to score features from the server model on public pretraining images; a reversed knowledge-distillation loss (KL divergence from the client-head predictions to the server model's predictions) is minimized by updating only a LoRA adapter, so the server model learns the new private-domain concepts without ever seeing private data. In Joint Alignment (JA) distillation, the aggregated client model and the LoRA-adapted server model are jointly optimized on the same public images with cross-entropy, bidirectional feature-space (L1, L2, cosine) and logit-level losses, restoring the feature-space compatibility that the LoRA update disturbed. The paper reports that this pair of updates yields an average improvement of 3.9 percentage points (top-1) and 2.7 percentage points (top-5) over the previous state of the art across five fine-grained domains, while client-side training remains limited to a small head and the LoRA adapters represent fewer than 1% of the server model's parameters.

Load-bearing premise

The public task-relevant images used for server-side distillation must be semantically close to the private client images, because the server only ever learns from those public images; when the public set is generic (ImageNet), the paper's own results show the advantage over FedPromo disappears.

Editorial extensions

If this is right

  • Server-side accuracy of the foundation model improves by an average of 3.9 points (top-1) and 2.7 points (top-5) over the previous state of the art across five fine-grained domains, without any client ever training the large model.
  • Client devices remain lightweight: they train only a small classification head on private data, and real deployments on low-power edge boards consume under 7 W and under 5 MB/s of network traffic.
  • Per-domain LoRA adapters and heads let the server add new domains without retraining old ones; a prototype-based discriminator selects the correct adapter when the domain is not specified.
  • Ablations show both distillation steps are needed: disabling C2S or JA individually costs about 2.5 points of top-1 accuracy, and the best results require both together with a rank-16 LoRA.
  • The gains depend on task-specific public pretraining: when the public set is replaced by ImageNet, the advantage over FedPromo disappears (28.3 vs. 28.5 top-1), so the method targets scenarios with semantically aligned public data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not explore is replacing the public task-specific images with synthetic or text-driven proxies; since the distillation losses operate on logits and features rather than raw pixels, class-conditional image generation or paired captions might supply the required alignment without any public dataset.
  • The reversed orientation of the C2S loss (KL from the client-head distribution to the server model's distribution) makes the server match the client heads' entropy; this may act as a regularizer that prevents overfitting to the public pretraining set, a role the paper does not directly test.
  • The reported 3.9-point average gain is concentrated in two of the five domains (NABirds and CompCars); a broader multi-domain benchmark would clarify whether the mechanism systematically helps fine-grained domains or mainly those with the largest public-private similarity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes EFFEKT, a multi-domain federated learning framework in which lightweight client-side proxy models (MobileNetV3-small) train only a small classification head, while a server-side DINOv2 foundation model is updated via domain-specific LoRA adapters. Two new server-side distillation objectives, Clients-to-Server (C2S, Eq. 1) and Joint Alignment (JA, Eq. 2), transfer client knowledge to the foundation model and realign the proxy feature space using public task-specific pretraining datasets. The evaluation compares EFFEKT with six baselines (FedAvg, FedAvg+EMA, FedProx, MOON, FedHEAL, FedPromo) on five fine-grained domain pairs, reports a multi-domain inference setting, on-device deployment on Raspberry Pi and Jetson devices, and extensive ablations.

Significance. If the headline result is taken at face value, EFFEKT is a practically useful contribution: it improves server-side accuracy of a frozen foundation model under strict client compute constraints, with public code, six-baseline comparisons, multi-seed statistics (Table A.9, Figures A.5 and A.6), a component ablation (Table 5), real-device validation with energy and bandwidth measurements, and an honest out-of-domain failure analysis (Table 8 and Table A.6). The paper also clearly separates simulated and on-device training. Its main weakness is that the claimed 3.9%/2.7% average gains are obtained only for source-target pairs that are semantically aligned, and the manuscript does not quantify how the gain degrades with domain similarity; the Introduction does not qualify the central claim accordingly.

major comments (2)
  1. [Section 7 (Out-of-Domain Pretraining Results, Tables 8 and A.6)] The central claim in the Introduction, an average increase of 3.9% top-1 and 2.7% top-5 over the state of the art across five fine-grained domains, is supported only by Table 1, where each target domain is paired with a semantically matched public pretraining set. When pretraining is switched to ImageNet-1k, Table 8 shows EFFEKT at 28.3 top-1 versus 28.5 for FedPromo, and Table A.6 shows that EFFEKT trails FedPromo on NABirds (16.5 vs 18.5), OxfordPets (89.0 vs 89.5), and Military Aircraft top-5 (24.6 vs 24.8). The Appendix A.5 domain-shift analysis reports DINO similarities averaging about 0.67, so the favorable source-target selection is the very condition under which the headline gains appear. Because no experiment interpolates between the matched pairs and the ImageNet extreme, the reader cannot determine at what level of alignment the advantage disappears. The Impact Statement already concedes that the method requires pretraining data with reasonable alignment; the main text should state the 3.9%/2.7% claim as conditional and add a controlled similarity sweep (for example, varying source-set mixing or using intermediate source datasets) to quantify the boundary.
  2. [Section 5 and Section 7 (Tables 6 and 7)] The two most important hyperparameters, lambda_KD and the LoRA rank, are selected by ablations run on CompCars (Tables 6 and 7), and CompCars is also one of the five test domains in the central comparison (Table 1). This makes the CompCars row of the headline table optimistically biased, and it is unclear how much of the 7.2% top-1 gain over FedPromo on CompCars is attributable to tuning on that same domain. The authors should either fix these hyperparameters before observing the test domains, tune them on a validation task not included in the five evaluation domains, or report the full Table 1 for rank 8 and rank 32 and for neighboring lambda_KD values so the reader can see the sensitivity on all domains.
minor comments (5)
  1. [Section 4.1 (Discussion on Convergence)] The paragraph on convergence explicitly disclaims a formal proof and then describes C2S/JA as a proximal alternating minimization strategy; since the required smoothness and bounded-drift assumptions are not stated, this should be labeled as intuition and empirical evidence rather than a convergence argument.
  2. [Section A.3 (Figures A.5 and A.6)] Figure A.6 is described in the text as Top-5 accuracy, but the y-axis label in the figure caption reads "Top-1 accuracy"; please correct the label.
  3. [Section A.1] The sentence "with n_r = 1000 rounds the top-1 accuracy is is42.1%" contains a duplicated "is"; please fix the typo.
  4. [Section 7 (Component Analysis)] The phrase "distillation on the task-specific pretraining datasets alone is meaningless... effectively tackle a data-free Unsupervised Domain Adaptation task" is potentially misleading, because C2S and JA do use the public pretraining data D^di_p; if the intended meaning is "without private client data," the wording should be changed accordingly.
  5. [Equation (1)] The loss L_KD^L(P,T) is introduced in the text as the reverse of the standard KD orientation, but in Eq. (1) both entries are outputs of the same client head on server and client features; a brief clarification of which side plays the teacher and which the student in this reverse setting would improve readability.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the C2S/JA objectives are new, measured against FedPromo, and the ablation with both disabled reproduces the FedPromo baseline exactly.

full rationale

EFFEKT's derivation chain is empirical and self-contained against external baselines. The server-side C2S objective (Eq. 1) distills the logits of frozen client heads through the LoRA-adapted server encoder on public pretraining data, and the JA objective (Eq. 2) combines cross-entropy on the public task-specific head, feature-space alignment, and bidirectional logit alignment. Neither objective is defined in terms of the private target labels, and the evaluation datasets have class sets disjoint from the public pretraining data, so no target metric is fitted by construction. The ablation in Table 5, with C2S and JA disabled, reproduces FedPromo's exact top-1/top-5 numbers (35.8/69.4), which is the expected reduction to the shared baseline rather than a circular construction. The only self-citation of note is the reuse of FedPromo's client-side ICP regularization and the same five domain pairs (Sec. 4.1 and Sec. 5), but FedPromo is used as the comparison baseline and its numbers are reproduced, so the cited component is not load-bearing in a way that forces the claimed 3.9% top-1 and 2.7% top-5 improvements. The Out-of-Domain experiments (Table 8 and Table A.6) show that EFFEKT's advantage largely disappears when the public data are mismatched, which is a limitation on external validity rather than a circularity: the headline gain is conditional on the chosen source-target pairs, but it is measured, not derived, from those pairs. No step reduces, by equation or by construction, to its own inputs; the score of 2 reflects only the presence of a minor, non-load-bearing self-citation to FedPromo.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or architectural entities. It reuses DINOv2, MobileNetV3, LoRA, and the FedPromo setup. The main additional assumptions are the domain-alignment assumption for public source data and white-box access to the foundation model, both acknowledged in the paper. The free parameters are training hyperparameters selected by ablation or hand, with lambda_KD and LoRA rank tuned on CompCars, which is also one of the test domains.

free parameters (4)
  • lambda_KD = 0.01
    Weight of the logit distillation term in Eq. 2; chosen by ablation on CompCars (Table 6), not by validation on held-out domains.
  • LoRA rank = 16
    Rank of the Q and V LoRA adapters; chosen by ablation on CompCars (Table 7). Rank 16 gives the best top-1 accuracy in that ablation.
  • server-side distillation learning rate = 1e-5
    Learning rate for C2S and JA updates on the server; set by hand and not ablated, but central to the distillation dynamics.
  • distillation frequency = 1 (every round)
    Table A.7 shows accuracy grows roughly linearly with frequency f; the paper uses f=1 throughout.
assumptions (4)
  • domain assumption The server and client encoders are pretrained so their feature spaces have the same dimension and can share a classification head.
    Section 3 and 4 define F_O and F_E in R^{n_f} and attach the same head to both encoders; this alignment is assumed to hold after pretraining and during federated training.
  • domain assumption Public task-specific pretraining data D^di_p are sufficiently aligned with the private target data D^di for server-side distillation on public images to improve target accuracy.
    Eqs. 1 and 2 are optimized on D^di_p only; Section A.5 quantifies domain shift and notes MilitaryAircraft is weakly aligned, and Out-of-Domain experiments (Table 8, Table A.6) show large degradation when the assumption fails.
  • domain assumption The server has white-box access to the foundation model's logits and latents and can run optimization on them.
    Stated as a limitation in the Impact Statement and in Section 1; the C2S and JA losses require F_O and the logits of the foundation model.
  • ad hoc to paper The proximal alternating minimization view of C2S/JA assumes smooth losses and bounded drift so that alternating updates converge.
    Section 4.1 Discussion on Convergence; the paper provides no formal proof and relies on empirical monotonic accuracy to support the claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EFFEKT: Efficient Federated Knowledge Transfer to Foundation Models." pith.science (2026). https://pith.science/paper/4OVP6QWR

@misc{pith2026260808138,
  author       = {Pith},
  title        = {Pith review of: EFFEKT: Efficient Federated Knowledge Transfer to Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4OVP6QWR}},
  note         = {Machine review of arXiv:2608.08138}
}
read the original abstract

Recent data protection laws have accelerated the adoption of Federated Learning (FL) for privacy-preserving decentralized training. Nevertheless, increasing model sizes impose substantial computational demands on client devices, limiting FL applicability in resource-constrained settings. We introduce a novel multi-domain federated learning framework in which lightweight client-side proxy models collaborate with a server-side Foundation Model (FM) to learn new concepts without sharing private data. Our approach, EFFEKT, enables efficient server-side training of domain-specific LoRA adapters while preserving feature-space alignment between the FM and proxy extractors via novel bi-directional cross-distillation strategies. Experiments on multiple real-world datasets and deployments on low-power edge devices demonstrate improvements over state-of-the-art baselines in most considered domains while maintaining lightweight computation at the client side.

Figures

Figures reproduced from arXiv: 2608.08138 by the authors.

Figure 1
Figure 1. EFFEKT is a heterogeneous federated learning pipeline that allows bidirectional, multidomain [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The EFFEKT architecture and cross-distillation strategies. a) pretraining pipeline; b) local update [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Real devices’ setup: 3 Raspberry Pi 4, 3 Raspberry Pi 5, 5 Jetson Nano, and 1 Jetson Orin-Nano. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 37 canonical work pages

  1. [1]

    Deep learning with differential privacy

    Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp.\ 308--318, 2016

    work page 2016

  2. [2]

    FedPromo : Federated lightweight proxy models at the edge bring new domains to foundation models

    Matteo Caligiuri, Francesco Barbato, Donald Shenaj, Umberto Michieli, and Pietro Zanuttigh. FedPromo : Federated lightweight proxy models at the edge bring new domains to foundation models. arXiv preprint arXiv:2508.03356, 2025

    arXiv 2025

  3. [3]

    Eclipse mosquitto [software]

    Eclipse Foundation . Eclipse mosquitto [software]. Available: https://mosquitto.org/, 2009

    work page 2009

  4. [4]

    Graphana dashboard [software]

    Grafana Labs . Graphana dashboard [software]. Available: https://grafana.com/, 2013

    work page 2013

  5. [5]

    Prometheus [software]

    Prometheus maintainers and contributors . Prometheus [software]. Available: https://prometheus.io, 2013

    work page 2013

  6. [6]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 22500--22510, 2023

    2023

  7. [7]

    Sonoff [hardware]

    Shenzhen Sonoff Technologies Co., Ltd. Sonoff [hardware]. Available: https://sonoff.tech/, 2015

    work page 2015

  8. [8]

    Tasmota firmware [software]

    Tasmota maintainers and contributors . Tasmota firmware [software]. Available: https://tasmota.github.io/docs/, 2017

    work page 2017

Show all 61 references
  1. [9]

    Theoretical analysis of privacy leakage in trustworthy federated learning: A perspective from linear algebra and optimization theory, 2024

    Xiaojin Zhang and Wei Chen. Theoretical analysis of privacy leakage in trustworthy federated learning: A perspective from linear algebra and optimization theory, 2024. URL https://arxiv.org/abs/2407.16735

    2024 arXiv

  2. [10]

    SL o RA : Federated parameter efficient fine-tuning of language models

    Sara Babakniya, Ahmed Elkordy, Yahya Ezzeldin, Qingfeng Liu, Kee-Bong Song, MOSTAFA EL-Khamy, and Salman Avestimehr. SL o RA : Federated parameter efficient fine-tuning of language models. In International Workshop on Federated Learning in the Age of Foundation Models in Conju...

    2023

  3. [11]

    Cross-architecture auxiliary feature space translation for efficient few-shot personalized object detection

    Francesco Barbato, Umberto Michieli, Jijoong Moon, Pietro Zanuttigh, and Mete Ozay. Cross-architecture auxiliary feature space translation for efficient few-shot personalized object detection. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), ...

    2024

  4. [12]

    Flower: A friendly federated learning research framework

    Daniel J Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Javier Fernandez-Marques, Yan Gao, Lorenzo Sani, Hei Li Kwing, Titouan Parcollet, Pedro PB de Gusmão, and Nicholas D Lane. Flower: A friendly federated learning research framework. arXiv preprint arXiv:2007.14390, 2020

    2007 arXiv

  5. [13]

    Food-101--mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101--mining discriminative components with random forests. In Proceedings of the European Conference on Computer Vision (ECCV), Part VI, pp.\ 446--461. Springer, 2014

    2014

  6. [14]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv \'e J \'e gou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 9650--9660, 2021

    2021

  7. [15]

    On the importance and applicability of pre-training for federated learning

    Hong-You Chen, Cheng-Hao Tu, Ziwei Li, Han Wei Shen, and Wei-Lun Chao. On the importance and applicability of pre-training for federated learning. In Proceedings of the 11th International Conference on Learning Representations (ICLR), 2023. URL https://openreview.net/forum?id=...

    2023

  8. [16]

    Fair federated learning under domain skew with local consistency and domain diversity

    Yuhang Chen, Wenke Huang, and Mang Ye. Fair federated learning under domain skew with local consistency and domain diversity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 12077--12086, 2024

    2024

  9. [17]

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009

    2009

  10. [18]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human langu...

    2019

  11. [19]

    Source-free unsupervised domain adaptation: A survey, 2023

    Yuqi Fang, Pew-Thian Yap, Weili Lin, Hongtu Zhu, and Mingxia Liu. Source-free unsupervised domain adaptation: A survey, 2023. URL https://arxiv.org/abs/2301.00265

    2023 arXiv

  12. [20]

    Evaluating federated dino’s performance on the segmentation task across diverse domains

    Marko Harasic, Dennis Lehmann, and Adrian Paschke. Evaluating federated dino’s performance on the segmentation task across diverse domains. In 2024 IEEE International Conference on Big Data (BigData), pp.\ 7784--7789, 2024. doi:10.1109/BigData62323.2024.10825380

    2024

  13. [21]

    Distilling the knowledge in a neural network

    Geoffrey Hinton. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015

    2015 arXiv

  14. [22]

    Cronus: Robust and heterogeneous collaborative learning with black-box knowledge transfer

    Chang Hongyan, Shejwalkar Virat, Shokri Reza, and Houmansadr Amir. Cronus: Robust and heterogeneous collaborative learning with black-box knowledge transfer. arXiv preprint arXiv:1912.11279, 2019

    1912 arXiv

  15. [23]

    Searching for mobilenetv3

    Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, et al. Searching for mobilenetv3. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 1314--1324, 2019

    2019

  16. [24]

    Measuring the effects of non-identical data distribution for federated visual classification

    Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint arXiv:1909.06335, 2019

    1909 arXiv

  17. [25]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1 0 (2): 0 3, 2022

    2022

  18. [26]

    Communication-efficient on-device machine learning: Federated distillation and augmentation under non-iid private data

    Eunjeong Jeong, Seungeun Oh, Hyesung Kim, Jihong Park, Mehdi Bennis, and Seong-Lyun Kim. Communication-efficient on-device machine learning: Federated distillation and augmentation under non-iid private data. arXiv preprint arXiv:1811.11479, 2018

    2018 arXiv

  19. [27]

    Scaffold: Stochastic controlled averaging for federated learning

    Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International Conference on Machine Learning, pp.\ 5132--5143. PMLR, 2020

    2020

  20. [28]

    Kawano and K

    Y. Kawano and K. Yanai. Automatic expansion of a food image dataset leveraging existing categories with domain adaptation. In Proceedings of the ECCV Workshop on Transferring and Adapting Source Knowledge in Computer Vision (TASK-CV), 2014

    2014

  21. [29]

    Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision ...

    2023

  22. [30]

    3d object representations for fine-grained categorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pp.\ 554--561, 2013

    2013

  23. [31]

    Fedmd: Heterogenous federated learning via model distillation

    Daliang Li and Junpu Wang. Fedmd: Heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581, 2019

    1910 arXiv

  24. [32]

    Model-contrastive federated learning

    Qinbin Li, Bingsheng He, and Dawn Song. Model-contrastive federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 10713--10722, 2021

    2021

  25. [33]

    Federated optimization in heterogeneous networks

    Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine Learning and Systems, 2: 0 429--450, 2020

    2020

  26. [34]

    Fedfms: Exploring federated foundation models for medical image segmentation

    Yuxi Liu, Guibo Luo, and Yuesheng Zhu. Fedfms: Exploring federated foundation models for medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp.\ 283--293. Springer, 2024

    2024

  27. [35]

    Fedclip: Fast generalization and personalization for clip in federated learning

    Wang Lu, Xixu Hu, Jindong Wang, and Xing Xie. Fedclip: Fast generalization and personalization for clip in federated learning. IEEE Data Engineering Bulletin, 2023

    2023

  28. [36]

    S. Maji, J. Kannala, E. Rahtu, M. Blaschko, and A. Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151, 2013

    2013 arXiv

  29. [37]

    Communication-efficient learning of deep networks from decentralized data

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics, pp.\ 1273--1282. PMLR, 2017

    2017

  30. [38]

    Nakamura

    T. Nakamura. Military aircraft detection dataset. Available: https://www.kaggle.com/datasets/a2015003713/militaryaircraftdetectiondataset, 2024

    2024

  31. [39]

    Where to begin? on the impact of pre-training and initialization in federated learning

    John Nguyen, Jianyu Wang, Kshitiz Malik, Maziar Sanjabi, and Michael Rabbat. Where to begin? on the impact of pre-training and initialization in federated learning. In Proceedings of the 11th International Conference on Learning Representations (ICLR), 2023. URL https://openre...

    2023

  32. [40]

    Maxime Oquab, Timothée Darcet, Theo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Russell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang-Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Nicolas B...

    2023 arXiv

  33. [41]

    Continual learning with foundation models: An empirical study of latent replay

    Oleksiy Ostapenko, Timothee Lesort, Pau Rodriguez, Md Rifat Arefin, Arthur Douillard, Irina Rish, and Laurent Charlin. Continual learning with foundation models: An empirical study of latent replay. In Conference on lifelong learning agents, pp.\ 60--91. PMLR, 2022

    2022

  34. [42]

    Cats and dogs

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 3498--3505. IEEE, 2012

    2012

  35. [43]

    Pytorch framework [software]

    PyTorch maintainers and contributors . Pytorch framework [software]. Available: https://github.com/pytorch/pytorch, 2016

    2016

  36. [44]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp...

    2021

  37. [45]

    Advances and open challenges in federated foundation models

    Chao Ren, Han Yu, Hongyi Peng, Xiaoli Tang, Bo Zhao, Liping Yi, Alysa Ziying Tan, Yulan Gao, Anran Li, Xiaoxiao Li, et al. Advances and open challenges in federated foundation models. IEEE Communications Surveys & Tutorials, 2025

    2025

  38. [46]

    Fedaux: Leveraging unlabeled auxiliary data in federated learning

    Felix Sattler, Tim Korjakow, Roman Rischke, and Wojciech Samek. Fedaux: Leveraging unlabeled auxiliary data in federated learning. IEEE Transactions on Neural Networks and Learning Systems, 34 0 (9): 0 5531--5543, 2021

    2021

  39. [47]

    Learning across domains and devices: Style-driven source-free domain adaptation in clustered federated learning

    Donald Shenaj, Eros Fan \` , Marco Toldo, Debora Caldarola, Antonio Tavera, Umberto Michieli, Marco Ciccone, Pietro Zanuttigh, and Barbara Caputo. Learning across domains and devices: Style-driven source-free domain adaptation in clustered federated learning. In Proceedings of...

    2023

  40. [48]

    Federated learning in computer vision

    Donald Shenaj, Giulia Rizzoli, and Pietro Zanuttigh. Federated learning in computer vision. IEEE Access, 2023 b

    2023

  41. [49]

    Unsupervised and semi-supervised learning with categorical generative adversarial networks

    Jost Tobias Springenberg. Unsupervised and semi-supervised learning with categorical generative adversarial networks. arXiv preprint arXiv:1511.06390, 2015

    2015 arXiv

  42. [50]

    Mingxing Tan and Quoc V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks, 2020. URL https://arxiv.org/abs/1905.11946

    2020 arXiv

  43. [51]

    Torchvision: Pytorch's computer vision library [software]

    TorchVision maintainers and contributors . Torchvision: Pytorch's computer vision library [software]. Available: https://github.com/pytorch/vision, 2016

    2016

  44. [52]

    Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection

    Grant Van Horn, Steve Branson, Ryan Farrell, Scott Haber, Jessie Barry, Panos Ipeirotis, Pietro Perona, and Serge Belongie. Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. In Proceedings of the...

    2015

  45. [53]

    The caltech-ucsd birds-200-2011 dataset

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. Technical Report CNS-TR-2011-001, Computation & Neural Systems, Caltech, 2011

    2011

  46. [54]

    Tinyvit: Fast pretraining distillation for small vision transformers

    Kan Wu, Jinnian Zhang, Houwen Peng, Mengchen Liu, Bin Xiao, Jianlong Fu, and Lu Yuan. Tinyvit: Fast pretraining distillation for small vision transformers. In European Conference on Computer Vision, pp.\ 68--85, 2022

    2022

  47. [55]

    Exploring the distributed knowledge congruence in proxy-data-free federated distillation

    Zhiyuan Wu, Sheng Sun, Yuwei Wang, Min Liu, Quyang Pan, Junbo Zhang, Zeju Li, and Qingxiang Liu. Exploring the distributed knowledge congruence in proxy-data-free federated distillation. ACM Transactions on Intelligent Systems and Technology, 15 0 (2): 0 1--34, 2024

    2024

  48. [56]

    A large-scale car dataset for fine-grained categorization and verification

    Linjie Yang, Ping Luo, Chen Change Loy, and Xiaoou Tang. A large-scale car dataset for fine-grained categorization and verification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 3973--3981, 2015

    2015

  49. [57]

    Fedgkd: Toward heterogeneous federated learning via global knowledge distillation

    Dezhong Yao, Wanning Pan, Yutong Dai, Yao Wan, Xiaofeng Ding, Chen Yu, Hai Jin, Zheng Xu, and Lichao Sun. Fedgkd: Toward heterogeneous federated learning via global knowledge distillation. IEEE Transactions on Computers, 73 0 (1): 0 3--17, 2023

    2023

  50. [58]

    pfedlora: Model-heterogeneous personalized federated learning with lora tuning

    Liping Yi, Han Yu, Gang Wang, Xiaoguang Liu, and Xiaoxiao Li. pfedlora: Model-heterogeneous personalized federated learning with lora tuning. arXiv preprint arXiv:2310.13283, 2023

    2023 arXiv

  51. [59]

    Decentralized federated learning: A survey and perspective

    Liangqi Yuan, Ziran Wang, Lichao Sun, Philip S Yu, and Christopher G Brinton. Decentralized federated learning: A survey and perspective. IEEE Internet of Things Journal, 11 0 (21): 0 34617--34638, 2024

    2024

  52. [60]

    Fedzkt: Zero-shot knowledge transfer towards resource-constrained federated learning with heterogeneous on-device models

    Lan Zhang, Dapeng Wu, and Xiaoyong Yuan. Fedzkt: Zero-shot knowledge transfer towards resource-constrained federated learning with heterogeneous on-device models. In 2022 IEEE 42nd International Conference on Distributed Computing Systems (ICDCS), pp.\ 928--938. IEEE, 2022

    2022

  53. [61]

    Data-free knowledge distillation for heterogeneous federated learning

    Zhuangdi Zhu, Junyuan Hong, and Jiayu Zhou. Data-free knowledge distillation for heterogeneous federated learning. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp.\ 12878--12889. PMLR, 18--24 J...

    2021

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.