Pith. sign in

REVIEW 3 major objections 7 minor 43 references

Federated Learning with Workload Reduction through Partial Training of Client Models and Entropy-Based Data Selection

T0 review · 3 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that fine-tuning only a pretrained model's upper layers on the highest-entropy 10–50% of local data beats FedAvg and FedProx while using about one-third of the client training time.

desk verdict Entropy-based selection plus partial fine-tuning is a sensible combo, and the random-selection control is the right test, but the small single-run gains do not yet establish that entropy beats random. read the letter →

arxiv 2501.00170 v1 pith:DFOUPAIH submitted 2024-12-30 cs.LG cs.AIcs.DC

Hongrui Shi , Valentin Radu , Po Yang This is my paper · ORCID
classification cs.LGcs.AIcs.DC
keywords federatedlearningentropy-baseddataselectionpartialfine-tuninghardenedsoftmaxnon-IIDworkloadreductionclientefficiencyCIFAR-100
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes FedFT-EDS, a federated learning method that combines two workload reductions: clients fine-tune only the upper layers of a pretrained model, and each round they train only on the local samples whose hardened-softmax prediction entropy is highest. On CIFAR-10, CIFAR-100, and a speech task under non-IID partitions, it reports higher global test accuracy than FedAvg and FedProx while using 10–50% of local data and roughly one-third of client training time. The paper's strongest claim is that selecting 50% of local data by entropy outperforms using all local data, concluding that not all client data is beneficial for federated learning at every round. This matters because it points to data selection, not just model compression, as a way to let resource-constrained devices participate.

What carries the argument

The machinery has two coupled pieces. First, partial fine-tuning: the global model is pretrained on a source domain, the lower feature-extractor parameters $\phi$ are frozen, and clients update only $\theta$, the upper part of the model (in the experiments, layers from layer 3 upward, with the best results when only the classifier is trainable). Second, entropy-based data selection: at each round each client computes the softmax output $\mathbf{p}$ for every local sample, applies a hardened softmax with temperature $\rho = 0.1$ to sharpen the distribution, computes Shannon entropy $H = -\sum_j p_j \log p_j$, and keeps the samples with the highest entropy as the most informative. The temperature-hardening is the mechanism that makes selection work: it compresses confident predictions into very low entropy, so that small gains in confidence produce large drops in entropy and easy samples are filtered out. The server then aggregates only the updated upper parts using FedAvg-style weighting.

What would settle it

Run FedFT-EDS against FedFT-RDS on a benchmark with controlled label noise or outlier contamination, where high-entropy samples are known to include many mislabeled points; if the entropy-selection accuracy gain over random selection disappears or turns negative while the rest of the method is unchanged, the claim that entropy identifies the most beneficial data would be refuted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that federated learning can be made both more accurate and much cheaper by freezing a pretrained feature extractor and fine-tuning only a small trainable upper part on each client, while ranking local samples each round by Shannon entropy of a temperature-hardened softmax output and updating only on the highest-entropy subset. The paper reports that with 10% or 50% of client data, FedFT-EDS beats FedAvg and FedProx trained on all data, improves on random data selection by roughly 0.4 to 3.8 percentage points in its tables, and reaches higher test accuracy per unit of client training time. It also reports that selecting 50% of data by entropy outperforms training on 100% of local data, which the authors interpret as evidence that not all user data is equally beneficial in every round.

Load-bearing premise

The load-bearing premise is that the current model's hardened-softmax prediction entropy marks the local samples whose training will most improve federated performance; if high-entropy samples are mostly mislabeled or outliers, or the temperature tuned on CIFAR-100 does not transfer to other tasks, the reported edge over random selection would vanish.

Editorial extensions

If this is right

  • Federated clients can cut local data to 10–50% and fine-tune only a fraction of the model without losing accuracy; FedFT-EDS reports top accuracy above FedAvg and FedProx in every tested setting.
  • Because only the upper-part parameters are communicated and updated, communication and client compute shrink together, easing the straggler problem and allowing a larger pool of clients to participate each round.
  • Entropy-based selection beats random selection at the same data budget, so the gain comes from which samples are chosen, not merely from training on fewer samples.
  • More data is not always better: with a pretrained model, FedFT-EDS at 50% selection beats FedFT-ALL at 100% data, suggesting that per-round data filtering can remove samples that would hurt or slow learning.
  • Learning efficiency, measured as test accuracy per unit of client training time, is roughly tripled on CIFAR-10 and improved even more on CIFAR-100.
  • The entropy ranking can be recomputed each round for free as part of the local forward pass, so the data-selection overhead is small relative to the training time saved.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The entropy proxy is a heuristic tied to the current global model; a natural extension would be to validate it against gradient-based importance or per-sample loss reduction, which the paper does not do.
  • The benefit of partial fine-tuning likely depends on the pretraining source being close enough to the target that the frozen bottom features remain useful; on far-domain tasks, both the fine-tuning depth and the entropy ranking would need recalibration, a point the paper itself qualifies.
  • In deployment, the extra forward pass for entropy scoring costs time even though it reuses the training pass; a testable extension is to cache or refresh the scores every few rounds instead of every round.
  • The claim that not all data is beneficial concerns per-round usefulness, not permanent data value; a sample excluded in one round may become informative later, so an adaptive selection threshold or a small random exploration component could be explored.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes FedFT-EDS, a federated learning method that combines fine-tuning of only the upper layers of a pretrained global model with per-round entropy-based local data selection. At each round, clients rank their local samples by Shannon entropy computed from a temperature-hardened softmax and train only on the highest-entropy subset; the server aggregates only the trainable upper part. Experiments on CIFAR-10, CIFAR-100, and Google Speech Commands report that FedFT-EDS outperforms FedAvg, FedProx, and random-selection controls while using 10–50% of local data, and that it roughly triples learning efficiency measured as test accuracy per unit of local training time. The paper also claims that entropy-selected 50% data can outperform training on all local data, supporting the thesis that not all client data is beneficial in FL.

Significance. If the reported results are reproducible, the paper makes a useful practical contribution: it demonstrates that a simple, cheap entropy-based selection rule combined with partial fine-tuning can reduce client workload without sacrificing global accuracy, and the construction of FedFT-RDS as a random-selection control is an appropriate way to isolate the effect of entropy selection. The CKA-based analysis of pretraining's effect on client model shift is a thoughtful addition, and the cross-domain speech experiment extends the evidence beyond image classification. The main weakness is statistical: the central entropy-selection gains over random selection are small (0.4–3.8 percentage points) and are reported from single runs without seeds, error bars, or significance tests, while the critical temperature hyperparameter is selected on the same benchmark used for evaluation. The paper does not provide code or reproducibility artifacts, which further limits verification.

major comments (3)
  1. [IV-C, Tables II–III] The central claim that entropy-based selection outperforms random selection is supported only by single-run accuracy differences of 0.4–2.7 points in Table II and 0.6–3.8 points in Table III. No seeds, error bars, or statistical tests are reported, and these margins are small relative to typical run-to-run variance in non-IID federated learning. Please report multiple seeds with confidence intervals and a paired comparison (e.g., Wilcoxon signed-rank or bootstrap) across datasets and heterogeneity levels; without this, the core contribution over FedFT-RDS is not statistically established.
  2. [IV-H3, Fig. 10c] The hardened-softmax temperature rho=0.1 is selected from an ablation on CIFAR-100, which is also the benchmark used for evaluation, and the advantage over FedFT-RDS disappears or reverses for rho values greater than or equal to 1. Because the success of the method is contingent on this hyperparameter, the evaluation should either choose rho on a held-out validation split or demonstrate robustness across datasets and alpha values without per-benchmark tuning. As written, the main result may reflect tuning on the test benchmark rather than a general property of entropy selection.
  3. [IV-E, Table III] The paper's headline insight that 'not all client data is beneficial for FL' rests on the comparison between FedFT-EDS (50%) and FedFT-ALL, where the observed advantage is 0.8–1.8 accuracy points in single runs without variance estimates. Please provide confidence intervals for this specific comparison, and ideally an analysis of why entropy selection removes harmful samples (e.g., label noise or distribution outliers), so that the claimed insight is supported beyond a single-run point estimate.
minor comments (7)
  1. [Abstract and Section IV-C] The abstract and conclusion emphasize using 50% of client data, while the main 10-client comparison in Table II and Figure 5 uses Pds=10%; please state explicitly which configuration is the headline result and clarify the role of each data-selection ratio.
  2. [Fig. 6] The legends for the CIFAR-100 panels include 'MOON', but this baseline is not introduced in Section IV-A or anywhere else in the text; either describe MOON and cite it, or remove it from the figure.
  3. [III-E, Eq. (6)] The text says the hardened softmax is 'parameterized by a temperature rho set to a value greater than 1', which is inconsistent with the following paragraph, where rho is set to a value smaller than 1, and with the experimental value rho=0.1; correct this contradiction.
  4. [IV-C] The text contains typographical fragments such as 'FedAVG-' and 'FedA VG'; please proofread and fix these broken references to FedAvg.
  5. [IV-A] The pretraining phase is described only as pretraining on Small ImageNet 32x32; please specify the pretraining epochs, optimizer, learning rate, data augmentation, and architecture details, as these are needed for reproducibility.
  6. [IV-A] The number of communication rounds is not stated in the setup; Figures 5, 8, and 9 show 50 rounds, but this should be stated explicitly in the experimental protocol, along with the FedProx proximal term mu if it is used.
  7. [IV-G] For the cross-domain Google Speech Commands experiment, please specify the model architecture used for the speech task; it is not clear whether the same Wide ResNet is applied to 1-D audio inputs or whether a different architecture is used.

Circularity Check

0 steps flagged · score 0.0 of 10

No derivational circularity: FedFT-EDS is an empirical combination of partial fine-tuning and entropy-based selection, with the key EDS-vs-RDS comparison tested against a proper random-selection control.

full rationale

The paper's central claims are empirical rather than derived from its own definitions. Entropy-based selection is defined by Equations 2-3 and the hardened softmax in Equation 6, but the claim that this selection improves global accuracy is not obtained from those equations; it is measured against FedFT-RDS, which uses the same partial fine-tuning and the same data fraction with random selection. The pretraining benefit is tested by comparing with and without pretraining, the partial fine-tuning benefit is isolated through FedFT-RDS, and the claim that not all local data is beneficial is assessed by comparing FedFT-EDS with 50% selected data against FedFT-ALL. None of these comparisons reduce to the method's own outputs by construction. There are no self-citations, no imported uniqueness theorems, and no ansatze smuggled in via citation; external references are used only as motivation. The main methodological weakness is that hyperparameters such as the temperature rho=0.1 and the fine-tuning depth are tuned on the same CIFAR-100/Dirichlet benchmarks used for evaluation, and the reported EDS-vs-RDS gains have no error bars or multiple seeds. This affects statistical robustness and generalizability, but it is not circularity of the derivation: no fitted parameter is renamed as a prediction, and no equation is equivalent to its input by definition.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard FL empirical assumptions plus a small set of tuned hyperparameters. There are no new physical or model entities; the hardened softmax is a modification of an existing activation function.

free parameters (5)
  • Hardened softmax temperature rho = 0.1
    Selected by ablation on CIFAR-100 with Diri(0.1) (Section IV-H, Fig. 10c); it directly controls which samples are considered high-entropy and is not set from a held-out validation split.
  • Fine-tuning depth (number of trainable top layers) = from layer 3 up, i.e., classifier-only in best configuration
    Ablation (Fig. 10a) shows classifier-only fine-tuning is best when source and target domains are close; this choice is used in the main experiments and determines both accuracy and workload reduction.
  • Data selection ratio Pds = 10% in Table II, 50% in Table III
    The paper reports both; the abstract highlights 50%, while close-domain results use 10%. The claimed advantage of entropy selection depends on this ratio.
  • Local update epochs E = 5
    Standard hyperparameter, no sensitivity analysis reported.
  • SGD learning rate and momentum = 0.1 and 0.5
    Used for all local updates; no sensitivity analysis shown.
assumptions (5)
  • domain assumption A pretrained model provides a transferable feature extractor for the downstream FL task.
    Invoked in Section III-B and Table I; the method freezes lower layers, so if transfer fails, partial fine-tuning loses its advantage.
  • domain assumption Prediction entropy is a valid proxy for the training value of a sample in federated learning.
    Core heuristic of Section III-C; the paper gives no proof and tests it only through the same benchmark results.
  • domain assumption Samples with high entropy are the most useful to train on, and discarding low-entropy samples does not hurt.
    This underlies the whole selection mechanism and the 'not all data is useful' claim.
  • domain assumption A large source domain, ImageNet Small 32x32, is available to the server for pretraining.
    The method assumes this resource is available, as stated in Section III-B.
  • domain assumption The Dirichlet-partitioned CIFAR-10/100 and Google Speech Commands settings are representative of real non-IID FL conditions.
    Used throughout Section IV to draw general conclusions about FL workload reduction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Federated Learning with Workload Reduction through Partial Training of Client Models and Entropy-Based Data Selection." pith.science (2026). https://pith.science/paper/DFOUPAIH

@misc{pith2026250100170,
  author       = {Pith},
  title        = {Pith review of: Federated Learning with Workload Reduction through Partial Training of Client Models and Entropy-Based Data Selection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DFOUPAIH}},
  note         = {Machine review of arXiv:2501.00170}
}
read the original abstract

With the rapid expansion of edge devices, such as IoT devices, where crucial data needed for machine learning applications is generated, it becomes essential to promote their participation in privacy-preserving Federated Learning (FL) systems. The best way to achieve this desiderate is by reducing their training workload to match their constrained computational resources. While prior FL research has address the workload constrains by introducing lightweight models on the edge, limited attention has been given to optimizing on-device training efficiency through reducing the amount of data need during training. In this work, we propose FedFT-EDS, a novel approach that combines Fine-Tuning of partial client models with Entropy-based Data Selection to reduce training workloads on edge devices. By actively selecting the most informative local instances for learning, FedFT-EDS reduces training data significantly in FL and demonstrates that not all user data is equally beneficial for FL on all rounds. Our experiments on CIFAR-10 and CIFAR-100 show that FedFT-EDS uses only 50% user data while improving the global model performance compared to baseline methods, FedAvg and FedProx. Importantly, FedFT-EDS improves client learning efficiency by up to 3 times, using one third of training time on clients to achieve an equivalent performance to the baselines. This work highlights the importance of data selection in FL and presents a promising pathway to scalable and efficient Federate Learning.

Figures

Figures reproduced from arXiv: 2501.00170 by the authors.

Figure 1
Figure 1. The workflow of our proposed FedFT-EDS. A global pretrained model is split into two parts, one frozen feature extractor [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Heatmaps of the CKA similarity in the scenario of [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 4
Figure 4. Averaged CKA similarity at different layer levels across [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (6 more)
Figure 6
Figure 6. Figure 6: Comparison of learning efficiency, calculated by divid [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 5
Figure 5. Figure 5: Learning curves of FedFT-EDS and baselines, with [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: The learning efficiency of FedFT-EDS, FedFT-RDS, [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: The learning curves of FedFT-EDS, FedFT-RDS and [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Varying the amount of training data selection for [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Ablation study of FedFT-EDS: a) Model part to fine [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 23 canonical work pages

  1. [1]

    Active learning: A survey,

    C. C. Aggarwal, X. Kong, Q. Gu, J. Han, and S. Y . Philip, “Active learning: A survey,” in Data classification . Chapman and Hall/CRC, 2014, pp. 599–634

    work page 2014

  2. [2]

    Federated active learning (f- al): an efficient annotation strategy for federated learning,

    J.-H. Ahn, Y . Ma, S. Park, and C. You, “Federated active learning (f- al): an efficient annotation strategy for federated learning,” IEEE Access, 2024

    work page 2024

  3. [3]

    Fedrolex: Model- heterogeneous federated learning with rolling sub-model extraction,

    S. Alam, L. Liu, M. Yan, and M. Zhang, “Fedrolex: Model- heterogeneous federated learning with rolling sub-model extraction,” Ad- vances in Neural Information Processing Systems , vol. 35, pp. 29 677– 29 690, 2022

    work page 2022

  4. [4]

    The power of ensembles for active learning in image classification,

    W. H. Beluch, T. Genewein, A. N ¨urnberger, and J. M. K ¨ohler, “The power of ensembles for active learning in image classification,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 9368–9377

    2018

  5. [5]

    A closer look at few-shot classification,

    W.-Y . Chen, Y .-C. Liu, Z. Kira, Y .-C. F. Wang, and J.-B. Huang, “A closer look at few-shot classification,” arXiv preprint arXiv:1904.04232, 2019

    arXiv 1904

  6. [6]

    Heterofl: Computation and commu- nication efficient federated learning for heterogeneous clients,

    E. Diao, J. Ding, and V . Tarokh, “Heterofl: Computation and commu- nication efficient federated learning for heterogeneous clients,” arXiv preprint arXiv:2010.01264, 2020

    arXiv 2010

  7. [7]

    Group knowledge transfer: Federated learning of large cnns at the edge,

    C. He, M. Annavaram, and S. Avestimehr, “Group knowledge transfer: Federated learning of large cnns at the edge,” Advances in Neural Information Processing Systems , vol. 33, pp. 14 068–14 080, 2020

    2020

  8. [8]

    Fedml: A research li- brary and benchmark for federated machine learning,

    C. He, S. Li, J. So, X. Zeng, M. Zhang, H. Wang, X. Wang, P. Vepakomma, A. Singh, H. Qiu et al. , “Fedml: A research li- brary and benchmark for federated machine learning,” arXiv preprint arXiv:2007.13518, 2020

    arXiv 2007

Show all 43 references
  1. [9]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , 2015

    2015 arXiv

  2. [10]

    Fjord: Fair and accurate federated learning under heteroge- neous targets with ordered dropout,

    S. Horvath, S. Laskaridis, M. Almeida, I. Leontiadis, S. Venieris, and N. Lane, “Fjord: Fair and accurate federated learning under heteroge- neous targets with ordered dropout,” Advances in Neural Information Processing Systems, vol. 34, pp. 12 876–12 889, 2021

    2021

  3. [11]

    Measuring the effects of non- identical data distribution for federated visual classification,

    T.-M. H. Hsu, H. Qi, and M. Brown, “Measuring the effects of non- identical data distribution for federated visual classification,” arXiv preprint arXiv:1909.06335, 2019

    1909 arXiv

  4. [12]

    Federated semi- supervised learning with inter-client consistency & disjoint learning,

    W. Jeong, J. Yoon, E. Yang, and S. J. Hwang, “Federated semi- supervised learning with inter-client consistency & disjoint learning,” arXiv preprint arXiv:2006.12097 , 2020

    2006 arXiv

  5. [13]

    Multi-class active learning for image classification,

    A. J. Joshi, F. Porikli, and N. Papanikolopoulos, “Multi-class active learning for image classification,” in 2009 ieee conference on computer vision and pattern recognition . IEEE, 2009, pp. 2372–2379

    2009

  6. [14]

    Re-thinking federated active learning based on inter-class diversity,

    S. Kim, S. Bae, H. Song, and S.-Y . Yun, “Re-thinking federated active learning based on inter-class diversity,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 3944–3953

    2023

  7. [15]

    Similarity of neural network representations revisited,

    S. Kornblith, M. Norouzi, H. Lee, and G. Hinton, “Similarity of neural network representations revisited,” in International conference on machine learning . PMLR, 2019, pp. 3519–3529

    2019

  8. [16]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” 2009

    2009

  9. [17]

    Sample-level data selection for federated learning,

    A. Li, L. Zhang, J. Tan, Y . Qin, J. Wang, and X.-Y . Li, “Sample-level data selection for federated learning,” in IEEE INFOCOM 2021-IEEE Conference on Computer Communications . IEEE, 2021, pp. 1–10

    2021

  10. [18]

    Fedmd: Heterogenous federated learning via model distillation,

    D. Li and J. Wang, “Fedmd: Heterogenous federated learning via model distillation,” arXiv preprint arXiv:1910.03581 , 2019

    1910 arXiv

  11. [19]

    A survey on deep active learning: Recent advances and new frontiers,

    D. Li, Z. Wang, Y . Chen, R. Jiang, W. Ding, and M. Okumura, “A survey on deep active learning: Recent advances and new frontiers,” IEEE Transactions on Neural Networks and Learning Systems , 2024

    2024

  12. [20]

    Selecting representative samples from complex biological datasets using k-medoids clustering,

    L. Li, L. Y .-L. Lan, L. Huang, C. Ye, J. Andrade, and P. C. Wilson, “Selecting representative samples from complex biological datasets using k-medoids clustering,” Frontiers in Genetics, vol. 13, p. 954024, 2022

    2022

  13. [21]

    Entropy-based sampling approaches for multi- class imbalanced problems,

    L. Li, H. He, and J. Li, “Entropy-based sampling approaches for multi- class imbalanced problems,” IEEE Transactions on Knowledge and Data Engineering, vol. 32, no. 11, pp. 2159–2170, 2019

    2019

  14. [22]

    Federated optimization in heterogeneous networks,

    T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networks,” Proceedings of Machine learning and systems , vol. 2, pp. 429–450, 2020

    2020

  15. [23]

    Ensemble distillation for robust model fusion in federated learning,

    T. Lin, L. Kong, S. U. Stich, and M. Jaggi, “Ensemble distillation for robust model fusion in federated learning,” Advances in Neural Information Processing Systems , vol. 33, pp. 2351–2363, 2020

    2020

  16. [24]

    No one left behind: Inclusive federated learning over heterogeneous devices,

    R. Liu, F. Wu, C. Wu, Y . Wang, L. Lyu, H. Chen, and X. Xie, “No one left behind: Inclusive federated learning over heterogeneous devices,” in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , 2022, pp. 3398–3406

    2022

  17. [25]

    Influence selection for active learning,

    Z. Liu, H. Ding, H. Zhong, W. Li, J. Dai, and C. He, “Influence selection for active learning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 9274–9283

    2021

  18. [26]

    Latent structured active learning,

    W. Luo, A. Schwing, and R. Urtasun, “Latent structured active learning,” Advances in Neural Information Processing Systems , vol. 26, 2013

    2013

  19. [27]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in AIStat. PMLR, 2017

    2017

  20. [28]

    Prioritized training on points that are learnable, worth learning, and not yet learnt,

    S. Mindermann, J. M. Brauner, M. T. Razzak, M. Sharma, A. Kirsch, W. Xu, B. H ¨oltgen, A. N. Gomez, A. Morisot, S. Farquhar et al. , “Prioritized training on points that are learnable, worth learning, and not yet learnt,” in International Conference on Machine Learning . PMLR,...

    2022

  21. [29]

    Is your data relevant?: Dynamic selection of relevant data for federated learning,

    L. Nagalapatti, R. S. Mittal, and R. Narayanam, “Is your data relevant?: Dynamic selection of relevant data for federated learning,” in Proceed- ings of the AAAI Conference on Artificial Intelligence , vol. 36, no. 7, 2022, pp. 7859–7867

    2022

  22. [30]

    Enhancing hetero- geneous federated learning with knowledge extraction and multi-model fusion,

    D. P. Nguyen, S. Yu, J. P. Mu ˜noz, and A. Jannesari, “Enhancing hetero- geneous federated learning with knowledge extraction and multi-model fusion,” in Proceedings of the SC’23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Ana...

    2023

  23. [31]

    Where to begin? on the impact of pre-training and initialization in federated learning,

    J. Nguyen, J. Wang, K. Malik, M. Sanjabi, and M. Rabbat, “Where to begin? on the impact of pre-training and initialization in federated learning,” arXiv preprint arXiv:2206.15387 , 2022

    2022 arXiv

  24. [32]

    Do wide and deep networks learn the same things? uncovering how neural network representations vary with width and depth,

    T. Nguyen, M. Raghu, and S. Kornblith, “Do wide and deep networks learn the same things? uncovering how neural network representations vary with width and depth,” arXiv preprint arXiv:2010.15327 , 2020

    2010 arXiv

  25. [33]

    Entropy to mitigate non-iid data problem on federated learning for the edge intelligence environment,

    F. C. Orlandi, J. C. Dos Anjos, J. F. d. P. Santana, V . R. Leithardt, and C. F. Geyer, “Entropy to mitigate non-iid data problem on federated learning for the edge intelligence environment,” IEEE Access, 2023

    2023

  26. [34]

    Fedentropy: Information- entropy-aided training optimization of semi-supervised federated learn- ing,

    D. Qian, Y . Cui, Y . Fu, F. Liu, and T. Wei, “Fedentropy: Information- entropy-aided training optimization of semi-supervised federated learn- ing,” Journal of Systems Architecture , vol. 137, p. 102851, 2023

    2023

  27. [35]

    A survey of deep active learning,

    P. Ren, Y . Xiao, X. Chang, P.-Y . Huang, Z. Li, B. B. Gupta, X. Chen, and X. Wang, “A survey of deep active learning,” ACM computing surveys (CSUR), vol. 54, no. 9, pp. 1–40, 2021

    2021

  28. [36]

    Active hidden markov models for information extraction,

    T. Scheffer, C. Decomain, and S. Wrobel, “Active hidden markov models for information extraction,” in International symposium on intelligent data analysis. Springer, 2001, pp. 309–318

    2001

  29. [37]

    Active learning literature survey,

    B. Settles, “Active learning literature survey,” 2009

    2009

  30. [38]

    A survey on active learning: State-of-the- art, practical challenges and research directions,

    A. Tharwat and W. Schenck, “A survey on active learning: State-of-the- art, practical challenges and research directions,” Mathematics, vol. 11, no. 4, p. 820, 2023

    2023

  31. [39]

    Rethink- ing few-shot image classification: a good embedding is all you need?

    Y . Tian, Y . Wang, D. Krishnan, J. B. Tenenbaum, and P. Isola, “Rethink- ing few-shot image classification: a good embedding is all you need?” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIV 16 . Springer, 2020, p...

    2020

  32. [40]

    Speech commands: A dataset for limited-vocabulary speech recognition,

    P. Warden, “Speech commands: A dataset for limited-vocabulary speech recognition,” arXiv preprint arXiv:1804.03209 , 2018

    2018 arXiv

  33. [41]

    Towards sustainable learning: Coresets for data-efficient deep learning,

    Y . Yang, H. Kang, and B. Mirzasoleiman, “Towards sustainable learning: Coresets for data-efficient deep learning,” in International Conference on Machine Learning . PMLR, 2023, pp. 39 314–39 330

    2023

  34. [42]

    Multiple instance active learning for object detection,

    T. Yuan, F. Wan, M. Fu, J. Liu, S. Xu, X. Ji, and Q. Ye, “Multiple instance active learning for object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 5330–5339

    2021

  35. [43]

    Wide residual networks,

    S. Zagoruyko and N. Komodakis, “Wide residual networks,” arXiv preprint arXiv:1605.07146, 2016

    2016 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.