TallyTrain is a hard-label distillation protocol for federated learning that uses argmax transmission and optional sparse merges to match soft-label performance at up to 1000x lower communication cost.
Federated optimization in heterogeneous networks
10 Pith papers cite this work. Polarity classification is still indexing.
10
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
verdicts
UNVERDICTED 10roles
background 1polarities
background 1representative citing papers
Framework quantifies intra- and inter-client memorization in FL LLMs, finding higher intra-client memorization influenced by decoding strategies, prefix length, and FL algorithms.
Proposes federated adaptive optimizers (FedAdagrad, FedAdam, FedYogi) with convergence analysis for non-convex objectives under data heterogeneity and reports empirical gains over FedAvg.
Echelon enables auditable aggregate-only adaptation of language models across privacy boundaries by training locally and sharing only boundary-level aggregates, achieving competitive performance in 1B LoRA experiments.
Proactive client selection in federated learning via differentially private mutual information and simulated annealing to optimize Potential Federation Loss for utility and fairness.
Non-identical data distributions degrade federated averaging accuracy on visual classification, but server momentum raises CIFAR-10 accuracy from 30.1% to 76.9% in the most skewed regimes.
XAI-SOH-FL extends SOH-FL with adaptive gamma via Bayesian optimization and SHAP interpretability, reporting 94.12% accuracy and 0.92 F1 on CICIDS2017 while converging faster than baseline.
FedAvg matches centralized training accuracy on mammography data split by breast density heterogeneity, showing standard FL can handle this clinical variation without special fixes.
FedProx outperforms FedAvg for deeper models under data heterogeneity, BSP reaches near-centralized accuracy at high communication cost, and LeNet gives the best accuracy-communication trade-off on the UC Merced dataset.
Position paper claiming that distributed training across massive edge devices can overcome data depletion and centralized compute monopolies in LLM scaling.
citing papers explorer
-
TallyTrain: Communication-Efficient Federated Distillation
TallyTrain is a hard-label distillation protocol for federated learning that uses argmax transmission and optional sparse merges to match soft-label performance at up to 1000x lower communication cost.
-
Exploring Cross-Client Memorization of Training Data in Large Language Models for Federated Learning
Framework quantifies intra- and inter-client memorization in FL LLMs, finding higher intra-client memorization influenced by decoding strategies, prefix length, and FL algorithms.
-
Adaptive Federated Optimization
Proposes federated adaptive optimizers (FedAdagrad, FedAdam, FedYogi) with convergence analysis for non-convex objectives under data heterogeneity and reports empirical gains over FedAvg.
-
Echelon: Auditable Aggregate-Only Language-Model Adaptation Across Privacy Boundaries
Echelon enables auditable aggregate-only adaptation of language models across privacy boundaries by training locally and sharing only boundary-level aggregates, achieving competitive performance in 1B LoRA experiments.
-
Choose Wisely and Privately: Proactive Client Selection for Fair and Efficient Federated Learning
Proactive client selection in federated learning via differentially private mutual information and simulated annealing to optimize Potential Federation Loss for utility and fairness.
-
Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
Non-identical data distributions degrade federated averaging accuracy on visual classification, but server momentum raises CIFAR-10 accuracy from 30.1% to 76.9% in the most skewed regimes.
-
XAI-SOH-FL: Enhancing SOH-FL with Adaptive Aggregation and Explainable AI for Intrusion Detection in Heterogeneous IoT
XAI-SOH-FL extends SOH-FL with adaptive gamma via Bayesian optimization and SHAP interpretability, reporting 94.12% accuracy and 0.92 F1 on CICIDS2017 while converging faster than baseline.
-
Evaluating Federated Learning approaches for mammography under breast density heterogeneity
FedAvg matches centralized training accuracy on mammography data split by breast density heterogeneity, showing standard FL can handle this clinical variation without special fixes.
-
The Impact of Federated Learning on Distributed Remote Sensing Archives
FedProx outperforms FedAvg for deeper models under data heterogeneity, BSP reaches near-centralized accuracy at high communication cost, and LeNet gives the best accuracy-communication trade-off on the UC Merced dataset.
-
Will LLMs Scaling Hit the Wall? Breaking Barriers via Distributed Resources on Massive Edge Devices
Position paper claiming that distributed training across massive edge devices can overcome data depletion and centralized compute monopolies in LLM scaling.