REVIEW 3 major objections 6 minor 1 cited by
E-3SFC: Communication-Efficient Federated Learning with Double-way Features Synthesizing
T0 review · 3 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read Compressing gradients into tiny synthetic features — regenerated by the model's own backward pass — lets federated learning cut communication by up to 111.6× while matching or beating uncompressed accuracy.
desk verdict The empirical package is broad and the extension is real, but the convergence theorems lean on an unproved bound (Assumption V.5) that is the only bridge between the compressor and the rates. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the 3SFC compressor and its compression–decompression symmetry. Compression means solving Eq. 9: a budget-constrained search over trainable synthetic features $D_{\mathrm{syn}}$ (plus a scalar $s$) to maximize the cosine similarity between the gradient the frozen model produces at those features and the target gradient; decompression is a single backward pass through the same frozen model. Error feedback (Eq. 6), inherited from sparsified SGD with memory, feeds the residual $\epsilon_t^i$ into the next compression target so that compression error does not accumulate unchecked. The E-3SFC extensions compress the server-to-client download through the same mechanism (Eq. 11, with clients reconstructing the global model via Eq. 12) and add a budget scheduler that solves Eq. 14 offline, shifting more of a fixed total budget to early rounds where compression efficiency is highest. The convergence analysis treats the compressor as a noisy oracle and expresses the error terms through the variance bound $\kappa_S^2 + \sigma_F^2$.
What would settle it
Measure the squared compression error with error feedback at every communication round on the paper's own MNIST+MLP setup and check whether it stays bounded by $\eta^2\kappa_S^2$ for a constant $\kappa_S$ that does not grow with the round number — Figure 8 reports only cosine efficiency, which is not the quantity the theorems bound. On the double-way scheme, also track the reconstruction residual of Eq. 12 over 200 rounds; if it drifts or amplifies, the approximate decompression is unstable.
Extended reading notes
Core claim
The central claim is that gradient compression in federated learning can be inverted: the model itself becomes the decompressor, and the compressed payload is a set of synthetic features. With the global model $w_t$ and the loss $F$ frozen as training priors, the 3SFC compressor solves the objective of Eq. 9 — find budget-constrained synthetic features $D_{\mathrm{syn}}$ whose gradient $\nabla_w F(D_{\mathrm{syn}}, w_t)$ points as nearly as possible in the direction of the true accumulated gradient $g_t^i + \epsilon_t^i$ (including the error-feedback residual) — and scales the result by $s_t^i$ via Eq. 8. The server aggregates the reconstructed gradients $s_t^i \nabla_w F(D_{\mathrm{syn}}^t, w_t)$, and clients or the server regenerate updates in one backward pass. The paper argues that this representation carries lower compression error per transmitted byte than sparsification or quantization, that the single simulation step avoids the multi-step optimization collapse of earlier synthetic-data compressors, and that under Assumptions V.1–V.5 the scheme converges linearly with aggregation noise in the strongly convex case and sub-linearly in the non-convex case, with rates $O(1/R)$/$O(1/R^2)$ and $O(1/R^{1/2})$/$O(1/R^{2/3})$ respectively, controlled by a compression-variance constant $\kappa_S^2$.
Load-bearing premise
The convergence theorems assume (Assumption V.5) that the compression error after $S$ inner optimization steps is bounded by a fixed constant $\kappa_S$ — $\mathbb{E}\|\eta\nabla F(w_t^i) - g_t^i\|^2 \le \eta^2\kappa_S^2$ — but this bound is never derived from the compressor's own optimization (Eq. 9) or from $S$, so if the real compressor does not satisfy it, the proved rates do not apply.
Editorial extensions
If this is right
- If the claimed rates hold, 3SFC stays within the convergence class of sparsified gradient methods — linear under strong convexity with aggregation noise, sub-linear otherwise — while transmitting orders of magnitude fewer bytes.
- Communication budget and local computation become interchangeable: increasing the inner optimization steps $S$ shrinks the variance term $\kappa_S^2$ that sets the convergence radius, so a fixed bandwidth can be traded for accuracy by spending more compute.
- The scheduler is solved before training begins, so the relaxed cumulative budget (Eq. 5) costs no extra online signalling and assigns more budget to early rounds, where compression efficiency is highest.
- At equal compression ratio, the paper reports 3SFC consistently above top-k sparsification with error feedback (up to 13.4% higher accuracy), which implies that each transmitted synthetic bit carries more usable gradient information than a sparsified gradient bit.
- Because downloads are compressed as well, the 111.6× traffic reduction claimed for E-3SFC comes from both FL communication phases, not just the upload.
Reading between the lines
- Beyond the paper: if the missing derivation of $\kappa_S$ from $S$ were supplied, the theory would produce a concrete design rule — how many inner optimization steps are needed to push compression noise below stochastic-gradient noise — which the paper currently states only qualitatively.
- Beyond the paper: the compressor reuses the machinery of gradient-inversion attacks (the paper notes the resemblance and argues safety because clients invert their own gradients); a natural next test is whether membership or attribute inference from the transmitted synthetic features is harder than from compressed gradients.
- Beyond the paper: the compression-efficiency metric (cosine between decompressed and raw gradients, Eq. 20) falls over training rounds even with error feedback; the budget scheduler is designed to counteract that, and a measurable prediction is that late-round per-byte efficiency under E-3SFC stays above that of constant-budget 3SFC.
- Beyond the paper: the error-feedback residual is never transmitted and Eq. 12 reconstruction is approximate, so an open benchmark is whether the reconstruction drift compounds over very long training horizons; tracking that residual would settle the stability of double-way compression.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes E-3SFC, a communication-efficient federated learning method whose core compressor, 3SFC, represents client gradients by synthetic features optimized through a short back-propagation loop. The claimed contributions are threefold: a single-step synthetic-features compressor with error feedback, a double-way compression scheme that also compresses model downloads, and a communication-budget scheduler. The paper provides convergence analyses for 3SFC under strongly convex and non-convex assumptions, claiming linear and sub-linear rates with aggregation noise, and reports experiments over six datasets and six models showing large communication reductions at comparable or better accuracy than several baselines. The main theoretical claim, however, is that convergence rates follow from an assumed bound on the compression error rather than from a property proved for the proposed compressor.
Significance. If the theoretical claims were established, this would be a valuable contribution: a gradient compression method with reported communication reductions of two to three orders of magnitude and broad empirical validation. The empirical work is extensive, covering six datasets, six models, 10-60 clients, ablations, budget schedulers, and compression-efficiency visualizations, and the paper points to a public code repository. The novel perspective of using the model itself as a decompressor is interesting and the experiments suggest practical promise. However, the central theoretical result is not established as stated: the convergence rates are conditional on an assumption about the compressor's error that is never derived, and the proof of the key lemma applies that assumption inconsistently. The paper's significance therefore rests mostly on its empirical findings until the theory is repaired or appropriately qualified.
major comments (3)
- [Section V, Assumption V.5 and Eq. (9)] The convergence theorems are conditional on an assumed bound, not on a property derived for 3SFC. Assumption V.5 asserts E||η∇F(w_t^i) − g_t^i||^2 ≤ η^2κ_S^2, but κ_S is never expressed in terms of the inner-loop objective Eq. (9), the optimization horizon S, the budget B, or the model's loss landscape. Theorems V.3 and V.4 then contain κ_S only through this assumption, so the argument proves that any compressor satisfying the bound converges at those rates; it does not establish that the optimization in Eq. (9) produces such a bound. Remark V.3.1 and the Insight paragraph state that S can be chosen to keep κ_S at the level of σ_F^2, but this is an instruction rather than a consequence of Eq. (9). This is load-bearing for the paper's central theoretical claim and needs either a derivation of κ_S from Eq. (9) or a substantially weakened claim.
- [Section IV-B, Eq. (12), and Section V] The formal analysis does not cover the double-way compression used in E-3SFC, which is the paper's titular algorithm. The server compresses the updated global model via Eq. (11), and clients reconstruct it via Eq. (12): w_t = w_{t−1} − s_t∇_{w_{t−1}}F(D_t_syn, w_{t−1}). This reconstruction is exact only if the compressed residual perfectly represents w_t − w_{t−1} + ϵ_t, which is not guaranteed and whose error is never bounded. Assumption V.5 and Lemma V.2 concern only the upload compressor applied to η∇F(w_t^i) + ϵ; they do not bound the download-compression error. Consequently, Theorems V.3 and V.4 do not provide convergence guarantees for Algorithm 1 as presented. Please either extend the analysis to the download compressor or state explicitly that the theoretical results apply to 3SFC without double-way compression.
- [Appendix, Lemma V.2 proof, Eqs. (30)-(32)] There is a mismatch between Assumption V.5 and its use in the proof of Lemma V.2. Assumption V.5 bounds E||η∇F(w_t^i) − g_t^i||^2 with the full local gradient ∇F, but Eq. (30) applies it to the stochastic gradient ∇F̃(w^{t+k}) in the first term, yielding E||g − η∇F̃||^2 ≤ η^2κ_S^2. The assumption as stated does not control this quantity; a bound would require an additional relationship between ∇F and ∇F̃ beyond Assumption V.2, which only controls E||∇F̃ − ∇F||^2. The same display also has missing norm symbols on the second and third terms. Since Lemma V.2 is the only route from the compressor to Theorems V.3 and V.4, this gap is load-bearing for the claimed convergence rates.
minor comments (6)
- [Eq. (9)] The denominator in the cosine term is printed as ||∇w_t + ϵ_t^i||; it should almost certainly be ||g_t^i + ϵ_t^i|| or ||∇_{w_t}F(D_syn,i, w_t)||. Please correct the typo, as it obscures the stated equivalence with Eq. (7).
- [Eq. (14)] The regularization term τ|H−B|/(H−B) is undefined when H=B, which is the constant-budget case used as a baseline in Table VII. Please specify a convention for this case or replace the term with a well-defined penalty.
- [Eq. (19) and Tables III-IV] The compression terminology is inconsistent. Eq. (19) defines Comp. Rate = Comp. Size / Uncomp. Size = 1 / Comp. Ratio, but the text and tables call a value such as 250× a 'compression ratio' and use 'higher compression ratio' to mean more compression. Please unify the definitions to avoid confusion.
- [Corollary V.3.1] The phrase 'quadratic convergence with total local epochs R' appears to mean a quadratic speed-up O(1/R^2); as written it conflicts with the linear convergence rate stated in Theorem V.3. Please rephrase to avoid a literal reading that contradicts the theorem.
- [Appendix, proof of Lemma V.1, Eq. (22)] The displayed expansion of term A contains a typo: the second inner product is written as −2⟨w_{t_a} − w∗, η̃⟩ and appears to be missing the gradient factor, so the step from Eq. (21) to Eq. (23) is difficult to follow.
- [Tables III and VI] Several entries appear to be mistyped, for example the 60-client E-3SFC entry for RegNet on Cifar10 reads '42.47 57.6×' rather than the '757.6×' that would match the corresponding column. Please proofread all table cells.
Circularity Check
Convergence claims are conditional on Assumption V.5: the rate bounds are expressed through κ_S, the same compression-variance constant assumed rather than derived from 3SFC's S-step optimization.
-
self definitional
[Section V, Assumption V.5; Lemma V.2; Theorems V.3 and V.4; Remark V.3.1; Insight paragraph]
"Assumption V.5 (Unbiased Compression Optimization Variance Bound). The expectation of the difference between local gradients and compressed local gradients is upper-bounded, i.e., E||η∇F (wt i) − gt i||2 ≤ η2κ2 S ... Theorem V.3 ... O(F ( ¯w) − F (w∗)) = O(μF e−˜ημF R 2 ∆0) + O( κ2 S + σ2 F μF ) + ..."
The convergence bounds in Theorems V.3 and V.4 are written directly in terms of κ_S², the very quantity introduced in Assumption V.5 as an upper bound on 3SFC's compression residual. κ_S is never derived from the cosine-based objective in Eq. 9, from the number of inner-loop steps S, or from the communication budget B. Instead, Remark V.3.1 and the Insight paragraph assert that 'S is chosen to maintain the same level variance of compression' and that κ_S 'can be effectively reduced by increasing S,' without proof. Thus the claimed linear and sub-linear convergence of 3SFC is not a derived consequence of the compressor; it is an assumed bound.
full rationale
The paper's central theoretical claims about 3SFC's convergence rates reduce to Assumption V.5. The proof chain is: Lemma V.1 bounds local shift in terms of the compression residual ||g−η∇F||², Lemma V.2 bounds that residual using κ_S² and σ_F², and Theorems V.3 and V.4 then present rates containing κ_S²+σ_F² as an additive term. Critically, κ_S² is never bounded in terms of the algorithm's actual optimization dynamics (Eq. 9's cosine objective, the number of inner steps S, or the budget B). The remark after Corollary V.3.1 simply instructs that S be chosen to keep κ_S² at the same level as σ_F², and the 'Insight' asserts without proof that increasing S reduces κ_S. Consequently, the theorem is conditional on an unverified property of the compressor; it would hold for any compressor satisfying that property, including ones that never run Eq. 9. This is a partial circularity: the prediction of linear/sub-linear convergence is not established for the specific 3SFC algorithm. The empirical comparisons against DGC, STC, etc. are independent and not circular. Because the core theoretical result rests on an assumed bound that is not shown to be satisfied, the score is 5 for a moderately circular, assumption-driven analysis.
Assumptions & free parameters
free parameters (3)
- kappa_S
- tau =
3.0
- lambda =
0
assumptions (7)
- standard math Assumption V.1: bounded local variance
- standard math Assumption V.2: bounded approximated gradient variance
- standard math Assumption V.3: bounded global optimal variance
- standard math Assumption V.4: expected smoothness
- ad hoc to paper Assumption V.5: bounded compression optimization variance
- domain assumption Exact global-model reconstruction in double-way compression
- ad hoc to paper Inner-loop optimization (Eq. 9) converges in S steps
Cite this review
Pith. "Pith review of E-3SFC: Communication-Efficient Federated Learning with Double-way Features Synthesizing." pith.science (2026). https://pith.science/paper/GQQAHG47
@misc{pith2026250203092,
author = {Pith},
title = {Pith review of: E-3SFC: Communication-Efficient Federated Learning with Double-way Features Synthesizing},
year = {2026},
howpublished = {\url{https://pith.science/paper/GQQAHG47}},
note = {Machine review of arXiv:2502.03092}
}
read the original abstract
The exponential growth in model sizes has significantly increased the communication burden in Federated Learning (FL). Existing methods to alleviate this burden by transmitting compressed gradients often face high compression errors, which slow down the model's convergence. To simultaneously achieve high compression effectiveness and lower compression errors, we study the gradient compression problem from a novel perspective. Specifically, we propose a systematical algorithm termed Extended Single-Step Synthetic Features Compressing (E-3SFC), which consists of three sub-components, i.e., the Single-Step Synthetic Features Compressor (3SFC), a double-way compression algorithm, and a communication budget scheduler. First, we regard the process of gradient computation of a model as decompressing gradients from corresponding inputs, while the inverse process is considered as compressing the gradients. Based on this, we introduce a novel gradient compression method termed 3SFC, which utilizes the model itself as a decompressor, leveraging training priors such as model weights and objective functions. 3SFC compresses raw gradients into tiny synthetic features in a single-step simulation, incorporating error feedback to minimize overall compression errors. To further reduce communication overhead, 3SFC is extended to E-3SFC, allowing double-way compression and dynamic communication budget scheduling. Our theoretical analysis under both strongly convex and non-convex conditions demonstrates that 3SFC achieves linear and sub-linear convergence rates with aggregation noise. Extensive experiments across six datasets and six models reveal that 3SFC outperforms state-of-the-art methods by up to 13.4% while reducing communication costs by 111.6 times. These findings suggest that 3SFC can significantly enhance communication efficiency in FL without compromising model performance.
Figures
Figures from the paper (4 more)
Forward citations
Cited by 1 Pith paper
-
Discerning and quantifying high frequency activities in EEG under normal and epileptic conditions
The authors claim that gamma-band EEG activity at roughly 50 Hz and 76 Hz shows distinct bi-stable dynamics in epilepsy, enabling 94-95% accuracy in separating normal from ictal states.
Reference graph
Works this paper leans on
-
[33]
Y . Zhou, M. Shi, Y . Li, Y . Sun, Q. Ye, and J. Lv, “Communication- efficient federated learning with single-step synthetic features compres- sor for faster convergence,” in Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision , 2023, pp. 5031–5040
work page 2023
-
[1]
Communication-efficient learning of deep networks from decentralized data,
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics . PMLR, 2017, pp. 1273– 1282
2017
-
[2]
A review of applications in federated learning,
L. Li, Y . Fan, M. Tse, and K.-Y . Lin, “A review of applications in federated learning,” Computers & Industrial Engineering , vol. 149, p. 106854, 2020
work page 2020
-
[3]
Large scale distributed deep networks,
J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, M. Mao, M. Ran- zato, A. Senior, P. Tucker, K. Yang et al., “Large scale distributed deep networks,” Advances in neural information processing systems , vol. 25, 2012
2012
-
[4]
Demystifying parallel and distributed deep learning: An in-depth concurrency analysis,
T. Ben-Nun and T. Hoefler, “Demystifying parallel and distributed deep learning: An in-depth concurrency analysis,” ACM Computing Surveys (CSUR), vol. 52, no. 4, pp. 1–43, 2019
work page 2019
-
[5]
Prior: Personalized prior for reactivating the information overlooked in federated learning,
M. Shi, Y . Zhou, K. Wang, H. Zhang, S. Huang, Q. Ye, and J. Lv, “Prior: Personalized prior for reactivating the information overlooked in federated learning,” Advances in Neural Information Processing System , 2023
work page 2023
-
[6]
Defta: A plug-and-play peer-to-peer decentralized federated learning framework,
Y . Zhou, M. Shi, Y . Tian, Q. Ye, and J. Lv, “Defta: A plug-and-play peer-to-peer decentralized federated learning framework,” Information Sciences, vol. 670, p. 120582, 2024
work page 2024
-
[7]
Robust and communication-efficient federated learning from non-iid data,
F. Sattler, S. Wiedemann, K.-R. M ¨uller, and W. Samek, “Robust and communication-efficient federated learning from non-iid data,” IEEE transactions on neural networks and learning systems , vol. 31, no. 9, pp. 3400–3413, 2019
2019
Show all 63 references
-
[8]
Communication-efficient federated learning with compensated overlap-fedavg,
Y . Zhou, Q. Ye, and J. Lv, “Communication-efficient federated learning with compensated overlap-fedavg,” IEEE Transactions on Parallel and Distributed Systems, vol. 33, no. 1, pp. 192–205, 2021
2021
-
[9]
Advances and open problems in federated learning,
P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings et al. , “Advances and open problems in federated learning,” Foundations and Trends® in Machine Learning , vol. 14, no. 1–2, pp. 1–210, 2021
2021
-
[10]
Language mod- els are few-shot learners,
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language mod- els are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020
1901
-
[11]
An empirical study of parameter efficient fine-tuning on vision-language pre-train model,
Y . Tian, M. Yang, Y . Li, D. Liu, X. Ren, X. Peng, and J. Lv, “An empirical study of parameter efficient fine-tuning on vision-language pre-train model,” in IEEE International Conference on Multimedia and Expo, ICME 2024 , pp. 1–6
2024
-
[12]
UNITE: multitask learning with sufficient feature for dense prediction,
Y . Tian, Y . Lin, Q. Ye, J. Wang, X. Peng, and J. Lv, “UNITE: multitask learning with sufficient feature for dense prediction,” IEEE Trans. Syst. Man Cybern. Syst. , vol. 54, no. 8, pp. 5012–5024, 2024
2024
-
[13]
Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,
W. Fedus, B. Zoph, and N. Shazeer, “Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,” 2021
2021
-
[14]
Towards federated learning at scale: System design,
K. Bonawitz, H. Eichner, W. Grieskamp, D. Huba, A. Ingerman, V . Ivanov, C. Kiddon, J. Kone ˇcn`y, S. Mazzocchi, B. McMahan et al. , “Towards federated learning at scale: System design,” Proceedings of Machine Learning and Systems , vol. 1, pp. 374–388, 2019
2019
-
[15]
Scalable distributed dnn training using commodity gpu cloud computing,
N. Strom, “Scalable distributed dnn training using commodity gpu cloud computing,” in Sixteenth annual conference of the international speech communication association, 2015
2015
-
[16]
Deep gradient compression: Reducing the communication bandwidth for distributed training,
Y . Lin, S. Han, H. Mao, Y . Wang, and W. J. Dally, “Deep gradient compression: Reducing the communication bandwidth for distributed training,” arXiv preprint arXiv:1712.01887 , 2017
2017 arXiv
-
[17]
Gradient sparsification for communication-efficient distributed optimization,
J. Wangni, J. Wang, J. Liu, and T. Zhang, “Gradient sparsification for communication-efficient distributed optimization,” Advances in Neural Information Processing Systems , vol. 31, 2018
2018
-
[18]
Qsgd: Communication-efficient sgd via gradient quantization and encoding,
D. Alistarh, D. Grubic, J. Li, R. Tomioka, and M. V ojnovic, “Qsgd: Communication-efficient sgd via gradient quantization and encoding,” Advances in neural information processing systems , vol. 30, 2017
2017
-
[19]
signsgd: Compressed optimisation for non-convex problems,
J. Bernstein, Y .-X. Wang, K. Azizzadenesheli, and A. Anandkumar, “signsgd: Compressed optimisation for non-convex problems,” in Inter- national Conference on Machine Learning. PMLR, 2018, pp. 560–569
2018
-
[20]
Error feedback fixes signsgd and other gradient compression schemes,
S. P. Karimireddy, Q. Rebjock, S. Stich, and M. Jaggi, “Error feedback fixes signsgd and other gradient compression schemes,” in International Conference on Machine Learning . PMLR, 2019, pp. 3252–3261
2019
-
[21]
Knowledge distillation: A survey,
J. Gou, B. Yu, S. J. Maybank, and D. Tao, “Knowledge distillation: A survey,” International Journal of Computer Vision , vol. 129, pp. 1789– 1819, 2021
2021
-
[22]
Federated learning via synthetic data,
J. Goetz and A. Tewari, “Federated learning via synthetic data,” arXiv preprint arXiv:2008.04489, 2020
2008 arXiv
-
[23]
Communication-efficient federated learning based on compressed sensing,
C. Li, G. Li, and P. K. Varshney, “Communication-efficient federated learning based on compressed sensing,”IEEE Internet of Things Journal, vol. 8, no. 20, pp. 15 531–15 541, 2021
2021
-
[24]
Fedsynth: Gradient compression via synthetic data in federated learning,
S. Hu, J. Goetz, K. Malik, H. Zhan, Z. Liu, and Y . Liu, “Fedsynth: Gradient compression via synthetic data in federated learning,” arXiv preprint arXiv:2204.01273, 2022
2022 arXiv
-
[25]
Sparse communication for distributed gradient descent,
A. F. Aji and K. Heafield, “Sparse communication for distributed gradient descent,” arXiv preprint arXiv:1704.05021 , 2017
2017 arXiv
-
[26]
z-signfedavg: A unified stochastic sign-based compression for federated learning,
Z. Tang, Y . Wang, and T.-H. Chang, “z-signfedavg: A unified stochastic sign-based compression for federated learning,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 14, 2024, pp. 15 301–15 309
2024
-
[27]
Optimization methods for large- scale machine learning,
L. Bottou, F. E. Curtis, and J. Nocedal, “Optimization methods for large- scale machine learning,” SIAM review, vol. 60, no. 2, pp. 223–311, 2018
2018
-
[28]
Deep leakage from gradients,
L. Zhu, Z. Liu, and S. Han, “Deep leakage from gradients,” Advances in neural information processing systems , vol. 32, 2019
2019
-
[29]
Sparsified sgd with memory,
S. U. Stich, J.-B. Cordonnier, and M. Jaggi, “Sparsified sgd with memory,” Advances in Neural Information Processing Systems , vol. 31, 2018
2018
-
[30]
A theoretical study of dataset distillation,
Z. Izzo and J. Zou, “A theoretical study of dataset distillation,” in NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning , 2023
2023
-
[31]
An image is worth 16x16 words: Transformers for image recognition at scale,
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929 , 2020
2010 arXiv
-
[32]
The convergence of sparsified gradient methods,
D. Alistarh, T. Hoefler, M. Johansson, N. Konstantinov, S. Khirirat, and C. Renggli, “The convergence of sparsified gradient methods,” Advances in Neural Information Processing Systems , vol. 31, 2018
2018
-
[34]
Communication-efficient federated learning,
M. Chen, N. Shlezinger, H. V . Poor, Y . C. Eldar, and S. Cui, “Communication-efficient federated learning,” Proceedings of the Na- tional Academy of Sciences , vol. 118, no. 17, p. e2024789118, 2021
2021
-
[35]
Rethinking gradient sparsification as total error minimiza- tion,
A. Sahu, A. Dutta, A. M Abdelmoniem, T. Banerjee, M. Canini, and P. Kalnis, “Rethinking gradient sparsification as total error minimiza- tion,” Advances in Neural Information Processing Systems , vol. 34, pp. 8133–8146, 2021
2021
-
[36]
Pre- dicting parameters in deep learning,
M. Denil, B. Shakibi, L. Dinh, M. Ranzato, and N. De Freitas, “Pre- dicting parameters in deep learning,” Advances in neural information processing systems, vol. 26, 2013
2013
-
[37]
1-bit stochastic gradient descent and its application to data-parallel distributed training of speech dnns,
F. Seide, H. Fu, J. Droppo, G. Li, and D. Yu, “1-bit stochastic gradient descent and its application to data-parallel distributed training of speech dnns,” in Fifteenth annual conference of the international speech communication association , 2014. JOURNAL OF LATEX CLASS FILES...
2014
-
[38]
Communication-efficient federated learning via knowledge distillation,
C. Wu, F. Wu, L. Lyu, Y . Huang, and X. Xie, “Communication-efficient federated learning via knowledge distillation,” Nature communications, vol. 13, no. 1, pp. 1–8, 2022
2022
-
[39]
Communication-efficient federated deep learning with layerwise asynchronous model update and temporally weighted aggregation,
Y . Chen, X. Sun, and Y . Jin, “Communication-efficient federated deep learning with layerwise asynchronous model update and temporally weighted aggregation,” IEEE transactions on neural networks and learning systems, vol. 31, no. 10, pp. 4229–4238, 2019
2019
-
[40]
A convergence theory for deep learn- ing via over-parameterization,
Z. Allen-Zhu, Y . Li, and Z. Song, “A convergence theory for deep learn- ing via over-parameterization,” in International conference on machine learning. PMLR, 2019, pp. 242–252
2019
-
[41]
A systematic review on overfitting control in shallow and deep neural networks,
M. M. Bejani and M. Ghatee, “A systematic review on overfitting control in shallow and deep neural networks,” Artificial Intelligence Review , vol. 54, no. 8, pp. 6391–6438, 2021
2021
-
[42]
idlg: Improved deep leakage from gradients,
B. Zhao, K. R. Mopuri, and H. Bilen, “idlg: Improved deep leakage from gradients,” arXiv preprint arXiv:2001.02610 , 2020
2001 arXiv
-
[43]
Gradient leakage attacks in federated learning: Research frontiers, taxonomy and future directions,
H. Yang, M. Ge, D. Xue, K. Xiang, H. Li, and R. Lu, “Gradient leakage attacks in federated learning: Research frontiers, taxonomy and future directions,” IEEE Network, 2023
2023
-
[44]
The mnist database of handwritten digit images for machine learning research,
L. Deng, “The mnist database of handwritten digit images for machine learning research,” IEEE Signal Processing Magazine , vol. 29, no. 6, pp. 141–142, 2012
2012
-
[45]
Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,
H. Xiao, K. Rasul, and R. V ollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747, 2017
2017 arXiv
-
[46]
Emnist: Extending mnist to handwritten letters,
G. Cohen, S. Afshar, J. Tapson, and A. Van Schaik, “Emnist: Extending mnist to handwritten letters,” in 2017 international joint conference on neural networks (IJCNN) . IEEE, 2017, pp. 2921–2926
2017
-
[47]
Learning multiple layers of features from tiny images,
A. Krizhevsky, G. Hinton et al. , “Learning multiple layers of features from tiny images,” 2009
2009
-
[48]
Character-level convolutional net- works for text classification,
X. Zhang, J. Zhao, and Y . LeCun, “Character-level convolutional net- works for text classification,” Advances in neural information processing systems, vol. 28, 2015
2015
-
[49]
Maximum likelihood estimation of dirichlet distribution parameters,
J. Huang, “Maximum likelihood estimation of dirichlet distribution parameters,” CMU Technique report, vol. 76, 2005
2005
-
[50]
Tackling the objective inconsistency problem in heterogeneous federated optimiza- tion,
J. Wang, Q. Liu, H. Liang, G. Joshi, and H. V . Poor, “Tackling the objective inconsistency problem in heterogeneous federated optimiza- tion,” Advances in neural information processing systems , vol. 33, pp. 7611–7623, 2020
2020
-
[51]
Federated learning on non-iid data silos: An experimental study,
Q. Li, Y . Diao, Q. Chen, and B. He, “Federated learning on non-iid data silos: An experimental study,” in 2022 IEEE 38th International Conference on Data Engineering (ICDE) . IEEE, 2022, pp. 965–978
2022
-
[52]
Deep residual learning for image recognition,
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778
2016
-
[53]
Designing network design spaces,
I. Radosavovic, R. P. Kosaraju, R. Girshick, K. He, and P. Doll ´ar, “Designing network design spaces,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 10 428–10 436
2020
-
[54]
Batch normalization: Accelerating deep network training by reducing internal covariate shift,
S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in International conference on machine learning . PMLR, 2015, pp. 448–456
2015
-
[55]
Dropout: a simple way to prevent neural networks from over- fitting,
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhut- dinov, “Dropout: a simple way to prevent neural networks from over- fitting,” The journal of machine learning research , vol. 15, no. 1, pp. 1929–1958, 2014
1929
-
[56]
Why batch normalization damage federated learning on non-iid data?
Y . Wang, Q. Shi, and T.-H. Chang, “Why batch normalization damage federated learning on non-iid data?” arXiv preprint arXiv:2301.02982 , 2023
2023 arXiv
-
[57]
Personalized federated learning with moreau envelopes,
C. T Dinh, N. Tran, and J. Nguyen, “Personalized federated learning with moreau envelopes,” Advances in neural information processing systems, vol. 33, pp. 21 394–21 405, 2020
2020
-
[58]
Federated cinn clustering for accurate clustered federated learning,
Y . Zhou, M. Shi, Y . Tian, Y . Li, Q. Ye, and J. Lv, “Federated cinn clustering for accurate clustered federated learning,” in ICASSP 2024- 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 5590–5594
2024
-
[59]
Step-ahead error feedback for distributed training with compressed gradient,
A. Xu, Z. Huo, and H. Huang, “Step-ahead error feedback for distributed training with compressed gradient,” in Proceedings of the AAAI Confer- ence on Artificial Intelligence , vol. 35, no. 12, 2021, pp. 10 478–10 486
2021
-
[60]
Detached error feedback for distributed sgd with random sparsification,
A. Xu and H. Huang, “Detached error feedback for distributed sgd with random sparsification,” in International Conference on Machine Learning. PMLR, 2022, pp. 24 550–24 575
2022
-
[61]
Federated reinforcement learning: Techniques, applications, and open challenges,
J. Qi, Q. Zhou, L. Lei, and K. Zheng, “Federated reinforcement learning: Techniques, applications, and open challenges,” arXiv preprint arXiv:2108.11887, 2021. IX. B IOGRAPHY SECTION Yuhao Zhou is currently working toward a Ph.D. degree in computer science with the College of ...
2021 arXiv
-
[62]
When δ∗ RC1 ≤ ˆη2 n or δ∗ C2 ≤ ˆη3 n, by choosing a proper ˜η = min{ δ∗ 1 2 C 1 2 1 R 1 2 , ∆∗ 1 3 C 1 3 2 R 1 3 }, the following bound can be derived: O(E||∇F (wt∗ )||2) = K(C0 + ∆∗ 1 2 C 1 2 1 R 1 2 + ∆∗ 2 3 C 1 3 2 R 2 3 ). (66)
-
[63]
When ∆∗ RC1 ≥ ˆη2 n or ∆∗ C2 ≥ ˆη3 n, by choosing a proper ˜η = ˆηn, the following bound can be derived: O(E||∇F (wt∗ )||2) = K( 1 ˜ηR ∆∗ + C0 + ∆∗ 1 2 C 1 2 1 R 1 2 + ∆∗ 2 3 C 1 3 2 R 2 3 ). (67) Finally, a general bound can be derived as follows, and the Theorem V .4 is ther...
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.