GP-FL: Model-Based Hessian Estimation for Second-Order Over-the-Air Federated Learning

Ali Bereyhi; H. Vincent Poor; Saba Asaad; Shayan Mohajer Hamidi

REVIEW 5 major objections 5 minor 1 cited by

GP-FL: Model-Based Hessian Estimation for Second-Order Over-the-Air Federated Learning

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GP-FL estimates the Hessian from noisy over-the-air gradients

desk verdict GP-FL is a plausible idea with a clean presentation, but the core posterior estimator has a sign error and the convergence theorem rests on an unproven δ-approximate premise. read the letter →

arxiv 2412.03867 v1 pith:VTHZHU3J submitted 2024-12-05 cs.LG

Shayan Mohajer Hamidi , Ali Bereyhi , Saba Asaad , H. Vincent Poor This is my paper · ORCID

classification cs.LG

keywords federatedlearningover-the-aircomputationsecond-orderoptimizationHessianestimationGaussianprocessquasi-Newtonmethodswirelesscommunicationconvergenceanalysis

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Second-order federated learning normally needs clients to send local Hessian matrices to the server, which is prohibitive over wireless links. This paper proposes GP-FL, where the server estimates the global Hessian from the noisy over-the-air aggregation of local gradients alone. The server models the unknown Hessian as a Gaussian process, computes a posterior from a window of recent noisy gradient differences, and samples a quasi-Hessian matrix that defines the update direction. The paper claims this achieves a linear-quadratic convergence rate, needing $O(\log \log 1/\varepsilon)$ rounds near the optimum or $O(\log(1/\varepsilon)/\log(1/\mu))$ otherwise. Experiments across LIBSVM, Fashion-MNIST, and CIFAR-10/100 report higher test accuracy than first- and second-order baselines.

What carries the argument

The central object is the Gaussian-process model of the quasi-Newton Hessian matrix: each entry of $\tilde B_t$ is treated as a Gaussian process whose posterior is conditioned on a window of the last $r$ noisy gradient differences, with covariance estimated by a radial basis kernel. Solving the secant equation $\tilde B_t w_t = \tilde y_t$ on samples gives the GP prior, and posterior sampling produces the Hessian estimate used in the Newton-type direction. This mechanism converts statistically known over-the-air noise into a probabilistic model, letting the server average curvature information across $r$ rounds instead of relying only on the latest two noisy gradients.

What would settle it

Run GP-FL on a strongly convex logistic-regression problem, record each round's $\delta_t = \|\hat B_t^{-1} - H_t^{-1}\| / \|H_t^{-1}\|$ using the true Hessian, and check whether $\max_t \delta_t < \lambda/L$; if it is ever violated, the stated linear-quadratic rate does not apply. Alternatively, attempt the method on a model with $d$ in the millions and check whether constructing the $d \times d$ posterior Hessian is computationally feasible; failure would falsify the practical claim for deep networks.

Watch

Extended reading notes

Core claim

The central claim is that the parameter server can form a stochastic estimator of the global Hessian matrix directly from the noisy aggregated gradients, without any local Hessian transmission. GP-FL's update is $\theta_{t+1} = \theta_t + \eta_t \tilde d_t$ with $\tilde d_t = -\hat B_t^{-1} \tilde g_t$, where $\tilde g_t$ is the AirComp gradient estimate and $\hat B_t$ is sampled from the Gaussian-process posterior of the quasi-Newton matrix. The posterior is built from the last $r$ noisy gradient differences via the secant equation and a radial basis kernel. The paper shows (Theorem 1) that under strong convexity, $L$-smoothness, and a $\delta$-approximation condition on the inverse quasi-Hessian, the expected distance to the optimum contracts with a linear-quadratic rate, and Corollaries 1 and 2 give the round complexities. The experiments support the claim that this scheme suppresses the directional bias introduced by over-the-air noise and outperforms AirComp first-order methods, GIANT, DANE, and the Sec-Order method.

Load-bearing premise

The load-bearing premise is that the inverse of the Gaussian-process posterior sample $\hat B_t^{-1}$ is a $\delta$-approximate of the true inverse Hessian with $\delta = \max_t \delta_t < \lambda/L$, which the theorem assumes but the paper never verifies; the practical scalability of estimating a full $d \times d$ Hessian for deep models is a second unproven premise.

Editorial extensions

If this is right

If GP-FL's central claim holds, a wireless FL system can enjoy second-order convergence rates without clients ever transmitting Hessian matrices or Hessian-vector products.
The round complexity of $O(\log \log 1/\varepsilon)$ near the optimum means high-accuracy solutions need dramatically fewer communication rounds than the linear $O(1/\varepsilon)$-type rates of first-order AirComp methods.
The Hessian estimate absorbs the channel noise statistics through the posterior, so the degradation seen in direct BFGS or Newton-direction AirComp baselines should be suppressed.
The observation-window study suggests a modest $r \approx 20$ is enough in practice, making the method implementable with a short memory of past gradients.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the same Gaussian-process Hessian estimator could be applied to any FL setting with noisy gradient aggregation, not only the specific zero-forcing AirComp receiver described here, because the noise model enters only through the posterior.
Editorial inference: the convergence guarantee is conditional on the $\delta$-approximate requirement, and the paper does not demonstrate that GP-FL's sampled quasi-Hessian satisfies $\delta < \lambda/L$; testing this condition would decide whether the linear-quadratic rate is actually achieved or only assumed.
Editorial inference: for deep models such as ResNet-18 the paper estimates a $d \times d$ Hessian, which is impractical at scale; a block-diagonal or low-rank approximation would be needed for the approach to transfer to larger architectures.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Dear colleague,

Quick take: GP-FL is a serious attempt to bring second-order methods to over-the-air FL without extra communication, but the math doesn't hold together as written. The central convergence claim is conditional on an assumption the paper never proves, and the posterior formula in Lemma 1 looks wrong.

What's genuinely new: applying Wills and Schön's GP-based Hessian modeling to the noisy AirComp setting, using a temporal window of aggregated gradients. That's a real application, and the paper is honest about borrowing the GP idea. The system model—device selection, receiver design, zero-forcing aggregation—is standard but competently assembled. The experiments cover a reasonable set of datasets and show GP-FL beating the listed baselines, which is evidence the idea has some practical life.

The soft spots are load-bearing. First, the conditional mean in Lemma 1 has a sign error. For the Gaussian model in (31) with Cov(b,o)=φ^T, the correct posterior mean is E[b|o]=μ_b + φ^T K^{-1}(o−μ_o). The paper gives μ_b − a^T(o−μ_o) in (32a) and repeats the minus in the proof. That is not a typo in one line; it propagates into the posterior sample used to build B_t in (34). Second, the covariance is not estimated from a GP trained on the window; it's a kernel matrix evaluated at the single concatenated vector z0_t. So the claim that this is a "posterior" from temporal observations is weak. Third, Theorem 1 assumes B_t^{-1} is δ-approximate to H^{-1} with δ < λ/L, but nothing in the paper shows the GP-FL estimator satisfies that. That assumption is doing all the work in the convergence proof; without it, the linear-quadratic rate is just a re-statement of [49]'s result plus an extra linear term. Finally, the ResNet-18 experiments require estimating a full d×d Hessian for a network with millions of parameters; that's computationally infeasible as described, and the paper doesn't explain any approximation.

None of this means the idea is worthless. The problem is real, and with the sign fixed and a proper analysis of the δ-approximate condition, the paper could be a decent contribution. A good referee could also ask for error bars and a description of how the Hessian is actually stored for deep models.

Bottom line: this deserves a serious referee—not a desk reject—because the core question (can you estimate curvature from noisy over-the-air gradients?) is interesting and the authors know the literature. But as it stands, I wouldn't rely on the convergence theorem or the exact posterior derivation.

Recommendation: send it out, with a referee who knows GP regression and can check the sign and the δ-approximate argument. If those get fixed, acceptance is plausible; if not, the paper needs major revision.

Referee Report

5 major / 5 minor

Summary. The paper proposes GP-FL, a second-order federated learning algorithm for over-the-air (AirComp) settings. The parameter server models the unknown Hessian as a Gaussian process, conditions on a window of noisy gradient differences to obtain a posterior for each Hessian entry, samples from that posterior to form a quasi-Hessian matrix, and updates the model with a quasi-Newton direction. The paper claims a linear-quadratic convergence rate (Theorem 1 and Corollaries 1-2) and reports accuracy improvements over FedAvg, GIANT, DANE, Sec-Order, and BFGS on LIBSVM, Fashion-MNIST, CIFAR-10, and CIFAR-100.

Significance. If the claims were substantiated, GP-FL would be a notable contribution: a communication-efficient second-order AirComp method that avoids Hessian transmission and provably accelerates convergence while mitigating aggregation noise. The paper contains useful components: a careful signal model with transmit-power normalization, a zero-forcing receiver design, a device selection framework, and an explicit convergence-analysis structure. However, the central mathematical claims are not established. The algorithm's core estimator is not the Gaussian-process posterior described in Lemma 1, the key delta-approximate premise of Theorem 1 is unproven, the corollary derivations are internally inconsistent with the stated rates, and the deep-network experiments cannot be realized with the described full-Hessian construction. In its current form the significance is therefore not realized.

major comments (5)

[Section IV-D, Eq. (33a)] Lemma 1 states zeta(o_t) = E_hat{tilde b0_t} - a_t^T(o_t - mu[o0_t]) with a_t = K_t^{-1} phi_t, and the proof repeats this minus sign in (33a). For the Gaussian model in (31), the standard conditional mean is E[tilde b_t | o_t] = E_hat{tilde b0_t} + phi_t^T K_t^{-1}(o_t - mu[o0_t]). The minus sign makes the claimed posterior mean incorrect for the stated model, so the matrix B_hat_t constructed from these conditional samples is not a posterior estimator. Since (34) and Theorem 1 rely on this estimator, this sign error is load-bearing.
[Section IV-C, Eqs. (29)-(32)] The covariance quantities K_t, phi_t, and beta_t are defined as kappa(z0_t, z0_t) evaluated at a single vector z0_t obtained by concatenating the r gradient differences and one deterministic quasi-Newton entry. This is a kernel transform of one point, not an estimate of the covariance from the r observations in Y_t; no averaging, maximum-likelihood fit, or empirical covariance is performed. Consequently the 'posterior' of Lemma 1 is not a Gaussian-process posterior trained on the window Y_t, and no statistical guarantee or concentration argument is provided for it.
[Theorem 1 and Appendix B] Theorem 1 assumes that the inverse of the sampled quasi-Hessian matrix B_hat_t^{-1} is a delta_t-approximate of the true inverse Hessian, with delta = max_t delta_t < lambda/L. No argument in the paper shows that the estimator from Lemma 1 satisfies this condition with the required small delta. The proof uses this assumption at Eq. (57)-(61) to obtain the contraction factor mu = L delta / lambda; without it the theorem has no force. Since the estimator is not even the posterior of the stated model, this premise is an unsupported hypothesis on which the main convergence claim rests.
[Corollaries 1-2 and Appendices C-D] Corollary 1 is stated with T_epsilon = O(log log 1/epsilon), but Appendix C's inequality (68), log(1/epsilon) <= -log(4lambda/L - L^2 delta/lambda) - 2(T_epsilon - t_0) log(gamma), directly yields T_epsilon = O(log 1/epsilon), not O(log log 1/epsilon). In addition, Theorem 1's bound contains the additive term ((mu^t - 1)/(mu - 1)) C_t, and C_t includes the channel-noise term sigma_n ||c_t||/(|D| sqrt(alpha_t^ZF)), which does not vanish as t grows. Corollary 2's condition (44) depends on t and requires the initial distance to exceed a threshold that grows like mu^{-t}; for any fixed initial condition this condition must eventually fail, so the corollary does not establish convergence to an arbitrary epsilon-neighborhood for the noisy problem.
[Section VI-C, VI-D and Eq. (34)] Algorithm 1 and Eq. (34) require forming the full d-by-d matrix B_hat_t and computing B_hat_t^{-1} times the estimated gradient. For the ResNet-18 experiments on CIFAR-10 and CIFAR-100, d is on the order of 10^7, so B_hat_t would have roughly 10^14 entries; storing, let alone inverting, such a matrix is infeasible. Remark 6 only asserts that the PS has high computational capability and does not address memory or per-entry GP computations. As written, the reported deep-network experiments cannot have implemented the described GP-FL algorithm, and no approximation or implementation details are given.

minor comments (5)

[Algorithm 1, input line] The input list contains the typo 'sindow size r'; it should be 'window size r'.
[Section III-B, after Eq. (21)] The noise vector tilde n_t is defined over complex Gaussian noise entries c^H n_{t,j}, while the gradients are treated as real; Remark 2 mentions complex gradients but the subsequent statistical model does not consistently handle complex versus real quantities.
[Remark 3] The text says 'assuming that g_t itself is an unbiased estimator of the true gradient,' but g_t is the exact global gradient; the wording should distinguish the exact gradient from its noisy estimate.
[Section VI and Table I] All reported accuracy curves and Table I lack error bars, confidence intervals, or the number of independent runs; this makes it impossible to judge whether the reported improvements over baselines are statistically significant.
[Section IV-A] The paper does not discuss how the RBF kernel bandwidth tau is set or tuned; tau is a free parameter whose influence on the Hessian estimate is not analyzed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the core Hessian estimator is built from noisy gradients via an explicit GP model, the convergence theorem is a conditional δ-approximate statement relying on an external Newton-step bound, and no fitted parameter is renamed as a prediction.

full rationale

I walked the claimed derivation chain. In Section IV, the Hessian estimator is constructed from the last r noisy gradient differences and a deterministic quasi-Newton solution: the PS forms z0, estimates the GP covariance with the radial-basis kernel κ(z0, z0), applies Lemma 1 to obtain the posterior mean/variance, builds B̂t, and computes the direction d̃t = −B̂t^{-1}g̃t (Eq. 34). This uses the same first-order observations to build the estimator, but it is not a fitted parameter being renamed as a prediction: the posterior expressions are explicit functions of the kernel and the observation window, and the Newton-direction output is not used to define the model. The convergence claim in Theorem 1 is explicitly conditional: it assumes the inverse of the posterior-sampled quasi-Hessian is a δt-approximate inverse Hessian, and the proof combines the external bound from Polyak and Tremba [49] with a triangle-inequality argument for M2. The δ-approximate premise is not proven for the GP-FL estimator, and the conditional-mean formula in Lemma 1 appears to have a sign inconsistency relative to the standard joint-Gaussian posterior; these are correctness or verification gaps, not circular reductions of the output to the input. The self-citations in the paper (e.g., [17], [27], [51], [52]) provide related-work context and experimental setups rather than load-bearing mathematical premises. The device-selection reference [20] is external and is not used to derive the convergence rate. No step satisfies the hard rule that a quoted equation reduces to its own input by construction, so the circularity score is 0.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on an unverified delta-approximate condition for the GP estimator, an independence assumption for Hessian entries, and a strong convexity assumption that is not satisfied by the tested neural networks. The method also has two under-specified hyperparameters (r and tau). No new physical entities are introduced.

free parameters (2)

observation window size r = 20
Chosen from a hyperparameter sweep on CIFAR-10 Setup I (Table I) and reused for all other experiments; controls how many gradient differences are used in the GP estimate.
RBF kernel bandwidth tau
Definition 1 introduces a bandwidth tau in the radial basis kernel but the paper never reports the value used in experiments, leaving the method under-specified.

assumptions (3)

domain assumption The global loss is L-smooth and lambda-strongly convex (Assumption 1).
Used in Theorem 1 and Corollaries 1-2; however, the experiments train non-convex neural networks, so the theory does not cover the main empirical claims.
domain assumption Entries of the quasi-Newton matrix are statistically independent Gaussian processes.
Section IV-B states entries of B_t are independent, but the entries are coupled through the secant equation B_t w_t = y_t, so the independence assumption is an approximation with no justification.
ad hoc to paper The posterior-sampled quasi-Hessian inverse is a delta-approximate of the true inverse Hessian.
Theorem 1 depends on this condition to get mu < 1 and the claimed convergence rates; no derivation or verification is offered, making the theorem conditional.

how reviews work

0 comments

Cite this review

Pith. "Pith review of GP-FL: Model-Based Hessian Estimation for Second-Order Over-the-Air Federated Learning." pith.science (2026). https://pith.science/paper/VTHZHU3J

@misc{pith2026241203867,
  author       = {Pith},
  title        = {Pith review of: GP-FL: Model-Based Hessian Estimation for Second-Order Over-the-Air Federated Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VTHZHU3J}},
  note         = {Machine review of arXiv:2412.03867}
}

read the original abstract

Second-order methods are widely adopted to improve the convergence rate of learning algorithms. In federated learning (FL), these methods require the clients to share their local Hessian matrices with the parameter server (PS), which comes at a prohibitive communication cost. A classical solution to this issue is to approximate the global Hessian matrix from the first-order information. Unlike in idealized networks, this solution does not perform effectively in over-the-air FL settings, where the PS receives noisy versions of the local gradients. This paper introduces a novel second-order FL framework tailored for wireless channels. The pivotal innovation lies in the PS's capability to directly estimate the global Hessian matrix from the received noisy local gradients via a non-parametric method: the PS models the unknown Hessian matrix as a Gaussian process, and then uses the temporal relation between the gradients and Hessian along with the channel model to find a stochastic estimator for the global Hessian matrix. We refer to this method as Gaussian process-based Hessian modeling for wireless FL (GP-FL) and show that it exhibits a linear-quadratic convergence rate. Numerical experiments on various datasets demonstrate that GP-FL outperforms all classical baseline first and second order FL approaches.

Figures

Figures reproduced from arXiv: 2412.03867 by the authors.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Distributed Quasi-Newton Method for Fair and Fast Federated Learning
cs.LG 2025-01 reject novelty 5.0 of 10

DQN-Fed updates a global model in a direction that makes every client's loss decrease at a rate tied to its local quasi-Newton step, with claimed linear-quadratic convergence.

Reference graph

Works this paper leans on

53 extracted references · 39 canonical work pages · cited by 1 Pith paper

[49]

New versions of newton method: step- size choice, convergence domain and under-determined equations,

B. Polyak and A. Tremba, “New versions of newton method: step- size choice, convergence domain and under-determined equations,” Optimization Methods and Software , vol. 35, no. 6, pp. 1272–1303, 2020

2020
[1]

Communication-efficient learning of deep networks from decentralized data,

B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS) . PMLR, 2017, pp. 1273–1282

work page 2017
[2]

Federated learning over wireless fading channels,

M. M. Amiri and D. G ¨und¨uz, “Federated learning over wireless fading channels,” IEEE Trans. Wirel. Commun. , vol. 19, no. 5, pp. 3546–3557, 2020

work page 2020
[3]

Systems and methods for communication resource usage control,

S. M. Hamidi, S. Herath, A. Bayesteh, and A. K. Khandani, “Systems and methods for communication resource usage control,” May 30 2019, uS Patent App. 15/824,352

work page 2019
[4]

Training neural networks on remote edge devices for unseen class classification,

S. M. Hamidi, “Training neural networks on remote edge devices for unseen class classification,” IEEE Signal Processing Letters , vol. 31, pp. 1004–1008, 2024. 13

work page 2024
[5]

Communication lower bounds for statistical estimation problems via a distributed data processing inequality,

M. Braverman, A. Garg, T. Ma, H. L. Nguyen, and D. P. Woodruff, “Communication lower bounds for statistical estimation problems via a distributed data processing inequality,” in Proceedings of the Forty-eighth Annual ACM Symposium on Theory of Computing , 2016, pp. 1011–1020

work page 2016
[6]

Geometric lower bounds for distributed parameter estimation under communication constraints,

Y . Han, A. ¨Ozg¨ur, and T. Weissman, “Geometric lower bounds for distributed parameter estimation under communication constraints,” in Proceedings of the 31st Conference On Learning Theory (COLT). PMLR, 2018, pp. 3163–3188

work page 2018
[7]

Quantization bits allocation for wireless federated learning,

M. Lan, Q. Ling, S. Xiao, and W. Zhang, “Quantization bits allocation for wireless federated learning,” IEEE Trans. Wirel. Commun. , vol. 22, no. 11, pp. 8336–8351, 2023

work page 2023

Show all 53 references

[8]

Novel gradient sparsification algorithm via bayesian inference,

A. Bereyhi, B. Liang, G. Boudreau, and A. Afana, “Novel gradient sparsification algorithm via bayesian inference,” in Proceedings of IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP), 2024, pp. 1–6

2024
[9]

A field guide to federated optimization,

J. Wang, Z. Charles, Z. Xu et al. , “A field guide to federated optimization,” 2021. [Online]. Available: https://arxiv.org/abs/2107.06917

2021 arXiv
[10]

Adaptive federated optimization,

S. J. Reddi, Z. Charles, M. Zaheer, K. Rush, J. Kone ˇcn`y, S. Kumar, and H. B. McMahan, “Adaptive federated optimization,” in Proceedings of the International Conference on Learning Representations , 2020

2020
[11]

Effective federated adaptive gradient methods with non-iid decentralized data,

Q. Tong, G. Liang, and J. Bi, “Effective federated adaptive gradient methods with non-iid decentralized data,” 2020. [Online]. Available: https://arxiv.org/abs/2009.06557

2020 arXiv
[12]

Federated learning via over- the-air computation,

K. Yang, T. Jiang, Y . Shi, and Z. Ding, “Federated learning via over- the-air computation,” IEEE Trans. Wirel. Commun., vol. 19, no. 3, pp. 2022–2035, 2020

2022
[13]

Device scheduling in over-the-air federated learning via matching pursuit,

A. Bereyhi, A. Vagollari, S. Asaad, R. R. M ¨uller, W. Gerstacker, and H. V . Poor, “Device scheduling in over-the-air federated learning via matching pursuit,” IEEE Trans. Signal Process. , vol. 71, pp. 2188–2203, 2023

2023
[14]

Broadband analog aggregation for low- latency federated edge learning,

G. Zhu, Y . Wang, and K. Huang, “Broadband analog aggregation for low- latency federated edge learning,” IEEE Trans. Wirel. Commun. , vol. 19, no. 1, pp. 491–506, 2019

2019
[15]

Machine learning at the wireless edge: Distributed stochastic gradient descent over-the-air,

M. M. Amiri and D. G ¨und¨uz, “Machine learning at the wireless edge: Distributed stochastic gradient descent over-the-air,” IEEE Trans. Signal Process., vol. 68, pp. 2155–2169, 2020

2020
[16]

On analog gradient descent learning over multiple access fading channels,

T. Sery and K. Cohen, “On analog gradient descent learning over multiple access fading channels,” IEEE Trans. Signal Process. , vol. 68, pp. 2897– 2911, 2020

2020
[17]

Over-the- air federated learning exploiting channel perturbation,

S. M. Hamidi, M. Mehrabi, A. K. Khandani, and D. G ¨und¨uz, “Over-the- air federated learning exploiting channel perturbation,” in 2022 IEEE 23rd International Workshop on Signal Processing Advances in Wireless Communication (SPAWC), 2022, pp. 1–5

2022
[18]

Privacy for free: Wireless federated learning via uncoded transmission with adaptive power control,

D. Liu and O. Simeone, “Privacy for free: Wireless federated learning via uncoded transmission with adaptive power control,” IEEE J. Sel. Areas Commun., vol. 39, no. 1, pp. 170–185, 2020

2020
[19]

Harnessing wireless channels for scalable and privacy-preserving federated learning,

A. Elgabli, J. Park, C. B. Issaid, and M. Bennis, “Harnessing wireless channels for scalable and privacy-preserving federated learning,” IEEE Trans. Commun., vol. 69, no. 8, pp. 5194–5208, 2021

2021
[20]

Over- the-air federated learning via second-order optimization,

P. Yang, Y . Jiang, T. Wang, Y . Zhou, Y . Shi, and C. N. Jones, “Over- the-air federated learning via second-order optimization,” IEEE Trans. Wirel. Commun., vol. 21, no. 12, pp. 10 560–10 575, 2022

2022
[21]

GIANT: Globally improved approximate newton method for distributed optimization,

S. Wang, F. Roosta, P. Xu, and M. W. Mahoney, “GIANT: Globally improved approximate newton method for distributed optimization,” in Advances in Neural Information Processing Systems , 2018, pp. 2332– 2342

2018
[22]

Disco: Distributed optimization for self-concordant empirical loss,

Y . Zhang and X. Lin, “Disco: Distributed optimization for self-concordant empirical loss,” in Proceedings of the International conference on machine learning. PMLR, 2015, pp. 362–370

2015
[23]

Distributed newton can communicate less and resist byzantine workers,

A. Ghosh, R. K. Maity, and A. Mazumdar, “Distributed newton can communicate less and resist byzantine workers,” Advances in Neural Information Processing Systems , vol. 33, pp. 18 028–18 038, 2020

2020
[24]

LIBSVM: A library for support vector machines,

C.-C. Chang and C.-J. Lin, “LIBSVM: A library for support vector machines,” ACM transactions on intelligent systems and technology (TIST), vol. 2, no. 3, pp. 1–27, 2011

2011
[25]

Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,

H. Xiao, K. Rasul, and R. V ollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” 2017. [Online]. Available: https://arxiv.org/abs/1708.07747

2017 arXiv
[26]

Learning multiple layers of features from tiny images,

A. Krizhevsky, “Learning multiple layers of features from tiny images,” Ph.D. dissertation, University of Toronto, 2009, technical Report

2009
[27]

Fed-it: Addressing class imbalance in federated learning through an information- theoretic lens,

S. M. Hamidi, R. Tan, L. Ye, and E.-H. Yang, “Fed-it: Addressing class imbalance in federated learning through an information- theoretic lens,” in 2024 IEEE International Symposium on Information Theory (ISIT) , 2024, pp. 1848–1853

2024
[28]

Adaptive subgradient methods for online learning and stochastic optimization

J. Duchi, E. Hazan, and Y . Singer, “Adaptive subgradient methods for online learning and stochastic optimization.” J. Mach. Learn. Res., vol. 12, no. 7, 2011

2011
[29]

Adam: A method for stochastic optimization,

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
[30]

Mime: Mimicking centralized stochastic algorithms in federated learning,

S. P. Karimireddy, M. Jaggi, S. Kale, M. Mohri, S. J. Reddi, S. U. Stich, and A. T. Suresh, “Mime: Mimicking centralized stochastic algorithms in federated learning,” 2021. [Online]. Available: https://arxiv.org/abs/2008.03606

2021 arXiv
[31]

Communication-efficient distributed optimization using an approximate newton-type method,

O. Shamir, N. Srebro, and T. Zhang, “Communication-efficient distributed optimization using an approximate newton-type method,” in Proceedings of the International Conference on Machine Learning . PMLR, 2014, pp. 1000–1008

2014
[32]

Feddane: A federated newton-type method,

T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smithy, “Feddane: A federated newton-type method,” in Proceedings of the 2019 53rd Asilomar Conference on Signals, Systems, and Computers . IEEE, 2019, pp. 1227–1231

2019
[33]

Aide: Fast and communication efficient distributed optimization,

S. J. Reddi, J. Kone ˇcn´y, P. Richt ´arik, B. P ´ocz´os, and A. Smola, “Aide: Fast and communication efficient distributed optimization,” 2016. [Online]. Available: https://arxiv.org/abs/1608.06879

2016 arXiv
[34]

Localnewton: Reducing communication bottleneck for distributed learning,

V . Gupta, A. Ghosh, M. Derezinski, R. Khanna, K. Ramchandran, and M. Mahoney, “Localnewton: Reducing communication bottleneck for distributed learning,” 2021. [Online]. Available: https://arxiv.org/abs/ 2105.07320

2021 arXiv
[35]

A reliable effective terascale linear learning system,

A. Agarwal, O. Chapelle, M. Dud´ık, and J. Langford, “A reliable effective terascale linear learning system,” J. Mach. Learn. Res. , vol. 15, no. 1, pp. 1111–1133, 2014

2014
[36]

CoCoA: A general framework for communication-efficient distributed optimization,

V . Smith, S. Forte, M. Chenxin, M. Tak ´aˇc, M. I. Jordan, and M. Jaggi, “CoCoA: A general framework for communication-efficient distributed optimization,” J. Mach. Learn. Res. , vol. 18, p. 230, 2018

2018
[37]

A distributed second-order algorithm you can trust,

C. Duenner, A. Lucchi, M. Gargiani, A. Bian, T. Hofmann, and M. Jaggi, “A distributed second-order algorithm you can trust,” in Proceedings of the 35th International Conference on Machine Learning, vol. 80. PMLR, 10–15 Jul 2018, pp. 1358–1366

2018
[38]

M. R. Hestenes, E. Stiefel et al. , Methods of conjugate gradients for solving linear systems . NBS Washington, DC, 1952, vol. 49, no. 1

1952
[39]

Distributed second order methods with fast rates and compressed communication,

R. Islamov, X. Qian, and P. Richt ´arik, “Distributed second order methods with fast rates and compressed communication,” in International conference on machine learning . PMLR, 2021, pp. 4617–4628

2021
[40]

Fednl: Making newton-type methods applicable to federated learning,

M. Safaryan, R. Islamov, X. Qian, and P. Richtarik, “Fednl: Making newton-type methods applicable to federated learning,” in Proceedings of the International Conference on Machine Learning . PMLR, 2022, pp. 18 959–19 010

2022
[41]

Local stochastic gradient descent ascent: Convergence analysis and communication efficiency,

Y . Deng and M. Mahdavi, “Local stochastic gradient descent ascent: Convergence analysis and communication efficiency,” in Proceedings of the International Conference on Artificial Intelligence and Statistics . PMLR, 2021, pp. 1387–1395

2021
[42]

Federated minimax optimization: Improved convergence analyses and algorithms,

P. Sharma, R. Panda, G. Joshi, and P. Varshney, “Federated minimax optimization: Improved convergence analyses and algorithms,” in Pro- ceedings of the International Conference on Machine Learning . PMLR, 2022, pp. 19 683–19 730

2022
[43]

Nocedal and S

J. Nocedal and S. J. Wright, Numerical Optimization. Springer, 1999

1999
[44]

Adjustment of an inverse matrix corre- sponding to changes in the elements of a given column or a given row of the original matrix,

J. Shermen and W. Morrison, “Adjustment of an inverse matrix corre- sponding to changes in the elements of a given column or a given row of the original matrix,” Annals of Mathmatical Statistics , vol. 20, pp. 621–625, 1949

1949
[45]

A uniform-forcing transceiver design for over-the-air function computation,

L. Chen, X. Qin, and G. Wei, “A uniform-forcing transceiver design for over-the-air function computation,” IEEE Wirel. Commun. Lett. , vol. 7, no. 6, pp. 942–945, 2018

2018
[46]

Stochastic quasi-newton with line-search regularisation,

A. G. Wills and T. B. Sch ¨on, “Stochastic quasi-newton with line-search regularisation,” Automatica, vol. 127, p. 109503, 2021

2021
[47]

A tutorial on gaussian process regression: Modelling, exploring, and exploiting functions,

E. Schulz, M. Speekenbrink, and A. Krause, “A tutorial on gaussian process regression: Modelling, exploring, and exploiting functions,” Journal of Mathematical Psychology , vol. 85, pp. 1–16, 2018

2018
[48]

Gaussian processes,

C. B. Do and H. Lee, “Gaussian processes,” Stanford University, Stanford, CA, accessed Dec , vol. 5, p. 2017, 2007

2017
[50]

D. W. Hosmer Jr, S. Lemeshow, and R. X. Sturdivant, Applied Logistic Regression. John Wiley & Sons, 2013

2013
[51]

Adafed: Fair federated learning via adaptive common descent direction,

S. M. Hamidi and E.-H. YANG, “Adafed: Fair federated learning via adaptive common descent direction,” Transactions on Machine Learning Research, 2024. [Online]. Available: https://openreview.net/forum?id= rFecyFpFUp

2024
[52]

Fair wireless federated learning through the identification of a common descent direction,

S. Mohajer Hamidi and O. Damen, “Fair wireless federated learning through the identification of a common descent direction,” IEEE Communications Letters, vol. 28, no. 3, pp. 567–571, 2024

2024
[2017]

Available: https://arxiv.org/abs/1412.6980

[Online]. Available: https://arxiv.org/abs/1412.6980

arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [49]

New versions of newton method: step- size choice, convergence domain and under-determined equations,

B. Polyak and A. Tremba, “New versions of newton method: step- size choice, convergence domain and under-determined equations,” Optimization Methods and Software , vol. 35, no. 6, pp. 1272–1303, 2020

2020

[2] [1]

Communication-efficient learning of deep networks from decentralized data,

B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS) . PMLR, 2017, pp. 1273–1282

work page 2017

[3] [2]

Federated learning over wireless fading channels,

M. M. Amiri and D. G ¨und¨uz, “Federated learning over wireless fading channels,” IEEE Trans. Wirel. Commun. , vol. 19, no. 5, pp. 3546–3557, 2020

work page 2020

[4] [3]

Systems and methods for communication resource usage control,

S. M. Hamidi, S. Herath, A. Bayesteh, and A. K. Khandani, “Systems and methods for communication resource usage control,” May 30 2019, uS Patent App. 15/824,352

work page 2019

[5] [4]

Training neural networks on remote edge devices for unseen class classification,

S. M. Hamidi, “Training neural networks on remote edge devices for unseen class classification,” IEEE Signal Processing Letters , vol. 31, pp. 1004–1008, 2024. 13

work page 2024

[6] [5]

Communication lower bounds for statistical estimation problems via a distributed data processing inequality,

M. Braverman, A. Garg, T. Ma, H. L. Nguyen, and D. P. Woodruff, “Communication lower bounds for statistical estimation problems via a distributed data processing inequality,” in Proceedings of the Forty-eighth Annual ACM Symposium on Theory of Computing , 2016, pp. 1011–1020

work page 2016

[7] [6]

Geometric lower bounds for distributed parameter estimation under communication constraints,

Y . Han, A. ¨Ozg¨ur, and T. Weissman, “Geometric lower bounds for distributed parameter estimation under communication constraints,” in Proceedings of the 31st Conference On Learning Theory (COLT). PMLR, 2018, pp. 3163–3188

work page 2018

[8] [7]

Quantization bits allocation for wireless federated learning,

M. Lan, Q. Ling, S. Xiao, and W. Zhang, “Quantization bits allocation for wireless federated learning,” IEEE Trans. Wirel. Commun. , vol. 22, no. 11, pp. 8336–8351, 2023

work page 2023