{"id":"9ee60d22-8c2f-4c92-af63-dd2590f5fabd","arxiv_id":"2412.03867","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"GP-FL estimates the global Hessian at the server with Gaussian process regression on recent noisy gradient differences, claiming linear-quadratic convergence in over-the-air federated learning, but the required approximation property is assumed, not derived.","lead":"This paper proposes GP-FL, a wireless federated learning method where the server estimates the curvature of the loss from noisy gradient sums instead of asking clients to send their local curvature. The authors claim faster convergence and fewer communication rounds than existing over-the-air methods, but the key proof step is assumed and the reported deep-network experiments are not backed by a feasible implementation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GP-FL's Hessian estimator is not the posterior of the stated Gaussian model: Lemma 1's conditional-mean sign is wrong and the covariance is built from a single sample, so the δ-approximate premise of Theorem 1 is unsupported.","rationale":"The reader's weakest_assumption is that GP-FL's convergence relies on an unverified δ-approximate condition. My concern sharpens that point: the estimator is not merely unverified; as stated it is internally inconsistent with the Gaussian model. Lemma 1's conditional mean has the wrong sign, and the covariance is constructed from a single sample rather than from the r-observation window, so the resulting B_t is not a posterior sample from the stated GP. This directly undermines the central algorithmic claim and the premise of Theorem 1. I partially agree with the reader because the δ-approximate issue is the same load-bearing failure, but I locate a more concrete mechanism (the sign error and one-sample covariance construction) rather than treating the δ condition as merely unproven. The secondary concern about estimating a full d×d Hessian for ResNet-18 is real but less central to the theoretical claim; the estimator construction already fails in small dimensions. Therefore the reader's REJECT verdict stands without modification.","tokens_in":21330,"tokens_out":7137,"duration_ms":69886,"concrete_test":"Check Lemma 1 analytically and numerically in a one-dimensional Gaussian case: set μ_o=0, μ_b=0, K=1, φ=0.9, β=1, draw o~N(0,1), and compare the true conditional mean E[b|o]=0.9o with (32a), which gives −0.9o; a Monte Carlo estimate over 10^4 samples will match 0.9o, confirming the sign error. Then, for a d=2 quadratic objective with known Hessian H under the paper's AirComp noise model, implement Section IV exactly and measure δ_t=∥B_t^{-1}−H^{-1}∥/∥H^{-1}∥ for the settings in Figure 1; if max_t δ_t ≥ λ/L, Theorem 1's convergence premise is violated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is that the estimator entering Theorem 1 is not a valid Gaussian-process posterior. In Lemma 1, for z=[o^T,b]^T with covariance [[K,φ],[φ^T,β]] as in (31), the standard conditional mean is E[b|o]=μ_b+φ^T K^{-1}(o-μ_o). Lemma 1 instead states ζ(o_t)=μ_b−a^T(o_t−μ[o_t]) with a=K^{-1}φ (32a), reversing the sign of the correction term. Since (34) uses this posterior sample to build B_t and then B_t^{-1}, the estimator is not the posterior estimator claimed. Moreover, the covariance in (31) is not estimated from the r past observations: K_t, φ_t, and β_t are set as κ(o0_t,o0_t), κ(o0_t,b0_t), and κ(b0_t,b0_t) from a single vector z0_t, so (32) is a kernel transform of one point rather than a posterior from a GP trained on the window Y_t. Consequently, no argument is given that B_t^{-1} is a δ-approximate inverse Hessian with δ<λ/L, which is exactly the premise Theorem 1 needs for the linear-quadratic rate.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GP-FL, a second-order federated learning algorithm for over-the-air (AirComp) settings. The parameter server models the unknown Hessian as a Gaussian process, conditions on a window of noisy gradient differences to obtain a posterior for each Hessian entry, samples from that posterior to form a quasi-Hessian matrix, and updates the model with a quasi-Newton direction. The paper claims a linear-quadratic convergence rate (Theorem 1 and Corollaries 1-2) and reports accuracy improvements over FedAvg, GIANT, DANE, Sec-Order, and BFGS on LIBSVM, Fashion-MNIST, CIFAR-10, and CIFAR-100.","tokens_in":21644,"tokens_out":9152,"duration_ms":96530,"significance":"If the claims were substantiated, GP-FL would be a notable contribution: a communication-efficient second-order AirComp method that avoids Hessian transmission and provably accelerates convergence while mitigating aggregation noise. The paper contains useful components: a careful signal model with transmit-power normalization, a zero-forcing receiver design, a device selection framework, and an explicit convergence-analysis structure. However, the central mathematical claims are not established. The algorithm's core estimator is not the Gaussian-process posterior described in Lemma 1, the key delta-approximate premise of Theorem 1 is unproven, the corollary derivations are internally inconsistent with the stated rates, and the deep-network experiments cannot be realized with the described full-Hessian construction. In its current form the significance is therefore not realized.","major_comments":[{"comment":"Lemma 1 states zeta(o_t) = E_hat{tilde b0_t} - a_t^T(o_t - mu[o0_t]) with a_t = K_t^{-1} phi_t, and the proof repeats this minus sign in (33a). For the Gaussian model in (31), the standard conditional mean is E[tilde b_t | o_t] = E_hat{tilde b0_t} + phi_t^T K_t^{-1}(o_t - mu[o0_t]). The minus sign makes the claimed posterior mean incorrect for the stated model, so the matrix B_hat_t constructed from these conditional samples is not a posterior estimator. Since (34) and Theorem 1 rely on this estimator, this sign error is load-bearing.","section":"Section IV-D, Eq. (33a)"},{"comment":"The covariance quantities K_t, phi_t, and beta_t are defined as kappa(z0_t, z0_t) evaluated at a single vector z0_t obtained by concatenating the r gradient differences and one deterministic quasi-Newton entry. This is a kernel transform of one point, not an estimate of the covariance from the r observations in Y_t; no averaging, maximum-likelihood fit, or empirical covariance is performed. Consequently the 'posterior' of Lemma 1 is not a Gaussian-process posterior trained on the window Y_t, and no statistical guarantee or concentration argument is provided for it.","section":"Section IV-C, Eqs. (29)-(32)"},{"comment":"Theorem 1 assumes that the inverse of the sampled quasi-Hessian matrix B_hat_t^{-1} is a delta_t-approximate of the true inverse Hessian, with delta = max_t delta_t < lambda/L. No argument in the paper shows that the estimator from Lemma 1 satisfies this condition with the required small delta. The proof uses this assumption at Eq. (57)-(61) to obtain the contraction factor mu = L delta / lambda; without it the theorem has no force. Since the estimator is not even the posterior of the stated model, this premise is an unsupported hypothesis on which the main convergence claim rests.","section":"Theorem 1 and Appendix B"},{"comment":"Corollary 1 is stated with T_epsilon = O(log log 1/epsilon), but Appendix C's inequality (68), log(1/epsilon) <= -log(4lambda/L - L^2 delta/lambda) - 2(T_epsilon - t_0) log(gamma), directly yields T_epsilon = O(log 1/epsilon), not O(log log 1/epsilon). In addition, Theorem 1's bound contains the additive term ((mu^t - 1)/(mu - 1)) C_t, and C_t includes the channel-noise term sigma_n ||c_t||/(|D| sqrt(alpha_t^ZF)), which does not vanish as t grows. Corollary 2's condition (44) depends on t and requires the initial distance to exceed a threshold that grows like mu^{-t}; for any fixed initial condition this condition must eventually fail, so the corollary does not establish convergence to an arbitrary epsilon-neighborhood for the noisy problem.","section":"Corollaries 1-2 and Appendices C-D"},{"comment":"Algorithm 1 and Eq. (34) require forming the full d-by-d matrix B_hat_t and computing B_hat_t^{-1} times the estimated gradient. For the ResNet-18 experiments on CIFAR-10 and CIFAR-100, d is on the order of 10^7, so B_hat_t would have roughly 10^14 entries; storing, let alone inverting, such a matrix is infeasible. Remark 6 only asserts that the PS has high computational capability and does not address memory or per-entry GP computations. As written, the reported deep-network experiments cannot have implemented the described GP-FL algorithm, and no approximation or implementation details are given.","section":"Section VI-C, VI-D and Eq. (34)"}],"minor_comments":[{"comment":"The input list contains the typo 'sindow size r'; it should be 'window size r'.","section":"Algorithm 1, input line"},{"comment":"The noise vector tilde n_t is defined over complex Gaussian noise entries c^H n_{t,j}, while the gradients are treated as real; Remark 2 mentions complex gradients but the subsequent statistical model does not consistently handle complex versus real quantities.","section":"Section III-B, after Eq. (21)"},{"comment":"The text says 'assuming that g_t itself is an unbiased estimator of the true gradient,' but g_t is the exact global gradient; the wording should distinguish the exact gradient from its noisy estimate.","section":"Remark 3"},{"comment":"All reported accuracy curves and Table I lack error bars, confidence intervals, or the number of independent runs; this makes it impossible to judge whether the reported improvements over baselines are statistically significant.","section":"Section VI and Table I"},{"comment":"The paper does not discuss how the RBF kernel bandwidth tau is set or tuned; tau is a free parameter whose influence on the Hessian estimate is not analyzed.","section":"Section IV-A"}],"recommendation":"reject","confidential_remarks":"The paper fits the journal's scope and the authors are addressing an important problem, but the technical core is not sound in its current form. The conditional-mean sign error in Lemma 1, the unproven delta-approximate assumption, the inconsistency in the claimed O(log log 1/epsilon) rate, and the computational infeasibility of the deep-network experiments together undermine the paper's central claims. These are load-bearing errors rather than presentation issues. A future revision that corrects the estimator, proves a verifiable condition for the delta-approximate property, and either provides a tractable approximate implementation or restricts experiments to feasible model sizes could warrant reconsideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nQuick take: GP-FL is a serious attempt to bring second-order methods to over-the-air FL without extra communication, but the math doesn't hold together as written. The central convergence claim is conditional on an assumption the paper never proves, and the posterior formula in Lemma 1 looks wrong.\n\nWhat's genuinely new: applying Wills and Schön's GP-based Hessian modeling to the noisy AirComp setting, using a temporal window of aggregated gradients. That's a real application, and the paper is honest about borrowing the GP idea. The system model—device selection, receiver design, zero-forcing aggregation—is standard but competently assembled. The experiments cover a reasonable set of datasets and show GP-FL beating the listed baselines, which is evidence the idea has some practical life.\n\nThe soft spots are load-bearing. First, the conditional mean in Lemma 1 has a sign error. For the Gaussian model in (31) with Cov(b,o)=φ^T, the correct posterior mean is E[b|o]=μ_b + φ^T K^{-1}(o−μ_o). The paper gives μ_b − a^T(o−μ_o) in (32a) and repeats the minus in the proof. That is not a typo in one line; it propagates into the posterior sample used to build B_t in (34). Second, the covariance is not estimated from a GP trained on the window; it's a kernel matrix evaluated at the single concatenated vector z0_t. So the claim that this is a \"posterior\" from temporal observations is weak. Third, Theorem 1 assumes B_t^{-1} is δ-approximate to H^{-1} with δ < λ/L, but nothing in the paper shows the GP-FL estimator satisfies that. That assumption is doing all the work in the convergence proof; without it, the linear-quadratic rate is just a re-statement of [49]'s result plus an extra linear term. Finally, the ResNet-18 experiments require estimating a full d×d Hessian for a network with millions of parameters; that's computationally infeasible as described, and the paper doesn't explain any approximation.\n\nNone of this means the idea is worthless. The problem is real, and with the sign fixed and a proper analysis of the δ-approximate condition, the paper could be a decent contribution. A good referee could also ask for error bars and a description of how the Hessian is actually stored for deep models.\n\nBottom line: this deserves a serious referee—not a desk reject—because the core question (can you estimate curvature from noisy over-the-air gradients?) is interesting and the authors know the literature. But as it stands, I wouldn't rely on the convergence theorem or the exact posterior derivation.\n\nRecommendation: send it out, with a referee who knows GP regression and can check the sign and the δ-approximate argument. If those get fixed, acceptance is plausible; if not, the paper needs major revision.","headline":"GP-FL is a plausible idea with a clean presentation, but the core posterior estimator has a sign error and the convergence theorem rests on an unproven δ-approximate premise.","tokens_in":22104,"tokens_out":4991,"would_cite":false,"duration_ms":111056,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GP-FL estimates the Hessian from noisy over-the-air gradients","keywords":["federated learning","over-the-air computation","second-order optimization","Hessian estimation","Gaussian process","quasi-Newton methods","wireless communication","convergence analysis"],"falsifier":"Run GP-FL on a strongly convex logistic-regression problem, record each round's $\\delta_t = \\|\\hat B_t^{-1} - H_t^{-1}\\| / \\|H_t^{-1}\\|$ using the true Hessian, and check whether $\\max_t \\delta_t < \\lambda/L$; if it is ever violated, the stated linear-quadratic rate does not apply. Alternatively, attempt the method on a model with $d$ in the millions and check whether constructing the $d \\times d$ posterior Hessian is computationally feasible; failure would falsify the practical claim for deep networks.","tokens_in":21171,"feed_emoji":"📡","tokens_out":5970,"duration_ms":56026,"temperature":0.7,"pith_summary":"Second-order federated learning normally needs clients to send local Hessian matrices to the server, which is prohibitive over wireless links. This paper proposes GP-FL, where the server estimates the global Hessian from the noisy over-the-air aggregation of local gradients alone. The server models the unknown Hessian as a Gaussian process, computes a posterior from a window of recent noisy gradient differences, and samples a quasi-Hessian matrix that defines the update direction. The paper claims this achieves a linear-quadratic convergence rate, needing $O(\\log \\log 1/\\varepsilon)$ rounds near the optimum or $O(\\log(1/\\varepsilon)/\\log(1/\\mu))$ otherwise. Experiments across LIBSVM, Fashion-MNIST, and CIFAR-10/100 report higher test accuracy than first- and second-order baselines.","feed_headline":"Gaussian-process Hessian estimation cuts wireless FL rounds","feed_subtitle":"Second-order federated learning without sharing Hessians: linear-quadratic convergence from noisy over-the-air gradients alone.","key_machinery":"The central object is the Gaussian-process model of the quasi-Newton Hessian matrix: each entry of $\\tilde B_t$ is treated as a Gaussian process whose posterior is conditioned on a window of the last $r$ noisy gradient differences, with covariance estimated by a radial basis kernel. Solving the secant equation $\\tilde B_t w_t = \\tilde y_t$ on samples gives the GP prior, and posterior sampling produces the Hessian estimate used in the Newton-type direction. This mechanism converts statistically known over-the-air noise into a probabilistic model, letting the server average curvature information across $r$ rounds instead of relying only on the latest two noisy gradients.","core_discovery":"The central claim is that the parameter server can form a stochastic estimator of the global Hessian matrix directly from the noisy aggregated gradients, without any local Hessian transmission. GP-FL's update is $\\theta_{t+1} = \\theta_t + \\eta_t \\tilde d_t$ with $\\tilde d_t = -\\hat B_t^{-1} \\tilde g_t$, where $\\tilde g_t$ is the AirComp gradient estimate and $\\hat B_t$ is sampled from the Gaussian-process posterior of the quasi-Newton matrix. The posterior is built from the last $r$ noisy gradient differences via the secant equation and a radial basis kernel. The paper shows (Theorem 1) that under strong convexity, $L$-smoothness, and a $\\delta$-approximation condition on the inverse quasi-Hessian, the expected distance to the optimum contracts with a linear-quadratic rate, and Corollaries 1 and 2 give the round complexities. The experiments support the claim that this scheme suppresses the directional bias introduced by over-the-air noise and outperforms AirComp first-order methods, GIANT, DANE, and the Sec-Order method.","pith_inferences":["Editorial inference: the same Gaussian-process Hessian estimator could be applied to any FL setting with noisy gradient aggregation, not only the specific zero-forcing AirComp receiver described here, because the noise model enters only through the posterior.","Editorial inference: the convergence guarantee is conditional on the $\\delta$-approximate requirement, and the paper does not demonstrate that GP-FL's sampled quasi-Hessian satisfies $\\delta < \\lambda/L$; testing this condition would decide whether the linear-quadratic rate is actually achieved or only assumed.","Editorial inference: for deep models such as ResNet-18 the paper estimates a $d \\times d$ Hessian, which is impractical at scale; a block-diagonal or low-rank approximation would be needed for the approach to transfer to larger architectures."],"forward_implications":["If GP-FL's central claim holds, a wireless FL system can enjoy second-order convergence rates without clients ever transmitting Hessian matrices or Hessian-vector products.","The round complexity of $O(\\log \\log 1/\\varepsilon)$ near the optimum means high-accuracy solutions need dramatically fewer communication rounds than the linear $O(1/\\varepsilon)$-type rates of first-order AirComp methods.","The Hessian estimate absorbs the channel noise statistics through the posterior, so the degradation seen in direct BFGS or Newton-direction AirComp baselines should be suppressed.","The observation-window study suggests a modest $r \\approx 20$ is enough in practice, making the method implementable with a short memory of past gradients."],"supporting_citations":[{"why":"Defines the AirComp aggregation model that GP-FL uses to build noisy gradient estimators over the wireless channel.","marker":"[12]"},{"why":"Provides the second-order over-the-air baseline 'Sec-Order' and the Gibbs-sampling device selection reused by GP-FL.","marker":"[20]"},{"why":"Supplies the GIANT baseline for distributed Newton-type optimization that GP-FL is compared against.","marker":"[21]"},{"why":"Supplies the DANE baseline for approximate Newton-type distributed optimization.","marker":"[31]"},{"why":"Introduces the Gaussian-process model of the quasi-Newton matrix that GP-FL adapts for Hessian estimation.","marker":"[46]"},{"why":"Gives the adaptive step-size rule and convergence result for Newton-type methods that Theorem 1 invokes.","marker":"[49]"}],"fun_headline_variants":["GP-FL: Hessian from noisy gradients, no sharing needed","Wireless FL: GP-based Hessian estimation from gradient noise","Second-order FL over the air without Hessian uploads","Gaussian process Hessian: linear-quadratic convergence for FL","Hessian estimated from AirComp noise: GP-FL speeds up FL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the inverse of the Gaussian-process posterior sample $\\hat B_t^{-1}$ is a $\\delta$-approximate of the true inverse Hessian with $\\delta = \\max_t \\delta_t < \\lambda/L$, which the theorem assumes but the paper never verifies; the practical scalability of estimating a full $d \\times d$ Hessian for deep models is a second unproven premise.","fun_headline_variants_meta":{"raw":{"variants":["GP-FL: Hessian from noisy gradients, no sharing needed","Wireless FL: GP-based Hessian estimation from gradient noise","Second-order FL over the air without Hessian uploads","Gaussian process Hessian: linear-quadratic convergence for FL","Hessian estimated from AirComp noise: GP-FL speeds up FL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000746,"raw_usage":{"total_tokens":3351,"prompt_tokens":996,"completion_tokens":2355,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":612,"completion_tokens_details":{"reasoning_tokens":2265}},"tokens_in":612,"tokens_out":2355,"duration_ms":17602,"temperature":1.0,"reasoning_tokens":2265,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:59:27.658322+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run GP-FL on a strongly convex logistic-regression problem, record each round's $\\delta_t = \\|\\hat B_t^{-1} - H_t^{-1}\\| / \\|H_t^{-1}\\|$ using the true Hessian, and check whether $\\max_t \\delta_t < \\lambda/L$; if it is ever violated, the stated linear-quadratic rate does not apply. Alternatively, attempt the method on a model with $d$ in the millions and check whether constructing the $d \\times d$ posterior Hessian is computationally feasible; failure would falsify the practical claim for deep networks.","supporting_citations":[{"cited_title":"Communication-efficient distributed optimization using an approximate newton-type method,","cited_arxiv_id":null,"evidence_quote":"Supplies the DANE baseline for approximate Newton-type distributed optimization."},{"cited_title":"Federated learning via over- the-air computation,","cited_arxiv_id":null,"evidence_quote":"Defines the AirComp aggregation model that GP-FL uses to build noisy gradient estimators over the wireless channel."},{"cited_title":"Over- the-air federated learning via second-order optimization,","cited_arxiv_id":null,"evidence_quote":"Provides the second-order over-the-air baseline 'Sec-Order' and the Gibbs-sampling device selection reused by GP-FL."},{"cited_title":"GIANT: Globally improved approximate newton method for distributed optimization,","cited_arxiv_id":null,"evidence_quote":"Supplies the GIANT baseline for distributed Newton-type optimization that GP-FL is compared against."},{"cited_title":"Stochastic quasi-newton with line-search regularisation,","cited_arxiv_id":null,"evidence_quote":"Introduces the Gaussian-process model of the quasi-Newton matrix that GP-FL adapts for Hessian estimation."}],"review_version":1}