{"id":"a4e17a39-5430-4ffa-869b-5f621d046592","arxiv_id":"2608.09074","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"VANEB generalizes nonparametric empirical Bayes to parameter-dependent noise and uses it to personalize federated models by shrinking local estimates toward a learned population prior.","lead":"VANEB is a new statistical method that lets federated learning systems personalize models per client by treating each client's local estimate as a noisy measurement of a hidden ideal, learning the population distribution of those ideals, and shrinking each client's estimate toward it. It derives a generalized Tweedie formula and error bounds, and shows strong accuracy on MNIST and CIFAR-10 personalization benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 4.2's advertised \\~O((nK)^{-1}) oracle bound requires n \\gtrsim K log^d K, which fails in the fixed-total-sample FL regime (small n, large K); the proof appears to need only a much weaker condition, so the stated hypothesis is either a load-bearing typo or the central rate does not apply.","rationale":"The reader's weakest_assumption concerns unknown and estimated covariance functions, which is a real limitation of the framework. That concern is valid, but it is an acknowledged modeling assumption rather than an internal inconsistency. The concern raised here is more immediately load-bearing for the central mathematical claim: Theorem 4.2 is the result that yields the \\~O((nK)^{-1}) excess-risk statement, and its stated hypothesis appears incompatible with the very FL regime used in the empirical sections. Moreover, the proof contains a condition on n that seems much weaker than the theorem's stated condition, indicating either a typo in a load-bearing inequality or an unstated stronger requirement. A reader who wants to rely on the paper's main theoretical contribution must know which condition is correct. If the weaker condition is correct, the manuscript still needs correction, but the central rate may hold in a useful regime; if the stronger condition is correct, the paper's headline claim of federation gain is not supported for large K with bounded local sample sizes. Either way, the verdict should remain conditional pending an analytical check of this condition. I therefore keep the reader's conditional verdict unchanged while flagging a distinct, more formal source of risk than the covariance-estimation gap.","tokens_in":57449,"tokens_out":12782,"duration_ms":119698,"concrete_test":"Re-derive the condition on n from equations (27), (30), and the bullet-bound for E(Q_2^2) in A.4.3, matching M^2/n + log(...)/(nK)(M^{d-2}/K + sum (mu/M)^q) against \\varepsilon_M^2(S_\\bullet,G_0) in (14). Check for d=3, K=1000, n=50 (Figure 5 regime) whether n >= K (\\log K)^3 is satisfied. If the proof's weaker condition is correct, correct the theorem hypothesis to n \\gtrsim K M^{2-d} \\log^{-d/2-1} K; if the stronger condition is genuinely needed, recompute the oracle bound in the K \\gg n regime and verify whether the \\~O(1/(nK)) claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Load-bearing concern: Theorem 4.2 states the oracle denoising inequality under the hypothesis n \\gtrsim K M^{d-2} log^{d/2+1} K, with M \\asymp \\sqrt{\\log K}, i.e. n \\gtrsim K log^d K. The paper's FL experiments (Tables 1-2, Figure 5) use a fixed total sample split among K clients, so the local sample size n \\approx 50,000/K shrinks as K grows; for K=500, n=100, and for K=1000, n=50. In this regime n \\ll K log^d K, so the theorem's hypothesis is false and the special-case rates Vol(S_1)(\\log K)^{...}/(Kn) are not in force. The internal proof in A.4.3 appears to require only n \\gtrsim K M^{2-d} \\log^{-d/2-1} K to control the E(Q_2^2) term, which for d>2 is a much weaker lower bound (for d=3, about K/\\log^3 K rather than K \\log^3 K). These conditions are inconsistent. Either the theorem statement has the exponent reversed, or the proof is missing a stronger step. As written, the central claim of a federation gain of order 1/(nK) is not established in the large-K, bounded-n regime that the paper emphasizes. This is independent of the known-covariance limitation; it concerns the formal hypothesis of the main theorem.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VANEB, a nonparametric empirical Bayes framework for personalized federated learning that treats local M-estimators as noisy observations with a parameter-dependent Gaussian covariance. The authors derive a generalized Tweedie formula for diagonal heteroskedastic covariances, formulate an NPMLE for the shared prior, propose a pseudo-EM algorithm with fixed-point guarantees, and prove non-asymptotic Hellinger rates and an oracle denoising inequality. They then introduce DNN-based extensions, VANEB-Head and VANEB-FT, that personalize the final layer under approximate diagonal Fisher-information covariances, and report experiments on simulations, MNIST, and CIFAR-10. The theoretical results are stated under Assumption 1, which requires the covariance functions to be known and diagonal.","tokens_in":57775,"tokens_out":4418,"duration_ms":43832,"significance":"If the theoretical results are correct, the paper addresses a genuine gap in NPEB methodology: prior work assumes known, parameter-independent covariances, while M-estimators in federated settings naturally produce parameter-dependent, unknown covariance matrices. The generalized Tweedie derivation and the variance-weighted marginal densities are novel and clearly presented, and the oracle denoising inequality would be a strong statement about the benefit of pooling local estimates. The empirical results, especially the balanced-evaluation gains of VANEB-FT, are encouraging. However, the significance is currently conditional: the main theorem's hypothesis appears to be stated with an exponent inconsistent with the proof, and the theory explicitly ignores the covariance-estimation error that is present in both the simulations and the deep-learning applications. These issues prevent the paper, in its present form, from establishing the advertised O((nK)^{-1}) federation gain in the regimes it emphasizes.","major_comments":[{"comment":"Theorem 4.2 states its oracle denoising inequality under the hypothesis n ≳ K M^{d-2} log^{d/2+1} K. Since M ≳ √(log K), this is equivalent to n ≳ K log^d K. However, in the proof's bound on E(Q_2^2), the text requires only n ≳ K M^{2-d} log^{-d/2-1} K, which for d>2 is a much weaker condition (for d=3, roughly K/log^3 K rather than K log^3 K). These two conditions are inconsistent. As written, the theorem's central claim of excess risk of order (1/(nK)) up to polylog factors is not established in the fixed-total-sample FL regime reported in Tables 1-2 and Figure 5, where n ≈ 50,000/K and the theorem's hypothesis fails for large K. The authors should either correct the theorem statement or supply the additional proof steps needed to justify the stated condition.","section":"Section 4.2 and Appendix A.4.3"},{"comment":"Assumption 1 requires the covariance function Σ_k(·) to be known and diagonal, and the working model (1) treats the local estimator as exactly Gaussian with this covariance. The paper explicitly states in Section 2 that 'We ignore the added uncertainty due to this here,' referring to clients reporting estimated covariance functions. This is load-bearing because the theory's Hellinger rates and oracle inequality do not cover the estimated-covariance case, yet the logistic and Poisson simulations use full, data-dependent Fisher information matrices and the DNN experiments use the estimated diagonal covariance in (20). The paper therefore does not currently provide a theoretical guarantee for the actual procedure being proposed for federated learning. The authors should either add an analysis of covariance-estimation error or explicitly restrict the theoretical claims to the known-covariance setting and present the practical method as a heuristic.","section":"Assumption 1 and Section 2"},{"comment":"In the special-case proof for Gaussian mixture priors, the text argues that the term involving the tail mass 'becomes negligible as K is assumed to diverge' without giving a quantitative rate. Since this special case is one of the headline Õ((nK)^{-1}) claims, the proof should explicitly track the dependence on the mixture parameters A*, γ, the radius r, and K, and show how Vol(S_r) arises in the stated bound. As written, the GMM case is not fully established.","section":"Section 4.2, Gaussian mixture case"}],"minor_comments":[{"comment":"There is a typo in the sentence defining the marginal law: 'F zk(z)and and define' should read 'F_{z_k}(z) and define'.","section":"Section 2"},{"comment":"The notation θ̂_k is reused for both the original local estimator and its rescaled version √(n_k) θ̂_k. This creates confusion in the likelihood expressions and in Algorithm 1; distinct symbols should be used.","section":"Section 2.2 and Algorithm 1"},{"comment":"The text says that 'contraction is controlled by Brouwer's fixed-point theorem,' but Brouwer's theorem gives existence of a fixed point, while contraction gives uniqueness and linear rate; the wording should distinguish these two ingredients.","section":"Section 3.1.3"},{"comment":"The axis label 'Log x' in the K-sweep panels is unclear; it appears to indicate a logarithmic horizontal axis, not a quantity named x. Please relabel for clarity.","section":"Figures 2-4"}],"recommendation":"major_revision","confidential_remarks":"The paper has substantial merit in its conceptual framing and the empirical results are interesting. However, the mismatch between Theorem 4.2's stated hypothesis and the proof's actual condition is a load-bearing issue that must be resolved before the main theoretical claim can be accepted. The known-covariance limitation is honestly acknowledged, but it sharply limits the applicability of the theory to the proposed FL method. I would encourage the authors to either fix the theorem statement and verify the proof, or add a careful covariance-estimation analysis; without one of these, the central claim of a federation gain of order (nK)^{-1} is not established."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, this paper is worth your attention. The core new idea is a generalized Tweedie formula for parameter-dependent covariance and an NPMLE theory that handles it. That genuinely extends the fixed-covariance results of Soloff et al. and Saha-Guntuboyina. The oracle denoising inequality is a real contribution, and the pseudo-EM algorithm is a sensible computational template. The simulation work is careful, and the DNN extension is honest about being heuristic.\n\nThe soft spots are real, and one is load-bearing. The stated hypothesis in Theorem 4.2 is n ≳ K M^{d-2} log^{d/2+1} K, i.e., n ≳ K log^d K, but the proof in A.4.3 only requires n ≳ K M^{2-d} log^{-d/2-1} K, which for d>2 is much weaker (roughly K / log^d K). Those can't both be right. If the proof's condition is the intended one, then the theorem actually applies to the fixed-total-sample FL experiments (n ≈ 50,000/K), and the \\~(O)((nK)^{-1}) claim is salvageable. If the theorem's condition is intended, then the main FL claim is not established in the regime the paper highlights. Either way the authors need to correct one of the two and verify the proof matches the statement. This is not a minor typo; it is the hypothesis of the central theorem.\n\nBeyond that, the known-covariance assumption is explicitly set aside for estimators, and the simulations and DNN variants use estimated or full covariances outside the diagonal theory. That is a standard theory-practice gap, but it should be acknowledged more directly. The deep learning experiments also lack error bars and code, which weakens the empirical claims. And there is no guarantee that pseudo-EM actually reaches the log-likelihood gap required by the theorems; Lemma 3.3 shows fixed-point convergence, not the specific likelihood condition.\n\nWho should read this? Statisticians working on empirical Bayes, NPMLE, and compound decision problems, and FL researchers who want a principled prior-learning framework. The paper deserves a serious referee, but the referee should hold the authors to fixing the Theorem 4.2 inconsistency and either proving the weaker condition is sufficient or adjusting the claim. I would not cite it until that is resolved, but I would definitely send it to review.","headline":"A serious NPEB paper with a real contribution, but Theorem 4.2's hypothesis and proof are inconsistent and the main FL claim is not established until that is fixed.","tokens_in":58361,"tokens_out":3611,"would_cite":false,"duration_ms":30706,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62C12","62G05","62F15"],"pacs":[],"model":"deepseek-v4-flash","headline":"Personalized federated learning can be cast as a variance-aware empirical Bayes problem: with a generalized Tweedie formula, the excess risk over the oracle Bayes estimator is near $\\tilde{O}((nK)^{-1})$ up to polylog factors.","keywords":["federated learning","personalization","nonparametric empirical Bayes","nonparametric maximum likelihood","Tweedie's formula","heteroskedasticity","oracle denoising inequality","deep neural networks"],"falsifier":"Simulate the paper's quadratic-variance scenario with the covariance function known exactly, as Assumption 1 requires, and measure the average squared excess risk against the true oracle posterior mean at fixed $n$ while $K$ grows; Theorem 4.2 predicts decay like $(\\log K)^{3\\vee(d/2-1)}\\varepsilon_M^2/n$ with $\\varepsilon_M^2\\approx (\\log^{d/2+1}K)/K$ for compact support. If the empirical excess risk fails to decrease with $K$ once the prior is nontrivial, the oracle denoising inequality is refuted.","tokens_in":57185,"feed_emoji":"📊","tokens_out":15571,"duration_ms":126300,"temperature":0.7,"pith_summary":"The paper tries to establish that personalized federated learning across heterogeneous clients is a compound decision problem: each client's asymptotically normal local estimator is a noisy observation of a client-specific parameter, all parameters are drawn from one unknown shared prior, and the server only has to learn that prior from the transmitted summaries. Its procedure, VANEB, handles the realistic case where the local noise covariance depends on the unknown parameter, which standard nonparametric empirical Bayes cannot. The main quantitative claim is an oracle denoising inequality: after substituting the estimated prior into a generalized Tweedie formula, the average squared error across clients is within roughly $\\tilde{O}((nK)^{-1})$ (up to polylogarithmic factors in $K$) of the infeasible oracle posterior mean, in discrete, compact-support, and Gaussian-mixture settings. If this bound is right, each client's personalized estimate can be almost as good as the posterior mean under the true population of client parameters, and the entire federation gain comes from learning that population. The paper also gives neural-network extensions, VANEB-Head and VANEB-FT, and reports that the fine-tuned variant has the highest reported accuracy under balanced evaluation on MNIST and CIFAR-10.","feed_headline":"Variance-aware shrinkage makes federated personalization near-oracle","feed_subtitle":"Clients get oracle-Bayes accuracy even when each local dataset is small, as long as the server learns the shared prior.","key_machinery":"The load-bearing object is the variance-aware NPMLE for the unknown prior $G_0$, computed by a pseudo-EM algorithm that alternates between fixed-covariance ridgeline updates and re-evaluating the covariance function at the updated atoms. The identity that carries the argument is the generalized Tweedie formula, which expresses the oracle posterior mean through variance-weighted marginal densities; the coordinate weights $\\sigma_{k,i}(\\theta)^2$ make shrinkage automatically stronger where the local estimate is least reliable. The oracle inequality then routes through the Hellinger convergence of both the ordinary and variance-weighted marginals, with the condition number $\\tau=\\bar{s}/s$ controlling the transfer between them. In the deep-learning extension, the same operation is applied only to the last-layer heads, using a diagonal Fisher-information estimate of the head covariance.","core_discovery":"The paper's central discovery is a generalized Tweedie formula for heteroskedastic Gaussian observations whose covariance matrix is a known function of the unknown parameter. Because the covariance cannot be pulled out of the integral as in the classical formula, the paper defines variance-weighted marginal densities $\\tilde{f}_{k,G,i}(\\hat\\theta_k)=\\int \\sigma_{k,i}(\\theta)^2\\varphi^{(k)}(\\hat\\theta_k;\\theta)\\,dG(\\theta)$ and shows that the oracle Bayes rule is $\\theta_k^o=\\hat\\theta_k/\\sqrt{n_k}+q_{k,G_0}(\\hat\\theta_k)/(\\sqrt{n_k}f_{k,G_0}(\\hat\\theta_k))$, where $q_{k,G}$ is the integral of the covariance-weighted likelihood score. Estimating $G_0$ by the nonparametric maximum likelihood estimator and plugging it into this formula gives the VANEB estimator. The proof transfers Hellinger accuracy of the NPMLE to the variance-weighted densities through the condition number $\\tau=\\bar{s}/s$, then converts that into the oracle denoising inequality of Theorem 4.2, whose excess risk is $(\\log K)^{3\\vee(d/2-1)}\\varepsilon_M^2(S_\\bullet,G_0)/n$; in the discrete, compact-support, and Gaussian-mixture cases this is $\\tilde{O}((nK)^{-1})$.","pith_inferences":["The paper sets aside the uncertainty from estimating the covariance function, so a natural follow-up is a bound with an extra covariance-estimation error term; if that term does not vanish, the advertised $(nK)^{-1}$ rate would degrade.","The diagonal-covariance assumption is violated by the paper's own full-covariance simulations and by the DNN head estimator, so the empirical success there suggests the variance-aware mechanism is more robust than the proved regime, not that the theory covers it.","A concrete testable extension is to use VANEB as a one-shot server-side post-processor for any asymptotically normal local summary, including differentially private summaries, and to measure how much privacy noise the oracle inequality tolerates.","The balanced-evaluation pattern implies that empirical-Bayes shrinkage acts as a regularizer that keeps local heads from collapsing onto the client's observed label mixture; a direct ablation varying the number of NPMLE atoms would show how much of that coverage comes from prior-learning rather than simple averaging."],"forward_implications":["Under the paper's Assumption 1, VANEB's average squared excess risk relative to the oracle posterior mean is $O((\\log K)^{3\\vee(d/2-1)}\\varepsilon_M^2/n)$, so more clients improve personalization even when every local sample size is fixed.","For discrete, compactly supported, and Gaussian-mixture priors the bound becomes $\\tilde O((nK)^{-1})$, so the federation gain enters entirely through learning the shared prior.","Local M-estimators used without pooling have excess risk that cannot fall below the oracle baseline's $1/n$-scale floor as $K$ grows; in VANEB's bound only the prior-estimation term $\\varepsilon_M^2$ carries the $K$-dependence.","Any approximate NPMLE whose log-likelihood gap is within the rate $\\varepsilon_M^2$ inherits the same Hellinger accuracy, which justifies the discretized support-constrained solver used in computation.","In the reported balanced evaluations on MNIST and CIFAR-10, VANEB-FT records the highest accuracy among the compared personalized methods, indicating that the shrinkage prevents local adaptation from discarding globally informative classes."],"supporting_citations":[{"why":"supplies the fixed-covariance multivariate NPEB framework and ridgeline-support machinery that VANEB extends to parameter-dependent variances.","marker":"[Soloff et al., 2025]"},{"why":"provides the Gaussian location-mixture NPMLE Hellinger rates and the regularized empirical-Bayes proof template used for the oracle inequality.","marker":"[Saha and Guntuboyina, 2020]"},{"why":"formulates the compound-decision empirical Bayes framework that motivates treating local estimates as noisy draws from a shared prior.","marker":"[Robbins, 1956]"},{"why":"gives Tweedie's formula and empirical Bayes modeling strategies that the paper generalizes to parameter-dependent covariance.","marker":"[Efron, 2014]"},{"why":"provides the ridgeline manifold theorem for Gaussian mixtures used as the fixed-covariance computational primitive in Lemma 3.2.","marker":"[Ray and Lindsay, 2005]"},{"why":"Berry-Esseen bounds for M-estimators justify the Gaussian working model for the local estimates.","marker":"[Shao and Zhang, 2022]"},{"why":"asymptotic normality of M-estimators supplies the starting distribution in the working model (1).","marker":"[Van der Vaart, 2000]"},{"why":"supplies the support-constrained NPMLE discretization used in the algorithm and Proposition 3.1.","marker":"[Koenker and Mizera, 2014]"},{"why":"gives the diagonal Fisher-information covariance approximation used for the neural-network head estimator in equation (20).","marker":"[Jhunjhunwala et al., 2024]"},{"why":"provides Federated Averaging, the baseline that the neural-network variants use for backbone aggregation and the main comparison method.","marker":"[McMahan et al., 2017a]"}],"fun_headline_variants":["VANEB: near-oracle federated learning without known variances","Generalized Tweedie formula unlocks heteroskedastic Bayes shrinkage","Variance-aware shrinkage: federated personalization gets oracle accuracy","Heteroskedastic NPEB personalizes federated models near-oracle","VANEB: shrink local models with variance-aware Bayes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is the working model that each client's rescaled local estimator is exactly Gaussian around the true parameter with a known, diagonal covariance function of that parameter; in practice that covariance must be estimated from local data, and the paper explicitly sets this added uncertainty aside.","fun_headline_variants_meta":{"raw":{"variants":["VANEB: near-oracle federated learning without known variances","Generalized Tweedie formula unlocks heteroskedastic Bayes shrinkage","Variance-aware shrinkage: federated personalization gets oracle accuracy","Heteroskedastic NPEB personalizes federated models near-oracle","VANEB: shrink local models with variance-aware Bayes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000626,"raw_usage":{"total_tokens":2966,"prompt_tokens":1087,"completion_tokens":1879,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":703,"completion_tokens_details":{"reasoning_tokens":1788}},"tokens_in":703,"tokens_out":1879,"duration_ms":14033,"temperature":1.0,"reasoning_tokens":1788,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:06:40.092942+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate the paper's quadratic-variance scenario with the covariance function known exactly, as Assumption 1 requires, and measure the average squared excess risk against the true oracle posterior mean at fixed $n$ while $K$ grows; Theorem 4.2 predicts decay like $(\\log K)^{3\\vee(d/2-1)}\\varepsilon_M^2/n$ with $\\varepsilon_M^2\\approx (\\log^{d/2+1}K)/K$ for compact support. If the empirical excess risk fails to decrease with $K$ once the prior is nontrivial, the oracle denoising inequality is refuted.","supporting_citations":[{"cited_title":"An empirical","cited_arxiv_id":null,"evidence_quote":"formulates the compound-decision empirical Bayes framework that motivates treating local estimates as noisy draws from a shared prior."}],"review_version":1}