DP-FedSOFIM: Differentially Private Federated Stochastic Optimization using Regularized Fisher Information Matrix

Mrinmay Sen; Sayantan Banerjee; Sidhant Nair; Tanmay Sen

REVIEW 4 major objections 4 minor 7 references

DP-FedSOFIM: Differentially Private Federated Stochastic Optimization using Regularized Fisher Information Matrix

T0 review · 4 major / 4 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Server-side preconditioning on already-privatized gradients can give differentially private federated learning second-order convergence benefits at no additional privacy cost, using only O(d) memory and communication.

desk verdict Sound privacy-preserving preconditioning idea, but the headline empirical claim is contradicted by the paper's own Table 2(b). read the letter →

arxiv 2601.09166 v3 pith:K6Z7S775 submitted 2026-01-14 cs.LG cs.CRcs.DC

Sidhant Nair , Tanmay Sen , Mrinmay Sen , Sayantan Banerjee This is my paper · ORCID

classification cs.LGcs.CRcs.DC

keywords differentialprivacyfederatedlearningsecond-orderoptimizationFisherinformationmatrixnaturalgradientSherman-Morrisonpost-processingprivacy-utilitytradeoff

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that useful second-order curvature information can be extracted entirely from the privatized gradients already flowing to the server in differentially private federated learning. The proposed method, DP-FedSOFIM, forms a rank-one regularized Fisher proxy from an exponential moving average of aggregated noisy gradients, inverts it in closed form with the Sherman-Morrison identity, and uses it to precondition the parameter update. Because every curvature computation acts only on already-private quantities, the post-processing property of differential privacy guarantees the method inherits the baseline's privacy budget unchanged. Empirical results are reported on CIFAR-10 and PathMNIST across two client scales and privacy budgets from ε=0.5 to ε=10, claiming consistently higher accuracy than first-order, client-side covariance, and variance-reduction baselines. A careful reader would care because this is a concrete route to second-order speedups without the O(d^2) client-side Hessian or covariance storage that previously made such methods impractical.

What carries the argument

The carrying object is the regularized rank-one Fisher proxy Î_t = ρI + M_tM_t^T, where M_t is the exponential moving average of privatized aggregated gradients. Its inverse is maintained implicitly through the Sherman-Morrison identity, H_t = (1/ρ)I − M_tM_t^T/(ρ²+ρ‖M_t‖²), so the preconditioned step H_tG_t costs two inner products and vector operations. The proxy does double duty: the momentum buffer smooths privacy noise (variance reduced by factor (1−β)/(1+β)), and the rank-one term captures the dominant curvature direction, contracting steps along M_t and leaving orthogonal directions scaled by 1/ρ.

What would settle it

Measure the eigenvalue spectrum of the empirical Fisher (gradient outer-product) matrix on the frozen-feature CIFAR-10 and PathMNIST tasks: if the top eigenvalue accounts for a small fraction of the trace, the rank-one proxy cannot be the source of the reported gains. A second check is to run DP-FedSOFIM with M_t replaced by an independent random unit vector scaled to the same norm; if accuracy gains persist, they are not curvature-driven.

Watch

Extended reading notes

Core claim

The central claim is that the privatized gradient stream carries enough curvature information to precondition training. Each round the server updates a momentum buffer M_t = βM_{t-1} + (1−β)G_t from the aggregated noisy gradient, forms the regularized rank-one proxy Î_t = ρI + M_tM_t^T, and updates θ_{t+1} = θ_t − ηH_tG_t with H_t = (ρI + M_tM_t^T)^{-1}. The Sherman-Morrison formula gives H_t in O(d): (1/ρ)I minus a scaled rank-one term, so no matrix is ever materialized. Privacy is structural: H_t is a deterministic function of G_t and the previous server state, so the mapping G_t → H_tG_t is post-processing and inherits the baseline's (ε,δ) guarantee. Convergence is linear to a bias-and-no

Load-bearing premise

The method's practical benefit rests on the premise that the Fisher or gradient-covariance structure is dominated by a single direction, so the rank-one term M_tM_t^T captures useful curvature; if the curvature is diffuse, the preconditioner only rescales the noisy gradient and can amplify noise in early rounds.

Editorial extensions

If this is right

Differentially private federated training can use curvature-aware preconditioning with client-side memory and communication still O(d), removing the main scalability barrier of earlier second-order DP methods.
Privacy accounting becomes modular: any improvement to the underlying noisy-gradient release (better composition, amplification, or clipping) transfers unchanged to the preconditioned method.
Under tight privacy budgets, where first-order updates are noise-dominated, the rank-one rescaling can reach a target accuracy in fewer communication rounds, lowering wall-clock time and client energy use.
The method is most beneficial when the frozen-feature loss landscape is ill-conditioned with concentrated curvature; on such tasks the reported accuracy gains persist across all tested privacy budgets.
Because server-side preconditioning adds no privacy loss, the approach composes cleanly with future privacy accounting advances and with partial-participation federated settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: if curvature is diffuse with no dominant direction, the preconditioner degenerates to a data-dependent scalar rescaling and adds no value; a direct test would compare accuracy against the same algorithm with M_t replaced by a fixed random unit vector of equal norm.
Editorial inference: the observed early-round instability under tight privacy suggests an untested warm-up variant that delays preconditioning until the momentum buffer has stabilized; this would likely preserve the late-round gains while removing the early deficit.
Editorial inference: the rank-one proxy could be extended to rank-k via the Woodbury identity to capture multiple curvature directions, at the cost of O(kd) instead of O(d) — a trade-off the paper does not explore.
Editorial inference: the convergence proof relies on strong convexity or the Polyak–Łojasiewicz condition, which match the frozen linear-head experiments, but extending the claimed gains to full deep-network training would require new analysis since same-step coupling and clipping bias behave differently in non-convex landscapes.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Dear colleague,

You can skip the hype and read the mechanism: the server-side rank-one Fisher proxy built from privatized gradients, with privacy by post-processing, is a legitimate and clean idea. The Sherman-Morrison update makes it O(d), and the proof that no additional privacy loss accrues is correct. That part is worth reading.

The problem is the paper's headline claim. It says DP-FedSOFIM 'consistently outperforms all baselines ... across both datasets, two client scales, and all privacy budgets tested.' Its own Table 2(b) shows the opposite: on CIFAR-10 with 100 clients, DP-FedSOFIM finishes at 63.86% at ε=2, below DP-FedGD (64.13%) and DP-FedFC (64.47%); at ε=5 it trails DP-FedGD (66.15 vs 66.58). So the central empirical claim is not just unproven—it is contradicted by the reported numbers. The results also come from single runs, with hyperparameters selected on final test accuracy, so the margins are not trustworthy.

The theory is honest but limited. It gives linear convergence to an error floor under strong convexity/PL, which is what you'd expect from any DP gradient method; it doesn't show DP-FedSOFIM converges faster than DP-FedGD. The spiked-spectrum justification for the rank-one proxy is plausible but untested—no eigenvalue spectrum is measured.

Where does that leave us? The core idea is a modest but real extension of SOFIM to the private federated setting, and the privacy argument is clean. The experimental section needs a major rework: report error bars, maybe multiple seeds, and tone down the claims to match the data. The writing is competent and the citations are relevant.

This paper deserves a serious referee—not because the conclusions hold, but because the idea has value and the evaluation is fixable. It should not be desk-rejected. My recommendation: send it to review, but the reviewer should require the empirical claims to be scaled back to what the data support.

Referee Report

4 major / 4 minor

Summary. The paper proposes DP-FedSOFIM, a differentially private federated optimization method that builds a rank-one, regularized Fisher information proxy at the server from the privacy-noise-corrupted aggregated gradients. The server applies this proxy as a preconditioner via the Sherman–Morrison formula, incurring only O(d) cost, and the authors argue that all curvature processing is post-processing of the privatized gradients, so the method inherits the DP guarantee of the underlying DP-FedGD mechanism. Theoretical results give linear convergence to a noise-and-bias floor under strong convexity and the PL condition. Experiments on CIFAR-10 and PathMNIST with frozen ResNet-20 features and 20 or 100 clients are presented for ε ∈ {0.5, 1, 2, 5, 10} and the non-private setting; the paper claims that DP-FedSOFIM consistently outperforms DP-FedGD, DP-FedFC, and DP-SCAFFOLD across all these settings.

Significance. If the central empirical claim were supported, the paper would make a useful contribution: a server-side second-order preconditioner with O(d) client and server cost and no additional privacy loss is an appealing design, and the privacy argument via post-processing is conceptually clean. The paper also includes machine-checkable-style proofs of the one-step descent and convergence to an error floor, and it explicitly derives the variance-reduction effect of the momentum buffer. These are genuine strengths. However, the headline claim of consistent, privacy-budget-independent superiority is contradicted by the paper's own tables, and the empirical evaluation lacks the statistical controls needed to support such a strong claim. The spiked-spectrum premise underlying the rank-one proxy is asserted rather than measured. As a result, the evidence for the paper's central contribution is not currently persuasive.

major comments (4)

[§5.3.1 and Table 2(b)] The claim that DP-FedSOFIM 'consistently outperforms' all baselines across all privacy budgets and client scales is directly contradicted by Table 2(b). On CIFAR-10 with 100 clients at ε=2, DP-FedSOFIM's final accuracy is 63.86%, below DP-FedGD's 64.13% and DP-FedFC's 64.47%; at ε=5 it finishes at 66.15%, below DP-FedGD's 66.58%. This is not a minor misstatement: the empirical superiority claim is the paper's central selling point, and Section 6 repeats it verbatim. The headline claim must be corrected or supported with evidence that does not appear in the present tables.
[§5.1 and Tables 2–3] Hyperparameters, including learning rate and clipping threshold, are selected by 'the configuration yielding optimal final test accuracy.' This means the test set is used for model selection, so the reported test accuracies are not an unbiased evaluation of the final configuration. Moreover, no standard deviations or multiple-seed results are reported. The differences at issue (e.g., 63.86 vs 64.13) are far smaller than the likely run-to-run variation in deep-net fine-tuning, so without error bars the claimed margins cannot be statistically distinguished from noise. The paper needs an evaluation protocol with validation-based selection and repeated runs.
[§3.4.2 and §5.3.5] The practical benefit of the rank-one Fisher proxy rests on the 'spiked-spectrum' assumption that gradient/Fisher structure is dominated by one direction. The paper offers qualitative speculation in §5.3.5 about CIFAR-10 being diffuse and PathMNIST being concentrated, but no direct measurement of the spectrum, eigengap, or condition number is provided. Without evidence that the rank-one MtMt^T captures useful curvature under DP noise, the method's advantage over first-order DP-FedGD is not established. A concrete spectral analysis of the effective loss landscape for the frozen-feature models would be needed to support the mechanism claimed.
[§4.5, Theorem 4.17] Theorem 4.17 bounds the error of DP-FedSOFIM, but it does not compare this bound with the corresponding bound for DP-FedGD. The error floor Γ in Eq. (35) contains an extra same-step coupling penalty ηG²maxρ^{-2}M̄², so the theorem does not establish that the preconditioned method converges faster or to a smaller neighborhood than the first-order baseline. Thus the theoretical section does not provide support for the paper's empirical superiority claim; at best it shows the preconditioned update retains linear convergence under assumptions.

minor comments (4)

[Abstract and §6] The abstract and conclusion state that DP-FedSOFIM 'consistently achieves higher accuracy' across all privacy budgets. Given the contradictions in Table 2(b) and the early-round instability acknowledged in §5.3.2, the claims should be hedged to specific regimes and datasets.
[Table 3(a), ε=No DP] The PathMNIST 20-client non-private row shows DP-FedSOFIM at 27.34% at round 10 versus 54.14% for DP-FedGD, yet §5.3.3 claims 'immediate dominance from round 10' in the non-private setting. This inconsistency should be resolved.
[§4, Remark 4.4] The convergence theorems assume strong convexity, but the experiments train a softmax head with no reported ℓ2 regularizer beyond the Fisher regularization ρ. The remark asserts strong convexity 'can be ensured' but does not state that it was, so the link between theory and experiments is incomplete.
[§5.1] The privacy accounting in Appendix E assumes full participation and gives a specific hockey-stick formula, but the experiments report only final accuracy curves; the noise multiplier σg selected for each ε is not listed. Reporting these values would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: server-side preconditioning is a genuine post-processing step and no derived claim reduces to its inputs by construction.

full rationale

The paper's central privacy claim is a direct post-processing argument: the momentum buffer, Fisher proxy, and Sherman-Morrison preconditioner are deterministic functions of already-privatized aggregated gradients, so the (epsilon, delta) guarantee is inherited from the client-side release mechanism (Section 3.4.3, Lemmas 4.25-4.26, Theorem 4.27). This is not circular; it is an application of a standard external theorem. The rank-one Fisher proxy is motivated by the authors' own SOFIM paper (Sen et al., 2024) and by Martens (2020), but the citation is motivational rather than load-bearing: the method's definition, convergence analysis, and experiments are self-contained, and no equation reduces to a fitted parameter or to the cited prior work. The contradiction between the Section 6 'consistently outperforms' claim and Table 2(b), and the test-set-based hyperparameter selection in Section 5.1, are empirical-validity/correctness concerns, not circularity of the derivation. No prediction is defined in terms of the fitted quantity, and no uniqueness or ansatz is imported as proof.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The algorithm needs no invented physics, but its claimed benefit depends on several tuning parameters (eta, C_g, rho, beta) and on the unverified spiked-spectrum premise. Theory adds standard smoothness/convexity/noise assumptions. The privacy-free claim is the cleanest part: it follows from post-processing.

free parameters (4)

per-regime learning rate eta = grid-selected per method/dataset/client/privacy regime (selected values not tabulated)
Selected on final test accuracy (Section 5.1), so reported gains partly reflect optimized step sizes.
gradient clipping threshold C_g = {5, 10}
Tuned per method and dataset; affects both optimization and DP noise scale.
regularization rho = 0.5
Fixed by hand (Section 5.1); controls the preconditioner's isotropic scaling and the convergence constant c_grad.
momentum beta = 0.9
Fixed by hand; sets the variance-reduction factor (1-beta)/(1+beta) and the curvature proxy dynamics.

assumptions (5)

domain assumption Global objective F is L-smooth and mu-strongly convex (or satisfies PL), Assumptions 4.2, 4.3, 4.20
Used for the linear-convergence Theorems 4.17 and 4.22. The experiments train a softmax head without explicit regularization, so strong convexity is not guaranteed.
domain assumption Bounded gradient norm along the iterate sequence, Assumption 4.8
Needed to control the same-step coupling between H_t and G_t in Lemma 4.15; not verified empirically.
ad hoc to paper Gradient-covariance spectrum is spiked, so the rank-one M_t M_t^T captures dominant curvature
Section 3.4.2 motivates the whole preconditioner; no spectrum measurement is given.
standard math Standard DP post-processing and composition theorems (Dwork et al., 2014)
Used to claim no extra privacy cost in Lemmas 4.25-4.26 and Theorem 4.27.
standard math Sherman-Morrison identity
Used for the efficient rank-one inverse update.

how reviews work

0 comments

Cite this review

Pith. "Pith review of DP-FedSOFIM: Differentially Private Federated Stochastic Optimization using Regularized Fisher Information Matrix." pith.science (2026). https://pith.science/paper/K6Z7S775

@misc{pith2026260109166,
  author       = {Pith},
  title        = {Pith review of: DP-FedSOFIM: Differentially Private Federated Stochastic Optimization using Regularized Fisher Information Matrix},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K6Z7S775}},
  note         = {Machine review of arXiv:2601.09166}
}

read the original abstract

Differentially private federated learning (DP-FL) often suffers from slow convergence under tight privacy budgets because the noise required for privacy preservation degrades gradient quality. Although second-order optimization can accelerate training, existing approaches for DP-FL face significant scalability limitations: Newton-type methods require clients to compute Hessians, while feature covariance methods scale poorly with model dimension. We propose DP-FedSOFIM, a simple and scalable Hessian approximation-based second-order optimization method for DP-FL. The method constructs a regularized proxy for the Fisher information matrix at the server using only privatized aggregated gradients, capturing useful curvature information without requiring full Hessian computations or feature covariance estimation. Efficient rank-one updates based on the Sherman-Morrison formula enable communication costs proportional to the model size and require only O(d) client-side memory. Because all curvature and preconditioning operations are performed at the server on already privatized gradients, DP-FedSOFIM introduces no additional privacy cost beyond the underlying privatized gradient release mechanism. Experiments on CIFAR-10 and PathMNIST demonstrate that DP-FedSOFIM converges faster and consistently achieves higher accuracy than several competitive differentially private federated learning baselines across a wide range of privacy budgets.

Figures

Figures reproduced from arXiv: 2601.09166 by the authors.

**Figure 2.** Convergence trajectories on PathMNIST across privacy regimes. DP-FedSOFIM (solid lines) [PITH_FULL_IMAGE:figures/full_fig_p023_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

7 extracted references · 3 linked inside Pith

[1]

Deep learning with differential privacy

Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. InProceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp. 308–318,

2016
[1998]

Galen Andrew, Om Thakkar, Brendan McMahan, and Swaroop Ramaswamy

doi: 10.1162/089976698300017746. Galen Andrew, Om Thakkar, Brendan McMahan, and Swaroop Ramaswamy. Differentially private learning with adaptive clipping.Advances in neural information processing systems, 34:17455–17466,

doi:10.1162/089976698300017746
[2014]

Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečn` y, Sanjiv Kumar, and H Brendan McMahan

URLhttps: //arxiv.org/abs/1301.3584. Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečn` y, Sanjiv Kumar, and H Brendan McMahan. Adaptive federated optimization.arXiv preprint arXiv:2003.00295,

arXiv 2003
[2015]

Communication- efficient learning of deep networks from decentralized data

Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication- efficient learning of deep networks from decentralized data. InArtificial intelligence and statistics, pp. 1273–1282. PMLR, 2017a. H Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang. Learning differentially private recurrent language models.arXiv...

arXiv
[2017]

Mounssif Krouka, Antti Koskela, and Tejas Kulkarni

URLhttps: //arxiv.org/abs/1412.6980. Mounssif Krouka, Antti Koskela, and Tejas Kulkarni. Communication efficient differentially private federated learning using second order information.Proceedings on Privacy Enhancing Technologies,

arXiv
[2020]

doi: 10.1038/s41598-020-69250-1

ISSN 2045-2322. doi: 10.1038/s41598-020-69250-1. URLhttps://doi.org/10.1038/ s41598-020-69250-1. Jack Sherman and Winifred J Morrison. Adjustment of an inverse matrix corresponding to a change in one element of a given matrix.The Annals of Mathematical Statistics, 21(1):124–127,

doi:10.1038/s41598-020-69250-1
[2022]

Maxence Noble, Aurélien Bellet, and Aymeric Dieuleveut

doi: 10.14722/ndss.2022.23054. Maxence Noble, Aurélien Bellet, and Aymeric Dieuleveut. Differentially private federated learning on heterogeneous data. InInternational conference on artificial intelligence and statistics, pp. 10110–10145. PMLR,

arXiv 2022

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [1]

Deep learning with differential privacy

Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. InProceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp. 308–318,

2016

[2] [1998]

Galen Andrew, Om Thakkar, Brendan McMahan, and Swaroop Ramaswamy

doi: 10.1162/089976698300017746. Galen Andrew, Om Thakkar, Brendan McMahan, and Swaroop Ramaswamy. Differentially private learning with adaptive clipping.Advances in neural information processing systems, 34:17455–17466,

doi:10.1162/089976698300017746

[3] [2014]

Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečn` y, Sanjiv Kumar, and H Brendan McMahan

URLhttps: //arxiv.org/abs/1301.3584. Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečn` y, Sanjiv Kumar, and H Brendan McMahan. Adaptive federated optimization.arXiv preprint arXiv:2003.00295,

arXiv 2003

[4] [2015]

Communication- efficient learning of deep networks from decentralized data

Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication- efficient learning of deep networks from decentralized data. InArtificial intelligence and statistics, pp. 1273–1282. PMLR, 2017a. H Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang. Learning differentially private recurrent language models.arXiv...

arXiv

[5] [2017]

Mounssif Krouka, Antti Koskela, and Tejas Kulkarni

URLhttps: //arxiv.org/abs/1412.6980. Mounssif Krouka, Antti Koskela, and Tejas Kulkarni. Communication efficient differentially private federated learning using second order information.Proceedings on Privacy Enhancing Technologies,

arXiv

[6] [2020]

doi: 10.1038/s41598-020-69250-1

ISSN 2045-2322. doi: 10.1038/s41598-020-69250-1. URLhttps://doi.org/10.1038/ s41598-020-69250-1. Jack Sherman and Winifred J Morrison. Adjustment of an inverse matrix corresponding to a change in one element of a given matrix.The Annals of Mathematical Statistics, 21(1):124–127,

doi:10.1038/s41598-020-69250-1

[7] [2022]

Maxence Noble, Aurélien Bellet, and Aymeric Dieuleveut

doi: 10.14722/ndss.2022.23054. Maxence Noble, Aurélien Bellet, and Aymeric Dieuleveut. Differentially private federated learning on heterogeneous data. InInternational conference on artificial intelligence and statistics, pp. 10110–10145. PMLR,

arXiv 2022