Pith. sign in

REVIEW 5 major objections 5 minor 38 references

pFedSOP : Accelerating Training Of Personalized Federated Learning Using Second-Order Optimization

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read pFedSOP replaces first-order local updates in personalized federated learning with a Fisher-information-matrix second-order step, claiming faster training in fewer communication rounds at FedAvg-level communication cost.

desk verdict The paper's second-order story collapses under inspection: the update is a scalar-scaled gradient step and the convergence theorem is vacuous, though the Gompertz aggregation idea and reported empirical gains are not nothing. read the letter →

arxiv 2506.07159 v1 pith:CBIJHCQL submitted 2025-06-08 cs.DC cs.LG

Mrinmay Sen , Chalavadi Krishna Mohan This is my paper
classification cs.DCcs.LG
keywords PersonalizedfederatedlearningSecond-orderoptimizationFisherinformationmatrixGompertzfunctionDataheterogeneityCommunicationefficiencyImageclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes pFedSOP, a personalized federated learning algorithm that replaces first-order local updates with a cheap second-order step. Each client blends its previous local gradient update with the global gradient update using a Gompertz-function weight computed from the angle between them, then uses a regularized Fisher information matrix built from that blended gradient as a stand-in for the Hessian. The Sherman–Morrison formula lets the client compute the inverse update in linear time without storing the Hessian, so the communication pattern stays similar to FedAvg: only gradient updates travel between clients and server. The central claim is that this accelerates training of per-client models and improves test accuracy within a fixed communication budget compared with first-order FL and PFL baselines, at per-round cost close to FedAvg.

What carries the argument

The two load-bearing pieces are the regularized Fisher information matrix $\hat{F}_{it} = \Delta^p_{it}{\Delta^p_{it}}^T + \rho I$, used as a stand-in for the Hessian under the FIM-Hessian equivalence for probabilistic objectives such as cross-entropy, and the Sherman–Morrison inversion $\hat{F}_{it}^{-1} = \frac{I}{\rho} - \frac{\Delta^p_{it}{\Delta^p_{it}}^T}{\rho^2 + \rho\,\Delta^p_{it}^T\Delta^p_{it}}$, which turns the second-order step into an $O(d)$ vector operation. The Gompertz-normalized angle between local and global gradient updates determines the aggregation weight $\beta$, deciding how much global information each client pulls into its personalized gradient. Together these pieces let each client take a curvature-aware step with no Hessian storage and no extra forward passes over local data.

What would settle it

Train a small two-layer network on a single client's non-quadratic cross-entropy objective and compute, at each pFedSOP update, the residual $x^*_i - (x_{i(t-1)} - (\nabla^2 P_i(x_{i(t-1)}))^{-1}\Delta^p_{it})$ from Eq. (23); on a non-quadratic objective this residual will not be zero, showing that the proof's load-bearing premise fails even if the algorithm still trains well. A direct test of the advertised convergence rate is to record $\|x_{it} - x^*_i\|$ over rounds on a quadratic local objective, where the bound should hold, and compare it with the same quantity on the real cross-entropy objective, where the proof's premise is violated.

Watch

Extended reading notes

Core claim

pFedSOP's central claim is that second-order optimization can be made practical in personalized federated learning by approximating each client's Hessian with a rank-one-plus-regularizer Fisher information matrix built from a personalized gradient update. Because the matrix has the form $\rho I + \Delta^p_{it}{\Delta^p_{it}}^T$, its inverse is available in closed form via the Sherman–Morrison formula, so the per-client update $x_{it} = x_{i(t-1)} - \eta_1(\rho I + \Delta^p_{it}{\Delta^p_{it}}^T)^{-1}\Delta^p_{it}$ costs $O(d)$ extra local computation rather than $O(d^3)$ for a true Hessian inverse. The personalized gradient $\Delta^p_{it}$ is a weighted blend of the local and global gradient updates, with weight $\beta = 1 - e^{-e^{-\lambda(\theta-1)}}$ determined by the angle between them, which is how client-specific global information enters the local step. The paper proves a per-model error bound $\|x_{it} - x^*_i\| \le \varepsilon_1\|x_{i(t-1)} - x^*_i\| + \varepsilon_2\|x_{i(t-1)} - x^*_i\|^2$, which it interprets as a linear-plus-quadratic convergence guarantee, and reports higher average test accuracy than the compared baselines within 100 communication rounds on CIFAR-10, CIFAR-100, and Tiny ImageNet under two heterogeneous partition schemes.

Load-bearing premise

The convergence proof assumes that one exact Newton step from the previous model lands on the local optimum, an identity that holds only for quadratic objectives; the experiments train deep networks with cross-entropy loss, which is not quadratic.

Editorial extensions

If this is right

  • If the central claim holds, personalized federated image classification can reach a target accuracy in fewer communication rounds than first-order PFL baselines, because each local step carries curvature information rather than only gradient information.
  • The per-client local computation stays at $O(N_i d + 2d)$ rather than the $O(N_i d + N_i d)$ incurred by methods that feed extra data during personalization, so the speed-up does not come from extra local forward passes.
  • Since only local and global gradient updates are exchanged, per-round communication cost matches FedAvg's $O(2d)$, making the approach usable on clients that cannot store or transmit a Hessian.
  • The method is limited to probabilistic objectives such as categorical cross-entropy, because the FIM-as-Hessian substitution rests on that equivalence; the paper states this limitation explicitly.
  • The sensitivity study indicates that the regularization parameter $\rho$ should stay near the learning rate in magnitude, while $\lambda$ in the tested range matters little, so the method needs only light tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implicit testable consequence is that the method's advantage over first-order baselines should be smallest on objectives that are nearly linear or exactly quadratic, where curvature information adds little; a synthetic PFL benchmark with quadratic local losses would isolate the contribution of the curvature step from the Gompertz aggregation.
  • Because the blended gradient weight depends only on the angle between local and global updates, not their magnitudes, a natural extension is a scale-aware blend; the paper does not explore this.
  • Since the update is a rank-one perturbation of a scaled identity, it acts like a preconditioner on the personalized gradient; comparing pFedSOP against momentum or adaptive-gradient optimizers under the same communication budget would test how much of the empirical gain is genuinely second-order rather than a generic acceleration effect.
  • The proof's Eq. (23) premise means the advertised linear-quadratic bound is strictly a quadratic-objective statement; if the empirical gains persist on non-quadratic networks, the mechanism is likely the personalized aggregation plus the regularized step acting as a preconditioner, not the quadratic-convergence guarantee as written.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes pFedSOP, a personalized federated learning algorithm whose claimed mechanism is FIM-based second-order optimization. Each client forms a personalized gradient update by a Gompertz-weighted combination of the previous local and global gradient updates, then constructs a regularized Fisher Information Matrix from that update and applies a Sherman-Morrison inversion to update the personalized model. The paper claims a linear-quadratic convergence guarantee for the local models and reports experimental results on CIFAR-10, CIFAR-100, and Tiny ImageNet under two heterogeneity settings, claiming fewer communication rounds and higher accuracy than several FL and PFL baselines.

Significance. If the central claims were valid, the paper would offer a communication-efficient personalized FL method with convergence faster than first-order methods and computation comparable to FedAvg. The paper is clearly written and reproduces standard algorithm descriptions, and its experimental tables cover several datasets and baselines. However, the load-bearing mathematical claims fail: the Sherman-Morrison update collapses to a scalar-scaled gradient step, so the method is not second-order in any meaningful sense; the convergence proof assumes a one-step Newton optimality that does not hold for the non-quadratic objectives used; and the stated error bound has a contraction factor greater than one. No code or supplementary proofs are provided. Because the central mechanism and the theoretical guarantee are both invalid as written, the paper does not establish its claimed contribution.

major comments (5)
  1. [Section IV-D, Eq. (18)] The Sherman-Morrison inversion in Eq. (18) reduces exactly to (Δp_it Δp_it^T + ρI)^{-1} Δp_it = Δp_it / (ρ + ||Δp_it||^2). Substituting into Eq. (19) gives x_it = x_i(t-1) - (η1/(ρ + ||Δp_it||^2)) Δp_it, which is a gradient step with a scalar, norm-dependent step size. The regularized FIM is a rank-one-plus-isotropic matrix, and when acting on the same vector Δp_it it cannot change the update direction. No curvature or eigen-direction information enters the update, so the algorithm is not second-order optimization and the stated acceleration mechanism is absent.
  2. [Section IV-E, Eq. (23)] Eq. (23) asserts that x*_i = x_i(t-1) - (∇^2 P_i(x_i(t-1)))^{-1} Δp_it for the update built with the approximate FIM. This equality is an exact one-step Newton optimality condition, which holds only for quadratic objectives (or in very special cases) and is not valid for categorical cross-entropy with ResNet models as used in the experiments. The subsequent substitution Δp_it = ∇^2 P_i(x_i(t-1)) e_{t-1} relies entirely on Eq. (23), so the proof assumes a strong form of the conclusion it is meant to establish.
  3. [Section IV-E, Theorem 1] The derived bound ||e_t|| ≤ ε1 ||e_{t-1}|| + ε2 ||e_{t-1}||^2 has ε1 = 1 + Γη1/ρ + Γη1||Δp||^2/(ρ^2 + ρ||Δp||^2), which is strictly larger than 1 for every positive Γ, η1, ρ and every nonzero Δp. A contraction factor greater than one cannot establish linear convergence, and the quadratic term amplifies large errors rather than controlling them. The claimed 'linear-quadratic convergence guarantee' is therefore vacuous and provides no support for the paper's central claim of faster training with fewer communication rounds.
  4. [Section IV-E] The convergence analysis is entirely per-client and treats the personalized gradient Δp_it as given. It does not model the server aggregation, partial client participation, stochastic local updates, or the dependence of Δp_it on the global gradient Δ(t-1). Even if the local bound were correct, it would not by itself establish the federated claim of reduced communication rounds, because the global interaction that drives personalization is not part of the analysis.
  5. [Section V-C, Table II] The headline accuracy metric is the per-client maximum test accuracy over all 100 communication rounds, averaged across clients. This does not measure the performance of a model actually produced at the end of training and can favor methods that pass through good points during optimization. Final-round accuracy or area under the round-accuracy curve should also be reported. The implementation reports a single seed (seed = 0) with no variance or statistical comparison, so the accuracy gaps shown in Table II are not established as significant.
minor comments (5)
  1. [Section II] 'Exiting methods' should be 'Existing methods'; the same typo appears in multiple places, including Section II-A and the experimental discussion.
  2. [Algorithms 2 and 3] T is used for the number of local iterations in Algorithm 2 but for the number of communication rounds in Algorithm 3; this notation conflict is confusing and should be resolved.
  3. [Section III-B, Eq. (8)] The paper writes F = E[g_i g_i^T] = E[g_i]E[g_i]^T + ρI, but the equality is not generally true; E[gg^T] = E[g]E[g]^T + Cov(g). The approximation and its assumptions should be stated explicitly.
  4. [Section IV-D] The symbol Δit is used both for the local gradient update returned by Algorithm 2 and for the model update step in Eq. (18); using distinct symbols would avoid ambiguity.
  5. [Section V-B.4] The text says 'we use same initialization and same settings for all the methods', but it does not specify whether the initialization is identical across the compared methods or how partial client selection is synchronized; this should be clarified for reproducibility.

Circularity Check

3 steps flagged · score 7.0 of 10

pFedSOP's 'second-order' update collapses by Sherman–Morrison to a scalar-scaled gradient step, and Theorem 1's proof assumes the one-step Newton optimality it aims to prove.

  1. renaming known result [Section IV-D, Eq. 18 (used in Eq. 19)]
    "∆it = Fit −1∆p it = [∆p it∆p it T + ρI] −1 ∆p it = ∆p it ρ − ∆p it∆p it T ∆p it ρ2 + ρ∆p it T ∆p it (18)"

    Applying the Sherman–Morrison formula used in this same derivation gives (Δp_itΔp_it^T + ρI)^{-1}Δp_it = Δp_it/(ρ + ||Δp_it||²), so Eq. 19 becomes x_it = x_{i(t-1)} − [η1/(ρ + ||Δp_it||²)]Δp_it. This is a scalar-scaled first-order gradient step: the 'regularized FIM' is rank-one plus isotropic, and acting on its own generating vector it contributes no curvature or eigen-direction information. The paper's central claim that Eq. 18 'efficiently utilizes second-order optimization' is therefore a renaming of a norm-scaled gradient step; the second-order mechanism is absent by construction.

  2. self definitional [Section IV-E, Eq. 23 in the proof of Theorem 1]
    "From the update rule of second-order optimization with exact Hessian, we can write that: x∗ i = xi(t−1) − (∇2Pi(xi(t−1)))−1 ∆p it (23)"

    Eq. 23 asserts that an exact Newton step from x_{i(t-1)} reaches the local optimum x*_i in one step, a property that holds only for quadratic objectives or when already at the optimum. The stated Assumptions 1–2 (twice differentiability and L-Lipschitz Hessian) do not imply it. The proof then uses Eq. 23 to substitute Δp_it = ∇²P_i(x_{i(t-1)}) e_{t-1} into the error recurrence and obtains the claimed linear-quadratic bound. Thus the theorem's conclusion is derived from an unstated premise that already contains the one-step optimality result; the premise is the conclusion restated for the exact Newton step, not derived from the pFedSOP update being analyzed.

1 more flagged steps
  1. ansatz smuggled in via citation [Section III-B, Eq. 8; used in Section IV-D, Eq. 17]
    "we replace the true FIM with a regularized FIM, motivated by the paper of SOFIM [12]. The regularized FIM is defined as follows : F = E[gigiT ] ≡ E[gi]E[gi]T + ρI"

    The regularized FIM is adopted from SOFIM, a prior work by the same first author, as an unproved definition rather than a derived or independently validated quantity. This self-cited ansatz is the load-bearing premise of the method: combined with Sherman–Morrison it forces the 'second-order' update to be a scalar-scaled gradient step (Eq. 18). The paper's central contribution therefore rests on a self-citation that supplies the very definition that eliminates the curvature information the method claims to use.

full rationale

The experimental comparisons are not circular: pFedSOP is tested against FedAvg, FedProx, Ditto, FedRep, FedALA, and FedDWA on standard benchmarks with shared initialization and fixed seed, which is independent evidence for the empirical accuracy gains. However, the paper's theoretical and mechanistic claims are circular. First, the 'FIM-based second-order' update of Eq. 18 reduces algebraically to a scalar-scaled gradient step, so the acceleration claim is a renaming of a first-order method with a norm-dependent step size. Second, Theorem 1's proof assumes one-step Newton optimality in Eq. 23, an unstated version of the convergence conclusion that the theorem purports to prove. Third, the regularized FIM definition is imported from the authors' own SOFIM paper as an unproved ansatz, and that ansatz is what causes the update to collapse to a first-order step. Because the central claimed mechanism reduces by construction to its own input—a rank-one-plus-isotropic FIM built from the update vector—the circularity score is 7.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method fits several hyperparameters (ρ, λ, learning rates) on the benchmark datasets and relies on a rank-one FIM approximation whose Hessian-equivalence is not justified at non-optimum points.

free parameters (4)
  • ρ (FIM regularization) = 1 (default); sensitivity over {1, 0.1, 0.01, 0.001}
    Controls the diagonal regularization of the FIM approximation; affects update magnitude and convergence. Tuned on the evaluation datasets.
  • λ (Gompertz steepness) = 1 (default); sensitivity over {5, 2.5, 1, 0.5}
    Controls the sharpness of the angle-to-weight mapping in the personalized gradient aggregation. Tuned on CIFAR-100.
  • η1 (personalization learning rate) = selected from {1, 0.1, 0.01, 0.001, 0.0001} per method
    Step size for the FIM-based personalized model update. Chosen per method as 'best' based on test accuracy, no fixed value reported.
  • η2 (local SGD learning rate) = selected from same set, per method
    Step size for local training iterations. Also selected per method by best performance.
assumptions (5)
  • domain assumption Equivalence of FIM and Hessian for probabilistic objectives (cited [13])
    The method assumes the regularized FIM [Δ^p Δ^p^T + ρI] approximates the Hessian of the local cross-entropy objective, though the equivalence holds only for the true distribution at the optimum, not for the rank-one empirical approximation used here.
  • standard math Assumption 1: local objective twice differentiable
    Section IV-E, used for the Hessian-based convergence proof.
  • standard math Assumption 2: Hessian of local objective is L-Lipschitz
    Section IV-E, used to bound the quadratic term in the convergence proof.
  • ad hoc to paper The Newton step from x_{i(t-1)} lands exactly on the local optimum (Eq. 23)
    The proof substitutes x*_i = x_{i(t-1)} - (∇²P_i(x_{i(t-1)}))^{-1} Δ^p_{it}, which is only exact for quadratic objectives. This is the invalid step in Theorem 1.
  • domain assumption Stale local gradient updates from previous rounds are valid in partial participation
    When a client is not sampled, the latest local gradient update from an earlier round is treated as Δ_{i(t-1)} (Section IV, Algorithm 3), without a staleness analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of pFedSOP : Accelerating Training Of Personalized Federated Learning Using Second-Order Optimization." pith.science (2026). https://pith.science/paper/CBIJHCQL

@misc{pith2026250607159,
  author       = {Pith},
  title        = {Pith review of: pFedSOP : Accelerating Training Of Personalized Federated Learning Using Second-Order Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CBIJHCQL}},
  note         = {Machine review of arXiv:2506.07159}
}
read the original abstract

Personalized Federated Learning (PFL) enables clients to collaboratively train personalized models tailored to their individual objectives, addressing the challenge of model generalization in traditional Federated Learning (FL) due to high data heterogeneity. However, existing PFL methods often require increased communication rounds to achieve the desired performance, primarily due to slow training caused by the use of first-order optimization, which has linear convergence. Additionally, many of these methods increase local computation because of the additional data fed into the model during the search for personalized local models. One promising solution to this slow training is second-order optimization, known for its quadratic convergence. However, employing it in PFL is challenging due to the Hessian matrix and its inverse. In this paper, we propose pFedSOP, which efficiently utilizes second-order optimization in PFL to accelerate the training of personalized models and enhance performance with fewer communication rounds. Our approach first computes a personalized local gradient update using the Gompertz function-based normalized angle between local and global gradient updates, incorporating client-specific global information. We then use a regularized Fisher Information Matrix (FIM), computed from this personalized gradient update, as an approximation of the Hessian to update the personalized models. This FIM-based second-order optimization speeds up training with fewer communication rounds by tackling the challenges with exact Hessian and avoids additional data being fed into the model during the search for personalized local models. Extensive experiments on heterogeneously partitioned image classification datasets with partial client participation demonstrate that pFedSOP outperforms state-of-the-art FL and PFL algorithms.

Figures

Figures reproduced from arXiv: 2506.07159 by the authors.

Figure 1
Figure 1. Flow diagram of the proposed pFedSOP, illustrating the different operations performed at each client [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Comparison of various methods in terms of communi [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Comparison of various methods in terms of communi [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Effect of personalization component (PC) on the [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Effect of ρ and λ on the performance of pFedSOp in term of communication round wise average training loss across all the available clients. 3) Parameters Sensitivity Analyses: pFedSOP has two ac￾tive hyperparameters. One is λ of the Gompertz function, which is used to …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 30 canonical work pages

  1. [1]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL, USA , vol. 54. PMLR, 2017, pp. 1273–1282

    work page 2017

  2. [2]

    Advances and open problems in federated learning,

    P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. A. Bonawitz, Z. Charles, G. Cormode, R. Cummings, R. G. L. D’Oliveira, H. Eichner, S. E. Rouayheb, D. Evans, J. Gard- ner, Z. Garrett, A. Gasc ´on, B. Ghazi, P. B. Gibbons, M. Gruteser, Z. Harchaoui, C. He, L. He, Z. Huo, B. Hutchinson, J. Hsu, M. Jaggi, T. Javidi, G. Joshi, M. K...

    work page 2021

  3. [3]

    Federated learning for generalization, robustness, fairness: A survey and benchmark,

    W. Huang, M. Ye, Z. Shi, G. Wan, H. Li, B. Du, and Q. Yang, “Federated learning for generalization, robustness, fairness: A survey and benchmark,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 12, pp. 9387–9406, 2024. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 12

    work page 2024

  4. [4]

    Towards personalized federated learning,

    A. Z. Tan, H. Yu, L. Cui, and Q. Yang, “Towards personalized federated learning,” IEEE Trans. Neural Networks Learn. Syst. , vol. 34, no. 12, pp. 9587–9603, 2023

    work page 2023

  5. [5]

    Backpropagation and stochastic gradient descent method,

    S.-i. Amari, “Backpropagation and stochastic gradient descent method,” Neurocomputing, vol. 5, no. 4-5, pp. 185–196, 1993

    1993

  6. [6]

    Numerical optimization,

    S. J. Wright, “Numerical optimization,” 2006

    work page 2006

  7. [7]

    Second-order stochastic opti- mization for machine learning in linear time,

    N. Agarwal, B. Bullins, and E. Hazan, “Second-order stochastic opti- mization for machine learning in linear time,” J. Mach. Learn. Res. , vol. 18, pp. 116:1–116:40, 2017

    work page 2017

  8. [8]

    Nys-Newton: Nystr\"om-Approximated Curvature for Stochastic Optimization

    H. Tankaria, D. Singh, and M. Yamada, “Nys-curve: Nystr ¨om- approximated curvature for stochastic optimization,” CoRR, vol. abs/2110.08577, 2021

    work page Pith review arXiv 2021

Show all 38 references
  1. [9]

    Optimizing neural networks with kronecker-factored approximate curvature,

    J. Martens and R. B. Grosse, “Optimizing neural networks with kronecker-factored approximate curvature,” in Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, ser. JMLR Workshop and Conference Proceed- ings, F. R. Ba...

    2015

  2. [10]

    Distributed estimation of the inverse hessian by determinantal averaging,

    M. Derezinski and M. W. Mahoney, “Distributed estimation of the inverse hessian by determinantal averaging,” in Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8- 14, 2019, Vancouver,...

    2019

  3. [11]

    On second- order optimization methods for federated learning,

    S. Bischoff, S. G ¨unnemann, M. Jaggi, and S. U. Stich, “On second- order optimization methods for federated learning,” arXiv preprint arXiv:2109.02388, 2021

    2021 arXiv

  4. [12]

    Sofim: Stochastic optimization using regularized fisher information matrix,

    M. Sen, A. Qin, C. Gayathri, Y .-W. Chen, B. Raman et al. , “Sofim: Stochastic optimization using regularized fisher information matrix,” in 2024 International Joint Conference on Neural Networks (IJCNN) . IEEE, 2024, pp. 1–7

    2024

  5. [13]

    New insights and perspectives on the natural gradient method,

    J. Martens, “New insights and perspectives on the natural gradient method,” The Journal of Machine Learning Research , vol. 21, no. 1, pp. 5776–5851, 2020

    2020

  6. [14]

    Variational gaussian process classi- fiers,

    M. N. Gibbs and D. J. MacKay, “Variational gaussian process classi- fiers,” IEEE Transactions on Neural Networks, vol. 11, no. 6, pp. 1458– 1464, 2000

    2000

  7. [15]

    Federated optimization in heterogeneous networks,

    T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networks,” in Proceedings of Machine Learning and Systems 2020, MLSys 2020, Austin, TX, USA, March 2-4, 2020 . mlsys.org, 2020

    2020

  8. [16]

    SCAFFOLD: stochastic controlled averaging for federated learning,

    S. P. Karimireddy, S. Kale, M. Mohri, S. J. Reddi, S. U. Stich, and A. T. Suresh, “SCAFFOLD: stochastic controlled averaging for federated learning,” in Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event , ser. Proce...

    2020

  9. [17]

    Model-contrastive federated learning,

    Q. Li, B. He, and D. Song, “Model-contrastive federated learning,” in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021 , 2021, pp. 10 713–10 722

    2021

  10. [18]

    Implicit gradient alignment in distributed and federated learning,

    Y . Dandi, L. Barba, and M. Jaggi, “Implicit gradient alignment in distributed and federated learning,” in Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Inno- vative Applications of Artificial Intelligence, IAAI 2022, The Twelv...

    2022

  11. [19]

    Fedcm: Federated learning with client-level momentum,

    J. Xu, S. Wang, L. Wang, and A. C.-C. Yao, “Fedcm: Federated learning with client-level momentum,” arXiv preprint arXiv:2106.10874 , 2021

    2021 arXiv

  12. [20]

    Generalizable heterogeneous federated cross-correlation and instance similarity learning,

    W. Huang, M. Ye, Z. Shi, and B. Du, “Generalizable heterogeneous federated cross-correlation and instance similarity learning,”IEEE Trans. Pattern Anal. Mach. Intell. , vol. 46, no. 2, pp. 712–728, 2024

    2024

  13. [21]

    Federated feature augmentation and alignment,

    T. Zhou, Y . Yuan, B. Wang, and E. Konukoglu, “Federated feature augmentation and alignment,” IEEE Trans. Pattern Anal. Mach. Intell. , vol. 46, no. 12, pp. 11 119–11 135, 2024

    2024

  14. [22]

    Fedexp: Speeding up federated averaging via extrapolation,

    D. Jhunjhunwala, S. Wang, and G. Joshi, “Fedexp: Speeding up federated averaging via extrapolation,” in The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net, 2023

    2023

  15. [23]

    Fedbabu: Towards enhanced representation for federated image classification,

    J. Oh, S. Kim, and S.-Y . Yun, “Fedbabu: Towards enhanced representation for federated image classification,” arXiv preprint arXiv:2106.06042, 2021

    2021 arXiv

  16. [24]

    Exploiting shared representations for personalized federated learning,

    L. Collins, H. Hassani, A. Mokhtari, and S. Shakkottai, “Exploiting shared representations for personalized federated learning,” in Interna- tional conference on machine learning . PMLR, 2021, pp. 2089–2099

    2021

  17. [25]

    Personalized federated learning with feature alignment and classifier collaboration,

    J. Xu, X. Tong, and S. Huang, “Personalized federated learning with feature alignment and classifier collaboration,” in The Eleventh Inter- national Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net, 2023

    2023

  18. [26]

    Fedcr: Personalized federated learning based on across-client common representation with conditional mutual information regularization,

    H. Zhang, C. Li, W. Dai, J. Zou, and H. Xiong, “Fedcr: Personalized federated learning based on across-client common representation with conditional mutual information regularization,” in International Confer- ence on Machine Learning . PMLR, 2023, pp. 41 314–41 330

    2023

  19. [27]

    Personalized federated learning with moreau envelopes,

    C. T. Dinh, N. H. Tran, and T. D. Nguyen, “Personalized federated learning with moreau envelopes,” in Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual , H. Larochel...

    2020

  20. [28]

    Ditto: Fair and robust federated learning through personalization,

    T. Li, S. Hu, A. Beirami, and V . Smith, “Ditto: Fair and robust federated learning through personalization,” in International conference on machine learning . PMLR, 2021, pp. 6357–6368

    2021

  21. [29]

    Fedphp: Federated personalization with inherited private models,

    X.-C. Li, D.-C. Zhan, Y . Shao, B. Li, and S. Song, “Fedphp: Federated personalization with inherited private models,” in Joint European Con- ference on Machine Learning and Knowledge Discovery in Databases . Springer, 2021, pp. 587–602

    2021

  22. [30]

    Per- sonalized cross-silo federated learning on non-iid data,

    Y . Huang, L. Chu, Z. Zhou, L. Wang, J. Liu, J. Pei, and Y . Zhang, “Per- sonalized cross-silo federated learning on non-iid data,” in Proceedings of the AAAI conference on artificial intelligence , vol. 35, no. 9, 2021, pp. 7865–7873

    2021

  23. [31]

    Per- sonalized federated learning with first order model optimization,

    M. Zhang, K. Sapra, S. Fidler, S. Yeung, and J. M. ´Alvarez, “Per- sonalized federated learning with first order model optimization,” in 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net, 2021

    2021

  24. [32]

    Feddwa: Personalized federated learning with dynamic weight adjustment,

    J. Liu, J. Wu, J. Chen, M. Hu, Y . Zhou, and D. Wu, “Feddwa: Personalized federated learning with dynamic weight adjustment,” in Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI 2023, 19th-25th August 2023, Macao, SAR, China. ij...

    2023

  25. [33]

    Fedala: Adaptive local aggregation for personalized federated learning,

    J. Zhang, Y . Hua, H. Wang, T. Song, Z. Xue, R. Ma, and H. Guan, “Fedala: Adaptive local aggregation for personalized federated learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 9, 2023, pp. 11 237–11 244

    2023

  26. [34]

    Personalized federated learning: A meta-learning approach,

    A. Fallah, A. Mokhtari, and A. E. Ozdaglar, “Personalized federated learning: A meta-learning approach,” CoRR, vol. abs/2002.07948, 2020

    2002 arXiv

  27. [35]

    Fast convergence of natural gradient descent for over-parameterized neural networks,

    G. Zhang, J. Martens, and R. B. Grosse, “Fast convergence of natural gradient descent for over-parameterized neural networks,” Advances in Neural Information Processing Systems , vol. 32, 2019

    2019

  28. [36]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, G. Hinton et al. , “Learning multiple layers of features from tiny images,” 2009

    2009

  29. [37]

    A downsampled variant of imagenet as an alternative to the cifar datasets,

    P. Chrabaszcz, I. Loshchilov, and F. Hutter, “A downsampled variant of imagenet as an alternative to the cifar datasets,” arXiv preprint arXiv:1707.08819, 2017

    2017 arXiv

  30. [38]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778. Mrinmay Sen is a joint research scholar in the Department of Artificial Intelligence, Indi...

    2016

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.