Pith. sign in

REVIEW 4 major objections 7 minor 22 references

Second-order federated optimization can be made stable and faster under non-IID data by monitoring and resetting ill-conditioned curvature updates.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 20:03 UTC pith:SRGCWXNP

load-bearing objection Cache mismatch: the supplied full text is FedRCO (second-order FL), not the neutrino 2p2h abstract; on that manuscript the contribution is a practical stabilizer stack with standard FL theory and strong small-CNN tables, limited by heuristic preconditioner control. the 4 major comments →

arxiv 2603.28314 v1 pith:SRGCWXNP submitted 2026-03-30 hep-ph hep-ex

Exclusive Hadron Observables in Neutrino Induced 2p2h Multinucleon Knockout

Vedantha Srinivas Kasturi , Juan Nieves , Federico S\'anchez , Joanna Ewa Sobczyk This is my paper
classification hep-ph hep-ex
keywords federated learningsecond-order optimizationK-FACnon-IID dataclient driftcurvature preconditioninggradient anomaly detection
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Federated learning usually relies on first-order gradient methods that ignore curvature, so they converge slowly and struggle when clients hold statistically different data. Second-order methods that use curvature (via approximations of the Fisher matrix) can in principle take better steps and cut communication rounds, but small local batches and non-IID distributions make those curvature estimates rank-deficient or mismatched to the global loss, producing exploding updates and divergence. This paper argues that the instability is not inevitable: a K-FAC-style curvature optimizer can be kept under control with a real-time gradient anomaly monitor, a fail-safe reset of local curvature state, and an aggregation rule that blends global parameters without erasing useful local geometry. The resulting method, FedRCO, is claimed to converge to a neighborhood of the optimum while bounding client drift, and experiments on CIFAR-10 and EMNIST under strong heterogeneity show higher accuracy and faster wall-clock progress than both standard first-order federated algorithms and prior second-order federated methods.

Core claim

The paper claims that the main failure modes of second-order federated learning—rank-deficient local Fisher matrices from small batches and curvature mismatch under non-IID data—produce unbounded updates, but that these can be tamed by a Gradient Anomaly Monitor, a Fail-Safe Resilience protocol that resets bad curvature state, and Curvature-Preserving Adaptive Aggregation, yielding stable local descent, bounded client drift, and faster, more accurate global convergence than existing first- and second-order federated optimizers.

What carries the argument

FedRCO: block-diagonal K-FAC natural-gradient client steps plus a look-ahead anomaly score on preconditioned gradient norms that triggers soft rollbacks or hard curvature resets, combined with accuracy-weighted local–global parameter interpolation that avoids overwriting local curvature geometry.

Load-bearing premise

The proofs and the practical claim both require that damping, monitoring, and resets keep the inverse-curvature eigenvalues bounded away from zero and infinity so that descent and drift bounds stay valid.

What would settle it

Run the same non-IID CIFAR-10 and EMNIST protocols with one mini-batch per local epoch: if FedRCO still produces gradient explosions, fails to reach the reported accuracy gaps over FedAvg/FedProx/FedPM, or needs far more communication rounds than claimed once the monitor thresholds or lazy-inverse interval are varied, the central stability-and-speed claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The manuscript proposes FedRCO, a K-FAC-based second-order federated optimizer intended to accelerate convergence under non-IID data while controlling numerical instability. It identifies rank deficiency (small local batches) and curvature mismatch (heterogeneous local Hessians/FIMs) as sources of unbounded updates, and introduces three mechanisms: a look-ahead Gradient Anomaly Monitor on preconditioned gradient norms, a Fail-Safe Resilience protocol (soft rollback / hard reset of curvature statistics), and Curvature-Preserving Adaptive Aggregation that mixes global and local parameters using local vs. global accuracy. Theoretical results (Theorems 5.1–5.4, Appendices C–D) give local descent, a server neighborhood rate, a client-drift bound proportional to λ_max², and a final error floor involving λ_max²/λ_min. Experiments on CIFAR-10 and EMNIST under Dirichlet, pathological, and IID partitions report large accuracy gains and faster wall-clock convergence versus FedAvg, FedProx, FedAdam, LocalNewton, and FedPM, with ablations on T_inv, the monitor, and participation.

Significance. If the stability mechanisms truly keep second-order updates well-conditioned under realistic federated heterogeneity, the work would be a useful step toward practical curvature-aware FL: it keeps O(d) communication like FedAvg, amortizes Kronecker inverses via lazy updates, and shows strong empirical gains on standard non-IID benchmarks. The rank-deficiency and curvature-mismatch analysis (Props. 3.1–3.2) is a clear contribution relative to prior second-order FL work that largely assumes well-behaved preconditioners. Code availability and extensive ablations (monitor trajectories, T_inv, client count/participation, wall-clock breakdowns) strengthen reproducibility. The main limitation on significance is that the superiority claim is demonstrated on a very small CNN with one mini-batch per local epoch, and the theorems rest on spectral control of the preconditioner that is enforced only heuristically.

major comments (4)
  1. Theorems 5.1–5.4 and the drift/error-floor bounds in Appendix D (edrift ≤ 2K²η²λ_max²(σ²+M²); Thm. 5.4 asymptotic term ∝ λ_max²/(μλ_min)) all require Assumption C.5-style control: (F̂)^{-1} ⪰ λ_min I with λ_min bounded away from zero and ∥(F̂)^{-1}∥ ≤ λ_max after damping. Props. 3.1–3.2 show that rank deficiency and non-IID mismatch can drive exactly the opposite (null-space 1/ε blow-up; Y/δ² global penalty). The Gradient Anomaly Monitor and Fail-Safe Resilience (§4.2, Eqs. 9–11) only threshold S_k with fixed τ_low≈10, τ_high≈1000, soft-clip to ∇̃_stable, or hard-reset Ω/Γ; there is no argument that these operations restore or preserve the (λ_min, λ_max) regime used in the proofs. Either prove a post-resilience spectral bound under Props. 3.1–3.2, or restate Theorems 5.1–5.4 as conditional on the monitor succeeding and report how often soft/hard interventions fire in the experiments.
  2. The experimental regime (§6.1, Appendix E.1) uses a two-conv + two-FC CNN, batch size 32, and “each epoch processing a single mini-batch,” with 20 local epochs per round. This is precisely the rank-deficient setting of Prop. 3.1, so the large gains in Table 1 and Fig. 1 may overstate benefits for more standard FL setups (multi-batch local epochs, ResNet-scale models, larger batches). At least one larger architecture and a multi-batch local-epoch setting are needed before the abstract claim of superiority over “state-of-the-art first-order and second-order methods” is warranted at the stated breadth.
  3. Convergence analysis (Appendix C–D) relies on L-smoothness, strong convexity / PL (μ), and bounded gradients, and several steps assume ∇L_c(θ*)≈0 and treat the product (F̂)^{-1}H̃ as having eigenvalues ≥ λ_min μ (around Eq. 61). Deep nets trained with cross-entropy are non-convex; the neighborhood rate in Thm. 5.2 is therefore only a formal guarantee under assumptions that do not hold for the models in §6. The paper should either (i) restrict the theorems to convex/PL surrogates and clearly separate them from the DNN experiments, or (ii) provide a non-convex stationarity analysis consistent with the K-FAC update and the hard-reset dynamics.
  4. Curvature-Preserving Adaptive Aggregation (§4.3, Eqs. 12–13) mixes with γ = Acc'/(Acc'+Acc) and falls back to θ_global when Acc' ≤ Acc. This requires a reliable local accuracy estimate each round (validation split or training accuracy). Training accuracy on highly skewed pathological clients is a poor proxy for representation quality and can bias γ toward overfitted local models—the opposite of the stated goal of avoiding curvature forgetting. The paper should specify how Acc'/Acc are computed in the reported runs, and include an ablation that replaces accuracy with a label-free criterion (e.g., local loss or parameter distance) to show the gains in Table 1 (FedRCO vs FedRCO-ori) are not an artifact of this choice.
minor comments (7)
  1. §4.2 refers to “Section 2.3,” but Related Work only has §§2.1–2.2; renumber or fix the cross-reference.
  2. Notation: Table 3 and the main text mix F, F̂, H, and λ_min/λ_max for both Hessian and inverse-preconditioner bounds; state explicitly whether λ_min in Thms. 5.1–5.4 is an eigenvalue of F̂ or of F̂^{-1}.
  3. Eq. (10) defines mean(H) with a sum of squared norms while H stores unsquared norms in the surrounding text; align the definition with the implementation.
  4. Fig. 2–4 and many appendix figures (6–26) are referenced with strong claims but several captions/panels are incomplete in the provided manuscript; ensure all panels and axis labels are present and that wall-clock numbers are hardware-reproducible.
  5. Hyperparameters τ_low, τ_high, ∇̃_stable=10, ε=0.03, α=0.95, T_inv=200 are fixed globally (§E.3). A short sensitivity table for τ_low/τ_high would strengthen the claim that the monitor is not brittle.
  6. Related work should more carefully position FedNL, LocalNewton, and FedPM communication costs against FedRCO’s O(d) pattern, and note other recent federated quasi-Newton / natural-gradient lines if space allows.
  7. The manuscript header/title block in the provided source (“Taming the Instability…”) does not match the arXiv metadata title about exclusive hadron observables in neutrino 2p2h; reconcile title, abstract, and body before any camera-ready version.

Circularity Check

0 steps flagged

No significant circularity: FedRCO's theory derives bounds from stated assumptions, and empirical claims are independently checkable against external baselines.

full rationale

Walking the claimed chain (Props 3.1–3.4 → method components → Theorems 5.1–5.4 → experiments), nothing reduces by construction to its own inputs. Rank-deficiency and curvature-mismatch arguments (Props 3.1–3.2) are standard spectral/quadratic analyses of the empirical FIM and local vs global Hessians; they do not define the later claims. Convergence theorems (5.1–5.4 and Appendices C–D) are ordinary descent/drift bounds under explicit assumptions (L-smoothness, strong convexity/PL, bounded variance/gradients, and eigenvalue control of the damped K-FAC inverse). The Gradient Anomaly Monitor, Fail-Safe Resilience, and Curvature-Preserving Aggregation are design choices with free hyperparameters (τ_low, τ_high, T_inv, ε, α, γ), not quantities fitted so that the reported accuracy or rate identity holds. Experiments compare FedRCO to external first- and second-order baselines (FedAvg, FedProx, FedAdam, LocalNewton, FedPM) on standard CIFAR-10/EMNIST partitions; superiority is empirical, not forced by the method definition. Citations (K-FAC, FedAvg, etc.) are to independent prior work, not load-bearing self-uniqueness theorems. No self-definitional loop, fitted-as-prediction, or renamed known result is present. Score 0 is appropriate.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 3 invented entities

The central claims rest on standard optimization regularity plus several paper-chosen numerical thresholds and the validity of block-diagonal K-FAC as a local preconditioner under one-mini-batch local epochs. No new physical entities are postulated; the “invented” pieces are algorithmic modules. Free parameters control anomaly detection, damping, EMA, inversion cadence, and aggregation mixing and are essential to the reported stability.

free parameters (6)
  • anomaly thresholds τ_low, τ_high = τ_low≈10, τ_high≈1000
    Hand-set cutoffs (~10 and ~1000) that decide soft rollback vs hard reset; load-bearing for the stability claim.
  • Tikhonov damping ε and π-correction = ε=0.03
    Damping that keeps Kronecker factors invertible; ε=0.03 used for all second-order methods in experiments.
  • EMA coefficient α for curvature factors = α=0.95
    Controls noise averaging of Ω/Γ under single mini-batch epochs.
  • inversion interval T_inv = T_inv=200 (preferred)
    Lazy inverse cadence; ablated, with T_inv=200 preferred; affects both cost and stability.
  • learning rate η and stable gradient bound ∇̃_stable = η=0.00625, ∇̃_stable=10
    Step size and soft-rollback cap used throughout experiments.
  • aggregation mix γ = Acc'/(Acc'+Acc) = dynamic in [0,1]
    Accuracy-gated interpolation weight; design choice that preserves local geometry when local Acc' is high.
axioms (6)
  • domain assumption Local objectives are L-smooth (Assumption C.1).
    Used for local descent lemma Theorem 5.1 and one-round global descent.
  • domain assumption Strong convexity / Polyak–Łojasiewicz structure with constant μ (Assumption C.2 / PL use in D.3).
    Needed for linear contraction factor ρ ∝ ημKλ_min in Theorems 5.2 and 5.4; deep nets are not globally SC.
  • domain assumption Stochastic gradients have bounded variance σ² and gradients are bounded by M (Assumptions C.3–C.4).
    Enter client-drift bound Theorem 5.3 and residual error floor.
  • domain assumption Preconditioner eigenvalues satisfy λ_min I ⪯ (F̂)^{-1} and ∥(F̂)^{-1}∥ ≤ λ_max after damping (Assumption C.5).
    Core regularity that converts K-FAC into controlled descent; exactly what rank deficiency threatens.
  • domain assumption Block-diagonal Kronecker-factored Fisher is a sufficiently faithful curvature model for the local update geometry.
    Inherited from K-FAC literature; required for natural-gradient interpretation and affine-invariance motivation.
  • standard math Standard matrix calculus / Kronecker inverse identity (Ω⊗Γ)^{-1}=Ω^{-1}⊗Γ^{-1} and Neumann expansion for stale inverses.
    Used in §4.1 and Appendix A.4 without controversy.
invented entities (3)
  • Gradient Anomaly Monitor (look-ahead score S_k) no independent evidence
    purpose: Detect accumulated divergence vs sudden explosion of preconditioned updates in real time.
    Algorithmic detector defined via sliding-window mean of ∥g̃∥; no independent physical meaning outside the method.
  • Fail-Safe Resilience protocol (soft rollback / hard reset) no independent evidence
    purpose: Restore stable local state when curvature becomes invalid.
    Control policy invented for FedRCO; evidence is internal ablation (Fig. 4), not external theory.
  • Curvature-Preserving Adaptive Aggregation no independent evidence
    purpose: Blend global and local parameters using accuracy-gated γ to avoid “curvature forgetting.”
    Aggregation rule specific to this paper; compared via FedRCO vs FedRCO-ori.

pith-pipeline@v1.1.0-grok45 · 24744 in / 4160 out tokens · 56150 ms · 2026-07-14T20:03:57.659266+00:00 · methodology

0 comments
read the original abstract

We explore the combined lepton and hadron kinematic observables from the exclusive Valencia $2p2h$ model. We present variables of interest which are available due to the exclusive kinematics and compare them with the democratically distributed outgoing nucleon kinematics as currently treated in neutrino event generators. We also show the effect of nuclear re-scattering based on the NEUT semi classical cascade. We comment on the observability of these variables in current and future long baseline neutrino detectors.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 4 linked inside Pith

  1. [1]

    Localnewton: Reducing communication bottleneck for distributed learning.arXiv preprint arXiv:2105.07320,

    Gupta, V ., Ghosh, A., Derezinski, M., Khanna, R., Ram- chandran, K., and Mahoney, M. Localnewton: Reducing communication bottleneck for distributed learning.arXiv preprint arXiv:2105.07320,

    Pith/arXiv arXiv

  2. [2]

    H., Qi, H., and Brown, M

    Hsu, T.-M. H., Qi, H., and Brown, M. Measuring the effects of non-identical data distribution for federated visual clas- sification.arXiv preprint arXiv:1909.06335,

    Pith/arXiv arXiv 1909

  3. [3]

    Fedpm: Federated learning using second- order optimization with preconditioned mixing of local parameters.arXiv preprint arXiv:2511.09100,

    Ishii, H., Niwa, K., Sawada, H., Fujino, A., Harada, N., and Yokota, R. Fedpm: Federated learning using second- order optimization with preconditioned mixing of local parameters.arXiv preprint arXiv:2511.09100,

    arXiv

  4. [4]

    Reddi, S., Charles, Z., Zaheer, M., Garrett, Z., Rush, K., Koneˇcn`y, J., Kumar, S., and McMahan, H. B. Adaptive federated optimization.arXiv preprint arXiv:2003.00295,

    Pith/arXiv arXiv 2003

  5. [5]

    Fednl: Making newton-type methods applicable to federated learning.arXiv preprint arXiv:2106.02969,

    Safaryan, M., Islamov, R., Qian, X., and Richt´arik, P. Fednl: Making newton-type methods applicable to federated learning.arXiv preprint arXiv:2106.02969,

    Pith/arXiv arXiv

  6. [6]

    9 Submission and Formatting Instructions for ICML 2026 A. Theoretical Analysis of Gradient Instability in Federated Second-Order Optimization In this section, we provide a theoretical analysis of the numerical instability observed when applying second-order optimization methods in a federated learning setting. We identify three primary sources of gradient...

    2026

  7. [7]

    These observations reveal a bias–variance trade-off in the choice of Tinv, which our method exploits to achieve both stability and efficiency in federated second-order optimization

    outperforms more frequent updates (e.g., Tinv = 20), while simultaneously reducing computational overhead. These observations reveal a bias–variance trade-off in the choice of Tinv, which our method exploits to achieve both stability and efficiency in federated second-order optimization. B. Theoretical Justification for the Efficacy of Second-Order in Fed...

    2026

  8. [8]

    16 Submission and Formatting Instructions for ICML 2026 C.3

    Second-order provides a significant advantage when the geometry is ill-conditioned, standard SGD would struggle with a large Lipschitz constant L, but our method effectively rescales the space. 16 Submission and Formatting Instructions for ICML 2026 C.3. Server-Side Global Convergence Theorem We now employ the local descent to prove the convergence of the...

    2026

  9. [9]

    The preconditioner λmin in ρ effectively improves the condition number, leading to faster convergence than SGD

    ρ .(70) The first term decays linearly to zero, while the second term represents the residual error floor due to stochastic noise and non-IID drift. The preconditioner λmin in ρ effectively improves the condition number, leading to faster convergence than SGD. D. Detailed Derivation of Optimization Bounds In this section, we rigorously derive the error bo...

    2026

  10. [10]

    Ideally, we want the update to align with the gradient. T1 ≈ −ηK∇L(θ t)⊤ 1 C X c F −1 c ∇Lc(θt) ! .(82) Using the preconditioner propertyv ⊤Gv≥λ min∥v∥2 and strong convexity: T1 ≤ −ηKλ min∥∇L(θt)∥2.(83) Using the Polyak-Lojasiewicz condition:∥∇L(θ)∥ 2 ≥2µ(L(θ)− L ∗): T1 ≤ −2ηKµλ min(L(θt)− L ∗).(84) 20 Submission and Formatting Instructions for ICML 2026 ...

    2026

  11. [11]

    In ill-conditioned problems, FedRCO ensures λmin is bounded away from zero, while in SGD it can be arbitrarily small

    2µλmin| {z } Asymptotic Error Floor .(88) For the linear decay speed ρ, the rate is governed by ρ∝λ min. In ill-conditioned problems, FedRCO ensures λmin is bounded away from zero, while in SGD it can be arbitrarily small. This proves faster convergence. For the error floor, the final error depends on the ratio λ2 max λmin . If the preconditioner is unsta...

    2026

  12. [12]

    Specifically, for each class k, we sample a distribution vector qk ∼Dir(α) and allocate a proportion of samples from class k to client i according to qk,i

    Dirichlet-based Non-IID DistributionThe Dirichlet-based partitioning simulates a realistic scenario where the label distributions across clients are unbalanced. Specifically, for each class k, we sample a distribution vector qk ∼Dir(α) and allocate a proportion of samples from class k to client i according to qk,i. The concentration parameter α controls t...

    2026

  13. [13]

    23 Submission and Formatting Instructions for ICML 2026 Figure 7.Dataset: EMNIST, Data distribution: Dirichlet 0.1, Party ratio: 0.5, Number clients:

    2026

  14. [14]

    24 Submission and Formatting Instructions for ICML 2026 Figure 9.Dataset: EMNIST, Data distribution: Dirichlet 0.1, Party ratio: 1, Number clients:

    2026

  15. [15]

    25 Submission and Formatting Instructions for ICML 2026 Figure 11.Dataset: EMNIST, Data distribution: Dirichlet 0.1, Party ratio: 0.8, Number clients:

    2026

  16. [16]

    26 Submission and Formatting Instructions for ICML 2026 Figure 13.Dataset: EMNIST, Data distribution: Dirichlet 1, Party ratio: 0.8, Number clients:

    2026

  17. [17]

    27 Submission and Formatting Instructions for ICML 2026 Figure 15.Dataset: EMNIST, Data distribution: Pathological 30, Party ratio: 0.8, Number clients:

    2026

  18. [18]

    28 Submission and Formatting Instructions for ICML 2026 Figure 17.Dataset: CIFAR-10, Data distribution: Dirichlet 0.1, Party ratio: 0.1, Number clients:

    2026

  19. [19]

    29 Submission and Formatting Instructions for ICML 2026 Figure 19.Dataset: CIFAR-10, Data distribution: Dirichlet 0.1, Party ratio: 1, Number clients:

    2026

  20. [20]

    30 Submission and Formatting Instructions for ICML 2026 Figure 21.Dataset: CIFAR-10, Data distribution: Dirichlet 0.1, Party ratio: 0.8, Number clients:

    2026

  21. [21]

    31 Submission and Formatting Instructions for ICML 2026 Figure 23.Dataset: CIFAR-10, Data distribution: Dirichlet 1, Party ratio: 0.8, Number clients:

    2026

  22. [22]

    32 Submission and Formatting Instructions for ICML 2026 Figure 25.Dataset: CIFAR-10, Data distribution: Pathological 5, Party ratio: 0.8, Number clients:

    2026