{"id":"ca0051c3-cf3d-4a59-ad47-741af182ee01","arxiv_id":"2601.09166","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A server-side, rank-one Fisher-information preconditioner built from privatized gradients gives faster convergence and higher accuracy than DP-FedGD, DP-FedFC, and DP-SCAFFOLD in some—but not all—reported DP federated-learning settings.","lead":"This paper proposes DP-FedSOFIM, a federated-learning optimizer that adds a server-side curvature adjustment to already-privatized gradients, aiming to speed up differentially private training without extra privacy cost. A smart generalist reader might care because privacy-preserving collaborative learning is bottlenecked by slow convergence, and an O(d) second-order-style fix could be widely useful—if the empirical claims hold.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central empirical claim is contradicted by Table 2(b): on CIFAR-10 with 100 clients, DP-FedSOFIM finishes below DP-FedGD at ε=2 (63.86 vs 64.13) and ε=5 (66.15 vs 66.58), so 'consistent superiority across all privacy budgets' is unsupported.","rationale":"The reader's final verdict is REJECT, and my read does not change that. The most load-bearing concern is the internal contradiction between the paper's stated claim of consistent empirical superiority and its own Table 2(b). The reader's rationale identifies this contradiction explicitly, but the 'weakest_assumption' field focuses on the spiked-spectrum premise. I consider the empirical contradiction more fundamental: if the reported table is accurate, the headline claim is false regardless of the spectral structure. The privacy analysis is sound and deserves credit; the Sherman–Morrison implementation is elegant and O(d). However, the paper's central claim is empirical, and that claim fails against its own numbers. The concrete test would settle whether the inconsistency is a reporting error or a genuine performance deficit; either way, the current manuscript does not support 'consistent superiority.'","tokens_in":29832,"tokens_out":4210,"duration_ms":44318,"concrete_test":"Re-run the CIFAR-10, 100-client conditions at ε=2 and ε=5 with at least five independent seeds, selecting hyperparameters on a held-out validation split rather than on final test accuracy. Report mean and standard deviation of final test accuracy for DP-FedGD, DP-FedFC, and DP-FedSOFIM. If the mean of DP-FedSOFIM is not above both baselines at ε=2 and ε=5, the claim of consistent superiority is falsified; if it is above, the original Table 2(b) still needs correction and the result should be reported with uncertainty.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's strongest claim is empirical: DP-FedSOFIM 'consistently outperforms all baselines ... across both CIFAR-10 and PathMNIST datasets, two client scales, and all privacy budgets tested' (Section 6). This is directly contradicted by the paper's own Table 2(b). For CIFAR-10 with 100 clients at ε=2, DP-FedSOFIM reaches 63.86%, below DP-FedGD (64.13%) and DP-FedFC (64.47%). At ε=5, it reaches 66.15%, below DP-FedGD (66.58%). Because the central contribution rests on consistent empirical superiority, this internal contradiction is load-bearing: either the table is wrong, or the headline claim is false as stated. The absence of error bars compounds the problem. Section 5.1 states that hyperparameters were selected by 'optimal final test accuracy,' so the reported margins are not independent of the test set. With a single run, the differences between 63.86% and 64.13% are within plausible noise, but the paper gives no way to assess this. Thus the evidence for the paper's main claim is both internally inconsistent and statistically unvalued. The privacy argument itself is sound: the server-side preconditioner is post-processing of privatized gradients, so it does not add privacy loss. The convergence theorem, however, only bounds error for DP-FedSOFIM; it does not show an advantage over first-order DP-FedGD. The spiked-spectrum justification in Section 3.4.2 is qualitative and untested, but the more immediate problem is that even the basic empirical comparison fails in the reported data.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DP-FedSOFIM, a differentially private federated optimization method that builds a rank-one, regularized Fisher information proxy at the server from the privacy-noise-corrupted aggregated gradients. The server applies this proxy as a preconditioner via the Sherman–Morrison formula, incurring only O(d) cost, and the authors argue that all curvature processing is post-processing of the privatized gradients, so the method inherits the DP guarantee of the underlying DP-FedGD mechanism. Theoretical results give linear convergence to a noise-and-bias floor under strong convexity and the PL condition. Experiments on CIFAR-10 and PathMNIST with frozen ResNet-20 features and 20 or 100 clients are presented for ε ∈ {0.5, 1, 2, 5, 10} and the non-private setting; the paper claims that DP-FedSOFIM consistently outperforms DP-FedGD, DP-FedFC, and DP-SCAFFOLD across all these settings.","tokens_in":30214,"tokens_out":3284,"duration_ms":35368,"significance":"If the central empirical claim were supported, the paper would make a useful contribution: a server-side second-order preconditioner with O(d) client and server cost and no additional privacy loss is an appealing design, and the privacy argument via post-processing is conceptually clean. The paper also includes machine-checkable-style proofs of the one-step descent and convergence to an error floor, and it explicitly derives the variance-reduction effect of the momentum buffer. These are genuine strengths. However, the headline claim of consistent, privacy-budget-independent superiority is contradicted by the paper's own tables, and the empirical evaluation lacks the statistical controls needed to support such a strong claim. The spiked-spectrum premise underlying the rank-one proxy is asserted rather than measured. As a result, the evidence for the paper's central contribution is not currently persuasive.","major_comments":[{"comment":"The claim that DP-FedSOFIM 'consistently outperforms' all baselines across all privacy budgets and client scales is directly contradicted by Table 2(b). On CIFAR-10 with 100 clients at ε=2, DP-FedSOFIM's final accuracy is 63.86%, below DP-FedGD's 64.13% and DP-FedFC's 64.47%; at ε=5 it finishes at 66.15%, below DP-FedGD's 66.58%. This is not a minor misstatement: the empirical superiority claim is the paper's central selling point, and Section 6 repeats it verbatim. The headline claim must be corrected or supported with evidence that does not appear in the present tables.","section":"§5.3.1 and Table 2(b)"},{"comment":"Hyperparameters, including learning rate and clipping threshold, are selected by 'the configuration yielding optimal final test accuracy.' This means the test set is used for model selection, so the reported test accuracies are not an unbiased evaluation of the final configuration. Moreover, no standard deviations or multiple-seed results are reported. The differences at issue (e.g., 63.86 vs 64.13) are far smaller than the likely run-to-run variation in deep-net fine-tuning, so without error bars the claimed margins cannot be statistically distinguished from noise. The paper needs an evaluation protocol with validation-based selection and repeated runs.","section":"§5.1 and Tables 2–3"},{"comment":"The practical benefit of the rank-one Fisher proxy rests on the 'spiked-spectrum' assumption that gradient/Fisher structure is dominated by one direction. The paper offers qualitative speculation in §5.3.5 about CIFAR-10 being diffuse and PathMNIST being concentrated, but no direct measurement of the spectrum, eigengap, or condition number is provided. Without evidence that the rank-one MtMt^T captures useful curvature under DP noise, the method's advantage over first-order DP-FedGD is not established. A concrete spectral analysis of the effective loss landscape for the frozen-feature models would be needed to support the mechanism claimed.","section":"§3.4.2 and §5.3.5"},{"comment":"Theorem 4.17 bounds the error of DP-FedSOFIM, but it does not compare this bound with the corresponding bound for DP-FedGD. The error floor Γ in Eq. (35) contains an extra same-step coupling penalty ηG²maxρ^{-2}M̄², so the theorem does not establish that the preconditioned method converges faster or to a smaller neighborhood than the first-order baseline. Thus the theoretical section does not provide support for the paper's empirical superiority claim; at best it shows the preconditioned update retains linear convergence under assumptions.","section":"§4.5, Theorem 4.17"}],"minor_comments":[{"comment":"The abstract and conclusion state that DP-FedSOFIM 'consistently achieves higher accuracy' across all privacy budgets. Given the contradictions in Table 2(b) and the early-round instability acknowledged in §5.3.2, the claims should be hedged to specific regimes and datasets.","section":"Abstract and §6"},{"comment":"The PathMNIST 20-client non-private row shows DP-FedSOFIM at 27.34% at round 10 versus 54.14% for DP-FedGD, yet §5.3.3 claims 'immediate dominance from round 10' in the non-private setting. This inconsistency should be resolved.","section":"Table 3(a), ε=No DP"},{"comment":"The convergence theorems assume strong convexity, but the experiments train a softmax head with no reported ℓ2 regularizer beyond the Fisher regularization ρ. The remark asserts strong convexity 'can be ensured' but does not state that it was, so the link between theory and experiments is incomplete.","section":"§4, Remark 4.4"},{"comment":"The privacy accounting in Appendix E assumes full participation and gives a specific hockey-stick formula, but the experiments report only final accuracy curves; the noise multiplier σg selected for each ε is not listed. Reporting these values would improve reproducibility.","section":"§5.1"}],"recommendation":"reject","confidential_remarks":"The privacy post-processing argument and the O(d) complexity analysis are sound and could form the basis of a useful paper, but the central empirical claim is contradicted by the paper's own Table 2(b), and the absence of error bars plus test-set-based hyperparameter selection undermine the statistical validity of the experiments. I do not think a revision within the current scope—without substantially new experiments and corrected claims—can fix this."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nYou can skip the hype and read the mechanism: the server-side rank-one Fisher proxy built from privatized gradients, with privacy by post-processing, is a legitimate and clean idea. The Sherman-Morrison update makes it O(d), and the proof that no additional privacy loss accrues is correct. That part is worth reading.\n\nThe problem is the paper's headline claim. It says DP-FedSOFIM 'consistently outperforms all baselines ... across both datasets, two client scales, and all privacy budgets tested.' Its own Table 2(b) shows the opposite: on CIFAR-10 with 100 clients, DP-FedSOFIM finishes at 63.86% at ε=2, below DP-FedGD (64.13%) and DP-FedFC (64.47%); at ε=5 it trails DP-FedGD (66.15 vs 66.58). So the central empirical claim is not just unproven—it is contradicted by the reported numbers. The results also come from single runs, with hyperparameters selected on final test accuracy, so the margins are not trustworthy.\n\nThe theory is honest but limited. It gives linear convergence to an error floor under strong convexity/PL, which is what you'd expect from any DP gradient method; it doesn't show DP-FedSOFIM converges faster than DP-FedGD. The spiked-spectrum justification for the rank-one proxy is plausible but untested—no eigenvalue spectrum is measured.\n\nWhere does that leave us? The core idea is a modest but real extension of SOFIM to the private federated setting, and the privacy argument is clean. The experimental section needs a major rework: report error bars, maybe multiple seeds, and tone down the claims to match the data. The writing is competent and the citations are relevant.\n\nThis paper deserves a serious referee—not because the conclusions hold, but because the idea has value and the evaluation is fixable. It should not be desk-rejected. My recommendation: send it to review, but the reviewer should require the empirical claims to be scaled back to what the data support.","headline":"Sound privacy-preserving preconditioning idea, but the headline empirical claim is contradicted by the paper's own Table 2(b).","tokens_in":30785,"tokens_out":3460,"would_cite":false,"duration_ms":28970,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Server-side preconditioning on already-privatized gradients can give differentially private federated learning second-order convergence benefits at no additional privacy cost, using only O(d) memory and communication.","keywords":["differential privacy","federated learning","second-order optimization","Fisher information matrix","natural gradient","Sherman-Morrison","post-processing","privacy-utility tradeoff"],"falsifier":"Measure the eigenvalue spectrum of the empirical Fisher (gradient outer-product) matrix on the frozen-feature CIFAR-10 and PathMNIST tasks: if the top eigenvalue accounts for a small fraction of the trace, the rank-one proxy cannot be the source of the reported gains. A second check is to run DP-FedSOFIM with M_t replaced by an independent random unit vector scaled to the same norm; if accuracy gains persist, they are not curvature-driven.","tokens_in":29631,"feed_emoji":"🔒","tokens_out":7058,"duration_ms":59990,"temperature":0.7,"pith_summary":"This paper tries to establish that useful second-order curvature information can be extracted entirely from the privatized gradients already flowing to the server in differentially private federated learning. The proposed method, DP-FedSOFIM, forms a rank-one regularized Fisher proxy from an exponential moving average of aggregated noisy gradients, inverts it in closed form with the Sherman-Morrison identity, and uses it to precondition the parameter update. Because every curvature computation acts only on already-private quantities, the post-processing property of differential privacy guarantees the method inherits the baseline's privacy budget unchanged. Empirical results are reported on CIFAR-10 and PathMNIST across two client scales and privacy budgets from ε=0.5 to ε=10, claiming consistently higher accuracy than first-order, client-side covariance, and variance-reduction baselines. A careful reader would care because this is a concrete route to second-order speedups without the O(d^2) client-side Hessian or covariance storage that previously made such methods impractical.","feed_headline":"Free curvature for DP federated learning via a rank-one Fisher proxy","feed_subtitle":"Server-side preconditioning of already-private gradients adds no privacy cost, cuts client memory to O(d), and lifts accuracy under tight bu","key_machinery":"The carrying object is the regularized rank-one Fisher proxy Î_t = ρI + M_tM_t^T, where M_t is the exponential moving average of privatized aggregated gradients. Its inverse is maintained implicitly through the Sherman-Morrison identity, H_t = (1/ρ)I − M_tM_t^T/(ρ²+ρ‖M_t‖²), so the preconditioned step H_tG_t costs two inner products and vector operations. The proxy does double duty: the momentum buffer smooths privacy noise (variance reduced by factor (1−β)/(1+β)), and the rank-one term captures the dominant curvature direction, contracting steps along M_t and leaving orthogonal directions scaled by 1/ρ.","core_discovery":"The central claim is that the privatized gradient stream carries enough curvature information to precondition training. Each round the server updates a momentum buffer M_t = βM_{t-1} + (1−β)G_t from the aggregated noisy gradient, forms the regularized rank-one proxy Î_t = ρI + M_tM_t^T, and updates θ_{t+1} = θ_t − ηH_tG_t with H_t = (ρI + M_tM_t^T)^{-1}. The Sherman-Morrison formula gives H_t in O(d): (1/ρ)I minus a scaled rank-one term, so no matrix is ever materialized. Privacy is structural: H_t is a deterministic function of G_t and the previous server state, so the mapping G_t → H_tG_t is post-processing and inherits the baseline's (ε,δ) guarantee. Convergence is linear to a bias-and-no","pith_inferences":["Editorial inference: if curvature is diffuse with no dominant direction, the preconditioner degenerates to a data-dependent scalar rescaling and adds no value; a direct test would compare accuracy against the same algorithm with M_t replaced by a fixed random unit vector of equal norm.","Editorial inference: the observed early-round instability under tight privacy suggests an untested warm-up variant that delays preconditioning until the momentum buffer has stabilized; this would likely preserve the late-round gains while removing the early deficit.","Editorial inference: the rank-one proxy could be extended to rank-k via the Woodbury identity to capture multiple curvature directions, at the cost of O(kd) instead of O(d) — a trade-off the paper does not explore.","Editorial inference: the convergence proof relies on strong convexity or the Polyak–Łojasiewicz condition, which match the frozen linear-head experiments, but extending the claimed gains to full deep-network training would require new analysis since same-step coupling and clipping bias behave differently in non-convex landscapes."],"forward_implications":["Differentially private federated training can use curvature-aware preconditioning with client-side memory and communication still O(d), removing the main scalability barrier of earlier second-order DP methods.","Privacy accounting becomes modular: any improvement to the underlying noisy-gradient release (better composition, amplification, or clipping) transfers unchanged to the preconditioned method.","Under tight privacy budgets, where first-order updates are noise-dominated, the rank-one rescaling can reach a target accuracy in fewer communication rounds, lowering wall-clock time and client energy use.","The method is most beneficial when the frozen-feature loss landscape is ill-conditioned with concentrated curvature; on such tasks the reported accuracy gains persist across all tested privacy budgets.","Because server-side preconditioning adds no privacy loss, the approach composes cleanly with future privacy accounting advances and with partial-participation federated settings."],"fun_headline_variants":["Curvature from gradients: DP-FL gets a Fisher proxy for free","Rank-one Fisher proxy makes DP federated learning faster and cheaper","Free curvature in DP-FL: rank-one Fisher proxy cuts memory and boosts accuracy","No extra privacy cost: DP-FedSOFIM preconditions with a rank-one Fisher","Fisher proxy from privatized gradients: faster DP-FL, no Hessian needed"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The method's practical benefit rests on the premise that the Fisher or gradient-covariance structure is dominated by a single direction, so the rank-one term M_tM_t^T captures useful curvature; if the curvature is diffuse, the preconditioner only rescales the noisy gradient and can amplify noise in early rounds.","fun_headline_variants_meta":{"raw":{"variants":["Curvature from gradients: DP-FL gets a Fisher proxy for free","Rank-one Fisher proxy makes DP federated learning faster and cheaper","Free curvature in DP-FL: rank-one Fisher proxy cuts memory and boosts accuracy","No extra privacy cost: DP-FedSOFIM preconditions with a rank-one Fisher","Fisher proxy from privatized gradients: faster DP-FL, no Hessian needed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001054,"raw_usage":{"total_tokens":4293,"prompt_tokens":808,"completion_tokens":3485,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":3392}},"tokens_in":552,"tokens_out":3485,"duration_ms":22216,"temperature":1.0,"reasoning_tokens":3392,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T10:40:22.877112+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the eigenvalue spectrum of the empirical Fisher (gradient outer-product) matrix on the frozen-feature CIFAR-10 and PathMNIST tasks: if the top eigenvalue accounts for a small fraction of the trace, the rank-one proxy cannot be the source of the reported gains. A second check is to run DP-FedSOFIM with M_t replaced by an independent random unit vector scaled to the same norm; if accuracy gains persist, they are not curvature-driven.","supporting_citations":[],"review_version":1}