{"id":"d9f77fd0-ebdc-4b22-9780-1a92612861f1","arxiv_id":"2502.01352","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Server-side metric privacy, with noise scaled by a distance between client model updates, improves federated learning accuracy over global differential privacy in their experiments, but the claimed privacy protection is not formally established.","lead":"The authors test a privacy method called metric-privacy on the server side of federated learning for Alzheimer MRI classification, comparing it with standard differential privacy across six aggregation strategies. They report that metric-privacy improves accuracy while offering similar protection against a client inference attack, but the attack evaluation and privacy guarantee have serious gaps.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central privacy claim is unsupported: the proposed metric-privacy mechanism scales noise by a data-dependent internal distance with no proof of Definition 7, and the CIA evaluation lacks a non-participating control.","rationale":"I agree with the reader's REJECT verdict, but I would place the load-bearing concern one level deeper than the stated weakest assumption. The reader identifies the missing non-participating control in the client-inference attack metric; that is a real and decisive experimental flaw. However, even if such a control were added, the paper's formal privacy claim would still be unsupported, because the mechanism in Section 6 is asserted to satisfy metric-privacy without any proof that the data-dependent distance d(n) yields the likelihood-ratio bound of Definition 7. The variance of the added Gaussian noise is itself a function of the private inputs, so the mechanism is not a standard Gaussian mechanism with a fixed sensitivity; the paper provides no sensitivity analysis for the distance-based calibration. The concrete counterexample above shows that the guarantee can fail catastrophically, which is a correctness risk rather than a tuning issue. The empirical accuracy contribution has independent support: Tables 4-7 and Appendix A report multiple runs and standard deviations, and the claim that metric-privacy improves convergence over global-DP in most tested configurations is plausible and useful. But the central abstract claim pairs that utility improvement with 'similar protection against client inference attacks', and that second half rests on two unsupported pillars: the missing proof of metric-privacy and the uncalibrated attack statistic. Both are critical, and either is sufficient to reject the central claim. Since my concern strengthens rather than redirects the reader's conclusion, I recommend no change to the REJECT verdict.","tokens_in":20446,"tokens_out":10755,"duration_ms":107579,"concrete_test":"Construct the one-round counterexample from Section 6: with clipping bound C=5, let W consist of four clipped updates differing by at most delta along a single coordinate, and W' consist of two updates equal to +C e1 and two equal to -C e1, so d(n) is delta in the first case and 2C in the second, while the FedAvg aggregate is identical. Compute the exact Gaussian mechanism output densities for both cases, using the paper's noise standard deviation ne*C/(nc*d(n)). Evaluate sup over y of log(P(M(W)=y)/P(M(W')=y)). With the two variances differing by a factor of 2C/delta, this supremum is unbounded as delta tends to 0, whereas the metric distance between the two inputs is finite (zero if the metric is aggregate-based).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The decisive flaw is upstream of the missing control: Section 6 defines d(n) as the maximum pairwise Frobenius distance among the received local updates and then sets the Gaussian noise scale to ne*C/(nc*d(n)), asserting that 'metric-privacy is fulfilled in each FL round'. For Definition 7 to hold, d must be a fixed metric on the input domain and the mechanism must bound likelihood ratios for every pair of inputs at distance r by exp(eps*r). But d(n) is not a distance between two possible inputs (e.g., with and without the target client); it is an internal statistic of the current client set. The noise variance therefore depends on the very data being privatized, and no sensitivity bound, Lipschitz constant, or composition argument is provided. Concretely, take one round with clipped updates: let W have four nearly identical updates (small d(W)=delta) and W' have two updates at +C e1 and two at -C e1 (d(W')=2C), so both have the same FedAvg aggregate. The mechanism outputs Gaussians with variances K/delta and K/(2C). As delta approaches 0, the likelihood ratio P(M(W)=y)/P(M(W')=y) is unbounded in the tail, while the distance between the two inputs under any finite input metric is finite. This directly violates the inequality in Definition 7. Appendix F confirms the distance is computed after clipping, so d(n) is bounded, but boundedness alone does not yield the required privacy inequality. The empirical CIA tables (Tables 9-10 and D.13-D.16) use a relative loss difference with no non-participating baseline and no calibrated decision rule, so they cannot rescue the claim. The accuracy comparison in Tables 4-7 is a useful heuristic contribution, but the load-bearing privacy half of the central claim fails.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a server-side metric-privacy mechanism for federated learning. Instead of the standard global-DP Gaussian mechanism with a fixed noise multiplier, the server computes, in each round, the maximum pairwise Frobenius distance d(n) among the received local model updates and scales the Gaussian noise standard deviation by 1/d(n). The approach is evaluated on an Alzheimer MRI dataset with six aggregation strategies (FedAvg, FedAvgM, FedMedian, FedProx, FedOpt, FedYogi) under homogeneous and non-i.i.d. client splits, and the authors introduce a 'Client Inference Attack' (CIA) in which a semi-honest client tries to determine whether another client participated in training. The reported experiments show better accuracy and convergence for metric-privacy than for global-DP, with similar or better CIA protection measured by the relative difference between loss on a target shadow dataset and loss on an aggregated test set.","tokens_in":20823,"tokens_out":5234,"duration_ms":50862,"significance":"If the central claim were established, the paper would be a useful practical contribution: it extends the small literature on server-side DP in federated learning to six aggregation strategies and proposes an attack scenario, CIA, that is relevant to cross-silo medical collaborations. The experimental setup is described in detail, the data are openly available, and the reproducibility appendix lists library versions and the modified Flower file. The systematic comparison across aggregation strategies and the explicit trust model are strengths. However, the significance is heavily limited by two load-bearing problems: the metric-privacy guarantee is asserted rather than proved and, as stated, is not compatible with the formal definition, and the CIA evaluation lacks the control condition needed to support the 'similar protection' conclusion.","major_comments":[{"comment":"The claim that the proposed mechanism satisfies metric-privacy in each round is unsupported and, as stated, incompatible with Definition 7. In Definition 7, d must be a fixed metric on the input domain and the mechanism must satisfy a likelihood-ratio bound uniformly over all pairs of inputs at a given distance. Here d(n) is a statistic of the current multiset of client updates, so the Gaussian noise variance depends on the very input being privatized. Concretely, consider one round with clipped updates: let W consist of four nearly identical updates, so d(W)=delta, and let W' consist of two updates at +C e1 and two at -C e1, so d(W')=2C while the FedAvg aggregate is the same. The output distributions have variances K/delta and K/(2C) respectively. As delta approaches 0, the likelihood ratio P(M(W)=y)/P(M(W')=y) is unbounded in the tail, even though the distance between the two inputs under any finite metric is finite. Thus the inequality in Definition 7 is violated. A correct mechanism requires a sensitivity bound or Lipschitz constant that is independent of the observed updates, followed by a composition argument across rounds; none is provided.","section":"Section 6, Definition 7"},{"comment":"The text immediately after the noise formula states that 'if d(n) > 1, we are adding more noise with metric-privacy', but the formula for the standard deviation is ne*C/(nc*d(n)). For d(n)>1 this reduces the noise, not increases it. The comparison between global-DP and metric-privacy in Tables 4-7 depends directly on the actual noise level, so this contradiction must be resolved and the experimental numbers reconciled with the corrected interpretation.","section":"Section 6, noise formula"},{"comment":"The CIA evaluation cannot support the conclusion of 'similar protection' because all experiments include the target client in the training set. The statistic used is the relative difference between the loss on the target client's shadow dataset and the loss on the aggregated test set. Without a control condition in which the target client does not participate, this difference may reflect only the distributional mismatch between the target dataset and the aggregate data, not the fact of participation. The paper should report the same relative-loss statistic for a model trained without client 3, and ideally provide a decision rule or ROC analysis. As reported, the numbers only show that the noisy model has a different loss profile; they do not quantify CIA success.","section":"Section 7.4, Tables 9-10 and D.13-D.16"},{"comment":"The noise multiplier ne=0.01 was selected because it 'allows us to protect against the client inference attacks that we are trying to prevent'. This is circular: the same CIA experiments used to conclude protection are used to calibrate the privacy parameter. Moreover, the distance d(n) used to scale the metric-privacy noise is computed from the model updates being protected, making the privacy parameter data-dependent. The paper should provide an independent calibration rule, for example by fixing epsilon and delta and deriving the noise scale from a sensitivity bound, and then evaluate CIA under that rule.","section":"Appendix F, final paragraph"}],"minor_comments":[{"comment":"The caption of Table 10 identifies the strategy as FedYogi, while the text in Section 7.4 says the results are for FedOpt; please correct this inconsistency. Also, 'V anilla' appears in Tables 9-10 and D.13-D.16 due to a formatting artifact.","section":"Table 10"},{"comment":"The formulas for d(n) and for the initial clipping-norm estimate contain a stray 'i,j' subscript under the max symbol; the intended domains of i and j should be written cleanly.","section":"Section 6, d(n) formula"},{"comment":"The displayed formula for Delta w^(r+1) is garbled: the denominator and the summation are not typeset correctly, and the expression ni(w^(r)-w_i^(r+1)) is missing the intended weighting or parentheses. Please rewrite the equation.","section":"Section 3, FedAvgM"},{"comment":"The paper describes the Flower code modification in words but provides no pseudocode or algorithm listing for the metric-privacy mechanism. An algorithm box with inputs (clipping norm, noise multiplier, d(n)) and outputs (perturbed aggregate) would make the method reproducible and easier to check.","section":"Appendix F"},{"comment":"Definition 8 states the CIA as a posterior probability P(sx=1|Dx,W), but the experiments never compute this probability; they use a relative loss difference. The paper should either derive why this statistic estimates the posterior or redefine the attack in terms of the measured quantity.","section":"Section 5, Definition 8"}],"recommendation":"reject","confidential_remarks":"For the editor: the paper's central formal claim appears incorrect as stated, not merely underproved; the data-dependent noise scaling in Section 6 is incompatible with Definition 7, and the counterexample is direct. The CIA evaluation also lacks the non-participating control needed for the 'similar protection' claim. These are load-bearing issues that cannot be fixed by local edits. The empirical accuracy comparison may be salvageable, but a resubmission would need a corrected privacy mechanism with a fixed sensitivity and a properly controlled attack evaluation, which is substantial new work."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. The paper has a useful empirical core: a clean comparison of server-side global-DP and a metric-privacy heuristic across six FL aggregators on an Alzheimer MRI benchmark, with open data and code details. The accuracy numbers are plausible and consistently show the metric-privacy variant beating global-DP. That part could be of interest to practitioners.\n\nThe problem is the privacy half. Section 6 defines d(n) as the maximum pairwise Frobenius distance among the received local updates and then scales Gaussian noise by 1/d(n), asserting metric-privacy holds per round. That assertion is unsupported. Definition 7 requires a fixed metric on the input domain and a mechanism satisfying a likelihood-ratio bound for every pair of inputs at distance r. Here d(n) is an internal statistic of the current input database, not a distance between two possible databases. Two databases can have the same aggregate but very different d(n), giving very different noise variances. The likelihood ratio can be unbounded in the tail, so Definition 7 is violated. The paper contains no theorem, no sensitivity analysis, no composition argument. This is not a nitpick; it's the central contribution.\n\nThere's also an internal contradiction: the text in Section 7.3.1 says if d(n)>1 the method adds more noise, but the formula std = ne*C/(nc*d(n)) says the opposite. One of those is wrong.\n\nThe CIA evaluation doesn't support 'similar protection.' The attack statistic is the relative difference between loss on the target's shadow set and loss on the aggregated test set. Without a non-participating control, you can't tell whether the noise is hiding participation or just shifting the distribution. And the noise multiplier 0.01 was chosen post hoc because it gives the desired protection, so the conclusion is circular.\n\nThe novelty claim of being 'first' to use metric-privacy in FL is also undercut by their own reference [29].\n\nIf the paper were reframed as a heuristic noise-calibration study without the metric-privacy guarantee, the empirical comparison would stand and could be publishable as a benchmarking note. As submitted, the load-bearing privacy claims fail. I'd send it to review rather than desk-reject, because the empirical part deserves scrutiny and the flaw is instructive, but my recommendation to the editor would be reject unless the authors supply a rigorous privacy analysis or substantially soften the claims.","headline":"Useful empirical comparison of server-side DP variants in FL, but the privacy claims are unsupported: no proof of the metric-privacy guarantee and no non-participating control for the client inference attack.","tokens_in":21382,"tokens_out":5031,"would_cite":false,"duration_ms":45009,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Metric privacy applied server-side improves convergence and matches differential privacy's protection against client inference attacks in federated medical imaging.","keywords":["federated learning","metric privacy","differential privacy","client inference attack","medical imaging","Gaussian mechanism","model aggregation","privacy utility trade-off"],"falsifier":"Train the identical federated setup with the target client excluded, then measure the same relative loss difference between shadow data and aggregated test set; if non-participating targets show a gap comparable to participating ones, the claimed protection against client inference attacks does not follow.","tokens_in":1510,"feed_emoji":"🩺","tokens_out":2040,"duration_ms":63221,"temperature":0.7,"pith_summary":"The paper argues that federated learning can keep the privacy benefits of server-side differential privacy while losing less model accuracy by switching to metric privacy, a relaxation that scales noise to a distance between local model updates. On six aggregation strategies and an Alzheimer MRI dataset, the authors report that metric privacy consistently outperforms global-DP in accuracy and convergence while providing comparable protection against a new attack they define, the Client Inference Attack, where a semi-honest client tries to infer whether another client participated in training. The reason to care is that, if true, federated systems with a trusted server can tune privacy noise to actual client disparity each round, reducing the utility cost of privacy without weakening protection against participation disclosure.","feed_headline":"Privacy noise tuned to client-model distances beats standard DP","feed_subtitle":"Server-side privacy noise scaled by how far client models diverge keeps accuracy high while hiding participation.","key_machinery":"The load-bearing object is the per-round distance $d^{(n)}$, defined as the maximum over pairs of clients $i,j$ of the average Frobenius norm of the layer-wise differences between their local model weights. It is computed server-side from the received model updates without touching client data, and it is used to scale the standard deviation of the Gaussian noise applied to the aggregated model: the noise becomes $\\mathcal{N}\\left(0, \\frac{\\text{ne}\\cdot C}{n_c\\cdot d^{(n)}}\\right)$ instead of depending on a fixed sensitivity. This adaptivity is what metric privacy requires, and what lets the method tune the noise more finely than fixed-clipping global-DP.","core_discovery":"The central claim is that metric privacy, applied from the server side, can improve the convergence and final accuracy of a federated model compared with standard global differential privacy while offering similar resistance to client inference attacks. The privacy guarantee is scaled by a distance computed each round as the maximum, over pairs of clients, of the average Frobenius norm of the differences between their local model weights. That distance is then used to scale the Gaussian noise added to the aggregated model, replacing the fixed-clipping sensitivity used by global-DP. Across homogeneous and non-i.i.d. client settings and all six tested aggregation strategies, the paper reports that metric privacy achieves higher accuracy and smoother convergence curves than global-DP, while the relative loss gap between a target client's shadow dataset and the aggregated test set remains similar for the two privacy methods.","pith_inferences":["If the attack statistic were validated with non-participating controls and ROC-style thresholds, metric privacy's protection could be quantified as a true detection rate; the current relative-loss comparison is at best a screening heuristic.","The distance-calibration recipe generalizes: any server-side privacy mechanism whose output space has a meaningful metric could be made adaptive in the same way, not just Gaussian noise on model weights.","Because $d^{(n)}$ is computed from the same updates the noise is meant to hide, a malicious client that can perturb its local model might distort the distance and therefore the noise level; checking that robustness would be a natural next step.","The reported gains are likely to grow with client heterogeneity, since standard DP adds noise independent of how far client models have drifted; testing on more heterogeneous non-i.i.d. medical data would show the boundary."],"forward_implications":["A trusted aggregator can calibrate privacy noise per round using only the received model updates, adding less noise when client models agree and more when they diverge.","Under the reported experimental conditions, metric privacy yields higher test accuracy than global-DP for all six aggregation strategies in homogeneous and non-i.i.d. settings, while keeping convergence curves closer to the non-private baseline.","The Client Inference Attack gives federated learning deployments a concrete, lightweight way to audit whether a semi-honest participant can infer another client's presence.","Because the defense is server-side, it also protects the final global model if it is later published, without requiring changes in client training."],"supporting_citations":[{"why":"Defines metric privacy / d-privacy, the relaxation this paper adapts to federated learning.","marker":"[8, 29]"},{"why":"Introduces FedAvg and the federated averaging formulation used as the primary aggregation baseline.","marker":"[2]"},{"why":"Defines membership inference attacks, the threat that client inference attacks extend.","marker":"[3]"},{"why":"Compares local and central DP in federated learning, providing the global-DP baseline the paper builds on.","marker":"[22]"},{"why":"Supplies the Alzheimer MRI dataset used in all experiments.","marker":"[30]"},{"why":"Provides the federated-learning implementation whose server-side DP wrapper is modified for metric privacy.","marker":"[24]"},{"why":"Defines FedOpt and adaptive federated optimization strategies evaluated in the experiments.","marker":"[27]"},{"why":"Defines FedProx, one of the six aggregation functions compared.","marker":"[26]"},{"why":"Defines FedMedian, another aggregation function compared.","marker":"[25]"}],"fun_headline_variants":["Distance-scaled noise boosts federated learning accuracy","Metric privacy improves convergence in federated medical imaging","Server-side distance scaling improves federated learning privacy","Federated learning with metric privacy: higher accuracy, equal attack defense","Privacy noise based on model distances outperforms standard DP"],"cache_read_input_tokens":23424,"weakest_assumption_plain":"The client-inference-attack evaluation rests on the assumption that a larger relative difference between loss on a target client's shadow dataset and loss on the aggregated test set reveals whether that client participated, yet the experiments never train a model without the target client to serve as a non-participation control.","fun_headline_variants_meta":{"raw":{"variants":["Distance-scaled noise boosts federated learning accuracy","Metric privacy improves convergence in federated medical imaging","Server-side distance scaling improves federated learning privacy","Federated learning with metric privacy: higher accuracy, equal attack defense","Privacy noise based on model distances outperforms standard DP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000398,"raw_usage":{"total_tokens":2089,"prompt_tokens":962,"completion_tokens":1127,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":1049}},"tokens_in":578,"tokens_out":1127,"duration_ms":9386,"temperature":1.0,"reasoning_tokens":1049,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T15:33:57.695571+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the identical federated setup with the target client excluded, then measure the same relative loss difference between shadow data and aggregated test set; if non-participating targets show a gap comparable to participating ones, the claimed protection against client inference attacks does not follow.","supporting_citations":[{"cited_title":"URL https://huggingface.co/datasets/Falah/Alzheimer_ MRI","cited_arxiv_id":null,"evidence_quote":"Supplies the Alzheimer MRI dataset used in all experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines FedProx, one of the six aggregation functions compared."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines FedMedian, another aggregation function compared."}],"review_version":1}