Detecting Atypical Clients in Federated Learning via Representation-Level Divergence

Alberto Fern\'andez-Hern\'andez; Cristian P\'erez-Corral; Enrique S. Quitana-Ort\'i; Jose I. Mestre; Manuel F. Dolz

arxiv: 2605.22266 · v1 · pith:JZPZQRF2new · submitted 2026-05-21 · 💻 cs.LG · cs.AI

Detecting Atypical Clients in Federated Learning via Representation-Level Divergence

Cristian P\'erez-Corral , Jose I. Mestre , Alberto Fern\'andez-Hern\'andez , Manuel F. Dolz , Enrique S. Quitana-Ort\'i This is my paper

Pith reviewed 2026-05-22 07:58 UTC · model grok-4.3

classification 💻 cs.LG cs.AI

keywords federated learningclient detectionrepresentation divergencefunctional deviationgeometric metricprobe setanomaly identificationmodel aggregation

0 comments

The pith

A geometric signal from activation partitions on a probe set identifies atypical clients in federated learning.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Federated learning often suffers from unstable updates due to data heterogeneity and potential anomalies among clients. This work introduces a lightweight method to measure a client's functional deviation from the global model by examining changes in the activation-induced partition of the input space. The evaluation happens on a shared probe set, producing a metric that is invariant to parameter permutations and easy to interpret. This signal can flag clients whose local updates cause significant shifts in how the model processes data. Consequently, it offers a practical way to monitor participants and apply safer aggregation rules.

Core claim

We propose a lightweight geometric signal to quantify the functional deviation of a client with respect to the global model by measuring how the local training of each client alters the activation-induced partition of the input space, evaluated on a shared probe set. This yields a permutation-invariant, interpretable metric of client-global divergence that captures differences in how data is processed by the model, effectively identifying clients that induce atypical functional changes.

What carries the argument

Activation-induced partition of the input space on a shared probe set as a geometric signal for functional deviation.

If this is right

The metric distinguishes stable heterogeneous clients from those with significant divergence.
It provides a tool for monitoring client behavior in federated learning systems.
The approach enables risk-aware aggregation strategies.
It addresses concerns about reliability in the aggregation process due to anomalous inputs or shifts.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Integrating this signal could lead to adaptive weighting of client contributions based on their divergence score.
The method might be extended to detect specific types of attacks like data poisoning in federated setups.
Similar representation-based checks could apply to other collaborative machine learning scenarios.

Load-bearing premise

That alterations in the activation-induced partition of the input space on a shared probe set provide a meaningful proxy for the functional deviation affecting global model aggregation.

What would settle it

Running the detection on a federated learning simulation where certain clients are engineered to have atypical updates and checking if the metric successfully flags them while ignoring normal heterogeneous clients.

Figures

Figures reproduced from arXiv: 2605.22266 by Alberto Fern\'andez-Hern\'andez, Cristian P\'erez-Corral, Enrique S. Quitana-Ort\'i, Jose I. Mestre, Manuel F. Dolz.

**Figure 1.** Figure 1: Per-client Dc across rounds for different Dirichlet α’s. Higher heterogeneity (α = 0.1) results in increased divergence and persistent client deviations, while near i.i.d. settings exhibit stable convergence. early layers define coarse separations and deeper layers refine them. To account for this structure, we define a hierarchical divergence that prioritizes discrepancies at early layers, in the sense th… view at source ↗

**Figure 2.** Figure 2: Per client Dc vs z-score of the different training rounds. The shifted clients are clearly identifiable, suggesting the validity of the metric to capture these anomalies. Dolz was supported by grant CNS2025-165098 funded by MICIU/AEI/10.13039/501100011033 and by the Plan Gen–T grant CIDEXG/2022/013 of the Generalitat Valenciana. REFERENCES [1] M. Chen, S. Mao, and Y. Liu, “Big data: A survey,” Mobile Netw… view at source ↗

read the original abstract

Federated learning enables collaborative training across distributed clients with heterogeneous data, but such heterogeneity often leads to unstable updates and degraded global performance. Moreover, in practical deployments, client updates may deviate from the expected behavior not only due to benign not i.i.d. distributions, but also due to distributional shifts or anomalous inputs, raising concerns about the reliability of the aggregation process. In this work, we propose a lightweight geometric signal to quantify the functional deviation of a client with respect to the global model. Instead of comparing model parameters or gradients, our approach measures how the local training of each client alters the activation-induced partition of the input space, evaluated on a shared probe set. This yields a permutation-invariant, interpretable metric of client--global divergence that captures differences in how data is processed by the model. We show that this signal effectively identifies clients that induce atypical functional changes, distinguishing stable yet heterogeneous clients from those whose updates significantly diverge from the global regime. As a result, the proposed metric provides a simple tool for monitoring client behavior and enabling risk-aware aggregation strategies in federated learning systems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper introduces a representation-level divergence metric for spotting atypical clients in federated learning by tracking activation partition shifts on a probe set, but the link to actual aggregation problems remains unproven.

read the letter

The main thing to know is that this paper proposes measuring client divergence in federated learning through changes in how the model partitions the input space based on activations, evaluated on a shared probe set. This replaces the usual parameter or gradient comparisons with a geometric signal meant to flag clients that cause atypical functional shifts during aggregation. The approach is presented as lightweight, permutation-invariant, and interpretable, aimed at distinguishing problematic updates from normal heterogeneity or distributional shifts. What stands out as new is the focus on representation-level effects rather than direct model differences, which could avoid some pitfalls in highly heterogeneous settings and provide a practical monitoring tool. The paper does a reasonable job framing the problem and explaining why this signal might support risk-aware aggregation strategies without heavy computation. On the soft spots, the central assumption that partition changes reliably proxy the functional deviations driving global instability is not strongly supported yet. There is no clear demonstration in the visible material that high metric values lead to measurable shifts in global loss or decision boundaries after averaging, nor does it rule out false positives from benign clients. Experiments or causal analysis would be needed to confirm the metric actually predicts aggregation issues rather than just capturing any difference. The work appears conceptual with limited validation details from the abstract, though if the full paper includes reproducible tests on standard FL benchmarks that tie the signal to performance impacts, that would address the main gap. This is for practitioners and researchers building monitoring or defense mechanisms in real-world federated deployments where client reliability matters. A reader focused on robust aggregation would get some value from the new framing and could adapt the probe-set idea. I would send it for peer review so referees can check the experiments and push for tighter evidence on the proxy's effectiveness.

Referee Report

2 major / 1 minor

Summary. The paper proposes a lightweight geometric signal for federated learning that quantifies a client's functional deviation from the global model by measuring alterations to the activation-induced partition of the input space, evaluated on a shared probe set. This yields a permutation-invariant metric claimed to identify atypical clients whose updates diverge significantly, distinguishing them from stable yet heterogeneous clients and supporting risk-aware aggregation strategies.

Significance. If the metric is shown to reliably flag clients whose updates produce atypical effects on the aggregated model, the approach could offer a practical, parameter-light monitoring tool for handling data heterogeneity, distributional shifts, and anomalies in federated systems without direct parameter or gradient comparisons.

major comments (2)

Abstract: the central claim that the signal 'effectively identifies clients that induce atypical functional changes' and distinguishes them from stable heterogeneous clients lacks any supporting equations, validation experiments, or error analysis. This is load-bearing for the identification utility asserted in the abstract.
Approach (activation-induced partition metric): the assumption that divergence in the activation-induced partition on a fixed probe set provides a meaningful and sufficient proxy for the functional deviations that affect global model aggregation is not demonstrated. The manuscript should include tests showing that high metric values causally correlate with larger post-aggregation shifts in global loss or decision boundaries, rather than being produced by benign heterogeneous clients.

minor comments (1)

The abstract introduces 'activation-induced partition' without a concise definition or reference to the relevant section or equation; adding this would improve immediate clarity for readers.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the insightful comments on our work. We provide detailed responses to each major comment and outline the revisions we will make to address the concerns raised.

read point-by-point responses

Referee: Abstract: the central claim that the signal 'effectively identifies clients that induce atypical functional changes' and distinguishes them from stable heterogeneous clients lacks any supporting equations, validation experiments, or error analysis. This is load-bearing for the identification utility asserted in the abstract.

Authors: While the abstract is a high-level summary, the manuscript provides supporting validation through experiments detailed in the results section. These experiments demonstrate the metric's effectiveness in identifying atypical clients by comparing divergence scores across different client types and showing correlation with model performance impacts. To strengthen the abstract, we will revise it to include a concise mention of the experimental evidence and the distinction from heterogeneous clients. We will also ensure that key equations defining the metric are referenced. revision: yes
Referee: Approach (activation-induced partition metric): the assumption that divergence in the activation-induced partition on a fixed probe set provides a meaningful and sufficient proxy for the functional deviations that affect global model aggregation is not demonstrated. The manuscript should include tests showing that high metric values causally correlate with larger post-aggregation shifts in global loss or decision boundaries, rather than being produced by benign heterogeneous clients.

Authors: We agree that demonstrating the link to aggregation effects is important. Our current results indicate that high metric values are associated with clients that cause notable shifts in the global model, unlike stable heterogeneous clients. To explicitly show the causal correlation, we will add new experiments measuring post-aggregation changes in global loss and decision boundaries as a function of the metric value. This will include comparisons to rule out benign heterogeneity and validate the probe set's role. These additions will be included in the revised manuscript. revision: yes

Circularity Check

0 steps flagged

No circularity: metric defined as direct observable on probe set

full rationale

The paper introduces a geometric signal that quantifies client-global divergence by measuring changes in activation-induced input-space partitions evaluated on a shared probe set. This definition is self-contained and does not rely on any fitted parameters, predictions that reduce to inputs by construction, or load-bearing self-citations. No derivation chain is presented that equates a claimed result to its own inputs; the metric is proposed as an independent, permutation-invariant observable for monitoring client behavior.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The abstract introduces no explicit free parameters, mathematical axioms, or new postulated entities; it relies on standard federated-learning assumptions about client heterogeneity and the utility of a shared probe set.

pith-pipeline@v0.9.0 · 5743 in / 1044 out tokens · 43212 ms · 2026-05-22T07:58:04.055306+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

we propose a lightweight geometric signal ... measures how the local training of each client alters the activation-induced partition of the input space, evaluated on a shared probe set ... K_θ_l(i,j) = 1 - (1/n_l) H(a_l(x_i;θ), a_l(x_j;θ)) ... D_hier(θ,θ') = d_1 + Σ (∏ λ_r) d_l
IndisputableMonolith/Foundation/AlexanderDuality.lean alexander_duality_circle_linking unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

activation patterns ... define a partition of the input space into regions ... piecewise-affine decomposition

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

15 extracted references · 15 canonical work pages

[1]

Big data: A survey,

M. Chen, S. Mao, and Y . Liu, “Big data: A survey,”Mobile Networks and Applications, 2014

work page 2014
[2]

Advances and open problems in federated learning,

P. Kairouz and H. B. McMahan, “Advances and open problems in federated learning,”Foundations and Trends in Machine Learning, vol. 14, no. 1-2, pp. 1–210, 2021

work page 2021
[3]

Communication-efficient learning of deep networks from decentralized data,

B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, “Communication-efficient learning of deep networks from decentralized data,” inProceedings of the 20th International Conference on Artificial Intelligence and Statistics, 2017

work page 2017
[4]

Federated optimization in heterogeneous networks,

T. Li, A. K. Sahuet al., “Federated optimization in heterogeneous networks,” inProceedings of Machine Learning and Systems, 2020

work page 2020
[5]

Adaptive federated optimization,

S. J. Reddi, Z. Charles, M. Zaheer, Z. Garrett, K. Rush, J. Kone ˇcný, S. Kumar, and H. B. McMahan, “Adaptive federated optimization,” in International Conference on Learning Representations, 2021

work page 2021
[6]

Fedbn: Federated learning on non-iid features via local batch normalization,

X. Li, M. Jiang, X. Zhang, M. Kamp, and Q. Dou, “Fedbn: Federated learning on non-iid features via local batch normalization,” inInterna- tional Conference on Learning Representations, 2021

work page 2021
[7]

Exploiting shared representations for personalized federated learning,

L. Collins, H. Hassani, A. Mokhtari, and S. Shakkottai, “Exploiting shared representations for personalized federated learning,” inInterna- tional Conference on Machine Learning, 2021

work page 2021
[8]

Fedbabu: Toward enhanced representation for federated image classification,

J. Oh, S. Kim, and S.-Y . Yun, “Fedbabu: Toward enhanced representation for federated image classification,” inInternational Conference on Learning Representations, 2022

work page 2022
[9]

On the number of linear regions of deep neural networks,

G. Montúfar, R. Pascanu, K. Cho, and Y . Bengio, “On the number of linear regions of deep neural networks,” inAdvances in Neural Information Processing Systems, 2014

work page 2014
[10]

On the expressive power of deep neural networks,

M. Raghu, B. Poole, J. Kleinberg, S. Ganguli, and J. Sohl-Dickstein, “On the expressive power of deep neural networks,” inInternational Conference on Machine Learning, 2017

work page 2017
[11]

Convergence of deep relu networks,

Y . Xu and H. Zhang, “Convergence of deep relu networks,”Neurocom- puting, 2024

work page 2024
[12]

Studying the evolution of neural activation patterns during training of feed-forward relu networks,

D. Hartmann, D. Franzen, and S. Brodehl, “Studying the evolution of neural activation patterns during training of feed-forward relu networks,” Frontiers in Artificial Intelligence, 2021

work page 2021
[13]

Practical secure aggregation for privacy-preserving machine learning,

K. Bonawitz, V . Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, “Practical secure aggregation for privacy-preserving machine learning,” inACM CCS, 2017

work page 2017
[14]

Ma- chine learning with adversaries: Byzantine tolerant gradient descent,

P. Blanchard, E. M. El Mhamdi, R. Guerraoui, and J. Stainer, “Ma- chine learning with adversaries: Byzantine tolerant gradient descent,” in NeurIPS, 2017

work page 2017
[15]

Regime change hypothesis: Foun- dations for decoupled dynamics in neural network training,

C. Pérez-Corral, A. Fernández-Hernández, J. I. Mestre, M. F. Dolz, J. Duato, and E. S. Quintana-Orti, “Regime change hypothesis: Foun- dations for decoupled dynamics in neural network training,” 2026

work page 2026

[1] [1]

Big data: A survey,

M. Chen, S. Mao, and Y . Liu, “Big data: A survey,”Mobile Networks and Applications, 2014

work page 2014

[2] [2]

Advances and open problems in federated learning,

P. Kairouz and H. B. McMahan, “Advances and open problems in federated learning,”Foundations and Trends in Machine Learning, vol. 14, no. 1-2, pp. 1–210, 2021

work page 2021

[3] [3]

Communication-efficient learning of deep networks from decentralized data,

B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, “Communication-efficient learning of deep networks from decentralized data,” inProceedings of the 20th International Conference on Artificial Intelligence and Statistics, 2017

work page 2017

[4] [4]

Federated optimization in heterogeneous networks,

T. Li, A. K. Sahuet al., “Federated optimization in heterogeneous networks,” inProceedings of Machine Learning and Systems, 2020

work page 2020

[5] [5]

Adaptive federated optimization,

S. J. Reddi, Z. Charles, M. Zaheer, Z. Garrett, K. Rush, J. Kone ˇcný, S. Kumar, and H. B. McMahan, “Adaptive federated optimization,” in International Conference on Learning Representations, 2021

work page 2021

[6] [6]

Fedbn: Federated learning on non-iid features via local batch normalization,

X. Li, M. Jiang, X. Zhang, M. Kamp, and Q. Dou, “Fedbn: Federated learning on non-iid features via local batch normalization,” inInterna- tional Conference on Learning Representations, 2021

work page 2021

[7] [7]

Exploiting shared representations for personalized federated learning,

L. Collins, H. Hassani, A. Mokhtari, and S. Shakkottai, “Exploiting shared representations for personalized federated learning,” inInterna- tional Conference on Machine Learning, 2021

work page 2021

[8] [8]

Fedbabu: Toward enhanced representation for federated image classification,

J. Oh, S. Kim, and S.-Y . Yun, “Fedbabu: Toward enhanced representation for federated image classification,” inInternational Conference on Learning Representations, 2022

work page 2022

[9] [9]

On the number of linear regions of deep neural networks,

G. Montúfar, R. Pascanu, K. Cho, and Y . Bengio, “On the number of linear regions of deep neural networks,” inAdvances in Neural Information Processing Systems, 2014

work page 2014

[10] [10]

On the expressive power of deep neural networks,

M. Raghu, B. Poole, J. Kleinberg, S. Ganguli, and J. Sohl-Dickstein, “On the expressive power of deep neural networks,” inInternational Conference on Machine Learning, 2017

work page 2017

[11] [11]

Convergence of deep relu networks,

Y . Xu and H. Zhang, “Convergence of deep relu networks,”Neurocom- puting, 2024

work page 2024

[12] [12]

Studying the evolution of neural activation patterns during training of feed-forward relu networks,

D. Hartmann, D. Franzen, and S. Brodehl, “Studying the evolution of neural activation patterns during training of feed-forward relu networks,” Frontiers in Artificial Intelligence, 2021

work page 2021

[13] [13]

Practical secure aggregation for privacy-preserving machine learning,

K. Bonawitz, V . Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, “Practical secure aggregation for privacy-preserving machine learning,” inACM CCS, 2017

work page 2017

[14] [14]

Ma- chine learning with adversaries: Byzantine tolerant gradient descent,

P. Blanchard, E. M. El Mhamdi, R. Guerraoui, and J. Stainer, “Ma- chine learning with adversaries: Byzantine tolerant gradient descent,” in NeurIPS, 2017

work page 2017

[15] [15]

Regime change hypothesis: Foun- dations for decoupled dynamics in neural network training,

C. Pérez-Corral, A. Fernández-Hernández, J. I. Mestre, M. F. Dolz, J. Duato, and E. S. Quintana-Orti, “Regime change hypothesis: Foun- dations for decoupled dynamics in neural network training,” 2026

work page 2026