Topology-Aware Differential Privacy in Hierarchical Federated Learning

Murtaza Rangwala; Rajkumar Buyya; Richard O. Sinnott

Topology-Aware Differential Privacy in Hierarchical Federated Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proves that a federation's communication topology leaks client statistics that DP-SGD noise cannot suppress, and derives a closed-form noise allocation provably better than uniform DP-SGD whenever the topology is asymmetric.

desk verdict Genuinely new threat model and a plausible bound, but the unconditional-safety claims outrun what the proxies prove; deserves a serious referee. read the letter →

arxiv 2506.19260 v3 pith:4SL5M7BW submitted 2025-06-24 cs.CR cs.DCcs.LG

Murtaza Rangwala , Richard O. Sinnott , Rajkumar Buyya This is my paper · ORCID

classification cs.CRcs.DCcs.LG

keywords federatedlearningdifferentialprivacynetworktopologymutualinformationdistributionalinferenceper-clientnoiseallocationcross-silostructuralleverage

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Federated learning sends only model updates to the server, and DP-SGD is supposed to bound what those updates reveal, but this paper argues that the network that carries them leaks on its own. A passive adversary who knows the communication topology and each client's organizational membership can infer per-site sensitive-class concentrations, such as a hospital's melanoma rate, through a channel that DP-SGD never touches. The authors prove that this leakage separates additively into a controllable mechanism term that per-client noise can shrink and an uncontrollable prior-coupling floor fixed by the client's structural position. From the bound they derive Fulcrum, a closed-form per-client noise allocation that minimizes the worst-case bound and strictly dominates uniform DP-SGD whenever leverage scores are asymmetric, degenerating exactly to uniform DP-SGD when they are not. If the result holds, federations get strictly more privacy per unit of utility cost by reallocating noise according to topology, with measured bound reductions up to about two nats (natural units of information) on real healthcare benchmarks.

What carries the argument

The load-bearing object is structural leverage $\ell_i^\circ$, the worst-case mutual information $I_P(p_i; D_{-i})$ between a client's sensitive-class concentration and all other clients' datasets, taken over the prior family consistent with the deployment graph and organizational labels; it quantifies how much a client's structural position alone reveals about its data. The additive bound is assembled from the data-processing inequality and the mutual-information chain rule; the mechanism term is bounded by Rényi-DP composition for the Gaussian mechanism with sensitivity $C/|B|$, converted to mutual information via the max-KL conversion of [26]. The allocation itself solves a convex min-max program with a slack variable $K$: KKT stationarity forces every per-client constraint active, giving the closed form $\sigma_i^{*2} = a/(K^* - \ell_i^\circ)$ with $K^*$ the unique root of the budget equation, which equalizes the worst-case bound across clients. Three practical proxies, group size, graph degree, and dataset size, stand in for the abstract $\ell_i^\circ$ in deployment, each matched to the dominant source of asymmetry.

What would settle it

Build a small federation ($n \le 10$) where the true structural leverage $\ell_i^\circ$ can be computed by enumerating priors in $\mathcal{F}_{G,\omega}$, and compare it with each deployed proxy: if any graph or dataset-size assignment yields $\ell_i^\circ$ greater than the proxy for some client, the proxy is not an upper bound and that client's per-client guarantee fails. Alternatively, run the full-channel TADI adversary against a Fulcrum-protected federation and check whether any client's measured leakage exceeds $T_{\max}/(2\sigma_i^{*2}|B|^2)$ plus the proxy value, an exceedance that would show the allocation under-noised an exposed client.

Watch

Extended reading notes

Core claim

The paper's central claim, Theorem V.2, is that under independent DP-SGD noise and disjoint client datasets, any deterministic adversary extracting an estimate $\hat{p}_i$ of client $i$'s sensitive-class concentration from the observed update tensor satisfies $I(p_i; \hat{p}_i \mid G, \omega, \{\sigma_j\}) \le T_{\max}/(2\sigma_i^2 |B|^2) + \ell_i^\circ$ for any prior in the family $\mathcal{F}_{G,\omega}$: a controllable mechanism term that decays as the client's noise $\sigma_i$ grows, plus an uncontrollable prior-coupling floor $\ell_i^\circ = \sup_{P \in \mathcal{F}_{G,\omega}} I_P(p_i; D_{-i})$ that measures how predictable $p_i$ is from the rest of the federation's data. Theorem V.3 then derives Fulcrum's balanced min-max allocation $\sigma_i^{*2} = a/(K^* - \ell_i^\circ)$, where $a = T_{\max}/(2|B|^2)$ and $K^*$ is the unique root of $\sum_i a/(K^* - \ell_i^\circ) = U$ for a fixed total noise-variance budget $U$; Corollary V.4 states the resulting worst-case bound $K^*$ is strictly below uniform DP-SGD's unless all leverage scores are equal. The TADI channel decomposition, a shadow-trained regressor with four ablations, confirms the additive structure empirically: the parameter channel never beats the constant-mean baseline, while the organizational channel reaches perfect client ranking exactly when the adversary's shadow prior matches the deployment prior. Across Fed-ISIC2019, Fed-Heart-Disease, and synthetic CIFAR-10 with nine topology configurations, the allocation reduces the privacy bound by up to 1.967 nats with statistically equivalent utility.

Load-bearing premise

The guarantee as deployed relies on the chosen leverage proxy never understating a client's true structural leverage $\ell_i^\circ$; the paper proves that upper-bound property only for the group-size proxy and only asymptotically, and explicitly leaves a tight upper bound open for the degree and dataset-size proxies, so a proxy that underestimates $\ell_i^\circ$ could let Fulcrum give a highly exposed client less noise than uniform DP-SGD would.

Editorial extensions

If this is right

Fulcrum lowers the worst-case per-client privacy bound at every tested utility budget and observation window across all three benchmarks, by up to 1.967 nats on Fed-ISIC2019, with no measurable utility cost (equivalence within ±0.5 percentage points).
On symmetric topologies, rings, complete graphs, balanced hierarchies, the allocation degenerates exactly to uniform DP-SGD, so adopting it never worsens privacy in a symmetric federation.
The TADI decomposition shows the parameter channel is already bounded by DP-SGD in every configuration tested, so the residual, noise-immune risk sits in the prior-coupling channel.
The prior-coupling floor is empirically realizable only when the adversary's shadow prior matches the deployment prior; under the realistic public-proxy construction it is not reached, making the bound conservative in the deployment's favor.
The group-size proxy asymptotically upper-bounds true leverage under stochastic block model priors, while the degree and dataset-size proxies are supported by ordering consistency and Pareto dominance across tested settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A concrete hazard follows from the paper's own limitation: if a degree or dataset-size proxy understates $\ell_i^\circ$ for any client, Fulcrum could allocate that client less noise than uniform DP-SGD; a safety-margin inflation of the proxies, calibrated on the TADI full-channel adversary, would close the gap.
The bound's two sources of slack, the dropped term $I(p_i; D_{-i} \mid \Theta)$ and the loose conversion at extreme noise, point to a tighter accountant (amplified RDP scaling the mechanism term from $1/(\sigma_i^2 |B|^2)$ toward $1/(\sigma_i^2 |D_i|^2)$), which would strengthen per-client guarantees when local datasets are much larger than batch sizes.
Equalizing the worst client is one policy; minimizing the average bound instead would trade a slightly worse worst case for lower typical leakage, a variant worth evaluating where aggregate exposure is the legal concern.
The channel decomposition suggests a deployment-time diagnostic: use matched-prior TADI lifts as an empirical estimate of $\ell_i^\circ$ and compare with the chosen proxy's ranking, letting operators validate the proxy before trusting the allocation.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my read on 2506.19260. The core idea is real: nobody has treated topology as a known leakage channel in DP-FL, and this paper is the first to decompose per-client leakage into a noise-controllable mechanism term and a structural prior-coupling floor. The TADI four-ablation decomposition is a sensible empirical instrument, and the water-filling allocation in Theorem V.3 is textbook-correct KKT. Credit where earned: the evaluation is unusually thorough — three benchmarks, six topology families, TOST equivalence testing, released code. Section VII is candid about what is not proven.

The soft spots are all in the gap between the theorems and the claims. The big one is the leverage proxies. Theorem V.2 only guarantees the per-client MI bound when the weights upper-bound the true structural leverage. Only the group-size proxy has a bounding result, and it's asymptotic; the degree and dataset-size proxies are explicitly unproven. If a proxy underestimates a high-exposure client's leverage, the allocation gives that client less noise than uniform DP-SGD and the per-client guarantee collapses. Section VII admits exactly this, yet the abstract and conclusion say 'safe to adopt unconditionally.' That is not supported. This is an internal gap.

Second, Figure 3's eta-sweep does not reproduce from the text. The proxies are described as functions of degree, group size, or dataset size — all eta-independent in Setting C — yet the gap grows monotonically with eta. Either an eta-dependent quantity is entering the leverage computation, or the figure was produced by something other than the described method. The central empirical validation is unreproducible as written.

Third, the 'uncontrollable floor' ell^o_i is defined as the supremum of I(p_i; D_-i), the very quantity in question. That makes the second term of Theorem V.2 definitional rather than derived. The bound isn't wrong — bounding a term by its supremum is fine — but the paper should say plainly that the floor is a worst-case construction, not a derived quantity. Note that Theorem V.3 holds for any weight vector, so the allocation itself is robust; only the per-client guarantee inherits the proxy problem.

Minor point: Lemma A.3's conditioning explanation is muddled. The inequality is standard sigma-algebra monotonicity, but the Markov-chain argument as written goes the wrong way. Easy fix.

Bottom line: the threat model and the channel decomposition deserve the FL privacy community's attention. This paper deserves peer review, not desk rejection, but it needs major revision: prove or drop the proxy upper-bound story, make the eta-sweep reproducible, and consistently separate bound reductions from measured privacy gains. I'd send it out.

Referee Report

3 major / 5 minor

Summary. The paper studies a passive adversary who knows the communication topology and organisational labels of a federated learning system and attempts to infer each client's sensitive-class concentration p_i from the sequence of DP-protected model updates. The authors define a 'structural leverage' ℓ^◦_i as the worst-case mutual information between p_i and the other clients' datasets, prove an additive per-client bound separating a noise-controllable term from this leverage floor (Theorem V.2), and derive a closed-form min-max noise allocation (FULCRUM, Theorem V.3) that they claim strictly dominates uniform DP-SGD whenever the leverage scores are non-uniform (Corollary V.4). They also present TADI, a shadow-trained regressor with four channel ablations, and evaluate FULCRUM and TADI on Fed-ISIC2019, Fed-Heart-Disease, and synthetic CIFAR-10 across several topology families.

Significance. If the main claims were fully established, this would be a useful contribution to an underexplored intersection of FL privacy and topology: framing topology as a known leakage channel rather than only as a privacy amplification mechanism is novel, and the closed-form allocation is elegant and easy to implement. The paper ships code and uses a careful equivalence-testing protocol for utility comparisons. However, the advertised unconditional guarantee is not supported: the theorem's per-client bound is only meaningful when the leverage proxy upper-bounds the true ℓ^◦_i, which the paper itself leaves open for two of the three proxies, and the reported η-dependent privacy-bound gaps in Section VI-C are not predicted by the stated theory for topology-only proxies. The contribution is potentially valuable, but the central claims need substantial clarification and narrowing.

major comments (3)

[Sections V-E and VII] The per-client guarantee of Theorem V.2 is not preserved under two of the three proposed proxies. Theorem V.2 bounds I(p_i; \hat p_i) by a/σ_i^2 + ℓ^◦_i, with ℓ^◦_i the true structural leverage of Definition V.1. When the allocation is computed from a proxy \tilde ℓ_i, the achieved per-client bound is a/σ_i^2 + ℓ^◦_i, where σ_i^2 = a/(K^* − \tilde ℓ_i). If \tilde ℓ_i < ℓ^◦_i for some client, that client receives less noise than the true optimum and the per-client bound of Eq. (2) is not guaranteed. Section V-E states that a tight upper bound on ℓ^◦_i remains open for the degree and dataset-size proxies, and Section VII explicitly concedes that 'the per-client bound of Theorem V.2 for that client is not guaranteed' when a proxy underestimates ℓ^◦_i. The abstract and conclusion nevertheless claim FULCRUM is 'safe to adopt unconditionally'. This is an internal inconsistency in the central advertised guarantee, not merely a presentation issue.
[Definition V.1 and Theorem V.2] The uncontrollable term in Theorem V.2 is definitional rather than derived. Since ℓ^◦_i is defined as sup_{P∈F_G,ω} I_P(p_i; D_{−i}), the second term of Eq. (2) is exactly the supremum of the lateral mutual information that the chain-rule proof already isolates. Consequently Theorem V.2 provides no upper bound that is computable from (G,ω) alone unless an independent upper bound on this supremum is supplied. The statement in Section V-E that Theorems V.2 and V.3 hold for any non-negative weight vector substituted in place of {ℓ^◦_i} confirms that the min-max allocation optimizes an abstract weight profile rather than a quantity tied to actual privacy risk; the paper should explicitly frame the result as a decomposition and should state that all concrete privacy guarantees require a validated conservative proxy.
[Section VI-C, Figure 3] The reported η-dependence of the privacy-bound gap is not a consequence of the stated theory. The leverage proxies in Section V-E are functions of (G,ω) and dataset sizes only, and ℓ^◦_i is a supremum over F_G,ω, so for a fixed topology the gap K_uniform − K^* in Corollary V.4 should be independent of the coupling parameter η. Figure 3 shows the gap growing monotonically in η and saturating at star and BA topologies, which indicates that the leverage values used to compute K^* and K_uniform in this experiment depend on the realized generative model rather than on the public topology alone. The paper does not specify how ℓ^◦_i is instantiated in Setting C, so the claimed empirical confirmation of Corollary V.4 is not reproducible from the described methodology.

minor comments (5)

[Abstract and Conclusion] The phrase 'privacy gains' should be replaced with 'privacy-bound reductions' throughout, because the 1.967 nats figure and the other reported gaps are reductions in an upper bound, not reductions in actual mutual information.
[Section V-E] The group-size proxy's claim that ℓ^◦_i ≤ H(Φ_{ω_i}) asymptotically needs a formal statement of the asymptotic regime and the assumed stochastic block model; as written, the sentence is not precise enough to be checked.
[Appendix A, Lemma A.3] The argument in Step 2 of Lemma A.3, which replaces conditioning on θ^{(<t)} by conditioning on θ^{(t−1)}, is compressed and does not immediately follow from the displayed Markov-chain statement; since this is a central composition step, a fuller proof should be supplied.
[Section VI-B] The evaluation section states that noise scales are set via σ^{*2}_i = a/(K^* − ℓ^◦_i), but the practical implementations use proxies; the paper should state explicitly which proxy is used for each setting and each figure, including Figure 3 and Figure 4.
[Figure 3] The colorbar label 'an/U Kuniform K*' is unclear; the figure caption and the axis label should state unambiguously that the color encodes K_uniform − K^* in nats.

Circularity Check

2 steps flagged · score 6.0 of 10

Uncontrollable floor is defined as the supremum of the MI it bounds, and proxy-based dominance gaps are algebraic consequences of arbitrary non-uniform weights.

self definitional [Definition V.1; Theorem V.2 proof sketch; Lemma A.4 (Appendix B)]
"Lemma A.4 (Lateral-leakage bound). Under (IA2): I(pi;D −i) ≤ ℓ◦_i. Proof. Immediate from Definition V .1 of the main paper, which defines ℓ◦_i as sup P∈FG,ω IP(pi;D−i)."

Theorem V.2's proof bounds the lateral term I(pi;D−i) by ℓ◦_i via Lemma A.4, but ℓ◦_i is defined (Definition V.1) as sup_{P∈FG,ω} I_P(pi;D−i), i.e., the supremum of that very quantity. The chain-rule step I(pi;Θ) ≤ I(pi;D−i) + I(pi;Θ|D−i) therefore contributes no independent bound on the 'uncontrollable floor'; the floor is the definitional input, not a derived prediction. The advertised additive separation of Theorem V.2 holds by construction: the second addend is a renamed worst-case of the same mutual information it is used to bound.
fitted input called prediction [Section V-E (Practical Leverage Proxies); Section VI-D; Section VII]
"Theorems V .2 and V .3 hold for any non-negative weight vector substituted in place of {ℓ◦_i}: the allocation strictly dominates uniform DP-SGD whenever the weights are non-uniform, regardless of whether they equal the true structural leverage. The proxies are therefore validated on two criteria: ordering consistency, meaning the proxy correctly ranks clients by structural exposure, which is sufficient for Corollary V .4 to apply; and empirical conservatism, meaning the allocation consistently reduces K_uniform in practice."

Corollary V.4 guarantees K⋆ < K_uniform for every non-uniform weight vector, so the 'empirical conservatism' criterion is automatically satisfied by any non-uniform proxy; it cannot distinguish a proxy that upper-bounds true leverage ℓ◦_i from an arbitrary weight vector. The Section VI-D Pareto gaps are computed from these proxy weights with the same closed-form bisection, so the reported privacy-bound improvements are algebraic consequences of the chosen weights rather than measurements of structural leakage. Section VII concedes that if a proxy underestimates ℓ◦_i, the Theorem V.2 per-client bound is not guaranteed and the Pareto dominance is only empirical evidence, not a theoretical guarantee.

full rationale

The min-max allocation of Theorem V.3 is a standard and correct KKT/water-filling derivation: for any given weight vector, σ*_i^2 = a/(K*−w_i) exactly solves min max_i(a/σ_i^2 + w_i) subject to sum σ_i^2 ≤ U, and Corollary V.4 is a direct algebraic consequence. There is no load-bearing self-citation: reference [39] only names the Murmura implementation framework, and the utility-equivalence experiments in Section VI-D are genuine and independent of the theoretical bound. However, two load-bearing steps do reduce by construction. First, Theorem V.2's 'uncontrollable prior-coupling floor' ℓ°_i is defined as the supremum of I(p_i;D_−i), the exact quantity that Lemma A.4 then 'bounds' by ℓ°_i; the additive split is therefore a definitional input rather than a derived prediction of the floor's value. Second, the validation of the degree and dataset-size proxies through 'empirical conservatism' is by construction: any non-uniform weight vector strictly reduces K_uniform, so the reported privacy-bound gaps cannot confirm that those proxies upper-bound true leverage. The paper itself acknowledges this in Section VII, conceding that the per-client guarantee is lost if a proxy underestimates ℓ°_i and that the Pareto dominance is then only empirical. That concession is a correctness limitation on the unconditional-safety claim, not itself a circular step. Overall, the controllable mechanism term is derived in a self-contained way, but the advertised 'safe to adopt unconditionally' conclusion rests on definitional and by-construction steps at its core, warranting a partial-circularity score.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The derivation rests on standard DP composition plus two independence assumptions. The main uncharged input is the structural leverage vector, which is replaced by proxies whose relationship to the true worst-case leverage is only partially proven. No new physical or mathematical entities are postulated.

free parameters (2)

Leverage proxy weights = Setting-dependent (e.g., normalized dataset size in Settings A and B, degree in Setting C)
Fulcrum replaces the unknown structural leverage ell_i^o with one of three proxies. The dominance result and the numerical gaps depend on these weights, and the paper does not prove they upper-bound ell_i^o for the degree and dataset-size proxies.
Leverage proxy normalization = Unit mean
Section V-E says proxies can be blended after normalizing to unit mean. The absolute scale of the proxy vector affects the water-filling solution and hence the reported privacy-bound gaps.

assumptions (5)

domain assumption IA1: DP-SGD noise is independent across all clients and rounds.
Stated in Section III-C. Required for the sequential composition argument in Lemma A.3, which chains per-round Gaussian mechanism bounds.
domain assumption IA2: Client datasets are disjoint.
Stated in Section III-A and III-C. Needed for the Markov/independence step that drops the dependence of client i's update on other clients' data.
domain assumption Priors in F_G,w factor over connected components, satisfy a Markov property with respect to G, and have bounded second moments.
Section V-A imposes these conditions to make ell_i^o finite and to justify the graphical-model-style chain-rule manipulations. The paper states that the deployment priors used in Section VI satisfy them.
standard math Cuff-Yu max-KL conversion bounds mutual information by the same constant as the KL divergence.
Used in Lemma A.2 to convert the Gaussian-mechanism KL bound into the mutual-information bound I(D_i; theta_i^{(t)} | theta^{(t-1)}, D_-i) <= 1/(2 sigma_i^2 |B|^2). The paper cites Cuff and Yu [26] for this conversion.
domain assumption Total noise variance U is a valid proxy for DP-SGD utility cost.
Section V-D motivates the budget constraint sum sigma_i^2 <= U by citing the convergence analysis of Wei et al. [24]. If utility cost is not governed by total variance, the budget constraint and the no-utility-cost claim may not generalize.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Topology-Aware Differential Privacy in Hierarchical Federated Learning." pith.science (2026). https://pith.science/paper/4SL5M7BW

@misc{pith2026250619260,
  author       = {Pith},
  title        = {Pith review of: Topology-Aware Differential Privacy in Hierarchical Federated Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4SL5M7BW}},
  note         = {Machine review of arXiv:2506.19260}
}

abstract

Hierarchical federated learning places regional aggregators between clients and the cloud, so a participant's update is observed only alongside its neighbours'. The concealment this arrangement provides depends on the size of the aggregation region, and regions in operational deployments vary widely. Prevailing practice applies a single noise multiplier to every participant, calibrated for the most exposed region, so every other participant carries more noise than its own exposure requires. We show that this allocation problem admits an explicit solution. We first give a silo-level differential privacy guarantee for the mechanism, then bound the mutual information between a participant's local class distribution and any estimate an observer positioned above the regional tier could form of it, using an adjacency notion matched to the quantity being protected. Minimising the worst-case bound under a fixed utility budget yields a min-max optimal allocation, which we call Fulcrum. The budget it recovers has a closed form we term the exposure dispersion, a measure of how unevenly aggregation weight is concentrated within regions relative to the most exposed one. Because this quantity follows from the region structure and the aggregation weights alone, a practitioner can evaluate it before training begins, and it vanishes precisely when all regions are equally exposed. On image and text classification at $\varepsilon = 0.99$, accuracy at a matched worst-case per-client guarantee improves by up to $14.84$ and $12.16$ percentage points where the dispersion is large, and is exactly zero on a balanced control for which the theory predicts parity.