Federated Learning: From Theory to Practice

A. Jung

REVIEW 2 major objections 4 minor 229 references

Federated Learning: From Theory to Practice

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The book's central claim: personalized federated learning reduces to one optimization principle—GTVMin—and FedAvg, FedSGD, FedProx, FedRelax, clustered, horizontal, vertical, and personalized FL are all instances of it.

desk verdict A useful, coherent textbook draft that unifies FL algorithms under GTVMin, but the advertised personalization guarantee is definitional rather than proven, and the statistical bounds only cover homogeneous clusters. read the letter →

arxiv 2505.19183 v2 pith:EHRCHBKT submitted 2025-05-25 cs.LG stat.ML

A. Jung This is my paper

classification cs.LGstat.ML MSC 68T0568W1590C2505C50

keywords federatedlearningpersonalizedgeneralizedtotalvariationminimizationFLnetworkgraphLaplaciandistributedoptimizationFedAvgProx

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The book argues that personalized federated learning is best understood not as a collection of separate algorithms, but as a single optimization problem: minimize each device's local loss plus a weighted penalty on how much connected devices' model parameters differ. This problem, called generalized total variation minimization (GTVMin), turns the federated learning network—which devices are linked and with what weight—into the central design choice. If GTVMin is the right frame, then FedSGD, FedAvg, FedProx, FedRelax, clustered, horizontal, vertical, and personalized FL all become instances of one principle, and improving an FL system means mainly choosing the network and the penalty. The book develops the mathematical consequences: convergence rates tied to the graph's Laplacian spectrum, a statistical bound showing similar devices learn similar parameters, and asynchronous algorithms whose convergence follows from contraction properties.

What carries the argument

The FL network: an undirected weighted graph whose nodes are devices and whose edge weights $A_{i,i'}$ represent similarity of learning tasks, together with the GTV regularizer $\sum_{\{i,i'\}\in E} A_{i,i'}\phi(w^{(i)}-w^{(i')})$, mainly $\phi=\|\cdot\|_2^2$. This pair turns personalization into a single convex objective whose quadratic form uses the graph Laplacian, so graph spectral quantities (especially $\lambda_2$) control both statistical accuracy and convergence speed of the resulting algorithms.

What would settle it

For a heterogeneous regression dataset, construct a graph with two connected nodes whose local ERM solutions are far apart in $\ell^2$ norm; if GTVMin with $\|\cdot\|_2^2$ cannot achieve lower average validation loss than either local ERM alone for any $\alpha>0$, then the squared-distance surrogate for task similarity fails.

Watch

Extended reading notes

Core claim

The central claim is that every major federated learning setup can be written as a GTVMin instance: $\min_{\{w^{(i)}\}} \sum_{i\in V} L_i(w^{(i)}) + \alpha \sum_{\{i,i'\}\in E} A_{i,i'} \phi(w^{(i)} - w^{(i')})$, where devices are nodes, weighted edges encode both communication links and statistical similarity between local datasets, and $\alpha$ controls how strongly similar devices are forced to share parameters. For the special case $\phi(\cdot)=\|\cdot\|_2^2$, the objective becomes a convex quadratic for linear local models, and its solution is characterized through the graph Laplacian $L(G)\otimes I$. The book shows that FedGD, FedSGD, FedAvg, FedProx, and FedRelax are all fixed-point iterations derived from GTVMin, and that single-model FL, clustered FL, horizontal FL, vertical FL, and personalized FL are special cases obtained by choosing the network and the GTV penalty. This is the author's intended contribution: a principled design principle, analogous to empirical risk minimization for centralized ML, that unifies theory and practice.

Load-bearing premise

The load-bearing premise is that a fixed undirected weighted graph can faithfully encode which devices have similar learning tasks, and that squared Euclidean distance between model parameters is an adequate measure of that similarity; asymmetric or higher-order similarities are explicitly out of scope.

Editorial extensions

If this is right

If GTVMin is the right frame, new FL algorithms can be designed systematically by picking a local loss, a GTV penalty, and a graph, then applying a distributed optimization method; the book does this for FedGD, FedSGD, FedAvg, FedProx, and FedRelax.
Because the objective is convex quadratic for linear models, convergence rates and optimal learning rates follow from the spectrum of the combined matrix formed by local Hessians plus $\alpha L(G)\otimes I$.
The bound in Proposition 3.1 implies that for connected networks and large $\alpha$, GTVMin solutions become nearly identical across devices, which is exactly the single-model FL regime; moderate $\alpha$ on cluster-structured graphs yields clustered FL.
Asynchronous variants converge when the local update operators form a pseudo-contraction, with a rate that degrades as the maximum delay between devices grows.
Model-agnostic FedRelax extends GTVMin to non-parametric models by comparing predictions on shared test data rather than parameter differences.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If GTVMin is taken as the design principle, then edge-weight construction—graph learning from local datasets—becomes as important as the optimization algorithm; one could test this directly by replacing hand-chosen graphs with learned ones and measuring downstream personalization quality.
The squared-Euclidean penalty assumption can be tested against asymmetric or higher-order task similarities; one could construct pairs of tasks where similarity is semantically directed and see whether a directed formulation is needed.
GTVMin also suggests a concrete benchmark: on heterogeneous FL datasets, compare FedAvg and FedProx against an exactly solved GTVMin instance to see how much of their empirical gains come from being approximations to the same objective.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a textbook draft, not a research preprint, and it should be judged as one. The GTVMin framing is genuinely useful: it gives a single design principle that contains FedSGD, FedAvg, FedProx, FedRelax, clustered FL, and personalized FL as special cases, and it does so with clean notation, self-contained proofs, and concrete pseudocode. Chapter 5's derivation of FedAvg from projected gradient descent is a nice pedagogical thread, and the connection to convex clustering is honestly cited. As an exposition, it succeeds. The soft spots are real but proportionate. There are no experiments, which is fine for a textbook but means every practical claim rests on the math. More importantly, the stress-test note is correct: Propositions 3.1 and 6.1 assume identical true parameters within the averaging set (w(i)=c, or y(i)=X(i)w(C) per cluster), so the bounds measure how well GTVMin enforces consensus or clustering, not how accurately it recovers genuinely heterogeneous personalized parameters. The abstract's "ensures that devices with similar learning tasks learn similar model parameters" is true by construction of the objective, not as a statistical consequence. The book does flag the homogeneous assumption in Section 3.4.2 and defers heterogeneous cases to clustered FL, but the personalization chapter (6.6) and the abstract do not carry that caveat. The bias introduced by a squared-Euclidean penalty when true parameters differ is never quantified, and no comparison to independent ERM is given. That is a genuine gap in the book's central narrative, though not a collapse: the optimization story is sound, and the pedagogical value survives. Who is this for? Advanced undergraduates and graduate students who want a principled map of FL algorithms, and researchers looking for a unified notation for personalization. It deserves a serious referee if submitted as a survey or textbook; the referee should push the author to either soften the personalization claim or add a proper bias-variance analysis for heterogeneous devices. I would not cite it as a source of new results, but I would point students to it.

Referee Report

2 major / 4 minor

Summary. This book develops a graph-based design principle for federated learning. It represents devices as nodes of a weighted undirected graph (the FL network), with edges encoding both communication links and task similarity, and proposes GTVMin (Eq. (53)) as a unified optimization formulation: minimize the sum of local losses plus a weighted squared-Euclidean penalty on parameter differences across edges. The book argues that FedSGD, FedAvg, FedProx, FedRelax, clustered FL, horizontal FL, vertical FL, and personalized FL are all instances of this principle, and it provides distributed gradient-based solvers, convergence bounds for quadratic objectives (Propositions 3.1, 5.1, 5.2, 6.1), and qualitative chapters on graph learning, trustworthiness, privacy, and cybersecurity, together with a glossary and exercises. The text is an accessible, theory-first textbook with self-contained proofs for the main bounds.

Significance. If the GTVMin unification is accepted, the book makes a valuable conceptual contribution: it reduces much of the FL algorithm design space to the choice of the FL network topology, edge weights, local losses, and the coupling strength alpha. The book is strong on presentation: the proofs of Propositions 3.1 and 6.1 are self-contained (Sections 3.8 and 6.9), the quadratic-form connection to graph Laplacians (Eq. (41)) is elegant, and the spectrum-based convergence analysis of Section 5.2 is informative. The interpretations via convex clustering, minimum-cost flow duality, and locally weighted learning (Section 3.6) are useful pedagogical bridges. However, the book includes no experimental evaluation, and more importantly, the statistical support for personalization under heterogeneous true parameters is incomplete; the abstract's 'ensures' statement is not established by the theory. The graph-encoding assumption is acknowledged explicitly in Section 3.2, which is honest, but it further conditions the practical scope of the claimed unification.

major comments (2)

[Abstract, §3.4.2 (Prop. 3.1), §6.3 (Prop. 6.1)] The book's central claim, stated in the abstract and Section 3.4 (Eq. (53)), is that GTVMin 'ensures that devices with similar learning tasks learn similar model parameters.' The only statistical results supporting this are Proposition 3.1 (bound (67)) and Proposition 6.1 (bound (167)). Both assume the true parameters are identical within the averaging set: Eq. (64) sets w(i)=c for all i, and Eq. (165) sets y(i)=X(i)w(C) with a single w(C) per cluster. Under these assumptions, the penalty acts on zero true variation, so the bounds measure how well GTVMin enforces consensus or clustering, not how accurately it recovers heterogeneous personalized parameters. When the true parameters differ across nodes, the squared-Euclidean penalty introduces a bias toward the graph-weighted mean of w*, scaled by alpha and the graph Laplacian of w*. The book provides no bound on this bias, no comparison with independent ERM, and no characterization of which graph structures or edge weights make the bias acceptable. Section 6.3 candidly acknowledges that (167) requires identical cluster parameters, but this does not reconcile the abstract's 'ensures' with the proven results. I recommend either adding a bias-variance analysis for heterogeneous w*(i) (e.g., bounding E||w_hat - w*||^2 relative to independent ERM in terms of lambda_2(L(G)), alpha, and the total variation of w*) or revising the claim to say that GTVMin encourages similar parameters and explicitly stating the heterogeneous regime as an open direction.
[§3.2, §7.2, §7.4] The book treats the FL network as the key design choice, but it never analyzes the effect of a mis-specified graph. Section 3.2 declares asymmetric similarity out of scope, and Section 3.3 focuses on squared Euclidean parameter distance as the penalty. If the true task similarities are asymmetric, higher-order, or not expressible as parameter proximity, the GTVMin solution is biased in ways the book does not quantify. Chapter 7 offers graph-learning heuristics (e.g., discrepancy measures and degree constraints) but no statistical guarantees connecting the estimated edge weights to the final personalized-model error. For a book with an applied title and a design principle stated as central, the absence of a sensitivity analysis or of sufficient conditions under which GTVMin outperforms independent ERM is a load-bearing gap: the 'ensures' claim relies entirely on the graph being a faithful encoding of task similarity.

minor comments (4)

[§6.4] The text contains an unresolved placeholder 'Figure??' in the discussion of horizontal FL and semi-supervised learning; this reference must be completed or removed.
[§5.5, Algorithms 8-9] The derivation of FedAvg from projected gradient descent should be labeled as a heuristic approximation. Algorithm 9 modifies Algorithm 8 by using R>1 local steps, approximate gradients, and random client subsets, so the projected-GD convergence analysis of Section 4.6 (bounds (86)-(90)) does not directly apply to FedAvg, and the book provides no separate convergence guarantee for it.
[§2.4, §5.4, §6.3] There are several small presentation issues: a stray closing parenthesis after 'Figure 2.3' in Section 2.4; in Section 5.4 the symbol B is used both for a batch set and for its size; and in the proof of Proposition 6.1 (Section 6.9.1) the notation f'(w(i)) is ambiguous because the same symbol denotes both a function of the stacked vector and a function of the local parameter block.
[§7.2] The approximation lambda_2(L(C)) ≈ d_max (Eq. (173)) is used to draw design conclusions, but the book states that a precise quantification of the approximation error is beyond scope. A pointer to the relevant concentration results (already cited as [96,98]) would suffice, but the approximation should be clearly marked as heuristic when used for the design guidance in the same section.

Circularity Check

1 steps flagged · score 2.0 of 10

One self-definitional “GTVMin ensures similar devices learn similar parameters” claim; the optimization and convergence derivations are otherwise self-contained, so circularity is minor.

self definitional [Abstract and Section 3.4, Eq. (53)]
"This optimization problem is referred to as generalized total variation minimization (GTVMin) and ensures that devices with similar learning tasks learn similar model parameters."

Similarity between learning tasks is defined in Section 3.2 as the weighted edges A_{i,i'} of the FL network. GTVMin (Eq. 53) is then defined as minimizing sum_i L_i(w(i)) + alpha * sum_{edges} A_{i,i'} ||w(i)-w(i')||_2^2. The penalty term directly penalizes parameter difference across precisely those edges that encode similarity, so the statement that connected devices learn similar parameters is a restatement of the objective, not a derived or empirically verified consequence. The statistical results in Propositions 3.1 and 6.1 only analyze the case of identical true parameters (Eqs. 64 and 165); they do not establish the heterogeneous personalization claim.

full rationale

The central derivation chain is largely self-contained: GTVMin is defined as a regularized ERM objective; the convergence bounds (Propositions 5.1, 5.2), statistical bounds (Propositions 3.1, 6.1), and asynchronous fixed-point convergence results are proven from stated assumptions (strong convexity, Laplacian spectral bounds, pseudo-contraction). No fitted parameter is renamed as a prediction, and no load-bearing claim relies on a self-citation chain: the book proves its key inequalities in-line, and its algorithmic equivalence claims (FedAvg, FedProx, FedRelax as GTVMin instances) are direct by-construction connections. The only self-definitional element is the abstract's 'ensures that devices with similar learning tasks learn similar model parameters': since edge weights are the defined carrier of task similarity and the objective penalizes ||w(i)-w(i')||^2 along exactly those edges, this statement is a property of the objective by construction rather than an empirical finding. The skeptic's concern that the personalization guarantee is unproven for heterogeneous true parameters is a correctness gap, not a circularity: the book's bounds assume homogeneous true parameters within clusters (Eqs. 64 and 165) and do not quantify the bias of the squared-Euclidean penalty when true parameters differ. That gap is real but falls outside the circularity definition, which requires the derivation to reduce to its inputs. Accordingly, the circularity score is 2: one minor self-definitional claim, with the core mathematical content independent.

Assumptions & free parameters 2 free parameters · 6 assumptions · 1 invented entities

The framework rests on two user-chosen inputs: the FL network edge weights and the regularization strength α. The analytic bounds also require linear-Gaussian local data and connectedness. No invented physical entities are introduced; the FL network is a mathematical abstraction whose validity is assumed.

free parameters (2)

GTVMin regularization parameter α = not fixed in the book; chosen by validation or probabilistic analysis
Controls the trade-off between local loss and parameter variation across edges. Section 3.4 says 'In practice, the choice of α can be guided by validation or by a probabilistic analysis.'
FL network edge weights A_{i,i'} = no default; treated as a design choice or learned in Chapter 7
The behavior of GTVMin depends entirely on which devices are connected and with what weight. Section 1.2 states 'we largely adopt the view of edges, and their weights, as a given design choice.'

assumptions (6)

domain assumption Undirected weighted graph edges encode task similarity between devices.
Section 3.2 defines edges as similarity; asymmetric similarity is explicitly out of scope.
domain assumption Local datasets follow the linear model y(i)=X(i)w(i)+ε(i) with i.i.d. Gaussian features and noise (Eq. 63) for the statistical bounds.
Used in Proposition 3.1 and 6.1; if the data are not linear, the bounds do not apply.
domain assumption For Proposition 3.1, all true local parameters are identical, w(i)=c (Eq. 64).
The bound on deviation from the average is derived under this homogeneity assumption.
domain assumption The FL network graph is connected (λ2>0) for Propositions 3.1 and 5.2.
The proofs invoke the lower bound (48) which requires λ2>0.
domain assumption Local loss functions are strongly convex with coefficient σ for the asynchronous convergence result.
Section 5.9 derives the contraction factor κ = 1/(1+σ/(2α d(i))) under this assumption.
standard math Standard convex analysis and spectral graph theory results are used without proof (CFW min-max, contraction mappings, proximal operator properties).
Invoked in Sections 3.3, 4.8, and 5.11.

invented entities (1)

FL network (weighted undirected graph over devices)
purpose: Formalizes which devices are similar and can communicate; the substrate on which GTVMin is defined.
A modeling abstraction, not an empirical entity; no independent falsifiable handle is provided beyond the book's own framing.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Federated Learning: From Theory to Practice." pith.science (2026). https://pith.science/paper/EHRCHBKT

@misc{pith2026250519183,
  author       = {Pith},
  title        = {Pith review of: Federated Learning: From Theory to Practice},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EHRCHBKT}},
  note         = {Machine review of arXiv:2505.19183}
}

read the original abstract

This book offers a hands-on introduction to building and understanding federated learning (FL) systems. FL enables multiple devices -- such as smartphones, sensors, or local computers -- to collaboratively train machine learning (ML) models, while keeping their data private and local. It is a powerful solution when data cannot or should not be centralized due to privacy, regulatory, or technical reasons. The book is designed for students, engineers, and researchers who want to learn how to design scalable, privacy preserving FL systems. Our main focus is on personalization: enabling each device to train its own model while still benefiting from collaboration with relevant devices. This is achieved by leveraging similarities between (the learning tasks associated with) devices that are encoded by the weighted edges (or links) of a federated learning network (FL network). The key idea is to represent real-world FL systems as networks of devices, where nodes correspond to device and edges represent communication links and data similarities between them. The training of personalized models for these devices can be naturally framed as a distributed optimization problem. This optimization problem is referred to as generalized total variation minimization (GTVMin) and ensures that devices with similar learning tasks learn similar model parameters. Our approach is both mathematically principled and practically motivated. While we introduce some advanced ideas from optimization theory and graph-based learning, we aim to keep the book accessible. Readers are guided through the core ideas step by step, with intuitive explanations.

Figures

Figures reproduced from arXiv: 2505.19183 by the authors.

**Figure 1.1.** Left: A basic ML method uses a single dataset to train a single [PITH_FULL_IMAGE:figures/full_fig_p022_1_1.png] view at source ↗

**Figure 2.1.** ERM (1) for linear regression minimizes a convex quadratic function wTQw + wT q. To train a ML model H means to solve ERM (1) (or (3) for linear regression); the dataset D is therefore referred to as a training set. The 15 [PITH_FULL_IMAGE:figures/full_fig_p035_2_1.png] view at source ↗

**Figure 2.** depicts the objective function of [PITH_FULL_IMAGE:figures/full_fig_p039_2.png] view at source ↗

Figures from the paper (87 more)

**Figure 2.2.** Figure 2.2: The estimation error of linear regression is determined by the [PITH_FULL_IMAGE:figures/full_fig_p039_2_2.png]

**Figure 2.3.** Figure 2.3: An example of an unlucky split of a dataset into a training set and [PITH_FULL_IMAGE:figures/full_fig_p046_2_3.png]

**Figure 2.4.** Figure 2.4: Equivalence between data augmentation and loss penalization. [PITH_FULL_IMAGE:figures/full_fig_p048_2_4.png]

**Figure 2.5.** Figure 2.5: Two devices i and i ′ learn personalized model parameters. Each device executes a separate instance of regularized ERM with the regularizer depending on the model parameters of the other device. 30 [PITH_FULL_IMAGE:figures/full_fig_p050_2_5.png]

**Figure 3.1.** Figure 3.1: Example of an FL network whose nodes i ∈ V represent devices. Each device i generates a local dataset D(i) and trains a local model H(i) . Some devices i, i′ are connected by an undirected edge {i, i′} with a positive edge weight Ai,i′. to directed graphs is beyond t…

**Figure 3.2.** Figure 3.2: Left: An FL network G consisting of n = 4 nodes. Right: Equivalent fully connected FL network G ′ with the same nodes and non-zero edge weights A′ i,i′ = Ai,i′ for {i, i′} ∈ E and A′ i,i′ = 0 for {i, i′} ∈ E / . 35 [PITH_FULL_IMAGE:figures/full_fig_p055_3_2.png]

**Figure 3.** Figure 3: illustrates the Laplacian matrix of a small graph. [PITH_FULL_IMAGE:figures/full_fig_p059_3.png]

**Figure 3.3.** Figure 3.3: Left: Example of an FL network G with three nodes i = 1, 2, 3. Right: Laplacian matrix L (G) ∈ R 3×3 of G. The Laplacian matrix is symmetric and psd, which follows from the identity w T (L (G) ⊗ I)w = X {i,i′}∈E Ai,i′ [PITH_FULL_IMAGE:figures/full_fig_p060_3_3.png]

**Figure 3.4.** Figure 3.4: An FL network G that consists of n=6 nodes forming two connected components C (1) , C (2) . Here, avg{w(i)} := (1/n) Pn i=1 w(i) is the average of all local model parameters. The bound (48) follows from (41) and the CFW for the eigenvalues of the matrix L (G) ⊗ I. Th…

**Figure 3.5.** Figure 3.5: Two nodes of an FL network that are connected by an edge [PITH_FULL_IMAGE:figures/full_fig_p076_3_5.png]

**Figure 3.6.** Figure 3.6: An undirected graph G that consists of two connected components C (1) , C (2) . Can you find corresponding orthonormal eigenvectors u (1) , u (2)? Are they unique? 58 [PITH_FULL_IMAGE:figures/full_fig_p078_3_6.png]

**Figure 3.7.** Figure 3.7: An FL network whose nodes i = 1, . . . , 5 represent devices that hold local datasets whose size is indicated next to each node. 3.4. Adding an Edge Increases Connectivity. Consider an undirected weighted graph G with Laplacian matrix L (G) . We construct a new graph…

**Figure 3.8.** Figure 3.8: An FL network that consists of a centre node [PITH_FULL_IMAGE:figures/full_fig_p081_3_8.png]

**Figure 3.9.** Figure 3.9: An FL network containing a node i having degree d (i) = 3 like all its neighbors i ′ ∈ N (i) . We use an FL algorithm to learn local model parameters w (i) . If the algorithm employs message passing, the first iteration provides access only to the local datasets of t…

**Figure 4.1.** Figure 4.1: We can approximate a differentiable function [PITH_FULL_IMAGE:figures/full_fig_p087_4_1.png]

**Figure 4.2.** Figure 4.2: Effect of inadequate learning rates η in the gradient step (82). (a) If η is too large, the gradient steps might “overshoot” such that the iterates w(k) might diverge from the optimum, i.e., f(w(k+1)) > f(w(k) )! (b) If η is too small, the gradient steps make very li…

**Figure 4.3.** Figure 4.3: The contraction factor κ (η) (Q) (87), used in the upper bound (86), as a function of the learning rate η. Note that κ (η) (Q) also depends on the eigenvalues of the matrix Q in (81). illustrates the dependence of κ (η) (Q) on the learning rate η. Thus, choosing a sm…

**Figure 4.** Figure 4: ) [PITH_FULL_IMAGE:figures/full_fig_p093_4.png]

**Figure 4.4.** Figure 4.4: Projected GD augments a basic gradient step with a projection back [PITH_FULL_IMAGE:figures/full_fig_p098_4_4.png]

**Figure 4.** Figure 4: depicts examples of a firmly non-expansive, a non-expansive and [PITH_FULL_IMAGE:figures/full_fig_p104_4.png]

**Figure 4.5.** Figure 4.5: Example of a non-expansive operator F (1), a firmly non-expansive operator F (2) and a contractive operator F (3) . 4.9 Exercises 4.1. Learning Rate Schedule. Consider the gradient step method applied to a differentiable objective function f(w), w (k+1) = w (k) − ηk∇…

**Figure 5.1.** Figure 5.1: At the beginning of iteration k, node i = 1 collects the current local model parameters w(2,k) and w(3,k) from its neighbors. Then, it computes the gradient step (123) to obtain the new local model parameters w(1,k+1). These updated parameters are then used in the ne…

**Figure 5.2.** Figure 5.2: Algorithm 4 alternates between message passing across the edges [PITH_FULL_IMAGE:figures/full_fig_p116_5_2.png]

**Figure 5.3.** Figure 5.3: Star-shaped graph G (star) with a centre node i = 1 representing a server that trains a (global) model which is shared with peripheral nodes. These peripheral nodes represent clients generating local datasets. The training process at the server is facilitated by rece…

**Figure 5.4.** Figure 5.4: A key computational step in many FL algorithms is the evaluation [PITH_FULL_IMAGE:figures/full_fig_p133_5_4.png]

**Figure 5.5.** Figure 5.5: The execution of an asynchronous FL algorithm consists of a sequence [PITH_FULL_IMAGE:figures/full_fig_p136_5_5.png]

**Figure 5.6.** Figure 5.6: Illustration of the factor κ k/(2B+1) in the convergence bound (150) for a partially asynchronous FL algorithm (143) using a pseudo-contraction (see (146)). 121 [PITH_FULL_IMAGE:figures/full_fig_p141_5_6.png]

**Figure 5.7.** Figure 5.7: Illustration of step (c) in (153). 5.11.2 Proof of Proposition 5.2 Similar to the upper bound (153) we also start with the CFW for the eigenvalues of Q in (114). In particular, λ1 = min ∥w∥ 2 2=1 w TQw. (154) We next analyze the right-hand side of (154) by partition…

**Figure 6.1.** Figure 6.1: Indeed, we might learn local model parameters [PITH_FULL_IMAGE:figures/full_fig_p151_6_1.png]

**Figure 6.2.** Figure 6.2: The solution of GTVMin (53) are local model parameters that are approximately identical for all nodes in a tight-knit cluster C. Chapter 3 discussed how the eigenvalues of the Laplacian matrix can be used to measure the connectivity of G. Similarly, we can measure th…

**Figure 6.3.** Figure 6.3: As the regularization parameter α increases, the solutions of the GTVMin (53) become approximately constant over larger subsets of nodes, i.e., they exhibit stronger clustering. 6.4 Horizontal FL HFL uses local datasets D(i) , for i ∈ V, that contain data points char…

**Figure 6.4.** Figure 6.4: HFL uses the same features to characterize data points in different [PITH_FULL_IMAGE:figures/full_fig_p158_6_4.png]

**Figure 6.5.** Figure 6.5: HFL includes SSL as a special case. SSL involves a subset of nodes [PITH_FULL_IMAGE:figures/full_fig_p159_6_5.png]

**Figure 6.6.** Figure 6.6: VFL uses local datasets that are derived from the same data points. [PITH_FULL_IMAGE:figures/full_fig_p161_6_6.png]

**Figure 6.7.** Figure 6.7: Personalized FL with local models being ANNs with a single hidden [PITH_FULL_IMAGE:figures/full_fig_p163_6_7.png]

**Figure 7.** Figure 7: illustrates this trade-off between per-iteration complexity and the [PITH_FULL_IMAGE:figures/full_fig_p173_7.png]

**Figure 7.1.** Figure 7.1: Computational trade-off in GTVMin-based methods such as Al [PITH_FULL_IMAGE:figures/full_fig_p174_7_1.png]

**Figure 7.2.** Figure 7.2: A generic autoencoder consists of an encoder that maps the [PITH_FULL_IMAGE:figures/full_fig_p178_7_2.png]

**Figure 8.** Figure 8: illustrates the update [PITH_FULL_IMAGE:figures/full_fig_p192_8.png]

**Figure 8.1.** Figure 8.1: For a convex local loss function Li (·), the update (190) becomes the evaluation of the proximal operator proxLi(·),ρ(·) with ρ = 2αd(i) . We can measure the robustness of (190) by the slope of proxLi(·),ρ(·) (see (192)). GTV penalty ϕ(·) = ∥·∥2 . Let us now study (1…

**Figure 8.2.** Figure 8.2: Illustration of the (zero-subgradient) optimality condition [PITH_FULL_IMAGE:figures/full_fig_p195_8_2.png]

**Figure 8.3.** Figure 8.3: Python code for a ML method that trains a decision tree on the [PITH_FULL_IMAGE:figures/full_fig_p202_8_3.png]

**Figure 8.4.** Figure 8.4: We can improve the fairness of a ML method by augmenting the [PITH_FULL_IMAGE:figures/full_fig_p204_8_4.png]

**Figure 9.1.** Figure 9.1: A FL algorithm maps the local datasets D(i) to the learnt model parameters wb (i) , for i = 1, . . . , n. dataset D := D(i) n i=1 (constituted by the local datasets D(i) for i = 1, . . . , n) and delivers the learnt local model parameters A [PITH_FULL_IMAGE:figure…

**Figure 9.** Figure 9: depicts the decision regions of a decision tree. This decision [PITH_FULL_IMAGE:figures/full_fig_p209_9.png]

**Figure 9.2.** Figure 9.2: Scatterplot of a dataset used to train a decision tree. We indicate [PITH_FULL_IMAGE:figures/full_fig_p210_9_2.png]

**Figure 9.3.** Figure 9.3: Probability distributions of the learnt model parameters [PITH_FULL_IMAGE:figures/full_fig_p210_9_3.png]

**Figure 9.4.** Figure 9.4: The solutions of the privacy funnel (205) trace out (for varying constraint R) a curve in the plane spanned by the values of I (s; Φ(x)) (measuring the privacy leakage) and I (y; Φ(x)) (measuring the usefulness of the transformed features for predicting the label). A…

**Figure 9.** Figure 9: illustrates the solution of [PITH_FULL_IMAGE:figures/full_fig_p219_9.png]

**Figure 9.** Figure 9: illustrates a dataset for which we want to find a linear feature map [PITH_FULL_IMAGE:figures/full_fig_p220_9.png]

**Figure 9.5.** Figure 9.5: A toy dataset D whose data points represent customers, each characterized by features x = [PITH_FULL_IMAGE:figures/full_fig_p221_9_5.png]

**Figure 10.1.** Figure 10.1: In many ML applications, such as monitoring livestock in remote [PITH_FULL_IMAGE:figures/full_fig_p226_10_1.png]

**Figure 10.2.** Figure 10.2: A GTVMin-based FL system from the perspective of a specific [PITH_FULL_IMAGE:figures/full_fig_p228_10_2.png]

**Figure 10.** Figure 10: illustrates the effect of a data poisoning attack on FL system [PITH_FULL_IMAGE:figures/full_fig_p230_10.png]

**Figure 10.3.** Figure 10.3: Propagation of the effect of a data poisoning attack that perturbs [PITH_FULL_IMAGE:figures/full_fig_p230_10_3.png]

**Figure 10.4.** Figure 10.4: A local dataset D(i) along with three hypothesis maps learnt via iterating (211) under three attack scenarios. 213 [PITH_FULL_IMAGE:figures/full_fig_p233_10_4.png]

**Figure 10.5.** Figure 10.5: An attack on (211) perturbs (adversarially) a fraction η of the received model parameters w(i ′ ,k) . against denial-of-service attacks and backdoor attacks. We now discuss how to protect (212) against privacy attacks. For a fixed time k, we can ensure a prescribed …

**Figure 10.6.** Figure 10.6: If the features and the label of a data point are drawn from a [PITH_FULL_IMAGE:figures/full_fig_p245_10_6.png]

**Figure 10.7.** Figure 10.7: 227 [PITH_FULL_IMAGE:figures/full_fig_p247_10_7.png]

**Figure 10.8.** Figure 10.8: Left: A convex set C ⊆ R d . Right: A convex function f : R d → R. See also: Euclidean space. Courant–Fischer–Weyl min-max characterization Consider a psd matrix Q ∈ R d×d with EVD (or spectral decomposition), Q = X d j=1 λju (j) [PITH_FULL_IMAGE:figures/full_fig_…

**Figure 10.9.** Figure 10.9: Data augmentation exploits intrinsic symmetries of data points [PITH_FULL_IMAGE:figures/full_fig_p253_10_9.png]

**Figure 10.10.** Figure 10.10: “Cows in the Swiss Alps” by User:Huhu Uet is licensed under [CC [PITH_FULL_IMAGE:figures/full_fig_p255_10_10.png]

**Figure 10.11.** Figure 10.11: Left: A decision tree is a flow-chart-like representation of a [PITH_FULL_IMAGE:figures/full_fig_p258_10_11.png]

**Figure 10.12.** Figure 10.12: Example of dimensionality reduction: High-dimensional image [PITH_FULL_IMAGE:figures/full_fig_p261_10_12.png]

**Figure 10.13.** Figure 10.13: Epigraph of the function f(x) = x 2 (i.e., shaded area). See also: graph, convex. Erdős-Rényi (ER) graph An Erdős-Rényi (ER) graph is a probabilistic model for graphs defined over a given node set i = 1, . . . , n. One way to 243 [PITH_FULL_IMAGE:figures/full_fig_…

**Figure 10.14.** Figure 10.14: The expectation of a discrete RV x is obtained by summing up its possible values xi , weighted by the corresponding probability p(xi) = p(x = xi). See also: feature vector, realization, RV, probability distribution, probability. expert ML aims to learn a hypothesi…

**Figure 10.15.** Figure 10.15: Illustration of a single iteration of FedAvg which consists of [PITH_FULL_IMAGE:figures/full_fig_p269_10_15.png]

**Figure 10.16.** Figure 10.16: We can interpret the temperature distribution over Finland as a [PITH_FULL_IMAGE:figures/full_fig_p272_10_16.png]

**Figure 10.17.** Figure 10.17: Two data points z (1) , z (2) that are used as a training set to learn a hypothesis hˆ via ERM. We can evaluate hˆ outside D(train) either by an i.i.d. assumption with some underlying probability distribution p(z) or by perturbing the data points. See also: model, …

**Figure 10.18.** Figure 10.18: A single gradient step (218) towards the minimizer [PITH_FULL_IMAGE:figures/full_fig_p278_10_18.png]

**Figure 10.19.** Figure 10.19: The basic gradient step (219) maps a given vector w to the updated vector w′ . It defines an operator T (f,η) (·) : R d → R d : w 7→ wb. Note that the gradient step (219) optimizes locally - in a neighborhood whose size is determined by the step size η - a linear a…

**Figure 10.20.** Figure 10.20: Five data points characterized by feature vectors [PITH_FULL_IMAGE:figures/full_fig_p285_10_20.png]

**Figure 10.21.** Figure 10.21: Left: Some undirected graph G with three nodes i = 1, 2, 3. Right: The Laplacian matrix L (G) ∈ R 3×3 of G. See also: graph, edge weight. large language model (LLM) LLMs is an umbrella term for ML methods that process and generate human-like text. These methods typ…

**Figure 10.22.** Figure 10.22: Some loss function L ((x, y), h) for a fixed data point, with feature vector x and label y, and a varying hypothesis h. ML methods try to find (or learn) a hypothesis that incurs minimal loss. See also: loss, data point, feature, label, hypothesis, prediction, feat…

**Figure 10.23.** Figure 10.23: An objective function maps each possible value [PITH_FULL_IMAGE:figures/full_fig_p298_10_23.png]

**Figure 10.24.** Figure 10.24: The parameter space W of an ML model H consists of all feasible choices for the model parameters. Each choice w for the model parameters selects a hypothesis map h (w) ∈ H. See also: ML, model, model parameters, Euclidean space, linear model, deep net, norm, hypoth…

**Figure 10.25.** Figure 10.25: Projected GD augments a basic gradient step with a projection [PITH_FULL_IMAGE:figures/full_fig_p305_10_25.png]

**Figure 10.26.** Figure 10.26: A generalized gradient step updates a vector [PITH_FULL_IMAGE:figures/full_fig_p306_10_26.png]

**Figure 10.** Figure 10: illustrates the above three routes to regularization. These [PITH_FULL_IMAGE:figures/full_fig_p309_10.png]

**Figure 10.27.** Figure 10.27: Three approaches to regularization: 1) data augmentation; 2) loss [PITH_FULL_IMAGE:figures/full_fig_p309_10_27.png]

**Figure 10.28.** Figure 10.28: A scatterplot with circle markers, where the data points represent [PITH_FULL_IMAGE:figures/full_fig_p314_10_28.png]

**Figure 10.29.** Figure 10.29: Consider an objective function f(w) that is β-smooth. Taking a gradient step, with step size η = 1/β, decreases the objective by at least 1 2β [PITH_FULL_IMAGE:figures/full_fig_p316_10_29.png]

**Figure 10.30.** Figure 10.30: Three datasets D(∗) , D(□) , and D(△) , each sampled independently from the same data-generating probability distribution. A stable ML method should return similar outputs when trained on any of these datasets. See also: ML, dataset, training set, model parameters,…

**Figure 10.31.** Figure 10.31: SGD for ERM approximates the gradient Pm r=1 ∇wL [PITH_FULL_IMAGE:figures/full_fig_p319_10_31.png]

**Figure 10.32.** Figure 10.32: VFL uses local datasets that are derived from the data points of [PITH_FULL_IMAGE:figures/full_fig_p326_10_32.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

229 extracted references · 52 canonical work pages

[1]

Rudin,Real and Complex Analysis, 3rd ed

W. Rudin,Real and Complex Analysis, 3rd ed. New York: McGraw-Hill, 1987

1987
[2]

Rudin,Principles of Mathematical Analysis, 3rd ed

W. Rudin,Principles of Mathematical Analysis, 3rd ed. New York: McGraw-Hill, 1976

1976
[3]

G. H. Golub and C. F. Van Loan,Matrix Computations, 4th ed. Balti- more, MD: Johns Hopkins University Press, 2013

2013
[4]

An analysis of the total least squares problem,

G. Golub and C. van Loan, “An analysis of the total least squares problem,”SIAM J. Numerical Analysis, vol. 17, no. 6, pp. 883–893, Dec. 1980

1980
[5]

The future of industrial communication: Automation networks in the era of the internet of things and industry 4.0,

M. Wollschlaeger, T. Sauter, and J. Jasperneite, “The future of industrial communication: Automation networks in the era of the internet of things and industry 4.0,”IEEE Industrial Electronics Magazine, vol. 11, no. 1, pp. 17–27, 2017

2017
[6]

The emergence of edge computing,

M. Satyanarayanan, “The emergence of edge computing,”Computer, vol. 50, no. 1, pp. 30–39, Jan. 2017. [Online]. Available: https: //doi.org/10.1109/MC.2017.9

doi:10.1109/mc.2017.9 2017
[7]

Wearable devices for the detection of covid-19,

H. Ates, A. Yetisen, F. Güder, and C. Dincer, “Wearable devices for the detection of covid-19,”Nature Electronics, vol. 4, no. 1, pp. 13–14, 2021. [Online]. Available: https://doi.org/10.1038/s41928-020-00533-1

work page doi:10.1038/s41928-020-00533-1 2021
[8]

The industrial internet of things (iiot): An analysis framework,

H. Boyes, B. Hallaq, J. Cunningham, and T. Watson, “The industrial internet of things (iiot): An analysis framework,” Computers in Industry, vol. 101, pp. 1–12, 2018. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0166361517307285

2018

Show all 229 references

[9]

S. Cui, A. Hero, Z.-Q. Luo, and J. Moura, Eds.,Big Data over Networks. Cambridge University Press, 2016

2016
[10]

Network medicine: a network- based approach to human disease,

A. Barabási, N. Gulbahce, and J. Loscalzo, “Network medicine: a network- based approach to human disease,”Nature Reviews Genetics, vol. 12, no. 56, 2011

2011
[11]

M. E. J. Newman,Networks: An Introduction. Oxford University Press, 2010

2010
[12]

Communication-Efficient Learning of Deep Networks from Decentralized Data,

B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” inProceedings of the 20th International Conference on Artificial Intelligence and Statistics, ser. Proceedings of Machine Learning Rese...

2017
[13]

Federated learning: Challenges, methods, and future directions,

T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning: Challenges, methods, and future directions,”IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 50–60, May 2020

2020
[14]

Federated learning for privacy- preserving ai,

Y. Cheng, Y. Liu, T. Chen, and Q. Yang, “Federated learning for privacy- preserving ai,”Communications of the ACM, vol. 63, no. 12, pp. 33–36, Dec. 2020

2020
[15]

cpSGD: Communication-efficient and differentially-private distributed sgd,

N. Agarwal, A. Suresh, F. Yu, S. Kumar, and H. McMahan, “cpSGD: Communication-efficient and differentially-private distributed sgd,” in Proc. Neural Inf. Proc. Syst. (NIPS), 2018

2018
[16]

Federated Multi-Task Learning,

V. Smith, C.-K. Chiang, M. Sanjabi, and A. Talwalkar, “Federated Multi-Task Learning,” inAdvances in Neural Information Processing Systems, vol. 30, 2017. [Online]. Available: https://proceedings.neurips. cc/paper/2017/file/6211080fa89981f66b1a0c9d55c61d0f-Paper.pdf

2017
[17]

Ship compute or ship data? why not both?

J. You, J. Wu, X. Jin, and M. Chowdhury, “Ship compute or ship data? why not both?” in18th USENIX Symposium on Networked Systems Design and Implementation (NSDI 21). USENIX Association, Apr. 2021, pp. 633–651. [Online]. Available: https://www.usenix.org/conference/nsdi21/prese...

2021
[18]

D. P. Bertsekas and J. Tsitsiklis,Parallel and Distributed Computation: Numerical Methods. Athena Scientific, 2015

2015
[19]

van Steen and A

M. van Steen and A. Tanenbaum,Distributed Systems, 3rd ed., Feb. 2017, self-published, open publication

2017
[20]

Tse and P

D. Tse and P. Viswanath,Fundamentals of Wireless Communication. Cambridge University Press, 2005

2005
[21]

Applied federated learning: Improving google keyboard query suggestions,

T. Yang, G. Andrew, H. Eichner, H. Sun, W. Li, N. Kong, D. Ramage, and F. Beaufays, “Applied federated learning: Improving google keyboard query suggestions,” 2018. [Online]. Available: https://arxiv.org/abs/1812.02903

2018 arXiv
[22]

An efficient frame- work for clustered federated learning,

A. Ghosh, J. Chung, D. Yin, and K. Ramchandran, “An efficient frame- work for clustered federated learning,” in34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada, 2020

2020
[23]

Clustered federated learning: Model-agnostic distributed multitask optimization under privacy con- straints,

F. Sattler, K. Müller, and W. Samek, “Clustered federated learning: Model-agnostic distributed multitask optimization under privacy con- straints,”IEEE Transactions on Neural Networks and Learning Systems, 2020

2020
[24]

Strang,Computational Science and Engineering

G. Strang,Computational Science and Engineering. Wellesley- Cambridge Press, MA, 2007

2007
[25]

Strang,Introduction to Linear Algebra, 5th ed

G. Strang,Introduction to Linear Algebra, 5th ed. Wellesley-Cambridge Press, MA, 2016

2016
[26]

H. H. Bauschke and P. L. Combettes,Convex Analysis and Monotone Operator Theory in Hilbert Spaces. New York: Springer, 2011

2011
[27]

Jung,Machine Learning: The Basics, 1st ed

A. Jung,Machine Learning: The Basics, 1st ed. Springer Singapore, Feb. 2022

2022
[28]

Shalev-Shwartz and S

S. Shalev-Shwartz and S. Ben-David,Understanding Machine Learning – from Theory to Algorithms. Cambridge University Press, 2014

2014
[29]

Can you program ethics into a self-driving car?

N. Goodall, “Can you program ethics into a self-driving car?”IEEE Spectrum, vol. 53, no. 6, pp. 28–58, June 2016

2016
[30]

Induction of decision trees,

J. R. Quinlan, “Induction of decision trees,”Machine Learning, vol. 1, no. 1, pp. 81–106
[31]

Schölkopf and A

B. Schölkopf and A. Smola,Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. Cambridge, MA, USA: MIT Press, Dec. 2002

2002
[32]

First-order methods for nonsmooth convex large-scale optimization, I: General purpose methods,

A. Juditsky and A. Nemirovski, “First-order methods for nonsmooth convex large-scale optimization, I: General purpose methods,” inOpti- mization for Machine Learning, S. Sra, S. Nowozin, and S. Wright, Eds. MIT press, 2011, pp. 121–147

2011
[33]

Billingsley,Probability and Measure, 3rd ed

P. Billingsley,Probability and Measure, 3rd ed. New York: Wiley, 1995

1995
[34]

An RKHS Approach to Estimation with Sparsity Constraints,

A. Jung, “An RKHS Approach to Estimation with Sparsity Constraints,” Ph.D. dissertation, Vienna University of Technology, 2011, available online: arXiv:1311.5768

2011 arXiv
[35]

E. L. Lehmann and G. Casella,Theory of Point Estimation, 2nd ed. New York: Springer, 1998

1998
[36]

Dermatologist-level classification of skin cancer with deep neural networks,

A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, and S. Thrun, “Dermatologist-level classification of skin cancer with deep neural networks,”Nature, vol. 542, 2017

2017
[37]

Lütkepohl,New Introduction to Multiple Time Series Analysis

H. Lütkepohl,New Introduction to Multiple Time Series Analysis. New York: Springer, 2005

2005
[38]

Wainwright,High-Dimensional Statistics: A Non-Asymptotic View- point

M. Wainwright,High-Dimensional Statistics: A Non-Asymptotic View- point. Cambridge: Cambridge University Press, 2019

2019
[39]

Clustered fed- erated learning via generalized total variation minimization,

Y. SarcheshmehPour, Y. Tian, L. Zhang, and A. Jung, “Clustered fed- erated learning via generalized total variation minimization,”IEEE Transactions on Signal Processing, vol. 71, pp. 4240–4256, 2023

2023
[40]

Spectral graph theory,

F. Chung, “Spectral graph theory,” inRegional Conference Series in Mathematics. American Mathematical Society, 1997, no. 92

1997
[41]

Spectral and algebraic graph theory,

D. Spielman, “Spectral and algebraic graph theory,” 2019

2019
[42]

Spectral graph theory,

D. Spielman, “Spectral graph theory,” inCombinatorial Scientific Com- puting, U. Naumann and O. Schenk, Eds. Chapman and Hall/CRC, 2012

2012
[43]

Hastie, R

T. Hastie, R. Tibshirani, and J. Friedman,The Elements of Statistical Learning, ser.SpringerSeriesinStatistics. NewYork, NY,USA:Springer, 2001

2001
[44]

Beck,First-Order Methods in Optimization

A. Beck,First-Order Methods in Optimization. Philadelphia, PA, USA: SIAM-Society for Industrial and Applied Mathematics, 2017

2017
[45]

Proximal algorithms,

N. Parikh and S. Boyd, “Proximal algorithms,”Foundations and Trends in Optimization, vol. 1, no. 3, pp. 123–231, 2013

2013
[46]

A primal–dual splitting method for convex optimization involving lipschitzian, proximable and linear composite terms,

L. Condat, “A primal–dual splitting method for convex optimization involving lipschitzian, proximable and linear composite terms,”Journal of Opt. Th. and App., vol. 158, no. 2, pp. 460–479, Aug. 2013

2013
[47]

An efficient parallel solver for SDD linear systems,

R. Peng and D. A. Spielman, “An efficient parallel solver for SDD linear systems,” inProc. ACM Symposium on Theory of Computing, New York, NY, 2014, pp. 333–342

2014
[48]

Lx = b — Laplacian solvers and their algorithmic applications,

N. K. Vishnoi, “ Lx = b — Laplacian solvers and their algorithmic applications,”Foundations and Trends in Theoretical Computer Science, vol. 8, no. 1–2, pp. 1–141, 2012. [Online]. Available: http://dx.doi.org/10.1561/0400000054

2012 doi
[49]

Convex clustering: Model, theoretical guarantee and efficient algorithm,

D. Sun, K.-C. Toh, and Y. Yuan, “Convex clustering: Model, theoretical guarantee and efficient algorithm,”Journal of Machine Learning Research, vol. 22, no. 9, pp. 1–32, 2021. [Online]. Available: http://jmlr.org/papers/v22/18-694.html

2021
[50]

Convex clus- tering shrinkage,

K. Pelckmans, J. D. Brabanter, J. Suykens, and B. D. Moor, “Convex clus- tering shrinkage,” inPASCAL Workshop on Statistics and Optimization of Clustering Workshop, 2005

2005
[51]

R. T. Rockafellar,Network Flows and Monotropic Optimization. Athena Scientific, Jul. 1998

1998
[52]

Boyd and L

S. Boyd and L. Vandenberghe,Convex Optimization. Cambridge, UK: Cambridge University Press, 2004

2004
[53]

D. P. Bertsekas,Network Optimization: Continuous and Discrete Models. Athena Scientific, 1998

1998
[54]

Locally weighted learning,

C. G. Atkeson, A. W. Moore, and S. Schaal, “Locally weighted learning,” Artif. Intell. Rev., vol. 11, no. 1–5, pp. 11–73, Feb. 1997. [Online]. Available: https://doi.org/10.1023/A:1006559212014

1997 doi
[55]

Neuraltangentkernel: Convergence and generalization in neural networks,

A.Jacot, F.Gabriel, andC.Hongler, “Neuraltangentkernel: Convergence and generalization in neural networks,” inAdvances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., vol. 31. Curran Associates,...

2018
[56]

Gradient descent provably optimizes over-parameterized neural networks,

S. S. Du, X. Zhai, B. Póczos, and A. Singh, “Gradient descent provably optimizes over-parameterized neural networks,” in7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. [Online]. Available: https://...

2019
[57]

A comparative analysis of optimization and generalization properties of two-layer neural network and random feature models under gradient descent dynamics,

W. E, C. Ma, and L. Wu, “A comparative analysis of optimization and generalization properties of two-layer neural network and random feature models under gradient descent dynamics,”Science China Mathematics, vol. 63, no. 7, pp. 1235–1258, 2020. [Online]. Available: https://doi...

2020 doi
[58]

Paszke, S

A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Köpf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala,PyTorch: An Imperative Style, High- Per...

2019
[59]

D. P. Bertsekas,Convex Optimization Algorithms. Athena Scientific, 2015

2015
[60]

No more pesky learning rates,

T. Schaul, X. Zhang, and Y. LeCun, “No more pesky learning rates,” in Proc. of the 30th International Conference on Machine Learning, PMLR 28(3), vol. 28, Atlanta, Georgia, June 2013, pp. 343–351

2013
[61]

Learning to learn by gradi- ent descent by gradient descent,

M. Andrychowicz, M. Denil, S. G. Colmenarejo, M. W. Hoffman, D. Pfau, T. Schaul, B. Shillingford, and N. de Freitas, “Learning to learn by gradi- ent descent by gradient descent,” inProceedings of the 30th International Conference on Neural Information Processing Systems, ser....

2016
[62]

Nesterov,Introductory lectures on convex optimization, ser

Y. Nesterov,Introductory lectures on convex optimization, ser. Applied Optimization. Kluwer Academic Publishers, Boston, MA, 2004, vol. 87, a basic course. [Online]. Available: http://dx.doi.org/10.1007/ 978-1-4419-8853-9

2004
[63]

Bauschke and P

H. Bauschke and P. Combettes,Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd ed. New York: Springer, 2017

2017
[64]

Istrăt,escu,Fixed point theory: An Introduction, ser

V. Istrăt,escu,Fixed point theory: An Introduction, ser. Mathematics and its applications ; 7. Dordrecht: Reidel, 1981

1981
[65]

Exponential graph is provably efficient for decentralized deep training,

B. Ying, K. Yuan, Y. Chen, H. Hu, P. PAN, and W. Yin, “Exponential graph is provably efficient for decentralized deep training,” inAdvances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, Eds., vol. 34. Curran Asso...

2021
[66]

Fastest mixing markov chain on a graph,

S. Boyd, P. Diaconis, and L. Xiao, “Fastest mixing markov chain on a graph,”SIAM Review, vol. 46, no. 4, pp. 667–689, 2004

2004
[67]

Internet time synchronization: the network time protocol,

D. Mills, “Internet time synchronization: the network time protocol,” IEEE Transactions on Communications, vol. 39, no. 10, pp. 1482–1493, 1991

1991
[68]

Hirvonen and J

J. Hirvonen and J. Suomela. (2023) Distributed algorithms 2020

2023
[69]

Diestel,Graph Theory

R. Diestel,Graph Theory. Springer Berlin Heidelberg, 2005

2005
[70]

Federated optimization in heterogeneous networks,

T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, “Federated optimization in heterogeneous networks,” in Proceedings of the Third Conference on Machine Learning and Systems, MLSys 2020, Austin, TX, USA, March 2-4, 2020, I. S. Dhillon, D. S. Papailiopoulos, ...

2020
[71]

Tanenbaum and D

A. Tanenbaum and D. Wetherall,Computer Networks, 5th ed. USA: Prentice Hall Press, 2010

2010
[72]

Convergence of a block coordinate descent method for nondifferentiable minimization,

P. Tseng, “Convergence of a block coordinate descent method for nondifferentiable minimization,”Journal of Optimization Theory and Applications, vol. 109, no. 3, pp. 475–494, 2001. [Online]. Available: https://doi.org/10.1023/A:1017501703105

2001 doi
[73]

S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein,Distributed Op- timization and Statistical Learning via the Alternating Direction Method of Multipliers. Hanover, MA: Now Publishers, 2010, vol. 3, no. 1

2010
[74]

Distributed learning systems with first-order methods,

J. Liu and C. Zhang, “Distributed learning systems with first-order methods,”Foundations and Trends in Databases, vol. 9, no. 1, p. 100
[75]

Towards efficient scheduling of federated mobile devices under computational and statistical heterogeneity,

C. Wang, Y. Yang, and P. Zhou, “Towards efficient scheduling of federated mobile devices under computational and statistical heterogeneity,”IEEE Transactions on Parallel and Distributed Systems, vol. 32, no. 2, pp. 394–410, 2021

2021
[76]

Asynchronous iterations in opti- mization: new sequence results and sharper algorithmic guarantees,

H. Feyzmahdavian and M. Johansson, “Asynchronous iterations in opti- mization: new sequence results and sharper algorithmic guarantees,”J. Mach. Learn. Res., vol. 24, no. 1, Jan. 2023

2023
[77]

A primer on monotone operator methods,

E. K. Ryu and S. Boyd, “A primer on monotone operator methods,” Applied and Computational Mathematics, vol. 15, no. 1, pp. 3–43, 2016, survey

2016
[78]

Q. Yang, Y. Liu, Y. Cheng, Y. Kang, T. Chen, and H. Yu,Federated Learning, 1st ed. Springer, 2022

2022
[79]

Attack robustness and centrality of complex networks

S. Iyer, T. Killingback, B. Sundaram, and Z. Wang, “Attack robustness and centrality of complex networks.”PLoS One, vol. 8, no. 4, p. e59613, 2013

2013
[80]

An omnibus test for normality for small samples,

D. J. Spiegelhalter, “An omnibus test for normality for small samples,” Biometrika, vol. 67, no. 2, pp. 493–496, 2024/03/25/ 1980. [Online]. Available: http://www.jstor.org/stable/2335498

2024
[81]

Q. Yang, Y. Liu, Y. Cheng, Y. Kang, T. Chen, and H. Yu,Horizontal Federated Learning. Cham: Springer International Publishing, 2020, pp. 49–67. [Online]. Available: https://doi.org/10.1007/978-3-031-01585-4_4

2020 doi
[82]

Chapelle, B

O. Chapelle, B. Schölkopf, and A. Zien, Eds.,Semi-Supervised Learning. Cambridge, Massachusetts: The MIT Press, 2006

2006
[83]

Q. Yang, Y. Liu, Y. Cheng, Y. Kang, T. Chen, and H. Yu,Vertical Federated Learning. Cham: Springer International Publishing, 2020, pp. 69–81. [Online]. Available: https://doi.org/10.1007/978-3-031-01585-4_5

2020 doi
[84]

Ludwig and N

H. Ludwig and N. Baracaldo, Eds.,Federated Learning: A Comprehensive Overview of Methods and Applications. Springer, 2022

2022
[85]

Personalized federated learning using hypernetworks,

A. Shamsian, A. Navon, E. Fetaya, and G. Chechik, “Personalized federated learning using hypernetworks,” inProceedings of the 38th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, M. Meila and T. Zhang, Eds., vol
[86]

Learning to compare: Relation network for few-shot learning,

F. Sung, Y. Yang, L. Zhang, T. Xiang, P. H. Torr, and T. M. Hospedales, “Learning to compare: Relation network for few-shot learning,” in2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 1199–1208

2018
[87]

Few-shot learning with graph neural networks

V. Satorras and J. Bruna, “Few-shot learning with graph neural networks.” inICLR (Poster). OpenReview.net, 2018. [Online]. Available: http://dblp.uni-trier.de/db/conf/iclr/iclr2018.html#SatorrasE18

2018
[88]

Network medicine: a network-based approach to human disease,

A.-L. Barabási, N. Gulbahce, and J. Loscalzo, “Network medicine: a network-based approach to human disease,”Nature Reviews Genetics, vol. 12, no. 1, pp. 56–68, 2011. [Online]. Available: https://doi.org/10.1038/nrg2918

2011 doi
[89]

Localized linear regression in networked data,

A. Jung and N. Tran, “Localized linear regression in networked data,” IEEE Sig. Proc. Lett., vol. 26, no. 7, Jul. 2019

2019
[90]

Network lasso: Clustering and optimization in large graphs,

D. Hallac, J. Leskovec, and S. Boyd, “Network lasso: Clustering and optimization in large graphs,” inProc. SIGKDD, 2015, pp. 387–396

2015
[91]

Graphical LASSO Based Model Selection for Time Series,

A. Jung, G. Hannak, and N. Görtz, “Graphical LASSO Based Model Selection for Time Series,”IEEE Sig. Proc. Letters, vol. 22, no. 10, Oct. 2015

2015
[92]

Learning the conditional independence structure of stationary time series: A multitask learning approach,

A. Jung, “Learning the conditional independence structure of stationary time series: A multitask learning approach,”IEEE Trans. Signal Process., vol. 63, no. 21, Nov. 2015

2015
[93]

How to learn a graph from smooth signals,

V. Kalofolias, “How to learn a graph from smooth signals,” inProceed- ings of the 19th International Conference on Artificial Intelligence and Statistics, ser. Proceedings of Machine Learning Research, A. Gretton and C. C. Robert, Eds., vol. 51. Cadiz, Spain: PMLR, 09–11 May 2...

2016
[94]

Learning graphs from data: A signal representation perspective,

X. Dong, D. Thanou, M. Rabbat, and P. Frossard, “Learning graphs from data: A signal representation perspective,”IEEE Signal Processing Magazine, vol. 36, no. 3, pp. 44–63, 2019

2019
[95]

Nešetřil and P

J. Nešetřil and P. O. de Mendez,Sparsity: Graphs, Structures, and Algorithms, ser. Algorithms and Combinatorics. Springer, 2012, vol. 28

2012
[96]

An introduction to matrix concentration inequalities,

J. Tropp, “An introduction to matrix concentration inequalities,”Found. Trends Mach. Learn., May 2015

2015
[97]

Clustering in partially labeled stochastic block models via total variation minimization,

A. Jung, “Clustering in partially labeled stochastic block models via total variation minimization,” inProc. 54th Asilomar Conf. Signals, Systems, Computers, Pacific Grove, CA, Nov. 2020

2020
[98]

Bollobas, W

B. Bollobas, W. Fulton, A. Katok, F. Kirwan, and P. Sarnak,Random graphs. Cambridge studies in advanced mathematics., 2001, vol. 73

2001
[99]

Keiser,Optical Fiber Communication, 4th ed

G. Keiser,Optical Fiber Communication, 4th ed. New Delhi: Mc-Graw Hill, 2011

2011
[100]

Tse and P

D. Tse and P. Viswanath,Fundamentals of wireless communication. USA: Cambridge University Press, 2005

2005
[101]

Algebraic connectivity of graphs,

M. Fiedler, “Algebraic connectivity of graphs,”Czechoslovak Mathemat- ical Journal,, vol. 23, no. 2, pp. 298–305, 1973

1973
[102]

Expander graphs and their applications,

S. Hoory, N. Linial, and A. Wigderson, “Expander graphs and their applications,”Bull. Amer. Math. Soc., vol. 43, no. 04, pp. 439–562, Aug. 2006

2006
[103]

Expander graph and communication-efficient decentralized optimization,

Y.-T. Chow, W. Shi, T. Wu, and W. Yin, “Expander graph and communication-efficient decentralized optimization,” in2016 50th Asilo- mar Conference on Signals, Systems and Computers, 2016, pp. 1715–1720

2016
[104]

S. M. Kay,Fundamentals of Statistical Signal Processing: Estimation Theory. Englewood Cliffs, NJ: Prentice Hall, 1993

1993
[105]

Learning sparse graphs under smoothness prior,

S. Chepuri, S. Liu, G. Leus, and A. Hero, “Learning sparse graphs under smoothness prior,” inProc. of the IEEE Int. Conf. on Acoustics, Speech and Signal Processing, 2017, pp. 6508–6512

2017
[106]

pFedSim: Similarity- Aware Model Aggregation Towards Personalized Federated Learning,

J. Tan, Y. Zhou, G. Liu, J. H. Wang, and S. Yu, “pFedSim: Similarity- Aware Model Aggregation Towards Personalized Federated Learning,” arXiv e-prints, p. arXiv:2305.15706, May 2023

2023 arXiv
[107]

Goodfellow, Y

I. Goodfellow, Y. Bengio, and A. Courville,Deep Learning. MIT Press, 2016

2016
[108]

Ethics guidelines for trustworthy AI,

H.-L. E. G. on Artificial Intelligence, “Ethics guidelines for trustworthy AI,” European Commission, Tech. Rep., April 2019

2019
[109]

Aus- tralia’s AI Ethics Principles,

Department of Industry, Science, Energy and Resources, “Aus- tralia’s AI Ethics Principles,” Government of Australia, 2024, accessed: 2024-09-30. [Online]. Available: https://www.industry. gov.au/publications/australias-artificial-intelligence-ethics-framework/ australias-ai-e...

2024
[110]

Oecd ai principles: Recommendation of the council on artificial intelligence,

OECD, “Oecd ai principles: Recommendation of the council on artificial intelligence,” https://oecd.ai/en/ai-principles, 2019, accessed: 2024-09- 30

2019
[111]

Interim measures for the man- agement of generative artificial intelligence services,

Cyberspace Administration of China, “Interim measures for the man- agement of generative artificial intelligence services,” https://www. chinalawtranslate.com/en/generative-ai/, 2023, accessed: 2025-05-02

2023
[112]

Artificial intelligence security gover- nance framework,

China Academy of Information and Communications Tech- nology (CAICT), “Artificial intelligence security gover- nance framework,” https://www.haynesboone.com/-/media/ project/haynesboone/haynesboone/pdfs/alert-pdfs/2024/ china-alert---china-publishes-the-ai-security-governance-...

2024
[113]

New generation artifi- cial intelligence ethics code,

Ministry of Science and Technology of China, “New generation artifi- cial intelligence ethics code,” https://www.chinalawvision.com/2025/01/ digital-economy-ai/ai-ethics-overview-china/, 2021, accessed: 2025-05- 02

2025
[114]

Artificial intelligence risk management framework (ai rmf 1.0),

National Institute of Standards and Technology, “Artificial intelligence risk management framework (ai rmf 1.0),” https://nvlpubs.nist.gov/ nistpubs/ai/nist.ai.100-1.pdf, 2023, accessed: 2025-05-02

2023
[115]

Blueprint for an ai bill of rights,

White House Office of Science and Technology Policy, “Blueprint for an ai bill of rights,” https://bidenwhitehouse.archives.gov/ostp/ ai-bill-of-rights/, 2022, accessed: 2025-05-02

2022
[116]

Executive order 14110: Safe, secure, and trustworthy development and use of artificial intelligence,

The White House, “Executive order 14110: Safe, secure, and trustworthy development and use of artificial intelligence,” https://www.federalregister.gov/documents/2023/11/01/2023-24283/ safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence, 2023, accessed: ...

2023
[117]

Internet addiction and problematic internet use: A systematic review of clinical research

D. Kuss and O. Lopez-Fernandez, “Internet addiction and problematic internet use: A systematic review of clinical research.”World J Psychiatry, vol. 6, no. 1, pp. 143–176, Mar 2016

2016
[118]

Angry by design: toxic communication and technical architectures,

L. Munn, “Angry by design: toxic communication and technical architectures,”Humanities and Social Sciences Communications, vol. 7, no. 1, p. 53, 2020. [Online]. Available: https://doi.org/10.1057/ s41599-020-00550-7

2020
[119]

A genocide incited on facebook, with posts from myanmar’s military,

P. Mozur, “A genocide incited on facebook, with posts from myanmar’s military,”The New York Times, 2018

2018
[120]

The persuasive effects of political microtargeting in the age of generative artificial intelligence

A. Simchon, M. Edwards, and S. Lewandowsky, “The persuasive effects of political microtargeting in the age of generative artificial intelligence.” PNAS Nexus, vol. 3, no. 2, p. pgae035, Feb 2024

2024
[121]

J. R. Taylor,An Introduction to Error Analysis: The study of uncer- tainties in physical measurements, second edition. ed. Sausalito, Calif: University Science Books, 1997

1997
[122]

A fixed-point of view on gradient methods for big data,

A. Jung, “A fixed-point of view on gradient methods for big data,” Frontiers in Applied Mathematics and Statistics, vol. 3, 2017. [Online]. Available: https://www.frontiersin.org/article/10.3389/fams.2017.00018

2017
[123]

Robust aggregation for federated learning,

K. Pillutla, S. M. Kakade, and Z. Harchaoui, “Robust aggregation for federated learning,”IEEE Transactions on Signal Processing, vol. 70, pp. 1142–1154, 2022

2022
[124]

Breakdown points of affine equivariant estimators of multivariate location and covariance matrices,

H. P. Lopuhaä and P. J. Rousseeuw, “Breakdown points of affine equivariant estimators of multivariate location and covariance matrices,” The Annals of Statistics, vol. 19, no. 1, pp. 229–248, 1991. [Online]. Available: http://www.jstor.org/stable/2241852

1991
[125]

A survey on sensor networks,

I. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci, “A survey on sensor networks,”IEEE Communications Magazine, vol. 40, no. 8, pp. 102–114, 2002

2002
[126]

Small Cuts and Connectivity Certificates: A Fault Tolerant Approach,

M. Parter, “Small Cuts and Connectivity Certificates: A Fault Tolerant Approach,” in33rd International Symposium on Distributed Computing (DISC 2019), ser. Leibniz International Proceedings in Informatics (LIPIcs), J. Suomela, Ed., vol. 146. Dagstuhl, Germany: Schloss Dagstuhl...

2019
[127]

Fault-tolerant spanners for general graphs,

S. Chechik, M. Langberg, D. Peleg, and L. Roditty, “Fault-tolerant spanners for general graphs,” inProceedings of the Forty-First Annual ACM Symposium on Theory of Computing, ser. STOC ’09. New York, NY, USA: Association for Computing Machinery, 2009, p. 435–444. [Online]. Ava...

2009
[128]

Guidelines for evaluating differential privacy guarantees,

J. Near and D. Darais, “Guidelines for evaluating differential privacy guarantees,” National Institute of Standards and Technology, Gaithers- burg, MD, Tech. Rep., 2023

2023
[129]

Data protection in the age of big data,

S. Wachter, “Data protection in the age of big data,”Nature Electronics, vol. 2, no. 1, pp. 6–7, 2019. [Online]. Available: https://doi.org/10.1038/s41928-018-0193-y

2019 doi
[130]

Protecting respondents identities in microdata release,

P. Samarati, “Protecting respondents identities in microdata release,” IEEE Transactions on Knowledge and Data Engineering, vol. 13, no. 6, pp. 1010–1027, 2001

2001
[131]

E. Comission, “Regulation (eu) 2016/679 of the european parliament and of the council of 27 april 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46/ec (general data prot...

2016
[132]

U. N. G. Assembly,The Universal Declaration of Human Rights (UDHR), New York, 1948

1948
[133]

What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods

J. Colin, T. Fel, R. Cadène, and T. Serre, “What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods.”Advances in Neural Information Processing Systems, vol. 35, pp. 2832–2845, 2022

2022
[134]

An information-theoretic approach to person- alized explainable machine learning,

A. Jung and P. Nardelli, “An information-theoretic approach to person- alized explainable machine learning,”IEEE Sig. Proc. Lett., vol. 27, pp. 825–829, 2020

2020
[135]

Explainable empirical risk minimization,

L. Zhang, G. Karakasidis, A. Odnoblyudova, L. Dogruel, Y. Tian, and A. Jung, “Explainable empirical risk minimization,”Neural Computing and Applications, vol. 36, no. 8, pp. 3983–3996, 2024. [Online]. Available: https://doi.org/10.1007/s00521-023-09269-3

2024 doi
[136]

Fairness in credit scoring: Assessment, implementation and profit implications,

N. Kozodoi, J. Jacob, and S. Lessmann, “Fairness in credit scoring: Assessment, implementation and profit implications,”European Journal of Operational Research, vol. 297, no. 3, pp. 1083–1094, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S0377...

2022
[137]

Gonçalves-Sá and F

J. Gonçalves-Sá and F. Pinheiro,Societal Implications of Rec- ommendation Systems: A Technical Perspective. Cham: Springer International Publishing, 2024, pp. 47–63. [Online]. Available: https://doi.org/10.1007/978-3-031-41264-6_3

2024 doi
[138]

Power optimization in 5g networks: A step towards green communication,

A. Abrol and R. Jha, “Power optimization in 5g networks: A step towards green communication,”IEEE Access, vol. 4, pp. 1355–1374, 2016

2016
[139]

9489–9502

PMLR, 18–24 Jul 2021, pp. 9489–9502. [Online]. Available: https://proceedings.mlr.press/v139/shamsian21a.html

2021
[140]

Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data,

M. J. Sheller, B. Edwards, G. A. Reina, J. Martin, S. Pati, A. Kotrotsou, M. Milchenko, W. Xu, D. Marcus, R. R. Colen, and S. Bakas, “Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data,”Scientific Reports, vol. 10, no. ...
[141]

R. B. Ash,Probability and Measure Theory, 2nd ed. New York: Academic Press, 2000

2000
[142]

Personalized health monitoring using predictive analytics,

P. Amin, N. R. Anikireddypally, S. Khurana, S. Vadakkemadathil, and W. Wu, “Personalized health monitoring using predictive analytics,” in2019 IEEE Fifth International Conference on Big Data Computing Service and Applications (BigDataService), 2019, pp. 271–278

2019
[143]

The algorithmic foundations of differential privacy,

C. Dwork and A. Roth, “The algorithmic foundations of differential privacy,”Foundations and Trends®in Theoretical Computer Science, vol. 9, no. 3–4, pp. 211–407, 2014. [Online]. Available: http://dx.doi.org/10.1561/0400000042

2014 doi
[144]

P. R. Halmos,Measure Theory. New York: Springer, 1974

1974
[145]

Understanding aggregate trends for apple intelligence using differential privacy,

Apple Machine Learning Research, “Understanding aggregate trends for apple intelligence using differential privacy,” https://machinelearning. apple.com/research/differential-privacy-aggregate-trends, April 2025, ac- cessed: 2025-05-20

2025
[146]

Rappor: Randomized aggregatable privacy-preserving ordinal response,

U. Erlingsson, V. Pihur, and A. Korolova, “Rappor: Randomized aggregatable privacy-preserving ordinal response,” inProceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’14. New York, NY, USA: Association for Computing Machinery, 2014,...

2014
[147]

Guidelines for evaluating differential privacy guarantees,

J. P. Near, D. Darais, N. Lefkovitz, and G. S. Howarth, “Guidelines for evaluating differential privacy guarantees,” National Institute of Standards and Technology, Gaithersburg, MD, NIST Special Publication NIST SP 800-226, 2025. [Online]. Available: https://doi.org/10.6028/N...

2025 doi
[148]

The u.s. census bureau adopts differential privacy,

J. M. Abowd, “The u.s. census bureau adopts differential privacy,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ser. KDD ’18. New York, NY, USA: Association for Computing Machinery, 2018, p. 2867. [Online]. Available: htt...

2018
[149]

Rényi differential privacy,

I. Mironov, “Rényi differential privacy,” in2017 IEEE 30th Computer Security Foundations Symposium (CSF), 2017, pp. 263–275

2017
[150]

A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergences,

S. Asoodeh, J. Liao, F. P. Calmon, O. Kosut, and L. Sankar, “A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergences,”arXiv e-prints, p. arXiv:2001.05990, Jan. 2020

2001 arXiv
[151]

The composition theorem for differential privacy,

P. Kairouz, S. Oh, and P. Viswanath, “The composition theorem for differential privacy,” inProceedings of the 32nd International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, F. Bach and D. Blei, Eds., vol. 37. Lille, France: PMLR, 07–09 Jul 20...

2015
[152]

S. M. Kay,Fundamentals of statistical signal processing. Vol. 2., Detec- tion theory, ser. Prentice-Hall signal processing series. Upper Saddle River, NJ: Prentice-Hall PTR, 1998

1998
[154]

The optimal noise-adding mechanism in differential privacy,

Q. Geng and P. Viswanath, “The optimal noise-adding mechanism in differential privacy,”IEEE Transactions on Information Theory, vol. 62, no. 2, pp. 925–951, 2016

2016
[155]

Sensitivity analysis of deep neural networks,

H. Shu and H. Zhu, “Sensitivity analysis of deep neural networks,” inProceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, ser. AAAI’19/IAAI’19/EAAI’19. AAAI Press, 2019. [Online]. Available: https://doi.org/10.1609/aaai.v33i01.33014943

2019 doi
[156]

How private are DP-SGD implementations?

L. Chua, B. Ghazi, P. Kamath, R. Kumar, P. Manurangsi, A. Sinha, and C. Zhang, “How private are DP-SGD implementations?” in Proceedings of the 41st International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, R. Salakhutdinov, Z. Kolter, K. Hell...

2024
[157]

Privacy amplification by sub- sampling: tight analyses via couplings and divergences,

B. Balle, G. Barthe, and M. Gaboardi, “Privacy amplification by sub- sampling: tight analyses via couplings and divergences,” inProceedings of the 32nd International Conference on Neural Information Processing Systems, ser. NIPS’18. Red Hook, NY, USA: Curran Associates Inc., 2...

2018
[158]

Label differential privacy and private training data release,

R. Busa-Fekete, A. Munoz-Medina, U. Syed, and S. Vassilvitskii, “Label differential privacy and private training data release,” in Proceedings of the 40th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol
[159]

From the information bottleneck to the privacy funnel,

A. Makhdoumi, S. Salamatian, N. Fawaz, and M. Médard, “From the information bottleneck to the privacy funnel,” in2014 IEEE Information Theory Workshop (ITW 2014), 2014, pp. 501–505

2014
[160]

Smashed: Sniffing and manipulating android sensor data for offensive purposes,

M. Mohamed, B. Shrestha, and N. Saxena, “Smashed: Sniffing and manipulating android sensor data for offensive purposes,”IEEE Transac- tions on Information Forensics and Security, vol. 12, no. 4, pp. 901–913, 2017

2017
[161]

Differential privacy as a mutual information constraint,

P. Cuff and L. Yu, “Differential privacy as a mutual information constraint,” inProceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’16. New York, NY, USA: Association for Computing Machinery, 2016, pp. 43–54. [Online]. Available: ht...

2016
[162]

Adversarial machine learning: A taxonomy and terminology of attacks and mitigations,

A. Vassilev, A. Oprea, A. Fordyce, and H. Anderson, “Adversarial machine learning: A taxonomy and terminology of attacks and mitigations,” National Institute of Standards and Technology, Gaithersburg, MD, NIST Artificial Intelligence (AI) Report NIST AI 100-2e2023, 2024. [Onli...

2024 doi
[163]

Machine learning with adversaries: Byzantine tolerant gradient descent,

P. Blanchard, E. M. El Mhamdi, R. Guerraoui, and J. Stainer, “Machine learning with adversaries: Byzantine tolerant gradient descent,” inAdvances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnet...

2017
[164]

Clean-label backdoor attacks,

A. Turner, D. Tsipras, and A. Madry, “Clean-label backdoor attacks,” 2019. [Online]. Available: https://openreview.net/forum?id= HJg6e2CcK7

2019
[165]

Model inversion attacks that exploit confidence information and basic countermeasures,

M. Fredrikson, S. Jha, and T. Ristenpart, “Model inversion attacks that exploit confidence information and basic countermeasures,” in Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’15. New York, NY, USA: Association for Computi...

2015
[166]

Robust multivariate mean estimation: The optimality of trimmed mean,

G. Lugosi and S. Mendelson, “Robust multivariate mean estimation: The optimality of trimmed mean,”Annals of Statistics, vol. 49, no. 1, pp. 393–410, Feb. 2021, publisher Copyright:©Institute of Mathematical Statistics, 2021

2021
[167]

How to backdoor federated learning,

E. Bagdasaryan, A. Veit, Y. Hua, D. Estrin, and V. Shmatikov, “How to backdoor federated learning,” inProceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, ser. Proceedings of Machine Learning Research, S. Chiappa and R. Calandra, ...

2020
[168]

The Asymptotic Distribution of the Trimmed Mean,

S. M. Stigler, “The Asymptotic Distribution of the Trimmed Mean,” The Annals of Statistics, vol. 1, no. 3, pp. 472 – 477, 1973. [Online]. Available: https://doi.org/10.1214/aos/1176342412

1973
[169]

Auror: defending against poisoning attacks in collaborative deep learning systems,

S. Shen, S. Tople, and P. Saxena, “Auror: defending against poisoning attacks in collaborative deep learning systems,” inProceedings of the 32nd Annual Conference on Computer Security Applications, ser. ACSAC ’16. New York, NY, USA: Association for Computing Machinery, 2016, p...

2016
[170]

Fltrust: Byzantine-robust federated learning via trust bootstrapping,

X. Cao, M. Fang, J. Liu, and N. Gong, “Fltrust: Byzantine-robust federated learning via trust bootstrapping,” inNetwork and Distributed Systems Security (NDSS) Symposium 2021, 01 2021

2021
[171]

Byzantine-robust decentralized federated learning,

M. Fang, Z. Zhang, Hairi, P. Khanduri, J. Liu, S. Lu, Y. Liu, and N. Gong, “Byzantine-robust decentralized federated learning,” in Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’24. New York, NY, USA: Association for Computi...

2024
[172]

Byzantine-robust distributed learning: Towards optimal statistical rates,

D. Yin, Y. Chen, R. Kannan, and P. Bartlett, “Byzantine-robust distributed learning: Towards optimal statistical rates,” inProceedings of the 35th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 80. PMLR, 10–15 Jul 2018, pp. 5650–

2018
[173]

Flare: Defending federated learning against model poisoning attacks via latent space representations,

N. Wang, Y. Xiao, Y. Chen, Y. Hu, W. Lou, and Y. T. Hou, “Flare: Defending federated learning against model poisoning attacks via latent space representations,” inProceedings of the 2022 ACM on Asia Conference on Computer and Communications Security, ser. ASIA CCS ’22. New Yor...

2022
[174]

Boosting and differential privacy,

C. Dwork, G. N. Rothblum, and S. Vadhan, “Boosting and differential privacy,” in2010 IEEE 51st Annual Symposium on Foundations of Computer Science, 2010, pp. 51–60

2010
[175]

T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein,Introduction to Algorithms. Cambridge, MA, USA: MIT Press, 2022. [Online]. Available: http://ebookcentral.proquest.com/lib/aalto-ebooks/detail. action?docID=6925615

2022
[176]

Sipser,Introduction to the Theory of Computation, 3rd ed

M. Sipser,Introduction to the Theory of Computation, 3rd ed. Andover, U.K.: Cengage Learning, 2013

2013
[177]

Differentially private empirical risk minimization,

K. Chaudhuri, C. Monteleoni, and A. Sarwate, “Differentially private empirical risk minimization,”J. Mach. Learn. Res., vol. 12, pp. 1069–1109, Mar. 2011

2011
[178]

Motwani and P

R. Motwani and P. Raghavan,Randomized Algorithms. Cambridge, U.K.: Cambridge Univ. Press, 1995

1995
[179]

R. G. Gallager,Stochastic Processes: Theory for Applications. Cam- bridge University Press, 2013

2013
[180]

A systematic review and meta-analysis of artificial intelligence versus clinicians for skin cancer diagnosis,

M. P. Salinas et al., “A systematic review and meta-analysis of artificial intelligence versus clinicians for skin cancer diagnosis,”npj Digit. Med., vol. 7, no. 1, May 2024, Art. no. 125, doi: 10.1038/s41746-024-01103-x

2024 doi
[181]

Bertsekas and J

D. Bertsekas and J. Tsitsiklis,Introduction to Probability, 2nd ed. Athena Scientific, 2008

2008
[182]

Gray,Probability, Random Processes, and Ergodic Properties, 2nd ed

R. Gray,Probability, Random Processes, and Ergodic Properties, 2nd ed. New York: Springer, 2009

2009
[183]

A relational model of data for large shared data banks,

E. F. Codd, “A relational model of data for large shared data banks,”Commun. ACM, vol. 13, no. 6, pp. 377–387, Jun. 1970, doi: 10.1145/362384.362685

1970
[184]

21, 2018

European Union, “Regulation (EU) 2018/1725 of the European Parliament and of the Council of 23 October 2018 on the protection of natural persons with regard to the processing of personal data by the Union institutions, bodies, offices and agencies and on the free movement of s...

2018
[185]

The computational complexity of probabilistic inference usingbayesianbeliefnetworks,

G. F. Cooper, “The computational complexity of probabilistic inference usingbayesianbeliefnetworks,”Artif. Intell., vol.42, no.2–3, pp.393–405, Mar. 1990, doi: 10.1016/0004-3702(90)90060-D

1990 doi
[186]

Privacy-enhanced federated learning against poisoning adversaries,

X. Liu, H. Li, G. Xu, Z. Chen, X. Huang, and R. Lu, “Privacy-enhanced federated learning against poisoning adversaries,”IEEE Transactions on Information Forensics and Security, vol. 16, pp. 4574–4588, 2021

2021
[187]

Poisongan: Generative poisoning attacks against federated learning in edge com- puting systems,

J. Zhang, B. Chen, X. Cheng, H. T. T. Binh, and S. Yu, “Poisongan: Generative poisoning attacks against federated learning in edge com- puting systems,”IEEE Internet of Things Journal, vol. 8, no. 5, pp. 3310–3322, 2021

2021
[188]

Silberschatz, H

A. Silberschatz, H. F. Korth, and S. Sudarshan,Database System Concepts, 7th ed. New York, NY, USA: McGraw-Hill Education, 2019. [Online]. Available: https://db-book.com/

2019
[189]

T. M. Cover and J. A. Thomas,Elements of Information Theory, 2nd ed. New Jersey: Wiley, 2006

2006
[190]

Hoberman,Data Modeling Made Simple: A Practical Guide for Business and IT Professionals, 2nd ed

S. Hoberman,Data Modeling Made Simple: A Practical Guide for Business and IT Professionals, 2nd ed. Basking Ridge, NJ, USA: Technics Publications, 2009

2009
[191]

Ramakrishnan and J

R. Ramakrishnan and J. Gehrke,Database Management Systems, 3rd ed. New York, NY, USA: McGraw-Hill, 2002

2002
[192]

Datasheets for datasets,

T. Gebru, J. Morgenstern, B. Vecchione, J. Vaughan, H. Wallach, H. Daumé, and K. Crawford, “Datasheets for datasets,”Commun. ACM, vol. 64, no. 12, pp. 86–92, nov 2021. [Online]. Available: https://doi.org/10.1145/3458723

2021 doi
[193]

Abiteboul, R

S. Abiteboul, R. Hull, and V. Vianu,Foundations of Databases. Read- ing, MA, USA: Addison-Wesley Publishing Company, 1995

1995
[194]

Cesa-Bianchi and G

N. Cesa-Bianchi and G. Lugosi,Prediction, Learning, and Games. New York, NY, USA: Cambridge University Press, 2006

2006
[195]

Hazan,Introduction to Online Convex Optimization

E. Hazan,Introduction to Online Convex Optimization. Now Publishers Inc., 2016

2016
[196]

Learning to explain: An information-theoretic perspective on model interpretation,

J. Chen, L. Song, M. Wainwright, and M. Jordan, “Learning to explain: An information-theoretic perspective on model interpretation,” inProc. 35th Int. Conf. on Mach. Learning, Stockholm, Sweden, 2018

2018
[197]

Bertsekas, A

D. Bertsekas, A. Nedic, and A. Ozdaglar,Convex Analysis and Opti- mization. Athena Scientific, 2003

2003
[198]

Dodge,The Oxford Dictionary of Statistical Terms

Y. Dodge,The Oxford Dictionary of Statistical Terms. Oxford Univer- sity Press, 2003

2003
[199]

Everitt,Cambridge Dictionary of Statistics

B. Everitt,Cambridge Dictionary of Statistics. Cambridge University Press, 2002

2002
[200]

“Why should i trust you?

M. Ribeiro, S. Singh, and C. Guestrin, ““Why should i trust you?”: Explaining the predictions of any classifier,” inProc. 22nd ACM SIGKDD, Aug. 2016, pp. 1135–1144

2016
[201]

Gujarati and D

D. Gujarati and D. Porter,Basic Econometrics. Mc-Graw Hill, 2009

2009
[202]

3233–3251

PMLR, 23–29 Jul 2023, pp. 3233–3251. [Online]. Available: https://proceedings.mlr.press/v202/busa-fekete23a.html

2023
[203]

C. E. Rasmussen and C. K. I. Williams,Gaussian Processes for Machine Learning. Cambridge, MA, USA: MIT Press, 2006

2006
[204]

Ross,A First Course in Probability, 9th ed

S. Ross,A First Course in Probability, 9th ed. Boston, MA, USA: Pearson Education, 2014

2014
[205]

Papoulis and S

A. Papoulis and S. U. Pillai,Probability, Random Variables, and Stochas- tic Processes, 4th ed. New York: Mc-Graw Hill, 2002

2002
[206]

Lapidoth,A Foundation in Digital Communication

A. Lapidoth,A Foundation in Digital Communication. New York: Cambridge University Press, 2009

2009
[207]

Kernel methods in computer vision,

C. Lampert, “Kernel methods in computer vision,”Foundations and Trends in Computer Graphics and Vision, 2009

2009
[208]

A tutorial on spectral clustering,

U. von Luxburg, “A tutorial on spectral clustering,”Statistics and Computing, vol. 17, no. 4, pp. 395–416, Dec. 2007

2007
[209]

One pixel attack for fooling deep neural networks,

J. Su, D. V. Vargas, and K. Sakurai, “One pixel attack for fooling deep neural networks,”IEEE Trans. Evol. Comput., vol. 23, no. 5, pp. 828–841, Oct. 2019, doi: 10.1109/TEVC.2019.2890858

2019
[210]

R. A. Horn and C. R. Johnson,Topics in Matrix Analysis. Cambridge, UK: Cambridge University Press, 1991

1991
[211]

Multitask learning,

R. Caruana, “Multitask learning,”Machine Learning, vol. 28, no. 1, pp. 41–75, 1997. [Online]. Available: https://doi.org/10.1023/A: 1007379606734

1997 doi
[212]

Stop explaining black box machine learning models for high- stakes decisions and use interpretable models instead,

C. Rudin, “Stop explaining black box machine learning models for high- stakes decisions and use interpretable models instead,”Nature Machine Intelligence, vol. 1, no. 5, pp. 206–215, 2019

2019
[213]

On spectral clustering: Analysis and an algorithm,

A. Y. Ng, M. I. Jordan, and Y. Weiss, “On spectral clustering: Analysis and an algorithm,” inAdv. Neur. Inf. Proc. Syst., 2001

2001
[214]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” inAdvances in neural information processing systems, vol. 30, 2017, pp. 5998–6008

2017
[215]

D. P. Bertsekas,Nonlinear Programming, 2nd ed. Belmont, MA: Athena Scientific, June 1999

1999
[216]

Information-theoretic approaches to differential privacy,

A. Ünsal and M. Önen, “Information-theoretic approaches to differential privacy,”ACM Comput. Surv., vol. 56, no. 3, Oct. 2023, Art. no. 76, doi: 10.1145/3604904

2023 doi
[217]

Hastie, R

T. Hastie, R. Tibshirani, and M. Wainwright,Statistical Learning with Sparsity. The Lasso and its Generalizations, 2015

2015
[218]

R. A. Horn and C. R. Johnson,Matrix Analysis, 2nd ed. Cambridge, UK: Cambridge University Press, 2013

2013
[219]

Generalized cutoff rates and Renyi’s information measures,

I. Csiszar, “Generalized cutoff rates and Renyi’s information measures,” IEEE Transactions on Information Theory, vol. 41, no. 1, pp. 26–34, 1995

1995
[220]

Convex optimization. algorithms and complexity

S. Bubeck, “Convex optimization. algorithms and complexity.” inFoun- dations and Trends in Machine Learning. Now Publishers, 2015, vol. 8

2015
[221]

Kallenberg,Foundations of modern probability

O. Kallenberg,Foundations of modern probability. New York: Springer, 1997

1997
[222]

Stochastic methods for l1 regularized loss minimization,

S. Shalev-Shwartz and A. Tewari, “Stochastic methods for l1 regularized loss minimization,” inProceedings of the 26th Annual International Conference on Machine Learning, ser. ICML ’09, New York, NY, USA, 2009, pp. 929–936

2009
[223]

Model cards for model reporting,

M. M. et.al., “Model cards for model reporting,” inProceedings of the Conference on Fairness, Accountability, and Transparency, ser. FAT* ’19. New York, NY, USA: Association for Computing Machinery, 2019, pp. 220–229. [Online]. Available: https://doi.org/10.1145/3287560.3287596

2019
[224]

IEEE standard review — Ethically aligned design: A vision for prioritizing human wellbeing with artificial intelligence and autonomous systems,

K. Shahriari and M. Shahriari, “IEEE standard review — Ethically aligned design: A vision for prioritizing human wellbeing with artificial intelligence and autonomous systems,” in2017 IEEE Canada Interna- tional Humanitarian Technology Conference, pp. 197–201, doi: 10.1109/I- ...
[225]

On-line learning and stochastic approximations,

L. Bottou, “On-line learning and stochastic approximations,” inOn- Line Learning in Neural Networks, D. Saad, Ed. New York, NY, USA: Cambridge Univ. Press, 1999, ch. 2, pp. 9–42

1999
[226]

The AI act proposal: A new right to technical interpretability?,

C. Gallese, “The AI act proposal: A new right to technical interpretability?,”SSRN Electron. J., Feb. 2023. [Online]. Available: https://ssrn.com/abstract=4398206

2023
[229]

Engineering trustworthy AI: A developer guide for empirical risk minimization,

D. Pfau and A. Jung, “Engineering trustworthy AI: A developer guide for empirical risk minimization,” Nov. 2024. [Online]. Available: https://arxiv.org/abs/2410.19361

2024 arXiv
[230]

Commission, C

E. Commission, C. Directorate-General for Communications Networks, and Technology,The Assessment List for Trustworthy Artificial Intelli- gence (ALTAI) for self assessment. Publications Office, 2020

2020
[2020]

Available: https://doi.org/10.1038/s41598-020-69250-1

[Online]. Available: https://doi.org/10.1038/s41598-020-69250-1

doi
[5659]

Available: https://proceedings.mlr.press/v80/yin18a.html

[Online]. Available: https://proceedings.mlr.press/v80/yin18a.html

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Rudin,Real and Complex Analysis, 3rd ed

W. Rudin,Real and Complex Analysis, 3rd ed. New York: McGraw-Hill, 1987

1987

[2] [2]

Rudin,Principles of Mathematical Analysis, 3rd ed

W. Rudin,Principles of Mathematical Analysis, 3rd ed. New York: McGraw-Hill, 1976

1976

[3] [3]

G. H. Golub and C. F. Van Loan,Matrix Computations, 4th ed. Balti- more, MD: Johns Hopkins University Press, 2013

2013

[4] [4]

An analysis of the total least squares problem,

G. Golub and C. van Loan, “An analysis of the total least squares problem,”SIAM J. Numerical Analysis, vol. 17, no. 6, pp. 883–893, Dec. 1980

1980

[5] [5]

The future of industrial communication: Automation networks in the era of the internet of things and industry 4.0,

M. Wollschlaeger, T. Sauter, and J. Jasperneite, “The future of industrial communication: Automation networks in the era of the internet of things and industry 4.0,”IEEE Industrial Electronics Magazine, vol. 11, no. 1, pp. 17–27, 2017

2017

[6] [6]

The emergence of edge computing,

M. Satyanarayanan, “The emergence of edge computing,”Computer, vol. 50, no. 1, pp. 30–39, Jan. 2017. [Online]. Available: https: //doi.org/10.1109/MC.2017.9

doi:10.1109/mc.2017.9 2017

[7] [7]

Wearable devices for the detection of covid-19,

H. Ates, A. Yetisen, F. Güder, and C. Dincer, “Wearable devices for the detection of covid-19,”Nature Electronics, vol. 4, no. 1, pp. 13–14, 2021. [Online]. Available: https://doi.org/10.1038/s41928-020-00533-1

work page doi:10.1038/s41928-020-00533-1 2021

[8] [8]

The industrial internet of things (iiot): An analysis framework,

H. Boyes, B. Hallaq, J. Cunningham, and T. Watson, “The industrial internet of things (iiot): An analysis framework,” Computers in Industry, vol. 101, pp. 1–12, 2018. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0166361517307285

2018