Pith. sign in

REVIEW 3 major objections 3 minor 1 cited by

This paper introduces distributed dynamic associative memory and proves that a tree-based online gradient descent algorithm achieves sublinear static regret and path-length-dependent dynamic regret, with communication delays appearing expli

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 19:32 UTC pith:A6E3KZS5

load-bearing objection Solid, mostly well-proved extension of OCO to distributed associative memory, but the main regret bound is not well-formed as stated (comparator path-length extends past T) and Corollary 2 drops a positive constant; both fixable, so worth peer review but not acceptance as is. the 3 major comments →

arxiv 2511.23347 v2 pith:A6E3KZS5 submitted 2025-11-28 cs.LG eess.SP

Distributed Dynamic Associative Memory via Online Convex Optimization

Bowen Wang , Matteo Zecchin , Osvaldo Simeone This is my paper
classification cs.LG eess.SP
keywords associative memorydistributed optimizationonline convex optimizationregret analysistree-based communicationdelayed gradientsdynamic regretmulti-agent learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces distributed dynamic associative memory (DDAM): a network of agents, each maintaining a local associative memory that must store both its own key–value associations and selected other agents' associations from streaming data. It proposes DDAM-TOGD, a tree-based distributed online gradient descent method, and proves that in stationary environments each agent's memory approaches the best fixed memory in hindsight, with sublinear static regret. In non-stationary environments, dynamic regret grows only with the path-length of the drifting optimal memories, scaled by communication delays. Delays enter through the sum of round-trip delays on routing trees and the spread between the longest and shortest delays. The paper also gives a combinatorial tree-design method that minimizes sum-delay and validates the approach on synthetic data and real wireless traffic data.

Core claim

The central claim is that a fully distributed, dynamic associative memory can be built by having each agent run online gradient descent on its own weighted retrieval cost while obtaining gradient feedback from other agents over routing trees. Under standard convexity and bounded-gradient assumptions, DDAM-TOGD achieves static regret O(Σ_n sqrt((1+τ_{n,sum})(T+Δτ_n))) and dynamic regret O(Σ_n (1+PL_T^{T+τ_{n,max}}) sqrt((1+τ_{n,sum})(T+Δτ_n))). Thus, in stationary settings the memory converges to the best fixed memory in hindsight, and in drifting environments regret degrades only proportionally to the total movement of the optimal memories plus the communication-delay overhead.

What carries the argument

The algorithm DDAM-TOGD: each agent n broadcasts its current memory X_{n,t} down a spanning tree to all agents m whose data it needs; those agents evaluate the gradient of their local retrieval loss at X_{n,t} and send it back; agent n then updates by online gradient descent using the delayed gradients. The proof's core is a regret decomposition that isolates four terms—standard OGD regret, drift caused by stale iterates, drift caused by delayed comparators, and a tail of gradients arriving after the horizon—each bounded using convexity, bounded gradients, and bounded domain diameter. The key identity is that all delay effects collapse into the quantities τ_{n,sum} and Δτ_n, so regret scales

Load-bearing premise

The protocol and all regret bounds assume a synchronous, reliable network in which every message arrives after an exactly known, fixed number of time steps; if delays are random, time-varying, or suffer loss, the update rule no longer matches the analysis and the bounds do not apply.

What would settle it

Run DDAM-TOGD on a small network with fixed delays but choose a comparator sequence that stays constant up to time T and then moves sharply afterward (so the paper's path-length PL_T^{T+τ_max} is positive while PL_T is zero); the bound predicts a regret penalty from this future movement—if the observed regret does not show that penalty, the bound's delay-path-length coupling is wrong. Alternatively, introduce random round-trip delays and check whether sublinear static regret still holds; a violation would confirm that synchronous fixed delays are load-bearing.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • In stationary environments, each agent's local memory asymptotically attains the performance of the best fixed memory chosen with hindsight, so distributed recall costs vanish relative to T.
  • In non-stationary environments, the algorithm tracks slowly drifting optimal memories, with dynamic regret proportional to the path-length of the drift; rapid changes necessarily increase regret.
  • Communication delays slow learning by a factor sqrt(1+τ_{n,sum}) and an additive Δτ_n term; when the horizon T dominates delay, the same sublinear-in-T behavior as centralized OGD is preserved.
  • Routing-tree design can be optimized by minimizing sum-delay, turning network topology into a tunable knob on regret.
  • DDAM-TOGD outperforms consensus-based distributed OGD when agents need personalized memories, since consensus forces shared parameters while DDAM allows per-agent specialization.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same delayed-gradient regret analysis could be applied to distributed attention-based sequence models, giving formal learning guarantees for federated or device-edge Transformers that exchange memory banks rather than gradients.
  • The tree-design problem is a variant of Steiner arborescence with flow-conservation constraints; one could extend it to dynamic graphs or to minimize max-delay rather than sum-delay, with the regret bound suggesting max-delay matters through Δτ_n.
  • The fixed-delay assumption could be relaxed to random delays; the decomposition suggests that expected regret would then depend on the first and second moments of round-trip delay, which is a testable extension.
  • Since the analysis is parameter-specific (learning rate depends on B, G_n, and τ), a parameter-free variant using coin-betting or FTRL could remove the need to know these constants, as the paper itself notes as future work.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. This paper introduces distributed dynamic associative memory (DDAM) as a multi-agent online convex optimization problem: each agent maintains a local associative-memory parameter and receives delayed gradient feedback from other agents over routing trees, according to a logical interest matrix. The proposed DDAM-TOGD algorithm is a projected online gradient descent rule with delayed gradients. The main theoretical result (Theorem 3, Eq. (16)) bounds the network dynamic regret by per-agent terms involving the time horizon, communication-delay statistics, and the path-length of the comparator sequence; Corollary 2 specializes this to static and dynamic regret with a tuned learning rate, giving rates of order O(Σ_n (1+PL_n) sqrt((1+τ_{n,sum})(T+Δτ_n))). The proof decomposes the regret into Reg*, DriftX, DriftU, and Tail and bounds each term in the appendices. The paper also proposes an ILP-based routing-tree design to minimize cumulative delay and reports experiments on synthetic data and a real wireless-traffic dataset.

Significance. If the formal issues below are corrected, this is a useful and largely self-contained extension of online convex optimization to a distributed associative-memory setting. The regret bounds have the standard sublinear-in-T form and explicitly quantify how communication delay and tree topology affect learning. The proof is detailed, and the tree-design optimization is a practical contribution. The experiments against consensus-based and centralized baselines support the qualitative claims that consensus is inappropriate for heterogeneous memories and that the proposed method tracks non-stationary comparators. The main concern is not the numerical validity of the regret bound but the well-definedness of the central theorem statement, which needs an explicit convention before publication.

major comments (3)
  1. [Sec. II-B, Eq. (5), Eq. (8), and Theorem 3 (Eq. (16))] Theorem 3 and Corollary 2 use the path-length PL_T^{T+τ_{n,max}}_n, but Eq. (8) defines PL_T^n only as a sum over t=2,...,T, and the comparator sequence U_{n,t} is defined only for t=1,...,T. As written, the right-hand side of Eq. (16) is therefore not a well-defined function of the problem instance: if U_{n,t} is extended after T arbitrarily, PL_T^{T+τ_{n,max}} can be made arbitrarily large and the bound becomes vacuous. The proof itself evaluates U_{n,t} for t up to T+τ_{n,max} in Eqs. (29)-(40), including the Tail term. The standard repair is to state explicitly that U_{n,t}=U_{n,T} for T<t≤T+τ_{n,max}; then PL_T^{T+τ_{n,max}}_n = PL_T^n and all occurrences in the proof become well defined. Please add this convention to the problem statement and adjust the notation in Theorem 3 and Corollary 2 accordingly.
  2. [Sec. IV-B, Corollary 2 (Eqs. (26)-(27))] Corollary 2's static-regret statement drops the constant C_n that appears in Theorem 3. Setting U_{n,t}=U*_n makes the path-length vanish, but Theorem 3 still contains the additive term C_n = K_n Δτ_n |W_n| B. The corollary is salvageable because C_n ≤ B K_n |W_n| Δτ_n ≤ B√7 |W_n| K_n τ_{n,max} = B√7 √J_n, which is absorbed by the first term in Eq. (27). However, this step is not stated, so as written Corollary 2 is not a direct consequence of Theorem 3. Please include the bounding argument explicitly.
  3. [Sec. V-A, Eq. (29)] The regret decomposition in Eq. (29) and the subsequent bounds on Reg*, DriftU, and Tail require values of U_{n,t} and X_{n,t} beyond the horizon T (up to T+τ_{n,max}). While X_{n,t} can be interpreted as the iterate generated by Algorithm 1 if the algorithm is run past T, the comparator U_{n,t} is only introduced for t=1,...,T in Eq. (5). The Tail term is especially problematic because it sums over t=T+τ_{n,m}+1,...,T+τ_{n,max}. The proof needs an explicit extension of the comparator sequence before these quantities can appear in a theorem statement.
minor comments (3)
  1. [Appendix A, Eq. (A-2)] The inner product in Eq. (A-2) is written as ⟨..., X_{n,t-τ_{n,m}} - U_{n,t}⟩, while Lemma 1 and the preceding display (A-1) use X_{n,t}-U_{n,t}. This appears to be a typo from the index shift; it should be corrected to X_{n,t}-U_{n,t} for consistency with the proof.
  2. [Sec. IV-A] The protocol assumes a synchronous, reliable network with deterministic delays τ_{n,m}. This is a modeling restriction; please state it as an explicit assumption (or as a limitation in the conclusions) so that readers do not expect Theorem 3 to cover clock skew, jitter, retransmission, or packet loss.
  3. [Sec. VI-A] The text refers to 'DAM-TOGD (ST)' in the experimental description; this should be 'DDAM-TOGD (ST)'. Also, in Fig. 7 the word 'sieze' should be 'size'.

Circularity Check

0 steps flagged

No significant circularity: the regret bounds are derived from stated assumptions via standard telescoping arguments; the only self-citation is a non-load-bearing earlier-version pointer.

full rationale

I traced the main derivation chain. Theorem 3's bound is obtained from the decomposition in Eq. (29) and the separate bounds in Eqs. (33), (36), (39), and (40), each of which follows from Assumptions 1–3, the standard projection inequality, Cauchy–Schwarz, and telescoping sums. No fitted parameter is renamed as a prediction, and the worst-case regret quantities are not defined in terms of the algorithm's own output. The routing-tree design in Sec. IV-C minimizes τ_n,sum because that term appears in the derived bound; this is legitimate design-under-the-bound, not circular reasoning. The paper's only self-citation is [1], an earlier-version pointer to the ICASSP submission, and it is not load-bearing for any theorem. I also considered the formal out-of-horizon path-length issue in PL_T^{T+τ_n,max} in Theorem 3 and Corollary 2, as well as the omission of C_n in Corollary 2; these are correctness/well-formedness concerns, not circular reductions, since they do not make the claimed bound equal to an input by construction. Accordingly, the circularity score is 1, reflecting only the presence of a minor non-load-bearing self-citation.

Axiom & Free-Parameter Ledger

1 free parameters · 7 axioms · 0 invented entities

The central claim rests on standard bounded-domain, convex, bounded-gradient assumptions plus a synchronous deterministic-communication model. No new physical entities are introduced. The only ad hoc element is the implicit extension of the comparator sequence beyond the time horizon in the path-length definition.

free parameters (1)
  • Per-agent learning rate η_n = sqrt(7B^2/4)/sqrt(Q_n(T+Δτ_n)+J_n)
    Set in Corollary 2 to balance regret terms; depends on T, delay, and gradient bounds. Not fitted to data, but a hand-chosen schedule that the regret bound requires.
axioms (7)
  • domain assumption Assumption 1: X is closed, convex, contains the origin; f_n,t is convex for all n,t.
    Stated in Sec. II-C; needed for the convexity inequality and for projection properties used throughout the proof.
  • domain assumption Assumption 2: The diameter of X is bounded by B.
    Stated in Sec. II-C; bounds telescoping terms and tail terms in the regret analysis.
  • domain assumption Assumption 3: Gradients of f_n,t are bounded by G_n in Euclidean norm.
    Stated in Sec. II-C; used to bound all drift and delayed-gradient terms in Theorem 3.
  • domain assumption The logical weight matrix W is row-stochastic, fixed, and known to all agents.
    Problem definition in Sec. II-A; the algorithm and regret analysis assume every agent can compute weighted sums over W_n.
  • domain assumption Physical graph G admits routing trees from each agent to all agents in W_n, and communication is synchronous with deterministic edge delays.
    Sec. IV-A and Algorithm 1; update (14) and the regret bounds depend on fixed delays τ_n,m and would break under random delays or packet loss.
  • standard math Standard online learning facts: projection nonexpansiveness, Jensen's inequality, and the convexity subgradient inequality.
    Used in Appendix A and Appendix B; these are standard results from [7].
  • ad hoc to paper Comparator sequence U_n,t is implicitly extended beyond t=T to define PL_T^{T+τ_max}.
    Theorem 3 and Corollary 2 use a path-length with horizon T+τ_max although the comparator is only defined for t=1,...,T in Eq. (5); no explicit extension is given.

pith-pipeline@v1.3.0-alltime-deepseek · 19767 in / 19552 out tokens · 172544 ms · 2026-08-03T19:32:14.762956+00:00 · methodology

0 comments
read the original abstract

An associative memory (AM) enables cue-response recall, and it has recently been recognized as a key mechanism underlying modern neural architectures such as Transformers. In this work, we introduce the concept of distributed dynamic associative memory (DDAM), which extends classical AM to settings with multiple agents and time-varying data streams. In DDAM, each agent maintains a local AM that must not only store its own associations but also selectively memorize information from other agents based on a specified interest matrix. To address this problem, we propose a novel tree-based distributed online gradient descent algorithm, termed DDAM-TOGD, which enables each agent to update its memory on the fly via inter-agent communication over designated routing trees. We derive rigorous performance guarantees for DDAM-TOGD, proving sublinear static regret in stationary environments and a path-length dependent dynamic regret bound in non-stationary environments. These theoretical results provide insights into how communication delays and network structure impact performance. Building on the regret analysis, we further introduce a combinatorial tree design strategy that optimizes the routing trees to minimize communication delays, thereby improving regret bounds. Numerical experiments demonstrate that the proposed DDAM-TOGD framework achieves superior accuracy and robustness compared to representative online learning baselines such as consensus-based distributed optimization, confirming the benefits of the proposed approach in dynamic, distributed environments.

Figures

Figures reproduced from arXiv: 2511.23347 by Bowen Wang, Matteo Zecchin, Osvaldo Simeone.

Figure 1
Figure 1. Figure 1: (a) In a distributed dynamic associative memory (DDAM) system, each agent maintains a local memory, and corresponding AM mechanism, by processing local, streaming, data, as well as by interacting with other agents over a physical network. Each agent n collects streaming data in the form of key kn,t and value vn,t over discrete time t = 1, 2, ..., T. The blue region represents the subset Nn of physical neig… view at source ↗
Figure 2
Figure 2. Figure 2: Physical topology considered in the experiments. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Regret versus time horizon T (ρ = 0.75, y0 = 2, and y1 = 10). VI. EXPERIMENTS In this section, we empirically evaluate the performance of the proposed DDAM-TOGD using a synthetic dataset and a real wireless traffic dataset [14]. A. Synthetic Dataset We consider memorization under the DeltaNet model [21], where memory retrieval is linear and the loss function for agent n at time t is given by the third entr… view at source ↗
Figure 5
Figure 5. Figure 5: Regret at T = 2500 versus Dirichlet parameter y0 (ρ = 0.75, and y1 = 10). 0 1 2 4 3 5 6 7 8 9 10 11 12 13 14 15 [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Physical topology considered in the experiments with the wireless [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Regret of proposed DDAM-TOGD⋆ and scaled path-length versus time horizon T with different Ω (y0 = 10, and y1 = 100). for every AP. For each AP n and time step t, we form a key–value pair (kn,t, vn,t), where the value vn,t ∈ R dv aggre￾gates the six (dv = 6) 10-minute traffic measurements of AP n within the hour associated with time step t. The corresponding key is constructed as kn,t = [kAP,n; ktime,t] ∈ R… view at source ↗
Figure 9
Figure 9. Figure 9: NMSE versus Dirichlet parameter y0 (y1 = 100). more slowly. Overall, the dynamic regret closely follows the evolution of the scaled path-length. Convergence: In [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Mesh Inference: A Formal Model of Collective Inference Without a Center

    cs.MA 2026-06 unverdicted novelty 8.0

    Mesh inference allows a network of agents to reach the centralized optimum through local relaxations of a coupled free energy using only admitted observations, with convergence guaranteed by M-matrix properties in the...

Reference graph

Works this paper leans on

35 extracted references · 9 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Distributed associative memory via online convex optimization,

    B. Wang, M. Zecchin, and O. Simeone, “Distributed associative memory via online convex optimization,”arXiv preprint arXiv:2509.22321, 2025

    Pith/arXiv arXiv 2025

  2. [2]

    Episodic and semantic memory,

    E. Tulvinget al., “Episodic and semantic memory,”Organization of memory, vol. 1, no. 381-403, p. 1, 1972

    1972

  3. [3]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”Proc. Adv. Neural Inf. Process. Syst., vol. 30, 2017

    2017

  4. [4]

    It’s all connected: A journey through test-time memorization, attentional bias, retention, and online optimization,

    A. Behrouz, M. Razaviyayn, P. Zhong, and V . Mirrokni, “It’s all connected: A journey through test-time memorization, attentional bias, retention, and online optimization,”arXiv preprint arXiv:2504.13173, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Test-time regression: a unifying framework for designing sequence models with associative memory,

    K. A. Wang, J. Shi, and E. B. Fox, “Test-time regression: a unifying framework for designing sequence models with associative memory,” arXiv preprint arXiv:2501.12352, 2025

    Pith/arXiv arXiv 2025

  6. [6]

    Understanding trans- former from the perspective of associative memory,

    S. Zhong, M. Xu, T. Ao, and G. Shi, “Understanding trans- former from the perspective of associative memory,”arXiv preprint arXiv:2505.19488, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    A modern introduction to online learning,

    F. Orabona, “A modern introduction to online learning,”arXiv preprint arXiv:1912.13213, 2019

    Pith/arXiv arXiv 1912

  8. [8]

    Online learning and online convex optimiza- tion,

    S. Shalev-Shwartzet al., “Online learning and online convex optimiza- tion,”Found. Trends Mach. Learn., vol. 4, no. 2, pp. 107–194, 2012

    2012

  9. [9]

    Localization via ultra-wideband radios: a look at positioning aspects for future sensor networks,

    S. Gezici, Z. Tian, G. B. Giannakis, H. Kobayashi, A. F. Molisch, H. V . Poor, and Z. Sahinoglu, “Localization via ultra-wideband radios: a look at positioning aspects for future sensor networks,”IEEE Signal Process. Mag., vol. 22, no. 4, pp. 70–84, 2005

    2005

  10. [10]

    Machine learning for resource management in cellular and IoT networks: Potentials, current solutions, and open challenges,

    F. Hussain, S. A. Hassan, R. Hussain, and E. Hossain, “Machine learning for resource management in cellular and IoT networks: Potentials, current solutions, and open challenges,”IEEE Commun. Surveys Tuts., vol. 22, no. 2, pp. 1251–1275, 2020

    2020

  11. [11]

    5G wire- less network slicing for eMBB, URLLC, and mMTC: A communication- theoretic view,

    P. Popovski, K. F. Trillingsgaard, O. Simeone, and G. Durisi, “5G wire- less network slicing for eMBB, URLLC, and mMTC: A communication- theoretic view,”IEEE Access, vol. 6, pp. 55 765–55 779, 2018

    2018

  12. [12]

    Distributed optimization in sensor networks,

    M. Rabbat and R. Nowak, “Distributed optimization in sensor networks,” inProc. 3rd Int. Symp. Inf. Process. Sensor Netw., 2004, pp. 20–27

    2004

  13. [13]

    J. D. Hamilton,Time series analysis. Princeton university press, 2020

    2020

  14. [14]

    Wireless traffic usage forecasting using real enterprise network data: Analysis and methods,

    S. P. Sone, J. J. Lehtom ¨aki, and Z. Khan, “Wireless traffic usage forecasting using real enterprise network data: Analysis and methods,” IEEE Open J. Commun. Soc., vol. 1, pp. 777–797, 2020

    2020

  15. [15]

    Neural networks and physical systems with emergent collective computational abilities

    J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities.”Proc. of the National Academy of Sciences, vol. 79, no. 8, pp. 2554–2558, 1982

    1982

  16. [16]

    Dense associative memory for pattern recognition,

    D. Krotov and J. J. Hopfield, “Dense associative memory for pattern recognition,”Proc. Adv. Neural Inf. Process. Syst., vol. 29, 2016

    2016

  17. [17]

    Large associative memory problem in neuro- biology and machine learning, arxiv,

    D. Krotov and J. Hopfield, “Large associative memory problem in neuro- biology and machine learning, arxiv,”arXiv preprint arXiv:2008.06996, 2020

    Pith/arXiv arXiv 2008

  18. [18]

    Hopfield networks is all you need,

    H. Ramsauer, B. Sch ¨afl, J. Lehner, P. Seidl, M. Widrich, T. Adler, L. Gruber, M. Holzleitner, M. Pavlovi ´c, G. K. Sandveet al., “Hopfield networks is all you need,”arXiv preprint arXiv:2008.02217, 2020

    Pith/arXiv arXiv 2008

  19. [19]

    Transformers are RNNs: Fast autoregressive transformers with linear attention,

    A. Katharopoulos, A. Vyas, N. Pappas, and F. Fleuret, “Transformers are RNNs: Fast autoregressive transformers with linear attention,” inProc. Int. Conf. Mach. Learn., Virtual Site, 2020, pp. 5156–5165

    2020

  20. [20]

    Gated linear attention transformers with hardware-efficient training,

    S. Yang, B. Wang, Y . Shen, R. Panda, and Y . Kim, “Gated linear attention transformers with hardware-efficient training,”arXiv preprint arXiv:2312.06635, 2023

    Pith/arXiv arXiv 2023

  21. [21]

    Linear transformers are secretly fast weight programmers,

    I. Schlag, K. Irie, and J. Schmidhuber, “Linear transformers are secretly fast weight programmers,” inProc. Int. Conf. Mach. Learn., Virtual Site, 2021, pp. 9355–9366

    2021

  22. [22]

    Forgetting transformer: Softmax attention with a forget gate,

    Z. Lin, E. Nikishin, X. O. He, and A. Courville, “Forgetting transformer: Softmax attention with a forget gate,”arXiv preprint arXiv:2503.02130, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    Online convex programming and generalized infinitesi- mal gradient ascent,

    M. Zinkevich, “Online convex programming and generalized infinitesi- mal gradient ascent,” inProc. Int. Conf. Mach. Learn., Washington, DC, USA, 2003, pp. 928–936

    2003

  24. [24]

    Tracking slowly moving clair- voyant: Optimal dynamic regret of online learning with true and noisy gradient,

    T. Yang, L. Zhang, R. Jin, and J. Yi, “Tracking slowly moving clair- voyant: Optimal dynamic regret of online learning with true and noisy gradient,” inProc. Int. Conf. Mach. Learn., New York, NY , USA, 2016, pp. 449–457

    2016

  25. [25]

    Dynamic regret of convex and smooth functions,

    P. Zhao, Y .-J. Zhang, L. Zhang, and Z.-H. Zhou, “Dynamic regret of convex and smooth functions,”Proc. Adv. Neural Inf. Process. Syst., vol. 33, pp. 12 510–12 520, 2020

    2020

  26. [26]

    Adaptive regret of convex and smooth functions,

    L. Zhang, T.-Y . Liu, and Z.-H. Zhou, “Adaptive regret of convex and smooth functions,” inProc. Int. Conf. Mach. Learn., California, USA, 2019, pp. 7414–7423

    2019

  27. [27]

    Communication-efficient distributed learning: An overview,

    X. Cao, T. Bas ¸ar, S. Diggavi, Y . C. Eldar, K. B. Letaief, H. V . Poor, and J. Zhang, “Communication-efficient distributed learning: An overview,” IEEE J. Sel. Areas Commun., vol. 41, no. 4, pp. 851–873, 2023

    2023

  28. [28]

    Advances and open problems in federated learning,

    P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummingset al., “Advances and open problems in federated learning,”Found. Trends Mach. Learn., vol. 14, no. 1–2, pp. 1–210, 2021

    2021

  29. [29]

    A general framework for decentralized optimization with first-order methods,

    R. Xin, S. Pu, A. Nedi ´c, and U. A. Khan, “A general framework for decentralized optimization with first-order methods,”Proc. IEEE, vol. 108, no. 11, pp. 1869–1889, 2020

    2020

  30. [30]

    Gossip algorithms for distributed signal processing,

    A. G. Dimakis, S. Kar, J. M. Moura, M. G. Rabbat, and A. Scaglione, “Gossip algorithms for distributed signal processing,”Proc. IEEE, vol. 98, no. 11, pp. 1847–1864, 2010

    2010

  31. [31]

    A survey on multi-task learning,

    Y . Zhang and Q. Yang, “A survey on multi-task learning,”IEEE Trans. Knowl. Data Eng., vol. 34, no. 12, pp. 5586–5609, 2021

    2021

  32. [32]

    Distributed multi-task relationship learning,

    S. Liu, S. J. Pan, and Q. Ho, “Distributed multi-task relationship learning,” inProc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., Halifax, NS, Canada, 2017, pp. 937–946

    2017

  33. [33]

    Multitask learning over graphs: An approach for distributed, streaming machine learning,

    R. Nassif, S. Vlaski, C. Richard, J. Chen, and A. H. Sayed, “Multitask learning over graphs: An approach for distributed, streaming machine learning,”IEEE Signal Process. Mag., vol. 37, no. 3, pp. 14–25, 2020

    2020

  34. [34]

    Distributed au- tonomous online learning: Regrets and intrinsic privacy-preserving prop- erties,

    F. Yan, S. Sundaram, S. Vishwanathan, and Y . Qi, “Distributed au- tonomous online learning: Regrets and intrinsic privacy-preserving prop- erties,”IEEE Trans. Knowl. Data Eng., vol. 25, no. 11, pp. 2483–2493, 2013

    2013

  35. [35]

    Coin betting and parameter-free online learn- ing,

    F. Orabona and D. P ´al, “Coin betting and parameter-free online learn- ing,”Proc. Adv. Neural Inf. Process. Syst., vol. 29, 2016

    2016