Adaptive Kernel Learning in Heterogeneous Networks

Alec Koppel; Amrit Singh Bedi; Hrusikesha Pradhan; Ketan Rajawat

REVIEW 3 major objections 5 minor 47 references

Adaptive Kernel Learning in Heterogeneous Networks

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper establishes that a decentralized kernel learning algorithm can match the O(T^{-1/2}) sub-optimality of unconstrained SGD while satisfying all network proximity constraints exactly on average.

desk verdict Worth taking seriously, but the proof of the main theorem has a load-bearing gap around Assumption 5: the norm-clipping step is not shown to preserve the KOMP error guarantee that Lemma 3 and Theorem 2 depend on. read the letter →

arxiv 1908.00510 v4 pith:3F7WUGLV submitted 2019-08-01 math.OC eess.SPstat.ML

Hrusikesha Pradhan , Amrit Singh Bedi , Alec Koppel , Ketan Rajawat This is my paper · ORCID

classification math.OCeess.SPstat.ML MSC 90C1590C2562L2068T05

keywords decentralizedoptimizationreproducingkernelHilbertspacestochasticprimal-dualmethodproximityconstraintsorthogonalmatchingpursuitonlinelearningconstraintviolationmodelcomplexity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a network of agents, each learning a nonlinear regression function from its own data stream, can coordinate through proximity constraints without sacrificing the optimal convergence rate or an affordable model size. The proposed algorithm, Heterogeneous Adaptive Learning with Kernels (HALK), runs a functional stochastic primal-dual method on an augmented Lagrangian and compresses each agent's function with greedy kernel matching pursuit, so the number of stored kernel centers stays bounded while iterates remain near the unprojected descent path. Under convexity, Lipschitz, and boundedness assumptions, the paper claims an average expected sub-optimality of $O(T^{-1/2} + \alpha)$ after $T$ iterations together with long-run average satisfaction of every proximity constraint, where $\alpha=\varepsilon/\eta$ is the compression budget. It also supplies a non-asymptotic bound of $O(\alpha^{-2p})$ on the dictionary size for data in $\mathbb{R}^p$, making explicit how memory trades off against accuracy. A sympathetic reader would care because the combination of the optimal unconstrained stochastic-gradient rate with exact long-run feasibility has not previously been available for kernel-based decentralized learning.

What carries the argument

The engine of the algorithm is a functional Arrow-Hurwicz primal-dual step on the augmented Lagrangian $L(f,\mu)=S(f)+\langle\mu,\mathcal{H}(f)+\nu\mathbf{1}-\gamma\rangle - \frac{\delta\eta}{2}\|\mu\|^2$. The primal descent is taken in the RKHS: the reproducing property turns the functional gradient into $\ell'_i(f_{i,t}(x_{i,t}),y_{i,t})\kappa(x_{i,t},\cdot)$ plus neighbor terms, and the dual ascent is an entry-wise projected gradient step for each edge. The $\nu$-tightening of the constraints and the dual regularization are what convert long-run feasibility from an approximation into an exact guarantee. To keep the representation finite, each unprojected iterate is passed through destructive kernel orthogonal matching pursuit (KOMP), which greedily removes kernel centers until the Hilbert-norm error against the unprojected function would exceed $\varepsilon$; a packing-number argument for Mercer kernels then gives the $O(\alpha^{-2p})$ model-order bound. A generalized Representer theorem for the constrained Lagrangian justifies writing every primal minimizer as a finite kernel expansion.

What would settle it

Run HALK on a compact domain with a Gaussian kernel and a stream whose unprojected gradients push $\|f_{i,t}\|_H$ above the claimed $R_B$, with the coefficient-rescaling rule active; measure $\|f_{i,t+1}-\tilde f_{i,t+1}\|_H$ at every step. If the rescaling makes this exceed $\varepsilon$ on a positive fraction of iterations, the projection-error bound behind Lemma 3 fails. Separately, set $\nu$ below the theorem's prescribed $\zeta T^{-1/2}+\Lambda\alpha$ and check whether the average constraint violation eventually becomes positive, which would contradict the claimed zero-violation regime.

Watch

Extended reading notes

Core claim

The central result is Theorem 2: with step-size $\eta=T^{-1/2}$, compression tolerance $\alpha=\varepsilon/\eta$, and a dual shift $\nu=\zeta T^{-1/2}+\Lambda\alpha$, the average expected sub-optimality of HALK is $O(T^{-1/2}+\alpha)$, and for every edge the average constraint violation satisfies $\frac{1}{T}\sum_{t=1}^T \mathbb{E}[H_{ij}(f_{i,t},f_{j,t})-\gamma_{ij}]\le 0$. Theorem 1 states that each agent's retained kernel dictionary has at most $\beta(R_M/\alpha)^{2p}$ elements, so fixing a memory budget $M$ places a floor of order $M^{-1/(2p)}$ on the achievable gap. Read together, the two theorems assert that the curse of kernelization can be controlled rather than merely lamented: lower $\alpha$ for accuracy and larger dictionaries, or higher $\alpha$ to save memory at a controlled cost. The proof is built from a per-iteration Lagrangian decrement inequality in which the KOMP projection error enters only as additive terms of order $\varepsilon/\eta$, and a lemma bounding the gap between the original and $\nu$-tightened problems by $O(\nu)$.

Load-bearing premise

The proof rests on Assumption 5: every compressed iterate and every optimum must lie in the same Hilbert ball of radius $R_B$, and the paper's suggested enforcement—dividing the KOMP coefficient vector by a large constant—is never analyzed; if that rescaling breaks the guarantee $\|f_{t+1}-\tilde f_{t+1}\|_H\le\varepsilon$, the optimality and feasibility bounds no longer follow.

Editorial extensions

If this is right

The decentralized kernel learning algorithm reaches the same $O(T^{-1/2})$ average sub-optimality as unconstrained stochastic gradient methods and, at the same time, satisfies network proximity constraints exactly on average over the run.
The memory-accuracy trade-off is governed by a single ratio $\alpha=\varepsilon/\eta$: to reduce the optimality floor by a factor of two, the allowed dictionary size must grow by a factor of order $2^{2p}$ in the data dimension.
The average feasibility guarantee holds for every edge regardless of the compression budget, so the method does not require re-tuning the tightening parameter when the dictionary is resized.
The coverage extends to non-differentiable convex losses through subgradients, so hinge-type losses are admissible without changing the rates.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The adaptive-bandwidth experiments in Section V-B are outside the formal coverage of Theorem 2, since the kernel itself changes over time; an extension would treat the bandwidth update in (39) as part of the stochastic iteration and verify whether the $O(T^{-1/2})$ rate survives the added randomness.
The enforcement of Assumption 5 by rescaling KOMP's coefficient vector is not analyzed in the paper, so the most defensible reading is that Theorem 2 is proved for iterates that happen to stay in the ball, and the clipping rule is an unverified repair rather than a proof step.
Since KOMP prunes dictionaries from local observations only, a testable extension would let agents exchange or diversify their kernel centers, potentially improving constraint satisfaction per unit of memory while preserving the same overall argument.
The exact long-run feasibility result suggests a practical tuning rule for networked field estimation: set the tightening $\nu$ just above the dominant $\Lambda\alpha$ term instead of hand-tuning penalty coefficients.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The one thing you should know: this is a legitimate paper, not a desk reject, but Theorem 2 has a proof gap that needs fixing before I'd trust the rates. The concern about Assumption 5 in the stress-test note is correct.

What is actually new: Theorem 1 gives a non-asymptotic model-order bound O(alpha^{-2p}) for multi-agent RKHS learning with proximity constraints, which prior work didn't have—earlier results were asymptotic or single-agent. Theorem 2 achieves O(T^{-1/2}+alpha) suboptimality with zero average constraint violation, matching the best vector-valued rates. The algorithmic machinery is largely assembled from the authors' own prior work—primal-dual from [15], KOMP from [29], constraint tightening from [9]—but the combination with heterogeneous proximity constraints plus the explicit dictionary bound is a real contribution. The proofs follow a standard template and are internally consistent up to the issue below.

The soft spot is load-bearing. Assumption 5 says the functions output from KOMP have Hilbert norm bounded by R_B, and the paper claims this can be enforced by dividing the coefficient vector by a large constant. That operation changes the output. If the unprojected KOMP output has norm larger than R_B, the rescaled point can be farther than epsilon from the unprojected function, and the guarantee ||f_{t+1} - f-tilde_{t+1}|| <= epsilon is exactly what Lemma 3 and the epsilon-dependent terms in Lemma 4 and Theorem 2 use. Algorithm 2 as printed has no such clipping, so the algorithm analyzed is not precisely the algorithm specified. This is fixable—prove an a priori bound on the unprojected iterates, or analyze the clipping with its true error bound—but it is not a cosmetic issue.

Minor concerns: K1 in Lemma 2 is an unspecified constant, so the rates are not fully explicit; the model-order bound in Theorem 1 depends on the dual iterates R_{i,t}, so it is not a clean uniform dictionary bound; the experiments lack error bars and don't connect the parsimony constant P to alpha, but they're illustrative rather than central.

My take: this is a solid extension from a capable group, and the Assumption 5 gap is likely patchable. For someone working on decentralized kernel methods, the model-order bound and the rate improvement are worth engaging with. I'd send it to a serious referee, expecting a request for major revision. If you're in the area, cite it; if you're refereeing, accept the assignment and push on the clipping step.

Referee Report

3 major / 5 minor

Summary. The paper considers decentralized online learning in which each agent estimates a function in a reproducing kernel Hilbert space and agents coordinate through nonlinear proximity constraints H_ij(f_i,f_j) <= gamma_ij. The authors formulate a stochastic constrained convex program, introduce a regularized Lagrangian, and propose a decentralized stochastic primal-dual algorithm with greedy dictionary compression (KOMP), called HALK. The main theoretical claims are: Theorem 1 gives a non-asymptotic model-order bound O(alpha^{-2p}); Theorem 2 gives, under Assumptions 1-5 and with eta = T^{-1/2} and nu = zeta T^{-1/2} + Lambda alpha, an average suboptimality O(T^{-1/2}+alpha) and zero average constraint violation; Corollary 2 translates this into O(1/epsilon^2) iteration complexity and O(1/epsilon^{2p}) model complexity. The numerical section validates the method on spatio-temporal random field estimation and on temperature and salinity data from the Gulf of Mexico, including an adaptive bandwidth variant.

Significance. If the main theorems hold, the paper makes a meaningful contribution: it provides the first non-asymptotic characterization of dictionary size for a decentralized RKHS primal-dual method, and it claims strict feasibility on average with the same O(T^{-1/2}) suboptimality rate as unconstrained SGD. The proof structure follows a standard primal-dual template and the appendices are unusually detailed. The experiments use real oceanographic data and compare against relevant baselines, which strengthens the practical claims. However, the central result currently depends on an unanalyzed norm-ball enforcement in Assumption 5, and the model-order bound depends on the dual trajectory without a uniform bound on the dual iterates. These are load-bearing gaps rather than presentational issues, so the paper needs a substantial revision before the claims can be accepted.

major comments (3)

[Section IV, Assumption 5; Algorithm 2; Lemma 3 (Eq. 100); Lemma 4 (Eq. 102)] The claimed enforcement of the Hilbert-ball bound in Assumption 5 is inconsistent with the KOMP approximation guarantee used in Lemmas 3 and 4. The paper states that the bound 'may be explicitly enforced by dividing the norm of the coefficient vector output from KOMP by a large constant,' but Algorithm 2 contains no such step, and the only guarantee provided by KOMP is ||f_{i,t+1} - \tilde f_{i,t+1}||_H <= epsilon. If the unprojected KOMP output has norm larger than R_B, rescaling changes f_{i,t+1}, and the distance from the rescaled output to \tilde f_{i,t+1} is at least ||\tilde f_{i,t+1}|| - R_B, which need not be <= epsilon and can be large because the dual multipliers mu_{ij,t} in (21) are not a priori bounded. Since Lemma 3 (Eq. 100), Lemma 4 (Eq. 102), and the bound ||f_t - f||_H <= 2 sqrt(V) R_B in Appendix B all require the KOMP error guarantee and the ball bound to hold simultaneously, Theorem 2 does not currently cover the algorithm as specified. A rigorous fix would prove a uniform a priori bound on \tilde f_{t+1}, or analyze the clipped projection with its exact error bound.
[Appendix E, Lemma 2 (Eq. 88); Theorem 2; Corollary 2] The bound on the dual stochastic gradient introduces an unspecified constant K1 ('for some 0 < K1 < infinity'), which propagates into the constant K in Theorem 2 and into Gamma used to set nu = zeta T^{-1/2} + Lambda alpha. The convergence rate O(T^{-1/2}+alpha) is unaffected by the value of K1, but the feasibility claim (37) is established only for zeta >= Gamma and Lambda >= 4 V R_B; because K1 is not identified in terms of the problem data, the prescribed parameter choice is not constructive. Please either give an explicit expression for K1 in terms of the known constants (C, L_h, gamma_ij, nu, X, R_B, V, E) or explicitly state that the feasibility condition uses a sufficiently large unspecified constant and therefore is non-constructive.
[Appendix A, Theorem 1 (Eq. 35)] The model-order bound in Theorem 1 is M_{i,t} <= beta (R_{M_{i,t}}/alpha)^{2p}, where R_{M_{i,t}} = C + L_h E R_{i,t} and R_{i,t} = max_{j in n_i} |mu_{ij,t}| is a time-dependent function of the dual iterates. The paper presents this as a non-asymptotic bound of order O(alpha^{-2p}), but without a uniform upper bound on ||mu_t|| or on R_{i,t}, the bound is not an absolute model-complexity guarantee; it is a trajectory-dependent bound. The proof in Appendix A bounds the denominator of the packing threshold using Assumptions 2 and 3 but does not bound R_{i,t} itself. Please either prove a uniform bound on the dual iterates (which would also help resolve the Assumption 5 issue) or restate Theorem 1 as a conditional statement under an additional bounded-dual-iterates assumption.

minor comments (5)

[Section V-A, Eq. (38) and following paragraph] The symbol delta is used both for the dual-regularization parameter in (13) and for the Huber-loss parameter in (38); the text says 'we choose delta = 10^4' while the figure caption sets delta = 10^{-5}. Please rename one of the two parameters to avoid confusion.
[Algorithm 2] The line 'while candidate dictionary is non-empty I/=empty' appears garbled, and the initialization 'f = \tilde f' is overwritten later; please use consistent notation for the current function, dictionary, and index set.
[Section V-B, Eq. (39)] The adaptive bandwidth update in (39) is not covered by the convergence analysis in Theorem 2, since the kernel is time-varying; the paper should state explicitly that this variant is heuristic and only validated experimentally.
[Paragraph after Theorem 1] The sentence 'the model order depends on the algorithm parameters only through the ratio alpha = eta/epsilon' contradicts the definition alpha = epsilon/eta in Theorem 1 and Corollary 2; please correct the ratio so that the prose matches the displayed equations.
[Table I] The entry for 'This Work' lists average constraint violation as 'zero'; this should read 'zero on average', since Theorem 2(ii) only guarantees the average violation is non-positive, not pathwise satisfaction.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Theorem 2 follows from explicit bounds in Appendix B; the Assumption 5 ball enforcement is a correctness gap, not a circular step.

full rationale

The derivation chain for Theorem 2 is not circular. Equations (61)-(70) in Appendix B exhibit an explicit bound on suboptimality and an explicit bound on constraint violation of the form (1/T) sum E[G_i(f_t)] <= Gamma/sqrt(T) + 4 V R_B alpha - nu. The constants zeta and Lambda are then chosen so that zeta >= Gamma and Lambda >= 4 V R_B, making the upper bound nonpositive by algebra; this is a standard dominance argument, not a fitted quantity renamed as a prediction. Lemma 3 (eq. 100) and Lemma 4 (eq. 102) are proved in the appendix from the KOMP error tolerance and Lipschitz assumptions; no equation is equivalent to its input by construction. The paper cites self-authored works [15] and [29] for the kernelized primal-dual method and KOMP, but the relevant representer and model-order arguments are re-proved in Appendices A and C, so the self-citations are not load-bearing. The one notable weakness is Assumption 5's claim that the R_B-ball condition 'may be explicitly enforced by dividing the norm of the coefficient vector output from KOMP by a large constant' (Sec. IV); this enforcement is not analyzed and could invalidate the KOMP guarantee used in Lemma 3. That is a correctness gap, not a circular reduction, so by the ground rules it does not raise the circularity score.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard convex-analysis assumptions (Slater, Lipschitz conditions, bounded kernel) plus one ad hoc assumption about bounded iterates that is enforced by an unanalyzed clipping procedure. The algorithm introduces no new physical or mathematical entities beyond the HALK procedure itself.

free parameters (4)

alpha = tunable
Compression parameter alpha = epsilon/eta controls the tradeoff between optimality gap O(alpha) and dictionary size O(alpha^{-2p}); the paper leaves it as a free design knob (Thm. 1 and Thm. 2).
zeta = lower bound given, not numeric
Dual-offset coefficient in nu = zeta T^{-1/2} + Lambda alpha; chosen large enough to force zero average constraint violation. The lower bound depends on problem constants such as R_B and K.
Lambda = Lambda >= 4 V R_B
Coefficient multiplying alpha in nu; chosen to dominate the alpha-term in the constraint-violation bound.
delta = chosen so C(delta) <= 0
Dual regularizer; must be large enough to make the coefficient of ||mu_t||^2 in the bound non-positive.

assumptions (5)

domain assumption Slater's condition (Assumption 4)
Strict feasibility with margin xi is assumed, giving strong duality and a Lipschitz gap between (3) and (12) (Lemma 1).
domain assumption Assumptions 1-3
Compact feature space, bounded kernel, Lipschitz losses and constraints; standard in online convex optimization.
ad hoc to paper Assumption 5
Boundedness of primal iterates and optimum in a Hilbert ball R_B; the paper proposes enforcing it by norm clipping, which is not analyzed.
standard math Representer theorem for constrained Lagrangians (Corollary 1 from [15, Thm. 1])
Basis expansion of primal minimizers over training samples; cited from prior work and proven in Appendix C.
standard math Covering number bound for Lipschitz Mercer kernels (from [46, Prop. 2.2])
On a compact set in R^p, the packing number of the feature map is O((1/vartheta)^{2p}); used to prove Theorem 1.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Adaptive Kernel Learning in Heterogeneous Networks." pith.science (2026). https://pith.science/paper/3F7WUGLV

@misc{pith2026190800510,
  author       = {Pith},
  title        = {Pith review of: Adaptive Kernel Learning in Heterogeneous Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3F7WUGLV}},
  note         = {Machine review of arXiv:1908.00510}
}

abstract

We consider learning in decentralized heterogeneous networks: agents seek to minimize a convex functional that aggregates data across the network, while only having access to their local data streams. We focus on the case where agents seek to estimate a regression \emph{function} that belongs to a reproducing kernel Hilbert space (RKHS). To incentivize coordination while respecting network heterogeneity, we impose nonlinear proximity constraints. To solve the constrained stochastic program, we propose applying a functional variant of stochastic primal-dual (Arrow-Hurwicz) method which yields a decentralized algorithm. To handle the fact that agents' functions have complexity proportional to time (owing to the RKHS parameterization), we project the primal iterates onto subspaces greedily constructed from kernel evaluations of agents' local observations. The resulting scheme, dubbed Heterogeneous Adaptive Learning with Kernels (HALK), when used with constant step-sizes, yields $\mathcal{O}(\sqrt{T})$ attenuation in sub-optimality and exactly satisfies the constraints in the long run, which improves upon the state of the art rates for vector-valued problems.

Figures

Figures reproduced from arXiv: 1908.00510 by the authors.

**Figure 1.** Convergence in terms of primal sub-optimality, constraint violation, and model complexity, for estimating a spatio-temporal correlated random field with parsimony constant P = 8, Gaussian kernel with bandwidth 0.05, λ = δ = 10−5 , η = 0.01 and penalty coefficient set at 0.08. Penalty method refers to an approximate constraint satisfaction approach to kernelized consensus optimization as in [35], whereas sine, degree… view at source ↗

**Figure 2.** Convergence in terms of primal sub-optimality and constraint violation, for temperature field of Gulf of Mexico with λ = δ = 10−5 , and η = 0.01. The centralized solver pools all information at a single location and applies projected stochastic gradient method [29], whereas P = 40 and P = 0.4 are two different parameter selections for the parsimony constant in Algorithm 1. Algorithm 1 is implemented with fixed bandw… view at source ↗

**Figure 3.** Convergence in terms of primal sub-optimality and constraint violation for salinity field of Gulf of Mexico with λ = δ = 10−5 , and η = 0.01. The centralized solver pools all information at a single location and applying projected stochastic gradient method [29], whereas P = 40 and P = 0.4 are two different parameter selections for the parsimony constant in Algorithm 1. Algorithm 1 is run a Gaussian kernel with band… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 46 canonical work pages

[15]

Projected st ochastic primal-dual method for constrained online learning with ke rnels,

A. Koppel, K. Zhang, H. Zhu, and T. Bas ¸ar, “Projected st ochastic primal-dual method for constrained online learning with ke rnels,” IEEE Transactions on Signal Processing, vol. 67, no. 10, pp. 2528–2542, 2019

work page 2019
[29]

Parsim onious online learning with kernels via sparse projections in function sp ace,

A. Koppel, G. Warnell, E. Stump, and A. Ribeiro, “Parsim onious online learning with kernels via sparse projections in function sp ace,” J. Mach. Learn. Res. , vol. 20, no. 1, pp. 83–126, 2019

work page 2019
[9]

Trading regret for efﬁci ency: online convex optimization with long term constraints,

M. Mahdavi, R. Jin, and T. Y ang, “Trading regret for efﬁci ency: online convex optimization with long term constraints,” J. Mach. Learn. Res. , vol. 13, no. Sep, pp. 2503–2528, 2012

work page 2012
[1]

Exact n onpara- metric decentralized online optimization,

H. Pradhan, A. S. Bedi, A. Koppel, and K. Rajawat, “Exact n onpara- metric decentralized online optimization,” in Proc. IEEE Global Conf. Signal Inf. Process. , Nov 2018, pp. 643–647

work page 2018
[2]

Consensus problems in networks of agents with switching topology and time-delays,

R. Olfati-Saber and R. M. Murray, “Consensus problems in networks of agents with switching topology and time-delays,” IEEE Trans. Autom. Control, vol. 49, no. 9, pp. 1520–1533, 2004

2004
[3]

Distributed processing over adaptive networks,

A. H. Sayed and C. G. Lopes, “Distributed processing over adaptive networks,” in 9th Int. Sym. on Signal Process. and Its Appl. IEEE, 2007, pp. 1–3

work page 2007
[4]

A multi-robo t control policy for information gathering in the presence of unknown hazards,

M. Schwager, P . Dames, D. Rus, and V . Kumar, “A multi-robo t control policy for information gathering in the presence of unknown hazards,” in Robotics Research. Springer, 2017, pp. 455–472

work page 2017
[5]

Source localization with d istributed sen- sor arrays and partial spatial coherence,

R. J. Kozick and B. M. Sadler, “Source localization with d istributed sen- sor arrays and partial spatial coherence,” IEEE Trans. Signal Process. , vol. 52, no. 3, pp. 601–616, 2004

work page 2004

Show all 47 references

[6]

Online Lea rning with Kernels,

J. Kivinen, A. J. Smola, and R. C. Williamson, “Online Lea rning with Kernels,” IEEE Trans. Signal Process. , vol. 52, pp. 2165–2176, August 2004

2004
[7]

Online convex optimization w ith stochastic constraints,

H. Y u, M. Neely, and X. Wei, “Online convex optimization w ith stochastic constraints,” in Proc. Adv. Neural Inf. Process. Syst. , 2017, pp. 1428–1438

2017
[8]

Subgradient methods for risk- sensitive optimization,

A. N. Madavan and S. Bose, “Subgradient methods for risk- sensitive optimization,” arXiv preprint arXiv:1908.01086 , 2019

1908 arXiv
[10]

Proximity with out consensus in online multiagent optimization,

A. Koppel, B. M. Sadler, and A. Ribeiro, “Proximity with out consensus in online multiagent optimization,” IEEE Trans. Signal Process., vol. 65, no. 12, pp. 3062–3077, 2017

2017
[11]

A low complexity algorithm with o( √ T ) regret and o(1) constraint violations for online convex optimization with long term constraints,

H. Y u and M. J. Neely, “A low complexity algorithm with o( √ T ) regret and o(1) constraint violations for online convex optimization with long term constraints,” J. Mach. Learn. Res. , vol. 21, no. 1, pp. 1–24, 2020

2020
[12]

Adaptive alg orithms for online convex optimization with long-term constraints,

R. Jenatton, J. Huang, and C. Archambeau, “Adaptive alg orithms for online convex optimization with long-term constraints,” i n Proc. Int. Conf. Mach. Learn. , 2016, pp. 402–411

2016
[13]

Online convex optimization f or cumulative constraints,

J. Y uan and A. Lamperski, “Online convex optimization f or cumulative constraints,” in Proc. Adv. Neural Inf. Process. Syst. , 2018, pp. 6137– 6146. 14

2018
[14]

Onli ne distributed learning over networks in rkh spaces using random fourier fe atures,

P . Bouboulis, S. Chouvardas, and S. Theodoridis, “Onli ne distributed learning over networks in rkh spaces using random fourier fe atures,” IEEE Transactions on Signal Processing , vol. 66, no. 7, pp. 1920–1932, 2017

1920
[16]

Decentral ised learning with random features and distributed gradient descent,

D. Richards, P . Rebeschini, and L. Rosasco, “Decentral ised learning with random features and distributed gradient descent,” in International Conference on Machine Learning . PMLR, 2020, pp. 8105–8115

2020
[17]

Coke: Communication - censored kernel learning for decentralized non-parametri c learning,

P . Xu, Y . Wang, X. Chen, and T. Zhi, “Coke: Communication - censored kernel learning for decentralized non-parametri c learning,” arXiv preprint arXiv:2001.10133 , 2020

2001 arXiv
[18]

A stochastic approximation me thod,

H. Robbins and S. Monro, “A stochastic approximation me thod,” Ann. Math. Statist. , vol. 22, no. 3, pp. 400–407, 09 1951

1951
[19]

Shapiro, D

A. Shapiro, D. Dentcheva, and A. Ruszczy´ nski, Lectures on stochastic programming: modeling and theory . SIAM, 2009

2009
[20]

Boyd and L

S. Boyd and L. V anderberghe, Convex Programming. New Y ork, NY: Wiley, 2004

2004
[21]

On the representation of continuous fu nctions of several variables as superpositions of continuous functions of one variable and addition,

V . Tikhomirov, “On the representation of continuous fu nctions of several variables as superpositions of continuous functions of one variable and addition,” in Selected W orks of AN Kolmogorov . Springer, 1991

1991
[22]

Image denoising via sparse and re dundant representations over learned dictionaries,

M. Elad and M. Aharon, “Image denoising via sparse and re dundant representations over learned dictionaries,” IEEE Trans. Img. Proc. , vol. 15, no. 12, pp. 3736–3745, Dec. 2006

2006
[23]

Neural networks: A comprehensive foundati on,

S. Haykin, “Neural networks: A comprehensive foundati on,” 1994

1994
[24]

Non-convex optimization for machine learning,

P . Jain, P . Kar et al. , “Non-convex optimization for machine learning,” F oundations and Trends® in Mach. Learn. , vol. 10, no. 3-4, pp. 142– 336, 2017

2017
[25]

Nonconvergence to unstable points in urn models and stochastic approximations,

R. Pemantle et al. , “Nonconvergence to unstable points in urn models and stochastic approximations,” The Annals of Probability, vol. 18, no. 2, pp. 698–712, 1990

1990
[26]

Online learning in reproducing kernel hilbert spaces,

K. Slavakis, P . Bouboulis, and S. Theodoridis, “Online learning in reproducing kernel hilbert spaces,” Sig. Process. Theory and Mach. Learn., pp. 883–987, 2013

2013
[27]

A generali zed representer theorem,

B. Sch¨ olkopf, R. Herbrich, and A. J. Smola, “A generali zed representer theorem,” Subseries of Lect. Notes in Comput. Sci. Edited by JG Carbonell and J. Siekmann , p. 416, 2001

2001
[28]

Matching pursuits with time- frequency dictionaries,

S. G. Mallat and Z. Zhang, “Matching pursuits with time- frequency dictionaries,” IEEE Trans. Signal Process. , vol. 41, no. 12, pp. 3397– 3415, Dec 1993

1993
[30]

Distributed stoch astic subgradient projection algorithms for convex optimization,

S. Ram, A. Nedic, and V . V eeravalli, “Distributed stoch astic subgradient projection algorithms for convex optimization,” J Optimiz. Theory App. , vol. 147, no. 3, pp. 516–545, Sep. 2010

2010
[31]

Online distri buted opti- mization via dual averaging,

S. Hosseini, A. Chapman, and M. Mesbahi, “Online distri buted opti- mization via dual averaging,” in Proc. 52nd IEEE Conf. Decis. Control . IEEE, 2013, pp. 1484–1489

2013
[32]

Distributed primal-dual met hods for online constrained optimization,

S. Lee and M. M. Zavlanos, “Distributed primal-dual met hods for online constrained optimization,” in ACC, 2016. IEEE, 2016, pp. 7171–7176

2016
[33]

Arrow, L

K. Arrow, L. Hurwicz, and H. Uzawa, Studies in Linear and Non-Linear Programming, ser. Stanford Mathematical Studies in the Social Sciences . Stanford University Press, Stanford, Dec. 1958, vol. II

1958
[34]

On the linear convergence of the admm in decentralized consensus optimiz ation

W. Shi, Q. Ling, K. Y uan, G. Wu, and W. Yin, “On the linear convergence of the admm in decentralized consensus optimiz ation.” IEEE Trans. Signal Process. , vol. 62, no. 7, pp. 1750–1761, 2014

2014
[35]

De centralized online learning with kernels,

A. Koppel, S. Paternain, C. Richard, and A. Ribeiro, “De centralized online learning with kernels,” IEEE Trans. Signal Process. , vol. 66, no. 12, pp. 3240–3255, June 2018

2018
[36]

Information theoretic le arning with adaptive kernels,

A. Singh and J. C. Pr´ ıncipe, “Information theoretic le arning with adaptive kernels,” Signal Processing, vol. 91, no. 2, pp. 203–213, 2011

2011
[37]

Some results on tchebychefﬁ an spline functions,

G. Kimeldorf and G. Wahba, “Some results on tchebychefﬁ an spline functions,” J. Math. Anal. Appl. , vol. 33, no. 1, pp. 82–95, 1971

1971
[38]

On stochastic optimization an d statistical learning in reproducing kernel hilbert spaces by support ve ctor machines (svm),

V . Norkin and M. Keyzer, “On stochastic optimization an d statistical learning in reproducing kernel hilbert spaces by support ve ctor machines (svm),” Informatica, vol. 20, no. 2, pp. 273–292, 2009

2009
[39]

Subgradient methods for sad dle-point problems,

A. Nedi´ c and A. Ozdaglar, “Subgradient methods for sad dle-point problems,” J. Opt. Theory and Appl. , vol. 142, no. 1, pp. 205–228, 2009

2009
[40]

Online le arning with kernels,

J. Kivinen, A. J. Smola, and R. C. Williamson, “Online le arning with kernels,” IEEE Transactions on Signal Processing , vol. 52, no. 8, pp. 2165–2176, 2004

2004
[41]

Breaking the curse of kerneliza- tion: Budgeted stochastic gradient descent for large-scal e svm training,

Z. Wang, K. Crammer, and S. Vucetic, “Breaking the curse of kerneliza- tion: Budgeted stochastic gradient descent for large-scal e svm training,” J. Mach. Learn. Res. , vol. 13, no. 1, pp. 3103–3131, 2012

2012
[42]

Kernel matching pursuit,

P . Vincent and Y . Bengio, “Kernel matching pursuit,” Mach. Learn. , vol. 48, no. 1, pp. 165–187, 2002

2002
[43]

Gulf of mexico regional climatolo gy (NCEI Accession 0123320),

T. P . Boyer, M. Biddle, M. Hamilton, A. V . Mishonov, C. Pa ver, D. Seidov, and M. . Zweng, “Gulf of mexico regional climatolo gy (NCEI Accession 0123320),” V ersion 1.1. NOAA National Centers for Environmental Inf
[44]

Asynchronous sad dle point algorithm for stochastic optimization in heterogeneous ne tworks,

A. S. Bedi, A. Koppel, and K. Rajawat, “Asynchronous sad dle point algorithm for stochastic optimization in heterogeneous ne tworks,” IEEE Trans. Signal Process. , vol. 67, no. 7, pp. 1742–1757, 2019

2019
[45]

Asynchronous onl ine learning in multi-agent systems with proximity constraints,

A. S. Bedi, A. Koppel, and K. Rajawat, “Asynchronous onl ine learning in multi-agent systems with proximity constraints,” IEEE Trans. Signal Inf. Process. Netw. , pp. 479 – 494, 2019

2019
[46]

The kernel recursive l east-squares algorithm,

Y . Engel, S. Mannor, and R. Meir, “The kernel recursive l east-squares algorithm,” IEEE Trans. Signal Process. , vol. 52, no. 8, pp. 2275–2285, Aug 2004

2004
[47]

The covering number in learning theory,

D.-X. Zhou, “The covering number in learning theory,” Journal of Complexity, vol. 18, no. 3, pp. 739–767, 2002. 15 Supplementary Material for: Adaptive Kernel Learning in Heterogeneous Networks by Hrusikesha Pradhan, Amrit Singh Bedi, Alec Koppel, and Ke tan Rajawat APPENDIX C...

2002

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [15]

Projected st ochastic primal-dual method for constrained online learning with ke rnels,

A. Koppel, K. Zhang, H. Zhu, and T. Bas ¸ar, “Projected st ochastic primal-dual method for constrained online learning with ke rnels,” IEEE Transactions on Signal Processing, vol. 67, no. 10, pp. 2528–2542, 2019

work page 2019

[2] [29]

Parsim onious online learning with kernels via sparse projections in function sp ace,

A. Koppel, G. Warnell, E. Stump, and A. Ribeiro, “Parsim onious online learning with kernels via sparse projections in function sp ace,” J. Mach. Learn. Res. , vol. 20, no. 1, pp. 83–126, 2019

work page 2019

[3] [9]

Trading regret for efﬁci ency: online convex optimization with long term constraints,

M. Mahdavi, R. Jin, and T. Y ang, “Trading regret for efﬁci ency: online convex optimization with long term constraints,” J. Mach. Learn. Res. , vol. 13, no. Sep, pp. 2503–2528, 2012

work page 2012

[4] [1]

Exact n onpara- metric decentralized online optimization,

H. Pradhan, A. S. Bedi, A. Koppel, and K. Rajawat, “Exact n onpara- metric decentralized online optimization,” in Proc. IEEE Global Conf. Signal Inf. Process. , Nov 2018, pp. 643–647

work page 2018

[5] [2]

Consensus problems in networks of agents with switching topology and time-delays,

R. Olfati-Saber and R. M. Murray, “Consensus problems in networks of agents with switching topology and time-delays,” IEEE Trans. Autom. Control, vol. 49, no. 9, pp. 1520–1533, 2004

2004

[6] [3]

Distributed processing over adaptive networks,

A. H. Sayed and C. G. Lopes, “Distributed processing over adaptive networks,” in 9th Int. Sym. on Signal Process. and Its Appl. IEEE, 2007, pp. 1–3

work page 2007

[7] [4]

A multi-robo t control policy for information gathering in the presence of unknown hazards,

M. Schwager, P . Dames, D. Rus, and V . Kumar, “A multi-robo t control policy for information gathering in the presence of unknown hazards,” in Robotics Research. Springer, 2017, pp. 455–472

work page 2017

[8] [5]

Source localization with d istributed sen- sor arrays and partial spatial coherence,

R. J. Kozick and B. M. Sadler, “Source localization with d istributed sen- sor arrays and partial spatial coherence,” IEEE Trans. Signal Process. , vol. 52, no. 3, pp. 601–616, 2004

work page 2004