Pith. sign in

REVIEW 3 major objections 4 minor 50 references

Joker: Joint Optimization Framework for Lightweight Kernel Machines

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that one dual block-coordinate-descent trust-region solver with random Fourier features trains KRR, KLR, and SVM on up to 10^7 samples using 0.7–5.9 GB GPU memory, with accuracy at or above Falkon, LogFalkon, EigenPro3…

desk verdict Joker is a credible empirical framework for large-scale kernel machines; its missing convergence analysis and lack of stationarity checks are real weaknesses, but not fatal to the practical claim. read the letter →

arxiv 2505.17765 v1 pith:RGGUAUB7 submitted 2025-05-23 cs.LG

Junhong Zhang , Zhihui Lai This is my paper · ORCID
classification cs.LG MSC 68T0590C2565K05
keywords kernelmethodsdualblockcoordinatedescenttrustregionrandomFourierfeatureslarge-scalelearningridgeregressionlogisticsupportvectormachines
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that a single optimization scheme can train the major kernel machines—kernel ridge regression, kernel logistic regression, and support vector machines—at scales up to roughly ten million samples on one consumer GPU. Its route is a dual formulation in which every listed model becomes the same kind of box-constrained quadratic-plus-conjugate problem, solved by a dual block coordinate descent method whose block steps are taken by a trust-region procedure. With random Fourier features replacing exact kernel evaluations, the method stores only the current block kernel and a maintained weight vector, which is why memory stays around 0.7–5.9 GB. The reported experiments on MSD, HEPC, SUSY, HIGGS, and CIFAR-5M show accuracy comparable to or better than Falkon, LogFalkon, EigenPro3, and ThunderSVM in similar or shorter time while using far less memory.

What carries the argument

The central object is the Fenchel-conjugate dual problem of Theorem 1, which converts each kernel model's primal regularized risk into a convex box-constrained problem whose Hessian is the kernel matrix. The carrying mechanism is DBCD-TR: block coordinate descent over fixed random blocks, with each block subproblem solved by a trust-region method—Algorithm 1 constructs a quadratic model using the block kernel and the Hessian of the conjugate loss, and Algorithm 2, a truncated CG-Steihaug method, computes a step inside the trust radius and projects it onto the box constraints. For inexact Joker, the random Fourier feature map $\psi(x)=\sqrt{2/M}\cos(Wx+b)$ supplies a finite-dimensional approximation, so the full kernel-gradient product $K_{B,:}\alpha$ becomes $\psi(X_B)^\top \theta$ and the maintained weight vector $\theta$ is updated incrementally after each block step. This reduces space to $O(|B|^2 + Md)$ and avoids the $O(M^2)$ preconditioner storage of Nyström-based methods, which is the concrete bottleneck the paper targets.

What would settle it

Take a medium dataset such as a $10^{5}$-sample subset of HIGGS, compute the exact dual optimum of problem (4) for KRR and KLR with a certified solver, run Joker with the paper's default iteration and block-size budgets, and check whether the dual objective approaches that optimum; if the gap remains large or validation accuracy is substantially below the exact solution, the convergence premise fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, Joker establishes that a single dual problem—minimizing $\frac{1}{2}\alpha^\top K\alpha + \frac{1}{\lambda}\sum_i \xi^*_{y_i}(-\lambda \alpha_i)$ over box constraints—captures KRR, Huber regression, L2-SVC, SVR, and KLR through their Fenchel conjugates. The authors derive this as Theorem 1, then solve it by DBCD-TR: each iteration picks a block $B$, forms a quadratic model with kernel block $K_{B,B}$ and gradient $K_{B,:}\alpha$, and takes a trust-region step computed by a truncated CG-Steihaug method with projection onto the box. For inexact Joker, random Fourier features $\psi(x)=\sqrt{2/M}\cos(Wx+b)$ make the expensive gradient reduce to $\psi(X_B)^\top \theta$, with $\theta$ maintained incrementally after each block update. The paper reports that this combination trains all three model families on datasets with $n$ up to $1.1\times 10^7$ using 0.7–5.9 GB GPU memory, with accuracy at or above the compared state-of-the-art methods in most cases, including better HIGGS and CIFAR-5M accuracy than Falkon, EigenPro3, and ThunderSVM.

Load-bearing premise

The load-bearing premise is that DBCD-TR actually reaches a near-optimal solution of the dual problem within the chosen iteration and block-size budgets; the paper does not prove this—Section 5 says the convergence speed of DBCD-TR is still unclear and Algorithm 2's truncation and projection are heuristic—so if those steps stall on KLR or SVR on the tested data, the reported accuracy and timing claims do not support the framework's stated superiority.

Editorial extensions

If this is right

  • Kernel logistic regression and SVM become as scalable as kernel ridge regression, because the same solver and the same per-iteration cost apply to classification losses.
  • A single RTX 3080 (10 GB) suffices for datasets with up to roughly 10^7 samples, whereas Falkon and LogFalkon need more than 50 GB at comparable model size.
  • Training time drops relative to EigenPro3 and ThunderSVM: on HIGGS, Joker-SVM converges in about one hour where EigenPro3 takes about 18 hours and ThunderSVM times out after a week.
  • The exact version of Joker stays competitive on medium datasets (about 5×10^5 samples), using under 1 GB of GPU memory and giving the best MSD relative error among the compared methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's own claims, the practical reach of Joker likely depends on DBCD-TR's unproved convergence behavior: on a dataset whose dual is badly scaled or whose proposed block steps are frequently rejected by the trust-region ratio, the fixed iteration budget may be far too small.
  • The same duality recipe using infimal convolutions of losses could extend to other composite losses and to multi-class machines, but the equality constraints of multi-class SVM and softmax would require a different subproblem solver than the box-constrained one used here.
  • Because the accuracy of inexact Joker is tied to the number of random Fourier features $M$, a natural testable extension is to plot test accuracy versus $M$ for each model family to see where the framework's memory savings start to cost generalization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Joker, a joint optimization framework for large-scale kernel machines that covers KRR, Huber regression, KLR, SVC, and SVR through a common dual formulation. The dual problem is solved by a dual block coordinate descent method with a trust-region subproblem solver (DBCD-TR), and the inexact variant replaces the kernel with random Fourier features. Experiments on MSD, HEPC, SUSY, HIGGS, and CIFAR-5M report lower GPU memory than Falkon, LogFalkon, EigenPro3, and ThunderSVM, with comparable or better accuracy and competitive training time. The dual derivation in Appendix A is standard and appears correct, and the code is released.

Significance. If the solver-convergence concern is resolved, this is a useful engineering contribution: a single dual BCD trust-region solver uniformly handles several kernel models, the inexact variant achieves large-scale results on a consumer GPU, and the empirical comparison is broad, with multiple baselines, error bars, and public datasets. The released code and the use of standard benchmarks strengthen reproducibility. However, the central empirical claims depend on DBCD-TR reaching near-optimal dual solutions within the reported budgets, and that point is currently unsupported; the significance of the results is therefore conditional on added convergence evidence or a proof.

major comments (3)
  1. [Section 5, Algorithm 2, Figure 3] The manuscript itself states in Section 5 that "the convergence speed of DBCD-TR is still unclear," and Section 2.2 introduces Algorithm 2 as "a heuristic truncated CG-Steihaug method" that terminates early on box violations and then projects the step. Because Algorithm 1 can reject steps and leave the block unchanged, there is no demonstrated monotone-decrease or stationarity guarantee, and no KKT residual or duality gap is reported for the runs in Table 4. Figure 3 shows primal/dual objective plots for only two settings (MSD exact KRR and HIGGS inexact KLR), so the reader cannot tell whether the reported accuracy and timing for Joker-KLR or Joker-SVM on the other datasets reflect solver quality or early stopping. I request either a convergence proof for DBCD-TR or, at minimum, final-iterate KKT residuals / duality gaps for every model-dataset combination in Table 4, together with a sensitivity check (e.g., doubling the iteration budget) on at least HIGGS and CIFAR-5M.
  2. [Section 3.2] For KLR, the feasible region is redefined as ε ≤ bα_i ≤ 1/λ − ε and the Hessian is capped via H̃_ii = min(∇²f(bα_B)_ii, ε^{-1/2}). This changes the dual objective that is actually minimized. The paper does not prove that a stationary point of this modified problem is close to a stationary point of the original KLR dual (4). Since the KLR results in Table 4 are produced with this modified problem and a larger block size, the reported KLR accuracy cannot be directly attributed to solving the original dual problem. Please provide a quantitative bound or an empirical verification (for example, reporting both the modified and true objective values at termination) that the truncation and Hessian cap do not materially change the solution.
  3. [Appendix B.2] For the nonsmooth SVR case, the trust-region subproblem is derived by replacing the ℓ1 term with sign(α_B)ᵀs, which is exact only when the signs of α_B and α_B + s agree. The text argues that this requires a sufficiently small trust-region radius, but it does not give a bound or a mechanism that enforces sign consistency, and no safeguard is described for cases where the projected truncated CG step crosses zero. Because the Joker-SVM results on MSD and HEPC in Table 4 rely on this approximation, the paper should either justify that the signs are preserved in practice (e.g., report sign-change statistics) or replace the approximation with a proximal trust-region step that handles the nonsmooth term exactly.
minor comments (4)
  1. [Section 2.3] There is a typo in the paragraph after Equation (10): "e can find a comprehensive summary" should read "We can find a comprehensive summary."
  2. [Appendix B heading] The heading "Details of Dual Block Coordinate Descent with T rust Region" contains a typo: "T rust" should be "Trust."
  3. [Abstract and Table 4] The abstract claims memory savings "up to 90%," but the largest datasets in Table 4 show about 81% savings on HIGGS (9.9 GB vs. 1.9 GB) and about 46% on CIFAR-5M (9.9 GB vs. 5.3 GB). The claim should state the exact comparison basis (e.g., the >50 GB Falkon configuration in Table 1) so the reader can reconcile it with the measured numbers in Table 4.
  4. [Section 4.1] In the sentence starting "Falkon-based methods are the fastest and have a substantial gap compared to EigenPro3 and ThunderSVM," the intended contrast is clear but the wording is awkward; consider rewriting to say that Falkon is fastest, EigenPro3 and ThunderSVM are much slower, and Joker sits between them.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found: the dual problem is obtained by standard Lagrangian/Fenchel duality, the RFF inexact variant follows from algebraic identities, and all performance claims are compared against external benchmarks; the admitted lack of a convergence proof for DBCD-TR is an unproven assumption rather than a circular reduction.

full rationale

The paper's central derivation is Theorem 1, which obtains the dual problem (4) from the primal (2) via Lagrange multipliers, first-order KKT conditions, and Fenchel conjugation, as shown in Appendix A. This is a standard duality argument and does not assume the conclusion. Proposition 2 extends the dual to infimal-convolution losses using the identity (ξ1□ξ2)* = ξ1* + ξ2*, which is likewise a standard mathematical result. The inexact Joker variant replaces the exact kernel with random Fourier features, defines ψ(x) in Eq. (10), and uses the identity KB,:α = ψ(XB)^T θ in Eq. (11) plus the θ-update in Eq. (12); these are algebraic identities, not fitted parameters renamed as predictions. The DBCD-TR solver is partly heuristic: Algorithm 2 is called a 'heuristic truncated CG-Steihaug method', and Section 5 explicitly states that 'the convergence speed of DBCD-TR is still unclear'. That is an open theoretical gap, and the empirical claims depend on the solver reaching near-optimal dual solutions within fixed budgets, but this is a correctness and robustness risk, not circular reasoning. All accuracy and timing claims are evaluated on external datasets (MSD, HEPC, SUSY, HIGGS, CIFAR-5M) against independently published baselines (Falkon, LogFalkon, EigenPro3, ThunderSVM) using the baselines' reported configurations. Hyperparameters are tuned per dataset in Appendix C, which weakens the strength of generalization claims but does not make any equation reduce to its own input. There are no load-bearing self-citations and no author-imported uniqueness theorem; the cited prior work on block coordinate descent and trust-region methods is external and is not used to certify the framework's correctness. Therefore, the derivation chain is self-contained with respect to circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper's dual derivation is standard and self-contained, so the ledger is not dominated by invented entities. The empirical results depend on a handful of user-chosen hyperparameters and on unproved convergence of the solver; those are the real costs the reader pays downstream.

free parameters (5)
  • regularization strength lambda = 2^-7 to 2^1 depending on dataset
    Chosen by grid search per dataset in Table A.3; directly affects all reported accuracy and timing numbers.
  • RFF dimension M = 1e4 to 2e5
    Set per dataset; larger M improves kernel approximation quality but increases memory and time, and this trade-off is central to the claimed memory savings.
  • block size |B| = 512, 1024, or 2048
    Chosen by hand as a convergence speed versus per-iteration cost trade-off; KLR uses larger blocks to compensate for ill-conditioning.
  • Huber delta and SVR epsilon = delta=1 or 2; epsilon=0.25
    Loss parameters selected per dataset without a stated selection rule, and they affect robust regression and SVR results.
  • kernel bandwidth sigma = median heuristic
    Data-dependent bandwidth for Gaussian and Laplacian kernels; not optimized during training but changes the kernel and therefore all results.
assumptions (5)
  • standard math Fenchel dual and KKT characterization of problem (2) are valid for all losses in Table 2.
    Used in Theorem 1 and Proposition 2 and Appendix A; requires convexity and closedness of losses, which are standard.
  • domain assumption Random Fourier features with M features provide a sufficiently accurate kernel approximation at the chosen M.
    Invoked in Section 2.3; the paper cites existing error bounds but does not use a concrete bound to set M.
  • ad hoc to paper DBCD-TR converges to a near-optimal dual solution within the reported iteration budgets.
    Section 5 says the convergence speed is still unclear; all experimental claims assume this premise.
  • standard math Slater's condition holds for the dual problem (4) for the losses used.
    Asserted in Section 2.1 to guarantee strong duality.
  • ad hoc to paper Algorithm 2's truncated CG step with projection remains an effective descent step for nonsmooth losses such as SVR.
    Appendix B.2 linearizes the l1 term and assumes sign consistency inside the trust region; this is heuristic and not proved.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Joker: Joint Optimization Framework for Lightweight Kernel Machines." pith.science (2026). https://pith.science/paper/RGGUAUB7

@misc{pith2026250517765,
  author       = {Pith},
  title        = {Pith review of: Joker: Joint Optimization Framework for Lightweight Kernel Machines},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RGGUAUB7}},
  note         = {Machine review of arXiv:2505.17765}
}
read the original abstract

Kernel methods are powerful tools for nonlinear learning with well-established theory. The scalability issue has been their long-standing challenge. Despite the existing success, there are two limitations in large-scale kernel methods: (i) The memory overhead is too high for users to afford; (ii) existing efforts mainly focus on kernel ridge regression (KRR), while other models lack study. In this paper, we propose Joker, a joint optimization framework for diverse kernel models, including KRR, logistic regression, and support vector machines. We design a dual block coordinate descent method with trust region (DBCD-TR) and adopt kernel approximation with randomized features, leading to low memory costs and high efficiency in large-scale learning. Experiments show that Joker saves up to 90\% memory but achieves comparable training time and performance (or even better) than the state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2505.17765 by the authors.

Figure 1
Figure 1. Performance versus the model size on HIGGS. [PITH_FULL_IMAGE:figures/full_fig_p014_1.png] view at source ↗
Figure 2
Figure 2. Test performance versus time. have a substantial gap compared to EigenPro3 and ThunderSVM. However, Joker alleviates this gap. Specifically, EigenPro3 and ThunderSVM use at least 10x training time compared to Falkon on MSD, and Joker reduces it to 5x time. On the other hand, EigenPro3 needs 36x time (18 hours) of Falkon (0.5 hour), and Joker reduces such gap to 2x time. Thus, Joker obtains comprehensively better res… view at source ↗
Figure 3
Figure 3. The primal, dual objectives, and validation loss of [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 46 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Aravkin, Robert Baraldi, and Dominique Orban

    Aleksandr Y. Aravkin, Robert Baraldi, and Dominique Orban. A proximal quasi- Newton trust-region method for nonsmooth regularized optimization. SIAM Journal on Optimization , 32(2):900--929, 2022

    work page 2022

  3. [3]

    Toward large kernel models

    Amirhesam Abedsoltan, Mikhail Belkin, and Parthe Pandit. Toward large kernel models. In the 40th International Conference on Machine Learning ( ICML ) , volume 202, pages 61--78. PRML, 2023

    work page 2023

  4. [4]

    Fast training of large kernel models with delayed projections, November 2024

    Amirhesam Abedsoltan, Siyuan Ma, Parthe Pandit, and Mikhail Belkin. Fast training of large kernel models with delayed projections, November 2024

    work page 2024

  5. [5]

    Convex optimization

    Stephen Boyd, Stephen P Boyd, and Lieven Vandenberghe. Convex optimization . Cambridge University press , 2004

    work page 2004

  6. [6]

    First-order methods in optimization

    Amir Beck. First-order methods in optimization . SIAM, 2017

    2017

  7. [7]

    Ellis, Brian Whitman, and Paul Lamere

    Thierry Bertin-Mahieux , Daniel P.W. Ellis, Brian Whitman, and Paul Lamere. The million song dataset. In Proceedings of the 12th International Conference on Music Information Retrieval ( ISMIR ) , 2011

    work page 2011

  8. [8]

    Baraldi and Drew P

    Robert J. Baraldi and Drew P. Kouri. Local convergence analysis of an inexact trust-region method for nonsmooth optimization. Optimization Letters , 18(3):663--680, April 2024

    work page 2024

Show all 50 references
  1. [9]

    Baraldi and Drew P

    Robert J. Baraldi and Drew P. Kouri. Efficient proximal subproblem solvers for a nonsmooth trust-region method. Computational Optimization and Applications , 90(1):193--226, January 2025

    2025

  2. [10]

    Searching for exotic particles in high-energy physics with deep learning

    Pierre Baldi, Peter Sadowski, and Daniel Whiteson. Searching for exotic particles in high-energy physics with deep learning. Nature communications , 5(1):4308, 2014

    2014

  3. [11]

    LIBSVM : A library for support vector machines

    Chih-Chung Chang and Chih-Jen Lin. LIBSVM : A library for support vector machines. ACM Transactions on Intelligent Systems and Technology , 2:27:1--27:27, 2011

    2011

  4. [12]

    On the algorithmic implementation of multiclass kernel-based vector machines

    Koby Crammer and Yoram Singer. On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research , 2:265--292, 2001

    2001

  5. [13]

    Support- Vector Networks

    Corinna Cortes and Vladimir Vapnik. Support- Vector Networks . Machine Leaming , 20(3):273--297, 1995

    1995

  6. [14]

    Deep neural tangent kernel and laplace kernel have the same RKHS

    Lin Chen and Sheng Xu. Deep neural tangent kernel and laplace kernel have the same RKHS . In the 8th International Conference on Learning Representations ( ICLR ) , October 2020

    2020

  7. [15]

    ReHLine : Regularized composite ReLU-ReHU loss minimization with linear computation and linear convergence

    Ben Dai and Yixuan Qiu. ReHLine : Regularized composite ReLU-ReHU loss minimization with linear computation and linear convergence. In Proceedings of the 37th Conference on Neural Information Processing Systems ( NeurIPS ) , volume 36, pages 14366--14386. Curran Associates, In...

    2023

  8. [16]

    Alexander G. de G. Matthews , Mark van der Wilk , Tom Nickson, Keisuke Fujii, Alexis Boukouvalas, Pablo Le \'o n-Villagr \'a , Zoubin Ghahramani, and James Hensman. GPflow : A Gaussian process library using TensorFlow . Journal of Machine Learning Research , 18(40):1--6, 2017

    2017

  9. [17]

    Scalable Kernel Methods via Doubly Stochastic Gradients

    Bo Dai, Bo Xie, Niao He, Yingyu Liang, Anant Raj, Maria-Florina F Balcan, and Le Song. Scalable Kernel Methods via Doubly Stochastic Gradients . In Proceedings of the 28th International Conference on Neural Information Processing Systems ( NeurIPS ) , volume 27. Curran Associa...

    2014

  10. [18]

    Gafni and Dimitri P

    Eli M. Gafni and Dimitri P. Bertsekas. Two-metric projection methods for constrained optimization. SIAM Journal on Control and Optimization , 22(6):936--964, 1984

    1984

  11. [19]

    Large sample analysis of the median heuristic, October 2018

    Damien Garreau, Wittawat Jitkrittum, and Motonobu Kanagawa. Large sample analysis of the median heuristic, October 2018

    2018

  12. [20]

    GPyTorch : Blackbox matrix-matrix gaussian process inference with GPU acceleration

    Jacob Gardner, Geoff Pleiss, Kilian Q Weinberger, David Bindel, and Andrew G Wilson. GPyTorch : Blackbox matrix-matrix gaussian process inference with GPU acceleration. In Proceedings of the 32nd Conference on Neural Information Processing Systems (NeurIPS) , volume 31, pages ...

    2018

  13. [21]

    On the similarity between the Laplace and neural tangent kernels

    Amnon Geifman, Abhay Yadav, Yoni Kasten, Meirav Galun, David Jacobs, and Basri Ronen. On the similarity between the Laplace and neural tangent kernels. In Proceedings of the 34th Conference on Neural Information Processing Systems ( NeurIPS ) , volume 33, pages 1451--1461. Cur...

    2020

  14. [22]

    Sathiya Keerthi, and S

    Cho-Jui Hsieh, Kai-Wei Chang, Chih-Jen Lin, S. Sathiya Keerthi, and S. Sundararajan. A dual coordinate descent method for large-scale linear SVM . In the 25th International Conference on Machine Learning (ICML) , pages 408--415. ACM Press, 2008

    2008

  15. [23]

    Neural tangent kernel: Convergence and generalization in neural networks

    Arthur Jacot, Franck Gabriel, and Clement Hongler. Neural tangent kernel: Convergence and generalization in neural networks. In Proceedings of the Conference on the 32nd Neural Information Processing Systems (NeurIPS) , volume 31, pages 8571--8580. Curran Associates, Inc., 2018

    2018

  16. [24]

    Samuel Lanthaler and Nicholas H. Nelsen. Error Bounds for Learning with Vector-Valued Random Features . In Proceedings of the 37th Conference on Neural Information Processing Systems ( NeurIPS ) , volume 36, pages 71834--71861, 2023

    2023

  17. [25]

    Stochastic gradient descent for gaussian processes done right

    Jihao Andreas Lin, Shreyas Padhy, Javier Antoran, Austin Tripp, Alexander Terenin, Csaba Szepesvari, Jos \'e Miguel Hern \'a ndez-Lobato , and David Janz. Stochastic gradient descent for gaussian processes done right. In the 12th International Conference on Learning Representa...

    2024

  18. [26]

    Diving into the shallows: A computational perspective on large-scale shallow learning

    Siyuan Ma and Mikhail Belkin. Diving into the shallows: A computational perspective on large-scale shallow learning. In Proceedings of the 31st Conference on Neural Information Processing Systems (NeurIPS) , volume 30, pages 3778--3787, 2017

    2017

  19. [27]

    Kernel machines that adapt to gpus for effective large batch training

    Siyuan Ma and Mikhail Belkin. Kernel machines that adapt to gpus for effective large batch training. In Proceedings of Machine Learning and Systems (MLSys) , volume 1, pages 360--373, 2019

    2019

  20. [28]

    Globally convergent newton methods for ill-conditioned generalized self-concordant losses

    Ulysse Marteau-Ferey , Francis Bach, and Alessandro Rudi. Globally convergent newton methods for ill-conditioned generalized self-concordant losses. In Proceedings of the 33rd Conference on Neural Information Processing Systems ( NeurIPS ) , volume 32, pages 7636--7646. Curran...

    2019

  21. [29]

    Kernel methods through the roof: Handling billions of points efficiently

    Giacomo Meanti, Luigi Carratino, Lorenzo Rosasco, and Alessandro Rudi. Kernel methods through the roof: Handling billions of points efficiently. In Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS) , volume 33, pages 14410--14422. Curran Ass...

    2020

  22. [30]

    Let's make block coordinate descent converge faster: Faster greedy rules, message-passing, active-set complexity, and superlinear convergence

    Julie Nutini, Issam Laradji, and Mark Schmidt. Let's make block coordinate descent converge faster: Faster greedy rules, message-passing, active-set complexity, and superlinear convergence. Journal of Machine Learning Research , 23(131):1--74, 2022

    2022

  23. [31]

    The deep bootstrap framework: Good online learners are good offline generalizers

    Preetum Nakkiran, Behnam Neyshabur, and Hanie Sedghi. The deep bootstrap framework: Good online learners are good offline generalizers. In the 9th International Conference on Learning Representations (ICLR) , 2021

    2021

  24. [32]

    Coordinate descent converges faster with the Gauss-Southwell rule than random selection

    Julie Nutini, Mark Schmidt, Issam Laradji, Michael Friedlander, and Hoyt Koepke. Coordinate descent converges faster with the Gauss-Southwell rule than random selection. In the 32nd International Conference on Machine Learning ( ICML ) , pages 1632--1641. PMLR, June 2015

    2015

  25. [33]

    John C. Platt. Sequential minimal optimization: A fast algorithm for training support vector machines. Technical report, Advances in Kernel Methods - Support Vector Learning , 1998

    1998

  26. [34]

    SDNA : Stochastic dual newton ascent for empirical risk minimization

    Zheng Qu, Peter Richtarik, Martin Takac, and Olivier Fercoq. SDNA : Stochastic dual newton ascent for empirical risk minimization. In the 33rd International Conference on Machine Learning , pages 1823--1832. PMLR, June 2016

    2016

  27. [35]

    On fast leverage score sampling and optimal learning

    Alessandro Rudi, Daniele Calandriello, Luigi Carratino, and Lorenzo Rosasco. On fast leverage score sampling and optimal learning. In Proceedings of the 32nd Conference on Neural Information Processing Systems (NeurIPS) , volume 31, pages 5672--5682. Curran Associates, Inc., 2018

    2018

  28. [36]

    FALKON : An optimal large scale kernel method

    Alessandro Rudi, Luigi Carratino, and Lorenzo Rosasco. FALKON : An optimal large scale kernel method. In Proceedings of the 31st Conference on Neural Information Processing Systems ( NeurIPS ) , volume 30, pages 3888--3898. Curran Associates, Inc., 2017

    2017

  29. [37]

    Have ASkotch : A neat solution for large-scale kernel ridge regression

    Pratik Rathore, Zachary Frangella, and Madeleine Udell. Have ASkotch : A neat solution for large-scale kernel ridge regression. arXiv preprint arXiv:2407.10070 , 2024

    2024

  30. [38]

    Random features for large-scale kernel machines

    Ali Rahimi and Ben Recht. Random features for large-scale kernel machines. In Proceedings of the 21st Conference on Neural Information Processing Systems ( NeurIPS ) , volume 14 E, pages 467--470, 2007

    2007

  31. [39]

    Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function

    Peter Richt \'a rik and Martin Tak \'a c . Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function. Mathematical Programming , 144(1):1--38, 2014

    2014

  32. [40]

    Graphical model structure learning with l1-regularization

    Mark Schmidt. Graphical model structure learning with l1-regularization. PhD Thesis, The University of British Columbia , page 26, 2010

    2010

  33. [41]

    Projected newton-type methods in machine learning

    Mark Schmidt, Dongmin Kim, and Suvrit Sra. Projected newton-type methods in machine learning. In Optimization for Machine Learning , pages 305--329. The MIT Press, September 2011

    2011

  34. [42]

    D. C. Sorensen. Newton's Method with a Model Trust Region Modification . SIAM Journal on Numerical Analysis , 19(2):409--426, April 1982

    1982

  35. [43]

    Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond

    Bernhard Sch \"o lkopf and Alexender Jorhannes Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond . MIT press, 1998

    1998

  36. [44]

    The conjugate gradient method and trust regions in large scale optimization

    Trond Steihaug. The conjugate gradient method and trust regions in large scale optimization. SIAM Journal on Numerical Analysis , 20(3):626--637, 1983

    1983

  37. [45]

    Stochastic dual coordinate ascent methods for regularized loss minimization

    Shai Shalev-Shwartz and Tong Zhang. Stochastic dual coordinate ascent methods for regularized loss minimization. Journal of Machine Learning Research , 14(16):567--599, 2013

    2013

  38. [46]

    Large scale kernel learning using block coordinate descent

    Stephen Tu, Rebecca Roelofs, Shivaram Venkataraman, and Benjamin Recht. Large scale kernel learning using block coordinate descent. arXiv preprint arXiv:1602.05310 , 2016

    2016 arXiv

  39. [47]

    Using the Nystr \"o m method to speed up kernel machines

    Christopher Williams and Matthias Seeger. Using the Nystr \"o m method to speed up kernel machines. In Proceedings of the 14th Conference on Neural Information Processing Systems ( NeurIPS ) , volume 13, pages 682--688. MIT Press, 2000

    2000

  40. [48]

    ThunderSVM : A fast SVM library on GPUs and CPUs

    Zeyi Wen, Jiashuai Shi, Qinbin Li, Bingsheng He, and Jian Chen. ThunderSVM : A fast SVM library on GPUs and CPUs . Journal of Machine Learning Research , 19(21):1--5, 2018

    2018

  41. [49]

    Dual coordinate descent methods for logistic regression and maximum entropy models

    Hsiang-Fu Yu, Fang-Lan Huang, and Chih-Jen Lin. Dual coordinate descent methods for logistic regression and maximum entropy models. Machine Learning , 85(1):41--75, October 2011

    2011

  42. [50]

    Nystr \"o m method vs random Fourier features: A theoretical and empirical comparison

    Tianbao Yang, Yu-feng Li, Mehrdad Mahdavi, Rong Jin, and Zhi-Hua Zhou. Nystr \"o m method vs random Fourier features: A theoretical and empirical comparison. In Proceedings of the 26th Conference on Neural Information Processing Systems ( NeurIPS ) , volume 25. Curran Associat...

    2012

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.