pith. sign in

arxiv: 2606.06742 · v2 · pith:I52XDYUJnew · submitted 2026-06-04 · 💻 cs.LG · stat.ML

TorchKM: A GPU-Oriented Library for Kernel Learning and Model Selection

Yikai Zhang , Gaoxiang Jia , Jie Ding , Boxiang Wang This is my paper

Pith reviewed 2026-06-28 02:03 UTC · model grok-4.3

classification 💻 cs.LG stat.ML
keywords kernel machinesGPU accelerationsupport vector machinesmodel selectionkernel logistic regressionkernel quantile regressionlibrary
0
0 comments X

The pith

TorchKM is a GPU-accelerated library for kernel machines that reuses matrix operations to speed up the full training and model-selection pipeline.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces TorchKM as an open-source library that implements support vector machines, kernel logistic regression, and kernel quantile regression with GPU acceleration. It uses a scikit-learn-style API and exploits GPU-friendly linear algebra to accelerate the complete pipeline by intelligently reusing matrix operations. The authors claim this yields competitive predictive performance together with substantial speedups over standard baselines and positions the library as a reusable component for AI-driven workflows.

Core claim

TorchKM accelerates kernel machine training and model selection on GPUs by intelligent reuse of matrix operations while preserving competitive predictive performance and providing a familiar API.

What carries the argument

Intelligent reuse of matrix operations in the GPU-accelerated training and model-selection pipeline for kernel machines.

If this is right

  • Kernel methods including SVMs and kernel regressions complete training and model selection faster on GPU hardware.
  • The full pipeline from data to selected model benefits from the matrix reuse without separate optimizations for each step.
  • The library serves as a drop-in component for larger AI workflows due to its programmable design and easy installation.
  • Users obtain the speedups while retaining a familiar scikit-learn-style interface.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Kernel methods could become practical for larger datasets where CPU implementations were previously too slow.
  • The approach may support hybrid systems that combine kernel models with neural networks at training time.
  • Extensions could test the same reuse pattern on other kernel-based algorithms not covered in the current library.

Load-bearing premise

That reuse of matrix operations on GPU will deliver substantial speedups while maintaining competitive predictive performance.

What would settle it

A side-by-side benchmark on the same datasets that shows either no meaningful speedup or lower accuracy than standard baselines would falsify the central claim.

Figures

Figures reproduced from arXiv: 2606.06742 by Boxiang Wang, Gaoxiang Jia, Jie Ding, Yikai Zhang.

Figure 1
Figure 1. Figure 1: Run time for standard SVM training and tuning under CPU and GPU compu [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Reliability curve for Platt-calibrated probabilities on the independent test set. The [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
read the original abstract

TorchKM is an open-source library for kernel machines, including support vector machines, kernel logistic regression, and kernel quantile regression, with GPU acceleration. The library features a scikit-learn-style API and is designed to exploit GPU-friendly linear algebra, accelerating the full training and model-selection pipeline through intelligent reuse of matrix operations. Benchmarks show competitive predictive performance with substantial speedups over standard baselines. The efficiency and programmable design also make TorchKM a kernel-learning component for AI-driven workflows. Code and documentation are available at https://github.com/YikaiZhang95/torchkm, and the package can be easily installed via PyPI.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 3 minor

Summary. The manuscript introduces TorchKM, an open-source GPU-accelerated library implementing kernel machines (SVM, kernel logistic regression, kernel quantile regression) with a scikit-learn-style API. It emphasizes intelligent reuse of matrix operations to accelerate the full training and hyperparameter-selection pipeline, reports competitive predictive performance, and claims substantial speedups over baselines such as scikit-learn and cuML. The code is publicly available on GitHub and installable via PyPI.

Significance. If the benchmark claims hold, the library supplies a practical, programmable component for kernel methods on GPUs that could integrate into larger AI-driven workflows. The open-source release and focus on reusable linear-algebra primitives constitute a clear engineering contribution; reproducible code and documented installation lower the barrier for adoption.

major comments (2)
  1. [§4, Table 2] §4, Table 2: the reported wall-clock speedups are presented without per-dataset matrix dimensions or conditioning numbers; without these quantities it is impossible to verify that the claimed matrix-reuse strategy is the dominant source of the observed gains rather than hardware or library differences.
  2. [§3.2] §3.2: the description of the model-selection loop reuses the Gram matrix across folds, yet no analysis is given of the numerical stability or memory-footprint trade-off when the kernel matrix exceeds GPU memory; this directly affects the practicality of the “full pipeline” acceleration claim.
minor comments (3)
  1. [Abstract] The abstract states “competitive predictive performance” but does not name the exact metrics (accuracy, AUC, pinball loss) or the cross-validation protocol; these details appear only later and should be summarized in the abstract.
  2. [Figure 3] Figure 3 caption does not indicate whether error bars represent standard deviation over 5 runs or over different random seeds; this reduces interpretability of the timing curves.
  3. [References] Several citations to cuML and scikit-learn are given only by package name; full bibliographic entries should be added for reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments and the recommendation for minor revision. We address each major comment below and will incorporate the suggested clarifications into the revised manuscript.

read point-by-point responses

  1. Referee: [§4, Table 2] §4, Table 2: the reported wall-clock speedups are presented without per-dataset matrix dimensions or conditioning numbers; without these quantities it is impossible to verify that the claimed matrix-reuse strategy is the dominant source of the observed gains rather than hardware or library differences.

    Authors: We agree that the absence of matrix dimensions and conditioning numbers limits the ability to attribute speedups specifically to the matrix-reuse strategy. In the revision we will expand Table 2 to report, for each dataset, the number of samples, number of features, and an estimate of the Gram-matrix condition number (computed via the ratio of largest to smallest eigenvalue where feasible). All timing comparisons were performed on identical hardware with the same underlying cuBLAS/cuSOLVER calls for the linear-algebra primitives, which isolates the effect of our reuse pattern; the added metadata will make this explicit. revision: yes

  2. Referee: [§3.2] §3.2: the description of the model-selection loop reuses the Gram matrix across folds, yet no analysis is given of the numerical stability or memory-footprint trade-off when the kernel matrix exceeds GPU memory; this directly affects the practicality of the “full pipeline” acceleration claim.

    Authors: The current implementation and experiments assume the Gram matrix resides in GPU memory, consistent with the moderate dataset sizes used. We acknowledge that a discussion of the memory-stability trade-off is missing. In the revised §3.2 we will add a paragraph describing (i) the memory footprint as a function of n and the kernel bandwidth, (ii) the numerical conditioning implications of reusing the same Gram matrix across folds, and (iii) the fallback behavior (CPU offload or block-wise processing) when the matrix does not fit. We will also report peak GPU memory usage alongside the timing results. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper describes a GPU-accelerated software library for kernel methods with scikit-learn-style API and benchmark comparisons. It contains no mathematical derivations, first-principles predictions, fitted parameters presented as predictions, or load-bearing self-citations. All claims are empirical implementation results supported by open code, hardware specs, and external baselines; the derivation chain is absent by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is a description of a software library. No free parameters, axioms, or invented entities are involved in a theoretical sense.

pith-pipeline@v0.9.1-grok · 5632 in / 1011 out tokens · 40128 ms · 2026-06-28T02:03:53.075260+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references

  1. [1]

    Journal of Computational and Graphical Statistics , volume=

    Sparse distance weighted discrimination , author=. Journal of Computational and Graphical Statistics , volume=. 2016 , publisher=

    2016

  2. [2]

    Parallel and distributed structured

    Jiang, Jiantong and Wen, Zeyi and Wang, Zeke and He, Bingsheng and Chen, Jian , journal=. Parallel and distributed structured. 2021 , publisher=

    2021

  3. [3]

    A method for solving the convex programming problem with convergence rate

    Nesterov, Yurii , booktitle=. A method for solving the convex programming problem with convergence rate

  4. [4]

    fastkqr:

    Tang, Qian and Gu, Yuwen and Wang, Boxiang , journal=. fastkqr:. 2026 , publisher=

    2026

  5. [5]

    Journal of Economic Perspectives , volume=

    Quantile regression , author=. Journal of Economic Perspectives , volume=. 2001 , publisher=

    2001

  6. [6]

    Technometrics , volume=

    A multicategory kernel distance weighted discrimination method for multiclass classification , author=. Technometrics , volume=. 2019 , publisher=

    2019

  7. [7]

    Journal of the American Statistical Association , volume=

    Distance-weighted discrimination , author=. Journal of the American Statistical Association , volume=. 2007 , publisher=

    2007

  8. [8]

    Advances in Kernel Methods: Support Vector Learning , pages=

    Fast Training of support vector machines using sequential minimal optimization , author=. Advances in Kernel Methods: Support Vector Learning , pages=. 1999 , publisher=

    1999

  9. [9]

    Machine Learning , volume=

    Support-vector networks , author=. Machine Learning , volume=. 1995 , publisher=

    1995

  10. [10]

    kernlab--an

    Karatzoglou, Alexandros and Smola, Alexandros and Hornik, Kurt and Zeileis, Achim , journal=. kernlab--an

  11. [11]

    Using the

    Williams, Christopher and Seeger, Matthias , journal=. Using the

  12. [12]

    Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=

    Another look at distance-weighted discrimination , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 2018 , publisher=

    2018

  13. [13]

    Neurocomputing , volume=

    A comprehensive survey on support vector machine classification: Applications, challenges and trends , author=. Neurocomputing , volume=. 2020 , publisher=

    2020

  14. [14]

    Technometrics , volume=

    Fast and exact leave-one-out analysis of large-margin classifiers , author=. Technometrics , volume=. 2022 , publisher=

    2022

  15. [15]

    Advances in Neural Information Processing Systems , volume=

    A Consolidated Cross-Validation Algorithm for Support Vector Machines via Data Reduction , author=. Advances in Neural Information Processing Systems , volume=

  16. [16]

    Programming with Tensor

    PyTorch , author=. Programming with Tensor. 2021 , publisher=

    2021

  17. [17]

    Nature , volume=

    Array programming with NumPy , author=. Nature , volume=. 2020 , publisher=

    2020

  18. [18]

    Scikit-learn: Machine learning in

    Pedregosa, Fabian and Varoquaux, Ga. Scikit-learn: Machine learning in. Journal of Machine Learning Research , volume=. 2011 , publisher=

    2011

  19. [19]

    Scalable parallel programming with

    Nickolls, John and Buck, Ian and Garland, Michael and Skadron, Kevin , journal=. Scalable parallel programming with. 2008 , publisher=

    2008

  20. [20]

    2011 , publisher=

    Chang, Chih-Chung and Lin, Chih-Jen , journal=. 2011 , publisher=

    2011

  21. [21]

    Diamond, Steven and Boyd, Stephen , journal=

  22. [22]

    Wen, Zeyi and Shi, Jiashuai and Li, Qinbin and He, Bingsheng and Chen, Jian , journal=. Thunder

  23. [23]

    Efficient multi-class probabilistic

    Wen, Zeyi and Shi, Jiashuai and He, Bingsheng and Chen, Jian and Chen, Yawen , journal=. Efficient multi-class probabilistic. 2018 , publisher=

    2018

  24. [24]

    Journal of Machine Learning Research , volume=

    Do we need hundreds of classifiers to solve real world classification problems? , author=. Journal of Machine Learning Research , volume=. 2014 , publisher=

    2014

  25. [25]

    Neural Computation , volume=

    The lack of a priori distinctions between learning algorithms , author=. Neural Computation , volume=. 1996 , publisher=

    1996

  26. [26]

    Advances in Neural Information Processing Systems , volume=

    Why do tree-based models still outperform deep learning on typical tabular data? , author=. Advances in Neural Information Processing Systems , volume=

  27. [27]

    Advances in Large Margin Classifiers , year =

    Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods , author =. Advances in Large Margin Classifiers , year =

  28. [28]

    Proceedings of the 22nd International Conference on Machine Learning (ICML) , year =

    Predicting good probabilities with supervised learning , author =. Proceedings of the 22nd International Conference on Machine Learning (ICML) , year =

  29. [29]

    Proceedings of the 34th International Conference on Machine Learning (ICML) , year =

    On Calibration of Modern Neural Networks , author =. Proceedings of the 34th International Conference on Machine Learning (ICML) , year =

  30. [30]

    2009 , publisher =

    The Elements of Statistical Learning: Data Mining, Inference, and Prediction , author =. 2009 , publisher =

    2009

  31. [31]

    A comprehensive survey on support vector machine classification: Applications, challenges and trends

    Jair Cervantes, Farid Garcia-Lamont, Lisbeth Rodr \' guez-Mazahua, and Asdrubal Lopez. A comprehensive survey on support vector machine classification: Applications, challenges and trends. Neurocomputing, 408: 0 189--215, 2020

    2020

  32. [32]

    LIBSVM : A library for support vector machines

    Chih-Chung Chang and Chih-Jen Lin. LIBSVM : A library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST), 2 0 (3): 0 1--27, 2011

    2011

  33. [33]

    Support-vector networks

    Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine Learning, 20 0 (3): 0 273--297, 1995

    1995

  34. [34]

    The Elements of Statistical Learning: Data Mining, Inference, and Prediction

    Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York, 2nd edition, 2009

    2009

  35. [35]

    Parallel and distributed structured SVM training

    Jiantong Jiang, Zeyi Wen, Zeke Wang, Bingsheng He, and Jian Chen. Parallel and distributed structured SVM training. IEEE Transactions on Parallel and Distributed Systems, 33 0 (5): 0 1084--1096, 2021

    2021

  36. [36]

    kernlab--an S 4 package for kernel methods in R

    Alexandros Karatzoglou, Alexandros Smola, Kurt Hornik, and Achim Zeileis. kernlab--an S 4 package for kernel methods in R . Journal of Statistical Software, 11: 0 1--20, 2004

    2004

  37. [37]

    Quantile regression

    Roger Koenker and Kevin F Hallock. Quantile regression. Journal of Economic Perspectives, 15 0 (4): 0 143--156, 2001

    2001

  38. [38]

    Distance-weighted discrimination

    James Stephen Marron, Michael J Todd, and Jeongyoun Ahn. Distance-weighted discrimination. Journal of the American Statistical Association, 102 0 (480): 0 1267--1271, 2007

    2007

  39. [39]

    A method for solving the convex programming problem with convergence rate O (1/k^2)

    Yurii Nesterov. A method for solving the convex programming problem with convergence rate O (1/k^2) . 269 0 (3): 0 543--547, 1983

    1983

  40. [40]

    Scalable parallel programming with CUDA : Is CUDA the parallel programming model that application developers have been waiting for? Queue, 6 0 (2): 0 40--53, 2008

    John Nickolls, Ian Buck, Michael Garland, and Kevin Skadron. Scalable parallel programming with CUDA : Is CUDA the parallel programming model that application developers have been waiting for? Queue, 6 0 (2): 0 40--53, 2008

    2008

  41. [41]

    Scikit-learn: Machine learning in P ython

    Fabian Pedregosa, Ga \"e l Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in P ython. Journal of Machine Learning Research, 12: 0 2825--2830, 2011

    2011

  42. [42]

    Fast training of support vector machines using sequential minimal optimization

    John Platt. Fast training of support vector machines using sequential minimal optimization. Advances in Kernel Methods: Support Vector Learning, pages 185--208, 1999

    1999

  43. [43]

    fastkqr: A fast algorithm for kernel quantile regression

    Qian Tang, Yuwen Gu, and Boxiang Wang. fastkqr: A fast algorithm for kernel quantile regression. Journal of Computational and Graphical Statistics, 35 0 (1): 0 395--405, 2026

    2026

  44. [44]

    Sparse distance weighted discrimination

    Boxiang Wang and Hui Zou. Sparse distance weighted discrimination. Journal of Computational and Graphical Statistics, 25 0 (3): 0 826--838, 2016

    2016

  45. [45]

    Another look at distance-weighted discrimination

    Boxiang Wang and Hui Zou. Another look at distance-weighted discrimination. Journal of the Royal Statistical Society Series B: Statistical Methodology, 80 0 (1): 0 177--198, 2018

    2018

  46. [46]

    A multicategory kernel distance weighted discrimination method for multiclass classification

    Boxiang Wang and Hui Zou. A multicategory kernel distance weighted discrimination method for multiclass classification. Technometrics, 61 0 (3): 0 396--408, 2019

    2019

  47. [47]

    Fast and exact leave-one-out analysis of large-margin classifiers

    Boxiang Wang and Hui Zou. Fast and exact leave-one-out analysis of large-margin classifiers. Technometrics, 64 0 (3): 0 291--298, 2022

    2022

  48. [48]

    Efficient multi-class probabilistic SVM s on GPU s

    Zeyi Wen, Jiashuai Shi, Bingsheng He, Jian Chen, and Yawen Chen. Efficient multi-class probabilistic SVM s on GPU s. IEEE Transactions on Knowledge and Data Engineering, 31 0 (9): 0 1693--1706, 2018 a

    2018

  49. [49]

    Thunder SVM : A fast SVM library on GPU s and CPU s

    Zeyi Wen, Jiashuai Shi, Qinbin Li, Bingsheng He, and Jian Chen. Thunder SVM : A fast SVM library on GPU s and CPU s. Journal of Machine Learning Research, 19 0 (21): 0 1--5, 2018 b

    2018