REVIEW 4 major objections 5 minor 37 references
Accelerating a Linear Programming Algorithm on AMD GPUs
T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read A PyTorch/ROCm implementation of the Primal-Dual Hybrid Gradient algorithm solves large linear programs up to 36x faster on an AMD GPU than the same solver on a CPU, with the advantage growing with problem size.
desk verdict A solid, honest engineering report on a PyTorch PDHG solver for AMD GPUs; the headline 36x speedup is a CPU-vs-GPU self-comparison, not a win over commercial CPU solvers. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
A PDHG iteration is only a matrix-vector product with the constraint matrix and its transpose, a vector add, and a projection onto a box or orthant, no factorization, hence massive parallelism. The paper makes that structure converge competitively by stacking four adaptations from the PDHG-for-LP literature: Ruiz diagonal preconditioning (balance row and column magnitudes), adaptive restarts keyed to a KKT-error measure, an adaptive step size that grows past the conservative spectral-norm bound, and primal-weight updating. Its original fishnet casting batches many PDHG trajectories as matrix-matrix multiplications, culls the worst half by duality gap, and repopulates via convex combinations,
What would settle it
Run the same Security-Constrained Economic Dispatch instance with a mature commercial CPU solver on the same EPYC processor: if it completes in well under an hour, the practical GPU-versus-CPU advantage is far smaller than 36x. Separately, run the fishnet-cast warm start on that instance 20 times and check whether the 4.5-6 minute solve times reproduce, or whether variance spans the 9.6-minute cold start.
Extended reading notes
Core claim
The paper's central claim is that PDHG, augmented with preconditioning, restarts, adaptive step sizes, and weight balancing, becomes a practical large-scale LP solver on GPUs, delivering order-of-magnitude speedups over the same solver on a CPU. The headline evidence: a Security-Constrained Economic Dispatch instance (17,682 variables, 17,706 constraints) solved to 10^-4 tolerance in 579.5 seconds on an AMD MI325X GPU versus 20,801 seconds on an AMD EPYC CPU, about 36x, with the MI325X also beating an AMD MI210 and an NVIDIA A100 on that instance and the Netlib suite. The paper also claims its original fishnet-casting warm start (spawning trajectories, culling by duality gap, recombining sur
Load-bearing premise
The headline speedup compares the GPU solver to the authors' own CPU implementation of the same PyTorch algorithm; if that CPU baseline is slower than a well-tuned professional solver, the 36x number overstates what a practitioner gains by switching to GPU.
Editorial extensions
If this is right
- If the 36x result is representative, GPU-based first-order solving becomes the faster option for large LPs, with the advantage widening as problem size grows.
- The same PyTorch code runs unchanged on CPU, AMD GPUs, and NVIDIA GPUs, so one solver can serve heterogeneous clusters without vendor lock-in.
- Integrated infeasibility detection means the solver can certify primal or dual infeasibility during the PDHG run, not as a separate costlier phase.
- Fishnet casting converts idle GPU parallelism into better starting points, implying time-to-solution gains beyond the per-iteration speedup for problems where warm starts help.
- The approach targets exactly the industrial-scale regimes, energy dispatch, finance, and logistics, where CPU solvers' latency and memory limits bite hardest.
Reading between the lines
- My read: the 36x ratio compares the GPU solver against the authors' own CPU implementation of the same algorithm; against a mature commercial CPU solver the practical gap would likely be far smaller, so the speedup is best read as a statement about parallelization headroom, not about beating the best CPU software.
- The fishnet results suggest a testable hypothesis: warm-start value grows with problem size and conditioning, which is why Netlib (small, ill-conditioned) showed no consistent gain while SCED (large) showed a strong one, a pattern the paper hints at but does not itself claim.
- The same batching machinery could solve many independent LPs simultaneously, e.g., scenario-based economic dispatch, since fishnet already maintains hundreds of parallel trajectories at once.
- The portability argument implies a direct extension: a CUDA-free, PyTorch-based LP solver could become a standard fallback for heterogeneous cloud fleets, but the paper's cross-vendor GPU comparison is a single instance and preliminary.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents torchPDLP, a PyTorch/ROCm implementation of the Primal-Dual Hybrid Gradient (PDHG) algorithm for general linear programs on AMD GPUs. It integrates several published enhancements (diagonal preconditioning, adaptive restarting, primal weight updates, adaptive step size, infeasibility detection) and introduces a stochastic multi-start heuristic called "fishnet casting." Benchmarks are reported on Netlib, MIPLIB 2017 LP relaxations, and a proprietary Security-Constrained Economic Dispatch (SCED) instance from Gridmatic, across an AMD EPYC CPU, AMD MI210/MI325X GPUs, and an NVIDIA A100. The headline result is an up-to-36x GPU-over-CPU speedup for torchPDLP on the SCED instance, with additional comparisons between the MI325X and the A100.
Significance. The engineering contribution is real: an open-source, portable PDHG LP solver that runs on AMD ROCm hardware through PyTorch, with public code and reproducible scripts. The cross-GPU comparison between AMD MI325X and NVIDIA A100 using the same codebase is useful, and the infeasibility-detection evaluation on Netlib infeasible instances is a concrete positive result. However, the significance as stated in the abstract is overstated: the 36x speedup is a self-comparison of torchPDLP on CPU versus GPU, not a comparison against established CPU solver implementations. If the claims are revised to describe the hardware-portability speedup honestly, the paper is a solid systems/engineering report; as written, the central performance claim is not supported by the experiments reported.
major comments (4)
- [Abstract; §4.4; Table 4.1] The abstract states that performance is evaluated "against established CPU-based solvers" and highlights a 36x speedup. The only CPU baseline in §4.4 is torchPDLP running on an AMD EPYC 7V13; no results against Gurobi, HiGHS, or another established CPU solver are reported. The 36x ratio in Table 4.1 is therefore the speedup of the authors' own PyTorch implementation on two hardware platforms, not a demonstration against mature CPU solvers. This is load-bearing for the central claim. Either add comparisons with established CPU solvers or revise the abstract and conclusion to claim only a GPU-over-CPU speedup for torchPDLP.
- [§4.3.5 versus §4.3.6] Section 4.3.5 states that torchPDLP incorporates all four enhancements but then says that only adaptive restarting is fully integrated by default, with the other three optional; it also says infeasibility detection and fishnet casting are "not included" because the current benchmarks are run on feasible problems. However, §4.3.6 immediately reports infeasibility-detection results on 29 Netlib infeasible instances. This internal contradiction makes it unclear which feature configuration produced the CPU/GPU benchmarks in §4.4 and Table 4.1. The text must specify the exact feature set used for each reported benchmark and reconcile the inconsistent statements.
- [§2.3, Algorithm 1; Appendix C] Algorithm 1 as printed updates x with projX(xn - tau(c + K^T y)) and y with projY(y + sigma(q + K xbar)), where K=[G;A] as defined in §2.2. The saddle-point Lagrangian in §2.2 and the derivation in Appendix C require the updates to use c - K^T y and q - K xbar (equivalently, K=-[G;A] in Appendix C's convention). As written, the main-text algorithm solves a different saddle-point problem. This is a reproducibility issue in the core algorithm statement; the sign convention should be corrected and unified with Appendix C.
- [§4.2; §4.4-4.6] Hyperparameters were tuned on a 20-problem Netlib sample at tolerance 10^-2 (Figures 4.2-4.4), then used for the final benchmarks at tolerance 10^-4. No holdout, cross-validation, or sensitivity analysis is reported, so the risk of overfitting the parameter choices to the small tuning sample is unquantified. Since the CPU/GPU speedup and the SCED timings depend on these settings, the paper should either report sensitivity of the headline numbers to the tuned parameters or explicitly state this as a limitation.
minor comments (5)
- [§2.3, Algorithm 1] Iterate subscripts are inconsistent in Algorithm 1: the left-hand side uses xt+1 while the right-hand side uses xn. Please clean up the notation.
- [§2.2 and Appendix B/C] The matrix K is defined with different signs in the main text and in Appendices B/C. The paper should include a forward reference or a small sign-convention table to avoid confusion.
- [§3.2, Figure 3.2] The architecture diagram labels "Preconditioning" and "PDHG" without clearly indicating which stages run on CPU versus GPU. The text says all performance-critical components run on the GPU, but the figure should be explicit, especially for preconditioning and termination checks.
- [§2.5 and §4.3.7] The fishnet heuristic uses r=||K||2 as a casting radius and states "we hope it contains" the feasible region. This is not a valid enclosing-radius estimate in general, and the paper admits mixed Netlib results. Please move this caveat to the main algorithmic description and quantify the failure/success cases with the data behind Figure 4.7.
- [§3.3.3 and §4.6] The SCED instance is proprietary, so the single 36x data point in Table 4.1 cannot be independently reproduced. The paper should explicitly state this limitation next to the headline result rather than only in the dataset description.
Circularity Check
No significant circularity: the algorithmic core is imported from external cited work and the performance comparisons are measured benchmarks, not derived predictions.
full rationale
The paper's central algorithmic content is explicitly imported from external, non-overlapping prior work: the PDHG update (Algorithm 1) and step-size reparameterization are attributed to Applegate et al. [7] and Chambolle–Pock [11]; adaptive restarts and primal-weight updates come from Applegate et al. [8]; infeasibility detection follows Applegate et al. [6]; the preconditioning uses Ruiz scaling [35]; presolve uses PaPILO [20]. No step in Sections 2.3–2.4 defines a quantity in terms of the quantity it is later said to predict, and no load-bearing premise is justified only by a self-citation. The fishnet heuristic (Section 2.5) is presented as an experimental, heuristic warm-start with no convergence guarantee, and the paper explicitly reports that it sometimes slows convergence and that further testing is required (Sections 4.3.7 and 5); this is an honest limitation, not a circular derivation. The headline 36x speedup (abstract; Table 4.1) is the measured ratio of torchPDLP runtime on an AMD EPYC CPU versus an AMD MI325X GPU on one SCED instance. The CPU baseline is the authors' own PyTorch implementation, and Section 4.4 is explicitly titled 'CPU vs GPU performances for torchPDLP'; no comparison against Gurobi, HiGHS, or another mature CPU solver is reported despite the abstract's phrase 'established CPU-based solvers.' This is a correctness/overclaim concern about what the speedup measures, not a circularity: the ratio is a benchmark measurement and is not constructed from the algorithm's own outputs or from a fitted parameter renamed as a prediction. The hyperparameter experiments (Section 4.2) tune a few scalar parameters on a 20-problem Netlib sample and then benchmark the tuned solver on Netlib, MIPLIB, and SCED; that is standard empirical practice and does not make the reported solve times equivalent to the tuning data by construction. The convergence proof in Appendix D is adapted from external work [11] and does not rely on the present paper's own claims. Overall, the derivation chain is self-contained with respect to circularity; the weaknesses in the performance claim are about benchmarking scope, not about circular reasoning.
Assumptions & free parameters
free parameters (8)
- Extrapolation parameter θ =
1
- Step size multiplier for η =
0.9
- Primal weight ω =
||c||2/||q||2
- Restart parameters β_sufficient, β_necessary, β_artificial =
0.2, 0.8, 0.36
- Fishnet initial point count =
32
- Fishnet cull fraction =
0.5
- Power iteration count for spectral norm
- Fishnet inner loop iterations k
assumptions (5)
- standard math A saddle point (x*, y*) exists for the LP saddle-point formulation
- standard math The step size condition τσ||K'||² < 1 ensures convergence
- domain assumption The benchmark LP instances are feasible and bounded
- domain assumption The power iteration estimate accurately approximates ||K||2
- ad hoc to paper The spectral radius r = ||K||2 contains the primal feasible region for fishnet casting
Cite this review
Pith. "Pith review of Accelerating a Linear Programming Algorithm on AMD GPUs." pith.science (2026). https://pith.science/paper/THSIVLYL
@misc{pith2026250816806,
author = {Pith},
title = {Pith review of: Accelerating a Linear Programming Algorithm on AMD GPUs},
year = {2026},
howpublished = {\url{https://pith.science/paper/THSIVLYL}},
note = {Machine review of arXiv:2508.16806}
}
read the original abstract
Linear Programming (LP) is a foundational optimization technique with widespread applications in finance, energy trading, and supply chain logistics. However, traditional Central Processing Unit (CPU)-based LP solvers often struggle to meet the latency and scalability demands of dynamic, high-dimensional industrial environments, creating a significant computational challenge. This project addresses these limitations by accelerating linear programming on AMD Graphics Processing Units (GPUs), leveraging the ROCm open-source platform and PyTorch. The core of this work is the development of a robust, high-performance, open-source implementation of the Primal-Dual Hybrid Gradient (PDHG) algorithm, engineered specifically for general LP problems on AMD hardware. Performance is evaluated against standard LP test sets and established CPU-based solvers, with a particular focus on challenging real- world instances including the Security-Constrained Economic Dispatch (SCED) to guide hyperparameter tuning. Our results show a significant improvement, with up to a 36x speedup on GPU over CPU for large-scale problems, highlighting the advantages of GPU acceleration in solving complex optimization tasks.
Figures
Figures from the paper (10 more)
Reference graph
Works this paper leans on
- [1]
-
[2]
Netlib lp collection - infeasible problems . Accessed: 2025-08-15
work page 2025
-
[3]
Advanced Micro Devices, AMD EPYC ™ 7003 Series Processors, 2022
work page 2022
-
[4]
, AMD Instinct ™ MI210 Accelerator, 2022
work page 2022
-
[5]
, AMD Instinct ™ MI325X Accelerators, 2024
work page 2024
-
[6]
Infeasibility detection with primal-dual hybrid gradient for large-scale linear programming
D. Applegate, M. D ´ıaz, H. Lu, and M. Lubin , Infeasibility detection with primal-dual hybrid gradient for large-scale linear programming , arXiv preprint arXiv:2102.04592, (2021)
work page Pith review arXiv 2021
-
[7]
D. Applegate, M. D ´ıaz, O. Hinder, H. Lu, M. Lubin, B. O’Donoghue, and W. Schudy, Practical large-scale linear programming using primal-dual hybrid gradi- ent, 2022
work page 2022
-
[8]
D. Applegate, O. Hinder, H. Lu, and M. Lubin , Faster first-order primal-dual methods for linear programming using restarts and sharpness , Mathematical Program- ming, 201 (2022), p. 133–184
work page 2022
Show all 37 references
-
[9]
R. H. Bartels and G. H. Golub , The simplex method of linear programming using lu decomposition, Communications of the ACM, 12 (1969), pp. 266–268
1969
-
[10]
Boyd and L
S. Boyd and L. V andenberghe, Convex Optimization, Cambridge University Press, March 2004
2004
-
[11]
Chambolle and T
A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging , Journal of mathematical imaging and vision, 40 (2011), pp. 120–145
2011
-
[12]
K. Chen, D. Sun, Y. Yuan, G. Zhang, and X. Zhao , Hpr-lp: An implementation of an hpr method for solving linear programming , 2025
2025
-
[13]
G. B. Dantzig, Programming in a linear structure, in Bulletin of the American Math- ematical Society, vol. 54, AMER MATHEMATICAL SOC 201 CHARLES ST, PROV- IDENCE, RI 02940-2213, 1948, pp. 1074–1074
1948
-
[14]
, Linear programming and extensions , Princeton university press, 2016
2016
-
[15]
I. I. Dikin , Iterative solution of problems of linear and quadratic programming , in Soviet Math. Dokl., vol. 8, 1967, pp. 674–675. 47
1967
-
[16]
A. V. Fiacco and G. P. McCormick , Computational algorithm for the sequen- tial unconstrained minimization technique for nonlinear programming , Management science, 10 (1964), pp. 601–617
1964
-
[17]
J. J. Forrest and D. Goldfarb , Steepest-edge simplex algorithms for linear pro- gramming, Mathematical programming, 57 (1992), pp. 341–374
1992
-
[18]
D. M. Gay, netlib/fp
-
[19]
P. E. Gill, W. Murray, and M. H. Wright , Numerical Linear Algebra and Op- timization, Society for Industrial and Applied Mathematics, Philadelphia, PA, 2021
2021
-
[20]
Gleixner, L
A. Gleixner, L. Gottwald, and A. Hoen , PaPILO: A parallel presolving library for integer and linear programming with multiprecision support , INFORMS Journal on Computing, (2023)
2023
-
[21]
Gleixner, G
A. Gleixner, G. Hendel, G. Gamrath, T. Achterberg, M. Bastubbe, T. Berthold, P. Christophel, K. Jarck, T. Koch, J. Linderoth, et al. , Miplib 2017: data-driven compilation of the 6th mixed-integer programming library , Mathematical Programming Computation, 13 (2021), pp. 443–490
2017
-
[22]
G. H. Golub and C. F. V an Loan, Matrix Computations, Johns Hopkins University Press, 4th ed., 2013
2013
-
[23]
Henk, L¨ owner-john ellipsoids, Documenta Math, 95 (2012), p
M. Henk, L¨ owner-john ellipsoids, Documenta Math, 95 (2012), p. 4
2012
-
[24]
X. Hu, T. Parker, C. Phillips, and Y. Yu , Pdlp-amd-rips. https://github. com/SimplySnap/PDLP-AMD-RIPS, 2025. Accessed: 2025-08-15
2025
-
[25]
Karmarkar , A new polynomial-time algorithm for linear programming , in Pro- ceedings of the sixteenth annual ACM symposium on Theory of computing, 1984, pp
N. Karmarkar , A new polynomial-time algorithm for linear programming , in Pro- ceedings of the sixteenth annual ACM symposium on Theory of computing, 1984, pp. 302–311
1984
-
[26]
Lu and J
H. Lu and J. Yang, cupdlp.jl: A gpu implementation of restarted primal-dual hybrid gradient for linear programming in julia , arXiv preprint arXiv:2311.12180, (2023)
2023 arXiv
-
[27]
Lu and J
H. Lu and J. Yang , Restarted halpern pdhg for linear programming , 2024
2024
-
[28]
, An overview of gpu-based first-order methods for linear programming and exten- sions, 2025
2025
-
[29]
H. Lu, J. Yang, H. Hu, Q. Huangfu, J. Liu, T. Liu, Y. Ye, C. Zhang, and D. Ge, cupdlp-c: A strengthened implementation of cupdlp for linear programming by c language , arXiv preprint arXiv:2312.14832, (2023)
2023 arXiv
-
[30]
Manning, R
T. Manning, R. D. Sleator, and P. W alsh , Naturally selecting solutions , Bio- engineered, 4 (2013), pp. 266–278. PMID: 23222169
2013
-
[31]
Mart´ı, M
R. Mart´ı, M. G. Resende, and C. C. Ribeiro , Multi-start methods for combina- torial optimization , European Journal of Operational Research, 226 (2013), pp. 1–8
2013
-
[32]
NVIDIA, NVIDIA A100 Tensor Core GPU Datasheet , 2020
2020
-
[33]
Ord ´onez and R
F. Ord ´onez and R. M. Freund , Computational experience and the explanatory value of condition measures for linear optimization , SIAM Journal on Optimization, 14 (2003), pp. 307–333. 48
2003
-
[34]
R. T. Rockafellar , Convex Analysis , vol. 28 of Princeton Mathematical Series, Princeton University Press, Princeton, NJ, 1970
1970
-
[35]
Ruiz, A scaling algorithm to equilibrate both rows and columns norms in matrices , tech
D. Ruiz, A scaling algorithm to equilibrate both rows and columns norms in matrices , tech. rep., CM-P00040415, 2001
2001
-
[36]
S. J. Wright , Primal-dual interior-point methods , SIAM, 1997
1997
-
[37]
S. J. Wright, Modified cholesky factorizations in interior-point algorithms for linear programming, SIAM Journal on Optimization, 9 (1999), pp. 1159–1191. 49 Appendix A Convex Analysis Background In this appendix, we summarize several foundational concepts from convex analysis...
1999
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.