REVIEW 4 major objections 6 minor 1 cited by
Accelerating Bidiagonalization of Banded Matrices through Memory-Aware Bulge-Chasing on GPUs
T0 review · 4 major / 6 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read A GPU-resident bulge-chasing kernel reduces banded matrices to bidiagonal form, beating multicore CPU libraries by over 100× at 32k.
desk verdict A genuine first GPU implementation of band-to-bidiagonal reduction with impressive speedups, but the correctness argument has a gap that needs fixing before the results are fully trustworthy. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the bulge: a nonzero created when a Householder transformation annihilates an element, which must be chased out of the band by further transformations. The mechanism is Algorithm 1's row-sweep scheduling: each sweep chases bulges down the diagonal, while a three-cycle separation between sweeps is claimed to prevent overlapping memory access and permit parallel execution. Around this, bandwidth tiling (reducing the bandwidth by an inner tilewidth per stage), a kernel that reuses shared memory for reflector formation and registers for row chunks, and a tunable Max-blocks occupancy limit convert the memory-bound kernel into a cache-local one. The paper's occupancy con
What would settle it
On a small banded matrix, accumulate the Householder transformations produced by the algorithm into matrices Q and Z and check whether Q^T A Z is bidiagonal to machine precision and Q^T Q and Z^T Z equal the identity. Any out-of-band entry that remains nonzero, or any loss of orthogonality, would falsify the claim that the reduction is an orthogonal reduction to bidiagonal form.
Extended reading notes
Core claim
The central claim is that the banded-to-bidiagonal reduction—historically treated as too data-movement-bound for accelerators—can be made GPU-resident and fast by combining three ideas: successive reduction of the bandwidth in tiles, row-sweep bulge chasing in which consecutive sweeps are separated by a three-cycle gap to avoid data races, and a kernel that computes Householder reflectors in shared memory but applies them through register-resident row chunks. The paper reports that the implementation outperforms multithreaded CPU libraries starting at 1024×1024 matrices and reaches over 100× speedup at 32k×32k, with accuracy validated on the resulting singular values.
Load-bearing premise
The claim depends on the three-cycle separation between row sweeps guaranteeing that all bulges are chased with no hidden nonzeros—a property the paper does not directly test, since it validates only the computed singular values, not the bidiagonal structure or orthogonality of the transformations.
Editorial extensions
If this is right
- Two-stage SVD can run fully on the GPU, removing CPU fallback and host-device sync from the middle stage.
- SVD pipelines can choose larger intermediate bandwidths without penalty, since the second stage now scales linearly with bandwidth.
- One hardware- and precision-agnostic kernel covers FP16, FP32, and FP64 across GPU vendors, avoiding vendor-specific rewrites.
- Memory-bound kernels in numerical linear algebra can be made competitive on GPUs when L1/L2 cache size and latency, not DRAM throughput alone, drive kernel design.
Reading between the lines
- If the reported speedups generalize, one-stage GPU SVD implementations that bypass the banded intermediate could be revisited, since the old reason for avoiding it—slow second stage—would no longer apply.
- The strong reported dependence on cache latency rather than cache capacity suggests a concrete hardware design target: future GPU memory hierarchies should optimize L1/L2 latency for memory-bound kernels, not just bandwidth.
- The algorithm's per-bulge kernel launches imply a synchronization cost that might be amortized differently on future architectures, so co-designing the scheduling into persistent kernels or cooperative groups is a plausible next step.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a GPU-resident algorithm for reducing a banded matrix to bidiagonal form by bulge-chasing, implemented with Julia's array abstractions and KernelAbstractions.jl so that one high-level kernel runs on NVIDIA, AMD, Intel, and Apple GPUs in FP16, FP32, and FP64. The authors claim this is the first fully GPU-resident band-to-bidiagonal reduction, that it outperforms PLASMA and SLATE starting at matrix sizes 1024x1024 and reaches over 100x speedup at 32k x 32k, and that its performance scales linearly with matrix bandwidth. The paper contributes a memory-aware kernel design, a bandwidth-tiling strategy for large bandwidths, a hardware-aware occupancy model, and a cross-vendor performance study.
Significance. If the correctness and performance claims hold, the paper addresses a recognized gap in two-stage SVD pipelines: the banded-to-bidiagonal stage has remained the last CPU-only stage on GPUs, and this work would remove that bottleneck. The single-source portable implementation across four GPU vendors and three precisions is a substantial engineering contribution, and the bandwidth-tiling strategy for larger matrix bandwidths could shift the practical trade-off between the first and second SVD stages. However, the central algorithmic correctness premise — that the parallel sweeps are race-free and that all bulges are annihilated so that the output is genuinely bidiagonal — is not directly verified; the current accuracy test only checks singular values through a solver that cannot see entries outside the bidiagonal storage. That premise must be established before the performance results can be fully credited.
major comments (4)
- [Algorithm 1; Section IV-B, Eq. (1)] Algorithm 1 cannot be the implemented schedule as written. (i) The inner loop (line 5) has no termination test; line 6 only guards the body, so when k>n the loop increments j forever. (ii) The separation rule is stated inconsistently: line 6 uses a fixed lag of three iterations, 3(R-1)<j, while Section IV-B and Eq. (1) state that the spacing between bulge-chasing blocks is 3*CBW. Since k advances by (TW+BW)*i (line 10), the coordinate separation produced by the line-6 lag depends on the stage index i, so Eq. (1) does not follow from the pseudocode. (iii) The bulge positions illustrated in Fig. 2 (k=3,7,10 for TW=2, BW=6) are not generated by line 10 for any consistent value of i. Please rewrite the pseudocode to match the implementation, add an explicit termination condition, and supply a correctness argument — or direct empirical evidence — that the separation makes concurrent sweeps ac
- [Section VI-A, Fig. 3] The numerical accuracy test is indirect. The authors form A=U*Sigma*V^T, reduce it to banded form, apply the GPU bidiagonal reduction, and then call LAPACK BDSDC. BDSDC reads only the stored bidiagonal diagonals; any nonzero entries left outside those diagonals by a missed or dropped bulge are invisible to the solver. Thus the reported singular-value errors cannot detect the failure modes most relevant to the paper's correctness claim: a silently dropped bulge or non-orthogonal accumulated transformations. Please add direct checks on the output of the reduction: (a) audit the zero pattern by measuring max |B_ij| outside the bidiagonal structure for a range of n, bandwidth, and precision; (b) if transformations are accumulated, verify orthogonality and the residual ||A - Q_left*B*Q_right^T|| / ||A||; or (c) compare singular values of the full reduced matrix stored as a dense banded matrix
- [Section IV-B, Eq. (1), Table I] The occupancy model is asserted without derivation. The number of concurrently active bulge-chasing blocks should follow from the sweep-separation rule, but with the line-6 lag of Algorithm 1 that number depends on n and on the stage-dependent stride, not simply on n/(3*CBW). Equation (1) and the minimum matrix sizes in Table I are therefore unsupported unless the link between the scheduling rule and the formula is provided. Please derive Eq. (1) from the actual implemented schedule, or revise the model and the conclusions drawn from it in Section VI-D.
- [Section IV-A, Algorithm 2] The kernel pseudocode omits the handling of boundaries and near-zero elements and defers details of the Householder reflector computation to reference [64]. That reference concerns tile-QR, not the bulge-chasing schedule used here, so it cannot justify the central race-freedom claim. In particular, Algorithm 2's column-annihilation phase (line 15) is described only as 'the procedure above', with no statement of how a column sweep interacts with simultaneously executing row sweeps. Please make the kernel pseudocode self-contained enough to check that two concurrent sweeps never touch overlapping rows or columns, or provide a formal/empirical race-freedom argument.
minor comments (6)
- [Section I] There is a sentence fragment near the middle of the introduction: 'in modern GPUs such as NVIDIA's Hopper and AMD's MI300X.' Please revise for readability.
- [Algorithm 1] The loop header 'j=0, j+=1' is unusual; please clarify whether j starts at 0 or 1 and whether the condition 3(R-1)<j is intended to be strict or non-strict.
- [Algorithm 2, line 10] The symbol 'CPB' is undefined; the row index r=k+l*CPB+i appears to be a typo for a quantity derived from TPB or CBW. Please define it or correct the expression.
- [Section VI-D, Fig. 6] Please state clearly whether PLASMA and SLATE timings include only the banded-to-bidiagonal reduction or also the dense-to-banded first stage, and describe the input storage/layout used for the CPU baselines. Without this, the comparison is difficult to interpret.
- [Open-source claim] The paper states the implementation is open-source but does not provide a repository URL or version identifier. Please add an artifact link.
- [Table II] The header contains apparent LaTeX spacing artifacts ('TH R E A D S P E R B L O C K', 'MA X B L O C K S', 'IN N E R T I L E W I D T H'). Please fix the header text.
Circularity Check
No significant circularity: performance claims are empirically self-contained; the only self-citation (ref [64]) is a non-load-bearing implementation detail.
full rationale
The paper's central claims are engineering/performance claims, not derivations whose conclusions are encoded in their inputs. The speedups vs PLASMA/SLATE are measured against external libraries, and the accuracy test starts from matrices with known singular values, applies the pipeline, and compares to the ground-truth Σ; nothing in that loop assumes the conclusion. The occupancy model in §IV-B (Eq. 1) is derived from the algorithm's own scheduling rule (spacing 3·CBW) and hardware ALU count, and is used to explain observed scaling, not to generate a result by fitting to the benchmark data; it is therefore not a fitted-input-called-prediction. The one self-citation, ref [64], supplies Householder-reflector implementation details and a prior numerical-fidelity claim; it is a standard numerical building block and is exercised end-to-end in the accuracy tests, so it is not load-bearing for the paper's central contribution. The manuscript has a genuine correctness-validation gap—it verifies singular values after BDSDC but not bidiagonality or orthogonality, and the Algorithm 1 / Eq. 1 sweep-spacing statements are not reconciled—but that is an omitted-proof/correctness risk, not circular reasoning. Accordingly, no circular step is identified.
Assumptions & free parameters
free parameters (4)
- inner tilewidth (TW) =
32 (FP32) / 16 (FP64)
- max blocks =
varies (e.g., 96-192 on RTX4060)
- threads per block (TPB) =
varies (e.g., 16-64)
- three-cycle separation factor (3) =
3
assumptions (4)
- standard math Householder reflections preserve singular values of the input matrix.
- ad hoc to paper The three-cycle separation between row sweeps is sufficient to prevent overlapping data accesses and missed bulges.
- domain assumption Synthetic matrices A = U Σ Vᵀ with prescribed singular values are representative for accuracy benchmarking.
- domain assumption The CPU baseline libraries PLASMA and SLATE were run in a state-of-the-art configuration.
Cite this review
Pith. "Pith review of Accelerating Bidiagonalization of Banded Matrices through Memory-Aware Bulge-Chasing on GPUs." pith.science (2026). https://pith.science/paper/O3TYZVKA
@misc{pith2026251012705,
author = {Pith},
title = {Pith review of: Accelerating Bidiagonalization of Banded Matrices through Memory-Aware Bulge-Chasing on GPUs},
year = {2026},
howpublished = {\url{https://pith.science/paper/O3TYZVKA}},
note = {Machine review of arXiv:2510.12705}
}
read the original abstract
The reduction of a banded matrix to bidiagonal form is a critical step in the calculation of Singular Values, a cornerstone of scientific computing and AI. Although inherently parallel, this step has traditionally been considered unsuitable for GPUs due to its memory-bound nature. However, recent advances in GPU architectures, such as increased L1 memory per Streaming Multiprocessor or Compute Unit and larger L2 caches, have shifted this paradigm. In this work, we present the first GPU-accelerated algorithm for reducing a banded matrix to bidiagonal form, integrated into an open-source software package. Our algorithm builds on prior multicore CPU cache-efficient bulge-chasing methods, adapted to modern GPU architectures to optimize throughput. Leveraging Julia's high-level array abstractions and KernelAbstractions.jl, we implement a single function that is both hardware-agnostic and data-precision-aware, running efficiently across NVIDIA, AMD, Intel, and Apple Metal GPUs. We develop a hardware-aware performance model to guide tuning and identify key hyperparameters that govern optimal GPU performance for memory-bound workloads. We show that such workloads, when carefully optimized, can achieve substantial speed-ups on modern GPUs: our implementation outperforms multithreaded CPU libraries (PLASMA,SLATE) starting from matrix sizes as small as 1024x1024, and achieves over 100x speed-up on 32k x 32k matrices. Moreover, the algorithm's performance scales linearly with the matrix bandwidth, enabling efficient reduction of matrices with larger bandwidths, previously considered impractical.
Figures
Figures from the paper (5 more)
Forward citations
Cited by 1 Pith paper
-
Hierarchical Recursive Precision for Accelerating Symmetric Linear Solves on MXUs
A tree-structured recursive Cholesky solver assigns FP16 to off-diagonal blocks and higher precision to diagonal blocks, achieving over 5x speedup on NVIDIA H200 and AMD MI300X with better accuracy than pure half precision.
Reference graph
Works this paper leans on
-
[64]
E. Ringoot, R. Alomairy, V . Churavy, and A. Edelman, “Performant Unified GPU Kernels for Portable Singular Value Computation Across Hardware and Precision,”arXiv preprint arXiv:2508.06339, 2025
arXiv 2025
-
[1]
Unveiling lora intrinsic ranks via salience analysis,
W. Ke, J. Wang, P. Wang, J. Liu, D. Nie, G. Li, and Y . Li, “Unveiling lora intrinsic ranks via salience analysis,” inAdvances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, Eds., vol. 37. Curran Associates, Inc., 2024, pp. 131 575–131 595. [Online]. Available: https://proceedin...
2024
-
[2]
Svft: Parameter-efficient fine-tuning with singular vectors,
V . Lingam, A. Tejaswi, A. Vavre, A. Shetty, G. K. Gudur, J. Ghosh, A. Dimakis, E. Choi, A. Bojchevski, and S. Sanghavi, “Svft: Parameter-efficient fine-tuning with singular vectors,” inAdvances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, Eds., vol. 37. Curran Associates, Inc...
2024
-
[3]
Lora: Low-rank adaptation of large language models
E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chenet al., “Lora: Low-rank adaptation of large language models.” ICLR, vol. 1, no. 2, p. 3, 2022
2022
-
[4]
The extraordinary svd,
C. D. Martin and M. A. Porter, “The extraordinary svd,”The American Mathematical Monthly, vol. 119, no. 10, pp. 838–851, 2012
2012
-
[5]
Diagnosing non- hermitian many-body localization and quantum chaos via singular value decomposition,
F. Roccati, F. Balducci, R. Shir, and A. Chenu, “Diagnosing non- hermitian many-body localization and quantum chaos via singular value decomposition,”Phys. Rev. B, vol. 109, p. L140201, Apr 2024. [Online]. Available: https://link.aps.org/doi/10.1103/PhysRevB.109.L140201
-
[6]
The singular value decomposition: Anatomy of optimizing an algorithm for extreme scale,
J. Dongarra, M. Gates, A. Haidar, J. Kurzak, P. Luszczek, S. Tomov, and I. Yamazaki, “The singular value decomposition: Anatomy of optimizing an algorithm for extreme scale,”SIAM Review, vol. 60, no. 4, pp. 808– 865, 2018. [Online]. Available: https://doi.org/10.1137/17M1117732
-
[7]
Fast tridiagonal solvers on the gpu,
Y . Zhang, J. Cohen, and J. D. Owens, “Fast tridiagonal solvers on the gpu,” inProceedings of the 15th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, ser. PPoPP ’10. New York, NY , USA: Association for Computing Machinery, 2010, p. 127–136. [Online]. Available: https://doi.org/10.1145/1693453.1693472
arXiv 2010
Show all 93 references
-
[8]
High performance solution of tridiagonal systems on the gpu,
D. Tolmachev, P. Marti, G. Castiglioni, and A. Jackson, “High performance solution of tridiagonal systems on the gpu,”ACM Trans. Parallel Comput., vol. 12, no. 2, May 2025. [Online]. Available: https://doi.org/10.1145/3716171
2025 doi
-
[9]
A scalable, numerically stable, high-performance tridiagonal solver using gpus,
L.-W. Chang, J. A. Stratton, H.-S. Kim, and W.-M. W. Hwu, “A scalable, numerically stable, high-performance tridiagonal solver using gpus,” inSC ’12: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, 2012, pp. 1–11
2012
-
[10]
A divide-and-conquer approach for solving singular value decomposition on a heterogeneous system,
D. Liu, R. Li, D. J. Lilja, and W. Xiao, “A divide-and-conquer approach for solving singular value decomposition on a heterogeneous system,” inProceedings of the ACM International Conference on Computing Frontiers, ser. CF ’13. New York, NY , USA: Association for Computing Mac...
2013
-
[11]
Accelerating numerical dense linear algebra calculations with gpus,
J. Dongarra, M. Gates, A. Haidar, J. Kurzak, P. Luszczek, S. Tomov, and I. Yamazaki, “Accelerating numerical dense linear algebra calculations with gpus,” inNumerical computations with GPUs. Springer, 2014, pp. 3–28
2014
-
[12]
Ponte vecchio across the atlantic: Single-node benchmarking of two intel gpu systems,
T. Applencourt, A. Sadawarte, S. Muralidharan, C. Bertoni, J. Kwack, Y . Luo, E. Rangel, J. Tramm, Y . Ghadar, A. Tameruset al., “Ponte vecchio across the atlantic: Single-node benchmarking of two intel gpu systems,” inSC24-W: Workshops of the International Conference for High...
2024
-
[13]
Effective Extensible Program- ming: Unleashing Julia on GPUs,
T. Besard, C. Foket, and B. De Sutter, “Effective Extensible Program- ming: Unleashing Julia on GPUs,”IEEE Transactions on Parallel and Distributed Systems, vol. 30, no. 4, pp. 827–841, 2019
2019
-
[14]
Gpuarrays.jl,
T. B. Simeon Danisch, “Gpuarrays.jl,” https://github.com/JuliaGPU/GPUArrays.jl, 2023
2023
-
[15]
Kernelabstractions.jl,
V . Churavy, “Kernelabstractions.jl,” https://github.com/JuliaGPU/ Ker- nelAbstractions.jl, 2023
2023
-
[16]
PLASMA: Parallel Linear Algebra Software for Multicore Using OpenMP,
J. Dongarra, M. Gates, A. Haidar, J. Kurzak, P. Luszczek, P. Wu, I. Ya- mazaki, A. YarKhan, M. Abalenkovs, N. Bagherpouret al., “PLASMA: Parallel Linear Algebra Software for Multicore Using OpenMP,”ACM Transactions on Mathematical Software (TOMS), vol. 45, no. 2, pp. 1– 35, 2019
2019
-
[17]
Accelerating the SVD Two Stage Bidiagonal Reduction and Divide and Conquer Using GPUs,
M. Gates, S. Tomov, and J. Dongarra, “Accelerating the SVD Two Stage Bidiagonal Reduction and Divide and Conquer Using GPUs,”Parallel Computing, vol. 74, pp. 3–18, 2018
2018
-
[18]
Communication Avoiding Successive Band Reduction,
G. Ballard, J. Demmel, and N. Knight, “Communication Avoiding Successive Band Reduction,”ACM SIGPLAN Notices, vol. 47, no. 8, pp. 35–44, 2012
2012
-
[19]
Families of algorithms for reducing a matrix to condensed form,
F. G. Van Zee, R. A. van de Geijn, G. Quintana-Ort ´ı, and G. J. Elizondo, “Families of algorithms for reducing a matrix to condensed form,”ACM Trans. Math. Softw., vol. 39, no. 1, Nov. 2012. [Online]. Available: https://doi.org/10.1145/2382585.2382587
2012
-
[20]
Parallel Reduction to Con- densed Forms for Symmetric Eigenvalue Problems Using Aggregated Fine-Grained and Memory-Aware Kernels,
A. Haidar, H. Ltaief, and J. Dongarra, “Parallel Reduction to Con- densed Forms for Symmetric Eigenvalue Problems Using Aggregated Fine-Grained and Memory-Aware Kernels,” inProceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Ana...
2011
-
[21]
Algorithm 807: The sbr toolbox—software for successive band reduction,
C. H. Bischof, B. Lang, and X. Sun, “Algorithm 807: The sbr toolbox—software for successive band reduction,”ACM Trans. Math. Softw., vol. 26, no. 4, p. 602–616, Dec. 2000. [Online]. Available: https://doi.org/10.1145/365723.365736
-
[22]
Restructuring the tridiagonal and bidiagonal qr algorithms for performance,
F. G. Van Zee, R. A. van de Geijn, and G. Quintana-Ort ´ı, “Restructuring the tridiagonal and bidiagonal qr algorithms for performance,”ACM Trans. Math. Softw., vol. 40, no. 3, Apr. 2014. [Online]. Available: https://doi.org/10.1145/2535371
2014 doi
-
[23]
The elpa library: scalable parallel eigenvalue solutions for electronic structure theory and compu- tational science,
A. Marek, V . Blum, R. Johanni, V . Havu, B. Lang, T. Auckenthaler, A. Heinecke, H.-J. Bungartz, and H. Lederer, “The elpa library: scalable parallel eigenvalue solutions for electronic structure theory and compu- tational science,”Journal of Physics: Condensed Matter, vol. 26...
2014
-
[24]
Reduction to band form for the singular value decomposition on graphics accelerators,
A. E. Tom ´as, R. Rodr ´ıguez-S´anchez, S. Catal ´an, and E. S. Quintana- Ort´ı, “Reduction to band form for the singular value decomposition on graphics accelerators,” inProceedings of the 9th International Workshop on Programming Models and Applications for Multicores and Ma...
2018
-
[25]
A novel hybrid cpu–gpu generalized eigensolver for electronic structure calcu- lations based on fine-grained memory aware tasks,
A. Haidar, S. Tomov, J. Dongarra, R. Solca, and T. Schulthess, “A novel hybrid cpu–gpu generalized eigensolver for electronic structure calcu- lations based on fine-grained memory aware tasks,”The International journal of high performance computing applications, vol. 28, no. 2...
2014
-
[26]
Accelerating the reduction to upper hessenberg, tridiagonal, and bidiagonal forms through hybrid gpu-based computing,
S. Tomov, R. Nath, and J. Dongarra, “Accelerating the reduction to upper hessenberg, tridiagonal, and bidiagonal forms through hybrid gpu-based computing,”Parallel Computing, vol. 36, no. 12, pp. 645–654, 2010. [Online]. Available: https://www.sciencedirect.com/science/article...
2010
-
[27]
Evolution of the SLATE Linear Algebra Library,
M. Gates, A. Abdelfattah, K. Akbudak, M. Al Farhan, R. Alomairy, D. Bielich, T. Burgess, S. Cayrols, N. Lindquist, D. Sukkariet al., “Evolution of the SLATE Linear Algebra Library,”The International Journal of High Performance Computing Applications, vol. 39, no. 1, pp. 3–17, 2025
2025
-
[28]
MAGMA: Enabling exascale performance with accelerated BLAS and LAPACK for diverse GPU ar- chitectures,
A. Abdelfattah, N. Beams, R. Carson, P. Ghysels, T. Kolev, T. Stitt, A. Vargas, S. Tomov, and J. Dongarra, “MAGMA: Enabling exascale performance with accelerated BLAS and LAPACK for diverse GPU ar- chitectures,”The International Journal of High Performance Computing Applicatio...
2024
-
[29]
rocsolver,
C. Bloor, J. Zuniga-Anaya, T. Alderso, and all, “rocsolver,” AMD,
-
[30]
Singular value decomposition on gpu using cuda,
S. Lahabar and P. J. Narayanan, “Singular value decomposition on gpu using cuda,” in2009 IEEE International Symposium on Parallel and Distributed Processing, 2009, pp. 1–10
2009
-
[31]
Gpu-based parallel householder bidiagonalization,
F. Liu and F. J. Seinstra, “Gpu-based parallel householder bidiagonalization,” inProceedings of the 19th ACM International Symposium on High Performance Distributed Computing, ser. HPDC ’10. New York, NY , USA: Association for Computing Machinery, 2010, p. 288–291. [Online]. A...
2010
-
[32]
Efficient gpu-centered singular value decomposition using the divide-and-conquer method,
S. Liu, H. Li, H. Sheng, H. Gui, and X. Zhang, “Efficient gpu-centered singular value decomposition using the divide-and-conquer method,” ACM Transactions on Architecture and Code Optimization, 2025
2025
-
[33]
Gpu- acceleration of the elpa2 distributed eigensolver for dense symmetric and hermitian eigenproblems,
V . W. zhe Yu, J. Moussa, P. K ˚us, A. Marek, P. Messmer, M. Yoon, H. Lederer, and V . Blum, “Gpu- acceleration of the elpa2 distributed eigensolver for dense symmetric and hermitian eigenproblems,”Computer Physics Communications, vol. 262, p. 107808, 2021. [Online]. Available...
2021
-
[34]
Eigen-g: Gpu-based eigen- value solver for real-symmetric dense matrices,
T. Imamura, S. Yamada, and M. Machida, “Eigen-g: Gpu-based eigen- value solver for real-symmetric dense matrices,” inParallel Processing and Applied Mathematics, R. Wyrzykowski, J. Dongarra, K. Karczewski, and J. Wa´sniewski, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg...
2014
-
[35]
Task-based, gpu- accelerated and robust library for solving dense nonsymmetric eigenvalue problems,
M. Myllykoski and C. C. Kjelgaard Mikkelsen, “Task-based, gpu- accelerated and robust library for solving dense nonsymmetric eigenvalue problems,”Concurrency and Computation: Practice and Experience, vol. 33, no. 11, p. e5915, 2021. [Online]. Available: https://onlinelibrary.w...
2021 doi
-
[36]
Introduction to starneig—a task-based library for solving non- symmetric eigenvalue problems,
——, “Introduction to starneig—a task-based library for solving non- symmetric eigenvalue problems,” inParallel Processing and Applied Mathematics, R. Wyrzykowski, E. Deelman, J. Dongarra, and K. Kar- czewski, Eds. Cham: Springer International Publishing, 2020, pp. 70– 81
2020
-
[37]
Toward a high performance tile divide and conquer algorithm for the dense symmetric eigenvalue problem,
A. Haidar, H. Ltaief, and J. Dongarra, “Toward a high performance tile divide and conquer algorithm for the dense symmetric eigenvalue problem,”SIAM Journal on Scientific Computing, vol. 34, no. 6, pp. C249–C274, 2012. [Online]. Available: https://doi.org/10.1137/110823699
2012 doi
-
[38]
cusolver,
NVIDIA, “cusolver,” NVIDIA, 2025. [Online]. Available: https://developer.nvidia.com/cusolver
2025
-
[39]
An efficient 2d fusion method for high-performance two-stage eigensolvers on modern heterogeneous architectures,
Y . Zhou, Y . Zong, Y . Jin, H. Li, and W. Xue, “An efficient 2d fusion method for high-performance two-stage eigensolvers on modern heterogeneous architectures,” inProceedings of the 39th ACM International Conference on Supercomputing, ser. ICS ’25. New York, NY , USA: Associ...
2025
-
[40]
Improving tridiagonalization performance on gpu architectures,
H. Wang, Z. Duan, Z. Zhao, S. Wu, S. Zheng, Q. Li, X. Jiang, and S. Zhang, “Improving tridiagonalization performance on gpu architectures,” inProceedings of the 30th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming, ser. PPoPP ’25. New York, NY ,...
2025
-
[41]
Cuda programming guide,
NVIDIA Corporation, “Cuda programming guide,” accessed: 2025-10-08. [Online]. Available: https://docs.nvidia.com/cuda/cuda- c-programming-guide/
2025
-
[42]
Zhang, personal email communication
S. Zhang, personal email communication
-
[43]
High-Performance Bidiagonal Reduction Using Tile Algorithms on Homogeneous Multicore Architec- tures,
H. Ltaief, P. Luszczek, and J. Dongarra, “High-Performance Bidiagonal Reduction Using Tile Algorithms on Homogeneous Multicore Architec- tures,”ACM Transactions on Mathematical Software (TOMS), vol. 39, no. 3, pp. 1–22, 2013
2013
-
[44]
Divide and conquer symmetric tridiagonal eigensolver for multicore architectures,
G. Pichon, A. Haidar, M. Faverge, and J. Kurzak, “Divide and conquer symmetric tridiagonal eigensolver for multicore architectures,” in2015 IEEE International Parallel and Distributed Processing Symposium, 2015, pp. 51–60
2015
-
[45]
Divide and conquer on hybrid gpu-accelerated multicore systems,
C. V ¨omel, S. Tomov, and J. Dongarra, “Divide and conquer on hybrid gpu-accelerated multicore systems,”SIAM Journal on Scientific Com- puting, vol. 34, no. 2, pp. C70–C82, 2012
2012
-
[46]
Gpu-acceleration of the elpa2 distributed eigensolver for dense symmetric and hermitian eigenproblems,
V . W.-z. Yu, J. Moussa, P. Kuus, A. Marek, P. Messmer, M. Yoon, H. Lederer, and V . Blum, “Gpu-acceleration of the elpa2 distributed eigensolver for dense symmetric and hermitian eigenproblems,”Com- puter Physics Communications, vol. 262, p. 107808, 2021
2021
-
[47]
Manycore algorithms for batch scalar and block tridiagonal solvers,
E. Laszlo, M. Giles, and J. Appleyard, “Manycore algorithms for batch scalar and block tridiagonal solvers,”ACM Transactions on Mathemat- ical Software (TOMS), vol. 42, no. 4, pp. 1–36, 2016
2016
-
[48]
High Perfor- mance Solution of Tridiagonal Systems on the GPU,
D. Tolmachev, P. Marti, G. Castiglioni, and A. Jackson, “High Perfor- mance Solution of Tridiagonal Systems on the GPU,”ACM Transactions on Parallel Computing, vol. 12, no. 2, pp. 1–25, 2025
2025
-
[49]
A fast gpu based bidiagonal solver for computational aeroacoustics,
S. Miao, X. Zhang, O. G. Parchment, and X. Chen, “A fast gpu based bidiagonal solver for computational aeroacoustics,”Computer Methods in Applied Mechanics and Engineering, vol. 286, pp. 22–39, 2015
2015
-
[50]
Symmetric tridiagonal eigenvalue solver across cpu graphics processing unit (gpu) nodes,
E. Hern ´andez-Rubio, A. Estrella-Cruz, A. Meneses-Viveros, J. A. Rivera-Rivera, L. I. Barbosa-Santill ´an, and S. V . Chapa-Vergara, “Symmetric tridiagonal eigenvalue solver across cpu graphics processing unit (gpu) nodes,”Applied Sciences, vol. 14, no. 22, 2024. [Online]. Av...
2024
-
[51]
Stable and efficient spectral divide and conquer algorithms for the symmetric eigenvalue decomposition and the svd,
Y . Nakatsukasa and N. J. Higham, “Stable and efficient spectral divide and conquer algorithms for the symmetric eigenvalue decomposition and the svd,”SIAM Journal on Scientific Computing, vol. 35, no. 3, pp. A1325–A1349, 2013. [Online]. Available: https://doi.org/10.1137/120876605
2013 doi
-
[52]
Bidiagonal svd computation via an associated tridiagonal eigenproblem,
O. Marques, J. Demmel, and P. B. Vasconcelos, “Bidiagonal svd computation via an associated tridiagonal eigenproblem,”ACM Trans. Math. Softw., vol. 46, no. 2, May 2020. [Online]. Available: https://doi.org/10.1145/3361746
2020 doi
-
[53]
A fast spectral divide-and-conquer method for banded matrices,
A. ˇSuˇsnjara and D. Kressner, “A fast spectral divide-and-conquer method for banded matrices,”Numerical Linear Algebra with Applications, vol. 28, no. 4, p. e2365, 2021. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1002/nla.2365
2021 doi
-
[54]
An improved divide-and-conquer algorithm for the banded matrices with narrow bandwidths,
X. Liao, S. Li, L. Cheng, and M. Gu, “An improved divide-and-conquer algorithm for the banded matrices with narrow bandwidths,”Computers & Mathematics with Applications, vol. 71, no. 10, pp. 1933–1943, 2016. [Online]. Available: https://www.sciencedirect.com/science/article/pi...
1933
-
[55]
A parallel structured banded dc algorithm for symmetric eigenvalue problems,
S. Li, X. Liao, Y . Lu, J. E. Roman, and X. Yue, “A parallel structured banded dc algorithm for symmetric eigenvalue problems,”CCF Transactions on High Performance Computing, vol. 5, no. 1, pp. 116–128, 2023. [Online]. Available: https://link.springer.com/article/10.1007/s4251...
2023 doi
-
[56]
Performance evaluation of the eigen exa eigensolver on oakleaf-fx: Tridiagonalization versus pentadiagonaliza- tion,
T. Fukaya and T. Imamura, “Performance evaluation of the eigen exa eigensolver on oakleaf-fx: Tridiagonalization versus pentadiagonaliza- tion,” in2015 IEEE International Parallel and Distributed Processing Symposium Workshop, 2015, pp. 960–969
2015
-
[57]
Parallel reduction of banded matrices to bidiagonal form,
B. Lang, “Parallel reduction of banded matrices to bidiagonal form,” Parallel Computing, vol. 22, no. 1, pp. 1–18, 1996. [Online]. Available: https://www.sciencedirect.com/science/article/pii/016781919500064X
1996
-
[58]
Lapack: a portable linear algebra library for supercomput- ers,
J. Demmel, “Lapack: a portable linear algebra library for supercomput- ers,” inIEEE Control Systems Society Workshop on Computer-Aided Control System Design, 1989, pp. 1–7
1989
-
[59]
A comprehensive study of task coalescing for selecting parallelism granularity in a two- stage bidiagonal reduction,
A. Haidar, H. Ltaief, P. Luszczek, and J. Dongarra, “A comprehensive study of task coalescing for selecting parallelism granularity in a two- stage bidiagonal reduction,” in2012 IEEE 26th International Parallel and Distributed Processing Symposium, 2012, pp. 25–35
2012
-
[60]
Efficient parallel reduction of bandwidth for symmetric matrices,
V . Manin and B. Lang, “Efficient parallel reduction of bandwidth for symmetric matrices,”Parallel Comput- ing, vol. 115, p. 102998, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0167819123000042
2023
-
[61]
Fine-grained bulge- chasing kernels for strongly scalable parallel qr algorithms,
L. Karlsson, B. K ˚agstr¨om, and E. Wadbro, “Fine-grained bulge- chasing kernels for strongly scalable parallel qr algorithms,”Parallel Computing, vol. 40, no. 7, pp. 271–288, 2014, 7th Workshop on Parallel Matrix Algorithms and Applications. [Online]. Available: https://www.s...
2014
-
[62]
Parallel two-stage reduction to hessenberg-triangular form,
T. Steel and R. Vandebril, “Parallel two-stage reduction to hessenberg-triangular form,”SIAM Journal on Scientific Computing, vol. 46, no. 2, pp. B56–B76, 2024. [Online]. Available: https://doi.org/10.1137/23M1547093
2024 doi
-
[63]
Algorithm 953: Parallel library software for the multishift qr algorithm with aggressive early deflation,
R. Granat, B. K ˚agstr¨om, D. Kressner, and M. Shao, “Algorithm 953: Parallel library software for the multishift qr algorithm with aggressive early deflation,”ACM Trans. Math. Softw., vol. 41, no. 4, Oct. 2015. [Online]. Available: https://doi.org/10.1145/2699471
2015 doi
-
[65]
Arrayfire: a gpu acceleration platform,
J. Malcolm, P. Yalamanchili, C. McClanahan, V . Venugopalakrishnan, K. Patel, and J. Melonakos, “Arrayfire: a gpu acceleration platform,” in Modeling and simulation for defense systems and applications VII, vol
-
[66]
C codegen considered unnecessary: go directly to binary, do not pass c. compilation of julia code for deployment in model-based engineering,
F. B. Carlson, C. Tapscott, G. Baraldi, and C. Rackauckas, “C codegen considered unnecessary: go directly to binary, do not pass c. compilation of julia code for deployment in model-based engineering,”
-
[67]
Large-scale distributed linear algebra with tensor processing units,
A. G. M. Lewis, J. Beall, M. Ganahl, M. Hauru, S. B. Mallick, and G. Vidal, “Large-scale distributed linear algebra with tensor processing units,”Proceedings of the National Academy of Sciences, vol. 119, no. 33, p. e2122762119, 2022. [Online]. Available: https://www.pnas.org/...
2022 doi
-
[68]
Juliagpu/amdgpu.jl: v0.7.3,
J. Samaroo, A. Smirnov, V . Churavy, L. R ¨ass, T. Hodgson, A. Montoison, W. Phillips, A. Ramadhan, J. Barmparesos, T. Besard, J. TagBot, M. Schanen, C. Bauer, M. Giordano, T. Arakaki, S. Antholzer, Alessandro, C. Elrod, G. Baraldi, H. Ranocha, M. Kunz, M. Raayai, T. Gymnich, ...
-
[69]
Available: https://arxiv.org/abs/2502.01128
[Online]. Available: https://arxiv.org/abs/2502.01128
-
[70]
Metal.jl,
T. Besard and M. Hawkins, “Metal.jl,” JuliaGPU, Jan. 2025. [Online]. Available: https://doi.org/10.5281/zenodo.14615291
2025 doi
-
[71]
Dynamic Task Scheduling with Data Dependency Awareness Using Julia,
R. Alomairy, F. Tome, J. Samaroo, and A. Edelman, “Dynamic Task Scheduling with Data Dependency Awareness Using Julia,” in2024 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 2024
2024
-
[72]
Toward Portable GPU Performance: Julia Recursive Im- plementation of TRMM and TRSM,
V . Carrica, M. Onyango, R. Alomairy, E. Ringoot, J. Schloss, and A. Edelman, “Toward Portable GPU Performance: Julia Recursive Im- plementation of TRMM and TRSM,”arXiv preprint arXiv:2504.13821, p. 10, 2025
2025 arXiv
-
[73]
oneAPI.jl,
T. Besard, “oneAPI.jl,” JuliaGPU, Jan. 2025. [Online]. Available: https://doi.org/10.5281/zenodo.14615352
2025 doi
-
[74]
NextLA.jl: Next-Gen Linear Algebra,
R. Alomairy, E. Ringoot, S. Xuan, V . Carrica, M. Onyango, and J. Samaroo, “NextLA.jl: Next-Gen Linear Algebra,” NextLinearAlgebra,
-
[75]
How to improve cuda kernel performance with shared memory register spilling,
NVIDIA Corporation, “How to improve cuda kernel performance with shared memory register spilling,” accessed: 2025-10-08. [Online]. Available: https://developer.nvidia.com/blog/how-to-improve- cuda-kernel-performance-with-shared-memory-register-spilling/
2025
-
[76]
Dissecting the nvidia blackwell architecture with microbenchmarks,
A. Jarmusch, N. Graddon, and S. Chandrasekaran, “Dissecting the nvidia blackwell architecture with microbenchmarks,” 2025. [Online]. Available: https://arxiv.org/abs/2507.10789
2025 arXiv
-
[77]
Synthesizing Numerical Linear Algebra using Julia,
S. Xuan, E. Ringoot, R. Alomairy, F. Tome, J. Samaroo, and A. Edelman, “Synthesizing Numerical Linear Algebra using Julia,” in2024 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 2024, p. 2
2024
-
[78]
Nvidia h100 tensor core gpu,
——, “Nvidia h100 tensor core gpu,” accessed: 2025-10-08. [Online]. Available: https://www.nvidia.com/en-us/data-center/h100/
2025
-
[79]
Available: https://doi.org/10.5281/zenodo.15049222
[Online]. Available: https://doi.org/10.5281/zenodo.15049222
-
[80]
Nvidia h100 product brief,
——, “Nvidia h100 product brief,” accessed: 2025-10-08. [Online]. Available: https://www.nvidia.com/content/dam/en-zz/Solutions/Data- Center/h100/PB-11773-001 v01.pdf
2025
-
[81]
Amd instinct mi250x accelerator,
Advanced Micro Devices, Inc., “Amd instinct mi250x accelerator,” accessed: 2025-10-08. [Online]. Available: https://www.amd.com/en/products/accelerators/instinct/mi200/mi250x.html
2025
-
[82]
Nvidia a100 tensor core gpu data sheet,
NVIDIA Corporation, “Nvidia a100 tensor core gpu data sheet,” accessed: 2025-10-08. [Online]. Available: https://www.nvidia.com/content/dam/en-zz/Solutions/Data- Center/a100/pdf/nvidia-a100-datasheet-nvidia-us-2188504-web.pdf
2025
-
[83]
Intel® xe gpu architec- ture,
Intel Corporation, “Intel® xe gpu architec- ture,” accessed: 2025-10-08. [Online]. Available: https://www.intel.com/content/www/us/en/docs/oneapi/optimization- guide-gpu/2023-1/intel-xe-gpu-architecture.html
2025
-
[84]
Nvidia ampere architecture whitepaper,
——, “Nvidia ampere architecture whitepaper,” accessed: 2025- 10-08. [Online]. Available: https://images.nvidia.com/aem-dam/en- zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf
2025
-
[85]
Intel’s ponte vecchio: Chiplets gone crazy,
C. Lam and G. Cozma, “Intel’s ponte vecchio: Chiplets gone crazy,” accessed: 2025-10-08. [Online]. Available: https://chipsandcheese.com/p/intels-ponte-vecchio-chiplets-gone-crazy
2025
-
[86]
Apple vs. oranges: Evaluating the apple silicon m-series socs for hpc performance and effi- ciency,
P. H ¨ubner, A. Hu, I. Peng, and S. Markidis, “Apple vs. oranges: Evaluating the apple silicon m-series socs for hpc performance and effi- ciency,” in2025 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). IEEE, 2025, pp. 45–54
2025
-
[87]
Amd instinct mi300x data sheet,
——, “Amd instinct mi300x data sheet,” accessed: 2025-10-08. [Online]. Available: https://www.amd.com/content/dam/amd/en/documents/instinct-tech- docs/data-sheets/amd-instinct-mi300x-data-sheet.pdf
2025
-
[88]
Bringing auto-tuning to hip: Analysis of tuning impact and difficulty on amd and nvidia gpus,
M. Lurati, S. Heldens, A. Sclocco, and B. van Werkhoven, “Bringing auto-tuning to hip: Analysis of tuning impact and difficulty on amd and nvidia gpus,” inEuro-Par 2024: Parallel Processing, J. Carretero, S. Shende, J. Garcia-Blas, I. Brandic, K. Olcoz, and M. Schreiber, Eds. ...
2024
-
[89]
Intel data center gpu max series product brief,
——, “Intel data center gpu max series product brief,” accessed: 2025-10-08. [Online]. Available: https://www.intel.com/content/dam/www/central- libraries/us/en/documents/2023-01/data-center-gpu-max-series-product- brief.pdf
2025
-
[92]
Parameter space visualization for large-scale datasets using parallel coordinate plots,
K. Glendenning, T. Wischgoll, J. Harris, R. Vickery, and L. Blaha, “Parameter space visualization for large-scale datasets using parallel coordinate plots,”Electronic Imaging, vol. 28, pp. 1–8, 2016
2016
-
[2023]
Available: https://doi.org/10.5281/zenodo.10040461
[Online]. Available: https://doi.org/10.5281/zenodo.10040461
-
[2025]
Available: https://github.com/ROCm/rocSOLVER
[Online]. Available: https://github.com/ROCm/rocSOLVER
-
[8403]
SPIE, 2012, pp. 49–56
2012
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.