Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Accelerating Bidiagonalization of Banded Matrices through Memory-Aware Bulge-Chasing on GPUs

T0 review · 4 major / 6 minor · reviewed 2026-08-04 · deepseek-v4-flash

Pith's one-line read A GPU-resident bulge-chasing kernel reduces banded matrices to bidiagonal form, beating multicore CPU libraries by over 100× at 32k.

desk verdict A genuine first GPU implementation of band-to-bidiagonal reduction with impressive speedups, but the correctness argument has a gap that needs fixing before the results are fully trustworthy. read the letter →

arxiv 2510.12705 v3 pith:O3TYZVKA submitted 2025-10-14 cs.DC cs.MS

classification cs.DCcs.MS MSC 65F1565F5065Y05
keywords bandedmatrixbidiagonalizationbulgechasingGPUsingularvaluedecompositionmemory-boundkernelsuccessivebandreductionperformanceportability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to overturn the long-held view that band-to-bidiagonal reduction, a memory-bound stage of singular value decomposition, cannot run efficiently on GPUs. Using recent growth in on-chip GPU caches, it adapts a CPU bulge-chasing strategy into a tiled, memory-aware GPU algorithm and reports the first fully GPU-resident implementation of this reduction. If the reported speedups hold, the last CPU-only stage in two-stage SVD disappears, and singular value computations can stay entirely on the accelerator. The authors also claim the algorithm's runtime scales linearly with matrix bandwidth, shifting the trade-off that previously forced SVD pipelines to keep intermediate bandwidths small.

What carries the argument

The load-bearing object is the bulge: a nonzero created when a Householder transformation annihilates an element, which must be chased out of the band by further transformations. The mechanism is Algorithm 1's row-sweep scheduling: each sweep chases bulges down the diagonal, while a three-cycle separation between sweeps is claimed to prevent overlapping memory access and permit parallel execution. Around this, bandwidth tiling (reducing the bandwidth by an inner tilewidth per stage), a kernel that reuses shared memory for reflector formation and registers for row chunks, and a tunable Max-blocks occupancy limit convert the memory-bound kernel into a cache-local one. The paper's occupancy con

What would settle it

On a small banded matrix, accumulate the Householder transformations produced by the algorithm into matrices Q and Z and check whether Q^T A Z is bidiagonal to machine precision and Q^T Q and Z^T Z equal the identity. Any out-of-band entry that remains nonzero, or any loss of orthogonality, would falsify the claim that the reduction is an orthogonal reduction to bidiagonal form.

Watch

Extended reading notes

Core claim

The central claim is that the banded-to-bidiagonal reduction—historically treated as too data-movement-bound for accelerators—can be made GPU-resident and fast by combining three ideas: successive reduction of the bandwidth in tiles, row-sweep bulge chasing in which consecutive sweeps are separated by a three-cycle gap to avoid data races, and a kernel that computes Householder reflectors in shared memory but applies them through register-resident row chunks. The paper reports that the implementation outperforms multithreaded CPU libraries starting at 1024×1024 matrices and reaches over 100× speedup at 32k×32k, with accuracy validated on the resulting singular values.

Load-bearing premise

The claim depends on the three-cycle separation between row sweeps guaranteeing that all bulges are chased with no hidden nonzeros—a property the paper does not directly test, since it validates only the computed singular values, not the bidiagonal structure or orthogonality of the transformations.

Editorial extensions

If this is right

  • Two-stage SVD can run fully on the GPU, removing CPU fallback and host-device sync from the middle stage.
  • SVD pipelines can choose larger intermediate bandwidths without penalty, since the second stage now scales linearly with bandwidth.
  • One hardware- and precision-agnostic kernel covers FP16, FP32, and FP64 across GPU vendors, avoiding vendor-specific rewrites.
  • Memory-bound kernels in numerical linear algebra can be made competitive on GPUs when L1/L2 cache size and latency, not DRAM throughput alone, drive kernel design.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reported speedups generalize, one-stage GPU SVD implementations that bypass the banded intermediate could be revisited, since the old reason for avoiding it—slow second stage—would no longer apply.
  • The strong reported dependence on cache latency rather than cache capacity suggests a concrete hardware design target: future GPU memory hierarchies should optimize L1/L2 latency for memory-bound kernels, not just bandwidth.
  • The algorithm's per-bulge kernel launches imply a synchronization cost that might be amortized differently on future architectures, so co-designing the scheduling into persistent kernels or cooperative groups is a plausible next step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents a GPU-resident algorithm for reducing a banded matrix to bidiagonal form by bulge-chasing, implemented with Julia's array abstractions and KernelAbstractions.jl so that one high-level kernel runs on NVIDIA, AMD, Intel, and Apple GPUs in FP16, FP32, and FP64. The authors claim this is the first fully GPU-resident band-to-bidiagonal reduction, that it outperforms PLASMA and SLATE starting at matrix sizes 1024x1024 and reaches over 100x speedup at 32k x 32k, and that its performance scales linearly with matrix bandwidth. The paper contributes a memory-aware kernel design, a bandwidth-tiling strategy for large bandwidths, a hardware-aware occupancy model, and a cross-vendor performance study.

Significance. If the correctness and performance claims hold, the paper addresses a recognized gap in two-stage SVD pipelines: the banded-to-bidiagonal stage has remained the last CPU-only stage on GPUs, and this work would remove that bottleneck. The single-source portable implementation across four GPU vendors and three precisions is a substantial engineering contribution, and the bandwidth-tiling strategy for larger matrix bandwidths could shift the practical trade-off between the first and second SVD stages. However, the central algorithmic correctness premise — that the parallel sweeps are race-free and that all bulges are annihilated so that the output is genuinely bidiagonal — is not directly verified; the current accuracy test only checks singular values through a solver that cannot see entries outside the bidiagonal storage. That premise must be established before the performance results can be fully credited.

major comments (4)
  1. [Algorithm 1; Section IV-B, Eq. (1)] Algorithm 1 cannot be the implemented schedule as written. (i) The inner loop (line 5) has no termination test; line 6 only guards the body, so when k>n the loop increments j forever. (ii) The separation rule is stated inconsistently: line 6 uses a fixed lag of three iterations, 3(R-1)<j, while Section IV-B and Eq. (1) state that the spacing between bulge-chasing blocks is 3*CBW. Since k advances by (TW+BW)*i (line 10), the coordinate separation produced by the line-6 lag depends on the stage index i, so Eq. (1) does not follow from the pseudocode. (iii) The bulge positions illustrated in Fig. 2 (k=3,7,10 for TW=2, BW=6) are not generated by line 10 for any consistent value of i. Please rewrite the pseudocode to match the implementation, add an explicit termination condition, and supply a correctness argument — or direct empirical evidence — that the separation makes concurrent sweeps ac
  2. [Section VI-A, Fig. 3] The numerical accuracy test is indirect. The authors form A=U*Sigma*V^T, reduce it to banded form, apply the GPU bidiagonal reduction, and then call LAPACK BDSDC. BDSDC reads only the stored bidiagonal diagonals; any nonzero entries left outside those diagonals by a missed or dropped bulge are invisible to the solver. Thus the reported singular-value errors cannot detect the failure modes most relevant to the paper's correctness claim: a silently dropped bulge or non-orthogonal accumulated transformations. Please add direct checks on the output of the reduction: (a) audit the zero pattern by measuring max |B_ij| outside the bidiagonal structure for a range of n, bandwidth, and precision; (b) if transformations are accumulated, verify orthogonality and the residual ||A - Q_left*B*Q_right^T|| / ||A||; or (c) compare singular values of the full reduced matrix stored as a dense banded matrix
  3. [Section IV-B, Eq. (1), Table I] The occupancy model is asserted without derivation. The number of concurrently active bulge-chasing blocks should follow from the sweep-separation rule, but with the line-6 lag of Algorithm 1 that number depends on n and on the stage-dependent stride, not simply on n/(3*CBW). Equation (1) and the minimum matrix sizes in Table I are therefore unsupported unless the link between the scheduling rule and the formula is provided. Please derive Eq. (1) from the actual implemented schedule, or revise the model and the conclusions drawn from it in Section VI-D.
  4. [Section IV-A, Algorithm 2] The kernel pseudocode omits the handling of boundaries and near-zero elements and defers details of the Householder reflector computation to reference [64]. That reference concerns tile-QR, not the bulge-chasing schedule used here, so it cannot justify the central race-freedom claim. In particular, Algorithm 2's column-annihilation phase (line 15) is described only as 'the procedure above', with no statement of how a column sweep interacts with simultaneously executing row sweeps. Please make the kernel pseudocode self-contained enough to check that two concurrent sweeps never touch overlapping rows or columns, or provide a formal/empirical race-freedom argument.
minor comments (6)
  1. [Section I] There is a sentence fragment near the middle of the introduction: 'in modern GPUs such as NVIDIA's Hopper and AMD's MI300X.' Please revise for readability.
  2. [Algorithm 1] The loop header 'j=0, j+=1' is unusual; please clarify whether j starts at 0 or 1 and whether the condition 3(R-1)<j is intended to be strict or non-strict.
  3. [Algorithm 2, line 10] The symbol 'CPB' is undefined; the row index r=k+l*CPB+i appears to be a typo for a quantity derived from TPB or CBW. Please define it or correct the expression.
  4. [Section VI-D, Fig. 6] Please state clearly whether PLASMA and SLATE timings include only the banded-to-bidiagonal reduction or also the dense-to-banded first stage, and describe the input storage/layout used for the CPU baselines. Without this, the comparison is difficult to interpret.
  5. [Open-source claim] The paper states the implementation is open-source but does not provide a repository URL or version identifier. Please add an artifact link.
  6. [Table II] The header contains apparent LaTeX spacing artifacts ('TH R E A D S P E R B L O C K', 'MA X B L O C K S', 'IN N E R T I L E W I D T H'). Please fix the header text.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: performance claims are empirically self-contained; the only self-citation (ref [64]) is a non-load-bearing implementation detail.

full rationale

The paper's central claims are engineering/performance claims, not derivations whose conclusions are encoded in their inputs. The speedups vs PLASMA/SLATE are measured against external libraries, and the accuracy test starts from matrices with known singular values, applies the pipeline, and compares to the ground-truth Σ; nothing in that loop assumes the conclusion. The occupancy model in §IV-B (Eq. 1) is derived from the algorithm's own scheduling rule (spacing 3·CBW) and hardware ALU count, and is used to explain observed scaling, not to generate a result by fitting to the benchmark data; it is therefore not a fitted-input-called-prediction. The one self-citation, ref [64], supplies Householder-reflector implementation details and a prior numerical-fidelity claim; it is a standard numerical building block and is exercised end-to-end in the accuracy tests, so it is not load-bearing for the paper's central contribution. The manuscript has a genuine correctness-validation gap—it verifies singular values after BDSDC but not bidiagonality or orthogonality, and the Algorithm 1 / Eq. 1 sweep-spacing statements are not reconciled—but that is an omitted-proof/correctness risk, not circular reasoning. Accordingly, no circular step is identified.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The performance claims rest on four hyperparameters tuned to the benchmark data (tilewidth, max blocks, threads per block, three-cycle separation) and on unproven assertions about sweep correctness and baseline fairness. No invented physical entities are introduced.

free parameters (4)
  • inner tilewidth (TW) = 32 (FP32) / 16 (FP64)
    Tuned per hardware and precision to match cache-line size; controls register footprint and synchronization frequency (Sec. IV-A, VI-B).
  • max blocks = varies (e.g., 96-192 on RTX4060)
    Limits active thread blocks per execution unit; trades occupancy for cache reuse and reduced memory contention (Sec. IV-C, VI-B).
  • threads per block (TPB) = varies (e.g., 16-64)
    Tuned per bandwidth and architecture; governs intra-block parallelism needed for applying Householder reflectors (Sec. VI-B).
  • three-cycle separation factor (3) = 3
    Chosen to avoid overlapping between row sweeps; asserted without proof in Sec. IV and Fig. 2, and not validated by direct bidiagonality checks.
assumptions (4)
  • standard math Householder reflections preserve singular values of the input matrix.
    Foundational property of orthogonal transformations used throughout Algorithm 1 and 2.
  • ad hoc to paper The three-cycle separation between row sweeps is sufficient to prevent overlapping data accesses and missed bulges.
    Stated in Section IV and Fig. 2, but no rigorous proof or direct validation of bidiagonality is provided.
  • domain assumption Synthetic matrices A = U Σ Vᵀ with prescribed singular values are representative for accuracy benchmarking.
    Used in Section VI-A to claim stable accuracy; real-world matrices may have different structures and conditioning.
  • domain assumption The CPU baseline libraries PLASMA and SLATE were run in a state-of-the-art configuration.
    The comparison in Section VI-D depends on the CPU libraries being well-tuned; the paper does not detail their tuning or storage formats.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Accelerating Bidiagonalization of Banded Matrices through Memory-Aware Bulge-Chasing on GPUs." pith.science (2026). https://pith.science/paper/O3TYZVKA

@misc{pith2026251012705,
  author       = {Pith},
  title        = {Pith review of: Accelerating Bidiagonalization of Banded Matrices through Memory-Aware Bulge-Chasing on GPUs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O3TYZVKA}},
  note         = {Machine review of arXiv:2510.12705}
}
read the original abstract

The reduction of a banded matrix to bidiagonal form is a critical step in the calculation of Singular Values, a cornerstone of scientific computing and AI. Although inherently parallel, this step has traditionally been considered unsuitable for GPUs due to its memory-bound nature. However, recent advances in GPU architectures, such as increased L1 memory per Streaming Multiprocessor or Compute Unit and larger L2 caches, have shifted this paradigm. In this work, we present the first GPU-accelerated algorithm for reducing a banded matrix to bidiagonal form, integrated into an open-source software package. Our algorithm builds on prior multicore CPU cache-efficient bulge-chasing methods, adapted to modern GPU architectures to optimize throughput. Leveraging Julia's high-level array abstractions and KernelAbstractions.jl, we implement a single function that is both hardware-agnostic and data-precision-aware, running efficiently across NVIDIA, AMD, Intel, and Apple Metal GPUs. We develop a hardware-aware performance model to guide tuning and identify key hyperparameters that govern optimal GPU performance for memory-bound workloads. We show that such workloads, when carefully optimized, can achieve substantial speed-ups on modern GPUs: our implementation outperforms multithreaded CPU libraries (PLASMA,SLATE) starting from matrix sizes as small as 1024x1024, and achieves over 100x speed-up on 32k x 32k matrices. Moreover, the algorithm's performance scales linearly with the matrix bandwidth, enabling efficient reduction of matrices with larger bandwidths, previously considered impractical.

Figures

Figures reproduced from arXiv: 2510.12705 by the authors.

Figure 2
Figure 2. In this work, we follow the parallel bulge-chasing [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 1
Figure 1. Illustration of the process of successive reduction of the bandwidth [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Visualization of the bandwidth-tiled, sweep-based GPU implementation of band-to-bidiagonal reduction using Householder reflectors. Each row sweep [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figures from the paper (5 more)
Figure 3
Figure 3. Figure 3: Relative error of singular values computed via GPU-based reduction [PITH_FULL_IMAGE:figures/full_fig_p007_3.png]
Figure 4
Figure 4. Figure 4: Performance of the GPU-based band reduction algorithm across kernel hyperparameters, hardware, and precisions. Parallel coordinate plots show [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Performance gains from GPU architectural advancements, showing the [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Runtime ratio of the GPU-accelerated band-to-bidiagonal reduc [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Runtime scaling of the band reduction algorithm across NVIDIA [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Hierarchical Recursive Precision for Accelerating Symmetric Linear Solves on MXUs

    cs.DC 2026-01 conditional novelty 6.0 of 10

    A tree-structured recursive Cholesky solver assigns FP16 to off-diagonal blocks and higher precision to diagonal blocks, achieving over 5x speedup on NVIDIA H200 and AMD MI300X with better accuracy than pure half precision.

Reference graph

Works this paper leans on

93 extracted references · 15 canonical work pages · cited by 1 Pith paper

  1. [64]

    Performant Unified GPU Kernels for Portable Singular Value Computation Across Hardware and Precision,

    E. Ringoot, R. Alomairy, V . Churavy, and A. Edelman, “Performant Unified GPU Kernels for Portable Singular Value Computation Across Hardware and Precision,”arXiv preprint arXiv:2508.06339, 2025

  2. [1]

    Unveiling lora intrinsic ranks via salience analysis,

    W. Ke, J. Wang, P. Wang, J. Liu, D. Nie, G. Li, and Y . Li, “Unveiling lora intrinsic ranks via salience analysis,” inAdvances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, Eds., vol. 37. Curran Associates, Inc., 2024, pp. 131 575–131 595. [Online]. Available: https://proceedin...

  3. [2]

    Svft: Parameter-efficient fine-tuning with singular vectors,

    V . Lingam, A. Tejaswi, A. Vavre, A. Shetty, G. K. Gudur, J. Ghosh, A. Dimakis, E. Choi, A. Bojchevski, and S. Sanghavi, “Svft: Parameter-efficient fine-tuning with singular vectors,” inAdvances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, Eds., vol. 37. Curran Associates, Inc...

  4. [3]

    Lora: Low-rank adaptation of large language models

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chenet al., “Lora: Low-rank adaptation of large language models.” ICLR, vol. 1, no. 2, p. 3, 2022

  5. [4]

    The extraordinary svd,

    C. D. Martin and M. A. Porter, “The extraordinary svd,”The American Mathematical Monthly, vol. 119, no. 10, pp. 838–851, 2012

  6. [5]

    Diagnosing non- hermitian many-body localization and quantum chaos via singular value decomposition,

    F. Roccati, F. Balducci, R. Shir, and A. Chenu, “Diagnosing non- hermitian many-body localization and quantum chaos via singular value decomposition,”Phys. Rev. B, vol. 109, p. L140201, Apr 2024. [Online]. Available: https://link.aps.org/doi/10.1103/PhysRevB.109.L140201

  7. [6]

    The singular value decomposition: Anatomy of optimizing an algorithm for extreme scale,

    J. Dongarra, M. Gates, A. Haidar, J. Kurzak, P. Luszczek, S. Tomov, and I. Yamazaki, “The singular value decomposition: Anatomy of optimizing an algorithm for extreme scale,”SIAM Review, vol. 60, no. 4, pp. 808– 865, 2018. [Online]. Available: https://doi.org/10.1137/17M1117732

  8. [7]

    Fast tridiagonal solvers on the gpu,

    Y . Zhang, J. Cohen, and J. D. Owens, “Fast tridiagonal solvers on the gpu,” inProceedings of the 15th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, ser. PPoPP ’10. New York, NY , USA: Association for Computing Machinery, 2010, p. 127–136. [Online]. Available: https://doi.org/10.1145/1693453.1693472

Show all 93 references
  1. [8]

    High performance solution of tridiagonal systems on the gpu,

    D. Tolmachev, P. Marti, G. Castiglioni, and A. Jackson, “High performance solution of tridiagonal systems on the gpu,”ACM Trans. Parallel Comput., vol. 12, no. 2, May 2025. [Online]. Available: https://doi.org/10.1145/3716171

  2. [9]

    A scalable, numerically stable, high-performance tridiagonal solver using gpus,

    L.-W. Chang, J. A. Stratton, H.-S. Kim, and W.-M. W. Hwu, “A scalable, numerically stable, high-performance tridiagonal solver using gpus,” inSC ’12: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, 2012, pp. 1–11

  3. [10]

    A divide-and-conquer approach for solving singular value decomposition on a heterogeneous system,

    D. Liu, R. Li, D. J. Lilja, and W. Xiao, “A divide-and-conquer approach for solving singular value decomposition on a heterogeneous system,” inProceedings of the ACM International Conference on Computing Frontiers, ser. CF ’13. New York, NY , USA: Association for Computing Mac...

  4. [11]

    Accelerating numerical dense linear algebra calculations with gpus,

    J. Dongarra, M. Gates, A. Haidar, J. Kurzak, P. Luszczek, S. Tomov, and I. Yamazaki, “Accelerating numerical dense linear algebra calculations with gpus,” inNumerical computations with GPUs. Springer, 2014, pp. 3–28

  5. [12]

    Ponte vecchio across the atlantic: Single-node benchmarking of two intel gpu systems,

    T. Applencourt, A. Sadawarte, S. Muralidharan, C. Bertoni, J. Kwack, Y . Luo, E. Rangel, J. Tramm, Y . Ghadar, A. Tameruset al., “Ponte vecchio across the atlantic: Single-node benchmarking of two intel gpu systems,” inSC24-W: Workshops of the International Conference for High...

  6. [13]

    Effective Extensible Program- ming: Unleashing Julia on GPUs,

    T. Besard, C. Foket, and B. De Sutter, “Effective Extensible Program- ming: Unleashing Julia on GPUs,”IEEE Transactions on Parallel and Distributed Systems, vol. 30, no. 4, pp. 827–841, 2019

  7. [14]

    Gpuarrays.jl,

    T. B. Simeon Danisch, “Gpuarrays.jl,” https://github.com/JuliaGPU/GPUArrays.jl, 2023

  8. [15]

    Kernelabstractions.jl,

    V . Churavy, “Kernelabstractions.jl,” https://github.com/JuliaGPU/ Ker- nelAbstractions.jl, 2023

  9. [16]

    PLASMA: Parallel Linear Algebra Software for Multicore Using OpenMP,

    J. Dongarra, M. Gates, A. Haidar, J. Kurzak, P. Luszczek, P. Wu, I. Ya- mazaki, A. YarKhan, M. Abalenkovs, N. Bagherpouret al., “PLASMA: Parallel Linear Algebra Software for Multicore Using OpenMP,”ACM Transactions on Mathematical Software (TOMS), vol. 45, no. 2, pp. 1– 35, 2019

  10. [17]

    Accelerating the SVD Two Stage Bidiagonal Reduction and Divide and Conquer Using GPUs,

    M. Gates, S. Tomov, and J. Dongarra, “Accelerating the SVD Two Stage Bidiagonal Reduction and Divide and Conquer Using GPUs,”Parallel Computing, vol. 74, pp. 3–18, 2018

  11. [18]

    Communication Avoiding Successive Band Reduction,

    G. Ballard, J. Demmel, and N. Knight, “Communication Avoiding Successive Band Reduction,”ACM SIGPLAN Notices, vol. 47, no. 8, pp. 35–44, 2012

  12. [19]

    Families of algorithms for reducing a matrix to condensed form,

    F. G. Van Zee, R. A. van de Geijn, G. Quintana-Ort ´ı, and G. J. Elizondo, “Families of algorithms for reducing a matrix to condensed form,”ACM Trans. Math. Softw., vol. 39, no. 1, Nov. 2012. [Online]. Available: https://doi.org/10.1145/2382585.2382587

  13. [20]

    Parallel Reduction to Con- densed Forms for Symmetric Eigenvalue Problems Using Aggregated Fine-Grained and Memory-Aware Kernels,

    A. Haidar, H. Ltaief, and J. Dongarra, “Parallel Reduction to Con- densed Forms for Symmetric Eigenvalue Problems Using Aggregated Fine-Grained and Memory-Aware Kernels,” inProceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Ana...

  14. [21]

    Algorithm 807: The sbr toolbox—software for successive band reduction,

    C. H. Bischof, B. Lang, and X. Sun, “Algorithm 807: The sbr toolbox—software for successive band reduction,”ACM Trans. Math. Softw., vol. 26, no. 4, p. 602–616, Dec. 2000. [Online]. Available: https://doi.org/10.1145/365723.365736

  15. [22]

    Restructuring the tridiagonal and bidiagonal qr algorithms for performance,

    F. G. Van Zee, R. A. van de Geijn, and G. Quintana-Ort ´ı, “Restructuring the tridiagonal and bidiagonal qr algorithms for performance,”ACM Trans. Math. Softw., vol. 40, no. 3, Apr. 2014. [Online]. Available: https://doi.org/10.1145/2535371

  16. [23]

    The elpa library: scalable parallel eigenvalue solutions for electronic structure theory and compu- tational science,

    A. Marek, V . Blum, R. Johanni, V . Havu, B. Lang, T. Auckenthaler, A. Heinecke, H.-J. Bungartz, and H. Lederer, “The elpa library: scalable parallel eigenvalue solutions for electronic structure theory and compu- tational science,”Journal of Physics: Condensed Matter, vol. 26...

  17. [24]

    Reduction to band form for the singular value decomposition on graphics accelerators,

    A. E. Tom ´as, R. Rodr ´ıguez-S´anchez, S. Catal ´an, and E. S. Quintana- Ort´ı, “Reduction to band form for the singular value decomposition on graphics accelerators,” inProceedings of the 9th International Workshop on Programming Models and Applications for Multicores and Ma...

  18. [25]

    A novel hybrid cpu–gpu generalized eigensolver for electronic structure calcu- lations based on fine-grained memory aware tasks,

    A. Haidar, S. Tomov, J. Dongarra, R. Solca, and T. Schulthess, “A novel hybrid cpu–gpu generalized eigensolver for electronic structure calcu- lations based on fine-grained memory aware tasks,”The International journal of high performance computing applications, vol. 28, no. 2...

  19. [26]

    Accelerating the reduction to upper hessenberg, tridiagonal, and bidiagonal forms through hybrid gpu-based computing,

    S. Tomov, R. Nath, and J. Dongarra, “Accelerating the reduction to upper hessenberg, tridiagonal, and bidiagonal forms through hybrid gpu-based computing,”Parallel Computing, vol. 36, no. 12, pp. 645–654, 2010. [Online]. Available: https://www.sciencedirect.com/science/article...

  20. [27]

    Evolution of the SLATE Linear Algebra Library,

    M. Gates, A. Abdelfattah, K. Akbudak, M. Al Farhan, R. Alomairy, D. Bielich, T. Burgess, S. Cayrols, N. Lindquist, D. Sukkariet al., “Evolution of the SLATE Linear Algebra Library,”The International Journal of High Performance Computing Applications, vol. 39, no. 1, pp. 3–17, 2025

  21. [28]

    MAGMA: Enabling exascale performance with accelerated BLAS and LAPACK for diverse GPU ar- chitectures,

    A. Abdelfattah, N. Beams, R. Carson, P. Ghysels, T. Kolev, T. Stitt, A. Vargas, S. Tomov, and J. Dongarra, “MAGMA: Enabling exascale performance with accelerated BLAS and LAPACK for diverse GPU ar- chitectures,”The International Journal of High Performance Computing Applicatio...

  22. [29]

    rocsolver,

    C. Bloor, J. Zuniga-Anaya, T. Alderso, and all, “rocsolver,” AMD,

  23. [30]

    Singular value decomposition on gpu using cuda,

    S. Lahabar and P. J. Narayanan, “Singular value decomposition on gpu using cuda,” in2009 IEEE International Symposium on Parallel and Distributed Processing, 2009, pp. 1–10

  24. [31]

    Gpu-based parallel householder bidiagonalization,

    F. Liu and F. J. Seinstra, “Gpu-based parallel householder bidiagonalization,” inProceedings of the 19th ACM International Symposium on High Performance Distributed Computing, ser. HPDC ’10. New York, NY , USA: Association for Computing Machinery, 2010, p. 288–291. [Online]. A...

  25. [32]

    Efficient gpu-centered singular value decomposition using the divide-and-conquer method,

    S. Liu, H. Li, H. Sheng, H. Gui, and X. Zhang, “Efficient gpu-centered singular value decomposition using the divide-and-conquer method,” ACM Transactions on Architecture and Code Optimization, 2025

  26. [33]

    Gpu- acceleration of the elpa2 distributed eigensolver for dense symmetric and hermitian eigenproblems,

    V . W. zhe Yu, J. Moussa, P. K ˚us, A. Marek, P. Messmer, M. Yoon, H. Lederer, and V . Blum, “Gpu- acceleration of the elpa2 distributed eigensolver for dense symmetric and hermitian eigenproblems,”Computer Physics Communications, vol. 262, p. 107808, 2021. [Online]. Available...

  27. [34]

    Eigen-g: Gpu-based eigen- value solver for real-symmetric dense matrices,

    T. Imamura, S. Yamada, and M. Machida, “Eigen-g: Gpu-based eigen- value solver for real-symmetric dense matrices,” inParallel Processing and Applied Mathematics, R. Wyrzykowski, J. Dongarra, K. Karczewski, and J. Wa´sniewski, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg...

  28. [35]

    Task-based, gpu- accelerated and robust library for solving dense nonsymmetric eigenvalue problems,

    M. Myllykoski and C. C. Kjelgaard Mikkelsen, “Task-based, gpu- accelerated and robust library for solving dense nonsymmetric eigenvalue problems,”Concurrency and Computation: Practice and Experience, vol. 33, no. 11, p. e5915, 2021. [Online]. Available: https://onlinelibrary.w...

  29. [36]

    Introduction to starneig—a task-based library for solving non- symmetric eigenvalue problems,

    ——, “Introduction to starneig—a task-based library for solving non- symmetric eigenvalue problems,” inParallel Processing and Applied Mathematics, R. Wyrzykowski, E. Deelman, J. Dongarra, and K. Kar- czewski, Eds. Cham: Springer International Publishing, 2020, pp. 70– 81

  30. [37]

    Toward a high performance tile divide and conquer algorithm for the dense symmetric eigenvalue problem,

    A. Haidar, H. Ltaief, and J. Dongarra, “Toward a high performance tile divide and conquer algorithm for the dense symmetric eigenvalue problem,”SIAM Journal on Scientific Computing, vol. 34, no. 6, pp. C249–C274, 2012. [Online]. Available: https://doi.org/10.1137/110823699

  31. [38]

    cusolver,

    NVIDIA, “cusolver,” NVIDIA, 2025. [Online]. Available: https://developer.nvidia.com/cusolver

  32. [39]

    An efficient 2d fusion method for high-performance two-stage eigensolvers on modern heterogeneous architectures,

    Y . Zhou, Y . Zong, Y . Jin, H. Li, and W. Xue, “An efficient 2d fusion method for high-performance two-stage eigensolvers on modern heterogeneous architectures,” inProceedings of the 39th ACM International Conference on Supercomputing, ser. ICS ’25. New York, NY , USA: Associ...

  33. [40]

    Improving tridiagonalization performance on gpu architectures,

    H. Wang, Z. Duan, Z. Zhao, S. Wu, S. Zheng, Q. Li, X. Jiang, and S. Zhang, “Improving tridiagonalization performance on gpu architectures,” inProceedings of the 30th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming, ser. PPoPP ’25. New York, NY ,...

  34. [41]

    Cuda programming guide,

    NVIDIA Corporation, “Cuda programming guide,” accessed: 2025-10-08. [Online]. Available: https://docs.nvidia.com/cuda/cuda- c-programming-guide/

  35. [42]

    Zhang, personal email communication

    S. Zhang, personal email communication

  36. [43]

    High-Performance Bidiagonal Reduction Using Tile Algorithms on Homogeneous Multicore Architec- tures,

    H. Ltaief, P. Luszczek, and J. Dongarra, “High-Performance Bidiagonal Reduction Using Tile Algorithms on Homogeneous Multicore Architec- tures,”ACM Transactions on Mathematical Software (TOMS), vol. 39, no. 3, pp. 1–22, 2013

  37. [44]

    Divide and conquer symmetric tridiagonal eigensolver for multicore architectures,

    G. Pichon, A. Haidar, M. Faverge, and J. Kurzak, “Divide and conquer symmetric tridiagonal eigensolver for multicore architectures,” in2015 IEEE International Parallel and Distributed Processing Symposium, 2015, pp. 51–60

  38. [45]

    Divide and conquer on hybrid gpu-accelerated multicore systems,

    C. V ¨omel, S. Tomov, and J. Dongarra, “Divide and conquer on hybrid gpu-accelerated multicore systems,”SIAM Journal on Scientific Com- puting, vol. 34, no. 2, pp. C70–C82, 2012

  39. [46]

    Gpu-acceleration of the elpa2 distributed eigensolver for dense symmetric and hermitian eigenproblems,

    V . W.-z. Yu, J. Moussa, P. Kuus, A. Marek, P. Messmer, M. Yoon, H. Lederer, and V . Blum, “Gpu-acceleration of the elpa2 distributed eigensolver for dense symmetric and hermitian eigenproblems,”Com- puter Physics Communications, vol. 262, p. 107808, 2021

  40. [47]

    Manycore algorithms for batch scalar and block tridiagonal solvers,

    E. Laszlo, M. Giles, and J. Appleyard, “Manycore algorithms for batch scalar and block tridiagonal solvers,”ACM Transactions on Mathemat- ical Software (TOMS), vol. 42, no. 4, pp. 1–36, 2016

  41. [48]

    High Perfor- mance Solution of Tridiagonal Systems on the GPU,

    D. Tolmachev, P. Marti, G. Castiglioni, and A. Jackson, “High Perfor- mance Solution of Tridiagonal Systems on the GPU,”ACM Transactions on Parallel Computing, vol. 12, no. 2, pp. 1–25, 2025

  42. [49]

    A fast gpu based bidiagonal solver for computational aeroacoustics,

    S. Miao, X. Zhang, O. G. Parchment, and X. Chen, “A fast gpu based bidiagonal solver for computational aeroacoustics,”Computer Methods in Applied Mechanics and Engineering, vol. 286, pp. 22–39, 2015

  43. [50]

    Symmetric tridiagonal eigenvalue solver across cpu graphics processing unit (gpu) nodes,

    E. Hern ´andez-Rubio, A. Estrella-Cruz, A. Meneses-Viveros, J. A. Rivera-Rivera, L. I. Barbosa-Santill ´an, and S. V . Chapa-Vergara, “Symmetric tridiagonal eigenvalue solver across cpu graphics processing unit (gpu) nodes,”Applied Sciences, vol. 14, no. 22, 2024. [Online]. Av...

  44. [51]

    Stable and efficient spectral divide and conquer algorithms for the symmetric eigenvalue decomposition and the svd,

    Y . Nakatsukasa and N. J. Higham, “Stable and efficient spectral divide and conquer algorithms for the symmetric eigenvalue decomposition and the svd,”SIAM Journal on Scientific Computing, vol. 35, no. 3, pp. A1325–A1349, 2013. [Online]. Available: https://doi.org/10.1137/120876605

  45. [52]

    Bidiagonal svd computation via an associated tridiagonal eigenproblem,

    O. Marques, J. Demmel, and P. B. Vasconcelos, “Bidiagonal svd computation via an associated tridiagonal eigenproblem,”ACM Trans. Math. Softw., vol. 46, no. 2, May 2020. [Online]. Available: https://doi.org/10.1145/3361746

  46. [53]

    A fast spectral divide-and-conquer method for banded matrices,

    A. ˇSuˇsnjara and D. Kressner, “A fast spectral divide-and-conquer method for banded matrices,”Numerical Linear Algebra with Applications, vol. 28, no. 4, p. e2365, 2021. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1002/nla.2365

  47. [54]

    An improved divide-and-conquer algorithm for the banded matrices with narrow bandwidths,

    X. Liao, S. Li, L. Cheng, and M. Gu, “An improved divide-and-conquer algorithm for the banded matrices with narrow bandwidths,”Computers & Mathematics with Applications, vol. 71, no. 10, pp. 1933–1943, 2016. [Online]. Available: https://www.sciencedirect.com/science/article/pi...

  48. [55]

    A parallel structured banded dc algorithm for symmetric eigenvalue problems,

    S. Li, X. Liao, Y . Lu, J. E. Roman, and X. Yue, “A parallel structured banded dc algorithm for symmetric eigenvalue problems,”CCF Transactions on High Performance Computing, vol. 5, no. 1, pp. 116–128, 2023. [Online]. Available: https://link.springer.com/article/10.1007/s4251...

  49. [56]

    Performance evaluation of the eigen exa eigensolver on oakleaf-fx: Tridiagonalization versus pentadiagonaliza- tion,

    T. Fukaya and T. Imamura, “Performance evaluation of the eigen exa eigensolver on oakleaf-fx: Tridiagonalization versus pentadiagonaliza- tion,” in2015 IEEE International Parallel and Distributed Processing Symposium Workshop, 2015, pp. 960–969

  50. [57]

    Parallel reduction of banded matrices to bidiagonal form,

    B. Lang, “Parallel reduction of banded matrices to bidiagonal form,” Parallel Computing, vol. 22, no. 1, pp. 1–18, 1996. [Online]. Available: https://www.sciencedirect.com/science/article/pii/016781919500064X

  51. [58]

    Lapack: a portable linear algebra library for supercomput- ers,

    J. Demmel, “Lapack: a portable linear algebra library for supercomput- ers,” inIEEE Control Systems Society Workshop on Computer-Aided Control System Design, 1989, pp. 1–7

  52. [59]

    A comprehensive study of task coalescing for selecting parallelism granularity in a two- stage bidiagonal reduction,

    A. Haidar, H. Ltaief, P. Luszczek, and J. Dongarra, “A comprehensive study of task coalescing for selecting parallelism granularity in a two- stage bidiagonal reduction,” in2012 IEEE 26th International Parallel and Distributed Processing Symposium, 2012, pp. 25–35

  53. [60]

    Efficient parallel reduction of bandwidth for symmetric matrices,

    V . Manin and B. Lang, “Efficient parallel reduction of bandwidth for symmetric matrices,”Parallel Comput- ing, vol. 115, p. 102998, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0167819123000042

  54. [61]

    Fine-grained bulge- chasing kernels for strongly scalable parallel qr algorithms,

    L. Karlsson, B. K ˚agstr¨om, and E. Wadbro, “Fine-grained bulge- chasing kernels for strongly scalable parallel qr algorithms,”Parallel Computing, vol. 40, no. 7, pp. 271–288, 2014, 7th Workshop on Parallel Matrix Algorithms and Applications. [Online]. Available: https://www.s...

  55. [62]

    Parallel two-stage reduction to hessenberg-triangular form,

    T. Steel and R. Vandebril, “Parallel two-stage reduction to hessenberg-triangular form,”SIAM Journal on Scientific Computing, vol. 46, no. 2, pp. B56–B76, 2024. [Online]. Available: https://doi.org/10.1137/23M1547093

  56. [63]

    Algorithm 953: Parallel library software for the multishift qr algorithm with aggressive early deflation,

    R. Granat, B. K ˚agstr¨om, D. Kressner, and M. Shao, “Algorithm 953: Parallel library software for the multishift qr algorithm with aggressive early deflation,”ACM Trans. Math. Softw., vol. 41, no. 4, Oct. 2015. [Online]. Available: https://doi.org/10.1145/2699471

  57. [65]

    Arrayfire: a gpu acceleration platform,

    J. Malcolm, P. Yalamanchili, C. McClanahan, V . Venugopalakrishnan, K. Patel, and J. Melonakos, “Arrayfire: a gpu acceleration platform,” in Modeling and simulation for defense systems and applications VII, vol

  58. [66]

    C codegen considered unnecessary: go directly to binary, do not pass c. compilation of julia code for deployment in model-based engineering,

    F. B. Carlson, C. Tapscott, G. Baraldi, and C. Rackauckas, “C codegen considered unnecessary: go directly to binary, do not pass c. compilation of julia code for deployment in model-based engineering,”

  59. [67]

    Large-scale distributed linear algebra with tensor processing units,

    A. G. M. Lewis, J. Beall, M. Ganahl, M. Hauru, S. B. Mallick, and G. Vidal, “Large-scale distributed linear algebra with tensor processing units,”Proceedings of the National Academy of Sciences, vol. 119, no. 33, p. e2122762119, 2022. [Online]. Available: https://www.pnas.org/...

  60. [68]

    Juliagpu/amdgpu.jl: v0.7.3,

    J. Samaroo, A. Smirnov, V . Churavy, L. R ¨ass, T. Hodgson, A. Montoison, W. Phillips, A. Ramadhan, J. Barmparesos, T. Besard, J. TagBot, M. Schanen, C. Bauer, M. Giordano, T. Arakaki, S. Antholzer, Alessandro, C. Elrod, G. Baraldi, H. Ranocha, M. Kunz, M. Raayai, T. Gymnich, ...

  61. [69]

    Available: https://arxiv.org/abs/2502.01128

    [Online]. Available: https://arxiv.org/abs/2502.01128

  62. [70]

    Metal.jl,

    T. Besard and M. Hawkins, “Metal.jl,” JuliaGPU, Jan. 2025. [Online]. Available: https://doi.org/10.5281/zenodo.14615291

  63. [71]

    Dynamic Task Scheduling with Data Dependency Awareness Using Julia,

    R. Alomairy, F. Tome, J. Samaroo, and A. Edelman, “Dynamic Task Scheduling with Data Dependency Awareness Using Julia,” in2024 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 2024

  64. [72]

    Toward Portable GPU Performance: Julia Recursive Im- plementation of TRMM and TRSM,

    V . Carrica, M. Onyango, R. Alomairy, E. Ringoot, J. Schloss, and A. Edelman, “Toward Portable GPU Performance: Julia Recursive Im- plementation of TRMM and TRSM,”arXiv preprint arXiv:2504.13821, p. 10, 2025

  65. [73]

    oneAPI.jl,

    T. Besard, “oneAPI.jl,” JuliaGPU, Jan. 2025. [Online]. Available: https://doi.org/10.5281/zenodo.14615352

  66. [74]

    NextLA.jl: Next-Gen Linear Algebra,

    R. Alomairy, E. Ringoot, S. Xuan, V . Carrica, M. Onyango, and J. Samaroo, “NextLA.jl: Next-Gen Linear Algebra,” NextLinearAlgebra,

  67. [75]

    How to improve cuda kernel performance with shared memory register spilling,

    NVIDIA Corporation, “How to improve cuda kernel performance with shared memory register spilling,” accessed: 2025-10-08. [Online]. Available: https://developer.nvidia.com/blog/how-to-improve- cuda-kernel-performance-with-shared-memory-register-spilling/

  68. [76]

    Dissecting the nvidia blackwell architecture with microbenchmarks,

    A. Jarmusch, N. Graddon, and S. Chandrasekaran, “Dissecting the nvidia blackwell architecture with microbenchmarks,” 2025. [Online]. Available: https://arxiv.org/abs/2507.10789

  69. [77]

    Synthesizing Numerical Linear Algebra using Julia,

    S. Xuan, E. Ringoot, R. Alomairy, F. Tome, J. Samaroo, and A. Edelman, “Synthesizing Numerical Linear Algebra using Julia,” in2024 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 2024, p. 2

  70. [78]

    Nvidia h100 tensor core gpu,

    ——, “Nvidia h100 tensor core gpu,” accessed: 2025-10-08. [Online]. Available: https://www.nvidia.com/en-us/data-center/h100/

  71. [79]

    Available: https://doi.org/10.5281/zenodo.15049222

    [Online]. Available: https://doi.org/10.5281/zenodo.15049222

  72. [80]

    Nvidia h100 product brief,

    ——, “Nvidia h100 product brief,” accessed: 2025-10-08. [Online]. Available: https://www.nvidia.com/content/dam/en-zz/Solutions/Data- Center/h100/PB-11773-001 v01.pdf

  73. [81]

    Amd instinct mi250x accelerator,

    Advanced Micro Devices, Inc., “Amd instinct mi250x accelerator,” accessed: 2025-10-08. [Online]. Available: https://www.amd.com/en/products/accelerators/instinct/mi200/mi250x.html

  74. [82]

    Nvidia a100 tensor core gpu data sheet,

    NVIDIA Corporation, “Nvidia a100 tensor core gpu data sheet,” accessed: 2025-10-08. [Online]. Available: https://www.nvidia.com/content/dam/en-zz/Solutions/Data- Center/a100/pdf/nvidia-a100-datasheet-nvidia-us-2188504-web.pdf

  75. [83]

    Intel® xe gpu architec- ture,

    Intel Corporation, “Intel® xe gpu architec- ture,” accessed: 2025-10-08. [Online]. Available: https://www.intel.com/content/www/us/en/docs/oneapi/optimization- guide-gpu/2023-1/intel-xe-gpu-architecture.html

  76. [84]

    Nvidia ampere architecture whitepaper,

    ——, “Nvidia ampere architecture whitepaper,” accessed: 2025- 10-08. [Online]. Available: https://images.nvidia.com/aem-dam/en- zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf

  77. [85]

    Intel’s ponte vecchio: Chiplets gone crazy,

    C. Lam and G. Cozma, “Intel’s ponte vecchio: Chiplets gone crazy,” accessed: 2025-10-08. [Online]. Available: https://chipsandcheese.com/p/intels-ponte-vecchio-chiplets-gone-crazy

  78. [86]

    Apple vs. oranges: Evaluating the apple silicon m-series socs for hpc performance and effi- ciency,

    P. H ¨ubner, A. Hu, I. Peng, and S. Markidis, “Apple vs. oranges: Evaluating the apple silicon m-series socs for hpc performance and effi- ciency,” in2025 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). IEEE, 2025, pp. 45–54

  79. [87]

    Amd instinct mi300x data sheet,

    ——, “Amd instinct mi300x data sheet,” accessed: 2025-10-08. [Online]. Available: https://www.amd.com/content/dam/amd/en/documents/instinct-tech- docs/data-sheets/amd-instinct-mi300x-data-sheet.pdf

  80. [88]

    Bringing auto-tuning to hip: Analysis of tuning impact and difficulty on amd and nvidia gpus,

    M. Lurati, S. Heldens, A. Sclocco, and B. van Werkhoven, “Bringing auto-tuning to hip: Analysis of tuning impact and difficulty on amd and nvidia gpus,” inEuro-Par 2024: Parallel Processing, J. Carretero, S. Shende, J. Garcia-Blas, I. Brandic, K. Olcoz, and M. Schreiber, Eds. ...

  81. [89]

    Intel data center gpu max series product brief,

    ——, “Intel data center gpu max series product brief,” accessed: 2025-10-08. [Online]. Available: https://www.intel.com/content/dam/www/central- libraries/us/en/documents/2023-01/data-center-gpu-max-series-product- brief.pdf

  82. [92]

    Parameter space visualization for large-scale datasets using parallel coordinate plots,

    K. Glendenning, T. Wischgoll, J. Harris, R. Vickery, and L. Blaha, “Parameter space visualization for large-scale datasets using parallel coordinate plots,”Electronic Imaging, vol. 28, pp. 1–8, 2016

  83. [2023]

    Available: https://doi.org/10.5281/zenodo.10040461

    [Online]. Available: https://doi.org/10.5281/zenodo.10040461

  84. [2025]

    Available: https://github.com/ROCm/rocSOLVER

    [Online]. Available: https://github.com/ROCm/rocSOLVER

  85. [8403]

    SPIE, 2012, pp. 49–56

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.