REVIEW 10 cited by
Dissecting the NVIDIA Hopper Architecture through Microbenchmarking and Multiple Level Analysis
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Dissecting the NVIDIA Hopper Architecture through Microbenchmarking and Multiple Level Analysis
read the original abstract
This study presents a comprehensive multi-level analysis of the NVIDIA Hopper GPU architecture, focusing on its performance characteristics and novel features. We benchmark Hopper's memory subsystem, highlighting improvements in the L2 partitioned cache and global memory access compared to Ampere and Ada Lovelace. The evaluation of Hopper's fourth-generation tensor cores reveals the benefits of FP8 precision and asynchronous wgmma instructions for matrix operations. Additionally, we investigate the performance of DPX instructions for dynamic programming, distributed shared memory (DSM) for inter-SM communication, and the Tensor Memory Accelerator (TMA) for asynchronous data movement. Through multi-level evaluation, we discover that the Hopper architecture demonstrates significant acceleration potential in real-world applications. For instance, the asynchronous programming model supported by TMA achieves a 1.5x speedup in matrix multiplication, FP8 delivers nearly double the performance of FP16, and DPX instructions accelerate a computational biology algorithm by at least 4.75x. Our findings provide actionable insights for optimizing compute-intensive workloads, from AI training to bioinformatics, on Hopper GPUs.
Forward citations
Cited by 10 Pith papers
-
TileLens: Efficiently Using Large-Granularity Memory Systems with Transparent Two-Dimensional Memory Layout
Tile-major memory layout plus TileLens SW/HW extensions and adaptive prefetching eliminate read amplification on kilobyte-granularity GPU memories, restoring near-HBM matmul speed for LLM kernels.
-
Non-Uniform L2 Cache Latency Across the Streaming Multiprocessors of an NVIDIA L40
L2 hit latency on NVIDIA L40 varies reproducibly by SM from 222-339 cycles following the AD102 GPC layout, enabling 11% makespan reduction via map-aware scheduling and 100% device separation via per-die fingerprint.
-
AsyncSparse: Accelerating Sparse Matrix-Matrix Multiplication on Asynchronous GPU Architectures
AsyncSparse presents BCSR and WCSR kernels that use TMA and warp specialization to accelerate SpMM, outperforming prior libraries by 1.47-6.24x on SuiteSparse and achieving 2.66x end-to-end speedup on Qwen2.5-7B at 90...
-
GPU-Tile-Sim: A Tile-Centric GPU Simulation Framework for LLM Hardware-Software Co-Design
A tile-graph GPU simulator predicts optimized LLM kernel performance on A100/H100 with MAPE about 1–9% by modeling dependencies and overlap rather than full instruction pipelines.
-
TLX: Hardware-Native, Evolvable MIMW GPU Compiler for Large-scale Production Environments
TLX is a Triton extension that exposes multi-warp, asynchronous, and cluster-level controls for modern GPUs, delivering competitive performance with low programmer effort and production deployment.
-
TLX: Hardware-Native, Evolvable MIMW GPU Compiler for Large-scale Production Environments
TLX introduces MIMW-based extensions to Triton that let developers orchestrate warp-group execution and asynchronous hardware features while preserving blocked programming productivity, with kernels deployed in large-...
-
Sim-FA: A GPGPU Simulator Framework for Fine-Grained Asynchronous Pipeline Analysis
Sim-FA is a new simulator that instruments FlashAttention-3 for cycle-accurate GPGPU analysis, achieving 5.7% average error on H800 while explaining inaccuracies in existing DRAM traffic models.
-
Sim-FA: A GPGPU Simulator Framework for Fine-Grained Asynchronous Pipeline Analysis
A trace-driven simulator models Hopper TMA/WGMMA pipelines at WarpGroup granularity, matching H800 FlashAttention-3 latency to 5.7% MAPE, and its analytical model explains GenZ's long-sequence DRAM underestimation.
-
DASH: Deterministic Attention Scheduling for High-throughput Reproducible LLM Training
DASH speeds up deterministic attention backprop by reversed and shift-based tile schedules, giving up to 1.28× throughput and ~5% end-to-end gains while keeping bitwise reproducibility.
-
Microbenchmark-Driven Analytical Performance Modeling Across Modern GPU Architectures
Microbenchmark-driven analytical models for B200 and MI300A achieve 1.31% and 0.09% MAE on validation kernels, far outperforming roofline baselines exceeding 95% error.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.