Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

KPerfIR: Towards an Open and Compiler-centric Ecosystem for GPU Kernel Performance Tooling on Modern AI Workloads

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read KPerfIR builds GPU profiling into the compiler IR itself, as reusable MLIR passes, and uses it to build a region-based intra-kernel timing tool that reports 8.2% overhead, 2% relative error, and guides a 24.1% improvement over Triton's…

desk verdict Worth engaging: the compiler-centric profiling dialect is a real contribution with open code; the empirical claims need error bars and the trace-replay assumption needs validation, but the architecture is solid. read the letter →

arxiv 2505.21661 v1 pith:U2CXXW6K submitted 2025-05-27 cs.DC cs.PL

Yue Guan , Yuanwei Fang , Keren Zhou , Corbin Robeck , Manman Ren , Zhongkai Yu , Yufei Ding , Adnan Aziz This is my paper
classification cs.DCcs.PL
keywords GPUprofilingcompiler-centricMLIRdialectTritonintra-kerneltimingtracereplayFlashAttention-3warpspecialization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

KPerfIR is a compiler-centric profiling infrastructure that lets performance tools be written as MLIR passes inside the Triton compiler, instead of as external black-box profilers. The paper claims this closes the gap between compilers and profilers: profiling operations are first-class IR operations (record, read counter, store counter) that lower through Triton's multi-level IR to GPU code for both Nvidia and AMD. To show the approach works, the authors build a region-based intra-kernel timing tool that instruments arbitrary code regions with start/end records, stores them in a circular shared-memory buffer, and replays the trace to correct for profiling-induced wait-time distortion. They report 8.2% end-to-end profiling overhead, about 2% relative measurement error, and use the tool's timeline to reorder barriers in FlashAttention-3, improving over the vanilla Triton FA3 kernel by 24.1%.

What carries the argument

The central object is the KPerfIR dialect plus its lowering path: a single high-level RecordOp is lowered through KPerfGPUIR (ReadCounterOp/StoreCounterOp, buffer allocation, init/finalize) to LLVM-level start/stop instrumentation. This multi-level IR chain is what carries the argument's portability: the same record markers work across Nvidia and AMD because the hardware-specific buffering and counter reads are inserted during lowering. The region-based timing tool's accuracy rests on trace replay, which cancels the profiler's own overhead by placing two start records around an asynchronous launch and one end record before the wait barrier, so the measured wait time is $(CLK2 - T_a) - (CLK1 - T_a) = CLK2 - CLK1$.

What would settle it

Profile a Triton kernel whose timed region is a short synchronous sequence, say a few back-to-back scalar or vector ALU operations with no tensor-core work, using KPerfIR's region timing, and compare each region's duration against a cycle-accurate hardware trace from NCU on the same H100. If the corrected wait time goes negative or the relative error exceeds the claimed 2% for these sub-1000-cycle regions, the trace-replay assumption is violated and the tool's accuracy claim does not hold for short synchronous regions.

Watch

Extended reading notes

Core claim

KPerfIR's central claim is that GPU performance profiling should be a compiler behavior, not an external tool. The paper introduces a KPerfIR dialect whose RecordOp marks profile-region boundaries; a lowering pass rewrites these markers into KPerfGPUIR operations (ReadCounterOp, StoreCounterOp, InitOp, FinalizeOp) and eventually into LLVM-level instrumentation with start/stop markers. Because instrumentation rides the compiler's own IR, it can report loop-iteration numbers, warp-group granularity, and region nesting, and it can be invoked from within the same pass pipeline that performs optimizations. The demonstration tool, a region-based timing profiler, records 8-byte timestamp entries into per-warp-group shared-memory buffers with a circular-overwrite strategy, then uses a trace-replay step to subtract the profiler's own clock-reading overhead from asynchronous wait times, provided the timed hardware unit (such as a tensor core) runs long enough to absorb the recording cost.

Load-bearing premise

The load-bearing measurement premise is Section 5.3's inequality $T_{MMA} - T_{exe} > T_a + T_b$: the asynchronous unit being timed must execute long enough (around 1000 cycles) for the profiler's own per-record overhead (under 25 cycles) to be absorbed, otherwise the trace-replay correction gives wrong wait times and the FA3 optimization built on those wait times is unreliable.

Editorial extensions

If this is right

  • If profiling runs as compiler passes, autotuning and feedback-guided optimization passes can consume profile results in the same compilation session, enabling closed-loop kernel optimization without external tooling.
  • The same KPerfIR tooling applies to both Nvidia and AMD backends, so a performance tool written once in IR terms does not need to be rewritten per vendor.
  • Intra-kernel region timing at warp-group granularity becomes available for kernels written in Triton, exposing pipeline-stage overlap, idle bubbles, and critical-path stages that whole-kernel profilers cannot see.
  • The FA3 case study shows that region-level wait-time data can directly identify a movable arrival barrier, leading to a concrete compiler-pass change with 24.1% speedup over the baseline Triton FA3 kernel.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension: The 24.1% FA3 improvement is reported for a single configuration (head dim 128, batch 16, seq 4096); whether the barrier-advancing optimization generalizes across batch sizes, sequence lengths, and head counts is not established by the paper, and the performance model's predicted TFLOPs suggest sensitivity to these parameters.
  • Extension: Because the trace-replay correction assumes the timed unit's execution time exceeds the recording overhead, the tool's accuracy is likely to degrade on short synchronous regions or on AMD instructions where scheduling is software-controlled; a natural test is to profile kernels with sub-100-cycle regions and compare against hardware counters.
  • Extension: The compiler-centric design implies that profiling becomes part of the kernel's binary signature and calling convention, which may interact with kernel caching, JIT specialization, and distributed-fusion workflows; these integration costs are not measured in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes KPerfIR, a compiler-centric profiling infrastructure implemented as MLIR dialects and passes inside the Triton compiler, enabling profiling tools to be written as compiler transformations that operate on high-level IR while lowering to GPU-specific instrumentation. The authors present a region-based intra-kernel timing tool built on this infrastructure, a trace-replay post-processing method intended to correct profiling-induced distortion of asynchronous wait times, and a case study on FlashAttention-3 in which profiling-guided changes to the arrival barrier of the V tensor are reported to improve performance by 24.1% over Triton's vanilla FA3 and by 7.6% over the manual FA3 kernel. The paper claims low profiling overhead (8.2%), 2% relative error, and portability across NVIDIA H100 and AMD MI300 platforms.

Significance. If the claims hold, KPerfIR would be a valuable open infrastructure: it bridges a real gap between compiler IR semantics and GPU profilers, enables tools to be expressed as reusable MLIR passes, and it is accompanied by open-source code integrated into the Triton tree. The region-based timing tool and the trace-replay correction are interesting technical contributions, and the FA3 case study demonstrates a plausible workflow in which compiler-level profiling feedback directly motivates a kernel transformation. The main strengths are the multi-level IR design, the explicit interfaces for third-party tools, and the concrete demonstration on a relevant modern AI workload. The significance is conditional on the empirical validation being made reproducible and on the trace-replay correction being shown to be accurate under the conditions where it is applied.

major comments (4)
  1. [§5.3] The trace-replay correction is load-bearing for the paper's accuracy claim and for the FA3 idle-bubble diagnosis, but its validity condition T_MMA − T_exe > T_a + T_b is never tested. The paper only bounds T_a + T_b (<25 cycles) and T_MMA (~1000 cycles); T_exe is unbounded, so the inequality can fail for short regions or for regions ending near a barrier, exactly the situation in the FA3 case where the arrival barrier of region 16 is reported as the bottleneck. Since §4.2 acknowledges that AMD exposes instruction scheduling to software, the same assumption is also less automatic on MI300. The evaluation in Table 5 measures end-to-end latency degradation, not per-region wait-time accuracy against a ground truth. Please add microbenchmarks that vary region length relative to T_MMA, compare corrected wait times against a ground-truth timer or a controlled injected-idle experiment, and report results for both H100 and MI300.
  2. [A Artifact Appendix] The artifact appendix states that 'the results for the OSDI'25 submission are derived from some feature branches' and that the implementation 'is still evolving.' This makes the headline numbers (8.2% overhead, 24.1% FA3 improvement, 7.6% over manual FA3) not independently reproducible from the cited artifact. Please pin the exact commits for the evaluated feature branches, include the profiling and timing scripts and raw outputs in the artifact, and state which reported result corresponds to which commit. Without this, the empirical contribution cannot be verified.
  3. [§6.4, Eq. (1), Table 5] Equation (1) defines T_theoretical = T_vanilla + N_record * Cyclerecord, and the text says the performance impact is within 2%, but Table 5 reports 199381 theoretical active cycles against 224981 actual, which is a 12.8% gap; the row 'Relative Performance' (0.89, 1, 1.02) is unexplained and does not by itself establish a 2% relative error. The abstract's '2% relative error' appears to refer to a different quantity than Table 5, since Table 5 measures degradation rather than timing accuracy. Please clarify exactly what quantity is 2%, how it was measured, and reconcile Table 5 and its caption with the surrounding text.
  4. [§6.2.2, Table 4] The performance model in Table 4 is asserted rather than derived or validated. The paper uses it to predict 582.44 TFLOPs for the improved FA3 kernel but does not compare this prediction against the measured TFLOPs of the improved kernel, nor against measured results for other configurations in Fig. 12. Since the model is presented as part of a 'performance modeling pass' that guides the overlapping optimization, please provide a predicted-versus-measured comparison across the swept batch and sequence-length configurations and on at least one additional workload (e.g., GEMM-SWP).
minor comments (5)
  1. [§4.2] The phrase 'even for instruction SMEM load and MFMA in amdgcm' contains a typo: 'amdgcm' should be 'amdgcn'.
  2. [§6.3] The text contains 'We hightlight that with the post-processing trace replay technique'; 'hightlight' should be 'highlight', and the sentence would be clearer if split.
  3. [Figure 10] Figure 10 uses 'Idel Time' and 'Vanilla Execution' in a way that is easy to misread; the labels should be corrected to 'Idle Time' and the two timelines should be distinguished more clearly.
  4. [Table 1] Table 1 uses '#' symbols in place of checkmarks/crosses, which is ambiguous; please use conventional symbols (e.g., ✓/✗ or yes/no) or add a legend.
  5. [§1 and §2.2] The claim of being the 'first region-based timing tool for GPUs' should be scoped more carefully, since §2.2 acknowledges that ThunderKitten also offers a region-based tracing interface; the novelty should be stated as the MLIR/compiler-centric mechanism rather than region timing per se.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: the core claims rest on an implemented MLIR instrumentation flow and measured, externally comparable kernel runs; the only weak point is an explicitly stated coverage assumption in trace replay, which is a validity condition, not a circular reduction.

full rationale

KPerfIR's central claims are infrastructural: profiling operations are lowered through MLIR to counter reads/stores, and the region-based timing tool is a concrete consumer of that infrastructure. No claimed prediction is defined in terms of what it purports to predict. The trace-replay correction in Sec. 5.3 cancels the record overhead algebraically (Twait = CLK2 - CLK1) and states its validity condition T_MMA - T_exe > T_a + T_b explicitly; whether that condition holds is a measurement-validity assumption, not a circular step. The overlapping performance model in Sec. 6.2.2 takes profiled stage latencies as inputs and forms closed-form SWP/WS latency expressions; the resulting 'predicted' 582.44 TFLOPs is a model output, and the reported 24.1% improvement over vanilla Triton FA3 is an actually measured kernel outcome, so the optimization claim is not forced by the model. The low-level overhead model (Eq. 1) adds a separately measured per-record cycle cost to the uninstrumented runtime and compares against the measured instrumented runtime, which is a falsifiable comparison rather than a fit by construction. Self-citations ([20], [22], [53], [54]) are used for prior-work context or buffer-communication mechanisms, not as load-bearing justification of the uniqueness or correctness of KPerfIR. The artifact appendix's disclosure that results come from evolving feature branches is a reproducibility caveat, not circularity. Overall, no equation or headline number reduces to its own inputs.

Assumptions & free parameters 2 free parameters · 4 assumptions · 2 invented entities

The central claim rests on several domain assumptions about GPU counter access, async instruction timing, and IR stability. No physical entities are invented; the introduced items are software abstractions with open-source artifacts. The performance model introduces a hand-asserted formula rather than fitted constants.

free parameters (2)
  • Per-record profiling overhead (Cyclerecord) = 33 cycles (H100, Fig. 15)
    Measured on the H100 and used in Eq. 1 for theoretical instrumented time; it is calibrated data, not a parameter fitted to the FA3 speedup.
  • Trace-replay overhead threshold = <25 cycles per record; T_MMA about 1000 cycles
    Assumed in Sec. 5.3 to guarantee accurate wait-time subtraction; not validated across the benchmark suite.
assumptions (4)
  • domain assumption GPU cycle counters (%clock on Nvidia, LSB of S_MEMTIME on AMD) are readable in kernel code and accurate enough for 32-bit timestamps.
    Needed for every KPerfGPUIR ReadCounterOp and StoreCounterOp lowering (Sec. 4.1 and Sec. 5.2).
  • domain assumption Asynchronous tensor-core operation time can cover the inserted profiling records: T_MMA - T_exe > T_a + T_b, with overhead below 25 cycles.
    Trace-replay correction in Sec. 5.3 relies on this inequality to subtract wait time accurately.
  • domain assumption Triton's MLIR pass ordering and multi-level IR semantics are stable enough for instrumentation at TTIR and TTGIR to remain meaningful after lowering.
    The portability and reusability claims in Sec. 4.1 and Sec. 7.2 depend on this.
  • ad hoc to paper The simplified performance model in Table 4, which ignores initialization and epilogue, can predict overlapping efficiency of FA3.
    The model is asserted in Sec. 6.2.2 and used to produce the predicted 582.44 TFLOPs; no derivation or cross-workload validation is provided.
invented entities (2)
  • KPerfIR MLIR dialect operations (RecordOp, ReadCounterOp, StoreCounterOp, InitOp, FinalizeOp) independent evidence
    purpose: Represent profiling markers and counter reads and stores in Triton's compiler IR so tools can be implemented as passes.
    The open-source Triton repository at the cited path exposes these operations, providing an external artifact, though the exact feature-branch implementation used for evaluation is not pinned.
  • KPerfGPUIR intermediate dialect independent evidence
    purpose: Vendor-independent but GPU-specific layer between high-level KPerfIR records and LLVM instrumentation.
    Documented at triton-lang.org/main/dialects/ProtonOps.html and present in the open-source tree.

how reviews work

0 comments
Cite this review

Pith. "Pith review of KPerfIR: Towards an Open and Compiler-centric Ecosystem for GPU Kernel Performance Tooling on Modern AI Workloads." pith.science (2026). https://pith.science/paper/U2CXXW6K

@misc{pith2026250521661,
  author       = {Pith},
  title        = {Pith review of: KPerfIR: Towards an Open and Compiler-centric Ecosystem for GPU Kernel Performance Tooling on Modern AI Workloads},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U2CXXW6K}},
  note         = {Machine review of arXiv:2505.21661}
}
read the original abstract

In this work, we propose KPerfIR, a novel multilevel compiler-centric infrastructure to enable the development of customizable, extendable, and portable profiling tools tailored for modern artificial intelligence (AI) workloads on modern GPUs. Our approach integrates profiling capabilities directly into the compiler workflow, allowing profiling functionalities to be implemented as compiler passes, offering a programmable and reusable framework for performance analysis. This design bridges the gap between compilers and profilers, enabling fine-grained insights into complex optimization challenges such as overlapping the execution of fine-grained function units on GPUs. KPerfIR is integrated into the Triton infrastructure to highlight the power of a compiler-centric approach to advance performance analysis and optimization in the ever-evolving landscape of AI compilers. Our evaluation shows that our tool incurs low overhead (8.2%), provides accurate measurements (2% relative error), and delivers actionable insights into complicated GPU intra-kernel optimizations.

Figures

Figures reproduced from arXiv: 2505.21661 by the authors.

Figure 1
Figure 1. Concept of the KPerfIR infrastructure and ecosystem for compiler-centric performance tool. (Left) Overview and comparison of KPerfIR’s compiler-centric design and prior profiler designs. (Right) Demonstrative examples of novel performance tools facilitated by the compiler-centric design of KPerfIR. able and reusable performance tools1 , as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. GPU overlapping techniques MosaicGPU profiler [10], which inserts PTX instructions using high-level Python bindings, represents the closest idea to our approach. However, it operates at the assembly code level rather than on the high-level IRs, restricting its ability to provide comprehensive and reusable profiling capabilities. ThunderKitten (TK) [44], a promising DSL for the Nvidia platform, also developed a custo… view at source ↗
Figure 3
Figure 3. Motivating examples the clustering and pipelining of matrix operations. Software Pipelining (SWP) transforms the execution of inde￾pendent loop iteration operations (i.e., memory and compute) into multiple stages to overlap between iterations, as shown in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: IR design and conversion passes into the compiler IR as a performance tooling infrastructure. The system is implemented upon the mainstream AI compiler Triton [47], which adopts MLIR as its compiler infrastructure. We demonstrate the multi-level IR design and runtime c…
Figure 5
Figure 5. Figure 5: An example of high-level record operations [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Novel use cases facilitated by KPerfIR’s compiler-centric approach specified analysis passes. This allows for handy manipulation of the target workload but lacks flexibility since users cannot skip kernels they are not interested in. The Python API lets the user specif…
Figure 7
Figure 7. Figure 7: Workflow of the region-based timing tool [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Memory management of the region-based tool [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Circular buffer and record buffer low and the store takes a vectorized store instruction. We use a 32-bit clock to capture the cycle tick of the current record, which may cause value overflow. We address this in the post-processing procedure, where we detect and throw …
Figure 11
Figure 11. Figure 11: Region-based timing results for FA3 kernels and overlapping improvements guided by profiling [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 12
Figure 12. Figure 12: Benchmarking FA3 kernels with a head dimension of 128 and 16 heads. The batch size and sequence length are set to [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]
Figure 14
Figure 14. Figure 14: Memory usage is practical for real-world scenarios without significantly im￾pacting kernel performance. For the most complicated SWP GEMM kernel with three stages, we insert many records to cover its three stages. Even in this case, the overhead is kept within 15%. Th…
Figure 15
Figure 15. Figure 15: Cycle-level benchmarks GEMM Theoretical Actual Active Cycles 199381 224981 229663 Relative Performance 0.89 1 1.02 [PITH_FULL_IMAGE:figures/full_fig_p013_15.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TileSight: A First-Principles Tile-Centric Analytical GPU Performance Model from Cores to Clusters

    cs.DC 2026-07 conditional novelty 7.0 of 10

    A tile-centric analytical model predicts GPU kernel latency and cache behavior purely from microbenchmark-calibrated hardware rates, reaching about 12% GEMM error and 13% end-to-end LLM serving error across five GPU lines.

Reference graph

Works this paper leans on

54 extracted references · 38 canonical work pages · cited by 1 Pith paper

  1. [1]

    AMD CDNA 3 Architec- ture, 2024

    Advanced Micro Devices, Inc. AMD CDNA 3 Architec- ture, 2024

    work page 2024

  2. [2]

    AMD Instinct MI300

    Advanced Micro Devices, Inc. "AMD Instinct MI300" Instruction Set Architecture, 2024

    work page 2024

  3. [3]

    Composable kernel (CK) library, 2024

    Advanced Micro Devices, Inc. Composable kernel (CK) library, 2024

    work page 2024

  4. [4]

    ROCm ROCProfiler, 2024

    Advanced Micro Devices, Inc. ROCm ROCProfiler, 2024

    work page 2024

  5. [5]

    Version 6.2.4

    Advanced Micro Devices, Inc.ROCm ROCTracer, 2024. Version 6.2.4

    work page 2024

  6. [6]

    rocBLAS Library, 2023

    AMD. rocBLAS Library, 2023

    work page 2023

  7. [7]

    Pytorch 2: Faster machine learning through dynamic python bytecode transformation and graph compilation

    Jason Ansel, Edward Yang, Horace He, Natalia Gimelshein, Animesh Jain, Michael V oznesensky, Bin Bao, Peter Bell, David Berard, Evgeni Burovski, et al. Pytorch 2: Faster machine learning through dynamic python bytecode transformation and graph compilation. In Proceedings of the 29th ACM International Confer- ence on Architectural Support for Programming L...

    work page 2024

  8. [8]

    Cu- daDMA: optimizing GPU memory bandwidth via warp specialization

    Michael Bauer, Henry Cook, and Brucek Khailany. Cu- daDMA: optimizing GPU memory bandwidth via warp specialization. In Proceedings of 2011 International Conference for High Performance Computing, Network- ing, Storage and Analysis, pages 1–11, Seattle Washing- ton, November 2011. ACM

    work page 2011

Show all 54 references
  1. [9]

    Hatchet: Pruning the overgrowth in parallel profiles

    Abhinav Bhatele, Stephanie Brink, and Todd Gamblin. Hatchet: Pruning the overgrowth in parallel profiles. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Anal- ysis, pages 1–21, 2019

    2019

  2. [10]

    JAX: com- posable transformations of Python+NumPy programs, 2018

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclau- rin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: com- posable transformations of Python+NumPy programs, 2018

    2018

  3. [11]

    Language models are few-shot learners

    Tom B Brown. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020

    2005 arXiv

  4. [12]

    Flux: fast software-based communication overlap on gpus through kernel fusion

    Li-Wen Chang, Wenlei Bao, Qi Hou, Chengquan Jiang, Ningxin Zheng, Yinmin Zhong, Xuanrun Zhang, Zuquan Song, Chengji Yao, Ziheng Jiang, et al. Flux: fast software-based communication overlap on gpus through kernel fusion. arXiv preprint arXiv:2406.06858, 2024

    2024 arXiv

  5. [13]

    {TVM}: An automated {End-to-End} optimizing compiler for deep learning

    Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, et al. {TVM}: An automated {End-to-End} optimizing compiler for deep learning. In 13th USENIX Symposium on Oper- ating Systems Design and Implemen...

    2018

  6. [14]

    Learning to optimize tensor programs

    Tianqi Chen, Lianmin Zheng, Eddie Yan, Ziheng Jiang, Thierry Moreau, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. Learning to optimize tensor programs. Advances in Neural Information Processing Systems, 31, 2018

    2018

  7. [15]

    Nvidia hopper h100 gpu: Scaling per- formance

    Jack Choquette. Nvidia hopper h100 gpu: Scaling per- formance. IEEE Micro, 43(3):9–17, 2023

    2023

  8. [16]

    V olta: Performance and programmability

    Jack Choquette, Olivier Giroux, and Denis Foley. V olta: Performance and programmability. Ieee Micro , 38(2):42–52, 2018

    2018

  9. [17]

    Crago, Sana Damani, Karthikeyan Sankar- alingam, and Stephen W

    Neal C. Crago, Sana Damani, Karthikeyan Sankar- alingam, and Stephen W. Keckler. WASP: Exploiting GPU Pipeline Parallelism with Hardware-Accelerated Automatic Warp Specialization. In 2024 IEEE Inter- national Symposium on High-Performance Computer Architecture (HPCA), pages 1–...

    2024

  10. [18]

    Davidson and Christopher W

    Jack W. Davidson and Christopher W. Fraser. Elim- inating redundant object code. In Proceedings of the 9th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’82, page 128–132, New York, NY , USA, 1982. Association for Computing Machinery

    1982

  11. [19]

    Chrome trace format, 2023

    Google. Chrome trace format, 2023

    2023

  12. [20]

    Amanda: Unified instrumentation 14 framework for deep neural networks

    Yue Guan, Yuxian Qiu, Jingwen Leng, Fan Yang, Shuo Yu, Yunxin Liu, Yu Feng, Yuhao Zhu, Lidong Zhou, Yun Liang, et al. Amanda: Unified instrumentation 14 framework for deep neural networks. In Proceedings of the 29th ACM International Conference on Architectural Support for Pro...

    2024

  13. [21]

    Profile inference revisited

    Wenlei He, Julián Mestre, Sergey Pupyrev, Lei Wang, and Hongtao Yu. Profile inference revisited. Proc. ACM Program. Lang., 6(POPL), January 2022

    2022

  14. [22]

    ALCOP: Automatic Load- Compute Pipelining in Deep Learning Compiler for AI- GPUs, May 2023

    Guyue Huang, Yang Bai, Liu Liu, Yuke Wang, Bei Yu, Yufei Ding, and Yuan Xie. ALCOP: Automatic Load- Compute Pipelining in Deep Learning Compiler for AI- GPUs, May 2023. arXiv:2210.16691

    2023 arXiv

  15. [23]

    Alcop: Automatic load- compute pipelining in deep learning compiler for ai- gpus

    Guyue Huang, Yang Bai, Liu Liu, Yuke Wang, Bei Yu, Yufei Ding, and Yuan Xie. Alcop: Automatic load- compute pipelining in deep learning compiler for ai- gpus. In D. Song, M. Carbin, and T. Chen, editors, Pro- ceedings of Machine Learning and Systems, volume 5, pages 680–694. C...

    2023

  16. [24]

    Multi- physics simulations: Challenges and opportunities

    David E Keyes, Lois C McInnes, Carol Woodward, William Gropp, Eric Myra, Michael Pernice, John Bell, Jed Brown, Alain Clo, Jeffrey Connors, et al. Multi- physics simulations: Challenges and opportunities. The International Journal of High Performance Computing Applications, 27...

    2013

  17. [25]

    Llvm: A compilation framework for lifelong program analysis & transforma- tion

    Chris Lattner and Vikram Adve. Llvm: A compilation framework for lifelong program analysis & transforma- tion. In International symposium on code generation and optimization, 2004. CGO 2004., pages 75–86. IEEE, 2004

    2004

  18. [26]

    Mlir: Scaling compiler infrastructure for do- main specific computation

    Chris Lattner, Mehdi Amini, Uday Bondhugula, Albert Cohen, Andy Davis, Jacques Pienaar, River Riddle, Ta- tiana Shpeisman, Nicolas Vasilache, and Oleksandr Zi- nenko. Mlir: Scaling compiler infrastructure for do- main specific computation. In 2021 IEEE/ACM Inter- national Symp...

    2021

  19. [27]

    Deep learning

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436–444, 2015

    2015

  20. [28]

    John Lu and Keith D. Cooper. Register promotion in c programs. SIGPLAN Not., 32(5):308–319, May 1997

    1997

  21. [29]

    Experimental FlashAttention3 using Triton, 2024

    Meta. Experimental FlashAttention3 using Triton, 2024. Version 2024.12.2

    2024

  22. [30]

    NVIDIA Turing GPU Architec- ture Whitepaper, 2018

    NVIDIA Corporation. NVIDIA Turing GPU Architec- ture Whitepaper, 2018

    2018

  23. [31]

    cuBLAS Library, 2023

    NVIDIA Corporation. cuBLAS Library, 2023. Retrieved from https://docs.nvidia.com/cuda/cublas/

    2023

  24. [32]

    CUPTI: CUDA Profiling Tools Interface, 2023

    NVIDIA Corporation. CUPTI: CUDA Profiling Tools Interface, 2023

    2023

  25. [33]

    NVIDIA Nsight Compute, 2024

    NVIDIA Corporation. NVIDIA Nsight Compute, 2024. Version 2022.4

    2024

  26. [34]

    NVIDIA Nsight Systems, 2024

    NVIDIA Corporation. NVIDIA Nsight Systems, 2024. Version 2024.7.1

    2024

  27. [35]

    NVIDIA PTX, 2024

    NVIDIA Corporation. NVIDIA PTX, 2024. Version 8.5

    2024

  28. [36]

    Group GEMM in Triton, 2024

    OpenAI Corpora. Group GEMM in Triton, 2024. Ver- sion 2024.11

    2024

  29. [37]

    Optimizing distributed ml communi- cation with fused computation-collective operations

    Kishore Punniyamurthy, Khaled Hamidouche, and Brad- ford M Beckmann. Optimizing distributed ml communi- cation with fused computation-collective operations. In SC24: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–17. IEEE, 2024

    2024

  30. [38]

    PyTorch Profiler

    PyTorch Core Team. PyTorch Profiler. https://pytorch.org/tutorials/intermediate/ profiler_tutorial.html, 2021. Accessed: 2025-04- 21

    2021

  31. [39]

    Rein- venting high performance computing: challenges and opportunities

    Daniel Reed, Dennis Gannon, and Jack Dongarra. Rein- venting high performance computing: challenges and opportunities. arXiv preprint arXiv:2203.02544, 2022

    2022 arXiv

  32. [40]

    Learning representations by back-propagating errors

    David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors. nature, 323(6088):533–536, 1986

    1986

  33. [41]

    Flashattention- 3: Fast and accurate attention with asynchrony and low- precision

    Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao. Flashattention- 3: Fast and accurate attention with asynchrony and low- precision. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

  34. [42]

    Flashattention- 3: Fast and accurate attention with asynchrony and low- precision

    Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao. Flashattention- 3: Fast and accurate attention with asynchrony and low- precision. arXiv preprint arXiv:2407.08608, 2024

    2024 arXiv

  35. [43]

    Tensor program opti- mization with probabilistic programs

    Junru Shao, Xiyou Zhou, Siyuan Feng, Bohan Hou, Rui- hang Lai, Hongyi Jin, Wuwei Lin, Masahiro Masuda, Cody Hao Yu, and Tianqi Chen. Tensor program opti- mization with probabilistic programs. Advances in Neu- ral Information Processing Systems, 35:35783–35796, 2022

    2022

  36. [44]

    Thunderkittens: Simple, fast, and adorable ai kernels

    Benjamin F Spector, Simran Arora, Aaryan Singhal, Daniel Y Fu, and Christopher Ré. Thunderkittens: Simple, fast, and adorable ai kernels. arXiv preprint arXiv:2410.20399, 2024

    2024 arXiv

  37. [45]

    CUTLASS, January 2023

    Vijay Thakkar, Pradeep Ramani, Cris Cecka, Aniket Shivam, Honghao Lu, Ethan Yan, Jack Kosaian, Mark Hoemmen, Haicheng Wu, Andrew Kerr, Matt Nicely, Duane Merrill, Dustyn Blasig, Fengqi Qiao, Piotr Ma- jcher, Paul Springer, Markus Hohnerbach, Jin Wang, and Manish Gupta. CUTLASS...

    2023

  38. [46]

    Large language models in medicine

    Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. Large language models in medicine. Nature medicine, 29(8):1930–1940, 2023

    1930

  39. [47]

    Triton: an intermediate language and compiler for tiled neural network computations

    Philippe Tillet, Hsiang-Tsung Kung, and David Cox. Triton: an intermediate language and compiler for tiled neural network computations. In Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages, pages 10–19, 2019

    2019

  40. [48]

    Nvbit: A dynamic binary instru- mentation framework for nvidia gpus

    Oreste Villa, Mark Stephenson, David Nellans, and Stephen W Keckler. Nvbit: A dynamic binary instru- mentation framework for nvidia gpus. In Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture, pages 372–383, 2019

    2019

  41. [49]

    Wlb-llm: Workload-balanced 4d paral- lelism for large language model training

    Zheng Wang, Anna Cai, Xinfeng Xie, Zaifeng Pan, Yue Guan, Weiwei Chu, Jie Wang, Shikai Li, Jianyu Huang, Chris Cai, et al. Wlb-llm: Workload-balanced 4d paral- lelism for large language model training. arXiv preprint arXiv:2503.17924, 2025

    2025 arXiv

  42. [50]

    Rap: Resource-aware automated gpu sharing for multi-gpu recommendation model training and input preprocessing

    Zheng Wang, Yuke Wang, Jiaqi Deng, Da Zheng, Ang Li, and Yufei Ding. Rap: Resource-aware automated gpu sharing for multi-gpu recommendation model training and input preprocessing. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming ...

    2024

  43. [51]

    Bloomberggpt: A large language model for finance

    Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravol- ski, Mark Dredze, Sebastian Gehrmann, Prabhan- jan Kambadur, David Rosenberg, and Gideon Mann. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564, 2023

    2023 arXiv

  44. [52]

    Ansor: Generating {High-Performance} tensor programs for deep learn- ing

    Lianmin Zheng, Chengfan Jia, Minmin Sun, Zhao Wu, Cody Hao Yu, Ameer Haj-Ali, Yida Wang, Jun Yang, Danyang Zhuo, Koushik Sen, et al. Ansor: Generating {High-Performance} tensor programs for deep learn- ing. In 14th USENIX symposium on operating systems design and implementatio...

    2020

  45. [53]

    Gvprof: A value profiler for gpu-based clusters

    Keren Zhou, Yueming Hao, John Mellor-Crummey, Xi- aozhu Meng, and Xu Liu. Gvprof: A value profiler for gpu-based clusters. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–16, 2020

    2020

  46. [54]

    Valueexpert: Exploring value patterns in gpu-accelerated applications

    Keren Zhou, Yueming Hao, John Mellor-Crummey, Xi- aozhu Meng, and Xu Liu. Valueexpert: Exploring value patterns in gpu-accelerated applications. In Proceed- ings of the 27th ACM International Conference on Ar- chitectural Support for Programming Languages and Operating Systems...

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.