Pith. sign in

REVIEW 4 major objections 51 references

Modern LLM GPU kernels are best simulated as warp-level tile graphs of data and order dependencies, not as instruction streams.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 05:55 UTC pith:2625BJ2U

load-bearing objection Solid middle-ground GPU simulator for modern LLM kernels: tile graphs + throughput models deliver real A100/H100 accuracy and useful co-design case studies, with the main caveats being TileLang-centric graphs and fitted constants rather than a broken core claim. the 4 major comments →

arxiv 2607.11262 v1 pith:2625BJ2U submitted 2026-07-13 cs.DC

GPU-Tile-Sim: A Tile-Centric GPU Simulation Framework for LLM Hardware-Software Co-Design

Yitong Ding , Jiawei Huang , Renyang Guan , Yangjie Zhou , Zihan Liu , Yu Feng , Shixuan Sun , Mingyi Guo
show 2 more authors
Jingwen Leng Jian Weng
This is my paper
classification cs.DC
keywords GPU simulationLLM workloadstile graphhardware-software co-designperformance modelingwarp specializationasynchronous pipelineskernel fusion
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Modern large-language-model kernels on GPUs win performance by fusing operators, software-pipelining data movement with compute, and specializing warps, so their runtime is controlled more by how tiles depend on and overlap with each other than by the latency of any single instruction. Existing tools either simulate full instruction streams, which are expensive to keep current with new architectures, or use coarse analytical mappings that miss fine-grained asynchronous coordination. GPU-Tile-Sim therefore represents a kernel as a directed acyclic graph of warp-level tile operations whose edges encode both dataflow and ordering constraints; an automatic frontend builds that graph from a tile-level intermediate representation, and a graph-driven backend schedules ready nodes onto throughput models of Tensor Cores, memory, and on-chip interconnect. On A100 and H100, across ordinary and heavily optimized GEMM and attention kernels as well as Llama-3-8B inference, the resulting cycle predictions stay within a few percent of hardware measurements and beat prior analytical models. The same abstraction extends to Blackwell with localized changes and is used to compare pipeline organizations, NoC-enabled fusion, and architecture-aware attention designs.

Core claim

The paper claims that modern LLM kernel performance is governed primarily by the dependency structure that controls tile-level execution order and overlap rather than by individual instruction latency, and that a warp-centric tile graph plus a graph-driven, throughput-oriented simulator is therefore sufficient to model conventional and highly optimized kernels on recent GPUs with low error while remaining extensible to new architectures.

What carries the argument

The warp-centric tile graph: a DAG whose nodes are tile-level operations executed by named warps or warp groups (with operation and tile descriptors) and whose edges are either data edges (producer–consumer) or order edges (synchronization, buffer reuse, same-warp program order).

Load-bearing premise

Once the tile graph encodes the right data and order edges, simple throughput models of compute units, memory bandwidth and latency, and on-chip interconnect are enough to predict cycles for heavily optimized, warp-specialized kernels without simulating full instruction pipelines.

What would settle it

Run the same optimized GEMM, FlashAttention-style, and Llama-3-8B kernels on A100 or H100 and check whether GTSim’s predicted cycles remain within the claimed MAPE range of roughly 1–9 percent against measured cycles; a large, systematic miss would falsify the sufficiency of the tile-graph-plus-throughput model.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Hardware–software co-design of LLM kernels can be explored by editing node roles and dependency edges rather than rewriting instruction-level simulators for each GPU generation.
  • Software-pipeline organizations (naive, warp-specialized cooperative, ping-pong, multi-stage) become first-class objects whose relative benefit can be predicted from the resulting tile-graph structure.
  • Inter-SM communication features such as distributed shared memory and NoC topology can be evaluated for fused attention pipelines by modeling remote tile accesses as NoC traffic.
  • New tensor-core primitives, operand memories, and CTA-pair execution (as on Blackwell) can be added mainly by new node types and resource models while reusing the same dependency-driven runtime.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the tile-graph abstraction continues to hold, compiler and DSL frontends that already expose warp roles and pipeline stages become the natural entry point for performance models, reducing the need for SASS or PTX traces.
  • The same dependency-first view may transfer to other accelerators whose kernels are tile-driven and asynchronous rather than purely instruction-stream driven.
  • Sustained accuracy on future generations will hinge on whether new microarchitectural effects remain expressible as resource throughput plus order edges rather than requiring detailed scoreboards or cache-coherence models.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 0 minor

Summary. The paper proposes GPU-Tile-Sim (GTSim), a tile-centric GPU simulator for LLM hardware–software co-design. Its central claim is that modern LLM kernel performance is governed primarily by tile-level data and ordering dependencies (and the overlap they enable) rather than per-instruction latency. Kernels are represented as warp-centric tile graphs (nodes = tile operations with execution groups; edges = data and order constraints), extracted automatically from TileLang IR after pipeline/warp-specialization lowering, and executed by a graph-driven backend with throughput-oriented compute, memory, TMA, and NoC models. On A100/H100 the authors report MAPE 1.22%–8.71% for GEMM, fused GEMM, FP8 GEMM, FlashAttention-3, Flash-Decoding, FlashMLA, and Llama-3-8B inference, outperforming adapted TileFlow and LLMCompass, with a faster-than-Accel-Sim A100 GEMM comparison, tile-graph ablations, MoE dynamic routing checks, and case studies on software pipelining, NoC-enabled fusion, and a preliminary Blackwell (B200) extension.

Significance. If the accuracy and extensibility claims hold under broader kernel sources and hardware generations, GTSim fills a genuine gap between costly instruction-driven simulators (hard to retarget to Hopper/Blackwell async/warpgroup/TMEM semantics) and coarse mapping/interval analytical models (weak on fusion, software pipelines, and warp specialization). The warp-centric tile graph with explicit order edges is a clean, reusable abstraction; the Tab. 4 ablations (order constraints and cross-warp sync) give direct evidence that dependency structure is load-bearing; and the co-design case studies (pipeline organizations, NoC topology/mapping, FA3 vs FA4 on Blackwell) show practical utility beyond cycle matching. Planned open-sourcing would further strengthen impact for LLM systems research.

major comments (4)
  1. Sec. 6.1 states that real-hardware kernels are “primarily based on TileLang,” and Sec. 4.3’s automatic path consumes TileLang IR after software-pipeline and warp-specialization injection. The headline accuracy claim (Abstract; Figs. 8–10) for “conventional and highly optimized” kernels is therefore demonstrated mainly on graphs whose structure is already explicit in that IR. Without at least one independent validation path against production implementations (e.g., CUTLASS/cuBLAS GEMM, reference FlashAttention-3/FlashMLA CUDA, or a CuBridge-lifted expert kernel as the paper itself cites), it remains unclear whether MAPE 1.22%–8.71% transfers when the same optimized dependency structure must be recovered or hand-specified outside TileLang. This is load-bearing for the claim that GTSim models highly optimized LLM kernels in general, not only TileLang-expressed ones. Please add non-TileLang
  2. Sec. 6.1–6.2: LLMCompass is adapted by modeling each fused kernel as serial constituent operators, and TileFlow is retargeted with H100 bandwidth parameters. On FlashAttention-3/Flash-Decoding (Fig. 9) the baselines often exceed 100% MAPE, which may partly reflect this serial/coarse treatment rather than an inherent ceiling of mapping-based models. The “consistently outperforms prior analytical models” claim is therefore only partially controlled. Please either (i) give fused/dataflow-faithful configurations for TileFlow (and any available fused path for LLMCompass) with explicit mapping fidelity notes, or (ii) reframe the comparison as “against these adapted baselines under stated limitations,” and discuss what a best-effort analytical model could capture without GTSim’s order edges and warp roles.
  3. Sec. 5.2 and Tab. 3: the backend relies on microbenchmark-calibrated free parameters (TMA ~100 B/cycle; L2/DRAM BW/latency; SRAM/TMEM; TC throughput) and a simple fully associative LRU L2 plus latency–bandwidth DRAM/NoC models. Tab. 4 shows that removing order/sync edges destroys accuracy, which supports the dependency thesis, but does not stress when the throughput models themselves fail (e.g., capacity thrashing, bank conflicts, non-tile-aligned reuse, or kernels outside the calibrated TMA/L2 regime). A short sensitivity study (perturb TMA rate / L2 capacity or latency within documented ranges) or an explicit failure-mode discussion is needed so readers can judge how much of the reported MAPE is graph structure versus fitted constants—especially given the weakest assumption that these abstractions suffice for warp-specialized optimized kernels.
  4. Sec. 7.3 / Fig. 14: Blackwell results are only a coarse comparison of simulated FA4 TFLOPS to published FA4 paper numbers, with no local B200 measurements and a simplified FA4 reimplementation. The Abstract’s “extend GTSim to Blackwell with preliminary validation” is appropriately cautious, but the FA3-vs-FA4 causal analysis (wave count, TMEM decoupling, exp approximation) is presented as if the absolute performance level is established. Please keep absolute B200 claims strictly preliminary, report how published FA4 numbers were extracted/normalized, and separate qualitative co-design insight from quantitative validation until device measurements exist.

Circularity Check

0 steps flagged

No significant circularity: GTSim is a calibrated simulator whose MAPE claims are checked against external real-device cycle counts, not quantities defined by the model itself.

full rationale

The paper's central claim is empirical accuracy of a new tile-graph simulator (MAPE 1.22%–8.71% on A100/H100 GEMM/attention/Llama-3-8B kernels, outperforming adapted TileFlow/LLMCompass). Hardware parameters (Tab. 3: TMA issue rate, L2/DRAM latency-bandwidth, etc.) are taken from docs, prior studies, and microbenchmarks, then used to drive graph simulation of full kernels; the reported errors are measured against independent real-hardware cycle counts (Figs. 8–10) and published FA4 numbers. Ablations (Tab. 4) that remove order edges or cross-warp sync increase MAPE, confirming the dependency representation has independent content rather than tautologically reproducing inputs. There is no self-definitional loop (tile graph is not defined via the target MAPE), no fitted parameter renamed as a prediction of the same quantity, no load-bearing uniqueness theorem imported from overlapping authors, and no ansatz smuggled via self-citation. Standard model calibration + external validation is not circularity under the stated criteria. The derivation chain (frontend IR → tile graph → throughput models → scheduled cycles) is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

The central accuracy claim rests on domain GPU scheduling facts, a modeling axiom that tile dependencies dominate instruction latency, and many measured hardware constants. The main invented construct is the warp-centric tile graph itself; independent evidence for it is the external hardware MAPE, not a formal proof.

free parameters (5)
  • TMA issue rate
    Calibrated from microbenchmarks (~100 B/cycle on Hopper/Blackwell models); directly affects memory-node timing.
  • L2/DRAM bandwidth and latency (per GPU)
    Tab. 3 values from docs, prior studies, and measurements; dominate memory-bound predictions.
  • SRAM/TMEM bandwidth and latency
    Modeled constants for on-chip paths; TMEM added for Blackwell.
  • Tensor Core / CUDA core / SFU throughput and pipeline width
    Architecture-table throughputs used to decompose nodes into sub-operations.
  • NoC bandwidth/latency/topology for DSMEM clusters
    Simplified cluster NoC parameters used in fusion/mapping case study.
axioms (5)
  • ad hoc to paper Modern LLM kernel performance is governed primarily by tile-level data movement, computation, and synchronization dependencies/overlap rather than individual instruction latency.
    Stated as the key insight (Abstract, Sec. 3); load-bearing for dropping instruction-level simulation.
  • domain assumption The warp is the right modeling granularity for scheduling units, specialization, and warp-group collectives on modern NVIDIA GPUs.
    Sec. 4.1 design choice; standard GPU fact but essential to the abstraction.
  • domain assumption Tile descriptors (shape, layout, coordinates, storage) suffice to expand memory traffic without per-thread addresses.
    Sec. 5.2 memory model; enables lightweight simulation.
  • ad hoc to paper Simple fully associative LRU L2 plus latency-bandwidth DRAM/NoC models capture dominant reuse and contention for evaluated LLM kernels.
    Sec. 5.2 admits simplicity; accuracy claim depends on this being enough.
  • domain assumption TileLang IR after pipeline/warp-specialization lowering preserves the execution structure needed for correct tile graphs.
    Sec. 4.3 frontend path; alternative manual graph construction assumed equivalent.
invented entities (2)
  • Warp-centric tile graph (nodes = tile ops with execution groups; data + order edges) independent evidence
    purpose: Unified representation of fused, pipelined, warp-specialized kernel execution for graph-driven simulation.
    Core abstraction introduced by the paper; not a physical particle but a new modeling entity the claims depend on.
  • GTSim graph-driven backend with ready-node issue and sub-operation decomposition independent evidence
    purpose: Simulate concurrency/contention without full instruction pipelines.
    Implementation of the method; evidence is empirical MAPE vs hardware.

pith-pipeline@v1.1.0-grok45 · 27628 in / 3335 out tokens · 36006 ms · 2026-07-14T05:55:00.317817+00:00 · methodology

0 comments
read the original abstract

Modern LLM (large language model) workloads increasingly rely on optimized GPU kernels through hardware-software co-design. These kernels achieve high-performance through fine-grained dependency scheduling and computation-memory overlap. As such, they incur new challenges on existing GPU performance models. Instruction-driven simulators are costly to adapt to evolving architectures, while analytical models are too coarse to capture kernels' characteristics. We propose GPU-Tile-Sim, a tile-centric GPU simulation framework for LLM hardware-software co-design. The key insight is that modern LLM kernel performance is governed less by individual instruction latency than by the dependency structure that controls execution order and overlap. Accordingly, GTSim represents kernel execution as a warp-level tile graph whose nodes capture tile-level operations and whose edges encode data and ordering constraints. Using this representation, we design an automatic tile-graph frontend and a graph-driven simulation backend. We evaluate GTSim on representative GEMM, attention, and end-to-end LLM inference workloads. On A100 and H100 across both conventional and highly optimized kernels, GTSim achieves high performance-modeling accuracy (MAPE, Mean Absolute Percentage Error, 1.22%--8.71%). We further extend GTSim to Blackwell with preliminary validation, and demonstrate its effectiveness in analyzing software and architectural design choices.

Figures

Figures reproduced from arXiv: 2607.11262 by Jian Weng, Jiawei Huang, Jingwen Leng, Mingyi Guo, Renyang Guan, Shixuan Sun, Yangjie Zhou, Yitong Ding, Yu Feng, Zihan Liu.

Figure 1
Figure 1. Figure 1: Representative kernel patterns in LLM workloads. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Architectural evolution across GPU generations. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of GTSim. roles explicit within a unified abstraction, rather than recovering them indirectly from instruction streams or coarse mappings [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Tile graph examples. (a) Simplified node and edge semantics. (b) A simple fused-GEMM tile graph with producer and [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Representative software pipeline organizations for [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Automatic tile-graph generation from TileLang IR. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Simulation workflow of the graph-driven simulator. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Parity plots for GEMM workloads on A100 and H100, comparing GTSim, TileFlow, and LLMCompass. [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: MAPE of GTSim, TileFlow, and LLMCompass on FlashAttention-3, Flash-Decoding, and FlashMLA. [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Measured and simulated results of Llama-3-8B in [PITH_FULL_IMAGE:figures/full_fig_p010_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Runtime and accuracy comparison on eight A100 square GEMMs. (a) Runtime normalized to Accel-Sim. (b) Simulated cycles normalized to hardware measurements. The error remains stable as both batch size and KV length increase, showing that GTSim can robustly model long-KV decode behavior beyond individual kernels [PITH_FULL_IMAGE:figures/full_fig_p010_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Software pipelining case study. (a) Normalized [PITH_FULL_IMAGE:figures/full_fig_p011_12.png] view at source ↗
Figure 14
Figure 14. Figure 14: FA3 vs. FA4 on B200. Left: TFLOPS. Middle: wave [PITH_FULL_IMAGE:figures/full_fig_p012_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 12 linked inside Pith

  1. [1]

    Yuan, Wilson W

    Ali Bakhoda, George L. Yuan, Wilson W. L. Fung, Henry Wong, and Tor M. Aamodt. 2009. Analyzing CUDA workloads using a detailed GPU simulator. In IEEE ISPASS

    2009

  2. [2]

    Reinhardt, Ali G

    Nathan Binkert, Bradford Beckmann, Gabriel Black, Steven K. Reinhardt, Ali G. Saidi, Arkaprava Basu, Joel Hestness, Derek R. Hower, Tushar Krishna, Somayeh Sardashti, Rathijit Sen, Korey Sewell, Muhammad Shoaib, Nilay Vaish, Mark D. Hill, and David A. Wood. 2011. The gem5 Simulator. InACM SIGARCH Computer Architecture News. 1–7. https://doi.org/10.1145/20...

    doi:10.1145/2024716.2024718 2011

  3. [3]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    Pith/arXiv arXiv 2020

  4. [4]

    Shiheng Cao, Junmin Wu, Junshi Chen, Hong An, and Zhibin Yu. 2025. AMALI: An Analytical Model for Accurately Modeling LLM Inference on Modern GPUs. InProceedings of the ACM/IEEE International Symposium on Computer Architecture (ISCA). 1495–1508. https://doi.org/10.1145/3695053.3731064

    doi:10.1145/3695053.3731064 2025

  5. [5]

    Hongzheng Chen, Bin Fan, Alexander Collins, Bastian Hagedorn, Evghenii Gaburov, Masahiro Masuda, Matthew Brookhart, Chris Sullivan, Jason Knight, Zhiru Zhang, and Vinod Grover. 2025. Tawa: Automatic Warp Specialization for Modern GPUs with Asynchronous References.arXiv preprint arXiv:2510.14719 (2025). https://arxiv.org/abs/2510.14719

    arXiv 2025

  6. [6]

    Rosen, Mark N

    Ron Cytron, Jeanne Ferrante, Barry K. Rosen, Mark N. Wegman, and F. Kenneth Zadeck. 1991. Efficiently Computing Static Single Assignment Form and the Control Dependence Graph.ACM Transactions on Programming Languages and Systems13, 4 (1991), 451–490. https://doi.org/10.1145/115372.115320

    doi:10.1145/115372.115320 1991

  7. [7]

    Tri Dao. 2023. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning.arXiv preprint arXiv:2307.08691(2023). https://arxiv.org/abs/ 2307.08691

    Pith/arXiv arXiv 2023

  8. [8]

    Fu, Stefano Ermon, Atri Rudra, and Christopher Ré

    Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. arXiv preprint arXiv:2205.14135(2022). https://arxiv.org/abs/2205.14135

    Pith/arXiv arXiv 2022

  9. [9]

    Tri Dao, Daniel Haziza, Francisco Massa, and Grigory Sizov. 2023. Flash-Decoding for Long-Context Inference. https://crfm.stanford.edu/2023/10/12/flashdecoding. html. Stanford CRFM blog post, Accessed: 2026-03-18

    2023

  10. [10]

    DeepSeek-AI. 2024. DeepSeek-V3 Technical Report.arXiv preprint arXiv:2412.19437(2024). https://arxiv.org/abs/2412.19437

    Pith/arXiv arXiv 2024

  11. [11]

    gpgpu-sim contributors. 2025. GPGPU-Sim Distribution. https://github.com/ gpgpu-sim/gpgpu-sim_distribution. Official repository README states the distribution is tested with a subset of CUDA versions up to 11, Accessed: 2026- 03-31

    2025

  12. [12]

    Riasanovsky, Manman Ren, Lei Wang, Shane Nay, Partha Kanuparthy, Zhijing Li, Ying Liu, Zaifeng Pan, Zhengding Hu, and Yufei Ding

    Yue Guan, Hongtao Yu, Peng Chen, Daohang Shi, Karthik Manivannan, Nicholas J. Riasanovsky, Manman Ren, Lei Wang, Shane Nay, Partha Kanuparthy, Zhijing Li, Ying Liu, Zaifeng Pan, Zhengding Hu, and Yufei Ding. 2026. TLX: Hardware- Native, Evolvable MIMW GPU Compiler for Large-scale Production Environ- ments.arXiv preprint arXiv:2605.10905(2026). https://arx...

    Pith/arXiv arXiv 2026

  13. [13]

    Jen-Cheng Huang, Joo Hwan Lee, Hyesoon Kim, and Hsien-Hsin S. Lee. 2014. GPUMech: GPU Performance Modeling Technique Based on Interval Analysis. In47th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). 268–279. https://doi.org/10.1109/MICRO.2014.59

    doi:10.1109/micro.2014.59 2014

  14. [14]

    Soojin Hwang, Sunho Lee, Jungwoo Kim, Hongbeen Kim, and Jaehyuk Huh. 2023. mNPUsim: Evaluating the Effect of Sharing Resources in Multi-core NPUs. In IEEE International Symposium on Workload Characterization (IISWC). 167–179. https://doi.org/10.1109/IISWC59245.2023.00018

    doi:10.1109/iiswc59245.2023.00018 2023

  15. [15]

    Hyndman and Anne B

    Rob J. Hyndman and Anne B. Koehler. 2006. Another Look at Measures of Forecast Accuracy.International Journal of Forecasting22, 4 (2006), 679–688. https://doi.org/10.1016/j.ijforecast.2006.03.001

    doi:10.1016/j.ijforecast.2006.03.001 2006

  16. [16]

    Aaron Jarmusch and Sunita Chandrasekaran. 2025. Microbenchmarking NVIDIA’s Blackwell Architecture: An in-depth Architectural Analysis.arXiv preprint arXiv:2512.02189(2025). https://arxiv.org/abs/2512.02189

    arXiv 2025

  17. [17]

    Aamodt, and Timothy G

    Mahmoud Khairy, Zhesheng Shen, Tor M. Aamodt, and Timothy G. Rogers. 2020. Accel-Sim: An Extensible Simulation Framework for Validated GPU Modeling. InISCA

    2020

  18. [18]

    Hyoukjun Kwon, Michael Pellauer, and Tushar Krishna. 2019. MAESTRO: A Data-Centric Approach to Understanding Reuse, Performance, and Hardware Cost of DNN Mappings. InMICRO

    2019

  19. [19]

    Jounghoo Lee, Yeonan Ha, Suhyun Lee, Jinyoung Woo, Jinho Lee, Hanhwi Jang, and Youngsok Kim. 2022. GCoM: A Detailed GPU Core Model for Accurate Analytical Modeling of Modern GPUs. InProceedings of the IEEE/ACM Interna- tional Symposium on Microarchitecture (MICRO). 424–436. https://doi.org/10. 1145/3470496.3527384

    arXiv 2022

  20. [20]

    Liqiang Lu, Naiqing Guan, Yuyue Wang, Liancheng Jia, Zizhang Luo, Jieming Yin, Jason Cong, and Yun Liang. 2021. TENET: A Framework for Modeling Tensor Dataflow Based on Relation-centric Notation. In48th ACM/IEEE Annual International Symposium on Computer Architecture (ISCA). 720–733. https://doi. org/10.1109/ISCA52012.2021.00062

    doi:10.1109/isca52012.2021.00062 2021

  21. [21]

    Weile Luo, Ruibo Fan, Zeyu Li, Dayou Du, Hongyuan Liu, Qiang Wang, and Xiaowen Chu. 2025. Dissecting the NVIDIA Hopper Architecture through Mi- crobenchmarking and Multiple Level Analysis.arXiv preprint arXiv:2501.12084 (2025). https://arxiv.org/abs/2501.12084

    Pith/arXiv arXiv 2025

  22. [22]

    Xinhao Luo, Zihan Liu, Yangjie Zhou, Shihan Fang, Ziyu Huang, Yu Feng, Chen Zhang, Shixuan Sun, Zhenzhe Zheng, Jingwen Leng, and Minyi Guo. 2025. Clus- terFusion: Expanding Operator Fusion Scope for LLM Inference via Cluster-Level Collective Primitive. arXiv:2508.18850 [cs.DC] https://arxiv.org/abs/2508.18850

    Pith/arXiv arXiv 2025

  23. [23]

    Xing Ma, Yangjie Zhou, Wu Sun, Zihan Liu, Jingwen Leng, Yun Lin, Shixuan Sun, Minyi Guo, and Jin Song Dong. 2026. CuBridge: An LLM-Based Framework for Understanding and Reconstructing High-Performance Attention Kernels.arXiv preprint arXiv:2605.05023(2026). https://arxiv.org/abs/2605.05023

    Pith/arXiv arXiv 2026

  24. [24]

    Paulius Micikevicius, Dusan Stosic, Neil Burgess, Marius Cornea, Pradeep Dubey, Richard Grisenthwaite, Sangwon Ha, Alexander Heinecke, Patrick Judd, John Kamalu, Naveen Mellempudi, Stuart Oberman, Mohammad Shoeybi, Michael Siu, and Hao Wu. 2022. FP8 Formats for Deep Learning.arXiv preprint arXiv:2209.05433(2022). https://arxiv.org/abs/2209.05433

    Pith/arXiv arXiv 2022

  25. [25]

    NVIDIA Corporation. [n. d.]. CUTLASS: Fast Linear Algebra in CUDA C++. https://github.com/NVIDIA/cutlass. Official GitHub repository, Accessed: 2026- 03-18

    2026

  26. [26]

    NVIDIA Corporation. 2017. NVIDIA Tesla V100 GPU Architecture. https://images.nvidia.com/content/volta-architecture/pdf/volta-architecture- whitepaper.pdf. Accessed: 2026-03-18

    2017

  27. [27]

    NVIDIA Corporation. 2018. NVIDIA Turing GPU Architecture. https://images. nvidia.com/aem-dam/en-zz/Solutions/design-visualization/technologies/ turing-architecture/NVIDIA-Turing-Architecture-Whitepaper.pdf. Accessed: 2026-03-31. 13 MICRO 2026, October 31–November 04, 2026, Athens, Greece Yitong Ding et al

    2018

  28. [28]

    NVIDIA Corporation. 2020. NVIDIA A100 Tensor Core GPU Architec- ture. https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/nvidia- ampere-architecture-whitepaper.pdf. Accessed: 2026-03-18

    2020

  29. [29]

    NVIDIA Corporation. 2022. NVIDIA H100 GPU Whitepaper. https://resources. nvidia.com/en-us-hopper-architecture/nvidia-h100-tensor-c. Official NVIDIA whitepaper landing page, Accessed: 2026-03-18

    2022

  30. [30]

    NVIDIA Corporation. 2025. Programming Blackwell Tensor Cores with CUTLASS. NVIDIA GTC 2025 Technical Session (S72720). https://www.nvidia.com/en- us/on-demand/session/gtc25-s72720/, Accessed: 2026-03-18

    2025

  31. [31]

    NVIDIA Corporation. 2026. CUDA C++ Programming Guide. https://docs.nvidia. com/cuda/cuda-programming-guide/index.html. Accessed: 2026-03-18

    2026

  32. [32]

    Keckler, Joel S

    Angshuman Parashar, Priyanka Raina, Yakun Sophia Shao, Yu-Hsin Chen, Vaish- nav Gopal, Shashank Agrawal, Brucek Khailany, Stephen W. Keckler, Joel S. Emer, and Vivienne Sze. 2019. Timeloop: A Systematic Approach to DNN Accelerator Evaluation. InISPASS

    2019

  33. [33]

    Orr, Mark D

    Jason Power, Joel Hestness, Marc S. Orr, Mark D. Hill, and David A. Wood. 2015. gem5-gpu: A Heterogeneous CPU-GPU Simulator.IEEE Computer Architecture Letters14, 1 (2015), 34–36. https://doi.org/10.1109/LCA.2014.2299539

    doi:10.1109/lca.2014.2299539 2015

  34. [34]

    Rogers, Mike O’Connor, and Tor M

    Timothy G. Rogers, Mike O’Connor, and Tor M. Aamodt. 2012. Cache-Conscious Wavefront Scheduling. InProceedings of the 2012 45th Annual IEEE/ACM Interna- tional Symposium on Microarchitecture. 72–83. https://doi.org/10.1109/MICRO. 2012.16

    doi:10.1109/micro 2012

  35. [35]

    Whatmough, Matthew Mattina, and Tushar Krishna

    Ananda Samajdar, Jan Moritz Joseph, Yuhao Zhu, Paul N. Whatmough, Matthew Mattina, and Tushar Krishna. 2020. A Systematic Methodology for Character- izing Scalability of DNN Accelerators using SCALE-Sim. InIEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). 58–68. https://doi.org/10.1109/ISPASS48437.2020.00016

    doi:10.1109/ispass48437.2020.00016 2020

  36. [36]

    Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao. 2024. FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision.arXiv preprint arXiv:2407.08608(2024). https://arxiv.org/abs/ 2407.08608

    Pith/arXiv arXiv 2024

  37. [37]

    Yifan Sun, Trinayan Baruah, Saiful A. Mojumder, Shi Dong, Xiang Gong, Shane Treadway, Yuhui Bao, Spencer Hance, Carter McCardwell, Vincent Zhao, Harrison Barclay, Amir Kavyan Ziabari, Zhongliang Chen, Rafael Ubal, José L. Abellán, John Kim, Ajay Joshi, and David R. Kaeli. 2019. MGPUSim: Enabling Multi-GPU Performance Modeling and Optimization. InACM/IEEE ...

    arXiv 2019

  38. [38]

    Tile-AI. 2025. TileLang GitHub Repository. https://github.com/tile-ai/tilelang. Accessed: 2026-03-29

    2025

  39. [39]

    Blaise Tine, Krishna Praveen Yalamarthy, Fares Elsabbagh, and Kim Hyesoon

  40. [40]

    In54th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)

    Vortex: Extending the RISC-V ISA for GPGPU and 3D-Graphics. In54th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). 754–

  41. [41]

    https://doi.org/10.1145/3466752.3480128

    doi:10.1145/3466752.3480128

  42. [42]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lam- ple. 2023. LLaMA: Open and Efficient Foundation Language Models.arXiv preprint arXiv:2302.13971(2023)

    Pith/arXiv arXiv 2023

  43. [43]

    Gomez, Łukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. InAdvances in Neural Information Processing Systems 30 (NeurIPS 2017). 5998–6008. https://proceedings.neurips.cc/paper/7181-attention-is-all-you-need

    2017

  44. [44]

    Oreste Villa, Daniel Lustig, Zi Yan, Evgeny Bolotin, Yaosheng Fu, Niladrish Chatterjee, Nan Jiang, and David Nellans. 2021. Need for Speed: Experi- ences Building a Trustworthy System-Level GPU Simulator. InIEEE Interna- tional Symposium on High-Performance Computer Architecture (HPCA). 868–880. https://doi.org/10.1109/HPCA51647.2021.00077

    doi:10.1109/hpca51647.2021.00077 2021

  45. [45]

    2016.Understanding Latency Hiding on GPUs

    Vasily Volkov. 2016.Understanding Latency Hiding on GPUs. Ph. D. Disserta- tion. University of California, Berkeley. https://www.escholarship.org/uc/item/ 1wb7f3h4

    2016

  46. [46]

    Lei Wang, Yu Cheng, Yining Shi, Zhengju Tang, Zhiwen Mo, Wenhao Xie, Lingx- iao Ma, Yuqing Xia, Jilong Xue, Fan Yang, and Zhi Yang. 2025. TileLang: A Composable Tiled Programming Model for AI Systems. arXiv:2504.17577 [cs.LG] https://arxiv.org/abs/2504.17577

    Pith/arXiv arXiv 2025

  47. [47]

    Lu Wang, Magnus Jahre, Almutaz Adileho, and Lieven Eeckhout. 2020. MDM: The GPU Memory Divergence Model. InProceedings of the 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). 1009–1021. https://doi. org/10.1109/MICRO50266.2020.00085

    doi:10.1109/micro50266.2020.00085 2020

  48. [48]

    Wonhyuk Yang, Yunseon Shin, Okkyun Woo, Geonwoo Park, Hyungkyu Ham, Jeehoon Kang, Jongse Park, and Gwangsun Kim. 2025. PyTorchSim: A Compre- hensive, Fast, and Accurate NPU Simulation Framework. In58th IEEE/ACM International Symposium on Microarchitecture (MICRO). 1363–1380. https: //doi.org/10.1145/3725843.3756045

    doi:10.1145/3725843.3756045 2025

  49. [49]

    Ted Zadouri, Markus Hoehnerbach, Jay Shah, Timmy Liu, Vijay Thakkar, and Tri Dao. 2026. FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling.arXiv preprint arXiv:2603.05451(2026). https: //arxiv.org/abs/2603.05451

    arXiv 2026

  50. [50]

    Hengrui Zhang, August Ning, Rohan Baskar Prabhakar, and David Wentzlaff. 2024. LLMCompass: Enabling Efficient Hardware Design for Large Language Model Inference. InProceedings of the 51st Annual International Symposium on Computer Architecture (ISCA). 1080–1096. https://doi.org/10.1109/ISCA59077.2024.00082

    doi:10.1109/isca59077.2024.00082 2024

  51. [51]

    Size Zheng, Siyuan Chen, Siyuan Gao, Liancheng Jia, Guangyu Sun, Runsheng Wang, and Yun Liang. 2023. TileFlow: A Framework for Modeling Fusion Dataflow via Tree-based Analysis. In56th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). https://doi.org/10.1145/3613424.3623792 14

    doi:10.1145/3613424.3623792 2023