Pith. sign in

REVIEW 3 major objections 5 minor 65 references

A training-free tile-centric model predicts GPU kernel latency to within about 12 percent and cache hit rates to within one point across four GPU generations.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 04:44 UTC pith:YJSD7HIN

load-bearing objection A genuinely useful first-principles tile-level GPU cost model with unusually broad validation; the deep-K cache optimism is real but looks like an addressable limitation, not a fatal flaw. the 3 major comments →

arxiv 2607.22432 v1 pith:YJSD7HIN submitted 2026-07-24 cs.DC cs.PF

TileSight: A First-Principles Tile-Centric Analytical GPU Performance Model from Cores to Clusters

Zhiwen Mo , Yu Cheng , Lei Wang , Zhengju Tang , Lei Xu , Guoyu Li , Yuqi Dong , Lingxiao Ma
show 7 more authors
Yuqing Xia Jilong Xue Fan Yang Luo Mai Zhi Yang Wayne Luk Hongxiang Fan
This is my paper
classification cs.DC cs.PF
keywords GPU performance modelinganalytical performance modeltile-centric programmingcache reuse distancepipeline overlaptensor-core GEMMdistributed inferenceLLM serving
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

TileSight tries to establish that the tile — the same unit modern GPU kernels are written in — can also be the unit of performance analysis, making kernel latency, cache behavior, and distributed communication fall out of one analytical engine. Its central claim is that with only one-time microbenchmark calibration per GPU, it predicts 703 tensor-core GEMM shapes across A100, H200, B200, and B6000 to a pooled mean absolute percentage error of 12.35%, beating prior learned and roofline-style predictors, and keeps L2 cache hit rates within about one percentage point. It extends the same tile abstraction to multi-GPU fused kernels and end-to-end LLM serving, reaching 16.18% and 13.52% weighted MAPE. A sympathetic reader would care because tile-centric programming is now the main way people write high-performance AI kernels, and existing tools are either coarse bounds, opaque learned models, or post-hoc profilers. The paper flags one regime where its uniform-SM assumption is optimistic — deep-K GEMMs — so the overall claim comes with that caveat.

Core claim

On the paper's own terms, the discovery is that a tile execution plan — a graph of tile actions annotated with per-tile resource vectors over tensor-core, CUDA-core, SFU, on-chip tensor memory, shared memory, L1.5, L2, DDR, and network pipelines — is a sufficient description of a kernel to predict how it will run. Tiles carry dependencies, concurrency, and execution order; a recursive prologue–steady–epilogue envelope computes overlap; a tile-granular reuse-distance analysis with a Gaussian approximation of the stochastic cache model derives multi-level cache hit rates from the schedule; and remote tensor accesses become tiles with a network resource entry routed through an alpha–beta stage

What carries the argument

The central machinery is the tile execution plan: each tile carries a resource vector, a vector of times on independently schedulable hardware pipelines; producer–consumer edges fix legal orderings; and a recursive pipeline envelope T = T_pro + max(N−d,0)T_steady + T_epi, with effective depth d = stages × resident tiles per SM − 1, models overlap. Cache behavior is derived from tile reuse distance — the number of distinct tile-sized blocks accessed between two uses of the same block — evaluated with a Gaussian approximation of the stochastic cache model, so L1.5/L2/DDR traffic fractions fall out of the schedule instead of trace simulation. Distributed movement enters the same vector as a net

Load-bearing premise

The load-bearing premise is that all SMs advance through the tile grid in lockstep, so cache traffic fractions can be computed from a single reuse-distance sequence; the paper's own deep-K GEMM case, 82% predicted versus 43% measured L2 hit rate, shows this assumption gives way exactly when SMs desynchronize.

What would settle it

Run a suite of high-occupancy GEMMs with very large K, for example K in the tens of thousands, on a modern GPU and compare predicted versus measured L2 hit rate and latency. If hit-rate predictions are systematically optimistic in this regime, as the paper's own 82%-versus-43% example suggests, then the uniform-SM premise fails precisely for the workloads that dominate large-batch AI serving, and the central latency claim should be scoped accordingly.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Kernel developers can evaluate tile shapes, swizzles, pipeline depths, and block counts analytically before running, replacing blind autotuning on the target machine.
  • The model transfers across GPU generations from one-time microbenchmarks, so new hardware needs only rate calibration, not re-training or corpus collection.
  • Cache hit-rate prediction within about one percentage point makes L2-locality optimizations such as traversal order and persistent blocks testable inside a cost model.
  • The same tile abstraction scales to multi-GPU fused kernels and end-to-end serving, so distributed scheduling and collective choice can be reasoned about before deployment.
  • Used as a compiler cost model, it can prune 95% of schedule candidates while retaining 99.66% of exhaustive-search best performance.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the optimistic deep-K bias comes from the uniform-SM-progress assumption, adding a model of SM desynchronization, such as per-wave jitter on reuse distances, should close the gap without abandoning the analytical approach.
  • The tile-granularity reuse-distance idea is not tied to one vendor; any accelerator with tile-shaped work and hierarchical caches could be calibrated the same way, though latency-bound small-batch kernels remain outside the current model.
  • A direct consequence the paper leaves implicit is that if this holds, learned predictors in compilers and schedulers could be replaced or bootstrapped by analytical models, reducing the cost and opacity of performance engineering for AI workloads.
  • The single-percentage-point L2 hit-rate claim suggests cache hierarchies can be modeled without detailed traces, which could make analytical models competitive with simulation at a fraction of the cost.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. TileSight proposes a tile-centric analytical GPU performance model that spans three levels: intra-tile resource-vector modeling, inter-tile scheduling and reuse-distance cache modeling, and cross-device alpha-beta communication modeling. The paper claims that, using only one-time per-architecture microbenchmark calibration, TileSight predicts single-GPU GEMM latency with 12.35% pooled MAPE across 703 BF16/FP16 tensor-core shapes on A100, B200, B6000, and H200, outperforming Roofline, NeuSight, PipeWeave, and GenZ. It further reports L2 hit-rate predictions within about one percentage point MAE on 4,680 persistent-kernel cases, 16.18% wMAPE on 304 distributed fused kernels, 13.52% wMAPE on 166 vLLM decode configurations, and competitive results as a schedule-pruning cost model.

Significance. If the central claims hold, this is a substantial contribution: it provides a white-box, schedule-sensitive, architecture-transferable performance model that matches the tile abstraction used by modern GPU programming frameworks, and it can serve as both a diagnostic tool and an analytical cost model for autotuning. The evaluation is unusually broad in hardware coverage (Ampere, Hopper, Blackwell, and an AMD GPU), workload breadth (GEMM, attention, collectives, fused distributed kernels, end-to-end serving), and comparison to learned and analytical baselines. The methodology is transparent about its calibrated hardware parameters, and the paper is honest about its limitations. The main risk is that the load-bearing cache-reuse model has a known systematic bias in the deep-K regime, and the paper does not quantify this bias's downstream effect on the headline latency MAPE.

major comments (3)
  1. [§5.3, §7, Eq. (1)] The uniform-SM progress assumption is load-bearing. §5.3 reports a GEMM with M=N=8192, K=28672 on H200 where TileSight predicts 82% L2 hit rate vs. 43% measured. Because the L1.5/L2/DDR traffic split computed in §3.5 populates the resource vector (Eq. 1) and flows into the pipeline envelope (Eqs. 2–5), this 39 pp error can bias latency predictions whenever the kernel is memory-side bound. The paper calls the regime 'rare' and 'mildly optimistic' but gives no distribution: how many of the 703 GEMM shapes fall into the deep-K/high-occupancy regime, and what is the latency MAPE restricted to those shapes? The workloads in Figure 12 include 8192×28672×8192 and 2048×12288×49152, which are exactly this regime. Please report the conditional error, and either correct the model or qualify the abstract's 'within roughly one percentage point' and the pooled 12.35% latency claim.
  2. [§5 (all)] The central empirical claims rest on 703 GEMM shapes, 4,680 cache cases, 304 distributed cases, and 166 vLLM configurations, but no code, data, or scripts are provided; the paper only promises open-sourcing 'upon publication.' For a systems paper whose main evidence is a numerical evaluation, this is insufficient for verification. Please release the implementation and the exact workload/config lists, and report run-to-run variance or confidence intervals for the reported MAPEs, since GPU latency measurements are noisy. At minimum, a detailed hardware-parameter table (e.g., effective L2 capacity B_T, per-pipeline calibrated rates) should be included as supplementary material.
  3. [§5.2, Fig. 5] The pooled 12.35% MAPE aggregates four GPU families with very different per-architecture errors (5.2% on B6000, 18.7% on A100). The paper does not report how the 703 shapes are distributed across architectures, nor whether the pooled number weights each shape equally or each architecture equally. This makes the transferability claim hard to assess. Also, the GEMM evaluation filters out stream-K and SIMT fallback paths; the number of filtered shapes and their behavior should be reported, since these paths are common in production GEMM workloads and are not covered by the headline claim.
minor comments (5)
  1. [Abstract and §5.3] The abstract says 'L2 cache-hit-rate predictions land within roughly one percentage point of the measured rate on every GPU.' This is true as MAE over the 4,680-case sweep, but the text should say 'mean absolute error' and give a quantile or outlier count, because the deep-K example in §5.3 is a 39 pp error on a single configuration.
  2. [§5.1] The baseline comparison is transparent but not apples-to-apples: PipeWeave falls back to H800 models for H200/B200, and NeuSight is retrained on PipeWeave's FP16 data. The text should state that the distributed and end-to-end baseline comparisons partly reflect dataset availability rather than only model quality, and consider adding an analytical/simulation baseline for the cache model.
  3. [§3.5.2] The symbol A is used for cache associativity in Eqs. (7)–(9), which is confusing given the matrix A in GEMM discussions. Rename to 'assoc' or a similar symbol.
  4. [References] Reference [12] appears incomplete: 'CUTLASS acce.' is not a valid author string. Reference [15] is a news brief; for a claim about CUDA Tile, consider citing NVIDIA's official documentation instead.
  5. [§1, Table 1] The 'No kernel profiling/training' mark is correct but should be clarified: the model uses per-architecture microbenchmarks for calibration. Without this clarification, 'first-principles' may be read as parameter-free rather than training-free.

Circularity Check

0 steps flagged

No significant circularity: microbenchmark-calibrated parameters are reused to predict externally measured latencies and hit rates, and the acknowledged deep-K miss shows the model is falsifiable rather than tautological.

full rationale

TileSight's derivation chain is not circular. Per-tile resource vectors (Eq. 1) are formed from operation, footprint, placement, and one-time hardware rates calibrated by bandwidth/throughput sweeps and short probes; the pipeline envelope (Eqs. 2–5), reuse-distance/SDCM cache model (Eqs. 6–10), and α–β stage cost (Eq. 11) are analytical compositions of those inputs. The claimed predictions — 12.35% pooled GEMM MAPE, L2 hit-rate MAEs, and 16.18%/13.52% distributed wMAPEs — are compared against external ground truth (cutlass_profiler, NCU, vLLM measurements), so no target quantity is set equal to an input by construction. The L2 validation is a genuine out-of-sample check, and the paper's own §5.3 counterexample (deep-K GEMM: 82% predicted vs. 43% measured L2 hit rate) is direct evidence that the cache model is falsifiable rather than fitted to those rates. The effective-cache-capacity calibration from bandwidth sweeps is a legitimate hardware-parameter measurement, not a fit to the hit-rate target. The uniform-SM assumption in §3.5.3/§7 is an acknowledged accuracy limitation, and the abstract's 'within roughly one percentage point' overstates the deep-K behavior; however, overclaiming and missing quantification are correctness risks, not circularity. The TileLang [10] and Tile-AI related citations involve overlapping authors but are motivational descriptions of existing DSLs, not load-bearing results; no uniqueness theorem or ansatz is imported from self-citation. Therefore no circular step is exhibited.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The model is analytical but carries calibrated hardware constants and a lockstep-cache assumption; the central claim does not fit any constants to the target GEMM/vLLM results, which is what keeps circularity low. The main risk is that the per-GPU 'effective cache capacity' and per-link alpha/beta are microbenchmark-derived inputs, not independently specified vendor facts.

free parameters (4)
  • Effective L2 cache capacity per GPU (B_T) = not tabulated; B200 L2 cliff ~83 MB, B6000 ~130 MB from Fig. 2
    SDCM (§3.5.2) requires cache capacity in tile units; TileSight calibrates effective capacity from bandwidth sweeps in §3.8/Figure 2 rather than using vendor spec. This fitted hardware parameter directly determines predicted L2 hit rates (§5.3).
  • L1.5/LRC effective capacity on H200/B200 = not tabulated; derived from B200 ~22.5 TB/s L1.5 tier in Fig. 2
    Cascade SDCM (§3.5.3) needs an intermediate capacity for the L1.5 tier; calibrated from bandwidth sweeps.
  • Per-pipeline throughput/bandwidth rates (TC, CUDA, SFU, TMEM, SMEM, L1.5, L2, DDR) = Table 3 lists measured values, e.g., B200 TC 2185/2382 TFLOPS, DDR 7.0/8.0 TB/s; per architecture
    Resource vector Eq. (1) divides tile work by these calibrated rates (§3.8). They are measured via short microbenchmarks, not taken from vendor spec.
  • Network alpha_l / beta_l per link = not tabulated in text
    Alpha-beta stage time Eq. (11) needs startup latency and inverse bandwidth per link; calibrated for NVLink, InfiniBand, PCIe (§3.6).
axioms (5)
  • standard math LRU-like cache and stochastic distance model with Gaussian approximation (Eqs. 7–10)
    Relies on SDCM and normal/Zelen-Severo approximations; standard statistical cache modeling from cited work.
  • domain assumption Prologue-steady-epilogue pipeline envelope (Eq. 2) with effective depth d (Eq. 3)
    Assumes tile execution is a regular pipeline; boundary costs are a fraction of steady state. Used recursively at every level (§3.4).
  • domain assumption Independent schedulable resources: T = max over resource sums in best topological order (Eqs. 4–5)
    Assumes resources in Eq. (1) serialize only on contention and otherwise overlap perfectly; no instruction-level issue modeling (§3.8).
  • domain assumption Uniform SM progress assumption for reuse distance
    Reuse-distance cache traffic assumes tiles advance at a uniform rate across SMs; §5.3 shows a large-K GEMM where lockstep prediction is 82% vs measured 43%; §7 acknowledges SMs desynchronize.
  • domain assumption Collective communication decomposable into staged logical exchanges with alpha-beta cost (Eq. 11)
    Standard alpha-beta model; collective algorithms (ring, tree, Rabenseifner) chosen by message size. Used §3.6.

pith-pipeline@v1.3.0-alltime-deepseek · 23757 in / 13480 out tokens · 136927 ms · 2026-08-01T04:44:24.272667+00:00 · methodology

0 comments
read the original abstract

Recent GPU programming frameworks such as Triton, TileLang, and CUDA Tile adopt tiles as first-class primitives, making tile-centric programming the prevailing approach for high-performance GPU kernels. Performance-analysis tooling has not followed: programmers still rely on coarse roofline bounds, opaque ML predictors, or post-hoc profilers to understand kernel execution. This gap is acute for modern AI workloads, where kernel fusion and distributed inference depend on tensor cores, CUDA cores, cache hierarchies, memory pipelines, and inter-GPU networks. We present TileSight, a tile-centric performance-modeling tool that elevates the tile from a programming primitive to an analysis primitive. Within a GPU core, TileSight models compute-memory pipeline overlap; across cores, it models the cache hierarchy; across GPUs, it models inter-node communication. All layers share the tile abstraction: the intra-tile layer expresses work as a resource vector spanning network, memory, and compute pipelines; the inter-tile layer schedules dependent and ordered actions to expose legal overlap and infers multi-level cache hit rates from tile reuse distance; and the cross-device layer maps remote tensor accesses to placements and routes them through an alpha-beta stage cost. On A100, H200, B200, and B6000, TileSight predicts single-GPU kernel latency with 12.35% pooled mean absolute percentage error (MAPE), outperforming state-of-the-art baselines and transferring better across architectures. Its L2 cache-hit-rate predictions are within roughly one percentage point of measurements on every GPU. At up to 32 GPUs, TileSight achieves 16.18% weighted MAPE (wMAPE) on fused distributed kernels and 13.52% wMAPE on end-to-end vLLM serving. In optimization, TileSight selects tile configurations competitive with strong vendor and expert baselines. TileSight will be open-sourced upon publication.

Figures

Figures reproduced from arXiv: 2607.22432 by Fan Yang, Guoyu Li, Hongxiang Fan, Jilong Xue, Lei Wang, Lei Xu, Lingxiao Ma, Luo Mai, Wayne Luk, Yu Cheng, Yuqi Dong, Yuqing Xia, Zhengju Tang, Zhiwen Mo, Zhi Yang.

Figure 1
Figure 1. Figure 1: FlashAttention-3 on H100: (a) the 10+ heteroge￾neous operations spanning Tensor Cores, CUDA cores, and SFUs; (b) their data-dependency DAG; (c) how scheduling order determines compute-memory pipeline overlap. 0 50 100 150 200 250 Data Size (MB) 7500 10000 12500 15000 17500 20000 22500 Effective Bandwidth (GB/s) L2 cliff: 83.12 MB L1.5 cliff B200 L1.5 = 22465 GB/s L2 = 20549 GB/s DDR = 7407 GB/s 0 50 100 15… view at source ↗
Figure 2
Figure 2. Figure 2: L2 bandwidth vs. working-set size on B200 and B6000, revealing the multi-level cache hierarchy. B200 (dual￾die) exposes a level-1.5 (L1.5)/LRC tier at ∼22.5 TB/s and a smeared L2 cliff at ∼83 MB; B6000 (single-die) shows a sharp cliff at ∼130 MB. TileSight uses these sweeps to calibrate effective cache capacity per GPU. either end-to-end runtime or analytical-model residuals from per-architecture traces, a… view at source ↗
Figure 3
Figure 3. Figure 3: TileSight design overview on all-gather–GEMM (AG-GEMM). (a) A workload is described by an operator and tensor placement only (𝑋 column-sharded across 𝑁 GPUs). (b) TileSight lifts it to a tile schedule whose DAG spans memory levels 𝐿0–𝐿4. (c) A single hardware abstraction exposes registers, SMEM, L2, HBM, and the inter-GPU fabric as a 5-level hierarchy. (d) Intra-tile resource vectors and inter-tile DAG/con… view at source ↗
Figure 4
Figure 4. Figure 4: Tile vs. cache-line reuse distance. Left: tradi￾tional cache-line reuse distance tracks tens of thousands of line entries and evaluates exact SDCM at line granularity. Right: TileSight lifts reuse distance to tile-sized blocks, applies a Gaussian SDCM approximation, and samples along reduction axes, preserving schedule sensitivity while making cache modeling lightweight. 35% to 72% in our motivating case; … view at source ↗
Figure 5
Figure 5. Figure 5: GEMM latency prediction vs. measured latency across A100, B200, B6000, H200, and MI210. Each point is one BF16/FP16 tensor-core GEMM shape; the diagonal indicates exact prediction. 0 200 400 600 800 1000 1200 Config index (shape × tiling) 0.0 0.2 0.4 0.6 0.8 1.0 L2 hit rate L2 hit-rate: TileSight vs NCU A100: MAE=1.46 pp, MAPE=2.33% H200: MAE=0.88 pp, MAPE=1.50% B200: MAE=1.05 pp, MAPE=1.52% B6000: MAE=0.7… view at source ↗
Figure 6
Figure 6. Figure 6: TileSight L2 hit-rate prediction vs. NCU ground truth across 4,680 GEMM persistent-kernel cases [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Pure-collective prediction on H200×8 and B200×8 across AllGather, AllReduce, ReduceScatter, and All-to-All [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Fused compute-communication kernel prediction on H200×8 and B200×8 (AllGather+GEMM, GEMM+ReduceScatter, Ulysses Attention). with a sigmoid, these out-of-range inputs drive it to zero, triggering division by zero and preventing robust end-to￾end prediction for these cases. This highlights a robustness limitation of ML-based predictors when extrapolating to unseen cases. TileSight achieves 7.5–18.0% per-mach… view at source ↗
Figure 9
Figure 9. Figure 9: vLLM decode throughput prediction across dense LLMs, MoE models, and multi-node configurations. Dense rows cover A100×1, B6000×2, B200×8, and H200-NVL. MoE rows cover B200×8, B200×32, and H200-NVL×8. Bars compare measured vLLM tokens per second with TileSight and PipeWeave where supported. PipeWeave does not support MoE. 10 1 10 2 10 3 10 4 Measured decode throughput (tok/s) 10 1 10 2 10 3 10 4 Predicted d… view at source ↗
Figure 10
Figure 10. Figure 10: Predicted vs. measured decode throughput across all healthy configurations. TileSight: 13.52% wMAPE over￾all. PipeWeave: 31.84% wMAPE on supported dense rows. 6 Related Work Tile-Centric Programming Frameworks. Triton [44], TileLang [10], TileLink [56], CUTLASS/CUTE [11], CuteDSL [12], ThunderKittens [38], FractalTensor [28], and NVIDIA’s CUDA Tile [33] have driven GPU programming toward tile-centric abst… view at source ↗
Figure 12
Figure 12. Figure 12: TileSight as cost model in TileLang: pruning 95% of candidate schedules and retaining the predicted top 5% reaches 99.66% of exhaustive-search best performance on average across 10 LLaMA-derived GEMM-FP16 workloads. using profiling-based kernel estimators, while DistServe [57], CrossPipe[8], Sailor [39], Metis [45], and RAPID-LLM [22] optimize parallel strategies with various communication and scheduling … view at source ↗
Figure 11
Figure 11. Figure 11: Kernel performance on H100 and MI210 when TileSight guides tile configuration selection in Triton and TileLang, replacing exhaustive autotuning. Reference lines are FlashAttention-3 for multi-head attention/grouped-query attention (MHA/GQA), FlashMLA for MLA, cuBLAS/rocBLAS for matrix multiplication, and vendor libraries for dequantized matrix multiplication. TileSight differs by being fully first-princip… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 11 linked inside Pith

  1. [1]

    1965.Handbook of mathemat- ical functions: with formulas, graphs, and mathematical tables

    Milton Abramowitz and Irene A Stegun. 1965.Handbook of mathemat- ical functions: with formulas, graphs, and mathematical tables. Vol. 55. Courier Corporation

    1965

  2. [2]

    Advanced Micro Devices, Inc. 2025. Omniperf Documenta- tion.https://rocm.docs.amd.com/projects/omniperf/en/docs-6.2.1/ what-is-omniperf.htmlAccessed: 2025-04-15

    2025

  3. [3]

    Amey Agrawal, Nitin Kedia, Jayashree Mohan, Ashish Panwar, Nipun Kwatra, Bhargav S Gulavani, Ramachandran Ramjee, and Alexey Tumanov. 2024. Vidur: A large-scale simulation framework for llm inference.Proceedings of Machine Learning and Systems6 (2024), 351– 366

    2024

  4. [4]

    Yehia Arafa, Abdel-Hameed Badawy, Gopinath Chennupati, Atanu Barai, Nandakishore Santhi, and Stephan Eidenbenz. 2020. Fast, accurate, and scalable memory modeling of GPGPUs using reuse profiles. InProceedings of the 34th ACM International Conference on supercomputing. 1–12

    2020

  5. [5]

    Yehia Arafa, Gopinath Chennupati, Atanu Barai, Abdel-Hameed A Badawy, Nandakishore Santhi, and Stephan Eidenbenz. 2019. Gpus cache performance estimation using reuse distance analysis. In2019 IEEE 38th International Performance Computing and Communications Conference (IPCCC). IEEE, 1–8

    2019

  6. [6]

    Abhimanyu Bambhaniya, Ritik Raj, Geonhwa Jeong, Souvik Kundu, Sudarshan Srinivasan, Suvinay Subramanian, Midhilesh Elavazhagan, Madhu Kumar, and Tushar Krishna. 2024. Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM models. arXiv preprint arXiv:2406.01698(2024)

    Pith/arXiv arXiv 2024

  7. [7]

    Jingwei Cai, Yuchen Wei, Zuotong Wu, Sen Peng, and Kaisheng Ma

  8. [8]

    Tiancheng Chen, Aleš Kubíček, Langwen Huang, and Torsten Hoefler

  9. [9]

    Conte, Mary Ann Hirsch, and W-MW Hwu

    Thomas M. Conte, Mary Ann Hirsch, and W-MW Hwu. 1998. Com- bining trace sampling with single pass methods for efficient cache simulation.IEEE Trans. Comput.47, 6 (1998), 714–720

    1998

  10. [10]

    Tile-AI Contributors. 2025. TileLang: A Domain-Specific Language for High-Performance GPU/CPU Kernels.https://github.com/tile- ai/tilelang

    2025

  11. [11]

    NVIDIA Corporation. 2024. CUTLASS: CUDA Templates for Linear Algebra Subroutines.https://github.com/NVIDIA/cutlass

    2024

  12. [12]

    NVIDIA CUTLASS.https://github.com/NVIDIA/ cutlass

    CUTLASS acce. NVIDIA CUTLASS.https://github.com/NVIDIA/ cutlass

  13. [13]

    Michael Davies, Neal Crago, Karthikeyan Sankaralingam, and Christos Kozyrakis. 2025. LIMINAL: Exploring The Frontiers of LLM Decode Performance.arXiv preprint arXiv:2507.14397(2025)

    arXiv 2025

  14. [14]

    Yicheng Feng, Yuetao Chen, Kaiwen Chen, Jingzong Li, Tianyuan Wu, Peng Cheng, Chuan Wu, Wei Wang, Tsung-Yi Ho, and Hong Xu. 2024. Echo: Simulating Distributed Training At Scale.ArXivabs/2412.12487 (2024).https://api.semanticscholar.org/CorpusID:274789139

    Pith/arXiv arXiv 2024

  15. [15]

    Futu News. 2026. NVIDIA launches CUDA 13.1 and CUDA Tile; Jensen Huang calls it the most significant advancement since CUDA’s introduction ∼20 years ago. Online news brief.https://news.futunn.com/en/post/65885271/futu-morning-brief- the-most-critical-week-of-the-yearAccessed: 2026-05-19

    arXiv 2026

  16. [16]

    Mingyu Gao, Xuan Yang, Jing Pu, Mark Horowitz, and Christos Kozyrakis. 2019. Tangram: Optimized coarse-grained dataflow for scalable nn accelerators. InProceedings of the Twenty-Fourth Interna- tional Conference on Architectural Support for Programming Languages and Operating Systems. 807–820

    2019

  17. [17]

    X Yu Geoffrey, Yubo Gao, Pavel Golikov, and Gennady Pekhimenko

  18. [18]

    Yue Guan, Yuanwei Fang, Keren Zhou, Corbin Robeck, Manman Ren, Zhongkai Yu, Yufei Ding, and Adnan Aziz. 2025. KPerfIR: Towards an Open and Compiler-centric Ecosystem for GPU Kernel Performance Tooling on Modern AI Workloads.arXiv preprint arXiv:2505.21661 (2025)

    Pith/arXiv arXiv 2025

  19. [19]

    Hanpeng Hu, Junwei Su, Juntao Zhao, Yanghua Peng, Yibo Zhu, Haibin Lin, and Chuan Wu. 2024. CDMPP: A device-model agnostic framework for latency prediction of tensor programs. InProceedings of the Nineteenth European Conference on Computer Systems. 1054–1074

    2024

  20. [20]

    Songlin Huang and Chenshu Wu. 2025. Neutrino: Fine-grained {GPU} Kernel Profiling via Programmable Probing. In19th USENIX Symposium on Operating Systems Design and Implementation (OSDI 25). 331–355

    2025

  21. [21]

    Rachid Karami, Sheng-Chun Kao, and Hyoukjun Kwon. 2025. Un- derstanding the Performance Horizon of the Latest ML Workloads with NonGEMM Workloads. In2025 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). IEEE, 1–14

    2025

  22. [22]

    George Karfakis, Faraz Tahmasebi, Bin Chen, Lime Yao, Saptarshi Mitra, Tian Pan, Hyoukjun Kwon, and Puneet Gupta. 2025. RAPID- LLM: Resilience-Aware Performance analysis of Infrastructure for Distributed LLM Training and Inference.ArXivabs/2512.19606 (2025). https://api.semanticscholar.org/CorpusID:284077588

    arXiv 2025

  23. [23]

    Hyoukjun Kwon, Prasanth Chatarasi, Vivek Sarkar, Tushar Krishna, Michael Pellauer, and Angshuman Parashar. 2020. Maestro: A data- centric approach to understand reuse, performance, and hardware cost of dnn mappings.IEEE micro40, 3 (2020), 20–29

    2020

  24. [24]

    Monica D Lam, Edward E Rothberg, and Michael E Wolf. 1991. The cache performance and optimizations of blocked algorithms.ACM SIGOPS Operating Systems Review25, Special Issue (1991), 63–74

    1991

  25. [25]

    Seonho Lee, Amar Phanishayee, and Divya Mahajan. 2025. Forecasting GPU Performance for Deep Learning Training and Inference. In Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1. 493–508

    2025

  26. [26]

    Ying Li, Yifan Sun, and Adwait Jog. 2023. Path Forward Beyond Simu- lators: Fast and Accurate GPU Execution Time Prediction for DNN Workloads. InProceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture. 380–394

    2023

  27. [27]

    Mingyu Liang, Hiwot Tadese Kassa, Wenyin Fu, Brian Coutinho, Louis Feng, and Christina Delimitrou. 2025. Lumos: Efficient Performance Modeling and Estimation for Large-scale LLM Training.ArXiv abs/2504.09307 (2025).https://api.semanticscholar.org/CorpusID: 277781663

    Pith/arXiv arXiv 2025

  28. [28]

    Siran Liu, Chengxiang Qi, Ying Cao, Chao Yang, Weifang Hu, Xuanhua Shi, Fan Yang, and Mao Yang. 2024. Uncovering nested data parallelism and data reuse in dnn computation with fractaltensor. InProceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles. 160–177

    2024

  29. [29]

    José Morgado, Leonel Sousa, and Aleksandar Ilic. 2024. CARM tool: cache-aware roofline model automatic benchmarking and application analysis. In2024 IEEE International Symposium on Workload Charac- terization (IISWC). IEEE, 68–81

    2024

  30. [30]

    Qingpeng Niu, James Dinan, Qingda Lu, and Ponnuswamy Sadayap- pan. 2012. PARDA: A fast parallel reuse distance analysis algorithm. In2012 IEEE 26th International Parallel and Distributed Processing Symposium. IEEE, 1284–1294

    2012

  31. [31]

    Cedric Nugteren, Gert-Jan Van den Braak, Henk Corporaal, and Henri Bal. 2014. A detailed GPU cache model based on reuse distance theory. In2014 IEEE 20th International Symposium on High Performance Computer Architecture (HPCA). IEEE, 37–48

    2014

  32. [32]

    NVIDIA Corporation. 2025. NVIDIA Nsight Compute.https: //developer.nvidia.com/nsight-computeAccessed: 2025-04-15. 15 Zhiwen Mo et al

    2025

  33. [33]

    NVIDIA Corporation. 2026. CUDA Tile | NVIDIA Developer.https: //developer.nvidia.com/cuda/tileAccessed: 2026-04-13

    2026

  34. [34]

    Santosh Pandey, Amir Yazdanbakhsh, and Hang Liu. 2024. Tao: re- thinking DL-based microarchitecture simulation.Proceedings of the ACM on Measurement and Analysis of Computing Systems8, 2 (2024), 1–25

    2024

  35. [35]

    Angshuman Parashar, Priyanka Raina, Yakun Sophia Shao, Yu-Hsin Chen, Victor A Ying, Anurag Mukkara, Rangharajan Venkatesan, Brucek Khailany, Stephen W Keckler, and Joel Emer. 2019. Timeloop: A systematic approach to dnn accelerator evaluation. In2019 IEEE international symposium on performance analysis of systems and software (ISPASS). IEEE, 304–315

    2019

  36. [36]

    Rajeev Patwari, Ashish Sirasao, and Devleena Das. 2025. Forecasting LLM inference performance via hardware-agnostic analytical model- ing.arXiv preprint arXiv:2508.00904(2025)

    Pith/arXiv arXiv 2025

  37. [37]

    Du Shen, Shuaiwen Leon Song, Ang Li, and Xu Liu. 2018. Cudaadvisor: Llvm-based runtime profiling for modern gpus. InProceedings of the 2018 International Symposium on Code Generation and Optimization. 214–227

    2018

  38. [38]

    Benjamin F Spector, Simran Arora, Aaryan Singhal, Daniel Y Fu, and Christopher Ré. 2024. ThunderKittens: Simple, Fast, and Adorable AI Kernels.arXiv preprint arXiv:2410.20399(2024)

    Pith/arXiv arXiv 2024

  39. [39]

    Foteini Strati, Zhendong Zhang, George Manos, Ixeia Sánchez Périz, Qinghao Hu, Tiancheng Chen, Berk Buzcu, Song Han, Pamela Delgado, and Ana Klimovic. 2025. Sailor: Automating distributed training over dynamic, heterogeneous, and geo-distributed clusters. InProceedings of the ACM SIGOPS 31st Symposium on Operating Systems Principles. 204–220

    2025

  40. [40]

    Stuart H Sul, Simran Arora, Benjamin F Spector, and Christopher Ré. 2025. ParallelKittens: Systematic and Practical Simplification of Multi-GPU AI Kernels.arXiv preprint arXiv:2511.13940(2025)

    arXiv 2025

  41. [41]

    Jonas Svedas, Hannah Watson, Nathan Laubeuf, Diksha Moolchandani, Abubakr Nada, Arjun Singh, Dwaipayan Biswas, James Myers, and Debjyoti Bhattacharjee. 2025. A survey of end-to-end modeling for distributed DNN training: Workloads, simulators, and TCO.arXiv preprint arXiv:2506.09275(2025)

    Pith/arXiv arXiv 2025

  42. [42]

    Ryan Swann, Muhammad Osama, Xiaohu Guo, Bryant Nelson, Lixun Zhang, Alex Brown, Yen Ong, Ali Yazdani, Sean Siddens, Ganesh Dasika, et al. 2025. tritonBLAS: Triton-based Analytical Approach for GEMM Kernel Parameter Selection.arXiv preprint arXiv:2512.04226 (2025)

    arXiv 2025

  43. [43]

    Rajeev Thakur, Rolf Rabenseifner, and William Gropp. 2005. Opti- mization of collective communication operations in MPICH.The International Journal of High Performance Computing Applications19, 1 (2005), 49–66

    2005

  44. [44]

    Philippe Tillet, Hsiang-Tsung Kung, and David Cox. 2019. Triton: an intermediate language and compiler for tiled neural network computations. InProceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages. 10–19

    2019

  45. [45]

    Taegeon Um, Byungsoo Oh, Minyoung Kang, Woo-Yeon Lee, Goeun Kim, Dongseob Kim, Youngtaek Kim, Mohd Muzzammil, and Myeong- jae Jeon. 2024. Metis: Fast automatic distributed training on heteroge- neous{GPUs}. In2024 USENIX Annual Technical Conference (USENIX ATC 24). 563–578

    2024

  46. [46]

    Xizheng Wang, Qingxu Li, Yichi Xu, Gang Lu, Dan Li, Li Chen, Heyang Zhou, Linkang Zheng, Sen Zhang, Yikai Zhu, Yang Liu, Pengcheng Zhang, Kun Qian, Kunling He, Jiaqi Gao, Ennan Zhai, Dennis Cai, and Binzhang Fu. 2025. SimAI: Unifying Architecture Design and Performance Tuning for Large-Scale Large Language Model Training with Scalability and Precision. In...

    2025

  47. [47]

    Zixian Wang, Cole Ramos, Muhammad A Awad, and Keith Lowery

  48. [48]

    Samuel Williams, Andrew Waterman, and David Patterson. 2009. Roofline: an insightful visual performance model for multicore ar- chitectures.Commun. ACM52, 4 (2009), 65–76

    2009

  49. [49]

    Feiyang Wu, Zhuohang Bian, Guoyang Duan, Tianle Xu, Junchi Wu, Teng Ma, Yongqiang Yao, Ruihao Gong, and Youwei Zhuo. 2025. TokenSim: Enabling Hardware and Software Exploration for Large Language Model Inference Systems. InAdvanced Parallel Programming Technologies.https://api.semanticscholar.org/CorpusID:276928157

    2025

  50. [50]

    Yannan Nellie Wu, Po-An Tsai, Angshuman Parashar, Vivienne Sze, and Joel S Emer. 2022. Sparseloop: An analytical approach to sparse tensor accelerator modeling. In2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 1377–1395

    2022

  51. [51]

    arXiv preprint arXiv:2506.20886(2025)

    Omniwise: Predicting GPU Kernels Performance with LLMs. arXiv preprint arXiv:2506.20886(2025)

    Pith/arXiv arXiv 2025

  52. [52]

    Zhihang Yuan, Yuzhang Shang, Yang Zhou, Zhen Dong, Zhe Zhou, Chenhao Xue, Bingzhe Wu, Zhikai Li, Qingyi Gu, Yong Jae Lee, et al

  53. [53]

    Alberto Zeni, Emanuele Del Sozzo, Eleonora D’Arnese, Davide Confic- coni, and Marco D Santambrogio. 2024. Starlight: A kernel optimizer for GPU processing.J. Parallel and Distrib. Comput.187 (2024), 104832

    2024

  54. [54]

    Kaixuan Zhang, Yunfan Cui, Shuhao Zhang, Chutong Ding, Shiyou Qian, Luping Wang, Jian Cao, Guangtao Xue, Cheng Huang, Guodong Yang, and Liping Zhang. 2026. PipeWeave: Synergizing Analytical and Learning Models for Unified GPU Performance Prediction.arXiv preprint(2026).https://arxiv.org/abs/2601.14910

    Pith/arXiv arXiv 2026

  55. [55]

    Srihas Yarlagadda, Amey Agrawal, Elton Pinto, Hakesh Darapaneni, Mitali Meratwal, Shivam Mittal, Pranavi Bajjuri, Srinivas Sridharan, and Alexey Tumanov. 2025. Maya: Optimizing Deep Learning Training Workloads using GPU Runtime Emulation.arXiv preprint arXiv:2503.20191(2025)

    arXiv 2025

  56. [56]

    Size Zheng, Jin Fang, Xuegui Zheng, Qi Hou, Wenlei Bao, Ningxin Zheng, Ziheng Jiang, Dongyang Wang, Jianxi Ye, Haibin Lin, et al. 2025. Tilelink: Generating efficient compute-communication overlapping kernels using tile-centric primitives.arXiv preprint arXiv:2503.20313 (2025)

    Pith/arXiv arXiv 2025

  57. [57]

    2024.{DistServe}: Disaggregating prefill and decoding for goodput-optimized large language model serving

    Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xu- anzhe Liu, Xin Jin, and Hao Zhang. 2024.{DistServe}: Disaggregating prefill and decoding for goodput-optimized large language model serving. In18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). 193–210

    2024

  58. [58]

    Keren Zhou, Xiaozhu Meng, Ryuichi Sai, Dejan Grubisic, and John Mellor-Crummey. 2021. An automated tool for analysis and tuning of gpu-accelerated code in hpc applications.IEEE Transactions on Parallel and Distributed Systems33, 4 (2021), 854–865

    2021

  59. [59]

    Keren Zhou, Xiaozhu Meng, Ryuichi Sai, and John Mellor-Crummey

  60. [60]

    Size Zheng, Siyuan Chen, Siyuan Gao, Liancheng Jia, Guangyu Sun, Runsheng Wang, and Yun Liang. 2023. TileFlow: A Framework for Modeling Fusion Dataflow via Tree-based Analysis. InProceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitec- ture. 1271–1288

    2023

  61. [65]

    In2021 IEEE/ACM International Symposium on Code Generation and Optimization (CGO)

    GPA: A GPU Performance Advisor Based on Instruction Sam- pling. In2021 IEEE/ACM International Symposium on Code Generation and Optimization (CGO). 115–125. 16

  62. [2021]

    In2021 USENIX Annual Technical Conference (USENIX ATC 21)

    Habitat: A {Runtime-Based} computational performance predictor for deep neural network training. In2021 USENIX Annual Technical Conference (USENIX ATC 21). 503–521

  63. [2023]

    InProceedings of the 50th Annual International Symposium on Computer Architecture

    Inter-layer scheduling space definition and exploration for tiled accelerators. InProceedings of the 50th Annual International Symposium on Computer Architecture. 1–17

  64. [2024]

    arXiv preprint arXiv:2402.16363(2024)

    Llm inference unveiled: Survey and roofline model insights. arXiv preprint arXiv:2402.16363(2024)

    Pith/arXiv arXiv 2024

  65. [2025]

    InUSENIX Annual Technical Conference.https: //api.semanticscholar.org/CorpusID:280049543

    CrossPipe: Towards Optimal Pipeline Schedules for Cross- Datacenter Training. InUSENIX Annual Technical Conference.https: //api.semanticscholar.org/CorpusID:280049543