Pith. sign in

REVIEW 4 major objections 4 minor 58 references

MCFuser: High-Performance and Rapid Fusion of Memory-Bound Compute-Intensive Operators

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MCFuser shows that memory-bound compute-intensive operator chains can be fused into kernels that beat Ansor by up to 5.9x and cut tuning time by over 70x.

desk verdict MCFuser is a credible systems paper with real engineering value, but the paper's own 'comprehensive/exhaustive' search-space claim is overbroad, and the mutation step in Algorithm 1 cannot even explore tiling expressions not present in the initial random population. read the letter →

arxiv 2506.22169 v1 pith:LIF6HK3O submitted 2025-06-27 cs.DC cs.PL

Zheng Zhang , Donglin Yang , Xiaobo Zhou , Dazhao Cheng This is my paper · ORCID
classification cs.DCcs.PL
keywords operatorfusionmemory-boundcompute-intensiveoperatorsGPUkernelgenerationtilingexpressionsDAG-basedmemoryoptimizationanalyticalperformancemodeltensorprogramauto-tuningself-attention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Operator fusion usually stops at compute-intensive operators because their compute throughput is saturated, but tensor dimensions can change that: when reduction dimensions shrink, a GEMM-like operator becomes memory-bandwidth-bound. MCFuser targets these memory-bound compute-intensive (MBCI) chains, with self-attention as the recurring example, and claims that a complete search space of loop nestings plus removal of redundant memory traffic yields fused kernels that beat general-purpose compilers. The paper reports up to 5.9x speedup over Ansor on A100 and RTX3080 GPUs and over 70x shorter tuning time, because an analytical performance model replaces ML-based cost-model training. If right, MCFuser makes fusion of such operator chains practical in deployment settings where tuning time matters.

What carries the argument

The load-bearing object is the high-level tiling expression: a string over tile loops where adjacency means nesting and parentheses mean sequential execution, such as $mhn(k(LA, LB, CC), LD, CE, SE)$ for a fused GEMM chain. Every candidate schedule is a permutation of the $m,n,k,h$ tile loops (24 deep forms) or one of two flat forms, with tile sizes restricted to multiples of 16. A DAG over loop nodes, load/compute/store nodes, and two dependency types (scope-dependent and order-dependent) is used to relocate memory statements to the outermost loop whose variables actually index the tensor, and to delete dead loop nodes when an extent is one. The analytical performance model $t_{est} = (t_{mem}+t_{comp})\times\alpha$ estimates execution time from tile sizes, trip counts, memory bandwidth, compute throughput, and the number of thread blocks relative to SMs, which lets the heuristic search rank candidates without training a cost model.

What would settle it

Take a GEMM chain covered by the paper (for example G12: batch 8, M=1024, K=N=H=128), hand-write or generate a kernel using split-K or warp specialization, schedules outside the tiling-expression space, and measure both on the same A100. If that kernel runs faster than MCFuser's best candidate, the claim that no configuration is overlooked is false; if it never wins across the G1-G12 set, the comprehensiveness claim is supported.

Watch

Extended reading notes

Core claim

The central claim is that memory-bound compute-intensive (MBCI) operator chains can be fused into one high-performance kernel by enumerating all loop-level tiling expressions, including all deep permutations and the two flat forms for a two-GEMM chain, and by using a DAG over loop, load, compute, and store nodes to move memory statements outward when a loop extent collapses to one. MCFuser then prunes equivalent or infeasible candidates and ranks survivors with an analytical performance model combining memory traffic, FLOPs, and a thread-block slowdown factor, so the best candidate is found by a short heuristic search rather than hours of measurement. The paper claims this makes the search space comprehensive in the sense that no tiling configuration recognized by the model is overlooked, and that the resulting kernels outperform Ansor, BOLT, FlashAttention, and PyTorch on GEMM chains and self-attention modules.

Load-bearing premise

The paper assumes that every high-performance fused kernel for these operator chains can be written as one of the tiling expressions it enumerates, namely all deep loop permutations plus exactly two flat forms, so that no promising schedule such as split-K or warp-specialized pipelines is ever considered.

Editorial extensions

If this is right

  • Self-attention modules can be compiled into a single fused kernel rather than the usual separate GEMM and softmax kernels, which the paper evaluates as 8.1x and 5.8x average speedups over PyTorch on A100 and RTX3080.
  • Auto-tuning a fused MBCI kernel no longer requires training an ML cost model: the analytical model's estimates correlate at 0.8 to 0.92 with measured time, and the search stops automatically on convergence.
  • Fusion benefits extend to end-to-end transformer models: MCFuser+Relay averages 1.45x over Relay, and MCFuser+Ansor averages 3.66x over BOLT on Bert models.
  • Flat tiling forms matter: including them gives up to 1.17x and 1.13x over a Chimera-style search restricted to deep tilings on A100 and RTX3080.
  • Tuning time drops sharply, by 74x to 139x on subgraphs and 1.36x to 1.57x end-to-end, because only a small top-k of candidates is measured.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the MBCI concept generalizes, the same machinery should apply to operator chains beyond GEMM pairs, such as GEMV, convolution plus pooling, or grouped GEMMs with shrinking reduction dimensions; the paper suggests this but evaluates only batched GEMM chains and self-attention.
  • The reported speedups over Ansor may partly measure Ansor's slow ML-guided search rather than the exhaustiveness of MCFuser's space; a fair test would compare MCFuser's best kernel against a split-K or warp-specialized implementation of the same chain, which MCFuser's representation cannot express.
  • Rule 3 discards any tile that needs more than 5% padding unless the dimension is a power of two, so on non-power-of-two shapes this could exclude the true optimum; relaxing the rule and remeasuring would isolate how much the pruning heuristic costs.
  • The analytical model's slowdown factor $\alpha = (N_{block}+N_{SM})/N_{block}$ is a rough occupancy proxy, so on future GPUs where block count and latency hiding decouple, the model may need recalibration.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper introduces MCFuser, a compiler framework for fusing chains of "memory-bound compute-intensive" (MBCI) operators on GPUs. The core mechanism is a tiling-expression search space (deep and flat loop nestings), a DAG-based transformation that moves load/store statements out of redundant loops, four pruning rules, and an analytical performance model that ranks candidates without an ML cost model. Search uses an evolutionary loop that measures only a small top-k subset each iteration. The paper reports average speedups of 6.6x/3.7x over PyTorch and 2.7x/1.6x over Ansor on GEMM chains, 8.1x/5.8x over PyTorch on self-attention, up to 5.9x over Ansor, and more than 70x tuning-time reductions on A100 and RTX 3080, together with end-to-end BERT results.

Significance. The empirical contribution is potentially useful: if reproduced, MCFuser would be a practical fusion tool for GEMM chains and self-attention, and the analytical performance model is an attractive alternative to ML cost models. The paper deserves credit for evaluating on two GPUs, multiple workloads, and an end-to-end setting, and for including an ablation (MCFuser-Chimera) that isolates the search-space contribution. The shared-memory estimator is also validated against measured usage. However, the paper's central universality claims - "comprehensive" search space and automatic identification of the optimal configuration - are not supported by the evidence, and the lack of numerical correctness validation is a serious gap for a compiler paper. The performance results and the underlying ideas may still be publishable after substantial revision.

major comments (4)
  1. [Section III-A] The assertion that the tiling-expression space is "comprehensive" and that "no potential configuration is overlooked" is load-bearing but not established. For a GEMM chain the space consists only of the 4! deep permutations and two flat forms; it cannot represent standard high-performance schedules such as split-K or stream-K, warp-specialized producer/consumer pipelines, or cross-block reduction partitioning. The experiments therefore compare the coverage of this hand-defined space, not a general fusion advantage. Please either prove a dominance or completeness property with respect to a well-defined schedule class, or replace "comprehensive/exhaustive" with a precise description of the represented schedule family and adjust the claims in Table I and Section VIII accordingly.
  2. [Section IV-B, Algorithm 1] The mutation step at line 17 changes only the tile size of one loop and never changes the tiling expression. Consequently, any tiling expression absent from the initial random population can never appear in a later generation, so the statement that the search "automatically identifies the optimal configuration in the search space" is unsupported. Either add a mutation or crossover operator over tiling expressions, or state explicitly that the evolutionary search explores tile sizes within the tiling expressions present in the initial sample and weaken the optimality claim accordingly.
  3. [Section IV-B, Algorithm 1] Lines 6-8 are internally inconsistent: after sorting the population by estimated time in ascending order, the pseudocode calls topk(..., reverse=True) and treats the result as the "top n" candidates. As written, this selects the candidates with the largest estimated times, contradicting the surrounding text and the notion of the "top1" best candidate. Please correct or clarify the pseudocode; this is central to the claimed search efficiency.
  4. [Section VI] No numerical correctness validation of any generated kernel is reported. The paper compares only normalized runtimes; it never states that outputs match a reference implementation or quantifies numerical error. This matters because Section III-B deliberately moves load/store statements across loops and removes loop nodes when k=1, and such transformations can change semantics if applied incorrectly. Please add correctness checks, e.g., against cuBLAS or PyTorch references with a stated tolerance, for every workload and configuration reported.
minor comments (4)
  1. [Section III] The first paragraph of Section III refers to "MCFusor," but the paper title and the rest of the text use "MCFuser." Please make the spelling consistent.
  2. [Section VI-B] The sentence "we achieve up to 5.9 × and speedup compared to Ansor" contains a typo; it should be "up to a 5.9× speedup."
  3. [Section VIII] The conclusion contains the duplicated phrase "machine learning machine learning compilation"; please remove the repetition.
  4. [Sections III-C and IV-B] The hyperparameters N, the convergence threshold epsilon, and the empirically set thresholds 1.2, 0.05, and n=8 are not given concrete values or a sensitivity analysis. At minimum, please state the values used and discuss how sensitive the reported tuning-time reductions are to them.

Circularity Check

2 steps flagged · score 2.0 of 10

Headline speedups are measured against external baselines and the analytical model is parameter-free, so the central claims are not circular; two mild issues (search-space 'comprehensiveness' asserted by construction; pruning threshold 1.2 validated on the same data) keep the score at 2.

  1. self definitional [Section III-A, Search Space Generation (candidate enumeration and comprehensiveness claim)]
    "Thus, the total number of candidates in our search space is determined by multiplying the number of possible tiling expressions with the viable tile size options. ... Informed by our comprehensive analysis of tiling expressions and tile sizes, we are able to fully enumerate the candidates within the search space for fused kernels. Unlike Chimera, which focuses on deep tiling expressions to the exclusion of flat tiling, our approach ensures no potential configuration is overlooked."

    The search space is literally defined as the enumeration: the candidate count is 'the number of possible tiling expressions' multiplied by 'the viable tile size options' (24 deep permutations plus 2 flat forms, times tile sizes). 'Fully enumerate the candidates within the search space' is therefore true by construction, because the space is the enumeration. The next sentence equates that definitional enumeration with 'no potential configuration is overlooked,' converting a definition of one representation into a universal coverage claim over all fused-kernel schedules, including forms the nested/sequential loop grammar cannot express (split-K, warp specialization, cross-block reductions).

  2. fitted input called prediction [Section III-C, Rule 4 and Section VI-E1, Shared Memory Estimation]
    "We prune candidates that satisfy Shmestm > 1.2Shmmax, where 1.2 accommodates estimation errors. ... The x-axis bifurcation at 1.2 ×Shmmax, an empirically determined threshold, aims to adequately account for estimation inaccuracies. Notably, over 90% of data points reside within quadrants I and III, affirming the precision of our estimation methodology in aptly identifying and pruning unviable schedules, thus curtailing the search space by 40%."

    The factor 1.2 is 'an empirically determined threshold' (Section VI-E1), i.e., chosen on the same candidate population (scheduled candidates from the Section VI-B experiments) that is then used to report 'over 90% of data points reside within quadrants I and III, affirming the precision of our estimation methodology' and the 40% reduction in candidates. The threshold is selected to make the quadrant split come out right and is then re-reported as validated accuracy on that same data: in-sample self-confirmation rather than independent support. The same applies to the top-k parameter 'n is empirically set to 8' in Algorithm 1. This does not propagate to the headline results: the rule only filters infeasible candidates, the best kernels are actually measured, and the analytical model (Eqs.

full rationale

The derivation chain is largely self-contained: the tiling-expression grammar (Section III-A) is built from explicit nested/sequential loop relations with its own enumeration formula; the DAG-based memory-access optimization (Section III-B) is a described transformation with a worked example; the analytical performance model (Eqs. 2-5) uses only hardware constants P, W, and NSM, is validated by correlation with measured times (Fig. 11, r = 0.80-0.92) rather than fitted to them, and Algorithm 1 measures the top-n candidates directly, so reported kernel performance is measured, not predicted. The headline results (up to 5.9x over Ansor, 70x tuning reduction, Table IV, speedups over BOLT/PyTorch/FlashAttention, and the MCFuser-vs-MCFuser-Chimera ablation) are anchored to external measurements and a controlled comparison. The authors' self-citations ([3], [4], [12], [28]) are background references on MoE, pipeline parallelism, and sparse transformers; none is load-bearing, and no uniqueness theorem or ansatz is imported from the authors' prior work. Two mild issues keep the score above zero. First, Section III-A equates the definitional enumeration of its own space with 'no potential configuration is overlooked,' a universality claim not established for schedules outside the nested/sequential grammar (split-K, warp specialization, cross-block reductions), and Algorithm 1's mutation changes only tile sizes, never the tiling expression, so the 'optimal candidate' claim overstates what the search can reach; these are coverage and correctness risks, not circular reductions of the measured results. Second, the pruning threshold 1.2 and the top-k size n=8 are empirically determined and then validated on the same candidate population (Section VI-E1), an in-sample self-confirmation limited to a secondary hyperparameter. No equation reduces a prediction to its inputs by construction; the central claims have independent, measured content.

Assumptions & free parameters 5 free parameters · 4 assumptions · 1 invented entities

The central claim rests on four unproved premises: the tiling-expression space is exhaustive, the analytical model ranks candidates well, the pruning rules preserve optimality, and the generated kernels are correct. The paper provides empirical validation for the shared-memory estimate (over 90% of points in quadrants I and III) and for the performance model (correlations 0.80-0.92 on four GEMM chains), but no proof or artifact for the other premises. Hand-set thresholds such as 1.2, 0.05, and n=8 are calibrated to the reported workloads, and Algorithm 1's N and epsilon are not disclosed.

free parameters (5)
  • Shared-memory pruning threshold = 1.2
    Rule 4 prunes candidates when estimated shared memory exceeds 1.2 times Shmmax; Section VI-E1 describes the value as empirically determined.
  • Padding ratio limit = 0.05
    Rule 3 discards candidates whose padding ratio is above 0.05; no derivation is given, and the limit is calibrated to the evaluated workloads.
  • Top-k measured candidates n = 8
    Algorithm 1 measures only the top n candidates per iteration; Section IV-B states n is empirically set to 8.
  • Convergence threshold epsilon = not specified
    Algorithm 1 terminates when the difference between successive top-1 measured times is below epsilon, but the paper never gives the value, making the stopping rule under-specified.
  • Initial population size N = not reported
    Algorithm 1 depends on N random initial candidates; the value is not stated, affecting both search coverage and the reported tuning time.
assumptions (4)
  • ad hoc to paper The tiling-expression space is exhaustive over high-performance fusion schedules.
    Section III-A calls the space comprehensive and says 'no potential configuration is overlooked'; no proof is provided and schedules such as split-K or warp specialization are not representable.
  • domain assumption The analytical performance model ranks candidates correctly.
    Equations (2)-(5) model time as serialized memory plus compute scaled by alpha; they ignore latency, overlap, and occupancy. Validation is limited to correlations 0.80-0.92 on four GEMM chains (Fig. 11).
  • ad hoc to paper The pruning rules preserve at least one optimal candidate.
    Rules 1-4 remove duplicates, padding-heavy tiles, and shared-memory-overflowing candidates; Rule 3 alone discards 99% of tile sizes. The paper gives no proof that pruned candidates are never the optimum.
  • domain assumption Generated fused kernels are numerically correct.
    The evaluation measures only speed; no output-equality check against PyTorch or CuBLAS is reported, so correctness is assumed.
invented entities (1)
  • MBCI (memory-bound compute-intensive) operator class
    purpose: A classification of compute-intensive operators whose effective compute/memory ratio falls below P/W for a given tensor shape, used to justify fusing them (Section II-A).
    This is a labeling device introduced by the paper, not a new physical object; it has no falsifiable handle outside the paper's own performance measurements, and its boundary depends on the assumed constants P and W.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MCFuser: High-Performance and Rapid Fusion of Memory-Bound Compute-Intensive Operators." pith.science (2026). https://pith.science/paper/LIF6HK3O

@misc{pith2026250622169,
  author       = {Pith},
  title        = {Pith review of: MCFuser: High-Performance and Rapid Fusion of Memory-Bound Compute-Intensive Operators},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LIF6HK3O}},
  note         = {Machine review of arXiv:2506.22169}
}
read the original abstract

Operator fusion, a key technique to improve data locality and alleviate GPU memory bandwidth pressure, often fails to extend to the fusion of multiple compute-intensive operators due to saturated computation throughput. However, the dynamicity of tensor dimension sizes could potentially lead to these operators becoming memory-bound, necessitating the generation of fused kernels, a task hindered by limited search spaces for fusion strategies, redundant memory access, and prolonged tuning time, leading to sub-optimal performance and inefficient deployment. We introduce MCFuser, a pioneering framework designed to overcome these obstacles by generating high-performance fused kernels for what we define as memory-bound compute-intensive (MBCI) operator chains. Leveraging high-level tiling expressions to delineate a comprehensive search space, coupled with Directed Acyclic Graph (DAG) analysis to eliminate redundant memory accesses, MCFuser streamlines kernel optimization. By implementing guidelines to prune the search space and incorporating an analytical performance model with a heuristic search, MCFuser not only significantly accelerates the tuning process but also demonstrates superior performance. Benchmarked against leading compilers like Ansor on NVIDIA A100 and RTX3080 GPUs, MCFuser achieves up to a 5.9x speedup in kernel performance and outpaces other baselines while reducing tuning time by over 70-fold, showcasing its agility.

Figures

Figures reproduced from arXiv: 2506.22169 by the authors.

Figure 1
Figure 1. Illustration of the Matmul operator implemented on the CUDA [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The performance of a MatMul operator across different K/M ratios [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The GEMM chain consists of two GEMM operators, with [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 6
Figure 6. Figure 6: Illustration of shared memory usage differences between tiling [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: The results of pruning search space candidates on an example of [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: The relative performance of fusing batch GEMM chains and self-attention modules on GPUs. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: The end-to-end model evaluation on A100 GPU. [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Model-predicted vs. actual shared memory usage on the NVIDIA [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 49 canonical work pages

  1. [1]

    Antman: Dynamic scaling on gpu clusters for deep learning,

    W. Xiao, S. Ren, Y . Li, Y . Zhang, P. Hou, Z. Li, Y . Feng, W. Lin, and Y . Jia, “Antman: Dynamic scaling on gpu clusters for deep learning,” in Proc. of USENIX OSDI , 2020, pp. 533–548

    2020

  2. [2]

    Whale: Efficient giant model training over heterogeneous gpus,

    X. Jia, L. Jiang, A. Wang, W. Xiao, Z. Shi, J. Zhang, X. Li, L. Chen, Y . Li, Z. Zheng et al. , “Whale: Efficient giant model training over heterogeneous gpus,” in USENIX Annual Technical Conference , 2022, pp. 673–688

    2022

  3. [3]

    Mpmoe: Memory efficient moe for pre-trained models with adaptive pipeline parallelism,

    Z. Zhang, Y . Xia, H. Wang, D. Yang, C. Hu, X. Zhou, and D. Cheng, “Mpmoe: Memory efficient moe for pre-trained models with adaptive pipeline parallelism,” IEEE Transactions on Parallel and Distributed Systems, vol. 35, no. 6, pp. 843–856, 2024

    work page 2024

  4. [4]

    Redundancy-free high-performance dynamic GNN training with hier- archical pipeline parallelism,

    Y . Xia, Z. Zhang, H. Wang, D. Yang, X. Zhou, and D. Cheng, “Redundancy-free high-performance dynamic GNN training with hier- archical pipeline parallelism,” in Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2023, Orlando, FL, USA, June 16-23, 2023 , A. R. Butt, N. Mi, and K. Chard, Eds. ACM, 2...

    work page 2023

  5. [5]

    Chimera: An analytical optimizing framework for effective compute-intensive operators fusion,

    S. Zheng, S. Chen, P. Song, R. Chen, X. Li, S. Yan, D. Lin, J. Leng, and Y . Liang, “Chimera: An analytical optimizing framework for effective compute-intensive operators fusion,” in IEEE International Symposium on High-Performance Computer Architecture, HPCA , 2023, pp. 1113– 1126

    work page 2023

  6. [6]

    Dnnfusion: accelerating deep neural networks execution with advanced operator fusion,

    W. Niu, J. Guan, Y . Wang, G. Agrawal, and B. Ren, “Dnnfusion: accelerating deep neural networks execution with advanced operator fusion,” in PLDI ’21: 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation , 2021, pp. 883– 898

    work page 2021

  7. [7]

    Astitch: enabling a new multi- dimensional optimization space for memory-intensive ML training and inference on modern SIMT architectures,

    Z. Zheng, X. Yang, P. Zhao, G. Long, K. Zhu, F. Zhu, W. Zhao, X. Liu, J. Yang, J. Zhai, S. L. Song, and W. Lin, “Astitch: enabling a new multi- dimensional optimization space for memory-intensive ML training and inference on modern SIMT architectures,” in ASPLOS ’22 , 2022, pp. 359–373

    work page 2022

  8. [8]

    Nvidia cublas

    “Nvidia cublas.” [Online]. Available: https://developer.nvidia.com/cublas

    work page

Show all 58 references
  1. [9]

    Nvidia cudnn

    “Nvidia cudnn.” [Online]. Available: https://developer.nvidia.com/cudnn

  2. [10]

    Nvidia cutlass

    “Nvidia cutlass.” [Online]. Available: https://developer.nvidia.com/cutlass

  3. [11]

    Nvidia tensorrt

    “Nvidia tensorrt.” [Online]. Available: https://docs.nvidia.com/deeplearning/tensorrt/ developer- guide/index.html

  4. [12]

    Mpipemoe: Memory efficient moe for pre-trained models with adaptive pipeline parallelism,

    Z. Zhang, D. Yang, Y . Xia, L. Ding, D. Tao, X. Zhou, and D. Cheng, “Mpipemoe: Memory efficient moe for pre-trained models with adaptive pipeline parallelism,” in 2023 IEEE International Parallel and Dis- tributed Processing Symposium (IPDPS) . IEEE, 2023, pp. 167–177

    2023

  5. [13]

    Ansor: Generating high-performance tensor programs for deep learning,

    L. Zheng, C. Jia, M. Sun, Z. Wu, C. H. Yu, A. Haj-Ali, Y . Wang, J. Yang, D. Zhuo, K. Sen, J. E. Gonzalez, and I. Stoica, “Ansor: Generating high-performance tensor programs for deep learning,” in 14th USENIX Symposium on Operating Systems Design and Implementation, OSDI , 202...

    2020

  6. [14]

    Learning to optimize tensor programs,

    T. Chen, L. Zheng, E. Yan, Z. Jiang, T. Moreau, L. Ceze, C. Guestrin, and A. Krishnamurthy, “Learning to optimize tensor programs,” Ad- vances in Neural Information Processing Systems , vol. 31, 2018

    2018

  7. [15]

    Tiramisu: A polyhedral compiler for expressing fast and portable code,

    R. Baghdadi, J. Ray, M. B. Romdhane, E. D. Sozzo, A. Akkas, Y . Zhang, P. Suriana, S. Kamil, and S. P. Amarasinghe, “Tiramisu: A polyhedral compiler for expressing fast and portable code,” in IEEE/ACM Interna- tional Symposium on Code Generation and Optimization, CGO , 2019

    2019

  8. [16]

    Astra: Exploiting predictability to optimize deep learning,

    M. Sivathanu, T. Chugh, S. S. Singapuram, and L. Zhou, “Astra: Exploiting predictability to optimize deep learning,” in ASPLOS, 2019, pp. 909–923

    2019

  9. [17]

    TASO: optimizing deep learning computation with automatic genera- tion of graph substitutions,

    Z. Jia, O. Padon, J. Thomas, T. Warszawski, M. Zaharia, and A. Aiken, “TASO: optimizing deep learning computation with automatic genera- tion of graph substitutions,” in Proceedings of the 27th ACM Symposium on Operating Systems Principles, SOSP , 2019

    2019

  10. [18]

    Scalable kernel fusion for memory-bound GPU applications,

    M. Wahib and N. Maruyama, “Scalable kernel fusion for memory-bound GPU applications,” in International Conference for High Performance Computing, Networking, Storage and Analysis, SC . IEEE Computer Society, 2014, pp. 191–202

    2014

  11. [19]

    AKG: automatic kernel generation for neural processing units using polyhedral transformations,

    J. Zhao, B. Li, W. Nie, Z. Geng, R. Zhang, X. Gao, B. Cheng, C. Wu, Y . Cheng, Z. Li, P. Di, K. Zhang, and X. Jin, “AKG: automatic kernel generation for neural processing units using polyhedral transformations,” in PLDI ’21: 42nd ACM SIGPLAN International Conference on Pro- gr...

    2021

  12. [20]

    Tensor comprehen- sions: Framework-agnostic high-performance machine learning abstrac- tions,

    N. Vasilache, O. Zinenko, T. Theodoridis, P. Goyal, Z. DeVito, W. S. Moses, S. Verdoolaege, A. Adams, and A. Cohen, “Tensor comprehen- sions: Framework-agnostic high-performance machine learning abstrac- tions,” CoRR, vol. abs/1802.04730, 2018

    2018 arXiv

  13. [21]

    Tensorflow xla

    “Tensorflow xla.” [Online]. Available: https://www.tensorflow.org/xla

  14. [22]

    Tvm: An automated end-to-end optimizing compiler for deep learning,

    T. Chen, T. Moreau, Z. Jiang, L. Zheng, E. Yan, H. Shen, M. Cowan, L. Wang, Y . Hu, L. Ceze et al. , “Tvm: An automated end-to-end optimizing compiler for deep learning,” in OSDI, 2018, pp. 578–594

    2018

  15. [23]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems , 2017, pp. 5998–6008

    2017

  16. [24]

    Xgboost: A scalable tree boosting system,

    T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 2016, pp. 785–794

    2016

  17. [25]

    Bolt: Bridg- ing the gap between auto-tuners and hardware-native performance,

    J. Xing, L. Wang, S. Zhang, J. Chen, A. Chen, and Y . Zhu, “Bolt: Bridg- ing the gap between auto-tuners and hardware-native performance,” in Proceedings of Machine Learning and Systems , 2022

    2022

  18. [26]

    Pytorch: An imperative style, high- performance deep learning library,

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. K ¨opf, E. Z. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, hi...

    2019

  19. [27]

    Bytetransformer: A high-performance transformer boosted for variable-length inputs,

    Y . Zhai, C. Jiang, L. Wang, X. Jia, S. Zhang, Z. Chen, X. Liu, and Y . Zhu, “Bytetransformer: A high-performance transformer boosted for variable-length inputs,” in IEEE International Parallel and Distributed Processing Symposium, IPDPS , 2023, pp. 344–355

    2023

  20. [28]

    Raptor-t: A fused and memory-efficient sparse transformer for long and variable-length sequences

    W. Hulin, D. Yang, Y . Xia, Z. Zhang, Q. Wang, J. Fan, X. Zhou, and D. Cheng, “Raptor-t: A fused and memory-efficient sparse transformer for long and variable-length sequences.” IEEE Transactions on Comput- ers, vol. 73, pp. 1852–1865, 2024

    2024

  21. [29]

    Fusionstitching: Boosting memory intensive computations for deep learning workloads,

    Z. Zheng, P. Zhao, G. Long, F. Zhu, K. Zhu, W. Zhao, L. Diao, J. Yang, and W. Lin, “Fusionstitching: Boosting memory intensive computations for deep learning workloads,” CoRR, vol. abs/2009.10924, 2020

    2009 arXiv

  22. [30]

    Flashattention: Fast and memory-efficient exact attention with io-awareness,

    T. Dao, D. Y . Fu, S. Ermon, A. Rudra, and C. R ´e, “Flashattention: Fast and memory-efficient exact attention with io-awareness,” in Advances in Neural Information Processing Systems , 2022

    2022

  23. [31]

    Huggingface’s trans- formers: State-of-the-art natural language processing,

    T. Wolf, L. Debut, V . Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz et al., “Huggingface’s trans- formers: State-of-the-art natural language processing,” arXiv preprint arXiv:1910.03771, 2019

    1910 arXiv

  24. [32]

    Flashattention-2: Faster attention with better parallelism and work partitioning,

    T. Dao, “Flashattention-2: Faster attention with better parallelism and work partitioning,” CoRR, vol. abs/2307.08691, 2023

    2023 arXiv

  25. [33]

    Hidet: Task-mapping programming paradigm for deep learning tensor programs,

    Y . Ding, C. H. Yu, B. Zheng, Y . Liu, Y . Wang, and G. Pekhimenko, “Hidet: Task-mapping programming paradigm for deep learning tensor programs,” in Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, V...

    2023

  26. [34]

    Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines,

    J. Ragan-Kelley, C. Barnes, A. Adams, S. Paris, F. Durand, and S. P. Amarasinghe, “Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines,” in SIGPLAN Conference on Programming Language Design and Imple- mentation,...

    2013

  27. [35]

    Flextensor: An automatic schedule exploration and optimization framework for tensor computation on heterogeneous system,

    S. Zheng, Y . Liang, S. Wang, R. Chen, and K. Sheng, “Flextensor: An automatic schedule exploration and optimization framework for tensor computation on heterogeneous system,” in Architectural Support for Programming Languages and Operating Systems, ASPLOS . ACM, 2020, pp. 859–873

    2020

  28. [36]

    AMOS: enabling automatic mapping for tensor computations on spatial accelerators with hardware abstraction,

    S. Zheng, R. Chen, A. Wei, Y . Jin, Q. Han, L. Lu, B. Wu, X. Li, S. Yan, and Y . Liang, “AMOS: enabling automatic mapping for tensor computations on spatial accelerators with hardware abstraction,” in ISCA ’22: The 49th Annual International Symposium on Computer Architecture. ...

    2022

  29. [37]

    Atomic dataflow based graph-level workload orchestration for scalable DNN accelerators,

    S. Zheng, X. Zhang, L. Liu, S. Wei, and S. Yin, “Atomic dataflow based graph-level workload orchestration for scalable DNN accelerators,” in International Symposium on High-Performance Computer Architecture, HPCA. IEEE, 2022, pp. 475–489

    2022

  30. [38]

    ROLLER: fast and efficient tensor compilation for deep learning,

    H. Zhu, R. Wu, Y . Diao, S. Ke, H. Li, C. Zhang, J. Xue, L. Ma, Y . Xia, W. Cui, F. Yang, M. Yang, L. Zhou, A. Cidon, and G. Pekhimenko, “ROLLER: fast and efficient tensor compilation for deep learning,” in 16th USENIX Symposium on Operating Systems Design and Implemen- tation...

    2022

  31. [39]

    Triton: an intermediate language and compiler for tiled neural network computations,

    P. Tillet, H. Kung, and D. D. Cox, “Triton: an intermediate language and compiler for tiled neural network computations,” in Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages , 2019, pp. 10–19

    2019

  32. [40]

    Nvidia, parallel thread execution isa

    “Nvidia, parallel thread execution isa.” [Online]. Available: http://docs.nvidia.com/cuda/parallel-thread-execution/index.html

  33. [41]

    Tensorflow: A system for large-scale machine learning,

    M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. A. Tucker, V . Vasudevan, P. Warden, M. Wicke, Y . Yu, and X. Zheng, “Tensorflow: A system for large-...

    2016

  34. [42]

    Available: https://onnx.ai/

    “Onnx.” [Online]. Available: https://onnx.ai/

  35. [43]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. D. M.-W. C. Kenton and L. K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of NAACL-HLT, 2019, pp. 4171–4186

    2019

  36. [44]

    An image is worth 16x16 words: Trans- formers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Trans- formers for image recognition at scale,” in 9th International Conference on Le...

    2021

  37. [45]

    Mlp-mixer: An all-mlp architecture for vision,

    I. O. Tolstikhin, N. Houlsby, A. Kolesnikov, L. Beyer, X. Zhai, T. Un- terthiner, J. Yung, A. Steiner, D. Keysers, J. Uszkoreit, M. Lucic, and A. Dosovitskiy, “Mlp-mixer: An all-mlp architecture for vision,” in Advances in Neural Information Processing Systems , 2021, pp. 24 2...

    2021

  38. [46]

    Relay: A high-level IR for deep learning,

    J. Roesch, S. Lyubomirsky, M. Kirisame, J. Pollock, L. Weber, Z. Jiang, T. Chen, T. Moreau, and Z. Tatlock, “Relay: A high-level IR for deep learning,” CoRR, vol. abs/1904.08368, 2019

    1904 arXiv

  39. [47]

    Intel oneapi deep neural network library

    “Intel oneapi deep neural network library.” [Online]. Available: https://github.com/oneapi-src/oneDNN

  40. [48]

    Intel oneapi math kernel library

    “Intel oneapi math kernel library.” [Online]. Available: https://software.intel.com/content/ www/us/en/develop/tools/oneapi/- components/onemkl.html

  41. [49]

    Rammer: Enabling holistic deep learning compiler optimizations with rtasks,

    L. Ma, Z. Xie, Z. Yang, J. Xue, Y . Miao, W. Cui, W. Hu, F. Yang, L. Zhang, and L. Zhou, “Rammer: Enabling holistic deep learning compiler optimizations with rtasks,” in OSDI, 2020

    2020

  42. [50]

    Accelerating deep learning inference with cross-layer data reuse on gpus,

    X. Wang, G. Li, X. Dong, J. Li, L. Liu, and X. Feng, “Accelerating deep learning inference with cross-layer data reuse on gpus,” in Euro-Par 2020: Parallel Processing - 26th International Conference on Parallel and Distributed Computing , ser. Lecture Notes in Computer Science...

    2020

  43. [51]

    Efficient GPU spatial-temporal multitasking,

    Y . Liang, H. P. Huynh, K. Rupnow, R. S. M. Goh, and D. Chen, “Efficient GPU spatial-temporal multitasking,” IEEE Trans. Parallel Distributed Syst. , vol. 26, no. 3, pp. 748–760, 2015

    2015

  44. [52]

    On optimizing machine learning workloads via kernel fusion,

    A. Ashari, S. Tatikonda, M. Boehm, B. Reinwald, K. Campbell, J. Keen- leyside, and P. Sadayappan, “On optimizing machine learning workloads via kernel fusion,” in Proceedings of the 20th ACM SIGPLAN Sympo- sium on Principles and Practice of Parallel Programming, PPoPP, 2015, p...

    2015

  45. [53]

    Learning to optimize halide with tree search and random programs,

    A. Adams, K. Ma, L. Anderson, R. Baghdadi, T. Li, M. Gharbi, B. Steiner, S. Johnson, K. Fatahalian, F. Durand, and J. Ragan-Kelley, “Learning to optimize halide with tree search and random programs,” ACM Trans. Graph. , vol. 38, no. 4, pp. 121:1–121:12, 2019

    2019

  46. [54]

    Nimble: Efficiently compiling dynamic neural networks for model inference,

    H. Shen, J. Roesch, Z. Chen, W. Chen, Y . Wu, M. Li, V . Sharma, Z. Tatlock, and Y . Wang, “Nimble: Efficiently compiling dynamic neural networks for model inference,” in Proceedings of Machine Learning and Systems, 2021

    2021

  47. [55]

    DISC: A dynamic shape compiler for machine learning workloads,

    K. Zhu, W. Zhao, Z. Zheng, T. Guo, P. Zhao, J. Bai, J. Yang, X. Liu, L. Diao, and W. Lin, “DISC: A dynamic shape compiler for machine learning workloads,” in EuroMLSys@EuroSys 2021, Proceedings of the 1st Workshop on Machine Learning and Systems Virtual Event , E. Yoneki and P...

    2021

  48. [56]

    Automatic generation of high-performance quantized machine learning kernels,

    M. Cowan, T. Moreau, T. Chen, J. Bornholt, and L. Ceze, “Automatic generation of high-performance quantized machine learning kernels,” in CGO ’20: 18th ACM/IEEE International Symposium on Code Genera- tion and Optimization , 2020, pp. 305–316

    2020

  49. [57]

    A code generator for high-performance tensor contractions on gpus,

    J. Kim, A. Sukumaran-Rajam, V . Thumma, S. Krishnamoorthy, A. Pa- nyala, L. Pouchet, A. Rountev, and P. Sadayappan, “A code generator for high-performance tensor contractions on gpus,” in IEEE/ACM Inter- national Symposium on Code Generation and Optimization,CGO , 2019, pp. 85–95

    2019

  50. [58]

    Neoflow: A flexible framework for enabling efficient compilation for high performance DNN training,

    S. Zheng, R. Chen, Y . Jin, A. Wei, B. Wu, X. Li, S. Yan, and Y . Liang, “Neoflow: A flexible framework for enabling efficient compilation for high performance DNN training,” IEEE Trans. Parallel Distributed Syst., vol. 33, no. 11, pp. 3220–3232, 2022

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.