Pith. sign in

REVIEW 5 major objections 5 minor 43 references

ComFuse: Fusing Complex Memory-Intensive Subgraphs with Compute-Intensive Kernels For Modern GPU Architectures

T0 review · 5 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that joint compute-memory fusion, through the Stage-Stream Execution Model and B2BGEMM scheduling, can fuse reduction-heavy subgraphs into GEMM kernels and outperform the standard PyTorch compiler by up to 1.24x while elim

desk verdict ComFuse is a real, mostly well-executed systems contribution with a novel Stage-Stream fusion model and B2BGEMM schedule; the main gaps are the missing artifact, an overbroad abstract that the self-attention results contradict, and unstated limits on the tile-locality assumption. read the letter →

arxiv 2608.03537 v1 pith:7PHI5YKC submitted 2026-08-04 cs.AR

Di Mu , Tengyuan Jin , Zhenkun Wang , Jialin Yang , Yusen Li , Mian Huo , Shusong Guo , Gang Wang
show 1 more author
Xiaoguang Liu
This is my paper
classification cs.AR
keywords operatorfusionGPUcompilationkernelschedulingmemory-intensiveepiloguesreductionback-to-backGEMMthreadblockclustersdeeplearningcompilers
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most GPU compilers treat matrix multiplications and their surrounding normalization, elementwise, and reduction operations as separate kernels, forcing intermediate tensors to be written to global memory and reread. This paper tries to remove that boundary with a scheduling model that streams a MatMul's output tiles directly into the downstream operations while the values are still on-chip, using cluster-level coordination to handle the synchronization reductions require. It claims this joint compute-memory fusion consistently outperforms the default compiler of a major deep-learning framework across all tested workloads—up to 1.24x end-to-end, and up to about 9.93x on the portion of runtime left after subtracting the matrix-multiply cost. It also fuses two GEMMs separated by an elementwise/reduction op, claiming the intermediate matrix never touches global memory, with measured speedups up to 1.97x on one attention variant. The benefit is conditional: when the reduction stage is heavy relative to the GEMM, or the tile count is too small to fill the pipeline, an unfused or specialized kernel can win, as the paper's self-attention result shows.

What carries the argument

The Stage-Stream Execution Model is the load-bearing mechanism: it uses reduction operators as stage boundaries, splits a memory-intensive epilogue into sequentially chained DAGs, and streams MatMul accumulator tiles through elementwise nodes at fragment granularity, pausing only at reduction points for a three-level hierarchical aggregation (intra-warp shuffle, intra-warp-group shared memory, inter-CTA via distributed shared memory). Its companion mechanism is the B2BGEMM cluster-cooperative dataflow schedule, which keeps the post-processed intermediate tile S_tile register-resident and distributes the second GEMM's output-tile accumulation across the cluster, followed by a four-stage pipel

What would settle it

Use profiler counters to count global-memory store and load bytes for the intermediate tensor in a fused GEMM + ReLU + GEMM kernel; the paper's zero-materialization claim implies zero bytes for that tensor, so any measured nonzero traffic for the full intermediate would refute the claim.

Watch

Extended reading notes

Core claim

ComFuse's central claim is that the usual materialization boundary between a compute-intensive GEMM and a memory-intensive epilogue is unnecessary for a broad class of graphs. Under two-dimensional tiling of the GEMM output P into tiles T_{m,n}, a single-dimension reduction only needs the tile strip sharing the same non-reduction index m; the paper uses this locality to build a Stage-Stream Execution Model in which reductions act as stage boundaries and the epilogue is decomposed into chained dataflow DAGs. Fragment-sized elementwise work streams through registers, reduction nodes pause the stream for a three-level hierarchical aggregation (warp shuffles, shared memory, then cross-CTA aggreg

Load-bearing premise

Everything rests on reductions in the fused epilogue aggregating along only one logical dimension, so each reduction can be completed by synchronizing only the tile strip that shares the same row index; if a reduction needs values from multiple rows or the whole tensor, the on-chip streaming stages no longer cover it and the intermediate must go to global memory.

Editorial extensions

If this is right

  • Normalizations such as LayerNorm, RMSNorm, and softmax-style score normalization can become epilogue stages of a GEMM kernel instead of separate kernels, reducing intermediate-tensor traffic.
  • The overlap of matrix-unit computation with general-purpose-core elementwise/reduction work means downstream memory-intensive stages can be hidden behind computation rather than adding latency after each GEMM.
  • Back-to-back GEMMs with an intervening elementwise/reduction op can run with the intermediate matrix staying in registers and on-chip paths, eliminating the global-memory store and reload.
  • Because the compiler lowers high-level tensor subprograms into kernel templates automatically, model variants that reorder or extend the epilogue do not require handwritten fused kernels.
  • The benefit is workload-dependent: it grows when the epilogue is computationally heavy and the pipeline reaches steady state, and can vanish when few tiles are assigned per CTA or when a specialized attention kernel is available.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The tile-strip locality argument implies the same fusion should extend to other single-axis reductions (e.g., max for top-k or log-sum-exp for online softmax), but not to reductions whose aggregation spans multiple non-reduction dimensions; fusing a two-dimensional reduction would likely force a return to global materialization.
  • The B2BGEMM schedule's elimination of intermediate traffic presumes the second GEMM's contraction dimension maps onto the cluster's layout along N1; for shapes where a CTA's register-resident tile does not align with the second GEMM's contraction segments, some inter-CTA shuffle or partial recomputation would be needed, so the zero-traffic claim likely has a shape-dependent boundary.
  • A testable extension is a cost model that predicts when the fused kernel wins: compare predicted GEMM time versus predicted epilogue time and choose fusion only when the GEMM is heavy enough to hide the epilogue; the paper's self-attention result is exactly the case where this condition fails.
  • The same stage-stream reduction idea could be composed with online single-pass softmax algorithms to fuse attention score computation across longer sequence tiles, since the reduction staging already separates local aggregation from broadcast.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. ComFuse is a GPU compilation system that fuses a compute-intensive MatMul (or two back-to-back MatMuls) with a memory-intensive subgraph containing elementwise and reduction operations. The core ideas are a Stage-Stream Execution Model that treats reductions as stage boundaries, a three-level hierarchical reduction using Thread Block Clusters and DSMEM, and a B2BGEMM scheduling paradigm that keeps the intermediate matrix in registers/on-chip memory. The system lowers high-level tensor subprograms automatically into CUTLASS EVT kernels. Evaluation compares against TorchInductor and TensorRT on MatMul+norm/softmax workloads and on Self-Attention, Target-Attention, and DLRM-style B2BGEMM workloads, reporting up to 1.24x, 1.97x, and 1.23x speedups respectively, but also a significant slowdown for Self-Attention.

Significance. The problem is well motivated: current compilers materialize MatMul outputs before epilogue reductions, and the paper's use of Hopper/Blackwell cluster DSMEM and TMA to consume intermediate tiles on-chip is a plausible and timely direction. The automatic lowering from a high-level tensor graph to CUTLASS EVT without fixed pattern matching is a genuine strength, as is the extension to B2BGEMM. The paper explicitly evaluates against strong baselines and acknowledges at least one failure case. If the scope and architectural assumptions are stated precisely, the Stage-Stream model and B2BGEMM scheduling are a useful contribution. The main gaps are that the central 'no GMEM materialization' claim is currently broader than the demonstrated conditions, the abstract overstates the results relative to the self-attention data, and the evaluation lacks basic reproducibility details (GPU model, precision, variance, artifact). No circularity is apparent; the performance claims are measured against external baselines, not derived from quantities defined by ComFuse itself.

major comments (5)
  1. [Sec. III, Eq. (1)] The tile-locality claim is the foundation of the Stage-Stream model, but it is stated without the conditions under which it holds. The synchronization boundary is confined to {T_{m,n}}_n only when the reduction is along the second (column) dimension of P and when all CTAs in that strip can be co-scheduled in one Thread Block Cluster. Reductions along the first dimension require the complementary strip, and for a strip wider than the cluster limit (e.g., N=8192 with TileN=128 gives N_T=64, exceeding the 16-CTA limit on current hardware) the proposed DSMEM-only aggregation cannot be applied. Neither condition is stated or tested. Please either restrict the claims to reductions that fit this pattern and cluster bound, or describe and evaluate a multi-cluster fallback.
  2. [Sec. IV-A, Eqs. (4)-(5)] The claim that B2BGEMM 'completely eliminates the GMEM materialization and reload of the intermediate matrix' depends on all CTAs holding contiguous S_tiles along N1 being co-scheduled in one cluster. Equation (5) aggregates partial results across the set C of co-scheduled CTAs; for N1 large enough that N1/TileN exceeds the cluster CTA limit, or for an intervening F that reduces along a different dimension than the tiling, the described register-resident consumption is not the mechanism that executes. Since the paper evaluates only small N1 (Target-Attention, DLRM with feature dims 512/128/256), the claim is not demonstrated in the general setting. Please state the supported geometries and the cluster-size constraint, or provide a multi-cluster mechanism.
  3. [Abstract / Sec. VI-B2, Fig. 9] The Abstract claims ComFuse kernels 'outperform those produced by TorchInductor across post-norm workloads and various complex computation scenarios', and Sec. VI-A2 says ComFuse 'consistently outperforms TorchInductor across all evaluated workloads'. This is contradicted by the Self-Attention results in Fig. 9, where the text itself states ComFuse exhibits a substantial performance gap and TorchInductor's FlashAttention path is best. The paper should qualify the claim: ComFuse outperforms TorchInductor on the Stage-Stream workloads and on Target-Attention/DLRM, but not on Self-Attention. As written, the central claim in the abstract is inaccurate.
  4. [Sec. VI-A, experimental setup] The evaluation omits the GPU model and numerical precision. The paper is about modern GPU architectures and cluster primitives, and the reported speedups are often small (1.03-1.10x); without hardware identity, precision, standard deviations, or number of runs, the results cannot be reproduced or compared across platforms. In addition, no code repository or commit hash is provided. Please add hardware details, precision (FP16/BF16/FP32), variance, and an artifact link or a clear statement of availability.
  5. [Sec. VI-A2, residual-time metric] The residual time defined as Time(Total)-Time(MatMul) is not a sound measure for isolating the memory-intensive subgraph in fused execution. The entire point of the fusion is that MatMul and epilogue overlap, so subtracting a serial MatMul time from total time can produce values that reflect scheduling overlap rather than actual epilogue cost, and small total-time differences are amplified. The 9.93x residual-speedup figure for MSTS should be interpreted very cautiously; please either report measured standalone epilogue costs with a proper overlap model, or remove the residual-time analysis from the conclusions.
minor comments (5)
  1. [Sec. V-A] Typo: 'translating the program into the the Fusion Spec IR' duplicates 'the'.
  2. [Fig. 7 and Fig. 9] The text in Sec. VI-A2 says TorchInductor execution time is normalized to 1.0, but the plots are described as showing speedups and the values below 1.0 are treated as slowdowns. The figures need a clear statement of whether bars are speedup or normalized time, and a legend explaining the numeric annotations.
  3. [Fig. 7-11] Legends contain 'T orchInductor' (spacing) and no explanation of the row/column labels or the batch-size axis. Please clean up the figure annotations.
  4. [Sec. III-A] The dual-mode adaptive aggregation uses a crossover threshold C_N, but the paper does not give its value, how it is chosen, or sensitivity to it. Since it is a free parameter, a brief sensitivity experiment or a stated default is needed.
  5. [Sec. VI-A2] The text says MSTS 'achieves the largest performance improvement', but Fig. 7 shows some MBLR speedups of 1.24x, larger than the typical MSTS values. Please make the comparison consistent with the plotted numbers.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the performance claims are benchmarked against external baselines and the scheduling constructions are not defined in terms of the results they explain.

full rationale

The paper's central contributions are a tiling observation (Eq. 1), a staged execution model (Sec. III), a B2BGEMM scheduling construction (Eqs. 2-5), and an automated lowering pipeline (Sec. V). None of these steps derives a predicted result from its own inputs. Eq. 1 is a statement about the synchronization scope of a reduction under a fixed tiling; it is an assumption about the reduction dimension, not a tautology, and any limitation there is a correctness/coverage issue, not circularity. The B2BGEMM equations (3)-(5) define the intermediate tile transformation and the segmented accumulation; the claim that this eliminates GMEM materialization follows from the construction, but it is then validated by measured comparisons against TorchInductor and TensorRT, which are external baselines. No parameter is fitted to benchmark outputs and then reported as a prediction. The paper contains no self-citation chain used to justify a central premise; references to CUTLASS and prior compiler work are background, not load-bearing reduction. The Self-Attention result, where ComFuse underperforms FlashAttention, is an honest external negative result. The residual-time metric is arithmetically defined, not used as evidence that would be circular. Overall, the derivation is self-contained in the sense relevant to circularity analysis: the empirical claims stand or fall on external measurements.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claims rest on hardware features (thread block clusters, DSMEM, TMA), on the tile-locality of epilogue reductions, and on the assumption that tensor-core MatMul and CUDA-core epilogue work can overlap. The only hand-chosen numerical heuristic visible is the crossover for dual-mode inter-CTA aggregation. No physical entities are introduced; the paper's contributions are compiler abstractions.

free parameters (1)
  • C_N crossover threshold in dual-mode adaptive aggregation = unspecified
    Sec III-A switches between peer-to-peer cross-broadcast and leader-follower inter-CTA aggregation based on 'when C_N is small' vs 'when C_N is large'; no value or tuning procedure is given, so the threshold is a hand-chosen implementation parameter.
assumptions (5)
  • domain assumption Target GPUs (Hopper/Blackwell) expose Thread Block Clusters, DSMEM, TMA, cluster-level sync, and co-scheduling of cooperating CTAs
    Invoked in Sec II-C and Sec III-A as the substrate for inter-CTA reduction and B2BGEMM register feeding.
  • domain assumption The epilogue reduction aggregates along only one logical dimension and its synchronization boundary is confined to tiles sharing the same non-reduction index (Eq. 1)
    Sec III opens the Stage-Stream model from this locality premise; reductions over the non-reduction dimension or over multiple dimensions are outside the stated mechanism.
  • domain assumption MainLoop and Epilogue use different hardware resources so epilogue latency can be overlapped with MatMul
    Sec III-C PingPong pipeline relies on this; Sec VI-B's self-attention case shows the overlap is not guaranteed.
  • domain assumption CTAs assigned to one reduction are co-scheduled concurrently in space and time within a cluster
    Sec III-A states ComFuse 'guarantees that cooperating CTAs are co-scheduled concurrently,' which is a hardware scheduler property, not proven.
  • domain assumption CUTLASS EVT and NVCC provide correct lower-level semantics for generated templates
    Sec V builds the compiler on CUTLASS and assumes template instantiations preserve semantics; the closure property in Sec V-B is stated, not machine-checked.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ComFuse: Fusing Complex Memory-Intensive Subgraphs with Compute-Intensive Kernels For Modern GPU Architectures." pith.science (2026). https://pith.science/paper/7PHI5YKC

@misc{pith2026260803537,
  author       = {Pith},
  title        = {Pith review of: ComFuse: Fusing Complex Memory-Intensive Subgraphs with Compute-Intensive Kernels For Modern GPU Architectures},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7PHI5YKC}},
  note         = {Machine review of arXiv:2608.03537}
}
read the original abstract

Modern deep learning workloads increasingly comprise heterogeneous computation graphs that combine compute-intensive operators with memory-intensive subgraphs. Existing deep learning compilers typically optimize these operator classes separately, creating rigid fusion boundaries that limit cross-operator optimization and on-chip data reuse. We observe that downstream memory-intensive operations can execute concurrently with compute-intensive operators, allowing their execution to be hidden behind computation; however, automatically exploiting this opportunity poses new compilation challenges. In this paper, we present ComFuse, an automated GPU compilation system that employs a novel operator fusion strategy to generate high-performance kernels for complex graph structures comprising compute-intensive operators and dependency-rich, memory-intensive elementwise-reduction subgraphs. ComFuse further supports the fusion of back-to-back GEMM (B2BGEMM) patterns, extending its applicability to more complex compute-memory interaction patterns. Additionally, it automatically lowers high-level tensor subprograms into optimized fused kernels, reducing the need for manual kernel engineering. Experimental results show that the fused kernels generated by ComFuse outperform those produced by TorchInductor across post-norm workloads and various complex computation scenarios, while supporting more flexible fusion patterns.

Figures

Figures reproduced from arXiv: 2608.03537 by the authors.

Figure 1
Figure 1. Joint Fusion of Compute-intensive and Memory-intensive Operators [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Data Flow in MatMul + RMSNorm Subgraph Structure [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Case for The Stage-Stream Execution Model: MatMul + LayerNorm [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Three-level Reduction Scheme in Stage-Stream Execution Model. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: B2BGEMM Parallel Task Schedule. materialize them in GMEM to allow GEMM2 to reload the required data. A. Dataflow Scheduling To reduce the memory access overhead caused by interme￾diate results in B2BGEMM, ComFuse introduces a cluster￾cooperative dataflow scheduling mec…
Figure 6
Figure 6. Figure 6: ComFuse Compilation Stack. patterns, ComFuse naturally generalizes to diverse subgraph variants, bypassing the brittleness of pattern-matching com￾pilers against structural variations. A. Code Translate The ComFuse system translates a tensor subprogram across three dis…
Figure 7
Figure 7. Figure 7: Overall performance comparison for Stage-Stream Execution Model. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Residual-time performance comparison for Stage-Stream Execution Model, as Residual-time is defined as [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Performance for Self-Attention. 2) Overall Results [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Performance for Target-Attention. 1 8 16 32 64 Batch L 0.8 0.9 1.0 1.1 1.2 1.3 DLRM Bottom MLP 0.94 1.10 1.18 1.21 1.23 1.00 1.00 1.00 1.00 1.00 1.08 1.20 1.21 1.19 1.21 ComFuse TorchInductor TensorRT [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: Performance for DLRM Bottom MLP. that the optimization capability of TensorRT largely relies on graph-level pattern matching. For regular structures and standard operator compositions, TensorRT can trigger highly optimized kernels through predefined patterns. However,…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 34 canonical work pages

  1. [1]

    TensorFlow: A system for Large-Scale machine learning,

    M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. Tucker, V . Vasudevan, P. Warden, M. Wicke, Y . Yu, and X. Zheng, “TensorFlow: A system for Large-Scale machine learning,” in12th USENIX Symposium on Operating Systems Design and Im...

    work page 2016

  2. [2]

    Learning to op- timize halide with tree search and random programs,

    A. Adams, K. Ma, L. Anderson, R. Baghdadi, T.-M. Li, M. Gharbi, B. Steiner, S. Johnson, K. Fatahalian, F. Durandet al., “Learning to op- timize halide with tree search and random programs,”ACM Transactions on Graphics (TOG), vol. 38, no. 4, pp. 1–12, 2019. 11

    work page 2019

  3. [3]

    Pytorch 2: Faster machine learning through dynamic python bytecode transformation and graph compilation,

    J. Ansel, E. Yang, H. He, N. Gimelshein, A. Jain, M. V oznesensky, B. Bao, P. Bell, D. Berard, E. Burovskiet al., “Pytorch 2: Faster machine learning through dynamic python bytecode transformation and graph compilation,” inProceedings of the 29th ACM international conference on architectural support for programming languages and operating systems, volume ...

    2024

  4. [4]

    TVM: An automated end-to-end optimizing compiler for deep learning,

    T. Chen, T. Moreau, Z. Jiang, L. Zheng, E. Yan, H. Shen, M. Cowan, L. Wang, Y . Hu, L. Cezeet al., “TVM: An automated end-to-end optimizing compiler for deep learning,” in13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), 2018, pp. 578–594

    work page 2018

  5. [5]

    Learning to optimize tensor programs,

    T. Chen, L. Zheng, E. Yan, Z. Jiang, T. Moreau, L. Ceze, C. Guestrin, and A. Krishnamurthy, “Learning to optimize tensor programs,”Ad- vances in Neural Information Processing Systems, vol. 31, 2018

    work page 2018

  6. [6]

    Evt: Accelerating deep learning training with epilogue visitor tree,

    Z. Chen, A. Kerr, R. Cai, J. Kosaian, H. Wu, Y . Ding, and Y . Xie, “Evt: Accelerating deep learning training with epilogue visitor tree,” in Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3, 2024, pp. 301–316

    work page 2024

  7. [7]

    cuDNN: Efficient primitives for deep learning,

    S. Chetlur, C. Woolley, P. Vandermersch, J. Cohen, J. Tran, B. Catanzaro, and E. Shelhamer, “cuDNN: Efficient primitives for deep learning,”arXiv preprint arXiv:1410.0759, 2014. [Online]. Available: https://arxiv.org/abs/1410.0759

    arXiv 2014

  8. [8]

    Flashattention-2: Faster attention with better parallelism and work partitioning,

    T. Dao, “Flashattention-2: Faster attention with better parallelism and work partitioning,” inInternational Conference on Learning Represen- tations (ICLR), 2024

    work page 2024

Show all 43 references
  1. [9]

    Analyzing the impact of kernel fusion on gpu tensor operation performance: A systematic performance study,

    M. Dodovi ´c, M. Veselinovi ´c, and M. Mi ˇsi´c, “Analyzing the impact of kernel fusion on gpu tensor operation performance: A systematic performance study,”Electronics, vol. 15, no. 5, 2026. [Online]. Available: https://www.mdpi.com/2079-9292/15/5/1034

    2026

  2. [10]

    Coda: Rewriting transformer blocks as gemm-epilogue programs,

    H. Guo, J. Zhang, A. Menon, D. Guessous, V . Thakkar, Y . Kim, and T. Dao, “Coda: Rewriting transformer blocks as gemm-epilogue programs,”arXiv preprint arXiv:2605.19269, 2026

    2026 arXiv

  3. [11]

    Mixed-input matrix multiplication performance optimiza- tions,

    M. Gupta, “Mixed-input matrix multiplication performance optimiza- tions,” https://research.google/blog/mixed-input-matrix-multiplication- performance-optimizations/, 2024, google Research Blog, accessed June 10, 2026

    2024

  4. [12]

    Fireiron: A data-movement-aware scheduling language for gpus,

    B. Hagedorn, A. S. Elliott, H. Barthels, R. Bodik, and V . Grover, “Fireiron: A data-movement-aware scheduling language for gpus,” in Proceedings of the ACM International Conference on Parallel Architec- tures and Compilation Techniques, 2020, pp. 71–82

    2020

  5. [13]

    Making deep learning go brrrr from first principles,

    H. He, “Making deep learning go brrrr from first principles,” https: //horace.io/brrrr intro.html, 2022

    2022

  6. [14]

    Data movement is all you need: A case study on optimizing transformers,

    A. Ivanov, N. Dryden, T. Ben-Nun, S. Li, and T. Hoefler, “Data movement is all you need: A case study on optimizing transformers,” Proceedings of Machine Learning and Systems, vol. 3, pp. 711–732, 2021

    2021

  7. [15]

    In-datacenter performance analysis of a tensor processing unit,

    N. P. Jouppi, C. Young, N. Patil, D. Patterson, G. Agrawal, R. Bajwa, S. Bates, S. Bhatia, N. Boden, A. Borcherset al., “In-datacenter performance analysis of a tensor processing unit,” inProceedings of the 44th annual international symposium on computer architecture, 2017, pp. 1–12

    2017

  8. [16]

    onednn graph compiler: A hybrid approach for high-performance deep learning compilation,

    J. Li, Z. Qin, Y . Mei, J. Cui, Y . Song, C. Chen, Y . Zhang, L. Du, X. Cheng, B. Jinet al., “onednn graph compiler: A hybrid approach for high-performance deep learning compilation,” in2024 IEEE/ACM International Symposium on Code Generation and Optimization (CGO). IEEE, 2024...

    2024

  9. [18]

    Deep learning recommendation model for personalization and recommenda- tion systems,

    M. Naumov, D. Mudigere, H.-J. M. Shi, J. Huang, N. Sundaraman, J. Park, X. Wang, U. Gupta, C.-J. Wu, A. G. Azzoliniet al., “Deep learning recommendation model for personalization and recommenda- tion systems,”arXiv preprint arXiv:1906.00091, 2019

    1906 arXiv

  10. [19]

    Dnnfusion: accelerating deep neural networks execution with advanced operator fusion,

    W. Niu, J. Guan, Y . Wang, G. Agrawal, and B. Ren, “Dnnfusion: accelerating deep neural networks execution with advanced operator fusion,” inProceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, 2021, pp. 883–898

    2021

  11. [20]

    Cutlass: Fast linear algebra in cuda c++,

    NVIDIA, “Cutlass: Fast linear algebra in cuda c++,”NVIDIA Developer Blog, December 2017. [Online]. Available: https://developer.nvidia.com/ blog/cutlass-linear-algebra-cuda/

    2017

  12. [21]

    NVIDIA TensorRT: An sdk for high-performance deep learn- ing inference,

    NVIDIA, “NVIDIA TensorRT: An sdk for high-performance deep learn- ing inference,” https://developer.nvidia.com/tensorrt, 2017, accessed: 2026-05-31

    2017

  13. [22]

    cuBLAS: The nvidia cuda basic linear algebra subroutines library,

    NVIDIA, “cuBLAS: The nvidia cuda basic linear algebra subroutines library,” https://docs.nvidia.com/cuda/cublas/, 2026, accessed: 2026-05- 31

    2026

  14. [23]

    Cuda c++ programming guide,

    NVIDIA, “Cuda c++ programming guide,” https://docs.nvidia.com/cuda/ cuda-c-programming-guide/, 2026, accessed: 2026-05-31

    2026

  15. [24]

    Triton: An open-source programming language for writing highly efficient gpu code,

    OpenAI, “Triton: An open-source programming language for writing highly efficient gpu code,” https://github.com/triton-lang/triton, 2019, accessed: 2026-05-31

    2019

  16. [25]

    Automatic kernel fusion for image processing dsls,

    B. Qiao, O. Reiche, F. Hannig, and J. Teich, “Automatic kernel fusion for image processing dsls,” inProceedings of the 21st International Workshop on Software and Compilers for Embedded Systems, 2018, pp. 76–85

    2018

  17. [26]

    Tensor program optimization with probabilistic programs,

    J. Shao, X. Zhou, S. Feng, B. Hou, R. Lai, H. Jin, W. Lin, M. Masuda, C. H. Yu, and T. Chen, “Tensor program optimization with probabilistic programs,” inAdvances in Neural Information Processing Systems, vol. 35, 2022, pp. 35 783–35 796

    2022

  18. [27]

    Astra: Exploiting predictability to optimize deep learning,

    M. Sivathanu, T. Chugh, S. S. Singapuram, and L. Zhou, “Astra: Exploiting predictability to optimize deep learning,” inProceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, 2019, pp. 909–923

    2019

  19. [28]

    XLA: Optimizing compiler for machine learning,

    TensorFlow, “XLA: Optimizing compiler for machine learning,” https: //www.tensorflow.org/xla, 2017, accessed: 2026-05-31

    2017

  20. [29]

    Tensor comprehen- sions: Framework-agnostic high-performance machine learning abstrac- tions,

    N. Vasilache, O. Zinenko, T. Theodoridis, P. Goyal, Z. DeVito, W. S. Moses, S. Verdoolaege, A. Adams, and A. Cohen, “Tensor comprehen- sions: Framework-agnostic high-performance machine learning abstrac- tions,”arXiv preprint arXiv:1802.04730, 2018

    2018 arXiv

  21. [30]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” inAdvances in Neural Information Processing Systems, 2017, pp. 5998–6008

    2017

  22. [31]

    Memory is all you need: An overview of compute-in-memory architectures for acceler- ating large language model inference,

    C. Wolters, X. Yang, U. Schlichtmann, and T. Suzumura, “Memory is all you need: An overview of compute-in-memory architectures for acceler- ating large language model inference,”arXiv preprint arXiv:2406.08413, 2024

    2024 arXiv

  23. [32]

    Mirage: A{Multi-Level}superoptimizer for tensor programs,

    M. Wu, X. Cheng, S. Liu, C. Shi, J. Ji, M. K. Ao, P. Velliengiri, X. Miao, O. Padon, and Z. Jia, “Mirage: A{Multi-Level}superoptimizer for tensor programs,” in19th USENIX Symposium on Operating Systems Design and Implementation (OSDI 25), 2025, pp. 21–38

    2025

  24. [33]

    {PluS}: Highly efficient and expandable{ML}compiler with pluggable graph schedules,

    R. Wu, Z. Zheng, F. Zhang, C. Liu, Z. Pan, J. Zhai, and X. Du, “{PluS}: Highly efficient and expandable{ML}compiler with pluggable graph schedules,” in2025 USENIX Annual Technical Conference (USENIX ATC 25), 2025, pp. 647–663

    2025

  25. [34]

    Bolt: Bridg- ing the gap between auto-tuners and hardware-native performance,

    J. Xing, L. Wang, S. Zhang, J. Chen, A. Chen, and Y . Zhu, “Bolt: Bridg- ing the gap between auto-tuners and hardware-native performance,” Proceedings of Machine Learning and Systems, vol. 4, pp. 204–216, 2022

    2022

  26. [35]

    Demystifying tensor cores to optimize half-precision matrix multiply,

    D. Yan, W. Wang, and X. Chu, “Demystifying tensor cores to optimize half-precision matrix multiply,” in2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, 2020, pp. 634–643

    2020

  27. [36]

    Flashlight: Pytorch compiler extensions to accelerate attention variants,

    B. You, I. Wang, Z. S. Mustafaoglu, A. Jangda, A. Moreira, R. Dathathri, D. Mahajan, K. Pingaliet al., “Flashlight: Pytorch compiler extensions to accelerate attention variants,”arXiv preprint arXiv:2511.02043, 2025

    2025 arXiv

  28. [37]

    Mcfuser: High- performance and rapid fusion of memory-bound compute-intensive operators,

    Z. Zhang, D. Yang, X. Zhou, and D. Cheng, “Mcfuser: High- performance and rapid fusion of memory-bound compute-intensive operators,” inSC24: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 2024, pp. 1–15

    2024

  29. [38]

    Apollo: Automatic partition-based operator fusion through layer by layer optimization,

    J. Zhao, X. Gao, R. Xia, Z. Zhang, D. Chen, L. Chen, R. Zhang, Z. Geng, B. Cheng, and X. Jin, “Apollo: Automatic partition-based operator fusion through layer by layer optimization,”Proceedings of Machine Learning and Systems, vol. 4, pp. 1–19, 2022

    2022

  30. [39]

    Operator fusion scheduling optimization for tvm deep learning compilers,

    G. Zheng, J. Li, W. Gao, L. Han, Y . Li, and J. Xu, “Operator fusion scheduling optimization for tvm deep learning compilers,” in2023 3rd International Symposium on Computer Technology and Information Science (ISCTIS). IEEE, 2023, pp. 273–277

    2023

  31. [40]

    Ansor: Generating{High-Performance}tensor programs for deep learning,

    L. Zheng, C. Jia, M. Sun, Z. Wu, C. H. Yu, A. Haj-Ali, Y . Wang, J. Yang, D. Zhuo, K. Senet al., “Ansor: Generating{High-Performance}tensor programs for deep learning,” in14th USENIX symposium on operating systems design and implementation (OSDI 20), 2020, pp. 863–879

    2020

  32. [41]

    Chimera: An analytical optimizing framework for effective 12 compute-intensive operators fusion,

    S. Zheng, S. Chen, P. Song, R. Chen, X. Li, S. Yan, D. Lin, J. Leng, and Y . Liang, “Chimera: An analytical optimizing framework for effective 12 compute-intensive operators fusion,” in2023 IEEE International Sym- posium on High-Performance Computer Architecture (HPCA). IEEE, ...

    2023

  33. [42]

    Astitch: enabling a new multi-dimensional optimization space for memory-intensive ml training and inference on modern simt architectures,

    Z. Zheng, X. Yang, P. Zhao, G. Long, K. Zhu, F. Zhu, W. Zhao, X. Liu, J. Yang, J. Zhaiet al., “Astitch: enabling a new multi-dimensional optimization space for memory-intensive ml training and inference on modern simt architectures,” inProceedings of the 27th ACM Interna- tion...

    2022

  34. [43]

    Fusionstitching: boosting memory intensive computations for deep learning workloads,

    Z. Zheng, P. Zhao, G. Long, F. Zhu, K. Zhu, W. Zhao, L. Diao, J. Yang, and W. Lin, “Fusionstitching: boosting memory intensive computations for deep learning workloads,”arXiv preprint arXiv:2009.10924, 2020

    2009 arXiv

  35. [44]

    Deep interest network for click-through rate prediction,

    G. Zhou, C. Song, X. Zhu, Y . Fan, H. Zhu, X. Ma, Y . Yan, J. Jin, H. Li, and K. Gai, “Deep interest network for click-through rate prediction,” inProceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2018, pp. 1059–1068. 13

    2018

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.