Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Dato: A Task-Based Programming Model for Dataflow Accelerators

T0 review · 4 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Type-checked streams and layouts let task-based Dato match hand-tuned NPU code with 12x less code.

desk verdict Useful programming model with credible NPU results, but the type-safety claim is unproven for concurrent tasks and the printed FPGA code doesn't run. read the letter →

arxiv 2509.06794 v1 pith:3CFMSQEN submitted 2025-09-08 cs.PL cs.ARcs.LG

Shihan Fang , Hongzheng Chen , Niansong Zhang , Jiajie Li , Han Meng , Adrian Liu , Zhiru Zhang This is my paper
classification cs.PLcs.ARcs.LG
keywords dataflowacceleratorstask-basedprogrammingmodelstreamtypeslayoutvirtual-to-physicalmappingDMAschedulingNPU/FPGAcodegenerationlineartypesystem
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that on dataflow accelerators, the trade-off between performance and productivity is not fundamental. Dato is a Python-embedded task-based language where communication is typed as hardware FIFO streams and tensor sharding is typed as layouts, so partitioning, placement, and buffer safety become compiler responsibilities. The compiler builds a virtual task graph and searches over topology-changing primitives to collapse it onto the physical array. Reported NPU results include up to 84% GEMM utilization and a 2.81x attention speedup over a commercial close-to-metal framework; on FPGA the generated systolic array reaches 98% of theoretical peak. If these numbers hold, accelerator programmers no longer have to choose between fine-grained control and high-level productivity.

What carries the argument

Three mechanisms carry the argument. First, the Stream type: put/get are typed with linear capability tokens Free and Used, and get returns a linear ready-future consumed exactly once by await; forward abstract interpretation over the CFG checks the token balance, making overflow and underflow compile-time errors. Second, the Layout type: a refinement type labeling each tensor axis Replicated or Sharded, with a pending-effect set that records which reductions still need an allreduce, so the compiler owns partitioning and collective insertion. Third, virtual-to-physical mapping: tasks declared with a mapping directive instantiate a virtual task lattice, and a branch-and-reduce search applies

What would settle it

Execute a Dato program that passes the stream type check with balanced put/get counts through the token-based DMA scheduler on the NPU at full throughput: a FIFO overflow, stall, or deadlock would refute the soundness claim. Feed a cyclic control-flow graph that reuses a stream across iterations to the type checker and see whether the token-multiset abstract interpretation reaches a fixed point; non-convergence would break the compile-time guarantee. Re-synthesize the 16x16 FPGA systolic array and re-run the i8xi4 GEMM roofline on the same boards: reproducing the claimed 300 MHz, 84% utilizati

Watch

Extended reading notes

Core claim

The paper's central claim is that communication and data layout can be first-class, statically checked types in a task-based model, and that a compiler using them can match or beat hand-tuned low-level flows. The Stream type enforces FIFO safety with linear capability tokens: put consumes a Free token and produces a Used token, get reverses this and returns a linear ready-future that await consumes exactly once, so overflow and underflow are untypeable by construction. The Layout type labels each tensor axis as replicated or sharded and carries pending-collective effects, allowing the compiler to insert allreduces automatically and reject incompatible sharding. From the annotated program, a

Load-bearing premise

The load-bearing premise is that the stream type checker and the DMA port scheduler are sound as implemented: no well-typed program can overflow or underflow a FIFO, futures are consumed exactly once, and the liveness-based port schedule cannot deadlock. The paper asserts these guarantees without a formal metatheorem or machine-checked artifact, and the abstract interpretation's convergence is not specified in full detail.

Editorial extensions

If this is right

  • Multi-kernel transformer pipelines can run as single spatially fused dataflow pipelines instead of sequential kernel launches, which produces the reported 2.81x attention and 1.64x FFN speedups.
  • GEMM code drops from 101 lines in the close-to-metal interface to 8 lines in Dato while matching or exceeding throughput, so the performance/productivity trade-off is eliminated for such kernels.
  • The same program retargets from a coarse-grained NPU to a fine-grained FPGA: the FPGA backend skips NPU-specific DMA passes and emits HLS C++, and the generated systolic array reaches 150 of 153.6 GOP/s theoretical peak.
  • Mixed-precision GEMM with i8 activations and i4 weights becomes expressible and reaches 84% hardware utilization, a configuration the comparison frameworks do not support.
  • Stream and layout errors such as FIFO overflow or underflow, unconsumed futures, and incompatible sharding surface at compile time rather than after synthesis or on-device debugging.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the linear-token typing gets a formal metatheorem, the same machinery could generalize to other streaming-hardware targets such as GPU tensor-memory-accelerator queues, where compilers must statically prove buffers never stall.
  • Because the token model is explicitly untimed, FIFO depths are not inferred by the type system; a natural testable extension is to let the mapping search also sweep stream capacities, using measured throughput as the objective.
  • The productivity story is strongest for regular tiled kernels: the FPGA systolic-array example still expresses per-region topology through compile-time meta_if branches, so the automatic-mapping claim presumes users can describe spatial structure in layout terms.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents Dato, a Python-embedded, task-based programming model for dataflow accelerators. Its central claims are that (i) explicit Stream types with linear capability tokens make FIFO overflow/underflow statically untypeable, (ii) Layout refinement types express sharding, and (iii) a virtual-to-physical mapping mechanism with .bundle()/.chain() primitives automatically maps a virtual task graph onto physical PEs while satisfying resource and port constraints. The compiler lowers to MLIR-AIE for AMD NPUs and C++ HLS for FPGAs. Experiments claim up to 84% hardware utilization for GEMM, a 2.81x speedup over IRON on multi-head attention, 98% of theoretical peak on a generated FPGA systolic array, and 12x fewer lines of code compared with IRON. The evaluation uses IRON and ARIES as NPU baselines and Allo as the FPGA baseline.

Significance. If the claims hold, Dato is a significant step toward eliminating the productivity/performance trade-off in dataflow accelerator programming: it gives developers explicit stream and layout abstractions while shifting the burden of partitioning, communication, and physical mapping to the compiler. The NPU results are concrete and measured against independent baselines (IRON, ARIES), with a reproducible-sounding methodology (20 warm-up + 400 timed runs), and the FPGA results include post place-and-route timing, which is rare in DSL papers. The ablation study is useful for attributing gains. The main risks are formal: the stream type system is presented as a safety guarantee but lacks a parallel-composition rule, and the DMA scheduling pass that is load-bearing for the NPU results is described at a high level without a correctness argument. These issues are fixable but need to be addressed before the safety and performance claims can be fully accepted.

major comments (4)
  1. [§3.3, Fig. 4] The paper states that overflow and underflow become 'untypeable by construction', but the typing rules shown in Fig. 4 are ordinary sequential rules, with the linear context Δ threaded through a single control-flow path. Dato's execution model is a graph of concurrently executing @task instances (§3.2, Fig. 2c). No rule is given for composing the token multisets of a producer task and a consumer task, and no account of interleavings is provided. A forward abstract interpretation over a single CFG cannot, by itself, establish that every schedule of concurrently running tasks respects FIFO capacity. The note in §3.3 that the model is untimed does not resolve this, since safety under all schedules is a property of the task graph and not of timing. The authors should either supply a parallel-composition rule / schedule-parametric analysis and a soundness argument, or explicitly weaken the cl
  2. [§7.2.1, Fig. 8] The headline 'up to 84% hardware utilization' comes from the configuration labeled i8xi4 (Tbest), whereas the plain i8xi4 curve reaches 64.22% and the i8xi8 curve reaches 61.58%. The text says that using i4 and i8 tensors allows larger tiling configurations, but 'Tbest' is never defined, and no selection procedure is given. If Tbest is the result of searching over tiling configurations and keeping the best measured one, the 84% figure is a best-case peak rather than the default path. The reader cannot tell whether this configuration is a fair representative of Dato's automatic mapping or a hand-picked point. Please define Tbest, describe the configuration search/tuning protocol, and report the performance of the default or automatically selected configuration. The shaded variance regions are also not quantified; given 400 timed runs, reporting standard deviations or confidence intervals
  3. [§6, DMA scheduling pass] The DMA scheduling pass is described as a 'token-based scheduling strategy grounded in a coarse liveness analysis' with epochs, multicast merging, spatial coalescing, and port-aware splitting. The load-bearing claim is that 'ports are acquired at first use and released at last use, ensuring arguments that share ports have non-overlapping lifetimes.' This invariant is essential for correctness under the NPU's 2-in/2-out port limit, and the GEMM and attention speedups depend on the generated schedules being deadlock-free. However, the epoch construction, the token assignment, and the coarse liveness approximation are not specified precisely enough to verify the invariant or the termination of the scheduling algorithm. Please provide the algorithm (or a formal description of its state and the invariant it maintains) and an argument for why no deadlock or port conflict can arise.
  4. [§7.5, Fig. 12] The FPGA portability claim ('Dato surpasses leading frameworks in performance') is supported by a comparison against Allo only, and Allo is a framework developed by the same research group. The comparison is further complicated by the fact that Allo's design fails timing at 132 MHz while Dato meets 300 MHz. The '98% of theoretical peak' result is for a systolic array whose dimensions and clock are chosen by the authors; this is a legitimate case study, but it should be framed as such. An independent baseline or a second FPGA framework would substantially strengthen the portability claim. At minimum, the paper should state the Allo throughput at its achieved clock and justify the choice of baseline.
minor comments (4)
  1. [§3.3, Fig. 4] The text defines Stream[T, N, P] with a packing parameter P, but the typing rules in Fig. 4 use Stream[T, N] only. Please reconcile the notation.
  2. [§7.4, Fig. 11] The text reports ablation times in milliseconds (13.65 ms, 6.42 ms, 3.13 ms, 2.83 ms), but Fig. 11 labels the y-axis 'Latency (us)' and the bar labels are 13654.84, 6419.11, 3133.56, 2825.92. Please harmonize the units and the labels.
  3. [Fig. 8] In the mixed-precision panel, the legend shows 'i8xi4' twice (once for the lower curve and once for Tbest), which is confusing. Please use distinct labels such as 'i8xi4 (default)' and 'i8xi4 (best)'.
  4. [§1, footnote 1] The code repository link points to the Allo repository rather than a Dato-specific release. Please provide a stable, versioned artifact for Dato so that the claims can be independently checked.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the main claims are supported by measured benchmarks and by type-system definitions; only a minor same-group baseline choice in the FPGA evaluation prevents a score of 0.

full rationale

The paper's central claims are (1) a linear-token stream type system makes FIFO overflow/underflow untypeable by construction, (2) layout types and virtual-to-physical mapping automate sharding and placement, and (3) Dato matches or beats IRON/ARIES on AMD NPUs and Allo on an Alveo FPGA. None of these reduces to its own input in a circular way. The stream-type invariant is genuinely definitional: tokens are introduced as Free/Used counts, so the overflow/underflow property is a direct consequence of the typing rules rather than an empirical prediction. The paper honestly notes the model is untimed and cannot give minimal FIFO depths, which is a limitation but not circularity. The NPU performance numbers are measured against IRON, an external vendor flow, and ARIES; even though some ARIES authors overlap with the Dato authors, the comparison is a measured benchmark, not an equation that forces the result. The FPGA comparison is against Allo, a framework built by overlapping authors, and the abstract's 'surpasses leading frameworks' is broader than the single baseline supports; however, this is a baseline-selection concern, not a circular derivation, because the reported throughput and timing come from actual synthesis and measurement. The 'highest measured performance' selection in the mapping search is an autotuning procedure rather than a fitted prediction. No step in the paper exhibits the pattern of defining an output in terms of the claimed result, fitting a parameter and then renaming it a prediction, or importing a uniqueness theorem from the authors' own prior work. The type-safety proof is not fully specified for parallel task composition, but that is a verification/completeness gap, not a circularity. For these reasons, a score of 2 reflects the mild same-group baseline issue while confirming that the core derivations are self-contained and externally evaluated.

Assumptions & free parameters 2 free parameters · 4 assumptions · 2 invented entities

The paper's compile-time guarantees rest on these unformalized semantic constructs; the performance results do not, which is why the ledger is lighter than the soundness score might suggest.

free parameters (2)
  • Mixed-precision GEMM tile config 'Tbest' = not stated (labeled 'Tbest' in Fig. 8)
    The 84.38% utilization headline comes only from this best-tile selection; the default i8xi4 config achieves 64.22%, so the headline number depends on a searched/tuned tiling choice.
  • Systolic array dimensions and clock = 16x16, 300 MHz
    The 98%-of-peak FPGA result is for this user-chosen array shape and clock, not an automatically synthesized mapping from a generic GEMM description.
assumptions (4)
  • domain assumption FIFO capacity semantics: each Stream[T,N] behaves like a hardware FIFO of depth N, with N slots modeled by linear tokens Free/Used.
    The linear typing rules in Fig. 4 assume this hardware-equivalence; it is asserted, not proved, and is the basis for the deadlock/overflow safety claims.
  • domain assumption SPMD execution model: tensor axes labeled S are partitioned evenly across PEs, R axes are replicated, and .allreduce() can combine partials across shards.
    The layout typing rules (Fig. 6) and VMG construction assume this sharding model; no proof that generated physical schedules realize it is given.
  • ad hoc to paper The compiler's forward abstract interpretation over the CFG with token multisets converges to a sound fixpoint.
    Section 3.3 states type checking is via forward abstract interpretation but gives no join operator, widening, or termination argument for loops; the safety guarantee depends on this unstated analysis detail.
  • domain assumption Collocating nodes with .bundle()/.chain() preserves external program semantics.
    Section 4.2 asserts semantic preservation with legality conditions sketched; no formal equivalence proof is given, and the DMA token schedule relies on it.
invented entities (2)
  • Linear ready-future type LFuture<T>
    purpose: Represents a value dequeued from a stream but not yet awaited, enabling latency hiding without blocking.
    Introduced in Fig. 4 type rules; no external implementation or machine-checked semantics provided; it is an internal compiler abstraction.
  • Linear capability tokens Free(S)/Used(S)
    purpose: Track FIFO occupancy at compile time to make overflow/underflow untypeable.
    Internal to the type system sketch in §3.3; no formalization artifact exists.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dato: A Task-Based Programming Model for Dataflow Accelerators." pith.science (2026). https://pith.science/paper/3CFMSQEN

@misc{pith2026250906794,
  author       = {Pith},
  title        = {Pith review of: Dato: A Task-Based Programming Model for Dataflow Accelerators},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3CFMSQEN}},
  note         = {Machine review of arXiv:2509.06794}
}
read the original abstract

Recent deep learning workloads increasingly push computational demand beyond what current memory systems can sustain, with many kernels stalling on data movement rather than computation. While modern dataflow accelerators incorporate on-chip streaming to mitigate off-chip bandwidth limitations, existing programming models struggle to harness these capabilities effectively. Low-level interfaces provide fine-grained control but impose significant development overhead, whereas high-level tile-based languages abstract away communication details, restricting optimization and forcing compilers to reconstruct the intended dataflow. We present Dato, a Python-embedded, task-based programming model for dataflow accelerators that elevates data communication and sharding to first-class type constructs. Developers write programs as a graph of tasks connected via explicit stream types, with sharded inputs specified using layout types. These tasks are first mapped virtually onto the accelerator's spatial fabric, and the compiler then generates a physical mapping that respects hardware constraints. Experimental results on both AMD Ryzen AI NPU and Alveo FPGA devices demonstrate that Dato achieves high performance while significantly reducing the burden of writing optimized code. On the NPU, Dato attains up to 84% hardware utilization for GEMM and delivers a 2.81x speedup on attention kernels compared to a state-of-the-art commercial framework. On the FPGA, Dato surpasses leading frameworks in performance when generating custom systolic arrays, achieving 98% of the theoretical peak performance.

Figures

Figures reproduced from arXiv: 2509.06794 by the authors.

Figure 1
Figure 1. A typical dataflow accelerator architecture [46, 54]. The actual number of hardware units may be greater. Other accelerators use scalable meshes of AI cores with con￾figurable memory and communication [15, 24, 46]. NVIDIA’s Hopper [36] and Blackwell [38] architectures likewise add tensor memory accelerators (TMA) to stage and stream ten￾sors asynchronously, overlapping data movement with com￾putation for higher util… view at source ↗
Figure 2
Figure 2. Example Allo and Dato programs of a simple producer-consumer pattern. abstract syntax tree (AST) is then lowered through an MLIR builder to construct the intermediate representation (IR). With the IR constructed, Dato generates a virtual compu￾tation graph that captures the program’s dataflow structure. Because dataflow accelerators often feature a limited num￾ber of processing engines (PEs), while user programs may… view at source ↗
Figure 5
Figure 5. Example dataflow programs that fail type check. these slots with linear capability tokens Free(𝑆) and Used(𝑆): .put() is well-typed only when a Free(𝑆) token is available, converting it into Used(𝑆) (tail advances); symmetrically, .get() is well-typed only in the presence of Used(𝑆), which it immediately turns back into Free(𝑆) while returning the payload as a linear ready-future, a value available without blocking … view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Typing rules for stream type. Γ is the ordinary typing context, whereas Δ is a multiset representing the linear context that tracks resources that must be used exactly once. ⊎ is the multiset union. 𝑁 is the depth of the stream [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: Layout type definition and minimal typing rules. 𝐷 is the base element domain. Γ is the typing context. 𝜏 ! Π denotes pending-collective effects Π to a type 𝜏. 3.4 Layout Type As deep learning scales across distributed, heterogeneous systems, treating individual PEs or…
Figure 7
Figure 7. Figure 7: Virtual-to-physical mapping of a tiled GEMM kernel. edges, connecting all task instances that participate in the col￾lective as dictated by the layout. For example, when the reduc￾tion dimension is sharded along a virtual axis, .allreduce() aggregates the partial resul…
Figure 8
Figure 8. Figure 8: Performance results on the GEMM kernel with different data types. Shaded regions denote the execution variance. 128 256 512 1024 2048 0 2 4 6 1e5 × MHA Dato IRON 128 256 512 1024 2048 0 2 4 6 1e4 FFN Sequence Length Latency (us) [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Performance of MHA and FFN on NPU (bf16). ultimately achieving up to 84% hardware utilization by fully leveraging the lower bitwidth. 7.2.2 Multi-Kernel Design. We evaluate two multi-kernel designs representative of Transformer workloads. The first is multi-head attent…
Figure 10
Figure 10. Figure 10: Left: Implementation of the Flash Attention kernel in Dato. Right: Physical mapping of the attention kernel. 10 1 10 0 10 1 0.0 0.5 1.0 1.5 1e4 1 × 4 2 × 4 4 × 4 Compute Workload (GOPs) Latency (us) Baseline + Virtual Mapping + Kernel Injection + Layout Opt. 0 2 4 6 L…
Figure 11
Figure 11. Figure 11: Left: Scalability results on GEMM. Right: Opti￾mization ablation study w/ GEMM. The numbers on the bar denote NPU time. subsequent layout optimization trims it further to 2.83 ms (–10%). Across the sequence, data/instruction copy costs re￾main negligible, and the tota…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. STEEL: Sparsity-Aware Fused Attention for Energy-Efficient Long-Sequence Inference on AMD's XDNA NPU

    cs.DC 2026-07 conditional novelty 6.0 of 10

    STEEL maps fused FlashAttention onto XDNA NPUs with sparsity-aware pipeline placement, cutting energy ~9 imes vs CPU and ~1.75 imes vs GPU and beating prior XDNA attention by ~9.6× latency.

Reference graph

Works this paper leans on

67 extracted references · 50 canonical work pages · cited by 1 Pith paper

  1. [1]

    Rink, Michael Schaarschmidt, Timur Sitdikov, Agnieszka Swietlik, Dimitrios Vytiniotis, and Joel Wee

    Sami Alabed, Daniel Belov, Bart Chrzaszcz, Juliana Franco, Dominik Grewe, Dougal Maclaurin, James Molloy, Tom Natan, Tamara Norman, Xiaoyue Pan, Adam Paszke, Norman A. Rink, Michael Schaarschmidt, Timur Sitdikov, Agnieszka Swietlik, Dimitrios Vytiniotis, and Joel Wee

    work page

  2. [2]

    AWS. 2023. Inferentia Architecture. https://awsdocs- neuron.readthedocs-hosted.com/en/latest/general/arch/neuron- hardware/inferentia.html

    work page 2023

  3. [3]

    Suhail Basalama and Jason Cong. 2025. Stream-HLS: Towards Auto- matic Dataflow Acceleration. In Proceedings of the 2025 ACM/SIGDA International Symposium on Field Programmable Gate Arrays . ACM, New York, NY, USA, 103–114

    work page 2025

  4. [4]

    Cerebras. 2024. The Future of AI is Wafer Scale. https://www.cerebras. ai/chip

    work page 2024

  5. [5]

    Hongzheng Chen, Cody Hao Yu, Shuai Zheng, Zhen Zhang, Zhiru Zhang, and Yida Wang. 2024. Slapo: A Schedule Language for Pro- gressive Optimization of Large Deep Learning Model Training. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 (La Jolla, CA, USA) (ASPLOS’24). ...

    work page 2024

  6. [6]

    Hongzheng Chen, Jiahao Zhang, Yixiao Du, Shaojie Xiang, Zichao Yue, Niansong Zhang, Yaohui Cai, and Zhiru Zhang. 2024. Understanding the potential of fpga-based spatial acceleration for large language model inference. ACM Transactions on Reconfigurable Technology and Systems 18, 1 (2024), 1–29

    work page 2024

  7. [7]

    Hongzheng Chen, Niansong Zhang, Shaojie Xiang, Zhichen Zeng, Mengjia Dai, and Zhiru Zhang. 2024. Allo: A programming model for composable accelerator design. Proceedings of the ACM on Program- ming Languages 8, PLDI (2024), 593–620

    work page 2024

  8. [8]

    Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Meghan Cowan, Haichen Shen, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2018. TVM: An Automated End-to-End Optimizing Compiler for Deep Learning. In Proceedings of the 13th USENIX Conference on Operating Systems Design and Implementation (Carlsbad, CA, US...

    work page 2018

Show all 67 references
  1. [9]

    Joseph W Cutler, Christopher Watson, Emeka Nkurumeh, Phillip Hilliard, Harrison Goldstein, Caleb Stanford, and Benjamin C Pierce

  2. [10]

    Vidushi Dadu and Tony Nowatzki. 2022. TaskStream: accelerating task-parallel workloads by recovering program structure. In Proceed- ings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (Lausanne, Switzer- land) ...

    2022

  3. [11]

    Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré

  4. [12]

    Yaoyao Ding, Bohan Hou, Xiao Zhang, Allan Lin, Tianqi Chen, Cody Yu Hao, Yida Wang, and Gennady Pekhimenko. 2025. Tilus: A Tile-Level GPGPU Programming Language for Low-Precision Com- putation. arXiv:2504.12984 [cs.LG] https://arxiv.org/abs/2504.12984

    2025 arXiv

  5. [13]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Ka- dian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al . 2024. The Llama 3 Herd of Models. 12 Dato: A Task-Based Programming Model for Dataflow Accelerators arXiv:2407.21783 [c...

    2024 arXiv

  6. [14]

    David Durst, Matthew Feldman, Dillon Huff, David Akeley, Ross Daly, Gilbert Louis Bernstein, Marco Patrignani, Kayvon Fatahalian, and Pat Hanrahan. 2020. Type-Directed Scheduling of Streaming Accelerators. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Langua...

    2020

  7. [15]

    Papka, Rick Stevens, Laura Florescu, Sumti Jairath, William Liu, Tejas Nama, and Arvind Sujeeth

    Murali Emani, Venkatram Vishwanath, Corey Adams, Michael E. Papka, Rick Stevens, Laura Florescu, Sumti Jairath, William Liu, Tejas Nama, and Arvind Sujeeth. 2021. Accelerating Scientific Applications With SambaNova Reconfigurable Dataflow Architecture. Computing in Science & E...

    2021 doi

  8. [16]

    Xinwei Fu, Zhen Zhang, Haozheng Fan, Guangtai Huang, Mohammad El-Shabani, Randy Huang, Rahul Solanki, Fei Wu, Ron Diamant, and Yida Wang. 2024. Distributed Training of Large Language Models on AWS Trainium. In Proceedings of the 2024 ACM Symposium on Cloud Computing (Redmond, ...

    2024

  9. [17]

    Amir Gholami, Zhewei Yao, Sehoon Kim, Coleman Hooper, Michael W Mahoney, and Kurt Keutzer. 2024. Ai and memory wall. IEEE Micro 44, 3 (2024), 33–39

    2024

  10. [18]

    Souradip Ghosh, Yufei Shi, Brandon Lucia, and Nathan Beckmann

  11. [19]

    Google. 2025. Ironwood: The first Google TPU for the age of infer- ence. https://blog.google/products/google-cloud/ironwood-tpu-age- of-inference/

    2025

  12. [20]

    Licheng Guo, Yuze Chi, Jason Lau, Linghao Song, Xingyu Tian, Moazin Khatti, Weikang Qiao, Jie Wang, Ecenur Ustun, Zhenman Fang, Zhiru Zhang, and Jason Cong. 2023. TAPA: A Scalable Task-Parallel Dataflow Programming Framework for Modern FPGAs with Co-Optimization of HLS and Phy...

    2023 doi

  13. [21]

    Bastian Hagedorn, Bin Fan, Hanfeng Chen, Cris Cecka, Michael Gar- land, and Vinod Grover. 2023. Graphene: An ir for optimized tensor computations on gpus. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating S...

    2023

  14. [22]

    Ripple: Asynchronous Programming for Spatial Dataflow Archi- tectures. Proc. ACM Program. Lang. 9, PLDI, Article 157 (June 2025), 28 pages. doi:10.1145/3729256

    2025 doi

  15. [23]

    Erika Hunhoff, Joseph Melber, Kristof Denolf, Andra Bisca, Samuel Bayliss, Stephen Neuendorffer, Jeff Fifield, Jack Lo, Pranathi Vasireddy, Phil James-Roxby, et al. 2025. Efficiency, Expressivity, and Extensi- bility in a Close-to-Metal NPU Programming Interface. In 2025 IEEE ...

    2025

  16. [24]

    IBM. 2024. Enhancing enterprise AI with the IBM Spyre Accelerator. https://research.ibm.com/blog/spyre-for-z

    2024

  17. [25]

    IBM. 2025. Why a decades old architecture decision is impeding the power of AI computing. https://research.ibm.com/blog/why-von- neumann-architecture-is-impeding-the-power-of-ai-computing

    2025

  18. [26]

    Tsung-Wei Huang, Dian-Lun Lin, Chun-Xun Lin, and Yibo Lin. 2022. Taskflow: A Lightweight Parallel and Heterogeneous Task Graph Computing System. IEEE Transactions on Parallel and Distributed Systems 33, 6 (2022), 1303–1320. doi:10.1109/TPDS.2021.3104255

    2022

  19. [27]

    Yuka Ikarashi, Kevin Qian, Samir Droubi, Alex Reinking, Gilbert Louis Bernstein, and Jonathan Ragan-Kelley. 2025. Exo 2: Growing a Schedul- ing Language. In Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Sy...

    2025

  20. [28]

    Norm Jouppi, George Kurian, Sheng Li, Peter Ma, Rahul Nagarajan, et al. 2023. TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings. In Proceedings of the 50th Annual International Symposium on Computer Architecture (Orland...

    2023

  21. [29]

    Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, et al

    Norman P. Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, et al . 2017. In-Datacenter Performance Analysis of a Tensor Processing Unit. In Proceedings of the 44th An- nual International Symposium on Computer Architecture (Toronto, ON, Canad...

    2017

  22. [30]

    Yuka Ikarashi, Gilbert Louis Bernstein, Alex Reinking, Hasan Genc, and Jonathan Ragan-Kelley. 2022. Exocompilation for Productive Programming of Hardware Accelerators. In Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementa...

    2022

  23. [31]

    Shadaj Laddad, Alvin Cheung, Joseph M Hellerstein, and Mae Milano

  24. [32]

    Yi-Hsiang Lai, Yuze Chi, Yuwei Hu, Jie Wang, Cody Hao Yu, Yuan Zhou, Jason Cong, and Zhiru Zhang. 2019. HeteroCL: A Multi-Paradigm Pro- gramming Infrastructure for Software-Defined Reconfigurable Com- puting. In Proceedings of the 2019 ACM/SIGDA International Symposium on Fiel...

    2019

  25. [33]

    Chris Lattner, Mehdi Amini, Uday Bondhugula, Albert Cohen, Andy Davis, Jacques Pienaar, River Riddle, Tatiana Shpeisman, Nicolas Vasi- lache, and Oleksandr Zinenko. 2021. MLIR: Scaling Compiler Infras- tructure for Domain Specific Computation. In Proceedings of the 2021 IEEE/A...

    2021

  26. [34]

    Gary A. Kildall. 1973. A Unified Approach to Global Program Op- timization. In Proceedings of the 1st Annual ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages (Boston, Mas- sachusetts) (POPL’73). Association for Computing Machinery, New York, NY, USA, 194–206...

    1973

  27. [35]

    NVIDIA. 2017. CUTLASS. https://github.com/NVIDIA/cutlass

    2017

  28. [36]

    Proceedings of the ACM on Programming Languages 9, POPL (2025), 241–270

    Flo: A Semantic Foundation for Progressive Stream Processing. Proceedings of the ACM on Programming Languages 9, POPL (2025), 241–270

    2025

  29. [37]

    NVIDIA. 2025. CUTLASS Documentation: CuTe Layouts. https: //docs.nvidia.com/cutlass/media/docs/cpp/cute/01_layout.html

    2025

  30. [38]

    NVIDIA. 2025. NVIDIA Blackwell Architecture Technical Brief. https: //resources.nvidia.com/en-us-blackwell-architecture

    2025

  31. [39]

    Zhaoying Li, Pranav Dangi, Chenyang Yin, Thilini Kaushalya Ban- dara, Rohan Juneja, Cheng Tan, Zhenyu Bai, and Tulika Mitra. 2025. Enhancing CGRA Efficiency Through Aligned Compute and Commu- nication Provisioning. In Proceedings of the 30th ACM International Conference on Arc...

    2025

  32. [40]

    Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean. 2023. Efficiently scaling transformer inference. Proceedings of machine learning and systems 5 (2023), 606–624

    2023

  33. [41]

    NVIDIA. 2022. NVIDIA Hopper Architecture. https://www.nvidia. com/en-us/data-center/technologies/hopper-architecture/

    2022

  34. [42]

    Alejandro Rico, Satyaprakash Pareek, Javier Cabezas, David Clarke, Baris Ozgul, Francisco Barat, Yao Fu, Stephan Münz, Dylan Stuart, Patrick Schlangen, Pedro Duarte, Sneha Date, Indrani Paul, Jian Weng, Sonal Santan, Vinod Kathail, Ashish Sirasao, and Juanjo Noguera. 2024. AMD...

    2024

  35. [43]

    Nick Rioux and Steve Zdancewic. 2025. Functional Meaning for Par- allel Streaming. Proc. ACM Program. Lang. 9, PLDI, Article 196 (June 2025), 25 pages. doi:10.1145/3729299

    2025 doi

  36. [44]

    Debjit Pal, Yi-Hsiang Lai, Shaojie Xiang, Niansong Zhang, Hongzheng Chen, Jeremy Casas, Pasquale Cocchini, Zhenkun Yang, Jin Yang, Louis-Noël Pouchet, and Zhiru Zhang. 2022. Accelerator Design with Decoupled Hardware Customizations: Benefits and Challenges: In- vited. In Proce...

    2022 doi

  37. [45]

    Amit Sabne. 2020. XLA : Compiling Machine Learning for Peak Per- formance

    2020

  38. [46]

    Jonathan Ragan-Kelley, Connelly Barnes, Andrew Adams, Sylvain Paris, Frédo Durand, and Saman Amarasinghe. 2013. Halide: A Lan- guage and Compiler for Optimizing Parallelism, Locality, and Recom- putation in Image Processing Pipelines. SIGPLAN Not. 48, 6 (jun 2013), 519–530. do...

    2013

  39. [47]

    James Thomas, Pat Hanrahan, and Matei Zaharia. 2020. Fleet: A Frame- work for Massively Parallel Streaming on FPGAs. InProceedings of the Twenty-Fifth International Conference on Architectural Support for Pro- gramming Languages and Operating Systems (Lausanne, Switzerland) (A...

    2020

  40. [48]

    Philippe Tillet, H. T. Kung, and David Cox. 2019. Triton: An Intermedi- ate Language and Compiler for Tiled Neural Network Computations. In Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages . ACM, New York, NY, USA, 10–19

    2019

  41. [49]

    Rucker, Shiv Sundram, Coleman Smith, Matthew Vilim, Raghu Prabhakar, Fredrik Kjølstad, and Kunle Olukotun

    Alexander C. Rucker, Shiv Sundram, Coleman Smith, Matthew Vilim, Raghu Prabhakar, Fredrik Kjølstad, and Kunle Olukotun. 2024. Revet: A Language and Compiler for Dataflow Threads. In 2024 IEEE Interna- tional Symposium on High-Performance Computer Architecture (HPCA) . IEEE Com...

    2024

  42. [50]

    Lei Wang, Yu Cheng, Yining Shi, Zhengju Tang, Zhiwen Mo, Wenhao Xie, Lingxiao Ma, Yuqing Xia, Jilong Xue, Fan Yang, and Zhi Yang. 2025. TileLang: A Composable Tiled Programming Model for AI Systems. arXiv:2504.17577 [cs.LG] https://arxiv.org/abs/2504.17577

    2025 arXiv

  43. [51]

    Tenstorrent. 2024. Blackhole. https://tenstorrent.com/en/hardware/ blackhole

    2024

  44. [52]

    Ningning Xie, Tamara Norman, Dominik Grewe, and Dimitrios Vytin- iotis. 2022. Synthesizing optimal parallelism placement and reduction strategies on hierarchical systems for deep learning. Proceedings of Machine Learning and Systems 4 (2022), 548–566

    2022

  45. [53]

    Xilinx. 2025. MLIR-AIE Dialect. https://github.com/Xilinx/mlir-aie

    2025

  46. [54]

    Gomez, Łukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, ...

    2017

  47. [55]

    AMD Xilinx. 2022. Vitis Accelerated Libraries. https://github.com/ Xilinx/Vitis_Libraries

    2022

  48. [56]

    Shaojie Xiang, Yi-Hsiang Lai, Yuan Zhou, Hongzheng Chen, Niansong Zhang, Debjit Pal, and Zhiru Zhang. 2022. HeteroFlow: An Accelerator Programming Model with Decoupled Data Placement for Software- Defined FPGAs. In Proceedings of the 2022 ACM/SIGDA International Symposium on F...

    2022

  49. [57]

    AMD Xilinx. 2023. Vitis HLS v2023.2. https://www.xilinx.com/ products/design-tools/vitis/vitis-platform.html

    2023

  50. [58]

    Yuanzhong Xu, HyoukJoong Lee, Dehao Chen, Blake Hechtman, Yanping Huang, Rahul Joshi, Maxim Krikun, Dmitry Lepikhin, Andy Ly, Marcello Maggioni, Ruoming Pang, Noam Shazeer, Shibo Wang, Tao Wang, Yonghui Wu, and Zhifeng Chen. 2021. GSPMD: General and Scalable Parallelization fo...

    2021 arXiv

  51. [59]

    AMD Xilinx. 2022. AI Engines and Their Applications. https://www.xilinx.com/content/dam/xilinx/support/documents/ white_papers/wp506-ai-engine.pdf

    2022

  52. [60]

    Xing, Joseph E

    Lianmin Zheng, Zhuohan Li, Hao Zhang, Yonghao Zhuang, Zhifeng Chen, Yanping Huang, Yida Wang, Yuanzhong Xu, Danyang Zhuo, Eric P. Xing, Joseph E. Gonzalez, and Ion Stoica. 2022. Alpa: Au- tomating Inter- and Intra-Operator Parallelism for Distributed Deep Learning. In 16th USE...

    2022

  53. [61]

    AMD Xilinx. 2022. Vitis AI: Adaptable & Real-Time AI Inference Acceleration. https://github.com/Xilinx/Vitis-AI

    2022

  54. [62]

    Jinming Zhuang, Shaojie Xiang, Hongzheng Chen, Niansong Zhang, Zhuoping Yang, Tony Mao, Zhiru Zhang, and Peipei Zhou. 2025. ARIES: An Agile MLIR-Based Compilation Flow for Reconfigurable Devices with AI Engines. In Proceedings of the 2025 ACM/SIGDA Inter- national Symposium on...

    2025

  55. [64]

    Hanchen Ye, Hyegang Jun, and Deming Chen. 2024. Hida: A hierar- chical dataflow compiler for high-level synthesis. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1 . ACM, New York, NY, US...

    2024

  56. [67]

    Keren Zhou, Mario Lezcano, Adam Goucher, Akhmed Rakhmati, Jeff Niu, Justin Lebar, Pawel Szczerbuk, Peter Bell, Phil Tillet, Thomas Raoux, et al. 2025. Linear Layouts: Robust Code Generation of Efficient Tensor Computation Using F2. arXiv:2505.23819 [cs.PL] https://arxiv. org/a...

    2025

  57. [578]

    https://www.usenix.org/conference/osdi22/presentation/zheng- lianmin

  58. [2022]

    Advances in neural information processing systems 35 (2022), 16344–16359

    Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in neural information processing systems 35 (2022), 16344–16359

    2022

  59. [2024]

    Proceedings of the ACM on Programming Languages 8, PLDI (2024), 1412–1436

    Stream types. Proceedings of the ACM on Programming Languages 8, PLDI (2024), 1412–1436

    2024

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.