Seamless acceleration of Fortran intrinsics via AMD AI engines

Gabriel Rodr\'iguez Canal; Nick Brown

REVIEW 3 major objections 5 minor 18 references

Seamless acceleration of Fortran intrinsics via AMD AI engines

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a compiler can automatically offload standard Fortran intrinsics—sum, transpose, and matmul—to the AI engine NPU inside AMD's Ryzen AI CPU, letting existing Fortran code get accelerator speedups without source…

desk verdict The compiler integration is real and the reduction/transpose measurements look honest, but the headline int32/float32 matmul speedups silently change the computation to int16/bfloat16 without any numerical validation, so the seamless-acceleration claim is not established for standard types. read the letter →

arxiv 2502.10254 v1 pith:YXUPK43T submitted 2025-02-14 cs.DC cs.ETcs.PF

Nick Brown , Gabriel Rodríguez Canal This is my paper · ORCID

classification cs.DCcs.ETcs.PF

keywords FortranintrinsicsAIEnginesNPUoffloadingMLIRFlanglinalgdialectxDSLAMDRyzen

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish that a compiler can automatically offload standard Fortran intrinsics to the AI engine array inside AMD's Ryzen AI CPU, so that scientific code gains accelerator performance without any source changes. It builds a compilation flow that recognises intrinsics after Flang lowers Fortran to MLIR, then specialises pre-built AIE kernels for each call and drives the NPU through a runtime wrapper. The measured results show that simple reductions only become competitive after the first-call setup cost is amortised, transpose wins at larger sizes that fit the NPU's memory tile, and matmul wins over the CPU for every data type tested. This matters because Fortran remains dominant in scientific computing, and a transparent compiler path would let existing codes inherit specialised-hardware speedups instead of requiring programmers to learn a new API.

What carries the argument

The mechanism is a compiler pipeline with three parts. First, a recognition pass maps linalg operations to specific Fortran intrinsics; for reductions this means parsing the arithmetic body of the operation. Second, a template library stores AIE-side IR fragments—built with AMD's MLIR tooling and represented in xDSL—that are loaded and specialised per call, replacing placeholders for types and sizes, and optionally calling external C++ kernels compiled with the chess compiler. Third, a new xrt_wrapper MLIR dialect lowers to a runtime wrapper around AMD's XRT so the CPU can find the NPU, allocate and map buffers, launch kernels, and wait; the AIE-side IR uses FIFO-linked tiles, 512-bit vector operations, and for transpose the memory tile's data movers rather than the vector units. The whole flow keeps the CPU implementation as a fallback and runtime-checks whether an NPU is present.

What would settle it

Compile a Fortran program that calls matmul on randomly filled int32 matrices, run it on both the CPU and the NPU-offloaded path, and compare every element of the result against a high-precision reference; any mismatch beyond the unavoidable rounding of the true product would show that the reduced-precision path is not the same computation.

Watch

Extended reading notes

Core claim

The central claim is that the compiler can be the bridge between unmodified Fortran source and AMD's AI engines. After Flang lowers a program to standard MLIR, the paper's transformation pass classifies each intrinsic call—matmul appears directly as linalg.matmul, while sum, product, maxval, and minval are recognised by inspecting the body of linalg.reduce. The compiler then loads a matching AIE kernel template from a library, specialises it for the data type and problem size, generates CPU-side IR that calls a custom XRT wrapper, and falls back to the CPU when the call is not suitable for the NPU. The authors report that this transparent path works for reductions, transpose, and matmul, and that matmul gives the NPU a clear advantage across all tested types, with int32 and float32 runs marked as using reduced-precision inputs and promoting only the output.

Load-bearing premise

The load-bearing premise is that computing matmul with int16 or bfloat16 inputs and promoting only the output to int32 or float32 counts as performing the same computation as the Fortran matmul intrinsic; if reduced precision changes the numerical result, the NPU's advantage for those data types is not a speedup for the same workload.

Editorial extensions

If this is right

If the approach is correct, recompiling an existing Fortran program is enough to move supported intrinsic calls onto the NPU; no library changes, annotations, or API calls are needed.
Repeated intrinsic calls amortise the NPU setup cost, so long-running scientific loops are the natural target; the paper reports subsequent-run times far below first-run times for reductions.
The matmul results imply that for compute-heavy linear algebra, the NPU can beat a CPU despite its transfer overhead, at least for the tested 256x256x512 problem size.
The memory-tile transpose result implies that data-movement-only operations on the NPU are useful only within the 512KB tile capacity, with CPU parity or better at the largest fitting sizes.
Keeping the CPU path as a fallback means unsupported data types or sizes remain correct by construction, which is essential for a transparent offload feature.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

This reader's inference: the paper's int32 and float32 matmul comparisons only count as like-for-like speedups if users accept that computing in int16 or bfloat16 and promoting the output is numerically equivalent; a documented error analysis would settle that.
A natural extension the author leaves implicit is an automatic size- and type-dependent dispatch that uses the CPU fallback for small transposes or single reductions, since first-run NPU overhead is large.
Because the ONNX dialect also lowers through linalg, the same template-library mechanism could plausibly accelerate neural-network operators on the NPU; the paper names this only as future work.
The template-library approach suggests that adding a new offloadable intrinsic is mostly a matter of contributing a new AIE kernel template, which would make the performance of the NPU depend on the breadth of that library.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The useful part of this paper is the compiler plumbing. Building on Brown's earlier Flang-to-standard-MLIR flow, the authors add an xrt_wrapper dialect, a template library of AIE kernels, and a lowering from linalg operations to AMD's AIE dialects. That integration is new, and it is a legitimate engineering contribution. The reductions and the transpose results are also measured in a way that inspires some confidence: they distinguish first-run from subsequent-run costs, break out transfer versus compute time, and use the memory tile for transpose, which is a sensible reuse of AMD's example. The paper is honest that reductions are mostly not faster on the NPU and that setup overhead matters. The soft spot is exactly where the stress-test note lands, and it is load-bearing. Table 4 reports int32 and float32 matmul speedups, but the asterisk says the NPU computes with int16 and bfloat16 inputs and only promotes the output. That is not the same Fortran intrinsic. int16 truncates any element outside its range; bfloat16 has eight mantissa bits, so even well-scaled float32 inputs acquire roughly 0.4% relative rounding per element. The paper gives no error analysis and no comparison of NPU outputs against CPU outputs. Those two rows should be labeled as lower-precision kernels, not as speedups for the user's matmul. The bfloat16 CPU baseline of 815 milliseconds also looks like software emulation, so the supposedly enormous speedup there is against a strawman. The CPU baseline being Flang's linalg lowering rather than an optimized BLAS is acceptable for a compiler-to-compiler comparison, but it should be stated more carefully. The central conclusion, that Fortran intrinsics can be seamlessly offloaded with significant advantages, is overstated. What is actually demonstrated is that a small set of intrinsics can be offloaded, and that for reductions the NPU is competitive only on repeated calls, for transpose it helps only near the memory tile limit, and for matmul it helps only when you quietly change the precision. That is still worth publishing, but only after the precision issue is confronted directly. Who is this for? Compiler engineers working on Flang/MLIR and on AIE programming tools. A serious referee should see it, but the revision must include numerical validation of the reduced-precision matmul, clearer labeling of what was actually computed, and a fairer CPU baseline for bfloat16. I would not cite it in its current form, but I would read a revised version.

Referee Report

3 major / 5 minor

Summary. The paper describes a compiler-based approach, built on Flang/MLIR and xDSL, that recognizes selected Fortran intrinsic calls (sum, product, maxval, minval, transpose, and matmul) in linalg form and offloads them to the AIE-ML array of an AMD Ryzen AI NPU. The CPU-side driver uses a new xrt_wrapper MLIR dialect, while the AIE-side kernels are drawn from a template library and specialized at compile time. The evaluation compares CPU and NPU runtimes for reductions, transpose, and matmul. The paper reports that reductions are competitive only when first-invocation setup is excluded, that transpose benefits appear only for the largest arrays fitting in the memory tile, and that matmul shows large speedups across all reported data types. However, the matmul speedups for int32 and float32 are obtained by computing with int16 and bfloat16 inputs, respectively, and no numerical error analysis is provided, so the central claim of seamless acceleration for standard Fortran data types is not yet established.

Significance. If the semantic-equivalence issue is resolved, this would be a useful contribution to compiler-assisted use of NPU hardware from Fortran: it shows a plausible path from standard MLIR dialects to AMD AIE dialects, it integrates with an open-source compiler stack, and it provides detailed runtime breakdowns including transfer and compute components. The paper is also transparent about the fact that the int32/float32 matmul rows use reduced-precision inputs. That transparency is valuable, but the current measurements do not support the abstract's unqualified claim that suitable Fortran intrinsics can be accelerated 'without any code modifications' while still computing the standard intrinsic result for int32 and float32 data.

major comments (3)

[§4, Table 4] The int32 and float32 matmul rows do not evaluate the same computation as the Fortran matmul intrinsic for those types. The table caption and §2 state that the AIE-ML has no native int32/float32 vector support and that the asterisk rows use int16 and bfloat16 as inputs while producing int32/float32 outputs. This changes the function being computed: int16 inputs cannot represent values outside [-32768,32767] and products can overflow, while bfloat16 has an 8-bit mantissa and introduces per-element rounding far larger than float32. The paper reports no output validation, no maximum or mean error, and no statement about the range of test inputs. Consequently, the row pairs (14032→1503 us for int32, 17566→1471 us for float32) conflate precision reduction with hardware speedup and do not support the claim of seamless acceleration of the standard matmul intrinsic. Please add numerical accuracy comparisons against the CPU reference for representative matrices, including values near the representable limits, or explicitly restrict the claimed speedups to a reduced-precision mode that the Fortran programmer must opt into.
[§4, Tables 1–2] The reduction results for float32 have the same semantic issue as matmul. The text states that 'bfloat16 is not defined by the Fortran standard, so we developed a transformation to replace float32 in the IR with MLIR's bfloat16 type.' Therefore the float32 rows for sum, product, maxval, and minval do not compute the Fortran intrinsic over float32 operands unless the numerical differences are shown to be negligible. The conv-bfloat16 variant is a reasonable alternative, but it still performs bfloat16 arithmetic on the NPU and the paper gives no accuracy measurements for it. Please report the numerical error of the NPU results relative to a standard float32 CPU computation, or narrow the claim to the explicitly converted data type.
[§4, Tables 2–4] The comparison methodology for NPU 'subsequent' timings versus CPU timings is not fully specified. The paper says all results are averaged over ten runs and that the NPU subsequent timings ignore the initial invocation, but it does not state whether the CPU timings are steady-state or include the first invocation. If the CPU averages include a first-call effect while the NPU rows exclude it, the speedups are not apples-to-apples. Please report whether the CPU timings also exclude warm-up or, alternatively, give both first and subsequent CPU timings, so the steady-state comparison is clearly defined.

minor comments (5)

[Abstract and §1] There are several typographical errors: 'incesent' should be 'incessant', 'To the best our our knowledge' should be 'To the best of our knowledge', and 'expret' should be 'expert'.
[§2.1] The phrase 'the enable the development' should read 'that enable the development'.
[Listing 2] 'varadic' should be 'variadic'.
[Table 4 caption] The caption should explicitly state that the CPU rows for int32 and float32 use the standard full-precision matmul, so it is unambiguous that the NPU rows for those types are not computing the identical operation.
[§4, Table 1] The bfloat16 CPU runtime (5187 us for sum) is much slower than float32 (962 us) because the CPU is emulating bfloat16 in software; the paper notes this, but a sentence explaining that this comparison is not representative of a Fortran-standard data type would help readers interpret the row.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper reports measured engineering results validated against a CPU baseline; self-citations are supporting infrastructure, not load-bearing definitions.

full rationale

The paper does not fit a model and then predict the same data; it measures runtimes of a concrete compiler prototype on CPU and NPU. The compilation flow builds on the authors' prior Flang-MLIR integration [5] and on AMD's published AIE examples [1], but these are engineering dependencies with independently checkable artifacts, not mathematical premises that already contain the target conclusion. The NPU-vs-CPU comparisons are externally obtained measurements, and the CPU baseline is also generated from Flang, so the central claim is not equivalent to an input by construction. The reduced-precision int32/float32 matmul rows are a potential benchmark-validity concern: the Fortran matmul intrinsic is defined over int32/float32, while the NPU rows use int16/bfloat16 inputs with only the output promoted. That is a semantic-equivalence question, not a circularity one, because the measured speedup is not derived from the claim it purportedly supports. Self-citations are present but are not used to forbid alternatives, to import a uniqueness theorem, or to rename a known result as a prediction. No load-bearing step reduces, by construction or by definition, to its own inputs.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No fitted parameters are present; the work is an engineering benchmark. Two load-bearing assumptions about numerical equivalence are made without supporting error analysis, and the third is a dependency on a prior self-authored compilation flow.

assumptions (3)

domain assumption The Flang to linalg mapping of Fortran intrinsics from Brown 2024 is a faithful representation of the intrinsic semantics.
The entire offload pipeline starts from linalg IR produced by the prior Flang integration [5]; if this mapping is incorrect, the offloaded kernels inherit those errors.
ad hoc to paper Replacing float32 with bfloat16 in the IR for sum, product, maxval, and minval preserves the result closely enough for the comparison to be meaningful.
Section 4 states bfloat16 is not defined by the Fortran standard and a transformation replaces float32 with bfloat16; no numerical error analysis is provided.
ad hoc to paper Using int16 or bfloat16 inputs in the matmul kernel while producing int32 or float32 outputs computes the same Fortran matmul intrinsic result.
Table 4's asterisk reveals this precision mismatch, yet the paper reports NPU speedups for int32 and float32 without validating outputs against the CPU result.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Seamless acceleration of Fortran intrinsics via AMD AI engines." pith.science (2026). https://pith.science/paper/YXUPK43T

@misc{pith2026250210254,
  author       = {Pith},
  title        = {Pith review of: Seamless acceleration of Fortran intrinsics via AMD AI engines},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YXUPK43T}},
  note         = {Machine review of arXiv:2502.10254}
}

read the original abstract

A major challenge that the HPC community faces is how to continue delivering the performance demanded by scientific programmers, whilst meeting an increased emphasis on sustainable operations. Specialised architectures, such as FPGAs and AMD's AI Engines (AIEs), have been demonstrated to provide significant energy efficiency advantages, however a major challenge is that to most effectively program these architectures requires significant expertise and investment of time which is a major blocker. Fortran in the lingua franca of scientific computing, and in this paper we explore automatically accelerating Fortran intrinsics via the AIEs in AMD's Ryzen AI CPU. Leveraging the open source Flang compiler and MLIR ecosystem, we describe an approach that lowers the MLIR linear algebra dialect to AMD's AIE dialects, and demonstrate that for suitable workloads the AIEs can provide significant performance advantages over the CPU without any code modifications required by the programmer.

Figures

Figures reproduced from arXiv: 2502.10254 by the authors.

**Figure 2.** Illustration of our overarching compiler approach that offloads selected linear algebra operations to the AIE array [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 17 canonical work pages

[1]

MLIR-based AI Engine toolchain

AMD Xilinx 2024. MLIR-based AI Engine toolchain . Retrieved Oct 8, 2024 from https://github.com/Xilinx/mlir-aie 6

work page 2024
[2]

Versal Adaptive SoC AIE-ML Architecture Manual

AMD Xilinx 2024. Versal Adaptive SoC AIE-ML Architecture Manual. Retrieved Oct 8, 2024 from https://docs.amd.com/r/en-US/am020-versal-aie-ml/Overview

work page 2024
[3]

George Bisbas et al. 2024. A shared compilation stack for distributed-memory parallelism in stencil DSLs. In Proceedings of the 29th ACM International Confer- ence on Architectural Support for Programming Languages and Operating Systems, Volume 3. 38–56

work page 2024
[4]

Nick Brown. 2023. Exploring the Versal AI engines for accelerating stencil- based atmospheric advection simulation. In Proceedings of the 2023 ACM/SIGDA International Symposium on Field Programmable Gate Arrays . 91–97

work page 2023
[5]

Nick Brown. 2024. Fully integrating the Flang Fortran compiler with standard MLIR. arXiv preprint arXiv:2409.18824 (2024)

work page Pith review arXiv 2024
[6]

Mathieu Fehr et al. 2022. IRDL: an IR definition language for SSA compilers. In Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation . 199–212

work page 2022
[7]

Design: Fortran IR

FIR 2024. Design: Fortran IR. Retrieved Aug 16, 2024 from https://flang.llvm.org/ docs/FIRLangRef.html

work page 2024
[8]

Flang Documentation

Flang 2024. Flang Documentation. Retrieved Aug 16, 2024 from https://flang. llvm.org/docs/

work page 2024

Show all 18 references

[9]

Carsten Heinz, Jaco Hofmann, Jens Korinth, Lukas Sommer, Lukas Weber, and Andreas Koch. 2021. The TaPaSCo Open-Source Toolflow: for the Automated Composition of Task-Based Parallel Reconfigurable Computing Systems. Journal of Signal Processing Systems 93 (2021), 545–563

2021
[10]

Carsten Heinz, Torben Kalkhof, Yannick Lavan, and Andreas Koch. 2024. TaPaS Co-AIE: An Open-Source Framework for Streaming-Based Heterogeneous Accel- eration Using AMD AI Engines. In2024 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). IEEE, 155–161

2024
[11]

High-Level Fortran IR (HLFIR)

HLFIR 2024. High-Level Fortran IR (HLFIR) . Retrieved Aug 16, 2024 from https: //flang.llvm.org/docs/HighLevelFIR.html

2024
[12]

Torben Kalkhof, Carsten Heinz, and Andreas Koch. 2024. Enabling FPGA and AI Engine Tasks in the HPX Programming Framework for Heterogeneous High- Performance Computing. In International Symposium on Applied Reconfigurable Computing. Springer, 75–89

2024
[13]

Mark Klaisoongnoen, Nick Brown, Tim Dykes, Jessica R Jones, and Utz-Uwe Haus. 2024. Evaluating Versal AI Engines for option price discovery in market risk analysis. In Proceedings of the 2024 ACM/SIGDA International Symposium on Field Programmable Gate Arrays. 176–182

2024
[14]

Chris Lattner and Vikram Adve. 2004. LLVM: A compilation framework for lifelong program analysis & transformation. In International symposium on code generation and optimization, 2004. CGO 2004. IEEE, 75–86

2004
[15]

Maksim Levental, Arham Khan, Ryan Chard, Kyle Chard, Stephen Neuendorffer, and Ian Foster. 2024. An End-to-End Programming Model for AI Engine Archi- tectures. In Proceedings of the 14th International Symposium on Highly Efficient Accelerators and Reconfigurable Technologies. 135–136

2024
[16]

Rodriguez-Canal et al. 2023. Fortran High-Level Synthesis: Reducing the barriers to accelerating HPC codes on FPGAs. In 2023 33rd International Conference on Field-Programmable Logic and Applications (FPL) . IEEE, 10–18

2023
[17]

A Python Compiler Design Toolkit

xDSL 2023. A Python Compiler Design Toolkit . Retrieved Aug 16, 2023 from https://github.com/xdslproject/xdsl

2023
[18]

Wenbo Zhang, Tianshuo Wang, Yiqi Liu, Yiming Li, and Zhenshan Bao. 2023. New Filter2D Accelerator on the Versal Platform Powered by the AI Engine. In International Symposium on Advanced Parallel Processing Technologies . Springer, 437–449. 7

2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

MLIR-based AI Engine toolchain

AMD Xilinx 2024. MLIR-based AI Engine toolchain . Retrieved Oct 8, 2024 from https://github.com/Xilinx/mlir-aie 6

work page 2024

[2] [2]

Versal Adaptive SoC AIE-ML Architecture Manual

AMD Xilinx 2024. Versal Adaptive SoC AIE-ML Architecture Manual. Retrieved Oct 8, 2024 from https://docs.amd.com/r/en-US/am020-versal-aie-ml/Overview

work page 2024

[3] [3]

George Bisbas et al. 2024. A shared compilation stack for distributed-memory parallelism in stencil DSLs. In Proceedings of the 29th ACM International Confer- ence on Architectural Support for Programming Languages and Operating Systems, Volume 3. 38–56

work page 2024

[4] [4]

Nick Brown. 2023. Exploring the Versal AI engines for accelerating stencil- based atmospheric advection simulation. In Proceedings of the 2023 ACM/SIGDA International Symposium on Field Programmable Gate Arrays . 91–97

work page 2023

[5] [5]

Nick Brown. 2024. Fully integrating the Flang Fortran compiler with standard MLIR. arXiv preprint arXiv:2409.18824 (2024)

work page Pith review arXiv 2024

[6] [6]

Mathieu Fehr et al. 2022. IRDL: an IR definition language for SSA compilers. In Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation . 199–212

work page 2022

[7] [7]

Design: Fortran IR

FIR 2024. Design: Fortran IR. Retrieved Aug 16, 2024 from https://flang.llvm.org/ docs/FIRLangRef.html

work page 2024

[8] [8]

Flang Documentation

Flang 2024. Flang Documentation. Retrieved Aug 16, 2024 from https://flang. llvm.org/docs/

work page 2024