Pith. sign in

REVIEW 3 major objections 5 minor 14 references

MXDOTP: A RISC-V ISA Extension for Enabling Microscaling (MX) Floating-Point Dot Products

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A RISC-V instruction that fuses MXFP8 dot products into a single operation delivers 25x speedup and 12.5x energy gains over a software MX baseline.

desk verdict Solid engineering paper on a RISC-V MXFP8 dot-product instruction; the 25x/12.5x numbers are credible, but the unproven exact-accumulation range needs a fix before the correctness claim is taken at face value. read the letter →

arxiv 2505.13159 v1 pith:CJMH6NWA submitted 2025-05-19 cs.AR

Gamze İslamoğlu , Luca Bertaccini , Arpan Suravi Prasad , Francesco Conti , Angelo Garofalo , Luca Benini This is my paper · ORCID
classification cs.AR
keywords MXFP8microscalingformatsRISC-VISAextensiondotproduct-accumulateSnitchcorestreamsemanticregisterslow-bitwidtharithmeticenergy-efficientAIinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Microscaling (MX) formats pack low-bitwidth numbers into blocks that share an exponent scale, saving memory while keeping precision, but their dot products require extra scaling and format-conversion work in software. This paper proposes a single RISC-V instruction, mxdotp, that fuses the scaled dot product of eight MXFP8 element pairs into one fused multiply-accumulate-style operation, eliminating that overhead. The instruction is implemented as a small extension to the Snitch RISC-V core, with block scales streamed in through Stream Semantic Registers so the register file does not need extra ports. On an eight-core 12 nm cluster, the authors report up to 102 GFLOPS and 356 GFLOPS/W for MXFP8 matrix multiplication, a 25x speedup and 12.5x energy-efficiency gain over a software FP8-to-FP32 baseline, at a 5.1% cluster area cost. If this holds, MX arithmetic can be made practical on open RISC-V cores without a dedicated tensor accelerator.

What carries the argument

The load-bearing object is the MXDOTP dot product-accumulate unit inside the FPU, together with the mxdotp instruction and the three Stream Semantic Registers that feed it. The unit computes the MX general dot product for a block of eight elements: product of the two scale factors, product of eight FP8 pairs, accumulation in a 95-bit fixed-point representation with an anchor at 34, and a final round-to-nearest-even conversion to FP32. Early accumulation, borrowed from fused FP8 dot-product work, adds the shifted FP32 accumulator directly into the fixed-point sum before conversion rather than through a separate FP32 adder. The SSR extension is what makes four operands per cycle possible: two SSRs stream the A and B vectors through the FPU's two 64-bit inputs, and a third streams both scales merged with the accumulator through the third input, sidestepping the three-read-port register file.

What would settle it

Write an optimized software MXFP8 matrix multiplication kernel—one that batches FP8-to-FP32 conversions, loads scales ahead of the inner loop, and uses wider SIMD registers—run it on the baseline Snitch cluster under the same conditions, and compare the GFLOPS and GFLOPS/W against the MXDOTP numbers. If the optimized kernel approaches or exceeds the 20-25x gap reported, the central claim of a large hardware benefit would need to be restated.

Watch

Extended reading notes

Core claim

The central claim is that the overhead that keeps MX formats from being efficient in software—per-block scale loading, FP8-to-FP32 conversion, explicit exponent shifts, and separate accumulation—can be collapsed into a single hardware operation. MXDOTP is an ISA extension whose instruction reads eight FP8 elements of each operand packed in two 64-bit registers, plus two 8-bit block scales and an FP32 accumulator, and writes back the scaled accumulator. To keep operand bandwidth without enlarging the register file, the scales are merged with the accumulator and delivered through the third 64-bit input, fed by a Stream Semantic Register. The datapath converts both E5M2 and E4M3 FP8 inputs to an FP9 intermediate and accumulates in a 95-bit fixed-point form with an anchor at 34, so the eight-product sum plus shifted accumulator is exact before rounding to FP32. The paper reports 79.7% utilization of the ideal throughput, 3.1-3.4x over FP32 and 20.9-25.0x over the software MX kernel, and a 5.1% cluster area increase, concluding that native MX dot-product hardware is necessary to unlock MX benefits.

Load-bearing premise

The headline speedup and energy ratios are measured against a software FP8-to-FP32 matrix-multiplication kernel that does per-element conversions and explicit scalar scale loads; a more optimized software routine would reduce those ratios.

Editorial extensions

If this is right

  • MXFP8 matrix multiplication can run at near-peak throughput on a RISC-V core without a tensor accelerator, since the scaling and format-conversion steps no longer cost instructions.
  • The ISA extension keeps software flexibility: block sizes stay configurable and mixed-precision execution is possible, so accuracy can be traded against speed per layer.
  • The cluster-level area and idle-power costs are small enough (5.1% area, 1.9% power) that MX dot-product support could be added to general-purpose cores rather than only dedicated accelerators.
  • The speedup over FP32 means low-bitwidth MX formats can deliver their memory-footprint advantages without sacrificing throughput on embedded-class cores.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Since the software baseline kernel is written with straightforward per-element conversions and scalar scale loads, a more optimized software routine—batching conversions or overlapping scale loads with arithmetic—would likely shrink the reported 25x and 12.5x margins; the paper's qualitative conclusion about needing hardware support would probably survive, but the exact ratios would not.
  • The same instruction shape could extend to other MX formats (MXFP6, MXFP4, MXINT8) by swapping the element datapath, which would make the extension a general MX engine rather than an FP8-only one.
  • The trick of merging block scales with the accumulator to fit three register ports may transfer to other cores that already have stream registers, giving a low-cost path to MX support in a family of RISC-V designs.
  • If the 95-bit fixed-point accumulator is truly exact for the eight-product block, the same design could be re-purposed as an exact narrow-bitwidth dot-product primitive for numerical-analysis use, not just AI inference.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes MXDOTP, a RISC-V ISA extension for MXFP8 dot products, and integrates a dedicated dot-product-accumulate unit into the Snitch FPU. The unit consumes eight FP8 elements per operand plus two 8-bit block scales, uses an FP9 E5M3 intermediate format and a 95-bit fixed-point early-accumulation datapath with FP32 output, and streams operands via Snitch's SSR mechanism. The authors report post-layout results in 12 nm FinFET: up to 102 GFLOPS, 356 GFLOPS/W, 25x speedup and 12.5x energy-efficiency improvement over a software FP8-to-FP32 MX matrix-multiplication baseline, with a 5.1% cluster-level area increase and an 80% utilization of the MXDOTP unit.

Significance. If the quantitative claims hold, this is a worthwhile engineering contribution: it is the first open RISC-V ISA extension for MX dot products, it provides a full post-layout implementation in a publicly available flow, and it demonstrates a large efficiency gain over software-based MX processing. The use of SSRs to supply the extra scale operand without modifying the register file is a clean microarchitectural idea. However, the headline numbers depend on two load-bearing assumptions that are not currently established: the exact-accumulation guarantee of the fixed-point datapath, and the representativeness of the FP8-to-FP32 software baseline. These need to be addressed before the central claims can be accepted.

major comments (3)
  1. [Section III-A] The claim that the 95-bit fixed-point accumulator 'with an anchor at 34' guarantees an exact result for MXFP8 dot products is asserted without a derivation or a precise definition of 'anchor'. Under the natural reading that the integer part has about 34 bits, the range is too small: the largest finite E5M2 value is 1.75*2^15, so the sum of eight products of two such values is about 2^34.8, and the two E8M0 scales can each contribute exponents of up to +/-127, while the FP32 accumulator itself has a finite exponent range of roughly +/-126. Even if 'anchor at 34' instead means 34 fractional bits, the 95-bit format still cannot cover the full dynamic range of the FP32 accumulator together with the scale exponents. The paper must specify the exact fixed-point format, derive the minimum width as a function of the FP8 formats, the scale exponent range, and the FP32 accumulator range, and then validate it with directed edge-case tests (maximum exponent products, minimum subnormals, extreme scale values) against an exact software reference. Until this is provided, the 'exact result' statement in Section III-A and the numerical validity of the measured kernels are unsupported.
  2. [Section IV-B and Figure 2] The FP8-to-FP32 software baseline appears to be an unoptimized reference implementation. The code snippet reloads and recomputes the block scales for every group of eight elements even though MX blocks have size 32, performs per-element FP8-to-FP32 conversions with limited SIMD use, and applies the scale after each eight-element chunk. A software kernel that hoists scale loads to the outer loop, reuses the scale exponent over a full block, and uses the available Snitch SIMD/FREP facilities more aggressively could plausibly reduce the reported 25x speedup and 12.5x energy-efficiency ratios. Since these ratios are headline results, the authors should either justify that the presented baseline is representative of the best practical software MX implementation on Snitch, or report the comparison against a more optimized software kernel and discuss the sensitivity of the conclusions.
  3. [Section IV-C] No numerical accuracy comparison is reported among the FP32, FP8-to-FP32, and MXDOTP-based MXFP8 kernels. If the MXDOTP datapath is exact, its outputs should match the FP8-to-FP32 software baseline up to the order of additions and final rounding; reporting maximum/mean absolute and relative errors for the DeiT-Tiny workloads would directly validate the exactness claim and the MX compliance of the implementation. Without such validation, the measured throughput and energy numbers cannot be interpreted as delivering correct MX computation, even if the hardware performance itself is accurate.
minor comments (5)
  1. [Section III-A and Figure 1a] The datapath diagram omits the widths of the intermediate products, the scale-exponent alignment path, and the sign-bit handling; please add these details so that the fixed-point range can be checked by the reader.
  2. [Section IV-A] The power methodology is described only briefly. Please state how switching activities were extracted from the post-layout gate-level simulation, what clock-gating and activity factors were used, and the spread of power estimates across the ten DeiT-Tiny samples.
  3. [Table IV-D] The operation count for a single mxdotp instruction is not defined. The text says 1 FLOP is one multiplication or one addition, but an eight-element dot product involves eight multiplications and seven additions plus the accumulator add; the GFLOPS counting convention should be stated explicitly in the text or table caption.
  4. [Table II] The 'sl' field of the MXDOTP instruction is mentioned in the text but not explained in the encoding table; a sentence in the caption describing how bits 26-25 select the scales from rs3 would improve readability.
  5. [General] There are minor typographical and formatting inconsistencies (e.g., the author name 'Islamoglu' is rendered with unusual spacing in some places, and reference labels such as 'Lutzet al.' should be separated). These should be cleaned up in the final version.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MXDOTP results are direct hardware measurements with an openly specified, reproducible software baseline.

full rationale

The paper's central claims are engineering measurements, not a derivation from fitted parameters. The 25x speedup, 12.5x energy efficiency, and 356 GFLOPS/W figures come from post-layout simulations comparing explicitly listed kernels (FP32, FP8-to-FP32, MXFP8) in Figure 2 and Section IV-C. The software baseline is disclosed as source-level pseudocode and used as a comparison point; any question about whether a more optimized software kernel would reduce the speedup is a benchmark-fairness concern, not circularity. No parameter is fitted to one subset of data and then 'predicted' on a closely related quantity. The cited prior work (Snitch, SSR, MiniFloat-NN, Lutz et al.) is used as an architectural foundation and for state-of-the-art comparison, not as the sole justification of the paper's own results. There is no self-citation chain forcing the design choice, and no 'uniqueness theorem' is imported from the authors' prior work. The only notable weakness is the unproven exact-accumulation claim in Section III-A: the 95-bit fixed-point format 'with an anchor at 34' is asserted to 'guarantee an exact result' without a formal derivation. That is a correctness and numerical-analysis risk, not a circular reduction, because the measurement results do not define the accumulator width into existence. Therefore, no circular step is present.

Assumptions & free parameters 1 free parameters · 3 assumptions · 1 invented entities

All the numbers in the paper come from post-layout simulation of the authors' own RTL, so the central claims rest on the correctness of that RTL, the power methodology, and the fairness of the baseline. The accumulation exactness is the most technically load-bearing assertion; it is plausible but unproved.

free parameters (1)
  • Fixed-point accumulator width and anchor = 95 bits, anchor at 34
    The datapath's internal fixed-point format (Section III-A) is chosen by the designers; the claim that it exactly accumulates eight MXFP8 products plus the FP32 accumulator is asserted without a detailed proof, so the numerical correctness of the design depends on this choice.
assumptions (3)
  • domain assumption The 95-bit fixed-point datapath with anchor at 34 exactly sums eight MXFP8 products and the shifted FP32 accumulator without overflow or precision loss for all E5M2/E4M3 inputs
    Stated in Section III-A ('we conservatively select the minimum bitwidth required to guarantee an exact result') with no derivation; if false, MX dot product results would deviate from expected FP32-accurate behavior.
  • domain assumption Power estimated from post-layout gate-level simulation with switching activity from ten DeiT-Tiny samples is representative of the energy consumption of MX matrix multiplication kernels
    Section IV-A describes the power methodology; the reported 356 GFLOPS/W and 12.5x energy efficiency depend on this representativeness.
  • domain assumption The FP8-to-FP32 software kernel in Figure 2 is a fair software baseline for MX dot products
    The 25x speedup and 12.5x energy efficiency claims are measured against this kernel; a stronger software implementation would reduce the reported improvements.
invented entities (1)
  • MXDOTP instruction (mxdotp rd, rs1, rs2, rs3, sl) and its MXFP8 dot product-accumulate hardware unit independent evidence
    purpose: Fuses MXFP8 block scaling and eight-element dot product accumulation into a single RISC-V instruction
    The RTL is available on GitHub (pulp-platform/cvfpu, feature/mxdotp branch) and the encoding is specified in Table II, providing an external handle for verification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MXDOTP: A RISC-V ISA Extension for Enabling Microscaling (MX) Floating-Point Dot Products." pith.science (2026). https://pith.science/paper/CJMH6NWA

@misc{pith2026250513159,
  author       = {Pith},
  title        = {Pith review of: MXDOTP: A RISC-V ISA Extension for Enabling Microscaling (MX) Floating-Point Dot Products},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CJMH6NWA}},
  note         = {Machine review of arXiv:2505.13159}
}
read the original abstract

Fast and energy-efficient low-bitwidth floating-point (FP) arithmetic is essential for Artificial Intelligence (AI) systems. Microscaling (MX) standardized formats have recently emerged as a promising alternative to baseline low-bitwidth FP formats, offering improved accuracy with a block-wise shared exponent scale combined with per-element values. However, efficiently executing the key linear algebra primitives for AI applications on MX formats requires specialized hardware support for the fundamental operators such as scaled dot product. In this work, we propose MXDOTP, the first RISC-V ISA extension for MX dot products, focusing on the 8-bit MXFP8 FP format. We extend the open-source Snitch RISC-V core with a dedicated MXFP8 dot product-accumulate unit, which fully consumes blocks of eight 8-bit operands packed into 64-bit inputs. To feed MXDOTP at full utilization with four operands per cycle, including block scales, we exploit Snitch's Stream Semantic Registers (SSRs), achieving up to 80% utilization with minimal impact on the Snitch core's architecture and no modification to the register file. Implemented in 12 nm FinFET, a cluster with eight MXDOTP-extended cores reaches up to 356 GFLOPS/W when computing MXFP8 matrix multiplications at 0.8 V, 1 GHz. Compared to a software baseline, where MX dot products are computed by type casting FP8 inputs to FP32 for higher accumulation precision and applying explicit block scaling, the cluster achieves 25x speedup and 12.5x better energy efficiency at a minimal 5.1% area increase.

Figures

Figures reproduced from arXiv: 2505.13159 by the authors.

Figure 1
Figure 1. (a) Simplified MXDOTP datapath (sign bits omitted). (b) Integration of MXDOTP into the Snitch core. (c) Execution of MXDOTP over multiple cycles with three pipeline stages. SSRs enable efficient operand streaming, ensuring full utilization without stalls. TABLE I: Operands of the MXDOTP instruction. Symbol Operand Role Data Type P A / P B Elements of A / B Input 8 × FP8 XA / XB Scale factor of A / B Input INT8 C Acc… view at source ↗
Figure 2
Figure 2. Comparison of FP32, FP8-to-FP32, and MXFP8 MM kernels. M and N represent the output matrix’s row and column dimensions, respectively. FPU 151 76 FREP 15 8 FP RF 25 13 Other 7 4 FP Subsystem 197 kGE % FMA 78 51 MXDOTP 26 17 COMP 18 12 CAST 27 18 Other 3 2 FPU 151 kGE % [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Area breakdown of the MXDOTP-extended Snitch core. 64 128 192 256 Inner Dimension 0 25 50 75 100 GFLOPS 3.1x 3.3x 3.4x 20.9x 24.1x 25.0x 24.5x (a) Throughput 64 128 192 256 Inner Dimension 0 100 200 300 GFLOPS/W 3.0x 3.2x 3.2x 10.4x 12.0x 12.5x 12.2x (b) Energy Efficiency FP32 FP8-to-FP32 MXFP8 [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: (a) Throughput and (b) Energy Efficiency of [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 13 canonical work pages

  1. [1]

    Pushing the Limits of Narrow Precision Infer- encing at Cloud Scale with Microsoft Floating Point,

    B. D. Rouhaniet al., “Pushing the Limits of Narrow Precision Infer- encing at Cloud Scale with Microsoft Floating Point,” inNIPS, 2020

    work page 2020

  2. [2]

    Microscaling Data Formats for Deep Learning,

    B. D. Rouhaniet al., “Microscaling Data Formats for Deep Learning,”

    work page

  3. [3]

    Fused FP8 4-Way Dot Product With Scaling and FP32 Accumulation,

    D. R. Lutzet al., “Fused FP8 4-Way Dot Product With Scaling and FP32 Accumulation,” inARITH, 2024

    work page 2024

  4. [4]

    MiniFloats on RISC-V Cores: ISA Extensions With Mixed-Precision Short Dot Products,

    L. Bertacciniet al., “MiniFloats on RISC-V Cores: ISA Extensions With Mixed-Precision Short Dot Products,”IEEE Trans. on Emerging Topics in Computing, vol. 12, no. 4, pp. 1040–1055, 2024

    work page 2024

  5. [5]

    OCP Microscaling Formats (MX) Specification,

    B. D. Rouhaniet al., “OCP Microscaling Formats (MX) Specification,”

    work page

  6. [6]

    NVIDIA Tensor Cores: Versatility for HPC & AI

    “NVIDIA Tensor Cores: Versatility for HPC & AI.” [Online]. Available: https://www.nvidia.com/en-us/data-center/tensor-cores/

    work page

  7. [7]

    Available: https://www.opencompute.org/documents/ ocp-microscaling-formats-mx-v1-0-spec-final-pdf

    [Online]. Available: https://www.opencompute.org/documents/ ocp-microscaling-formats-mx-v1-0-spec-final-pdf

    work page

  8. [8]

    Data Formats and Math Fidelity — TT Buda documentation

    “Data Formats and Math Fidelity — TT Buda documentation.” [Online]. Available: https://docs.tenstorrent.com/pybuda/latest/dataformats.html

    work page

Show all 14 references
  1. [9]

    AMD Versal™ AI Edge Series Gen 2 Product Brief

    “AMD Versal™ AI Edge Series Gen 2 Product Brief.” [Online]. Avail- able: https://www.amd.com/content/dam/amd/en/documents/products/ adaptive-socs-and-fpgas/versal/versal-ai-edge-gen2-product-brief.pdf

  2. [10]

    Snitch: A Tiny Pseudo Dual-Issue Processor for Area and Energy Efficient Execution of Floating-Point Intensive Workloads,

    F. Zarubaet al., “Snitch: A Tiny Pseudo Dual-Issue Processor for Area and Energy Efficient Execution of Floating-Point Intensive Workloads,” IEEE Trans. Comput., vol. 70, no. 11, pp. 1845–1860, 2021

    2021

  3. [11]

    Stream Semantic Registers: A Lightweight RISC- V ISA Extension Achieving Full Compute Utilization in Single-Issue Cores,

    F. Schuikiet al., “Stream Semantic Registers: A Lightweight RISC- V ISA Extension Achieving Full Compute Utilization in Single-Issue Cores,”IEEE Trans. Comput., vol. 70, no. 2, pp. 212–227, 2021

    2021

  4. [12]

    Exact Dot Product Accumulate Operators for 8-bit Floating-Point Deep Learning,

    O. Desrenteset al., “Exact Dot Product Accumulate Operators for 8-bit Floating-Point Deep Learning,” inDSD, 2023

    2023

  5. [13]

    Training data-efficient image transformers & distil- lation through attention,

    H. Touvronet al., “Training data-efficient image transformers & distil- lation through attention,” inICML, 2021

    2021

  6. [2023]

    Available: http://arxiv.org/abs/2310.10537

    [Online]. Available: http://arxiv.org/abs/2310.10537

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.