Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

The Cambrian Explosion of Mixed-Precision Matrix Multiplication for Quantized Deep Learning Inference

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By recasting GEMM's innermost loop as INT8 dot products accumulating into INT32, the paper shows quantized ResNet-50 and BERT-Large inference runs about 1.7–2.3× faster than tuned FP32 on three edge CPUs, losing at most about one accuracy…

desk verdict Useful kernel porting guide with new K1 measurements, but the headline speedup ratios rest on an unvalidated FP32 baseline and an unmeasured energy claim. read the letter →

arxiv 2506.11728 v1 pith:W6LXFIKK submitted 2025-06-13 cs.CL

Héctor Martínez , Adrián Castelló , Francisco D. Igual , Enrique S. Quintana-Ortí This is my paper · ORCID
classification cs.CL
keywords mixed-precisionGEMMINT8quantizedinferencedot-productmicro-kernelsmatrixenginesARMNEONandSVE2IntelAMXRISC-VSpacemiTK1edgeCPU
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the classic recipe for fast matrix multiplication on CPUs, the Goto–van de Geijn framework built around vector AXPY operations in FP32, no longer matches what modern hardware offers, and shows how to rebuild it for mixed-precision integer (MIP) arithmetic: INT8 inputs multiplied and accumulated into INT32, the format behind quantized deep-learning inference. It details micro-kernels and packing layouts for six instruction sets, ARMv8.0-A NEON, ARMv8.2 NEON, ARM SVE2, ARM SME, Intel AMX, and the RISC-V SpacemiT K1's integrated matrix engine, while keeping the outer loop structure of the framework intact. On three edge boards, the MIP kernels deliver end-to-end speedups of 1.67×, 2×, and 2.1× for ResNet-50 and 2.25×, 1.67×, and 2.32× for BERT-Large relative to a tuned FP32 GEMM, with accuracy drops of roughly one percentage point or less. If these numbers hold, INT8 GEMM is a practical way to run latency-sensitive inference on commodity CPUs, not only on GPUs or dedicated accelerators.

What carries the argument

The carrying object is the GEMM micro-kernel recast from AXPY to DOT form inside the fixed Goto–van de Geijn macro-kernel, a set of five nested tiling loops with two packing stages that reorganize blocks of A and B into contiguous buffers. The decisive change sits in the innermost loop and in the packer: instead of spreading a rank-1 update over vector FMA instructions on FP32, the micro-kernel issues dot-product instructions, for example vdotq_laneq_s32 in ARMv8.2 NEON (four 4-element INT8 dot products accumulated into INT32 per call), vmadot in the SpacemiT K1 (a 4×4×8 INT8-to-INT32 matrix multiply), tile dpbssd for Intel AMX (16×16×64), and smopa for ARM SME, and the packing routines lay out micro-panels so that each instruction's operands are read with stride-1 access in the native register or tile lane order. The mechanical consequence is that packing and compute are co-designed per ISA while the macro-kernel, the parallelism, and the blocking parameters are inherited from the classical framework.

What would settle it

Re-run the two workloads on the same three boards with the paper's FP32 GEMM, the paper's INT8+INT32 GEMM, and a well-tuned FP32 GEMM from a mainstream BLAS library such as OpenBLAS, BLIS, or the ARM Performance Library. If the vendor FP32 kernel materially outperforms the paper's FP32 baseline, the reported speedups shrink proportionally, and if a vendor INT8 GEMM matches or beats the MIP kernels, the claim that hand-tuned micro-kernels are required for these gains would need qualification.

Watch

Extended reading notes

Core claim

The discovery is that the Goto–van de Geijn GEMM blueprint survives the transition to quantized inference, but only if its core is inverted: instead of decomposing the micro-kernel's rank-1 update into AXPY operations on FP32 vectors, the micro-kernel must be expressed as DOT-product operations that feed INT8 operands into mixed-precision accumulators (INT8×INT8→INT32), and the packing routines must be redesigned so that the micro-panels of A and B arrive in the exact lane and tile layout each ISA's dot or matrix-engine instruction expects. The authors show this working across a deliberately heterogeneous set of ISAs, from the small ARM Cortex-A72 with ARMv8.0-A NEON to the ARM Cortex-A78AE with ARMv8.2 dot products, the RISC-V SpacemiT K1's vmadot matrix engine, Intel AMX tiles, and ARM SME's outer-product tiles, an era of diversification they call the Cambrian period of matrix multiplication. A hardware-native DOT instruction compresses the micro-kernel to a few vdotq_laneq_s32 calls, where the older ARMv8.0-A NEON needs multi-stage widening, multiply, and pairwise-add sequences. The empirical payoff is that MIP inference speeds up by roughly 1.7–2.3× end-to-end versus FP32 across both a vision model (ResNet-50 v1.5 on ImageNet) and a language model (BERT-Large on SST-2), quantized accuracy stays within about one percentage point of the FP32 model, and memory footprint drops to roughly a quarter to a third of the FP32 model.

Load-bearing premise

The reported speedups rest on the premise that the paper's FP32 GEMM is nearly as fast as the best possible FP32 GEMM on each of the three boards; the authors cite their earlier study for that competitiveness rather than benchmarking vendor libraries head-to-head here, so a slower-than-optimal FP32 baseline would inflate the gains.

Editorial extensions

If this is right

  • Quantized INT8 inference on CPUs reaches a performance tier that previously motivated dedicated accelerators: on the tested boards, the MIP GEMMs run ResNet-50 and BERT-Large end-to-end at 1.7–2.3× the FP32 rate with accuracy cost of about one point or less.
  • The Goto–van de Geijn framework remains the right scaffolding for high-performance GEMM; the portability burden has moved entirely into the micro-kernel and packing routines, which must be regenerated for each instruction set.
  • As mixed-precision instructions define the new peak throughput of CPUs, mainstream and vendor BLAS libraries will need MIP GEMM variants as first-class kernels to reach the hardware ceiling.
  • In the measured cases, energy on the Jetson platform drops 5.1× for ResNet-50 and 2.72× for BERT-Large, making INT8 GEMM a direct lever for battery- and thermally-constrained edge deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The measured boards are early adopters of the new instructions; the peak ratios reported in the paper's Table 1 for newer matrix engines (Intel AMX near 65× and ARM SME near 34× the FP32 FMA rate) imply that on those chips the MIP-versus-FP32 gap could dwarf the roughly 2× observed here, provided packing and the memory pipeline keep the engines fed.
  • The paper quantizes only weight-to-activation GEMMs in BERT and uses dynamic quantization; applying the same MIP micro-kernels to the attention-score and softmax GEMMs, or to INT4 inputs, is a natural stress test of whether the DOT-centric packing generalizes beyond the reported accuracy results.
  • On the SpacemiT K1, the four IME accelerators did not beat the eight integer cores (one configuration was 1.2× slower), so the practical way to use the IMEs on RISC-V remains an open tuning question, likely resolved by larger matrices or by overlapping core and accelerator work rather than the static split tested here.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents a systematic study of mixed-precision integer (INT8 inputs, INT32 accumulation) GEMM kernels for quantized deep learning inference on CPU architectures. It reviews the evolution of SIMD and matrix-engine ISAs (NEON, ARMv8.2 dot product, SVE2, SME, Intel AMX, and the SpacemiT K1 IME), describes micro-kernel and packing designs for each, and evaluates two representative workloads (ResNet50v1.5 on ImageNet and BERT-Large on SST-2) on three edge platforms: Raspberry Pi (Cortex-A72), Jetson AGX Orin (Cortex-A78AE), and Banana Pi BPI-F3 (SpacemiT K1). The reported end-to-end speedups over a custom FP32 GEMM range from about 1.67x to 2.32x, with accuracy drops near 1%. The paper also makes qualitative and quantitative claims about energy savings. The central claims are plausible given the hardware throughput ratios, but the credibility of the speedup numbers depends on the competitiveness of the FP32 baseline, which is not demonstrated in the paper.

Significance. The paper addresses a timely and practically important topic: fast quantized GEMM on heterogeneous CPU/edge hardware. Its strengths include a broad taxonomy of mixed-precision ISA extensions, concrete micro-kernel and packing designs for several modern ISAs, direct measurements on three quite different platforms, and quantified accuracy impact for two standard DL workloads. The per-layer GOPS improvements are broadly consistent with the instruction-level throughput ratios reported in Table 1, which lends internal credibility to the implementation work. However, the headline speedups are ratios against a self-implemented FP32 GEMM whose competitiveness is only asserted via a previous self-citation and never benchmarked in this paper. The energy figures in the conclusions are also not backed by a described measurement. If the baseline comparison is added and the energy claims are corrected, the paper would be a useful reference for practitioners implementing quantized inference on CPUs.

major comments (4)
  1. [Section 7.4] The speedups reported in Section 7.5 for ResNet50v1.5 (1.67x, 2x, 2.1x) and in Section 7.6 for BERT-Large (2.25x, 1.67x, 2.32x) are ratios against 'our FP32 GEMM' described in Section 7.4. The text claims this baseline is competitive with BLIS, OpenBLAS, and ARM PL only by citing the authors' previous work [33]; no benchmark of the FP32 baseline against those libraries is included for the three boards evaluated here. If the FP32 baseline is slower than a well-tuned vendor library, all headline speedups are inflated. Please add a head-to-head FP32 GEMM comparison (GOPS and/or total inference time) against at least one established BLAS implementation on each platform, and report the achieved fraction of the theoretical FP32 peak. It would also strengthen the paper to compare the MIP GEMM against an existing quantized GEMM library (e.g., ruy, QNNPACK, or oneDNN) on the same platforms.
  2. [Section 8] The conclusion states that energy was 'not fully measured' but then gives precise energy values: 0.98 J vs 0.20 J for a single-image ResNet50 inference on the ARM Cortex-A78AE and 0.79 J vs 0.29 J for BERT-Large with l=512, b=1. These numbers are not backed by any described measurement methodology, and the caveat 'not fully measured' directly contradicts their presentation as results. Please either add a full energy measurement methodology (power measurement device, sampling rate, CPU frequency/power-state settings) and make energy a real experimental result, or delete the quantitative Joule figures and keep only the qualitative statement.
  3. [Section 7.3] The dynamic quantization method is not specified precisely enough to reproduce the accuracy results. The paper does not say whether the INT8 scale factors are per-tensor or per-channel, how the activation ranges are determined (e.g., calibration set, running statistics, or runtime min/max), how biases and scaling are fused, or which parts of the models (softmax, layer normalization, GELU, the M5/M7 attention GEMMs) remain in FP32. Since Tables 6 and 7 make a quantitative claim of at most about 1% accuracy drop, please provide a complete, step-by-step description of the quantization pipeline.
  4. [Table 6] The FP32 baseline top-1 accuracy of 0.711 on 50,000 ImageNet images is approximately five percentage points below the expected performance of ResNet50v1.5 on ImageNet (typically around 0.76). If the evaluation pipeline (preprocessing, resize/crop, batch size, or data split) differs from standard practice, please describe it; otherwise this suggests a measurement issue, and the claim that quantization drops accuracy by only about 1% may not transfer to standard conditions.
minor comments (5)
  1. [Throughout] There are several typos and spacing issues, including 'ResNet50v1.9' in Section 8 (should be v1.5), 'propietary', 'descructively', 'atractive', 'wokloads', and inconsistent spacing in 'A VX' and 'V oltage'.
  2. [Figures 6 and 7] The right-hand 'Aggregated time' plots label the x-axis as 'Layer' but appear to show cumulative time over layers; please clarify the x-axis semantics and how the cumulative curve is constructed.
  3. [Section 7.4] The sentence 'In a previous work [33] we demonstrated...' is a self-citation; if the revised paper includes the requested head-to-head baseline benchmarks, this sentence should be updated or removed so that the competitiveness claim is evidenced in this manuscript.
  4. [Section 7.5] The statement that average per-layer speedups of 2.3x and 4.2x 'matches the number of SIMD units in each processor' is unclear and not substantiated; please replace it with a direct reference to the measured instruction throughput ratios or to Table 1.
  5. [Section 7.1] The performance results are reported as averages over at least 50 seconds, but no standard deviation or run-to-run variation is given; please report at least min/max or standard deviation for the end-to-end times, particularly on the 8-core SpacemiT K1 where thread/IME scheduling may vary.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; speedups are direct measurements and the only self-citation is non-load-bearing support for the FP32 baseline.

full rationale

The paper's central quantitative claims are direct empirical measurements rather than derivations. Section 7.5 reports speedups computed as ratios of measured execution times between the paper's FP32 GEMM and its MIP INT8+INT32 GEMM on three boards; no parameter is fitted to a subset of data and then renamed as a prediction, and no quantity is defined in terms of the result it is supposed to explain. The MIP micro-kernels and packing schemes in Sections 5 and 6 are concrete implementations built from documented ISA intrinsics and instructions, and their performance is measured, not derived from self-referential assumptions. The only self-citation occurs in Section 7.4, where the paper supports the competitiveness of its FP32 baseline by citing the authors' previous work [33]; this is a mild self-citation, but it is not load-bearing in a circular sense, because the reported speedup ratio would remain a measured quantity even if the baseline's competitiveness were doubted. The baseline-quality concern raised by the weakest assumption is a benchmarking-validity question, not a circularity of the paper's derivation chain. Similarly, the energy figures in Section 8 are inconsistent with the caveat that energy was 'not fully measured' in the study, but this is an internal consistency issue rather than a circular step. No equation reduces to its own inputs, and no claimed prediction is forced by a fitted value or by a self-citation chain.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central performance claim rests on tuned but undisclosed kernel parameters, on the validity of the Goto framework when extended to MIP hardware, and on the assumption that INT8 quantization preserves accuracy. No new physical or mathematical entities are postulated; the 'Cambrian explosion' is a descriptive label, not a technical entity.

free parameters (2)
  • Cache blocking and micro-kernel tile sizes (mc, nc, kc, mr, nr) = Not reported in the paper
    Performance results depend on platform-specific tuning of these parameters; values are mentioned as tailored per platform in Section 7.4 but not disclosed, blocking re-implementation.
  • Dynamic quantization ranges (scale factors) for weights and activations = Per-tensor ranges from min/max, exact values not reported
    The accuracy outcomes in Tables 6 and 7 depend on the quantization ranges chosen; the paper describes dynamic quantization without reporting per-tensor details.
assumptions (4)
  • domain assumption The Goto and van de Geijn macro-kernel framework remains the correct organizing structure for mixed-precision integer GEMM on modern ISAs.
    Sections 3 to 6 adapt this framework to dot-product and matrix-engine ISAs without proving it is optimal for these new units; the framework is standard but its extension to MIP is assumed.
  • domain assumption INT8 inputs with INT32 accumulation preserve model accuracy within about one percentage point for the evaluated models.
    Tables 6 and 7 show this for ResNet50v1.5 and BERT-Large on ImageNet and SST-2 respectively; generalization to other models and datasets is assumed.
  • domain assumption The peak performance numbers in Table 1 are measured or estimated on representative platforms using the stated instructions.
    Section 2.1 says the table reports empirical performance numbers but never describes the benchmark, platform configuration, or measurement method; cross-ISA ratios are therefore indicative only.
  • standard math The count of useful operations is 2mnk for GEMM and 2mn for matrix-vector products, as used for GOPS rates.
    This definition is used in Section 7.4 to compute GOPS rates; it is a conventional metric in the high-performance computing literature.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Cambrian Explosion of Mixed-Precision Matrix Multiplication for Quantized Deep Learning Inference." pith.science (2026). https://pith.science/paper/W6LXFIKK

@misc{pith2026250611728,
  author       = {Pith},
  title        = {Pith review of: The Cambrian Explosion of Mixed-Precision Matrix Multiplication for Quantized Deep Learning Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W6LXFIKK}},
  note         = {Machine review of arXiv:2506.11728}
}
read the original abstract

Recent advances in deep learning (DL) have led to a shift from traditional 64-bit floating point (FP64) computations toward reduced-precision formats, such as FP16, BF16, and 8- or 16-bit integers, combined with mixed-precision arithmetic. This transition enhances computational throughput, reduces memory and bandwidth usage, and improves energy efficiency, offering significant advantages for resource-constrained edge devices. To support this shift, hardware architectures have evolved accordingly, now including adapted ISAs (Instruction Set Architectures) that expose mixed-precision vector units and matrix engines tailored for DL workloads. At the heart of many DL and scientific computing tasks is the general matrix-matrix multiplication gemm, a fundamental kernel historically optimized using axpy vector instructions on SIMD (single instruction, multiple data) units. However, as hardware moves toward mixed-precision dot-product-centric operations optimized for quantized inference, these legacy approaches are being phased out. In response to this, our paper revisits traditional high-performance gemm and describes strategies for adapting it to mixed-precision integer (MIP) arithmetic across modern ISAs, including x86_64, ARM, and RISC-V. Concretely, we illustrate novel micro-kernel designs and data layouts that better exploit today's specialized hardware and demonstrate significant performance gains from MIP arithmetic over floating-point implementations across three representative CPU architectures. These contributions highlight a new era of gemm optimization-driven by the demands of DL inference on heterogeneous architectures, marking what we term as the "Cambrian period" for matrix multiplication.

Figures

Figures reproduced from arXiv: 2506.11728 by the authors.

Figure 1
Figure 1. Reference GEMM algorithm (macro-kernel) in GotoBLAS2. To fully optimize GEMM on modern multicore CPUs, de￾velopers must address three key challenges: • Carefully tune the blocking (tiling) parameters mc, nc, kc to make effective use of the cache hierarchy [11, 19]. • Ensure ample parallelism within the loop structure to fa￾cilitate a balanced workload distribution across CPU cores and to leverage both private and sh… view at source ↗
Figure 2
Figure 2. Packing (top) and AXPY-oriented micro-kernel (bottom) for mr × nr = 4 × 8, FP32 data, and VL=128 bits in ARMv8.2 NEON. In the top figure, ⊙ refers to an AXPY between the left-hand side scalar operand and the right-hand side vector register. Operation ARMv8.2 NEON ARM SVE RVV 1.0 Load vld1q f32 svld1 f32 vle32 v f32m1 Computation vfmaq laneq f32 svmla lane f32 vfmacc vf f32m1 Store vst1q f32 svst1 f32 vse32 v f32m1 … view at source ↗
Figure 3
Figure 3. Packing (top) and DOT-oriented micro-kernel (bottom) for mr × nr = 2 × 8, MIP INT8+INT32 data, and VL=128 bits in ARMv8.0-A NEON. In the top figure, ⊗ refers to DOT operations between the left-hand side vector and each vector of the right-hand side; ⊕ refers to the element-wise addition. box for the micro-tile Cr in the top part of [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Packing (top) and DOT-oriented micro-kernel (bottom) for mr × nr = 4 × 16, MIP INT8+INT32 data, and VL=128 bits in ARMv8.2 NEON. data is basically the same given in [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Packing (top) and matrix-oriented micro-kernel (bottom) for [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Performance of FP32 and MIP inference for ResNet50v1.5 model [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Performance of FP32 and MIP inference for a single encoder block of BERT-Large model [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. INT8 Quantization Makes ARM Edge Inference Dispatch-Invariant

    cs.ET 2026-07 conditional novelty 6.0 of 10

    INT8 QDQ CNNs are byte-exact across ARM Cortex-A53/A72/A76 under XNNPACK even with different SIMD kernels, unlike FP32 or x86 INT8.

Reference graph

Works this paper leans on

33 extracted references · 26 canonical work pages · cited by 1 Pith paper

  1. [33]

    Mart ´ınez, F

    H. Mart ´ınez, F. D. Igual, R. Rodr´ıguez-S´anchez, S. Catal´an, A. Castell ´o, E. S. Quintana-Ort ´ı, Inference with transformer encoders on ARM and RISC-V multicore processors, in: Euro-Par 2024: Parallel Processing, 2024, pp. 377–392. 16

    work page 2024

  2. [1]

    Hubara, M

    I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, Y . Bengio, Quantized neural networks: training neural networks with low precision weights and activations, J. Machine Learning Research 18 (1) (2017) 6869–6898

    work page 2017

  3. [2]

    Dettmers, M

    T. Dettmers, M. Lewis, Y . Belkada, L. Zettlemoyer, LLM.int8(): 8-bit matrix multiplication for transformers at scale, in: 36th NIPS, Curran Associates Inc., Red Hook, NY , USA, 2024

    work page 2024

  4. [3]

    G. Xiao, J. Lin, M. Seznec, H. Wu, J. Demouth, S. Han, SmoothQuant: accurate and efficient post-training quantization for large language mod- els, in: 40th ICML, JMLR.org, 2023

    work page 2023

  5. [4]

    Mart ´ınez, S

    H. Mart ´ınez, S. Catal ´an, A. Castell ´o, J. I. Mestre, E. S. Quintana- Ort´ı, Latency-critical quantized inference with transformer decoders on ARM and RISC-V CPUs, IEEE Internet of Things Journal (2025) 1– 1doi:10.1109/JIOT.2025.3560382

    work page arXiv 2025

  6. [5]

    Kågstr ¨om, P

    B. Kågstr ¨om, P. Ling, C. H. V . Loan, GEMM-based level 3 BLAS: High-performance model implementations and performance evalua- tion benchmark, ACM Trans. Math. Softw. 24 (3) (1998) 268–302. doi:10.1145/290328.290332

    work page arXiv 1998

  7. [6]

    J. W. Demmel, Applied Numerical Linear Algebra, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, 1997

    work page 1997

  8. [7]

    Goodfellow, Y

    I. Goodfellow, Y . Bengio, A. Courville, Deep Learning, MIT Press, 2016

    2016

Show all 33 references
  1. [8]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, in: Advances in Neural Information Processing Systems, V ol. 30, 2017

    2017

  2. [9]

    Chollet, Deep Learning with Python, Manning Publications, 2017

    F. Chollet, Deep Learning with Python, Manning Publications, 2017

    2017

  3. [10]

    J. J. Dongarra, J. Du Croz, S. Hammarling, I. Duff, A set of level 3 basic linear algebra subprograms, ACM Trans. Math. Softw. 16 (1) (1990) 1– 17

    1990

  4. [11]

    K. Goto, R. van de Geijn, Anatomy of high-performance matrix multipli- cation, ACM Trans. Math. Softw. 34 (3) (2008) 12:1–12:25

    2008

  5. [12]

    C. L. Lawson, R. J. Hanson, D. R. Kincaid, F. T. Krogh, Basic linear algebra subprograms for Fortran usage, ACM Trans. Math. Soft. 5 (3) (1979) 308–323

    1979

  6. [13]

    H. Kim, G. Ye, N. Wang, A. Yazdanbakhsh, N. S. Kim, Exploiting intel advanced matrix extensions (amx) for large language model in- ference, IEEE Computer Architecture Letters 23 (1) (2024) 117–120. doi:10.1109/LCA.2024.3397747

    2024

  7. [14]

    A. F. AbouElhamayed, J. Dotzel, Y . Akhauri, C.-C. Chang, S. Gobriel, J. P. Mu˜noz, V . S. Chua, N. Jain, M. S. Abdelfattah, SparAMX: Accelerat- ing compressed LLMs token generation on AMX-powered CPUs (2025). arXiv:2502.12444. URLhttps://arxiv.org/abs/2502.12444

    2025 arXiv

  8. [15]

    J. Xiao, Q. Huang, X. Chen, C. Tian, Understanding large language mod- els in your pockets: Performance study on cots mobile devices (2025). arXiv:2410.03613. URLhttps://arxiv.org/abs/2410.03613

    2025

  9. [16]

    Releases · space-mit/riscv-ime-extension-spec — github.com,https:// github.com/space-mit/riscv-ime-extension-spec/releases, [Accessed 02-06-2025]

    2025

  10. [17]

    Remke, A

    S. Remke, A. Breuer, Hello sme! generating fast matrix multiplication kernels using the scalable matrix extension (2024). arXiv:2409.18779. URLhttps://arxiv.org/abs/2409.18779

    2024 arXiv

  11. [18]

    URLhttps://www.intel.com/content/ www/us/en/content-details/671488/ intel-64-and-ia-32-architectures-optimization-\ \reference-manual-volume-1.html

    Intel, Intel®64 and IA-32 architectures optimization reference manual volume 1 (Apr 2024). URLhttps://www.intel.com/content/ www/us/en/content-details/671488/ intel-64-and-ia-32-architectures-optimization-\ \reference-manual-volume-1.html

    2024

  12. [19]

    T. M. Low, F. D. Igual, T. M. Smith, E. S. Quintana-Ort´ı, Analytical mod- eling is enough for high-performance BLIS, ACM Trans. Math. Softw. 43 (2) (2016) 12:1–12:18

    2016

  13. [20]

    F. G. Van Zee, R. A. van de Geijn, The BLIS framework: Experiments in portability, ACM Trans. Math. Softw. 42 (2) (2016) 12:1–12:19

    2016

  14. [21]

    T. M. Smith, R. van de Geijn, M. Smelyanskiy, J. R. Hammond, F. G. V . Zee, Anatomy of high-performance many-threaded matrix multiplication, in: Proc. IEEE 28th IPDPS, 2014, pp. 1049–1059. 15

    2014

  15. [22]

    Williams, A

    S. Williams, A. Waterman, D. Patterson, Roofline: An insightful perfor- mance model for multicore architectures, Communications of the ACM 52 (4) (2009) 65–76

    2009

  16. [23]

    K. Dowd, C. R. Severance, High Performance Computing, 2nd Edition, O’Reilly, 1998

    1998

  17. [24]

    Natesh, H

    V . Natesh, H. T. Kung, PQS (prune, quantize, and sort): Low-bitwidth accumulation of dot products in neural network computations (2025). arXiv:2504.09064. URLhttps://arxiv.org/abs/2504.09064

    2025 arXiv

  18. [25]

    Finkelstein, U

    A. Finkelstein, U. Almog, M. Grobman, Fighting quantization bias with bias (2019). arXiv:1906.03193. URLhttps://arxiv.org/abs/1906.03193

    2019 arXiv

  19. [26]

    K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recog- nition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

    2016

  20. [27]

    Sze, Y .-H

    V . Sze, Y .-H. Chen, T.-J. Yang, J. S. Emer, Efficient processing of deep neural networks: A tutorial and survey, Proceedings of the IEEE 105 (12) (2017) 2295–2329. doi:10.1109/JPROC.2017.2761740

    2017

  21. [28]

    Krizhevsky, I

    A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, in: Proceedings of the 25th Interna- tional Conference on Neural Information Processing Systems - V olume 1, NIPS’12, Curran Associates Inc., USA, 2012, pp. 1097–1105. U...

    2012

  22. [29]

    Chellapilla, S

    K. Chellapilla, S. Puri, P. Simard, High performance convolutional neural networks for document processing, in: International Workshop on Fron- tiers in Handwriting Recognition, 2006

    2006

  23. [30]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, BERT: Pre-training of deep bidirectional transformers for language understanding, in: Proc. 2019 Conf. North American Chapter Assoc. Computational Linguistics: Human Language Techn., 2019, pp. 4171–4186

    2019

  24. [31]

    Socher, A

    R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng, C. Potts, Recursive deep models for semantic compositionality over a sentiment treebank, in: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Association for Computational L...

    2013

  25. [32]

    Kluska, M

    P. Kluska, M. Zikeba, Post-training quantization methods for deep learning models, in: N. T. Nguyen, K. Jearanaitanakij, A. Selamat, B. Trawi ´nski, S. Chittayasothorn (Eds.), Intelligent Information and Database Systems, Springer International Publishing, Cham, 2020, pp. 467–479

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.