Pith. sign in

REVIEW 4 major objections 5 minor 50 references

Refining Datapath for Microscaling ViTs

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MXInt datapaths run the whole Vision Transformer on FPGA fabric.

desk verdict Genuine MXInt datapath work with a real but fixable overclaim problem and an unvalidated alignment bound; worth serious review after revision. read the letter →

arxiv 2505.22194 v2 pith:Z6CPUH35 submitted 2025-05-28 cs.AR

Can Xiao , Jianyi Cheng , Aaron Zhao This is my paper
classification cs.AR
keywords VisionTransformerFPGAacceleratorMXIntmicroscalingintegerpost-trainingquantizationLayerNormSoftmaxGELU
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the Microscaling Integer (MXInt) format, in which a block of values shares one exponent, lets every operation of a Vision Transformer—not just the matrix multiplications—run on FPGA fabric with very small mantissas. Prior ViT accelerators left accuracy-sensitive non-linear operations such as LayerNorm, Softmax and GELU on the CPU, paying a communication penalty. The authors show that after re-quantizing MXInt inputs to a common shared exponent, these operations become integer-only arithmetic plus small lookup tables, reducing their datapath to 2–5 mantissa bits. They report that within a 1% ImageNet top-1 accuracy loss, this achieves at least 93× speedup compared to a Float16 baseline and at least 1.9× speedup compared to related work, with up to 4.99× memory density improvement. If correct, it removes the CPU–FPGA split for ViT inference and makes fully on-fabric ViT accelerators practical.

What carries the argument

The load-bearing object is the MXInt number format: a block of values shares one exponent while each value keeps its own sign and small signed mantissa. The argument runs through three datapath identities. For LayerNorm, re-quantizing every input to the block's maximum exponent ($x = 2^{x_{\max,e}} x_m'$) lets the $2^{x_{\max,e}}$ cancel between numerator and denominator, reducing the operator to mantissa-only arithmetic with a $(x_v')^{-1/2}$ lookup table and exponent shifts. GELU uses the small mantissa width to justify a piecewise design: ReLU outside $|x| \ge a$ and a lookup table inside, with the exponent forwarded unchanged. Softmax splits $e^x = 2^{x\log_2 e}$ into an integer part handled by shifts and a fixed-point remainder $r$ of only two bits feeding a $2^r$ lookup table, and the division of two such values becomes mantissa division with exponent subtraction. Together these convert every non-linear operator into integer datapaths whose cost scales with mantissa width, not full floating-point width.

What would settle it

Run the same post-training MXInt pipeline on a ViT with a wider internal dynamic range than DeiT-Base—for example DeiT-Large, or DeiT-Small at 384×384 input resolution—and measure ImageNet top-1. If the accuracy loss exceeds 1% with the paper's 5/5/2-bit datapaths and 16/256 block sizes, the central claim fails; additionally, logging the per-tensor exponent spread of LayerNorm inputs would show directly whether the right-shift range (up to 6 bits) is the limiting factor.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that MXInt's block-shared exponent is not a quantization burden to work around in non-linear operators but a resource that makes them cheap. By forcing all values entering LayerNorm, Softmax and GELU to share one exponent ($x = 2^{x_{\max,e}} x_m'$), the exponent cancels out of the ratio in LayerNorm, so the whole operator can be computed on signed mantissas with integer arithmetic and a small reciprocal-square-root LUT. GELU becomes a lookup table over a narrow domain ($|x| < a$) with ReLU outside, and Softmax becomes a two-bit fixed-point remainder feeding a power-of-two LUT with the integer part handled by shifts. The resulting datapaths use 5 mantissa bits for LayerNorm and GELU and 2 for Softmax, cutting LUT entries at least 16× against vanilla MXInt implementations while keeping accuracy loss under 1% on DeiT Tiny, Small and Base. The system-level claim is that the whole ViT workload maps onto the FPGA, yielding at least 93× speedup over a Float16 datapath and at least 1.9× over related fixed-point accelerators.

Load-bearing premise

The load-bearing premise is that after re-quantizing all LayerNorm, Softmax and GELU inputs to one shared exponent and truncating mantissas to 2–5 bits, ImageNet top-1 accuracy stays within 1% of the floating-point model; the paper checks this only on DeiT Tiny, Small and Base, so a ViT with a wider internal dynamic range could break the accuracy budget.

Editorial extensions

If this is right

  • ViT inference can be performed end-to-end on the FPGA fabric, eliminating the CPU–accelerator communication overhead that prior designs pay for LayerNorm, Softmax and GELU.
  • The same 1%-accuracy budget shifts the design point to 2–5 mantissa bits, so area and memory scale down sharply; LUT entries for the non-linear operators drop at least 16× versus vanilla MXInt.
  • Because the results come from post-training quantization without fine-tuning, the accelerator can be deployed without GPU retraining, and the reported accuracy is a lower bound that fine-tuning could improve.
  • Larger DeiT models tolerate low-bit MXInt better than smaller ones, suggesting the approach becomes more attractive as model sizes grow.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same exponent-cancellation trick for LayerNorm and the two-bit Softmax remainder should transfer to LLM inference on MXInt hardware, where Softmax over longer sequences has a wider logit spread; the 2-bit remainder may need to grow with sequence length, a testable extension.
  • The block-size-equals-tile-size choice couples numerical grouping to the control path; decoupling them could shrink activations' shared-exponent blocks and reduce quantization error at the cost of regroup logic, a latency–accuracy trade the paper leaves unexplored.
  • The underutilized DSP blocks noted by the authors suggest that mapping MXInt operators onto hardened DSPs rather than LUTs is the next obvious lever; if synthesis closes the gap, the reported 93× speedup could rise further on the same FPGA.
  • A direct stress test would run the pipeline on DeiT-Large or on higher-resolution inputs; if the internal dynamic range widens, the right-shift range and mantissa widths would need re-sweeping, which would quantify how robust the 1% budget is.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes MXInt (microscaling integer) datapath optimizations for vision transformers (ViTs), claiming the first open-source FPGA accelerator that maps all ViT operations, including LayerNorm, GELU, and Softmax, onto the fabric. The authors use post-training quantization to the MXInt format and design custom integer-only hardware units for the non-linear operations, exploiting the shared-exponent property to reduce mantissa bitwidths to 2–5 bits while keeping ImageNet top-1 accuracy loss under 1% for DeiT Tiny, Small, and Base. System-level results from cycle-accurate simulation and Vivado reports are compared against a Float8 implementation of the same architecture, claiming speedups of 93×, 96×, and 171× on the three models, plus area savings in LUT entries of at least 16× for the approximated operators.

Significance. If the claims hold, the work is a useful contribution to FPGA-based ViT acceleration: it demonstrates that the shared exponents of MXInt can be exploited not just for linear operators but also for non-linear ones, yielding significantly narrower datapaths than existing fixed-point designs. The strength is the empirical design-space exploration of LUT bitwidths and domains, and the reporting of accuracy and hardware results side by side for the DeiT family. The LayerNorm algebraic extraction in Section III-B1 is clean (for positive shared exponents), and the accuracy tables (V and VI) largely support the bitwidth reduction story. However, the headline performance claims in the abstract and contributions are not all backed by the experiments: the speedup is measured against Float8, not Float16, and the claimed 1.9× speedup over related work is undermined by the paper's own 'unfair comparison' caveat.

major comments (4)
  1. [Abstract; Section I contribution 3; Section IV-2; Figure 10] The abstract and contribution list state 'at least 93× speedup compared to Float16', but the experiments in Section IV-2 compare against a Float8 implementation of the same architecture: Figure 10 explicitly says 'The red bars represent the same architecture in Float8', and the caption labels the baseline 'Float8'. No Float16 baseline is reported anywhere. The 93×/96×/171× numbers are therefore speedups over Float8, not Float16. This is a load-bearing claim in the abstract; either add a Float16 baseline or change the wording to 'compared to Float8' throughout.
  2. [Abstract; Section I contribution 3; Table VII; Section IV-3] The claim of 'at least 1.9× speedup compared to related work' is not supported by the reported data. Table VII explicitly states that comparing with related work 'is unfair due to differences in hardware microarchitectures ... and algorithm optimizations' and the systems run on different FPGAs (ZCU102 vs U250) with different pruning and fine-tuning. The 1.9× figure appears to come from the DeiT Base row (66.06 FPS vs 34.00 FPS for Auto-ViT-Acc), but DeiT Tiny shows Ours at 589.44 FPS vs 616.10 FPS for [9], i.e., slower, and no controlled or normalized comparison is provided. Please either perform a fair comparison (same FPGA, same mapping strategy, same quantization/fine-tuning status) or remove the '1.9× speedup compared to related work' claim.
  3. [Section III-B1, Eq. (3); Figure 3] The LayerNorm re-quantization in Eq. (3) is implemented in hardware as 'exponent alignment (>> up to 6 bits)' (Figure 3), meaning any element whose exponent differs from the block maximum xmax_e by more than 6 bits is truncated to zero. This is an unstated dynamic-range bound on the LayerNorm inputs. The paper never reports the distribution of (xmax_e - xe) for the evaluated models, nor does it sweep this shift limit in the design-space explorations of Figures 4, 7, or 9. Consequently the claim of mapping 'all operations of the ViT models' onto fabric with under 1% accuracy loss is only validated for the DeiT family; a ViT with a wider internal exponent spread (e.g., Swin, a differently trained DeiT, or a larger model) could violate the accuracy budget. Please measure the exponent spread on the DeiT models and either justify the 6-bit bound or add a sensitivity study.
  4. [Abstract; Section VI] The paper describes the contribution as 'the first open-source ViT accelerator', but no repository URL, source code release, or artifact availability statement appears anywhere in the manuscript. Without a public artifact, the 'open-source' claim cannot be verified. Please provide the repository link in the camera-ready version, or remove the word 'open-source' from the abstract and conclusions.
minor comments (5)
  1. [Table VI] The row labeled 'Float32' lists accuracies as 79.83, 72.13, 81.80 under the column headers 'DeiT Tiny, DeiT Small, DeiT Base'. The correct ordering should be 72.13, 79.83, 81.80; either the column headers or the row values are swapped. Please verify all rows of the table.
  2. [Section III-B1, paragraph after Eq. (2)] The sentence 'When computing Softmax, we are effectively dealing with MXInt values coming from different blocks' appears in the LayerNorm subsection and seems to be a copy-paste error; the surrounding text and Figure 3 describe LayerNorm re-quantization, not Softmax.
  3. [Section III-B1, Eq. (3) and surrounding text] The text writes 'λ = xmax_e' but the preceding equation defines x = 2^{xmax_e} xm', so λ should be the scale factor 2^{xmax_e}, not the exponent value xmax_e. The notation is inconsistent and should be fixed.
  4. [Figure 10] The caption labels the bars 'Float8 Ours', while the body text says the green bars represent 'our work without and with inter-layer pipelining'. The figure should clearly distinguish the two green-bar variants, perhaps with different hatching or labels.
  5. [References, [10]] Reference [10] is cited as the source of the DeiT family, but it points to 'DeiT III: Revenge of the ViT', which is a training/distillation recipe, not the original DeiT model family. Please cite the original DeiT paper (Touvron et al., 'Training data-efficient image transformers & distillation through attention') or clarify why DeiT III is the appropriate reference for the pretrained models used.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: the MXInt datapath equations are self-contained re-expressions, and the only self-citation is motivational rather than load-bearing.

full rationale

The paper's derivation chain is not circular in any load-bearing sense. The LayerNorm optimization (Equations (3)-(7)) is a representation transform: values are re-quantized to a shared exponent, and the scale factor cancels in the normalized expression, so the integer-only mantissa datapath is a mathematical rearrangement of the original operator rather than a conclusion fed back as an input. The GELU and Softmax LUT designs (Figures 4, 7, 8, 9) are tuned by sweeping bitwidths and domains against measured ImageNet accuracy; the resulting 'within 1% accuracy loss' is a selection constraint and an evaluated outcome, not a quantity predicted from the same fitted parameters, so it is not a fitted input masquerading as a prediction. Speedups are obtained from cycle-accurate simulation and Vivado implementation reports of the authors' own architecture, not from a self-referential identity. The only self-citation is [7], which is used to motivate the 1% accuracy budget and to note that MXInt is accurate for LLM quantization; this is not load-bearing because the ViT-specific accuracy is verified independently in Figure 1b and Tables V-VI, and the central hardware claims stand on the paper's own measurements. No uniqueness theorem is imported from the authors, and the llama.cpp Softmax decomposition is explicitly attributed to [24] rather than relabeled as a novel first-principles result. Overall, no equation or hardware claim reduces to its own inputs; the score reflects one minor, non-load-bearing self-citation.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

All free parameters are hardware design choices tuned against the external ImageNet accuracy metric, not derived analytically. No invented entities are introduced: MXInt is prior work [6,7,8,34]. The main assumptions are algebraic identities and domain assumptions about requantized MXInt tensors, validated only on the DeiT family.

free parameters (7)
  • LUT1/sqrt mantissa bitwidth (xv_m) = 4 bits by Figure 4; 5 bits in final LayerNorm datapath
    Chosen by design-space sweep to stay within 1% ImageNet accuracy loss on DeiT models.
  • LUTGELU domain boundary a = a = 3
    Selected from Figure 7 sweep; inputs outside [-3,3] are treated as ReLU or zero.
  • LUTGELU mantissa bitwidth = 4 bits in exploration; 5 bits in final design
    Selected from Figure 8 sweep under the 1% accuracy budget.
  • LUTpow2 r bitwidth for Softmax = 2 bits
    Selected from Figure 9 sweep; final Softmax mantissa width on fabric is 2 bits.
  • Accumulator mantissa width for linear operators = 12 bits
    Empirically set in Section III-A to make the accumulator additions lossless for DeiT models.
  • MXInt block sizes = 16 for activations, 256 for weights
    Chosen to match tile sizes and avoid regroup logic (Section III-A); directly affects quantization accuracy and hardware area.
  • Per-model mantissa precision from greedy quantization search = MXInt8/MXInt8 (Tiny), MXInt8/MXInt6 or MXInt6/MXInt8 (Small), etc., as in Table V
    Greedy PTQ search determines the smallest mantissa width per model that keeps accuracy loss under 1%.
assumptions (6)
  • ad hoc to paper epsilon in LayerNorm is set to zero so sqrt(Var + eps) becomes sqrt(Var) and the shared exponent can be pulled out of the square root.
    Used in Section III-B1, Equations (4)-(6). This is a numerical approximation; standard LayerNorm uses a small epsilon (typically 1e-5), and the paper does not quantify its effect across architectures.
  • domain assumption All values entering a nonlinear operator can be re-quantized to a single shared exponent xmax_e without losing the accuracy needed for the 1% budget.
    Used in Equation (3) and the requantization step in Figure 3. Verified only on DeiT/ImageNet; no formal bound on precision loss from the dynamic right shift.
  • domain assumption For GELU inputs, the exponent of the output equals the exponent of the input, so the exponent can be forwarded without recomputation.
    Used in Figure 6 and Section III-B2. Holds when the function is close to identity or zero outside the LUT region, which is checked by sweeps but not proven for other models.
  • domain assumption The base-2 decomposition exp(x) = 2^n * 2^r with r quantized to 2 bits is accurate enough for Softmax in MXInt.
    Used in Equations (14)-(19). Adopted from llama.cpp [24] and validated only on DeiT.
  • standard math Standard identities 2^{a+b} = 2^a 2^b and (2^a)^b = 2^{ab}, plus LUT interpolation, are valid.
    Used throughout the datapath derivations; these are uncontroversial.
  • domain assumption MXInt block formats with shared exponents represent ViT tensors accurately at the chosen block sizes (16 for activations, 256 for weights).
    Borrowed from prior MXInt work [6,7,8] and independently checked in Table V only for DeiT.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Refining Datapath for Microscaling ViTs." pith.science (2026). https://pith.science/paper/Z6CPUH35

@misc{pith2026250522194,
  author       = {Pith},
  title        = {Pith review of: Refining Datapath for Microscaling ViTs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z6CPUH35}},
  note         = {Machine review of arXiv:2505.22194}
}
abstract

Vision Transformers (ViTs) leverage the transformer architecture to effectively capture global context, demonstrating strong performance in computer vision tasks. A major challenge in ViT hardware acceleration is that the model family contains complex arithmetic operations that are sensitive to model accuracy, such as the Softmax and LayerNorm operations, which cannot be mapped onto efficient hardware with low precision. Existing methods only exploit parallelism in the matrix multiplication operations of the model on hardware and keep these complex operations on the CPU. This results in suboptimal performance due to the communication overhead between the CPU and accelerator. Can new data formats solve this problem? In this work, we present the first ViT accelerator that maps all operations of the ViT models onto FPGAs. We exploit a new arithmetic format named Microscaling Integer (MXInt) for datapath designs and evaluate how different design choices can be made to trade off accuracy, hardware performance, and hardware utilization. Our contributions are twofold. First, we quantize ViTs using the MXInt format, achieving both high area efficiency and accuracy. Second, we propose MXInt-specific hardware optimization that map these complex arithmetic operations into custom hardware. Within 1\% accuracy loss, our method achieves at least 93$\times$ speedup compared to Float16 and at least 1.9$\times$ speedup compared to related work.

Figures

Figures reproduced from arXiv: 2505.22194 by the authors.

Figure 1
Figure 1. Motivating example: dataflow hardware acceleration of a ViT in MXInt. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Comparison between the standard floating-point format and MXInt on the dot product operator. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Optimized datapath for MXInt LayerNorm. 1) LayerNorm Approximation: LayerNorm operations have been widely used in transformers, including ViTs, which scale values in a tensor to a fixed range [19]. The standard expression of the LayerNorm operation is presented as follows. y = x − E(x) p V ar(x) + ϵ γ + β (1) x and y denote the input and output tensors, and E(.) computes the expectation and V ar(.) computes the vari… view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: The LUT domain covers the non-linear region of the [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 7
Figure 7. Figure 7: Accuracy loss over different LUT domains for GELU. [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]
Figure 8
Figure 8. Figure 8: Accuracy loss over different bitwidths for GELU. LUT [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 10
Figure 10. Figure 10: Speedups achieved by our work (shown in green). high model accuracy. This leads to an efficient datapath design for the exponential function in MXInt. Table IV compares our design with related work. We present an intermediate design point to compare with the state-of-…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 41 canonical work pages

  1. [9]

    An integer- only and group-vector systolic accelerator for efficiently mapping vision transformer on edge,

    M. Huang, J. Luo, C. Ding, Z. Wei, S. Huang, and H. Yu, “An integer- only and group-vector systolic accelerator for efficiently mapping vision transformer on edge,” IEEE Transactions on Circuits and Systems I: Regular Papers, 2023

    work page 2023

  2. [1]

    Flightllm: Efficient large language model inference with a complete mapping flow on fpga,

    S. Zeng, J. Liu, G. Dai, X. Yang, T. Fu, H. Wang, W. Ma, H. Sun, S. Li, Z. Huang, et al. , “Flightllm: Efficient large language model inference with a complete mapping flow on fpga,” arXiv preprint arXiv:2401.03868, 2024

    arXiv 2024

  3. [2]

    Heatvit: Hardware-efficient adaptive token pruning for vision transformers,

    P. Dong, M. Sun, A. Lu, Y . Xie, K. Liu, Z. Kong, X. Meng, Z. Li, X. Lin, Z. Fang, et al., “Heatvit: Hardware-efficient adaptive token pruning for vision transformers,” in 2023 IEEE International Symposium on High- Performance Computer Architecture (HPCA), pp. 442–455, IEEE, 2023

    work page 2023

  4. [3]

    Auto-vit-acc: An fpga-aware automatic acceleration framework for vision transformer with mixed-scheme quantization,

    Z. Li, M. Sun, A. Lu, H. Ma, G. Yuan, Y . Xie, H. Tang, Y . Li, M. Leeser, Z. Wang, et al. , “Auto-vit-acc: An fpga-aware automatic acceleration framework for vision transformer with mixed-scheme quantization,” in 2022 32nd International Conference on Field-Programmable Logic and Applications (FPL), pp. 109–116, IEEE, 2022

    work page 2022

  5. [4]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    D. Alexey, “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv: 2010.11929 , 2020

    arXiv 2010

  6. [5]

    Sda: Low-bit stable diffusion acceleration on edge fpgas,

    G. Yang, Y . Xie, Z. J. Xue, S.-E. Chang, Y . Li, P. Dong, J. Lei, W. Xie, Y . Wang, X. Lin, et al. , “Sda: Low-bit stable diffusion acceleration on edge fpgas,” in 2024 34th International Conference on Field- Programmable Logic and Applications (FPL), pp. 264–273, IEEE, 2024

    work page 2024

  7. [6]

    Pushing the limits of narrow precision inferencing at cloud scale with microsoft floating point,

    B. Darvish Rouhani, D. Lo, R. Zhao, M. Liu, J. Fowers, K. Ovtcharov, A. Vinogradsky, S. Massengill, L. Yang, R. Bittner, et al. , “Pushing the limits of narrow precision inferencing at cloud scale with microsoft floating point,” Advances in neural information processing systems , vol. 33, pp. 10271–10281, 2020

    work page 2020

  8. [7]

    Revisiting block-based quantisation: What is important for sub-8-bit llm inference?,

    C. Zhang, J. Cheng, I. Shumailov, G. A. Constantinides, and Y . Zhao, “Revisiting block-based quantisation: What is important for sub-8-bit llm inference?,” 2023

    work page 2023

Show all 50 references
  1. [8]

    Ex- ploring fpga designs for mx and beyond,

    E. Samson, N. Mellempudi, W. Luk, and G. A. Constantinides, “Ex- ploring fpga designs for mx and beyond,” in 2024 34th International Conference on Field-Programmable Logic and Applications (FPL) , pp. 304–310, IEEE, 2024

    2024

  2. [10]

    Deit iii: Revenge of the vit,

    H. Touvron, M. Cord, and H. J ´egou, “Deit iii: Revenge of the vit,” in European conference on computer vision , pp. 516–533, Springer, 2022

    2022

  3. [11]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition , pp. 248–255, Ieee, 2009

    2009

  4. [12]

    Ieee standard 754 for binary floating-point arithmetic,

    W. Kahan, “Ieee standard 754 for binary floating-point arithmetic,” Lecture Notes on the Status of IEEE , vol. 754, no. 94720-1776, p. 11, 1996

    1996

  5. [13]

    Understanding the potential of fpga-based spatial accel- eration for large language model inference,

    H. Chen, J. Zhang, Y . Du, S. Xiang, Z. Yue, N. Zhang, Y . Cai, and Z. Zhang, “Understanding the potential of fpga-based spatial accel- eration for large language model inference,” ACM Transactions on Reconfigurable Technology and Systems , 2024

    2024

  6. [14]

    fpgaconvnet: A framework for map- ping convolutional neural networks on fpgas,

    S. I. Venieris and C.-S. Bouganis, “fpgaconvnet: A framework for map- ping convolutional neural networks on fpgas,” in2016 IEEE 24th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), pp. 40–47, IEEE, 2016

    2016

  7. [15]

    Finn: A framework for fast, scalable binarized neural network inference,

    Y . Umuroglu, N. J. Fraser, G. Gambardella, M. Blott, P. Leong, M. Jahre, and K. Vissers, “Finn: A framework for fast, scalable binarized neural network inference,” in Proceedings of the 2017 ACM/SIGDA interna- tional symposium on field-programmable gate arrays , pp. 65–74, 2017

    2017

  8. [16]

    Hida: A hierarchical dataflow compiler for high-level synthesis,

    H. Ye, H. Jun, and D. Chen, “Hida: A hierarchical dataflow compiler for high-level synthesis,” arXiv preprint arXiv:2311.03379 , 2023

    2023 arXiv

  9. [17]

    Automating constraint- aware datapath optimization using e-graphs,

    S. Coward, G. A. Constantinides, and T. Drane, “Automating constraint- aware datapath optimization using e-graphs,” in 2023 60th ACM/IEEE Design Automation Conference (DAC) , pp. 1–6, IEEE, 2023

    2023

  10. [18]

    Online alignment and ad- dition in multiterm floating-point adders,

    K. Alexandridis and G. Dimitrakopoulos, “Online alignment and ad- dition in multiterm floating-point adders,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems , 2024

    2024

  11. [19]

    PyTorch LayerNorm, 2025

    2025

  12. [22]

    PyTorch Softmax, 2025

    2025

  13. [23]

    I-vit: Integer-only quantization for efficient vision transformer inference,

    Z. Li and Q. Gu, “I-vit: Integer-only quantization for efficient vision transformer inference,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , pp. 17065–17075, 2023

    2023

  14. [25]

    Packqvit: Faster sub-8-bit vision transformers via full and packed quantization on the mobile,

    P. Dong, L. Lu, C. Wu, C. Lyu, G. Yuan, H. Tang, and Y . Wang, “Packqvit: Faster sub-8-bit vision transformers via full and packed quantization on the mobile,” Advances in Neural Information Processing Systems, vol. 36, pp. 9015–9028, 2023

    2023

  15. [26]

    Pytorch Image Models, 2023

    2023

  16. [27]

    AMD Alveo V80 Compute Accelerator, 2025

    2025

  17. [28]

    Towards accurate binary convolutional neural network,

    X. Lin, C. Zhao, and W. Pan, “Towards accurate binary convolutional neural network,” 2017

    2017

  18. [29]

    Lq-nets: Learned quantization for highly accurate and compact deep neural networks,

    D. Zhang, J. Yang, D. Ye, and G. Hua, “Lq-nets: Learned quantization for highly accurate and compact deep neural networks,” 2018

    2018

  19. [30]

    Training and inference with integers in deep neural networks,

    S. Wu, G. Li, F. Chen, and L. Shi, “Training and inference with integers in deep neural networks,” arXiv preprint arXiv:1802.04680 , 2018

    2018 arXiv

  20. [31]

    Quantizing deep convolutional networks for effi- cient inference: A whitepaper,

    R. Krishnamoorthi, “Quantizing deep convolutional networks for effi- cient inference: A whitepaper,” arXiv preprint arXiv:1806.08342, 2018

    2018 arXiv

  21. [32]

    Vs-quant: Per-vector scaled quantization for accurate low-precision neural network inference,

    S. Dai, R. Venkatesan, M. Ren, B. Zimmer, W. Dally, and B. Khailany, “Vs-quant: Per-vector scaled quantization for accurate low-precision neural network inference,” Proceedings of Machine Learning and Sys- tems, vol. 3, pp. 873–884, 2021

    2021

  22. [33]

    Accuracy boosters: Epoch-driven mixed-mantissa block floating-point for dnn training,

    S. B. Harma, C. S ¨onmez, B. Falsafi, M. Jaggi, and Y . Oh, “Accuracy boosters: Epoch-driven mixed-mantissa block floating-point for dnn training,” arXiv preprint arXiv:2211.10737 , 2022

    2022 arXiv

  23. [34]

    With shared microexponents, a little shifting goes a long way,

    B. Darvish Rouhani, R. Zhao, V . Elango, R. Shafipour, M. Hall, M. Mesmakhosroshahi, A. More, L. Melnick, M. Golub, G. Varatkar, et al., “With shared microexponents, a little shifting goes a long way,” in Proceedings of the 50th Annual International Symposium on Computer Archi...

    2023

  24. [35]

    Going further with winograd convolutions: Tap-wise quantization for efficient inference on 4x4 tiles,

    R. Andri, B. Bussolino, A. Cipolletta, L. Cavigelli, and Z. Wang, “Going further with winograd convolutions: Tap-wise quantization for efficient inference on 4x4 tiles,” in 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO) , pp. 582–598, IEEE, 2022

    2022

  25. [36]

    Drq: dynamic region-based quantization for deep neural network ac- celeration,

    Z. Song, B. Fu, F. Wu, Z. Jiang, L. Jiang, N. Jing, and X. Liang, “Drq: dynamic region-based quantization for deep neural network ac- celeration,” in 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA) , pp. 1010–1021, IEEE, 2020

    2020

  26. [37]

    Mokey: Enabling narrow fixed-point inference for out-of-the-box floating-point transformer models,

    A. H. Zadeh, M. Mahmoud, A. Abdelhadi, and A. Moshovos, “Mokey: Enabling narrow fixed-point inference for out-of-the-box floating-point transformer models,” in Proceedings of the 49th Annual International Symposium on Computer Architecture , pp. 888–901, 2022

    2022

  27. [38]

    Cambricon-q: A hybrid architecture for efficient training,

    Y . Zhao, C. Liu, Z. Du, Q. Guo, X. Hu, Y . Zhuang, Z. Zhang, X. Song, W. Li, X. Zhang, et al., “Cambricon-q: A hybrid architecture for efficient training,” in 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA), pp. 706–719, IEEE, 2021

    2021

  28. [39]

    Llm. int8 (): 8-bit matrix multiplication for transformers at scale,

    T. Dettmers, M. Lewis, Y . Belkada, and L. Zettlemoyer, “Llm. int8 (): 8-bit matrix multiplication for transformers at scale,” arXiv preprint arXiv:2208.07339, 2022

    2022 arXiv

  29. [40]

    Gptq: Accurate post-training quantization for generative pre-trained transformers,

    E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh, “Gptq: Accurate post-training quantization for generative pre-trained transformers,” arXiv preprint arXiv:2210.17323, 2022

    2022 arXiv

  30. [41]

    Smoothquant: Accurate and efficient post-training quantization for large language models,

    G. Xiao, J. Lin, M. Seznec, J. Demouth, and S. Han, “Smoothquant: Accurate and efficient post-training quantization for large language models,” arXiv preprint arXiv:2211.10438 , 2022

    2022 arXiv

  31. [42]

    Zeroquant: Efficient and affordable post-training quantization for large- scale transformers,

    Z. Yao, R. Yazdani Aminabadi, M. Zhang, X. Wu, C. Li, and Y . He, “Zeroquant: Efficient and affordable post-training quantization for large- scale transformers,” Advances in Neural Information Processing Sys- tems, vol. 35, pp. 27168–27183, 2022

    2022

  32. [43]

    Psq: An automatic search framework for data-free quantization on pim-based architecture,

    F. Liu, N. Yang, and L. Jiang, “Psq: An automatic search framework for data-free quantization on pim-based architecture,” in 2023 IEEE 41st International Conference on Computer Design (ICCD) , pp. 507–514, IEEE, 2023

    2023

  33. [44]

    Spark: Scalable and precision-aware acceleration of neural networks via efficient encoding,

    F. Liu, N. Yang, H. Li, Z. Wang, Z. Song, S. Pei, and L. Jiang, “Spark: Scalable and precision-aware acceleration of neural networks via efficient encoding,” in 2024 IEEE International Symposium on High- Performance Computer Architecture (HPCA) , pp. 1029–1042, IEEE, 2024

    2024

  34. [45]

    Msd: Mixing signed digit representations for hardware-efficient dnn accelera- tion on fpga with heterogeneous resources,

    J. Wu, J. Zhou, Y . Gao, Y . Ding, N. Wong, and H. K.-H. So, “Msd: Mixing signed digit representations for hardware-efficient dnn accelera- tion on fpga with heterogeneous resources,” in 2023 IEEE 31st Annual International Symposium on Field-Programmable Custom Computing Machi...

    2023

  35. [46]

    Adaptable butterfly accelerator for attention-based nns via hardware and algorithm co-design,

    H. Fan, T. Chau, S. I. Venieris, R. Lee, A. Kouris, W. Luk, N. D. Lane, and M. S. Abdelfattah, “Adaptable butterfly accelerator for attention-based nns via hardware and algorithm co-design,” in 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO) , pp. 599–6...

    2022

  36. [47]

    Aˆ 3: Accelerating attention mechanisms in neural networks with approximation,

    T. J. Ham, S. J. Jung, S. Kim, Y . H. Oh, Y . Park, Y . Song, J.-H. Park, S. Lee, K. Park, J. W. Lee, et al. , “Aˆ 3: Accelerating attention mechanisms in neural networks with approximation,” in 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA)...

    2020

  37. [48]

    Dfx: A low-latency multi-fpga appliance for accelerating transformer-based text generation,

    S. Hong, S. Moon, J. Kim, S. Lee, M. Kim, D. Lee, and J.-Y . Kim, “Dfx: A low-latency multi-fpga appliance for accelerating transformer-based text generation,” in 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO), pp. 616–630, IEEE, 2022

    2022

  38. [49]

    Flat: An optimized dataflow for mitigating attention bottlenecks,

    S.-C. Kao, S. Subramanian, G. Agrawal, A. Yazdanbakhsh, and T. Kr- ishna, “Flat: An optimized dataflow for mitigating attention bottlenecks,” in Proceedings of the 28th ACM International Conference on Archi- tectural Support for Programming Languages and Operating Systems, Vol...

    2023

  39. [50]

    Sanger: A co-design framework for enabling sparse attention using reconfigurable architecture,

    L. Lu, Y . Jin, H. Bi, Z. Luo, P. Li, T. Wang, and Y . Liang, “Sanger: A co-design framework for enabling sparse attention using reconfigurable architecture,” in MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture, pp. 977–991, 2021

    2021

  40. [51]

    Gobo: Quan- tizing attention-based nlp models for low latency and energy efficient inference,

    A. H. Zadeh, I. Edo, O. M. Awad, and A. Moshovos, “Gobo: Quan- tizing attention-based nlp models for low latency and energy efficient inference,” in 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pp. 811–824, IEEE, 2020

    2020

  41. [52]

    Edgebert: Sentence-level energy optimizations for latency-aware multi- task nlp inference,

    T. Tambe, C. Hooper, L. Pentecost, T. Jia, E.-Y . Yang, M. Donato, V . Sanh, P. Whatmough, A. M. Rush, D. Brooks, and G.-Y . Wei, “Edgebert: Sentence-level energy optimizations for latency-aware multi- task nlp inference,” in MICRO-54: 54th Annual IEEE/ACM International Sympos...

    2021

  42. [53]

    Fact: Ffn-attention co-optimized transformer architecture with eager correlation prediction,

    Y . Qin, Y . Wang, D. Deng, Z. Zhao, X. Yang, L. Liu, S. Wei, Y . Hu, and S. Yin, “Fact: Ffn-attention co-optimized transformer architecture with eager correlation prediction,” in Proceedings of the 50th Annual International Symposium on Computer Architecture , pp. 1–14, 2023

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.