FastMamba: A High-Speed and Efficient Mamba Accelerator on FPGA with Accurate Quantization

Aotao Wang; Haikuo Shao; Shaobo Ma; Zhongfeng Wang

REVIEW 4 major objections 5 minor 1 cited by

FastMamba: A High-Speed and Efficient Mamba Accelerator on FPGA with Accurate Quantization

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Hadamard outlier suppression lets an FPGA run Mamba2 prefill 8.9x faster than an RTX 3090.

desk verdict A solid but under-verified FPGA accelerator for Mamba2—good quantization results on the 130M model, but an energy-efficiency discrepancy and an algorithm/hardware scale mismatch need fixing before the headline numbers are trusted. read the letter →

arxiv 2505.18975 v4 pith:RG2KBGKT submitted 2025-05-25 cs.AR cs.AI

Aotao Wang , Haikuo Shao , Shaobo Ma , Zhongfeng Wang This is my paper

classification cs.ARcs.AI

keywords MambaMamba2StateSpaceModelsFPGAacceleration8-bitquantizationHadamardtransformationpower-of-twoalgorithm-hardwareco-design

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FastMamba aims to show that Mamba2, a state-space language model, can be deployed efficiently on FPGAs without losing accuracy. It combines an 8-bit quantization scheme that uses a Hadamard transform to spread out outlier activation values, a power-of-two fixed-point format for the SSM block and convolution layer, and first-order linear approximations for the exponential and SoftPlus functions. On the Xilinx VC709 FPGA, the design reports a maximum 68.80x speedup over an Intel Xeon 4210R CPU and 8.90x over an NVIDIA RTX 3090 during prompt prefill, the context-ingestion pass, on Mamba2-130M, and 1.65x better energy efficiency than the RTX 3090 when decoding Mamba2-2.7B. If these results hold, resource-constrained edge devices could run state-space models locally instead of relying on cloud GPUs.

What carries the argument

The load-bearing identity is the Hadamard factorization $Y=(XH)(H^T W^T)$, where the Hadamard matrix, normalized by $\sqrt{n}$, is orthonormal, so the rotation concentrates outlier energy and gives 8-bit quantization a narrower dynamic range. The SSM block and convolution use fine-grained power-of-two scaling factors $2^p$, turning multiplications into shifts, and the nonlinearities are reduced to an 8-segment first-order linear approximation of $2^v$ built from $e^x = 2^{x\log_2 e}$ and the symmetry $\mathrm{SoftPlus}(x)=x+\mathrm{SoftPlus}(-x)$. On the hardware side, five vector processing units (parallel adder, parallel multiplier, parallel multiply-add, Hadamard adder tree, and multiplier adder tree) execute these fixed-point operations in a pipelined dataflow.

What would settle it

Run the same Hadamard-based W8A8 quantization on Mamba2-2.7B and compare Lambada perplexity and zero-shot accuracy to FP16; a degradation well above the roughly one percentage point seen at 130M, or post-Hadamard activations that still fall outside the 8-bit clipping range, would break the central claim.

Watch

Extended reading notes

Core claim

The paper claims that Mamba2's three deployment obstacles—outlier-heavy linear-layer activations, irregular element-wise SSM operations, and hardware-unfriendly nonlinearities—can all be removed algorithmically. Hadamard rotations spread activation and weight outliers before 8-bit quantization, power-of-two scaling makes SSM and convolution arithmetic shift-dominated, and exp and SoftPlus collapse into one linear-approximation unit. The implemented FPGA reaches a maximum 68.80x prefill speedup over the CPU and 8.90x over the RTX 3090 on Mamba2-130M, and a 1.65x energy-efficiency gain over the RTX 3090 when decoding Mamba2-2.7B, with reported accuracy degradation within about one percentage point on the 130M model.

Load-bearing premise

The design assumes that the method that keeps accuracy loss under about one percent on the small 130M model will also keep accuracy acceptable on the 2.7B model used in the decode tests, but no accuracy measurements are reported at that scale.

Editorial extensions

If this is right

The Hadamard-based W8A8 linear quantizer beats both normal quantization and SmoothQuant on perplexity and seven zero-shot tasks for Mamba2-130M, so outlier rotation rather than per-channel smoothing carries the accuracy.
Power-of-two quantization plus linearized nonlinearities reduces the SSM block to fixed-point shifts, adds, and matrix products that map directly onto vector processing units.
The pipelined FPGA reaches a maximum 68.80x prefill speedup over the Intel Xeon 4210R CPU and 8.90x over the NVIDIA RTX 3090, with average speedups of 55.70x and 6.06x at typical evaluation lengths.
The shared Nonlinear Approximation Unit saves 56% of DSP resources and 49% of flip-flop resources compared with an FP16 nonlinear unit.
The decode-stage comparison on Mamba2-2.7B reports 1.65x energy efficiency over the RTX 3090, but the accuracy of the quantized 2.7B model is not measured in the paper.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the Hadamard-based quantization holds up at larger scale, the same outlier-rotation trick should transfer to other state-space and attention architectures whose activation tensors have extreme channels; the paper only demonstrates it on the 130M model.
The energy-efficiency result covers decode only; a prefill comparison at 2.7B would require either more on-chip memory or a sequence-chunked schedule, so the measured 1.65x advantage may not carry over to that stage.
A direct testable extension is to record the post-Hadamard activation ranges at every linear layer of Mamba2-2.7B; if any remain outside the 8-bit clipping range, the accuracy-degradation assumption breaks before any hardware measurement.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick verdict: FastMamba is a real FPGA implementation of Mamba2 with a sensible quantization stack, and the accuracy tables suggest the quantization actually works. But the version you showed me has a few loose ends that need attention before I'd trust the headline numbers: an abstract/body inconsistency on energy efficiency, and a mismatch between the algorithm's input-dependent quantization scales and the apparent static scales in the hardware.

What's good: the paper adapts QuaRot-style Hadamard rotation to Mamba2's linear layers, adds power-of-two quantization for the SSM and convolution, and shares a single approximation unit for exp and SoftPlus. That's a coherent co-design, not just a repackaging of known tricks. The accuracy results on Mamba2-130M are genuinely decent: FastMamba-LQ matches FP16 average accuracy (42.6 vs 42.6), and the full quantized model drops only to 42.2 with PPL 17.9 vs 16.9. For an 8-bit linear + PoT SSM that's a good outcome. The nonlinear approximation unit saves 56% DSP and 49% FF versus an FP16 unit, which is a concrete hardware win. The implementation uses real resources (77% LUT, 92% DSP) and the throughput/energy numbers are reported on actual hardware.

Now the soft spots. First, the abstract in the arXiv listing says 6x energy efficiency over RTX 3090, but the body consistently says 1.65x. That's a serious discrepancy; it needs to be fixed and explained. Second, Algorithm 1 computes quantization scales from the max over the input (a two-pass, input-dependent operation), while the Hadamard linear module in Fig. 6 quantizes with a multiplier and shifter and has no max-reduction unit. The paper never says whether the hardware uses static scales. If it does, the reported accuracy may not hold for inputs outside the calibration set. The accuracy evaluation doesn't state which version was measured. This isn't necessarily fatal, but it's a gap that needs to be closed. Third, the speedup comparisons to CPU and GPU don't tell us what software optimizations were used (e.g., thread count, batch size, library), and there's no comparison to prior Mamba accelerators like MARCA, which makes it hard to place the 8.9x speedup. Fourth, the 2.7B decode measurements have no accuracy validation at that scale, so we're asked to trust that the quantization transfers from 130M.

Overall: the core idea is sound, and the accuracy evidence from the 130M model is the strongest part. The hardware implementation is real and resource reporting is transparent. The issues are about completeness and verification, not about a fundamentally broken approach. I'd send this to peer review, but I'd want the authors to reconcile abstract and body, explain the scale computation in hardware, and add at least one comparison with an existing Mamba accelerator.

Referee Report

4 major / 5 minor

Summary. FastMamba presents a hardware-algorithm co-design for deploying Mamba2 on FPGA. The algorithm contributions are Hadamard-transform-based 8-bit quantization for linear layers, power-of-two quantization for convolution and SSM blocks, and first-order linear approximation of exponential and SoftPlus. The hardware contributions are vector processing units, a Hadamard-based linear module, an SSM module with a nonlinear approximation unit, and a pipelined dataflow. Experiments on Xilinx VC709 report that W8A8 quantization of Mamba2-130M stays within about 1% average accuracy of FP16, prefill speedups up to 68.80x over a Xeon CPU and 8.90x over an RTX 3090, and 1.65x energy efficiency over the RTX 3090 for decode on Mamba2-2.7B.

Significance. The problem is timely and the paper contains useful engineering: it gives a full resource-utilization breakdown, quantifies DSP/FF savings of the nonlinear unit, and evaluates against two standard platforms. If the accuracy results are reproducible on the actual hardware, the Hadamard-based quantization plus custom datapath is a credible approach to SSM inference at the edge. However, the central claims currently rest on an unresolved mismatch between the dynamic quantization algorithm and the static-looking hardware, and on accuracy evidence only at the smaller model scale.

major comments (4)

[III-A, IV-B] Algorithm 1 (lines 7-8) computes the activation scale sX from the maximum of the Hadamard-transformed activations XH, which is an input-dependent, two-pass procedure, and the hardware in Fig. 6 and Section IV-B uses fixed coefficients (x scoe, >> sshift) with no max-reduction or scale-computation unit. The manuscript never states whether the implemented module uses dynamic or static scales. If static, the Table II accuracy results (which appear to use dynamic scales) do not validate the FPGA implementation, because activations outside the calibration range would be clipped in 8 bits. Please either describe the hardware dynamic-scale path, or report accuracy and quantization-error statistics using the same static-scale procedure as the hardware, including the calibration set and clipping behavior.
[V.A, V.C.2] The accuracy evaluation (Table II) is performed only on Mamba2-130M, while the decode-stage throughput and energy-efficiency results (Table III, Section V.C.2) use Mamba2-2.7B. The paper assumes that the Hadamard 8-bit linear quantization and the PoT quantization of the SSM block preserve accuracy at 2.7B scale, but no perplexity/zero-shot accuracy or outlier statistics are reported for that model. Since the energy-efficiency claim is for the 2.7B configuration, provide accuracy results (or at least an outlier-range analysis) for Mamba2-2.7B under the exact quantization scheme used by the hardware.
[V.C.1, Table III] The speedup comparison in Section V.C.1 reports maximum 68.80x vs CPU and 8.90x vs GPU but does not describe the CPU/GPU software configuration: framework, batch size, sequence length, thread count, FP16/int8 kernels, or whether the comparison uses optimized prefill/decode kernels. The paper also does not compare against any existing Mamba accelerator, despite citing Marca [9] in the introduction. Without these details the speedup numbers are difficult to interpret. Provide the exact evaluation protocol and, ideally, a comparison with prior FPGA/ASIC Mamba accelerators on the same workload.
[Table III, V.C.2] Energy efficiency is reported as token/(s·W) without any description of how power is measured (e.g., board-level power meter, vendor tools, or model estimates) or which components are included. The GPU and FPGA run at different frequencies and use different memory systems, so the 1.65x energy-efficiency ratio needs a stated measurement methodology to be credible.

minor comments (5)

[Abstract, V.C.2] The abstract states a 6x energy-efficiency improvement in the decode experiment, while Section V.C.2, Table III, and the conclusion report 1.65x. Please make these numbers consistent.
[Algorithm 1] Line 4, 'FindHadamard(X[i])', suggests the Hadamard matrix is derived from data; since H depends only on the dimension, rename to something like 'FindHadamard(dim(X[i]))' to avoid confusion.
[Table III] The Throughput row shows GPU 111 token/s and FastMamba 5.68 token/s, yet the text emphasizes higher energy efficiency for FastMamba. State explicitly that the throughput is for decode of Mamba2-2.7B and clarify the workload and batch size so readers can understand the throughput/energy trade-off.
[Fig. 10, IV-D] The 'Half Float Nonlinear Unit' in Fig. 10 is not defined in the text; specify whether it is an FP16 reference implementation and how its resource numbers were obtained.
[III-B.2] Equation (5) states SoftPlus(x) ≈ e^x without specifying the domain; the subsequent symmetry argument implies x ≤ 0. Add the domain and quantify the approximation error of the first-order linear method.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central claims are validated by external FP16 and CPU/GPU baselines, with no load-bearing self-citation or definitional reduction.

full rationale

The paper's central claims are empirical and externally anchored. The 8-bit Hadamard quantization is credited to QuaRot [8], an external prior work, and is evaluated against an FP16 Mamba2-130M baseline in Table II; the reported 0.4-1% accuracy degradation is a measured comparison, not an assumed result. The power-of-two SSM quantization and the first-order exponential/SoftPlus approximations are tested through the same Table II accuracy evaluation, and the hardware speedups in Fig. 9 and Table III are measured against an Intel Xeon 4210R CPU and an NVIDIA RTX 3090 GPU. No fitted parameter is renamed as a prediction, and no derived quantity reduces to an input by construction. The paper contains self-citations only in the ordinary sense of building on the authors' own prior work in related areas (e.g., references [15] and [16] for the exponential and SoftPlus approximation formulas), but those citations are not load-bearing in a circular way: the approximations are implemented in hardware and their accuracy is empirically checked. The potential concern that Fig. 6's static quantization scales may not match Algorithm 1's dynamic max-based scales is a hardware-accuracy consistency issue, not a circularity issue, because it does not make the claimed results true by definition. Overall, the derivation chain is self-contained against external benchmarks and no circular step is present.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central performance and accuracy claims are not fitted to a target result; quantization scaling factors are data-dependent, and approximation coefficients are fixed design choices. The main assumptions are that the Hadamard rotation retains orthonormality, that the quantization/approximation accuracy carries to the 2.7B model, and that the CPU/GPU baselines are fair. No new physical or conceptual entities are introduced.

assumptions (4)

standard math Hadamard matrices are orthonormal when normalized by sqrt(n), so rotation preserves inner products and does not change the linear layer output.
Used in Section III.A to justify applying the Hadamard transform to X and W without altering the linear layer result.
domain assumption The outlier distribution in Mamba2 linear layer activations can be sufficiently mitigated by Hadamard rotation to enable 8-bit quantization.
Empirically asserted in Fig. 3, which shows only one example; not proven analytically.
domain assumption Power-of-two quantization and the first-order linear approximations of exponential and SoftPlus introduce negligible accuracy loss in the SSM block.
Accuracy is tested only on Mamba2-130M; the approximation coefficients are chosen by the authors, not derived from a formal optimality criterion.
domain assumption The CPU and GPU baselines represent reasonable, fair comparisons for the claimed speedups.
The paper gives no detail on the software implementation (e.g., whether fused Mamba kernels are used), so the fairness of the baseline is assumed.

how reviews work

0 comments

Cite this review

Pith. "Pith review of FastMamba: A High-Speed and Efficient Mamba Accelerator on FPGA with Accurate Quantization." pith.science (2026). https://pith.science/paper/RG2KBGKT

@misc{pith2026250518975,
  author       = {Pith},
  title        = {Pith review of: FastMamba: A High-Speed and Efficient Mamba Accelerator on FPGA with Accurate Quantization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RG2KBGKT}},
  note         = {Machine review of arXiv:2505.18975}
}

read the original abstract

State Space Models (SSMs), like recent Mamba2, have achieved remarkable performance and received extensive attention. However, deploying Mamba2 on resource-constrained edge devices encounters many problems: severe outliers within the linear layer challenging the quantization, diverse and irregular element-wise tensor operations, and hardware-unfriendly nonlinear functions in the SSM block. To address these issues, this paper presents FastMamba, a dedicated accelerator on FPGA with hardware-algorithm co-design to promote the deployment efficiency of Mamba2. Specifically, we successfully achieve 8-bit quantization for linear layers through Hadamard transformation to eliminate outliers. Moreover, a hardware-friendly and fine-grained power-of-two quantization framework is presented for the SSM block and convolution layer, and a first-order linear approximation is developed to optimize the nonlinear functions. Based on the accurate algorithm quantization, we propose an accelerator that integrates parallel vector processing units, pipelined execution dataflow, and an efficient SSM Nonlinear Approximation Unit, which enhances computational efficiency and reduces hardware complexity. Finally, we evaluate FastMamba on Xilinx VC709 FPGA. For the input prefill task on Mamba2-130M, FastMamba achieves 68.80\times and 8.90\times speedup over Intel Xeon 4210R CPU and NVIDIA RTX 3090 GPU, respectively. In the output decode experiment with Mamba2-2.7B, FastMamba attains 6\times higher energy efficiency than RTX 3090 GPU.

Figures

Figures reproduced from arXiv: 2505.18975 by the authors.

**Figure 2.** Computational flow in Mamba2 block and SSM block. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** The distributions of activation values before Hadamard transforming [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 6.** Figure 6: The architecture of Hadamard-base Linear Module (4 [PITH_FULL_IMAGE:figures/full_fig_p004_6.png]

**Figure 5.** Figure 5: The structure of the multipliers and adders in the VPUs. [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]

**Figure 7.** Figure 7: Architecture and computational flow of the SSM module (24 [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]

**Figure 9.** Figure 9: Comparison of speedup improvement to CPU and GPU on Mamba2- [PITH_FULL_IMAGE:figures/full_fig_p006_9.png]

**Figure 10.** Figure 10: The hardware resource utilization rates of the Nonlinear Approxima [PITH_FULL_IMAGE:figures/full_fig_p006_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Systolic Array-based Accelerator for Structured State-Space Models
cs.LG 2025-07 reject novelty 6.0 of 10

A specialized systolic-array accelerator with a reconfigurable processing element and diagonal dataflow claims 2000x inference speedup over GPUs for S4 and Liquid-S4 state-space models.

Reference graph

Works this paper leans on

17 extracted references · 12 canonical work pages · cited by 1 Pith paper

[9]

Marca: Mamba accelerator with reconfigurable architec- ture,

J. Li et al. , “Marca: Mamba accelerator with reconfigurable architec- ture,” arXiv preprint arXiv:2409.11440 , 2024

arXiv 2024
[1]

Mamba: Linear-time sequence modeling with selective state spaces,

A. Gu et al. , “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752 , 2024

arXiv 2024
[2]

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality,

A. Gu et al., “Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality,” in ICML, 2024

work page 2024
[3]

Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model,

L. Zhu et al., “Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model,” in ICML, 2024

work page 2024
[4]

VMamba: Visual State Space Model,

Y . Liu et al., “VMamba: Visual State Space Model,” in NeurIPS, 2024, pp. 103031-103063

work page 2024
[5]

VideoMamba: State Space Model for Efficient Video Understanding,

K. Li et al. , “VideoMamba: State Space Model for Efficient Video Understanding,” in ECCV, 2024, pp. 237-255

work page 2024
[6]

Attention is All you Need,

A. Vaswani et al., “Attention is All you Need,” in NeurIPS, 2017

work page 2017
[7]

SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models,

G. Xiao et al., “SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models,” in ICML, 2023

work page 2023

Show all 17 references

[8]

QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs,

S. Ashkboos et al. , “QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs,” in NeurIPS, 2024, pp. 100213-100240

2024
[10]

Root Mean Square Layer Normalization,

B. Zhang et al. , “Root Mean Square Layer Normalization,” arXiv preprint arXiv:1910.07467, 2019

1910 arXiv
[11]

Searching for Activation Functions,

P. Ramachandran et al. , “Searching for Activation Functions,” arXiv preprint arXiv:1710.05941, 2017

2017 arXiv
[12]

Digit-Serial DA-Based Fixed-Point RNNs: A Unified Approach for Enhancing Architectural Efficiency,

M. T. Khan et al., “Digit-Serial DA-Based Fixed-Point RNNs: A Unified Approach for Enhancing Architectural Efficiency,” IEEE Trans. Neural Networks and Learn. Syst. , vol. 36, no. 5, pp. 8240-8254, 2025

2025
[13]

Incorporating Second-Order Functional Knowledge for Bet- ter Option Pricing,

C. Dugas, “Incorporating Second-Order Functional Knowledge for Bet- ter Option Pricing,”, Neural Information Processing Systems ,2000

2000
[14]

Zero-order hold discretization of general state space systems with input delay,

G. Pechlivanidou et al., “Zero-order hold discretization of general state space systems with input delay,” IMA J. Math. Control. Inf., vol. 39, pp. 708-730, June 2022

2022
[15]

Efficient Precision-Adjustable Architecture for Softmax Function in Deep Learning,

D. Zhu et al., “Efficient Precision-Adjustable Architecture for Softmax Function in Deep Learning,” IEEE Trans. Circuits Syst. , vol. 67, pp. 3382-3386, December 2020

2020
[16]

A High-Precision Flexible Symmetry-Aware Architec- ture for Element-Wise Activation Functions,

X. Feng et al., “A High-Precision Flexible Symmetry-Aware Architec- ture for Element-Wise Activation Functions,” in ICFPT, 2021, pp. 1-4

2021
[17]

A framework for few-shot language model evaluation,

L. Gao et al. , “A framework for few-shot language model evaluation,” Zenodo, July 2024. [Online]. Available: https://zenodo.org/records/12608602

2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [9]

Marca: Mamba accelerator with reconfigurable architec- ture,

J. Li et al. , “Marca: Mamba accelerator with reconfigurable architec- ture,” arXiv preprint arXiv:2409.11440 , 2024

arXiv 2024

[2] [1]

Mamba: Linear-time sequence modeling with selective state spaces,

A. Gu et al. , “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752 , 2024

arXiv 2024

[3] [2]

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality,

A. Gu et al., “Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality,” in ICML, 2024

work page 2024

[4] [3]

Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model,

L. Zhu et al., “Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model,” in ICML, 2024

work page 2024

[5] [4]

VMamba: Visual State Space Model,

Y . Liu et al., “VMamba: Visual State Space Model,” in NeurIPS, 2024, pp. 103031-103063

work page 2024

[6] [5]

VideoMamba: State Space Model for Efficient Video Understanding,

K. Li et al. , “VideoMamba: State Space Model for Efficient Video Understanding,” in ECCV, 2024, pp. 237-255

work page 2024

[7] [6]

Attention is All you Need,

A. Vaswani et al., “Attention is All you Need,” in NeurIPS, 2017

work page 2017

[8] [7]

SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models,

G. Xiao et al., “SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models,” in ICML, 2023

work page 2023