ITERA-LLM: Boosting Sub-8-Bit Large Language Model Inference via Iterative Tensor Decomposition

Christos-Savvas Bouganis; Keran Zheng; Yinting Huang; Zhewen Yu

REVIEW 3 major objections 7 minor 21 references

ITERA-LLM: Boosting Sub-8-Bit Large Language Model Inference via Iterative Tensor Decomposition

T0 review · 3 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ITERA-LLM claims that putting quantization inside an iterative SVD residual loop, instead of after it, recovers up to 4.9% BLEU over quantization-only W4A8 compression and reduces FPGA linear-layer latency by up to 41.1% at comparable…

desk verdict A real algorithmic contribution in iterative SVD plus sensitivity-based rank allocation, but the headline 41.1% hardware speedup is model-only and not yet measured. read the letter →

arxiv 2505.08981 v1 pith:X3AIZBQK submitted 2025-05-13 cs.AR

Keran Zheng , Yinting Huang , Zhewen Yu , Christos-Savvas Bouganis This is my paper · ORCID

classification cs.AR

keywords largelanguagemodelssub-8-bitquantizationsingularvaluedecompositionlow-ranktensorFPGAaccelerationco-designsensitivity-basedrankallocationneuralmachinetranslation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that sub-8-bit LLM inference can go beyond quantization-only compression by adding an iterative, quantization-aware SVD decomposition of each weight matrix. The method repeatedly extracts and quantizes the dominant rank-one singular component, subtracts it from a residual, and repeats, then uses sensitivity-based rank allocation to assign ranks per layer. On OPUS-MT translation models, this approach improves BLEU by up to 4.9% over W4A8 quantization at the same compression ratio, and on FPGA analytical models it cuts linear-layer latency by up to 41.1% at comparable accuracy. The paper also proposes hardware-aware analytical models and design-space exploration to select Pareto-optimal accuracy-latency configurations.

What carries the argument

The central mechanism is the SVD-based iterative tensor decomposition loop: starting from a weight matrix $W$, each iteration takes the rank-1 singular value decomposition of the current residual, quantizes the two singular-vector factors to the target word length, subtracts their product from the residual, and accumulates the quantized factors into two low-rank matrices $W'_1$ and $W'_2$. Placing quantization inside the residual loop lets the approximation explicitly account for quantization error at every rank. The supporting mechanism is sensitivity-based rank allocation, which uses finite-difference estimates of model accuracy with respect to each layer's rank to shift rank budget from insensitive to sensitive layers. On the hardware side, the Single and Cascade SVD MatMul engines reuse or cascade the same systolic MatMul tile and keep the intermediate $XW'_1$ tile on-chip, supported by analytical latency, DSP, BRAM, and bandwidth models for design-space exploration.

What would settle it

Synthesize the best W6A8 SVD-iterative design point and the quantization-only baseline on a ZCU111 with Vitis HLS 2023.2 at 200 MHz, run the EN-DE and FR-EN OPUS-MT linear layers at batch size 512, and compare measured latency and BLEU against the predicted 41.1% reduction and 4.9% improvement.

Watch

Extended reading notes

Core claim

The central claim is that quantization error and low-rank truncation error can be compensated together by running SVD as a quantized rank-one residual refinement loop, rather than decomposing first and quantizing afterward. This iterative decomposition produces compression Pareto fronts that dominate both quantization-only and one-shot SVD baselines across compression ratios. With sensitivity-based rank allocation, W4A8 improves BLEU by up to 4.9% over quantization-only at the same compression ratio, and W6A8 reduces the number of fixed-point operations by 12.5% at similar model accuracy. Under the resource constraints of the ZCU111 FPGA, the framework reports linear-layer latency reductions between 12.1% and 41.1% compared to a quantization-only MatMul baseline at comparable accuracy.

Load-bearing premise

The reported latency reductions and Pareto fronts rest on analytical performance and resource models in Section VI that the paper does not validate with measured bitstreams on the ZCU111 board; if those rate and workload estimates are inaccurate, the 41.1% latency reduction may not materialize in real hardware.

Editorial extensions

If this is right

Sub-8-bit post-training LLM compression can avoid the accuracy drop of quantization-only W4A8 by using iterative SVD with sensitivity-based rank allocation, producing Pareto fronts that dominate quantization-only baseline.
Under compute-bound resources, W6A8 iterative SVD with SRA gives better BLEU and lower latency than both W4A8 and the quantization baseline, while under bandwidth-limited resources W4A8 with SRA gives the best Pareto front.
At similar model accuracy, the method reduces fixed-point operations by 12.5% at W6A8 compared to quantization-only compression.
SVD MatMul engines can match baseline MatMul latency in the bandwidth-limited region while requiring less off-chip bandwidth, and beat it in the compute-bound region due to reduced operations.
For selected design points, per-layer occupancy variation remains below 5%, indicating padding overhead is small with the chosen tiling configurations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the analytical hardware models prove accurate on real hardware, the same co-design search should transfer to other transformer workloads, but the paper only demonstrates OPUS-MT translation models, so generality to other LLM families remains untested.
The accuracy gains concentrate at low compression ratios where more residual refinement iterations are available, suggesting a direct test: sweep the rank budget on a single model and check whether per-iteration BLEU recovery stays roughly constant or decays.
The 41.1% figure is specifically linear-layer latency, not end-to-end inference; extending it to whole-model speedup requires assuming non-linear layers, embeddings, and decoding overheads are not the dominant bottleneck.
A natural extension would be to apply the same iterative residual idea to other decompositions, such as Tucker or CP, provided each iteration can produce a quantized low-rank factor update, though the paper does not investigate this.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The part of this paper that deserves attention is the algorithm, not the headline number. Iterative quantized SVD residual refinement is a genuinely useful idea: it quantizes rank-1 singular vectors inside a loop, subtracts them from the residual, and repeats. On OPUS-MT, that clearly beats both a quantization-only baseline and plain SVD followed by quantization, at the same compression ratio and operation count. The sensitivity-based rank allocation (SRA) also looks sensible as a way to distribute a rank budget across layers, and the comparison against a uniform-rank SVD baseline gives a fair view of what SRA buys. The two hardware scheduling schemes (single engine vs. cascade) are a nice co-design touch, even if they are not the main intellectual contribution.

Now the soft spots, in proportion. The 41.1% latency reduction in the abstract and Fig. 11 comes entirely from the analytical performance model in Section VI. No synthesis, no RTL simulation, no on-board measurement is reported, despite the ZCU111 and Vitis HLS being mentioned. The stress-test note is right: Eq. 15 takes a max of workload/rate ratios, implicitly assuming perfectly overlapped input, output, and compute, with no intermediate buffering cost for the two sequential MatMuls. That is not an innocent assumption, and the claimed 0.589x speedup could move substantially if real scheduling overheads are counted. For a hardware paper, this is a load-bearing gap.

The SRA circularity concern is real but less severe. The rank allocation is tuned against BLEU on a calibration set and then evaluated with BLEU on the same benchmark family, so part of the gain is fitted. That is common in co-design DSE, but it would be easy to address with a held-out split or multiple calibration seeds. Also, no error bars or multiple seeds are given for the 4.9% BLEU improvement; that number appears to be the max across conditions. A missing baseline that would isolate the benefit of the iterative refinement itself is SRA applied to plain SVD (rather than uniform-rank SVD). Without it, you cannot tell how much of the gain comes from the iterative loop versus the rank allocation.

Citation pattern looks fine, and the paper is honest about comparing to a quantization-only baseline. It is not trying to hide the weaknesses. My verdict is conditional: the algorithmic claims are plausible and interesting, but the hardware claims need to be backed by measurement or at least cycle-accurate simulation before I would trust them.

This deserves a serious referee. It is a coherent co-design paper with a novel combination and fair algorithmic comparisons. I would assign it for review, and I would ask for hardware validation on a few design points, a clearer train/calibration/eval split, and error bars on the BLEU numbers.

Referee Report

3 major / 7 minor

Summary. ITERA-LLM proposes a software-hardware co-design framework for sub-8-bit LLM inference on FPGAs. The algorithmic contribution is an iterative SVD-based tensor decomposition in which each rank-1 factor is quantized and subtracted from the residual, followed by a Sensitivity-based Rank Allocation (SRA) heuristic that adjusts per-layer ranks via finite-difference estimates of BLEU sensitivity under a total rank budget. The hardware contribution is a set of analytical performance and resource models for two SVD MatMul engine variants (single-engine and cascade), used to prune and explore the design space under ZCU111 constraints. The paper reports up to 41.1% linear-layer latency reduction relative to a quantization-only MatMul baseline at comparable BLEU, and up to 4.9% BLEU improvement at W4A8.

Significance. If the latency and accuracy claims were independently validated, the paper would make a useful contribution to low-bitwidth LLM acceleration on FPGAs: the iterative residual compensation is a plausible mechanism for recovering quantization loss at sub-8-bit precision, SRA is a principled way to allocate a rank budget, and the DSE coupling of compression Pareto fronts with hardware configurations addresses a real gap. The paper is also careful to compare against a quantization-only baseline and to separate compute-bound from bandwidth-limited regimes. However, the headline hardware speedup is produced entirely by an unvalidated analytical model, and the accuracy improvements are partly fitted on the evaluation metric, so the significance is conditional on additional evidence. No code, bitstream, or synthesis report is provided, which further limits reproducibility.

major comments (3)

[Section VI.A, Eq. (15), Fig. 11] The central latency claim is not supported by any implementation evidence. Eq. (15) computes latency as the maximum of three workload/rate ratios, which assumes that input, output, and computation are perfectly overlapped and ignores pipeline fill and drain, port contention, DRAM efficiency, and the cost of buffering and re-reading the intermediate Mt×R tile described in Section V. The paper reports no RTL simulation, synthesis timing, resource utilization report, or on-board ZCU111 measurement, so the 0.589× (41.1% reduction) latency ratio and the Pareto fronts in Fig. 11 are model-predicted values only. Adding at least a synthesis-based validation of the cycle and resource models, and ideally on-board measurements, is required to support the headline result.
[Section IV.B, Eq. (8), Section VIII] The SRA procedure optimizes ranks by evaluating BLEU on a calibration set (Eqs. 6-8) and the same BLEU metric on the same WMT2019 benchmark family is used for the final evaluation, with no held-out split or multiple-seed/error-bar reporting. The accuracy improvement over the quantization-only baseline is therefore partly an optimized quantity rather than an independent prediction. Please report results on a separate validation or test set, provide variance over at least several runs, and state the calibration set size and sampling procedure.
[Section VI.B, Eqs. (17)-(18)] The BRAM model assigns one BRAM18K per DSP at depth K/Kf, which does not account for dual-port conflicts, padding to the DSP/Kf granularity, or the actual FIFO implementation after synthesis. Since the resource model is used to prune the design space and to define the ZCU111 constraints in Figs. 10-11, an error in this model could change which configurations are feasible and thus alter the reported Pareto fronts. Please validate the resource estimates against synthesis reports for representative configurations.

minor comments (7)

[Algorithm 1] The residual update line has a typo: it reads "˜R = ˜R− W′k_1 , W′k_2" but should subtract the product W′k_1 W′k_2, not two separate terms.
[Eq. (4)] Equation (4) writes the rank-1 factors as W′k_1 and W′k_2 without explicitly stating their dimensions; please clarify that they are rank-1 outer products to avoid ambiguity with the augmented rank-r matrices W′1 and W′2.
[Section VIII.B] The experimental setup labels [8] as a post-training quantization scheme, but [8] (Q8BERT) is a quantization-aware training method; please correct the citation or the description of the baseline.
[Section IV.B] The SRA evaluation uses a "randomly sampled calibration set" without reporting its size, sampling distribution, or number of sentences; this information is needed for reproducibility and for judging the stability of the finite-difference sensitivities.
[Section V] The paper describes weight word lengths in detail but does not explicitly state how activations X are quantized in the hardware path (beyond the A8 notation); please make the activation quantization scheme and its implementation in the PE explicit.
[Fig. 10 caption] The caption says "corresponding bandwidth requirement to run at full throughput" without noting that these values come from the analytical model of Eq. (19); please state that these are modeled estimates, not measurements.
[Abstract and Section IV] The paper uses "model accuracy" and "BLEU Score" interchangeably; please use consistent terminology, since BLEU is a translation quality metric rather than accuracy in the classification sense.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ITERA-LLM's compression, rank search, and analytical latency estimates are explicit optimizations/models rather than re-labeled inputs, though hardware claims remain model-only.

full rationale

The paper's central claims do not reduce to their inputs by construction. Algorithm 1 defines an explicit iterative SVD-with-quantization residual refinement loop; the output is not assumed to equal the target accuracy or latency. The SRA procedure (Section IV) solves the stated optimization problem in Eq. 5 by finite-difference BLEU sensitivity estimates and greedy rank adjustment; the reported BLEU improvements are empirical outcomes of that search, and the paper does not state that the calibration set used for rank allocation is the same as the WMT2019 evaluation set, so the result is not a fitted parameter renamed as a prediction. The latency and resource models (Section VI, Eqs. 12-19) are analytical estimates used for design-space exploration; they are not fitted to the claimed 41.1% speedup, so the latency comparison is model-based rather than circular. The self-citations ([15], [17]) appear only as related-work context and are not load-bearing for the paper's unique claims; no uniqueness theorem or ansatz is imported from the authors' prior work. The main weakness is that the hardware latency claims are not validated against synthesis or on-board measurement, but that is a correctness or validation gap, not circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper's central claims rest on empirical rank allocation, hand-chosen bit-widths, and unvalidated analytical hardware models. No new physical entities are introduced.

free parameters (3)

per-layer decomposition ranks r_i = variable, set by SRA
Allocated by sensitivity-based rank allocation to maximize BLEU on a calibration set (Eq. 8-11), so the accuracy result depends on these fitted ranks.
weight bit-width wl (W4A8, W6A8) = 4 and 6 bits in the reported Pareto points
Chosen via design space exploration across bit-widths; the reported gains are only for the selected configurations.
SRA hyperparameters delta0 and alpha = not specified
Initial perturbation and decay constant in Eq. 11 are hand-chosen; the paper does not report values or sensitivity to them.

assumptions (4)

domain assumption SVD-based low-rank approximation combined with quantization can recover accuracy lost by sub-8-bit weight quantization in LLMs.
The central premise of the method, stated in Sections I and III, is that residual iterative SVD compensates quantization error; no theoretical guarantee is given.
domain assumption Finite-difference sensitivity of BLEU with respect to rank is a valid guide for rank allocation across layers.
Section IV.B assumes that the partial derivative approximation in Eq. 8 reflects true layer sensitivity; this is heuristic for non-differentiable BLEU and could be noisy.
domain assumption The analytical rate and resource models (Eq. 12-19) predict actual FPGA latency and utilization accurately enough for design ranking.
Section VI builds the latency claims on these models; no comparison to hardware measurements is provided.
domain assumption The randomly sampled calibration set used for SRA represents the WMT2019 test distribution.
Section IV.B uses a calibration set for BLEU evaluation; if it overlaps or differs from the test set, the reported accuracies could be inflated or misleading.

how reviews work

0 comments

Cite this review

Pith. "Pith review of ITERA-LLM: Boosting Sub-8-Bit Large Language Model Inference via Iterative Tensor Decomposition." pith.science (2026). https://pith.science/paper/X3AIZBQK

@misc{pith2026250508981,
  author       = {Pith},
  title        = {Pith review of: ITERA-LLM: Boosting Sub-8-Bit Large Language Model Inference via Iterative Tensor Decomposition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X3AIZBQK}},
  note         = {Machine review of arXiv:2505.08981}
}

read the original abstract

Recent advancements in Large Language Models (LLMs) have demonstrated impressive capabilities as their scale expands to billions of parameters. Deploying these large-scale models on resource-constrained platforms presents significant challenges, with post-training fixed-point quantization often used as a model compression technique. However, quantization-only methods typically lead to significant accuracy degradation in LLMs when precision falls below 8 bits. This paper addresses this challenge through a software-hardware co-design framework, ITERA-LLM, which integrates sub-8-bit quantization with SVD-based iterative low-rank tensor decomposition for error compensation, leading to higher compression ratios and reduced computational complexity. The proposed approach is complemented by a hardware-aware Design Space Exploration (DSE) process that optimizes accuracy, latency, and resource utilization, tailoring the configuration to the specific requirements of the targeted LLM. Our results show that ITERA-LLM achieves linear layer latency reduction of up to 41.1%, compared to quantization-only baseline approach while maintaining similar model accuracy.

Figures

Figures reproduced from arXiv: 2505.08981 by the authors.

**Figure 2.** The overview of proposed ITERA-LLM framework. This work focuses on the topic of post-training compression and FPGA accelerator co-design for producing Pareto-optimal design points on the accuracy-latency frontier. sensitivity-based rank allocation. Our approach achieves up to 4.9% improvement in model accuracy at W4A8 compared to existing quantization-only methods at a comparable compression ratio. Furthermore, for … view at source ↗

**Figure 3.** The proposed iterative tensor decomposition algorithm with SVD and [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: The sensitivity analysis measures the reduction in [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]

**Figure 5.** Figure 5: Dataflow and parallelism scheme for a target dense matrix-matrix [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]

**Figure 6.** Figure 6: Single SVD MatMul engine architecture (left) vs. Cascade SVD [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]

**Figure 7.** Figure 7: Pareto fronts of BLEU score (accuracy) versus model compression ratio obtained from design space exploration. The Pareto fronts correspond to the design points of SVD Iterative (SRA) W4A8, highlighted in red with a cross symbol. decomposition and quantization baseline …

**Figure 9.** Figure 9: Bar plot of BLEU score versus compression ratio for OPUSMT models with different source-to-target language pairs, demonstrating the generality of our approach. At compression ratio 8, our iterative SVD tensor decomposition W4A8 improves accuracy by 1.2% over quantizat…

**Figure 10.** Figure 10: Pareto fronts of different modes of MatMul engines’ latencies and [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]

**Figure 11.** Figure 11: Trade-offs between BLEU score and performance (latency) across compression methods mapped onto MatMul kernels, evaluated with batch size 512 under resource constraints of ZCU111 under two different off-chip bandwidth constraints. Left: original bandwidth of ZCU111, Ri…

**Figure 12.** Figure 12: Layer-wise MatMul Tile occupancy for selected design points [PITH_FULL_IMAGE:figures/full_fig_p008_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 18 canonical work pages

[1]

Rae, Oriol Vinyals, and Laurent Sifre

Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre...

2022
[2]

M4bram: Mixed-precision matrix-matrix multiplication in fpga block rams

Yuzong Chen, Jordan Dotzel, and Mohamed S Abdelfattah. M4bram: Mixed-precision matrix-matrix multiplication in fpga block rams. In 2023 International Conference on Field Programmable Technology (ICFPT), pages 69–78. IEEE, 2023

work page 2023
[3]

Msd: Mixing signed digit representations for hardware-efficient dnn acceleration on fpga with heterogeneous resources

Jiajun Wu, Jiajun Zhou, Yizhao Gao, Yuhao Ding, Ngai Wong, and Hayden Kwok-Hay So. Msd: Mixing signed digit representations for hardware-efficient dnn acceleration on fpga with heterogeneous resources. In 2023 IEEE 31st Annual International Symposium on Field- Programmable Custom Computing Machines (FCCM) , pages 94–104. IEEE, 2023

work page 2023
[4]

Democratizing neural ma- chine translation with OPUS-MT

J ¨org Tiedemann, Mikko Aulamo, Daria Bakshandaeva, Michele Boggia, Stig-Arne Gr ¨onroos, Tommi Nieminen, Alessandro Raganato Yves Scherrer, Raul Vazquez, and Sami Virpioja. Democratizing neural ma- chine translation with OPUS-MT. Language Resources and Evaluation, (58):713–755, 2023

work page 2023
[5]

Omniquant: Omnidirectionally calibrated quantization for large language models, 2024

Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. Omniquant: Omnidirectionally calibrated quantization for large language models, 2024

2024
[6]

Qllm: Accurate and efficient low-bitwidth quantization for large language models, 2024

Jing Liu, Ruihao Gong, Xiuying Wei, Zhiwei Dong, Jianfei Cai, and Bohan Zhuang. Qllm: Accurate and efficient low-bitwidth quantization for large language models, 2024

work page 2024
[7]

Efficient arbitrary precision acceleration for large language models on gpu tensor cores, 2024

Shaobo Ma, Chao Fang, Haikuo Shao, and Zhongfeng Wang. Efficient arbitrary precision acceleration for large language models on gpu tensor cores, 2024

work page 2024
[8]

Q8bert: Quantized 8bit bert

Ofir Zafrir, Guy Boudoukh, Peter Izsak, and Moshe Wasserblat. Q8bert: Quantized 8bit bert. In 2019 Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing - NeurIPS Edition (EMC2-NIPS) , pages 36–39. IEEE, December 2019

work page 2019

Show all 21 references

[9]

Q-bert: Hessian based ultra low precision quantization of bert

Sheng Shen, Zhen Dong, Jiayu Ye, Linjian Ma, Zhewei Yao, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. Q-bert: Hessian based ultra low precision quantization of bert. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 8815– 8821, 2020

2020
[10]

Owq: Outlier-aware weight quantization for efficient fine-tuning and inference of large language models

Changhun Lee, Jungyu Jin, Taesu Kim, Hyungjun Kim, and Eunhyeok Park. Owq: Outlier-aware weight quantization for efficient fine-tuning and inference of large language models. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 38, pages 13355–13364, 2024

2024
[11]

Hawq: Hessian aware quantization of neural networks with mixed-precision

Zhen Dong, Zhewei Yao, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. Hawq: Hessian aware quantization of neural networks with mixed-precision. In Proceedings of the IEEE/CVF international conference on computer vision , pages 293–302, 2019

2019
[12]

Bitmod: Bit-serial mixture-of-datatype llm acceleration

Yuzong Chen, Ahmed F AbouElhamayed, Xilai Dai, Yang Wang, Marta Andronic, George A Constantinides, and Mohamed S Abdelfattah. Bitmod: Bit-serial mixture-of-datatype llm acceleration. arXiv preprint arXiv:2411.11745, 2024

2024 arXiv
[13]

Optimizing bit-serial matrix multiplica- tion for reconfigurable computing

Yaman Umuroglu, Davide Conficconi, Lahiru Rasnayake, Thomas B Preusser, and Magnus Sj¨alander. Optimizing bit-serial matrix multiplica- tion for reconfigurable computing. ACM Transactions on Reconfigurable Technology and Systems (TRETS) , 12(3):1–24, 2019

2019
[14]

Hihispmv: Sparse matrix vector multiplication with hierarchical row reductions on fpgas with high bandwidth memory

Abdul Rehman Tareen, Marius Meyer, Christian Plessl, and Tobias Kenter. Hihispmv: Sparse matrix vector multiplication with hierarchical row reductions on fpgas with high bandwidth memory. In 2024 IEEE 32nd Annual International Symposium on Field-Programmable Custom Computing M...

2024
[15]

Hass: Hardware-aware sparsity search for dataflow dnn accelerator

Zhewen Yu, Sudarshan Sreeram, Krish Agrawal, Junyi Wu, Alexander Montgomerie-Corcoran, Cheng Zhang, Jianyi Cheng, Christos-Savvas Bouganis, and Yiren Zhao. Hass: Hardware-aware sparsity search for dataflow dnn accelerator. arXiv preprint arXiv:2406.03088 , 2024

2024 arXiv
[16]

Adaptable butterfly accelerator for attention-based nns via hardware and algorithm co-design

Hongxiang Fan, Thomas Chau, Stylianos I Venieris, Royson Lee, Alexandros Kouris, Wayne Luk, Nicholas D Lane, and Mohamed S Abdelfattah. Adaptable butterfly accelerator for attention-based nns via hardware and algorithm co-design. In 2022 55th IEEE/ACM International Symposium o...

2022
[17]

Streamsvd: Low-rank ap- proximation and streaming accelerator co-design

Zhewen Yu and Christos-Savvas Bouganis. Streamsvd: Low-rank ap- proximation and streaming accelerator co-design. In 2021 International Conference on Field-Programmable Technology (ICFPT) , pages 1–9. IEEE, 2021

2021
[18]

Charm: Composing heterogeneous accelerators for matrix multiply on versal acap architec- ture, 2023

Jinming Zhuang, Jason Lau, Hanchen Ye, Zhuoping Yang, Yubo Du, Jack Lo, Kristof Denolf, Stephen Neuendorffer, Alex Jones, Jingtong Hu, Deming Chen, Jason Cong, and Peipei Zhou. Charm: Composing heterogeneous accelerators for matrix multiply on versal acap architec- ture, 2023

2023
[19]

Film-qnn: Efficient fpga acceleration of deep neural networks with intra-layer, mixed-precision quantization

Mengshu Sun, Zhengang Li, Alec Lu, Yanyu Li, Sung-En Chang, Xiaolong Ma, Xue Lin, and Zhenman Fang. Film-qnn: Efficient fpga acceleration of deep neural networks with intra-layer, mixed-precision quantization. In Proceedings of the 2022 ACM/SIGDA International Symposium on Fie...

2022
[20]

Understanding the potential of fpga-based spatial acceleration for large language model inference

Hongzheng Chen, Jiahao Zhang, Yixiao Du, Shaojie Xiang, Zichao Yue, Niansong Zhang, Yaohui Cai, and Zhiru Zhang. Understanding the potential of fpga-based spatial acceleration for large language model inference. ACM Transactions on Reconfigurable Technology and Systems, 18(1):...

2024
[21]

James W. Demmel. Applied numerical linear algebra . Society for Industrial and Applied Mathematics, USA, 1997

1997

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Rae, Oriol Vinyals, and Laurent Sifre

Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre...

2022

[2] [2]

M4bram: Mixed-precision matrix-matrix multiplication in fpga block rams

Yuzong Chen, Jordan Dotzel, and Mohamed S Abdelfattah. M4bram: Mixed-precision matrix-matrix multiplication in fpga block rams. In 2023 International Conference on Field Programmable Technology (ICFPT), pages 69–78. IEEE, 2023

work page 2023

[3] [3]

Msd: Mixing signed digit representations for hardware-efficient dnn acceleration on fpga with heterogeneous resources

Jiajun Wu, Jiajun Zhou, Yizhao Gao, Yuhao Ding, Ngai Wong, and Hayden Kwok-Hay So. Msd: Mixing signed digit representations for hardware-efficient dnn acceleration on fpga with heterogeneous resources. In 2023 IEEE 31st Annual International Symposium on Field- Programmable Custom Computing Machines (FCCM) , pages 94–104. IEEE, 2023

work page 2023

[4] [4]

Democratizing neural ma- chine translation with OPUS-MT

J ¨org Tiedemann, Mikko Aulamo, Daria Bakshandaeva, Michele Boggia, Stig-Arne Gr ¨onroos, Tommi Nieminen, Alessandro Raganato Yves Scherrer, Raul Vazquez, and Sami Virpioja. Democratizing neural ma- chine translation with OPUS-MT. Language Resources and Evaluation, (58):713–755, 2023

work page 2023

[5] [5]

Omniquant: Omnidirectionally calibrated quantization for large language models, 2024

Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. Omniquant: Omnidirectionally calibrated quantization for large language models, 2024

2024

[6] [6]

Qllm: Accurate and efficient low-bitwidth quantization for large language models, 2024

Jing Liu, Ruihao Gong, Xiuying Wei, Zhiwei Dong, Jianfei Cai, and Bohan Zhuang. Qllm: Accurate and efficient low-bitwidth quantization for large language models, 2024

work page 2024

[7] [7]

Efficient arbitrary precision acceleration for large language models on gpu tensor cores, 2024

Shaobo Ma, Chao Fang, Haikuo Shao, and Zhongfeng Wang. Efficient arbitrary precision acceleration for large language models on gpu tensor cores, 2024

work page 2024

[8] [8]

Q8bert: Quantized 8bit bert

Ofir Zafrir, Guy Boudoukh, Peter Izsak, and Moshe Wasserblat. Q8bert: Quantized 8bit bert. In 2019 Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing - NeurIPS Edition (EMC2-NIPS) , pages 36–39. IEEE, December 2019

work page 2019