Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Transitive Array: An Efficient GEMM Accelerator with Result Reuse

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a quantized GEMM can be made roughly 8x cheaper by reusing earlier bit-row sums, and that a multiplication-free accelerator array realizes this with 7.46x speedups at equal accuracy.

desk verdict Genuine co-design for bit-sliced GEMM reuse with a fixable sign-arithmetic gap and a single-block evaluation; deserves serious refereeing. read the letter →

arxiv 2504.16339 v1 pith:NUKTPVHF submitted 2025-04-23 cs.AR

Cong Guo , Chiyue Wei , Jiaming Tang , Bowen Duan , Song Han , Hai Li , Yiran Chen This is my paper · ORCID
classification cs.AR
keywords transitivesparsityGEMMaccelerationbit-slicingLLMinferencequantizationHassegraphresultreusemultiplication-freeaccelerator
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that quantized matrix multiplication contains a previously untapped, lossless form of sparsity: after bit-slicing, many row patterns repeat or extend one another, so a row's partial sum can be reused instead of recomputed. It calls this transitive sparsity and argues it cuts the operation count of an 8-bit GEMM by up to 8x (87.5% sparsity) on LLaMA-7B weights without changing quantized results. The companion claim is that this sparsity can be exploited in hardware: the Transitive Array replaces multipliers with adders, uses a Hasse-graph scoreboard to find execution orders in linear time, and reports about 7.46x and 3.97x speedups over Olive and BitVert with 2.31x and 1.65x energy reductions at comparable LLaMA perplexity. A sympathetic reader would care because if true, low-cost LLM inference can get large speedups from a purely structural property of quantized tensors, not from model pruning or accuracy trade-offs.

What carries the argument

The central object is the Hasse graph of TransRows, the directed acyclic graph whose nodes are the 2^T possible T-bit row patterns and whose edges join patterns that differ by exactly one 1-bit. Each row's prefix is the nearest earlier node (smallest Hamming distance) that has already been computed, and the paper's Scoreboard algorithm—a forward pass that records candidate prefixes and a backward pass that keeps only the closest single prefix—turns this graph into a balanced forest of independent trees. The Scoreboard Information (SI) table stores for each TransRow its prefix and lane assignment; static SI is computed offline for weights, while a dynamic Scoreboard computes SI on the fly for attention tensors. The Transitive Array then executes each row as an XOR between row and prefix to isolate the new bits, a 12-bit adder in the Prefix PE, and a 24-bit accumulator in the Accumulation PE, so no multipliers are involved.

What would settle it

Re-run the paper's Scoreboard analysis on every Transformer block (not just the first) and on prefill lengths above 2048, counting the fraction of TransRows with reuse distance greater than 1 and the fraction of distance-3 prefixes. The central claim predicts these stay near 1.67% and below 0.1% across all measured blocks; a measurement where either fraction grows with depth or sequence length, or where fresh-row count approaches the 162-out-of-256 random expectation on real weights, would falsify the efficiency premise and collapse the speedup toward the unconditional 1/T bound.

Watch

Extended reading notes

Core claim

Bit-slicing turns each quantized weight row into multiple binary rows (TransRows). The paper's discovery claim is that real LLM tensors contain heavy overlap among these rows: some are identical, and others are supersets that differ by one or a few bits, so a row such as 1011 can be computed as the stored sum of 0011 plus one additional input value rather than as three fresh additions. Because every node with the same number of 1s has no ordering with its peers, a Hasse graph over the 2^T rows gives a partial order whose level structure exposes parallelism, and assigning each node a single smallest-distance prefix splits the graph into independent trees. On this basis the paper claims the theoretical 8x computation reduction for LLaMA-7B versus dense GEMM, and states that the Transitive Array obtains the reported speedup and energy gains while matching the perplexity of baselines on LLaMA-1/2/3.

Load-bearing premise

The speedup stands on the claim that real LLM tensors, everywhere and at any length, keep the dense row-overlap structure measured on the first Transformer block at 2048 tokens; if later blocks or longer sequences produce more diverse bit patterns, the reuse degrades to ordinary one-addition-per-bit performance.

Editorial extensions

If this is right

  • For any 8-bit bit-sliced GEMM whose rows exhibit the measured overlap, each T-bit row needs roughly one accumulation instead of T, giving a theoretical upper bound of 1/T density (12.5% for 8-bit) and a 8x operation reduction over dense GEMM.
  • Because the array only contains adders, a hardware implementation is compact; the paper computes a core area of 0.443 mm² at 28nm for six units, below all five baselines.
  • Dynamic Scoreboarding makes the accelerator compatible with attention layers' on-the-fly Q/K/V tensors, which the paper says prior accelerators Olive, Tender, and BitVert do not support.
  • The design is quantization-agnostic and supports mixed precision: 4-bit and 8-bit weights and activations are handled by splitting or widening the adder arrays, and it can run on top of group-wise quantization schemes.
  • With 4-bit weights the paper reports 4.91x, 7.46x, and 3.97x speedups over ANT, Olive, and BitVert at iso-accuracy, extending the reuse benefit beyond the 8-bit case.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 8x reduction is an upper bound tied to 8-bit rows and the measured distribution; if the roughly 1.67% long-distance fraction rises at longer prefill lengths, per-row cost grows toward the unconditional bound of one addition per T bits, so the reported speedups are an upper envelope rather than a guaranteed mean.
  • The static-Scoreboard mode hints at a compiler or runtime pass that picks tile sizes and row order to minimize SI misses; such a pass could remove the dedicated Scoreboard hardware and its area cost, an extension the paper does not develop.
  • Nothing in the principle limits it to Transformer weights; applying the same reuse analysis to im2col-based convolutions, sparse matrices, or training-time gradients would test whether transitive sparsity is a general tensor property or an LLM-specific one.
  • A direct comparison at the same hardware budget and same sequence lengths, including end-to-end attention plus FC layers, would clarify how much of the 7.46x is algorithmic reuse versus the area saved by removing multipliers; the paper's headline comparisons are per-layer cycle counts.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper introduces 'transitive sparsity' for bit-sliced GEMM: rows of a binary weight matrix that share set bits can reuse partial sums, and the induced set-inclusion partial order is represented as a Hasse graph. A Scoreboard algorithm computes a balanced forest of reuse chains, and the paper presents a corresponding hardware design, the Transitive Array, with tiling, Benes-network dispatch, adder-only PPE/APE units, and static and dynamic Scoreboards. Evaluations on LLaMA-1/2/3 and ResNet-18 compare the design against BitFusion, ANT, Olive, Tender, and BitVert, with headline results of approximately 7.46x and 3.97x speedup and 2.31x and 1.65x energy reduction over Olive and BitVert at similar perplexity, plus a theoretical 8x reduction relative to dense 8-bit GEMM.

Significance. If correct, the paper opens a new axis of reduction beyond bit-level sparsity and weight sparsity, and the 1/T lower bound on achievable density gives the idea a clean theoretical footing. The paper's strengths include the concrete Hasse-graph formulation, the hardware details of the Scoreboard and balanced forest, a cycle-level simulator complemented by RTL synthesis at 28nm, and the fact that the measured 12.57% density for T=8 matches the predicted 1/T bound. The main weakness is that the adder-only datapath is never shown to implement signed two's-complement GEMM; the perplexity parity in Table 3 is currently the only evidence that the arithmetic is correct, which makes the headline speedup and iso-accuracy claims conditional. If the signed-arithmetic gap is closed, the overall contribution would be significant for low-bit LLM inference.

major comments (3)
  1. [Sec. 2.2, Sec. 4.5, Fig. 7(c)] The data path contains only adders, yet the paper claims two's-complement representation. In two's-complement bit-slicing, the MSB plane contributes a negative weight; treating every one-bit as positive 1 (as stated in Sec. 2.2) computes a different value whenever any negative weight is present. No zero-point offset, sign-extension compensation, or subtraction/negation path appears in Fig. 7(c) or Sec. 4.5. Please specify the exact integer encoding used for weights and activations, and either (i) add the sign-plane subtraction and demonstrate it in the PPE/APE data path, or (ii) describe an affine transformation (e.g., zero-point) that maps signed integers to unsigned before bit-slicing and show that the dequantization recovers the original GEMM. Until this is resolved, the perplexity parity in Table 3 cannot be taken as evidence that the simulated and synthesized design computes the same GEMM as the quantized baseline.
  2. [Sec. 5.1, Sec. 5.2, Sec. 5.9] The empirical density argument, namely that only about 1.67% of TransRows have reuse distance greater than 1 and that distance-3 prefixes occur in fewer than 0.1% of rows, is measured only on the first Transformer block at prefill length 2048. The sentence 'all Transformer blocks are identical and exhibit similar computational behavior' is an unverified extrapolation: activation distributions and therefore bit-sliced TransRow distributions can differ across blocks and sequence lengths. Please report density statistics for multiple blocks and at least one other sequence length, or bound the sensitivity of the headline speedups to this assumption. This is load-bearing because if reuse distances degrade, the design approaches the unconditional 1/T lower bound and the speedup over bit-sparse baselines shrinks.
  3. [Sec. 4.5, Table 1] The 12-bit PPE and 24-bit APE widths are asserted to be sufficient ('maintaining sufficiently high precision'), but no range analysis is provided. For an 8-bit TransRow, the largest prefix-sum magnitude depends on the number of accumulated terms and the signed input range, and after accounting for bit-plane shifts and group-wise scale factors the bound is not obvious. Please provide a worst-case bound for all intermediate sums, including multi-hop distance-2 and distance-3 chains, and state the exact bit widths needed for the T=4 and T=8 configurations. If the widths are insufficient, the 'lossless' claim in Sec. 2.1 fails.
minor comments (5)
  1. [Sec. 5.4, Table 3] The BitVert perplexity values are taken from the original paper rather than reproduced under the same setup; this should be stated in the table caption and considered when interpreting the iso-accuracy speedup numbers.
  2. [Sec. 5.1, Table 2] Table 2 lists different PE-array dimensions for the baselines (e.g., 28x32 for BitFusion, 36x64 for ANT) and for TransArray (8x32 per unit), so please clarify how the array sizes and lane counts are normalized in the speedup and energy comparisons.
  3. [Sec. 2.2, Fig. 3] The worked example is internally inconsistent: the input vector is written as (6, -5, -2, 4), while the text says TransRow 1011 accumulates 6, -2, and 4 and skips -5; the bit-to-element alignment should be drawn explicitly.
  4. [Sec. 3.4] The phrase 'reducing significant (T times for T-bit) memory overhead' is awkward and should be rewritten for clarity.
  5. [General] There are several typographical issues, including 'Socreboard' in Table 2, 'Futhermore' in Sec. 4.6, and the repeated duplicate reference entries for BBS/BitVert in the reference list.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: measured speedups, derived 1/T lower bound, and external baselines; flagged concerns are correctness/evidence issues, not circular reductions.

full rationale

The central speedup and energy claims are simulator measurements against externally published baselines, not quantities fitted from the same data they are used to predict. The 1/T lower bound on density is derived directly from bit-slicing arithmetic ('we must perform at least one accumulation operation for every T-bit element'), independent of the paper's own results, and the observed density approaching that bound is an empirical measurement, not a self-defined output. The PPL parity in Table 3 is external, quantized-GEMM evidence (Qserve, ANT-style group quantization), so the iso-accuracy premise is not established by circular reasoning. Self-citations to Olive and ANT appear as comparison baselines and simulator infrastructure, not as justification of the core result. The two substantive reviewer concerns are correctness/evidence issues rather than circularity: the extrapolation from one Transformer block to all blocks is an unsupported empirical assumption, and the absence of an explicit two's-complement sign-plane path in the adder-only datapath is a potential arithmetic correctness gap. Neither concern makes any claimed result equal to its input by construction. No circular step is present.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The hardware configuration (T=8, N=256, adder widths, distance cutoff) is chosen by hand and by empirical sweeps; these choices, not a derivation, set the advertised 8x lower bound. The correctness of the lossless claim hinges on standard integer arithmetic plus two empirical data-distribution assumptions that were only spot-checked. No new physical entities are postulated.

free parameters (5)
  • TransRow width T = 8
    Chosen from design-space exploration (Fig. 9a) as the Pareto-optimal bit-width; T directly sets the 1/T theoretical density lower bound and the maximum reuse granularity.
  • Tiling row size N = 256
    Chosen because overall density stabilizes beyond 256 rows (Fig. 9c); determines Hasse graph occupancy and Scoreboard table size.
  • Distance cutoff for prefix search = 3 (Distance < 4)
    Alg. 1 line 7 stops at distance >= 4; motivated by the measured rarity of distance-3 prefixes, making the cutoff an empirical design choice that affects correctness for sparser tiles.
  • PPE adder width = 12-bit
    Chosen to be 'sufficiently high' for lossless prefix sums; no worst-case bound is derived for the stated input and tile ranges.
  • APE accumulator width = 24-bit
    Same issue; the paper's tiling description uses 32-bit partial sums, so the 24-bit accumulator may be inconsistent, and no overflow proof is given.
assumptions (5)
  • standard math Integer bit-slicing with reordered accumulations is lossless (relies on associativity and commutativity of integer addition).
    Sec. 2.1 states exactness of bit-slicing when addition precision is sufficient, citing prior work [1,49]; this is standard integer arithmetic.
  • domain assumption Signed 2's complement weights can be handled with unsigned TransRows and 'appropriate shifts' using only adders.
    Sec. 2.2 asserts 2's complement representation but never specifies sign-bit subtraction or zero-point correction; the hardware (Sec. 4.5) contains only adders and a shifter.
  • domain assumption Real LLaMA weight and activation bit patterns are subset-rich enough: unique TransRows below the uniform expectation and distance greater than 1 rare.
    Secs. 3.2 and 5.9 report <0.1% distance-3 prefixes and 1.67% distance greater than 1; these statistics back the utilization and speedup claims but were measured on one block.
  • domain assumption All Transformer blocks in LLaMA exhibit similar computational behavior, so a first-block trace represents the whole model.
    Sec. 5.1 states this without quantitative support; different blocks have different weight distributions and KV-cache dynamics.
  • domain assumption Scoreboarding time is hidden behind PPE and APE execution, including sorting and T-way parallel passes completing within the pipeline.
    Sec. 4.6 argues min(n,2^T)/T < n/T, but this ignores the bitonic sorter's O(log^2 n) latency and hardware resource sharing with the compute arrays.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Transitive Array: An Efficient GEMM Accelerator with Result Reuse." pith.science (2026). https://pith.science/paper/NUKTPVHF

@misc{pith2026250416339,
  author       = {Pith},
  title        = {Pith review of: Transitive Array: An Efficient GEMM Accelerator with Result Reuse},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NUKTPVHF}},
  note         = {Machine review of arXiv:2504.16339}
}
abstract

Deep Neural Networks (DNNs) and Large Language Models (LLMs) have revolutionized artificial intelligence, yet their deployment faces significant memory and computational challenges, especially in resource-constrained environments. Quantization techniques have mitigated some of these issues by reducing data precision, primarily focusing on General Matrix Multiplication (GEMM). This study introduces a novel sparsity paradigm, transitive sparsity, which leverages the reuse of previously computed results to substantially minimize computational overhead in GEMM operations. By representing transitive relations using a directed acyclic graph, we develop an efficient strategy for determining optimal execution orders, thereby overcoming inherent challenges related to execution dependencies and parallelism. Building on this foundation, we present the Transitive Array, a multiplication-free accelerator designed to exploit transitive sparsity in GEMM. Our architecture effectively balances computational workloads across multiple parallel lanes, ensuring high efficiency and optimal resource utilization. Comprehensive evaluations demonstrate that the Transitive Array achieves approximately 7.46$\times$ and 3.97$\times$ speedup and 2.31$\times$ and 1.65$\times$ energy reduction compared to state-of-the-art accelerators such as Olive and BitVert while maintaining comparable model accuracy on LLaMA models.

Figures

Figures reproduced from arXiv: 2504.16339 by the authors.

Figure 1
Figure 1. Comparison of bit sparsity and transitive sparsity [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Quantization and bit-slicing. A weight tensor (FP16) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Hasse graph and definition of prefix, suffix, and [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (7 more)
Figure 5
Figure 5. Figure 5: Scoreboarding process. Step 1 : Sort TransRows by Hamming order. Step 2 : Record present TransRows in the Hasse graph. Step 3 (Forward pass): Assign candidate prefixes to all nodes. Step 4 (Backward pass): Propagate suffix requests for nodes without a Distance = 1 pref…
Figure 6
Figure 6. Figure 6: Bit field diagram of 4-bit Scoreboard and prefix and [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: (a) Overview of Transitive Array architecture. (b) [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 9
Figure 9. Figure 9: Design space exploration among various bit-width and tiling row sizes: (a) and (b), and 8-bit TranSparsity with tiling [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Runtime and energy consumption on FC layers of LLaMA models. *BitFusion (8-bit) and Tender (4-bit) exhibit [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: TransArray energy breakdown on LLaMA-1-7B. [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 14
Figure 14. Figure 14: Speedup comparison on ResNet18 models. resembles a uniform distribution. (2) Certain patterns may exist in the weight tensor of DNN models. For uniform random data, the occurrence of repeated values follows the classical problem of dis￾crete uniform distributions. The…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Phi: Leveraging Pattern-based Hierarchical Sparsity for High-Efficiency Spiking Neural Networks

    cs.AR 2025-05 conditional novelty 6.0 of 10

    Phi decomposes SNN activations into pre-computed pattern rows plus sparse +/-1 corrections, yielding a 3.45x speedup and 4.93x energy savings over the Stellar accelerator.

Reference graph

Works this paper leans on

69 extracted references · 30 canonical work pages · cited by 1 Pith paper

  1. [1]

    Jorge Albericio, Alberto Delmas, Patrick Judd, Sayeh Sharify, Gerard O’Leary, Roman Genov, and Andreas Moshovos. 2017. Bit-Pragmatic Deep Neural Network Computing. 50th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) (2017)

    work page 2017

  2. [2]

    Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman

    Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. 2024. QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs. arXiv:2404.00456 [cs.LG] https://arxiv.org/abs/2404.00456

    arXiv 2024

  3. [3]

    Kenneth E Batcher. 1968. Sorting Networks and Their Applications. Proceedings of the April 30–May 2, 1968, Spring Joint Computer Conference (1968), 307–314

    work page 1968

  4. [4]

    Vaclav E. Benes. 1964. Mathematical Theory of Connecting Networks and Tele- phone Traffic. Mathematics in Science and Engineering 17 (1964)

    work page 1964

  5. [5]

    Peter F Brown, Stephen A Della Pietra, Vincent J Della Pietra, Jennifer C Lai, and Robert L Mercer. 1992. An estimate of an upper bound for the entropy of English. Computational Linguistics 18, 1 (1992), 31–40

    work page 1992

  6. [7]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    arXiv 2020

  7. [9]

    Abdelfattah

    Yuzong Chen, Jian Meng, Jae sun Seo, and Mohamed S. Abdelfattah

    work page

  8. [10]

    Sharan Chetlur, Cliff Woolley, Philippe Vandermersch, Jonathan Cohen, John Tran, Bryan Catanzaro, and Evan Shelhamer. 2014. cuDNN: Efficient Primitives for Deep Learning. arXiv:1410.0759 [cs.NE] https://arxiv.org/abs/1410.0759

    arXiv 2014

Show all 69 references
  1. [11]

    Ping Chi, Shuangchen Li, Cong Xu, Tao Zhang, Jishen Zhao, Yongpan Liu, Yu Wang, and Yuan Xie. 2016. Prime: A novel processing-in-memory architecture for neural network computation in reram-based main memory. ACM SIGARCH Computer Architecture News 44, 3 (2016), 27–39

    2016

  2. [12]

    Wikipedia contributors. [n. d.]. Hasse diagram — Wikipedia, The Free Encyclo- pedia. https://en.wikipedia.org/wiki/Hasse_diagram

  3. [13]

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition . Ieee, 248–255

    2009

  4. [14]

    Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

    2018 arXiv

  5. [15]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ah- mad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sra- vankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston...

    2024 arXiv

  6. [16]

    Michael J. Flynn. 1972. Some Computer Organizations and Their Effectiveness. IEEE Trans. Comput. C-21 (1972), 948–960. https://api.semanticscholar.org/ CorpusID:18573685 ISCA ’25, June 21–25, 2025, Tokyo, Japan Cong Guo, Chiyue Wei, Jiaming Tang, Bowen Duan, Song Han, Hai Li, ...

    1972

  7. [17]

    Ian Goodfellow. 2016. Deep learning

    2016

  8. [18]

    Cong Guo, Feng Cheng, Zhixu Du, James Kiessling, Jonathan Ku, Shiyu Li, Ziru Li, Mingyuan Ma, Tergel Molom-Ochir, Benjamin Morris, et al. 2025. A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models. IEEE Circuits and Systems Magazine 25, 1 (2...

    2025

  9. [19]

    Cong Guo, Bo Yang Hsueh, Jingwen Leng, Yuxian Qiu, Yue Guan, Zehuan Wang, Xiaoying Jia, Xipeng Li, Minyi Guo, and Yuhao Zhu. 2020. Accelerating sparse dnn models without hardware-support via tile-wise sparsity. In SC20: International Conference for High Performance Computing, ...

    2020

  10. [20]

    Cong Guo, Yuxian Qiu, Jingwen Leng, Xiaotian Gao, Chen Zhang, Yunxin Liu, Fan Yang, Yuhao Zhu, and Minyi Guo. 2022. SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximation. In International Conference on Learning Representations. https://openreview.net/for...

    2022

  11. [21]

    Cong Guo, Jiaming Tang, Weiming Hu, Jingwen Leng, Chen Zhang, Fan Yang, Yunxin Liu, Minyi Guo, and Yuhao Zhu. 2023. Olive: Accelerating large language models via hardware-friendly outlier-victim pair quantization. In Proceedings of the 50th Annual International Symposium on Co...

    2023

  12. [22]

    Cong Guo, Fengchen Xue, Jingwen Leng, Yuxian Qiu, Yue Guan, Weihao Cui, Quan Chen, and Minyi Guo. 2024. Accelerating sparse dnns based on tiled gemm. IEEE Trans. Comput. (2024)

    2024

  13. [23]

    Cong Guo, Chen Zhang, Jingwen Leng, Zihan Liu, Fan Yang, Yunxin Liu, Minyi Guo, and Yuhao Zhu. 2022. Ant: Exploiting adaptive numerical data type for low-bit deep neural network quantization. In 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO) . IEEE, 1414–1433

    2022

  14. [24]

    Song Han, Xingyu Liu, Huizi Mao, Jing Pu, Ardavan Pedram, Mark A Horowitz, and William J Dally. 2016. EIE: Efficient inference engine on compressed deep neural network. ACM SIGARCH Computer Architecture News 44, 3 (2016), 243– 254

    2016

  15. [25]

    Song Han, Huizi Mao, and William J Dally. 2015. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149 (2015)

    2015 arXiv

  16. [26]

    Helmut Hasse. 1967. Grundlagen der Mathematik in historischer Entwicklung. Mathematisch-Naturwissenschaftliche Bibliothek 31 (1967)

    1967

  17. [27]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition . 770–778

    2016

  18. [28]

    Kevin Hsieh, Eiman Ebrahimi, Gwangsun Kim, Niladrish Chatterjee, Mike O’Connor, Nandita Vijaykumar, Onur Mutlu, and Stephen W Keckler. 2016. Transparent offloading and mapping (TOM) enabling programmer-transparent near-data processing in GPU systems. ACM SIGARCH Computer Archi...

    2016

  19. [29]

    Weiming Hu, Haoyan Zhang, Cong Guo, Yu Feng, Renyang Guan, Zhendong Hua, Zihan Liu, Yue Guan, Minyi Guo, and Jingwen Leng. 2025. M-ANT: Efficient Low- bit Group Quantization for LLMs via Mathematically Adaptive Numerical Type. In 2025 IEEE International Symposium on High Perfo...

    2025

  20. [30]

    Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. 2017. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. arXiv:1712.05877 [cs.LG] https://arxiv.org/abs/1712.05877

    2017 arXiv

  21. [31]

    Dong-Ik Jeon, Kyeong-Bin Park, and Ki-Seok Chung. 2017. HMC-MAC: Processing-in memory architecture for multiply-accumulate operations with hybrid memory cube. IEEE Computer Architecture Letters 17, 1 (2017), 5–8

    2017

  22. [32]

    Norman P. Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, Rick Boyle, Pierre luc Cantin, Clifford Chao, Chris Clark, Jeremy Coriell, Mike Daley, Matt Dau, Jeffrey Dean, Ben Gelb, Tara Vazi...

    2017 arXiv

  23. [33]

    Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling Laws for Neural Language Models. arXiv:2001.08361 [cs.LG] https: //arxiv.org/abs/2001.08361

    2020 arXiv

  24. [34]

    Donald E. Knuth. 2009. The Art of Computer Programming, Volume 4A: Combi- natorial Algorithms, Part 1. InThe Art of Computer Programming. Addison-Wesley Professional

    2009

  25. [35]

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436–444

    2015

  26. [36]

    Jungi Lee, Wonbeom Lee, and Jaewoong Sim. 2024. Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantization. arXiv preprint arXiv:2406.12930 (2024)

    2024 arXiv

  27. [37]

    Yuhang Li, Mingzhu Shen, Jian Ma, Yan Ren, Mingxin Zhao, Qi Zhang, Rui- hao Gong, Fengwei Yu, and Junjie Yan. 2021. Mqbench: Towards reproducible and deployable model quantization benchmark. arXiv preprint arXiv:2111.03759 (2021)

    2021 arXiv

  28. [38]

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration. arXiv:2306.00978 [cs.CL] https://arxiv.org/abs/2306.00978

    2024 arXiv

  29. [39]

    Yujun Lin, Haotian Tang, Shang Yang, Zhekai Zhang, Guangxuan Xiao, Chuang Gan, and Song Han. 2024. QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving. arXiv:2405.04532 [cs.CL] https://arxiv.org/abs/2405. 04532

    2024 arXiv

  30. [40]

    Zihan Liu, Xinhao Luo, Junxian Guo, Wentao Ni, Yangjie Zhou, Yue Guan, Cong Guo, Weihao Cui, Yu Feng, Minyi Guo, et al. 2025. VQ-LLM: High-performance Code Generation for Vector Quantization Augmented LLM Inference. In 2025 IEEE International Symposium on High Performance Comp...

    2025

  31. [41]

    Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choud- hary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tij- men Blankevoort. 2024. SpinQuant: LLM quantization with learned rotations. arXiv:2405.16406 [cs.LG] https://arxiv.org/abs/2405.16406

    2024 arXiv

  32. [42]

    Hang Lu, Liang Chang, Chenglong Li, Zixuan Zhu, Shengjian Lu, Yanhuan Liu, and Mingzhe Zhang. 2021. Distilling Bit-level Sparsity Parallelism for Gen- eral Purpose Deep Learning Acceleration. 54th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) (2021)

    2021

  33. [43]

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843 (2016)

    2016 arXiv

  34. [44]

    NVIDIA Corporation. 2020. NVIDIA A100 Tensor Core GPU Architecture . Tech- nical Report. NVIDIA Corporation. https://images.nvidia.com/aem-dam/en- zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf Accessed: 2024-11-23

    2020

  35. [45]

    Eunhyeok Park, Dongyoung Kim, and Sungjoo Yoo. 2018. Energy-efficient neural network ‘accel’erator based on outlier-aware low-precision computation. In 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA). IEEE, 688–698

    2018

  36. [46]

    Eric Qin, Ananda Samajdar, Hyoukjun Kwon, Vineet Nadella, Sudarshan Srini- vasan, Dipankar Das, Bharat Kaul, and Tushar Krishna. 2020. Sigma: A sparse and irregular gemm accelerator with flexible interconnects for dnn training. In 2020 IEEE International Symposium on High Perf...

    2020

  37. [47]

    Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. 2024. Omni- Quant: Omnidirectionally Calibrated Quantization for Large Language Models. arXiv:2308.13137 [cs.LG] https://arxiv.org/abs/2308.13137

    2024 arXiv

  38. [48]

    Hardik Sharma, Jongse Park, Naveen Suda, Liangzhen Lai, Benson Chau, Joon Kyung Kim, Vikas Chandra, and Hadi Esmaeilzadeh. 2018. Bit fusion: Bit- level dynamically composable architecture for accelerating deep neural network. In 2018 ACM/IEEE 45th Annual International Symposiu...

    2018

  39. [49]

    Man Shi, Vikram Jain, Antony Joseph, Maurice Meijer, and Marian Verhelst

  40. [50]

    Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2020. Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism. arXiv:1909.08053 [cs.CL] https: //arxiv.org/abs/1909.08053

    2020 arXiv

  41. [51]

    Proceedings of the 30th IEEE International Symposium on High- Performance Computer Architecture (HPCA) (2024)

    BitWave: Exploiting Column-Based Bit-Level Sparsity for Deep Learning Acceleration. Proceedings of the 30th IEEE International Symposium on High- Performance Computer Architecture (HPCA) (2024)

    2024

  42. [52]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023)

    2023 arXiv

  43. [53]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  44. [54]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2023. Attention Is All You Need. arXiv:1706.03762 [cs.CL] https://arxiv.org/abs/1706.03762

    2023 arXiv

  45. [55]

    A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems (2017)

    2017

  46. [56]

    Zhongwei Wan, Xin Wang, Che Liu, Samiul Alam, Yu Zheng, Jiachen Liu, Zhong- nan Qu, Shen Yan, Yi Zhu, Quanlu Zhang, Mosharaf Chowdhury, and Mi Zhang

  47. [57]

    Abraham Waksman. 1968. A permutation network. In 8th Annual Symposium on Switching and Automata Theory . IEEE, 111–117. Transitive Array: An Efficient GEMM Accelerator with Result Reuse ISCA ’25, June 21–25, 2025, Tokyo, Japan

    1968

  48. [58]

    Yang Wang, Chen Zhang, Zhiqiang Xie, Cong Guo, Yunxin Liu, and Jingwen Leng

  49. [59]

    arXiv:2312.03863 [cs.CL] https://arxiv.org/abs/2312.03863

    Efficient Large Language Models: A Survey. arXiv:2312.03863 [cs.CL] https://arxiv.org/abs/2312.03863

    arXiv

  50. [60]

    Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma, Fan Yang, Ruiping Wang, Yi Wu, and Furu Wei. 2023. BitNet: Scaling 1- bit Transformers for Large Language Models. arXiv:2310.11453 [cs.CL] https: //arxiv.org/abs/2310.11453

    2023 arXiv

  51. [61]

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han

  52. [62]

    Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Con- glong Li, and Yuxiong He. 2022. ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers. In Advances in Neural Information Processing Systems , S. Koyejo, S. Mohamed, A. Agar...

    2022

  53. [63]

    Chiyue Wei, Cong Guo, Feng Cheng, Shiyu Li, Hao Frank Yang, Hai Helen Li, and Yiran Chen. 2025. Prosperity: Accelerating Spiking Neural Networks via Product Sparsity. In 2025 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 806–820

    2025

  54. [64]

    Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. 2016. Learning structured sparsity in deep neural networks. Advances in neural information processing systems 29 (2016)

    2016

  55. [65]

    Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, and Beidi Chen. [n. d.]. H$_2$O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models. arXiv:2306...

    arXiv

  56. [66]

    arXiv:2211.10438 [cs.CL] https://arxiv.org/abs/2211.10438

    SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models. arXiv:2211.10438 [cs.CL] https://arxiv.org/abs/2211.10438

    arXiv

  57. [68]

    Ali Hadi Zadeh, Isak Edo, Omar Mohamed Awad, and Andreas Moshovos. 2020. Gobo: Quantizing attention-based nlp models for low latency and energy ef- ficient inference. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 811–824

    2020

  58. [69]

    Chen Zhang, Yang Wang, Zhiqiang Xie, Cong Guo, Yunxin Liu, Jingwen Leng, Guangyu Sun, Zhigang Ji, Runsheng Wang, Yuan Xie, et al. 2024. DSTC: Dual- Side Sparsity Tensor Core for DNNs Acceleration on Modern GPU Architectures. IEEE Trans. Comput. (2024)

    2024

  59. [71]

    Maohua Zhu, Tao Zhang, Zhenyu Gu, and Yuan Xie. 2019. Sparse tensor core: Algorithm and hardware co-design for vector-wise sparse neural networks on modern gpus. In Proceedings of the 52nd Annual IEEE/ACM International Sympo- sium on Microarchitecture. 359–371

    2019

  60. [2021]

    In 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA)

    Dual-side sparse tensor core. In 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA) . IEEE, 1083–1095

    2021

  61. [2024]

    arXiv:2409.05227 [cs.LG] https://arxiv.org/abs/2409.05227

    BBS: Bi-directional Bit-level Sparsity for Deep Learning Acceleration. arXiv:2409.05227 [cs.LG] https://arxiv.org/abs/2409.05227

    arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.