Pith. sign in

REVIEW 3 major objections 4 minor 44 references

Accelerating LLM Inference with Flexible N:M Sparsity via A Fully Digital Compute-in-Memory Accelerator

T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Tuning both $N$ and block size $M$ per layer, based on outlier density and spread, preserves more LLM accuracy at high sparsity than fixed or $N$-only $N{:}M$ patterns, while a partitioned digital compute-in-memory macro accelerates the…

desk verdict FLOW + FlexCiM is a credible algorithm-hardware co-design that extends N:M pruning to per-layer N and M and builds a low-overhead DCiM macro for it; the pruning results look solid, but the headline hardware numbers rest on an all-weights-on-chip assumption that does not hold for the evaluated 7B/70B models. read the letter →

arxiv 2504.14365 v1 pith:34CWI64X submitted 2025-04-19 cs.LG cs.AIcs.AR

Akshat Ramachandran , Souvik Kundu , Arnab Raha , Shamik Kundu , Deepak K. Mathaikutty , Tushar Krishna This is my paper · ORCID
classification cs.LGcs.AIcs.AR
keywords N:Msparsitystructuredpruningoutlier-awaredigitalcompute-in-memoryLLMinferenceaccelerationlayer-wiseFLOWFlexCiM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FLOW is a pruning-side method that assigns each layer of an LLM its own $N{:}M$ structured-sparsity pattern by looking at both the fraction of outlier weights and how spread out those outliers are. FlexCiM is a companion digital compute-in-memory accelerator that can execute any supported $N{:}M$ pattern without rebuilding the macro, by splitting a macro into sub-macros and adding distribution and merging units. The paper's central claim is that this paired flexibility is worth having: FLOW keeps perplexity closer to the dense model than fixed $N{:}M$ or $N$-only-varied baselines at 50-60% sparsity, and FlexCiM delivers up to 1.75x lower inference latency and 1.5x lower energy than existing sparse accelerators with only about 6% area overhead. If correct, it means one memory-compute macro can serve a whole family of sparsity patterns, so the choice of sparsity shape becomes a per-layer optimization problem rather than a hardware constraint.

What carries the argument

The load-bearing mechanism is the combination of two per-layer outlier signals with a partitioned-macro dataflow. FLOW computes the outlier fraction $O_l$ from an importance score based on weight magnitude times input-activation L2 norm, thresholded at a multiple of the layer's standard deviation, and the outlier distribution $D_l$ as the mean pairwise L1 distance among outliers inside 128x128 blocks, normalized across layers. An integer linear program then assigns $N_l$ and $M_l$ so that $N$ tracks the outlier fraction and $M$ tracks one minus the normalized outlier distribution, subject to the average sparsity budget. On the hardware side, FlexCiM splits a 128x32x8 DCiM macro into $P=4$ sub-macros; a distribution unit of $P:1$ multiplexers feeds activations to sub-macros, CSC metadata selects which sub-macro receives which input, and a merging adder tree combines partial sums. This division of labor keeps large multiplexers out of the memory cells and confines per-cell logic to a 2:1 mux, which is why flexible $N{:}M$ costs only about 6% area.

What would settle it

Run the paper's cycle-accurate simulator with the global SRAM assumption removed for a 70B-class 8-bit model, letting weights stream from DRAM, and compare FlexCiM's end-to-end energy and latency against the dense systolic-array baseline at iso-throughput; if the advantage drops below the claimed 1.5x energy / 1.75x latency, the end-to-end efficiency claim is refuted.

Watch

Extended reading notes

Core claim

The central claim is that both dimensions of $N{:}M$ sparsity should vary by layer, not just $N$. A layer's tolerance to pruning is set by two separate properties of its outliers: their abundance (presence) and their pairwise distance (distribution). Clustered outliers call for a larger block $M$ so the pruning pass has room to keep them; sparse, low-count outliers call for smaller blocks and lower $N$. FLOW encodes this as an integer program with $N$ restricted to 1, 2, 4, or 8 and $M$ restricted to 2, 4, or 8, using the outlier fraction and a normalized outlier-distribution score to guide the assignment while matching a target average sparsity. On the hardware side, FlexCiM partitions one DCiM macro into four row-wise sub-macros and uses distribution and merging units so that a block of $M$ activations can be handled by one or several sub-macros according to $N$. The paper reports that FLOW improves zero-shot accuracy by up to 36% relative to prior pruning methods and improves perplexity by up to 18% over OWL-style $N$-only assignment, while FlexCiM reduces inference latency by up to 1.75x and energy by up to 1.5x versus the sparse accelerators it is compared with.

Load-bearing premise

The load-bearing premise is that all model weights fit inside the on-chip global SRAM, so the energy and latency comparisons exclude DRAM access; for a 70B-parameter model even at 8-bit precision this is optimistic, and any weight streaming from DRAM would erode the reported gains.

Editorial extensions

If this is right

  • A FLOW-pruned model keeps WikiText2 perplexity closer to the dense baseline than fixed 4:8/3:8 patterns and than OWL's $N$-only assignment at the same target sparsity.
  • FlexCiM accelerates dense, 1:2, 1:4, 2:4, 1:8, 2:8, and 4:8 patterns in one macro, so layer-wise heterogeneous sparsity no longer forces a fixed hardware pattern.
  • The 1.75x latency and 1.5x energy gains come with flexible $N{:}M$ support at about 6% area overhead over a fixed 1:2 sparse DCiM.
  • FLOW also improves perplexity of Mamba-based state space models, so the method is not tied to transformers.
  • Under an unconstrained sparsity budget, FLOW achieves both higher average sparsity and lower perplexity than OWL on all tested models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If a benchmark includes DRAM traffic for a 70B-class model, the end-to-end advantage over dense baselines would likely shrink below the headline 1.5x energy figure, because weight streaming adds a cost both designs pay.
  • The outlier-distribution score could serve as a general layer-difficulty signal for other compression choices, such as per-layer quantization bit width or mixture-of-experts routing; the paper does not test those uses.
  • Allowing non-power-of-two $M$ values would expand the representational freedom FLOW can select, at the price of wider multiplexers in the distribution unit; that trade-off is not explored.
  • A sensitivity study of the ILP weights would show whether the chosen $N{:}M$ patterns transfer across model families without retuning; the paper fixes these values once.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. FLOW is a post-training pruning method that assigns layer-wise N:M sparsity patterns to LLM weights based on both the fraction and the spatial distribution of outliers, formulated as an integer linear program (Eq. 2). FlexCiM is a digital compute-in-memory (DCiM) macro that supports multiple N:M patterns by partitioning the macro into sub-macros and adding distribution and merging units. The paper evaluates FLOW on transformer LLMs, SSMs, and VLMs, reporting perplexity and zero-shot accuracy gains over SparseGPT, Wanda, and OWL, and evaluates FlexCiM via Verilog RTL synthesis and a cycle-accurate simulator, reporting up to 1.75x lower latency and 1.5x lower energy than VEGETA and SDP baselines. The two central caveats are that the hardware claims assume all model weights fit in global SRAM, which is unrealistic for the evaluated 7B/70B class, and the 'optimal' N:M selection is asserted from an ILP with hand-picked hyperparameters and no oracle comparison.

Significance. If the results hold, the paper makes a useful contribution: it identifies outlier distribution as a signal for choosing M per layer, provides a concrete ILP-based allocation, and demonstrates a low-overhead hardware mechanism for flexible N:M sparsity in DCiM. Strengths include machine-checked RTL synthesis, a cycle-accurate simulator, public code release, and evaluation across a broad set of models including SSMs and VLMs. The perplexity improvements over OWL at 60% sparsity and the up-to-36% zero-shot accuracy gains are substantial and externally evaluated on WikiText2 and standard tasks, so the algorithm-side results are not circular. However, the significance is tempered by the on-chip SRAM assumption in the hardware evaluation, which makes the 1.75x/1.5x claims upper bounds for the models actually tested, and by the lack of an oracle baseline to validate the 'optimal' selection language.

major comments (3)
  1. [VI-A and VI-C] Sections VI-A and VI-C: The energy and latency claims (Figures 1 and 7) are computed under the assumption that 'all model parameters fit within the global SRAM for simplicity' (Section VI-A). For the evaluated LLaMA2-7B and LLaMA3-70B at 8-bit weights, this requires roughly 7 GB and 70 GB of on-chip SRAM, which is not realistic for a DCiM system. Since LLM decode is memory-bound, once weights must be fetched from off-chip DRAM, the reported 1.5x energy and 1.75x latency advantages over dense and VEGETA baselines are dominated by DRAM access; the statement in Section VI-C that off-chip access 'will be approximately similar for DCiM or digital accelerators' does not compensate, because equal DRAM cost can mask the claimed benefits, and if all designs are DRAM-bandwidth limited the latency gap can collapse. The paper should either re-scope the hardware claims to models that fit on-chip or add a DRAM-inclusive memory model to the simulation.
  2. [IV-B, Eq. (2)] Eq. (2), Section IV-B: The ILP objective encodes the paper's own hypothesis that N_l is proportional to outlier fraction O_l and M_l is proportional to (1 - ND_l), with hyperparameters (alpha, beta, k, h) = (1, 4, 8, 8) chosen empirically on the evaluated models. Calling the result 'optimal N:M' is unsupported because no comparison against an oracle search over the feasible set (N in {1,2,4,8}, M in {2,4,8}) is reported. Since this search space is only 12 patterns per layer, an exhaustive-oracle baseline is computationally cheap and would directly validate whether the ILP objective and its weights are justified. Without it, the central algorithmic claim of optimality is an assertion rather than a demonstrated result.
  3. [IV-A] Section IV-A: The outlier threshold tau is stated as 'we empirically identify tau = 3 or tau = 5 provide the best results,' but no ablation is shown. Because tau directly determines O_l and ND_l, which drive the entire FLOW allocation in Eq. (2), the sensitivity of final perplexity and zero-shot accuracy to tau should be reported for at least one model. This is load-bearing for the algorithm's central claim: if results change significantly with tau, the method's robustness and the 'empirically identify' statement need stronger support.
minor comments (4)
  1. [Abstract, VI-C, Conclusion] The latency improvement number is inconsistent: the abstract and conclusion say 'up to 1.75x lower inference latency,' while Section VI-C reports 'up to 1.72x' for the same results. Please harmonize these numbers.
  2. [III-A, Eq. (1)] The prose before Eq. (1) says D_l measures 'the summation of the pairwise distance between the outliers averaged over the number of outliers,' but the equation shows an average over nC2 outlier pairs. Please make the description match the equation.
  3. [IV-B] The block size of 128x128 used to compute D_l^b is not justified; since it is a hyperparameter that affects ND_l, adding a one-sentence rationale or a small sensitivity check would improve reproducibility.
  4. [Figure 5] Figure 5 is very dense and difficult to parse; the walkthrough in Section V-D would benefit from larger fonts, clearer sub-macro labels, and a more explicit mapping between the metadata bits and the multiplexer select lines.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: FLOW's ILP is a stated heuristic validated on external benchmarks, and FlexCiM's hardware results come from an independent RTL/simulator comparison.

full rationale

The paper's central FLOW claim is that per-layer N:M selection guided by outlier fraction O_l and outlier distribution ND_l improves LLM perplexity and accuracy. The ILP in Eq. (2) does encode the paper's own heuristic relationships (N_l ∝ O_l and M_l ∝ 1−ND_l), and the weights (α,β,k,h)=(1,4,8,8) are empirically chosen; in that sense 'optimal' is relative to the stated objective rather than derived from an independent theorem. This is a modeling assumption, not a circular derivation: the ILP objective does not contain the test metrics (WikiText2 PPL, zero-shot accuracy), and the evaluation compares the resulting sparse models against SparseGPT, Wanda, OWL, and magnitude baselines on external benchmarks. The hardware latency/energy claims are produced by an RTL synthesis plus cycle-accurate simulator flow against VEGETA and SDP baselines; the 'all parameters fit in the global SRAM' assumption is an optimistic modeling boundary that limits external validity, but it is not an input that is later reported as the predicted output. The self-citations ([25], [26], [27]) are background/related-work references and do not carry the derivation. No step was found where a fitted parameter is renamed as a prediction or where a claimed result equals its own input by construction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 2 invented entities

The central algorithm depends on outlier thresholds, ILP weights, scaling factors, and a block size that are all selected by hand or by fitting to the evaluated models. The hardware claims rely on two new units that exist only in synthesis and simulation, and on an optimistic memory model that excludes DRAM. These are the main things a reader would need to independently verify.

free parameters (5)
  • tau (outlier threshold) = 3 or 5
    Weights with importance score above tau times sigma are classified as outliers; the paper says tau equals 3 or 5 provides the best results (Section IV-A). No sensitivity analysis is shown.
  • alpha and beta (ILP objective weights) = alpha=1, beta=4
    Trade-off between matching outlier fraction for N and outlier distribution for M in Eq. 2; chosen empirically, no sensitivity analysis.
  • k and h (scaling factors in ILP) = k=8, h=8
    Normalize outlier fraction and distribution to the same scale as the N and M choice sets; chosen empirically.
  • block size for outlier distribution = 128x128
    The layer is partitioned into 128x128 blocks to compute pairwise outlier distances (Section IV-B); no ablation over block size is reported.
  • partition size P in FlexCiM = 4
    The DCiM macro is split into P sub-macros; P=4 is the demonstrated implementation but the paper does not sweep P.
assumptions (4)
  • domain assumption The Wanda importance score |Wij| * ||Xj||2 is a valid proxy for weight importance and outlier identification (Section IV-A).
    Borrowed from Wanda [33]; not re-derived or separately evaluated in this paper.
  • domain assumption The mean-plus-tau*sigma threshold on importance scores correctly separates outliers from inliers (Section IV-A).
    Adopted from OWL [38]; the paper states tau is chosen empirically.
  • ad hoc to paper Layers with clustered outliers prefer larger M values, and this relationship is monotonic in (1-ND_l) (Sections III-A and IV-B).
    This is the core hypothesis of FLOW; the paper labels it 'We hypothesize' and provides no independent test that the ranking is optimal.
  • ad hoc to paper The ILP hyperparameters alpha, beta, k, and h chosen on the evaluated models generalize to other models and tasks.
    The paper fixes these values globally; no cross-validation or sensitivity study is reported.
invented entities (2)
  • Distribution unit
    purpose: Routes input activations to the P sub-macros based on N:M metadata, replacing large per-cell multiplexers (Section V-B).
    Implemented in RTL and synthesized, but not fabricated; area and power estimates come from synthesis and simulation only.
  • Merging unit
    purpose: Accumulates partial sums from sub-macros to produce the final output for flexible N:M patterns (Section V-B).
    Same status as the distribution unit: designed and simulated, with no silicon measurement.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Accelerating LLM Inference with Flexible N:M Sparsity via A Fully Digital Compute-in-Memory Accelerator." pith.science (2026). https://pith.science/paper/34CWI64X

@misc{pith2026250414365,
  author       = {Pith},
  title        = {Pith review of: Accelerating LLM Inference with Flexible N:M Sparsity via A Fully Digital Compute-in-Memory Accelerator},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/34CWI64X}},
  note         = {Machine review of arXiv:2504.14365}
}
read the original abstract

Large language model (LLM) pruning with fixed N:M structured sparsity significantly limits the expressivity of the sparse model, yielding sub-optimal performance. In contrast, supporting multiple N:M patterns to provide sparse representational freedom introduces costly overhead in hardware. To address these challenges for LLMs, we first present a flexible layer-wise outlier-density-aware N:M sparsity (FLOW) selection method. FLOW enables the identification of optimal layer-wise N and M values (from a given range) by simultaneously accounting for the presence and distribution of outliers, allowing a higher degree of representational freedom. To deploy sparse models with such N:M flexibility, we then introduce a flexible, low-overhead digital compute-in-memory architecture (FlexCiM). FlexCiM supports diverse sparsity patterns by partitioning a digital CiM (DCiM) macro into smaller sub-macros, which are adaptively aggregated and disaggregated through distribution and merging mechanisms for different N and M values. Extensive experiments on both transformer-based and recurrence-based state space foundation models (SSMs) demonstrate that FLOW outperforms existing alternatives with an accuracy improvement of up to 36%, while FlexCiM achieves up to 1.75x lower inference latency and 1.5x lower energy consumption compared to existing sparse accelerators. Code is available at: https://github.com/FLOW-open-project/FLOW

Figures

Figures reproduced from arXiv: 2504.14365 by the authors.

Figure 1
Figure 1. Normalized energy comparison between several digital and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) Outlier distribution measured by pairwise [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Example of efficient N:M assignment based on outlier [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: (a) FlexCiM overview with a partition size [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Example of FlexCiM running 1:4 and 4:8 sparsity patterns. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: (a) Accuracy (%) comparison across 4 zero-shot tasks with [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Normalized performance comparison between different dense [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 20 canonical work pages

  1. [1]

    Metron: Holistic performance evaluation framework for llm inference systems,

    A. Agrawal, A. Agarwal, N. Kedia, J. Mohan, S. Kundu, N. Kwatra, R. Ramjee, and A. Tumanov, “Metron: Holistic performance evaluation framework for llm inference systems,” arXiv preprint:2407.07000, 2024

    arXiv 2024

  2. [2]

    Openflamingo: An open-source framework for training large autoregressive vision-language models,

    A. Awadalla, I. Gao, J. Gardner, J. Hessel, Y . Hanafy, W. Zhu, K. Marathe, Y . Bitton, S. Gadre, S. Sagawa et al., “Openflamingo: An open-source framework for training large autoregressive vision-language models,” arXiv preprint arXiv:2308.01390 , 2023

    arXiv 2023

  3. [3]

    Piqa: Reasoning about physical commonsense in natural language,

    Y . Bisk, R. Zellers, J. Gao, Y . Choi et al. , “Piqa: Reasoning about physical commonsense in natural language,” in Proceedings of the AAAI conference on artificial intelligence , vol. 34, no. 05, 2020, pp. 7432– 7439

    2020

  4. [4]

    Microsoft coco captions: Data collection and evaluation server,

    X. Chen, H. Fang, T.-Y . Lin, R. Vedantam, S. Gupta, P. Doll ´ar, and C. L. Zitnick, “Microsoft coco captions: Data collection and evaluation server,” arXiv preprint arXiv:1504.00325 , 2015

    arXiv 2015

  5. [5]

    15.3 a 65nm 3t dynamic analog ram- based computing-in-memory macro and cnn accelerator with retention enhancement, adaptive analog sparsity and 44tops/w system energy efficiency,

    Z. Chen, X. Chen, and J. Gu, “15.3 a 65nm 3t dynamic analog ram- based computing-in-memory macro and cnn accelerator with retention enhancement, adaptive analog sparsity and 44tops/w system energy efficiency,” in 2021 IEEE International Solid-State Circuits Conference (ISSCC), vol. 64. IEEE, 2021, pp. 240–242

    work page 2021

  6. [6]

    Boolq: Exploring the surprising difficulty of natural yes/no questions,

    C. Clark, K. Lee, M.-W. Chang, T. Kwiatkowski, M. Collins, and K. Toutanova, “Boolq: Exploring the surprising difficulty of natural yes/no questions,” arXiv preprint arXiv:1905.10044 , 2019

    arXiv 1905

  7. [7]

    Towards Efficient SRAM-PIM Architecture Design by Exploiting Unstructured Bit-Level Sparsity

    C. Duan et al. , “Towards efficient sram-pim architecture de- sign by exploiting unstructured bit-level sparsity,” arXiv preprint arXiv:2404.09497, 2024

    work page Pith review arXiv 2024

  8. [8]

    Integrating nvidia deep learning accelerator (nvdla) with risc-v soc on firesim,

    F. Farshchi, Q. Huang, and H. Yun, “Integrating nvidia deep learning accelerator (nvdla) with risc-v soc on firesim,” in 2019 2nd Workshop on Energy Efficient Machine Learning and Cognitive Computing for Embedded Applications (EMC2) . IEEE, 2019, pp. 21–25

    work page 2019

Show all 44 references
  1. [9]

    SparseGPT: Massive language models can be accurately pruned in one-shot,

    E. Frantar and D. Alistarh, “SparseGPT: Massive language models can be accurately pruned in one-shot,” 2023

    2023

  2. [10]

    A 5-nm 254-tops/w 221-tops/mm 2 fully-digital computing-in-memory macro supporting wide-range dynamic-voltage- frequency scaling and simultaneous mac and write operations,

    H. Fujiwara et al. , “A 5-nm 254-tops/w 221-tops/mm 2 fully-digital computing-in-memory macro supporting wide-range dynamic-voltage- frequency scaling and simultaneous mac and write operations,” in 2022 IEEE International Solid-State Circuits Conference (ISSCC) , vol. 65. IEEE...

    2022

  3. [11]

    The pile: An 800gb dataset of diverse text for language modeling,

    L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima et al. , “The pile: An 800gb dataset of diverse text for language modeling,” arXiv preprint arXiv:2101.00027 , 2020

    2020 arXiv

  4. [12]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752 , 2023

    2023 arXiv

  5. [13]

    Sparsity-aware and re-configurable npu architecture for samsung flagship mobile soc,

    J.-W. Jang et al. , “Sparsity-aware and re-configurable npu architecture for samsung flagship mobile soc,” in ISCA, 2021, pp. 15–28

    2021

  6. [14]

    Vegeta: Vertically-integrated extensions for sparse/dense gemm tile acceleration on cpus,

    G. Jeong, S. Damani, A. R. Bambhaniya, E. Qin, C. J. Hughes, S. Subramoney, H. Kim, and T. Krishna, “Vegeta: Vertically-integrated extensions for sparse/dense gemm tile acceleration on cpus,” in 2023 IEEE International Symposium on High-Performance Computer Archi- tecture (HPC...

    2023

  7. [15]

    Mixtral of experts,

    A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch, B. Savary, C. Bam- ford, D. S. Chaplot, D. d. l. Casas, E. B. Hanna, F. Bressand et al. , “Mixtral of experts,” arXiv preprint arXiv:2401.04088 , 2024

    2024 arXiv

  8. [16]

    Colonnade: A reconfigurable sram-based digital bit- serial compute-in-memory macro for processing neural networks,

    H. Kim et al. , “Colonnade: A reconfigurable sram-based digital bit- serial compute-in-memory macro for processing neural networks,” IEEE Journal of Solid-State Circuits , vol. 56, no. 7, pp. 2221–2233, 2021

    2021

  9. [17]

    Laxor: A bit-accurate bnn accelerator with latch-xor logic for local computing,

    D. Li, T. Yamasaki, A. Mani, A. T. Do, N. Chen, and B. Wang, “Laxor: A bit-accurate bnn accelerator with latch-xor logic for local computing,” in 2023 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED) . IEEE, 2023, pp. 1–6

    2023

  10. [18]

    Awq: Activation-aware weight quanti- zation for on-device llm compression and acceleration,

    J. Lin, J. Tang, H. Tang, S. Yang, W.-M. Chen, W.-C. Wang, G. Xiao, X. Dang, C. Gan, and S. Han, “Awq: Activation-aware weight quanti- zation for on-device llm compression and acceleration,” Proceedings of Machine Learning and Systems , vol. 6, pp. 87–100, 2024

    2024

  11. [19]

    33.2 a fully integrated analog reram based 78.4 tops/w compute-in-memory chip with fully parallel mac computing,

    Q. Liu, B. Gao, P. Yao, D. Wu, J. Chen, Y . Pang, W. Zhang, Y . Liao, C.-X. Xue, W.-H. Chen et al. , “33.2 a fully integrated analog reram based 78.4 tops/w compute-in-memory chip with fully parallel mac computing,” in 2020 IEEE International Solid-State Circuits Conference- (...

    2020

  12. [20]

    S2ta: Exploiting structured sparsity for energy-efficient mobile cnn acceleration,

    Z.-G. Liu, P. N. Whatmough, Y . Zhu, and M. Mattina, “S2ta: Exploiting structured sparsity for energy-efficient mobile cnn acceleration,” in 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 2022, pp. 573–586

    2022

  13. [21]

    An analysis of neural language modeling at multiple scales,

    S. Merity, N. S. Keskar, and R. Socher, “An analysis of neural language modeling at multiple scales,” arXiv preprint arXiv:1803.08240 , 2018

    2018 arXiv

  14. [22]

    Introducing meta llama 3: The most capable openly available llm to date,

    A. Meta, “Introducing meta llama 3: The most capable openly available llm to date,” Meta AI, 2024

    2024

  15. [23]

    Accelerating sparse deep neural networks,

    A. Mishra, J. A. Latorre, J. Pool, D. Stosic, D. Stosic, G. Venkatesh, C. Yu, and P. Micikevicius, “Accelerating sparse deep neural networks,” arXiv preprint arXiv:2104.08378 , 2021

    2021 arXiv

  16. [24]

    Scnn: An accelerator for compressed-sparse convolutional neural networks,

    A. Parashar, M. Rhu, A. Mukkara, A. Puglielli, R. Venkatesan, B. Khailany, J. Emer, S. W. Keckler, and W. J. Dally, “Scnn: An accelerator for compressed-sparse convolutional neural networks,” ACM SIGARCH computer architecture news , vol. 45, no. 2, pp. 27–40, 2017

    2017

  17. [25]

    FlexNN: A dataflow-aware flexible deep learning accelerator for energy-efficient edge devices,

    A. Raha, D. A. Mathaikutty, S. K. Ghosh, and S. Kundu, “FlexNN: A dataflow-aware flexible deep learning accelerator for energy-efficient edge devices,” arXiv preprint:2403.09026, 2024

    2024 arXiv

  18. [26]

    MicroScopiQ: Acceler- ating foundational models through outlier-aware microscaling quantiza- tion,

    A. Ramachandran, S. Kundu, and T. Krishna, “MicroScopiQ: Acceler- ating foundational models through outlier-aware microscaling quantiza- tion,” arXiv preprint arXiv:2411.05282 , 2024

    2024 arXiv

  19. [27]

    Algorithm-hardware co-design of distribution-aware logarithmic-posit encodings for efficient dnn inference,

    A. Ramachandran, Z. Wan, G. Jeong, J. Gustafson, and T. Krishna, “Algorithm-hardware co-design of distribution-aware logarithmic-posit encodings for efficient dnn inference,” arXiv:2403.05465, 2024

    2024 arXiv

  20. [28]

    Deepscaletool: A tool for the accurate esti- mation of technology scaling in the deep-submicron era,

    S. Sarangi and B. Baas, “Deepscaletool: A tool for the accurate esti- mation of technology scaling in the deep-submicron era,” in 2021 IEEE ISCAS. IEEE, 2021, pp. 1–5

    2021

  21. [29]

    Dnnweaver: From high-level deep network models to fpga acceleration,

    H. Sharma, J. Park, E. Amaro, B. Thwaites, P. Kotha, A. Gupta, J. K. Kim, A. Mishra, and H. Esmaeilzadeh, “Dnnweaver: From high-level deep network models to fpga acceleration,” inthe Workshop on Cognitive Architectures, 2016

    2016

  22. [30]

    Towards vqa models that can read,

    A. Singh, V . Natarajan, M. Shah, Y . Jiang, X. Chen, D. Batra, D. Parikh, and M. Rohrbach, “Towards vqa models that can read,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 8317–8326

    2019

  23. [31]

    Sp-imc: A sparsity aware in-memory-computing macro in 28nm cmos with configurable sparse representation for highly sparse dnn workloads,

    A. Sridharan, F. Zhang, J.-S. Seo, and D. Fan, “Sp-imc: A sparsity aware in-memory-computing macro in 28nm cmos with configurable sparse representation for highly sparse dnn workloads,” in 2024 IEEE Custom Integrated Circuits Conference (CICC) . IEEE, 2024, pp. 1–2

    2024

  24. [32]

    A fully-digital and row-pipelined compute-in-memory neural network accelerator with soc-level benchmarking for ar/vr applications,

    H. E. Sumbul, J.-s. Seo, D. H. Morris, and E. Beigne, “A fully-digital and row-pipelined compute-in-memory neural network accelerator with soc-level benchmarking for ar/vr applications,” IEEE Micro, 2023

    2023

  25. [33]

    A simple and effective pruning approach for large language models,

    M. Sun, Z. Liu, A. Bair, and J. Z. Kolter, “A simple and effective pruning approach for large language models,” ICLR, 2024

    2024

  26. [34]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron et al. , “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint:2307.09288, 2023

    2023 arXiv

  27. [35]

    Sdp: Co-designing algorithm, dataflow, and architecture for in-sram sparse nn acceleration,

    F. Tu, Y . Wang, L. Liang, Y . Ding, L. Liu, S. Wei, S. Yin, and Y . Xie, “Sdp: Co-designing algorithm, dataflow, and architecture for in-sram sparse nn acceleration,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , vol. 42, no. 1, pp. 109–121, 2022

    2022

  28. [36]

    Highlight: Efficient and flexible dnn acceleration with hierarchical structured sparsity,

    Y . N. Wu, P.-A. Tsai, S. Muralidharan, A. Parashar, V . Sze, and J. Emer, “Highlight: Efficient and flexible dnn acceleration with hierarchical structured sparsity,” in Proceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture , 2023, pp. 1106–1120

    2023

  29. [37]

    A task-centric angle of llm pre-trained weights through sparsity,

    L. Yin, S. Liu, A. Jaiswal, S. Kundu, and Z. Wang, “A task-centric angle of llm pre-trained weights through sparsity,” ICML, 2024

    2024

  30. [38]

    Outlier weighed layerwise sparsity: A missing secret sauce for pruning llms to high sparsity,

    L. Yin, Y . Wu, Z. Zhang, C.-Y . Hsieh, Y . Wang, Y . Jia, G. Li, A. Jaiswal, M. Pechenizkiy, Y . Lianget al., “Outlier weighed layerwise sparsity: A missing secret sauce for pruning llms to high sparsity,” ICML, 2024

    2024

  31. [39]

    Compute-in-memory chips for deep learning: Recent trends and prospects,

    S. Yu, H. Jiang, S. Huang, X. Peng, and A. Lu, “Compute-in-memory chips for deep learning: Recent trends and prospects,” IEEE circuits and systems magazine, vol. 21, no. 3, pp. 31–56, 2021

    2021

  32. [40]

    15.2 a 2.75-to-75.9 tops/w computing-in-memory nn processor supporting set-associate block-wise zero skipping and ping- pong cim with simultaneous computation and weight updating,

    J. Yue et al. , “15.2 a 2.75-to-75.9 tops/w computing-in-memory nn processor supporting set-associate block-wise zero skipping and ping- pong cim with simultaneous computation and weight updating,” in 2021 IEEE International Solid-State Circuits Conference (ISSCC) , vol. 64. I...

    2021

  33. [41]

    Hel- laswag: Can a machine really finish your sentence?

    R. Zellers, A. Holtzman, Y . Bisk, A. Farhadi, and Y . Choi, “Hel- laswag: Can a machine really finish your sentence?” arXiv preprint arXiv:1905.07830, 2019

    1905 arXiv

  34. [42]

    A 28-nm 18.7 tops/mm2 89.4-to-234.6 tops/w 8b single-finger edram compute- in-memory macro with bit-wise sparsity aware and kernel-wise weight update/refresh,

    Y . Zhan, W.-H. Yu, K.-F. Un, R. P. Martins, and P.-I. Mak, “A 28-nm 18.7 tops/mm2 89.4-to-234.6 tops/w 8b single-finger edram compute- in-memory macro with bit-wise sparsity aware and kernel-wise weight update/refresh,” IEEE Journal of Solid-State Circuits , 2024

    2024

  35. [43]

    Dynamic sparse no training: Training-free fine-tuning for sparse llms,

    Y . Zhang, L. Zhao, M. Lin, Y . Sun, Y . Yao, X. Han, J. Tanner, S. Liu, and R. Ji, “Dynamic sparse no training: Training-free fine-tuning for sparse llms,” arXiv preprint arXiv:2310.08915 , 2023

    2023 arXiv

  36. [44]

    A digital sram computing-in-memory design utilizing activation unstructured sparsity for high-efficient dnn inference,

    B. Zhong, M. Wang, C. Zhang, Y . Mai, X. Li, and Z. Yu, “A digital sram computing-in-memory design utilizing activation unstructured sparsity for high-efficient dnn inference,” in2023 IEEE Computer Society Annual Symposium on VLSI (ISVLSI) . IEEE, 2023, pp. 1–6

    2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.