Pith. sign in

REVIEW 5 major objections 7 minor 55 references

SwiftPrune: Hessian-Free Weight Pruning for Large Language Models

T0 review · 5 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SwiftPrune prunes large language models to 50% sparsity in seconds — about 7.7 seconds for LLaMA2-7B — with benchmark accuracy within roughly a point of SparseGPT and Wanda, by replacing Hessian inversion with a per-weight contribution…

desk verdict SwiftPrune is a fast, plausible pruning heuristic with strong 2:4 structured results, but the calibration-data handling is under-specified enough that the headline accuracy numbers are not yet reproducible from the manuscript. read the letter →

arxiv 2501.16376 v2 pith:MXG43QGQ submitted 2025-01-24 cs.LG cs.AI

Yuhan Kang , Yang Shi , Mei We , Jun He , Jianchao Yang , Zeyu Xue , Jing Feng , Xinwang Liu This is my paper · ORCID
classification cs.LGcs.AI
keywords weightpruninglargelanguagemodelsHessian-freeEWMAthresholdpost-trainingstructuredsparsity2:4LLMcompression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that post-training pruning of large language models does not need Hessian matrices or global sorting. It introduces SwiftPrune, which scores each weight by the closed-form metric $L_i = \tfrac12 w_i^2/(1-x_i^2/S)$ and selects weights for removal with an EWMA threshold test, cutting per-row complexity from roughly cubic to linear. On LLaMA2, LLaMA3, and Pythia, the paper reports pruning LLaMA2-7B to 50% sparsity in about 7.7 seconds with zero-shot accuracy within about a percentage point of SparseGPT and Wanda, and an average 12.29x speedup over existing approaches. If correct, this makes fast, training-free, repeatable pruning practical for very large models and for workloads that require frequent pruning operations.

What carries the argument

The central object is the row-wise contribution metric $L_i = \tfrac12 w_i^2/(1-x_i^2/S)$, where $x_i$ is the input activation to column $i$ and $S = \sum_j x_j^2$ is the sum of squared activations in that row. It is the paper's stand-in for the OBS saliency $\tfrac12 w_i^2/[H^{-1}]_{ii}$, obtained by showing that $H^{-1}_{qq}/(1-x_q^2/S)$ is nearly constant. The second mechanism is the EWMA threshold test: the streaming estimates $\text{est}$ and $\text{dev}$ are updated by $\text{est} \leftarrow (1-\alpha)\text{est}+\alpha L_i$ and $\text{dev} \leftarrow (1-\beta)\text{dev}+\beta|\text{est}-L_i|$, and weights with $L_i < \text{est} - l_a\,\text{dev}$ are pruned on the fly, avoiding any sorting step. A single activation vector $X$ of size $1 \times n$ is the only input used to compute $S$ and hence the entire row's importance ordering.

What would settle it

Run SwiftPrune on LLaMA2-7B twice at 50% sparsity: once with $S$ taken from one randomly drawn calibration activation per row and once with $S$ averaged over the full calibration set, then compare WikiText2 perplexity and the two masks.

Watch

Extended reading notes

Core claim

The central claim is that the inverse-Hessian saliency used by OBS-style pruning can be reduced, for the linear layers of an LLM, to a Hessian-free per-weight score. Starting from the per-row Hessian $H = 2XX^\top$ and a small diagonal perturbation, the paper derives $H^{-1}_{qq}/(1-x_q^2/S) \approx C$, a constant, under the assumption that $S = \sum_i x_i^2$ is far larger than any single $x_q^2$. Because only relative magnitudes matter, it replaces $H^{-1}_{qq}$ with $1-x_q^2/S$, giving the importance metric $L_i = \tfrac12 w_i^2/(1-x_i^2/S)$. It then replaces sorting by the exponentially weighted moving average of the $L_i$ stream, pruning $w_i$ whenever $L_i < \text{est} - l_a \cdot \text{dev}$, which brings the per-row cost to $O(n)$. The paper claims this preserves accuracy comparable to Hessian-based and diagonal-Hessian baselines while pruning LLaMA2-7B in seconds, and that the same selection logic extends to 2:4 structured sparsity for hardware acceleration.

Load-bearing premise

Every row's pruning decision depends on one activation vector $X$ from which $S = \sum_i x_i^2$ is computed, and the paper never specifies how a calibration dataset is aggregated into that vector; if the true input distribution is not represented by this single sample, the selected weights are not the minimal-loss set.

Editorial extensions

If this is right

  • Pruning a 7B-parameter model to 50% sparsity can be done in under ten seconds without retraining, Hessian construction, or weight updates.
  • Because the per-row cost is $O(n)$ rather than $O(n^3)$, the method's runtime grows roughly linearly with hidden width, so it scales to models where Hessian-based pruning becomes computationally prohibitive.
  • The same streaming test extends to 2:4 structured sparsity, and the paper reports about 1.48x end-to-end inference speedup on an RTX 4090 with 41% fewer DRAM access conflicts, making the pruned weights usable on hardware sparse tensor cores.
  • Pruned models remain compatible with subsequent LoRA or full-parameter fine-tuning, and the paper shows fine-tuning recovers part of the accuracy gap for both unstructured 50% and 2:4 structured sparsity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The metric's dependence on a single activation vector $X$ per row is under-specified: the paper never states how a multi-input calibration set is aggregated into $S$, so a natural test is to compute masks with per-row $S$ averaged over the calibration batch versus $S$ from one randomly chosen sample.
  • Because the EWMA threshold adapts to each row's local mean and deviation, the same $l_a$ value will likely produce different sparsity levels across rows; an online or per-row adjustment of $l_a$ could make the method hit an exact global sparsity target more reliably than the one fixed mapping in Table 3.
  • The formula $L_i \propto w_i^2/(1-x_i^2/S)$ sits between Wanda's $|w_i|\,\|x_i\|$ and the full inverse-Hessian saliency; a systematic ablation would isolate when the extra input-dependent denominator helps and when it merely tracks Wanda's ranking.
  • The derivation assumes $S \gg x_q^2$ for every column, which may fail for outlier activations; if those outliers are exactly the columns whose removal matters most, the approximation could misorder the tail of the importance distribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes SwiftPrune, a post-training pruning method for large language models that replaces Hessian-based importance scoring with a contribution metric L_i = (1/2) w_i^2 / (1 - x_i^2 / S), replaces sorting with an EWMA threshold test, and adds support for 2:4 structured sparsity. The authors report pruning LLaMA2, LLaMA3, and Pythia models within seconds, with an average speedup of 12.29x over Wanda and SparseGPT while retaining comparable average zero-shot accuracy. The central claims are an O(n) pruning pass per row and accuracy parity with Hessian-based methods without retraining or weight updates.

Significance. If the method works as described, it is practically significant: pruning an LLM in seconds instead of minutes-to-hours would make repeated pruning, adaptive sparsity, and deployment-time compression much more feasible. The experimental matrix is reasonably broad, covering three model families and several sparsity levels, and the reported speedups are large. The paper also provides a structured-sparsity extension and a fine-tuning compatibility study. However, the manuscript as written is not reproducible: the calibration protocol is not specified, the algorithm contains an internally inconsistent update of S, the EWMA threshold is tuned per sparsity level rather than derived, and several accuracy statements contradict the reported tables. The O(n) complexity claim is plausible, but it rests on the EWMA heuristic replacing sorting, and the paper provides no analysis or sensitivity study showing that the heuristic selects approximately the same weights as the bottom quantile of L_i.

major comments (5)
  1. [Section 3.1, Eq. (3)] The regularized Hessian in Eq. (3) is undefined as written. If S = sum x_i^2, then H = 2XX^T + (sum_i diag(2XX^T)) I equals 2XX^T + 2S I, but the next paragraph and Eq. (4) use H_qq = 2(x_q^2 + S/n), which corresponds to a damping term of 2S/n, not 2S. This is load-bearing because the derivation of H^{-1}_{qq} and of the final metric in Eq. (8) depends on the damping being 2S/n. Please state the intended regularizer explicitly and re-derive Eqs. (4)-(7) consistently.
  2. [Algorithm 1 and Table 2] The calibration protocol is not specified. Algorithm 1 accepts a single vector X_{1xn} and defines S = sum x_i^2, but the text and Table 6 say calibration data are used, and no statement explains how multiple calibration samples are aggregated into this X (for example, by averaging squared activations, by using one random sample, or by processing a sequence of inputs). As published, the metric L_i depends on one vector, so the reported accuracy cannot be reproduced from the manuscript. Moreover, line 9 updates S <- S - w_i^2, mixing activation squares with weight squares; S was defined from x_i^2, so the update is dimensionally inconsistent and changes subsequent L_i values in an unexplained way. Specify the aggregation over calibration data and either justify or remove the S update.
  3. [Section 3.2, Algorithm 1, Table 3] The EWMA threshold does not enforce the requested sparsity sp; the fraction pruned is set indirectly by la, which is tuned per sparsity level in Table 3. This makes the accuracy at a given sparsity dependent on a calibration curve rather than on a principled sparsity control. Table 8 reports results at 10-40% sparsity without giving the corresponding la values. In addition, the threshold is order-dependent because L_i values are scanned in the fixed row order and est/dev are updated online; the paper provides no analysis or ablation showing how close the selected set is to the bottom quantile of L_i. Please provide the la values used for Table 8 and an evaluation of sensitivity to row ordering and to la.
  4. [Section 4.2, Table 5] The accuracy summary contradicts the reported numbers. The text claims that at 50% pruning SwiftPrune keeps the average performance decline within 2 percentage points of the dense model and that the average difference from baselines is below 1%; however, Table 5 shows, for example, LLaMA2-7B at 50% unstructured dropping from 64.10 to 58.70 (-5.40), at 50% 2:4 dropping to 53.37 (-10.73), and Pythia-2.8B at 2:4 dropping from 53.07 to 46.39 (-6.68). These statements should be corrected or replaced with per-condition numbers, and the claimed parity with Wanda and SparseGPT should be stated per configuration rather than as a global average.
  5. [Table 6 and Section 4.2] Table 6 lists SwiftPrune's pruning metric as |W_ij| * n, which is inconsistent with Eq. (8), L_i = (1/2) w_i^2 / (1 - x_i^2/S), and with Algorithm 1. Section 4.2 also states that SwiftPrune works "without requiring training data," while Section 4.1 and Table 6 indicate that calibration data are used. These contradictions make the method's precise input requirements unclear and should be resolved.
minor comments (7)
  1. [Section 2, Eq. (1)] The notation dE/dDelta w is nonstandard and should be written in terms of the gradient and Hessian of E with respect to w.
  2. [Section 3.1, Eq. (4)] The cofactor H*_{qq} is used without being defined; please define it before Eq. (4).
  3. [Section 3.1, Eq. (7)] The quantity C is called a constant, but it depends on S and therefore varies across rows; say "constant for fixed S" or express it as a proportionality factor.
  4. [Section 3.3] There is a typo: "mtehod" should be "method."
  5. [Table 4 and Section 3.3] The text reports a 1.48x mean speedup, while the table entries average to 1.47x; please align the numbers.
  6. [Table 7] The table layout is confusing: the left column mixes sparsity patterns (50%, 2:4) with fine-tuning methods (NO, LoRA, Full), and the "Zero-Shot" and "Perplexity" labels are not clearly associated with the rows; please restructure it.
  7. [Section 7] The statement that the pruning approach "prevents the amplification of existing biases" is not supported by any experiment reported in the paper; please remove or qualify it.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the pruning metric is derived from an explicit loss approximation, the EWMA criterion is presented as a heuristic, and the reported accuracies are measured rather than fitted.

full rationale

The derivation chain is self-contained. Section 3.1 starts from the layerwise output-difference loss E = sum_i ||w_i x - w_hat_i x||^2, forms H = 2XX^T + diag(2XX^T)I, and algebraically derives H^{-1}_{qq}/(1 - x_q^2/S) approximately constant, which justifies replacing H^{-1}_{qq} by (1 - x_q^2/S) for ranking weights. This is an explicit approximation with stated assumptions (large row dimension, S >> x_q^2), not a definition of the target result. The metric L_i = 1/2 w_i^2/(1 - x_i^2/S) is then applied in Algorithm 1, and the EWMA threshold test is introduced as a deliberate heuristic to avoid sorting; its quality is validated empirically on held-out benchmarks. The parameter la is calibrated to the desired sparsity level via Table 3, but this is standard hyperparameter tuning rather than a fitted prediction: the reported zero-shot accuracies, perplexities, and latencies are measured after pruning, not constructed to match the baselines. The unspecified aggregation of the calibration vector X into S is a reproducibility gap, but it does not make any output equivalent to an input by construction. There are no load-bearing self-citations and no imported uniqueness claims. Therefore the central claims are not circular.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The method rests on two structural assumptions, row independence and single-vector activation summary, and one hand-tuned parameter la that controls sparsity. No new physical or mathematical entities are postulated.

free parameters (2)
  • la (EWMA threshold multiplier) = -1.5, -0.9, -0.2, 0.2, 0.5 for 90/80/70/60/50% sparsity (Table 3)
    Hand-tuned per pruning ratio to make the online threshold yield the target sparsity; the exact achieved sparsity is not reported.
  • EWMA smoothing factors alpha, beta = 0.125, 0.125 (RFC 6298)
    Fixed from TCP RTT estimation, not tuned per model; used in Figure 2 and Algorithm 1.
assumptions (3)
  • domain assumption Weights across different rows of a linear layer are independent and can be optimized separately.
    Section 2 states that rows never multiply with each other, so the Hessian is block-diagonal per row; this is a standard layerwise pruning assumption.
  • domain assumption The approximation S - x_q^2 approximately equals S holds for all weights in LLM layers because n is large (e.g., 4096).
    Section 3.1, after Eq. 6, uses S >> x_q^2 to treat H^{-1}_{qq}(1 - x_q^2/S) as constant; no empirical check of this assumption is provided.
  • domain assumption EWMA estimates of mean and deviation correctly identify the smallest L values without a full sort.
    Section 3.2, Algorithm 1 prunes when L_i < est - la times dev; this assumes the online state tracks the distribution tail, which is not proven.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SwiftPrune: Hessian-Free Weight Pruning for Large Language Models." pith.science (2026). https://pith.science/paper/MXG43QGQ

@misc{pith2026250116376,
  author       = {Pith},
  title        = {Pith review of: SwiftPrune: Hessian-Free Weight Pruning for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MXG43QGQ}},
  note         = {Machine review of arXiv:2501.16376}
}
read the original abstract

Post-training pruning, as one of the key techniques for compressing large language models, plays a vital role in lightweight model deployment and model sparsity. However, current mainstream pruning methods dependent on the Hessian matrix face significant limitations in both pruning speed and practical effectiveness due to the computationally intensive nature of second-order derivative calculations. This paper presents SwiftPrune, a novel Hessian-free weight pruning method that achieves hardware-efficient model compression through two key innovations: 1) SwiftPrune eliminates the need for computationally intensive Hessian matrix calculations by introducing a contribution-based weight metric, which evaluates the importance of weights without relying on second-order derivatives. 2) we employ the Exponentially Weighted Moving Average (EWMA) technique to bypass weight sorting, enabling the selection of weights that contribute most to LLM accuracy and further reducing time complexity. Our approach is extended to support structured sparsity pruning, facilitating efficient execution on modern hardware accelerators. We validate the SwiftPrune on three LLMs (namely LLaMA2, LLaMA3, and Pythia), demonstrating that it significantly enhances compression performance. The experimental findings reveal that SwiftPrune completes the pruning process within seconds, achieving an average speedup of 12.29x (up to 56.02x) over existing SOTA approaches.

Figures

Figures reproduced from arXiv: 2501.16376 by the authors.

Figure 1
Figure 1. Our design of novel pruning method, using [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Statistical magnitude detection of L with EWMA method in LLaMA2-7B MLP blocks. x axis presents the sequence number of each weight, and y axis presents the numerical values. Ideal algorithms should show est approaches real mean and dev approaches real dev. racy improvements across multiple benchmarks through innovative fine-grained pruning strategies. These empirical findings validate the innovation of the SwiftPrune… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 25 canonical work pages

  1. [1]

    Ron Banner, Yury Nahshan, and Daniel Soudry. 2019. Post training 4-bit quantization of convolutional networks for rapid-deployment. Advances in Neural Information Processing Systems, 32

    work page 2019

  2. [2]

    Stella Biderman, Hailey Schoelkopf, Quentin G. Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar van der Wal. 2023. https://arxiv.org/abs/2304.01373 Pythia: A suite for analyzing large language models across training and scaling . ArXiv...

    arXiv 2023

  3. [3]

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. 2020. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432--7439

    2020

  4. [4]

    Michael Boratko, Harshit Padigela, Divyendra Mikkilineni, Pritish Yuvraj, Rajarshi Das, Andrew McCallum, Maria Chang, Achille Fokoue-Nkoutche, Pavan Kapanipathi, Nicholas Mattei, et al. 2018. A systematic classification of knowledge, reasoning, and context within the arc dataset. arXiv preprint arXiv:1806.00358

    arXiv 2018

  5. [5]

    OpenCompass Contributors. 2023. Opencompass: A universal evaluation platform for foundation models. https://github.com/open-compass/opencompass

    2023

  6. [6]

    Peijie Dong, Lujun Li, Zhenheng Tang, Xiang Liu, Xinglin Pan, Qiang Wang, and Xiaowen Chu. 2024. Pruner-zero: evolving symbolic pruning metric from scratch for large language models. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org

    work page 2024

  7. [7]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

  8. [8]

    Utku Evci, Trevor Gale, Jacob Menick, Pablo Samuel Castro, and Erich Elsen. 2020. Rigging the lottery: Making all tickets winners. In International conference on machine learning, pages 2943--2952. PMLR

    work page 2020

Show all 55 references
  1. [9]

    Gongfan Fang, Xinyin Ma, Mingli Song, Michael Bi Mi, and Xinchao Wang. 2023. Depgraph: Towards any structural pruning. The IEEE/CVF Conference on Computer Vision and Pattern Recognition

    2023

  2. [10]

    Elias Frantar and Dan Alistarh. 2022. Optimal brain compression: A framework for accurate post-training quantization and pruning. Advances in Neural Information Processing Systems, 35:4475--4488

    2022

  3. [11]

    Elias Frantar and Dan Alistarh. 2023. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning, pages 10323--10337. PMLR

    2023

  4. [12]

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022. https://arxiv.org/abs/2210.17323 Gptq: Accurate post-training quantization for generative pre-trained transformers . ArXiv, abs/2210.17323

    2022 arXiv

  5. [13]

    Qichen Fu, Minsik Cho, Thomas Merth, Sachin Mehta, Mohammad Rastegari, and Mahyar Najibi. 2024. https://arxiv.org/abs/2407.14057 Lazyllm: Dynamic token pruning for efficient long context llm inference . Preprint, arXiv:2407.14057

    2024 arXiv

  6. [14]

    Yonggan Fu, Haichuan Yang, Jiayi Yuan, Meng Li, Cheng Wan, Raghuraman Krishnamoorthi, Vikas Chandra, and Yingyan Lin. 2022. Depthshrinker: a new compression paradigm towards boosting real-hardware efficiency of compact neural networks. In International Conference on Machine Le...

    2022

  7. [15]

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang S...

    2024 doi

  8. [16]

    Song Han, Huizi Mao, and William J Dally. 2015. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149

    2015 arXiv

  9. [17]

    Simla Burcu Harma, Ayan Chakraborty, Elizaveta Kostenok, Danila Mishin, Dongho Ha, Babak Falsafi, Martin Jaggi, Ming Liu, Yunho Oh, Suvinay Subramanian, et al. 2024. Effective interplay between sparsity and quantization: From theory to practice. arXiv preprint arXiv:2405.20935

    2024 arXiv

  10. [18]

    Babak Hassibi, David G Stork, and Gregory J Wolff. 1993. Optimal brain surgeon and general network pruning. In IEEE international conference on neural networks, pages 293--299. IEEE

    1993

  11. [19]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3

    2022

  12. [20]

    Roy, Jonathan Frankle, and Gintare Karolina Dziugaite

    Tian Jin, Michael Carbin, Daniel M. Roy, Jonathan Frankle, and Gintare Karolina Dziugaite. 2022. Pruning's effect on generalization through the lens of training and regularization. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NI...

    2022

  13. [21]

    Woosuk Kwon, Sehoon Kim, Michael W Mahoney, Joseph Hassoun, Kurt Keutzer, and Amir Gholami. 2022 a . https://proceedings.neurips.cc/paper_files/paper/2022/file/987bed997ab668f91c822a09bce3ea12-Paper-Conference.pdf A fast post-training pruning framework for transformers . In Ad...

    2022

  14. [22]

    Woosuk Kwon, Sehoon Kim, Michael W Mahoney, Joseph Hassoun, Kurt Keutzer, and Amir Gholami. 2022 b . A fast post-training pruning framework for transformers. Advances in Neural Information Processing Systems, 35:24101--24116

    2022

  15. [23]

    Qi Le, Enmao Diao, Ziyan Wang, Xinran Wang, Jie Ding, Li Yang, and Ali Anwar. 2025. https://openreview.net/forum?id=WOt1owGfuN Probe pruning: Accelerating LLM s through dynamic pruning via model-probing . In The Thirteenth International Conference on Learning Representations

    2025

  16. [24]

    Yixiao Li, Yifan Yu, Qingru Zhang, Chen Liang, Pengcheng He, Weizhu Chen, and Tuo Zhao. 2023. Losparse: Structured compression of large language models based on low-rank and sparse approximation. In International Conference on Machine Learning, pages 20336--20350. PMLR

    2023

  17. [25]

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems, 6:87--100

    2024

  18. [26]

    Zhiqiang Liu, Yong Dou, Jingfei Jiang, Jinwei Xu, Shijie Li, Yongmei Zhou, and Yingnan Xu. 2017. Throughput-optimized fpga accelerator for deep convolutional neural networks. ACM Transactions on Reconfigurable Technology and Systems (TRETS), 10(3):1--23

    2017

  19. [27]

    Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Re, et al. 2023. Deja vu: Contextual sparsity for efficient llms at inference time. In International Conference on Machine Learning, pages 22137--2...

    2023

  20. [28]

    Kai Lu, Yaohua Wang, Yang Guo, Chun Huang, Sheng Liu, Ruibo Wang, Jianbin Fang, Tao Tang, Zhaoyun Chen, Biwei Liu, et al. 2022. Mt-3000: a heterogeneous multi-zone processor for hpc. CCF Transactions on High Performance Computing, 4(2):150--164

    2022

  21. [29]

    Kai Lv, Yuqing Yang, Tengxiao Liu, Qinghui Gao, Qipeng Guo, and Xipeng Qiu. 2023. Full parameter fine-tuning for large language models with limited resources. arXiv preprint arXiv:2306.09782

    2023 arXiv

  22. [30]

    X Ma, G Fang, and X Wang. 2023 a . On the structural pruning of large language models. NeurIPS, Llm-pruner

    2023

  23. [31]

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023 b . Llm-pruner: on the structural pruning of large language models. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA. Curran Associates Inc

    2023

  24. [32]

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023 c . Llm-pruner: On the structural pruning of large language models. Advances in neural information processing systems, 36:21702--21720

    2023

  25. [33]

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843

    2016 arXiv

  26. [34]

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789

    2018 arXiv

  27. [35]

    Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort. 2020. Up or down? adaptive rounding for post-training quantization. In International Conference on Machine Learning, pages 7197--7206. PMLR

    2020

  28. [36]

    Vern Paxson, Mark Allman, Jerry Chu, and Matt Sargent. 2011. https://www.rfc-editor.org/rfc/rfc6298 Rfc6298: Computing tcp's retransmission timer . Technical report

    2011

  29. [37]

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99--106

    2021

  30. [38]

    Victor Sanh, Thomas Wolf, and Alexander Rush. 2020. Movement pruning: Adaptive sparsity by fine-tuning. Advances in neural information processing systems, 33:20378--20389

    2020

  31. [39]

    Michael Santacroce, Zixin Wen, Yelong Shen, and Yuanzhi Li. 2023. What matters in the structured pruning of generative language models? arXiv preprint arXiv:2302.03773

    2023 arXiv

  32. [40]

    Shashata Sawmya, Linghao Kong, Ilia Markov, Dan Alistarh, and Nir Shavit. 2024. Sparse expansion and neuronal disentanglement. arXiv preprint arXiv:2405.15756

    2024 arXiv

  33. [41]

    Hang Shao, Bei Liu, and Yanmin Qian. 2024. One-shot sensitivity-aware mixed sparsity pruning for large language models. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 11296--11300. IEEE

    2024

  34. [42]

    Maying Shen, Pavlo Molchanov, Hongxu Yin, and Jose M. Alvarez. 2022. When to prune? a policy towards early structural pruning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12247--12256

    2022

  35. [43]

    Sidak Pal Singh and Dan Alistarh. 2020. Woodfisher: Efficient second-order approximation for neural network compression. Advances in Neural Information Processing Systems, 33:18098--18109

    2020

  36. [44]

    Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. 2023. A simple and effective pruning approach for large language models. arXiv preprint arXiv:2306.11695

    2023 arXiv

  37. [45]

    Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. 2024. https://openreview.net/forum?id=PxoFut3dWW A simple and effective pruning approach for large language models . In The Twelfth International Conference on Learning Representations

    2024

  38. [46]

    Minjin Tang, Mei Wen, Yasong Cao, Junzhong Shen, Jianchao Yang, Jiawei Fei, Yang Guo, and Sheng Liu. 2022. Mentha: Enabling sparse-packing computation on systolic arrays. In Proceedings of the 51st International Conference on Parallel Processing, pages 1--11

    2022

  39. [47]

    Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M

    Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cristian Cant \'o n Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes,...

    2023 arXiv

  40. [48]

    Tycho F. A. van der Ouderaa, Markus Nagel, Mart Van Baalen, and Tijmen Blankevoort. 2024. https://openreview.net/forum?id=DYIIRgwg2i The LLM surgeon . In The Twelfth International Conference on Learning Representations

    2024

  41. [49]

    Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, and Danqi Chen. 2024. https://openreview.net/forum?id=09iOdaeOzp Sheared LL a MA : Accelerating language model pre-training via structured pruning . In The Twelfth International Conference on Learning Representations

    2024

  42. [50]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830

    2019 arXiv

  43. [51]

    Ritchie Zhao, Yuwei Hu, Jordan Dotzel, Chris De Sa, and Zhiru Zhang. 2019. Improving neural network quantization without retraining using outlier channel splitting. In International conference on machine learning, pages 7543--7552. PMLR

    2019

  44. [52]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223

    2023 arXiv

  45. [53]

    Michael Zhu and Suyog Gupta. 2017. To prune, or not to prune: exploring the efficacy of pruning for model compression. arXiv preprint arXiv:1710.01878

    2017 arXiv

  46. [54]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  47. [55]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.