Pith. sign in

REVIEW 3 major objections 5 minor 31 references

TriSP claims that the best structured-pruning decisions come from a per-channel score that combines activation-weighted weight magnitude with gradient sensitivity, and that the best recovery after pruning comes from low-rank adaptation, not

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 11:47 UTC pith:EGD433OB

load-bearing objection Core pruning method for MHA models is solid, but Mistral results are not reproducible without a GQA-aware algorithm. the 3 major comments →

arxiv 2607.22587 v1 pith:EGD433OB submitted 2026-06-09 cs.AI

TriSP: Tri-Signal Structured Pruning for Large Language Models

Manel Kara laoua , Soumia Bouyahiaoui , Aicha Boutorh This is my paper
classification cs.AI
keywords structured pruninglarge language modelsimportance metricgradient sensitivityactivation normlow-rank adaptationLoRA recoveryadaptive allocation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

TriSP is a structured pruning pipeline for decoder-only large language models. It introduces a per-channel importance score that combines three signals: weight magnitude, input activation norm, and first-order loss gradient sensitivity, folded together by a geometric mean. The paper argues this single score makes better removal decisions than either activation-only or gradient-only criteria, and that adaptive per-layer budget allocation matters once more than about 30% of the model is removed. It also argues that post-pruning recovery with low-rank adaptation (LoRA) consistently beats the fixed bias-correction used by FLAP, with the gap widening at aggressive ratios. On LLaMA-7B, the method reaches 6.80 WikiText-2 perplexity at 20% pruning and an 82% throughput gain at 50% pruning.

Core claim

The central claim is that per-channel importance for structured pruning should be measured as the geometric mean of two activation-aware quantities: WIFN, the average absolute weight magnitude scaled by the square root of the accumulated activation norm, and GS, the average of |weight x gradient| scaled by the same activation norm. Grouping these channel scores by attention head and MLP neuron, standardizing them per layer, applying a compression-weighted global threshold, and recovering the pruned model with LoRA on the attention projections produces the best results in every configuration the paper tests: lowest WikiText-2 and PTB perplexity and highest average zero-shot commonsense accura

What carries the argument

The load-bearing object is the TriSP score S_l,j = sqrt(WIFN_l,j * GS_l,j). WIFN_l,j is the average absolute weight entering channel j, times the square root of that channel's mean squared activation norm; GS_l,j is the average absolute product of weight and gradient for that channel, also scaled by the activation norm. The geometric mean prevents either the structural or the loss-sensitivity signal from dominating. Scores are z-score normalized per layer, grouped into attention heads (by contiguous output-projection channels) and MLP neurons, ranked with compression weights that reflect how many parameters each head or neuron removes, and cut by a global threshold that meets the target prun

Load-bearing premise

The load-bearing premise is that a head's importance can be read off from the output projection's per-channel scores alone; the Q, K, and V projections of that head are removed based on that score even though their own loss sensitivity is never measured.

What would settle it

On LLaMA-2-7B at 20% pruning, rank heads using the same TriSP formula computed separately for the query, key, value, and output projections, or by the direct first-order loss change when each head is removed, and compare the pruned models' WikiText-2 perplexity with TriSP's output-projection-only ranking. If the alternative ranking selects a different head subset and attains lower perplexity, the head-scoring shortcut fails; if the rankings coincide, it holds.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • At 20% pruning, LLaMA-2-7B's WikiText-2 perplexity rises from a dense 5.47 to 6.80 with TriSP+LoRA, while average zero-shot accuracy drops from 63.79% to 60.98% - a small quality cost for removing a fifth of the model.
  • At 50% pruning, TriSP+LoRA keeps perplexity at 13.44 and average accuracy at 48.18%, while raising inference throughput by 82% (74.6k tokens/s vs 41.1k), making aggressive pruning practical for latency-sensitive deployments.
  • The metric ablation shows the full TriSP score beats both WIFN-only and GS-only at every ratio tested, e.g. 15.83 vs 16.12 and 17.14 perplexity at 40% pruning on Vicuna-7B.
  • Adaptive layer-module allocation only pays off beyond 30% pruning; at 50% it reduces perplexity from 53.46 (uniform) to 27.63 on LLaMA-2-7B.
  • LoRA recovery outperforms bias compensation at every pruning ratio on Vicuna-7B, with the gap widening from about 0.1 points at 20% to roughly 6 points at 50%.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the geometric mean is symmetric, TriSP assumes WIFN and GS contribute equally; weighting them differently by layer or ratio might improve aggressive-ratio behavior, but that experiment is not in the paper.
  • Editorial inference: heads are scored only through the output projection; a natural stress test is to score each of WQ, WK, WV, WO separately and check whether the head rankings, and the resulting pruned models, diverge. The paper does not run this experiment.
  • Manuscript note: Section E refers to a 'Table ??' for qualitative generation examples; no such table appears in the text, so the qualitative fluency claim currently lacks its supporting artifact.
  • Editorial inference: since LoRA was applied only to attention projections and the paper says extending it to MLP is future work, the reported 50% results might improve further with MLP adapters; conversely, the current gains could partly reflect attention recovery rather than the pruning metric itself.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes TriSP, a structured pruning pipeline for decoder-only LLMs. The importance score combines activation-weighted weight magnitude (WIFN) with a gradient-sensitivity term (GS) via a geometric mean, scores are z-standardized across layers, and head/neuron components are pruned according to an adaptive layer-module allocation with a compression-weighted global threshold. A LoRA-based recovery stage is compared against bias compensation and no recovery. Experiments on LLaMA-2-7B, Vicuna-7B, DeepSeek-7B, and Mistral-7B report the lowest WikiText-2/PTB perplexity and highest zero-shot commonsense accuracy at 20%, 30%, and 50% pruning ratios across all baselines tested, along with inference-throughput gains.

Significance. If the empirical claims hold, TriSP would be a useful and practical contribution: it transfers gradient-augmented importance from unstructured to structured pruning, provides a direct comparison of recovery strategies under a fixed criterion, and reports consistent gains over FLAP and LLM-Pruner. The paper includes careful ablations of the metric, allocation, and recovery components, and the core MHA (non-GQA) experimental setups are internally consistent. However, the claimed generalization to Mistral-7B is not supported by the described algorithm, and the absence of code, seeds, and error bars leaves key quantitative claims unverifiable as presented.

major comments (3)
  1. [Section 3.3-3.4, Algorithm 1, Table 5] The method as written cannot be applied to Mistral-7B. Mistral uses GQA (Table 1: H_Q=32, H_KV=8), but Algorithm 1 and Section 3.4 delete 'pruned head rows from WQ, WK, WV and columns from WO' treating Q/K/V/O heads as one-to-one. Head scores (Eq. 8) are computed on 32 WO column blocks, giving 32 independent decisions, but WK and WV have only 8 KV row blocks. The paper gives no rule for pruning WK/WV rows when only a subset of the four query heads sharing a KV head is pruned. Moreover, the compression weight c_i=4d_h/3 in Eq. 9 assumes four full projections per head; under GQA the per-query-head parameter cost is WQ+WO plus a quarter of WK+WV (or a different group-level cost if entire KV groups are pruned). Therefore the nominal 20% and 30% ratios in Table 5 do not correspond to the stated compression objective, and the Mistral perplexity/accuracy results are not reproducible from the pr
  2. [Section 4.1, Tables 2-5] The baseline comparison may be confounded by undisclosed LoRA hyperparameters. The paper states all methods use the same calibration set (512 WikiText-2 samples) but does not state whether LLM-Pruner's LoRA recovery uses the same rank r=8, alpha=16, dropout=0.05, 25k Alpaca samples, 2 epochs, and learning rate as TriSP. If LLM-Pruner's LoRA was not similarly tuned, the 'outperforms LLM-Pruner + LoRA' claims in Tables 3-5 could reflect suboptimal baseline recovery rather than a superior pruning metric. Please report the exact hyperparameters used for each baseline or run matched-configuration ablations.
  3. [Section 4.2, Tables 2, 4, 5] No variance or multiple-seed results are reported. Some of the claimed differences are small (e.g., Table 4: 20% DeepSeek WikiText-2 8.27 vs. 8.53; Table 5: 20% Mistral TriSP w/o LoRA 6.15 vs. TriSP w/ LoRA 6.35), and single-run perplexity in this setting typically has run-to-run noise. Since the central claim is 'lowest perplexity at all tested configurations,' the absence of error bars or seeds means the quantitative ordering is not statistically supported as presented. Add at least 2-3 seeds with standard deviations, or clearly state that results are single-run and temper the 'lowest' claim accordingly.
minor comments (5)
  1. [Table 2 caption] The table header says 'LLaMA-7B' while the text and Table 1 refer to 'LLaMA-2-7B'. Please use consistent model naming throughout.
  2. [Appendix E] The qualitative generation section references 'Table ??' and no table is actually included. Either add the table or remove the reference.
  3. [Table 4] The FLAP w/ bias row at 20% appears to have missing spaces between numbers ('66.5865.5167.8967.89'), making values hard to read. Fix the formatting.
  4. [Section 3.3] The sentence 'removing one attention head eliminates 4×d_h ×d_hidden parameters' later uses c_i = 4d_h/3 for heads, which is correct only under the assumption that all four projections are equally pruned per head. Clarify this MHA-specific assumption in the text.
  5. [Section 4.5, Table 6] At 50% pruning LLM-Pruner has both lower latency (26.11 ms vs. 27.46 ms) and higher throughput (78.4k vs. 74.6k) than TriSP. The paper's abstract and conclusion emphasize TriSP's 82% throughput improvement without mentioning that LLM-Pruner is slightly faster; please qualify the claim to avoid an overstatement.

Circularity Check

0 steps flagged

No significant circularity: the TriSP importance score is computed from model weights, activations, and calibration gradients, and the claimed perplexity/accuracy results are held-out evaluations.

full rationale

The derivation chain is self-contained. Equations 4–6 define WIFN, GS, and the TriSP score directly from the pretrained weight matrices, calibration activation norms (Eq. 3), and calibration gradient magnitudes (Eq. 2); Eq. 7 only standardizes these scores, Eq. 8 aggregates them into head scores, and Eq. 9 selects a threshold from the target pruning ratio and the models' own parameter counts. No parameter is fitted to WikiText-2, PTB, or zero-shot benchmark results, and the reported perplexities/accuracies are measured after pruning on held-out data. The ablations compare internal variants rather than tuning components to target metrics. Citations to FLAP, Wanda, GBLM-Pruner, LLM-Pruner, and LoRA are external prior work used as baselines or as sources of standard score components; there are no load-bearing self-citations and no imported uniqueness theorem. The Mistral-7B grouped-query-attention issue (Table 1 lists H_Q=32 and H_KV=8 while Algorithm 1 removes one-to-one Q/K/V/O structures) and the empty Table ?? reference are reproducibility/completeness concerns, not circularity: they do not show that any 'prediction' is equal by construction to its inputs. Therefore the appropriate circularity score is 0.

Axiom & Free-Parameter Ledger

7 free parameters · 4 axioms · 0 invented entities

The paper does not introduce new theoretical entities; it combines known signals. The main assumptions are about the validity of gradient proxies, the sufficiency of output-projection scores for heads, and the fairness of baseline comparisons. Free parameters are standard hyperparameters but their sensitivity is unexplored.

free parameters (7)
  • LoRA rank r = 8
    Chosen by the authors (Section 4.1). No sensitivity analysis is reported, so the best results may depend on this value.
  • LoRA alpha = 16
    Chosen with scaling factor alpha=16 (Section 4.1). Standard choice but not justified experimentally.
  • LoRA dropout = 0.05
    Chosen in Section 4.1. Impact on final quality is not analyzed.
  • Recovery epochs = 2
    Fine-tuning LoRA for 2 epochs on 25k Alpaca samples (Section 4.1). No epoch-by-epoch results.
  • Calibration set size = 512
    Uses 512 WikiText-2 samples (Section 4.1). The stability of the gradient statistics over this sample count is not assessed.
  • Epsilon in WIFN/GS = 1e-6
    Numerical stabilizer in Eqs. (4)-(5), minor effect but labeled as fixed.
  • MLP rounding multiple = 64
    Rounds retained MLP neurons to multiples of 64 for hardware efficiency (Section 3.4). This could change effective pruning ratios and is not accounted for in the reported pruning percentages.
axioms (4)
  • domain assumption Gradient magnitudes averaged over 512 calibration samples reliably approximate loss sensitivity for every channel.
    The method computes per-weight gradients (Eq. 2) on a small calibration set and uses them as a proxy for removal impact (Section 3.2). If gradients are noisy or unrepresentative, the scores are unreliable.
  • domain assumption The output projection channel scores are a sufficient proxy for whole-head importance.
    Head-level importance is computed by averaging the output projection's channel scores (Eq. 8), ignoring WQ, WK, WV. If the output projection under- or over-weights heads, the pruning decisions are suboptimal.
  • domain assumption LoRA fine-tuning on Alpaca instruction data transfers to the downstream zero-shot benchmarks.
    Recovery fine-tuning uses 25k Alpaca samples; the evaluation uses WikiText-2/PTB and six commonsense benchmarks (Section 4.1). No analysis of distribution mismatch.
  • ad hoc to paper All baselines (FLAP, LLM-Pruner) are run with equally tuned hyperparameters.
    The paper states 'all methods use the same calibration set' but does not report whether the same LoRA settings were used for all methods. If baselines used suboptimal tuning, the comparisons are biased.

pith-pipeline@v1.3.0-alltime-deepseek · 13966 in / 9799 out tokens · 110315 ms · 2026-08-02T11:47:35.888500+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) achieve strong performance across diverse tasks but their deployment is constrained by the memory and compute cost of their parameters. Structured pruning addresses this by removing entire structures such as attention heads and Multi-Layer Perceptron (MLP) neurons to produce smaller dense models that run efficiently on standard hardware. However, existing methods rely on either gradient-based importance estimation, which is memory-prohibitive, or activation-based statistical proxies, which do not directly measure the effect of removal on the loss. Furthermore, the interaction between the importance criterion and the post-pruning recovery strategy has not been systematically studied. We propose TriSP (Tri-Signal Structured Pruning), an importance metric that combines weight magnitude scaled by activation norm with first-order gradient sensitivity via a geometric mean, producing a channel-level score that captures both structural and loss-sensitivity signals. Combined with adaptive per-layer budget allocation and low-rank adaptation (LoRA) recovery, TriSP achieves the lowest perplexity and highest zero-shot accuracy across all tested configurations, reaching 6.80 WikiText-2 perplexity at 20% pruning on LLaMA-7B. Inference throughput improves by 82% at 50% pruning, while still maintaining competitive performance.

Figures

Figures reproduced from arXiv: 2607.22587 by Aicha Boutorh, Manel Kara laoua, Soumia Bouyahiaoui.

Figure 1
Figure 1. Figure 1: Overview of the proposed TriSP structured pruning pipeline. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Vicuna-7B: WikiText-2 perplexity under TriSP pruning with different recovery strategies. [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: WikiText-2 perplexity: uniform (UL-UM) vs. adaptive (AL-AM) allocation on LLaMA-2-7B. [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: WikiText-2 perplexity scaling with pruning ratio on LLaMA-2-7B. TriSP + LoRA maintains the lowest [PITH_FULL_IMAGE:figures/full_fig_p013_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Inference throughput of LLaMA-2-7B (seq=64, batch=32, H100). At 20% pruning, TriSP achieves 50.0k [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Throughput vs. batch size for TriSP-pruned DeepSeek-7B at sequence length 512. Structured pruning yields [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Per-layer retention ratio under AL-AM allocation for LLaMA-2-7B at four pruning ratios. Values above the [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 10 linked inside Pith

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  2. [2]

    Fluctuation-based adaptive structured pruning for large language models

    Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. Fluctuation-based adaptive structured pruning for large language models. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10865–10873, 2024

    2024

  3. [3]

    Attention is all you need.Advances in neural information processing systems, 30:I, 2017

    Vaswani Ashish. Attention is all you need.Advances in neural information processing systems, 30:I, 2017. 11 APREPRINT- JULY28, 2026

    2017

  4. [4]

    Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman

    Saleh Ashkboos, Maximilian L. Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman. Slicegpt: Compress large language models by deleting rows and columns.arXiv preprint arXiv:2401.15024, 2024

    Pith/arXiv arXiv 2024

  5. [5]

    Deepseek llm: Scaling open-source language models with longtermism.arXiv preprint arXiv:2401.02954, 2024

    Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. Deepseek llm: Scaling open-source language models with longtermism.arXiv preprint arXiv:2401.02954, 2024

    Pith/arXiv arXiv 2024

  6. [6]

    Piqa: Reasoning about physical commonsense in natural language

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about physical commonsense in natural language. InProceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432–7439, 2020

    2020

  7. [7]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.LMSYS blog, 2023

    2023

  8. [8]

    Think you have solved question answering? try arc, the ai2 reasoning challenge.arXiv preprint arXiv:1803.05457, 2018

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge.arXiv preprint arXiv:1803.05457, 2018

    Pith/arXiv arXiv 2018

  9. [9]

    Beyond size: How gradients shape pruning decisions in large language models.arXiv preprint arXiv:2311.04902, 2023

    Rocktim Jyoti Das, Mingjie Sun, Liqun Ma, and Zhiqiang Shen. Beyond size: How gradients shape pruning decisions in large language models.arXiv preprint arXiv:2311.04902, 2023

    Pith/arXiv arXiv 2023

  10. [10]

    Sparsegpt: Massive language models can be accurately pruned in one-shot

    Elias Frantar and Dan Alistarh. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning, pages 10323–10337. PMLR, 2023

    2023

  11. [11]

    Gptq: Accurate post-training quantization for generative pre-trained transformers.arXiv preprint arXiv:2210.17323, 2022

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers.arXiv preprint arXiv:2210.17323, 2022

    Pith/arXiv arXiv 2022

  12. [12]

    Second order derivatives for network pruning: Optimal brain surgeon.Advances in neural information processing systems, 5, 1992

    Babak Hassibi and David Stork. Second order derivatives for network pruning: Optimal brain surgeon.Advances in neural information processing systems, 5, 1992

    1992

  13. [13]

    Structured pruning for deep convolutional neural networks: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):2900–2919, 2023

    Yang He and Lingao Xiao. Structured pruning for deep convolutional neural networks: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):2900–2919, 2023

    2023

  14. [14]

    Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    2022

  15. [15]

    Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b.arXiv preprint arXiv:2310.06825, 2023

    Pith/arXiv arXiv 2023

  16. [16]

    Everybody prune now: Structured pruning of llms with only forward passes.arXiv preprint arXiv:2402.05406, 2024

    Steven Kolawole, Lucio Dery, Jean-François Kagy, Virginia Smith, Graham Neubig, and Ameet Talwalkar. Everybody prune now: Structured pruning of llms with only forward passes.arXiv preprint arXiv:2402.05406, 2024

    arXiv 2024

  17. [17]

    Probe pruning: Accelerating llms through dynamic pruning via model probing.arXiv preprint arXiv:2502.15618, 2025

    Qi Le, Enmao Diao, Ziyan Wang, Xinran Wang, Jie Ding, Li Yang, and Ali Anwar. Probe pruning: Accelerating llms through dynamic pruning via model probing.arXiv preprint arXiv:2502.15618, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Dynamic low-rank adaptation based pruning algorithm for large language models

    Linfeng Li and Lei Guo. Dynamic low-rank adaptation based pruning algorithm for large language models. In 2024 7th International Conference on Pattern Recognition and Artificial Intelligence (PRAI), pages 1094–1099. IEEE, 2024

    2024

  19. [19]

    Slimgpt: Layer-wise structured pruning for large language models.Advances in Neural Information Processing Systems, 37:107112–107137, 2024

    Gui Ling, Ziyang Wang, Yuliang Yan, and Qingwen Liu. Slimgpt: Layer-wise structured pruning for large language models.Advances in Neural Information Processing Systems, 37:107112–107137, 2024

    2024

  20. [20]

    Llm-pruner: On the structural pruning of large language models

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. Llm-pruner: On the structural pruning of large language models. Advances in Neural Information Processing Systems, 36:21702–21720, 2023

    2023

  21. [21]

    Shortgpt: Layers in large language models are more redundant than you expect

    Xin Men, Mingyu Xu, Qingyu Zhang, Qianhao Yuan, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, and Weipeng Chen. Shortgpt: Layers in large language models are more redundant than you expect. InFindings of the Association for Computational Linguistics: ACL 2025, pages 20192–20204, 2025

    2025

  22. [22]

    Can a suit of armor conduct electricity? a new dataset for open book question answering

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. InProceedings of the 2018 conference on empirical methods in natural language processing, pages 2381–2391, 2018

    2018

  23. [23]

    Winogrande: An adversarial winograd schema challenge at scale.Communications of the ACM, 64(9):99–106, 2021

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale.Communications of the ACM, 64(9):99–106, 2021

    2021

  24. [24]

    Zico Kolter

    Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter. A simple and effective pruning approach for large language models.arXiv preprint arXiv:2306.11695, 2023. 12 APREPRINT- JULY28, 2026

    Pith/arXiv arXiv 2023

  25. [25]

    Alpaca: A strong, replicable instruction-following model.Stanford Center for Research on Foundation Models, 3(6):7, 2023

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Alpaca: A strong, replicable instruction-following model.Stanford Center for Research on Foundation Models, 3(6):7, 2023

    2023

  26. [26]

    Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    Pith/arXiv arXiv 2023

  27. [27]

    uller, Jonas M. K

    Yifan Yang, Kai Zhen, Bhavana Ganesh, Aram Galstyan, Goeric Huybrechts, Markus M"uller, Jonas M. K"ubler, Rupak Vignesh Swaminathan, Athanasios Mouchtaris, Sravan Babu Bodapati, et al. Wanda++: Pruning large language models via regional gradients. InFindings of the Association for Computational Linguistics: ACL 2025, pages 4321–4333, 2025

    2025

  28. [28]

    Hellaswag: Can a machine really finish your sentence? InProceedings of the 57th annual meeting of the association for computational linguistics, pages 4791–4800, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? InProceedings of the 57th annual meeting of the association for computational linguistics, pages 4791–4800, 2019

    2019

  29. [29]

    Loraprune: Structured pruning meets low-rank parameter-efficient fine-tuning

    Mingyang Zhang, Hao Chen, Chunhua Shen, Zhen Yang, Linlin Ou, Xinyi Yu, and Bohan Zhuang. Loraprune: Structured pruning meets low-rank parameter-efficient fine-tuning. InFindings of the Association for Computa- tional Linguistics: ACL 2024, pages 3013–3026, 2024

    2024

  30. [30]

    Blockpruner: Fine-grained pruning for large language models

    Longguang Zhong, Fanqi Wan, Ruijun Chen, Xiaojun Quan, and Liangzhi Li. Blockpruner: Fine-grained pruning for large language models. InFindings of the Association for Computational Linguistics: ACL 2025, pages 5065–5080, 2025

    2025

  31. [31]

    A survey on model compression for large language models.Transactions of the Association for Computational Linguistics, 12:1556–1577, 2024

    Xunyu Zhu, Jian Li, Yong Liu, Can Ma, and Weiping Wang. A survey on model compression for large language models.Transactions of the Association for Computational Linguistics, 12:1556–1577, 2024. A Perplexity Scaling Figure 4 visualizes the perplexity scaling behavior on LLaMA-2-7B across pruning ratios. Both FLAP and TriSP degrade gracefully at moderate r...

    2024