Pith. sign in

REVIEW 3 major objections 6 minor 47 references

Lightweight and Post-Training Structured Pruning for On-Device Large Lanaguage Models

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read COMP, a post-training structured pruning method, can prune LLaMA-2-7B with only 8GB of memory at a 20% pruning ratio while retaining about 91.2% of its average zero-shot performance.

desk verdict A useful post-training pruning recipe whose headline number is off and whose key neuron-importance metric is never actually tested. read the letter →

arxiv 2501.15255 v1 pith:GRZ5JYUL submitted 2025-01-25 cs.LG cs.AI

Zihuai Xu , Yang Xu , Hongli Xu , Yunming Liao , Zhiwei Yao , Zuan Xie This is my paper · ORCID
classification cs.LGcs.AI
keywords structuredpruningpost-traininglargelanguagemodelsneuronimportancemasktuningmatrixconditionnumberhybridgranularityon-devicedeployment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces COMP, a structured pruning method that compresses a large language model after training, with no fine-tuning, using little GPU memory. The aim is to make pruning practical on the devices where the model will actually run. COMP prunes whole layers first, then individual neurons inside the remaining layers, ranking neurons by a new matrix-condition-number score, and repairs the damage with mask tuning. The reported result is that LLaMA-2-7B can be pruned at a 20% ratio in about 8GB of memory while keeping roughly 91.2% of its average zero-shot accuracy, beating LLM-Pruner, ShortGPT, and SliceGPT on the same benchmark.

What carries the argument

The load-bearing mechanism is the condition-number-based neuron importance metric. For each dense layer, the mask-tuning reconstruction is a least-squares problem whose coefficient matrix is $\hat{A}^T\hat{A}$; the paper argues that pruning neurons that increase the condition number of this matrix most makes the mask solution most sensitive to errors. A second-order Taylor expansion of the condition number, with the Hessian approximated by a diagonal Fisher matrix, yields a per-neuron score $eI_f = -g_f + \tfrac12 F_{ff}$ that sorts neurons for pruning. This score is embedded in an iterative pruning loop: neurons are pruned one dense at a time until the variance of the tuned nonzero mask entries crosses an adaptively raised threshold, and each layer is pruned using the original model's input to stop errors from accumulating. Layer importance, in contrast, is simply one minus the cosine similarity between a layer's input and output, and layers are removed iteratively so later removals see the updated network.

What would settle it

Take a fixed layer-pruning schedule and mask-tuning protocol, then replace COMP's condition-number neuron ordering with random or uniform neuron ordering at the same pruning ratios and compare perplexity on WikiText2. If the randomly pruned model matches COMP's perplexity, the condition-number metric is not doing the work; equivalently, compute the metric's rank correlation with the actual leave-one-out output error for each neuron on the calibration data and check whether low score truly means low damage.

Watch

Extended reading notes

Core claim

The central claim is that post-training structured pruning can be both lightweight and broadly applicable if pruning is done at two granularities and performance is restored by mask tuning rather than fine-tuning. On the paper's own account, COMP first removes low-importance layers, then prunes input neurons inside the denses of surviving layers using an importance score derived from the condition number of the mask-tuning least-squares coefficient matrix. The pruned model's outputs are then reconstructed by solving for a tuned mask, iteratively increasing the number of pruned neurons per dense while keeping the variance of mask entries under a threshold. The paper reports that this pipeline cuts the memory needed to prune a 7B model to 8GB, retains 91.2% of original average zero-shot accuracy at 20% pruning, and outperforms the compared post-training baselines across LLaMA-2, OPT, and ChatGLM3 models.

Load-bearing premise

The whole neuron-pruning stage depends on a mathematical approximation: that a simplified sensitivity score estimated from a diagonal Fisher matrix ranks neurons correctly, so pruning the lowest-scoring ones does the least damage; if that ranking is wrong, the method's choice of which neurons to prune is arbitrary.

Editorial extensions

If this is right

  • Fine-tuning can be removed from the structured-pruning pipeline for LLMs, which removes the need for large labeled datasets and the privacy risk of shipping user data to a cloud server.
  • A 7B-parameter model can be pruned on hardware with roughly 8GB of GPU memory at 20% compression, with most zero-shot accuracy intact.
  • The hybrid layer-plus-neuron strategy dominates either granularity alone, and the gap grows at higher pruning ratios such as 30%.
  • Because the pipeline is agnostic to layer-internal structure, the same pruning recipe applies to LLaMA-2, OPT, and ChatGLM3 families without model-specific adapters.
  • Iterative layer removal and identical-layer-input mask tuning are both shown to matter; recomputing importance after each layer removal cuts perplexity nearly in half when six layers are removed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The condition-number criterion is defined on any linear layer's coefficient matrix, so a natural extension is to apply the same score to other removables such as attention heads or to semi-structured 2:4 masks; the paper does not test those cases.
  • The memory footprint scales with the largest dense input dimension rather than total parameter count, so models with unusually wide linear layers like OPT will not all reach the same 8GB budget; the paper's own OPT results hint at this.
  • A direct test of the neuron-importance metric is to compare COMP's nonrandom neuron selection against random or uniform selection with mask tuning held fixed; if perplexity does not change, the condition-number score is not the reason the method works.
  • The calibration budget is only 10 samples, so the variance of the method across random calibration draws is an open question and a cheap falsifiability check.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. COMP is a post-training structured pruning method for LLMs. It first removes layers based on cosine-similarity redundancy, then performs input-neuron pruning in each remaining layer using a condition-number-based importance metric derived from the mask-tuning normal equations, and finally reconstructs dense outputs by mask tuning. The pruning ratio per layer is allocated via Eq. (10), and an iterative variance-threshold loop determines how many neurons to remove. Experiments on LLaMA-2, OPT, and ChatGLM3 compare with LLM-Pruner, SliceGPT, and ShortGPT on perplexity and zero-shot tasks; the paper reports lower perplexity than baselines at 20-30% pruning and memory usage of 8GB for LLaMA-2-7B.

Significance. The practical motivation is strong: on-device pruning without fine-tuning is an important problem, and the paper demonstrates consistent perplexity improvements over three baselines on five models with only 10 calibration samples. The layer-wise dynamic loading and the use of mask tuning are sensible engineering choices, and the comparison is fairly broad. However, the central novelty—the condition-number neuron-importance metric—is neither theoretically justified nor empirically isolated. The empirical tables support the claim that the full COMP pipeline works better than the tested baselines, but not the claim that the proposed metric is responsible for that advantage. If the missing ablations confirm the metric's value, the paper would be a useful contribution to post-training structured pruning.

major comments (3)
  1. [Section III.B, Eqs. (7)-(9)] The proposed second-order approximation is not actually a Hessian expansion. The Hessian H of κ(A^T A) is replaced by F = gg^T, which for a vector g is rank-one; after assuming diagonality, the "second-order" term reduces to 1/2 g_f^2, a function of first derivatives only, and the off-diagonal curvature that the Taylor expansion would contribute is discarded without justification. Since this metric is the paper's central novelty, the derivation needs either a rigorous justification or a direct validation (e.g., correlation between eI_f and the actual reconstruction-loss increase when pruning f). The sign convention in Eq. (7) versus Eq. (9) also needs clarification: if g = ∂κ/∂mc, pruning neuron f changes mc_f from 1 to 0, giving a first-order term -g_f; the expression for eI_f should be stated in terms of that change, not as a generic importance measure.
  2. [Section V.C and Figure 2] The paper never tests whether the condition-number-based neuron ranking is better than random or uniform selection. In Figure 2, neuron pruning removes equal counts from every dense; the hybrid strategy is compared with layer-only and uniform neuron-only pruning, but not with selective ranking. The ablations in Section V.C cover only iterative layer ordering and identical layer inputs. Consequently, the claim that COMP's hybrid strategy with condition-number-based importance achieves the reported results is not supported by any experiment that isolates the metric. Please add ablations in which the same layer-pruning and mask-tuning pipeline is run with (i) the proposed eI_f ranking, (ii) random neuron selection, (iii) magnitude-based selection, and (iv) uniform per-dense counts.
  3. [Section IV.B, Eq. (10), and Algorithm 1] The per-layer pruning-ratio formula is not reproducible as written. w_l is described as "the harmonic mean of the rest layer's importance," but no normalization is specified, so the r_l values are not guaranteed to sum to the target r; the term n \hat N appears to subtract removed-layer parameters using the current layer's parameter count rather than the removed layers' actual counts; and the variance-threshold loop increments v_T without specifying its step or a termination condition, so the algorithm's behavior at the target ratio is under-specified. Please clarify the formula and the loop, or provide the exact update rules used in the experiments.
minor comments (6)
  1. [Section V.A and References] ShortGPT is cited as [28] (SliceGPT) in the experimental setup; the correct reference is [18].
  2. [Abstract and Table I] The abstract claims a 6.13% improvement, but Table I shows 58.74 vs 55.60 for LLaMA-2-7B at 20% pruning, a relative gain of 5.65%, and no 6.13% appears in the tables; please correct the number or specify the comparison used.
  3. [Section III.A] Equation (3) uses X^{l+1}_t for the output of layer l while X^{l,k} denotes inputs to denses; define the layer-input/output notation consistently.
  4. [Table III and Section IV.B] There are typos in "Pruing ratio" and "remainig"; please proofread these passages.
  5. [Table I] The SliceGPT row for LLaMA-2-13B at 20% shows the same value 929.68 for WikiText2 and PTB; please verify this entry.
  6. [Section VI (Limitation)] The Limitation paragraph reports 30 minutes for LLaMA-2-7B at 30% pruning and about 1 hour for LLaMA-2-13B; consider including a runtime comparison with baselines, since on-device deployment claims depend on practical time as well as memory.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: pruning decisions are evaluated on external held-out benchmarks and no load-bearing self-citation or definitional equivalence is present.

full rationale

I walked the paper's derivation chain. Layer importance (Eq. 3) is an independent cosine-similarity redundancy heuristic; neuron importance (Eq. 9) is derived from a Taylor/Fisher approximation of the condition number of the mask-tuning coefficient matrix (Eqs. 4-8), which is an internally consistent design rather than a definitional equivalence. The pruning ratios (20%, 25%, 30%) and the number of removed layers are fixed externally, and the method is evaluated on held-out WikiText2, PTB, Alpaca, and zero-shot tasks, so no parameter is fitted to the reported benchmark scores. Mask tuning does minimize a reconstruction objective on the same calibration samples used for pruning, but that is a standard compression procedure and not a circular prediction. The runtime limitation acknowledged in Section VI is a performance concern, not a circularity. The main weakness is empirical rather than circular: the ablation study in Section V.C does not compare the Eq. (9) neuron-importance metric against random or uniform neuron selection, so its individual contribution is under-supported; however, the absence of an ablation is a validation gap, not a derivation that reduces to its own inputs. No load-bearing self-citation or imported uniqueness theorem appears in the references.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The method introduces no new physical or architectural entities. Its free parameters are mostly implementation hyperparameters, none fitted to the evaluation sets. The main axiomatic load is the ad hoc Fisher-style approximation of the Hessian of the condition number and the domain assumptions about layer redundancy and mask variance controlling pruning quality.

free parameters (4)
  • number of removed layers n
    Algorithm 1 takes n as input, but the paper never states how n is chosen for each pruning ratio such as 20%, 25%, or 30%. The balance between layer-level and neuron-level removal strongly affects results.
  • variance threshold v_T increment
    Algorithm 1 increases v_T iteratively until the layer reaches its pruning ratio, but the increment step size is unspecified, making exact neuron counts unreproducible.
  • epsilon for matrix inversion
    Section III.B adds a small positive epsilon to A^T A to make it invertible for Cholesky decomposition; the value is not given even though it affects the condition number and the resulting masks.
  • calibration sample count and length = 10 samples, 128 tokens
    Implementation Details fixes 10 random C4 samples truncated to 128 tokens, following LLM-Pruner; results may be sensitive to this small sample and no variance is reported.
assumptions (6)
  • domain assumption Layer redundancy, measured as cosine similarity between a layer's input and output, is a valid proxy for layer importance (Eq. 3).
    This motivates removing low-importance layers; it is not proven and Figure 1 shows the redundancy pattern varies across models.
  • domain assumption Mask tuning can reconstruct a dense layer's output after pruning using a least-squares solve (Eq. 2).
    Borrowed from FPT; the paper does not prove that reconstruction error translates to downstream task preservation.
  • ad hoc to paper The Hessian of the condition number can be approximated by the outer product of its gradient, and off-diagonal terms can be dropped (Eqs. 7-9).
    This is the core new heuristic and is not derived from a standard Fisher information argument; it is assumed to hold for the saliency metric.
  • domain assumption A larger variance of the tuned mask hurts generalization (Section IV.B, citing [33]).
    This is used to stop pruning per dense and to set the threshold v_T; the cited reference is about model selection and overfitting, not specifically about masks.
  • domain assumption Using the original model's input when pruning a layer avoids cumulative error and overfitting (Section IV.B, Cumulative Deviation).
    Supported by the ablation in Table III but presented as a design choice, not a theorem.
  • standard math Positive semidefinite matrix A^T A can be made positive definite by adding epsilon E and inverted via Cholesky decomposition.
    Standard numerical linear algebra; low risk.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Lightweight and Post-Training Structured Pruning for On-Device Large Lanaguage Models." pith.science (2026). https://pith.science/paper/GRZ5JYUL

@misc{pith2026250115255,
  author       = {Pith},
  title        = {Pith review of: Lightweight and Post-Training Structured Pruning for On-Device Large Lanaguage Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GRZ5JYUL}},
  note         = {Machine review of arXiv:2501.15255}
}
read the original abstract

Considering the hardware-friendly characteristics and broad applicability, structured pruning has emerged as an efficient solution to reduce the resource demands of large language models (LLMs) on resource-constrained devices. Traditional structured pruning methods often need fine-tuning to recover performance loss, which incurs high memory overhead and substantial data requirements, rendering them unsuitable for on-device applications. Additionally, post-training structured pruning techniques typically necessitate specific activation functions or architectural modifications, thereby limiting their scope of applications. Herein, we introduce COMP, a lightweight post-training structured pruning method that employs a hybrid-granularity pruning strategy. COMP initially prunes selected model layers based on their importance at a coarse granularity, followed by fine-grained neuron pruning within the dense layers of each remaining model layer. To more accurately evaluate neuron importance, COMP introduces a new matrix condition-based metric. Subsequently, COMP utilizes mask tuning to recover accuracy without the need for fine-tuning, significantly reducing memory consumption. Experimental results demonstrate that COMP improves performance by 6.13\% on the LLaMA-2-7B model with a 20\% pruning ratio compared to LLM-Pruner, while simultaneously reducing memory overhead by 80\%.

Figures

Figures reproduced from arXiv: 2501.15255 by the authors.

Figure 2
Figure 2. Wikitext2 perplexity of different models with three strategies under certain pruning ratios. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the pruning process of COMP. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. GPU memory cost with different pruning methods. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: The perplexity variation with the numbers of removed [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 18 canonical work pages

  1. [1]

    A survey on recent advances in llm-based multi-turn dialogue systems,

    Z. Yi, J. Ouyang, Y . Liu, T. Liao, Z. Xu, and Y . Shen, “A survey on recent advances in llm-based multi-turn dialogue systems,” arXiv preprint arXiv:2402.18013, 2024

    arXiv 2024

  2. [2]

    Toolqa: A dataset for llm question answering with external tools,

    Y . Zhuang, Y . Yu, K. Wang, H. Sun, and C. Zhang, “Toolqa: A dataset for llm question answering with external tools,” Advances in Neural Information Processing Systems , vol. 36, pp. 50 117–50 143, 2023

    2023

  3. [3]

    A comprehensive survey on process-oriented automatic text summarization with exploration of llm-based methods,

    H. Jin, Y . Zhang, D. Meng, J. Wang, and J. Tan, “A comprehensive survey on process-oriented automatic text summarization with exploration of llm-based methods,” arXiv preprint arXiv:2403.02901 , 2024

    arXiv 2024

  4. [4]

    Exploding ai power use: an opportunity to rethink grid planning and management,

    L. Lin, R. Wijayawardana, V . Rao, H. Nguyen, E. W. GNIBGA, and A. A. Chien, “Exploding ai power use: an opportunity to rethink grid planning and management,” in Proceedings of the 15th ACM International Conference on Future and Sustainable Energy Systems , 2024, pp. 434– 441

    work page 2024

  5. [5]

    Carbon emissions and large neural network training,

    D. Patterson, J. Gonzalez, Q. Le, C. Liang, L.-M. Munguia, D. Rothchild, D. So, M. Texier, and J. Dean, “Carbon emissions and large neural network training,” arXiv preprint arXiv:2104.10350 , 2021

    arXiv 2021

  6. [6]

    Drive as you speak: Enabling human-like interaction with large language models in au- tonomous vehicles,

    C. Cui, Y . Ma, X. Cao, W. Ye, and Z. Wang, “Drive as you speak: Enabling human-like interaction with large language models in au- tonomous vehicles,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2024, pp. 902–909

    2024

  7. [7]

    Llm as a system service on mobile devices,

    W. Yin, M. Xu, Y . Li, and X. Liu, “Llm as a system service on mobile devices,” arXiv preprint arXiv:2403.11805 , 2024

    arXiv 2024

  8. [8]

    Language models are few-shot learners advances,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, and A. Askell, “Language models are few-shot learners advances,” in Proceedings of the 34th International Conference on Neural Information Processing Systems , 2020, p. 33

    work page 2020

Show all 47 references
  1. [9]

    Energy and policy consid- erations for modern deep learning research,

    E. Strubell, A. Ganesh, and A. McCallum, “Energy and policy consid- erations for modern deep learning research,” Proceedings of the AAAI Conference on Artificial Intelligence , vol. 34, no. 09, pp. 13 693–13 696, Apr. 2020

    2020

  2. [10]

    Learning both weights and connections for efficient neural network,

    S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and connections for efficient neural network,” Advances in neural information processing systems, vol. 28, 2015

    2015

  3. [11]

    The lottery ticket hypothesis: Finding sparse, trainable neural networks,

    J. Frankle and M. Carbin, “The lottery ticket hypothesis: Finding sparse, trainable neural networks,” in International Conference on Learning Representations, 2018

    2018

  4. [12]

    Stabilizing the lottery ticket hypothesis,

    J. Frankle, G. K. Dziugaite, D. M. Roy, and M. Carbin, “Stabilizing the lottery ticket hypothesis,” arXiv preprint arXiv:1903.01611 , 2019

    1903 arXiv

  5. [13]

    Sparsegpt: Massive language models can be accurately pruned in one-shot,

    E. Frantar and D. Alistarh, “Sparsegpt: Massive language models can be accurately pruned in one-shot,” in International Conference on Machine Learning. PMLR, 2023, pp. 10 323–10 337

    2023

  6. [14]

    Plug-and-play: An efficient post-training pruning method for large language models,

    Y . Zhang, H. Bai, H. Lin, J. Zhao, L. Hou, and C. V . Cannistraci, “Plug-and-play: An efficient post-training pruning method for large language models,” in The Twelfth International Conference on Learning Representations, 2024

    2024

  7. [15]

    Nvidia a100 tensor core gpu: Performance and innovation,

    J. Choquette, W. Gandhi, O. Giroux, N. Stam, and R. Krashinsky, “Nvidia a100 tensor core gpu: Performance and innovation,” IEEE Micro, vol. 41, no. 2, pp. 29–35, 2021

    2021

  8. [16]

    A fast post-training pruning framework for transformers,

    W. Kwon, S. Kim, M. W. Mahoney, J. Hassoun, K. Keutzer, and A. Gholami, “A fast post-training pruning framework for transformers,” Advances in Neural Information Processing Systems , vol. 35, pp. 24 101– 24 116, 2022

    2022

  9. [17]

    Loraprune: Structured pruning meets low-rank parameter-efficient fine- tuning,

    M. Zhang, H. Chen, C. Shen, Z. Yang, L. Ou, X. Yu, and B. Zhuang, “Loraprune: Structured pruning meets low-rank parameter-efficient fine- tuning,” in Findings of the Association for Computational Linguistics ACL 2024, 2024, pp. 3013–3026

    2024

  10. [18]

    Shortgpt: Layers in large language models are more redundant than you expect,

    X. Men, M. Xu, Q. Zhang, B. Wang, H. Lin, Y . Lu, X. Han, and W. Chen, “Shortgpt: Layers in large language models are more redundant than you expect,” arXiv preprint arXiv:2403.03853 , 2024

    2024 arXiv

  11. [19]

    Llm-pruner: On the structural pruning of large language models,

    X. Ma, G. Fang, and X. Wang, “Llm-pruner: On the structural pruning of large language models,” in Advances in Neural Information Processing Systems, 2023

    2023

  12. [20]

    A simple and effective pruning approach for large language models,

    M. Sun, Z. Liu, A. Bair, and J. Z. Kolter, “A simple and effective pruning approach for large language models,” arXiv preprint arXiv:2306.11695 , 2023

    2023 arXiv

  13. [21]

    Parameter-efficient fine- tuning for large models: A comprehensive survey,

    Z. Han, C. Gao, J. Liu, S. Q. Zhang et al. , “Parameter-efficient fine- tuning for large models: A comprehensive survey,” arXiv preprint arXiv:2403.14608, 2024

    2024 arXiv

  14. [22]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685 , 2021

    2021 arXiv

  15. [23]

    Compacter: Efficient low-rank hypercomplex adapter layers,

    R. Karimi Mahabadi, J. Henderson, and S. Ruder, “Compacter: Efficient low-rank hypercomplex adapter layers,” Advances in Neural Information Processing Systems, vol. 34, pp. 1022–1035, 2021

    2021

  16. [24]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,”

  17. [25]

    Privacy-preserving large language models for structured medical information retrieval,

    I. C. Wiest, D. Ferber, J. Zhu, M. van Treeck, S. K. Meyer, R. Juglan, Z. I. Carrero, D. Paech, J. Kleesiek, M. P. Ebert et al., “Privacy-preserving large language models for structured medical information retrieval,” NPJ Digital Medicine, vol. 7, no. 1, p. 257, 2024

    2024

  18. [26]

    Optimizing llm training for financial services: Best practices for model accuracy, risk management, and compliance in ai-powered financial applications,

    D. Paul, G. Namperumal, and Y . Surampudi, “Optimizing llm training for financial services: Best practices for model accuracy, risk management, and compliance in ai-powered financial applications,” Journal of Artificial Intelligence Research and Applications , vol. 3, no. 2, p...

    2023

  19. [27]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al., “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  20. [28]

    Slicegpt: Compress large language models by deleting rows and columns,

    S. Ashkboos, M. L. Croci, M. G. d. Nascimento, T. Hoefler, and J. Hensman, “Slicegpt: Compress large language models by deleting rows and columns,” arXiv preprint arXiv:2401.15024 , 2024

    2024 arXiv

  21. [29]

    Deja vu: Contextual sparsity for efficient llms at inference time,

    Z. Liu, J. Wang, T. Dao, T. Zhou, B. Yuan, Z. Song, A. Shrivastava, C. Zhang, Y . Tian, C. Reet al., “Deja vu: Contextual sparsity for efficient llms at inference time,” in International Conference on Machine Learning. PMLR, 2023, pp. 22 137–22 176. 8

    2023

  22. [30]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologi...

    2019

  23. [31]

    An estimate for the condition number of a matrix,

    A. K. Cline, C. B. Moler, G. W. Stewart, and J. H. Wilkinson, “An estimate for the condition number of a matrix,” SIAM Journal on Numerical Analysis, vol. 16, no. 2, pp. 368–375, 1979

    1979

  24. [32]

    Matrix inversion using cholesky decomposition,

    A. Krishnamoorthy and D. Menon, “Matrix inversion using cholesky decomposition,” in 2013 signal processing: Algorithms, architectures, arrangements, and applications (SPA) . IEEE, 2013, pp. 70–72

    2013

  25. [33]

    Points of significance: model selection and overfitting,

    J. Lever, M. Krzywinski, and N. Altman, “Points of significance: model selection and overfitting,” Nature methods, vol. 13, no. 9, pp. 703–705, 2016

    2016

  26. [34]

    Opt: Open pre-trained transformer language models,

    S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V . Lin et al. , “Opt: Open pre-trained transformer language models,” arXiv preprint arXiv:2205.01068 , 2022

    2022 arXiv

  27. [35]

    Glm-130b: An open bilingual pre-trained model,

    A. Zeng, X. Liu, Z. Du, Z. Wang, H. Lai, M. Ding, Z. Yang, Y . Xu, W. Zheng, X. Xia et al. , “Glm-130b: An open bilingual pre-trained model,” arXiv preprint arXiv:2210.02414 , 2022

    2022 arXiv

  28. [36]

    Pointer sentinel mixture models,

    S. Merity, C. Xiong, J. Bradbury, and R. Socher, “Pointer sentinel mixture models,” arXiv preprint arXiv:1609.07843 , 2016

    2016 arXiv

  29. [37]

    Building a large annotated corpus of english: The penn treebank,

    M. Marcus, B. Santorini, and M. A. Marcinkiewicz, “Building a large annotated corpus of english: The penn treebank,” Computational linguistics, vol. 19, no. 2, pp. 313–330, 1993

    1993

  30. [38]

    Compressing pre-trained language models by matrix decomposition,

    M. B. Noach and Y . Goldberg, “Compressing pre-trained language models by matrix decomposition,” in Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Pr...

    2020

  31. [39]

    BoolQ: Exploring the surprising difficulty of natural yes/no questions,

    C. Clark, K. Lee, M.-W. Chang, T. Kwiatkowski, M. Collins, and K. Toutanova, “BoolQ: Exploring the surprising difficulty of natural yes/no questions,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human La...

    2019

  32. [40]

    Winogrande: An adversarial winograd schema challenge at scale,

    K. Sakaguchi, R. L. Bras, C. Bhagavatula, and Y . Choi, “Winogrande: An adversarial winograd schema challenge at scale,” 2019

    2019

  33. [41]

    Logiqa: A challenge dataset for machine reading comprehension with logical reasoning,

    J. Liu, L. Cui, H. Liu, D. Huang, Y . Wang, and Y . Zhang, “Logiqa: A challenge dataset for machine reading comprehension with logical reasoning,” arXiv preprint arXiv:2007.08124 , 2020

    2007 arXiv

  34. [42]

    Measuring massive multitask language understanding,

    D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt, “Measuring massive multitask language understanding,” arXiv preprint arXiv:2009.03300 , 2020

    2009 arXiv

  35. [43]

    Piqa: Reasoning about physical commonsense in natural language,

    Y . Bisk, R. Zellers, R. Le bras, J. Gao, and Y . Choi, “Piqa: Reasoning about physical commonsense in natural language,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 05, pp. 7432–7439, Apr. 2020

    2020

  36. [44]

    Crowdsourcing multiple choice science questions,

    J. Welbl, N. F. Liu, and M. Gardner, “Crowdsourcing multiple choice science questions,” arXiv preprint arXiv:1707.06209 , 2017

    2017 arXiv

  37. [45]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of machine learning research , vol. 21, no. 140, pp. 1–67, 2020

    2020

  38. [46]

    Cupy: A numpy-compatible library for nvidia gpu calculations,

    R. Nishino and S. H. C. Loomis, “Cupy: A numpy-compatible library for nvidia gpu calculations,” 31st confernce on neural information processing systems, vol. 151, no. 7, 2017. 9

    2017

  39. [2019]

    Available: https://arxiv.org/abs/1711.05101

    [Online]. Available: https://arxiv.org/abs/1711.05101

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.