Pith. sign in

REVIEW 32 cited by

SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2301.00774 v3 pith:6XMQUCC4 submitted 2023-01-02 cs.LG

Elias Frantar , Dan Alistarh This is my paper · ORCID
classification cs.LG
keywords modelssparsegptaccuratelyavailablemassiveone-shotprunedsparsity
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

We show for the first time that large-scale generative pretrained transformer (GPT) family models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal loss of accuracy. This is achieved via a new pruning method called SparseGPT, specifically designed to work efficiently and accurately on massive GPT-family models. We can execute SparseGPT on the largest available open-source models, OPT-175B and BLOOM-176B, in under 4.5 hours, and can reach 60% unstructured sparsity with negligible increase in perplexity: remarkably, more than 100 billion weights from these models can be ignored at inference time. SparseGPT generalizes to semi-structured (2:4 and 4:8) patterns, and is compatible with weight quantization approaches. The code is available at: https://github.com/IST-DASLab/sparsegpt.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 32 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 69 citations worldwide. Full citation record

  1. Is Your NPU Ready for LLMs? Dissecting the Hidden Efficiency Bottlenecks in Mobile LLM Inference

    cs.AR 2026-07 conditional novelty 7.0 of 10

    Cross-layer measurements of five mobile LLM frameworks on CPU/GPU/NPU reveal amplified NPU framework gaps, a prefill–decode backend phase split, and up to ~55% NPU energy savings from scheduling fixes.

  2. You Only Prune Once: Designing Calibration-Free Model Compression With Policy Learning

    cs.CL 2025-01 conditional novelty 7.0 of 10

    PruneNet learns a calibration-free stochastic policy to prune FFN rows in LLMs while preserving the singular-value spectrum, retaining about 80% of LLaMA-2-7B's zero-shot accuracy at 30% compression.

  3. Reduced Matrix Multiplication: Input-Adaptive Matrix-Product Reduction for LLM Inference

    cs.LG 2026-08 conditional novelty 6.0 of 10

    Selecting the highest-norm columns of the activation matrix inside each matrix product preserves quality at reduced compute, with attention far more reducible than MLP layers.

  4. Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Unsigned differential activations locate a few GLU-MLP neurons whose zeroing surgically destabilizes demographic bias while retaining ~99.5% of measured capabilities.

  5. Rethinking Depth Pruning for Vision Transformers: A Heterogeneity-Aware Perspective

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Heterogeneity-aware depth pruning of attention and activation layers, guided by a polynomial model-accuracy predictor, delivers up to 1.58× speedup on DeiT-B and 5.19× when combined with width pruning.

  6. Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models

    cs.LG 2026-01 reject novelty 6.0 of 10

    SALR combines static pruning of frozen weights with a trainable truncated-SVD low-rank residual adapter to match LoRA accuracy at 50% sparsity, cutting model size ~2x and giving ~1.7x inference speedup.

  7. Detection Transformers Under the Knife: A Neuroscience-Inspired Approach to Ablations

    cs.CV 2025-07 conditional novelty 6.0 of 10

    Ablation experiments on DETR, DDETR, and DINO show a clear resilience gradient, with DINO's static content queries becoming largely expendable after training.

  8. The Case for Instance-Optimized LLMs in OLAP Databases

    cs.DB 2025-07 conditional novelty 6.0 of 10

    IOLM-DB compresses a general-purpose LLM on a per-query basis to speed up LLM-invoking OLAP queries, reporting throughput gains up to 3.31x and smaller models in a three-workload prototype.

  9. CoNNect: Connectivity-Based Regularization for Structural Pruning

    cs.LG 2025-02 conditional novelty 6.0 of 10

    CoNNect is a differentiable connectivity regularizer that encourages sparse but connected networks, improving structural pruning accuracy when added to DepGraph and LLM-pruner.

  10. The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws

    cs.LG 2025-01 conditional novelty 6.0 of 10

    Replacing the parameter count in a Chinchilla-style scaling law with the average active parameter count during pre-training predicts final loss for both dense and sparsely pre-trained LLMs.

  11. TrimLLM: Progressive Layer Dropping for Domain-Specific LLMs

    cs.LG 2024-12 conditional novelty 6.0 of 10

    Progressive layer dropping during domain fine-tuning can halve an LLM's depth with small accuracy loss, yielding 2-5x throughput gains on consumer GPUs.

  12. Preserving Deep Representations In One-Shot Pruning: A Hessian-Free Second-Order Optimization Framework

    cs.LG 2024-11 conditional novelty 6.0 of 10

    SNOWS prunes vision networks in one shot by optimizing a K-step nonlinear reconstruction objective with Hessian-free Newton steps, improving accuracy over layer-wise least-squares methods.

  13. SAM Decoding: Speculative Decoding via Suffix Automaton

    cs.CL 2024-11 conditional novelty 6.0 of 10

    By replacing n-gram matching with suffix automaton longest-suffix retrieval over a static corpus and the current text, SAM-Decoding accelerates LLM generation and adds speedup when combined with EAGLE-2.

  14. P$^2$ Law: Scaling Law for Post-Training After Model Pruning

    cs.AI 2024-11 conditional novelty 6.0 of 10

    Post-training loss of a pruned LLM follows a power-law curve fixed by pre-pruning model size, pruning rate, number of tokens, and the model's original loss.

  15. PALS: Percentile-Aware Layerwise Sparsity for LLM Pruning

    cs.CL 2026-07 conditional novelty 5.0 of 10

    Activation-percentile-based per-layer sparsity allocation improves LLaMA-2-7B pruning by 15% perplexity over uniform Wanda, while gradient-based allocation fails worse than random.

  16. Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM

    cs.CL 2025-10 conditional novelty 5.0 of 10

    A single per-layer error tolerance, instead of a per-layer rank budget, improves zero-shot low-rank compression of LLMs and VLMs empirically, but the theorem that this uniform tolerance is Pareto-optimal is proven onl...

  17. On the transferability of Sparse Autoencoders for interpreting compressed models

    cs.LG 2025-07 conditional novelty 5.0 of 10

    Pruning a pretrained sparse autoencoder can produce an interpretability tool for a WANDA-pruned LLM that is roughly comparable to retraining an SAE on the pruned model, though with notable caveats in the reported metrics.

  18. A Sparsity Predicting Approach for Large Language Models via Activation Pattern Clustering

    cs.LG 2025-07 reject novelty 5.0 of 10

    A clustering method for LLM activation patterns achieves up to 79% centroid precision and a best perplexity of 12.49, but the reported perplexity assumes 100% accurate cluster selection and no predictor is built.

  19. Neural Parameter Search for Slimmer Fine-Tuned Models and Better Transfer

    cs.LG 2025-05 conditional novelty 5.0 of 10

    Neural Parameter Search (NPS) prunes fine-tuned models by evolutionary reweighting of magnitude-based task vector subspaces, improving transfer, fusion, and compression.

  20. Constrained Edge AI Deployment: Fine-Tuning vs Distillation for LLM Compression

    cs.LG 2025-05 conditional novelty 5.0 of 10

    Under identical MLP-only pruning schedules, KL self-distillation matches or exceeds cross-entropy fine-tuning for recovering test accuracy of a pruned 7B language model, with a 2.5 to 5.5 point edge at 50% retention.

  21. Lillama: Large Language Models Compression via Low-Rank Feature Distillation

    cs.LG 2024-12 conditional novelty 5.0 of 10

    Lillama compresses LLMs by SVD-initialized low-rank layers trained with a local Teacher plus Student activation distillation loss, achieving 20-40% parameter reduction with only 13 million calibration tokens.

  22. Curvature-Weighted Capacity Allocation: A Minimum Description Length Framework for Layer-Adaptive Large Language Model Optimization

    cs.IT 2026-03 reject novelty 4.0 of 10

    A curvature-based layer gain is converted into optimal allocation and pruning via water-filling and convex penalties, but the experiments substitute the baseline's influence scores and show only modest, mixed gains.

  23. Enhancing Large Multimodal Models with Adaptive Sparsity and KV Cache Compression

    cs.AI 2025-07 conditional novelty 4.0 of 10

    A TPE-guided search over layer-wise pruning ratios and KV cache bit-widths compresses LLaVA-1.5 7B/13B with small accuracy loss, outperforming Wanda and SparseGPT on most tested benchmarks.

  24. Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compression

    cs.LG 2025-05 conditional novelty 4.0 of 10

    ACBench tests compressed LLMs on agentic tasks and finds 4-bit quantization keeps tool use and workflow generation strong while hurting real-world application performance.

  25. CE-LSLM: Efficient Large-Small Language Model Inference and Communication via Cloud-Edge Collaboration

    cs.NI 2025-05 reject novelty 4.0 of 10

    CE-LSLM reports lower edge inference latency by reusing cloud LLM KV caches in edge SLMs, but the cross-model reuse mechanism is asserted, not demonstrated.

  26. Position: Enough of Scaling LLMs! Lets Focus on Downscaling

    cs.CL 2025-05 conditional novelty 4.0 of 10

    The paper advocates for downscaling LLMs, derives that training CO2 scales linearly with parameters and data, and proposes a condition under which ensembles of pruned small models beat the original at equal compute.

  27. The Efficiency vs. Accuracy Trade-off: Optimizing RAG-Enhanced LLM Recommender Systems Using Multi-Head Early Exit

    cs.IR 2025-01 reject novelty 4.0 of 10

    A GCN retriever plus multi-head early exit speeds up LLM click-through rate prediction, but the reported AUC numbers are internally inconsistent.

  28. Edge Intelligence in Civil Aviation: Paradigms, Techniques, and Applications

    cs.AI 2026-07 conditional novelty 3.0 of 10

    A survey of edge-AI techniques and four cloud–edge–device deployment paradigms, mapped to civil aviation applications and future trends.

  29. CPTQuant - A Novel Mixed Precision Post-Training Quantization Techniques for Large Language Models

    cs.CL 2024-12 reject novelty 3.0 of 10

    CPTQuant proposes three sensitivity-based mixed precision quantization methods and claims up to 4x compression on BERT and OPT models, but the comparisons and method details are incomplete.

  30. The Hitchhiker's Guide to Agentic AI: From Foundations to Systems

    cs.AI 2026-06 unverdicted novelty 2.0 of 10

    A survey-style reference book mapping the full agentic-AI stack from transformer internals to production deployment, with no new research result.

  31. The 4/$\delta$ Bound: Designing Predictable LLM-Verifier Systems for Formal Method Guarantee

    cs.AI 2025-11 reject novelty 2.0 of 10

    The 4/δ bound is the mean of four geometric distributions, not a new theorem, and the simulation validation is circular.

  32. On Accelerating Edge AI: Optimizing Resource-Constrained Environments

    cs.LG 2025-01 conditional novelty 2.0 of 10

    The paper argues that model compression, neural architecture search, and compiler optimizations work together to accelerate edge AI, but it provides no new experimental evidence.

Pith tools