Pith. sign in

REVIEW 9 cited by

Compresso: Structured Pruning with Collaborative Prompting Learns Compact Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.05015 v2 pith:RQVJWTCG submitted 2023-10-08 cs.AI

Song Guo , Jiahang Xu , Li Lyna Zhang , Mao Yang This is my paper · ORCID
classification cs.AI
keywords pruningcompressoalgorithmchallengesperformancecollaborationcollaborativecollection
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
abstract

Despite the remarkable success of Large Language Models (LLMs), the massive size poses significant deployment challenges, particularly on resource-constrained hardware. While existing LLM compression methods focus on quantization, pruning remains relatively unexplored due to the high cost of training-based approaches and data collection challenges. One-shot pruning methods, although cost-effective and data-free, have become dominant in LLM pruning, but lead to performance decline under the structured pruning setting. In this work, we introduce a new paradigm for structurally pruning LLMs, called Compresso. Our approach, through the collaboration of the proposed resource-efficient pruning algorithm and the LLM itself, learns optimal pruning decisions during the training process. Compresso addresses the challenges of expensive training costs and data collection by incorporating Low-Rank Adaptation (LoRA) into the $L_0$ regularization during the instruction tuning process. Then, we further augment the pruning algorithm by introducing a collaborative prompt that fosters collaboration between the LLM and the pruning algorithm, significantly boosting the overall performance. To this end, Compresso prunes LLaMA-7B to 5.4B, maintaining original performance and even surpassing LLaMA-7B in reading comprehension by 2.62%. Extensive experiments demonstrate that Compresso significantly outperforms one-shot pruning baselines across various sparsity ratios, achieving up to 2.21%, 11.43%, 7.04%, and 4.81% higher scores on the commonsense reasoning, reading comprehension, MMLU, and BBH benchmarks, respectively.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models

    cs.LG 2026-01 reject novelty 6.0 of 10

    SALR combines static pruning of frozen weights with a trainable truncated-SVD low-rank residual adapter to match LoRA accuracy at 50% sparsity, cutting model size ~2x and giving ~1.7x inference speedup.

  2. SDMPrune: Self-Distillation MLP Pruning for Efficient Large Language Models

    cs.CL 2025-06 conditional novelty 6.0 of 10

    SDMPrune combines a self-distillation loss with Taylor-based importance scoring to prune only MLP neurons, improving zero-shot performance of compressed LLaMA models over existing pruning methods.

  3. All-in-One Tuning and Structural Pruning for Domain-Specific LLMs

    cs.CL 2024-12 conditional novelty 6.0 of 10

    ATP jointly searches for pruning decisions and fine-tunes LLaMA models with LoRA in one stage, outperforming two-stage pruning on domain-specific tasks.

  4. CLOVER: Cross-Layer Orthogonal Vectors Pruning and Fine-Tuning

    cs.LG 2024-11 reject novelty 6.0 of 10

    Attention pairs (Q-K and V-O) are SVD-decomposed so pruning or fine-tuning touches only a small singular-factor matrix, yielding better pruning tolerance and small PEFT gains.

  5. SOS-LoRA: Static Orthogonal-Subspace Low-Rank Adaptation with Fixed Multi-Scale Scaling

    cs.LG 2026-06 conditional novelty 5.0 of 10

    A LoRA update split into several fixed, differently-scaled low-rank experts with orthogonal input directions improves fine-tuning accuracy at the same parameter count.

  6. Olica: Efficient Structured Pruning of Large Language Models without Retraining

    cs.CL 2025-06 conditional novelty 5.0 of 10

    Olica prunes LLMs without retraining by low-rank decomposing attention-layer matrix products and adding small linear corrections to pruned feed-forward layers, matching or beating retraining-based methods on several b...

  7. PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks

    cs.DC 2025-01 conditional novelty 5.0 of 10

    PICE uses progressive inference, cloud sketches plus parallel edge expansion, to raise LLM serving throughput by 1.5-2x and cut latency by up to 43 percent in testbed experiments.

  8. SlimGPT: Layer-wise Structured Pruning for Large Language Models

    cs.AI 2024-12 conditional novelty 5.0 of 10

    A layer-wise structured pruning method for LLMs that uses batched greedy head and channel selection and a log-increasing per-layer pruning ratio.

  9. Deploying Foundation Model Powered Agent Services: A Survey

    cs.DC 2024-12 accept novelty 4.0 of 10

    This survey proposes a layered framework (execution, resource, model, agent, application) for deploying foundation-model-powered agent services across edge-cloud environments, and reviews optimization techniques at ea...

Pith tools