REVIEW 2 cited by
SDQ: Sparse Decomposed Quantization for LLM Inference
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recently, large language models (LLMs) have shown surprising performance in task-specific workloads as well as general tasks with the given prompts. However, to achieve unprecedented performance, recent LLMs use billions to trillions of parameters, which hinder the wide adaptation of those models due to their extremely large compute and memory requirements. To resolve the issue, various model compression methods are being actively investigated. In this work, we propose SDQ (Sparse Decomposed Quantization) to exploit both structured sparsity and quantization to achieve both high compute and memory efficiency. From our evaluations, we observe that SDQ can achieve 4x effective compute throughput with <1% quality drop.
Forward citations
Cited by 2 Pith papers
-
Amber Pruner: Leveraging N:M Activation Sparsity for Efficient Prefill in Large Language Models
Amber Pruner proposes training-free N:M activation sparsity for LLM prefill; however, the supplied manuscript body is an unrelated paper.
-
Semantic Retention and Extreme Compression in LLMs: Can We Have Both?
A 25% pruning plus 4-bit quantization configuration retains roughly 20% more benchmark performance than 3-bit quantization alone at matching theoretical compression rates, across two LLMs.
Discussion (0). Continue with ORCID to comment.