SWSC combines channel K-means clustering with an SVD low-rank error correction to compress LLM weights, and reports lower perplexity than RTN quantization on Llama-2-7B Q and K projections at 2 to 3 average bits.
Numerical inverting of matrices of high order,
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
SWSC: Shared Weight for Similar Channel in LLM
SWSC combines channel K-means clustering with an SVD low-rank error correction to compress LLM weights, and reports lower perplexity than RTN quantization on Llama-2-7B Q and K projections at 2 to 3 average bits.