REVIEW 7 cited by
OneBit: Towards Extremely Low-bit Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Model quantification uses low bit-width values to represent the weight matrices of existing models to be quantized, which is a promising approach to reduce both storage and computational overheads of deploying highly anticipated LLMs. However, current quantization methods suffer severe performance degradation when the bit-width is extremely reduced, and thus focus on utilizing 4-bit or 8-bit values to quantize models. This paper boldly quantizes the weight matrices of LLMs to 1-bit, paving the way for the extremely low bit-width deployment of LLMs. For this target, we introduce a 1-bit model compressing framework named OneBit, including a novel 1-bit parameter representation method to better quantize LLMs as well as an effective parameter initialization method based on matrix decomposition to improve the convergence speed of the quantization framework. Sufficient experimental results indicate that OneBit achieves good performance (at least 81% of the non-quantized performance on LLaMA models) with robust training processes when only using 1-bit weight matrices.
Forward citations
Cited by 7 Pith papers
-
LiquidGEMM: Hardware-Efficient W4A8 GEMM Kernel for High-Performance LLM Serving
A W4A8 GEMM kernel using shift-based overflow-safe dequantization and a hardware-scheduled pipeline reports up to 2.9x speedup over prior W4A8 kernels.
-
GeLaCo: An Evolutionary Approach to Layer Compression
Evolutionary search over layer-merging configurations, scored by module-wise activation similarity, yields competitive LLM compression and the first size-quality Pareto fronts.
-
CCQ: Convolutional Code for Extreme Low-bit Quantization in LLMs
CCQ compresses LLMs to 2.0-2.75 bits per weight using convolutional codes and bit-shift decoding, shrinking 671B-parameter models to under 200GB.
-
Q-resafe: Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models
Q-resafe restores much of the safety lost in quantized LLMs by distilling the original model's responses through DPO while selectively updating only safety-critical weights.
-
Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs
The study introduces TruthfulnessEval and reports that 4-bit quantization preserves simple true/false accuracy, but explicit 'lie' prompts make quantized and full-precision LLMs output falsehoods even when internal pr...
-
APT-LLM: Exploiting Arbitrary-Precision Tensor Core Computing for LLM Acceleration
A bit-sliced Tensor Core scheme with a bipolar integer encoding and adaptive kernel autotuning reports 1.65-3.99x speedups for ultra-low-bit GPTQ-quantized LLM inference on three NVIDIA GPUs.
-
Cross-Layer Error Compensation and Finite-Sample Feature-Statistics Matching for Extreme Low-Bit Quantization of Large Language Models
A whole-network final-feature and statistics matching objective improves 1.125-bit and 4.125-bit LLM quantization over layer-local and distillation baselines, but its cross-layer mechanism reduces exactly to final-fea...
Discussion (0). Sign in to comment.