Pith. sign in

REVIEW 5 cited by

Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2309.05516 v5 pith:7IMGNHPU submitted 2023-09-11 cs.CL cs.AIcs.LG

Wenhua Cheng , Weiwei Zhang , Haihao Shen , Yiyang Cai , Xin He , Kaokao Lv , Yi Liu This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords quantizationsignroundaccuracyacrossaveragebitsdescentexceptional
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Large Language Models (LLMs) have demonstrated exceptional proficiency in language-related tasks, but their deployment poses significant challenges due to substantial memory and storage requirements. Weight-only quantization has emerged as a promising solution, significantly reducing memory and storage needs without sacrificing too much performance. In this study, we introduce SignRound, a method that leverages signed gradient descent (SignSGD) to optimize rounding values and weight clipping in just 200 steps. SignRound integrates the advantages of Quantization-Aware Training (QAT) and Post-Training Quantization (PTQ), delivering exceptional results across 2 to 4 bits while minimizing tuning costs and avoiding additional inference overhead. For example, SignRound achieved absolute average accuracy improvements ranging from 6.91% to 33.22% at 2bits, as measured by the average zero-shot accuracy across 11 tasks. It also demonstrates strong generalization in recent models, achieving near-lossless 4-bit quantization in most scenarios. The source code is publicly available at https://github.com/intel/auto-round.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FP4 All the Way: Fully Quantized Training of LLMs

    cs.LG 2025-05 conditional novelty 7.0 of 10

    A 7-billion parameter LLM trained almost entirely in 4-bit float precision matches a standard 16-bit baseline after a brief quantization-aware fine-tuning phase.

  2. MoPEQ: Mixture of Mixed Precision Quantized Experts

    cs.LG 2025-09 conditional novelty 6.0 of 10

    Assigning 2, 3, or 4 bits to MoE experts by Hessian trace sensitivity keeps VLM accuracy close to uniform 4-bit while reducing model size.

  3. Unifying Block-wise PTQ and Distillation-based QAT for Progressive Quantization toward 2-bit Instruction-Tuned LLMs

    cs.LG 2025-06 conditional novelty 6.0 of 10

    UPQ, a progressive FP16-to-INT4-to-INT2 pipeline with teacher-student distillation, is the first to quantize open-source instruction-tuned LLMs to 2-bit without proprietary post-training data.

  4. Dual Precision Quantization for Efficient and Accurate Deep Neural Networks Inference

    cs.CV 2025-05 conditional novelty 6.0 of 10

    DPQ is a post-training quantization algorithm that stores weights in INT4, computes in FP8, and uses Hessian-based group-aware reordering to keep accuracy near the full-precision model.

  5. Qrazor: Reliable and Effortless 4-bit LLM Quantization by Significant Data Razoring

    cs.LG 2025-01 conditional novelty 5.0 of 10

    A post-training quantization scheme that uses per-group leading-one detection to keep four salient bits from an 8/16-bit integer base, achieving 4-bit weights, activations, and KV cache without fine-tuning or rotation.

Pith tools