Pith. sign in

REVIEW 17 cited by

RPTQ: Reorder-based Post-training Quantization for Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2304.01089 v4 pith:4JN4UB67 submitted 2023-04-03 cs.CL

RPTQ: Reorder-based Post-training Quantization for Large Language Models

Zhihang Yuan , Lin Niu , Jiawei Liu , Wenyu Liu , Xinggang Wang , Yuzhang Shang , Guangyu Sun , Qiang Wu
show 2 more authors
Jiaxiang Wu Bingzhe Wu
This is my paper
classification cs.CL
keywords rptqchannelsllmsmemoryquantizationquantizingchallengelanguage
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

Large-scale language models (LLMs) have demonstrated impressive performance, but their deployment presents challenges due to their significant memory usage. This issue can be alleviated through quantization. In this paper, we identify that the challenge in quantizing activations in LLMs arises from varying ranges across channels, rather than solely the presence of outliers. To address this challenge, we introduce a quantization method called RPTQ, which utilizes a reorder-based approach. By rearranging the channels and quantizing them in clusters, RPTQ effectively mitigates the impact of range differences between channels. To minimize the overhead of the reorder operation, we fuse it into the layer norm operation and weights in linear layers. In our experiments, RPTQ achieved a significant breakthrough by utilizing 3-bit activation in LLMs for the first time, resulting in a substantial reduction in memory usage. For instance, quantizing OPT-175b can lead to a memory consumption reduction of up to 80%.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. QuantVLA: Scale-Calibrated Post-Training Quantization for Vision-Language-Action Models

    cs.LG 2026-02 unverdicted novelty 7.0

    QuantVLA is the first post-training quantization framework for VLA models that quantizes the diffusion transformer action head and reports higher task success rates than full-precision baselines with roughly 70% memor...

  2. SAB-LVLM: Significance-Aware Binarization for Large Vision-Language Models

    cs.CV 2026-07 unverdicted novelty 6.0

    SAB-LVLM proposes a significance-aware binarization technique for LVLMs that uses modality-guided Hessian-based maps to reweight binarization errors and improve performance under 1-bit constraints.

  3. BitNet Text Embeddings

    cs.CL 2026-06 unverdicted novelty 6.0

    BITEMBED converts LLM backbones to ternary BitNet-style encoders, adapts them with contrastive pre-training and teacher distillation, and produces text embeddings at multiple precisions that perform comparably to full...

  4. OSAQ: Outlier Self-Absorption for Accurate Low-bit LLM Quantization

    cs.LG 2026-05 unverdicted novelty 6.0

    OSAQ uses the low-rank structure of the Hessian to construct a closed-form additive weight transformation that suppresses outliers without changing task loss, enabling better low-bit LLM quantization.

  5. OSAQ: Outlier Self-Absorption for Accurate Low-bit LLM Quantization

    cs.LG 2026-05 unverdicted novelty 6.0

    OSAQ suppresses weight outliers in LLMs via a closed-form additive transformation from the Hessian's stable null space, improving 2-bit quantization perplexity by over 40% versus vanilla GPTQ with no inference overhead.

  6. LBLLM: Lightweight Binarization of Large Language Models via Three-Stage Distillation

    cs.LG 2026-04 unverdicted novelty 6.0

    LBLLM achieves better accuracy than prior binarization methods for LLMs by decoupling weight and activation quantization through initialization, layer-wise distillation, and learnable activation scaling.

  7. Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimate

    cs.LG 2026-04 unverdicted novelty 6.0

    DASH-Q uses a stable diagonal curvature estimate and weighted least squares to achieve robust ultra-low-bit post-training quantization of LLMs, improving zero-shot accuracy by 7% on average over baselines.

  8. Rethinking Residual Errors in Compensation-based LLM Quantization

    cs.LG 2026-04 conditional novelty 6.0

    Redefining residual errors to include compensation-aware discrepancies and realigning calibration to full-precision outputs improves GPTQ and GPTAQ performance on LLMs.

  9. You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations

    cs.CL 2025-11 conditional novelty 6.0

    TAQ estimates per-layer importance from hidden representations and output sensitivity on task calibration data to allocate mixed precision in a training-free PTQ setting, outperforming task-agnostic baselines on accur...

  10. AHCQ-SAM: Toward Accurate and Hardware-Compatible Post-Training Segment Anything Model Quantization

    cs.CV 2025-03 unverdicted novelty 6.0

    AHCQ-SAM introduces ACNR, HLUQ, CAG, and LNQ quantization techniques that deliver 15.2% mAP gain on 4-bit SAM-B and 14.01% J&F gain on 4-bit SAM2-Tiny versus prior PTQ methods.

  11. ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models

    cs.CL 2023-12 unverdicted novelty 6.0

    ASVD compresses LLMs by 10-30% and KV caches by 50% via activation-aware SVD that absorbs outliers into transformed weights and calibrates per-layer sensitivity.

  12. Efficient Reasoning on the Edge

    cs.LG 2026-03 accept novelty 5.5

    LoRA adapters, budget-forced GRPO, dynamic switching, parallel verification and FPTQuant enable practical chain-of-thought reasoning on quantized Qwen2.5-7B for edge devices.

  13. SEPTQ: A Simple and Effective Post-Training Quantization Paradigm for Large Language Models

    cs.CL 2026-04 unverdicted novelty 5.0

    SEPTQ simplifies LLM post-training quantization to two steps via static global importance scoring and mask-guided column-wise weight updates, claiming superior results over baselines in low-bit settings.

  14. MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Design

    cs.LG 2024-12 unverdicted novelty 5.0

    MixLLM uses global output-feature importance to set mixed bit-widths for LLM quantization and adds two-step dequantization plus software pipelining for system efficiency.

  15. A Survey on Efficient Inference for Large Language Models

    cs.CL 2024-04 accept novelty 3.0

    The paper surveys techniques to speed up and reduce the resource needs of LLM inference, organized by data-level, model-level, and system-level changes, with comparative experiments on representative methods.

  16. An Empirical Study of OpenPangu Quantization on Ascend NPUs

    cs.LG 2026-06 unverdicted novelty 2.0

    Empirical benchmarking finds 8-bit weight-only quantization lossless for OpenPangu 1B/7B on Ascend NPUs, 4-bit usable for 7B but harmful for 1B on reasoning/math/code, and 2-bit/binary settings mostly collapse.

  17. An Empirical Study of OpenPangu Quantization on Ascend NPUs

    cs.LG 2026-06 unverdicted novelty 2.0

    Empirical tests show 8-bit weight-only quantization is lossless on both models while 4-bit works for the 7B but harms the 1B on reasoning/math/code tasks, and 2-bit or lower settings collapse performance.