Pith. sign in

REVIEW 5 cited by

QTIP: Quantization with Trellises and Incoherence Processing

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.11235 v4 pith:62RH3ZGZ submitted 2024-06-17 cs.LG

Albert Tseng , Qingyao Sun , David Hou , Christopher De Sa This is my paper · ORCID
classification cs.LG
keywords quantizationinferenceqtiptrellisachievecodebookcodesdimension
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
abstract

Post-training quantization (PTQ) reduces the memory footprint of LLMs by quantizing weights to low-precision datatypes. Since LLM inference is usually memory-bound, PTQ methods can improve inference throughput. Recent state-of-the-art PTQ approaches use vector quantization (VQ) to quantize multiple weights at once, which improves information utilization through better shaping. However, VQ requires a codebook with size exponential in the dimension. This limits current VQ-based PTQ works to low VQ dimensions ($\le 8$) that in turn limit quantization quality. Here, we introduce QTIP, which instead uses trellis coded quantization (TCQ) to achieve ultra-high-dimensional quantization. TCQ uses a stateful decoder that separates the codebook size from the bitrate and effective dimension. QTIP introduces a spectrum of lookup-only to computed lookup-free trellis codes designed for a hardware-efficient "bitshift" trellis structure; these codes achieve state-of-the-art results in both quantization quality and inference speed.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Pushing the Limits of Large Language Model Quantization via the Linearity Theorem

    cs.LG 2024-11 conditional novelty 7.0 of 10

    A new theorem and method (HIGGS) make per-layer quantization error a reliable predictor of final model perplexity, enabling state-of-the-art data-free and dynamic bit-width LLM compression.

  2. SoftWater: Class-Aware Rate Allocation for Softmax Quantization

    cs.LG 2026-08 conditional novelty 6.0 of 10

    SoftWater, a KL-divergence-based quantizer for LLM softmax heads, allocates bit rate by class frequency and variance and beats WaterSIC at matched head rates on 59 of 60 test points.

  3. Price of metric universality in vector quantization is at most 0.11 bit

    cs.IT 2026-02 conditional novelty 6.0 of 10

    A universal vector-quantization codebook exists that is within 0.11 bit/coordinate of covariance-adaptive waterfilling simultaneously for all input covariances, for Gaussian weights.

  4. High-Rate Nested-Lattice Quantized Matrix Multiplication with Small Lookup Tables

    cs.IT 2025-05 accept novelty 6.0 of 10

    A layered nested-lattice quantizer computes approximate inner products with a lookup table of size 2^{2dR/M}, enabling high-rate quantized matrix multiplication with near-Voronoi distortion.

  5. L$^3$: Large Lookup Layers

    cs.LG 2026-01 conditional novelty 5.0 of 10

    Adding token-ID-routed embedding lookup tables (L3 layers) between decoder layers improves language-model perplexity and downstream scores over iso-FLOP dense and MoE baselines, with static routing that permits cheap ...

Pith tools