Pith. sign in

REVIEW 3 major objections 8 minor 82 references

Binary codes on a hypersphere compress LLM weights to ~2 bits without codebooks

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-10 03:47 UTC pith:A3KFYMMT

load-bearing objection Promising codebook-free 2-bit LLM compression, but the actual effective bit-width is never reported in the results, making the central storage claim unverifiable. the 3 major comments →

arxiv 2607.08643 v1 pith:A3KFYMMT submitted 2026-07-09 cs.LG

BiSCo-LLM: Lookup-Free Binary Spherical Coding for Extreme Low-Bit Large Language Model Compression

Yuantian Shao , Peisong Wang , Zhilei Liu , Chuangyi Li , Yuanteng Chen , Pengcheng Xie , Yiwu Yao , Zhihui Wei
show 1 more author
Jian Cheng
This is my paper
classification cs.LG
keywords weightcompressionsphericalbinaryexplicitlow-bitstoragebisco-llm
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that large language model weights can be compressed to approximately 2 bits per weight without storing explicit vector quantization codebooks. The central mechanism is binary spherical quantization (BSQ): each chunk of pretrained weights is mapped onto a unit hypersphere and binarized into a sign pattern, producing a bit-packed payload that carries directional information without any lookup table. A compact neural decoder shared across all weight chunks of a given Transformer module category reconstructs the original weight block from these binary codes. A second BSQ stage then encodes the residual error left by the first stage, rather than naively enlarging a single binary code — the paper shows that most of the implicit code space of a large one-stage binary code is never occupied by the finite set of pretrained weight chunks, so residual allocation is more efficient than code-space enlargement. After each module category (attention projections, MLP projections) is compressed and replaced, category-wise recovery distillation is performed with the compressed category active in the full model, to close the gap between local weight reconstruction quality and assembled model behavior. A small 1% of sensitive channels are stored in 8-bit as a stabilization mechanism, counted separately from the main BSQ payload. On Qwen3-8B, the method achieves WikiText-2 perplexity of 10.18 versus 9.73 for the full-precision model, and an average downstream accuracy of 68.05 versus 69.92, with all auxiliary components (binary codes, decoder parameters, protected channels, LoRA adapters, metadata) included in the reported storage budget.

Core claim

The load-bearing finding is that the direction of a weight chunk on a unit hypersphere, encoded as a binary sign pattern, carries enough information for a shared neural decoder to reconstruct the chunk to within a useful accuracy — and that the residual error from this first reconstruction is itself structured enough to benefit from a second binary spherical coding stage. This two-stage base-plus-residual decomposition outperforms a single larger binary code at the same nominal bit rate, because the finite set of pretrained weight chunks occupies only a vanishingly small fraction of a large implicit binary code space (the paper measures approximately 524,286 occupied patterns out of 2^64 on

What carries the argument

Binary Spherical Quantization (BSQ) maps a weight chunk to a latent vector, normalizes it onto a unit hypersphere, and binarizes it into sign values. The straight-through estimator handles gradient flow through the non-differentiable sign operation. A two-stage residual pipeline trains the second BSQ codec on the reconstruction error of the first. Category-wise recovery distillation trains LoRA adapters after each Transformer module category is replaced, with the compressed category active in the full model context. An 8-bit protected-channel path stores 1% of sensitive channels separately, selected by activation-weighted magnitude scoring.

Load-bearing premise

The storage claim depends on the neural decoder parameters being small enough to amortize across all weight chunks in a module category. If decoder size must grow with model scale or with the diversity of weight distributions across layers, the real bit budget could exceed 2 bits per weight, undermining the central storage advantage over explicit codebook methods.

What would settle it

If a model with substantially more layers or more diverse weight distributions required decoder parameters that grew faster than linearly with the number of weight chunks, the real bit-per-weight budget would exceed 2 bits and the comparison against explicit-VQ methods would no longer hold.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If codebook-free binary spherical coding generalizes beyond Qwen3-8B, deployment pipelines for multi-hundred-billion-parameter models could avoid the storage and lookup overhead of explicit VQ codebooks entirely, simplifying both checkpoint distribution and inference kernel design.
  • The residual-stage architecture suggests a natural rate-distortion curve: additional BSQ stages could be added or removed to hit specific storage targets, making the method adaptable to heterogeneous deployment budgets without re-architecting the base codec.
  • The finding that category-wise recovery outperforms layer-wise recovery implies that compression-aware distillation should be organized by functional module type rather than by layer index, which could change how recovery training is scheduled in other extreme-compression pipelines.
  • The 8-bit protected-channel mechanism, if it scales, suggests that a very small fraction of channels carries most of the irreducible sensitivity at 2 bits, and that protecting them is more storage-efficient than increasing the global bit budget.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper presents BiSCo-LLM, a codebook-free binary spherical coding framework for extreme low-bit (approximately 2-bit) LLM weight compression. The method has three main components: (1) a base BSQ codec that maps weight chunks onto a unit hypersphere and binarizes them into sign codes, with reconstruction performed by shared neural decoders rather than explicit codebook lookup; (2) a second-stage residual BSQ codec that allocates additional bits to the reconstruction error left by the base stage; and (3) category-wise recovery distillation performed after replacing each Transformer module category. An auxiliary 8-bit protected-channel path handles sensitive channels. The paper reports results on Qwen3-8B, achieving WikiText-2 perplexity of 10.18 (vs. 9.73 FP16) and average downstream accuracy of 68.05 (vs. 69.92 FP16) over a seven-task set.

Significance. The codebook-free binary spherical coding approach is a meaningful contribution to the extreme low-bit LLM compression literature. The residual BSQ design (§III-D) is well-motivated by the code-space utilization diagnostic (Fig. 1) and the granularity sweep (Fig. 2), which together provide concrete evidence that naively enlarging a single-stage binary code is ineffective. The category-wise recovery distillation (§III-F) addresses a real problem — the mismatch between local reconstruction loss and assembled model behavior — and the diagnostic in Fig. 3 provides a useful motivating example. The explicit storage accounting framework (Eq. 3) is a positive design choice that, if properly populated with actual numbers, would enable fair comparison with VQ-based methods. The overall pipeline is thoughtfully constructed.

major comments (3)
  1. The actual effective bit-width is never reported for the main Qwen3-8B result. Equation 3 defines the storage accounting as S_tot = S_code + S_dec + S_8bit + S_LoRA + S_meta, and the paper repeatedly emphasizes that real storage accounting is essential for fair comparison (§III-A, §V). However, Table III omits the bit-width column 'for compactness,' and no experimental section reports the computed bpw for the Qwen3-8B result. The nominal code rate (S*b/d) is discussed in §III-A, but the real rate including decoder parameters, LoRA adapters, protected-channel payloads, and metadata is never stated. This is load-bearing for the central claim that BiSCo-LLM operates 'under a real storage budget of approximately 2 bits per weight.' Without the actual number, the comparison to other 2-bit methods (AQLM, QuIP#, UniSVQ) cannot be verified, and the reader cannot assess whether decoder amortizis.
  2. Table III shows entirely blank entries for BiSCo-LLM on LLaMA3-8B (all columns are dashes). The experimental setup (§IV-A) states the plan to evaluate on 'LLaMA-3 8B, Qwen3 8B,' and Table III includes the LLaMA3-8B section with baselines populated, but the BiSCo-LLM row is empty. The paper acknowledges this: 'the BISCO-LLM results on LLaMA3-8B remain to be filled in' (§IV-C). This means the method is validated on only one model, which is insufficient for a claim that 'codebook-free spherical coding can preserve model behavior under an extreme low-bit storage budget.' Generalization across model families is a load-bearing requirement for this claim.
  3. The average accuracy comparison in Table III is not apples-to-apples. The table caption states that BiSCo-LLM's average on Qwen3-8B is computed over seven tasks (BoolQ, RTE, WinoGrande, ARC-Easy, ARC-Challenge, OpenBookQA, PIQA), excluding HellaSwag, while the FP16/BF16 baseline average of 69.92 appears to include HellaSwag (the HS column shows a dash for BiSCo-LLM but is listed for baselines). The paper acknowledges that 'HellaSwag results are not yet available for BISCO-LLM on Qwen3-8B' (§IV-C). Comparing averages computed over different task sets can be misleading, especially when the gap to the baseline is only 1.87 points. The comparison to vector baselines (AQLM, QuIP#, UniSVQ) is similarly affected if their averages include HellaSwag. The paper should either complete the HellaSwag evaluation or recompute all averages over the common task subset.
minor comments (8)
  1. §IV-A uses future tense throughout ('We plan to evaluate,' 'we will report,' 'we will also evaluate'). For a submitted manuscript, these should be rewritten as completed experiments or clearly labeled as planned experiments.
  2. Table III caption states that 'C4 perplexity, RTE, OpenBookQA, MMLU, and bit-width columns are omitted for compactness.' However, RTE and OpenBookQA columns are actually present in the table. The caption should be corrected.
  3. The notation in Eq. (7) introduces L_ent with a gamma parameter that is not defined in the equation or surrounding text. Clarifying whether gamma is a fixed hyperparameter or learned would help reproducibility.
  4. Fig. 3 caption references multiple hyperparameter settings with date-stamped identifiers (e.g., '260612_080538'). These are not interpretable by readers and should be replaced with descriptive labels.
  5. §III-D references 'the paper should present the two-stage form as the default setting unless later experiments show that a third stage gives a clearly better real-bpw trade-off.' This self-referential phrasing should be rephrased.
  6. The term 'approximately 2 bits per weight' is used throughout but never given a precise numeric value. Even an approximate range (e.g., 2.0–2.3 bpw) would make the claim more precise.
  7. Table I uses '16D' and '32D' notation without explicitly stating whether these refer to chunk dimension d, binary length b, or both. Clarification would help.
  8. Reference [28] (Leech lattice VQ) and [31] (PocketLLM) are cited in §II-B but appear to be concurrent or very recent work; the paper should clarify the temporal relationship and whether these methods are directly comparable.

Circularity Check

0 steps flagged

No circularity found: BSQ encoding, residual decomposition, and recovery distillation are standard formulations not defined in terms of their target results

full rationale

The paper's core technical derivation is self-contained and does not exhibit circular reasoning. The BSQ encoding (Eqs. 4-8) maps weight chunks to binary spherical codes via hypersphere normalization and sign binarization — this is a standard formulation from visual tokenization (Zhao et al. 2024, ref [36]) adapted to weight compression, not defined in terms of the target perplexity or accuracy. The residual decomposition (Eqs. 9-11) defines the second-stage codec as encoding the reconstruction error r^(1) = x - x_hat^(1), which is a standard residual coding formulation, not a self-referential definition. The code-space utilization diagnostic (Fig. 1) provides independent empirical motivation: it measures that only ~524,286 of 2^64 possible code patterns are occupied by Qwen3-8B weight chunks, supporting the residual design without circularity. The granularity sweep (Fig. 2) independently shows that enlarging a one-stage code does not monotonically improve results. The category-wise recovery distillation (Eq. 15) uses standard KD and hidden-state alignment losses, not defined in terms of the final accuracy. The 8-bit protected-channel selection (Eqs. 12-13) uses activation statistics and weight magnitudes as independent criteria. The main comparison (Table III) evaluates against external baselines (AQLM, QuIP#, UniSVQ) on standard benchmarks. While the paper does not report the final computed bpw in Table III (a transparency concern, not a circularity issue), the storage accounting framework (Eq. 3) is additive and non-circular. No step in the derivation chain reduces to its own inputs by construction.

Axiom & Free-Parameter Ledger

8 free parameters · 4 axioms · 2 invented entities

The paper introduces no new physical entities or postulated objects. The neural decoder is a standard architecture. The free parameters are mostly empirical hyperparameters typical of compression pipelines, though several key values (loss weights, temperature, LoRA rank) are unreported, which limits reproducibility. The axioms are domain assumptions standard in the quantization literature, with the shared-decoder assumption being the most load-bearing and least independently verified.

free parameters (8)
  • chunk dimension d = 32 (default)
    Controls the granularity of weight chunking; chosen empirically based on ablation in Table I and Fig. 2.
  • binary code length b = 32 or 64 per stage
    Determines the implicit code space size; set to achieve target bit rate b/d ≈ 1-2.
  • number of residual stages S = 2
    Default two-stage base+residual; chosen based on Table I ablation.
  • protected-channel ratio ρ = 0.01 (1%)
    Selected based on Table VI showing diminishing returns beyond 1%.
  • loss weights λ_c, λ_e = not specified
    Codec training loss weights in Eq. 6; values not reported in the paper.
  • entropy temperature τ = not specified
    Controls softness of binary assignment in Eq. 8; value not reported.
  • recovery loss weights λ_kd, λ_hid, λ_aux = not fully specified
    Weights for distillation objective in Eq. 15; specific values for final reported results not given.
  • LoRA rank and alpha = not specified
    Used in category-wise and end-to-end recovery; not reported for the main result.
axioms (4)
  • domain assumption Straight-through estimator provides unbiased or sufficiently low-variance gradients for binary quantization of weight chunks.
    Eq. 5 uses STE for the non-differentiable sign operation; this is standard in BSQ/VQ-VAE literature but unverified for the weight compression setting.
  • domain assumption Weight chunks from the same module category across layers share sufficient distributional similarity to be decoded by a single shared neural decoder.
    The category-batched codec optimization (§III-G) assumes all layers in a category can share one decoder, which is critical for amortization.
  • domain assumption Local weight reconstruction error (MSE) is a useful proxy for downstream model quality before recovery distillation.
    The BSQ codec is trained with MSE (Eq. 7), but the paper itself shows (Fig. 3) that local loss reduction does not guarantee global recovery.
  • domain assumption The 8-bit protected channel set selected by activation-weighted magnitude scores captures the most compression-sensitive channels.
    Eq. 12-13 define the selection criterion; the paper verifies 1% helps (Table II) but does not prove this is optimal.
invented entities (2)
  • Category-shared neural BSQ decoder independent evidence
    purpose: Reconstructs weight chunks from binary spherical codes without storing explicit codebook centroids.
    The decoder is a standard neural network trained on weight chunks; its effectiveness is measured by reconstruction error and downstream accuracy. No new physical entity is postulated.
  • Coupled intermediate-channel protection for MLP independent evidence
    purpose: Selects the same intermediate index across gate_proj, up_proj, and down_proj for 8-bit protection to preserve structural coupling.
    This is an algorithmic design choice, not a postulated entity. Its effectiveness is measured in Table II.

pith-pipeline@v1.1.0-glm · 28249 in / 2885 out tokens · 475881 ms · 2026-07-10T03:47:57.612824+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) are increasingly constrained by memory capacity, weight bandwidth, and checkpoint storage during deployment. Existing low-bit compression methods mainly follow two directions. Scalar or group-wise quantization is simple and compatible with efficient low-precision kernels, but its representation capacity becomes limited when the target budget approaches 2 bits per weight. Vector-quantized weight compression provides a richer block-level representation, but usually introduces explicit codebooks, index lookup, and additional storage accounting. This paper presents BiSCo-LLM, a codebook-free binary spherical coding framework for extreme low-bit LLM weight compression. The core pipeline is built on three components. First, local weight chunks are mapped onto a unit hypersphere and binarized into compact spherical codes, so that the main payload is a bit-packed sign stream rather than explicit VQ centroids. Second, a residual BSQ stage encodes the reconstruction error left by the base spherical codec, providing an explicit rate-distortion path without stored codebooks. Third, category-wise recovery distillation is performed after replacing each Transformer module category, reducing the mismatch between local weight reconstruction and assembled model behavior. A small 8-bit protected-channel path is used as an auxiliary stabilization mechanism for sensitive channels and is counted separately from the BSQ payload. The reported storage budget includes binary codes, neural decoders, protected-channel payloads, LoRA adapters, and metadata.

Figures

Figures reproduced from arXiv: 2607.08643 by Chuangyi Li, Jian Cheng, Peisong Wang, Pengcheng Xie, Yiwu Yao, Yuanteng Chen, Yuantian Shao, Zhihui Wei, Zhilei Liu.

Figure 1
Figure 1. Figure 1: Empirical utilization of the implicit BSQ code space on Qwen3-8B [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Nominal-rate-matched BSQ granularity sweep under category-prefix [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Diagnostic comparison between layer-wise local convergence and prefix-level model behavior on Qwen3-8B. Left: all-layer normalized local losses [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

82 extracted references · 82 canonical work pages · 20 internal anchors

  1. [1]

    Language models are few-shot learners,

    T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert- V oss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Am...

    work page 2020

  2. [2]

    LLaMA: Open and Efficient Foundation Language Models

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “Llama: Open and efficient foundation language models,”arXiv preprint arXiv:2302.13971, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  3. [3]

    GPT-4 Technical Report

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkatet al., “Gpt-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  4. [4]

    DeepSeek-V3 Technical Report

    A. Liu, B. Feng, B. Xue, B. Wanget al., “V3 technical report,”arXiv preprint arXiv:2412.19437, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  5. [5]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

    D. Guo, D. Yang, H. Zhang, J. Songet al., “R1: Incentivizing rea- soning capability in llms via reinforcement learning,”arXiv preprint arXiv:2501.12948, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  6. [6]

    Kimi K2: Open Agentic Intelligence

    Kimi Team, “Kimi k2: Open agentic intelligence,”arXiv preprint arXiv:2507.20534, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  7. [7]

    GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models

    A. Zeng, X. Lv, Q. Zheng, Z. Houet al., “Arc foundation models,”arXiv preprint arXiv:2508.06471, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  8. [8]

    GPTQ: Accurate post-training quantization for generative pre-trained transformers,

    E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh, “GPTQ: Accurate post-training quantization for generative pre-trained transformers,” in International Conference on Learning Representations, 2023

    work page 2023

  9. [9]

    Flexgen: High-throughput generative inference of large language models with a single gpu,

    Y . Sheng, L. Zheng, B. Yuan, Z. Li, M. Ryabinin, D. Y . Fu, Z. Xie, B. Chen, C. Barrett, J. E. Gonzalez, P. Liang, C. R ´e, I. Stoica, and C. Zhang, “Flexgen: High-throughput generative inference of large language models with a single gpu,” inInternational Conference on Machine Learning, 2023

    work page 2023

  10. [10]

    FineQuant: Unlocking Efficiency with Fine-Grained Weight-Only Quantization for LLMs

    Y . J. Kim, R. Henry, R. Fahim, and H. H. Awadalla, “Finequant: Unlocking efficiency with fine-grained weight-only quantization for llms,”arXiv preprint arXiv:2308.09723, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  11. [11]

    AWQ: Activation-aware weight quantization for large language model com- pression and acceleration,

    J. Lin, J. Tang, H. Tang, S. Yang, X. Dang, and S. Han, “AWQ: Activation-aware weight quantization for large language model com- pression and acceleration,” inProceedings of Machine Learning and Systems, 2024

    work page 2024

  12. [12]

    Apple intel- ligence foundation language models,

    T. Gunter, Z. Wang, C. Wang, R. Pang, A. Narayananet al., “Apple intel- ligence foundation language models,”arXiv preprint arXiv:2407.21075, 2024

    work page arXiv 2024

  13. [13]

    Cerebrum (AIOS SDK): A Platform for Agent Development, Deployment, Distribution, and Discovery

    B. Rama, K. Mei, and Y . Zhang, “Cerebrum (aios sdk): A platform for agent development, deployment, distribution, and discovery,”arXiv preprint arXiv:2503.11444, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  14. [14]

    The OpenHands Software Agent SDK: A Composable and Extensible Foundation for Production Agents

    X. Wang, S. Rosenberg, J. Michelini, C. Smith, H. Tranet al., “Open- hands software agent sdk,”arXiv preprint arXiv:2511.03690, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  15. [15]

    ZeroQuant: Efficient and affordable post-training quantization for large- scale transformers,

    Z. Yao, R. Y . Aminabadi, M. Zhang, X. Wu, C. Li, and Y . He, “ZeroQuant: Efficient and affordable post-training quantization for large- scale transformers,” inAdvances in Neural Information Processing Systems, 2022. 14

    work page 2022

  16. [16]

    OmniQuant: Omnidirectionally calibrated quantization for large language models,

    W. Shao, M. Chen, Z. Zhang, P. Xu, L. Zhao, Z. Li, K. Zhang, P. Gao, Y . Qiao, and P. Luo, “OmniQuant: Omnidirectionally calibrated quantization for large language models,” inInternational Conference on Learning Representations, 2024

    work page 2024

  17. [17]

    LLM.int8(): 8-bit matrix multiplication for transformers at scale,

    T. Dettmers, M. Lewis, Y . Belkada, and L. Zettlemoyer, “LLM.int8(): 8-bit matrix multiplication for transformers at scale,” inAdvances in Neural Information Processing Systems, 2022

    work page 2022

  18. [18]

    RPTQ: Reorder-based post-training quantization for large language models,

    Z. Yuan, C. Xue, Y . Chen, Q. Wu, and G. Sun, “RPTQ: Reorder-based post-training quantization for large language models,” 2023

    work page 2023

  19. [19]

    SmoothQuant: Accurate and efficient post-training quantization for large language models,

    G. Xiao, J. Lin, M. Seznec, H. Wu, J. Demouth, and S. Han, “SmoothQuant: Accurate and efficient post-training quantization for large language models,” inInternational Conference on Machine Learn- ing, 2023

    work page 2023

  20. [20]

    SpQR: A sparse-quantized representation for near-lossless LLM weight compres- sion,

    T. Dettmers, R. Svirschevski, V . Egiazarian, D. Kuznedelev, E. Frantar, S. Ashkboos, A. Borzunov, T. Hoefler, and D. Alistarh, “SpQR: A sparse-quantized representation for near-lossless LLM weight compres- sion,” inInternational Conference on Learning Representations, 2024

    work page 2024

  21. [21]

    SqueezeLLM: Dense-and-sparse quantization,

    S. Kim, C. Hooper, A. Gholami, M. W. Mahoney, Y . Shao, and K. Keutzer, “SqueezeLLM: Dense-and-sparse quantization,” inInterna- tional Conference on Machine Learning, 2024

    work page 2024

  22. [22]

    Atom: Low-bit quantization for efficient and accurate LLM serving,

    Y . Zhao, J. Lin, J. Tang, S. Yang, and S. Han, “Atom: Low-bit quantization for efficient and accurate LLM serving,” inProceedings of Machine Learning and Systems, 2024

    work page 2024

  23. [23]

    QuaRot: Outlier-free 4-bit inference in rotated LLMs,

    S. Ashkboos, M. L. Croci, M. G. d. Nascimento, T. Hoefler, and J. Hensman, “QuaRot: Outlier-free 4-bit inference in rotated LLMs,” 2024

    work page 2024

  24. [24]

    SpinQuant: LLM quantization with learned rotations,

    Z. Liu, C. Zhao, I. Fedorov, B. Soran, D. Choudhary, R. Krishnamoorthi, V . Chandra, and Y . Tian, “SpinQuant: LLM quantization with learned rotations,” 2024

    work page 2024

  25. [25]

    Squeeze10-LLM: Squeezing LLMs' Weights by 10 Times via a Staged Mixed-Precision Quantization Method

    Q. Zhu, Y . Ren, L. Yang, M. Lin, Y . Liet al., “Squeeze10-llm: Squeezing llms’ weights by 10 times via a staged mixed-precision quantization method,”arXiv preprint arXiv:2507.18073, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  26. [26]

    ICQuant: Index Coding enables Low-bit LLM Quantization

    X. Li, O. Hanna, C. Fragouli, and S. Diggavi, “Icquant: Index coding enables low-bit llm quantization,”arXiv preprint arXiv:2505.00850, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  27. [27]

    Learning grouped lat- tice vector quantizers for low-bit llm compression,

    X. Zhang, X. Wu, J. Wang, and W. Lin, “Learning grouped lat- tice vector quantizers for low-bit llm compression,”arXiv preprint arXiv:2510.20984, 2025

    work page arXiv 2025

  28. [28]

    Leech Lattice Vector Quantization for Efficient LLM Compression

    T. F. A. van der Ouderaa, M. van Baalen, P. Whatmough, and M. Nagel, “Leech lattice vector quantization for efficient llm compression,”arXiv preprint arXiv:2603.11021, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  29. [29]

    Extreme compression of large language models via additive quantization,

    V . Egiazarian, A. Panferov, D. Kuznedelev, E. Frantar, A. Babenko, and D. Alistarh, “Extreme compression of large language models via additive quantization,” inInternational Conference on Machine Learning, 2024

    work page 2024

  30. [30]

    VPTQ: Extreme low-bit vector post-training quantization for large language models,

    Y . Liu, J. Wen, Y . Wang, S. Ye, L. L. Zhang, T. Cao, C. Li, and M. Yang, “VPTQ: Extreme low-bit vector post-training quantization for large language models,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Miami, Florida, USA: Association for Computational Linguistics, 2024, pp. 8181–8196

    work page 2024

  31. [31]

    Pocketllm: Ulti- mate compression of large language models via meta networks,

    Y . Tian, C. Wang, J. Han, Y . Tang, and K. Han, “Pocketllm: Ulti- mate compression of large language models via meta networks,”arXiv preprint arXiv:2511.17637, 2025

    work page arXiv 2025

  32. [32]

    LiftQuant: Continuous bit-width LLM via dimensional lifting and projection,

    L. He, X. Liu, J. Liu, T. Feng, T. Lu, C. Gan, Z. Peng, Y . Du, H. Yang, Y . Liu, and L. Du, “LiftQuant: Continuous bit-width LLM via dimensional lifting and projection,” 2026

    work page 2026

  33. [33]

    UniSVQ: 2-bit Unified Scalar-Vector Quantization

    H. Wang, H. Zhao, X. Yu, Z. Yao, X. Han, Z. Liu, and M. Sun, “Unisvq: 2-bit unified scalar-vector quantization,”arXiv preprint arXiv:2606.10520, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  34. [34]

    LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization

    H. Wang, X. Yu, H. Zhao, F. Wang, and X. Han, “Lc-qat: Data-efficient 2-bit qat for llms via linear-constrained vector quantization,”arXiv preprint arXiv:2606.10531, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  35. [35]

    Language model beats diffusion: Tokenizer is key to visual generation,

    L. Yu, J. Lezama, N. B. Gundavarapu, L. Versari, K. Sohn, D. Minnen, Y . Cheng, V . Birodkar, A. Gupta, X. Gu, A. G. Hauptmann, B. Gong, M.-H. Yang, I. Essa, D. A. Ross, and L. Jiang, “Language model beats diffusion: Tokenizer is key to visual generation,” inInternational Conference on Learning Representations, 2024

    work page 2024

  36. [36]

    Image and video tokenization with binary spherical quantization,

    Y . Zhao, Y . Xiong, and P. Kr ¨ahenb¨uhl, “Image and video tokenization with binary spherical quantization,” 2024

    work page 2024

  37. [37]

    OWQ: Outlier-aware weight quantization for efficient fine-tuning and inference of large language models,

    C. Lee, J. Jin, T. Kim, H. Kim, and E. Park, “OWQ: Outlier-aware weight quantization for efficient fine-tuning and inference of large language models,” inProceedings of the AAAI Conference on Artificial Intelligence, 2024

    work page 2024

  38. [38]

    AffineQuant: Affine transformation quantization for large language models,

    Y . Ma, H. Li, X. Zheng, F. Ling, X. Xiao, R. Wang, S. Wen, F. Chao, and R. Ji, “AffineQuant: Affine transformation quantization for large language models,” inInternational Conference on Learning Representations, 2024. [Online]. Available: https://openreview.net/ forum?id=of2rhALq8l

    work page 2024

  39. [39]

    LLM-QAT: Data-free quantization aware training for large language models,

    Z. Liu, B. Oguz, C. Zhao, E. Chang, P. Stock, Y . Mehdad, Y . Shi, R. Krishnamoorthi, and V . Chandra, “LLM-QAT: Data-free quantization aware training for large language models,” inFindings of the Associ- ation for Computational Linguistics: ACL 2024. Bangkok, Thailand: Association for Computational Linguistics, 2024, pp. 467–484

    work page 2024

  40. [40]

    BitDistiller: Unleashing the potential of sub-4-bit LLMs via self- distillation,

    D. Du, Y . Zhang, S. Cao, J. Guo, T. Cao, X. Chu, and N. Xu, “BitDistiller: Unleashing the potential of sub-4-bit LLMs via self- distillation,” inProceedings of the 62nd Annual Meeting of the Associa- tion for Computational Linguistics (Volume 1: Long Papers). Bangkok, Thailand: Association for Computational Linguistics, 2024, pp. 102– 116

    work page 2024

  41. [41]

    OneBit: Towards extremely low-bit large language models,

    Y . Xu, X. Han, Z. Yang, S. Wang, Q. Zhu, Z. Liu, W. Liu, and W. Che, “OneBit: Towards extremely low-bit large language models,” inAdvances in Neural Information Processing Systems, 2024

    work page 2024

  42. [42]

    Neural discrete representation learning,

    A. van den Oord, O. Vinyals, and K. Kavukcuoglu, “Neural discrete representation learning,” inAdvances in Neural Information Processing Systems, 2017

    work page 2017

  43. [43]

    GPTVQ: The blessing of dimensionality for LLM quantization,

    M. van Baalen, A. Kuzmin, M. Nagel, P. Couperus, C. Bastoul, E. Mahurin, T. Blankevoort, and P. Whatmough, “GPTVQ: The blessing of dimensionality for LLM quantization,” 2024

    work page 2024

  44. [44]

    QuIP: 2-bit quantization of large language models with guarantees,

    J. Chee, Y . Cai, V . Kuleshov, and C. M. De Sa, “QuIP: 2-bit quantization of large language models with guarantees,” inAdvances in Neural Information Processing Systems, 2023

    work page 2023

  45. [45]

    QuIP#: Even better LLM quantization with hadamard incoherence and lattice codebooks,

    A. Tseng, J. Chee, Q. Sun, V . Kuleshov, and C. De Sa, “QuIP#: Even better LLM quantization with hadamard incoherence and lattice codebooks,” inInternational Conference on Machine Learning, 2024

    work page 2024

  46. [46]

    QTIP: Quantization with trellises and incoherence processing,

    ——, “QTIP: Quantization with trellises and incoherence processing,” 2024

    work page 2024

  47. [47]

    Finite scalar quantization: VQ-V AE made simple,

    F. Mentzer, D. Minnen, E. Agustsson, and M. Tschannen, “Finite scalar quantization: VQ-V AE made simple,” inInternational Conference on Learning Representations, 2024

    work page 2024

  48. [48]

    Open-MAGVIT2: An open-source project toward democratizing auto-regressive visual generation,

    Z. Luo, F. Shi, Y . Ge, Y . Yang, L. Wang, and Y . Shan, “Open-MAGVIT2: An open-source project toward democratizing auto-regressive visual generation,” 2024

    work page 2024

  49. [49]

    MaskBit: Embedding-free image generation via bit tokens,

    M. Weber, L. Yu, Q. Yu, X. Deng, X. Shen, D. Cremers, and L.- C. Chen, “MaskBit: Embedding-free image generation via bit tokens,” Transactions on Machine Learning Research, 2024

    work page 2024

  50. [50]

    SimVQ: Addressing representation collapse in vector quantized models with one linear layer,

    Y . Zhu, T. Ibuki, M. Reid, Y . Matsuo, S. S. Gu, and Y . Iwasawa, “SimVQ: Addressing representation collapse in vector quantized models with one linear layer,” 2024

    work page 2024

  51. [51]

    Restructuring vector quantization with the rotation trick,

    C. Fifty, R. G. Junkins, D. Duan, A. Iyengar, J. W. Liu, E. Amid, S. Thrun, and C. R´e, “Restructuring vector quantization with the rotation trick,” 2024

    work page 2024

  52. [52]

    Scaling image tokenizers with grouped spherical quantization,

    J. Wang, Z. Qin, Y . Zhang, V . T. Hu, B. Ommer, R. Briq, and S. Kessel- heim, “Scaling image tokenizers with grouped spherical quantization,” 2024

    work page 2024

  53. [53]

    SparseGPT: Massive language models can be accurately pruned in one-shot,

    E. Frantar and D. Alistarh, “SparseGPT: Massive language models can be accurately pruned in one-shot,” inProceedings of the 40th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 202. PMLR, 2023

    work page 2023

  54. [54]

    A simple and effective pruning approach for large language models,

    M. Sun, Z. Liu, A. Bair, and J. Z. Kolter, “A simple and effective pruning approach for large language models,” inInternational Conference on Learning Representations, 2024

    work page 2024

  55. [55]

    LLM-Pruner: On the structural pruning of large language models,

    X. Ma, G. Fang, and X. Wang, “LLM-Pruner: On the structural pruning of large language models,” inAdvances in Neural Information Process- ing Systems, 2023

    work page 2023

  56. [56]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” inInternational Conference on Learning Representations, 2022

    work page 2022

  57. [57]

    QLoRA: Efficient finetuning of quantized LLMs,

    T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “QLoRA: Efficient finetuning of quantized LLMs,” inAdvances in Neural Infor- mation Processing Systems, 2023

    work page 2023

  58. [58]

    QA-LoRA: Quantization-aware low-rank adaptation of large language models,

    Y . Xu, L. Xie, X. Gu, X. Chen, H. Chang, H. Zhang, Z. Chen, X. Zhang, and Q. Tian, “QA-LoRA: Quantization-aware low-rank adaptation of large language models,” inInternational Conference on Learning Rep- resentations, 2024

    work page 2024

  59. [59]

    LoftQ: LoRA-fine-tuning-aware quantization for large language mod- els,

    Y . Li, Y . Yu, C. Liang, N. Karampatziakis, P. He, W. Chen, and T. Zhao, “LoftQ: LoRA-fine-tuning-aware quantization for large language mod- els,” inInternational Conference on Learning Representations, 2024

    work page 2024

  60. [60]

    LQ-LoRA: Low- rank plus quantized matrix decomposition for efficient language model finetuning,

    H. Guo, P. Greengard, E. P. Xing, and Y . Kim, “LQ-LoRA: Low- rank plus quantized matrix decomposition for efficient language model finetuning,” 2023

    work page 2023

  61. [61]

    L4Q: Parameter efficient quantization- aware fine-tuning on large language models,

    H. Jeon, Y . Kim, and J.-J. Kim, “L4Q: Parameter efficient quantization- aware fine-tuning on large language models,” inProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers). Vienna, Austria: Association for Computational Linguistics, 2025, pp. 2002–2024. 15

    work page 2025

  62. [62]

    RILQ: Rank-insensitive LoRA-based quantization error compensation for boosting 2-bit large language model accuracy,

    G. Lee, J. Lee, S. Hong, M. Kim, E. Ahn, D.-S. Chang, and J. Choi, “RILQ: Rank-insensitive LoRA-based quantization error compensation for boosting 2-bit large language model accuracy,” inProceedings of the AAAI Conference on Artificial Intelligence, 2025

    work page 2025

  63. [63]

    SERQ: Saliency-aware low-rank error reconstruction for LLM quantization,

    Y . Park, H. Kim, and S. Choi, “SERQ: Saliency-aware low-rank error reconstruction for LLM quantization,” 2026

    work page 2026

  64. [64]

    SpenseGPT: Practical one- shot pruning enabling sparse and dense GEMMs for LLM inference,

    J. Lee, S.-w. Hwang, and S. Rajbhandari, “SpenseGPT: Practical one- shot pruning enabling sparse and dense GEMMs for LLM inference,” 2026

    work page 2026

  65. [65]

    EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillation

    S.-H. Zhang, L.-T. Huang, X.-S. Deng, X.-Y . Zou, C. Wu, N. Li, and S.-Q. Zhang, “EdgeRazor: A lightweight framework for large language models via mixed-precision quantization-aware distillation,” arXiv preprint arXiv:2605.04062, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  66. [66]

    Stanford alpaca: An instruction- following LLaMA model,

    R. Taori, I. Gulrajani, T. Zhang, Y . Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto, “Stanford alpaca: An instruction- following LLaMA model,” https://github.com/tatsu-lab/stanford alpaca, 2023

    work page 2023

  67. [67]

    OpenOrca: An open dataset of GPT augmented FLAN reasoning traces,

    W. Lian, B. Goodson, E. Pentland, A. Cook, C. V ong, and Teknium, “OpenOrca: An open dataset of GPT augmented FLAN reasoning traces,” https://huggingface.co/datasets/Open-Orca/OpenOrca, 2023

    work page 2023

  68. [68]

    The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

    G. Penedo, H. Kydl ´ıˇcek, L. Ben Allal, A. Lozhkov, M. Mitchell, C. Raf- fel, L. V on Werra, and T. Wolf, “The FineWeb datasets: Decanting the web for the finest text data at scale,”arXiv preprint arXiv:2406.17557, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  69. [69]

    RACE: Large-scale ReAding comprehension dataset from examinations,

    G. Lai, Q. Xie, H. Liu, Y . Yang, and E. Hovy, “RACE: Large-scale ReAding comprehension dataset from examinations,” inProceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017, pp. 785–794

    work page 2017

  70. [70]

    Crowdsourcing multiple choice science questions,

    J. Welbl, N. F. Liu, and M. Gardner, “Crowdsourcing multiple choice science questions,” inProceedings of the 3rd Workshop on Noisy User- generated Text, 2017, pp. 94–106

    work page 2017

  71. [71]

    LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models

    Y . Chen, S. Qian, H. Tang, X. Lai, Z. Liu, S. Han, and J. Jia, “Lon- gLoRA: Efficient fine-tuning of long-context large language models,” arXiv preprint arXiv:2309.12307, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  72. [72]

    LongAlign: A Recipe for Long Context Alignment of Large Language Models

    Y . Bai, X. Lv, J. Zhang, Y . He, J. Qi, L. Hou, J. Tang, Y . Dong, and J. Li, “LongAlign: A recipe for long context alignment of large language models,”arXiv preprint arXiv:2401.18058, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  73. [73]

    Pointer Sentinel Mixture Models

    S. Merity, C. Xiong, J. Bradbury, and R. Socher, “Pointer sentinel mixture models,”arXiv preprint arXiv:1609.07843, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  74. [74]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,”Journal of Machine Learning Research, vol. 21, no. 140, pp. 1–67, 2020

    work page 2020

  75. [75]

    BoolQ: Exploring the surprising difficulty of natural yes/no questions,

    C. Clark, K. Lee, M.-W. Chang, T. Kwiatkowski, M. Collins, and K. Toutanova, “BoolQ: Exploring the surprising difficulty of natural yes/no questions,” inProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019, pp. 2924–2936

    work page 2019

  76. [76]

    GLUE: A multi-task benchmark and analysis platform for natural language understanding,

    A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman, “GLUE: A multi-task benchmark and analysis platform for natural language understanding,” inInternational Conference on Learning Rep- resentations, 2019

    work page 2019

  77. [77]

    WinoGrande: An adversarial winograd schema challenge at scale,

    K. Sakaguchi, R. L. Bras, C. Bhagavatula, and Y . Choi, “WinoGrande: An adversarial winograd schema challenge at scale,”Communications of the ACM, vol. 64, no. 9, pp. 99–106, 2021

    work page 2021

  78. [78]

    Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge

    P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord, “Think you have solved question answering? try ARC, the AI2 reasoning challenge,” inarXiv preprint arXiv:1803.05457, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  79. [79]

    Can a suit of armor conduct electricity? a new dataset for open book question answering,

    T. Mihaylov, P. Clark, T. Khot, and A. Sabharwal, “Can a suit of armor conduct electricity? a new dataset for open book question answering,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018, pp. 2381–2391

    work page 2018

  80. [80]

    PIQA: Reasoning about physical commonsense in natural language,

    Y . Bisk, R. Zellers, R. L. Bras, J. Gao, and Y . Choi, “PIQA: Reasoning about physical commonsense in natural language,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 05, 2020, pp. 7432–7439

    work page 2020

Showing first 80 references.