{"id":"883ff52d-4655-46bb-b8a8-02e362262cb1","arxiv_id":"2505.03031","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Radio assigns bit depths to LLM weight groups by minimizing a rate-distortion objective via dual ascent, improving perplexity at 2-4 bits compared with GPTQ, AWQ, and OWQ.","lead":"Radio is a new method for compressing large language models that decides how many bits to give each group of weights by balancing a rate-distortion tradeoff. It aims to make 2-4 bit quantization of models up to 70B+ parameters more accurate than existing post-training methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Appendix B's additive distortion decomposition assumes quantization errors across weight groups are uncorrelated; for a fixed model these errors are deterministic, and the paper provides no test that off-diagonal Hessian cross-terms vanish at 3-4 bits.","rationale":"Reading the paper in good faith, I find a coherent rate-distortion formulation, a reproducible-sounding algorithm, and empirical gains that mostly persist on C4 validation (Table 5), although the margins shrink. The reader's conditional verdict is well matched: the paper needs a code release, clearer model-selection reporting, and a check of its theoretical assumptions. My review focuses on the load-bearing theoretical assumption. The derivation of Eq. (5) in Appendix B requires quantization errors across groups to be uncorrelated so that cross terms in the quadratic form vanish. This is not a high-rate approximation that one might reasonably extrapolate; it is a structural requirement for the objective to be additive. For a fixed model, the errors are deterministic, so the only way the cross terms vanish is if the Jacobian's off-diagonal blocks are small in the relevant directions or the errors happen to be orthogonal. Neither is established. The paper itself flags the zero-mean assumption as false, so the linear term in (13) is not actually zero without bias correction; the bias correction partially addresses the mean but not the covariance structure. A direct measurement of the cross-term fraction would settle whether the additive decomposition is numerically adequate. If it is, the optimality claim stands; if not, the method's success is empirical only, and the 'rate-distortion foundations' language overstates the guarantee. Either way, the conditional verdict is appropriate, so I recommend no change to the reader's verdict.","tokens_in":26500,"tokens_out":11357,"duration_ms":108222,"concrete_test":"Quantize OPT-125M with Radio at 3 and 4 bits (group size 512, 128 C4 calibration examples). For each group n, form the error vector Δ_n = Θ_n^q - Θ_n and compute the Jacobian J_n = ∂f/∂Θ_n via autograd on the calibration minibatch. Evaluate the full quadratic form S_full = E_X[Σ_{n,m} Δ_n^T (J_n^T J_m) Δ_m] and the diagonal-only approximation S_diag = Σ_n E_X[Δ_n^T (J_n^T J_n) Δ_n]. Report the ratio |S_full - S_diag| / S_full. If this ratio exceeds, say, 10%, the additive decomposition in Eq. (13) fails, and the optimality of Radio's bit allocation is unsupported. Repeating at 2.5-bit Llama-2-7B would test the regime where the paper claims its largest gains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central optimality claim (Eqs. 4-5) rests on the additive distortion decomposition in Appendix B. Step (b) drops all cross terms E_X[(J^T J)_{nm} Δ_n Δ_m] for n≠m by appealing to 'uncorrelatedness of Δ_1,...,Δ_N'. For a fixed pretrained model, however, the Δ_n = Θ_n^q - Θ_n are deterministic values, not random variables; the expectation over calibration inputs X does not by itself make these cross terms vanish. The Discussion repeats the assertion ('off-diagonal elements disappear in the expectation due to multiplication by uncorrelated quantization errors') but no evidence is given. The paper explicitly acknowledges the zero-mean assumption is false (Sec. 3.2) and patches it with bias correction, yet no analogous patch or check is provided for uncorrelatedness. If the off-diagonal terms are non-negligible at 2-4 bits — plausible because companding and pruning introduce structured, layer-correlated errors — then the objective optimized by Algorithm 1 (the sum of per-group d_n(B_n)) is not the true output distortion, and Eq. (4) does not characterize the optimal bit allocation. This would undercut the theoretical 'foundations' claim even if the empirical perplexity gains survive.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Radio, a post-training LLM weight quantization method based on a rate-distortion formulation. The central idea is to allocate bit depths to weight groups by minimizing an approximate expected-output-distortion objective subject to an average-bit-rate constraint. The authors derive a high-rate approximation in which the per-group distortion is d_n(B_n) = P_n H_n G_n^2 S_n^2 2^(-2B_n), leading to the optimality condition that the marginal distortion per bit be equalized across groups. They solve the relaxed problem by dual ascent, using backpropagated gradient variances to set bit depths, and combine the allocation with companded scalar quantization, bias correction, and row-column grouping. Experiments on OPT and Llama-2 at 3-4 bits and 2.x bits report perplexity and downstream-task results comparing favorably with GPTQ, AWQ, OWQ, QuIP, SqueezeLLM, and OmniQuant.","tokens_in":26780,"tokens_out":7351,"duration_ms":74972,"significance":"If the theoretical derivation were fully justified, Radio would offer a principled, per-group mixed-precision allocation with a clean rate-distortion interpretation, going beyond the coarser mixed-precision schemes in much of the prior literature. The empirical study is broad: it covers multiple model families and scales, includes 2.x-bit results where Radio shows large perplexity improvements over OWQ, provides ablations separating step-size, bit-depth, and companding effects, and reports runtime and kernel-level speedups. The appendices give detailed derivations, algorithm parameters, and a CUDA kernel, and the paper states that code is available. The main caveat is that the optimality claim rests on high-rate and zero-mean/uncorrelated quantization-error assumptions that are acknowledged to be only partially valid at the operating bit depths; the empirical results are promising, but the theoretical 'foundations' claim is not yet established as stated.","major_comments":[{"comment":"The derivation of Eq. (5), and hence the optimality conditions in Eq. (4), requires the quantization errors Δ_1,...,Δ_N to be zero-mean and mutually uncorrelated. Section 3.2 explicitly notes that zero-mean is empirically false and patches it with bias correction, but no analogous patch or check is provided for uncorrelatedness. For a fixed pretrained model, the Δ_n are deterministic functions of the weights, and the expectation over calibration inputs X does not make the cross terms E_X[(J^T J)_{nm}] Δ_n Δ_m vanish for n≠m. The paper should either provide evidence that the off-diagonal Hessian cross-terms are negligible at 2-4 bits, or reformulate the derivation using a stochastic/dithered quantizer that makes the uncorrelatedness assumption meaningful. Without this, Eq. (4) does not actually characterize the rate-distortion optimal bit allocation.","section":"Appendix B, Eqs. (12)-(13)"},{"comment":"The identity E[Δ_n^2(B_n)] = H_n S_n^2 2^(-2B_n) is a high-rate asymptotic result from Gersho and Gray, and the companding transform in Eq. (8) is derived as asymptotically optimal as B→∞. The experiments, however, operate at 2-4 bits, where the asymptotic expansion is strained, and the paper provides no validation that the measured per-group quantization error follows the assumed exponential form at these depths. A direct diagnostic, such as plotting measured group distortion versus 2^(-2B) for representative layers, or a statement quantifying the approximation error, would be needed to support the claim that Algorithm 1 optimizes the true output distortion.","section":"Appendix B, step (c), and Sec. 3.2"},{"comment":"The printed dual-ascent update in Sec. 3.1, B_n ← clamp(1/2 log2(G_n^2 S_n^2 V / (2 ln2)), 0, 8), appears to invert the argument of the logarithm relative to Algorithm 1 line 15 and the optimality condition in Eq. (4). The correct form, following from -2 ln2 H G_n^2 S_n^2 2^(-2B_n) = -V, is B_n ← clamp(1/2 log2(2 ln2 H G_n^2 S_n^2 / V), 0, 8). As printed, Eq. (6) would allocate fewer bits as V increases, which is the opposite of the intended dual-ascent behavior. This inconsistency must be corrected.","section":"Eq. (6) and Algorithm 1, line 15"},{"comment":"The statement that the paper formulates weight quantization 'as a convex optimization problem' is inaccurate for the original objective in Eq. (3), which is a discrete, generally nonconvex least-squares problem. Only the high-rate surrogate with fixed G_n^2 and S_n^2 is convex in the relaxed bit depths B_n. The discussion should be reworded to distinguish the surrogate objective actually optimized from the original problem.","section":"Discussion, Sec. 5"}],"minor_comments":[{"comment":"The appendix says 'Table 6 lists output produced by different quantized Llama-2-70b models,' but Table 6 is already used for running times; the example outputs should be renumbered as a separate table.","section":"Appendix E"},{"comment":"The citation to '(Young et al., 2021)' for coarse 1D grid fine-tuning does not match the reference list, which contains Young et al. (2019) on transform quantization for CNN compression; please correct the year or the reference.","section":"Sec. 3.2"},{"comment":"The labels 'C4 (Full)' and 'WT2 (Full)' in Figure 4 are not defined in the text; please clarify what 'Full' refers to.","section":"Figure 4"},{"comment":"The reproducibility statement mentions a 'GitHub project website' but does not provide the URL; please include the exact repository address.","section":"Reproducibility Statement"},{"comment":"Some 4-bit Radio perplexity values are below the FP16 baseline (e.g., OPT-125M: 27.23 vs. 27.65), which is plausible as a regularization effect but deserves a one-sentence comment in the text.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is applying classical rate-distortion bit allocation to LLM weight compression: per-group bit depths are chosen by dual ascent on gradient variances estimated with autograd, then combined with companded quantization and bias correction. The empirical payoff is real, especially at 2.x bits on Llama-2 (e.g., 7B: 7.05 vs 9.98 PPL at 2.4 bits versus OWQ/256). At 3–4 bits, Radio is competitive and occasionally better than GPTQ, AWQ, and OmniQuant. The empirical claim is the contribution, not the theoretical framing.\n\nThe paper does several things well. The formulation is clean and implementable, the companding step is standard but sensibly applied, and bias correction addresses a real failure of the zero-mean assumption. The experiments are unusually broad: OPT 125M–66B and Llama-2 7B–70B, perplexity plus GSM8K and five common-sense QA benchmarks, plus a custom CUDA kernel. Ablations in Table 3 are useful, and Appendix F's recasting of OBS is a nice pedagogical addition. The reference list covers the relevant quantization and rate-distortion literature; the self-citations to earlier transform-quantization work are relevant, not turf-building.\n\nThe soft spots are real but not fatal. The derivation in Appendix B assumes quantization errors across groups are uncorrelated and that the high-rate formula E[Δ^2] = H S^2 2^{-2B} holds at 2–4 bits. The paper explicitly admits zero-mean error is false and patches it, but never tests uncorrelatedness. The stress-test point is legitimate: for a fixed model, the errors are deterministic, not random, so the additive distortion decomposition is at best a modeling assumption. That undercuts the \"foundations\" language and the exact-optimality claim in Eq. (4), but it does not undercut the empirical gains, which are heuristics that clearly work. The reproducibility section has two small holes: the smoothing factor α in Algorithm 1 is never given a value, and the text says code is on GitHub but no URL appears. The WikiText2 model-selection issue is minor; the paper says it selects on validation and reports test, which is normal.\n\nTake-home: I trust the numbers and would cite this as a competitive post-training quantization method. The theory needs an honest caveat paragraph before I would call it foundational. This deserves peer review, with a request to measure off-diagonal Hessian terms on a small model and fill the reproducibility gaps.","headline":"A real empirical contribution in LLM quantization, packaged with a rate-distortion theory that overclaims its assumptions; worth a serious referee.","tokens_in":27330,"tokens_out":4210,"would_cite":true,"duration_ms":44931,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Quantizing an LLM's weights is rate–distortion optimal when each weight group contributes equal marginal output distortion per bit, and the paper's Radio algorithm realizes this rule, beating GPTQ, AWQ, OWQ, QuIP, SqueezeLLM, and…","keywords":["rate-distortion theory","LLM quantization","post-training compression","mixed-precision bit allocation","companding quantization","dual ascent","perplexity"],"falsifier":"Quantize OPT-1.3B at 3 bits with Radio, then, keeping the average bit rate fixed, move one bit from a low-$G_n^2 S_n^2$ group to a high-$G_n^2 S_n^2$ group and measure calibration-set output MSE; if the equal-marginal-distortion allocation is truly optimal, the perturbed allocation must have equal or higher MSE. As a second check, compute the correlation of quantization error vectors across groups: significant correlations would violate the uncorrelatedness assumption in Appendix B, step (b).","tokens_in":26287,"feed_emoji":"⚖️","tokens_out":14260,"duration_ms":114199,"temperature":0.7,"pith_summary":"The paper establishes a rate–distortion foundation for post-training LLM weight quantization. It argues that, at a fixed average bit budget, the bit depth assigned to each group of weights should equalize the marginal decrease in the model's output distortion per extra bit across groups, so that groups whose quantization errors propagate more strongly to the output receive more bits. From this rule the paper derives a dual-ascent algorithm, Radio, that alternates between estimating gradient variances by backpropagating through the quantized model and updating bit depths. Quantizing with a companding transform and bias correction, Radio matches or beats GPTQ, AWQ, OWQ, QuIP, SqueezeLLM, and OmniQuant in perplexity at 3–4 bits on the OPT and Llama-2 families, and substantially outperforms OWQ at 2.1–2.8 bits. If the theory holds, it provides a principled way to compress any LLM post-training to a user-specified model size or accuracy.","feed_headline":"Rate-distortion rule sets optimal bits for every LLM weight group","feed_subtitle":"A per-group bit rule, applied by the Radio algorithm, beats GPTQ, AWQ, OWQ, and QuIP at 3-4 bits on OPT and Llama-2","key_machinery":"The load-bearing object is the per-group distortion function $d_n(B_n) = P_n H_n G_n^2 S_n^2 2^{-2B_n}$, which decomposes the model's expected squared output error into additive contributions from each weight group. The identity that carries the argument is the high-rate quantization law $\\mathbb{E}[\\Delta_n^2(B_n)] = H_n S_n^2 2^{-2B_n}$ combined with a Jacobian-linearized model, which together imply that the optimal bit allocation equalizes marginal distortion per bit across groups, $-d_n'(B_n)/P_n = V$. This converts a combinatorial mixed-precision search into a smooth convex resource-allocation problem solved by dual ascent, with the gradient variances $G_n^2$ re-estimated by backpropagation at every iteration. The quantizer itself uses companding, a sigmoid transform derived from the cube root of the weight density, so that uniform quantization of the transformed weights approximates the optimal non-uniform quantizer, and bias correction $\\mathbf{b}_n^q \\leftarrow \\mathbf{b}_n + (\\boldsymbol{\\Theta}_n^q - \\boldsymbol{\\Theta}_n)\\bar{\\mathbf{X}}_n$ compensates for the empirically non-zero mean of quantization errors.","core_discovery":"The paper's central discovery is that mixed-precision LLM weight quantization has a closed-form rate–distortion solution. Measuring output distortion as the expected squared deviation of the model's predictions, the distortion contributed by quantizing weight group $n$ at $B_n$ bits is approximately $d_n(B_n) = P_n H_n G_n^2 S_n^2 2^{-2B_n}$, where $P_n$ is the group size, $G_n^2$ is the average squared norm of the output Jacobian with respect to that group's weights, $S_n^2$ is the weight variance, and $H_n$ is a distribution-dependent constant. Because this family is monotone and convex in $B_n$, the constrained optimum equalizes the marginal distortion per bit, $-d_n'(B_n)/P_n = V$ for all $n$, which is equivalent to the allocation rule $B_n = \\tfrac{1}{2}\\log_2(2\\ln 2\\, G_n^2 S_n^2 / V)$ with the dual variable $V$ tuned to meet the bit budget. The paper turns this identity into an algorithm: estimate $G_n^2$ by backpropagating a PCA-projected calibration loss through the quantized model, update bit depths and $V$ by dual ascent, group weights into rows or columns to exploit variance heterogeneity, quantize with a companding transform matched to the weight distribution, and correct the induced output bias by updating each layer's bias vector. Empirically, Radio achieves lower WikiText2 perplexity than GPTQ, AWQ, OWQ, QuIP, SqueezeLLM, and OmniQuant at 3–4 bits on OPT and Llama-2, and lower perplexity than OWQ at 2.1–2.8 bits.","pith_inferences":["The equal-marginal-distortion rule generalizes beyond squared output error: replacing the output Jacobian with gradients of a task loss (e.g., accuracy on GSM8K) would produce a task-aware bit allocation; the paper does not test this, but the derivation does not depend on the loss being squared.","Equation (9) predicts that the bit saving from grouping grows with the heterogeneity of $G_n^2 S_n^2$ across groups, which suggests a testable extension: searching over permutations or orthogonal rotations of each weight matrix to maximize that heterogeneity, beyond the row/column grouping Radio uses, should yield further compression.","The paper's silence on whether quantization errors are uncorrelated across groups is testable: computing the empirical cross-group error correlation matrix on a Radio-quantized model would show whether the additive distortion decomposition (Appendix B, step b) holds, and if not, a covariance-aware bit allocation would be needed.","Table 1 shows that large models (66B, 70B) are compressed almost as well by simple round-to-nearest as by Radio, which suggests a practical scaling rule: for very large models, companding plus bias correction may capture most of the benefit, while the full dual-ascent allocation matters most for small- and medium-size models."],"forward_implications":["At any user-specified average bit rate, Radio's bit-depth assignment is the exact minimizer of the least-squares output distortion up to integer rounding, removing the combinatorial search that restricts mixed-precision methods to coarse bit-depth options.","The method quantizes post-training to any target model size or fractional bit rate and scales to hundreds of billions of parameters, quantizing a 7B model in about 47 minutes.","Radio achieves the lowest WikiText2 perplexity among GPTQ, QuIP, OWQ, AWQ, SqueezeLLM, and OmniQuant at 3–4 bits on the OPT and Llama-2 families, with the largest gains on small models (up to 4.55 perplexity on 3-bit OPT-125M).","At 2.1–2.8 bits, Radio-quantized Llama-2 models are substantially more accurate than OWQ at the same bit rates, with perplexity gaps that grow as bit depth decreases.","Quantizing low-variance weights to zero yields a mild pruning effect that slightly improves generalization, and a custom kernel dequantizes mixed-precision weights on the fly to accelerate matrix-vector multiplication by 1.4–3.3× for 3-bit models."],"supporting_citations":[{"why":"Supplies the high-rate quantization law $\\mathbb{E}[\\Delta^2] = H S^2 2^{-2B}$ and the companding transform that equations (5) and (8) are built on.","marker":"Gersho & Gray, 1991"},{"why":"Provides the dual ascent framework used by Algorithm 1 to alternate bit-depth and dual-variable updates.","marker":"Boyd et al., 2011"},{"why":"GPTQ is the primary baseline and the OBS-derived method that Radio is contrasted with throughout.","marker":"Frantar et al., 2022"},{"why":"OWQ is the baseline for the 2.1–2.8 bit comparison and supplies the code used for RTN and GPTQ baselines.","marker":"Lee et al., 2024"},{"why":"AWQ is the grouped mixed-precision baseline whose overhead and perplexity Radio is compared against.","marker":"Lin et al., 2024"},{"why":"QuIP is a 2-bit quantization baseline that appears in the perplexity and runtime comparisons.","marker":"Chee et al., 2024"},{"why":"OBS is the convex pruning algorithm that GPTQ descends from; Appendix F uses it to contrast pruning with quantization.","marker":"Hassibi & Stork, 1992"},{"why":"Defines the OPT model family on which the main quantization experiments are run.","marker":"S. Zhang et al., 2022"},{"why":"Defines the Llama-2 model family used for the 3–4 bit and 2.x-bit experiments.","marker":"Touvron et al., 2023"},{"why":"C4 supplies the 128 calibration examples used to estimate gradient variances in Algorithm 1.","marker":"Raffel et al., 2020"}],"fun_headline_variants":["Rate-distortion formula sets optimal bits per LLM weight","Radio: closed-form bit allocation beats GPTQ, AWQ, OWQ","Optimal LLM quantization via a simple rate-distortion rule","Mix-precision bits solved: rate-distortion for LLMs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The derivation assumes that the high-rate quantization law (error variance halves per added bit) stays accurate at the 2–4 bit depths used in the experiments and that quantization errors in different weight groups are uncorrelated; if either assumption fails, the additive distortion that Radio optimizes no longer equals the true output distortion.","fun_headline_variants_meta":{"raw":{"variants":["Rate-distortion formula sets optimal bits per LLM weight","Radio: closed-form bit allocation beats GPTQ, AWQ, OWQ","Optimal LLM quantization via a simple rate-distortion rule","Mix-precision bits solved: rate-distortion for LLMs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0002,"raw_usage":{"total_tokens":1416,"prompt_tokens":1024,"completion_tokens":392,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":640,"completion_tokens_details":{"reasoning_tokens":318}},"tokens_in":640,"tokens_out":392,"duration_ms":4009,"temperature":1.0,"reasoning_tokens":318,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:01:33.535058+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Quantize OPT-1.3B at 3 bits with Radio, then, keeping the average bit rate fixed, move one bit from a low-$G_n^2 S_n^2$ group to a high-$G_n^2 S_n^2$ group and measure calibration-set output MSE; if the equal-marginal-distortion allocation is truly optimal, the perturbed allocation must have equal or higher MSE. As a second check, compute the correlation of quantization error vectors across groups: significant correlations would violate the uncorrelatedness assumption in Appendix B, step (b).","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"AWQ is the grouped mixed-precision baseline whose overhead and perplexity Radio is compared against."}],"review_version":1}