{"id":"ed46010f-0f66-4cbf-81da-4e52532009b3","arxiv_id":"2501.09107","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A soft-thresholding rule equivalent to selecting the top alpha percent of weights by magnitude is presented as a statistical pre-calibration for LLM quantization, with results close to but not clearly better than calibration-based baselines.","lead":"This paper proposes a data-free 'pre-calibration' step that classifies weights before quantization, using a soft-thresholding rule to single out a small fraction of important weights. The authors argue this preserves the weight distribution and can match calibration-based quantization methods, but the theoretical foundation has serious gaps and the gains are modest.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Section 5 proof only upper-bounds one KL term by the adaptive LASSO penalty, and it models quantized weights as a smoothed convolution with independent noise; on actual discrete quantization, the claimed KL-minimization proxy is not established.","rationale":"The reader identified the same load-bearing weakness: the adaptive LASSO penalty is only an upper bound on a KL-related term, not a valid proxy, and the treatment of quantization error as a small, constant-mean, independent perturbation is inconsistent with the actual deterministic rounding used in the algorithm. My stress test sharpens this into two precise mathematical objections: (1) the direction of Claim 2's inequality means a small penalty does not control the KL term, and (2) Eq. (6)'s convolution model contradicts the atomic nature of quantized weights, making the stated KL divergence ill-defined. Both objections target the paper's main novelty, the theoretical foundation; without it, the method is a magnitude-based outlier selector, which is a known heuristic rather than a new information-preserving pre-calibration step. The empirical results are competitive but not decisive: the reported perplexity and zero-shot scores are close to RTN/AWQ/SpQR, often at higher average bit-widths, and the hyperparameters alpha and clipping range are tuned on the evaluation benchmarks. No code is provided, and no simple top-k magnitude baseline is compared, so the empirical claim is under-supported. Since the reader's REJECT verdict is already based on this core failure, my assessment does not move the verdict; it confirms it.","tokens_in":15511,"tokens_out":2882,"duration_ms":34527,"concrete_test":"Run a microbenchmark on 1,000 weights drawn from a heavy-tailed distribution: quantize to 4-bit min-max grids under two policies with the same average bit-width—(A) keep the top 8% magnitudes in higher precision, (B) keep a random 8% in higher precision. Compute the empirical KL between original and quantized weight distributions and the adaptive-LASSO penalty sum_i |hat w_i / w_i| for both. If policy A shows far lower penalty but similar or higher KL than policy B, the proxy is vacuous. Separately, re-derive Eq. (6) with the deterministic rounding map q(w) = round((w - z)/s)*s + z to verify that the induced distribution is atomic, not a convolution with independent noise.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that pre-calibration preserves weight distribution and hence matches calibration-based PTQ accuracy—rests entirely on the theoretical bridge in Section 5. Claim 2 proves only |mu_delta sum_i f''(w_i)(hat w_i - w_i)| <= C(sum_i |hat w_i/w_i| + 1). That is an upper bound, not an equivalence: minimizing the adaptive LASSO penalty does not guarantee the KL-related term is small. Moreover, mu_delta is treated as a constant, but in the actual pipeline it is determined by the minmax quantization bins, which depend on the classification decision being made. More fundamentally, Eq. (6) represents the quantized-weight distribution as f_hatW = fW * f_delta with delta independent of w. Real quantization maps each weight deterministically to a discrete grid value, producing an atomic distribution; a KL divergence between continuous fW and that atomic distribution is not well-defined (infinite unless supports match). Without a valid distribution-preservation argument, Algorithm 1 reduces to selecting the top-alpha weights by absolute value—a magnitude threshold—and the empirical parity with calibration-based methods has no supporting mechanism beyond tuned hyperparameters.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a data-free 'pre-calibration' step for post-training quantization of LLMs. The method uses an adaptive LASSO penalty with pseudo-activations (XX^T = bI) to derive a soft-thresholding rule (Eq. 5), which classifies weights as salient or non-salient; the two classes are then quantized separately with minmax quantization. The authors claim this classification approximately minimizes the KL divergence between original and quantized weight distributions, thereby preserving Shannon information, and that the resulting quantized models match the accuracy of calibration-based PTQ methods on LLaMA, LLaMA-2, OPT, Falcon, and Code-Llama, while being far faster. Section 5 presents a derivation intended to show that the adaptive LASSO penalty is a proxy for the KL divergence.","tokens_in":15731,"tokens_out":4105,"duration_ms":42608,"significance":"If the central claim were valid, a fast, data-free, deterministic pre-calibration method that matches calibration-based PTQ would be practically valuable, especially for deployment robustness across domains. The proposed method is computationally attractive: soft-thresholding is simple, quantization is fast, and no calibration data are needed. However, the theoretical foundation in Section 5 is not sound as written, and the empirical parity claim is weakened by per-model and per-bit-width tuning of the outlier percentage and clipping range on the same benchmarks used for evaluation. The method's actual behavior may reduce to a magnitude-based outlier selection, without the promised information-theoretic guarantee.","major_comments":[{"comment":"The distributional model underlying the KL derivation is invalid for actual quantization. Eq. (6) represents the quantized-weight density as a convolution f_hatW = fW * f_delta with delta independent of w, but in the proposed algorithm each weight is mapped deterministically to a discrete grid value, producing an atomic distribution. The KL divergence between a continuous density fW and an atomic distribution is infinite unless the supports coincide, so the approximate expansion in Eqs. (7)-(11) is not a valid approximation of the quantity the paper claims to minimize. Additionally, Eq. (10) writes D_KL as a sum over sampled weights fW(hat w_i) ln(f_hatW(hat w_i)/fW(hat w_i)), whereas KL divergence for continuous densities is an integral over the support; the notation conflates a density with an empirical sum.","section":"Section 5, Eqs. (6)-(11)"},{"comment":"Claim 2 establishes only an upper bound: |mu_delta sum_i f''(w_i)(hat w_i - w_i)| <= C(sum_i |hat w_i/w_i| + 1). An upper bound does not imply that minimizing the adaptive LASSO penalty minimizes the KL-related term; a small penalty can still leave the upper bound loose and the KL term large. The paper then states that 'we may replace D_KL(fW||f_hatW) with sum_i |hat w_i/w_i|', but this replacement is not justified by the preceding inequality. Furthermore, the first term mu_delta sum_i f'(w_i) in Eq. (11) is treated as constant, yet mu_delta is the mean quantization error, which depends on the minmax quantization bins and therefore on the very salient/non-salient classification the algorithm is choosing; it is not a fixed constant across the optimization.","section":"Section 5, Claim 2 and Eq. (12)"},{"comment":"The closed-form soft-thresholding solution relies on the pseudo-activation assumption XX^T = bI. This assumption is introduced as 'pseudo activations' but is not justified for any actual LLM layer: weight matrices in transformers do not generally satisfy orthogonality of activations, and the method does not use real activations at all. If the assumption fails, the derivation of Eq. (5) from the adaptive LASSO loss does not hold, and the algorithm's classification rule has no direct connection to the optimization problem (2). The paper needs either a justification for why the orthogonal-activation idealization captures the behavior of real layers or a demonstration that the soft-thresholding rule is optimal under a different principled criterion.","section":"Section 4.1, Eq. (5)"},{"comment":"The central empirical claim is weakened by hyperparameter tuning on the evaluation benchmarks. The outlier percentage alpha is chosen separately per model and per bit-width (e.g., Table 3: alpha = 8%, 6%, 5%, 9% across LLaMA-7B/13B/30B and 3-bit settings; Table 6: alpha = 4%, 5%, 6%), and Section 6 states that the 3-bit clipping range of 90-95% was selected because it 'yields similar accuracy' on the same tasks reported in the tables. These choices mean part of the reported performance is fitted to the test benchmarks, so the comparison does not independently establish that the pre-calibration mechanism, rather than the tuned alpha and clipping values, is responsible for the results.","section":"Algorithm 1 and Section 6"},{"comment":"The accuracy comparisons are not at matched average bit-widths. For example, in Table 3, the 4-bit pre-calibration setting for LLaMA-7B uses 4.81 avg bits while AWQ uses 4.25 bits, and in Table 4 the same pre-calibration configuration is compared against 4.25-bit baselines. The claim of being 'on par' with calibration-based methods is therefore at a higher bit budget in several rows, which changes the interpretation of the perplexity and zero-shot results; an iso-bit comparison is needed before concluding parity.","section":"Tables 3 and 4"}],"minor_comments":[{"comment":"The expansion of the quadratic term omits the trace: (WX - hat W X)(WX - hat W X)^T is a matrix, not a scalar, and the loss should be its trace (or Frobenius norm squared). The subsequent coordinate-wise decomposition in Eq. (4) implicitly assumes this, but the notation should be corrected.","section":"Section 4.1, Eq. (3)"},{"comment":"The procedure for reducing lambda' until alpha percent of weights are selected is underspecified: the paper does not state the step size, the initialization, or the stopping rule, which makes the exact classification rule non-reproducible from the text alone.","section":"Algorithm 1"},{"comment":"There are typographical issues such as 'Sta tistical' in the title and some missing spaces in the abstract; these should be cleaned up.","section":"Abstract and title"},{"comment":"The quantization time comparison would be more informative if the hardware, software versions, and any warm-up or batching details were reported for all baselines, since the speedup claim depends on implementation factors.","section":"Table 2"}],"recommendation":"reject","confidential_remarks":"The paper does not release code, and the appendix hyperparameter tables are terse (e.g., SpQR settings). In my view, the theoretical bridge in Section 5 is not a local fix: the distributional model is incompatible with deterministic quantization, and the penalty-proxy claim relies on an upper bound in the wrong direction. The empirical section also tunes key hyperparameters on the evaluation benchmarks, so the central parity claim is not independently established. A resubmission would need a substantially different theoretical justification and controlled, iso-bit experiments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I'll be direct: the central theoretical claim doesn't survive contact with the paper. Section 5 tries to show that adaptive LASSO is a proxy for minimizing KL divergence between original and quantized weights, but the derivation models quantized weights as a convolution of the original density with a small independent noise. Real quantization maps each weight to a discrete grid, so the quantized distribution is atomic; the KL divergence from a continuous density to that atomic distribution isn't even finite unless the supports match. Even granting the continuous approximation, Claim 2 only upper-bounds one term by C(Σ|ŵ_i/w_i| + 1), so a small penalty doesn't force the KL term small. And μδ is treated as a constant when it actually depends on the minmax bins that are being decided. So the paper's stated mechanism—distribution preservation—is not established. What's left is Algorithm 1: pick the top α% of weights by |w_i|, quantize them at higher precision. That's a known outlier-aware baseline (SpQR, AWQ, LLM.int8() all use similar ideas, though with different saliency criteria). The paper never compares against a simple magnitude-threshold baseline, and α is tuned per model/bit-width on the same benchmarks, with the 90-95% clipping range also chosen on eval data. The perplexity gains are modest too: pre-calibration trails SpQR in most 4-bit and 3-bit comparisons, and the average bit widths are often higher (4.81 vs 4.25 for LLaMA-7B).\n\nThat said, the paper does have value. The pseudo-activation trick that reduces the objective to soft-thresholding is clean and the speed numbers (10-100x faster than AWQ/SpQR) are credible. The Code-Llama/HumanEval/MBPP results make a real point: calibration on the wrong domain can hurt, and a data-free method can be more robust. That is the paper's most honest contribution, and it deserves attention.\n\nWho is this for? PTQ practitioners who want a quick, data-free initialization. I would not cite it as a theoretical result, but I might point to the domain-shift experiment in a footnote. It deserves a serious referee—the empirical claims are concrete and checkable—but in current form I'd recommend rejection. The theory should be either fixed or dropped, a magnitude baseline added, code released, and the story reframed around the robustness/speed trade-off rather than information preservation.","headline":"A fast data-free PTQ method that is really magnitude-threshold outlier selection, wrapped in an unsound KL-divergence argument; the domain-shift experiments are the only genuinely useful part.","tokens_in":16315,"tokens_out":3570,"would_cite":false,"duration_ms":38836,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A data-free pre-calibration step can match calibration-based methods for quantizing large language models.","keywords":["Post-training quantization","adaptive LASSO","large language models","weight distribution preservation","KL divergence","soft-thresholding","model compression","calibration-free quantization"],"falsifier":"Compare the perplexity and zero-shot accuracy of the proposed thresholding against a plain magnitude-based outlier selector that keeps the same fraction of largest absolute weights in high precision, holding the total bit budget fixed. If the two match across models and bit-widths, the distribution-preservation mechanism adds nothing beyond magnitude thresholding. A direct check is to compute the KL divergence between original and quantized weight tensors before and after pre-calibration and show that it decreases relative to round-to-nearest.","tokens_in":15254,"feed_emoji":"⚡","tokens_out":5571,"duration_ms":48557,"temperature":0.7,"pith_summary":"This paper proposes a pre-calibration step for post-training quantization of large language models that uses no calibration data at all. The step classifies weights into salient and common groups by solving an adaptive-LASSO problem with pseudo-activations, which reduces to a simple soft-thresholding rule. The authors argue this minimizes the Kullback-Leibler divergence between original and quantized weight distributions, preserving the model's information content. If the claim holds, quantized LLMs become robust to domain shifts in calibration data and far cheaper to produce.","feed_headline":"Data-free thresholding matches calibration-based LLM quantization","feed_subtitle":"A statistical pre-calibration step preserves weight distributions, matching calibration-based PTQ at a fraction of the cost.","key_machinery":"The load-bearing object is the adaptive LASSO penalty, a weighted L1 penalty $\\sum_i |\\hat w_i / w_i|$ in which the original weight serves as the gold-standard weight for each coefficient. Combined with pseudo-activations chosen so that $XX^\\top = bI$, the loss separates per weight and its minimizer is the soft-thresholding rule $\\hat w_i = \\operatorname{sign}(w_i)\\operatorname{ReLU}(|w_i| - \\lambda'/|w_i|)$. This rule is what classifies weights into salient and non-salient groups, and its closed-form nature is what makes pre-calibration fast.","core_discovery":"The paper's central claim is that minimizing the adaptive LASSO penalty $\\sum_i |\\hat w_i/w_i|$ under pseudo-activations is a proxy for minimizing $D_{KL}(f_W \\| f_{\\hat W})$, so that a data-free soft-thresholding classification of weights followed by separate quantization preserves the weight distribution. Empirically, this pre-calibration achieves perplexity and zero-shot accuracy on par with calibration-based methods such as SpQR, AWQ, and OPTQ on LLaMA, LLaMA 2, OPT, and Falcon models, while being roughly ten to a hundred times faster. The method does not shrink weights; it only uses the threshold to decide which weights are outliers, then quantizes both classes with minmax quantization.","pith_inferences":["A simpler magnitude-based or relative-magnitude thresholding may reproduce the reported accuracy, since the soft-thresholding step ultimately selects the largest $|\\hat w_i/w_i|$ ratios; the information-theoretic proof establishes an upper bound, not a guarantee of distribution preservation.","The pre-calibration step could be composed with activation-aware calibration methods as the improved initial point the paper proposes, potentially combining robustness with task-specific accuracy.","A direct test of the KL claim on real layer weights—computing $D_{KL}$ before and after pre-calibration versus round-to-nearest—would clarify whether distribution preservation or simply outlier protection drives the gains; the paper does not report that measurement."],"forward_implications":["LLMs can be quantized to 3-4 bits with no calibration data, avoiding the risk that calibration data mismatches the deployment task.","Quantization time drops by at least an order of magnitude compared with calibration-based methods such as AWQ and SpQR.","The pre-calibrated weight split can serve as a starting point for existing calibration methods, potentially improving their final accuracy.","Because no data is sampled, the quantization result is deterministic and insensitive to random seeds.","The framework is not fixed to adaptive LASSO: other f-divergences and penalties could guide the same pre-calibration classification step."],"supporting_citations":[{"why":"Introduces the adaptive LASSO penalty that the paper uses as the distribution-preservation proxy.","marker":"Zou, 2006"},{"why":"SpQR is the calibration-based baseline the pre-calibration results are compared against and the motivating counterexample on Code-Llama.","marker":"Dettmers et al., 2024b"},{"why":"AWQ is the calibration-based baseline whose quantization time and perplexity are compared in the experiments.","marker":"Lin et al., 2023"},{"why":"OPTQ supplies the layer-wise reconstruction formulation the paper contrasts with its classification approach.","marker":"Frantar et al., 2023"},{"why":"WikiText2 is the primary perplexity benchmark used for the quantized LLaMA models.","marker":"Merity et al., 2016"},{"why":"C4 provides the second perplexity benchmark in the evaluation tables.","marker":"Raffel et al., 2020"},{"why":"The LM Evaluation Harness is used to run the zero-shot commonsense reasoning tasks.","marker":"Gao et al., 2021"}],"fun_headline_variants":["Data-free pre-calibration matches calibration-based PTQ","Statistical pre-calibration: LLM quantization without data","Adaptive LASSO quantization rivals calibration methods","Faster PTQ: same accuracy, no calibration data","Pre-calibration step preserves weight distribution for LLMs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument depends on the claim that the adaptive LASSO penalty is a reliable proxy for distribution preservation, but the proof establishes an upper bound rather than an equivalence, so a small penalty does not guarantee a small KL divergence.","fun_headline_variants_meta":{"raw":{"variants":["Data-free pre-calibration matches calibration-based PTQ","Statistical pre-calibration: LLM quantization without data","Adaptive LASSO quantization rivals calibration methods","Faster PTQ: same accuracy, no calibration data","Pre-calibration step preserves weight distribution for LLMs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000227,"raw_usage":{"total_tokens":1451,"prompt_tokens":904,"completion_tokens":547,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":472}},"tokens_in":520,"tokens_out":547,"duration_ms":5957,"temperature":1.0,"reasoning_tokens":472,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:10:18.251159+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the perplexity and zero-shot accuracy of the proposed thresholding against a plain magnitude-based outlier selector that keeps the same fraction of largest absolute weights in high precision, holding the total bit budget fixed. If the two match across models and bit-widths, the distribution-preservation mechanism adds nothing beyond magnitude thresholding. A direct check is to compute the KL divergence between original and quantized weight tensors before and after pre-calibration and show that it decreases relative to round-to-nearest.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"OPTQ supplies the layer-wise reconstruction formulation the paper contrasts with its classification approach."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"C4 provides the second perplexity benchmark in the evaluation tables."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The LM Evaluation Harness is used to run the zero-shot commonsense reasoning tasks."}],"review_version":1}