{"id":"0d34d93c-2e15-42b6-98bd-dbaa1e63747d","arxiv_id":"2505.17595","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"NeUQI improves low-bit uniform quantization of LLMs by relaxing the integer zero-point constraint and efficiently searching a near-optimal scale, beating existing PTQ baselines at 2-4 bits.","lead":"NeUQI is a new method for setting the scale and zero-point when compressing large language models to very low bit widths (2-4 bits) with uniform quantization. It replaces the standard Min-Max formula with an efficient search that allows non-integer zero-points, and consistently produces quantized models with lower perplexity and higher accuracy than existing baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The unproved radius-1 localization of the exact zero-point optimum in §4.2.1 is the load-bearing gap: if Eq. 7's minimizer can lie more than 1 from the Eq. 8 minimizer zS, NeUQI's near-optimal claim for a fixed scale fails.","rationale":"The paper makes a clear, useful contribution: a fast scale-only optimization with an efficient zero-point search, and the experimental tables show large gains at 2-bit over the configured baselines. Reading §4.2.1 carefully, the load-bearing step is the reduction of the exact piecewise quadratic zero-point problem (Eq. 7) to a flattened surrogate (Eq. 8) whose minimizer zS is assumed to be within 1 of the true minimizer. This is exactly where near-optimality is earned, and it is stated as an assumption without proof or a systematic sensitivity analysis. Table 1 provides partial evidence: 'NeUQI' versus the exact no-acceleration baseline shows relative loss about 1.002 on LLaMA-2-7B 2-bit, which suggests the interval is adequate there. But that table covers one model family, one bit-width, and aggregates over blocks; it does not establish a bound for smaller scales in the grid, for 3-bit or 4-bit, or for Qwen distributions. The practical risk is concrete: for scales where the normalized weight range exceeds 2^k − 1, the plateau intervals in Eq. 8 do not all overlap, the surrogate minimizer is not pinned to an intersection of plateaus, and the phase of the exact loss's periodic valleys can dominate; nothing in the paper rules out |z* − zS| > 1 in that regime. If that happens, the exact search window misses the true minimizer, the zero-point is suboptimal, and the scale-only objective is evaluated at a worse point than the algorithm claims. The empirical comparison to baselines could still survive, but the central 'near-optimal initialization' claim would be overstated. Because the issue is a missing proof or validation rather than a known contradiction, the appropriate disposition remains conditional on the proposed check, matching the reader's verdict. Secondary concerns such as MagR stripped of coordinate descent, PV-tuning numbers from other papers, and the absence of released code are real but less central to the algorithm's core claim.","tokens_in":22958,"tokens_out":16182,"duration_ms":136547,"concrete_test":"Re-run the exact zero-point search (Algorithm 2, without the [zS−1, zS+1] restriction) for a random sample of weight rows from LLaMA-2-7B and Qwen-2.5-7B at 2 and 3 bits, using the same Hessian diagonals, across the full scale grid of Eq. 9. For each row compute z_exact = argmin of Eq. 7 over all real z and compare with NeUQI's restricted argmin, reporting the fraction of rows and scale candidates with |z_exact − z_restricted| > 1 and the resulting relative loss increase. If the fraction is non-negligible or the loss increase exceeds 0.1%, the near-optimality claim fails in those regimes.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's zero-point optimization replaces the exact piecewise quadratic loss (Eq. 7) with the flattened surrogate (Eq. 8), whose minimizer zS is used to restrict the exact search to [zS−1, zS+1] in Section 4.2.1. This localization is assumed, not proved. Eq. 8 erases the fine structure of the loss: within each plateau interval it charges a flat hi/4, discarding the 1-periodic valleys at z = j − x_i where the exact loss vanishes. The exact minimizer z* is determined by those valleys weighted by Hessian diagonals, and for scales where the normalized weight range exceeds 2^k − 1, the plateau intervals in Eq. 8 need not overlap, so zS is not constrained to a common flat region and the phase selected by the surrogate can in principle be several units away from the phase preferred by the exact loss. No bound of the form |z* − zS| ≤ 1 is derived. Table 1 reports aggregate relative loss for LLaMA-2-7B at 2 bits and provides partial support, but it cannot rule out larger violations for other scales, bit-widths, or model families. Since every scale evaluation uses this restricted zero-point, a violation would make the reported scale-only loss suboptimal and directly undermine the 'near-optimal' part of the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes NeUQI, a post-training uniform quantization parameter initialization method for large language models. It formulates the quantization loss under a diagonal Hessian approximation, derives a near-optimal zero-point for a fixed scale by exact minimization of a piecewise quadratic function via sorted transition points, and reduces the joint scale/zero-point optimization to a one-dimensional scale search with a coarse-to-fine grid. The authors argue that the conventional Min-Max initialization imposes two constraints (extreme-value-determined parameters and an integer zero-point) and show empirically that relaxing them improves perplexity and zero-shot accuracy across LLaMA and Qwen families at 2-4 bits. They also demonstrate gains when NeUQI is used as an initialization for distillation and quantization-aware fine-tuning.","tokens_in":23275,"tokens_out":6068,"duration_ms":66023,"significance":"If the claims hold, NeUQI is a valuable practical contribution: it is a fast, well-specified initialization scheme with O(n log n) per-scale zero-point computation, and the experiments show consistent and often large improvements over strong PTQ baselines at 2-3 bits, with additional benefits as a warm start for fine-tuning. The paper provides detailed pseudo-code for the optimization algorithms and a runtime analysis, and the experimental coverage (three model families, sizes 7B-72B, multiple bit-widths and group sizes) is extensive. The main caveat is that the theoretical 'near-optimal' guarantee is asserted rather than proven; the central claim is therefore empirical, and the empirical evaluation has some comparability issues.","major_comments":[{"comment":"The paper restricts the exact zero-point search to [zS−1, zS+1] based on the unproved assumption that the true minimizer of Eq. (7) lies within this interval. This is load-bearing: the scale-only loss of Section 4.2.2 is evaluated with this restricted zero-point, so any violation would make the reported losses suboptimal and directly undermine the 'near-optimal' part of the central claim. Table 1 reports only aggregate relative loss for LLaMA-2-7B at 2 bits, which is not sufficient to validate the bound across scales, bit-widths, or model families. Please provide a proof of the localization bound (or a counterexample analysis), or present a validation across the full grid of scales and bit-widths, or weaken the claim to 'approximately optimal under the localization assumption'.","section":"§4.2.1, Eq. (8) and Algorithm 4"},{"comment":"MagR is evaluated without the coordinate descent iterations used in the original MagR paper. Since MagR's published results are obtained with those iterations, this modification makes the comparison potentially misleading: the table may understate MagR's performance, and it is unclear whether 'fairness' is achieved by disabling a component of the baseline. Please report MagR with its standard configuration, or provide a clear justification for the modified setting and show that the conclusions are unchanged.","section":"§5.2, Tables 2 and 11"},{"comment":"Several baseline numbers, including LeanQuant† in Table 4, are taken from the original papers rather than computed in the authors' evaluation pipeline. Differences in hardware, calibration data, or evaluation setups can affect perplexity and accuracy, so the claim of 'consistently outperforms existing methods' is not supported on equal footing for all baselines. Please either run all baselines under the same setting or explicitly quantify the expected sensitivity to these differences.","section":"§5.1, Table 4"}],"minor_comments":[{"comment":"The formulation's relaxation of the integer zero-point constraint is central to the method, but the hardware support discussion is relegated to Appendix E; a brief pointer in the main text would help readers assess deployability.","section":"§3.1, Eq. (2)"},{"comment":"The MagR row for Qwen 2.5 7B contains entries like '23.1242.09' and '37.7660.9951.30' that are missing separators and are difficult to read; please format them consistently.","section":"Table 2"},{"comment":"The text refers to 'Min-Max+1', but the method is defined as 'Min-Max+' in Appendix A; please use a consistent name.","section":"§5.4"},{"comment":"The phrase 'sharing NeUQI’s loss function 5' should read 'loss function (5)' to avoid ambiguity with a numbered section.","section":"§4.2.1"},{"comment":"LeanQuant† has missing values (—) for LLaMA 3 8B on Wiki2 and C4; please indicate whether these are unavailable or infeasible.","section":"Table 4"},{"comment":"The search space is restricted to scales no larger than the Min-Max scale, but no justification is given for this upper bound; a sentence explaining why the optimal scale cannot exceed this value would be useful, especially for heavy-tailed weight distributions.","section":"§4.2.2, Eq. (9)"},{"comment":"The comment 'Since(⌈zS −1 +x i − 1/2⌉+ 2) +1/2 −x i ≥ zS + 1' is difficult to parse; consider reformatting it as a clearer derivation.","section":"Appendix C, Algorithm 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's claim to be 'the first to identify the constraints of the Min-Max formula' is strong given prior work on clipping-based and MSE-based initialization; the authors should temper this or survey more carefully. The paper is an empirical contribution with a light theoretical core; the unproved localization assumption in §4.2.1 is the key risk and should be addressed before publication. The paper is otherwise within the scope of the journal and the experiments are extensive."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does something real: for a fixed scale, it reduces the zero-point problem to a piecewise-quadratic optimization and solves it exactly in O(n log n) time, then uses a flat surrogate to speed things up and a coarse-to-fine scale search. That is a legitimate algorithmic contribution, and the floating-point zero-point relaxation is a sensible move that the experiments support. The empirical work is extensive—multiple model families, bit-widths, group sizes, tasks—and the gains over GPTQ, GPTAQ, and MagR are consistent and large at 2 bits. The runtime table shows the optimizations buy a lot. I believe the central claim that NeUQI is a better initialization than Min-Max and its variants.\n\nThe soft spots are real but not disqualifying. The load-bearing concern is the unproved assumption in Section 4.2.1 that the exact zero-point optimum lies within one unit of the surrogate optimum zS. The stress-test note is correct that the surrogate flattens the 1-periodic valleys of Eq. 7, so in principle the exact minimizer could be several units away. The paper just asserts the neighborhood. Table 1 gives partial empirical support on one model at 2 bits, but that does not cover other scales, bit-widths, or families. This needs to be fixed, either with a bound or a much broader validation.\n\nThe baseline fairness is a smaller issue. MagR is run without its coordinate descent iterations, which is a choice the authors mark with a dagger and justify as fair, but it does soften the comparison. Some numbers (LeanQuant, PV-tuning, EfficientQAT) are taken from prior papers without re-running in a unified setup. That is common practice but worth flagging. The 'first to identify constraints' claim is overstated; prior work already knows Min-Max is suboptimal, though the specific framing as two constraints is theirs. No code is released, which hurts reproducibility in a field where that is now expected.\n\nOverall, the paper deserves a serious referee. The method is clearly specified, the improvements are plausible and mostly reproducible from the description, and the open question about the neighborhood assumption is a well-defined technical gap rather than a sign of sloppy thinking. I would send it to review and ask the authors to prove or thoroughly test the localization assumption, report MagR with its full configuration, and release code.","headline":"Solid PTQ initialization paper with a genuine algorithmic contribution, weakened by an unproved localization assumption and some baseline fairness issues.","tokens_in":23776,"tokens_out":2340,"would_cite":true,"duration_ms":21074,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Deriving a near-optimal zero-point for each scale makes low-bit LLM quantization consistently better.","keywords":["post-training quantization","uniform quantization","low-bit large language models","quantization parameter initialization","zero-point optimization","scale search","Hessian diagonal approximation"],"falsifier":"Compute the exact global minimizer of the zero-point loss in Eq. 7 by a full transition-point sweep for all rows of a real 2-bit quantized LLaMA layer, and compare each to the returned $z_S$; if any exact minimizer lies more than 1 away, the restricted search has found a suboptimal zero-point and the near-optimality claim is falsified for realistic data.","tokens_in":22753,"feed_emoji":"📉","tokens_out":8988,"duration_ms":94341,"temperature":0.7,"pith_summary":"Quantizing a model to very few bits requires choosing two numbers for each weight block: a scale, the grid spacing, and a zero-point, the grid's offset. The paper argues that the usual Min-Max formula picks these numbers badly at 2-3 bits, and that this initialization, not the quantization procedure itself, is what limits low-bit LLMs. NeUQI derives a near-optimal zero-point for any given scale, reducing the joint two-variable search to a one-variable search over the scale, and solves that search efficiently. In experiments on LLaMA and Qwen families at 2-4 bits, this starting point consistently outperforms existing post-training quantization baselines, and it is strong enough that a light distillation step beats much heavier fine-tuning methods.","feed_headline":"Near-optimal quantizer start beats Min-Max on low-bit LLMs","feed_subtitle":"NeUQI derives the zero-point from the scale, turning a two-parameter quantization problem into one fast search.","key_machinery":"The load-bearing object is the piecewise-quadratic zero-point loss $L(z)=\\sum_i h_i (x_i + z - \\mathrm{clip}(\\lfloor x_i+z\\rceil,0,2^k-1))^2$, where $x_i$ is a weight divided by the scale and $h_i$ its Hessian diagonal weight. Because each term changes at transition points spaced exactly 1 apart, sweeping all transition points in sorted order and updating a running quadratic in $O(n\\log n)$ time yields the exact zero-point optimum for a fixed scale. A simplified two-transition-point surrogate produces $z_S$, and the coarse-to-fine scale search reduces the number of zero-point solves to $O(\\sqrt{T})$.","core_discovery":"NeUQI's central claim is that a much better uniform-quantization initialization is available almost for free once the zero-point is treated as a floating-point variable rather than an integer fixed by the Min-Max formula. For a fixed scale, the Hessian-weighted quantization loss becomes a piecewise quadratic function of the zero-point, and the paper gives an $O(n\\log n)$ sweep over its transition points that finds its exact global minimum; a two-transition-point approximation gives a fast starting point $z_S$, after which the exact search restricted to $[z_S-1, z_S+1]$ is claimed to be near-optimal. With that zero-point solver, only the scale remains, and a coarse-to-fine grid search over it costs $O(\\sqrt{T})$ zero-point evaluations. The paper reports that this recipe beats GPTQ, GPTAQ, MagR, and search-based LeanQuant across LLaMA 2/3 and Qwen 2.5 at 2-4 bits, and that replacing only the initialization inside the fine-tuning method EfficientQAT improves it further.","pith_inferences":["Inference: The same transition-point sweep works for any per-sample convex quadratic of the quantizer parameter, so a similar near-optimal initialization could be derived for clipping thresholds, per-group scales, or activation quantizers.","Inference: The $[z_S-1,z_S+1]$ window is a one-unit guarantee that is asserted rather than proved; if weighted outlier distributions ever push the exact optimum further away, widening the window adaptively would be a cheap safety valve.","Inference: The results suggest the bottleneck at very low bit-widths is not optimization capacity but the quality of the starting parameters, which would shift future work toward better priors and initialization-first recipes.","Inference: Making floating-point zero-points a first-class citizen in inference kernels would remove the main deployment obstacle; if that happens, the integer-constrained formulations used by most libraries may become an unnecessary handicap."],"forward_implications":["At 2-bit channel-wise quantization, NeUQI lowers perplexity and raises average zero-shot accuracy on every LLaMA and Qwen model tested, with the largest gains on 70B-scale models.","Relaxing the integer zero-point constraint costs less than 0.01 average bit-width in the 2-bit channel-wise setting, and NeUQI still beats integer-constrained methods running at a slightly higher average bit-width.","NeUQI composes with Hadamard-transform-based quantization pipelines, improving W2A16, W2A4, and W4A4 results over the same pipelines with other initializations.","With about 0.5M tokens of lightweight distillation on LLaMA 2 7B at 2 bits, NeUQI surpasses PV-tuning, which uses roughly 1B tokens on a per-token basis, and also improves the strong fine-tuning method EfficientQAT when only its initialization is replaced.","Swapping only the initialization inside EfficientQAT improves its 2-bit results on LLaMA 2 7B and LLaMA 3 8B, and freezing the NeUQI-provided scale in the first stage helps further."],"supporting_citations":[{"why":"Defines the Min-Max quantization initialization and the integer zero-point convention that NeUQI identifies as the limiting constraints.","marker":"Jacob et al., 2017"},{"why":"Supplies the standard uniform/asymmetric affine quantization formulation that the paper extends by allowing floating-point zero-points.","marker":"Krishnamoorthi, 2018"},{"why":"Provides the layer-wise quantization loss and the GPTQ baseline that NeUQI's calibration, loss, and comparisons build on.","marker":"Frantar et al., 2023"},{"why":"Originates the diagonal Hessian approximation that turns the layer loss into the piecewise-quadratic zero-point objective.","marker":"LeCun et al., 1989"},{"why":"Also supports the use of the diagonal Hessian approximation in the low-bit post-training quantization setting.","marker":"Liu et al., 2024"},{"why":"LeanQuant is the search-based baseline whose Min-Max grid search NeUQI's design is explicitly contrasted with in efficiency and flexibility.","marker":"Zhang & Shrivastava, 2025"},{"why":"GPTAQ is a uniform post-training quantization baseline that NeUQI outperforms in the 2-bit comparisons.","marker":"Li et al., 2025"},{"why":"MagR is the weight-magnitude-reduction baseline that NeUQI's 2-bit results are compared against.","marker":"Zhang et al., 2024"},{"why":"PV-tuning is the resource-intensive fine-tuning baseline that NeUQI plus light distillation is shown to surpass.","marker":"Malinovskii et al., 2024"},{"why":"EfficientQAT is the strong fine-tuning method whose results improve when only its quantization initialization is replaced by NeUQI.","marker":"Chen et al., 2025"}],"fun_headline_variants":["NeUQI: near-optimal starts for low-bit LLM quantization","Better LLM quantization start: NeUQI beats Min-Max","Near-optimal zero-point search cuts LLM quantization loss","One sweep finds near-optimal quantization for 2-4 bit LLMs","Zero-point trick makes low-bit LLM quantization near-optimal"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that, for realistic weight vectors and Hessian weights, the exact best zero-point always falls within one unit of the fast approximate zero-point $z_S$, so a search restricted to $[z_S-1,z_S+1]$ cannot miss it.","fun_headline_variants_meta":{"raw":{"variants":["NeUQI: near-optimal starts for low-bit LLM quantization","Better LLM quantization start: NeUQI beats Min-Max","Near-optimal zero-point search cuts LLM quantization loss","One sweep finds near-optimal quantization for 2-4 bit LLMs","Zero-point trick makes low-bit LLM quantization near-optimal"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000336,"raw_usage":{"total_tokens":1901,"prompt_tokens":1029,"completion_tokens":872,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":645,"completion_tokens_details":{"reasoning_tokens":781}},"tokens_in":645,"tokens_out":872,"duration_ms":8818,"temperature":1.0,"reasoning_tokens":781,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:44:15.466578+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the exact global minimizer of the zero-point loss in Eq. 7 by a full transition-point sweep for all rows of a real 2-bit quantized LLaMA layer, and compare each to the returned $z_S$; if any exact minimizer lies more than 1 away, the restricted search has found a suboptimal zero-point and the near-optimality claim is falsified for realistic data.","supporting_citations":[{"cited_title":"S., and Solla, S","cited_arxiv_id":null,"evidence_quote":"Originates the diagonal Hessian approximation that turns the layer loss into the piecewise-quadratic zero-point objective."}],"review_version":1}