{"id":"756f0dfb-e62c-44ba-9b41-6333483b0c2a","arxiv_id":"2608.07019","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"ReQuant refines already-quantized language models on a fixed integer grid via backpropagation-free coordinate descent, consistently lowering perplexity, KL divergence, and zero-shot accuracy gaps across models, bit-widths, and PTQ initializers.","lead":"ReQuant is a post-processing step for quantized language models that keeps adjusting the discrete integer weights after an existing quantization method has finished, improving how closely the compressed model reproduces the original. It can be added to current quantization pipelines as a plug-and-play refinement and gives the largest gains for simple methods and very low-bit settings.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ReQuant scores moves with activation statistics frozen at the initial quantized prefix; the paper never specifies a layer order that keeps eH/B valid, so the monotone-decrease guarantee applies only to a stale surrogate, not the deployed model.","rationale":"The reader's weakest assumption identifies the same root: activation statistics are captured once and never refreshed. I agree that this is the most load-bearing issue because the mathematical guarantees (Section 3.3, Appendix A.2) are all relative to the fixed-statistics surrogate L(e) defined in Eq. 5, whereas the empirical claims concern perplexity, KL, and zero-shot accuracy of the actual quantized network. The missing piece is the layer traversal order. If ReQuant processes layers in reverse topological order, each layer's eX remains exact because no upstream layer has changed when that layer is refined; if it processes in forward or arbitrary order, eX is stale for every layer after the first, and the accepted moves can be suboptimal or harmful for the deployed model. The paper does not state the order, making the method under-specified. The no-QuaRot GPTQ regression in Table 13 is a concrete instance of the surrogate diverging: WikiText PPL/KL improve while average zero-shot accuracy drops, a trade-off shift that should not happen if refinement reliably reduced true reconstruction error. A controlled layer-order and refresh experiment would settle whether this is noise or a structural failure. This does not change the reader's CONDITIONAL verdict: the method is promising and largely supported, but the unspecified refresh/layer-order detail and the qualification of 'consistently improves' should be addressed before acceptance as stated.","tokens_in":24514,"tokens_out":10395,"duration_ms":104688,"concrete_test":"Run ReQuant on Llama-3 8B, GPTQ initializer, W4A16 without QuaRot (the setting of Table 13) under three layer-processing regimes: (a) forward order with no refresh, (b) reverse topological order with no refresh, (c) forward order with eH/B refreshed by a calibration forward pass after each layer. If (b) and (c) improve the ten-task average accuracy while (a) reproduces the Table 13 drop, then the unstated layer order is responsible and the paper must specify it or adopt a refresh scheme. Also compare, per layer, the surrogate's predicted loss decrease to the true reconstruction error computed with freshly collected eX after each sweep; if the surrogate predicts a decrease while the true error increases, the fixed-statistics assumption is decisively falsified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"ReQuant precomputes eH = eX eX^T and B = ΔX eX^T once from the initial quantized prefix and never refreshes them (Eq. 6, Algorithm 1). For a single row's sweeps this is exact: changing that layer's weights does not change its input activations eX. However, once any upstream layer is refined, the inputs to all downstream layers change, so a downstream layer's precomputed eH/B no longer describe the activations that layer will actually see at deployment. The finite-termination and monotone-decrease proof in Appendix A.2 explicitly assumes fixed calibration statistics; it says nothing about the true reconstruction error ||W X - W_q eX_new||^2 with refreshed activations. No layer traversal order is stated in Section 3, so if layers are refined front-to-back (or in parallel), accepted moves can increase the true deployment error. The paper's own no-QuaRot W4A16 table (Table 13) shows GPTQ+ReQuant improving WikiText PPL/KL but decreasing ten-task average accuracy from 64.63 to 64.41, consistent with the surrogate diverging from the deployment objective. This is the load-bearing gap because the central claim of consistent improvement requires the fixed-statistics surrogate to remain aligned with the objective actually evaluated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ReQuant, a post-training quantization refinement stage that treats an existing quantized model as a feasible starting point and optimizes its integer codes on the fixed quantization grid. The layer-wise objective is the GPTAQ-style activation-aware reconstruction error ||WX - W_q \\tilde X||^2, where \\tilde X denotes activations collected once from the initial quantized prefix (Eq. 3). Candidate coordinate moves are scored in closed form via Eq. 7, and updates that strictly decrease the row loss are accepted (Algorithm 1). The paper proves finite termination under fixed calibration statistics (Appendix A.2), analyzes per-layer complexity (Appendix A.3), and reports experiments across Llama-3 8B/70B, Qwen3-14B, and Qwen3-235B, several bit widths, and RTN/AWQ/GPTQ/GPTAQ initializers, with perplexity, KL, and ten-task zero-shot accuracy as metrics.","tokens_in":24774,"tokens_out":9999,"duration_ms":100716,"significance":"If the claims hold, ReQuant provides a format-preserving, initializer-agnostic post-processing stage that can be composed with existing PTQ pipelines, and the results suggest that a substantial share of the performance gap between simple and advanced PTQ initializers is correctable fixed-grid assignment error. The derivations and the finite-termination proof in Appendix A are clear and, as far as I checked, correct; the experimental coverage is unusually broad, including a 235B MoE model. The paper is also honest about several small reversals and about seed variability in Appendix B.9. The main weakness is a mismatch between the theoretical guarantee, which is stated for a frozen-statistics surrogate, and the empirical claim about the deployed model; this gap, together with the absence of repeated-run statistics for the main tables, needs to be addressed before the 'consistently improves' claim is fully supported.","major_comments":[{"comment":"The layer traversal order is unspecified, and the activation statistics \\tilde H = \\tilde X \\tilde X^T and B are collected once from the initial quantized prefix and never refreshed in Algorithm 1. If layers are refined front-to-back or in parallel, the input activations \\tilde X of downstream layers change after upstream layers are refined, so the monotone-decrease guarantee of Appendix A.2 applies only to the stale surrogate L(W_q) = ||WX - W_q \\tilde X_initial||^2, not to the actual reconstruction error of the deployed model ||WX - W_q \\tilde X_new||^2. The no-QuaRot W4A16 GPTQ result in Table 13 (average accuracy 64.63 to 64.41 while WikiText PPL/KL improve) is consistent with this mismatch. The authors should specify a traversal order that keeps \\tilde X valid during refinement (e.g., reverse layer order), or refresh the statistics between sweeps, and should state explicitly whether the theoretical guarantee concerns the surrogate or the deployed objective; if it concerns only the surrogate, the abstract's claim that accepted updates 'strictly reduce the mean squared reconstruction error' needs qualification.","section":"§3.2, §3.3, Appendix A.2"},{"comment":"The main tables (Tables 1, 2, 3, 4, 5, 6, and the appendix tables) report single runs without error bars. The only repeated-run estimate in the paper is the QuaRot seed-stability study in Appendix B.9, which reports average accuracy 65.27 ± 0.55 for one configuration. Several headline improvements on strong initializers are smaller than this spread: for example, Table 1 reports GPTQ+ReQuant W4A16 on Llama-3 8B as +0.33 and GPTAQ+ReQuant W4A16 on Qwen3-14B as +0.04, and Table 3 reports GPTAQ+ReQuant W3A4 as +0.06. To support the central claim of consistent improvement across all four initializers, the authors should report repeated runs or seed-averaged results with confidence intervals for at least the main comparisons, or restrict the claim to settings where the gains exceed the measured variability.","section":"§4, Appendix B.9"},{"comment":"The paper states that WikiText-2 is used as the calibration dataset and that perplexity is also reported on WikiText-2, but it does not state whether the calibration and evaluation splits are disjoint. If the same sequences are used for both, the WikiText-2 PPL/KL improvements are in-domain and may be optimistic. Please specify the exact split (e.g., train/validation/test) and, ideally, calibrate on a different corpus or report held-out perplexity so that the reported WikiText-2 numbers are not confounded with calibration-set overlap.","section":"§4.1"}],"minor_comments":[{"comment":"The text claims that ReQuant can be composed with AdaRound/BRECQ/FlexRound-style construction-time optimization, but the experiments only compare with FlexRound; consider adding a sentence noting that the AdaRound/BRECQ composition is not experimentally evaluated, or softening the claim.","section":"§4.5"},{"comment":"The main text refers to 'Top-20 KL' for the Qwen3-235B experiment, but the table header simply says 'KL'; please define Top-20 KL and make the table header consistent with the text.","section":"§4.6, Table 6"},{"comment":"The note 'GPTQ rows reuse Table 4' is confusing because Table 4 reports RTN and GPTQ under different headings; please make the reuse explicit by repeating the rows or by clearly labeling the source table.","section":"Appendix B.8, Table 14"},{"comment":"The sentence 'if the remaining network is locally Lipschitz, smaller layer residuals tighten a bound on later hidden-state perturbations' is used to connect the layer-wise objective to downstream metrics; a formal statement or a reference for this bound would help the reader assess how tight that connection is expected to be.","section":"§3.3"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"ReQuant is worth a serious look. The idea—treat a finished PTQ model as a feasible starting point and keep optimizing its integer codes on the fixed grid—is genuinely distinct from AdaRound/BRECQ/FlexRound, which do their rounding during construction. The paper's math is correct: Eq. 7 gives the exact loss change for a coordinate move, the gradient update is a clean rank-one refresh, and the finite-termination proof in Appendix A.2 is sound. The experiments are broad: Llama-3 8B/70B, Qwen3-14B, even a 235B MoE, across W4A16/W4A4/W3A4/W2A4 and four initializers. The headline result—RTN refined by ReQuant approaching or beating GPTAQ—is believable and practically useful. The limitations section is honest about calibration-set dependence and the empirical nature of held-out metrics.\n\nSoft spots, in order of importance. First, the activation statistics issue. eH and B are computed once from the initial quantized prefix and never refreshed. That is exact only if you refine a single layer in isolation. Once an upstream layer changes, downstream layers see different activations, so the objective being optimized is a stale surrogate. The monotone-decrease guarantee is for that surrogate, not for the deployed model. The paper never states the layer traversal order or a refresh strategy, so a front-to-back sweep could in principle accept moves that increase true reconstruction error. This is not a fatal flaw—the empirical gains are consistent enough that the method clearly helps—but the gap between the guarantee and the claim should be closed in revision.\n\nSecond, 'consistently improves' is too strong. Table 13 shows GPTQ+ReQuant dropping ten-task average accuracy from 64.63 to 64.41, and the appendix has a few KL/PPL cells where the baseline wins. The paper acknowledges some of this in B.5 but the abstract and Section 4 still say 'consistently.' Third, there are no error bars and no code. Given that QuaRot seed variability alone moves average accuracy by ±0.55, some of the small deltas in Tables 1–3 could be noise. Releasing code and adding seeds/error bars would firm this up.\n\nWho it's for: people doing practical PTQ deployment, especially at lower bit-widths. It deserves a serious referee; the idea is clean, the analysis is correct, and the weaknesses are addressable. Send it out.","headline":"ReQuant is a clean, well-executed fixed-grid refinement stage for PTQ with correct math and broad experiments; the 'consistent improvement' claim is slightly too strong and the stale-activation-statistics assumption deserves an explicit fix, but it merits serious peer review.","tokens_in":25311,"tokens_out":2710,"would_cite":true,"duration_ms":27432,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Post-training quantization outputs are not final: a fixed-grid discrete refinement pass can still improve them.","keywords":["post-training quantization","large language models","discrete coordinate descent","fixed-grid refinement","reconstruction error","integer weight codes","low-bit quantization","model compression"],"falsifier":"Run ReQuant with the same starting model and calibration data but recompute $\\mathbf{H}$ and $\\mathbf{B}$ from the actual quantized activations after each sweep instead of reusing the initial statistics. If refreshing the statistics yields lower perplexity or a different set of accepted moves than the paper's one-pass version, the one-pass surrogate is not tracking the true deployment objective.","tokens_in":24307,"feed_emoji":"⚙️","tokens_out":4858,"duration_ms":49789,"temperature":0.7,"pith_summary":"The paper proposes ReQuant, a post-training quantization post-processing stage that keeps improving a quantized model after an initial PTQ method has produced it. The central claim is that the initial integer assignments are not final: ReQuant revisits each weight code on the fixed quantization grid, accepts only moves that strictly reduce the layer-wise mean squared reconstruction error, and preserves bit-width, scales, zero-points, and inference kernels. If true, any existing PTQ pipeline can be improved by plugging in this refinement, with the largest gains for simple initializers and low bit-widths. The paper reports that refining a simple round-to-nearest initialization across sweeps approaches or surpasses the stronger GPTAQ initializer under the same format.","feed_headline":"One refinement stage lifts every LLM quantization baseline","feed_subtitle":"ReQuant swaps integer codes on the same grid, matching or beating expensive quantizers after a few offline sweeps.","key_machinery":"The load-bearing object is the exact discrete coordinate score $\\Delta L(\\Delta q_j) = -\\Delta q_j\\, g_j + (\\Delta q_j)^2 \\, H_{jj}$, derived from the row-wise quadratic reconstruction loss. Here $\\mathbf{H} = \\widetilde{X}\\widetilde{X}^\\top$ is the row-level Hessian of the quantized-branch activations and $\\mathbf{B} = \\Delta X \\widetilde{X}^\\top$ captures the activation mismatch between full-precision and quantized prefixes; both are precomputed once. This score lets ReQuant evaluate every $K$-neighborhood grid move in constant time, update the gradient incrementally after each accepted move, and iterate sweeps until the local objective is exhausted.","core_discovery":"ReQuant establishes that completed PTQ outputs are feasible starting points rather than terminal solutions. Treating the layer-wise objective $L(W^q) = \\|WX - W^q \\widetilde{X}\\|_F^2$, it performs discrete coordinate descent over integer codes on the inherited grid; since each output row depends only on the corresponding weight row, the loss decomposes row-wise, and each candidate move is scored in O(1) using a cached Hessian and gradient. A move is accepted exactly when it lowers the row reconstruction loss, so every accepted update strictly decreases the full layer objective, and finite termination follows from finiteness of the grid. The experiments show consistent improvements in perplexity, KL divergence, and ten-task average accuracy across RTN, AWQ, GPTQ, and GPTAQ initializers on Llama-3 and Qwen3 models, with RTN+ReQuant approaching or beating GPTAQ under W4A4.","pith_inferences":["Because $\\mathbf{H}$ and $\\mathbf{B}$ are collected once from the initial quantized prefix, the monotone guarantee covers the surrogate objective, not necessarily the true deployment objective; a testable extension is to refresh these statistics every sweep and compare the resulting assignments.","The paper's own no-QuaRot GPTQ result, where perplexity improves while average downstream accuracy edges down, shows that the layer-wise objective is a proxy; practitioners should validate refinement on the specific downstream metric they care about.","The same row-wise discrete coordinate machinery transfers to any layer with linear structure, such as convolutional layers or sparse expert projections in mixture-of-experts models."],"forward_implications":["Any existing PTQ pipeline can be extended by a refinement stage that touches only integer codes, so the deployed bit-width, scales, zero-points, and kernels stay exactly as they were.","A simple round-to-nearest start, after enough sweeps, approaches or exceeds a strong activation-aware initializer under the same format, meaning much of the PTQ performance gap is correctable discrete assignment error.","The improvement is largest where quantization is hardest: lower bit-widths and weaker initializers, with gains still present on GPTQ and GPTAQ.","Offline refinement cost is controlled by the sweep count $T$, so practitioners can trade offline time for accuracy and pay nothing at serving time."],"supporting_citations":[{"why":"Provides the greedy column-wise PTQ method whose fixed assignments ReQuant revisits.","marker":"[24]"},{"why":"Supplies the activation-aware objective in Eq. 2 and serves as the strongest baseline that ReQuant refines.","marker":"[25]"},{"why":"Provides the activation-aware scaling initializer used as a baseline.","marker":"[13]"},{"why":"Provides the rotation preprocessing used in most reported experiments.","marker":"[27]"},{"why":"Supplies the simple round-to-nearest baseline that shows the largest gains after refinement.","marker":"[30]"},{"why":"Serves as the construction-time adaptive rounding baseline in the pipeline-level comparison.","marker":"[29]"}],"fun_headline_variants":["ReQuant refines any quantized LLM to match costly methods","Post-training quantization gets a grid-based refinement stage","No-backprop discrete refinement lifts all PTQ baselines","Swap integer codes to improve every LLM quantizer","Fixed-grid moves make PTQ outputs iteratively improvable"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The refinement scores are computed from activation statistics collected once from the initial quantized model, and the paper assumes those statistics stay accurate as the integer codes change; if accepted updates shift downstream activations, the optimized solution may be optimal for outdated statistics rather than for the actual deployment.","fun_headline_variants_meta":{"raw":{"variants":["ReQuant refines any quantized LLM to match costly methods","Post-training quantization gets a grid-based refinement stage","No-backprop discrete refinement lifts all PTQ baselines","Swap integer codes to improve every LLM quantizer","Fixed-grid moves make PTQ outputs iteratively improvable"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000349,"raw_usage":{"total_tokens":1932,"prompt_tokens":994,"completion_tokens":938,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":858}},"tokens_in":610,"tokens_out":938,"duration_ms":10549,"temperature":1.0,"reasoning_tokens":858,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T16:31:06.989609+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run ReQuant with the same starting model and calibration data but recompute $\\mathbf{H}$ and $\\mathbf{B}$ from the actual quantized activations after each sweep instead of reusing the initial statistics. If refreshing the statistics yields lower perplexity or a different set of accepted moves than the paper's one-pass version, the one-pass surrogate is not tracking the true deployment objective.","supporting_citations":[{"cited_title":"AWQ: Activation-aware weight quantization for on-device llm compression and acceleration.Proceedings of Machine Learning and Systems (MLSys), 6:87–100, 2024","cited_arxiv_id":null,"evidence_quote":"Provides the activation-aware scaling initializer used as a baseline."},{"cited_title":"Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman","cited_arxiv_id":null,"evidence_quote":"Provides the rotation preprocessing used in most reported experiments."},{"cited_title":"Post-training 4-bit quantization of convolutional networks for rapid deployment","cited_arxiv_id":null,"evidence_quote":"Supplies the simple round-to-nearest baseline that shows the largest gains after refinement."},{"cited_title":"FlexRound: Learnable rounding based on element-wise division for post-training quantization","cited_arxiv_id":null,"evidence_quote":"Serves as the construction-time adaptive rounding baseline in the pipeline-level comparison."}],"review_version":1}