{"id":"7ec6d96d-7db7-4834-a126-bb2a17a7b838","arxiv_id":"2608.11045","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A calibration-free rounding scheme uses a diffusion model trained on a model's own weights to resolve ambiguous midpoint rounding, improving 3- and 4-bit weight-only quantization on small LLMs.","lead":"ReRound trains a diffusion model on a small language model's own weight matrices, then uses the model's reconstructions to decide how to round weights that sit near the midpoint between two quantized levels. It is calibration-free, runs offline, and improves 3-bit and 4-bit weight-only quantization accuracy over round-to-nearest on the eight small models tested.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'consistent improvement' claim may depend on per-model hyperparameters: Table 8 sets different candidate sets and beta for each model with no disclosed calibration-free selection rule, and the only automatic weight-only selector (spectral, Eq. 17-18) is validated on a single model.","rationale":"The reader picked the spectral proxy as the weakest assumption. I agree that proxy is unproven, but I think the sharper and more actionable weak point is Table 8: the per-model T and beta have no disclosed selection rule. Because ReRound is advertised as calibration-free, the only legitimate ways to set these are a fixed default or a weight-only criterion; the paper provides neither, so every reported gain is conditional on an unspecified tuning procedure. The proposed fixed-configuration rerun is a direct falsification test: if ReRound with one global setting still beats RTN on all 16 model-bit pairs, the central claim is robust and the Table 8 variation is likely harmless; if not, the 'consistent improvement' headline is unsupported. This is consistent with a conditional verdict: the algorithm is well specified, the supporting math is internally consistent, and the method is reproducible, but the empirical claim needs this robustness check before acceptance. No change to the reader's verdict is needed; the conditional recommendation already reflects the required additional evidence.","tokens_in":23935,"tokens_out":14996,"duration_ms":149810,"concrete_test":"Run ReRound on all eight checkpoints at W3 and W4 with a single fixed configuration, e.g., the Gemma 2 2B entry in Table 8: T_linear={0,0.15,0.25}, T_head={0,0.15,0.25}, beta=8, delta=0.1, revision cap=1%, with no per-model changes. Compare four-task average accuracy against matched group-wise RTN for all 16 model-bit pairs. If any model-bit pair fails to beat RTN, the consistent-improvement claim depends on per-model tuning. Report the diffusion training seed and per-task standard errors so small gains can be distinguished from evaluation noise.","verdict_should_be":"UNCHANGED","load_bearing_attack":"ReRound's advertised advantage is improving RTN without calibration data. But Appendix D (Table 8) gives different T_linear, T_head, and beta for every model, and the paper never describes how these are chosen. If they were selected after seeing the four-task accuracies in Tables 1-2 and 10, then the 'no calibration data' claim is violated at the hyperparameter level. Even if selected from earlier development runs, a practitioner applying ReRound to a new model has no stated way to set T and beta except a validation set, which would be calibration data. The only automatic, weight-only mechanism is spectral selection (Eq. 17-18), whose evidence for tracking task accuracy is one ablation on OLMo 2 1B (Table 4). The reported gains are 0.1-1.6 points, and beta values from 8 to 128 change the tolerance sharpening substantially (Figure 6). These per-model choices could determine whether an average gain is positive. The central claim that ReRound 'consistently improves' RTN would survive a fixed configuration; it may not.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ReRound, a calibration-free post-training quantization method for LLMs that revises round-to-nearest (RTN) assignments near quantization-interval midpoints. ReRound trains a per-model conditional diffusion model (a ControlNet attached to a frozen, image-pretrained U-Net, conditioned on 2-bit stochastic-rounding patches) to reconstruct continuous weights from low-bit observations. For each weight matrix, reconstructed entries propose flipping the RTN floor-or-ceiling decision; a position-dependent tolerance metric tau_pos(r;tau) (Eq. 13) admits only proposals near the midpoint, and sweeping tau generates candidate quantized integer matrices. The candidate whose dequantized matrix most closely matches the full-precision matrix's leading singular values under D_spec (Eqs. 17-18) is selected, with tau=0 (RTN) always included in the candidate set. Experiments on eight small LLMs at 3 and 4 bits report positive four-task-average gains over matched group-wise RTN in every model-bit pair (0.1-1.6 points), best-in-table results at 3 bits on all five main models, competitive results against GPTQ/AdaRound/SignRound at 4 bits, and improvements when applied on top of SINQ quantization parameters. Appendix J provides derivations of the conditional-denoising relation, bounds on the scalar error of accepted revisions, and a spectral-selection guarantee relative to RTN.","tokens_in":24157,"tokens_out":17416,"duration_ms":142193,"significance":"If the claims hold, ReRound makes a concrete contribution: it shows that a weight-computed diffusion prior can disambiguate midpoint rounding without any activation or text calibration data, while preserving the low-bit representation and inference procedure. The paper is strong on reproducibility: Algorithms 1-2 fully specify the procedure, Tables 7, 8, 9, and 12 give software versions and per-model settings, code is released, and offline runtime is reported (Table 6). Section J's derivations are correct and useful, including Tweedie's relation for the conditional denoiser (Eqs. 29-38), the bound that every accepted revision lies within tau of the midpoint (Eq. 48), the scalar flip-cost bound (Eq. 51), and the guarantee D_spec(tau*) <= D_spec(0) (Eq. 56). The main risk is empirical: the headline 'consistent improvement' rests on single-run four-task averages with no variance estimates, some with margins as small as 0.1-0.2 points, and the method's per-model hyperparameters (Table 8) lack a disclosed calibration-free selection rule.","major_comments":[{"comment":"The manuscript states that ReRound 'uses the same formulation across models' and claims 'consistent improvement' over RTN, but Table 8 assigns different tolerance candidate sets (T_linear, T_head) and different beta values to every model (e.g., beta=8 for Gemma 2 2B versus beta=128 for Qwen3 1.7B), and no weight-only selection rule for these hyperparameters is described anywhere. Because tau=0 makes RTN an always-available candidate, these per-model settings are not outputs of the spectral selection; they are inputs chosen outside the method. If T and beta were set using the task accuracies in Tables 1-2 and 10, then the 'without activation or text calibration data' claim is violated at the hyperparameter level, and the claim 'ReRound consistently improves RTN' would be better stated as 'for each model there exist settings under which ReRound improves RTN.' The paper should either (a) disclose a fixed, calibration-free protocol for choosing T and beta, (b) supply a fixed-configuration experiment (one T and one beta for all eight models) showing the gains survive, or (c) temper the abstract and conclusion claims accordingly.","section":"§4.1; Appendix D (Table 8)"},{"comment":"All task accuracies are single-run with no variance estimates, and several headline gains are small: the W3 gain is 0.1 points on Qwen3 1.7B and 0.2 points on Gemma 3 1B, and the W4 gain is 0.2 points on SmolLM2 1.7B. Individual tasks within those averages move in the opposite direction; for example, Table 10 shows Llama 3.2 1B W3 WinoGrande dropping from 53.6 (RTN) to 51.4 (ReRound), and Table 1 shows Gemma 3 1B W4 BoolQ dropping from 62.6 to 61.9. With four-task averages and no seed variance, a 0.1-0.4 point difference is within the typical noise of these benchmarks. The sign-consistency across eight models and two bit widths is suggestive, but the quantitative claim 'consistently improves' needs either multiple seeds (or a variance estimate) on at least a subset of models, or a more modest framing of the claim.","section":"Tables 1-3, 10"},{"comment":"The selection rule that distinguishes ReRound from RTN is the spectral proxy D_spec, but its empirical validation is a single ablation on OLMo 2 1B. Table 4 shows that the proxy outperforms weight-MSE selection and matched random flips on that model, which is encouraging, yet the paper's own Section 5 concedes that 'spectral selection operates only in weight space and may not identify the best candidate for every downstream task,' and Appendix J.4 states that the derived scalar bound 'does not characterize changes in model outputs or downstream task accuracy.' Since tau=0 is always a candidate, Eq. (56) protects against spectral regression but not against task-accuracy regression. A direct check without calibration data—for example, for several models, reporting the rank correlation between D_spec(tau) and the four-task average across the candidate set, or the fraction of matrices where the selected tau* differs from 0—would substantiate the proxy.","section":"Eqs. (17)-(18); Table 4; Appendix J.4"},{"comment":"The claim that ReRound 'learns a diffusion prior from the pretrained LLM's own weights' is incomplete as stated: the U-Net backbone is frozen and initialized from the image-pretrained IF-II-M-v1.0 checkpoint, and only the ControlNet is trained on LLM weight patches (Appendix C). This means an external, image-domain generative prior is part of the method, and the main text should state this in Section 3.2 rather than only in the appendix. A control experiment (for example, a randomly initialized backbone with the same ControlNet training, or a from-scratch diffusion model) would clarify how much of the reconstruction signal comes from the learned weight statistics versus the frozen generic backbone.","section":"§3.2; Appendix C; Figure 1"}],"minor_comments":[{"comment":"The abstract's claim of 'superior accuracy compared to an extensive set of calibration-free methods' overstates Tables 1-2: at W4 on OLMo 2 1B, both HQQ (60.4) and BNB FP4 (61.1) exceed ReRound (59.4). I suggest rewording to 'best or near-best average accuracy on most models.'","section":"Abstract; Tables 1-2"},{"comment":"CafeQ is listed among the calibration-free baselines in Section 4.1, but Tables 1 and 2 contain no CafeQ rows; CafeQ appears only in Table 3 for Gemma 2 2B at W4. Please state explicitly the coverage of the CafeQ comparison.","section":"§4.1; Tables 1-3"},{"comment":"The 'Midpoint window' ablation variant is under-specified: it is not stated whether the acceptance threshold inside [0.5-delta, 0.5+delta] is still tau and what happens outside the window. There is also a typo in the text, '[0.5-delta, ,0.5+delta]', with a doubled comma.","section":"§4.4; Table 4"},{"comment":"The main text's sentence 'Using l, u, and r from Eq. (19)' refers to an appendix equation, while the same quantities are defined in Eq. (1) in the main text; cite Eq. (1) instead. The duplicate numbering between the main text and Appendix J (Eqs. (17)-(18) vs (54)-(55), and Eqs. (13) vs (44)) is confusing and should be unified.","section":"§3.3; Appendix A; Appendix J"},{"comment":"The differing treatment of the output logit layer deserves an explicit sentence: for several models T_head={0}, meaning no revisions are considered for lm_head; please explain whether this was decided per model and why.","section":"Appendix D; Table 8"},{"comment":"The abstract's statement that 'the method applies to AI models beyond LLMs' is conjectural, as the experiments cover only small LLMs and Section 5 appropriately limits the claim; either add a small non-LLM proof-of-concept or move this sentence to future work.","section":"§5; Abstract"},{"comment":"Table 9 uses per-bit-width candidate sets for the SINQ experiments (W4 {0,0.15} vs W3 {0,0.15,0.25,0.35}), while Table 8's caption says the same settings are used at 3 and 4 bits; please disambiguate the scope of that statement in the caption.","section":"Appendix D; Table 9"},{"comment":"The caption should state explicitly that diffusion training uses two GPUs while diffusion inference and ReRound PTQ use one GPU, to avoid the impression that the listed hours are single-GPU times.","section":"Table 6 caption"}],"recommendation":"major_revision","confidential_remarks":"The reviewer agreement with the skeptic's concern is the key risk: per-model hyperparameters in Table 8 are chosen outside the method, and the paper never discloses a calibration-free selection rule for T and beta; the 'no calibration data' claim therefore needs either a disclosed protocol or a fixed-configuration experiment. The small single-run margins make the 'consistent improvement' headline brittle, but the conversion is not the only issue. The appendix derivations are correct, and the method is novel; I expect the concerns can be addressed within the scope of a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new piece is using a per-model conditional diffusion prior, learned from the LLM's own weight patches, to propose floor/ceiling flips for midpoint-ambiguous weights, with a position-dependent tolerance and a spectral-preservation candidate selector. The method is fully specified: Appendix J has a correct Tweedie-style derivation, Algorithm 2 is precise, and code is promised. The empirical support is real: ReRound beats its matched group-wise RTN on all eight models at 3 and 4 bits, and the gains come only from changed integer assignments, since scales and zero-points are fixed. That is a clean experimental design. The ablation shows each component earns its keep on OLMo 2 1B.\n\nThe soft spots are in proportion to their size. First, Table 8 gives different T and beta for every model, with no disclosed way to set them without seeing task accuracies. The paper calls itself calibration-free, but hyperparameter selection on a validation set is calibration in the looser sense. Even if the authors chose these from development runs, a practitioner applying ReRound to a new model has no stated recipe. This is the load-bearing issue: the 'consistently improves' claim may not survive a single fixed configuration, and the paper does not test that. Second, all task accuracies are single-run, with no error bars; gains of 0.1 to 1.6 points are small, and several individual tasks move the wrong way. Third, the abstract's 'superior accuracy compared to an extensive set of calibration-free methods' overstates it: on OLMo 2 1B at 4 bits, HQQ and BNB both beat ReRound (60.4 and 61.1 vs 59.4), which the paper admits in Section 4.2. Fourth, the spectral selection proxy is validated on only one model; the theoretical connection to downstream accuracy is absent, though the paper honestly notes this in Section 5.\n\nWho this is for: people working on weight-only PTQ for small models, especially on-device. It deserves a serious referee: the idea is original, the math is solid, and the main claim is credible enough to warrant scrutiny. I would send it out, but ask the authors for error bars or multiple seeds, a disclosed hyperparameter selection rule (or a fixed-default sensitivity analysis), and a correction to the abstract's overstatement.","headline":"Novel diffusion-guided rounding that likely helps small-LLM PTQ, but per-model hyperparameters and missing error bars make the 'calibration-free, consistent improvement' claim softer than stated.","tokens_in":24683,"tokens_out":3164,"would_cite":true,"duration_ms":28544,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ReRound uses a diffusion prior learned from the model's own weights to resolve midpoint-ambiguous rounding decisions, consistently improving 3- and 4-bit weight-only LLM quantization without any calibration data.","keywords":["post-training quantization","weight-only quantization","LLM quantization","diffusion prior","round-to-nearest","calibration-free PTQ","spectral preservation","midpoint ambiguity"],"falsifier":"Find a model, bit width, and task combination where the spectral-selected candidate has lower downstream accuracy than plain RTN while a weight-MSE-selected or randomly flipped candidate does better; or run the same ReRound pipeline on a larger model (e.g., 7B) and observe the gains over RTN vanish or reverse.","tokens_in":23722,"feed_emoji":"🔢","tokens_out":5569,"duration_ms":43764,"temperature":0.7,"pith_summary":"The paper introduces ReRound, a calibration-free post-training quantization method that improves round-to-nearest (RTN) by resolving the rounding decisions that RTN makes on the slimmest of margins. ReRound trains a conditional diffusion model on the pretrained LLM's own weight matrices to reconstruct continuous weights from low-bit observations, then uses those reconstructions to flip RTN assignments only near quantization interval midpoints, where the two adjacent rounding choices have nearly identical scalar error. A position-dependent tolerance metric and a selection rule that matches the leading singular values of the de-quantized candidates to the full-precision weights produce the final matrix. On small LLMs (1B–2.7B), ReRound consistently improves RTN at 3 and 4 bits, matches or exceeds calibration-based methods like SignRound without using any activations or text, and leaves the low-bit representation and inference procedure unchanged.","feed_headline":"ReRound: diffusion prior resolves midpoint rounding in low-bit LLMs","feed_subtitle":"Flips RTN assignments near midpoints, beating calibration-based PTQ on 1–2.7B models without calibration text.","key_machinery":"The central mechanism is the conditional diffusion prior over weight patches. A U-Net diffusion model is trained on 64×64 patches of the LLM's full-precision weight matrices, conditioned on 2-bit stochastic-rounding versions of the same patches, and at inference reconstructs continuous weights from deterministic 2-bit RTN conditions. The reconstruction signal is filtered by a position-dependent tolerance metric $\\tau_{\\mathrm{pos}}(r;\\tau)=\\tau\\exp\\!\\left[-\\beta\\left(\\frac{[0.5-\\delta-d(r)]_+}{0.5-\\delta}\\right)^2\\right]$, which allows flips only near the midpoint $r=0.5$ and increasingly defers to RTN toward the interval ends; the flip cost $c_{\\mathrm{flip}}(r)=\\Delta^2|1-2r|$ justifies why midpoints are the low-cost place to revise. The selection rule compares candidates through the spectral discrepancy $D_{\\mathrm{spec}}(\\tau)=\\|\\sigma_{1:k}(W_\\tau)-\\sigma_{1:k}(W)\\|_2/\\|\\sigma_{1:k}(W)\\|_2$, and because $\\tau=0$ (pure RTN) is always included, the chosen candidate never has larger spectral discrepancy than RTN itself.","core_discovery":"The central claim is that the ambiguity of round-to-nearest at interval midpoints is a useful, exploitable degree of freedom in post-training quantization: flipping a small number of RTN assignments (at most 1% per matrix) can improve downstream accuracy, provided the flips are guided by a learned prior over the model's own weight structure. ReRound implements this with a conditional diffusion model that, given a low-bit observation of a weight patch, produces a continuous reconstruction; the reconstruction proposes an alternative assignment only when it contradicts RTN, and the proposal is accepted only within a tolerance region around the midpoint. Among the candidate matrices generated by sweeping the tolerance, ReRound picks the one whose de-quantized leading singular values best match the original full-precision matrix. The paper reports consistent average-accuracy gains over group-wise RTN across eight LLMs at 3 and 4 bits, and competitive or better results than calibration-based rounding methods on the evaluated tasks.","pith_inferences":["If the spectral-preservation proxy is the true driver of the gains, then cheaper matrix-structure criteria (e.g., low-rank projections or randomized SVD) might replace the diffusion prior, making the diffusion model just a way to generate diverse candidate matrices.","The paper evaluates only models with 1B–2.7B parameters; an untested implication is whether the advantage persists at 7B+ scale, where wider matrices may require larger patch context for the diffusion prior.","The tolerance metric's hyperparameters ($\\beta$, candidate sets) are tuned per model in the appendix; a principled calibration-free procedure for setting them would be needed before the method can be applied without task accuracy feedback.","Since the reconstructed matrix is computed once and reused across bit widths, the approach could support runtime selection of bit width with no additional diffusion training."],"forward_implications":["ReRound consistently improves group-wise RTN for every tested model at both 3 and 4 bits, including models outside the main comparison such as Llama 3.2 1B, Pythia 1.4B, and Phi-2 2.7B.","ReRound matches or exceeds calibration-based PTQ methods (SignRound, AdaRound) on the evaluated small LLMs despite using no activation or text calibration data.","The selected candidate has no larger spectral discrepancy than RTN, since the $\\tau=0$ candidate is always in the set; the offline PTQ cost is modest (42–124 seconds per model).","ReRound can also improve quantization setups produced by other PTQ methods, as demonstrated on top of SINQ's scales on Qwen3 1.7B.","Because only selected floor-or-ceiling assignments change, any deployment stack supporting group-wise RTN can adopt the improvement with no inference-side changes."],"supporting_citations":[{"why":"Supplies the conditional diffusion architecture and training procedure for dequantization from low-bit conditions, which ReRound adapts to weight patches.","marker":"(Vavilala, Shaik, and Forsyth 2025)"},{"why":"Provides the DDPM forward/reverse process and the conditional-mean relation that underlies the denoising estimate.","marker":"(Ho, Jain, and Abbeel 2020)"},{"why":"Defines the rounding-assignment optimization problem (AdaRound) and serves as a calibration-based baseline that ReRound surpasses without calibration.","marker":"(Nagel et al. 2020)"},{"why":"SignRound is the strongest calibration-based rounding baseline in the comparison and the one ReRound claims to beat in Table 3.","marker":"(Cheng et al. 2024)"},{"why":"GPTQ establishes the weight-only PTQ framework with approximate second-order error minimization and is a key calibration-based baseline context.","marker":"(Frantar et al. 2022)"},{"why":"Supplies Tweedie's formula used in the appendix to justify the denoising estimate as a conditional mean.","marker":"(Efron 2011)"}],"fun_headline_variants":["Diffusion prior fixes LLM rounding at midpoints","ReRound: smart rounding for small LLMs without calibration","Calibration-free quantization gets a diffusion-guided rounding boost","ReRound: learn when to round up or down in LLM weights","Diffusion-based rounding beats RTN in low-bit LLM quantization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the candidate whose de-quantized weight matrix best matches the full-precision weights' leading singular values will also be the candidate with the best downstream task accuracy; the paper validates this on one ablation but gives no theoretical reason why spectral preservation should track language modeling or commonsense performance.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion prior fixes LLM rounding at midpoints","ReRound: smart rounding for small LLMs without calibration","Calibration-free quantization gets a diffusion-guided rounding boost","ReRound: learn when to round up or down in LLM weights","Diffusion-based rounding beats RTN in low-bit LLM quantization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00021,"raw_usage":{"total_tokens":1463,"prompt_tokens":1051,"completion_tokens":412,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":667,"completion_tokens_details":{"reasoning_tokens":328}},"tokens_in":667,"tokens_out":412,"duration_ms":4012,"temperature":1.0,"reasoning_tokens":328,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:25:32.449471+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Find a model, bit width, and task combination where the spectral-selected candidate has lower downstream accuracy than plain RTN while a weight-MSE-selected or randomly flipped candidate does better; or run the same ReRound pipeline on a larger model (e.g., 7B) and observe the gains over RTN vanish or reverse.","supporting_citations":[],"review_version":1}