{"id":"344d45d5-3c48-4b63-954d-2114d85292bb","arxiv_id":"2606.03458","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"KVarN uses Hadamard rotation plus dual-axis variance normalization on K and V matrices to cut token-scale errors and error accumulation in KV-cache quantization, reaching new SOTA at 2-bit on MATH500, AIME24 and HumanEval.","lead":"KVarN applies a Hadamard rotation followed by dual-scaling variance normalization to quantize the KV cache in large language models to 2 bits. This targets error accumulation during autoregressive decoding on reasoning tasks and ships with a vLLM implementation.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Central claim rests on unverified premise that per-token scale errors are the primary driver of accumulation and that fixed dual-variance normalization corrects them without calibration or model-specific tuning.","rationale":"Reader's weakest_assumption directly matches the load-bearing premise required for the accumulation-mitigation claim. Because the supplied context contains only the abstract, no additional internal evidence (ablations, scale-error plots, or cross-model results) is available to strengthen or refute it; the concrete test above would resolve the uncertainty without requiring the full manuscript.","tokens_in":1704,"tokens_out":388,"duration_ms":10728,"concrete_test":"On a 7B or 13B model, run 128-step autoregressive decoding on 50 MATH500 problems with (a) standard 2-bit KV quantizer, (b) same quantizer but with oracle per-token scales injected at every step, and (c) KVarN; measure both final accuracy and the L2 norm of scale deviation from FP16 at each layer. If oracle scales close most of the accuracy gap while KVarN does not, the 'primarily token-scale' premise is falsified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract states that accumulation is 'driven primarily by incorrect token scales' and that the Hadamard + dual-scaling procedure 'fixes outlying token-scale errors'. This is the load-bearing assumption: if scale mismatch is not the dominant source (e.g., if accumulation arises more from attention-score quantization, value projection drift, or interaction with rotary embeddings), or if the fixed normalization fails to stabilize scales on models outside the tested set, the SOTA claim on MATH500/AIME24/HumanEval at 2-bit does not follow. The calibration-free claim further requires that the dual-axis variance normalization generalizes without any per-model statistics or hyperparameter search.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that KV-cache quantization errors in autoregressive decoding accumulate primarily due to incorrect per-token scales, unlike in prefill settings. It introduces KVarN, a calibration-free quantizer that applies a Hadamard rotation followed by dual-scaling variance normalization across both axes of the K and V matrices. This is reported to fix outlying token-scale errors, reduce accumulation relative to baselines, and achieve new state-of-the-art results on generative reasoning benchmarks (MATH500, AIME24, HumanEval) at 2-bit precision, with a vLLM implementation provided.","tokens_in":1868,"tokens_out":582,"duration_ms":14101,"significance":"If the central attribution to token-scale errors and the calibration-free generalization hold, the work would be significant for memory-efficient long-context inference in LLMs, particularly enabling test-time scaling on reasoning tasks. The calibration-free nature and public implementation are concrete strengths that would facilitate adoption if the error-mitigation mechanism is rigorously validated.","major_comments":[{"comment":"Results section: The manuscript reports benchmark improvements but provides no quantitative error-accumulation curves (e.g., per-timestep scale deviation, MSE on KV states, or attention-score drift over decoding steps) for KVarN versus baselines. This absence leaves the load-bearing claim—that accumulation is driven primarily by incorrect token scales and is fixed by the dual normalization—without direct empirical support.","section":"Results"},{"comment":"Ablation or method evaluation: There is no ablation isolating the dual-variance normalization (across both K/V axes) from the Hadamard rotation alone, nor any test of whether the fixed normalization generalizes without per-model statistics. This is required to substantiate that the procedure corrects scales without calibration or model-specific tuning on the reported 2-bit SOTA results.","section":"Ablation"},{"comment":"Experimental setup: Details on baseline re-implementations (e.g., whether prior KV quantizers were adapted to the autoregressive regime with identical 2-bit settings and no hidden calibration) are insufficient to rule out implementation differences as the source of the reported gains on MATH500/AIME24/HumanEval.","section":"Experiments"}],"minor_comments":[{"comment":"Notation for the dual-scaling factors (e.g., how variance is computed per-axis and applied) could be clarified with an explicit equation in the method section to aid reproducibility.","section":"Method"},{"comment":"Figure captions for any error or benchmark plots should explicitly state the number of decoding steps and models evaluated to allow direct comparison with the accumulation claim.","section":"Figures"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback on our manuscript. We address each major comment below and will incorporate revisions to strengthen the empirical support and clarity of the work.","responses":[{"response":"We agree that explicit quantitative curves would provide stronger direct support for the central claim on token-scale errors. In the revised manuscript we will add per-timestep analyses, including scale deviation, MSE on KV states, and attention-score drift, comparing KVarN against baselines.","revision_made":"yes","referee_comment":"[Results] Results section: The manuscript reports benchmark improvements but provides no quantitative error-accumulation curves (e.g., per-timestep scale deviation, MSE on KV states, or attention-score drift over decoding steps) for KVarN versus baselines. This absence leaves the load-bearing claim—that accumulation is driven primarily by incorrect token scales and is fixed by the dual normalization—without direct empirical support."},{"response":"We acknowledge that an ablation isolating the dual-variance normalization from the Hadamard rotation, together with tests of generalization without per-model statistics, would strengthen the claims. We will add such an ablation study and cross-model generalization experiments in the revision.","revision_made":"yes","referee_comment":"[Ablation] Ablation or method evaluation: There is no ablation isolating the dual-variance normalization (across both K/V axes) from the Hadamard rotation alone, nor any test of whether the fixed normalization generalizes without per-model statistics. This is required to substantiate that the procedure corrects scales without calibration or model-specific tuning on the reported 2-bit SOTA results."},{"response":"We will expand the experimental setup section to include detailed descriptions of baseline re-implementations, confirming adaptation to the autoregressive regime, identical 2-bit settings, and absence of hidden calibration steps beyond those specified in the original works.","revision_made":"yes","referee_comment":"[Experiments] Experimental setup: Details on baseline re-implementations (e.g., whether prior KV quantizers were adapted to the autoregressive regime with identical 2-bit settings and no hidden calibration) are insufficient to rule out implementation differences as the source of the reported gains on MATH500/AIME24/HumanEval."}],"tokens_in":1447,"tokens_out":453,"duration_ms":17001,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper introduces KVarN, a calibration-free quantizer that applies a Hadamard rotation followed by variance normalization along both axes of the K and V matrices. The core idea is that autoregressive decoding makes per-token scale errors the main driver of accumulation, and this fixed procedure corrects them.\n\nWhat stands out is the shift in focus. Prior KV quantization work often targets prefill settings, while this one measures behavior across long decoding steps on generative tasks. The authors report better accuracy than baselines on MATH500, AIME24, and HumanEval at 2 bits, and they release a vLLM implementation, which makes the result immediately usable.\n\nThe soft spots are in the evidence for the claimed mechanism. The abstract states that incorrect token scales drive accumulation and that the dual scaling fixes outlying errors, yet it supplies no error-accumulation curves over tokens and no ablation that isolates the dual normalization from the rotation. Baseline re-implementations also lack enough detail to judge fairness. These gaps leave open whether the gains come from the stated scale correction or from other properties of the transform.\n\nThe work targets practitioners who need to stretch test-time compute on fixed hardware for reasoning models. A reader building or tuning long-horizon inference pipelines would find the method and code directly relevant.\n\nIt deserves a serious referee. The engineering contribution is concrete, the benchmarks are standard, and the calibration-free property is a practical plus. Reviewers can ask for the missing curves and ablations, but the paper is coherent enough to warrant that step.\n\nRecommendation: send it for peer review.","headline":"KVarN pairs Hadamard rotation with dual-axis variance normalization to cut error growth in autoregressive KV caches and reports 2-bit gains on reasoning benchmarks, but the mechanism rests on limited supporting analysis.","tokens_in":2363,"tokens_out":409,"would_cite":false,"duration_ms":19117,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Variance normalization after Hadamard rotation on KV caches prevents error accumulation during autoregressive reasoning.","keywords":["KV-cache quantization","error accumulation","variance normalization","autoregressive decoding","Hadamard rotation","2-bit precision","reasoning benchmarks"],"falsifier":"Measuring error accumulation rates on MATH500 or AIME24 after applying the Hadamard-plus-dual-variance-normalization procedure and finding no reduction relative to baselines would falsify the claim.","tokens_in":2616,"feed_emoji":"📉","tokens_out":654,"duration_ms":25094,"temperature":0.7,"pith_summary":"The paper shows that quantization errors in KV caches build up over successive decoding steps in autoregressive generation, driven mainly by wrong scales for individual tokens. KVarN counters this through a fixed sequence of Hadamard rotation followed by variance normalization along both axes of the K and V matrices, without any calibration data or model tuning. This procedure corrects outlying token scales and cuts error growth relative to prior quantizers. The result is stronger performance on generative reasoning benchmarks at 2-bit precision. A reader would care because the method targets a core memory constraint that limits how far test-time scaling can be pushed for improved model reasoning.","feed_headline":"Variance fix stops error buildup in KV cache quantization","feed_subtitle":"Dual normalization after rotation corrects token scales, improving 2-bit results on math and code generation tests.","key_machinery":"Dual-scaling variance normalization applied after Hadamard rotation, which normalizes K and V matrices along both sequence and feature dimensions to correct token scales.","core_discovery":"In autoregressive decoding, KV-cache quantization errors accumulate across timesteps primarily due to incorrect per-token scales. KVarN applies a Hadamard rotation followed by dual-scaling variance normalization across both axes of the K and V matrices to fix outlying token-scale errors. This combination reduces error accumulation over existing baselines and establishes new state-of-the-art results on benchmarks including MATH500, AIME24, and HumanEval at 2-bit precision.","pith_inferences":["The same normalization pattern could be tested on other cached states inside transformers to check for similar drift reduction.","Combining KVarN with complementary quantization methods might push viable precision below 2 bits on the same tasks.","The calibration-free property suggests easier deployment across varied model families without repeated tuning runs.","If the scale-correction mechanism holds, it points to a broader design principle for stabilizing sequential generation under quantization."],"forward_implications":["Longer reasoning sequences become feasible at 2-bit KV precision without proportional accuracy loss.","KV-cache quantization no longer requires per-model calibration data.","Reasoning benchmarks such as MATH500 and HumanEval see improved results at low bit widths.","Memory bottlenecks during extended autoregressive decoding are alleviated."],"fun_headline_variants":["Variance norm stops KV cache error buildup","Dual scaling fixes token scale errors in KV cache","Hadamard plus variance norm reduces error buildup","KVarN mitigates error accumulation during autoregressive decoding"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"That incorrect per-token scales are the primary driver of error accumulation and that a fixed variance normalization procedure can correct them reliably without calibration or tuning.","fun_headline_variants_meta":{"raw":{"variants":["Variance norm stops KV cache error buildup","Dual scaling fixes token scale errors in KV cache","Hadamard plus variance norm reduces error buildup","KVarN mitigates error accumulation during autoregressive decoding"]},"model":"grok-4.3","cost_usd":0.00794,"raw_usage":{"total_tokens":3606,"prompt_tokens":645,"num_sources_used":0,"completion_tokens":57,"cost_in_usd_ticks":79399500,"prompt_tokens_details":{"text_tokens":645,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2904,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":645,"tokens_out":57,"duration_ms":21993,"temperature":1.0,"reasoning_tokens":2904,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-28T11:34:35.910708+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Measuring error accumulation rates on MATH500 or AIME24 after applying the Hadamard-plus-dual-variance-normalization procedure and finding no reduction relative to baselines would falsify the claim.","supporting_citations":[],"review_version":1}