{"id":"4ba8eb4b-1186-4e1f-b278-c525917995a8","arxiv_id":"2608.08506","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A training-free compression pipeline that updates calibration activations layer by layer and re-allocates ranks iteratively improves compressed LLM accuracy by up to a few points on five zero-shot tasks.","lead":"The paper identifies two sources of drift in training-free low-rank LLM compression, calibration activation errors and layer importance shifts, and adds corrections for both. The corrected pipeline improves zero-shot accuracy by around 1 to 2.5 points on several Llama and Qwen3 models, which matters for deploying smaller compressed models on edge devices.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1's C+R gains are selected, not fixed: α and N are chosen per cell as the best LM-Eval configuration, so the headline improvements and W/T/L record may be test-set selection artifacts.","rationale":"The strongest claim is that applying the two corrections to a joint-decomposition baseline yields consistently better zero-shot accuracy. Three conditions would have to hold: (a) the calibration correction is beneficial on average, (b) the rank-allocation correction is beneficial once calibrated, and (c) the reported comparisons reflect a fair, reproducible protocol. The least secure condition is (c). The paper does not list α and N for the Table 1 cells, and the appendix explicitly says those entries are the best LM-Eval configuration. Because only five tasks are averaged and the grid is tiny (two α values and up to three N values), per-cell selection can easily account for the roughly 0.2–1pp differences that drive the W/T/L record. The paper's own ablations show negative average R gains on several model groups for α=0.05 at N=1, so the method is not robust to a reasonable fixed choice. A reviewer therefore cannot yet distinguish 'the correction helps' from 'the chosen configuration overfits these five tasks.' The reader's weakest_assumption focused on BI-score reliability; that is a real secondary concern, but the selection issue is more load-bearing because it affects the headline numbers even before the validity of the BI metric is considered. I do not see a fatal logical flaw in the calibration correction itself, and the C-only 0.50pp gap is a useful partial result, which is why I recommend keeping the conditional verdict rather than moving to reject.","tokens_in":22384,"tokens_out":10583,"duration_ms":105314,"concrete_test":"Recompute Table 1 with a pre-registered held-out selection protocol: for each model and compression rate, tune α∈{0.01,0.05} and N∈{1,2,3} on a random half of the five LM-Eval tasks, evaluate on the complementary half, and average over repeated random splits. Then recompute the Average Gap from Best and the W/T/L record against UniQL for C+R. If the held-out-selected gap is not below UniQL's 1.22pp and the W/T/L record is not at least as favorable as 18/3/3, the reported per-cell best-configuration numbers in Table 1 are selection artifacts rather than evidence for the correction.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central empirical support for the rank-allocation correction rests on per-cell hyperparameter selection on the same evaluation benchmarks. Section 4.4 states that 'the choice of α and N is quintessential' and reports experiments over α∈{0.01,0.05} and N≤3; Table 8's caption confirms that the Table 1 C+R entries 'are for the best LM-Eval configuration from Table 1.' Appendix C.5 quantifies the fragility: at N=1, α=0.05 gives mean changes of −0.20pp for Llama-3.2-1B/3B and −0.07pp for the Qwen3-4B pair, while α=0.01 gives only +0.01pp on the small Llama group. A fixed or validation-selected hyperparameter can therefore erase or reverse the R gains, and the 18/3/3 W/T/L record versus UniQL plus the 'up to 1–2.5pp' improvements are not evidence for a single robust training-free correction. The 0.50pp Average Gap shown in Figure 3 is computed for the C component alone, so the abstract and introduction attribute that number to 'our approach' with both corrections, which is a mismatch. The combined C+R story, which is the central claim, needs held-out hyperparameter selection before it supports the conclusions.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies two sources of misalignment in training-free low-rank LLM compression: residual calibration-data activation errors that accumulate across layers, and drift in layer importance scores after compression. It proposes two training-free corrections—layer-by-layer calibration correction (C) and iterative rank-allocation correction (R)—and implements them on top of the UniQL joint-decomposition framework. Evaluation across eight Llama and Qwen3 models at 15%, 30%, and 40% compression rates on five zero-shot LM-Eval tasks is reported, with claims of up to 1–2.5 percentage-point accuracy improvements and an average gap from the best method of 0.50pp versus 1.22pp for UniQL and 1.71pp for MoDeGPT. The paper also includes ablations of the two corrections, repeated-seed variance, a Qwen2.5-32B scaling experiment, and a compatibility study with the post-hoc correction method EoRA.","tokens_in":22696,"tokens_out":6157,"duration_ms":54559,"significance":"If the two corrections are as effective as claimed, the paper would make a useful, simple, and training-free contribution to LLM compression, since the corrections are compatible with existing joint-decomposition pipelines and add no inference cost. The paper's strengths include a broad evaluation across model families and compression rates, per-benchmark breakdowns, wall-clock compression-time comparisons, an explicit compatibility study with EoRA, and a candid limitations section. The aggregate average-gap metric is favorable for the calibration correction alone. However, the headline claims for the combined C+R method are weakened by per-cell hyperparameter selection on the evaluation benchmarks, a mismatch between the reported average-gap figure and the claims in the introduction, and a 32B experiment that contradicts the generality of the calibration-correction mechanism. These issues bear directly on the central claim and require substantive revision.","major_comments":[{"comment":"The C+R results in Table 1 appear to be selected per cell as the best configuration over α and N on the same LM-Eval benchmarks used for evaluation. Table 8's caption states that the (C+R) scores are for the best LM-Eval configuration from Table 1, and Section 4.4 describes the choice of α and N as 'quintessential.' Appendix C.5 quantifies the fragility: at N=1, α=0.05 gives mean changes of −0.20pp for Llama-3.2-1B/3B and −0.07pp for the Qwen3-4B pair, while α=0.01 gives only +0.01pp on the small Llama group. A fixed or validation-selected hyperparameter can therefore erase or reverse the R gains, and the 18/3/3 W/T/L record versus UniQL plus the 'up to 1–2.5pp' improvements are not evidence for a single robust training-free correction. The paper needs held-out hyperparameter selection, or a fixed configuration report, before the combined C+R claim is supported.","section":"Section 4.4, Table 8 caption, Appendix C.5 (Tables 10–11)"},{"comment":"The abstract and the contribution bullet attribute the 0.50pp average gap from the best method to 'our approach' with both corrections, but Figure 3 explicitly states that 'Ours here indicates after calibration correction (C)' and Section 4.3 confirms that the Average Gap analysis focuses on the calibration correction (C) alone. The combined C+R average gap is not reported. This is a mismatch between the headline summary statistic and the actual computation, and it overstates what the evidence supports. The introduction and Section 6 should attribute the 0.50pp figure to C only, or the combined C+R average gap should be computed and reported.","section":"Figure 3 caption, Section 4.3, and introduction contribution bullet"},{"comment":"The Qwen2.5-32B scaling experiment contradicts the calibration-correction claim. Relative to UniQL at 40% compression, Ours (C) is worse on WikiText-2 PPL (24.16 vs. 20.50), 0-shot LM-Eval average (57.42 vs. 59.84), and MMLU (27.20 vs. 32.40). The text states that 'pure corrective calibration (C) does not lead to performance improvements' for larger models. This undermines the mechanism claim in Sections 1 and 4.3 that residual calibration error accumulation is a general source of misalignment that the C correction mitigates. The authors should present the 32B result as an explicit boundary condition and discuss what it implies for the claimed universality of the calibration-correction mechanism.","section":"Section 4.5, Table 4"},{"comment":"The repeated-seed experiment does not provide statistical support for the claim that 'the findings hold.' For Llama-3.2-1B at 40% compression, UniQL is 41.78±0.48 and Ours (C+R) is 41.93±0.55, fully overlapping within one standard deviation; for Llama-3.2-3B, the two methods differ by 0.77pp with standard deviations of 0.63 and 0.45 at n=3, which is not significant without a proper test. The paper reports single-run results throughout the main tables, and the repeated-seed subsection is the only place where variance is quantified; as it stands, it does not demonstrate that the reported improvements are robust.","section":"Table 3"},{"comment":"The diagnostic evidence for the calibration correction is partly tautological. The NMSE reduction in Table 9 and the BI-score convergence in Algorithm 1 are computed on the same 128-sample calibration set whose activations the correction updates at each layer; feeding compressed-model activations back into the compression loop will, by construction, reduce calibration-set NMSE relative to a one-pass method. This does not by itself establish that the correction reduces true inference-time error, which is an assumption (stated in the paper's framing). The held-out LM-Eval accuracy numbers are not circular, but the paper's mechanistic interpretation should be validated on held-out activations or explicitly tied to the held-out task improvements rather than presented as direct evidence of error propagation being fixed.","section":"Section 3.1, Table 9, and Algorithm 1"}],"minor_comments":[{"comment":"Equation (9) has unbalanced parentheses in the second term: 'f(|| ˆX; ˆW1,· · ·ˆWk)||)2' appears to be missing a closing parenthesis for the norm or for f(·). Please fix the notation.","section":"Equation (9)"},{"comment":"The caption of Figure 3 says 'Ours here indicates after calibration correction (C),' but the legend and text sometimes refer to 'Ours' without the '(C)' qualifier. Consider relabeling the legend to 'Ours (C)' for consistency with Table 1.","section":"Figure 3"},{"comment":"The bullet list in Section 4.2 has formatting issues: 'Model Family.Llama-3.2 models...' should have a space and consistent punctuation after the bolded item labels. This is a minor presentation issue.","section":"Section 4.2"},{"comment":"The paper does not state whether code or checkpoints will be released. Given the empirical nature of the work and the claim of compatibility with existing frameworks, a statement about code release would improve reproducibility.","section":"Section 4.5 and Appendix E"}],"recommendation":"major_revision","confidential_remarks":"The key barrier is the per-cell selection of α and N on the same LM-Eval benchmarks used to report the headline C+R improvements; the paper's own Appendix C.5 shows that fixed configurations can reverse the gains. The 0.50pp average gap is also attributed to the combined approach when it is computed for C alone. These are fixable with additional experiments or revised claims, but they are central to the paper's message. I would like to see either held-out hyperparameter selection or a clear reframing of the claims before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth reading for the empirical characterization, not for the headline accuracy gains. It shows clearly that calibration activations accumulate error across layers in joint low-rank decomposition, and that Block Influence scores drift after compression. Those two effects are real, and the figures make the case. The proposed corrections are simple, training-free, and compatible with UniQL/MoDeGPT, which is a genuinely useful property.\n\nWhat's new: the layer-by-layer calibration correction generalizes PTQ-style online calibration to low-rank joint decomposition, and the iterative rank re-allocation via a delta rule is new as far as I can tell. The ablation work is decent: repeated seeds, MMLU, perplexity, EoRA synergy, and a 32B scaling experiment.\n\nThe main soft spot is the selection issue. Table 8's caption admits the C+R numbers are the best LM-Eval configuration chosen from Table 1's grid. Appendix C.5 shows that at N=1, alpha=0.05 hurts the small Llama and Qwen3 groups, and alpha=0.01 gives negligible gains. So the 18/3/3 W/T/L and the 'up to 1-2.5pp' claims are test-set selection, not a fixed method. A fixed or validation-selected hyperparameter would likely erase much of the R gain. Second, Figure 3's 0.50pp average gap is for C alone, while the intro attributes it to 'our approach' with both corrections; that mismatch should be fixed. Third, the main table has no error bars, and the repeated-seed experiment shows overlapping standard deviations, so the headline differences are not individually significant. Fourth, the 32B result contradicts the calibration-correction story: C makes things worse on Qwen2.5-32B, and while the paper handles that honestly, it limits generality.\n\nThe circularity concern is real but not fatal. The NMSE and BI diagnostics are computed on the same calibration set used for the corrections, so they partly measure self-consistency; the accuracy numbers come from held-out LM-Eval tasks, so the central claim isn't label-fitting. The citation pattern looks fine; they cite the PTQ works that did online calibration, and Section 6 discusses hyperparameter choice openly.\n\nThis is a useful paper for people working on training-free compression. It identifies real failure modes and offers a cheap, compatible patch that often helps and sometimes hurts. It deserves a serious referee, but acceptance should be conditional on reporting held-out hyperparameter selection, adding error bars, and clarifying that the pooled gap is C-only. I'd send it to review with those revisions requested.","headline":"Useful empirical study of two real misalignment effects in training-free low-rank compression, but the headline C+R gains rest on per-cell hyperparameter selection on the test benchmarks; the C-only average-gap result is more solid but more modest.","tokens_in":23202,"tokens_out":2645,"would_cite":true,"duration_ms":26767,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training-free low-rank compression of LLMs silently drifts from its own calibration estimates as layers are compressed, and the paper shows that two no-training corrections—refreshing calibration activations layer by layer and re-running…","keywords":["low-rank compression","training-free","LLM compression","calibration correction","rank allocation","truncation error","Block Influence score","zero-shot accuracy"],"falsifier":"Compute two rank allocations for one model at 40% compression: one from the paper's BI-drift rule and one from a sensitivity oracle such as leave-one-layer-out accuracy loss or per-layer Fisher information. If the BI-drift allocation is not consistently closer in accuracy to the oracle allocation than the original static allocation, the central claim fails. A cheaper signal: on Qwen3-1.7B, where the paper's own table shows the combined corrections below the baseline at 40%, check whether the reallocated ranks move budget into layers whose BI drift is positive but whose oracle sensitivity is flat.","tokens_in":22206,"feed_emoji":"🎯","tokens_out":7344,"duration_ms":72391,"temperature":0.7,"pith_summary":"The paper claims that training-free low-rank compression of large language models quietly invalidates its own assumptions as it runs: residual errors in the calibration activations that guide each layer's decomposition accumulate and grow deeper into the model, and the layer-importance scores used to allocate ranks shift once weights are compressed. It proposes two corrections that require no training: recompute calibration activations layer by layer through the freshly compressed layers, and rerun the rank allocation for a few rounds, nudging each layer's rank ratio by the drift in its Block Influence score. Implemented on top of an existing joint-decomposition compressor and tested on eight Llama and Qwen3 models at 15, 30, and 40 percent compression, the corrections raise zero-shot accuracy by about 1 to 2.5 points over the per-weight and joint baselines and cut the average gap to the best method from 1.22 and 1.71 points down to 0.50 points. If right, this means existing training-free compressors can be made more accurate for free at inference time, with little added compression cost.","feed_headline":"Two fixes halve the accuracy gap of compressed LLMs","feed_subtitle":"Refreshing calibration activations and reranking layers lifts zero-shot scores by up to 2.5 points.","key_machinery":"The load-bearing mechanism is a revised compression loop. In the standard pipeline, calibration activations and per-layer rank ratios are collected once on the uncompressed model; in Algorithm 1 of the paper, the two are refreshed inside the compression pass. The calibration correction replaces the original activation $X$ in the reconstruction objective $O=\\min\\|WX-\\hat{W}\\hat{X}\\|^2$ with the activation $\\hat{X}$ produced by the already-compressed earlier layers, eliminating layer-by-layer error accumulation by construction. The rank-allocation correction recomputes Block Influence scores (the complement of the cosine similarity between a layer's input and output activations) on the compressed model and adjusts each layer's rank ratio with the delta update $\\hat{\\phi}_i^n = \\phi_i + \\mathrm{Pr}(\\alpha\\Delta_i^n, r_{\\text{target}})$, where $\\Delta_i^n$ is the normalized BI drift for layer $i$ in round $n$ and $\\mathrm{Pr}$ projects the scores back onto the global retention budget. The two mechanisms are decoupled: calibration correction works at the level of the forward activations, rank correction at the level of the layer-wise budget.","core_discovery":"The central discovery is that the two misalignment channels share one root cause: training-free low-rank compression treats the original model as a static reference, while in fact every compression step changes the signals on which later steps depend. Calibration residuals grow with depth, reaching normalized mean squared errors up to 0.27 at the final layer, because each layer's decomposition is fit to activations from the uncompressed model rather than to the activations the compressed prefix will actually produce. Layer importance also drifts, with BI scores shifting by up to 0.17 after compression, so the precomputed rank-ratio vector no longer matches the compressed model. The paper's fixes update the calibration activations on the fly by passing data through each just-compressed layer, and refine the rank ratios through a damped delta rule over a few compression rounds. On the evaluated models, the combined corrections achieve 18 wins, 3 ties, and 3 losses against the joint-decomposition baseline, and 18 wins and 6 losses against the modular baseline.","pith_inferences":["A natural untested extension is to swap the Block Influence score for a different sensitivity oracle, such as Fisher information or leave-one-layer-out accuracy loss; if the accuracy gains persist, the correction comes from the iteration scheme itself, not from the specific importance metric.","Because the delta rule's dampening coefficient α behaves like a learning rate, annealing α across rounds or selecting it with a quick holdout estimate is a plausible improvement over the fixed α ∈ {0.01, 0.05} explored here.","The 0.50 pp average-gap number is relative to the three baselines included in this paper; other training-free compressors not in the comparison could set a different best-oracle, so the absolute claim is benchmark-relative.","For small Qwen3 models at aggressive 40% compression, the combined correction occasionally trails the base method, suggesting the rank-delta update can move budget in the wrong direction when BI drift is noisy; a signed or heavier-tailed update rule would be worth testing."],"forward_implications":["Applying corrective calibration to an existing joint-decomposition compressor adds no inference cost and roughly 1.5–1.8x the compression time of the base joint method, yet remains 2.5–5x faster than the modular pseudo-inverse baseline.","Across all configurations, the average gap from the best-performing method drops to 0.50 percentage points from 1.22 for the joint baseline and 1.71 for the modular baseline, and the combined correction wins or ties 21 of 24 comparisons against the joint baseline.","The corrections transfer across model families and sizes: Llama models benefit most, and a 32B model improves through rank correction alone even when calibration correction alone does not help.","The corrections are complementary to post-hoc error compensation: adding a block-level residual path improves all tested configurations by 0.41 points on average and does not change which compression configuration is best.","At aggressive 40% compression, the corrected method achieves the best zero-shot scores across all tested Llama models, extending the accuracy–compression trade-off frontier beyond the baselines."],"supporting_citations":[{"why":"Provides the UniQL joint-decomposition method on top of which both corrections are implemented, and is the main baseline for accuracy comparisons.","marker":"[Chiang et al., 2026]"},{"why":"Provides the MoDeGPT modular-decomposition baseline and the joint-decomposition and global rank-allocation ideas that this paper's analysis targets.","marker":"[Lin et al., 2025]"},{"why":"Supplies the Block Influence score used to measure layer importance, set initial rank ratios, and compute drift for the rank-allocation correction.","marker":"[Men et al., 2025]"},{"why":"Supplies the SVD-LLM per-weight activation-aware decomposition baseline.","marker":"[Wang et al., 2025b]"},{"why":"Supplies the LM-Eval harness and the five zero-shot tasks used to measure all reported accuracy numbers.","marker":"[Gao et al., 2023]"}],"fun_headline_variants":["Calibration correction lifts compressed LLM accuracy by 2.5 points","Training-free fix halves accuracy gap in compressed LLMs","Layer-by-layer calibration rescues compressed LLM performance","Rank and calibration fixes narrow LLM compression gap"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on the assumption that the drift of Block Influence scores, measured on 128 calibration samples, faithfully reveals which layers lost importance after compression; if a layer's BI drift points the wrong way, the rank reallocation can hurt accuracy.","fun_headline_variants_meta":{"raw":{"variants":["Calibration correction lifts compressed LLM accuracy by 2.5 points","Training-free fix halves accuracy gap in compressed LLMs","Layer-by-layer calibration rescues compressed LLM performance","Rank and calibration fixes narrow LLM compression gap"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000183,"raw_usage":{"total_tokens":1310,"prompt_tokens":935,"completion_tokens":375,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":309}},"tokens_in":551,"tokens_out":375,"duration_ms":4140,"temperature":1.0,"reasoning_tokens":309,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:33:14.285584+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute two rank allocations for one model at 40% compression: one from the paper's BI-drift rule and one from a sensitivity oracle such as leave-one-layer-out accuracy loss or per-layer Fisher information. If the BI-drift allocation is not consistently closer in accuracy to the oracle allocation than the original static allocation, the central claim fails. A cheaper signal: on Qwen3-1.7B, where the paper's own table shows the combined corrections below the baseline at 40%, check whether the reallocated ranks move budget into layers whose BI drift is positive but whose oracle sensitivity is flat.","supporting_citations":[],"review_version":1}