{"id":"d0e6e74a-e451-4f8f-8080-542382951ec1","arxiv_id":"2608.08684","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"RippleKV allocates KV cache budgets across transformer layers by measuring how much perturbing each layer's value cache shifts the model's output distribution, and it reports the best average LongBench score among the compared compression methods at matched budgets.","lead":"RippleKV is a new way to split the memory budget of a large language model's cache across its layers: it measures which layers the model's output is most sensitive to and gives those layers more cache. On the LongBench long-context benchmark it beats existing cache-compression methods by a small but consistent margin at matched memory budgets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Full-cache Gaussian value perturbation is never shown to rank compression damage at the actual 10–30% budgets; the asserted budget-invariance of the sensitivity profile is untested.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: perturbation-response proxy fidelity, including the asserted but untested budget-invariance. I considered other candidate concerns — missing calibration-set details, absent error bars, and unreported hyperparameters such as alpha, gamma, and the calibration protocol — but the most load-bearing is the proxy-fidelity/budget-invariance link, because it is the causal chain connecting the method to the reported gains. If that link fails, the observed improvements could be explained by task-specific calibration or noise; if it holds, the paper is a solid conditional contribution. The Table 1 correlation is suggestive but is measured against a D_l computed at an unspecified compression setting, and no test checks whether the same profile remains valid at 10%, 20%, and 30% budgets. The proposed concrete test directly measures rank correlation at each budget and tests whether re-estimating the profile per budget changes the result. Since the reader already marked the paper CONDITIONAL for closely related reasons, no verdict change is needed; the condition should explicitly require the budget-invariance check and calibration-set disclosure.","tokens_in":12780,"tokens_out":6790,"duration_ms":75518,"concrete_test":"On Llama-3.1-8B-Instruct, compute isolated compression damage D_l (Eq. 1) under SnapKV eviction separately at 10%, 20%, and 30% budgets; compute Spearman correlation between D_l at each budget and the Eq. (8) perturbation profile s_l; then compare LongBench averages under the fixed reused profile versus a profile recomputed at each budget, using a calibration set disjoint from the LongBench test tasks. If rank correlation drops materially (e.g., below ~0.6) at either budget, or if per-budget profiling improves average score by more than ~0.5 points, the budget-invariance premise is false and the central improvement claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"RippleKV's central premise is that the perturbation-response profile s_l (Eqs. 6–8), measured by adding norm-adaptive Gaussian noise to value vectors at the full cache with keys and attention weights fixed, ranks layers by the damage caused by real token-eviction compression at the deployment budget. The only direct evidence is Table 1, where mean |rho| is 0.799 (range 0.723–0.857) against an isolated-compression damage D_l whose budget is not stated. This does not support the reuse claim: the paper asserts 'The resulting sensitivity profile is independent of the target cache budget' without measuring D_l separately at 10%, 20%, and 30%. Eviction removes keys as well as values, so attention weights are renormalized after compression, whereas the perturbation keeps keys fixed; these two effects need not have the same layer ordering at different compression severities. If the ranking changes with budget, the single offline profile misdirects budgets at 20% and 30%, and the reported margins (0.28–0.83 average points) may not transfer. The missing calibration-set details (source, size, overlap with LongBench) compound this: if the calibration set is drawn from the evaluation tasks, part of the gain is task-specific rather than model-level.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes RippleKV, a cross-layer KV cache allocation method that assigns each Transformer layer a cache budget based on how perturbing that layer's value cache shifts the model's final predictive distribution. Sensitivity is measured offline by adding norm-adaptive Gaussian noise to a layer's value vectors (keeping keys and other layers fixed), averaging the resulting KL divergence over a calibration set (Eqs. 6-8), and converting the normalized sensitivity profile into budget multipliers through an exponential mapping with a ratio parameter and a clipping-normalization step (Eqs. 9-12). The authors report experiments on LongBench with Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Mistral-7B-Instruct-v0.3, comparing against StreamingLLM, H2O, SnapKV, and PyramidKV at 10%, 20%, and 30% cache budgets. They claim RippleKV achieves the highest average score in all five displayed compressed settings while matching baselines in latency and memory, and that the sensitivity profile is a better allocation signal than layer depth or layer-local response.","tokens_in":12859,"tokens_out":4465,"duration_ms":50395,"significance":"If the central claim holds, RippleKV is a useful, low-cost addition to the layer-wise KV cache allocation literature: it requires no gradient computation, the profile is computed once per model, and the budget mapping is scale-invariant and budget-preserving. The paper has real strengths: the perturbation-response idea is clearly motivated, Table 1 provides a direct correlation statistic (mean |rho|=0.799) linking the proposed signal to isolated compression damage, per-dataset results are disclosed in Table 2, and the ablation table (Table 4) is internally consistent with the main results. The contribution, however, is incremental rather than transformative, and the empirical support is currently incomplete: only one of the two load-bearing premises (perturbation-response fidelity at the actual deployment budget) is directly tested, and even that test leaves roughly a third of ranking variance unexplained. The reported gains over the strongest baseline are 0.28 to 0.83 average points with no error bars or significance tests. The missing experiments are feasible, so the idea is defensible, but the manuscript is not yet at the standard of a definitive systems/empirical paper.","major_comments":[{"comment":"The load-bearing proxy-fidelity claim is not directly tested at the deployment budgets. The paper states that the sensitivity profile is independent of the target cache budget, but Table 1 reports only the correlation of the perturbation response with an isolated compression damage D_l whose compression budget is not specified. Actual eviction at 10-30% removes keys and renormalizes attention weights, whereas the perturbation in Eq. (6) adds noise to values while keeping keys and attention weights fixed; these two operations need not have the same layer ordering at different compression severities. The authors should measure D_l separately at 10%, 20%, and 30% budgets, compute the Spearman correlation between s_l and D_l at each budget, and verify either that the ranking is stable or that the profile is recomputed per budget. Without this, the claimed reuse of a single profile is unsupported, and this is load-bearing for the method's correctness.","section":"Perturbation Response section and Methodology (Eqs. 6-8)"},{"comment":"The calibration set D_cal is never described: its size, source, and overlap with the LongBench evaluation tasks are all absent. If the calibration examples are drawn from the test tasks, part of the reported gains could be task-specific rather than a general model-level sensitivity profile. The authors should specify the calibration data, its size, how the evaluation windows W_i are chosen, and ideally demonstrate that a profile computed on held-out calibration data still transfers to the LongBench datasets.","section":"Algorithm 1 / Experimental Settings"},{"comment":"The central empirical claim is supported by only five compressed settings: Llama-3.1-8B at 10%, 20%, and 30%, and Qwen2.5 and Mistral at 10% only. The 20% and 30% rows for Qwen and Mistral are missing from Table 2, which weakens the claim of consistent improvement across 'multiple cache budgets' and three model families. Given that the margins over the strongest baseline are 0.28 to 0.83 average points, the paper needs error bars (multiple seeds or multiple calibration draws) and, preferably, a paired significance test before claiming a robust advantage.","section":"Table 2 / Main Results"},{"comment":"The paper claims RippleKV 'can be integrated with existing KV cache compression methods,' but all experiments instantiate it only with SnapKV token scoring. The generality of the allocation layer would be much better supported by at least one additional base method (e.g., H2O or StreamingLLM scoring). If such experiments are not provided, the claim should be weakened to compatibility with SnapKV-style selection.","section":"Methodology / Implementation Details"}],"minor_comments":[{"comment":"Please state the compression budget and the compression method used to compute the isolated compression damage D_l; the table currently leaves this unspecified.","section":"Table 1"},{"comment":"The normalized sensitivity score is written bs_l, which collides notationally with the budget symbol b_l; please use a different symbol for the normalized score.","section":"Algorithm 1, line 13"},{"comment":"The allocation ratio is denoted r in Algorithm 1 and Eq. (10) but R in Table 5; please unify the notation.","section":"Table 5 and Eq. (10)"},{"comment":"The column header 'Full' is ambiguous: clarify that it denotes the full RippleKV model rather than the Full Cache reference, and consider adding an average row to the table.","section":"Ablation Study, Table 4"},{"comment":"Clarify how the evaluation window W_i is selected and how 'positions protected by the base compression method are excluded from perturbation' interacts with the budget-dependent token selection; this is important for reproducibility.","section":"Equation (7) and profiling procedure"},{"comment":"The offline profiling cost (N times |A| forward passes) is not quantified. A brief report of profiling runtime or FLOPs would help readers assess the practical one-time overhead.","section":"Efficiency analysis"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about budget-invariance is real and lands: the paper asserts profile reuse across budgets without measuring the correlation at the actual 10-30% budgets, and the calibration-set details are absent. Both are fixable with experiments, so I recommend major revision rather than rejection. The contribution is modest but publishable if the empirical support is completed; the missing 20/30% results for Qwen and Mistral and the unspecified calibration set are the most urgent items."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: RippleKV is a solid, well-scoped engineering contribution. It takes the older perturbation-sensitivity idea from quantization (HAWQ-V2 and the like) and turns it into a cross-layer KV cache budget allocator, and it is honest about that lineage. What is actually new is the specific mechanism: norm-adaptive Gaussian perturbation of the value cache with keys and attention fixed, converting the output KL response into layer budgets through a normalized exponential mapping. The empirical finding that this signal ranks layers far better than depth (mean |rho|=0.799 vs 0.359) is genuinely useful.\n\nThe experiments are mostly fine. The main claim — best average LongBench score across three model families and three budget levels — holds up in the displayed tables. The ablation and hyperparameter sensitivity tables are internally consistent, per-dataset losses are disclosed, and the efficiency numbers confirm that budget reallocation adds no measurable online cost. That is the right thing to check.\n\nNow the soft spots, in rough order of seriousness. First, the margins over the strongest baseline are 0.28 to 0.83 average points with no error bars or multi-seed runs; for all we know those differences are within noise. That is the biggest single weakness. Second, the proxy-fidelity premise is only partly established: Table 1 shows rho=0.799, which leaves a third of the ranking variance unexplained, and the budget-invariance statement is asserted without a direct test. The stress-test worry that the profile might misdirect at 20% and 30% does not fully land, because the method uses one profile at all three budgets and still shows gains at all three — that is indirect evidence, not a clean test, but it is evidence. Third, r=1.50 is selected on LongBench itself; the sensitivity sweep shows the choice is mild, so this is a minor issue, not a load-bearing one. Fourth, the calibration set (source, size, overlap with LongBench) and profiling parameters alpha and gamma are unreported, and the appendix that supposedly contains them is missing. That is a genuine reproducibility blocker.\n\nWho is this for? Anyone working on long-context inference efficiency. It is not a deep theoretical paper and the gains are small, but the method is cheap, plausible, and the evaluation is more thorough than most in this niche. It deserves a serious referee. I would send it to review with the expectation of moderate revision: add variance or multi-seed results, report the calibration protocol, and ideally test budget-invariance directly at 10%, 20%, and 30%.","headline":"RippleKV is a legitimate engineering contribution that repurposes perturbation sensitivity for cross-layer KV cache allocation; the gains are real but small and the paper is missing variance estimates, calibration details, and a direct test of budget-invariance.","tokens_in":13573,"tokens_out":3398,"would_cite":false,"duration_ms":34760,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RippleKV shows that a layer's value-cache sensitivity to perturbation predicts how much it can be compressed, and that allocating cache budgets by this signal beats depth-based rules.","keywords":["KV cache compression","layer-wise budget allocation","perturbation propagation","output sensitivity","long-context inference","token eviction","LongBench","KL divergence"],"falsifier":"Take a held-out model not used in the paper, compute per-layer isolated compression damage $D_\\ell$ (Eq. 1) at a budget such as 5% or 10%, and compare its rank ordering against RippleKV's perturbation profile $s_\\ell$; if the Spearman correlation falls well below 0.799, or if the budget ranking from one budget underperforms uniform allocation on the target budget, the central proxy is falsified.","tokens_in":12422,"feed_emoji":"🎯","tokens_out":6412,"duration_ms":61587,"temperature":0.7,"pith_summary":"RippleKV's core claim is that a layer's true cache need is how strongly changes to its value cache ripple into the model's final predictions, and that this can be measured offline with controlled noise rather than by trying every compression schema. Across 32 layers of one model, isolated compression damage varied 17.7-fold and did not track depth; the perturbation response predicted that damage with mean Spearman |rho| = 0.799 versus 0.359 for layer index. The paper proposes allocating a fixed total KV cache budget by this sensitivity profile through a normalized exponential mapping, and reports the highest average LongBench scores across three model families at 10-30% cache budgets while leaving the within-layer token scoring unchanged. A sympathetic reader would take the contribution as showing that end-to-end output sensitivity is a practical, reusable signal for cross-layer cache budgeting.","feed_headline":"Perturbation-guided KV cache budgets beat layer-position heuristics","feed_subtitle":"By injecting norm-scaled noise into value caches and watching the output, RippleKV posts the top average LongBench scores.","key_machinery":"The load-bearing object is the per-layer perturbation response $s_\\ell = \\frac{1}{N}\\sum_i d_{i,\\ell}$, where $d_{i,\\ell}$ is the average KL divergence between the full-cache output and the output after adding norm-scaled Gaussian noise $\\alpha\\|v_{\\ell,h,u}\\|_2 \\epsilon$ to that layer's value vectors. The argument runs on the claim that this response ranks layers like true compression damage; the allocation machinery then maps it to budgets by $\\tilde{s}_\\ell = (s_\\ell - s_{\\min})/(s_{\\max} - s_{\\min})$, multiplier $r^{\\tilde{s}_\\ell}$ with $r \\ge 1$, clipping to $[1/\\gamma, \\gamma]$, and rescaling to mean one, so the total budget $B$ is preserved.","core_discovery":"On the paper's own terms, the discovery is a proxy that works: norm-adaptive perturbations of a layer's value cache, with keys and attention fixed, produce final-output KL shifts that rank layers in nearly the same order as actually evicting tokens from that layer (mean |rho| = 0.799 vs 0.359 for depth). This makes layer budgets a model-specific, nonmonotonic function of the sensitivity profile rather than a fixed depth schedule. RippleKV converts the profile into budgets via min-max normalization, an exponential mapping with ratio r, clipping, and normalization to preserve the global budget; it only redistributes memory across layers, leaving the token selection strategy (SnapKV in the paper's implementation) untouched. On LongBench, that redistribution yields the best average scores in every compressed setting tested, with larger gains on multi-document QA and synthetic tasks.","pith_inferences":["Beyond the paper's experiments, the same perturbation-response principle could be used to set per-head or per-token budgets within a layer, not just per-layer totals, since the measurement is defined at any granularity of value-cache units.","Because profiling happens at full cache, the method implicitly assumes rank-order stability of the sensitivity profile across budgets; a direct test at very low retention (e.g. 5%) or on longer contexts than the calibration set would show whether the profile is truly budget-invariant.","The calibration set's composition matters: if deployment shifts to tasks whose attention or value statistics differ sharply from the calibration window, the KL responses may re-rank, so an online low-cost variant that recomputes the profile on a small prefix of each document is a natural extension."],"forward_implications":["Any fixed total cache budget can be redistributed by the same profile; the profile is reused across budgets without re-profiling.","Layer allocations become nonmonotonic with depth, so intermediate layers that are most damage-prone get more cache than the first or last layers.","RippleKV adds no per-token inference cost: profiling is offline and deployment only changes per-layer budget sizes, so it layers on top of existing eviction scorers.","The reported gains concentrate in tasks that need multi-hop evidence (multi-document QA) and synthetic retrieval, where a single evicted token in a sensitive layer can break the answer."],"supporting_citations":[{"why":"Supplies the LongBench benchmark used for all task-category evaluations and averages.","marker":"Bai et al. 2024"},{"why":"SnapKV, whose token scoring and selection RippleKV keeps fixed so the only change is per-layer budgets.","marker":"Li et al. 2024"},{"why":"PyramidKV, the depth-schedule layer-allocation baseline RippleKV is compared against and outperforms.","marker":"Cai et al. 2024"},{"why":"H2O, a heavy-hitter attention baseline that must be matched under equal total cache budget.","marker":"Zhang et al. 2023"},{"why":"StreamingLLM, attention-sink baseline, one of the four compression methods in the comparison.","marker":"Xiao et al. 2024"},{"why":"Cited for the claim that perturbations propagate through later layers, justifying measuring the end-to-end output response.","marker":"Jing et al. 2026"},{"why":"Cited alongside for propagation-based sensitivity in networks; supports the choice of output-level response over layer-local statistics.","marker":"Dong et al. 2020"}],"fun_headline_variants":["RippleKV: Perturbation-guided KV cache budgets top LongBench","Sensitivity profiles, not depth, guide KV cache budgets","RippleKV: Perturbation-based cache allocation beats depth heuristics","RippleKV turns perturbation responses into smarter KV budgets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a layer's output response to norm-scaled random value-noise, measured with the full cache and fixed attention weights, ranks layers the same way that token eviction at the actual target budget does, and that the small calibration set represents the deployment tasks; if either fails, the budgets are misdirected and the reported gains vanish.","fun_headline_variants_meta":{"raw":{"variants":["RippleKV: Perturbation-guided KV cache budgets top LongBench","Sensitivity profiles, not depth, guide KV cache budgets","RippleKV: Perturbation-based cache allocation beats depth heuristics","RippleKV turns perturbation responses into smarter KV budgets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000706,"raw_usage":{"total_tokens":3175,"prompt_tokens":932,"completion_tokens":2243,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":548,"completion_tokens_details":{"reasoning_tokens":2171}},"tokens_in":548,"tokens_out":2243,"duration_ms":18182,"temperature":1.0,"reasoning_tokens":2171,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:28:22.234549+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out model not used in the paper, compute per-layer isolated compression damage $D_\\ell$ (Eq. 1) at a budget such as 5% or 10%, and compare its rank ordering against RippleKV's perturbation profile $s_\\ell$; if the Spearman correlation falls well below 0.799, or if the budget ranking from one budget underperforms uniform allocation on the target budget, the central proxy is falsified.","supporting_citations":[],"review_version":1}