{"id":"cfbbc152-9b4b-404d-b9a2-844143913bc0","arxiv_id":"2506.19549","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"RCStat uses pre-softmax attention logits to define a Relative Contextualization score that improves adaptive KV-cache eviction and attention-head selection for attribution on LLaMA models.","lead":"This paper introduces a statistical score, Relative Contextualization, computed from raw pre-softmax attention logits in large language models. The score is used to pick which attention heads and which key-value cache entries matter, improving KV-cache compression and attribution accuracy without retraining.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"KV-compression eviction is scored against a prompt-window proxy for future generations; if that proxy is unrepresentative, the compression gains do not establish RC as a better relevance signal.","rationale":"The reader's weakest_assumption—that Section 4.1's last-window proxy ĝ may not represent future generations and therefore the threshold c has no principled meaning—is precisely the step I would stress-test. The theoretical bound in Theorem 3.6 is a correct copula-based inequality, and the attribution experiments provide some internal evidence (e.g., Table 3 pre- vs post-softmax RC) that logit-space scores carry signal. But the compression application, which is half of the headline claim, depends on the proxy being representative. The paper itself flags this as a design choice ('Following SnapKV', 'we observe that a small-sized ĝ achieves better performance') without a validation that the proxy distribution matches the true generation distribution. I do not elevate the missing recent baselines (QFilter, PyramidKV, Ada-KV) to the primary concern because those affect the external 'state-of-the-art' claim rather than the internal mechanism; similarly, the token/sentence-level attribution claim is under-supported but would not invalidate the chunk-level results. The proposed oracle test directly separates the proxy effect from the RC effect. Since this concern is addressable and the reader already returned CONDITIONAL, my stress-test does not move the verdict.","tokens_in":109351,"tokens_out":10283,"duration_ms":113472,"concrete_test":"Run a diagnostic oracle-window experiment on a held-out subset of QMSum, 2WikiMultiHop, and SQuAD: compute the Eq. (9) eviction set twice—once with the standard last-window ĝ (w=8 and 16) and once with ĝ replaced by the actual continuation (reference answer or a model-generated continuation). At matched compression ratios, report (i) Jaccard overlap of the two eviction sets, (ii) VER and RougeL-F1 under the oracle policy vs the window policy. If the oracle does not materially reduce VER (e.g., no >0.05 decrease) and eviction sets largely coincide, the proxy concern is resolved; if the oracle improves quality or changes eviction substantially, the reported compression gains are driven by the window proxy, not by RC's relevance signal.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The compression claim in §4.1 rests on Eq. (9): evict token ti when E[Zp(pi,ĝ)] ≤ c E[Zp(p\\ĝ,ĝ)], where ĝ is the last w prompt tokens used as a proxy for the future generation g. The score thus compares each prompt token against the last few prompt tokens, and Yp(ĝ,ĝ) is self-contextualization within the prompt window—not against tokens the model will actually generate. The paper reports that small w works better (§5.2) but provides no evidence that the window distribution matches the generation distribution. For long or multi-hop prompts, relevant tokens may not be well represented by the final window, so the RC scores are computed against the wrong reference distribution and the scalar threshold c has no demonstrated link to the value-fidelity objective in Eq. (8). Under that condition, the improved VER/RougeL frontiers in Figs. 5–6 could reflect a favorable proxy choice rather than the claimed superiority of pre-softmax RC. This is the load-bearing step for the compression half of the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"RCStat formalizes pre-softmax attention logits as random variables for an autoregressive transformer. For a sequence s = p ⊕ g, the paper defines cross-contextualization X_s(p1, g') and self-contextualization Y_s(g1, g') as empirical distributions over logits between specified token subsets, then defines relative contextualization Z_s = max(X_s − Y_s, 0). Theorem 3.6 proves that E[Z] is bounded above by A = ∫ min(F_Y(t), 1 − F_X(t)) dt and below by a = ∫ max(F_Y(t) − F_X(t), 0) dt, using Sklar's theorem and the Fréchet–Hoeffding bounds; Algorithm 1 computes A in Õ(|p||g| + |g|²) time. Two applications are presented. In KV-cache compression (§4.1), a prompt token t_i is evicted when its expected RC against a window ĝ of the last w prompt tokens is at most c times the prompt-level expected RC against ĝ, with ĝ used as a proxy for the unknown future generation. In attribution (§4.2), heads are ranked by sequence-level RC, the top-k are kept, and spans are scored by normalized RC. Experiments on LLaMA-3.2-3B and LLaMA-3.1-8B Instruct report lower Value Error Rate and better RougeL-F1 frontiers than KNorm, SnapKV, StreamingLLM, and TOVA on QMSum, 2WikiMultiHop, and SQuAD v2.0, and 2–3% higher chunk-level attribution accuracy on QuoteSum and VERI-GRAN from selecting the top-20 heads.","tokens_in":109523,"tokens_out":18169,"duration_ms":155311,"significance":"The theoretical core is the paper's strongest part and it is correct: in Appendix B, the proof of Theorem 3.6 validly applies the Fréchet–Hoeffding bounds through Sklar's theorem, and the authors appropriately note that with discrete marginals the copula is not unique and the bounds need not be tight; the same argument yields both the upper and lower bound. Algorithm 1 is a concrete, efficient realization of the bound, and the framework's ability to address both compression and attribution from one statistic is attractive. The paper also includes honest internal controls: Table 2 shows that the independence (IOT) approximation degrades VER, and Appendix Table 3 shows that applying the identical RC-ranking procedure to post-softmax weights loses the attribution gain, which is the key evidence for the pre- vs. post-softmax thesis.","major_comments":[{"comment":"The compression claim stands or falls on the proxy ĝ. Eq. (9) evicts token t_i when E[max(X_p({t_i}, ĝ) − Y_p(ĝ, ĝ), 0)] ≤ c E[Z_p(p\\ĝ, ĝ)], where ĝ is the last w prompt tokens used in place of the future generation g. Each prompt token is therefore scored against the last few prompt tokens rather than against the tokens the model will actually generate, and the paper offers no evidence that the window's query distribution matches the generation distribution: it only reports that w=8 beats w=16. On long or multi-hop prompts (including 2WikiMultiHop, which is in the benchmark suite), relevant tokens may not be well represented by the final window, and in that case the threshold c has no demonstrated connection to the value-fidelity objective in Eq. (8); the improved VER and RougeL-F1 frontiers in Figures 5 and 6 could then reflect a favorable proxy choice rather than the superiority of pre-softmax RC. I ask for (i) a direct comparison between window-based scores and scores computed against truly generated tokens (e.g., the first few decoded tokens) to validate the proxy, and (ii) an ablation with alternative proxies (random prompt window, question-only tokens, full prompt) to show that the gains are attributable to RC rather than to the proxy.","section":"§4.1, Eq. (9)"},{"comment":"The 'state-of-the-art' claim is not supported by the baseline selection. In Section 2 the paper identifies QFilter [22], PyramidKV [6], and Ada-KV [20] as recent variable-budget methods, but none of them appears in the compression experiments of Section 5.2, which compare only KNorm, SnapKV, StreamingLLM, and TOVA. Likewise, the attribution discussion in Section 2 centers on the trained explainer of Cohen-Wang et al. [11], but Table 1 includes no comparison against it. Consequently the abstract's quantitative claims — 'improves generation quality by 15–40%' and 'achieves 2–5% higher compression than prior SOTA' — cannot be traced to any reported number or figure; the figures show VER and RougeL-F1 at matched compression ratios, not a percentage quality improvement or a compression gain at matched quality. Please either add the missing recent baselines or restrict the claims to the methods actually evaluated, and make the abstract numbers derivable from the included experiments.","section":"§5.2 and §2"},{"comment":"The claim of a 'clear anti-correlation' between head-wise compression ratios and RC scores is contradicted by the paper's own data: Example 2 in Figure 4a reports a correlation of −0.01, and the claim rests on only two examples. A correlation indistinguishable from zero in one of two illustrative cases does not support the adaptive head-wise eviction narrative. Please report the distribution of head-wise compression-ratio/RC correlations over a sizable sample of examples (e.g., all 2000 SQuAD or QMSum examples) with mean and spread, and discuss the fraction of examples for which the anti-correlation actually holds.","section":"§5.1, Figure 4a"},{"comment":"The attribution gains are small (93.91 vs. 90.54 on QuoteSum, 79.37 vs. 77.91 on VERI-GRAN) and are reported without error bars, standard deviations, or significance tests, so it is not possible to exclude sampling noise at these margins. In addition, only k=20 is reported in Table 1, and although Figure 7 shows the accuracy-versus-k curve, it carries no uncertainty information; the choice of k should be justified or swept with confidence intervals. The conclusion that RC 'boosts' attribution accuracy should be based on a test (e.g., paired bootstrap over examples) that shows the 2–3% difference is not noise.","section":"§5.3, Table 1"}],"minor_comments":[{"comment":"The sentence 'We define conditional SC similar to conditional CC in Theorem 3.2' should refer to Definition 3.2, not Theorem 3.2.","section":"§3.3, Definition 3.3"},{"comment":"The max(·) in Eq. (9) is missing its second argument; it should read max(X_p(p_i, ĝ) − Y_p(ĝ, ĝ), 0).","section":"§4.1, Eq. (9)"},{"comment":"The citation 'Knorm [33]' points to MiniCache [33]; the K-norm baseline is Devoto et al. [12], which is cited correctly in Section 2. Please fix the inconsistent citation.","section":"§5.2, baseline list"},{"comment":"The sentence beginning 'In fact, even at 80~90% compression an the LLM generates answers...' is grammatically broken and appears to introduce a claim about hallucination ('internal model weights learnt during pre-training') that is not connected to the RougeL ordering discussion. Rewrite or remove it.","section":"§5.2, last paragraph"},{"comment":"The text refers to 'the efficient upper bound (Eq. 11, visualized in Figure 3b)', but Eq. (11) is numbered in the appendix; within the main text this bound appears as Eq. (4). Use a single numbering scheme.","section":"§4.2"},{"comment":"The 'p<0.05' in the task-complexity comparison is reported without any description of the statistical test, and the threshold τ=1.5 used for the '130 vs. 12 heads' count appears to be chosen ad hoc. Please specify the test and the choice of τ.","section":"§5.1"},{"comment":"Appendix G refers to csv files with 'mean and standard deviations' and Appendix A states that code 'is to be made available', but no code or data link is actually included in the manuscript. Since the empirical claims depend on these artifacts, a link to a public repository should be provided.","section":"Appendix A and G"},{"comment":"The complexity notation 'Õ(|p||g|+|g|²) computations, where Õ(n)=O(n log n)' is imprecise: Õ denotes suppression of polylogarithmic factors, not a single function evaluation. Please state the intended meaning.","section":"§3.3"},{"comment":"The formula for the i.i.d. approximation of E[Z|s] contains unbalanced parentheses and brackets; as written it does not parse.","section":"§3.3, i.i.d. approximation"},{"comment":"The x-axis labels such as '(22,0) (13,13)' are not explained in the caption; the caption should state that these are (layer, head) indices of the selected heads.","section":"Figure 4b"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the load-bearing risk is the compression half of the central claim. The window-proxy approximation in §4.1 is acknowledged by the authors but unvalidated, and the baseline set in §5.2 excludes exactly the methods (QFilter, PyramidKV, Ada-KV) that the paper itself describes as the current state of the art; the abstract's '15–40%' and '2–5%' figures are not derivable from the reported experiments. The theory is correct and the pre- vs. post-softmax control in Appendix Table 3 is valuable, so I see this as fixable in revision rather than as a reject. I would also encourage the editor to require the code link promised in Appendix A, since the empirical claims are the manuscript's main deliverable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper is more solid than the typical KV-cache compression submission, and its theoretical core holds up. The RC random variable and the overlap-area bound are new in this literature as far as I can tell, and the proof via Sklar and Fréchet–Hoeffding is correct. The bound is not fitted to downstream metrics; task labels never enter the derivation. The attribution half is the cleaner part of the paper: the full sequence is known up front, the head ranking is a straightforward RC computation, and the gains on QuoteSum and VERI-GRAN are small but consistent. The bottom-k heads dropping to near-random accuracy is a good sanity check that the ranking carries signal.\n\nThe soft spots are real but not disqualifying. The KV eviction rule uses the last prompt tokens as a proxy for future generations, following SnapKV, and the paper never checks whether that proxy matches the generation distribution. If the proxy is unrepresentative, the VER and RougeL frontiers could partly reflect proxy choice rather than RC being a better relevance signal. I would want a direct test, e.g., comparing eviction decisions made with the proxy against decisions made with a held-out sample of actual generated tokens. Second, the paper cites QFilter, PyramidKV, and Ada-KV but does not evaluate against them; that weakens the state-of-the-art claim. Third, no error bars, no code release, and the attribution hyperparameters k and tau are chosen without a stated validation procedure. These are all fixable in revision.\n\nThe citation pattern is fine; the paper attributes the classical identity properly. The upper bound is tight only for continuous CDFs, which the authors acknowledge, and for discrete logits it is still a valid bound, just looser. That is a minor point.\n\nThis is for people working on KV-cache eviction or attention-based attribution. The RC formalism is a genuinely different way to use pre-softmax logits, and the bound makes it cheap to compute. I would send it to peer review and ask the authors to add the missing baselines, error bars, code, and a robustness check on the proxy. It deserves a serious referee.","headline":"A correct copula bound on pre-softmax logits is the real contribution; the KV-compression claims are plausible but rest on an unvalidated prompt-window proxy.","tokens_in":110130,"tokens_out":3424,"would_cite":true,"duration_ms":38670,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that pre-softmax attention logits encode token relevance that softmax normalization obscures, and that a single statistic built from them — Relative Contextualization — can drive both KV-cache compression and attribution…","keywords":["relative contextualization","pre-softmax attention logits","KV cache compression","token attribution","attention head selection","transformer interpretability","LLM inference efficiency","attention sinks"],"falsifier":"Re-run the compression experiments with the reference window $\\hat g$ drawn from the beginning or middle of the prompt, or from the actual generated continuation, and compare Value Error Rate and RougeL-F1 at matched compression ratios; if RCStat no longer dominates SnapKV and TOVA, the claimed benefit is an artifact of the recency proxy rather than of logit-space contextualization.","tokens_in":109104,"feed_emoji":"🧠","tokens_out":8261,"duration_ms":82487,"temperature":0.7,"pith_summary":"The paper argues that softmax-normalized attention weights flatten and distort the token-relevance information that lives in raw pre-softmax query–key logits, and that this lost information matters for two practical tasks. The authors define Relative Contextualization (RC), a random variable that compares how strongly prompt tokens align with generated tokens versus how strongly generated tokens align among themselves, and derive a near-linear-time upper bound for its expectation. RC then serves as a per-head relevance score: KV-cache eviction is driven by a per-token threshold comparing that token's RC with the prompt's overall RC, and attribution is improved by keeping only the top-RC heads. On LLaMA-3.2-3B and LLaMA-3.1-8B across QA and summarization datasets, the paper reports the lowest Value Error Rate at every compression ratio studied and better RougeL-F1 frontiers than post-softmax baselines, plus 2–3% higher attribution accuracy from about 2% of heads, all without retraining. A sympathetic reader should take the central claim as: raw logit-space statistics are a general-purpose, training-free relevance signal for transformer internals.","feed_headline":"LLM relevance lives in pre-softmax logits, not attention weights","feed_subtitle":"RCStat scores prompt tokens by logit overlap to prune KV caches and pick explanatory heads, with no retraining.","key_machinery":"The central object is the Relative Contextualization random variable, $Z_s(p_1,g_1)=\\max(X_s(p_1,g)-Y_s(g_1,g),0)$, where $X$ is the distribution of $\\langle q,k\\rangle$ logits between a prompt subset and the generated tokens and $Y$ is the distribution of logits among generated tokens; large $Z$ means the prompt part is actively grounding the generation in that head. Since $X$ and $Y$ are dependent and their joint law is intractable, the paper proves the bound $\\mathbb{E}[Z]\\le A=\\int_{-\\infty}^{\\infty}\\min(F_Y(t),1-F_X(t))\\,dt$ using copula concentration bounds, and computes $A$ by a Lebesgue-style sweep over sorted logit samples. In the KV-compression application the mechanism is an eviction rule: evict token $t_i$ when $\\mathbb{E}[Z_p(\\{t_i\\},\\hat g)]\\le c\\,\\mathbb{E}[Z_p(p\\setminus\\hat g,\\hat g)]$, with $\\hat g$ a window of the last few prompt tokens standing in for future generation, so each head's cache keeps only tokens whose expected contextual contribution clears the threshold.","core_discovery":"RCStat's central discovery is that the expected surplus of cross-contextualization over self-contextualization—RC, in short—ranks attention heads and prompt tokens more faithfully than post-softmax attention. Concretely, pre-softmax logits from prompt tokens to generated tokens (cross-contextualization) and among generated tokens (self-contextualization) are treated as two distributions; their degree of separation, measured by the overlap area between the self-contextualization CDF and the cross-contextualization survival function, upper-bounds the expected RC and is computable in $~O(|p||g|+|g|^2)$ time. The authors use exact expected RC for eviction decisions and the upper bound for long-context attribution, and report that heads with high RC are anti-correlated with per-head compression ratios, that high-RC heads cluster in middle layers, and that keeping only those heads improves attribution while bottom-ranked heads degrade it sharply.","pith_inferences":["The clearest untested consequence is RC as a hallucination or grounding detector: if low RC measures weak prompt-to-generation alignment, then generations with low RC against their context should be the ones that fabricate content; the authors name this as a future direction, and it is directly checkable on an existing hallucination benchmark.","A stress test on the proxy window would separate contextual signal from recency artifact: re-running eviction with $\\hat g$ taken from the beginning or middle of the prompt, or from the true continuation, should preserve the reported ordering; if it does not, the gains depend on the proxy rather than on RC itself.","The theory suggests an architectural probe: since the upper bound is tight only for continuous CDFs and the paper notes that the discrete case does not guarantee tightness, measuring the gap between exact expected RC and its overlap-area bound per head would show which heads' rankings are most sensitive to the copula bound, and could motivate a sharper estimator."],"forward_implications":["The same scalar $c$, without retraining or per-layer schedules, produces head-specific cache budgets: heads with higher RC retain more keys, which the paper verifies by showing per-head compression ratio is anti-correlated with RC.","Attention attribution can be reduced to a head-selection step: a compact top-RC subset of heads (roughly 2% of all heads) outperforms using all heads on QuoteSum and VERI-GRAN, while the bottom-RC subset performs near chance.","RC scores are sensitive to task difficulty and context relevance: a prompt unrelated to the question activates far fewer high-RC heads than a relevant prompt, and QMSum summaries recruit about nine times more high-RC heads than SQuAD single-hop answers.","Because the framework only touches logits during inference, it is applicable to any auto-regressive transformer with exposed attention logits and can be composed with quantization or low-rank KV methods, which the paper positions as future work."],"supporting_citations":[{"why":"SnapKV supplies the last-window proxy $\\hat g$ for future generation and is the main post-softmax compression baseline.","marker":"[30]"},{"why":"TOVA is the strongest post-softmax eviction baseline in the paper and sets the Value Error Rate numbers RCStat must beat.","marker":"[34]"},{"why":"StreamingLLM contributes the attention-sink baseline and the observation that post-softmax weights concentrate on sink positions.","marker":"[54]"},{"why":"KNorm is the norm-based eviction baseline used in the compression frontier comparisons.","marker":"[33]"},{"why":"Phukan et al. (2024) provides the VERI-GRAN attribution benchmark, the hidden-state attribution baseline, and the head-ranking paradigm RCStat improves.","marker":"[41]"},{"why":"Vallender supplies the area-between-CDF expression for $\\mathbb{E}[\\max(X-Y,0)]$ that Theorem 3.6 extends.","marker":"[50]"},{"why":"Durante and Sempi supplies the copula and Fréchet–Hoeffding bounds used to sandwich the joint term in the proof of Theorem 3.6.","marker":"[17]"}],"fun_headline_variants":["Pre-softmax logits beat attention weights for token relevance","RCStat prunes LLM caches using pre-softmax logit scores","Raw logits, not attention, reveal token importance in LLMs","No-retrain LLM compression and explanation via logit overlap"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The compression claim stands on treating the last few prompt tokens as a proxy $\\hat g$ for the tokens the model will actually generate; if those window tokens are not representative of future generation, the RC scores are computed against the wrong reference distribution and the threshold $c$ no longer has a principled meaning.","fun_headline_variants_meta":{"raw":{"variants":["Pre-softmax logits beat attention weights for token relevance","RCStat prunes LLM caches using pre-softmax logit scores","Raw logits, not attention, reveal token importance in LLMs","No-retrain LLM compression and explanation via logit overlap"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000576,"raw_usage":{"total_tokens":2691,"prompt_tokens":892,"completion_tokens":1799,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":508,"completion_tokens_details":{"reasoning_tokens":1723}},"tokens_in":508,"tokens_out":1799,"duration_ms":13687,"temperature":1.0,"reasoning_tokens":1723,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:32:12.490345+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the compression experiments with the reference window $\\hat g$ drawn from the beginning or middle of the prompt, or from the actual generated continuation, and compare Value Error Rate and RougeL-F1 at matched compression ratios; if RCStat no longer dominates SnapKV and TOVA, the claimed benefit is an artifact of the recency proxy rather than of logit-space contextualization.","supporting_citations":[{"cited_title":"Minicache: Kv cache compression in depth dimension for large language models.Advances in Neural Information Processing Systems, 37:139997–140031, 2024","cited_arxiv_id":null,"evidence_quote":"KNorm is the norm-based eviction baseline used in the compression frontier comparisons."},{"cited_title":"Peering into the mind of language models: An approach for attribution in contextual question answering","cited_arxiv_id":null,"evidence_quote":"Phukan et al. (2024) provides the VERI-GRAN attribution benchmark, the hidden-state attribution baseline, and the head-ranking paradigm RCStat improves."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Vallender supplies the area-between-CDF expression for $\\mathbb{E}[\\max(X-Y,0)]$ that Theorem 3.6 extends."},{"cited_title":"Copula theory: An introduction","cited_arxiv_id":null,"evidence_quote":"Durante and Sempi supplies the copula and Fréchet–Hoeffding bounds used to sandwich the joint term in the proof of Theorem 3.6."}],"review_version":2}