{"id":"c855bf6b-887b-4611-afdc-f27aebdec8b3","arxiv_id":"2610.03135","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Pooling GQA query heads into one entropy-weighted vector before selecting which KV pages survive eliminates the up-to-r-fold union overhead and retains retrieval needles in a small pilot.","lead":"This paper introduces a way to shrink the key-value cache that long-context AI models must keep in memory during decoding, designed around grouped-query attention where several heads share one buffer. It shows that scoring tokens per head forces the engine to keep the union of all heads' choices, inflating cache up to the group ratio, and proposes pooling heads first with entropy weights so exactly the budgeted pages survive.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"The needle-preservation guarantee (Theorem 4/Cor. 2) requires, via Eq. (26), near-uniform sibling heads with T-independent deficit; pilot data show it unmet and the mechanism already degenerate (pooled≈mean, Fig.","rationale":"The reader's weakest_assumption (Theorem 4's head-specialization premise not being reached) is the right location; I agree that CONDITIONAL is the correct verdict and the pre-registered Phase 1 work is the appropriate gate. My stress-test sharpens that diagnosis in two ways the reader did not fully capture. First, the premise failure is not merely a pilot-scale numerical shortfall ('holds directionally') but structurally threatening at the target scale: the operative condition is on sibling heads (Eq. 26), whose residual distributions are far from uniform; nothing in real attention makes sibling sink-isolated entropy approach ln(T−|S_sink|) as T grows, so the implied deficit c_j likely grows with ln T, in which case D(T) never decays and the amplification mechanism of Theorem 4(ii) provably never activates — the guarantee is then vacuous in deployment regardless of how sharp the retrieval head becomes. The paper's own Fig. C.9b (pooled ≈ mean within 1.06× on real text) is direct evidence the mechanism is already inert at T≈2k, and its Section 6.5 reconciliation path (sharper retrieval heads) targets the retrieval head, not the sibling premise that actually drives D(T). Second, I note a tension deserving scrutiny: if pooled and mean needle masses agree within 1.06× yet Table 6a reports 100%-vs-0% retention, the pooled-selection result likely reflects page-boundary effects (the needle rides in a page whose retention is decided by other tokens), so the headline 'needle recall 100%' may not measure what it advertises — consistent with the paper's own disclosure that this criterion is weak. The set-algebra theorems (Proposition 1, Theorem 1, Theorem 3, Propositions 2-4) are elementary and correct; my concern does not touch them. It therefore strengthens the conditionality rather than overturning the verdict, so verdict_should_be stays UNCHANGED (the existing CONDITIONAL stands), with the concrete test providing a decisive measurement of whether the Eq. (26) premise can ever hold at Phase-1 scale.","tokens_in":39590,"tokens_out":9872,"duration_ms":74777,"concrete_test":"On the Phase-1 model (Llama-3.1-8B-Instruct, eager attention capture), record each sibling head's sink-isolated collision entropy across T ∈ {4k, 16k, 64k, 128k} and compute c_j(T) = ln(T−4) − H2^sink(A_{l,j}) per group. If c_j grows with T (saturation, i.e., c_j/ΔlnT tending to a constant rather than to 0), evaluate D(T) = Σ_{j≠h*} e^{c_j/τ_g}(T−4)^{-1/τ_g} with the measured values and τ_g=0.5: if D(T) does not decay well below C_ret ≥ e^{-η_ret/τ_g} at 128k, then Eq. (26)'s premise is structurally empty at deployment scale, Theorem 4/Corollary 2 do not bind, and the '100% vs 0%' needle claim rests solely on the weak pilot criterion. As a confirmation, re-tabulate Table 6a restricting trials to those where the needle page's rank is decided by the needle's own pooled mass (not by other in-page content) to see whether the gap is a mechanism effect or a page-boundary artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The UOR≡1 result (Theorem 3) is sound by construction, so the load-bearing part of the central claim is the needle-preservation guarantee (Theorem 4/Corollary 2) and its advertised empirical face ('100% vs 0%'). Theorem 4 rests entirely on Eq. (26): every sibling head's sink-isolated collision norm must satisfy ∥Ã∥² ≤ e^{c_j}/(T−|S_sink|) with c_j independent of T — the siblings' residuals must be essentially uniform over the non-sink context, with deficits that do not grow with context. Real attention is not uniform and does not become more uniform as T grows. The paper's own pilot already shows the premise unmet: retrieval-head mass 1−ε≈0.11–0.17 versus the assumed near-1 regime, sink-isolated retrieval entropy 2.6–2.8 nats versus ln T≈7.6 at T≈2k (Section 6.4), and decisively Fig. C.9b shows entropy-weighted pooling degenerating to arithmetic-mean pooling (pooled≈mean within 1.06×) on real text. If sibling sink-isolated entropy saturates while ln T→∞, then the implied c_j(T) grows like ln T, D(T) = Σ_j e^{c_j/τ_g}(T−4)^{-1/τ_g} in Corollary 2 does not vanish, the amplification w*→1 (Theorem 4 ii) never activates, and Δ(T) is O(1), not 10^-9. The Phase-1 reconciliation path stated in Section 6.5 ('retrieval heads are sharper') addresses only the retrieval head, not the sibling side of Eq. (26) — the side that makes the bound bind. Additionally, the advertised '100% vs 0%' needle recall (Section 6.4, Table 6a) is measured under the weak pooled-selection criterion on pages whose survival is driven by in-page pooled mass other than the ~10^-3 needle (Fig. C.9b reports pooled needle mass ~10^-3 with pooled≈mean), so the gap may be a boundary effect of page-level ranking rather than mechanism-level preservation of the needle. None of this questions the set-algebra results, which are correct; it targets the practically advertised needle claim, which is premise-gated in a way the current evidence suggests never binds at the 4k–128k scale where it is advertised.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper formalizes KV-cache eviction at the physical-group granularity of GQA serving engines. It defines the union overhead ratio (UOR) and a Jaccard disagreement index, proves an exact UOR–disagreement identity for two-head groups (Proposition 1) and two-sided inflation bounds for general ratios (Theorem 1), and proposes entropy-weighted group pooling using sink-isolated Rényi-2 collision entropy (Theorem 2), with page-block mapping and budget-exact selection (Theorem 3, Propositions 3–4). A conditional finite-context needle-retention result (Theorem 4, Corollary 2) shows that entropy-weighted pooling preserves a non-sink needle when one retrieval head carries mass 1−ε and all siblings have near-maximal sink-isolated entropy, while arithmetic mean pooling asymptotes to (1−ε)/r. The pilot on Qwen2.5-1.5B-Instruct (r=6) reports UOR≡1.000 across 1,008 pooled measurements, a sink-to-context weight inversion (2.10×→0.66×), and 100% versus 0% needle retention under the pooled-selection criterion at a 20% budget; strict all-layer retention fails at depths 25–75%, which the authors report as a limitation. The full 4k–128k sweep, larger models, and serving measurements are pre-registered and pending.","tokens_in":40255,"tokens_out":20019,"duration_ms":221885,"significance":"If the structural results stand, the paper makes a clean and useful formal contribution: it identifies a real systems inefficiency (union inflation under per-head eviction with shared GQA buffers), measures it, and packages the fix as a small set of elementary, fully derived results. The proofs are checkable and the appendix derivations are complete; the pilot logs and reference implementation are released, and the paper is unusually candid — Theorem 3's UOR≡1 is honestly labeled as holding by construction, the strict-criterion needle failures are reported rather than hidden, and Fig. C.9 deliberately pairs a premise-satisfying synthetic check with a premise-violating real-text check. The load-bearing piece is Theorem 4/Corollary 2: it is conditional on head specialization (Eq. 26), the pilot shows the premise is only directionally met, and the mechanism degenerates toward mean pooling on real text; the paper's own framing (a formal framework with pilot-validated structural core, not a validated serving system) is accurate. Whether Corollary 2 binds at deployment scale is an open, falsifiable question.","major_comments":[{"comment":"Theorem 4 and Corollary 2 are stated for sink-isolated entropies over the full non-sink sequence, but Algorithm 1 (line 4) and Definition 5 compute the pooling weights from the observation window W_max=4096; since the windowed distribution lives on at most W_max tokens, its collision entropy is bounded by ln W_max, so for T>W_max the windowed statistic cannot satisfy Eq. (26)'s ln(T−|S_sink|) growth and the asymptotic amplification of Theorem 4(ii) is not certified by Corollary 2 for the mechanism as specified. Remark 4 and Table C.10 instantiate the bound at T=131,072 using full-sequence entropies, and Fig. C.9a reports that the 'full engine path' reproduces the bound to 3×10^{-7} at 128k; the paper must state which entropy the engine-path check used, and must either restate Theorem 4/Corollary 2 for the windowed statistic (with W_obs in place of T, turning Δ(T) into a constant-in-T deficit) or justify full-sequence entropy in the deployed kernels. The pilot's needle runs with T≤2.5k do not exercise this regime, but the advertised 128k deployment does.","section":"§4.1 (Def. 5), §5 (Thm. 4, Eq. 26), §C.9 (Fig. C.9a)","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The binding side of Theorem 4's premise is the sibling condition of Eq. (26): every head j≠h* must have H_sink_2(A_{l,j}) ≥ ln(T−|S_sink|) − c_j with T-independent c_j. The pilot does not support this premise: Section 6.4 reports sink-isolated retrieval entropy of 2.6–2.8 nats at T≈2k (versus ln T≈7.6), and Table 5b shows the measured α=2 entropy range topping out near 5.9 nats in the certified runs up to 8.2k, implying deficits of at least 1.7–3.1 nats that already exceed any plausible fixed c_j. If sibling entropy saturates while ln T grows, the implied c_j(T) grows like ln T, D(T) in Corollary 2 does not vanish, Δ(T) stays O(1), and the w*→1 amplification of Theorem 4(ii) never activates. Section 6.5's reconciliation item targets retrieval-head sharpness (1−ε large) but contains no planned measurement of sibling entropy scaling in T, so the stated path to 'reconcile the asymptotic guarantee with deployed budgets at Phase-1 scale' lacks its quantitative target; add to Phase 1 a measurement of H_sink_2(T) for sibling heads across T and a sensitivity analysis of Δ(T) under T-dependent c_j. Fig. C.9b is the pilot-scale face of the same issue: entropy-weighted pooling there agrees with mean pooling to within a factor 1.06 on real text.","section":"§5 (Eq. 26), §6.3–6.4, §6.5","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The headline claim 'needle recall is 100% versus 0% for mean pooling at a 20% budget' appears in the abstract without its success-criterion qualifier, while Table 6a defines it under the pooled-selection criterion (the needle's page must survive the pooled eviction decision, with no cross-layer requirement) and Table 6b shows that under the strict all-layer criterion both methods score 0.00 at depths 25–75% and both trivially score 1.00 at depth 100%. Under the paper's own 'claims follow tables' governance, every occurrence of the 100%-versus-0% claim (abstract, introduction, conclusion) should carry the phrasing 'under the pooled-selection criterion', and the abstract should note that strict all-layer retention fails at pilot scale; as written, the unqualified headline materially overstates what Table 6 supports.","section":"Abstract; §6.4 (Table 6)","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The two abstract versions disagree on headline numbers: 'up to 4.75x at a 2% budget' versus 'up to 5.1×' in Table 4a, and 'a 13x sink masquerade' has no counterpart in the body, which reports a 2.10×→0.66× sink-to-context inversion (Section 6.3, Fig. 8); reconcile every abstract number with the certified logs.","section":"Abstract and §6.2/§6.3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The merged '75%/100%' column conflates a failing depth (0.00) with a trivially passing one (1.00); splitting the two depths would make the strict-criterion failure pattern much easier to read.","section":"Table 6b","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The title of the ThinK reference contains 'Kertia', presumably a typo for 'Key-Value'; please verify it against the published version.","section":"Reference [20]","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The phrase 'an 800% memory expansion for r=8 architectures' is loose: the bound is 8× the per-head budget, i.e., a 700% increase over budget; rephrase to avoid the factor-versus-percent confusion.","section":"§4.2","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"Colleague,\n\nThe useful part of this paper is the union-overhead formalization. For GQA serving, the observation that head-independent top-k selection forces the engine to retain the union of per-head choices is real, and the exact r=2 identity (Prop 1) plus the sandwich bounds of Theorem 1 are correct, elementary set algebra — I checked the Cauchy–Schwarz and Jensen steps and they hold. The UOR≡1 result (Theorem 3) is a tautology, but an honestly labeled one, and it is genuinely useful: Ada-KV works the budget side and EntroKV the layer-budget side; nobody else attacks the union at the selection point and maps it onto page frames. That is the contribution, and it survives scrutiny. The budget-conservation proposition with Hamilton rounding is also clean.\n\nThe paper is worth reading for its reporting discipline. It discloses, unprompted, that strict all-layer needle retention fails at pilot scale, that the Theorem 4 premise is only directionally met (measured needle mass 0.11–0.17, not near 1), that horizons top out at 2.5k, that there is no uncompressed baseline, and that Fig. C.9b shows entropy-weighted pooling degenerating to within 1.06× of mean pooling on real text. Imagined or fabricated red flags: none.\n\nWhere it slips is the abstract and intro: \"needle recall 100% versus 0% at a 20% budget\" appears without the pooled-selection-criterion caveat that only shows up in Section 6.4. Under the strict all-layer criterion both methods fail at depths 25–75%. The 100-vs-0 gap is plausibly a page-ranking boundary effect — pooled needle mass is ~1e-3 — rather than mechanism-level preservation. And the stress-test point on Eq. (26) stands: the bound requires sibling heads to be near-uniform over non-sink positions with deficits that don't grow with T. Real attention isn't uniform, and the Phase-1 reconciliation (\"retrieval heads will be sharper\") addresses only the retrieval head, not the sibling side that makes the bound bind. I'll caveat one thing: the stress-test says the guarantee \"never binds at the 4k–128k scale where it is advertised,\" but the paper doesn't actually claim it binds there — that sweep is pre-registered and pending. The genuine defect is narrower: the abstract presenting a weak-criterion empirical result as headline without its condition.\n\nBottom line: the structural/memory argument is solid, novel in framing, and pilot-verified at the level of exact cardinality and sink-isolation ordering. The needle claim should be demoted to a conditional result with the criterion caveat in the abstract. This deserves a serious referee with a request for a rebalanced abstract, not a desk reject. I would cite the UOR formalism in my own serving work.","headline":"The union-overhead formalism is correct and genuinely new; the needle-retention headline is premise-gated and the pilot never reaches the regime where the advertised guarantee binds — the paper knows this, but the abstract outruns the evidence.","tokens_in":40802,"tokens_out":2846,"duration_ms":21423},"pith_extraction":{"msc":["68T07","94A17"],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"Pooling sibling attention heads into one vector before selecting what to keep removes GQA's union-overhead inflation exactly — UOR ≡ 1.0 by construction — while entropy weighting preserves factual needles that arithmetic-mean pooling…","keywords":["grouped-query attention","KV-cache eviction","PagedAttention","Rényi-2 collision entropy","attention sinks","needle retention","union overhead ratio","entropy-weighted pooling"],"falsifier":"Run the pre-registered phase-1 needle sweep (Llama-3.1-8B, r=4, contexts 4k–128k, depths 0–100%) and measure the retrieval head's non-sink mass $1-\\epsilon$ and the pooled needle mass under the strict all-layer criterion: if needle mass stays near 0.1–0.2 with pooled mass near $10^{-3}$, the Theorem 4 premise fails at scale and the needle-retention claim must be downgraded to the pooled-selection sense; if $1-\\epsilon$ approaches 1 at long context, Corollary 2's deficit bound ($\\le 1.6\\times10^{-9}$ at 128k) predicts retention in every layer and the pilot failures are a premise gap, not a framework gap.","tokens_in":41143,"feed_emoji":"🎯","tokens_out":11509,"duration_ms":121226,"temperature":0.7,"pith_summary":"Page-EntroKV claims that the memory inflation built into grouped-query attention (GQA) eviction can be removed exactly, not approximately: pool each physical group's heads into one attention vector first, then evict from that single vector. Because a weighted average of probability vectors is itself a probability vector, selecting the top-k from it retains exactly k entries for every attention pattern, so the union overhead ratio (UOR) is identically 1.0 by construction (Theorem 3). The paper pairs this structural identity with a calibration-free weighting: heads are pooled by Boltzmann weights built from sink-isolated Rényi-2 collision entropy, one inner product per head evaluated once at prefill, which keeps attention-sink heads from masquerading as retrieval heads and lets a single sharp head's factual needle survive where arithmetic-mean pooling dilutes it by the factor 1−1/r (Theorems 2 and 4). A pilot on Qwen2.5-1.5B-Instruct (r=6) reproduces the structural predictions — measured union inflation up to 5.1× under head-independent replay versus exactly 1.000 under pooling, needle retention 100% versus 0% for mean pooling at a 20% budget — while the strict all-layer needle guarantee is openly reported as failing at pilot scale because the theorem's strong premise (needle mass near 1) is not reached. If the framework holds at scale, long-context GQA serving can evict at page granularity with zero union overhead and without sacrificing the tokens that carry factual recall.","feed_headline":"Pooling before eviction cuts GQA cache overhead to exactly 1.0","feed_subtitle":"Sibling heads stop forcing the union of their picks, so factual needles survive where mean pooling drops them.","key_machinery":"The load-bearing object is the pooled-before-selection group vector $A^{(l,g)}_{\\mathrm{pooled}}(t) = \\sum_{h \\in \\mathrm{Group}(g)} w_{l,h} A_{l,h}(t)$ with Boltzmann weights $w_{l,h} \\propto \\exp(-H^{\\mathrm{sink}}_2(A_{l,h})/\\tau_g)$, where $H^{\\mathrm{sink}}_2(A_{l,h}) = -\\ln \\|\\tilde{A}_{l,h}\\|_2^2$ is the sink-isolated Rényi-2 collision entropy — a single inner product per head, computed once at prefill over a 4,096-token observation window with no calibration. Sink isolation renormalizes attention off the first four (sink) positions, so a head that parks its mass on sink tokens cannot masquerade as a sharp retrieval head; its weight decays as $(T-4)^{-1/\\tau_g}$ (Theorem 2). Because pooling preserves the simplex, top-k selection on the pooled vector has exact cardinality — the entire content of Theorem 3 — and a block-max reduction projects the pooled vector onto PagedAttention page frames so eviction runs at the hardware tuple (layer, group, page). The supporting identities are the exact UOR–disagreement relation UOR $= 2/(2-D)$ for two-head groups and the Cauchy–Schwarz and Jensen sandwich bounds at general group ratios (Proposition 1, Theorem 1).","core_discovery":"The paper's central claim is that GQA's shared physical KV buffer fixes the correct granularity for eviction, and that head-level selection is structurally wrong: sibling heads sharing a buffer force the engine to retain the union of their top-k sets, inflating memory by up to the group ratio r. Page-EntroKV therefore pools before selecting — the group's pooled vector $A^{(l,g)}_{\\mathrm{pooled}}(t) = \\sum_h w_{l,h} A_{l,h}(t)$ is a convex combination of probability vectors and hence itself a probability vector, so top-k selection on it has exact cardinality k and UOR ≡ 1.0 for every attention pattern (Theorem 3). Around that identity the paper proves an exact UOR–disagreement relation for two-head groups and two-sided sandwich bounds at every ratio (Proposition 1, Theorem 1), characterizes the Boltzmann pooling family whose temperature limits interpolate from one-hot on the lowest-entropy head to arithmetic mean (Proposition 2), proves sink-isolated collision entropy suppresses sink heads at rate $(T-4)^{-1/\\tau_g}$ while retrieval heads keep weight bounded away from zero (Theorem 2), and derives a finite-context needle-retention bound in which entropy-weighted pooling preserves needle mass $1-\\epsilon$ while arithmetic-mean pooling collapses it to $(1-\\epsilon)/r$ (Theorem 4, Corollary 2). The pilot validates the structural core — 1,008 pooled measurements at UOR exactly 1.000, sink-to-context weight ordering inverted from 2.10× to 0.66×, and 100% versus 0% needle retention under the pooled-selection criterion — and the paper states plainly that strict all-layer retention fails at pilot scale because the measured retrieval-head mass (0.11–0.17) is far from the premise's near-1 regime.","pith_inferences":["My inference: because Theorem 3's budget guarantee depends only on pooling-before-selection and not on how the weights are chosen, the exact UOR≡1 result would survive any group aggregator — max-over-heads, mean, or any learned scorer; only the needle-retention quality, not the memory bound, is at stake.","My inference: the paper's own pilot makes the Theorem 4 premise the decisive open question — if phase-1 models at 128k context keep measured retrieval-head mass near 0.1–0.2 rather than near 1, the strict all-layer needle guarantee will still not bind, and the needle-retention claim holds only in the weaker pooled-selection sense that the paper already reports.","My inference: the exact UOR–disagreement identity doubles as a production diagnostic — an operator can measure pairwise Jaccard disagreement among sibling heads during serving and immediately compute the union inflation any head-independent policy would incur, quantifying what pooling saves.","My inference: the temperature family suggests a cheap runtime safeguard — when within-group weight dispersion collapses toward the $1/r$ uniform limit, the group is visibly degenerating toward mean pooling, and switching that group to max-over-heads restores needle sensitivity without violating the UOR≡1 budget guarantee (Proposition 2)."],"forward_implications":["Under every attention pattern, every temperature, and every page size, pooled group-level eviction retains exactly the budgeted number of tokens, so UOR ≡ 1.0 holds by construction and needs no calibration or tuning.","Sibling-head disagreement becomes cost-free: the union inflation measured up to 5.1× under head-independent replay disappears without any budget reallocation, because selection happens on one pooled vector.","Factual needles survive where mean pooling loses them: at a 20% budget the needle's page is retained in 100% of pilot trials under the pooled-selection criterion versus 0% for arithmetic-mean pooling, since entropy weighting amplifies the one sharp retrieval head instead of dividing its mass by r.","The compression composes with channel-dimension quantization: a 20% page budget (5× along the sequence axis) multiplies with 4-bit KV storage (4× along the channel axis) for roughly 20× combined reduction, with the pooling logic unaffected by storage precision.","Eviction accounting becomes hardware-exact: per-layer page unions are bounded between $K_l$ and $H_{\\mathrm{KV}} K_l$ pages, and the fused on-chip kernels add no DRAM traffic beyond the attention computation that produces the per-head distributions."],"supporting_citations":[{"why":"Defines grouped-query attention — the splitting of H_Q query heads into H_KV physical groups with ratio r — whose shared buffers create the union-inflation problem the paper formalizes.","marker":"[3]"},{"why":"Specifies the PagedAttention block table and physical KV layout (blocks, H_KV, B, D_head) that Page-EntroKV aligns its eviction tuple (layer, group, page) to.","marker":"[7]"},{"why":"Supplies the attention-sink phenomenon and the convention of treating the first four positions as sinks, which sink isolation builds on for its renormalization.","marker":"[13]"},{"why":"Establishes that specialized retrieval heads mechanistically carry long-context factual recall, the premise for preserving those heads from dilution by pooling.","marker":"[24]"},{"why":"The budget-side fix for GQA union inflation (adaptive per-head quotas within a group) that Page-EntroKV contrasts with its selection-side pooling guarantee.","marker":"[17]"},{"why":"The concurrent entropy-based method (per-layer budget allocation, offline calibration) from which Page-EntroKV distinguishes its calibration-free, selection-side use of entropy.","marker":"[18]"},{"why":"Provides the observation-window convention (final W_max = 4096 tokens) over which prefill entropies are computed once.","marker":"[11]"}],"fun_headline_variants":["Pool first, evict later: GQA cache overhead hits exactly 1.0","Entropy-weighted pooling kills GQA's union bloat, UOR=1","Sink-proof pooling keeps GQA cache lean, needles intact","Group-pooled eviction trims GQA cache, UOR stays 1.000","For GQA, pool before you evict to avoid 4.75x waste"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The needle-retention guarantee of Theorem 4 rests on the premise that one head in a group concentrates mass near 1 on a single non-sink factual token while every sibling head has near-maximal sink-isolated entropy — a regime the pilot does not reach, since measured needle mass is only about 0.11–0.17 with sink-isolated retrieval entropy near 2.6–2.8 nats, and where the premise fails the paper accepts that strict all-layer retention fails at depths 25–75%.","fun_headline_variants_meta":{"raw":{"variants":["Pool first, evict later: GQA cache overhead hits exactly 1.0","Entropy-weighted pooling kills GQA's union bloat, UOR=1","Sink-proof pooling keeps GQA cache lean, needles intact","Group-pooled eviction trims GQA cache, UOR stays 1.000","For GQA, pool before you evict to avoid 4.75x waste"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000204,"raw_usage":{"cost":0.000204008,"is_byok":false,"cost_details":{"upstream_inference_cost":0.000204008,"upstream_inference_prompt_cost":0.000155288,"upstream_inference_completions_cost":4.872e-05},"total_tokens":1488,"prompt_tokens":1314,"completion_tokens":174,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":256,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":66}},"tokens_in":1314,"tokens_out":174,"cost_source":"openrouter_usage","duration_ms":2696,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":66,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-10-05T12:23:09.946607+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"Run the pre-registered phase-1 needle sweep (Llama-3.1-8B, r=4, contexts 4k–128k, depths 0–100%) and measure the retrieval head's non-sink mass $1-\\epsilon$ and the pooled needle mass under the strict all-layer criterion: if needle mass stays near 0.1–0.2 with pooled mass near $10^{-3}$, the Theorem 4 premise fails at scale and the needle-retention claim must be downgraded to the pooled-selection sense; if $1-\\epsilon$ approaches 1 at long context, Corollary 2's deficit bound ($\\le 1.6\\times10^{-9}$ at 128k) predicts retention in every layer and the pilot failures are a premise gap, not a framework gap.","supporting_citations":[{"cited_title":"Ainslie, J","cited_arxiv_id":null,"evidence_quote":"Defines grouped-query attention — the splitting of H_Q query heads into H_KV physical groups with ratio r — whose shared buffers create the union-inflation problem the paper formalizes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Specifies the PagedAttention block table and physical KV layout (blocks, H_KV, B, D_head) that Page-EntroKV aligns its eviction tuple (layer, group, page) to."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the attention-sink phenomenon and the convention of treating the first four positions as sinks, which sink isolation builds on for its renormalization."},{"cited_title":"Retrieval Head Mechanistically Explains Long-Context Factuality","cited_arxiv_id":"2404.15574","evidence_quote":"Establishes that specialized retrieval heads mechanistically carry long-context factual recall, the premise for preserving those heads from dilution by pooling."},{"cited_title":"Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference","cited_arxiv_id":"2407.11550","evidence_quote":"The budget-side fix for GQA union inflation (adaptive per-head quotas within a group) that Page-EntroKV contrasts with its selection-side pooling guarantee."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The concurrent entropy-based method (per-layer budget allocation, offline calibration) from which Page-EntroKV distinguishes its calibration-free, selection-side use of entropy."},{"cited_title":"SnapKV: LLM Knows What You are Looking for Before Generation","cited_arxiv_id":"2404.14469","evidence_quote":"Provides the observation-window convention (final W_max = 4096 tokens) over which prefill entropies are computed once."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}