Page-EntroKV: Hardware-Aligned, Entropy-Weighted KV-Cache Eviction under Grouped-Query Attention

The useful part of this paper is the union-overhead formalization. For GQA serving, the observation that head-independent top-k selection forces the engine to retain the union of per-head choices is real, and the exact r=2 identity (Prop 1) plus the sandwich bounds of Theorem 1 are correct, elementary set algebra — I checked the Cauchy–Schwarz and Jensen steps and they hold. The UOR≡1 result (Theorem 3) is a tautology, but an honestly labeled one, and it is genuinely useful: Ada-KV works the budget side and EntroKV the layer-budget side; nobody else attacks the union at the selection point and maps it onto page frames. That is the contribution, and it survives scrutiny. The budget-conservation proposition with Hamilton rounding is also clean.

The paper is worth reading for its reporting discipline. It discloses, unprompted, that strict all-layer needle retention fails at pilot scale, that the Theorem 4 premise is only directionally met (measured needle mass 0.11–0.17, not near 1), that horizons top out at 2.5k, that there is no uncompressed baseline, and that Fig. C.9b shows entropy-weighted pooling degenerating to within 1.06× of mean pooling on real text. Imagined or fabricated red flags: none.

Where it slips is the abstract and intro: "needle recall 100% versus 0% at a 20% budget" appears without the pooled-selection-criterion caveat that only shows up in Section 6.4. Under the strict all-layer criterion both methods fail at depths 25–75%. The 100-vs-0 gap is plausibly a page-ranking boundary effect — pooled needle mass is ~1e-3 — rather than mechanism-level preservation. And the stress-test point on Eq. (26) stands: the bound requires sibling heads to be near-uniform over non-sink positions with deficits that don't grow with T. Real attention isn't uniform, and the Phase-1 reconciliation ("retrieval heads will be sharper") addresses only the retrieval head, not the sibling side that makes the bound bind. I'll caveat one thing: the stress-test says the guarantee "never binds at the 4k–128k scale where it is advertised," but the paper doesn't actually claim it binds there — that sweep is pre-registered and pending. The genuine defect is narrower: the abstract presenting a weak-criterion empirical result as headline without its condition.

Bottom line: the structural/memory argument is solid, novel in framing, and pilot-verified at the level of exact cardinality and sink-isolation ordering. The needle claim should be demoted to a conditional result with the criterion caveat in the abstract. This deserves a serious referee with a request for a rebalanced abstract, not a desk reject. I would cite the UOR formalism in my own serving work.

The UOR≡1 result (Theorem 3) is sound by construction, so the load-bearing part of the central claim is the needle-preservation guarantee (Theorem 4/Corollary 2) and its advertised empirical face ('100% vs 0%'). Theorem 4 rests entirely on Eq. (26): every sibling head's sink-isolated collision norm must satisfy ∥Ã∥² ≤ e^{c_j}/(T−|S_sink|) with c_j independent of T — the siblings' residuals must be essentially uniform over the non-sink context, with deficits that do not grow with context. Real attention is not uniform and does not become more uniform as T grows. The paper's own pilot already shows the premise unmet: retrieval-head mass 1−ε≈0.11–0.17 versus the assumed near-1 regime, sink-isolated retrieval entropy 2.6–2.8 nats versus ln T≈7.6 at T≈2k (Section 6.4), and decisively Fig. C.9b shows entropy-weighted pooling degenerating to arithmetic-mean pooling (pooled≈mean within 1.06×) on real text. If sibling sink-isolated entropy saturates while ln T→∞, then the implied c_j(T) grows like ln T, D(T) = Σ_j e^{c_j/τ_g}(T−4)^{-1/τ_g} in Corollary 2 does not vanish, the amplification w*→1 (Theorem 4 ii) never activates, and Δ(T) is O(1), not 10^-9. The Phase-1 reconciliation path stated in Section 6.5 ('retrieval heads are sharper') addresses only the retrieval head, not the sibling side of Eq. (26) — the side that makes the bound bind. Additionally, the advertised '100% vs 0%' needle recall (Section 6.4, Table 6a) is measured under the weak pooled-selection criterion on pages whose survival is driven by in-page pooled mass other than the ~10^-3 needle (Fig. C.9b reports pooled needle mass ~10^-3 with pooled≈mean), so the gap may be a boundary effect of page-level ranking rather than mechanism-level preservation of the needle. None of this questions the set-algebra results, which are correct; it targets the practically advertised needle claim, which is premise-gated in a way the current evidence suggests never binds at the 4k–128k scale where it is advertised.

The reader's weakest_assumption (Theorem 4's head-specialization premise not being reached) is the right location; I agree that CONDITIONAL is the correct verdict and the pre-registered Phase 1 work is the appropriate gate. My stress-test sharpens that diagnosis in two ways the reader did not fully capture. First, the premise failure is not merely a pilot-scale numerical shortfall ('holds directionally') but structurally threatening at the target scale: the operative condition is on sibling heads (Eq. 26), whose residual distributions are far from uniform; nothing in real attention makes sibling sink-isolated entropy approach ln(T−|S_sink|) as T grows, so the implied deficit c_j likely grows with ln T, in which case D(T) never decays and the amplification mechanism of Theorem 4(ii) provably never activates — the guarantee is then vacuous in deployment regardless of how sharp the retrieval head becomes. The paper's own Fig. C.9b (pooled ≈ mean within 1.06× on real text) is direct evidence the mechanism is already inert at T≈2k, and its Section 6.5 reconciliation path (sharper retrieval heads) targets the retrieval head, not the sibling premise that actually drives D(T). Second, I note a tension deserving scrutiny: if pooled and mean needle masses agree within 1.06× yet Table 6a reports 100%-vs-0% retention, the pooled-selection result likely reflects page-boundary effects (the needle rides in a page whose retention is decided by other tokens), so the headline 'needle recall 100%' may not measure what it advertises — consistent with the paper's own disclosure that this criterion is weak. The set-algebra theorems (Proposition 1, Theorem 1, Theorem 3, Propositions 2-4) are elementary and correct; my concern does not touch them. It therefore strengthens the conditionality rather than overturning the verdict, so verdict_should_be stays UNCHANGED (the existing CONDITIONAL stands), with the concrete test providing a decisive measurement of whether the Eq. (26) premise can ever hold at Phase-1 scale.

If the structural results stand, the paper makes a clean and useful formal contribution: it identifies a real systems inefficiency (union inflation under per-head eviction with shared GQA buffers), measures it, and packages the fix as a small set of elementary, fully derived results. The proofs are checkable and the appendix derivations are complete; the pilot logs and reference implementation are released, and the paper is unusually candid — Theorem 3's UOR≡1 is honestly labeled as holding by construction, the strict-criterion needle failures are reported rather than hidden, and Fig. C.9 deliberately pairs a premise-satisfying synthetic check with a premise-violating real-text check. The load-bearing piece is Theorem 4/Corollary 2: it is conditional on head specialization (Eq. 26), the pilot shows the premise is only directionally met, and the mechanism degenerates toward mean pooling on real text; the paper's own framing (a formal framework with pilot-validated structural core, not a validated serving system) is accurate. Whether Corollary 2 binds at deployment scale is an open, falsifiable question.

Page-EntroKV claims that the memory inflation built into grouped-query attention (GQA) eviction can be removed exactly, not approximately: pool each physical group's heads into one attention vector first, then evict from that single vector. Because a weighted average of probability vectors is itself a probability vector, selecting the top-k from it retains exactly k entries for every attention pattern, so the union overhead ratio (UOR) is identically 1.0 by construction (Theorem 3). The paper pairs this structural identity with a calibration-free weighting: heads are pooled by Boltzmann weights built from sink-isolated Rényi-2 collision entropy, one inner product per head evaluated once at prefill, which keeps attention-sink heads from masquerading as retrieval heads and lets a single sharp head's factual needle survive where arithmetic-mean pooling dilutes it by the factor 1−1/r (Theorems 2 and 4). A pilot on Qwen2.5-1.5B-Instruct (r=6) reproduces the structural predictions — measured union inflation up to 5.1× under head-independent replay versus exactly 1.000 under pooling, needle retention 100% versus 0% for mean pooling at a 20% budget — while the strict all-layer needle guarantee is openly reported as failing at pilot scale because the theorem's strong premise (needle mass near 1) is not reached. If the framework holds at scale, long-context GQA serving can evict at page granularity with zero union overhead and without sacrificing the tokens that carry factual recall.

What carries the argument. The load-bearing object is the pooled-before-selection group vector $A^{(l,g)}_{\mathrm{pooled}}(t) = \sum_{h \in \mathrm{Group}(g)} w_{l,h} A_{l,h}(t)$ with Boltzmann weights $w_{l,h} \propto \exp(-H^{\mathrm{sink}}_2(A_{l,h})/\tau_g)$, where $H^{\mathrm{sink}}_2(A_{l,h}) = -\ln \|\tilde{A}_{l,h}\|_2^2$ is the sink-isolated Rényi-2 collision entropy — a single inner product per head, computed once at prefill over a 4,096-token observation window with no calibration. Sink isolation renormalizes attention off the first four (sink) positions, so a head that parks its mass on sink tokens cannot masquerade as a sharp retrieval head; its weight decays as $(T-4)^{-1/\tau_g}$ (Theorem 2). Because pooling preserves the simplex, top-k selection on the pooled vector has exact cardinality — the entire content of Theorem 3 — and a block-max reduction projects the pooled vector onto PagedAttention page frames so eviction runs at the hardware tuple (layer, group, page). The supporting identities are the exact UOR–disagreement relation UOR $= 2/(2-D)$ for two-head groups and the Cauchy–Schwarz and Jensen sandwich bounds at general group ratios (Proposition 1, Theorem 1).

Core claim. The paper's central claim is that GQA's shared physical KV buffer fixes the correct granularity for eviction, and that head-level selection is structurally wrong: sibling heads sharing a buffer force the engine to retain the union of their top-k sets, inflating memory by up to the group ratio r. Page-EntroKV therefore pools before selecting — the group's pooled vector $A^{(l,g)}_{\mathrm{pooled}}(t) = \sum_h w_{l,h} A_{l,h}(t)$ is a convex combination of probability vectors and hence itself a probability vector, so top-k selection on it has exact cardinality k and UOR ≡ 1.0 for every attention pattern (Theorem 3). Around that identity the paper proves an exact UOR–disagreement relation for two-head groups and two-sided sandwich bounds at every ratio (Proposition 1, Theorem 1), characterizes the Boltzmann pooling family whose temperature limits interpolate from one-hot on the lowest-entropy head to arithmetic mean (Proposition 2), proves sink-isolated collision entropy suppresses sink heads at rate $(T-4)^{-1/\tau_g}$ while retrieval heads keep weight bounded away from zero (Theorem 2), and derives a finite-context needle-retention bound in which entropy-weighted pooling preserves needle mass $1-\epsilon$ while arithmetic-mean pooling collapses it to $(1-\epsilon)/r$ (Theorem 4, Corollary 2). The pilot validates the structural core — 1,008 pooled measurements at UOR exactly 1.000, sink-to-context weight ordering inverted from 2.10× to 0.66×, and 100% versus 0% needle retention under the pooled-selection criterion — and the paper states plainly that strict all-layer retention fails at pilot scale because the measured retrieval-head mass (0.11–0.17) is far from the premise's near-1 regime.

The premise it rests on. The needle-retention guarantee of Theorem 4 rests on the premise that one head in a group concentrates mass near 1 on a single non-sink factual token while every sibling head has near-maximal sink-isolated entropy — a regime the pilot does not reach, since measured needle mass is only about 0.11–0.17 with sink-isolated retrieval entropy near 2.6–2.8 nats, and where the premise fails the paper accepts that strict all-layer retention fails at depths 25–75%.

Under every attention pattern, every temperature, and every page size, pooled group-level eviction retains exactly the budgeted number of tokens, so UOR ≡ 1.0 holds by construction and needs no calibration or tuning.

Sibling-head disagreement becomes cost-free: the union inflation measured up to 5.1× under head-independent replay disappears without any budget reallocation, because selection happens on one pooled vector.

Factual needles survive where mean pooling loses them: at a 20% budget the needle's page is retained in 100% of pilot trials under the pooled-selection criterion versus 0% for arithmetic-mean pooling, since entropy weighting amplifies the one sharp retrieval head instead of dividing its mass by r.

The compression composes with channel-dimension quantization: a 20% page budget (5× along the sequence axis) multiplies with 4-bit KV storage (4× along the channel axis) for roughly 20× combined reduction, with the pooling logic unaffected by storage precision.

Eviction accounting becomes hardware-exact: per-layer page unions are bounded between $K_l$ and $H_{\mathrm{KV}} K_l$ pages, and the fused on-chip kernels add no DRAM traffic beyond the attention computation that produces the per-head distributions.

My inference: because Theorem 3's budget guarantee depends only on pooling-before-selection and not on how the weights are chosen, the exact UOR≡1 result would survive any group aggregator — max-over-heads, mean, or any learned scorer; only the needle-retention quality, not the memory bound, is at stake.

My inference: the paper's own pilot makes the Theorem 4 premise the decisive open question — if phase-1 models at 128k context keep measured retrieval-head mass near 0.1–0.2 rather than near 1, the strict all-layer needle guarantee will still not bind, and the needle-retention claim holds only in the weaker pooled-selection sense that the paper already reports.

My inference: the exact UOR–disagreement identity doubles as a production diagnostic — an operator can measure pairwise Jaccard disagreement among sibling heads during serving and immediately compute the union inflation any head-independent policy would incur, quantifying what pooling saves.

My inference: the temperature family suggests a cheap runtime safeguard — when within-group weight dispersion collapses toward the $1/r$ uniform limit, the group is visibly degenerating toward mean pooling, and switching that group to max-over-heads restores needle sensitivity without violating the UOR≡1 budget guarantee (Proposition 2).

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review