Pith. sign in

REVIEW 2 major objections 3 minor 37 references

Deterministic top-k KV-cache eviction is provably unable to diagnose the error it causes; randomized eviction restores a per-step error certificate.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 07:19 UTC pith:JEZ5GPIV

load-bearing objection Novel impossibility theorem and an honest, well-executed empirical study; but Theorem 2 rests on an unproven uniform weight bound that needs fixing. the 2 major comments →

arxiv 2607.21475 v2 pith:JEZ5GPIV submitted 2026-07-23 cs.LG cs.AIcs.CL

Error Certificates for KV-Cache Eviction via Randomized Design

Peng Xie This is my paper
classification cs.LG cs.AIcs.CL
keywords KV-cache evictionerror certificaterandomized designPoisson samplingHajek correctionSen-Yates-Grundysilent failureattribution
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Keeping only the top-k KV tokens makes the retained state independent of every evicted value, so a serving system that looks only at what it kept cannot tell how much the deletion changed the answer—even in principle. The paper proves this impossibility for any deterministic value-blind evictor and then shows that randomizing the tail restores the information: if eviction is a Poisson sample with known inclusion probabilities and one logit offset performs the Hajek correction inside the softmax, a survey-sampling variance estimator computed from the retained set alone becomes a per-step error certificate, measured to cover the true attention error about 97% of the time. On real long-context workloads the certificate is not a failure predictor—output confidence beats it at that—but it is the only online signal that separates cache-induced failures from inherently difficult ones, and scheduling recomputation on it beats random or confidence gating. The upshot the paper presses: deterministic eviction is structurally blind to its own damage, and randomization buys attribution, not prediction.

Core claim

Theorem 1 states that for any deterministic, value-blind eviction mechanism, no estimator measurable with respect to the online information can be consistent for the induced attention-output error: one can perturb all evicted values so that the retained keys, scores, and query history are bit-identical while the true output error grows arbitrarily. The paper then gives the design that restores identifiability: keep a certainty set, evict the tail by independent Bernoulli sampling with algorithm-chosen inclusion probabilities, and add log(1/pi) to the retained logit so the softmax denominator performs a Hajek correction. Theorem 2 shows that the resulting single-sum Sen-Yates-Grundy statistic

What carries the argument

The core mechanism is a randomized eviction design: a certainty set with inclusion probability 1, a Poisson-sampled tail with known inclusion probabilities, and a single logit offset per retained tail token that turns the softmax denominator into a Hajek estimator. The variance estimate is the Sen-Yates-Grundy single-sum form, which reduces to a Horvitz-Thompson-style sum over retained tail tokens, and an empirical-Bernstein radius (variance term plus range term) turns it into a per-step certificate. The proof machinery is classical design-based inference: unbiased variance estimation from the retained set without any assumptions on score quality, grounded in a uniform bounded-weight assumpt

Load-bearing premise

The per-step certificate's validity rests on the uniform bounded-weight assumption: normalized softmax weights on the tail are bounded by B/m, which the paper says the probability floor and certainty layer guarantee; if the inclusion probabilities are computed from a different importance score than the softmax weight, no such bound follows, and the certificate's remainder and range term may understate the true error.

What would settle it

Compute the ratio a_i/(pi_i * sum_j a_j) for tail tokens in a real model using the paper's probability formula. If the ratio is not uniformly bounded in practice, run the replay cells where an unbounded token is retained and check whether the certificate covers the realized error at near-nominal rates; a persistent deficit would falsify the design-based guarantee. Separately, a powered study finding a deterministic self-signal that predicts its own eviction-induced failures substantially above chance on natural data would weaken the practical significance of the impossibility theorem.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Any deterministic top-k serving system that claims a self-diagnostic of eviction damage is making an unsupportable promise: the failure is structural, not a matter of finding a better monitor.
  • The randomized design costs one scalar logit offset per retained tail token plus O(|tail|) work per head per step, with no retraining, and yields a per-step certificate with measured coverage of 96.9-97.7% at 12.5-50% budgets.
  • At question-aware budgets of 25-50%, eviction is nearly free, so the damage regime is streaming and agent memory, where history is compressed before future queries arrive.
  • Mean output log-probability is a stronger predictor of overall failure than any cache-side signal; the certificate's value is attribution, such as deciding whether a failure was caused by the cache or by the model itself.
  • Certificate-gated recomputation captures about 36% of the oracle's recoverable gain in streaming settings, beating random gating by roughly a factor of 1.7-1.8 and confidence gating, which can fall below random.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same unidentifiability argument likely transfers to other deterministic deletion or selection channels in inference—token pruning, early-exit dropping, context distillation—where the selection decision is independent of the deleted content; a known-randomness design could provide similar certificates there.
  • A cheaper two-run baseline of independent Poisson draws reaches only 0.62-0.69 attribution AUC against the certificate's 0.73-0.75 from a single draw, suggesting that the certificate's advantage comes from design-based validity rather than extra compute; this predicts that an optimized single threshold on a learned model would not close the gap.
  • The unablated implementation choices—head and layer subsampling, six-step window, normalizer floor—are natural places to test whether a cheaper certificate retains ranking quality; a single-layer, single-step version is a plausible deployment-ready extension.
  • The paper's stated but untested prediction that merging-based eviction error scales with within-stratum dispersion of evicted values rather than evicted attention mass is a direct testable extension of its stratified-ratio view, and would bound the certificate's relevance to merging-style methods.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper studies whether KV-cache eviction can estimate its own induced attention-output error at serving time. It proves a negative result for deterministic top-k eviction: no estimator measurable from the retained information can be uniformly consistent. It then proposes a randomized Poisson-tail design with known inclusion probabilities and a single logit offset implementing a Hajek correction, and derives a Sen-Yates-Grundy-style variance estimator over the retained set (Theorem 2), which is used to build an empirical-Bernstein certificate. Extensive pre-registered experiments on LongBench and synthetic tasks evaluate the certificate as a failure predictor and as an attribution/scheduling signal, with several claims deliberately killed. The paper honestly distinguishes theoretical guarantees from empirically validated behavior.

Significance. The negative result (Theorem 1) is clean and conceptually important: deterministic evictors cannot self-diagnose their induced error. The proposed randomized design is elegant, cheap, and supported by an unusually credible empirical study: pre-registered claims with kill conditions, per-run logs, released scripts, and explicit reporting of what failed. If Assumption 2 can be properly justified, or the design adjusted so that it holds, the certificate would be a valuable new capability for streaming compression. As it stands, the theoretical guarantee behind the positive result is incomplete, but the gap is local and fixable.

major comments (2)
  1. [Section 3.3, Assumption 2] The assertion that a_i/(pi_i * sum_j a_j) <= B/m is 'guaranteed constructively by the floor epsilon and the certainty layer' is not established. From Eq (2), pi_i is a clipped version of m * score_i / sum_j score_j, whereas a_i = exp(s_i). The floor only gives pi_i >= epsilon, so the stated bound would require B >= m/epsilon (with epsilon = 1e-6, B >= 1e6 * m); such a B is not the constant implied by the O(B^2/m^2) remainder. The certainty layer removes high-softmax-weight tokens from the tail, but the ratio a_i/score_i for a tail token with small score, or with a_i large relative to score_i, is unconstrained. This is load-bearing: Theorem 2 and the range term in Eq (4) both depend on the uniform bound. Please either prove Assumption 2 from the design (e.g., choose pi_i proportional to a_i on the tail) or replace it with an explicit, verifiable condition.
  2. [Appendix A.1, proof of Theorem 2] The proof asserts that 'each summand contributes at most B/m after normalization' and uses this to conclude bN/N = 1 + O_p(B/m) and a final O(B^2/m^2) remainder. If B is as large as m/epsilon, the first-order term is O(1/epsilon) and the remainder is O(1/epsilon^2), so the asymptotic statement is vacuous; if B is meant to be an absolute constant independent of m, that needs to be stated and proved. The unbiasedness claim E[bV_t] = Var(e_lin^t) + O(B^2/m^2) is therefore not established as written. This directly affects the certificate radius (4), whose variance term is bV_t and whose range term has the same B-dependence.
minor comments (3)
  1. [Eqs. (1)-(2)] Equation (2) uses 'score_i' while Eq. (1) defines s_i and a_i = exp(s_i). Clarify whether score_i is the same as s_i or an arbitrary importance score; this matters for interpreting Assumption 2 and for replicating the experiments.
  2. [Theorem 1] The proof assumes S != C ('whenever eviction occurs'). Add this condition explicitly to the theorem statement, or state that the result is vacuous when the full cache is retained.
  3. [Reproducibility] The reproducibility section says logs and scripts are 'packaged for release'; for a claims-based paper, a repository URL or supplement link would strengthen verifiability.

Circularity Check

0 steps flagged

No significant circularity: the impossibility proof and the design-based certificate derivations are self-contained; the noted Assumption 2 gap is a correctness risk, not a circular step.

full rationale

The paper's central derivation chain is not circular. Theorem 1 is a direct indistinguishability construction: altering evicted values while keeping all retained keys, values, scores, and downstream statistics identical leaves the online information Ft unchanged, yet shifts the true attention output arbitrarily. No fitted parameter or renamed input is involved. Theorem 2 is an application of classical survey-sampling identities (Horvitz-Thompson, Sen-Yates-Grundy, and the Hajek ratio expansion) to the algorithm's own stored inclusion probabilities; the retained-set variance estimator bVt contains no fitted or free parameter, and the unbiasedness claim is a theorem under the explicitly stated Assumptions 1-2. The empirical coverage of 96.9-97.7% is a direct measurement on replay cells with pass lines fixed in advance, not a fit of the certificate to those cells; the paper also explicitly states that the deployed certificate rests on measured coverage rather than claiming an unearned theorem. The seven pre-registered claims with kill conditions, three of which died, and the explicit labeling of post-hoc analyses, work against circularity rather than for it. The only self-referential element is the companion-manuscript remark in Remark 3, which is explicitly non-load-bearing. The skeptical concern about Assumption 2 - that the uniform bound B/m is asserted as 'guaranteed constructively by the floor epsilon and the certainty layer' without a proof relating the arbitrary importance score in Eq. (2) to the softmax weights ai - is a genuine correctness gap in the theorem's assumptions, but it is not circularity: Assumption 2 is not defined in terms of the theorem's conclusion, and the O(B^2/m^2) remainder is disclosed rather than hidden. An unsupported or false assumption makes the theorem incomplete, not self-referential. No load-bearing self-citation chain, no unique-theorem import from the authors' own prior work, and no fitted-input-called-prediction pattern are present.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 0 invented entities

The design-based theory rests on two algorithmic assumptions (known design, bounded weights) plus classical survey-sampling results. No fitted parameters are used to obtain the estimator or the certificate, though several hand-set constants (epsilon, epsilon_0, delta, tau) and subsampling choices affect the measured coverage. The main concern is Assumption 2, which is asserted rather than proved for arbitrary importance scores.

free parameters (4)
  • epsilon (inclusion-probability floor) = 1e-6
    Floor in Eq (2) chosen by hand; guarantees pi_i >= epsilon and affects variance/certificate magnitude, not fitted to data.
  • epsilon_0 (normalizer floor in certificate radius) = 1e-9
    Constant in Eq (4) chosen by hand to avoid division by zero.
  • delta (per-step certificate confidence) = 0.1
    Confidence level in Eq (4); chosen by hand; coverage measured against it.
  • tau (red-flag threshold) = 1
    Label-free operating threshold for flagging; argued as a design-scale boundary, not fitted, but still a hand-set constant.
axioms (6)
  • domain assumption Known design (Assumption 1): inclusion probabilities pi_i are chosen, stored, and bounded below by epsilon on the tail.
    Needed for design-based unbiasedness of bV_t in Theorem 2 (Section 3.3).
  • domain assumption Bounded normalized weights (Assumption 2): a_i/(pi_i * sum_j a_j) <= B/m for all tail i.
    Controls the O(B^2/m^2) remainder in Theorem 2 and the range term in the certificate (Eq 4); the paper asserts it is guaranteed but does not prove it for the deployed importance scores.
  • domain assumption Poisson sampling: tail retention indicators I_i are independent Bernoulli(pi_i).
    Reduces the Sen-Yates-Grundy double sum to a single sum; used throughout Theorem 2 and Appendix A.1.
  • standard math HT/SYG variance identities and the Hajek ratio expansion are correct.
    Classical design-based inference results (Horvitz-Thompson, Sen-Yates-Grundy, Sarndal et al.) invoked in Appendix A.1.
  • standard math Empirical-Bernstein and e-process bounds (Howard et al.; Waudby-Smith-Ramdas; Ville's inequality) hold as stated.
    Used to turn the variance estimate into the certificate radius (Eq 4) and the anytime construction (Remark 2, Appendix A.2).
  • domain assumption Task-level transfer of a per-layer attention certificate is empirical, not derived.
    Paper explicitly does not claim a bound across LayerNorm, residual streams, or autoregressive sampling; deployed validity rests on measured coverage.

pith-pipeline@v1.3.0-alltime-deepseek · 19934 in / 17249 out tokens · 156670 ms · 2026-08-01T07:19:18.139810+00:00 · methodology

0 comments
read the original abstract

Deterministic KV-cache eviction keeps the top-$k$ tokens under an importance score and deletes the rest. We prove that this design cannot know what it destroyed: evicted values can be altered so that everything the serving system retains is unchanged while the true attention-output error grows arbitrarily, so no serving-time estimator of that error is consistent. Randomized eviction restores identifiability. With a Poisson-sampled tail at known inclusion probabilities, one logit offset performs the H\'ajek correction inside the softmax, and a survey-sampling variance estimator over the retained set becomes a per-step error certificate with 0.97 empirical coverage at no accuracy cost. On real workloads, seven pre-registered claims locate the certificate's value precisely. Prediction goes to output confidence: question-aware eviction at 25--50\% budgets is nearly free, output log-probability predicts failure better than any cache-side signal, and certificate-gated budget escalation adds nothing. Attribution stays with the certificate: it separates cache-induced from inherent failures (AUC 0.65--0.75, against 0.47--0.54 for output confidence) and schedules recomputation better than random or confidence gating. Randomization buys attribution, not prediction.

Figures

Figures reproduced from arXiv: 2607.21475 by Peng Xie.

Figure 1
Figure 1. Figure 1: One dialogue, one budget, two fates (verbatim run; Qwen2.5-7B-Instruct; thirty-turn [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Eviction-induced failure rate (among examples the full cache answers correctly) on exact [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The self-signal panel on eviction-induced failures, pooled over 6k and 16k LongBench [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Certificate against output log-probability on three questions, at 6k, 16k, and natural [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Certificate-gated recomputation under streaming compression at 16k, re-running 25% of [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Agent-memory vignette under streaming compression to a 25% budget (two instruct [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The gated system across the merged 10–30% budget grid (Qwen2.5-7B, fifty dialogues, [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 17 linked inside Pith

  1. [1]

    LongBench: A bilingual, multitask benchmark for long context understanding

    Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. LongBench: A bilingual, multitask benchmark for long context understanding. InProceedings of ACL, 2024. arXiv:2308.14508

    Pith/arXiv arXiv 2024

  2. [2]

    PyramidKV: Dynamic KV cache compression based on pyramidal information funneling.arXiv preprint arXiv:2406.02069, 2024

    Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Baobao Chang, Junjie Hu, and Wen Xiao. PyramidKV: Dynamic KV cache compression based on pyramidal information funneling.arXiv preprint arXiv:2406.02069, 2024

    Pith/arXiv arXiv 2024

  3. [3]

    Value-aware stochastic KV cache eviction for reasoning models.arXiv preprint arXiv:2606.03928, 2026

    Ting-Yun Chang, Harvey Yiyun Fu, Deqing Fu, Chenghao Yang, Jesse Thomason, and Robin Jia. Value-aware stochastic KV cache eviction for reasoning models.arXiv preprint arXiv:2606.03928, 2026

    Pith/arXiv arXiv 2026

  4. [4]

    MagicPIG: LSH sampling for efficient LLM generation

    Zhuoming Chen, Ranajoy Sadhukhan, Zihao Ye, Yang Zhou, Jianyu Zhang, Niklas Nolte, Yuandong Tian, Matthijs Douze, L´ eon Bottou, Zhihao Jia, and Beidi Chen. MagicPIG: LSH sampling for efficient LLM generation. InInternational Conference on Learning Representations,

  5. [5]

    vAttention: Verified sparse attention.arXiv preprint arXiv:2510.05688, 2025

    Aditya Desai, Kumar Krishna Agrawal, et al. vAttention: Verified sparse attention.arXiv preprint arXiv:2510.05688, 2025. 17

    Pith/arXiv arXiv 2025

  6. [6]

    Forget without compromise: Nexus sampling for streaming KV-cache eviction under fixed budgets.arXiv preprint arXiv:2606.23961, 2026

    Duc Duong, Hoang Anh Duy Le, Jianwen Xie, Anshumali Shrivastava, and Zhaozhuo Xu. Forget without compromise: Nexus sampling for streaming KV-cache eviction under fixed budgets.arXiv preprint arXiv:2606.23961, 2026

    Pith/arXiv arXiv 2026

  7. [7]

    Generalized friendship paradox in complex networks: The case of scientific collaboration.Scientific Reports, 4:4603, 2014

    Young-Ho Eom and Hang-Hyun Jo. Generalized friendship paradox in complex networks: The case of scientific collaboration.Scientific Reports, 4:4603, 2014

    2014

  8. [8]

    Scott L. Feld. Why your friends have more friends than you do.American Journal of Sociology, 96(6):1464–1477, 1991

    1991

  9. [9]

    Kevin Zhou

    Yuan Feng, Junlin Lv, Yukun Cao, Xike Xie, and S. Kevin Zhou. Ada-KV: Optimizing KV cache eviction by adaptive budget allocation for efficient LLM inference.arXiv preprint arXiv:2407.11550, 2024

    Pith/arXiv arXiv 2024

  10. [10]

    Selective classification for deep neural networks

    Yonatan Geifman and Ran El-Yaniv. Selective classification for deep neural networks. In Advances in Neural Information Processing Systems 30, 2017

    2017

  11. [11]

    CAOTE: KV cache selection for LLMs via attention output error- based token eviction.arXiv preprint arXiv:2504.14051, 2025

    Raghavv Goel, Junyoung Park, Mukul Gagrani, Dalton Jones, Matthew Morse, Harper Langston, Mingu Lee, and Chris Lott. CAOTE: KV cache selection for LLMs via attention output error- based token eviction.arXiv preprint arXiv:2504.14051, 2025

    arXiv 2025

  12. [12]

    The Llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori et al. The Llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Asymptotic theory of rejective sampling with varying probabilities from a finite population.Annals of Mathematical Statistics, 35(4):1491–1523, 1964

    Jaroslav H´ ajek. Asymptotic theory of rejective sampling with varying probabilities from a finite population.Annals of Mathematical Statistics, 35(4):1491–1523, 1964

    1964

  14. [14]

    A baseline for detecting misclassified and out-of-distribution examples in neural networks

    Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. InInternational Conference on Learning Representations, 2017. arXiv:1610.02136

    Pith/arXiv arXiv 2017

  15. [15]

    Horvitz and Donovan J

    Daniel G. Horvitz and Donovan J. Thompson. A generalization of sampling without replacement from a finite universe.Journal of the American Statistical Association, 47(260):663–685, 1952

    1952

  16. [16]

    Howard, Aaditya Ramdas, Jon McAuliffe, and Jasjeet Sekhon

    Steven R. Howard, Aaditya Ramdas, Jon McAuliffe, and Jasjeet Sekhon. Time-uniform, nonparametric, nonasymptotic confidence sequences.Annals of Statistics, 49(2):1055–1080, 2021

    2021

  17. [17]

    RULER: What’s the real context size of your long-context language models? InFirst Conference on Language Modeling, 2024

    Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. RULER: What’s the real context size of your long-context language models? InFirst Conference on Language Modeling, 2024. arXiv:2404.06654

    Pith/arXiv arXiv 2024

  18. [18]

    Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7B.arXiv preprint arXiv:2310.06825, 2023

    Pith/arXiv arXiv 2023

  19. [19]

    Wiley, 1965

    Leslie Kish.Survey Sampling. Wiley, 1965

    1965

  20. [20]

    MomentKV: Closing the directional gap in KV cache eviction for long-context inference.arXiv preprint arXiv:2606.01563, 2026

    Yu Li, Binxu Li, and Tian Lan. MomentKV: Closing the directional gap in KV cache eviction for long-context inference.arXiv preprint arXiv:2606.01563, 2026

    Pith/arXiv arXiv 2026

  21. [21]

    SnapKV: LLM knows what you are looking for before generation

    Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. SnapKV: LLM knows what you are looking for before generation. InAdvances in Neural Information Processing Systems 37, 2024. arXiv:2404.14469. 18

    Pith/arXiv arXiv 2024

  22. [22]

    On the two different aspects of the representative method.Journal of the Royal Statistical Society, 97(4):558–625, 1934

    Jerzy Neyman. On the two different aspects of the representative method.Journal of the Royal Statistical Society, 97(4):558–625, 1934

    1934

  23. [23]

    Transformers are multi-state RNNs

    Matanel Oren, Michael Hassid, Nir Yarden, Yossi Adi, and Roy Schwartz. Transformers are multi-state RNNs. InProceedings of EMNLP, 2024. arXiv:2401.06104

    Pith/arXiv arXiv 2024

  24. [24]

    Qwen2.5 technical report.arXiv preprint arXiv:2412.15115, 2024

    Qwen Team. Qwen2.5 technical report.arXiv preprint arXiv:2412.15115, 2024

    Pith/arXiv arXiv 2024

  25. [25]

    Game-theoretic statistics and safe anytime-valid inference.Statistical Science, 38(4):576–601, 2023

    Aaditya Ramdas, Peter Gr¨ unwald, Vladimir Vovk, and Glenn Shafer. Game-theoretic statistics and safe anytime-valid inference.Statistical Science, 38(4):576–601, 2023

    2023

  26. [26]

    Springer, 1992

    Carl-Erik S¨ arndal, Bengt Swensson, and Jan Wretman.Model Assisted Survey Sampling. Springer, 1992

    1992

  27. [27]

    Amode R. Sen. On the estimate of the variance in sampling with varying probabilities.Journal of the Indian Society of Agricultural Statistics, 5:119–127, 1953

    1953

  28. [28]

    Quest: Query-aware sparsity for efficient long-context LLM inference

    Jiaming Tang, Yilong Zhao, Kan Zhu, Guangxuan Xiao, Baris Kasikci, and Song Han. Quest: Query-aware sparsity for efficient long-context LLM inference. InInternational Conference on Machine Learning, 2024. arXiv:2406.10774

    Pith/arXiv arXiv 2024

  29. [29]

    ´Etude critique de la notion de collectif

    Jean Ville. ´Etude critique de la notion de collectif. Gauthier-Villars, 1939

    1939

  30. [30]

    E-values: Calibration, combination and applications.Annals of Statistics, 49(3):1736–1754, 2021

    Vladimir Vovk and Ruodu Wang. E-values: Calibration, combination and applications.Annals of Statistics, 49(3):1736–1754, 2021

    2021

  31. [31]

    Estimating means of bounded random variables by betting.Journal of the Royal Statistical Society: Series B, 86(1):1–27, 2024

    Ian Waudby-Smith and Aaditya Ramdas. Estimating means of bounded random variables by betting.Journal of the Royal Statistical Society: Series B, 86(1):1–27, 2024

    2024

  32. [32]

    Efficient streaming language models with attention sinks

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. InInternational Conference on Learning Representations,

  33. [33]

    Michael Grundy

    Frank Yates and P. Michael Grundy. Selection without replacement from within strata with probability proportional to size.Journal of the Royal Statistical Society: Series B, 15(2): 253–261, 1953

    1953

  34. [34]

    When does value-aware KV eviction help? A fixed-contract diagnostic for non-monotone cache compression

    Ruijie Zhang, Haozhe Liang, Da Chang, Li Hu, Fanqi Kong, Huaxiao Yin, and Yu Li. When does value-aware KV eviction help? A fixed-contract diagnostic for non-monotone cache compression. arXiv preprint arXiv:2605.08234, 2026

    Pith/arXiv arXiv 2026

  35. [35]

    CaM: Cache merging for memory-efficient LLMs inference

    Yuxin Zhang, Yuxuan Du, Gen Luo, Yunshan Zhong, Zhenyu Zhang, Shiwei Liu, and Rongrong Ji. CaM: Cache merging for memory-efficient LLMs inference. InInternational Conference on Machine Learning, 2024

    2024

  36. [36]

    H2O: Heavy-hitter oracle for efficient generative inference of large language models

    Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher R´ e, Clark Barrett, Zhangyang Wang, and Beidi Chen. H2O: Heavy-hitter oracle for efficient generative inference of large language models. InAdvances in Neural Information Processing Systems 36, 2023. arXiv:2306.14048. 19 A Proofs A.1 Theo...

    Pith/arXiv arXiv 2023

  37. [37]

    converts Mt into the time-uniform statement, and the boundary in [ 16] gives the stated radius shape (a variance term plus a range term, both computable from the retained set). If tail indicators are redrawn every Tr steps the increments are independent across blocks; if the same draw is reused, increments within a block are identical and the product is t...

    2026