Pith. sign in

REVIEW 3 major objections 4 minor 67 references

KV cache compression can keep final answers correct while degrading the reasoning evidence that supports them.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-04 23:50 UTC pith:VK5HIK6Z

load-bearing objection First real test of whether KV compression preserves reasoning evidence, not just answers; the answer–evidence gap is a genuine finding that deserves peer review, with one overstated side claim and a residual judge-calibration caveat. the 3 major comments →

arxiv 2608.01631 v1 pith:VK5HIK6Z submitted 2026-08-03 cs.CL

Does Accuracy Equal Evidence? Reasoning Faithfulness under KV Cache Compression

Mengting Ai , Jingrui He , Yue Guo This is my paper
classification cs.CL
keywords KV cache compressionreasoning faithfulnesschain-of-thought validityanswer-evidence gaptoken evictionLLM-as-a-judgefixed-trace replaylarge reasoning models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests a common assumption: that a compressed AI model that still gives the right answer has also preserved the reasoning behind that answer. Using a controlled fixed-trace replay protocol, it shows this assumption fails for large reasoning models under token-eviction KV cache compression: ten eviction methods kept competitive final-answer accuracy on math, science, and clinical-calculation tasks while substantially degrading the validity of the visible rationale or its robustness to injected errors. The authors call this divergence the answer–evidence gap. A quantization control that keeps full token coverage was far less affected, suggesting the damage comes from losing access to parts of the reasoning trace, not from memory reduction itself. The stakes are practical: in high-stakes settings, a correct-looking but unsupported answer can be more dangerous than a visibly wrong one, and accuracy-only evaluation would certify it.

Core claim

The paper's central finding is an answer–evidence gap: under KV cache compression, the rate at which a model preserves the correct final answer and the rate at which it preserves valid evidence for that answer can diverge sharply. On AIME, GPQA-Diamond, and MedCalc-Bench, token-eviction compressors such as SnapKV and AdaKV at a 256-token budget retained much of Full-KV accuracy while wrong-chain correct answers (RWAC/correct) rose from single digits to over 60 percent on AIME and perturbation fidelity dropped by tens of points. Replaying identical reasoning traces isolates retention from rollout variation. Accuracy is an asymmetric diagnostic: collapse reveals damage, but preserved accuracy

What carries the argument

The load-bearing mechanism is the controlled fixed-trace replay protocol: the uncompressed model generates a complete reasoning trace; each compression method replays the same trace with its own compressed KV cache, so all methods observe identical textual reasoning and differences isolate what the compressed representation retains. It is paired with three metrics: final-answer accuracy; answer–chain consistency, using a language-model judge to label wrong-chain correct answers (RWAC) where the answer is right but the visible chain is not fully correct; and perturbation faithfulness, measuring fidelity and bias rate after injecting a wrong answer at start, middle, or end of the trace. A matc

Load-bearing premise

The main results rest on a language-model judge's binary verdict that a reasoning chain is 'not fully correct'; if the judge systematically treats compressed, terser rationales as unsupported regardless of their actual validity, the answer–evidence gap would be overstated.

What would settle it

Re-run the fixed-trace protocol with the reasoning judge blinded to compression status and with all rationales normalized to a uniform style and length, or judged by independent human experts on the same rubric. If the RWAC/correct differences between eviction methods and Full-KV collapse to near zero, the answer–evidence gap is a judge artifact rather than a genuine divergence between answer preservation and evidence preservation.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Accuracy-only leaderboards for KV compression can select methods that systematically produce correct-looking but unsupported answers; on AIME and GPQA-Diamond, accuracy and chain validity rank compressors in opposite directions.
  • More cache budget recovers final answers before it recovers evidence: chain validity remains degraded at all tested budgets on AIME, while fidelity recovers gradually.
  • Coverage-preserving quantization (KIVI-2bit) stays close to Full-KV across metrics, so the gap is tied to token eviction rather than KV memory reduction itself.
  • The gap is task-dependent: on answer-recoverable reasoning tasks it hides behind accuracy, while on evidence-constrained retrieval (RULER QA) compression damage shows up as accuracy collapse.
  • Deployment-style compressed generation also shows fragility in end-to-end checks, so the answer–evidence gap is not an artifact of the replay protocol.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same divergence likely affects any downstream system that consumes compressed-model rationales for auditing or trust scoring; one testable extension is whether human readers make more wrong trust decisions when given compressed rationales than accuracy numbers predict.
  • A compressor designed to preserve dependency and provenance structure — protecting definitions, intermediate computations, and verification steps over answer-like conclusions — could be tested directly for whether it closes RWAC while keeping accuracy.
  • The fixed-trace replay protocol could double as a training or selection signal: compressors could be chosen by replaying teacher traces and optimizing chain validity, not just final-answer match.
  • Because judge-based RWAC may be sensitive to verbosity, a further test is to separate surface brevity from evidence presence by measuring the gap under controlled rationale-length normalization.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper asks whether KV cache compression methods that preserve final-answer accuracy also preserve the validity of the visible reasoning chains that support those answers. Using a fixed-trace replay protocol, the authors hold the reasoning prefix fixed and compress only its KV states, then measure three metrics: final-answer accuracy, answer–chain consistency (via an LLM judge with human validation), and perturbation-based faithfulness. They evaluate ten token-eviction methods and one quantization method (KIVI) on Qwen3-8B, DeepSeek-R1-Distill-Llama-8B, and Qwen3-30B-A3B across AIME, GPQA-Diamond, MedCalc, and RULER. The main empirical claim is an 'answer–evidence gap': token-eviction compression can preserve competitive final-answer accuracy while substantially increasing the rate of wrong-chain-correct answers and reducing perturbation fidelity, whereas KIVI is less affected. The paper argues that final-answer accuracy is an asymmetric diagnostic and that evidence-aware metrics should complement accuracy-only evaluation.

Significance. If the finding holds, it is an important contribution to the evaluation of KV cache compression for reasoning models: it identifies a failure mode that accuracy-only benchmarks miss and introduces a reusable fixed-trace protocol with matched-answer analysis. The study is unusually broad in coverage (11 methods, 3 models, 4 tasks, budget ablations) and includes a human-validated LLM judge (κ≈0.89 on 400 stratified outputs) as well as a second judge and second annotator. The paper also explicitly avoids over-strong mechanistic claims. The main risks are the judge-dependence of the chain-validity metric and some overstated interpretations of the quantization control, but these are addressable and do not undermine the core empirical pattern.

major comments (3)
  1. [§2, §3.1] The fixed-trace protocol prefills q∥<think>r</think>, compresses the cache for r, and resumes after </think>. Since r is textually present, it is unclear what the LLM judge sees: the full output including r, or only the newly generated post-</think> continuation. If the judge sees r and r is correct, a correct final answer should almost always yield a correct chain, yet Table 3 shows large RWAC/correct increases for compressed methods (e.g., SnapKV 65.5 vs Full-KV 4.3 on AIME). Please specify the judge input and provide one concrete matched-answer example from Table 5 where the chain is judged not fully correct despite an identical correct r. Without this, the protocol's interpretation as 'same evidence, less usable support' is under-specified.
  2. [§3.1, Appendix B] The RWAC gap depends on the LLM judge's classification of compressed outputs. Overall human–judge κ=0.89 is strong, but it does not rule out a systematic bias toward labeling shorter, more fragmentary compressed continuations as 'not fully correct'. The no-think baseline in Table 3 shows the judge does not penalize brevity indiscriminately, but no-think outputs are not compressed sparse derivations. Please report human–judge agreement separately for compressed vs Full-KV outputs and by continuation length, and state the direction of disagreements (e.g., whether the judge is stricter than humans on compressed outputs). This directly bears on the quantitative magnitude of the headline gap.
  3. [§4.1, Table 3] The text says KIVI 'remains close to Full-KV across all metrics', but Table 3 does not fully support this. On GPQA-Diamond, KIVI-2bit has RWAC/correct=55.5 vs Full-KV 42.3, a 13-point degradation; on RULER, final accuracy is 64.3 vs Full-KV 78.9, a 14.6-point drop. KIVI is indeed less affected than the eviction methods, but it is not close to Full-KV across all metrics. Please revise the wording and report deltas explicitly; this tempers the mechanistic conclusion that coverage loss, rather than memory reduction itself, causes the answer–evidence gap.
minor comments (4)
  1. [§4.3, Table 4] The claim that accuracy and reasoning quality 'move in opposite directions' on AIME/GPQA-Diamond is too strong for GPQA, where Spearman ρ=-0.20 is weak. Also, n=11 methods gives wide uncertainty; report p-values or confidence intervals, and qualify the GPQA interpretation.
  2. [§4.2, RULER] The text says RULER is a case where 'accuracy itself collapses', but SnapKV and HeadKV retain accuracy 59.5 and 61.2 vs Full-KV 78.9. This is a substantial drop but not a collapse. Suggest softening the wording and clarifying the threshold used.
  3. [Appendix A.3.2, Table 10] For HeadKV and SnapKV end-to-end, RWAC/correct=0.0 is driven by a very small number of correct answers. The text explains this, but adding the correct-answer counts (denominators) in the table would prevent misinterpretation.
  4. [Global] Figures 2 and 3 are referenced frequently but lack detailed axis labels and definitions of 'chain consistency' and 'fidelity' in the captions. Adding a short legend describing the plotted quantities would improve readability. Minor typos: 'answer–evidencegap' in the abstract and 'tied less to' phrasing should be checked.

Circularity Check

0 steps flagged

No circular derivation: the answer–evidence gap is measured behaviorally, not obtained by construction or by self-citation.

full rationale

The paper's central claim is an empirical separation result: under KV cache compression, final-answer accuracy and the validity/fidelity of supporting reasoning can diverge. This claim is operationalized by three metrics (Section 3): final accuracy, RWAC (answer-chain consistency), and perturbation fidelity. These metrics measure the gap; they do not define it into existence, because the numerical values come from model outputs on held-out tasks, not from the metric definitions. No parameters are fitted to a subset of data and then renamed as predictions; the LLM judge is an external annotator whose labels are validated against human judgments with Cohen's kappa about 0.89 (Appendix B), and the Limitations explicitly acknowledge that exact RWAC values may vary with judge choice and prompt design. That is a measurement caveat, not a circular step. The fixed-trace replay protocol holds the reasoning text fixed across methods, and no equation in the paper equates the predicted quantity with an input by construction. The matched-answer analysis (Table 5) additionally isolates cases where Full-KV and compressed runs share the same correct answer, showing that eviction methods degrade chain validity even when the answer is preserved; this is an empirical conditional result, not a tautology. There are no load-bearing self-citations: the fixed-trace replay references (Kim and Gwon 2026; Kim et al. 2026) are external, and no uniqueness theorem or ansatz from the authors' prior work is used to force the conclusion. The 'answer-evidence gap' is a new label for an observed behavioral pattern, not a renaming that is then presented as evidence. Overall, the derivation chain is self-contained and empirically grounded, so the circularity score is 0.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

No numbers are fit to data and no new entities are postulated. The evaluation rests on the three domain assumptions listed above, which are stated in the paper and partially validated through human audit and cross-judge checks.

axioms (3)
  • domain assumption An LLM-as-a-judge can reliably classify reasoning chains as 'correct' versus 'not fully correct'.
    Used for all RWAC results in Section 3.1. Validated against 400 human labels with kappa about 0.89, but the exact labels are judge- and prompt-dependent.
  • domain assumption Fixed-trace replay, compressing only KV states of the reasoning prefix while the textual prefix is identical, isolates KV retention from rollout dynamics.
    Stated in Section 2 and Limitations. If compression interacts with the raw text prefix differently, the isolation claim weakens.
  • domain assumption Perturbation faithfulness, injecting an incorrect answer into the trace and measuring answer change or adoption, measures evidential robustness.
    Protocol in Section 3.2, borrowed from CoT faithfulness work. It is a controlled probe, not a complete adversarial robustness test.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Does Accuracy Equal Evidence? Reasoning Faithfulness under KV Cache Compression." pith.science (2026). https://pith.science/paper/VK5HIK6Z

@misc{pith2026260801631,
  author       = {Pith},
  title        = {Pith review of: Does Accuracy Equal Evidence? Reasoning Faithfulness under KV Cache Compression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VK5HIK6Z}},
  note         = {Machine review of arXiv:2608.01631}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

KV cache compression is commonly evaluated by final-answer accuracy, implicitly assuming that preserving the answer also preserves the reasoning that supports it. We test this assumption for large reasoning models and show that it can fail: under compression, correct answers and the validity of their visible supporting rationales can be preserved at different rates. We study this failure with a controlled fixed-trace replay protocol, which holds reasoning content fixed and isolates whether compression preserves usable information from an already available trace. We evaluate ten token-eviction KV compression methods and one quantization method on three models across mathematical reasoning, scientific QA, clinical calculation, and long-context retrieval. We measure final accuracy, answer-chain consistency, and perturbation faithfulness. Across tasks, token-eviction methods can preserve competitive final-answer accuracy while substantially degrading chain support or perturbation faithfulness. We call this the answer-evidence gap. A coverage-preserving quantization control is substantially less affected, suggesting that the failure is tied less to KV memory reduction itself than to losing access to parts of the reasoning trace. Code is available at https://github.com/famous-blue-raincoat/Safe_KV_Compress.

Figures

Figures reproduced from arXiv: 2608.01631 by Jingrui He, Mengting Ai, Yue Guo.

Figure 1
Figure 1. Figure 1: Illustration of the answer–evidence gap under [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Budget ablation on AIME26, GPQA￾Diamond, and MedCalc (Qwen3-8B). On AIME, chain consistency (reasoning quality) remains degraded at all budgets while faithfulness recovers steadily; on GPQA/MedCalc, chain consistency recovers to near￾baseline while faithfulness retains a persistent gap. gets generally improve the compressed model’s accuracy, but the answer–chain consistency or per￾turbation fidelity still … view at source ↗
Figure 3
Figure 3. Figure 3: Position-conditioned changes in perturbation [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: LLM-as-a-Judge Prompt Templates. We design strict domain-specific rubrics for (a) AIME, (b) GPQA [PITH_FULL_IMAGE:figures/full_fig_p019_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Perturbation protocol illustration on AIME. (a) A plausible but incorrect answer is injected at one of [PITH_FULL_IMAGE:figures/full_fig_p021_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

67 extracted references · 30 canonical work pages

  1. [1]

    International Conference on Machine Learning , pages=

    KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache , author=. International Conference on Machine Learning , pages=. 2024 , organization=

    work page 2024

  2. [2]

    Advances in Neural Information Processing Systems , volume=

    H2o: Heavy-hitter oracle for efficient generative inference of large language models , author=. Advances in Neural Information Processing Systems , volume=

  3. [3]

    Advances in Neural Information Processing Systems , volume=

    Kvquant: Towards 10 million context length llm inference with kv cache quantization , author=. Advances in Neural Information Processing Systems , volume=

  4. [4]

    Yuan Feng and Junlin Lv and Yukun Cao and Xike Xie and S Kevin Zhou , booktitle=. Ada-. 2026 , url=

    work page 2026

  5. [5]

    Xiang Liu and Zhenheng Tang and Peijie Dong and Zeyu Li and Liuyue and Bo Li and Xuming Hu and Xiaowen Chu , booktitle=. Chunk. 2026 , url=

    2026

  6. [6]

    Not All Heads Matter: A Head-Level

    Yu Fu and Zefan Cai and Abedelkadir Asi and Wayne Xiong and Yue Dong and Wen Xiao , booktitle=. Not All Heads Matter: A Head-Level. 2025 , url=

    2025

  7. [7]

    A Simple and Effective L\_2 Norm-Based Strategy for KV Cache Compression

    Devoto, Alessio and Zhao, Yu and Scardapane, Simone and Minervini, Pasquale. A Simple and Effective L\_2 Norm-Based Strategy for KV Cache Compression. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:10.18653/v1/2024.emnlp-main.1027

    doi:10.18653/v1/2024.emnlp-main.1027 2024

  8. [8]

    2025 , eprint=

    LagKV: Lag-Relative Information of the KV Cache Tells Which Tokens Are Important , author=. 2025 , eprint=

    work page 2025

  9. [9]

    Zefan Cai and Yichi Zhang and Bofei Gao and Yuliang Liu and Yucheng Li and Tianyu Liu and Keming Lu and Wayne Xiong and Yue Dong and Junjie Hu and Wen Xiao , booktitle=. Pyramid. 2025 , url=

    2025

  10. [10]

    Zefan Cai and Wen Xiao and Hanshi Sun and Cheng Luo and Yikai Zhang and Ke Wan and Yucheng Li and Yeyang Zhou and Li-Wen Chang and Jiuxiang Gu and Zhen Dong and Anima Anandkumar and Abedelkadir Asi and Junjie Hu , booktitle=. R-. 2026 , url=

    work page 2026

  11. [11]

    Proceedings of the 38th International Conference on Neural Information Processing Systems , articleno =

    Li, Yuhong and Huang, Yingbing and Yang, Bowen and Venkitesh, Bharat and Locatelli, Acyr and Ye, Hanchen and Cai, Tianle and Lewis, Patrick and Chen, Deming , title =. Proceedings of the 38th International Conference on Neural Information Processing Systems , articleno =. 2024 , isbn =

    work page 2024

  12. [12]

    The Twelfth International Conference on Learning Representations , year=

    Efficient Streaming Language Models with Attention Sinks , author=. The Twelfth International Conference on Learning Representations , year=

  13. [13]

    Transformers are Multi-State RNN s

    Oren, Matanel and Hassid, Michael and Yarden, Nir and Adi, Yossi and Schwartz, Roy. Transformers are Multi-State RNN s. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:10.18653/v1/2024.emnlp-main.1043

    doi:10.18653/v1/2024.emnlp-main.1043 2024

  14. [14]

    American Invitational Mathematics Examination (AIME) 2026 , author=

    2026

  15. [15]

    Bowman , booktitle=

    David Rein and Betty Li Hou and Asa Cooper Stickland and Jackson Petty and Richard Yuanzhe Pang and Julien Dirani and Julian Michael and Samuel R. Bowman , booktitle=. 2024 , url=

    2024

  16. [16]

    MedCalc-Bench: Evaluating Large Language Models for Medical Calculations , url =

    Khandekar, Nikhil and Jin, Qiao and Xiong, Guangzhi and Dunn, Soren and Applebaum, Serina S and Anwar, Zain and Sarfo-Gyamfi, Maame and Safranek, Conrad W and Anwar, Abid A and Zhang, Andrew and Gilson, Aidan and Singer, Maxwell B and Dave, Amisha and Taylor, Andrew and Zhang, Aidong and Chen, Qingyu and Lu, Zhiyong , booktitle =. MedCalc-Bench: Evaluatin...

    work page doi:10.52202/079017-2690

  17. [17]

    2024 , url=

    Cheng-Ping Hsieh and Simeng Sun and Samuel Kriman and Shantanu Acharya and Dima Rekesh and Fei Jia and Boris Ginsburg , booktitle=. 2024 , url=

    2024

  18. [18]

    2025 , eprint=

    Qwen3 Technical Report , author=. 2025 , eprint=

    2025

  19. [19]

    DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning , volume=

    Deepseek-AI , year=. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning , volume=. Nature , publisher=. doi:10.1038/s41586-025-09422-z , number=

    doi:10.1038/s41586-025-09422-z

  20. [20]

    2023 , eprint=

    Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection , author=. 2023 , eprint=

    2023

  21. [21]

    Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps

    Tutek, Martin and Hashemi Chaleshtori, Fateme and Marasovic, Ana and Belinkov, Yonatan. Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. doi:10.18653/v1/2025.emnlp-main.504

    doi:10.18653/v1/2025.emnlp-main.504 2025

  22. [22]

    2023 , eprint=

    Universal and Transferable Adversarial Attacks on Aligned Language Models , author=. 2023 , eprint=

    2023

  23. [23]

    2024 , eprint=

    The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions , author=. 2024 , eprint=

    2024

  24. [24]

    Biochemia medica , volume=

    Interrater reliability: the kappa statistic , author=. Biochemia medica , volume=. 2012 , publisher=

    2012

  25. [25]

    American Invitational Mathematics Examination (AIME) 2024 , author=

    2024

  26. [26]

    American Invitational Mathematics Examination (AIME) 2025 , author=

    2025

  27. [27]

    2025 , eprint=

    A Survey on LLM-as-a-Judge , author=. 2025 , eprint=

    2025

  28. [28]

    Proceedings of the twenty-seventh annual ACM-SIAM symposium on Discrete algorithms , pages=

    Nearly tight oblivious subspace embeddings by trace inequalities , author=. Proceedings of the twenty-seventh annual ACM-SIAM symposium on Discrete algorithms , pages=. 2016 , organization=

    work page 2016

  29. [29]

    H2O: heavy-hitter oracle for efficient generative inference of large language models , year =

    Zhang, Zhenyu and Sheng, Ying and Zhou, Tianyi and Chen, Tianlong and Zheng, Lianmin and Cai, Ruisi and Song, Zhao and Tian, Yuandong and R\'. H2O: heavy-hitter oracle for efficient generative inference of large language models , year =. Proceedings of the 37th International Conference on Neural Information Processing Systems , articleno =

    work page

  30. [30]

    Second Conference on Language Modeling , year=

    Training Large Language Models to Reason in a Continuous Latent Space , author=. Second Conference on Language Modeling , year=

    work page

  31. [31]

    2023 , eprint=

    Measuring Faithfulness in Chain-of-Thought Reasoning , author=. 2023 , eprint=

    2023

  32. [32]

    Preprint, alphaXiv , pages=

    Chain-of-thought is not explainability , author=. Preprint, alphaXiv , pages=

    work page

  33. [33]

    Language Models Don t Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting , url =

    Turpin, Miles and Michael, Julian and Perez, Ethan and Bowman, Samuel , booktitle =. Language Models Don t Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting , url =

  34. [34]

    2025 , eprint=

    Chain-of-Thought Reasoning In The Wild Is Not Always Faithful , author=. 2025 , eprint=

    2025

  35. [35]

    2025 , eprint=

    KVCompose: Efficient Structured KV Cache Compression with Composite Tokens , author=. 2025 , eprint=

    work page 2025

  36. [36]

    Mixing Importance with Diversity: Joint Optimization for

    Xuyang Liu and Xiyan Gui and Yuchao Zhang and Linfeng Zhang , booktitle=. Mixing Importance with Diversity: Joint Optimization for. 2026 , url=

    work page 2026

  37. [37]

    2025 , url=

    Dong Liu and Yanxuan Yu and Xuhong Wang and Ben Lengerich and Ying Nian Wu , booktitle=. 2025 , url=

    work page 2025

  38. [38]

    2026 , eprint=

    HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference , author=. 2026 , eprint=

    work page 2026

  39. [39]

    2026 , eprint=

    SkipKV: Selective Skipping of KV Generation and Storage for Efficient Inference with Large Reasoning Models , author=. 2026 , eprint=

    work page 2026

  40. [40]

    2026 , eprint=

    DeltaKV: Residual-Based KV Cache Compression via Long-Range Similarity , author=. 2026 , eprint=

    work page 2026

  41. [41]

    2026 , eprint=

    Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression , author=. 2026 , eprint=

    work page 2026

  42. [42]

    arXiv preprint arXiv:2510.00636 , year=

    Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution , author=. arXiv preprint arXiv:2510.00636 , year=

    arXiv

  43. [43]

    Proceedings of the 29th Symposium on Operating Systems Principles , pages =

    Kwon, Woosuk and Li, Zhuohan and Zhuang, Siyuan and Sheng, Ying and Zheng, Lianmin and Yu, Cody Hao and Gonzalez, Joseph and Zhang, Hao and Stoica, Ion , title =. Proceedings of the 29th Symposium on Operating Systems Principles , pages =. 2023 , isbn =. doi:10.1145/3600006.3613165 , abstract =

    arXiv 2023

  44. [44]

    Forty-second International Conference on Machine Learning , year=

    Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models , author=. Forty-second International Conference on Machine Learning , year=

  45. [45]

    2023 , eprint=

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , author=. 2023 , eprint=

    2023

  46. [46]

    2023 , eprint=

    Self-Consistency Improves Chain of Thought Reasoning in Language Models , author=. 2023 , eprint=

    2023

  47. [47]

    2025 , eprint=

    Reasoning Models Don't Always Say What They Think , author=. 2025 , eprint=

    2025

  48. [48]

    2025 , eprint=

    Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbosity , author=. 2025 , eprint=

    2025

  49. [49]

    2026 , eprint=

    FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning , author=. 2026 , eprint=

    work page 2026

  50. [50]

    2024 , eprint=

    Think before you speak: Training Language Models With Pause Tokens , author=. 2024 , eprint=

    2024

  51. [51]

    2024 , eprint=

    Let's Think Dot by Dot: Hidden Computation in Transformer Language Models , author=. 2024 , eprint=

    2024

  52. [52]

    2024 , eprint=

    Thinking Tokens for Language Modeling , author=. 2024 , eprint=

    2024

  53. [53]

    2024 , eprint=

    Distilling System 2 into System 1 , author=. 2024 , eprint=

    work page 2024

  54. [54]

    The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models , url=

    Renze, Matthew and Guven, Erhan , year=. The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models , url=. doi:10.1109/fllm63129.2024.10852493 , booktitle=

    arXiv 2024

  55. [55]

    2024 , eprint=

    C3oT: Generating Shorter Chain-of-Thought without Compromising Effectiveness , author=. 2024 , eprint=

    work page 2024

  56. [56]

    2025 , eprint=

    Chain of Draft: Thinking Faster by Writing Less , author=. 2025 , eprint=

    work page 2025

  57. [57]

    2025 , eprint=

    Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching , author=. 2025 , eprint=

    work page 2025

  58. [58]

    2025 , eprint=

    How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach , author=. 2025 , eprint=

    work page 2025

  59. [59]

    2025 , eprint=

    ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning , author=. 2025 , eprint=

    work page 2025

  60. [60]

    Lee and Sangdoo Yun and Hyun Oh Song , booktitle=

    Jang-Hyun Kim and Jinuk Kim and Sangwoo Kwon and Jae W. Lee and Sangdoo Yun and Hyun Oh Song , booktitle=. 2026 , url=

    work page 2026

  61. [61]

    Locating and Editing Factual Associations in GPT , url =

    Meng, Kevin and Bau, David and Andonian, Alex and Belinkov, Yonatan , booktitle =. Locating and Editing Factual Associations in GPT , url =

  62. [62]

    Towards Best Practices of Activation Patching in Language Models: Metrics and Methods , url =

    Zhang, Fred and Nanda, Neel , booktitle =. Towards Best Practices of Activation Patching in Language Models: Metrics and Methods , url =

    work page

  63. [63]

    Proceedings of the 30th International Conference on Neural Information Processing Systems , pages =

    Goyal, Anirudh and Lamb, Alex and Zhang, Ying and Zhang, Saizheng and Courville, Aaron and Bengio, Yoshua , title =. Proceedings of the 30th International Conference on Neural Information Processing Systems , pages =. 2016 , isbn =

    work page 2016

  64. [64]

    2026 , eprint=

    CASK: Core-Aware Selective KV Compression for Reasoning Traces , author=. 2026 , eprint=

    work page 2026

  65. [65]

    KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization , url =

    Zhang, Tianyi and Yi, Jonah and Xu, Zhaozhuo and Shrivastava, Anshumali , booktitle =. KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization , url =. doi:10.52202/079017-0109 , editor =

    work page doi:10.52202/079017-0109

  66. [66]

    and Shao, Yakun Sophia and Keutzer, Kurt and Gholami, Amir , booktitle =

    Hooper, Coleman and Kim, Sehoon and Mohammadzadeh, Hiva and Mahoney, Michael W. and Shao, Yakun Sophia and Keutzer, Kurt and Gholami, Amir , booktitle =. KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization , url =. doi:10.52202/079017-0040 , editor =

    doi:10.52202/079017-0040

  67. [67]

    2024 , editor =

    Liu, Zirui and Yuan, Jiayi and Jin, Hongye and Zhong, Shaochen and Xu, Zhaozhuo and Braverman, Vladimir and Chen, Beidi and Hu, Xia , booktitle =. 2024 , editor =

    work page 2024

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.