Pith. sign in

REVIEW 3 major objections 5 minor 54 references

Task-driven evolutionary search beats KV cache heuristics at equal memory.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 20:49 UTC pith:G7BBQMYQ

load-bearing objection Worth a serious referee: the evolutionary allocation idea is new and the held-out results are consistently positive, but the search objective mismatch and single-run reporting need to be addressed before I'd trust the headline margins. the 3 major comments →

arxiv 2509.08315 v1 pith:G7BBQMYQ submitted 2025-09-10 cs.LG cs.CLcs.NE

EvolKV: Evolutionary KV Cache Compression for LLM Inference

Bohan Yu , Yekun Chai This is my paper
classification cs.LG cs.CLcs.NE
keywords KV cache compressionevolutionary searchCMA-ESlayer-wise budget allocationLLM inferencelong-contextGSM8Ktask-driven optimization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that the best way to compress a frozen LLM's KV cache is to let a black-box optimizer decide how many tokens each layer may keep, guided only by the downstream task score. It claims that existing rule-based allocations — same budget everywhere, fixed positions, or a pyramid that shrinks with depth — miss the task-specific importance of layers. If correct, the same memory footprint can be made to retain more task-relevant information, and extreme compression can even beat the uncompressed model on code completion. The practical payoff would be plug-and-play inference compression: no fine-tuning, no architecture changes, just a per-task layer-budget table found by search.

Core claim

EvolKV's central claim is that per-layer KV cache budgets should be treated as optimizable parameters, not as a fixed heuristic shape. It groups contiguous layers, searches over group budgets with CMA-ES, and scores each candidate by downstream task performance (F1, accuracy, recall) penalized by deviation from the target average cache size. On LongBench, GSM8K, Needle-in-a-Haystack, and RULER, with Mistral-7B-Instruct and Llama-3-8B-Instruct, it reports consistent average gains over uniform, fixed-position, and pyramidal baselines at equal total budgets; a 128-budget allocation reaches 95.7% of full-model GSM8K accuracy where the strongest baseline keeps only 84.5%, and at 1.5% of the cache

What carries the argument

The central object is the per-layer KV cache budget vector K, partitioned into groups and optimized by CMA-ES, a black-box evolutionary optimizer. Fitness combines the downstream metric f(S) with a cache-efficiency term that discounts allocations whose average budget exceeds the target c. The search proceeds group by group from bottom to top, freezing optimized groups, which shrinks the search space and stabilizes optimization. A scale-up rule then proportionally expands a low-budget allocation to larger target budgets, and the paper argues this preserves near-optimality.

Load-bearing premise

The fitness computed on 30 randomly sampled optimization examples ranks budget allocations the same way the full evaluation would, and the shape optimized at a 128-token budget stays near-optimal when scaled to larger budgets.

What would settle it

Repeat the optimization at c=128 on 30 different random draws from NarrativeQA, evaluate each resulting allocation on 1,000 held-out NarrativeQA questions, and compare the spread; if the variance across draws is comparable to or larger than the reported average gain over PyramidKV, the search is fitting the sample, not a stable layer-importance pattern.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the claim holds, KV cache compression becomes task-adaptive: a small labeled sample per task yields a layer-budget table that transfers to that task's evaluation set.
  • Equal-memory compression can become strictly more accurate than uniform or pyramidal heuristics, because the optimizer can concentrate budget on layers that matter for the task.
  • Extreme compression budgets are not necessarily a monotone loss: at 1.5% of the cache, the optimized allocation can beat the full-cache model on code completion, suggesting redundancy that is better removed by task-aware allocation.
  • The optimized low-budget shape can be expanded to higher budgets without re-running the search, making per-task optimization cheap at deployment.
  • Because the method works on frozen models and arbitrary metrics, it can be applied to proprietary or instruction-tuned models without access to training gradients.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: the fitness-transfer assumption could be tested by measuring how many optimization instances are needed before the chosen allocation stabilizes; if the 30-sample fitness is noisy, the gains would shrink or vanish.
  • Inference: the scale-invariance of the optimized budget shape is a nontrivial empirical claim; if layer-importance shifts with total cache size, the expansion rule would need per-budget re-optimization.
  • Inference: the same evolutionary framework could be applied at finer granularity, e.g., attention-head-level budgets, which the paper lists as future work and which would test whether middle-layer peaks are head-specific or layer-wide.
  • Inference: because the paper finds consistent mid-layer maxima in cache budgets, an independent check is whether those layers correspond to measurable attention or representation properties, which would connect the allocation to model interpretability.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. EvolKV proposes to replace heuristic layer-wise KV cache budget allocations (uniform, fixed-position, pyramidal) with an evolutionary search. Using CMA-ES, it optimizes per-group KV budgets on a small set of downstream-task instances under a target average cache budget c, then evaluates the resulting allocation on LongBench, GSM8K, NIAH, and RULER with Mistral-7B-Instruct, Llama-3-8B-Instruct, and two Qwen models. The central claims are that EvolKV outperforms all tested baselines across budgets on long-context and math tasks, that the c=128 optimized allocation transfers to larger budgets by proportional expansion, and that on some code-completion tasks it exceeds the full-cache model at extreme compression.

Significance. If the central claims are correct, the paper makes a useful empirical contribution: it demonstrates that layer-wise, task-driven budget allocation can beat simple heuristics at equal memory cost, and it provides evidence that the optimal allocation is non-monotonic and not pyramidal. The paper also has genuine strengths: optimization instances are removed from evaluation; stability is checked across three optimization rounds (Appendix C.2, std 0.078); alternate optimization-data mixes are tested (Table 9); and cross-dataset transfer from NIAH to LongBench is examined (Appendix C.5). These design choices support the empirical comparisons as measured. However, the load-bearing search objective in Algorithm 1 is not the same as the objective stated in Eq. (1), and several quantitative claims in the abstract are stronger than the tables support.

major comments (3)
  1. [Section 3.2, Eq. (1), Algorithm 1] The stated objective in Eq. (1) maximizes f(S)(1+λ CACHESCORE(S,c)) over complete schemes S with a global average-budget constraint, and the 'KV Cache Budget Completion' step rescales all layer budgets to exactly T=cL. But Algorithm 1 (lines 10-11) evaluates each candidate as r = f(˜G)(1 + λ CACHESCORE(Sg,c)), where CACHESCORE is applied only to the candidate group Sg, not to the completed global allocation. Thus CMA-ES is guided by a proxy that differs from the evaluated configuration: a group with a very low uncompleted budget can receive a high cache-efficiency score and then be proportionally expanded to match the target, changing its actual layer-wise shape. The paper provides no evidence that rankings on uncompleted partial allocations correlate with rankings after completion. Since the paper's central claim is that evolutionary search 'directly maximizes downstream performance' un
  2. [Section 4.2.1, Appendix C.4, Table 10] Most results at c=256-2048 are obtained by proportionally expanding the allocation optimized at c=128 rather than by optimizing at the target budget. The paper argues that expansion is competitive or better, but Table 10 shows that at c=256 direct optimization outperforms expansion (EvolKV.opt 39.34 vs EvolKV.ex 39.05), and the table contains no c=2048 row even though the text states expansion beats optimization at c=2048. Therefore the scale-invariance assumption is not uniformly supported, and the c=2048 rows in Tables 1-2 rely on extrapolation without direct comparison. Please report c=2048 direct-optimization results or otherwise justify why the extrapolation is reliable at that budget.
  3. [Abstract, Section 4.2.1, Tables 1-2] The abstract and text claim EvolKV 'achieves superior performance over the full KV cache setting on code completion while utilizing only 1.5% of the original budget.' This is not supported by Table 1 for Mistral-7B-Instruct: at c=128, EvolKV scores 51.64/47.05 on LCC/RepoBench-P versus Full scores 57.27/53.88. For Llama-3-8B-Instruct (Table 2), EvolKV's code average is 56.26 versus Full's 56.24, and only RepoBench-P exceeds the full model (55.11 vs 53.59). The claim should be qualified to the specific model and sub-task, otherwise it overstates the empirical finding.
minor comments (5)
  1. [Eq. (1)-(2)] The notation is inconsistent: Eq. (1) uses 'CACHE SCORE' with a space, while Eq. (2) and Algorithm 1 use 'CACHESCORE'. Please unify.
  2. [Appendix C.4, Table 10] The text refers to c=2048 in the expansion-versus-optimization comparison, but Table 10 lists only c=256, 512, and 1024. Add the missing row or revise the text.
  3. [Section 4.2.3] The NIAH optimization setup uses 'no more than 35 instances whose average scores ... are below 60.' The threshold and instance-selection procedure are not justified; please specify how these instances were chosen and how sensitive the results are to that threshold.
  4. [Introduction] Minor typos: 'While effectively for memory reduction' should be 'While effective for memory reduction'; the table header 'A vg.' in several tables appears to be a rendering artifact.
  5. [Figures 2-3] The axis labels in Figures 2 and 3 are difficult to read in the provided version; please ensure the final figures are legible, especially the heatmap in Figure 2c.

Circularity Check

0 steps flagged

No significant circularity: the central claims rest on held-out evaluations and a genuine c=128-to-larger-budget extrapolation; the only self-citation is non-load-bearing.

full rationale

EvolKV's central claim is that evolutionary search over per-layer KV cache budgets improves downstream performance. The optimization fitness is computed on a small held-out training subset (e.g., 30 NarrativeQA instances for LongBench, 30 GSM8K training instances), and the reported test numbers are measured on disjoint held-out data (e.g., LongBench test samples with training samples removed; GSM8K test set). The c=128 optimized allocation is proportionally expanded to c=256-2048 using the budget-completion procedure described in Section 3.2, and the resulting performance is then measured; this is a genuine extrapolation, not a quantity fitted to the test set. The paper explicitly removes optimization instances from evaluation ('with all training samples removed') and tests transfer across datasets (NIAH to LongBench, NIAH to RULER), which are held-out predictions. The only self-citation is 'inspired by (Chai et al., 2022)' in the introduction; it is used only as a general inspiration for evolutionary algorithms and is not load-bearing for any derivation or uniqueness claim. The skeptical concern that Algorithm 1 evaluates fitness on uncompleted group budgets while the final evaluation uses budget-completed allocations is a correctness/objective-mismatch issue, not circularity: the search proxy differs from the evaluated configuration, but the reported results are not definitionally equal to the search objective. Similarly, the 30-sample representativeness and scale-invariance assumptions are empirical assumptions that could fail, but they do not make the predictions equivalent to their inputs by construction. Therefore no specific circular step can be exhibited, and the paper's central experimental claims have independent content.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

The method's contribution is the allocation shape; everything else is pulled from prior literature or set by hand. The search is calibrated by lambda, gamma, sigma, ng, and the 30-example fitness set, all hand-set or tuned on the benchmark, and the final per-group budgets are themselves fitted values. The two domain assumptions carrying the most weight are the 30-example fitness proxy and the scale-invariance of the optimal shape, both only partially tested. No new physical entities are introduced.

free parameters (7)
  • lambda (cache-efficiency weight) = 0.3
    Balances task fitness against budget deviation in Eq. (1); set by hand and never ablated in the paper.
  • gamma (CACHESCORE smoothing factor) = 0.2
    Shapes the smooth discount in Eq. (2); hand-set, not ablated.
  • sigma (CMA-ES step size) = 0.3
    CMA-ES learning-rate/step-size; hand-set, not ablated.
  • ng (group size) = 8
    Number of layers optimized together; chosen because downstream performance 'peaks at ng = 8' in the ablation (Fig. 4a, Table 7), i.e., tuned on the reported benchmark.
  • number of optimization instances = 30
    30 (or at most 35 for NIAH) sampled examples used as the fitness set; hand-set, sensitivity not reported.
  • NIAH optimization-instance selection threshold = baseline score < 60
    For NIAH/RULER, EvolKV optimizes only on instances whose baseline scores on both models are below 60 (Section 4.2.3); a deliberate non-random choice whose effect is not analyzed.
  • optimized per-group KV budgets = 4 budgets per model/task (see Fig. 3a,b)
    The final fitted object of the method: per-group token budgets maximizing fitness on the 30-instance set; these are the quantities the paper then reports and extrapolates.
axioms (5)
  • domain assumption The downstream-task fitness f computed on 30 sampled instances reliably ranks candidate budget allocations for the full evaluation distribution.
    Entire search depends on the 30-example fitness proxy; if noisy or unrepresentative, CMA-ES overfits budgets to the sample and held-out gains vanish. Partial support: 3-run stability (Table 8, std 0.078) and alternate training-data results (Table 9), but no bootstrap or sensitivity analysis. Sections 4.2.1 and 4.3.
  • domain assumption The optimal allocation shape is scale-invariant: budgets optimized at c=128, proportionally expanded, remain near-optimal at c=256-2048.
    Most reported results (Tables 1-2, budgets 256-2048) come from expansion of the c=128 shape, not from search at that budget. App. C.4 shows expansion even beats direct optimization at high budgets, evidence for the assumption, but the mechanism is unexplained. Section 3.2 'KV Cache Budget Completion', App. C.4.
  • domain assumption The optimal allocation can be found by sequential bottom-up optimization of contiguous 8-layer groups with previously optimized groups frozen.
    Algorithm 1 is a greedy coordinate-style search over 4 blocks; it assumes earlier-group choices stay optimal when later groups change, and that contiguous groupings suffice. No comparison against joint optimization or non-contiguous groupings. Section 3.2, Algorithm 1.
  • domain assumption SnapKV's window-based token selection is an unbiased base scheduler, so comparisons isolate the budget-shape effect.
    EvolKV is evaluated as SnapKV selection plus EvolKV budgets, so all reported gains over SnapKV/PyramidKV are attributed to the per-layer budget shape; if token ranking within layers interacts with budget shapes, the attribution is confounded. Section 4.1.
  • standard math CMA-ES, run on a continuous relaxation with an unspecified discretization of integer budgets, converges to a reasonable surrogate of the integer optimum within M iterations.
    The paper never specifies how candidate integer budgets are encoded, bounded, or discretized, nor the number of generations M; the optimizer's behavior on this small discrete search space is taken as given. Section 3.2, Algorithm 1.

pith-pipeline@v1.3.0-alltime-deepseek · 29308 in / 25822 out tokens · 273839 ms · 2026-08-04T20:49:11.560740+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of EvolKV: Evolutionary KV Cache Compression for LLM Inference." pith.science (2026). https://pith.science/paper/G7BBQMYQ

@misc{pith2026250908315,
  author       = {Pith},
  title        = {Pith review of: EvolKV: Evolutionary KV Cache Compression for LLM Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G7BBQMYQ}},
  note         = {Machine review of arXiv:2509.08315}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Existing key-value (KV) cache compression methods typically rely on heuristics, such as uniform cache allocation across layers or static eviction policies, however, they ignore the critical interplays among layer-specific feature patterns and task performance, which can lead to degraded generalization. In this paper, we propose EvolKV, an adaptive framework for layer-wise, task-driven KV cache compression that jointly optimizes the memory efficiency and task performance. By reformulating cache allocation as a multi-objective optimization problem, EvolKV leverages evolutionary search to dynamically configure layer budgets while directly maximizing downstream performance. Extensive experiments on 11 tasks demonstrate that our approach outperforms all baseline methods across a wide range of KV cache budgets on long-context tasks and surpasses heuristic baselines by up to 7 percentage points on GSM8K. Notably, EvolKV achieves superior performance over the full KV cache setting on code completion while utilizing only 1.5% of the original budget, suggesting the untapped potential in learned compression strategies for KV cache budget allocation.

Figures

Figures reproduced from arXiv: 2509.08315 by Bohan Yu, Yekun Chai.

Figure 1
Figure 1. Figure 1: Illustration of the EvolKV framework. Compared to rule-based strategies (top row), EvolKV performs bottom-up, group-wise KV cache budget optimization using evolutionary search, progressively refining each layer group based on task-specific fitness feedback. Optimization Objectives of Evolutionary Compres￾sion Evolutionary algorithms generate candidate solu￾tions and evaluate their fitness, iteratively refi… view at source ↗
Figure 2
Figure 2. Figure 2: KV cache budget allocation of EvolKV—comparison, optimization trajectory, and allocation heatmap. cache budget c is 128 and the optimization objective is the recall score. During the evaluation, the KV cache budget is fixed at 128 [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Visualization of Mistral-7B-Instruct’s layer KV cache budgets in LongBench and Llama-3-8B￾Instruct’ budgets in GSM8K. Discussion on Optimized Budget Allocation We op￾timize EvolKV on Mistral-7B-Instruct with randomly selected 30 NarrativeQA instances at c = 128, 256, 512 and 1024. Our experiments reveal that EvolKV discov￾ers totally distinct KV cache allocation patterns com￾pared to heuristic approaches, … view at source ↗
Figure 4
Figure 4. Figure 4: Effects of layer grouping and KV cache ex￾pansion vs. KV cache optimization on LongBench. Generalization Analysis of EvolKV We optimize the layer-wise KV cache budgets of Mistral-7B-Instruct on a 30-sample subset of NarrativeQA with target aver￾age KV cache budget c = 256, 512, and 1024. The re￾sults are shown in Figure 4b and the detailed results are shown in [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Average performance comparison across six major task categories in LongBench between baseline methods and our proposed EvolKV on Llama-3-8B-Instruct and Mistral-7B-Instruct. Method Single-Document QA Multi-Document QA Summarization Few-shot Learning Synthetic Code Avg. NrtvQA Qasper MF-en HotpotQA 2WikiMQA Musique GovReport QMSum MultiNews TREC TriviaQA SAMSum PCount PRE Lcc RB-P KV Size = 128 SnapKV 22.12… view at source ↗
Figure 6
Figure 6. Figure 6: Performance comparison of baselines and EvolKV on Mistral-7B-Instruct and Llama-3-8B-Instruct across six LongBench task categories [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Evaluation of Llama-3-8B-Instruct and Mistral-7B-Instruct on NIAH at a KV cache budget of 128. 0 800 1600 2400 3200 4000 Generation Length 0 50 100 150 200 250 300 350 400 Time (s) FullKV SnapKV PyramidKV StreamingLLM EvolKV (a) Comparison results of inference time. 1k 2k 4k 8k 16k 32k Context Length 15 20 25 30 35 40 45 50 Peak Memory (GB) FullKV SnapKV PyramidKV StreamingLLM EvolKV (b) Comparison results… view at source ↗
Figure 8
Figure 8. Figure 8: Comparison of inference time and peak memory usage between baseline methods and EvolKV [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

54 extracted references · 10 canonical work pages · 2 internal anchors

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Tanweer Alam, Shamimul Qamar, Amit Dixit, and Mohamed Benaida. 2020. http://arxiv.org/abs/2007.12673 Genetic algorithm: Reviews, implementations, and applications

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  4. [4]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng X...

    Pith/arXiv arXiv 2023

  5. [5]

    Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2024. http://arxiv.org/abs/2308.14508 Longbench: A bilingual, multitask benchmark for long context understanding

    Pith/arXiv arXiv 2024

  6. [6]

    Nacim Belkhir, Johann Dr\' e o, Pierre Sav\' e ant, and Marc Schoenauer. 2015. https://doi.org/10.1007/978-3-319-31471-6_9 Parameter setting for multicore cma-es with\' z large populations . In Revised Selected Papers of the 12th International Conference on Artificial Evolution - Volume 9554, page 109–122, Berlin, Heidelberg. Springer-Verlag

    work page doi:10.1007/978-3-319-31471-6_9 2015

  7. [7]

    Peters, and Arman Cohan

    Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. http://arxiv.org/abs/2004.05150 Longformer: The long-document transformer

    Pith/arXiv arXiv 2020

  8. [8]

    Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Baobao Chang, Junjie Hu, and Wen Xiao. 2024. http://arxiv.org/abs/2406.02069 Pyramidkv: Dynamic kv cache compression based on pyramidal information funneling

    Pith/arXiv arXiv 2024

  9. [9]

    Qi Cao, Takeshi Kojima, Yutaka Matsuo, and Yusuke Iwasawa. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.550 Unnatural error correction: GPT -4 can almost perfectly handle unnatural scrambled text . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8898--8913, Singapore. Association for Computational Linguistics

    work page doi:10.18653/v1/2023.emnlp-main.550 2023

  10. [10]

    Yekun Chai, Yewei Fang, Qiwei Peng, and Xuhong Li. 2024 a . https://doi.org/10.18653/v1/2024.findings-emnlp.86 Tokenization falling short: On subword robustness in large language models . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1582--1599, Miami, Florida, USA. Association for Computational Linguistics

    doi:10.18653/v1/2024.findings-emnlp.86 2024

  11. [11]

    Yekun Chai, Qingyi Liu, Jingwu Xiao, Shuohuan Wang, Yu Sun, and Hua Wu. 2024 b . https://doi.org/10.18653/v1/2024.emnlp-main.182 Autoregressive pre-training on pixels and texts . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 3106--3125, Miami, Florida, USA. Association for Computational Linguistics

    work page doi:10.18653/v1/2024.emnlp-main.182 2024

  12. [12]

    Yekun Chai, Shuohuan Wang, Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.8 Clip-tuning: Towards derivative-free prompt learning with a mixture of rewards . In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 108--117, Abu Dhabi, United Arab Emirates. Association for Computatio...

    work page doi:10.18653/v1/2022.findings-emnlp.8 2022

  13. [13]

    Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. 2019. http://arxiv.org/abs/1904.10509 Generating long sequences with sparse transformers

    Pith/arXiv arXiv 2019

  14. [14]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. http://arxiv.org/abs/2110.14168 Training verifiers to solve math word problems

    Pith/arXiv arXiv 2021

  15. [15]

    Fu, Stefano Ermon, Atri Rudra, and Christopher Ré

    Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. http://arxiv.org/abs/2205.14135 Flashattention: Fast and memory-efficient exact attention with io-awareness

    Pith/arXiv arXiv 2022

  16. [16]

    Smith, and Matt Gardner

    Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, and Matt Gardner. 2021. http://arxiv.org/abs/2105.03011 A dataset of information-seeking questions and answers anchored in research papers

    Pith/arXiv arXiv 2021

  17. [17]

    Alexander Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir Radev. 2019. https://doi.org/10.18653/v1/P19-1102 Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1074--1084, Florence, Italy. Association for C...

    doi:10.18653/v1/p19-1102 2019

  18. [18]

    Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, and Jianfeng Gao. 2024. http://arxiv.org/abs/2310.01801 Model tells you what to discard: Adaptive kv cache compression for llms

    Pith/arXiv arXiv 2024

  19. [19]

    Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. 2019. https://doi.org/10.18653/v1/d19-5409 Samsum corpus: A human-annotated dialogue dataset for abstractive summarization . In Proceedings of the 2nd Workshop on New Frontiers in Summarization. Association for Computational Linguistics

    doi:10.18653/v1/d19-5409 2019

  20. [20]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and et al Amy Yang. 2024. http://arxiv.org/abs/2407.21783 The llama 3 herd of models

    Pith/arXiv arXiv 2024

  21. [21]

    Daya Guo, Canwen Xu, Nan Duan, Jian Yin, and Julian McAuley. 2023. http://arxiv.org/abs/2306.14893 Longcoder: A long-range pre-trained language model for code completion

    Pith/arXiv arXiv 2023

  22. [22]

    Müller, and Petros Koumoutsakos

    Nikolaus Hansen, Sibylle D. Müller, and Petros Koumoutsakos. 2003. https://doi.org/10.1162/106365603321828970 Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (cma-es) . Evolutionary Computation, 11(1):1--18

    doi:10.1162/106365603321828970 2003

  23. [23]

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. http://arxiv.org/abs/2011.01060 Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps

    Pith/arXiv arXiv 2020

  24. [24]

    John H. Holland. 1992. https://doi.org/10.7551/mitpress/1090.001.0001 Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence . The MIT Press

    doi:10.7551/mitpress/1090.001.0001 1992

  25. [25]

    Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. 2024. http://arxiv.org/abs/2404.06654 Ruler: What's the real context size of your long-context language models?

    Pith/arXiv arXiv 2024

  26. [26]

    Hui Huang, Bing Xu, Xinnian Liang, Kehai Chen, Muyun Yang, Tiejun Zhao, and Conghui Zhu. 2024. https://doi.org/https://doi.org/10.1016/j.inffus.2024.102480 Multi-view fusion for instruction mining of large language model . Information Fusion, 110:102480

    arXiv 2024

  27. [27]

    Luyang Huang, Shuyang Cao, Nikolaus Parulian, Heng Ji, and Lu Wang. 2021. http://arxiv.org/abs/2104.02112 Efficient attentions for long document summarization

    Pith/arXiv arXiv 2021

  28. [28]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. http://arxiv.org/abs/2310.06...

    Pith/arXiv arXiv 2023

  29. [29]

    Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. https://doi.org/10.18653/v1/P17-1147 T rivia QA : A large scale distantly supervised challenge dataset for reading comprehension . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601--1611, Vancouver, Canada. Assoc...

    doi:10.18653/v1/p17-1147 2017

  30. [30]

    Kennedy and R

    J. Kennedy and R. Eberhart. 1995. https://doi.org/10.1109/ICNN.1995.488968 Particle swarm optimization . In Proceedings of ICNN'95 - International Conference on Neural Networks, volume 4, pages 1942--1948 vol.4

    arXiv 1995

  31. [31]

    John R. Koza. 1992. Genetic programming: on the programming of computers by means of natural selection. MIT Press, Cambridge, MA, USA

    work page 1992

  32. [32]

    Tomáš Kočiský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. 2017. http://arxiv.org/abs/1712.07040 The narrativeqa reading comprehension challenge

    Pith/arXiv arXiv 2017

  33. [33]

    Xin Li and Dan Roth. 2002. https://aclanthology.org/C02-1150/ Learning question classifiers . In COLING 2002: The 19th International Conference on Computational Linguistics

    work page 2002

  34. [34]

    Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. 2024. http://arxiv.org/abs/2404.14469 Snapkv: Llm knows what you are looking for before generation

    Pith/arXiv arXiv 2024

  35. [35]

    Tianyang Liu, Canwen Xu, and Julian McAuley. 2023 a . http://arxiv.org/abs/2306.03091 Repobench: Benchmarking repository-level code auto-completion systems

    Pith/arXiv arXiv 2023

  36. [36]

    Zichang Liu, Aditya Desai, Fangshuo Liao, Weitao Wang, Victor Xie, Zhaozhuo Xu, Anastasios Kyrillidis, and Anshumali Shrivastava. 2023 b . http://arxiv.org/abs/2305.17118 Scissorhands: Exploiting the persistence of importance hypothesis for llm kv cache compression at test time

    Pith/arXiv arXiv 2023

  37. [37]

    Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, et al. 2024. Starcoder 2 and the stack v2: The next generation. arXiv preprint arXiv:2402.19173

    Pith/arXiv arXiv 2024

  38. [38]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, and et al Mohammad Bavarian. 2024. http://arxiv.org/abs/2303.08774 Gpt-4 technical report

    Pith/arXiv arXiv 2024

  39. [39]

    Matanel Oren, Michael Hassid, Nir Yarden, Yossi Adi, and Roy Schwartz. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.1043 Transformers are multi-state RNN s . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 18724--18741, Miami, Florida, USA. Association for Computational Linguistics

    doi:10.18653/v1/2024.emnlp-main.1043 2024

  40. [40]

    Phillip Rust, Jonas F Lotz, Emanuele Bugliarello, Elizabeth Salesky, Miryam de Lhoneux, and Desmond Elliott. 2022. Language modelling with pixels. arXiv preprint arXiv:2207.06991

    Pith/arXiv arXiv 2022

  41. [41]

    Fu, Zhiqiang Xie, Beidi Chen, Clark W

    Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, Max Ryabinin, Daniel Y. Fu, Zhiqiang Xie, Beidi Chen, Clark W. Barrett, Joseph Gonzalez, Percy Liang, Christopher R \'e , Ion Stoica, and Ce Zhang. 2023. https://api.semanticscholar.org/CorpusID:257495837 High-throughput generative inference of large language models with a single gpu . In International ...

    work page 2023

  42. [42]

    Luohe Shi, Hongyi Zhang, Yao Yao, Zuchao Li, and Hai Zhao. 2024. http://arxiv.org/abs/2407.18003 Keep the cost down: A review on methods to optimize llm' s kv-cache consumption

    Pith/arXiv arXiv 2024

  43. [43]

    Oscar Skean, Md Rifat Arefin, Dan Zhao, Niket Patel, Jalal Naghiyev, Yann LeCun, and Ravid Shwartz-Ziv. 2025. http://arxiv.org/abs/2502.02013 Layer by layer: Uncovering hidden representations in language models

    Pith/arXiv arXiv 2025

  44. [44]

    Rainer Storn and Kenneth Price. 1997. https://doi.org/10.1023/A:1008202821328 Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces . J. of Global Optimization, 11(4):341–359

    doi:10.1023/a:1008202821328 1997

  45. [45]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, and et al Lukas Blecher. 2023. http://arxiv.org/abs/2307.09288 Llama 2: Open foundation and fine-tuned chat models

    Pith/arXiv arXiv 2023

  46. [46]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. http://arxiv.org/abs/2108.00573 Musique: Multihop questions via single-hop question composition

    Pith/arXiv arXiv 2022

  47. [47]

    W. Vent. 1975. https://doi.org/https://doi.org/10.1002/fedr.19750860506 Rechenberg, ingo, evolutionsstrategie — optimierung technischer systeme nach prinzipien der biologischen evolution. 170 s. mit 36 abb. frommann-holzboog-verlag. stuttgart 1973. broschiert . Feddes Repertorium, 86(5):337--337

    work page doi:10.1002/fedr.19750860506 1975

  48. [48]

    Wenxuan Wang and Zhaopeng Tu. 2020. http://arxiv.org/abs/2011.03803 Rethinking the value of transformer components

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  49. [49]

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024. http://arxiv.org/abs/2309.17453 Efficient streaming language models with attention sinks

    Pith/arXiv arXiv 2024

  50. [50]

    Dongjie Yang, XiaoDong Han, Yan Gao, Yao Hu, Shilin Zhang, and Hai Zhao. 2024. http://arxiv.org/abs/2405.12532 Pyramidinfer: Pyramid kv cache compression for high-throughput llm inference

    Pith/arXiv arXiv 2024

  51. [51]

    Cohen, Ruslan Salakhutdinov, and Christopher D

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. http://arxiv.org/abs/1809.09600 Hotpotqa: A dataset for diverse, explainable multi-hop question answering

    Pith/arXiv arXiv 2018

  52. [52]

    Yang Zhang, Yanfei Dong, and Kenji Kawaguchi. 2024. http://arxiv.org/abs/2409.14381 Investigating layer importance in large language models

    Pith/arXiv arXiv 2024

  53. [53]

    Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, and Beidi Chen. 2023. http://arxiv.org/abs/2306.14048 H _2 o: Heavy-hitter oracle for efficient generative inference of large language models

    Pith/arXiv arXiv 2023

  54. [54]

    Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, and Dragomir Radev. 2021. http://arxiv.org/abs/2104.05938 Qmsum: A new benchmark for query-based multi-domain meeting summarization

    Pith/arXiv arXiv 2021