Pith. sign in

REVIEW 4 major objections 6 minor 30 references

DAC: A Dynamic Attention-aware Approach for Task-Agnostic Prompt Compression

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A prompt compression method that blends token entropy with accumulated attention scores beats entropy-only baselines on LongBench, GSM8K, and BBH.

desk verdict A useful empirical idea for prompt compression, but Eq. (7) as written deletes 93% of the prompt in the first stage, so the paper's central mechanism is internally inconsistent until that sign error is fixed. read the letter →

arxiv 2507.11942 v1 pith:SOY6UCY4 submitted 2025-07-16 cs.CL

Yi Zhao , Zuchao Li , Hai Zhao , Baoyuan Qi , Guoming Liu This is my paper
classification cs.CL
keywords promptcompressiontask-agnosticinformationentropyattentionscoresdynamicin-contextlearninglong-contextLLMLongBench
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Task-agnostic prompt compression removes redundant tokens from a prompt without knowing the downstream task. Existing entropy-based methods treat each token's information content as fixed and ignore which tokens the model actually attends to. This paper proposes DAC, which scores each token by a weighted blend of information entropy and accumulated attention, and recomputes the entropy in stages as compression proceeds. On LongBench, DAC reports a 4.03-point gain over entropy-only baselines and an average 1.33-point gain over the previous state of the art, with consistent advantages on GSM8K and BBH. If correct, this would make cheap, task-agnostic compression a more reliable way to let black-box LLMs handle longer contexts at lower cost.

What carries the argument

The central object is the integrated compression metric, which combines a token's information entropy $I_t(x)$ with its accumulated attention score $s_t$—the mean, across all layers and heads, of the attention weights the token receives from all other tokens. The paper compares additive and multiplicative fusion and selects the additive form with $\alpha=0.8$. The second mechanism is the dynamic procedure: compression runs in $D$ stages, each with its own compression rate $\Delta\tau=\tau^{1/D}+\Delta P$, where $\Delta P$ is the fraction of tokens retained in the previous stage because of a limit on consecutive-token deletion. At each stage, entropy is recomputed by a small language model, a percentile threshold on the integrated metric is applied, and tokens whose predecessor was already removed are protected from deletion. These two mechanisms together carry the claimed improvement.

What would settle it

Take 200 prompts from NarrativeQA, compute the accumulated attention score vectors from the small compressor model (Qwen2-0.5B) and from the target LLM (Qwen2-7B), and measure their Pearson correlation. If the average correlation is near zero, or if removing the tokens with highest small-model attention does not lower the target LLM's F1 score, then the attention term is not doing the work the paper attributes to it.

Watch

Extended reading notes

Core claim

The paper claims that the two blind spots of entropy-based prompt compression are measurable and fixable. First, attention-critical tokens—the ones that receive high accumulated attention from other tokens—are not the same as high-entropy tokens, with an average Pearson correlation of 0.095 on NarrativeQA; removing them hurts downstream F1 even more than random token removal. Second, entropy is not static: after a token is removed, the entropy of the following tokens shifts, and the shift grows with the compression ratio. DAC addresses both with an integrated metric $M_t^a=(1-\alpha)I_t(x)+\alpha s_t$ (choosing $\alpha=0.8$) and with a staged procedure that recomputes entropy via a small language model, limits consecutive-token deletion, and adjusts each stage's compression rate. The paper reports that this dynamic attention-aware procedure outperforms the three strongest existing methods across LongBench, GSM8K, and BBH, on both Qwen2 and LLaMA3 model families.

Load-bearing premise

The load-bearing premise is that accumulated attention scores computed from the small language model used for compression identify the same attention-critical tokens as the target LLM; if the small model attends to different tokens, the attention term could degrade compression rather than help it.

Editorial extensions

If this is right

  • At a 50% compression rate, DAC's all-task average on LongBench is 37.76, and at 20% retention it is 32.63, both above the entropy-only baselines, meaning users can compress harder without losing as much task performance.
  • Because the compressor is a 0.5B or 1B model, it can be applied to black-box LLMs without fine-tuning or architecture changes, an advantage the paper highlights over trained-compressor baselines.
  • The dynamic procedure lets DAC hold up better at high compression: on BBH at $\tau=0.2$ it beats LLMLingua by 0.8 points, while at $\tau=0.5$ it trails by only 0.67.
  • The added compression time is small relative to inference savings: on a 12,908-token prompt compressed to 20%, the total compression-plus-inference time stays below the full-prompt inference time, and the paper notes that larger production LLMs amplify the savings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper validates the small-model proxy only for entropy similarity across model sizes, not for attention similarity; a direct test would measure the Pearson correlation between the small model's accumulated attention scores and the target LLM's on the same sequences. If that correlation is weak, the attention term could be a liability rather than a help.
  • Because the additive fusion with $\alpha=0.8$ weights attention four times more than entropy, the metric is close to a pure attention ranking; tuning $\alpha$ per dataset, task type, or compression rate is an obvious extension the paper does not explore.
  • The dynamic entropy recomputation is the main source of added compression time. A cheaper approximation that recomputes entropy only for tokens whose predecessor was removed could preserve the entropy-shift insight while cutting overhead, and could be tested against the current algorithm.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes DAC, a task-agnostic hard prompt compression method that integrates token-level information entropy with accumulated attention scores computed by a small language model. Compression is performed in D iterative stages; at each stage the information entropy is recomputed, tokens with low integrated metric are deleted unless they directly follow an already-deleted token, and the per-stage deletion budget is adjusted by a term ΔP. The method is evaluated on LongBench, GSM8K and BBH with Qwen2 and LLaMA3 model pairs, reporting gains over entropy-only baselines and prior state-of-the-art methods at compression rates τ=0.5 and τ=0.2. The paper also includes ablation studies, an overhead analysis, and a publicly available code repository.

Significance. If the empirical results survive the issues below, the paper would provide a useful, lightweight improvement over entropy-only prompt compression, with a practical recipe (small-model attention plus dynamic entropy recomputation) that appears to transfer across two model families. The strengths are the two motivating observations, the ablation isolating the attention and dynamic components, the inclusion of multiple benchmarks and model pairs, the overhead measurement, and the public code. The significance is moderated, however, by an internal inconsistency in the stage-wise compression formula, by the absence of a validation split when selecting the fusion weight α, and by the unverified transfer of attention-critical tokens from the small model to the target LLM. These points must be resolved before the reported state-of-the-art claims can be fully assessed.

major comments (4)
  1. [Sec. 4.2, Eq. (7) and Algorithm 1] The per-stage compression amount is defined as Δτ = τ^(1/D) + ΔP, but Algorithm 1 uses Δτ as a percentile threshold for deletion (Eq. (8), line 6). Since tokens below the threshold are deleted, a larger Δτ means more deletion. With the paper's own settings τ=0.5 and D=L_input/100 capped at 15, τ^(1/D) ≈ 0.933, so the first stage would delete roughly 93.3% of tokens instead of the 6.7% needed to keep τ^(1/D) of them; repeating this over D stages would leave a tiny fraction of the prompt, contradicting the reported ~50% kept length. The intended formula appears to be Δτ = 1 − τ^(1/D) + ΔP (or an equivalent retention-based update). The manuscript as printed does not permit reproduction of the reported compression rates. Please correct Eq. (7), verify Algorithm 1 against the released code, and report the achieved τ per benchmark condition.
  2. [Sec. 5.2 and Table 1] The fusion weight α=0.8 is selected by maximizing F1 on the same Single-Doc QA datasets (NarrativeQA, QASPER, MultiFieldQA) that are then reported in the main LongBench results. Because the selection is made on the evaluation set, the reported gains on those rows partly reflect tuning on the test set, and the headline 4.03-point improvement over entropy-only baselines is not obtained under a clean held-out protocol. Please either use a separate validation split for α, show that the fusion-strategy ranking is stable across task categories, or exclude the tuned categories from the headline average. No variance or significance testing is reported, so the 1.33-point average gain over the prior SOTA is not accompanied by an uncertainty estimate.
  3. [Sec. 4.1, Appendix A, and Algorithm 1] Observation 1 validates the importance of attention-critical tokens using Qwen2-7B attention, but Algorithm 1 computes the accumulated attention scores from the small SLM only. Appendix A demonstrates entropy similarity across model sizes, but it provides no analogous evidence that attention-critical tokens identified by the 0.5B SLM coincide with those of the 7B evaluation model. If the small model's attention pattern differs from the target LLM's, the attention term could preserve the wrong tokens and degrade compression. Please provide a quantitative overlap analysis (e.g., top-k agreement or Jaccard similarity of attention-critical token sets) between Qwen2-0.5B and Qwen2-7B, and between LLaMA-3.2-1B and LLaMA-3.1-8B.
  4. [Sec. 5.1] The statement that 'all compression rates in experiments are actual compression rates' is not supported by any reported achieved lengths. Because DAC's consecutive-compression exception and dynamic ΔP cause the kept fraction to drift from the nominal τ, the reader cannot verify that all methods are compared at equal actual compression. Please report the empirical τ (or the number of kept tokens) for each method, task, and compression-rate condition, and state the tolerance used for 'approximately consistent' final lengths.
minor comments (6)
  1. [Throughout] There are several typos: 'simular' (Related Work), 'mertic' (Sec. 5.1), 'Infromation Entropy' (Figure 2), and 'compresstion' (Appendix B).
  2. [Algorithm 1, line 7] The line 'for j in range(ex)' is not well-defined; it should be 'for j in range(len(ex))' and the deletion loop should iterate over a copy of the current sequence so that deleting tokens does not skip subsequent indices.
  3. [Eq. (9)] The notation uses ex for both the output set and the current sequence; please use distinct symbols (e.g., S for the current sequence and ex for the compressed output) to avoid ambiguity.
  4. [Figure 5] The legend says compression time is indicated 'by blue', but color labels are not legible in black-and-white printing; please add data labels or use a grayscale-safe pattern.
  5. [Sec. 5.5] Table 4 reports only category averages for the LLaMA3 series; per-task scores would help verify the claim that DAC achieves state-of-the-art results 'in most task types'.
  6. [References] LongLLMLingua (Jiang et al., 2024) appears in the reference list but is neither discussed in the Related Work nor included as a baseline; please either add it to the relevant discussion or remove the reference.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: DAC's metric and dynamic schedule are not defined in terms of benchmark results, and alpha=0.8 is standard hyperparameter selection rather than a fitted prediction.

full rationale

DAC's compression metric (Eq. 5) is computed from information entropy (Eq. 2) and accumulated attention scores (Eqs. 3-4) on the input prompt via a small LM, not from the target outputs or evaluation labels; no equation defines the metric, threshold (Eq. 8), or dynamic rate (Eq. 7) in terms of the reported F1 or EM scores. The fusion weight alpha=0.8 is chosen in Sec. 5.2 on Single-Doc QA and then applied unchanged in Table 1 and Table 2, but this is ordinary hyperparameter selection rather than fitting a parameter to the predicted quantity itself. The same fixed alpha transfers to Multi-Doc QA, Summarization, Few-shot, GSM8K, BBH, and the LLaMA3 experiments, so the headline improvements do not reduce to the fit by construction. Self-citations such as PyramidInfer and the KV-cache survey motivate attention-aware compression but are not the load-bearing evidence for the main claims, which are benchmarked against external baselines. Two concerns noted in the manuscript are weaknesses rather than circularity: Appendix A validates only entropy similarity across model sizes, not attention-score similarity, so the small-model attention proxy is an unverified assumption; and Eq. 7 appears algebraically inconsistent with the stated budget and Algorithm 1's deletion rule, which is an internal correctness question, not a circular reduction. No step of the claimed derivation is equivalent to its own inputs by construction.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the empirical validity of entropy and attention as token-importance signals, plus a dynamic update rule. No new physical or conceptual entities are introduced. The main free parameters are the fusion weight and the number of dynamic iterations.

free parameters (2)
  • alpha (fusion weight) = 0.8
    Selected by grid search over {0.2, 0.4, 0.6, 0.8} and multiplicative on Single-Doc QA (Sec 5.2, Figure 4); used across all other experiments.
  • D (dynamic iterations) = L_input/100, capped at 15
    Chosen by hand with no sensitivity analysis (Sec 5.1).
assumptions (4)
  • domain assumption Information entropy from a small language model approximates the entropy of the target LLM well enough for compression decisions.
    Appendix A gives Pearson correlation 0.835 between Qwen2-0.5B and Qwen2-72B, used to justify this.
  • domain assumption High accumulated attention identifies tokens that must be preserved for downstream performance.
    Observation 1 shows dropping attention-critical tokens degrades F1, but this is demonstrated only on one dataset family with one model.
  • domain assumption Tokens whose entropy shifts after context removal can be protected by limiting consecutive token deletion and recomputing entropy.
    The paper provides descriptive evidence in Figure 3, not a formal guarantee.
  • domain assumption Percentile-threshold filtering of the integrated metric is a valid token-selection policy.
    Assumed by the algorithm (Eq. 8) without theoretical justification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DAC: A Dynamic Attention-aware Approach for Task-Agnostic Prompt Compression." pith.science (2026). https://pith.science/paper/SOY6UCY4

@misc{pith2026250711942,
  author       = {Pith},
  title        = {Pith review of: DAC: A Dynamic Attention-aware Approach for Task-Agnostic Prompt Compression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SOY6UCY4}},
  note         = {Machine review of arXiv:2507.11942}
}
read the original abstract

Task-agnostic prompt compression leverages the redundancy in natural language to reduce computational overhead and enhance information density within prompts, especially in long-context scenarios. Existing methods predominantly rely on information entropy as the metric to compress lexical units, aiming to achieve minimal information loss. However, these approaches overlook two critical aspects: (i) the importance of attention-critical tokens at the algorithmic level, and (ii) shifts in information entropy during the compression process. Motivated by these challenges, we propose a dynamic attention-aware approach for task-agnostic prompt compression (DAC). This approach effectively integrates entropy and attention information, dynamically sensing entropy shifts during compression to achieve fine-grained prompt compression. Extensive experiments across various domains, including LongBench, GSM8K, and BBH, show that DAC consistently yields robust and substantial improvements across a diverse range of tasks and LLMs, offering compelling evidence of its efficacy.

Figures

Figures reproduced from arXiv: 2507.11942 by the authors.

Figure 1
Figure 1. Framework of our proposed DAC for task-agnostic prompt compression. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Left: The information entropy (red) and corresponding accumulated attention scores (blue) on one [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Left: Entropy differences highlight significant shifts in tokens whose preceding tokens were removed [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: The performance of DAC method with differ [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The comparison of compress and inference [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: The performance of DAC at various compres [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 8
Figure 8. Figure 8: Quantitative analysis of the entropy similar [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 7
Figure 7. Figure 7: Visualization of the entropy similarities between different model parameter sizes (Qwen2 series models). [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 9
Figure 9. Figure 9: Cases study on GSM8K dataset. Original Prompt Distinguish deductively valid arguments from formal fallacies. Q: "It is not always easy to see who is related to whom -- and in which ways. The following argument pertains to this question: To begin with, Lesley is a close…
Figure 10
Figure 10. Figure 10: Cases study on formal_fallacies of BBH dataset. [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

30 extracted references · 7 canonical work pages

  1. [1]

    Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2024. https://doi.org/10.18653/v1/2024.acl-long.172 L ong B ench: A bilingual, multitask benchmark for long context understanding . In Proceedings of the 62nd Annual Meeting of the Association for ...

    doi:10.18653/v1/2024.acl-long.172 2024

  2. [2]

    Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165

    arXiv 2020

  3. [3]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

  4. [4]

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, et al. 2024. A survey on in-context learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1107--1128

    2024

  5. [5]

    Tao Ge, Jing Hu, Haixun Wang, Si-Qing Chen, and Furu Wei. 2023. https://www.microsoft.com/en-us/research/publication/in-context-autoencoder-for-context-compression-in-a-large-language-model/ In-context autoencoder for context compression in a large language model . In ICLR 2024

    work page 2023

  6. [6]

    Xijie Huang, Li Lyna Zhang, Kwang-Ting Cheng, Fan Yang, and Mao Yang. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.758 Fewer is more: Boosting math reasoning with reinforced context pruning . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13674--13695, Miami, Florida, USA. Association for Computationa...

    work page doi:10.18653/v1/2024.emnlp-main.758 2024

  7. [7]

    Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.825 LLML ingua: Compressing prompts for accelerated inference of large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13358--13376, Singapore. Association for Computation...

    doi:10.18653/v1/2023.emnlp-main.825 2023

  8. [8]

    Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2024. https://doi.org/10.18653/v1/2024.acl-long.91 L ong LLML ingua: Accelerating and enhancing LLM s in long context scenarios via prompt compression . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa...

    doi:10.18653/v1/2024.acl-long.91 2024

Show all 30 references
  1. [9]

    Hoyoun Jung and Kyung-Joong Kim. 2024. Discrete prompt compression with reinforcement learning. IEEE Access

    2024

  2. [10]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \"u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \"a schel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Proc...

    2020

  3. [11]

    Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.391 Compressing context to enhance inference efficiency of large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, ...

    2023 doi

  4. [12]

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems, 6:87--100

    2024

  5. [13]

    Zichang Liu, Aditya Desai, Fangshuo Liao, Weitao Wang, Victor Xie, Zhaozhuo Xu, Anastasios Kyrillidis, and Anshumali Shrivastava. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/a452a7c6c463e4ae8fbdc614c6e983e6-Paper-Conference.pdf Scissorhands: Exploiting the...

    2023

  6. [14]

    Jesse Mu, Xiang Li, and Noah Goodman. 2024. Learning to compress prompts with gist tokens. Advances in Neural Information Processing Systems, 36

    2024

  7. [15]

    Vicky Zhao, Lili Qiu, and Dongmei Zhang

    Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor R \"u hle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Dongmei Zhang. 2024. https://doi.org/10.18653/v1/2024.findings-acl.57 LLML ingua-2: Data distillation for efficie...

    2024 doi

  8. [16]

    Luohe Shi, Hongyi Zhang, Yao Yao, Zuchao Li, and Hai Zhao. 2024. https://arxiv.org/abs/2407.18003 Keep the cost down: A review on methods to optimize llm' s kv-cache consumption . Preprint, arXiv:2407.18003

    2024 arXiv

  9. [17]

    Yutao Sun, Li Dong, Yi Zhu, Shaohan Huang, Wenhui Wang, Shuming Ma, Quanlu Zhang, Jianyong Wang, and Furu Wei. 2024. You only cache once: Decoder-decoder architectures for language models. arXiv preprint arXiv:2405.05254

    2024 arXiv

  10. [18]

    Mirac Suzgun, Nathan Scales, Nathanael Sch \"a rli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and Jason Wei. 2023. https://doi.org/10.18653/v1/2023.findings-acl.824 Challenging BIG -bench tasks and whether chain-of-thought c...

    2023 doi

  11. [19]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  12. [20]

    David Wingate, Mohammad Shoeybi, and Taylor Sorensen. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.412 Prompt compression and contrastive conditioning for controllability and toxicity reduction in language models . In Findings of the Association for Computational Ling...

    2022 doi

  13. [21]

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. 2023. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864

    2023 arXiv

  14. [22]

    Fangyuan Xu, Weijia Shi, and Eunsol Choi. 2023. https://arxiv.org/abs/2310.04408 Recomp: Improving retrieval-augmented lms with compression and selective augmentation . Preprint, arXiv:2310.04408

    2023 arXiv

  15. [23]

    Dongjie Yang, Xiaodong Han, Yan Gao, Yao Hu, Shilin Zhang, and Hai Zhao. 2024 a . https://doi.org/10.18653/v1/2024.findings-acl.195 P yramid I nfer: Pyramid KV cache compression for high-throughput LLM inference . In Findings of the Association for Computational Linguistics: A...

    2024 doi

  16. [24]

    Yifei Yang, Zouying Cao, Qiguang Chen, Libo Qin, Dongjie Yang, Hai Zhao, and Zhi Chen. 2024 b . https://arxiv.org/abs/2410.18517 Kvsharer: Efficient inference via layer-wise dissimilar kv cache sharing . Preprint, arXiv:2410.18517

    2024 arXiv

  17. [25]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024 a . Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36

    2024

  18. [26]

    Yao Yao, Zuchao Li, and Hai Zhao. 2024 b . https://doi.org/10.18653/v1/2024.findings-naacl.183 G o T : Effective graph-of-thought reasoning in language models . In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2901--2921, Mexico City, Mexico. Ass...

    2024 doi

  19. [27]

    Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher R\' e , Clark Barrett, Zhangyang "Atlas" Wang, and Beidi Chen. 2023 a . https://proceedings.neurips.cc/paper_files/paper/2023/file/6ceefa7b15572587b78ecfcebb28...

    2023

  20. [28]

    Zhuosheng Zhang, Yao Yao, Aston Zhang, Xiangru Tang, Xinbei Ma, Zhiwei He, Yiming Wang, Mark Gerstein, Rui Wang, Gongshen Liu, and Hai Zhao. 2023 b . https://arxiv.org/abs/2311.11797 Igniting language intelligence: The hitchhiker's guide from chain-of-thought reasoning to lang...

    2023 arXiv

  21. [29]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  22. [30]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.