Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

ECoRAG: Evidentiality-guided Compression for Long Context RAG

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Evidentiality-guided compression—keeping only sentences that support the correct answer and adding more only until the context is sufficient—lets retrieval-augmented generation answer open-domain questions more accurately than feeding the…

desk verdict A practical compression framework with genuine gains, held back by missing significance tests and an un-audited labeler; deserves review with revisions. read the letter →

arxiv 2506.05167 v2 pith:H3KYW26S submitted 2025-06-05 cs.CL cs.AIcs.IR

Yeonseok Jeong , Jinsu Kim , Dohyeon Lee , Seung-won Hwang This is my paper
classification cs.CLcs.AIcs.IR
keywords retrieval-augmentedgenerationcontextcompressionevidentialityopen-domainquestionansweringadaptivesentencerankingRAGefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ECoRAG tries to prove that the right way to compress long retrieval contexts for open-domain question answering is to keep only sentences that help the reader LLM produce the correct answer, and to stop adding sentences as soon as the accumulated evidence is sufficient. The paper defines evidentiality hierarchically—a sentence is strong evidence if the answer can be generated with it but not without it, weak evidence if it supports without answering, and a distractor if it actively misleads—and trains a dual-encoder compressor to rank sentences in that order. A small (770M-parameter) evaluator then reads the ranked sentences and adds more only when the current compression is judged non-evidential, so the compression ratio adapts per question. If the claim holds, long-context RAG can drop from about 14,000 input tokens to a few hundred per question while matching or beating the accuracy of feeding the reader everything, which would make retrieval augmentation substantially cheaper and less prone to distractor-driven errors. The authors report gains on NQ, TQA, and WQ with GPT-4o-mini and across three other reader models, plus a HotpotQA result indicating that the ranking tracks human-annotated evidence.

What carries the argument

The load-bearing object is the three-way evidentiality label: strong evidence (with the sentence the LLM can generate the gold answer; without it it cannot), weak evidence (it does not answer alone but does not interfere), and distractor (it interferes). The compressor is a dual encoder initialized from Contriever, trained with two InfoNCE losses, $L_{se}$ and $L_{we}$, that enforce strong > weak > distractor in the similarity score between question and sentence. The second mechanism is evidentiality reflection: a 770M-parameter Flan-T5-large evaluator, trained to emit a single special token <EVI> or <NOT>, tests the accumulated compression from the top-ranked sentence upward and adds evidence in chunks of four until the context is judged evidential or a limit of 20 evidence pieces is reached. That loop is what turns fixed-ratio compression into question-dependent compression.

What would settle it

Re-mine all evidentiality labels on the same training questions with a different decoding temperature or sampling budget, or with the gold answer removed from the mining prompt, and retrain the compressor from scratch; if ECoRAG's edge over standard RAG on NQ, TQA, and WQ disappears under any of these regenerations, the reported gains depend on the specific labeling procedure rather than on evidentiality itself. A sharper version: replace Flan-UL2 with a model verified to have no exposure to the NQ/TQA/WQ test splits and check whether ECoRAG still beats standard RAG.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that evidentiality—not relevance and not answerability—is the signal that should drive context compression in LLM-based RAG. The paper builds its case by labeling each sentence of 100 DPR-retrieved documents as strong evidence, weak evidence, or distractor using leave-one-out answer generation with Flan-UL2, training a Contriever-initialized dual encoder to rank sentences accordingly, and then using a Flan-T5-large evaluator to adaptively grow the compressed set until the accumulated context is judged sufficient. On Natural Questions, TriviaQA, and WebQuestions, ECoRAG reports higher exact match than standard RAG (36.48 vs 36.09, 65.34 vs 56.21, and 30.17 vs 21.11 with GPT-4o-mini) while using roughly 440–630 tokens instead of about 14,000, and it also outperforms prior compressors including RECOMP and CompAct. The ablation study attributes part of the gain to the evidentiality labels themselves and part to the adaptive evaluator, and the HotpotQA NDCG experiment aligns the learned ranking with human-annotated evidence.

Load-bearing premise

The whole method rests on the assumption that the sentence-level 'strong evidence' labels mined from Flan-UL2 by comparing answers with and without each sentence are correct and stable enough to train a compressor that transfers to test questions where the gold answer is hidden; the paper does not specify how the binary comparisons are computed, and its data-exposure audit covers the Flan-T5 evaluator but not the Flan-UL2 model that produces the labels.

Editorial extensions

If this is right

  • Long-context RAG can run at roughly 3–5% of the tokens: on the three benchmarks the compressed context is 441–632 tokens instead of about 14,000, and accuracy does not drop.
  • Question-dependent compression replaces fixed ratios: the evaluator stops adding evidence as soon as the accumulated sentences are judged sufficient, so easy questions get shorter contexts than hard ones.
  • Evidentiality ranking generalizes across readers: results with Flan-UL2, Gemma2, and Llama3 show the same compressor improves on or at least matches the best compression baselines without retraining for each reader.
  • The framework scales beyond 100 documents: with 1,000 retrieved documents, uncompressed RAG on NQ collapses to EM 0.44 while ECoRAG holds 35.51, suggesting the method is robust when retrieval noise grows.
  • The same evidence-ranking signal transfers to multi-hop QA: on HotpotQA and LongBench multi-hop tasks, ECoRAG outperforms standard RAG and RECOMP, which the paper attributes to its explicit handling of weak evidence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One consequence the authors leave implicit: if evidentiality labels are this useful for compression, the same three-way labels could be used as a training signal for the retriever itself, collapsing the retrieve-then-compress pipeline into a single evidence-scoring step.
  • The paper trains on gold-answer supervision and notes in Section 7 that label mining is computationally expensive; a natural extension is to mine evidentiality without the gold answer, e.g., by treating sentences whose removal changes the model's own generated answer as evidence, which would make the method applicable where no answer key exists.
  • Because the evaluator emits a single token rather than generating a summary, the reflection loop is cheap; the paper does not explore whether the same evaluator could also score answer confidence or trigger re-retrieval when no sentence reaches the evidence threshold.
  • The ablation shows the evaluator matters most on TQA (+1.80 EM); a testable prediction of the paper's view is that evidentiality-guided compression will show the largest benefit on datasets with many lexically relevant but misleading passages, and near-zero benefit on datasets where retrieved documents are already concise.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes ECoRAG, a two-stage framework for open-domain QA with RAG: (1) a sentence-level extractive compressor trained with evidentiality labels mined from an LLM (Flan-UL2), and (2) a lightweight Flan-T5-based 'evidentiality evaluator' that adaptively decides how many top-ranked sentences to keep, stopping once it judges the compressed context sufficient. The central claim, stated in the abstract and Table 1, is that ECoRAG outperforms standard RAG and existing compression baselines in EM/F1 on NQ, TQA, and WQ while using far fewer tokens (e.g., roughly 14,000 tokens down to 441–632 tokens with GPT-4o-mini). The paper also reports ablations, reader generalization across GPT-4o-mini, Flan-UL2, Llama3, and Gemma2, retriever generalization with Contriever, multi-hop experiments, and latency measurements.

Significance. If the claims hold, ECoRAG is a practically useful contribution: it offers a concrete, re-implementable recipe for reducing long-context RAG inputs by more than an order of magnitude while maintaining or improving answer accuracy, and it demonstrates the value of evidentiality as a training signal. The paper's strengths are its breadth of experiments (three ODQA datasets, four reader models, two retrievers, multi-hop benchmarks), the inclusion of a human-annotated evidence alignment study on HotpotQA, the efficiency analysis, and the release of code. Those strengths are substantial. However, the load-bearing adaptive component is validated in a partially self-referential way, and the central NQ result depends on a small difference that is not accompanied by uncertainty quantification, so the evidence as presented is not yet conclusive.

major comments (3)
  1. [§3.2.1, §5.2, Table 3] The evidentiality evaluator is trained on labels produced by Flan-UL2 in §3.1.1, and its validation in §5.2 (Figures 4 and 5) uses Flan-UL2's own labels as ground truth. This establishes only that the evaluator approximates the labeler, not that it detects evidence sufficiency for the deployed reader GPT-4o-mini. The concern is load-bearing: Table 3 (row D) shows that removing the evaluator drops NQ EM from 36.48 to 35.71, below the standard-RAG baseline of 36.09 in Table 1. The paper should validate the evaluator against an independent evidence ground truth (e.g., human-annotated evidence or a reader-specific oracle) and should report whether the evaluator's stopping decisions correlate with the actual answer correctness of the reader across compression lengths.
  2. [§3.1.1, §B.2, Figure 3] The definition of strong evidence as sentences 'with which the LLM can generate the correct answer and without which it cannot' is not operationalized: the paper does not specify the number of samples, decoding temperature, or probability threshold used to convert this definition into binary labels. This is not merely a reproducibility detail, because the mining prompt includes the gold answer (Figure 3) and Flan-UL2 is the same model family that later validates the evaluator. If Flan-UL2 has memorized NQ/TQA/WQ test answers or produces noisy labels, the downstream gains in Table 1 could be inflated. The contamination audit in Section B.2 and Table 19 covers only the Flan-T5 evaluator, not the Flan-UL2 labeler; the paper should add an exposure audit for Flan-UL2 and report label-quality statistics (e.g., agreement with a different labeler or with human evidence).
  3. [Tables 1 and 3] The headline improvements are small in several key comparisons (NQ EM: 36.48 vs. 36.09 standard RAG; ablation: 36.48 vs. 35.71 without evaluator), yet the paper reports no confidence intervals, bootstrap estimates, or significance tests. Given that the adaptive component is what pushes ECoRAG above standard RAG on NQ, the paper should report variance or significance for at least the central NQ comparisons, and should state whether the reported differences are stable across random seeds and across the choice of token limit and increment step in §B.1.
minor comments (4)
  1. [Eq. (3)] The notation below Eq. (3) says 's±j = sim(q, d−j)', which is inconsistent with the text stating that the negative set is D− ∪ D+; this should read sim(q, d±j) with d±j ranging over the union of weak evidence and distractors.
  2. [§B.1] There is a typo in the training description: 'we we used 8 positive contexts' should be 'we used 8 positive contexts'.
  3. [Tables 13 and 1] The token count for RECOMP (extractive) on NQ is reported as 165 in Table 13 but as 662 in Table 1; even accounting for different reader tokenizers, this discrepancy should be explained or corrected.
  4. [Abstract] The phrase 'To reduce RAG overhead, from longer context, context compression is necessary' is grammatically awkward and should be revised.

Circularity Check

1 steps flagged · score 3.0 of 10

End-task gains for GPT-4o-mini are not circular, but the evidentiality evaluator is trained and validated against the same Flan-UL2 labeler, so its reported agreement is a distillation check rather than independent evidence of sufficiency.

  1. self definitional [Section 3.2.1 and Section 5.2 (RQ2), Figures 4-5]
    "Section 3.2.1: 'we reuse our evidentiality labels obtained from the LLM in Section 3.1.1 and distill them from our reader LLM into smaller model, Flan-T5-large ... to build the evaluator.' Section 5.2: 'we define ground-truth labels for retrieved documents as either <EVI>, which lead to generating the correct answer as in Section 3.2.1, or <NOT>' and 'our evidentiality evaluator, despite its much smaller size (770M), closely approximates the performance of Flan-UL2 (-0.08p%).'"

    The evaluator's supervision is the set of evidentiality labels mined by Flan-UL2, and its evaluation 'ground truth' is defined by the same procedure ('as in Section 3.2.1') using the same labeler. The reported -0.08p% gap from Flan-UL2 therefore measures how well the student reproduced the teacher's labels on held-out questions; it does not independently certify that the stopping signal detects evidence sufficiency for a different reader such as GPT-4o-mini. The main Table 1 results are not circular because the reader was not used to generate labels, but the adaptive component that contributes +0.77 EM on NQ (Table 3) is validated only against the labeler's own judgments.

full rationale

The central end-task comparison (Table 1) is a standard supervised pipeline: Flan-UL2 mines sentence-level evidentiality labels on training questions; a Contriever-based compressor and Flan-T5-large evaluator are trained on those labels; and final QA accuracy is measured on held-out test sets with GPT-4o-mini, a reader that did not produce the labels. No equation in the paper reduces end-task accuracy to the training labels by construction, and the compressor receives an external check against human-annotated HotpotQA evidence (Table 2). The main circularity concern is limited to the evaluator's validation: since the same Flan-UL2 model defines 'evidentiality' and supplies the 'ground-truth' labels against which the distilled evaluator is measured, the reported agreement is a teacher-student distillation check, not independent evidence that the stopping signal generalizes to unseen readers. This is a real limitation, especially because the evaluator contributes the NQ gain over standard RAG (Table 3: 36.48 vs 35.71, with standard RAG at 36.09), but it does not make the GPT-4o-mini results circular. Self-citations in the paper (Lee et al. 2021, Han et al. 2023, Song et al. 2024, Jeong et al. 2025) are background or future-work references and are not load-bearing for the method's correctness. Overall score 3: one self-referential validation in a load-bearing component; the central end-task claim retains independent content.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The framework introduces a labeled construct (hierarchical evidentiality), which is a training signal rather than an entity in the schema's sense. The free parameters are mostly standard, with the evidence cap of 20 being the most consequential since it directly controls the compression ratio. The axioms are domain assumptions about the reliability of LLM-generated labels and the transferability of a gold-answer-conditioned model to test time.

free parameters (5)
  • evidence_limit = 20
    Maximum number of evidence pieces collected in adaptive compression; set to match RECOMP's compression level (Section B.1).
  • increment_step = 4
    Number of evidence pieces added per reflection iteration to reduce latency (Section B.1).
  • temperature_tau = 1.0
    Temperature for both InfoNCE losses Lwe and Lse (Section B.1).
  • hard_negative_ratio = 0.15 weak : 0.85 distractor
    Ratio of weak to distractor negatives used in computing Lse (Section B.1).
  • evaluator_label_ratio = 1 EVI : 3 NOT
    Ratio of positive to negative sentences used to train the evidentiality evaluator (Section B.1).
assumptions (5)
  • domain assumption Flan-UL2's leave-one-out answer generation provides a stable oracle for whether a sentence enables the correct answer. Sentence-level labels (strong/weak/distractor) derived by this oracle are correct and reproducible.
    Used in Section 3.1.1 to define evidentiality and to mine training labels; no sampling procedure or thresholds are specified.
  • domain assumption A dot-product dual-encoder (Contriever initialization) can rank sentences by evidentiality sufficiently well for downstream QA.
    The compressor uses sim(q,d) = EQ(q) * ED(d) (Section 3.1.2). This assumes the encoded space captures evidential support.
  • domain assumption Distractor sentences can be identified by a second condition ('does not interfere with evidence') and this interference is measurable from LLM outputs.
    Condition 2 in Section 3.1.1; the paper does not specify how interference is operationalized.
  • domain assumption Training with the gold answer available (shown in Figure 3) transfers to test time where the reader sees only the question and compressed documents.
    Evidentiality mining uses the gold answer as input, but deployment does not. The paper does not address this train/deploy mismatch.
  • domain assumption Flan-T5-large is an appropriate teacher-student architecture for the evaluator because instruction-tuned T5 is reliable for document classification.
    Choice justified in Section B.2, but the model's prior exposure is the only contamination check performed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ECoRAG: Evidentiality-guided Compression for Long Context RAG." pith.science (2026). https://pith.science/paper/H3KYW26S

@misc{pith2026250605167,
  author       = {Pith},
  title        = {Pith review of: ECoRAG: Evidentiality-guided Compression for Long Context RAG},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H3KYW26S}},
  note         = {Machine review of arXiv:2506.05167}
}
read the original abstract

Large Language Models (LLMs) have shown remarkable performance in Open-Domain Question Answering (ODQA) by leveraging external documents through Retrieval-Augmented Generation (RAG). To reduce RAG overhead, from longer context, context compression is necessary. However, prior compression methods do not focus on filtering out non-evidential information, which limit the performance in LLM-based RAG. We thus propose Evidentiality-guided RAG, or ECoRAG framework. ECoRAG improves LLM performance by compressing retrieved documents based on evidentiality, ensuring whether answer generation is supported by the correct evidence. As an additional step, ECoRAG reflects whether the compressed content provides sufficient evidence, and if not, retrieves more until sufficient. Experiments show that ECoRAG improves LLM performance on ODQA tasks, outperforming existing compression methods. Furthermore, ECoRAG is highly cost-efficient, as it not only reduces latency but also minimizes token usage by retaining only the necessary information to generate the correct answer. Code is available at https://github.com/ldilab/ECoRAG.

Figures

Figures reproduced from arXiv: 2506.05167 by the authors.

Figure 1
Figure 1. Comparison of performance between prepend [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. This figure illustrates the overall framework of ECoRAG. First, the evidentiality-guided compressor [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. This figure illustrates the evidentiality mining [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Evidentiality evaluation metrics using differ [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Evidentiality evaluation metrics using differ [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: An input prompt for LLM for question answering, including few-shot examples, input documents, and a [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: An input prompt for LLM for evidentiality evaluation, including few-shot examples, compressed [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Measuring Alignment With Reader Highlights Net of Position and Length

    cs.IR 2026-07 conditional novelty 7.0 of 10

    Language-model importance rankings retain 38.4% of crowd-highlighted sentences versus 19.9% of matched non-highlighted sentences, an enrichment of +0.196 net of position and length confounds.

Reference graph

Works this paper leans on

53 extracted references · 15 canonical work pages · cited by 1 Pith paper

  1. [1]

    Akari Asai, Matt Gardner, and Hannaneh Hajishirzi. 2022. Evidentiality-guided generation for knowledge-intensive nlp tasks. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2226--2243

    work page 2022

  2. [2]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. https://openreview.net/forum?id=hSyW5go0v8 Self- RAG : Learning to retrieve, generate, and critique through self-reflection . In The Twelfth International Conference on Learning Representations

    2024

  3. [3]

    Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, et al. 2024. Longbench: A bilingual, multitask benchmark for long context understanding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3119--3137

    2024

  4. [4]

    Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on freebase from question-answer pairs. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1533--1544

    work page 2013

  5. [5]

    Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. https://doi.org/10.18653/v1/2024.findings-acl.137 M 3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation . In Findings of the Association for Computational Linguistics: ACL 2024, pages 2318--2335, Bangkok,...

    doi:10.18653/v1/2024.findings-acl.137 2024

  6. [6]

    Cheng-Han Chiang and Hung-yi Lee. 2023. https://doi.org/10.18653/v1/2023.acl-long.870 Can large language models be an alternative to human evaluations? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15607--15631, Toronto, Canada. Association for Computational Linguistics

    doi:10.18653/v1/2023.acl-long.870 2023

  7. [7]

    Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping H...

    arXiv 2022

  8. [8]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

Show all 53 references
  1. [9]

    Sang-eun Han, Yeonseok Jeong, Seung-won Hwang, and Kyungjae Lee. 2023. On monotonic aggregation for open-domain qa. In Proc. Interspeech 2023, pages 3432--3436

    2023

  2. [10]

    Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, and Boris Ginsburg. 2024. https://openreview.net/forum?id=kIoBbc76Sy RULER : What s the real context size of your long-context language models? In First Conference on Language Modeling

    2024

  3. [11]

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022. https://openreview.net/forum?id=jKN1pXi7b0 Unsupervised dense information retrieval with contrastive learning . Transactions on Machine Learning Research

    2022

  4. [12]

    Gautier Izacard and Edouard Grave. 2021. https://doi.org/10.18653/v1/2021.eacl-main.74 Leveraging passage retrieval with generative models for open domain question answering . In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Li...

    2021 doi

  5. [13]

    Yeonseok Jeong, Minsoo Kim, Seung-won Hwang, and Byung-Hak Kim. 2025. Agent-as-judge for factual summarization of long narratives. arXiv preprint arXiv:2501.09993

    2025

  6. [14]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023 a . Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  7. [15]

    Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2023 b . https://doi.org/10.18653/v1/2023.emnlp-main.825 LLML ingua: Compressing prompts for accelerated inference of large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natu...

    2023 doi

  8. [16]

    Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2024. https://aclanthology.org/2024.acl-long.91 L ong LLML ingua: Accelerating and enhancing LLM s in long context scenarios via prompt compression . In Proceedings of the 62nd Annua...

    2024

  9. [17]

    Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. https://doi.org/10.18653/v1/P17-1147 T rivia QA : A large scale distantly supervised challenge dataset for reading comprehension . In Proceedings of the 55th Annual Meeting of the Association for Computational...

    2017 doi

  10. [18]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.550 Dense passage retrieval for open-domain question answering . In Proceedings of the 2020 Conference on Empiric...

    2020 doi

  11. [19]

    Jaehyung Kim, Jaehyun Nam, Sangwoo Mo, Jongjin Park, Sang-Woo Lee, Minjoon Seo, Jung-Woo Ha, and Jinwoo Shin. 2024. https://openreview.net/forum?id=w4DW6qkRmt Sure: Improving open-domain question answering of LLM s via summarized retrieval . In The Twelfth International Confer...

    2024

  12. [20]

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: A benchmark for question answering research. Transactions of the Association for C...

    2019

  13. [21]

    Tiziano Labruna, Jon Ander Campos, and Gorka Azkune. 2024. https://arxiv.org/abs/2404.19705 When to retrieve: Teaching llms to utilize information retrieval effectively . Preprint, arXiv:2404.19705

    2024 arXiv

  14. [22]

    Kyungjae Lee, Seung-won Hwang, Sang-eun Han, and Dohyeon Lee. 2021. Robustifying multi-hop qa through pseudo-evidentiality training. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natura...

    2021

  15. [23]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \"u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \"a schel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Proc...

    2020

  16. [24]

    Tianle Li, Ge Zhang, Quy Duc Do, Xiang Yue, and Wenhu Chen. 2024. Long-context llms struggle with long in-context learning. arXiv preprint arXiv:2404.02060

    2024 arXiv

  17. [25]

    Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.391 Compressing context to enhance inference efficiency of large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, ...

    2023 doi

  18. [26]

    Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. https://doi.org/10.1162/tacl_a_00638 Lost in the middle: How language models use long contexts . Transactions of the Association for Computational Linguistics, 12...

    2024 doi

  19. [27]

    Ziyang Luo, Can Xu, Pu Zhao, Xiubo Geng, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023. https://arxiv.org/abs/2305.04757 Augmented large language models with parametric knowledge guiding . Preprint, arXiv:2305.04757

    2023 arXiv

  20. [28]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2024. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36

    2024

  21. [29]

    Jesse Mu, Xiang Li, and Noah Goodman. 2024. Learning to compress prompts with gist tokens. Advances in Neural Information Processing Systems, 36

    2024

  22. [30]

    Yilin Niu, Fangkai Jiao, Mantong Zhou, Ting Yao, Jingfang Xu, and Minlie Huang. 2020. https://doi.org/10.18653/v1/2020.acl-main.361 A self-training method for machine reading comprehension with soft evidence extraction . In Proceedings of the 58th Annual Meeting of the Associa...

    2020 doi

  23. [31]

    OpenAI. 2023. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . Preprint, arXiv:2303.08774

    2023 arXiv

  24. [32]

    Vicky Zhao, Lili Qiu, and Dongmei Zhang

    Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Ruhle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Dongmei Zhang. 2024. https://aclanthology.org/2024.findings-acl.57 LLML ingua-2: Data distillation for efficient and ...

    2024

  25. [33]

    Cheng Qian, Xinran Zhao, and Tongshuang Wu. 2024. https://openreview.net/forum?id=Pvn1dKreZW ''merge conflicts!''' exploring the impacts of external knowledge distractors to parametric knowledge graphs . In First Conference on Language Modeling

    2024

  26. [34]

    Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics, 11:1316--1331

    2023

  27. [35]

    Rohit Saxena and Frank Keller. 2024. Select and summarize: Scene saliency for movie script summarization. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3439--3455

    2024

  28. [36]

    Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Sch \"a rli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, pages 31210--31227. PMLR

    2023

  29. [37]

    Yongho Song, Dahyun Lee, Myungha Jang, Seung-won Hwang, Kyungjae Lee, Dongha Lee, and Jinyoung Yeo. 2024. Evidentiality-aware retrieval for overcoming abstractiveness in open-domain question answering. In Findings of the Association for Computational Linguistics: EACL 2024, pa...

    2024

  30. [38]

    Tran, Xavier Garcia, Jason Wei, Xuezhi Wang, Hyung Won Chung, Dara Bahri, Tal Schuster, Steven Zheng, Denny Zhou, Neil Houlsby, and Donald Metzler

    Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Xavier Garcia, Jason Wei, Xuezhi Wang, Hyung Won Chung, Dara Bahri, Tal Schuster, Steven Zheng, Denny Zhou, Neil Houlsby, and Donald Metzler. 2023. https://openreview.net/forum?id=6ruVLB727MC UL 2: Unifying language learning paradigms . ...

    2023

  31. [39]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L \'e onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram \'e , et al. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118

    2024 arXiv

  32. [40]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  33. [41]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. ♫ musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539--554

    2022

  34. [42]

    Chenguang Wang, Xiao Liu, and Dawn Song. 2020. Language models are open knowledge graphs. arXiv preprint arXiv:2010.11967

    2020 arXiv

  35. [43]

    Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, et al. 2022. Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks. In ...

    2022

  36. [44]

    Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzm \'a n, Armand Joulin, and Edouard Grave. 2020. https://aclanthology.org/2020.lrec-1.494 CCN et: Extracting high quality monolingual datasets from web crawl data . In Proceedings of the Twel...

    2020

  37. [45]

    Kevin Wu, Eric Wu, and James Zou. 2024. How faithful are rag models? quantifying the tug-of-war between rag and llms' internal prior. arXiv preprint arXiv:2404.10198

    2024 arXiv

  38. [46]

    Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. C-pack: Packed resources for general chinese embeddings. In Proceedings of the 47th international ACM SIGIR conference on research and development in information retrieval, pages 641--649

    2024

  39. [47]

    Fangyuan Xu, Weijia Shi, and Eunsol Choi. 2024. Recomp: Improving retrieval-augmented lms with context compression and selective augmentation. In The Twelfth International Conference on Learning Representations

    2024

  40. [48]

    Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, and Zhen-Hua Ling. 2024. https://arxiv.org/abs/2401.15884 Corrective retrieval augmented generation . Preprint, arXiv:2401.15884

    2024 arXiv

  41. [49]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. https://doi.org/10.18653/v1/D18-1259 H otpot QA : A dataset for diverse, explainable multi-hop question answering . In Proceedings of the 2018 Conference...

    2018 doi

  42. [50]

    Chanwoong Yoon, Taewhoo Lee, Hyeon Hwang, Minbyul Jeong, and Jaewoo Kang. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.1194 C omp A ct: Compressing retrieved documents actively for question answering . In Proceedings of the 2024 Conference on Empirical Methods in Natural ...

    2024 doi

  43. [51]

    Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang. 2023. https://openreview.net/forum?id=fB0hRu9GZUS Generate rather than retrieve: Large language models are strong context generators . In The Eleventh Inter...

    2023

  44. [52]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  45. [53]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.