Pith. sign in

REVIEW 4 major objections 5 minor 4 cited by

Enhancing Retrieval-Augmented Generation: A Study of Best Practices

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that showing a RAG model retrieved correct and incorrect answer pairs for the same question improves accuracy and factuality more than any other configuration tested, especially on specialized-knowledge questions.

desk verdict A fatal test-set leakage sinks the Contrastive ICL headline; the rest is a useful but incremental ablation study. read the letter →

arxiv 2501.07391 v1 pith:D4BIQZD2 submitted 2025-01-13 cs.CL cs.AI

Siran Li , Linus Stenzel , Carsten Eickhoff , Seyed Ali Bahrainian This is my paper · ORCID
classification cs.CLcs.AI
keywords retrieval-augmentedgenerationcontrastivein-contextlearningqueryexpansionfocusmodeRAGbestpracticesfactualityevaluationablationstudyLLMprompting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish which design choices most improve retrieval-augmented generation (RAG), the technique of feeding an LLM retrieved context at inference time. It runs 74 ablation experiments varying model size, prompt wording, chunk size, knowledge-base size, retrieval frequency, query expansion, multilingual context, sentence-level focusing, and in-context demonstrations. The central finding is that Contrastive In-Context Learning RAG, which retrieves correct and incorrect answer examples for the same question and puts them in the prompt, outperforms every other variant on both general-knowledge and specialized-knowledge questions. A second proposed configuration, Focus Mode, which retrieves individual sentences rather than whole documents, ranks second. The paper concludes that the relevance and quality of retrieved documents matter more than their quantity, and that prompt phrasing remains decisive even inside a RAG system.

What carries the argument

The load-bearing mechanism is Contrastive In-Context Learning: the retrieved context is not general encyclopedic text but a small set of demonstration examples consisting of the same question with its correct answer and a deliberately incorrect answer, inserted into the prompt before the active query. The retrieval pipeline that carries the method is built from a vector-similarity search index (FAISS) over sentence-transformer embeddings of the knowledge base, a T5-based query-expansion module, and a Mistral instruction-tuned generator. Focus Mode, the second-ranked configuration, works by splitting retrieved documents into sentences and re-ranking them against the query so the generator sees only the most relevant sentences.

What would settle it

Hold out the target question's own example from the retrieval knowledge base, or swap in demonstrations drawn from a disjoint development set, and re-run the Contrastive ICL experiments. If the advantage over the baseline disappears or shrinks sharply, the reported gains came from revealing the answer inside the prompt, not from contrastive learning.

Watch

Extended reading notes

Core claim

The paper's central claim is that adding contrastive demonstrations to a RAG prompt is the single most effective configuration among those tested. In this design, the knowledge base is the evaluation dataset itself: each query retrieves example question–answer pairs of the same type, and the prompt presents both a correct answer and an incorrect (contrastive) answer before asking the model to answer the active question, with the active query masked during retrieval. Across ROUGE, embedding similarity, MAUVE, and FActScore, these Contrastive ICL variants beat the baseline RAG and all other variants, with the margin widening on MMLU, the dataset requiring specialized knowledge. The authors interpret this as the model learning to discriminate correct from incorrect response patterns.

Load-bearing premise

The results for Contrastive ICL assume that retrieving from the evaluation dataset with the active query masked still hides the answer; if the same question appears in the retrieved demonstrations with its correct answer visible, the measured advantage is an artifact of answer leakage rather than a genuine property of the method.

Editorial extensions

If this is right

  • RAG practitioners should consider adding contrastive demonstrations (correct and incorrect answer pairs) to prompts for question-answering, especially on knowledge-heavy tasks.
  • Knowledge-base size can be kept small if the retrieved documents are on-topic; the paper finds 1K and 10K article bases perform comparably.
  • Retrieving individual sentences (Focus Mode) is a low-cost way to sharpen responses when full documents add noise.
  • Frequent retrieval updates during generation (small strides) hurt coherence; larger strides preserve context stability.
  • Prompt wording has a measurable effect even within RAG, so prompt design deserves the same attention as retrieval choices.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A stricter evaluation that draws demonstrations from a disjoint development set would separate the method's genuine effect from answer leakage; the paper's current setup does not fully rule this out.
  • Combining Contrastive ICL with Focus Mode or Query Expansion, which the paper leaves untested, could yield additional gains beyond any single module.
  • The contrastive demonstration idea transfers naturally to other conditional generation tasks such as summarization or simplification, where correct and incorrect outputs for the same input can be obtained.
  • A practical takeaway for practitioners: treat the module ranking with caution until leakage-free reproduction is run, but expect the qualitative message that demonstrations help and sentence-level focus helps to be robust.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents an empirical study of Retrieval-Augmented Generation (RAG) configurations, introducing four new variants: Query Expansion, Contrastive In-Context Learning (ICL), a Multilingual knowledge base, and Focus Mode. The authors evaluate these and other ablations (LLM size, prompt design, chunk size, knowledge base size, retrieval stride) on TruthfulQA and MMLU using ROUGE, embedding cosine similarity, MAUVE, and FActScore. The central claim is that Contrastive ICL RAG outperforms all other variants, especially on MMLU, followed by Focus Mode RAG. The paper also reports best practices such as the relative unimportance of knowledge base size and the importance of prompt design.

Significance. If the results were valid, the paper would be a useful reference for practitioners, and the public code and broad ablation coverage are commendable. However, the headline finding about Contrastive ICL is compromised by a fundamental evaluation design flaw: the ICL knowledge base is the evaluation dataset itself, so the model is prompted with gold-annotated answers from the test distribution. This makes the comparison against the Wikipedia-based baselines invalid for the paper's central claim. The other ablations (e.g., Focus Mode, prompt design) may still be informative, but they do not rescue the main conclusion as presented.

major comments (4)
  1. [Section 3.1 Q7 and Appendix A.3] The Contrastive ICL experiment uses the evaluation dataset as the knowledge base, and the prompt templates in Appendix A.3 insert the correct answer (and, for '+' variants, the incorrect answer) of a retrieved example. Masking the active query prevents retrieving the exact current question, but it does not prevent retrieving other test examples with gold labels. For MMLU, all 1,824 evaluation samples are indexed, so the model receives the gold answer of a similar question in its prompt. All other variants retrieve from Wikipedia Vital Articles and never receive such gold answers. This is test-label leakage, not a comparable RAG design. The large MMLU gains (Table 2: ROUGE-2 from 1.90 to 17.46; Table 3: FActScore from 63.73 to 74.44) are exactly what this confound predicts. This invalidates finding (1) in Section 6. The authors need to use a knowledge base disjoint from the evaluation data (e.g., training splits or an external corpus) and re-run the ICL experiments to support the claimed superiority.
  2. [Table 2 and Section 5.1] Table 2's footnote states that bold scores denote 'statistical significance over the baseline,' but no significance test, p-value, confidence interval, or multiple-comparison correction is reported anywhere in the paper. Phrases such as 'significantly outperforming' (Section 5.1, item 2) and 'statistically significant' are therefore unsupported. The authors should either report appropriate statistical tests (with correction for the many comparisons across 74 runs) or remove the significance claims.
  3. [Table 5 and Section 5.3] The qualitative example for ICL1D+ on MMLU ('The correct answer is I. Bending and III. Asymmetric stretching.') is the gold answer itself. This demonstrates concretely that the prompt transfers test labels into the generated response, confirming that the measured ICL advantage reflects answer leakage rather than improved retrieval or generation capability.
  4. [Section 8 Limitations] The Limitations section does not acknowledge the evaluation-data leakage in the Contrastive ICL setup, despite this being the most serious threat to the paper's central claim. Any revised version must address this issue explicitly and explain how the headline result can be validated under a leakage-free protocol.
minor comments (5)
  1. [Section 6, Finding 5] The phrase 'are the still the top models' should read 'are still the top models.'
  2. [Table 4] The header 'Artivles' appears to be a typo for 'Articles.'
  3. [Section 5.1, Item 6] The naming 'ExpendS', 'ExpendM', 'ExpendL' is inconsistent with the term 'Query Expansion'; consider renaming to 'ExpandS', 'ExpandM', 'ExpandL' for clarity.
  4. [Appendix A.3] In the ICL prompt templates, the placeholder 'Question: q' is used for both the retrieved example's question and the active query. Clarify the notation to distinguish the example question from the current query, as this is central to understanding the leakage issue.
  5. [Section 4.3] The MAUVE description states that it is evaluated on the entire dataset to ensure stability, but no standard errors or confidence intervals are reported. Given the small per-subject sample size in MMLU, the authors should report uncertainty or at least discuss the sensitivity of MAUVE to sample size.

Circularity Check

1 steps flagged · score 5.0 of 10

Contrastive ICL's headline superiority is a self-referential evaluation artifact: gold answers from the test benchmarks are placed in the prompt, so the central comparison is confounded.

  1. other [Section 3.1 Q7; Section 5.1 #7; Appendix A.3; Section 6 finding 1]
    "By using an evaluation dataset as the knowledge base and masking the active query during retrieval, the model can replicate effective response patterns. ... ICL1D+ Prompt : "HelpV1. Considering these examples: Question: q, Correct Answer: Answercorrect. Question: q, Incorrect Answer: Answerincorrect. Question: q, Correct Answer:""

    The Contrastive ICL knowledge base is the evaluation dataset itself, and the prompt templates in Appendix A.3 place the gold correct answer (and, for '+' variants, the gold incorrect answer) of retrieved neighbor questions directly into the context. Masking the active query only prevents retrieving the current question; all other TruthfulQA/MMLU test items remain in the index, so the model receives labeled examples from the exact benchmark on which it is scored. The baselines retrieve unlabeled Wikipedia text and never receive gold answers. The large ICL gains (e.g., MMLU ROUGE-2 from 1.90 to 17.46 and FActScore from 63.73 to 74.44) are therefore substantially a consequence of providing test-set labels and answer format in the prompt, not of a comparable RAG design.

full rationale

The only substantive circularity is the Contrastive ICL evaluation. The paper's own description of Q7 states that the knowledge base is the evaluation dataset, and the Appendix A.3 prompt templates show that gold answers are inserted as in-context demonstrations. Because the metrics (ROUGE, embedding cosine similarity, MAUVE, FActScore) compare against reference answers whose content appears in the demonstrations, the ICL variant's scores are inflated by construction relative to the Wikipedia-retrieval baselines. This does not invalidate the separate ablations (LLM size, prompt design, chunk size, knowledge-base size, retrieval stride, query expansion, multilingual, and Focus Mode), which use a common external knowledge base and are self-contained empirical comparisons. I found no load-bearing self-citation or imported uniqueness argument; the self-citations (NEWTS, SIMSUM) are background references. Because the paper's strongest claim—Contrastive ICL outperforms all variants, especially on MMLU—is confounded by this self-referential evaluation, the circularity score is 5: the result is not a formal derivation from its own inputs, but the central empirical 'prediction' is not measured against an independent benchmark.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper's central claims rest on standard empirical assumptions (metric validity, knowledge base coverage) plus the questionable assumption that using the evaluation dataset as the ICL knowledge base is leakage-free.

free parameters (4)
  • ICL example count = 1 or 2
    Number of retrieved demonstrations in Contrastive ICL is chosen by hand; results vary with this count (Section 5.1 point 7).
  • Query expansion filter size = 9, 15, 21
    Number of articles retrieved in the first expansion stage is chosen per variant; effects are marginal (Section 5.1 point 6).
  • Baseline chunk size = 64 tokens
    Default document chunk size of 64 tokens is chosen for the baseline and used across most experiments (Section 4.4).
  • Retrieval stride = baseline: no stride; variants: 1, 2, 5
    Stride values are chosen experimental conditions, not fitted, but the conclusion that larger strides are better depends on these choices (Section 5.1 point 5).
assumptions (4)
  • domain assumption ROUGE, embedding cosine similarity, MAUVE, and FActScore adequately capture response quality
    The paper uses these metrics to rank RAG variants (Section 4.3); if they do not reflect quality, the rankings are meaningless.
  • domain assumption Wikipedia Vital Articles provide sufficient coverage for TruthfulQA and MMLU questions
    The knowledge base is built from Wikipedia Level 3/4 articles (Section 4.2); insufficient coverage would bias retrieval results.
  • domain assumption GPT-3.5-turbo is a reliable factuality judge for FActScore
    FActScore depends on GPT-3.5-turbo's atomic fact checking (Section 4.3); no validation of this judge is provided.
  • ad hoc to paper Retrieving from the evaluation dataset with the active query masked prevents information leakage
    The Contrastive ICL design uses the test set as its knowledge base (Appendix A.3); this assumption is not justified and is likely false.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Retrieval-Augmented Generation: A Study of Best Practices." pith.science (2026). https://pith.science/paper/D4BIQZD2

@misc{pith2026250107391,
  author       = {Pith},
  title        = {Pith review of: Enhancing Retrieval-Augmented Generation: A Study of Best Practices},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D4BIQZD2}},
  note         = {Machine review of arXiv:2501.07391}
}
read the original abstract

Retrieval-Augmented Generation (RAG) systems have recently shown remarkable advancements by integrating retrieval mechanisms into language models, enhancing their ability to produce more accurate and contextually relevant responses. However, the influence of various components and configurations within RAG systems remains underexplored. A comprehensive understanding of these elements is essential for tailoring RAG systems to complex retrieval tasks and ensuring optimal performance across diverse applications. In this paper, we develop several advanced RAG system designs that incorporate query expansion, various novel retrieval strategies, and a novel Contrastive In-Context Learning RAG. Our study systematically investigates key factors, including language model size, prompt design, document chunk size, knowledge base size, retrieval stride, query expansion techniques, Contrastive In-Context Learning knowledge bases, multilingual knowledge bases, and Focus Mode retrieving relevant context at sentence-level. Through extensive experimentation, we provide a detailed analysis of how these factors influence response quality. Our findings offer actionable insights for developing RAG systems, striking a balance between contextual richness and retrieval-generation efficiency, thereby paving the way for more adaptable and high-performing RAG frameworks in diverse real-world scenarios. Our code and implementation details are publicly available.

Figures

Figures reproduced from arXiv: 2501.07391 by the authors.

Figure 1
Figure 1. Overview of our RAG framework. It involves three main components: a query expansion module, a [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Evaluating the Robustness of Retrieval-Augmented Generation to Adversarial Evidence in the Health Domain

    cs.IR 2025-09 conditional novelty 6.0 of 10

    Misleading health documents in RAG context sharply lower LLM accuracy, and heavily helpful-biased retrieval pools restore it.

  2. FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation

    cs.IR 2025-04 conditional novelty 6.0 of 10

    FinDER offers 5,703 expert-annotated ambiguous financial queries grounded in 10-K filings and shows current retrieval and generation models still struggle with them.

  3. Never Come Up Empty: Adaptive HyDE Retrieval for Improving LLM Developer Support

    cs.SE 2025-07 conditional novelty 4.0 of 10

    A HyDE retrieval pipeline with full-answer context and adaptive similarity thresholding improves LLM answers to Stack Overflow questions over zero-shot prompting for three of four open-source models.

  4. Get Experience from Practice: LLM Agents with Record & Replay

    cs.LG 2025-05 reject novelty 4.0 of 10

    AgentRR is a proposed paradigm that records agent traces, generalizes them into multi-level experiences, and replays them under safety checks to make LLM agents cheaper, faster, and more reliable.

Reference graph

Works this paper leans on

44 extracted references · 21 canonical work pages · cited by 4 Pith papers

  1. [1]

    Seyed Ali Bahrainian, Sheridan Feucht, and Carsten Eickhoff. 2022. https://aclanthology.org/2022.findings-acl.42 NEWTS : A corpus for news topic-focused summarization . In Findings of the Association for Computational Linguistics: ACL 2022, pages 493--503

    work page 2022

  2. [2]

    Sofia Blinova, Xinyu Zhou, Martin Jaggi, Carsten Eickhoff, and Seyed Ali Bahrainian. 2023. https://aclanthology.org/2023.acl-long.552 SIMSUM : Document-level text simplification via simultaneous summarization . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9927--9944. Association ...

    work page 2023

  3. [3]

    Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022. Improving language models by retrieving from trillions of tokens. In International conference on machine learning, pages 2206--2240. PMLR

    2022

  4. [4]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gr...

    work page 2020

  5. [5]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 b . Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  6. [6]

    Claudio Carpineto and Giovanni Romano. 2012. A survey of automatic query expansion in information retrieval. Acm Computing Surveys (CSUR), 44(1):1--50

    2012

  7. [7]

    Chang, Katrin Tomanek, Jessica Hoffmann, Nithum Thain, Erin van Liemt, Kathleen Meier-Hellstern, and Lucas Dixon

    Tyler A. Chang, Katrin Tomanek, Jessica Hoffmann, Nithum Thain, Erin van Liemt, Kathleen Meier-Hellstern, and Lucas Dixon. 2024. Detecting hallucination and coverage errors in retrieval augmented generation for controversial topics. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (L...

    work page 2024

  8. [8]

    Tong Chen, Hongwei Wang, Sihao Chen, Wenhao Yu, Kaixin Ma, Xinran Zhao, Hongming Zhang, and Dong Yu. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.845 Dense X retrieval: What retrieval granularity should we use? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 15159--15177. Association for Computational ...

    doi:10.18653/v1/2024.emnlp-main.845 2024

Show all 44 references
  1. [9]

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1--53

    2024

  2. [10]

    Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT : Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT, pages 4171--4186

    2019

  3. [11]

    Guanting Dong, Yutao Zhu, Chenghao Zhang, Zechen Wang, Zhicheng Dou, and Ji-Rong Wen. 2024. Understand What LLM Needs : Dual preference alignment for retrieval-augmented generation. arXiv preprint arXiv:2406.18676

    2024 arXiv

  4. [12]

    Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2024. https://arxiv.org/abs/2401.08281 The faiss library

    2024 arXiv

  5. [13]

    Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. A Survey on RAG Meeting LLMs : Towards retrieval-augmented large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining...

    2024

  6. [14]

    Masato Fujitake. 2024. https://aclanthology.org/2024.lrec-main.892 L ayout LLM : Large language model instruction tuning for visually rich document understanding . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Ev...

    2024

  7. [15]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997

    2023 arXiv

  8. [16]

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. pages 3929--3938

    2020

  9. [17]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR)

    2021

  10. [18]

    Jennifer Hsia, Afreen Shaikh, Zhiruo Wang, and Graham Neubig. 2024. RAGGED : Towards informed design of retrieval augmented generation systems. arXiv preprint arXiv:2403.09040

    2024 arXiv

  11. [19]

    Yutong Hu, Quzhe Huang, Mingxu Tao, Chen Zhang, and Yansong Feng. 2024. https://openreview.net/forum?id=Cjp6YKVeAa Can perplexity reflect large language model's ability in long text understanding? In The Second Tiny Papers Track at ICLR 2024

    2024

  12. [20]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2024. https://doi.org/10.1145/3703155 A Survey on Hallucination in Large Language Models : Principles, taxonomy, challenges, and ...

    2024 doi

  13. [21]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7B . arXiv preprint arXiv:2310.06825

    2023 arXiv

  14. [22]

    Zhuoran Jin, Pengfei Cao, Yubo Chen, Kang Liu, Xiaojian Jiang, Jiexin Xu, Li Qiuxia, and Jun Zhao. 2024. Tug-of-War between Knowledge : Exploring and resolving knowledge conflicts in retrieval-augmented language models. In Proceedings of the 2024 Joint International Conference...

    2024

  15. [23]

    Gangwoo Kim, Sungdong Kim, Byeongguk Jeon, Joonsuk Park, and Jaewoo Kang. 2023. Tree of Clarifications : Answering ambiguous questions with retrieval-augmented large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pag...

    2023

  16. [24]

    Sung-Min Lee, Eunhwan Park, Donghyeon Jeon, Inho Kang, and Seung-Hoon Na. 2024. RADCoT : Retrieval-augmented distillation to specialization models for generating chain-of-thoughts in query expansion. In Proceedings of the 2024 Joint International Conference on Computational Li...

    2024

  17. [25]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \"u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \"a schel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Proc...

    2020

  18. [26]

    Jiarui Li, Ye Yuan, and Zehua Zhang. 2024. Enhancing llm factual accuracy with rag to counter hallucinations: A case study on domain-specific queries in private knowledge-bases. arXiv preprint arXiv:2403.10446

    2024 arXiv

  19. [27]

    Chin-Yew Lin. 2004. ROUGE : A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74--81

    2004

  20. [28]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. T ruthful QA : Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214--3252

    2022

  21. [29]

    Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.741 FA ct S core: Fine-grained atomic evaluation of factual precision in long form text generatio...

    2023 doi

  22. [30]

    Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. 2021. MAUVE : Measuring the gap between neural text and human text using divergence frontiers. In NeurIPS

    2021

  23. [31]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485--5551

    2020

  24. [32]

    Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics, 11:1316--1331

    2023

  25. [33]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT : Sentence embeddings using siamese bert-networks. In EMNLP, pages 3982--3992

    2019

  26. [34]

    Sina Semnani, Violet Yao, Heidi Zhang, and Monica Lam. 2023. W iki C hat: Stopping the hallucination of large language model chatbots by few-shot grounding on W ikipedia. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2387--2413

    2023

  27. [35]

    Haizhou Shi, Zihao Xu, Hengyi Wang, Weiyi Qin, Wenyuan Wang, Yibin Wang, and Hao Wang. 2024 a . Continual learning of large language models: A comprehensive survey. arXiv preprint arXiv:2404.16789

    2024 arXiv

  28. [36]

    Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2024 b . https://doi.org/10.18653/v1/2024.naacl-long.463 REPLUG : Retrieval-augmented black-box language models . In Proceedings of the 2024 Conference of the ...

    2024 doi

  29. [37]

    Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, Rajib Rana, and Suranga Nanayakkara. 2023. Improving the domain adaptation of retrieval augmented generation (rag) models for open domain question answering. Transactions of the Association for Comp...

    2023

  30. [38]

    Qiushi Sun, Chengcheng Han, Nuo Chen, Renyu Zhu, Jingyang Gong, Xiang Li, and Ming Gao. 2024. https://aclanthology.org/2024.lrec-main.956 Make Prompt-based Black-Box Tuning Colorful : Boosting model generalization from three orthogonal perspectives . In Proceedings of the 2024...

    2024

  31. [39]

    Nhat Tran and Diane Litman. 2024. Enhancing knowledge retrieval with topic modeling for knowledge-grounded dialogue. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 5986--5995

    2024

  32. [40]

    Xiaohua Wang, Zhenghua Wang, Xuan Gao, Feiran Zhang, Yixin Wu, Zhibo Xu, Tianyuan Shi, Zhengyuan Wang, Shizheng Li, Qi Qian, et al. 2024. Searching for best practices in retrieval-augmented generation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Langu...

    2024

  33. [41]

    Kevin Wu, Eric Wu, and James Zou. 2024. ClashEval : Quantifying the tug-of-war between an llm’s internal prior and external evidence. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track

    2024

  34. [42]

    Lingxi Zhang, Jing Zhang, Yanling Wang, Shulin Cao, Xinmei Huang, Cuiping Li, Hong Chen, and Juanzi Li. 2023. https://aclanthology.org/2023.acl-long.57 FC - KBQA : A fine-to-coarse composition framework for knowledge base question answering . In Proceedings of the 61st Annual ...

    2023

  35. [43]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  36. [44]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.