Pith. sign in

REVIEW 4 major objections 6 minor 31 references

Knowledge Augmented Finetuning Matters in both RAG and Agent Based Dialog Systems

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Finetuning a dialog model on retrieved knowledge beats prompting for factual accuracy in both RAG and agent systems.

desk verdict KAFT paper's headline comparison is confounded (GPT-2 finetuned vs GPT-3.5 prompted), but the Table V within-model ablation is a genuine and useful result. read the letter →

arxiv 2506.22852 v1 pith:X34UR2S3 submitted 2025-06-28 cs.CL

Yucheng Cai , Yuxuan Wu , Yi Huang , Junlan Feng , Zhijian Ou This is my paper
classification cs.CL
keywords knowledgeaugmentedfinetuningretrievalgenerationLLMagentsdialogsystemsfactualaccuracyhallucinationMobileCS2customerservice
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a language model serving in a retrieval-augmented (RAG) or tool-calling (agent) dialog system should be finetuned on domain dialogs together with the retrieved knowledge, rather than merely prompted with that knowledge. It calls this knowledge augmented finetuning (KAFT) and tests it on MobileCS2, a real customer-service dataset. On both architectures, KAFT with a small GPT-2 model beats GPT-3.5 prompting on fluency, semantic similarity, and especially Inform Rate, a measure of whether the response supplies the information the user asked for. If the result is right, it means the largest gains in knowledge-intensive dialog come from teaching the model to use external knowledge during training, not from scaling up prompts at inference time.

What carries the argument

The load-bearing mechanism is knowledge augmented finetuning itself: supervised finetuning of the generator (and, in the agent setting, the decision maker) on domain data where each training example includes the knowledge the system would actually retrieve at test time. For RAG, the generator is trained with the auto-regressive loss over $p_\theta(r_t \mid c_t, h_t)$ using retriever output $h_t$; for the agent, the same loss is applied with API search results, and a second finetuned module predicts the search decision $a_t$. The key design choice is that training uses retrieved knowledge instead of the clean annotated knowledge, which aligns training with test-time conditions and teaches the model to cope with imperfect retrieval.

What would settle it

A controlled run on MobileCS2 that holds the base model fixed, for example prompting GPT-2 with the same five examples and instructions or finetuning GPT-3.5 with KAFT, would settle the claim: if the Inform Rate gap largely disappears or reverses, the advantage is due to the base model rather than the finetuning method.

Watch

Extended reading notes

Core claim

The central claim is that training the response generator on pairs of dialog context and retrieved knowledge pieces, which the paper calls knowledge augmented finetuning (KAFT), substantially improves knowledge-intensive dialog over prompting, with the largest effect on factual accuracy. In the RAG system, KAFT finetunes the generation model $p_\theta(r_t \mid c_t, h_t)$ on retrieved (rather than oracle) knowledge; in the agent system, it additionally finetunes the decision maker $p_\theta(a_t \mid c_t)$ that chooses which API to call. On the MobileCS2 test set, KAFT with GPT-2 reaches an Inform Rate of 0.145 in RAG and 0.147 in the agent system, versus 0.063 and 0.082 for GPT-3.5 prompting, and also raises BLEU and BERTScore. The paper also finds that training with retrieved knowledge outperforms training with annotated knowledge when the system is tested with retrieved knowledge, because the generator learns to judge whether retrieved pieces are correct.

Load-bearing premise

The comparison assumes that the performance gap comes from KAFT and not from the fact that KAFT uses a finetuned GPT-2 while prompting uses GPT-3.5, since the two methods are never tested with the same base model.

Editorial extensions

If this is right

  • A dialog system can reach higher factual accuracy by finetuning a relatively small generator on retrieved knowledge than by prompting a much larger model, implying that training data construction matters more than model scale for this task.
  • Agent systems benefit from finetuning not only the response generator but also the decision maker that chooses which knowledge API to call: KAFT raises decision accuracy on Personal, Product, and FAQ searches over both 0-shot and 5-shot prompting.
  • Training a RAG generator on the retriever's actual noisy output improves test-time performance, so retrieval noise should be simulated during finetuning rather than hidden behind oracle knowledge.
  • Because RAG and agent systems perform on par despite the agent's better retrieval, improving the agent's low decision-making accuracy is a concrete route to further gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: a natural extension, implied but not tested by the paper, is that the same KAFT recipe should transfer to larger open-weight models and other vertical domains, since the mechanism is a training-data construction choice rather than a GPT-2-specific trick.
  • Editorial inference: the large drop when the retriever's recall@1 is low for Product search suggests that improving retrieval accuracy would amplify KAFT's gains more than further prompting would; this is a bottleneck the paper identifies but does not quantify end-to-end.
  • Editorial inference: because the paper measures only one dataset and one evaluation script, the strongest test would be to re-run KAFT on a second knowledge-intensive dialog dataset with the same base model for prompting and finetuning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes knowledge augmented finetuning (KAFT): instead of prompting a large language model with retrieved knowledge in RAG- or agent-based dialog systems, the authors finetune the generator (GPT-2) on domain-specific dialogs together with the knowledge pieces retrieved by the system's retriever/APIs. Experiments on the MobileCS2 customer-service dataset compare KAFT (finetuned GPT-2) against prompting (GPT-3.5 with 0- or 5-shot in-context examples) in both a RAG pipeline and an agent pipeline, with additional ablations on using retrieved vs. annotated knowledge in training and testing. The paper claims that KAFT substantially surpasses prompting, especially in factual accuracy measured by Inform Rate, and that this is the first solid empirical work on the KAFT idea.

Significance. If the central claim were cleanly established, the paper would provide a useful practical result: domain-specific finetuning with retrieved knowledge can improve factual accuracy of dialog systems. The study has some real strengths: it uses a realistic human-human customer-service dataset, follows the official evaluation scripts, and includes thoughtful ablations (retrieved vs. oracle knowledge in training and testing, RAG vs. agent retrieval quality). The ablations showing that training with retrieved knowledge improves test-time robustness are informative. However, as reported, the headline comparison is confounded and therefore cannot support the claim that KAFT as a method outperforms prompting. The paper also overstates its novelty given the cited work on agent finetuning and retriever-augmented finetuning.

major comments (4)
  1. [§IV-A and Table I] The central comparison 'KAFT substantially surpasses prompting' is confounded. The paper states 'For the KAFT method, we finetune the GPT2, while for the prompting method, we use the GPT3.5.' Thus every KAFT result is a finetuned GPT-2 and every prompting result is an unfinetuned GPT-3.5. The observed gaps (e.g., Inform 0.145 vs. 0.063 in RAG, 0.147 vs. 0.082 in agent) could be due to the base model, the finetuning status, or their interaction, rather than to KAFT. No matched control is reported: there is no prompted GPT-2, no KAFT-finetuned (or LoRA-tuned) GPT-3.5, and no same-parameter-count prompt-based system. Without such controls, the abstract's and Section IV-B's claim that KAFT beats prompting is not supported as a statement about the method.
  2. [§IV-A and §IV-B] The results are reported as single point estimates with no variance, significance testing, or information about the number of runs and seeds. Table I reports one BLEU, one BERTScore, one Inform, and one Score per setting. For the main substantive claim about factual accuracy, the difference in Inform Rate should be accompanied by a significance test or at least confidence intervals. This is especially important because the prompting baselines use in-context examples selected 'randomly from the dataset' (Section III-B.2), so their variance may be substantial.
  3. [§I and §II (related work)] The claim that 'this paper represents the first solid empirical work to investigate the KAFT idea' is inconsistent with the paper's own citations. FireAct [11] investigates finetuning language agents for tool use, and RAFT [26] explicitly adapts language models to domain-specific RAG by finetuning on retrieved knowledge with distractors; the latter is described in Section III-B.1 as being similar to the paper's approach of using retrieved rather than oracle knowledge in training. The novelty claim should be substantially revised, and the relation to RAFT and FireAct should be discussed explicitly rather than asserted to leave a gap.
  4. [§IV-B] The sentence 'It is found in our experiments that a small model like GPT2 with KAFT can beat GPT3.5 with prompting' is a model-vs-method comparison, not a method-vs-method comparison. It demonstrates at most that finetuning a small model on a specific dataset can outperform a general-purpose API model in this task, which is not the same as showing that KAFT is superior to prompting for a fixed model. Section IV-A's statement that 'Using GPT-2 suffices to investigate the research question' is an assumption, not a demonstrated fact, and it is in tension with the stated goal of systematically comparing prompting and KAFT.
minor comments (6)
  1. [§I] Typo: 'Futuremore' should be 'Furthermore'.
  2. [§I] 'close-book QA' should be 'closed-book QA'.
  3. [§III-C.2] The text reads 'We use the ICL) method'; the opening parenthesis is missing.
  4. [§IV-A and references] The paper refers to 'the SLT 2024 FutureDial Challenge [17]' and 'MobileCS2 dataset, released from the SLT 2024 FutureDial Challenge', but reference [17] is titled 'The 2nd FutureDial challenge: Dialog systems with retrieval augmented generation (FutureDial-RAG)'. The terminology should be made consistent.
  5. [§IV-A] The combined score formula 'score = 0.5*(BLEU/100 + BERTScore) + Inform' is unusual because BERTScore and Inform can be on different scales; a brief explanation of why this formula is used would help readers interpret Table I.
  6. [§IV-C, Table IV] Table IV shows that using oracle knowledge in testing yields a combined score of 0.992, leaving almost no room for variation; this near-ceiling effect should be noted when interpreting the retrieval-error analysis in the surrounding text.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation found; KAFT results come from held-out test evaluation, not from construction.

full rationale

The paper's central claim—that KAFT surpasses prompting in RAG and agent systems—is an empirical comparison on the held-out MobileCS2 test set. Training KAFT on the official train split and scoring with the official scripts does not make the reported Inform Rate, BLEU, or BERTScore numbers follow from the training objective by construction. The KAFT generator is trained with Eq. (3) on context–knowledge–response triples, and the retriever with Eq. (2); neither equation embeds the test evaluation metric. There is no fitted parameter that is later renamed as a prediction, no uniqueness theorem imported from overlapping authors, and no ansatz smuggled in via citation. The main validity concern is a confound: Section IV-A states 'For the KAFT method, we finetune the GPT2, while for the prompting method, we use the GPT3.5,' so the method comparison is entangled with base-model choice and finetuning status. That is an experimental-design weakness, not circularity, because the test measurements are not algebraically forced by the training setup. A minor self-referential element exists: MobileCS2 [17] and its scoring script were released by overlapping authors and are used as the benchmark, and the 'first solid empirical work' claim is inconsistent with the cited RAFT [26] and FireAct [11]. These affect attribution and control, not the derivation chain, so the circularity score is low.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

No mathematical free parameters are introduced. The central claim depends on dataset assumptions, the choices of GPT-2 vs GPT-3.5, and the use of domain-specific annotations, which are the real burden of the experimental comparison.

assumptions (6)
  • domain assumption The MobileCS2 annotations (knowledge pieces, search decisions, responses) are correct and sufficient for measuring factual accuracy.
    The entire evaluation uses the official dataset and scripts from [17]. Incorrect annotations would invalidate the Inform Rate measurements.
  • domain assumption BLEU, BERTScore, and Inform Rate adequately capture response quality and factual accuracy.
    These three metrics are used as the sole evaluation; Inform Rate is a coarse proxy and may not reflect full factual correctness.
  • ad hoc to paper Using retriever-generated knowledge during training approximates test-time retrieval distribution, improving robustness.
    Section III.B.1 chooses generated ht over annotated ht to align train/test, assuming the noise distribution of the retriever is representative.
  • ad hoc to paper GPT-2 is a sufficient model for investigating the KAFT research question.
    Section IV.B states 'Using GPT-2 suffices', but this is an assumption that results transfer to larger models.
  • domain assumption GPT-3.5 with in-context examples is a representative prompting baseline.
    The comparison treats GPT-3.5 prompting as the prompting method, without testing other prompted models or sizes.
  • domain assumption The search APIs in the agent system simulate real API calls appropriately using annotated data.
    Section III.C.1 builds Product/FAQ APIs from annotated search decisions; this assumes the annotation is a faithful simulator of a real API.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Knowledge Augmented Finetuning Matters in both RAG and Agent Based Dialog Systems." pith.science (2026). https://pith.science/paper/X34UR2S3

@misc{pith2026250622852,
  author       = {Pith},
  title        = {Pith review of: Knowledge Augmented Finetuning Matters in both RAG and Agent Based Dialog Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X34UR2S3}},
  note         = {Machine review of arXiv:2506.22852}
}
read the original abstract

Large language models (LLMs) have recently been applied to dialog systems. Despite making progress, LLMs are prone to errors in knowledge-intensive scenarios. Recently, approaches based on retrieval augmented generation (RAG) and agent have emerged to improve the factual accuracy by enhancing the LLMs with knowledge retrieved from external knowledge bases (KBs). This is mostly implemented by prompting the LLMs with instructions, examples and the retrieved knowledge. However, LLMs may have difficulty using the retrieved knowledge effectively for response generation, because they are not well trained to do such generation for specific domains. To mitigate this problem, we propose to finetune the LLMs in the RAG-based and agent-based systems with domain-specific data, together with domain-specific external knowledge, which is called knowledge augmented finetuning (KAFT). We base our study on the MobileCS2 dataset, a real-life customer service dialog dataset that features intensive knowledge interactions, to systematically compare the prompting and KAFT techniques in the RAG-based and agent-based systems. Experiment results show that KAFT substantially surpasses prompting in both RAG and agent systems, particularly in terms of factual accuracy. To the best of our knowledge, this paper represents the first solid empirical work to investigate the KAFT idea.

Figures

Figures reproduced from arXiv: 2506.22852 by the authors.

Figure 1
Figure 1. Overview of the three methods to improve factual [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the RAG-based dialog systems: (a) the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of the agent-based dialog systems. The [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: An illustration of the prompts for the LLMs, as well as an example turn in the dialog in the RAG-based and agent-based [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 21 canonical work pages

  1. [11]

    Fireact: Toward language agent fine- tuning,

    Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Collier, Karthik Narasimhan, and Shunyu Yao, “Fireact: Toward language agent fine- tuning,”arXiv preprint arXiv:2310.05915, 2023

    arXiv 2023

  2. [26]

    Raft: Adapting language model to domain specific rag,

    Tianjun Zhang, Shishir G Patil, Naman Jain, Sheng Shen, Matei Zaharia, Ion Stoica, and Joseph E Gonzalez, “Raft: Adapting language model to domain specific rag,”arXiv preprint arXiv:2403.10131, 2024

    arXiv 2024

  3. [1]

    GPT-4 technical report,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al., “GPT-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  4. [2]

    Palm: Scaling language modeling with pathways,

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al., “Palm: Scaling language modeling with pathways,”Journal of Machine Learning Research, 2023

    work page 2023

  5. [3]

    Survey of hallucination in natural language generation,

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung, “Survey of hallucination in natural language generation,”ACM Computing Surveys, 2023

    work page 2023

  6. [4]

    Retrieval-augmented generation for knowledge-intensive NLP tasks,

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K ¨uttler, Mike Lewis, Wen-tau Yih, Tim Rockt ¨aschel, et al., “Retrieval-augmented generation for knowledge-intensive NLP tasks,”NeurIPS, 2020

    work page 2020

  7. [5]

    Few-shot learning with retrieval augmented language models,

    Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave, “Few-shot learning with retrieval augmented language models,”arXiv preprint arXiv:2208.03299, 2022

    arXiv 2022

  8. [6]

    Knowledge-retrieval task-oriented dialog systems with semi- supervision,

    Yucheng Cai, Hong Liu, Zhijian Ou, Yi Huang, and Junlan Feng, “Knowledge-retrieval task-oriented dialog systems with semi- supervision,” inINTERSPEECH, 2023

    work page 2023

Show all 31 references
  1. [7]

    Generative agents: Interactive simulacra of human behavior,

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein, “Generative agents: Interactive simulacra of human behavior,” inProceedings of the 36th annual acm symposium on user interface software and technology, 2023, pp. 1–22

    2023

  2. [8]

    Toolformer: Language models can teach themselves to use tools,

    Timo Schick, Jane Dwivedi-Yu, Roberto Dess `ı, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom, “Toolformer: Language models can teach themselves to use tools,”NeurIPS, 2024

    2024

  3. [9]

    React: Synergizing reasoning and acting in language models,

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao, “React: Synergizing reasoning and acting in language models,” inThe Eleventh International Conference on Learning Representations, 2023

    2023

  4. [10]

    Can generalist foundation models outcompete special-purpose tuning? Case study in medicine,

    Harsha Nori, Yin Tat Lee, Sheng Zhang, Dean Carignan, Richard Edgar, Nicolo Fusi, Nicholas King, Jonathan Larson, Yuanzhi Li, Weishung Liu, Renqian Luo, Scott Mayer McKinney, Robert Osazuwa Ness, Hoifung Poon, Tao Qin, Naoto Usuyama, Chris White, and Eric Horvitz, “Can general...

    2023 arXiv

  5. [12]

    Efficient tool use with chain-of-abstraction reasoning,

    Silin Gao, Jane Dwivedi-Yu, Ping Yu, Xiaoqing Ellen Tan, Ramakanth Pasunuru, Olga Golovneva, Koustuv Sinha, Asli Celikyilmaz, Antoine Bosselut, and Tianlu Wang, “Efficient tool use with chain-of-abstraction reasoning,”arXiv preprint arXiv:2401.17464, 2024

    2024 arXiv

  6. [13]

    Rag vs fine-tuning: Pipelines, tradeoffs, and a case study on agriculture,

    Aman Gupta, Anup Shirgaonkar, Angels de Luis Balaguer, Bruno Silva, Daniel Holstein, Dawei Li, Jennifer Marsman, Leonardo O Nunes, Mahsa Rouzbahman, Morris Sharp, et al., “Rag vs fine-tuning: Pipelines, tradeoffs, and a case study on agriculture,”arXiv preprint arXiv:2401.08406, 2024

    2024 arXiv

  7. [14]

    Fine tuning vs. retrieval augmented generation for less popular knowledge,

    Heydar Soudani, Evangelos Kanoulas, and Faegheh Hasibi, “Fine tuning vs. retrieval augmented generation for less popular knowledge,” in Proceedings of the 2024 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Re...

    2024

  8. [15]

    Lora: Low-rank adaptation of large language models.,

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al., “Lora: Low-rank adaptation of large language models.,”ICLR, vol. 1, no. 2, pp. 3, 2022

    2022

  9. [16]

    Scaling instruction-finetuned language models,

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Sid- dhartha Brahma, et al., “Scaling instruction-finetuned language models,” Journal of Machine Learning Research, vol. 25, no. 70, pp. 1–53, 2024

    2024

  10. [17]

    The 2nd FutureDial challenge: Dialog systems with retrieval aug- mented generation (FutureDial-RAG),

    Yucheng Cai, Si Chen, Yuxuan Wu, Yi Huang, Junlan Feng, and Zhijian Ou, “The 2nd FutureDial challenge: Dialog systems with retrieval aug- mented generation (FutureDial-RAG),” in2024 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2024, pp. 1091–1098

    2024

  11. [18]

    Language models are few-shot learners,

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al., “Language models are few-shot learners,” NeurIPS, 2020

    2020

  12. [19]

    Chain-of-thought prompting elicits reasoning in large language models,

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al., “Chain-of-thought prompting elicits reasoning in large language models,”NeurIPS, 2022

    2022

  13. [20]

    Training language models to follow instructions with human feedback,

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wain- wright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al., “Training language models to follow instructions with human feedback,”NeurIPS, 2022

    2022

  14. [21]

    Llama 2: Open foundation and fine- tuned chat models,

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Alma- hairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al., “Llama 2: Open foundation and fine- tuned chat models,”arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  15. [22]

    Dense passage retrieval for open-domain question answering,

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih, “Dense passage retrieval for open-domain question answering,” inEMNLP, 2020

    2020

  16. [23]

    Re2G: Retrieve, rerank, generate,

    Michael Glass, Gaetano Rossiello, Md Faisal Mahbub Chowdhury, Ankita Naik, Pengshan Cai, and Alfio Gliozzo, “Re2G: Retrieve, rerank, generate,” inNAACL-HLT, 2022

    2022

  17. [24]

    Unsupervised dense information retrieval with contrastive learning,

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave, “Unsupervised dense information retrieval with contrastive learning,”Transactions on Machine Learning Research, 2022

    2022

  18. [25]

    REALM: retrieval-augmented language model pre-training,

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang, “REALM: retrieval-augmented language model pre-training,” in ICML, 2020

    2020

  19. [27]

    Self-RAG: Learning to retrieve, generate, and critique through self-reflection,

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi, “Self-RAG: Learning to retrieve, generate, and critique through self-reflection,” inICLR, 2024

    2024

  20. [28]

    Demonstrate-search- predict: Composing retrieval and language models for knowledge- intensive NLP,

    Omar Khattab, Keshav Santhanam, Xiang Lisa Li, David Hall, Percy Liang, Christopher Potts, and Matei Zaharia, “Demonstrate-search- predict: Composing retrieval and language models for knowledge- intensive NLP,”arXiv preprint arXiv:2212.14024, 2022

    2022 arXiv

  21. [29]

    BERT: Pre-training of deep bidirectional transformers for language understanding,

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” inNAACL-HLT, 2019

    2019

  22. [30]

    Language models are unsupervised multitask learners,

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever, “Language models are unsupervised multitask learners,”OpenAI Blog, 2019

    2019

  23. [31]

    Bertscore: Evaluating text generation with bert,

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi, “Bertscore: Evaluating text generation with bert,” inICLR, 2019

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.