Pith. sign in

REVIEW 4 major objections 6 minor 37 references

QuIM-RAG: Advancing Retrieval-Augmented Generation with Inverted Question Matching for Enhanced QA Performance

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that question-to-question matching via an inverted index of machine-generated questions raises closed-corpus QA faithfulness to 1.00 and BERTScore F1 to 0.67, versus 0.69 and 0.31 for traditional chunk-vector RAG.

desk verdict Plausible question-to-question retrieval system, but the evaluation is too thin to support the 'outperforms traditional RAG' claim. read the letter →

arxiv 2501.02702 v1 pith:3AQ3B2Y2 submitted 2025-01-06 cs.CL cs.AIcs.LG

Binita Saha , Utsha Saha , Muhammad Zubair Malik This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords retrieval-augmentedgenerationquestionansweringinvertedindexembeddingquantizationhallucinationmitigationBERTScoreRAGAS
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes QuIM-RAG (question-to-question inverted index matching), a retrieval design in which each chunk of a document collection is first turned into a set of likely questions, and a user's query is matched against those questions rather than against raw text. Its claim is that this retrieval index, built from generated questions and quantized prototype buckets, retrieves more relevant context and lets an open-source generator produce answers that are more faithful and less prone to hallucination. On a custom 500+ page university corpus with manually checked ground truth, the reported result is BERTScore F1 of 0.67 and RAGAS faithfulness of 1.00 for the proposed system, against 0.31 and 0.69 for a traditional chunk-vector RAG baseline. The paper also argues that the carefully curated question-based dataset itself improves both systems, so data preparation and retrieval design are treated as joint causes of the gain. If the result holds, organizations with stable document collections could get more reliable, source-linked answers from a general-purpose open LLM without fine-tuning.

What carries the argument

The load-bearing mechanism is the inverted index that maps each quantized prototype to the set of question embeddings and source chunks that were assigned to it. Question generation turns raw chunks into searchable intents, embedding places those intents in the same vector space as the user query, quantization groups them into prototype buckets by cosine similarity, and the top-3 selection inside the matched bucket supplies the generator's context. The index is what converts a large corpus into a small, relevant context before generation.

What would settle it

On a held-out set of fresh questions from the same corpus, measure whether the top-3 retrieved chunks contain the sentence from which the ground-truth answer was written; if retrieval recall is much lower than the reported faithfulness of 1.00, the shortfall lies in question coverage or prototype quantization, not in the generator. Re-running the same retrieval without quantization and comparing recall would further separate the two.

Watch

Extended reading notes

Core claim

The central claim is that the QA task for a limited corpus should be posed as question-to-question matching: for every chunk, an instruction-following language model generates a set of questions intended to cover the chunk's key information; those questions are embedded, quantized to nearest prototypes, and stored in an inverted index that maps each prototype to its question embeddings and source chunks. A user query follows the same embed-and-quantize path, and the top three matching questions in the prototype bucket bring their chunks into the generator's context. In the reported evaluation, this pipeline reaches RAGAS faithfulness 1.00, answer relevancy 0.99, context precision 0.92, context recall 0.74, and BERTScore F1 0.67, while the traditional chunk-vector RAG baseline reaches 0.69, 0.79, 0.45, 0.39, and 0.31 on the same corpus. The paper reads these numbers as evidence that question-indexed retrieval plus a curated domain corpus mitigates information dilution and hallucination in domain-specific QA.

Load-bearing premise

The load-bearing premise is that the question-generation step produces questions covering every piece of information a user might later ask about, since a chunk can only be retrieved through one of its generated questions.

Editorial extensions

If this is right

  • If the reported numbers hold, question-to-question matching is a practical alternative to direct chunk-vector retrieval for closed-corpus QA, with BERTScore F1 rising from 0.31 to 0.67 and RAGAS faithfulness from 0.69 to 1.00 in this setting.
  • The custom question-derived dataset improves every reported metric for both systems, so corpus preparation is a separable source of the gain.
  • Retrieving the top-3 question matches yields context that grounds the generator's answers and lets the system return source links alongside each response.
  • Prompting the generator to decline out-of-domain questions gives the system a defined boundary, which is part of why the reported answers avoid hallucinated content.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I infer that retrieval quality is capped by question-generation coverage: any factual content that the question generator never turned into a question is unreachable, so adding more or more diverse questions per chunk should directly improve recall.
  • A testable extension the paper leaves implicit is removing the quantization step and comparing retrieval recall, which would show whether the efficiency gain costs any matching accuracy.
  • I infer that the approach transfers best to corpora with predictable user intents, such as catalogs, policy documents, and handbooks, where generated questions can anticipate the space of queries; for open-ended corpora, question coverage becomes harder to guarantee.
  • Because the same corpus and generator were used across conditions, the paper's comparison isolates retrieval and data preparation as the sources of the difference, but a human user study would be the next check that the automatic scores correspond to more useful answers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes QuIM-RAG, a retrieval-augmented generation architecture in which each corpus chunk is first converted into a set of GPT-3.5-turbo-instruct-generated questions. These questions are embedded with BAAI/bge-large-en-v1.5, quantized to prototype vectors, and stored in an inverted index that maps prototypes back to chunks. At query time, the user query is embedded and quantized, the top-3 closest stored questions are selected, and their associated chunks are passed as context to Llama-3-8B-instruct together with a custom prompt designed to reduce hallucination and handle out-of-domain queries. The authors construct a custom corpus from 687 pages of the NDSU Career Advising and Catalog websites (22,609 generated questions from 731 chunks), prepare manual ground-truth QA pairs, and compare QuIM-RAG against a traditional RAG baseline using BERTScore and RAGAS. The headline result is a large improvement, e.g., F1 from 0.31 (traditional RAG on traditional data) to 0.67 (QuIM-RAG on custom data).

Significance. If the reported gains are real, the question-to-question inverted matching idea is a practically interesting alternative to chunk-vector retrieval, and the detailed corpus-construction pipeline, including manual review and source-link preservation, is a useful engineering contribution. The paper also has the merit of evaluating with modern semantic metrics (BERTScore, RAGAS) rather than n-gram metrics. However, the current quantitative evidence is too thin to support the central claim: the entire evaluation is one table of point estimates without test-set size, variance, significance tests, or a retrieval-level evaluation, and the baseline is underspecified. The architecture's load-bearing assumption, that generated questions cover all information a user might ask about, is never measured. As it stands, the paper is a plausible engineering proposal whose validation is incomplete.

major comments (4)
  1. [Section 5, Table 2] The entire quantitative evaluation is a single table of point estimates. The paper does not report the number of test queries, any measure of variance or confidence intervals, or significance tests, and the F1 cell for QuIM-RAG on custom data is printed as '0.0.67'. With no indication of the test-set size or per-query score distribution, the large claimed gaps (e.g., 0.31 vs 0.67 in F1) cannot be distinguished from noise, from a small or hand-picked test set, or from coincidental evaluation conditions. The constant Harmfulness value of 0 across all systems further suggests that this metric may be too coarse or the test set too small to be informative. Please provide the number of test queries, per-query score distributions, error bars or bootstrap intervals, and a paired statistical test where appropriate.
  2. [Section 3.2 and Section 4.4] The retrieval mechanism is the paper's core contribution, but it is never directly evaluated. The paper reports only end-to-end BERTScore and RAGAS and never reports retrieval recall@k, precision@k, or any measurement of how often the correct chunk is among the top-3 retrieved chunks. This is load-bearing because in the proposed design a chunk is reachable only through the questions generated from it: if GPT-3.5-turbo-instruct fails to generate a question that is semantically close to a user query, that chunk is simply invisible to retrieval. The manual review described in Section 4.1 checks the quality of generated questions against their own chunks but does not measure coverage over the space of plausible user queries. Please add a retrieval evaluation with held-out queries and known gold chunks, and include an ablation in which test queries target facts deliberately withheld during question generation.
  3. [Section 4.7 and Table 2] The 'traditional RAG' baseline is underspecified, which makes the comparison non-reproducible. The paper does not state which embedding model, chunking procedure, retriever, top-k value, prompt, or generation model were used for the traditional RAG baseline, nor whether the baseline was run by the authors with identical downstream components and only the retrieval mechanism changed. In addition, the 'Traditional' versus 'Custom' dataset conditions differ in multiple ways: raw web chunks versus manually reviewed, question-enriched chunks. The architecture effect and the dataset-curation effect are therefore entangled. Please describe the baseline precisely and, ideally, run an ablation that holds the dataset fixed and varies only the retrieval index (vector-chunk retrieval vs. question-to-question inverted matching).
  4. [Section 4.2 and Section 3.2] There is a potential circularity between the retrieval index and the evaluation data. The ground-truth QA pairs are constructed from the same curated chunks that define the retrieval index, and the retrieval index is built from questions generated from those same chunks. If the test questions were written with knowledge of the generated questions, or if they overlap substantially with them, retrieval will look artificially effective because the system is matching test questions to near-duplicates of themselves rather than to genuinely new user queries. The paper needs to state explicitly how the test questions were selected, whether they are disjoint from the GPT-generated questions, and how the annotators were prevented from relying on the generated questions. At minimum, report the lexical and semantic overlap between the test questions and the generated question set.
minor comments (6)
  1. [Table 2] The F1 score for QuIM-RAG on custom data is printed as '0.0.67'; this should be '0.67'.
  2. [Section 4.4] The text refers to 'llAma3-7b-instruct', while Section 4.5 and the abstract refer to 'Llama3-8b-instruct'; the model name should be consistent and correct.
  3. [Section 3.2] The quantization equations use 'arg min_p CosineSimilarity(v, p)'; since one normally minimizes cosine distance (1 - cosine similarity), the text should clarify the distance measure and avoid the confusing phrase 'minimizing the cosine similarity distance'.
  4. [Section 4.1] The chunking description says 'chunking the data into 1000 tokens and creating overlapping chunks of 200 characters', mixing token and character units; please specify the chunk size and overlap in consistent units.
  5. [References] References [34] and [36] are not the standard citations for BLEU and ROUGE; the original sources (Papineni et al., 2002, and Lin, 2004) should be cited.
  6. [Section 5.1] The BERTScore formulas contain notation slips, for example the subscripts in the pairwise cosine similarity expression and the P_BERT/R_BERT definitions; these should be corrected for reproducibility.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; QuIM-RAG's reported gain rests on an empirical comparison with external metrics, not on a derivation that reduces to its own inputs.

full rationale

QuIM-RAG is an engineered retrieval pipeline, not a derived mathematical claim. The load-bearing assertion—that matching user queries to generated questions outperforms chunk-vector retrieval—is supported by a direct empirical comparison (Table 2) using external metrics (BERTScore and RAGAS) against a traditional RAG baseline. Those numbers could have come out differently, so the result is not forced by construction. The only self-citation, [8], is used to introduce the general concept of RAG and carries none of the argument; it is not a load-bearing uniqueness claim. The principal evidence-based concerns are coverage and evaluation, not circularity: Section 3.2 makes a chunk reachable only through its generated questions, and Section 4.4 reports top-3 retrieval without reporting retrieval recall@k or a coverage ablation; Section 5.2.1's RAGAS answer-relevance metric generates questions from the answer, which mirrors QuIM-RAG's question-matching design; and the ground-truth QA pairs are manually prepared from the same corpus. These are measurement and threat-to-validity issues that could inflate the apparent gain, but they do not reduce the claimed result to its own inputs. No equation defines the claimed output in terms of fitted inputs, and no parameter is fitted and then renamed a prediction. Accordingly, there is no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The retrieval pipeline's success depends on unstated or untested choices: GPT-3.5 question coverage, prototype quantization loss, top-k, chunk size, and the representativeness of manually built ground truth. None are measured or ablated, so the reported comparison carries the full burden of these assumptions.

free parameters (4)
  • Top-k retrieved questions = 3
    Section 4.4 fixes k=3 with no ablation or sensitivity analysis.
  • Chunk size and overlap = 1000 tokens, 200-character overlap
    Section 4.1 sets these values; the units are inconsistent and no sensitivity analysis is reported.
  • Number and construction of prototype vectors = unspecified
    Section 4.3 quantizes embeddings to prototypes stored in chromaDb, but does not state how many prototypes exist or how they are selected, so an unstated hyperparameter controls retrieval quality.
  • Minimum page length filter = 250 characters
    Section 4.1 filters out short pages at this threshold without justification.
assumptions (5)
  • domain assumption GPT-3.5-turbo-instruct-generated questions fully cover all information in each chunk.
    Section 3.2 retrieves chunks only via generated questions, so any chunk content not turned into a question is unreachable. No coverage measurement or ablation is provided.
  • domain assumption Cosine-similarity quantization to the nearest prototype preserves retrieval accuracy.
    Sections 3.2 and 4.3 replace exact vector matching with a single prototype lookup. Queries and relevant questions that fall in different prototypes are missed, and no quantization error analysis is given.
  • domain assumption The manually curated ground truth QA set is representative of real user queries.
    Section 4.2 describes manual curation by the authors without stating the number of test questions, sampling strategy, or inter-annotator agreement.
  • standard math Standard vector-space retrieval mathematics (cosine similarity, inverted index) works as expected.
    The algorithmic steps in Section 3.2 rely on conventional IR machinery, which is not controversial.
  • domain assumption BERTScore and RAGAS are valid proxies for answer quality on this corpus.
    The paper adopts these metrics without human validation on this corpus; RAGAS answer relevance generates questions from answers using an LLM, which mirrors the question-generation design of QuIM-RAG and may bias the comparison.

how reviews work

0 comments
Cite this review

Pith. "Pith review of QuIM-RAG: Advancing Retrieval-Augmented Generation with Inverted Question Matching for Enhanced QA Performance." pith.science (2026). https://pith.science/paper/3AQ3B2Y2

@misc{pith2026250102702,
  author       = {Pith},
  title        = {Pith review of: QuIM-RAG: Advancing Retrieval-Augmented Generation with Inverted Question Matching for Enhanced QA Performance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3AQ3B2Y2}},
  note         = {Machine review of arXiv:2501.02702}
}
read the original abstract

This work presents a novel architecture for building Retrieval-Augmented Generation (RAG) systems to improve Question Answering (QA) tasks from a target corpus. Large Language Models (LLMs) have revolutionized the analyzing and generation of human-like text. These models rely on pre-trained data and lack real-time updates unless integrated with live data tools. RAG enhances LLMs by integrating online resources and databases to generate contextually appropriate responses. However, traditional RAG still encounters challenges like information dilution and hallucinations when handling vast amounts of data. Our approach addresses these challenges by converting corpora into a domain-specific dataset and RAG architecture is constructed to generate responses from the target document. We introduce QuIM-RAG (Question-to-question Inverted Index Matching), a novel approach for the retrieval mechanism in our system. This strategy generates potential questions from document chunks and matches these with user queries to identify the most relevant text chunks for generating accurate answers. We have implemented our RAG system on top of the open-source Meta-LLaMA3-8B-instruct model by Meta Inc. that is available on Hugging Face. We constructed a custom corpus of 500+ pages from a high-traffic website accessed thousands of times daily for answering complex questions, along with manually prepared ground truth QA for evaluation. We compared our approach with traditional RAG models using BERT-Score and RAGAS, state-of-the-art metrics for evaluating LLM applications. Our evaluation demonstrates that our approach outperforms traditional RAG architectures on both metrics.

Figures

Figures reproduced from arXiv: 2501.02702 by the authors.

Figure 1
Figure 1. Overall Architecture of Corpus Preparation for Modified RAG [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Illustration of Inverted Index Construction for Question Matching [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Overall Retrieval and Generation Architecture for RAG [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The upper section details a prompt designed for creating a custom dataset, focusing [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Workflow of QuIM-RAG system and Traditional RAG system for User Query Process [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 18 canonical work pages

  1. [1]

    Language models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” Advances in neural information processing systems , vol. 33, pp. 1877–1901, 2020

    work page 1901

  2. [2]

    Empower large language model to perform better on industrial domain-specific question answering,

    Z. Wang, F. Yang, P. Zhao, L. Wang, J. Zhang, M. Garg, Q. Lin, and D. Zhang, “Empower large language model to perform better on industrial domain-specific question answering,” arXiv preprint arXiv:2305.11541 , 2023

    arXiv 2023

  3. [3]

    Large language models struggle to learn long-tail knowledge,

    N. Kandpal, H. Deng, A. Roberts, E. Wallace, and C. Raffel, “Large language models struggle to learn long-tail knowledge,” in International Conference on Machine Learning . PMLR, 2023, pp. 15 696–15 707

    2023

  4. [4]

    Fine-tuning or retrieval? comparing knowledge injection in llms,

    O. Ovadia, M. Brief, M. Mishaeli, and O. Elisha, “Fine-tuning or retrieval? comparing knowledge injection in llms,” arXiv preprint arXiv:2312.05934 , 2023. 16

    arXiv 2023

  5. [5]

    Overcoming catastrophic forgetting in neural networks,

    J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al. , “Overcoming catastrophic forgetting in neural networks,” Proceedings of the national academy of sciences , vol. 114, no. 13, pp. 3521–3526, 2017

    2017

  6. [6]

    An empirical in- vestigation of catastrophic forgetting in gradient-based neural networks,

    I. J. Goodfellow, M. Mirza, D. Xiao, A. Courville, and Y. Bengio, “An empirical in- vestigation of catastrophic forgetting in gradient-based neural networks,” arXiv preprint arXiv:1312.6211, 2013

    arXiv 2013

  7. [7]

    Recall and learn: Fine-tuning deep pretrained language models with less forgetting,

    S. Chen, Y. Hou, Y. Cui, W. Che, T. Liu, and X. Yu, “Recall and learn: Fine-tuning deep pretrained language models with less forgetting,” arXiv preprint arXiv:2004.12651 , 2020

    arXiv 2004

  8. [8]

    Enhancing international graduate student experience through ai- driven support systems: A llm and rag-based approach,

    B. Saha and U. Saha, “Enhancing international graduate student experience through ai- driven support systems: A llm and rag-based approach,” in 2024 International Conference on Data Science and Its Applications (ICoDSA) , North Dakota State University. IEEE, 2024, pp. 300–304, in press

    work page 2024

Show all 37 references
  1. [9]

    Retrieval-augmented generation for knowledge-intensive nlp tasks,

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. K¨ uttler, M. Lewis, W.-t. Yih, T. Rockt¨ aschelet al. , “Retrieval-augmented generation for knowledge-intensive nlp tasks,” Advances in Neural Information Processing Systems , vol. 33, pp. 9459–9474, 2020

    2020

  2. [10]

    Text and code embeddings by contrastive pre-training,

    A. Neelakantan, T. Xu, R. Puri, A. Radford, J. M. Han, J. Tworek, Q. Yuan, N. Tezak, J. W. Kim, C. Hallacy et al., “Text and code embeddings by contrastive pre-training,”arXiv preprint arXiv:2201.10005, 2022

    2022 arXiv

  3. [11]

    Retrieval-augmented generation for large language models: A survey,

    Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, and H. Wang, “Retrieval-augmented generation for large language models: A survey,” arXiv preprint arXiv:2312.10997, 2023

    2023 arXiv

  4. [12]

    On the dangers of stochas- tic parrots: Can language models be too big?

    E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell, “On the dangers of stochas- tic parrots: Can language models be too big?” in Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , 2021, pp. 610–623

    2021

  5. [13]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al. , “Language models are unsupervised multitask learners,” OpenAI blog, vol. 1, no. 8, p. 9, 2019

    2019

  6. [14]

    Palm: Scaling language modeling with pathways,

    A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann et al., “Palm: Scaling language modeling with pathways,” Journal of Machine Learning Research , vol. 24, no. 240, pp. 1–113, 2023

    2023

  7. [15]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Ba- tra, P. Bhargava, S. Bhosaleet al., “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288 , 2023

    2023 arXiv

  8. [16]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing sys- tems, vol. 30, 2017

    2017

  9. [17]

    A survey on large language models: Applications, challenges, limitations, and practical usage,

    M. U. Hadi, R. Qureshi, A. Shah, M. Irfan, A. Zafar, M. B. Shaikh, N. Akhtar, J. Wu, S. Mirjalili et al., “A survey on large language models: Applications, challenges, limitations, and practical usage,” Authorea Preprints, 2023. 17

    2023

  10. [18]

    Retrieving supporting evidence for llms generated answers,

    S. Huo, N. Arabzadeh, and C. L. Clarke, “Retrieving supporting evidence for llms generated answers,” arXiv preprint arXiv:2306.13781 , 2023

    2023 arXiv

  11. [19]

    A bibliometric review of large language models research from 2017 to 2023,

    L. Fan, L. Li, Z. Ma, S. Lee, H. Yu, and L. Hemphill, “A bibliometric review of large language models research from 2017 to 2023,” ArXiv, vol. abs/2304.02020, 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:257952516

    2017 arXiv

  12. [20]

    A non-factoid question- answering taxonomy,

    V. Bolotova, V. Blinov, F. Scholer, W. B. Croft, and M. Sanderson, “A non-factoid question- answering taxonomy,” in Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval , 2022, pp. 1196–1207

    2022

  13. [21]

    Qa dataset explosion: A taxonomy of nlp resources for question answering and reading comprehension,

    A. Rogers, M. Gardner, and I. Augenstein, “Qa dataset explosion: A taxonomy of nlp resources for question answering and reading comprehension,” ACM Computing Surveys , vol. 55, no. 10, pp. 1–45, 2023

    2023

  14. [22]

    Multi-domain multilingual question answering,

    S. Ruder and A. Sil, “Multi-domain multilingual question answering,” Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: Tutorial Abstracts,

    2021

  15. [23]

    Survey of hallucination in natural language generation,

    Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung, “Survey of hallucination in natural language generation,” ACM Computing Surveys, vol. 55, no. 12, pp. 1–38, 2023

    2023

  16. [24]

    Augmented language models: a survey,

    G. Mialon, R. Dess ` ı, M. Lomeli, C. Nalmpantis, R. Pasunuru, R. Raileanu, B. Rozi` ere, T. Schick, J. Dwivedi-Yu, A. Celikyilmaz et al. , “Augmented language models: a survey,” arXiv preprint arXiv:2302.07842 , 2023

    2023 arXiv

  17. [25]

    Internet-augmented lan- guage models through few-shot prompting for open-domain question answering,

    A. Lazaridou, E. Gribovskaya, W. Stokowiec, and N. Grigorev, “Internet-augmented lan- guage models through few-shot prompting for open-domain question answering,” arXiv preprint arXiv:2203.05115, 2022

    2022 arXiv

  18. [26]

    Paq: 65 million probably-asked questions and what you can do with them,

    P. Lewis, Y. Wu, L. Liu, P. Minervini, H. K¨ uttler, A. Piktus, P. Stenetorp, and S. Riedel, “Paq: 65 million probably-asked questions and what you can do with them,” Transactions of the Association for Computational Linguistics , vol. 9, pp. 1098–1115, 2021

    2021

  19. [27]

    A reliable knowledge processing framework for combustion science using foundation models,

    V. Sharma and V. Raman, “A reliable knowledge processing framework for combustion science using foundation models,” arXiv preprint arXiv:2401.00544 , 2023

    2023 arXiv

  20. [28]

    Conditioning chat-gpt for information retrieval: The unipa- gpt case study,

    I. Siragusa and R. Pirrone, “Conditioning chat-gpt for information retrieval: The unipa- gpt case study,” in Proceedings of the Seventh Workshop on Natural Language for Artificial Intelligence (NL4AI 2023) co-located with 22th International Conference of the Italian As- sociat...

    2023

  21. [29]

    Reinforcement learning for optimizing rag for domain chatbots,

    M. Kulkarni, P. Tangarajan, K. Kim, and A. Trivedi, “Reinforcement learning for optimizing rag for domain chatbots,” arXiv preprint arXiv:2401.06800 , 2024

    2024 arXiv

  22. [30]

    Quadro: Dataset and models for question- answer database retrieval,

    S. Campese, I. Lauriola, and A. Moschitti, “Quadro: Dataset and models for question- answer database retrieval,” arXiv preprint arXiv:2304.01003 , 2023

    2023 arXiv

  23. [31]

    Mteb: Massive text embedding benchmark,

    N. Muennighoff, N. Tazi, L. Magne, and N. Reimers, “Mteb: Massive text embedding benchmark,” arXiv preprint arXiv:2210.07316 , 2022

    2022 arXiv

  24. [32]

    Bertscore: Evaluating text generation with bert,

    T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “Bertscore: Evaluating text generation with bert,” arXiv preprint arXiv:1904.09675 , 2019. 18

    1904 arXiv

  25. [33]

    Ragas: Automated evaluation of retrieval augmented generation,

    S. Es, J. James, L. Espinosa-Anke, and S. Schockaert, “Ragas: Automated evaluation of retrieval augmented generation,” arXiv preprint arXiv:2309.15217 , 2023

    2023 arXiv

  26. [34]

    Rouge-ss: A new rouge variant for the evaluation of text summarization,

    S. Kumar, A. Solanki, and N. Z. Jhanjhi, “Rouge-ss: A new rouge variant for the evaluation of text summarization,” 2024

    2024

  27. [35]

    Meteor: An automatic metric for mt evaluation with improved correlation with human judgments,

    S. Banerjee and A. Lavie, “Meteor: An automatic metric for mt evaluation with improved correlation with human judgments,” in Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization , 2005, pp. 65– 72

    2005

  28. [36]

    Rouge metric evaluation for text summarization techniques,

    M. Barbella and G. Tortora, “Rouge metric evaluation for text summarization techniques,” Available at SSRN 4120317 , 2022. 19

    2022

  29. [2021]

    Available: https://api.semanticscholar.org/CorpusID:245289877

    [Online]. Available: https://api.semanticscholar.org/CorpusID:245289877

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.