The reviewed record of science sign in
Pith

arxiv: 2504.14891 · v1 · pith:G3M3HF4E · submitted 2025-04-21 · cs.CL

Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey

Aoran Gan , Hao Yu , Kai Zhang , Qi Liu , Wenyu Yan , Zhenya Huang , Shiwei Tong , Guoping Hu This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:G3M3HF4Erecord.jsonopen to challenge →

classification cs.CL
keywords evaluationgenerationcomprehensivelanguageretrievalsurveyframeworksknowledge
0
0 comments X
read the original abstract

Recent advancements in Retrieval-Augmented Generation (RAG) have revolutionized natural language processing by integrating Large Language Models (LLMs) with external information retrieval, enabling accurate, up-to-date, and verifiable text generation across diverse applications. However, evaluating RAG systems presents unique challenges due to their hybrid architecture that combines retrieval and generation components, as well as their dependence on dynamic knowledge sources in the LLM era. In response, this paper provides a comprehensive survey of RAG evaluation methods and frameworks, systematically reviewing traditional and emerging evaluation approaches, for system performance, factual accuracy, safety, and computational efficiency in the LLM era. We also compile and categorize the RAG-specific datasets and evaluation frameworks, conducting a meta-analysis of evaluation practices in high-impact RAG research. To the best of our knowledge, this work represents the most comprehensive survey for RAG evaluation, bridging traditional and LLM-driven methods, and serves as a critical resource for advancing RAG development.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. VikingMem: A Memory Base Management System for Stateful LLM-based Applications

    cs.AI 2026-05 unverdicted novelty 5.0

    VikingMem implements the Memory Base paradigm via event-centric extraction and entity updates on VikingDB with temporal compression, claiming up to 30% better retrieval effectiveness on long-term memory benchmarks.

  2. Retrieval-Augmented Generation Must Move Beyond Factual Grounding to Represent Diverse Opinions

    cs.AI 2026-04 unverdicted novelty 5.0

    Opinion-aware RAG with LLM opinion extraction and entity-linked graphs improves retrieval diversity by 26-42% over factual baselines on e-commerce forum data.

  3. Empirical Evaluation of PDF Parsing and Chunking for Financial Question Answering with RAG

    cs.CL 2026-04 unverdicted novelty 5.0

    Systematic tests show that specific PDF parsers combined with overlapping chunking strategies better preserve structure and improve RAG answer correctness on financial QA benchmarks including the new TableQuest dataset.

  4. Reducing Hallucinations in Complex Question Answering using Simple Graph-based Retrieval-Augmented Generation (long version)

    cs.CL 2026-06 unverdicted novelty 4.0

    A graph-augmented RAG system with vector and graph query tools halves hallucinations and raises factual correctness scores on the MoNaCo complex QA benchmark.

  5. Retrieval Augmented Generation Framework for the Nepali Legal Domain Question Answering

    cs.CL 2026-04 conditional novelty 4.0

    A RAG pipeline using BM25 retrieval and GPT-o3 generation achieves 91% Precision@1 and 85% truthfulness for Nepali legal question answering on a curated 100-query benchmark.