Pith. sign in

REVIEW 4 major objections 5 minor 43 references

GuidedRAG claims that pre-filtering the knowledge base by explicit semantics before retrieval makes RAG more relevant, faster, and more precise than standard dense retrieval.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 07:34 UTC pith:UVWIAV6B

load-bearing objection A useful framework for semantic pre-filtering in RAG, but the headline numbers are oracle-driven and the evaluation needs to be more honest before I'd trust them. the 4 major comments →

arxiv 2607.26071 v1 pith:UVWIAV6B submitted 2026-07-10 cs.IR cs.AI

GuidedRAG: Semantic Steering of Retrieval-Augmented Generation

Matthijs Jansen op de Haar , Tobias St\"ahle , Lorenzo Gatti This is my paper
classification cs.IR cs.AI
keywords retrieval-augmented generationsemantic steeringselection spacedense retrievalretrieval precisionretrieval efficiencyknowledge graphuser intent
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that the retrieval stage of RAG suffers because it searches the whole knowledge base using only a text prompt. GuidedRAG adds a selection stage that uses explicit semantic labels—entities, relations, categories—to carve out a small subset of the knowledge base first, then ranks chunks only inside that subset. On a knowledge graph of 300 Wikipedia articles about the Roman Empire, the authors report 14.0–15.8% higher retrieval relevance, mitigation of a 19.7–27.4% loss in retrieval precision, and a search space reduced by orders of magnitude. They also map 15 existing RAG variants onto the framework, arguing that GuidedRAG extends any canonical RAG pipeline and reduces to the base method when no selection is made.

Core claim

The central claim is that semantic selection before retrieval—rather than a better embedding model, a knowledge graph, or a larger context window—is the lever that improves retrieval. GuidedRAG lets a user specify semantics such as an entity or relation type, forms a selection space by applying set operations over the knowledge base, and retrieves only within that space. The paper reports that relevant facts appear earlier in the ranked chunk list, that retrieval becomes orders of magnitude cheaper, and that queries about non-existent semantics return an empty selection, which prevents false-positive chunks. It further claims that 15 existing RAG variants can be expressed inside the framewor

What carries the argument

The central mechanism is the selection stage: a set of semantic labels (entity, category, relation, metric) combined with set-theoretic operations (union, intersection, difference) defines a selection space S inside the knowledge base. Retrieval then runs cosine similarity only over S, which yields the efficiency gain; when the requested semantics do not exist, S is empty, which yields the precision gain. The second piece is semantic steering: the same labels can switch ranking or augmentation strategies, and they can be passed into the augmented prompt that reaches the generator.

Load-bearing premise

The load-bearing premise is that a user or system can supply semantic labels that match the knowledge base's annotations; the paper's own evaluation uses ground-truth entity semantics rather than automatically tagged ones (and in its precision test, GuidedRAG is omitted because non-existent semantics give the empty set), so with missing, noisy, or misaligned semantics the measured gains are not guaranteed.

What would settle it

Run the same knowledge base and question set but replace ground-truth entity semantics with an automatic tagger, then compare GuidedRAG's ranked-hit rate with base RAG's. If the advantage disappears or reverses at tagger accuracy near 57%—the level the paper reports for its best automatic tagger—the 14–15.8% relevance gain depends on perfect semantics rather than on the selection mechanism itself.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Retrieval cost scales with the size of the selection space instead of the whole corpus, so larger knowledge bases need not slow retrieval when selections are specific.
  • Relevant chunks appear earlier in the ranked list, letting systems fetch fewer chunks and thereby cutting token usage and generation latency.
  • Queries about non-existent semantics yield an empty selection, letting a system detect unanswerable questions before paying retrieval or generation cost.
  • Because the framework specialises to the base RAG method when no selection is made, it can be added to existing pipelines rather than replacing them.
  • Removing semantically irrelevant chunks before ranking reduces the chance that irrelevant context reaches the generator, which should lower hallucination risk.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The reported gains presume ideal semantics. In practice, if semantics are noisy or user-supplied with errors, the benefit is an upper bound; the paper's own tagging experiment (best automatic tagger around 57% accuracy) suggests this gap is material.
  • Editorial inference: A testable extension is to replace entity-level annotations with cheaper document-level metadata—source, date, category. If the relevance and scaling benefits persist, semantic steering becomes practical for corpora that lack knowledge graphs.
  • Editorial inference: The selection-space view offers a formal way to compare RAG systems: each variant can be described by the semantics it uses, the set operation it applies, and its behaviour on an empty selection. That taxonomy could guide design choices independently of implementation.
  • Editorial inference: The framework implies that selection quality, not retriever sophistication, drives retrieval performance. Holding the retriever and generator fixed while varying only the precision of the semantic selection would test this directly.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces GuidedRAG, a framework that adds Configuration and Selection stages to the canonical RAG pipeline. Semantics supplied by the user (or inferred automatically) are used to restrict the knowledge base to a selection space before retrieval, and ranking is performed only within this subset. The authors instantiate GuidedRAG versions of base RAG and GraphRAG and evaluate them on the ARLtR benchmark, reporting that GuidedRAG improves retrieval relevance by 14.0–15.8%, eliminates a 19.7–27.4% loss in retrieval precision, reduces retrieval overhead by orders of magnitude, retrieves fact chunks earlier in the ranking, lowers retrieval turbulence, and achieves 'full coverage' of 15 surveyed RAG variants via a stage-mapping exercise.

Significance. If the quantitative claims held, the paper would demonstrate a simple, general, and cheap way to improve RAG. The framework is clearly described, and the decomposition into Configuration/Selection/Retrieval/Augmentation/Generation is a useful organizing device. The stage mapping of 15 existing approaches in Appendix A is a useful conceptual contribution even without empirical validation. The authors also deserve credit for holding the retrieval method fixed across the compared variants and for validating the relevance annotations with a Cohen's κ check. The Limitations section explicitly acknowledges the dependence on semantic quality. However, the headline numbers are obtained under oracle semantic conditions and an asymmetric precision protocol, so the current empirical contribution is an upper-bound demonstration rather than a deployment-level evaluation.

major comments (4)
  1. [§5.2, Table 4, Fig. 5] The main relevance comparison feeds the GuidedRAG variants ground-truth entity annotations from ARLtR: 'entity semantics are taken directly from ARLtR rather than being automatically tagged prior to retrieval.' The automatic taggers evaluated in §5.1 achieve at most 56.8% accuracy (Table 3), yet none is used in the headline relevance experiment. The 14.0–15.8% improvement is therefore an upper bound conditional on perfect entity specification, and §6.3 itself concedes that 'inaccuracies in the automated annotations may reduce its effectiveness.' Please add an end-to-end experiment using the realistic taggers from Table 3, or at least a sensitivity analysis over semantic noise, reporting hit rates. Without this, the central claim that GuidedRAG improves retrieval relevance is not established for actual use.
  2. [§5.4, Fig. 7] The precision comparison is definitional rather than empirical. GuidedRAG is 'omitted because it scopes the KB using predefined semantics,' making the search space the empty set and yielding zero precision loss by construction. In practice, a user or tagger can supply a semantic that is absent from the KB, or miss the correct semantic; then GuidedRAG either returns nothing or must fall back on unconstrained retrieval. The paper does not measure the recall cost of incorrect or missing semantics. Please run GuidedRAG on the unanswerable questions with predicted semantics, and report precision and recall simultaneously (or a combined measure). The claim in §6.1 that GuidedRAG 'avoids the precision-recall dilemma' is not supported by the current protocol.
  3. [§6.2, Table 6] 'Full coverage across 15 diverse RAG variants, demonstrating generalizability across the literature' overstates what is shown. Table 6 is a conceptual mapping, not an empirical evaluation: for most entries the Selection stage is 'N/A' and no GuidedRAG version is implemented or tested. The claim that GuidedRAG can extend any canonical RAG approach may be plausible, but 'demonstrating generalizability' goes beyond the evidence. Please reframe this as a coverage analysis, or implement and evaluate at least one or two additional variants beyond base RAG and GraphRAG.
  4. [§4.1, §5] All quantitative results come from a single benchmark, ARLtR, which is created by the same group (ref. [27]), and no independent dataset is used. Moreover, the reported experiments provide no confidence intervals or significance tests over questions; the 'three iterations' in §5.2 apply only to LLM relevance annotation, not to the retrieval results. At minimum, report bootstrap confidence intervals for the headline hit-rate differences in Table 4/Fig. 5, and validate on at least one independent knowledge base. This is standard for retrieval papers making generalizable claims.
minor comments (5)
  1. [§5.2] The sentence 'This results in 12,000 total chunks' is confusing: 600 questions × 5 chunks = 3,000 chunks per approach, and 12,000 only arises if the four approaches are counted separately. Please clarify the arithmetic.
  2. [Fig. 7] The figure labels are corrupted in the manuscript (sequences of '/uni000000...' glyphs), making the figure unreadable. Please regenerate the figure with proper text rendering.
  3. [Table 4] The column 'Avg. Hit (k)' is ambiguous; please state explicitly that lower values are better and that questions with no hit are counted at k = 6. A short note in the caption would prevent misinterpretation.
  4. [Eq. (1)] The turbulence measure T = (D_c − 5)/10 should be explained more clearly: the reader must infer that D_c ranges from 5 to 15. Please state the range and the interpretation.
  5. [§6.3] The manuscript states that no other benchmark exists for semantic steering in RAG. This is a strong claim; please cite the absence more carefully, or soften it, since several RAG benchmarks exist even if they lack explicit semantic annotations.

Circularity Check

2 steps flagged

Headline relevance and precision gains are built into the evaluation: gold ARLtR entity annotations are given to GuidedRAG as free inputs while defining relevance, and non-existent semantics are declared to yield the empty set, making the precision advantage definitional.

specific steps
  1. fitted input called prediction [§5.2 Retrieval Relevance; selection definitions in §4.3 Table 2; dataset attributes in §4.1]
    "Furthermore, entity semantics are taken directly from ARLtR rather than being automatically tagged prior to retrieval. ... Each factoid has a corresponding fact (i.e., answer), grounded in the KB. Utilizing this fact, we can determine whether a chunk is relevant to answer a question in the generation stage."

    Relevance is defined by the retrieved chunk containing the fact, and ARLtR links each question to the specific entity used in Table 2 to build the GuidedRAG selection space (E1, E1∩E2, E1∩Arel). Supplying the dataset's own gold entity as the semantic therefore puts the fact-bearing chunks into the candidate set by construction, while RAG and GraphRAG receive no such oracle. The 14.0–15.8% relevance gain is consequently an upper bound of the gold-label linkage, not a measured property of a deployed semantic-steering pipeline; the automatic taggers in §5.1 (55.0%/56.8%) are never connected to the relevance result.

  2. self definitional [§5.4 Retrieval Precision; unanswerable-set definition in §4.1]
    "GuidedRAG is omitted because it scopes the KB using predefined semantics. This makes the search space equivalent to the empty set (Ø), resulting in no loss of precision."

    Qn is defined (in §4.1) as consisting of question-answer pairs 'about semantics that are not present in the KB.' Since GuidedRAG is given exactly those non-existent semantics, an empty selection is a logical consequence of the problem definition. The precision-loss mitigation (19.7–27.4%) is therefore not measured against any retrieval method: any system that returns nothing when handed a non-existent filter would achieve the same zero false-positive rate. The claimed mitigation is true by definition.

full rationale

The paper is transparent about both moves, and parts of the framework are not circular: the search-space reduction in §5.3 is a real design consequence, the turbulence metric (Eq. 1) is measured, and Table 6 is a mapping exercise rather than a circular derivation. However, the two headline empirical claims reduce by construction. In §5.2, GuidedRAG/GraphRAG* are fed ARLtR's own gold entity attributes, while relevance is defined as the retrieved chunk containing the fact and ARLtR links each question to the entity/answer in the KB; the fact-bearing chunk is thus placed in the candidate set by the same annotations that constitute the method's input. The automatic taggers evaluated in §5.1 (55.0% and 56.8% accuracy) are never used in this comparison, so the 14.0–15.8% gain is an oracle upper bound, not a deployment estimate. In §5.4, unanswerable questions are by definition about semantics absent from the KB, and GuidedRAG is omitted because its gold semantic yields the empty set; the 19.7–27.4% precision-loss mitigation is therefore true by definition and never empirically compared. The evaluation also relies entirely on ARLtR, a dataset created by the same authors with no external benchmark check (§6.3), which amplifies—though does not independently establish—the circularity. Score 8 rather than 10 because retrieval ranking within the selected subset and turbulence remain empirical results; nevertheless, the central claims of relevance and precision improvement reduce, respectively, to gold-label input and to the empty-set consequence.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 0 invented entities

The central claim rests on the availability and correctness of semantic annotations, the representativeness of a single self-created dataset, and the validity of LLM-based relevance judgments. The evaluation's use of gold semantics makes it an upper-bound test, and the precision advantage is partly definitional.

free parameters (1)
  • similarity threshold = 0.83
    Chosen in §5.4 as a 'sensible threshold that balances precision and recall' based on the cumulative distributions; used to quantify baseline precision loss.
axioms (4)
  • domain assumption User-provided semantics are available and accurate for each query
    The framework and evaluation assume the Selection stage can be populated with correct semantics; §3.1 says semantics are generated by user or inferred, and §6.3 acknowledges dependence on semantic quality.
  • domain assumption ARLtR is representative of RAG knowledge bases
    All experiments use a single self-created benchmark; §6.3 states 'the reported results may not directly generalize to other knowledge bases or application domains.'
  • ad hoc to paper Ground-truth entity labels used for selection approximate real-world semantic steering
    §5.2 uses entity semantics 'taken directly from ARLtR', giving GuidedRAG oracle semantics not available to the baselines; this assumption is load-bearing for the reported improvements.
  • domain assumption LLM relevance annotations are valid ground truth
    Relevance of chunks is judged by an LLM at temperature zero with author validation on a subset; §5.2 and Appendix D.

pith-pipeline@v1.3.0-alltime-deepseek · 18130 in / 9193 out tokens · 94815 ms · 2026-08-02T07:34:17.244401+00:00 · methodology

0 comments
read the original abstract

In this work, we propose GuidedRAG, a novel extension to traditional Retrieval-Augmented Generation (RAG) that introduces a dedicated selection stage and semantic steering during retrieval. In contrast to current state-of-the-art RAG approaches, which depend on increasingly complex retrieval and knowledge structures, GuidedRAG constrains the knowledge base using semantics before retrieval, aligning the retrieval space with user intent while substantially reducing the search space. Our evaluation shows that GuidedRAG improves retrieval relevance by 14.0-15.8%, mitigates a 19.7-27.4% loss in retrieval precision, and reduces retrieval overhead by orders of magnitude. Moreover, relevant chunks are consistently retrieved earlier in the ranking process, while alignment with user intent improves by 31.8-36.8%. We further show that GuidedRAG achieves full coverage across 15 diverse RAG variants, demonstrating generalizability across the literature. Together, these findings establish semantic steering and selections as a powerful and generalizable paradigm for improving the current state-of-the-art in RAG.

Figures

Figures reproduced from arXiv: 2607.26071 by Lorenzo Gatti, Matthijs Jansen op de Haar, Tobias St\"ahle.

Figure 1
Figure 1. Figure 1: In GuidedRAG we investigate five stages, namely: The Configuration, Selection, Retrieval, Augmentation, and Generation stage. The Configuration stage specifies the knowledge base (KB) and underlying semantics, and configures the degree of user control. In the Retrieval-Augmented Generation (RAG) process, a user specifies semantics and a prompt, which are used in subsequent steps. In the Selection stage, a … view at source ↗
Figure 2
Figure 2. Figure 2: Selection methods use different semantics (e.g., entities or relations) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: RAG Methods over GuidedRAG Framework Stages over specific relation types rather than all relations. These relation hops are often infeasible in traditional GraphRAG systems because inaccuracies in entity taggers can introduce irrelevant context into the selection space. In the implementation and evaluation, we use an LLM (i.e., minimax 2.7 2 ) and an EM (i.e., gemini-embedding-23 ). 4.3 Selection and Retri… view at source ↗
Figure 4
Figure 4. Figure 4: Percentage of Correct Tags per Tagging Method [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Percentage of Questions with at Least One Correct Chunk [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Turbulence of Chunks in Retrieval (0 = low, 1 = high) [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Cumulative Chunks Retained at Similarity Threshold (%) [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: In this example, we illustrate an instance of using [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 2 canonical work pages

  1. [1]

    Chen Amiraz, Florin Cuconasu, Simone Filice, et al. 2025. The Distracting Effect: Understanding Irrelevant Passages in RAG. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 18228–18258. https://doi.org/10.18653/v1/2025.acl-long.892

    doi:10.18653/v1/2025.acl-long.892 2025

  2. [2]

    Dang Anh-Hoang, Vu Tran, and Le-Minh Nguyen. 2025. Survey and analysis of hallucinations in large language models: attribution to prompting strategies or model behavior.Frontiers in Artificial IntelligenceVolume 8 - 2025 (2025). https://doi.org/10.3389/frai.2025.1622292

    arXiv 2025

  3. [3]

    Akari Asai, Zeqiu Wu, Yizhong Wang, et al . 2024. Self-RAG: Learning to Re- trieve, Generate, and Critique through Self-Reflection. InThe Twelfth Interna- tional Conference on Learning Representations. https://openreview.net/forum?id= hSyW5go0v8

    2024

  4. [4]

    Krisztian Balog, Marc Bron, and Maarten De Rijke. 2011. Query modeling for entity search based on terms, categories, and examples.ACM Trans. Inf. Syst.29, 4, Article 22 (2011), 31 pages. https://doi.org/10.1145/2037661.2037667

    arXiv 2011

  5. [5]

    Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, et al. 2022. Improving language models by retrieving from trillions of tokens. https://arxiv.org/abs/ 2112.04426

    Pith/arXiv arXiv 2022

  6. [6]

    Brown, Benjamin Mann, Nick Ryder, et al

    Tom B. Brown, Benjamin Mann, Nick Ryder, et al. 2020. Language models are few-shot learners. InProceedings of the 34th International Conference on Neural Information Processing Systems. Article 159, 25 pages

    2020

  7. [7]

    Jaime Carbonell and Jade Goldstein. 1998. The use of MMR, diversity-based reranking for reordering documents and producing summaries. InProceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 335–336. https://doi.org/10.1145/290941.291025

    arXiv 1998

  8. [8]

    Jacob Cohen. 1960. A Coefficient of Agreement for Nominal Scales.Educational and Psychological Measurement20, 1 (1960), 37–46. https://doi.org/10.1177/ 001316446002000104

    1960

  9. [9]

    Yufeng Du, Minyang Tian, Srikanth Ronanki, et al. 2025. Context Length Alone Hurts LLM Performance Despite Perfect Retrieval. InFindings of the Association for Computational Linguistics: EMNLP 2025. 23281–23298. https://doi.org/10. 18653/v1/2025.findings-emnlp.1264

    2025

  10. [10]

    Darren Edge, Ha Trinh, Newman Cheng, et al . 2025. From Local to Global: A Graph RAG Approach to Query-Focused Summarization. https://arxiv.org/abs/ 2404.16130

    Pith/arXiv arXiv 2025

  11. [11]

    Darío Garigliotti and Krisztian Balog. 2017. On Type-Aware Entity Retrieval. In Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval. 27–34. https://doi.org/10.1145/3121050.3121054

    arXiv 2017

  12. [12]

    Zirui Guo, Lianghao Xia, Yanhua Yu, et al . 2025. LightRAG: Simple and Fast Retrieval-Augmented Generation. InFindings of the Association for Computational Linguistics: EMNLP 2025. 10746–10761. https://doi.org/10.18653/v1/2025.findings- emnlp.568

    doi:10.18653/v1/2025.findings- 2025

  13. [13]

    Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, et al. 2024. HippoRAG: neurobio- logically inspired long-term memory for large language models. InProceedings of the 38th International Conference on Neural Information Processing Systems. Article 1902, 59532–59569 pages

    2024

  14. [14]

    Kelvin Guu, Kenton Lee, Zora Tung, et al. 2020. REALM: retrieval-augmented language model pre-training. InProceedings of the 37th International Conference on Machine Learning. Article 368, 3929–3938 pages

    2020

  15. [15]

    Haoyu Han, Li Ma, Yu Wang, et al. 2025. Rag vs. graphrag: A systematic evaluation and key insights.arXiv preprint arXiv:2502.11371(2025)

    arXiv 2025

  16. [16]

    Sebastian Hofstätter, Jiecao Chen, Karthik Raman, et al. 2023. FiD-Light: Efficient and Effective Retrieval-Augmented Text Generation. InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1437–1447. https://doi.org/10.1145/3539618.3591687

    arXiv 2023

  17. [17]

    Aidan Hogan, Eva Blomqvist, Michael Cochez, et al. 2021. Knowledge graphs. ACM Computing Surveys (Csur)54, 4 (2021), 1–37

    2021

  18. [18]

    Yizheng Huang and Jimmy Xiangji Huang. 2026. A Survey on Retrieval- Augmented Text Generation for Large Language Models.ACM Comput. Surv.58, 12 (2026), 1–38. https://doi.org/10.1145/3805774

    doi:10.1145/3805774 2026

  19. [19]

    Gautier Izacard, Patrick Lewis, Maria Lomeli, et al. 2023. Atlas: few-shot learning with retrieval augmented language models.J. Mach. Learn. Res.24, 1, Article 251 (2023), 11912 – 11954 pages

    2023

  20. [20]

    Soyeong Jeong, Jinheon Baek, Sukmin Cho, et al. 2024. Adaptive-RAG: Learn- ing to Adapt Retrieval-Augmented Large Language Models through Question Complexity. https://arxiv.org/abs/2403.14403

    Pith/arXiv arXiv 2024

  21. [21]

    Zhengbao Jiang, Frank Xu, Luyu Gao, et al. 2023. Active Retrieval Augmented Generation. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 7969–7992. https://doi.org/10.18653/v1/2023.emnlp-main. 495

    doi:10.18653/v1/2023.emnlp-main 2023

  22. [22]

    Hideo Joho and Joemon M Jose. 2025. An Instruction-Response Perspective on Large Language Models in Information Retrieval Tasks. InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. 3843–3852. https://doi.org/10.1145/3726302.3730346

    arXiv 2025

  23. [23]

    Amita Kamath, Robin Jia, and Percy Liang. 2020. Selective Question Answering under Domain Shift. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 5684–5696. https://doi.org/10.18653/v1/2020.acl- main.503

    doi:10.18653/v1/2020.acl- 2020

  24. [24]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, et al. 2020. Dense Passage Retrieval for Open-Domain Question Answering. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 6769–6781. https: //doi.org/10.18653/v1/2020.emnlp-main.550

    doi:10.18653/v1/2020.emnlp-main.550 2020

  25. [25]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, et al. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. InProceedings of the 34th In- ternational Conference on Neural Information Processing Systems. Article 793, 9459–9474 pages

    2020

  26. [26]

    Liu, Kevin Lin, John Hewitt, et al

    Nelson F. Liu, Kevin Lin, John Hewitt, et al . 2024. Lost in the Middle: How Language Models Use Long Contexts.Transactions of the Association for Compu- tational Linguistics12 (2024), 157–173. https://doi.org/10.1162/tacl_a_00638

    doi:10.1162/tacl_a_00638 2024

  27. [27]

    Matthijs Jansen op de Haar, Tobias Stähle, and Lorenzo Gatti. 2026. All Relations Lead to Rome: Automated Knowledge Graph Creation and Question Generation. arXiv:2606.22645 [cs.IR] https://arxiv.org/abs/2606.22645

    Pith/arXiv arXiv 2026

  28. [28]

    Anders Ericsson

    Antti Oulasvirta, Mikael Wahlström, and K. Anders Ericsson. 2011. What does it mean to be good at using a mobile device? An investigation of three levels of experience and skill.International Journal of Human-Computer Studies69, 3 (2011), 155–169. https://doi.org/10.1016/j.ijhcs.2010.11.003

    doi:10.1016/j.ijhcs.2010.11.003 2011

  29. [29]

    Vijay Raghavan, Peter Bollmann, and Gwang S. Jung. 1989. A critical investigation of recall and precision as measures of retrieval system performance.ACM Trans. Inf. Syst.7, 3 (1989), 205–229. https://doi.org/10.1145/65943.65945

    arXiv 1989

  30. [30]

    Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How Much Knowledge Can You Pack Into the Parameters of a Language Model?. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 5418–5426. https://doi.org/10.18653/v1/2020.emnlp-main.437

    doi:10.18653/v1/2020.emnlp-main.437 2020

  31. [31]

    Salton, A

    G. Salton, A. Wong, and C. S. Yang. 1975. A vector space model for automatic indexing.Commun. ACM18, 11 (1975), 613–620. https://doi.org/10.1145/361219. 361220

    work page doi:10.1145/361219 1975

  32. [32]

    Manveer Singh Tamber, Forrest Sheng Bao, Chenyu Xu, et al. 2025. Benchmarking LLM Faithfulness in RAG with Evolving Leaderboards. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track. 799–811. https://doi.org/10.18653/v1/2025.emnlp-industry.54

    doi:10.18653/v1/2025.emnlp-industry.54 2025

  33. [33]

    Haoyu Tian, Yingchaojie Feng, Zhen Wen, et al. 2026. RAGExplorer: A Visual Analytics System for the Comparative Diagnosis of RAG Systems. https://arxiv. org/abs/2601.12991

    arXiv 2026

  34. [34]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, et al. 2017. Attention is all you need. InProceedings of the 31st International Conference on Neural Information Processing Systems. 6000–6010

    2017

  35. [35]

    Elena Volkanovska. 2025. A Study of Errors in the Output of Large Language Models for Domain-Specific Few-Shot Named Entity Recognition.Journal for Language Technology and Computational Linguistics38, 2 (2025), 31–42. https: //doi.org/10.21248/jlcl.38.2025.281

    work page doi:10.21248/jlcl.38.2025.281 2025

  36. [36]

    Liang Wang, Nan Yang, and Furu Wei. 2023. Query2doc: Query Expansion with Large Language Models. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 9414–9423. https://doi.org/10.18653/v1/ 2023.emnlp-main.585

    doi:10.18653/v1/ 2023

  37. [37]

    Watts and Steven H

    Duncan J. Watts and Steven H. Strogatz. 1998. Collective dynamics of ‘small- world’ networks.Nature393, 6684 (1998), 440–442. https://doi.org/10.1038/30918

    doi:10.1038/30918 1998

  38. [38]

    ChengXiang Zhai. 2024. Large Language Models and Future of Information Retrieval: Opportunities and Challenges. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 481–490. https://doi.org/10.1145/3626772.3657848

    arXiv 2024

  39. [39]

    Tianjun Zhang, Shishir G Patil, Naman Jain, et al. 2024. RAFT: Adapting Language Model to Domain Specific RAG. InFirst Conference on Language Modeling. https: //openreview.net/forum?id=rzQGHXNReU

    2024

  40. [40]

    Penghao Zhao, Hailin Zhang, Qinhan Yu, et al . 2024. Retrieval-Augmented Generation for AI-Generated Content: A Survey.CoRRabs/2402.19473 (2024), 1–29. https://doi.org/10.48550/arxiv.2402.19473

    Pith/arXiv arXiv doi:10.48550/arxiv.2402.19473 2024

  41. [41]

    Yutao Zhu, Huaying Yuan, Shuting Wang, et al. 2025. Large Language Models for Information Retrieval: A Survey.ACM Transactions on Information Systems44, 1 (2025), 1–54. https://doi.org/10.1145/3748304 9 Matthijs Jansen op de Haar, Tobias Stähle and Lorenzo Gatti A COMPARISON OF RAG APPROACHES Table 6. This table presents a mapping of 15 distinct RAG appro...

    doi:10.1145/3748304 2025

  42. [42]

    Augmentation You are an LLM assistant

    Hannibal invaded... Augmentation You are an LLM assistant. Only answer the question based on the specified chunks. What is the Capital of the Roman Empire?

  43. [43]

    relevant

    Hannibal invaded... Roman Empire Roman Empire Roman Empire Cosine Similarit y Fig. 8. In this example, we illustrate an instance of using GuidedRAG.(i)During the Configuration stage, a user specifies a prompt and a semantic. In this case, the question concerns the Roman Empire.(ii)Using this semantic, the knowledge base is scoped in the Selection stage to...