Pith. sign in

REVIEW 4 major objections 6 minor 53 references

Am I on the Right Track? What Can Predicted Query Performance Tell Us about the Search Behaviour of Agentic RAG

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Query performance prediction on the first query an agentic RAG model generates gives a weak but consistent signal for final answer quality, even without relevance labels.

desk verdict Honest, well-scoped workshop study with a real proxy-validation gap and an overstated significance claim; worth refereeing for a workshop, not a desk reject. read the letter →

arxiv 2507.10411 v1 pith:SSF4S44B submitted 2025-07-14 cs.IR

Fangzheng Tian , Jinyuan Fang , Debasis Ganguly , Zaiqiao Meng , Craig Macdonald This is my paper
classification cs.IR
keywords AgenticRAGQueryPerformancePredictionretrieval-augmentedgenerationadaptiveretrievalNaturalQuestionsSearch-R1R1-Searcherpost-retrievalQPP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Agentic RAG lets a reasoning model decide when and how to search while answering a question. This paper asks whether query performance prediction (QPP) can substitute for missing relevance judgments on the model-generated search queries, so that the quality of an intermediate retrieval can be estimated without human labels. Using two reinforcement-learning-trained agentic RAG models and three retrievers on the Natural Questions benchmark, the paper finds that better retrievers shorten the reasoning process and improve answer quality, and that the QPP score of the first generated query is positively, though weakly, correlated with final answer F1. The authors read this as a first step toward adaptive retrieval, where QPP would tell the agent whether the retrieved results are worth using.

What carries the argument

The central object is the post-retrieval QPP estimate attached to the first query the agentic generator emits. QPP models predict retrieval effectiveness without human relevance judgments; here they are computed on the retrieved documents for each generated query and then correlated with the F1 score of the final answer. Depending on the retriever, the paper uses NQC (standard deviation of retrieval scores), Max(Score) (the top score as an upper bound on relevance), A-Pair-Ratio (a coherence ratio between top-ranked and bottom-ranked documents), and Dense-QPP (the geometric volume of embeddings). Because the correlation is measured on the first iteration only, the first retrieval acts as the load-bearing early signal for answer quality.

What would settle it

On a dataset with gold evidence passages, such as HotpotQA, one could compute the correlation between each generated query's QPP score and whether its top retrieved documents contain the gold supporting facts; if that correlation is near zero, QPP is not measuring retrieval usefulness and the claimed link to answer quality would not follow.

Watch

Extended reading notes

Core claim

The paper's central finding is that estimated retrieval quality, measured by post-retrieval QPP of the first generated query, is consistently positively correlated with final answer quality across two agentic RAG models (Search-R1 and R1-Searcher) and three retrieval configurations (BM25, BM25 with MonoT5 re-ranking, and E5). Spearman correlations are small, roughly 0.01 to 0.25, and all are significant; the strongest value comes from a coherence-based QPP method applied to E5 retrieval in Search-R1. The same experiments show that more effective retrievers reduce the average number of reasoning-retrieval iterations, and that estimated query quality tends to decrease across iterations. The paper treats the correlation as evidence that QPP offers a usable proxy for retrieval usefulness in a setting where no relevance labels exist for the intermediate queries.

Load-bearing premise

The argument assumes that a QPP score computed against a generated subquery correctly approximates how useful the retrieved documents really are for producing the final answer, even though the paper never validates this proxy against any ground-truth measure of retrieval usefulness.

Editorial extensions

If this is right

  • A higher QPP estimate for the first generated query predicts, with low-to-moderate accuracy, that the agentic model will produce a better final answer.
  • Switching from BM25 to E5 or MonoT5 for the same generator both shortens the average reasoning-retrieval chain and raises answer exact-match and F1 scores.
  • Estimated retrieval quality tends to decline as reasoning continues, so later retrievals on average contribute less useful context than the first one.
  • A practical gate can be built on QPP: if the first query's estimated performance is low, the system may decide to reformulate or retrieve differently.
  • Because the correlations hold across different models and retrievers, QPP can serve as an online signal for answer quality without requiring relevance labels.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if first-query QPP is mostly a question-difficulty signal, then it should predict answer F1 even when the retrieved documents are replaced by random passages; this is a test the paper does not run.
  • Editorial inference: the paper only correlates first-query QPP, so aggregating QPP estimates across all iterations is a natural extension that could lift the observed Spearman values above 0.25.
  • Editorial inference: QPP could serve as a shaped reward in the reinforcement-learning training of future agentic RAG models, rewarding the model for issuing queries whose predicted performance is high; the paper mentions this direction but does not test it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper investigates the role of Query Performance Prediction (QPP) in agentic RAG models, specifically Search-R1 and R1-Searcher, on the Natural Questions dataset with three retrievers (BM25, MonoT5, E5). It reports three findings: (1) more effective retrievers yield higher answer quality (EM/F1) with fewer reasoning iterations; (2) QPP estimates of generated queries tend to decrease over reasoning iterations; and (3) QPP estimates of the first generated query are positively, though weakly, correlated with final answer F1. The paper interprets these results as evidence that QPP can approximate the usefulness of retrieved documents and could support adaptive retrieval in agentic RAG.

Significance. If the central correlation claim is reliable, this is a useful initial study opening a new direction: using QPP as a signal for adaptive retrieval or as a training reward in agentic RAG. The paper's strengths include a large-scale evaluation (3,610 NQ questions), the use of established unsupervised QPP methods, a transparent experimental setup, and appropriate hedging of the practical strength of the correlations. The main weakness is that the load-bearing assumption that QPP estimates approximate retrieval usefulness is not independently validated, and the blanket significance claim over Table 3 is not supported by the magnitudes reported.

major comments (4)
  1. [Section 3.3 and Section 5.3 (RQ-3)] The paper's central claim—that QPP estimates of generated queries are positively correlated with final answer quality—depends on treating QPP as 'a reasonable approximation' of retrieval usefulness (Section 3.3). This proxy is never validated against any independent measure of retrieval usefulness. The confound identified by the reader is real: for the many questions answered in one or two iterations (Figure 3), the first generated query is often the original user question, so QPP on that query largely reflects query specificity and difficulty, and the same difficulty can depress both QPP and final-answer F1. The positive correlations in Table 3 could therefore arise without QPP carrying any information about retrieval quality. To support RQ-3 and the RQ-2 interpretation, the paper should validate the proxy, for example by testing whether QPP correlates with the presence of gold-answer information in the retrieved documents, or by stratifying the analysis across question-difficulty groups.
  2. [Table 3 and Section 5.3] The paper asserts that 'All observed correlations are significant', but with n=3,610, several reported Spearman values are far from significant at the two-sided 0.05 level. For example, rho=0.0096 (NQC/MonoT5 for R1-Searcher) has an approximate p-value of 0.56, and rho=0.0297 (A-Pair-Ratio/E5 for R1-Searcher) has an approximate p-value of 0.07. This directly contradicts the blanket significance claim and weakens the RQ-3 conclusion that QPP can serve as a useful signal. The paper should report p-values or bootstrap confidence intervals for each correlation and clearly state which correlations are statistically reliable.
  3. [Section 5.2 (RQ-2) and Figures 4/5] The decreasing QPP-over-iterations trend is interpreted as evidence that retrieval quality deteriorates as reasoning progresses. This interpretation is confounded by selection: questions that require more iterations are a non-random subset, consisting disproportionately of harder or more ambiguous questions (Figure 3 shows most questions converge in 1-2 iterations). The paper itself acknowledges this in the final paragraph of Section 5.2, but the RQ-2 answer is still stated as a general trend about retrieval quality. The analysis should be conditional on question-level difficulty or should track QPP changes within trajectories that actually reach later iterations, to separate a true retrieval-quality decline from a composition effect.
  4. [Section 5.2 (NQC/MonoT5 exception)] The paper explains the NQC-on-MonoT5 exception by asserting that 'NQC is not an accurate predictor for query performance in MonoT5' and cites Datta et al. [8]. However, no direct evidence is provided for this claim in the present setting. If this exception is used to support the general decreasing-trend conclusion, it should be backed by a diagnostic, such as comparing NQC behavior on MonoT5 against a predictor known to work well on neural re-rankers, rather than only a citation.
minor comments (6)
  1. [Section 5.3] There is a typo: 'We now to turn to examine' should be 'We now turn to examine'.
  2. [Section 5.1] The model name 'Searcher-R1' appears once; this should be 'Search-R1' for consistency.
  3. [Section 3.3] The phrase 'with with' appears in the sentence about the QPP estimates and the downstream answer quality; it should be 'with'.
  4. [Appendix A.2] The text refers to 'a cutoff of 3 iterations' when reporting that 262 out of 3610 queries generated identical sub-queries; this is likely meant to be the top-3 document cutoff used by the retrievers. Please clarify the wording.
  5. [Figure 5] The caption says 'legend as per Figure 4', but Figure 4's legend includes DenseQPP and A-ratio, which are only applicable to E5. In Figure 5, only the applicable predictors appear to be plotted; please make the per-figure legend explicit to avoid ambiguity.
  6. [Section 4.3] The description of Max(Score) says it reflects an 'estimated upper bound of the relevance of the retrieved context', but no formal definition or reference equation is given for how this score is extracted for each retriever. Including a concrete formula or pseudocode would improve reproducibility.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the paper reports empirical correlations against external NQ labels and fits no parameters whose values force the conclusions.

full rationale

The paper does not present a derivation chain that reduces a predicted result to its inputs. RQ-1 compares answer quality and iteration counts across retrievers (Table 1), RQ-2 shows average QPP trends over iterations (Figures 4 and 5), and RQ-3 reports Spearman correlations between QPP scores of the first generated query and final-answer F1 (Table 3). QPP scores come from established unsupervised predictors (NQC, Max(Score), Dense-QPP, A-Pair-Ratio) with fixed settings, and answer quality is scored against the external NQ gold labels; no parameter is fitted to those labels, so the correlations are not forced by construction. Section 3.3 does assume, without independent validation, that post-retrieval QPP is "a reasonable approximation" of retrieval usefulness, and question difficulty could be a confound. However, this is a construct-validity assumption, not a circular definition: QPP is not defined in terms of final-answer quality, and final-answer quality is not computed from QPP. Self-citations appear for the implementation framework and for some QPP methods, but they are not load-bearing for the validity of the measured correlations. One reporting concern is the blanket claim that all correlations in Table 3 are significant, given small values such as 0.0096; that is a statistical reporting issue, not circularity.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The paper makes no derivation and introduces no fitted coefficients. Its load-bearing input is the assumption that QPP scores of intermediate generated queries proxy retrieval usefulness, plus standard choices of QA metrics and corpus. Configuration constants (top-k, QPP windows) are adopted from prior work and may affect the measured correlations.

free parameters (4)
  • RAG top-k cutoff = 3
    Top-3 documents per retrieval are injected into the reasoning context; chosen to match the original Search-R1 and R1-Searcher configurations and affects both answer quality and QPP values.
  • NQC score window = top-100
    NQC uses retrieval scores of the top 100 documents; a standard setting from prior QPP work, but it influences the NQC estimates reported.
  • A-Pair-Ratio window = top-50, first and last five
    For E5 only, the coherence ratio is computed over the first five and last five of the top-50 retrieved documents, following the cited QPP method; this choice determines the A-Pair-Ratio values used in correlations.
  • Dense-QPP top-k = 3
    The hypercube volume is computed over the top-3 document embeddings, matching the RAG cutoff; this choice directly determines the Dense-QPP values used in correlations.
assumptions (3)
  • domain assumption Post-retrieval QPP estimates approximate the usefulness or relevance of retrieved documents for the downstream answer.
    Introduced in Section 3.3 as a 'reasonable approximation' without ground truth relevance judgments for generated queries; all RQ-2 and RQ-3 interpretations rely on this.
  • domain assumption F1 and Exact Match against NQ gold answers capture answer quality.
    Standard QA metrics, but they are short-answer string metrics and can misjudge semantically correct answers; no human evaluation is performed (Section 4.2).
  • domain assumption Agentic RAG behavior on NQ with a 2018 Wikipedia snapshot is representative of agentic RAG behavior generally.
    The authors limit the study to NQ and note in Section 6 that R1-Searcher was not trained on NQ and that multi-hop datasets like HotpotQA may behave differently.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Am I on the Right Track? What Can Predicted Query Performance Tell Us about the Search Behaviour of Agentic RAG." pith.science (2026). https://pith.science/paper/SSF4S44B

@misc{pith2026250710411,
  author       = {Pith},
  title        = {Pith review of: Am I on the Right Track? What Can Predicted Query Performance Tell Us about the Search Behaviour of Agentic RAG},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SSF4S44B}},
  note         = {Machine review of arXiv:2507.10411}
}
read the original abstract

Agentic Retrieval-Augmented Generation (RAG) is a new paradigm where the reasoning model decides when to invoke a retriever (as a "tool") when answering a question. This paradigm, exemplified by recent research works such as Search-R1, enables the model to decide when to search and obtain external information. However, the queries generated by such Agentic RAG models and the role of the retriever in obtaining high-quality answers remain understudied. To this end, this initial study examines the applicability of query performance prediction (QPP) within the recent Agentic RAG models Search-R1 and R1-Searcher. We find that applying effective retrievers can achieve higher answer quality within a shorter reasoning process. Moreover, the QPP estimates of the generated queries, used as an approximation of their retrieval quality, are positively correlated with the quality of the final answer. Ultimately, our work is a step towards adaptive retrieval within Agentic RAG, where QPP is used to inform the model if the retrieved results are likely to be useful.

Figures

Figures reproduced from arXiv: 2507.10411 by the authors.

Figure 1
Figure 1. A conceptual overview of a search-enhanced reasoner based [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Example prompt for Search-R1 and its output over two reasoning-retrieval iterations for a query from the Natural Questions dataset. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Distribution of the reasoning lengths (number of iterations) across 3610 NQ dataset questions for (a) Search-R1 and (b) R1-Searcher [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Average QPP estimates for the generated queries in each [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Average QPP estimates of generated queries in each [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The output from Search-R1 for the negative example “chris cornell she’s going to change the world”. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: The output from Search-R1 for the positive example “when was the last time villanova won the ncaa basketball championship”. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 26 canonical work pages

  1. [8]

    Suchana Datta, Debasis Ganguly, Mandar Mitra, and Derek Greene. 2022. A Relative Information Gain-based Query Performance Prediction Framework with Generated Query Variants. ACM Trans. Inf. Syst. 41, 2, Article 38 (dec 2022), 31 pages. https://doi.org/10.1145/3545112

    work page doi:10.1145/3545112 2022

  2. [1]

    Marwah Alaofi, Negar Arabzadeh, Charles L. A. Clarke, and Mark Sanderson

    work page

  3. [2]

    Negar Arabzadeh, Amin Bigdeli, Morteza Zihayat, and Ebrahim Bagheri. 2021. Query Performance Prediction Through Retrieval Coherency. In Advances in Information Retrieval: 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28 – April 1, 2021, Proceedings, Part II . Springer-Verlag, Berlin, Heidelberg, 193–200. https://doi.org/10.1007/...

    work page doi:10.1007/978-3-030-72240-1_15 2021

  4. [4]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    work page 2020

  5. [5]

    Anthony Chen, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019. Eval- uating Question Answering Evaluation. In Proceedings of the 2nd Workshop on Machine Reading for Question Answering . Association for Computational Linguis- tics, Hong Kong, China, 119–124. https://aclanthology.org/D19-5817/

    work page 2019

  6. [6]

    Bruce Croft

    Steve Cronen-Townsend, Yun Zhou, and W. Bruce Croft. 2002. Predicting query performance (SIGIR ’02). Association for Computing Machinery, New York, NY, USA, 299–306. https://doi.org/10.1145/564376.564429

    arXiv 2002

  7. [7]

    Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, and Fabrizio Silvestri. 2024. The Power of Noise: Redefining Retrieval for RAG Systems. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (Washington DC, USA) (SIGIR ...

    arXiv 2024

  8. [9]

    Suchana Datta, Sean MacAvaney, Debasis Ganguly, and Derek Greene. 2022. A ’Pointwise-Query, Listwise-Document’ based Query Performance Prediction Approach. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (Madrid, Spain) (SIGIR ’22). Association for Computing Machinery, New York, NY, USA, 2...

    arXiv 2022

Show all 53 references
  1. [10]

    Abhimanyu Dubey et al . 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI] https://arxiv.org/abs/2407.21783

    2024 arXiv

  2. [11]

    Nouha Dziri, Sivan Milton, Mo Yu, Osmar Zaiane, and Siva Reddy. 2022. On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguisti...

    2022

  3. [12]

    Guglielmo Faggioli, Nicola Ferro, Cristina Ioana Muntean, Raffaele Perego, and Nicola Tonellotto. 2023. A Geometric Framework for Query Performance Pre- diction in Conversational Search. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development ...

    2023

  4. [13]

    Jinyuan Fang, Zaiqiao Meng, and Craig Macdonald. 2025. KiRAG: Knowledge- Driven Iterative Retriever for Enhancing Retrieval-Augmented Generation.arXiv preprint arXiv:2502.18397 (2025)

    2025 arXiv

  5. [14]

    Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. 2024. Detecting hallucinations in large language models using semantic entropy. Nature (London) 630, 8017 (2024), 625–630. Am I on the Right Track? IR-RAG Workshop at SIGIR ’25, July 17, 2025, Padua, Italy

    2024

  6. [15]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2024. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv:2312.10997 [cs.CL] https://arxiv.org/abs/2312.10997

    2024 arXiv

  7. [16]

    Claudia Hauff, Djoerd Hiemstra, and Franciska de Jong. 2008. A survey of pre- retrieval query performance predictors. InProceedings of the 17th ACM Conference on Information and Knowledge Management (Napa Valley, California, USA)(CIKM ’08). 1419–1420. https://doi.org/10.1145/1...

    2008

  8. [17]

    Bolei He, Nuo Chen, Xinran He, Lingyong Yan, Zhenkai Wei, Jinchang Luo, and Zhen-Hua Ling. 2024. Retrieving, Rethinking and Revising: The Chain-of- Verification Can Improve Retrieval Augmented Generation. In Findings of the Association for Computational Linguistics: EMNLP 2024...

    2024

  9. [18]

    Ben He and Iadh Ounis. 2004. Inferring Query Performance Using Pre-retrieval Predictors. In String Processing and Information Retrieval . 43–54

    2004

  10. [19]

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reason- ing Steps. In Proceedings of the 28th International Conference on Computational Linguistics. 6609–6625. https://aclanthology.org/2...

    2020

  11. [20]

    Gautier Izacard and Edouard Grave. 2021. Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume . Association for Comput...

    2021

  12. [21]

    Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave

  13. [22]

    Zhengbao Jiang, Frank Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi- Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Active Retrieval Augmented Generation. In Proceedings of the 2023 Conference on Empirical Meth- ods in Natural Language Processing . Association for...

    2023 doi

  14. [23]

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. arXiv:2503.09516 [cs.CL] https://arxiv.org/abs/2503.09516

    2025 arXiv

  15. [24]

    Ashutosh Joshi, Sheikh Muhammad Sarwar, Samarth Varshney, Sreyashi Nag, Shrivats Agrawal, and Juhi Naik. 2024. REAPER: Reasoning based Retrieval Planning for Complex RAG Systems. InProceedings of the 33rd ACM International Conference on Information and Knowledge Management (Bo...

    2024

  16. [25]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav...

    2019

  17. [26]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. InProceedings o...

    2020 arXiv

  18. [27]

    Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. 2025. Search-o1: Agentic Search-Enhanced Large Reasoning Models. arXiv:2501.05366 [cs.AI] https://arxiv.org/abs/2501.05366

    2025 arXiv

  19. [28]

    Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics 12 (2024), 157–173. https://doi.org/10.11...

    2024 doi

  20. [29]

    Craig Macdonald, Jinyuan Fang, Andrew Parry, and Zaiqiao Meng. 2025. Con- structing and Evaluating Declarative RAG Pipelines in PyTerrier. InProceedings of SIGIR 2025

    2025

  21. [30]

    Craig Macdonald, Nicola Tonellotto, Sean MacAvaney, and Iadh Ounis. 2021. PyTerrier: Declarative Experimentation in Python from BM25 to Dense Retrieval. In Proceedings of the 30th ACM International Conference on Information & Knowl- edge Management (Virtual Event, Queensland, ...

    2021

  22. [31]

    Chuan Meng, Negar Arabzadeh, Mohammad Aliannejadi, and Maarten de Rijke

  23. [32]

    Rodrigo Frassetto Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. 2020. Document Ranking with a Pretrained Sequence-to-Sequence Model. In Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020 (Findings of ACL, Vol. EMNL...

    2020

  24. [33]

    Md Rizwan Parvez, Wasi Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai- Wei Chang. 2021. Retrieval Augmented Code Generation and Summarization. In Findings of the Association for Computational Linguistics: EMNLP 2021. Association for Computational Linguistics, Punta Cana, Do...

    2021

  25. [34]

    In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (Taipei, Taiwan) (SIGIR ’23)

    Query Performance Prediction: From Ad-hoc to Conversational Search. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (Taipei, Taiwan) (SIGIR ’23). Association for Computing Machinery, New York, NY, USA, 2583–259...

  26. [35]

    Alireza Salemi and Hamed Zamani. 2024. Evaluating Retrieval Quality in Retrieval-Augmented Generation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (Wash- ington DC, USA) (SIGIR ’24). Association for Computi...

    2024

  27. [36]

    Anna Shtok, Oren Kurland, and David Carmel. 2010. Using statistical decision theory and relevance models for query-performance prediction. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval (Geneva, Switzerland) (...

    2010

  28. [37]

    Nirmal Roy, Leonardo F. R. Ribeiro, Rexhina Blloshmi, and Kevin Small. 2024. Learning When to Retrieve, What to Rewrite, and How to Respond in Conversa- tional QA. In Findings of the Association for Computational Linguistics: EMNLP

    2024

  29. [38]

    https://aclanthology.org/2024.findings-emnlp.622/

    Association for Computational Linguistics, Miami, Florida, USA, 10604– 10625. https://aclanthology.org/2024.findings-emnlp.622/

    2024

  30. [39]

    Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. 2025. R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning. arXiv:2503.05592 [cs.AI] https: //arxiv.org/abs/2503.05592

    2025 arXiv

  31. [40]

    Yongquan Tao and Shengli Wu. 2014. Query Performance Prediction By Consid- ering Score Magnitude and Variance Together. InProceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Manage- ment (Shanghai, China) (CIKM ’14). Association for ...

    2014

  32. [41]

    Anna Shtok, Oren Kurland, David Carmel, Fiana Raiber, and Gad Markovits. 2012. Predicting Query Performance by Query-Drift Estimation. ACM Trans. Inf. Syst. 30, 2, Article 11 (May 2012), 35 pages. https://doi.org/10.1145/2180868.2180873

    2012

  33. [42]

    Aditi Singh, Abul Ehtesham, Saket Kumar, and Tala Talaei Khoei. 2025. Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG. arXiv:2501.09136 [cs.AI] https://arxiv.org/abs/2501.09136

    2025 arXiv

  34. [43]

    Neeraj Varshney, Man Luo, and Chitta Baral. 2022. Can Open-Domain QA Reader Utilize External Knowledge Efficiently like Humans? arXiv:2211.12707 [cs.CL] https://arxiv.org/abs/2211.12707

    2022 arXiv

  35. [44]

    Maria Vlachou. 2025. Predicting Retrieval Failures in Conversational Recommen- dation Systems. Ph.D. thesis. University of Glasgow, Glasgow, UK

    2025

  36. [45]

    Fangzheng Tian, Debasis Ganguly, and Craig Macdonald. 2025. Is Relevance Propagated from Retriever to Generator in RAG?. In Advances in Information Retrieval: 47th European Conference on Information Retrieval, ECIR 2025, Lucca, Italy (Lucca, Italy). 18 pages

    2025

  37. [46]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal

  38. [47]

    In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge- Intensive Multi-Step Questions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 10014–10037

  39. [48]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2025 arXiv

  40. [49]

    chris cornell she’s going to change the world

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language...

    2018

  41. [50]

    Maria Vlachou and Craig Macdonald. 2024. Coherence-based Query Performance Measures for Dense Retrieval. In Proceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval (Washington DC, USA) (ICTIR ’24). Association for Computing Machinery, New ...

    2024

  42. [51]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533 (2022)

    2022 arXiv

  43. [52]

    Zihao Wang, Anji Liu, Haowei Lin, Jiaqi Li, Xiaojian Ma, and Yitao Liang. 2024. RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long- Horizon Generation. arXiv:arXiv:2403.05313

    2024 arXiv

  44. [2023]

    Atlas: few-shot learning with retrieval augmented language models. J. Mach. Learn. Res. 24, 1, Article 251 (Jan. 2023), 43 pages

    2023

  45. [2024]

    arXiv:2404.08137 [cs.IR] https://arxiv.org/abs/2404.08137

    Generative Information Retrieval Evaluation. arXiv:2404.08137 [cs.IR] https://arxiv.org/abs/2404.08137

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.