Pith. sign in

REVIEW 4 major objections 5 minor 41 references

LeTS: Learning to Think-and-Search via Process-and-Outcome Reward Hybridization

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Rewarding each retrieval step, not just the final answer, improves retrieval-augmented reasoning and cuts search count by about 31%.

desk verdict LeTS is a plausible, well-ablated recipe for adding process rewards to RL-trained RAG, but the 'consistently outperforms' headline is undermined by one reversed result and a missing error-bar analysis. read the letter →

arxiv 2505.17447 v1 pith:W4XKIS4G submitted 2025-05-23 cs.CL

Qi Zhang , Shouqing Yang , Lirong Gao , Hao Chen , Xiaomeng Hu , Jinglei Chen , Jiexiang Wang , Sheng Guo
show 3 more authors
Bo Zheng Haobo Wang Junbo Zhao
This is my paper
classification cs.CL
keywords reinforcementlearningretrieval-augmentedgenerationprocess-levelrewardoutcome-leveladvantagerescalingGRPOmulti-hopquestionansweringknowledgeredundancy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that reinforcement learning for retrieval-augmented question answering is poorly served by outcome-only rewards, because the model never learns which intermediate searches were useful. It proposes LeTS, a training scheme based on group-relative policy optimization (GRPO) that adds two rule-based process rewards: one penalizes re-retrieving overlapping document sets within a rollout, and the other gives partial credit to underperforming rollouts whose steps align with steps of the group's best rollout. Those step scores are normalized and used to rescale each rollout's outcome advantage, giving sharper per-step credit assignment. Across four multi-hop and three single-hop benchmarks, LeTS outperforms the outcome-only RL baseline on the same backbone and reduces average search time by roughly 30%.

What carries the argument

The load-bearing mechanism is a stepwise advantage-rescaling identity $A_{i,j}=(1+\mathrm{sgn}(A_i^o)\lambda\hat{r}^p_{i,j})A_i^o$, which multiplies the group-normalized outcome advantage of each rollout by a factor derived from that step's normalized process reward. The process rewards come from two rule-based modules: the knowledge redundancy reward uses Jaccard overlap between the current retrieved document set and all previous sets in the same rollout, and the knowledge match reward uses the Kuhn-Munkres (Hungarian) algorithm, a maximum-weight bipartite matching routine, to align steps of underperforming rollouts to steps of the best-performing, exact-match rollout and reward their document-level overlap. Because $\hat{r}^p$ is normalized within each rollout, the rescaling only changes the relative weight of steps and never flips the sign of the outcome advantage, preserving trajectory-level signal while making credit assignment finer-grained. The final reasoning step and malformed rollouts are excluded from rescaling to keep gradient propagation stable.

What would settle it

Retrain LeTS with the two process rewards replaced by random scores matched to the same mean and variance; if EM, token count, and search count stay near the reported values, the specific content of the Jaccard and Hungarian signals is not what drives the improvement. A sharper test comes from the paper's Appendix D, where the same F1 outcome reward collapses a 3B base model to zero searches: running LeTS on that model and checking whether search time remains nonzero while reward rises would directly test whether the hybrid reward prevents outcome-reward hacking.

Watch

Extended reading notes

Core claim

LeTS's central claim is that hybridizing process-level and outcome-level rewards produces more accurate and cheaper think-and-search behavior than outcome-only RL. Concretely, the paper categorizes each group of GRPO rollouts into outperforming and underperforming, then assigns every step a process reward: a knowledge redundancy term $r^p_{i,j}=1-\max_t |d_{i,j}\cap d_{i,t}|/|d_{i,j}\cup d_{i,t}|$ that penalizes retrieving documents already covered by earlier steps, and a knowledge match reward computed by Hungarian-aligning each underperforming rollout's steps to an outperforming rollout's steps and taking the Jaccard similarity of retrieved document sets. These step rewards are z-scored within each rollout and fused into the outcome-level advantage through $A_{i,j}=(1+\mathrm{sgn}(A_i^o)\lambda\hat{r}^p_{i,j})A_i^o$. The paper reports consistent EM and LLM-as-judge gains over the outcome-only baseline across Qwen-7B, 7B-Instruct, and 3B-Instruct backbones, with average search count falling by 30.85% and generated tokens by 11.15%.

Load-bearing premise

The load-bearing premise is that how much a step's retrieved documents overlap with earlier documents or with successful steps' documents is a reliable sign of whether that search step helped; if overlap does not track step quality, the extra rewards would just teach the model to search less, not search better.

Editorial extensions

If this is right

  • On the four multi-hop benchmarks (HotpotQA, 2WikiMultiHopQA, MusiQue, Bamboogle), LeTS beats the outcome-only RL baseline on average EM and LLM-as-judge accuracy for Qwen-7B, Qwen-7B-Instruct, and Qwen-3B-Instruct, with the largest absolute gain on Bamboogle.
  • The same trained models generalize to single-hop benchmarks (NQ, PopQA, TriviaQA), so the learned search policy is not overfit to multi-hop query structures.
  • Average search count drops by 30.85% and generated tokens by 11.15%, so the process rewards convert into inference-time savings, not just higher scores.
  • Ablations show each process reward reduces search time on its own, and combining them raises EM from 22.18 to 23.21 on MusiQue, indicating the two signals are complementary.
  • The framework works on base and instruct-tuned backbones and on a 3B model, suggesting the hybrid-reward recipe transfers across model sizes and training stages.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: the same advantage-rescaling trick could be lifted out of retrieval and applied to other sparse-outcome agentic RL settings, such as web navigation, code execution, or tool use, whenever a cheap rule-based proxy for step quality exists; the paper's framing points toward this but does not test it.
  • My inference: the group-level knowledge match reward is a form of implicit dense credit assignment that only engages when at least one rollout in the group reaches an exact match; dropping the exact-match requirement and aligning to the highest-reward rollout instead could extend the method to harder tasks where full success is rare.
  • My inference: the paper's stated limitation, that it was not validated on larger language models, leaves open whether the reported gains survive scale; running LeTS on a 30B-plus instruct model would be the direct check.
  • My inference: the 30.85% search-count reduction could reflect earlier stopping rather than better retrieval focus; measuring per-step recall before and after training would separate 'search less' from 'search better', a test the paper does not report.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes LeTS, a reinforcement-learning framework for retrieval-augmented generation that extends GRPO with two rule-based process-level rewards: a knowledge redundancy reward based on Jaccard overlap of retrieved document sets within a rollout (Eq. 4) and a knowledge match reward that aligns underperforming rollouts to exact-match rollouts via maximum-weight bipartite matching (Eqs. 5-6). These process rewards are hybridized with the outcome reward by rescaling the outcome advantage per step with a factor lambda (Eq. 7). The method is evaluated on multi-hop benchmarks (HotpotQA, 2WikiMultiHopQA, MusiQue, Bamboogle) and single-hop benchmarks (NQ, PopQA, TriviaQA) using Qwen2.5-7B, Qwen2.5-7B-Instruct, and Qwen2.5-3B-Instruct backbones, reporting modest EM/LJ improvements over ReSearch as well as reductions in search time and generated tokens.

Significance. If the central claim holds, LeTS provides a simple, annotation-free recipe for combining process-level and outcome-level rewards in RL-trained RAG, with the practical benefit of roughly 30% fewer search steps. The strengths of the paper are its rule-based process reward design, the detailed pseudocode in Appendix E, the transparent training setup inherited from ReSearch, and the use of publicly available benchmarks. However, the process reward is validated only indirectly through final accuracy and efficiency metrics, and no control isolates whether the improvements come from better step-level credit assignment or merely from pressure toward shorter, less-overlapping search trajectories. The reported gains are small and accompanied by per-dataset exceptions to the 'consistently outperforms' claim, so the evidence is not yet strong enough for the paper's conclusions.

major comments (4)
  1. [Section 3.3, Eq. (4)] The knowledge redundancy reward is presented as a measure of step quality, but any step whose retrieved document set is disjoint from all earlier sets receives the maximum reward regardless of whether the documents are relevant or useful. The reductions in search time (Table 4) and in-rollout redundancy frequency (Table 5) are exactly what a policy that emits fewer and less-overlapping queries would produce, so these measurements do not by themselves demonstrate improved credit assignment. To support the central claim, the authors should add a control that isolates the step-quality content of the reward, such as random per-step rewards or a pure search-count penalty with the same efficiency pressure, and show that LeTS outperforms that control.
  2. [Section 3.4, Eq. (7)] The normalized process reward divides by the standard deviation of the per-step process rewards within a rollout. This quantity is undefined when all process rewards are equal, which occurs for single-step rollouts and for any trajectory in which every Jaccard overlap is zero. No fallback is specified in Eq. (7), Algorithm 1, or Appendix B. Please state the convention (for example, set the normalized reward to zero in the degenerate case) and verify that the reported results are not sensitive to this choice.
  3. [Section 4.2, Section 4.3, Tables 1 and 2] The claim that LeTS 'consistently outperforms all baselines' is contradicted by the paper's own tables: in Table 1, LeTS-Qwen-7B-Instruct is below ReSearch-Qwen-7B-Instruct on HotpotQA EM (43.16 vs. 43.52), and in Table 2, LeTS is below ReSearch on PopQA EM (43.91 vs. 44.58). Moreover, the reported average gains of 1.67–2.61 EM points are given without error bars or significance tests. Please correct the overclaim and add variance or significance information for the main comparisons.
  4. [Section 3.3, Eqs. (5)-(6)] The knowledge match reward defines the correctness of an underperforming step by its document-level overlap with the group's exact-match rollout, and when multiple exact-match rollouts exist it selects the one with the highest total process reward. Systematic errors in the winning trajectory are therefore propagated to underperforming rollouts as if they were correct. Because this is the only process signal applied to underperforming rollouts, a concrete test is needed, such as comparing KM against a variant that aligns to a randomly chosen outperforming rollout or to an oracle-relevant document set, to show that KM contributes genuine step-quality supervision rather than simply reducing the variance of the search behavior.
minor comments (5)
  1. [Section 4.1] The sentence 'please refer to A' is an incomplete cross-reference; it should point to Appendix A and briefly describe the evaluation benchmarks and metrics.
  2. [Section 5.4, Table 4] Table 4 shows that on MusiQue the average number of generated tokens increases by 2.36% relative to ReSearch, so the statement that LeTS 'reduces both' tokens and search times should be qualified as applying on average rather than on every dataset.
  3. [Appendix D, Figure 6] Figure 6 plots 'Training Reward' and 'Average Search Time' on the same axes without a legend or clear axis assignment; please add a legend and separate y-axes.
  4. [Appendix F, Table 10] The ReSearch answer in the case study contains a typo ('M arch4, 1801'), and the discussion of which president set the two-term precedent should be factually consistent, since the example is used to illustrate the qualitative benefit of LeTS.
  5. [Limitations] The Limitations section does not discuss the unvalidated assumption that document-overlap statistics approximate step quality; given that this assumption underlies the method, the limitations should acknowledge it and mention the missing control experiments identified above.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported LeTS gains are empirical results of RL training on independent public benchmarks, not reductions of the method's inputs by construction.

full rationale

The paper's central claim is that LeTS improves performance and efficiency over outcome-supervised RL baselines. The derivation chain is an empirical RL pipeline: rule-based process rewards (Eq. 4-6) are computed from retrieved document sets and from exact-match rollout alignment, and the hybridized advantage (Eq. 7) is used to train the policy. None of these equations is fitted to the reported evaluation metrics, and none of the reported EM/LJ gains is a mathematical consequence of the reward definitions. The KM reward does derive its notion of step correctness from exact-match winning rollouts, but that is an algorithmic design choice rather than a circular prediction: the paper does not claim that the process reward is annotation-free evidence of step quality independent of outcome, and the held-out benchmark results are not forced by this construction. The reductions in search time and tokens are consistent with the knowledge-redundancy reward's objective, and the paper presents them as measured training effects, not as quantities predicted from the reward. The hyperparameter lambda is tuned on one backbone and then fixed, which is standard practice and does not make the benchmark results fitted values. There are no load-bearing self-citations, no imported uniqueness theorems, and no renaming of known results presented as a derivation. The undefined normalization when all per-step process rewards are equal is a robustness gap, not a circularity. Overall, the evaluation is self-contained against external benchmarks, so the paper warrants a low circularity score.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on unverified modeling assumptions about document overlap as a proxy for step quality, on the safety of the F1 plus format reward, and on gpt-4o-mini as a judge. No new entities are introduced. The only tuned scalar is lambda.

free parameters (2)
  • Rescaling factor lambda = 0.1
    Tuned over {0.05, 0.1, 0.2} on Qwen2.5-7B-Instruct and reused for all experiments; controls the strength of process reward rescaling in Eq. 7.
  • Outcome reward mixing coefficients = 0.9 (F1) + 0.1 (format bonus)
    Hand-chosen constants in Eq. 3, with no sensitivity analysis reported.
assumptions (4)
  • domain assumption Jaccard overlap between retrieved document sets is a reliable measure of redundant knowledge.
    Used in Eq. 4 and Eq. 5 without empirical validation; lexical overlap may miss semantic redundancy or penalize useful repeated context.
  • domain assumption Exact-match-answer rollouts contain search steps that are reliable enough to supervise underperforming rollouts.
    Used in Eq. 5 and Eq. 6 to define the knowledge match reward; a correct final answer does not guarantee each intermediate search is optimal.
  • domain assumption The F1-based outcome reward with a format check provides sufficient signal to train think-and-search behavior without reward hacking.
    Defined in Eq. 3; contradicted by the paper's own Appendix D, where Qwen2.5-3B-Base collapses to zero search time.
  • domain assumption gpt-4o-mini LLM-as-a-Judge is a valid evaluation metric.
    Used for LJ scores in Tables 1 to 3; no human agreement or calibration is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LeTS: Learning to Think-and-Search via Process-and-Outcome Reward Hybridization." pith.science (2026). https://pith.science/paper/W4XKIS4G

@misc{pith2026250517447,
  author       = {Pith},
  title        = {Pith review of: LeTS: Learning to Think-and-Search via Process-and-Outcome Reward Hybridization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W4XKIS4G}},
  note         = {Machine review of arXiv:2505.17447}
}
read the original abstract

Large language models (LLMs) have demonstrated impressive capabilities in reasoning with the emergence of reasoning models like OpenAI-o1 and DeepSeek-R1. Recent research focuses on integrating reasoning capabilities into the realm of retrieval-augmented generation (RAG) via outcome-supervised reinforcement learning (RL) approaches, while the correctness of intermediate think-and-search steps is usually neglected. To address this issue, we design a process-level reward module to mitigate the unawareness of intermediate reasoning steps in outcome-level supervision without additional annotation. Grounded on this, we propose Learning to Think-and-Search (LeTS), a novel framework that hybridizes stepwise process reward and outcome-based reward to current RL methods for RAG. Extensive experiments demonstrate the generalization and inference efficiency of LeTS across various RAG benchmarks. In addition, these results reveal the potential of process- and outcome-level reward hybridization in boosting LLMs' reasoning ability via RL under other scenarios. The code will be released soon.

Figures

Figures reproduced from arXiv: 2505.17447 by the authors.

Figure 1
Figure 1. Illustrations of redundant searching issue and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Collaborative comparison of performance and [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The core framework of LeTS extends conventional outcome-supervised GRPO by introducing a fine￾grained categorization of rollouts within each group. Specifically, generated rollouts are classified as either outperforming or underperforming, based on their relative returns. Separate reward functions are then applied to each category to compute process-level rewards. Finally, the rollout-level advantages are rescaled a… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Training reward during training for the four [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Knowledge redundancy frequency of the four [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: The variance of reward and average search [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 1 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations

    2023

  4. [4]

    Ning Bian, Hongyu Lin, Peilin Liu, Yaojie Lu, Chunkang Zhang, Ben He, Xianpei Han, and Le Sun. 2024. https://doi.org/10.1109/TCSS.2024.3476030 Influence of external information on large language models mirrors social cognitive patterns . IEEE Transactions on Computational Social Systems, pages 1--17

    arXiv 2024

  5. [5]

    Chi-Min Chan, Chunpu Xu, Ruibin Yuan, Hongyin Luo, Wei Xue, Yike Guo, and Jie Fu. 2024. Rq-rag: Learning to refine queries for retrieval augmented generation. arXiv preprint arXiv:2404.00610

    arXiv 2024

  6. [6]

    Hung-Ting Chen, Fangyuan Xu, Shane A Arora, and Eunsol Choi. 2023. Understanding retrieval augmentation for long-form question answering. arXiv preprint arXiv:2310.12150

    arXiv 2023

  7. [7]

    Pan, Wen Zhang, Huajun Chen, Fan Yang, Zenan Zhou, and Weipeng Chen

    Mingyang Chen, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z. Pan, Wen Zhang, Huajun Chen, Fan Yang, Zenan Zhou, and Weipeng Chen. 2025. https://arxiv.org/abs/2503.19470 Research: Learning to reason with search for llms via reinforcement learning . Preprint, arXiv:2503.19470

    arXiv 2025

  8. [8]

    Marco Cuturi. 2013. https://proceedings.neurips.cc/paper_files/paper/2013/file/af21d0c97db2e27e13572cbf59eb343d-Paper.pdf Sinkhorn distances: Lightspeed computation of optimal transport . In Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc

    2013

Show all 41 references
  1. [9]

    Xuefeng Du, Chaowei Xiao, and Sharon Li. 2024. Haloscope: Harnessing unlabeled llm generations for hallucination detection. Advances in Neural Information Processing Systems, 37:102948--102972

    2024

  2. [10]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997, 2:1

    2023 arXiv

  3. [11]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  4. [12]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  5. [13]

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International conference on machine learning, pages 3929--3938. PMLR

    2020

  6. [14]

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. https://doi.org/10.18653/v1/2020.coling-main.580 Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps . In Proceedings of the 28th International Conference on Computational Li...

    2020 doi

  7. [15]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and 1 others. 2025. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions ...

    2025

  8. [16]

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720

    2024 arXiv

  9. [17]

    Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024. A survey on large language models for code generation. arXiv preprint arXiv:2406.00515

    2024 arXiv

  10. [18]

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516

    2025 arXiv

  11. [19]

    Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. https://doi.org/10.18653/v1/P17-1147 T rivia QA : A large scale distantly supervised challenge dataset for reading comprehension . In Proceedings of the 55th Annual Meeting of the Association for Computational...

    2017 doi

  12. [20]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav...

    2019 doi

  13. [21]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \"u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \"a schel, and 1 others. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural informati...

    2020

  14. [22]

    Yanming Liu, Xinyue Peng, Xuhong Zhang, Weihao Liu, Jianwei Yin, Jiannan Cao, and Tianyu Du. 2024. Ra-isf: Learning to answer and understand from retrieval augmentation via iterative self-feedback. arXiv preprint arXiv:2403.06840

    2024 arXiv

  15. [23]

    Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. 2023. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint arXiv:2308.09583

    2023 arXiv

  16. [24]

    Ruotian Ma, Peisong Wang, Cheng Liu, Xingyan Liu, Jiaqi Chen, Bang Zhang, Xin Zhou, Nan Du, and Jia Li. 2025. S ^2 r: Teaching llms to self-verify and self-correct via reinforcement learning. arXiv preprint arXiv:2502.12853

    2025 arXiv

  17. [25]

    Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, and Nan Duan. 2023. Query rewriting in retrieval-augmented large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5303--5315

    2023

  18. [26]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Hannaneh Hajishirzi, and Daniel Khashabi. 2022. When not to trust language models: Investigating effectiveness and limitations of parametric and non-parametric memories. arXiv preprint

    2022

  19. [27]

    Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah Smith, and Mike Lewis. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.378 Measuring and narrowing the compositionality gap in language models . In Findings of the Association for Computational Linguistics: EMNLP 20...

    2023 doi

  20. [28]

    Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. 2023. Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy. arXiv preprint arXiv:2305.15294

    2023 arXiv

  21. [29]

    Li, and Daya Guo Y

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y.K. Li, and Daya Guo Y. Wu. 2024. https://arxiv.org/abs/2402.03300 Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    2024 arXiv

  22. [30]

    Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval augmentation reduces hallucination in conversation. arXiv preprint arXiv:2104.07567

    2021 arXiv

  23. [31]

    Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Ji-Rong Wen, Yang Lu, and Xu Miu. 2025. https://github.com/RUCAIBox/R1-searcher R1-searcher: Incentivizing the search capability in llms via reinforcement learning

    2025

  24. [32]

    Hexiang Tan, Fei Sun, Wanli Yang, Yuanzhuo Wang, Qi Cao, and Xueqi Cheng. 2024. https://doi.org/10.18653/v1/2024.acl-long.337 Blinded by generated contexts: How language models merge generated and retrieved contexts when knowledge conflicts? In Proceedings of the 62nd Annual M...

    2024 doi

  25. [33]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022 a . Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. arXiv preprint arXiv:2212.10509

    2022 arXiv

  26. [34]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022 b . https://doi.org/10.1162/tacl_a_00475 ♫ M u S i Q ue: Multihop questions via single-hop question composition . Transactions of the Association for Computational Linguistics, 10:539--554

    2022 doi

  27. [35]

    Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. https://openreview.net/forum?id=1PL1NIMMrw Self-consistency improves chain of thought reasoning in language models . In The Eleventh International Conferenc...

    2023

  28. [36]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

    2024 arXiv

  29. [37]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. https://doi.org/10.18653/v1/D18-1259 H otpot QA : A dataset for diverse, explainable multi-hop question answering . In Proceedings of the 2018 Conference...

    2018 doi

  30. [38]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models, 2023. URL https://arxiv. org/abs/2210.03629

    2023 arXiv

  31. [39]

    Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant. 2024. https://openreview.net/forum?id=ZS4m74kZpH Making retrieval-augmented language models robust to irrelevant context . In The Twelfth International Conference on Learning Representations

    2024

  32. [40]

    Murong Yue. 2025. A survey of large language model agents for question answering. arXiv preprint arXiv:2503.19213

    2025 arXiv

  33. [41]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, and 1 others. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223, 1(2)

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.