Pith. sign in

REVIEW 5 major objections 4 minor 2 cited by

Curriculum Guided Reinforcement Learning for Efficient Multi Hop Retrieval Augmented Generation

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A two-stage, time-varying reward curriculum makes multi-hop retrieval agents more accurate and more efficient.

desk verdict The method is a plausible new combination, but the paper's core claim of consistent gains is contradicted by its own Table 3 on Bamboogle, and the HotpotQA numbers are partly fitted. read the letter →

arxiv 2505.17391 v1 pith:HBE2GPYF submitted 2025-05-23 cs.CL

Yuelyu Ji , Rui Meng , Zhuochun Li , Daqing He This is my paper
classification cs.CL
keywords InformationRetrievalReinforcementLearningQueryRewritingMulti-HopQuestionAnsweringRetrieval-AugmentedGenerationCurriculumDirectPreferenceOptimizationRewardScheduling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a query-rewriting agent for multi-hop retrieval-augmented generation can be trained to be both more accurate and more efficient if its reward signal changes shape over the course of an episode. The proposed framework, EVO-RAG, runs the agent through two curricula: an early Discovery stage that rewards broad exploration, then a Refinement stage that up-weights correctness, redundancy, and step-cost penalties. The agent is trained with direct preference optimization over a seven-head reward model, so it learns when each of four actions—search, backtrack, answer, refuse—is actually useful. Across four multi-hop question-answering benchmarks, the authors report Exact Match gains of up to 4.6 points over strong baselines while cutting average retrieval depth by roughly 15%. If the claim holds, this is a general recipe for making RAG pipelines cheaper and less prone to wandering.

What carries the argument

The load-bearing mechanism is the two-stage curriculum with dynamic reward scheduling: a seven-component step-level reward vector $r_t$, aggregated as $R_t=\sum_i w_i(t) r_i(s_t,a_t)$, where the weights $w_i(t)$ are linearly interpolated inside each episode by the progress ratio $p(t)=t/T_{\max}$. This is the 'gearbox' that lets the agent explore early and refine late. The second piece is the multi-head preference model: a frozen LLM encoder with seven parallel linear heads, one per reward, trained by a pairwise logistic loss that reduces to direct preference optimization; it ranks sibling actions by cumulative weighted reward and supplies the preferences that update the policy. Together they turn abstract goals like 'don't waste retrievals' and 'answer correctly' into per-step pressure on a discrete action space of SEARCH, BACKTRACK, ANSWER, and REFUSE.

What would settle it

Train the same EVO-RAG pipeline on 1,000 queries drawn from 2WikiMultiHopQA or MuSiQue while keeping every reward weight and early-stopping rule fixed, then evaluate on HotpotQA and Bamboogle. If the method is a general curriculum recipe, performance should remain close to the HotpotQA-trained numbers; if it collapses or drops below RAG-Gym on HotpotQA, the reported gains are at least partly artifacts of tuning to one benchmark.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that phase-agnostic, episode-level rewards are the wrong learning signal for multi-hop RAG, and that a time-varying, step-level reward vector fixes the failure modes. The reward vector has seven components: a retrieval bonus for hitting gold-supporting documents, a sub-query overlap penalty for redundant queries, a backtrack penalty, a refusal reward for truthfully abstaining when evidence is insufficient, a step cost, a terminal answer-correctness reward combining EM and F1, and a retrieval-action penalty that grows as the episode advances. During Discovery the weights favor retrieval-oriented terms; during Refinement they shift toward the overlap penalty, step cost, and answer correctness; and inside each episode the weights are linearly interpolated by the progress ratio $p(t)=t/T_{\max}$. Training uses direct preference optimization over a multi-head reward model, with a threshold on reward differences to keep preference pairs stable. The reported result is that this schedule cuts redundant sub-queries and retrieval depth while improving Exact Match and F1 on HotpotQA, 2WikiMultiHopQA, MuSiQue, and Bamboogle, and the qualitative traces show the agent backtracking deliberately, refusing unanswerable questions, and stopping after concise chains instead of hallucinating.

Load-bearing premise

The entire comparison hinges on the assumption that a policy trained on a single 1,000-question HotpotQA sample, with every reward weight hand-tuned and early stopping done on HotpotQA dev, transfers to the other multi-hop benchmarks without further tuning; the paper's own Bamboogle result, where EVO-RAG trails the strongest baseline, is partial counter-evidence.

Editorial extensions

If this is right

  • If EVO-RAG's central claim is correct, a query-rewriting agent trained only on 1,000 HotpotQA questions can carry its gains to other multi-hop benchmarks such as 2WikiMultiHopQA and MuSiQue without retraining.
  • The explicit refusal reward gives the agent a learned 'I don't know' action, so unanswerable questions should be answered with abstention rather than hallucination; the paper's qualitative trace shows this on a labelled unanswerable Hugo Award question.
  • The dynamic scheduler should make retrieval chains shorter: the full reward suite achieves the best EM/F1 while keeping average chain length at 10.4 steps, and the per-episode interpolation cuts steps from 11.5 to 9.8 in the reported comparison.
  • Backtracking and answer correctness are complementary: adding backtrack reward to answer correctness yields a +3.1 EM gain over no reward, and adding the overlap penalty improves efficiency without hurting accuracy.
  • No single reward suffices: single-reward training reaches at best 70.3% internal preference accuracy with the backtrack reward, while the full suite reaches 57.6% EM, so the contribution is the combination plus scheduling, not any one term.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the same three 8-billion-parameter backbones show similar gains, the effect likely lives mostly in the learned decision policy rather than in parametric knowledge; a direct test would be swapping in a much larger or smaller backbone and checking whether the reward schedule still transfers.
  • Editorial inference: the paper's own Bamboogle result, where EVO-RAG scores below the strongest baseline, suggests the cross-benchmark transfer claim has a boundary; an extension would train on a different source benchmark and see whether the HotpotQA-tuned weights remain optimal.
  • Editorial inference: the time-varying scheduler is a manually coded curriculum on reward weights; the natural next step, which the paper lists as future work, is to meta-learn the schedule rather than interpolate it linearly, which would make the method self-tuning.
  • Editorial inference: the seven reward signals are task-generic, so the same recipe could be ported to retrieval-augmented summarization or fact verification rather than only multi-hop question answering.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes EVO-RAG, a reinforcement-learned query-rewriting agent for multi-hop RAG. The agent chooses among SEARCH, BACKTRACK, ANSWER, and REFUSE at each step and is trained with Direct Preference Optimization over preferences derived from a seven-component step-level reward vector whose weights are interpolated within episodes across a two-stage Discovery/Refinement curriculum. Experiments are reported on HotpotQA, 2WikiMultiHopQA, MuSiQue, and Bamboogle, with all policies trained on 1,000 HotpotQA queries and evaluated on official validation sets. The paper claims up to +4.6 Exact Match and roughly 15% reduction in retrieval steps over strong baselines.

Significance. If the empirical claims were fully supported, EVO-RAG would be a useful contribution: the decomposition of process rewards into retrieval, redundancy, efficiency, and correctness signals is well motivated, and the qualitative step-level traces in Table 7 clearly illustrate how the scheduler changes behavior. The two-stage curriculum and time-varying reward interpolation are easy to understand and reproduce from the text. However, the evidence as presented does not support the central claim of consistent superiority: the main table contradicts the text on Bamboogle, the HotpotQA numbers are partly fitted by design, and the headline baseline ReARTeR is absent from the comparison table. The contribution is therefore currently not established.

major comments (5)
  1. [§4.2, Table 3] The statement in §4.2 that "EVO-RAG consistently achieves superior performance, notably improving Exact Match (EM) and F1 scores across all datasets" is contradicted by Table 3: on Bamboogle, all three EVO-RAG variants (45.3–46.0 EM) are 5.2–5.9 points below RAG-Gym (51.2 EM). Since Bamboogle is one of the four benchmarks used to support the general-recipe claim, the claim of consistent superiority is false as stated and must be revised or supported with additional analysis.
  2. [§3.4, §4.1, §7] The HotpotQA evaluation is not an independent test. Section 3.4 states that training stops when HotpotQA dev EM stalls, Section 4.1 reports training on 1,000 HotpotQA queries, and Section 7 concedes that reward parameters were manually tuned primarily on HotpotQA. Consequently the HotpotQA results in Table 3 are partly fitted values rather than out-of-sample measurements, and the cross-dataset results, where Bamboogle shows a clear deficit, are the only transfer evidence.
  3. [§4.2, Table 3] The paper claims in §4.2 to beat ReARTeR by +2.0 EM and +1.8 F1 on HotpotQA and says Table 3 compares against RAG-Gym and ReARTeR, but Table 3 lists neither ReARTeR nor its scores. This missing baseline makes the headline improvement over the stated previous best unverifiable and should be added or the claim removed.
  4. [Table 3, Table 4] All numbers are reported as 3-seed averages without standard deviations or significance tests, and the margins are small in several cases, e.g., +1.7 EM on 2WikiMultiHopQA in Table 3 and the LLaMA-3.1-8B row of Table 4 where time-dynamic scheduling is 1.8 EM worse than two-stage. Without variance estimates, the reported improvements cannot be distinguished from seed noise.
  5. [§3.3, §3.6] The preference pairs used for DPO are constructed from cumulative returns computed with the same manually tuned reward weights, as described in Section 3.3, and those weights were tuned on HotpotQA. Because the preference signal is derived from the tuned weights, the policy improvement on HotpotQA is at least partly a consequence of fitting the reward to the evaluation set; a validation-free weighting scheme or a fully held-out evaluation would be needed to break this circularity.
minor comments (4)
  1. [§3.4, §3.5] The symbol β denotes both the Retrieval Bonus weight in the weight vector of Section 3.5 and the DPO temperature (β = 0.1) in Section 3.4; please rename one to avoid ambiguity.
  2. [Table 5] The column "Eval Accuracy (%)" is defined in the text as the accuracy of the learned preference model, not end-task QA accuracy; the caption should state this explicitly to prevent misreading.
  3. [Table 3] The RAG-Gym row is labeled "ReSearch + PRM" but no retriever or retrieval configuration is specified for either baseline; please state whether the same retriever and document pool are used, since retrieval quality is a major confound in multi-hop RAG comparisons.
  4. [References] References [12], [13], [26], [37], [40], [45], and [47] appear unrelated to the RAG topics they are cited for; please replace them with relevant citations or remove them.

Circularity Check

1 steps flagged · score 4.0 of 10

The HotpotQA headline gain is partly a dev-set fit rather than an out-of-sample prediction; the cross-dataset results remain independent, and the Bamboogle contradiction is a correctness issue, not circularity.

  1. fitted input called prediction [Section 3.2 (Aggregated Reward), Section 3.4 (Policy Optimisation), Section 4.1, Section 7 Limitations]
    "We empirically tuned each 𝑤𝑖 to balance effectiveness and efficiency... Training stops if HotpotQA dev EM stalls for two cycles... reward parameters and scheduling were manually tuned primarily on the HotpotQA dataset."

    The paper's headline 'boosts Exact Match by up to 4.6 points' is anchored to HotpotQA, but that number comes from a configuration whose reward weights and stopping epoch were selected by watching HotpotQA dev EM. The reported metric and the selection criterion are the same quantity, so the HotpotQA EM result is a fitted value on the evaluation set rather than an out-of-sample prediction. This compromises the in-distribution headline claim, though the cross-dataset results are genuinely trained on HotpotQA and evaluated elsewhere, so the method's core derivation is not reduced to a fit.

full rationale

The algorithmic chain (seven-step reward vector to preference pairs to multi-head reward model to DPO policy update to evaluation) is self-contained: no predicted quantity is defined in terms of itself, and using EM/F1 as a terminal reward on training trajectories while reporting EM/F1 on held-out data is standard RL evaluation, not circularity. There is no load-bearing self-citation chain; the process-supervision recipe is attributed to external work (RAG-Gym), and no uniqueness argument is imported from the authors' prior work. The only fitted-input step is the HotpotQA dev tuning described in Sections 3.2, 3.4, and acknowledged in Section 7, which makes the abstract's 'up to 4.6 points' gain partly a selected dev-set number. The cross-dataset results are independent out-of-sample evidence and are not circular. Separately, Section 4.2's claim of 'consistently superior performance ... across all datasets' is contradicted by Table 3 on Bamboogle (EVO-RAG 45.3-46.0 EM vs RAG-Gym 51.2 EM); that is an internal-consistency/correctness problem, so it is noted here but does not raise the circularity score.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The method depends on several hand-picked quantities and domain assumptions. The most consequential is the set of reward weights and scheduler anchors, all tuned on HotpotQA, together with the early-stopping rule on HotpotQA dev. The reliance on gold supporting documents and an LLM verifier for the refusal reward are additional domain assumptions that shape the learned behavior.

free parameters (7)
  • Reward weight schedule (7 components) = Table 1 values (e.g., beta 2.0 to 0.5; eta 0.02 to 0.10; kappa 0.05 to 1.0)
    Manually tuned on HotpotQA to balance exploration and efficiency; central to the method.
  • Preference threshold Delta = 0.3
    Filters preference pairs by absolute reward difference to ensure stable training; chosen by hand.
  • Preference margin delta = 0.2
    Selects positive/negative action pairs per state during preference extraction.
  • DPO temperature beta = 0.1
    Standard DPO hyperparameter, set by hand.
  • Step penalty raw value = -1 (alternatives -0.5, -2 tested)
    Chosen because it balanced EM against step count in preliminary experiments.
  • Maximum episode length Tmax = 20
    Used in p(t)=t/Tmax for weight interpolation; no sensitivity analysis given.
  • Early stopping criterion = HotpotQA dev EM stalls for 2 cycles
    Makes HotpotQA dev part of model selection, a choice that affects the reported numbers.
assumptions (5)
  • domain assumption The action space SEARCH, BACKTRACK, ANSWER, REFUSE is sufficient for multi-hop RAG
    Section 3.2 defines A as the full policy space; if a necessary behavior is missing, the policy cannot express it.
  • domain assumption Gold supporting documents D* are available during training to compute the retrieval bonus
    Section 3.2 defines r_ret using D*; this holds for HotpotQA training data but may not transfer to other datasets.
  • domain assumption The ChatGPT-4o verifier's enough_evidence judgment is a reliable oracle for the refusal reward
    Section 3.2 explains that unanswerable is judged by ChatGPT-4o; errors in this judgment propagate to the refusal signal.
  • domain assumption Reward components are linearly combinable with time-varying weights
    Section 3.2 and Table 1 assume total reward R_t is a weighted sum with no interaction terms.
  • domain assumption Training on 1,000 HotpotQA questions transfers to other multi-hop datasets
    Section 4.1 trains only on HotpotQA; the Bamboogle regression vs RAG-Gym challenges this assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Curriculum Guided Reinforcement Learning for Efficient Multi Hop Retrieval Augmented Generation." pith.science (2026). https://pith.science/paper/HBE2GPYF

@misc{pith2026250517391,
  author       = {Pith},
  title        = {Pith review of: Curriculum Guided Reinforcement Learning for Efficient Multi Hop Retrieval Augmented Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HBE2GPYF}},
  note         = {Machine review of arXiv:2505.17391}
}
read the original abstract

Retrieval-augmented generation (RAG) grounds large language models (LLMs) in up-to-date external evidence, yet existing multi-hop RAG pipelines still issue redundant subqueries, explore too shallowly, or wander through overly long search chains. We introduce EVO-RAG, a curriculum-guided reinforcement learning framework that evolves a query-rewriting agent from broad early-stage exploration to concise late-stage refinement. EVO-RAG couples a seven-factor, step-level reward vector (covering relevance, redundancy, efficiency, and answer correctness) with a time-varying scheduler that reweights these signals as the episode unfolds. The agent is trained with Direct Preference Optimization over a multi-head reward model, enabling it to learn when to search, backtrack, answer, or refuse. Across four multi-hop QA benchmarks (HotpotQA, 2WikiMultiHopQA, MuSiQue, and Bamboogle), EVO-RAG boosts Exact Match by up to 4.6 points over strong RAG baselines while trimming average retrieval depth by 15 %. Ablation studies confirm the complementary roles of curriculum staging and dynamic reward scheduling. EVO-RAG thus offers a general recipe for building reliable, cost-effective multi-hop RAG systems.

Figures

Figures reproduced from arXiv: 2505.17391 by the authors.

Figure 1
Figure 1. Illustration of EVO-RAG’s two-stage curriculum. In [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The query rewriting agent (top) interacts with the environment through four high-level actions and observes retrieved [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Reward weights for EVO-RAG training. Stage 1 and [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Sub-query length (top) and step count (bottom) [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Wiki-R1: Incentivizing Multimodal Reasoning for Knowledge-based VQA via Data and Sampling Curriculum

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Controllable retrieval-difficulty curriculum plus reward-propagation sampling lets RL close the pretrain-to-KB-VQA gap and beat prior SOTA on two hard encyclopedic VQA benchmarks.

  2. An Intelligent Fault Self-Healing Mechanism for Cloud AI Systems via Integration of Large Language Models and Deep Reinforcement Learning

    cs.AI 2025-06 reject novelty 3.0 of 10

    An LLM-plus-deep-RL hybrid is proposed for cloud fault self-healing, claiming 37% faster recovery on unknown faults with weak experimental documentation.

Reference graph

Works this paper leans on

54 extracted references · 10 canonical work pages · cited by 2 Pith papers

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  2. [2]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gr...

    2020

  3. [3]

    Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. 2024. Benchmarking large language models in retrieval-augmented generation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 17754–17762

    2024

  4. [4]

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. arXiv preprint arXiv:2402.03216 (2024)

    arXiv 2024

  5. [5]

    Yiqun Chen, Lingyong Yan, Weiwei Sun, Xinyu Ma, Yi Zhang, Shuaiqiang Wang, Dawei Yin, Yiming Yang, and Jiaxin Mao. 2025. Improving Retrieval- Augmented Generation through Multi-Agent Reinforcement Learning. arXiv preprint arXiv:2501.15228 (2025)

    arXiv 2025

  6. [6]

    Qixin Deng, Qikai Yang, Ruibin Yuan, Yipeng Huang, Yi Wang, Xubo Liu, Zeyue Tian, Jiahao Pan, Ge Zhang, Hanfeng Lin, et al. 2024. ComposerX: Multi-Agent Symbolic Music Composition with LLMs. In The 25th International Society for Music Information Retrieval Conference

    work page 2024

  7. [7]

    Zhicheng Ding, Panfeng Li, Qikai Yang, and Siyang Li. 2024. Enhance image- to-image generation with llava-generated prompts. In 2024 5th International Conference on Information Science, Parallel and Distributed Systems (ISPDS) . IEEE, 77–81

    work page 2024

  8. [8]

    Jingsheng Gao, Linxu Li, Weiyuan Li, Yuzhuo Fu, and Bin Dai. 2024. SmartRAG: Jointly Learn RAG-Related Tasks From the Environment Feedback.arXiv preprint arXiv:2410.18141 (2024)

    arXiv 2024

Show all 54 references
  1. [9]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al . 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025)

    2025 arXiv

  2. [10]

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reason- ing Steps. InProceedings of the 28th International Conference on Computational Lin- guistics. International Committee on Computati...

    2020

  3. [11]

    Jerry Huang, Siddarth Madala, Risham Sidhu, Cheng Niu, Julia Hockenmaier, and Tong Zhang. 2025. RAG-RL: Advancing Retrieval-Augmented Generation via RL and Curriculum Learning. arXiv preprint arXiv:2503.12759 (2025)

    2025 arXiv

  4. [12]

    Yihong Jin and Ze Yang. 2025. Scalability Optimization in Cloud-Based AI Inference Services: Strategies for Real-Time Load Balancing and Automated Scaling. arXiv preprint arXiv:2504.15296 (2025)

    2025 arXiv

  5. [13]

    Yihong Jin, Ze Yang, and Xinhe Xu. 2024. Scam Detection for Ethereum Smart Contracts: Leveraging Graph Representation Learning for Secure Blockchain. arXiv preprint arXiv:2412.12370 (2024)

    2024 arXiv

  6. [14]

    Magdalena Kaiser and Gerhard Weikum. 2025. Preference-based Learning with Retrieval Augmented Generation for Conversational Question Answering. arXiv preprint arXiv:2503.22303 (2025)

    2025 arXiv

  7. [15]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing s...

    2020

  8. [16]

    Zilinghan Li, Shilan He, Ze Yang, Minseok Ryu, Kibaek Kim, and Ravi Madduri

  9. [17]

    Dong Liu and Kaiser Pister. 2024. LLMEasyQuant–An Easy to Use Toolkit for LLM Quantization. arXiv preprint arXiv:2406.19657 (2024)

    2024

  10. [18]

    Dong Liu, Roger Waleffe, Meng Jiang, and Shivaram Venkataraman. 2024. Graph- snapshot: Graph machine learning acceleration with fast storage and retrieval. arXiv preprint arXiv:2406.17918 (2024)

    2024 arXiv

  11. [19]

    Dong Liu and Yanxuan Yu. 2024. Mt2st: Adaptive multi-task to single-task learning. arXiv preprint arXiv:2406.18038 (2024)

    2024 arXiv

  12. [20]

    Tianci Liu, Haoxiang Jiang, Tianze Wang, Ran Xu, Yue Yu, Linjun Zhang, Tuo Zhao, and Haoyu Wang. 2025. Roserag: Robust retrieval-augmented generation with small-scale llms via margin-aware preference optimization. arXiv preprint arXiv:2502.10993 (2025)

    2025 arXiv

  13. [21]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35...

    2022

  14. [22]

    Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. 2022. Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03350 (2022)

    2022 arXiv

  15. [23]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research 21, 140 (2020), 1–67

    2020

  16. [24]

    Stephen Robertson, Hugo Zaragoza, et al . 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends® in Information Retrieval 3, 4 (2009), 333–389

    2009

  17. [25]

    Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. 2025. R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning. arXiv preprint arXiv:2503.05592 (2025)

    2025 arXiv

  18. [26]

    Pei-Chiang Su, Shi-Yi Tan, Zhenyao Liu, and Wei-Chang Yeh. 2022. A mixed- heuristic quantum-inspired simplified swarm optimization algorithm for sched- uling of real-time tasks in the multiprocessor system. Applied Soft Computing 131 (2022), 109807

    2022

  19. [27]

    Shiqi Sun, Kun Zhang, Jingyuan Li, Min Yu, Kun Hou, Yuanzhuo Wang, and Xueqi Cheng. 2025. Retriever-generator-verification: A novel approach to enhancing factual coherence in open-domain question answering. Information Processing & Management 62, 4 (2025), 104147. doi:10.1016/...

    2025

  20. [28]

    Zhongxiang Sun, Qipeng Wang, Weijie Yu, Xiaoxue Zang, Kai Zheng, Jun Xu, Xiao Zhang, Song Yang, and Han Li. 2025. ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewarding. arXiv preprint arXiv:2501.07861 (2025)

    2025 arXiv

  21. [29]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  22. [31]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal

  23. [32]

    Yiteng Tu, Weihang Su, Yujia Zhou, Yiqun Liu, and Qingyao Ai. 2025. RbFT: Ro- bust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defects. arXiv preprint arXiv:2501.18365 (2025)

    2025 arXiv

  24. [33]

    Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou

  25. [34]

    Transactions of the Association for Computational Linguistics 10 (2022), 539–554

    MuSiQue: Multihop Questions via Single-hop Question Composition. Transactions of the Association for Computational Linguistics 10 (2022), 539–554

    2022

  26. [35]

    Zilong Wang, Zifeng Wang, Long Le, Huaixiu Steven Zheng, Swaroop Mishra, Vincent Perot, Yuwei Zhang, Anush Mattapalli, Ankur Taly, Jingbo Shang, et al

  27. [36]

    Zhepei Wei, Wei-Lin Chen, and Yu Meng. 2024. InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales. arXiv preprint arXiv:2406.13629 (2024)

    2024 arXiv

  28. [37]

    Ao Xiang, Bingjie Huang, Xinyu Guo, Haowei Yang, and Tianyao Zheng. 2024. A neural matrix decomposition recommender system model based on the multi- modal large language model. InProceedings of the 2024 7th International Conference on Machine Learning and Machine Intelligence ...

    2024

  29. [38]

    Yujing Wang, Hainan Zhang, Liang Pang, Binghui Guo, Hongwei Zheng, and Zhiming Zheng. 2024. MaFeRw: Query rewriting with multi-aspect feedbacks for retrieval-augmented large language models. arXiv preprint arXiv:2408.17072 (2024)

    2024 arXiv

  30. [39]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 Technical Report. arXiv preprint arXiv:2505.09388 (2025)

    2025 arXiv

  31. [40]

    arXiv preprint arXiv:2407.08223 (2024)

    Speculative rag: Enhancing retrieval augmented generation through draft- ing. arXiv preprint arXiv:2407.08223 (2024)

    2024 arXiv

  32. [41]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A dataset for di- verse, explainable multi-hop question answering. arXiv preprint arXiv:1809.09600 (2018)

    2018 arXiv

  33. [42]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR)

    2023

  34. [43]

    Guangzhi Xiong, Qiao Jin, Xiao Wang, Yin Fang, Haolin Liu, Yifan Yang, Fangyuan Chen, Zhixing Song, Dengyu Wang, Minjia Zhang, et al. 2025. Rag-gym: Opti- mizing reasoning and search agents with process supervision. arXiv preprint arXiv:2502.13957 (2025). Curriculum-Guided Rei...

    2025 arXiv

  35. [44]

    Nan Zhang, Prafulla Kumar Choubey, Alexander Fabbri, Gabriel Bernadett- Shapiro, Rui Zhang, Prasenjit Mitra, Caiming Xiong, and Chien-Sheng Wu. 2024. SiReRAG: Indexing Similar and Related Information for Multihop Reasoning. arXiv preprint arXiv:2412.06206 (2024)

    2024 arXiv

  36. [45]

    Qikai Yang, Cheng Ji, Huaiying Luo, Panfeng Li, and Zhicheng Ding. 2025. Data Augmentation Through Random Style Replacement. arXiv preprint arXiv:2504.10563 (2025)

    2025

  37. [46]

    Zongmeng Zhang, Yufeng Shi, Jinhua Zhu, Wengang Zhou, Xiang Qi, Peng Zhang, and Houqiang Li. 2024. Trustworthy alignment of retrieval-augmented large language models via reinforcement learning. arXiv preprint arXiv:2410.16843 (2024)

    2024 arXiv

  38. [47]

    Chuqing Zhao and Yisong Chen. 2025. LLM-powered Topic Modeling for Dis- covering Public Mental Health Trends in Social Media. (2025)

    2025

  39. [48]

    Hanning Zhang, Juntong Song, Juno Zhu, Yuanhao Wu, Tong Zhang, and Cheng Niu. 2025. RAG-Reward: Optimizing RAG with Reward Modeling and RLHF. arXiv preprint arXiv:2501.13264 (2025)

    2025

  40. [49]

    Rongzhi Zhu, Xiangyu Liu, Zequn Sun, Yiwei Wang, and Wei Hu. 2025. Miti- gating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answering. arXiv preprint arXiv:2502.14245 (2025). Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009

    2025 arXiv

  41. [50]

    Zheyu Zhang, Yutong Luo, Yongzhou Chen, Haopeng Zhao, Zhichao Ma, and Hao Liu. 2025. Automated Parking Trajectory Generation Using Deep Reinforcement Learning. arXiv preprint arXiv:2504.21071 (2025)

    2025 arXiv

  42. [53]

    Haopeng Zhao, Zhichao Ma, Lipeng Liu, Yang Wang, Zheyu Zhang, and Hao Liu

  43. [2020]

    Advances in neural information processing systems 33 (2020), 5776–5788

    Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. Advances in neural information processing systems 33 (2020), 5776–5788

    2020

  44. [2022]

    arXiv preprint arXiv:2212.10509 (2022)

    Interleaving retrieval with chain-of-thought reasoning for knowledge- intensive multi-step questions. arXiv preprint arXiv:2212.10509 (2022)

    2022 arXiv

  45. [2024]

    arXiv preprint arXiv:2409.11585 (2024)

    Advances in appfl: A comprehensive and extensible federated learning framework. arXiv preprint arXiv:2409.11585 (2024)

    2024 arXiv

  46. [2025]

    arXiv preprint arXiv:2504.05339 (2025)

    Optimized Path Planning for Logistics Robots Using Ant Colony Algorithm under Multiple Constraints. arXiv preprint arXiv:2504.05339 (2025)

    2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.