Pith. sign in

REVIEW 4 major objections 3 minor 7 cited by

ParallelSearch: Train your LLMs to Decompose Query and Search Sub-queries in Parallel with Reinforcement Learning

T0 review · 4 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ParallelSearch trains LLM agents to split queries into independent sub-queries and execute them concurrently, improving question-answering accuracy by 2.9% on average and using 69.6% of the LLM calls of sequential search on parallelizable q

desk verdict Abstract-only read: plausible RLVR extension that teaches parallel query decomposition, but the efficiency number is an optimized objective and the parallel-execution quality assumption is unexamined. read the letter →

arxiv 2508.09303 v1 pith:LSKGWY2I submitted 2025-08-12 cs.CL cs.AIcs.IR

Shu Zhao , Tan Yu , Anbang Xu , Japinder Singh , Aaditya Shukla , Rama Akkiraju This is my paper
classification cs.CLcs.AIcs.IR
keywords reinforcementlearningsearchagentsquerydecompositionparallelexecutionquestionansweringtooluseLLMmulti-stepretrieval
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper targets a bottleneck in reasoning-augmented search agents: they issue one search query at a time even when a question can be split into independent parts. ParallelSearch trains a large language model with reinforcement learning to decompose such questions, run the resulting sub-queries concurrently, and combine the answers. The authors report that the trained agent beats sequential baselines by 2.9% on average across seven question-answering benchmarks, and on parallelizable questions it gains 12.7% while using 69.6% of the LLM calls. If these numbers hold, parallel decomposition is a practical way to make search-augmented reasoning both more accurate and cheaper.

What carries the argument

The central mechanism is a reinforcement-learning training scheme with a three-part reward: correctness of the final answer, quality of the query decomposition, and the parallel-execution benefit measured by model calls saved. The decomposition-quality term is what lets the model learn to spot independent question components instead of following a fixed step-by-step search routine; the parallel-benefit term ties that skill directly to efficiency.

What would settle it

Take the trained ParallelSearch policy on the parallelizable benchmark questions and run the same sub-queries in two modes, concurrent tool calls and serialized tool calls, while logging per-sub-query retrieval success and final answer accuracy; if serialized execution matches or beats concurrent execution, the reported gains come from the search environment rather than from parallel execution itself.

Watch

Extended reading notes

Core claim

The paper's claim is that parallel search can be trained, not hard-coded: a reinforcement-learning procedure can teach a large language model to recognize when an input question decomposes into logically independent sub-queries, issue those sub-queries concurrently, and use the results to answer. The reward design simultaneously scores final-answer correctness, decomposition quality, and the reduction in model calls from parallel execution. Across seven question-answering benchmarks the trained agent outperforms state-of-the-art sequential search baselines by 2.9% on average, and on questions classified as parallelizable the gain is 12.7% while using only 69.6% of the model calls of sequenti

Load-bearing premise

The reported accuracy and efficiency gains assume that running several search sub-queries at the same time returns the same per-query result quality as running them one after another, with no extra rate-limit failures, duplicate results, truncated contexts, or tool interference.

Editorial extensions

If this is right

  • Queries that compare several entities, the clearest parallelizable case, should show the largest accuracy gains, consistent with the reported 12.7% jump.
  • Sequential search-agent pipelines can be retrained with these rewards without changing the underlying model architecture.
  • The 69.6%-of-calls figure implies roughly one-third fewer LLM invocations on parallelizable questions, lowering per-query cost.
  • Because the gains are averaged over seven benchmarks, the benefit is not confined to a single domain or question type.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment is an ablation that decomposes sub-queries but serializes their execution; if that ablation preserves most of the accuracy gain, the learned planning skill matters more than concurrency itself.
  • The reported call count is not wall-clock time; measuring end-to-end latency with a real search API could reveal a smaller or larger efficiency gain depending on tool latency and rate limits.
  • The same decomposition-plus-execution reward could transfer to other independent tool calls, such as parallel database queries, code checks, or unit tests, whenever the sub-tasks are truly independent.
  • If future question sets tilt toward multi-entity comparisons, the average advantage over sequential agents may exceed the 2.9% reported here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper proposes ParallelSearch, an RLVR-trained framework that extends reasoning-augmented search agents such as Search-R1 by allowing an LLM to decompose a query into independent sub-queries and execute them in parallel. The method uses a composite reward incorporating answer correctness, query decomposition quality, and parallel execution benefit. The abstract reports an average 2.9% performance improvement over state-of-the-art baselines across seven question-answering benchmarks, a 12.7% improvement on parallelizable questions, and a reduction to 69.6% of the LLM calls relative to sequential approaches. The central claim is that RL with parallel-execution-aware rewards simultaneously improves accuracy and reduces tool calls.

Significance. If the reported results hold, the work addresses a real architectural bottleneck in current reasoning-augmented search agents and offers a principled RL-based approach to make such agents more efficient. The combination of accuracy and efficiency gains is practically valuable, and the decomposition-based reward structure is a plausible mechanism for inducing parallelizable search behavior. However, the manuscript as available (essentially the abstract) provides no experimental detail, no definition of the parallelizable subset, no baseline list, no variability measures, and no ablation separating the effects of decomposition quality from the parallel-execution reward. The significance therefore depends on evidence that is not yet visible.

major comments (4)
  1. [Abstract, 'parallelizable questions'] The 12.7% gain is stated only for 'parallelizable questions,' but the abstract neither defines this subset nor explains how it is identified. Without a transparent, pre-specified criterion, the reader cannot rule out selection bias or circularity (e.g., the subset may be chosen based on the model's own decomposition behavior). This is load-bearing for the headline accuracy claim.
  2. [Abstract, reward design] The abstract states that the reward jointly considers correctness, decomposition quality, and 'parallel execution benefits.' The 69.6% call reduction is therefore a direct optimization target of the reward, not an independent emergent property. The paper needs an ablation that isolates the contribution of the parallel-benefit reward term from the correctness term, otherwise the efficiency claim is partly a restatement of the training objective.
  3. [Abstract, environment assumption] The efficiency and accuracy results assume that issuing multiple sub-queries in parallel returns retrieval results of the same quality as issuing them sequentially. The abstract presents no evidence on per-query retrieval quality, rate limits, snippet duplication, or context truncation under concurrent tool calls. If parallel execution degrades retrieval or sequential execution is handicapped by tool-side effects, the reported gains would be artifacts of the environment rather than the learned policy. This is a missing empirical block, not a demonstrated flaw, but it is central to both headline numbers.
  4. [Abstract, 'seven QA benchmarks' and '2.9%'] The average 2.9% gain is reported without naming the seven benchmarks, the baselines, the standard deviations, or the number of runs. Given that the gain is a small single-digit percentage, run-to-run variance could plausibly account for it. The paper must supply benchmark details, error bars, and a comparison protocol to make the claim interpretable.
minor comments (3)
  1. [Abstract] The term 'state-of-the-art baselines' is vague; specific baseline names and versions should be given.
  2. [Abstract] The phrase 'requiring only 69.6% of the LLM calls compared to sequential approaches' is ambiguous: it should state the exact counting convention (e.g., calls to the LLM for generating sub-queries, retrieval calls, or both).
  3. [Abstract] The acronym 'RLVR' is used without expansion; the manuscript should define it at first use.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found in available text; abstract reports empirical RL-trained results against external benchmarks, with no quoted reduction to inputs.

full rationale

Only the abstract is available in the provided manuscript text. The paper's central claims are empirical: a reinforcement learning framework with reward functions for correctness, query decomposition quality, and parallel execution benefits is trained and then evaluated on seven question-answering benchmarks against state-of-the-art baselines. The reported performance improvements (2.9% average, 12.7% on parallelizable questions) and efficiency improvement (69.6% of LLM calls) are measured outcomes of training, not identities derived from the reward definitions. There is no quoted equation showing that a predicted quantity is equal by construction to a fitted input, no fitted parameter renamed as a prediction, and no load-bearing self-citation chain. The efficiency gain being aligned with a parallel-execution reward is the method's intended objective, not a circular derivation: reward optimization and post-training evaluation are distinct stages. Under the hard rule that circularity must be demonstrated by specific quoted reduction, and with no full-text equations or ablation available, no circular step can be flagged. The absence of evidence for retrieval-quality equivalence under parallel execution is a legitimate empirical concern, but it is an unverified tooling assumption, not a circularity in the paper's derivation chain. Therefore the honest finding is no significant circularity, score 0.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim borrows the whole RLVR training stack from prior work (Search-R1-style agents) and adds a multi-term reward. The visible free parameters are the reward-mixture weights and the definition or threshold of 'parallelizable questions.' The main domain assumptions are that search tool calls can execute concurrently without quality loss and that decomposition quality is verifiable without the trained model's own preferences leaking into the reward. No new entities such as particles, mediators, or forces are introduced.

free parameters (2)
  • reward mixture weights (correctness, decomposition quality, parallel benefit) = not reported in abstract
    The reward 'jointly considering correctness, query decomposition quality, and parallel execution benefits' requires weights or normalization chosen by hand; the abstract does not report them.
  • threshold or definition of 'parallelizable questions' = not reported in abstract
    The 12.7% gain is measured on this subset; if the definition is tuned or applied post hoc, the headline gain is selective.
assumptions (3)
  • domain assumption RLVR training with verifiable rewards transfers to LLM search agents (as established for Search-R1).
    The method builds directly on the Search-R1 training paradigm cited in the abstract; correctness of the whole framework inherits from it.
  • domain assumption The search or tool environment supports concurrent execution of independent sub-queries without degrading per-query retrieval quality.
    Sustains the 69.6% call figure and the parallel-decomposition reward; not stated or evidenced in the abstract.
  • domain assumption Query decomposition quality can be scored automatically without access to the final answer or leakage from the trained model's own preferences.
    The 'query decomposition quality' reward term must be computed during RL; its reliability is invisible in the abstract.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ParallelSearch: Train your LLMs to Decompose Query and Search Sub-queries in Parallel with Reinforcement Learning." pith.science (2026). https://pith.science/paper/LSKGWY2I

@misc{pith2026250809303,
  author       = {Pith},
  title        = {Pith review of: ParallelSearch: Train your LLMs to Decompose Query and Search Sub-queries in Parallel with Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LSKGWY2I}},
  note         = {Machine review of arXiv:2508.09303}
}
read the original abstract

Reasoning-augmented search agents such as Search-R1, trained via reinforcement learning with verifiable rewards (RLVR), demonstrate remarkable capabilities in multi-step information retrieval from external knowledge sources. These agents address the limitations of their parametric memory by dynamically gathering relevant facts to address complex reasoning tasks. However, existing approaches suffer from a fundamental architectural limitation: they process search queries strictly sequentially, even when handling inherently parallelizable and logically independent comparisons. This sequential bottleneck significantly constrains computational efficiency, particularly for queries that require multiple entity comparisons. To address this critical limitation, we propose ParallelSearch, a novel reinforcement learning framework that empowers large language models (LLMs) to recognize parallelizable query structures and execute multiple search operations concurrently. Our approach introduces dedicated reward functions that incentivize the identification of independent query components while preserving answer accuracy through jointly considering correctness, query decomposition quality, and parallel execution benefits. Comprehensive experiments demonstrate that ParallelSearch outperforms state-of-the-art baselines by an average performance gain of 2.9% across seven question-answering benchmarks. Notably, on parallelizable questions, our method achieves a 12.7% performance improvement while requiring only 69.6% of the LLM calls compared to sequential approaches.

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. HyperEyes: Dual-Grained Efficiency-Aware Reinforcement Learning for Parallel Multimodal Search Agents

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    HyperEyes uses a dual-grained RL framework with parallel tool actions and efficiency rewards to achieve 9.9% higher accuracy and 5.3x fewer tool calls than prior open-source multimodal agents.

  2. LatentRAG: Latent Reasoning and Retrieval for Efficient Agentic RAG

    cs.CL 2026-05 unverdicted novelty 7.0 of 10

    LatentRAG performs agentic RAG by generating latent tokens for thoughts and subqueries in one forward pass, matching explicit methods' accuracy on seven benchmarks while reducing latency by ~90%.

  3. Planner-Centric Reinforcement Learning for Deep Research with Structure-Aware Reward

    cs.AI 2026-05 unverdicted novelty 6.0 of 10

    DecomposeR represents research plans as typed DAGs and uses two-stage planner-then-answerer RL to improve long-form research performance by 5.1-8.0 points over baselines.

  4. HyperEyes: Dual-Grained Efficiency-Aware Reinforcement Learning for Parallel Multimodal Search Agents

    cs.LG 2026-05 unverdicted novelty 6.0 of 10

    HyperEyes presents a parallel multimodal search agent using dual-grained efficiency-aware RL with a new TRACE reward and IMEB benchmark, claiming 9.9% higher accuracy and 5.3x fewer tool calls than prior open-source agents.

  5. Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learning

    cs.CL 2025-12 unverdicted novelty 6.0 of 10

    NPR trains LLMs to reason in parallel via self-distilled RL, delivering up to 24.5% performance gains and 4.6x speedups with 100% genuine parallel execution on reasoning benchmarks.

  6. Hybrid Deep Searcher: Scalable Parallel and Sequential Search Reasoning

    cs.AI 2025-08 conditional novelty 6.0 of 10

    Fine-tuning Qwen3-8B on a synthetic hybrid-hop dataset with parallel and sequential search trajectory supervision yields a search agent that outperforms several agentic RAG baselines on five QA benchmarks.

  7. Erase to Improve: Erasable Reinforcement Learning for Search-Augmented LLMs

    cs.CL 2025-10 unverdicted novelty 5.0 of 10

    ERL trains LLMs to erase faulty reasoning steps and regenerate them in place, yielding gains of up to 8.48% EM on multi-hop QA benchmarks like HotpotQA.

Reference graph

Works this paper leans on

43 extracted references · 25 canonical work pages · cited by 6 Pith papers

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Back to basics: Revisiting reinforce-style optimization for learning from human feedback in llms

    Arash Ahmadian, Chris Cremer, Matthias Gall \' e , Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet \" U st \" u n, and Sara Hooker. Back to basics: Revisiting reinforce-style optimization for learning from human feedback in llms. In Annual Meeting of the Association for Computational Linguistics (ACL), 2024

    work page 2024

  3. [3]

    A review of factors influencing user satisfaction in information retrieval

    Azzah Al - Maskari and Mark Sanderson. A review of factors influencing user satisfaction in information retrieval. Journal of the Association for Information Science and Technology (JASIST), 2010

    work page 2010

  4. [4]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    arXiv 2024

  5. [5]

    Retrieval-augmented generation for large language models: A survey

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, and Haofen Wang. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997, 2023

    arXiv 2023

  6. [6]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  7. [7]

    Retrieval augmented language model pre-training

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming - Wei Chang. Retrieval augmented language model pre-training. In International Conference on Machine Learning (ICML), 2020

    work page 2020

  8. [8]

    Constructing A multi-hop QA dataset for comprehensive evaluation of reasoning steps

    Xanh Ho, Anh - Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing A multi-hop QA dataset for comprehensive evaluation of reasoning steps. In International Conference on Computational Linguistics (COLING), 2020

    work page 2020

Show all 43 references
  1. [9]

    ORPO: monolithic preference optimization without reference model

    Jiwoo Hong, Noah Lee, and James Thorne. ORPO: monolithic preference optimization without reference model. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024

    2024

  2. [10]

    Survey of hallucination in natural language generation

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 2023

    2023

  3. [11]

    Bowen Jin, Jinsung Yoon, Jiawei Han, and Sercan \" O . Arik. Long-context llms meet RAG: overcoming challenges for long inputs in RAG . In International Conference on Learning Representations (ICLR), 2025 a

    2025

  4. [12]

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516, 2025 b

    2025 arXiv

  5. [13]

    Weld, and Luke Zettlemoyer

    Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Annual Meeting of the Association for Computational Linguistics (ACL), 2017

    2017

  6. [14]

    Dense passage retrieval for open-domain question answering

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen - tau Yih. Dense passage retrieval for open-domain question answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020

    2020

  7. [15]

    A survey of reinforcement learning from human feedback

    Timo Kaufmann, Paul Weng, Viktor Bengs, and Eyke H \"u llermeier. A survey of reinforcement learning from human feedback. arXiv preprint arXiv:2312.14925, 2023

    2023

  8. [16]

    Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming - Wei Chang, Andrew M

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur P. Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming - Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and...

    2019

  9. [17]

    Miranda, Bill Yuchen Lin, Khyathi Raghavi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A

    Nathan Lambert, Valentina Pyatkin, Jacob Morrison, Lester James V. Miranda, Bill Yuchen Lin, Khyathi Raghavi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi. Rewardbench: Evaluating reward models for language modeling. In North A...

    2025

  10. [18]

    Large language models in finance: A survey

    Yinheng Li, Shaofei Wang, Han Ding, and Hang Chen. Large language models in finance: A survey. In International Conference on AI in Finance (ICAIF), 2023

    2023

  11. [19]

    When not to trust language models: Investigating effectiveness of parametric and non-parametric memories

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Annual Meeting of the Association for Computational Linguistics (ACL), 2023

    2023

  12. [20]

    O2-searcher: A searching-based agent model for open-domain open-ended question answering

    Jianbiao Mei, Tao Hu, Daocheng Fu, Licheng Wen, Xuemeng Yang, Rong Wu, Pinlong Cai, Xing Gao, Yu Yang, Chengjun Xie, et al. O2-searcher: A searching-based agent model for open-domain open-ended question answering. arXiv preprint arXiv:2505.16582, 2025

    2025 arXiv

  13. [21]

    Simpo: Simple preference optimization with a reference-free reward

    Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference-free reward. In Conference on Neural Information Processing Systems (NeurIPS), 2024

    2024

  14. [22]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leik...

    2022

  15. [23]

    Iterative reasoning preference optimization

    Richard Yuanzhe Pang, Weizhe Yuan, He He, Kyunghyun Cho, Sainbayar Sukhbaatar, and Jason Weston. Iterative reasoning preference optimization. In NeurIPS, 2024

    2024

  16. [24]

    Smith, and Mike Lewis

    Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. Measuring and narrowing the compositionality gap in language models. In Conference on Empirical Methods in Natural Language Processing (EMNLP) Findings, 2023

    2023

  17. [25]

    Manning, Stefano Ermon, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Conference on Neural Information Processing Systems (NeurIPS), 2023

    2023

  18. [26]

    Toolformer: Language models can teach themselves to use tools

    Timo Schick, Jane Dwivedi - Yu, Roberto Dess \` , Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. In Conference on Neural Information Processing Systems (NeurIPS), 2023

    2023

  19. [27]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  20. [28]

    Spurious rewards: Rethinking training signals in rlvr

    Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, et al. Spurious rewards: Rethinking training signals in rlvr. arXiv preprint arXiv:2506.10947, 2025

    2025 arXiv

  21. [29]

    Zerosearch: Incentivize the search capability of llms without searching

    Hao Sun, Zile Qiao, Jiayan Guo, Xuanbo Fan, Yingyan Hou, Yong Jiang, Pengjun Xie, Fei Huang, and Yan Zhang. Zerosearch: Incentivize the search capability of llms without searching. arXiv preprint arXiv:2505.04588, 2025

    2025 arXiv

  22. [30]

    Sutton and Andrew G

    Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction. IEEE Transactions on Neural Networks, 1998

    1998

  23. [31]

    Multihop- RAG : Benchmarking retrieval-augmented generation for multi-hop queries

    Yixuan Tang and Yi Yang. Multihop- RAG : Benchmarking retrieval-augmented generation for multi-hop queries. In Conference on Language Modeling (COLM), 2024

    2024

  24. [32]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  25. [33]

    Musique: Multihop questions via single-hop question composition

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics (TACL), 2022

    2022

  26. [34]

    Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. In Annual Meeting of the Association for Computational Linguistics (ACL), 2023

    2023

  27. [35]

    Acting less is reasoning more! teaching model to act efficiently

    Hongru Wang, Cheng Qian, Wanjun Zhong, Xiusi Chen, Jiahao Qiu, Shijue Huang, Bowen Jin, Mengdi Wang, Kam-Fai Wong, and Heng Ji. Acting less is reasoning more! teaching model to act efficiently. arXiv preprint arXiv:2504.14870, 2025 a

    2025 arXiv

  28. [36]

    Text embeddings by weakly-supervised contrastive pre-training

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533, 2022

    2022 arXiv

  29. [37]

    Stepsearch: Igniting llms search ability via step-wise proximal policy optimization

    Ziliang Wang, Xuhui Zheng, Kang An, Cijun Ouyang, Jialu Cai, Yuhang Wang, and Yichao Wu. Stepsearch: Igniting llms search ability via step-wise proximal policy optimization. arXiv preprint arXiv:2505.15107, 2025 b

    2025 arXiv

  30. [38]

    Reasoning or memorization? unreliable results of reinforcement learning due to data contamination

    Mingqi Wu, Zhihao Zhang, Qiaole Dong, Zhiheng Xi, Jun Zhao, Senjie Jin, Xiaoran Fan, Yuhao Zhou, Yanwei Fu, Qin Liu, et al. Reasoning or memorization? unreliable results of reinforcement learning due to data contamination. arXiv preprint arXiv:2507.10532, 2025 a

    2025

  31. [39]

    Masksearch: A universal pre-training framework to enhance agentic search capability

    Weiqi Wu, Xin Guan, Shen Huang, Yong Jiang, Pengjun Xie, Fei Huang, Jiuxin Cao, Hai Zhao, and Jingren Zhou. Masksearch: A universal pre-training framework to enhance agentic search capability. arXiv preprint arXiv:2505.20285, 2025 b

    2025 arXiv

  32. [40]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  33. [41]

    Cohen, Ruslan Salakhutdinov, and Christopher D

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018

    2018

  34. [42]

    Narasimhan, and Yuan Cao

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023

    2023

  35. [43]

    R-search: Empowering llm reasoning with search via multi-reward reinforcement learning

    Qingfei Zhao, Ruobing Wang, Dingling Xu, Daren Zha, and Limin Liu. R-search: Empowering llm reasoning with search via multi-reward reinforcement learning. arXiv preprint arXiv:2506.04185, 2025

    2025 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.