REVIEW 3 major objections 5 minor 4 cited by
MaskSearch: A Universal Pre-Training Framework to Enhance Agentic Search Capability
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Pre-training a language model to fill masked spans by web search transfers to open-domain QA and beats single-stage training across model sizes.
desk verdict A genuine, well-engineered pre-training recipe for agentic search, but the central causal claim is undercut by a missing same-scale control. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the RAMP instance: a Wikipedia paragraph, 1 to 4 salient spans replaced by a [mask] token, and a search tool the model may call multiple times before emitting the reconstructed paragraph. Salient spans are chosen to require world knowledge rather than local context. Training examples are chain-of-thought trajectories from a multi-agent system (planner, rewriter, observer) filtered by LLM-as-Judge, then scaled by iterative self-evolve distillation with the growing model as teacher. SFT optimizes the trajectory directly, while RL uses DAPO with a reward that is half format compliance and half answer correctness, with the paper adopting a model-based answer judge after finding token-recall rewards invite reward hacking. Curriculum learning sorts examples by number of masks, from 1 to 4, to increase difficulty gradually.
What would settle it
Fine-tune the same base model on a same-size 10M corpus of control trajectories that include the same searches but require no mask-filling (for example, copying the retrieved passage), then fine-tune on HotpotQA; if downstream recall matches the RAMP-pretrained model, the specific mask-filling claim is refuted.
Extended reading notes
Core claim
The central claim is that a universal, infinitely scalable search-agent pre-training task exists, and that RAMP is one. RAMP turns any paragraph into a training example by masking salient spans (entities, dates, ontologies, numbers) and requiring the model to plan, issue search queries, read retrieved passages, and output the complete restored text. Trained this way, the model internalizes a general decompose-search-observe-reason capability rather than a dataset-specific pattern. The paper demonstrates the transfer by first pre-training on RAMP, using up to 10M examples generated through multi-agent synthesis followed by self-evolving distillation, and then post-training on HotpotQA, reporting consistent recall improvements over the same models trained only on HotpotQA, with larger out-of-domain gains on FanoutQA, Musique, 2WikiMultiHopQA, Bamboogle, and FreshQA. It further claims that applying RL to RAMP via DAPO with a hybrid format-and-answer reward yields even higher downstream upper bounds, and that curriculum learning over mask counts strengthens the effect.
Load-bearing premise
The attribution of the gains to the RAMP task design assumes that the effect is not just extra training data and extra search-tool practice, because the pre-training corpus has 10M examples while the compared baselines see only 58K HotpotQA trajectories.
Editorial extensions
If this is right
- RAMP pre-training before HotpotQA post-training raises average token recall by several points across all six tested QA datasets and all tested model sizes, so the benefit is not tied to one architecture or benchmark.
- Smaller models pre-trained with RAMP can reach recall levels of larger models that only received downstream training, suggesting pre-training can partially substitute for model scale.
- Reinforcement learning on the RAMP task, not just supervised learning, improves later agentic QA performance, indicating the pre-training signal works for both training paradigms.
- Curriculum learning ordered by number of masks outperforms mixing all difficulties, so example difficulty ordering matters for the pre-training stage.
- The 10M-example scaling experiment shows performance continues to rise with more RAMP data for small models, supporting the scalability of the task.
Reading between the lines
- If the transfer is real, any task expressible as 'fill missing spans by searching' (entity resolution, fact-checking, data wrangling) could inherit the same pre-training benefit without new labeled data, since RAMP data is generated from raw text alone.
- The paper does not test whether the gains come from the mask-filling objective itself or from sheer additional search-tool practice; a same-scale control pre-training on search trajectories without masks would isolate the mechanism.
- The multi-tool extension the authors list as future work is a natural next test: masking spans that require dates, numbers, code, or images would turn RAMP into a general agentic pre-training task beyond retrieval.
- Because the 10M dataset is generated by Qwen-family teachers and still transfers to LLaMA models, the task may enable cross-family pre-training data sharing at scale.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MaskSearch, a two-stage training framework for LLM-based search agents. In the first stage, models are trained on a Retrieval-Augmented Mask Prediction (RAMP) task: given a paragraph with salient spans masked, the model must use a search tool and multi-step reasoning to fill them. The RAMP corpus is constructed from Wikipedia with agent-based trajectory generation (planner/rewriter/observer) followed by iterative self-evolve distillation, yielding 10M CoT examples (14B tokens). In the second stage, models are trained on downstream HotpotQA via SFT or RL (DAPO with a hybrid reward), and evaluated on HotpotQA plus out-of-domain multi-hop QA benchmarks (FanoutQA, Musique, 2WikiMultiHopQA, Bamboogle, FreshQA). The central claim is that RAMP pre-training produces stable recall gains over single-stage training on HotpotQA alone, with larger out-of-domain gains, across Qwen and LLaMA model families.
Significance. If the causal attribution holds, MaskSearch would be a useful scalable objective for pre-training agentic retrieval-and-reasoning behavior, and the paper provides a plausible and well-motivated recipe. The strengths are the breadth of the empirical sweep (six model sizes across two families, six datasets), the explicit scaling analysis in Figure 2, the curriculum-learning analysis, and the release of code. The gains are consistent across model families, which makes the phenomenon credible. However, the paper's central transfer claim currently rests on a comparison that confounds task design with additional training data and compute, so the significance is conditional on an added control experiment.
major comments (3)
- [Section 4 (Baselines); Section 3.3; Table 2] The comparison in Table 2 does not isolate RAMP's task design. MaskSearch pre-trains on a 10M-example / 14B-token RAMP corpus (Section 3.3) before the 58K HotpotQA SFT, whereas Distilled Search-R1 and Search-R1 receive only the 58K HotpotQA trajectories described in Section 4. No baseline pre-trains the same corpus at the same scale with an alternative objective, so the reported 3-8 point gains could be explained by the extra training tokens, the extra exposure to the search tool, or the self-evolve data-generation pipeline rather than by masked-span prediction. Figure 2's scaling curve is equally consistent with generic continued pre-training. The admitted need for deeper analysis in Section H does not substitute for this control. Please add a same-scale control that continues pre-training on the same Wikipedia corpus with a standard language-modeling objective, and ideally a second control that trains on the same corpus with masks removed while keeping the search-tool interaction.
- [Section 5.1; Table 1; Table 2] All headline results are single-run token-level recall scores on development splits (Table 1 marks several datasets with a dagger to indicate that the dev split is used as the test set), with no error bars, multiple seeds, or significance tests. The claimed improvements are 3-8 points on the in-domain task and vary across datasets, and the authors themselves show in Section 6.2 that the token-level recall metric can be gamed by longer answers. To support the wording 'significantly enhances,' the paper should report variance over at least a few seeds or a paired significance test, especially for the smaller improvements.
- [Section 3.3; Appendix E; Experimental setup] The search tool used for RAMP pre-training, downstream training, and evaluation is never concretely specified: no API or retriever name, result count, snippet length, or number of retrieval tokens is provided beyond the prompt template in Appendix E. Because the paper's central claim is that the model learns to use a search tool, this omission blocks replication and makes it difficult to determine how much of the observed gain is specific to the RAMP task versus to the particular retrieval environment. Please document the retrieval backend, the number and length of retrieved results, and any query-rewriting or retrieval-token limits used in both stages.
minor comments (5)
- [Section 5.1] In the sentence reporting the Bamboogle gain, 'Qwen2.15-1.5B' should be 'Qwen2.5-1.5B'.
- [Section 6.2] The sentence 'Model-based reward offer significant improvements of 34.71 and 19.48 over the other two reward methods' is ungrammatical and the numbers are not tied to any table or axis in Figure 5; please state what quantities these numbers refer to or remove them.
- [Table 3] The table header 'w/ RAMP w/o RAMP CL Mix' is ambiguous; it is unclear which columns correspond to curriculum learning and mixed training under each condition. Also, the text says CL outperforms Mix on Qwen2.5-7B and LLaMA-3.2-1B, but the table shows the opposite direction for Qwen2.5-1.5B and LLaMA-3.1-8B, so the sentence 'indicating that the approach can generalize across different model architectures' is too strong.
- [Section 6.1] The PPL-based masking experiment uses a 40K RAMP dataset with 6K HotpotQA trajectories, whereas the main results use the 10M RAMP corpus with 58K HotpotQA trajectories; the comparison in Figure 4(b) should state explicitly that these are smaller-scale settings and that the two settings are not directly comparable.
- [Section 5.2] The text says that results at training step 0 align precisely with the Direct SFT baseline, but no Direct SFT baseline is plotted or labeled in Figure 2; please add the baseline curve or state where it appears.
Circularity Check
No significant circularity: the central claim is an empirical transfer result measured on held-out QA benchmarks, not a derivation from its own inputs.
full rationale
The paper's load-bearing claim is that two-stage RAMP pre-training plus HotpotQA post-training improves recall on HotpotQA and other external QA sets. This is supported by benchmark comparisons in Table 2, Figure 2 and Table 5 against baselines trained only on downstream data. Nothing in the paper defines the reported downstream recall in terms of the RAMP objective: the masked-span answer comes from the original Wikipedia paragraph, while the evaluation answers come from public QA datasets; the two are connected only through empirical transfer. The self-evolve distillation loop (Eq. 3-4) does generate new pretraining trajectories from a model fine-tuned on its own earlier outputs, which is a self-training data-construction loop, but it is not used to compute the evaluation numbers and therefore is not a circular derivation. The absence of a same-scale control pre-training with an alternative objective means the causal attribution to the mask-prediction design is not fully controlled (extra tokens and extra search-tool practice are confounded), but that is an experimental-control limitation rather than the logical equivalence or fitted-input-as-prediction pattern this pass targets. Self-citations (Refs. 6 and 25) appear only as background related work and are not load-bearing for the main result.
Assumptions & free parameters
free parameters (5)
- Answer reward weight in hybrid reward =
0.5
- Length penalty alpha =
0.2
- Length penalty beta =
8
- Length penalty gamma =
4
- Number of masked spans k =
1 to 4
assumptions (4)
- domain assumption Wikipedia is a representative corpus for learning universal agentic search capabilities.
- domain assumption Token-level recall is a valid proxy for answer correctness in open-domain QA.
- domain assumption LLM-as-Judge filtering and model-based RL rewards reliably identify correct answers.
- domain assumption The search engine returns sufficiently relevant results to support multi-step retrieval.
Cite this review
Pith. "Pith review of MaskSearch: A Universal Pre-Training Framework to Enhance Agentic Search Capability." pith.science (2026). https://pith.science/paper/7FA7HPHQ
@misc{pith2026250520285,
author = {Pith},
title = {Pith review of: MaskSearch: A Universal Pre-Training Framework to Enhance Agentic Search Capability},
year = {2026},
howpublished = {\url{https://pith.science/paper/7FA7HPHQ}},
note = {Machine review of arXiv:2505.20285}
}
read the original abstract
Retrieval-Augmented Language Models (RALMs) represent a classic paradigm where models enhance generative capabilities using external knowledge retrieved via a specialized module. Recent advancements in Agent techniques enable Large Language Models (LLMs) to autonomously utilize tools for retrieval, planning, and reasoning. While existing training-based methods show promise, their agentic abilities are limited by inherent characteristics of the task-specific data used during training. To further enhance the universal search capability of agents, we propose a novel pre-training framework, MaskSearch. In the pre-training stage, we introduce the Retrieval Augmented Mask Prediction (RAMP) task, where the model learns to leverage search tools to fill masked spans on a large number of pre-training data, thus acquiring universal retrieval and reasoning capabilities for LLMs. After that, the model is trained on downstream tasks to achieve further improvement. We apply both Supervised Fine-tuning (SFT) and Reinforcement Learning (RL) for training. For SFT, we combine agent-based and distillation-based methods to generate training data, starting with a multi-agent system consisting of a planner, rewriter, observer, and followed by a self-evolving teacher model. While for RL, we employ DAPO as the training framework and adopt a hybrid reward system consisting of answer rewards and format rewards. Additionally, we introduce a curriculum learning approach that allows the model to learn progressively from easier to more challenging instances based on the number of masked spans. We evaluate the effectiveness of our framework in the scenario of open-domain multi-hop question answering. Through extensive experiments, we demonstrate that MaskSearch significantly enhances the performance of LLM-based search agents on both in-domain and out-of-domain downstream tasks.
Figures
Figures from the paper (2 more)
Forward citations
Cited by 4 Pith papers
-
Fetch-then-Explore: Decoupling Selection from Extraction over a Persistent Workspace for Search Agents
Fetch-then-Explore, which stores fetched pages in a per-question workspace and extracts evidence on demand with grep/read, beats visit-and-read and browsing baselines on BrowseComp across three LLM backbones.
-
Reasoning and Tool-use Compete in Agentic RL:From Quantifying Interference to Disentangled Tuning
Jointly optimizing reasoning and tool use in agentic RL interferes with both; separating the two into disjoint LoRA adapters (DART) improves retrieval-augmented QA.
-
ParallelSearch: Train your LLMs to Decompose Query and Search Sub-queries in Parallel with Reinforcement Learning
ParallelSearch applies reinforcement learning with parallel-decomposition rewards to teach LLM search agents to issue independent sub-queries concurrently, lifting QA accuracy by 2.9% on average and 12.7% on paralleli...
-
WebShaper: Agentically Data Synthesizing via Information-Seeking Formalization
WebShaper formalizes web information-seeking tasks as set-theoretic queries, synthesizes training questions by layer-wise expansion, and uses the resulting data to train open-source agents that outperform prior open-s...
Reference graph
Works this paper leans on
-
[1]
Qwen2.5 technical report.arXiv preprint arXiv:2501.15383, 2025
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2.5 technical report.arXiv preprint arXiv:2501.15383, 2025. URL https://arxiv.org/abs/2412.15115
arXiv 2025
-
[2]
DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Jun-Mei Song, Ruoyu Zhang, Runxin Xu, Qi- hao Zhu, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.ArXiv, abs/2501.12948, 2025. URLhttps://api.semanticscholar.org/CorpusID:275789950
arXiv 2025
-
[3]
The llama 3 herd of models, 2024
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models, 2024. URL https://arxiv.org/abs/2407.21783
arXiv 2024
-
[4]
Large language models: A survey.ArXiv, abs/2402.06196, 2024
Shervin Minaee, Tomás Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. Large language models: A survey.ArXiv, abs/2402.06196, 2024. URL https://api. semanticscholar.org/CorpusID:267617032
arXiv 2024
-
[5]
How do large language models acquire factual knowledge during pretraining?, 2024
Hoyeon Chang, Jinho Park, Seonghyeon Ye, Sohee Yang, Youngkyung Seo, Du-Seong Chang, and Minjoon Seo. How do large language models acquire factual knowledge during pretraining?, 2024. URL https://arxiv.org/abs/2406.11813
arXiv 2024
-
[6]
Weiqi Wu, Chengyue Jiang, Yong Jiang, Pengjun Xie, and Kewei Tu. Do PLMs know and understand ontological knowledge? In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors,Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers), pages 3080–3101, Toronto, Canada, July 2023. Association for Co...
doi:10.18653/v1/2023 2023
-
[7]
Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H. Miller, and Sebastian Riedel. Language models as knowledge bases?ArXiv, abs/1909.01066, 2019. URL https: //api.semanticscholar.org/CorpusID:202539551
arXiv 1909
-
[8]
A survey on rag meets llms: Towards retrieval-augmented large language models.ArXiv, abs/2405.06211,
Yujuan Ding, Wenqi Fan, Liang bo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. A survey on rag meets llms: Towards retrieval-augmented large language models.ArXiv, abs/2405.06211,
Show all 110 references
-
[9]
Retrieval- augmented generation for knowledge-intensive nlp tasks.ArXiv, abs/2005.11401, 2020
Patrick Lewis, Ethan Perez, Aleksandara Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kuttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval- augmented generation for knowledge-intensive nlp tasks.ArXiv, abs/2005.1140...
2005 arXiv
-
[10]
In-context retrieval-augmented language models, 2023
Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. In-context retrieval-augmented language models, 2023. URL https://arxiv.org/abs/2302. 00083
2023
-
[11]
Retrieval augmented language model pre-training
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. Retrieval augmented language model pre-training. In Hal Daumé III and Aarti Singh, editors,Proceedings of the 37th International Conference on Machine Learning, volume 119 ofProceedings of Machine Learning...
2020
-
[12]
Search-o1: Agentic search-enhanced large reasoning models, 2025
Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-o1: Agentic search-enhanced large reasoning models, 2025. URL https://arxiv.org/abs/ 2501.05366
2025 arXiv
-
[13]
Search-r1: Training llms to reason and leverage search engines with reinforcement learning, 2025
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning, 2025. URL https://arxiv.org/abs/2503.09516
2025 arXiv
-
[14]
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio, editors,Proceedings of the 2019 Conference of the North American Chapter of ...
2019 doi
-
[15]
Dapo: An open-source llm reinforcement learning system at scale, 2025
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source llm reinforcement learning system at scale, 2025. URL https://arxiv.org/abs/2503.14476. 10
2025 arXiv
-
[16]
Corrective retrieval augmented generation.ArXiv, abs/2401.15884, 2024
Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, and Zhen-Hua Ling. Corrective retrieval augmented generation.ArXiv, abs/2401.15884, 2024. URLhttps://api.semanticscholar.org/CorpusID:267312595
2024 arXiv
-
[17]
Retrieval-augmented generation for large language models: A survey.ArXiv, abs/2312.10997, 2023
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Qianyu Guo, Meng Wang, and Haofen Wang. Retrieval-augmented generation for large language models: A survey.ArXiv, abs/2312.10997, 2023. URLhttps://api.semanticscholar.org/CorpusID:266359151
2023 arXiv
-
[18]
Patil, Naman Jain, Sheng Shen, Matei A
Tianjun Zhang, Shishir G. Patil, Naman Jain, Sheng Shen, Matei A. Zaharia, Ion Stoica, and Joseph Gonzalez. Raft: Adapting language model to domain specific rag.ArXiv, abs/2403.10131, 2024. URL https://api.semanticscholar.org/CorpusID:268510197
2024 arXiv
-
[19]
Augmented non-hallucinating large lan- guage models as medical information curators.NPJ Digital Medicine, 7, 2024
Stephen Gilbert, Jakob Nikolas Kather, and Aidan Hogan. Augmented non-hallucinating large lan- guage models as medical information curators.NPJ Digital Medicine, 7, 2024. URL https://api. semanticscholar.org/CorpusID:269327466
2024
-
[20]
Enhancing llm factual accuracy with rag to counter hallucinations: A case study on domain-specific queries in private knowledge-bases, 2024
Jiarui Li, Ye Yuan, and Zehua Zhang. Enhancing llm factual accuracy with rag to counter hallucinations: A case study on domain-specific queries in private knowledge-bases, 2024. URL https://arxiv.org/abs/ 2403.10446
2024 arXiv
-
[21]
A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.ArXiv, abs/2311.05232, 2023
Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.ArXiv, abs/2311.05232, 2...
2023 arXiv
-
[22]
Agentic retrieval-augmented generation: A survey on agentic rag, 2025
Aditi Singh, Abul Ehtesham, Saket Kumar, and Tala Talaei Khoei. Agentic retrieval-augmented generation: A survey on agentic rag, 2025. URLhttps://arxiv.org/abs/2501.09136
2025 arXiv
-
[23]
Agentic retrieval-augmented generation for time series analysis, 2024
Chidaksh Ravuru, Sagar Srinivas Sakhinana, and Venkataramana Runkana. Agentic retrieval-augmented generation for time series analysis, 2024. URLhttps://arxiv.org/abs/2408.14484
2024 arXiv
-
[24]
Golden-retriever: High-fidelity agentic retrieval augmented generation for industrial knowledge base, 2024
Zhiyu An, Xianzhong Ding, Yen-Chun Fu, Cheng-Chung Chu, Yan Li, and Wan Du. Golden-retriever: High-fidelity agentic retrieval augmented generation for industrial knowledge base, 2024. URL https:// arxiv.org/abs/2408.00798
2024 arXiv
-
[25]
Vidorag: Visual document retrieval-augmented generation via dynamic iterative reasoning agents, 2025
Qiuchen Wang, Ruixue Ding, Zehui Chen, Weiqi Wu, Shihang Wang, Pengjun Xie, and Feng Zhao. Vidorag: Visual document retrieval-augmented generation via dynamic iterative reasoning agents, 2025. URL https://arxiv.org/abs/2502.18017
2025 arXiv
-
[26]
Pasa: An llm agent for comprehensive academic paper search, 2025
Yichen He, Guanhua Huang, Peiyuan Feng, Yuan Lin, Yuchen Zhang, Hang Li, and Weinan E. Pasa: An llm agent for comprehensive academic paper search, 2025. URLhttps://arxiv.org/abs/2501.10120
2025 arXiv
-
[27]
Open deep search: Democratizing search with open-source reasoning agents, 2025
Salaheddin Alzubi, Creston Brooks, Purva Chiniya, Edoardo Contente, Chiara von Gerlach, Lucas Irwin, Yihan Jiang, Arda Kaz, Windsor Nguyen, Sewoong Oh, Himanshu Tyagi, and Pramod Viswanath. Open deep search: Democratizing search with open-source reasoning agents, 2025. URL htt...
2025 arXiv
-
[28]
A real-world webagent with planning, long context understanding, and program synthesis.ArXiv, abs/2307.12856, 2023
Izzeddin Gur, Hiroki Furuta, Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, and Aleksandra Faust. A real-world webagent with planning, long context understanding, and program synthesis.ArXiv, abs/2307.12856, 2023. URLhttps://api.semanticscholar.org/CorpusID:260126067
2023 arXiv
-
[29]
Codeagent: Enhancing code generation with tool-integrated agent systems for real-world repo-level coding challenges, 2024
Kechi Zhang, Jia Li, Ge Li, Xianjie Shi, and Zhi Jin. Codeagent: Enhancing code generation with tool-integrated agent systems for real-world repo-level coding challenges, 2024
2024
-
[30]
Robust planning with llm-modulo framework: Case study in travel planning, 2024
Atharva Gundawar, Mudit Verma, Lin Guan, Karthik Valmeekam, Siddhant Bhambri, and Subbarao Kambhampati. Robust planning with llm-modulo framework: Case study in travel planning, 2024. URL https://arxiv.org/abs/2405.20625
2024 arXiv
-
[31]
Reflexion: Language agents with verbal reinforcement learning.Advances in Neural Information Processing Systems, 36, 2024
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning.Advances in Neural Information Processing Systems, 36, 2024
2024
-
[32]
O’Brien, Carrie J
Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. Generative agents: Interactive simulacra of human behavior, 2023. URL https://arxiv.org/ abs/2304.03442
2023 arXiv
-
[33]
ReAct: Synergizing reasoning and acting in language models
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. InInternational Conference on Learning Representations (ICLR), 2023. 11
2023
-
[34]
AutoAgent: A Fully-Automated and Zero-Code Framework for LLM Agents, 2025
Chao Huang Jiabin Tang, Tianyu Fan. AutoAgent: A Fully-Automated and Zero-Code Framework for LLM Agents, 2025. URLhttps://arxiv.org/abs/2502.05957
2025
-
[35]
Agent q: Advanced reasoning and learning for autonomous ai agents.arXiv preprint arXiv:2408.07199, 2024
Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov. Agent q: Advanced reasoning and learning for autonomous ai agents.arXiv preprint arXiv:2408.07199, 2024
2024 arXiv
-
[36]
Agile: A novel framework of llm agents.arXiv preprint arXiv:2405.14751, 2024
Peiyuan Feng, Yichen He, Guanhua Huang, Yuan Lin, Hanchong Zhang, Yuchen Zhang, and Hang Li. Agile: A novel framework of llm agents.arXiv preprint arXiv:2405.14751, 2024
2024 arXiv
-
[37]
Start: Self-taught reasoner with tools, 2025
Chengpeng Li, Mingfeng Xue, Zhenru Zhang, Jiaxi Yang, Beichen Zhang, Xiang Wang, Bowen Yu, Binyuan Hui, Junyang Lin, and Dayiheng Liu. Start: Self-taught reasoner with tools, 2025. URL https: //arxiv.org/abs/2503.04625
2025 arXiv
-
[38]
Openai o1 system card, 2024
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card, 2024. URL https://arxiv.org/ abs/2412.16720
2024 arXiv
-
[39]
s1: Simple test-time scaling, 2025
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling, 2025. URLhttps://arxiv.org/abs/2501.19393
2025 arXiv
-
[40]
Scaling llm test-time compute optimally can be more effective than scaling model parameters.ArXiv, abs/2408.03314, 2024
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters.ArXiv, abs/2408.03314, 2024. URL https://api. semanticscholar.org/CorpusID:271719990
2024 arXiv
-
[41]
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors,Advances in Ne...
2022
-
[42]
Griffiths, Yuan Cao, and Karthik Narasimhan
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models, 2023. URL https://arxiv.org/ abs/2305.10601
2023 arXiv
-
[43]
Question-analysis prompting improves llm performance in reasoning tasks, 2024
Dharunish Yugeswardeenoo, Kevin Zhu, and Sean O’Brien. Question-analysis prompting improves llm performance in reasoning tasks, 2024. URLhttps://arxiv.org/abs/2407.03624
2024 arXiv
-
[44]
Complexity-based prompting for multi-step reasoning, 2023
Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark, and Tushar Khot. Complexity-based prompting for multi-step reasoning, 2023. URLhttps://arxiv.org/abs/2210.00720
2023 arXiv
-
[45]
Tenenbaum, Antonio Torralba, Shuang Li, and Igor Mordatch
Vighnesh Subramaniam, Yilun Du, Joshua B. Tenenbaum, Antonio Torralba, Shuang Li, and Igor Mordatch. Multiagent finetuning: Self improvement with diverse reasoning chains, 2025. URL https://arxiv.org/ abs/2501.05707
2025 arXiv
-
[46]
Ra-dit: Retrieval-augmented dual instruction tuning, 2024
Xi Victoria Lin, Xilun Chen, Mingda Chen, Weijia Shi, Maria Lomeli, Rich James, Pedro Rodriguez, Jacob Kahn, Gergely Szilvasy, Mike Lewis, Luke Zettlemoyer, and Scott Yih. Ra-dit: Retrieval-augmented dual instruction tuning, 2024. URLhttps://arxiv.org/abs/2310.01352
2024 arXiv
-
[47]
Orca: Progressive learning from complex explanation traces of gpt-4, 2023
Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. Orca: Progressive learning from complex explanation traces of gpt-4, 2023. URL https://arxiv. org/abs/2306.02707
2023 arXiv
-
[48]
Towards reasoning ability of small language models,
Gaurav Srivastava, Shuxiang Cao, and Xuan Wang. Towards reasoning ability of small language models,
-
[49]
Distilling reasoning capabilities into smaller language models
Kumar Shridhar, Alessandro Stolfo, and Mrinmaya Sachan. Distilling reasoning capabilities into smaller language models. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors,Findings of the Association for Computational Linguistics: ACL 2023, pages 7059–7073, Toronto...
2023 doi
-
[50]
Specializing smaller language models towards multi-step reasoning, 2023
Yao Fu, Hao Peng, Litu Ou, Ashish Sabharwal, and Tushar Khot. Specializing smaller language models towards multi-step reasoning, 2023. URLhttps://arxiv.org/abs/2301.12726
2023 arXiv
-
[51]
Cole, Aditi Chaudhary, Bhuwan Dhingra, and Partha Talukdar
Jeremy R. Cole, Aditi Chaudhary, Bhuwan Dhingra, and Partha Talukdar. Salient span masking for temporal understanding, 2023. URLhttps://arxiv.org/abs/2303.12860. 12
2023 arXiv
-
[52]
Cohen, Ruslan Salakhutdinov, and Christopher D
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018
2018
-
[53]
FanOutQA: A multi-hop, multi- document question answering benchmark for large language models
Andrew Zhu, Alyssa Hwang, Liam Dugan, and Chris Callison-Burch. FanOutQA: A multi-hop, multi- document question answering benchmark for large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association fo...
2024 doi
-
[54]
MuSiQue: Multihop questions via single-hop question composition.Transactions of the Association for Computational Linguistics, 2022
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. MuSiQue: Multihop questions via single-hop question composition.Transactions of the Association for Computational Linguistics, 2022
2022
-
[55]
Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps. InProceedings of the 28th International Conference on Computational Linguistics, pages 6609–6625, Barcelona, Spain (Online), De...
2020
-
[56]
Measuring and narrowing the compositionality gap in language models
Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah Smith, and Mike Lewis. Measuring and narrowing the compositionality gap in language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Findings of the Association for Computational Linguistics: EMNLP 2023, page...
2023 doi
-
[57]
Freshllms: Refreshing large language models with search engine augmentation, 2023
Tu Vu, Mohit Iyyer, Xuezhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny Zhou, Quoc Le, and Thang Luong. Freshllms: Refreshing large language models with search engine augmentation, 2023. URLhttps://arxiv.org/abs/2310.03214
2023 arXiv
-
[58]
Generalization through memorization: Nearest neighbor language models, 2020
Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Generalization through memorization: Nearest neighbor language models, 2020. URLhttps://arxiv.org/abs/1911.00172
2020 arXiv
-
[59]
Rae, Erich Elsen, and Laurent Sifre
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore,...
2022 arXiv
-
[60]
Atlas: Few-shot learning with retrieval augmented language models, 2022
Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi- Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. Atlas: Few-shot learning with retrieval augmented language models, 2022. URLhttps://arxiv.org/abs/2208.03299
2022 arXiv
-
[61]
<think>{Overall thought process}</think> <search>[{
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.CoRR, abs/2402.03300, 2024. doi: 10.48550/ARXIV .2402.03300. URLhttps://doi.org/10...
-
[64]
Sense and Sensibility
First, find out who directed the movie "Sense and Sensibility"
-
[65]
Investigate the educational background of the director, particularly their undergraduate school
-
[66]
Sense and Sensibility
Identify the specific institution where the director completed their undergraduate studies. Now, I will start with the first step and search for the director of the movie "Sense and Sensibility".</think> <search>[{"query": "Who is the director of the movie ’Sense and Sensibili...
-
[67]
Continued on next page 15 Table 6: Prompt for Agent-Based CoT trajectory construction
First, identify who the performer of Live and Beyond is. Continued on next page 15 Table 6: Prompt for Agent-Based CoT trajectory construction
-
[68]
Then find out the birthplace of this performer
-
[69]
Next, search for which state Knowles is from
-
[70]
query":
Finally, determine when the birth city of the performer of Live and Beyond became the capital of Knowles’ state. Now, I will start with the first step and search online to determine who the performer of Live and Beyond is.</think> <search>[{"query": "Find out who the performer...
-
[71]
Sense and Sensibility
First, determine who the director of the movie "Sense and Sensibility" is
-
[72]
Sense and Sensibility
Then, search for educational background of this director, particularly undergraduate education. Now, I’ll proceed with the first step by using online searches to identify the director of the movie "Sense and Sensibility".</think> <search>[{"query": "Find out who the director o...
1995
-
[73]
Carefully analyze the search results to determine if they resolve the previous step, and provide a summary
-
[74]
<think>{thought process}</think> <answer>{answer}</answer>
Update the execution steps and propose new steps based on the search results. You may encounter the following scenarios: a. If the current search results are sufficient to arrive at the Final Answer to the problem, provide the final answer directly. Use the format: "<think>{th...
-
[76]
Sense and Sensibility
Then, search for this director’s educational background, focusing on undergraduate education. I will now begin with step one by searching online to identify the director of the movie "Sense and Sensibility".</think> <search>[{"query": "Find out who directed the movie ’Sense an...
-
[77]
Sense and Sensibility
The movie "Sense and Sensibility" was released in 1995 and is directed by Ang Lee, a renowned director known for his work in both Eastern and Western cinema
1995
-
[78]
Crouching Tiger, Hidden Dragon
Ang Lee is recognized for directing major films such as "Crouching Tiger, Hidden Dragon" and "Life of Pi"
-
[79]
Sense and Sensibility
Ang Lee’s background in filmmaking and storytelling has earned him multiple Academy Awards and international acclaim.</information> Thought:<think>After a detailed analysis of the search results, I concluded that Ang Lee is the director of the movie "Sense and Sensibility". Th...
-
[80]
Sense and Sensibility
First, determine who directed the movie "Sense and Sensibility"
-
[81]
Sense and Sensibility
Then, search for this director’s educational background, focusing on undergraduate education. I will now begin with step one by searching online to identify the director of the movie "Sense and Sensibility".</think> <search>[{"query": "Find out who directed the movie "Sense an...
-
[82]
to receive a B.F.A
Ang Lee graduated from the National Taiwan College of Arts in 1975 and then came to the U.S. to receive a B.F.A. Degree in Theatre/Theater Direction at the University of Illinois at Urbana-Champaign, and a Masters Degree in Film Production at New York University
1975
-
[83]
After high school Lee attended the Taiwan Academy of Art, where he became interested in acting. He moved to the United States in 1978 to study theater at the University of Illinois at Urbana-Champaign.</information> Thought:<think>After a thorough analysis of the search result...
1978
-
[84]
First, I need to identify which suspense drama premiered on MBC
-
[85]
query":
Then, I will look up who directed that suspense drama. First, I will search for recent suspense dramas that premiered on MBC.</think> <search>[{ "query": "What recent suspense dramas premiered on MBC?", "intent": 1}] </search> Observation:<information>1. Premiering soon! A 12-...
-
[86]
Chen Weiting’s suspense drama is airing, featuring time travel and criminal investigation, with top actors like award-winning actors joining."
-
[87]
Though the series is framed as a historical drama, it unfolds as a large-scale suspense espionage drama
The best 10 suspense dramas of the past 4 years, with ’The Big Squad’ at the bottom. Though the series is framed as a historical drama, it unfolds as a large-scale suspense espionage drama
-
[88]
However, recent suspense dramas such as ’Dim Flames’ and ’Rebirth’ have not been well-received
Premiering soon! An 18-episode suspense drama is coming, starring Sun Li, with a strong cast. However, recent suspense dramas such as ’Dim Flames’ and ’Rebirth’ have not been well-received
-
[89]
The story is a suspenseful crime drama about a vulnerable lawyer who is framed and battles powerful forces
Lee Jong-suk’s return! New drama ’The Black Lawyer’ premiered on MBC, achieving high viewership. The story is a suspenseful crime drama about a vulnerable lawyer who is framed and battles powerful forces
-
[90]
The top 10 suspense dramas to watch in 2024, featuring leading actors like Liao Fan, Duan Yihong, and Zhang Songwen
2024
-
[91]
Premiering tonight! Another suspenseful drama is coming! ’Thirteen Years in Dust’ will air on iQIYI’s Mist Theater, starring Chen Jianbin and Chen Xiao
-
[92]
</information> Thought:<think>From the search results, I found several relevant pieces of information:
Chen Weiting’s new drama ’The Sun and Stars’ debuted with impressive suspenseful plotlines. </information> Thought:<think>From the search results, I found several relevant pieces of information:
-
[93]
The suspense drama ’The Black Lawyer’ premiered on MBC, starring Lee Jong-suk and Im Yoon-ah, and it is a crime suspense drama
-
[94]
The premiere of ’The Black Lawyer’ achieved high ratings, telling a story about a vulnerable lawyer who is framed and fights back against powerful figures
-
[95]
query":
However, the search results did not provide the director’s information for ’The Black Lawyer.’ Filter out unrelated content: The search results also mention suspense dramas on other platforms like iQIYI and Mist Theater, and other non-MBC dramas, such as ’The Sun and Stars.’ T...
-
[96]
is an advanced RL algorithm that enhances the Group Relative Policy Optimization (GRPO) [61] by incorporating techniques such as dynamic sampling and token-level policy gradient loss. DAPO samples a group of outputs {oi}G i=1 for each question q paired with the answer a, and o...
1975
-
[97]
First, search for David Hoelscher’s brief NFL career to determine the specific league and team he played for in 1998
1998
-
[98]
query":
Then confirm the name of the college where he played football before entering the NFL. Now, I will start with the first step and search for David Hoelscher’s NFL career details. </think> <search>[{"query": "David Hoelscher one game NFL" "David Henry Hoelscher NFL team 1998" "D...
1998
-
[99]
David Hoelscher, , - Career Stats - NFL - CBSSports.com: Get the latest career stats for David Hoelscher of the on CBS Sports
-
[100]
David Hoelscher, , - 2021-22 Game Log - NFL - CBSSports.com: Get up-to-date stats for every game played by David Hoelscher during the 2021-22 NFL season on CBS Sports
2021
-
[102]
Class Freshman
David Hoelscher - 2021-22 - Esports - Florida Southern College: David Hoelscher. Class Freshman. Hometown Orchard Park, N.Y . Continued on next page 21 Table 9: RAMP Case
2021
-
[103]
query":
David Hoelscher | American Football Database - Fandom: American football defensive tackle in the National Football League for the Washington Redskins. He played college football at Eastern Kentucky University. </information> <think> After finding out that David Hoelscher is in...
1998
-
[104]
TEAM, W, L, T, PCT
David Hoelscher 1998 Game Log | StatMuse: David Hoelscher’s game log for the 1998 NFL regular season and playoffs. TEAM, W, L, T, PCT. Eagles NFL 2024 Betting. Best Record ATS · Chargers. 12-5-1
1998
-
[105]
David Hoelscher Stats, News and Video - DE | NFL.com: David Hoelscher DE Player Info Height 6-6 Weight 261 Arms Hands Experience 2 College Eastern Kentucky
-
[106]
He played but didn’t record a tackle
David Hoelscher - Washington Redskins Defensive Lineman: David Hoelscher played 1 season for the Redskins. He played but didn’t record a tackle
-
[107]
Colonels in the NFL - Eastern Kentucky University Athletics: Colonels in the NFL; David Hoelscher (1994-97), Washington Redskins (1998); Joe Hollingsworth (1946-47), Pittsburgh Steelers (1949-51); Tyrone
1998
-
[108]
query":
David Hoelscher Pro Football Stats, Position, College, Draft: DA VID HOELSCHER; 1998 Washington Redskins (NFL), 91, DT, 1, 0; 1999 Frankfurt Galaxy (NFLE), 91, RDT-RDE. </information> <think> After confirming that David Hoelscher played for the Washington Redskins in 1998, I w...
1998
-
[109]
Hoelscher, David Henry Height: 6-6 Weight: 256
David Hoelscher Pro Football Stats, Position, College, Draft: DA VID HOELSCHER. Hoelscher, David Henry Height: 6-6 Weight: 256. Born: November 27, 1975 Coldwater, OH High School: Versailles (OH). Year, College, Status. 1994
1975
-
[110]
HOELSCHER; 1994, Eastern Kentucky, Roster; 1995, Eastern Kentucky, Roster; 1996, Eastern Kentucky, Roster; 1997, Eastern Kentucky, Played
1994
-
[111]
Class Freshman
David Hoelscher - 2021-22 - Esports - Florida Southern College: David Hoelscher. Class Freshman. Hometown Orchard Park, N.Y .. Prev School Athletics. 111 Lake Hollingsworth Drive | Lakeland FL 33801 | 863-680-4245. 4.David Hoelscher Stats, Height, Weight, Position, Draft, Coll...
2021
-
[112]
Stub icon, This biographical article relating to an American football defensive lineman born in the
David Hoelscher | American Football Database - Fandom: He played college football at Eastern Kentucky University. Stub icon, This biographical article relating to an American football defensive lineman born in the. </information> <think> After verifying the college where David...
1975
-
[2024]
URLhttps://api.semanticscholar.org/CorpusID:276185331
-
[2025]
URLhttps://arxiv.org/abs/2502.11569
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.