Pith. sign in

REVIEW 5 major objections 4 minor 7 cited by

Long-horizon web agents fail mainly because their context fills with noise; separating search from browse and periodically summarizing fixes it.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 08:44 UTC pith:XD6K4VKJ

load-bearing objection A genuinely simple and well-tested framework for long-horizon agentic search, but the headline 'beats all open-source frameworks' claim is overreach until the closest summarization-based baseline is actually run. the 5 major comments →

arxiv 2510.18939 v2 pith:XD6K4VKJ submitted 2025-10-21 cs.CL

Lost in the Maze: Overcoming Context Limitations in Long-Horizon Agentic Search

Howard Yen , Yoonsang Lee , Ashwin Paranjape , Mengzhou Xia , Thejas Venkatesh , Jack Hessel , Danqi Chen , Yuhao Zhang This is my paper
classification cs.CL
keywords SLIMagentic searchlong-horizon web searchcontext managementtrajectory summarizationBrowseCompHLEhallucination analysis
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Long-horizon web search agents often fail not because the underlying model is weak, but because their context windows accumulate noisy, irrelevant content, forcing them to hit limits or stop early. The paper proposes SLIM, a minimal framework with three components: a search tool that returns only titles, URLs, and snippets; a browse tool that fetches only the most relevant section of a page; and a summarizer that periodically compresses the whole trajectory. On BrowseComp and HLE, SLIM with the o3 model reaches 56% and 31% accuracy respectively, beating the open-source frameworks it compared against by 8 and 4 points while using roughly a quarter of the tool calls. A trajectory-level error analysis attributes the gain mainly to reduced hallucination. If correct, the paper shows that architecture-level context management, not more complex agent orchestration, is the key to scaling agentic search.

Core claim

SLIM's central claim is that poor context management is the primary bottleneck in long-horizon agentic search. Existing frameworks either dump all scraped content into the model's context, summarize search results but still scrape every link, or orchestrate many specialized agents; all three patterns fill context with noise, exhaust budgets, or stop early. SLIM instead gives the model a search tool that returns only lightweight snippets and a browse tool that returns just the section of a page most relevant to the current query, then compresses the entire conversation every n turns. With o3 as the base model, this design achieves 56% on BrowseComp and 31% on HLE, 8 and 4 points above the bes

What carries the argument

The central mechanism is the separation of retrieval into three cooperating components: a search tool that returns only the top-k titles, URLs, and short snippets; a browse tool that scrapes a chosen URL and returns only the section whose content best matches the current query; and a summarization module that compresses the entire conversation history every n turns. The search tool keeps initial results light, the browse tool lets the model select which pages deserve deeper inspection, and the periodic trajectory summary replaces the growing noisy history with a compact status report. Together these components let the agent run longer trajectories without overflowing the context window, and

Load-bearing premise

The claim that SLIM outperforms all open-source frameworks rests on the four baselines selected being strong and faithfully implemented, and on a cost model that excludes cached input tokens and weights output tokens 4x; if either assumption fails, the reported margins and cost advantage shrink.

What would settle it

Run a controlled ablation: add the same periodic trajectory summarization to the best-performing baseline that already summarizes per result, keeping its search step unchanged. If it reaches SLIM's accuracy with comparable tool calls, the claim that SLIM's search/browse separation is the key mechanism fails; alternatively, if a pricing model that counts cached input tokens at full price reverses the cost ordering, the efficiency claim is not robust.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If SLIM's account is right, scaling a search agent's tool budget directly buys accuracy: on BrowseComp, going from 10 turns to 150 turns raises accuracy from 17.7% to 56.0% without overflowing the context window.
  • The tool-call savings are structural and model-general: SLIM uses 15-25% of the tool calls across three different base models, making cheaper and smaller models more viable for deep-research workloads.
  • The error taxonomy gives a concrete next target: roughly 30% of incorrect SLIM trajectories contain the groundtruth answer but fail to use it, so teaching agents to re-read and verify their own search histories should yield further gains.
  • Because the framework is only search plus browse plus summarize, it can be combined with stronger base models or RL-trained agents as drop-in components rather than requiring bespoke multi-agent orchestration.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the paper's cost metric excludes cached input tokens and weights output tokens 4x; under a deployment that pays full price for all tokens, the efficiency gap could narrow, so the reported 4-6x cost advantage is conditional on that accounting.
  • Editorial inference: the fixed 'summarize every n turns' schedule is a crude proxy for context pressure; an adaptive trigger that summarizes when the trajectory's information gain per token drops is a natural, untested extension.
  • Editorial inference: the browse tool's relevance scoring is itself a retrieval step, so swapping ROUGE-L paragraph scoring for a learned reranker could improve recall of hard-to-find answers while preserving the architecture.
  • Editorial inference: the high 'answer ignored' rate suggests a simple final verification or answer-extraction pass over the summarized history could close much of the remaining gap, and the released trajectory logs make this easy to test.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper studies why open-source agentic search frameworks degrade on long-horizon tasks, attributes the primary failure modes to poor context management (context-window overflow, tool-budget exhaustion, and early stopping), and proposes SLIM, a single-agent framework with separate search and browse tools plus periodic trajectory summarization. SLIM is evaluated on 300-instance subsets of BrowseComp and HLE with o3, o4-mini, and Claude-4-Sonnet as base models, and compared against ReAct, Search-o1, HF-ODR, and GPT-Researcher. The authors report that SLIM reaches 56.0 on BrowseComp and about 31 points on HLE with o3 while using substantially fewer tool calls and lower cost than the compared baselines. The paper also introduces an automated trajectory-level error taxonomy and reports that SLIM hallucinates less than the baselines.

Significance. If the reported comparisons are accurate, the paper provides a valuable and simple design pattern—separating search from browsing and periodically compressing the trajectory—that improves long-horizon agentic search efficiency, and it releases code and outputs for reproducibility. The evaluation across three base models and two benchmarks, with ablations of the main design choices, is a genuine strength. However, the headline claim that SLIM outperforms 'all open-source frameworks' is not yet established: the closest prior open-source system based on context summarization, ReSum, is never evaluated, and the numerical margins shift substantially depending on which baseline configuration is used. The paper's central contribution is therefore plausible but needs a corrected, more carefully scoped comparison before the claims as stated can be accepted.

major comments (5)
  1. [Abstract, §5, Table 3] The headline HLE numbers are internally inconsistent. The arXiv metadata abstract states 33% and a 6-point margin; the full-text abstract states 31% and 4 points; §5 says '8 and 4 points'; Table 3 gives SLIM HLE scores of 31.3 at T=100 and 30.7 at T=150, versus 27.0 for the best Search-o1 setting. The claimed 6-point margin is not supported anywhere in the table, and even the 4-point margin holds only against the T=50 Search-o1 row. These numbers must be reconciled.
  2. [§5, Table 3] The claimed 8-point BrowseComp advantage is an artifact of cost-matched baseline selection. SLIM at T=150 (56.0, $1.24) is compared to Search-o1 at T=50 (48.3, $1.27), but Search-o1 at T=100 reaches 55.7 ($2.23)—a 0.3-point gap—and Search-o1 at T=50 is not the strongest configuration in the table. The claim 'outperforms all open-source frameworks by 8 points' is therefore misleading; it should be explicitly framed as a cost-matched comparison against a specific baseline configuration, or the strongest baseline configuration should be used for the headline margin.
  3. [§7, References] ReSum (Wu et al., 2025b) is an open-source framework whose core component is context summarization, the same mechanism credited for SLIM's gains, yet it is listed in related work but never run as a baseline. Without ReSum in Tables 3/13/14, the claim of outperforming 'all open-source frameworks' is unsupported. The authors should either add ReSum as a baseline or restrict the claim to the four selected frameworks. The current citation placement in the reinforcement-learning paragraph also misdescribes the work.
  4. [§6, §A.3] The trajectory-level error taxonomy is presented as a contribution, but the LLM-as-a-judge detectors are not validated against human annotation. The paper states that frontier LLMs are 'powerful enough to reliably check' simple yes/no questions, but no agreement, precision/recall, or calibration numbers are reported for confirmation-bias, unfocused-search, answer-ignored, abstention, or hallucination detectors. Since the fine-grained analysis is a stated contribution, some form of validation is needed to support the conclusions drawn from it (e.g., the hallucination-rate comparison in Table 4).
  5. [§5, §A.5] The main results appear to be based on a single run per configuration. Given the stochasticity of LLM agents and the small 0.3-point gap between SLIM at T=150 and Search-o1 at T=100 on BrowseComp, confidence intervals or repeated runs are needed to assess whether the reported differences are meaningful. The released outputs are helpful, but the paper should report variance (e.g., bootstrap over instances or multiple seeds) for the primary comparisons.
minor comments (4)
  1. [Table 3] Several SLIM HLE rows are typeset without column separators (e.g., '31.37.7', '31.318.4'), making the scores and token counts ambiguous. The table formatting should be fixed.
  2. [Abstract] The arXiv metadata version of the abstract includes a GLM-4.7 Flash result ('10 points improvement ... using a third of the cost') that does not appear in the full-text abstract or in the main results. The manuscript should be internally consistent: either add the corresponding experiment or remove the sentence.
  3. [Figure 1 caption] The phrase 'more than 4-6x fewer tool calls' is redundant; '4-6x fewer tool calls' is sufficient.
  4. [§7] ReSum is cited in the reinforcement-learning paragraph, but its title and contribution are about context summarization rather than RL training. The citation should be moved and described accurately.

Circularity Check

0 steps flagged

No significant circularity: SLIM is an empirical framework design evaluated on external benchmarks; no load-bearing derivation reduces to its own inputs.

full rationale

The paper's central claim is an empirical comparison: SLIM is introduced as a simple framework with search, browse, and summarization, and evaluated against baselines on BrowseComp and HLE. There is no chain of equations in which an output is defined as the input or a fitted parameter is renamed a prediction. The failure-mode analysis in Section 3 motivates design choices, but the design is not derived from the outcome; the trajectory taxonomy in Section 6 was built by manual inspection and is used to characterize, not to fit, SLIM's headline accuracy. Hyperparameters (n, k, L, chunking) are selected through reported ablations rather than fit to the headline result. The only self-citation (Yen et al., 2025) supports the reliability of the LLM-as-judge hallucination checks in the appendix; it is not load-bearing for the main accuracy/cost comparison. Concerns about baseline selection (ReSum cited but not run; Search-o1 at T=100 nearly matching SLIM on BrowseComp) and inconsistent reported HLE numbers are substantive correctness or evaluation-coverage risks, but they are not circularity: nothing in the paper's own derivation forces these numbers by construction. The paper is self-contained against external benchmarks, so the appropriate circularity finding is 0.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

SLIM is a hand-designed framework rather than a formal derivation. Its load-bearing choices are hyperparameters tuned on small validation subsets; the paper's comparisons also rely on assumptions about faithful baselines and a specific cost model. No new physical or formal entities are introduced.

free parameters (5)
  • summarization interval n = 50 turns
    Default interval chosen after ablations on 50-sample subsets; ablations show sensitivity (n=25 gives 40 vs 50 on BrowseComp).
  • search result count k = 10
    Default top-k results; ablation with k=20 and L=3000 gives 42 vs 50 on BrowseComp, so performance varies with k.
  • browse content length L = 10000 characters
    Maximum scraped content length; ablation L=3000 yields 54 vs 50 on BrowseComp, indicating a tuned value.
  • chunking/similarity metric = newline chunks + ROUGE-L
    Ablation with word chunks + BM25 yields 52 vs 50 on BrowseComp, so the choice matters somewhat.
  • output token weighting in cost model = 4x output token cost
    Token cost is weighted as non-cached input + 4*output; this weighting affects cost comparisons and is a modeling choice, not an empirical fit.
axioms (5)
  • domain assumption Search engine snippets (title, URL, short snippet) are sufficient for the agent to select relevant pages without full content.
    Core to SLIM's search tool; if snippets are misleading, browse selectivity fails (Sec. 4).
  • domain assumption ROUGE-L/(BM25) similarity between query and chunks identifies the most relevant passage.
    Basis of browse tool B(u,q); no guarantee of retrieval quality (Sec. 4, Alg. 5).
  • domain assumption LLM-as-a-judge and rule-based heuristics can reliably annotate failure modes.
    Used to measure hallucination, abstention, etc., without human agreement studies (Sec. A.3).
  • domain assumption Subsets of 300 instances are representative of BrowseComp and HLE.
    Evaluation is on random subsets due to cost; full benchmark statistics may differ (Sec. 2.2).
  • domain assumption Cost model excluding cached input tokens is a fair basis for comparing systems.
    Stated assumption that production systems use caching (Sec. 5).

pith-pipeline@v1.3.0-alltime-deepseek · 28676 in / 12957 out tokens · 101061 ms · 2026-08-04T08:44:49.475509+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Lost in the Maze: Overcoming Context Limitations in Long-Horizon Agentic Search." pith.science (2026). https://pith.science/paper/XD6K4VKJ

@misc{pith2026251018939,
  author       = {Pith},
  title        = {Pith review of: Lost in the Maze: Overcoming Context Limitations in Long-Horizon Agentic Search},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XD6K4VKJ}},
  note         = {Machine review of arXiv:2510.18939}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Long-horizon agentic search requires iteratively exploring the web over long trajectories and synthesizing information across many sources, enabling powerful applications like deep research systems. In this work, we show that popular agentic search frameworks struggle to scale to long trajectories primarily due to context limitations--they accumulate long, noisy content, hit context window and tool budgets, or stop early. We therefore introduce SLIM (Simple Lightweight Information Management), a simple framework that separates retrieval into distinct search and browse tools, and periodically summarizes the trajectory, keeping context concise while enabling longer, more focused searches. Across a wide range of long-horizon tasks, SLIM achieves comparable performance at substantially lower cost and far fewer tool calls than strong open-source frameworks with both proprietary and open-weight models, including RL-trained models for deep research. Specifically, with o3 as the base model, SLIM achieves 56% on BrowseComp and 33% on HLE, outperforming all open-source frameworks by 8 and 6 absolute points, respectively, while incurring 4-6x fewer tool calls. With GLM-4.7 Flash, SLIM achieves 10 points improvement over the next best open-source framework, Search-o1, on BrowseComp using a third of the cost. To systematically understand failure modes in long-horizon agentic search, we develop an automated fine-grained trajectory analysis pipeline and error taxonomy, and find that SLIM exhibits significantly fewer hallucinations than prior systems. We hope our analysis framework and simple tool design inform future long-horizon agents.

Figures

Figures reproduced from arXiv: 2510.18939 by Ashwin Paranjape, Danqi Chen, Howard Yen, Jack Hessel, Mengzhou Xia, Thejas Venkatesh, Yoonsang Lee, Yuhao Zhang.

Figure 1
Figure 1. Figure 1: With o3 as the base model, SLIM achieves better performance than existing frameworks on both BrowseComp and HLE while using more than 4-6x fewer tool calls and lower overall costs, which account for LLM token usage and tool costs. powerful components—search, browse, and summarization—that effectively manage the context size of long-horizon systems. The simple tool design allows LLMs to interleave searching… view at source ↗
Figure 2
Figure 2. Figure 2: Example queries and their relevant documents for BrowseComp (Wei et al., 2025) and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Each framework exhibits distinct outcome trends— [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Compared to REACT and SEARCH-O1, the cooperation between search, browse, and summarization modules allows SLIM to accumulate shorter contexts and less noisy information after exploring the same amount of searches. Browse tool B. Our browse tool is designed to complement the search tool by allowing the LLM to dig deeper into promising search results. Specifically, the browse tool B(u, q) → maxci∈c sim(ci , … view at source ↗
Figure 6
Figure 6. Figure 6: SLIM consistently achieves the highest performance across these models and all datasets compared to other frameworks, suggesting that our simple design generalizes well to models of different sizes and training strategies. Furthermore, our effective context management also results in fewer tool calls and often lower overall costs compared to the baselines. SLIM also shows consistent trends across all three… view at source ↗
Figure 5
Figure 5. Figure 5: With o4-mini as the base model, SLIM consistently outperforms other baselines on BrowseComp while using fewer tool calls and lower overall costs. On HLE, SLIM can achieve overall higher performance and use fewer tool calls. 0 1 2 3 Cost ($) 0 2 4 6 8 10 BrowseComp 0 50 100 150 Tools (Count) 0 2 4 6 8 10 0 1 2 3 Cost ($) 6 8 10 12 14 16 HLE 0 50 Tools (Count) 6 8 10 12 14 16 Base (claude-4-sonnet) SLIM (Our… view at source ↗
Figure 6
Figure 6. Figure 6: With Claude-4-Sonnet as the base model, SLIM consistently outperforms other baselines on BrowseComp while using fewer tool calls and lower overall costs. On HLE, SLIM can achieve overall higher performance and use fewer tool calls. Based on the taxonomy, we develop an automated error analysis pipeline that annotates each trajectory with the failure modes using a mix of rule-based heuristics and LLM-as-a-ju… view at source ↗
Figure 7
Figure 7. Figure 7: Examples of each trajectory-level failure mode on a BrowseComp sample. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: An example of a SLIM trajectory. Algorithm 5: SLIM Data: Task input x, LLM θ, maximum number of turns T, summary interval n Function search(q): return (titlei , urli , snippeti) k i=1; Function browse(u, q): D ← scrape(u); D ← split(D) = {di} m i=1; if q = ∅ then return d ′ ← d1; else d ′ ← arg maxdi∈D ROUGE-L(di , q); return d ′ ; Result: Task output y Turn t ← 1; Context C ← {x}; T ← {search, browse}; wh… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. LLM Agents Are Latent Context Managers: Eliciting Self-Managed Context via State Proprioception

    cs.CL 2026-06 conditional novelty 7.0

    Exposing per-block token, recency, and access metadata with lossless archive/recovery elicits latent context management in untrained LLM agents and roughly doubles LOCA-Bench success under pressure.

  2. LLM Agents Are Latent Context Managers: Eliciting Self-Managed Context via State Proprioception

    cs.CL 2026-06 conditional novelty 7.0

    A training-free, model-agnostic dashboard that exposes per-block context state (tokens, age, budget) with lossless archive/recovery improves long-horizon tool-agent performance on LOCA-Bench, BrowseComp-Plus, and GAIA.

  3. The Context Gathering Decision Process: A POMDP Framework for Agentic Search

    cs.AI 2026-05 accept novelty 7.0

    Framing LLM agent loops as a Context Gathering Decision Process POMDP yields a predicate-based belief state that boosts multi-hop reasoning up to 11.4% and an exhaustion gate that cuts token use up to 39% with no perf...

  4. LLM Agents Are Latent Context Managers: Eliciting Self-Managed Context via State Proprioception

    cs.CL 2026-06 unverdicted novelty 6.0

    VISTA supplies LLM agents with a visible proprioceptive dashboard of typed context blocks, enabling untrained self-management that lifts performance on long-horizon tool-use benchmarks across multiple model scales.

  5. LLM Agents Are Latent Context Managers: Eliciting Self-Managed Context via State Proprioception

    cs.CL 2026-06 conditional novelty 6.0

    Giving LLM agents a visible dashboard of their context blocks, plus lossless archive and recovery, sharply improves long-horizon tool-agent performance without any training.

  6. SlimSearcher: Training Efficiency-Aware Web Agents via Adaptive Reward Gating

    cs.LG 2026-06 unverdicted novelty 5.0

    SlimSearcher reduces tool-call rounds by 17-58% on GAIA, BrowseComp and XBenchDeepSearch while maintaining accuracy via Pareto filtration in SFT and Adaptive Reward Gating in RL.

  7. On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Length

    cs.AI 2026-05 unverdicted novelty 5.0

    Longer action horizons bottleneck LLM agent training through instability, but training with reduced horizons stabilizes learning and enables better generalization to longer horizons.

Reference graph

Works this paper leans on

40 extracted references · 15 linked inside Pith · cited by 4 Pith papers

  1. [1]

    Attributed question answering: Evaluation and modeling for attributed large language models

    Bernd Bohnet, Vinh Q Tran, Pat Verga, Roee Aharoni, Daniel Andor, Livio Baldini Soares, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herzig, Kai Hui, et al. Attributed question answering: Evaluation and modeling for attributed large language models. arXiv preprint arXiv:2212.08037, 2022. URL https://arxiv.org/pdf/2212.08037.pdf

    Pith/arXiv arXiv 2022

  2. [2]

    Pan, Wen Zhang, Huajun Chen, Fan Yang, Zenan Zhou, and Weipeng Chen

    Mingyang Chen, Linzhuang Sun, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z. Pan, Wen Zhang, Huajun Chen, Fan Yang, Zenan Zhou, and Weipeng Chen. Research: Learning to reason with search for llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2503.19470

    Pith/arXiv arXiv 2025

  3. [3]

    Deepresearch bench: A comprehensive benchmark for deep research agents, 2025

    Mingxuan Du, Benfeng Xu, Chiwei Zhu, Xiaorui Wang, and Zhendong Mao. Deepresearch bench: A comprehensive benchmark for deep research agents, 2025. URL https://arxiv.org/abs/2506.11763

    Pith/arXiv arXiv 2025

  4. [4]

    gpt-researcher , July 2023

    Assaf Elovic. gpt-researcher , July 2023. URL https://github.com/assafelovic/gpt-researcher

    2023

  5. [5]

    Enabling large language models to generate text with citations

    Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. Enabling large language models to generate text with citations. In Empirical Methods in Natural Language Processing (EMNLP), 2023

    2023

  6. [6]

    Gemini deep research — your personal research assistant, September 2025

    Google. Gemini deep research — your personal research assistant, September 2025. URL https://gemini.google/overview/deep-research/

    2025

  7. [7]

    Atlas: few-shot learning with retrieval augmented language models

    Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. Atlas: few-shot learning with retrieval augmented language models. J. Mach. Learn. Res., 24 0 (1), January 2023. ISSN 1532-4435

    2023

  8. [8]

    Arik, and Jiawei Han

    Bowen Jin, Jinsung Yoon, Priyanka Kargupta, Sercan O. Arik, and Jiawei Han. An empirical study on reinforcement learning for reasoning-search interleaved llm agents, 2025 a . URL https://arxiv.org/abs/2505.15117

    Pith/arXiv arXiv 2025

  9. [9]

    Search-r1: Training LLM s to reason and leverage search engines with reinforcement learning

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan O Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training LLM s to reason and leverage search engines with reinforcement learning. In Second Conference on Language Modeling, 2025 b . URL https://openreview.net/forum?id=Rwhi91ideu

    2025

  10. [10]

    T rivia QA : A large scale distantly supervised challenge dataset for reading comprehension

    Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. T rivia QA : A large scale distantly supervised challenge dataset for reading comprehension. In Regina Barzilay and Min-Yen Kan (eds.), Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\ 1601--1611, Vancouver, Canada, July 2017....

    doi:10.18653/v1/p17-1147 2017

  11. [11]

    WiCE: Real-World Entailment for Claims in Wikipedia

    Ryo Kamoi, Tanya Goyal, Juan Diego Rodriguez, and Greg Durrett. WiCE: Real-World Entailment for Claims in Wikipedia . arXiv preprint arXiv:2303.01432, 2023. URL https://arxiv.org/abs/2303.01432

    Pith/arXiv arXiv 2023

  12. [12]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural questions: A benchmark for question answering research. Transac...

    doi:10.1162/tacl_a_00276 2019

  13. [13]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\" u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\" a schel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Proceedings of the 34th International Conference on Neural Information Processing Systems,...

    2020

  14. [14]

    Websailor: Navigating super-human reasoning for web agent, 2025 a

    Kuan Li, Zhongwang Zhang, Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baixuan Li, Zhengwei Tao, Xinyu Wang, Weizhou Shen, Junkai Zhang, Dingchu Zhang, Xixi Wu, Yong Jiang, Ming Yan, Pengjun Xie, Fei Huang, and Jingren Zhou. Websailor: Navigating super-human reasoning for web agent, 2025 a . URL https://arxiv.org/abs/2507.02592

    Pith/arXiv arXiv 2025

  15. [15]

    Search-o1: Agentic search-enhanced large reasoning models, 2025 b

    Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-o1: Agentic search-enhanced large reasoning models, 2025 b . URL https://arxiv.org/abs/2501.05366

    Pith/arXiv arXiv 2025

  16. [16]

    Webthinker: Empowering large reasoning models with deep research capability

    Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yutao Zhu, Yongkang Wu, Ji - Rong Wen, and Zhicheng Dou. Webthinker: Empowering large reasoning models with deep research capability. CoRR, abs/2504.21776, 2025 c . doi:10.48550/ARXIV.2504.21776. URL https://doi.org/10.48550/arXiv.2504.21776

    Pith/arXiv arXiv doi:10.48550/arxiv.2504.21776 2025

  17. [17]

    Sfr-deepresearch: Towards effective reinforcement learning for autonomously reasoning single agents, 2025

    Xuan-Phi Nguyen, Shrey Pandit, Revanth Gangi Reddy, Austin Xu, Silvio Savarese, Caiming Xiong, and Shafiq Joty. Sfr-deepresearch: Towards effective reinforcement learning for autonomously reasoning single agents, 2025. URL https://arxiv.org/abs/2509.06283

    Pith/arXiv arXiv 2025

  18. [18]

    Introducing deep research, February 2025

    OpenAI. Introducing deep research, February 2025. URL https://openai.com/index/introducing-deep-research/

    2025

  19. [19]

    KILT : a benchmark for knowledge intensive language tasks

    Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rockt \"a schel, and Sebastian Riedel. KILT : a benchmark for knowledge intensive language tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Ass...

    2021

  20. [20]

    Wang, John-Clark Levin, Mstyslav Kazakov, Fiona Feng, Steven Y

    Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, Michael Choi, Anish Agrawal, Arnav Chopra, Adam Khoja, Ryan Kim, Richard Ren, Jason Hausenloy, Oliver Zhang, Mantas Mazeika, Dmitry Dodonov, Tung Nguyen, Jaeho Lee, Daron Anderson, Mikhail Doroshenko, Alun Cennyth Stokes, ...

    Pith/arXiv arXiv 2025

  21. [21]

    Webresearcher: Unleashing unbounded reasoning capability in long-horizon agents, 2025

    Zile Qiao, Guoxin Chen, Xuanzhong Chen, Donglei Yu, Wenbiao Yin, Xinyu Wang, Zhen Zhang, Baixuan Li, Huifeng Yin, Kuan Li, Rui Min, Minpeng Liao, Yong Jiang, Pengjun Xie, Fei Huang, and Jingren Zhou. Webresearcher: Unleashing unbounded reasoning capability in long-horizon agents, 2025. URL https://arxiv.org/abs/2509.13309

    arXiv 2025

  22. [22]

    Measuring Attribution in Natural Language Generation Models

    Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Turc, and David Reitter. Measuring Attribution in Natural Language Generation Models . Computational Linguistics, pp.\ 1--64, 08 2023. ISSN 0891-2017. doi:10.1162/coli_a_00486. URL https://doi.org/10.1162/coli\_a\_00486

    doi:10.1162/coli_a_00486 2023

  23. [23]

    The probabilistic relevance framework: Bm25 and beyond

    Stephen Robertson and Hugo Zaragoza. The probabilistic relevance framework: Bm25 and beyond. Found. Trends Inf. Retr., 3 0 (4): 0 333–389, apr 2009. ISSN 1554-0669. doi:10.1561/1500000019. URL https://doi.org/10.1561/1500000019

    doi:10.1561/1500000019 2009

  24. [24]

    Open-source DeepResearch -- Freeing our search agents, January 2025

    Aymeric Roucher, Albert Villanova del Moral, Merve Noyan, Thomas Wolf, and Clémentine Fourrier. Open-source DeepResearch -- Freeing our search agents, January 2025. URL https://huggingface.co/blog/open-deep-research

    2025

  25. [25]

    REPLUG : Retrieval-augmented black-box language models

    Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. REPLUG : Retrieval-augmented black-box language models. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Languag...

    2024

  26. [26]

    Simpledeepsearcher: Deep information seeking via web-powered reasoning trajectory synthesis, 2025

    Shuang Sun, Huatong Song, Yuhao Wang, Ruiyang Ren, Jinhao Jiang, Junjie Zhang, Fei Bai, Jia Deng, Wayne Xin Zhao, Zheng Liu, Lei Fang, Zhongyuan Wang, and Ji-Rong Wen. Simpledeepsearcher: Deep information seeking via web-powered reasoning trajectory synthesis, 2025. URL https://arxiv.org/abs/2505.16834

    arXiv 2025

  27. [27]

    Webshaper: Agentically data synthesizing via information-seeking formalization, 2025

    Zhengwei Tao, Jialong Wu, Wenbiao Yin, Junkai Zhang, Baixuan Li, Haiyang Shen, Kuan Li, Liwen Zhang, Xinyu Wang, Yong Jiang, Pengjun Xie, Fei Huang, and Jingren Zhou. Webshaper: Agentically data synthesizing via information-seeking formalization, 2025. URL https://arxiv.org/abs/2507.15061

    Pith/arXiv arXiv 2025

  28. [28]

    Executable code actions elicit better llm agents

    Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable code actions elicit better llm agents. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024

    2024

  29. [29]

    Browsecomp: A simple yet challenging benchmark for browsing agents, 2025

    Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese. Browsecomp: A simple yet challenging benchmark for browsing agents, 2025. URL https://arxiv.org/abs/2504.12516

    Pith/arXiv arXiv 2025

  30. [30]

    W eb W alker: Benchmarking LLM s in web traversal

    Jialong Wu, Wenbiao Yin, Yong Jiang, Zhenglin Wang, Zekun Xi, Runnan Fang, Linhai Zhang, Yulan He, Deyu Zhou, Pengjun Xie, and Fei Huang. W eb W alker: Benchmarking LLM s in web traversal. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Lingu...

    2025

  31. [31]

    Resum: Unlocking long-horizon search intelligence via context summarization, 2025 b

    Xixi Wu, Kuan Li, Yida Zhao, Liwen Zhang, Litu Ou, Huifeng Yin, Zhongwang Zhang, Yong Jiang, Pengjun Xie, Fei Huang, Minhao Cheng, Shuai Wang, Hong Cheng, and Jingren Zhou. Resum: Unlocking long-horizon search intelligence via context summarization, 2025 b . URL https://arxiv.org/abs/2509.13313

    arXiv 2025

  32. [32]

    Grok 3 beta — the age of reasoning agents, February 2025

    xAI. Grok 3 beta — the age of reasoning agents, February 2025. URL https://x.ai/news/grok-3

    2025

  33. [33]

    Open data synthesis for deep research, 2025

    Ziyi Xia, Kun Luo, Hongjin Qian, and Zheng Liu. Open data synthesis for deep research, 2025. URL https://arxiv.org/abs/2509.00375

    Pith/arXiv arXiv 2025

  34. [34]

    ReAct : Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct : Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023. URL https://arxiv.org/abs/2210.03629

    Pith/arXiv arXiv 2023

  35. [35]

    Helmet: How to evaluate long-context language models effectively and thoroughly

    Howard Yen, Tianyu Gao, Minmin Hou, Ke Ding, Daniel Fleischer, Peter Izsak, Moshe Wasserblat, and Danqi Chen. Helmet: How to evaluate long-context language models effectively and thoroughly. In International Conference on Learning Representations (ICLR), 2025

    2025

  36. [36]

    Deepresearcher: Scaling deep research via reinforcement learning in real-world environments, 2025

    Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, and Pengfei Liu. Deepresearcher: Scaling deep research via reinforcement learning in real-world environments, 2025. URL https://arxiv.org/abs/2504.03160

    Pith/arXiv arXiv 2025

  37. [37]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  38. [38]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  39. [39]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  40. [40]

    yes" or

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    arXiv 2021