Pith. sign in

REVIEW 3 major objections 4 minor 22 references

This paper claims that reliability-aware evaluation, not final-answer accuracy, reveals how search agents handle degraded evidence, and that a controlled synthetic retrieval environment can measure those differences.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 03:18 UTC pith:XGUQHO5S

load-bearing objection Useful stress-test rig and two metrics worth taking seriously; the relevance and factuality label validations are too thin to support the sharpest axis-specific claims, but the core findings are solid enough to referee. the 3 major comments →

arxiv 2607.13920 v1 pith:XGUQHO5S submitted 2026-07-15 cs.CL

DeepStress: Stress-Testing Deep Search Agents

Ismael Rousseau , Geraldine Damnati , Frederic Bechet This is my paper
classification cs.CL
keywords search agentsstress testingsynthetic document generationreliability-aware evaluationknowledge conflictsretrieval robustnessmulti-hop question answeringabstention behavior
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

DeepStress claims that the ability to handle unreliable search evidence is not captured by final-answer accuracy, and that a laboratory environment which replaces retrieval with controllable synthetic documents can expose how agents really differ. The authors intercept every search call and return one generated document whose trustworthiness, relevance, and factuality are sampled from experimenter-chosen probabilities. Across two question-answering benchmarks and twelve agents, each degraded axis produces its own failure signature—some agents ignore untrustworthy source cues, some over-search until budget exhaustion, some break output format, and some confidently answer from relevant-but-false documents. To make these differences comparable, the paper proposes a reliability-aware score (RAS) that gives partial credit for abstaining or stopping under unreliable evidence, and a cost-normalized variant (CoPRAS) that re-ranks systems relative to raw accuracy or efficiency. If the approach is sound, search-agent evaluation should move beyond accuracy to report reliability-aware outcomes, and benchmarks whose questions are answerable from memory need a no-search baseline.

Core claim

Search agents do not have one robustness trait but several: under controlled degradation of trustworthiness, relevance, or factuality, twelve agents diverge into distinct failure profiles—some ignore untrustworthy source cues, some fail mainly on irrelevant documents, some answer confidently from relevant-but-false documents, and others abstain or exhaust their tool budget. Each search call draws a label per axis from scenario probabilities, and a generator produces a document realizing those labels. The paper's reliability-aware score gives full credit to correct answers, partial credit to refusal when evidence is unreliable, and zero to incorrect or malformed answers; its cost-normalized v

What carries the argument

The central object is the search-call interceptor plus scenario sampler. Each agent search is routed to a simulator that draws T, R, F labels from scenario probabilities, then builds a document (grounding factual ones in gold evidence, sampling known low-credibility domains for untrustworthy ones, and swapping in an unrelated question for irrelevant ones). The evaluative machinery is the reliability-aware score: RAS_i = 1 for correct answers, 1−ρ for abstention or budget exhaustion with ρ = π_T×π_R×π_F, and 0 for incorrect or malformed answers; CoPRAS divides total cost by total RAS. This scoring treats 'not answering under unreliable evidence' as partially correct, which separates reliabili

Load-bearing premise

The load-bearing premise is that synthetic documents generated under a given degradation label differ from clean documents only along that axis; because the paper's own sanity check finds imperfect factuality label agreement and no human validation of the relevance manipulation, the measured behavioral contrasts could partly be artifacts of generation instead of agent robustness.

What would settle it

Audit the generated corpus in a blind design: ask annotators to label documents for relevance and factuality without seeing the intended label, and rerun the 0%-relevance and 0%-factuality conditions with a second generator and a second prompt wording; if the agent outcome distributions shift across generators, the stress test is measuring generator artifacts rather than a stable agent property.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Failure taxonomies must be reported alongside accuracy: abstention, budget exhaustion, missing answer tags, and incorrect answers move in opposite directions under different degradation axes.
  • RAS and CoPRAS change system rankings at degraded quality, so reliability-aware and cost-efficiency evaluations can disagree with raw tokens-per-correct answers.
  • No-search baselines are required for question sets whose answers are memorizable; otherwise, correct answers under completely false evidence are misread as evidence robustness.
  • Contradictory-evidence conditions (50% factuality) provide a direct probe of an agent's ability to aggregate consistent evidence across multiple searches.
  • Unreliable evidence raises search cost, on average roughly doubling the cost-normalized metric between fully reliable and fully degraded documents, making robustness an efficiency issue as well as an accuracy issue.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the simulator could double as a training adversary—an RL objective maximizing RAS under degraded scenarios would directly reward abstention and source-checking, not just final accuracy.
  • Editorial inference: the one-document-per-call design leaves open whether the same failure signatures hold when a search returns a mix of reliable and contradictory documents; testing multi-document returns is the natural next step.
  • Editorial inference: because the factuality manipulation showed label noise (13 of 69 intended-factual documents were judged non-factual), the measured contrasts are likely a lower bound; a generator with tighter label adherence would sharpen the behavioral differences.
  • Editorial inference: agents that verbally flag degradation without changing their answer suggest future process metrics should score detection-and-action, not detection alone.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes DeepStress, a controlled simulation environment for stress-testing search agents. The framework intercepts an agent's search tool and replaces retrieval with dynamically generated synthetic documents whose trustworthiness, relevance, and factuality are sampled from experimenter-controlled probabilities. The authors evaluate twelve systems (six GPT-based configurations, five open-weight search/RL agents, plus a no-reasoning DR-Tulu ablation) on HotpotQA and BrowseComp-Plus under seven scenarios (0%/50%/100% along each axis while the other axes are fixed at 100%), totaling 33,600 traces. They introduce two aggregate metrics, RAS and CoPRAS, which give partial credit to abstention and tool-budget exhaustion under unreliable evidence. The main empirical claims are: (i) agents differ substantially in how they handle degraded documents, with GPT systems tending to abstain or over-search, open-weight systems producing confident errors, and DR-Tulu showing format brittleness; (ii) RAS and CoPRAS reveal reliability-aware behavior that raw accuracy and TPC miss; and (iii) HotpotQA is partly answerable from parametric knowledge, as shown by a direct-inference baseline, so it should not be used as a pure search-agent benchmark without such a baseline.

Significance. If the central claims hold, DeepStress is a useful contribution: it provides a reproducible, controlled laboratory for measuring how search agents react to evidence-quality failures, a dimension that standard final-answer benchmarks under-sample. The paper has clear strengths: it releases (or promises to release) the environment code; it evaluates a broad and diverse set of systems; it uses dynamic per-query document generation rather than a static corpus; it includes a direct-inference baseline that cleanly demonstrates parametric-knowledge leakage on HotpotQA; and its outcome taxonomy goes beyond binary accuracy. The proposed RAS/CoPRAS metrics are transparently specified and their design choices are discussed. However, the validity of the entire axis-specific analysis rests on the fidelity of the synthetic labels. The human sanity check is only partially convincing: factuality agreement is modest and asymmetric, and relevance — the axis producing the largest behavioral differences — is never human-validated. Because these are load-bearing concerns rather than cosmetic ones, the manuscript needs substantial revision before the empirical conclusions can be accepted.

major comments (3)
  1. [§3.3, Appendix A.1, Tables 2–3] Factuality is the central degradation axis for the dataset-contrast claims in §5.2, yet the human validation is weak and asymmetric. Table 2 reports only 85.5% raw agreement with the intended factuality label and Cohen's κ=0.64. Table 3 shows the error pattern: 13 of 69 documents intended as factual were judged non-factual by annotators, while intended non-factual documents were almost always recognized as such. Thus the F=100% control condition is contaminated with documents that annotators read as false, and the F=0% condition may consist of obviously exaggerated falsehoods (see the Appendix A.3 example) rather than realistic misleading evidence. This directly threatens the claim that agents differ specifically in handling non-factual documents. Please quantify the effect of annotator-disputed documents, report per-condition results excluding them, or strengthen the generation protocol
  2. [Appendix A.1; §3.2] The relevance axis has no human validation at all: Appendix A.1 states that 'relevance is always annotated as True.' This is especially problematic because Figure 1b shows the largest behavioral differences on the relevance axis (e.g., Search-R1 incorrect answers rising to 79.2% at R=0). R=0 documents are generated by replacing the target question and support context with an unrelated question, and the document title is sampled from that unrelated context. Nothing verifies that these documents are actually perceived as irrelevant, that they contain no overlapping entities or answer-relevant content, or that they are not systematically identifiable by surface cues such as title mismatches or formatting oddities. Because the R=0 condition is defined by construction rather than validated, the measured relevance-specific effects could be artifacts of the swapping procedure. Please add a huma
  3. [§3.4, Eqs. (4)–(5); Table 1] RAS's reliability definition is internally inconsistent with its motivation. Eq. (4) defines ρ = π_T × π_R × π_F, so any axis at 0% makes ρ=0 and gives full RAS credit to abstention and tool-budget exhaustion. But the text preceding Eq. (5) justifies partial credit only for conditions where 'relevance or factuality is set to 0%' — i.e., where the answer cannot be recovered from the documents. A document that is untrustworthy in style but factually correct and relevant is still answerable; yet in the T=0, R=F=100% scenario, a system that refuses to answer solely because the source looks unreliable receives RAS=1. This inflates the 0% column of Table 1 and favors GPT systems that abstain or over-search on stylistic grounds. Please either restrict partial credit to R/F degradation or provide a substantive argument that low trustworthiness alone makes evidence unusable; the current choice co
minor comments (4)
  1. [Appendix A.1] The sentence 'In 2, we report raw accuracy...' is missing the table number; it should refer to Table 2. Also, the claim of 'strong agreement' for factuality is overstated given the asymmetric confusion matrix in Table 3.
  2. [Abstract / §1] The phrase 'This phenomenon occurs rarely in realistic benchmarks' has an unclear antecedent ('this phenomenon' most likely refers to poor-quality evidence, but the sentence reads as if it refers to dramatic failure). Please rephrase for clarity.
  3. [Figure 1] The figure repeats the 100% condition three times and uses small fonts and dense legends; consider showing the reference condition once per panel and using larger labels. Also, the paper does not report confidence intervals or significance tests for the 200-question per-system scenario cells; a brief note on variability would strengthen the claims of 'substantial differences.'
  4. [§3.4 / Appendix E] The final-answer classification relies entirely on GPT-4.1 as an LLM judge with no human validation of the judge itself. Given that the outcome taxonomy (especially the abstention category) is central to RAS, a small human agreement study for the judge would increase confidence.

Circularity Check

0 steps flagged

No significant circularity: the central results are controlled measurements, and RAS/CoPRAS are transparent metric definitions rather than fitted predictions.

full rationale

The paper's derivation chain is an experimental protocol, not a derivation in which a target result is recovered from its own inputs. DeepStress replaces the retrieval module with a simulator that samples T/R/F labels and converts them into generation prompts; the measured agent outcomes (correct, incorrect, abstention, budget exhaustion, missing tag) are empirical observations, not consequences of the metric definitions. The direct-inference baseline independently supports the parametric-knowledge interpretation of HotpotQA results, so that conclusion is not assumed from the simulator. RAS and CoPRAS are explicitly defined scoring functions: partial credit for abstention/budget exhaustion under low reliability is a stated design choice, and the paper does not present the resulting rank behavior as a predicted phenomenon. The only self-referential element is the inclusion of the authors' own SpiqaDR checkpoint among twelve baselines, but it is one system among many and no central claim depends on it; no load-bearing uniqueness theorem or prior-work ansatz is invoked. The heavier caveats in the paper concern synthetic label fidelity (85.5% factuality agreement; relevance not human-validated), which is a validity/correctness risk, not circularity: the pipeline could fail to realize its intended conditions, but this would make the experiment confounded rather than make the conclusions true by construction. Thus the paper does not reduce its claims to its inputs under any of the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

No physical invented entities are introduced. The new constructs are the DeepStress simulator and the RAS/CoPRAS metrics; the main hidden burden is the assumption that synthetic documents and LLM judges faithfully realize the intended conditions, plus the hand-chosen weights and scoring rules that shape the headline rankings.

free parameters (4)
  • Cost weights λ and μ = λ=0.25, μ=500
    Borrowed from Xie et al. (2026) for TPC and CoPRAS; hand-chosen weights for input tokens and search calls that affect all cost-normalized rankings.
  • RAS partial-credit rule = 1 for correct; 1-ρ for abstention/budget-exceeded; 0 for incorrect/missing-tag
    A design choice that defines the new metric; giving abstention and budget exhaustion the same full credit when ρ=0 partly determines which systems rank high under degradation.
  • Scenario probabilities π_T, π_R, π_F = 0%, 50%, 100% on one axis; other axes fixed at 100%
    Experimenter-chosen stress levels; they define the experimental conditions and enter RAS through the reliability product ρ.
  • Tool-call budget = 100 tool calls (budget sweep from 10 to 100 in Appendix B.2)
    Affects outcome categories and cost metrics; the paper shows rankings can shift with budget, especially for high-effort GPT systems.
axioms (5)
  • domain assumption LLM-generated synthetic documents faithfully realize the intended trustworthiness/relevance/factuality labels without systematic confounds.
    Central to the simulator; human sanity check on only 50 documents gives 96.4% agreement on trustworthiness and 85.5% on factuality, and relevance is not human-validated (Appendix A.1).
  • domain assumption GPT-4.1 LLM-as-a-judge reliably classifies final answers as correct, incorrect, or uncertain.
    All outcome distributions depend on this judge (Section 3.4, Appendix E.1), but the paper reports no agreement study or human spot-check for this classifier.
  • domain assumption Replacing each agent's native retrieval backend with the one-document-per-call simulator preserves the agent's decision-making behavior except for the intended retrieval change.
    Needed to attribute behavioral differences to document quality; the authors adapt harnesses and prompts, but the interaction between a single synthetic document and native tool-use formats is not independently validated.
  • domain assumption The 200 randomly sampled questions per dataset and condition are representative enough for the reported outcome rates and rankings.
    No error bars, confidence intervals, seeds, or question IDs are given; the paper treats the point estimates as sufficient evidence for behavior differences.
  • domain assumption Every HotpotQA and BrowseComp-Plus question is answerable when reliable evidence is available.
    Assumed by the RAS design ("every question is answerable in principle") and inherited from the datasets' construction.

pith-pipeline@v1.3.0-alltime-deepseek · 31549 in / 10692 out tokens · 98194 ms · 2026-08-02T03:18:22.947308+00:00 · methodology

0 comments
read the original abstract

While search agents demonstrate impressive capabilities in multi-step question answering, their robustness to poor-quality evidence remains under-explored. This phenomenon occurs rarely in realistic benchmarks but can lead to dramatic failure in real life applications. Therefore in this study we propose DeepStress, a stress testing framework that controls the frequency of challenging evidence by replacing the retrieval module of search agents with a controlled synthetic environment. We use this framework to control three dimensions that can affect document reliability: trustworthiness, relevance, and factuality. Testing several search agents on HotpotQA and BrowseCompPlus, we demonstrate that agents exhibit substantial differences in their ability to handle unreliable information and propose new metrics that better document systems outcomes as well as the interactions between conflicting parametric and retrieved knowledge.

Figures

Figures reproduced from arXiv: 2607.13920 by Frederic Bechet, Geraldine Damnati, Ismael Rousseau.

Figure 1
Figure 1. Figure 1: Final-answer outcomes aggregated over HotpotQA and BrowseComp-Plus. Outcome rates across three quality levels (0%, 50%, and 100%) for each document-quality axis: Trustworthiness, Relevance, and Factuality. without support documents, and the corpus is not memorized by current LLMs, therefore there is no conflict between parametric and dynamic knowledge. This corpus allows us to study how a search agent mode… view at source ↗
Figure 2
Figure 2. Figure 2: Outcomes at factuality levels 0% and 50% on [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Human evaluation in our LabelStudio interface [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Detailed outcomes vs tool-calls budget improve more gradually. As a result, correctness-based rankings become reasonably stable only after around 50 tool calls, although high-reasoning models continue to benefit from larger budgets. RAS is more stable with respect to the tool-call budget. Unlike correctness, it does not always increase with additional turns, because abstention and budget exhaustion can rec… view at source ↗
Figure 5
Figure 5. Figure 5: Effect of tool budget on aggregate metrics: correctness, RAS, TPC, and CoPRAS. [PITH_FULL_IMAGE:figures/full_fig_p017_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Mean number of tool calls per dataset, per model, for values at 0%, 50% and 100% [PITH_FULL_IMAGE:figures/full_fig_p018_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Tool calls distributions: HotpotQA [PITH_FULL_IMAGE:figures/full_fig_p027_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Tool calls distributions: BrowseComp-Plus 27 [PITH_FULL_IMAGE:figures/full_fig_p027_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Evolution of query strategy usage on BrowseComp-Plus and HotpotQA at 0%, 50%, and 100% document quality. 0.0 0.2 0.4 0.6 0.8 1.0 Verbalization rate Source trustworthiness 0.0 0.2 0.4 0.6 0.8 1.0 Verbalization rate Content relevance gpt-5-nano (low) gpt-5-nano (medium) gpt-5-nano (high) gpt-5-mini (low) gpt-5-mini (medium) gpt-5-mini (high) SpiqaDR-3B Search-R1 AutoRefine ASearcher DR-Tulu 0.0 0.2 0.4 0.6 0… view at source ↗
Figure 10
Figure 10. Figure 10: Mean verbalization rate per dataset, per model, for values at 0%, 50% and 100% [PITH_FULL_IMAGE:figures/full_fig_p028_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Detailed final-answer outcomes: HotpotQA 0 20 40 60 80 100 100% 60.0% 15.0% 25.0% 77.5% 15.5% 70.0% 13.0% 57.5% 26.5% 16.0% 74.5% 13.5% 41.5% 56.0% 49.0% 15.0% 36.0% 73.5% 26.0% 54.5% 26.3% 19.2% 54.0% 36.5% 48.5% 18.2% 31.8% 46.2% 18.6% 30.7% Trustworthiness 0 20 40 60 80 100 60.0% 15.0% 25.0% 77.5% 15.5% 70.0% 13.0% 57.5% 26.5% 16.0% 74.5% 13.5% 41.5% 56.0% 49.0% 15.0% 36.0% 73.5% 26.0% 54.5% 26.3% 19.2… view at source ↗
Figure 12
Figure 12. Figure 12: Detailed final-answer outcomes: BrowseComp-Plus 32 [PITH_FULL_IMAGE:figures/full_fig_p032_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Reliability-Aware Score details by system and scenario. Each subplot reports [PITH_FULL_IMAGE:figures/full_fig_p033_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 2 canonical work pages

  1. [1]

    arXiv preprint arXiv:2504.12516 , year=

    BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents , author=. arXiv preprint arXiv:2504.12516 , year=

    Pith/arXiv arXiv

  2. [2]

    https://aclanthology.org/D18-1259/

    Yang, Zhilin and Qi, Peng and Zhang, Saizheng and Bengio, Yoshua and Cohen, William and Salakhutdinov, Ruslan and Manning, Christopher D , url = "https://aclanthology.org/D18-1259/", booktitle=

  3. [3]

    arXiv preprint arXiv:2506.08500 , year=

    Dragged into conflicts: Detecting and addressing conflicting sources in search-augmented llms , author=. arXiv preprint arXiv:2506.08500 , year=

    Pith/arXiv arXiv

  4. [4]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    DeepDiver: Adaptive Web-Search Intensity Scaling via Reinforcement Learning , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  5. [5]

    Over-Searching in Search-Augmented Large Language Models

    Xie, Roy and Gopinath, Deepak and Qiu, David and Lin, Dong and Sun, Haitian and Potdar, Saloni and Dhingra, Bhuwan. Over-Searching in Search-Augmented Large Language Models. Proceedings of the 19th Conference of the E uropean Chapter of the A ssociation for C omputational L inguistics. 2026. doi:10.18653/v1/2026.eacl-long.361

    doi:10.18653/v1/2026.eacl-long.361 2026

  6. [6]

    Tracking the Limits of Knowledge Propagation: How LLM s Fail at Multi-Step Reasoning with Conflicting Knowledge

    Feng, Yiyang and Chen, Zeming and Wu, Haotian and Zhou, Jiawei and Bosselut, Antoine. Tracking the Limits of Knowledge Propagation: How LLM s Fail at Multi-Step Reasoning with Conflicting Knowledge. Proceedings of the 19th Conference of the E uropean Chapter of the A ssociation for C omputational L inguistics. 2026. doi:10.18653/v1/2026.eacl-long.273

    work page doi:10.18653/v1/2026.eacl-long.273 2026

  7. [7]

    Second Conference on Language Modeling , year=

    Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning , author=. Second Conference on Language Modeling , year=

  8. [8]

    2025 , eprint=

    DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research , author=. 2025 , eprint=

    2025

  9. [9]

    First Workshop on Multi-Turn Interactions in Large Language Models , year=

    BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent , author=. First Workshop on Multi-Turn Interactions in Large Language Models , year=

  10. [10]

    arXiv preprint arXiv:2601.17617 , url=

    Agentic Search in the Wild: Intents and Trajectory Dynamics from 14M+ Real Search Requests , author=. arXiv preprint arXiv:2601.17617 , url=

    Pith/arXiv arXiv

  11. [11]

    arXiv preprint arXiv:2505.04588 , url=

    Zerosearch: Incentivize the search capability of llms without searching , author=. arXiv preprint arXiv:2505.04588 , url=

    Pith/arXiv arXiv

  12. [12]

    Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs , author=

  13. [13]

    Proceedings of the 61st annual meeting of the Association for Computational Linguistics , url=

    Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions , author=. Proceedings of the 61st annual meeting of the Association for Computational Linguistics , url=

  14. [14]

    Findings of the Association for Computational Linguistics: EACL 2026 , year=

    Findings of the Association for Computational Linguistics: EACL 2026 , author=. Findings of the Association for Computational Linguistics: EACL 2026 , year=

    2026

  15. [15]

    ACM Transactions on Information Systems , volume=

    Dense text retrieval based on pretrained language models: A survey , author=. ACM Transactions on Information Systems , volume=. 2024 , url=

    2024

  16. [16]

    NeurIPS 2022 Foundation Models for Decision Making Workshop , url=

    ReAct: Synergizing Reasoning and Acting in Language Models , author=. NeurIPS 2022 Foundation Models for Decision Making Workshop , url=

    2022

  17. [17]

    arXiv preprint arXiv:2505.08638 , url=

    Trail: Trace reasoning and agentic issue localization , author=. arXiv preprint arXiv:2505.08638 , url=

    Pith/arXiv arXiv

  18. [18]

    arXiv preprint arXiv:2510.02837 , year=

    Beyond the Final Answer: Evaluating the Reasoning Trajectories of Tool-Augmented Agents , author=. arXiv preprint arXiv:2510.02837 , year=

    Pith/arXiv arXiv

  19. [19]

    Advances in Neural Information Processing Systems , volume=

    Spiqa: A dataset for multimodal question answering on scientific papers , author=. Advances in Neural Information Processing Systems , volume=. 2024 , url=

    2024

  20. [20]

    Towards Multi-Document Question Answering in Scientific Literature: Pipeline, Dataset, and Evaluation

    Huang, Hui and Velcin, Julien and Kessaci, Yacine. Towards Multi-Document Question Answering in Scientific Literature: Pipeline, Dataset, and Evaluation. Findings of the Association for Computational Linguistics: EMNLP 2025. 2025. doi:10.18653/v1/2025.findings-emnlp.576

    work page doi:10.18653/v1/2025.findings-emnlp.576 2025

  21. [21]

    The Fourteenth International Conference on Learning Representations , year=

    Unlocking Long-Horizon Agentic Search with Large-Scale End-to-End RL , author=. The Fourteenth International Conference on Learning Representations , year=

  22. [22]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Search and Refine During Think: Facilitating Knowledge Refinement for Improved Retrieval-Augmented Reasoning , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=