Pith. sign in

REVIEW 4 major objections 5 minor 39 references

A learned summarizer inserted after each retrieval step in an RL-trained search agent reduces context length by 35% and improves exact-match accuracy on seven QA benchmarks, especially multi-hop ones.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 10:16 UTC pith:HO26VIYL

load-bearing objection Useful idea and credible efficiency gains, but the accuracy claim is confounded by changing the retrieval budget (5/5 vs 3/3) with no controlled ablation. the 4 major comments →

arxiv 2510.10448 v2 pith:HO26VIYL submitted 2025-10-12 cs.CL

RECON: Reasoning with Condensation for Efficient Retrieval-Augmented Generation

Zhichao Xu , Minheng Wang , Yawei Wang , Wenqian Ye , Yuntao Du , Yunpu Ma , Yijun Tian This is my paper
classification cs.CL
keywords retrieval-augmented generationcontext compressionreinforcement learningsummarizationmulti-hop question answeringsearch agentsknowledge distillationexact match
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

RECON claims that compressing retrieved documents as they arrive, not after the fact, is a key ingredient for reinforcement-learned search agents. The paper inserts a dedicated summarizer into the Search-R1 loop, trained first to pick relevant passages and then to mimic a stronger teacher's summaries, and freezes it while the policy is trained with PPO. On seven QA benchmarks it reports a 35% cut in average context length, roughly 5% faster training and 31% lower inference latency, and average exact-match gains from 0.303 to 0.347 for the 3B policy and from 0.431 to 0.444 for the 7B policy. The largest accuracy improvements appear on multi-hop questions, where cleaner evidence supports multi-step reasoning. If this holds, learned observation compression becomes a standard component for building scalable RL-based retrieval-augmented generation systems.

Core claim

The paper's central claim is that a frozen, dedicated summarizer interleaved at every retrieval step, rather than a one-time postprocessing step, improves both cost and cognition in RL-trained RAG. The summarizer is trained in two stages: a relevance-pretraining stage on MS MARCO that teaches it to distinguish useful from irrelevant passages, and a distillation stage that aligns it with multi-aspect summaries (factual correctness, completeness, coverage, coherence, clarity, logicality) generated by a proprietary LLM teacher. During reinforcement learning it stays frozen, so the policy receives concise, human-readable evidence at each search turn while the summarizer's behavior remains fixed.

What carries the argument

The load-bearing component is the observation compressor: a lightweight summarizer (initialized from Qwen2.5-3B-Instruct) that takes the documents returned by the retriever and emits a compact, query-focused summary in place of the raw concatenated passages. Its two-stage training—relevance pretraining on MS MARCO followed by multi-aspect distillation from a stronger teacher—gives it both filtering and factuality behavior. The integration pattern matters too: the summarizer runs after every search call, its output is inserted as the observation, and the policy is trained with token-level loss masking so only the policy's own tokens are optimized. This decoupling lets the compressor be swappe

Load-bearing premise

The comparison that supports the main accuracy claim changes two variables at once: RECON retrieves 5 documents per turn and allows 5 turns, while Search-R1 retrieves 3 documents and allows 3 turns, so the reported EM gains may come from the larger search budget rather than from the condensation module.

What would settle it

Run the Search-R1 policy with the same enlarged budget (5 documents per turn, 5 turns) but no summarizer, feeding raw retrieved documents into the context instead of summaries, and compare exact match on the same seven benchmarks. If that budget-matched baseline matches or exceeds RECON's 0.347 (3B) and 0.444 (7B), the summarizer contributes little beyond added retrieval; if it falls short, condensation is doing the work.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Average context length drops from about 948 to 620 tokens across the seven benchmarks, directly reducing serving cost and memory pressure during RL rollouts.
  • A 3B policy gains 14.5% relative exact match (0.303 to 0.347) and a 7B policy gains 3.0% (0.431 to 0.444), with the gains concentrating in multi-hop QA.
  • Because the summarizer is frozen and separate from the policy, the same compressor can be dropped into other RL-based agentic RAG systems that suffer from context accumulation.
  • With condensed observations, the agent can afford more documents per turn and more turns (5 vs. 3) without blowing up the context window, enabling deeper search behavior.
  • Training time improves by about 5% and inference latency by about 31%, so the added summarization step pays for itself in wall-clock time.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The headline accuracy comparison changes the search budget alongside the summarizer: RECON uses 5 documents per turn and up to 5 turns, while the Search-R1 baseline uses 3 and 3. A budget-matched baseline without any summarizer is never run, so the fraction of the EM gain owed to condensation alone remains untested—that is the experiment to demand.
  • If the summarizer is indeed the cause, the same 35% context reduction could be traded for longer-horizon search (more turns, more sources) at constant cost, a scaling direction the paper does not explore.
  • The teacher-dependence the paper acknowledges suggests a natural stress test: replace the proprietary summarizer teacher with an open-weight model and check whether the multi-hop gains persist; if they do, the mechanism is compatible with a fully open stack.
  • Because efficiency gains are larger than accuracy gains at 7B, the method may be most valuable as a cost-reduction lever on strong policies, with accuracy benefits tightest where the policy is weak—a pattern worth testing across model scales.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes RECON, a framework that inserts a dedicated summarization/condensation module into the multi-turn RL training loop of Search-R1. The summarizer is trained in two stages: first, a relevance-pretraining stage on MS MARCO; second, a multi-aspect distillation stage from GPT-4o-mini using query-document pairs harvested from Search-R1 rollouts on NQ and HotpotQA. During RL training the summarizer is frozen, and retrieved documents are replaced by its summaries before being appended to the policy context. The authors claim that, relative to Search-R1, RECON reduces total context length by about 35%, speeds up training by 5.2% and inference by 30.9%, and improves average exact match from 0.303 to 0.347 on Qwen2.5-3B-Base and from 0.431 to 0.444 on Qwen2.5-7B-Base across seven QA benchmarks.

Significance. If the central claim holds, RECON would be a useful, practical contribution: it attacks a real bottleneck in RL-trained RAG agents, where raw retrieved observations accumulate across turns and make training/inference expensive. The proposed design is decoupled from policy optimization, which is important for training stability, and the paper provides a fairly detailed experimental protocol, including hyperparameters, dataset statistics, and code. The idea of applying learned observation compression inside a multi-turn RL loop, rather than only as a post-processing step, is timely and goes beyond the single-pass RECOMP paradigm. The strongest evidence is the substantially lower context length and wall-clock latency in Table 3/Figure 2, and the larger relative EM gains on the 3B model. However, the empirical comparison as currently presented does not isolate the contribution of the summarizer from the effect of a larger retrieval budget, so the headline accuracy claim is not yet fully supported.

major comments (4)
  1. [§3.3, Table 1] The central comparison is confounded. RECON changes two hyperparameters relative to Search-R1: the number of retrieved documents per turn is increased from 3 to 5, and the maximum number of turns is extended from 3 to 5. The Search-R1 baseline in Table 1 uses 3/3. The paper never runs Search-R1 with the 5/5 budget and no summarizer, so the EM gains (e.g., 3B: 0.303→0.347; 7B: 0.431→0.444) could be caused by the larger search budget and longer horizon rather than by condensation. Please add a 2×2 ablation: Search-R1 with 5 docs/5 turns (no summarizer), RECON with 5/5, RECON with 3/3, and/or Search-R1 with 3/3. Without this, the paper's central claim that learned observation compression improves accuracy is not established.
  2. [§4.2, Table 3, Figure 2] The efficiency claims also do not isolate the summarizer's effect. The reduction in average context length is partly mechanical: replacing 5 raw documents with a summary always shortens the input. The reported 'turns used' (RECON 1.84 vs Search-R1 2.13) reflects the policy's learned stopping behavior under a different maximum-turn budget, so it is not a clean measure of the summarizer's benefit. Please report efficiency numbers at matched retrieval budgets (e.g., 5/5 with and without summarizer), and ideally also report the summarizer's own compute cost separately from the policy's wall-clock time.
  3. [§4.1, §B.4, Table 1] All reported EM numbers are single-seed (random seed=1 in §B.4), with no error bars or multiple seeds. This is especially important for the 7B result, where the average improvement is only 0.013 EM, and several per-dataset differences are within plausible run-to-run noise (e.g., PopQA 0.457→0.454, Musique 0.196→0.206). Please provide mean and standard deviation over at least three seeds, or a bootstrap confidence interval, to support the claim of 'consistent improvements'.
  4. [§3.2, §B.2, Table 1] The summarizer is distilled on query-document pairs collected from Search-R1 rollouts on NQ and HotpotQA, and these same two datasets are used for in-domain training/evaluation in Table 1. The paper says this is 'consistent with RECON's training distribution to avoid leakage,' but it is unclear whether the distillation data comes from the training split while evaluation is on the test split. Even without direct leakage, optimizing the summarizer toward teacher summaries on this distribution and then evaluating on the same distribution can overstate the in-domain gains. Please clarify the train/test separation explicitly and report the average excluding or conditioning on fully held-out datasets.
minor comments (5)
  1. [Abstract vs §4.2] The Abstract reports a 5.4% training speedup, while §4.2 reports 5.2% (13.9 vs 14.7 hours). Please make these consistent.
  2. [§D, Table 1] The Search-R1 baseline numbers are said to be from Jin et al. (2025c). Please state explicitly which numbers were reproduced under the authors' own evaluation harness and which are copied from the prior paper; otherwise differences could be due to evaluation protocol rather than method.
  3. [Algorithm 1] The caption says 'Colored text denotes modifications,' but in a monochrome/PDF version the colored text may not be distinguishable. Use a visible marker (e.g., bold or a comment) instead.
  4. [Table 2] MuSiQue's test size is missing. Please fill it in or explain why only validation is used.
  5. [§B.2] The sentence 'Training data is drawn from NQ and HotpotQA queries, consistent with RECON's training distribution to avoid leakage' is ambiguous. Clarify whether the distillation queries overlap with the evaluation sets, and what was done to prevent leakage into the policy's training data.

Circularity Check

0 steps flagged

No significant circularity: the EM gains are not forced by any fitted parameter; the main weakness is a missing 5/5 no-summarizer control, which is a confound rather than a circular reduction.

full rationale

The paper's central comparison is experimental, not derivational: the summarizer is an external trained artifact (Qwen2.5-3B, Stage-1 relevance SFT on MS MARCO, Stage-2 distillation from GPT-4o-mini) and is frozen during PPO, so no parameter inside RECON is fitted to the reported EM numbers. The RL reward is exact match and evaluation uses exact match; that is standard policy optimization, not a prediction forced by construction. The only mild self-referential element is that the Stage-2 distillation data are generated from NQ and HotpotQA queries in Search-R1 rollouts (Section 3.2 / B.2: 'Training data is drawn from NQ and HotpotQA queries, consistent with RECON ’s training distribution to avoid leakage'), and NQ/HotpotQA are also in-domain evaluation sets (Table 2). This can inflate the in-domain columns, but the OOD benchmarks (TriviaQA, PopQA, 2Wiki, Musique, Bamboogle) are external and also improve, so the central claim is not equivalent to its input. A more serious issue is the uncontrolled comparison: Section 3.3 changes 'the number of retrieved documents per turn is increased from 3 to 5, and the maximum number of turns is extended from 3 to 5', with no Search-R1 run at the same 5/5 budget without the summarizer; under the stated rules this is a confound / missing control (correctness risk), not circularity. There is no load-bearing self-citation chain and no uniqueness theorem imported from the authors.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

No new theoretical entities are introduced. The summarizer is a trained artifact composed of an existing backbone (Qwen2.5-3B-Instruct), LoRA, and distillation data. The central claim rests on empirical evaluation, not on a derived equation; the free parameters above are design choices that affect the headline numbers, and the axioms are the measurement and data assumptions that support the empirical comparison.

free parameters (5)
  • retrieved_docs_per_turn = 5
    Increased from Search-R1's 3 when integrating RECON; the comparison does not control for this, so accuracy gains may be partly attributable to a larger retrieval budget.
  • max_search_turns = 5
    Extended from Search-R1's 3 in the RECON variant; confounded with the summarizer in the comparison.
  • distillation_data_scale = 1.47M (468k HotpotQA + 1.0M NQ)
    Hand-selected mixture; no sensitivity analysis to data size or ratio is reported.
  • summarizer_aspect_setting = clarity
    Six distillation aspects are trained, but evaluation 'primarily' uses the clarity setting; other aspects are not compared in main results.
  • summarizer_LoRA_hyperparameters = rank 8, alpha 32
    Standard LoRA choices; no ablation or sensitivity study.
axioms (4)
  • domain assumption Exact match on seven QA benchmarks is a valid measure of retrieval-agent quality.
    EM is used both as the RL reward and as the headline metric; it ignores partial credit and answer justification quality.
  • domain assumption The official Search-R1 PPO checkpoint is a strong, representative baseline.
    Core comparison; the Search-R1 numbers are taken from Jin et al. (2025c) rather than rerun by the authors in the same harness.
  • domain assumption GPT-4o-mini's multi-aspect summaries are trustworthy teacher targets.
    The student summarizer is distilled from these summaries; any teacher errors or stylistics propagate to the student (also acknowledged in Limitations).
  • domain assumption E5-base-v2 retrieves enough evidence for the evaluation questions.
    The evidence available to the agent comes entirely from this frozen retriever; if it fails to retrieve relevant passages, compression cannot recover them.

pith-pipeline@v1.3.0-alltime-deepseek · 14387 in / 11011 out tokens · 90400 ms · 2026-08-04T10:16:43.836284+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of RECON: Reasoning with Condensation for Efficient Retrieval-Augmented Generation." pith.science (2026). https://pith.science/paper/HO26VIYL

@misc{pith2026251010448,
  author       = {Pith},
  title        = {Pith review of: RECON: Reasoning with Condensation for Efficient Retrieval-Augmented Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HO26VIYL}},
  note         = {Machine review of arXiv:2510.10448}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Search agents trained with reinforcement learning (RL) interleave reasoning with tool calls in a multi-turn, tool-integrated reasoning (TIR) loop, where each tool invocation returns an environment observation that is appended to the agent's context. As the rollout proceeds, these raw observations accumulate, inflating token cost and diluting the signal available for downstream reasoning. Unlike single-pass retrieve-then-read pipelines, where context compression is a one-time postprocessing step, the multi-turn RL setting requires compression that runs at every observation step while remaining decoupled from policy optimization. We introduce RECON (REasoning with CONdensation), a framework that addresses this challenge by inserting a dedicated observation compressor into the reasoning loop. The compressor is trained via a two-stage curriculum: relevance pretraining on QA datasets followed by multi-aspect distillation from proprietary LLMs, and remains frozen during RL training to preserve policy stability. Integrated into the Search-R1 search-agent pipeline, RECON reduces total context length by 35%, improves training speed by 5.4% and inference latency by 30.9%, while boosting average exact-match by 14.5% on the 3B agent and 3.0% on the 7B agent, with particular strength in multi-hop QA. These results establish learned observation compression as a key component for building practical, scalable RL-trained search agents.

Figures

Figures reproduced from arXiv: 2510.10448 by Minheng Wang, Wenqian Ye, Yawei Wang, Yijun Tian, Yunpu Ma, Yuntao Du, Zhichao Xu.

Figure 1
Figure 1. Figure 1: Training pipeline of our method. In the rollout module, instead of directly using retrieval results from the search engine (dashed line), an additional summarization model is used to condense the retrieved information and remove noises from document sources. This way we reduce the context length and achieve efficient and effective rollout in both training and inference. Refer to § C for algorithmic details… view at source ↗
Figure 2
Figure 2. Figure 2: Inference efficiency of RECON vs. Search [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 23 linked inside Pith

  1. [1]

    My action is not correct. Let me rethink

    Colored text denotes modifications from the original Search R1 algorithm. For completeness, we present the PPO objective used for policy optimization with the summarizer- augmented retrieverR sum. JPPO(θ) =Ex∼D, y∼πθ(·|x;Rsum) " 1 |y| |y|X t=1 I(yt)·min rt(θ)At,clip rt(θ),1−ϵ,1+ϵ At # rt(θ) = πθ yt |x, y<t;R sum πold yt |x, y<t;R sum .(1) Here, πθ and πol...

    2017

  2. [3]

    Alexis Chevalier, Alexander Wettig, Anirudh Ajith, and Danqi Chen

    Learning to reason with search for llms via reinforcement learning.arXiv preprint arXiv:2503.19470. Alexis Chevalier, Alexander Wettig, Anirudh Ajith, and Danqi Chen

    Pith/arXiv arXiv

  3. [5]

    Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen

    Group-in-group policy optimization for llm agent training.arXiv preprint arXiv:2505.10978. Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. 2023a. Enabling large language models to gener- ate text with citations. InThe 2023 Conference on Empirical Methods in Natural Language Processing. Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi ...

    Pith/arXiv arXiv 2023

  4. [6]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shi- rong Ma, Peiyi Wang, Xiao Bi, and 1 others

    Towards an ai co- scientist.arXiv preprint arXiv:2502.18864. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shi- rong Ma, Peiyi Wang, Xiao Bi, and 1 others

    Pith/arXiv arXiv

  5. [7]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, and 1 others

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948. Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, and 1 others

    Pith/arXiv arXiv

  6. [9]

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richard- son, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others

    Gpt-4o system card.arXiv preprint arXiv:2410.21276. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richard- son, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others

    Pith/arXiv arXiv

  7. [10]

    Bowen Jin, Jinsung Yoon, Jiawei Han, and Sercan O Arik

    Openai o1 system card.arXiv preprint arXiv:2412.16720. Bowen Jin, Jinsung Yoon, Jiawei Han, and Sercan O Arik. 2025a. Long-context llms meet rag: Overcom- ing challenges for long inputs in rag. InThe Thir- teenth International Conference on Learning Repre- sentations. Bowen Jin, Jinsung Yoon, Priyanka Kargupta, Sercan O Arik, and Jiawei Han. 2025b. An emp...

    Pith/arXiv arXiv

  8. [11]

    Jeff Johnson, Matthijs Douze, and Hervé Jégou

    Flashrag: A modular toolkit for efficient retrieval-augmented generation research.CoRR, abs/2405.13576. Jeff Johnson, Matthijs Douze, and Hervé Jégou

    Pith/arXiv arXiv

  9. [13]

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, and 1 others

    Chain of thought monitorability: A new and fragile opportunity for ai safety.arXiv preprint arXiv:2507.11473. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, and 1 others

    Pith/arXiv arXiv

  10. [14]

    Shahar Levy, Nir Mazor, Lihi Shalmon, Michael Hassid, and Gabriel Stanovsky

    Measuring faithfulness in chain-of-thought reasoning.arXiv preprint arXiv:2307.13702. Shahar Levy, Nir Mazor, Lihi Shalmon, Michael Hassid, and Gabriel Stanovsky

    Pith/arXiv arXiv

  11. [15]

    More documents, same length: Isolating the challenge of multiple documents in rag.arXiv preprint arXiv:2503.04388. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Hein- rich Küttler, Mike Lewis, Wen-tau Yih, Tim Rock- täschel, Sebastian Riedel, and Douwe Kiela. 2020a. Retrieval-augmented generation for knowle...

    arXiv

  12. [16]

    Nelson F

    Search-o1: Agentic search- enhanced large reasoning models.arXiv preprint arXiv:2501.05366. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paran- jape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Pith/arXiv arXiv

  13. [17]

    Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettle- moyer, and Hannaneh Hajishirzi

    A scalable framework for evaluating health language models.arXiv preprint arXiv:2503.23339. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettle- moyer, and Hannaneh Hajishirzi

    arXiv

  14. [18]

    InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12076–12100, Singa- pore

    FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12076–12100, Singa- pore. Association for Computational Linguistics. Artidoro Pagnoni, Vidhisha Balachandran, and Yulia Tsvetkov

    2023

  15. [20]

    InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 5687–5711, Singa- pore

    Measuring and narrowing the compositionality gap in language mod- els. InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 5687–5711, Singa- pore. Association for Computational Linguistics. Jack W. Rae, Anna Potapenko, Siddhant M. Jayaku- mar, Chloe Hillier, and Timothy P. Lillicrap

    2023

  16. [21]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, and 1 others

    Proxi- mal policy optimization algorithms.arXiv preprint arXiv:1707.06347. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, and 1 others

    Pith/arXiv arXiv

  17. [22]

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu

    Deepseek- math: Pushing the limits of mathematical reason- ing in open language models.arXiv preprint arXiv:2402.03300. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu

    Pith/arXiv arXiv

  18. [23]

    Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji- Rong Wen

    Hybridflow: A flex- ible and efficient rlhf framework.arXiv preprint arXiv:2409.19256. Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji- Rong Wen

    Pith/arXiv arXiv

  19. [24]

    arXiv preprint arXiv:2503.05592

    R1-searcher: Incentivizing the search capability in llms via reinforcement learning. arXiv preprint arXiv:2503.05592. Jiashuo Sun, Xianrui Zhong, Sizhe Zhou, and Jiawei Han

    Pith/arXiv arXiv

  20. [25]

    Richard S Sutton, Andrew G Barto, and 1 others

    Dynamicrag: Leveraging outputs of large language model as feedback for dynamic reranking in retrieval-augmented generation.arXiv preprint arXiv:2505.07233. Richard S Sutton, Andrew G Barto, and 1 others

    Pith/arXiv arXiv

  21. [27]

    Fangyuan Xu, Weijia Shi, and Eunsol Choi

    Resum: Unlocking long-horizon search intelli- gence via context summarization.arXiv preprint arXiv:2509.13313. Fangyuan Xu, Weijia Shi, and Eunsol Choi

    arXiv

  22. [29]

    Zhichao Xu, Fengran Mo, Zhiqi Huang, Crystina Zhang, Puxuan Yu, Bei Wang, Jimmy Lin, and Vivek Sriku- mar

    A survey of model architectures in infor- mation retrieval.arXiv preprint arXiv:2502.14822. Zhichao Xu, Fengran Mo, Zhiqi Huang, Crystina Zhang, Puxuan Yu, Bei Wang, Jimmy Lin, and Vivek Sriku- mar

    arXiv

  23. [30]

    Zhichao Xu, Jinghua Yan, Ashim Gupta, and Vivek Srikumar

    Distillation versus Contrastive Learn- ing: How to Train Your Rerankers.arXiv preprint arXiv:2507.08336. Zhichao Xu, Jinghua Yan, Ashim Gupta, and Vivek Srikumar

    arXiv

  24. [31]

    InProceedings of the 10th Workshop on Representation Learning for NLP (RepL4NLP- 2025), pages 152–169, Albuquerque, NM

    State Space Models are Strong Text Rerankers. InProceedings of the 10th Workshop on Representation Learning for NLP (RepL4NLP- 2025), pages 152–169, Albuquerque, NM. Asso- ciation for Computational Linguistics. https:// aclanthology.org/2025.repl4nlp-1.12/. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, ...

    2025

  25. [32]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Ben- gio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning

    Qwen2.5 technical report.arXiv preprint arXiv:2412.15115. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Ben- gio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning

    Pith/arXiv arXiv

  26. [34]

    Table 2: QA Dataset statistics.†denotes in-domain trainset and ∗ denotes OOD dataset

    Deepresearcher: Scaling deep research via reinforce- ment learning in real-world environments.arXiv preprint arXiv:2504.03160. Table 2: QA Dataset statistics.†denotes in-domain trainset and ∗ denotes OOD dataset. Dataset # Train # Val # Test Corpus Task License Natural Questions† (Kwiatkowski et al.,

    Pith/arXiv arXiv

  27. [35]

    pre-training

    – – 125 Web Adversarial QA MIT A Formulation of Search-R1 The training of Search-R1 (Jin et al., 2025c) is formulated as a reinforcement learning (RL) problem where the policy model πθ alternates between (i) generating intermediate reasoning spans and (ii) issuing search queries to an external engine R, with the objective of producing a factually correct ...

    2016

  28. [36]

    pre- training

    to retrieval top-5 passages for each query and construct corresponding {(xij,D ij)} query-documents pairs. For each (xij,D ij) pair, we instruct GPT-4o-mini to generate a multi-document query-focused summary focusing on each aspect ak of six aspects. Together, we collect 468,547 (xij,D ij, ak) triplet from the HotpotQA dataset and 1,002,329 (xij,D ij, ak)...

    2024

  29. [37]

    We retrieve top-5 passages (baseline: top-3)

    with FAISS GPU acceleration (Johnson et al., 2019); retriever is frozen during RL training. We retrieve top-5 passages (baseline: top-3). •Randomization.Random seed=1. • Backbone and system.Backbone: Qwen2.5-Base at 3B and 7B scales. We use XFORMERS as attention backend and PyTorch FSDP for distributed training. You are a helpful assistant in a retrieval-...

    2019

  30. [2016]

    Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi

    Ms marco: A human generated ma- chine reading comprehension dataset.arXiv preprint arXiv:1611.09268. Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi

    Pith/arXiv arXiv

  31. [2017]

    Tomek Korbak, Mikita Balesni, Elizabeth Barnes, Yoshua Bengio, Joe Benton, Joseph Bloom, Mark Chen, Alan Cooney, Allan Dafoe, Anca Dragan, and 1 others

    Triviaqa: A large scale distantly supervised challenge dataset for reading comprehen- sion.arXiv preprint arXiv:1705.03551. Tomek Korbak, Mikita Balesni, Elizabeth Barnes, Yoshua Bengio, Joe Benton, Joseph Bloom, Mark Chen, Alan Cooney, Allan Dafoe, Anca Dragan, and 1 others

    Pith/arXiv arXiv

  32. [2018]

    Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Be- rant

    Hotpotqa: A dataset for diverse, explainable multi-hop question answer- ing.arXiv preprint arXiv:1809.09600. Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Be- rant

    Pith/arXiv arXiv

  33. [2019]

    Other baselines.Baseline methods in Table 1 include (1) prompting-based method: Direct Inference and Chain-of-Thought prompting (Wei et al., 2022)

    and HotpotQA (Yang et al., 2018), using Wikipedia-18 dump as the knowledge corpus, with a unified preprocessing pipeline. Other baselines.Baseline methods in Table 1 include (1) prompting-based method: Direct Inference and Chain-of-Thought prompting (Wei et al., 2022). (2) Retrieval-based method: RAG (Lewis et al., 2020b), IRCoT (Trivedi et al., 2023), Se...

    2018

  34. [2020]

    In Proceedings of the 2020 Conference on empirical methods in natural language processing (EMNLP), pages 3632–3645

    Coarse-to-fine query focused multi-document summarization. In Proceedings of the 2020 Conference on empirical methods in natural language processing (EMNLP), pages 3632–3645. Zhichao Xu, Fengran Mo, Zhiqi Huang, Crystina Zhang, Puxuan Yu, Bei Wang, Jimmy Lin, and Vivek Sriku- mar

    2020

  35. [2021]

    InProceedings of the 2021 Con- ference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Lan- guage Technologies, pages 4812–4829, Online

    Understanding factuality in abstrac- tive summarization with FRANK: A benchmark for factuality metrics. InProceedings of the 2021 Con- ference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Lan- guage Technologies, pages 4812–4829, Online. As- sociation for Computational Linguistics. Ofir Press, Muru Zhang, Sewon M...

    2021

  36. [2022]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others

    Text embeddings by weakly- supervised contrastive pre-training.arXiv preprint arXiv:2212.03533. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others

    Pith/arXiv arXiv

  37. [2023]

    InProceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, pages 3829–3846, Singapore

    Adapting language models to compress contexts. InProceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, pages 3829–3846, Singapore. Associa- tion for Computational Linguistics. Feiteng Fang, Yuelin Bai, Shiwen Ni, Min Yang, Xiao- jun Chen, and Ruifeng Xu

    2023

  38. [2024]

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa

    Deepseek- coder: When the large language model meets programming–the rise of code intelligence.arXiv preprint arXiv:2401.14196. Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa

    Pith/arXiv arXiv

  39. [2025]

    Monitoring reasoning models for misbehavior and the risks of promoting obfuscation.arXiv preprint arXiv:2503.11926. Shiyi Cao, Sumanth Hegde, Dacheng Li, Tyler Griggs, Shu Liu, Eric Tang, Jiayi Pan, Xingyao Wang, Akshay Malik, Graham Neubig, Kourosh Hakhamaneshi, Richard Liaw, Philipp Moritz, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica

    Pith/arXiv arXiv