GraphPO represents reasoning rollouts as a DAG to merge semantically equivalent paths, share suffixes, and assign separate efficiency and correctness advantages for lower variance and better performance than chain or tree baselines.
hub Canonical reference
Beyond ten turns: Unlocking long-horizon agentic search with large-scale asynchronous rl
Canonical reference. 80% of citing Pith papers cite this work as background.
26
Pith papers citing it
1
external citations · external index
Background
80% of classified citations
hub tools
citation-role summary
background 4
baseline 1
citation-polarity summary
representative citing papers
A new benchmark and clean-room harness show frontier AI agents reach only 0.337 factual F1 when synthesizing conclusions from scientific evidence.
HiViG is a test-time critic that combines macro-action history summarization with visual grounding of execution coordinates to reduce short-sighted and visually erroneous actions in long-horizon GUI agents.
AstraFlow decouples RL components into autonomous dataflow services to natively support multi-policy agentic LLM training, elastic scaling, and cross-region execution with 2.7x speedup on math, code, search, and AgentBench workloads.
ActGuide-RL uses human action data as plan-style guidance in mixed-policy RL to overcome exploration barriers in LLM agents, matching SFT+RL performance on search benchmarks without cold-start training.
Self-play’s natural question-construction paths supply free privileged information that turns sparse-reward self-play into dense self-distillation, yielding stronger data-free search agents.
PaperScope is a new multi-modal multi-document benchmark that evaluates AI agents on deep scientific research by requiring integration of evidence across multiple papers including figures and tables.
A verifiable offline Wikipedia tool environment plus iterative scaffold-to-ReAct self-distillation lets a 9B agent reach competitive BrowseComp/GAIA/HotpotQA scores without stronger-model distillation.
SearchEyes unifies multimodal search-agent training via Perception-Knowledge Chains on Wikidata5M and Hop-Anchored Policy Optimization, claiming a 6.2-point average gain over the strongest open-source baseline on six benchmarks.
BioInsight is a multi-agent orchestration system that converts disease-associated protein tables into interactive evidence dashboards with typed artifact contracts, improving reported traceability and ranking over baselines given the same retrieved evidence.
ReSum's contrastive RL branching on self-summarization points improves LLM math reasoning accuracy by about 4% and shortens rollouts by about 18.6% across tested backbones.
GDCR assigns step-level rewards via distance to the answer node in a training-time ER graph and SAPO combines these with trajectory advantages for credit assignment in agentic search.
SR²AM achieves competitive Pass@1 accuracy on diverse tasks with 25.8-95.3% fewer reasoning tokens than much larger models by using self-regulated simulative planning trained via supervised learning and RL.
Imagining in 360° decouples visual search into a single-step probabilistic semantic layout predictor and an actor, removing the need for multi-turn CoT reasoning and trajectory annotations while improving efficiency in 360° environments.
Agent-World autonomously synthesizes verifiable real-world tasks and uses continuous self-evolution to train 8B and 14B agents that outperform proprietary models on 23 benchmarks.
Survey that defines agentic RL for LLMs via POMDPs, introduces a taxonomy of planning/tool-use/memory/reasoning capabilities and domains, and compiles open environments from over 500 papers.
Scaling the delegation backbone in hierarchical search agents improves EM by ~11 points while scaling the executor moves EM by only ~2.6 points, and a 1.7B SFT executor matches a frontier sub-agent at 37% fewer tokens.
KbSD uses a same-size hint-augmented teacher and quadrant-adaptive KL objectives to deliver dense supervision for calibrated behavior across knowledge states in agentic search.
APPO improves LLM agent training by branching at tokens selected for both uncertainty and future impact, then scaling credit for consequential reasoning procedures.
SlimSearcher reduces tool-call rounds by 17-58% on GAIA, BrowseComp and XBenchDeepSearch while maintaining accuracy via Pareto filtration in SFT and Adaptive Reward Gating in RL.
CAPF improves Qwen3-4B exact-match scores from 44.7% to 48.5% on seven QA benchmarks by allowing privileged verifier feedback during RLVR training with attenuated credit for the feedback step.
ZipRL is a new RL-based adaptive compression method for multi-turn LLM agents that adds multi-granularity prompts and hindsight response replay to GRPO, reporting 27.9-34.7% gains on five agent tasks with maintained token efficiency.
Injecting 1% targeted synthetic data into GPT-2's pre-training substantially improves performance on 8 of 9 failing BLiMP grammatical paradigms, indicating data scarcity causes formal linguistic failures.
ERL trains LLMs to erase faulty reasoning steps and regenerate them in place, yielding gains of up to 8.48% EM on multi-hop QA benchmarks like HotpotQA.
citing papers explorer
-
GraphPO: Graph-based Policy Optimization for Reasoning Models
GraphPO represents reasoning rollouts as a DAG to merge semantically equivalent paths, share suffixes, and assign separate efficiency and correctness advantages for lower variance and better performance than chain or tree baselines.
-
Can AI Agents Synthesize Scientific Conclusions?
A new benchmark and clean-room harness show frontier AI agents reach only 0.337 factual F1 when synthesizing conclusions from scientific evidence.
-
A History-Aware Visually Grounded Critic for Computer Use Agents
HiViG is a test-time critic that combines macro-action history summarization with visual grounding of execution coordinates to reduce short-sighted and visually erroneous actions in long-horizon GUI agents.
-
AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMs
AstraFlow decouples RL components into autonomous dataflow services to natively support multi-policy agentic LLM training, elastic scaling, and cross-region execution with 2.7x speedup on math, code, search, and AgentBench workloads.
-
Learning Agentic Policy from Action Guidance
ActGuide-RL uses human action data as plan-style guidance in mixed-policy RL to overcome exploration barriers in LLM agents, matching SFT+RL performance on search benchmarks without cold-start training.
-
$\pi$-Play: Multi-Agent Self-Play via Privileged Self-Distillation without External Data
Self-play’s natural question-construction paths supply free privileged information that turns sparse-reward self-play into dense self-distillation, yielding stronger data-free search agents.
-
PaperScope: A Multi-Modal Multi-Document Benchmark for Agentic Deep Research Across Massive Scientific Papers
PaperScope is a new multi-modal multi-document benchmark that evaluates AI agents on deep scientific research by requiring integration of evidence across multiple papers including figures and tables.
-
DeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environment
A verifiable offline Wikipedia tool environment plus iterative scaffold-to-ReAct self-distillation lets a 9B agent reach competitive BrowseComp/GAIA/HotpotQA scores without stronger-model distillation.
-
SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulation
SearchEyes unifies multimodal search-agent training via Perception-Knowledge Chains on Wikidata5M and Hop-Anchored Policy Optimization, claiming a 6.2-point average gain over the strongest open-source baseline on six benchmarks.
-
BioInsight: Multi-Agent Orchestration for Interactive Biomedical Knowledge Discovery
BioInsight is a multi-agent orchestration system that converts disease-associated protein tables into interactive evidence dashboards with typed artifact contracts, improving reported traceability and ranking over baselines given the same retrieved evidence.
-
ReSum: Synergizing LLM Reasoning and Summarization with Reinforcement Learning
ReSum's contrastive RL branching on self-summarization points improves LLM math reasoning accuracy by about 4% and shortens rollouts by about 18.6% across tested backbones.
-
Beyond Trajectory Rewards: Step-level Credit Assignment for Agentic Search via Graph Modeling
GDCR assigns step-level rewards via distance to the answer node in a training-time ER graph and SAPO combines these with trajectory advantages for credit assignment in agentic search.
-
Efficient Agentic Reasoning Through Self-Regulated Simulative Planning
SR²AM achieves competitive Pass@1 accuracy on diverse tasks with 25.8-95.3% fewer reasoning tokens than much larger models by using self-regulated simulative planning trained via supervised learning and RL.
-
Beyond Thinking: Imagining in 360$^\circ$ for Humanoid Visual Search
Imagining in 360° decouples visual search into a single-step probabilistic semantic layout predictor and an actor, removing the need for multi-turn CoT reasoning and trajectory annotations while improving efficiency in 360° environments.
-
Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence
Agent-World autonomously synthesizes verifiable real-world tasks and uses continuous self-evolution to train 8B and 14B agents that outperform proprietary models on 23 benchmarks.
-
The Landscape of Agentic Reinforcement Learning for LLMs: A Survey
Survey that defines agentic RL for LLMs via POMDPs, introduces a taxonomy of planning/tool-use/memory/reasoning capabilities and domains, and compiles open environments from over 500 papers.
-
Think Big, Search Small: Where Capacity Matters in Hierarchical Search Agents?
Scaling the delegation backbone in hierarchical search agents improves EM by ~11 points while scaling the executor moves EM by only ~2.6 points, and a 1.7B SFT executor matches a frontier sub-agent at 37% fewer tokens.
-
KbSD: Knowledge Boundary aware Self-Distillation for Behavioral Calibration in Agentic Search
KbSD uses a same-size hint-augmented teacher and quadrant-adaptive KL objectives to deliver dense supervision for calibrated behavior across knowledge states in agentic search.
-
APPO: Agentic Procedural Policy Optimization
APPO improves LLM agent training by branching at tokens selected for both uncertainty and future impact, then scaling credit for consequential reasoning procedures.
-
SlimSearcher: Training Efficiency-Aware Web Agents via Adaptive Reward Gating
SlimSearcher reduces tool-call rounds by 17-58% on GAIA, BrowseComp and XBenchDeepSearch while maintaining accuracy via Pareto filtration in SFT and Adaptive Reward Gating in RL.
-
CAPF: Guiding Search-Agent Rollouts with Credit-Attenuated Privileged Feedback
CAPF improves Qwen3-4B exact-match scores from 44.7% to 48.5% on seven QA benchmarks by allowing privileged verifier feedback during RLVR training with attenuated credit for the feedback step.
-
ZipRL: Adaptive Multi-Turn Context Compression with Hindsight Response Replay
ZipRL is a new RL-based adaptive compression method for multi-turn LLM agents that adds multi-granularity prompts and hindsight response replay to GRPO, reporting 27.9-34.7% gains on five agent tasks with maintained token efficiency.
-
LiteResearcher: A Scalable Agentic RL Training Framework for Deep Research Agent
Injecting 1% targeted synthetic data into GPT-2's pre-training substantially improves performance on 8 of 9 failing BLiMP grammatical paradigms, indicating data scarcity causes formal linguistic failures.
-
Erase to Improve: Erasable Reinforcement Learning for Search-Augmented LLMs
ERL trains LLMs to erase faulty reasoning steps and regenerate them in place, yielding gains of up to 8.48% EM on multi-hop QA benchmarks like HotpotQA.
-
DocArena: Turning Raw Documents into Controllable Training Environments for Document Search Agents
DocArena automates creation of multimodal document QA training data via MLLM-based structuring and cross-page reasoning pairs, yielding agents with top retrieval and QA performance in unified tests.
-
A Survey of Reinforcement Learning for Large Reasoning Models
A survey compiling RL methods, challenges, data resources, and applications for enhancing reasoning in large language models and large reasoning models since DeepSeek-R1.