{"arxiv_id":"2608.21946","title":"EDGE: Experience-Distillation for Guided Exploration in Agentic Reinforcement Learning","abstract":"Reinforcement learning with outcome-based objectives such as GRPO enables LLM-based agents to solve complex, long-horizon tasks, yet the reusable exploration patterns embedded in interaction trajectories are largely discarded after a single policy update. Existing experience-augmented approaches retrieve historical guidance at inference time, but they apply experiences without accounting for the policy's evolving capability and create persistent dependencies on external retrieval. We propose EDGE (Experience-Distillation for Guided Exploration), a framework that treats retrieved experiences as temporary training-time scaffolds and progressively internalizes their benefits into the parametric policy. Concretely, EDGE partitions each rollout group into experience-conditioned and experience-free trajectories to estimate and admit only positive marginal gains without extra sampling, then distills the induced behavior into the base policy via a reverse-KL objective on its own empirical support. A co-evolutionary experience bank further synthesizes guidance from emerging failure modes and prunes obsolete entries as the policy evolves. Across embodied, web, and search-based QA tasks, EDGE improves over strong RL baselines by up to 12.5 points and remains effective without inference-time scaffolds or a proprietary reflector. The code is available at https://github.com/xvolcano02/EDGE.","primary_cat":"cs.CL","cross_cats":["cs.AI","cs.LG"],"submitted_at":"2026-08-22T13:16:25+00:00","latest_version":2,"source":"arxiv","source_label":"arXiv","source_url":null,"has_pdf":true,"has_source":false,"verdict":"CONDITIONAL","confidence":"MODERATE","one_line_summary":"EDGE frames experience reuse as a dynamic scaffold-to-parameter transition, gating distillation on measured marginal gain and evolving an experience bank, improving agentic RL across three task families.","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","upvotes":0,"downvotes":0,"pith_score":0.0,"inbound_count":5,"_links":{"html":"https://pith.science/paper/2608.21946","verdict":"/api/v1/verdicts/2608.21946"}}