{"id":"f3f26c29-bde6-42c3-bf3e-fd1a0254378d","arxiv_id":"2608.10743","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A distilled context refiner plus refinement during RL training improves reliability and efficiency of multi-turn LLM search agents, with the latest retrieved documents identified as the main source of context interference.","lead":"This paper studies why multi-turn search agents that use large language models sometimes fail even when the right documents are retrieved, and finds that newly retrieved document chunks add the most distracting noise. It proposes a distilled context refiner and a reinforcement-learning pipeline that removes this noise, improving both answer accuracy and search efficiency.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No Search-GRPO-plus-refiner control means the claimed CRRL training benefit in Sec. 4.3 is confounded with inference-time context refinement; Table 2 shows inference-only refinement already yields most of the gain.","rationale":"Good-faith reading: the paper makes a useful empirical contribution in showing that context refinement at inference helps search agents, and the distilled context refiner is evaluated against reasonable baselines including GPT-Refine and self-refinement. The released code and the seven-benchmark evaluation are genuine evidence. The load-bearing weakness is not the refiner itself but the attribution of CRRL's improvement to the RL training procedure. The paper's own Table 2 shows the refiner alone adds 4.7 EM to IRCoT on Qwen2.5-7b, which is larger than the 2.0 EM CRRL advantage over Search-GRPO in Table 3. Therefore the headline claim that incorporating context refinement into RL training pipelines improves reliability is underdetermined by the reported experiments. This is an omitted control, not a mathematical error, and it can be fixed with one additional evaluation condition. The training-data inconsistency between Sec. 4.2 (40k samples for CRRL) and Appendix D (60k samples for training-based baselines) is a secondary fairness concern, but the missing Search-GRPO-plus-refiner condition is the decisive confound because it directly targets RQ iii. I agree with the reader's weakest assumption. The conditional verdict remains appropriate pending this control; if the control matches CRRL, the paper should be revised to claim only that context refinement improves search agents, not that training on refined contexts adds further gains.","tokens_in":23105,"tokens_out":4148,"duration_ms":38280,"concrete_test":"Run the released Search-GRPO checkpoint(s) with the same trained context refiner F applied at inference: for each turn, replace retrieved documents d_i with \\tilde{d}_i = F(q_{i-1}, d_i) before the policy generates the next action, keeping all other decoding and retrieval settings identical (K=3, same prompts, same max turns). Evaluate EM/ART/AIT on the seven benchmarks, with at least three seeds, and compare to Table 3 CRRL. If Search-GRPO+refiner matches CRRL within error bars, the claimed benefit of training on refined contexts is unsupported; if CRRL remains consistently better (e.g., >1 EM with non-overlapping error bars), the training component is validated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's RQ iii claim—that 'incorporating context refinement into the RL training pipelines of search agents' improves reliability and efficiency (Sec. 4.3)—is supported by comparing CRRL against Search-GRPO in Table 3. But CRRL's pipeline (Sec. 4.1) applies the context refiner F during rollouts and at inference, whereas Search-GRPO is both trained and evaluated on raw retrieved documents. The experiment therefore contrasts 'trained on refined contexts plus inference refinement' with 'trained on raw contexts plus no inference refinement,' a two-factor difference. Table 2 quantifies the inference-only factor: applying the refiner F to IRCoT raises average EM from 27.5 to 32.2 for Qwen2.5-7b and cuts ART from 2.6 to 1.2. The reported CRRL advantage over Search-GRPO is only 36.6 vs 34.6 EM and 1.6 vs 2.1 ART (Table 3). Thus inference-time refinement alone accounts for most of the observed gap, and no condition isolates the effect of training the policy on refined contexts. Without a Search-GRPO-plus-refiner control, the central training-pipeline conclusion is not identifiable. The absence of error bars or multiple seeds further means the remaining 2.0 EM gap could be within noise.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies context interference in multi-turn LLM search agents, addressing three questions: which parts of the context cause interference, how to refine contexts to mitigate it, and whether incorporating context refinement into RL training improves search agents. The authors report that masking previous retrieved documents, queries, and thinking steps yields only small gains, from which they conclude that the latest retrieved documents are the primary source of interference. They then distill a context refinement dataset from GPT-4 and train a small context refiner that extracts query-relevant information from retrieved documents. Finally, they integrate this refiner into a GRPO-based RL training pipeline (CRRL), where rollouts refine retrieved documents before the policy generates actions. Experiments on seven QA benchmarks with two Qwen models compare inference-time refinement methods and training-based methods, reporting improvements in Exact Match and reductions in average retrieval times, context length, and inference time. The paper also includes an analysis of recall rate versus recall accuracy and a discussion of data contamination.","tokens_in":23369,"tokens_out":5200,"duration_ms":50628,"significance":"If the conclusions hold, the paper provides a clear and actionable finding: in multi-turn search agents, the latest retrieved documents are the dominant source of context interference, and a distilled, relatively small context refiner can mitigate it at inference time and during RL training. The strengths are the breadth of evaluation (seven QA datasets, two model sizes, multiple efficiency metrics), the consistency of improvements in the inference-time refinement results (Table 2), the release of code, and the careful separation of training and test data. The paper also draws attention to an important problem—context interference in agentic search—that is less studied than in single-turn RAG. However, the central claim about the benefit of incorporating refinement into RL training is not cleanly isolated: the CRRL comparison in Table 3 varies both the training contexts and the inference-time context processing, and no control separates these factors. The causal attribution to 'latest documents' in Section 3.2 is also indirect.","major_comments":[{"comment":"The main training results are reported without variance or multiple seeds. The differences between CRRL and Search-o1 in Table 3 are small (36.6 vs 36.2 EM for Qwen2.5-7b; 31.5 vs 30.3 for Qwen2.5-3b), and the 2.0-point improvement over Search-GRPO on the 7b model could be within run-to-run noise, especially given that the Appendix D training details mention instability ('In cases where training diverges, we evaluate at the most recent stable checkpoint'). Reporting the mean and standard deviation over at least three seeds for the headline comparisons, or at least for CRRL and the strongest baselines, is necessary to support the 'significantly improve' language used in the abstract and Section 4.3.","section":"Sec. 4.3, Tables 3"}],"minor_comments":[{"comment":"Minor typo: 'close performance to GPT-Refine' should be 'close performance to GPT-Refine' or 'performance close to GPT-Refine's'; also 'a distill dataset' is more naturally 'a distilled dataset'.","section":"Sec. 3.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical study with a useful analysis of context interference in search agents, but the central claim about RL training benefits is not yet identifiable because of the missing Search-GRPO-plus-refiner control. The indirect causal attribution in Section 3.2 and the lack of variance reporting further weaken the current version. I would support publication after these issues are addressed, as the underlying idea and the inference-time refinement results are valuable. The authors should also be asked to clarify the training-data-size discrepancy in Appendix D, since it could give a misleading impression of the comparison."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper is worth engaging with. It gives a systematic diagnosis of where context interference comes from in multi-turn search agents, and it shows a distilled context refiner that matches GPT-4-driven refinement at a fraction of the cost. The cleanest result is that refining the latest retrieved documents before feeding them to the policy improves both exact-match accuracy and reduces retrieval counts on seven QA benchmarks. That is a real, reproducible contribution.\n\nThe masking experiments (IRCoT-o, -oq, -oqp) are a sensible way to attribute interference, and the conclusion that previous documents and queries contribute little relative to the latest observation is reasonably supported. The authors also check that their refined texts are entailed by the original documents, which is the right guard against hallucinated compression. The efficiency metrics (ART, context length, AIT) are more meaningful than accuracy alone for search agents.\n\nThe soft spots are real but mostly addressable. First, the 'latest documents are the primary cause' claim is inferred from the persistence of the recall gap after masking older context, not directly tested by masking the latest observation. An intervention that only refines the latest documents works, which is supportive, but it isn't the same as a causal test.\n\nSecond—and this is the main issue—the CRRL claim in Sec. 4.3 is not identifiable from the reported comparisons. CRRL applies the refiner during rollouts and at inference; Search-GRPO is trained and evaluated on raw contexts. That is a two-factor difference. Table 2 shows that applying the same refiner at inference to IRCoT already gives 32.2 EM vs 27.5 baseline, and the CRRL advantage over Search-GRPO is only 36.6 vs 34.6. Without a Search-GRPO-plus-refiner control, you cannot attribute the gain to training on refined contexts. The missing control matters because the training corpus sizes also differ (40k for CRRL, 60k for the baselines), and the paper admits that difference is left to a future revision. No error bars or multiple seeds are reported, so the remaining 2 EM gap could be noise.\n\nThese are not fatal flaws. The diagnostic framing and the refiner itself are solid, and the RL idea is plausible. But the paper needs one more experiment and some rigor additions before the central training-pipeline conclusion is supported. For peer review: yes, send it. The authors have done a careful study with a clear method, and the confound is addressable. A serious referee could ask for the missing control, error bars, and training-set-size ablation, and the paper would be stronger for it.","headline":"Solid empirical study of context interference in multi-turn search agents; the distilled refiner works, but the RL-training benefit is confounded with inference-time refinement.","tokens_in":23919,"tokens_out":3417,"would_cite":true,"duration_ms":30663,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A multi-turn search agent's accuracy and speed are improved by refining the newest retrieved documents, not the accumulated history, and the paper shows how to train a small refiner and fold it into reinforcement learning.","keywords":["context interference","multi-turn search agents","context refinement","reinforcement learning","retrieval-augmented generation","question answering","distillation","efficiency"],"falsifier":"Train Search-GRPO exactly as in the paper and then run it at inference with the context refiner switched on, without any refinement-aware training. If that control matches CRRL's exact-match scores, the paper's claim that refinement must enter RL training is not supported; if CRRL still wins, the training-time contribution is real.","tokens_in":22909,"feed_emoji":"🔍","tokens_out":6886,"duration_ms":59452,"temperature":0.7,"pith_summary":"Multi-turn search agents get long contexts, and the paper argues that the newest batch of retrieved documents is what distracts them most; earlier documents, earlier queries, and earlier reasoning are much less harmful. To remove that interference, the authors train a small context refiner that, given the current search query and the top documents, outputs only the query-relevant sentences, and they replace the full retrieved documents with that output at every turn. They then fold the refiner into reinforcement learning (CRRL), so training rollouts are also generated over refined contexts. Across seven single- and multi-hop question-answering benchmarks, the refined agents improve exact-match accuracy and need fewer retrieval calls, so the paper concludes that refine context and then generate is a workable recipe for cheaper, more reliable search agents.","feed_headline":"Refining latest retrieved docs lifts search-agent accuracy","feed_subtitle":"A small distilled refiner plus RL training improves answers and cuts retrieval calls on seven QA benchmarks.","key_machinery":"Two components carry the argument. The first is the context refiner $F$: a small instruction-tuned language model fine-tuned to map a search query $q_{i-1}$ and a retrieved document set $\\mathbf{d}_i$ to a short refined text $\\tilde{\\mathbf{d}}_i$ containing only the information relevant to answering $q_{i-1}$, trained on distilled teacher extractions that an entailment check keeps faithful to the source. The second is CRRL, a GRPO-based RL procedure whose rollouts use refined observations $\\tilde{\\mathbf{d}}_i$ instead of raw documents, so the policy learns search and reasoning behaviour on interference-free contexts while token-level loss masking keeps retrieved tokens out of the policy gradient. The refiner is what removes the noise identified in the causal analysis; CRRL is what makes refinement part of learning rather than a separate preprocessing step.","core_discovery":"Context interference in a multi-turn search agent is caused mainly by the latest observation—the top-$K$ documents returned by the most recent retrieval—rather than by older retrieved documents, past search queries, or old reasoning steps, which the paper shows by masking segments of the history one at a time. On that basis the authors introduce a context refiner $F$ trained by supervised fine-tuning on a distilled dataset of query-relevant extractions produced by a stronger teacher and verified by an entailment model to stay strictly inside the source documents. The refiner replaces each observation $\\mathbf{d}_i$ with $\\tilde{\\mathbf{d}}_i=F(q_{i-1},\\mathbf{d}_i)$, and the resulting cleaned contexts are used both at inference and inside GRPO rollouts in the proposed CRRL; the claim is that refinement during RL training further improves reliability (exact match) and efficiency (average retrieval times, context length, and inference time) over retrieval-based and RL baselines on all seven datasets.","pith_inferences":["The localization of interference to the latest retrieved documents suggests a cheap diagnostic for deployed agents: if accuracy collapses just after a retrieval turn, the problem is likely the newest documents, not the accumulated history.","The same distillation-plus-entailment recipe could transfer to other observation-heavy agent settings, such as tool outputs or code execution logs, but the interference source would have to be re-localized for each setting rather than assumed to be the latest observation.","A direct control that applies the refiner at inference time to the RL-trained baseline without refinement-aware training would separate the value of training on clean contexts from the value of reading clean contexts; this is the comparison the current table does not report.","If the refiner is also used at inference for CRRL, its efficiency gains may partly reflect a shorter effective context length rather than a better policy, which matters for how the method would port to agents with longer action budgets."],"forward_implications":["Search-agent accuracy can be improved by fixing the input context, not only by scaling the policy model.","Average retrieval calls drop, so per-question latency and cost fall even after accounting for the refiner's extra inference.","A weak model can approximate a strong teacher's context refinement via distillation, reducing dependence on expensive teacher calls at inference time.","Rollout quality in RL training matters enough that context hygiene during training yields better final policies than applying refinement only at inference."],"supporting_citations":[{"why":"Supplies the multi-turn search agent framework, special-token prompt format, and the Search-GRPO baseline that CRRL is compared against.","marker":"[Jin et al., 2025]"},{"why":"IRCoT interleaves retrieval with chain-of-thought; it is the prompting baseline and the inference protocol used to generate trajectories for distillation.","marker":"[Trivedi et al., 2023a]"},{"why":"IRCoT variant used when running the teacher model to produce the step-wise extracted data for the context refinement dataset.","marker":"[Trivedi et al., 2023b]"},{"why":"GRPO is the underlying group-relative policy optimization that CRRL extends with refined context rollouts.","marker":"[Shao et al., 2024]"},{"why":"Rejection fine-tuning (RFT) is the retrieval-based training baseline; CRRL must beat its reliability and efficiency numbers.","marker":"[Yuan et al., 2023]"},{"why":"Introduces the in-context interference phenomenon the paper operationalizes for multi-turn search agents.","marker":"[Coleman et al., 2023]"},{"why":"Provides the 2018 Wikipedia dump as the external knowledge base used for all retrievals, load-bearing for the recall-rate analysis.","marker":"[Karpukhin et al., 2020]"}],"fun_headline_variants":["Latest docs cause search-agent confusion; refiner fixes it","Distilled refiner trims latest retrieval noise for smarter agents","Search agents lose focus on newest docs; refiner restores it","Refine newest retrieval to boost search-agent accuracy and speed","Context refiner cuts irrelevant latest docs from multi-turn search"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central comparison assumes that CRRL's gains come from refinement during RL training, since the refiner is switched on at CRRL inference but not at the baseline's inference, and no baseline-plus-refiner control is reported.","fun_headline_variants_meta":{"raw":{"variants":["Latest docs cause search-agent confusion; refiner fixes it","Distilled refiner trims latest retrieval noise for smarter agents","Search agents lose focus on newest docs; refiner restores it","Refine newest retrieval to boost search-agent accuracy and speed","Context refiner cuts irrelevant latest docs from multi-turn search"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000198,"raw_usage":{"total_tokens":1372,"prompt_tokens":950,"completion_tokens":422,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":338}},"tokens_in":566,"tokens_out":422,"duration_ms":4436,"temperature":1.0,"reasoning_tokens":338,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:12:47.029891+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train Search-GRPO exactly as in the paper and then run it at inference with the context refiner switched on, without any refinement-aware training. If that control matches CRRL's exact-match scores, the paper's claim that refinement must enter RL training is not supported; if CRRL still wins, the training-time contribution is real.","supporting_citations":[],"review_version":1}