{"id":"c8caf90e-6099-4d36-b290-4273e6b8dea5","arxiv_id":"2608.08768","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"BOUND distills teacher-constructed corrective preferences at decision states where a locally plausible continuation would reinforce search drift, gaining up to 5.6 EM on Bamboogle and 4.8 accuracy on BrowseComp-Plus over Trajectory SFT.","lead":"BOUND is a training method that helps LLM search agents avoid drifting away from the original question by having a teacher write short summaries of each search state and use them to create corrective preference examples. It reports consistent accuracy gains over trajectory imitation on several multi-hop question answering and deep-search benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The no-observation teacher judgment at Algorithm 1 lines 3-4 is the load-bearing premise; with only a 50-state self-audit, mislabeled local preferences could explain the gains without genuine drift correction.","rationale":"I agree with the reader that the weakest assumption is the reliability of the teacher's local error judgment. The manuscript's own Appendix B limits direct validation to 50 states, and the audit's drift-status labels are not tied to executed outcomes, so the premise that every retained preference is genuinely preferable is unverified. The ablations (Table 3, Table 6, Figure 3) support the mechanism internally, but they compare variants of the same teacher pipeline and cannot rule out that the teacher's corrections are effective only because they encode parametric priors about what good search looks like. The single-run Table 1 numbers further increase the risk that the headline gains are not stable. I do not see a contradiction or a hidden circularity: the method is a valid empirical recipe, and the concern is about label quality and external validation, which is exactly what the reader's CONDITIONAL verdict should require. The execution-based test above would settle whether the preferences are actually preferable, so no adjustment to the reader's verdict is needed.","tokens_in":21481,"tokens_out":5528,"duration_ms":64613,"concrete_test":"Sample 150 corrective preference pairs from D_pref. From the recorded state s_t, re-execute each pair's original continuation and teacher correction in the real retrieval environment with the Reroute transition from Appendix D.4, then continue both rollouts with the initial student policy and score the final answer. If the corrected rollouts do not yield significantly higher answer accuracy (or, at minimum, better next-state evidence sufficiency) than the original rollouts, the no-observation teacher labels are not reliable and the central drift-correction explanation is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the teacher can reliably assess the student's original continuation from only the decision-time state s_t, the generated brief b_t, and the continuation y0_t (Algorithm 1 lines 3-4; Section 3.3.1). The observation produced by the continuation is deliberately excluded (Appendix D.1), so an 'erroneous' label is a counterfactual prediction about whether the continuation will fail to advance the search, not a label verified against the retrieved evidence. If the teacher mislabels a locally useful continuation as erroneous, or prefers a correction that, under the actual retriever and the Reroute transition (Appendix D.4), would not retrieve the missing evidence, DPO is trained on false preferences. The only direct check is a 50-state manual audit by the authors (Appendix B, Table 8), which does not execute corrections and reports 94% drift-status accuracy on a small, self-annotated sample. Reported gains could therefore come from the teacher's parametric knowledge or a generic preference for well-formed queries rather than from the drift-correction mechanism the paper claims. This premise is load-bearing because every retained pair in D_pref inherits the teacher's no-observation judgment.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BOUND, a training-time preference-distillation method for LLM-based deep search agents. BOUND collects rollouts from a student policy, constructs a teacher-side 'search-state brief' for each decision-time state, and uses a teacher model to assess whether the student's continuation contains a local search-control error. Depending on the rollout outcome, it then constructs state-matched preference pairs: a corrective pair contrasting a student-specific correction with the original continuation in unsuccessful rollouts, or a termination pair contrasting an evidence-supported answer with an unnecessary retrieval in successful rollouts. These pairs are distilled into the student via DPO, with the brief and teacher computation used only at training time. The method is evaluated on HotpotQA, MuSiQue, 2WikiMultiHopQA, Bamboogle, FRAMES, GAIA, and BrowseComp-Plus, and the paper reports that BOUND outperforms Trajectory SFT by 5.6 EM on Bamboogle and 4.8 accuracy on BrowseComp-Plus under matched settings.","tokens_in":21698,"tokens_out":3263,"duration_ms":38834,"significance":"If the reported results hold, BOUND makes a useful contribution: it addresses a real failure mode in multi-step search (persistent drift) with an on-policy, state-matched preference construction procedure, rather than relying on full teacher trajectories or coarse outcome rewards. The paper has clear strengths: the controlled comparison with Trajectory SFT in Table 4 shares teacher, student, retriever, and evaluation protocol; the ablations in Table 3 and Figure 3 isolate specific components; the decontamination audit in Appendix C is careful; and the authors provide code. The central risk is not circularity or leakage but measurement reliability and the validity of the teacher's no-observation local-error judgment. The paper's main claims rest on single-run benchmark numbers with no variance or significance testing, and the validation of the teacher's load-bearing judgment is a small, author-conducted audit. These issues are fixable and do not invalidate the approach, but they currently limit the confidence in the reported gains.","major_comments":[{"comment":"The central claim that BOUND 'achieves the best performance on five of the six datasets' is based on single-run numbers with no error bars, confidence intervals, or significance tests. Table 1 does not report how many seeds or rollouts were used, and no p-values are given for any comparison in that table. Because many reported differences between BOUND and the next-best baseline are only a few points (e.g., HotpotQA EM 42.4 versus 41.1 for R1-Searcher, MuSiQue F1 26.3 versus 24.7), the claim of consistent superiority is not statistically supported. I request repeated evaluation with multiple seeds (or bootstrap intervals) and significance tests for the main comparisons, or a clear statement of which differences are robust.","section":"Section 4.1, Table 1"},{"comment":"The load-bearing premise is that the teacher can reliably identify a correctable local search-control error from only the student-visible state, the generated brief, and the original continuation, without seeing the observation that the continuation produces. The manuscript states this explicitly in Section 3.3.1: 'The observation produced by the continuation is excluded, so the assessment relies only on information available at decision time.' The only direct check is a manual audit of 50 states by the authors, reported in Appendix B, Table 8, with 94 percent drift-status accuracy and 11 of 13 drift cases detected. This sample is small, self-annotated, and does not execute the corrections to verify that they retrieve the missing evidence under the actual rerouting transition. If the teacher mislabels locally appropriate continuations as erroneous, or if the preferred correction does not actually retrieve better evidence under the Reroute execution in Appendix D.4, the DPO training signal would be wrong. I recommend additional validation: execute a sample of teacher corrections in the environment and compare the resulting evidence and outcomes against the original continuations, and independently annotate a larger sample of teacher labels to measure inter-annotator agreement.","section":"Section 3.3.1, Algorithm 1 lines 3-4; Appendix B Table 8"},{"comment":"The controlled Trajectory SFT comparison is the strongest evidence for the method, but the statistical claim is reported only for one metric: 'The BrowseComp-Plus accuracy gain is statistically significant (p<0.001).' No significance test is reported for the Bamboogle EM/F1 gains (5.6 and 6.5 points, respectively) or for the BrowseComp-Plus recall gain. Since the abstract and Section 4.4 highlight these numbers, please clarify whether the other gains are also statistically significant, or report them as non-significant trends. At minimum, give the test procedure (number of samples, test type) used for the p-value.","section":"Section 4.4, Table 4"}],"minor_comments":[{"comment":"The table omits standard errors or variance, which makes it hard to assess the stability of all comparisons; adding a note about the number of evaluation runs would help readers interpret the percentages.","section":"Table 1 and Section 4.1"},{"comment":"The bar chart reports four strategies without error bars or significance tests; adding confidence intervals or at least a statement about single-run evaluation would improve interpretability.","section":"Figure 3"},{"comment":"The manual audit is described as being done by the authors; since the audit is used to support the teacher-assessment quality, it would be better to have independent annotation or a second rater with agreement statistics, even on a small sample.","section":"Appendix B"},{"comment":"The definition of 'search-control boundary' is distributed across Section 3.1, Section 3.3.2, and Appendix D.1; a concise formal definition placed near its first use would reduce ambiguity.","section":"Section 3.3.2"},{"comment":"The abstract says '12 of 14 metrics' and Section 4.2 says 'best performance on five of the six datasets'; consider aligning the phrasing and noting the 2Wiki exception explicitly in the abstract for full transparency.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The paper is well within scope for the journal and the code release and decontamination audit are commendable. My main concern is that the headline claims overstate the evidence given the absence of variance and significance testing, and the teacher's no-observation label validity is only weakly checked. Both issues are addressable with additional experiments and do not appear to require a fundamentally different method. I would support acceptance after the measurement and teacher-validation concerns are substantively addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nBOUND is a competent, well-controlled empirical paper on a real failure mode in LLM search agents: persistent drift where locally plausible retrieval and reasoning steps reinforce a wrong anchor, a dropped constraint, or a displaced topic. The headline result—gains over Trajectory SFT and existing 7B baselines under matched settings—is credible, and the paper deserves a serious referee look.\n\nWhat's actually new is the structured teacher-side search-state brief (target, constraints, confirmed evidence, missing information, drift status) used as privileged information to build state-matched DPO pairs over student-induced states, with corrective contrasts in failed rollouts and termination contrasts in successful ones. CSO and DAS cover adjacent ground, but the combination of a brief-anchored assessment with student-specific corrections is not in the cited literature. The paper does the controlled work: Table 4 isolates Trajectory SFT under the same interface; Table 3 ablates each component; Table 6 shows the brief helps with two different teachers. That is real evidence for the mechanism, not just a stacked recipe.\n\nThe soft spots are real but proportionate. The load-bearing premise is the teacher's no-observation judgment (Algorithm 1, lines 3–4): the teacher labels the student's continuation as erroneous or not without seeing the observation it produces. That is a counterfactual prediction about whether the continuation will advance the search, not a verified label. The only direct check is a 50-state self-audit (94% drift-status accuracy) that does not execute the corrections. Mislabeled preferences would still train DPO, and some of the gain could come from the teacher's parametric knowledge rather than genuine drift correction. That said, the brief/no-brief ablation and the teacher-vs-student correction ablation suggest the signal is meaningful, not just noise. The other gap is statistical: Table 1 reports single-run numbers without variance or significance tests for most metrics. That is common in this area, but it means the headline margins could move on reseeding.\n\nThe citation pattern is fine. The paper builds on its own group's SimpleDeepSearcher and R1-Searcher, but the comparisons are transparent, and the 2Wiki exception is fairly explained by training-data inclusion. Training data is public or synthetic with a contamination audit, and code is released.\n\nBottom line: this is a paper for people working on search-agent post-training. It does not reorganize the field, but it is a solid, reproducible recipe with a clear mechanism. I would send it to review. The main revision requests should be repeated-seed results with error bars and a more substantial verification of the teacher's error labels—ideally by executing corrections on a few hundred states and checking whether Reroute actually retrieves the missing evidence.","headline":"BOUND is a well-controlled, credible recipe for correcting persistent search drift in LLM agents; the main risks are the unverified no-observation teacher judgments and single-run headline numbers.","tokens_in":22260,"tokens_out":3340,"would_cite":true,"duration_ms":33955,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Persistent search drift in LLM-based search agents can be corrected by distilling teacher-built preferences at decision-time search-control boundaries, using a brief that pins the original target and constraints.","keywords":["persistent search drift","search control","preference distillation","direct preference optimization","LLM search agents","deep search","multi-hop question answering","retrieval-augmented generation"],"falsifier":"Take a held-out set of a few hundred student decision-time states, let the student execute each continuation, and record whether the next observation actually pushes the search away from the target and constraints. Compare the teacher's error flags (made without that observation) against this ground truth. The central claim would be falsified if the teacher's precision on drift-producing states is near chance, or if retraining the student on the same pairs but with the 'correction' replaced by a randomly chosen alternative continuation of similar quality retains the benchmark gains.","tokens_in":21245,"feed_emoji":"🧭","tokens_out":7723,"duration_ms":74147,"temperature":0.7,"pith_summary":"BOUND claims that the persistent search drift seen in LLM-based search agents — wrong-anchor drift, dropped constraints, and local-topic drift — can be suppressed by distilling teacher-built preferences at decision-time \"search-control boundaries.\" The core assertion is that a teacher can decide, from only the student-visible state plus a structured \"search-state brief\" that fixes the original target and constraints, whether the student's next move is a correctable local error, and then contrast that move with a student-specific correction (or, on successful rollouts, contrast an evidence-supported answer with an unnecessary further retrieval). These state-matched pairs are distilled into the student with direct preference optimization (DPO), while the brief and teacher stay in training. If correct, the recipe is a training-time fix that transfers to held-out deep-search tasks, with the reported gains being 5.6 EM points on Bamboogle and 4.8 accuracy points on BrowseComp-Plus over Trajectory SFT.","feed_headline":"Brief-guided training beats trajectory imitation for LLM search","feed_subtitle":"Corrective preferences at decision-time boundaries lead on five of six deep-search benchmarks","key_machinery":"The search-state brief: a five-field, teacher-only summary generated from the student-visible state (question, action history, evidence context) that pins the original search target and key constraints while recording confirmed evidence, missing information, and drift status. It does the work of giving the teacher a stable task-level reference so that assessment and correction are anchored to what should stay fixed, rather than to a potentially drifted context. The second piece is the state-matched preference pair — chosen and rejected continuations sharing the same state — which isolates the local search-control decision; DPO then pushes the student toward the chosen side.","core_discovery":"At the paper's center is a claim about what makes search drift persist: a continuation can be locally plausible — a reasonable query that retrieves relevant-looking evidence — while reinforcing a wrong anchor, a dropped constraint, or a displaced topic, so neither trajectory imitation nor outcome reward cleanly supervises the misstep. BOUND's discovery is that a teacher-side \"search-state brief\" (original search target, key constraints, confirmed evidence, missing information, drift status) lets the teacher separate locally plausible from task-aligned continuations using only the decision-time state, and that pairing the student's actual continuation with a brief-guided correction — or with an unnecessary retrieval step when the answer is already supported — yields preference pairs that survive validation and improve the student after DPO training. The paper reports that this transfer holds under shared retrieval and inference settings: best performance on five of six datasets, and consistent gains over a trajectory-SFT control trained with the same teacher, questions, and search-control interface.","pith_inferences":["One implication the paper leaves implicit: the density of corrective pairs on a dataset should predict how much BOUND helps; measuring performance against the fraction of states flagged as drifted would test whether the preference signal, rather than the teacher's parametric knowledge, drives the gains.","The observation-exclusion design suggests teacher judgments should degrade as drift deepens; a controlled study varying drift magnitude would clarify the boundary of the brief's usefulness.","BOUND's local corrective preferences and outcome-level RL (e.g., reward on final answer) plausibly complement each other: local DPO to stop drift, outcome RL to allocate global credit, with the rerouting transition as a shared action space.","The 'unnecessary retrieval' contrast could be extended to rollouts that end in failure but contain a late evidence-supported answer, possibly teaching earlier stopping when the student is about to go astray."],"forward_implications":["If BOUND is right, a training-time brief is enough: the student learns to avoid drift with no inference-time privileged information.","The corrective contrast targets the student's own misstep rather than a generic teacher move, so the signal is specific to errors that actually occur on the student's rollout distribution.","Termination preferences on successful rollouts teach the student to stop when evidence suffices, which should reduce oversearching and unnecessary retrieval cost.","The rerouting ablation implies that explicit re-anchoring — dropping passages tied to the abandoned search direction — is what makes the correction effective, not just the rewritten query.","The gains transfer across benchmarks (Bamboogle, BrowseComp-Plus, FRAMES, GAIA) and across two teachers, suggesting the mechanism generalizes beyond the training distribution."],"supporting_citations":[{"why":"Direct preference optimization (DPO) is the distillation objective that converts the constructed preference pairs into student policy updates.","marker":"[26]"},{"why":"ReAct-style iterative search defines the baseline interactive search loop and action vocabulary that BOUND's search-control interface extends.","marker":"[42]"},{"why":"Search-R1 is a trained RL search baseline that BOUND must outperform under shared retrieval and inference settings.","marker":"[11]"},{"why":"R1-Searcher is a trained RL baseline that, unlike BOUND, incorporates 2Wiki questions into its training data; BOUND's comparison against it supports transfer claims.","marker":"[30]"},{"why":"SimpleDeepSearcher supplies a trajectory-synthesis baseline and the GAIA validation-subset setting that BOUND follows.","marker":"[31]"},{"why":"ORBIT-4B is the comparably sized trained baseline; BOUND's consistent gains over it support the preference-based approach over outcome-level RL at equal scale.","marker":"[33]"},{"why":"BrowseComp-Plus is the fixed-corpus deep-search benchmark used to measure transfer of the learned search-control behavior.","marker":"[3]"},{"why":"HotpotQA supplies part of the training questions from which student rollouts and corrective preference pairs are constructed.","marker":"[41]"},{"why":"MuSiQue supplies the other part of the training questions used for corrective preference construction.","marker":"[34]"},{"why":"DeepSeek-V4-Flash acts as the teacher model performing brief-guided assessment and correction during preference construction.","marker":"[6]"}],"fun_headline_variants":["Corrective preference pairs at decision boundaries cut search drift","Brief-guided training beats trajectory imitation on five datasets","LLM search drift fixed by teacher briefs and DPO preference pairs","BOUND: brief-guided contrasts top trajectory SFT in deep search","Search error classification via briefs improves LLM agent training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the teacher can tell, from just the student-visible state, the constructed brief, and the student's proposed next move — without seeing what that move retrieves — whether the move is a correctable local error rather than a reasonable step; if that judgment is wrong, the DPO pairs teach the student bad preferences and the reported gains could come from noise or from the teacher's own parametric knowledge.","fun_headline_variants_meta":{"raw":{"variants":["Corrective preference pairs at decision boundaries cut search drift","Brief-guided training beats trajectory imitation on five datasets","LLM search drift fixed by teacher briefs and DPO preference pairs","BOUND: brief-guided contrasts top trajectory SFT in deep search","Search error classification via briefs improves LLM agent training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000796,"raw_usage":{"total_tokens":3547,"prompt_tokens":1031,"completion_tokens":2516,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":647,"completion_tokens_details":{"reasoning_tokens":2433}},"tokens_in":647,"tokens_out":2516,"duration_ms":21547,"temperature":1.0,"reasoning_tokens":2433,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:25:03.339593+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of a few hundred student decision-time states, let the student execute each continuation, and record whether the next observation actually pushes the search away from the target and constraints. Compare the teacher's error flags (made without that observation) against this ground truth. The central claim would be falsified if the teacher's precision on drift-producing states is near chance, or if retraining the student on the same pairs but with the 'correction' replaced by a randomly chosen alternative continuation of similar quality retains the benchmark gains.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ORBIT-4B is the comparably sized trained baseline; BOUND's consistent gains over it support the preference-based approach over outcome-level RL at equal scale."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DeepSeek-V4-Flash acts as the teacher model performing brief-guided assessment and correction during preference construction."}],"review_version":1}