{"id":"a3874483-226c-44a6-9fdc-986324510ae4","arxiv_id":"2605.12913","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"DAgger-style training with turn-level policy interpolation raises 4B and 8B LLM agents to 27.3% and 29.8% on SWE-bench Verified, beating several larger published systems.","lead":"The paper adapts the classic DAgger algorithm to train LLM-based agents on long-horizon tasks such as software engineering by mixing student and teacher actions at each turn and then training the student with teacher labels. A smart reader should care because it offers a practical middle path between the covariate-shift problems of pure supervised fine-tuning and the sparse-reward problems of pure reinforcement learning for agent training.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Mixed student-teacher trajectories risk new state-distribution shifts if student errors leave the environment in states outside the teacher's training distribution","rationale":"The reader's weakest assumption correctly isolates the state-continuity requirement that must hold for the interpolation to avoid reintroducing distribution shift. Because the reported gains rest on this unverified property and no ablations or state-distribution diagnostics are described, the CONDITIONAL verdict remains appropriate.","tokens_in":1813,"tokens_out":309,"duration_ms":32800,"concrete_test":"Re-run the DAgger collection loop on a 100-trajectory subset of SWE-bench, once with full state persistence after every student turn and once with an oracle reset to the teacher-only prefix after each student turn; if the final student performance gap between the two variants exceeds 2 points, the mitigation of covariate shift cannot be attributed solely to interpolation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that turn-level policy interpolation mitigates covariate shift by exposing the student to realistic states while still supplying dense teacher labels. This holds only if a student action at turn t does not corrupt the persistent environment state (e.g., a broken code edit in SWE-bench) in a way that makes the subsequent teacher trajectory off-distribution relative to both pure-teacher and pure-student rollouts. The paper provides no analysis or ablation of state continuity after mixed actions, nor any metric showing that the visited state distribution under interpolation is closer to deployment than SFT baselines.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper revisits DAgger for long-horizon LLM agents by collecting trajectories via turn-level interpolation between student and teacher policies, then training the student via supervised imitation on teacher labels. This is claimed to combine dense supervision (avoiding sparse RL rewards) with on-policy exposure to realistic states (mitigating covariate shift from pure SFT). Experiments on SWE-bench Verified report +3.9 point gains for a 4B model (reaching 27.3%) and +3.6 points for an 8B model (reaching 29.8%), outperforming several larger published baselines, with consistent gains on a held-out SWE-Gym split.","tokens_in":1966,"tokens_out":551,"duration_ms":51088,"significance":"If the results hold, the work supplies a practical, low-overhead recipe for improving multi-turn agent training that avoids the full machinery of RL while still addressing distribution shift. The empirical outperformance of larger models by smaller DAgger-trained agents on a standard benchmark is noteworthy and could influence post-training pipelines for agentic LLMs.","major_comments":[{"comment":"§4 (Experiments) and Table 1: the reported +3.9 / +3.6 point gains on SWE-bench Verified are presented without error bars, multiple random seeds, or statistical tests; given that the central claim rests on these numeric improvements over the strongest baseline, the absence of variance estimates leaves the reliability of the result unclear.","section":"§4 (Experiments) and Table 1"},{"comment":"§3.2 (DAgger for LM Agents): the claim that turn-level interpolation mitigates covariate shift by exposing the student to realistic states assumes that a student action at turn t does not corrupt persistent environment state in a way that renders subsequent teacher actions off-distribution; no ablation, state-distribution metric, or continuity analysis is provided to support this assumption, which is load-bearing for the core argument.","section":"§3.2 (DAgger for LM Agents)"}],"minor_comments":[{"comment":"The abstract and §4.1 should explicitly name the strongest post-training baseline and the exact interpolation probability schedule used, as these details are needed to reproduce the claimed gains.","section":"Abstract and §4.1"},{"comment":"Notation for the interpolation probability (mentioned as a free parameter) is introduced without a clear equation or pseudocode block; adding a short algorithm box would improve clarity.","section":"§3.2"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback. We address each major comment below and indicate where revisions will be made to strengthen the manuscript.","responses":[{"response":"We agree that variance estimates and statistical tests would improve the reliability assessment of the reported gains. In the revised manuscript we will rerun the key 4B and 8B experiments with three random seeds, report mean and standard deviation in Table 1, and include a brief statistical significance note (paired t-test against the strongest baseline).","revision_made":"yes","referee_comment":"[§4 (Experiments) and Table 1] §4 (Experiments) and Table 1: the reported +3.9 / +3.6 point gains on SWE-bench Verified are presented without error bars, multiple random seeds, or statistical tests; given that the central claim rests on these numeric improvements over the strongest baseline, the absence of variance estimates leaves the reliability of the result unclear."},{"response":"The turn-level schedule ensures the teacher intervenes after every student action, limiting state drift to a single step; because the teacher then restores the trajectory toward its own distribution, subsequent states remain close to the teacher policy’s support. We will expand §3.2 with a short continuity argument and add an appendix figure comparing state-feature histograms (e.g., file-system and repository state embeddings) between pure-teacher and interpolated trajectories to quantify the limited divergence.","revision_made":"partial","referee_comment":"[§3.2 (DAgger for LM Agents)] §3.2 (DAgger for LM Agents): the claim that turn-level interpolation mitigates covariate shift by exposing the student to realistic states assumes that a student action at turn t does not corrupt persistent environment state in a way that renders subsequent teacher actions off-distribution; no ablation, state-distribution metric, or continuity analysis is provided to support this assumption, which is load-bearing for the core argument."}],"tokens_in":1539,"tokens_out":424,"duration_ms":50790,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's core result is that turn-level interpolation between student and teacher policies during data collection improves LLM agents on long-horizon coding tasks. They report +3.9 points for the 4B model and +3.6 for the 8B on SWE-bench Verified, with the smaller agent beating some published 8B systems and both showing gains on a held-out split. This is a direct, practical adaptation of classic DAgger rather than a new algorithm, and the numbers are the part worth paying attention to because they come from a real benchmark with consistent scaling behavior across two model sizes.","headline":"DAgger with turn-level mixing gives measurable gains on SWE-bench for 4B/8B agents, but the state-distribution claim after mixed actions still needs direct checks.","tokens_in":2448,"tokens_out":203,"would_cite":true,"duration_ms":33877,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[],"headline":"DAgger mixture rollouts address covariate shift in agent training with no overlap to RS recognition-cost or forcing chain","alignment":"orthogonal","rationale":"The paper's central machinery (turn-level policy interpolation μ_i = β_i π_e + (1-β_i) π_θ, teacher-labeled states from mixed trajectories, cross-entropy training on d^turn_i or d^traj_i) is standard imitation learning for mitigating distribution shift in long-horizon agents. It invokes no J-cost functional, no ratio-symmetric cost J(x) = ½(x + x⁻¹) − 1, no φ-ladder, no 8-tick periodicity, and no parameter-free derivation of constants. The domain (cs.LG post-training of LLM agents on SWE tasks) lies entirely outside the RS forcing chain from one distinction to spacetime and constants.","tokens_in":57607,"confidence":"high","tokens_out":194,"duration_ms":12943,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"DAgger with turn-level interpolation mitigates covariate shift in multi-turn LLM agents while retaining dense teacher supervision.","keywords":["DAgger","LLM agents","covariate shift","SWE-bench Verified","software engineering agents","multi-turn interaction","policy interpolation","supervised fine-tuning"],"falsifier":"A direct comparison showing that training on purely student-generated trajectories or purely teacher-generated trajectories yields smaller gains on SWE-bench Verified than the interpolated version.","tokens_in":2735,"feed_emoji":"🛠️","tokens_out":479,"duration_ms":52194,"temperature":0.7,"pith_summary":"The paper establishes that long-horizon LM agents face a core tradeoff: supervised fine-tuning supplies dense teacher labels but trains on off-policy trajectories that differ from deployment states, while reinforcement learning uses on-policy rollouts yet receives only sparse outcome signals. Revisiting DAgger addresses this by collecting trajectories via turn-level mixing of student and teacher actions, then training the student to match the teacher on those same trajectories. This exposes the model to realistic states likely to arise during actual use and still supplies rich step-by-step guidance. The approach is demonstrated on software-engineering agents at 4B and 8B scales, where it improves over the strongest post-training baseline by several points on SWE-bench Verified and on a held-out split.","feed_headline":"DAgger training gives 3.9 point boost to 4B LLM agents on SWE-bench","feed_subtitle":"Turn-level mixing of student and teacher actions supplies realistic states plus dense labels, outperforming pure SFT and RL baselines.","key_machinery":"Turn-level interpolation of student and teacher policies inside the DAgger loop, which generates mixed trajectories for subsequent supervised training on teacher labels.","core_discovery":"Collecting trajectories through turn-level interpolation of student and teacher policies, then training the student by mimicking the teacher on those trajectories, allows the model to encounter realistic environment states while still receiving dense supervision, thereby mitigating covariate shift that arises in pure supervised fine-tuning of multi-turn LM agents.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["DAgger improves 4B agent by 3.9 points on SWE-bench","4B agent reaches 27.3% on SWE-bench with DAgger","8B agent reaches 29.8% on SWE-bench with DAgger","DAgger supplies realistic states and dense labels to agents"],"cache_read_input_tokens":64,"weakest_assumption_plain":"A reliable teacher policy remains available and affordable to query at every training step, and the environment can continue or reset after mixed student-teacher actions without creating new distribution shifts.","fun_headline_variants_meta":{"raw":{"variants":["DAgger improves 4B agent by 3.9 points on SWE-bench","4B agent reaches 27.3% on SWE-bench with DAgger","8B agent reaches 29.8% on SWE-bench with DAgger","DAgger supplies realistic states and dense labels to agents"]},"model":"grok-4.3","cost_usd":0.009515,"raw_usage":{"total_tokens":4210,"prompt_tokens":754,"num_sources_used":0,"completion_tokens":82,"cost_in_usd_ticks":95153000,"prompt_tokens_details":{"text_tokens":754,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3374,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":754,"tokens_out":82,"duration_ms":45222,"temperature":1.0,"reasoning_tokens":3374,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-14T19:50:59.965269+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A direct comparison showing that training on purely student-generated trajectories or purely teacher-generated trajectories yields smaller gains on SWE-bench Verified than the interpolated version.","supporting_citations":[],"review_version":1}