{"id":"9b615205-2045-46d4-a218-514140e669fb","arxiv_id":"2610.07898","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Failure-conditioned RL, with per-attempt rewards and active-set advantages, improves SWE-bench Verified resolution over GRPO for a 4B coding agent.","lead":"A new training method makes coding agents learn from their failed patches by feeding the error messages into the next attempt. On a 500-task benchmark, it raises first-try success from 38.9% to 41.7% and two-try success from 48.5% to 52.8%.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"FC-SWE's reported checkpoint was selected on a 100-task subset of the 500-task SWE-bench Verified benchmark, with no equivalent selection rule reported for GRPO; the headline advantage may be a checkpoint-selection artifact.","rationale":"The paper is transparent about the FC-SWE selection procedure in Appendix B.1 but silent about the GRPO checkpoint selection. Table 1's central comparison depends on a single FC-SWE step chosen using a 100-task slice of the benchmark. Table 4 shows the selected step is not interchangeability: local-reward checkpoints at steps 67 and 84 differ by 1.8 percentage points in conditional recovery and 2.2 percentage points in Resolved@2 on one seed. Thus, the missing symmetric selection rule is a concrete fairness risk for the numerical claim, not a theoretical complaint about the method. I do not see an equally compelling internal flaw in the algorithm itself: trajectory-local rewards (Eq. 5), active-set advantage estimation (Eqs. 6-7), and the disclosed outcome-dependence bias are coherent, and ablations support the role of failure context. Compute-matching and benchmark-feedback access concerns are acknowledged by the authors and apply to both policies under the stated protocol. The checkpoint-selection issue is exactly the reader's weakest assumption, so my read does not move the verdict; it keeps the existing CONDITIONAL posture pending the symmetric checkpoint-selection check.","tokens_in":20840,"tokens_out":7928,"duration_ms":100661,"concrete_test":"Evaluate every saved GRPO checkpoint at the canonical steps (36, 50, 67, 84, and any other baseline checkpoints) on the same fixed 100-task SWE-bench Verified subset used for FC-SWE selection; choose the best GRPO checkpoint by the same stated selection rule and tie-break. Then evaluate that checkpoint on all 500 tasks with the three decoding seeds and recompute Resolved@1 and Resolved@2. If the symmetrically selected GRPO checkpoint yields 38.9/48.5 and FC-SWE still leads by roughly 2.9/4.3 percentage points, checkpoint selection is exonerated. If it yields higher numbers or shrinks the gap below about one percentage point, the headline advantage is not robust. This test requires only saved checkpoints and the existing evaluation harness.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Appendix B.1 says the final FC-SWE checkpoint (step 67) \"was selected based on performance across multiple validation sets, including a fixed 100-task subset of SWE-bench Verified\"; no equivalent selection rule is reported for the GRPO checkpoint in Table 1. The central numerical claim (41.7 vs 38.9 Resolved@1; 52.8 vs 48.5 Resolved@2) therefore rests on an asymmetric checkpoint-selection premise. Because the 100-task subset is drawn from the same 500-task evaluation benchmark, selecting FC-SWE's step on it and reporting a single, unspecified GRPO checkpoint can inflate both components of the headline advantage. This is not an internal inconsistency in the algorithm: the method sections are clearly specified. It is a fairness condition on the headline numbers, and the paper discloses the FC-SWE side while leaving the GRPO side unstated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"FC-SWE extends GRPO training for repository-level software engineering agents with failure-conditioned recovery: after an initial patch fails verification, the repository is reset and a recovery trajectory is generated conditioned on the failed patch and verifier feedback. Two mechanisms adapt GRPO to these outcome-dependent chains: trajectory-local verifier rewards (Section 3.3) and active-set advantage estimation over the executed trajectories of an issue (Section 3.4). On all 500 SWE-bench Verified tasks with SWE-agent, the authors report that FC-SWE trained on Qwen3.5-4B reaches 41.7% Resolved@1 and 52.8% Resolved@2 versus 38.9% and 48.5% for a GRPO baseline trained from the same initialization, and 70.7% Resolved@11 with an eleven-attempt test-time budget. Ablations isolate the training-time failure context, trajectory-local rewards, and verifier-evidence conditioning; per-seed counts and checkpoint details are reported in the appendices.","tokens_in":21063,"tokens_out":12326,"duration_ms":77781,"significance":"If the headline comparison holds, the paper makes a solid methodological contribution: a clean, well-specified recipe for training multi-turn SWE agents to reuse verifier diagnostics after failure, with the two mechanisms clearly formalized in Section 3. The strengths are real and worth naming: the controlled same-initialization comparison, full per-seed counts in Table 5, clean separation of training (R2E-Gym) from evaluation (SWE-bench Verified), and unusually honest limitation statements (verifier-assisted protocol in Appendix A, training-compute asymmetry in Appendix G, and the acknowledged lack of baseline unbiasedness in Section 3.4). The training-context ablation in Table 2A is especially informative: extra recovery trajectories without failure conditioning do not reproduce the Resolved@2 gain (47.9% vs. 48.5% for GRPO and 52.8% for FC-SWE), which supports attributing the gain to failure-conditioned training rather than to the additional trajectories alone.","major_comments":[{"comment":"The headline comparison in Table 1 rests on a checkpoint-selection procedure that is disclosed for only one side. Appendix B.1 states that the reported FC-SWE checkpoint (step 67) \"was selected based on performance across multiple validation sets, including a fixed 100-task subset of SWE-bench Verified,\" but no selection rule is documented for the GRPO row, which is reported as a single number. Since the 100-task subset is drawn from the same 500-task benchmark used for Table 1, FC-SWE's numbers reflect selection on part of the evaluation set while GRPO's do not, so the reported 2.9 and 4.3 percentage-point advantages can be inflated by this asymmetry. I ask that the paper either document the GRPO checkpoint-selection rule and apply the identical protocol (same 100-task subset, same criterion) to both methods, or report both policies' benchmark results at matched training steps (e.g., step 67 and the final step for each). I note that the disclosure of the FC-SWE selection rule is commendable; the issue is the missing counterpart.","section":"Appendix B.1 / Table 1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The paper reports only point estimates for the central comparison, and the per-seed counts in Table 5 show substantial overlap between the two policies: FC-SWE Resolved@1 is 216/207/203 (43.2/41.4/40.6%) versus 206/179/198 (41.2/35.8/39.6%) for GRPO, and Resolved@2 is 274/261/257 (54.8/52.2/51.4%) versus 257/228/242 (51.4/45.6/48.4%). Because the same 500 tasks are evaluated under every seed, a paired task-level analysis (e.g., a McNemar-style comparison per seed, or pooled across seeds) is the natural way to quantify the 2.9 and 4.3 percentage-point gaps, and I ask that it be added or that the absence of a significant difference be stated. As Appendix B.5 acknowledges, the three seeds are decoding seeds rather than independent training runs, so between-training-run variance is not captured by the current numbers; this makes the requested paired analysis more important, not less.","section":"Table 1 / Table 5","authors_acknowledge":"We do not infer statistical superiority between verifier-only and combined context from these percentages alone.","authors_acknowledge_p":0.66}],"minor_comments":[{"comment":"The active-set advantage estimator is acknowledged in Section 3.4 to lack the unbiasedness guarantee of an action-independent baseline, and Appendix D.1 concedes that alternative advantage-group constructions are not ablated; since trajectory-local rewards are ablated but the comparison-group construction is not, an ablation of the grouping (active-set versus per-attempt-index) would complete the attribution of the gains to the second mechanism.","section":"Section 3.4 / Appendix D.1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The author names \"Terry Kong\", \"Yuki Huang\", \"Ashwath Aithal\", \"Wenming Yang\", and \"Jun Yang\" contain spurious internal spaces in the compiled text; these formatting artifacts should be fixed in the camera-ready version.","section":"Title page","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The token-efficiency comparison reports the token budgets at the 50% and 60% resolution targets (3.15M, 4.39M, 8.22M and 5.08M, 6.29M, 13.57M) but not the attained resolutions at those budgets in the main text; stating them (e.g., 52.8%, 54.1%, and 50.7% at the 50% target, per Appendix E.3) would allow readers to judge the matched-resolution claim without consulting the appendix.","section":"Section 4.5","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The training-compute asymmetry (FC-SWE generates up to twice as many trajectories per step as GRPO) is properly disclosed in Appendix G, and the w/o-failure-context ablation partially addresses it; a GRPO variant matched in total number of trained trajectories per task would further quantify how much of the gain comes from recovery conditioning rather than additional data.","section":"Appendix G","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"Read this one if you work on RL for coding agents. The core is a clean recipe: after a patch fails verification, reset the repo and generate a recovery trajectory conditioned on the failed patch and verifier output, then train with trajectory-local rewards (each attempt keeps its own binary verifier outcome) and active-set advantages (comparison group is only the trajectories actually executed for that issue). LaMer, PivoARL, MulFeRL, and ContextRL are in the neighborhood, but the reward-assignment and grouping choices are concretely different, and the paper argues those differences matter. The strongest evidence is the inference-time ablation: feedback-conditioned retries hurt Base and GRPO relative to resampling but help FC-SWE, which is what you would expect if the training really teaches the policy to use failure evidence rather than just adding attempts. Training and evaluation data are cleanly separated (R2E-Gym vs SWE-bench Verified), so there is no leakage concern there.\n\nThe paper is honestly run and honestly written: same Qwen3.5-4B initialization for the controlled comparison, same controller and attempt budgets, three decoding seeds with per-seed counts in the appendix, ablations for failure context, reward broadcasting, and runtime penalties, plus a candid limitations section covering deployment verifiers, compute matching, single training config, and the fact that the three seeds are inference seeds, not training seeds.\n\nThe soft spot is the one the reader flagged, and it is real. Appendix B.1 says the reported FC-SWE checkpoint (step 67) was selected using a fixed 100-task subset of SWE-bench Verified, drawn from the same 500 tasks used for the headline numbers. No equivalent selection rule is stated for the GRPO checkpoint. That asymmetry is exactly the kind of thing that can inflate a 2.9-point gap. The disclosure rules out hidden intent, but the GRPO side needs the same treatment. This is a reporting fix, not a redesign: state the GRPO selection rule, or report both methods' best checkpoints under an identical procedure. Also keep in mind the numbers are verifier-assisted recovery numbers, not Pass@K; the paper says this clearly, so it is a comparability warning rather than a flaw.\n\nWho it is for: people building retry and recovery loops for SWE agents, and anyone thinking about how to define advantage groups in outcome-dependent trajectory chains. It deserves a serious referee. My recommendation: send it to review, and require the symmetric checkpoint-selection reporting and artifacts before acceptance.","headline":"A well-run recovery-RL study whose main risk—asymmetric checkpoint selection on the eval benchmark—is a reporting fix, not a fatal flaw.","tokens_in":21670,"tokens_out":3023,"duration_ms":35604},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"A failure-conditioned RL recipe lifts SWE-bench Verified two-attempt resolution to 52.8%, versus 48.5% for GRPO.","keywords":["failure-conditioned RL","software engineering agents","GRPO","verifier feedback","recovery trajectories","SWE-bench Verified","advantage estimation","long-horizon agent training"],"falsifier":"Pick the GRPO checkpoint by the same rule used for FC-SWE, best performance on the fixed 100-task subset of SWE-bench Verified, and re-evaluate both policies on all 500 tasks at the two-attempt budget; if the best GRPO Resolved@2 reaches or exceeds 52.8%, the claimed training advantage is not established. A separate check at inference: feed the trained policy a verifier output taken from a different failed task; if conditional recovery stays near 19.0%, the diagnostic content itself is not what drives recovery.","tokens_in":24360,"feed_emoji":"🐛","tokens_out":4345,"duration_ms":72717,"temperature":0.7,"pith_summary":"The paper claims that an RL-trained coding agent becomes better at both first attempts and follow-up recovery if its training loop includes recovery trajectories: when a patch fails verification, the repository resets and the agent tries again with the failed patch and the verifier's diagnostic output as context. Standard GRPO training samples independent attempts and discards that feedback, so the agent never learns to exploit what a rejection tells it. FC-SWE is that training recipe, built on two adjustments to GRPO: each attempt keeps its own verifier outcome as its reward, and advantages are computed over exactly the attempts executed for the issue. On all 500 SWE-bench Verified tasks, a 4B-parameter model trained this way reaches 41.7% first-attempt and 52.8% two-attempt resolution, versus 38.9% and 48.5% for GRPO. The training also transfers beyond its two-attempt horizon, reaching 70.7% resolution with eleven attempts at test time.","feed_headline":"Training on failed patches lifts two-attempt resolution to 52.8%","feed_subtitle":"Failure-conditioned RL beats GRPO on first try and recovery, and stays ahead through 11 attempts.","key_machinery":"The central object is the failure-conditioned recovery chain with two GRPO adaptations. After a patch fails, the system restores the repository to its original state and builds the next attempt's context as $h_{c,r+1} = C(x, patch(\\tau_{c,r}), verify(\\tau_{c,r}))$: the original issue, the failed patch, and the verifier's diagnostic output, truncated to a fixed budget. Trajectory-local rewards assign each attempt its own binary verifier outcome, so a successful recovery raises the advantage of the recovery trajectory without relabeling the earlier failure. Active-set advantage estimation then computes leave-one-out mean baselines over the set of trajectories actually executed for the issue, normalizing by the sample standard deviation with the zero-variance fallback of the paper's GRPO implementation. Together these convert verifier failure from a terminal scalar into structured context and contrastive signal for the policy, which is what the paper argues standard GRPO cannot do.","core_discovery":"FC-SWE claims that post-failure recovery can be trained directly, rather than left to chance at inference. The framework turns each issue into outcome-dependent recovery chains: the initial trajectory is always generated; if its patch fails the verifier, the repository is restored to the original task state and a new trajectory is generated from the issue plus the failed patch and diagnostic verifier output as context. Two mechanisms adapt GRPO to these chains. Trajectory-local reward assignment gives every trajectory the verifier outcome of its own patch, so a chain with outcomes [0, 1] is trained with rewards [0, 1], never [1, 1]; the later success does not retroactively reward the earlier failure. Active-set advantage estimation forms each trajectory's comparison group from all initial and recovery trajectories actually executed for the same issue, keeping failed attempts in the group while excluding unexecuted recovery slots. The paper's central empirical claim is that on all 500 SWE-bench Verified tasks, with Qwen3.5-4B and SWE-agent, FC-SWE achieves 41.7% Resolved@1 and 52.8% Resolved@2 versus 38.9% and 48.5% for GRPO, and that the gap comes from learning to use failure evidence: feedback-conditioned retries beat independent resampling only for the FC-SWE-trained policy (19.0% versus 17.0% conditional recovery), and removing verifier evidence from the context drops conditional recovery by 3.3 points.","pith_inferences":["The size of the headline margin is the number to stress-test: if a GRPO checkpoint selected on the same 100-task SWE-bench Verified validation slice closed most of the 2.9/4.3-point gap, the mechanism could still be real but the headline advantage would be weaker than reported.","The paper itself notes that the active-set comparison is outcome-dependent and does not inherit the unbiasedness guarantee of an action-independent baseline; a natural follow-up is to test whether a selection-bias-corrected or doubly-robust advantage changes training dynamics, which would separate conditioning effects from group-composition effects.","The recipe generalizes: any verifier-gated agent domain (theorem proving, web automation, scientific code) with diagnostic failure output could treat recovery trajectories as first-class training data instead of inference-time repair.","Since Resolved@K gains shrink after K=8, the policy's recovery skill, not the attempt budget, appears to be the binding constraint at test time; training with more than two attempts per chain or a curriculum over attempt number is the natural next experiment to test whether that ceiling moves."],"forward_implications":["Failure-conditioned RL raises both first-attempt solving and recovery: 41.7% vs 38.9% Resolved@1 and 52.8% vs 48.5% Resolved@2 over GRPO on all 500 SWE-bench Verified tasks.","The benefit is specifically learning from failure evidence: removing failure context during training drops two-attempt resolution by 4.9 points and conditional recovery from 19.0% to 12.6%, and at inference verifier evidence adds 3.3 points over patch-only context.","Only the failure-trained policy benefits from feedback at a matched budget: FC-SWE recovers 19.0% of first-attempt failures with feedback versus 17.0% by independent resampling, while Base and GRPO both do better with resampling than with feedback.","Recovery training transfers beyond its two-attempt training horizon: the same policy reaches 70.7% resolution with eleven verifier-guided attempts at test time, versus 67.3% for GRPO.","FC-SWE is also inference-token-efficient: at roughly 50% resolution it uses about 3.15M tokens per task versus 4.39M for GRPO and 8.22M for Base."],"supporting_citations":[{"why":"Provides the GRPO objective that FC-SWE extends and the baseline it is compared against.","marker":"(Shao et al., 2024)"},{"why":"Supplies the leave-one-out reward baseline used in both the GRPO baseline and FC-SWE's active-set advantage estimation.","marker":"(Ahmadian et al., 2024)"},{"why":"Defines SWE-bench Verified, the 500-task evaluation set, and the verification procedure applied during training and evaluation.","marker":"(Jimenez et al., 2023)"},{"why":"Supplies SWE-agent, the agent-computer interface and evaluation controller used for benchmark runs.","marker":"(Yang et al., 2024)"},{"why":"Supplies R2E-Gym, the 4,518-task training environment and its verifiers, with no repository overlap with SWE-bench Verified.","marker":"(Jain et al., 2025)"},{"why":"Supplies OpenHands, the harness that generates the training rollouts.","marker":"(Wang et al., 2024)"},{"why":"Motivates using execution feedback from a verifier as a learning signal for code-synthesis RL.","marker":"(Gehring et al., 2024)"},{"why":"Supplies the finding that self-repair can underperform independent resampling at fixed budgets, the pattern FC-SWE claims to reverse.","marker":"(Olausson et al., 2024)"}],"fun_headline_variants":["Failure-conditioned RL turns failed patches into recovery training","FC-SWE learns from verifier failure, beating GRPO on SWE-bench Verified","Training on failed patches: 41.7% first-try, 52.8% two-attempt resolution","Recovery trajectories trained from failed patches beat independent resampling","Failed patches become context for trained recovery in SWE agents"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The reported 2.9-point and 4.3-point advantages rest on checkpoint-selection fairness: the FC-SWE checkpoint (step 67) was chosen using validation performance that included a fixed 100-task slice of SWE-bench Verified, and the paper reports no analogous selection procedure for the GRPO checkpoint it is compared against.","fun_headline_variants_meta":{"raw":{"variants":["Failure-conditioned RL turns failed patches into recovery training","FC-SWE learns from verifier failure, beating GRPO on SWE-bench Verified","Training on failed patches: 41.7% first-try, 52.8% two-attempt resolution","Recovery trajectories trained from failed patches beat independent resampling","Failed patches become context for trained recovery in SWE agents"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000303,"raw_usage":{"cost":0.00030324,"is_byok":false,"cost_details":{"upstream_inference_cost":0.00030324,"upstream_inference_prompt_cost":0.00018032,"upstream_inference_completions_cost":0.00012292},"total_tokens":1727,"prompt_tokens":1288,"completion_tokens":439,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":0,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":350}},"tokens_in":1288,"tokens_out":439,"cost_source":"openrouter_usage","duration_ms":4027,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":350,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-10-07T21:20:04.476432+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"Pick the GRPO checkpoint by the same rule used for FC-SWE, best performance on the fixed 100-task subset of SWE-bench Verified, and re-evaluate both policies on all 500 tasks at the two-attempt budget; if the best GRPO Resolved@2 reaches or exceeds 52.8%, the claimed training advantage is not established. A separate check at inference: feed the trained policy a verifier output taken from a different failed task; if conditional recovery stays near 19.0%, the diagnostic content itself is not what drives recovery.","supporting_citations":[],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}