{"id":"79676c28-2a72-4e71-941e-cdabc970d4e1","arxiv_id":"2608.13292","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A post-generation refinement adapter, trained with SFT and DPO on constructed verbose-to-concise pairs, makes LLM repair patches much smaller across four SWE-bench systems while preserving or improving resolution.","lead":"LLM-generated patches that fix software bugs are usually much larger and more complex than developer-written patches, even when they pass tests. This paper presents RECAP, an add-on that shrinks those patches after generation while preserving or improving how often the fix works.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"RQ-2's headline size reduction mixes different resolved-instance sets before and after refinement, so part of the reported drop from +242% to +4% may be compositional rather than per-patch refinement.","rationale":"I chose the unmatched-instance comparison over the reader's synthetic-data concern because the central claim is explicitly about four host systems, and those hosts were evaluated directly; the synthetic-distribution worry mainly affects generalization beyond those hosts. The unmatched-set issue, by contrast, can change the reported numbers for the claimed setting itself. The paper's Table IX provides exactly the transition labels needed to test this, so the check is cheap and decisive. I do not treat this as evidence of any error in the method; the refiner may well shrink patches on matched instances, but the current tables do not demonstrate that. The reader's conditions (release artifacts, qualify overclaims, add error bars, validate synthetic data) remain reasonable; I would add matched-instance size reporting as an additional precondition. Hence the conditional verdict is unchanged.","tokens_in":23133,"tokens_out":8814,"duration_ms":100362,"concrete_test":"Recompute Table VI for each host x mode with total and net changes restricted to (i) the C->C transition subset (instances resolved by both the host and RECAP), (ii) the I->C subset, and (iii) the C->I subset, using the same gold-patch reference. If the C->C-only average still drops from roughly +242% to near +4% in total changes, and from +348% to near -40% in net changes, then the size reduction reflects genuine per-instance refinement. If the C->C-only drop is much smaller or positive, the headline tradeoff is largely compositional. This requires only rerunning the existing evaluation harness with the Table IX transition labels; no new training is needed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central RQ-2 claim is that RECAP cuts average total changes from +242.14% to +4.24% and net changes from +348.24% to -39.75% while preserving or improving resolution. These averages are per-instance ratios to the gold patch, averaged over each method's resolved instances, consistent with the RQ-1 protocol in Section V-C. The resolved-instance sets are not the same before and after refinement. For example, SWE-agent under UR has 333 resolved before and 357 after, and Table IX shows this is composed of 314 C->C, 19 C->I, and 43 I->C instances. The after-average therefore includes the 43 recovered instances and excludes the 19 broken ones. If recovered instances tend to have small patch ratios, the average drops even when refined patches on the original 314 C->C instances are no smaller than the host patches. The effect is strongest for OGR, where Table IX guarantees C->I=0 while I->C gains enlarge the after-set, so the reported +23 to +42 resolution improvements and the size reduction are partly a selection artifact: OGR returns refined patches only when they pass the oracle, and the post-refinement average is taken over a different, larger set. The paper reports transition counts but not size metrics per transition, so the headline size-correctness tradeoff is not yet evidenced at the per-instance level.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper identifies patch verbosity as a systemic problem in LLM-based APR. It characterizes 28 SWE-bench Verified approaches and reports that the median resolved patch is 121.78% larger in total changes, 80.91% larger in net changes, and 43.99% higher in cyclomatic complexity than the developer (gold) patch. It then proposes RECAP, a post-generation adapter with a collector, a filter, and a compact refiner trained via SFT and DPO on a multi-source dataset (function-level pairs, tangled commits, and synthetic inflated SWE-bench patches). The evaluation on four host systems across three deployment modes (UR, JGR, OGR) claims a substantially better size-correctness tradeoff than prompting, commit-untangling, and minimality-aware baselines, cutting average total changes from +242.14% to +4.24% while preserving or improving resolution. The paper also presents an ablation showing SFT alone over-compresses and DPO restores correctness, and a generalizability experiment with three open-source backbones.","tokens_in":23423,"tokens_out":5885,"duration_ms":64365,"significance":"If the central claims hold, this is a valuable contribution. The RQ-1 characterization of verbosity across 28 public leaderboard systems is a useful empirical resource, and the proposed adapter design is practical: it is plug-and-play, tested on multiple hosts and backbones, and the transition decomposition in Table IX is a thoughtful way to expose broken and recovered instances. The paper ships concrete examples of refinement beyond simple line deletion, and the SFT/DPO ablation cleanly illustrates a known failure mode of aggressive minimization. The significance is somewhat tempered by the fact that the size-reduction headline is measured as distance to the gold patch, which is also the training target, and by the absence of repeated runs; nevertheless, the empirical characterization and the adapter framework are likely to be useful to the APR community.","major_comments":[{"comment":"The headline RQ-2 size-reduction numbers are not computed on a fixed set of instances, so the reported drop from +242.14% to +4.24% is partly compositional. The per-instance Δ metrics are averaged over each method's resolved instances (Section V-B3, applied in Section V-C), but Table IX shows the resolved sets differ before and after refinement: for SWE-agent UR, 314 instances are C→C, 19 are C→I, and 43 are I→C, so the post-refinement average includes 43 recovered instances and excludes 19 broken ones. The effect is strongest for OGR, where the oracle filter forces C→I=0 and adds I→C gains by construction (Eq. 4), so the post-refinement set is systematically different from the host's resolved set. Without size statistics per transition, the paper cannot attribute the size reduction to per-patch refinement on the original resolved instances. Please report ΔTotal/ΔNet on the C→C intersection, and separately for the I→C recovered set, or run a per-instance paired comparison on the common resolved subset.","section":"§V-D, Table IX"},{"comment":"The abstract's claim that RECAP preserves or improves resolution is contradicted by the paper's own Table VI: on Moatless, RECAP-UR resolves 350/500 (-4) and RECAP-JGR resolves 348/500 (-6). Similarly, Table VIII shows resolution losses for Gemma-4-4B on SWE-agent (-5), Moatless (-24), and Openhands (-7), and for Mistral on Moatless (-8). The claim should be qualified to the OGR mode, or to 'preserving or improving resolution in most settings', and the exceptions should be discussed. This matters because the 'preserving or improving resolution' phrase is a central part of the advertised tradeoff.","section":"Abstract and §V-D, Table VI"},{"comment":"The transferability of the refiner rests on an unverified assumption that the synthetic training distribution resembles real APR verbosity. The paper constructs verbose inputs by asking GPT-5.2 to inflate gold patches according to the hand-written Table I patterns, or by rejection-sampling 10 LLM-written patches and keeping the largest. No evidence is provided that real host patches (e.g., SWE-agent's or Openhands's patches) exhibit the same structure of verbosity, rather than, say, agentic exploration noise or poor localization. The Threats to Validity section acknowledges LLM-judging and inflation as possible noise sources, but does not validate the synthetic distribution against real host patches. A concrete test would be to compare distributional properties (edit locality, file scope, line-level vs block-level changes) of synthetic and real host patches, or to train on a sample of real host patches and measure whether the gains persist.","section":"§III-C, Threats to Validity"},{"comment":"The size-reduction target and the evaluation anchor are both the gold developer patch: the model is trained to map inflated gold patches back to gold patches, and results are reported as percentage distance from the same kind of reference. This alignment is close to construction for the concision metric, as the reader's circularity concern notes. The paper acknowledges that developer patches are not guaranteed to be minimal, but calling them 'relative reference points' does not resolve the circularity. Please include absolute patch sizes, report a reference-free measure (e.g., human reviewability judgments on a sample of refined patches), or at least discuss what happens when the gold patch is itself not minimal.","section":"§V-C and §III-C"},{"comment":"No repeated runs or error bars are reported. The differences of a few resolved instances (e.g., Moatless UR -4 vs JGR -6, Agentless UR +4 vs JGR +11) are likely within stochastic variation of both the host agents and the refiner. The resolved-instance counts, and especially the small deltas, should be accompanied by at least three independent runs or a bootstrap over instances with confidence intervals. As written, the 'preserving or improving resolution' claim rests on single-run point estimates.","section":"§V-B, Tables VI-VIII"}],"minor_comments":[{"comment":"The text states that OGR 'guarantees it cannot drop an already-resolved instance', but Eq. (4) only guarantees this with respect to the test oracle T. A refined patch that passes the tests may still alter behavior in untested ways. The later discussion acknowledges this, but the earlier phrasing should be corrected.","section":"§IV-D, Eq. (4)"},{"comment":"The collector uses an LLM to generate a Python extraction script from a single localization artifact; it is unclear how failures or hallucinations in the generated script are detected and handled. Please describe the fallback mechanism or a sanity check on the extraction script.","section":"§IV-B, Figure 1"},{"comment":"The 'Resolved' column reports absolute counts and, in parentheses, deltas relative to the host's resolved count; this should be stated explicitly in the caption to avoid confusion for the baseline rows.","section":"Table VI caption"},{"comment":"Some entries appear to be missing leading signs (e.g., the SWE-agent row for Gemma-4-4B shows '5.94', '47.71', '47.15' without signs). Ensure all values are signed consistently with the metric definitions.","section":"Table VIII"},{"comment":"Net Changes is defined as added minus deleted LOC, so it can be negative for a patch that is smaller than the gold patch. The interpretation of negative ΔNet values in the tables should be clarified, since a negative net change is not always desirable if it means removing necessary lines.","section":"§V-B, metrics"},{"comment":"The curriculum λ values (0, 0.3, 0.6, 0.9, 1.0) appear ad hoc. A sensitivity analysis or a citation to a prior work using the same schedule would strengthen the claim that curriculum learning is beneficial.","section":"§IV-C, curriculum scheduling"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a good empirical fit for a software-engineering venue. I would ask the authors to prioritize the compositional issue in RQ-2: if the per-transition size analysis shows that the headline reduction is mostly a selection artifact, the central novelty is substantially weakened. The synthetic-data representativeness concern is the second most important issue. If both are convincingly addressed, the paper could be acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the RQ-1 characterization of patch verbosity across 28 SWE-bench Verified approaches is genuinely useful. It separates textual size from structural complexity, shows the median approach lands at +122% total changes relative to gold, and ties verbosity to capability-oriented design choices like iterative refinement and broad context. This is a clean empirical pass and the best part of the paper.\n\nSecond, the headline RQ-2 claim—cutting average total changes from +242% to +4% while preserving or improving resolution—needs a serious caveat. The averages are taken over each method's resolved instances, and those sets change after refinement. Table IX shows SWE-agent UR goes from 333 resolved to 357, composed of 314 C→C, 19 C→I, and 43 I→C. The after-average includes 43 recovered instances and excludes 19 broken ones. If recovered instances tend to have small patch ratios, the average drops even when refined patches on the original 314 are no smaller than the host patches. The effect is strongest for OGR, where the oracle enforces C→I=0 and the after-set gains I→C instances by construction. The paper reports transition counts but no per-transition size metrics, so the tradeoff is not yet evidenced at the per-instance level. The compositionality concern in the stress-test note holds up.\n\nThe core idea is not wrong. The authors show concrete refinement examples in Figures 2 and 3, and the RQ-3 ablation is coherent: SFT over-compresses, DPO restores correctness. That SFT-alone resolves fewer instances than an untrained base model is a believable result, and the evaluation does use genuinely verbose host patches, not only the synthetic inflated inputs. The abstract's 'preserving or improving resolution' is contradicted by Moatless UR (-4) and JGR (-6), which is sloppy but minor. No error bars or repeated runs either, which is common for SWE-bench but should be stated.\n\nI would accept this for peer review. The RQ-1 study alone justifies referee time, and the refinement formulation is distinct from the cited minimality-aware generators. But I would condition acceptance on per-transition size reporting and a qualified abstract. This is a good candidate for the reading group; it will generate useful discussion about what counts as evidence in APR.","headline":"The verbosity characterization is the real contribution; the headline size-correctness tradeoff is partly an artifact of averaging over different resolved-instance sets before and after refinement.","tokens_in":23970,"tokens_out":3262,"would_cite":true,"duration_ms":33810,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A small post-generation refiner can reduce LLM repair patches to developer-size changes while preserving or improving resolution.","keywords":["automatic program repair","patch verbosity","patch refinement","post-generation refinement","direct preference optimization","LLM-based repair","minimal patches","code review"],"falsifier":"Take a set of real candidate patches from a new agent whose verbosity comes from exploration leftovers—failed alternatives, reverted edits, repeated tool calls—rather than from the paper's listed inflation patterns, run RECAP on them, and check whether they shrink without failing tests. If the refiner cannot reduce this structurally different verbosity, the central assumption about the synthetic training distribution is false.","tokens_in":22926,"feed_emoji":"🩹","tokens_out":7617,"duration_ms":74101,"temperature":0.7,"pith_summary":"Large language models now resolve real repository-level bugs, but the patches they produce are far bulkier than what a developer would write. This paper claims that patch verbosity is systemic: across 28 leading repair approaches, every one exceeded the developer's patch in total line changes, with the median at +121.78%. It then proposes treating minimality as a separate post-generation step rather than a generator constraint. The paper's central result is that a small, trained refiner placed after generation can cut average total changes from +242.14% to +4.24% relative to developer patches across four host systems, while preserving or increasing the number of resolved issues by up to 42. If true, concise repairs do not require retraining large repair agents.","feed_headline":"LLM repair patches shrink to developer size without lost fixes","feed_subtitle":"Across four repair agents, one adapter cut average total changes from +242% to +4% while preserving fixes.","key_machinery":"The central mechanism is post-generation patch refinement, formalized as a task that takes problem context, code context, and a candidate patch and returns a patched program that satisfies the intended fix specification while making a smaller change. The instantiation is RECAP, a three-part adapter: a collector that standardizes context and patch format from a host framework, a filter that decides whether refinement is safe under an unconditional, judge-guided, or oracle-guided deployment mode, and a small open-source refiner trained on the refinement task. The training data is constructed from function-level repair pairs, tangled commits paired with atomic fixes, and synthetic repository-level examples made by inflating concise developer patches; SFT teaches the verbose-to-concise transformation and DPO supplies the negative signal that stops the model from over-compressing.","core_discovery":"On the paper's own terms, the discovery is that correctness and concision can be decoupled: a compact model that never sees the original bug can take a successful but verbose candidate patch, along with the issue and code context, and rewrite it into a smaller, less complex patch that still passes the tests. The evidence is a refinement pipeline trained on 5,540 verbosity-reducing pairs from three sources, using supervised fine-tuning followed by direct preference optimization with distilled reasoning traces. Across four host systems, this adapter reduced average total changes from +242.14% to +4.24% and net changes from +348.24% to -39.75% relative to developer patches, while strict test-gated deployment improved resolution by up to 42 instances and never dropped a resolved one. The paper also reports a mechanical origin for the problem: iterative refinement and higher-resolution capability are the design factors most associated with larger patches, while output format and minimality prompts have little effect.","pith_inferences":["Inference beyond the paper: a refinement adapter of this shape could sit in any diff-producing pipeline—code review bots, CI auto-fixers, or commit squashing tools—where the goal is to make machine-written edits reviewable, not just to repair bugs.","Inference beyond the paper: the reported capability–concision tradeoff implies that as repair agents get stronger through more iterative search, their patches will get larger; post-generation refinement may therefore become a necessary companion to scaling, not a temporary patch.","Inference beyond the paper: using a stronger oracle than test-passing, such as property-based tests or differential testing, would test whether the refiner's smaller patches preserve behavior beyond what a benchmark test suite can verify."],"forward_implications":["Verbosity is a systemic property, not an outlier effect: all 28 evaluated approaches exceed developer patch size, so future repair systems should be measured on patch size as well as resolution.","Minimality prompts and structured output formats will not fix verbosity; only changing how the patch is produced or adding a post-generation step changes the outcome.","A compact refiner can be bolted onto an existing repair pipeline without retraining the generator, and the biggest size gains come on the most verbose agents.","Refinement can turn failing candidates into passing ones, recovering up to 43 instances on one host, so removing extraneous edits is itself a correctness improvement.","Optimizing a single size metric is unsafe: the SFT-only refiner wins on one metric but resolves the fewest instances, while DPO restores correctness without giving back concision."],"supporting_citations":[{"why":"Supplies the repository-level benchmark, gold developer patches, and test oracle used for evaluation and training data.","marker":"[6]"},{"why":"Provides the function-level repair pairs whose code-consistency-rate contrast seeds the refinement dataset; also serves as a minimality-aware baseline.","marker":"[20]"},{"why":"Provides the edit-aware reward baseline for minimality-aware generation that RECAP is compared against.","marker":"[21]"},{"why":"Supplies direct preference optimization, the training objective that restores correctness after supervised fine-tuning.","marker":"[23]"},{"why":"One of the four host systems evaluated, contributing the most verbose candidate patches that RECAP refines.","marker":"[28]"},{"why":"One of the four host systems, with already-compact patches that RECAP still pushes below gold size.","marker":"[30]"},{"why":"Provides tangled commit pairs from which repository-level non-minimal to atomic fix targets are built.","marker":"[43]"},{"why":"Supplies the annotated conventional-commit dataset underlying the tangled commit pairs.","marker":"[44]"}],"fun_headline_variants":["Refine after generation: patch verbosity down, fixes preserved","Post-hoc refinement trims LLM patches without hurting fixes","Smaller repair patches, same or more fixes: refine after generation","Patch size: from +242% to +4% via refinement after generation","Post-hoc adapter: smaller patches, fixes preserved or improved"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole approach depends on the synthetic training data—gold patches inflated according to a hand-written list of bad patterns—being a faithful stand-in for how real repair agents actually make patches verbose; if real verbosity has a different structure, the refiner's size gains may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Refine after generation: patch verbosity down, fixes preserved","Post-hoc refinement trims LLM patches without hurting fixes","Smaller repair patches, same or more fixes: refine after generation","Patch size: from +242% to +4% via refinement after generation","Post-hoc adapter: smaller patches, fixes preserved or improved"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000887,"raw_usage":{"total_tokens":3895,"prompt_tokens":1080,"completion_tokens":2815,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":696,"completion_tokens_details":{"reasoning_tokens":2727}},"tokens_in":696,"tokens_out":2815,"duration_ms":18997,"temperature":1.0,"reasoning_tokens":2727,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:26:14.936975+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of real candidate patches from a new agent whose verbosity comes from exploration leftovers—failed alternatives, reverted edits, repeated tool calls—rather than from the paper's listed inflation patterns, run RECAP on them, and check whether they shrink without failing tests. If the refiner cannot reduce this structurally different verbosity, the central assumption about the synthetic training distribution is false.","supporting_citations":[{"cited_title":"Less is more: Adaptive program repair with bug localization and preference learning,","cited_arxiv_id":null,"evidence_quote":"Provides the function-level repair pairs whose code-consistency-rate contrast seeds the refinement dataset; also serves as a minimality-aware baseline."},{"cited_title":"QiMeng-PRepair: Precise Code Repair via Edit-Aware Reward Optimization","cited_arxiv_id":"2604.05963","evidence_quote":"Provides the edit-aware reward baseline for minimality-aware generation that RECAP is compared against."},{"cited_title":"Detecting multiple semantic concerns in tangled code commits,","cited_arxiv_id":null,"evidence_quote":"Provides tangled commit pairs from which repository-level non-minimal to atomic fix targets are built."},{"cited_title":"A first look at conventional commits classification,","cited_arxiv_id":null,"evidence_quote":"Supplies the annotated conventional-commit dataset underlying the tangled commit pairs."}],"review_version":1}