{"id":"991fe767-48f4-4037-a67e-5d895846e9de","arxiv_id":"2511.01043","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"DPO-f+ fine-tunes LLMs with reward-augmented DPO to generate developer-profile-aware code-repair feedback, claiming gains over DPO that rest on LLM-judge evaluations.","lead":"This paper presents DPO-f+, a training method that blends Direct Preference Optimization with a learned reward margin to make LLM code-repair feedback match what novices and experienced developers say they need. It reports improved LLM-judged feedback alignment and small repair-accuracy gains, but the abstract claims a 200-developer human study that the full text says is still future work.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract's 200-developer preference result appears nowhere in the body, which defers human evaluation to future work; the alignment claim rests on an LLM-as-judge rubric that also creates the training labels, so no direct evidence ties DPO-f+ to real developer preferences.","rationale":"The reader's weakest assumption — that GPT-4's rubric scores are a valid proxy for real developer preferences — is the same load-bearing concern I identify. The evidence offered for that proxy is thin: a 100-item calibration with reported 95% agreement, but no evidence that the raters were developers or that the agreement was with the automated scores rather than among human raters. Moreover, the paper's own limitation section (§6.2) explicitly defers direct human evaluation to future work, directly contradicting the abstract's claim of a 200-developer human study. This is not a matter of fine-tuning; it is the core of the paper's stated contribution. If the human-preference claim is removed, what remains is a modest code-accuracy improvement whose statistical reliability is not established (no error bars, small SWE-bench counts) plus an LLM-judge alignment gain that is partially circular because the same rubric that generated the training labels is used as the evaluation instrument. I do not accuse the authors of fabrication; the discrepancy could be a reporting error or an artifact of version mismatch. But for the preprint as presented, the central claim is not supported by the body text. The reader's REJECT verdict is appropriate, and my read does not change it.","tokens_in":17823,"tokens_out":8647,"duration_ms":101439,"concrete_test":"Conduct a preregistered pairwise preference study with the claimed 200 developers on a stratified random sample of the existing 1,000 novice-task feedback instances, comparing DPO-f+ against both DPO and baseline, with pre-specified analysis (e.g., exact binomial test against 50%). If observed preference for DPO-f+ is not significantly above chance or differs materially from the abstract's 71.5%, the alignment claim fails. As an immediate audit, also check the anonymized repository for the 200-developer dataset and the 100-item calibration instrument; absence of either would confirm the body-text gap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim — that DPO-f+ 'aligns code-repair feedback with developer preferences' — requires evidence that real developers prefer its outputs. The body provides none. §6.2 explicitly says 'future work will include direct human evaluation to further validate our findings,' and §7.1 again defers human studies. The abstract's reported 71.5% preference (p=0.0057) from 200 developers is absent from the experimental sections and tables. The only human check is a 100-item calibration (§4.2.3) described as 95% inter-annotator agreement; the text does not establish that these items were judged by the target population or that agreement was with GPT-4's rubric scores rather than among raters.\n\nThe alignment evaluation is also circular: GPT-4's seven-dimension rubric both labels preference pairs (§4.2.3) and scores the final outputs (§4.4.3). High G-Eval and pairwise win rates therefore show that DPO-f+ tracks the rubric, not that it tracks developers. The objective accuracy gains are additionally fragile: Table 3 reports no confidence intervals, and Table 5's headline SWE-bench difference (20 vs 15 resolved out of 300) is within sampling noise (Fisher exact p≈0.49). But the load-bearing defect is the missing link to human preference: without direct developer judgments, 'aligned with developers' preferences' is an interpretation, not a demonstrated result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DPO-f+, a fine-tuning framework that augments Direct Preference Optimization (DPO) with a frozen learned reward-model margin, aiming to align code-repair feedback with the needs of novice and experienced developers. The framework defines a seven-dimension rubric for feedback quality, constructs preference pairs by scoring candidate feedback with GPT-4, trains a reward model and policy on those pairs, and evaluates the resulting model on novice programming tasks and SWE-bench Lite. The authors report improvements over a base model and standard DPO in preference accuracy, Pass@k/executability on novice tasks, SWE-bench resolution rate, and rubric-based alignment scores.","tokens_in":18189,"tokens_out":5677,"duration_ms":62198,"significance":"If the central claim were supported, the idea of adding a graded, frozen reward-margin signal to DPO for profile-aware code-repair feedback would be a useful and inexpensive contribution to the SE/LLM alignment literature. The paper also has positive features: it ships source data and scripts, evaluates on two base models, uses an official SWE-bench harness, and separates code-accuracy from feedback-quality metrics. However, the headline claim—that DPO-f+ aligns feedback with real developer preferences—is not established by the evidence in the manuscript. The abstract reports a 200-developer human study that never appears in the body; the alignment evaluation is circular because the same GPT-4 rubric that creates the training labels is also used to score the final outputs; and the objective accuracy improvements are reported without confidence intervals or significance tests, with the SWE-bench difference between DPO-f+ and DPO being within sampling noise. The core contribution may be salvageable with additional human evaluation and a reframed claim, but as written the central empirical assertion is unsupported.","major_comments":[{"comment":"The abstract states that a human study with 200 developers found DPO-f+ preferred in 71.5% of comparisons with p=0.0057. No such study is reported in the body. The evaluation sections only present LLM-as-judge rubric scores and pairwise comparisons; §6.2 explicitly says 'future work will include direct human evaluation to further validate our findings,' and §7.1 again defers human studies. Either the human study must be fully described (participants, materials, procedure, results) or the abstract claim must be removed. As it stands, the manuscript's headline finding is absent from the reported experiments.","section":"Abstract; §5.1.3; §5.2.3; §6.2; §7.1"},{"comment":"The alignment evaluation is circular. Preference labels for training are produced by GPT-4 scoring feedback with the seven-dimension rubric (§4.2.3). The main alignment evaluation then uses GPT-4 to score generated feedback on the same seven dimensions and DeepSeek-V3 for pairwise comparison (§4.4.3). Therefore the reported preference accuracy, G-Eval scores, and win rates measure agreement with the GPT-4 rubric, not agreement with actual developer preferences. The 100-item human calibration mentioned in §6.2 does not resolve this, because the text does not specify that raters were from the target developer population, how the 100 items were selected, or whether the 95% figure is inter-rater agreement or agreement with GPT-4's scores.","section":"§4.2.3; §4.4.3; Tables 2 and 4"},{"comment":"The objective accuracy claims lack statistical grounding. Table 3 reports executability and Pass@k values without confidence intervals or significance tests. Table 5 shows SWE-bench Lite resolutions of 20/300 for DPO-f+, 15/300 for DPO, and 6/300 for the baseline; the DPO-f+ vs. DPO difference is within sampling noise (e.g., Fisher's exact test gives a large p-value). The claimed +1.67 pp over DPO is not compelling evidence of an improvement. Confidence intervals and appropriate tests should be reported for all headline accuracy comparisons.","section":"§5.1.2, Table 3; §5.2.2, Table 5"},{"comment":"Preference accuracy is computed on test pairs drawn from the same GPT-4-labeled distribution used for training. The paper itself acknowledges that this metric 'can be susceptible to overfitting to specific rubric cues.' The large gains (e.g., Qwen baseline 0.4511 to DPO-f+ 0.8184) may therefore reflect the model learning to exploit rubric-specific patterns rather than developing a generalizable ability to predict developer preferences. Independent validation on human preference judgments or on out-of-distribution pairs is needed before these numbers can be interpreted as alignment with developers.","section":"§5.1.1, Table 2"}],"minor_comments":[{"comment":"The inverse temperature β is mentioned in the text but does not appear in the equations. Also, the data-dependent weight λ(x) is never defined; please specify how it is computed.","section":"Eq. (1), Eq. (4)"},{"comment":"The text says 'Table 1 reports per-metric scores' but the actual per-metric alignment scores appear in Table 4. Please correct the cross-reference.","section":"§5.1.3"},{"comment":"There is a typo: 'i.e,.SWE-bench Lite' should be 'i.e., SWE-bench Lite'.","section":"§1"},{"comment":"The inference setup says 'default temperature' but does not give the temperature value. Please specify the decoding hyperparameters.","section":"§4.4.2"},{"comment":"The max sequence length of 64 tokens seems very short for natural-language feedback plus corrected code. Please clarify whether this is an input or output truncation value and whether it affects the evaluation.","section":"§4.3.4"}],"recommendation":"reject","confidential_remarks":"The most serious issue is the mismatch between the abstract's claim of a 200-developer human study and the body, which defers human evaluation to future work. This is not a minor presentation error; it directly concerns the paper's central claim. I would ask the editor to verify whether such a study exists. The work could become publishable after a major revision that includes the human study (or removes the claim), breaks the evaluative circularity, and adds statistical rigor to the accuracy results, but as submitted the central empirical claim is unsupported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper has a genuinely useful empirical pipeline, but the central claim as written is not supported. The abstract reports a 200-developer human preference study (71.5% preference, p=0.0057) that never appears in the body. The only human data is a 100-item calibration with 95% inter-annotator agreement, and the text doesn't say who those raters were or what they agreed on. Sections 6.2 and 7.1 explicitly defer real human evaluation to future work. That's an abstract-versus-body mismatch, and it directly affects the meaning of \"aligned with developer preferences.\"\n\nWhat's genuinely new: the developer-profile rubric (separate novice and experienced interpretations of seven feedback dimensions) and the automatic pair-construction pipeline for code-repair feedback. The reward-augmented DPO loss is close to AlphaDPO, but applying it to feedback alignment in SE, with a concrete evaluation protocol, is a legitimate contribution. The SWE-bench Lite setup, with a fixed retrieval helper and official evaluator, is an honest attempt at external validity, and they release data and scripts.\n\nThe soft spots are real but not all equal. The circularity worry is the biggest: GPT-4's seven-dimension rubric creates the training pairs and scores the final outputs. The paper acknowledges LLM-as-judge is a proxy and calibrates on 100 items, but that doesn't rescue the abstract's claim. The SWE-bench numbers (20 vs 15 vs 6 out of 300) are presented without confidence intervals or significance tests; the difference between 20 and 15 is within sampling noise. Pass@k lacks error bars too. That said, the novice-task Pass@1 improvement (+5.71 pp over baseline) and the independent DeepSeek-V3 pairwise judgments are at least consistent with a real effect, so I don't think the objective gains are fabricated.\n\nMy call: send it to serious peer review, but the editor should insist the abstract be corrected to match the body and the human study be either added or clearly removed. The reviewers will want error bars and a non-circular alignment check. The core idea is worth engaging with; the current packaging overstates what's shown.","headline":"The empirical pipeline is real, but the abstract sells a human-preference result the paper doesn't contain, and the rubric circularity is load-bearing.","tokens_in":18673,"tokens_out":2077,"would_cite":false,"duration_ms":23648,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding a frozen reward-model margin to preference training makes code-repair feedback more accurate and more aligned with developer needs, the paper argues.","keywords":["code repair","feedback alignment","direct preference optimization","reward-augmented DPO","developer preferences","code comprehension","human-AI teaming","LLM-as-judge"],"falsifier":"Run a fully blind human study in which novice and experienced developers rate baseline, standard DPO, and DPO-f+ feedback on the same seven dimensions without knowing the source; if DPO-f+ does not beat the alternatives in blind ratings, or if GPT-4 rubric scores correlate at or below chance with human developers' ratings on a validation set larger than the reported 100-item check, the alignment claim fails.","tokens_in":17700,"feed_emoji":"🔧","tokens_out":5402,"duration_ms":58704,"temperature":0.7,"pith_summary":"The paper argues that code repair assistance is not just about producing corrected code: the natural-language feedback accompanying the fix must match the developer's experience level and comprehension needs. It introduces DPO-f+, a training method that extends Direct Preference Optimization with a frozen reward-model margin, so preferred versus dispreferred feedback pairs are weighted by how decisive the preference is rather than treated as equal binary labels. On novice programming tasks, DPO-f+ raises Pass@1 by 5.71 percentage points over the baseline and 3.30 over standard DPO; on SWE-bench Lite it resolves 4.67 percentage points more issues than the baseline. The framework also scores highest on all seven feedback-alignment dimensions and wins a majority of pairwise comparisons, with beginners preferring DPO-f+ feedback in 71.5% of human evaluations (p=.0057). If correct, the paper shows that structured, developer-profiled alignment of natural-language guidance is a trainable property that improves both objective code accuracy and perceived feedback quality.","feed_headline":"Graded feedback signal lifts code-repair accuracy and human preference","feed_subtitle":"On novice tasks Pass@1 rises 5.7 points; real GitHub issue fixes double over baseline.","key_machinery":"The load-bearing mechanism is the reward-augmented DPO objective (Eq. 4), in which the policy's log-likelihood score for each response is combined with a frozen reward model's scaled score, and the loss contrasts the combined margin of the preferred versus rejected response against the reference model's margin. The frozen reward supplies a graded preference signal that standard DPO lacks. The other key piece is the seven-dimension rubric — Conciseness, Quality, Explainability, Understandability, Completeness, Actionability, and Contextual Relevance — interpreted differently for novice and experienced developers; it defines what 'alignment' means and drives the automatic construction of accep","core_discovery":"The paper's central claim is that code-repair feedback — the natural-language explanations that accompany corrected code — is a first-class alignment target, not a byproduct of learning to repair. It presents DPO-f+, which augments Direct Preference Optimization with a learned reward model's margin signal: instead of treating every accepted/rejected preference pair as equally informative, the combined score in the DPO objective up-weights clear wins and down-weights near-ties. Empirically, DPO-f+ improves Pass@1 on novice programming tasks by 5.71 percentage points over the baseline and 3.30 over standard DPO, and it raises the issue-resolution rate on SWE-bench Lite by 4.67 points over the","pith_inferences":["The paper does not isolate whether the frozen reward-margin signal is the active ingredient or whether the gains come from the extra training signal alone; a controlled ablation that keeps the reward model but removes the margin would identify the mechanism.","Preference-accuracy gains are much larger than downstream Pass@k gains, suggesting rubric alignment and bug-fixing skill are partially separate; a human task measuring comprehension — time-to-understand or ability to apply the suggested fix — would test whether alignment buys comprehension directly.","Because the human calibration used only a small sample, a larger blind evaluation with developers across experience levels would determine whether GPT-4 rubric scores track real preferences beyond the reported 71.5% preference rate.","The recipe could plausibly transfer to code review comments, test generation, or documentation, where natural-language guidance is similarly the bottleneck for human-AI teaming."],"forward_implications":["If DPO-f+ works as reported, preference-aligned feedback is not merely a courtesy: the alignment gains transfer to code correctness, with Pass@1 improving by 5.71 points over baseline on novice tasks and SWE-bench Lite resolution rising by 4.67 points.","Preference pairs can be built automatically from execution-verified, rubric-scored code, so the framework can be re-applied to new tasks without manual preference labeling at each step.","The same training recipe generalizes from introductory C++ tasks to real GitHub issue-fix pairs, producing measurable gains under the official SWE-bench Lite evaluator with identical retrieval support.","Feedback alignment can be evaluated at scale with LLM-as-judge protocols: rubric scores and pairwise judgments both favor DPO-f+, and the reported human calibration shows 95% agreement on a 100-item sample."],"fun_headline_variants":["Feedback that fits: DPO-f+ aligns code-repair explanations with developer preferences","Better code fixes start with feedback developers actually want","Reward-weighted DPO boosts code-repair feedback and fixes","Developer-preferred feedback: DPO-f+ improves code repair and clarity"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The alignment claim rests on the assumption that GPT-4's seven-dimension rubric scores are a reliable proxy for real developer preferences, and that the preference pairs built from those scores are trustworthy ground truth; if that proxy fails, the claim that DPO-f+ is aligned with developers loses its force even if the code-accuracy gains are genuine.","fun_headline_variants_meta":{"raw":{"variants":["Feedback that fits: DPO-f+ aligns code-repair explanations with developer preferences","Better code fixes start with feedback developers actually want","Reward-weighted DPO boosts code-repair feedback and fixes","Developer-preferred feedback: DPO-f+ improves code repair and clarity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000727,"raw_usage":{"total_tokens":3162,"prompt_tokens":878,"completion_tokens":2284,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":622,"completion_tokens_details":{"reasoning_tokens":2210}},"tokens_in":622,"tokens_out":2284,"duration_ms":17647,"temperature":1.0,"reasoning_tokens":2210,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T00:24:18.455232+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a fully blind human study in which novice and experienced developers rate baseline, standard DPO, and DPO-f+ feedback on the same seven dimensions without knowing the source; if DPO-f+ does not beat the alternatives in blind ratings, or if GPT-4 rubric scores correlate at or below chance with human developers' ratings on a validation set larger than the reported 100-item check, the alignment claim fails.","supporting_citations":[],"review_version":1}