{"id":"0c976098-1c79-44b7-8984-462d1f307700","arxiv_id":"2502.04306","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"ScoreFlow uses a score-weighted variant of direct preference optimization to automatically generate and refine per-task LLM agent workflows, reporting an average 8.2% improvement over baselines on six benchmarks.","lead":"This paper introduces ScoreFlow, a method that automatically builds and improves multi-step AI agent workflows by optimizing the workflow generator with score-weighted preference learning. If it works, it could make complex LLM-based systems cheaper and easier to design, letting smaller models match or beat larger ones.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Appendix Table 5 says the optimization process ran on 'test data'; if literal, the headline test-set numbers are contaminated, and this must be resolved before the results can be accepted.","rationale":"The reader's stated weakest assumption is about Theorem 3.2's condition. That is not the most load-bearing issue: with the default f(x)=x, |rz|<=1 indeed implies the theorem's bounds, since -1/(1-s)<=-1 and 1/s>=1 for s in (0,1], so the reader's 'rz<=sz' reading is inaccurate. Moreover, the empirical contribution does not stand or fall on that monotonicity theorem. The test-split ambiguity in Appendix A.3.4 is more serious because it threatens every headline number. I frame it conditionally because the caption may be loose language, but it must be resolved before acceptance. The judge/evaluator overlap is a second, concrete threat to the robustness of the claimed gains. Verdict remains CONDITIONAL, now conditioned on resolving the split and evaluator-overlap issues rather than on the theorem condition.","tokens_in":18841,"tokens_out":11561,"duration_ms":108958,"concrete_test":"Trace the released code at github.com/Gen-Verse/ScoreFlow to identify which split is passed to Algorithm 1's preference-collection loop for each benchmark. If any optimization iteration samples from the same split used to produce Table 1 test numbers, re-run the full pipeline with optimization restricted to the 20% validation split and re-report test-set solve rates; if Tables 1–3 shift materially, the central claim is unsupported. Independently, re-evaluate MATH, DROP, and HotpotQA final outputs with a judge that was not used during optimization (e.g., exact-answer matching or GPT-4o) and report per-seed standard deviations.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central empirical claim—85.3% average solve rate and 8.2% improvement—depends on the test split being held out from Score-DPO's optimization loop. Section 4.1 says data are split 1:4 validation/test and Tables 1–3 report test-set numbers, but Appendix A.3.4's Table 5 is explicitly captioned 'The detailed cost value ($) in Figure 4b (optimization process on test data).' If Algorithm 1's preference-collection loop consumed the test split for the cost experiment on HumanEval, the same queries used to compute the 95.9% HumanEval number were seen by the generator during optimization. The paper never states that only the validation split was used for optimization, and this caption suggests otherwise. That would invalidate the central claim, independent of whether Theorem 3.2's sufficient condition is met. At minimum, a precise statement of which split feeds Algorithm 1 is required. Secondary but related: for MATH, DROP, and HotpotQA the final judge is GPT-4o-mini, the same model used as executor during optimization, and no error bars are reported, so part of the margin could reflect judge alignment or noise.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"ScoreFlow proposes an automated framework that generates per-task LLM multi-agent workflows expressed as Python code. The workflow generator, a small open-source LLM (default Llama-3.1-8B-Instruct), is iteratively fine-tuned with Score-DPO, a variant of DPO that uses quantitative evaluation scores through an upweighting function d(x,y) and a score-scaling function f(x). The paper reports an average solve rate of 85.3% across six benchmarks, an 8.2% improvement over baselines, and shows that smaller executors can outperform larger ones after workflow optimization. It also contains a theoretical analysis claiming that Score-DPO makes per-sample influence monotonically increasing with the evaluation score under a stated condition.","tokens_in":19135,"tokens_out":6293,"duration_ms":64689,"significance":"If the empirical claims hold, the paper makes a practically valuable contribution: it reduces the cost of workflow optimization by using a small open-source generator, demonstrates gains across QA, coding, and math benchmarks, and includes comparisons against SFT, PPO, DPO, AFlow, and ADAS. The cost analyses in Figure 4 and the cross-model ablations in Table 3 are useful, and the idea of incorporating quantitative scores into DPO is likely to be of independent interest. However, two load-bearing points need attention before the claims can be accepted: the possibility that optimization used test data, and an inconsistency between the stated sufficient condition in Theorem 3.2 and the default choice f(x)=x. The absence of error bars further weakens the empirical claims.","major_comments":[{"comment":"The caption of Table 5 reads '(optimization process on test data).' Section 4.1 says the data are split 1:4 into validation and test, and Tables 1-3 report test-set solve rates. Algorithm 1 consumes its input dataset D for preference collection and generator fine-tuning. If the optimization cost experiment was run on the test split, then the workflows used to compute the final test numbers were optimized on test queries, which would invalidate the headline results regardless of the theoretical analysis. The paper never states that only the validation split was used for optimization; please state explicitly which split feeds Algorithm 1 for every reported experiment, and if any optimization step used test data, recompute all test-set results with optimization confined to the validation split.","section":"A.3.4, Table 5 and Algorithm 1"},{"comment":"As printed, the theorem states the condition as -(1 - f(s_z))^{-1} <= r_z <= f^{-1}(s_z). With the default f(x)=x, f^{-1}(s_z)=s_z, so the condition is -(1 - s_z)^{-1} <= r_z <= s_z. This is not implied by |r_z| <= 1; for example, s_z=0.1 and r_z=0.5 satisfy |r_z|<=1 but violate r_z <= s_z. The proof's use of Lemma A.1 suggests the intended upper bound may be 1/f(s_z) rather than f^{-1}(s_z), and with that reciprocal interpretation |r_z|<=1 would be sufficient, but the notation in the theorem statement is then incorrect. As written, the theorem is not established for the default function, and Appendix A.3.3 verifies only P(|r_z|<=1), not the condition stated in the theorem. The authors should correct the condition and report the empirical frequency of the actual condition used.","section":"3.5, Theorem 3.2 and Appendix A.1"},{"comment":"All reported results are averages over three evaluations without standard deviations, confidence intervals, or significance tests. The central claim of an 8.2% average improvement and several per-dataset margins (for example, DROP at 86.2 vs 83.5 and MBPP at 84.7 vs 82.9) might be within run-to-run variation. The paper should provide variance estimates or significance tests for the main comparison against AFlow and against the DPO ablation; without these, the empirical strength of the claim is difficult to assess.","section":"4.2, Tables 1-3"},{"comment":"For MATH, DROP, and HotpotQA the final judge is GPT-4o-mini, which is also the model used as the executor during optimization. This creates a potential confound: part of the reported improvement may reflect agreement with the judge model rather than correctness on the underlying task. The paper should acknowledge this limitation, provide a small human-verified or rule-based evaluation for these datasets, or report judge-agreement statistics to show the result is not driven by judge bias.","section":"4.1, Metrics and Evaluation Scores"}],"minor_comments":[{"comment":"There is a typo: 'validaton' should be 'validation.'","section":"4.1"},{"comment":"The Input list numbers the items incorrectly: there are two items labeled '5).' Please renumber the inputs.","section":"A.2.3, Algorithm 1"},{"comment":"The caption says 'All methods are executed using GPT-4o-mini,' but ScoreFlow uses Llama-3.1-8B-Instruct as the generator for the main results. Please clarify that GPT-4o-mini is the executor, not the generator for ScoreFlow, to avoid confusion.","section":"Table 1 caption"},{"comment":"Several training hyperparameters are not reported: LoRA rank and alpha, learning rate, batch size, and the number of Score-DPO update steps per iteration. These are needed to reproduce the method.","section":"A.2.3, Algorithm 1 and 4.1"},{"comment":"The notation P^*(w,l) is used both for the weighted sampling distribution and, in Algorithm 1, for the sampled preference loss; please define the relationship to P(w,l) explicitly in the algorithm listing.","section":"3.4"}],"recommendation":"major_revision","confidential_remarks":"The most serious issue is the Table 5 caption stating that optimization was run on test data. If that is literal, the paper's main results are contaminated and the manuscript should not be accepted unless all test-set numbers are recomputed with optimization on the validation split only. The Theorem 3.2 condition also needs correction; the current text is internally inconsistent with the default f(x)=x choice. I would not raise the threshold to reject at this stage because both issues may be fixable with clarification and reruns, but they must be resolved before the paper can be considered for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about arXiv:2502.04306. First, the method is a sensible incremental step: Score-DPO takes DPO and adds score-scaled rewards and score-weighted sampling, then applies it to generate per-task agent workflows in code. That is a real extension of AFlow, and the experiments are broad. Second, there is a credible contamination red flag: Appendix A.3.4's Table 5 is captioned \"optimization process on test data.\" If that caption is literal, the optimization loop consumed the test split on HumanEval, and the headline 95.9% and the 85.3% average are not trustworthy. The paper never says that only the validation split feeds Algorithm 1, so the burden is on the authors to clarify.\n\nThe design choices are reasonable: code as workflow representation, per-task adaptivity, LoRA fine-tuning a small open generator, and a third-party executor for scoring. The ablations against SFT/PPO/DPO and across executors are useful. The cost-efficiency story is plausible.\n\nThe theory is overclaimed. Theorem 3.2 gives a condition involving f; with f(x)=x the condition is -1/(1-sz) <= rz <= sz, and |rz|<=1 does not imply rz<=sz. The 91.1% statistic in A.3.3 checks the wrong condition, so the monotonic-influence claim is not supported. That weakens the theoretical section, but the empirical result does not rest on it. Also, no variance bars are reported; \"evaluated three times\" without spread is thin. The same GPT-4o-mini is both optimization-time executor and final judge for three benchmarks, so judge alignment may inflate the gap. The cost experiment on test data is the main issue, not these secondary ones.\n\nThis paper is for people working on automated workflow generation and preference optimization. It deserves a serious referee, not a desk reject, but the first thing a referee should demand is a precise statement of which split feeds Algorithm 1, and a rerun or explanation if the caption is wrong. If the test-data leak is real, the headline comparisons need to be redone. I would not cite the numbers until that is settled.","headline":"A useful incremental method with a serious unresolved data-split question; the theory is overclaimed but the empirical comparison is the real issue.","tokens_in":19631,"tokens_out":2570,"would_cite":false,"duration_ms":24868,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ScoreFlow claims that LLM agent workflows can be automated by fine-tuning a code-writing generator with score-weighted preference pairs, reaching 85.3% average solve rate and letting smaller models beat larger ones at lower cost.","keywords":["LLM multi-agent workflows","workflow optimization","score-based preference optimization","direct preference optimization","code representation","automatic workflow generation","mathematical reasoning","question answering"],"falsifier":"Log the pair $(s_z, r_z)$ for every sampled workflow during a ScoreFlow run; if the measured per-sample influence is not increasing in $s_z$ whenever the exact condition $-(1-f(s_z))^{-1}\\le r_z\\le f^{-1}(s_z)$ fails, while $|r_z|\\le1$ still holds, then the paper's sufficient condition is shown not to guarantee the mechanism on those samples.","tokens_in":18664,"feed_emoji":"🤖","tokens_out":11111,"duration_ms":97897,"temperature":0.7,"pith_summary":"ScoreFlow claims that the labor of designing multi-agent LLM workflows can be automated by treating workflow generation as code generation and optimizing the generator with a score-aware version of direct preference optimization. Rather than searching over discrete workflow structures, the method samples several workflows per task, executes them, scores the results, and fine-tunes the generator on preference pairs weighted by score gaps and rescaled rewards. On six benchmarks covering question answering, coding, and mathematical reasoning, ScoreFlow reports an average solve rate of 85.3%, which the paper reports as an 8.2% improvement over existing baselines. The practical payoff would be that smaller, cheaper models can outperform larger ones while lowering inference and optimization cost.","feed_headline":"85.3% solve rate follows from score-weighted workflow tuning","feed_subtitle":"Gradient-based optimization replaces discrete workflow search, letting small models beat larger ones for less.","key_machinery":"The mechanism is the Score-DPO loss\n$$L_{\\mathrm{Score-DPO}} = -\\mathbb{E}_{(w,l)\\sim P^*}\\left[\\log\\$\\sigma$\\left(f(s_w)r_w - (1-f(s_l))r_l\\right)\\right],$$\nwith $r_y=\\beta\\log(\\pi_\\theta(y\\mid x)/\\pi_{\\mathrm{ref}}(y\\mid x))$ and sampling distribution $P^*(w,l)\\propto d(s_w,s_l)P(w,l)$, where $f$ and $d$ are strictly increasing in the score and score gap respectively (defaults $f(x)=x$, $d(x,y)=(x-y)^3$). The loss is what carries the argument: it up-weights confident preference pairs and scales each item's reward by its score, so that Theorem 3.2 can show per-sample influence grows with score under the condition $-(1-f(s_z))^{-1}\\le r_z\\le f^{-1}(s_z)$. The surrounding pipeline iteratively generates $k=8$ workflows per task, executes them with an independent executor, and fine-tunes the code-writing generator on the resulting scored pairs.","core_discovery":"Score-DPO is the paper's central proposal: it takes the standard DPO objective and injects the quantitative evaluation score $s$ into both the sampling distribution and the ranking target. Preference pairs $(w,l)$ are sampled with probability proportional to $d(s_w,s_l)$, and the implicit reward gap becomes $f(s_w) r_w - (1-f(s_l)) r_l$, so pairs with decisive scores dominate the gradient. Theorem 3.2 states that, under a range condition on $r_z$, the per-sample influence of a workflow on the loss is strictly increasing in its score, which the paper argues reduces the harm of noisy evaluation scores that slows DPO. Empirically, the full ScoreFlow pipeline, which generates eight workflows per problem per iteration, executes them, and runs three Score-DPO fine-tuning iterations, outperforms the baselines and, on HumanEval, lets small executors beat the chain-of-thought outputs of a larger model.","pith_inferences":["The score-weighting recipe is not tied to workflow generation: the same loss could be applied to any preference-tuning task with continuous rewards, such as human quality ratings or verifier scores, though the paper only demonstrates it on six agentic benchmarks.","The approach suggests a natural curriculum: because pairs with larger score gaps are up-weighted first, later iterations could anneal $d(x,y)$, for example from cubic toward linear, to transition from coarse to fine preference learning.","Since the generator optimizes per-task workflows, the method could be extended to reuse trained workflows across similar tasks, amortizing the one-time optimization cost over a whole dataset; the paper does not test transfer."],"forward_implications":["Automated workflow optimization shifts from discrete search over structures to gradient-based fine-tuning of the generator, which the paper shows scales to a combined dataset of math, coding, and question-answering problems better than the discrete-search baseline.","Score-DPO can be applied wherever a scalar score, not just a binary preference, is available, so any evaluation metric that can be computed per output can feed the loss.","Because the generator is an open-source 8B model and converges within two to three iterations, the optimization and inference costs are below those of methods that repeatedly call a large optimizer model.","Smaller executors such as GPT-4o-mini and DeepSeek-V3, once wrapped in a ScoreFlow-optimized workflow, beat the chain-of-thought output of the larger GPT-4o on HumanEval, implying that workflow quality can matter more than raw model size."],"supporting_citations":[{"why":"Defines direct preference optimization, the base objective that Score-DPO extends with score weighting and reward scaling.","marker":"[27]"},{"why":"Supplies the Bradley-Terry pairwise ranking model behind the sigmoid objective in the Score-DPO loss.","marker":"[5]"},{"why":"Documents that preference-learning algorithms like DPO do not learn preference rankings, motivating the score-guided correction.","marker":"[6]"},{"why":"Introduces code representation for agentic workflows and serves as a baseline ScoreFlow must beat.","marker":"[14]"},{"why":"Establishes the AFlow code-based workflow search baseline and the operator set ScoreFlow reuses.","marker":"[46]"},{"why":"Provides the PPO alternative against which Score-DPO is compared within the same pipeline.","marker":"[30]"}],"fun_headline_variants":["ScoreFlow: Score-DPO tunes agent workflows, lifts accuracy 8.2%","Gradient-based workflow tuning uses scores to let small models excel","Score-weighted preference optimization sharpens LLM agent workflows","Quantitative feedback into DPO yields better agent workflow tuning","Small models beat large ones with score-aware workflow optimization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the range condition in Theorem 3.2 holds during optimization so that each sample's influence on the loss rises with its evaluation score; the paper verifies only the sufficient proxy $|r_z|\\le1$, which it reports holds with probability about 91.1% before convergence, leaving the remaining samples without a proven monotonicity guarantee.","fun_headline_variants_meta":{"raw":{"variants":["ScoreFlow: Score-DPO tunes agent workflows, lifts accuracy 8.2%","Gradient-based workflow tuning uses scores to let small models excel","Score-weighted preference optimization sharpens LLM agent workflows","Quantitative feedback into DPO yields better agent workflow tuning","Small models beat large ones with score-aware workflow optimization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001358,"raw_usage":{"total_tokens":5476,"prompt_tokens":874,"completion_tokens":4602,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":490,"completion_tokens_details":{"reasoning_tokens":4516}},"tokens_in":490,"tokens_out":4602,"duration_ms":34675,"temperature":1.0,"reasoning_tokens":4516,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T22:49:58.647210+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Log the pair $(s_z, r_z)$ for every sampled workflow during a ScoreFlow run; if the measured per-sample influence is not increasing in $s_z$ whenever the exact condition $-(1-f(s_z))^{-1}\\le r_z\\le f^{-1}(s_z)$ fails, while $|r_z|\\le1$ still holds, then the paper's sufficient condition is shown not to guarantee the mechanism on those samples.","supporting_citations":[{"cited_title":"Automated design of agentic systems","cited_arxiv_id":null,"evidence_quote":"Introduces code representation for agentic workflows and serves as a baseline ScoreFlow must beat."}],"review_version":1}