{"id":"83a077b7-5013-4346-95ce-b8c434a858e0","arxiv_id":"2504.15210","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Adding symbolic-execution-generated tests to APPS improves reward model accuracy for code generation, but barely changes final code-generation quality relative to CodeRL.","lead":"The paper uses symbolic execution to generate extra programming test cases, then uses them to train reward models that score code from LLMs. The scoring models improve, but the final code generators perform about the same as the existing CodeRL baseline.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Critic evaluation set is unspecified; Table 1 may compare in-sample SE-trained critics against an out-of-distribution CodeRL baseline.","rationale":"The reader's weakest assumption correctly identifies the missing critic evaluation set and the reliance on the same CrossHair pipeline for all described labeled data. My stress-test sharpens this into a concrete risk: the critics may be evaluated in-sample on the exact tasks and code-feedback pairs used for training, while the CodeRL baseline is necessarily out-of-distribution. This is not an accusation of fraud; it is a simple gap in experimental reporting that the paper itself never fills. The authors do give an honest negative actor result and are appropriately cautious about interpreting critic gains, which is credit to the paper. However, the abstract and Section 5.2 present the critic improvement as the main contribution, so the missing evaluation protocol is load-bearing. The proposed check—re-evaluating all critics on a held-out task set with both CrossHair and original APPS labels—would settle whether the symbolic-execution data genuinely improves reward modeling or merely improves performance on the training distribution. Until this is done, the paper cannot be accepted as establishing the central claim, but the concern is addressable, so CONDITIONAL remains the right verdict and no change to the reader's recommendation is needed.","tokens_in":7032,"tokens_out":3706,"duration_ms":36742,"concrete_test":"Request or reconstruct the exact critic evaluation split and check whether any of the tasks or code-feedback pairs used for Table 1 also appear in the critic training set (the 2,402 tasks and their generated codes). Then re-run Table 1 on a held-out set of tasks never used for critic training, using two label sets: (a) CrossHair-generated test outcomes and (b) original APPS test outcomes. If the accuracy gap between CodeRL-SE-critic and CodeT5-finetuned-critic shrinks or reverses under either held-out label set, the central claim fails; if it persists under both, the claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central positive result of the paper is the critic accuracy improvement in Table 1, but the evaluation protocol for that table is never described. Section 3.3 explains that critic training data consists of code-feedback pairs obtained by testing 100 actor-generated codes per task against CrossHair-generated inputs on the 2,402 filtered APPS training tasks. No held-out task set or label source is specified for the critic evaluation reported in Table 1. If that table is computed on the same 2,402 tasks or on a random split of code samples from those tasks, then CodeRL-SE-critic and CodeT5-SE-critic are being evaluated in-distribution relative to their training data, while CodeT5-finetuned-critic, trained on original APPS test results, is evaluated on SE-generated test cases it never saw. The 37.19% relative accuracy gain could then be an artifact of training/evaluation overlap plus distribution mismatch, rather than evidence that symbolic-execution-augmented data produces better reward models. The paper reports no error bars, no multiple seeds, and no external evaluation labels that would disambiguate this. Because the actor results are honestly negative, the entire positive claim of the paper rests on this unspecified critic evaluation; without it, the headline improvement is not yet established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies whether symbolic-execution-generated test inputs improve reward-based fine-tuning of code-generating LLMs. The authors filter the APPS train set down to 2,402 single-function tasks, use MonkeyType to infer type annotations and CrossHair to generate additional input-output test cases, and then train two CodeT5-based critic models (CodeRL-SE-critic and CodeT5-SE-critic) on code-feedback pairs derived from those tests. The critics are used in a CodeRL-style actor-critic loop to fine-tune CodeT5-large-ntp-py, and a separate DPO training run is performed. The headline result is that both new critics beat the existing CodeRL critic in classification accuracy (0.4250 and 0.3449 vs. 0.3098), while actor pass@5 is only marginally better than the CodeRL baseline (4.37 vs. 4.23) and DPO underperforms. The paper concludes that symbolic-execution-augmented data improves reward models, but that this improvement does not yet translate into substantial actor gains.","tokens_in":7290,"tokens_out":6151,"duration_ms":53645,"significance":"If the critic improvement is real, the paper provides useful evidence that automatically generated symbolic-execution test cases can serve as a denser and more objective reward signal than sparse human-written unit tests. The pipeline itself, combining MonkeyType and CrossHair to augment APPS, is concrete and reproducible in principle, and the honest reporting of the near-parity actor results is a valuable negative result for the RL fine-tuning community. The paper also raises a worthwhile research question: why a stronger critic does not translate into a stronger actor. However, the central positive claim currently rests on an unspecified critic evaluation protocol, and the paper provides no error bars, multiple seeds, or code release, so the significance of the 37.19% accuracy gain is not yet established.","major_comments":[{"comment":"The evaluation protocol for Table 1 is unspecified, and this is load-bearing for the paper's central claim. The paper never states which tasks or which ground-truth labels produce the accuracy numbers for the three critics. Section 3.3 describes critic training on code-feedback pairs obtained by testing actor-generated code against CrossHair-generated inputs on the 2,402 filtered APPS training tasks, but it does not say whether the Table 1 evaluation uses those same tasks, a held-out subset, or the 2,500-task APPS test set. If the SE-trained critics are evaluated on labels produced by the same CrossHair pipeline that generated their training labels, while the CodeRL baseline critic is evaluated on original APPS test labels, then the reported 37.19% relative gain is confounded by training/evaluation overlap and by label-distribution shift. The authors must specify the evaluation task set, the label source, the overlap with training data, and whether the baseline critic is evaluated under identical conditions; they should also report multiple seeds or confidence intervals, since no error bars are given.","section":"§3.3, §4.1, Table 1"},{"comment":"The paper motivates the contribution by claiming that symbolic execution 'ensures the coverage of all CFG paths,' but it never measures path coverage. Figure 3 and the text in Section 5.1 only report the number of test cases per task, which is not the same as CFG path coverage. The claim of complete path coverage is therefore not supported by the presented evidence. If the authors wish to retain the coverage motivation, they should either report a coverage metric (e.g., branch or statement coverage achieved by the generated inputs) or soften the wording to 'increased path coverage' with appropriate evidence.","section":"§3.2, §5.1"},{"comment":"The comparison of the DPO and RL actor results in Table 2 is hard to interpret because the paper does not state the amount of training data used for DPO (how many tasks, how many chosen-rejected pairs) or the number of independently trained models over which the pass@5 numbers are computed. Given that the actor improvements are small (0.14 absolute overall), a single seed is not sufficient to conclude that RL with CodeRL-SE-critic outperforms the CodeRL baseline. The authors should report variance across at least a few seeds, or explicitly acknowledge that the actor differences are within noise.","section":"§4.3, §5.3"}],"minor_comments":[{"comment":"The sentence 'The mean number of test cases increases from 1 to 5, and the median from 5.16 to 7.22' appears to swap the mean and median values; as written, an original median of 5.16 contradicts the earlier statement that 2012 of the 5000 train tasks contain exactly one test case. Please correct the numbers or the assignment.","section":"§5.1"},{"comment":"The text 'we use ak of 5' should read 'we use a k of 5' or 'we use k=5'.","section":"§3.5"},{"comment":"The phrase 'There have been invented several frameworks' is awkward and should be rephrased.","section":"§2"},{"comment":"No code, data, or configuration files are released; providing the filtering script, the CrossHair/MonkeyType pipeline, and the evaluation script would materially support reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the journal's scope and the idea is reasonable, but the central claim rests entirely on the Table 1 critic comparison, whose evaluation protocol is missing. I am not suggesting misconduct; this looks like an incomplete experimental description. The authors should be given a chance to supply the missing protocol and, if necessary, rerun the critic evaluation on a properly held-out set with identical labels for all models."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper tries something new — using symbolic-execution-generated tests as training data for the reward model in RL/DPO fine-tuning of code LLMs. That combination is not in the cited prior work, which uses symbolic execution at inference or for test generation, and RL/DPO separately. The paper also reports the actor result honestly: the better critic does not translate into meaningfully better code generation, which is a useful, if sobering, data point.\n\nWhat it does well: the pipeline is straightforward and the idea is plausible — denser, path-sensitive test feedback should give the critic a less coarse training signal. They evaluate both an RL variant and DPO, and they put the negative actor result front and center rather than hiding it. The paper is clearly written and the methodology is easy to follow.\n\nWhere it is soft: the central positive claim, the critic accuracy improvement in Table 1, is not yet established. The paper never says which tasks or labels were used to compute those critic accuracies. The only labeled data described comes from the same CrossHair pipeline that generated the training data. If the SE-trained critics are evaluated in-distribution on SE-generated labels while the CodeRL baseline is evaluated on original APPS labels, part of the 37.19% gain is distribution mismatch. The stress-test note is right about this. No error bars, no multiple seeds, and no code/data release make it hard to assess variance. The filtering from 5000 to 2402 train tasks is a generalization concern, though not a fatal one. There is also a minor typo where mean and median seem swapped in Section 5.1.\n\nThat said, the paper is not sloppy in its reasoning; the actor discussion anticipates the critique that better critic quality doesn't automatically help the policy. The missing evaluation protocol is a fixable omission rather than a fundamental flaw.\n\nWho this is for: people working on reward models and RL fine-tuning for code generation will find the combination worth knowing about, especially the negative actor result. It deserves a serious referee, but the authors should be asked to specify the critic evaluation set, compare all critics on identical external labels, run multiple seeds, and ideally release the data and code.\n\nBottom line: engage with it, but treat the headline critic number as conditional until the evaluation protocol is released.","headline":"A genuinely new combination of symbolic execution and reward-model fine-tuning, with an honest negative actor result, but the headline critic improvement rests on an unspecified evaluation protocol.","tokens_in":7789,"tokens_out":2174,"would_cite":false,"duration_ms":19048,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Symbolic execution sharpens LLM code critics, but code generators barely improve.","keywords":["symbolic execution","reinforcement learning","reward model","code generation","CodeT5","APPS dataset","direct preference optimization","pass@k"],"falsifier":"Re-run the Table 1 comparison on a jointly fixed evaluation set—for example, the original APPS test cases for all three critics—and recompute accuracy and mean absolute error. If the symbolic-execution-trained critic no longer beats the CodeRL critic by a wide margin, the headline improvement is at least partly a distribution artifact. A second check is to train one critic on APPS's original tests while evaluating on symbolic-execution-generated labels, and another critic with the roles reversed, to separate label-source effects from genuine cross-distribution skill.","tokens_in":6825,"feed_emoji":"🧪","tokens_out":5468,"duration_ms":44601,"temperature":0.7,"pith_summary":"This paper asks whether reward models that judge generated code can be trained on better test data than human-written unit tests. It augments the APPS coding benchmark with inputs generated by symbolic execution, which systematically walks every control-flow path instead of relying on a few hand-picked cases. Reward models fine-tuned on this enriched data classify code into pass/fail categories far more accurately than the CodeRL baseline critic, with accuracy rising from 0.3098 to 0.4250, a relative gain of 37.19 percent. The paper also checks whether this sharper reward signal improves the code generators themselves, and here the answer is muted: the best actor gains only 0.14 absolute pass@5 over CodeRL, while a DPO-trained model does worse. The contribution is a demonstration that symbolic execution can supply denser and more objective training labels for learned code evaluation, leaving open why better critics do not translate into better policies.","feed_headline":"Symbolic execution lifts LLM code critics by 37 percent","feed_subtitle":"Machine-generated test cases make code-quality judges more accurate, yet the code generators barely improve.","key_machinery":"The load-bearing mechanism is the pairing of CrossHair, a symbolic execution engine that enumerates one representative input per control-flow-graph path, with MonkeyType, which supplies the type annotations CrossHair needs in order to explore a Python function meaningfully. Together they turn the original APPS train set into a 2,402-task dataset with roughly five times as many test cases per problem, each path exercised exactly once. That dataset produces code-feedback pairs used to train critic models, which then score actor-generated samples inside the CodeRL actor-critic loop; DPO uses the same pairs stripped of the explicit critic. The design isolates the effect of the reward signal by keeping the base models (CodeT5-base for critics, CodeT5-large-ntp-py for actors) identical to CodeRL's.","core_discovery":"On the paper's own terms, the central claim is that symbolic-execution-generated test cases are a superior training signal for learned code-quality judges. By running CrossHair over 2,402 APPS tasks with MonkeyType-inferred type annotations, the authors expand the average test count per task and cover every control-flow path exactly once. Reward models fine-tuned on the resulting code-feedback pairs—either by fine-tuning the existing CodeRL critic or by training CodeT5-base from scratch—beat the stock CodeRL critic on classification accuracy (0.4250 and 0.3449 versus 0.3098) and on mean absolute error. The same enriched signal does not produce a corresponding jump in the code-generating actor: the best RL-trained actor reaches 4.37 pass@5 versus CodeRL's 4.23, and DPO training lands at 3.81. The paper reads this as evidence that symbolic execution improves reward-model quality substantially while leaving the policy-optimization bottleneck open.","pith_inferences":["A sharper critic may be selecting for agreement with the ground-truth program's control-flow paths, which could bias the reward toward code that mirrors the reference implementation rather than any correct program; testing on diverse correct solutions would reveal this.","The missing shared evaluation set for critics is a testable extension: grading all critics on the original APPS unit tests could shrink or flip the reported 37 percent gain if the new critics were trained to predict symbolic-execution-generated labels.","The actor plateau hints that the current RL hyperparameters or sample budget, not reward accuracy, cap pass@5; varying those while holding the critic fixed would separate the two effects.","Since symbolic execution produces deterministic labels, the approach could also be used to measure reward-model calibration, not just classification accuracy, on path-based test suites."],"forward_implications":["A better critic does not automatically make a better policy: the 37 percent critic gain shrinks to a 0.14 pass@5 gain for the actor, so future work must treat reward quality and policy optimization as separate bottlenecks.","Automated test generation can replace hand-written unit tests as the labeling source for RL fine-tuning of code models, removing human test-design bias at least for single-function tasks.","Because DPO underperforms CodeRL, the benefit of symbolic-execution data appears tied to the explicit reward-model pathway, not simply to having better chosen-rejected pairs.","The pipeline's applicability is limited by the requirement that each task reduce to a single standalone function with inferable types; tasks that do not fit are discarded, so the method's reach is currently narrow.","If the critic gains generalize, the same enrichment recipe could be applied to other benchmarks and languages, but the actor results suggest expectations should be modest."],"supporting_citations":[{"why":"Supplies the APPS benchmark, the 5,000 train and 5,000 test tasks the paper augments and evaluates on.","marker":"Hendrycks et al., 2021"},{"why":"Provides the CodeRL actor-critic framework, the baseline critic and actor, and the warmed-up CodeT5-large-ntp-py model.","marker":"Le et al., 2022"},{"why":"Provides CodeT5-base, the shared backbone for both the new critic and the CodeRL models.","marker":"Wang et al., 2021"},{"why":"Defines symbolic execution, the path-enumeration technique behind the generated test cases.","marker":"King, 1976"},{"why":"Supplies the DPO training algorithm used for the implicit-reward comparison.","marker":"Rafailov et al., 2024"},{"why":"Defines pass@k, the metric used to evaluate all actor models.","marker":"Chen et al., 2021"},{"why":"Cited for the distribution-mismatch explanation of why a correct reward may not improve the actor.","marker":"Casper et al., 2023"},{"why":"Cited for DPO's tendency to over-weight out-of-distribution data, invoked to explain DPO underperformance.","marker":"Xu et al., 2024"}],"fun_headline_variants":["Symbolic execution boosts LLM code judges, not generators","Better code critics via symbolic tests, but generators lag","Test-case synthesis sharpens code-quality models, yet actors stall","Reward models gain from symbolic execution; code generation barely moves","Code judges get 37% more accurate, but code generators stay flat"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The 37 percent critic gain rests on the assumption that all three critics were graded on the same held-out task set with the same ground-truth labels; the paper never specifies that shared evaluation set, and the only labeled data it describes comes from the same symbolic-execution pipeline used for training.","fun_headline_variants_meta":{"raw":{"variants":["Symbolic execution boosts LLM code judges, not generators","Better code critics via symbolic tests, but generators lag","Test-case synthesis sharpens code-quality models, yet actors stall","Reward models gain from symbolic execution; code generation barely moves","Code judges get 37% more accurate, but code generators stay flat"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000218,"raw_usage":{"total_tokens":1401,"prompt_tokens":871,"completion_tokens":530,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":487,"completion_tokens_details":{"reasoning_tokens":445}},"tokens_in":487,"tokens_out":530,"duration_ms":5382,"temperature":1.0,"reasoning_tokens":445,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:30:15.661881+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the Table 1 comparison on a jointly fixed evaluation set—for example, the original APPS test cases for all three critics—and recompute accuracy and mean absolute error. If the symbolic-execution-trained critic no longer beats the CodeRL critic by a wide margin, the headline improvement is at least partly a distribution artifact. A second check is to train one critic on APPS's original tests while evaluating on symbolic-execution-generated labels, and another critic with the roles reversed, to separate label-source effects from genuine cross-distribution skill.","supporting_citations":[],"review_version":1}