{"id":"5bb1da91-af71-4478-a996-40a41358ba13","arxiv_id":"2607.10474","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"RLVP post-trains one LLM across eight PDE families with hybrid validity-plus-continuous physics rewards, improving solver accuracy and enabling selective compositional transfer to held-out PDEs.","lead":"Researchers post-trained language models with continuous physics rewards so the models write better numerical PDE solvers, not just code that runs. Smaller trained models beat much larger frontier models on in-distribution solver generation and partially transfer numerical motifs to related unseen equations.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Continuous physics rewards may overfit to training-reference solvers and operator-recombination structure rather than teaching scheme-agnostic physical correctness.","rationale":"The reader’s weakest assumption correctly identifies the load-bearing soft spot: whether the hand-tuned continuous verifier is a generalizable physics signal or an environment-specific ranking over particular references, grids, and operator recombinations. In-distribution Base/SFT/RLVP gains and the frontier direct-generation comparison (Table 1; Fig. 3) are empirically solid under the stated protocol; the continuous-reward ablation further supports that R_traj matters beyond validity. Those results do not need to be overturned. What remains least secure is the interpretation that the policy has internalized transferable numerical methods rather than optimized to the verifier’s reference distribution. The held-out suite is selective by construction and does not stress-test reference- or grid-invariance. Artifact non-release and limited multiphysics/geometry scope already justify CONDITIONAL; the reference-overfitting concern reinforces that verdict without requiring a harsher one. A single independent-reference re-evaluation would settle whether the concern lands. No stronger internal inconsistency was found; novelty remains compositional but real.","tokens_in":35118,"tokens_out":733,"duration_ms":49113,"concrete_test":"Re-evaluate the released 7B RLVP checkpoint on the full 143 seen-PDE cases using independently implemented references from a different scheme family than the training verifier (e.g., spectral/IMEX where training used FD/FV, or higher-order FV where training used low-order). Keep prompts, k=8, and nRMSE≤10^{-2} fixed. If pass@8 drops by >0.15 or median-best nRMSE@8 worsens by >5× relative to Table 1, the continuous reward has overfit to reference-solver artifacts rather than scheme-agnostic physics.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that RLVP internalizes transferable numerical reliability rests on R(y,c)=V·R_traj·R_phys (§3.5 Eq. 5), with R_traj=exp(−L_traj/T_traj), T_traj=0.05, and pass success defined by nRMSE≤10^{-2} (App. A). L_traj is measured only against the environment’s fixed numerical references on the training grid/parameter envelope. Nothing in the design prevents the policy from matching reference-solver idiosyncrasies (numerical diffusion, projection residuals, CFL substep patterns) rather than learning scheme-agnostic correctness. The residual term is reference-relative (Eq. 4), so it cannot fully break that coupling. Held-out transfer (§4.3; App. F) is strongest precisely where operators recombine seen motifs (heat ⊂ RD; 2D advection as 1D lift; ARD as advection+reaction+diffusion) and weak on KdV, Darcy–reaction, and advection–diffusion—consistent with template recombination inside a narrow operator algebra, not broad physics grounding. The continuous-vs-validity ablation (§4.4) shows R_traj helps, but does not show that the learned ranking is reference- or grid-invariant. If the dense signal is partly reference-matching, both the in-distribution nRMSE gains (Table 1) and the compositionality narrative are less secure than claimed.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper introduces RLVP, a GRPO-based post-training method that adapts LLMs to generate numerical PDE solver code using a hybrid verifier: binary execution validity gates multiplied by continuous rewards for function-space trajectory error (R_traj) and reference-relative residual consistency (R_phys). After SFT on a multi-scheme solver bank spanning eight PDE families (hyperbolic, parabolic, elliptic, incompressible flow), a single policy is trained jointly and evaluated by pass@k and best-of-k nRMSE against hidden numerical references. Across 3B/7B/14B Qwen2.5-Coder models, RLVP substantially improves over base and SFT checkpoints on seen PDEs, can outperform static prompting of larger frontier models under the same direct-generation protocol, and yields selective gains on held-out PDEs that recombine trained numerical motifs. An ablation shows that adding R_traj improves accuracy beyond validity-only RL.","tokens_in":35533,"tokens_out":1343,"duration_ms":38195,"significance":"If the results hold under broader scrutiny, this is a clear and timely contribution: it moves scientific code generation from inference-time scaffolding to amortized post-training with graded physical feedback, and it is among the first multi-PDE RLVR pipelines with continuous rather than binary verifiers. Strengths include a carefully documented solver bank with self-convergence and MMS checks, controlled Base/SFT/RLVP comparisons at three scales, frontier baselines under a matched protocol, a validity-only ablation, and an intended open release of the verifier environment and recipes. The practical claim that a smaller post-trained model can beat frontier prompting on in-distribution solver generation is well supported and of real interest to scientific computing and LLM post-training communities.","major_comments":[{"comment":"§3.5 Eqs. (3)–(5) and Appendix A fix T_traj=0.05, T_phys=2.0, and the success threshold nRMSE≤10^{-2} without sensitivity analysis. These free parameters define both the dense training signal and the pass@k metric. Because the central claim is that continuous physics rewards (not just validity) drive gains, the paper should report at least a limited sweep or robustness check on T_traj (and preferably T_phys and the success threshold) for the 7B model on seen and held-out aggregates; otherwise it is hard to know how much of Table 1 and Figs. 8–9 depends on this particular temperature/threshold choice.","section":null},{"comment":"§4.3 and Appendix F present compositionality and zero-shot transfer as recombination of stencils, CFL control, and time-stepping motifs. The quantitative gains on held-out PDEs are real, but the strongest cases (heat ⊂ reaction–diffusion; 2D advection as a 1D lift; ARD as advection+reaction+diffusion) sit inside the trained operator algebra, while KdV, Darcy–reaction, and advection–diffusion remain weak (Table 16). The abstract and contribution list should more tightly match this selectivity. Please either (i) add a controlled motif-ablation or feature-frequency analysis that separates recombination from generic code quality, or (ii) explicitly frame transfer as selective numerical-motif reuse rather than broad physics internalization.","section":null},{"comment":"§3.5 Eq. (1) and Eq. (4): L_traj is measured only against the environment’s fixed numerical references, and R_phys is reference-relative. This is a legitimate RLVR design, but it leaves open whether the policy is learning scheme-agnostic physical correctness or matching reference-solver idiosyncrasies (numerical diffusion, projection residuals, CFL patterns). A load-bearing check for the “verifiable physics” claim would be evaluation under a changed reference family, grid family, or residual discretization not used in training—at least for one or two PDEs—or an explicit limitation stating that rewards are reference-anchored and transfer is within a related numerical-methods neighborhood.","section":null}],"minor_comments":[{"comment":"Figure 2 and Table 1: clarify whether Claude Sonnet 4.5 and other API baselines used identical system/user prompt constraints (NumPy/SciPy only, fixed interface) as the local models; any prompt mismatch would weaken the “smaller model beats frontier” comparison.","section":null},{"comment":"§4.2: Incompressible Navier–Stokes is the clear capacity bottleneck (only 14B RLVP succeeds strongly). A short discussion of why projection structure is harder to learn from the current bank would help readers set expectations.","section":null},{"comment":"Related Work: CodePDE and SciML-agent systems are cited; a short explicit contrast on compute amortization (post-training once vs. per-query refinement) would sharpen positioning without new experiments.","section":null},{"comment":"Appendix C’s exponential-tolerance derivation is useful; cross-reference it more clearly from §3.5 so readers see that R_traj is an expected-pass probability, not an ad-hoc soft score.","section":null},{"comment":"Typos/clarity: “DeepSeek V4 Flash” labeling in Table 1 should be double-checked against the actual API model name; ensure consistent PDE naming (e.g., reaction_diffusion1d vs. Reaction Diffusion 1D) between tables and figures.","section":null},{"comment":"Figure 11 (solver component frequencies) is informative; reporting sample sizes and a simple statistical comparison (RLVP vs SFT) would make the motif-reuse narrative less anecdotal.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The manuscript is a solid systems/ML contribution rather than a pure theory paper; fit is good for a methods-oriented ML or scientific ML venue. The skeptic concern about reference-matching is real but does not, on my reading, collapse the main empirical claim—gains over SFT and validity-only RL are well controlled. I would not require a full redesign of the verifier; sensitivity checks and tighter transfer language should suffice. Novelty relative to concurrent scientific coding agents should be watched at camera-ready, but the continuous multi-PDE RLVR angle is distinct enough as written."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"Punchline: this is a clean systems result. They take the usual SFT-then-GRPO recipe and replace binary unit-test rewards with a hybrid gate—execute/shape/finite, then continuous function-space nRMSE and a reference-relative residual—so the policy is ranked by the field it returns, not just whether the code runs. One policy across eight PDE families, three model sizes, and the 7B RLVP checkpoint beats Claude Sonnet 4.5 under the same direct-generation protocol on in-distribution solvers.\n\nWhat is actually new is not PDE theory; it is the post-training loop. Prior work mostly does inference-time refinement or binary RLVR. Here the continuous term matters: their validity-only ablation is worse on pass@k and nRMSE on both seen and held-out sets. The solver bank is curated and audited (self-convergence, MMS), the evaluation protocol is fixed and combinatorial pass@k is reported honestly, and the authors do not oversell Navier–Stokes or the hard held-outs.\n\nSoft spots, in proportion. The dense reward is still measured against the environment’s numerical references with fixed temperatures (T_traj=0.05, T_phys=2.0) and a fixed success cut at nRMSE≤1e-2; there is no sensitivity study, and nothing fully rules out matching reference idiosyncrasies rather than scheme-agnostic correctness. Held-out gains line up with operator recombination (heat ⊂ RD, 2D advection as a lift, ARD as advection+reaction+diffusion) and stay weak on KdV and Darcy–reaction—so the compositionality story is selective motif reuse, not broad physics grounding. Code and verifier are promised, not shipped. Scope is structured 1D/2D families, which they already flag.\n\nNone of that sinks the central claim. For people building scientific coding agents or graded RLVR, this is worth reading carefully. I would send it to referees; it is important enough and empirically sharp enough to deserve that time, even if reviewers push on reward sensitivity and artifact release.","headline":"Solid multi-PDE RL post-training paper: continuous physics rewards beat binary validity and can make a small coder model beat frontier prompting on in-distribution solver generation; transfer is real but selective.","tokens_in":36153,"tokens_out":536,"would_cite":true,"duration_ms":14195,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Post-training language models with continuous physics rewards lets one policy write accurate multi-PDE solvers, and a smaller trained model can beat prompting a much larger frontier model.","keywords":["reinforcement learning","PDE solvers","code generation","verifiable rewards","large language models","scientific computing","post-training","numerical methods"],"falsifier":"Run the same GRPO setup with validity-only reward versus full continuous physics reward under matched seeds and rollouts: if validity-only matches or beats full RLVP on median best-of-8 nRMSE and held-out transfer, continuous physics is not the driver; or re-evaluate RLVP solvers on new grids, parameters, and independent reference solvers outside the paper’s suite—if high training scores collapse while validity stays high, the signal is overfit to the hidden references.","tokens_in":36002,"feed_emoji":"🧮","tokens_out":1120,"duration_ms":34952,"temperature":0.7,"pith_summary":"Building a reliable numerical PDE solver still requires expert choices about discretizations, stability limits, and boundary handling—choices that can make two programs that both run differ by orders of magnitude in solution quality. This paper claims that those graded scientific preferences can be written into a language model’s parameters, not only recovered at inference time with prompting and debugging loops. The method, RLVP, first warms up a model on a bank of teacher solvers, then applies reinforcement learning with a hybrid verifier: hard checks require the code to execute and return finite, correctly shaped fields, while continuous rewards score trajectory accuracy against hidden references and residual consistency with the PDE. One policy trained jointly across eight equation families improves pass rates and solution error over base and supervised baselines, shows selective zero-shot gains on held-out PDEs by recombining numerical motifs, and allows a smaller post-trained model to outperform direct prompting of a frontier model on in-distribution solver generation. If the claim holds, scientific code agents can amortize numerical reliability into the model rather than relying only on larger static models or per-problem scaffolding.","feed_headline":"Smaller LLMs beat frontier models at writing PDE solvers","feed_subtitle":"Continuous physics rewards teach one policy to recombine numerical methods across equation families.","key_machinery":"The hybrid RLVP reward R = V · R_traj · R_phys. Validity V is a hard gate (executes, correct shape, finite values). Continuous factors are expected pass probabilities under exponential tolerances on normalized space-time L2 trajectory error and reference-relative residual error. After supervised fine-tuning on a multi-PDE solver bank, GRPO uses group-relative advantages from this reward to shift probability toward programs that are both executable and physically accurate.","core_discovery":"The paper establishes that Reinforcement Learning with Verifiable Physics (RLVP)—post-training a single language-model policy with a hybrid binary–continuous verifier that multiplies program validity by function-space trajectory accuracy and PDE-residual consistency—substantially improves multi-PDE solver code generation. Across model scales, RLVP raises pass@1 and pass@8 and drives median best-of-8 nRMSE to about 6×10^{-4} on eight seen families spanning hyperbolic, parabolic, elliptic, and incompressible-flow systems; a smaller RLVP-trained model outperforms prompting a frontier model under the same direct-generation protocol; and the policy shows selective zero-shot transfer to held-out P","pith_inferences":["The same hybrid continuous-verifier pattern may transfer to other scientific code tasks that admit graded residual or conservation-error signals, such as molecular-dynamics integrators or finite-element assembly.","If motif recombination is the transfer mechanism, expanding the solver bank with more operator types could widen zero-shot coverage without a proportional increase in training PDE families.","Reference-relative rewards may under-reward stylistically different solvers that still satisfy the PDE better than the hidden reference on new instances.","Combining RLVP with retrieval, debugging loops, or verifier-guided search—settings the paper leaves for future work—could compound the amortized gains isolated under direct generation."],"forward_implications":["A single post-trained policy can serve multiple PDE families instead of specializing one model per equation.","Continuous physical accuracy rewards improve solution quality and cross-PDE transfer beyond binary execution feedback alone.","Smaller verifier-trained models can outperform larger static frontier models on in-distribution PDE solver generation under direct generation.","Trained policies recombine numerical motifs (stencils, CFL substepping, RK schemes, boundary handling) into solvers for related unseen PDEs.","Future solver-writing systems should put more weight on parameter updates from verifiable physics feedback, not only on inference-time prompting and debugging."],"fun_headline_variants":["RL with continuous physics rewards lifts multi-PDE solver code gen","Smaller model with RLVP beats frontier prompting on PDE solvers","Hybrid binary-continuous verifier trains one policy across PDE families","Verifiable physics post-training enables zero-shot PDE solver transfer","Continuous residual rewards teach LLMs to recombine numerical motifs"],"cache_read_input_tokens":32896,"weakest_assumption_plain":"The method assumes that continuous scores against hidden numerical reference trajectories and residuals, with fixed exponential temperatures and a fixed success error threshold, measure true physical correctness rather than closeness to particular reference solvers, grids, and operator combinations.","fun_headline_variants_meta":{"raw":{"variants":["RL with continuous physics rewards lifts multi-PDE solver code gen","Smaller model with RLVP beats frontier prompting on PDE solvers","Hybrid binary-continuous verifier trains one policy across PDE families","Verifiable physics post-training enables zero-shot PDE solver transfer","Continuous residual rewards teach LLMs to recombine numerical motifs"]},"model":"grok-4.5","effort":"low","cost_usd":0.00499,"raw_usage":{"total_tokens":1525,"prompt_tokens":944,"num_sources_used":0,"completion_tokens":87,"cost_in_usd_ticks":49900000,"prompt_tokens_details":{"text_tokens":944,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":494,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":944,"tokens_out":87,"duration_ms":6053,"temperature":1.0,"reasoning_tokens":494,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T11:25:37.463523+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Run the same GRPO setup with validity-only reward versus full continuous physics reward under matched seeds and rollouts: if validity-only matches or beats full RLVP on median best-of-8 nRMSE and held-out transfer, continuous physics is not the driver; or re-evaluate RLVP solvers on new grids, parameters, and independent reference solvers outside the paper’s suite—if high training scores collapse while validity stays high, the signal is overfit to the hidden references.","supporting_citations":[],"review_version":1}