{"id":"f40e465b-3276-45d5-ae49-fcbd6bf2cc8b","arxiv_id":"2607.26490","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An LLM-guided, execution-verified evolutionary search discovered PINN training algorithms that beat the seed network on four PDE benchmarks and matched expert-designed baselines on three.","lead":"This paper introduces EvoPINN, a system in which a large language model proposes changes to physics-informed neural networks, runs the candidates under a fixed compute budget, and keeps only changes that lower solution error. Tests on four equations show discovered algorithms beat the starting network everywhere and match or beat expert-designed baselines on three of four — with caveats about statistical strength and missing code.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Budget matching matches steps and parameter counts, not FLOPs or wall-clock; Appendix D.1 concedes this, so SLRC-PINN's parameter-matched gains may be uncharged compute, and expert baselines may be under-tuned.","rationale":"The reader's weakest assumption and my stress-test converge: the central comparison protocol is fair at the level of step counts and parameter counts but not at the level of FLOPs/wall-clock or exact expert protocols. This matters specifically because the paper's headline uses the word 'rigorous parameter-matched comparisons' to establish SLRC-PINN's novelty; if a simple width-230 MLP were given the same per-step compute, the 55.1% gap might shrink or disappear. The paper deserves credit for the many controls it does include (search/report separation, frozen retraining on fresh seeds, stateless TCROP control, five seeds, disclosed caveats in D.1 and C.4), and I found no internal contradiction. The concern is addressable: profile per-candidate compute and perform a FLOP-matched rerun; optionally re-run original-protocol experts. Since the reader already required exactly this kind of revision under CONDITIONAL, I recommend UNCHANGED rather than a harsher verdict.","tokens_in":20228,"tokens_out":5691,"duration_ms":55751,"concrete_test":"Measure per-training-run FLOPs (or MACs) and wall-clock for SLRC-PINN, the width-230 MLP, and the four frozen Table 1 algorithms under the reported 20,000-step budget on identical hardware. Then retrain the width-230 MLP with per-step FLOPs matched to SLRC-PINN (e.g., scale width/forward cost to equal SLRC-PINN's measured per-step cost) over the same five seeds. If the 55.1% mean-error gap collapses, the parameter-matched advantage is an artifact of uncharged compute. As a secondary check, re-run one expert (e.g., FBPINN) with its original published protocol; if its error drops materially below Table 3, the baseline pool is under-tuned.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The pivotal comparison protocol matches nominal resources (optimization steps, initial collocation counts, optimizer family) but Appendix D.1 concedes that this 'does not imply identical wall-clock time or floating-point operation count.' Discovered programs may therefore spend more FLOPs per step—SLRC-PINN evaluates six Gaussian bases plus a gating path and two decoders, and TCROP adds global resampling every 1,000 steps—so the claimed gains over the seed and over the parameter-matched MLP/APINN/FBPINN/HyResPINNs may partly reflect uncharged compute rather than a novel mechanism. Table 3's 'matched capacity and identical training budgets' matches parameter counts and step counts, not per-step FLOPs or wall-clock; the 55.1% reduction over the width-230 MLP could be the MLP running at a FLOP disadvantage. In addition, Appendix C.4 states the expert baselines are 'architecture-faithful reimplementations... rather than claiming exact reproduction of the numerical protocols in their original studies,' so the expert pool may be under-tuned. Both limitations are disclosed but unquantified; because the central claim asserts 'rigorous parameter-matched comparisons,' this gap is load-bearing.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EvoPINN, an agentic framework that treats PINN design as an execution-grounded search over a two-module algorithm space A=(M,T), where M is the neural representation and T is the training program. An LLM proposes one-module-at-a-time mutations guided by training diagnostics, evolutionary memory, and diversified search focuses; candidates are screened by structural AST comparison, executable smoke tests, and budget-matched PDE evaluation. The frozen winner is retrained from scratch on five fresh seeds and evaluated on a disjoint reporting set. Across four PDE benchmarks the discovered algorithms improve on the seed PINN and are competitive with or better than three of four expert baselines; the framework also discovers SLRC-PINN and TCROP. The paper includes ablations and zero-shot transfer experiments.","tokens_in":20404,"tokens_out":6559,"duration_ms":60662,"significance":"If the results are robust, EvoPINN is a meaningful step toward automated discovery of PINN learning mechanisms rather than hyperparameter configuration. The paper is unusually careful in several respects: frozen algorithms are retrained from scratch, search- and reporting-time point sets are separated, the TCROP control is executed as a stateless program, and the main protocol limitations are explicitly disclosed. The central claim is concrete and falsifiable. The main weakness is that the quantitative fairness controls are not fully quantified: the budget envelope matches steps and parameter counts but not FLOPs or wall-clock time, and the expert baselines are acknowledged to be reimplementations of published methods rather than exact reproductions. These gaps are disclosed, but they are load-bearing for the abstract's claim of 'rigorous parameter-matched comparisons.'","major_comments":[{"comment":"The abstract's 'rigorous parameter-matched comparisons' and Table 3's 'identical training budgets' match parameter counts and optimization-step counts, but Appendix D.1 explicitly states that matching the envelope does not imply identical wall-clock time or floating-point operation count. SLRC-PINN evaluates six Gaussian basis responses, a gating field, and two decoders each forward pass; the width-230 global MLP does not. If SLRC-PINN has materially higher per-step FLOPs, the 55.1% error reduction over the MLP may reflect additional computational budget rather than the discovered topology. Since the claim of a 'genuinely new mechanism' depends on excluding this explanation, please add a FLOP-matched or wall-clock-matched control, or report per-step FLOP counts and show the advantage survives an equal-compute comparison.","section":"Section 5.4 and Appendix D.1 (Eq. 38, Table 3)"},{"comment":"The expert baselines are 'architecture-faithful reimplementations... rather than claiming exact reproduction of the numerical protocols in their original studies.' If the reimplemented APINN, FBPINN, HyResPINNs, and other expert variants are less tuned than their published versions, the 'match-or-beat expert' results in Table 1 and the controlled comparison in Table 3 may be inflated. The paper reports that experts were screened under a unified pipeline but does not report whether the screened variants reach accuracy levels comparable to the original publications. Please provide screening scores, compare against published reference accuracies, or otherwise demonstrate that the expert pool is not systematically weaker than the source literature.","section":"Appendix C.4 and Tables 1/3"}],"minor_comments":[{"comment":"The diagnostic summary includes 'the log-scale gap between the final training loss and the search-time solution error,' which is derived from reference-solution values. The prompt protocol in B.2 says the prompt 'never contains reference-solution values.' This should be qualified to make clear that summary statistics derived from reference values are permitted; as written it is easy to read as contradictory.","section":"Appendix B.1/B.2"},{"comment":"The main text presents SLRC-PINN as the discovered Burgers1D algorithm, but C.4 states that the SLRC-PINN controlled study is distinct from the frozen Burgers1D algorithm used in Table 1. This distinction should appear in the main text to prevent readers from comparing the two sets of numbers as if they were the same experiment.","section":"Section 5.4 vs C.4"},{"comment":"Formatting glitch: the Poisson2D expert-baseline entry '4.7025e-3±8.23e-44.1413e-3' has no separating whitespace between the expert and EvoPINN columns. Please correct the table rendering.","section":"Table 1"},{"comment":"The comparison with LLM Best-of-56 uses 56 independent samples, matching EvoPINN's 56 proposals. Please state explicitly whether the Best-of-56 and PINNsAgent baselines also regenerate from seed with the same five reporting seeds and disjoint reporting points; the text implies this but it should be explicit in the protocol.","section":"Section 5.3/Table 2"}],"recommendation":"major_revision","confidential_remarks":"The paper's main risk is overstatement rather than methodological carelessness. The search protocol is unusually disciplined, and the two main fairness gaps are disclosed. I would support publication if the authors quantify the FLOP/wall-clock issue and demonstrate that the expert baselines are not systematically under-tuned. The recommendation is major_revision rather than reject because both issues are addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, read this one if you care about LLM-guided scientific discovery. The real news: EvoPINN treats PINN design as executable algorithm search over a modular (representation, training program) space, with an LLM mutating one module at a time under diagnostic feedback and evolutionary memory. The search object—the complete PINN algorithm, not just hyperparameters—is genuinely broader than prior PINN auto-design work. The evaluation protocol is better than most papers of this type: search-time selection uses one point set, discovered algorithms are frozen and retrained from scratch on five fresh seeds for reporting, and the TCROP case study includes a stateless control that reproduces the reported accuracy. The paper also flags its own limitations rather than hiding them. Citation pattern looks honest; the related work is current and the framing against FunSearch, Eureka, and ReEvo is fair.\n\nWhere it gets soft: the abstract says 'significantly reduce,' but Table 1 has overlapping error bars at n=5 on three of four benchmarks and no significance test is given. More importantly, 'budget-matched' and 'parameter-matched' are not the same as compute-matched. Appendix D.1 concedes the shared envelope 'does not imply identical wall-clock time or floating-point operation count.' SLRC-PINN evaluates six localized bases, a gating path, and two decoders per step, and TCROP adds global resampling; so some of the gains may be uncharged FLOPs, not mechanism. Appendix C.4 also concedes the expert baselines are 'architecture-faithful reimplementations' rather than exact reproductions of the original numerical protocols, so the expert pool may be under-tuned. Neither concession is quantified. Add to that: no code, prompts, discovered programs, or logs are released, and the generator is proprietary, so the specific discoveries cannot be reproduced from the text.\n\nNone of this is fatal. The SLRC-PINN topology is checkable from the equations, the search/report separation is a real strength, and the framework's central viability claim is plausible. But the strongest sentence in the paper should be scaled back until compute is measured and artifacts ship. For people working on LLM-driven scientific computing and PINN auto-design, this is a useful and thought-provoking framework paper. I would send it to serious peer review, but require significance testing, per-candidate FLOP/wall-clock accounting, and a systematic prior-art check for the 'novel architecture' claim before accepting the headline.","headline":"EvoPINN is a serious, honestly reported agentic search framework, but the headline 'rigorous parameter-matched' claim outruns the protocol: budget matches steps and parameter counts, not FLOPs or wall-clock, and the artifacts needed to reproduce the discoveries are not released.","tokens_in":21052,"tokens_out":2855,"would_cite":true,"duration_ms":50011,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Execution-grounded search can discover entirely new PINN learning algorithms, and one discovered architecture, SLRC-PINN, retains its advantage under parameter-matched retraining.","keywords":["physics-informed neural networks","automated algorithm discovery","large language models","evolutionary search","SLRC-PINN","PDE solving","execution-grounded search"],"falsifier":"Measure per-step FLOPs and wall-clock time of SLRC-PINN versus the parameter-matched MLP on Burgers1D under the reported 20,000-step budget, and retrain both with a hard equal-FLOPs envelope; alternatively rerun the expert baselines using their original published numerical protocols. If the relative-L2 gap vanishes or reverses, the claimed advantage would be refuted.","tokens_in":19941,"feed_emoji":"🤖","tokens_out":7291,"duration_ms":61411,"temperature":0.7,"pith_summary":"This paper claims that open-ended algorithm discovery for scientific computing is possible in practice when a large language model is tied to an execution loop: the agent proposes changes to a physics-informed neural network, runs them, reads diagnostics, and keeps only changes that survive structural checks and improve solution error under a shared budget. The central proposal is to split a PINN into a representation module and a training-program module, mutate one at a time, and let a diagnostic-guided memory steer the search, converting PINN design from handcrafting into a verifiable search problem. Across Poisson, Burgers, wave, and heat benchmarks, the frozen discovered algorithms reduce relative L2 error relative to the seed PINN and match or beat a pool of expert-designed variants on three of four tasks. The paper's flagship discovery is SLRC-PINN, a representation u(q)=u_G(q)+alpha(q)u_L(q) with a zero-initialized corrective branch and a learned bank of localized bases, and the paper argues its advantage is due to this topology, not capacity. If true, this means an automated agent can invent new, transferable computational mechanisms rather than merely select among known ones.","feed_headline":"An AI agent invents better neural-net solvers than handcrafted ones","feed_subtitle":"Run diagnostics guide an evolutionary search to a global-local PINN whose gains survive matched retraining.","key_machinery":"The load-bearing mechanism is the global-local decomposition of SLRC-PINN: u(q)=u_G(q)+alpha(q)u_L(q), with u_G a persistent global decoder, u_L a corrective decoder whose final layer is initialized to zero, and alpha(q) a smooth gating field computed from a learned bank of localized basis functions with learned centers and input-dependent widths. Zero initialization guarantees the network starts as the global branch, so local enrichment is introduced as a continuous, function-preserving extension rather than as a competing subnetwork. Around it, the EvoPINN loop does the discovery work: one-module-at-a-time mutation of M versus T, UCB-style scheduling between the two modules, LLM proposals","core_discovery":"On its own terms, the paper establishes that EvoPINN, an evolutionary search where an LLM proposes programmatic changes and an execution pipeline verifies them, can discover complete and reusable PINN learning algorithms. The algorithm is represented as a pair A=(M,T), with M the neural representation and T the training program; each generation mutates only one module, conditions the LLM on a compact diagnostic summary plus evolutionary memory, and accepts a candidate only if it passes structural abstract-syntax-tree verification and beats the parent's relative L2 error under a shared training budget. Frozen, freshly retrained versions of the discovered algorithms report lower mean relative","pith_inferences":["A strict test of the 'budget-matched' claim would cap total floating-point operations per candidate rather than optimization steps; if the gap shrinks, part of the reported gain may be uncharged computation—an extension the paper itself flags by noting the shared envelope does not imply equal wall-clock or FLOP counts.","The zero-init global-plus-local template could be extracted as a hand-crafted prior and tested on other shock-forming PDEs or multiscale learning tasks, which would tell whether the discovered topology is a general principle or a special fix for Burgers.","Varying the search-time reference set across independent runs and checking whether the same mechanisms emerge would sharpen the evidence that the search is discovering robust structures rather than overfitting to one evaluation set."],"forward_implications":["Frozen discovered algorithms, retrained from scratch, improve over the seed PINN in all four benchmark regimes and beat the selected expert baseline on three of them.","SLRC-PINN outperforms a parameter-matched MLP and domain-decomposition variants under the same training budget, so the architecture's effect is separable from parameter count.","The zero-initialized local-correction principle means the model can begin from the smooth global solution and grow local capacity only where the physics residual demands it.","The discovered training policy for the Poisson problem, which spreads optimizer effort and refresh points based on run telemetry, outperforms the standard equal-split handoff, showing the search can invent training programs as well as representations.","Ablation results indicate that run-informed diagnostics and modular mutation are both needed: removing them degrades accuracy or stability, especially on the wave benchmark."],"fun_headline_variants":["Agentic search discovers PINN algorithm beating hand-tuned","LLM-driven evolution finds new solver architecture","EvoPINN: AI invents reusable PDE solver algorithms","Autonomous agent invents novel PINN architecture"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The quantitative claims rest on the fairness of the comparison protocol, which matches optimization steps and initial collocation counts but not wall-clock time or floating-point work per step; if discovered algorithms spend more compute per step, or if the expert baselines are weaker than their original published versions, the performance gap could shrink or disappear.","fun_headline_variants_meta":{"raw":{"variants":["Agentic search discovers PINN algorithm beating hand-tuned","LLM-driven evolution finds new solver architecture","EvoPINN: AI invents reusable PDE solver algorithms","Autonomous agent invents novel PINN architecture"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000195,"raw_usage":{"total_tokens":1203,"prompt_tokens":760,"completion_tokens":443,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":380}},"tokens_in":504,"tokens_out":443,"duration_ms":4655,"temperature":1.0,"reasoning_tokens":380,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T14:38:39.667551+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure per-step FLOPs and wall-clock time of SLRC-PINN versus the parameter-matched MLP on Burgers1D under the reported 20,000-step budget, and retrain both with a hard equal-FLOPs envelope; alternatively rerun the expert baselines using their original published numerical protocols. If the relative-L2 gap vanishes or reverses, the claimed advantage would be refuted.","supporting_citations":[],"review_version":1}