{"id":"0c07dd89-9db8-4af0-8bd5-29c203ba03ce","arxiv_id":"2509.10500","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Benchmarking four multi-view symbolic regression packages on five real scientific datasets shows all find accurate compact models; parameter limits and shared constants emerge as key design features.","lead":"This paper benchmarks four symbolic regression packages that each search for one mathematical formula to fit several related datasets at once. It finds all four often produce accurate, simple formulas and proposes design guidelines for future multi-dataset symbolic regression tools.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reduced-MSE metric's inconsistent parameter counting and zero penalty on test loss could invert method rankings; the paper's comparative claims and design guidelines depend on this metric.","rationale":"I have read the paper carefully. The central claim is that MvSR methods often produce accurate and simple models on real-world multi-view datasets. The paper supports this with distributions of reduced MSE and visual fits that indeed show many good models. The most fragile point is the evaluation metric: Eq. 4.1's n_params is not unambiguously defined for reused parameters and shared constants, and test loss is computed with n_params=0. Since the paper uses this metric to rank methods and to justify design guidelines, an inconsistency here could change the conclusions. This is exactly the reader's identified weak assumption. I do not see a more fundamental flaw: the datasets are real, the code is public, the authors acknowledge the artificiality of the Nikuradse partition and the lack of uncertainties. The qualitative claim that MvSR often finds good model forms is supported by the plots independently of the metric. However, the comparative statements are quantitative and metric-dependent, so a re-analysis with a consistent complexity penalty is necessary before the guidelines can be taken as definitive. Hence the CONDITIONAL verdict stands.","tokens_in":15025,"tokens_out":6928,"duration_ms":70636,"concrete_test":"Use the public repository (https://github.com/erusseil/mvsr) to recompute the analysis with two modified metrics: (1) count n_params as the number of distinct free parameters (view-specific plus shared constants, reused symbols counted once) in Eq. 4.1, and (2) replace Eq. 4.1 with AICc (or BIC) using the same parameter count. Regenerate the cumulative distributions in Figure 1 and the per-dataset best-model tables, then check whether the ordinal claims (eggp > PySR, etc.) and the guideline conclusions survive. If the rankings invert or the effect sizes shrink below significance, the central claim is not robust to the metric choice.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4 defines the reduced MSE (Eq. 4.1) as MSE/(n_obs - n_params), but never specifies how n_params is counted when a parameter symbol appears multiple times in an expression or when constants (K) are shared across views. The four implementations differ in these respects: Operon forbids parameter reuse, PySR/eggp allow it, and ϕ-SO separates view-specific parameters from class constants. If n_params counts occurrences rather than distinct symbols, the complexity penalty is not comparable across methods. Further, in evaluating the test loss the paper sets n_params=0 (Section 4), so the reported test scores are not reduced and are on a different scale from training scores. This asymmetry can mis-order models: e.g., a model with many reused parameters or constants may get a favorable training MSE* and then an unpenalized test MSE, while a genuinely simpler model is ranked lower. The paper's comparative statements (eggp produced the largest number of satisfactory models; PySR finds significantly less good models) and the derived guidelines (limit parameters, limit constants) rely on these orderings. The authors admit a rigorous comparison is impossible, yet the quantitative conclusions presented are metric-dependent. The absence of uncertainty weighting is acknowledged by the authors and is a lesser issue because all methods are treated equally, but the inconsistent metric directly biases method comparison.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an empirical comparison of four symbolic regression (SR) implementations that support multi-view symbolic regression (MvSR): Operon, PySR, ϕ-SO, and eggp. The authors apply these methods to five real-world datasets from physics and network science, partitioning each dataset into multiple views and training a parametric model with view-specific parameters. They evaluate models using a reduced mean squared error (MSE*) aggregated with the worst-fitness rule, and report both training and test scores. The central claim is that all MvSR implementations 'often achieve good accuracy while proposing solutions with only few free parameters' and that certain implementation features (parameter limits, shared-constant limits, aggregation control) are desirable for future development. The paper concludes with guidelines for MvSR design and makes all code and results publicly available.","tokens_in":15362,"tokens_out":2860,"duration_ms":33678,"significance":"If the results hold, this is a useful and timely empirical study: MvSR is a recent extension of SR, and a comparison of the four publicly available implementations on real scientific data has not been performed before. The paper benefits from using real, domain-relevant datasets rather than synthetic benchmarks, and from releasing code and results. The authors are also transparent about several limitations, including the absence of data uncertainties, the restriction to one-dimensional problems, and the impossibility of a fully fair cross-implementation comparison. However, the quantitative conclusions—especially the comparative statements about which methods produce more satisfactory models—depend on a metric (MSE*) whose parameter-counting rules are ambiguous and applied inconsistently between training and test phases. The central qualitative message is defensible, but the load-bearing metric needs clarification or revision before the comparative claims can be accepted as stated.","major_comments":[{"comment":"The reduced MSE is defined as MSE/(n_obs − n_params), but the paper never specifies how n_params is counted for expressions where the same parameter symbol appears multiple times, nor for constants shared across views. The four implementations differ in exactly these respects: Operon forbids parameter reuse, PySR and eggp allow it, and ϕ-SO distinguishes view-specific parameters from globally shared constants. If n_params counts occurrences rather than distinct symbols, the complexity penalty is not comparable across methods. Compounding this, the test loss is computed with n_params = 0, so test MSE* is just raw MSE, putting training and test scores on different scales. This asymmetry can misorder models: a complex model with many reused parameters may receive a favorable training penalty and an unpenalized test score, while a genuinely simpler model is ranked lower. This directly affect","section":"Section 4, Eq. (4.1)"},{"comment":"There is an inconsistency in how the 'best' model is selected. Section 5(a) states that for each dataset and implementation the authors study 'the model with the lowest MSE_test that was produced,' yet the caption of Table 2 says the solutions are 'from the lowest MSE_train.' Figure 5's caption also says 'lowest MSE_train solution,' while other figure captions refer to 'lowest reduced MSE solution.' This inconsistency matters because the showcased models and the discussion of their qualities (e.g., PySR's galaxy model, ϕ-SO's Michaelis–Menten model) depend on which criterion was used. If the selection was based on training MSE, the 'test accuracy' claims for those models are not properly supported. Please clarify the selection criterion and ensure it is applied consistently.","section":"Section 5(a) and Table 2"},{"comment":"The global cumulative distributions pool all datasets, hyperparameter configurations, and seeds into a single histogram. Because the datasets have very different numbers of views, points per view, and intrinsic difficulty, the pooled distribution may be dominated by the easiest datasets or by the worst-fitness aggregation behavior. The statement that 'PySR finds significantly less good models' rests on this pooled comparison. A per-dataset breakdown (e.g., separate cumulative distributions or median MSE* per dataset) is needed to support the comparative claims. Without it, the observed differences could be driven by one or two datasets rather than by general method behavior.","section":"Figure 1 and Section 5(a)"}],"minor_comments":[{"comment":"Typo: 'The data offers a with very high sampling' should likely be 'The data offers a very high sampling.'","section":"Section 3, Nikuradse dataset"},{"comment":"Typo: 'This is evidence by' should be 'This is evidenced by.'","section":"Section 6"},{"comment":"In the sentence 'In this case nparams = 0and M SE∗ = M SE,' there is a missing space and the notation could be formatted consistently.","section":"Section 4"},{"comment":"The definition of n as 'the number of parameters in the model (θ)' with bounds l < n < u is slightly confusing because θ is indexed per view (θ_i). Clarify that n is the number of view-specific parameters.","section":"Section 2, Eq. (2.1)"},{"comment":"The caption states 'Medians not shown are exceeding the clipping threshold.' Consider indicating the actual median values or marking the clipping threshold more explicitly, as the reader cannot tell which medians are absent.","section":"Figure 1"},{"comment":"Some expressions are difficult to parse due to formatting (e.g., square-root continuations and stacked fractions). Consider using a cleaner typesetting or adding a brief explanation of the notation used for constants and parameters.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable empirical study, but the metric inconsistency in Eq. (4.1) is a substantive issue that affects the main comparative conclusions. I would encourage the editor to request a revision where the authors either fix the metric definition and re-analyze, or soften the comparative claims to match the ambiguity. The public code and data are a plus, and the paper is within the scope of the journal's interest in symbolic regression for physical sciences."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is the first systematic real-world comparison of four multi-view symbolic regression implementations (Operon, PySR, ϕ-SO, eggp) on five scientific datasets, with code and results public. That is worth having. The central qualitative claim—that MvSR methods often find compact, accurate models without fine-tuning—is supported by the cumulative distributions and by the individual fit plots. The authors also deserve credit for being upfront about several limitations: one-dimensional datasets, no uncertainty handling, the artificial view partition for Nikuradse, and the statement that a rigorous comparison is impossible.\n\nThe soft spots are real but mostly affect the comparative statements, not the main message. The reduced-MSE metric (Eq. 4.1) is inconsistently applied. Parameter counting is never specified for reused parameters or shared constants, and the four implementations differ exactly there. Worse, test loss is computed with n_params = 0, so it is plain MSE, not reduced MSE—making the training and test scores not directly comparable and potentially reordering methods. The paper's claims like \"eggp produced the largest number of satisfactory models\" and \"PySR finds significantly less good models\" depend on that ordering. There is also no significance testing, the \"good model\" threshold is heuristic, and the design guidelines (limit parameters, limit constants, expose aggregation) are reasonable but inferred partly from the same metric. These are acknowledged in part, but not fixed.\n\nThat said, the qualitative result that all methods frequently find good models does not depend on the fragile parts of the metric. And the guidelines themselves are sensible even if the evidence for them is softer than the text suggests. The paper is an honest empirical contribution, not a method paper, and the authors do not oversell a hierarchy among methods in the conclusion.\n\nI would send this to peer review. The benchmark is useful and the public artifacts make it reproducible. The referee should push the authors to clarify parameter counting, either fix the test metric or explicitly justify why n_params = 0 there, and soften the comparative rankings to match the evidence. For anyone working on symbolic regression or applying MvSR, this paper is a practical reference—with the metric caveat in mind.","headline":"Useful first real-world benchmark of four MvSR implementations; the central qualitative claim holds, but the comparative rankings rest on a metric with inconsistent parameter counting and should be treated as suggestive, not definitive.","tokens_in":15809,"tokens_out":1214,"would_cite":true,"duration_ms":16593,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Across five real-world datasets, the four public multi-view symbolic regression implementations frequently produce compact, accurate models even without fine-tuning.","keywords":["multi-view symbolic regression","symbolic regression","interpretable models","phenomenological equations","benchmark","parameter sparsity","physical sciences"],"falsifier":"Recompute the benchmark with a loss that weights each point by its reported uncertainty where available, counts every distinct parameter and shared constant when computing degrees of freedom on test data, and treats Nikuradse's pipe roughness as a continuous covariate rather than as view labels. If the ranking of implementations changes materially—say, eggp no longer yields the most satisfactory models or PySR no longer the fewest—the paper's central comparison is an artifact of its metric.","tokens_in":14958,"feed_emoji":"📈","tokens_out":6120,"duration_ms":70066,"temperature":0.7,"pith_summary":"Multi-view symbolic regression (MvSR) searches for one parametric function that, after view-specific parameter fitting, describes several datasets produced by the same underlying phenomenon. The paper compares the four public implementations that currently support MvSR—Operon, PySR, ϕ-SO, and eggp—on five real-world physical datasets covering supernova light curves, galaxy rotation curves, Michaelis–Menten kinetics, pipe friction, and network degree distributions. Its central finding is that all four methods often produce accurate models with only a few free parameters, even under fixed hyperparameters with no expert fine-tuning. The paper does not claim a definitive winner; instead it identifies design features that matter: hard limits on the number of parameters, regulated use of constants shared across views, and an explicit choice of how per-view losses are aggregated. If correct, these results are a practical green light for using MvSR on real one-dimensional datasets and a concrete checklist for future tool development.","feed_headline":"Multi-dataset symbolic regression finds compact physics models","feed_subtitle":"Across five real datasets, all four compared tools often beat the paper's good-fit threshold with few parameters and no tuning.","key_machinery":"The Multi-view Symbolic Regression objective: minimize over the shared functional form f of an aggregate (max or mean) of per-view losses, where each view i minimizes its own parameters θ_i, i.e. min_f agg_i (min_{θ_i} L(f(x_i; θ_i), y_i)). This formulation is what lets a single equation absorb view-specific variation into a few reused parameters and keeps the model from overfitting any one dataset. The paper's evaluation engine is the reduced mean-squared error, MSE* = Σ(y−ŷ)^2/(n_obs − n_params), aggregated by the worst (maximum) per-view value; it is the metric that grounds the 'good model' threshold and the comparative statements.","core_discovery":"The paper claims that, despite the fundamental difficulty of comparing implementations that use different algorithms, Multi-view Symbolic Regression as it exists today already delivers on its promise: across five real-world benchmarks, the MvSR variants in Operon, PySR, ϕ-SO, and eggp frequently found solutions whose training reduced mean-squared error fell below the paper's 5×10−3 'good model' threshold, and many of those solutions use two to four view-specific parameters and few global constants. The authors also claim that specific features drive success: restricting the maximum number of parameters is essential for simple, relevant models (unrestricted runs fit best on raw accuracy but a","pith_inferences":["Extension: treating Nikuradse's pipe roughness as a continuous covariate rather than as discrete view labels would remove an artificial partition the paper itself notes, and could change the ranking of methods on that dataset.","Extension: extending the same benchmark to multidimensional real-world inputs would test whether MvSR's parameter-sharing benefit grows with view count and input dimension, a natural next step the paper leaves open.","Extension: a corrected complexity metric that counts reused parameters and shared constants in n_params, instead of setting n_params to zero on test data, might reorder methods when judged on out-of-sample parsimony.","Extension: because only the single best returned model is judged per run, exporting and exploring the top-N solutions visited during the search, as the paper recommends, could change which tool looks most useful in practice."],"forward_implications":["Practitioners can run current MvSR tools on one-dimensional multi-dataset problems with stock hyperparameters and reasonably expect compact, accurate models that interpolate and sometimes extrapolate well.","MvSR developers should implement or default to a hard parameter-count limit (or a degrees-of-freedom penalty such as AIC/BIC/MDL), because unconstrained models win on accuracy but lose interpretability and physical plausibility.","Shared constants across views are a valuable feature, but tools need a user-set maximum; without one, constants multiply and make models hard to read, as when PySR's light-curve model needs seven constants where ϕ-SO uses one.","The aggregation function should be exposed as an explicit hyperparameter, with worst-fitness recommended when all views are equally important and mean when outliers or large samples are present.","Uncertainty handling should become a standard MvSR capability; currently only PySR and ϕ-SO accept error bars, so measurement noise is silently ignored in cross-method comparisons."],"supporting_citations":[{"why":"Defines the MvSR formulation and the Operon extension, supplying the baseline multi-view loss and worst-fitness aggregation used in the comparison.","marker":"[13]"},{"why":"Introduces Class Symbolic Regression for ϕ-SO, one of the four implementations benchmarked, and the prior evidence that multi-dataset fitting favors fewer parameters.","marker":"[14]"},{"why":"Provides PySR and its Parametrized Template Expressions, the implementation whose constant overuse and high runtime are analyzed.","marker":"[2]"},{"why":"Introduces eggp with equality-graph search, the implementation reported to produce the largest number of satisfactory models.","marker":"[3]"},{"why":"Describes the Operon genetic-programming framework, whose speed and lack of parameter limits shape the operon and operon_increased results.","marker":"[1]"},{"why":"Supplies the contemporary symbolic-regression benchmark context used to justify Operon's performance profile.","marker":"[17]"},{"why":"Provides the co-added galaxy rotation curves used as a high-number-of-views benchmark.","marker":"[25]"},{"why":"Supplies the historical Michaelis–Menten dataset, the low-data-per-view stress case in the evaluation.","marker":"[26]"},{"why":"Provides the Nikuradse pipe-friction data, the historically difficult benchmark evaluated as six artificial views.","marker":"[27]"},{"why":"Supplies the KONECT network collection from which the network degree distribution views are drawn.","marker":"[33]"}],"fun_headline_variants":["MvSR tools beat accuracy thresholds across five physics benchmarks","Compact equations from multi-view symbolic regression comparisons","Parameter limits key to multi-dataset symbolic regression success","Four MvSR implementations rival manual physics derivation","Multi-view regression: fewer parameters, reliable fits across five datasets"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The comparative conclusions rest on the reduced mean-squared-error metric aggregated by worst per-view fit, which ignores measurement uncertainties, counts reused and shared constants inconsistently, and sets n_params to zero on test data; if that metric mis-orders models, the claims about which implementations generate better models lose support.","fun_headline_variants_meta":{"raw":{"variants":["MvSR tools beat accuracy thresholds across five physics benchmarks","Compact equations from multi-view symbolic regression comparisons","Parameter limits key to multi-dataset symbolic regression success","Four MvSR implementations rival manual physics derivation","Multi-view regression: fewer parameters, reliable fits across five datasets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000785,"raw_usage":{"total_tokens":3290,"prompt_tokens":721,"completion_tokens":2569,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":465,"completion_tokens_details":{"reasoning_tokens":2493}},"tokens_in":465,"tokens_out":2569,"duration_ms":21937,"temperature":1.0,"reasoning_tokens":2493,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T12:31:20.484271+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the benchmark with a loss that weights each point by its reported uncertainty where available, counts every distinct parameter and shared constant when computing degrees of freedom on test data, and treats Nikuradse's pipe roughness as a continuous covariate rather than as view labels. If the ranking of implementations changes materially—say, eggp no longer yields the most satisfactory models or PySR no longer the fewest—the paper's central comparison is an artifact of its metric.","supporting_citations":[{"cited_title":"Multi-View Symbolic Regression","cited_arxiv_id":"2402.04298","evidence_quote":"Defines the MvSR formulation and the Operon extension, supplying the baseline multi-view loss and worst-fitness aggregation used in the comparison."},{"cited_title":"2020 Operon C++: An Efficient Genetic Programming Framework for Symbolic Regression","cited_arxiv_id":null,"evidence_quote":"Describes the Operon genetic-programming framework, whose speed and lack of parameter limits shape the operon and operon_increased results."},{"cited_title":"2020 Navarro-Frenk-White dark matter profile and the dark halos around disk systems","cited_arxiv_id":null,"evidence_quote":"Provides the co-added galaxy rotation curves used as a high-number-of-views benchmark."},{"cited_title":"2011 The original Michaelis constant: translation of the 1913 Michaelis-Menten paper","cited_arxiv_id":null,"evidence_quote":"Supplies the historical Michaelis–Menten dataset, the low-data-per-view stress case in the evaluation."},{"cited_title":"1933 Strömungsgestze in rauhen Rohren","cited_arxiv_id":null,"evidence_quote":"Provides the Nikuradse pipe-friction data, the historically difficult benchmark evaluated as six artificial views."},{"cited_title":"Handbook of Network Analysis [KONECT -- the Koblenz Network Collection]","cited_arxiv_id":"1402.5500","evidence_quote":"Supplies the KONECT network collection from which the network degree distribution views are drawn."}],"review_version":1}