{"id":"7ebd20b2-6b20-4dd2-8b44-5a6b1bc50cc0","arxiv_id":"2501.10339","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"PASTIS is a new information criterion that uses extreme value statistics to add a library-size penalty, allowing sparse stochastic differential equations to be selected with a controlled false-positive rate.","lead":"The authors derive a new model-selection score, PASTIS, that adds a penalty proportional to the log of the candidate-library size, correcting the combinatorial overfitting that standard AIC misses when learning stochastic differential equations from data. A generalist might care because it makes automated discovery of minimal stochastic equations more reliable, including ecological networks, reaction-diffusion systems, and other noisy experimental trajectories.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The p-calibration of PASTIS rests on independence and chi-square statistics for superfluous-term information gains that are violated by correlated polynomial libraries and multiplicative noise; the paper quantifies this only indirectly in Supplement S1f.","rationale":"The reader's weakest-assumption analysis matches mine. I checked the derivation of Eq. (9): the Gumbel law (Eq. 8) and the exact CDF (Eq. 11) both require independence of Delta I_s, and End Matter 2 states the sufficient condition as orthogonality of superfluous basis functions along the trajectory. The monomial libraries used in every benchmark violate that condition. Supplement S1f is the only direct evidence about the multiplicative/correlated regime and it shows the theoretical prediction is less accurate; the sign and magnitude of the miscalibration are not quantified. This matters because the paper's novelty is the principled significance-level interpretation of p, not just a new penalty. The method may be conservative in practice, and the benchmarks are extensive, but the calibration claim is not yet demonstrated in the target regime. A direct false-positive-rate measurement would settle it. I do not see an internal inconsistency under the stated assumptions, and the public code and extensive benchmarks are independent support, so conditional acceptance is the right level; my stress-test does not move the reader's verdict, hence UNCHANGED.","tokens_in":17379,"tokens_out":13755,"duration_ms":147193,"concrete_test":"Run a calibration experiment on the Lotka-Volterra model of Fig. 3c / Supplement S1f: for p in {0.001, 0.01, 0.1}, generate at least 1000 independent long trajectories (tau = 10^4) with the true model B* known, and measure the empirical rate at which max_{s in B0\\B*} [I(B*+{s}) - I(B*)] exceeds log(n0/p). Compare this rate to p and to the theoretical g(p). Repeat with an orthogonalized library, e.g., Hermite polynomials under the stationary measure, to separate basis-correlation effects from multiplicative-noise effects. If the empirical rate for the monomial library differs from p by more than the 95% binomial confidence interval, the p-calibration claim fails for the correlated-library regime the paper targets.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central guarantee that the user-set p in Eq. (9) is approximately the false-positive probability rests on End Matter 2, where Delta I* = max_{s in B0\\B*} [I(B*+{s}) - I(B*)] is treated as the maximum of n0-n* independent 1/2 chi-square_1 variables, yielding Eq. (8). The stated sufficient condition is <s(x_t)s'(x_t)> = 0 for superfluous s != s', which is essentially never satisfied by the monomial libraries used in the benchmarks: along a bounded trajectory, x, x^2, x^3 and cross terms are strongly correlated. If the Delta I_s are positively correlated, the effective number of independent tests is smaller than n0, making the penalty log(n0/p) conservative and the realized false-positive rate lower than p; but the paper does not quantify this. Supplement Fig. S1f shows that for the multiplicative-noise Lotka-Volterra benchmark even the exact-CDF prediction g(p) is inaccurate, and since Eq. (3) is only a quasi-likelihood for multiplicative noise, the marginal 1/2 chi-square_1 behavior from Wilks' theorem (Eq. 7) can also fail, so the discrepancy could be in either direction. Without a direct false-positive-rate measurement in this regime, the advertised significance-level interpretation of p is not established; this is load-bearing because it is the mechanism by which PASTIS claims to control overfitting.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces Parsimonious Stochastic Inference (PASTIS), a model-selection criterion for stochastic differential equations (SDEs) and stochastic partial differential equations (SPDEs). The criterion is IPASTIS(B) = I(B) − nB log(n0/p), where I(B) is a quasi-likelihood-based information measure, n0 is the size of the candidate library, and p is a user-set significance level. The penalty is derived from extreme-value statistics: if the information gains ΔIs for adding each superfluous basis function are independent and asymptotically 1/2 χ²₁, the maximum gain is Gumbel-distributed, leading to the log(n0/p) term. The authors show that AIC and cross-validation overfit in large libraries, and they benchmark PASTIS against AIC, BIC, SINDy, CV, and LASSO on stochastic Lorenz, Ornstein–Uhlenbeck, Lotka–Volterra, and Gray–Scott models, reporting higher exact-match accuracy and competitive prediction error. Extensions to large sampling intervals, measurement noise, and SPDEs are discussed, and code is provided.","tokens_in":17612,"tokens_out":7589,"duration_ms":75329,"significance":"If the claimed calibration holds, PASTIS is a substantial methodological contribution: it gives a principled, interpretable way to control overfitting risk in sparse SDE discovery, with a single user parameter p and an explicit combinatorial penalty. The derivation chain from Eq. (6) through Wilks' theorem to the Gumbel penalty is transparent and internally consistent under the stated assumptions, and the paper provides an exact finite-n0 correction g(p) in the Supplement. The benchmarks are synthetic but cover a useful range of noise regimes, and the comparison methods are tuned with care. The paper also ships reproducible code and presents falsifiable predictions (e.g., the functional form of the penalty and g(p)). However, the practical validity of the central p-as-significance-level claim is not fully established for the benchmark settings, because the derivation assumes independence and exact Wilks behavior that are not guaranteed for correlated polynomial bases or multiplicative noise. The paper's own Supplement Fig. S1f shows a deviation for the multiplicative-noise Lotka–Volterra case.","major_comments":[{"comment":"The derivation of the Gumbel penalty treats the information gains ΔIs as independent and asymptotically 1/2 χ²₁, with the stated sufficient condition ⟨s(xt)s′(xt)⟩ = 0 for superfluous functions s ≠ s′. For the polynomial bases used in Fig. 3 (monomials and cross terms), this condition is typically violated along a bounded trajectory: x, x², x³ and cross terms are strongly correlated. The paper does not quantify how such correlations affect the realized false-positive rate, even though p is defined as that rate. I ask the authors to report direct false-positive-rate measurements (using the known true models in the OU and Lorenz benchmarks at large τ) as a function of p, for both uncorrelated and correlated bases, and either confirm that p remains a good approximation or replace the claim with an explicit, evidence-based upper bound.","section":"End Matter 2, Eqs. (8)–(9)"},{"comment":"Wilks' theorem is invoked to justify the 1/2 χ²₁ distribution of ΔIs, but Eq. (3) is a quasi-likelihood for multiplicative noise, not the true likelihood (as the main text acknowledges). The Lotka–Volterra benchmark is multiplicative, and Supplement Fig. S1f shows that the theoretical g(p) is inaccurate in this case. Since this system is one of the four central benchmarks and the central claim is that p is the false-positive probability, the paper should either (i) restrict the significance-level interpretation to additive-noise cases, or (ii) provide a direct empirical calibration of p for multiplicative noise and state the regime of validity. Without this, the claim that PASTIS controls overfitting at a chosen significance level is not established for the multiplicative-noise setting.","section":"Eq. (7) and Supplement Fig. S1f"},{"comment":"Part of the empirical support for PASTIS is that the asymptotic exact-match accuracy converges to a value > 1 − p (e.g., Fig. 3). This comparison is partly definitional, because End Matter 3 defines p as the probability of selecting a one-superfluous-term model. The genuinely testable predictions are the functional form of the penalty and the finite-n0 correction g(p); for the additive-noise benchmarks g(p) matches, which is encouraging. However, the paper should separate the definitional statement (error ≈ p) from the testable prediction (the shape and n0-dependence of g(p)) and present direct false-positive rates versus p for correlated bases. The current Fig. S1 panels do not isolate the effect of basis correlations on the calibration.","section":"End Matter 3 and Fig. 5 / Fig. S1"}],"minor_comments":[{"comment":"There is a typo: 'F ailure of AIC' should read 'Failure of AIC'.","section":"Main text, 'Failure of AIC'"},{"comment":"The displayed equation appears garbled in the typeset version (the term '1q 4⟨ ˆD⟩' in the denominator is not readable); please fix the formatting.","section":"End Matter 4, Eq. (12)"},{"comment":"The sentence 'Its minimization leads to the result presented in Fig. 4b' should read 'maximization', since IPASTIS is a criterion to be maximized.","section":"End Matter 4, text after Eq. (14)"},{"comment":"The text says AIC, BIC, and CV use the same greedy forward-selection algorithm as employed for PASTIS, while the main text describes PASTIS as hill-climbing with both additions and removals; please clarify whether removals are included for the baselines or whether only forward selection is used.","section":"Supplement Appendix B"},{"comment":"The transferability of the Wilks theorem and extreme-value argument to discretized fields is asserted rather than demonstrated; given the spatial correlations in the basis (e.g., differential operators), a brief argument or a numerical check would strengthen the SPDE extension.","section":"End Matter 5 (SPDE)"},{"comment":"The paper mentions extended BIC (EBIC) as a consistency perspective but does not include it in the benchmarks; a numerical comparison with EBIC would help position PASTIS relative to existing multiplicity-adjusted criteria.","section":"Discussion"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely to interest the soft-matter and data-driven-discovery communities, and the code availability is a strength. The main concern is the gap between the p-calibration derivation and the benchmark conditions, particularly correlated libraries and multiplicative noise; this is fixable by adding direct false-positive-rate measurements and adjusting the claims accordingly. I recommend major revision rather than rejection because the core penalty form is well-motivated and the additive-noise benchmarks are encouraging."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this paper earns its place. It introduces a simple, defensible information criterion for sparse SDE model selection that accounts for library size via the penalty nB log(n0/p), and it backs the criterion with a clear derivation from Wilks' theorem and extreme value statistics. The p-as-false-positive-rate interpretation is genuinely new in this context, and it gives practitioners exactly what they want: a knob that means something. For additive-noise systems with decorrelated superfluous terms, the benchmarks support it, and the code and data are public.\n\nWhat is new and good: Eq. 9 with the library-size log penalty, the Gumbel derivation in End Matter 2, and the exact finite-library CDF Eq. 11. The authors also position the work honestly relative to EBIC and Bayesian multiplicity priors, and the misspecified-model appendix shows the method is not just an exact-recovery trick.\n\nThe soft spot is real but not fatal. The independence assumption behind Eq. 8 and Eq. 11 fails for polynomial libraries on a bounded trajectory, and for multiplicative noise the Wilks 1/2-chi-square premise can fail too. The stress test is right that this is the load-bearing part of the claim that p equals the false-positive probability. However, the paper does not hide it: Supplement Fig. S1f shows the exact-CDF prediction g(p) deviating for the Lotka-Volterra system, and the text concedes the prediction is less accurate there. What is missing is a direct measurement of the realized false-positive rate under correlated libraries and multiplicative noise, telling the reader how far p actually drifts. That should be added. Without it, the p interpretation is approximate in the regime where users most need the method.\n\nMinor points: the main-text figures have no error bars (averages of 48-100 runs are plotted as single curves), and the greedy search, while efficient and reasonable, is heuristic. Neither changes the conclusions.\n\nWho this is for: anyone doing data-driven inference of stochastic dynamics from a large candidate library. It deserves a serious referee. My recommendation: send it to review, and have the authors add direct false-positive calibration under correlated bases and multiplicative noise, with error bars on the benchmark curves.","headline":"A genuinely useful p-controlled information criterion for stochastic model selection, with a real but honestly flagged caveat about correlated libraries and multiplicative noise.","tokens_in":18195,"tokens_out":2513,"would_cite":true,"duration_ms":25515,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["60G70","60H10","62M05","62F03"],"pacs":[],"model":"deepseek-v4-flash","headline":"PASTIS finds true stochastic models that AIC and BIC miss.","keywords":["model selection","stochastic differential equations","sparse identification","extreme value theory","information criteria","multiple hypothesis testing","likelihood inference","stochastic dynamics"],"falsifier":"Run PASTIS with a fixed $p$ on a simulated system whose true drift is known but whose candidate library is deliberately redundant (e.g., monomials up to degree 20 on a trajectory that stays in a narrow range, so many functions are nearly collinear), and count how often the selected model contains at least one term outside the true set in the long-trajectory limit; if the measured false-positive rate is substantially larger than $p$ or than the exact finite-library prediction $g(p)$, the independence assumption underlying the Gumbel penalty is the culprit.","tokens_in":17132,"feed_emoji":"🎲","tokens_out":10468,"duration_ms":98831,"temperature":0.7,"pith_summary":"Learning stochastic differential equations from trajectory data usually means picking a small set of drift functions from a large library. The paper argues that standard selection tools miss a key source of overfitting: the number of candidate models grows combinatorially with library size, so the best-fitting spurious model will eventually beat the true one by chance alone. Its central proposal, PASTIS, adds a penalty proportional to $n_B \\log(n_0/p)$ to the log-likelihood, where $n_0$ is the library size and $p$ is a user-set false-positive rate. The paper shows on synthetic benchmarks that PASTIS recovers the exact generating model more often than AIC, BIC, cross-validation, SINDy, and LASSO, and that the same criterion works for stochastic partial differential equations and under sampling and measurement imperfections.","feed_headline":"PASTIS finds true stochastic models that AIC and BIC miss","feed_subtitle":"The criterion penalizes each candidate term by a log-size factor, keeping spurious terms out of large-library searches.","key_machinery":"The load-bearing object is the PASTIS information criterion $I_{\\mathrm{PASTIS}}(B) = I(B) - n_B \\log(n_0/p)$, built on the quasi-likelihood information $I(B)$ of the paper's estimation approach. It does its work through two statistical facts: Wilks' theorem, which says the information gain from a single superfluous term is asymptotically $\\frac12 \\chi^2_1$, and extreme value theory, which says the maximum of many such independent gains concentrates near $\\log(n_0)$ and fluctuates with a Gumbel law. The $\\log(n_0/p)$ penalty is exactly the threshold that keeps the probability that any spurious term beats the true model at approximately $p$, turning a combinatorial search over $2^{n_0}$ candidate models into a tractable, significance-controlled selection rule.","core_discovery":"The paper's central claim is that the right penalty for sparse model selection in stochastic dynamics is not one unit per parameter, as in Akaike's criterion, but $\\log(n_0/p)$ per parameter, where $n_0$ is the size of the candidate library and $p$ is the target probability of accepting a superfluous term. The criterion, called PASTIS, is $I_{\\mathrm{PASTIS}}(B) = I(B) - n_B \\log(n_0/p)$, with $I(B)$ the quasi-likelihood information gain of basis $B$ relative to a zero-force null model. Because the largest information gain among $n_0-n_*$ superfluous terms grows like $\\log(n_0)$ (a Gumbel extreme-value effect), a fixed per-parameter penalty like AIC's lets one spurious term slip in with probability approaching one; the $\\log(n_0/p)$ penalty instead caps that probability near $p$. The paper demonstrates on four stochastic benchmark models that PASTIS selects the exact generating model with probability above $1-p$ given enough data, and that it outperforms AIC, BIC, cross-validation, SINDy, and LASSO in exact-recovery speed.","pith_inferences":["Beyond the paper: the $\\log(n_0/p)$ penalty is the same multiplicity correction that physicists call the look-elsewhere effect, so PASTIS could be read as a general template for any likelihood-based search over a large structured hypothesis space, not only SDE drift selection.","Beyond the paper: the independence assumption behind the Gumbel bound will be violated when candidate basis functions are strongly correlated along the trajectory, as typical polynomial libraries are; the supplementary material shows one such discrepancy for multiplicative-noise systems, and a natural extension is to replace the single penalty $\\log(n_0/p)$ by a data-dependent effective-number-of-","Beyond the paper: a testable extension suggested by the paper's own framing is to make $p$ adaptive, small early in the search and larger later, which could speed up convergence to the true model at short times without sacrificing the long-time false-positive guarantee."],"forward_implications":["With enough data, PASTIS recovers the exact sparse generating model in more than $1-p$ of simulation trials, while AIC and cross-validation keep adding superfluous terms even in the long-time limit.","BIC eventually selects the true model asymptotically but converges far more slowly than PASTIS, so PASTIS reaches exact recovery with much shorter trajectories.","The criterion extends unchanged in form to stochastic partial differential equations when the library includes finite-difference operators, allowing the same procedure to identify reaction-diffusion and other field equations.","With modest modifications (a trapezoid likelihood, a three-point diffusion estimator, and a Stratonovich form), PASTIS maintains high exact-match accuracy at large sampling intervals and under measurement noise.","In misspecified settings where the true force is not in the library, PASTIS selects smaller, more parsimonious approximations than AIC, BIC, CV, or SINDy while retaining competitive prediction error."],"supporting_citations":[{"why":"Provides the quasi-likelihood information measure $I(B)$ that PASTIS penalizes, and the estimation framework used throughout.","marker":"[13]"},{"why":"Defines the AIC baseline whose per-parameter penalty of one is what PASTIS replaces.","marker":"[18]"},{"why":"Supplies the asymptotic $\\chi^2_1$ distribution of the information gain of a single superfluous term.","marker":"[20]"},{"why":"Supplies the extreme-value law for maxima of independent squared Gaussians that yields the $\\log(n_0)$ shift.","marker":"[41]"},{"why":"Defines the SINDy sparse-identification baseline; the benchmarks show it degrades under dynamical noise.","marker":"[6]"},{"why":"Defines the BIC baseline with penalty $(n_B/2)\\log \\tau$, which PASTIS overtakes in recovery speed.","marker":"[22]"},{"why":"Defines EBIC, the extended Bayesian criterion whose combinatorial penalty is conceptually related to PASTIS.","marker":"[34]"}],"fun_headline_variants":["PASTIS fixes AIC's blind spot in stochastic model selection","Log-size penalty beats AIC for stochastic dynamics","PASTIS picks true models where AIC fails","Extreme value theory sharpens stochastic model choice","AIC adds spurious terms; PASTIS stops them"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The derivation assumes that the information gain of each irrelevant candidate term is statistically independent of the others and follows the same standard chi-squared shape, so the best-scoring irrelevant term obeys the extreme-value formula; when the candidate functions are strongly correlated with each other along the trajectory, that independence fails and the user-chosen false-positive probability $p$ may no longer equal the actual rate of accepting a spurious term.","fun_headline_variants_meta":{"raw":{"variants":["PASTIS fixes AIC's blind spot in stochastic model selection","Log-size penalty beats AIC for stochastic dynamics","PASTIS picks true models where AIC fails","Extreme value theory sharpens stochastic model choice","AIC adds spurious terms; PASTIS stops them"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00065,"raw_usage":{"total_tokens":2961,"prompt_tokens":906,"completion_tokens":2055,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":522,"completion_tokens_details":{"reasoning_tokens":1976}},"tokens_in":522,"tokens_out":2055,"duration_ms":14139,"temperature":1.0,"reasoning_tokens":1976,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:11:55.192891+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run PASTIS with a fixed $p$ on a simulated system whose true drift is known but whose candidate library is deliberately redundant (e.g., monomials up to degree 20 on a trajectory that stays in a narrow range, so many functions are nearly collinear), and count how often the selected model contains at least one term outside the true set in the long-trajectory limit; if the measured false-positive rate is substantially larger than $p$ or than the exact finite-library prediction $g(p)$, the independence assumption underlying the Gumbel penalty is the culprit.","supporting_citations":[{"cited_title":"Nabeel, A","cited_arxiv_id":null,"evidence_quote":"Provides the quasi-likelihood information measure $I(B)$ that PASTIS penalizes, and the estimation framework used throughout."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the AIC baseline whose per-parameter penalty of one is what PASTIS replaces."},{"cited_title":"Lebarbier and T","cited_arxiv_id":null,"evidence_quote":"Supplies the asymptotic $\\chi^2_1$ distribution of the information gain of a single superfluous term."},{"cited_title":"Rish and G","cited_arxiv_id":null,"evidence_quote":"Supplies the extreme-value law for maxima of independent squared Gaussians that yields the $\\log(n_0)$ shift."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the BIC baseline with penalty $(n_B/2)\\log \\tau$, which PASTIS overtakes in recovery speed."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines EBIC, the extended Bayesian criterion whose combinatorial penalty is conceptually related to PASTIS."}],"review_version":1}