{"id":"3d8dfabe-7c5b-4b53-b051-98b948beb3fa","arxiv_id":"2412.01181","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Explicit integrating factor Euler trains stiff neural ODEs stably at low cost, but its first-order accuracy and fixed-Jacobian approximation limit precision.","lead":"This paper evaluates a classical exponential integrator, integrating factor Euler, as a low-cost way to train neural networks that model stiff differential equations. It reports accurate recovery of stiff linear systems and stable training of a stiff Van der Pol oscillator where implicit solvers failed, while acknowledging first-order accuracy as a limit.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Frozen-Jacobian linearization (Eq. 12) is the load-bearing assumption for the nonlinear stiff examples; without an error estimate or a Jacobian-variation test, the Van der Pol success may be benchmark-specific rather than a property of IF Euler.","rationale":"The reader's weakest-assumption analysis is correct: Eq. (12) is the only place where nonlinearity is imported into the exponential stabilization, and it is not justified by the linear examples. I focus on this rather than the unsupported implicit-failure claim because even a perfect baseline comparison would not establish the method's robustness for nonlinear stiff systems; the Van der Pol demonstration is the sole supporting nonlinear success. The concrete ablation is decisive: if the midpoint or substep variants produce the same recovered dynamics, then the frozen Jacobian is not the critical ingredient and the paper's method can be taken at face value. If they do not, the reported success is an artifact of the anchoring convention, and the paper would need a local error bound or an adaptive re-linearization strategy before the central claim can be accepted. The linear Examples 1 and 2 are strong and mathematically sound, and the paper honestly acknowledges first-order accuracy limitations, so the concern does not overturn the conditional verdict; it sharpens the required revision.","tokens_in":31017,"tokens_out":7683,"duration_ms":74690,"concrete_test":"A controlled ablation on Example 4 (Eq. 21): keep the pi-net, optimizer, and data identical, but evaluate L at the midpoint of each integration interval instead of at y0 (Eq. 12), and then also re-evaluate L at every substep when subdividing each interval into, say, 5 substeps. Compare the recovered parameter relative errors in Figure 11. If either variant changes the results by more than the spread between adjacent n-values, the frozen-Jacobian anchoring is arbitrary and the method needs an error estimate; if both variants are stable, the concern is weakened. Alongside this, record sup_{t in [t_i,t_{i+1}]} ||J(t,y(t))-L|| along the true trajectory to show whether the frozen linearization is representative.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The nonlinear extension of the method rests on Eq. (12): the Jacobian L is evaluated once at the initial condition y0 of each integration interval and kept fixed for the whole step h. For linear systems the matrix exponential solves the interval exactly, which explains Examples 1 and 2. For nonlinear systems, the paper gives no bound on how much J(t,y(t)) may vary over the interval, no criterion for when a frozen linearization is representative, and no stability analysis for the explicit treatment of N(t,y) in Eq. (17). Example 3 already shows IF Euler as the least accurate method among those compared, so the nonlinear claim rests almost entirely on Example 4. That example is reported without any data for the claimed implicit-method failures, and the recovered equations for n=100 are far from the true coefficients (e.g., the y-damping coefficient 604.86 vs. 1000), so 'succeeded' is not quantified. The choice of y0 as the anchoring point is arbitrary; a midpoint or endpoint linearization would be an equally plausible choice and may change training outcomes. Because the central claim is about training stiff neural ODEs generally, this untested fixed-linearization assumption is load-bearing.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes applying the explicit exponential integrating factor Euler (IF Euler) method to train neural ODEs on stiff systems. The authors derive the scheme yn+1 = e^{Lh}(yn + hNn) with L frozen at the initial Jacobian of each integration interval, test it on a one-dimensional linear stiff equation, a ten-dimensional linear stiff system, a three-dimensional nonlinear stiff system, and the stiff Van der Pol oscillator (μ=1000), and compare against backward Euler, trapezoid, Radau3, and Radau5. The main claims are that IF Euler recovers stiff linear systems to numerical precision, trains the Van der Pol model where the implicit methods are asserted to fail, and is more efficient than implicit methods because it avoids nonlinear solves. The paper also states that higher-order explicit exponential methods were not found to be viable, referring to a companion paper.","tokens_in":31200,"tokens_out":6120,"duration_ms":51310,"significance":"If the nonlinear results were fully supported, the paper would provide a simple, low-cost alternative to implicit solvers for stiff neural ODEs, potentially widening the use of neural ODEs in scientific applications. The derivation of IF Euler is clear and the linear-system experiments are detailed and internally consistent; however, the exact recovery of linear systems is a by-construction property of integrating the linear part exactly, so it does not by itself validate the method for general stiff problems. The paper's broader significance hinges on Examples 3 and 4 and on the comparative claims against implicit and higher-order exponential methods, all of which are currently under-documented. The manuscript is clearly written but does not yet provide enough evidence to establish the central claims.","major_comments":[{"comment":"The statement that 'every implicit integration technique we tried failed to properly train the stiff neural ODE' is made without any supporting data. Table 11 and Figure 11 report results only for IF Euler; no loss curves, parameter errors, or descriptions of the convergence failures are provided for backward Euler, trapezoid, Radau3, or Radau5. Since the paper's central contrast between IF Euler and implicit methods rests on this claim, the authors should supply quantitative evidence of the implicit-method failures, such as best-attained parameter errors and a statement of the failure mode (e.g., NaN loss, non-convergence of Newton iterations).","section":"Section III D, Example 4"},{"comment":"The method freezes the Jacobian L = df/dy(y0) at the initial condition of each integration interval. For nonlinear systems (Examples 3 and 4), no error bound or quantitative criterion is given for when this fixed linearization remains representative over the integration step. The matrix exponential may stabilize a linearization that is far from the local dynamics, and the explicit treatment of N(t,y) in Eq. (17) is not analyzed. The favorable Van der Pol result could be specific to that problem. The authors should provide a local error estimate in terms of the Jacobian variation over the interval, or at least report the Jacobian variation along the training trajectories and test the sensitivity to using the midpoint or endpoint linearization instead of the initial-condition linearization.","section":"Section II E, Eqs. (12) and (17)"},{"comment":"The claim that no higher-order explicit exponential integration method is viable for stiff neural ODEs is deferred to a companion paper (Ref. 75) by the same authors, which is not included. This assertion is load-bearing for the paper's framing as leaving higher-order methods as an open problem and for the exclusive focus on IF Euler. Without presenting at least a summary of the accuracy and stability results for representative higher-order exponential methods (e.g., ETD2RK or a higher-order Lawson method) on a stiff benchmark, the claim should be weakened to a statement about the methods the authors tried, and the universality removed.","section":"Sections I, II E, and Conclusion, Ref. 75"},{"comment":"The experiments report a single trial per configuration with no seeds, error bars, or key hyperparameters (optimizer, learning rate, number of epochs, network width, initialization). The quantitative interpretation is further obscured because 'success' is not defined: in Example 4 the recovered equations deviate substantially from the true system (Table 11, n=100: the y-coefficient is 604.86 instead of 1000 and the x^2 y coefficient is -573.02 instead of -1000), and the error decays slowly with n. The authors should state a success criterion (e.g., relative parameter error below a threshold or trajectory error on a hold-out set) and report mean and variance over multiple random seeds.","section":"Section III C and D, Examples 3 and 4"}],"minor_comments":[{"comment":"The text calls IF Euler an 'A-stable first-order' scheme, but the stability statement applies only to the exact integration of the linear part; the explicit Euler treatment of the nonlinear remainder can still impose step-size restrictions. This should be clarified to avoid overstating the stability of the full scheme.","section":"Section II E, Eq. (17)"},{"comment":"The caption contains a duplicated word: 'explicit exponential integrating factor Euler method method' should read 'explicit exponential integrating factor Euler method'.","section":"Figure 5 caption"},{"comment":"The IF Euler rows for Example 3 contain many spurious higher-degree terms (e.g., y1^2 and y1 y2 terms with large coefficients) that are not present in the true model; this is a relevant observation that could be reported in the main text as a consequence of the first-order error and the frozen Jacobian, rather than appearing only in the tables.","section":"Tables 7-10"},{"comment":"The text says the Van der Pol stiffness is parameterized by mu and then sets mu=1000, but Eq. (21) is written in the standard two-dimensional form with the coefficient 1000 appearing explicitly; stating the relation between Eq. (21) and the usual x'' + mu(x^2-1)x' + x = 0 form would help readers.","section":"Section III D, Eq. (21)"}],"recommendation":"major_revision","confidential_remarks":"The paper's key comparative claims (implicit failure on Van der Pol, higher-order exponential methods unviable) are both deferred to other work by the same authors or presented without data; a revised version should either include those results or clearly circumscribe the claims. The linear-system experiments are exact by construction, so I would advise the editor that significance should be judged on the nonlinear evidence, which currently is a single un-seeded experiment. The companion papers Refs. 63 and 75 are closely related and should be checked for overlap with this submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick read of Fronk & Petzold on exponential integration for stiff neural ODEs. The genuinely new thing is empirical: they apply the classical integrating-factor Euler method to training neural ODEs and show it handles a stiff Van der Pol oscillator (µ=1000) where, they report, implicit single-step methods fail. That negative result for the implicit methods is stated without any failure data, so take it on faith. The method itself is Lawson/Cox–Matthews, and the exactness on linear systems is a known property of exponential integrators.\n\nWhat the paper does well: the linear-system experiments are clean and convincing. With n=17 points in a 10-dimensional stiff linear system, IF Euler recovers the coefficients to near machine precision; implicit methods do not. That makes sense analytically, and the tables back it up. The paper is also honest about its big limitation: first-order accuracy means the recovered Van der Pol coefficients are off (e.g., the damping coefficient 604.86 vs. the true 1000), and they explicitly leave higher-order methods as an open problem.\n\nThe soft spots are real but not fatal. The nonlinear examples lean on a frozen Jacobian (Eq. 12) evaluated once at the initial condition of each interval. For a genuinely nonlinear stiff system, the Jacobian can drift, and the paper gives no error estimate or criterion for when the frozen linearization is adequate. Example 3, the 3D nonlinear system, is actually a loss for IF Euler — it is the least accurate of the five methods compared. So the nonlinear case rests almost entirely on the Van der Pol example, which lacks seeds, error bars, hyperparameters, or the promised implicit-method failure logs. The dismissal of higher-order explicit exponential methods is outsourced to a companion paper (Ref. 75) not included here, which weakens the \"IF Euler is the only viable option\" framing — though the abstract itself is more careful, calling higher-order methods an open problem.\n\nThe citation pattern is fine; the paper engages with the classical numerical analysis literature and does not oversell its own novelty. It is a reasonable first step, not a breakthrough.\n\nWho would get value: researchers working on neural ODE training for stiff systems who want a cheap, explicit alternative to implicit solvers, and who are willing to accept first-order accuracy. The paper deserves a serious referee — the central empirical claim is interesting enough to warrant a closer look — but a revision should add experimental detail, include the implicit failure data, and either incorporate the higher-order analysis or drop the \"only viable option\" language.\n\nRecommendation: send it for review, with the expectation of moderate revision.","headline":"Classical IF Euler applied to stiff neural ODEs: the linear results are clean and exact, the Van der Pol claim is interesting but under-documented, and the higher-order method discussion is outsourced to a companion paper.","tokens_in":31753,"tokens_out":2599,"would_cite":false,"duration_ms":24085,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["65L04","65L20","65F60","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper establishes that the explicit integrating factor Euler (IF Euler) method can train stiff neural ODEs stably and cost-effectively, succeeding on the stiff Van der Pol oscillator where all tested implicit single-step methods fail.","keywords":["neural ordinary differential equations","stiff ODEs","exponential integration","integrating factor Euler","Van der Pol oscillator","polynomial neural networks","discretize-optimize","matrix exponential"],"falsifier":"Take a stiff nonlinear system whose Jacobian's dominant eigenvalue changes by orders of magnitude inside a single integration interval (for example, a Van der Pol oscillator with a time-dependent $\\mu(t)$ varying between 1 and 1000), and train it with IF Euler using the paper's large-step protocol; if training diverges or the recovered system is not stiff, the frozen-Jacobian assumption is the limiting factor and the method's robustness does not extend beyond systems with locally stable fast manifolds.","tokens_in":30744,"feed_emoji":"🧮","tokens_out":8618,"duration_ms":73673,"temperature":0.7,"pith_summary":"The paper argues that an explicit exponential integration method, the integrating factor Euler (IF Euler) scheme, can train stiff neural ODEs stably and cheaply, a task where standard implicit single-step solvers fail on the stiff Van der Pol oscillator. Because IF Euler avoids solving nonlinear systems at each step, it is much cheaper per step than backward Euler, trapezoid, or Radau methods. The method recovers stiff linear systems almost exactly even from very few training points, and it scales to a 10-dimensional stiff linear model. Its first-order accuracy, however, means that for nonlinear stiff problems it trades accuracy for stability, which the paper presents as an open limitation.","feed_headline":"Explicit method trains stiff neural ODEs implicit solvers can't","feed_subtitle":"One matrix exponential per step stabilizes training on stiff Van der Pol, at a fraction of implicit-solver cost.","key_machinery":"The engine of the method is the integrating factor transformation $w(t) = e^{-Lt}y(t)$, which converts the ODE into $\\dot{w} = e^{-Lt}N(t, e^{Lt}w)$ and lets forward Euler handle only the slow nonlinear remainder while the stiff linear part is solved exactly by the matrix exponential. The Jacobian $L$ is evaluated once per integration interval at the initial condition, so the matrix exponential $e^{Lh}$ acts as a fixed stabilizing preconditioner for that interval. This is combined with a $\\pi$-net polynomial neural network for the right-hand side, whose polynomial outputs allow exact equation recovery, and the discretize-optimize training paradigm, which makes gradients independent of the forward solver's accuracy.","core_discovery":"The central claim is that the explicit integrating factor Euler method is the most robust exponential integration method for training stiff neural ODEs. The scheme decomposes the right-hand side into a linear part frozen at the interval's initial Jacobian $L = df/dy(y_0)$ and a nonlinear remainder $N(t,y) = f(t,y) - Ly$, then exactly integrates the linear part with the matrix exponential while applying forward Euler to the transformed remainder: $y_{n+1} = e^{Lh}(y_n + hN_n)$. On the stiff Van der Pol oscillator with $\\mu = 1000$, every implicit single-step method the authors tried diverged during training, whereas IF Euler remained stable even with large step sizes and recovered equations whose dominant terms approximate the true system as training data increase. The method also reproduces stiff linear systems to near machine precision, matching or beating Radau5 with scarce data.","pith_inferences":["If the frozen-Jacobian assumption is the real source of stability, IF Euler should extend most naturally to systems whose stiff subspace is roughly constant along the trajectory; systems with rapidly rotating or sign-changing Jacobians would likely need a per-step Jacobian update.","The Van der Pol success may owe to its slow-fast structure, where a locally fixed linearization captures the fast stable manifold; a stiff system without such a clean timescale separation should be a harder test.","One could test this by running IF Euler on a stiff system with a time-dependent stiffness parameter (e.g., $\\mu(t)$) and comparing against a variant that recomputes $L$ at each step.","A higher-order extension might come from Rosenbrock-W methods, which share the same one-Jacobian-per-step structure but add order conditions without reintroducing nonlinear solves; the paper's negative result on other exponential methods suggests this route is not obvious."],"forward_implications":["Stiff neural ODE training can be done with explicit steps, removing the per-step nonlinear solves that make implicit methods costly and hard to differentiate.","Linear or nearly linear stiff dynamics can be identified from very few data points, as the 10D linear model is recovered almost exactly from 17 points, which matters for scientific system identification.","The stiff Van der Pol oscillator becomes trainable with large step sizes, a benchmark that previously defeated implicit single-step training in the authors' tests.","The first-order accuracy ceiling means that the practical benefit for nonlinear stiff problems is stability at coarse resolution, not high accuracy; a stable higher-order exponential method would be the next step.","The same explicit-exponential strategy could be applied to other differentiable simulations, such as neural PDEs, where repeated low-cost integration is required."],"supporting_citations":[{"why":"Defines neural ODEs as the model class this paper trains and backpropagates through.","marker":"[14]"},{"why":"Established that implicit single-step methods can train stiff neural ODEs; the benchmark implicit schemes here extend that study and its failure on Van der Pol motivates this work.","marker":"Ref. 63"},{"why":"Previous systematic comparison that concluded IF Euler is the most robust exponential integrator, which justifies focusing this paper on IF Euler alone.","marker":"Ref. 75"},{"why":"Supplies the exponential time-differencing formulation, including the A-stable first-order IF Euler scheme used here.","marker":"[64]"},{"why":"Origin of the integrating-factor (generalized Runge-Kutta) approach that the IF Euler scheme instantiates.","marker":"[98]"},{"why":"Provides the pi-net polynomial neural network architecture used to learn interpretable, recoverable ODE right-hand sides.","marker":"[33]"},{"why":"Supports the discretize-optimize training paradigm the paper adopts for gradient accuracy and efficiency.","marker":"[53]"}],"fun_headline_variants":["Explicit IF Euler beats implicit solvers on stiff neural ODEs","Matrix exponential step stabilizes stiff neural ODE training","IF Euler: explicit method that trains stiff neural ODEs","First-order explicit method tames stiff Van der Pol","Exponential integration outdoes implicit for stiff ODEs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's stability rests on assuming that the Jacobian evaluated once at the start of each integration interval continues to represent the stiff dynamics for the entire step; if the stiffness changes direction or magnitude within a step, the matrix exponential may stabilize the wrong linearization and the training can fail or learn incorrect dynamics.","fun_headline_variants_meta":{"raw":{"variants":["Explicit IF Euler beats implicit solvers on stiff neural ODEs","Matrix exponential step stabilizes stiff neural ODE training","IF Euler: explicit method that trains stiff neural ODEs","First-order explicit method tames stiff Van der Pol","Exponential integration outdoes implicit for stiff ODEs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000888,"raw_usage":{"total_tokens":3813,"prompt_tokens":910,"completion_tokens":2903,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":526,"completion_tokens_details":{"reasoning_tokens":2820}},"tokens_in":526,"tokens_out":2903,"duration_ms":18521,"temperature":1.0,"reasoning_tokens":2820,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:36:56.380362+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a stiff nonlinear system whose Jacobian's dominant eigenvalue changes by orders of magnitude inside a single integration interval (for example, a Van der Pol oscillator with a time-dependent $\\mu(t)$ varying between 1 and 1000), and train it with IF Euler using the paper's large-step protocol; if training diverges or the recovered system is not stiff, the frozen-Jacobian assumption is the limiting factor and the method's robustness does not extend beyond systems with locally stable fast manifolds.","supporting_citations":[],"review_version":1}