Pith. sign in

REVIEW 4 major objections 4 minor 37 references

Semi-Implicit Neural Ordinary Differential Equations

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that splitting a neural ODE into a nonlinear part handled explicitly and a fixed linear part handled implicitly, with a discrete-adjoint backward pass, allows stable training of stiff systems at time steps two orders of…

desk verdict A practical semi-implicit neural ODE trainer with real speedups on stiff partitioned systems, but the headline claim is narrower than it looks: every experiment hard-codes the linear part. read the letter →

arxiv 2412.11301 v1 pith:VCBDYQBD submitted 2024-12-15 cs.LG cs.AI

Hong Zhang , Ying Liu , Romit Maulik This is my paper · ORCID
classification cs.LGcs.AI MSC 65L0668T07
keywords neuralordinarydifferentialequationssemi-implicitintegrationimplicit-explicitRunge-KuttastiffODEsdiscreteadjointgraphnetworksscientificmachinelearningchaoticdynamics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Neural ODEs trained with explicit time integrators become unstable on stiff problems, forcing tiny time steps that make training impractical. This paper claims that the instability can be removed by splitting the right-hand side into a nonlinear part $G(u)$, handled explicitly, and a fixed linear part $J u$, handled implicitly by an implicit-explicit Runge--Kutta (IMEX-RK) integrator. Because the implicit part is linear, each stage solve is a linear system with a fixed matrix whose factorization can be reused, and gradients are computed by a discrete adjoint that matches the forward discretization. The result, called SINODE, is claimed to train stiff graph and PDE models with time steps two orders of magnitude larger than explicit methods, and to succeed where both explicit and fully implicit training fail.

What carries the argument

The load-bearing object is the additive split of the vector field into a nonlinear $G(u)$ and a linear $J u$, plus an IMEX-RK time integrator that advances $G$ explicitly and $Ju$ implicitly. Each implicit stage reduces to the constant-coefficient linear system in Eq. (12), $(I-\Delta t\,\tilde a_{ii}J)U^{(i)}=u_n+\Delta t\sum_{j<i}a_{ij}G^{(j)}+\Delta t\sum_{j<i}\tilde a_{ij}JU^{(j)}$, which avoids nonlinear Newton iteration altogether; with $J$ constant, the factorization of $I-\Delta t\,\tilde a_{ii}J$ is reused across stages, steps, and mini-batches. The backward pass is the discrete adjoint of the same IMEX-RK map (Theorem 4.1), giving a reverse-accurate gradient whose memory cost depends on the network, not on the number of solver iterations or time steps.

What would settle it

Run the GRAND Cora benchmark with the linear operator learned instead of fixed to $-I$ in Eq. (19), updating it every optimizer step so factorizations must be recomputed each epoch; if per-epoch time and loss no longer beat explicit Adams and RK4 by a large margin, then the reusable constant linear system, not IMEX stability alone, is the load-bearing mechanism and the method's generality is limited to problems with a known fixed linear split.

Watch

Extended reading notes

Core claim

The paper's central claim is that a learnable ODE $\frac{du}{dt}=G(u)+Ju$ can be trained stably and efficiently when $G$ is integrated explicitly and $Ju$ implicitly, using IMEX-RK time stepping and a reverse-accurate discrete adjoint. The linearity of the implicit term turns every implicit stage into the linear system $(I-\Delta t\,\tilde a_{ii}J)U^{(i)}=\text{known}$, whose coefficient matrix does not change across stages, time steps, or (when $J$ is fixed) mini-batches, so off-the-shelf linear solvers and reused factorizations become available during training. On the GRAND graph benchmark and on Kuramoto--Sivashinsky and viscous Burgers learning problems, the paper reports stable training at step sizes roughly two orders of magnitude larger than explicit Runge--Kutta methods permit, with far fewer network evaluations per epoch and successful training in cases where fully implicit methods blow up or fail to converge.

Load-bearing premise

The whole speed and stability advantage rests on the model's dynamics being expressible as a nonlinear part plus a fixed linear part, because without a fixed linear part there is no cheap reusable linear equation to solve.

Editorial extensions

If this is right

  • Stiff neural ODEs that previously needed step sizes near $10^{-3}$ can use step sizes near $1$ in the tested problems, cutting per-epoch network evaluations from tens of thousands to tens or hundreds.
  • On the GRAND graph benchmark, SINODE trains stably where explicit RK4 and explicit Adams diverge and where fully implicit Crank--Nicolson and implicit Adams blow up on Cora.
  • For the Kuramoto--Sivashinsky and viscous Burgers learning tasks, SINODE remains trainable at fine grid resolutions where explicit methods require step sizes around $10^{-7}$ and fully implicit methods suffer nonlinear-solver divergence.
  • The discrete adjoint computes gradients consistent with the discrete forward pass, so the large-step training is not biased by the discretization mismatch of continuous adjoint methods.
  • When $J$ is fixed, one LU factorization is shared across every stage, time step, and mini-batch within an epoch, so training cost scales with network evaluations rather than with the cost of solving the linear systems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the tested benchmarks, any stiff learning problem with a known linear operator, such as a diffusion, Laplacian, or spectral term, could use the same split, but the paper does not demonstrate a general recipe for finding such splits in arbitrary neural networks.
  • Allowing $J$ to be learned while held fixed for several epochs at a time would likely preserve most of the factorization-reuse benefit and extend SINODE to problems without a prescribed linear part; this is an extension, not a paper claim.
  • Because each fixed factorization is reused more often as the grid resolution or the number of time steps grows, the speed advantage over explicit training should widen on larger problem sizes than those tested.
  • The failures reported for fully implicit methods suggest that the practical bottleneck in stiff neural ODE training is the cost and fragility of Newton iterations on ill-conditioned systems, and that replacing only the stiff part with a linear implicit solve removes that bottleneck; this diagnosis is inferred from the reported blow-ups rather than proved.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes SINODE, a semi-implicit approach for training neural ODEs whose right-hand side can be additively split into a nonlinear part G(u) and a linear part J u. The forward pass uses IMEX-RK methods, treating G explicitly and the linear part implicitly, which reduces each stage to a linear system when J is constant. The backward pass uses a discrete adjoint (Theorem 4.1) to obtain reverse-accurate gradients. Experiments on graph classification with GRAND, and on learning the Kuramoto–Sivashinsky and viscous Burgers equations, show that SINODE can use much larger time steps than explicit methods and remains stable where fully implicit methods fail, yielding substantial speedups in training time.

Significance. If the central claim holds, SINODE provides a practical way to train stiff neural ODEs by leveraging well-understood IMEX time integrators and off-the-shelf linear solvers, with the advantage of reverse-accurate discrete adjoints. The paper includes a concrete algorithm, a stated adjoint formula, and a reproducible code repository, and the experiments demonstrate clear stability and efficiency gains on problems with a known linear part. The main caveat is that the method's applicability currently hinges on the availability of a constant linear-nonlinear split, which is supplied externally in every experiment; the paper's headline claim about general challenging neural ODEs is therefore broader than what is demonstrated.

major comments (4)
  1. [Abstract and Sec. 7 (Limitations)] The abstract claims SINODE 'can train challenging neural ODEs where both explicit methods and fully implicit methods are intractable.' This claim is not qualified by the requirement, evident from Secs. 5.1–5.3, that a constant linear part J must be known a priori or supplied externally (e.g., Eq. (19) for GRAND, the fixed finite-difference convolution for KS, and the fixed finite-difference layer for Burgers). The paper does not propose a procedure for discovering a suitable split when the dynamics are not already separated, and if J were learned, the reusable-factorization argument in Sec. 4.1 would weaken to at most once per training iteration. The Limitations section does not list this as a limitation, which makes the scope of the headline claim easy to overread. The authors should either provide a method for constructing the split or explicitly restrict the central claims to problems with a known constant linear part.
  2. [Sec. 5.1 (GRAND experiments)] The GRAND experiments report NFEs per epoch (Table 1) and show testing accuracy versus training time (Fig. 1), but no final test accuracy table is provided, and no standard deviations across multiple seeds are reported for any dataset or method. Given that graph classification results are typically reported as mean±std over seeds, the absence of such numbers makes it difficult to assess whether the observed accuracy differences are significant. The authors should add a table of final test accuracies with error bars for all methods on Cora, CoauthorCS, and Photo.
  3. [Tables 1 and 2; Sec. 5.1-5.3] The NFE comparisons exclude the cost of linear solves, which are central to IMEX and fully implicit methods. While the paper also shows wall-clock training-time curves (Figs. 1, 2, 4, 6), these are not tabulated, and the reader cannot easily compare the efficiency of the methods in a quantitative way. The claim that NFE is 'usually a good metric' (Sec. 5.1) needs support: for implicit methods, the per-step linear solve cost can dominate the NN evaluation cost, especially at higher grid resolutions. The authors should either justify NFE as a fair metric in the presence of linear solves or report per-epoch wall-clock times in a table for all datasets and step sizes.
  4. [Appendix A (Proof of Theorem 4.1)] Theorem 4.1 is the backbone of the training algorithm, but the proof in Appendix A is only carried out for explicit RK (Eqs. (22)–(28)); the extension to IMEX-RK is asserted in a single sentence without a derivation. Since the IMEX adjoint involves two coefficient tableaux and mixed explicit/implicit stage treatments, a complete derivation (or a precise reference to one) is needed to make the reverse-accuracy claim fully verifiable. Additionally, the appendix is titled 'Proof of Theorem 5.1' while the theorem in the main text is numbered 4.1.
minor comments (4)
  1. [Sec. 4.2, after Eq. (15)] The text says the same strategy as Sec. 4.1 can be used to solve the transposed linear system, but for a non-symmetric J the transpose is generally not the same matrix as the forward system, so the LU factorization from the forward pass cannot be reused directly; this should be clarified.
  2. [Sec. 4.1, Eq. (13)] The Kronecker-product expression in Eq. (13) is not fully clear: the identity matrix I_m acts on the batch dimension, but the product with (I - Δt a_ij J) seems to mix state and batch indices. The matrix-free reshape interpretation in Eq. (14) is correct, but the notation in (13) should be cleaned up to avoid confusion.
  3. [Appendix A, Eq. (26)] The adjoint equations in the ERK proof appear to have a minor indexing issue in the last line of Eq. (26) (the factor b_i is omitted on the right-hand side of the second equation). While the final implementation in Eq. (30) looks correct, the proof's intermediate formulas should be corrected for consistency.
  4. [Sec. 5 and Appendix C] The exact time-step sizes used for each method and dataset are mentioned in the text but are not collected in one place. A table summarizing the step sizes for explicit, implicit, and IMEX methods across GRAND, KS, and Burgers would improve reproducibility.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the SINODE derivation is self-contained and the efficiency gains are empirically benchmarked against external IMEX-RK and integrator baselines.

full rationale

The central construction is not a renamed fit or a self-citation chain. Eq. (10) defines the additive split du/dt = G(u) + J u, and Eqs. (11)-(12) specialize the standard IMEX-RK stage equations to the case where H is linear, giving the linear system (I - Delta t a_ii J) U = ... by direct algebra. The discrete adjoint in Eqs. (15)-(17) is derived in Appendix A from the stage equations via a Lagrangian calculation, not imported from a self-citation. Stability properties of IMEX-RK are attributed to the external numerical-analysis literature (Ascher-Ruuth-Spiteri; Kennedy-Carpenter) and then validated by direct comparisons of NFEs, time-to-accuracy, and loss trajectories against explicit RK/Adams and fully implicit Crank-Nicolson/Adams on GRAND, KS, and Burgers tasks. No fitted parameter is renamed as a prediction; J is supplied by the physics (identity in GRAND; fixed finite-difference operators in KS and Burgers) and is not tuned to reproduce the target metric. Self-citations to the authors' PNODE framework and checkpointing papers support implementation details and do not carry the load of either the theoretical derivation or the empirical claims. The main caveat is scope: the strongest general claim about 'challenging neural ODEs' relies on a known constant linear part, and that requirement is not listed in the Limitations section. This is a scope limitation, not circularity, because Eq. (12) and the stability comparison remain independently checkable without assuming the conclusion.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new postulates or entities; its background is standard numerical analysis and neural ODE theory. The main assumption is partitionability, and the discrete adjoint theorem is asserted with an incomplete proof, so it is listed as an assumption here.

assumptions (3)
  • domain assumption The right-hand side of the ODE can be written as G(u) + J u with J linear and constant.
    This is the core assumption of SINODE (Eq. 10); the paper provides no automatic way to find such a split.
  • standard math IMEX-RK methods have the stability and accuracy properties stated in the cited numerical analysis literature.
    The paper relies on Ascher et al. 1997, Kennedy and Carpenter 2003, and Pareschi and Russo 2005 for the schemes' properties without re-deriving them.
  • ad hoc to paper Theorem 4.1 (the discrete adjoint for IMEX-RK) is correct as stated.
    The appendix proves the ERK adjoint and asserts the IMEX extension is straightforward; the actual IMEX formula (15) is not fully derived in the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Semi-Implicit Neural Ordinary Differential Equations." pith.science (2026). https://pith.science/paper/VCBDYQBD

@misc{pith2026241211301,
  author       = {Pith},
  title        = {Pith review of: Semi-Implicit Neural Ordinary Differential Equations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VCBDYQBD}},
  note         = {Machine review of arXiv:2412.11301}
}
read the original abstract

Classical neural ODEs trained with explicit methods are intrinsically limited by stability, crippling their efficiency and robustness for stiff learning problems that are common in graph learning and scientific machine learning. We present a semi-implicit neural ODE approach that exploits the partitionable structure of the underlying dynamics. Our technique leads to an implicit neural network with significant computational advantages over existing approaches because of enhanced stability and efficient linear solves during time integration. We show that our approach outperforms existing approaches on a variety of applications including graph classification and learning complex dynamical systems. We also demonstrate that our approach can train challenging neural ODEs where both explicit methods and fully implicit methods are intractable.

Figures

Figures reproduced from arXiv: 2412.11301 by the authors.

Figure 2
Figure 2. Testing accuracy vs. training time for various step [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Ground truth (Left) vs. prediction (Right) using IMEX-RK3 with ∆t = 0.2. In [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Training loss vs. training time for KS. Left: Grid size 64. Right: Grid size 512. Only SINODE with IMEX methods achieve feasible training completion for the case with grid size 512. diffusion term. We are not able to make the fully implicit methods work with the step size 0.2 because the nonlinear solver diverges frequently during the training process. This is not uncommon for classical PDE solvers since the condi￾t… view at source ↗
Figures from the paper (9 more)
Figure 5
Figure 5. Figure 5: Snapshots of VBE trajectories for grid size [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Training loss vs. training time for VBE. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Training loss vs. epoch for a variety of time integrators using explicit, implicit, and semi-implicit methods. The [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Training loss for KSE on a uniform grid of size [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Training loss for KSE on a uniform grid of size [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Snapshots of VBE trajectories of ground truth and model predictions for grid size [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Snapshots of VBE trajectories of ground truth and model predictions for grid size [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: Training loss for the Burgers equation on a uniform grid of size [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: Training loss for the Burgers equation on a uniform grid of size [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 32 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    M.; Ruuth, S

    Ascher, U. M.; Ruuth, S. J.; and Spiteri, R. J. 1997. Implicit-explicit Runge--Kutta methods for time-dependent partial differential equations. Applied Numerical Mathematics, 25(2-3): 151--167

    work page 1997

  4. [4]

    Z.; and Koltun, V

    Bai, S.; Kolter, J. Z.; and Koltun, V. 2019. Deep equilibrium models. In Advances in Neural Information Processing Systems, volume 32

    work page 2019

  5. [5]

    Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEs

    Baker, J.; Xia, H.; Wang, Y.; Cherkaev, E.; Narayan, A.; Chen, L.; Xin, J.; Bertozzi, A. L.; Osher, S. J.; and Wang, B. 2022. Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEs . arXiv:2204.08621

    work page Pith review arXiv 2022

  6. [6]

    F.; Benson, S.; Brown, J.; Brune, P.; Buschelman, K.; Constantinescu, E.; Dalcin, L.; Dener, A.; Eijkhout, V.; Faibussowitsch, J.; Gropp, W

    Balay, S.; Abhyankar, S.; Adams, M. F.; Benson, S.; Brown, J.; Brune, P.; Buschelman, K.; Constantinescu, E.; Dalcin, L.; Dener, A.; Eijkhout, V.; Faibussowitsch, J.; Gropp, W. D.; Hapla, V.; Isaac, T.; Jolivet, P.; Karpeev, D.; Kaushik, D.; Knepley, M. G.; Kong, F.; Kruger, S.; May, D. A.; McInnes, L. C.; Mills, R. T.; Mitchell, L.; Munson, T.; Roman, J....

    work page 2023

  7. [7]

    Boscarino, S. 2007. Error analysis of IMEX Runge--Kutta methods derived from differential-algebraic systems. SIAM Journal on Numerical Analysis, 45(4): 1600--1621

    work page 2007

  8. [8]

    Boscarino, S.; and Russo, G. 2009. On a class of uniformly accurate IMEX Runge--Kutta schemes and applications to hyperbolic systems with relaxation. SIAM Journal on Scientific Computing, 31(3): 1926--1945

    work page 2009

Show all 37 references
  1. [9]

    I.; Bronstein, M.; Webb, S.; and Rossi, E

    Chamberlain, B.; Rowbottom, J.; Gorinova, M. I.; Bronstein, M.; Webb, S.; and Rossi, E. 2021. Grand: Graph neural diffusion . In International Conference on Machine Learning, 1407--1418. PMLR

    2021

  2. [10]

    Q.; Rubanova, Y.; Bettencourt, J.; Duvenaud, D.; Chen, R

    Chen, T. Q.; Rubanova, Y.; Bettencourt, J.; Duvenaud, D.; Chen, R. T.; Rubanova, Y.; Bettencourt, J.; and Duvenaud, D. 2018. Neural Ordinary Differential Equations. In Advances in Neural Information Processing Systems

    2018

  3. [11]

    Daulbaev, T.; Katrutsa, A.; Markeeva, L.; Gusak, J.; Cichocki, A.; and Oseledets, I. 2020. Interpolation Technique to Speed Up Gradients Propagation in Neural ODEs . In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.; and Lin, H., eds., Advances in Neural Information Proc...

    2020

  4. [12]

    Dupont, E.; Doucet, A.; and Teh, Y. W. 2019. Augmented Neural ODEs . In Wallach, H.; Larochelle, H.; Beygelzimer, A.; d Alch\' e -Buc, F.; Fox, E.; and Garnett, R., eds., Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc

    2019

  5. [13]

    W.; Heaton, H.; Li, Q.; McKenzie, D.; Osher, S.; and Yin, W

    Fung, S. W.; Heaton, H.; Li, Q.; McKenzie, D.; Osher, S.; and Yin, W. 2022. JFB: Jacobian -Free Backpropagation for Implicit Networks. In Proceedings of the 36th AAAI Conference on Artificial Intelligence, AAAI 2022, volume 36

    2022

  6. [14]

    Gholaminejad, A.; Keutzer, K.; and Biros, G. 2019. ANODE : Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs . In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19 , 730--736. International Joint Conferences ...

    2019

  7. [15]

    Haber, E.; Lensink, K.; Treister, E.; and Ruthotto, L. 2019. IMEX net A Forward Stable Deep Neural Network. In Chaudhuri, K.; and Salakhutdinov, R., eds., Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Researc...

    2019

  8. [16]

    Kassam, A.-K.; and Trefethen, L. N. 2005. Fourth-Order Time-Stepping for Stiff PDEs . SIAM Journal on Scientific Computing, 26(4): 1214--1233

    2005

  9. [17]

    A.; and Carpenter, M

    Kennedy, C. A.; and Carpenter, M. H. 2003 a . Additive Runge--Kutta schemes for convection--diffusion--reaction equations. Applied Numerical Mathematics, 44(1-2): 139--181

    2003

  10. [18]

    A.; and Carpenter, M

    Kennedy, C. A.; and Carpenter, M. H. 2003 b . Additive Runge--Kutta schemes for convection–diffusion–reaction equations. Applied Numerical Mathematics, 44(1): 139--181

    2003

  11. [19]

    Kidger, P. 2022. On Neural Differential Equations. arXiv preprint arXiv:2202.02435

    2022 arXiv

  12. [20]

    Hey, that's not an ODE

    Kidger, P.; Chen, R. T. Q.; and Lyons, T. 2020. "Hey, that's not an ODE": Faster ODE Adjoints via Seminorms. In International Conference on Machine Learning

    2020

  13. [21]

    Kim, S.; Ji, W.; Deng, S.; Ma, Y.; and Rackauckas, C. 2021. Stiff neural ordinary differential equations . Chaos: An Interdisciplinary Journal of Nonlinear Science, 31(9)

    2021

  14. [22]

    J.; Burby, J

    Linot, A. J.; Burby, J. W.; Tang, Q.; Balaprakash, P.; Graham, M. D.; and Maulik, R. 2022. Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systems. Journal of Computational Physics, 474

    2022

  15. [23]

    Massaroli, S.; Poli, M.; Park, J.; Yamashita, A.; and Asama, H. 2020. Dissecting Neural ODEs . In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.; and Lin, H., eds., Advances in Neural Information Processing Systems, volume 33, 3952--3963. Curran Associates, Inc

    2020

  16. [24]

    Matsubara, T.; Miyatake, Y.; and Yaguchi, T. 2023. The Symplectic Adjoint Method: Memory-Efficient Backpropagation of Neural-Network-Based Differential Equations. IEEE Transactions on Neural Networks and Learning Systems, 1--13

    2023

  17. [25]

    T.; Adams, M

    Mills, R. T.; Adams, M. F.; Balay, S.; Brown, J.; Dener, A.; Knepley, M.; Kruger, S. E.; Morgan, H.; Munson, T.; Rupp, K.; Smith, B. F.; Zampini, S.; Zhang, H.; and Zhang, J. 2021. Toward performance-portable PETSc for GPU-based exascale systems . Parallel Computing, 108: 102831

    2021

  18. [26]

    Onken, D.; and Ruthotto, L. 2020. Discretize-Optimize vs. Optimize-Discretize for Time-Series Regression and Continuous Normalizing Flows. arXiv preprint arXiv:2005.13420

    2020 arXiv

  19. [27]

    Pareschi, L.; and Russo, G. 2005. Implicit--explicit Runge--Kutta schemes and applications to hyperbolic systems with relaxation. Journal of Scientific Computing, 25: 129--155

    2005

  20. [28]

    Poli, M.; Massaroli, S.; Park, J.; Yamashita, A.; Asama, H.; and Park, J. 2019. Graph Neural Ordinary Differential Equations. arXiv preprint arXiv:1911.07532

    2019 arXiv

  21. [29]

    Winston, E.; and Kolter, J. Z. 2020. Monotone operator equilibrium networks. In Advances in Neural Information Processing Systems, volume 2020-December

    2020

  22. [30]

    Xhonneux, L.-P.; Qu, M.; and Tang, J. 2020. Continuous Graph Neural Networks. In III, H. D.; and Singh, A., eds., Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, 10432--10441. PMLR

    2020

  23. [31]

    Zhang, H.; and Constantinescu, E. M. 2021. Revolve-based adjoint checkpointing for multistage time integration . In International Conference on Computational Science . (online), in main track

    2021

  24. [32]

    Zhang, H.; and Constantinescu, E. M. 2023. Optimal checkpointing for adjoint multistage time-stepping schemes. Journal of Computational Science, 66: 101913

    2023

  25. [33]

    Zhang, H.; and Sandu, A. 2014. FATODE: a library for forward, adjoint, and tangent linear integration of ODEs . SIAM Journal on Scientific Computing, 36(5): C504--C523

    2014

  26. [34]

    Zhang, H.; and Zhao, W. 2022. A Memory-Efficient Neural Ordinary Differential Equation Framework Based on High-Level Adjoint Differentiation. IEEE Transactions on Artificial Intelligence, 1--11

    2022

  27. [35]

    E.; Keutzer, K.; Mahoney, M

    Zhang, T.; Yao, Z.; Gholami, A.; Gonzalez, J. E.; Keutzer, K.; Mahoney, M. W.; and Biros, G. 2019. ANODEV2 : A Coupled Neural ODE Framework. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc

    2019

  28. [36]

    Zhuang, J.; Dvornek, N.; Li, X.; Tatikonda, S.; Papademetris, X.; and Duncan, J. 2020. Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODE . In Proceedings of the 37th International Conference on Machine Learning, volume 119, 11639--11649

    2020

  29. [37]

    C.; Tatikond, S.; and Duncan, J

    Zhuang, J.; Dvornek, N. C.; Tatikond, S.; and Duncan, J. S. 2021. MALI : A memory efficient and reverse accurate integrator for Neural ODE s. In International Conference on Learning Representations

    2021

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.