REVIEW 5 cited by
How to train your neural ODE: the world of Jacobian and kinetic regularization
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Training neural ODEs on large datasets has not been tractable due to the necessity of allowing the adaptive numerical ODE solver to refine its step size to very small values. In practice this leads to dynamics equivalent to many hundreds or even thousands of layers. In this paper, we overcome this apparent difficulty by introducing a theoretically-grounded combination of both optimal transport and stability regularizations which encourage neural ODEs to prefer simpler dynamics out of all the dynamics that solve a problem well. Simpler dynamics lead to faster convergence and to fewer discretizations of the solver, considerably decreasing wall-clock time without loss in performance. Our approach allows us to train neural ODE-based generative models to the same performance as the unregularized dynamics, with significant reductions in training time. This brings neural ODEs closer to practical relevance in large-scale applications.
Forward citations
Cited by 5 Pith papers
-
Convex Relaxations for the Optimization of Markov Processes
Sequential-coupling convex relaxations using local marginals and cluster moments solve high-dimensional Markov process optimization, recovering Benamou–Brenier dynamics and general kernels as special cases.
-
IRIS: A Real-World Benchmark for Inverse Recovery and Identification of Physical Dynamic Systems from Monocular Video
IRIS releases 220 real 4K videos of eight dynamical systems with ground-truth parameters plus a protocol that measures parameter recovery, equation selection, and multi-body failure modes of unsupervised video-to-phys...
-
A Kinetic Energy Perspective of Flow Matching
A per-sample kinetic-energy score along flow-matching trajectories tracks semantic quality and data rarity, with an extreme-energy regime that predicts memorization, and a two-phase inference-time shaping that improve...
-
Generalization Bound for a General Class of Neural Ordinary Differential Equations
Claims a first generalization bound for nonlinear neural ODEs, but bounds the complexity of time trajectories rather than input-output maps, leaving the main theorem unproven.
-
Deep Potential: Recovering the gravitational potential and local pattern speed in the solar neighborhood with GDR3 using normalizing flows
Using normalizing flows and a neural network on Gaia DR3 data, the authors recover a local pattern speed of 28.2 km/s/kpc and a total matter density of 0.086 solar masses per cubic parsec within 1 kpc of the Sun.
Discussion (0). Continue with ORCID to comment.