ERNNs set each hidden state to the fixed point of an implicit ODE, making the state-to-state Jacobian exactly -I (norm 1) at equilibrium, eliminating vanishing/exploding gradients in theory and giving strong empirical training speedups.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2019 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
RNNs Evolving on an Equilibrium Manifold: A Panacea for Vanishing and Exploding Gradients?
ERNNs set each hidden state to the fixed point of an implicit ODE, making the state-to-state Jacobian exactly -I (norm 1) at equilibrium, eliminating vanishing/exploding gradients in theory and giving strong empirical training speedups.