REVIEW 3 cited by
A recurrent neural network without chaos
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We introduce an exceptionally simple gated recurrent neural network (RNN) that achieves performance comparable to well-known gated architectures, such as LSTMs and GRUs, on the word-level language modeling task. We prove that our model has simple, predicable and non-chaotic dynamics. This stands in stark contrast to more standard gated architectures, whose underlying dynamical systems exhibit chaotic behavior.
Forward citations
Cited by 3 Pith papers
-
Chaos-Free Networks are Stable Recurrent Neural Networks
Decoupled-Gate Networks, a structural simplification of Chaos-Free Networks, are unconditionally incrementally input-to-state stable by design for nonlinear system identification.
-
Algorithm Development in Neural Networks: Insights from the Streaming Parity Task
RNNs trained on short parity sequences can suddenly generalize to arbitrary length by merging hidden states that agree on future outputs, forming a finite automaton.
-
Hyperpruning: Efficient Search through Pruned Variants of Recurrent Neural Networks Leveraging Lyapunov Spectrum
A Lyapunov-spectrum-based distance to the dense network lets hyperparameter search for pruned RNNs stop early and select models that beat both loss-based baselines and the dense originals.
Discussion (0). Sign in to comment.