Central-difference random gradient estimation (CD-RGE) trains RNNs up to 1.1B parameters without backpropagation through time, matching or beating BPTT on several small tasks while using less GPU memory.
Aditya Rawal and Risto Miikkulainen
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Scaling Recurrent Neural Networks to a Billion Parameters with Zero-Order Optimization
Central-difference random gradient estimation (CD-RGE) trains RNNs up to 1.1B parameters without backpropagation through time, matching or beating BPTT on several small tasks while using less GPU memory.