REVIEW 6 cited by
Training End-to-End Analog Neural Networks with Equilibrium Propagation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We introduce a principled method to train end-to-end analog neural networks by stochastic gradient descent. In these analog neural networks, the weights to be adjusted are implemented by the conductances of programmable resistive devices such as memristors [Chua, 1971], and the nonlinear transfer functions (or `activation functions') are implemented by nonlinear components such as diodes. We show mathematically that a class of analog neural networks (called nonlinear resistive networks) are energy-based models: they possess an energy function as a consequence of Kirchhoff's laws governing electrical circuits. This property enables us to train them using the Equilibrium Propagation framework [Scellier and Bengio, 2017]. Our update rule for each conductance, which is local and relies solely on the voltage drop across the corresponding resistor, is shown to compute the gradient of the loss function. Our numerical simulations, which use the SPICE-based Spectre simulation framework to simulate the dynamics of electrical circuits, demonstrate training on the MNIST classification task, performing comparably or better than equivalent-size software-based neural networks. Our work can guide the development of a new generation of ultra-fast, compact and low-power neural networks supporting on-chip learning.
Forward citations
Cited by 6 Pith papers
-
Equilibrium Propagation for Non-Conservative Systems
A modified Equilibrium Propagation with an antisymmetric-Jacobian correction computes exact cost gradients for non-conservative neural dynamics.
-
Circuit realization and hardware linearization of monotone operator equilibrium networks
Resistor-diode circuits realize ReLU monotone operator equilibrium networks, and their exact gradient can be computed in the same hardware by linearizing the diodes.
-
An analog-electronic implementation of a harmonic oscillator recurrent neural network
An analog circuit implementing a four-node harmonic oscillator network preserves enough information to match its digital twin's sMNIST classification accuracy with a retrained linear readout.
-
Equilibrium Propagation for Dissipative Dynamics
An effective action with time-reversed trajectories extends equilibrium propagation to damped linear reciprocal networks, enabling temporal learning demonstrated on mechanical and RLC systems.
-
A Blueprint for Equilibrium-Based Differentiable Continuous-Variable Thermodynamic Computing
Tunable energy landscapes whose thermal averages equal sigmoid, softmax, and matrix-vector products can, in principle, form the basis of a low-energy analog computer, with a superconducting double-well device as a fir...
-
Bridging Predictive Coding and MDL: A Two-Part Code Framework for Deep Learning
A theoretical framework claims that predictive coding performs block-coordinate descent on a two-part code objective and bounds true risk by empirical risk plus codelength divided by sample size.
Discussion (0). Sign in to comment.