REVIEW 3 major objections 5 minor 41 references
Noise-based Local Learning using Stochastic Magnetic Tunnel Junctions
T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read This paper demonstrates that the two-level, autocorrelated noise of a stochastic magnetic tunnel junction is sufficient to train multilayer neural networks under the decorrelated activity-based node perturbation rule, approaching…
desk verdict DANP learning genuinely works with real sMTJ noise in simulation, but the abstract's hardware claim outruns what was actually built: only the noise source is physical, not the network. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central machinery is the decorrelated activity-based node perturbation (DANP) rule: after two forward passes with different noise perturbations, the update to layer $l$ weights is $\delta W_l = N \langle \delta L \, \delta a_l / \|\delta a\|^2 \, \bar{x}_{l-1}^\top \rangle$, where $\delta L$ is the loss difference, $\delta a$ the activity difference, and $\bar{x}_{l-1}$ the decorrelated input, with the decorrelation transform $\bar{x}_l = R_l x_l$ learned by an iterative local rule. Because it compares two noisy passes, DANP needs no clean reference pass, no access to the injected noise values, and no measurement of spatial correlations between nodes. The physical noise source is a stochastic magnetic tunnel junction, a magnetic device whose resistance randomly toggles between two levels via thermally driven magnetization switching.
What would settle it
If a full analog 2x2x2x1 network built from the proposed sMTJ noise-injection circuit is trained and its loss does not decrease over 100 epochs, or if the same loss decrease occurs when the sMTJ is replaced with a fixed voltage, then the claim that physical sMTJ noise drives learning is falsified.
Extended reading notes
Core claim
The paper's claim is that the two-level, autocorrelated, non-Gaussian noise actually produced by a stochastic magnetic tunnel junction is a sufficient learning signal for node-perturbation training. In simulations, multilayer networks with three hidden layers of 500 units trained by DANP with noise drawn either from a hidden Markov model fitted to a measured sMTJ time series or from that same time series fed serially to all nodes reach accuracies approaching those of backpropagation on MNIST, CIFAR-10, and CIFAR-100; the serially fed worst-case noise with strong autocorrelation still learns. A live setup, in which a microcontroller reads a single sMTJ and passes its samples to a Python 2x2x2x1 network, shows loss decreasing over 100 epochs. The paper also gives a circuit proposal for a fully analog version, while noting that the complete CMOS learning circuitry is left for future work.
Load-bearing premise
The paper assumes that injecting the measured sMTJ noise into a software model faithfully represents what would happen in the proposed analog circuit, since the physical experiment itself keeps the network, weights, and updates in software rather than in the resistive network.
Editorial extensions
If this is right
- Learning in a physical network requires only two noisy forward passes and a scalar loss difference; no backward pass and no clean reference pass are needed.
- The injected noise does not need to be Gaussian, measurable, or independent across nodes: real two-level sMTJ noise, even serially autocorrelated, trains multilayer networks.
- Because DANP is local except for the global loss signal, the weight updates and the layer-level decorrelation can in principle be implemented with analog circuitry, reducing the energy cost relative to digital backpropagation.
- The approach generalizes beyond magnetic tunnel junctions to any stochastic physical device whose noise can be coupled into node pre-activations, such as memristive devices.
- Tunability of sMTJ dwell times, which can reach nanoseconds, sets the bandwidth of the learning process and allows the training loop to speed up as the hardware improves.
Reading between the lines
- A direct test the paper leaves implicit is whether the proposed analog circuit, with resistive weights, diode activation, and sMTJ voltage-adder noise injection, reproduces the simulated DANP dynamics; the reported experiment keeps the network itself in software.
- Because the worst-case single-source autocorrelated noise still trains nearly as well as independent simulated sources, the effective requirement on the physical noise may be much weaker than white noise, broadening the class of usable devices.
- Since DANP uses only a global scalar feedback signal, the same rule should transfer to physical systems whose forward dynamics are not differentiable, such as optical or mechanical networks, where backpropagation cannot be defined.
- If sMTJ fluctuation rates can be tuned from milliseconds to nanoseconds, the approach makes a concrete energy-accuracy trade-off: faster noise speeds up learning iterations but raises the bandwidth and power cost of the readout and update circuitry.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes using the intrinsic stochastic noise of superparamagnetic magnetic tunnel junctions (sMTJs) as the perturbation source for decorrelated activity-based node perturbation (DANP), a local learning rule that avoids backpropagation. After recapping DANP in Section II.A, the paper characterizes measured sMTJ noise, fits a two-state hidden Markov model to it, and proposes an analog circuit concept in Figure 1 in which sMTJ noise is injected at each node. Section III reports three levels of evidence: single-layer ANP robustness sweeps on MNIST (Figure 3); multilayer DANP simulations on MNIST, CIFAR-10, and CIFAR-100 using both simulated and measured sMTJ noise (Figure 4); and a proof-of-concept experiment in which an Arduino reads a single sMTJ and feeds its voltage samples to a 2x2x2x1 network that otherwise runs in Python (Figure 5). The central numerical claim is that DANP with real, autocorrelated, two-level sMTJ noise learns effectively in multilayer networks and approaches backpropagation performance. The abstract further claims that learning was demonstrated experimentally 'in a small network composed of physical stochastic magnetic tunnel junctions,' but the hardware experiment does not implement the network in physical sMTJ hardware.
Significance. If the claims are restricted to what is actually demonstrated, this is a valuable contribution: the simulations show that strongly autocorrelated two-level noise, including noise sampled sequentially from a single physical device, can drive DANP training in deep networks without requiring the idealized Gaussian perturbations of the original ANP formulation. The work also provides a useful benchmark for noise-based learning in physical substrates, and it ships code and data (Appendix C), which aids reproducibility. I find no circularity in the learning evaluation: the HMM parameters are fit to device measurements, and the algorithm is compared against backpropagation as an external baseline. The main weakness is the gap between the proposed analog learning circuit and the hardware experiment, which uses the sMTJ only as a noise source for a software network. With the abstract and hardware claims revised to match the evidence, the paper would be a sound and informative step toward noise-based physical learning.
major comments (3)
- [Abstract; §III.D; §II.C] The abstract's statement that learning can be achieved 'in a small network composed of physical stochastic magnetic tunnel junctions' is not supported by the experiments in §III.D. There, the network runs in Python: the Arduino reads a single sMTJ and supplies voltage samples, which are stored in a 200-sample buffer and shuffled before injection; the forward pass, loss, and weight updates all occur in software. The analog circuit in Figure 1 is a proposal described in §II.C, and Section IV explicitly defers embedding the forward pass, decorrelation, and parameter updates in physical hardware to future work. The load-bearing equivalence between Eq. (4) and the proposed analog circuit is therefore untested; unmodeled effects such as the sMTJ's two-level quantization interacting with the summing amplifier, diode nonlinearities, and resistive weight mismatch could alter the effective update. Please revise the claims to state that a single physical sMTJ was used as a noise source for an otherwise simulated DANP network, or provide a physical implementation of the network itself.
- [§III.D; Fig. 5] The hardware proof-of-concept reports decreasing loss over 100 epochs, but Figure 5 shows no baseline condition, no chance-level reference, and no statistical test. Without a control, a reader cannot distinguish learning from a downward drift or from a favorable initialization. I recommend adding a control condition with fixed random noise or with the noise source disconnected, and reporting the distribution of final loss across multiple network initializations.
- [Appendix A] The hyperparameters for the multilayer simulations were selected by sweeping η and ϵ and choosing the pair with the highest test accuracy at 1000 epochs for a single seed. This test-set-based selection makes the reported test curves optimistic and complicates the comparison with backpropagation, because the same selection protocol is not described for the BP baselines. I suggest reporting performance under the same selection rule for BP, or performing selection on a held-out validation split and reporting test performance only for the final configuration.
minor comments (5)
- [§II.A; Appendix A] The loss is defined in Eq. (1) as squared error L = ||y* - y||^2, but Appendix A states that categorical cross-entropy was used for training. Please unify the notation and state the loss used in each experiment.
- [§III.D] The text states that the Arduino Uno has 14-bit resolution on its analog pins; the Arduino Uno's ADC is 10-bit. Please correct this factual detail or specify the exact hardware used.
- [References] Reference [38] is listed as accessed on 2010-09-30, which predates the Telemetrix project; the access date is likely a typo and should be corrected.
- [§III.B; Fig. 3] The text says ANP is robust to 'Gaussian or Binomial distribution sampled noise,' but the panels of Figure 3 are not labeled clearly enough to identify which noise distribution each panel corresponds to. Please add explicit panel labels.
- [§II.C] The sentence 'This is the case for a resistance-based setup, as shown here' is ambiguous; please clarify whether the series connection of sMTJs applies to the voltage-adder circuit in Figure 1 or to the multi-level noise generalization.
Circularity Check
No significant circularity: the DANP learning results are benchmarked against an external backpropagation baseline and reproduced with measured sMTJ noise; the hardware-claim overreach is an evidence gap, not a circular reduction.
full rationale
The paper's derivation chain is not circular. The core update rule (Eq. 4) is DANP/ANP from prior work (Ref. [20]) and the decorrelation rule comes from Ref. [30]; these are self-citations by overlapping authors, but they are not the load-bearing evidence here because the current paper independently re-runs the resulting algorithm on MNIST, CIFAR-10, and CIFAR-100 using both simulated and measured sMTJ noise (Fig. 4) and compares it against an external backpropagation baseline. The HMM parameters in Appendix A (p = 0.0809, means 0.0480 and 0.0362) are fitted to the sMTJ device statistics, not to the learning outcome, so no fitted quantity is renamed as a prediction. The main weakness is that Section III.D's 'experimental validation' runs the network in Python and only reads one sMTJ through an Arduino, while Section II.C's analog circuit is proposed rather than built; the authors explicitly state that a complete CMOS implementation 'is reserved for future work.' That is a mismatch between the abstract's hardware claim and the actual experiment, but it is not a circularity: no equation is defined in terms of the result it is supposed to predict, and no parameter is fitted to the target accuracy. The self-citations are background for the algorithm and are independently confirmed by the paper's own simulations, so the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (6)
- HMM transition probability p =
0.0809
- HMM state means mu1, mu2 =
0.0480, 0.0362
- HMM observation noise sigma =
0.001
- DANP learning rates eta (simulations, per dataset) =
MNIST: 0.1*2^-11; CIFAR10: 0.1*2^-12; CIFAR100: 0.1*2^-12; CIFAR100 measured noise: 0.1*2^-10
- Decorrelation learning rates epsilon (simulations, per dataset) =
MNIST: 0.1*2^-19; CIFAR10: 0.1*2^-19; CIFAR100: 0.1*2^-20; CIFAR100 measured noise: 0.1*2^-14
- Hardware demo learning rates eta and epsilon =
eta=1e-3, epsilon=1e-4
assumptions (5)
- domain assumption ANP and DANP update rules (Eqs. 3 and 4 plus decorrelation) are effective gradient-aligned learning rules
- domain assumption Iterative decorrelation Rl <- Rl - epsilon(...) is local and improves multilayer DANP
- domain assumption Neel-Arrhenius law describes sMTJ switching dwell times
- domain assumption A two-state HMM and symmetric Bernoulli with Poisson dwell times approximate the measured sMTJ noise
- domain assumption Serially feeding one measured sMTJ time series to all nodes is a valid worst-case noise source for DANP
Cite this review
Pith. "Pith review of Noise-based Local Learning using Stochastic Magnetic Tunnel Junctions." pith.science (2026). https://pith.science/paper/PXSEK3TI
@misc{pith2026241212783,
author = {Pith},
title = {Pith review of: Noise-based Local Learning using Stochastic Magnetic Tunnel Junctions},
year = {2026},
howpublished = {\url{https://pith.science/paper/PXSEK3TI}},
note = {Machine review of arXiv:2412.12783}
}
read the original abstract
Brain-inspired learning in physical hardware has enormous potential to learn fast at minimal energy expenditure. One of the characteristics of biological learning systems is their ability to learn in the presence of various noise sources. Inspired by this observation, we introduce a novel noise-based learning approach for physical systems implementing multi-layer neural networks. Simulation results show that our approach allows for effective learning whose performance approaches that of the conventional effective yet energy-costly backpropagation algorithm. Using a spintronics hardware implementation, we demonstrate experimentally that learning can be achieved in a small network composed of physical stochastic magnetic tunnel junctions. These results provide a path towards efficient learning in general physical systems which embraces rather than mitigates the noise inherent in physical devices.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
- [1]
-
[2]
D. Markovic, A. Mizrahi, D. Querlioz, and J. Grollier, Physics for neuromorphic computing, Nat. Rev. Phys. 2, 499 (2020)
work page 2020
-
[3]
B. Scellier and Y. Bengio, Equilibrium propagation: Bridging the gap between energy-based models and backpropagation, Front. comput. neurosci. 11 (2017)
work page 2017
- [4]
-
[5]
M. Nakajima, K. Inoue, K. Tanaka, Y. Kuniyoshi, T. Hashimoto, and K. Nakajima, Physical deep learning with biologically inspired training method: gradient-free approach for physical hardware, Nat. Commun. 13, 1 (2022)
work page 2022
-
[6]
V. L´ opez-Pastor and F. Marquardt, Self-learning machines based on hamiltonian echo back- propagation, Phys. Rev. X 13, 031020 (2023)
work page 2023
- [7]
-
[8]
E. R. W. Van Doremaele, T. Stevens, S. Ringeling, S. Spolaor, M. Fattori, and Y. van de Burgt, Hardware implementation of backpropagation using progressive gradient descent for in 18 situ training of multilayer neural networks, Sci. Adv. 10, 8999 (2024)
work page 2024
Show all 41 references
-
[9]
Kendall, R
J. Kendall, R. Pantone, K. Manickavasagam, Y. Bengio, and B. Scellier, Training end-to-end analog neural networks with equilibrium propagation, arXiv:2006.01981 [cs.NE] (2020)
2020 arXiv
-
[10]
Dillavou, M
S. Dillavou, M. Stern, A. J. Liu, and D. J. Durian, Demonstration of decentralized physics- driven learning, Phys. Rev. Appl. 18, 014040 (2022)
2022
-
[11]
Laborieux and F
A. Laborieux and F. Zenke, Holomorphic equilibrium propagation computes exact gradients through finite size oscillations, in NeurIPS , Vol. 36 (2022) pp. 1–14
2022
-
[12]
V. R. Anisetti, B. Scellier, and J. M. Schwarz, Learning by non-interfering feedback chemical signaling in physical networks, Phys. Rev. Res. 5 (2023)
2023
-
[13]
A. A. Faisal, L. P. J. Selen, and D. M. Wolpert, Noise in the nervous system, Nat. Rev. Neurosci. 9, 292 (2008)
2008
-
[14]
Shimizu, K
G. Shimizu, K. Yoshida, H. Kasai, and T. Toyoizumi, Computational roles of intrinsic synaptic dynamics, Curr. Opin. Neurobiol. 70, 34 (2021)
2021
-
[15]
K. Y. Camsari, R. Faria, B. M. Sutton, and S. Datta, Stochastic p-bits for invertible logic, Phys. Rev. X 7, 031014 (2017)
2017
-
[16]
W. A. Borders, A. Z. Pervaiz, S. Fukami, K. Y. Camsari, H. Ohno, and S. Datta, Integer factorization using stochastic magnetic tunnel junctions, Nature 573, 390 (2019)
2019
-
[17]
Grimaldi, L
A. Grimaldi, L. Mazza, E. Raimondo, P. Tullo, D. Rodrigues, K. Y. Camsari, V. Crupi, M. Carpentieri, V. Puliafito, and G. Finocchio, Evaluating spintronics-compatible implemen- tations of ising machines, Phys. Rev. Appl. 20, 024005 (2023)
2023
-
[18]
J. Si, S. Yang, Y. Cen, J. Chen, Y. Huang, Z. Yao, D. J. Kim, K. Cai, J. Yoo, X. Fong, and H. Yang, Energy-efficient superparamagnetic ising machine and its application to traveling salesman problems, Nat. Commun. 15 (2024)
2024
-
[19]
Kaiser, W
J. Kaiser, W. A. Borders, K. Y. Camsari, S. Fukami, H. Ohno, and S. Datta, Hardware-aware in situ learning based on stochastic magnetic tunnel junctions, Phys. Rev. Appl. 17, 014016 (2022)
2022
-
[20]
S. Dalm, M. van Gerven, and N. Ahmad, Effective learning with node perturbation in deep neural networks, arXiv:2310.00965 [cs.LG] (2023)
2023 arXiv
-
[21]
Dembo and T
A. Dembo and T. Kailath, Model-free distributed learning, IEEE Trans Neural Netw. 1, 58 (1990). 19
1990
-
[22]
Cauwenberghs, A fast stochastic error-descent algorithm for supervised learning and opti- mization, in NeurIPS , Vol
G. Cauwenberghs, A fast stochastic error-descent algorithm for supervised learning and opti- mization, in NeurIPS , Vol. 5 (1992)
1992
-
[23]
Lanza, A
M. Lanza, A. Sebastian, W. D. Lu, M. L. Gallo, M.-F. Chang, D. Akinwande, F. M. Puglisi, H. N. Alshareef, M. Liu, and J. B. Roldan, Memristive technologies for data storage, compu- tation, encryption, and radio-frequency communication, Science 376 (2022)
2022
-
[24]
Grollier, D
J. Grollier, D. Querlioz, K. Camsari, K. Everschor-Sitte, S. Fukami, and M. Stiles, Neuromor- phic spintronics, Nat. Electron. 3, 360 (2020)
2020
-
[25]
Z´ azvorka, F
J. Z´ azvorka, F. Jakobs, D. Heinze, N. Keil, S. Kromin, S. Jaiswal, K. Litzius, G. Jakob, P. Virnau, D. Pinna, K. Everschor-Sitte, L. R´ ozsa, A. Donges, U. Nowak, and M. Kl¨ aui, Thermal skyrmion diffusion used in a reshuffler device, Nat. Nanotechnol. 14, 658 (2019)
2019
-
[26]
K. Raab, M. A. Brems, G. Beneke, T. Dohi, J. Roth¨ orl, F. Kammerbauer, J. H. Mentink, and M. Kl¨ aui, Brownian reservoir computing realized using geometrically confined skyrmion dynamics, Nat. Commun. 13, 6982 (2022)
2022
-
[27]
Kanai, K
S. Kanai, K. Hayakawa, H. Ohno, and S. Fukami, Theory of relaxation time of stochastic nanomagnets, Phys. Rev. B 103, 094423 (2021)
2021
-
[28]
A. G. Baydin, B. A. Pearlmutter, D. Syme, F. Wood, and P. Torr, Gradients without back- propagation, ArXiv:2202.08587 [cs] (2022)
2022 arXiv
-
[29]
M. Ren, S. Kornblith, R. Liao, and G. Hinton, Scaling forward gradient with local losses, in ICLR, Vol. 11 (2023)
2023
-
[30]
Ahmad, E
N. Ahmad, E. Schrader, and M. van Gerven, Constrained parameter inference as a principle for learning, Trans. Mach. Learn. Res. (2023)
2023
-
[31]
Schnitzspan, M
L. Schnitzspan, M. Kl¨ aui, and G. Jakob, Nanosecond true-random-number generation with superparamagnetic tunnel junctions: Identification of joule heating and spin-transfer-torque effects, Phys. Rev. Appl. 20, 024002 (2023)
2023
-
[32]
Hayakawa, S
K. Hayakawa, S. Kanai, T. Funatsu, J. Igarashi, B. Jinnai, W. Borders, H. Ohno, and S. Fukami, Nanosecond random telegraph noise in in-plane magnetic tunnel junctions, Phys. Rev. Lett. 126, 117202 (2021)
2021
-
[33]
Safranski, J
C. Safranski, J. Kaiser, P. Trouilloud, P. Hashemi, G. Hu, and J. Z. Sun, Demonstration of nanosecond operation in stochastic magnetic tunnel junctions, Nano Lett. 21, 2040 (2021)
2021
-
[34]
N´ eel, Th´ eorie du tra ˆ ınage magn´ etique des ferromagn´ etiques en grains fins avec application aux terres cuites, in Ann
L. N´ eel, Th´ eorie du tra ˆ ınage magn´ etique des ferromagn´ etiques en grains fins avec application aux terres cuites, in Ann. Geophys., Vol. 5 (1949) pp. 99–136. 20
1949
-
[35]
Schnitzspan, J
L. Schnitzspan, J. Cramer, J. Kubik, M. Tarequzzaman, G. Jakob, and M. Kl¨ aui, Impact of annealing temperature on tunneling magnetoresistance multilayer stacks, IEEE Magn. Lett. 11, 1 (2020)
2020
-
[36]
Schnitzspan, M
L. Schnitzspan, M. Kl¨ aui, and G. Jakob, Electrical coupling of superparamagnetic tunnel junctions mediated by spin-transfer-torques, Appl. Phys. Lett. 123 (2023)
2023
-
[37]
J. G. Fern´ andez, S. Keemink, and M. van Gerven, Gradient-free training of recurrent neural networks using random perturbations, Front. Neurosci. 18 (2024)
2024
-
[38]
Yorinks, The telemetrix project, https://mryslab.github.io/telemetrix, accessed: 2010-09-30
A. Yorinks, The telemetrix project, https://mryslab.github.io/telemetrix, accessed: 2010-09-30
2010
-
[39]
Belouze, Optimization without backpropagation, ArXiv:2209.06302v1 [cs.LG] (2022)
G. Belouze, Optimization without backpropagation, ArXiv:2209.06302v1 [cs.LG] (2022)
2022 arXiv
-
[40]
Jaeger, B
H. Jaeger, B. Noheda, and W. G. van der Wiel, Toward a formal theory for computing machines made out of whatever physics offers, Nat. Commun. 14, 1 (2023)
2023
-
[41]
Kammerbauer and L
F. Kammerbauer and L. Schnitzspan, Time series sMTJ switching data, 10.5281/zen- odo.15222667 (2025). 21
2025 doi
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.