RECORD
Deep learning incorporating biologically-inspired neural dynamics
Neural and Evolutionary Computing · Submitted 17 Dec 2018
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Thus, they open a new avenue for a widespread adoption of SNNs in practical applications.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Neural networks have become the key technology of artificial intelligence and have contributed to breakthroughs in several machine learning tasks, primarily owing to advances in deep learning applied to Artificial Neural Networks (ANNs). Simultaneously, Spiking Neural Networks (SNNs) incorporating biologically-feasible spiking neurons have held great promise because of their rich temporal dynamics and high-power efficiency. However, the developments in SNNs were proceeding separately from those in ANNs, effectively limiting the adoption of deep learning research insights. Here we show an alternative perspective on the spiking neuron that casts it as a particular ANN construct called Spiking Neural Unit (SNU), and a soft SNU (sSNU) variant that generalizes its dynamics to a novel recurrent ANN unit. SNUs bridge the biologically-inspired SNNs with ANNs and provide a methodology for seamless inclusion of spiking neurons in deep learning architectures. Furthermore, SNU enables highly-efficient in-memory acceleration of SNNs trained with backpropagation through time, implemented with the hardware in-the-loop. We apply SNUs to tasks ranging from hand-written digit recognition, language modelling, to music prediction. We obtain accuracy comparable to, or better than, that of state-of-the-art ANNs, and we experimentally verify the efficacy of the in-memory-based SNN realization for the music-prediction task using 52,800 phase-change memory devices. The new generation of neural units introduced in this paper incorporate biologically-inspired neural dynamics in deep learning. In addition, they provide a systematic methodology for training neuromorphic computing hardware. Thus, they open a new avenue for a widespread adoption of SNNs in practical applications.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Neural networks have become the key technology of artificial intelligence and have contributed to breakthroughs in several machine learning tasks, primarily owing to advances in deep learning applied to Artificial Neural Networks (ANNs). | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s1 | Simultaneously, Spiking Neural Networks (SNNs) incorporating biologically-feasible spiking neurons have held great promise because of their rich temporal dynamics and high-power efficiency. | background 1.00 | 0.04 | 0.03 | asserted 0.81 |
| s2 | However, the developments in SNNs were proceeding separately from those in ANNs, effectively limiting the adoption of deep learning research insights. | gap 0.89 | 0.06 | 0.03 | asserted 0.99 |
| s3 | Here we show an alternative perspective on the spiking neuron that casts it as a particular ANN construct called Spiking Neural Unit (SNU), and a soft SNU (sSNU) variant that generalizes its dynamics to a novel recurrent ANN unit. | contribution 0.97 | 0.93 | 0.19 | asserted 1.00 |
| s4 | SNUs bridge the biologically-inspired SNNs with ANNs and provide a methodology for seamless inclusion of spiking neurons in deep learning architectures. | contribution 0.79 | 0.70 | 0.30 | asserted 1.00 |
| s5 | Furthermore, SNU enables highly-efficient in-memory acceleration of SNNs trained with backpropagation through time, implemented with the hardware in-the-loop. | contribution 0.78 | 0.85 | 0.09 | asserted 0.99 |
| s6 | We apply SNUs to tasks ranging from hand-written digit recognition, language modelling, to music prediction. | contribution 0.63 | 0.95 | 0.05 | asserted 0.99 |
| s7 | We obtain accuracy comparable to, or better than, that of state-of-the-art ANNs, and we experimentally verify the efficacy of the in-memory-based SNN realization for the music-prediction task using 52,800 phase-change memory devices. | result 1.00 | 0.97 | 0.22 | asserted 1.00 |
| s8 | The new generation of neural units introduced in this paper incorporate biologically-inspired neural dynamics in deep learning. | contribution 0.95 | 0.80 | 0.41 | asserted 1.00 |
| s9 | In addition, they provide a systematic methodology for training neuromorphic computing hardware. | contribution 0.92 | 0.75 | 0.11 | asserted 1.00 |
| s10 | Thus, they open a new avenue for a widespread adoption of SNNs in practical applications. | interpretation 0.98 | 0.28 | 0.45 | asserted 0.84 |
What was examined, and what was not
- arXiv listing abstract v2
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Neural networks have become the key technology of artificial intelligence and have contributed to breakthroughs in several machine learning tasks, primarily owing to advances in deep learning applied to Artificial Neural Networks (ANNs). Simultaneously, Spiking Neural Networks (SNNs) incorporating biologically-feasible spiking neurons have held great promise because of their rich temporal dynamics and high-power efficiency. However, the developments in SNNs were proceeding separately from those in ANNs, effectively limiting the adoption of deep learning research insights. Here we show an alternative perspective on the spiking neuron that casts it as a particular ANN construct called Spiking Neural Unit (SNU), and a soft SNU (sSNU) variant that generalizes its dynamics to a novel recurrent ANN unit. SNUs bridge the biologically-inspired SNNs with ANNs and provide a methodology for seamless inclusion of spiking neurons in deep learning architectures. Furthermore, SNU enables highly-efficient in-memory acceleration of SNNs trained with backpropagation through time, implemented with the hardware in-the-loop. We apply SNUs to tasks ranging from hand-written digit recognition, language modelling, to music prediction. We obtain accuracy comparable to, or better than, that of state-of-the-art ANNs, and we experimentally verify the efficacy of the in-memory-based SNN realization for the music-prediction task using 52,800 phase-change memory devices. The new generation of neural units introduced in this paper incorporate biologically-inspired neural dynamics in deep learning. In addition, they provide a systematic methodology for training neuromorphic computing hardware. Thus, they open a new avenue for a widespread adoption of SNNs in practical applications.
Human review
How the literature uses it
6 arXiv papers cite this in 10 sentences
Papers citing it · 8 sentences without a confident label
Uses its method or tool 1 paper
-
To investigate system performance and possible trade-offs with complexity of implementation, we consider the following CTE-based SNN models and learning techniques: * System 1: one-layer SNN with unsupervised or supervised STDP learning; * System 2: one and two-layer SNNs, with probabilistic backpropagation [ref] ; * System 3: multi-layer SNNs, with determininstic backpropagation [ref] .
File Classification Based on Spiking Neural Networks · Implemented Models · 0.99
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.