Time to Spike? Understanding the Representational Power of Spiking Neural Networks in Discrete Time

Adalbert Fono; Duc Anh Nguyen; Ernesto Araya; Gitta Kutyniok

REVIEW 2 major objections 4 minor 79 references

Time to Spike? Understanding the Representational Power of Spiking Neural Networks in Discrete Time

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Discrete-time LIF spiking networks are universal approximators with explicit neuron counts, and their input-space partitions grow quadratically with latency while depth adds no new regions.

desk verdict A genuinely new O(T^2) partitioning bound for discrete-time LIF-SNNs, with a fixable approximation-rate typo and a tightness proof that needs to be completed. read the letter →

arxiv 2505.18023 v2 pith:AFINLK2U submitted 2025-05-23 cs.LG cs.NE

Duc Anh Nguyen , Ernesto Araya , Adalbert Fono , Gitta Kutyniok This is my paper · ORCID

classification cs.LGcs.NE MSC 68T0752C3541A25

keywords spikingneuralnetworksdiscrete-timeLIFexpressivityuniversalapproximationinputspacepartitioninghyperplanearrangementslatencypolyhedralregions

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish a quantitative theory of what discrete-time leaky integrate-and-fire spiking neural networks (LIF-SNNs) can represent when inputs and outputs are static vectors. Its main claims are that these networks realize piecewise-constant functions on polyhedral regions, that two-layer networks with a single time step are universal approximators of continuous functions with explicit neuron counts, and that the number of input-space regions produced by a first-layer neuron grows only quadratically with latency $T$: at most $(T^2+T+2)/2$, despite $2^{T-1}$ possible spike histories at time $T$. A consequence is that depth does not increase the region count, since later layers only merge regions, so latency is the resource that separates these networks from ReLU and Heaviside ANNs. The paper also reports experiments on CIFAR-10 and SVHN that are consistent with the first hidden layer acting as a bottleneck, with larger $T$ raising training accuracy.

What carries the argument

The engine of the argument is the closed-form expression for a single first-layer neuron's spike output, $$s_k(t) = H\Bigl(\langle w_k,x\rangle + b_k + g_{t-1}(s_k(1),\ldots,s_k(t-1))\Bigr),$$ where $g_{t-1}$ is a scalar shift depending only on the neuron's past spikes and on the temporal parameters $u_k(0),\beta,\vartheta$ and $t$. Because the weight vector $w_k$ is shared across time steps, every decision boundary of this neuron is a hyperplane parallel to $\langle w_k,x\rangle+b_k=0$, and the shift only moves that hyperplane along its normal. The proof counts how many of these shifted hyperplanes can actually lie inside the region they are supposed to cut: at transition $t-1\to t$, at most $t$ regions are split, summing to $1+\sum_{t=1}^T t=(T^2+T+2)/2$. The extension to many neurons applies a deletion-restriction count for families of parallel hyperplanes, a variant of Zaslavsky's theorem, and a general-position construction for the tightness claim.

What would settle it

Enumerate, with exact rational arithmetic, all spike histories of a single first-layer LIF neuron for a fine sweep of $\beta,\vartheta,u_k(0)$ and $T=8$: the quadratic bound predicts at most $(8^2+8+2)/2=37$ distinct spike time series over inputs, and tightness predicts that some parameter choice attains 37. Finding 38 or more histories would refute Lemma 4.2; an exhaustive search that never reaches 37 would refute the tightness claim.

Watch

Extended reading notes

Core claim

The central discovery is that the representational complexity of a discrete-time LIF-SNN is fixed already by the first hidden layer together with the latency $T$: each neuron in that layer separates the input space by a family of parallel hyperplanes, and one neuron can create at most $(T^2+T+2)/2$ regions, on each of which its spike time series is constant. Across $n_1$ first-layer neurons and input dimension $n_{\mathrm{in}}$, the maximum number of activation regions is bounded by $\sum_{i=0}^{n_{\mathrm{in}}} ((T^2+T)/2)^i \binom{n_1}{i}$ when $n_1 \ge n_{\mathrm{in}}$, and by $((T^2+T+2)/2)^{n_1}$ when $n_1 < n_{\mathrm{in}}$; the paper claims the bound is attainable by suitable network parameters. The same section proves that adding layers beyond the first cannot create new activation regions, only merge them. On the approximation side, the paper proves that $T=1$ two-layer discrete-time LIF-SNNs realize any step function on a grid of hypercubes, hence approximate every continuous function on a compact set, with explicit width bounds in the Lipschitz case and a matching lower bound on $[0,1]$ for $f(x)=\Gamma x$.

Load-bearing premise

The claim that the upper bound on region counts can actually be attained assumes one can arrange the parallel decision lines of all first-layer neurons so that they never coincide or cross in forbidden ways; the proof gives a translation construction for this but does not check every possible shift coincidence across time steps.

Editorial extensions

If this is right

Increasing latency $T$ in a shallow discrete-time LIF-SNN raises the maximum number of activation regions by a factor quadratic in $T$ per first-layer neuron, whereas increasing depth leaves the region count unchanged.
With $n_1 < n_{\mathrm{in}}$, a first hidden layer of $n_1$ neurons can generate at most $((T^2+T+2)/2)^{n_1}$ regions, so the width of the first layer is an exact ceiling on the number of distinct spike patterns the network can produce.
At $T=1$ the model reduces to a Heaviside ANN, and the paper's approximation results give explicit worst-case-optimal neuron counts for Lipschitz functions, improving the previous quadratic width bound to a linear one.
Because deeper layers only merge activation regions, a network that needs many separable input regions must put width or latency into the first layer; later layers can refine the function on existing regions but cannot create new ones.
The experiments indicate that high latency raises training accuracy mainly through the refined first-layer partition, and that the practical benefit of further increasing $T$ can be limited because the added regions shrink in width.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the quadratic region bound is generically attained, the first-layer geometry is an arrangement of parallel slabs rather than a generic hyperplane arrangement; this suggests the network's effective decision surface is closer to a set of correlated thresholded projections, which may explain why a narrow first layer cannot be rescued by depth.
The empirical observation that shifts can become periodic for leaky parameters below 1 (for example $\beta=0.8$) suggests the theoretical $O(T^2)$ growth may overstate practically realizable region counts at large $T$; one test is to count non-empty regions under random initialization and increasing latency.
The order-optimal approximation lower bound is proven only in one dimension; identifying worst-case Lipschitz targets in higher dimensions would determine whether the linear width scaling of the upper bound is fundamental.
Viewing the membrane potential as a hidden state, the quadratic region bound resembles a reachable-state count for a one-dimensional threshold automaton; this suggests direct comparisons with recurrent networks and state machines that the paper does not draw.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

I've read arXiv:2505.18023 (Nguyen et al.). The genuinely new thing is the O(T^2) bound on input-space regions for discrete-time LIF-SNNs. Each first-layer neuron, despite 2^{T-1} possible spike histories, can only carve its input into (T^2+T+2)/2 regions using parallel hyperplanes. The proof via lexicographic ordering (Lemma B.12) is solid, and the tightness construction with beta=1 and irrational initial potential is convincing. The upper bound in Theorem 4.3, built on the deletion-restriction recursion for families of parallel hyperplanes, is a clean and useful contrast with ReLU ANNs: latency is doing the work that depth does in ReLU nets, and later layers just merge regions.

But there are two soft spots. First, the approximation-rate formulas in Theorem 3.2 and Corollary B.8 have an arithmetic error. With delta = epsilon/Gamma, the number of grid intervals per dimension is diam(Gamma/epsilon), not diam/(epsilon Gamma). As printed, the bound scales inversely with Gamma, which is backwards for approximating a Gamma-Lipschitz function. The proof text in the appendix actually derives the correct scaling, so this is a typo, but it is in the main theorems and it matters.

Second, the tightness of Theorem 4.3 depends on Lemma B.13, and that proof is not complete. The iterative translation construction never verifies that after shifting, every neuron's hyperplanes still lie inside the slabs from Lemma B.12, that the spike-history shifts are pairwise distinct, or that no new hyperplane passes through an intersection of several prior families. Any one of those failures would drop the region count below the claimed bound. The upper bound is safe; the tight Theta(T^2) claim is not fully established. An explicit construction for a small T and n_in, or a more careful degeneracy argument, would settle it.

The experiments are a nice sanity check, but there's no released code and no variance estimates. That's minor for a theory paper.

Overall, the partitioning result is a real contribution and the paper deserves a serious referee. It needs the formula fix and a rigorous Lemma B.13 before the tight claim is usable. I'd send it to review, and I'd probably cite the per-neuron bound even with a note.

Referee Report

2 major / 4 minor

Summary. This paper studies the expressivity of discrete-time LIF spiking neural networks with static inputs, direct encoding, and membrane-potential decoding. The main theoretical claims are: (1) every continuous function on a compact set can be uniformly approximated by a network with T=1 time step and L=2 hidden layers, with explicit width bounds for Lipschitz functions (Theorem 3.2) and a matching lower bound for f(x)=Γx on [0,1] (Proposition 3.3); (2) the first hidden layer partitions the input space into polyhedral regions, and although a naive count allows 2^T regions per neuron, a single neuron creates at most (T^2+T+2)/2 regions (Lemma 4.2), leading to the upper bound in Theorem 4.3 on activation and constant regions in terms of nin, n1, and T; (3) the bound is claimed to be tight under a general-position condition for families of parallel hyperplanes (Lemma B.13); (4) depth does not increase the number of activation regions, in contrast to ReLU ANNs, and this is supported by experiments on CIFAR10 and SVHN and by a toy region-counting experiment. The paper also discusses how the leaky parameter β affects the shift of parallel hyperplanes.

Significance. The paper addresses a timely and under-theorized model class: discrete-time LIF-SNNs, which are the standard SNN implementation framework but are much less analyzed than continuous-time spike-response models. If the results are correct after revision, the main contributions are the explicit width bounds for universal approximation with T=1, the per-neuron O(T^2) region bound in Lemma 4.2/B.12, and the sharp contrast with ReLU ANNs: latency plays a role analogous to width for the first hidden layer, while depth does not increase the number of activation regions. The proofs are largely self-contained and constructive: the identity construction, the step-function realization, and the lexicographic-region argument for the quadratic bound are genuine strengths, as is the use of families of parallel hyperplanes rather than generic arrangements. The experiments illustrate the predicted bottleneck behavior, though they do not by themselves prove the theory. However, the approximation-rate formula in the main theorem and the proof of tightness in Lemma B.13 need correction before the claimed rates can be relied upon.

major comments (2)

[§3.2, Eq. (3); Corollary B.8] The printed width formulas invert the Lipschitz constant. The proof in Corollary B.8 chooses δ = min{ε/Γ, 2K}, so m = ceil(2K/δ)^n = ceil(2KΓ/ε)^n, and hence n1 = (ceil(2KΓ/ε)+1)n and n2 = ceil(2KΓ/ε)^n. The formulas printed as ceil(2K/(εΓ)) have the reciprocal dependence: as Γ decreases (an easier function class), the required width increases, which contradicts the Γ=0 remark just below Theorem 3.2 and the lower bound in Proposition 3.3. Since the stated approximation rates are a central contribution, Eq. (3) and Corollary B.8 must be corrected consistently, with diam∞(Ω)Γ/ε in place of diam∞(Ω)/(εΓ).
[§B.2.3, Lemma B.13] The construction of families in general position is incomplete. The proof only ensures that the hyperplanes of a newly added family avoid the 0-dimensional intersection points of nin previous families; it does not rule out that a new hyperplane contains a lower-dimensional intersection (e.g., in R^3 the line of intersection of two previous hyperplanes), which would violate general position for n1 > nin. It also does not explicitly verify that the translations used to push the family away from previous intersections preserve the per-neuron splitting condition (11) of Lemma B.12 for all k hyperplanes, nor that the bound B on the shifts is compatible with the choices of u_k(0), β, and ϑ needed to attain k distinct active hyperplanes. Because Theorem 4.3's tightness statement rests entirely on this lemma, a complete existence proof (e.g., a genericity argument over the free parameters (w_k, b_k, u_k(0), β_k, ϑ_k)) is required.

minor comments (4)

[§B.2.3, Lemma B.13] The text says 'the shifts (as defined in the proof of Lemma B.11) are bounded,' but the shift term is defined in equation (10) of Lemma B.12, not in Lemma B.11.
[§3.2, Remark after Theorem 3.2] The remark that 'K can be chosen as diam∞(Ω)' is off by a factor of two: for a compact set Ω, the enclosing cube has side length at least diam∞(Ω), so the correct statement is 2K = diam∞(Ω) or K = diam∞(Ω)/2. As written, the constant in Corollary B.8 differs by a factor of two from Eq. (3).
[§B.2.2, Lemma B.12, step 3] The statement that the values Σ a_{t-i}β^i are 'decreasing' with lexicographic order is not literally true for β=1, where all codes with the same number of ones have equal shift. The argument still works because in that case the hyperplanes coincide, but the wording should be adjusted to 'non-increasing' with the equality case handled separately.
[§B.2.4, Theorem B.14] In the proof that |C| = |R|, the identity-mapping construction via Proposition 3.1 only propagates spikes; to ensure that distinct activation patterns produce distinct outputs, the decoder must be injective on {0,1}^{n1×T} (e.g., a_t = 2^{t-1} and V = I). This is straightforward but should be stated explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the region-counting and approximation bounds are proven from the LIF dynamics themselves, and self-citations are contextual only.

full rationale

The paper's central derivation chain is self-contained. Lemma B.12 derives the per-neuron bound (T^2+T+2)/2 directly from the explicit closed form of the spike decision in equation (9), with tightness shown by an explicit parameter choice (beta = 1, theta = 1, suitable u_k(0)), so the bound is not an input to the derivation. Theorem 4.3 combines Lemma B.12 with Lemma B.11, an independent counting argument for families of parallel hyperplanes, and with Lemma B.13, which is an existence construction over the weights and biases; nothing in the theorem is obtained by fitting a parameter to the quantity being predicted. The only self-citations (Singh et al., 2023; Guhring et al., 2020; Bolcskei et al., 2019; Kutyniok et al., 2022) are used for context or comparison in the related-work sections, not as premises of the proofs. The skeptical concern about Lemma B.13 (that the general-position construction does not fully verify all distinctness and non-degeneracy conditions) is a completeness or correctness risk, not a circularity: a failed construction would weaken the tightness claim, but it would not make any theorem equivalent to its own statement or to a fitted value. The paper also explicitly flags, in the Remark after Proposition 3.3, that the multidimensional lower bound is not formally proven; that is a scoping limitation, not a circular step. The experimental sections compare observed region counts or accuracies against the stated theory rather than using the theory to define the outcome, so there is no fitted-input-called-prediction pattern.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claims rely on the LIF discretization, the direct-encoding and membrane-potential decoding choice, and standard counting results for hyperplane arrangements. No parameter fitting is used in the theoretical results; the analysis is over all parameter configurations. The most fragile step is the general-position construction in Lemma B.13, which is asserted by the paper rather than machine-checked.

assumptions (5)

domain assumption Discrete-time LIF dynamics with reset-by-subtraction (Eq. 1) is the model of SNN computation.
The entire analysis is built on this specific discretization with reset-by-subtraction; other reset mechanisms or encodings could change region counts.
domain assumption Direct encoding E(x)(t) = x for all t and membrane potential output decoder (Definition 2.2).
The theoretical results are proven for this coding scheme; the parallel-hyperplane structure relies on the same weight matrix being applied at every time step.
standard math Heaviside step function with H(0) = 1.
Used to define spike activations; closed half-spaces determine polyhedral regions and boundary conventions.
standard math Zaslavsky's arrangement bound and the deletion-restriction recursion.
Invoked in Lemma B.11 to count regions of families of parallel hyperplanes.
standard math Uniform continuity of continuous functions on compact sets.
Used in Lemma B.7 to show that step functions can approximate any continuous function on a compact domain.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Time to Spike? Understanding the Representational Power of Spiking Neural Networks in Discrete Time." pith.science (2026). https://pith.science/paper/AFINLK2U

@misc{pith2026250518023,
  author       = {Pith},
  title        = {Pith review of: Time to Spike? Understanding the Representational Power of Spiking Neural Networks in Discrete Time},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AFINLK2U}},
  note         = {Machine review of arXiv:2505.18023}
}

read the original abstract

Recent years have seen significant progress in developing spiking neural networks (SNNs) as a potential solution to the energy challenges posed by conventional artificial neural networks (ANNs). However, our theoretical understanding of SNNs remains relatively limited compared to the ever-growing body of literature on ANNs. In this paper, we study a discrete-time model of SNNs based on leaky integrate-and-fire (LIF) neurons, referred to as discrete-time LIF-SNNs, a widely used framework that still lacks solid theoretical foundations. We demonstrate that discrete-time LIF-SNNs with static inputs and outputs realize piecewise constant functions defined on polyhedral regions, and more importantly, we quantify the network size required to approximate continuous functions. Moreover, we investigate the impact of latency (number of time steps) and depth (number of layers) on the complexity of the input space partitioning induced by discrete-time LIF-SNNs. Our analysis highlights the importance of latency and contrasts these networks with ANNs employing piecewise linear activation functions. Finally, we present numerical experiments to support our theoretical findings.

Figures

Figures reproduced from arXiv: 2505.18023 by the authors.

**Figure 1.** Each color represents a distinct constant value of R(Φ) in the corresponding region. At t = 2, neurons define parallel hyperplanes, while at ℓ = 2 some regions merge. This section establishes a theoretical framework for understanding input patterns in discrete-time LIF-SNNs. We extend the concepts of activation patterns and regions from 5 [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗

**Figure 2.** presents results for CIFAR10 classification, where the first hidden layer is set to 20 neurons, and the number of (a) Training accuracy achieved by an ANN and SNNs with different numbers of time steps but identical spatial architectures (b) The improvement of training accuracy when increasing the width of the subsequent hidden layers [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 4.** Illustration of the toy dataset consisting of points in [−1, 1]2 from two classes separated by the yellow line and visualized by color. regions often decreases after training, as the model is trying to fit the very simple decision boundary of this toy dataset. A natural extension of our theory will be to prove whether randomly initialized discrete-time LIF-SNNs achieve or approximate the theoretical bound in Theore… view at source ↗

Figures from the paper (5 more)

**Figure 3.** Figure 3: Value of gt−1 for β = 1 and β = 0.8 and ⟨w, x⟩ + b = 0.7. 5.3. Counting regions: pre-training versus post-training We consider a discrete-time LIF SNN with L = 2 and T = 1, 2, and count the number of linear regions before and after training on a linearly separable data…

**Figure 5.** Figure 5: Illustration of the grid and cube partition used in the proof of Lemma B.7. Corollary B.8. Let f be a Γ-Lipschitz continuous function on the hypercube [−K, K] n. Then for all ε > 0 there exists a discrete-time LIF-SNN with T = 1, L = 2 and n1 = max 2K ε Γ , 1 +…

**Figure 6.** Figure 6: P Illustration of the proof. From time step t − 1 to time step t, at most 1 additional region is created in the regions with t−1 i=1 ai = m for any fixed m ∈ {0, . . . , t − 1}. Therefore, at most t regions are additionally created in this time step. of all binary code…

**Figure 7.** Figure 7: Comparison of train accuracies achieved by ANN and SNNs on SVHN dataset with different numbers of time steps. Both types of networks share the same spatial architectures: the first hidden layer is a bottleneck with only 20 neurons and the subsequent layers are progress…

**Figure 8.** Figure 8: Test accuracies achieved by ANN and SNNs on CIFAR10 and SVHN dataset with different numbers of time steps. Both types of networks share the same spatial architectures: the first hidden layer is a bottleneck with only 20 neurons and the subsequent layers are progressive…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

79 extracted references · 73 canonical work pages

[1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
[2]

I., Jantan, A., Omolara, A

Abiodun, O. I., Jantan, A., Omolara, A. E., Dada, K. V., Mohamed, N. A., and Arshad, H. State-of-the-art in artificial neural network applications: A survey. Heliyon, 4 0 (11): 0 e00938, 2018

work page 2018
[3]

Discrete Mathematics of Neural Networks

Anthony, M. Discrete Mathematics of Neural Networks. Society for Industrial and Applied Mathematics, 2001

work page 2001
[4]

Understanding deep neural networks with rectified linear units

Arora, R., Basu, A., Mianjy, P., and Mukherjee, A. Understanding deep neural networks with rectified linear units. In International Conference on Learning Representations, (ICLR) , 2018

work page 2018
[5]

and Baraniuk, R

Balestriero, R. and Baraniuk, R. A spline theory of deep learning. In Proceedings of the International Conference on Machine Learning (ICML), volume 80, pp.\ 374--383, 2018

work page 2018
[6]

Balestriero, R., Cosentino, R., Aazhang, B., and Baraniuk, R. G. The geometry of deep networks: power diagram subdivision. In Advances on Neural Information Processing Systems (NeurIPS), 2019

work page 2019
[7]

Optimal approximation with sparsely connected deep neural networks

B\" o lcskei, H., Grohs, P., Kutyniok, G., and Petersen, P. Optimal approximation with sparsely connected deep neural networks. SIAM Journal on Mathematics of Data Science, 1 0 (1): 0 8--45, 2019

work page 2019
[8]

W., Choudhary, A., Agrawal, A., Billinge, S

Choudhary, K., DeCost, B., Chen, C., Jain, A., Tavazza, F., Cohn, R., Park, C. W., Choudhary, A., Agrawal, A., Billinge, S. J. L., Holm, E., Ong, S. P., and Wolverton, C. Recent advances and applications of deep learning methods in materials science. npj Computational Materials, 8 0 (1): 0 59, Apr 2022

work page 2022

Show all 79 references

[9]

M., Potempa, K., Versari, L., Fischbacher, T., Gesmundo, A., and Alakuijala, J

Comsa, I. M., Potempa, K., Versari, L., Fischbacher, T., Gesmundo, A., and Alakuijala, J. Temporal coding in spiking neural networks with alpha synaptic function. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 8529--8533, 2020

2020
[10]

Cybenko, G. V. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 2: 0 303--314, 1989

1989
[11]

Are SNNs really more energy-efficient than ANNs ? A n in-depth hardware-aware study

Dampfhoffer, M., Mesquida, T., Valentian, A., and Anghel, L. Are SNNs really more energy-efficient than ANNs ? A n in-depth hardware-aware study. IEEE Transactions on Emerging Topics in Computational Intelligence IEEE, 7 0 (3): 0 731--741, 2023

2023
[12]

K., Ward, M., Neftci, E

Eshraghian, J. K., Ward, M., Neftci, E. O., Wang, X., Lenz, G., Dwivedi, G., Bennamoun, M., Jeong, D. S., and Lu, W. D. Training spiking neural networks using lessons from deep learning. Proceedings of the IEEE, 111 0 (9): 0 1016--1054, 2023

2023
[13]

Incorporating learnable membrane time constant to enhance learning of spiking neural networks

Fang, W., Yu, Z., Chen, Y., Masquelier, T., Huang, T., and Tian, Y. Incorporating learnable membrane time constant to enhance learning of spiking neural networks. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 2641--2651, 2021

2021
[14]

Spikingjelly: An open-source machine learning infrastructure platform for spike-based intelligence

Fang, W., Chen, Y., Ding, J., Yu, Z., Masquelier, T., Chen, D., Huang, L., Zhou, H., Li, G., and Tian, Y. Spikingjelly: An open-source machine learning infrastructure platform for spike-based intelligence. Science Advances, 9 0 (40): 0 eadi1480, 2023 a

2023
[15]

Parallel spiking neurons with high efficiency and ability to learn long-term dependencies

Fang, W., Yu, Z., Zhou, Z., Chen, D., Chen, Y., Ma, Z., Masquelier, T., and Tian, Y. Parallel spiking neurons with high efficiency and ability to learn long-term dependencies. In Conference on Neural Information Processing Systems (NeurIPS), 2023 b

2023
[16]

and van Hemmen, J

Gerstner, W. and van Hemmen, J. L. Associative memory in a network of ‘spiking’ neurons. Network: Computation in Neural Systems, 3 0 (2): 0 139--164, 1992

1992
[17]

M., Naud, R., and Paninski, L

Gerstner, W., Kistler, W. M., Naud, R., and Paninski, L. Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition. Cambridge University Press, USA, 2014

2014
[18]

A., Huang, J., Kelber, F., Nazeer, K

Gonzalez, H. A., Huang, J., Kelber, F., Nazeer, K. K., Langer, T. H., Liu, C., Lohrmann, M. A., Rostami, A., Sch \"o ne, M., Vogginger, B., Wunderlich, T., Yan, Y., Akl, M., and Mayr, C. Spi NN aker2: A large-scale neuromorphic system for event-based and asynchronous machine l...

2023
[19]

Grigsby, J. E. and Lindsey, K. On transversality of bent hyperplane arrangements and the topological expressiveness of relu neural networks. SIAM Journal on Applied Algebra and Geometry, 6 0 (2): 0 216--242, 2022

2022
[20]

Error bounds for approximations with deep R e LU neural networks in W^ s,p norms

G\" u hring, I., Kutyniok, G., and Petersen, P. Error bounds for approximations with deep R e LU neural networks in W^ s,p norms. Analysis and Applications, 18 0 (05): 0 803--859, 2020

2020
[21]

Direct learning-based deep spiking neural networks: a review

Guo, Y., Huang, X., and Ma, Z. Direct learning-based deep spiking neural networks: a review. Frontiers in Neuroscience, Volume 17 - 2023, 2023

2023
[22]

Fast and energy-efficient neuromorphic deep learning with first-spike times

Göltz, J., Kriener, L., Baumbach, A., Billaudelle, S., Breitwieser, O., Cramer, B., Dold, D., Kungl, A., Senn, W., Schemmel, J., Meier, K., and Petrovici, M. Fast and energy-efficient neuromorphic deep learning with first-spike times. Nature Machine Intelligence, 3: 0 823--835, 2021

2021
[23]

Universal function approximation by deep neural nets with bounded width and R e LU activations

Hanin, B. Universal function approximation by deep neural nets with bounded width and R e LU activations. Mathematics, 7 0 (10), 2019

2019
[24]

and Rolnick, D

Hanin, B. and Rolnick, D. Complexity of linear regions in deep networks. In Proceedings of the International Conference on Machine Learning (ICML), volume 97, pp.\ 2596--2604, 2019 a

2019
[25]

and Rolnick, D

Hanin, B. and Rolnick, D. Deep relu networks have surprisingly few activation patterns. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, 2019 b

2019
[26]

and Jones, M

Har-Peled, S. and Jones, M. On separating points by lines. In Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA '18, pp.\ 918–932. Society for Industrial and Applied Mathematics, 2018. ISBN 9781611975031

2018
[27]

Delving deep into rectifiers: Surpassing human-level performance on imagenet classification

He, K., Zhang, X., Ren, S., and Sun, J. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), ICCV '15, pp.\ 1026–1034, USA, 2015. IEEE Computer Society. ...

2015 doi
[28]

K., and Wessels, H

Henkes, A., Eshraghian, J. K., and Wessels, H. Spiking neural networks for nonlinear regression. Royal Society Open Science, 11 0 (5): 0 231606, 2024

2024
[29]

Multilayer feedforward networks are universal approximators

Hornik, K., Stinchcombe, M., and White, H. Multilayer feedforward networks are universal approximators. Neural Networks, 2 0 (5): 0 359--366, 1989

1989
[30]

When deep learning meets polyhedral theory: A survey

Huchette, J., Muñoz, G., Serra, T., and Tsay, C. When deep learning meets polyhedral theory: A survey. arXiv:2305.00241, 2023

2023 arXiv
[31]

I., Balestriero, R., and Baraniuk, R

Humayun, A. I., Balestriero, R., and Baraniuk, R. Deep networks always grok and here is why. In High-dimensional Learning Dynamics 2024: The Emergence of Structure and Reasoning, 2024

2024
[32]

Mathematical introduction to deep learning: Methods, implementations, and theory

Jentzen, A., Kuckuck, B., and von Wurstemberger, P. Mathematical introduction to deep learning: Methods, implementations, and theory. arXiv:2310.20360, 2023

2023 arXiv
[33]

Neural networks with linear threshold activations: structure and algorithms

Khalife, S., Cheng, H., and Basu, A. Neural networks with linear threshold activations: structure and algorithms. Mathematical Programming, 206 0 (1): 0 333--356, 2024

2024
[34]

Neural architecture search for spiking neural networks

Kim, Y., Li, Y., Park, H., Venkatesha, Y., and Panda, P. Neural architecture search for spiking neural networks. In Proceedings of the European Conference on Computer Vision(ECCV), pp.\ 36–56, 2022

2022
[35]

Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In International Conference on Learning Representations, ICLR 2015, , 2015

2015
[36]

A theoretical analysis of deep neural networks and parametric pdes

Kutyniok, G., Petersen, P., Raslan, M., and Schneider, R. A theoretical analysis of deep neural networks and parametric pdes. Constructive Approximation, 55 0 (1): 0 73--125, 2022

2022
[37]

H., Delbruck, T., and Pfeiffer, M

Lee, J. H., Delbruck, T., and Pfeiffer, M. Training deep spiking neural networks using backpropagation. Frontiers in Neuroscience, 10, 2016

2016
[38]

An analytical estimation of spiking neural networks energy efficiency

Lemaire, E., Cordone, L., Castagnetti, A., Novac, P.-E., Courtois, J., and Miramond, B. An analytical estimation of spiking neural networks energy efficiency. In Neural Information Processing: ICONIP 2022, pp.\ 574--587. Springer International Publishing, 2023

2022
[39]

Y., Pinkus, A., and Schocken, S

Leshno, M., Lin, V. Y., Pinkus, A., and Schocken, S. Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Networks, 6 0 (6): 0 861--867, 1993

1993
[40]

The expressive power of neural networks: a view from the width

Lu, Z., Pu, H., Wang, F., Hu, Z., and Wang, L. The expressive power of neural networks: a view from the width. In Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS), pp.\ 6232–6240, 2017

2017
[41]

Efficient and effective time-series forecasting with spiking neural networks

Lv, C., Wang, Y., Han, D., Zheng, X., Huang, X., and Li, D. Efficient and effective time-series forecasting with spiking neural networks. arXiv:2402.01533, 2024

2024 arXiv
[42]

On the computational complexity of networks of spiking neurons

Maass, W. On the computational complexity of networks of spiking neurons. In Advances in Neural Information Processing Systems (NeurIPS), volume 7, 1994

1994
[43]

On the computational power of noisy spiking neurons

Maass, W. On the computational power of noisy spiking neurons. In Advances in Neural Information Processing Systems, volume 8, 1995

1995
[44]

Lower bounds for the computational power of networks of spiking neurons

Maass, W. Lower bounds for the computational power of networks of spiking neurons. Neural Computation, 8 0 (1): 0 1--40, 1996 a

1996
[45]

Noisy spiking neurons with temporal coding have more computational power than sigmoidal neurons

Maass, W. Noisy spiking neurons with temporal coding have more computational power than sigmoidal neurons. In Advances in Neural Information Processing Systems, volume 9, 1996 b

1996
[46]

Networks of spiking neurons: The third generation of neural network models

Maass, W. Networks of spiking neurons: The third generation of neural network models. Neural Networks, 10 0 (9): 0 1659--1671, 1997 a

1997
[47]

Fast sigmoidal networks via spiking neurons

Maass, W. Fast sigmoidal networks via spiking neurons. Neural Computation, 9 0 (2): 0 279--304, 02 1997 b

1997
[48]

G., Chawla, N., Desoli, G., Malavena, G., Monzio Compagnoni, C., Wang, Z., Yang, J

Mehonic, A., Ielmini, D., Roy, K., Mutlu, O., Kvatinsky, S., Serrano-Gotarredona, T., Linares-Barranco, B., Spiga, S., Savel’ev, S., Balanov, A. G., Chawla, N., Desoli, G., Malavena, G., Monzio Compagnoni, C., Wang, Z., Yang, J. J., Sarwat, S. G., Sebastian, A., Mikolajick, T....

2024
[49]

F., Pascanu, R., Cho, K., and Bengio, Y

Mont \' u far, G. F., Pascanu, R., Cho, K., and Bengio, Y. On the number of linear regions of deep neural networks. In Advances in Neural Information Processing Systems (NeuRIPS), volume 27, 2014

2014
[50]

Supervised learning based on temporal coding in spiking neural networks

Mostafa, H. Supervised learning based on temporal coding in spiking neural networks. IEEE Transactions on Neural Networks and Learning Systems, 29 0 (7): 0 3227--3235, 2018

2018
[51]

O., Mostafa, H., and Zenke, F

Neftci, E. O., Mostafa, H., and Zenke, F. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks. IEEE Signal Processing Magazine, 36 0 (6): 0 51--63, 2019

2019
[52]

Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., and Ng, A. Y. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011

2011
[53]

M., Dold, D., and Petersen, P

Neuman, A. M., Dold, D., and Petersen, P. C. Stable learning using spiking neural networks equipped with affine encoders and decoders. arXiv:2404.04549, 2024

2024 arXiv
[54]

P., Rubin, D

Orchard, G., Frady, E. P., Rubin, D. B. D., Sanborn, S., Shrestha, S. B., Sommer, F. T., and Davies, M. Efficient neuromorphic signal processing with loihi 2. In 2021 IEEE Workshop on Signal Processing Systems (SiPS), pp.\ 254--259, 2021

2021
[55]

On the number of inference regions of deep feed forward networks with piece-wise linear activations

Pascanu, R., Mont \' u far, G., and Bengio, Y. On the number of inference regions of deep feed forward networks with piece-wise linear activations. In International Conference on Learning Representations, (ICLR) 2014 , 2014

2014
[56]

and Montúfar, G

Patel, N. and Montúfar, G. On the local complexity of linear regions in deep relu networks. arXiv:2412.18283, 2024

2024 arXiv
[57]

and Voigtlaender, F

Petersen, P. and Voigtlaender, F. Optimal approximation of piecewise smooth functions using deep relu neural networks. Neural Networks, 108: 0 296--330, 2018

2018
[58]

and Zech, J

Petersen, P. and Zech, J. Mathematical theory of deep learning. arXiv:2407.18384, 2024

2024
[59]

On the expressive power of deep neural networks

Raghu, M., Poole, B., Kleinberg, J., Ganguli, S., and Sohl-Dickstein, J. On the expressive power of deep neural networks. In Proceedings of the International Conference on Machine Learning (ICML), volume 70, pp.\ 2847--2854, 2017

2017
[60]

and Roy, K

Rathi, N. and Roy, K. Diet-snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization. IEEE Transactions on Neural Networks and Learning Systems, 34 0 (6): 0 3174--3182, 2023

2023
[61]

Exploring neuromorphic computing based on spiking neural networks: Algorithms to hardware

Rathi, N., Chakraborty, I., Kosta, A., Sengupta, A., Ankit, A., Panda, P., and Roy, K. Exploring neuromorphic computing based on spiking neural networks: Algorithms to hardware. ACM Comput. Surv., 55 0 (12), 2023

2023
[62]

Conversion of continuous-valued deep networks to efficient event-driven networks for image classification

Rueckauer, B., Lungu, I.-A., Hu, Y., Pfeiffer, M., and Liu, S.-C. Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Frontiers in Neuroscience, 11, 2017

2017
[63]

Sarker, I. H. Deep learning: A comprehensive overview on techniques, taxonomy, applications and research directions. SN Computer Science, 2 0 (6): 0 420, Aug 2021

2021
[64]

Bounding and counting linear regions of deep neural networks

Serra, T., Tjandraatmadja, C., and Ramalingam, S. Bounding and counting linear regions of deep neural networks. In Proceedings of the International Conference on Machine Learning (ICML), volume 80, pp.\ 4558--4566, 2018

2018
[65]

Rethinking the membrane dynamics and optimization objectives of spiking neural networks

Shen, H., Zheng, Q., Wang, H., and Pan, G. Rethinking the membrane dynamics and optimization objectives of spiking neural networks. In The Annual Conference on Neural Information Processing Systems (NeuRIPS), 2024

2024
[66]

Deep network approximation characterized by number of neurons

Shen, Z., Yang, H., and Zhang, S. Deep network approximation characterized by number of neurons. Communications in Computational Physics, 28 0 (5): 0 1768--1811, 2020

2020
[67]

Expressivity of spiking neural networks through the spike response model

Singh, M., Fono, A., and Kutyniok, G. Expressivity of spiking neural networks through the spike response model. In UniReps: the First Workshop on Unifying Representations in Neural Models, 2023

2023
[68]

Stanley, R. P. Enumerative Combinatorics: Volume 1. Cambridge University Press, USA, 2nd edition, 2011

2011
[69]

High-performance deep spiking neural networks with 0.3 spikes per neuron

Stanojevic, A., Wo \' z niak, S., Bellec, G., Cherubini, G., Pantazi, A., and Gerstner, W. High-performance deep spiking neural networks with 0.3 spikes per neuron. Nature Communications, 15 0 (1): 0 6793, 2024

2024
[70]

Benefits of depth in neural networks

Telgarsky, M. Benefits of depth in neural networks. In Annual Conference on Learning Theory, volume 49, pp.\ 1517--1539, 2016

2016
[71]

C., Greenewald, K., Lee, K., and Manso, G

Thompson, N. C., Greenewald, K., Lee, K., and Manso, G. F. Deep learning's diminishing returns: The cost of improvement is becoming unsustainable. IEEE Spectrum, 58 0 (10): 0 50--55, 2021

2021
[72]

Direct training for spiking neural networks: Faster, larger, better

Wu, Y., Deng, L., Li, G., Zhu, J., Xie, Y., and Shi, L. Direct training for spiking neural networks: Faster, larger, better. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp.\ 1311--1318, 2019

2019
[73]

Spiking neural networks and their applications: A review

Yamazaki, K., Vo-Ho, V.-K., Bulsara, D., and Le, N. Spiking neural networks and their applications: A review. Brain Sciences, 12 0 (7), 2022

2022
[74]

Error bounds for approximations with deep relu networks

Yarotsky, D. Error bounds for approximations with deep relu networks. Neural Networks, 94: 0 103--114, 2017

2017
[75]

J., Li, G., Xiao, Z., Jing, Z., Yang, K., Liu, C., Ge, C., Huang, R., and Yang, Y

Yuan, R., Duan, Q., Tiw, P. J., Li, G., Xiao, Z., Jing, Z., Yang, K., Liu, C., Ge, C., Huang, R., and Yang, Y. A calibratable sensory neuron based on epitaxial VO 2 for spike-based neuromorphic multisensory system. Nature Communications, 13, 2022

2022
[76]

Facing up to arrangements: Face-count formulas for partitions of space by hyperplanes

Zaslavsky, T. Facing up to arrangements: Face-count formulas for partitions of space by hyperplanes. Number 154 in Memoirs of the American Mathematical Society. American Mathematical Society, Providence, RI, 1975

1975
[77]

and Zhou, Z.-H

Zhang, S.-Q. and Zhou, Z.-H. Theoretically provable spiking neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2022

2022
[78]

On the intrinsic structures of spiking neural networks

Zhang, S.-Q., Chen, J.-Y., Wu, J.-H., Zhang, G., Xiong, H., Gu, B., and Zhou, Z.-H. On the intrinsic structures of spiking neural networks. Journal of Machine Learning Research, 25 0 (194): 0 1--74, 2024

2024
[79]

Universality of deep convolutional neural networks

Zhou, D.-X. Universality of deep convolutional neural networks. Applied and Computational Harmonic Analysis, 48 0 (2): 0 787--794, 2020

2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

[2] [2]

I., Jantan, A., Omolara, A

Abiodun, O. I., Jantan, A., Omolara, A. E., Dada, K. V., Mohamed, N. A., and Arshad, H. State-of-the-art in artificial neural network applications: A survey. Heliyon, 4 0 (11): 0 e00938, 2018

work page 2018

[3] [3]

Discrete Mathematics of Neural Networks

Anthony, M. Discrete Mathematics of Neural Networks. Society for Industrial and Applied Mathematics, 2001

work page 2001

[4] [4]

Understanding deep neural networks with rectified linear units

Arora, R., Basu, A., Mianjy, P., and Mukherjee, A. Understanding deep neural networks with rectified linear units. In International Conference on Learning Representations, (ICLR) , 2018

work page 2018

[5] [5]

and Baraniuk, R

Balestriero, R. and Baraniuk, R. A spline theory of deep learning. In Proceedings of the International Conference on Machine Learning (ICML), volume 80, pp.\ 374--383, 2018

work page 2018

[6] [6]

Balestriero, R., Cosentino, R., Aazhang, B., and Baraniuk, R. G. The geometry of deep networks: power diagram subdivision. In Advances on Neural Information Processing Systems (NeurIPS), 2019

work page 2019

[7] [7]

Optimal approximation with sparsely connected deep neural networks

B\" o lcskei, H., Grohs, P., Kutyniok, G., and Petersen, P. Optimal approximation with sparsely connected deep neural networks. SIAM Journal on Mathematics of Data Science, 1 0 (1): 0 8--45, 2019

work page 2019

[8] [8]

W., Choudhary, A., Agrawal, A., Billinge, S

Choudhary, K., DeCost, B., Chen, C., Jain, A., Tavazza, F., Cohn, R., Park, C. W., Choudhary, A., Agrawal, A., Billinge, S. J. L., Holm, E., Ong, S. P., and Wolverton, C. Recent advances and applications of deep learning methods in materials science. npj Computational Materials, 8 0 (1): 0 59, Apr 2022

work page 2022