REVIEW 3 major objections 5 minor 15 references
Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read By rewriting LIF dynamics as a fixed point over all timesteps, the paper claims SNN training can run in about three parallel iterations instead of T sequential ones, matching BPTT accuracy while leaving the architecture untouched.
desk verdict The fixed-point framework is a genuinely neat unification of PSN/PSU, but the 'constant time' claim doesn't hold up—per-iteration cost grows with T. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the vectorized fixed-point map $\hat\Phi_\alpha(u) = -V_{\mathrm{th}}(\Lambda - I)S_\alpha(u - V_{\mathrm{th}}) + \Lambda c$, built from the lower-triangular Toeplitz decay matrix $\Lambda$ whose $(i,j)$ entry is $\lambda^{i-j}$ for $i \ge j$ and zero otherwise; this matrix encodes how inputs decay into every later timestep. Writing the membrane potential as a $T$-vector $u$ turns the chronological LIF recurrence into a single simultaneous equation, so each application of $\hat\Phi_\alpha$ updates all timesteps in parallel, and the contraction condition $V_{\mathrm{th}} L_\alpha \lVert \Lambda - I \rVert_\infty < 1$, with $\lVert \Lambda - I \rVert_\infty = \lambda(1-\lambda^{T-1})/(1-\lambda)$, guarantees convergence to a unique fixed point. The split steepness parameters, forward $\alpha_f$ and backward $\alpha_b = \alpha_f/3$, keep the forward trajectory close to the true Heaviside dynamics while giving the backward pass a smooth surrogate gradient through $K$ unrolled iterations, avoiding the Jacobian inversion of deep equilibrium models. With a learnable decay matrix $A$ and threshold $B$, the same map yields PSN at $K=1$ and PSU at $K=2$ as truncated special cases.
What would settle it
Run FPT's forward pass with $K=3$ fixed on one GPU and measure per-batch time at $T=8$, $32$, $128$, $512$, and $2048$, holding the network and batch size constant; the paper's own Table 3 places the curve at 1.55 ms ($T=8$), 2.01 ms ($T=32$), and 5.58 ms ($T=128$), so a rising curve — or a direct operation count showing the $\Lambda c$ product grows with $T$ — would settle that the complexity is not constant in $T$ as literally claimed.
Extended reading notes
Core claim
FPT's central discovery is that the LIF dynamics $u_t = \lambda(u_{t-1} - V_{\mathrm{th}} s_{t-1}) + c_t$, $s_t = H(u_t - V_{\mathrm{th}})$ across $T$ timesteps can be re-expressed exactly as one simultaneous system in vector form: $u = -V_{\mathrm{th}}(\Lambda - I)s + \Lambda c$, $s = H(u - V_{\mathrm{th}})$, where $\Lambda$ is the lower-triangular matrix of exponential decay weights $\lambda^{i-j}$. Replacing the Heaviside step $H$ with a sigmoid surrogate $S_\alpha$ makes the induced map $\hat\Phi_\alpha(u) = -V_{\mathrm{th}}(\Lambda - I)S_\alpha(u - V_{\mathrm{th}}) + \Lambda c$ a contraction under the 1-norm whenever $V_{\mathrm{th}} L_\alpha \lambda(1-\lambda^{T-1})/(1-\lambda) < 1$, so $K$ iterations — $K=3$ in practice — reach a unique fixed point whose spike outputs reproduce the original neuron's trajectory. Splitting the surrogate steepness, with a large $\alpha_f$ in the forward pass and a smoother $\alpha_b = \alpha_f/3$ in the backward pass, keeps the forward map faithful while stabilizing gradients, and training needs no architecture change. The paper further shows that PSN ($K=1$) and PSU ($K=2$) are special cases of this same iterated map for a learnable variant, and that the preserved reset mechanism — absent from those models — is what keeps FPT accuracy at LIF levels. The claimed time complexity is $O(K)$, versus $O(T)$ for BPTT, because each iteration updates all $T$ timesteps simultaneously.
Load-bearing premise
The paper's constant-time claim counts each full pass over all $T$ timesteps as a single unit of work, even though that pass multiplies by the $T$-by-$T$ decay matrix $\Lambda$; the cost of forming $\Lambda c$ and $(\Lambda - I)s$ is never analyzed, so the $O(K)$ headline holds only if that matrix-vector cost is ignored.
Editorial extensions
If this is right
- Training cost decouples from sequence length: FPT reports per-batch training about 10 times faster than BPTT at $T=32$–$128$ and about 21 times faster at $T=512$, with inference time nearly constant in $T$.
- Accuracy matches full BPTT: t-tests on the Amazon Photos/DRSGNN benchmark give $p > 0.05$ at $T=8$, $32$, and $128$, while FPT beats the LocalZO baseline by over 3 accuracy points on DVS-CIFAR10 and by about 1.7 points on ImageNet-100.
- Because the converged forward dynamics equal the original LIF dynamics, a model trained with FPT can be deployed unchanged on sequential neuromorphic hardware, and the paper argues this makes FPT a practical pretraining scheme for long-sequence SNN tasks.
- PSN and PSU are revealed as one- and two-step truncations of a single iterated map; their known accuracy gap relative to LIF is attributable to the missing iterations and the dropped reset mechanism, both of which FPT retains.
- Three iterations suffice across all reported settings: increasing to $K=4$ or $K=5$ leaves DVS-Gesture accuracy unchanged, and non-learnable FPT with $K=3$ reproduces ordinary LIF accuracy on sequential CIFAR10 (81.54 vs 81.50).
Reading between the lines
- A literal reading of the $O(K)$ claim counts only the number of iteration rounds: each round computes the $T$-vector $-V_{\mathrm{th}}(\Lambda - I)s + \Lambda c$, and producing any $T$-dimensional output costs $\Omega(T)$ work — $O(T)$ work and $O(\log T)$ parallel depth with a scan, or $O(T \log T)$ with an FFT — so the honest complexity is closer to $O(K \log T)$ parallel depth and $\Omega(KT)$
- Because $\Lambda$ is Toeplitz, $\Lambda c$ is a causal exponential filter — effectively a parallel prefix or convolution — so FPT's remaining $T$-dependence can be delegated to a purpose-built scan or FFT kernel; the method's true complexity would then be that of the convolution, not of the fixed-point iterations.
- Since the fixed point is unique and contractive, a deep-equilibrium-style rule that differentiates the equilibrium equation directly is a natural alternative to unrolling $K$ iterations; the paper chooses the unroll because $K$ is small, and that trade-off would flip for larger $K$, steeper $\alpha_f$, or cheaper Jacobian solvers.
- The same vectorization should transfer to any contractive leaky recurrence $u_t = \lambda g(u_{t-1}) + c_t$ with a Lipschitz $g$; the paper treats only the LIF/Heaviside pair, but the fixed-point template suggests a route to parallelizing other gated recurrent cells that have a reset.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes Fixed-point Parallel Training (FPT), a method that rewrites the LIF recurrence over T timesteps as a fixed-point equation u = -Vth(Λ-I)s + Λc with s = S_α(u - Vth), and solves it by K fixed-point iterations so that all timesteps can be updated simultaneously. The backward pass backpropagates through the K iterations using surrogate gradients. The authors claim O(K) training time, prove convergence of the surrogate fixed-point map, identify PSN and PSU as K=1 and K=2 special cases, and report competitive accuracy and large GPU speedups on DVS-CIFAR10, DVS-Gesture, ImageNet-100, sequential CIFAR10/CIFAR100, and an Amazon Photos graph task.
Significance. If the headline complexity claim were supported, FPT would be a valuable architecture-agnostic SNN training method because it preserves the reset mechanism and can be applied without modifying the network. The empirical part is careful in several respects: results are averaged over three runs with standard deviations, t-tests are used against BPTT, and speedups are measured on actual GPUs. The theoretical connection to PSN and PSU is also a useful conceptual contribution. However, the central O(K) time-complexity claim is not backed by a cost analysis of the T-dimensional matrix operations, and the paper's own Table 3 shows a clear dependence of FPT training time on T. With a corrected complexity statement, the contribution remains solid, but as written the abstract, title, and Section 5.3 overstate what is established.
major comments (3)
- [§4.3.1, Eq. (9); Algorithm 1; §5.3, Table 1] The claim that FPT has O(K) time complexity is not supported. Each iteration computes u^{(k)} = -Vth(Λ-I)s^{(k-1)} + Λc, where Λ is a T×T lower-triangular Toeplitz matrix defined in Eq. (8). Applying Λc or (Λ-I)s costs Ω(T^2) with a dense matvec, or Ω(T) work and O(log T) depth with a prefix scan. The term (Λ-I)s must be recomputed in every iteration because s changes, so the per-iteration cost is not O(1); the total cost is Ω(LKT) work for an L-layer network, not O(LK). The paper's own Table 3 is consistent with a T-dependent cost: with K=3 fixed, FPT training time increases from 1.55 ms at T=8 to 2.01 ms at T=32 and 5.58 ms at T=128. The abstract's and title's 'constant time complexity' claim therefore needs to be revised, or a cost model for the T-dimensional operations must be provided.
- [§5.1, Lemma 5.1; Section 6.1, Figure 2] Lemma 5.1 proves contraction of the surrogate map Φ̂_α(u) = -Vth(Λ-I)S_α(u-Vth)+Λc, but it does not bound the distance between the fixed point of Φ̂_α and the fixed point of the original LIF map with the Heaviside step. The statement that FPT 'preserves LIF dynamics' is therefore not established by the theory alone; Figure 2 provides empirical evidence for specific α and K values. A quantitative approximation bound, such as ||u*_α - u*_LIF|| as a function of α, λ, and T, would be needed to substantiate the theoretical claim, especially since the schedules in Table 5 use α_f values as small as 12, for which S_α is not close to the Heaviside step in the usual uniform sense.
- [§5.3, Table 1] The complexity table is internally inconsistent with the algorithm. Table 1 reports training time O(LK) and memory O(LT)+λO(LKT), but Algorithm 1 constructs and stores the T×T matrix Λ, which alone is O(LT^2) memory if materialized per layer. It is also unclear what the coefficient λ denotes in a complexity expression. If Λ is never materialized and the matrix-vector products are applied implicitly via a scan or a recurrence, this should be stated explicitly; as written, the memory comparison undercounts the cost of the method and makes Table 1 difficult to interpret against the BPTT entries.
minor comments (5)
- [Table 2] In the ImageNet-100 block, the method name 'Normal BPPT' appears to be a typo for 'BPTT'.
- [Table 5 and surrounding text] The caption of Table 5 says the hyperparameters correspond to 'Table 1 in the main text,' but the relevant experimental results are in Table 2; Table 1 is the complexity comparison.
- [Table 5] The hyperparameter λTET is used in Table 5 but is never defined or explained in the text.
- [§2.2 and Table 4] The abbreviation IPSU is used in Section 2.2 and Table 4 but is never expanded or defined, and the related term RPSU is mentioned without a definition or reference.
- [Figure 2(c)] The caption says FPT achieves 'linear speedup' as T increases, but the axes and the reference baseline for the speedup factor are not specified, making the claim hard to verify from the figure alone.
Circularity Check
No significant circularity: the FPT fixed-point scheme is an algebraic reformulation of the LIF recurrence, the PSN/PSU special-case claims are comparisons with independent prior models, and the evaluation is externally benchmarked.
full rationale
The derivation chain is self-contained. Equation (9) is obtained by unrolling the LIF recurrence Eq. (1) into vector form with the Toeplitz decay matrix Λ from Eq. (8), so the fixed-point map Φ(u) in Eq. (10) is equivalent to the original dynamics by algebraic rearrangement rather than by assuming the desired conclusion. The convergence analysis in Lemma 5.1 and Appendix A relies on the Banach fixed-point theorem and Hölder's inequality, both external mathematical facts, and computes its contraction constant from Λ and the surrogate Lipschitz constant without presupposing convergence of FPT. The relation to PSN and PSU in Section 5.2 is a genuine comparison with externally defined prior models: PSN's update u=Ac is shown to coincide with the K=1 iteration of the generalized Eq. (22), and PSU with K=2, so neither model is defined in terms of FPT. Experimental accuracy claims are tested against independent baselines (LocalZO, BPTT, OTTT, SLTT, etc.) on standard datasets, making them externally falsifiable. There is no load-bearing self-citation and no imported uniqueness theorem from the authors' own prior work. The concern that the O(K) time-complexity claim omits the cost of the T-dimensional Λ operations is a substantive correctness or complexity-modeling critique, but it is not a circularity: the headlined claim does not reduce by construction to a fit, a definition, or a self-referential citation.
Assumptions & free parameters
free parameters (3)
- K (fixed-point iteration count) =
3 (typically)
- Forward surrogate steepness alpha_f schedule =
[3,12,12] on DVS-CIFAR10 and DVS-Gesture; [2,6,12] on ImageNet-100; [1,3,12] on sequential CIFAR
- Backward surrogate steepness alpha_b =
alpha_f / 3
assumptions (4)
- domain assumption The LIF neuron update (Eq. 1-2) is the correct model for the tasks under study.
- ad hoc to paper Replacing the Heaviside step with sigmoid S_alpha in the iteration, and then taking hard spikes at the end, yields a close approximation to the original LIF dynamics.
- standard math Banach fixed-point theorem applies to the surrogate mapping in Eq. (11).
- ad hoc to paper The matrix-vector products with Lambda and Lambda-I have negligible or constant cost with respect to T.
Cite this review
Pith. "Pith review of Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity." pith.science (2026). https://pith.science/paper/K2W32HTS
@misc{pith2026250612087,
author = {Pith},
title = {Pith review of: Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity},
year = {2026},
howpublished = {\url{https://pith.science/paper/K2W32HTS}},
note = {Machine review of arXiv:2506.12087}
}
abstract
Spiking Neural Networks (SNNs) often suffer from high time complexity $O(T)$ due to the sequential processing of $T$ spikes, making training computationally expensive. In this paper, we propose a novel Fixed-point Parallel Training (FPT) method to accelerate SNN training without modifying the network architecture or introducing additional assumptions. FPT reduces the time complexity to $O(K)$, where $K$ is a small constant (usually $K=3$), by using a fixed-point iteration form of Leaky Integrate-and-Fire (LIF) neurons for all $T$ timesteps. We provide a theoretical convergence analysis of FPT and demonstrate that existing parallel spiking neurons can be viewed as special cases of our proposed method. Experimental results show that FPT effectively simulates the dynamics of original LIF neurons, significantly reducing computational time without sacrificing accuracy. This makes FPT a scalable and efficient solution for real-world applications, particularly for long-term tasks. Our code will be released at \href{https://github.com/WanjinVon/FPT}{\texttt{https://github.com/WanjinVon/FPT}}.
Figures
Reference graph
Works this paper leans on
-
[2]
doi: 10.1007/978-3-031-72754-2
-
[4]
doi: 10.1609/AAAI.V38I10.29066. 9 Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity El Ghaoui, L., Gu, F., Travacca, B., Askari, A., and Tsai, A. Implicit Deep Learning.SIAM Journal on Mathematics of Data Science, 3(3):930–958,
-
[9]
Taylor, L., King, A., and Harper, N
doi: 10.1016/j.neunet.2022.12.008. Taylor, L., King, A., and Harper, N. S. Addressing the speed-accuracy simulation trade-off for adaptive spiking neurons. InAdvances in Neural Information Processing Systems, volume 36,
-
[10]
SSF: Accelerating Training of Spiking Neural Networks with Stabilized Spiking Flow
Wang, J., Song, Z., Wang, Y ., Xiao, J., Yang, Y ., Mei, S., and Zhang, Z. SSF: Accelerating Training of Spiking Neural Networks with Stabilized Spiking Flow. InProceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5982–5991, 2023a. Wang, X., Wu, Z., Jiang, B., Bao, Z., Zhu, L., Li, G., Wang, Y ., and Tian, Y . HARDVS: Revisiting Hu...
-
[11]
Wang, Z., Zhang, Y ., Lian, S., Cui, X., Yan, R., and Tang, H. Toward high-accuracy and low-latency spiking neural net- works with two-stage optimization.IEEE Transactions on Neural Networks and Learning Systems, 2023b. Wang, Z., Tao, P., and Chen, L. Brain-inspired Chaotic Spiking Backpropagation.National Science Review, pp. nwae037, 2024b. doi: 10.1093/...
- [12]
-
[13]
Zheng, H., Wu, Y ., Deng, L., Hu, Y ., and Li, G
doi: 10.1609/AAAI.V38I15.29640. Zheng, H., Wu, Y ., Deng, L., Hu, Y ., and Li, G. Going deeper with directly-trained larger spiking neural net- works. InProceedings of the AAAI Conference on Artifi- cial Intelligence, volume 35, pp. 11062–11070,
-
[14]
doi: 10.1038/s41467-023-44614-z. Zhu, V . and Rosenbaum, R. Learning fixed points of re- current neural networks by reparameterizing the network model.Neural Computation, 36(8):1568–1600,
Show all 15 references
-
[15]
Convergence Proof for FPT Lemma A.1.Assume the substitution function Sα(·) is Lipschitz continuous with a constant Lα
11 Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity A. Convergence Proof for FPT Lemma A.1.Assume the substitution function Sα(·) is Lipschitz continuous with a constant Lα. If the condition VthLα λ(1−λT−1 ) 1−λ <1 , where 0< λ <1 ,...
2001
-
[16]
Optimizer: Adam with betas: (0.9, 0.999), Rate Scheduler: cosine annealing
13 Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity Table 5.Training parameters for FPT on various datasets. Optimizer: Adam with betas: (0.9, 0.999), Rate Scheduler: cosine annealing. DVS-CIFAR-10 DVS-Gesture ImageNet-100 Number ep...
-
[2015]
doi: 10.1007/s11263-014-0788-3
ISSN 0920-5691, 1573-1405. doi: 10.1007/s11263-014-0788-3. Cao, Z., Li, M., Wang, X., Wang, H., Wang, F., Li, Y ., and Huang, Z.-G. Efficient Training of Spiking Neural Networks with Multi-parallel Implicit Stream Architec- ture. InEuropean Conference on Computer Vision, vol- ...
-
[2016]
doi: 10.1007/s40840-015-0255-5
ISSN 0126-6705, 2180-4206. doi: 10.1007/s40840-015-0255-5. Singh, A. K., Saraswat, V ., Baghini, M. S., and Ganguly, U. Quantum tunneling based ultra-compact and energy efficient spiking neuron enables hardware SNN.IEEE Transactions on Circuits and Systems I: Regular Papers, 6...
-
[2021]
doi: 10.1137/20M1358517
ISSN 2577-0187. doi: 10.1137/20M1358517. Fang, W., Yu, Z., Zhou, Z., Chen, D., Chen, Y ., Ma, Z., Masquelier, T., and Tian, Y . Parallel spiking neurons with high efficiency and ability to learn long-term depen- dencies. InAdvances in Neural Information Processing Systems, volume 36,
-
[2023]
doi: 10.1016/j.patter.2023.100831
ISSN 26663899. doi: 10.1016/j.patter.2023.100831. Meng, Q., Xiao, M., Yan, S., Wang, Y ., Lin, Z., and Luo, Z.- Q. Training high-performance low-latency spiking neural networks by differentiation on spike representation. In Proceedings of the IEEE/CVF Conference on Computer Vi...
2023
-
[2024]
doi: 10.1609/AAAI.V38I10. 28964. Bai, S., Kolter, J. Z., and Koltun, V . Deep equilibrium mod- els.Advances in neural information processing systems, 32,
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.