{"id":"18472ef0-694d-43db-861a-e34c3f23bcb9","arxiv_id":"2506.12087","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Fixed-point iteration with a sigmoid surrogate lets all T timesteps of a LIF neuron be processed in K=3 parallel iterations, matching BPTT accuracy with large speedups.","lead":"This paper introduces Fixed-point Parallel Training (FPT), a method that processes all timesteps of a spiking neural network simultaneously through a fixed-point iteration, rather than step by step. The authors claim near-constant training time and report accuracy matching backpropagation-through-time on several benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline O(K) time-complexity claim is not supported: the per-iteration update in Eq. (9) is a T×T lower-triangular matvec or scan, so each iteration costs at least Ω(T) work and the total cost is Ω(KT), not O(K).","rationale":"The reader's weakest_assumption is exactly this complexity issue, and I agree it is load-bearing: the entire contribution, from the title through the abstract and Table 1, rests on reducing a T-sequential recurrence to K parallel iterations. Eq. (9) does not eliminate T from the cost; it moves the T-dependence into a matrix-vector product with a T×T lower-triangular matrix. Unlike the O(T) sequential recurrence, the parallel scan form is neither constant-time in work nor constant-time in wall-clock on real hardware. The Table 3 measurements support this rather than contradicting it. I therefore do not see a way to defend the O(K) complexity claim as stated. On the other hand, the empirical results are substantial: FPT matches BPTT accuracy within noise across several datasets, preserves a reset-like mechanism that PSN lacks, and speeds up training relative to BPTT. These are genuine contributions even if the complexity bound is corrected to O(KT) work or O(K log T) parallel depth. A secondary concern is the convergence proof in Lemma A.1, which uses a matrix norm not induced by the 1-norm, since the operator norm for ||·||_1 is the maximum column sum, not the maximum row sum; moreover, the stated condition is not met for the α_f=12, λ=0.5 configurations in Appendix C. However, the complexity issue alone is sufficient to undermine the headline claim. Since the reader already conditions acceptance on correcting the complexity and convergence claims and releasing code, my concern does not move the verdict.","tokens_in":16978,"tokens_out":6825,"duration_ms":83934,"concrete_test":"With K fixed (e.g., K=3), microbenchmark the single operation in Algorithm 1, u ← -Vth(Λ-I)s + Λc, in isolation for T = 8, 32, 128, 512 on the same GPU, measuring wall time and FLOP count per iteration. If per-iteration cost grows with T—as it must for any lower-triangular matvec or prefix-scan implementation—the O(K) claim is false. Re-deriving Eq. (9) and counting Ω(T) flops per iteration would settle the issue analytically without needing the authors' code.","verdict_should_be":"UNCHANGED","load_bearing_attack":"FPT's central claim is that training time is O(K), with K a small constant independent of T. This holds only if each fixed-point iteration is O(1) work. But Eq. (9) defines u = -Vth(Λ-I)s + Λc, where Λ is a T×T lower-triangular Toeplitz matrix. Computing Λc once costs Ω(T^2) via a dense matvec or Ω(T) via a parallel prefix scan; the term (Λ-I)s must be recomputed each iteration because s changes, and that again costs Ω(T^2) directly or Ω(T) work with Ω(log T) parallel depth using a scan. Thus the per-iteration cost is not O(1), and the total training cost is Ω(LKT) in work, not O(LK). The paper's own Table 3 is consistent with this: FPT training time rises from 1.55 ms at T=8 to 5.58 ms at T=128 with K=3 fixed. Table 1's O(LK) training-time entry treats T-dimensional vector operations as free. This is not a quibble about constants: the title, abstract, and Algorithm 1 predict a T-independent training time, which the stated complexity measure cannot deliver. The empirical speedups relative to BPTT may still be useful, but the 'constant time complexity' headline is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes Fixed-point Parallel Training (FPT), a method that rewrites the LIF recurrence over T timesteps as a fixed-point equation u = -Vth(Λ-I)s + Λc with s = S_α(u - Vth), and solves it by K fixed-point iterations so that all timesteps can be updated simultaneously. The backward pass backpropagates through the K iterations using surrogate gradients. The authors claim O(K) training time, prove convergence of the surrogate fixed-point map, identify PSN and PSU as K=1 and K=2 special cases, and report competitive accuracy and large GPU speedups on DVS-CIFAR10, DVS-Gesture, ImageNet-100, sequential CIFAR10/CIFAR100, and an Amazon Photos graph task.","tokens_in":17304,"tokens_out":6967,"duration_ms":87252,"significance":"If the headline complexity claim were supported, FPT would be a valuable architecture-agnostic SNN training method because it preserves the reset mechanism and can be applied without modifying the network. The empirical part is careful in several respects: results are averaged over three runs with standard deviations, t-tests are used against BPTT, and speedups are measured on actual GPUs. The theoretical connection to PSN and PSU is also a useful conceptual contribution. However, the central O(K) time-complexity claim is not backed by a cost analysis of the T-dimensional matrix operations, and the paper's own Table 3 shows a clear dependence of FPT training time on T. With a corrected complexity statement, the contribution remains solid, but as written the abstract, title, and Section 5.3 overstate what is established.","major_comments":[{"comment":"The claim that FPT has O(K) time complexity is not supported. Each iteration computes u^{(k)} = -Vth(Λ-I)s^{(k-1)} + Λc, where Λ is a T×T lower-triangular Toeplitz matrix defined in Eq. (8). Applying Λc or (Λ-I)s costs Ω(T^2) with a dense matvec, or Ω(T) work and O(log T) depth with a prefix scan. The term (Λ-I)s must be recomputed in every iteration because s changes, so the per-iteration cost is not O(1); the total cost is Ω(LKT) work for an L-layer network, not O(LK). The paper's own Table 3 is consistent with a T-dependent cost: with K=3 fixed, FPT training time increases from 1.55 ms at T=8 to 2.01 ms at T=32 and 5.58 ms at T=128. The abstract's and title's 'constant time complexity' claim therefore needs to be revised, or a cost model for the T-dimensional operations must be provided.","section":"§4.3.1, Eq. (9); Algorithm 1; §5.3, Table 1"},{"comment":"Lemma 5.1 proves contraction of the surrogate map Φ̂_α(u) = -Vth(Λ-I)S_α(u-Vth)+Λc, but it does not bound the distance between the fixed point of Φ̂_α and the fixed point of the original LIF map with the Heaviside step. The statement that FPT 'preserves LIF dynamics' is therefore not established by the theory alone; Figure 2 provides empirical evidence for specific α and K values. A quantitative approximation bound, such as ||u*_α - u*_LIF|| as a function of α, λ, and T, would be needed to substantiate the theoretical claim, especially since the schedules in Table 5 use α_f values as small as 12, for which S_α is not close to the Heaviside step in the usual uniform sense.","section":"§5.1, Lemma 5.1; Section 6.1, Figure 2"},{"comment":"The complexity table is internally inconsistent with the algorithm. Table 1 reports training time O(LK) and memory O(LT)+λO(LKT), but Algorithm 1 constructs and stores the T×T matrix Λ, which alone is O(LT^2) memory if materialized per layer. It is also unclear what the coefficient λ denotes in a complexity expression. If Λ is never materialized and the matrix-vector products are applied implicitly via a scan or a recurrence, this should be stated explicitly; as written, the memory comparison undercounts the cost of the method and makes Table 1 difficult to interpret against the BPTT entries.","section":"§5.3, Table 1"}],"minor_comments":[{"comment":"In the ImageNet-100 block, the method name 'Normal BPPT' appears to be a typo for 'BPTT'.","section":"Table 2"},{"comment":"The caption of Table 5 says the hyperparameters correspond to 'Table 1 in the main text,' but the relevant experimental results are in Table 2; Table 1 is the complexity comparison.","section":"Table 5 and surrounding text"},{"comment":"The hyperparameter λTET is used in Table 5 but is never defined or explained in the text.","section":"Table 5"},{"comment":"The abbreviation IPSU is used in Section 2.2 and Table 4 but is never expanded or defined, and the related term RPSU is mentioned without a definition or reference.","section":"§2.2 and Table 4"},{"comment":"The caption says FPT achieves 'linear speedup' as T increases, but the axes and the reference baseline for the speedup factor are not specified, making the claim hard to verify from the figure alone.","section":"Figure 2(c)"}],"recommendation":"major_revision","confidential_remarks":"The empirical work is credible and the method appears practically useful, but the headline complexity claim is a load-bearing statement that is not supported by the manuscript's own equations and runtime measurements. I would be willing to accept a revised version that either provides a full cost model for the T-dimensional operations or honestly reframes the claim as O(K) fixed-point iterations with O(KT) work and O(K log T) parallel depth on a GPU, updating the abstract, title, and Section 5.3 accordingly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a read if you work on SNN training speed. The core idea — rewriting the LIF recurrence as a fixed-point iteration and running K steps in parallel — is a nice unification: PSN and PSU fall out as K=1 and K=2, and the reset mechanism is preserved. The experimental work is solid: accuracy matches BPTT on DVS-CIFAR10, DVS-Gesture, and ImageNet-100, and the speedups are real, with t-tests and standard deviations reported. The code is promised but not yet released.\n\nThe soft spot is the headline. 'Constant time O(K)' is not supported. Each fixed-point iteration requires applying the T×T lower-triangular matrix Λ to a vector, which is Ω(T) work (or Ω(T²) if done densely). The paper's own Table 3 shows FPT training time rising from 1.55 ms at T=8 to 5.58 ms at T=128 — a factor of 3.6, not constant. So the claim that training time is independent of T doesn't hold. What they can honestly claim is that FPT replaces O(T) sequential steps with K parallel steps at the cost of O(KT) work, and that it is much faster than BPTT in practice.\n\nThe convergence proof also needs a fix. The bound uses the infinity norm of Λ−I for a 1-norm contraction. For this specific Toeplitz matrix the max row and column sums coincide, so the result is salvageable, but as written it's wrong. Also their own α_f=12 sits exactly at the boundary of the derived condition α<12, so the proof doesn't cover the configurations used in the experiments.\n\nThat said, the method itself is sensible. The fixed-point view is a real contribution, the ablations with learnable A show the reset mechanism matters, and the empirical comparison with BPTT is honest. The main fix is to correct the complexity statement and the norm argument.\n\nVerdict: send to review, but the authors should not present the complexity claim as O(K) without qualification. A careful referee will catch this.","headline":"The fixed-point framework is a genuinely neat unification of PSN/PSU, but the 'constant time' claim doesn't hold up—per-iteration cost grows with T.","tokens_in":17820,"tokens_out":4175,"would_cite":false,"duration_ms":43722,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","65H10"],"pacs":[],"model":"deepseek-v4-flash","headline":"By rewriting LIF dynamics as a fixed point over all timesteps, the paper claims SNN training can run in about three parallel iterations instead of T sequential ones, matching BPTT accuracy while leaving the architecture untouched.","keywords":["spiking neural networks","fixed-point iteration","parallel training","LIF neuron","surrogate gradient","backpropagation through time","time complexity","parallel spiking neuron"],"falsifier":"Run FPT's forward pass with $K=3$ fixed on one GPU and measure per-batch time at $T=8$, $32$, $128$, $512$, and $2048$, holding the network and batch size constant; the paper's own Table 3 places the curve at 1.55 ms ($T=8$), 2.01 ms ($T=32$), and 5.58 ms ($T=128$), so a rising curve — or a direct operation count showing the $\\Lambda c$ product grows with $T$ — would settle that the complexity is not constant in $T$ as literally claimed.","tokens_in":16760,"feed_emoji":"⚡","tokens_out":25278,"duration_ms":245247,"temperature":0.7,"pith_summary":"The paper claims that the main bottleneck of spiking-neural-network training — stepping through $T$ timesteps one at a time — can be bypassed by rewriting the Leaky Integrate-and-Fire (LIF) neuron, the standard model whose membrane potential decays between spikes and fires when crossing a threshold, as a single fixed-point equation over the entire time axis. The proposed method, FPT, solves this equation with about three surrogate-smoothed iterations that update every timestep in parallel, and a contraction proof guarantees convergence to the same trajectory the sequential neuron would produce, reset mechanism included. If the claim holds, model architecture and trained weights stay identical to ordinary backpropagation-through-time (BPTT) training, yet measured training time decouples from sequence length: roughly 10 times faster than BPTT at $T=32$ and about 21 times faster at $T=512$, with accuracy statistically indistinguishable from BPTT. Existing parallel spiking neuron (PSN) and parallel spiking unit (PSU) models turn out to be the one- and two-iteration truncations of the same scheme, which explains why they deviate from true LIF dynamics.","feed_headline":"Three fixed-point passes replace T timesteps of SNN training","feed_subtitle":"A fixed-point rewrite of LIF neurons trains all timesteps in parallel, matching BPTT accuracy at up to 21x speed.","key_machinery":"The load-bearing object is the vectorized fixed-point map $\\hat\\Phi_\\alpha(u) = -V_{\\mathrm{th}}(\\Lambda - I)S_\\alpha(u - V_{\\mathrm{th}}) + \\Lambda c$, built from the lower-triangular Toeplitz decay matrix $\\Lambda$ whose $(i,j)$ entry is $\\lambda^{i-j}$ for $i \\ge j$ and zero otherwise; this matrix encodes how inputs decay into every later timestep. Writing the membrane potential as a $T$-vector $u$ turns the chronological LIF recurrence into a single simultaneous equation, so each application of $\\hat\\Phi_\\alpha$ updates all timesteps in parallel, and the contraction condition $V_{\\mathrm{th}} L_\\alpha \\lVert \\Lambda - I \\rVert_\\infty < 1$, with $\\lVert \\Lambda - I \\rVert_\\infty = \\lambda(1-\\lambda^{T-1})/(1-\\lambda)$, guarantees convergence to a unique fixed point. The split steepness parameters, forward $\\alpha_f$ and backward $\\alpha_b = \\alpha_f/3$, keep the forward trajectory close to the true Heaviside dynamics while giving the backward pass a smooth surrogate gradient through $K$ unrolled iterations, avoiding the Jacobian inversion of deep equilibrium models. With a learnable decay matrix $A$ and threshold $B$, the same map yields PSN at $K=1$ and PSU at $K=2$ as truncated special cases.","core_discovery":"FPT's central discovery is that the LIF dynamics $u_t = \\lambda(u_{t-1} - V_{\\mathrm{th}} s_{t-1}) + c_t$, $s_t = H(u_t - V_{\\mathrm{th}})$ across $T$ timesteps can be re-expressed exactly as one simultaneous system in vector form: $u = -V_{\\mathrm{th}}(\\Lambda - I)s + \\Lambda c$, $s = H(u - V_{\\mathrm{th}})$, where $\\Lambda$ is the lower-triangular matrix of exponential decay weights $\\lambda^{i-j}$. Replacing the Heaviside step $H$ with a sigmoid surrogate $S_\\alpha$ makes the induced map $\\hat\\Phi_\\alpha(u) = -V_{\\mathrm{th}}(\\Lambda - I)S_\\alpha(u - V_{\\mathrm{th}}) + \\Lambda c$ a contraction under the 1-norm whenever $V_{\\mathrm{th}} L_\\alpha \\lambda(1-\\lambda^{T-1})/(1-\\lambda) < 1$, so $K$ iterations — $K=3$ in practice — reach a unique fixed point whose spike outputs reproduce the original neuron's trajectory. Splitting the surrogate steepness, with a large $\\alpha_f$ in the forward pass and a smoother $\\alpha_b = \\alpha_f/3$ in the backward pass, keeps the forward map faithful while stabilizing gradients, and training needs no architecture change. The paper further shows that PSN ($K=1$) and PSU ($K=2$) are special cases of this same iterated map for a learnable variant, and that the preserved reset mechanism — absent from those models — is what keeps FPT accuracy at LIF levels. The claimed time complexity is $O(K)$, versus $O(T)$ for BPTT, because each iteration updates all $T$ timesteps simultaneously.","pith_inferences":["A literal reading of the $O(K)$ claim counts only the number of iteration rounds: each round computes the $T$-vector $-V_{\\mathrm{th}}(\\Lambda - I)s + \\Lambda c$, and producing any $T$-dimensional output costs $\\Omega(T)$ work — $O(T)$ work and $O(\\log T)$ parallel depth with a scan, or $O(T \\log T)$ with an FFT — so the honest complexity is closer to $O(K \\log T)$ parallel depth and $\\Omega(KT)$ ","Because $\\Lambda$ is Toeplitz, $\\Lambda c$ is a causal exponential filter — effectively a parallel prefix or convolution — so FPT's remaining $T$-dependence can be delegated to a purpose-built scan or FFT kernel; the method's true complexity would then be that of the convolution, not of the fixed-point iterations.","Since the fixed point is unique and contractive, a deep-equilibrium-style rule that differentiates the equilibrium equation directly is a natural alternative to unrolling $K$ iterations; the paper chooses the unroll because $K$ is small, and that trade-off would flip for larger $K$, steeper $\\alpha_f$, or cheaper Jacobian solvers.","The same vectorization should transfer to any contractive leaky recurrence $u_t = \\lambda g(u_{t-1}) + c_t$ with a Lipschitz $g$; the paper treats only the LIF/Heaviside pair, but the fixed-point template suggests a route to parallelizing other gated recurrent cells that have a reset."],"forward_implications":["Training cost decouples from sequence length: FPT reports per-batch training about 10 times faster than BPTT at $T=32$–$128$ and about 21 times faster at $T=512$, with inference time nearly constant in $T$.","Accuracy matches full BPTT: t-tests on the Amazon Photos/DRSGNN benchmark give $p > 0.05$ at $T=8$, $32$, and $128$, while FPT beats the LocalZO baseline by over 3 accuracy points on DVS-CIFAR10 and by about 1.7 points on ImageNet-100.","Because the converged forward dynamics equal the original LIF dynamics, a model trained with FPT can be deployed unchanged on sequential neuromorphic hardware, and the paper argues this makes FPT a practical pretraining scheme for long-sequence SNN tasks.","PSN and PSU are revealed as one- and two-step truncations of a single iterated map; their known accuracy gap relative to LIF is attributable to the missing iterations and the dropped reset mechanism, both of which FPT retains.","Three iterations suffice across all reported settings: increasing to $K=4$ or $K=5$ leaves DVS-Gesture accuracy unchanged, and non-learnable FPT with $K=3$ reproduces ordinary LIF accuracy on sequential CIFAR10 (81.54 vs 81.50)."],"supporting_citations":[{"why":"Defines PSN, the K=1 special case of FPT's learnable variant, and supplies the sequential-CIFAR baselines and experimental settings the ablation adopts.","marker":"(Fang et al., 2023)"},{"why":"Defines PSU/IPSU/RPSU, the K=2 truncations of the same iterated map, which serve as the other special-case family and ablation baselines.","marker":"(Li et al., 2024)"},{"why":"Introduces deep equilibrium models, the fixed-point training paradigm FPT distinguishes itself from by unrolling only K=3 iterations instead of inverting a Jacobian.","marker":"(Bai et al., 2019)"},{"why":"Supplies the generalized Banach fixed-point theorem invoked in the contraction proof that guarantees convergence of the FPT iteration.","marker":"(Shukla et al., 2016)"},{"why":"Provides the Hölder-inequality generalization used to bound the Lipschitz constant of the surrogate fixed-point map in the convergence proof.","marker":"(Cheung, 2001)"},{"why":"LocalZO is the zeroth-order training baseline FPT compares against for accuracy and speed on DVS-CIFAR10, DVS-Gesture, and ImageNet-100.","marker":"(Mukhoty et al., 2023)"},{"why":"Source of the probabilistic Bernoulli firing mechanism used to produce final spike outputs at the FPT equilibrium.","marker":"(Ma et al., 2023)"},{"why":"Provides the DRSGNN model and Amazon Photos experimental setup whose BPTT training FPT matches in accuracy while accelerating.","marker":"(Zhao et al., 2024)"}],"fun_headline_variants":["Constant-time SNN training via fixed-point LIF","Three fixed-point steps train all timesteps at once","O(K) training: parallel SNNs without accuracy loss","Fixed-point method speeds SNN training 21x","Parallel spiking networks in constant time"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's constant-time claim counts each full pass over all $T$ timesteps as a single unit of work, even though that pass multiplies by the $T$-by-$T$ decay matrix $\\Lambda$; the cost of forming $\\Lambda c$ and $(\\Lambda - I)s$ is never analyzed, so the $O(K)$ headline holds only if that matrix-vector cost is ignored.","fun_headline_variants_meta":{"raw":{"variants":["Constant-time SNN training via fixed-point LIF","Three fixed-point steps train all timesteps at once","O(K) training: parallel SNNs without accuracy loss","Fixed-point method speeds SNN training 21x","Parallel spiking networks in constant time"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000206,"raw_usage":{"total_tokens":1812,"prompt_tokens":1152,"completion_tokens":660,"prompt_tokens_details":{"cached_tokens":1024},"prompt_cache_hit_tokens":1024,"prompt_cache_miss_tokens":128,"completion_tokens_details":{"reasoning_tokens":585}},"tokens_in":128,"tokens_out":660,"duration_ms":318230,"temperature":1.0,"reasoning_tokens":585,"cache_read_input_tokens":1024,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:03:59.968425+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run FPT's forward pass with $K=3$ fixed on one GPU and measure per-batch time at $T=8$, $32$, $128$, $512$, and $2048$, holding the network and batch size constant; the paper's own Table 3 places the curve at 1.55 ms ($T=8$), 2.01 ms ($T=32$), and 5.58 ms ($T=128$), so a rising curve — or a direct operation count showing the $\\Lambda c$ product grows with $T$ — would settle that the complexity is not constant in $T$ as literally claimed.","supporting_citations":[],"review_version":1}