{"id":"e1bb1fb7-b870-4029-8042-5a1abca57266","arxiv_id":"2506.14464","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"HYPR parallelizes the online learning rule e-prop over sequence segments using associative scans, achieving constant memory, large speedups, and near-BPTT accuracy on several tasks with oscillatory spiking neurons.","lead":"Researchers at TU Graz introduce HYPR, a training method for recurrent spiking neural networks that keeps memory use fixed regardless of input sequence length while enabling large GPU speedups. The practical payoff is that brain-like spiking networks become much easier to train on very long, continuous data streams, which standard backpropagation handles poorly.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Empirical 'low gap' claim is confounded: BPTT and HYPR use different loss objectives and hyperparameters, λ is unreported, and BPTT baselines fail to reproduce published numbers.","rationale":"The reader's conditional verdict is appropriate and should be kept. My reading supports the reader's concern about BPTT baseline quality but sharpens it: the empirical comparison is additionally confounded by different loss functions and hyperparameters between algorithms, and by the unreported subsequence length λ. These issues specifically attack the 'unprecedented low gap' claim in the abstract and Table 1, while leaving the algorithmic contribution, parallelized e-prop-style gradient accumulation with constant memory, largely intact. Appendix F's algebraic rearrangement of cumulative e-prop APGs into a backward-scan form is credible, and the reported speedups and memory scaling are plausible given the released code. A secondary ambiguity is that Algorithm 1 places the optimizer update after the whole time series, whereas Algorithm A1 appears to update inside the subsequence loop; this should be clarified because it affects both the 'online' characterization and the claimed mathematical equivalence to fully-online e-prop. The proposed concrete check would settle whether the narrow-gap claim survives a controlled comparison; until then, conditional acceptance with requested revisions is the right verdict.","tokens_in":23292,"tokens_out":13536,"duration_ms":141529,"concrete_test":"Rerun Table 1 with matched controls: identical loss for BPTT and HYPR (start with per-timestep cross-entropy for both, plus a separate sum-of-softmax BPTT-only ablation), identical learning rate, schedule, batch size, t0, surrogate gradient, and gradient clipping, and report λ, including λ=1 and λ=full sequence. Independently rerun the published ALIF/SE-adLIF BPTT configurations to verify the baselines. If the HYPR-BPTT gap on SHD, ECG, or SMNIST shifts by more than about 2 percentage points relative to Table 1, the 'unprecedented low gap' claim is not robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline empirical claim of an 'unprecedentedly low task performance gap' (Section 5.2) requires a controlled comparison between HYPR and BPTT. That control is missing. Appendix M.1 states that 'depending on the task and the training algorithm (BPTT or HYPR), we used different functions to compute a loss': sum-of-softmax is used for BPTT where applicable, while HYPR is forced to use a summative per-timestep cross-entropy because sum-of-softmax has no per-timestep decomposition. The reported accuracy gap therefore mixes the algorithm choice with a change of loss objective. Tables A4-A7 also list different hyperparameters for HYPR, shown in parentheses, including learning rate, schedule, batch size, t0, surrogate gradient, and gradient clipping; examples are ECG SE-adLIF (learning rate 0.01 vs 0.005, batch size 64 vs 32) and SMNIST BRF (learning rate 0.1 vs 0.01). In addition, the Table 1 footnote admits the ALIF BPTT baselines could not be reproduced (published 90.4/85.9/98.7 versus reproduced 85.61/84.24/97.57), so a weaker baseline can make the HYPR-BPTT gap appear artificially small. Finally, the subsequence length λ is not reported for the benchmark experiments, so it is unclear whether HYPR is evaluated in a truly online configuration or one closer to offline gradient accumulation. The algorithmic core, parallel associative-scan computation of eligibility products, appears mathematically sound; the difficulty is that Table 1 does not yet establish the claimed narrow gap.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HYPR, a segment-wise training algorithm for recurrent spiking neural networks that combines approximate forward eligibility propagation (e-prop-style) with parallel, scan-based computation over subsequences. The authors show that cumulative HYPR parameter updates are mathematically equivalent to cumulative e-prop updates (Appendix F), that eligibility products can be computed via associative scans in O(log λ) time per subsequence, and that memory use is constant with respect to the total sequence length. Experiments on a long-delay cue task report a 108× speedup over e-prop and constant memory, and benchmark runs on SHD, ECG, sMNIST, sCIFAR, and Pathfinder-E report BPTT-competitive accuracies, especially for oscillatory neuron models (BRF, SE-adLIF).","tokens_in":23660,"tokens_out":3713,"duration_ms":39076,"significance":"If the empirical claims hold, the paper makes a useful contribution: it addresses the sequentiality bottleneck of forward-propagation learning for RSNNs without sacrificing the constant-memory, infinite-context property of e-prop. The core algorithmic idea is elegant, the equivalence derivation in Appendix F is sound, and the availability of code plus 5-seed evaluations strengthens reproducibility. The main unresolved issue is the controlled comparison to BPTT, which is load-bearing for the headline claim of an unprecedentedly low performance gap; that issue is fixable within the manuscript's scope.","major_comments":[{"comment":"The HYPR-versus-BPTT benchmark comparison is not controlled. Appendix M.1 states that BPTT uses the sum-of-softmax loss where applicable, while HYPR is forced to use a summative per-timestep cross-entropy because sum-of-softmax has no per-timestep decomposition. In addition, Tables A4–A7 report different hyperparameters for HYPR and BPTT, e.g., ECG SE-adLIF learning rate 0.01 vs 0.005 and batch size 64 vs 32 (Table A5), sMNIST BRF learning rate 0.1 vs 0.01 (Table A6), and sCIFAR t0 500 vs 300 and surrogate DG vs SLAYER (Table A7). The reported accuracy gap therefore mixes the algorithm choice with a change of loss objective and with different tuning. Since the 'unprecedentedly low gap' claim is a central empirical contribution, the authors should report a matched comparison, at minimum BPTT with the same per-timestep loss, and ideally both algorithms under the same loss, surrogate, learning rate, and schedule.","section":"Section 5.2; Appendix M.1; Tables A4–A7"},{"comment":"The ALIF BPTT baselines are not reproduced: the published accuracies are 90.4/85.9/98.7, while the reproduced BPTT accuracies are 85.61/84.24/97.57. Since HYPR is compared against the authors' own reproduced BPTT numbers, the statement that HYPR is 'mostly on par' with BPTT and the claim of an unprecedentedly low gap may be overstated. The footnote argues that the discrepancy is not critical for a relative comparison, but it is critical for the specific claim that approximate forward learning approaches BPTT; the authors should either reproduce the published baselines or explicitly quantify the HYPR-to-published-BPTT gap.","section":"Table 1 footnote"},{"comment":"The subsequence length λ is not reported for any benchmark experiment, and no choice of λ is listed in Tables A4–A7. This matters for two reasons. First, λ controls the trade-off between parallelism and memory, so the claimed constant-memory property is only meaningful if λ is fixed and reported. Second, without λ, a reader cannot tell whether the reported accuracies were obtained in a near-online configuration or with a subsequence equal to the full sequence, in which case HYPR would reduce to a parallelized approximate BPTT variant. Please report λ, and the number of subsequences, for every benchmark configuration.","section":"Section 5.2; Appendix M.2"}],"minor_comments":[{"comment":"The phrase 'complex neuon state dynamics' contains a typo and should read 'complex neuron state dynamics'.","section":"Introduction"},{"comment":"Equation (A11) uses the notation Δθti for the cumulative approximate parameter gradient, whereas the main text and Eq. (6) use ∇~θti; please unify the notation.","section":"Appendix F"},{"comment":"The associative-scan construction is defined on the reversed sequence [pλ,...,p0]; the text should state explicitly how the returned sequence is ordered so that the implementation can be checked against Eq. (A12).","section":"Appendix G"},{"comment":"The code repository is referenced, but there is no version tag or commit hash; adding one would make the reproducibility claim verifiable.","section":"Appendix M.4"}],"recommendation":"major_revision","confidential_remarks":"The algorithmic core and the equivalence proof are strong and appear sound. The empirical benchmarking, however, needs substantial rework before the paper can be accepted: the loss mismatch, hyperparameter differences, unreported λ, and unreproduced ALIF baselines all bear directly on the headline 'low gap' claim. This is a fixable issue within the manuscript's scope, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"HYPR is worth knowing about. The core idea—writing e-prop's eligibility and q-vector recurrences as linear state-space recurrences and then using associative scans to compute segment-wise cumulative updates in parallel—is genuinely new, and the derivation in Appendix F that the cumulative HYPR gradient equals e-prop's is correct. That gives you constant memory in sequence length and a real speedup over step-by-step e-prop; the 108× number on the cue task is credible, and they ship code and 5-seed results. On the algorithmic side this is a solid contribution.\n\nThe soft spot is the empirical 'narrow gap' claim. Appendix M.1 says plainly that BPTT and HYPR use different loss functions (sum-of-softmax vs. per-timestep CE), and Tables A4–A7 show different learning rates, batch sizes, t0, surrogate gradients, and clipping magnitudes between the two. The Table 1 footnote admits the ALIF BPTT baselines couldn't be reproduced. And λ, the subsequence length, is not reported for the benchmarks, so I can't tell whether HYPR is running in a truly online regime or something much closer to truncated BPTT. Those confounds don't sink the central algorithmic claim, but they do mean the 'unprecedentedly low gap' should be read as 'low gap under favorable conditions,' not as a clean apples-to-apples comparison.\n\nThe oscillatory-neuron finding is interesting and worth following up, but it's part of the same uncontrolled comparison. The authors are honest about the limitation—they say the gap persists on sCIFAR/Pathfinder-E—so the framing isn't misleading. It just needs a tighter experiment.\n\nWho it's for: anyone working on online SNN training, neuromorphic learning, or efficient RNN training on long sequences. It deserves a serious referee; I'd send it to review but with a clear request to report λ, align loss objectives where possible, and either reproduce published BPTT baselines or soften the claim. Those are fixable in a revision. The algorithmic contribution is the stronger half of the paper.","headline":"Solid algorithmic contribution with a genuinely parallel e-prop reformulation; the 'low gap' empirical claim is real but confounded and needs a cleaner comparison.","tokens_in":24165,"tokens_out":2329,"would_cite":true,"duration_ms":27083,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HYPR is a training algorithm for recurrent spiking neural networks that parallelizes approximate online gradient learning over sequence segments, achieving constant memory independent of sequence length and, with oscillatory neuron…","keywords":["recurrent spiking neural networks","online learning","eligibility propagation","e-prop","backpropagation through time","associative scan","oscillatory neuron models","constant memory training"],"falsifier":"Reproduce the BPTT baselines on SHD, ECG, and sMNIST using the original released code and hyperparameters for each neuron model, then measure the gap to HYPR; if the gap is much larger than the reported few points, the 'unprecedentedly low gap' claim collapses. A second decisive check is to compare HYPR's cumulative parameter updates with fully online e-prop on the same network and seed: any numerical difference would falsify the Appendix F equivalence, and measuring GPU memory at fixed $\\lambda$ while growing input length would test the constant-memory claim.","tokens_in":23127,"feed_emoji":"⚡","tokens_out":7471,"duration_ms":71144,"temperature":0.7,"pith_summary":"This paper introduces HYPR, a training algorithm for recurrent spiking neural networks that aims to combine the two things gradient-based sequence training usually cannot have together: online, sequence-length-independent memory and GPU parallelism. The core move is to rewrite the per-neuron eligibility updates of approximate forward learning as a linear state-space recurrence, so that cumulative parameter updates over a sequence segment can be computed with associative scans instead of a step-by-step forward pass. The authors show that the resulting cumulative updates are mathematically equivalent to fully online e-prop, while being computable up to 108 times faster on a GPU and with memory that depends only on the chosen subsequence length, not on total sequence length. On benchmarks with oscillatory spiking neuron models, HYPR comes within a few percentage points of backpropagation-through-time, a much smaller gap than earlier approximate forward-gradient methods. A careful reader would care because this points toward practical online training of recurrent spiking networks on long or infinite streams, on standard hardware and potentially on neuromorphic systems.","feed_headline":"Online spiking-network training nearly matches backprop accuracy","feed_subtitle":"HYPR parallelizes gradient updates over segments, cutting training time up to 108x at constant memory.","key_machinery":"The load-bearing object is the eligibility matrix $e_i^t$ of approximate forward learning, recast with per-step Jacobians $A_i^t$ and parameter derivatives $\\delta_i^t$ into the linear state-space form $e_i^t = A_i^t e_i^{t-1} + \\delta_i^t$, with the approximate parameter gradient $\\tilde{\\nabla}_{\\theta_i}^t = B_i^t e_i^t$. Because this recurrence is linear in $e$, it can be unrolled and evaluated over a segment of length $\\lambda$ with associative scans: the cumulative transition matrices $\\phi^{\\lambda:t} = \\prod_{k=t}^{\\lambda} A_i^k$ are obtained for all $t$ in parallel, and the backward-accumulated vectors $q_i^t = q_i^{t+1}A_i^{t+1} + dL^t/ds_i^t$ are obtained by a reverse scan. The final combination $[\\tilde{\\nabla}_{\\theta_i}]_{1:\\lambda} = q_i^0 e_i^0 + \\sum_t q_i^t \\delta_i^t$ reproduces the cumulative e-prop update without ever materializing the intermediate eligibility matrices; low-rank factorization of $\\delta_i^t$ keeps memory at $O(\\lambda)$ rather than $O(T)$. This machinery is what makes the memory constant in total sequence length and the computation parallelizable over time.","core_discovery":"The central claim is that approximate online forward learning and parallel backpropagation-through-time can be fused without changing the learning rule. Given a neuron with state $s_i^t$, the eligibility matrix $e_i^t$ follows the linear recurrence $e_i^t = A_i^t e_i^{t-1} + \\delta_i^t$ with $A_i^t = \\partial s_i^t/\\partial s_i^{t-1}$ and $\\delta_i^t = \\partial s_i^t/\\partial \\theta_i$, and the approximate gradient is $\\tilde{\\nabla}_{\\theta_i}^t = B_i^t e_i^t$ with $B_i^t = (dL^t/dy_i^t)(\\partial y_i^t/\\partial s_i^t)$. This is a linear state-space model even when the neuron itself is highly nonlinear, because the nonlinearity lives inside the Jacobians, which are linear first-order approximations. HYPR splits the input into subsequences, computes state-transition products $\\phi^{\\lambda:t}$ and back-accumulated vectors $q^t$ with associative scans in parallel, and combines them to form the cumulative approximate parameter gradient $[\\tilde{\\nabla}_{\\theta_i}]_{1:\\lambda} = q_i^0 e_i^0 + \\sum_{t=1}^{\\lambda} q_i^t \\delta_i^t$. The result is mathematically equivalent to fully online e-prop but with the per-step materialization of large eligibility matrices replaced by parallel scans over low-rank factors. The paper's discovery is that this equivalence holds for a wide class of spiking neuron models, and that with oscillatory models the accuracy gap to BPTT nearly disappears.","pith_inferences":["The same linear-SSM reformulation likely applies to other forward-gradient algorithms whose updates are built from eligibility-like quantities, so the associative-scan trick may extend beyond e-prop to broader families of online recurrent learning rules; the paper itself does not explore this.","A testable prediction is that other neuron models with rich internal or oscillatory dynamics will show similarly small HYPR-to-BPTT gaps, while simpler leaky or threshold-adaptation models will not; choosing neuron models for forward-gradient trainability could become a design principle.","If the equivalence and constant-memory properties hold at scale, HYPR could serve as a fast surrogate for studying e-prop-style learning on long sequences, and could motivate simplified variants tailored for neuromorphic hardware, though the paper notes that pure forward propagation may be simpler to implement on such hardware."],"forward_implications":["HYPR can train an RSNN on arbitrarily long input streams on a single GPU, because memory grows only with subsequence length $\\lambda$, not with total sequence length.","Because its cumulative updates are equivalent to fully online e-prop, HYPR retains e-prop's infinite training context and online spirit while running up to 108 times faster in the reported medium-size setup.","Oscillatory neuron models such as BRF and SE-adLIF train particularly well under HYPR, reducing the accuracy gap to BPTT on SHD, ECG, and sMNIST to roughly one to three percentage points.","HYPR is the first approximate forward-learning RSNN method reported to work on sCIFAR and Pathfinder-E, though with a larger gap to BPTT there.","Recurrent connections are still used by HYPR-trained networks: removing them lowers accuracy for both BPTT and HYPR, so the method does not collapse to a feedforward learner despite ignoring recurrent gradient pathways."],"supporting_citations":[{"why":"Defines the e-prop algorithm to which HYPR is mathematically equivalent and against which the speedup is measured.","marker":"[12]"},{"why":"Introduces RTRL, the exact forward-gradient method whose approximations, including e-prop, HYPR builds on.","marker":"[13]"},{"why":"Supplies the associative scan (parallel prefix sum) algorithm used to parallelize eligibility and gradient accumulation over subsequences.","marker":"[33]"},{"why":"Explains the parallel state-space model formulation and associative scan usage that HYPR adapts for eligibility matrices.","marker":"[34]"},{"why":"Introduces the Balanced Resonate-and-Fire oscillatory neuron model used in HYPR experiments and provides its BPTT accuracy baseline.","marker":"[17]"},{"why":"Introduces the SE-adLIF oscillatory neuron model used in experiments, with BPTT baselines and the initialization used for training.","marker":"[15]"},{"why":"Introduces the ALIF neuron model and the ECG benchmark preprocessing used as a non-oscillatory comparison baseline.","marker":"[35]"},{"why":"Provides the Spiking Heidelberg Digits dataset used as a benchmark.","marker":"[36]"},{"why":"Provides the ECG dataset used as a benchmark.","marker":"[37]"},{"why":"Provides the long-range arena tasks (sCIFAR and Pathfinder-E) used to probe long-range dependencies.","marker":"[39]"}],"fun_headline_variants":["HYPR: online spiking training at backprop accuracy, constant memory","Parallel scans make online spiking learning match backprop","Constant-memory online training rivals BPTT for spiking networks","HYPR: 108x faster spiking training with near-backprop accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that HYPR nearly matches BPTT rests on the BPTT baselines being comparably strong, and the authors report that they could not exactly reproduce the published BPTT accuracies with their own code.","fun_headline_variants_meta":{"raw":{"variants":["HYPR: online spiking training at backprop accuracy, constant memory","Parallel scans make online spiking learning match backprop","Constant-memory online training rivals BPTT for spiking networks","HYPR: 108x faster spiking training with near-backprop accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001155,"raw_usage":{"total_tokens":4883,"prompt_tokens":1140,"completion_tokens":3743,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":756,"completion_tokens_details":{"reasoning_tokens":3668}},"tokens_in":756,"tokens_out":3743,"duration_ms":29111,"temperature":1.0,"reasoning_tokens":3668,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:17:18.762479+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Reproduce the BPTT baselines on SHD, ECG, and sMNIST using the original released code and hyperparameters for each neuron model, then measure the gap to HYPR; if the gap is much larger than the reported few points, the 'unprecedentedly low gap' claim collapses. A second decisive check is to compare HYPR's cumulative parameter updates with fully online e-prop on the same network and seed: any numerical difference would falsify the Appendix F equivalence, and measuring GPU memory at fixed $\\lambda$ while growing input length would test the constant-memory claim.","supporting_citations":[{"cited_title":"A solution to the learning dilemma for recurrent networks of spiking neurons","cited_arxiv_id":null,"evidence_quote":"Defines the e-prop algorithm to which HYPR is mathematically equivalent and against which the speedup is measured."},{"cited_title":"Williams and David Zipser","cited_arxiv_id":null,"evidence_quote":"Introduces RTRL, the exact forward-gradient method whose approximations, including e-prop, HYPR builds on."},{"cited_title":"Simplified state space layers for sequence modeling","cited_arxiv_id":null,"evidence_quote":"Explains the parallel state-space model formulation and associative scan usage that HYPR adapts for eligibility matrices."},{"cited_title":"Balanced resonate-and- fire neurons","cited_arxiv_id":null,"evidence_quote":"Introduces the Balanced Resonate-and-Fire oscillatory neuron model used in HYPR experiments and provides its BPTT accuracy baseline."},{"cited_title":"Accurate and efficient time-domain classi- fication with adaptive spiking recurrent neural networks","cited_arxiv_id":null,"evidence_quote":"Introduces the ALIF neuron model and the ECG benchmark preprocessing used as a non-oscillatory comparison baseline."},{"cited_title":"The Hei- delberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks","cited_arxiv_id":null,"evidence_quote":"Provides the Spiking Heidelberg Digits dataset used as a benchmark."},{"cited_title":"A database for evaluation of algorithms for measurement of qt and other waveform intervals in the ecg","cited_arxiv_id":null,"evidence_quote":"Provides the ECG dataset used as a benchmark."}],"review_version":1}