Pith. sign in

REVIEW 4 major objections 6 minor 40 references

Energy-Efficient Deep Reinforcement Learning with Spiking Transformers

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that replacing dense activations in a Decision Transformer with multi-step spiking LIF neurons produces a Transformer that is both more accurate and more energy-efficient for offline maze navigation.

desk verdict A plausible spiking-transformer architecture for offline RL, but the headline accuracy gap and energy claim are unsupported by the evidence as presented. read the letter →

arxiv 2505.14533 v1 pith:PX6G6AZ7 submitted 2025-05-20 cs.LG cs.AI

Mohammad Irfan Uddin , Nishad Tasnim , Md Omor Faruk , Zejian Zhou This is my paper
classification cs.LGcs.AI
keywords SpikingNeuralNetworksTransformerReinforcementLearningLeakyIntegrate-and-FireOfflineRLMazeNavigationEnergy-EfficientAIDecision
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that a Transformer built from spiking neurons can solve long-horizon offline reinforcement learning tasks more accurately and more energy-efficiently than a conventional agent-based Transformer. The proposed Spike-Transformer Reinforcement Learning (STRL) algorithm keeps multi-head self-attention but replaces every dense activation with a multi-step Leaky Integrate-and-Fire (LIF) neuron, so tokens are processed over several spiking micro-steps. On a dataset of 50,000 A*-generated maze trajectories, the paper reports 99.64% test accuracy for STRL versus 79.82% for a Decision Transformer, and similar success on the D4RL maze2d-umaze-v1 task. A sympathetic reading is that the threshold-and-integrate dynamics work as a temporal filter, improving action prediction while preserving the promise of event-driven, low-energy computation.

What carries the argument

The central object is the multi-step Leaky Integrate-and-Fire (LIF) neuron inserted after every linear projection and attention operation. Its membrane potential follows $U_{\ell,i,t+1} = \alpha U_{\ell,i,t} + I_{\ell,i,t} - R_{\ell,i,t}$, with a Heaviside spike emission $S = H(U - \theta)$ and reset when the threshold $\theta$ is crossed. In the SNN Transformer, queries, keys, and values are each spiked over $T_s$ micro-steps and averaged, producing the spiking self-attention $\mathrm{softmax}(\tilde{Q}\tilde{K}^\top/\sqrt{d})\tilde{V}$. This temporal integration is the mechanism claimed to filter noise and preserve long-horizon information across the stacked spiking attention and spiking MLP blocks.

What would settle it

Rerun the maze experiment with a Decision Transformer whose block count, embedding dimension, optimizer, and training schedule are matched to the six-block, 256-dimension SNN Transformer; if the matched Decision Transformer also exceeds 99% test accuracy, the claimed 20-point gap is not caused by the spiking mechanism.

Watch

Extended reading notes

Core claim

The paper claims that the spiking mechanism is not a performance tax but an improvement: replacing the Decision Transformer's instantaneous dense activations with multi-step LIF neurons yields a Transformer that reaches 99.64% test accuracy on 50,000 maze navigation trajectories, while the same setup with a standard Decision Transformer reaches 79.82%. The explanation offered is that the multi-step integration and thresholding smooth out transient noise and retain persistent signals, so the spiking Transformer captures long-range dependencies in trajectories better than a memoryless dense Transformer. The paper also claims this comes with an energy advantage because spiking neurons fire sparsely and avoid continuous activation updates, making the architecture suited to neuromorphic deployment.

Load-bearing premise

The load-bearing premise is that the 79.82% Decision Transformer result came from a fairly tuned model of comparable capacity, since the paper gives no architecture, hyperparameter, or tuning budget for that baseline.

Editorial extensions

If this is right

  • A spiking Transformer can match or exceed the action-prediction accuracy of a dense Decision Transformer on offline maze navigation, so the spiking nonlinearity does not weaken sequence modeling.
  • The reported 99.64% test accuracy on held-out maze layouts implies the learned policy generalizes to unseen maze configurations with negligible overfitting.
  • Because the model is built from event-driven LIF neurons with $T_s = 4$ time steps, it is a candidate for deployment on neuromorphic hardware with lower energy per decision than a dense Transformer.
  • The same architecture transfers to the continuous-state D4RL maze2d-umaze-v1 dataset at 99.59% validation accuracy, indicating the approach is not limited to discrete grid mazes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A next step the paper does not take is to measure actual joules per decision on neuromorphic chips; the energy claim is inferred from spiking sparsity rather than measured in the experiments.
  • Since the trajectories come from A* and are nearly deterministic, a stricter test of the spiking advantage would be stochastic or partially observable environments where temporal integration has more work to do.
  • Sweeping the number of LIF micro-steps $T_s$ would reveal a latency-accuracy trade-off and identify the shortest spike train that still yields 99% accuracy.
  • If a matched Decision Transformer also reaches 99% on this dataset, the mechanism to study is the temporal averaging itself, not the sparse firing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript proposes STRL, a Transformer architecture in which dense activations are replaced by multi-step Leaky Integrate-and-Fire (LIF) spiking neurons, and evaluates it on offline maze-navigation datasets. The central empirical claim is that STRL reaches 99.64% test accuracy on 50,000 procedurally generated 21x21 mazes, compared with 79.82% for a Decision Transformer, while also being energy-efficient due to spiking dynamics. The energy-efficiency claim is argued on the basis of prior neuromorphic hardware benchmarks rather than measured for the proposed model.

Significance. If substantiated, the architecture would be a useful contribution to spiking transformers for sequential decision making, combining multi-head attention with multi-step LIF neurons. The paper gives a fairly detailed specification of the STRL model and training configuration, and it reports consistent convergence to above 99% accuracy on the maze benchmark. However, the two headline claims — superiority over a Decision Transformer and energy efficiency — are not currently supported by the evidence: the baseline is unspecified and appears undertuned, and no energy measurements or even FLOP/synaptic-operation counts are reported for STRL. The work is therefore best viewed as an incomplete proof of concept rather than a validated result.

major comments (4)
  1. [Section 5.3, Table 2] The Decision Transformer baseline is not specified: the paper provides no architecture details (number of layers, heads, embedding dimension), no hyperparameters, no optimization configuration, no tuning budget, and no error bars or number of seeds. Since 4-way action classification on deterministic A* trajectories should be easy for a reasonably sized transformer, the reported 79.82% accuracy strongly suggests an undertuned baseline. Without a matched and properly tuned comparison, the 20-point gap cannot be attributed to the spiking mechanism.
  2. [Abstract, Section 1, Section 4.6] The energy-efficiency claim is never measured. The abstract and contributions assert lower power consumption, and Section 4.6 argues qualitatively that spiking activations reduce computation, but the paper reports no energy measurements, no FLOPs or synaptic-operation counts, and no hardware evaluation of STRL. The Loihi, TrueNorth, and SpiNNaker benchmarks cited in Section 1 pertain to other SNN systems and do not substantiate anything about this model's efficiency.
  3. [Sections 4.7 and 5.1] What is actually trained is a supervised action classifier on expert demonstrations, not a reinforcement learning agent: the only training objective is the cross-entropy loss in Eq. (7), and the agent never interacts with the environment. Consequently, the 99.64% figure is action-prediction accuracy on held-out mazes, not policy performance in an MDP sense. Claims of 'policy optimality' and 'reinforcement learning' require a closed-loop or return-based evaluation, which is not provided.
  4. [Section 5.4, Table 3, Figure 7] The D4RL experimental protocol is internally inconsistent: the text says the model is trained for 20 epochs, Table 3 reports metrics only up to epoch 10, and the text states 'by epoch 10' while the Figure 7 caption says 'across 20 epochs'. The number of epochs and the final reported metrics must be stated consistently, and the claim of 99.59% validation accuracy should be tied to a specific epoch count.
minor comments (6)
  1. [Section 4.1] The word 'trditional' should be 'traditional'.
  2. [Section 5.1] The phrase 'a extensive empirical experiments' should be corrected, e.g., to 'extensive empirical experiments'.
  3. [Section 4.7] The phrase 'expert dmonstrations' should be 'expert demonstrations'.
  4. [Sections 4.2, 4.6, 5.2] The notation for the simulation time steps is inconsistent: Tmax, Ts, and T are all used for what appears to be the same quantity, and the relationship between trajectory timesteps and micro-timesteps should be clarified.
  5. [Section 5.1] The equation numbering jumps from Eq. (7) to Eq. (18) with no equations in between, which appears to be a formatting artifact; the equations should be renumbered.
  6. [Section 5.5] The paper states that every one of ten runs exceeded 99% accuracy, but no per-seed results, standard deviations, or confidence intervals are reported; reporting these would support the robustness claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the SNN Transformer's accuracy is measured on held-out mazes and its architecture is fully specified by the paper's own equations.

full rationale

I examined the paper for circular derivation. The STRL architecture is defined by explicit equations in Sections 4.1-4.5: input embeddings, multi-step LIF dynamics (Eq. 2), spiking self-attention (Eqs. 3-5), spiking MLP, and action head (Eq. 6). The performance claim is not derived from these equations by construction; it is measured on held-out mazes in Section 5.3 (Tables 1-2). The 99.64% test accuracy is an external measurement against a separate test split, and the claim that the SNN Transformer outperforms the Decision Transformer is an empirical comparison, not a consequence of the model definition. The energy-efficiency claim is supported only by citations to external neuromorphic benchmarks [15-17] and by a qualitative sparsity argument, not by an on-chip measurement in this paper; however, importing external benchmark evidence is not circular reasoning. No equation is defined in terms of the target result, no fitted parameter is renamed as a prediction, and no load-bearing argument reduces to a self-citation chain. The baseline comparison is uncontrolled - the Decision Transformer's architecture, hyperparameters, and tuning budget are not reported - which is a validity risk, but it is not a circularity. Therefore the circularity score is 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claims rest on standard SNN training techniques, an imported energy-efficiency assumption, and a behavior-cloning benchmark treated as RL. There are no newly invented physical entities. The free parameters are architectural hyperparameters; they do not involve fitting to match specific quantitative outcomes, but they are hand-chosen and not all reported.

free parameters (4)
  • Leak factor alpha = not reported
    The leak factor in the LIF dynamics (Eq. 2) is chosen by hand and controls how much membrane potential persists across micro-steps. Its value is not reported, but the temporal smoothing behavior that the paper credits for performance depends on it.
  • Firing threshold theta = not reported
    The threshold theta determines sparsity and which activations fire. It is a free hyperparameter, not fitted to data in a principled way, and its value is not reported.
  • Number of simulation time steps Ts = 4
    The multi-step LIF uses T=4 micro-steps in all experiments (Section 5.2). This is a chosen hyperparameter that directly controls the temporal integration window.
  • Embedding dimension d = 256
    The model uses embedding dimension 256 (Section 5.2). This was tuned on validation accuracy and is a free architectural choice that affects capacity.
assumptions (3)
  • domain assumption Surrogate gradient training is a valid way to train spiking neural networks.
    The paper uses backpropagation through the Heaviside spike function via surrogate gradients (Section 4.2) without proof of convergence. This is standard practice in SNN literature.
  • domain assumption SNNs are more energy-efficient than dense ANNs on neuromorphic hardware.
    The entire energy-efficiency claim rests on external benchmarks from Loihi, TrueNorth, and SpiNNaker2 (Introduction), not on any measurement of the proposed model. This is an imported assumption, not a demonstrated property of this architecture.
  • domain assumption The A* maze trajectory dataset is a meaningful benchmark for reinforcement learning decision-making.
    The paper frames the task as offline RL, but the training data is deterministic A* demonstrations with no stochasticity or reward feedback beyond returns-to-go. This makes the task essentially behavior cloning, and the assumption that success here transfers to RL is not validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Energy-Efficient Deep Reinforcement Learning with Spiking Transformers." pith.science (2026). https://pith.science/paper/PX6G6AZ7

@misc{pith2026250514533,
  author       = {Pith},
  title        = {Pith review of: Energy-Efficient Deep Reinforcement Learning with Spiking Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PX6G6AZ7}},
  note         = {Machine review of arXiv:2505.14533}
}
read the original abstract

Agent-based Transformers have been widely adopted in recent reinforcement learning advances due to their demonstrated ability to solve complex tasks. However, the high computational complexity of Transformers often results in significant energy consumption, limiting their deployment in real-world autonomous systems. Spiking neural networks (SNNs), with their biologically inspired structure, offer an energy-efficient alternative for machine learning. In this paper, a novel Spike-Transformer Reinforcement Learning (STRL) algorithm that combines the energy efficiency of SNNs with the powerful decision-making capabilities of reinforcement learning is developed. Specifically, an SNN using multi-step Leaky Integrate-and-Fire (LIF) neurons and attention mechanisms capable of processing spatio-temporal patterns over multiple time steps is designed. The architecture is further enhanced with state, action, and reward encodings to create a Transformer-like structure optimized for reinforcement learning tasks. Comprehensive numerical experiments conducted on state-of-the-art benchmarks demonstrate that the proposed SNN Transformer achieves significantly improved policy performance compared to conventional agent-based Transformers. With both enhanced energy efficiency and policy optimality, this work highlights a promising direction for deploying bio-inspired, low-cost machine learning models in complex real-world decision-making scenarios.

Figures

Figures reproduced from arXiv: 2505.14533 by the authors.

Figure 1
Figure 1. SNN-Transformer Overview. and spatial dependencies required for RL tasks that demand sophisticated foresight and planning [18, 19]. Efforts have been made to enhance tempo￾ral extraction.For example, surrogate-gradient frameworks such as SLAYER propagate errors across hundreds of simulation steps and improve speech and gesture recognition [20]; the e-prop algorithm introduces local eligibility traces that approximat… view at source ↗
Figure 2
Figure 2. The improvement from Decision Transformer ( [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. (Top) Training and validation loss, showing mean and standard deviation per epoch. (Bottom) Training and validation accuracy. Both curves illustrate stable convergence and close alignment between training and validation sets. Convergence and Learning Curves [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Test confusion matrix over 4 actions. The model maintains high recall and [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: (Left) Decision Transformer training and validation loss, showing mean and standard deviation per epoch. (Right) Training and validation accuracy for Decision Transformer. The model exhibits stable convergence but achieves lower final accuracy compared to the SNN Trans…
Figure 6
Figure 6. Figure 6: Training loss per episode (mini-batch) for a single epoch on [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: (Left) Training (blue) and validation (red) loss with variance shading. The loss drops below 0.01 by epoch 15, demonstrating stable convergence. (Right) Training (blue) and validation (red) accuracy on maze2d-umaze-v1 across 20 epochs. The model surpasses 99% accuracy …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 12 canonical work pages

  1. [1]

    Jiang, Z

    R. Jiang, Z. Wang, B. He, Y. Zhou, G. Li, Z. Zhu, A data- efficient goal-directed deep reinforcement learning method for robot visuomotor skill, Neurocomputing 462 (2021) 389–401. doi:https://doi.org/10.1016/j.neucom.2021.08.023. URL https://www.sciencedirect.com/science/article/pii/ S0925231221012078

    work page doi:10.1016/j.neucom.2021.08.023 2021

  2. [2]

    F. Li, Q. Jiang, S. Zhang, M. Wei, R. Song, Robot skill acquisition in assembly process using deep reinforcement learning, Neurocomputing 345 (2019) 92–102, deep Learning for Intelligent Sensing, Decision-Making and Control.doi:https://doi.org/10.1016/j.neucom.2019.01.087. URL https://www.sciencedirect.com/science/article/pii/ S0925231219301316

    work page doi:10.1016/j.neucom.2019.01.087 2019

  3. [3]

    R. Zhou, H. Cao, J. Huang, X. Song, J. Huang, Z. Huang, Hybrid lane change strategy of autonomous vehicles based on soar cognitive architecture and deep reinforcement learning, Neurocomputing 611 (2025) 128669.doi:https://doi.org/10.1016/j.neucom.2024.128669. URL https://www.sciencedirect.com/science/article/pii/ S0925231224014401 20

    work page arXiv 2025

  4. [4]

    O. t. Vinyals, Grandmaster level in starcraft ii using multi-agent reinforcement learning, Nature 575 (2019) 350–354. doi:10.1038/ s41586-019-1724-z

    work page 2019

  5. [5]

    J. Yan, Y. Huang, A. Gupta, A. Gupta, C. Liu, J. Li, L. Cheng, Energy-aware systems for real-time job schedul- ing in cloud data centers: A deep reinforcement learning ap- proach, Computers and Electrical Engineering 99 (2022) 107688. doi:https://doi.org/10.1016/j.compeleceng.2022.107688. URL https://www.sciencedirect.com/science/article/pii/ S0045790622000106

    arXiv 2022

  6. [6]

    Hochreiter, J

    S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Compu- tation 9 (8) (1997) 1735–1780.arXiv:https://direct.mit.edu/neco/ article-pdf/9/8/1735/813796/neco.1997.9.8.1735.pdf, doi:10. 1162/neco.1997.9.8.1735. URLhttps://doi.org/10.1162/neco.1997.9.8.1735

    doi:10.1162/neco.1997.9.8.1735 1997

  7. [7]

    J. L. Elman, Finding structure in time, Cognitive Science 14 (2) (1990) 179–211.doi:https://doi.org/10.1016/0364-0213(90)90002-E. URL https://www.sciencedirect.com/science/article/pii/ 036402139090002E

    doi:10.1016/0364-0213(90)90002-e 1990

  8. [8]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, I. Polosukhin, Attention is all you need, in: I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems, Vol. 30, Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/pape...

    work page 2017

Show all 40 references
  1. [9]

    Devlin, M

    J. Devlin, M. Chang, K. Lee, K. Toutanova, BERT: pre-training of deep bidirectional transformers for language understanding, in: J. Burstein, C. Doran, T. Solorio (Eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Ling...

    2019 doi

  2. [10]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Un- terthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, N. Houlsby, An image is worth 16x16 words: Transformers for image recognition at scale, ArXiv abs/2010.11929 (2020). URLhttps://api...

    2020 arXiv

  3. [11]

    L. Chen, K. Lu, A. Rajeswaran, K. Lee, A. Grover, M. Laskin, P. Abbeel, A. Srinivas, I. Mordatch, Decision transformer: Reinforcement learning via sequence modeling (2021).arXiv:2106.01345. URLhttps://arxiv.org/abs/2106.01345

    2021 arXiv

  4. [12]

    E. O. Neftci, H. Mostafa, F. Zenke, Surrogate gradient learning in spiking neural networks (2019).arXiv:1901.09948. URLhttps://arxiv.org/abs/1901.09948

    2019 arXiv

  5. [13]

    Davies, N

    M. Davies, N. Srinivasa, T.-H. Lin, G. Chinya, Y. Cao, S. H. Choday, G. Dimou, P. Joshi, N. Imam, S. Jain, Y. Liao, C.-K. Lin, A. Lines, R. Liu, D. Mathaikutty, S. McCoy, A. Paul, J. Tse, G. Venkataramanan, Y.-H. Weng, A. Wild, Y. Yang, H. Wang, Loihi: A neuromorphic manycore ...

    2018

  6. [14]

    P. A. Merolla, J. V. Arthur, R. Alvarez-Icaza, A. S. Cassidy, J. Sawada, F. Akopyan, B. L. Jackson, N. Imam, C. Guo, Y. Nakamura, B. Brezzo, I. Vo, S. K. Esser, R. Appuswamy, B. Taba, A. Amir, M. D. Flickner, W. P. Risk, R. Manohar, D. S. Modha, A million spiking-neuron integr...

    2014 doi

  7. [15]

    Blouw, X

    P. Blouw, X. Choo, E. Hunsberger, C. Eliasmith, Benchmarking keyword spotting efficiency on neuromorphic hardware, in: Proceedings of the 7th Annual Neuro-Inspired Computational Elements Workshop, NICE ’19, Association for Computing Machinery, New York, NY, USA, 2019. doi:10.1...

    2019

  8. [16]

    S. K. Esser, P. A. Merolla, J. V. Arthur, A. S. Cassidy, R. Appuswamy, A. Andreopoulos, D. J. Berg, J. L. McKinstry, T. Melano, D. R. Barch, C. di Nolfo, P. Datta, A. Amir, B. Taba, M. D. Flickner, D. S. Modha, Convolutional networks for fast, energy-efficient neuromorphic com...

    2016 doi

  9. [17]

    Rostami, B

    A. Rostami, B. Vogginger, Y. Yan, C. G. Mayr, E-prop on spin- naker 2: Exploring online learning in spiking rnns on neuromorphic hardware, Frontiers in Neuroscience Volume 16 - 2022 (2022). doi:10.3389/fnins.2022.1018006. URL https://www.frontiersin.org/journals/neuroscience/ ...

    2022

  10. [18]

    G. Wu, D. Liang, S. Luan, J. Wang, Training spiking neural networks for reinforcement learning tasks with temporal cod- ing method, Frontiers in Neuroscience Volume 16 - 2022 (2022). doi:10.3389/fnins.2022.877701. URL https://www.frontiersin.org/journals/neuroscience/ articles...

    2022

  11. [19]

    G. Tang, N. Kumar, K. P. Michmizos, Reinforcement co-learning of deep and spiking neural networks for energy-efficient mapless navigation with neuromorphic hardware (2020).arXiv:2003.01157. URLhttps://arxiv.org/abs/2003.01157

    2020 arXiv

  12. [20]

    S. B. Shrestha, G. Orchard, Slayer: Spike layer error reassignment in time (2018).arXiv:1810.08646. URLhttps://arxiv.org/abs/1810.08646

    2018 arXiv

  13. [21]

    Bellec, F

    G. Bellec, F. Scherr, A. Subramoney, E. Hajek, D. Salaj, R. Legenstein, W. Maass, A solution to the learning dilemma for recurrent networks of spiking neurons, Nature Communications 11 (07 2020).doi:10.1038/ s41467-020-17236-y

    2020

  14. [22]

    Y. Wu, L. Deng, G. Li, J. Zhu, L. Shi, Spatio-temporal backpropagation for training high-performance spiking neural networks, Frontiers in Neu- roscience Volume 12 - 2018 (2018).doi:10.3389/fnins.2018.00331. 23 URL https://www.frontiersin.org/journals/neuroscience/ articles/10...

    2018

  15. [23]

    W. Fang, Z. Yu, Y. Chen, T. Masquelier, T. Huang, Y. Tian, Incorpo- rating learnable membrane time constant to enhance learning of spiking neural networks (2021).arXiv:2007.05785. URLhttps://arxiv.org/abs/2007.05785

    2021 arXiv

  16. [24]

    S. Gao, X. Fan, X. Deng, Z. Hong, H. Zhou, Z. Zhu, Te- spikformer:temporal-enhanced spiking neural network with transformer, Neurocomputing 602 (2024) 128268. doi:https: //doi.org/10.1016/j.neucom.2024.128268. URL https://www.sciencedirect.com/science/article/pii/ S0925231224010397

    2024

  17. [25]

    Gerstner, W

    W. Gerstner, W. M. Kistler, Spiking Neuron Models: Single Neurons, Populations, Plasticity, Cambridge University Press, 2002. URLhttps://psycnet.apa.org/doi/10.1017/CBO9780511815706

    2002 doi

  18. [26]

    Janner, Q

    M. Janner, Q. Li, S. Levine, Offline reinforcement learning as one big sequence modeling problem (2021).arXiv:2106.02039. URLhttps://arxiv.org/abs/2106.02039

    2021 arXiv

  19. [27]

    Ghanem, P

    A. Ghanem, P. Ciblat, M. Ghogho, Multi-objective decision transformers for offline reinforcement learning (08 2023).doi:10.48550/arXiv.2308. 16379

    2023 doi

  20. [28]

    Correia, L

    A. Correia, L. Alexandre, Hierarchical decision transformer, 2023, pp. 1661–1666.doi:10.1109/IROS55552.2023.10342230

    2023

  21. [29]

    S. G. Konan, E. Seraj, M. Gombolay, Contrastive decision transformers, in: K. Liu, D. Kulic, J. Ichnowski (Eds.), Proceedings of The 6th Conference on Robot Learning, Vol. 205 of Proceedings of Machine Learning Research, PMLR, 2023, pp. 2159–2169. URLhttps://proceedings.mlr.pr...

    2023

  22. [30]

    Z. Li, F. Nie, Q. Sun, F. Da, H. Zhao, Uncertainty-aware decision transformer for stochastic driving environments (2024). arXiv:2309. 16397. URLhttps://arxiv.org/abs/2309.16397 24

    2024 arXiv

  23. [31]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu, J. Ibarz, B. Ichter, A. Irpan, T. Jackson, S. Jesmonth, N. J. Joshi, R. Julian, D. Kalashnikov, Y. Kuang, I. Leal, K.-H. Lee, S. Levine, Y. Lu, U. Malla, D. Manj...

    2023 arXiv

  24. [32]

    V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. A. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, D. Hassabis, Human-level control through deep re...

    2015

  25. [33]

    Sutton, A

    R. Sutton, A. Barto, Reinforcement learning: An introduction, IEEE Transactions on Neural Networks 9 (5) (1998) 1054–1054.doi:10.1109/ TNN.1998.712192

    1998

  26. [34]

    URLhttps://api.semanticscholar.org/CorpusID:208329736

    K.Roy, A.R.Jaiswal, P.Panda, Towardsspike-basedmachineintelligence with neuromorphic computing, Nature 575 (2019) 607 – 617. URLhttps://api.semanticscholar.org/CorpusID:208329736

    2019

  27. [35]

    Y. Li, Y. Lei, X. Yang, Spikeformer: A novel architecture for training high-performance low-latency spiking neural network (2022). arXiv: 2211.10686. URLhttps://arxiv.org/abs/2211.10686

    2022 arXiv

  28. [36]

    Z. Zhou, Y. Zhu, C. He, Y. Wang, S. Yan, Y. Tian, L. Yuan, Spikformer: When spiking neural network meets transformer (2022).arXiv:2209. 15425. URLhttps://arxiv.org/abs/2209.15425

    2022 arXiv

  29. [37]

    J. Fu, A. Kumar, O. Nachum, G. Tucker, S. Levine, D4rl: Datasets for deep data-driven reinforcement learning (2021).arXiv:2004.07219. URLhttps://arxiv.org/abs/2004.07219 25

    2021 arXiv

  30. [38]

    P. E. Hart, N. J. Nilsson, B. Raphael, A formal basis for the heuristic determination of minimum cost paths, IEEE Transactions on Systems Science and Cybernetics 4 (2) (1968) 100–107.doi:10.1109/TSSC.1968. 300136

    1968 doi

  31. [39]

    Loshchilov, F

    I. Loshchilov, F. Hutter, Decoupled weight decay regularization (2019). arXiv:1711.05101. URLhttps://arxiv.org/abs/1711.05101

    2019 arXiv

  32. [40]

    Loshchilov, F

    I. Loshchilov, F. Hutter, Sgdr: Stochastic gradient descent with warm restarts (2017).arXiv:1608.03983. URLhttps://arxiv.org/abs/1608.03983 26

    2017 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.