Pith. sign in

REVIEW 4 major objections 5 minor 25 references

Spiking control agents lose performance because tiny action errors snowball across decision steps, and a training-free voltage carry-over can stop the snowball.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 06:49 UTC pith:RJB2EKH6

load-bearing objection The diagnosis of error amplification in continuous-control ANN-to-SNN conversion is real and worth publishing, but the proposed CRPI remedy is weakened by per-task test-set tuning of its free parameter and a few derivation/reporting slips. the 4 major comments →

arxiv 2601.21778 v3 pith:RJB2EKH6 submitted 2026-01-29 cs.NE cs.LG

Error Amplification Limits ANN-to-SNN Conversion in Continuous Control

Zijie Xu , Zihan Huang , Yiting Dong , Kang Chen , Wenxuan Liu , Zhaofei Yu This is my paper
classification cs.NE cs.LG
keywords spiking neural networksANN-to-SNN conversioncontinuous controlreinforcement learningerror amplificationtemporal correlationmembrane potential initializationtraining-free conversion
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish why ANN-to-SNN conversion, which works well in classification and discrete control, degrades sharply in continuous control. It argues that the main culprit is not instantaneous action mismatch but small action approximation errors that become positively correlated across decision steps, letting the closed-loop environment amplify them into cumulative state drift. The paper supports this with a reward decomposition showing that state distribution shift dominates the return loss, and with cosine-similarity metrics showing converted spiking policies drift consistently while ANN policies actively compensate. It then proposes Cross-Step Residual Potential Initialization, a training-free mechanism that carries residual membrane voltage across decision steps to decorrelate those errors, and reports consistent recovery of lost performance on continuous-control benchmarks with vector and pixel observations.

Core claim

The paper claims that ANN-to-SNN conversion in continuous control fails because of error amplification, not per-step approximation quality: small action errors become positively correlated across decision steps, and the environment turns that correlation into progressive state-trajectory divergence that destroys return. Reward decomposition shows the loss is dominated by state distribution shift, and cosine-similarity metrics show converted spiking policies drift consistently while ANN policies compensate. The proposed fix, CRPI, records residual membrane potentials at the end of each decision step, scales them by α, and adds them to the next step's initial potentials, effectively subtractin

What carries the argument

Cross-Step Residual Potential Initialization (CRPI): after each decision step, the difference between the final and initial membrane potentials of each neuron is saved, scaled by a coefficient α, and added to the initial membrane potential at the next decision step (with clipping to keep potentials in the valid firing range). It implements a first-order error correction, modified residual equals current residual minus α times the previous residual, which drives the cosine similarity of consecutive conversion errors toward zero and thereby stops the environment from amplifying them.

Load-bearing premise

The fix rests on the assumption that the error left over from one decision step reliably predicts the next step's error through one scalar coefficient α; if residual errors are dominated by something other than the previous-step residual, or the final membrane potential is strongly tied to initialization, the correction is unjustified and becomes an ad hoc damping heuristic.

What would settle it

Measure cross-step cosine similarity of action errors and the ANN/SNN return gap in a continuous-control task with a very short horizon or near-open-loop dynamics. If a large return gap persists while the SNN Drift correlation is near zero, the claim that temporal error correlation drives amplification is wrong; conversely, if manipulating α restores performance in exact proportion to the measured residual correlation, the mechanism is supported.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Performance of converted spiking policies in continuous control must be assessed through induced state trajectories over the full horizon, because per-step action accuracy alone can look fine while long-horizon return collapses.
  • Existing training-free conversion pipelines can inherit the fix by changing only how membrane potentials are initialized at decision boundaries, with no architectural change or extra training.
  • The decorrelation mechanism transfers across neuron models, simulation lengths, and RL training algorithms, meaning the error-amplification problem is general rather than an artifact of one conversion recipe.
  • On vision-based continuous control, the corrected conversion can match or exceed directly trained spiking baselines on average, which strengthens the case for training-free conversion as a practical route to energy-efficient control policies.
  • The fix preserves the energy advantage of spiking inference, since the added computation is a per-step state transfer rather than extra network operations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If positive cross-step error correlation is the true failure driver, the same phenomenon should appear in any closed-loop use of converted spiking networks, such as imitation learning or model-predictive control; CRPI-style initialization is a natural thing to test there.
  • A parameter-free extension would estimate α online from the measured cosine similarity of consecutive residuals during deployment, removing the per-task grid search and making the method adaptive to changing error statistics.
  • The paper's diagnostics (state-decomposition of return and cross-step error similarity) could be used as a pre-deployment screening tool: if a converted policy's SNN Drift is positive and large, long-horizon failure is predictable before full rollout evaluation.
  • Since ANN policies show negative, error-correcting correlation while converted SNNs show positive correlation, a more ambitious research direction is to shape the conversion itself to mimic the ANN's corrective behavior rather than only damp the SNN's errors.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper studies ANN-to-SNN conversion in continuous control reinforcement learning and argues that the dominant cause of performance loss is not instantaneous action mismatch but temporally correlated action approximation errors that produce cumulative state distribution shift. It supports this with a reward decomposition (Section 4.1), trajectory divergence analyses (Section 4.2), and cosine-similarity measures of cross-step error correlation (Section 4.3). To mitigate the problem, the paper proposes Cross-Step Residual Potential Initialization (CRPI), which initializes membrane potentials at the start of a decision step using the previous step's residual potential with a strength α, effectively modifying the residual error as ε_{k+1} − α ε_k. Experiments on MuJoCo and DeepMind Control Suite tasks report consistent improvements in Average Performance Ratio (APR) across IF, SNM, MT, and DC neuron models, with some configurations exceeding ANN performance, and include energy-efficiency estimates. The paper makes its code publicly available.

Significance. If the central claims hold, the paper makes a useful and timely contribution: it identifies a failure mode specific to continuous-control ANN-to-SNN conversion—closed-loop amplification of correlated conversion errors—and proposes a simple, training-free remedy that is compatible with several conversion pipelines. The counterfactual reward decomposition in Section 4.1 is a clean way to separate action-mismatch effects from state-distribution-shift effects, and the breadth of neuron-model compatibility is a strength. The code release is also a positive feature. However, the quantitative support for CRPI is currently weakened by the hyperparameter-selection protocol and by a partly tautological validation of the error-decorrelation mechanism; these issues need to be addressed before the performance claims can be taken at face value.

major comments (4)
  1. [Section 6.1, Figure 5, Tables 2–3] The text states that α is selected via a coarse grid search over {0,0.1,...,1.0} and fixed across seeds, but it does not describe any validation split. Figure 5 plots relative performance versus α for each DMC task, and the APR values in Tables 2–3 appear to correspond to per-task optimal α chosen on the evaluation environments. If that is the case, the reported gains are the maximum over 11 candidates and do not estimate the performance of CRPI with a fixed, non-oracle α. Please specify the selection criterion explicitly; if α is tuned on test environments, report results with α chosen on a separate validation set or with a single pre-registered value across tasks. This is load-bearing for the paper's headline claim that CRPI substantially recovers lost performance.
  2. [Section 5.1, Eq. (21)] There is a sign inconsistency in the derivation. Substituting Eq. (17) into Eq. (19) gives (v_{k+1}[T] − v_{k+1}[0] − α(v_k[T]−v_k[0]))/T, which is what Eq. (20) states. However Eq. (21) rewrites the numerator as v_{k+1}[T] − (v_{k+1}[0] − α(v_k[T]−v_k[0])), which changes the sign of the α term. Eq. (22) is consistent with Eq. (19) but not with the printed Eq. (21). Correct Eq. (21) to use a plus sign before α. As written, the derivation does not logically lead to the update implemented in Algorithm 1.
  3. [Section 6.2, Figure 4] The claim that CRPI 'effectively suppresses cross-step error correlation' is circular for the residual-potential metric: CRPI explicitly constructs the modified residual as ε_{k+1} − α ε_k, so the cosine similarity between successive residual potentials is guaranteed to decrease as α increases. The non-trivial evidence is the action-level SNN Consistency and SNN Drift metrics, which are not mechanically tied to the definition of CRPI. Please rephrase the interpretation, and either drop the residual-potential panel or present it as a sanity check rather than as empirical support for the mechanism.
  4. [Section 5.1 and Algorithm 1] The derivation of Eq. (22) relies on the assumption that the final membrane potential is approximately uniform in (0,θ) and weakly dependent on initialization, so that changing v[0] by α(v_k[T]−v_k[0]) realizes the desired residual ε_{k+1} − α ε_k. However, Algorithm 1 clips both the carried residual and the initialized membrane potential to a finite range; when clipping is active, the actual correction is smaller than the linear form. The paper does not report how often clipping occurs or measure the discrepancy between the assumed and actual residual dynamics. Moreover, the AR(1) model with a single scalar α is assumed rather than tested. Please include an analysis of clipping frequency/effect and fit diagnostics for the AR(1) assumption, or verify empirically that Eq. (22) is accurately realized in the tested settings.
minor comments (5)
  1. [Section 5.2] 'Equitation (6)' should be 'Equation (6)'. Also 'v_l_k[T]−v_l_0[T]' should be 'v_l_k[T]−v_l_k[0]'.
  2. [Section 6.4, Table 3] '± denotes half an standard deviations' should be 'half a standard deviation'. Table 2 would benefit from confidence intervals or standard deviations, as provided for individual environments in Appendix B.2.
  3. [Figure 2(a)] The text states that R_SNN|ANN degradation is less than 0.5%, but the figure labels are not sufficiently clear; please label bars precisely so the reader can verify this claim.
  4. [Section 4.3] The cosine-similarity metrics in Table 1 are reported without error bars or significance tests. Since the paper's diagnosis rests on these correlations, please include variability estimates across seeds/rollouts.
  5. [References] The reference for OpenAI Gym is listed as 'Brockman, 2016'; the standard format is 'Brockman et al., 2016'. Please check citation formatting throughout.

Circularity Check

2 steps flagged

CRPI's correlation suppression is true by construction, and reported gains are selected per-task on the evaluation environments via alpha grid search, so the headline performance claim is partially fitted, not predicted.

specific steps
  1. self definitional [Section 5.1, Eqs. (18)-(22); Section 6.2, Fig. 4]
    "we consider a first-order approximation of residual error dynamics across consecutive decision steps: ε̃l k+1 = εl k+1 − α εl k, where α > 0 captures the empirically observed positive alignment between successive residual errors ... Increasing α reduces the expected cosine similarity in Eq. (18) and can drive it below zero."

    CRPI is derived from the ansatz ε̃_{k+1} = ε_{k+1} − α ε_k, and Eq. (22) implements exactly that subtraction in the initial potential. Therefore the cosine similarity cos(ε̃_{k+1}, ε_k) = (cos(ε_{k+1}, ε_k) − α·std(ε_k)/std(ε_{k+1}))·... is a decreasing linear function of α by the paper's own definition. Section 6.2 then reports this algebraic consequence as empirical evidence ('Figure 4 shows that the temporal correlation ... decreases monotonically as α increases, indicating that CRPI effectively suppresses cross-step error correlation'). This is measuring the defining identity, not an independent confirmation of the mechanism.

  2. fitted input called prediction [Section 6.1 (Evaluation Protocol); Section 6.4, Tables 2-3; Fig. 5]
    "The hyperparameter α is selected via a coarse grid search over {0, 0.1, 0.2, . . . , 0.9, 1.0} and is fixed across all seeds. ... Across all evaluated configurations, CRPI consistently improves the performance ratio compared to the corresponding baseline ANN-to-SNN conversion methods."

    The paper does not describe a validation split: the reported APR values in Tables 2 and 3 and Figure 5 correspond to the per-task α chosen on the evaluation environments themselves. With 11 grid candidates, the reported 'consistent improvements' are effectively the maximum over α per task, i.e., test-set tuning. The claim that 'CRPI substantially recovers lost performance' is therefore the fitted value of the selection procedure, not an out-of-sample prediction of CRPI with a fixed, non-oracle α. A held-out α or pre-registered value would be needed to support the quantitative headline claim independently.

full rationale

The paper's diagnosis of error amplification is an independent, non-circular empirical contribution: Section 4.1 decomposes return loss with counterfactual state distributions, and Section 4.3 measures cosine similarities of action errors. These analyses stand on their own. However, the proposed remedy has two load-bearing circular elements. First, CRPI's defining equation is ε̃_{k+1} = ε_{k+1} − α ε_k, so the monotone decrease in residual-membrane-potential correlation with α reported in Figure 4 is an algebraic identity, not evidence that the mechanism works. Second, the headline performance gains are obtained after per-task grid-search selection of α on the evaluation environments, with no validation split described, so the reported APR is a fitted maximum rather than a held-out prediction. The self-citations to Bu et al. (2022a,b) provide a standard uniform-potential assumption that is independently published and not by itself circular, so it does not raise the score further. Overall, the paper contains a valid analysis of the problem but the central claims of the proposed method—decorrelation and quantitative recovery—reduce partly by construction and partly by test-set fitting, yielding a partial circularity score of 6.

Axiom & Free-Parameter Ledger

1 free parameters · 5 axioms · 0 invented entities

The central contribution rests on one fitted scalar α and several assumptions about residual error behavior. There are no new physical entities or conserved quantities. The main fragility is the first-order residual error model and the proxy claim that membrane residual correlation mirrors action error correlation.

free parameters (1)
  • α (correlation coefficient in CRPI) = Grid search over {0, 0.1, ..., 1.0}; per-task selected value not reported
    α controls how much of the previous residual membrane potential is carried over. It is selected per environment/task by maximizing performance on the evaluation environments, so the reported gains are tuned rather than predicted.
axioms (5)
  • standard math Integrate-and-Fire neuron dynamics (Eqs. 1-5) and the standard rate-coding convergence result (Eq. 8) hold.
    The paper builds on the conventional IF model and the well-known averaging result for ANN-to-SNN conversion.
  • ad hoc to paper The residual error dynamics across decision steps follow a first-order autoregressive model: \tilde{ε}_{k+1} = ε_{k+1} − α ε_k (Eq. 19).
    This is the key modeling assumption behind CRPI. A single scalar α is assumed to capture the cross-step alignment of all residual errors in all layers; no empirical validation of this functional form is provided.
  • domain assumption The final membrane potential is approximately uniformly distributed in (0, θ) and weakly dependent on initialization.
    Used in Section 5.1 to justify adjusting the initial membrane potential as in Eq. (22); the assumption is attributed to Bu et al. 2022a and is not re-validated in the RL setting.
  • ad hoc to paper Positive temporal correlation of residual membrane potentials is a valid proxy for positive action-level error correlation.
    The paper states this in Section 5.1 and uses it to link the membrane-level correction to action-level error decorrelation, but the proxy relationship is not formally established.
  • domain assumption Closed-loop MDP dynamics amplify small per-step action errors into state distribution shift.
    This is the central phenomenon being claimed, supported by empirical trajectory divergence rather than proved; it is a structural property of the environments used.

pith-pipeline@v1.3.0-alltime-deepseek · 19966 in / 9250 out tokens · 100866 ms · 2026-08-03T06:49:45.733304+00:00 · methodology

0 comments
read the original abstract

Spiking Neural Networks (SNNs) can achieve competitive performance by converting already existing well-trained Artificial Neural Networks (ANNs), avoiding further costly training. This property is particularly attractive in Reinforcement Learning (RL), where training through environment interaction is expensive and potentially unsafe. However, existing conversion methods perform poorly in continuous control, where suitable baselines are largely absent. We identify error amplification as the key cause: small action approximation errors become temporally correlated across decision steps, inducing cumulative state distribution shift and severe performance degradation. To address this issue, we propose Cross-Step Residual Potential Initialization (CRPI), a lightweight gradient-free mechanism that carries over residual membrane potentials across decision steps to suppress temporally correlated errors. Experiments on continuous control benchmarks with both vector and visual observations demonstrate that CRPI can be integrated into existing conversion pipelines and substantially recovers lost performance. Our results highlight continuous control as a critical and challenging benchmark for ANN-to-SNN conversion, where small errors can be strongly amplified and impact performance. Code is available at https://github.com/xuzijie32/ANN2SNN-CRPI.

Figures

Figures reproduced from arXiv: 2601.21778 by Kang Chen, Wenxuan Liu, Yiting Dong, Zhaofei Yu, Zihan Huang, Zijie Xu.

Figure 1
Figure 1. Figure 1: Challenges of ANN-to-SNN conversion across different task categories. (a) Classification accuracy on ImageNet (Huang et al., 2025). (b) Average returns in discrete control tasks on Atari (Patel et al., 2019). (c) Relative returns in continuous control tasks, averaged over six environments from the DeepMind Control Suite. Additional results in the experimental section confirm that the per￾formance degradati… view at source ↗
Figure 2
Figure 2. Figure 2: Analysis of performance degradation in ANN-to-SNN conversion in the HalfCheetah-v4 environment. The ANN policy is trained with TD3 for 3 million environment steps and converted using IF neurons with 8 simulation steps. (a) Expected returns under different combinations of policies and state distributions. (b) t-SNE visualization of state trajectories induced by ANN and con￾verted SNN policies, revealing sig… view at source ↗
Figure 4
Figure 4. Figure 4: Cosine similarity of residual membrane potential and action errors across consecutive decision steps under different values of α. Results are obtained on MuJoCo environments using TD3 and IF neurons with 16 simulation steps. rect empirical evidence that CRPI decorrelates conversion￾induced errors across decision steps, addressing the root cause of error amplification identified in Section 4.3. 6.3. Enhanci… view at source ↗
Figure 6
Figure 6. Figure 6: Representative MuJoCo continuous control tasks used in our experiments. From left to right: Ant-v4, HalfCheetah-v4, Hopper-v4, and Walker2d-v4 [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Representative DeepMind Control Suite tasks used in our experiments. From left to right: acrobot swingup, cartpole swingup, cheetah run, finger spin, reacher easy, and quadruped walk. As shown in [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 14 linked inside Pith

  1. [2]

    TD3 and SAC have 256 units in both hidden layers. For image-based continuous control tasks in the DeepMind Control Suite, we adopt Data-Regularized Q-v2 (DrQ-v2) (Yarats et al., 2021b;a), a sample-efficient off-policy algorithm designed for high-dimensional visual observations. DrQ-v2 employs a convolutional encoder followed by an actor-critic architectur...

    2024

  2. [6]

    Reduc- ing ann-snn conversion error through residual membrane potential

    Hao, Z., Bu, T., Ding, J., Huang, T., and Yu, Z. Reduc- ing ann-snn conversion error through residual membrane potential. InProceedings of the AAAI conference on artificial intelligence, volume 37, pp. 11–21, 2023a. Hao, Z., Ding, J., Bu, T., Huang, T., and Yu, Z. Bridging the gap between anns and snns by calibrating offset spikes. arXiv preprint arXiv:23...

    Pith/arXiv arXiv

  3. [7]

    Table 5.State and action space dimensions for MuJoCo environments. ENVIRONMENTSTATEDIMENSIONACTIONDIMENSION ANT-V4 27 8 HALFCHEETAH-V4 17 6 HOPPER-V4 11 3 WALKER2D-V4 17 6 Tables 5 and 6 summarize the dimensionalities of state and action spaces for all evaluated environments. MuJoCo tasks use low-dimensional vector states, while DMC tasks rely on pixel ob...

    2015

  4. [8]

    and Zeng, Y

    Li, Y . and Zeng, Y . Efficient and accurate conversion of spiking neural network with burst spikes.arXiv preprint arXiv:2204.13271,

    Pith/arXiv arXiv

  5. [11]

    and Lee, Y

    10 Error Amplification Limits ANN-to-SNN Conversion in Continuous Control Oh, H. and Lee, Y . Sign gradient descent-based neuronal dynamics: Ann-to-snn conversion beyond relu network. arXiv preprint arXiv:2407.01645,

    Pith/arXiv arXiv

  6. [12]

    Fast and efficient information transmission with burst spikes in deep spik- ing neural networks

    Park, S., Kim, S., Choe, H., and Yoon, S. Fast and efficient information transmission with burst spikes in deep spik- ing neural networks. InProceedings of the 56th Annual Design Automation Conference 2019, pp. 1–6,

    2019

  7. [13]

    A low latency adaptive coding spiking framework for deep reinforcement learning.arXiv preprint arXiv:2211.11760,

    Qin, L., Yan, R., and Tang, H. A low latency adaptive coding spiking framework for deep reinforcement learning.arXiv preprint arXiv:2211.11760,

    Pith/arXiv arXiv

  8. [14]

    and Liu, S.-C

    Rueckauer, B. and Liu, S.-C. Conversion of analog to spik- ing neural networks using sparse temporal coding. In 2018 IEEE international symposium on circuits and sys- tems (ISCAS), pp. 1–5. IEEE,

    2018

  9. [15]

    Trust region policy optimization.arXiv preprint arXiv:1502.05477,

    Schulman, J. Trust region policy optimization.arXiv preprint arXiv:1502.05477,

    Pith/arXiv arXiv

  10. [16]

    Deep reinforcement learning with population-coded spiking neural network for continuous control

    Tang, G., Kumar, N., Yoo, R., and Michmizos, K. Deep reinforcement learning with population-coded spiking neural network for continuous control. InConference on Robot Learning, pp. 2016–2029. PMLR,

    2016

  11. [19]

    Mastering visual continuous control: Improved data-augmented re- inforcement learning.arXiv preprint arXiv:2107.09645, 2021a

    Yarats, D., Fergus, R., Lazaric, A., and Pinto, L. Mastering visual continuous control: Improved data-augmented re- inforcement learning.arXiv preprint arXiv:2107.09645, 2021a. Yarats, D., Kostrikov, I., and Fergus, R. Image aug- mentation is all you need: Regularizing deep reinforce- ment learning from pixels. InInternational Conference on Learning Repre...

    Pith/arXiv arXiv

  12. [20]

    Additional Experiments Details A.1

    12 Error Amplification Limits ANN-to-SNN Conversion in Continuous Control A. Additional Experiments Details A.1. Reinforcement Learning Environments We evaluate the proposed method on a diverse set of continuous control benchmarks covering both vector-based and vision- based observations. Specifically, we consider standard MuJoCo (Todorov et al., 2012; Todorov,

    2012

  13. [21]

    tasks from OpenAI Gymnasium (Brockman, 2016; Towers et al.,

    2016

  14. [22]

    These environments are widely used for benchmarking reinforcement learning algorithms

    and visual control tasks from the DeepMind Control Suite (DMC) (Tunyasuvunakool et al., 2020). These environments are widely used for benchmarking reinforcement learning algorithms. Figure 6.Representative MuJoCo continuous control tasks used in our experiments. From left to right: Ant-v4, HalfCheetah-v4, Hopper-v4, and Walker2d-v4. Figure 7.Representativ...

    2020

  15. [25]

    This work employs differential coding methods based on MT neurons. For linear layers, including fully connected and convolutional layers that can be represented by Equation (38), xl =W lxl−1 +b l,(38) where W l and bl is the weight and bias of layer l. Under differential coding in SNNs, this is equivalent to eliminating the bias termb l and initializing t...

    1982

  16. [2014]

    Playing atari with deep reinforcement learning

    Mnih, V . Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602,

    Pith/arXiv arXiv

  17. [2015]

    Deep reinforce- ment learning with spiking q-learning.arXiv preprint arXiv:2201.09754,

    Chen, D., Peng, P., Huang, T., and Tian, Y . Deep reinforce- ment learning with spiking q-learning.arXiv preprint arXiv:2201.09754,

    Pith/arXiv arXiv

  18. [2017]

    Soft actor- critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    9 Error Amplification Limits ANN-to-SNN Conversion in Continuous Control Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. Soft actor- critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. InInternational Con- ference on Machine Learning, pp. 1861–1870. PMLR, 2018a. Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha...

    Pith/arXiv arXiv 1912

  19. [2018]

    Proxy tar- get: Bridging the gap between discrete spiking neu- ral networks and continuous control.arXiv preprint arXiv:2505.24161, 2025a

    Xu, Z., Bu, T., Hao, Z., Ding, J., and Yu, Z. Proxy tar- get: Bridging the gap between discrete spiking neu- ral networks and continuous control.arXiv preprint arXiv:2505.24161, 2025a. Xu, Z., Shi, X., Dong, Y ., Huang, Z., and Yu, Z. Care- bn: Precise moving statistics for stabilizing spiking neu- ral networks in reinforcement learning.arXiv preprint arX...

    arXiv

  20. [2019]

    and Gu, S

    Deng, S. and Gu, S. Optimal conversion of conventional ar- tificial neural networks to spiking neural networks.arXiv preprint arXiv:2103.00476,

    Pith/arXiv arXiv

  21. [2020]

    Openai gym.arXiv preprint arXiv:1606.01540,

    Brockman, G. Openai gym.arXiv preprint arXiv:1606.01540,

    Pith/arXiv arXiv

  22. [2021]

    Continuous control with deep reinforcement learning.arXiv preprint arXiv:1509.02971,

    Lillicrap, T. Continuous control with deep reinforcement learning.arXiv preprint arXiv:1509.02971,

    Pith/arXiv arXiv

  23. [2022]

    Optimized potential initialization for low-latency spiking neural networks

    Bu, T., Ding, J., Yu, Z., and Huang, T. Optimized potential initialization for low-latency spiking neural networks. In Proceedings of the AAAI conference on artificial intelli- gence, volume 36, 2022a. Bu, T., Fang, W., Ding, J., Dai, P., Yu, Z., and Huang, T. Optimal ANN-SNN conversion for high-accuracy and ultra-low-latency spiking neural networks. InIn...

    Pith/arXiv arXiv

  24. [2024]

    Differential coding for training-free ann-to-snn conversion.arXiv preprint arXiv:2503.00301,

    Huang, Z., Fang, W., Bu, T., Xue, P., Hao, Z., Liu, W., Tang, Y ., Yu, Z., and Huang, T. Differential coding for training-free ann-to-snn conversion.arXiv preprint arXiv:2503.00301,

    Pith/arXiv arXiv

  25. [2025]

    A cat-like robot real-time learning to run

    Wawrzy´nski, P. A cat-like robot real-time learning to run. In Adaptive and Natural Computing Algorithms: 9th Inter- national Conference, ICANNGA 2009, Kuopio, Finland, April 23-25, 2009, Revised Selected Papers 9, pp. 380–

    2009