Pith. sign in

REVIEW 5 major objections 6 minor 42 references

An Interpretable Neural Control Network with Adaptable Online Learning for Sample Efficient Robot Locomotion Learning

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A 72-parameter interpretable controller, updated by a relevance-weighted rule, learns hexapod walking from zero-initialized weights in about 10 minutes on real hardware, with 40 percent fewer samples and 150 percent higher final reward…

desk verdict Physical from-scratch hexapod learning in ~10 minutes is the real result; the headline speedup versus CPGRBF-PIBB is plausible but not yet cleanly separated from tuning effort. read the letter →

arxiv 2501.10698 v1 pith:PEGWUII4 submitted 2025-01-18 cs.RO cs.LG

Arthicha Srisuchinnawong , Poramate Manoonpong This is my paper · ORCID
classification cs.ROcs.LG
keywords LocomotionlearningReinforcementExplainableartificialintelligenceNeuralcontrolSampleefficiencyOnlineHexapodrobotCentralpatterngenerator
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that interpretability can be used as a learning accelerator, not just an after-the-fact explanation tool. It proposes SME-AGOL, where a Sequential Motion Executor encodes a gait cycle as four sequential key poses realized by triangular basis signals, and an Adaptable Gradient-weighting Online Learning rule updates only the few output-mapping weights that strongly influence the current motor command. In simulation the authors report that SME-AGOL reaches a fixed walking-speed threshold with about 40 percent fewer trials than the CPGRBF-PIBB baseline and finishes with roughly 150 percent higher per-trial reward; on a physical hexapod it learns forward locomotion from all-zero parameters in about 10 minutes. If the comparison holds, the paper's thesis is that a small, analyzable controller with a sparse update rule can outperform large black-box policies on both sample efficiency and final performance.

What carries the argument

The machinery is the SME-AGOL update loop. SME is a three-layer, discrete-time controller whose first layer produces four sequentially propagating internal states, whose second layer shapes them into triangular bases with only neighbor overlap, and whose third layer maps the bases to 18 motor commands through a learned weight matrix; each basis is interpretable as a key pose. AGOL is the update rule $$\$\Delta$\$\theta$ \approx \eta_\$\theta$ \sum_\tau \sum_t \frac{|\nabla_{\tilde\$\theta$} \tilde a_t|\,(\tilde\theta_t-\$\theta$)}{\sigma_\$theta^{2}$} A_t,$$ with an analogous online adaptation of the exploration scale $\sigma_\theta$. The absolute gradient $|\nabla_{\tilde\theta} \tilde a_t|$ acts as a relevance mask: parameters that do not influence the emitted action get near-zero updates, so at any timestep only the weights of the currently active basis move. That sparse, phase-local update is what the paper identifies as the source of sample efficiency.

What would settle it

Run SME-AGOL on the same hexapod with the hand-selected SME constants varied around their reported values ($w_\tau = 0.05$, $w_{b_i c_n} = 0.5 w_\tau$, $w_{b_i c_m} = 0.25 w_\tau$) while keeping the learning rates fixed; if the episodes-to-0.2-reward advantage over CPGRBF-PIBB survives a wide range of those constants, the claimed mechanism, not the tuning, is the source.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that a controller with only 72 learnable parameters can learn a coordinated hexapod gait from scratch because the architecture and update rule conspire to keep almost all parameters irrelevant at any instant. The SME builds a gait cycle from four central-pattern-generator states, each low-pass filtered into a triangular basis that overlaps only with its neighbors; the four bases are interpreted as four key poses, and the only learned weights are the 18-by-4 matrix mapping bases to motor commands. AGOL then weights the parameter update by the absolute gradient of the current motor commands with respect to the explored parameters, so a parameter that does not shape the current action receives a near-zero update, and the exploration variance is adapted online by a companion gradient rule. The authors measure that SME-AGOL reaches a per-episode walking reward of 0.2 in 35 simulated episodes, versus 60 episodes for CPGRBF-PIBB, and ends at an average final reward of 0.58 versus 0.23; on the physical robot, ten repetitions from zero-initialized weights reach about 5 cm/s after roughly 100 episodes and slightly above 8 cm/s at 200 episodes.

Load-bearing premise

The argument rests on the assumption that the 40 percent sample reduction and 150 percent reward gain come from the triangular bases and relevance-weighted updates, rather than from the hand-chosen SME constants or the grid-searched learning rates, since the paper does not test what happens when those fixed values are changed.

Editorial extensions

If this is right

  • If the reported comparison holds, learning a new gait on a physical robot no longer requires massive simulated datasets or sim-to-real transfer; a zero-initialized controller can reach usable walking speeds in minutes using only a speed reward.
  • Architectural interpretability is doing optimization work, so future controller design should favor separable units with clear per-neuron meanings rather than dense end-to-end networks.
  • Because only the 72 output-mapping weights are learned, the resulting policy is small enough to inspect: the weight matrix can be read as four learned key poses, and gait frequency can be adjusted after training through the single parameter $w_\tau$.
  • The online adaptation of exploration variance lets the same run shift focus between leg lifting and leg swinging as learning proceeds, which points toward policies that keep adapting after deployment instead of freezing at a fixed gait.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extension left implicit in the paper is that the relevance-gated update should transfer to any periodic control task with a low-dimensional action space, not just legged locomotion, because the gating only requires that few parameters be active at each phase.
  • If the triangular-basis interference reduction is the true cause of the gain, controllers using more orthogonal basis functions should show proportionally larger sample-efficiency improvements from the same AGOL rule, a scaling prediction that could be tested by varying basis width and overlap.
  • Because intralimb and interlimb coordination live in the same learned mapping matrix, a natural next experiment is to keep the SME bases fixed and re-learn only $W^o_b$ on a second robot, isolating how much of a learned gait transfers across platforms.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces SME-AGOL, a two-part controller/learning system for legged locomotion. SME is a three-layer interpretable network with sequential central pattern generator states, triangular basis functions, and a learned output mapping; AGOL is an online policy-gradient-style rule that weights parameter updates by the magnitude of the action gradient with respect to each parameter. The central quantitative claims are that SME-AGOL needs 40% fewer samples and achieves 150% higher final reward than CPGRBF-PIBB on a simulated hexapod, and that it learns to walk on a physical hexapod from zero-initialized parameters in about 10 minutes. The paper supports these claims with a 20-condition simulation comparison, learning curves, statistical tests, and a physical robot experiment repeated 10 times.

Significance. If the comparative claims held, this would be a noteworthy result: an interpretable, low-parameter policy class combined with a simple relevance-weighted update achieving locomotion learning on physical hardware in minutes, without sim-to-real transfer. The physical robot demonstration with 10 repetitions, the zero-init learning, and the comparison to a manually designed controller are concrete strengths. The paper also provides useful interpretability analyses (key poses, basis overlap, reward landscapes) and makes its videos available. However, as presented, the headline speedup claims are not fully protected against alternative explanations involving hyperparameter tuning and the absence of key ablations, so the significance is conditional on those issues being resolved.

major comments (5)
  1. [Section 3.1] The simulation comparison is load-bearing, but Section 3.1 states that "the hyperparameters of each testing condition were obtained from a grid search" without reporting the grid ranges or the selected values for any of the 20 conditions. Because the five learning algorithms have different exploration schedules and learning rates, a per-method, per-condition grid search can encode large performance differences that are unrelated to the SME architecture or the AGOL weighting. The authors should report the full hyperparameter table and, ideally, a sensitivity analysis showing that the reported SME-AGOL advantages are not driven by favorable hyperparameter choices.
  2. [Table 1] The learning-speed threshold of 0.2 is partly post hoc: the table caption defines it as "the reward value achievable within the first 100 episodes by half of the methods tested in this work." Choosing the threshold after observing which methods cross it weakens the claim that the episode counts measure sample efficiency. The authors should either justify the 0.2 threshold independently of the results (e.g., as a fixed fraction of a physically meaningful speed) or report results for several thresholds, such as 0.15, 0.2, and 0.25, to show that the ranking is robust.
  3. [Section 2.2, Eq. (8)] The substitution of the relevance score |Rel_theta| by the action-gradient magnitude |nabla_theta a_t| in Eq. (8) is asserted rather than derived. These quantities are not equivalent in general: a parameter can have a large action gradient without being relevant to the return, and relevance propagation as introduced in [32] is defined through a different conservation property. Since the proposed mechanism is exactly this weighting, the paper should either provide a formal relationship between |Rel_theta| and |nabla_theta a_t| under the SME structure, or empirically validate the substitution, for example by comparing Eq. (8) with a variant using an actual relevance-propagation computation.
  4. [Section 3.1, Table 1] There is no ablation that isolates the AGOL weighting term. Setting |nabla_theta a_t| to a constant in Eq. (8) reduces AGOL to a PIBB-style parameter-exploring update, yet this control condition is not reported. Given that SME-PIBB+ reaches 0.29 while SME-AGOL reaches 0.58, the claimed improvement could come from the triangular bases alone, from the weighting, or from their interaction. Reporting SME with PIBB, SME with constant-weight AGOL, and SME with full AGOL would separate these contributions and is essential to support the mechanistic story.
  5. [Table 1] Table 1 reports final rewards as point estimates only, without standard deviations, confidence intervals, or per-trial values, even though each condition was repeated 10 times. The text cites p-values from Mann-Whitney U tests, but p-values alone do not convey the magnitude or variability of the 150% final-reward difference. The authors should add error bars or a confidence interval to Table 1 (or to a companion table) so that the reader can assess whether the reported differences are robust across repetitions.
minor comments (6)
  1. [Section 2.1.1] In the sentence describing Fig. 1a, "radish/yellowish" should be "reddish/yellowish."
  2. [Section 3.1] The text says "Mann-Wallis test" in one place; this should be "Mann-Whitney U test."
  3. [Section 3.1] The sentence beginning "In this experiment, the comparison was performed using two matrices" should read "two metrics."
  4. [Section 3.2] There is a typo in "receptively" near Fig. 11; it should be "respectively."
  5. [Section 2.2] The reference to "Section II-B" should be "Section 2.2" to match the paper's own numbering style.
  6. [Table 2] The table lists a simulated quadruped row for SME-AGOL, but the body text does not describe this condition; either provide details or remove the row to avoid unverifiable entries.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the SME-AGOL claims are supported by controlled re-implemented baselines and a physical experiment, not by definitional equivalence or a self-citation chain.

full rationale

The derivation chain is self-contained. SME's CPG weights are obtained by solving the linear boundary system in Eq. (2) for chosen constants (gamma=0.5, omega=8, iota=0.95, epsilon=0.01), and the basis parameters are then fixed; the AGOL update in Eqs. (5)-(9) follows the standard parameter-exploring policy-gradient algebra (PGPE/PIBB) with an added |gradient| factor, and the 'relevance' label is a description of that factor rather than an independently fitted quantity. The headline numbers (150% final reward, 40% fewer samples) are computed from Table 1, where all five learning algorithms were cross-implemented in this paper on the same simulator, reward, and episode length, so the comparison to CPGRBF-PIBB [14] is a re-measured baseline rather than an imported result. The 0.2 reward threshold is defined from a physical speed calculation (40% of Vmax) even though the table caption also notes it was achievable by half the methods. The main weaknesses - unreported grid-search hyperparameter values, hand-fixed SME constants without sensitivity analysis, and no AGOL ablation - are reporting/robustness concerns, not circularity: none of the claimed predictions reduces by construction to a fitted input or to a self-citation. Self-citations to [14] (CPGRBF baseline and the 'eight episodes' batch size) are not load-bearing for the derivation because the baseline values are regenerated here and the batch-size choice is an experimental setting, not a premise that entails the result.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The method is not a derivation from first principles; it is an engineered controller whose fixed parameters, grid-searched hyperparameters, and several unverified statistical assumptions carry much of the reported performance. The absence of the grid-search values is the largest gap in the ledger.

free parameters (7)
  • wτ (basis lowpass coefficient) = 0.05
    Controls transition speed and walking frequency (0.3 Hz); selected by hand in Section 2.1.2, not learned.
  • wbi_cn and wbi_cm (basis shaping weights) = 0.5wτ and 0.25wτ
    Chosen empirically to reduce interference between non-neighbor bases; described in Section 2.1.2 and Figure 2.
  • CPG boundary parameters γ, ω, ι, ε = γ=0.5, ω=8, ι=0.95, ε=0.01
    Chosen so the fixed CPG weights solve Eq. 2; these define the internal state dynamics and are not learned.
  • Per-condition grid-search hyperparameters for 20 simulation conditions = not reported
    Section 3.1 states all hyperparameters were obtained from a grid search but the selected values are not listed; this is fitting to the test task without a held-out check.
  • Online update window and episode length = 8 episodes, 70 timesteps
    Window size taken from prior work [14]; episode length is fixed at 70, about one gait cycle.
  • Initial and adapted exploration standard deviation σθ = not reported
    σθ appears throughout Eqs. 6-9 and Eq. 9 adapts it, but its initialization and update schedule are not specified.
  • Physical experiment learning rates ηθ and ησ = ηθ=0.5, ησ=0.1
    Reported as empirically chosen in Section 3.2.
assumptions (5)
  • domain assumption Deterministic environment for policy gradient: ∇θ ln p(s0)=0 and ∇θ ln p(st+1|st,ãt)=0.
    Invoked in Section 2.2 to derive Eq. 5; if the environment is stochastic, the gradient estimate changes.
  • domain assumption Parameter-space exploration determinism: p(ãt|θ̃t) is deterministic given explored parameters θ̃t, so ∇θ ln p(ãt|θ̃t)=0.
    Used to go from Eq. 5 to Eq. 6 following PGPE [24].
  • ad hoc to paper Relevance |Rel_θ̃| can be replaced by action-gradient magnitude |∇_θ̃ ãt|.
    Substituted in Eq. 8 for PyTorch autograd convenience; no derivation is given that this preserves the intended relevance ordering.
  • domain assumption Advantage estimator A_t is an unbiased estimate of the true policy-gradient direction.
    Used in Eqs. 7-9 with normalization by mean/std or by a learned baseline; small sample sizes (8 episodes) make unbiasedness questionable.
  • domain assumption Hand-fixed CPG and basis parameters produce a smooth enough reward landscape for sparse updates to converge.
    Underlies the claim that interpretability causes sample efficiency; this is never ablated in the experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Interpretable Neural Control Network with Adaptable Online Learning for Sample Efficient Robot Locomotion Learning." pith.science (2026). https://pith.science/paper/PEGWUII4

@misc{pith2026250110698,
  author       = {Pith},
  title        = {Pith review of: An Interpretable Neural Control Network with Adaptable Online Learning for Sample Efficient Robot Locomotion Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PEGWUII4}},
  note         = {Machine review of arXiv:2501.10698}
}
read the original abstract

Robot locomotion learning using reinforcement learning suffers from training sample inefficiency and exhibits the non-understandable/black-box nature. Thus, this work presents a novel SME-AGOL to address such problems. Firstly, Sequential Motion Executor (SME) is a three-layer interpretable neural network, where the first produces the sequentially propagating hidden states, the second constructs the corresponding triangular bases with minor non-neighbor interference, and the third maps the bases to the motor commands. Secondly, the Adaptable Gradient-weighting Online Learning (AGOL) algorithm prioritizes the update of the parameters with high relevance score, allowing the learning to focus more on the highly relevant ones. Thus, these two components lead to an analyzable framework, where each sequential hidden state/basis represents the learned key poses/robot configuration. Compared to state-of-the-art methods, the SME-AGOL requires 40% fewer samples and receives 150% higher final reward/locomotion performance on a simulated hexapod robot, while taking merely 10 minutes of learning time from scratch on a physical hexapod robot. Taken together, this work not only proposes the SME-AGOL for sample efficient and understandable locomotion learning but also emphasizes the potential exploitation of interpretability for improving sample efficiency and learning performance.

Figures

Figures reproduced from arXiv: 2501.10698 by the authors.

Figure 1
Figure 1. (a) An overview of the Sequential Motion Executor-Adaptable Gradient-weighting Online Learning (SME [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. (a) Central pattern generator neurons/internal states ( [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Online locomotion learning process. Firstly, the robot interacts with the environment through the use of [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Mann-Whitney U test p-value corresponding to all comparison pairs in terms of (a) the final episodic reward [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Average episodic rewards, i.e., learning curves and the corresponding min-max range (shade) obtained from [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Average episodic rewards, i.e., learning curves and the corresponding min-max range (shade) obtained [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Five examples of the explored robot joint trajectories during the first learning episodes obtained from (left) an [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Reward landscape [40] estimated along the update direction ∆θ of (left) CPGRBF-AGOL, (middle) SME￾AGOL, and (right) SME-PGPE+ , where the parameter θ = Wo b . To further analyze the accuracy of gradient estimation, [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Snapshots from a trial of physical robot locomotion learning (a) during the first 40 episodes and (b) between [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Average episodic reward obtained from 10 repetitions of physical robot locomotion learning, along with the [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Graphical representation of the top view robot kinematics and evolution of the adaptive exploration rates [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 32 canonical work pages

  1. [32]

    Layer- wise relevance propagation: an overview

    Grégoire Montavon, Alexander Binder, Sebastian Lapuschkin, Wojciech Samek, and Klaus-Robert Müller. Layer- wise relevance propagation: an overview. Explainable AI: interpreting, explaining and visualizing deep learning, pages 193–209, 2019

    work page 2019

  2. [1]

    Deep learning

    Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016

    2016

  3. [2]

    Adaptive modular neural control for online gait synchronization and adaptation of an assistive lower-limb exoskeleton

    Arthicha Srisuchinnawong, Chaicharn Akkawutvanich, and Poramate Manoonpong. Adaptive modular neural control for online gait synchronization and adaptation of an assistive lower-limb exoskeleton. IEEE Transactions on Neural Networks and Learning Systems, Early Access, 2023

    work page 2023

  4. [3]

    Adaptive bipedal robot walking on industrial pipes under neural multimodal locomotion control: Toward robotic out-pipe inspection

    Arthicha Srisuchinnawong, Kitti Phongaksorn, Wasuthorn Ausrivong, and Poramate Manoonpong. Adaptive bipedal robot walking on industrial pipes under neural multimodal locomotion control: Toward robotic out-pipe inspection. IEEE/ASME Transactions on Mechatronics, Early Access, 2023

    work page 2023

  5. [4]

    why should i trust you?

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, 2016

    2016

  6. [5]

    Multi-expert learning of adaptive legged locomotion

    Chuanyu Yang, Kai Yuan, Qiuguo Zhu, Wanming Yu, and Zhibin Li. Multi-expert learning of adaptive legged locomotion. Science Robotics, 5(49):2174, 2020

    work page 2020

  7. [6]

    Rapid locomotion via reinforcement learning

    Gabriel B Margolis, Ge Yang, Kartik Paigwar, Tao Chen, and Pulkit Agrawal. Rapid locomotion via reinforcement learning. The International Journal of Robotics Research, 43(4):572–587, 2024

    2024

  8. [7]

    Learning quadrupedal locomotion on deformable terrain

    Suyoung Choi, Gwanghyeon Ji, Jeongsoo Park, Hyeongjun Kim, Juhyeok Mun, Jeong Hyun Lee, and Jemin Hwangbo. Learning quadrupedal locomotion on deformable terrain. Science Robotics, 8(74):eade2256, 2023

    work page 2023

Show all 42 references
  1. [8]

    Learning to walk in minutes using massively parallel deep reinforcement learning

    Nikita Rudin, David Hoeller, Philipp Reist, and Marco Hutter. Learning to walk in minutes using massively parallel deep reinforcement learning. In Aleksandra Faust, David Hsu, and Gerhard Neumann, editors, Proceedings of the 5th Conference on Robot Learning, volume 164 of Proc...

    2022

  2. [9]

    Deep reinforcement learning amidst lifelong non-stationarity

    Annie Xie, James Harrison, and Chelsea Finn. Deep reinforcement learning amidst lifelong non-stationarity. In 4th Lifelong Machine Learning Workshop at ICML 2020, 2020

    2020

  3. [10]

    Hliferl: A hierarchical lifelong reinforcement learning framework

    Fan Ding and Fei Zhu. Hliferl: A hierarchical lifelong reinforcement learning framework. Journal of King Saud University-Computer and Information Sciences, 34(7):4312–4321, 2022

    2022

  4. [11]

    Legged robots that keep on learning: Fine-tuning locomotion policies in the real world

    Laura Smith, J Chase Kew, Xue Bin Peng, Sehoon Ha, Jie Tan, and Sergey Levine. Legged robots that keep on learning: Fine-tuning locomotion policies in the real world. In 2022 International Conference on Robotics and Automation (ICRA), pages 1593–1599. IEEE, 2022

    2022

  5. [12]

    Modular deep reinforcement learning for emergent locomotion on a six-legged robot

    Malte Schilling, Kai Konen, and Timo Korthals. Modular deep reinforcement learning for emergent locomotion on a six-legged robot. In 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob), pages 946–953. IEEE, 2020

    2020

  6. [13]

    Deepcpg policies for robot locomotion

    Aditya M Deshpande, Eric Hurd, Ali A Minai, and Manish Kumar. Deepcpg policies for robot locomotion. IEEE Transactions on Cognitive and Developmental Systems, 15:2105–2121, 2023

    2023

  7. [14]

    Generic neural locomotion control framework for legged robots

    Mathias Thor, Tomas Kulvicius, and Poramate Manoonpong. Generic neural locomotion control framework for legged robots. IEEE transactions on Neural Networks and Learning Systems, 32(9):4013–4025, 2020

    2020

  8. [15]

    Experience-learning inspired two-step reward method for efficient legged locomotion learning towards natural and robust gaits.arXiv preprint arXiv:2401.12389, 2024

    Yinghui Li, Jinze Wu, Xin Liu, Weizhong Guo, and Yufei Xue. Experience-learning inspired two-step reward method for efficient legged locomotion learning towards natural and robust gaits.arXiv preprint arXiv:2401.12389, 2024

    2024 arXiv

  9. [16]

    Fully spiking actor network with intralayer connections for reinforcement learning

    Ding Chen, Pfeixi Peng, Tiejun Huang, and Yonghong Tian. Fully spiking actor network with intralayer connections for reinforcement learning. IEEE Transactions on Neural Networks and Learning Systems, Early Access, 2024

    2024

  10. [17]

    Dreamwaq: Learning robust quadrupedal locomotion with implicit terrain imagination via deep reinforcement learning

    I Made Aswin Nahrendra, Byeongho Yu, and Hyun Myung. Dreamwaq: Learning robust quadrupedal locomotion with implicit terrain imagination via deep reinforcement learning. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 5078–5084. IEEE, 2023

    2023

  11. [18]

    Learning locomotion for quadruped robots via distributional ensemble actor-critic

    Sicen Li, Yiming Pang, Panju Bai, Jiawei Li, Zhaojin Liu, Shihao Hu, Liquan Wang, and Gang Wang. Learning locomotion for quadruped robots via distributional ensemble actor-critic. IEEE Robotics and Automation Letters, 9, 2024

    2024

  12. [19]

    Viability leads to the emergence of gait transitions in learning agile quadrupedal locomotion on challenging terrains

    Milad Shafiee, Guillaume Bellegarda, and Auke Ijspeert. Viability leads to the emergence of gait transitions in learning agile quadrupedal locomotion on challenging terrains. Nature Communications, 15(1):3073, 2024

    2024

  13. [20]

    Reinforcement learning: An introduction

    Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018

    2018

  14. [21]

    Anymal parkour: Learning agile navigation for quadrupedal robots

    David Hoeller, Nikita Rudin, Dhionis Sako, and Marco Hutter. Anymal parkour: Learning agile navigation for quadrupedal robots. Science Robotics, 9(88):eadi7566, 2024

    2024

  15. [22]

    Learning to walk: Spike based rein- forcement learning for hexapod robot central pattern generation

    Ashwin Sanjay Lele, Yan Fang, Justin Ting, and Arijit Raychowdhury. Learning to walk: Spike based rein- forcement learning for hexapod robot central pattern generation. In 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), pages 208...

    2020

  16. [23]

    Simple statistical gradient-following algorithms for connectionist reinforcement learning

    Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Reinforcement learning, pages 5–32, 1992

    1992

  17. [24]

    Parameter-exploring policy gradients

    Frank Sehnke, Christian Osendorfer, Thomas Rückstieß, Alex Graves, Jan Peters, and Jürgen Schmidhuber. Parameter-exploring policy gradients. Neural Networks, 23(4):551–559, 2010

    2010

  18. [25]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  19. [26]

    Policy improvement methods: Between black-box optimization and episodic reinforcement learning

    Freek Stulp and Olivier Sigaud. Policy improvement methods: Between black-box optimization and episodic reinforcement learning. hal-00738463, 2012

    2012

  20. [27]

    A survey on interpretable reinforcement learning

    Claire Glanois, Paul Weng, Matthieu Zimmer, Dong Li, Tianpei Yang, Jianye Hao, and Wulong Liu. A survey on interpretable reinforcement learning. Machine Learning, pages 1–44, 2024

    2024

  21. [28]

    Neurovis: real-time neural information measurement and visualization of embodied neural systems

    Arthicha Srisuchinnawong, Jettanan Homchanthanakul, and Poramate Manoonpong. Neurovis: real-time neural information measurement and visualization of embodied neural systems. Frontiers in Neural Circuits, 15:743101, 2021

    2021

  22. [29]

    Versatile modular neural locomotion control with fast learning

    Mathias Thor and Poramate Manoonpong. Versatile modular neural locomotion control with fast learning. Nature Machine Intelligence, 4(2):169–179, 2022

    2022

  23. [30]

    Continuous online adaptation of bioinspired adaptive neuroendocrine control for autonomous walking robots

    Jettanan Homchanthanakul and Poramate Manoonpong. Continuous online adaptation of bioinspired adaptive neuroendocrine control for autonomous walking robots. IEEE Transactions on Neural Networks and Learning Systems, 33(5):1833–1845, 2021. 17 An Interpretable Neural Control Net...

    2021

  24. [31]

    Designing worm- inspired neural networks for interpretable robotic control

    Mathias Lechner, Ramin Hasani, Manuel Zimmer, Thomas A Henzinger, and Radu Grosu. Designing worm- inspired neural networks for interpretable robotic control. In 2019 International Conference on Robotics and Automation (ICRA), pages 87–94. IEEE, 2019

    2019

  25. [33]

    Striving for simplicity: The all convolutional net

    Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. In ICLR (workshop track), 2015

    2015

  26. [34]

    Morf—modular robot framework

    Mathias Thor, Jørgen Christian Larsen, and Poramate Manoonpong. Morf—modular robot framework. In Proc. 2nd Int. Youth Conf. Bionic Eng.(IYCBE), pages 21–23, 2018

    2018

  27. [35]

    V-rep: A versatile and scalable robot simulation framework

    Eric Rohmer, Surya PN Singh, and Marc Freese. V-rep: A versatile and scalable robot simulation framework. In 2013 IEEE/RSJ international conference on intelligent robots and systems, pages 1321–1326. IEEE, 2013

    2013

  28. [36]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems, pages 5026–5033. IEEE, 2012

    2012

  29. [37]

    Correlated Exploration in Deep Reinforcement Learning

    Maximilian Hensel. Correlated Exploration in Deep Reinforcement Learning. PhD thesis, Technische Universität Darmstadt, 2019

    2019

  30. [38]

    Parameter space noise for exploration.arXiv preprint arXiv:1706.01905, 2017

    Matthias Plappert, Rein Houthooft, Prafulla Dhariwal, Szymon Sidor, Richard Y Chen, Xi Chen, Tamim Asfour, Pieter Abbeel, and Marcin Andrychowicz. Parameter space noise for exploration.arXiv preprint arXiv:1706.01905, 2017

    2017 arXiv

  31. [39]

    Automatic differentiation in pytorch

    Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In Conference on Neural Information Processing Systems, 2017

    2017

  32. [40]

    A closer look at deep policy gradients

    Andrew Ilyas, Logan Engstrom, Shibani Santurkar, Dimitris Tsipras, Firdaus Janoos, Larry Rudolph, and Alek- sander Madry. A closer look at deep policy gradients. In International Conference on Learning Representations, 2020

    2020

  33. [41]

    Grad-cam: Visual explanations from deep networks via gradient-based localization

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618–626, 2017

    2017

  34. [42]

    Improvement in deep networks for optimization using explainable artificial intelligence

    Jin Ha Lee, Ik hee Shin, Sang gu Jeong, Seung-Ik Lee, Muhamamad Zaigham Zaheer, and Beom-Su Seo. Improvement in deep networks for optimization using explainable artificial intelligence. In 2019 International Conference on Information and Communication Technology Convergence (I...

    2019

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.