Pith. sign in

REVIEW 5 major objections 5 minor 80 references

Growable and Interpretable Neural Control with Online Continual Learning for Autonomous Lifelong Locomotion Learning Machines

T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GOLLUM claims a real hexapod can learn new gaits continuously, keep old ones, and reuse them, all without pretraining or task context.

desk verdict A real-robot continual learning result with real strengths, but the 'full action space' claim does not survive contact with its own equations; worth peer review, not worth taking at face value. read the letter →

arxiv 2505.12029 v1 pith:X4MSCAOL submitted 2025-05-17 cs.RO

Arthicha Srisuchinnawong , Poramate Manoonpong This is my paper · ORCID
classification cs.RO
keywords continuallearninglocomotioncentralpatterngeneratorinterpretableneuralcontrolneurogenesisreinforcementhexapodrobotcatastrophicforgetting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GOLLUM (Growable Online Locomotion Learning Under Multicondition) is a control-and-learning framework that lets a real hexapod robot acquire new locomotion skills while it is running, keep the skills it has already learned, and reuse those skills to make later learning faster. The paper's central claim is that interpretability, not more capacity or more data, is what resolves the four standard problems of lifelong locomotion learning: sample inefficiency, lack of knowledge reuse, catastrophic forgetting, and incomprehensibility. On a physical hexapod, GOLLUM learned walking, slope climbing, and gaits for deformable terrains from scratch within roughly 100 to 200 episodes, about ten to twenty minutes per skill, using a single simple reward term. It later combined previously learned skills to improve performance in new conditions, and the paper claims this makes GOLLUM, among the compared approaches, the only one that realizes online continual locomotion learning in the real world without task context or human intervention in the tested setting.

What carries the argument

The load-bearing object is the interpretable neural control network, a discrete-time non-spiking network organized in two dimensions at once. Horizontally, each column is a ring-like central pattern generator (CPG) subnetwork, a loop of neurons that keeps producing rhythmic patterns without external input, and each column is meant to encode one primary skill. Vertically, seven layers form four functional modules: feedback and two preprocessing layers ($FB$, $I'$, $I$) classify and gate the current condition; a sequential central pattern generator layer ($C$) emits discrete internal states; a basis layer ($B$) smooths them into sparse triangular signals; a premotor layer ($PM$) holds shared action patterns; and an output layer ($M$, $V$, $O$) maps patterns to motor commands, value predictions, and observation predictions. Three mechanisms carry the argument: primary learning updates only the active column's $PM\to M$ weights, so inactive skills stay untouched; supplementary learning adjusts the $B\to PM$ weights that blend skills; and neurogenesis, triggered when the return falls below the value-prediction boundary while an observation exceeds its prediction boundary, appends a column initialized from the most similar existing skill. The claimed full action space is preserved because the ring structure imposes a rhythmic prior while the bases are combined linearly, and the claimed interpretability follows from the sparse structure, where the connection matrix $\kappa$ displays which behaviors transition to which.

What would settle it

Run GOLLUM on a terrain whose optimal gait requires an inter-leg phase pattern that is not generated by the four-action ring sequence, for example a prescribed asymmetric tripod. If the controller either fails to reach the reward level achieved by a directly parameterized gait controller, or needs neurogenesis to keep adding columns without converging, the assumption that the fixed triangular bases span a full action space is falsified.

Watch

Extended reading notes

Core claim

The central claim is that lifelong locomotion learning can be solved by making the controller itself interpretable rather than by adding complicated memory or regularization mechanisms. On the paper's own terms, GOLLUM's contribution is the combination of column-wise skill encoding, layer-wise function encoding, a dual-layer learning rule, and neurogenesis-driven network growth. The experiments report that this combination lets a physical hexapod learn walking from scratch on level ground in about 200 episodes, roughly ten minutes, acquire separate skills for slopes, a frozen joint, and soft or rough terrains in about an hour, recall earlier skills after later training without performance loss, and improve a recalled level-ground skill by about 40 percent by mixing in slope skills. The Discussion then states the strongest form of the claim: GOLLUM is the only locomotion learning framework that achieves online continual locomotion learning in the real world without catastrophic forgetting while exploiting task similarity during both direct knowledge transfer and learning, and without task context or human intervention, under unlimited space.

Load-bearing premise

The whole scheme depends on the idea that the hand-built ring network, with four action phases per skill and manually chosen smoothing parameters, can still produce every movement pattern the robot might need. If a useful gait cannot be written as a weighted sum of those fixed triangular bases, then the promise of learning diverse new skills without re-parameterizing the gait fails.

Editorial extensions

If this is right

  • A legged robot can be deployed without simulation pretraining and still acquire new skills on the fly: GOLLUM reports usable gaits from scratch in roughly 100 to 200 episodes, about ten to twenty minutes per condition on the real robot.
  • Skills learned earlier remain recallable after later training: when the robot returned to a level floor after climbing a 25-degree slope, it recovered its earlier flat-ground gait instead of the sliding-backward behavior learned on the slope.
  • New conditions can be learned faster by reusing old skills: initializing a new column from the most similar prior skill, plus supplementary blending, let the robot combine a slope skill and a motor-dysfunction skill to climb a slope with a frozen joint in about 30 episodes.
  • Because the learned controller is interpretable, a designer can read the behavior hierarchy from the connection matrix and modify parameters such as the frequency parameter without retraining, and post-hoc decision-tree explanations of GOLLUM are reported to be smaller and lower-error than those for the compared deep networks.
  • The same core network is claimed to transfer to other embodiments and tasks: changing the output dimension from 18 to 12 joints adapts it to a quadruped, and replacing the locomotion reward with a fitting error gives a programming-by-demonstration method.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The dual-layer separation, per-skill primary weights plus a small mixing layer, is arguably a general recipe for lifelong learning: it avoids forgetting by construction rather than by regularization or replay, so it could be ported to other control problems such as manipulation or navigation wherever an interpretable rhythmic prior is available.
  • The prediction-boundary trigger for neurogenesis is an intrinsic novelty signal; a natural extension is to compare GOLLUM against common intrinsic-motivation bonuses such as prediction-error or count-based exploration, and to test whether the explicit two-channel surprise, value plus observation, is necessary or whether either channel alone suffices.
  • The strongest claim is explicitly qualified as holding under unlimited space; a stress test that would bound the claim is to check how skill count, memory, and learning time scale as the number of conditions grows, since the paper demonstrates five to six skills within an hour but does not establish an unbounded regime.
  • Because the basis-layer parameters are partly selected empirically, one could ask whether learning those parameters online, while keeping the sparsity structure, improves generalization to terrains with very different gait dynamics, or whether it breaks the layer-wise interpretability the approach relies on.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The manuscript introduces GOLLUM, a modular interpretable neural controller for legged robots that grows new CPG-like ring subnetworks when online-fitted value and observation prediction boundaries are exceeded. Primary learning updates PM-to-M weights to shape motor commands for the active skill, while supplementary learning updates B-to-PM weights to recombine previously learned action patterns. Experiments on the MORF hexapod report flat-ground walking from scratch in about 200 episodes, continual acquisition of slope, deformable-terrain, and motor-dysfunction skills within about an hour, recall of old skills, and ablations suggesting that neurogenesis and supplementary learning prevent catastrophic forgetting and exploit similarity. The paper claims that GOLLUM is the only real-world locomotion learning framework that achieves online continual learning without pretraining or task context.

Significance. If the empirical claims hold, GOLLUM is a valuable contribution: it demonstrates that a strongly structured, interpretable controller can be grown online on physical hardware, with substantially better sample efficiency than the reported deep RL baselines, and it provides a concrete mechanism—separate primary and supplementary weight spaces—by which new skills can be added without interfering with old ones. The provision of videos and open code is a strength, as are the quantitative compactness and fidelity comparisons. The significance is currently capped by two gaps: the lack of a formal characterization of the representable motor-command space, and the under-specified statistics for the continual-learning ablations; both need to be fixed before the strongest claims can be accepted.

major comments (5)
  1. [Sequential Central Pattern Generator (C), Eqs. (5)–(7)] The claim in this section that GOLLUM 'incorporates a rhythmic prior ... while maintaining a full action space' is not supported by the equations. From Eqs. (5)–(7), every motor command is a linear combination of the triangular basis signals B_k[t], because PM is a linear map of B and M is a linear map of PM. With four C neurons and four B bases per subnetwork, a single active subnetwork spans at most four temporal basis functions, and the off-diagonal weights w_{PM_i,B_k} merely recombine those same bases without enlarging the temporal shape space. The paper therefore has not established that arbitrary leg-coordination patterns (e.g., a hexapod wave gait with six independent phase offsets) can be represented. Please either prove the coverage claim for the relevant gait class, or replace 'full action space' with a precise statement of the low-dimensional subspace that the architecture can represent and explain why that subspace suffices for the demonstrated skills.
  2. [Basis (B), Eq. (5)] The basis-shape parameters are central to the representation, but their specification is incomplete. The text states that the C-layer parameters are 'set analytically as described in Eqs.??–?? (also in Figures?? and??) in the supplementary material,' and the off-diagonal B-layer weights are 'selected empirically as depicted in the supplementary document.' The unresolved equation and figure numbers and the empirical selection leave the actual basis functions unspecified in the manuscript. Because Eqs. (5)–(7) show that all learned motor behavior is a linear combination of these fixed bases, the reader cannot reproduce the approach or verify the claimed absence of gait parameterization without these parameter values and their selection procedure.
  3. [Experiments and Results, statistical tests in Figures 11–14] The continual-learning support for the main claims rests on paired t-tests with n=20, but the manuscript does not define what n is. If n refers to episodes within a single physical run, those comparisons are pseudoreplicated and cannot support the stated conclusions; if n refers to independent runs, the number of runs and the per-condition variance need to be reported. The same issue affects the speed comparisons in 'Primitive Locomotion Learning' (n=20, Figure 8). Please state the experimental design, report means and error bars for repeated runs, and adjust the statistical claims accordingly.
  4. [Subnetwork Neurogenesis, Eqs. (8)–(11)] The novelty detector is itself adaptive: Vδ and Oδ are trained online to cover the observed reward and feedback deviations, and the text reports that a reward-only drop at episode 100 expanded the prediction boundary without triggering neurogenesis. This creates a need to specify the condition under which boundary expansion stops and a new subnetwork is created; otherwise the mechanism is indistinguishable from an always-expanding threshold, and the claimed autonomy of 'detecting new conditions' is not established. A worked analysis of the boundary dynamics at the episode-100 versus episode-150 transitions would resolve this concern.
  5. [Discussion, final paragraph of 'Life-long Locomotion Learning Research Aspect'] The statement that GOLLUM is 'the only locomotion learning framework' with the listed properties is stronger than the evidence in Table 2. The table covers 18 selected methods and gives no systematic inclusion criteria or search protocol, and several table entries are not directly comparable because they differ in robot platform, simulation-versus-real setting, and task definition. Please narrow the claim to the methods actually compared, or provide a systematic literature selection protocol that justifies 'only.'
minor comments (5)
  1. [Eq. (4)] The term w_{Ci,Ii} I_i[t] is inconsistent with the text's description that the selection input is I_{i+1}; align the index.
  2. [Eqs. (3) and (5)] The relation between τ_i in the I-layer and w_{Bi,Bi}=1−τ_i in the B-layer should be stated explicitly; as written, 'set to the same value as that in the basis layer' is ambiguous.
  3. [Throughout] There are several typos: 'nework' in the Output Layer subsection, 'within an hours' in General Continual Locomotion Learning, 'refered to behavior model' and 'refereed to' in Separation and Incrementation, and 'unsupervisely' in the Abstract.
  4. [Eq. (19)] The notation t+H min τ=t r[τ] is malformed; use min_{t≤τ≤t+H} r[τ].
  5. [Figure 15a] Figure 15a is labeled a conceptual representation; consider stating in the caption that the axes are illustrative, since the figure is used to support the interpretability/performance trade-off claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; GOLLUM's claims rest on physical-robot experiments and standard learning rules, not on self-referential derivation.

full rationale

GOLLUM's central claims are empirical: a physical hexapod learned locomotion skills, and the learning curves, ablations, and comparisons are reported as experimental results rather than derived from the model equations. The policy-gradient update (Eqs. 12–13) and the connectionist controller (Eqs. 5–10) are standard function-approximation machinery; no fitted parameter is renamed as a prediction of an independent phenomenon. The value and observation predictions (Eqs. 8–11) are learned online from the same reward/feedback stream and function as novelty detectors for neurogenesis, not as predictive claims about external outcomes, so fitting them to that stream does not create a circular prediction. The direct knowledge transfer initialization is an experimental intervention compared against a naive baseline, and the 'only framework' claim is a literature comparison, not a mathematical consequence of the model. Self-citations (e.g., the low-pass-filter basis parameters from Srisuchinnawong et al. 2023, 2021b) are used as ordinary references for standard design choices and are not load-bearing proofs. The paper does contain unresolved equation references in the C-layer parameter description ('as describe in Eqs.??–??'), but that is an incompleteness or correctness concern, not circularity. The skeptical 'full action space' critique concerns whether the four-action triangular-basis CPG can represent all needed gaits; that is an assumption-strength and correctness issue, not a case where an output is equivalent to an input by construction.

Assumptions & free parameters 6 free parameters · 5 assumptions · 3 invented entities

The central mechanism relies on a set of hand-set CPG and basis parameters, on an unproven representational assumption about the fixed ring structure, and on online-fitted prediction boundaries for neurogenesis. The paper does not introduce new physical entities; the invented items are algorithmic components of the framework.

free parameters (6)
  • CPG connection weights = w_Ci,Ci=20, bias=-13, w_transition=7, w_inhibit=-26
    Set analytically to produce the desired ring sequence; not learned. Central to rhythmic pattern generation.
  • Basis shape parameters = w_Bi,Ck = -0.5 tau_i for predecessor, -0.25 tau_i for subsequent; tau_i chosen manually
    Selected empirically to make triangular bases; affects the temporal shape of all motor commands.
  • Prediction boundary floors = epsilon_v = 0.02, epsilon_o = 0.02
    Arbitrary small constants; affect the sensitivity of neurogenesis detection.
  • Learning rates for value/observation predictors = not specified in main text (one high, one low)
    Empirically selected; affect the speed and stability of the prediction boundaries used by neurogenesis.
  • Horizon and episode length = H=14, episode=30 timesteps (5 s)
    Chosen by authors; influence the return definition and the neurogenesis timing.
  • Experience replay length = N=8 episodes
    Taken from prior works (Thor et al. 2020); affects sample efficiency.
assumptions (5)
  • domain assumption The fixed CPG ring structure with four actions per subnetwork spans the needed action space for all tested skills
    The paper states it maintains a full action space without proving that the four-action, triangular-basis parametrization can represent bouncing or other discovered gaits.
  • domain assumption The observation vector (pitch, 18 joint states, hue mean, hue std) is sufficient to distinguish environment conditions that require new skills
    Neurogenesis relies on observation prediction mismatches; if the features are not discriminative, new conditions would be missed or confused.
  • domain assumption The policy gradient rule (Eqs. 12-13) with adaptive exploration converges adequately in this non-stationary physical setting
    No convergence proof is offered; the empirical success is the only support.
  • domain assumption The low-level position controller accurately tracks the 20 Hz commands from the external computer
    Motor commands are treated as executed joint trajectories; tracking errors are not modeled.
  • domain assumption Reward (forward speed or inverse COT) is a sufficient objective for locomotion skill acquisition
    The reward functions are single-term; the paper argues this removes the need for reward shaping, but it also assumes these terms capture the desired behaviors.
invented entities (3)
  • Supplementary connections (B to PM between subnetworks)
    purpose: Allow an active skill to blend in patterns from previously learned skills
    New algorithmic structure; its contribution is measured by weight magnitudes, but it is not independently falsifiable outside the paper.
  • Neurogenesis trigger based on value and observation prediction boundaries
    purpose: Autonomously decide when to add a new subnetwork
    New online detector; depends on fitted prediction boundaries, so it is not an external measurement.
  • Observation prediction templates (O_i)
    purpose: Serve as supervised targets for training the sensory preprocessing layer and for mismatch detection
    The templates are derived from the network's own weight matrix (w_Oi,Bk), not from external data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Growable and Interpretable Neural Control with Online Continual Learning for Autonomous Lifelong Locomotion Learning Machines." pith.science (2026). https://pith.science/paper/X4MSCAOL

@misc{pith2026250512029,
  author       = {Pith},
  title        = {Pith review of: Growable and Interpretable Neural Control with Online Continual Learning for Autonomous Lifelong Locomotion Learning Machines},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X4MSCAOL}},
  note         = {Machine review of arXiv:2505.12029}
}
read the original abstract

Continual locomotion learning faces four challenges: incomprehensibility, sample inefficiency, lack of knowledge exploitation, and catastrophic forgetting. Thus, this work introduces Growable Online Locomotion Learning Under Multicondition (GOLLUM), which exploits the interpretability feature to address the aforementioned challenges. GOLLUM has two dimensions of interpretability: layer-wise interpretability for neural control function encoding and column-wise interpretability for robot skill encoding. With this interpretable control structure, GOLLUM utilizes neurogenesis to unsupervisely increment columns (ring-like networks); each column is trained separately to encode and maintain a specific primary robot skill. GOLLUM also transfers the parameters to new skills and supplements the learned combination of acquired skills through another neural mapping layer added (layer-wise) with online supplementary learning. On a physical hexapod robot, GOLLUM successfully acquired multiple locomotion skills (e.g., walking, slope climbing, and bouncing) autonomously and continuously within an hour using a simple reward function. Furthermore, it demonstrated the capability of combining previous learned skills to facilitate the learning process of new skills while preventing catastrophic forgetting. Compared to state-of-the-art locomotion learning approaches, GOLLUM is the only approach that addresses the four challenges above mentioned without human intervention. It also emphasizes the potential exploitation of interpretability to achieve autonomous lifelong learning machines.

Figures

Figures reproduced from arXiv: 2505.12029 by the authors.

Figure 1
Figure 1. a Growable online locomotion learning under multicondition (GOLLUM) consists of an interpretable neural control for motor command generation, a dual learning mechanism (primary learning for efficient locomotion learning and supplementary learning for exploiting shared skills), and a neurogenesis for implementing new skills. The interpretable neural control has two interpretation dimensions (column-wise and layer-wis… view at source ↗
Figure 2
Figure 2. GOLLUM framework, presented along with the corresponding neural activity signals: feedback (F B[t], where θ denotes the robot pitch angle), first sensory preprocessing (I ′ [t], classification score), second sensory preprocessing (I[t], internal state selection), sequential central pattern (C[t], discrete internal state), basis (B, smooth internal state), premotor (PM[t], pattern), and output (M[t], motor command). … view at source ↗
Figure 3
Figure 3. MORF hexapod robot employed in this work presented long with its sensors, GOLLUM, the robot interface, and the training process. 1 st Input Preprocessing (I’) The first input preprocessing layer (I’) maps the sensory feedback/observation (F Bk[t]) to the intermediate preprocessed input (I ′ i [t]), which represents the behavior/environment classification signals for activating the proper columns/subnetworks. The map… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: a Feedback independent and b feedback dependent sequential central pattern generator neurons (Ci). The former always propagates the activity of the former neuron (Ci−1) forward to the next (Ci+1) due to the excitatory connections from Ci to Ci+1, Ci to Bi, and from Bi …
Figure 5
Figure 5. Figure 5: Visualization of the learning rule (Eq. 12) applied to the connection weights between two premotor neurons (PM1 and PM2) and a motor output (M1), where the star denotes the coordinate of the current parameter values, blue dots denote the coordinates of the explored par…
Figure 6
Figure 6. Figure 6: a Graphical illustration of locomotion learning on different terrains: (I) flat rigid floor, (II) thin mat (soft/deformable terrain), (III) thick sponge (highly soft/highly deformable terrain), (IV) rough paver, (V) inclined paver, and (VI) gravel field. b Correspondin…
Figure 8
Figure 8. Figure 8: Average speed reward per gait cycle and its range (min–max), obtained from the simulated hexapod robot trained with different methods (GOLLUM, CPGRBF + PIBB, DNN + DroQ, and DNN + PPO, see text for details). Note that the hyper-parameters of the methods (see the supple…
Figure 7
Figure 7. Figure 7: b. By the 100th episode, the robot had developed a gait with a forward speed of approximately 5 cm/s on average, which is equivalent to the result obtained from a manually designed controller (Homchanthanakul and Manoonpong 2021), validated on the same robot. Finally, …
Figure 9
Figure 9. Figure 9: Snapshots and inverse cost of transport (COT)-based rewards (Eq. 19) obtained from a physical hexapod robot under online continual locomotion learning a on different slopes (0◦ , 10◦ , 15◦ , and 25◦ ), b on different slopes with potential motor dysfunction (0◦ , 10◦ , …
Figure 10
Figure 10. Figure 10: a Returns and lower prediction boundary obtained from locomotion learning on different terrains. b Trajectory of the sensory feedback in the observation space (hue mean H¯ , hue standard deviation σH, and body pitch θ) presented along with its prediction boundary and …
Figure 11
Figure 11. Figure 11: Comparison of the rewards from locomotion learning on different slopes (blue line) with and (gray line) without neurogenesis [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 13
Figure 13. Figure 13: also shows that, after adding a new subnetwork, the skill initialization could affect performance at the very first few episodes. Using GOLLUM with the direct knowledge transfer mechanism (blue line) automatically selected the previously learned knowledge according to…
Figure 14
Figure 14. Figure 14: Snapshots and connection weight matrices for exploiting task similarity obtained from the locomotion learning a on different slopes, b on different slopes with potential motor dysfunction, and c on different terrains. The rows highlighted in green indicate the supplem…
Figure 15
Figure 15. Figure 15: a Trade-off between interpretability and performance in various machine learning methods (DW 2019), including GOLLUM, which has a higher interpretability and performance (see [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

80 extracted references · 70 canonical work pages

  1. [1]

    Neuron 46(4): 681--692

    Angela JY and Dayan P (2005) Uncertainty, neuromodulation, and attention. Neuron 46(4): 681--692

    work page 2005

  2. [2]

    (2020) Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai

    Arrieta AB, D \' az-Rodr \' guez N, Del Ser J, Bennetot A, Tabik S, Barbado A, Garc \' a S, Gil-L \'o pez S, Molina D, Benjamins R et al. (2020) Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information fusion 58: 82--115

    work page 2020

  3. [3]

    Journal of Intelligent & Robotic Systems 99(3-4): 659--671

    Azayev T and Zimmerman K (2020) Blind hexapod locomotion in complex terrain with gait adaptation using deep reinforcement learning and classification. Journal of Intelligent & Robotic Systems 99(3-4): 659--671

    work page 2020

  4. [4]

    IEEE Robotics and Automation Letters 7(4): 12547--12554

    Bellegarda G and Ijspeert A (2022) Cpg-rl: Learning central pattern generators for quadruped locomotion. IEEE Robotics and Automation Letters 7(4): 12547--12554

    work page 2022

  5. [5]

    Science Robotics 8(74): eade2256

    Choi S, Ji G, Park J, Kim H, Mun J, Lee JH and Hwangbo J (2023) Learning quadrupedal locomotion on deformable terrain. Science Robotics 8(74): eade2256

    work page 2023

  6. [6]

    PLoS computational biology 11(9): e1004402

    Christopoulos V and Schrater PR (2015) Dynamic integration of value information into a common probability currency as a theory for flexible decision making. PLoS computational biology 11(9): e1004402

    work page 2015

  7. [7]

    Journal of Neuroscience 26(38): 9761--9770

    Cisek P (2006) Integrated neural processes for defining potential actions and deciding between them: a computational model. Journal of Neuroscience 26(38): 9761--9770

    work page 2006

  8. [8]

    Nature 521(7553): 503--507

    Cully A, Clune J, Tarapore D and Mouret JB (2015) Robots that can adapt like animals. Nature 521(7553): 503--507

    work page 2015

Show all 80 references
  1. [9]

    IEEE Transactions on Cognitive and Developmental Systems

    Deshpande AM, Hurd E, Minai AA and Kumar M (2023) Deepcpg policies for robot locomotion. IEEE Transactions on Cognitive and Developmental Systems

    2023

  2. [10]

    Journal of King Saud University-Computer and Information Sciences 34(7): 4312--4321

    Ding F and Zhu F (2022) Hliferl: A hierarchical lifelong reinforcement learning framework. Journal of King Saud University-Computer and Information Sciences 34(7): 4312--4321

    2022

  3. [11]

    AI Mag 40(2): 44

    DW GDA (2019) Darpa’s explainable artificial intelligence program. AI Mag 40(2): 44

    2019

  4. [12]

    Feber J, Szadkowski R and Faigl J (2022) Gait adaptation after leg amputation of hexapod walking robot without sensory feedback. In: Artificial Neural Networks and Machine Learning--ICANN 2022: 31st International Conference on Artificial Neural Networks, Bristol, UK, September...

    2022

  5. [13]

    Entropy 26(1): 93

    Gai S, Lyu S, Zhang H and Wang D (2024) Continual reinforcement learning for quadruped robot locomotion. Entropy 26(1): 93

    2024

  6. [14]

    IEEE Transactions on Software Engineering

    Ghzouli R, Berger T, Johnsen EB, Wasowski A and Dragule S (2023) Behavior trees and state machines in robotics applications. IEEE Transactions on Software Engineering

    2023

  7. [15]

    arXiv preprint arXiv:2112.13112

    Glanois C, Weng P, Zimmer M, Li D, Yang T, Hao J and Liu W (2021) A survey on interpretable reinforcement learning. arXiv preprint arXiv:2112.13112

    2021 arXiv

  8. [16]

    Pearson education india

    Gonzalez RC (2009) Digital image processing. Pearson education india

    2009

  9. [17]

    Journal of Experimental Biology 221(7): jeb138438

    Gravish N and Lauder GV (2018) Robotics-inspired biology. Journal of Experimental Biology 221(7): jeb138438

    2018

  10. [18]

    Frontiers in Neurorobotics 17: 1127642

    Hafez MB, Immisch T, Weber T and Wermter S (2023) Map-based experience replay: a memory-efficient solution to catastrophic forgetting in reinforcement learning. Frontiers in Neurorobotics 17: 1127642

    2023

  11. [19]

    IEEE Transactions on Cognitive and Developmental Systems

    Hafez MB and Wermter S (2023) Continual robot learning using self-supervised task inference. IEEE Transactions on Cognitive and Developmental Systems

    2023

  12. [20]

    arXiv preprint arXiv:2110.02034

    Hiraoka T, Imagawa T, Hashimoto T, Onishi T and Tsuruoka Y (2021) Dropout q-functions for doubly efficient reinforcement learning. arXiv preprint arXiv:2110.02034

    2021 arXiv

  13. [21]

    IEEE Transactions on Neural Networks and Learning Systems 33(5): 1833--1845

    Homchanthanakul J and Manoonpong P (2021) Continuous online adaptation of bioinspired adaptive neuroendocrine control for autonomous walking robots. IEEE Transactions on Neural Networks and Learning Systems 33(5): 1833--1845

    2021

  14. [22]

    In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

    Homchanthanakul J, Ngamkajornwiwat P, Teerakittikul P and Manoonpong P (2019) Neural control with an artificial hormone system for energy-efficient compliant terrain locomotion and adaptation of walking robots. In: 2019 IEEE/RSJ International Conference on Intelligent Robots a...

    2019

  15. [23]

    arXiv preprint arXiv:1807.02401

    Hu K and O'Connor P (2018) Learning a representation map for robot navigation using deep variational autoencoder. arXiv preprint arXiv:1807.02401

    2018 arXiv

  16. [24]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, volume 32

    Isele D and Cosgun A (2018) Selective experience replay for lifelong learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, volume 32

    2018

  17. [25]

    Citeseer

    Jaeger H (1995) Identification of behaviors in an agent's phase space. Citeseer

    1995

  18. [26]

    arXiv preprint arXiv:2104.08212

    Kalashnikov D, Varley J, Chebotar Y, Swanson B, Jonschkowski R, Finn C, Levine S and Hausman K (2021) Mt-opt: Continuous multi-task robotic reinforcement learning at scale. arXiv preprint arXiv:2104.08212

    2021 arXiv

  19. [27]

    Journal of Artificial Intelligence Research 75: 1401--1476

    Khetarpal K, Riemer M, Rish I and Precup D (2022) Towards continual reinforcement learning: A review and perspectives. Journal of Artificial Intelligence Research 75: 1401--1476

    2022

  20. [28]

    (2022) Biological underpinnings for lifelong learning machines

    Kudithipudi D, Aguilar-Simon M, Babb J, Bazhenov M, Blackiston D, Bongard J, Brna AP, Chakravarthi Raja S, Cheney N, Clune J et al. (2022) Biological underpinnings for lifelong learning machines. Nature Machine Intelligence 4(3): 196--210

    2022

  21. [29]

    The International Journal of Robotics Research 31(3): 330--345

    Kuli \'c D, Ott C, Lee D, Ishikawa J and Nakamura Y (2012) Incremental learning of full body motion primitives and their sequencing through human motion observation. The International Journal of Robotics Research 31(3): 330--345

    2012

  22. [30]

    In: 2019 International Conference on Robotics and Automation (ICRA)

    Lechner M, Hasani R, Zimmer M, Henzinger TA and Grosu R (2019) Designing worm-inspired neural networks for interpretable robotic control. In: 2019 International Conference on Robotics and Automation (ICRA). IEEE, pp. 87--94

    2019

  23. [31]

    In: International Conference on Learning Representations

    Lee D, Ahn D, Kim T, Chuang PI and Kim JJ (2018) Viterbi-based pruning for sparse matrix with fixed and high index compression ratio. In: International Conference on Learning Representations

    2018

  24. [32]

    Science robotics 5(47): eabc5986

    Lee J, Hwangbo J, Wellhausen L, Koltun V and Hutter M (2020) Learning quadrupedal locomotion over challenging terrain. Science robotics 5(47): eabc5986

    2020

  25. [33]

    In: 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS)

    Lele AS, Fang Y, Ting J and Raychowdhury A (2020) Learning to walk: Spike based reinforcement learning for hexapod robot central pattern generation. In: 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS). IEEE, pp. 208--212

    2020

  26. [34]

    Applied Intelligence 51(1): 185--201

    Li C, Li Y, Zhao Y, Peng P and Geng X (2021) Sler: Self-generated long-term experience replay for continual reinforcement learning. Applied Intelligence 51(1): 185--201

    2021

  27. [35]

    IEEE Robotics and Automation Letters

    Li G, Ijspeert A and Hayashibe M (2024) Ai-cpg: Adaptive imitated central pattern generators for bipedal locomotion learned through reinforced reflex neural networks. IEEE Robotics and Automation Letters

    2024

  28. [36]

    Queue 16(3): 31--57

    Lipton ZC (2018) The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue 16(3): 31--57

    2018

  29. [37]

    Luneckas M, Luneckas T, Udris D, Plonis D, Maskeli \=u nas R and Dama s evi c ius R (2019) Energy-efficient walking over irregular terrain: A case of hexapod robot

    2019

  30. [38]

    In: Proc

    Mansard N, Aycard O and Koike C (2005) Hierarchy of behaviors. In: Proc. of the IEEE International Conference on Robotics and Biomimetics

    2005

  31. [39]

    arXiv preprint arXiv:2205.02824

    Margolis GB, Yang G, Paigwar K, Chen T and Agrawal P (2022) Rapid locomotion via reinforcement learning. arXiv preprint arXiv:2205.02824

    2022 arXiv

  32. [40]

    In: 2023 IEEE International Conference on Robotics and Automation (ICRA)

    Nahrendra IMA, Yu B and Myung H (2023) Dreamwaq: Learning robust quadrupedal locomotion with implicit terrain imagination via deep reinforcement learning. In: 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp. 5078--5084

    2023

  33. [41]

    ACM Computing Surveys 55(13s): 1--42

    Nauta M, Trienes J, Pathak S, Nguyen E, Peters M, Schmitt Y, Schl \"o tterer J, Van Keulen M and Seifert C (2023) From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable ai. ACM Computing Surveys 55(13s): 1--42

    2023

  34. [42]

    Neural networks 113: 54--71

    Parisi GI, Kemker R, Part JL, Kanan C and Wermter S (2019) Continual lifelong learning with neural networks: A review. Neural networks 113: 54--71

    2019

  35. [43]

    Pasemann F, Hild M and Zahedi K (2003) So (2)-networks as neural oscillators. In: Computational Methods in Neural Modeling: 7th International Work-Conference on Artificial and Natural Neural Networks, IWANN 2003 Ma \'o , Menorca, Spain, June 3--6, 2003 Proceedings, Part I . Sp...

    2003

  36. [45]

    arXiv preprint arXiv:1909.10008

    Ribeiro J, Melo FS and Dias J (2019 b ) Multi-task learning and catastrophic forgetting in continual reinforcement learning. arXiv preprint arXiv:1909.10008

    2019 arXiv

  37. [46]

    Evolutionary Intelligence 7: 119--131

    Rossi C and Eiben A (2014) Simultaneous versus incremental learning of multiple skills by modular robots. Evolutionary Intelligence 7: 119--131

    2014

  38. [47]

    Nature machine intelligence 1(5): 206--215

    Rudin C (2019) Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence 1(5): 206--215

    2019

  39. [48]

    In: Faust A, Hsu D and Neumann G (eds.) Proceedings of the 5th Conference on Robot Learning, Proceedings of Machine Learning Research, volume 164

    Rudin N, Hoeller D, Reist P and Hutter M (2022) Learning to walk in minutes using massively parallel deep reinforcement learning. In: Faust A, Hsu D and Neumann G (eds.) Proceedings of the 5th Conference on Robot Learning, Proceedings of Machine Learning Research, volume 164. ...

    2022

  40. [49]

    Nature Machine Intelligence 4(7): 652--660

    Ruppert F and Badri-Spr \"o witz A (2022) Learning plastic matching of robot dynamics in closed-loop central pattern generators. Nature Machine Intelligence 4(7): 652--660

    2022

  41. [50]

    In: 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob)

    Schilling M, Konen K and Korthals T (2020) Modular deep reinforcement learning for emergent locomotion on a six-legged robot. In: 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob). IEEE, pp. 946--953

    2020

  42. [51]

    arXiv preprint arXiv:1707.06347

    Schulman J, Wolski F, Dhariwal P, Radford A and Klimov O (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

    2017 arXiv

  43. [52]

    In: International conference on machine learning

    Schwarz J, Czarnecki W, Luketina J, Grabska-Barwinska A, Teh YW, Pascanu R and Hadsell R (2018) Progress & compress: A scalable framework for continual learning. In: International conference on machine learning. PMLR, pp. 4528--4537

    2018

  44. [53]

    Neural Networks 23(4): 551--559

    Sehnke F, Osendorfer C, R \"u ckstie T, Graves A, Peters J and Schmidhuber J (2010) Parameter-exploring policy gradients. Neural Networks 23(4): 551--559

    2010

  45. [54]

    Nature Communications 15(1): 3073

    Shafiee M, Bellegarda G and Ijspeert A (2024) Viability leads to the emergence of gait transitions in learning agile quadrupedal locomotion on challenging terrains. Nature Communications 15(1): 3073

    2024

  46. [55]

    In: 2022 International Conference on Robotics and Automation (ICRA)

    Smith L, Kew JC, Peng XB, Ha S, Tan J and Levine S (2022 a ) Legged robots that keep on learning: Fine-tuning locomotion policies in the real world. In: 2022 International Conference on Robotics and Automation (ICRA). IEEE, pp. 1593--1599

    2022

  47. [56]

    arXiv preprint arXiv:2208.07860

    Smith L, Kostrikov I and Levine S (2022 b ) A walk in the park: Learning to walk in 20 minutes with model-free reinforcement learning. arXiv preprint arXiv:2208.07860

    2022 arXiv

  48. [57]

    Frontiers in Neural Circuits 15: 743101

    Srisuchinnawong A, Homchanthanakul J and Manoonpong P (2021 a ) Neurovis: real-time neural information measurement and visualization of embodied neural systems. Frontiers in Neural Circuits 15: 743101

    2021

  49. [58]

    IEEE/ASME Transactions on Mechatronics

    Srisuchinnawong A, Phongaksorn K, Ausrivong W and Manoonpong P (2023) Adaptive bipedal robot walking on industrial pipes under neural multimodal locomotion control: Toward robotic out-pipe inspection. IEEE/ASME Transactions on Mechatronics

    2023

  50. [59]

    In: 2019 19th International Conference on Advanced Robotics (ICAR)

    Srisuchinnawong A, Shao D, Ngamkajornwiwat P, Teerakittikul P, Dai Z, Ji A and Manoonpong P (2019) Neural control for gait generation and adaptation of a gecko robot. In: 2019 19th International Conference on Advanced Robotics (ICAR). IEEE, pp. 468--473

    2019

  51. [60]

    Journal of Intelligent & Robotic Systems 101: 1--15

    Srisuchinnawong A, Wang B, Shao D, Ngamkajornwiwat P, Dai Z, Ji A and Manoonpong P (2021 b ) Modular neural control for gait adaptation and obstacle avoidance of a tailless gecko robot. Journal of Intelligent & Robotic Systems 101: 1--15

    2021

  52. [61]

    Reviews in the Neurosciences 30(2): 107--164

    Steuer I and Guertin PA (2019) Central pattern generators in the brainstem and spinal cord: an overview of basic principles, similarities and differences. Reviews in the Neurosciences 30(2): 107--164

    2019

  53. [62]

    Stulp F and Sigaud O (2012) Policy improvement methods: Between black-box optimization and episodic reinforcement learning

    2012

  54. [63]

    MIT press

    Sutton RS and Barto AG (2018) Reinforcement learning: An introduction. MIT press

    2018

  55. [64]

    IEEE transactions on neural networks and learning systems 32(9): 4013--4025

    Thor M, Kulvicius T and Manoonpong P (2020) Generic neural locomotion control framework for legged robots. IEEE transactions on neural networks and learning systems 32(9): 4013--4025

    2020

  56. [65]

    In: Proc

    Thor M, Larsen JC and Manoonpong P (2018) Morf---modular robot framework. In: Proc. 2nd Int. Youth Conf. Bionic Eng.(IYCBE). pp. 21--23

    2018

  57. [66]

    Nature Machine Intelligence 4(2): 169--179

    Thor M and Manoonpong P (2022) Versatile modular neural locomotion control with fast learning. Nature Machine Intelligence 4(2): 169--179

    2022

  58. [67]

    Frontiers in Neural Circuits : 135

    Thor M, Strohmer B and Manoonpong P (2021) Locomotion control with frequency and motor pattern adaptations. Frontiers in Neural Circuits : 135

    2021

  59. [68]

    (2020) The neuroanatomical ultrastructure and function of a biological ring attractor

    Turner-Evans DB, Jensen KT, Ali S, Paterson T, Sheridan A, Ray RP, Wolff T, Lauritzen JS, Rubin GM, Bock DD et al. (2020) The neuroanatomical ultrastructure and function of a biological ring attractor. Neuron 108(1): 145--163

    2020

  60. [69]

    Journal of Neuroendocrinology 33(1): e12911

    Van Damme S, De Fruyt N, Watteyne J, Kenis S, Peymen K, Schoofs L and Beets I (2021) Neuromodulatory pathways in learning and memory: Lessons from invertebrates. Journal of Neuroendocrinology 33(1): e12911

    2021

  61. [70]

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    Wang L, Zhang X, Su H and Zhu J (2024) A comprehensive survey of continual learning: theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence

    2024

  62. [71]

    IEEE Transactions on Neural Networks and Learning Systems 33(8): 4003--4016

    Wang Z, Chen C and Dong D (2021) Lifelong incremental reinforcement learning with online bayesian inference. IEEE Transactions on Neural Networks and Learning Systems 33(8): 4003--4016

    2021

  63. [72]

    Journal of Comparative Neurology 523(7): 997--1037

    Wolff T, Iyer NA and Rubin GM (2015) Neuroarchitecture and neuroanatomy of the drosophila central complex: A gal4-based dissection of protocerebral bridge neurons and circuits. Journal of Comparative Neurology 523(7): 997--1037

    2015

  64. [73]

    arXiv preprint arXiv:2006.10701

    Xie A, Harrison J and Finn C (2020) Deep reinforcement learning amidst lifelong non-stationarity. arXiv preprint arXiv:2006.10701

    2020 arXiv

  65. [74]

    Nature 550(7677): 519--523

    Yan G, V \'e rtes PE, Towlson EK, Chew YL, Walker DS, Schafer WR and Barab \'a si AL (2017) Network control principles predict neuron function in the caenorhabditis elegans connectome. Nature 550(7677): 519--523

    2017

  66. [75]

    Science Robotics 5(49): eabb2174

    Yang C, Yuan K, Zhu Q, Yu W and Li Z (2020) Multi-expert learning of adaptive legged locomotion. Science Robotics 5(49): eabb2174

    2020

  67. [76]

    Nature Machine Intelligence 5(8): 919--932

    Yu W, Yang C, McGreavy C, Triantafyllidis E, Bellegarda G, Shafiee M, Ijspeert AJ and Li Z (2023) Identifying important sensory feedback for learning locomotion skills. Nature Machine Intelligence 5(8): 919--932

    2023

  68. [77]

    In: 2013 IEEE/ASME International Conference on Advanced Intelligent Mechatronics

    Zenker S, Aksoy EE, Goldschmidt D, W \"o rg \"o tter F and Manoonpong P (2013) Visual terrain classification for selecting energy efficient gaits of a hexapod robot. In: 2013 IEEE/ASME International Conference on Advanced Intelligent Mechatronics. IEEE, pp. 577--584

    2013

  69. [78]

    arXiv preprint arXiv:2311.06015

    Zhang H, Shi D, Zhuang Z, Zhao H, Wei Z, Zhao F, Gai S, Lyu S and Wang D (2023) Rsg: Fast learning adaptive skills for quadruped robots by skill graph. arXiv preprint arXiv:2311.06015

    2023 arXiv

  70. [79]

    The Journal of Supercomputing : 1--33

    Zhao Z, Zhang G, Wu Y, Hong R, Yang Y and Fu Y (2024) Block-wise dynamic mixed-precision for sparse matrix-vector multiplication on gpus. The Journal of Supercomputing : 1--33

    2024

  71. [80]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution isbn journal key month note number organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence...

  72. [81]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.