Pith. sign in

REVIEW 4 major objections 8 minor 1 cited by

Arnold: a generalist muscle transformer policy

T0 review · 4 major / 8 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Arnold, a single transformer policy trained by on-policy behavior cloning, per-task PPO fine-tuning, and self-distillation, matches or exceeds specialist experts across 14 musculoskeletal control tasks spanning four body models.

desk verdict Solid empirical generalist policy for musculoskeletal control; the 14-task claim is credible but hinges on a few addressable verification gaps. read the letter →

arxiv 2508.18066 v1 pith:OENFZFZX submitted 2025-08-25 cs.RO cs.AIcs.LGq-bio.QM

Alberto Silvio Chiappa , Boshi An , Merkourios Simos , Chengkun Li , Alexander Mathis This is my paper
classification cs.ROcs.AIcs.LGq-bio.QM
keywords musculoskeletalcontrolmulti-taskpolicytransformeron-policybehaviorcloningimitationlearningmusclesynergiesMyoSuiteself-distillation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to show that one neural network can act as a generalist motor controller for the human musculoskeletal system. The agent, Arnold, is a transformer that represents each sensory channel and each muscle as tokens drawn from a 214-word sensorimotor vocabulary, so the same policy can handle different body models, observation spaces, and action spaces without architectural changes. Its training is a three-stage distillation recipe: on-policy behavior cloning from specialist expert policies, per-task PPO fine-tuning to exceed those experts, and self-distillation of the improved specialists back into the single generalist. On the 14 MyoSuite tasks spanning four musculoskeletal models, the paper reports expert or super-expert performance, on average 105.13% of the expert reward. The paper further reports that Arnold's muscle activations are low-dimensional within each task but are not shared across tasks, which it reads as evidence against universal muscle synergies in this setting.

What carries the argument

The load-bearing object is the sensorimotor vocabulary: a set of 214 learned word embeddings, combined by addition to form a role embedding for each sensory input and each actuator. Role embeddings are added to linear sensory encodings before an encoder-decoder transformer, and the decoder emits one action token per muscle plus a value token, making variable observation and action spaces a matter of token composition rather than architecture change. The training mechanism is on-policy behavior cloning (OBC): the student collects rollouts in the environment and the teacher experts label those states with target actions, avoiding the distributional mismatch of offline behavior cloning. Around

What would settle it

During OBC pretraining, record for each task the expert policy's action error on the states the student actually visits. The paper's explanation predicts that this on-policy error is small enough to drive imitation. A finding that one or more teachers are systematically wrong on the student's state distribution—or, equivalently, an experiment in which corrupting one teacher's labels on student rollouts leaves that task's final performance unchanged—would undercut the claim that OBC is what carries Arnold's success.

Watch

Extended reading notes

Core claim

Arnold's central claim is that expert-level multi-task musculoskeletal control can be obtained by distillation rather than by solving each task from scratch. The key correction to ordinary behavior cloning is to run the imitation on the student's own rollouts: expert policies label the states Arnold visits, so the training distribution matches deployment. With that change, Arnold reaches expert performance across all 14 tasks; with per-task PPO fine-tuning followed by distillation back into the generalist, it reaches super-expert performance. The second claim is that a compositional sensorimotor vocabulary of 214 learned tokens—where a sensory channel like 'right soleus muscle length' is the

Load-bearing premise

OBC assumes that the teacher policies give correct target actions on the states the student visits during its own rollouts; if a teacher is unreliable precisely where Arnold goes, the imitation signal degrades and the reported expert-level results on all 14 tasks would not transfer.

Editorial extensions

If this is right

  • A single 4.4-million-parameter transformer policy can match or exceed task-specific experts across four musculoskeletal models, so specialist policies do not have to be discarded when a generalist is needed.
  • OBC closes the imitation gap that standard BC leaves open: on the 14-task set it reaches 98.97% of expert reward versus 80.89% for BC and 50.05% for PPO trained from scratch.
  • RL fine-tuning followed by self-distillation raises the generalist to 105.13% of expert reward, so a distilled multi-task policy can serve as a better starting point for further learning rather than a ceiling.
  • A policy pretrained on ten tasks learns four held-out tasks faster than randomly initialized specialists, indicating transferable sensorimotor representations, though not zero-shot generalization.
  • Within each task Arnold's muscle activations lie near a low-dimensional subspace, but these subspaces differ across tasks, suggesting muscle synergies are task-specific in this generalist.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The vocabulary's compositionality implies that adding a new body part or task should require only a small number of new tokens, since shared words like muscle names, side labels, and semantic roles are reused; the paper demonstrates 214 tokens for 14 tasks but does not test scaling limits.
  • The OBC recipe depends on having queryable teacher policies in the loop; it would not transfer directly to fixed offline demonstration datasets, a boundary implicit in the method's design.
  • The absence of transferable synergies may reflect the specific task mix and the distillation objective rather than a general property of multi-task motor learning; the paper itself suggests that more tasks or more similar tasks could change this conclusion.
  • A direct test of the vocabulary's role: remove the shared tokens by replacing compositional role embeddings with disjoint per-task embeddings on a held-out task and measure how much slower learning becomes; the paper's Table 1 already shows large drops on five tasks, suggesting the effect should be measurable on new tasks too.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. Arnold is a generalist transformer policy for musculoskeletal control. Using a 214-token compositional sensorimotor vocabulary, a single encoder-decoder transformer is trained across 14 MyoSuite tasks spanning four musculoskeletal models. The training pipeline consists of on-policy behavior cloning (OBC) from specialist teachers, per-task PPO fine-tuning, and self-distillation. The paper reports expert or super-expert performance (105.13 ± 1.60% of expert reward) averaged over 200 test episodes and three seeds, data-efficient transfer to novel tasks, and a CSI analysis indicating that muscle synergies are task-specific rather than universal.

Significance. If the results hold, this is a substantial empirical contribution: it demonstrates that a single policy can control multiple musculoskeletal embodiments and tasks, and it provides a compositional tokenization scheme for variable observation/action spaces. The paper has real strengths: three-seed error bars for the main comparison, ablations of observation normalization, vocabulary sharing, and OBC versus BC, and the final evaluation is measured by environment reward rather than derived from the teachers, so the central claim is not circular. The CSI analysis is a useful functional measure of control dimensionality. However, several load-bearing points need strengthening before the main claims are fully supported: the OBC learning signal is not diagnosed, the expert baseline used for normalization is ambiguous, and the vocabulary ablation is single-seed.

major comments (4)
  1. [Training Arnold on 14 tasks (pre-training); Figure 4; Methods (Table 5)] The central 14-task expert-level claim rests on OBC. As described, the student starts from a random policy (initial action standard deviation 1.0 in Table 5) and is trained solely by MSE regression to expert actions on the student's own rollouts, with no initial expert dataset. Unlike DAgger, nothing anchors the student to the expert's state distribution at the start. The paper reports no state-distribution divergence measure, no early-training diagnostic, and no ablation that adds an expert-initialized buffer. If the teacher policies output arbitrary or unreliable actions on the student's early out-of-distribution states, the imitation gradient is not a reliable learning signal and the 14-task result would be fragile. Please provide a quantitative comparison of student and expert state distributions during training and/or an empirical BC-with-expert-buffer control.
  2. [Arnold reaches super-expert performance with RL fine-tuning and self-distillation; Figure 5] The headline results are expressed as percentages of 'expert performance' (e.g., 105.13 ± 1.60 %), but the manuscript never defines how the expert baseline was computed: deterministic vs sampled actions, number of episodes, reward vs solved fraction, and whether the protocol is identical across BC, OBC, PPO and Arnold. Without this, the 'super-expert' claim is not interpretable. Please specify the expert normalization and report absolute rewards/success rates.
  3. [What is the role of the learned sensorimotor vocabulary?; Table 1] The vocabulary-sharing ablation is based on one seed and reports approximate drops (∼35%, ∼19%, ∼11%, ∼37%, ∼10%) without error bars or a significance test. Since the compositional sensorimotor vocabulary is presented as a key innovation, this is load-bearing. Please provide multi-seed results with s.e.m. and a statistical comparison.
  4. [Arnold is more data-efficient for novel tasks; Figure 5C] The contribution that pretrained Arnold learns novel tasks 'with a fraction of the data' is supported only by a visual comparison of learning curves, without error bars, a predefined success threshold, or a statistical test. Please quantify steps-to-threshold or area under the curve over multiple seeds.
minor comments (8)
  1. [General] Typos: 'Boading' -> 'Baoding' (Results), 'musculoseketal' -> 'musculoskeletal' (Results), 'suing' -> 'using' (Methods), 'substantialy' -> 'substantially' (Multi-task RL).
  2. [Figure 2 caption] The caption contains a stray 'and': 'muscle activations as words in the target language and.'
  3. [Methods, Sensorimotor vocabulary] 'Transformer networks are positionally invariant' is imprecise; standard transformers without positional encoding are permutation-invariant/equivariant, not positionally invariant. Please rephrase.
  4. [Table 3] Several entries such as 'Pos dist. 1 1' are hard to parse; please format the reward parameters consistently.
  5. [Figure 5B] Please specify whether the error bars are across seeds, episodes, or both; the caption says '200 episodes, mean ± standard error' but not how seeds enter.
  6. [Figure S2] Clarify what 'solved fraction' means for tasks without a binary solved condition, e.g., Walk to point.
  7. [Table S2] Table S2 duplicates Table 1. Either remove it or present it as a full version of the same ablation.
  8. [Reproducibility] The manuscript does not state whether code, trained checkpoints, or the teacher policies will be released. Please add a data/code availability statement.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Arnold's performance claims are empirical, externally benchmarked, and not derived from its training inputs or self-citations.

full rationale

Arnold's central claim—expert or super-expert performance on 14 MyoSuite tasks—is an empirical result measured by environment reward (105.13 ± 1.60% of expert performance, 200 test episodes, mean ± s.e.m.), not a quantity computed from the teacher policies or from the sensorimotor vocabulary by construction. The training pipeline (OBC, PPO fine-tuning, self-distillation) uses expert policies as imitation targets, but the final evaluation is external to those teachers; there is no equation in which the reported performance is equivalent to the teachers' rewards or to the imitation loss. The self-citations in the paper provide teacher policies (e.g., Chiappa et al. 2024 for Lattice-PPO and hand experts; Caggiano et al. 2022/2023 for MyoChallenge winners; Simos et al. 2025 for the locomotion expert) and analysis methods (CSI from Chiappa et al. 2024), but these are not invoked as a uniqueness theorem or as a substitute for empirical validation. The loss-of-function study (Figure 5B) directly compares OBC, BC, PPO, and OBC-PPO, and the transfer experiment (Figure 5C) compares pretrained versus randomly initialized networks against the same held-out tasks—both are controlled empirical comparisons. The synergy analysis is a fresh application of PCA/CSI to activations collected from Arnold; its conclusion of limited transferability is a measured outcome, not an input. The Limitations section states that Arnold 'struggled to discover new control policies without demonstrations' and 'was not able to solve these tasks zero-shot'; these are honest limitations about generalization, and they do not indicate circularity. No load-bearing step reduces to its own inputs.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

No new physical entities are postulated. The central numerical claims rest on hand-chosen environment allocations, an observation history length, and the hand-designed vocabulary sharing scheme, plus fidelity of the simulator and competence of the teacher policies.

free parameters (3)
  • Per-task environment allocation (Table S1) = Elbow pose 2; Thumb/Index/Middle/Ring/Little reach 2 each; Pen/Die reorient 2 each; Baoding CW 2; Baoding CCW 6; Baoding
    Chosen heuristically from single-task imitation difficulty (Figure S1); changes task sampling balance and affects multi-task training outcome.
  • Observation history length = 5 time steps
    Set by design; an attention-over-time variant with 96 steps was tested and not adopted for deterministic experts (Figure S3). Load-bearing for architecture size.
  • Sensorimotor vocabulary token sharing scheme = 214 tokens, hand-designed semantic groups
    The exact partition of sensors and actuators into shared compositional tokens is a design choice; the disjoint-vocabulary ablation (Table 1) shows performance depends on it.
assumptions (3)
  • domain assumption MyoSuite/MuJoCo Hill-type muscle models are a valid proxy for human musculoskeletal dynamics
    All results are simulation-based; the paper's claims about motor control insights inherit this assumption.
  • domain assumption The prior expert policies used as OBC teachers are competent and queried correctly on student states
    Teacher quality caps student performance; experts come from prior competitions and the authors' earlier works.
  • domain assumption PPO-style on-policy training with the given hyperparameters (Table 5) converges in 55M steps
    The training recipe is empirical; no convergence guarantee is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Arnold: a generalist muscle transformer policy." pith.science (2026). https://pith.science/paper/OENFZFZX

@misc{pith2026250818066,
  author       = {Pith},
  title        = {Pith review of: Arnold: a generalist muscle transformer policy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OENFZFZX}},
  note         = {Machine review of arXiv:2508.18066}
}
read the original abstract

Controlling high-dimensional and nonlinear musculoskeletal models of the human body is a foundational scientific challenge. Recent machine learning breakthroughs have heralded policies that master individual skills like reaching, object manipulation and locomotion in musculoskeletal systems with many degrees of freedom. However, these agents are merely "specialists", achieving high performance for a single skill. In this work, we develop Arnold, a generalist policy that masters multiple tasks and embodiments. Arnold combines behavior cloning and fine-tuning with PPO to achieve expert or super-expert performance in 14 challenging control tasks from dexterous object manipulation to locomotion. A key innovation is Arnold's sensorimotor vocabulary, a compositional representation of the semantics of heterogeneous sensory modalities, objectives, and actuators. Arnold leverages this vocabulary via a transformer architecture to deal with the variable observation and action spaces of each task. This framework supports efficient multi-task, multi-embodiment learning and facilitates rapid adaptation to novel tasks. Finally, we analyze Arnold to provide insights into biological motor control, corroborating recent findings on the limited transferability of muscle synergies across tasks.

Figures

Figures reproduced from arXiv: 2508.18066 by the authors.

Figure 1
Figure 1. Arnold is a generalist musculoskeletal policy that controls multiple human embodiments and solves 14 different motor control tasks. We train Arnold in three steps. Step 1: Arnold learns to imitate multiple experts in parallel using on-policy behavior cloning (OBC), thus building a general foundation. Step 2: Different versions of Arnold are specialized to individual tasks using PPO. These specialists might outperfor… view at source ↗
Figure 2
Figure 2. Architecture of the muscle transformer policy. Observation time series are transformed into sensory encodings by a linear transformation. Role embeddings are extracted from the sensorimotor vocabulary and added to the sensory encoding before they are processed by the transformer encoder. A transformer decoder processes actuator and value embeddings attending to the encoder’s output, to perform value estimation and t… view at source ↗
Figure 3
Figure 3. Performance comparison between three variants of PPO and Multitask-PPO (MT-PPO). Reward standardization (rew norm) and per￾component observation standardization (obs norm) are ablated as variants of PPO. They are essential to reach satisfying performance on the "reach" tasks, but overall PPO fails to tackle the locomotion and manipulation tasks in contrast to the experts. d Training Arnold on 14 tasks from scratch (… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Difference between standard (top) and on-policy BC (OBC) (bot￾tom). By using on-policy trajectories to train the student policy, we ensure that the transitions adhere to the same distribution as the ones the agent will encounter during deployment. Training Arnold on 14…
Figure 5
Figure 5. Figure 5: A Multi-task performance comparison between PPO, BC and Arnold, compared to the single task expert performance (mean ± s.e.m. across 3 random seeds). B Single task and average performance of BC, OBC, OBC without observation normalization, OBC-PPO and Arnold (200 episod…
Figure 6
Figure 6. Figure 6: Arnold’s sensorimotor vocabulary includes 214 tokens that can represent the sensory input and motor output for 14 tasks involving 4 different embodiments. sensorimotor vocabulary containing a set of words that can be arranged to describe a wide variety of sensory input…
Figure 7
Figure 7. Figure 7: Average relative task performance across the 11 MyoHand tasks, when the control signal is projected on action subspaces generated by progressively more principal components (from 1 to 39 muscles, left to right). Blue: task-specific principal components. Red: shared pri…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. MyoChallenge 2025: A New Benchmark for Human Athletic Intelligence

    cs.RO 2026-05 unverdicted novelty 6.0 of 10

    MyoChallenge 2025 introduces standardized table tennis and soccer tasks for musculoskeletal models in the MyoSuite simulation framework to benchmark athletic motor control algorithms.

Reference graph

Works this paper leans on

88 extracted references · 53 canonical work pages · cited by 1 Pith paper

  1. [1]

    Deepmind control suite

    Yuval Tassa, Y otam Doron, Alistair Muldal, Tom Erez, Y azhe Li, Diego de Las Casas, David Budden, Abbas Abdolmaleki, Josh Merel, Andrew Lefrancq, et al. Deepmind control suite. arXiv preprint arXiv:1801.00690 , 2018

    arXiv 2018

  2. [2]

    Isaac gym: High performance gpu-based physics simulation for robot learning

    Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, and Gavriel State. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470 , 2021

    arXiv 2021

  3. [3]

    Jacobsen, E

    S. Jacobsen, E. Iversen, D. Knutti, R. Johnson, and K. Biggers. Design of the Utah/M.I.T. Dextrous Hand. In 1986 IEEE International Conference on Robotics and Automation Proceedings , volume 3, pages 1520–1532, April 1986. doi: 10.1109/ROBOT.1986.1087395

    work page arXiv 1986

  4. [4]

    Fast, strong and compliant pneumatic actuation for dexterous tendon-driven hands

    Vikash Kumar, Zhe Xu, and Emanuel Todorov. Fast, strong and compliant pneumatic actuation for dexterous tendon-driven hands. arXiv preprint arXiv:2013.6630771, 2013

    work page arXiv 2013

  5. [5]

    Michelis, Lewis S

    Oncay Y asa, Y asunori Toshimitsu, Mike Y . Michelis, Lewis S. Jones, Miriam Filippi, Thomas Buchner, and Robert K. Katzschmann. An Overview of Soft Robotics. Annual Review of Control, Robotics, and Autonomous Systems , 6(Volume 6, 2023):1–29, May 2023. ISSN 2573-5144. doi: 10.1146/annurev-control-062322-100607

    doi:10.1146/annurev-control-062322-100607 2023

  6. [6]

    LEAP Hand: Low-cost, efficient, and anthropomorphic hand for robot learning.arXiv preprint arXiv:2309.06440, 2023

    Kenneth Shaw, Ananye Agarwal, and Deepak Pathak. LEAP Hand: Low-cost, efficient, and anthropomorphic hand for robot learning.arXiv preprint arXiv:2309.06440, 2023

    arXiv 2023

  7. [7]

    https://www.allegrohand.com

    Allegro Hand | robot hand. https://www.allegrohand.com

  8. [8]

    Christoph, Maximilian Eberlein, Filippos Katsimalis, Arturo Roberti, Aristotelis Sympetheros, Michel R

    Clemens C. Christoph, Maximilian Eberlein, Filippos Katsimalis, Arturo Roberti, Aristotelis Sympetheros, Michel R. Vogt, Davide Liconti, Chenyu Y ang, Barnabas Gavin Cangan, Ronan J. Hinchet, and Robert K. Katzschmann. Orca: An open-source, reliable, cost-effective, anthropomorphic robotic hand for uninterrupted dexterous task learning. arXiv preprint arX...

    arXiv 2025

Show all 88 references
  1. [9]

    Robotic artificial muscles: Current progress and future perspectives

    Jun Zhang, Jun Sheng, Ciarán T O’Neill, Conor J Walsh, Robert J Wood, Jee-Hwan Ryu, Jaydev P Desai, and Michael C Yip. Robotic artificial muscles: Current progress and future perspectives. IEEE transactions on robotics , 35(3):761–781, 2019

    2019

  2. [10]

    Temporal difference learning for model predictive control

    Nicklas Hansen, Xiaolong Wang, and Hao Su. Temporal difference learning for model predictive control. arXiv preprint arXiv:2203.04955 , 2022

    2022 arXiv

  3. [11]

    Learning with Muscles: Benefits for Data- Efficiency and Robustness in Anthropomorphic Tasks

    Isabell Wochner, Pierre Schumacher, Georg Martius, Dieter Büchler, Syn Schmitt, and Daniel Haeufle. Learning with Muscles: Benefits for Data- Efficiency and Robustness in Anthropomorphic Tasks. InProceedings of The 6th Conference on Robot Learning , pages 1178–1188. PMLR, March 2023

    2023

  4. [12]

    Contrasting action and posture coding with hierarchical deep neural network models of proprioception

    Kai J Sandbrink, Pranav Mamidanna, Claudio Michaelis, Matthias Bethge, Mackenzie W Mathis, and Alexander Mathis. Contrasting action and posture coding with hierarchical deep neural network models of proprioception. Elife, 12:e81499, 2023

    2023

  5. [13]

    Chiappa, Chris Versteeg, Lee E

    Marin Vargas, Alessandro, Axel Bisi, Alberto S. Chiappa, Chris Versteeg, Lee E. Miller, and Alexander Mathis. Task-driven neural network models predict neural dynamics of proprioception. Cell, 187(7):1745–1761.e19, March 2024. ISSN 0092-8674, 1097-4172. doi: 10.1016/j.cell.2024.02.036

    2024 doi

  6. [14]

    Acquiring musculoskeletal skills with curriculum-based reinforcement learning

    Alberto Silvio Chiappa, Pablo Tano, Nisheet Patel, Abigaïl Ingster, Alexandre Pouget, and Alexander Mathis. Acquiring musculoskeletal skills with curriculum-based reinforcement learning. Neuron, 112(23):3969–3983.e5, December 2024. ISSN 0896-6273. doi: 10.1016/j.neuron.2024.09.002

    2024 doi

  7. [15]

    Deep-learning models of the ascending proprioceptive pathway are subject to illusions

    Adriana Perez Rotondo, Merkourios Simos, Florian David, Sebastian Pigeon, Olaf Blanke, and Alexander Mathis. Deep-learning models of the ascending proprioceptive pathway are subject to illusions. Experimental Physiology, 2025

    2025

  8. [16]

    OpenSim: Open-source software to create and analyze dynamic simulations of movement

    Scott L Delp, Frank C Anderson, Allison S Arnold, Peter Loan, Ayman Habib, Chand T John, Eran Guendelman, and Darryl G Thelen. OpenSim: Open-source software to create and analyze dynamic simulations of movement. IEEE transactions on biomedical engineering , 54(11):1940–1950, 2007

    1940

  9. [17]

    OpenSim: A musculoskeletal modeling and simulation framework for in silico investigations and exchange

    Ajay Seth, Michael Sherman, Jeffrey A Reinbolt, and Scott L Delp. OpenSim: A musculoskeletal modeling and simulation framework for in silico investigations and exchange. Procedia Iutam, 2:212–232, 2011

    2011

  10. [18]

    Scone: Open source software for predictive simulation of biological motion

    Thomas Geijtenbeek. Scone: Open source software for predictive simulation of biological motion. Journal of Open Source Software , 4(38):1421, 2019

    2019

  11. [19]

    MyoSuite: A contact-rich simulation suite for muscu- loskeletal motor control

    Vittorio Caggiano, Huawei Wang, Guillaume Durandau, Massimo Sartori, and Vikash Kumar. MyoSuite: A contact-rich simulation suite for muscu- loskeletal motor control. In Learning for Dynamics and Control Conference , pages 492–507. PMLR, 2022

    2022

  12. [20]

    DEP-RL: Embodied exploration for reinforcement learning in overactuated and musculoskeletal systems

    Pierre Schumacher, Daniel Haeufle, Dieter Büchler, Syn Schmitt, and Georg Martius. DEP-RL: Embodied exploration for reinforcement learning in overactuated and musculoskeletal systems. In The Eleventh International Conference on Learning Representations , 2022

    2022

  13. [21]

    Ostrichrl: A muscu- loskeletal ostrich simulation to study bio-mechanical locomotion

    Vittorio La Barbera, Fabio Pardo, Yuval Tassa, Monica Daley, Christopher Richards, Petar Kormushev, and John Hutchinson. Ostrichrl: A muscu- loskeletal ostrich simulation to study bio-mechanical locomotion. arXiv preprint arXiv:2112.06061 , 2021

    2021 arXiv

  14. [22]

    Deep reinforcement learning for modeling human locomotion control in neuromechanical simulation.Journal of neuroengineering and rehabilitation, 18(1):126, 2021

    Seungmoon Song, Łukasz Kidzi ´nski, Xue Bin Peng, Carmichael Ong, Jennifer Hicks, Sergey Levine, Christopher G Atkeson, and Scott L Delp. Deep reinforcement learning for modeling human locomotion control in neuromechanical simulation.Journal of neuroengineering and rehabilitat...

    2021

  15. [23]

    C. Berg, V. Caggiano, and Vikash Kumar. Sar: Generalization of physiological agility and dexterity via synergistic action representation. arXiv preprint arXiv:2307.03716, 2023

    2023 arXiv

  16. [24]

    Latent exploration for Reinforcement Learning

    Alberto Silvio Chiappa, Alessandro Marin Vargas, Ann Huang, and Alexander Mathis. Latent exploration for Reinforcement Learning. Advances in Neural Information Processing Systems , 36:56508–56530, December 2023

    2023

  17. [25]

    Myochallenge 2022: Learning contact-rich manipulation using a musculoskeletal hand

    Vittorio Caggiano, Guillaume Durandau, Huwawei Wang, Alberto Chiappa, Alexander Mathis, Pablo Tano, Nisheet Patel, Alexandre Pouget, Pierre Schumacher, Georg Martius, Daniel Haeufle, Yiran Geng, Boshi An, Yifan Zhong, Jiaming Ji, Yuanpei Chen, Hao Dong, Y aodong Y ang, Rahul S...

    2022

  18. [26]

    MuscleVAE: Model-based controllers of muscle-actuated characters

    Yusen Feng, Xiyan Xu, and Libin Liu. MuscleVAE: Model-based controllers of muscle-actuated characters. arXiv preprint arXiv:2312.07340 , 2023

    2023 arXiv

  19. [27]

    DynSyn: Dynamical synergistic representation for efficient learning and control in overac- tuated embodied systems

    Kaibo He, Chenhui Zuo, Chengtian Ma, and Y anan Sui. DynSyn: Dynamical synergistic representation for efficient learning and control in overac- tuated embodied systems. arXiv preprint arXiv:2407.11472 , 2024

    2024 arXiv

  20. [28]

    Emergence of natural and robust bipedal walking by learning from biologically plausible objectives

    Pierre Schumacher, Thomas Geijtenbeek, Vittorio Caggiano, Vikash Kumar, Syn Schmitt, Georg Martius, and Daniel FB Haeufle. Emergence of natural and robust bipedal walking by learning from biologically plausible objectives. iScience, 2025

    2025

  21. [29]

    Reinforcement learning-based motion imitation for physiologically plausible musculoskeletal motor control

    Merkourios Simos, Alberto Silvio Chiappa, and Alexander Mathis. Reinforcement learning-based motion imitation for physiologically plausible musculoskeletal motor control. arXiv preprint arXiv:2503.14637 , 2025

    2025

  22. [30]

    Myochallenge 2023: Towards human-level dexterity and agility

    Vittorio Caggiano, Guillaume Durandau, Huiyi Wang, Chun Kwang Tan, Pierre Schumacher, Huawei Wang, Alberto Silvio Chiappa, Alessan- dro Marin Vargas, Alexander Mathis, Jungdam Won, Jungnam Park, Gunwoo Park, Beomsoo Shin, Minsueng Kim, Seungbum Koo, Zhuo Y ang, Wei Dang, Heng ...

    2023

  23. [31]

    MyoChallenge 2024: Physiological Dexterity and Agility in Bionic Humans

    Vittorio Caggiano, Guillaume Durandau, Seungmoon Song, Chun Kwang Tan, Huiyi Wang, Balint Hodossy, Pierre Schumacher, Letizia Gionfrida, Massimo Sartori, and Vikash Kumar. MyoChallenge 2024: Physiological Dexterity and Agility in Bionic Humans. In NeurIPS 2024 Competition T ra...

    2024

  24. [32]

    Schmidt, Timothy D

    Richard A. Schmidt, Timothy D. Lee, Carolee J. Winstein, Gabriele Wulf, and Howard N. Zelaznik. Motor Control and Learning: A Behavioral Emphasis. Human Kinetics, Champaign, IL, 6 edition, 2018. ISBN 9781492547754. With Web Resource

    2018

  25. [33]

    Evolution of behavioural control from chordates to primates

    Paul Cisek. Evolution of behavioural control from chordates to primates. Philosophical T ransactions of the Royal Society B: Biological Sciences , 377(1844):20200522, December 2021. doi: 10.1098/rstb.2020.0522

    2021

  26. [34]

    DMAP: A Distributed Morphological Attention Policy for learning to 16 locomote with a changing body

    Alberto Silvio Chiappa, Alessandro Marin Vargas, and Alexander Mathis. DMAP: A Distributed Morphological Attention Policy for learning to 16 locomote with a changing body. Advances in Neural Information Processing Systems , 35:37214–37227, December 2022

    2022

  27. [35]

    Auto-bidding in real-time auctions via oracle imitation learning (oil)

    Alberto Silvio Chiappa, Briti Gangopadhyay, Zhao Wang, and Shingo Takamatsu. Auto-bidding in real-time auctions via oracle imitation learning (oil). arXiv preprint arXiv:2412.11434 , 2024

    2024 arXiv

  28. [36]

    Mohanty, Carmichael F

    Łukasz Kidzi ´nski, Sharada P . Mohanty, Carmichael F . Ong, Jennifer L. Hicks, Sean F . Carroll, Sergey Levine, Marcel Salathé, and Scott L. Delp. Learning to Run Challenge: Synthesizing Physiologically Accurate Motion Using Deep Reinforcement Learning. In Sergio Escalera and...

    2018 doi

  29. [37]

    Learning to run challenge solutions: Adapting reinforcement learning methods for neuromusculoskeletal environments

    Łukasz Kidzi ´nski, Sharada Prasanna Mohanty, Carmichael Ong, Zhewei Huang, Shuchang Zhou, Anton Pechenko, Adam Stelmaszczyk, Piotr Jarosik, Mikhail Pavlov, Sergey Kolesnikov, Sergey Plis, Zhibo Chen, Zhizheng Zhang, Jiale Chen, Jun Shi, Zhuobin Zheng, Chun Yuan, Zhihui Lin, H...

    2018 arXiv

  30. [38]

    Łukasz Kidzi ´nski, Carmichael Ong, Sharada Prasanna Mohanty, Jennifer Hicks, Sean F . Carroll, Bo Zhou, Hongsheng Zeng, Fan Wang, Rongzhong Lian, Hao Tian, Wojciech Ja´skowski, Garrett Andersen, Odd Rune Lykkebø, Nihat Engin Toklu, Pranav Shyam, Rupesh Kumar Srivastava, Serge...

    1902 arXiv

  31. [39]

    Artificial intelligence for prosthetics: Challenge solutions

    Łukasz Kidzi ´nski, Carmichael Ong, Sharada Prasanna Mohanty, Jennifer Hicks, Sean Carroll, Bo Zhou, Hongsheng Zeng, Fan Wang, Rongzhong Lian, Hao Tian, et al. Artificial intelligence for prosthetics: Challenge solutions. In The NeurIPS’18 Competition: From Machine Learning to...

    2020

  32. [40]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  33. [41]

    Controlvae: Model-based learning of generative controllers for physics-based charac- ters

    Heyuan Y ao, Zhenhua Song, Baoquan Chen, and Libin Liu. Controlvae: Model-based learning of generative controllers for physics-based charac- ters. ACM T rans. Graph., 41(6):183:1–183:16, 2022. doi: 10.1145/3550454.3555434

    2022

  34. [42]

    Td-mpc2: Scalable, robust world models for continuous control

    Nicklas Hansen, Hao Su, and Xiaolong Wang. Td-mpc2: Scalable, robust world models for continuous control. In The T welfth International Conference on Learning Representations (ICLR) , 2024

    2024

  35. [43]

    Rusu, Sergio Gomez Colmenarejo, Caglar Gulcehre, Guillaume Desjardins, James Kirkpatrick, Razvan Pascanu, Volodymyr Mnih, Koray Kavukcuoglu, and Raia Hadsell

    Andrei A. Rusu, Sergio Gomez Colmenarejo, Caglar Gulcehre, Guillaume Desjardins, James Kirkpatrick, Razvan Pascanu, Volodymyr Mnih, Koray Kavukcuoglu, and Raia Hadsell. Policy distillation. arXiv preprint arXiv:1511.06295 , 2016

    2016 arXiv

  36. [44]

    Actor-mimic: Deep multitask and transfer reinforcement learning

    Emilio Parisotto, Jimmy Lei Ba, and Ruslan Salakhutdinov. Actor-mimic: Deep multitask and transfer reinforcement learning. arXiv preprint arXiv:1511.06342, 2016

    2016 arXiv

  37. [45]

    Czarnecki, John Quan, James Kirkpatrick, Raia Hadsell, Nicolas Heess, and Razvan Pascanu

    Y ee Teh, Victor Bapst, Wojciech M. Czarnecki, John Quan, James Kirkpatrick, Raia Hadsell, Nicolas Heess, and Razvan Pascanu. Distral: Robust multitask reinforcement learning. In Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017

    2017

  38. [46]

    Active fine-tuning of generalist policies

    Marco Bagatella, Jonas Hübotter, Georg Martius, and Andreas Krause. Active fine-tuning of generalist policies. arXiv preprint arXiv:2410.05026 , 2024

    2024 arXiv

  39. [48]

    RLDG: Robotic generalist policy distillation via reinforcement learning

    Charles Xu, Qiyang Li, Jianlan Luo, and Sergey Levine. RLDG: Robotic generalist policy distillation via reinforcement learning. arXiv preprint arXiv:2412.09858, 2024

    2024 arXiv

  40. [49]

    Hiroki Furuta, Yusuke Iwasawa, Yutaka Matsuo, and S. Gu. A system for morphology-task generalization via unified representation and behavior distillation. International Conference on Learning Representations , 2022. doi: 10.48550/arXiv.2211.14296

    2022 doi arXiv

  41. [50]

    Improving Policy Optimization with Generalist-Specialist Learning

    Zhiwei Jia, Xuanlin Li, Zhan Ling, Shuang Liu, Yiran Wu, and Hao Su. Improving Policy Optimization with Generalist-Specialist Learning. In Proceedings of the 39th International Conference on Machine Learning , pages 10104–10119. PMLR, June 2022

    2022

  42. [51]

    Decision transformer: Reinforcement learning via sequence modeling

    Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling. Advances in neural information processing systems , 34:15084–15097, 2021

    2021

  43. [52]

    A generalist agent

    Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vin...

    2022 arXiv

  44. [53]

    Robotkeyframing: Learning locomotion with high-level objectives via mixture of dense and sparse rewards

    Fatemeh Zargarbashi, Jin Cheng, Dongho Kang, Robert Sumner, and Stelian Coros. Robotkeyframing: Learning locomotion with high-level objectives via mixture of dense and sparse rewards. Conference on Robot Learning , 2024. doi: 10.48550/arXiv.2407.11562

    2024 doi arXiv

  45. [54]

    Octo: An open-source generalist robot policy

    Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, Jianlan Luo, Y ou Liang Tan, Lawrence Yunliang Chen, Pannag Sanketi, Quan Vuong, Ted Xiao, Dorsa Sadigh, Chelsea Finn, and Sergey Levine. Oct...

    2024 arXiv

  46. [55]

    AnyMorph: Learning transferable polices by inferring agent morphology

    Brandon Trabucco, Mariano Phielipp, and Glen Berseth. AnyMorph: Learning transferable polices by inferring agent morphology. International Conference on Machine Learning, pages 21677–21691. PMLR, 2022

    2022

  47. [56]

    Scaling proprioceptive-visual learning with heterogeneous pre-trained transformers

    Lirui Wang, Xinlei Chen, Jialiang Zhao, and Kaiming He. Scaling proprioceptive-visual learning with heterogeneous pre-trained transformers. In The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024

    2024

  48. [57]

    Body transformer: Leveraging robot embodiment for policy learning

    Carmelo Sferrazza, Dun-Ming Huang, Fangchen Liu, Jongmin Lee, and Pieter Abbeel. Body transformer: Leveraging robot embodiment for policy learning. arXiv preprint arXiv:2408.06316 , 2024

    2024 arXiv

  49. [58]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems , pages 5026–5033. IEEE, 2012

    2012

  50. [59]

    Jack M. Winters. Hill-Based Muscle Models: A Systems Engineering Perspective. In Jack M. Winters and Savio L-Y . Woo, editors, Multiple Muscle Systems: Biomechanics and Movement Organization , pages 69–93. Springer, New Y ork, NY , 1990. ISBN 978-1-4613-9030-5. doi: 10.1007/97...

    1990 doi

  51. [60]

    Attention is All you Need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is All you Need. In Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017

    2017

  52. [61]

    Julian, Karol Hausman, Chelsea Finn, and S

    Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan C. Julian, Karol Hausman, Chelsea Finn, and S. Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. Conference on Robot Learning , 2019

    2019

  53. [62]

    Pomerleau

    Dean A. Pomerleau. ALVINN: An Autonomous Land Vehicle in a Neural Network. InAdvances in Neural Information Processing Systems, volume 1. Morgan-Kaufmann, 1988

    1988

  54. [63]

    Hayes-Michie

    Donald Michie, Michael Bain, and Jean E. Hayes-Michie. Cognitive models from subcognitive skills. In M. Grimble, S. McGhee, and P . Mowforth, editors, Knowledge-Based Systems in Industrial Control . Peter Peregrinus, Stevenage, 1990

    1990

  55. [64]

    Learning to fly

    Claude Sammut, Scott Hurst, Dana Kedzier, and Donald Michie. Learning to fly. In D. Sleeman and P . Edwards, editors, Proceedings of the Ninth International Conference on Machine Learning , pages 385–393, San Francisco, 1992. Morgan Kaufmann

    1992

  56. [65]

    Gordon, and J

    Stephane Ross, Geoffrey J. Gordon, and J. Andrew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. 17 arXiv preprint arXiv:1011.0686 , 2011

    2011 arXiv

  57. [66]

    DeepSeek-AI, Daya Guo, Dejian Y ang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F . Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei...

    2025 arXiv

  58. [67]

    Learning complex dexterous manipulation with deep reinforcement learning and demonstrations

    Aravind Rajeswaran, Vikash Kumar, Abhishek Gupta, Giulia Vezzani, John Schulman, Emanuel Todorov, and Sergey Levine. Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. arXiv preprint arXiv:1709.10087 , 2018

    2018 arXiv

  59. [68]

    Linear combinations of primitives in vertebrate motor control

    Ferdinando A Mussa-Ivaldi, Simon F Giszter, and Emilio Bizzi. Linear combinations of primitives in vertebrate motor control. Proceedings of the National Academy of Sciences , 91(16):7534–7538, 1994

    1994

  60. [69]

    Modularity in the motor system: Decomposition of muscle patterns as combinations of time-varying synergies

    Andrea d’Avella and MMCM Tresch. Modularity in the motor system: Decomposition of muscle patterns as combinations of time-varying synergies. Advances in neural information processing systems , 14, 2001

    2001

  61. [70]

    Combinations of muscle synergies in the construction of a natural motor behavior

    Andrea d’Avella, Philippe Saltiel, and Emilio Bizzi. Combinations of muscle synergies in the construction of a natural motor behavior. Nature neuroscience, 6(3):300–308, 2003

    2003

  62. [71]

    The case for and against muscle synergies

    Matthew C Tresch and Anthony Jarc. The case for and against muscle synergies. Current opinion in neurobiology , 19(6):601–607, 2009

    2009

  63. [72]

    Muscle synergies in neuroscience and robotics: From input-space to task-space perspectives

    Cristiano Alessandro, Ioannis Delis, Francesco Nori, Stefano Panzeri, and Bastien Berret. Muscle synergies in neuroscience and robotics: From input-space to task-space perspectives. Frontiers in computational neuroscience , 7:43, 2013

    2013

  64. [73]

    Learning to use muscles

    Gerald E Loeb. Learning to use muscles. Journal of human kinetics , 76(1):9–33, 2021

    2021

  65. [74]

    Responses to spinal microstimulation in the chronically spinalized rat and their relationship to spinal systems activated by low threshold cutaneous stimulation

    Matthew C Tresch and Emilio Bizzi. Responses to spinal microstimulation in the chronically spinalized rat and their relationship to spinal systems activated by low threshold cutaneous stimulation. Experimental brain research , 129:401–416, 1999

    1999

  66. [75]

    Microstimulation activates a handful of muscle synergies

    Simon A Overduin, Andrea d’Avella, Jose M Carmena, and Emilio Bizzi. Microstimulation activates a handful of muscle synergies. Neuron, 76(6): 1071–1077, 2012

    2012

  67. [76]

    Adaptive intelligence: leveraging insights from adaptive behavior in animals to build flexible ai systems.arXiv preprint arXiv:2411.15234, 2024

    Mackenzie Weygandt Mathis. Adaptive intelligence: leveraging insights from adaptive behavior in animals to build flexible ai systems.arXiv preprint arXiv:2411.15234, 2024

    2024

  68. [77]

    Are transformers truly foundational for robotics? npj Robotics, 3(1):9, 2025

    James AR Marshall and Andrew B Barron. Are transformers truly foundational for robotics? npj Robotics, 3(1):9, 2025

    2025

  69. [78]

    Jeffrey Johnston and Stefano Fusi

    W. Jeffrey Johnston and Stefano Fusi. Abstract representations emerge naturally in neural networks trained to perform multiple tasks. Nature Communications, 14(1):1040, February 2023. ISSN 2041-1723. doi: 10.1038/s41467-023-36583-0

    2023 doi

  70. [79]

    Driscoll, Krishna Shenoy, and David Sussillo

    Laura N. Driscoll, Krishna Shenoy, and David Sussillo. Flexible multitask computation in recurrent networks utilizes shared dynamical motifs. Nature Neuroscience, 27(7):1349–1363, July 2024. ISSN 1546-1726. doi: 10.1038/s41593-024-01668-6

    2024 doi

  71. [80]

    Natural language instructions induce compositional generalization in networks of neurons

    Reidar Riveland and Alexandre Pouget. Natural language instructions induce compositional generalization in networks of neurons. Nature Neuro- science, 27(5):988–999, May 2024. ISSN 1546-1726. doi: 10.1038/s41593-024-01607-5

    2024 doi

  72. [81]

    G. E. Alexander, M. R. DeLong, and P . L. Strick. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annual Review of Neuroscience, 9:357–381, 1986

    1986

  73. [82]

    Large language models and the reverse turing test

    Terrence J Sejnowski. Large language models and the reverse turing test. Neural computation, 35(3):309–342, 2023

    2023

  74. [83]

    Paul Morris Fitts and Michael I. Posner. Human Performance. Brooks/Cole Publishing Company, Belmont, California, 1967

    1967

  75. [84]

    Wolpert, Jörn Diedrichsen, and J

    Daniel M. Wolpert, Jörn Diedrichsen, and J. Randall Flanagan. Principles of sensorimotor learning. Nature reviews. Neuroscience , 12(12): 739–751, December 2011. ISSN 1471-0048. doi: 10.1038/nrn3112

    2011 doi

  76. [85]

    Cortical control of arm movements: a dynamical systems perspective

    Krishna V Shenoy, Maneesh Sahani, and Mark M Churchland. Cortical control of arm movements: a dynamical systems perspective. Annual review of neuroscience, 36(1):337–359, 2013

    2013

  77. [86]

    Bert: Pre-training of deep bidirectional transformers for language understand- ing

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. Naacl, 2019

    2019

  78. [87]

    Self-attention with relative position representations

    Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-attention with relative position representations. arXiv preprint arXiv:1803.02155 , 2018

    2018 arXiv

  79. [88]

    RoFormer: Enhanced transformer with rotary position embedding

    Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. RoFormer: Enhanced transformer with rotary position embedding. arXiv preprint arXiv:2104.09864 , 2023

    2023 arXiv

  80. [89]

    Analysis of the synergies underlying complex hand manipulation

    Emanuel Todorov and Zoubin Ghahramani. Analysis of the synergies underlying complex hand manipulation. In The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society , volume 2, pages 4637–4640. IEEE, 2004. 18 Appendix Figure S1. Single-tas...

    2004

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.