{"id":"8f4aa0dd-e3d4-40c9-99e7-25d1c2e25915","arxiv_id":"2508.18066","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A single transformer policy with a compositional sensorimotor vocabulary achieves expert or super-expert performance on 14 musculoskeletal control tasks spanning four embodiments.","lead":"Researchers created Arnold, one AI policy that controls simulated human muscles across 14 tasks and four body models, from finger precision to walking. It reaches expert-level or better performance by imitating specialists, then improving on them.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"OBC's validity depends on teachers providing competent actions on the student's off-distribution states; the paper reports no divergence or early-learning diagnostics, leaving the 14-task expert-level claim more fragile than the final curves suggest.","rationale":"The reader correctly identifies OBC's teacher-competence assumption as the weakest point. The paper's strongest claim—expert or super-expert performance across 14 tasks—depends on OBC being a valid imitation procedure in this setting. The concern is not that OBC is impossible in principle, but that the paper provides no direct evidence for its central premise: that teacher actions remain valid targets on the student's on-policy states, especially early in training when the student is far from the expert. The final performance curves (Figure 5B) are consistent with the claim, and the 3-seed runs and ablations are good empirical practice. However, without measuring student-teacher state divergence or testing an alternative initialization, the 14-task result is conditional on an untested assumption. The reader's CONDITIONAL verdict already captures this risk, so no change to the verdict is necessary. The proposed concrete test would settle whether the concern actually lands by directly measuring the divergence and the effect of expert anchoring. This is a substantive, testable concern rather than a disagreement with the field's consensus, and it does not impugn the authors' integrity.","tokens_in":21703,"tokens_out":8013,"duration_ms":105736,"concrete_test":"During OBC pretraining on a hard task (e.g., Object relocation or Baoding harder), record student and expert state distributions at fixed checkpoints (0, 1M, 5M, 20M, 50M steps) and compute a distributional distance between them, such as Wasserstein distance on a PCA projection of the observation space or maximum mean discrepancy. Also measure the expert's action-prediction error or action variance on the student's states. If student-expert divergence is large in early training yet final performance reaches expert level, the loss is not relying on teacher competence in the way the paper implies. Additionally, run OBC starting from a student initialized with the expert's behavior policy or with an initial expert dataset (DAgger-style), and compare final performance; a large drop without expert anchoring would confirm the assumption is load-bearing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central result rests on on-policy behavior cloning (OBC): the student collects its own rollouts and is trained to match expert actions on those states. In the paper's Figure 4 and the 'Training Arnold on 14 tasks (pre-training)' section, this is presented as solving distributional shift. But OBC, as implemented, has no initial expert dataset to anchor the student near the expert's state distribution. At initialization the student's action standard deviation is 1.0, so early rollouts explore states far from the expert's support. In high-dimensional, contact-rich tasks like Object relocation and Baoding harder, an expert's action on such out-of-distribution states may be arbitrary or even physically meaningless, so the MSE gradient is not a reliable learning signal. Unlike DAgger, which aggregates expert data onto the student's distribution while retaining an initial expert dataset, OBC starts from scratch and relies wholly on the teacher's ability to label states the teacher never visits. The paper provides no state-distribution divergence measure, no early-training diagnostics, and no ablation that would show OBC works despite, rather than because of, this assumption. If the student's rollout distribution happens to remain close to the expert's, the assumption is benign; if not, the reported expert-level results could depend on unstated initialization or environment structure. This is the load-bearing premise for the 14-task claim, and it is currently unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"Arnold is a generalist transformer policy for musculoskeletal control. Using a 214-token compositional sensorimotor vocabulary, a single encoder-decoder transformer is trained across 14 MyoSuite tasks spanning four musculoskeletal models. The training pipeline consists of on-policy behavior cloning (OBC) from specialist teachers, per-task PPO fine-tuning, and self-distillation. The paper reports expert or super-expert performance (105.13 ± 1.60% of expert reward) averaged over 200 test episodes and three seeds, data-efficient transfer to novel tasks, and a CSI analysis indicating that muscle synergies are task-specific rather than universal.","tokens_in":22031,"tokens_out":8012,"duration_ms":96663,"significance":"If the results hold, this is a substantial empirical contribution: it demonstrates that a single policy can control multiple musculoskeletal embodiments and tasks, and it provides a compositional tokenization scheme for variable observation/action spaces. The paper has real strengths: three-seed error bars for the main comparison, ablations of observation normalization, vocabulary sharing, and OBC versus BC, and the final evaluation is measured by environment reward rather than derived from the teachers, so the central claim is not circular. The CSI analysis is a useful functional measure of control dimensionality. However, several load-bearing points need strengthening before the main claims are fully supported: the OBC learning signal is not diagnosed, the expert baseline used for normalization is ambiguous, and the vocabulary ablation is single-seed.","major_comments":[{"comment":"The central 14-task expert-level claim rests on OBC. As described, the student starts from a random policy (initial action standard deviation 1.0 in Table 5) and is trained solely by MSE regression to expert actions on the student's own rollouts, with no initial expert dataset. Unlike DAgger, nothing anchors the student to the expert's state distribution at the start. The paper reports no state-distribution divergence measure, no early-training diagnostic, and no ablation that adds an expert-initialized buffer. If the teacher policies output arbitrary or unreliable actions on the student's early out-of-distribution states, the imitation gradient is not a reliable learning signal and the 14-task result would be fragile. Please provide a quantitative comparison of student and expert state distributions during training and/or an empirical BC-with-expert-buffer control.","section":"Training Arnold on 14 tasks (pre-training); Figure 4; Methods (Table 5)"},{"comment":"The headline results are expressed as percentages of 'expert performance' (e.g., 105.13 ± 1.60 %), but the manuscript never defines how the expert baseline was computed: deterministic vs sampled actions, number of episodes, reward vs solved fraction, and whether the protocol is identical across BC, OBC, PPO and Arnold. Without this, the 'super-expert' claim is not interpretable. Please specify the expert normalization and report absolute rewards/success rates.","section":"Arnold reaches super-expert performance with RL fine-tuning and self-distillation; Figure 5"},{"comment":"The vocabulary-sharing ablation is based on one seed and reports approximate drops (∼35%, ∼19%, ∼11%, ∼37%, ∼10%) without error bars or a significance test. Since the compositional sensorimotor vocabulary is presented as a key innovation, this is load-bearing. Please provide multi-seed results with s.e.m. and a statistical comparison.","section":"What is the role of the learned sensorimotor vocabulary?; Table 1"},{"comment":"The contribution that pretrained Arnold learns novel tasks 'with a fraction of the data' is supported only by a visual comparison of learning curves, without error bars, a predefined success threshold, or a statistical test. Please quantify steps-to-threshold or area under the curve over multiple seeds.","section":"Arnold is more data-efficient for novel tasks; Figure 5C"}],"minor_comments":[{"comment":"Typos: 'Boading' -> 'Baoding' (Results), 'musculoseketal' -> 'musculoskeletal' (Results), 'suing' -> 'using' (Methods), 'substantialy' -> 'substantially' (Multi-task RL).","section":"General"},{"comment":"The caption contains a stray 'and': 'muscle activations as words in the target language and.'","section":"Figure 2 caption"},{"comment":"'Transformer networks are positionally invariant' is imprecise; standard transformers without positional encoding are permutation-invariant/equivariant, not positionally invariant. Please rephrase.","section":"Methods, Sensorimotor vocabulary"},{"comment":"Several entries such as 'Pos dist. 1 1' are hard to parse; please format the reward parameters consistently.","section":"Table 3"},{"comment":"Please specify whether the error bars are across seeds, episodes, or both; the caption says '200 episodes, mean ± standard error' but not how seeds enter.","section":"Figure 5B"},{"comment":"Clarify what 'solved fraction' means for tasks without a binary solved condition, e.g., Walk to point.","section":"Figure S2"},{"comment":"Table S2 duplicates Table 1. Either remove it or present it as a full version of the same ablation.","section":"Table S2"},{"comment":"The manuscript does not state whether code, trained checkpoints, or the teacher policies will be released. Please add a data/code availability statement.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the teachers are largely the authors' own prior work or competitions in which they participated, so the percentage-normalization must be fully transparent. This is not circular because final evaluation uses environment rewards, but it raises the bar for reporting the expert baseline. The paper would also benefit from a code/data release statement. I see no reason to doubt the empirical results, but the OBC diagnostics and the one-seed vocabulary ablation need strengthening before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read the Arnold paper. The headline claim holds up: a single transformer policy with a compositional sensorimotor vocabulary reaches expert or super-expert performance on 14 MyoSuite tasks spanning four embodiments. That is new. Prior work was single-task specialists or rigid-body generalists; nobody had done multi-embodiment musculoskeletal policy distillation at this scale.\n\nThe recipe is OBC + PPO fine-tuning + self-distillation, and the paper does the right kind of homework. The loss-of-function analysis is informative: PPO from scratch fails, BC struggles, OBC reaches near-expert, and the final distillation gives 105% ± 1.6% of expert reward. Three seeds with error bars on the main curves. The sensorimotor vocabulary is the genuine innovation: 214 tokens with compositional role embeddings, and the ablation shows a disjoint vocabulary degrades performance. The transfer results—pretraining on ten tasks speeds learning on four held-out tasks—are clean.\n\nThe weaknesses are mostly verification gaps. There is no code release, which makes the 14-task claim impossible to fully reproduce. Table 1 (vocabulary ablation) is one seed, so treat that number as a hint. The exact expert baseline used for percentage normalization is not pinned down. And the stress-test concern about OBC is real in principle: early in training, the student visits states the expert may not handle well, and the paper never measures student-teacher state distribution divergence. But the empirical evidence—three seeds of OBC hitting 99% expert performance across all 14 tasks—suggests the teacher labels were good enough in practice. The concern is a missing diagnostic, not a demonstrated flaw.\n\nThe muscle synergy analysis is a nice extra, and the negative result (no transferable synergies across tasks) is honest and interesting. The paper is careful not to overclaim zero-shot generalization or biological universality.\n\nWho is this for? Anyone working on multi-task RL, policy distillation, or musculoskeletal control. It is a solid foundation-model step, even if it falls short of a full generalist agent. I would send it to a serious referee. The referee should ask for code, single-seed details, and ideally a divergence metric for OBC, but the central claim is credible and the contribution is worth archiving.\n\nMy recommendation: send to peer review. The core result is solid; the missing pieces are reproducible-support, not correctness.","headline":"Solid empirical generalist policy for musculoskeletal control; the 14-task claim is credible but hinges on a few addressable verification gaps.","tokens_in":22488,"tokens_out":2295,"would_cite":true,"duration_ms":26708,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Arnold, a single transformer policy trained by on-policy behavior cloning, per-task PPO fine-tuning, and self-distillation, matches or exceeds specialist experts across 14 musculoskeletal control tasks spanning four body models.","keywords":["musculoskeletal control","multi-task policy","transformer","on-policy behavior cloning","imitation learning","muscle synergies","MyoSuite","self-distillation"],"falsifier":"During OBC pretraining, record for each task the expert policy's action error on the states the student actually visits. The paper's explanation predicts that this on-policy error is small enough to drive imitation. A finding that one or more teachers are systematically wrong on the student's state distribution—or, equivalently, an experiment in which corrupting one teacher's labels on student rollouts leaves that task's final performance unchanged—would undercut the claim that OBC is what carries Arnold's success.","tokens_in":21625,"feed_emoji":"🦾","tokens_out":6931,"duration_ms":76981,"temperature":0.7,"pith_summary":"This paper aims to show that one neural network can act as a generalist motor controller for the human musculoskeletal system. The agent, Arnold, is a transformer that represents each sensory channel and each muscle as tokens drawn from a 214-word sensorimotor vocabulary, so the same policy can handle different body models, observation spaces, and action spaces without architectural changes. Its training is a three-stage distillation recipe: on-policy behavior cloning from specialist expert policies, per-task PPO fine-tuning to exceed those experts, and self-distillation of the improved specialists back into the single generalist. On the 14 MyoSuite tasks spanning four musculoskeletal models, the paper reports expert or super-expert performance, on average 105.13% of the expert reward. The paper further reports that Arnold's muscle activations are low-dimensional within each task but are not shared across tasks, which it reads as evidence against universal muscle synergies in this setting.","feed_headline":"One transformer policy beats 14 muscle-task specialists","feed_subtitle":"Imitation on the student's own rollouts plus PPO fine-tuning and self-distillation nets 105 percent of expert reward.","key_machinery":"The load-bearing object is the sensorimotor vocabulary: a set of 214 learned word embeddings, combined by addition to form a role embedding for each sensory input and each actuator. Role embeddings are added to linear sensory encodings before an encoder-decoder transformer, and the decoder emits one action token per muscle plus a value token, making variable observation and action spaces a matter of token composition rather than architecture change. The training mechanism is on-policy behavior cloning (OBC): the student collects rollouts in the environment and the teacher experts label those states with target actions, avoiding the distributional mismatch of offline behavior cloning. Around","core_discovery":"Arnold's central claim is that expert-level multi-task musculoskeletal control can be obtained by distillation rather than by solving each task from scratch. The key correction to ordinary behavior cloning is to run the imitation on the student's own rollouts: expert policies label the states Arnold visits, so the training distribution matches deployment. With that change, Arnold reaches expert performance across all 14 tasks; with per-task PPO fine-tuning followed by distillation back into the generalist, it reaches super-expert performance. The second claim is that a compositional sensorimotor vocabulary of 214 learned tokens—where a sensory channel like 'right soleus muscle length' is the","pith_inferences":["The vocabulary's compositionality implies that adding a new body part or task should require only a small number of new tokens, since shared words like muscle names, side labels, and semantic roles are reused; the paper demonstrates 214 tokens for 14 tasks but does not test scaling limits.","The OBC recipe depends on having queryable teacher policies in the loop; it would not transfer directly to fixed offline demonstration datasets, a boundary implicit in the method's design.","The absence of transferable synergies may reflect the specific task mix and the distillation objective rather than a general property of multi-task motor learning; the paper itself suggests that more tasks or more similar tasks could change this conclusion.","A direct test of the vocabulary's role: remove the shared tokens by replacing compositional role embeddings with disjoint per-task embeddings on a held-out task and measure how much slower learning becomes; the paper's Table 1 already shows large drops on five tasks, suggesting the effect should be measurable on new tasks too."],"forward_implications":["A single 4.4-million-parameter transformer policy can match or exceed task-specific experts across four musculoskeletal models, so specialist policies do not have to be discarded when a generalist is needed.","OBC closes the imitation gap that standard BC leaves open: on the 14-task set it reaches 98.97% of expert reward versus 80.89% for BC and 50.05% for PPO trained from scratch.","RL fine-tuning followed by self-distillation raises the generalist to 105.13% of expert reward, so a distilled multi-task policy can serve as a better starting point for further learning rather than a ceiling.","A policy pretrained on ten tasks learns four held-out tasks faster than randomly initialized specialists, indicating transferable sensorimotor representations, though not zero-shot generalization.","Within each task Arnold's muscle activations lie near a low-dimensional subspace, but these subspaces differ across tasks, suggesting muscle synergies are task-specific in this generalist."],"supporting_citations":[{"why":"Provides the MyoSuite simulator with the musculoskeletal models and the 14 tasks that define the benchmark.","marker":"[19]"},{"why":"Provides curriculum-trained Baoding experts and the CSI analysis method used for the muscle-synergy study.","marker":"[14]"},{"why":"Provides expert policies for finger-reaching and pen/die reorientation tasks used as teachers.","marker":"[24]"},{"why":"Supplies the winning Baoding-ball manipulation experts used as teachers for four tasks.","marker":"[25]"},{"why":"Provides the motion-imitation expert for the Walk to point locomotion task.","marker":"[29]"},{"why":"Supplies the object-relocation expert used as the teacher for that task.","marker":"[30]"},{"why":"Introduces the oracle imitation learning variant of on-policy behavior cloning that OBC adapts.","marker":"[35]"},{"why":"Supplies PPO, used for fine-tuning and as the baseline reinforcement learning algorithm.","marker":"[40]"},{"why":"Provides the encoder-decoder transformer architecture that Arnold is built on.","marker":"[60]"},{"why":"Introduces DAgger's distribution-matching idea that OBC extends by discarding offline data.","marker":"[65]"}],"fun_headline_variants":["One transformer masters 14 musculoskeletal tasks","Generalist policy beats single-task muscle specialists","Self-training yields super-expert muscle controller","Arnold: one policy for 14 body control skills","Distillation makes a super-expert muscle policy"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"OBC assumes that the teacher policies give correct target actions on the states the student visits during its own rollouts; if a teacher is unreliable precisely where Arnold goes, the imitation signal degrades and the reported expert-level results on all 14 tasks would not transfer.","fun_headline_variants_meta":{"raw":{"variants":["One transformer masters 14 musculoskeletal tasks","Generalist policy beats single-task muscle specialists","Self-training yields super-expert muscle controller","Arnold: one policy for 14 body control skills","Distillation makes a super-expert muscle policy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000254,"raw_usage":{"total_tokens":1382,"prompt_tokens":698,"completion_tokens":684,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":442,"completion_tokens_details":{"reasoning_tokens":615}},"tokens_in":442,"tokens_out":684,"duration_ms":8416,"temperature":1.0,"reasoning_tokens":615,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T16:35:37.005716+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"During OBC pretraining, record for each task the expert policy's action error on the states the student actually visits. The paper's explanation predicts that this on-policy error is small enough to drive imitation. A finding that one or more teachers are systematically wrong on the student's state distribution—or, equivalently, an experiment in which corrupting one teacher's labels on student rollouts leaves that task's final performance unchanged—would undercut the claim that OBC is what carries Arnold's success.","supporting_citations":[{"cited_title":"MyoSuite: A contact-rich simulation suite for muscu- loskeletal motor control","cited_arxiv_id":null,"evidence_quote":"Provides the MyoSuite simulator with the musculoskeletal models and the 14 tasks that define the benchmark."},{"cited_title":"Acquiring musculoskeletal skills with curriculum-based reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Provides curriculum-trained Baoding experts and the CSI analysis method used for the muscle-synergy study."},{"cited_title":"Latent exploration for Reinforcement Learning","cited_arxiv_id":null,"evidence_quote":"Provides expert policies for finger-reaching and pen/die reorientation tasks used as teachers."},{"cited_title":"Myochallenge 2022: Learning contact-rich manipulation using a musculoskeletal hand","cited_arxiv_id":null,"evidence_quote":"Supplies the winning Baoding-ball manipulation experts used as teachers for four tasks."},{"cited_title":"Myochallenge 2023: Towards human-level dexterity and agility","cited_arxiv_id":null,"evidence_quote":"Supplies the object-relocation expert used as the teacher for that task."},{"cited_title":"Auto-bidding in real-time auctions via Oracle Imitation Learning (OIL)","cited_arxiv_id":"2412.11434","evidence_quote":"Introduces the oracle imitation learning variant of on-policy behavior cloning that OBC adapts."},{"cited_title":"Attention is All you Need","cited_arxiv_id":null,"evidence_quote":"Provides the encoder-decoder transformer architecture that Arnold is built on."}],"review_version":1}