Pith. sign in

REVIEW 4 major objections 5 minor 32 references

LooperMuscle claims a structured mixture-of-experts actor, an expert-aware distributional critic, and contribution-routed replay can train humanoid whole-body tracking policies in about 45 minutes while recovering 72% of the quality of a si

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 00:11 UTC pith:KKJAY5JL

load-bearing objection A credible, cleanly reported empirical advance on fast humanoid tracking; the central attribution to MoE structure needs a capacity control, but the 34% error reduction over FastSAC-MLP is a real, useful result. the 4 major comments →

arxiv 2608.00820 v1 pith:KKJAY5JL submitted 2026-08-01 cs.RO

LooperMuscle: Fast and Stable Learning of Humanoid Whole-Body Tracking via Structured Mixture-of-Experts

Boyi Liu , Qijin Li , Tianqi Yu , Qinrui Yan , Xingxing Zuo This is my paper
classification cs.RO
keywords whole-body trackinghumanoid controlmixture-of-expertsoff-policy reinforcement learningdistributional criticreplay buffermotion imitationFastSAC
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

LooperMuscle is a reinforcement-learning recipe for whole-body humanoid motion tracking that claims to close much of the quality gap between fast off-policy training and slow, high-quality on-policy training. On 40 motion-capture sequences, the paper reports reducing mean body-position error from 0.153 m to 0.101 m relative to the FastSAC baseline, a 34 percent improvement, while reaching 72 percent of PPO's converged normalized reward in roughly 45 minutes of GPU training instead of PPO's six hours. The claimed mechanism is a structured design: a policy decomposed into four expert networks gated by upper-body and lower-body groups, a distributional critic that mirrors the experts' gating for credit assignment, and a replay buffer that routes samples according to per-expert contribution with a progress-dependent curriculum. If the claim holds, humanoid motion iteration cycles could shrink from hours to under an hour without giving up most of the tracking quality that today requires on-policy training.

Core claim

The paper's central claim is that the speed-performance gap in humanoid whole-body tracking has structural causes: a monolithic actor network, a scalar value function, and uniform experience replay reinforce each other. LooperMuscle replaces all three at once: a mixture-of-experts actor with per-joint-group gating and anti-collapse regularization; a distributional critic with per-expert value heads aggregated by the same gating weights; and quota-routed replay that lets expert contribution steer which transitions enter the batch and when. These components form a closed loop: expert contributions route data, routed data shapes value learning, and value gradients refine specialization. The pap

What carries the argument

The load-bearing object is the closed training loop coupling three mechanisms: (1) a K-expert actor fused by per-joint-group gating weights, with learnable per-joint scaling and output alignment plus a KL-based load-balance regularizer; (2) an expert-aware distributional critic whose per-expert categorical return distributions are aggregated by the actor's own gating weights, making value structure isomorphic to policy structure; (3) contribution-routed replay, where each transition is tagged by each expert's mean absolute action contribution and sampled under per-expert quotas with deferred buckets released by progress-dependent schedules. The loop is what carries the argument: expert contr

Load-bearing premise

The speed-quality numbers come from a simulator that hands the policy ground-truth body positions and orientations that no real sensor can measure; the load-bearing assumption is that the ranking of LooperMuscle against FastSAC and PPO stays the same when only onboard-estimable signals are used, an interface on which the real robot is currently validated only qualitatively.

What would settle it

Run the same 40-sequence benchmark and full ablation suite with the ground-truth body-state signals removed from the observation, keeping only joint encoders, base IMU, and body-relative references; if LooperMuscle's body-error advantage over the monolithic baseline shrinks to a small margin or its ranking relative to PPO inverts, the central practical claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Off-policy whole-body tracking policies can be trained in about 45 minutes on a single GPU and reach roughly 72 percent of PPO's converged reward, letting developers iterate on motion libraries in minutes rather than hours.
  • The 34 percent body-error reduction over FastSAC is consistent across all six motion categories, from 26.7 percent for walking to 40.6 percent for fall-and-get-up, so the gain concentrates where tracking is hardest.
  • The ablations attribute the largest share of the gain to the MoE actor itself; removing it returns the method to the monolithic baseline, and removing quota replay, deferred scheduling, or the expert-aware critic each degrades performance by 12 to 26 percent.
  • The paper's cross-paradigm comparison suggests expert decomposition helps under on-policy PPO methods as well as under off-policy FastSAC, pointing to a general design principle for whole-body tracking.
  • Deployment on a physical humanoid with an onboard-only observation interface, while qualitative, indicates the recipe produces policies that can be retrained for real hardware without the simulator's privileged signals.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The per-expert action-contribution vector stored with each transition could serve as a generic saliency signal beyond replay, for example to adapt the group partition, expert count, or per-group temperatures during training based on observed specialization.
  • The deferred-unlock schedule is essentially a self-paced curriculum keyed to policy progress rather than task difficulty; the same idea may transfer to other off-policy continuous-control settings where difficult transitions destabilize early learning.
  • If the privileged-interface ranking survives evaluation under the deployable interface, a practical workflow would be to benchmark in the simulator at 45 minutes per motion and then retrain the final policy under onboard observations, making per-motion fine-tuning economically feasible.
  • The KL load-balance regularizer acting on batch-level gating statistics may also preserve action diversity, since keeping weaker experts active maintains exploration that a collapsed gate would remove.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes LooperMuscle, a FastSAC-based off-policy framework for 29-DoF humanoid whole-body tracking. It combines a semantically structured mixture-of-experts actor (K=4 experts, per-joint-group gating), an expert-aware distributional critic, and quota-routed replay with deferred scheduling. On 40 LAFAN1 sequences in MJLab, the authors report 0.101 m mean body error versus 0.153 m for FastSAC-MLP and 0.082 m for PPO, normalized reward 0.723 versus 0.648 and 1.000, and training time about 45 min versus 15 min and 360 min. The paper also reports ablations for each component, a hardware deployment on Unitree G1 via the Holosoma interface, and a contextual comparison with concurrent MoE-based methods.

Significance. If substantiated, the result would be a meaningful step toward closing the speed-performance gap in off-policy humanoid whole-body tracking: it would show that structured policy decomposition and expert-aware value learning can substantially improve tracking quality while retaining a large wall-clock advantage over PPO. The paper is commendably candid about its limitations, including the privileged observation interface, the missing parameter-matched baseline, and the fixed hyperparameter choices. It also promises release of code and reports multiple seeds. However, as it stands, the central structural claim is not fully verified because the main comparison does not isolate architecture from capacity, and the quantitative benchmark is run under an interface that is not deployable. These issues affect the headline conclusions, although they appear fixable within the scope of a revision.

major comments (4)
  1. [Abstract, Fig. 1, Table I] The headline claim of '47.5% higher reward than FastSAC' is not supported by Table I: normalized reward is 0.723 for LooperMuscle and 0.648 for FastSAC-MLP, a relative increase of about 11.6%, not 47.5%. If the 47.5% refers to raw training reward on a single sequence or a different checkpoint, the metric, sequence, and training phase must be specified. As written, the abstract and Table I contradict each other on a central quantitative claim.
  2. [Sec. IV-B, Sec. IV-D, Table III] The claim that the improvement comes from structured MoE rather than capacity/compute is not established. LooperMuscle uses K=4 experts with per-joint scaling and output alignment, and trains about 3x longer than FastSAC-MLP. The 'w/o MoE Actor' row in Table III reduces to FastSAC-MLP, removing not only the MoE architecture but also the expert-aware critic and quota routing; it therefore cannot isolate the actor's contribution. The statement that 'expert decomposition is the primary source of representational gain' is unsupported. The manuscript itself names parameter-matched baselines as a missing control (Sec. IV-B). Without a monolithic MLP with comparable parameter count and training budget, the 34.0% reduction may be an effect of capacity, not structure.
  3. [Sec. IV-A, Sec. IV-G, Sec. V] All quantitative benchmarking (Table I, Table II, Table III) uses MJLab's privileged ground-truth global key-body positions/orientations, which no onboard sensor provides. The hardware policy is retrained under a different 154-dim Holosoma interface and validated only qualitatively. Therefore the central quantitative claims (0.101 m body error, 0.723 normalized reward, 45 min) are not shown to transfer to a deployable observation interface. The paper states this limitation, but the conclusion still frames LooperMuscle as delivering 'practical efficiency for rapid policy iteration.' Either add a quantitative deployable-interface benchmark or substantially qualify the practical claim.
  4. [Sec. IV-D, Table III] The ablation evidence for the 'closed-loop' coupling is weaker than the text claims. The rows for 'w/o Quota Replay' and 'w/o Deferred Scheduling' still retain the MoE actor, and 'w/o Expert-Aware Critic' likely retains the MoE actor and quota replay. The non-additivity interpretation is asserted without reporting a fuller ablation matrix or interaction analysis. A complete factorial design is not required, but the claim that the components 'form a closed-loop training system rather than independent heuristics' goes beyond the data presented.
minor comments (5)
  1. [Sec. IV-B] The text states that all methods in Fig. 1 (top) are trained for the same number of steps, yet Table I reports very different wall-clock times. Please report the exact step counts and throughput so the reader can reconcile 'same steps' with 15 vs 45 min.
  2. [Sec. IV-F, Table IV] The cross-regime comparison is explicitly labeled as non-comparable, but the table still invites the reader to infer a consistent MoE benefit. Consider moving this material to the discussion or clearly separating it from the paper's own benchmark evidence.
  3. [Sec. III-D, Eq. (9)] The quota fractions q_k are not constrained to sum to 1. Please specify the normalization used so that the per-expert sample counts N_k are well defined and sum to N_exp.
  4. [Sec. V] The conclusion lists 'parameter-matched baselines' as a limitation, but the abstract and Sec. I state the central claim without this caveat. Please align the emphatic claims in the abstract/introduction with the actual evidence.
  5. [Fig. 5d] The t-SNE visualization is interpreted as showing that clusters align with dynamic regimes rather than motion categories. This is a visual claim; a quantitative alignment metric or per-cluster labeling would make it more convincing.

Circularity Check

1 steps flagged

Central speed-quality benchmark is externally measured; only the expert-specialization t-SNE is self-referential.

specific steps
  1. self definitional [Section IV-E, Fig. 5d (t-SNE of primitive trunk features)]
    "To verify that expert decomposition produces meaningful specialization, we visualize the primitive trunk features—for each state, the penultimate-layer (trunk) activation of its dominant expert k∗ = arg maxk et,k — via t-SNE [30] (Fig. 5d). States routed to different dominant experts occupy well-separated regions, confirming that the anti-collapse regularizer drives non-redundant specialization."

    The 'dominant expert' k* is defined in Sec. III-D from the expert contribution vector e_{t,k} = (1/d) Σ_j |w_{k,g(j)} c_k S_{k,j} μ_{k,j}|, i.e., from the same expert networks whose trunk activations are being plotted. Because each expert is a separate neural network, assigning states to the network with the largest action contribution and then projecting that network's own penultimate-layer features will produce well-separated per-expert clusters regardless of whether the experts have learned semantically meaningful control modes. The t-SNE separation is therefore a property of the labeling scheme, not independent evidence of specialization. This is a supporting Q4 claim, not the central tracking-error result, which is measured against external reference motions and baselines.

full rationale

The paper's headline results (Table I: 34.0% body-error reduction, normalized reward 0.723 vs 0.648, ~45 min vs ~360 min) are measured against the external LAFAN1 reference motions and compared with PPO and FastSAC-MLP baselines under a fixed protocol. None of these numbers are constructed from LooperMuscle's own parameters or definitions. The 'closed training loop' (actor contributions -> routing -> critic -> actor) is a positive feedback mechanism, not a definitional identity. The acknowledged absence of a parameter-matched monolithic MLP baseline is a genuine experimental confound that weakens the attribution of the gain to structured MoE, but it is a missing control and alternative explanation, not circularity. The one self-referential step is the t-SNE specialization plot, where the dominant-expert label and the plotted trunk features come from the same expert networks, so the observed separation is partly tautological. That plot is a secondary mechanism check and does not bear the central speed-quality claim, so the overall circularity is low (2/10).

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The method rests on standard RL machinery and on hand-chosen hyperparameters (K, G, tau_g, lambda_lb, eta, quotas, curriculum thresholds, Natom) whose values are not reported in the preprint. The key domain assumptions are the semantic joint grouping and the transferability of the privileged-interface results to hardware.

free parameters (6)
  • K (number of experts) and G (number of joint groups) = K=4, G=2
    Chosen by hand; not systematically varied in the paper (Sec. III-B, Sec. V notes fixed design choices).
  • Per-group gating temperature tau_g = not reported in text
    Controls softmax sharpness; value deferred to released codebase (Sec. III-B).
  • Load-balance coefficient lambda_lb and annealing schedule alpha_lb(t) = not reported in text
    Weights the KL anti-collapse regularizer; annealing schedule not specified in the text (Eq. 3).
  • Routing fraction eta and expert quotas q_k = eta in [0,1], q_k not reported
    Determines replay batch composition; values deferred to codebase (Eq. 9).
  • Curriculum thresholds rho_h, rho_s, rho_e, h_max, u_max = not reported in text
    Define holdout and deferred unlock schedules (Eqs. 10-11); values not in the paper.
  • Natom for C51 distributional critic = not reported
    Number of atoms in the categorical distribution (Eq. 6); deferred to codebase.
axioms (5)
  • domain assumption Splitting the 29 joints into upper-body and lower-body groups captures the semantically relevant control structure.
    A single fixed G=2 grouping is used for all 40 sequences and claimed to transfer across motion categories (Sec. III-B, IV-A).
  • domain assumption A KL load-balance regularizer on batch-averaged gating weights prevents gate collapse without preventing state-dependent specialization.
    The anti-collapse mechanism is an inductive heuristic, not a theorem (Eq. 3).
  • domain assumption The MJLab privileged-observation interface yields method rankings representative of deployable interfaces.
    The paper's benchmark uses ground-truth anchor points; hardware is retrained under a different interface. The transfer assumption is acknowledged in Sec. IV-A and V.
  • domain assumption The SAC base recipe with dual critics, distributional value functions, and observation/layer normalization from FastSAC is a stable off-policy training backbone.
    Adopted from [6] without modification, as stated in Sec. III-A.
  • domain assumption The LAFAN1 40-sequence set is a representative benchmark for whole-body tracking difficulty.
    Six categories are chosen to span difficulty, but no external validation of the benchmark's representativeness is given (Sec. IV-A).

pith-pipeline@v1.3.0-alltime-deepseek · 11770 in / 15816 out tokens · 159679 ms · 2026-08-05T00:11:35.797860+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of LooperMuscle: Fast and Stable Learning of Humanoid Whole-Body Tracking via Structured Mixture-of-Experts." pith.science (2026). https://pith.science/paper/KKJAY5JL

@misc{pith2026260800820,
  author       = {Pith},
  title        = {Pith review of: LooperMuscle: Fast and Stable Learning of Humanoid Whole-Body Tracking via Structured Mixture-of-Experts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KKJAY5JL}},
  note         = {Machine review of arXiv:2608.00820}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

FastSAC-style methods significantly reduce humanoid motion training time but often suffer from notable performance degradation compared with PPO in whole-body tracking tasks. We target this speed-performance gap by introducing LooperMuscle, a composed expert policy learning framework that restores tracking quality while preserving high training efficiency. LooperMuscle combines a semantically structured mixture-of-experts actor, an expert-aware distributional critic, and contribution-routed replay with deferred curriculum scheduling. These three components form a closed training loop in which expert contributions guide data routing, routed data shape value learning, and value gradients in turn refine expert specialization. Empirically, our approach substantially outperforms vanilla FastSAC in motion tracking accuracy while requiring far less wall-clock time than PPO: where FastSAC trains in about 15 minutes but underperforms, and PPO achieves stronger results but requires about 6 hours, LooperMuscle recovers a substantial fraction of the remaining gap to PPO in roughly 45 minutes of simulation training, delivering practical efficiency for rapid policy iteration. The code will be released to benefit the research community at https://loopermuscle.github.io/.

Figures

Figures reproduced from arXiv: 2608.00820 by Boyi Liu, Qijin Li, Qinrui Yan, Tianqi Yu, Xingxing Zuo.

Figure 1
Figure 1. Figure 1: LooperMuscle overview. Top: Training reward curves on a reference motion. LooperMuscle converges in ∼45 min with 47.5% higher reward than FastSAC and ∼8× faster than PPO. Bottom: Real-world deployment on Unitree G1 executing a motion sequence from the KungfuBot2 motion library [9], demonstrating stable and expressive whole-body tracking. We argue that this gap stems from three interrelated struc￾tural limi… view at source ↗
Figure 2
Figure 2. Figure 2: System overview of LooperMuscle. (A) The policy receives proprioceptive observations and motion references. (B) The Muscle-Inspired Expert Actor decomposes the policy into K experts fused via per-joint-group gating wk,g, with per-joint scaling Sk,j and output alignment ck. The Expert-Aware Distributional Critic maintains per-expert DVF heads aggregated through the same gating weights. (C) Routed Replay & C… view at source ↗
Figure 3
Figure 3. Figure 3: MoE Actor architecture. The input observation is processed by a recruitment network that produces per-joint-group gating weights wk,g(st). Each Muscle Unit (expert) outputs actions µk (st), modulated by learnable per-joint scaling Sk,j and output alignment ck before fusion via Eq. (1). A KL-based gate load-balance regularizer prevents gate collapse. B. Multi-Expert Actor with Anti-Collapse Gating A monolit… view at source ↗
Figure 4
Figure 4. Figure 4: Quota-Routed Replay and Expert-Aware Critic over training time. Top: Each transition is stored with expert contribution metadata and routed to the main buffer, per-expert buckets, and deferred buck￾ets. Middle: Mini-batches combine uniform samples, expert-quota samples (Nk = ⌊qkNexp⌋), and deferred samples (Ndef = ⌊u(ρ)Nexp⌋). Bottom: Holdout ratio h(ρ) releases held-back data early, while unlock ratio u(ρ… view at source ↗
Figure 5
Figure 5. Figure 5: Main quantitative results and expert specialization. (a) LooperMuscle occupies a favorable middle point in the speed–quality tradeoff: much better tracking quality than FastSAC-MLP and much less training time than PPO. Ellipses denote ±1 standard deviation over multiple random seeds. (b) Reward vs. wall-clock time (0–197 min) under the same reference task in the short-budget regime. (c) Per-category body/j… view at source ↗
Figure 6
Figure 6. Figure 6: Tracking error radar profiles at 15 min, 30 min, 45 min, 240 min, and 360 min (lower area is better). This time-profiled view shows how method-level error structure evolves from early to late training under the same task setup [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Real-world deployment on Unitree G1. The LooperMuscle policy tracks the whole-body motion of a fighting sequence from the KungfuBot2 motion library [9], with stable balance and smooth transitions between kicks, demonstrating sim-to-real transfer via the Holosoma runtime stack at 50 Hz. TABLE IV: MoE consistently outperforms monolithic MLP across different training regimes. Each regime row uses a different … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

32 extracted references · 16 canonical work pages

  1. [1]

    Learning to walk in minutes using massively parallel deep reinforcement learning,

    N. Rudin, D. Hoeller, P. Reist, and M. Hutter, “Learning to walk in minutes using massively parallel deep reinforcement learning,” in Proceedings of the 5th Conference on Robot Learning, vol. 164. PMLR, 2022, pp. 91–100

    work page 2022

  2. [2]

    Isaac gym: High performance gpu-based physics simulation for robot learning,

    V . Makoviychuk, L. Wawrzyniak, Y . Guo, M. Lu, K. Storey, M. Mack- lin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, and G. State, “Isaac gym: High performance gpu-based physics simulation for robot learning,” 2021

    2021

  3. [3]

    Mujoco playground,

    K. Zakka, B. Tabanpour, Q. Liao, M. Haiderbhai, S. Holt, J. Y . Luo, A. Allshire, E. Frey, K. Sreenath, L. A. Kahrset al., “Mujoco playground,”arXiv preprint arXiv:2502.08844, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    Sim-to-real transfer in deep reinforcement learning for robotics: A survey,

    W. Zhao, J. P. Queralta, and T. Westerlund, “Sim-to-real transfer in deep reinforcement learning for robotics: A survey,” in2020 IEEE Symposium Series on Computational Intelligence (SSCI), 2020

    work page 2020

  5. [5]

    Closing the sim-to-real loop: Adapting simula- tion randomization with real world experience,

    Y . Chebotar, A. Handa, V . Makoviychuk, M. Macklin, J. Issac, N. Ratliff, and D. Fox, “Closing the sim-to-real loop: Adapting simula- tion randomization with real world experience,” in2019 International Conference on Robotics and Automation (ICRA), 2019

    work page 2019

  6. [6]

    Learning sim-to-real humanoid locomotion in 15 minutes,

    Y . Seo, C. Sferrazza, J. Chen, G. Shi, R. Duan, and P. Abbeel, “Learning sim-to-real humanoid locomotion in 15 minutes,”arXiv preprint arXiv:2512.01996, 2025

    arXiv 2025

  7. [7]

    Fasttd3: Simple, fast, and capable reinforcement learning for hu- manoid control,

    Y . Seo, C. Sferrazza, H. Geng, M. Nauman, Z.-H. Yin, and P. Abbeel, “Fasttd3: Simple, fast, and capable reinforcement learning for hu- manoid control,”arXiv preprint arXiv:2505.22642, 2025

    Pith/arXiv arXiv 2025

  8. [8]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  9. [9]

    KUNGFUBOT2: Learning versatile motion skills for humanoid whole-body control,

    J. Han, W. Xie, J. Zheng, J. Shi, W. Zhang, T. Xiao, and C. Bai, “KUNGFUBOT2: Learning versatile motion skills for humanoid whole-body control,”arXiv preprint arXiv:2509.16638, 2025

    arXiv 2025

  10. [10]

    A distributional per- spective on reinforcement learning,

    M. G. Bellemare, W. Dabney, and R. Munos, “A distributional per- spective on reinforcement learning,” inInternational Conference on Machine Learning, 2017

    work page 2017

  11. [11]

    Deepmimic: Example-guided deep reinforcement learning of physics-based charac- ter skills,

    X. B. Peng, P. Abbeel, S. Levine, and M. Van de Panne, “Deepmimic: Example-guided deep reinforcement learning of physics-based charac- ter skills,”ACM Transactions on Graphics (TOG), 2018

    work page 2018

  12. [12]

    Amp: Adversarial motion priors for stylized physics-based character control,

    X. B. Peng, Z. Ma, P. Abbeel, S. Levine, and A. Kanazawa, “Amp: Adversarial motion priors for stylized physics-based character control,” inACM SIGGRAPH, 2021

    work page 2021

  13. [13]

    Perpetual humanoid control for real-time simulated avatars,

    Z. Luo, J. Cao, A. Weng, K. Kitani, and W. Xu, “Perpetual humanoid control for real-time simulated avatars,”arXiv preprint arXiv:2305.06456, 2023

    Pith/arXiv arXiv 2023

  14. [14]

    Universal humanoid motion representations for physics-based control,

    Z. Luo, J. Cao, J. Merel, A. Winkler, J. Huang, K. Kitani, and W. Xu, “Universal humanoid motion representations for physics-based control,” inInternational Conference on Learning Representations (ICLR), 2024

    work page 2024

  15. [15]

    Beyondmimic: From motion tracking to versatile humanoid control via guided diffusion,

    Q. Liao, T. E. Truong, X. Huang, G. Tevet, K. Sreenath, and C. K. Liu, “Beyondmimic: From motion tracking to versatile humanoid control via guided diffusion,”arXiv preprint arXiv:2508.08241, 2025

    Pith/arXiv arXiv 2025

  16. [16]

    Hover: Versatile neural whole-body controller for humanoid robots,

    T. He, W. Xiao, T. Lin, Z. Luo, Z. Xu, Z. Jiang, J. Kautz, C. Liu, G. Shi, X. Wanget al., “Hover: Versatile neural whole-body controller for humanoid robots,” in2025 IEEE International Conference on Robotics and Automation (ICRA), 2025

    work page 2025

  17. [17]

    Asap: Aligning simulation and real-world physics for learning agile humanoid whole-body skills,

    T. He, J. Gao, W. Xiao, Y . Zhang, Z. Wang, J. Wang, Z. Luo, G. He, N. Sobanbab, C. Panet al., “Asap: Aligning simulation and real-world physics for learning agile humanoid whole-body skills,” inRobotics: Science and Systems, 2025

    work page 2025

  18. [18]

    Parallel q- learning: Scaling off-policy reinforcement learning under massively parallel simulation,

    Z. Li, T. Chen, Z.-W. Hong, A. Ajay, and P. Agrawal, “Parallel q- learning: Scaling off-policy reinforcement learning under massively parallel simulation,” inInternational Conference on Machine Learning. PMLR, 2023, pp. 19 440–19 459

    work page 2023

  19. [19]

    Getting sac to work on a massive parallel simulator: An rl journey with off-policy algorithms,

    A. Raffin, “Getting sac to work on a massive parallel simulator: An rl journey with off-policy algorithms,” araffin.github.io, 2025

    work page 2025

  20. [20]

    Speeding up sac with massively parallel simulation,

    A. Shukla, “Speeding up sac with massively parallel simulation,” arthshukla.substack.com, 2025

    work page 2025

  21. [21]

    Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,

    N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, “Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,”arXiv preprint arXiv:1701.06538, 2017

    Pith/arXiv arXiv 2017

  22. [22]

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,

    W. Fedus, B. Zoph, and N. Shazeer, “Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,”Journal of Machine Learning Research, vol. 23, no. 120, pp. 1–39, 2022

    2022

  23. [23]

    Multi-task reinforcement learning with soft modularization,

    R. Yang, H. Xu, Y . Wu, and X. Wang, “Multi-task reinforcement learning with soft modularization,” inAdvances in Neural Information Processing Systems, 2020

    work page 2020

  24. [24]

    Probabilistic mixture- of-experts for efficient deep reinforcement learning,

    J. Ren, Y . Li, Z. Ding, W. Pan, and H. Dong, “Probabilistic mixture- of-experts for efficient deep reinforcement learning,”arXiv preprint arXiv:2104.09122, 2021

    Pith/arXiv arXiv 2021

  25. [25]

    Distributional reinforcement learning with quantile regression,

    W. Dabney, M. Rowland, M. Bellemare, and R. Munos, “Distributional reinforcement learning with quantile regression,” inProceedings of the AAAI Conference on Artificial Intelligence, 2018

    work page 2018

  26. [26]

    Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,” inInternational conference on machine learning. PMLR, 2018, pp. 1861–1870

    2018

  27. [27]

    mjlab: A lightweight framework for gpu-accelerated robot learning,

    K. Zakka, Q. Liao, B. Yi, L. L. Lay, K. Sreenath, and P. Abbeel, “mjlab: A lightweight framework for gpu-accelerated robot learning,” arXiv preprint arXiv:2601.22074, 2026

    arXiv 2026

  28. [28]

    Robust mo- tion in-betweening,

    F. G. Harvey, M. Yurick, D. Nowrouzezahrai, and C. Pal, “Robust mo- tion in-betweening,”ACM Transactions on Graphics (TOG), vol. 39, no. 4, pp. 60:1–60:12, 2020

    work page 2020

  29. [29]

    Holosoma: An open- source framework for humanoid robot learning,

    Amazon Frontier AI & Robotics (FAR), “Holosoma: An open- source framework for humanoid robot learning,” https://github.com/ amazon-far/holosoma, 2025

    work page 2025

  30. [30]

    Visualizing data using t-sne

    L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.” Journal of machine learning research, vol. 9, no. 11, 2008

    2008

  31. [31]

    Exbody2: Advanced expressive humanoid whole-body control,

    M. Ji, X. Peng, F. Liu, J. Li, G. Yang, X. Cheng, and X. Wang, “Exbody2: Advanced expressive humanoid whole-body control,”arXiv preprint arXiv:2412.13196, 2024

    Pith/arXiv arXiv 2024

  32. [32]

    Gmt: General motion tracking for humanoid whole-body control,

    Z. Chen, M. Ji, X. Cheng, X. Peng, X. B. Peng, and X. Wang, “Gmt: General motion tracking for humanoid whole-body control,”arXiv preprint arXiv:2506.14770, 2025

    Pith/arXiv arXiv 2025