{"id":"4d8749d8-a86f-4f4f-a0c5-b8c0fd58bf4c","arxiv_id":"2608.00820","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"LooperMuscle trains a humanoid whole-body tracking policy in about 45 minutes, cutting body error by 34% versus FastSAC and reaching 72% of PPO's reward, versus PPO's 6 hours.","lead":"This paper introduces LooperMuscle, a training framework combining a mixture-of-experts policy, a distributional critic, and a contribution-aware replay buffer to make humanoid whole-body motion tracking faster while keeping quality high. A generalist should read it because it targets the speed-versus-quality gap in robot learning, showing a 45-minute training recipe that closes much of the gap to a 6-hour baseline.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing parameter-matched MLP baseline leaves the central claim that structured MoE (not capacity or extra training time) drives the 34% error reduction unverified.","rationale":"The reader's conditional verdict is appropriate, but I identify a different load-bearing concern than the reader's stated weakest assumption. The reader emphasizes privileged-interface transfer; that is an external-validity concern and is explicitly disclosed by the authors. The more direct threat to the paper's central architectural claim is the missing parameter-matched monolithic baseline. The paper explicitly acknowledges this gap, so it is not a hidden flaw, but it is load-bearing: without it, the 34% improvement over FastSAC-MLP cannot be attributed to the structured MoE design rather than to additional parameters and training budget. The measured benchmark result may still hold, so I do not recommend moving from CONDITIONAL to REJECT; I recommend keeping the verdict CONDITIONAL, with the parameter-matched control as a required condition.","tokens_in":12077,"tokens_out":6417,"duration_ms":78065,"concrete_test":"Run a monolithic FastSAC-MLP (no MoE, with MLP width/depth adjusted to match LooperMuscle's total trainable parameters) under the same MJLab privileged interface, same number of gradient steps, same 45-minute budget, and same 40-sequence benchmark. If its mean body error is within one std of LooperMuscle's 0.101 m (or below), the structured-MoE attribution fails. Also record normalized reward and per-category errors to see where any gap remains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central causal claim is that the structured MoE actor plus expert-aware critic/replay closes the speed-performance gap. This is not established by the controls presented. LooperMuscle trains ~3x longer than FastSAC-MLP (45 vs 15 min) and contains K=4 experts with per-joint scaling and output alignment, so it has substantially more parameters. Sec. IV-B argues that FastSAC-MLP saturates by 15 min and that longer training (up to 197/360 min) leaves the gap unchanged, concluding 'representational rather than compute bottleneck.' But saturating a small monolithic MLP does not test whether a monolithic MLP with LooperMuscle's parameter count could achieve comparable error. Table III's 'w/o MoE Actor' row reduces to FastSAC-MLP, removing not only expert structure but also the expert-dependent critic and replay; so it cannot isolate architecture from capacity. The manuscript itself lists 'parameter-matched baselines' as a missing control in Sec. IV-B and Sec. V. If a parameter-matched MLP reaches ~0.10 m body error, the 34.0% reduction in Table I is explained by capacity/compute, and the proposed contributions (semantic gating, quota routing, expert-aware critic) are not necessary for the headline result. This is a missing control, not a contradiction; the measured comparison may still hold, but the central 'structured' claim is unverified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LooperMuscle, a FastSAC-based off-policy framework for 29-DoF humanoid whole-body tracking. It combines a semantically structured mixture-of-experts actor (K=4 experts, per-joint-group gating), an expert-aware distributional critic, and quota-routed replay with deferred scheduling. On 40 LAFAN1 sequences in MJLab, the authors report 0.101 m mean body error versus 0.153 m for FastSAC-MLP and 0.082 m for PPO, normalized reward 0.723 versus 0.648 and 1.000, and training time about 45 min versus 15 min and 360 min. The paper also reports ablations for each component, a hardware deployment on Unitree G1 via the Holosoma interface, and a contextual comparison with concurrent MoE-based methods.","tokens_in":12432,"tokens_out":5817,"duration_ms":67718,"significance":"If substantiated, the result would be a meaningful step toward closing the speed-performance gap in off-policy humanoid whole-body tracking: it would show that structured policy decomposition and expert-aware value learning can substantially improve tracking quality while retaining a large wall-clock advantage over PPO. The paper is commendably candid about its limitations, including the privileged observation interface, the missing parameter-matched baseline, and the fixed hyperparameter choices. It also promises release of code and reports multiple seeds. However, as it stands, the central structural claim is not fully verified because the main comparison does not isolate architecture from capacity, and the quantitative benchmark is run under an interface that is not deployable. These issues affect the headline conclusions, although they appear fixable within the scope of a revision.","major_comments":[{"comment":"The headline claim of '47.5% higher reward than FastSAC' is not supported by Table I: normalized reward is 0.723 for LooperMuscle and 0.648 for FastSAC-MLP, a relative increase of about 11.6%, not 47.5%. If the 47.5% refers to raw training reward on a single sequence or a different checkpoint, the metric, sequence, and training phase must be specified. As written, the abstract and Table I contradict each other on a central quantitative claim.","section":"Abstract, Fig. 1, Table I"},{"comment":"The claim that the improvement comes from structured MoE rather than capacity/compute is not established. LooperMuscle uses K=4 experts with per-joint scaling and output alignment, and trains about 3x longer than FastSAC-MLP. The 'w/o MoE Actor' row in Table III reduces to FastSAC-MLP, removing not only the MoE architecture but also the expert-aware critic and quota routing; it therefore cannot isolate the actor's contribution. The statement that 'expert decomposition is the primary source of representational gain' is unsupported. The manuscript itself names parameter-matched baselines as a missing control (Sec. IV-B). Without a monolithic MLP with comparable parameter count and training budget, the 34.0% reduction may be an effect of capacity, not structure.","section":"Sec. IV-B, Sec. IV-D, Table III"},{"comment":"All quantitative benchmarking (Table I, Table II, Table III) uses MJLab's privileged ground-truth global key-body positions/orientations, which no onboard sensor provides. The hardware policy is retrained under a different 154-dim Holosoma interface and validated only qualitatively. Therefore the central quantitative claims (0.101 m body error, 0.723 normalized reward, 45 min) are not shown to transfer to a deployable observation interface. The paper states this limitation, but the conclusion still frames LooperMuscle as delivering 'practical efficiency for rapid policy iteration.' Either add a quantitative deployable-interface benchmark or substantially qualify the practical claim.","section":"Sec. IV-A, Sec. IV-G, Sec. V"},{"comment":"The ablation evidence for the 'closed-loop' coupling is weaker than the text claims. The rows for 'w/o Quota Replay' and 'w/o Deferred Scheduling' still retain the MoE actor, and 'w/o Expert-Aware Critic' likely retains the MoE actor and quota replay. The non-additivity interpretation is asserted without reporting a fuller ablation matrix or interaction analysis. A complete factorial design is not required, but the claim that the components 'form a closed-loop training system rather than independent heuristics' goes beyond the data presented.","section":"Sec. IV-D, Table III"}],"minor_comments":[{"comment":"The text states that all methods in Fig. 1 (top) are trained for the same number of steps, yet Table I reports very different wall-clock times. Please report the exact step counts and throughput so the reader can reconcile 'same steps' with 15 vs 45 min.","section":"Sec. IV-B"},{"comment":"The cross-regime comparison is explicitly labeled as non-comparable, but the table still invites the reader to infer a consistent MoE benefit. Consider moving this material to the discussion or clearly separating it from the paper's own benchmark evidence.","section":"Sec. IV-F, Table IV"},{"comment":"The quota fractions q_k are not constrained to sum to 1. Please specify the normalization used so that the per-expert sample counts N_k are well defined and sum to N_exp.","section":"Sec. III-D, Eq. (9)"},{"comment":"The conclusion lists 'parameter-matched baselines' as a limitation, but the abstract and Sec. I state the central claim without this caveat. Please align the emphatic claims in the abstract/introduction with the actual evidence.","section":"Sec. V"},{"comment":"The t-SNE visualization is interpreted as showing that clusters align with dynamic regimes rather than motion categories. This is a visual claim; a quantitative alignment metric or per-cluster labeling would make it more convincing.","section":"Fig. 5d"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a robotics/RL venue and the authors are transparent about limitations, which is a strength. The main obstacle is the missing parameter-matched MLP control: without it, the central 'structured MoE' claim is not distinguishable from a capacity/compute effect. The privileged-interface issue is acceptable if the paper is framed as a controlled algorithmic comparison, but the practical claims must be softened. I would be willing to review a revised version that adds the parameter-matched baseline, reconciles the reward figures, and clarifies the scope of the quantitative claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should read this one if you care about off-policy RL for humanoid whole-body tracking. The authors combine known ingredients—MoE actor, distributional critics, replay shaping—into a single recipe that trains in ~45 minutes and beats FastSAC-MLP by 34% body-position error (0.153→0.101 m) while reaching 72% of PPO's normalized reward in an eighth of the wall-clock time. That's genuinely useful for sim-to-real iteration.\n\nThe paper's honesty is a real strength. The ablations show each component matters: removing quota replay costs 25.7%, deferred scheduling 17.8%, expert-aware critic 11.9%, and the MoE actor 51.5%. The per-category improvement is consistent (26.7–40.6%). They also separate the privileged MJLab benchmark from the retrained Holosoma hardware policy, which is more than most papers do.\n\nNow the soft spots, in proportion. The stress-test note is right but not fatal. The \"w/o MoE Actor\" row collapses to FastSAC-MLP, which removes the entire expert structure along with the critic and replay—so it can't isolate architecture from capacity. The authors themselves list parameter-matched baselines as future work. That means the 34% reduction is a solid engineering result, but the scientific claim that \"structured MoE, not extra parameters/time, drives the gain\" is unproven. A monolithic MLP with ~4x capacity and 45 minutes of training is the missing control.\n\nTwo more issues. The abstract's \"47.5% higher reward\" is for one motion in Fig. 1, not the benchmark table—the representative number is 34%. And the main quantitative numbers come from a privileged observation interface (global key-body positions from the simulator state), which no real sensor gives you. The authors acknowledge this and retrain for hardware, but we don't know whether the ranking LooperMuscle vs. FastSAC-MLP survives the deployable interface.\n\nCode is not yet available, and hyperparameters are promised but not in the paper. Single morphology (G1) limits generality.\n\nOverall: this is a credible, well-communicated empirical package, not a formal proof. It deserves a serious referee, and I'd engage with it. My recommendation: send it out, but require the parameter-matched baseline experiment (or a tight argument for why it's impossible) and a clear statement of what the 47.5% figure refers to.","headline":"A credible, cleanly reported empirical advance on fast humanoid tracking; the central attribution to MoE structure needs a capacity control, but the 34% error reduction over FastSAC-MLP is a real, useful result.","tokens_in":12971,"tokens_out":2915,"would_cite":true,"duration_ms":30188,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LooperMuscle claims a structured mixture-of-experts actor, an expert-aware distributional critic, and contribution-routed replay can train humanoid whole-body tracking policies in about 45 minutes while recovering 72% of the quality of a si","keywords":["whole-body tracking","humanoid control","mixture-of-experts","off-policy reinforcement learning","distributional critic","replay buffer","motion imitation","FastSAC"],"falsifier":"Run the same 40-sequence benchmark and full ablation suite with the ground-truth body-state signals removed from the observation, keeping only joint encoders, base IMU, and body-relative references; if LooperMuscle's body-error advantage over the monolithic baseline shrinks to a small margin or its ranking relative to PPO inverts, the central practical claim fails.","tokens_in":11956,"feed_emoji":"🤖","tokens_out":6787,"duration_ms":66969,"temperature":0.7,"pith_summary":"LooperMuscle is a reinforcement-learning recipe for whole-body humanoid motion tracking that claims to close much of the quality gap between fast off-policy training and slow, high-quality on-policy training. On 40 motion-capture sequences, the paper reports reducing mean body-position error from 0.153 m to 0.101 m relative to the FastSAC baseline, a 34 percent improvement, while reaching 72 percent of PPO's converged normalized reward in roughly 45 minutes of GPU training instead of PPO's six hours. The claimed mechanism is a structured design: a policy decomposed into four expert networks gated by upper-body and lower-body groups, a distributional critic that mirrors the experts' gating for credit assignment, and a replay buffer that routes samples according to per-expert contribution with a progress-dependent curriculum. If the claim holds, humanoid motion iteration cycles could shrink from hours to under an hour without giving up most of the tracking quality that today requires on-policy training.","feed_headline":"LooperMuscle cuts body-tracking error 34% in 45 minutes","feed_subtitle":"Structured expert policy reaches 72% of PPO's reward in 45 minutes, up from FastSAC's 65%.","key_machinery":"The load-bearing object is the closed training loop coupling three mechanisms: (1) a K-expert actor fused by per-joint-group gating weights, with learnable per-joint scaling and output alignment plus a KL-based load-balance regularizer; (2) an expert-aware distributional critic whose per-expert categorical return distributions are aggregated by the actor's own gating weights, making value structure isomorphic to policy structure; (3) contribution-routed replay, where each transition is tagged by each expert's mean absolute action contribution and sampled under per-expert quotas with deferred buckets released by progress-dependent schedules. The loop is what carries the argument: expert contr","core_discovery":"The paper's central claim is that the speed-performance gap in humanoid whole-body tracking has structural causes: a monolithic actor network, a scalar value function, and uniform experience replay reinforce each other. LooperMuscle replaces all three at once: a mixture-of-experts actor with per-joint-group gating and anti-collapse regularization; a distributional critic with per-expert value heads aggregated by the same gating weights; and quota-routed replay that lets expert contribution steer which transitions enter the batch and when. These components form a closed loop: expert contributions route data, routed data shapes value learning, and value gradients refine specialization. The pap","pith_inferences":["The per-expert action-contribution vector stored with each transition could serve as a generic saliency signal beyond replay, for example to adapt the group partition, expert count, or per-group temperatures during training based on observed specialization.","The deferred-unlock schedule is essentially a self-paced curriculum keyed to policy progress rather than task difficulty; the same idea may transfer to other off-policy continuous-control settings where difficult transitions destabilize early learning.","If the privileged-interface ranking survives evaluation under the deployable interface, a practical workflow would be to benchmark in the simulator at 45 minutes per motion and then retrain the final policy under onboard observations, making per-motion fine-tuning economically feasible.","The KL load-balance regularizer acting on batch-level gating statistics may also preserve action diversity, since keeping weaker experts active maintains exploration that a collapsed gate would remove."],"forward_implications":["Off-policy whole-body tracking policies can be trained in about 45 minutes on a single GPU and reach roughly 72 percent of PPO's converged reward, letting developers iterate on motion libraries in minutes rather than hours.","The 34 percent body-error reduction over FastSAC is consistent across all six motion categories, from 26.7 percent for walking to 40.6 percent for fall-and-get-up, so the gain concentrates where tracking is hardest.","The ablations attribute the largest share of the gain to the MoE actor itself; removing it returns the method to the monolithic baseline, and removing quota replay, deferred scheduling, or the expert-aware critic each degrades performance by 12 to 26 percent.","The paper's cross-paradigm comparison suggests expert decomposition helps under on-policy PPO methods as well as under off-policy FastSAC, pointing to a general design principle for whole-body tracking.","Deployment on a physical humanoid with an onboard-only observation interface, while qualitative, indicates the recipe produces policies that can be retrained for real hardware without the simulator's privileged signals."],"supporting_citations":[{"why":"Supplies the FastSAC off-policy training recipe and the 15-minute locomotion baseline that LooperMuscle extends.","marker":"[6]"},{"why":"Provides PPO, the on-policy quality reference whose six-hour training defines the speed-performance gap.","marker":"[8]"},{"why":"Supplies the categorical distributional value function (C51) used for the expert-aware critic heads.","marker":"[10]"},{"why":"Supplies SAC, the base actor-critic algorithm and dual-critic stability that the framework preserves.","marker":"[26]"},{"why":"Supplies the simulation benchmarking environment in which the quantitative comparisons and ablations are run.","marker":"[27]"},{"why":"Supplies the 40 motion-capture tracking sequences used as the whole-body tracking benchmark.","marker":"[28]"},{"why":"Supplies concurrent MoE evidence for the cross-paradigm comparison and the motion library used in hardware deployment.","marker":"[9]"},{"why":"Supplies the deployable onboard-only observation interface under which the hardware policy is retrained.","marker":"[29]"}],"fun_headline_variants":["Humanoid tracking: 45-min training nears 6-hour PPO quality","Speed meets stability: LooperMuscle for humanoid tracking","Fast humanoid tracking: LooperMuscle closes PPO gap in 45 min","MoE-driven humanoid tracking: PPO-level quality in 45 min","45-minute humanoid tracking rivals PPO's 6-hour quality"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The speed-quality numbers come from a simulator that hands the policy ground-truth body positions and orientations that no real sensor can measure; the load-bearing assumption is that the ranking of LooperMuscle against FastSAC and PPO stays the same when only onboard-estimable signals are used, an interface on which the real robot is currently validated only qualitatively.","fun_headline_variants_meta":{"raw":{"variants":["Humanoid tracking: 45-min training nears 6-hour PPO quality","Speed meets stability: LooperMuscle for humanoid tracking","Fast humanoid tracking: LooperMuscle closes PPO gap in 45 min","MoE-driven humanoid tracking: PPO-level quality in 45 min","45-minute humanoid tracking rivals PPO's 6-hour quality"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001019,"raw_usage":{"total_tokens":4134,"prompt_tokens":737,"completion_tokens":3397,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":481,"completion_tokens_details":{"reasoning_tokens":3298}},"tokens_in":481,"tokens_out":3397,"duration_ms":25361,"temperature":1.0,"reasoning_tokens":3298,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T00:11:35.797860+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same 40-sequence benchmark and full ablation suite with the ground-truth body-state signals removed from the observation, keeping only joint encoders, base IMU, and body-relative references; if LooperMuscle's body-error advantage over the monolithic baseline shrinks to a small margin or its ranking relative to PPO inverts, the central practical claim fails.","supporting_citations":[{"cited_title":"A distributional per- spective on reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the categorical distributional value function (C51) used for the expert-aware critic heads."},{"cited_title":"Robust mo- tion in-betweening,","cited_arxiv_id":null,"evidence_quote":"Supplies the 40 motion-capture tracking sequences used as the whole-body tracking benchmark."},{"cited_title":"Holosoma: An open- source framework for humanoid robot learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the deployable onboard-only observation interface under which the hardware policy is retrained."}],"review_version":1}