{"id":"909fe53b-3104-46db-a5e4-8e2bf4e479c5","arxiv_id":"2502.01143","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"ASAP trains a residual action model on real-world rollouts and fine-tunes simulation policies through it, reducing humanoid whole-body motion tracking error in sim-to-real transfer.","lead":"This paper presents ASAP, a two-stage pipeline that first trains humanoid motion-tracking policies in simulation and then uses real-world data to learn a residual 'delta action' model that corrects the simulator's physics. On a Unitree G1 humanoid, the approach reduces tracking errors and enables dynamic motions like jumps, kicks, and a single-leg balance, suggesting a practical way to close the sim-to-real gap for agile robots.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Real-world claim rests on an untested 4-DoF ankle-only delta action model with no variance reporting; the paper's own Fig. 13 shows substantial knee/hip corrections in the sim-to-sim analog.","rationale":"The reader's weakest_assumption—that 4-DoF ankle-only corrections may miss substantial sim-to-real mismatch—is the same point I identify as most load-bearing. For ASAP's central claim to hold, the delta action model used in real-world fine-tuning must align the simulator well enough that the adapted policy improves on hardware; if the correction is confined to the ankles while knees and hips also contribute (as Figure 13 indicates for the sim-to-sim case), the fine-tuned policy is being optimized against a simulator that is only partially corrected. The real-world table contains no per-trial statistics, so the modest point-estimate gains (7.8–29.6%) cannot be distinguished from noise, and the 'up to 52.7%' sentence in the conclusion is not supported by Table V. I would not reject the paper: the sim-to-sim results, the OOD LeBron test, and the released code are genuine supporting evidence, and the framework is not circular—the delta action is fit to state-transition discrepancies and then used to adapt a policy. The appropriate disposition is the same conditional acceptance the reader reached: the method is plausible, but the real-world leg of the central claim needs stronger data (per-trial variance, more tasks, or a 4-DoF vs 23-DoF ablation) before it can be taken as established. A single feasible check—the 4-DoF sim-to-sim ablation—can directly test whether the restriction is the weak point.","tokens_in":22441,"tokens_out":10129,"duration_ms":107934,"concrete_test":"Run the existing IsaacGym→IsaacSim and IsaacGym→Genesis benchmarks with a 4-DoF ankle-only delta action model (matching the real-world restriction) and compare closed-loop Eg-mpjpe/Empjpe against the 23-DoF results in Table IV. If the 4-DoF ablation loses most of ASAP's improvement over Vanilla (e.g., relative reduction in Eg-mpjpe drops by more than half), then the real-world restriction is missing a substantial part of the dynamics gap and the central whole-body claim is not supported. If the 4-DoF ablation reproduces most of the Table IV gains, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that ASAP closes the sim-to-real gap for agile whole-body skills—requires the delta action model to capture a large enough share of the real dynamics mismatch that fine-tuning through it lowers real-world tracking error. In the real-world experiments (Section IV-C), the delta action model is restricted to 4 ankle DoF, justified by the G1's ankle linkage and by data scarcity, but no real-world measurement is provided showing that ankle corrections dominate the gap. The paper's own Figure 13, for the IsaacGym→IsaacSim case, shows the largest corrections at the ankle pitch but also pronounced values at knee joints, with non-ankle discrepancies throughout the lower body; Table V then reports only point estimates, with reductions of 7.8–29.6% across metrics and no variance or significance testing. If the 4-DoF restriction omits a substantial part of the mismatch, the fine-tuned policy is being adapted to a simulator that is only partially aligned, and the modest real-world gains may be an artifact of ankle-specific tuning or noise rather than the whole-body alignment claimed in the abstract. The conclusion's 'up to 52.7% in sim-to-real tasks' is not derivable from Table V (max ≈29.6%), suggesting the headline real-world result is overstated and the evidence for the central claim is weaker than presented.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"ASAP proposes a two-stage framework for closing the sim-to-real dynamics gap in agile humanoid whole-body motion tracking. Stage one pre-trains phase-conditioned motion-tracking policies in IsaacGym using retargeted human motion; stage two collects real-world rollouts under the pre-trained policy, trains a delta (residual) action model with RL so that the simulator reproduces real-world next states, freezes this model into the simulator as s_{t+1}=f_sim(s_t, a_t+pi_delta(s_t,a_t)), and fine-tunes the policy; deployment then uses the fine-tuned policy without the delta model. The paper evaluates on IsaacGym-to-IsaacSim, IsaacGym-to-Genesis, and IsaacGym-to-Unitree G1, reporting lower tracking errors than Vanilla, SysID, and DeltaDynamics baselines, with open-loop replay metrics, closed-loop policy metrics, and a small real-world table.","tokens_in":22639,"tokens_out":6119,"duration_ms":58591,"significance":"If the central claim holds, residual action alignment is an attractive alternative to SysID and DR: it avoids hand-picked parameter spaces, preserves agility, and is trainable from policy rollouts. The paper's strengths include an open-source multi-simulator codebase, a systematic comparison against SysID and learned-dynamics baselines, and ablations (dataset size, horizon, action norm weight) that show the delta action model improves out-of-distribution generalization. The real-world G1 demonstrations of kicks, jumps, and balance are impressive and constitute a demanding testbed. However, the evidence for the central real-world claim currently rests on point estimates from a reduced 4-DoF ankle model, and some empirical statements in the paper are contradicted by its own tables.","major_comments":[{"comment":"The sentence \"Quantitative results in Table III demonstrate that ASAP consistently outperforms the OpenLoop baseline across all replayed motion lengths\" is not supported by Table III: at the 0.25 s replay length in IsaacSim, ASAP has Eg-mpjpe=19.9 mm and Empjpe=15.6 mm versus OpenLoop's 19.5 mm and 15.1 mm, so it is slightly worse on both metrics; at 0.5 s in IsaacSim, DeltaDynamics has lower Empjpe (16.4) than ASAP (19.2). Please either restrict the claim to the longer horizons and specify the primary metric, or add significance or paired comparisons.","section":"Section IV-A, Table III"},{"comment":"The claim that ASAP \"consistently outperforms baselines across all difficulty levels\" and achieves the \"lowest Eg-mpjpe and Empjpe\" is contradicted by Table IV on root-relative MPJPE: in IsaacSim Hard, Vanilla Empjpe=51.6 versus ASAP=56.5; in Genesis Easy, Vanilla Empjpe=70.1 versus ASAP=73.5; and in IsaacSim Medium, Vanilla Empjpe=49.2 versus ASAP=49.3. Even on Eg-mpjpe, several Easy and Medium differences are only 1-7 mm, which is within or near one standard error. Please clarify whether Eg-mpjpe is the sole primary outcome and provide error bars or statistical tests for the closed-loop comparisons.","section":"Section IV-B, Table IV"},{"comment":"The real-world evaluation is the direct evidence for Q3, yet Table V reports single point estimates per metric with no variance, confidence intervals, or significance tests, even though Section IV-C states that the tracking policy was executed 30 times per task. With n=30, paired mean differences and standard errors should be reported; without them, reductions of 7.8-29.6% cannot be distinguished from run-to-run variability.","section":"Section IV-C, Table V"},{"comment":"The decision to train a 4-DoF ankle-only delta action model rests on the claim that the G1 ankle linkage dominates the sim-to-real gap, but no real-world measurement is provided to support that dominance. Figure 13, for the IsaacGym-to-IsaacSim case, shows pronounced corrections at the knee as well as the ankle, and the paper's own limitation section notes that the full 23-DoF model is data-hungry. If the ankle-only model misses a substantial share of the mismatch, the fine-tuned policy is adapted to only a partially aligned simulator. Please add per-joint delta action statistics from real-world data or otherwise validate the 4-DoF restriction before using it as the basis for the real-world claim.","section":"Section IV-C, Figure 13"},{"comment":"The conclusion states reductions \"up to 52.7% in sim-to-real tasks\", but the maximum reduction in Table V is about 29.6% (LeBron Eg-mpjpe: 159 to 112). The 52.7% figure appears to correspond to a sim-to-sim open-loop comparison in Table III (for example, IsaacSim 1.0 s: 80.8 to 37.9, which is a 53.1% reduction). Please correct the conclusion or explicitly attribute this number to the appropriate experiment.","section":"Conclusion"}],"minor_comments":[{"comment":"The sentence \"the minimum required to train the full 23-DoF delta action model in simulation, as discussed in Section III-B\" appears in Section IV-C, but Section III-B does not discuss data requirements; this belongs to the dataset-size analysis in Section V-A or the limitations section. Please fix the cross-reference.","section":"Section IV-C"},{"comment":"Please add a color scale and numeric labels to Figure 13 so that the claimed ankle and knee dominance can be assessed quantitatively.","section":"Figure 13"},{"comment":"The abstract and Section IV-B use \"significantly\" without reporting significance tests; please either add statistical tests or use more neutral phrasing such as \"consistently reduces\" where appropriate.","section":"Abstract and Section IV-B"},{"comment":"Equation (1) in Section VIII-D states f_real(s,pi(s)) = f_sim(s,pi(s)+pi_delta(s,pi(s))), but this is an assumed one-step consistency condition, not a derived identity; labeling it explicitly as an assumption would clarify the derivation.","section":"Section VIII-D"}],"recommendation":"major_revision","confidential_remarks":"This is a solid systems paper with a clear central idea and a useful code release. The main revision need is statistical reporting and reconciling the text claims with the tables. I would weight the hardware effort positively, but the manuscript needs one more round of evidence before the real-world claims can be accepted at face value."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Net take: a solid, well-engineered application of residual action learning to whole-body humanoid sim-to-real, with real code and a real robot demo, but the real-world evidence is thinner than the abstract suggests and the headline '52.7%' is not supported by the real-world table.\n\nWhat's actually new: the two-stage pipeline — pretrain a motion-tracking policy in IsaacGym, roll it out on hardware, train a delta action model on the state-transition discrepancy, freeze that model into the simulator as st+1 = f_sim(st, at + π∆(st, at)), then fine-tune the policy inside the corrected simulator. Residual action learning itself is old (RGAT, residual RL), but applying it to agile whole-body humanoid tracking, and validating across two alternate simulators plus a Unitree G1, is a legitimate and useful contribution. The open-loop replay experiments (Table III) are the most convincing part: the delta action model reduces long-horizon replay error substantially, and it generalizes to an out-of-distribution motion (LeBron's 'Silencer').\n\nThe paper earns credit for shipping an open-source multi-simulator codebase and for an honest limitations section: two G1s broke during data collection, MoCap is required, and the full 23-DoF delta model is impractical with the data they can collect. These are real constraints, not afterthoughts.\n\nSoft spots, in proportion. The real-world evaluation (Table V) is the load-bearing claim for 'sim-to-real,' and it is thin: two tasks, no variance or significance despite 30 runs per task, and the 'up to 52.7%' in the conclusion does not come from that table — max reduction there is about 30%. It looks like it comes from the open-loop sim-to-sim replay, which is a different claim. The 4-DoF ankle-only delta action model is a pragmatic choice, but the paper does not show that ankle corrections dominate the real gap; Figure 13 (in sim-to-sim) shows substantial knee and hip discrepancies too. The SysID baseline is a coarse discrete grid search, so beating it is not a high bar. In the sim-to-sim closed-loop tables, gains on easy/medium motions are often within a few millimeters or within one standard deviation; the method clearly shines on hard motions, which is itself worth reporting.\n\nNone of this is fatal. The framework is not circular — the delta model is fit to state-transition discrepancies, and the OOD deployment is a genuine test. The central idea holds up; the evidence is just uneven.\n\nWho will get value: anyone working on sim-to-real transfer for legged or humanoid robots, and people who want a working multi-simulator training setup. It deserves a serious referee. My recommendation: send it to review, and in the report push hard on variance reporting, a justification for ankle-only, and a correction of the 52.7% claim.","headline":"A solid, well-engineered application of residual action learning to whole-body humanoid sim-to-real with real code and a real robot demo, but the real-world evidence is thinner than the abstract suggests and the headline '52.7%' is not supported by the real-world table.","tokens_in":23307,"tokens_out":3510,"would_cite":true,"duration_ms":33310,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A residual action model learned from real-world rollouts and frozen into the simulator reduces motion-tracking error for agile humanoid skills.","keywords":["sim-to-real transfer","humanoid whole-body control","residual action model","reinforcement learning","motion tracking","agile skills","dynamics mismatch"],"falsifier":"Apply the same training protocol but freeze a full 23-joint delta action model trained with comparable real data (or estimate per-joint errors from the 100 clips) and compare real-world global tracking error to the 4-DoF version; if the full model's error is materially lower, the ankle-only restriction is the wrong load-bearing choice.","tokens_in":22172,"feed_emoji":"🤖","tokens_out":10620,"duration_ms":98612,"temperature":0.7,"pith_summary":"The paper tries to establish that the gap between simulated and real physics for agile humanoid whole-body skills can be closed by learning a small residual action correction from real-world rollouts and freezing it into the simulator before fine-tuning the control policy. If true, this gives a practical recipe that avoids laborious system identification and avoids the conservative behavior that domain randomization can induce. The recipe is: pre-train a motion-tracking policy in simulation from human-video-derived reference motions, collect roughly one hundred real motion clips, train a delta action model that compensates for the state discrepancy when those clips are replayed in simulation, fine-tune the policy in the corrected simulator, then deploy the fine-tuned policy alone. The authors demonstrate the recipe across two sim-to-sim transfers and on a real 23-degree-of-freedom humanoid, with lower tracking error than baselines and with a 4-degree-of-freedom ankle-only correction sufficient for the real motions tested.","feed_headline":"Residual actions close the sim-to-real gap for humanoids","feed_subtitle":"Pre-train in simulation, collect ~100 real clips, fine-tune with a frozen delta-action model; tracking error drops on jumps and kicks.","key_machinery":"The load-bearing object is the RL-trained delta action model $\\pi_\\Delta(s_t,a_t)$, a residual policy that maps the current state and commanded action to a corrective action. It is trained by initializing the simulator at a recorded real-world state $s^r_t$ and rewarding agreement between the simulated next state and the recorded real next state $s^r_{t+1}$, with an action-norm penalty to keep corrections small. Once trained, the model is frozen and wrapped into the simulator transition $s_{t+1}=f_{\\mathrm{sim}}(s_t,a_t+\\pi_\\Delta(s_t,a_t))$, so that policy fine-tuning experiences the shifted dynamics; at deployment the delta model is removed and only the fine-tuned policy runs.","core_discovery":"On the paper's own terms, the discovery is that the sim-to-real dynamics gap for agile whole-body humanoid control can be treated as a correction to actions rather than to parameters or states. The authors train a delta action model $\\pi_\\Delta$ from real-world rollouts, freeze it into the simulator as $s_{t+1}=f_{\\mathrm{sim}}(s_t,a_t+\\pi_\\Delta(s_t,a_t))$, and fine-tune the pre-trained motion-tracking policy inside that corrected simulator. They report that this reduces tracking error against system identification, domain randomization, and learned-residual-dynamics baselines in two sim-to-sim transfers and on a real 23-degree-of-freedom humanoid, where a 4-degree-of-freedom ankle-only correction suffices for the motions tested. The measured gains include a drop in global body-position error from about 159 mm to 112 mm on an out-of-distribution real-world motion, and the method achieves 100 percent success on hard sim-to-sim tasks where baselines fail. If true, the useful story is that a small, structured residual in action space can absorb the physics mismatch that system identification and domain randomization cannot.","pith_inferences":["A natural extension the paper does not pursue: the same delta-action idea could run online, updating $\\pi_\\Delta$ on each new hardware unit or as motors wear, turning a one-time calibration into continuous self-alignment.","The dataset-size saturation in the paper (4300 versus 43000 samples changes closed-loop error by only about 0.65 percent) suggests that active collection targeting high-error joints and motion phases could make real-world data use far more efficient.","The paper's own per-joint magnitude plot shows knee and hip discrepancies inside the full 23-DoF model, so a low-rank correction over ankle, knee, and hip trained from the same 100 clips is a direct test of whether the 4-DoF restriction costs agility on hard jumps.","Because the delta model only changes the transition function inside the simulator, it should compose with stronger pre-training (for example, better domain randomization or richer reference data) rather than competing with it."],"forward_implications":["A policy fine-tuned in a simulator that contains the frozen delta action model tracks reference motions with lower global body-position and joint-angle errors in the target environment, including a real-world out-of-distribution motion whose global error falls from about 159 mm to 112 mm.","The method keeps 100 percent task success on hard sim-to-sim transfers where system identification and delta-dynamics baselines drop to 60 to 83 percent, so the correction mechanism, not just extra training, is what preserves agility.","RL fine-tuning with the delta model beats training-free fixes such as fixed-point iteration and gradient search, showing the compensation must be optimized closed-loop rather than solved one step at a time.","The learned corrections are per-joint and asymmetric (ankle and knee deltas are largest, upper-body deltas small), so uniform random action noise cannot reproduce the effect.","Sim-to-sim transfer between different physics engines can serve as a fast proxy for real-world delta-action alignment before any hardware rollout is collected."],"supporting_citations":[{"why":"Supplies the phase-variable motion goal and reference-state initialization that make agile single-motion tracking trainable.","marker":"[67]"},{"why":"Reconstructs 3D human motion from video, the raw data source for imitation.","marker":"[93]"},{"why":"A physics-based motion tracker used to filter reconstructed human motions so that only physically feasible ones are retargeted.","marker":"[86]"},{"why":"Supplies the human-to-robot retargeting procedure that turns cleaned human motions into robot reference trajectories.","marker":"[25]"},{"why":"The physics simulator in which the tracking policy, delta action model, and fine-tuning are all trained.","marker":"[58]"},{"why":"A target physics simulator used to establish the sim-to-sim transfer benchmark and the generalization claim.","marker":"[6]"},{"why":"Documents the passive-ankle design that motivates the 4-DoF ankle-only delta action model in real-world deployment.","marker":"[37]"},{"why":"The reinforcement learning algorithm used at every training stage of the pipeline.","marker":"[80]"},{"why":"The random torque-noise fine-tuning baseline; beating it shows structured delta actions, not added noise, drive the improvement.","marker":"[7]"}],"fun_headline_variants":["Residual actions bridge sim-to-real for agile humanoids","Delta-action model boosts humanoid agility in real world","Fine-tune in sim with real data: humanoid skills leap","Sim-to-real: small action fix solves dynamics gap","Ankle-only correction unlocks agile humanoid moves"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim rests on the assumption that correcting the robot's four ankle degrees of freedom is enough to capture the dominant mismatch between simulation and real physics; if knee and hip discrepancies are substantial, the real-world benefit of the method shrinks.","fun_headline_variants_meta":{"raw":{"variants":["Residual actions bridge sim-to-real for agile humanoids","Delta-action model boosts humanoid agility in real world","Fine-tune in sim with real data: humanoid skills leap","Sim-to-real: small action fix solves dynamics gap","Ankle-only correction unlocks agile humanoid moves"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000199,"raw_usage":{"total_tokens":1440,"prompt_tokens":1081,"completion_tokens":359,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":697,"completion_tokens_details":{"reasoning_tokens":280}},"tokens_in":697,"tokens_out":359,"duration_ms":4283,"temperature":1.0,"reasoning_tokens":280,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T16:24:19.091760+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Apply the same training protocol but freeze a full 23-joint delta action model trained with comparable real data (or estimate per-joint errors from the 100 clips) and compare real-world global tracking error to the 4-DoF version; if the full model's error is materially lower, the ankle-only restriction is the wrong load-bearing choice.","supporting_citations":[{"cited_title":"Deepmimic: Example-guided deep reinforcement learning of physics-based character skills","cited_arxiv_id":null,"evidence_quote":"Supplies the phase-variable motion goal and reference-state initialization that make agile single-motion tracking trainable."},{"cited_title":"Tram: Global trajectory and motion of 3d humans from in-the-wild videos","cited_arxiv_id":null,"evidence_quote":"Reconstructs 3D human motion from video, the raw data source for imitation."},{"cited_title":"PIP-Loco: A Proprioceptive Infinite Horizon Planning Framework for Quadrupedal Robot Locomotion","cited_arxiv_id":"2409.09441","evidence_quote":"A physics-based motion tracker used to filter reconstructed human motions so that only physically feasible ones are retargeted."},{"cited_title":"Isaac gym: High performance gpu based physics simulation for robot learning","cited_arxiv_id":null,"evidence_quote":"The physics simulator in which the tracking policy, delta action model, and fine-tuning are all trained."},{"cited_title":"Dy- namic locomotion for passive-ankle biped robots and humanoids using whole-body locomotion control","cited_arxiv_id":null,"evidence_quote":"Documents the passive-ankle design that motivates the 4-DoF ankle-only delta action model in real-world deployment."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The reinforcement learning algorithm used at every training stage of the pipeline."}],"review_version":1}