Pith. sign in

REVIEW 3 major objections 5 minor 53 references

ComplexMimic: Human-Scene Interaction Imitation in Complex 3D Environments

T0 review · 3 major / 5 minor · reviewed 2026-07-12 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Simulated humanoids can imitate complex human-scene interaction by training separate motion-fidelity and collision experts, then distilling them with difficulty-aware sampling.

desk verdict Solid systems paper: dual experts plus difficulty-aware distillation give real gains on dense indoor HSI imitation; the τ/0.5 m coupling is real but overstated as circularity. read the letter →

arxiv 2607.02034 v2 pith:O2F6OJXF submitted 2026-07-02 cs.CV

Lu Pan , Hongwei Zhao This is my paper
classification cs.CV
keywords HumanoidImitationLearningHuman-SceneInteractionDifficulty-AwareDistillationPhysics-basedCharacterControlMulti-teacherMotionTrackingComplex3DEnvironments
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that physics-based imitation of human-scene interaction in dense, cluttered 3D environments is limited by a real trade-off: strict tracking of motion-capture references keeps poses natural but causes frequent collisions and early failures, while looser tracking raises success at the cost of fidelity. ComplexMimic resolves that conflict in two stages. First it trains an imitation expert with a tight early-termination threshold in a scene-free simulator and an interaction expert with a relaxed threshold plus a local height map of the full scene mesh. Then it freezes both teachers and distills them into one student, routing each motion to the expert that handles it best and reweighting training toward hard-yet-learnable clips using online failure rates and return-improvement signals. On three benchmarks—including real scanned rooms—the unified policy raises success while lowering joint error relative to prior physics-based imitators. A reader cares because this is a concrete route from imperfect MoCap to whole-body control that can actually move through furniture-filled spaces.

What carries the argument

Dual Flow Strategy plus Difficulty-Aware Distillation (DAD). Dual Flow produces two frozen PPO experts (imitation vs interaction). DAD then assigns each motion to the better expert, distills Gaussian action distributions with a joint PPO+KL objective, and samples via inter-group difficulty softmax, intra-group hard prioritization, and a learnability filter that downweights clips whose episode return has stopped improving.

What would settle it

Hold out a test set of clips whose reference motions severely penetrate scene geometry; if success rate and global MPJPE on that set no longer beat single-expert or uniform-distillation baselines, the claim that dual-flow difficulty-aware distillation resolves the trade-off does not hold for the hardest realistic cases.

Watch

Extended reading notes

Core claim

In complex scene meshes, the early-termination threshold on tracking error creates a feasibility–faithfulness trade-off: larger thresholds improve success but inflate global joint error, smaller ones do the reverse. The authors show this can be broken by Dual Flow training of complementary experts—one strict and scene-free, one relaxed and scene-aware—followed by Difficulty-Aware Distillation that routes motions by regime, mixes KL distillation with on-policy PPO, and samples by motion-wise failure statistics plus learnability. The resulting student outperforms strong baselines on TRUMANS, LINGO, and GIMO.

Load-bearing premise

The method assumes that imperfect MoCap clips, fixed tracking-error cutoffs, and a simple half-metre average joint-distance failure rule yield online difficulty and return-improvement scores that correctly flag which trajectories are hard yet still learnable; clips that heavily penetrate the mesh still fail.

Editorial extensions

If this is right

  • Physics-based humanoid controllers can be trained for dense furniture-filled rooms rather than empty or single-object scenes.
  • Success rate and global joint error can improve together instead of trading off under complex meshes.
  • Zero-shot transfer to unseen scene layouts and real scanned rooms with reconstruction noise becomes practical.
  • Multi-teacher imitation benefits from online failure and progress signals rather than uniform sampling.
  • Regime-specialized teacher routing avoids the gradient interference of action averaging or random routing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same dual-expert plus difficulty-aware recipe may transfer to human-object manipulation, where contact feasibility also conflicts with kinematic fidelity.
  • Filtering or repairing severe mesh penetrations in the MoCap references before training could shrink the residual failure mode without changing the architecture.
  • Motion-wise failure EMAs could serve as a general automatic curriculum for multi-skill humanoid policies beyond HSI.
  • Cross-simulator and scaled-body results in the supplement suggest the distilled policy is less brittle to dynamics and morphology shifts than single-expert trackers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. ComplexMimic addresses physics-based human–scene interaction (HSI) imitation in dense 3D environments, where prior work largely uses simplified scenes. The authors identify a feasibility–faithfulness trade-off controlled by the early-termination threshold τ (Fig. 1b). They propose a two-stage method: (i) Dual Flow Strategy training an imitation expert (scene-free, τ≤0.25) and an interaction expert (height-map scene input, τ≥0.5); (ii) Difficulty-Aware Distillation that routes motions into G_Imitate / G_Inter by imitation-expert success, then samples via online failure EMA and return-improvement EMA (Eqs. 2–9). Student training uses joint PPO+KD (Eq. 4). Evaluation on TRUMANS (in-domain), LINGO (OOD), and GIMO (noisy real scans) reports gains over DeepMimic, AMP, PHC, and MaskedMimic (Tables 1–3), with ablations of experts, DAS components, PPO, and routing (Table 4 and Supp.).

Significance. If the gains hold under independent metrics and broader baselines, this is a useful systems contribution for physics-based HSI: it is the first systematic treatment of full-body imitation under complex meshes rather than single-object scenes, and the dual-expert + difficulty-aware multi-teacher design is a clear, reusable recipe. Strengths include three-dataset evaluation (in-domain / OOD / real scans), consistent ablations attributing gains to both experts and DAS pieces, baselines re-run with the same height-map input, and supplementary cross-simulator (Isaac Lab) and cross-embodiment (scaled SMPL) checks. Code is promised. The work is incremental relative to PHC/MaskedMimic-style imitation, but the complex-scene setting and the explicit feasibility–fidelity framing are timely for embodied humanoid control.

major comments (3)
  1. Sec. 3.2, Metrics, and Implementation Details couple the headline Succ metric to the DAS mining signal: failure is defined as mean joint distance >0.5 m, which is exactly the Succ criterion (δ=0.5 m), and student training keeps τ=0.5. DAS then prioritizes via EMA of that same binary failure indicator plus return improvement. Because Succ is the complement of this criterion, reported lifts (e.g., MaskedMimic 0.872→0.906 Succ on TRUMANS, Table 1) and the attribution of gains to DAS (Table 4: w/o learnability filtering drops Succ to 0.877) may be partly metric-aligned rather than a pure resolution of the trade-off. Please report Succ / MPJPE under at least one alternate threshold (e.g., 0.3 m and 0.7 m), and/or a contact/penetration or energy metric independent of the 0.5 m rule, and show that DAS still helps under those criteria.
  2. Fig. 1b motivates the dual-flow design via the τ trade-off, but the main tables do not include a single-policy control that varies only τ (or only scene input) under the same student architecture and training budget. Table 4 removes experts after dual-flow training; it does not show that dual-flow + DAD beats a well-tuned single expert with τ∈{0.25,0.5,adaptive} and height maps. Without that control, it remains possible that much of the gain is from relaxed termination and scene observation rather than complementary teachers and difficulty-aware routing. A single-policy τ sweep (or curriculum on τ) under identical student capacity would make the central claim load-bearing.
  3. Related Work and baselines omit recent physics-based HSI controllers that already target scene interaction (e.g., TokenHSI, UniHSI, and related closed-loop HSI systems discussed in Sec. 2.2). Tables 1–3 only compare pure motion-imitation methods (DeepMimic, AMP, PHC, MaskedMimic) augmented with height maps. For a claim of SOTA on complex HSI imitation, either include at least one such HSI controller (adapted to the imitation setting) or clearly restrict the claim to “motion-imitation baselines under complex meshes” and justify why HSI controllers are out of scope. As written, the SOTA framing overreaches the comparison set.
minor comments (5)
  1. Eq. (3) and the surrounding text: the student always conditions on hmt in the KL terms, while the imitation teacher does not; a short note on how the student uses height maps when routed to the imitation expert would avoid ambiguity.
  2. Implementation Details list many free parameters (τ, EMA rates, τ_inter/τ_intra, ε, λ_u, λ_PPO/λ_KD, reward coefficients in Supp. Eq. 12). Supp. Tabs. 7–8 ablate temperatures and ε_u; a brief sensitivity note for λ_PPO/λ_KD and reward weights in the main text would help reproducibility.
  3. Fig. 4 qualitative examples are useful but sparse; adding failure cases from Supp. Fig. 8 (severe penetration) into the main qualitative figure would better bound the method’s scope.
  4. Typographical and formatting issues: missing spaces in several compound phrases (e.g., “Physics-basedHuman-Scene”, “Difficulty-AwareDistillation”), and the arXiv id / version line should be cleaned for camera-ready.
  5. Sec. 4.1: clarify how many TRUMANS clips fall into G_Imitate vs G_Inter and whether group sizes are stable across training, so readers can judge inter-group sampling balance.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: dual-flow experts and DAS are empirical training choices evaluated on held-out metrics, not predictions forced by their own inputs.

full rationale

ComplexMimic is a methods paper whose central claims are empirical (Succ / Eg-mpjpe / Empjpe / Eacc / Evel on TRUMANS, LINGO, GIMO vs DeepMimic, AMP, PHC, MaskedMimic; ablations in Table 4). The Dual Flow Strategy trains two PPO experts under different τ and scene observations, then distills them with a joint PPO+KL objective; none of these steps define the reported metrics by construction. Difficulty-Aware Sampling reweights training clips via online EMA of a binary failure indicator and return-improvement EMA (Sec. 3.2, Eqs. 5–9). That failure indicator uses the same 0.5 m mean-joint-distance threshold that defines Succ (following PHC), but this is a shared community evaluation convention, not a self-definitional loop: DAS only reallocates training budget on the training set, while Succ and continuous tracking errors are measured on held-out clips and out-of-domain datasets. Improving continuous Eg-mpjpe, Empjpe, Eacc, and Evel cannot be obtained merely by mining the binary Succ threshold. There is no fitted parameter renamed as a prediction, no uniqueness theorem imported from the authors, and no load-bearing self-citation chain. Prior work by overlapping authors (e.g., TokenHSI) appears only as related-work context. The residual risk that τ and the 0.5 m criterion couple curriculum to the headline metric is an evaluation-design concern, not circularity under the stated patterns.

Assumptions & free parameters 6 free parameters · 5 assumptions · 3 invented entities

The work is empirical RL, not a formal derivation. Load-bearing choices are standard MDP/PPO assumptions, the imitation reward form, fixed early-termination thresholds that define the two experts, the 0.5 m success criterion, height-map scene encoding, and several hand-tuned sampling/EMA constants that steer DAD. Invented entities are algorithmic constructs (experts, DAD/DAS, routing groups), not physical objects; they have no independent evidence outside the reported sim metrics.

free parameters (6)
  • early_termination_threshold_tau = 0.25 / 0.5
    τ=0.25 for imitation expert and τ=0.5 for interaction expert/student; Fig. 1(b) shows the feasibility–fidelity trade-off is controlled by this hand-chosen threshold.
  • success_failure_joint_distance_delta = 0.5 m
    Binary failure when average joint distance exceeds 0.5 m; used both for evaluation and for difficulty EMA.
  • DAS_temperatures_and_uniform_mix = 1.0 / 1.0 / 0.05
    τ_inter=τ_intra=1.0, ϵ=0.05; ablated in Supp. Table 7.
  • EMA_and_learnability_filter = 0.95 / 0.9 / 0.1 / 1.0
    Failure EMA β=0.95, return EMA η=0.9, ϵu=0.1, λu=1.0; ablated in Supp. Table 8.
  • PPO_and_KD_loss_weights_and_learning_rates = LR actor-MLP 2e-5, transformer 2e-6, critic 1e-4
    λ_PPO, λ_KD and actor/critic learning rates (Supp. Table 9) balance distillation vs on-policy RL; not derived.
  • imitation_reward_coefficients = 0.5, 0.3, 0.1, 0.1
    Fixed weights and temperature constants in r_imit (Supp. Eq. 12): 0.5/0.3/0.1/0.1 and exp scales 100/10/0.1/0.1.
assumptions (5)
  • domain assumption HSI imitation is an MDP solvable by PPO with GAE and a hand-designed imitation reward that matches joint positions/rotations/velocities.
    Sec. 3 and Supp. D–E; standard in DeepMimic-style work but not proved optimal for complex scenes.
  • domain assumption A local 20×20 height map (8 cm spacing) is a sufficient scene observation for collision-aware control.
    Sec. 3.1 Interaction Expert; inherited from MaskedMimic-style perception.
  • ad hoc to paper Partitioning motions by whether the scene-free imitation expert succeeds in the scene-aware simulator yields complementary teachers without harmful gradient interference.
    Regime-specialized routing Eq. (2); validated only by ablation vs random routing / action averaging.
  • ad hoc to paper Online failure rate EMA plus return-improvement EMA identify hard-yet-learnable clips better than uniform sampling.
    Sec. 3.2 DAS; core of DAD, supported by ablations but not by theory.
  • domain assumption Mean SMPL body shape and Isaac Gym contact dynamics are adequate proxies for evaluation of HSI fidelity.
    Implementation details Sec. 4.1; body-scale transfer is tested but real hardware is not.
invented entities (3)
  • Dual Flow Strategy (imitation expert + interaction expert)
    purpose: Decouple strict tracking from collision-aware adaptation before distillation.
    Stage I of ComplexMimic; algorithmic split, not a physical entity.
  • Difficulty-Aware Distillation / Difficulty-Aware Sampling (DAD/DAS)
    purpose: Reweight multi-teacher supervision toward hard-yet-learnable motions via failure and learnability signals.
    Stage II; defined by paper-specific scores D_k, I_k, u_k.
  • Regime-specialized teacher routing groups G_Imitate / G_Inter
    purpose: Assign each motion to one teacher to avoid conflicting distillation targets.
    Eq. (2); constructed from imitation-expert success on training motions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ComplexMimic: Human-Scene Interaction Imitation in Complex 3D Environments." pith.science (2026). https://pith.science/paper/O2F6OJXF

@misc{pith2026260702034,
  author       = {Pith},
  title        = {Pith review of: ComplexMimic: Human-Scene Interaction Imitation in Complex 3D Environments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O2F6OJXF}},
  note         = {Machine review of arXiv:2607.02034}
}
read the original abstract

Physics-based Human-Scene Interaction (HSI) imitation learning is crucial for embodied intelligence as it bridges the gap between kinematic 3D motions and real-world dynamics. However, most existing methods focus on simplified scene settings, leaving complex environments largely unexplored, which limits their applicability in real-world scenarios. In this paper, we focus on HSI mimicry in complex environments. Under this complex setting, we observe an inherent trade-off between successfully performing interaction and maintaining natural, physically plausible motions. To address this challenge, we propose ComplexMimic, a framework that reconstructs diverse HSI by interpreting imperfect MoCap data. First, we introduce a Dual Flow Strategy, which learns two complementary experts: an imitation expert for accurate motion tracking and an interaction expert for collision-aware adaptation in complex scenes. Second, naive multi-expert distillation, which treats all experts equally, often under-samples challenging behaviors, limiting effective learning. To mitigate this issue, we propose a difficulty-aware distillation strategy that adaptively weights supervision and prioritizes hard-yet-learnable trajectories guided by failure statistics and learning progress signals. Extensive experiments on three benchmark datasets demonstrate that our approach outperforms current state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2607.02034 by the authors.

Figure 1
Figure 1. (a) Prior humanoid imitation-learning studies [20] typically focus on scene￾free or overly simplified environments (upper row), whereas our work targets realistic HSI imitation in complex 3D scenes (lower row). (b) Feasibility–faithfulness trade-off in HSI imitation. Varying the early termination threshold τ during training reveals a clear trade-off: larger τ improves task feasibility (higher success rate) but reduc… view at source ↗
Figure 2
Figure 2. Overview of ComplexMimic, a two-stage framework for human–scene inter￾action imitation in complex environments. In Stage I, an imitation expert and an interaction expert are trained to achieve motion-faithful tracking and collision-aware adaptation, respectively. In Stage II, these experts are frozen and distilled into a unified policy via Difficulty-Aware Distillation (DAD). Black arrows denote policy updates; red … view at source ↗
Figure 3
Figure 3. Examples of return improvement under different motion cases. The first refer￾ence motion conflicts heavily with the scene meshes, resulting in limited return gains. In contrast, the second motion achieves better tracking performance and larger return gains, demonstrating the effectiveness of our learning progress signal. A simple and effective choice is to set sk(m) proportional to the motion difficulty, sk(m) = Dk(… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Qualitative comparisons on the TRUMANS [16] dataset. Red boxes highlight typical failure modes under complicated scenes, such as undesired collisions, unstable contacts, and noticeable deviations from the reference. More visual results can be found in the supplementary…
Figure 5
Figure 5. Figure 5: Qualitative ablation comparisons on TRUMANS [16]. showing that (i) focusing updates on harder motions within each regime and (ii) balancing training across regimes are both beneficial. The learnability filtering is particularly important: removing it significantly degr…
Figure 6
Figure 6. Figure 6: Visualization of our method in Isaac Lab [PITH_FULL_IMAGE:figures/full_fig_p024_6.png]
Figure 7
Figure 7. Figure 7: Visualization under different body scales. J Discussion Limitations and Future Work. Although our method demonstrates strong performance on human–scene interaction imitation under complex 3D environ￾ments, it still has several limitations. First, our method cannot hand…
Figure 8
Figure 8. Figure 8: Examples of failure cases. The reference motion is severely penetrated with the 3D environments. in [PITH_FULL_IMAGE:figures/full_fig_p025_8.png]

Review history (2 revisions) →

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 14 linked inside Pith

  1. [1]

    In: Proceed- ings of the IEEE/CVF International Conference on Computer Vision

    Collorone, L., Gioia, M., Pappa, M., Leoni, P., Ficarra, G., Litany, O., Spinelli, I., Galasso, F.: Monster: a unified model for motion, scene, text retrieval. In: Proceed- ings of the IEEE/CVF International Conference on Computer Vision. pp. 10940– 10949 (2025)

    2025

  2. [2]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Guo, C., Zou, S., Zuo, X., Wang, S., Ji, W., Li, X., Cheng, L.: Generating di- verse and natural 3d human motions from text. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 5152–5161 (2022)

    2022

  3. [3]

    In: European Conference on Computer Vision

    Guo, C., Zuo, X., Wang, S., Cheng, L.: Tm2t: Stochastic and tokenized model- ing for the reciprocal generation of 3d human motions and texts. In: European Conference on Computer Vision. pp. 580–597. Springer (2022)

    2022

  4. [4]

    In: Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision

    Hassan, M., Ceylan, D., Villegas, R., Saito, J., Yang, J., Zhou, Y., Black, M.J.: Stochastic scene-aware motion prediction. In: Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision. pp. 11374–11384 (2021)

    2021

  5. [5]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Hassan, M., Choutas, V., Tzionas, D., Black, M.J.: Resolving 3d human pose ambi- guities with 3d scene constraints. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 2282–2292 (2019)

    2019

  6. [6]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Hassan, M., Ghosh, P., Tesch, J., Tzionas, D., Black, M.J.: Populating 3d scenes by learning human-scene interaction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14708–14718 (2021)

    2021

  7. [7]

    In: ACM SIGGRAPH 2023 Conference Pro- ceedings

    Hassan, M., Guo, Y., Wang, T., Black, M., Fidler, S., Peng, X.B.: Synthesizing physical character-scene interactions. In: ACM SIGGRAPH 2023 Conference Pro- ceedings. pp. 1–9 (2023)

    2023

  8. [8]

    arXiv preprint arXiv:2502.01143 (2025)

    He, T., Gao, J., Xiao, W., Zhang, Y., Wang, Z., Wang, J., Luo, Z., He, G., Soban- bab, N., Pan, C., et al.: Asap: Aligning simulation and real-world physics for learn- ing agile humanoid whole-body skills. arXiv preprint arXiv:2502.01143 (2025)

    arXiv 2025

Show all 53 references
  1. [9]

    arXiv preprint arXiv:2406.08858 (2024)

    He,T.,Luo,Z.,He,X.,Xiao,W.,Zhang,C.,Zhang,W.,Kitani,K.,Liu,C.,Shi,G.: Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning. arXiv preprint arXiv:2406.08858 (2024)

    2024 arXiv

  2. [10]

    arXiv preprint arXiv:1503.02531 (2015) 16 L

    Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015) 16 L. Pan and H. Zhao

    2015 arXiv

  3. [11]

    Advances in neural information processing systems33, 6840–6851 (2020)

    Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Advances in neural information processing systems33, 6840–6851 (2020)

    2020

  4. [12]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Huang, C.H.P., Yi, H., Höschle, M., Safroshkin, M., Alexiadis, T., Polikovsky, S., Scharstein, D., Black, M.J.: Capturing and inferring dense full-body human-scene contact. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13274–13285 (2022)

    2022

  5. [13]

    In: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision

    Hwang,I.,Zhou,B.,Kim,Y.M.,Wang,J.,Guo,C.:Scenemi:Motionin-betweening for modeling human-scene interaction. In: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision. pp. 6034–6045 (2025)

    2025

  6. [14]

    Neural computation3(1), 79–87 (1991)

    Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural computation3(1), 79–87 (1991)

    1991

  7. [15]

    In: SIGGRAPH Asia 2024 Conference Papers

    Jiang, N., He, Z., Wang, Z., Li, H., Chen, Y., Huang, S., Zhu, Y.: Autonomous character-scene interaction synthesis from text instruction. In: SIGGRAPH Asia 2024 Conference Papers. pp. 1–11 (2024)

    2024

  8. [16]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Jiang, N., Zhang, Z., Li, H., Ma, X., Wang, Z., Chen, Y., Liu, T., Zhu, Y., Huang, S.: Scaling up dynamic human-scene interaction modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1737– 1747 (2024)

    2024

  9. [17]

    arXiv preprint arXiv:1412.6980 (2014)

    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

    2014 arXiv

  10. [18]

    arXiv preprint arXiv:2508.08241 (2025)

    Liao, Q., Truong, T.E., Huang, X., Gao, Y., Tevet, G., Sreenath, K., Liu, C.K.: Beyondmimic: From motion tracking to versatile humanoid control via guided dif- fusion. arXiv preprint arXiv:2508.08241 (2025)

    2025 arXiv

  11. [19]

    In: European Conference on Computer Vision

    Liu, X., Hou, H., Yang, Y., Li, Y.L., Lu, C.: Revisit human-scene interaction via space occupancy. In: European Conference on Computer Vision. pp. 1–19. Springer (2024)

    2024

  12. [20]

    arXiv preprint arXiv:2412.17730 (2024)

    Liu, Y., Yang, B., Zhong, L., Wang, H., Yi, L.: Mimicking-bench: A benchmark for generalizable humanoid-scene interaction learning via human mimicking. arXiv preprint arXiv:2412.17730 (2024)

    2024 arXiv

  13. [21]

    Liu, Z., Ge, J., Xiong, M., Gu, J., Tang, B., Jing, W., Chen, S.: It takes two: Learning interactive whole-body control between humanoid robots (2025)

    2025

  14. [22]

    Luo, Z., Cao, J., Kitani, K., Xu, W., et al.: Perpetual humanoid control for real- timesimulatedavatars.In:ProceedingsoftheIEEE/CVFInternationalConference on Computer Vision. pp. 10895–10904 (2023)

    2023

  15. [23]

    arXiv preprint arXiv:2108.10470 (2021)

    Makoviychuk, V., Wawrzyniak, L., Guo, Y., Lu, M., Storey, K., Macklin, M., Hoeller, D., Rudin, N., Allshire, A., Handa, A., et al.: Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470 (2021)

    2021 arXiv

  16. [24]

    In: 2024 International Conference on 3D Vision (3DV)

    Pan, L., Wang, J., Huang, B., Zhang, J., Wang, H., Tang, X., Wang, Y.: Syn- thesizing physically plausible human motions in 3d scenes. In: 2024 International Conference on 3D Vision (3DV). pp. 1498–1507. IEEE (2024)

    2024

  17. [25]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Pan, L., Yang, Z., Dou, Z., Wang, W., Huang, B., Dai, B., Komura, T., Wang, J.: Tokenhsi: Unified synthesis of physical human-scene interactions through task tokenization. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 5379–5391 (2025)

    2025

  18. [26]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A.A., Tzionas, D., Black, M.J.: Expressive body capture: 3d hands, face, and body from a single im- age. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10975–10985 (2019) C...

    2019

  19. [27]

    ACM Transactions On Graphics (TOG)37(4), 1–14 (2018)

    Peng, X.B., Abbeel, P., Levine, S., Van de Panne, M.: Deepmimic: Example-guided deep reinforcement learning of physics-based character skills. ACM Transactions On Graphics (TOG)37(4), 1–14 (2018)

    2018

  20. [28]

    Peng, X.B., Guo, Y., Halper, L., Levine, S., Fidler, S.: Ase: Large-scale reusable adversarialskillembeddingsforphysicallysimulatedcharacters.ACMTransactions On Graphics (TOG)41(4), 1–17 (2022)

    2022

  21. [29]

    ACM Transactions on Graphics (ToG)40(4), 1–20 (2021)

    Peng, X.B., Ma, Z., Abbeel, P., Levine, S., Kanazawa, A.: Amp: Adversarial motion priors for stylized physics-based character control. ACM Transactions on Graphics (ToG)40(4), 1–20 (2021)

    2021

  22. [30]

    arXiv preprint arXiv:1506.02438 (2015)

    Schulman, J., Moritz, P., Levine, S., Jordan, M., Abbeel, P.: High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438 (2015)

    2015 arXiv

  23. [31]

    arXiv preprint arXiv:1707.06347 (2017)

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)

    2017 arXiv

  24. [32]

    arXiv preprint arXiv:2508.07842 (2025)

    Shen, Y., Liu, H., Zhang, L., Liu, P., Xia, R., Yao, T., Feng, T.: Detach: Cross- domain learning for long-horizon tasks via mixture of disentangled experts. arXiv preprint arXiv:2508.07842 (2025)

    2025

  25. [33]

    In: nternational conference on machine learning

    Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., Ganguli, S.: Deep unsuper- vised learning using nonequilibrium thermodynamics. In: nternational conference on machine learning. pp. 2256–2265 (2015)

    2015

  26. [34]

    arXiv preprint arXiv:2010.02502 (2020)

    Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 (2020)

    2010 arXiv

  27. [35]

    arXiv preprint arXiv:2011.13456 (2020)

    Song, Y., Sohl-Dickstein, J., Kingma, D.P., Kumar, A., Ermon, S., Poole, B.: Score- based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 (2020)

    2011 arXiv

  28. [36]

    ACM Trans- actions On Graphics (TOG)43(6), 1–21 (2024)

    Tessler, C., Guo, Y., Nabati, O., Chechik, G., Peng, X.B.: Maskedmimic: Unified physics-based character control through masked motion inpainting. ACM Trans- actions On Graphics (TOG)43(6), 1–21 (2024)

    2024

  29. [37]

    arXiv preprint arXiv:2209.14916 (2022)

    Tevet, G., Raab, S., Gordon, B., Shafir, Y., Cohen-Or, D., Bermano, A.H.: Human motion diffusion model. arXiv preprint arXiv:2209.14916 (2022)

    2022 arXiv

  30. [38]

    In: Proceedings of the IEEE/CVF International Con- ference on Computer Vision

    Wang, W., Pan, L., Dou, Z., Mei, J., Liao, Z., Lou, Y., Wu, Y., Yang, L., Wang, J., Komura, T.: Sims: Simulating stylized human-scene interactions with retrieval- augmented script generation. In: Proceedings of the IEEE/CVF International Con- ference on Computer Vision. pp. 14...

    2025

  31. [39]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wang, Y., Zhao, Q., Yu, R., Tsui, H.W., Zeng, A., Lin, J., Luo, Z., Yu, J., Li, X., Chen, Q., et al.: Skillmimic: Learning basketball interaction skills from demon- strations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 17540–17549 (2025)

    2025

  32. [40]

    arXiv preprint arXiv:2506.12779 (2025)

    Wang, Y., Yang, M., Ding, Z., Zhang, Y., Zeng, W., Xu, X., Jiang, H., Lu, Z.: From experts to a generalist: Toward general whole-body control for humanoid robots. arXiv preprint arXiv:2506.12779 (2025)

    2025 arXiv

  33. [41]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wang,Z.,Chen,Y.,Jia,B.,Li,P.,Zhang,J.,Zhang,J.,Liu,T.,Zhu,Y.,Liang,W., Huang, S.: Move as you say interact as you can: Language-guided human motion generation with scene affordance. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 433–...

    2024

  34. [42]

    Advances in Neural Informa- tion Processing Systems35, 14959–14971 (2022)

    Wang, Z., Chen, Y., Liu, T., Zhu, Y., Liang, W., Huang, S.: Humanise: Language- conditioned human motion generation in 3d scenes. Advances in Neural Informa- tion Processing Systems35, 14959–14971 (2022)

    2022

  35. [43]

    arXiv preprint arXiv:2309.07918 (2023) 18 L

    Xiao, Z., Wang, T., Wang, J., Cao, J., Zhang, W., Dai, B., Lin, D., Pang, J.: Unified human-scene interaction via prompted chain-of-contacts. arXiv preprint arXiv:2309.07918 (2023) 18 L. Pan and H. Zhao

    2023 arXiv

  36. [44]

    ACM Transactions on Graphics (TOG)44(6), 1–14 (2025)

    Xu, P., Wu, Z., Wang, R., Sarukkai, V., Fatahalian, K., Karamouzas, I., Zordan, V., Liu, C.K.: Learning to ball: Composing policies for long-horizon basketball moves. ACM Transactions on Graphics (TOG)44(6), 1–14 (2025)

    2025

  37. [45]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Xu, S., Ling, H.Y., Wang, Y.X., Gui, L.Y.: Intermimic: Towards universal whole- body control for physics-based human-object interactions. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 12266–12277 (2025)

    2025

  38. [46]

    In: Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers

    Yu,R.,Wang,Y.,Zhao,Q.,Tsui,H.W.,Wang,J.,Tan,P.,Chen,Q.:Skillmimic-v2: Learning robust and generalizable interaction skills from sparse and noisy demon- strations. In: Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conferenc...

    2025

  39. [47]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Zhang, J., Fan, H., Yang, Y.: Energymogen: Compositional human motion gen- eration with energy-based diffusion model in latent space. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 17592–17602 (2025)

    2025

  40. [48]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Zhang, J., Fan, H., Yang, Y.: Towards decompositional human motion generation with energy-based diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 30650–30660 (2026)

    2026

  41. [49]

    In: Proceedings of theIEEE/CVF conference on computer vision and pattern recognition

    Zhang, J., Zhang, Y., Cun, X., Zhang, Y., Zhao, H., Lu, H., Shen, X., Shan, Y.: Generating human motion from textual descriptions with discrete representa- tions. In: Proceedings of theIEEE/CVF conference on computer vision and pattern recognition. pp. 14730–14740 (2023)

    2023

  42. [50]

    IEEE transactions on pattern analysis and machine intelligence46(6), 4115–4128 (2024)

    Zhang, M., Cai, Z., Pan, L., Hong, F., Guo, X., Yang, L., Liu, Z.: Motiondiffuse: Text-driven human motion generation with diffusion model. IEEE transactions on pattern analysis and machine intelligence46(6), 4115–4128 (2024)

    2024

  43. [51]

    In: European Conference on Computer Vision (ECCV) (2022)

    Zhao, K., Wang, S., Zhang, Y., Beeler, T., Tang, S.: Compositional human-scene interaction synthesis with semantic control. In: European Conference on Computer Vision (ECCV) (2022)

    2022

  44. [52]

    In: Proceedings of the IEEE/CVF international con- ference on computer vision

    Zhao, K., Zhang, Y., Wang, S., Beeler, T., Tang, S.: Synthesizing diverse human motions in 3d indoor scenes. In: Proceedings of the IEEE/CVF international con- ference on computer vision. pp. 14738–14749 (2023)

    2023

  45. [53]

    w/o” indicates removal; “–

    Zheng, Y., Yang, Y., Mo, K., Li, J., Yu, T., Liu, Y., Liu, C.K., Guibas, L.J.: Gimo: Gaze-informed human motion prediction in context. In: European Conference on Computer Vision. pp. 676–694. Springer (2022) ComplexMimic 19 Supplementary Material In this supplementary material...

    2022

Pith tools

Reviewed July 12, 2026 · model on record in the stance chip above.