Pith. sign in

REVIEW 4 major objections 5 minor 73 references

PhysiInter: Integrating Physical Mapping for High-Fidelity Human Interaction Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Physical mapping, a projection of training and generated motions through a physics simulator, removes sliding, floating, and penetration artifacts while preserving semantic meaning.

desk verdict A genuinely new way to use motion imitation for both data augmentation and inference-time post-processing in text-to-motion, but the headline numbers overstate the result because baselines don't get the same post-processing and the imitation failure rate is never reported. read the letter →

arxiv 2506.07456 v1 pith:ADO3VTAO submitted 2025-06-09 cs.CV

Wei Yao , Yunlian Sun , Chang Liu , Hongwen Zhang , Jinhui Tang This is my paper
classification cs.CV
keywords text-to-motiongenerationhumaninteractionphysicalmappingphysics-basedmotionimitationplausibilitydiffusionmodelscapturedataaugmentationmarker-basedloss
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the recurring artifacts in text-to-motion generation—foot sliding, floating, ground penetration, and body-body interpenetration—can be removed at the source by projecting every motion through a physics simulator before it is used for training or released as output. The core proposal, physical mapping, trains a simulated humanoid agent to imitate a captured or generated motion; the agent's physically valid replay replaces the original sequence while keeping its semantic content. The authors demonstrate the idea on both two-person interaction data and single-person motion data, reporting 3%–89% improvements across physical-plausibility metrics and user-study preferences. They also introduce a non-canonical motion representation and two losses, Motion Consistency and Marker-based Interaction, that make the generated position, velocity, and rotation components agree with each other and sharpen human-human contact. If correct, the method offers a generator-agnostic way to make physically grounded motion the default rather than an afterthought.

What carries the argument

The load-bearing mechanism is physical mapping, a two-stage projection through a physics-based motion imitation system. A humanoid agent with human-like mass and collision geometry is trained by reinforcement learning to follow target joint positions via PD controllers in a physics simulator; the policy's replay is the physically valid version of the motion. The paper also introduces a non-canonical motion representation of the form $[p, v, r]$ (global joint positions, velocities, and 6D local rotations) so that interactions can be modeled without a canonical root at the origin, and two loss functions: the Motion Consistency loss, which aligns the velocity and position components with forward kinematics from rotations, and the Marker-based Interaction loss, which computes a $67 \times 67$ pairwise marker-distance map between interacting humans to supervise contact and near-contact.

What would settle it

Take a held-out set of interaction classes that are rare in the imitation policy's training data—for example backflips, handstands, or one person lifting another—and run the full pipeline on them. If most of these samples are discarded by the 'failed imitation' filter, or if text-motion alignment drops sharply after physical mapping, that would show the central claim is bounded by imitation coverage rather than general physical fidelity.

Watch

Extended reading notes

Core claim

The central claim is that physical constraints can be integrated throughout the human interaction generation pipeline by treating motion imitation as a projection: any motion sequence, whether from a motion-capture dataset or a diffusion model, is fed to a reinforcement-learned policy that drives a simulated humanoid to reproduce it under gravity, friction, and collision. The imitated motion replaces the input after failed imitations are discarded, yielding a physically valid sequence with the same semantics. On the data side this cleans the training distribution; on the generation side it acts as post-processing that corrects residual artifacts. Supporting this pipeline, the paper shows that a non-canonical representation encoding global position, velocity, and 6D rotations is needed for interactions, and that two new losses—Motion Consistency, which forces position, velocity, and rotation to describe the same motion, and Marker-based Interaction, which supervises pairwise distances between 67 surface markers—materially improve quality.

Load-bearing premise

The whole pipeline depends on the ability of a simulated humanoid to imitate a motion without changing its meaning; motions it cannot imitate are thrown away, so the reported gains are measured only on the motions it can imitate.

Editorial extensions

If this is right

  • Motion-capture datasets cleaned by physical mapping train models that reproduce fewer ground-penetration, floating, and sliding artifacts; on the two-person interaction dataset the mapping reduces penetration by 64% and interpenetration by 27% relative to the raw data.
  • Generated motions passed through physical mapping as post-processing gain physical fidelity even when the generator was trained on uncorrected data, with reported gains of 3%–89% across physical metrics.
  • With the Motion Consistency loss, position-based and rotation-based readings of the same generated motion converge (MPJPE below 15 mm), so rotations can be used directly to drive a mesh without extra fitting.
  • The Marker-based Interaction loss improves contact modeling for two-person motions, with 8 of 13 physical metrics improving in the ablation.
  • Because the framework only requires a data-driven generator over compatible motion units, the same physical mapping and losses apply to single-person and multi-person generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable consequence the paper does not pursue: the fraction of motions discarded by imitation is itself a coverage diagnostic. If the discard rate is low on everyday interactions but high on acrobatic or load-bearing motions, the method's guarantees are strongest precisely where current baselines already do reasonably well.
  • Physical mapping could be embedded inside the diffusion loop as a differentiable or iterative projection instead of a one-shot post-process, which might reduce the semantic drift that a single replay can introduce.
  • The proposed physical metrics (Penetration, Float, Foot Contact, Skate, PFC, Interpenetration) measure the exact failure modes the method targets; a natural extension is to validate them against perceptual judgments on a broader set of motions.
  • The Motion Consistency idea, enforcing cross-component consistency in a composite representation, should transfer to other multi-channel generative tasks such as audio-driven gesture or full-body grasping, where positions, velocities, and rotations must also agree.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces "physical mapping," a method that uses physics-based motion imitation (specifically PHC) to project captured and generated human motions into a physically plausible space. This mapping is applied in two ways: as a preprocessing/augmentation step for motion-capture training data and as an inference-time post-processor for generated motions. The authors also propose a non-canonical global motion representation for two-person interactions, a Motion Consistency (MC) loss to enforce agreement between position, velocity, and rotation components, and a Marker-Based Interaction (MI) loss to better model body-surface contacts. Experiments are conducted on the InterHuman and HumanML3D datasets against InterGen and MDM baselines, reporting 3%-89% improvements in physical-fidelity metrics and a user-study preference for the proposed method.

Significance. If the claims are validated, the work would offer a practical recipe for reducing foot sliding, floating, and interpenetration artifacts in text-to-motion generation, particularly for multi-person interactions, by leveraging recent advances in physics-based motion imitation. The paper's strengths include the clear formulation of physical mapping as a training-data augmentation plus inference-time post-processing pipeline, the introduction of a marker-based interaction loss that targets contact semantics rather than raw joint distances, and the inclusion of a user study to complement automatic metrics. The central quantitative claim, however, is not yet established because the main comparisons confound post-processing with training changes, because the custom physical metrics are aligned with the post-processor's objectives, and because the coverage of the imitation policy is unquantified.

major comments (4)
  1. [Section IV.B.1, Table I] The main inference-time comparison applies the physics post-processor only to the proposed method while evaluating baselines (InterGen-Pos, InterGen-Rot, MDM) on their raw outputs. The physical metrics reported in Table I (Penetration, Float, Foot Contact, Skate, PFC, Interpenetration) are precisely the quantities that physical mapping is designed to minimize, so a method that projects onto a physics-constrained manifold will improve these scores by construction, independent of generative quality. To separate the contribution of post-processing from that of augmented training data and the proposed losses, the authors should apply the same post-processor to the baselines and report those numbers, or report pre-post-processing metrics for all methods.
  2. [Section III-B and Section V.A] The paper states in Section III-B that "failed imitation samples" are discarded after physical mapping, and Section V.A concedes that imitation failure is particularly pronounced in complex human-human interactions, giving falls for backflips and handstands as examples. Yet no coverage statistic is reported: the fraction of InterHuman sequences discarded, their semantic categories, or the success rate of post-processing generated motions are all absent. If many motions are dropped or distorted, the augmented training set is biased toward easily imitable motions, and the test-time evaluation may either include failed post-processed outputs (dragging down averages) or exclude them (making the comparison not representative of the full distribution). The headline 3%-89% improvement is therefore not interpretable without a coverage statistic.
  3. [Section IV.C.2, Eq. (3) and Table III] The MC loss ablation is expected by construction. The MPJPE metric measures the discrepancy between position-derived and rotation-derived joint positions, and the second term of Eq. (3) directly minimizes ∥x - FK(r)∥ on the same quantity. Reporting that MC reduces MPJPE from above 50 mm to below 15 mm therefore does not independently validate motion quality; it only confirms that the loss optimizes the reported metric. The authors should provide an evaluation metric that is not the training objective of MC loss, or should explicitly frame the MPJPE result as a consistency check rather than a quality improvement.
  4. [Section IV.B.1, Table I] The single-person results are mixed and do not support the abstract's blanket claim of "3%-89% improvement in physical fidelity." In Table I, against HumanML3D the proposed method worsens Penetration (3.1509 to 4.5257) and Skate (1.2962 to 3.0985); against MDM it worsens Penetration (1.3565 to 3.4764) and Skate (0.5622 to 4.6395). The paper should report per-metric direction and either restrict the headline claim to the multi-person setting where the gains are consistent or provide a principled aggregation that justifies the range quoted in the abstract.
minor comments (5)
  1. [Section III-B, Eq. (1)] The second line of Eq. (1) contains a typo: the right-hand side has X_b^(t) + σ_t ε_b for both denoising arguments, and the conditioning on X_a is missing; this appears to be a copy-paste error from the first line.
  2. [Section III-C, Eq. (4)] The text says "M denotes a 67 × 67 distance map distance map," with a duplicated phrase; please correct this.
  3. [Table I and Table III] The table legends are ambiguous about what "Ours" refers to in the dataset rows (e.g., the InterHuman row), because "Ours" seems to denote the physically mapped dataset rather than a generative model output; please clarify the legend and row semantics.
  4. [Section IV.A.2, PFC definition] The definition "PFC = v_left_foot · v_right_foot · a_root" is not mathematically well-defined as written because the dot product of three vectors is not defined; please specify the intended operation (e.g., elementwise product or triple product) and describe how the scalar is computed.
  5. [Section II.A] Reference [32] is listed as "Auto-encoding variational bayes" without the actual author list for Kingma and Welling's VAE paper; this reference should be corrected to include the full citation.

Circularity Check

1 steps flagged · score 6.0 of 10

The headline 3%-89% physical-fidelity gain is partially self-referential because the evaluation metrics measure exactly the constraints that the physical-mapping post-processor is built to enforce.

  1. self definitional [Section III-B (Pipeline), Section IV-A.2 (Metrics), Table I footnote]
    "The second application of physical mapping is post-processing to refine generated outputs. ... To address this, we extract the positional component of the generated sequence and apply physical mapping to produce the final result. ... O URS DENOTES BASELINES INTEGRATED WITH OUR METHOD ... (1) Penetration: The average depth by which the human mesh penetrates the ground plane. (2) Float: The average height of the lowest point of the human mesh above the ground. (3) Foot Contact: Defined as P enetration+ F loat ..."

    The physical metrics (Penetration, Float, Foot Contact, Skate, PFC, Interpenetration) are precisely the quantities that the physics-simulation post-processor is designed to remove or constrain: the simulator prevents ground penetration, enforces ground contact, and penalizes interpenetration. The 'Ours' rows in Table I are baselines plus physical mapping applied as post-processing to every generated sequence before evaluation. Therefore, large improvements on these metrics are largely guaranteed by the post-processing operation itself; they confirm that the post-processor implements its intended projection rather than that the generative model independently learned physical plausibility.

full rationale

The clearest circular element is the inference-time evaluation: physical mapping is applied as a post-processor, and the paper's physical metrics are the same artifact categories that the physics simulator enforces, so the Table I 'Ours' gains on those metrics reduce, to a substantial degree, to the post-processor doing what it was built to do. The training-data augmentation comparison (Phys Data vs Baseline) is more independent because it shows that training on physically mapped data improves generation even before post-processing, though it is still evaluated on the same physical metrics and on the filtered subset of motions that imitation could successfully map. The paper's own Section V.A limitation concedes that imitation fails on complex interactions such as backflips, handstands, and lifting motions, but this is a coverage/correctness concern rather than a circularity concern. I do not find a load-bearing self-citation chain: the motion-imitation backbone PHC is external prior work, and no uniqueness theorem from the present authors is invoked. The MC and MI losses are supervised training objectives; reporting that MC loss reduces the MPJPE consistency metric is a standard loss-metric match, not a circular prediction. Overall, the central contribution has independent content through the data-augmentation experiments, but the headline physical-fidelity claim is partially circular, supporting a score of 6.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No new particles, forces, or conserved quantities are introduced; 'physical mapping' is an algorithmic process, not an entity. The free parameters are hand-chosen thresholds in the MI loss. The domain assumptions all concern the fidelity of the simulation and imitation stack, which the paper itself flags as imperfect.

free parameters (2)
  • MI contact threshold = 0.1 m
    Threshold in the first term of Eq. (4); chosen by hand to decide when two markers are in contact.
  • MI distance threshold = 1 m
    Threshold in the second term of Eq. (4); interactions farther than this are masked out, chosen by hand.
assumptions (4)
  • domain assumption Simulated physics in Isaac Gym approximates real-world mechanics closely enough for the corrected motions to be physically valid.
    The whole physical mapping is built on this. The authors admit the simulator is imperfect (Section V-A.3).
  • domain assumption The PHC imitation policy can reproduce arbitrary single- and multi-person motions without semantic distortion.
    Physical mapping inherits PHC's policy; the paper discards failed imitation samples, which means this assumption is false for some motions (Section V-A.1).
  • domain assumption Sparse surface markers (67 per person) are sufficient to capture interaction semantics.
    MI loss depends on this; the paper cites prior marker literature but does not validate it with an ablation.
  • domain assumption Discarding failed imitation samples does not bias the motion distribution used for training and evaluation.
    The paper removes failed samples without reporting what fraction fails or how that affects the dataset (Section III-B).

how reviews work

0 comments
Cite this review

Pith. "Pith review of PhysiInter: Integrating Physical Mapping for High-Fidelity Human Interaction Generation." pith.science (2026). https://pith.science/paper/ADO3VTAO

@misc{pith2026250607456,
  author       = {Pith},
  title        = {Pith review of: PhysiInter: Integrating Physical Mapping for High-Fidelity Human Interaction Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ADO3VTAO}},
  note         = {Machine review of arXiv:2506.07456}
}
read the original abstract

Driven by advancements in motion capture and generative artificial intelligence, leveraging large-scale MoCap datasets to train generative models for synthesizing diverse, realistic human motions has become a promising research direction. However, existing motion-capture techniques and generative models often neglect physical constraints, leading to artifacts such as interpenetration, sliding, and floating. These issues are exacerbated in multi-person motion generation, where complex interactions are involved. To address these limitations, we introduce physical mapping, integrated throughout the human interaction generation pipeline. Specifically, motion imitation within a physics-based simulation environment is used to project target motions into a physically valid space. The resulting motions are adjusted to adhere to real-world physics constraints while retaining their original semantic meaning. This mapping not only improves MoCap data quality but also directly informs post-processing of generated motions. Given the unique interactivity of multi-person scenarios, we propose a tailored motion representation framework. Motion Consistency (MC) and Marker-based Interaction (MI) loss functions are introduced to improve model performance. Experiments show our method achieves impressive results in generated human motion quality, with a 3%-89% improvement in physical fidelity. Project page http://yw0208.github.io/physiinter

Figures

Figures reproduced from arXiv: 2506.07456 by the authors.

Figure 1
Figure 1. Insight All human activities inherently adhere to the laws of physics. Motivated by this observation, we introduce physical mapping, a framework that enforces physical constraints on human motion generation, thereby producing higher-fidelity motion sequences. Both MoCap systems and generative models often over￾look the fact that human motion adheres to physical laws. Compared to single-person MoCap, multi-person sce… view at source ↗
Figure 2
Figure 2. Comparison with Previous Methods Prior approaches directly leverage motion capture data to train generative models, which often results in motion artifacts. To address this, we propose physical mapping, a method that preprocesses mocap data during training and directly optimizes the generated motion sequences, thereby significantly enhancing motion quality. the semantic intent of the original motion while satisfying… view at source ↗
Figure 3
Figure 3. Overview of Pipeline (a) Data: A reinforcement learning policy is employed to map motion data into a physical simulator. This step enhances motion data quality while ensuring physical plausibility. (b) Imitation: Motion imitation serves as the core component of our framework, enabling data augmentation and post-processing of generated motions. (c) Generator: A diffusion model is utilized, guided by text prompts to d… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Marker-Based Interaction Loss The pink points represent pre￾defined motion capture (mocap) markers, with a total of 67 markers per hu￾man. The proposed marker interaction (MI) loss computes pairwise distances between markers to construct a 67 × 67 distance matrix. This…
Figure 5
Figure 5. Figure 5: Visualization of Human Mesh Simplification To reduce compu￾tational complexity for interpenetration volume estimation, the human body mesh is approximated as a geometric structure composed of 45 spheres. And the yellow points here are joints. Physical Metrics A core co…
Figure 6
Figure 6. Figure 6: User Study Results We use a scoring system to compare ours with various benchmark datasets and methods, using physical plausibility and text matching as the criteria. The red dashed line indicates 50%. Exceeding the red line means ours is preferred. one greets the othe…
Figure 7
Figure 7. Figure 7: Comparison of Position-based and Rotation-based GT represents the ground-truth motion representation in the dataset. Generate represents the motion generated by the model. POS and ROT represent position-based and rotation-based motions. scriptions. As demonstrated in …
Figure 8
Figure 8. Figure 8: Quantitative Results of Motion Quality. FID* evaluates motion quality by comparing it with the motion distribution of real motion. The smaller the value, the better the motion quality. Each group has two values measured using position-based and rotation-based motion. T…
Figure 9
Figure 9. Figure 9: Qualitative Results We present qualitative results for both single-person and multi-person motion generation. Our method produces motion sequences that align with text prompts while maintaining high physical plausibility [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: R-Precision performance on the InterHuman dataset. Experimental [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

73 extracted references · 48 canonical work pages

  1. [1]

    Pymaf: 3d human pose and shape regression with pyramidal mesh alignment feedback loop,

    H. Zhang, Y . Tian, X. Zhou, W. Ouyang, Y . Liu, L. Wang, and Z. Sun, “Pymaf: 3d human pose and shape regression with pyramidal mesh alignment feedback loop,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 11 446–11 456

    work page 2021

  2. [2]

    Pymaf-x: Towards well-aligned full-body model regression from monocular im- ages,

    H. Zhang, Y . Tian, Y . Zhang, M. Li, L. An, Z. Sun, and Y . Liu, “Pymaf-x: Towards well-aligned full-body model regression from monocular im- ages,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023

    work page 2023

  3. [3]

    Staf: 3d human mesh recovery from video with spatio-temporal alignment fusion,

    W. Yao, H. Zhang, Y . Sun, and J. Tang, “Staf: 3d human mesh recovery from video with spatio-temporal alignment fusion,” IEEE Transactions on Circuits and Systems for Video Technology , 2024

    work page 2024

  4. [4]

    Vibe: Video inference for human body pose and shape estimation,

    M. Kocabas, N. Athanasiou, and M. J. Black, “Vibe: Video inference for human body pose and shape estimation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 5253–5263

    work page 2020

  5. [5]

    Spec: Seeing people in the wild with an estimated camera,

    M. Kocabas, C.-H. P. Huang, J. Tesch, L. M ¨uller, O. Hilliges, and M. J. Black, “Spec: Seeing people in the wild with an estimated camera,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 11 035–11 045

    work page 2021

  6. [6]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems , vol. 33, pp. 6840– 6851, 2020

    2020

  7. [7]

    Human motion diffusion model,

    G. Tevet, S. Raab, B. Gordon, Y . Shafir, D. Cohen-or, and A. H. Bermano, “Human motion diffusion model,” in The Eleventh International Conference on Learning Representations , 2023. [Online]. Available: https://openreview.net/forum?id=SJ1kSyO2jwu

    work page 2023

  8. [8]

    Human motion diffusion as a generative prior,

    Y . Shafir, G. Tevet, R. Kapon, and A. H. Bermano, “Human motion diffusion as a generative prior,” arXiv preprint arXiv:2303.01418 , 2023

    arXiv 2023

Show all 73 references
  1. [9]

    Mo- tiondiffuse: Text-driven human motion generation with diffusion model,

    M. Zhang, Z. Cai, L. Pan, F. Hong, X. Guo, L. Yang, and Z. Liu, “Mo- tiondiffuse: Text-driven human motion generation with diffusion model,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024

    2024

  2. [10]

    Generating human motion from textual descriptions with discrete representations,

    J. Zhang, Y . Zhang, X. Cun, S. Huang, Y . Zhang, H. Zhao, H. Lu, and X. Shen, “Generating human motion from textual descriptions with discrete representations,” 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 14 730–14 740, 2023. [Online]. Ava...

    2023

  3. [11]

    Motiongpt: Human motion as a foreign language,

    B. Jiang, X. Chen, W. Liu, J. Yu, G. Yu, and T. Chen, “Motiongpt: Human motion as a foreign language,” ArXiv, vol. abs/2306.14795,

    arXiv

  4. [12]

    Motionclip: Exposing human motion generation to clip space,

    G. Tevet, B. Gordon, A. Hertz, A. H. Bermano, and D. Cohen-Or, “Motionclip: Exposing human motion generation to clip space,” in European Conference on Computer Vision , 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID:247450907

    2022

  5. [13]

    Motion mamba: Efficient and long sequence motion generation with hierarchical and bidirectional selective ssm,

    Z. Zhang, A. Liu, I. Reid, R. Hartley, B. Zhuang, and H. Tang, “Motion mamba: Efficient and long sequence motion generation with hierarchical and bidirectional selective ssm,”ArXiv, vol. abs/2403.07487,

    arXiv

  6. [14]

    Humantomato: Text-aligned whole-body motion generation,

    S. Lu, L.-H. Chen, A. Zeng, J. de Lin, R. Zhang, L. Zhang, and H. yeung Shum, “Humantomato: Text-aligned whole-body motion generation,” ArXiv, vol. abs/2310.12978, 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:264306297

    2023 arXiv

  7. [15]

    Omg: Towards open-vocabulary motion generation via mixture of controllers,

    H. Liang, J. Bao, R. Zhang, S. Ren, Y . Xu, S. Yang, X. Chen, J. Yu, and L. Xu, “Omg: Towards open-vocabulary motion generation via mixture of controllers,” 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 482–493, 2023. [Online]. Available: htt...

    2024

  8. [16]

    Isaac gym: High performance gpu-based physics simulation for robot learning,

    V . Makoviychuk, L. Wawrzyniak, Y . Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa et al. , “Isaac gym: High performance gpu-based physics simulation for robot learning,” arXiv preprint arXiv:2108.10470, 2021. JOURNAL OF LATEX CLASS FILES, VOL. ...

    2021 arXiv

  9. [17]

    Mujoco: A physics engine for model- based control,

    E. Todorov, T. Erez, and Y . Tassa, “Mujoco: A physics engine for model- based control,” in 2012 IEEE/RSJ international conference on intelligent robots and systems . IEEE, 2012, pp. 5026–5033

    2012

  10. [18]

    Remos: 3d motion-conditioned reaction synthesis for two-person inter- actions,

    A. Ghosh, R. Dabral, V . Golyanik, C. Theobalt, and P. Slusallek, “Remos: 3d motion-conditioned reaction synthesis for two-person inter- actions,” in European Conference on Computer Vision , 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:265466318

    2023

  11. [19]

    Intergen: Diffusion- based multi-human motion generation under complex interactions,

    H. Liang, W. Zhang, W. Li, J. Yu, and L. Xu, “Intergen: Diffusion- based multi-human motion generation under complex interactions,” International Journal of Computer Vision , pp. 1–21, 2024

    2024

  12. [20]

    Inter-x: Towards versatile human-human interaction analysis,

    L. Xu, X. Lv, Y . Yan, X. Jin, S. Wu, C. Xu, Y . Liu, Y . Zhou, F. Rao, X. Sheng et al. , “Inter-x: Towards versatile human-human interaction analysis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 22 260–22 271

    2024

  13. [21]

    in2in: Leveraging individual information to generate human interactions,

    P. Ruiz-Ponce, G. Barquero, C. Palmero, S. Escalera, and J. Garc ´ıa- Rodr´ıguez, “in2in: Leveraging individual information to generate human interactions,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 1941–1951

    2024

  14. [22]

    Intermask: 3d human interaction generation via collaborative masked modelling,

    M. G. Javed, C. Guo, L. Cheng, and X. Li, “Intermask: 3d human interaction generation via collaborative masked modelling,” ArXiv, vol. abs/2410.10010, 2024. [Online]. Available: https://api.semanticscholar. org/CorpusID:273346671

    2024 arXiv

  15. [23]

    Interdance:reactive 3d dance generation with realistic duet interactions,

    R. Li, Y .-Z. Zhang, Y . Zhang, Y . Zhang, M. Su, J. Guo, Z. Liu, Y . Liu, and X. Li, “Interdance:reactive 3d dance generation with realistic duet interactions,” ArXiv, vol. abs/2412.16982, 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:274982678

    2024 arXiv

  16. [24]

    Interact: Capture and modelling of realistic, expressive and interactive activities between two persons in daily scenarios,

    Y . Huang, L. Ho, D. Qin, M. Shi, and T. Komura, “Interact: Capture and modelling of realistic, expressive and interactive activities between two persons in daily scenarios,” ArXiv, vol. abs/2405.11690, 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:269921862

    2024 arXiv

  17. [25]

    Regennet: Towards human action-reaction synthesis,

    L. Xu, Y . Zhou, Y . Yan, X. Jin, W. Zhu, F. Rao, X. Yang, and W. Zeng, “Regennet: Towards human action-reaction synthesis,” 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 1759–1769, 2024. [Online]. Available: https://api.semanticscholar.org/C...

    2024

  18. [26]

    Role-aware interaction generation from textual description,

    M. Tanaka and K. Fujiwara, “Role-aware interaction generation from textual description,” 2023 IEEE/CVF International Conference on Computer Vision (ICCV) , pp. 15 953–15 963, 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:267024730

    2023

  19. [27]

    Generating diverse and natural 3d human motions from text,

    C. Guo, S. Zou, X. Zuo, S. Wang, W. Ji, X. Li, and L. Cheng, “Generating diverse and natural 3d human motions from text,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 5152–5161

    2022

  20. [28]

    Dance revolution: Long-term dance generation with music via curriculum learning,

    R. Huang, H. Hu, W. Wu, K. Sawada, M. Zhang, and D. Jiang, “Dance revolution: Long-term dance generation with music via curriculum learning,” arXiv preprint arXiv:2006.06119 , 2020

    2006 arXiv

  21. [29]

    Synthesis of compositional animations from textual descriptions,

    A. Ghosh, N. Cheema, C. Oguz, C. Theobalt, and P. Slusallek, “Synthesis of compositional animations from textual descriptions,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 1396–1406

    2021

  22. [30]

    Learning a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networks,

    M. Plappert, C. Mandery, and T. Asfour, “Learning a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networks,” Robotics and Autonomous Systems , vol. 109, pp. 13–26, 2018

    2018

  23. [31]

    Generative adversarial nets,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial nets,” Advances in neural information processing systems , vol. 27, 2014

    2014

  24. [32]

    Auto-encoding variational bayes,

    R. Lopez, P. Boyeau, N. Yosef, M. I. Jordan, and J. Regier, “Auto-encoding variational bayes,” 2020. [Online]. Available: https: //api.semanticscholar.org/CorpusID:211146177

    2020

  25. [33]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” ArXiv, vol. abs/2006.11239, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:219955663

    2006 arXiv

  26. [34]

    Temos: Generating diverse human motions from textual descriptions,

    M. Petrovich, M. J. Black, and G. Varol, “Temos: Generating diverse human motions from textual descriptions,” in European Conference on Computer Vision. Springer, 2022, pp. 480–497

    2022

  27. [35]

    Executing your commands via motion diffusion in latent space,

    X. Chen, B. Jiang, W. Liu, Z. Huang, B. Fu, T. Chen, and G. Yu, “Executing your commands via motion diffusion in latent space,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 18 000–18 010

    2023

  28. [36]

    Momask: Generative masked modeling of 3d human motions,

    C. Guo, Y . Mu, M. G. Javed, S. Wang, and L. Cheng, “Momask: Generative masked modeling of 3d human motions,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 1900–1910

    2024

  29. [37]

    Robust vector quantized-variational autoencoder,

    C.-H. Lai, D. Zou, and G. Lerman, “Robust vector quantized-variational autoencoder,” ArXiv, vol. abs/2202.01987, 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID:246608006

    2022 arXiv

  30. [38]

    Physdiff: Physics- guided human motion diffusion model,

    Y . Yuan, J. Song, U. Iqbal, A. Vahdat, and J. Kautz, “Physdiff: Physics- guided human motion diffusion model,” inProceedings of the IEEE/CVF international conference on computer vision , 2023, pp. 16 010–16 021

    2023

  31. [39]

    Diffusionphase: Motion diffusion in frequency domain,

    W. Wan, Y . Huang, S. Wu, T. Komura, W. Wang, D. Jayaraman, and L. Liu, “Diffusionphase: Motion diffusion in frequency domain,” arXiv preprint arXiv:2312.04036, 2023

    2023 arXiv

  32. [40]

    Digital life project: Autonomous 3d characters with social intelligence,

    Z. Cai, J. Jiang, Z. Qing, X. Guo, M. Zhang, Z. Lin, H. Mei, C. Wei, R. Wang, W. Yin et al., “Digital life project: Autonomous 3d characters with social intelligence,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 582–592

    2024

  33. [41]

    Sitcom-crafter: A plot-driven human motion generation system in 3d scenes,

    J. Chen, P. Hu, X. Chang, Z. Shi, M. C. Kampffmeyer, and X. Liang, “Sitcom-crafter: A plot-driven human motion generation system in 3d scenes,” ArXiv, vol. abs/2410.10790, 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:273345513

    2024 arXiv

  34. [42]

    Amass: Archive of motion capture as surface shapes,

    N. Mahmood, N. Ghorbani, N. F. Troje, G. Pons-Moll, and M. J. Black, “Amass: Archive of motion capture as surface shapes,” in Proceedings of the IEEE/CVF international conference on computer vision , 2019, pp. 5442–5451

    2019

  35. [43]

    Residual force control for agile human be- havior imitation and extended motion synthesis,

    Y . Yuan and K. Kitani, “Residual force control for agile human be- havior imitation and extended motion synthesis,” Advances in Neural Information Processing Systems , vol. 33, pp. 21 763–21 774, 2020

    2020

  36. [44]

    Ase: Large- scale reusable adversarial skill embeddings for physically simulated characters,

    X. B. Peng, Y . Guo, L. Halper, S. Levine, and S. Fidler, “Ase: Large- scale reusable adversarial skill embeddings for physically simulated characters,” ACM Transactions On Graphics (TOG) , vol. 41, no. 4, pp. 1–17, 2022

    2022

  37. [45]

    Amp: Adversarial motion priors for stylized physics-based character control,

    X. B. Peng, Z. Ma, P. Abbeel, S. Levine, and A. Kanazawa, “Amp: Adversarial motion priors for stylized physics-based character control,” ACM Transactions on Graphics (ToG) , vol. 40, no. 4, pp. 1–20, 2021

    2021

  38. [46]

    Universal humanoid motion representations for physics-based control,

    Z. Luo, J. Cao, J. Merel, A. Winkler, J. Huang, K. Kitani, and W. Xu, “Universal humanoid motion representations for physics-based control,” arXiv preprint arXiv:2310.04582 , 2023

    2023 arXiv

  39. [47]

    Perpetual humanoid control for real-time simulated avatars,

    Z. Luo, J. Cao, K. Kitani, W. Xu et al. , “Perpetual humanoid control for real-time simulated avatars,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 10 895–10 904

    2023

  40. [48]

    Physhoi: Physics-based imitation of dynamic human-object interaction,

    Y . Wang, J. Lin, A. Zeng, Z. Luo, J. Zhang, and L. Zhang, “Physhoi: Physics-based imitation of dynamic human-object interaction,” arXiv preprint arXiv:2312.04393, 2023

    2023 arXiv

  41. [49]

    Anyskill: Learning open-vocabulary physical skill for interactive agents,

    J. Cui, T. Liu, N. Liu, Y . Yang, Y . Zhu, and S. Huang, “Anyskill: Learning open-vocabulary physical skill for interactive agents,” in Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 852–862

    2024

  42. [50]

    Learning physically simulated tennis skills from broadcast videos,

    Y . YUAN and V . Makoviychuk, “Learning physically simulated tennis skills from broadcast videos,” 2023

    2023

  43. [51]

    Simpoe: Simu- lated character control for 3d human pose estimation,

    Y . Yuan, S.-E. Wei, T. Simon, K. Kitani, and J. Saragih, “Simpoe: Simu- lated character control for 3d human pose estimation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 7159–7169

    2021

  44. [52]

    Embodied scene-aware human pose estimation,

    Z. Luo, S. Iwase, Y . Yuan, and K. Kitani, “Embodied scene-aware human pose estimation,” Advances in Neural Information Processing Systems , vol. 35, pp. 6815–6828, 2022

    2022

  45. [53]

    Multiphys: Multi-person physics-aware 3d motion estimation,

    N. Ugrinovic, B. Pan, G. Pavlakos, D. Paschalidou, B. Shen, J. Sanchez- Riera, F. Moreno-Noguer, and L. Guibas, “Multiphys: Multi-person physics-aware 3d motion estimation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 2331–2340

    2024

  46. [54]

    Ego-pose estimation and forecasting as real-time pd control,

    Y . Yuan and K. Kitani, “Ego-pose estimation and forecasting as real-time pd control,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2019, pp. 10 082–10 092

    2019

  47. [55]

    Emage: Towards unified holis- tic co-speech gesture generation via masked audio gesture modeling,

    H. Liu, Z. Zhu, G. Becherini, Y . Peng, M. Su, Y . Zhou, X. Zhe, N. Iwamoto, B. Zheng, and M. J. Black, “Emage: Towards unified holis- tic co-speech gesture generation via masked audio gesture modeling,” arXiv e-prints , pp. arXiv–2401, 2023

    2023

  48. [56]

    Generating holistic 3d human motion from speech,

    H. Yi, H. Liang, Y . Liu, Q. Cao, Y . Wen, T. Bolkart, D. Tao, and M. J. Black, “Generating holistic 3d human motion from speech,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 469–480

    2023

  49. [57]

    Cocogesture: Toward coherent co-speech 3d gesture generation in the wild,

    X. Qi, H. Zhang, Y . Wang, J. Pan, C. Liu, P. Li, X. Chi, M. Li, Q. Zhang, W. Xue et al. , “Cocogesture: Toward coherent co-speech 3d gesture generation in the wild,” arXiv preprint arXiv:2405.16874 , 2024

    2024 arXiv

  50. [58]

    Smpl: A skinned multi-person linear model,

    M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black, “Smpl: A skinned multi-person linear model,” ACM transactions on graphics (TOG), vol. 34, no. 6, pp. 1–16, 2015

    2015

  51. [59]

    Expressive body capture: 3d hands, face, and body from a single image,

    G. Pavlakos, V . Choutas, N. Ghorbani, T. Bolkart, A. A. Osman, D. Tzionas, and M. J. Black, “Expressive body capture: 3d hands, face, and body from a single image,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 10 975–10 985....

    2019

  52. [60]

    Deepmimic: Example-guided deep reinforcement learning of physics-based character skills,

    X. B. Peng, P. Abbeel, S. Levine, and M. Van de Panne, “Deepmimic: Example-guided deep reinforcement learning of physics-based character skills,” ACM Transactions On Graphics (TOG) , vol. 37, no. 4, pp. 1–14, 2018

    2018

  53. [61]

    Deep whole-body control: learning a unified policy for manipulation and locomotion,

    Z. Fu, X. Cheng, and D. Pathak, “Deep whole-body control: learning a unified policy for manipulation and locomotion,” in Conference on Robot Learning . PMLR, 2023, pp. 138–149

    2023

  54. [62]

    Keep it smpl: Automatic estimation of 3d human pose and shape from a single image,

    F. Bogo, A. Kanazawa, C. Lassner, P. Gehler, J. Romero, and M. J. Black, “Keep it smpl: Automatic estimation of 3d human pose and shape from a single image,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, P...

    2016

  55. [63]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al. , “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PMLR, 2021, pp. 8748–8763

    2021

  56. [64]

    Refit: Recurrent fitting network for 3d human recovery,

    Y . Wang and K. Daniilidis, “Refit: Recurrent fitting network for 3d human recovery,” in Proceedings of the IEEE/CVF International Con- ference on Computer Vision , 2023, pp. 14 644–14 654

    2023

  57. [65]

    W-hmr: Human mesh recovery in world space with weak-supervised camera calibration and orientation correction,

    W. Yao, H. Zhang, Y . Sun, and J. Tang, “W-hmr: Human mesh recovery in world space with weak-supervised camera calibration and orientation correction,” arXiv preprint arXiv:2311.17460 , 2023

    2023 arXiv

  58. [66]

    Ai choreographer: Music conditioned 3d dance generation with aist++,

    R. Li, S. Yang, D. A. Ross, and A. Kanazawa, “Ai choreographer: Music conditioned 3d dance generation with aist++,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 13 401–13 412

    2021

  59. [67]

    Edge: Editable dance generation from music,

    J. Tseng, R. Castellon, and K. Liu, “Edge: Editable dance generation from music,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 448–458

    2023

  60. [68]

    Rethinking diffusion for text-driven human motion generation,

    Z. Meng, Y . Xie, X. Peng, Z. Han, and H. Jiang, “Rethinking diffusion for text-driven human motion generation,” arXiv preprint arXiv:2411.16575, 2024

    2024 arXiv

  61. [69]

    Beat-it: Beat-synchronized multi-condition 3d dance generation,

    C. Zheng, J. Qin, and S. He, “Beat-it: Beat-synchronized multi-condition 3d dance generation,” 2024

    2024

  62. [70]

    Action2motion: Conditioned generation of 3d human motions,

    C. Guo, X. Zuo, S. Wang, S. Zou, Q. Sun, A. Deng, M. Gong, and L. Cheng, “Action2motion: Conditioned generation of 3d human motions,” in Proceedings of the 28th ACM International Conference on Multimedia, 2020, pp. 2021–2029

    2020

  63. [71]

    Dancing to music,

    H.-Y . Lee, X. Yang, M.-Y . Liu, T.-C. Wang, Y .-D. Lu, M.-H. Yang, and J. Kautz, “Dancing to music,”Advances in neural information processing systems, vol. 32, 2019. Wei Yao received the B.E. degree from the Uni- versity of South China, Hengyang, China, in 2021. He is now a P...

    2019

  64. [2023]

    Available: https://api.semanticscholar.org/CorpusID: 259262201

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 259262201

  65. [2024]

    Available: https://api.semanticscholar.org/CorpusID: 268364256

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 268364256

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.