Pith. sign in

REVIEW 3 major objections 5 minor 35 references

Grounded Gesture Generation: Language, Motion, and Space

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces a unified multimodal dataset of over 7.7 hours of synchronized motion, speech, and 3D scene data for spatially grounded referential gestures, and shows that fine-tuning a diffusion-based motion generation model on it…

desk verdict A useful standardization effort and a plausible resource, but the synthetic speech-motion pairing is unvalidated, so the '7.7 hours synchronized' claim is stronger than the evidence supports. read the letter →

arxiv 2507.04522 v1 pith:JD3C3PFH submitted 2025-07-06 cs.CV cs.AIcs.RO

Anna Deichler , Jim O'Regan , Teo Guichoux , David Johansson , Jonas Beskow This is my paper
classification cs.CVcs.AIcs.RO
keywords groundedgesturegenerationreferentialgesturesmultimodaldatasetHumanML3Dspatialgroundingco-speechVRdialoguemotion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that gesture generation for embodied agents needs to be spatially grounded, and that the missing piece has been data that ties motion, speech, and 3D scenes together in one format. To that end it introduces a combined corpus: a synthetic set of pointing gestures paired with generated demonstrative utterances, and MM-Conv, a VR-based corpus of two-party dialogues, standardized into the HumanML3D pose representation. The central empirical claim is that fine-tuning a diffusion-based motion generation model (OmniControl) on this task-specific data improves both motion naturalness (FID) and spatial control accuracy (Control L2) relative to the pretrained base model. If correct, the field gains a reusable resource and a strong baseline recipe for building virtual humans that point at and refer to objects while speaking.

What carries the argument

The load-bearing mechanism is the standardization of heterogeneous motion capture into the HumanML3D representation: each frame becomes a 263-dimensional vector containing root rotational and linear velocities, root height, rotation-invariant joint positions, joint rotation parameters and velocities, and binary foot-contact indicators. This common encoding lets a single diffusion-based generation model, OmniControl, be fine-tuned across the synthetic and VR datasets, with a spatial guidance loss that enforces that controlled joints—pelvis, left wrist, right wrist—match target positions in global coordinates during denoising. Synthetic speech is matched to pointing motions using the Hungarian algorithm (with duration padding), and word-level timestamps from WhisperX provide the temporal reference used to filter and align utterances.

What would settle it

Take a random sample of the 1,406 matched synthetic pairs and measure the temporal gap between the WhisperX word-level timestamp of the demonstrative ('this'/'that') and the peak pointing frame of the matched motion (e.g., maximum wrist extension toward the referent); if the median gap is large or human raters find the pointed object inconsistent with the mentioned object, the claimed synchronization does not hold.

Watch

Extended reading notes

Core claim

The central discovery is that two previously separate research threads—co-speech gesture synthesis and scene-, text-conditioned motion generation—can be joined through a common motion representation. The paper constructs a unified dataset by converting a synthetic pointing-gesture corpus and the VR dialogue dataset MM-Conv into the HumanML3D format, a 263-dimensional per-frame pose encoding derived from SMPL-X, supplemented with 3D scene information and object-reference annotations; the combined corpus totals over 7.7 hours of synchronized motion, speech, and scene data. The paper then claims that fine-tuning OmniControl, a diffusion-based motion generation model, on this corpus consistently lowers FID and Control L2 error across three training splits (synthetic only, synthetic plus referential, and the full set) and three control joints (pelvis, left wrist, right wrist), with the synthetic-only split giving the best wrist control and the addition of referential VR data improving pelvis control.

Load-bearing premise

The dataset is only 'synchronized' if the Hungarian-algorithm matching of generated TTS utterances to pointing motions, including padding to equalize durations, preserves genuine semantic and temporal gesture-speech coherence; if that matching is coarse or the padding distorts the relationship, the resource's central claim fails.

Editorial extensions

If this is right

  • Researchers gain a ready-to-use corpus that removes the format mismatch between BVH-based gesture datasets and HumanML3D-based motion models.
  • Fine-tuning on task-specific gesture data is shown to be a reliable way to improve a general motion model's naturalness and spatial control, so other labs can adopt the same recipe for their own data.
  • The synthetic-only training split achieves the lowest wrist errors, suggesting that focused, high-quality synthetic pointing data can outperform broader mixed data for deictic control.
  • Adding referential VR conversations improves pelvis control, indicating that conversational context contributes to better whole-body coordination.
  • The framework's modular perception-to-generation pipeline means spatial cues derived from a vision-language model or 3D simulator can drive gesture generation without retraining the motion generator.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: if alignment quality withstands closer scrutiny, this corpus could become a standard benchmark for grounded referential gesture generation, in the way that HumanML3D became a benchmark for text-to-motion.
  • My inference: the paper's own concession that position-based spatial loss is not the right objective for pointing suggests a natural follow-up—redefining the spatial loss as directional alignment (elbow-to-wrist vector toward the referent) could produce substantially better communicative pointing, and the dataset is built to support such training.
  • My inference: because the synthetic utterances are machine-generated by a VLM with exophoric demonstratives, the corpus could double as a testbed for embodied reference resolution and demonstrative generation, not just motion synthesis.
  • My inference: the duration-based filtering (0.5–20 seconds) and Hungarian matching may bias the corpus toward short, single-point episodes, so generalization to long, multi-reference deictic dialogues remains an open test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents a multimodal dataset and framework for grounded gesture generation, combining a synthetic pointing-gesture dataset with TTS-generated demonstrative utterances and the VR-based MM-Conv dialogue dataset, both converted to HumanML3D format. The authors report 7.66 hours of motion data with accompanying speech and scene annotations, define three training splits, and fine-tune OmniControl to show improved FID and Control L2 metrics after task-specific adaptation. The central claims are that the combined corpus supports spatially grounded, referential gesture research and that fine-tuning on it improves motion quality and control.

Significance. If the synchronization of the synthetic subset is validated, this is a useful standardized resource: it addresses a real gap, provides reference annotations and scene information in HumanML3D format, and offers a modular simulation framework. The authors ship a converted version of MM-Conv with resolved object references and a proof-of-concept fine-tuning of a controllable diffusion model. The main contribution is empirical rather than theoretical; the value will hinge on dataset release and on the credibility of the synthetic alignment. The fine-tuning results are a promising demonstration but would benefit from stronger statistical reporting.

major comments (3)
  1. [3.2] The synthetic pointing dataset's 'synchronized' claim is not substantiated. The Hungarian-algorithm matching is described only as 'based on the demonstrative location,' with no definition of the location representation, no cost function, and no validation that matched speech and motion are semantically or temporally aligned. The WhisperX WER threshold (0.3) filters ASR errors, not gesture-speech alignment, and duration padding cannot create co-speech timing for motions originally captured without speech. Because roughly 1.53 of the 7.66 hours (the ST split) rests on this pipeline, the abstract's claim of 'over 7.7 hours of synchronized motion, speech, and 3D scene information' is not yet supported. The authors should provide either (a) quantitative alignment validation, e.g., the distribution of times between the deictic word onset and the pointing stroke peak, or (b) a precisely qualified claim that only the MM-Conv portion is genuinely synchronized while the synthetic portion offers task-specific loose pairing.
  2. [4.3] Table 5 reports FID and Control L2 for base vs. fine-tuned models across three splits, but all numbers are point estimates from a single run. FID estimates are notoriously noisy on small evaluation sets, and the paper does not report the number of evaluation samples, seeds, or confidence intervals. The claim that 'fine-tuning consistently improves both metrics' needs at least three seeds with mean±std or a significance test before it can be considered load-bearing evidence for the benefit of task-adaptive training.
  3. [3.2] The count of audio files vs. matched motions is unclear: 1406 audio files are said to be matched with 1135 pointing motions, but the synthetic ST split contains 1135 clips. If only 1135 audio-motion pairs result, the fate of the remaining 271 files (or 271 motions if audio files are matched multiple times) should be stated. Additionally, the matching cost combines 'demonstrative location' and duration, but the two modalities come from different capture sessions and different scene coordinate systems; how locations are made comparable is not explained.
minor comments (5)
  1. [3.4] The text says 'over 7.7 hours' but Table 4 gives 459.86 minutes, which is 7.66 hours; 'approximately 7.7 hours' would be more accurate.
  2. [Table 1] Table 1 lists MM-Conv duration as 6.5 hours, while Section 3.3 reports 6.14 hours after filtering; please clarify which value is raw and which is post-filter.
  3. [Figure 2] Figure 2 shows distributions for 'Synthetic Audio' and 'Original Motion' but the figure legend is minimal; consider labeling whether the audio durations are before or after padding and add a panel showing the aligned pairing of durations.
  4. [4.2] The 'Control L2 error' metric is not explicitly defined; it should be tied to the spatial loss in Eq. (1) and described as the average Euclidean distance between predicted and target joint positions over controlled joints and frames.
  5. [3.1] The notation for the HumanML3D vector components (ωr, vr, yr, p, R, Ṙ, c) appears with formatting artifacts in the PDF; please ensure the subscript and dimension descriptions are legible.

Circularity Check

0 steps flagged · score 1.0 of 10

No material circularity: the dataset construction and fine-tuning results are empirical, and self-citations are provenance for prior resources, not load-bearing derivations.

full rationale

The paper's central contributions are a standardized multimodal dataset and a fine-tuning study, not a first-principles derivation. The synthetic pointing subset follows the authors' earlier pipeline from [5] and [8], and MM-Conv is the authors' own dataset [7]; however, these are cited as existing resources that are then converted to SMPL-X/HumanML3D format via the independent preprocessing described in Section 3.1. The 7.7-hour total is arithmetic over the reported clip statistics in Table 4, and the fine-tuning results in Table 5 are empirical FID and Control L2 measurements on a fixed pretrained model. No fitted parameter is renamed as a prediction, and no claimed result reduces by construction to its own input. The most fragile step is the Hungarian-algorithm matching of synthesized TTS utterances to pre-recorded pointing motions in Section 3.2, where 'synchronized' may rest on an unvalidated operationalization of alignment; but that is a validity or correctness concern, not circularity, and the paper itself concedes its evaluation 'does not fully reflect the desired behavior in pointing tasks' (Section 4.3). The self-citations are substantial but are citations to prior datasets and methods, not circular support for the present claim. Accordingly, no specific circular step can be exhibited from the paper's own equations or definitions.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper's central claim is a dataset resource, so the ledger contains no invented entities or fitted physical constants. The load-bearing choices are data-construction assumptions: suitability of the HumanML3D representation for gestures, validity of the Hungarian speech-motion alignment, validity of FID/Control L2 as gesture metrics, and correctness of the MM-Conv reference annotations from [2].

free parameters (3)
  • WER filter threshold = 0.3
    Section 3.2 discards 94 synthetic speech samples whose WhisperX transcription deviates from the original text by WER > 0.3; this threshold determines which clips enter the dataset.
  • Motion segment duration bounds = 0.5 to 20 seconds
    Section 3.3 filters referential and non-referential segments to this window, following 'filtering practices in prior motion-language datasets' but the exact bounds are hand-chosen and affect the final statistics.
  • Hungarian matching distance criterion = unspecified
    Section 3.2 states utterances are matched to pointing motions 'based on the demonstrative location', but the metric (e.g., Euclidean distance between target and object centroid) is not defined, so the pairing is not reproducible.
assumptions (4)
  • domain assumption HumanML3D representation is a suitable common format for both locomotion and co-speech gesture data.
    Section 3.1 converts all motion to the 263-dimensional HumanML3D representation without validating that this representation preserves deictic gesture information (e.g., wrist orientation relative to referent).
  • ad hoc to paper The Hungarian alignment yields temporally and semantically synchronized speech-motion pairs.
    Section 3.2 aligns independently generated TTS audio with pre-recorded pointing motions; the alignment is a heuristic and no synchrony metric (e.g., gesture onset vs. demonstrative mention) is reported.
  • domain assumption FID and Control L2 are valid metrics for gesture naturalness and spatial control.
    Section 4.3 uses the HumanML3D evaluation protocol without re-validating it for communicative gestures; FID is distributional and may not capture gesture-speech coherence.
  • domain assumption MM-Conv reference annotations from the anonymous under-review work [2] are accurate.
    Section 3.3 segments motion into referential and non-referential clips based on these annotations; errors directly propagate to all dataset statistics and training splits.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Grounded Gesture Generation: Language, Motion, and Space." pith.science (2026). https://pith.science/paper/JD3C3PFH

@misc{pith2026250704522,
  author       = {Pith},
  title        = {Pith review of: Grounded Gesture Generation: Language, Motion, and Space},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JD3C3PFH}},
  note         = {Machine review of arXiv:2507.04522}
}
read the original abstract

Human motion generation has advanced rapidly in recent years, yet the critical problem of creating spatially grounded, context-aware gestures has been largely overlooked. Existing models typically specialize either in descriptive motion generation, such as locomotion and object interaction, or in isolated co-speech gesture synthesis aligned with utterance semantics. However, both lines of work often treat motion and environmental grounding separately, limiting advances toward embodied, communicative agents. To address this gap, our work introduces a multimodal dataset and framework for grounded gesture generation, combining two key resources: (1) a synthetic dataset of spatially grounded referential gestures, and (2) MM-Conv, a VR-based dataset capturing two-party dialogues. Together, they provide over 7.7 hours of synchronized motion, speech, and 3D scene information, standardized in the HumanML3D format. Our framework further connects to a physics-based simulator, enabling synthetic data generation and situated evaluation. By bridging gesture modeling and spatial grounding, our contribution establishes a foundation for advancing research in situated gesture generation and grounded multimodal interaction. Project page: https://groundedgestures.github.io/

Figures

Figures reproduced from arXiv: 2507.04522 by the authors.

Figure 1
Figure 1. Ground truth bounding boxes of objects in current view, [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 3
Figure 3. Rendered SMPL-X frames from the VR-based referential dataset (MM-Conv). [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 5
Figure 5. Examples of rendered data sample of SMPL-X character [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figures from the paper (2 more)
Figure 6
Figure 6. Figure 6: Proposed framework. 4.1. Motion generation with spatial conditioning OmniControl [31] is a diffusion-based motion genera￾tion model trained on the HumanML3D dataset, which en￾ables precise control over any joint in time, guided by text prompts and spatial constraints. …
Figure 7
Figure 7. Figure 7: Rendered aligned synthetic pointing gesture examples in third and first person views. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 28 canonical work pages

  1. [1]

    GestureDiffu- CLIP: Gesture diffusion model with CLIP latents

    Tenglong Ao, Zeyi Zhang, and Libin Liu. GestureDiffu- CLIP: Gesture diffusion model with CLIP latents. ACM Transactions on Graphics (TOG), 42(4):1–18, 2023. 3

    work page 2023

  2. [2]

    Multimodal referencing: Grounding generative models in situated communication, 2025

    Anonymous Authors. Multimodal referencing: Grounding generative models in situated communication, 2025. Under review. 5

    work page 2025

  3. [3]

    WhisperX: Time-accurate speech transcription of long- form audio

    Max Bain, Jaesung Huh, Tengda Han, and Andrew Zisser- man. WhisperX: Time-accurate speech transcription of long- form audio. INTERSPEECH 2023, 2023. 4

    work page 2023

  4. [4]

    F5-TTS: A fairytaler that fakes fluent and faithful speech with flow matching

    Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chun- hui Wang, Jian Zhao, Kai Yu, and Xie Chen. F5-TTS: A fairytaler that fakes fluent and faithful speech with flow matching. arXiv preprint arXiv:2410.06885, 2024. 4

    arXiv 2024

  5. [5]

    In- corporating spatial awareness in data-driven gesture gener- ation for virtual agents

    Anna Deichler, Simon Alexanderson, and Jonas Beskow. In- corporating spatial awareness in data-driven gesture gener- ation for virtual agents. In Proceedings of the ACM Inter- national Conference on Intelligent Virtual Agents (IVA ’24), pages 1–4, Glasgow, United Kingdom, 2024. ACM. 4

    work page 2024

  6. [6]

    Diffusion-based co-speech gesture genera- tion using joint text and audio representation

    Anna Deichler, Shivam Mehta, Simon Alexanderson, and Jonas Beskow. Diffusion-based co-speech gesture genera- tion using joint text and audio representation. InProceedings of the 25th International Conference on Multimodal Interac- tion, pages 755–762, 2023. 3

    work page 2023

  7. [7]

    MM-Conv: A multi-modal conversational dataset for virtual humans

    Anna Deichler, Jim O’Regan, and Jonas Beskow. MM-Conv: A multi-modal conversational dataset for virtual humans. arXiv preprint arXiv:2410.00253, 2024. 3, 4

    arXiv 2024

  8. [8]

    Towards context-aware human-like point- ing gestures with rl motion imitation

    Anna Deichler, Siyang Wang, Simon Alexanderson, and Jonas Beskow. Towards context-aware human-like point- ing gestures with rl motion imitation. In Context-Awareness in Human-Robot Interaction: Approaches and Challenges, workshop at 2022 ACM/IEEE International Conference on Human-Robot Interaction, 2022. 4

    work page 2022

Show all 35 references
  1. [9]

    Investigating the use of recurrent motion modelling for speech gesture generation

    Ylva Ferstl and Rachel McDonnell. Investigating the use of recurrent motion modelling for speech gesture generation. In Proceedings of the 18th International Conference on Intelli- gent Virtual Agents, pages 93–98, 2018. 3

    2018

  2. [10]

    IV A: Investigating the use of recurrent motion modelling for speech gesture gen- eration

    Ylva Ferstl and Rachel McDonnell. IV A: Investigating the use of recurrent motion modelling for speech gesture gen- eration. In IVA ’18 Proceedings of the 18th International Conference on Intelligent Virtual Agents, Nov 2018. 3

    2018

  3. [11]

    Learning individual styles of conversational gesture

    Shiry Ginosar, Amir Bar, Gefen Kohavi, Caroline Chan, An- drew Owens, and Jitendra Malik. Learning individual styles of conversational gesture. CoRR, abs/1906.04160, 2019. 3

    1906 arXiv

  4. [12]

    Generating diverse and natural 3d human motions from text

    Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In CVPR 2022, pages 5152–5161, June 2022. 3, 7

    2022

  5. [13]

    Generating diverse and nat- ural 3D human motions from text

    Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and nat- ural 3D human motions from text. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5152–5161, 2022. 2, 3, 5

    2022

  6. [14]

    Automatic deictic ges- tures for animated pedagogical agents

    Sri Rama Kartheek Kappagantula, Nicoletta Adamo-Villani, Meng-Lin Wu, and V oicu Popescu. Automatic deictic ges- tures for animated pedagogical agents. IEEE Transactions on Learning Technologies, 13(1):1–13, 2019. 3

    2019

  7. [15]

    AI2-THOR: An interactive 3D environment for visual AI

    Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli Vander- Bilt, Luca Weihs, Alvaro Herrasti, Matt Deitke, Kiana Ehsani, Daniel Gordon, Yuke Zhu, et al. AI2-THOR: An interactive 3D environment for visual AI. arXiv preprint arXiv:1712.05474, 2017. 4, 6

    2017 arXiv

  8. [16]

    Talking With Hands 16.2M: A large-scale dataset of synchronized body- finger motion and audio for conversational motion analy- sis and synthesis

    Gilwoo Lee, Zhiwei Deng, Shugao Ma, Takaaki Shiratori, Siddhartha S Srinivasa, and Yaser Sheikh. Talking With Hands 16.2M: A large-scale dataset of synchronized body- finger motion and audio for conversational motion analy- sis and synthesis. In Proceedings of the IEEE/CVF Int...

  9. [17]

    Deictic believability: Coordinated ges- ture, locomotion, and speech in lifelike pedagogical agents

    James C Lester, Jennifer L V oerman, Stuart G Towns, and Charles B Callaway. Deictic believability: Coordinated ges- ture, locomotion, and speech in lifelike pedagogical agents. Applied Artificial Intelligence, 13(4-5):383–414, 1999. 3

    1999

  10. [18]

    Controllable human-object interaction synthesis

    Jiaman Li, Alexander Clegg, Roozbeh Mottaghi, Jiajun Wu, Xavier Puig, and C Karen Liu. Controllable human-object interaction synthesis. In European Conference on Computer Vision, pages 54–72. Springer, 2024. 7

    2024

  11. [19]

    Black, Hao Li, and Javier Romero

    Tianye Li, Timo Bolkart, Michael J. Black, Hao Li, and Javier Romero. Learning a model of facial shape and ex- pression from 4d scans. ACM Trans. Graph. , 36(6), Nov

  12. [20]

    GroundingGPT: Language enhanced multi-modal grounding model

    Zhaowei Li, Qi Xu, Dong Zhang, Hang Song, Yiqing Cai, Qi Qi, Ran Zhou, Junting Pan, Zefeng Li, Van Tu Vu, et al. GroundingGPT: Language enhanced multi-modal grounding model. arXiv preprint arXiv:2401.06071, 2024. 4

    2024 arXiv

  13. [21]

    EMAGE: Towards unified holistic co-speech gesture generation via expressive masked audio gesture modeling

    Haiyang Liu, Zihao Zhu, Giorgio Becherini, Yichen Peng, Mingyang Su, You Zhou, Xuefei Zhe, Naoya Iwamoto, Bo Zheng, and Michael J Black. EMAGE: Towards unified holistic co-speech gesture generation via expressive masked audio gesture modeling. In Proceedings of the IEEE/CVF Co...

    2024

  14. [22]

    BEAT: A large-scale semantic and emotional multi-modal dataset for conversational gestures synthesis

    Haiyang Liu, Zihao Zhu, Naoya Iwamoto, Yichen Peng, Zhengqing Li, You Zhou, Elif Bozkurt, and Bo Zheng. BEAT: A large-scale semantic and emotional multi-modal dataset for conversational gestures synthesis. arXiv preprint arXiv:2203.05297, 2022. 3

    2022 arXiv

  15. [23]

    Matthew Loper, Naureen Mahmood, and Michael J. Black. MoSh: Motion and Shape capture from sparse markers. ACM Trans. Graph., 33(6), Nov. 2014. 3

    2014

  16. [24]

    AMASS: Archive of motion capture as surface shapes

    Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Ger- ard Pons-Moll, and Michael J Black. AMASS: Archive of motion capture as surface shapes. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5442–5451, 2019. 2

    2019

  17. [25]

    Design of a virtual human presenter

    Tsukasa Noma, Liwei Zhao, and Norman I Badler. Design of a virtual human presenter. IEEE Computer Graphics and Applications, 20(4):79–85, 2002. 3

    2002

  18. [26]

    Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3D hands, face, and body from a single image. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pa...

    2019

  19. [27]

    The KIT Motion-Language Dataset

    Matthias Plappert, Christian Mandery, and Tamim Asfour. The KIT Motion-Language Dataset. Big Data , 4(4):236– 252, Dec. 2016. 3

    2016

  20. [28]

    Animated agents for pro- cedural training in virtual reality: Perception, cognition, and motor control

    Jeff Rickel and W Lewis Johnson. Animated agents for pro- cedural training in virtual reality: Perception, cognition, and motor control. Applied artificial intelligence , 13(4-5):343– 382, 1999. 3

    1999

  21. [29]

    Move as you say interact as you can: Language-guided human motion generation with scene af- fordance

    Zan Wang, Yixin Chen, Baoxiong Jia, Puhao Li, Jinlu Zhang, Jingze Zhang, Tengyu Liu, Yixin Zhu, Wei Liang, and Siyuan Huang. Move as you say interact as you can: Language-guided human motion generation with scene af- fordance. In Proceedings of the IEEE/CVF Conference on Compu...

  22. [30]

    HUMANISE: Language-conditioned hu- man motion generation in 3D scenes

    Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, Wei Liang, and Siyuan Huang. HUMANISE: Language-conditioned hu- man motion generation in 3D scenes. In Advances in Neural Information Processing Systems (NeurIPS), 2022. 2, 3

    2022

  23. [31]

    OmniControl: Control any joint at any time for human motion generation

    Yiming Xie, Varun Jampani, Lei Zhong, Deqing Sun, and Huaizu Jiang. OmniControl: Control any joint at any time for human motion generation. In The Twelfth International Conference on Learning Representations, 2024. 2, 4, 7

    2024

  24. [32]

    Inter-X: Towards versatile human- human interaction analysis

    Liang Xu, Xintao Lv, Yichao Yan, Xin Jin, Shuwen Wu, Congsheng Xu, Yifan Liu, Yizhou Zhou, Fengyun Rao, Xingdong Sheng, et al. Inter-X: Towards versatile human- human interaction analysis. In CVPR, pages 22260–22271,

  25. [33]

    Hongwei Yi, Hualin Liang, Yifei Liu, Qiong Cao, Yandong Wen, Timo Bolkart, Dacheng Tao, and Michael J. Black. Generating holistic 3D human motion from speech. InCVPR 2023, pages 469–480, Los Alamitos, CA, USA, June 2023. IEEE Computer Society. 3

    2023

  26. [34]

    Generating human interaction motions in scenes with text control

    Hongwei Yi, Justus Thies, Michael J Black, Xue Bin Peng, and Davis Rempe. Generating human interaction motions in scenes with text control. In European Conference on Com- puter Vision, pages 246–263. Springer, 2024. 2

    2024

  27. [35]

    See that bench over THERE, we could sit there

    Appendix 6.1. Exophoric Demonstrative Generation Prompt The following prompt was used to generate exophoric referring expressions involving demonstratives (‘this’ or ‘that’) in a conversational style, using GPT-4o: Listing 2. Prompt used for generating exophoric demonstrative ...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.