Pith. sign in

REVIEW 4 major objections 5 minor 47 references

CigTime: Corrective Instruction Generation Through Inverse Motion Editing

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read CigTime turns a source-target motion pair into the corrective instruction that transforms one into the other.

desk verdict New task formulation with a plausible method, but the synthetic data loop and self-scoring editor make the headline gains provisional until human-validated evaluation. read the letter →

arxiv 2412.05460 v1 pith:BCPES4MI submitted 2024-12-06 cs.CV

Qihang Fang , Chengcheng Tang , Bugra Tekin , Yanchao Yang This is my paper · ORCID
classification cs.CV
keywords correctiveinstructiongenerationmotioneditingtext-conditioneddiffusionlargelanguagemodelsVQ-VAEtokenizationmotion-languagemodelsportscoaching
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces CigTime, a motion-language model that takes two motion sequences, the user's current movement and a target movement, and writes the corrective instruction that would turn the first into the second. The authors' central claim is that this task is the inverse of text-conditioned motion editing, so they can generate unlimited training triplets by asking a pretrained motion editor to edit source motions according to randomly selected captions. A VQ-VAE tokenizer converts both motions into discrete tokens, and a large language model is fine-tuned on the template pairing source tokens, target tokens, and the instruction. On a motion-caption benchmark, CigTime beats large-language-model and motion-language baselines on both text metrics and reconstruction accuracy. If the claim holds, coaching feedback can be synthesized without expert annotation, which matters for sports training and motor rehabilitation.

What carries the argument

The central machinery is the inverse of the paper's editing equation: the target motion is defined as $x_O = m \odot x_L + (1-m) \odot x_I$, where $x_L$ is the motion generated by a pretrained text-conditioned diffusion editor from an instruction $L$ and $m$ is a mask selecting the edited body part. CigTime is trained to invert this map: a VQ-VAE tokenizer turns $x_I$ and $x_O$ into discrete token lists, the token lists are arranged into a prompt, and a large language model is fine-tuned with cross-entropy loss to emit $L$. An anchor loss $\lambda\|W-W_0\|_2^2$ keeps the newly learned motion-token embeddings from drifting away from the original vocabulary while still allowing them to adapt.

What would settle it

Take a set of source-target motion pairs from real coaching sessions and ask coaches whether the generated instructions describe the intended correction; if instructions that reconstruct well under the motion editor are frequently judged as describing a different action, the core claim fails. Concretely, a pair whose true correction is 'bend your knees on landing' that yields an instruction the editor reconstructs well but a coach reads as a different move would contradict the inverse-motion-editing assumption.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that corrective instruction generation is the inverse problem of text-conditioned motion editing, and that inverting an existing editor with synthetic triplets is enough to train a model to do it. Given any source motion and desired target motion, the trained model outputs a textual correction; when that text is fed back through the motion editor, it reproduces the target better than text from generic large language models or motion-captioning models. The paper reports a mean-per-joint position error of 0.13 and an FID of 1.44 on its benchmark, against best baseline values of 0.21 and 2.09, and reports that its advantage persists when the evaluation uses a different motion editor than the one that generated the training data.

Load-bearing premise

The load-bearing premise is that synthetic triplets made by a pretrained motion editor, with a randomly chosen caption standing in for expert feedback, are valid corrective instruction data, and that the same editor is a faithful oracle for scoring them.

Editorial extensions

If this is right

  • Corrective instruction data no longer has to be manually annotated: any source motion from a motion-caption dataset can be turned into training triplets through editing.
  • The generated text is actionable in the sense that feeding it back into the motion editor moves the source toward the target, so the same evaluation loop can be automated without human judges.
  • The method generalizes beyond the training editor: evaluated with a different motion-diffusion editor, the generated instructions still reconstruct targets better than baselines, suggesting the model captures motion-difference semantics rather than memorizing the data editor's behavior.
  • The trained generator can be plugged into pose-estimation-based coaching systems, as demonstrated in the paper's real-world example where motions extracted from monocular video are turned into corrective text.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the same pretrained motion editor is used to create the training triplets and to score reconstruction, the reported MPJPE and FID numbers partly measure how faithfully the model imitates that editor's caption-to-edit mapping, not how good the coaching is in absolute terms.
  • Generalization to a different motion-caption collection shows n-gram scores dropping sharply while reconstruction metrics stay similar; a plausible reading is that the model learns transferable motion-difference semantics but not dataset-specific vocabulary, and human evaluation would be needed to decide which property matters more.
  • A natural next test is to gather a small set of expert-written correction pairs: if the model can be fine-tuned on them or evaluated with human raters, one can separate the synthetic-editor bias from genuine coaching quality.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces CigTime, a method for generating corrective text instructions from a pair of source and target 3D human motions. The authors propose a data-collection pipeline that uses a pretrained motion diffusion model (MDM) as an editor: a randomly chosen HumanML3D action caption is used with MDM to edit the source motion, and the target is formed by mask-blending this edited motion with the source. The resulting (source, target, instruction) triplets are tokenized with a VQ-VAE and used to fine-tune Llama-3-8B. The model is evaluated with text-similarity metrics (BLEU, ROUGE, METEOR, CLIPScore) and with reconstruction accuracy, where the generated instruction is fed back into a motion editor and compared with the target via MPJPE and FID. Experiments on HumanML3D, KIT, and Fit3D, plus ablations over backbone and motion editor, are reported. The central claim is that CigTime substantially outperforms LLM and motion-language baselines on this new task.

Significance. If the claims are substantiated, the paper addresses a genuinely useful and understudied task: generating corrective coaching feedback from motion pairs. The idea of inverting motion editing to create training data is creative and could reduce annotation cost. The paper includes a real-world demonstration with WHAM-estimated motions, which is a good step toward practical coaching applications. It also attempts to probe generalization across datasets and motion editors. However, the current evidence is weakened by a partly circular evaluation loop, the use of non-expert synthetic corrective labels, and the absence of statistical uncertainty estimates. The ablation with PriorMDM (Table 3) is a constructive attempt to address editor circularity, and the extension to KIT shows some generalization, but the headline claim of large improvement is not yet convincingly established.

major comments (4)
  1. [§3.2, Eq. (5) and §4.2] The evaluation loop is closed: the target motion xO is constructed by MDM from a randomly chosen instruction L via Eq. (5), and the reconstruction metrics in §4.2 feed the generated instruction back into MDM and compare the result with xO. This measures whether the generated text is executable by MDM to reproduce a self-consistently defined target, not whether it is a good corrective instruction for a user. Table 3 partially mitigates the same-editor concern by using PriorMDM, but the instruction-quality metrics still compare against the randomly selected caption, and Figure 5 shows that semantically different instructions can produce nearly identical reconstructions. To support the central claim, an evaluation with a motion editor not used during data generation, and ideally human ratings on real source-target pairs, is needed.
  2. [§3.2 and §5 (limitation 1)] The corrective instruction label L is not an expert correction tied to a specific deficiency in xI; it is a randomly sampled HumanML3D action caption used to edit xI into xO. The paper's own limitation section admits that the dataset "lacks targeted feedback on form and dynamics that are specific to actions and sports." This raises the risk that the model learns to caption the target motion given the source, rather than to compare the two motions and identify what needs to change. An ablation that removes the source motion tokens, or swaps the source and target, would directly test whether source information is actually used; no such ablation is reported.
  3. [Tables 1 and 5] No error bars, standard deviations, or significance tests are reported, so the "large margin" claim in Table 1 is not shown to be statistically robust. On KIT (Table 5), the margins shrink substantially; e.g., BLEU is 0.14 for CigTime and 0.14 for Qwen-1.5-7B-LoRA, and ROUGE is 0.27 vs 0.25. The magnitude of the improvement is highly dataset-dependent, and the reader cannot tell whether the HumanML3D gains are within run-to-run variance. Reporting multiple seeds with standard deviations and appropriate significance tests is necessary to support the headline conclusion.
  4. [Appendix B.3, Figure 5] The paper's own qualitative examples show that different corrective instructions (e.g., "forehand table tennis" vs "throwing a frisbee") can lead to motions that are nearly identical to each other and to the target. This means the reconstruction metrics (MPJPE/FID) are insensitive to important semantic differences between instructions, so low MPJPE cannot be taken as evidence that the generated instruction is the correct one. The paper acknowledges this diversity but does not adjust its evaluation conclusions; a semantic or human-judgment evaluation is needed to establish instruction quality beyond n-gram overlap.
minor comments (5)
  1. [Global] There are several typographical inconsistencies: "METERO" appears in Tables 1 and 2 (should be "METEOR"), "Geasture" appears in Figure 5, and "kerne_size" appears in Table 7. The heading "Ablation study with different network structurer" contains a typo.
  2. [Abstract/introduction] The paper alternates between "corrective" and "correctional" instruction; please choose one consistent term throughout.
  3. [Appendix C.1] The description "training for 50000 epochs" for MDM is likely a typo; please clarify the actual training schedule.
  4. [Appendix E, Figure 6] The text refers to "the global response pdf" in the figure caption; this is an artifact and should be replaced with a proper reference to the figure.
  5. [References] Reference [30] is listed as "Meta. Llama3, 2024" without a version or URL; please provide a complete citation.

Circularity Check

2 steps flagged · score 6.0 of 10

The core training/evaluation loop is closed around MDM: targets are MDM edits of random captions and reconstruction is scored by re-feeding predicted text into the same MDM; PriorMDM and KIT/Fit3D checks partially break the loop.

  1. self definitional [Section 3.2, Eq. (5); Section 4.1 Datasets]
    "We utilize the corrective instruction, L, as the generation condition, c, to generate the corresponding corrective motion sequence, xL. We then calculate the target motion sequence, xO, by combining the source motion sequence, xI, and the corrective motion sequence, xL, xO = m ⊙ xL + (1− m) ⊙ xI."

    The supervised label L is not an independently annotated correction; it is the text condition fed into MDM to synthesize the target. Eq. (5) defines xO as a mask-blend of xI with MDM's output under L, so the 'ground-truth' corrective text is the editor's input by construction. Training T(xI, xO) to output L therefore teaches the inverse of MDM's editing function on synthetic pairs, and agreement with that L measures recovery of the generator's condition, not coaching quality. The paper's own limitation (Sec. 5) concedes the data 'lacks targeted feedback on form and dynamics that are specific to actions and sports.'

  2. other [Section 4.1, Evaluation Metrics (Reconstruction accuracy); Section 4.2, Table 1]
    "To evaluate the quality, we use the generated corrective instruction as an editing condition to modify the source motion sequences and obtain the generated target motion. We then compare this with the ground-truth target motion."

    The ground-truth target xO was created by Eq. (5) with MDM and the same instruction L. Using MDM again to turn the predicted instruction into a motion and measuring MPJPE/FID against that xO is a self-consistency check of the inverse mapping, not an external test of corrective validity. Table 1's Ground-Truth row reports 0.00 MPJPE and 0.00 FID, which is possible because the evaluator and the data generator are the same editor. The headline reconstruction numbers are therefore partly forced by construction. Table 3 partly mitigates this by switching to PriorMDM, where the Ground-Truth row is no longer zero, so the loop is not fully closed, but the primary Table 1 results remain within the closed loop.

full rationale

The paper's derivation chain is an inverse-editing loop. In Sec. 3.2, Eq. 5 defines the target xO as a mask-blend of the source xI with MDM's generation conditioned on a randomly selected caption L, so the 'corrective instruction' label is the editor's input, not an independent expert correction. The LLM is then trained to output L from (xI, xO), i.e., to invert MDM's editing map. In Sec. 4.1, reconstruction accuracy is measured by feeding the predicted instruction back into MDM and comparing with the MDM-defined target; the perfect Ground-Truth row in Table 1 shows how fully the evaluator is aligned with the generator. This makes the headline BLEU/ROUGE/METEOR and MPJPE/FID gains partly measures of self-consistency rather than corrective quality. The paper partially breaks the closed loop: Table 3 evaluates with PriorMDM, where ground truth no longer gives zero error, and Sec. B.1/B.2 test Fit3D and KIT, so the result is not fully tautological. The conclusion's first limitation concedes the dataset 'lacks targeted feedback on form and dynamics,' which is precisely the coaching content the task claims to generate. No load-bearing self-citation chain is present; the circularity is in the same-editor data generation and evaluation design. Score 6: partial circularity via the closed synthetic loop.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on four premises: MDM is a faithful motion editor, HumanML3D captions can stand in for corrective instructions, VQ-VAE tokens preserve the motion information the LLM needs, and reconstruction through the same editor measures instruction quality. No new physical entities are introduced. The tokenizer and LLM add hyperparameters but no externally fitted constants.

free parameters (3)
  • VQ-VAE codebook size K = 64
    Set in Appendix C tokenizer architecture; determines how coarsely motions are discretized before the LLM sees them.
  • Anchor loss weight lambda = not reported
    Introduced in Eq. 10 to prevent learned embeddings from drifting; its value is not given, so the sensitivity of the result to this choice is unknown.
  • MDM reverse diffusion steps = 50
    Appendix C.1 sets maximum reverse diffusion steps to 50; this affects how faithfully the editor turns an instruction into a target motion and therefore shapes the whole dataset.
assumptions (4)
  • domain assumption MDM produces plausible target motions when conditioned on a text instruction.
    Section 3.2 and Eq. 5 generate every target motion in the dataset through MDM; if MDM is biased or broken, the corrective instruction signal is corrupted.
  • ad hoc to paper A HumanML3D action caption is a valid corrective instruction.
    Section 4.1 randomly selects one instruction from the HumanML3D split as the editing condition; these are neutral action descriptions, and Section 5 concedes the data lacks action-specific form feedback.
  • domain assumption VQ-VAE tokenization preserves the information needed to compare two motions.
    Section 3.3 feeds only discrete token lists to the LLM; if quantization discards timing or joint detail, the text cannot encode the motion difference.
  • ad hoc to paper Reconstruction with the same editor is a valid proxy for instruction quality.
    Section 4.2 evaluates by using the predicted instruction to edit the source with MDM and measuring MPJPE/FID against the MDM-generated target; this assumes self-consistency with the data generator equals real coaching effectiveness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CigTime: Corrective Instruction Generation Through Inverse Motion Editing." pith.science (2026). https://pith.science/paper/BCPES4MI

@misc{pith2026241205460,
  author       = {Pith},
  title        = {Pith review of: CigTime: Corrective Instruction Generation Through Inverse Motion Editing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BCPES4MI}},
  note         = {Machine review of arXiv:2412.05460}
}
read the original abstract

Recent advancements in models linking natural language with human motions have shown significant promise in motion generation and editing based on instructional text. Motivated by applications in sports coaching and motor skill learning, we investigate the inverse problem: generating corrective instructional text, leveraging motion editing and generation models. We introduce a novel approach that, given a user's current motion (source) and the desired motion (target), generates text instructions to guide the user towards achieving the target motion. We leverage large language models to generate corrective texts and utilize existing motion generation and editing frameworks to compile datasets of triplets (source motion, target motion, and corrective text). Using this data, we propose a new motion-language model for generating corrective instructions. We present both qualitative and quantitative results across a diverse range of applications that largely improve upon baselines. Our approach demonstrates its effectiveness in instructional scenarios, offering text-based guidance to correct and enhance user performance.

Figures

Figures reproduced from arXiv: 2412.05460 by the authors.

Figure 1
Figure 1. Overview of CigTime. Left: We leverage source motion tokens and corrective instructions as input to a motion editor to produce target motion tokens. Right: We then employ a language model to generate precise corrective instructions based on a given source and target motion. We demonstrate in the example generating corrective instructions for lifting weights with the upper body. motion editor to edit the source motio… view at source ↗
Figure 2
Figure 2. Template for LLM fine-tuning. The LLM is required to output the corrective instructions given token lists for the source and target motion sequences (i.e., Action 1 and Action 2) as well as instructions on the expected output. 3.3 Fine-tuning LLMs for Corrective Instruction Generation With the prepared dataset of triplets from the motion editing process, we learn the inverse process of motion editing, a function, T … view at source ↗
Figure 3
Figure 3. Visualization of corrective instructions and reconstructed motions for different methods. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: In-context learning for corrective instruction generation. The prompt for the LLMs in in-context learning includes a task description and several examples. This information is given to the LLMs, instructing them to generate correctional instructions for new motion pair…
Figure 5
Figure 5. Figure 5: Diversity of the corrective instructions. We present some examples where the recon￾structed motions have a similar appearance to the target motions, but the corrective instructions still differ from the ground truth, demonstrating the robustness of our approach generat…
Figure 6
Figure 6. Figure 6: Real-world application. This figure illustrates the source and target motions collected from real-world participants, alongside the corrective instructions generated by different methods. Left to right: the source motion, target motion, generated corrective instruction…
Figure 7
Figure 7. Figure 7: Additional visualizations. Qualitative results for the corrective instructions and recon￾structed motion sequences. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: Additional visualizations. Qualitative results for the corrective instructions and recon￾structed motion sequences. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 24 canonical work pages

  1. [1]

    Language2pose: Natural language grounded pose forecasting

    Chaitanya Ahuja and Louis-Philippe Morency. Language2pose: Natural language grounded pose forecasting. In 2019 International Conference on 3D Vision (3DV), pages 719–728. IEEE, 2019

    2019

  2. [2]

    Teach: Temporal action composition for 3d humans

    Nikos Athanasiou, Mathis Petrovich, Michael J Black, and Gül Varol. Teach: Temporal action composition for 3d humans. In 2022 International Conference on 3D Vision (3DV) , pages 414–423. IEEE, 2022

    work page 2022

  3. [3]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

  4. [4]

    Meteor: An automatic metric for mt evaluation with improved correlation with human judgments

    Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72, 2005

    2005

  5. [5]

    Motionllm: Understanding human behaviors from human motions and videos

    Ling-Hao Chen, Shunlin Lu, Ailing Zeng, Hao Zhang, Benyou Wang, Ruimao Zhang, and Lei Zhang. Motionllm: Understanding human behaviors from human motions and videos. arXiv preprint arXiv:2405.20340, 2024

    arXiv 2024

  6. [6]

    Pose trainer: correcting exercise posture using pose estimation

    Steven Chen and Richard R Yang. Pose trainer: correcting exercise posture using pose estimation. arXiv preprint arXiv:2006.11718, 2020

    arXiv 2006

  7. [7]

    Pose- fix: Correcting 3d human poses with natural language

    Ginger Delmas, Philippe Weinzaepfel, Francesc Moreno-Noguer, and Grégory Rogez. Pose- fix: Correcting 3d human poses with natural language. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15018–15028, 2023

    work page 2023

  8. [8]

    Pose tutor: an explainable system for pose correction in the wild

    Bhat Dittakavi, Divyagna Bavikadi, Sai Vikas Desai, Soumi Chakraborty, Nishant Reddy, Vineeth N Balasubramanian, Bharathi Callepalli, and Ayon Sharma. Pose tutor: an explainable system for pose correction in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3540–3549, 2022

    work page 2022

Show all 47 references
  1. [9]

    A survey on in-context learning

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. A survey on in-context learning. arXiv preprint arXiv:2301.00234, 2022. 10

    2022 arXiv

  2. [10]

    C· ase: Learning conditional adversarial skill embeddings for physics-based characters

    Zhiyang Dou, Xuelin Chen, Qingnan Fan, Taku Komura, and Wenping Wang. C· ase: Learning conditional adversarial skill embeddings for physics-based characters. In SIGGRAPH Asia 2023 Conference Papers, pages 1–11, 2023

    2023

  3. [11]

    Aifit: Automatic 3d human-interpretable feedback models for fitness training

    Mihai Fieraru, Mihai Zanfir, Silviu Cristian Pirlea, Vlad Olaru, and Cristian Sminchisescu. Aifit: Automatic 3d human-interpretable feedback models for fitness training. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9919–9928, 2021

    2021

  4. [12]

    Syn- thesis of compositional animations from textual descriptions

    Anindita Ghosh, Noshaba Cheema, Cennet Oguz, Christian Theobalt, and Philipp Slusallek. Syn- thesis of compositional animations from textual descriptions. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1396–1406, 2021

    2021

  5. [13]

    Generating diverse and natural 3d human motions from text

    Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5152–5161, 2022

    2022

  6. [14]

    Tm2t: Stochastic and tokenized modeling for the reciprocal generation of 3d human motions and texts

    Chuan Guo, Xinxin Zuo, Sen Wang, and Li Cheng. Tm2t: Stochastic and tokenized modeling for the reciprocal generation of 3d human motions and texts. In European Conference on Computer Vision, pages 580–597. Springer, 2022

    2022

  7. [15]

    Action2motion: Conditioned generation of 3d human motions

    Chuan Guo, Xinxin Zuo, Sen Wang, Shihao Zou, Qingyao Sun, Annan Deng, Minglun Gong, and Li Cheng. Action2motion: Conditioned generation of 3d human motions. In Proceedings of the 28th ACM International Conference on Multimedia, pages 2021–2029, 2020

    2021

  8. [16]

    Synthesizing physical character-scene interactions

    Mohamed Hassan, Yunrong Guo, Tingwu Wang, Michael Black, Sanja Fidler, and Xue Bin Peng. Synthesizing physical character-scene interactions. In ACM SIGGRAPH 2023 Conference Proceedings, pages 1–9, 2023

    2023

  9. [17]

    A deep learning framework for character motion synthesis and editing

    Daniel Holden, Jun Saito, and Taku Komura. A deep learning framework for character motion synthesis and editing. ACM Transactions on Graphics (TOG), 35(4):1–11, 2016

    2016

  10. [18]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  11. [19]

    Mistral 7b

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023

    arXiv

  12. [20]

    Motiongpt: Human motion as a foreign language

    Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. Motiongpt: Human motion as a foreign language. Advances in Neural Information Processing Systems, 36, 2024

    2024

  13. [21]

    Action-gpt: Leveraging large-scale language models for improved and generalized action generation

    Sai Shashank Kalakonda, Shubh Maheshwari, and Ravi Kiran Sarvadevabhatla. Action-gpt: Leveraging large-scale language models for improved and generalized action generation. In 2023 IEEE International Conference on Multimedia and Expo (ICME) , pages 31–36. IEEE, 2023

    2023

  14. [22]

    Guided motion diffusion for controllable human motion synthesis

    Korrawe Karunratanakul, Konpat Preechakul, Supasorn Suwajanakorn, and Siyu Tang. Guided motion diffusion for controllable human motion synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2151–2162, 2023

    2023

  15. [23]

    Fixmypose: Pose correctional captioning and retrieval

    Hyounghun Kim, Abhay Zala, Graham Burri, and Mohit Bansal. Fixmypose: Pose correctional captioning and retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 13161–13170, 2021

    2021

  16. [24]

    Flame: Free-form language-based motion synthesis & editing

    Jihoon Kim, Jiseob Kim, and Sungjoon Choi. Flame: Free-form language-based motion synthesis & editing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8255–8263, 2023

    2023

  17. [25]

    A brand new dance partner: Music-conditioned pluralistic dancing controlled by multiple dance genres

    Jinwoo Kim, Heeseok Oh, Seongjean Kim, Hoseok Tong, and Sanghoon Lee. A brand new dance partner: Music-conditioned pluralistic dancing controlled by multiple dance genres. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3490–3500, 2022

    2022

  18. [26]

    Dancing to music

    Hsin-Ying Lee, Xiaodong Yang, Ming-Yu Liu, Ting-Chun Wang, Yu-Ding Lu, Ming-Hsuan Yang, and Jan Kautz. Dancing to music. Advances in neural information processing systems, 32, 2019

    2019

  19. [27]

    Finedance: A fine-grained choreography dataset for 3d full body dance 11 generation

    Ronghui Li, Junfan Zhao, Yachao Zhang, Mingyang Su, Zeping Ren, Han Zhang, Yansong Tang, and Xiu Li. Finedance: A fine-grained choreography dataset for 3d full body dance 11 generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10234–10243, 2023

    2023

  20. [28]

    Ai choreographer: Music conditioned 3d dance generation with aist++

    Ruilong Li, Shan Yang, David A Ross, and Angjoo Kanazawa. Ai choreographer: Music conditioned 3d dance generation with aist++. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13401–13412, 2021

    2021

  21. [29]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. InText summarization branches out, pages 74–81, 2004

    2004

  22. [31]

    Posefix: Model-agnostic general human pose refinement network

    Gyeongsik Moon, Ju Yong Chang, and Kyoung Mu Lee. Posefix: Model-agnostic general human pose refinement network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7773–7781, 2019

    2019

  23. [32]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002

    2002

  24. [33]

    Deepmimic: Example- guided deep reinforcement learning of physics-based character skills

    Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel Van de Panne. Deepmimic: Example- guided deep reinforcement learning of physics-based character skills. ACM Transactions On Graphics (TOG), 37(4):1–14, 2018

    2018

  25. [34]

    Temos: Generating diverse human motions from textual descriptions

    Mathis Petrovich, Michael J Black, and Gül Varol. Temos: Generating diverse human motions from textual descriptions. In European Conference on Computer Vision , pages 480–497. Springer, 2022

    2022

  26. [35]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

  27. [36]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020

    2020

  28. [37]

    Human motion diffusion as a generative prior

    Yonatan Shafir, Guy Tevet, Roy Kapon, and Amit H Bermano. Human motion diffusion as a generative prior. arXiv preprint arXiv:2303.01418, 2023

    2023 arXiv

  29. [38]

    Wham: Reconstructing world- grounded humans with accurate 3d motion

    Soyong Shin, Juyong Kim, Eni Halilaj, and Michael J Black. Wham: Reconstructing world- grounded humans with accurate 3d motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2070–2080, 2024

    2024

  30. [39]

    Motionclip: Exposing human motion generation to clip space

    Guy Tevet, Brian Gordon, Amir Hertz, Amit H Bermano, and Daniel Cohen-Or. Motionclip: Exposing human motion generation to clip space. In European Conference on Computer Vision, pages 358–374. Springer, 2022

    2022

  31. [40]

    Human motion diffusion model

    Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen-Or, and Amit H Bermano. Human motion diffusion model. arXiv preprint arXiv:2209.14916, 2022

    2022 arXiv

  32. [41]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017

    2017

  33. [42]

    Hierarchical quantized autoencoders

    Will Williams, Sam Ringer, Tom Ash, David MacLeod, Jamie Dougherty, and John Hughes. Hierarchical quantized autoencoders. Advances in Neural Information Processing Systems , 33:4524–4535, 2020

    2020

  34. [43]

    Omnicontrol: Control any joint at any time for human motion generation

    Yiming Xie, Varun Jampani, Lei Zhong, Deqing Sun, and Huaizu Jiang. Omnicontrol: Control any joint at any time for human motion generation. arXiv preprint arXiv:2310.08580, 2023

    2023 arXiv

  35. [44]

    T2m-gpt: Generating human motion from textual descriptions with discrete representations

    Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Shaoli Huang, Yong Zhang, Hongwei Zhao, Hongtao Lu, and Xi Shen. T2m-gpt: Generating human motion from textual descriptions with discrete representations. arXiv preprint arXiv:2301.06052, 2023

    2023 arXiv

  36. [45]

    Motiondiffuse: Text-driven human motion generation with diffusion model

    Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondiffuse: Text-driven human motion generation with diffusion model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  37. [46]

    Finemogen: Fine-grained spatio-temporal motion generation and editing

    Mingyuan Zhang, Huirong Li, Zhongang Cai, Jiawei Ren, Lei Yang, and Ziwei Liu. Finemogen: Fine-grained spatio-temporal motion generation and editing. Advances in Neural Information Processing Systems, 36, 2024. 12

    2024

  38. [47]

    Motiongpt: Finetuned llms are general-purpose motion generators

    Yaqi Zhang, Di Huang, Bin Liu, Shixiang Tang, Yan Lu, Lu Chen, Lei Bai, Qi Chu, Nenghai Yu, and Wanli Ouyang. Motiongpt: Finetuned llms are general-purpose motion generators. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 7368–7376, 2024

    2024

  39. [48]

    forehand table tennis

    Zixiang Zhou, Yu Wan, and Baoyuan Wang. Avatargpt: All-in-one framework for motion understanding planning generation and beyond. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1357–1366, 2024. 13 Instruction: I utilize some tokens t...

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.