Pith. sign in

REVIEW 4 major objections 6 minor 43 references

AToM: Aligning Text-to-Motion Model at Event-Level with GPT-4Vision Reward

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AToM fine-tunes a text-to-motion model on GPT-4V-scored preference data to improve event-level alignment across integrity, temporal order, and frequency.

desk verdict Genuinely new reward source for text-to-motion event-level alignment, but the reward model is not validated on the actual scoring tasks and the quantitative gains are modest; still deserves a serious referee. read the letter →

arxiv 2411.18654 v1 pith:YEIMMZLI submitted 2024-11-27 cs.CV

Haonan Han , Xiangzuo Wu , Huan Liao , Zunnan Xu , Zhongyuan Hu , Ronghui Li , Yachao Zhang , Xiu Li This is my paper · ORCID
classification cs.CV
keywords text-to-motiongenerationevent-levelalignmentGPT-4VrewardpreferenceoptimizationAIfeedbackMotionPrefermotion-textLoRAfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes that a vision-language model's scores of rendered motions can act as a training signal for text-to-motion generation, without motion-capture annotation or human preference labeling. It builds MotionPrefer, a dataset of 5,276 event-level prompts and 80K motion preference pairs, where each motion is graded by GPT-4V on whether it contains all described actions, puts them in the right order, and repeats them the right number of times. Fine-tuning MotionGPT on this data with LoRA and an IPO preference loss is reported to improve event-level alignment across all three dimensions, beating both the base model and the human-feedback baseline InstructMotion on most metrics. A human study backs the result, with AToM preferred in 70 to 84 percent of comparisons depending on the task. If the finding holds, scalable AI feedback can replace costly human labels for fine-grained motion-text alignment.

What carries the argument

The load-bearing object is MotionPrefer, a preference dataset built from event-level prompts and GPT-4V scores, together with the reward paradigm that produces those scores. Central to the paradigm is the motion injection format: generated 3D motion is rendered and sampled as a sequence of frames spaced eight apart, which GPT-4V sees alongside the prompt and a task-specific scoring rubric. The fine-tuning stage converts these scores into preference pairs by grouping motions by prompt, ranking them by score, and keeping only pairs whose score gap exceeds a threshold. The model is then updated with the IPO loss, a preference-optimization objective that trains the policy directly on the ranked pairs, combined with LoRA low-rank adaptation for parameter-efficient tuning.

What would settle it

Take a held-out set of event-level prompts with ground-truth annotations for which motions appear, in what order, and how many times; generate motions; then compare AToM against the base model on the subset where GPT-4V's scores disagree with human labels. If AToM's advantage disappears or reverses there, the GPT-4V reward is the weak link. An even sharper test is to retrain the same pipeline with GPT-4V scores replaced by random or inverted scores: if the reported metric gains persist, they are not caused by the reward's semantic content.

Watch

Extended reading notes

Core claim

The paper's central claim is that event-level alignment failures, such as missing motion events, wrong temporal order, and wrong repetition counts, can be corrected by reward fine-tuning on GPT-4V judgments. The authors generate motions from constructed prompts, render them to video, sample frames every eight frames, and ask GPT-4V to score each motion-prompt pair under task-specific rubrics: integrity on a 0/5 scale, temporal order on a 1 to 5 scale, and frequency on a 1 to 3 scale. These scores form the MotionPrefer dataset, from which preference pairs are selected only when score differences exceed a threshold. Fine-tuning MotionGPT with LoRA and the IPO loss on these pairs is shown to lower text-motion distance, raise retrieval precision, and improve FID relative to MotionGPT and InstructMotion, with human win rates of 84.4 percent for integrity, 74.4 percent for temporal order, and 70.0 percent for frequency.

Load-bearing premise

The whole gain rests on GPT-4V being a trustworthy judge of event-level alignment from a handful of still frames, even though the paper's own check puts its agreement with human labels at only 69.77 percent.

Editorial extensions

If this is right

  • Event-level alignment can be improved without new motion-capture data: the reward comes from rendered images and a frozen vision-language model.
  • The same reward paradigm can be reused with other vision-language models as they improve, since only the scoring instructions would need to change.
  • Fine-grained preference signals covering integrity, order, and frequency isolate which failure mode a motion model still has, making the dataset a diagnostic tool as well as a training set.
  • The approach offers a scalable alternative to human RLHF data collection, reducing the cost of aligning text-to-motion models to detailed event descriptions.
  • The reported reductions in diversity and multimodality indicate that alignment gains come with a coverage trade-off that downstream motion applications will need to manage.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If GPT-4V scoring is the bottleneck, as the paper's own 69.77 percent agreement with human labels suggests, then augmenting or replacing it with a small human-labeled reward model could push the same pipeline further.
  • The render-and-score preference loop could generalize to other generative domains whose outputs can be visualized, such as text-to-video, audio spectrograms, or 3D scene generation, whenever event-level correctness matters.
  • The visible effects of score filtering and preference dataset volume imply that data curation quality may matter more than the choice of RL algorithm, so a cleaner larger dataset could outperform a more sophisticated loss.
  • The diversity and multimodality trade-off visible in the metrics points to a potential failure mode: over-optimizing the reward could collapse the motion distribution, making a diversity-regularized variant a natural next experiment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents AToM, a three-stage framework for improving event-level text-to-motion alignment. In stage one, GPT-4 constructs 5.3K synthetic prompts from HumanML3D labels, and MotionGPT generates multiple candidate motions per prompt to form the MotionPrefer dataset (Table 3). In stage two, GPT-4V scores the alignment of rendered, 8-frame-interval frame sequences against each prompt using task-specific rubrics for integrity, temporal order, and frequency (Eq. 5, Table 4). In stage three, preference pairs are constructed from score gaps (Algorithm 1) and used to fine-tune MotionGPT via IPO with LoRA (Eqs. 7–8). The authors report automated metric improvements on filtered HumanML3D test sets, human-study win rates of 70.0%–84.4% over MotionGPT (Fig. 4), and ablations on injection format, score filtering, LoRA, RL strategy, sampling interval, and dataset volume. The central claim is that GPT-4V feedback provides a scalable AI reward signal that improves event-level alignment beyond both the pretrained model and a human-feedback baseline.

Significance. If the central claim holds, the paper offers a practical recipe for replacing expensive human preference annotation with VLM-based reward scoring in text-to-motion alignment, and it introduces a fine-grained, multi-aspect preference dataset that could support future work. The strengths include the scale and structure of MotionPrefer, the clear ablation of motion-injection formats, the inclusion of a human evaluation, and the systematic comparison with alternative RL losses. However, the significance is currently bounded by two unverified load-bearing assumptions: that GPT-4V's event-level rubric scores are reliable enough to construct preference pairs, and that the filtered HumanML3D test set is not biased toward the synthetic prompt distribution used for fine-tuning. The paper also overstates the consistency of its quantitative gains, given that several metrics worsen or overlap within standard deviations. These issues are fixable within the manuscript's scope.

major comments (4)
  1. [Sec. 3.2, Eq. (5); Appendix 6.4] The reward signal is not validated for the exact task it is used for. GPT-4V's only reported agreement with human preference is 69.77% pairwise accuracy on InstructMotion's human-preference dataset (Appendix 6.4, Table 7), which is a binary preference task, not the 0/5, 1–5, and 1–3 event-level rubrics of Table 4 used in Eq. (5). Since Algorithm 1 and the IPO loss in Eqs. (7)–(8) rely entirely on these scores, the 69.77% figure provides no direct evidence that the constructed preference pairs are correct; if GPT-4V relies on visual heuristics when judging 8-frame-interval frame sequences, the fine-tuning could amplify those heuristics rather than improve true event-level alignment. Please validate the exact scoring rubrics against human annotations on a held-out subset of MotionPrefer, or provide an explicit noise-robustness analysis of IPO under the measured label-error rate.
  2. [Sec. 4.1] The filtering of the HumanML3D test set is not described. The sentence 'we filtered the HumanML3D test set, obtaining 418, 506, and 234 text-motion pairs' gives no inclusion criteria. If the filter selects prompts that match the synthetic template used to construct MotionPrefer (random HumanML3D labels joined by conjunctions), then both the automated metrics and the user study in Figure 4 are evaluated on a distribution close to the training distribution, which would inflate the reported gains and weaken the generalization claim. Please specify the filter procedure and report the overlap between the filtered test set and the synthetic prompt distribution.
  3. [Table 5; Sec. 4.2] The claim that AToM 'consistently outperforms' baselines is too strong given the reported statistics. For integrity, FID worsens (0.400 vs. 0.340) and MM Dist and Top-1 are unchanged within standard error; MModality decreases in all three tasks (e.g., 3.495 vs. 3.846 for temporal); for frequency, Top-1/Top-2/Top-3 are essentially flat. Most reported differences are within overlapping standard deviations, including the General-task comparison against InstructMotion. Please temper the conclusion, report effect sizes or confidence intervals, and explicitly discuss the trade-offs in diversity and multimodality rather than attributing them to 'slight reductions.'
  4. [Sec. 3.1, Table 3] The synthetic prompt construction uses GPT-4 to join randomly selected HumanML3D labels with conjunctions, but the paper provides no evidence that this distribution resembles natural event-level prompts. Because both training and evaluation are built on this template, the approach may be fitting the template rather than improving general event-level understanding. Please include a distributional comparison between the synthetic prompts and natural event-level prompts, or a human assessment of the naturalness of the constructed prompts.
minor comments (6)
  1. [Sec. 3.1; Supplementary Tables 11–12] The main text cites 'Table 12' for the temporal prompt-construction instruction, but in the supplementary material Table 12 is the frequency instruction and Table 11 is the temporal instruction; please correct the cross-reference.
  2. [Sec. 3.2, Eq. (6)] Equation (6) introduces the symbol 'Dscore' but the variable defined earlier in the same section is 'Dreward'; please unify the notation.
  3. [Table 1] The row for 'Guo et al. [11]' lists no annotator, prompt count, or pair count, which makes the comparison incomplete; either fill in the entries or explain why the data are unavailable.
  4. [Figure 5] The caption for Figure 5 does not identify the axes or the units, and the statement that FID has been 'negatively treated' is unexplained; please clarify the visualization.
  5. [Sec. 4.3, Figure 6] The text says an 8-frame interval was chosen because it provides strong retrieval precision, but Figure 6 shows lower FID at interval 4; please justify the choice or acknowledge the trade-off.
  6. [Throughout] The term 'event-level' is used repeatedly but is never formally defined; a brief definition in Section 1 would improve precision and help readers connect the three sub-tasks to the concept.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the GPT-4V reward is evaluated against independent HumanML3D retrieval metrics and human judgments, and the only self-citation is a non-load-bearing related-work mention.

full rationale

The paper's derivation chain is not circular. The training signal is GPT-4V's event-level alignment scores (Eq. 5, Table 4), while the main quantitative evaluation uses HumanML3D retrieval metrics (MM-Dist, R-Precision), FID, Diversity, and MModality, plus a human study (Sec. 4.1-4.2). These are independent measurement channels: the model is optimized to satisfy GPT-4V's rubric, but the reported improvements are measured by text-motion encoders and human judgments, not by the reward itself. The human study in Fig. 4 asks participants to compare motions on frequency, integrity, and temporal alignment; although these criteria match the reward rubric, human evaluation is an external check rather than a formal consequence of the training objective. Appendix 6.4 validates GPT-4V against InstructMotion's external human preference data (69.77% accuracy), which is a separate benchmark and does not make the central claim true by construction. The only potential overlaps are that synthetic training prompts and the filtered test set both draw on HumanML3D, and the test filtering criteria are not described; these are data-distribution or leakage risks, not circularity. The sole self-citation is [17] (BATON), used only as an example of aligning generative audio models with human feedback in the related-work section; it is not load-bearing for any derivation. Therefore no circular step can be exhibited, and the score reflects only the presence of a minor, non-load-bearing self-citation.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claim rests on the validity of GPT-4V as a reward model, on the representativeness of the filtered test set, and on several hyperparameters tuned on the same test metrics used for final claims. There are no invented physical entities; the new artifact is the MotionPrefer dataset and the GPT-4V reward paradigm.

free parameters (6)
  • IPO beta (β) = 0.10
    Selected by ablation on test-set MM Dist, R-Precision, FID, and diversity (Appendix 6.3, Figure 9); the same test metrics report the final results.
  • Frame sampling interval = 8 frames
    Selected by comparing intervals 4, 8, 16, 32 on test metrics (Figure 6); the choice is a tuned hyperparameter, not a principled constant.
  • Score filtering threshold = rating > 3 as positive
    Table 6(b) compares with and without filtering and reports improved metrics; the threshold is chosen post hoc on the same test set.
  • Preference-pair score gap delta (δ) = not reported
    Algorithm 1 selects pairs only when ri - rj > delta; the value of delta is not given, so this threshold is an unreported free parameter.
  • LoRA rank and alpha = R=8, alpha=16
    LoRA configuration is a hyperparameter choice (Sec 4.1); it affects capacity and is not fitted to data.
  • Training epochs = 20
    Chosen from an iteration ablation (Appendix 6.2); beyond 30 iterations overfits, and 20 is selected as the operating point.
assumptions (6)
  • domain assumption GPT-4V's visual scores from sampled frames are a valid proxy for human event-level alignment.
    The whole reward paradigm depends on this; the paper validates it only to 69.77% agreement on InstructMotion pairs, so the assumption is partially supported but not established.
  • domain assumption Rendered videos sampled at 8-frame intervals preserve the information needed to judge integrity, temporal order, and frequency.
    Sec 3.2 Eq 4 uses Sampler(Dmotion-video, 8); very short or fast motions could be invisible between frames, especially for counting.
  • domain assumption The filtered HumanML3D test set is a representative benchmark for event-level alignment.
    Sec 4.1 filters the test set without stating criteria; if the filter selects prompts similar to MotionPrefer, the evaluation is biased toward the method's training distribution.
  • domain assumption MM Dist, R-Precision, FID, Diversity, and MModality capture event-level alignment quality.
    These metrics use HumanML3D contrastive encoders and distribution distances; they were designed for general text-motion alignment and may not reflect event-level errors such as wrong order or wrong count.
  • standard math IPO loss and LoRA provide a valid optimization framework for preference learning.
    Standard results from Azar et al. and Hu et al., cited in Sec 3.3; not derived in this paper.
  • ad hoc to paper Synthetic prompts constructed from HumanML3D labels with conjunctions resemble real event-level user prompts.
    Prompt construction in Sec 3.1 relies on GPT-4 and fixed label groups; distribution shift to natural prompts is not measured.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AToM: Aligning Text-to-Motion Model at Event-Level with GPT-4Vision Reward." pith.science (2026). https://pith.science/paper/YEIMMZLI

@misc{pith2026241118654,
  author       = {Pith},
  title        = {Pith review of: AToM: Aligning Text-to-Motion Model at Event-Level with GPT-4Vision Reward},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YEIMMZLI}},
  note         = {Machine review of arXiv:2411.18654}
}
read the original abstract

Recently, text-to-motion models have opened new possibilities for creating realistic human motion with greater efficiency and flexibility. However, aligning motion generation with event-level textual descriptions presents unique challenges due to the complex relationship between textual prompts and desired motion outcomes. To address this, we introduce AToM, a framework that enhances the alignment between generated motion and text prompts by leveraging reward from GPT-4Vision. AToM comprises three main stages: Firstly, we construct a dataset MotionPrefer that pairs three types of event-level textual prompts with generated motions, which cover the integrity, temporal relationship and frequency of motion. Secondly, we design a paradigm that utilizes GPT-4Vision for detailed motion annotation, including visual data formatting, task-specific instructions and scoring rules for each sub-task. Finally, we fine-tune an existing text-to-motion model using reinforcement learning guided by this paradigm. Experimental results demonstrate that AToM significantly improves the event-level alignment quality of text-to-motion generation.

Figures

Figures reproduced from arXiv: 2411.18654 by the authors.

Figure 1
Figure 1. Showcases of motion samples for three scenarios. The two motion samples for each scenario were generated based on the prompt [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The framework of AToM. AToM encompasses three stages: (1) A motion generation process using task-specific prompts constructed by LLM; (2) Evaluation of alignment score for text-motion pairs using a predefined reward paradigm based on LVLM; (3) A fine-tuning mechanism based on LoRA and RL strategy that enhances the original motion generator using the dataset MotionPrefer. Given a label group, Xtask, the three labels … view at source ↗
Figure 3
Figure 3. Generated qualitative samples comparison of pretrained model MotionGPT and finetuned model AToM. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (10 more)
Figure 5
Figure 5. Figure 5: Performance distri￾bution of different reinforce￾ment learning strategies after generative model finetuning. each across three criteria: frequency, integrity, and tempo￾ral alignment. For each pair, the participant chose the better motion or marked a tie if both were c…
Figure 6
Figure 6. Figure 6: Impact of different frame sampling intervals on align [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Generated qualitative samples comparison of pretrained model MotionGPT and finetuned model AToM. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Impact of different epoch numbers on alignment and [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Impact of β on alignment and quality metrics [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Impact of preference pair quantity on alignment and [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Full-Image Example [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: Trajectory-Image Example 8. Human Evaluation We present an example of the user study for the frequency task in [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: Frame-by-Frame Example Task1: Frenquency Task 1 Description (Must Read Before Answering) • Target: Select the video with higher quality in terms of action frequency from each pair of videos. • Definition of Action Frequency: ◦ Definition of High Quality: The video sho…
Figure 14
Figure 14. Figure 14: User study example of frequency task 5 [PITH_FULL_IMAGE:figures/full_fig_p015_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 23 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  2. [2]

    Text2action: Generative adversarial synthesis from language to action

    Hyemin Ahn, Timothy Ha, Yunho Choi, Hwiyeon Yoo, and Songhwai Oh. Text2action: Generative adversarial synthesis from language to action. In2018 IEEE International Confer- ence on Robotics and Automation (ICRA), pages 5915–5920. IEEE, 2018. 2

    2018

  3. [3]

    Lan- guage2pose: Natural language grounded pose forecasting

    Chaitanya Ahuja and Louis-Philippe Morency. Lan- guage2pose: Natural language grounded pose forecasting. In 2019 International Conference on 3D Vision (3DV), pages 719–728. IEEE, 2019. 2

    work page 2019

  4. [4]

    The claude 3 model family: Opus, sonnet, haiku

    AI Anthropic. The claude 3 model family: Opus, sonnet, haiku. Claude-3 Model Card, 1, 2024. 3

    work page 2024

  5. [5]

    Teach: Temporal action composition for 3d hu- mans

    Nikos Athanasiou, Mathis Petrovich, Michael J Black, and G¨ul Varol. Teach: Temporal action composition for 3d hu- mans. In 2022 International Conference on 3D Vision (3DV), pages 414–423. IEEE, 2022. 3

    work page 2022

  6. [6]

    A general theoretical paradigm to un- derstand learning from human preferences

    Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bi- lal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. A general theoretical paradigm to un- derstand learning from human preferences. In International Conference on Artificial Intelligence and Statistics , pages 4447–4455. PMLR, 2024. 2, 5, 8

    work page 2024

  7. [7]

    Training a helpful and harmless assistant with reinforcement learning from human feedback

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022. 3

    arXiv 2022

  8. [8]

    Con- stitutional ai: Harmlessness from ai feedback

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Con- stitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073, 2022. 3

    arXiv 2022

Show all 43 references
  1. [9]

    Executing your commands via motion diffusion in latent space

    Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands via motion diffusion in latent space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18000–18010, 2023. 1, 3

    2023

  2. [10]

    Synthesis of compositional animations from textual descriptions

    Anindita Ghosh, Noshaba Cheema, Cennet Oguz, Christian Theobalt, and Philipp Slusallek. Synthesis of compositional animations from textual descriptions. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1396–1406, 2021. 2

    2021

  3. [11]

    Generating diverse and natural 3d human motions from text

    Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5152–5161, 2022. 2, 4, 6, 7, 1

    2022

  4. [12]

    Tm2t: Stochastic and tokenized modeling for the reciprocal genera- tion of 3d human motions and texts

    Chuan Guo, Xinxin Zuo, Sen Wang, and Li Cheng. Tm2t: Stochastic and tokenized modeling for the reciprocal genera- tion of 3d human motions and texts. InEuropean Conference on Computer Vision, pages 580–597. Springer, 2022. 1

    2022

  5. [13]

    Momask: Generative masked model- ing of 3d human motions

    Chuan Guo, Yuxuan Mu, Muhammad Gohar Javed, Sen Wang, and Li Cheng. Momask: Generative masked model- ing of 3d human motions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1900–1910, 2024. 1

    1900

  6. [14]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 2, 6, 8

    2021 arXiv

  7. [15]

    Motiongpt: Human motion as a foreign lan- guage

    Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. Motiongpt: Human motion as a foreign lan- guage. Advances in Neural Information Processing Systems, 36:20067–20079, 2023. 1, 2, 3, 4, 6, 7, 8

    2023

  8. [16]

    Aligning text- to-image models using human feedback

    Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, and Shixiang Shane Gu. Aligning text- to-image models using human feedback. arXiv preprint arXiv:2302.12192, 2023. 3

    2023 arXiv

  9. [17]

    Baton: Aligning text-to-audio model with human prefer- ence feedback

    Huan Liao, Haonan Han, Kai Yang, Tianjiao Du, Rui Yang, Zunnan Xu, Qinmei Xu, Jingquan Liu, Jiasheng Lu, and Xiu Li. Baton: Aligning text-to-audio model with human prefer- ence feedback. In IJCAI 2024, 2024. 3

    2024

  10. [18]

    Generating animated videos of human activities from natural language descrip- tions

    Angela S Lin, Lemeng Wu, Rodolfo Corona, Kevin Tai, Qix- ing Huang, and Raymond J Mooney. Generating animated videos of human activities from natural language descrip- tions. Learning, 1(2018):1, 2018. 2

    2018

  11. [19]

    Peft: State- of-the-art parameter-efficient fine-tuning methods

    Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, and B Bossan. Peft: State- of-the-art parameter-efficient fine-tuning methods. URL: https://github. com/huggingface/peft, 2022. 6

    2022

  12. [20]

    Learning generalizable human mo- tion generator with reinforcement learning

    Yunyao Mao, Xiaoyang Liu, Wengang Zhou, Zhenbo Lu, and Houqiang Li. Learning generalizable human mo- tion generator with reinforcement learning. arXiv preprint arXiv:2405.15541, 2024. 2, 3, 1

    2024 arXiv

  13. [21]

    Gpt-4 technical report, 2024

    OpenAI. Gpt-4 technical report, 2024. 2

    2024

  14. [22]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Car- roll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Ad- vances in neural information processing systems, 35:...

    2022

  15. [23]

    Modipo: text-to-motion alignment via ai-feedback-driven direct preference optimiza- tion

    Massimiliano Pappa, Luca Collorone, Giovanni Ficarra, In- dro Spinelli, and Fabio Galasso. Modipo: text-to-motion alignment via ai-feedback-driven direct preference optimiza- tion. arXiv preprint arXiv:2405.03803, 2024. 2, 3

    2024 arXiv

  16. [24]

    Black, and G ¨ul Varol

    Mathis Petrovich, Michael J. Black, and G ¨ul Varol. Tmr: Text-to-motion retrieval using contrastive 3d human motion synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9488–9497,

  17. [25]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024. 3

    2024

  18. [26]

    Proximal policy optimization algo- rithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Rad- ford, and Oleg Klimov. Proximal policy optimization algo- rithms. arXiv preprint arXiv:1707.06347, 2017. 3 9

    2017 arXiv

  19. [27]

    Human motion diffusion model

    Yonatan Shafir, Guy Tevet, Roy Kapon, and Amit H Bermano. Human motion diffusion model. arXiv preprint arXiv:2209.14916, 2022. 1, 3

    2022 arXiv

  20. [28]

    Human motion diffusion as a generative prior

    Yonatan Shafir, Guy Tevet, Roy Kapon, and Amit H Bermano. Human motion diffusion as a generative prior. arXiv preprint arXiv:2303.01418, 2023. 1

    2023 arXiv

  21. [29]

    Exploring text-to-motion generation with human preference

    Jenny Sheng, Matthieu Lin, Andrew Zhao, Kevin Pruvost, Yu-Hui Wen, Yangguang Li, Gao Huang, and Yong-Jin Liu. Exploring text-to-motion generation with human preference. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1888–1899, 2024....

    2024

  22. [30]

    Motionclip: Exposing human motion generation to clip space

    Guy Tevet, Brian Gordon, Amir Hertz, Amit H Bermano, and Daniel Cohen-Or. Motionclip: Exposing human motion generation to clip space. In European Conference on Com- puter Vision, pages 358–374. Springer, 2022. 3

    2022

  23. [31]

    Motiongpt-2: A general-purpose motion- language model for motion generation and understanding

    Yuan Wang, Di Huang, Yaqi Zhang, Wanli Ouyang, Jile Jiao, Xuetao Feng, Yan Zhou, Pengfei Wan, Shixiang Tang, and Dan Xu. Motiongpt-2: A general-purpose motion- language model for motion generation and understanding. arXiv preprint arXiv:2410.21747, 2024. 6

    arXiv

  24. [32]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. Advances in Neural Information Pro- cessing Systems, 36, 2024. 3

    2024

  25. [33]

    Generating human motion from textual descrip- tions with discrete representations

    Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Yong Zhang, Hongwei Zhao, Hongtao Lu, Xi Shen, and Ying Shan. Generating human motion from textual descrip- tions with discrete representations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, ...

    2023

  26. [34]

    Motiondif- fuse: Text-driven human motion generation with diffusion model

    Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondif- fuse: Text-driven human motion generation with diffusion model. arXiv preprint arXiv:2208.15001, 2022. 3

    2022 arXiv

  27. [35]

    Re- modiffuse: Retrieval-augmented motion diffusion model

    Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong, Huirong Li, Lei Yang, and Ziwei Liu. Re- modiffuse: Retrieval-augmented motion diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 364–373, 2023. 1

    2023

  28. [36]

    Temo: Towards text-driven 3d stylization for multi-object meshes

    Xuying Zhang, Bo-Wen Yin, Yuming Chen, Zheng Lin, Yun- heng Li, Qibin Hou, and Ming-Ming Cheng. Temo: Towards text-driven 3d stylization for multi-object meshes. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19531–19540, 2024. 3

    2024

  29. [37]

    Slic-hf: Sequence like- lihood calibration with human feedback

    Yao Zhao, Rishabh Joshi, Tianqi Liu, Misha Khalman, Mo- hammad Saleh, and Peter J Liu. Slic-hf: Sequence like- lihood calibration with human feedback. arXiv preprint arXiv:2305.10425, 2023. 3

    2023 arXiv

  30. [38]

    Fine-tuning language models from human preferences

    Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019. 3 10 AToM: Aligning Text-to-Motion Model at Event-Level with ...

    1909 arXiv

  31. [39]

    a person walks forward, turns around, walks backward, and then squats

    Additional Results 6.1. More Qualitative Results Figure 7 presents additional qualitative comparisons be- tween AToM and the baseline models, highlighting AToM’s superior performance. 6.2. Number of Iterations for Fine-tuning We increased the number of iterations for fine-tuni...

  32. [40]

    Event1, Conjunction1, Event2, Conjunction2, ..., Conjunction4, Event5

    Details of MotionPrefer Construction 7.1. GPT-4 Instruction for Prompt Construction We instruct GPT-4 to generate motion-event-based prompt. The designed instruction for three tasks are as follows: The distribution of prompts with varying numbers of mo- tion events is shown in...

  33. [41]

    Human Evaluation We present an example of the user study for the frequency task in Figure 14. 4

  34. [42]

    [8] [16] [24] [32] [40] [48] [56] [64] [72] [80] [88]

  35. [96]

    Equivalent Quality

    [104] [112] [120] [128] [136] [144] [152] [160] [168] [176] [184] [192] [Frame Index] Figure 13. Frame-by-Frame Example Task1: Frenquency Task 1 Description (Must Read Before Answering) • Target: Select the video with higher quality in terms of action frequency from each pair ...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.