Toward Rich Video Human-Motion2D Generation

Cairong Zhao; Feng Wei; Ruihao Xi; Shuhua Li; Xuekuan Wang; Yiwei Wang; Yongcheng Li; Zichen Wang

REVIEW 3 major objections 6 minor 55 references

Toward Rich Video Human-Motion2D Generation

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces a 150,000-sequence dataset of text-annotated 2D human motions weighted toward two-person interactions, and an RVHM2D diffusion model that generates such motions from text, alone or with a first-frame reference, using…

desk verdict The dataset is worth a look, but the 'FID-based RL' is mislabeled: Eqs. (8)-(10) are cosine-similarity losses with no RL update, and the evaluation shares encoders with the model. read the letter →

arxiv 2506.14428 v1 pith:VI6OBTJU submitted 2025-06-17 cs.CV

Ruihao Xi , Xuekuan Wang , Yongcheng Li , Shuhua Li , Zichen Wang , Yiwei Wang , Feng Wei , Cairong Zhao This is my paper

classification cs.CV

keywords text-to-motiongeneration2Dhumanmotionmulti-characterinteractiondiffusionmodeldatasetFIDrewardreinforcementlearningskeleton-drivenvideosynthesis

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that 2D human-motion generation can handle the hard case of two interacting people, not just one person, if the training data is large enough and the model is built to exploit rich text. To that end it contributes Motion2D-Video-150K, 150,000 text-annotated 2D skeleton sequences with roughly 1.5 double-person clips for every single-person clip, and RVHM2D, a diffusion model that reads both global and word-level text features and is fine-tuned with FID-style rewards. If the claim holds, the 2D motion prior it provides could directly feed pose-guided video generators, making text-to-video of coordinated human interaction more controllable and less prone to distorted limbs.

What carries the argument

The load-bearing object is the dataset-plus-conditioning loop. Motion2D-Video-150K is built by running RTMPose-Large on existing action datasets and about 500,000 web clips, captioning skeletons with Gemini 2.0 Flash or Owl3, and filtering on limb confidence, motion smoothness, and tracking stability; this supplies the paired two-person data that prior 3D motion datasets lack. On the model side, RVHM2D uses a dual-tower transformer with shared parameters for the two characters, injects global and local text features through cross-attention after each self-attention layer, and optionally inserts a first-frame reference motion through an extra self-attention over the concatenated sequence. The second-stage training signal is a reinforcement-learning-style objective in which the loss adds $1 - \cos(\mathrm{Enc}_{\text{text}}(c_t),\mathrm{Enc}_{\text{motion}}(m_{\text{pred}}))$ and $1 - \cos(\mathrm{Enc}_{\text{motion}}(m_{\text{gt}}),\mathrm{Enc}_{\text{motion}}(m_{\text{pred}}))$, i.e. text-alignment and motion-FID rewards, on top of bone-length, velocity, distance-map, joint-awareness, and reconstruction losses.

What would settle it

Re-run the Motion2D-Video-150K test evaluation with an R-Precision and FID evaluator built on a different text encoder, for example T5-XXL or a frozen CLIP variant not used in training, and compare RVHM2D against InterGen; if the Top-1 R-Precision lead and the FID improvements are not reproduced, the reported gains depend on sharing the generator's CLIP feature space.

Watch

Extended reading notes

Core claim

The paper's central claim is that, on the new Motion2D-Video-150K benchmark, RVHM2D achieves leading performance among the compared re-implemented baselines (MDM and InterGen) for both single- and double-character text-to-motion generation, with R-Precision Top-1 of 36.64 versus 33.67 for single-character and 31.48 versus 30.70 for double-character, alongside higher diversity scores. The authors attribute this to three mechanisms: enriched text conditioning using dual CLIP-L/B encoders (or T5-XXL) with both pooled and local token features, a unified dual-tower diffusion architecture that shares weights between the two characters and adds interaction cross-attention, and a two-stage training scheme in which a standard diffusion objective is followed by fine-tuning where cosine-similarity terms derived from a pretrained FID evaluation model act as rewards. The dataset claim is that 150,000 mined and cleaned sequences, with a 1.5:1 two-person to one-person ratio and captions generated by multimodal LLMs, are sufficient to train such a model for motions up to 300 frames.

Load-bearing premise

The evaluation setup assumes that judging text-motion alignment with an evaluator that uses the same CLIP-L/B text encoders as the generation model gives fair R-Precision scores; if the evaluator's text encoder is replaced by an independent one, the reported text-alignment gains may shrink.

Editorial extensions

If this is right

Text prompts describing two numbered people can produce skeleton sequences that drive pose-guided video models, yielding interactive videos of two characters.
A single unified model handles both single- and double-character 2D motion by using the person count in the prompt and duplicating the skeleton for single-character cases.
Adding global and local text features from stronger encoders, especially T5-XXL, raises R-Precision and lowers FID relative to pooled CLIP features alone.
Fine-tuning with text-FID and motion-FID rewards after standard diffusion training improves text alignment and distributional realism of generated motions.
The held-out test set, with 6,260 single-character and 5,422 double-character sequences, provides a reusable benchmark for interactive 2D text-to-motion evaluation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A useful further benchmark would measure interaction plausibility separately from single-person pose quality, since the current metrics do not isolate whether two-character motions are physically coordinated.
Because single-character cases are modeled by duplicating one skeleton twice, part of the double-character result may inherit from single-character generation; training on true paired interaction data alone would clarify how much the interaction cross-attention contributes.
If the dataset scales as described, it could serve as a pretraining source for lifting 2D skeletons into 3D motion models, a direction the paper leaves implicit.
A testable extension is to use the FID-reward fine-tuning stage with a stronger independent text encoder as the reward model, which might further close the text-alignment gap without sharing the generator's CLIP feature space.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick read: the dataset is the real contribution; the model's headline mechanism—reinforcement learning with an FID reward—is not what the equations implement. That mismatch is the main thing to know before you spend time here.

What's genuinely new is Motion2D-Video-150K. 150K video clips with 2D skeletons and Gemini-generated captions, roughly 1.5:1 two-person to single-person, and a cleaning pipeline based on limb integrity and motion smoothness. Prior 2D motion datasets are overwhelmingly single-character, so if this is actually released it fills a real gap. The annotation and cleaning description is concrete enough to reproduce. I also credit the T5-XXL result in Table 3—the architecture clearly can exploit a stronger text encoder, and that is a useful data point for the field.

Now the soft spots, and they are substantial. The core model contribution is mislabeled. Eq. (8) is 1 − cosine(text, motion) and Eq. (9) is 1 − cosine(gt, pred). That is not Fréchet Inception Distance, which is a distributional metric over batch statistics, and Eq. (10) is a weighted sum of supervised losses with no policy gradient, no reward sampling, no RL update. The abstract and the contribution list say 'reinforcement learning with an FID-based reward,' but the method section itself only calls it 'RL-inspired' and 'akin to a reward.' That internal contradiction is not a wording nitpick: the ablations attribute the gains to 'FID-based refinement,' and that is unsupported by the paper's own math.

The evaluation compounds the issue. The R-Precision feature extractor uses the same CLIP-L/B text encoders as the generation model, so the text-alignment numbers are partly self-referential. And Table 1 undercuts the 'leading performance' claim: RVHM2D's FID is 1.92/2.41 versus MDM's 0.27/0.39. The only lead is R-Precision Top1 over InterGen, which is exactly the metric most likely to be inflated by the shared-encoder setup. There are also no error bars, and the two 2D-motion datasets cited in the related work—Holistic-Motion2D and HumanDreamer—are never used as baselines.

Who is this for? People building 2D motion priors for video generation would want the dataset. The model, as presented, needs major revision: rename the second stage, re-run with an independent evaluator, and add those missing comparisons. I would send this to peer review rather than desk reject, because the dataset is a legitimate resource and the flaws are fixable in revision. But in current form I would not accept it; the central mechanism is not what it claims to be.

Referee Report

3 major / 6 minor

Summary. The paper introduces Motion2D-Video-150K, a 150,000-sequence 2D human-motion dataset with single- and double-character action captions, and RVHM2D, a diffusion-based model that generates 2D skeleton sequences from text and an optional first-frame reference motion. The model uses dual CLIP-L/B or T5-XXL encoders with global and local features, and a two-stage training procedure whose second stage is described as reinforcement learning with an FID-based reward. Experiments on the new dataset compare RVHM2D with re-implemented MDM and InterGen baselines and include ablations over text encoders, caption sources, and reference-motion conditioning.

Significance. The dataset contribution is potentially valuable: 150K annotated 2D motion sequences with balanced single- and double-character interactions address a real data scarcity for multi-character 2D motion priors, and the unified single/double-character generation framework is a sensible goal. If the model claims were supported, the work would be useful for text-to-video generation of interacting people. The paper also provides a code URL and re-implements baselines, although no dataset URL or evaluation code is provided. The central model claim, however, is not supported by the equations and tables as presented.

major comments (3)

[Reinforcement Learning-Inspired Refinement, Eqs. (8)-(10)] The paper's headline contribution, namely reinforcement learning fine-tuning with an FID-based reward, is not present in the equations. Equations (8) and (9) define per-sample cosine-similarity auxiliary losses, not Fréchet Inception Distances, which are distributional metrics computed over batches of real and generated samples. Equation (10) is a weighted sum of supervised losses; there is no policy gradient, no reward sampling, no value model, and no policy optimization step. Therefore the abstract and the contribution list overstate what is implemented, and the improvements attributed to 'FID-based refinement' in Table 2 and the ablation discussion cannot be credited to the stated mechanism.
[Evaluation Metrics paragraph] The evaluation is circular with respect to the model's text conditioning. The paper states that the evaluation feature extractor 'utilizes CLIP-L/B text encoders as our proposed model,' and Eq. (8) trains the model with a loss computed between the same text encoder and a motion encoder. Because the model is optimized to maximize similarity in the same embedding space used for R-Precision evaluation, the reported text-alignment gains may reflect overfitting to that feature space rather than genuine improvements in text-motion alignment. An independent evaluation feature space, such as a different text encoder or a separately trained motion encoder, is needed to support the claimed R-Precision improvements.
[Table 1] The quantitative results contradict the claimed 'leading performance' on realism. For single-character generation, RVHM2D's FID is 1.9241, compared with 0.2674 for MDM and 1.4317 for InterGen; for two-character generation it is 2.4123, compared with 0.3926 for MDM and 2.0633 for InterGen. Since FID is the metric most directly tied to the paper's realism claims, being substantially worse on FID than both baselines undercuts the central assertion. The statement that MDM's best FID reflects overfitting is not substantiated by any diagnostic evidence.

minor comments (6)

[Data Splits and Availability] The paper calls Motion2D-Video-150K 'publicly available' but provides no dataset URL or release mechanism, only a code URL; please clarify how the dataset will be released.
[Table 1 caption] The Diversity column is annotated with '→ closer to ground truth human motion diversity is better,' but no ground-truth diversity value is reported, so the reader cannot verify the 'closer' claim.
[Table 2] The row labels for the ablation conditions are missing; the reader must infer which check marks correspond to CLIP-L/B, Text FID, and Motion FID. The row that adds both FID losses is particularly ambiguous.
[Table 4] Table 4 varies caption source and reference-motion conditioning simultaneously, but there is no Owl3-with-reference-motion condition, so the effect of reference motion is not cleanly isolated.
[Method, dataset paragraph] The sentence 'as discussed in ,' contains an empty reference and should be completed.
[Implementation Details] The loss weights lambda_textfid and lambda_motionfid and the statement that the 'reward signal is set very low' are not quantified, making the refinement stage difficult to reproduce.

Circularity Check

2 steps flagged · score 6.0 of 10

The second-stage 'FID-based' losses share the evaluation model's CLIP-L/B encoders, so the reported R-Precision gains are partly trained into the evaluation metric itself; the FID/RL mechanism is additionally a renamed per-sample cosine loss.

fitted input called prediction [Experiments, Evaluation Metrics; Method, Eq. (8)]
"However, to align with the text encoding capabilities of our RVHM2D model, our evaluation feature extractor utilizes CLIP-L/B text encoders as our proposed model. ... L_textfid = 1 − sim_cos(Enc_text(c_t), Enc_motion(m_pred)) (8)"

The second-stage loss is computed with the same CLIP-L/B text and motion encoders that the paper's own evaluation model uses to measure R-Precision and FID features. Minimizing Eq. (8) is therefore a direct gradient step toward the evaluator's embedding space, so the Table 2 gains attributed to 'Text FID' and 'Motion FID' losses are partly an artifact of training on the metric's own representation rather than independent evidence of improved text-motion alignment. The claimed prediction of better alignment reduces, for the text-alignment metric, to 'the model is closer to the evaluator because it was trained to be close to the evaluator.'
renaming known result [Method, Reinforcement Learning-Inspired Refinement; Eqs. (8)-(10)]
"In the second stage, we incorporate FID-based scores directly into our loss function to serve as a strong learning signal, akin to a reward in RL ... L_textfid = 1 − sim_cos(Enc_text(c_t), Enc_motion(m_pred)) (8) ... L_motionfid = 1 − sim_cos(Enc_motion(m_gt), Enc_motion(m_pred)) (9) ... L_total = L_first_stage + λ_textfid L_textfid + λ_motionfid L_motionfid (10)"

Fréchet Inception Distance is a distributional statistic computed from the means and covariances of batches of real and generated feature vectors, whereas Eqs. (8) and (9) are per-sample cosine similarities between single embeddings. The final objective is a weighted supervised auxiliary loss with no policy gradient, no reward sampling, and no policy optimization step. The paper's headline contribution of 'reinforcement learning with an FID-based reward' is therefore a renaming of a per-sample cosine auxiliary loss; the claimed mechanism behind Table 2's improvements is not specified by the paper's own equations.

full rationale

The dataset contribution, Motion2D-Video-150K, is an independent empirical artifact and is not circular: it is constructed from video sources, pose estimation, LLM-generated captions, and a cleaning pipeline, and it is evaluated on a held-out split. There is no load-bearing self-citation or imported uniqueness theorem. The central circularity is in the evaluation and refinement loop: the paper deliberately builds its evaluation feature extractor with the same CLIP-L/B text encoders used by RVHM2D, and then the second-stage 'FID' losses in Eqs. (8)-(9) are computed with the encoders from that same evaluation model. This makes the reported R-Precision improvements from the refinement stage, at least in part, a consequence of training directly against the metric's feature space rather than an independent confirmation of better text-motion alignment. Additionally, the paper calls these cosine-similarity losses 'FID-based' and describes the stage as reinforcement learning, but no Fréchet distance and no RL update appear in the equations. This is a naming/mechanism mismatch rather than pure circularity, but it is load-bearing because it is the stated source of the claimed quality gains. Overall, the core text-alignment claim is partially circular, while the dataset and architecture retain independent content, so a score of 6 is appropriate.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The ledger is dominated by domain assumptions about the annotation pipeline and the 2D representation; no new physical entities or forces are introduced. The main unaccounted degrees of freedom are the unreported loss weights and the smoothness threshold, which directly shape both the dataset and the model.

free parameters (2)

Loss weighting hyperparameters
Weights lambda_BL, lambda_VEL, lambda_DM, lambda_JA, lambda_Recon, lambda_textfid, lambda_motionfid in Eq. (7) and (10) are not reported; they are chosen by hand and control the final training objective.
Motion smoothness threshold tau_smooth
Used in the data cleaning pipeline to discard irregular motion, but its value is never specified; it determines which 150K sequences remain.

assumptions (4)

domain assumption RTMPose-Large and LLM captions (Gemini 2.0 Flash, Owl3) produce accurate enough annotations for 150K videos without human verification.
The annotation pipeline in 'The Motion2D-Video-150K Dataset' relies on these models as ground truth, with no manual validation or error rate reported.
domain assumption 17-keypoint 2D skeletons are a sufficient motion representation for rich single- and double-character interaction generation and downstream video synthesis.
The Introduction argues 2D motion is a compelling alternative to 3D capture; all experiments assume this representation captures the needed interaction dynamics.
domain assumption The dual-tower diffusion architecture and interactive attention from InterGen transfer to the new 2D dataset.
Section 'Model Architecture' states RVHM2D adopts a dual-tower structure similar to InterGen without evidence that this design is optimal for the new input modality and dataset.
standard math The diffusion and DDIM equations from Ho et al. (2020) and Song et al. (2020) apply as background math.
The method section uses standard diffusion formulas without modification; these are accepted background results.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Toward Rich Video Human-Motion2D Generation." pith.science (2026). https://pith.science/paper/VI6OBTJU

@misc{pith2026250614428,
  author       = {Pith},
  title        = {Pith review of: Toward Rich Video Human-Motion2D Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VI6OBTJU}},
  note         = {Machine review of arXiv:2506.14428}
}

read the original abstract

Generating realistic and controllable human motions, particularly those involving rich multi-character interactions, remains a significant challenge due to data scarcity and the complexities of modeling inter-personal dynamics. To address these limitations, we first introduce a new large-scale rich video human motion 2D dataset (Motion2D-Video-150K) comprising 150,000 video sequences. Motion2D-Video-150K features a balanced distribution of diverse single-character and, crucially, double-character interactive actions, each paired with detailed textual descriptions. Building upon this dataset, we propose a novel diffusion-based rich video human motion2D generation (RVHM2D) model. RVHM2D incorporates an enhanced textual conditioning mechanism utilizing either dual text encoders (CLIP-L/B) or T5-XXL with both global and local features. We devise a two-stage training strategy: the model is first trained with a standard diffusion objective, and then fine-tuned using reinforcement learning with an FID-based reward to further enhance motion realism and text alignment. Extensive experiments demonstrate that RVHM2D achieves leading performance on the Motion2D-Video-150K benchmark in generating both single and interactive double-character scenarios.

Figures

Figures reproduced from arXiv: 2506.14428 by the authors.

**Figure 1.** Qualitative results illustrating our proposed text-to-motion generation (RVHM2D) and subsequent skeleton-driven [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** The architecture of our proposed RVHM2D model for human motion generation. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** The annotation and data cleaning pipeline for our Motion2D-Video-150K human motion 2D dataset. This pipeline [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Data Splits and Availability: The complete Motion2DVideo-150K dataset comprises 150,000 motion sequences. For evaluation purposes, we randomly sampled 11,682 sequences to constitute the test set. This test set is carefully balanced to reflect the diverse nature of ou…

**Figure 4.** Figure 4: An example from the Motion2D-Video-150K 2D rich motion dataset. The textual caption is generated by Gemini 2.0 [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Qualitative comparison of generated 2D human motions from different models: (a) InterGen, (b) InterGen with [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 30 canonical work pages

[1]

Bar-Tal, O.; Chefer, H.; Tov, O.; Herrmann, C.; Paiss, R.; Zada, S.; Ephrat, A.; Hur, J.; Liu, G.; Raj, A.; et al. 2024. Lumiere: A space-time diffusion model for video generation. In SIGGRAPH Asia 2024 Conference Papers, 1--11

work page 2024
[2]

Barquero, G.; Escalera, S.; and Palmero, C. 2023. Belfusion: Latent diffusion for behavior-driven human motion prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2317--2327

work page 2023
[3]

Cai, Z.; Jiang, J.; Qing, Z.; Guo, X.; Zhang, M.; Lin, Z.; Mei, H.; Wei, C.; Wang, R.; Yin, W.; et al. 2024. Digital life project: Autonomous 3d characters with social intelligence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 582--592

work page 2024
[4]

Cao, H.; Tan, C.; Gao, Z.; Xu, Y.; Chen, G.; Heng, P.-A.; and Li, S. Z. 2024. A survey on generative diffusion models. IEEE Transactions on Knowledge and Data Engineering

work page 2024
[5]

Chen, F.; Zhao, S.; Xu, C.; and Lan, L. 2025. JointTuner: Appearance-Motion Adaptive Joint Training for Customized Video Generation. arXiv preprint arXiv:2503.23951

arXiv 2025
[6]

Chen, X.; Jiang, B.; Liu, W.; Huang, Z.; Fu, B.; Chen, T.; and Yu, G. 2023. Executing your Commands via Motion Diffusion in Latent Space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18000--18010

work page 2023
[7]

Chung, J.; Wuu, C.-h.; Yang, H.-r.; Tai, Y.-W.; and Tang, C.-K. 2021. Haa500: Human-centric atomic action dataset with curated videos. In Proceedings of the IEEE/CVF international conference on computer vision, 13465--13474

work page 2021
[8]

Cideron, G.; Girgin, S.; Verzetti, M.; Vincent, D.; Kastelic, M.; Borsos, Z.; McWilliams, B.; Ungureanu, V.; Bachem, O.; Pietquin, O.; et al. 2024. Musicrl: Aligning music generation to human preferences. arXiv preprint arXiv:2402.04229

arXiv 2024

Show all 55 references

[9]

M.; Kim, N.; Bitton, Y.; Rieser, V.; Omidshafiei, S.; Hu, Y.; Chen, S.; Dutta, S.; Chang, M.; Lee, K.; et al

Collins, K. M.; Kim, N.; Bitton, Y.; Rieser, V.; Omidshafiei, S.; Hu, Y.; Chen, S.; Dutta, S.; Chang, M.; Lee, K.; et al. 2024. Beyond Thumbs Up/Down: Untangling Challenges of Fine-Grained Feedback for Text-to-Image Generation. In Proceedings of the AAAI/ACM Conference on AI, ...

2024
[10]

Dai, W.; Chen, L.-H.; Wang, J.; Liu, J.; Dai, B.; and Tang, Y. 2024. Motionlcm: Real-time controllable motion generation via latent consistency model. In European Conference on Computer Vision, 390--408. Springer

2024
[11]

DeepMind, G. 2025. Gemini 2.0 Flash. url = https://deepmind.google/technologies/gemini/flash/ . Gemini 2.0 Flash model released by Google DeepMind

2025
[12]

B.; and Dynkin, E

Dynkin, E. B.; and Dynkin, E. B. 1965. Markov processes. Springer

1965
[13]

Ethayarajh, K.; Xu, W.; Muennighoff, N.; Jurafsky, D.; and Kiela, D. 2024. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306

2024 arXiv
[14]

Feng, M.; Liu, J.; Yu, K.; Yao, Y.; Hui, Z.; Guo, X.; Lin, X.; Xue, H.; Shi, C.; Li, X.; et al. 2023. Dreamoving: A human video generation framework based on diffusion models. arXiv preprint arXiv:2312.05107

2023 arXiv
[15]

GPT4o. 2024. https://openai.com/index/hello-gpt-4o/

2024
[16]

G.; Wang, S.; and Cheng, L

Guo, C.; Mu, Y.; Javed, M. G.; Wang, S.; and Cheng, L. 2024. Momask: Generative masked modeling of 3d human motions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1900--1910

2024
[17]

Guo, C.; Zou, S.; Zuo, X.; Wang, S.; Ji, W.; Li, X.; and Cheng, L. 2022. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 5152--5161

2022
[18]

Ho, J.; Jain, A.; and Abbeel, P. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 6840--6851

2020
[19]

Hu, L. 2024. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8153--8163

2024
[20]

Jiang, B.; Chen, X.; Liu, W.; Yu, J.; Yu, G.; and Chen, T. 2023 a . Motiongpt: Human motion as a foreign language. Advances in Neural Information Processing Systems, 36: 20067--20079

2023
[21]

Jiang, T.; Lu, P.; Zhang, L.; Ma, N.; Han, R.; Lyu, C.; Li, Y.; and Chen, K. 2023 b . Rtmpose: Real-time multi-person pose estimation based on mmpose. arXiv preprint arXiv:2303.07399

2023 arXiv
[22]

Khirodkar, R.; Bagautdinov, T.; Martinez, J.; Zhaoen, S.; James, A.; Selednik, P.; Anderson, S.; and Saito, S. 2024. Sapiens: Foundation for human vision models. In European Conference on Computer Vision, 206--228. Springer

2024
[23]

Kong, W.; Tian, Q.; Zhang, Z.; Min, R.; Dai, Z.; Zhou, J.; Xiong, J.; Li, X.; Wu, B.; Zhang, J.; et al. 2024. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603

2024 arXiv
[24]

Li, R.; Xing, D.; Sun, H.; Ha, Y.; Shen, J.; and Ho, C. 2025. TokenMotion: Decoupled Motion Control via Token Disentanglement for Human-centric Video Generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, 1951--1961

2025
[25]

Li, Z.; Yuan, W.; He, Y.; Qiu, L.; Zhu, S.; Gu, X.; Shen, W.; Dong, Y.; Dong, Z.; and Yang, L. T. 2024. LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioning. arXiv preprint arXiv:2410.07093

2024 arXiv
[26]

Liang, H.; Zhang, W.; Li, W.; Yu, J.; and Xu, L. 2024. Intergen: Diffusion-based multi-human motion generation under complex interactions. International Journal of Computer Vision, 1--21

2024
[27]

Ma, Y.; He, Y.; Cun, X.; Wang, X.; Chen, S.; Li, X.; and Chen, Q. 2024. Follow your pose: Pose-guided text-to-video generation using pose-free videos. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 4117--4125

2024
[28]

Peng, B.; Chen, X.; Wang, Y.; Lu, C.; and Qiao, Y. 2024. ConditionVideo: training-free condition-guided video generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 4459--4467

2024
[29]

Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; and Liu, P. J. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140): 1--67

2020
[30]

Ramesh, A.; Dhariwal, P.; Nichol, A.; Chu, C.; and Chen, M. 2022. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2): 3

2022 arXiv
[31]

Ruiz, N.; Li, Y.; Jampani, V.; Pritch, Y.; Rubinstein, M.; and Aberman, K. 2023. Dreambooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 22500--22510

2023
[32]

Shafir, Y.; Tevet, G.; Kapon, R.; and Bermano, A. H. 2023. Human motion diffusion as a generative prior. arXiv preprint arXiv:2303.01418

2023 arXiv
[33]

Song, J.; Meng, C.; and Ermon, S. 2020. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502

2020 arXiv
[34]

R.; and Shah, M

Soomro, K.; Zamir, A. R.; and Shah, M. 2012. UCF101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402

2012 arXiv
[35]

Sun, H.; Zheng, R.; Huang, H.; Ma, C.; Huang, H.; and Hu, R. 2024. LGTM: Local-to-Global Text-Driven Human Motion Diffusion Model. In ACM SIGGRAPH 2024 Conference Papers, 1--9

2024
[36]

Tan, S.; Gong, B.; Wang, X.; Zhang, S.; Zheng, D.; Zheng, R.; Zheng, K.; Chen, J.; and Yang, M. 2024. Animate-X: Universal Character Image Animation with Enhanced Motion Representation. arXiv preprint arXiv:2410.10306

2024 arXiv
[37]

Tevet, G.; Raab, S.; Gordon, B.; Shafir, Y.; Cohen-or, D.; and Bermano, A. H. 2023. Human Motion Diffusion Model. In The Eleventh International Conference on Learning Representations

2023
[38]

Wallace, B.; Dang, M.; Rafailov, R.; Zhou, L.; Lou, A.; Purushwalkam, S.; Ermon, S.; Xiong, C.; Joty, S.; and Naik, N. 2024. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8228--8238

2024
[39]

Wan, T.; Wang, A.; Ai, B.; Wen, B.; Mao, C.; Xie, C.-W.; Chen, D.; Yu, F.; Zhao, H.; Yang, J.; et al. 2025. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314

2025 arXiv
[40]

Wang, B.; Wang, X.; Ni, C.; Zhao, G.; Yang, Z.; Zhu, Z.; Zhang, M.; Zhou, Y.; Chen, X.; Huang, G.; et al. 2025. HumanDreamer: Generating Controllable Human-Motion Videos via Decoupled Generation. arXiv preprint arXiv:2503.24026

2025 arXiv
[41]

Wang, Y.; Wang, Y.; Guo, D.; Chen, J.; Zhang, R.; Ma, Y.; and Zheng, Z. 2024 a . Rlcoder: Reinforcement learning for repository-level code completion. arXiv preprint arXiv:2407.19487

2024 arXiv
[42]

Wang, Y.; Wang, Z.; Gong, J.; Huang, D.; He, T.; Ouyang, W.; Jiao, J.; Feng, X.; Dou, Q.; Tang, S.; et al. 2024 b . Holistic-motion2d: Scalable whole-body human motion generation in 2d space. arXiv preprint arXiv:2406.11253

2024 arXiv
[43]

Wang, Z.; Li, A.; Zhu, L.; Guo, Y.; Dou, Q.; and Li, Z. 2024 c . Customvideo: Customizing text-to-video generation with multiple subjects. arXiv preprint arXiv:2401.09962

2024
[44]

Xu, J.; Liu, X.; Wu, Y.; Tong, Y.; Li, Q.; Ding, M.; Tang, J.; and Dong, Y. 2024. Imagereward: Learning and Evaluating Human Preferences for Text-to-Image Generation. Advances in Neural Information Processing Systems, 36

2024
[45]

Ye, J.; Xu, H.; Liu, H.; Hu, A.; Yan, M.; Qian, Q.; Zhang, J.; Huang, F.; and Zhou, J. 2024. mplug-owl3: Towards long image-sequence understanding in multi-modal large language models. arXiv preprint arXiv:2408.04840

2024 arXiv
[46]

H.; and Son, J

Yu, S.; Seo, P. H.; and Son, J. 2023. Zero-shot referring image segmentation with global-local context features. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 19456--19465

2023
[47]

Zhai, Y.; Lin, K.; Li, L.; Lin, C.-C.; Wang, J.; Yang, Z.; Doermann, D.; Yuan, J.; Liu, Z.; and Wang, L. 2024. Idol: Unified dual-modal latent diffusion for human-centric joint video-depth generation. In European Conference on Computer Vision, 134--152. Springer

2024
[48]

Zhang, C.; Zhang, C.; Zhang, M.; and Kweon, I. S. 2023. Text-to-Image Diffusion Models in Generative AI: A Survey. arXiv preprint arXiv:2303.07909

2023 arXiv
[49]

Zhang, L.; Rao, A.; and Agrawala, M. 2023. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 3836--3847

2023
[50]

Zhang, M.; Cai, Z.; Pan, L.; Hong, F.; Guo, X.; Yang, L.; and Liu, Z. 2024 a . Motiondiffuse: Text-driven human motion generation with diffusion model. IEEE Transactions on Pattern Analysis and Machine Intelligence

2024
[51]

Zhang, W.; Zhu, M.; and Derpanis, K. G. 2013. From actemes to action: A strongly-supervised representation for detailed action understanding. In Proceedings of the IEEE international conference on computer vision, 2248--2255

2013
[52]

Zhang, Y.; Gu, J.; Wang, L.-W.; Wang, H.; Cheng, J.; Zhu, Y.; and Zou, F. 2024 b . Mimicmotion: High-quality human motion video generation with confidence-aware pose guidance. arXiv preprint arXiv:2406.19680

2024 arXiv
[53]

Zhang, Y.; Huang, D.; Liu, B.; Tang, S.; Lu, Y.; Chen, L.; Bai, L.; Chu, Q.; Yu, N.; and Ouyang, W. 2024 c . Motiongpt: Finetuned llms are general-purpose motion generators. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 7368--7376

2024
[54]

Zhang, Z.; Liu, A.; Reid, I.; Hartley, R.; Zhuang, B.; and Tang, H. 2024 d . Motion mamba: Efficient and long sequence motion generation. In European Conference on Computer Vision, 265--282. Springer

2024
[55]

Zhao, S.; Chen, D.; Chen, Y.-C.; Bao, J.; Hao, S.; Yuan, L.; and Wong, K.-Y. K. 2024. Uni-controlnet: All-in-one control to text-to-image diffusion models. Advances in Neural Information Processing Systems, 36

2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Bar-Tal, O.; Chefer, H.; Tov, O.; Herrmann, C.; Paiss, R.; Zada, S.; Ephrat, A.; Hur, J.; Liu, G.; Raj, A.; et al. 2024. Lumiere: A space-time diffusion model for video generation. In SIGGRAPH Asia 2024 Conference Papers, 1--11

work page 2024

[2] [2]

Barquero, G.; Escalera, S.; and Palmero, C. 2023. Belfusion: Latent diffusion for behavior-driven human motion prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2317--2327

work page 2023

[3] [3]

Cai, Z.; Jiang, J.; Qing, Z.; Guo, X.; Zhang, M.; Lin, Z.; Mei, H.; Wei, C.; Wang, R.; Yin, W.; et al. 2024. Digital life project: Autonomous 3d characters with social intelligence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 582--592

work page 2024

[4] [4]

Cao, H.; Tan, C.; Gao, Z.; Xu, Y.; Chen, G.; Heng, P.-A.; and Li, S. Z. 2024. A survey on generative diffusion models. IEEE Transactions on Knowledge and Data Engineering

work page 2024

[5] [5]

Chen, F.; Zhao, S.; Xu, C.; and Lan, L. 2025. JointTuner: Appearance-Motion Adaptive Joint Training for Customized Video Generation. arXiv preprint arXiv:2503.23951

arXiv 2025

[6] [6]

Chen, X.; Jiang, B.; Liu, W.; Huang, Z.; Fu, B.; Chen, T.; and Yu, G. 2023. Executing your Commands via Motion Diffusion in Latent Space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18000--18010

work page 2023

[7] [7]

Chung, J.; Wuu, C.-h.; Yang, H.-r.; Tai, Y.-W.; and Tang, C.-K. 2021. Haa500: Human-centric atomic action dataset with curated videos. In Proceedings of the IEEE/CVF international conference on computer vision, 13465--13474

work page 2021

[8] [8]

Cideron, G.; Girgin, S.; Verzetti, M.; Vincent, D.; Kastelic, M.; Borsos, Z.; McWilliams, B.; Ungureanu, V.; Bachem, O.; Pietquin, O.; et al. 2024. Musicrl: Aligning music generation to human preferences. arXiv preprint arXiv:2402.04229

arXiv 2024