Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DanceTogether claims to be the first end-to-end diffusion framework that turns a single reference image plus per-person pose and mask streams into long, photorealistic two-person videos while strictly preserving every identity, by binding…

desk verdict A plausible multi-person animation method with a real new mechanism and, if released, useful datasets, but the evaluation does not actually test the paper's headline claim of strict identity preservation. read the letter →

arxiv 2505.18078 v1 pith:ULJ2WH2N submitted 2025-05-23 cs.CV

Junhao Chen , Mingjin Chen , Jianjin Xu , Xiang Li , Junting Dong , Mingze Sun , Puhua Jiang , Hongxiang Li
show 4 more authors
Yuhang Yang Hao Zhao Xiaoxiao Long Ruqi Huang
This is my paper
classification cs.CV
keywords controllablevideogenerationidentitypreservationmulti-personinteractiondiffusionmodelpose-guidedanimationmask-posefusionbenchmarkhuman-robot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DanceTogether claims that multi-person controllable video generation fails because existing systems never bind identity to motion in a persistent way: a frame-wise pipeline lets who a person is drift apart from what they are doing. The paper's fix is to feed the diffusion model two control streams per actor — a pose map that says how the body moves and a tracking mask that says where this particular person is — fused at every denoising step so the two can never decouple. To make the claim testable, the authors build their own two-person figure-skating dataset (PairFS-4K), a one-hour robot-interaction set, and a three-track benchmark on 100 clips spanning dance, boxing, wrestling, yoga, and skating. On that benchmark the method beats the strongest prior model by a wide margin, and the paper shows that the gains come specifically from the mask-pose fusion and identity tokens rather than from more training data alone.

What carries the argument

The MaskPoseAdapter is the load-bearing module: it encodes each person's RGB pose map with a shared PoseNet, runs binary tracking masks through a light 3-channel processor, gates both streams per-pixel with a learnable weight (λ≈0.8 favoring pose), enhances and LayerNorms the fused features, then applies temperature-scaled soft attention across the person dimension so each subject's identity and motion are bound in a single latent at every denoising step. It works alongside the MultiFace Encoder, which turns an ArcFace vector into K=4 Perceiver-refined identity tokens appended to the UNet's cross-attention keys/values, and a masked reconstruction loss that weights body regions 1× and face regions 2×. The paper's ablation shows that removing the adapter collapses HOTA from 83.94 to 48.95, which is the concrete evidence that the fusion — not just the extra data — carries the result.

What would settle it

Take the DanceTogEval-100 outputs and have human raters, or an independent detector suite never used in training, judge identity switches and pose faithfulness; if human-visible identity drift or detector-free judgment diverges from the HOTA/IDF1/MPJPE2D numbers, the benchmark's margin is partly an artifact of the model learning to be legible to SAMURAI-DWPose-OSNet rather than a true gain in identity preservation.

Watch

Extended reading notes

Core claim

The central claim is that an explicit, persistent binding between identity and action is what lets a diffusion model generate long, photorealistic multi-actor videos from a single reference image. DanceTogether implements this with three coupled modules: a MultiFace Encoder that compresses an ArcFace identity embedding into four refined tokens injected into every cross-attention layer; a MaskPoseAdapter that fuses per-person tracking masks with semantically rich but noisy pose heat-maps through gated, attention-reordered integration, producing a single pose-mask latent; and a StableAnimator-style video diffusion backbone with a human-tracking masked reconstruction loss that upweights face and body regions. Trained on the paper's curated multi-person data, the pipeline lifts HOTA from 71.35 to 83.94, cuts MPJPE2D from 1555 px to 492 px, and improves masked-region FVD from 29.0 to 17.1 relative to the strongest baseline on the new TogetherVideoBench.

Load-bearing premise

The reported identity and motion scores assume that the automated detectors used to measure them are neutral, accurate instruments, even though the same mask tracker, pose estimator, and ReID model also produced the training annotations those scores reward.

Editorial extensions

If this is right

  • Identity drift and appearance bleeding in two-person generation become tractable: the same single-reference-image interface used for single-person animation now works for pairs, including position exchanges and occlusion-heavy interactions.
  • Because the conditioning is per-person, a user can choreograph two actors independently and the model composes them into one coherent video, realizing what the paper calls compositionally controllable multi-actor interaction.
  • A one-hour fine-tune on the HumanRob-300 robot set yields human-robot interaction videos that retain each agent's identity, pointing to cheap cross-domain transfer for embodied-AI and human-robot interaction work.
  • TogetherVideoBench gives the field three standard tracks (identity consistency, interaction coherence, video quality), and the reported margins set the bar that future multi-person controllable video generation systems must beat.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The benchmark's detectors (SAMURAI masks, DWPose keypoints, OSNet ReID) also produced the training annotations, so the reported HOTA/IDF1 gains may partly measure how legible the generated video is to those specific models rather than to human viewers; a human-rating study or a held-out detector suite would separate the two effects.
  • The gated-fusion recipe is portable: any control task pairing a semantically rich but noisy signal with a sparse but reliable one (depth plus keypoint tracks, text prompts plus object masks, robot joints plus silhouettes) could reuse the same MaskPoseAdapter-style design.
  • If identity-action binding is the true cause of the gains, the framework should transfer to non-human multi-instance generation (multiple objects, multiple camera viewpoints of a scene) without architectural rethinking — a directly testable prediction of the paper's framing.
  • The ablation ordering (framework +8.91 HOTA on swing data versus PairFS-4K adding only +2.15 HOTA on top of full data) suggests that once interaction types are represented, the fusion machinery rather than raw data volume is the binding constraint.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes DanceTogether, an end-to-end diffusion framework for generating two-person interaction videos from a single reference image plus per-person pose and mask sequences. The method introduces a MaskPoseAdapter that fuses tracking masks with pose heatmaps to bind identity to action, and a MultiFace Encoder that injects identity tokens into cross-attention. The authors also contribute PairFS-4K and HumanRob-300 datasets, and a three-track benchmark (TogetherVideoBench) with 100 curated test clips. They report large improvements over prior baselines on identity-consistency (HOTA 83.94 vs. 71.35), interaction-coherence (MPJPE2D 492 vs. 1555 px), and video quality (masked FVD 17.1 vs. 29.0).

Significance. If the reported gains reflect genuine identity preservation, the work is a useful step toward controllable multi-person video generation: the MaskPoseAdapter design and the ablations isolating the mask input are informative, and the datasets/benchmark are a potential community resource. The paper also includes a same-data comparison (DanceTog w. Dataswing vs. StableAnimator w. Dataswing) that partially controls for training data, and the module ablation in Table 6 clearly shows the mask input is critical. However, the central claim of 'strictly preserving every identity' is not substantiated by the chosen metrics, and the evaluation pipeline's reliance on the same detectors used to create training annotations is a serious correctness risk.

major comments (3)
  1. [Sec. F.1, Eq. (21), Eq. (23)] The Identity-Consistency track does not measure identity preservation relative to the reference image. HOTA (Eq. 23) is sqrt(DetA * AssA) and IDF1 (Eq. 21) is a harmonic mean of ID precision/recall; both are computed from per-frame detections, ReID associations, and ground-truth trajectory labels. They measure whether the automated pipeline yields temporally stable, separable tracks, not whether each actor's appearance matches the reference identity. A generated video with the two identities swapped, or with both faces blended into an average appearance, can still produce stable separate tracks and hence high HOTA/IDF1. Consequently, Table 2 and the Abstract's 'strictly preserving every identity' and Sec. 3.3's 'eliminating appearance bleeding' are not supported by the reported evidence. Please add a reference-to-generated appearance metric (e.g., ArcFace cosine similarity between the reference face crop and each generated per-person face crop averaged over frames) and ideally a human perception study.
  2. [Sec. D.3, Sec. F.1, Eq. (25)] All three tracks rely on the same automated pipeline (DWPose, SAMURAI, OSNet) used to produce training annotations, which creates a circularity risk. MPJPE2D (Eq. 25) is computed from DWPose keypoints on generated frames; the model is trained to satisfy DWPose-derived pose conditions, so it may be optimized to be legible to DWPose rather than to produce accurate human pose. Similarly, Track-3 masked FVD/FID use SAMURAI masks on the generated video, and Track-1 uses OSNet ReID embeddings; all are the same detectors/embeddings used in Sec. D.3 to annotate training data. A model that learns to produce videos 'legible' to these exact detectors can score high even if human-visible identity and pose fidelity are worse. Please evaluate at least one independent detector/tracker (e.g., a different pose estimator and a different segmentation or ReID model) on a subset, or validate the automated scores against human ratings on a subset, and report the agreement.
  3. [Sec. 4.2, Tables 2–5] The headline comparisons are asymmetric. In Sec. 4.2 only StableAnimator is fine-tuned, on the swing-dance subset for 40 epochs; all other baselines are used with released weights, while DanceTogether is trained on the full multi-person dataset (Tab. 1) including the new PairFS-4K. The only controlled comparison that isolates method from data is 'DanceTog w. Dataswing' versus 'StableAnimator w. Dataswing' in Tables 2–5, which still shows substantial gains. But the prose and Abstract quote the fully trained model's numbers. Please either fine-tune all baselines on the same training data, or restrict the 'SOTA' claims to the same-data row, and discuss the contribution of additional training data explicitly.
minor comments (6)
  1. [Sec. 4.1] 'Pre-trained weights from StableAnimator [5]' cites reference [5] (Stable Video Diffusion); the StableAnimator reference is [79]. This citation error should be corrected.
  2. [Sec. 2.2] The sentence 'Tevet et al. [77] generate high-quality action sequences but lack robust identity modeling' describes a human motion diffusion model, not a video generation method; please rephrase to avoid confusion.
  3. [Tables 2–5] The tables report no confidence intervals or significance tests over generation seeds; given the stochastic nature of diffusion sampling and only 100 test clips, please report at least three seeds with error bars, or per-clip bootstrap intervals, for the main comparisons.
  4. [Sec. 3.3, Eqs. (7) and (10)] The text calls lambda a learnable weight and reports lambda approx 0.8, while s_p is set to 1.5 and alpha_res to 0.5. Please clarify which quantities are learned and which are fixed, and report the final learned values.
  5. [Secs. 3.5, D.2, E] The detection model is referred to as YOLOv8x in Sec. 3.5 and D.2 but as YOLOX in Sec. E; please make the detector name consistent.
  6. [Tables 2–5] Some table entries have formatting artifacts (e.g., 'DanceTog w.Data f ull+DataP airF S'); please clean the row labels for readability.

Circularity Check

1 steps flagged · score 4.0 of 10

Identity-consistency scores are measured with the same detector/tracker pipeline that generated the training conditions; the headline identity claim is not independently tested.

  1. fitted input called prediction [Sec. F.1 (TogetherVideoBench overview) and Sec. F.3 Track 1, Eqs. (21)-(24); training annotation in Sec. D.3]
    "All tracks share a unified Data Curation Pipeline that automatically extracts per-person pose, mask, face-crop, and bounding box for both ground truth and generated videos, ensuring reproducibility and fair comparison. ... We extract pose information of 133 keypoints using DW-Pose [107] and assign each pose to independent subject IDs via an IOU matching approach utilizing the masks generated by SAMURAI."

    The headline 'strictly preserving every identity' is scored with HOTA/IDF1 computed by running the same DWPose/SAMURAI/ReID pipeline on generated videos that was used to create the training masks, poses, and IDs. Eq. (23) defines HOTA as sqrt(DetA*AssA) over pipeline-extracted boxes/tracks, and Eq. (21) defines IDF1 over Hungarian-ID matches; neither references the reference image or the two ArcFace identities. A model optimized to be legible to these detectors (the same instruments that generated its targets) can inflate the scores through stable separable tracks even if identities are swapped or blended. The reported identity 'prediction' is therefore partially forced by the training/evaluation instrumentation rather than an independent test of reference-to-video identity preservation.

full rationale

There is no equation-level circularity in the diffusion architecture: MaskPoseAdapter fusion, MultiFace Encoder tokens, and the reconstruction loss are all stated with their own equations and are not derived from the benchmark numbers. The main circularity is confined to the evaluation of the central claim. Track 1's HOTA/IDF1 are computed from the unified Data Curation Pipeline that also generated DWPose/SAMURAI training annotations, and the metrics do not use the reference image, so the identity-consistency score partly measures detector legibility rather than identity preservation. Masked FVD/FID and qualitative stills provide some independent visual support, which keeps the paper from being wholly circular, but the paper does not report a reference-to-generated face similarity or human study, so the strongest claim rests on an instrument shared with the training targets. Accordingly the score is a moderate 4, not higher.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The central claim rests on no equation-level tautology; it is an empirical systems claim. The main burden is the domain assumption that the automated detection pipeline is a valid, neutral evaluator, plus a modest set of hand-chosen fusion coefficients in MaskPoseAdapter. The introduced datasets and benchmark are concrete artifacts with external availability, not postulated entities.

free parameters (8)
  • gate fusion weight lambda = learned, reported ~0.8
    Eq. (7); balances pose vs mask features in MaskPoseAdapter, directly controls identity-action binding strength.
  • residual fusion alpha_res = 0.5
    Eq. (8) and Sec. C.4; selected by hand after feature-map experiments.
  • pose feature scale s_p = 1.5
    Eq. (10); manually chosen before final integration.
  • final fusion weights = 0.95 and 0.05
    Eq. (15); fixed blend of attention-weighted and per-person mean features.
  • loss weights for body and face masks = 1 and 2
    Eq. (3); hand-chosen to emphasize face regions.
  • softmax temperature tau = learned
    Eq. (12); temperature-scaled cross-person attention.
  • mask processor output channels = 3
    Sec. C.4 and Fig. 9; chosen to keep mask features from dominating pose features.
  • identity tokens K = 4
    Eq. (17); number of Perceiver tokens per identity.
assumptions (5)
  • standard math Diffusion model training, cross-attention, and Perceiver building blocks as established in Stable Video Diffusion and StableAnimator.
    Assumed background throughout Sec. 3; the paper builds on these without re-deriving them.
  • domain assumption Per-person masks, 133-point poses, and ReID tracks can be reliably extracted from monocular RGB video via SAMURAI, DWPose, and OSNet.
    Used to build PairFS-4K and all training/evaluation annotations (Sec. D.2-D.3); if these tools fail under occlusion or blur, the identity-action binding is corrupted.
  • domain assumption ArcFace plus CLIP embeddings of a single reference image provide sufficient identity information for multiple individuals.
    MultiFace Encoder (Sec. 3.4) uses these features to preserve identity; face crops must be resolvable for both persons.
  • domain assumption Automated metrics computed from DWPose/SAMURAI outputs on generated videos faithfully measure identity preservation and interaction coherence.
    All benchmark tracks (Sec. F.1) rely on this; same detectors generated training targets, creating potential circularity.
  • domain assumption Evaluation domain assumes mostly static cameras and simple backgrounds.
    Stated limitation in Sec. A; the method is not claimed to handle dynamic camera or cluttered scenes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation." pith.science (2026). https://pith.science/paper/ULJ2WH2N

@misc{pith2026250518078,
  author       = {Pith},
  title        = {Pith review of: DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ULJ2WH2N}},
  note         = {Machine review of arXiv:2505.18078}
}
read the original abstract

Controllable video generation (CVG) has advanced rapidly, yet current systems falter when more than one actor must move, interact, and exchange positions under noisy control signals. We address this gap with DanceTogether, the first end-to-end diffusion framework that turns a single reference image plus independent pose-mask streams into long, photorealistic videos while strictly preserving every identity. A novel MaskPoseAdapter binds "who" and "how" at every denoising step by fusing robust tracking masks with semantically rich-but noisy-pose heat-maps, eliminating the identity drift and appearance bleeding that plague frame-wise pipelines. To train and evaluate at scale, we introduce (i) PairFS-4K, 26 hours of dual-skater footage with 7,000+ distinct IDs, (ii) HumanRob-300, a one-hour humanoid-robot interaction set for rapid cross-domain transfer, and (iii) TogetherVideoBench, a three-track benchmark centered on the DanceTogEval-100 test suite covering dance, boxing, wrestling, yoga, and figure skating. On TogetherVideoBench, DanceTogether outperforms the prior arts by a significant margin. Moreover, we show that a one-hour fine-tune yields convincing human-robot videos, underscoring broad generalization to embodied-AI and HRI tasks. Extensive ablations confirm that persistent identity-action binding is critical to these gains. Together, our model, datasets, and benchmark lift CVG from single-subject choreography to compositionally controllable, multi-actor interaction, opening new avenues for digital production, simulation, and embodied intelligence. Our video demos and code are available at https://DanceTog.github.io/.

Figures

Figures reproduced from arXiv: 2505.18078 by the authors.

Figure 1
Figure 1. DanceTogether generates complex two-person interaction videos with interactive details and consistent identity preservation from a single reference image (see the left-most of each row), using independent multi-person pose and mask sequences as control signals. Abstract Controllable video generation (CVG) has advanced rapidly, yet current systems falter when more than one actor must move, interact, and exchange posi… view at source ↗
Figure 2
Figure 2. DanceTogether pipeline overview: A single reference image and per-person pose/mask sequences enter the system; the MaskPoseAdapter fuses these control signals, the MultiFace Encoder injects identity tokens, and the video-diffusion backbone synthesizes an interaction video that preserves consistent identities for all actors. 3 Method 3.1 Overview: DanceTogether Pipeline Given a reference image Iref and per-person con… view at source ↗
Figure 3
Figure 3. Data Curation Pipeline Overview. Our pipeline processes raw videos through human [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (18 more)
Figure 4
Figure 4. Figure 4: The RGB image in the “Ref Image” row is the input reference frame, and the two pose [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Ablation study animation results (1/2) [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Ablation study animation results (2/2). are missing in some occluded frames. In contrast, the original PoseNet’s output makes it difficult to distinguish each individual pose, and pose features may be lost in occluded frames. C.4 Experiments on residual alpha and mask …
Figure 7
Figure 7. Figure 7: Comparison of PoseNet and MaskPoseAdapter outputs under identical frame inputs. [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: The effect of residual alpha on MaskPoseAdapter output. [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: The effect of different output channel numbers in the Light mask processor on [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: We use individual human masks for each person to conduct quantitative evaluation. The [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]
Figure 11
Figure 11. Figure 11: Additional animation results (1/4). The image with red borders is the reference images. [PITH_FULL_IMAGE:figures/full_fig_p027_11.png]
Figure 12
Figure 12. Figure 12: Additional animation results (2/4). The image with red borders is the reference images. [PITH_FULL_IMAGE:figures/full_fig_p027_12.png]
Figure 13
Figure 13. Figure 13: Additional animation results (3/4). The image with red borders is the reference images. [PITH_FULL_IMAGE:figures/full_fig_p028_13.png]
Figure 14
Figure 14. Figure 14: Additional animation results (4/4). The image with red borders is the reference images. [PITH_FULL_IMAGE:figures/full_fig_p028_14.png]
Figure 15
Figure 15. Figure 15: Additional animation results (4/18). 29 [PITH_FULL_IMAGE:figures/full_fig_p029_15.png]
Figure 16
Figure 16. Figure 16: Additional animation results (5/18). 30 [PITH_FULL_IMAGE:figures/full_fig_p030_16.png]
Figure 17
Figure 17. Figure 17: Additional animation results (8/18). 31 [PITH_FULL_IMAGE:figures/full_fig_p031_17.png]
Figure 18
Figure 18. Figure 18: Additional animation results (10/18). 32 [PITH_FULL_IMAGE:figures/full_fig_p032_18.png]
Figure 19
Figure 19. Figure 19: Additional animation results (11/18). 33 [PITH_FULL_IMAGE:figures/full_fig_p033_19.png]
Figure 20
Figure 20. Figure 20: Additional animation results (12/18). 34 [PITH_FULL_IMAGE:figures/full_fig_p034_20.png]
Figure 21
Figure 21. Figure 21: Using the first frame as the reference image, we perform inference on human–robot [PITH_FULL_IMAGE:figures/full_fig_p035_21.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Go to Zero: Towards Zero-shot Motion Generation with Million-scale Data

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A 7B text-to-motion model trained on the new 2M-clip MotionMillion dataset is reported to generalize zero-shot to complex, out-of-domain prompts.

  2. Animate-X++: Universal Character Image Animation with Dynamic Backgrounds

    cs.CV 2025-08 conditional novelty 5.0 of 10

    Animate-X++ turns cartoon images into pose-driven animations with text-controlled moving backgrounds, claiming state-of-the-art results on a new synthetic anthropomorphic benchmark.

Reference graph

Works this paper leans on

122 extracted references · 32 canonical work pages · cited by 2 Pith papers

  1. [1]

    Kling ai: Next-generation ai creative studio

    Kling AI. Kling ai: Next-generation ai creative studio. 10

  2. [2]

    Nil: No- data imitation learning by leveraging pre-trained video diffusion models.arXiv preprint arXiv:2503.10626, 2025

    Mert Albaba, Chenhao Li, Markos Diomataris, Omid Taheri, Andreas Krause, and Michael Black. Nil: No- data imitation learning by leveraging pre-trained video diffusion models.arXiv preprint arXiv:2503.10626, 2025

    arXiv 2025

  3. [3]

    Evaluating multiple object tracking performance: the clear mot metrics.EURASIP Journal on Image and Video Processing, 2008:1–10, 2008

    Keni Bernardin and Rainer Stiefelhagen. Evaluating multiple object tracking performance: the clear mot metrics.EURASIP Journal on Image and Video Processing, 2008:1–10, 2008

    2008

  4. [4]

    Gen2act: Human video generation in novel scenarios enables generalizable robot manipulation.arXiv preprint arXiv:2409.16283, 2024

    Homanga Bharadhwaj, Debidatta Dwibedi, Abhinav Gupta, Shubham Tulsiani, Carl Doersch, Ted Xiao, Dhruv Shah, Fei Xia, Dorsa Sadigh, and Sean Kirmani. Gen2act: Human video generation in novel scenarios enables generalizable robot manipulation.arXiv preprint arXiv:2409.16283, 2024

    arXiv 2024

  5. [5]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    arXiv 2023

  6. [6]

    Align your latents: High-resolution video synthesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22563– 22575, 2023

    2023

  7. [7]

    What are you doing? a closer look at controllable human video generation.arXiv preprint arXiv:2503.04666, 2025

    Emanuele Bugliarello, Anurag Arnab, Roni Paiss, Pieter-Jan Kindermans, and Cordelia Schmid. What are you doing? a closer look at controllable human video generation.arXiv preprint arXiv:2503.04666, 2025

    arXiv 2025

  8. [8]

    Deep video generation, prediction and completion of human action sequences

    Haoye Cai, Chunyan Bai, Yu-Wing Tai, and Chi-Keung Tang. Deep video generation, prediction and completion of human action sequences. InProceedings of the European conference on computer vision (ECCV), pages 366–382, 2018

    2018

Show all 122 references
  1. [9]

    Realtime multi-person 2d pose estimation using part affinity fields

    Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. Realtime multi-person 2d pose estimation using part affinity fields. InCVPR, 2017

    2017

  2. [10]

    Everybody dance now

    Caroline Chan, Shiry Ginosar, Tinghui Zhou, and Alexei A Efros. Everybody dance now. InProceedings of the IEEE/CVF international conference on computer vision, pages 5933–5942, 2019

    2019

  3. [11]

    A survey on evaluation of large language models.ACM transactions on intelligent systems and technology, 15(3):1–45, 2024

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A survey on evaluation of large language models.ACM transactions on intelligent systems and technology, 15(3):1–45, 2024

    2024

  4. [12]

    Idea23d: Collaborative lmm agents enable 3d model generation from interleaved multimodal inputs

    Junhao Chen, Xiang Li, Xiaojun Ye, Chao Li, Zhaoxin Fan, and Hao Zhao. Idea23d: Collaborative lmm agents enable 3d model generation from interleaved multimodal inputs. InProceedings of the 31st International Conference on Computational Linguistics, pages 4149–4166, 2025

    2025

  5. [13]

    Ultraman: Ultra-fast and high-resolution texture generation for 3d human reconstruction from a single image

    Mingjin Chen, Junhao Chen, Huan-ang Gao, Xiaoxue Chen, Zhaoxin Fan, and Hao Zhao. Ultraman: Ultra-fast and high-resolution texture generation for 3d human reconstruction from a single image. 2025

    2025

  6. [14]

    Follow-your-canvas: Higher-resolution video outpainting with extensive content generation.arXiv preprint arXiv:2409.01055, 2024

    Qihua Chen, Yue Ma, Hongfa Wang, Junkun Yuan, Wenzhe Zhao, Qi Tian, Hongmei Wang, Shaobo Min, Qifeng Chen, and Wei Liu. Follow-your-canvas: Higher-resolution video outpainting with extensive content generation.arXiv preprint arXiv:2409.01055, 2024

    2024 arXiv

  7. [15]

    Control3d: Towards controllable text-to-3d generation

    Yang Chen, Yingwei Pan, Yehao Li, Ting Yao, and Tao Mei. Control3d: Towards controllable text-to-3d generation. InProceedings of the 31st ACM International Conference on Multimedia, pages 1148–1156, 2023

    2023

  8. [16]

    Abo: Dataset and benchmarks for real-world 3d object understanding

    Jasmine Collins, Shubham Goel, Kenan Deng, Achleshwar Luthra, Leon Xu, Erhan Gundogdu, Xi Zhang, Tomas F Yago Vicente, Thomas Dideriksen, Himanshu Arora, et al. Abo: Dataset and benchmarks for real-world 3d object understanding. InProceedings of the IEEE/CVF conference on comp...

    2022

  9. [17]

    Arcface: Additive angular margin loss for deep face recognition

    Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4690–4699, 2019

    2019

  10. [18]

    Magicdance: Realistic human dance video generation with motions & facial expressions transfer.arXiv preprint arXiv:2311.12052, 2(3):4, 2023

    Yichun Shi Di Chang, Quankai Gao, Jessica Fu, Hongyi Xu, Guoxian Song, Qing Yan, Xiao Yang, and Mohammad Soleymani. Magicdance: Realistic human dance video generation with motions & facial expressions transfer.arXiv preprint arXiv:2311.12052, 2(3):4, 2023

    2023 arXiv

  11. [19]

    Image quality assessment: Unifying structure and texture similarity.IEEE transactions on pattern analysis and machine intelligence, 44(5):2567–2581, 2020

    Keyan Ding, Kede Ma, Shiqi Wang, and Eero P Simoncelli. Image quality assessment: Unifying structure and texture similarity.IEEE transactions on pattern analysis and machine intelligence, 44(5):2567–2581, 2020

    2020

  12. [20]

    A survey of embodied ai: From simulators to research tasks.IEEE Transactions on Emerging Topics in Computational Intelligence, 6(2):230–244, 2022

    Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan. A survey of embodied ai: From simulators to research tasks.IEEE Transactions on Emerging Topics in Computational Intelligence, 6(2):230–244, 2022

    2022

  13. [21]

    Dreamoving: A human video generation framework based on diffusion models

    Mengyang Feng, Jinlin Liu, Kai Yu, Yuan Yao, Zheng Hui, Xiefan Guo, Xianhui Lin, Haolan Xue, Chen Shi, Xiaowen Li, et al. Dreamoving: A human video generation framework based on diffusion models. arXiv preprint arXiv:2312.05107, 2023

    2023 arXiv

  14. [22]

    Reconstructing three-dimensional models of interacting humans.arXiv preprint arXiv:2308.01854, 2023

    Mihai Fieraru, Mihai Zanfir, Elisabeta Oneata, Alin-Ionut Popa, Vlad Olaru, and Cristian Sminchisescu. Reconstructing three-dimensional models of interacting humans.arXiv preprint arXiv:2308.01854, 2023

    2023 arXiv

  15. [23]

    Iw-bench: Evaluating large multimodal models for converting image-to-web

    Hongcheng Guo, Wei Zhang, Junhao Chen, Yaonan Gu, Jian Yang, Junjia Du, Binyuan Hui, Tianyu Liu, Jianxin Ma, Chang Zhou, and Zhoujun Li. Iw-bench: Evaluating large multimodal models for converting image-to-web. 2024

    2024

  16. [24]

    Reparo: Compositional 3d assets generation with differentiable 3d layout alignment.arXiv preprint arXiv:2405.18525, 2024

    Haonan Han, Rui Yang, Huan Liao, Jiankai Xing, Zunnan Xu, Xiaoming Yu, Junwei Zha, Xiu Li, and Wanhua Li. Reparo: Compositional 3d assets generation with differentiable 3d layout alignment.arXiv preprint arXiv:2405.18525, 2024. 11

    2024 arXiv

  17. [25]

    Controllable video generation with sparse trajectories

    Zekun Hao, Xun Huang, and Serge Belongie. Controllable video generation with sparse trajectories. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7854–7863, 2018

    2018

  18. [26]

    Clipscore: A reference-free evaluation metric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7514–7528, 2021

    2021

  19. [27]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In I. Guyon, U. V on Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editor...

    2017

  20. [28]

    Video diffusion models.Advances in Neural Information Processing Systems, 35:8633–8646, 2022

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models.Advances in Neural Information Processing Systems, 35:8633–8646, 2022

    2022

  21. [29]

    Animate anyone: Consistent and controllable image-to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8153–8163, 2024

    2024

  22. [30]

    Make it move: controllable image-to-video generation with text descriptions

    Yaosi Hu, Chong Luo, and Zhenzhong Chen. Make it move: controllable image-to-video generation with text descriptions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18219–18228, 2022

    2022

  23. [31]

    T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation.Advances in Neural Information Processing Systems, 36:78723–78747, 2023

    Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation.Advances in Neural Information Processing Systems, 36:78723–78747, 2023

    2023

  24. [32]

    Learning high fidelity depths of dressed humans by watching social media dance videos

    Yasamin Jafarian and Hyun Soo Park. Learning high fidelity depths of dressed humans by watching social media dance videos. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12753–12762, 2021

    2021

  25. [33]

    Yolo by ultralytics

    Glenn Jocher, Ayush Chaurasia, and Jing Qiu. Yolo by ultralytics. https://github.com/ ultralytics/ultralytics, 2023

    2023

  26. [34]

    Dreampose: Fashion image-to-video synthesis via stable diffusion

    Johanna Karras, Aleksander Holynski, Ting-Chun Wang, and Ira Kemelmacher-Shlizerman. Dreampose: Fashion image-to-video synthesis via stable diffusion. In2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 22623–22633. IEEE, 2023

    2023

  27. [35]

    Text2video-zero: Text-to-image diffusion models are zero-shot video generators

    Levon Khachatryan, Andranik Movsisyan, Vahram Tadevosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text-to-image diffusion models are zero-shot video generators. InProceedings of the IEEE/CVF International Conference on Computer V...

    2023

  28. [36]

    Harmony4d: A video dataset for in-the-wild close human interactions.Advances in Neural Information Processing Systems, 37:107270–107285, 2024

    Rawal Khirodkar, Jyun-Ting Song, Jinkun Cao, Zhengyi Luo, and Kris Kitani. Harmony4d: A video dataset for in-the-wild close human interactions.Advances in Neural Information Processing Systems, 37:107270–107285, 2024

    2024

  29. [37]

    Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    2024 arXiv

  30. [38]

    World knowledge from ai image generation for robot control

    Jonas Krumme and Christoph Zetzsche. World knowledge from ai image generation for robot control. arXiv preprint arXiv:2503.16579, 2025

    2025 arXiv

  31. [39]

    Collaborative video diffusion: Consistent multi-video generation with camera control.Advances in Neural Information Processing Systems, 37:16240–16271, 2024

    Zhengfei Kuang, Shengqu Cai, Hao He, Yinghao Xu, Hongsheng Li, Leonidas J Guibas, and Gordon Wet- zstein. Collaborative video diffusion: Consistent multi-video generation with camera control.Advances in Neural Information Processing Systems, 37:16240–16271, 2024

    2024

  32. [40]

    Interactive control of avatars animated with human motion data

    Jehee Lee, Jinxiang Chai, Paul SA Reitsma, Jessica K Hodgins, and Nancy S Pollard. Interactive control of avatars animated with human motion data. InProceedings of the 29th annual conference on Computer graphics and interactive techniques, pages 491–500, 2002

    2002

  33. [41]

    Dispose: Disentangling pose guidance for controllable human image animation

    Hongxiang Li, Yaowei Li, Yuhang Yang, Junjie Cao, Zhihong Zhu, Xuxin Cheng, and Long Chen. Dispose: Disentangling pose guidance for controllable human image animation. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  34. [42]

    Magicmotion: Controllable video generation with dense-to-sparse trajectory guidance.arXiv preprint arXiv:2503.16421, 2025

    Quanhao Li, Zhen Xing, Rui Wang, Hui Zhang, Qi Dai, and Zuxuan Wu. Magicmotion: Controllable video generation with dense-to-sparse trajectory guidance.arXiv preprint arXiv:2503.16421, 2025

    2025

  35. [43]

    Ai choreographer: Music conditioned 3d dance generation with aist++

    Ruilong Li, Shan Yang, David A Ross, and Angjoo Kanazawa. Ai choreographer: Music conditioned 3d dance generation with aist++. InProceedings of the IEEE/CVF international conference on computer vision, pages 13401–13412, 2021

    2021

  36. [44]

    Lodge: A coarse to fine diffusion network for long dance generation guided by the characteristic dance primitives

    Ronghui Li, YuXiang Zhang, Yachao Zhang, Hongwen Zhang, Jie Guo, Yan Zhang, Yebin Liu, and Xiu Li. Lodge: A coarse to fine diffusion network for long dance generation guided by the characteristic dance primitives. InProceedings of the IEEE/CVF Conference on Computer Vision and...

    2024

  37. [45]

    Finedance: A fine-grained choreography dataset for 3d full body dance generation

    Ronghui Li, Junfan Zhao, Yachao Zhang, Mingyang Su, Zeping Ren, Han Zhang, Yansong Tang, and Xiu Li. Finedance: A fine-grained choreography dataset for 3d full body dance generation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 10234–10243, 2023

    2023

  38. [46]

    Evaluation of text-to-video generation models: A dynamics perspective.Advances in Neural Information Processing Systems, 37:109790–109816, 2024

    Mingxiang Liao, Qixiang Ye, Wangmeng Zuo, Fang Wan, Tianyu Wang, Yuzhong Zhao, Jingdong Wang, Xinyu Zhang, et al. Evaluation of text-to-video generation models: A dynamics perspective.Advances in Neural Information Processing Systems, 37:109790–109816, 2024

    2024

  39. [47]

    Open-sora plan: Open-source large video generation model.arXiv 12 preprint arXiv:2412.00131, 2024

    Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al. Open-sora plan: Open-source large video generation model.arXiv 12 preprint arXiv:2412.00131, 2024

    2024 arXiv

  40. [48]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InEuropean Conference on Computer Vision (ECCV), pages 740–755, 2014

    2014

  41. [49]

    Rich: Robust implicit clothed humans reconstruction from multi-scale spatial cues

    Yukang Lin, Ronghui Li, Kedi Lyu, Yachao Zhang, and Xiu Li. Rich: Robust implicit clothed humans reconstruction from multi-scale spatial cues. InChinese Conference on Pattern Recognition and Computer Vision (PRCV), pages 193–206. Springer, 2023

    2023

  42. [50]

    Fr \’echet video motion distance: A metric for evaluating motion consistency in videos.arXiv preprint arXiv:2407.16124, 2024

    Jiahe Liu, Youran Qu, Qi Yan, Xiaohui Zeng, Lele Wang, and Renjie Liao. Fr \’echet video motion distance: A metric for evaluating motion consistency in videos.arXiv preprint arXiv:2407.16124, 2024

    2024 arXiv

  43. [51]

    Evalcrafter: Benchmarking and evaluating large video generation models

    Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang, Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng, Raymond Chan, and Ying Shan. Evalcrafter: Benchmarking and evaluating large video generation models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...

    2024

  44. [52]

    Fetv: A benchmark for fine-grained evaluation of open-domain text-to-video generation.Advances in Neural Information Processing Systems, 36:62352–62387, 2023

    Yuanxin Liu, Lei Li, Shuhuai Ren, Rundong Gao, Shicheng Li, Sishuo Chen, Xu Sun, and Lu Hou. Fetv: A benchmark for fine-grained evaluation of open-domain text-to-video generation.Advances in Neural Information Processing Systems, 36:62352–62387, 2023

    2023

  45. [53]

    Hota: A higher order metric for evaluating multi-object tracking.International Journal of Computer Vision, 129(2):548–578, 2021

    Jonathon Luiten, Aljosa Osep, Patrick Dendorfer, Laura Leal-Taixé, Daniel Cremers, Ian Reid, Stefan Roth, Simone Milani, Alexander Kirillov, and Paul V oigtlaender. Hota: A higher order metric for evaluating multi-object tracking.International Journal of Computer Vision, 129(2...

    2021

  46. [54]

    Dreamactor-m1: Holistic, expressive and robust human image animation with hybrid guidance.arXiv preprint arXiv:2504.01724, 2025

    Yuxuan Luo, Zhengkun Rong, Lizhen Wang, Longhao Zhang, Tianshu Hu, and Yongming Zhu. Dreamactor-m1: Holistic, expressive and robust human image animation with hybrid guidance.arXiv preprint arXiv:2504.01724, 2025

    2025 arXiv

  47. [55]

    Notice of removal: Videofusion: Decomposed diffusion models for high-quality video generation

    Zhengxiong Luo, Dayou Chen, Yingya Zhang, Yan Huang, Liang Wang, Yujun Shen, Deli Zhao, Jingren Zhou, and Tieniu Tan. Notice of removal: Videofusion: Decomposed diffusion models for high-quality video generation. In2023 IEEE/CVF Conference on Computer Vision and Pattern Recogn...

    2023

  48. [56]

    Follow your pose: Pose-guided text-to-video generation using pose-free videos

    Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow your pose: Pose-guided text-to-video generation using pose-free videos. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 4117–4125, 2024

    2024

  49. [57]

    Foundation models for video understanding: A survey.Authorea Preprints, 2024

    Neelu Madan, Andreas Møgelmose, Rajat Modi, Yogesh S Rawat, and Thomas B Moeslund. Foundation models for video understanding: A survey.Authorea Preprints, 2024

    2024

  50. [58]

    Synergy and synchrony in couple dances.arXiv preprint arXiv:2409.04440, 2024

    V ongani Maluleke, Lea Müller, Jathushan Rajasegaran, Georgios Pavlakos, Shiry Ginosar, Angjoo Kanazawa, and Jitendra Malik. Synergy and synchrony in couple dances.arXiv preprint arXiv:2409.04440, 2024

    2024 arXiv

  51. [59]

    Benchmarking counterfactual image generation.Advances in Neural Information Processing Systems, 37:133207–133230, 2024

    Thomas Melistas, Nikos Spyrou, Nefeli Gkouti, Pedro Sanchez, Athanasios Vlontzos, Yannis Panagakis, Giorgos Papanastasiou, and Sotirios Tsaftaris. Benchmarking counterfactual image generation.Advances in Neural Information Processing Systems, 37:133207–133230, 2024

    2024

  52. [60]

    Efficient motion weighted spatio-temporal video ssim index

    Anush K Moorthy and Alan C Bovik. Efficient motion weighted spatio-temporal video ssim index. In Human Vision and Electronic Imaging XV, volume 7527, pages 440–448. SPIE, 2010

    2010

  53. [61]

    Sora: Creating video from text.https://openai.com/sora, 2024

    OpenAI. Sora: Creating video from text.https://openai.com/sora, 2024. Accessed: 2024-06-01

    2024

  54. [62]

    Controlnext: Powerful and efficient control for image and video generation.arXiv preprint arXiv:2408.06070, 2024

    Bohao Peng, Jian Wang, Yuechen Zhang, Wenbo Li, Ming-Chang Yang, and Jiaya Jia. Controlnext: Powerful and efficient control for image and video generation.arXiv preprint arXiv:2408.06070, 2024

    2024 arXiv

  55. [63]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. In The Twelfth International Conference on Learning Representations, 2024

    2024

  56. [64]

    Dreamfusion: Text-to-3d using 2d diffusion.arXiv preprint arXiv:2209.14988, 2022

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion.arXiv preprint arXiv:2209.14988, 2022

    2022 arXiv

  57. [65]

    Worldsimbench: Towards video generation models as world simulators.arXiv preprint arXiv:2410.18072, 2024

    Yiran Qin, Zhelun Shi, Jiwen Yu, Xijun Wang, Enshen Zhou, Lijun Li, Zhenfei Yin, Xihui Liu, Lu Sheng, Jing Shao, et al. Worldsimbench: Towards video generation models as world simulators.arXiv preprint arXiv:2410.18072, 2024

    2024 arXiv

  58. [66]

    Performance measures and a data set for multi-target, multi-camera tracking

    Ergys Ristani, Francesco Solera, Roger Zou, Rita Cucchiara, and Carlo Tomasi. Performance measures and a data set for multi-target, multi-camera tracking. InEuropean Conference on Computer Vision (ECCV), pages 17–35, 2016

    2016

  59. [67]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InCVPR, 2022

    2022

  60. [68]

    Motion-i2v: Consistent and controllable image-to- video generation with explicit motion modeling

    Xiaoyu Shi, Zhaoyang Huang, Fu-Yun Wang, Weikang Bian, Dasong Li, Yi Zhang, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, et al. Motion-i2v: Consistent and controllable image-to- video generation with explicit motion modeling. InACM SIGGRAPH 2024 Conference Papers, pa...

    2024

  61. [69]

    Make-a-video: Text-to-video generation without text-video data.arXiv preprint arXiv:2209.14792, 2022

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data.arXiv preprint arXiv:2209.14792, 2022

    2022 arXiv

  62. [70]

    Directorllm for human-centric video generation.arXiv preprint arXiv:2412.14484, 2024

    Kunpeng Song, Tingbo Hou, Zecheng He, Haoyu Ma, Jialiang Wang, Animesh Sinha, Sam Tsai, Yaqiao Luo, Xiaoliang Dai, Li Chen, et al. Directorllm for human-centric video generation.arXiv preprint arXiv:2412.14484, 2024

    2024 arXiv

  63. [71]

    Transnet v2: An effective deep network architecture for fast shot transition detection

    Tomás Soucek and Jakub Lokoc. Transnet v2: An effective deep network architecture for fast shot transition detection. InProceedings of the 32nd ACM International Conference on Multimedia, pages 13 11218–11221, 2024

    2024

  64. [72]

    T2v-compbench: A comprehensive benchmark for compositional text-to-video generation.arXiv preprint arXiv:2407.14505, 2024

    Kaiyue Sun, Kaiyi Huang, Xian Liu, Yue Wu, Zihan Xu, Zhenguo Li, and Xihui Liu. T2v-compbench: A comprehensive benchmark for compositional text-to-video generation.arXiv preprint arXiv:2407.14505, 2024

    2024 arXiv

  65. [73]

    Journeydb: A benchmark for generative image understanding.Advances in neural information processing systems, 36:49659–49678, 2023

    Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, et al. Journeydb: A benchmark for generative image understanding.Advances in neural information processing systems, 36:49659–49678, 2023

    2023

  66. [74]

    Drive: Diffusion-based rigging empowers generation of versatile and expressive characters.arXiv preprint arXiv:2411.17423, 2024

    Mingze Sun, Junhao Chen, Junting Dong, Yurun Chen, Xinyu Jiang, Shiwei Mao, Puhua Jiang, Jingbo Wang, Bo Dai, and Ruqi Huang. Drive: Diffusion-based rigging empowers generation of versatile and expressive characters.arXiv preprint arXiv:2411.17423, 2024

    2024 arXiv

  67. [75]

    Beyond talking–generating holistic 3d human dyadic motion for communication.International Journal of Computer Vision, 133(5):2910–2926, 2025

    Mingze Sun, Chao Xu, Xinyu Jiang, Yang Liu, Baigui Sun, and Ruqi Huang. Beyond talking–generating holistic 3d human dyadic motion for communication.International Journal of Computer Vision, 133(5):2910–2926, 2025

    2025

  68. [76]

    Video understanding with large language models: A survey.IEEE Transactions on Circuits and Systems for Video Technology, 2025

    Yunlong Tang, Jing Bi, Siting Xu, Luchuan Song, Susan Liang, Teng Wang, Daoan Zhang, Jie An, Jingyang Lin, Rongyi Zhu, et al. Video understanding with large language models: A survey.IEEE Transactions on Circuits and Systems for Video Technology, 2025

    2025

  69. [77]

    Human motion diffusion model.arXiv preprint arXiv:2209.14916, 2022

    Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen-Or, and Amit H Bermano. Human motion diffusion model.arXiv preprint arXiv:2209.14916, 2022

    2022 arXiv

  70. [78]

    Emo2: End-effector guided audio-driven avatar video generation.arXiv preprint arXiv:2501.10687, 2025

    Linrui Tian, Siqi Hu, Qi Wang, Bang Zhang, and Liefeng Bo. Emo2: End-effector guided audio-driven avatar video generation.arXiv preprint arXiv:2501.10687, 2025

    2025 arXiv

  71. [79]

    Stablean- imator: High-quality identity-preserving human image animation.arXiv preprint arXiv:2411.17697, 2024

    Shuyuan Tu, Zhen Xing, Xintong Han, Zhi-Qi Cheng, Qi Dai, Chong Luo, and Zuxuan Wu. Stablean- imator: High-quality identity-preserving human image animation.arXiv preprint arXiv:2411.17697, 2024

    2024 arXiv

  72. [80]

    Fvd: A new metric for video generation

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. Fvd: A new metric for video generation. 2019

    2019

  73. [81]

    Articulated mesh animation from multi-view silhouettes

    Daniel Vlasic, Ilya Baran, Wojciech Matusik, and Jovan Popovi ´c. Articulated mesh animation from multi-view silhouettes. InAcm Siggraph 2008 papers, pages 1–9. 2008

    2008

  74. [82]

    This&that: Language-gesture controlled video generation for robot planning.arXiv preprint arXiv:2407.05530, 2024

    Boyang Wang, Nikhil Sridhar, Chao Feng, Mark Van der Merwe, Adam Fishman, Nima Fazeli, and Jeong Joon Park. This&that: Language-gesture controlled video generation for robot planning.arXiv preprint arXiv:2407.05530, 2024

    2024 arXiv

  75. [83]

    Cove: Unleashing the diffusion feature correspondence for consistent video editing.arXiv preprint arXiv:2406.08850, 2024

    Jiangshan Wang, Yue Ma, Jiayi Guo, Yicheng Xiao, Gao Huang, and Xiu Li. Cove: Unleashing the diffusion feature correspondence for consistent video editing.arXiv preprint arXiv:2406.08850, 2024

    2024 arXiv

  76. [84]

    Taming rectified flow for inversion and editing.arXiv preprint arXiv:2411.04746, 2024

    Jiangshan Wang, Junfu Pu, Zhongang Qi, Jiayi Guo, Yue Ma, Nisha Huang, Yuxin Chen, Xiu Li, and Ying Shan. Taming rectified flow for inversion and editing.arXiv preprint arXiv:2411.04746, 2024

    2024 arXiv

  77. [85]

    Vividpose: Advancing stable video diffusion for realistic human image animation.arXiv preprint arXiv:2405.18156, 2024

    Qilin Wang, Zhengkai Jiang, Chengming Xu, Jiangning Zhang, Yabiao Wang, Xinyi Zhang, Yun Cao, Weijian Cao, Chengjie Wang, and Yanwei Fu. Vividpose: Advancing stable video diffusion for realistic human image animation.arXiv preprint arXiv:2405.18156, 2024

    2024 arXiv

  78. [86]

    Disco: Disentangled control for realistic human dance generation

    Tan Wang, Linjie Li, Kevin Lin, Yuanhao Zhai, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, and Lijuan Wang. Disco: Disentangled control for realistic human dance generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (C...

    2024

  79. [87]

    Unianimate: Taming unified video diffusion models for consistent human image animation

    Xiang Wang, Shiwei Zhang, Changxin Gao, Jiayu Wang, Xiaoqiang Zhou, Yingya Zhang, Luxin Yan, and Nong Sang. Unianimate: Taming unified video diffusion models for consistent human image animation. Science China Information Sciences, 2025

    2025

  80. [88]

    Unianimate-dit: Human image animation with large-scale video diffusion transformer.arXiv preprint arXiv:2504.11289, 2025

    Xiang Wang, Shiwei Zhang, Longxiang Tang, Yingya Zhang, Changxin Gao, Yuehuan Wang, and Nong Sang. Unianimate-dit: Human image animation with large-scale video diffusion transformer.arXiv preprint arXiv:2504.11289, 2025

    2025

  81. [89]

    Instructavatar: Text-guided emotion and motion control for avatar generation

    Yuchi Wang, Junliang Guo, Jianhong Bai, Runyi Yu, Tianyu He, Xu Tan, Xu Sun, and Jiang Bian. Instructavatar: Text-guided emotion and motion control for avatar generation. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 8132–8140, 2025

    2025

  82. [90]

    Bovik, H.R

    Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity.IEEE Transactions on Image Processing, 13(4):600–612, 2004

    2004

  83. [91]

    Humanvid: Demystifying training data for camera-controllable human image animation

    Zhenzhi Wang, Yixuan Li, Yanhong Zeng, Youqing Fang, Yuwei Guo, Wenran Liu, Jing Tan, Kai Chen, Tianfan Xue, Bo Dai, et al. Humanvid: Demystifying training data for camera-controllable human image animation. InThe Thirty-eight Conference on Neural Information Processing System...

    2024

  84. [92]

    Multi- identity human image animation with structural video diffusion.arXiv preprint arXiv:2504.04126, 2025

    Zhenzhi Wang, Yixuan Li, Yanhong Zeng, Yuwei Guo, Dahua Lin, Tianfan Xue, and Bo Dai. Multi- identity human image animation with structural video diffusion.arXiv preprint arXiv:2504.04126, 2025

    2025

  85. [93]

    Mo- tionctrl: A unified and flexible motion controller for video generation.arXiv preprint arXiv:2312.03641, 2023

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Mo- tionctrl: A unified and flexible motion controller for video generation.arXiv preprint arXiv:2312.03641, 2023

    2023 arXiv

  86. [94]

    Easyanimate: A high-performance long video generation method based on transformer architecture.arXiv preprint arXiv:2405.18991, 2024

    Jiaqi Xu, Xinyi Zou, Kunzhe Huang, Yunkuo Chen, Bo Liu, MengLi Cheng, Xing Shi, and Jun Huang. Easyanimate: A high-performance long video generation method based on transformer architecture.arXiv preprint arXiv:2405.18991, 2024

    2024

  87. [95]

    Xagen: 3d expressive human avatars generation.Advances in Neural Information Processing Systems, 36, 2024

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Jiashi Feng, and Mike Zheng Shou. Xagen: 3d expressive human avatars generation.Advances in Neural Information Processing Systems, 36, 2024. 14

    2024

  88. [96]

    Magicanimate: Temporally consistent human image animation using diffusion model

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human image animation using diffusion model. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition...

    2024

  89. [97]

    Magicanimate: Temporally consistent human image animation using diffusion model

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human image animation using diffusion model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2024

  90. [98]

    Human motion video generation: A survey.Authorea Preprints, 2024

    Haiwei Xue, Xiangyang Luo, Zhanghao Hu, Xin Zhang, Xunzhi Xiang, Yuqin Dai, Jianzhuang Liu, Zhensong Zhang, Minglei Li, Jian Yang, et al. Human motion video generation: A survey.Authorea Preprints, 2024

    2024

  91. [99]

    Follow-your-pose v2: Multiple-condition guided character image animation for stable pose control.arXiv preprint arXiv:2406.03035, 2024

    Jingyun Xue, Hongfa Wang, Qi Tian, Yue Ma, Andong Wang, Zhiyuan Zhao, Shaobo Min, Wenzhe Zhao, Kaihao Zhang, Heung-Yeung Shum, et al. Follow-your-pose v2: Multiple-condition guided character image animation for stable pose control.arXiv preprint arXiv:2406.03035, 2024

    2024 arXiv

  92. [100]

    Video quality assessment via gradient magnitude similarity deviation of spatial and spatiotemporal slices

    Peng Yan, Xuanqin Mou, and Wufeng Xue. Video quality assessment via gradient magnitude similarity deviation of spatial and spatiotemporal slices. InMobile Devices and Multimedia: Enabling Technologies, Algorithms, and Applications 2015, volume 9411, pages 182–191. SPIE, 2015

    2015

  93. [101]

    Dialoguenerf: Towards realistic avatar face-to-face conversation video generation.Visual Intelligence, 2(1):24, 2024

    Yichao Yan, Zanwei Zhou, Zi Wang, Jingnan Gao, and Xiaokang Yang. Dialoguenerf: Towards realistic avatar face-to-face conversation video generation.Visual Intelligence, 2(1):24, 2024

    2024

  94. [102]

    Pose guided human video generation

    Ceyuan Yang, Zhe Wang, Xinge Zhu, Chen Huang, Jianping Shi, and Dahua Lin. Pose guided human video generation. InProceedings of the European conference on computer vision (ECCV), pages 201–216, 2018

    2018

  95. [103]

    Samurai: Adapting segment anything model for zero-shot visual tracking with motion-aware memory.arXiv preprint arXiv:2411.11922, 2024

    Cheng-Yen Yang, Hsiang-Wei Huang, Wenhao Chai, Zhongyu Jiang, and Jenq-Neng Hwang. Samurai: Adapting segment anything model for zero-shot visual tracking with motion-aware memory.arXiv preprint arXiv:2411.11922, 2024

    2024 arXiv

  96. [104]

    MatAnyone: Stable video matting with consistent memory propagation

    Peiqing Yang, Shangchen Zhou, Jixin Zhao, Qingyi Tao, and Chen Change Loy. MatAnyone: Stable video matting with consistent memory propagation. InCVPR, 2025

    2025

  97. [105]

    Holodeck: Language guided generation of 3d embodied ai environments

    Yue Yang, Fan-Yun Sun, Luca Weihs, Eli VanderBilt, Alvaro Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingjie Liu, et al. Holodeck: Language guided generation of 3d embodied ai environments. InProceedings of the IEEE/CVF Conference on Computer Vision and Patte...

    2024

  98. [106]

    Idea2img: Iterative self-refinement with gpt-4v for automatic image design and generation

    Zhengyuan Yang, Jianfeng Wang, Linjie Li, Kevin Lin, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. Idea2img: Iterative self-refinement with gpt-4v for automatic image design and generation. InEuropean Conference on Computer Vision, pages 167–184. Springer, 2024

    2024

  99. [107]

    Effective whole-body pose estimation with two- stages distillation

    Zhendong Yang, Ailing Zeng, Chun Yuan, and Yu Li. Effective whole-body pose estimation with two- stages distillation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4210–4220, 2023

    2023

  100. [108]

    Mmad: Multi-modal movie audio description

    Xiaojun Ye, Junhao Chen, Xiang Li, Haidong Xin, Chao Li, Sheng Zhou, and Jiajun Bu. Mmad: Multi-modal movie audio description. InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 114...

    2024

  101. [109]

    Whac: World-grounded humans and cameras

    Wanqi Yin, Zhongang Cai, Ruisi Wang, Fanzhou Wang, Chen Wei, Haiyi Mei, Weiye Xiao, Zhitao Yang, Qingping Sun, Atsushi Yamashita, Lei Yang, and Ziwei Liu. Whac: World-grounded humans and cameras. InEuropean Conference on Computer Vision, pages 20–37. Springer, 2024

    2024

  102. [110]

    Hi4d: 4d instance segmentation of close human interaction

    Yifei Yin, Chen Guo, Manuel Kaufmann, Juan Jose Zarate, Jie Song, and Otmar Hilliges. Hi4d: 4d instance segmentation of close human interaction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17016–17027, 2023

    2023

  103. [111]

    Pose-guided human animation from a single image in the wild

    Jae Shin Yoon, Lingjie Liu, Vladislav Golyanik, Kripasindhu Sarkar, Hyun Soo Park, and Christian Theobalt. Pose-guided human animation from a single image in the wild. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15039–15048, 2021

    2021

  104. [112]

    Follow-your-multipose: Tuning-free multi-character text-to-video generation via pose guidance.arXiv preprint arXiv:2412.16495, 2024

    Beiyuan Zhang, Yue Ma, Chunlei Fu, Xinyang Song, Zhenan Sun, and Ziqiang Li. Follow-your-multipose: Tuning-free multi-character text-to-video generation via pose guidance.arXiv preprint arXiv:2412.16495, 2024

    2024 arXiv

  105. [113]

    Zhujiu: A multi-dimensional, multi-faceted chinese benchmark for large language models

    Baoli Zhang, Haining Xie, Pengfan Du, Junhao Chen, Pengfei Cao, Yubo Chen, Shengping Liu, Kang Liu, and Jun Zhao. Zhujiu: A multi-dimensional, multi-faceted chinese benchmark for large language models. InProceedings of the 2023 Conference on Empirical Methods in Natural Langua...

    2023

  106. [114]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023

    2023

  107. [115]

    Mimicmotion: High-quality human motion video generation with confidence-aware pose guidance.arXiv preprint arXiv:2406.19680, 2024

    Yuang Zhang, Jiaxi Gu, Li-Wen Wang, Han Wang, Junqi Cheng, Yuefeng Zhu, and Fangyuan Zou. Mimicmotion: High-quality human motion video generation with confidence-aware pose guidance.arXiv preprint arXiv:2406.19680, 2024

    2024 arXiv

  108. [116]

    Taste- rob: Advancing video generation of task-oriented hand-object interaction for generalizable robotic manipulation.arXiv preprint arXiv:2503.11423, 2025

    Hongxiang Zhao, Xingchen Liu, Mutian Xu, Yiming Hao, Weikai Chen, and Xiaoguang Han. Taste- rob: Advancing video generation of task-oriented hand-object interaction for generalizable robotic manipulation.arXiv preprint arXiv:2503.11423, 2025. 15

    2025 arXiv

  109. [117]

    Torchreid: A library for deep learning person re-identification in pytorch

    Kaiyang Zhou and Tao Xiang. Torchreid: A library for deep learning person re-identification in pytorch. arXiv preprint arXiv:1910.10093, 2019

    1910 arXiv

  110. [118]

    Omni-scale feature learning for person re-identification

    Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, and Tao Xiang. Omni-scale feature learning for person re-identification. InICCV, 2019

    2019

  111. [119]

    Learning generalisable omni-scale representations for person re-identification.TPAMI, 2021

    Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, and Tao Xiang. Learning generalisable omni-scale representations for person re-identification.TPAMI, 2021

    2021

  112. [120]

    Genimage: A million-scale benchmark for detecting ai-generated image

    Mingjian Zhu, Hanting Chen, Qiangyu Yan, Xudong Huang, Guanyu Lin, Wei Li, Zhijun Tu, Hailin Hu, Jie Hu, and Yunhe Wang. Genimage: A million-scale benchmark for detecting ai-generated image. Advances in Neural Information Processing Systems, 36:77771–77782, 2023

    2023

  113. [121]

    Champ: Controllable and consistent human image animation with 3d parametric guidance

    Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Zilong Dong, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. InEuropean Conference on Computer Vision, pages 145–162. Springer, 2024

    2024

  114. [122]

    Inference with HJB-based Face Optimization

    Tianyi Zhu, Dongwei Ren, Qilong Wang, Xiaohe Wu, and Wangmeng Zuo. Generative inbetweening through frame-wise conditions-driven video generation.arXiv preprint arXiv:2412.11755, 2024. A Limitations While DanceTogether achieves state-of-the-art performance on two-person interac...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.