Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

AnimateAnywhere: Rouse the Background in Human Image Animation

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that in human image animation, the background should move in harmony with the person, and that the human pose sequence alone carries enough information to infer plausible camera and background motion.

desk verdict A solid, well-engineered human animation paper that tackles a real gap—dynamic backgrounds without camera trajectories—but its central causality claim needs a direct test. read the letter →

arxiv 2504.19834 v2 pith:FICWSR3E submitted 2025-04-28 cs.CV

Xiaoyu Liu , Mingshuai Yao , Yabo Zhang , Xianhui Lin , Peiran Ren , Xiaoming Li , Ming Liu , Wangmeng Zuo This is my paper · ORCID
classification cs.CV
keywords humanimageanimationbackgroundmotionvideodiffusionmodelpose-drivenepipolarconstraintcamera-freeLoRA3Dattention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that in human image animation, the background should not stay frozen, and that the human pose sequence alone carries enough information to infer plausible camera and background motion. It introduces AnimateAnywhere, built on a pretrained image-to-video diffusion model, which animates a reference human and a reference background from a pose sequence without needing any camera trajectory at inference. The central bet is that relative changes in a person's size and position across frames reveal zoom, pan, and other camera motion, and a learned Background Motion Learner turns pose features into background motion. If this bet holds, ordinary users could generate videos with lively, coherent backgrounds from a single image pair plus pose data, matching what previously required camera estimation or manual trajectories.

What carries the argument

The two load-bearing components are the Background Motion Learner (BML) and the adaptive epipolar loss. BML is a LoRA module added to each DiT block of the base video diffusion model; it receives the sum of the denoising features and the human pose features extracted by a ControlNet, so the network can shift the background according to the implied camera motion. The epipolar loss uses the fundamental matrix from labeled camera poses during training to build, for each pixel, an epipolar line in every other frame; the model suppresses attention outside that line only when the vanilla attention score is below a 30th-percentile threshold ($(1-M_{ij}(u,v)) \cap (A_{ij}(u,v) < \delta)$), penalizing geometrically unreasonable cross-frame attention without destroying learned correlations. The combined effect is that geometry is injected as a soft training signal rather than as a control input at inference.

What would settle it

Take a fixed pose sequence of a person who is stationary in world space while the camera zooms in, and the same pose sequence with the person moving forward while the camera zooms out so the on-screen pose is identical; run AnimateAnywhere on both. If the generated background motion is the same or chosen arbitrarily, or if on a held-out set with known camera trajectories the predicted background motion disagrees with the recorded camera path for these ambiguous cases, the central claim that pose sequences determine background motion fails in exactly the regime the paper concedes.

Watch

Extended reading notes

Core claim

The paper claims that background motion can be learned directly from human pose sequences. Concretely, AnimateAnywhere takes a reference human image, a reference background image, and a pose sequence, and produces a video in which the background moves coherently with the human. The Background Motion Learner, a LoRA-style adapter inserted into each DiT block of the base model, reads combined pose and denoising features and predicts background motion end-to-end, avoiding the error accumulation of explicitly estimating camera trajectories. To keep that motion geometrically plausible, training applies an adaptive epipolar constraint to the 3D attention map: attention from a background pixel to another frame is suppressed only where it falls outside the epipolar line and is already low-confidence, leaving the model's own learned correlations intact. At inference no epipolar masks or camera poses are required. The paper reports that this outperforms prior human-animation methods on the Humanvid test set and a wild-collected BL200 set, including methods that use ground-truth camera poses at test time.

Load-bearing premise

The paper assumes that a human pose sequence alone reveals the background or camera motion, so that pose changes like apparent size and horizontal position translate into zoom and pan; this mapping is genuinely ambiguous in cases such as a skateboarder whose forward motion is indistinguishable from a camera zoom, and if such ambiguities dominate real usage, the learned background motion will be arbitrary or implausible.

Editorial extensions

If this is right

  • Users can animate a character with a dynamic background using only a reference human, a reference background, and a pose sequence; no camera trajectory or reference video is needed at inference.
  • Because the reference human and reference background are injected separately, the same pose sequence can be reused with arbitrary human and background combinations, enabling cross-character and cross-scene customization.
  • Reported quantitative gains over static-background baselines and over a camera-pose-based method that needs test-time camera poses suggest the pose-to-background shortcut is learnable from camera-annotated video.
  • The adaptive epipolar masking avoids the artifacts of hard suppression, making geometric consistency compatible with the free-form 3D attention of a diffusion transformer.
  • When a pose sequence is ambiguous, the method still commits to a plausible background, and the paper names text descriptions as the next step for disambiguation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit is that the BML's reliability should degrade precisely on pose sequences whose camera motion is underdetermined, such as a stationary person with a zoom; a text condition or a learned distribution over camera hypotheses would be a natural extension.
  • The same 'pose implies camera' principle could transfer to non-human subjects, where object size and position changes relative to the frame are equally informative about camera motion.
  • The low-confidence-gated epipolar mask is a general trick: any video-diffusion model with 3D attention could use it during training whenever camera labels are available, even for tasks outside human animation.
  • A testable question is whether BML actually learns geometry or merely copies common background motions from the training distribution; ablating with a pose sequence whose implied camera motion contradicts the training prior would settle this.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes AnimateAnywhere, a human image animation framework built on CogVideoX that generates videos with dynamic backgrounds while being controlled only by a human pose sequence and separate reference human and background images. The two main technical contributions are a Background Motion Learner (BML), a LoRA-based module that injects pose-derived features into the denoising DiT to predict background motion, and an adaptive epipolar constraint applied to the 3D attention map, which suppresses low-confidence attention that is geometrically unreasonable according to the camera motion available during training. The method is evaluated on the Humanvid test set and a new Bilibili200 (BL200) dataset, compared against MagicAnimate, AnimateAnyone, Champ, MusePose, DynamiCtrl, and Humanvid, and ablated with respect to the BML, the epipolar loss, the VGG loss, and the attention threshold. The authors report state-of-the-art quantitative results and qualitative examples showing plausible dynamic backgrounds.

Significance. If the central claim is validated, the paper makes a practical contribution by removing the need for camera trajectories or reference videos at inference, which broadens the applicability of human image animation to ordinary users. The architecture is clearly described, and the ablations do isolate the contributions of the BML and the epipolar loss, which is a genuine strength. The idea of learning background motion from pose features and of regularizing 3D attention with an adaptive epipolar mask is reasonable and worthy of publication. However, the paper's headline claim that the BML 'learns background motions from human pose sequences' is not directly supported by the reported evidence, and the paper's own Limitations section concedes a fundamental ambiguity in the pose-to-background mapping. The provided metrics are also based on single runs and a hyperparameter selected on the test set, so the strength of the empirical claims is currently not fully established.

major comments (3)
  1. [Section III-B, Table III] The central claim that the BML learns background motion from pose sequences is not directly supported by the reported experiments. Table III and Table I report only global frame-quality metrics (PSNR, SSIM, LPIPS, FID, FVD) aggregated over the entire video, so the gain from adding BML could come from extra model capacity, from the reference-background conditioning path, or from learning a generic camera-motion prior that is only weakly related to the specific pose content. A controlled experiment is needed: fix the reference human and reference background, vary only the pose sequence (for example, left-to-right walking versus in-place zooming, or a temporally scrambled pose sequence), and measure whether the generated background motion changes accordingly, for instance via optical-flow correlation between the background and the pose direction or via a human study. Section V's skateboard example shows that the pose-to-background mapping is ambiguous even in simple cases. Without such a test, the abstract's assertion that the method 'effectively learns the background motion from human pose sequences' remains unverified.
  2. [Section III-C, Eq. (4)] Equation (4) sums the epipolar loss over all pixels (u,v) without an explicit background/human mask, while the surrounding text says the constraint is applied to background pixels. Human body motion is not rigid under a global camera transformation, so applying the epipolar penalty to human-region queries would be geometrically incorrect and could distort the generated figure. The authors should state explicitly how the background region is identified in the loss, and either include the mask in Eq. (4) or define Aij(u,v) and Omega_ij(u,v) as restricted to background tokens. As written, the loss is ambiguous and cannot be reproduced from the paper.
  3. [Section IV-A, Table IV] The threshold delta is tuned on the Humanvid test set: Table IV reports the metrics for different percentiles and the authors then adopt the value that gives the best FVD and PSNR on that same set, and the final Table I numbers are produced with this selected value. This makes the headline comparison optimistic; a validation split or a sensitivity analysis that does not reuse the test set for model selection should be reported. Relatedly, all quantitative results are single runs without error bars or significance tests, and several differences from the nearest competitor are small (for example, PSNR 18.07 versus 17.91 in Table I), so run-to-run variance could change the ranking of methods.
minor comments (6)
  1. [Table II] The caption of Table II does not state that the numbers are computed on the subset of Humanvid videos with significant background motion, even though the text introduces such a subset; the table should be labeled accordingly to avoid confusion with Table I.
  2. [Table II] In Table II, the row 'Ours with Lepipolar((1-M) cap (A<delta))' reports SSIM = 0.5831, which is identical to the 'Ours' row in Table I while all other metrics differ; the authors should confirm whether this is a copying error and clarify the evaluation subset.
  3. [Section III-C and Fig. 2] The notation around the attention map is inconsistent: Aij(u,v) is sometimes a scalar and sometimes a map over the j-th frame, and the dimensions of Omega_ij(u,v) are not defined. The figures would be much easier to read if the masks and maps were defined with explicit tensor shapes.
  4. [Section III-D, Eq. (6)] The VGG perceptual loss in Eq. (6) uses a hand-region mask m, but the range and resolution of m and the choice of VGG features are not specified, which makes this term non-reproducible.
  5. [Related Work and Experiments] DynaScene [18], which is discussed in Section II-A as a method for realistic dynamic backgrounds, is not included in the quantitative or qualitative comparisons; if it requires camera poses at test time, this should be stated explicitly.
  6. [General] There are minor typos and formatting issues, including 'V AE' for 'VAE', inconsistent capitalization of 'AnimateAnyone', and the use of 'eq. (4)' without the equation being labeled consistently; a careful proofread would improve the presentation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper trains a learned background-motion module against ground-truth video and camera-pose supervision, and its self-referential attention masking is a regularization mechanism, not a definitional reduction.

full rationale

The paper's central mechanism is a learned Background Motion Learner that maps human pose features to background motion signals, trained end-to-end with a latent diffusion loss (Eq. 5), a VGG perceptual loss (Eq. 6), and an epipolar loss (Eq. 4) supervised by labeled camera trajectories. The epipolar constraint is applied during training only, and the paper explicitly states that at inference 'our method no longer requires epipolar information or camera trajectories' (Section III-C). This is standard supervised learning, not a fitted parameter renamed as a prediction. The adaptive epipolar mask uses the model's own 3D attention map and a threshold delta to decide which low-confidence attention values to suppress; this is a self-regularization design, not an equivalence between the input and the claimed output. The threshold is selected via sensitivity analysis on a test set, which is a methodological concern about evaluation, not circularity. The paper's citation of DynaScene [18] is related-work context and is not load-bearing for the proposed derivation. The Limitations section (Section V) concedes that a pose sequence can correspond to multiple plausible background motions; that is an acknowledged ambiguity in the task, not a circular step. No equation or claim in the paper reduces by construction to its own inputs, and no self-citation chain is used to justify a core result. Therefore the derivation is self-contained with respect to circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a domain assumption about pose-to-background information and on epipolar-geometry supervision from labeled camera poses. The main fitted parameters are the attention threshold delta, tuned on evaluation data, and the two loss weights set by hand.

free parameters (4)
  • delta (attention threshold) = 30th percentile
    Chosen as best on the evaluation set through sensitivity analysis in Table IV; controls how much attention is suppressed outside the epipolar mask.
  • lambda_vgg = 0.2
    Hand-set loss weight for VGG perceptual loss in Eq. (6)-(7).
  • lambda_epipolar = 0.005
    Hand-set loss weight for epipolar loss in Eq. (4) and (7).
  • CFG scale = 2.5
    Inference-time classifier-free guidance scale, set without reported sensitivity analysis.
assumptions (5)
  • standard math Epipolar geometry: a point in one frame must lie on the epipolar line in another frame defined by the fundamental matrix.
    Used to build the epipolar mask M_ij in Sec. III-C.
  • domain assumption Human pose sequences carry recoverable information about camera and background motion.
    The central premise stated in Sec. I and acknowledged as ambiguous in Sec. V (skateboard example).
  • domain assumption The 3D attention map in the diffusion model encodes cross-frame correspondences that can be regularized for geometric consistency.
    Required for the epipolar constraint to improve background quality in Sec. III-C.
  • domain assumption Camera poses annotated in Humanvid are accurate enough to supervise the epipolar loss.
    Training uses Humanvid camera poses via SLAM (Sec. IV-A); errors would misplace epipolar lines.
  • domain assumption Grounded-SAM human and background masks are reliable for separating foreground and background training signals.
    Used to construct reference human, reference background, and human motion sequences (Sec. IV-A).

how reviews work

0 comments
Cite this review

Pith. "Pith review of AnimateAnywhere: Rouse the Background in Human Image Animation." pith.science (2026). https://pith.science/paper/FICWSR3E

@misc{pith2026250419834,
  author       = {Pith},
  title        = {Pith review of: AnimateAnywhere: Rouse the Background in Human Image Animation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FICWSR3E}},
  note         = {Machine review of arXiv:2504.19834}
}
read the original abstract

Human image animation aims to generate human videos of given characters and backgrounds that adhere to the desired pose sequence. However, existing methods focus more on human actions while neglecting the generation of background, which typically leads to static results or inharmonious movements. The community has explored camera pose-guided animation tasks, yet preparing the camera trajectory is impractical for most entertainment applications and ordinary users. As a remedy, we present an AnimateAnywhere framework, rousing the background in human image animation without requirements on camera trajectories. In particular, based on our key insight that the movement of the human body often reflects the motion of the background, we introduce a background motion learner (BML) to learn background motions from human pose sequences. To encourage the model to learn more accurate cross-frame correspondences, we further deploy an epipolar constraint on the 3D attention map. Specifically, the mask used to suppress geometrically unreasonable attention is carefully constructed by combining an epipolar mask and the current 3D attention map. Extensive experiments demonstrate that our AnimateAnywhere effectively learns the background motion from human pose sequences, achieving state-of-the-art performance in generating human animation results with vivid and realistic backgrounds. The source code and model will be available at https://github.com/liuxiaoyu1104/AnimateAnywhere.

Figures

Figures reproduced from arXiv: 2504.19834 by the authors.

Figure 1
Figure 1. Comparison with existing paradigms of human image animation. (a) [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) Overview Pipeline of AnimateAnywhere. Given a reference background image, a reference human image, and a human pose sequence as inputs, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Epipolar line and epipolar mask. (a) The epipolar line [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison on Humanvid [4] test dataset. Our proposed method generates more realistic videos while ensuring harmonious motion between [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparisons on the BL200 test dataset. Our method not only generates realistic background motion but also maintains the temporal [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 4
Figure 4. Figure 4: fig. 4. In contrast, our method synthesizes photorealistic videos [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: The results are obtained from our method by cross-using the reference human image and background image. TABLE II ABLATION STUDY ON THE EPIPOLAR LOSS Lepipolar . Method PSNR↑ SSIM↑ LPIPS↓ FID↓ FVD↓ Ours w/o Lepipolar 15.83 0.4823 0.4063 96.48 1269.42 Ours with Lepipolar…
Figure 7
Figure 7. Figure 7: Qualitative comparison of epipolar loss. Without the epipolar loss or [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: The generated results with and without the VGG perceptual loss. The [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Examples of multiple camera poses based on a human pose sequence. [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AniCrafter: Customizing Realistic Human-Centric Animation via Avatar-Background Conditioning in Video Diffusion Models

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A diffusion model animates a character into arbitrary dynamic backgrounds by conditioning on a rendered 3D-avatar video, reframing open-domain animation as a restoration problem.

Reference graph

Works this paper leans on

51 extracted references · 13 canonical work pages · cited by 1 Pith paper

  1. [1]

    Animate anyone: Consistent and controllable image-to-video synthesis for character animation,

    L. Hu, “Animate anyone: Consistent and controllable image-to-video synthesis for character animation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 8153–8163

    2024

  2. [2]

    Magicanimate: Temporally consistent human image animation using diffusion model,

    Z. Xu, J. Zhang, J. H. Liew, H. Yan, J.-W. Liu, C. Zhang, J. Feng, and M. Z. Shou, “Magicanimate: Temporally consistent human image animation using diffusion model,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 1481–1490

    work page 2024

  3. [3]

    Champ: Controllable and consistent human image animation with 3d parametric guidance,

    S. Zhu, J. L. Chen, Z. Dai, Y . Xu, X. Cao, Y . Yao, H. Zhu, and S. Zhu, “Champ: Controllable and consistent human image animation with 3d parametric guidance,” arXiv preprint arXiv:2403.14781 , 2024

    arXiv 2024

  4. [4]

    Humanvid: Demystifying training data for camera- controllable human image animation,

    Z. Wang, Y . Li, Y . Zeng, Y . Fang, Y . Guo, W. Liu, J. Tan, K. Chen, T. Xue, B. Daiet al., “Humanvid: Demystifying training data for camera- controllable human image animation,” arXiv preprint arXiv:2407.17438, 2024

    arXiv 2024

  5. [5]

    Disentangling foreground and background motion for enhanced realism in human video genera- tion,

    J. Liu, K. Yu, M. Feng, X. Guo, and M. Cui, “Disentangling foreground and background motion for enhanced realism in human video genera- tion,” arXiv preprint arXiv:2405.16393 , 2024

    arXiv 2024

  6. [6]

    Realisdance: Equip controllable character animation with realistic hands,

    J. Zhou, B. Wang, W. Chen, J. Bai, D. Li, A. Zhang, H. Xu, M. Yang, and F. Wang, “Realisdance: Equip controllable character animation with realistic hands,” arXiv preprint arXiv:2409.06202 , 2024. IEEE TRANSACTIONS ON MULTIMEDIA 10

    arXiv 2024

  7. [7]

    Realishuman: A two-stage approach for refining malformed human parts in generated images,

    B. Wang, J. Zhou, J. Bai, Y . Yang, W. Chen, F. Wang, and Z. Lei, “Realishuman: A two-stage approach for refining malformed human parts in generated images,” arXiv preprint arXiv:2409.03644 , 2024

    arXiv 2024

  8. [8]

    Unianimate: Taming unified video diffusion models for consistent human image animation,

    X. Wang, S. Zhang, C. Gao, J. Wang, X. Zhou, Y . Zhang, L. Yan, and N. Sang, “Unianimate: Taming unified video diffusion models for consistent human image animation,” arXiv preprint arXiv:2406.01188 , 2024

    arXiv 2024

Show all 51 references
  1. [9]

    Cogvideox: Text-to-video diffusion models with an expert transformer,

    Z. Yang, J. Teng, W. Zheng, M. Ding, S. Huang, J. Xu, Y . Yang, W. Hong, X. Zhang, G. Feng et al., “Cogvideox: Text-to-video diffusion models with an expert transformer,” arXiv preprint arXiv:2408.06072 , 2024

    2024 arXiv

  2. [10]

    Lora: Low-rank adaptation of large language models

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chen et al., “Lora: Low-rank adaptation of large language models.” ICLR, vol. 1, no. 2, p. 3, 2022

    2022

  3. [11]

    First or- der motion model for image animation,

    A. Siarohin, S. Lathuili `ere, S. Tulyakov, E. Ricci, and N. Sebe, “First or- der motion model for image animation,” Advances in neural information processing systems, vol. 32, 2019

    2019

  4. [12]

    Gac-gan: A general method for appearance-controllable human video motion transfer,

    D. Wei, X. Xu, H. Shen, and K. Huang, “Gac-gan: A general method for appearance-controllable human video motion transfer,” IEEE Trans- actions on Multimedia , vol. 23, pp. 2457–2470, 2020

    2020

  5. [13]

    One-shot free-view neural talking-head synthesis for video conferencing,

    T.-C. Wang, A. Mallya, and M.-Y . Liu, “One-shot free-view neural talking-head synthesis for video conferencing,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 10 039–10 049

    2021

  6. [14]

    Humangan: A generative model of human images,

    K. Sarkar, L. Liu, V . Golyanik, and C. Theobalt, “Humangan: A generative model of human images,” in 2021 International Conference on 3D Vision (3DV) . IEEE, 2021, pp. 258–267

    2021

  7. [15]

    A good image generator is what you need for high- resolution video synthesis,

    Y . Tian, J. Ren, M. Chai, K. Olszewski, X. Peng, D. N. Metaxas, and S. Tulyakov, “A good image generator is what you need for high- resolution video synthesis,” arXiv preprint arXiv:2104.15069 , 2021

    2021 arXiv

  8. [16]

    Coherent image animation using spatial-temporal correspondence,

    W. Wang, W. Zhou, J. Bao, and H. Li, “Coherent image animation using spatial-temporal correspondence,” IEEE Transactions on Multimedia , vol. 25, pp. 3397–3408, 2022

    2022

  9. [17]

    One-shot human motion transfer via occlusion-robust flow prediction and neural texturing,

    Y . Ji, C. Zheng, and T.-J. Cham, “One-shot human motion transfer via occlusion-robust flow prediction and neural texturing,” IEEE Transac- tions on Multimedia , 2025

    2025

  10. [18]

    Beyond static scenes: Camera-controllable back- ground generation for human motion,

    M. Yao, M. Chen, Q. Zhou, Y . Zhang, M. Liu, X. Li, S. Liu, C. Ju, S. Xiao, Q. Liu et al., “Beyond static scenes: Camera-controllable back- ground generation for human motion,” arXiv preprint arXiv:2504.02004, 2025

    2025 arXiv

  11. [19]

    Musepose: a pose-driven image-to-video framework for virtual human generation,

    Z. Tong, C. Li, Z. Chen, B. Wu, and W. Zhou, “Musepose: a pose-driven image-to-video framework for virtual human generation,” arxiv, 2024

    2024

  12. [20]

    Animate-x: Universal character image animation with enhanced motion representation,

    S. Tan, B. Gong, X. Wang, S. Zhang, D. Zheng, R. Zheng, K. Zheng, J. Chen, and M. Yang, “Animate-x: Universal character image animation with enhanced motion representation,” arXiv preprint arXiv:2410.10306, 2024

    2024 arXiv

  13. [21]

    Mimo: Controllable character video synthesis with spatial decomposed modeling,

    Y . Men, Y . Yao, M. Cui, and L. Bo, “Mimo: Controllable character video synthesis with spatial decomposed modeling,” arXiv preprint arXiv:2409.16160, 2024

    2024 arXiv

  14. [22]

    Animate anyone 2: High-fidelity charac- ter image animation with environment affordance,

    L. Hu, G. Wang, Z. Shen, X. Gao, D. Meng, L. Zhuo, P. Zhang, B. Zhang, and L. Bo, “Animate anyone 2: High-fidelity charac- ter image animation with environment affordance,” arXiv preprint arXiv:2502.06145, 2025

    2025 arXiv

  15. [23]

    Tram: Global trajectory and motion of 3d humans from in-the-wild videos,

    Y . Wang, Z. Wang, L. Liu, and K. Daniilidis, “Tram: Global trajectory and motion of 3d humans from in-the-wild videos,” arXiv preprint arXiv:2403.17346, 2024

    2024 arXiv

  16. [24]

    Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras,

    Z. Teed and J. Deng, “Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras,” Advances in neural information processing systems, vol. 34, pp. 16 558–16 569, 2021

    2021

  17. [25]

    Cameractrl: Enabling camera control for text-to-video generation,

    H. He, Y . Xu, Y . Guo, G. Wetzstein, B. Dai, H. Li, and C. Yang, “Cameractrl: Enabling camera control for text-to-video generation,” arXiv preprint arXiv:2404.02101 , 2024

    2024 arXiv

  18. [26]

    Video diffusion models,

    J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet, “Video diffusion models,” Advances in Neural Information Processing Systems, vol. 35, pp. 8633–8646, 2022

    2022

  19. [27]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning,

    Y . Guo, C. Yang, A. Rao, Z. Liang, Y . Wang, Y . Qiao, M. Agrawala, D. Lin, and B. Dai, “Animatediff: Animate your personalized text- to-image diffusion models without specific tuning,” arXiv preprint arXiv:2307.04725, 2023

    2023 arXiv

  20. [28]

    Stable video diffusion: Scaling latent video diffusion models to large datasets,

    A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y . Levi, Z. English, V . V oleti, A. Lettset al., “Stable video diffusion: Scaling latent video diffusion models to large datasets,” arXiv preprint arXiv:2311.15127, 2023

    2023 arXiv

  21. [29]

    Lumiere: A space-time diffusion model for video generation,

    O. Bar-Tal, H. Chefer, O. Tov, C. Herrmann, R. Paiss, S. Zada, A. Ephrat, J. Hur, G. Liu, A. Raj et al. , “Lumiere: A space-time diffusion model for video generation,” arXiv preprint arXiv:2401.12945 , 2024

    2024 arXiv

  22. [30]

    Photorealistic video generation with diffusion models,

    A. Gupta, L. Yu, K. Sohn, X. Gu, M. Hahn, F.-F. Li, I. Essa, L. Jiang, and J. Lezama, “Photorealistic video generation with diffusion models,” in European Conference on Computer Vision. Springer, 2025, pp. 393– 411

    2025

  23. [31]

    Con- trolvideo: Training-free controllable text-to-video generation,

    Y . Zhang, Y . Wei, D. Jiang, X. Zhang, W. Zuo, and Q. Tian, “Con- trolvideo: Training-free controllable text-to-video generation,” arXiv preprint arXiv:2305.13077, 2023

    2023 arXiv

  24. [32]

    Sparsectrl: Adding sparse controls to text-to-video diffusion models,

    Y . Guo, C. Yang, A. Rao, M. Agrawala, D. Lin, and B. Dai, “Sparsectrl: Adding sparse controls to text-to-video diffusion models,” in European Conference on Computer Vision . Springer, 2025, pp. 330–348

    2025

  25. [33]

    Dreamvideo: Composing your dream videos with customized subject and motion,

    Y . Wei, S. Zhang, Z. Qing, H. Yuan, Z. Liu, Y . Liu, Y . Zhang, J. Zhou, and H. Shan, “Dreamvideo: Composing your dream videos with customized subject and motion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 6537–6549

    2024

  26. [34]

    Videodreamer: Customized multi-subject text-to-video generation with disen-mix finetuning on language-video foundation models,

    H. Chen, X. Wang, G. Zeng, Y . Zhang, Y . Zhou, F. Han, Y . Wu, and W. Zhu, “Videodreamer: Customized multi-subject text-to-video generation with disen-mix finetuning on language-video foundation models,” IEEE Transactions on Multimedia , 2025

    2025

  27. [35]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685 , 2021

    2021 arXiv

  28. [36]

    Direct-a-video: Customized video generation with user- directed camera movement and object motion,

    S. Yang, L. Hou, H. Huang, C. Ma, P. Wan, D. Zhang, X. Chen, and J. Liao, “Direct-a-video: Customized video generation with user- directed camera movement and object motion,” in ACM SIGGRAPH 2024 Conference Papers, 2024, pp. 1–12

    2024

  29. [37]

    Motionctrl: A unified and flexible motion controller for video generation,

    Z. Wang, Z. Yuan, X. Wang, Y . Li, T. Chen, M. Xia, P. Luo, and Y . Shan, “Motionctrl: A unified and flexible motion controller for video generation,” in ACM SIGGRAPH 2024 Conference Papers , 2024, pp. 1–11

    2024

  30. [38]

    Camco: Camera-controllable 3d-consistent image-to-video generation,

    D. Xu, W. Nie, C. Liu, S. Liu, J. Kautz, Z. Wang, and A. Vahdat, “Camco: Camera-controllable 3d-consistent image-to-video generation,” arXiv preprint arXiv:2406.02509 , 2024

    2024 arXiv

  31. [39]

    Cami2v: Camera-controlled image-to-video diffusion model,

    G. Zheng, T. Li, R. Jiang, Y . Lu, T. Wu, and X. Li, “Cami2v: Camera-controlled image-to-video diffusion model,” arXiv preprint arXiv:2410.15957, 2024

    2024 arXiv

  32. [40]

    Training-free camera control for video generation,

    C. Hou, G. Wei, Y . Zeng, and Z. Chen, “Training-free camera control for video generation,” arXiv preprint arXiv:2406.10126 , 2024

    2024 arXiv

  33. [41]

    Adding conditional control to text-to-image diffusion models,

    L. Zhang, A. Rao, and M. Agrawala, “Adding conditional control to text-to-image diffusion models,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 3836–3847

    2023

  34. [42]

    T2i- adapter: Learning adapters to dig out more controllable ability for text- to-image diffusion models,

    C. Mou, X. Wang, L. Xie, Y . Wu, J. Zhang, Z. Qi, and Y . Shan, “T2i- adapter: Learning adapters to dig out more controllable ability for text- to-image diffusion models,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 5, 2024, pp. 4296–4304

    2024

  35. [43]

    Scope of validity of psnr in im- age/video quality assessment,

    Q. Huynh-Thu and M. Ghanbari, “Scope of validity of psnr in im- age/video quality assessment,” Electronics letters, vol. 44, no. 13, pp. 800–801, 2008

    2008

  36. [44]

    Image quality assessment: from error visibility to structural similarity,

    Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing , vol. 13, no. 4, pp. 600–612, 2004

    2004

  37. [45]

    The unreasonable effectiveness of deep features as a perceptual metric,

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 586–595

    2018

  38. [46]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium,

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” Advances in neural information processing systems , vol. 30, 2017

    2017

  39. [47]

    Fvd: A new metric for video generation,

    T. Unterthiner, S. van Steenkiste, K. Kurach, R. Marinier, M. Michalski, and S. Gelly, “Fvd: A new metric for video generation,” 2019

    2019

  40. [48]

    Grounded sam: Assembling open-world models for diverse visual tasks,

    T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y . Chen, F. Yan, Z. Zeng, H. Zhang, F. Li, J. Yang, H. Li, Q. Jiang, and L. Zhang, “Grounded sam: Assembling open-world models for diverse visual tasks,” 2024

    2024

  41. [49]

    ProPainter: Improving propagation and transformer for video inpainting,

    S. Zhou, C. Li, K. C. Chan, and C. C. Loy, “ProPainter: Improving propagation and transformer for video inpainting,” in Proceedings of IEEE International Conference on Computer Vision (ICCV) , 2023

    2023

  42. [50]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101 , 2017

    2017 arXiv

  43. [51]

    Dynamictrl: Rethinking the basic structure and the role of text for high-quality human image animation,

    H. Zhao, Z. Qi, C. Wang, Q. Zheng, G. Lu, F. Chen, H. Xu, and Z. Wu, “Dynamictrl: Rethinking the basic structure and the role of text for high-quality human image animation,” arXiv preprint arXiv:2503.21246, 2025

    2025 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.