Pith. sign in

REVIEW 3 major objections 5 minor 58 references

AnyCharV: Bootstrap Controllable Character Video Generation with Fine-to-Coarse Guidance

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A two-stage, fine-to-coarse guidance schedule lets a diffusion model put any reference character into any target video, beating four prior systems on identity, motion, and scene preservation.

desk verdict New training strategy, weak evaluation: the FVD reference set is mismatched to the task and the user-study margins over Viggle are within error bars. read the letter →

arxiv 2502.08189 v2 pith:HNTESG6Q submitted 2025-02-12 cs.CV

Zhao Wang , Hao Wen , Lingting Zhu , Chenming Shang , Yujiu Yang , Qi Dou This is my paper · ORCID
classification cs.CV
keywords charactervideogenerationcontrollablediffusionmodelfine-to-coarseguidanceself-boostingtrainingposeidentitypreservationcomposition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to establish that character video generation can be made fully flexible: given any reference character image and any target driving video, a diffusion model can put the character into the video, following the target motion and preserving the target scene. The proposed two-stage scheme first trains a base model under a fine segmentation mask to compose character and scene, then uses the base model's own outputs to build 64,000 character-replacement pairs and fine-tunes with only a coarse bounding-box mask. The authors report that this fine-to-coarse guidance outperforms open-source and closed-source state-of-the-art systems in quantitative metrics and user studies, and extends to characters and scenes synthesized by text-to-image and text-to-video models. If correct, the result is a practical character-replacement tool for film and creative production that requires no 3D reconstruction and no per-video tuning.

What carries the argument

The load-bearing mechanism is the fine-to-coarse guidance schedule applied in two stages. Stage 1 trains the denoising UNet with a ReferenceNet and a pose guider, conditioning generation on the reference image via CLIP cross-attention and ReferenceNet spatial features, and on the target video via a fine segmentation mask concatenated to the noisy latent and augmented at its borders, together with 2D pose skeletons. Stage 2 freezes ReferenceNet and pose guider and fine-tunes only the denoising UNet on self-generated (reference-driving video, target-driving video) pairs, replacing the fine mask with a coarse bounding-box mask and feeding the unmasked generated video as the scene input. This self-boosting loop turns mask shape from a hard constraint into a soft location cue, which is what the paper credits for preserving character details.

What would settle it

Measure identity preservation, such as CLIP-I or face distance, of the final model while tracking the visual fidelity of the 64,000 synthesized training pairs; if identity gains appear only when those pairs are nearly artifact-free and disappear when pairs contain visible mask-shaped artifacts, the self-boosting assumption is the active ingredient. A more direct test is to corrupt a subset of the generated pairs, for instance by blurring or misaligning the character, and check whether the second stage amplifies those corruptions.

Watch

Extended reading notes

Core claim

The central discovery is that mask shape granularity controls the trade-off between compositional accuracy and identity fidelity. A fine segmentation mask in the first stage teaches the model where and how to place the reference character into the target scene, but its exact boundary leaks into the generated character and produces blur and artifacts. The fix is a self-boosting stage: the first-stage model generates 64,000 pairs of videos that share scene and motion but differ in character, and the model is then fine-tuned on these pairs using a coarse bounding-box mask on the unmasked video. This looser constraint lets the model recover the reference character's details while still knowing where to put it. The paper reports consistent gains over using fine masks throughout, coarse masks throughout, or no masks, and superior identity, motion, and scene scores against Make-A-Protagonist, Viggle, MIMO, and VACE.

Load-bearing premise

The second stage assumes that videos synthesized by the first stage are faithful enough to train on; if those synthesized pairs contain systematic shape, texture, or motion artifacts, the self-boosting retraining will reinforce them rather than correct them, and the paper does not quantify failure rates across the 64,000 pairs.

Editorial extensions

If this is right

  • Character replacement in arbitrary footage becomes a single-image-plus-video operation, with no 3D reconstruction, no per-video fine-tuning, and no manual compositing.
  • The method composes with text-to-image and text-to-video generators, so a user can describe a character and a scene in text and get that character animated in that scene.
  • Because only the UNet is fine-tuned in stage 2, the self-boosting stage is comparatively cheap and could be re-run on new data to adapt to new character types.
  • The reported 5-minute inference for a 5-second 576x1024 clip places the method within reach of interactive editing tools.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The self-boosting stage is effectively self-distillation: the model trains on its own outputs to relax a conditioning signal. The same fine-to-coarse relaxation might transfer to other guidance modalities such as depth, skeleton, or normal maps, where fine annotations can leak shape artifacts.
  • Because the pipeline depends on SAM2 and DWPose for masks and poses, its robustness is bounded by those detectors; occluded or unusual poses may fail before the diffusion model gets a chance to correct them.
  • The paper does not measure how the quality of the 64,000 synthesized pairs varies; if a subset contains systematic errors, the second stage could be silently reinforcing them. A straightforward diagnostic would be to plot identity metrics against per-pair confidence scores.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes AnyCharV, a two-stage framework for controllable character video generation. Given a reference character image and a target driving video, the method synthesizes a new video in which the reference character performs the target video's motion within the target scene, guided by 2D pose and masks. Stage 1 trains a base model in a self-supervised manner with a fine segmentation mask. Stage 2 constructs 64,000 generated video pairs using the stage-1 model and fine-tunes the UNet with a coarse bounding-box mask, a 'self-boosting' step intended to preserve identity details. Experiments compare against Make-A-Protagonist, Viggle, MIMO, and VACE using FVD, DOVER++, CLIP-I, and a user study.

Significance. The task is timely and practically useful, and the framework is elegant in avoiding explicit 3D reconstruction. The self-supervised composition strategy and the fine-to-coarse self-boosting design are interesting contributions, and the paper includes thorough ablations and a substantial user study. However, the quantitative evidence for the central superiority claim is not currently convincing: the FVD protocol is mismatched to the task, the user-study error intervals overlap with the closest baseline, and the self-boosting mechanism's reliability is not quantitatively characterized. If the evaluation is strengthened, the method could be a valuable practical pipeline.

major comments (3)
  1. [Appendix A.3, Table 1] The FVD values in Table 1 are computed between the 200 generated videos and 1,000 real character videos from the CharVG training set (Appendix A.3), not against the 10 target driving videos that define the evaluated task. FVD is a distribution-level realism metric; a method that ignores both conditioning inputs and emits generic plausible character videos can achieve a low FVD. Consequently, the reported FVD improvements (e.g., 5.55% over VACE, 3.88% over MIMO) are uninterpretable as evidence of identity, motion, or scene preservation, and the reference set's overlap with training data is not addressed. The paper also does not report standard errors or confidence intervals for any of the three metrics in Table 1, although the evaluation consists of only 200 generated videos. To support the central claim, the authors should either compute FVD against the target driving videos (or a task-matched paired reference set) or drop FVD as a headline metric and rely on metrics that directly measure conditional fidelity.
  2. [Table 2, Section 4.2] The user study's average rankings show overlapping error intervals between AnyCharV and the strongest closed-source baseline, Viggle, on all three criteria (Identity: 1.89±0.24 vs. 2.04±0.24; Motion: 2.02±0.37 vs. 2.20±0.33; Scene: 1.99±0.27 vs. 2.04±0.31). The paper does not report how the intervals were computed, per-rater statistics, or a significance test; the Section 4.2 claim that AnyCharV 'significantly outperforms' Viggle is therefore not supported. Given that the user study is the only evidence directly aligned with the claimed identity/motion/scene preservation, the authors should add a paired significance test and either report the full distribution or acknowledge the lack of statistical separation.
  3. [Section 3.3, Eq. (3)] The self-boosting stage trains on 64,000 video pairs generated by the stage-1 model. Because the conditioning input in Eq. (3) is the generated reference-driving video x_r^d, any systematic artifact in the base model's outputs (e.g., shape distortion at mask borders, blurred hands, motion jitter) can be reinforced during fine-tuning. The regression target is the original target video, so this is not classic circularity, but the reliability of the synthesized pairs is nonetheless a load-bearing assumption. The paper shows selected examples in Figure 7 and an ablation in Table 3, but it does not quantify artifact frequency, apply any filtering, or compare the distribution of generated pairs to real data. The authors should validate this assumption, for example by measuring pose/identity error on the pairs or by ablating with a quality-filtered subset.
minor comments (5)
  1. [Section 4.2] In the user-study paragraph, the text refers to 'the open-source Make-A-Protagonist and MACE'; 'MACE' is undefined and appears to be a typo for 'VACE'.
  2. [Appendix A.3] In the user-study metric definitions, 'Character identity preservation (Identity in Table 1)' should reference Table 2 rather than Table 1.
  3. [Section 3.3] The first paragraph says 'The based model developed in Section 3.2' and should read 'The base model developed in Section 3.2'.
  4. [Section 4.4] The sentence 'driving by a target video generated by text-to-video (T2V) models' should be 'driven by a target video...' for grammatical correctness.
  5. [Section 3.2, Figure 2] The notation 'md = J - md' for the complementary mask is confusing because the same symbol 'md' denotes both the mask and its complement; consider using a different symbol such as 'm_d' for the complement. The figure caption also contains unresolved LaTeX artifacts for the scene symbol.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the self-boosting stage targets the real driving video, and the main claims rest on external metrics and ablations; only a non-load-bearing self-citation remains.

full rationale

The paper's central claim is empirical: a two-stage fine-to-coarse training pipeline yields better controllable character video generation. No load-bearing step reduces a prediction to its own inputs by construction. In Stage 2, the diffusion model is trained against the real target driving video (Eq. (3): 'z′ is the latent of the target driving video x′d'), while the self-generated reference driving video serves only as an input condition; the regression target is not the model's own output, so the identity-preservation result is not forced by the loss definition. The reported gains are checked by external metrics (FVD, DOVER++, CLIP-I) and human ratings, and the component ablations (Tables 3-4) test the claimed contributions independently. The only self-citation is CustomVideo [6] in related work, which is contextual and not load-bearing. Thus there is no significant circularity; the self-boosting data-quality risk is a training-robustness concern, not a circularity concern.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The ledger captures hand-chosen training choices and unstated domain assumptions the empirical claim depends on. No new physical or conceptual entities are introduced; the method recombines existing conditioning signals.

free parameters (3)
  • Number of self-boosting video pairs = 64,000
    Chosen without sensitivity analysis; the scale of the synthetic pair set directly affects the second-stage training signal (Section 3.3).
  • Classifier-free guidance scale = 3.0
    Inference-time hyperparameter selected by hand; no ablation is shown for how identity versus scene fidelity varies with this value (Section A.2).
  • Stage-wise training schedule = 50k + 3k + 10k + 10k steps at 768x768 and 704x704; batch sizes 64 and 8; learning rate 1e-5
    All schedule components are hand-chosen; no sensitivity study is reported (Section A.2).
assumptions (4)
  • domain assumption DWPose, SAM2, and YOLOv8 provide sufficiently accurate pose, segmentation, and bounding-box masks for training and inference.
    The method feeds these estimates directly as conditioning signals; errors would propagate into the generated composition. Introduced in Sections 3.2 and A.1.
  • domain assumption Videos generated by the first-stage base model are reliable enough to serve as training inputs in the second stage.
    The self-boosting stage is the paper's main novelty, and its benefit depends on the quality of the 64,000 synthesized pairs. The paper shows qualitative examples but no quantitative quality control (Section 3.3).
  • domain assumption Stable Diffusion 1.5 and AnimateDiff priors are sufficient backbones for high-fidelity character video generation.
    The model is initialized from these pretrained weights and does not train a new backbone from scratch (Section 3.4 and A.2).
  • domain assumption The CharVG dataset of 9,055 internet videos covers the diversity needed for arbitrary source characters and target scenes.
    The 'arbitrary' generalization claim depends on training distribution coverage; no distribution analysis is provided (Sections 4.1 and A.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of AnyCharV: Bootstrap Controllable Character Video Generation with Fine-to-Coarse Guidance." pith.science (2026). https://pith.science/paper/HNTESG6Q

@misc{pith2026250208189,
  author       = {Pith},
  title        = {Pith review of: AnyCharV: Bootstrap Controllable Character Video Generation with Fine-to-Coarse Guidance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HNTESG6Q}},
  note         = {Machine review of arXiv:2502.08189}
}
read the original abstract

Character video generation is a significant real-world application focused on producing high-quality videos featuring specific characters. Recent advancements have introduced various control signals to animate static characters, successfully enhancing control over the generation process. However, these methods often lack flexibility, limiting their applicability and making it challenging for users to synthesize a source character into a desired target scene. To address this issue, we propose a novel framework, AnyCharV, that flexibly generates character videos using arbitrary source characters and target scenes, guided by pose information. Our approach involves a two-stage training process. In the first stage, we develop a base model capable of integrating the source character with the target scene using pose guidance. The second stage further bootstraps controllable generation through a self-boosting mechanism, where we use the generated video in the first stage and replace the fine mask with the coarse one, enabling training outcomes with better preservation of character details. Extensive experimental results demonstrate the superiority of our method compared with previous state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2502.08189 by the authors.

Figure 1
Figure 1. Our method works by giving a reference character image (left) and a target driving video [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overview of our proposed AnyCharV. We design a two-stage pipeline with fine-to-coarse [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Qualitative results of our method compared with previous SOTA methods. The reference [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 5
Figure 5. Figure 5: Visualization for the effect of different [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Qualitative results of combining AnyCharV with FLUX [ [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Qualitative visualization results of our generated data pairs with a reference image (left) [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Qualitative visualization results of our method given a reference image (left) and a target [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Qualitative visualization results of our method given a reference image (left) and a target [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Our user interface for user study. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 30 canonical work pages

  1. [1]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 2, 3, 4

    arXiv 2023

  2. [2]

    Video diffusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633–8646, 2022. 2

    work page 2022

  3. [3]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205.15868, 2022. 2

    arXiv 2022

  4. [4]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7623–7633, 2023. 2

    2023

  5. [5]

    I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models

    Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou. I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models. arXiv preprint arXiv:2311.04145, 2023. 2, 3

    arXiv 2023

  6. [6]

    Customvideo: Customizing text-to-video generation with multiple subjects

    Zhao Wang, Aoxue Li, Lingting Zhu, Yong Guo, Qi Dou, and Zhenguo Li. Customvideo: Customizing text-to-video generation with multiple subjects. arXiv preprint arXiv:2401.09962, 2024. 2, 3

    arXiv 2024

  7. [7]

    Motionbooth: Motion-aware customized text-to-video generation

    Jianzong Wu, Xiangtai Li, Yanhong Zeng, Jiangning Zhang, Qianyu Zhou, Yining Li, Yunhai Tong, and Kai Chen. Motionbooth: Motion-aware customized text-to-video generation. arXiv preprint arXiv:2406.17758,

    arXiv

  8. [8]

    Motiondirector: Motion customization of text-to-video diffusion models

    Rui Zhao, Yuchao Gu, Jay Zhangjie Wu, David Junhao Zhang, Jia-Wei Liu, Weijia Wu, Jussi Keppo, and Mike Zheng Shou. Motiondirector: Motion customization of text-to-video diffusion models. In European Conference on Computer Vision, pages 273–290. Springer, 2025. 2, 3

    work page 2025

Show all 58 references
  1. [9]

    Animate anyone: Consistent and controllable image-to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8153– 8163, 2024. 2, 3, 5

    2024

  2. [10]

    Magicanimate: Temporally consistent human image animation using diffusion model

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human image animation using diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn...

    2024

  3. [11]

    Mimicmotion: High-quality human motion video generation with confidence-aware pose guidance

    Yuang Zhang, Jiaxi Gu, Li-Wen Wang, Han Wang, Junqi Cheng, Yuefeng Zhu, and Fangyuan Zou. Mimicmotion: High-quality human motion video generation with confidence-aware pose guidance. arXiv preprint arXiv:2406.19680, 2024. 2

    2024 arXiv

  4. [12]

    Champ: Controllable and consistent human image animation with 3d parametric guidance

    Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Zilong Dong, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. In European Conference on Computer Vision, pages 145–162. Springer, 2025. 2, 3

    2025

  5. [13]

    Make-your-video: Customized video generation using textual and structural guidance

    Jinbo Xing, Menghan Xia, Yuxin Liu, Yuechen Zhang, Yong Zhang, Yingqing He, Hanyuan Liu, Haoxin Chen, Xiaodong Cun, Xintao Wang, et al. Make-your-video: Customized video generation using textual and structural guidance. IEEE Transactions on Visualization & Computer Graphics, 3...

  6. [14]

    Mimo: Controllable character video synthesis with spatial decomposed modeling

    Yifang Men, Yuan Yao, Miaomiao Cui, and Liefeng Bo. Mimo: Controllable character video synthesis with spatial decomposed modeling. arXiv preprint arXiv:2409.16160, 2024. 2, 3, 7, 14

    2024 arXiv

  7. [15]

    Viggle ai mix application

    Viggle. Viggle ai mix application. https://viggle.ai/home, 2024. 2, 3, 7, 14

    2024

  8. [16]

    Make-a-protagonist: Generic video editing with an ensemble of experts

    Yuyang Zhao, Enze Xie, Lanqing Hong, Zhenguo Li, and Gim Hee Lee. Make-a-protagonist: Generic video editing with an ensemble of experts. arXiv preprint arXiv:2305.08850, 2023. 2, 3, 7, 14

    2023 arXiv

  9. [17]

    Moviecharac- ter: A tuning-free framework for controllable character video synthesis

    Di Qiu, Zheng Chen, Rui Wang, Mingyuan Fan, Changqian Yu, Junshi Huan, and Xiang Wen. Moviecharac- ter: A tuning-free framework for controllable character video synthesis. arXiv preprint arXiv:2410.20974,

    arXiv

  10. [18]

    Vace: All-in-one video creation and editing

    Zeyinzi Jiang, Zhen Han, Chaojie Mao, Jingfeng Zhang, Yulin Pan, and Yu Liu. Vace: All-in-one video creation and editing. arXiv preprint arXiv:2503.07598, 2025. 2, 3, 7, 14

    2025 arXiv

  11. [19]

    Stablevideo: Text-driven consistency-aware diffusion video editing

    Wenhao Chai, Xun Guo, Gaoang Wang, and Yan Lu. Stablevideo: Text-driven consistency-aware diffusion video editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 23040–23050, 2023. 3

    2023

  12. [20]

    Controlvideo: Training-free controllable text-to-video generation

    Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian. Controlvideo: Training-free controllable text-to-video generation. arXiv preprint arXiv:2305.13077, 2023. 3 10

    2023 arXiv

  13. [21]

    Videocomposer: Compositional video synthesis with motion controllability

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Jiuniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jingren Zhou. Videocomposer: Compositional video synthesis with motion controllability. Advances in Neural Information Processing Systems, 36, 2024. 3

    2024

  14. [22]

    Id- animator: Zero-shot identity-preserving human video generation

    Xuanhua He, Quande Liu, Shengju Qian, Xin Wang, Tao Hu, Ke Cao, Keyu Yan, and Jie Zhang. Id- animator: Zero-shot identity-preserving human video generation. arXiv preprint arXiv:2404.15275, 2024. 3

    2024 arXiv

  15. [23]

    Dreamvideo: Composing your dream videos with customized subject and motion

    Yujie Wei, Shiwei Zhang, Zhiwu Qing, Hangjie Yuan, Zhiheng Liu, Yu Liu, Yingya Zhang, Jingren Zhou, and Hongming Shan. Dreamvideo: Composing your dream videos with customized subject and motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogniti...

    2024

  16. [24]

    Direct-a-video: Customized video generation with user-directed camera movement and object motion

    Shiyuan Yang, Liang Hou, Haibin Huang, Chongyang Ma, Pengfei Wan, Di Zhang, Xiaodong Chen, and Jing Liao. Direct-a-video: Customized video generation with user-directed camera movement and object motion. In ACM SIGGRAPH 2024 Conference Papers, pages 1–12, 2024. 3

    2024

  17. [25]

    Skinned motion retargeting with residual perception of motion semantics & geometry

    Jiaxu Zhang, Junwu Weng, Di Kang, Fang Zhao, Shaoli Huang, Xuefei Zhe, Linchao Bao, Ying Shan, Jue Wang, and Zhigang Tu. Skinned motion retargeting with residual perception of motion semantics & geometry. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

    2023

  18. [26]

    Skinned motion retargeting with dense geometric interaction perception

    Zijie Ye, Jia-Wei Liu, Jia Jia, Shikun Sun, and Mike Zheng Shou. Skinned motion retargeting with dense geometric interaction perception. Advances in Neural Information Processing Systems, 37:125907– 125934, 2024. 3

    2024

  19. [27]

    Star: skeleton-aware text-based 4d avatar generation with in-network motion retargeting

    Zenghao Chai, Chen Tang, Yongkang Wong, and Mohan Kankanhalli. Star: skeleton-aware text-based 4d avatar generation with in-network motion retargeting. arXiv preprint arXiv:2406.04629, 2024. 3

    2024 arXiv

  20. [28]

    Semantics-aware motion retargeting with vision-language models

    Haodong Zhang, Zhike Chen, Haocheng Xu, Lei Hao, Xiaofei Wu, Songcen Xu, Zhensong Zhang, Yue Wang, and Rong Xiong. Semantics-aware motion retargeting with vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2155–2...

    2024

  21. [29]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 3

    2020

  22. [30]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 3, 13

    2010 arXiv

  23. [31]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775–5787, 2022. 3

    2022

  24. [32]

    Sora: A review on background, technology, limitations, and opportunities of large vision models

    Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jianfeng Gao, et al. Sora: A review on background, technology, limitations, and opportunities of large vision models. arXiv preprint arXiv:2402.17177, 2024. 3

    2024 arXiv

  25. [33]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 3, 4

    2024 arXiv

  26. [34]

    Openhumanvid: A large-scale high-quality dataset for enhancing human-centric video generation

    Hui Li, Mingwang Xu, Yun Zhan, Shan Mu, Jiaye Li, Kaihui Cheng, Yuxuan Chen, Tan Chen, Mao Ye, Jingdong Wang, et al. Openhumanvid: A large-scale high-quality dataset for enhancing human-centric video generation. arXiv preprint arXiv:2412.00115, 2024. 3

    2024 arXiv

  27. [35]

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 3

    2023 arXiv

  28. [36]

    Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models

    Fan Bao, Chendong Xiang, Gang Yue, Guande He, Hongzhou Zhu, Kaiwen Zheng, Min Zhao, Shilong Liu, Yaole Wang, and Jun Zhu. Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models. arXiv preprint arXiv:2405.04233, 2024. 3

    2024 arXiv

  29. [37]

    Controlnext: Powerful and efficient control for image and video generation

    Bohao Peng, Jian Wang, Yuechen Zhang, Wenbo Li, Ming-Chang Yang, and Jiaya Jia. Controlnext: Powerful and efficient control for image and video generation. arXiv preprint arXiv:2408.06070, 2024. 3

    2024 arXiv

  30. [38]

    Magicdance: Realistic human dance video generation with motions & facial expressions transfer

    Di Chang, Yichun Shi, Quankai Gao, Jessica Fu, Hongyi Xu, Guoxian Song, Qing Yan, Xiao Yang, and Mohammad Soleymani. Magicdance: Realistic human dance video generation with motions & facial expressions transfer. arXiv preprint arXiv:2311.12052, 2023. 3

    2023 arXiv

  31. [39]

    Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion

    Di Chang, Yichun Shi, Quankai Gao, Hongyi Xu, Jessica Fu, Guoxian Song, Qing Yan, Yizhe Zhu, Xiao Yang, and Mohammad Soleymani. Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion. In Forty-first International Conference on Machine...

    2024

  32. [40]

    Idol: Unified dual-modal latent diffusion for human-centric joint video-depth generation

    Yuanhao Zhai, Kevin Lin, Linjie Li, Chung-Ching Lin, Jianfeng Wang, Zhengyuan Yang, David Doermann, Junsong Yuan, Zicheng Liu, and Lijuan Wang. Idol: Unified dual-modal latent diffusion for human-centric joint video-depth generation. In European Conference on Computer Vision, ...

  33. [41]

    Disco: Disentangled control for realistic human dance generation

    Tan Wang, Linjie Li, Kevin Lin, Yuanhao Zhai, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, and Lijuan Wang. Disco: Disentangled control for realistic human dance generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p...

    2024

  34. [42]

    Pct-net: Full resolution image harmonization using pixel-wise color transformations

    Julian Jorge Andrade Guerreiro, Mitsuru Nakazawa, and Björn Stenger. Pct-net: Full resolution image harmonization using pixel-wise color transformations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5917–5926, 2023. 3

    2023

  35. [43]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7310–7320, 2024. 4

    2024

  36. [44]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024. 4, 13

    2024 arXiv

  37. [45]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

  38. [46]

    Effective whole-body pose estimation with two- stages distillation

    Zhendong Yang, Ailing Zeng, Chun Yuan, and Yu Li. Effective whole-body pose estimation with two- stages distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4210–4220, 2023. 5, 13

    2023

  39. [47]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 5

    2013 arXiv

  40. [48]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 7, 13

    2022

  41. [49]

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. In The Twelfth International Conference on Learning Representations,...

    2024

  42. [50]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847,

  43. [51]

    Towards accurate generative models of video: A new metric & challenges

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018. 7, 13

    2018 arXiv

  44. [52]

    Exploring video quality assessment on user generated contents from aesthetic and technical perspectives

    Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jingwen Hou, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring video quality assessment on user generated contents from aesthetic and technical perspectives. In Proceedings of the IEEE/CVF International Conference on...

    2023

  45. [53]

    Labs Black Forest. Flux. https://github.com/black-forest-labs/flux , 2023. 9

    2023

  46. [54]

    Hunyuanvideo: A systematic framework for large video generative models

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024. 9

    2024 arXiv

  47. [55]

    Real-time flying object detection with yolov8

    Dillon Reis, Jordan Kupec, Jacqueline Hong, and Ahmad Daoudi. Real-time flying object detection with yolov8. arXiv preprint arXiv:2305.09972, 2023. 13

    2023 arXiv

  48. [56]

    Edge: Editable dance generation from music

    Jonathan Tseng, Rodrigo Castellon, and Karen Liu. Edge: Editable dance generation from music. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 448–458,

  49. [57]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021. 13

    2021

  50. [58]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32,...

    2019

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.