Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

DualReal: Adaptive Joint Training for Lossless Identity-Motion Fusion in Video Customization

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Jointly training identity and motion adapters in one alternating loop, with gradient masking and stage/depth-dependent weighting, fuses both dimensions without loss in customized text-to-video generation, lifting CLIP-I by 21.7% and…

desk verdict A plausible joint-training recipe for identity-motion video customization, but the reported gains are not cleanly attributable to the method because the head-to-head comparisons confound backbone, compute, and training budget. read the letter →

arxiv 2505.02192 v2 pith:CZWR6IGU submitted 2025-05-04 cs.CV cs.AI

Wenchuan Wang , Mengqi Huang , Yijing Tu , Zhendong Mao This is my paper · ORCID
classification cs.CVcs.AI
keywords videocustomizationtext-to-videogenerationidentityconsistencymotionjointtrainingdiffusiontransformergradientmaskingadaptiveweighting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DualReal addresses a specific failure mode in customized text-to-video generation: when identity and motion are tuned separately, improving one degrades the other. The paper argues that the two dimensions are intrinsically coupled—stable identity restricts possible motions, and motion trajectories force identity changes—so a single coordinated training procedure should outperform isolated adapter tuning. Its central claim is that adaptive joint training, alternating identity and motion steps while masking gradients to prevent knowledge leakage and weighting contributions by denoising stage and network depth, fuses both patterns without loss. The reported result is a 21.7% average gain in CLIP-I and 31.8% in DINO-I over existing methods, with top or near-top scores on motion consistency metrics. A sympathetic reader would care because it suggests a general recipe for training multi-dimension customization in diffusion models.

What carries the argument

The central mechanism is a pair of complementary units. Dual-aware Adaptation alternates the training stage with a binary selector variable $Z$, applies gradient masks $M_m$ and $M_i$ that activate only the motion or identity adapter parameters, and keeps the frozen adapter as a latent regularizer during the other's forward pass. StageBlender Controller is a gated MLP conditioned on timestep embeddings and pooled text-visual features; it outputs softmax-weighted groups $\omega^{(1)}\dots\omega^{(n)}$ that scale each DiT block's motion contribution, with identity scaled by $1-\omega^i$, through the residual update $\hat{f}^i_{\text{out}} = \omega^i f^i_{\text{mo}} + (1-\omega^i) f^i_{\text{id}} + f^i_{\text{dit}}$. The two together let the model vary the identity/motion balance across denoising steps and network depths.

What would settle it

Re-run all baselines (DreamVideo, MotionBooth, LoRA, full fine-tuning) on the same CogVideoX-5B backbone with matched training budgets; if DualReal's identity-similarity margins shrink to near zero or its motion metrics no longer lead, the joint-training claim would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that the 'isolated customized paradigm'—training identity and motion adapters separately, then blending at inference—systematically degrades both dimensions because it ignores their interdependence and applies uniform optimization across all denoising steps. DualReal instead trains the two adapters in one shared loop: at each step a binary switch chooses identity or motion data, the active adapter is guided by the frozen prior of the other dimension, and a gradient mask blocks updates to the non-training adapter, preventing cross-dimension knowledge leakage. A StageBlender Controller then generates per-block scaling weights conditioned on the denoising timestep and fused text-visual features, so identity receives fine-grained attention in shallow, early blocks and motion receives growing weight in the deepest block as denoising proceeds. The paper reports that this arrangement improves identity similarity scores by 21.7% on CLIP-I and 31.8% on DINO-I on average across a constructed benchmark of 50 identities and 21 motion sequences, while matching or exceeding baselines on temporal consistency, motion smoothness, and flickering.

Load-bearing premise

The reported improvements assume that comparing DualReal (trained on CogVideoX-5B) against baselines that run on different base models (MotionBooth on LaVie-base, DreamVideo on ModelScopeT2V) isolates the effect of the proposed training method; if the base model drives most of the metric gap, the gains would not be attributable to joint training.

Editorial extensions

If this is right

  • If DualReal's joint training is right, then the standard practice of training adapters for different attributes separately and merging at inference should be revisited; joint alternating training with gradient masking offers an alternative that avoids mutual degradation.
  • The denoising-stage and depth-dependent weighting suggests that identity and motion occupy different timescales in the diffusion process; one could schedule other attribute pairs, such as style and viewpoint, in the same way.
  • The gradient-masking regularization gives a concrete low-cost mechanism to prevent catastrophic interference in multi-task adapter training.
  • The constructed benchmark of 50 identities, 21 motion sequences, and 50 prompts per case could become a common testbed for identity-motion customization if other groups adopt it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the method's contribution would be to re-run DreamVideo and MotionBooth on the exact CogVideoX-5B backbone used for DualReal; the paper's quantitative comparison keeps different base models for the baselines, so a same-backbone comparison would isolate the joint-training effect from the base model.
  • The identity-weight curve in Figure 7—identity weight rising with denoising step except in the deepest block, which goes the opposite way—suggests a learned curriculum that maps naturally onto other multi-concept generation tasks, where attribute-specific schedules could be read off from similar controller analyses.
  • The framework implicitly predicts that any two customizable dimensions with asymmetric spatial or temporal structure will benefit from stage-dependent weighting; a cheap test would be applying the same controller to style-and-layout or subject-and-environment customization.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes DualReal, a framework for customized text-to-video generation that jointly trains identity and motion adapters on a Diffusion Transformer backbone (CogVideoX-5B). The method alternates identity- and motion-focused training steps with gradient masking to prevent cross-dimension leakage, and uses a StageBlender Controller to modulate the contribution of the two adapters as a function of denoising timestep and DiT block depth. The authors claim average improvements of 21.7% in CLIP-I and 31.8% in DINO-I over existing methods, and state that the approach achieves top performance on nearly all motion quality metrics. The paper includes quantitative comparisons in Table 1, component ablations in Tables 2 and 3, qualitative results, and supplementary implementation details.

Significance. If the central claim were established, the paper would make a useful contribution to video customization: the idea of alternating identity and motion training with gradient masking, combined with stage- and depth-dependent weighting, is a plausible mechanism for reducing identity-motion conflicts, and the paper provides a clearly described framework with detailed hyperparameters in the supplementary material. The work also assembles a benchmark with 50 identities and 21 motion sequences, which is a useful evaluation resource. However, the quantitative evidence as presented does not support the causal attribution of the reported gains to the proposed method, because the headline comparisons are confounded by base model and training budget, and the claimed average improvement percentages are not reproducible from the reported table. The central technical idea remains defensible, but the evaluation needs substantial strengthening before the paper's main claims can be accepted.

major comments (4)
  1. [Sec. 4.1, Table 1, Supp. 7.1] The main quantitative comparison does not isolate the proposed method. DualReal is evaluated on CogVideoX-5B, while MotionBooth uses LaVie-base and DreamVideo uses ModelScopeT2V, so the two published baselines confound method with backbone. The same-backbone baselines are also not matched in training budget: LoRA runs 600 total steps, full-parameter fine-tuning runs 330 steps, and DualReal runs 1,000 steps, with different learning rates (1e-3 vs 1e-4). The large same-backbone gaps in CLIP-I (0.629 vs 0.425/0.521) and DINO-I (0.551 vs 0.286/0.424) could therefore reflect additional optimization compute rather than the proposed architecture. The authors should either run baselines on the same backbone with matched training steps and learning-rate schedules, or clearly report the backbone and budget of every row in Table 1 and restrict causal claims to controlled comparisons.
  2. [Abstract and Table 1] The claimed average improvements of 21.7% on CLIP-I and 31.8% on DINO-I are not reproducible from Table 1 under any standard aggregation. Averaging the four baselines gives (0.566+0.425+0.521+0.458)/4 = 0.4925 for CLIP-I, for which 0.629 is a 27.7% relative improvement, not 21.7%; for DINO-I the baseline average is 0.3758, and 0.551 is a 46.6% improvement, not 31.8%. The paper should state the exact formula used for the average improvement and report per-case or per-baseline numbers with error bars.
  3. [Table 1, Dynamic Degree row] The 'lossless fusion' claim and the statement of 'top performance on nearly all motion metrics' are contradicted by the Dynamic Degree results. Relative to the benchmark average of 12.02, DualReal deviates by +2.94, whereas MotionBooth deviates by only -1.07, and DreamVideo deviates by -3.18, so DualReal is not the closest to the reference motion intensity. The authors acknowledge that their DD is 'not high,' but this undercuts the lossless-fusion language in the title and abstract. The paper should either soften the claim or provide a principled tolerance band within which a deviation is considered acceptable, with statistical support.
  4. [Tables 2 and 3] The ablation studies report single runs with no variance or significance testing, and they are performed on a smaller subset whose size is not specified. Because identity and motion metrics are noisy for generative model evaluation, the observed differences (e.g., DINO-I 0.771 vs 0.766 for removing weight groups in Table 2) may not be meaningful. The authors should report standard deviations over multiple seeds or, at minimum, the number of cases in the ablation subset and the per-case metric distribution.
minor comments (5)
  1. [Sec. 4.1, Evaluation metrics] Temporal Flickering is described as a mean absolute difference between adjacent frames, which normally increases with flicker, yet the table labels it with an up-arrow as a positive metric. The paper should clarify the sign convention and whether higher or lower is better.
  2. [Sec. 3.2, Eq. (1)-(2)] The dimensions of W_down, W_up, and W_cond are not specified; providing these would make the adapter architecture and parameter counts fully reproducible.
  3. [Sec. 3.2, Eq. (4)-(6)] The mask notation is confusing: M is defined as Z*Mm + (1-Z)*Mi, but the conditions in Eq. (6) use l for the motion mask and k for the identity mask without defining the indexing over layers or parameter blocks. Please clarify that the masks are applied elementwise to the adapter parameters and specify how the per-block grouping from the StageBlender Controller interacts with this masking.
  4. [Figure 3] The text 'Deprecated prompts' in the figure appears to be a typo and should be replaced with a meaningful label such as 'diverse prompts' or 'depicted prompts'.
  5. [Tables 2 and 3] The row labeled 'w/o Weight Groups' in Table 2 is numerically identical to the n=1 row in Table 3; the paper should state explicitly that these settings are the same, or explain why the same configuration appears in both tables.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: DualReal is an empirical training method evaluated on external metrics; the backbone-mismatch concern is a validity issue, not circular reasoning.

full rationale

This paper is an empirical systems contribution with no derivation chain to walk. DualReal's adapter and controller weights are learned from data, and the claimed gains are measured on external benchmarks (CLIP-T, CLIP-I, DINO-I, and VBench motion metrics) using held-out prompt variations. No equation defines a predicted quantity in terms of a fitted parameter or an author-proprietary uniqueness theorem, and the few self-citations (e.g., CustomContrast, RealCustom) appear only as background references in related work and are not load-bearing premises. The ablation studies compare variants of the method itself and involve no definitional circularity. The comparison against DreamVideo and MotionBooth on different backbones is a legitimate experimental-fairness concern, but it is a confound, not circular reasoning; therefore the circularity score is 0.

Assumptions & free parameters 4 free parameters · 2 assumptions · 0 invented entities

The method relies on a pretrained DiT backbone and the assumption that adapters can inject identity and motion priors without destructive interference. Hyperparameters such as the training switch ratio (0.5), learning rate (1e-3), and group cardinality (n=7) are hand-chosen or tuned on the authors' subset. No new physical entities are introduced; the proposed DA-Block and StageBlender Controller are architectural components evaluated only within this paper.

free parameters (4)
  • training switch ratio gamma = 0.5
    Set to give each step a 50% chance of motion training; chosen by hand in Sec. 7.1, not derived.
  • learning rate = 1e-3
    Chosen by hand in Sec. 7.1, standard AdamW.
  • group cardinality n = 7
    Selected via ablation on a smaller evaluation subset (Table 3); a hyperparameter tuned on the authors' own data.
  • number of training steps per case = 1000
    Set in Sec. 7.1, chosen by hand for each test case.
assumptions (2)
  • domain assumption The pretrained DiT model (CogVideoX-5B) contains sufficient identity and motion priors for adapter-based injection.
    The whole method assumes the base model can represent both dimensions and adapters are sufficient, per Sec. 3.1.
  • domain assumption Frozen adapter features act as intrinsic regularization that prevents dimensional overfitting during alternating training.
    Stated in Sec. 3.2 as the mechanism; no independent evidence beyond the paper's ablations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DualReal: Adaptive Joint Training for Lossless Identity-Motion Fusion in Video Customization." pith.science (2026). https://pith.science/paper/CZWR6IGU

@misc{pith2026250502192,
  author       = {Pith},
  title        = {Pith review of: DualReal: Adaptive Joint Training for Lossless Identity-Motion Fusion in Video Customization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CZWR6IGU}},
  note         = {Machine review of arXiv:2505.02192}
}
read the original abstract

Customized text-to-video generation with pre-trained large-scale models has recently garnered significant attention by focusing on identity and motion consistency. Existing works typically follow the isolated customized paradigm, where the subject identity or motion dynamics are customized exclusively. However, this paradigm completely ignores the intrinsic mutual constraints and synergistic interdependencies between identity and motion, resulting in identity-motion conflicts throughout the generation process that systematically degrade. To address this, we introduce DualReal, a novel framework that employs adaptive joint training to construct interdependencies between dimensions collaboratively. Specifically, DualReal is composed of two units: (1) Dual-aware Adaptation dynamically switches the training step (i.e., identity or motion), learns the current information guided by the frozen dimension prior, and employs a regularization strategy to avoid knowledge leakage; (2) StageBlender Controller leverages the denoising stages and Diffusion Transformer depths to guide different dimensions with adaptive granularity, avoiding conflicts at various stages and ultimately achieving lossless fusion of identity and motion patterns. We constructed a more comprehensive evaluation benchmark than existing methods. The experimental results show that DualReal improves CLIP-I and DINO-I metrics by 21.7% and 31.8% on average, and achieves top performance on nearly all motion metrics. Page: https://wenc-k.github.io/dualreal-customization

Figures

Figures reproduced from arXiv: 2505.02192 by the authors.

Figure 1
Figure 1. Generated customization results of our proposed novel paradigm [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Visual analysis of isolated training paradigm. We se￾lect different identities with the same motion pattern, fix the num￾ber of identity training steps, and gradually increase the number of motion training steps to achieve two-dimensional customiza￾tion. The red box marking the relative optimal position of the same identity’s fidelity. Experiments show that (1) adding motion prior significantly damages identity cons… view at source ↗
Figure 3
Figure 3. Overall framework of DualReal. At each training step, we first dynamically switch the training step Z (i.e., identity or motion) to determine the data processing path. The specific data undergoes noise injection and combines with the text embeddings. StageBlender Controller governs two-dimensional adapters’ contributions in Dual-Aware Block (DA-Block) through time-aware conditioning of current denoising step and fus… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Illustration of proposed StageBlender Controller, which employs an Adaptive LayerNorm mechanism that mod￾ulates text-visual feature based on timestep-conditional embed￾dings, then maps the feature to multiple groups after residual gated connections. These scaled weight…
Figure 5
Figure 5. Figure 5: Qualitative comparison with existing methods. Compared with other methods, DualReal achieves high identity consistency with coherent motion, demonstrating the advantage of joint training in balancing pattern conflicts. ize in processing concrete, fine-grained features.…
Figure 7
Figure 7. Figure 7: Controller Visual Analysis. We show the Identity Weights trends across denoising steps for different block depths. (1) As denoising progresses, the diffusion model’s emphasis shifts monotonically between identity and motion, with a growing focus on identity(orange dash…
Figure 8
Figure 8. Figure 8: Generated customization results of our proposed novel paradigm [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Generated customization results of our proposed novel paradigm [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: More Qualitative comparison with existing methods. The result shows that while MotionBooth maintains identity fidelity, it fails to model motion patterns effectively. DreamVideo suffers from pattern conflicts during inference, resulting in inconsistent identity. Simil…
Figure 11
Figure 11. Figure 11: Qualitative ablations studies on each component. Omitting Dual-aware Adaptation introduces artifacts on the subject’s hands, significantly reducing clarity. Moreover, using fixed weights for the dimensional adapters without the StageBlender Controller causes over-adap…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LongAnimation: Long Animation Generation with Dynamic Global-Local Memory

    cs.CV 2025-07 conditional novelty 6.0 of 10

    LongAnimation uses a dynamic global-local memory, built from a long-video-understanding model's KV cache, to colorize animation sequences of about 500 frames with stable color consistency.

Reference graph

Works this paper leans on

60 extracted references · 18 canonical work pages · cited by 1 Pith paper

  1. [1]

    Latent-shift: Latent diffu- sion with temporal shift for efficient text-to-video genera- tion

    Jie An, Songyang Zhang, Harry Yang, Sonal Gupta, Jia-Bin Huang, Jiebo Luo, and Xi Yin. Latent-shift: Latent diffu- sion with temporal shift for efficient text-to-video genera- tion. arXiv preprint arXiv:2304.08477, 2023. 3

    arXiv 2023

  2. [2]

    Uniedit: A unified tuning- free framework for video motion and appearance editing

    Jianhong Bai, Tianyu He, Yuchi Wang, Junliang Guo, Haoji Hu, Zuozhu Liu, and Jiang Bian. Uniedit: A unified tuning- free framework for video motion and appearance editing. arXiv preprint arXiv:2402.13185, 2024. 3

    arXiv 2024

  3. [3]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 3

    arXiv 2023

  4. [4]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 7

    2021

  5. [5]

    Still-moving: Customized video generation without customized video data

    Hila Chefer, Shiran Zada, Roni Paiss, Ariel Ephrat, Omer Tov, Michael Rubinstein, Lior Wolf, Tali Dekel, Tomer Michaeli, and Inbar Mosseri. Still-moving: Customized video generation without customized video data. ACM Transactions on Graphics (TOG), 43(6):1–11, 2024. 3

    work page 2024

  6. [6]

    Disenbooth: Disentangled parameter-efficient tuning for subject-driven text-to-image generation

    Hong Chen, Yipeng Zhang, Xin Wang, Xuguang Duan, Yuwei Zhou, and Wenwu Zhu. Disenbooth: Disentangled parameter-efficient tuning for subject-driven text-to-image generation. arXiv preprint arXiv:2305.03374 , 3(4), 2023. 3

    arXiv 2023

  7. [7]

    Customcontrast: A multilevel contrastive perspective for subject-driven text-to-image cus- tomization

    Nan Chen, Mengqi Huang, Zhuowei Chen, Yang Zheng, Lei Zhang, and Zhendong Mao. Customcontrast: A multilevel contrastive perspective for subject-driven text-to-image cus- tomization. In Proceedings of the AAAI Conference on Arti- ficial Intelligence, pages 2123–2131, 2025. 3

    work page 2025

  8. [8]

    Subject-driven text-to-image generation via apprenticeship learning

    Wenhu Chen, Hexiang Hu, Yandong Li, Nataniel Ruiz, Xuhui Jia, Ming-Wei Chang, and William W Cohen. Subject-driven text-to-image generation via apprenticeship learning. Advances in Neural Information Processing Sys- tems, 36:30286–30305, 2023. 3

    2023

Show all 60 references
  1. [9]

    Structure and content-guided video synthesis with diffusion models

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 7346–7356, 2023. 3, 7

    2023

  2. [10]

    An image is worth one word: Personalizing text-to- image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patash- nik, Amit H Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to- image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022. 3

    2022 arXiv

  3. [11]

    Tokenflow: Consistent diffusion features for consistent video editing

    Michal Geyer, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Tokenflow: Consistent diffusion features for consistent video editing. arXiv preprint arXiv:2307.10373, 2023. 3

    2023 arXiv

  4. [12]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 2, 3

    2023 arXiv

  5. [13]

    Svdiff: Compact param- eter space for diffusion fine-tuning

    Ligong Han, Yinxiao Li, Han Zhang, Peyman Milanfar, Dimitris Metaxas, and Feng Yang. Svdiff: Compact param- eter space for diffusion fine-tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7323–7334, 2023. 3

    2023

  6. [14]

    Id-animator: Zero-shot identity-preserving human video generation

    Xuanhua He, Quande Liu, Shengju Qian, Xin Wang, Tao Hu, Ke Cao, Keyu Yan, and Jie Zhang. Id-animator: Zero-shot identity-preserving human video generation. arXiv preprint arXiv:2404.15275, 2024. 3

    2024 arXiv

  7. [15]

    Gaussian error linear units (gelus)

    Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016. 4

    2016 arXiv

  8. [16]

    Video dif- fusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models. Advances in Neural Information Processing Systems, 35:8633–8646, 2022. 3

    2022

  9. [17]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205.15868, 2022. 3

    2022 arXiv

  10. [18]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022. 6, 7

    2022

  11. [19]

    Dse-gan: Dynamic semantic evolu- tion generative adversarial network for text-to-image gener- ation

    Mengqi Huang, Zhendong Mao, Penghui Wang, Quan Wang, and Yongdong Zhang. Dse-gan: Dynamic semantic evolu- tion generative adversarial network for text-to-image gener- ation. In Proceedings of the 30th ACM International Confer- ence on Multimedia, pages 4345–4354, 2022. 3

    2022

  12. [20]

    Towards accurate image coding: Improved au- toregressive image generation with dynamic vector quantiza- tion

    Mengqi Huang, Zhendong Mao, Zhuowei Chen, and Yong- dong Zhang. Towards accurate image coding: Improved au- toregressive image generation with dynamic vector quantiza- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 22596–22605,

  13. [21]

    Realcustom: narrowing real text word for real-time open-domain text-to-image customization

    Mengqi Huang, Zhendong Mao, Mingcong Liu, Qian He, and Yongdong Zhang. Realcustom: narrowing real text word for real-time open-domain text-to-image customization. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 7476–7485, 2024. 3

    2024

  14. [22]

    Vbench: Comprehensive bench- mark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive bench- mark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and P...

    2024

  15. [23]

    Scalable adap- tive computation for iterative generation

    Allan Jabri, David Fleet, and Ting Chen. Scalable adap- tive computation for iterative generation. arXiv preprint arXiv:2212.11972, 2022. 3

    2022 arXiv

  16. [24]

    9 Videobooth: Diffusion-based video generation with image prompts

    Yuming Jiang, Tianxing Wu, Shuai Yang, Chenyang Si, Dahua Lin, Yu Qiao, Chen Change Loy, and Ziwei Liu. 9 Videobooth: Diffusion-based video generation with image prompts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6689– 6700, 2024. 2

    2024

  17. [25]

    Hunyuanvideo: A systematic framework for large video generative models

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024. 3, 4

    2024 arXiv

  18. [26]

    Multi-concept customization of text-to-image diffusion, 2023

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion, 2023. 6

    2023

  19. [27]

    Cones 2: Customizable image synthesis with multiple subjects, 2023

    Zhiheng Liu, Yifei Zhang, Yujun Shen, Kecheng Zheng, Kai Zhu, Ruili Feng, Yu Liu, Deli Zhao, Jingren Zhou, and Yang Cao. Cones 2: Customizable image synthesis with multiple subjects, 2023. 6

    2023

  20. [28]

    Magic-me: Identity-specific video customized diffu- sion

    Ze Ma, Daquan Zhou, Chun-Hsiao Yeh, Xue-She Wang, Xi- uyu Li, Huanrui Yang, Zhen Dong, Kurt Keutzer, and Jiashi Feng. Magic-me: Identity-specific video customized diffu- sion. arXiv preprint arXiv:2402.09368, 2024. 3

    2024 arXiv

  21. [29]

    Realcustom++: Represent- ing images as real-word for real-time customization

    Zhendong Mao, Mengqi Huang, Fei Ding, Mingcong Liu, Qian He, and Yongdong Zhang. Realcustom++: Represent- ing images as real-word for real-time customization. arXiv preprint arXiv:2408.09744, 2024. 3

    2024

  22. [30]

    Snap video: Scaled spatiotemporal transformers for text-to-video synthesis

    Willi Menapace, Aliaksandr Siarohin, Ivan Skorokhodov, Ekaterina Deyneka, Tsai-Shien Chen, Anil Kag, Yuwei Fang, Aleksei Stoliar, Elisa Ricci, Jian Ren, et al. Snap video: Scaled spatiotemporal transformers for text-to-video synthesis. In Proceedings of the IEEE/CVF Conference...

    2024

  23. [31]

    Dreamix: Video diffusion models are general video editors

    Eyal Molad, Eliahu Horwitz, Dani Valevski, Alex Rav Acha, Yossi Matias, Yael Pritch, Yaniv Leviathan, and Yedid Hoshen. Dreamix: Video diffusion models are general video editors. arXiv preprint arXiv:2302.01329, 2023. 3

    2023 arXiv

  24. [32]

    Sora, 2024

    OpenAI. Sora, 2024. 3, 4

    2024

  25. [33]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 4195–4205,

  26. [34]

    Hier- archical spatio-temporal decoupling for text-to-video gener- ation

    Zhiwu Qing, Shiwei Zhang, Jiayu Wang, Xiang Wang, Yujie Wei, Yingya Zhang, Changxin Gao, and Nong Sang. Hier- archical spatio-temporal decoupling for text-to-video gener- ation. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 6635–6645,

  27. [35]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  28. [36]

    Customize-a-video: One-shot motion customization of text- to-video diffusion models

    Yixuan Ren, Yang Zhou, Jimei Yang, Jing Shi, Difan Liu, Feng Liu, Mingi Kwon, and Abhinav Shrivastava. Customize-a-video: One-shot motion customization of text- to-video diffusion models. In European Conference on Com- puter Vision, pages 332–349. Springer, 2024. 3

    2024

  29. [37]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, pa...

    2015

  30. [38]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2250...

    2023

  31. [39]

    Hyperdreambooth: Hypernetworks for fast personalization of text-to-image models

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Wei Wei, Tingbo Hou, Yael Pritch, Neal Wadhwa, Michael Rubinstein, and Kfir Aberman. Hyperdreambooth: Hypernetworks for fast personalization of text-to-image models. In Proceedings of the IEEE/CVF conference on computer vision and pat...

    2024

  32. [40]

    1st place solution in google universal image embedding challenge

    Shihao Shao, Lijun Yu, Yifan Zhao, and Yixiao Ge. 1st place solution in google universal image embedding challenge. https://github.com/ShihaoShao-GH/1st-Place- Solution-in-Google-Universal-Image-Embedding, 2023. 7

    2023

  33. [41]

    In- stantbooth: Personalized text-to-image generation without test-time finetuning

    Jing Shi, Wei Xiong, Zhe Lin, and Hyun Joon Jung. In- stantbooth: Personalized text-to-image generation without test-time finetuning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 8543–8552, 2024. 3

    2024

  34. [42]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 ,

    arXiv

  35. [43]

    Continual dif- fusion: Continual customization of text-to-image diffusion with c-lora

    James Seale Smith, Yen-Chang Hsu, Lingyu Zhang, Ting Hua, Zsolt Kira, Yilin Shen, and Hongxia Jin. Continual dif- fusion: Continual customization of text-to-image diffusion with c-lora. arXiv preprint arXiv:2304.06027, 2023. 3

    2023 arXiv

  36. [44]

    Action recognition in realistic sports videos

    Khurram Soomro and Amir R Zamir. Action recognition in realistic sports videos. In Computer vision in sports , pages 181–208. Springer, 2015. 6

    2015

  37. [45]

    Ucf101: A dataset of 101 human actions classes from videos in the wild, 2012

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild, 2012. 6

    2012

  38. [46]

    Animate-x: Universal character image ani- mation with enhanced motion representation

    Shuai Tan, Biao Gong, Xiang Wang, Shiwei Zhang, Dandan Zheng, Ruobing Zheng, Kecheng Zheng, Jingdong Chen, and Ming Yang. Animate-x: Universal character image ani- mation with enhanced motion representation. arXiv preprint arXiv:2410.10306, 2024. 3

    arXiv

  39. [47]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Proceedings, Part II 16, pages 402–419. Springer,

    2020

  40. [48]

    Modelscope text-to-video technical report

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-video technical report. arXiv preprint arXiv:2308.06571, 2023. 3

    2023 arXiv

  41. [49]

    Videocomposer: Compositional video synthesis 10 with motion controllability

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Ji- uniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jin- gren Zhou. Videocomposer: Compositional video synthesis 10 with motion controllability. Advances in Neural Information Processing Systems, 36:7594–7611, 2023. 3

    2023

  42. [50]

    Customvideo: Customizing text-to- video generation with multiple subjects

    Zhao Wang, Aoxue Li, Lingting Zhu, Yong Guo, Qi Dou, and Zhenguo Li. Customvideo: Customizing text-to- video generation with multiple subjects. arXiv preprint arXiv:2401.09962, 2024. 2, 4

    2024

  43. [51]

    Elite: Encoding visual con- cepts into textual embeddings for customized text-to-image generation

    Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. Elite: Encoding visual con- cepts into textual embeddings for customized text-to-image generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15943–15953, 2023. 3

    2023

  44. [52]

    Dreamvideo: Composing your dream videos with customized subject and motion

    Yujie Wei, Shiwei Zhang, Zhiwu Qing, Hangjie Yuan, Zhi- heng Liu, Yu Liu, Yingya Zhang, Jingren Zhou, and Hong- ming Shan. Dreamvideo: Composing your dream videos with customized subject and motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...

    2024

  45. [53]

    Mo- tionbooth: Motion-aware customized text-to-video genera- tion

    Jianzong Wu, Xiangtai Li, Yanhong Zeng, Jiangning Zhang, Qianyu Zhou, Yining Li, Yunhai Tong, and Kai Chen. Mo- tionbooth: Motion-aware customized text-to-video genera- tion. arXiv preprint arXiv:2406.17758, 2024. 2, 3, 4, 5, 6, 7

    2024 arXiv

  46. [54]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference...

    2023

  47. [55]

    Cus- tomcrafter: Customized video generation with preserving motion and concept composition abilities

    Tao Wu, Yong Zhang, Xintao Wang, Xianpan Zhou, Guang- cong Zheng, Zhongang Qi, Ying Shan, and Xi Li. Cus- tomcrafter: Customized video generation with preserving motion and concept composition abilities. arXiv preprint arXiv:2408.13239, 2024. 5

    2024 arXiv

  48. [56]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 3, 4, 5, 6, 7

    2024 arXiv

  49. [57]

    Show-1: Marrying pixel and latent diffusion models for text-to-video generation

    David Junhao Zhang, Jay Zhangjie Wu, Jia-Wei Liu, Rui Zhao, Lingmin Ran, Yuchao Gu, Difei Gao, and Mike Zheng Shou. Show-1: Marrying pixel and latent diffusion models for text-to-video generation. International Journal of Com- puter Vision, pages 1–15, 2024. 3

    2024

  50. [58]

    Motiondirector: Motion customization of text-to-video diffusion models

    Rui Zhao, Yuchao Gu, Jay Zhangjie Wu, David Jun- hao Zhang, Jia-Wei Liu, Weijia Wu, Jussi Keppo, and Mike Zheng Shou. Motiondirector: Motion customization of text-to-video diffusion models. In European Conference on Computer Vision, pages 273–290. Springer, 2024. 2

    2024

  51. [59]

    Magicvideo: Efficient video generation with latent diffusion models

    Daquan Zhou, Weimin Wang, Hanshu Yan, Weiwei Lv, Yizhe Zhu, and Jiashi Feng. Magicvideo: Efficient video generation with latent diffusion models. arXiv preprint arXiv:2211.11018, 2022. 3 11

    2022 arXiv

  52. [60]

    Experimental Details This section describes the implementation of our primary experiments and ablation studies

    Supplementary 7.1. Experimental Details This section describes the implementation of our primary experiments and ablation studies. For each method, we pro- vide detailed information on the setup. We list hyperparam- eter values, data pre-processing and post-processing steps, t...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.