Pith. sign in

REVIEW 2 major objections 5 minor 51 references

Tora2: Motion and Appearance Customized Diffusion Transformer for Multi-Entity Video Generation

T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Tora2 claims the first single diffusion-transformer model that customizes both the appearance and the motion of several entities at once.

desk verdict Tora2 is a real integration of multi-entity appearance and per-entity trajectory control in a DiT, but the ReID branch's value for non-human objects is asserted, not demonstrated, and the metrics are all point estimates. read the letter →

arxiv 2507.05963 v2 pith:ZWKHH53X submitted 2025-07-08 cs.CV

Zhenghao Zhang , Junchao Liao , Xiangyu Meng , Long Qin , Weizhi Wang This is my paper
classification cs.CV
keywords controllablevideogenerationmulti-entitypersonalizationdiffusiontransformertrajectorycontrolidentitypreservationgatedself-attentioncontrastivelearningopen-setcustomization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Tora2 addresses a problem that previous video generators handled only partially: given several reference images, per-entity text descriptions, and per-entity motion trajectories, produce one video in which each entity keeps its identity, matches its description, and follows its own path. The paper argues that existing methods either personalize a single subject's appearance or control motion from a fixed first frame, but not both for multiple entities simultaneously. It presents a tuning-free diffusion-transformer framework that injects appearance, text, and trajectory conditions at different stages and binds them per entity. If the claim holds, it matters because video creators could specify a man, a cat, and a balloon with separate pictures and paths and get a coherent clip without fine-tuning.

What carries the argument

The load-bearing components are the decoupled personalization extractor, the gated self-attention binding, and the contrastive loss. The extractor concatenates DINOv2 global features with high-frequency identity features taken from a face-recognition backbone for humans and a re-identification backbone for generic objects, then uses a Q-Former to turn the combined representation into personalization embeddings. The gated self-attention mechanism concatenates personalization, motion, and word tokens and lets them interact through self-attention with learned gates, which the paper says is what keeps each trajectory attached to the right entity. The contrastive loss is a symmetric entity-motion InfoNCE objective that pulls matching entity-motion pairs together and pushes non-matching entity-entity and motion-motion pairs apart, reducing trajectory error by roughly three pixels in the ablation.

What would settle it

Take a set of non-human object categories far from common re-identification training data, such as furniture, vehicles, or abstract shapes, and compare subject similarity with and without the high-frequency branch enabled. If removing the re-identification branch for non-human entities produces no measurable drop in subject similarity, then the decoupled extractor's claimed open-set benefit for non-human objects is not doing the work.

Watch

Extended reading notes

Core claim

Tora2's central claim is that multi-entity appearance and motion customization can be achieved together in a diffusion transformer without test-time tuning. The method separates identity information into low-frequency semantic features and high-frequency identity-specific features, compresses them into personalization embeddings through a query transformer, and binds each entity's visual, textual, and trajectory tokens with a gated self-attention mechanism. Motion embeddings are injected before text conditioning, and personalization embeddings are injected last via cross-attention, an order the ablations say best balances trajectory fidelity and identity preservation. A contrastive loss explicitly maps each entity's personalization embedding to its motion embedding. On the MSRVTT-Personalization benchmark the paper reports subject similarity and face similarity competitive with a leading multi-entity personalization method while also yielding trajectory errors around 13 to 17 pixels, far lower than a two-stage appearance-then-motion pipeline.

Load-bearing premise

The decoupled personalization extractor assumes pretrained face-recognition and person-re-identification networks provide identity-strong features for arbitrary open-set entities, including non-human objects; if those features do not transfer to unseen object categories, the high-frequency branch contributes little for non-human entities and the extractor falls back to global DINOv2 features.

Editorial extensions

If this is right

  • A single model can drive several open-set entities on independent trajectories while preserving each entity's identity, without per-subject fine-tuning.
  • Joint training of motion and appearance in one DiT avoids the identity drift and positional drift seen in two-stage pipelines that generate a customized first frame and then animate it.
  • The decoupled high-frequency features improve face similarity from 0.389 with DINOv2-only features to 0.413 in the paper's ablation, suggesting that face and object identity are best handled by separate backbones.
  • The binding strategy lowers trajectory error from 29.95 pixels with no binding and 20.74 with linear projection to 17.31 with gated self-attention, and the contrastive loss further lowers it to 14.16.
  • The curated dataset of 1.1 million video clips with entity-text-trajectory triplets is itself a reusable resource for training other controllable video generators.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The motion-personalization contrastive loss is a generic alignment mechanism that could be ported to other DiT video models with trajectory encoders, since it only needs paired motion and identity embeddings.
  • The paper's open-set claim is strongest for human faces; for non-human objects the high-frequency branch relies on a re-identification backbone, so a reader should expect the largest gains on faces and should test object categories far from the training distribution before assuming full open-set generality.
  • Because the per-entity binding attaches a reference, a text token, and a trajectory, the framework could naturally support localized video editing, such as replacing one entity's appearance or path while leaving other entities untouched.
  • A fairer comparison to the closed-source multi-entity personalization baseline would need implementation details of that baseline; the paper's qualitative comparisons use a two-stage open pipeline instead.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper presents Tora2, an extension of the Tora trajectory-controlled video diffusion transformer. It adds a decoupled personalization extractor (DPE) that concatenates DINOv2 global features with high-frequency features from ArcFace and a ReID backbone, projects them with appearance adapters, and queries them with a Q-Former to obtain per-entity personalization embeddings. A gated self-attention module binds each personalization embedding to its text tokens and trajectory motion embeddings, and a symmetric contrastive loss aligns entity and motion embeddings. The model is trained on 1.1M curated video clips, initialized from Tora/CogVideoX-5B. The authors report quantitative results on MSRVTT-Personalization showing Tora2 near Video Alchemist on subject similarity and slightly better face similarity, with trajectory error far below a Tora+Flux.1 two-stage baseline, and they claim to be the first method to handle simultaneous multi-entity appearance and motion customization.

Significance. If the claims hold, Tora2 would be a practically useful step: a tuning-free DiT that keeps several reference identities while each follows its own trajectory and text description. The architecture is coherent and the ablations support the value of the binding mechanism and the contrastive loss (Tables 3-4). However, the central open-set non-human appearance claim depends on the ReID high-frequency branch, whose effectiveness is neither properly cited nor isolated in the ablations; moreover, all quantitative comparisons are point estimates whose headline differences with Video Alchemist are under 1-2%. The significance is therefore conditional on the revision of these issues. The paper does not include code or model releases, so reproducibility rests on the textual description, which currently omits key specifications.

major comments (2)
  1. [§3.2, Eq. (3); Table 2] The claim that pretrained facial recognition and ReID backbones supply identity-strong high-frequency features for human and generic objects respectively is unsupported. The ReID model is never specified; reference [21] is an open-world tracking paper rather than a re-identification model. More seriously, the only ablation that isolates ReID, Table 2 row (b), shows that adding ReID features to DINOv2 degrades all four metrics (Text-S 0.262→0.253, Vid-S 0.717→0.698, Subj-S 0.602→0.599, Face-S 0.389→0.362). Because the full DPE also includes the face branch and the Q-Former, the improvement from DINOv2 to DPE cannot be attributed to the ReID branch, and no decomposition by non-human vs human entities is given. If ReID features do not transfer to arbitrary open-set objects, the high-frequency branch contributes nothing for non-human entities and the claimed open-set multi-entity customization reduces to the DINOv2 global features already used by Video Alchemist. This is load-bearing for the paper's main novelty claim and should be resolved by either specifying the ReID backbone and reporting a non-human-only ablation, or removing the ReID branch from the central claim.
  2. [§4.1, Table 1] All reported metrics are point estimates without error bars, confidence intervals, or significance tests, for both the MSRVTT-Personalization benchmark and the 200-video ablation set. The main quantitative claim is that Tora2 achieves 'competitive performance' with Video Alchemist: the margins are 0.008 in Face-S, 0.011 in non-human Subj-S, and 0.005 in non-human Text-S. At these magnitudes, sampling variation and evaluation randomness could easily reverse the ordering, so the comparison is not established. The same issue affects Table 2, where the DPE vs DINOv2 gain of 0.019 Subj-S should be accompanied by an estimate of variance or at least multiple seeds. Please add error bars over evaluation runs or a significance analysis for the headline comparisons.
minor comments (5)
  1. [§3.2, Eq. (5)] The text says 'two gated mechanisms' and the figure shows gates on p_i and m_i, but the notation in Eq. (5) does not show a corresponding gate for c_i; state explicitly whether the text embeddings are also gated or are updated without a gate.
  2. [§3.2, Eq. (6)] The second term of the contrastive loss is written with a dummy index j in 'log exp(m_j^T p_j / tau)' while the first term uses i; this makes the symmetric InfoNCE structure harder to read. Rewrite with explicit positive-pair indices or a cleaner symmetric form.
  3. [§3.3] The description of the filtering thresholds is ambiguous: 'camera movement angles within [0°,30°], [150°,200°], [330°,360°]' should be explained in the context of the camera-motion detector's output convention.
  4. [§4.1] The DPE components are not specified enough for reproduction: no architecture or output token count is given for the Q-Former, the appearance adapters are described only as 'linear projection' in the ablation but not in the main text, and the 'ReID backbone' is never named. Provide standard implementation details (backbone names, feature dimensions, number of learnable queries).
  5. [Abstract and Introduction] The phrase 'first method to achieve simultaneous multi-entity customization of appearance and motion' is hedged in the abstract but stated flatly in the Introduction; since the related work section lists several recent customization methods, please either add a comparison or soften the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Tora2's claims rest on trained components and out-of-sample evaluation, not on a derivation chain that reduces to its inputs.

full rationale

This is a trained-system paper rather than a derivation with fitted constants. The personalization embeddings are produced by Eqs. (3)-(4) from reference images using DINOv2 plus pretrained recognition/ReID features and a Q-Former; the gated binding, contrastive loss, and injection order are defined by Eqs. (5)-(8). None of these equations is defined in terms of the evaluation metrics used later. The reported Face-S and Subj-S scores use ArcFace and DINO features, which are the same feature families injected as conditioning, so there is a metric-feature overlap that can partially inflate identity metrics. However, the model is trained with a denoising objective and a motion-appearance contrastive loss, not by optimizing those metric scores, and the MSRVTT-Personalization benchmark is an external, out-of-distribution test set. Thus the identity improvements are not forced by construction. The ReID-transfer concern for non-human entities is an empirical-support limitation rather than circularity: the paper's own ablation (Table 2) shows adding ReID alone degrades several metrics, and the final DPE gains are attributed to the full combination including the face branch and Q-Former, not to ReID alone. Self-citations to Tora [51] are use of a prior published base model and are checked by comparison against the Tora+Flux.1 pipeline, so they are not load-bearing circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The system's stated advantages depend on feature-transfer assumptions (DINOv2 semantics plus face/ReID identity details), on annotation correctness for 1.1 million clips, and on several hand-tuned hyperparameters. These assumptions are plausible but not independently verified; no released code or data allows the reader to test them.

free parameters (5)
  • contrastive loss weight lambda = 0.2
    Set in Section 4.1; balances the contrastive loss against the denoising loss, tuned by the authors on a 200-video validation set and not justified by a derivation.
  • contrastive temperature tau = not reported
    Appears in Eq. 6; the value is not stated, so the contrastive loss is under-specified.
  • classifier-free guidance drop rates = caption 50%, reference 33%, trajectory 33%
    Set in Section 4.1; these conditional-drop probabilities are design choices that affect generation fidelity and motion control.
  • data filtering thresholds = aesthetic score above 5, flow score above 2, zoom detection 0.4 to 0.6, camera angle ranges
    Section 3.3; thresholds are hand-selected to build the 1.1 million clip training set and determine what the model learns.
  • inference guidance scale and sampling steps = text guidance 6.0, DPM 50 steps
    Section 4.1; affects generation quality and trajectory fidelity.
assumptions (5)
  • standard math Standard diffusion denoising objective with classifier-free guidance is an appropriate training target for controllable video generation.
    Section 3.1 and Eq. 1; inherited from Tora and CogVideoX without proof in this paper.
  • domain assumption DINOv2 global features capture semantic content, while face and ReID features capture identity-strong high-frequency details that are not redundant with DINOv2.
    Section 3.2, Eq. 3; the decoupling premise is supported only by the paper's own ablations and metric choices.
  • domain assumption Pretrained face recognition (ArcFace) and ReID features generalize to arbitrary open-set entities, including non-human objects.
    Section 3.2; ReID and face backbones are not trained for open-set object identity, yet the method relies on them for generic objects.
  • domain assumption The annotation pipeline (Qwen2.5-Max nouns, LISA masks, CoTracker3 tracklets) produces correct entity-text-trajectory triplets for all 1.1 million training clips.
    Section 3.3; errors in masks or tracklets would mislabel positives and negatives in the contrastive loss.
  • ad hoc to paper Gated self-attention over concatenated personalization, text, and motion tokens learns correct entity binding without explicit token-level correspondence supervision.
    Section 3.2, Eq. 5; the binding mechanism is not proven to associate the right image with the right path and word, and the paper argues empirically.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Tora2: Motion and Appearance Customized Diffusion Transformer for Multi-Entity Video Generation." pith.science (2026). https://pith.science/paper/ZWKHH53X

@misc{pith2026250705963,
  author       = {Pith},
  title        = {Pith review of: Tora2: Motion and Appearance Customized Diffusion Transformer for Multi-Entity Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZWKHH53X}},
  note         = {Machine review of arXiv:2507.05963}
}
read the original abstract

Recent advances in diffusion transformer models for motion-guided video generation, such as Tora, have shown significant progress. In this paper, we present Tora2, an enhanced version of Tora, which introduces several design improvements to expand its capabilities in both appearance and motion customization. Specifically, we introduce a decoupled personalization extractor that generates comprehensive personalization embeddings for multiple open-set entities, better preserving fine-grained visual details compared to previous methods. Building on this, we design a gated self-attention mechanism to integrate trajectory, textual description, and visual information for each entity. This innovation significantly reduces misalignment in multimodal conditioning during training. Moreover, we introduce a contrastive loss that jointly optimizes trajectory dynamics and entity consistency through explicit mapping between motion and personalization embeddings. Tora2 is, to our best knowledge, the first method to achieve simultaneous multi-entity customization of appearance and motion for video generation. Experimental results demonstrate that Tora2 achieves competitive performance with state-of-the-art customization methods while providing advanced motion control capabilities, which marks a critical advancement in multi-condition video generation. Project page: https://ali-videoai.github.io/Tora2_page/.

Figures

Figures reproduced from arXiv: 2507.05963 by the authors.

Figure 1
Figure 1. Given text prompts, motion trajectories, and reference images for human entities (e.g., man, woman) and non-human [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An overview of Tora2, which consists of a decoupled personalization extractor (DPE), a trajectory extractor, a video [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Qualitative comparison of appearance and motion customization for multiple entities. The depth map from the first [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Ablation study of cross-model binding strategies. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 14 canonical work pages

  1. [1]

    Fan Bao, Chendong Xiang, Gang Yue, Guande He, Hongzhou Zhu, Kaiwen Zheng, Min Zhao, Shilong Liu, Yaole Wang, and Jun Zhu. 2024. Vidu: a Highly Con- sistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models. arXiv:2405.04233

    arXiv 2024

  2. [2]

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, and Robin Rombach. 2023. Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets. arXiv:2311.15127

    arXiv 2023

  3. [3]

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. 2024. Video generation models as world simulators. https: //openai.com/research/video-generation-models-as-world-simulators

    work page 2024

  4. [4]

    Haoxin Chen, Menghan Xia, Yin-Yin He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, Chao-Liang Weng, and Ying Shan. 2023. VideoCrafter1: Open Diffusion Models for High-Quality Video Generation. arXiv:2310.19512

    arXiv 2023

  5. [5]

    Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Yuwei Fang, Kwot Sin Lee, Ivan Skorokhodov, Kfir Aberman, Jun-Yan Zhu, Ming-Hsuan Yang, and Sergey Tulyakov. 2025. Multi-subject Open-set Personalization in Video Generation. arXiv:2501.06187 [cs.CV]

    arXiv 2025

  6. [6]

    Zuozhuo Dai, Zhenghao Zhang, Yao Yao, Bingxue Qiu, Siyu Zhu, Long Qin, and Weizhi Wang. 2023. Fine-Grained Open Domain Image Animation with Motion Guidance. arXiv:2311.12886

    arXiv 2023

  7. [7]

    Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. 2019. ArcFace: Additive Angular Margin Loss for Deep Face Recognition. InIEEE Conf. Comput. Vis. Pattern Recog.Computer Vision Foundation / IEEE, 4690–4699. doi:10.1109/ CVPR.2019.00482

    arXiv 2019

  8. [8]

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Yaohui Wang, Yu Qiao, Dahua Lin, and Bo Dai. 2023. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv:2307.04725

    arXiv 2023

Show all 51 references
  1. [9]

    Xuanhua He, Quande Liu, Shengju Qian, Xin Wang, Tao Hu, Ke Cao, Keyu Yan, and Jie Zhang. 2024. ID-Animator: Zero-Shot Identity-Preserving Human Video Generation. arXiv:2404.15275 [cs.CV]

    2024 arXiv

  2. [10]

    Jonathan Ho and Tim Salimans. 2022. Classifier-free diffusion guidance. arXiv:2207.12598

    2022 arXiv

  3. [11]

    Belongie

    Xun Huang and Serge J. Belongie. 2017. Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization. InInt. Conf. Comput. Vis.IEEE Computer Society, 1510–1519. doi:10.1109/ICCV.2017.167

    2017 doi

  4. [12]

    Yuzhou Huang, Ziyang Yuan, Quande Liu, Qiulin Wang, Xintao Wang, Ruimao Zhang, Pengfei Wan, Di Zhang, and Kun Gai. 2025. ConceptMaster: Multi- Concept Video Customization on Diffusion Transformer Models Without Test- Time Tuning. arXiv:2501.04698 [cs.CV]

    2025 arXiv

  5. [14]

    Nikita Karaev, Iurii Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. 2024. CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. arXiv:2410.11831 [cs.CV]

    2024 arXiv

  6. [15]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Opti- mization. InInt. Conf. Learn. Represent., Yoshua Bengio and Yann LeCun (Eds.)

    2015

  7. [16]

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, Kathrina Wu, Qin Lin, Junkun Yuan, Yanxin Long, Aladdin Wang, Andong Wang, Changlin Li, Duojun Huang, Fang Yang, Hao Tan, Hongmei Wang, Jacob Song, Jiawang Bai, J...

    2025 arXiv

  8. [17]

    Shakker Labs. [n. d.]. Flux.1-dev-controlnet-union-pro. Accessed 2024 [Online]. https://huggingface.co/Shakker-Labs/FLUX.1-dev-ControlNet-Union-Pro

    2024

  9. [19]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven C. H. Hoi. 2022. BLIP: Boot- strapping Language-Image Pre-training for Unified Vision-Language Under- standing and Generation. InInt. Conf. Mach. Learn. (Proceedings of Machine Learning Research, Vol. 162), Kamalika Chaudhuri, St...

    2022

  10. [20]

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, and Lei Zhang. 2024. Grounding DINO: Marrying DINO with Grounded Pre-training for Open-Set Object Detection. InEur. Conf. Comput. Vis. (Lecture Notes ...

    2024 doi

  11. [22]

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu

  12. [23]

    Lewis, and W

    Wan-Duo Kurt Ma, John P. Lewis, and W. Bastiaan Kleijn. 2024. TrailBlazer: Trajectory Control for Diffusion-Based Video Generation. InSIGGRAPH Asia, Takeo Igarashi, Ariel Shamir, and Hao (Richard) Zhang (Eds.). ACM, 97:1–97:11. doi:10.1145/3680528.3687652

    2024

  13. [24]

    Arun Mallya, Ting-Chun Wang, and Ming-Yu Liu. 2022. Implicit Warp- ing for Animation with Image Sets. InAdv. Neural Inform. Process. Syst., Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (Eds.). 22438–22450. http://papers.nips.cc/paper_files/paper/2...

    2022

  14. [25]

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po- Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabb...

    2024

  15. [26]

    William Peebles and Saining Xie. 2023. Scalable Diffusion Models with Transform- ers. InInt. Conf. Comput. Vis.IEEE, 4172–4182. doi:10.1109/ICCV51070.2023.00387

    2023

  16. [27]

    Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, David Yan, Dhruv Choudhary, Dingkang Wang, Geet Sethi, Guan Pang, Haoyu Ma, Ishan Misra, Ji Hou, Jialiang Wang, Kiran Jagadeesh, Kunpeng Li, Lu...

    2025 arXiv

  17. [28]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. InInt. Conf. M...

    2021

  18. [29]

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. 2024. Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks. ...

    2024 arXiv

  19. [30]

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. InMedical Image Computing and Computer-Assisted Intervention (Lecture Notes in Computer Science, Vol. 9351), Nassir Navab, Joachim Hornegger, William M. W...

    2015 doi

  20. [32]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. InAdv. Neural Inform. Process. Syst., Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S...

    2017

  21. [33]

    Chien-Yao Wang, I-Hau Yeh, and Hong-Yuan Mark Liao. 2024. YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information. InEur. Conf. Comput. Vis. (Lecture Notes in Computer Science, Vol. 15089), Ales Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Tor...

    2024 doi

  22. [34]

    Hanlin Wang, Hao Ouyang, Qiuyu Wang, Wen Wang, Ka Leong Cheng, Qifeng Chen, Yujun Shen, and Limin Wang. 2024. LeviTor: 3D Trajectory Oriented Image-to-Video Synthesis. arXiv:2412.15214 [cs.CV]

    2024 arXiv

  23. [35]

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. 2023. Modelscope text-to-video technical report. arXiv:2308.06571

    2023 arXiv

  24. [36]

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Jiuniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jingren Zhou. 2023. Video- Composer: Compositional Video Synthesis with Motion Controllabil- ity. InAdv. Neural Inform. Process. Syst., Alice Oh, Tristan Naumann, Amir Glo...

    2023

  25. [38]

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Tianshui Chen, Menghan Xia, Ping Luo, and Yin Shan. 2023. MotionCtrl: A Unified and Flexible Motion Controller for Video Generation. arXiv:2312.03641

    2023 arXiv

  26. [39]

    Yujie Wei, Shiwei Zhang, Zhiwu Qing, Hangjie Yuan, Zhiheng Liu, Yu Liu, Yingya Zhang, Jingren Zhou, and Hongming Shan. 2024. Dream Video: Composing Your Dream Videos with Customized Subject and Motion. InIEEE Conf. Comput. Vis. Pattern Recog.IEEE, 6537–6549. doi:10.1109/CVPR52...

    2024

  27. [40]

    Yujie Wei, Shiwei Zhang, Hangjie Yuan, Xiang Wang, Haonan Qiu, Rui Zhao, Yutong Feng, Feng Liu, Zhizhong Huang, Jiaxin Ye, Yingya Zhang, and Hongming Shan. 2024. DreamVideo-2: Zero-Shot Subject-Driven Video Customization with Precise Motion Control. arXiv:2410.13830 [cs.CV]

    2024 arXiv

  28. [41]

    Jianzong Wu, Xiangtai Li, Yanhong Zeng, Jiangning Zhang, Qianyu Zhou, Yining Li, Yunhai Tong, and Kai Chen. 2024. MotionBooth: Motion-Aware Customized Text-to-Video Generation. arXiv:2406.17758 [cs.CV]

    2024 arXiv

  29. [42]

    Weijia Wu, Zhuang Li, Yuchao Gu, Rui Zhao, Yefei He, David Junhao Zhang, Mike Zheng Shou, Yan Li, Tingting Gao, and Di Zhang. 2024. DragAnything: Motion Control for Anything Using Entity Representation. InEur. Conf. Comput. Vis. (Lecture Notes in Computer Science, Vol. 15080),...

    2024 doi

  30. [43]

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, Fisher Yu, Dacheng Tao, and Andreas Geiger. 2023. Unifying Flow, Stereo and Depth Estimation.IEEE Trans. Pattern Anal. Mach. Intell.45, 11 (2023), 13941–13958. doi:10.1109/TPAMI. 2023.3298645

    2023

  31. [44]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al . 2024. Qwen2.5 technical report. arXiv:2412.15115

    2024 arXiv

  32. [45]

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al . 2024. CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer. arXiv:2408.06072

    2024 arXiv

  33. [46]

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. 2023. IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models. arXiv:2308.06721 [cs.CV]

    2023 arXiv

  34. [47]

    Shengming Yin, Chenfei Wu, Jian Liang, Jie Shi, Houqiang Li, Gong Ming, and Nan Duan. 2023. DragNUWA: Fine-grained control in video generation by integrating text, image, and trajectory. arXiv:2308.08089

    2023 arXiv

  35. [48]

    Hauptmann, Ming-Hsuan Yang, Yuan Hao, Irfan Essa, and Lu Jiang

    Lijun Yu, Yong Cheng, Kihyuk Sohn, José Lezama, Han Zhang, Huiwen Chang, Alexander G. Hauptmann, Ming-Hsuan Yang, Yuan Hao, Irfan Essa, and Lu Jiang

  36. [49]

    Shenghai Yuan, Jinfa Huang, Xianyi He, Yunyuan Ge, Yujun Shi, Liuhan Chen, Jiebo Luo, and Li Yuan. 2024. Identity-Preserving Text-to-Video Generation by Frequency Decomposition. arXiv:2411.17440 [cs.CV]

    2024 arXiv

  37. [50]

    Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou. 2023. I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models. arXiv:2311.04145

    2023 arXiv

  38. [51]

    Zhenghao Zhang, Junchao Liao, Menghao Li, Zuozhuo Dai, Bingxue Qiu, Siyu Zhu, Long Qin, and Weizhi Wang. 2024. Tora: Trajectory-oriented Diffusion Transformer for Video Generation. arXiv:2407.21705 [cs.CV]

    2024 arXiv

  39. [52]

    Jian Zhao and Hui Zhang. 2022. Thin-Plate Spline Motion Model for Image Animation. InIEEE Conf. Comput. Vis. Pattern Recog.IEEE, 3647–3656. doi:10. 1109/CVPR52688.2022.00364

    2022

  40. [53]

    Yupeng Zhou, Daquan Zhou, Ming-Ming Cheng, Jiashi Feng, and Qibin Hou

  41. [2022]

    DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps. InAdv. Neural Inform. Process. Syst., Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35. 5775–5787. http://papers.nips.cc/paper_files/paper/202...

    2022

  42. [2023]

    InIEEE Conf

    MAGVIT: Masked Generative Video Transformer. InIEEE Conf. Comput. Vis. Pattern Recog.IEEE, 10459–10469. doi:10.1109/CVPR52729.2023.01008

    2023

  43. [2024]

    StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generation. InAdv. Neural Inform. Process. Syst., Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (Eds.), Vol. 37. 110315–110340. http://pap...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.