Pith. sign in

REVIEW 2 major objections 4 minor 50 references

Video Diffusion Transformers encode motion and structure in distinct attention heads that can be controlled for training-free motion transfer.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 07:10 UTC pith:WAT4GHOU

load-bearing objection Solid training-free motion-transfer method that turns a clean head-level analysis of video DiTs into usable control primitives; the timestep-stability gap is real but secondary to the empirical gains. the 2 major comments →

arxiv 2607.11081 v1 pith:WAT4GHOU submitted 2026-07-13 cs.CV cs.AI

Controlling Motion Transfer in Diffusion Transformers via Attention Heads

Sunyoung Jung , Jiwoo Park , Yoonseok Choi , Kyobin Choo , Ming-Hsuan Yang , Seong Jae Hwang This is my paper
classification cs.CV cs.AI
keywords Motion TransferDiffusion TransformersAttention HeadsVideo GenerationTraining-free Controllable GenerationSemantic CorrespondenceDisplacement Maps
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Video Diffusion Transformers produce high-quality, coherent video, yet turning them into motion-transfer tools—copying a reference video’s movement while obeying a new text prompt—has been hard because no one knew how those models store motion versus layout. This paper shows that individual attention heads already specialize: some track cross-frame motion, others lock onto spatial structure. By reading only the motion heads, refining their displacement maps with semantic correspondences, and injecting features only from the low-entropy structure heads, the authors build a training-free method called HALO that follows the reference motion while staying faithful to the target prompt. A sympathetic reader cares because the same head-level view both beats prior DiT baselines on fidelity metrics and supplies an interpretable control surface for video generation more generally.

Core claim

Video DiTs contain two functionally distinct subsets of attention heads: motion-specific heads whose cross-frame attention yields displacement maps aligned with optical flow, and structure-specialized heads whose attention maps have low entropy and preserve spatial layout. Controlling only those heads—optimizing semantic-aware displacements from the former and selectively injecting value features from the latter—produces accurate motion transfer without any parameter updates.

What carries the argument

HALO: a head-aware pipeline that constructs and semantically reweights displacement maps from motion-specific heads, then injects value features solely from low-entropy structure-specialized heads during denoising.

Load-bearing premise

The same heads stay specialized for motion or structure across different videos, denoising steps, and model variants, so a fixed selection rule remains valid.

What would settle it

On held-out videos or a third DiT architecture, check whether the temporal-pattern heads still show higher directional alignment and correlation with optical flow, and whether low-entropy heads still improve structural metrics, relative to random or all-head baselines; if not, the specialization claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Existing video DiTs can perform motion transfer without fine-tuning or extra parameters.
  • The same structure-specialized heads support identity-preserving video editing by selective value injection.
  • Head specialization transfers across at least two DiT architectures (CogVideoX and Wan) under identical hyperparameters.
  • Head-level analysis supplies an interpretable foundation for other controllable video generation tasks such as motion direction or intensity control.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If head specialization is largely architecture-invariant, the same labeling procedure could unlock control in larger future DiTs without re-deriving the heads from scratch.
  • Entropy-based head selection may generalize to other generative transformers in which spatial and temporal roles are entangled inside unified attention.
  • The improved CLIP–motion-fidelity balance suggests that disentangling motion from semantics at the head level can reduce the classic trade-off that has plagued motion-transfer methods.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper analyzes attention heads in video Diffusion Transformers (primarily CogVideoX, with transfer to Wan) and identifies two functional subsets: motion-specific heads (temporal-pattern masks whose cross-frame displacement maps align with optical flow via Directional Alignment and Correlation) and structure-specialized heads (low attention-map entropy that correlates with low spatial feature entropy). Building on this analysis, HALO constructs a reference displacement map from motion heads, refines it with semantic correspondence from diffusion features (SCR + SRW), optimizes the latent via an L2 semantic-motion loss L_SM, and selectively injects value features from low-entropy heads to preserve spatial layout. The method is training-free. Experiments on a standard motion-transfer benchmark and a new Movie Scene Dataset show gains in CLIP, Motion Fidelity and FTD over DiTFlow, RoPECraft and GWTF, supported by ablations (Table 3), head-configuration controls (Table 6), a user study and qualitative failure-mode analysis.

Significance. If the head specialization is reliable, the work supplies both a practical training-free motion-transfer method that improves the CLIP–MF trade-off and an interpretable control primitive for DiT-based video generation. Strengths include explicit head-level diagnostics (DA/Corr vs. optical flow; entropy–feature-entropy correlation), component ablations that isolate SCR and selective injection, cross-architecture transfer to Wan with the same hyperparameters, a production-oriented Movie Scene Dataset, and open code/presentation materials. These elements make the contribution more than an incremental engineering tweak and give the community a concrete mechanistic handle on motion versus structure inside video DiTs.

major comments (2)
  1. Sec. 3 and Supp. C.4/Figs. 15–16 establish strong cross-sample entropy consistency (SSIM 0.988) and a median ≈7, and the Wan transfer is reassuring. However, the statistics are averages over timesteps. The optimization loop (Alg. 1, t > T_opt) runs at high-noise steps; there is no per-timestep ranking of the same heads by DA/Corr or entropy. If the top motion heads or the τ<7 set change substantially with noise level, the claim that these heads are fixed, content-stable control primitives is only partially supported. A short per-timestep stability plot (or a statement that the ranking is invariant) would close this gap without altering the empirical pipeline.
  2. Table 1 and the CLIP–MF discussion (Supp. F.2) correctly note that TC can favor static outputs. The paper already reports higher motion dynamics for HALO and a favorable user-study TC ranking. Still, the main-table claim of “balanced” superiority would be clearer if the authors either (a) report a joint CLIP–MF Pareto figure in the main text or (b) explicitly qualify that the modest TC drop is expected under higher dynamics. This is a presentation-of-evidence issue rather than a contradiction of the numbers.
minor comments (4)
  1. Fig. 3(b) and the DA/Corr definitions (Supp. C.3) would benefit from a one-sentence statement of how many frame pairs and which optical-flow estimator (RAFT) are used for the head ranking, so the numbers are fully reproducible from the main text alone.
  2. Hyper-parameter choices (β=0.1, τ<7, top-k=4, T_opt=12) are justified in Supp. C.1; a brief pointer in Sec. 5 would help readers who do not immediately consult the supplement.
  3. The Movie Scene Dataset is a useful addition; a short note on licensing/availability (beyond the RASCA FX acknowledgment) would strengthen its utility as a community resource.
  4. Occasional typographical inconsistencies (e.g., “HALOgeneratesvideosthatfollow…”) appear in the abstract and figure captions; a light copy-edit pass would improve polish.

Circularity Check

0 steps flagged

No circularity: head specialization is measured by independent optical-flow/entropy statistics; motion transfer optimizes an external L2 displacement objective and is scored on held-out CLIP/MF/FTD benchmarks.

full rationale

The paper's derivation chain is self-contained and non-circular. Motion-specific heads are identified by matching attention maps to temporal pattern masks (SparseGen-style) and by computing directional alignment / Pearson correlation of the resulting displacement maps against independent RAFT optical flow (Sec. 3, Analysis 1, Eqs. 1-2, Fig. 3). Structure-specialized heads are identified by low attention-map entropy correlating with low spatial feature entropy (Sec. 3, Analysis 2, Fig. 4). These labels are then used as fixed selectors inside a training-free pipeline: D_ref is built only from the motion heads and refined by external DIFT semantic correspondences (Sec. 4.1, Eqs. 3-5, SCR/SRW); structure is preserved by injecting value features only from the low-entropy heads (Sec. 4.2). The optimization objective L_SM is simply the L2 distance between the refined reference displacement and the current latent's displacement; final claims are ordinary empirical comparisons (CLIP, MF, FTD, user study) against prior methods on public and newly curated benchmarks (Tables 1-6). No equation equates a claimed performance gain to a quantity defined solely by the fitted hyper-parameters (beta, tau, top-k, T_opt), no uniqueness theorem is imported from the authors' prior work, and no known empirical pattern is merely renamed. Minor internal cross-references to the authors' own analysis sections do not create definitional dependence. The result is therefore an ordinary empirical engineering contribution resting on independent measurements, not a circular derivation.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 2 invented entities

The central claim rests on empirical regularities observed inside two concrete DiT checkpoints plus a handful of hand-chosen scalars that control the optimization and injection stages. No new physical entities are postulated; the ‘motion-specific’ and ‘structure-specialized’ heads are labels for subsets discovered by the analysis rather than free inventions.

free parameters (4)
  • semantic bias strength β = 0.1
    Fixed to 0.1 after a small sweep; larger values suppress motion dynamics (Table 8).
  • entropy threshold τ = <7
    Set to the observed median entropy ≈7; used to select structure heads for injection (Table 6b, Supp. C.4).
  • optimization steps T_opt = 12
    Chosen as 12 after observing diminishing returns (Table 7).
  • top-k attention candidates = 4
    Fixed to 4; larger k injects noisy matches and degrades MF (Table 10).
axioms (3)
  • domain assumption Cross-frame attention of temporal heads yields displacement maps that are faithful proxies for optical flow.
    Validated only on the tested models and RAFT flow; invoked throughout Sec. 3 and 4.1.
  • domain assumption Attention-map entropy is a reliable indicator of structural content in the corresponding value features.
    Supported by correlation plots (Fig. 4) but treated as general for head selection.
  • domain assumption Diffusion intermediate features (DIFT-style) supply object-level semantic correspondence usable for reweighting.
    Imported from prior correspondence literature and used without re-derivation in Sec. 4.1.
invented entities (2)
  • motion-specific heads (subset M) no independent evidence
    purpose: Source of clean inter-frame displacement maps for motion optimization.
    Defined operationally by temporal-pattern masks and DA/Corr scores; no independent existence outside the analysis.
  • structure-specialized heads (low-entropy subset) no independent evidence
    purpose: Source of clean value features for selective injection that preserves layout.
    Defined by entropy threshold; shown to help editing but still an internal label of the model.

pith-pipeline@v1.1.0-grok45 · 25183 in / 2667 out tokens · 24883 ms · 2026-07-14T07:10:27.720745+00:00 · methodology

0 comments
read the original abstract

Diffusion Transformers (DiTs) have advanced video generation with high-quality, temporally coherent results. However, extending them to motion transfer, which requires following reference motion while aligning with a target prompt, remains challenging due to limited understanding of motion and structure representations within DiTs. We analyze video DiTs at the attention-head level and identify distinct heads specialized for motion and spatial structure. Based on this insight, we propose a head-aware controllable motion transfer framework that requires no parameter updates. Our method refines motion cues from motion-specialized heads via semantic correspondence guidance and preserves structure through selective feature injection. This head-level control not only enables accurate motion transfer but also provides an interpretable foundation for controllable video generation with DiTs.

Figures

Figures reproduced from arXiv: 2607.11081 by Jiwoo Park, Kyobin Choo, Ming-Hsuan Yang, Seong Jae Hwang, Sunyoung Jung, Yoonseok Choi.

Figure 1
Figure 1. Figure 1: Overview. We present HALO, a head-aware controllable motion transfer framework for video Diffusion Transformers, which identifies motion- and structure￾specialized attention heads within the model. Leveraging these findings, HALO gener￾ates videos that follow the target prompt while remaining motion- and structure-aligned with reference videos, achieving accurate motion transfer. Abstract. Diffusion Transf… view at source ↗
Figure 2
Figure 2. Figure 2: Limitations of displacement-based motion transfer. Comparison between HALO and displacement-only optimization [35]. (a) Lack of semantic alignment causes motion errors. (b) Missing structural preservation leads to spatial misalignment. HALO ensures consistent motion and spatial fidelity. introduce semantic guidance modules that align motion cues with semantic sim￾ilarities derived from diffusion features, … view at source ↗
Figure 3
Figure 3. Figure 3: Head Configuration Comparison. (a) Attention maps show distinct patterns: temporal heads capture cross-frame diag￾onals, while spatial heads maintain intra￾frame locality. (b) Quantitative evaluation using directional alignment and correla￾tion shows that temporal heads align more closely with reference motion. (c) Displace￾ment maps further confirm temporal heads more accurately capture motion flow. Atten… view at source ↗
Figure 5
Figure 5. Figure 5: Overview of HALO. (a) From a reference video, we extract displacement maps and head features. Displacements from motion-specific heads guide motion by opti￾mizing the latent representation, while selected head features from the reference are injected to preserve structure during generation. (b) To enhance motion guidance, se￾mantic correspondence derived from diffusion features refines the displacement map… view at source ↗
Figure 6
Figure 6. Figure 6: Details of Semantic Correspondence Refinement (SCR) and Semantic Reweight￾ing (SRW). (a) I ref f,f′ is obtained by choosing, among top-k attention candidates, the patch closest to the semantic best match I cor f,f′ . (b) SRW manipulates target cross-frame attention by adding a correspondence-based bias at I cor f,f′ . we form the bias matrix Bf,f′ : \mathcal {B}_{f,f'}(i,j) = \begin {cases} \beta , & \text… view at source ↗
Figure 7
Figure 7. Figure 7: Effect of SCR and SRW on displacement maps. The refinements improve ro￾bustness to fine-grained object motion and better preserve object shape. We therefore select structurally informative heads via entropy-based anal￾ysis (Analysis 2, Sec. 3). Low-entropy heads exhibit strong diagonal attention and produce spatially coherent features; we inject their value features into the corresponding heads, supplying … view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative comparison between U-Net- and DiT-based baselines and HALO. 5.1 Main Results Qualitative Results [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative results in the movie scene dataset. works exhibit a trade-off between these metrics, as shown in [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Qualitative results of our method across ablation studies. Exp.# corresponds to [PITH_FULL_IMAGE:figures/full_fig_p012_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Qualitative results on complex motion and video editing. (a) Generated sam￾ples illustrating the HALO’s capability in handling complex motion dynamics. (b) Application to video editing using the structure-specialized head. our analysis showing a median entropy of 7, we adopt τ=7 as the threshold. Injecting features from high-entropy heads (τ>7) yields lower MF and FTD scores, indicating that such heads co… view at source ↗
Figure 12
Figure 12. Figure 12: Limitations of existing DiT-based motion transfer methods. (a) The gener￾ated subject exhibits structural distortion and misalignment, failing to preserve the spatial layout of the reference. (b) Identity cues from the reference video leak into the generated subject, causing identity corrup￾tion and appearance inconsistency across frames [PITH_FULL_IMAGE:figures/full_fig_p017_12.png] view at source ↗
Figure 14
Figure 14. Figure 14: Qualitative results with corresponding reference displacement maps Dref in hyperparameter Top-k analysis [PITH_FULL_IMAGE:figures/full_fig_p019_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: (a) Attention Entropy Distribution: Distribution of attention-map entropy values across samples. (b) Layer-wise Head Entropy Consistency: Distribution of head￾level entropy values across layers, showing consistent patterns across different samples. Similarity Metric (SSIM): A heatmap is used to visualize pairwise similarity between samples. \text {DA} = \mathrm {mean}\Big \langle \frac {f_{\text {ref}}}{\… view at source ↗
Figure 16
Figure 16. Figure 16: Entropy distribution across all layers, computed by averaging the entropy of all attention heads within each layer. The orange dashed line denotes the median entropy value across layers, which is 6.78. A monkey is riding a bicycle on a trail next to a mural-covered wall in an urban park. Reference HALO (a) (b) [PITH_FULL_IMAGE:figures/full_fig_p022_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Qualitative results under different entropy thresholds. (a) Injecting features only from heads with entropy greater than 7. (b) Injecting features from all heads. this consistency, we compute similarity metrics using heat maps that visualize entropy values across the entire benchmark ( [PITH_FULL_IMAGE:figures/full_fig_p022_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Qualitative results and reference displacement maps Dref across various at￾tention head configurations. We evaluate various entropy ranges to confirm that our selected entropy range, τ , consistently identifies structural-specialized heads [PITH_FULL_IMAGE:figures/full_fig_p023_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Displacement-field comparison between temporal and spatial heads in Wan. Temporal heads better capture reference motion, showing clearer separation of back￾ground and object motion [PITH_FULL_IMAGE:figures/full_fig_p024_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Quantitative comparison of each head in Wan using Directional Alignment (DA) and Correlation (Corr) with ground￾truth optical flow [PITH_FULL_IMAGE:figures/full_fig_p024_20.png] view at source ↗
Figure 22
Figure 22. Figure 22: Motion transfer results of HALO applied to Wan [PITH_FULL_IMAGE:figures/full_fig_p025_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Overview of our movie scene dataset: video samples and corresponding prompts [PITH_FULL_IMAGE:figures/full_fig_p025_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: Qualitative comparison across diverse motion transfer tasks with the movie scene dataset. HALO is evaluated against U-Net- and DiT-based baselines across mul￾tiple subjects and motions. F Additional Experiment Results F.1 More Qualitative Results We provide additional qualitative results in [PITH_FULL_IMAGE:figures/full_fig_p027_24.png] view at source ↗
Figure 25
Figure 25. Figure 25: Qualitative results of HALO on benchmark and movie-scene datasets. The top rows illustrate performance on the motion transfer benchmark, while the bottom rows show generalization to movie scene videos [PITH_FULL_IMAGE:figures/full_fig_p028_25.png] view at source ↗
Figure 26
Figure 26. Figure 26: Qualitative comparison across diverse motion transfer tasks. HALO is evalu￾ated against U-Net- and DiT-based baselines across multiple subjects and motions. F.2 Quantitative Trade-off and metric Analysis CLIP vs. MF trade-off Given that motion transfer necessitates a delicate bal￾ance between preserving target semantics and following reference motion, CLIP and MF serve as complementary metrics that should… view at source ↗
Figure 27
Figure 27. Figure 27: Objective comparison across motion-transfer methods in terms of Motion Fidelity (MF) and CLIP Score (CLIP). Baseline methods exhibit a trade-off between MF and CLIP, whereas HALO (Ours) achieves balanced performance. ranks HALO highest in perceived temporal consistency, indicating that the lower automated TC score does not necessarily imply weaker temporal coherence [PITH_FULL_IMAGE:figures/full_fig_p030… view at source ↗
Figure 28
Figure 28. Figure 28: Failure cases of our method. In these examples, the output closely follows the reference motion but exhibits unnatural articulated movements (e.g., a penguin walk￾ing like a bear), which occurs when semantic correspondence over-aligns structurally incompatible articulations. methods. Specifically, our approach remains highly competitive with DiTFlow and RopeCraft, achieving faithful motion transfer withou… view at source ↗
Figure 29
Figure 29. Figure 29: Fine-grained local motion transfer. HALO preserves coarse facial motion and head pose, but subtle non-rigid deformations, such as detailed facial expressions, remain challenging due to the patch-level displacement representation. Reference cow → rhino cow → tiger Reference flamingo → crane flamingo → goose [PITH_FULL_IMAGE:figures/full_fig_p032_29.png] view at source ↗
Figure 30
Figure 30. Figure 30: Application to video editing using the structure-specialized heads. to remain unchanged from the reference. As shown in [PITH_FULL_IMAGE:figures/full_fig_p032_30.png] view at source ↗
Figure 31
Figure 31. Figure 31: Structural consistency in long-video generation. We evaluate HALO on 112- frame sequences and visualize the behavior of selected low-entropy heads. The selected heads maintain stable structural attention patterns over extended temporal horizons, supporting long-sequence structural preservation. over time. This indicates that entropy-based head selection is not limited to short clips but provides stable st… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

50 extracted references · 12 linked inside Pith

  1. [1]

    arXiv e-prints pp

    Ahn, D., Kang, J., Lee, S., Kim, M., Min, J., Jang, W., Lee, S., Paul, S., Hong, S., Kim, S.: Fine-grained perturbation guidance via attention head selection. arXiv e-prints pp. arXiv–2506 (2025)

    2025

  2. [2]

    arXiv preprint arXiv:2412.07750 (2024)

    Atzmon, Y., Gal, R., Tewel, Y., Kasten, Y., Chechik, G.: Motion by queries: Identity-motion trade-offs in text-to-video generation. arXiv preprint arXiv:2412.07750 (2024)

    Pith/arXiv arXiv 2024

  3. [3]

    In: CVPR

    Avrahami, O., Patashnik, O., Fried, O., Nemchinov, E., Aberman, K., Lischinski, D., Cohen-Or, D.: Stable flow: Vital layers for training-free image editing. In: CVPR. pp. 7877–7888 (2025)

    2025

  4. [4]

    In: SIGGRAPH Asia 2024 Conference Papers

    Bar-Tal, O., Chefer, H., Tov, O., Herrmann, C., Paiss, R., Zada, S., Ephrat, A., Hur, J., Liu, G., Raj, A., et al.: Lumiere: A space-time diffusion model for video generation. In: SIGGRAPH Asia 2024 Conference Papers. pp. 1–11 (2024)

    2024

  5. [5]

    Geographical analysis6(1), 1–31 (1974)

    Batty, M.: Spatial entropy. Geographical analysis6(1), 1–31 (1974)

    1974

  6. [6]

    arXiv preprint arXiv:2311.15127 (2023)

    Blattmann, A., Dockhorn, T., Kulal, S., Mendelevitch, D., Kilian, M., Lorenz, D., Levi, Y., English, Z., Voleti, V., Letts, A., et al.: Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127 (2023)

    Pith/arXiv arXiv 2023

  7. [7]

    In: CVPR

    Burgert, R., Xu, Y., Xian, W., Pilarski, O., Clausen, P., He, M., Ma, L., Deng, Y., Li, L., Mousavi, M., et al.: Go-with-the-flow: Motion-controllable video diffusion models using real-time warped noise. In: CVPR. pp. 13–23 (2025)

    2025

  8. [8]

    In: CVPR

    Cai, M., Cun, X., Li, X., Liu, W., Zhang, Z., Zhang, Y., Shan, Y., Yue, X.: Ditctrl: Exploring attention control in multi-modal diffusion transformer for tuning-free multi-prompt longer video generation. In: CVPR. pp. 7763–7772 (2025)

    2025

  9. [9]

    In: CVPR

    Chen, H., Zhang, Y., Cun, X., Xia, M., Wang, X., Weng, C., Shan, Y.: Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In: CVPR. pp. 7310–7320 (2024)

    2024

  10. [10]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Chen, S., Xu, M., Ren, J., Cong, Y., He, S., Xie, Y., Sinha, A., Luo, P., Xiang, T., Perez-Rua, J.M.: Gentron: Diffusion transformers for image and video generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6441–6451 (2024)

    2024

  11. [11]

    ACM Transactions on Graphics (ToG)41(2), 1–21 (2021)

    Du, T., Wu, K., Ma, P., Wah, S., Spielberg, A., Rus, D., Matusik, W.: Diffpd: Differentiable projective dynamics. ACM Transactions on Graphics (ToG)41(2), 1–21 (2021)

    2021

  12. [12]

    In: ICML (2024)

    Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F., et al.: Scaling rectified flow transformers for high-resolution image synthesis. In: ICML (2024)

    2024

  13. [13]

    In: CVPR

    Gao,J.,Yin,Z.,Hua,C.,Peng,Y.,Liang,K.,Ma,Z.,Guo,J.,Liu,Y.:Conmo:Con- trollable motion disentanglement and recomposition for zero-shot motion transfer. In: CVPR. pp. 7191–7200 (2025)

    2025

  14. [14]

    arXiv preprint arXiv:2505.13344 (2025)

    Gokmen, A.B., Ekin, Y., Bilecen, B.B., Dundar, A.: Ropecraft: Training-free mo- tion transfer with trajectory-guided rope optimization on diffusion transformers. arXiv preprint arXiv:2505.13344 (2025)

    arXiv 2025

  15. [15]

    In: ICLR (2024),https://openreview.net/forum?id= Fx2SbBgcte

    Guo, Y., Yang, C., Rao, A., Liang, Z., Wang, Y., Qiao, Y., Agrawala, M., Lin, D., Dai, B.: Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. In: ICLR (2024),https://openreview.net/forum?id= Fx2SbBgcte

    2024

  16. [16]

    arXiv preprint arXiv:2501.00103 (2024) 36 S

    HaCohen, Y., Chiprut, N., Brazowski, B., Shalem, D., Moshe, D., Richardson, E., Levin, E., Shiran, G., Zabari, N., Gordon, O., et al.: Ltx-video: Realtime video latent diffusion. arXiv preprint arXiv:2501.00103 (2024) 36 S. Jung et al

    Pith/arXiv arXiv 2024

  17. [17]

    NeurIPS36, 8266– 8279 (2023)

    Hedlin, E., Sharma, G., Mahajan, S., Isack, H., Kar, A., Tagliasacchi, A., Yi, K.M.: Unsupervised semantic correspondence using stable diffusion. NeurIPS36, 8266– 8279 (2023)

    2023

  18. [18]

    In: Proceedings of the 2021 conference on empirical methods in natural language processing

    Hessel, J., Holtzman, A., Forbes, M., Le Bras, R., Choi, Y.: Clipscore: A reference- free evaluation metric for image captioning. In: Proceedings of the 2021 conference on empirical methods in natural language processing. pp. 7514–7528 (2021)

    2021

  19. [19]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024)

    Huang, Z., He, Y., Yu, J., Zhang, F., Si, C., Jiang, Y., Zhang, Y., Wu, T., Jin, Q., Chanpaisit, N., Wang, Y., Chen, X., Wang, L., Lin, D., Qiao, Y., Liu, Z.: VBench: Comprehensive benchmark suite for video generative models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024)

    2024

  20. [20]

    arXiv preprint arXiv:2410.21276 (2024)

    Hurst, A., Lerer, A., Goucher, A.P., Perelman, A., Ramesh, A., Clark, A., Os- trow, A., Welihinda, A., Hayes, A., Radford, A., et al.: Gpt-4o system card. arXiv preprint arXiv:2410.21276 (2024)

    Pith/arXiv arXiv 2024

  21. [21]

    In: CVPR (2023)

    Jeong, H., Park, G.Y., Ye, J.C.: Vmc: Video motion customization using temporal attention adaption for text-to-video diffusion models. In: CVPR (2023)

    2023

  22. [22]

    In: CVPR

    Kang, S., Kim, J., Kim, J., Hwang, S.J.: Your large vision-language model only needs a few attention heads for visual grounding. In: CVPR. pp. 9339–9350 (2025)

    2025

  23. [23]

    In: ICCV

    Khachatryan, L., Movsisyan, A., Tadevosyan, V., Henschel, R., Wang, Z., Navasardyan, S., Shi, H.: Text2video-zero: Text-to-image diffusion models are zero- shot video generators. In: ICCV. pp. 15954–15964 (2023)

    2023

  24. [24]

    arXiv preprint arXiv:2509.18096 (2025)

    Kim, C., Shin, H., Hong, E., Yoon, H., Arnab, A., Seo, P.H., Hong, S., Kim, S.: Seg4diff: Unveiling open-vocabulary segmentation in text-to-image diffusion transformers. arXiv preprint arXiv:2509.18096 (2025)

    arXiv 2025

  25. [25]

    arXiv preprint arXiv:2412.03603 (2024)

    Kong, W., Tian, Q., Zhang, Z., Min, R., Dai, Z., Zhou, J., Xiong, J., Li, X., Wu, B., Zhang, J., et al.: Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603 (2024)

    Pith/arXiv arXiv 2024

  26. [26]

    Li, B., Zhang, Y., Wang, Q., Ma, L., Shi, X., Wang, X., Wan, P., Yin, Z., Zhuge, Y., Lu, H., Jia, X.: Vfxmaster: Unlocking dynamic visual effect generation via in-context learning (2025),https://arxiv.org/abs/2510.25772

    arXiv 2025

  27. [27]

    arXiv preprint arXiv:2406.05338 (2024)

    Ling, P., Bu, J., Zhang, P., Dong, X., Zang, Y., Wu, T., Chen, H., Wang, J., Jin, Y.: Motionclone: Training-free motion cloning for controllable video generation. arXiv preprint arXiv:2406.05338 (2024)

    Pith/arXiv arXiv 2024

  28. [28]

    arXiv preprint arXiv:2504.12027 (2025)

    Liu, B., Wang, C., Su, T., Ten, H., Huang, J., Guo, K., Jia, K.: Understanding attention mechanism in video diffusion models. arXiv preprint arXiv:2504.12027 (2025)

    Pith/arXiv arXiv 2025

  29. [29]

    arXiv preprint arXiv:2402.17177 (2024)

    Liu, Y., Zhang, K., Li, Y., Yan, Z., Gao, C., Chen, R., Yuan, Z., Huang, Y., Sun, H., Gao, J., et al.: Sora: A review on background, technology, limitations, and opportunities of large vision models. arXiv preprint arXiv:2402.17177 (2024)

    Pith/arXiv arXiv 2024

  30. [30]

    arXiv preprint arXiv:2507.16869 (2025)

    Ma, Y., Feng, K., Hu, Z., Wang, X., Wang, Y., Zheng, M., He, X., Zhu, C., Liu, H., He, Y., et al.: Controllable video generation: A survey. arXiv preprint arXiv:2507.16869 (2025)

    arXiv 2025

  31. [31]

    arXiv preprint arXiv:2506.17220 (2025)

    Nam, J., Son, S., Chung, D., Kim, J., Jin, S., Hur, J., Kim, S.: Emer- gent temporal correspondences from video diffusion transformers. arXiv preprint arXiv:2506.17220 (2025)

    Pith/arXiv arXiv 2025

  32. [32]

    arXiv preprint arXiv:2303.06457 (2023)

    Pardyl, A., Rypeść, G., Kurzejamski, G., Zieliński, B., Trzciński, T.: Active vi- sual exploration based on attention-map entropy. arXiv preprint arXiv:2303.06457 (2023)

    Pith/arXiv arXiv 2023

  33. [33]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Park, G.Y., Jeong, H., Lee, S.W., Ye, J.C.: Spectral motion alignment for video motion transfer using diffusion models. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 39, pp. 6398–6405 (2025) Controlling Motion Transfer 37

    2025

  34. [34]

    Machine Learning 113(9), 6945–6975 (2024)

    Peruzzo, E., Sangineto, E., Liu, Y., De Nadai, M., Bi, W., Lepri, B., Sebe, N.: Spatial entropy as an inductive bias for vision transformers. Machine Learning 113(9), 6945–6975 (2024)

    2024

  35. [35]

    In: CVPR

    Pondaven, A., Siarohin, A., Tulyakov, S., Torr, P., Pizzati, F.: Video motion trans- fer with diffusion transformers. In: CVPR. pp. 22911–22921 (2025)

    2025

  36. [36]

    In: ICCV

    Shi, Q., Wu, J., Bai, J., Zhang, J., Qi, L., Tong, Y., Li, X.: Decouple and track: Benchmarking and improving video diffusion transformers for motion transfer. In: ICCV. pp. 10995–11005 (2025)

    2025

  37. [37]

    In: CVPR

    Siarohin, A., Lathuilière, S., Tulyakov, S., Ricci, E., Sebe, N.: Animating arbitrary objects via deep motion transfer. In: CVPR. pp. 2377–2386 (2019)

    2019

  38. [38]

    NeurIPS36, 1363–1389 (2023)

    Tang, L., Jia, M., Wang, Q., Phoo, C.P., Hariharan, B.: Emergent correspondence from image diffusion. NeurIPS36, 1363–1389 (2023)

    2023

  39. [39]

    In: ECCV

    Teed, Z., Deng, J.: Raft: Recurrent all-pairs field transforms for optical flow. In: ECCV. pp. 402–419. Springer (2020)

    2020

  40. [40]

    arXiv preprint arXiv:2503.20314 (2025)

    Wan, T., Wang, A., Ai, B., Wen, B., Mao, C., Xie, C.W., Chen, D., Yu, F., Zhao, H., Yang, J., Zeng, J., Wang, J., Zhang, J., Zhou, J., Wang, J., Chen, J., Zhu, K., Zhao, K., Yan, K., Huang, L., Feng, M., Zhang, N., Li, P., Wu, P., Chu, R., Feng, R., Zhang, S., Sun, S., Fang, T., Wang, T., Gui, T., Weng, T., Shen, T., Lin, W., Wang, W., Wang, W., Zhou, W.,...

    Pith/arXiv arXiv 2025

  41. [41]

    NeurIPS37, 96541– 96565 (2024)

    Wang, J., Ma, Y., Guo, J., Xiao, Y., Huang, G., Li, X.: Cove: Unleashing the diffusion feature correspondence for consistent video editing. NeurIPS37, 96541– 96565 (2024)

    2024

  42. [42]

    In: Proceedings of the Spe- cial Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers

    Wang, L., Mai, Z., Shen, G., Liang, Y., Tao, X., Wan, P., Zhang, D., Li, Y., Chen, Y.C.: Motion inversion for video customization. In: Proceedings of the Spe- cial Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers. pp. 1–12 (2025)

    2025

  43. [43]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wang, Y., Wang, L., Ma, Z., Hu, Q., Xu, K., Guo, Y.: Videodirector: Precise video editing via text-to-video models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 2589–2598 (2025)

    2025

  44. [44]

    arXiv preprint arXiv:2502.01776 (2025)

    Xi, H., Yang, S., Zhao, Y., Xu, C., Li, M., Li, X., Lin, Y., Cai, H., Zhang, J., Li, D., et al.: Sparse videogen: Accelerating video diffusion transformers with spatial- temporal sparsity. arXiv preprint arXiv:2502.01776 (2025)

    Pith/arXiv arXiv 2025

  45. [45]

    NeurIPS37, 76115–76138 (2024)

    Xiao, Z., Zhou, Y., Yang, S., Pan, X.: Video diffusion models are training-free motion interpreter and controller. NeurIPS37, 76115–76138 (2024)

    2024

  46. [46]

    IEEE Transactions on Visualization and Computer Graphics 31(2), 1526–1541 (2024)

    Xing, J., Xia, M., Liu, Y., Zhang, Y., Zhang, Y., He, Y., Liu, H., Chen, H., Cun, X., Wang, X., et al.: Make-your-video: Customized video generation using textual and structural guidance. IEEE Transactions on Visualization and Computer Graphics 31(2), 1526–1541 (2024)

    2024

  47. [47]

    In: ICLR (2025),https://openreview.net/forum?id=LQzN6TRFg9

    Yang, Z., Teng, J., Zheng, W., Ding, M., Huang, S., Xu, J., Yang, Y., Hong, W., Zhang, X., Feng, G., Yin, D., Yuxuan.Zhang, Wang, W., Cheng, Y., Xu, B., Gu, X., Dong, Y., Tang, J.: Cogvideox: Text-to-video diffusion models with an expert transformer. In: ICLR (2025),https://openreview.net/forum?id=LQzN6TRFg9

    2025

  48. [48]

    In: CVPR

    Yatim, D., Fridman, R., Bar-Tal, O., Kasten, Y., Dekel, T.: Space-time diffusion features for zero-shot text-driven motion transfer. In: CVPR. pp. 8466–8476 (2024)

    2024

  49. [49]

    In: The Eleventh International Conference on Learning Representations (2023) 38 S

    Zhang, H., Li, F., Liu, S., Zhang, L., Su, H., Zhu, J., Ni, L., Shum, H.Y.: Dino: Detr with improved denoising anchor boxes for end-to-end object detection. In: The Eleventh International Conference on Learning Representations (2023) 38 S. Jung et al

    2023

  50. [50]

    In: ECCV

    Zhao, R., Gu, Y., Wu, J.Z., Zhang, D.J., Liu, J.W., Wu, W., Keppo, J., Shou, M.Z.: Motiondirector: Motion customization of text-to-video diffusion models. In: ECCV. pp. 273–290. Springer (2024)

    2024