Pith. sign in

REVIEW 5 major objections 8 minor 66 references

Interactive Video Generation via Domain Adaptation

T0 review · 5 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that the quality loss in training-free, mask-guided video generation comes from two measurable domain shifts, and that both can be corrected at inference time so a frozen text-to-video diffusion model follows user-drawn…

desk verdict A plausible training-free fix for masked-attention drift in video diffusion, but the SOTA claim rests on tuning and baselines that need to be hardened before the numbers convince me. read the letter →

arxiv 2505.24253 v1 pith:DT2X2KIL submitted 2025-05-30 cs.CV cs.AIcs.MM

Ishaan Rawal , Suryansh Kumar This is my paper
classification cs.CVcs.AIcs.MM
keywords interactivevideogenerationtrajectorycontroltraining-freediffusionattentionmaskingdomainadaptationmasknormalizationintrinsicdenoisingmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tackles interactive video generation: steering a pretrained text-to-video diffusion model so that a specified subject follows a user-drawn bounding-box trajectory. It argues that existing training-free masking methods pay for control with perceptual quality because attention masking knocks activations out of their training distribution, and because the randomly sampled starting noise does not match what trajectory-conditioned inference expects. To fix the first, it proposes mask normalization, a pre-normalization layer that matches the distribution of masked attention outputs to unmasked ones using exact feature distribution matching. To fix the second, it proposes temporal intrinsic denoising, which refines the latent before each sampling step with a diffusion prior plus a correlation-based temporal gradient. The paper reports that together these yield better trajectory alignment and better perceptual scores than the two prior training-free baselines.

What carries the argument

Two inference-time modules carry the argument. Mask normalization is a pre-normalization layer that, for each masked attention head, replaces the masked output's values by the unmasked output's values with the same rank (exact feature distribution matching), aligning the masked distribution to the unmasked one and reducing the activation variance shift the paper measures. Temporal intrinsic denoising (TID) modifies the DDIM sampling loop: before each step it runs several intrinsic denoising updates on the current latent, where the update combines the model's own score (classifier-free guidance) with the gradient of a temporal consistency term, the average Pearson correlation between foreground crops across consecutive frames, evaluated on a Tweedie one-step clean estimate of the latent. The temporal prior is what injects the bounding-box trajectory into latent refinement, while the intrinsic denoising is what pulls the latent back toward the model's high-probability manifold.

What would settle it

Run the method with the sign of the temporal-prior gradient flipped on the same prompts and boxes. If the paper's mechanism is correct, the flipped sign should push the subject away from the trajectory and measurably lower per-frame mIoU; if the temporal gradient is not what drives control, mIoU would stay roughly unchanged.

Watch

Extended reading notes

Core claim

The central claim is that both failure modes of training-free interactive video generation are domain shifts, and both can be corrected at inference time without retraining. Masked attention changes the distribution of attention outputs, so the paper inserts a mask-normalization layer that rank-matches masked outputs to unmasked outputs per attention head before the residual connection. The initialization gap is handled by treating the current latent as a noisy image and running intrinsic denoising steps whose gradient includes a temporal prior: the Pearson correlation of foreground crops between consecutive frames, computed on a Tweedie-denoised one-step estimate of the clean latent. With these two modules, the paper reports that a frozen Zeroscope model produces videos that follow the boxes more accurately while keeping semantic similarity and perceptual quality at or above the baselines.

Load-bearing premise

The load-bearing premise is that the gradient of Pearson correlation between foreground crops is a faithful stand-in for the gradient of the true conditional likelihood; if this approximation fails, the guidance in Eq. (3) will pull whatever content sits in the boxes toward self-similarity instead of moving the subject along the trajectory.

Editorial extensions

If this is right

  • Training-free trajectory control no longer has to trade control for quality: the two modules attach to a frozen model and are active only during the early masked steps, shifting rather than accepting the trade-off.
  • The same frozen video diffusion backbone can be reused across different user trajectories without per-trajectory fine-tuning, LoRA adaptation, or a reference image.
  • Because mask normalization acts only on attention outputs, it applies to any attention-masked inference scheme, not just bounding-box video control.
  • Because TID uses a differentiable temporal objective, other user-specified constraints, such as size, speed, or direction, could be added as additional guidance terms in the same latent refinement loop.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The variance-shift measurement could serve as a cheap diagnostic: a practitioner could detect whether a proposed mask schedule will hurt quality before generating a full video by checking activation variance in the first few denoising steps.
  • The Pearson-correlation temporal prior assumes a single foreground subject whose appearance is roughly stable across frames; extending the method to multiple interacting subjects would likely require per-subject masks and a richer spatial prior.
  • The large reported gradient scale suggests the temporal gradient is much weaker than the score term, so a testable extension is adaptive normalization of only the temporal gradient, which the paper's own ablation suggests can help or hurt depending on the sequence.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. This paper proposes a training-free interactive video generation (IVG) method built on a frozen text-to-video diffusion model (Zeroscope). The method combines two components: mask normalization, which uses exact feature distribution matching (EFDM) to align masked attention outputs with unmasked counterparts and thereby reduce the variance shift caused by attention masking; and temporal intrinsic denoising (TID), which refines the latent before each sampling step by combining classifier-free guidance with a Pearson-correlation temporal prior computed over foreground crops. The authors report conditional metrics (CLIP-SIM, coverage, mIoU) and unconditional metrics (FID, KID, JeDi) on the 126 prompt–bounding-box pairs from Peekaboo, comparing against Peekaboo and Trailblazer, and conclude that the method achieves state-of-the-art performance among training-free IVG methods.

Significance. If the empirical claims were fully supported, the paper would make a useful contribution: it would demonstrate that a frozen video diffusion model can be steered by bounding boxes with less quality loss than Peekaboo and Trailblazer, and the mask-normalization idea is simple and could transfer to other masked-attention inference settings. The paper also provides a clear algorithm (Algorithm 1) and an interesting domain-adaptation framing for two known failure modes. However, the current evidence is not yet sufficient to establish the headline SOTA claim: the main hyperparameter is tuned on the evaluation set, the temporal-prior gradient is an unjustified proxy for a classifier log-likelihood, and the evaluation protocol filters metrics on detector success and uses an asymmetric FID/KID frame-count protocol.

major comments (5)
  1. [Section 4 and Supplementary B.2, Tables 3–4] The temporal-prior scale c_g is selected by comparing c_g ∈ {0, 2000, 5000, 7000, 10000} on the same 126-prompt benchmark that is used for the headline mIoU and FID/KID results. No validation split is held out, no seed variance is reported, and the other free parameters (γ, M, classifier-free guidance scale, frozen steps) are fixed without sensitivity analysis. The reported gains, especially the mIoU improvement from 25.97% to 33.82% on the static subset, may therefore be tuning artifacts rather than properties of the method. Please evaluate with c_g (and ideally all hyperparameters) fixed on a validation split before computing test metrics, and report error bars or significance tests.
  2. [Section 3.3.2, Eq. (3)] The derivation replaces the classifier log-likelihood gradient ∇_z log g_t(\tilde b | z^{m-1}_t | y, b) with the gradient of the Pearson-correlation metric τ(\tilde b, \hat z^{m-1}_{0,t}) without any justification that this correlation is a log-likelihood or that its gradient points toward the intended trajectory. As written, the gradient only encourages the foreground crops to be linearly correlated across consecutive frames, which can homogenize whatever content is inside the boxes rather than move the specified subject along the trajectory. This approximation is load-bearing because TID is one of the two core contributions; please provide a derivation or explicitly reframe the temporal prior as a heuristic and ablate it against a control (e.g., a random or fixed-direction gradient of the same magnitude).
  3. [Supplementary A.2 and Tables 1–2] CLIP-SIM and mIoU are computed only on the filtered subset of videos in which OWL-ViT detects the subject in at least 12 frames. Because coverage differs across methods (e.g., Tables 1 and 3 show Trailblazer static CoV 35 versus Peekaboo 40), the metrics on the filtered subsets are not comparable across methods: a method with low coverage can appear to have high mIoU on the small set of videos it happens to track. Please report metrics on the full set or use a fixed, pre-registered filtering rule, and report per-method sample sizes.
  4. [Section 4 and Tables 1–2] The experimental comparison includes only Peekaboo and Trailblazer, yet the abstract and Section 1 claim that the method 'outperforms all prior training-free IVG methods' and Section 2 names FreeTraj, Motion-Zero, and other training-free approaches. The claim is not supported by the evidence. Please add comparisons to the named methods or revise the claim to be specific to the compared baselines.
  5. [Supplementary A.2 and Table 2] FID and KID are computed with all 24 frames from each generated video but only 8 uniformly sampled frames from each real reference video, resized to 299×299. This asymmetry in frame counts and sampling can bias the unconditional quality estimate. In addition, Table 2 reports JeDi but then dismisses the low JeDi scores of the baselines as 'deceptively low' while the proposed method has the worst JeDi (1.86); this selective interpretation is not backed by an independent analysis. Please align the reference and generated frame protocols, and either report JeDi without post-hoc discounting or provide a criterion for when it is trustworthy.
minor comments (8)
  1. [Section 2] 'diffusion models have surpass GANs' should be 'diffusion models have surpassed GANs'.
  2. [Section 3.2] 'updates that remain ODD' should be 'updates that remain out-of-distribution (OOD)'.
  3. [Section 3.3.2] 'Morover' is a typo for 'Moreover'.
  4. [Section 4.1] 'for for per-frame quality evaluation' has a duplicated 'for', and 'without loosing semantic quality' should be 'without losing semantic quality'.
  5. [Supplementary B.2] 'intrisnic' should be 'intrinsic' in the table captions, and the KID entry '2.88±0.09' in Table 4 is missing the percent sign used elsewhere.
  6. [Section 3.3.2] The notation \hat z^{m-1}_{0,t} for the Tweedie estimate and the conditional expression log g_t(z^{m-1}_t | y,b | \tilde b) are both confusing; please define the estimate and rewrite the conditioning in standard notation.
  7. [Figure 2] The comparison labeled 'our method' includes both mask normalization and temporal intrinsic denoising, so the plot does not isolate the effect of mask normalization; a mask-normalization-only curve would support the stated conclusion.
  8. [References] References [53] and [54] are the same FreeInit paper; please merge them to avoid duplicate citations.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the central quality and control claims are empirical comparisons against external metrics.

full rationale

The paper's headline claims are empirical: mask normalization and temporal intrinsic denoising are evaluated with FID, KID, JeDi, CLIP-SIM, Coverage, and mIoU on a fixed benchmark, against Peekaboo and Trailblazer with the same Zeroscope backbone. None of these metrics is computed from the method's own inputs by construction. Mask normalization is a test-time distribution-matching operation using the frozen model's unmasked attention outputs as the reference; the variance-reduction result in Fig. 2 is largely a property of the EFDM construction rather than an independent prediction, but the perceptual-quality and trajectory-control claims do not rest on that tautology. Temporal intrinsic denoising adds a hand-defined Pearson-correlation gradient to the diffusion update; optimizing that gradient improves its own objective by design, yet the reported FID/KID and mIoU improvements are not forced by the update rule. The c_g=10000 hyperparameter is selected via ablations on the same 126-prompt set used for the headline tables, which is a benchmark-tuning concern that could inflate gains, but it is not a circular reduction because the final metrics are not algebraically determined by that choice. The claim of outperforming all prior training-free IVG methods is under-supported since only Peekaboo and Trailblazer are compared, but that is a completeness and statistical-validity issue, not a circularity. There is no load-bearing self-citation chain and no imported uniqueness theorem; the derivation is self-contained relative to external benchmarks.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central method rests on a small set of hand-set hyperparameters and three modeling assumptions: that masking causes a fixable distribution shift, that a correlation score can stand in for a classifier log-likelihood, and that one backbone plus one 126-prompt benchmark is enough to establish general improvement. No new physical or conceptual entity is posited.

free parameters (5)
  • gamma = 0.05
    Step-size multiplier for intrinsic denoising updates in Eq. (2); fixed by hand for all experiments.
  • M = 2
    Number of intrinsic denoising refinement iterations before each DDIM step; chosen without a stated tuning procedure.
  • c_g = 10000
    Scale for the temporal-prior gradient in Eq. (3); selected by ablation on the same 126-prompt evaluation set (Supp. Table 3).
  • classifier-free guidance scale = 9
    Guidance scale used for all generations; hand-set following common practice.
  • frozen steps = 4
    Number of early denoising steps where attention masks and mask normalization are active; hand-set.
assumptions (4)
  • domain assumption Masked attention outputs in a pre-trained video diffusion model are out-of-distribution, and this distribution shift is the cause of perceptual degradation.
    Motivates mask normalization; supported only by a variance analysis on Zeroscope (Section 3.2), not across models.
  • ad hoc to paper The Pearson-correlation temporal prior tau can replace the classifier log-likelihood log g_t in the intrinsic denoising update.
    Section 3.3.2 states the substitution without a derivation that tau is a log-probability or that its gradient has the correct scale.
  • standard math Tweedie's one-shot denoising estimate is accurate enough to compute temporal gradients at every intrinsic step.
    Uses a standard result (Eq. 3), but the regime of validity for video latents at high noise levels is not examined.
  • domain assumption Results on Zeroscope with the 126-prompt Peekaboo benchmark generalize to other video diffusion models.
    The paper claims applicability to any video diffusion model, but experiments use a single backbone and a single benchmark; Section 5 acknowledges the single-subject, static-camera limitation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Interactive Video Generation via Domain Adaptation." pith.science (2026). https://pith.science/paper/DT2X2KIL

@misc{pith2026250524253,
  author       = {Pith},
  title        = {Pith review of: Interactive Video Generation via Domain Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DT2X2KIL}},
  note         = {Machine review of arXiv:2505.24253}
}
read the original abstract

Text-conditioned diffusion models have emerged as powerful tools for high-quality video generation. However, enabling Interactive Video Generation (IVG), where users control motion elements such as object trajectory, remains challenging. Recent training-free approaches introduce attention masking to guide trajectory, but this often degrades perceptual quality. We identify two key failure modes in these methods, both of which we interpret as domain shift problems, and propose solutions inspired by domain adaptation. First, we attribute the perceptual degradation to internal covariate shift induced by attention masking, as pretrained models are not trained to handle masked attention. To address this, we propose mask normalization, a pre-normalization layer designed to mitigate this shift via distribution matching. Second, we address initialization gap, where the randomly sampled initial noise does not align with IVG conditioning, by introducing a temporal intrinsic diffusion prior that enforces spatio-temporal consistency at each denoising step. Extensive qualitative and quantitative evaluations demonstrate that mask normalization and temporal intrinsic denoising improve both perceptual quality and trajectory control over the existing state-of-the-art IVG techniques.

Figures

Figures reproduced from arXiv: 2505.24253 by the authors.

Figure 1
Figure 1. The task is to generate videos conditioned on a text prompt and a bounding box trajectory mask. Each row shows one instance: the gray panel on the left visualizes the bounding box trajectory (darker boxes appear later in time; arrows indicate direction for clarity), and the following frames show the generated video. The text prompt is shown below each video, with the subject in bold. Our model generates coherent, tr… view at source ↗
Figure 2
Figure 2. Effect of attention masking on activation variance. The attention layers in the first 4 diffusion steps are masked for trajectory control (indicated in gray). No masking is applied for the baseline configuration. (a) The masked variant (blue) exhibits internal variance shift relative to the baseline (green), whereas our method (orange) results in more aligned variance distribution, even in presence of masking. (b) T… view at source ↗
Figure 3
Figure 3. Mask Normalization aligns the empirical cumulative distributions of masked and unmasked attention outputs using EFDM (Q, K, V are query, key, value tensors of attention). For simplicity, we assume that Am and Au are N × 1- dimensional vectors. Exact Feature Distribution Matching (EFDM) [62], based on Exact Histogram Matching [9], is a popular method in style transfer for matching the empirical Cumulative Distributio… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison and ablations. Peekaboo (top row) exhibits prominent artifacts in both static (left panel) and dynamic bounding box (right panel) settings. Trailblazer (2nd row) struggles with realism and often ignores the spatial constraint. Mask normalization …
Figure 5
Figure 5. Figure 5: Additional Results: Additional examples of trajectory control in videos using mask normalization and temporal intrinsic denoising [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Additional Qualitative Comparison We compare our method (mask normalization with temporal intrinsic denoising; bottom row) with Peekaboo (top row) and Trailblazer (middle row). We find that while Trailblazer and Peekaboo either fail to generate subject within the bound…
Figure 7
Figure 7. Figure 7: Ablation: effect of cg We compare the effect of cg in temporal intrinsic denoising on the generated videos, for values 10000, 7000, 5000, 2000 and 0 (corresponds to intrinsic denoising), with values decreasing along the column. Lower cg values fail to enforce spatio-te…
Figure 8
Figure 8. Figure 8: Effect of gradient normalization: We compare two variants of temporal intrinsic denoising: with gradient normalization (“norm”, bottom row) and without normalization (“unnorm”, top row). While “norm” yields improved results in some cases (right panel), it frequently in…
Figure 9
Figure 9. Figure 9: Failure Cases: Top: The model fails to follow the user-guided trajectory. Middle: The model generates extra subjects in addition to the correct one following the trajectory. Bottom: The background appears unrealistic or inconsistent. B.3 Effect of gradient normalizatio…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

66 extracted references · 48 canonical work pages

  1. [1]

    Agarwal, S

    A. Agarwal, S. Karanam, K. Joseph, A. Saxena, K. Goswami, and B. V . Srinivasan. A-star: Test-time attention segregation and retention for text-to-image synthesis. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 2283–2293, 2023

    work page 2023

  2. [2]

    Bansal, H.-M

    A. Bansal, H.-M. Chu, A. Schwarzschild, R. Sengupta, M. Goldblum, J. Geiping, and T. Goldstein. Universal guidance for diffusion models. InICLR, 2024. URL https: //openreview.net/forum?id=pzpWBbnwiJ

    work page 2024

  3. [3]

    Bi´nkowski, D

    M. Bi´nkowski, D. J. Sutherland, M. Arbel, and A. Gretton. Demystifying MMD GANs. In International Conference on Learning Representations, 2018. URL https://openreview. net/forum?id=r1lUOzWCW

    work page 2018

  4. [4]

    Brooks, B

    T. Brooks, B. Peebles, C. Holmes, W. DePue, Y . Guo, L. Jing, D. Schnurr, J. Taylor, T. Luhman, E. Luhman, et al. Video generation models as world simulators.OpenAI Blog, 1:8, 2024

    work page 2024

  5. [5]

    M. Cao, X. Wang, Z. Qi, Y . Shan, X. Qie, and Y . Zheng. Masactrl: Tuning-free mutual self- attention control for consistent image synthesis and editing. InProceedings of the IEEE/CVF international conference on computer vision, pages 22560–22570, 2023

    work page 2023

  6. [6]

    C. Chen, J. Shu, L. Chen, G. He, C. Wang, and Y . Li. Motion-zero: Zero-shot moving object control framework for diffusion-based video generation.arXiv preprint arXiv:2401.10150, 2024

    arXiv 2024

  7. [7]

    T.-S. Chen, C. H. Lin, H.-Y . Tseng, T.-Y . Lin, and M.-H. Yang. Motion-conditioned diffusion model for controllable video synthesis.arXiv preprint arXiv:2304.14404, 2023

    arXiv 2023

  8. [8]

    W. Chen, Y . Ji, J. Wu, H. Wu, P. Xie, J. Li, X. Xia, X. Xiao, and L. Lin. Control-a-video: Controllable text-to-video generation with diffusion models.arXiv preprint arXiv:2305.13840, 2023

    arXiv 2023

Show all 66 references
  1. [9]

    Coltuc, P

    D. Coltuc, P. Bolon, and J.-M. Chassery. Exact histogram specification.IEEE TIP, 15(5): 1143–1152, 2006

    2006

  2. [10]

    Dhariwal and A

    P. Dhariwal and A. Q. Nichol. Diffusion models beat GANs on image synthesis. In A. Beygelzimer, Y . Dauphin, P. Liang, and J. W. Vaughan, editors,NeurIPS, 2021. URL https://openreview.net/forum?id=AAWuCvzaVt

    2021

  3. [11]

    B. Efron. Tweedie’s formula and selection bias.J. Am. Stat. Assoc., 106(496):1602–1614, 2011

    2011

  4. [12]

    Epstein, A

    D. Epstein, A. Jabri, B. Poole, A. Efros, and A. Holynski. Diffusion self-guidance for control- lable image generation.Advances in Neural Information Processing Systems, 36:16222–16239, 2023

    2023

  5. [13]

    Esser, J

    P. Esser, J. Chiu, P. Atighehchian, J. Granskog, and A. Germanidis. Structure and content- guided video synthesis with diffusion models. InProceedings of the IEEE/CVF international conference on computer vision, pages 7346–7356, 2023

    2023

  6. [14]

    Geng and A

    D. Geng and A. Owens. Motion guidance: Diffusion-based image editing with differentiable motion estimators. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  7. [15]

    Goodfellow, J

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio. Generative adversarial nets.Advances in neural information processing systems, 27, 2014

    2014

  8. [16]

    Heusel, H

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    2017

  9. [17]

    J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633–8646, 2022

    2022

  10. [18]

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021. 10

    2021 arXiv

  11. [19]

    Y . Hu, Z. Chen, and C. Luo. Lamd: Latent motion diffusion for video generation.arXiv preprint arXiv:2304.11603, 2023

    2023 arXiv

  12. [20]

    Huang, Y

    H. Huang, Y . Feng, C. Shi, L. Xu, J. Yu, and S. Yang. Free-bloom: Zero-shot text-to-video generator with llm director and ldm animator.Advances in Neural Information Processing Systems, 36:26135–26158, 2023

    2023

  13. [21]

    Y . Jain, A. Nasery, V . Vineet, and H. Behl. Peekaboo: Interactive video generation via masked- diffusion. InCVPR, pages 8079–8088, 2024

    2024

  14. [22]

    Jeong, J

    H. Jeong, J. Chang, G. Y . Park, and J. C. Ye. Dreammotion: Space-time self-similar score distillation for zero-shot video editing. InEuropean Conference on Computer Vision, pages 358–376. Springer, 2024

    2024

  15. [23]

    Jeong, G

    H. Jeong, G. Y . Park, and J. C. Ye. Vmc: Video motion customization using temporal attention adaption for text-to-video diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9212–9221, 2024

    2024

  16. [24]

    Khachatryan, A

    L. Khachatryan, A. Movsisyan, V . Tadevosyan, R. Henschel, Z. Wang, S. Navasardyan, and H. Shi. Text2video-zero: Text-to-image diffusion models are zero-shot video generators. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15954– 15964, 2023

    2023

  17. [25]

    M. Li, B. Wan, M.-F. Moens, and T. Tuytelaars. Animate your motion: Turning still images into dynamic videos. InEuropean Conference on Computer Vision, pages 409–425. Springer, 2024

    2024

  18. [26]

    X. Li, Z. Lai, L. Xu, Y . Qu, L. Cao, S. Zhang, B. Dai, and R. Ji. Director3d: Real-world camera trajectory and 3d scene generation from text.Advances in Neural Information Processing Systems, 37:75125–75151, 2025

    2025

  19. [27]

    Y . Li, H. Liu, Q. Wu, F. Mu, J. Yang, J. Gao, C. Li, and Y . J. Lee. Gligen: Open-set grounded text-to-image generation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22511–22521, 2023

    2023

  20. [28]

    S. Lin, B. Liu, J. Li, and X. Yang. Common diffusion noise schedules and sample steps are flawed. InProceedings of the IEEE/CVF winter conference on applications of computer vision, pages 5404–5411, 2024

    2024

  21. [29]

    S. Liu, Y . Zhang, W. Li, Z. Lin, and J. Jia. Video-p2p: Video editing with cross-attention control. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8599–8608, 2024

    2024

  22. [30]

    Lugmayr, M

    A. Lugmayr, M. Danelljan, A. Romero, F. Yu, R. Timofte, and L. Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11461–11471, 2022

    2022

  23. [31]

    G. Y . Luo, G. M. Favero, Z. Luo, A. Jolicoeur-Martineau, and C. Pal. Beyond FVD: An enhanced evaluation metrics for video generation distribution quality. InThe Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum? id=cC3LxGZasH

    2025

  24. [32]

    W.-D. K. Ma, J. P. Lewis, and W. B. Kleijn. Trailblazer: Trajectory control for diffusion-based video generation. InSIGGRAPH Asia, pages 1–11, 2024

    2024

  25. [33]

    Minderer, A

    M. Minderer, A. Gritsenko, A. Stone, M. Neumann, D. Weissenborn, A. Dosovitskiy, A. Ma- hendran, A. Arnab, M. Dehghani, Z. Shen, et al. Simple open-vocabulary object detection. In European conference on computer vision, pages 728–755. Springer, 2022

    2022

  26. [34]

    G. Monge. Mémoire sur la théorie des déblais et des remblais.Mem. Math. Phys. Acad. Royale Sci., pages 666–704, 1781

  27. [35]

    X. L. Ng, K. E. Ong, Q. Zheng, Y . Ni, S. Y . Yeo, and J. Liu. Animal kingdom: A large and diverse dataset for animal behavior understanding. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19023–19034, 2022. 11

    2022

  28. [36]

    Torch- Metrics - Measuring Reproducibility in PyTorch, Feb

    Nicki Skafte Detlefsen, Jiri Borovec, Justus Schock, Ananya Harsh, Teddy Koker, Luca Di Liello, Daniel Stancl, Changsheng Quan, Maxim Grechkin, and William Falcon. Torch- Metrics - Measuring Reproducibility in PyTorch, Feb. 2022. URL https://github.com/ Lightning-AI/torchmetrics

    2022

  29. [37]

    M. Niu, X. Cun, X. Wang, Y . Zhang, Y . Shan, and Y . Zheng. Mofa-video: Controllable image animation via generative motion field adaptions in frozen image-to-video diffusion model. In European Conference on Computer Vision, pages 111–128. Springer, 2024

    2024

  30. [38]

    Phung, S

    Q. Phung, S. Ge, and J.-B. Huang. Grounded text-to-image synthesis with attention refocusing. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7932–7942, 2024

    2024

  31. [39]

    H. Qiu, Z. Chen, Z. Wang, Y . He, M. Xia, and Z. Liu. Freetraj: Tuning-free trajectory control in video diffusion models.arXiv preprint arXiv:2406.16863, 2024

    2024 arXiv

  32. [40]

    H. Qiu, M. Xia, Y . Zhang, Y . He, X. Wang, Y . Shan, and Z. Liu. Freenoise: Tuning-free longer video diffusion via noise rescheduling. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  33. [41]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pages 8748–8763. PMLR, 2021

    2021

  34. [42]

    Y . Ren, Y . Zhou, J. Yang, J. Shi, D. Liu, F. Liu, M. Kwon, and A. Shrivastava. Customize-a-video: One-shot motion customization of text-to-video diffusion models. InEuropean Conference on Computer Vision, pages 332–349. Springer, 2024

    2024

  35. [43]

    J. P. Rolland, V . V o, B. Bloss, and C. K. Abbey. Fast algorithms for histogram matching: Application to texture synthesis.Journal of Electronic Imaging, 9(1):39–45, 2000

    2000

  36. [44]

    Ronneberger, P

    O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. InMedical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, page...

    2015

  37. [45]

    Singer, A

    U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafni, et al. Make-a-video: Text-to-video generation without text-video data.arXiv preprint arXiv:2209.14792, 2022

    2022 arXiv

  38. [46]

    Ulyanov, A

    D. Ulyanov, A. Vedaldi, and V . Lempitsky. Deep image prior. InCVPR, pages 9446–9454, 2018

    2018

  39. [47]

    J. Wang, Y . Zhang, J. Zou, Y . Zeng, G. Wei, L. Yuan, and H. Li. Boximator: Generating rich and controllable motions for video synthesis. InForty-first International Conference on Machine Learning (ICML)

  40. [48]

    X. Wang, H. Yuan, S. Zhang, D. Chen, J. Wang, Y . Zhang, Y . Shen, D. Zhao, and J. Zhou. Videocomposer: Compositional video synthesis with motion controllability.Advances in Neural Information Processing Systems, 36:7594–7611, 2023

    2023

  41. [49]

    Z. Wang, Z. Yuan, X. Wang, Y . Li, T. Chen, M. Xia, P. Luo, and Y . Shan. Motionctrl: A unified and flexible motion controller for video generation. InACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024

    2024

  42. [50]

    Y . Wei, S. Zhang, Z. Qing, H. Yuan, Z. Liu, Y . Liu, Y . Zhang, J. Zhou, and H. Shan. Dreamvideo: Composing your dream videos with customized subject and motion. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6537–6549, 2024

    2024

  43. [51]

    J. Z. Wu, Y . Ge, X. Wang, S. W. Lei, Y . Gu, Y . Shi, W. Hsu, Y . Shan, X. Qie, and M. Z. Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7623–7633, 2023. 12

    2023

  44. [52]

    R. Wu, L. Chen, T. Yang, C. Guo, C. Li, and X. Zhang. Lamp: Learn a motion pattern for few-shot video generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7089–7098, 2024

    2024

  45. [53]

    T. Wu, C. Si, Y . Jiang, Z. Huang, and Z. Liu. Freeinit: Bridging initialization gap in video diffusion models. InECCV, pages 378–394. Springer, 2024

    2024

  46. [54]

    T. Wu, C. Si, Y . Jiang, Z. Huang, and Z. Liu. Freeinit: Bridging initialization gap in video diffusion models. InEuropean Conference on Computer Vision, pages 378–394. Springer, 2024

    2024

  47. [55]

    T. Wu, Y . Zhang, X. Wang, X. Zhou, G. Zheng, Z. Qi, Y . Shan, and X. Li. Customcrafter: Customized video generation with preserving motion and concept composition abilities.arXiv preprint arXiv:2408.13239, 2024

    2024 arXiv

  48. [56]

    J. Xiao, R. Feng, H. Zhang, Z. Liu, Z. Yang, Y . Zhu, X. Fu, K. Zhu, Y . Liu, and Z.-J. Zha. Dreamclean: Restoring clean image using deep diffusion prior. InICLR, 2024. URL https: //openreview.net/forum?id=6ALuy19mPa

    2024

  49. [57]

    Z. Xiao, Y . Zhou, S. Yang, and X. Pan. Video diffusion models are training-free motion interpreter and controller. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  50. [58]

    W. Yan, Y . Zhang, P. Abbeel, and A. Srinivas. Videogpt: Video generation using vq-vae and transformers.arXiv preprint arXiv:2104.10157, 2021

    2021 arXiv

  51. [59]

    S. Yang, L. Hou, H. Huang, C. Ma, P. Wan, D. Zhang, X. Chen, and J. Liao. Direct-a-video: Customized video generation with user-directed camera movement and object motion. In SIGGRAPH, pages 1–12, 2024

    2024

  52. [60]

    S. Yang, L. Hou, H. Huang, C. Ma, P. Wan, D. Zhang, X. Chen, and J. Liao. Direct-a-video: Customized video generation with user-directed camera movement and object motion. InACM SIGGRAPH 2024 Conference Papers, pages 1–12, 2024

    2024

  53. [61]

    Zhang, A

    L. Zhang, A. Rao, and M. Agrawala. Adding conditional control to text-to-image diffusion models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023

    2023

  54. [62]

    Zhang, M

    Y . Zhang, M. Li, R. Li, K. Jia, and L. Zhang. Exact feature distribution matching for arbitrary style transfer and domain generalization. InCVPR, pages 8035–8045, 2022

    2022

  55. [63]

    Zhang, Y

    Y . Zhang, Y . Wei, D. Jiang, X. Zhang, W. Zuo, and Q. Tian. Controlvideo: Training-free controllable text-to-video generation.arXiv preprint arXiv:2305.13077, 2023

    2023 arXiv

  56. [64]

    Zhang, J

    Z. Zhang, J. Liao, M. Li, Z. Dai, B. Qiu, S. Zhu, L. Qin, and W. Wang. Tora: Trajectory-oriented diffusion transformer for video generation.arXiv preprint arXiv:2407.21705, 2024

    2024 arXiv

  57. [65]

    R. Zhao, Y . Gu, J. Z. Wu, D. J. Zhang, J.-W. Liu, W. Wu, J. Keppo, and M. Z. Shou. Motiondi- rector: Motion customization of text-to-video diffusion models. InEuropean Conference on Computer Vision, pages 273–290. Springer, 2024

    2024

  58. [66]

    norm”, bottom row) and without normalization (“unnorm

    D. Zhou, W. Wang, H. Yan, W. Lv, Y . Zhu, and J. Feng. Magicvideo: Efficient video generation with latent diffusion models.arXiv preprint arXiv:2211.11018, 2022. 13 Supplementary Material: Interactive Video Generation via Domain Adaptation Abstract This supplementary material ...

    2022 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.