Pith. sign in

REVIEW 5 major objections 5 minor 72 references

MagicPrompt: Ultra-Lightweight Prompt Tuning for Video Generation

T0 review · 5 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MagicPrompt claims that a frozen billion-parameter video diffusion model can be steered to new tasks by training only attention-embedded soft prompts and tiny shift-bias vectors (under 1% of parameters), with a dual pixel/latent reward maki

desk verdict A serious PEFT recipe with impressive parameter efficiency, but the self-referential latent reward needs a hard look and the paper's own Section 5 undercuts the 'preserves pre-trained knowledge' framing. read the letter →

arxiv 2607.14595 v2 pith:A6MFLODX submitted 2026-07-16 cs.CV

Yinhan Zhang , Dingwei Tan , Xianghao Kong , Yue Ma , Yeying Jin , Anyi Rao This is my paper
classification cs.CV
keywords videodiffusionmodelsparameter-efficientfine-tuningsoftprompttuningattention-embeddedpromptsrewardoptimizationtext-to-videogenerationimage-to-videocontrol-to-video
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MagicPrompt tries to establish that the attention layers of a large video diffusion transformer are enough of an adaptation surface: learn a small number of soft prompt tokens, insert them into self- and cross-attention, add per-layer scale/bias terms, and leave the pretrained backbone completely frozen. The paper couples this with a reward scheme that combines pixel-space scores from decoded frames with a latent-space score that minimizes the distance between guided and unguided denoising latents, treating the classifier-free-guidance trajectory as the quality target. Across text-to-video, image-to-video, and control-to-video tasks, and across 1.3B, 5B, and 14B models, the paper reports quality competitive with full fine-tuning and low-rank adaptation while updating under 1% of parameters. The reason this matters is practical: if true, customizing billion-scale video models becomes a small-memory, small-data operation feasible on ordinary GPUs, without overwriting the pretrained distribution.

What carries the argument

Attention-Embedded Prompt Tuning: visual soft prompts are concatenated into the key/value sequences of self-attention, textual soft prompts are prepended to the text-embedding context in cross-attention, and learnable shift-bias vectors gamma and beta recalibrate attention outputs as X' = gamma*X + beta. Dual-Space Reward Feedback Optimization: final reward is a weighted sum of a pixel-space reward (Human Preference Score and Motion Preference Score on decoded frames) and a latent-space reward R_latent = 1 - MSE(z_cfg, z_uncond), computed by comparing the CFG-guided and unconditional latents at the same timestep. The soft prompts are the adaptation mechanism; the latent reward is the mechani

What would settle it

Compare the latent reward against human preference ratings on a fixed prompt set across timesteps: if R_latent is flat or negatively correlated at high-noise stages, the CFG-as-target premise fails. Alternatively, retrain with the CFG target replaced by a random shifted latent; if quality does not drop, the latent reward is not doing the claimed work.

Watch

Extended reading notes

Core claim

In the paper's own terms, the claim is that non-intrusive steering—learnable visual prompts prepended to self-attention key/value sequences, learnable textual prompts prepended to cross-attention text embeddings, and a learned shift/scale calibration of attention outputs—is sufficient to adapt a frozen video diffusion model to a new condition or domain. The second claim is that reward-based training can be made stable by mixing a pixel-space reward (perceptual human-preference and motion/optical-flow scores on decoded frames) with a latent-space reward defined as 1 minus the mean squared error between classifier-free-guidance (CFG) latents and unconditional latents at the same timestep. The

Load-bearing premise

The latent reward treats the model's own CFG-guided latents at every timestep as the quality target, so if those latents are unreliable under high noise or the MSE mostly measures guidance scale instead of semantic quality, the training signal loses its foundation and the stability claim collapses.

Editorial extensions

If this is right

  • Adaptation of a 14B video model can be done with roughly 0.1% trainable parameters, moving billion-scale customization toward single-GPU budgets.
  • Freezing the backbone avoids overwriting pretrained generation priors, which should curb catastrophic forgetting when training data is scarce.
  • Reward training no longer requires group sampling or a separately trained reward model for the latent component, lowering both data and compute overhead.
  • One soft-prompt recipe covers text-, image-, and control-conditioned generation, so a single tuning framework can serve diverse downstream tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not explore composition, but because only attention streams are modified, MagicPrompt-style prompts could likely be stacked with weight-based adapters to add new domains and new controls simultaneously.
  • The latent reward's dependence on classifier-free guidance suggests the same two-space reward should transfer to other CFG-based conditional diffusion models (e.g., image or audio); testing it there would show whether the stability claim is general or video-specific.
  • The Discussion's noted identity shift and shot-transition tendency imply the soft prompts change more than alignment—they reshape the model's generative priors; a testable safeguard would be regularizing the shift-bias norm to preserve subject identity while keeping temporal gains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces MagicPrompt, a parameter-efficient fine-tuning (PEFT) method for video diffusion models. It injects learnable soft prompts into the key/value projections of self- and cross-attention, plus a per-layer scale-and-shift 'shift bias', while freezing the backbone. Training uses a dual-space reward: a pixel-space reward (HPS/MPS) evaluated on decoded frames, and a latent-space reward defined as the negative MSE between the model's own CFG-guided and unconditional latents. Experiments on T2V, I2V, and Control2V with Wan models up to 14B claim competitive performance with less than 1% trainable parameters.

Significance. If the results hold, the parameter efficiency is practically significant, enabling adaptation of 14B video models on limited hardware. The paper includes cross-scale experiments and ablations, and the Discussion is unusually candid about failure modes. However, the central reward design is self-referential and may collapse the CFG gap, and the experimental reporting lacks error bars, hyperparameters, and code. These issues must be resolved before the claims are fully credible.

major comments (5)
  1. [§3.3.2, Eq. (8)] The latent reward R_latent = 1 - MSE(z_cfg_t, z_uncond_t) is self-referential: both latents are outputs of the same prompt-conditioned model at the same timestep, so minimizing their MSE incentivizes the conditional and unconditional trajectories to converge, i.e., to collapse the CFG guidance gap, rather than to 'reach CFG-level quality.' The CFG trajectory is not a fixed target; it shifts as the soft prompts are updated. The paper's own Discussion (§5) reports identity shifts in T2V and shot transitions in I2V, which are consistent with this degeneracy. Please provide evidence (e.g., monitoring the CFG gap during training, or a control with a normalized/non-collapsing objective) that the latent reward is not simply reducing conditioning.
  2. [§4.1, Eqs. (7) and (9)] The reward pipeline is underspecified: alpha in Eq. (7) and lambda_pixel/lambda_latent in Eq. (9) are not reported; the token counts L_vis and L_txt are given only as 64 in the experimental setup, without specifying the split between visual/textual prompts. No error bars or seed variance are provided for any table, and the project page does not contain code or configs. The central comparison therefore cannot be reproduced or assessed for statistical significance.
  3. [§4.2, Table 3] The 'Fine-Tuning' row for Wan2.1-1.3B in T2V (CLIP 0.660, LPIPS 0.654, FID 103.534, FVD 824.528) is numerically identical (within rounding) to the 'Pre-Trained' row in Table 1 (0.66, 0.69, 103.53, 824.53). This suggests a labeling or copy-paste error that casts doubt on the cross-scale comparison. Additionally, in Table 1 Control2V, Ours has FID 34.43, worse than Pre-Trained (30.35) and LoRA (26.09), but the text only mentions 'comparable CLIP and LPIPS scores' without acknowledging the FID regression.
  4. [§3.3.1] The pixel reward is described as computed at 'low-noise timesteps (t≥3T/4)'. With standard diffusion timestep indexing (t=T is pure noise, t=0 is clean), t≥3T/4 is the high-noise regime where visual semantics are not clear, contradicting the stated purpose of evaluating visual quality. This needs clarification or correction; it directly affects the pixel reward signal and the claimed stable optimization.
  5. [§5] The Discussion reports that soft prompts and shift bias cause subject-identity changes in T2V and a tendency to produce shot transitions in I2V instead of staying conditioned on the initial frame. These are not merely qualitative curiosities; they indicate the method can break conditioning. The paper does not quantify these effects or provide a mitigation, yet the abstract claims 'competitive performance' and 'stable reward optimization.' This is a load-bearing limitation that should be addressed with analyses or a caveat in the claims.
minor comments (5)
  1. [Abstract] The claim of 'less than 1% trainable parameters' is not backed by a table of exact parameter counts or a formula for how the percentage is computed; please provide exact numbers.
  2. [§3.2.2] The 'shift bias' as defined in Eq. (4) is an affine transform (scale and shift). Consider renaming to 'scale-and-shift' to avoid confusion with the conventional meaning of 'bias' as an additive term only.
  3. [Eq. (8)] Since MSE can be greater than 1, R_latent can be negative. Clarify whether the reward is clipped or normalized, and what range is expected.
  4. [Table 2] The text states that compared to 'No Reward' the full method 'greatly enhances the generation effect,' but the CLIP score is identical (0.92 vs. 0.92); the improvement is in FID/FVD. Please qualify the claim accordingly.
  5. [§4.1] The evaluation metric definitions list FID for 'image quality' and FVD for 'temporal dynamics', but the tables also report LPIPS and CLIP; the text should clarify which metrics are computed on which reference frames and how the test sets are constructed.

Circularity Check

1 steps flagged · score 4.0 of 10

Latent reward (Eq. 8) defines its quality target as the same model's own CFG output, making the claimed 'stable reward optimization' self-referential; the parameter-efficiency claim remains independently supported.

  1. self definitional [Section 3.3.2, Eq. (8)]
    "Rlatent = 1 − MSE(z_cfg_t, z_uncond_t) ... the CFG trajectory at timestep t serves as the target quality level we aim for the model to achieve."

    Both z_cfg_t and z_uncond_t are produced by the same frozen-backbone model with the same trainable soft prompts and shift bias at the same timestep. The 'target quality level' is therefore not an external or fixed standard; it is the current model's own conditional output. Maximizing R_latent reduces to minimizing the conditional/unconditional gap of the model itself, whose trivial optimum is for the text condition to have no effect. The claimed 'stable reward optimization' is thus a self-consistency objective renamed as reward feedback, rather than an optimization toward an independent quality target.

full rationale

The paper's primary parameter-efficiency and competitive-performance claims are supported by comparisons against external methods (LoRA, VACE, full fine-tuning) and do not depend on Eq. (8); those results would remain meaningful even if the latent reward were removed. No load-bearing self-citation or imported uniqueness theorem is used: the many self-citations appear in related work and do not justify the core derivation. However, the paper's second central claim, 'stable reward optimization', is anchored in Eq. (8), and that objective is self-referential by construction: the reward target is generated by the very model being optimized. Section 5 even reports behaviors consistent with this concern (identity shift, shot transitions), though that is a limitation rather than circularity itself. Overall, one source of partial circularity in a central technical claim; the efficiency claim stands independently, so the score is moderate rather than high.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on frozen-backbone sufficiency, the validity of HPS/MPS reward models, and the self-referential CFG-vs-unconditional latent reward. Balancing coefficients and token counts are tuned but not fully reported.

free parameters (4)
  • alpha (HPS/MPS balance in Eq. 7) = not reported
    Eq. 7 combines HPS and MPS with weight alpha; no value or tuning procedure is given.
  • lambda_pixel and lambda_latent (reward balance in Eq. 9) = not reported
    Eq. 9 requires weights for pixel and latent rewards; values are absent and affect the ablation conclusions.
  • Soft prompt token count (L_vis = L_txt) = 64
    The ablation in Table 4 selects 64 tokens; performance is non-monotonic across 4/8/32/64, so this is a tuned hyperparameter.
  • Per-model-size learning rates = 5e-3 (1.3B/5B), 1e-3 (14B)
    Learning rates are chosen separately for different model sizes; this is standard but still a tuned choice.
assumptions (4)
  • domain assumption Frozen backbone weights suffice; only soft prompts and shift bias are optimized (Eq. 1).
    Central premise of PEFT; if intermediate features drift, prompts may not preserve pretrained knowledge.
  • ad hoc to paper The CFG trajectory is a valid quality target at all timesteps (Section 3.3.2).
    R_latent = 1 - MSE(z_cfg, z_uncond) assumes CFG output is the desired target and that MSE similarity tracks quality; no independent evidence is provided.
  • domain assumption HPS/MPS reward models provide reliable perceptual and motion quality signals (Eq. 7).
    The method relies on external reward models' validity for video; this is not revalidated against human judgments here.
  • domain assumption OpenVid/OpenHumanVid splits and TikTok evaluation are representative (Section 4.1).
    Generalization claims rest on dataset choice; few-shot claims are only shown qualitatively.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MagicPrompt: Ultra-Lightweight Prompt Tuning for Video Generation." pith.science (2026). https://pith.science/paper/A6MFLODX

@misc{pith2026260714595,
  author       = {Pith},
  title        = {Pith review of: MagicPrompt: Ultra-Lightweight Prompt Tuning for Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A6MFLODX}},
  note         = {Machine review of arXiv:2607.14595}
}
read the original abstract

Large-scale video diffusion models deliver strong generation performance, but full fine-tuning for downstream tasks incurs prohibitive computational costs. Existing parameter-efficient fine-tuning (PEFT) methods have two critical flaws on billion-scale models: they still require substantial trainable parameters, and reward-based training suffers from noise-induced optimization instability in condition-guided tasks. We propose MagicPrompt, a lightweight framework that achieves extreme parameter efficiency and stable reward optimization. It first adopts Attention-Embedded Prompt Tuning, which steers generation via lightweight soft prompts with orders of magnitude fewer parameters while preserving pre-trained knowledge. It further introduces Dual-Space Reward Feedback Optimization, which uses self-supervised latent objectives to improve condition-guided reward training. Experiments show MagicPrompt reaches competitive performance with less than 1% trainable parameters and notably reduces training costs.

Figures

Figures reproduced from arXiv: 2607.14595 by the authors.

Figure 1
Figure 1. Overview of MagicPrompt. We introduce MagicPrompt, an efficient attention-embedded prompt tuning framework for video diffusion models. Across diverse model sizes, MagicPrompt achieves competitive generation quality compared to Full Fine-tuning and LoRA, while requiring the fewest trainable parameters. Our method demonstrates seamless adaptation to Text-to-Video (T2V), Image-to￾Video (I2V), and Control-to-Video (Cont… view at source ↗
Figure 2
Figure 2. Comparison of PEFT paradigms. (a) Adapter-based methods insert auxiliary modules in parallel alongside the backbone. (b) Representation-based methods (e.g., LoRA) approximate weight updates via low-rank decomposition. (c) Prompt-Tuning methods add trainable soft prompts to each layer and combine with the layer input. (d) Our method employs attention-embedded prompt tuning, injecting trainable prompts directly into a… view at source ↗
Figure 3
Figure 3. Framework overview of MagicPrompt. Visual soft prompt prefill with latent feature in the key and value in the self-attention module, and textual soft prompt combined with text embedding in the cross-attention module to reweight the attention distribution. To better align with human preference, the reward feedback optimization is guided by both pixel-space semantic reward and latent objectives. denoising trajectory a… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Overview of dual reward feedback optimization. To enhance generalization in reward training, MagicPrompt integrates regularization signals from two domains. Pixel Space: Perceptual rewards (HPS, MPS) evaluate decoded frames to ensure visual fidelity. Latent Space: Comp…
Figure 5
Figure 5. Figure 5: Qualitative comparison with baselines. Our method demonstrates superior temporal coherence and visual fidelity across T2V, I2V, and Control2V tasks. Unlike baselines exhibiting motion blur and structural distortion, our method preserves fine details and ensures consist…
Figure 6
Figure 6. Figure 6: New condition adaptation. The pre-trained model suffers from notable performance degradation on unseen new conditions, as it has not learned to model the novel condition prior. After tuning with our method, the model achieves substantial performance gains and effective…
Figure 7
Figure 7. Figure 7: Few-Shot result visualization. Under one-shot and few-shot training regimes, our method adaptively adjusts the de￾noising distribution and consistently achieves better generation quality. 4.4. Ablation Study To validate the design choices of MagicPrompt, we conduct abl…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

72 extracted references · 6 linked inside Pith

  1. [1]

    Wele Gedara Chaminda Bandara and Vishal M. Patel. Atten- tion Prompt Tuning: Parameter-efficient Adaptation of Pre- trained Models for Spatiotemporal Modeling, 2024. 2

    2024

  2. [2]

    Align your Latents: High-resolution Video Synthesis with Latent Diffusion Models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your Latents: High-resolution Video Synthesis with Latent Diffusion Models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22563–22575, 2023. 3

    2023

  3. [3]

    Adaptformer: Adapt- ing Vision Transformers for Scalable Visual Recognition

    Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo. Adaptformer: Adapt- ing Vision Transformers for Scalable Visual Recognition. Advances in Neural Information Processing Systems, 35: 16664–16678, 2022. 2

    2022

  4. [4]

    Con- textFlow: Training-Free Video Object Editing via Adap- tive Context Enrichment.arXiv preprint arXiv:2509.17818,

    Yiyang Chen, Xuanhua He, Xiujun Ma, and Yue Ma. Con- textFlow: Training-Free Video Object Editing via Adap- tive Context Enrichment.arXiv preprint arXiv:2509.17818,

    arXiv

  5. [5]

    Directly Fine-Tuning Diffusion Models on Differentiable Rewards

    Kevin Clark, Paul Vicol, Kevin Swersky, and David Fleet. Directly Fine-Tuning Diffusion Models on Differentiable Rewards. InInternational Conference on Learning Repre- sentations, pages 4793–4822, 2024. 6

    2024

  6. [6]

    Compact 3d gaussian splatting for Dense Visual SLAM.arXiv preprint arXiv:2403.11247, 2024

    Tianchen Deng, Yaohui Chen, Leyan Zhang, Jianfei Yang, Shenghai Yuan, Jiuming Liu, Danwei Wang, Hesheng Wang, and Weidong Chen. Compact 3d gaussian splatting for Dense Visual SLAM.arXiv preprint arXiv:2403.11247, 2024. 3

    arXiv 2024

  7. [7]

    GaussianDWM: 3D Gaussian Driving World Model for Unified Scene Understanding and Multi-Modal Generation

    Tianchen Deng, Xuefeng Chen, Yi Chen, Qu Chen, Yuyao Xu, Lijin Yang, Le Xu, Yu Zhang, Bo Zhang, Wuxiong Huang, and Hesheng Wang. GaussianDWM: 3D Gaussian Driving World Model for Unified Scene Understanding and Multi-Modal Generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10656–10667, 2026. 3

    2026

  8. [8]

    Dit4edit: Dif- fusion Transformer for Image Editing

    Kunyu Feng, Yue Ma, Bingyuan Wang, Chenyang Qi, Haozhe Chen, Qifeng Chen, and Zeyu Wang. Dit4edit: Dif- fusion Transformer for Image Editing. InProceedings of the AAAI Conference on Artificial Intelligence, pages 2969– 2977, 2025. 3

    2025

Show all 72 references
  1. [9]

    PAI-Studio: Cin- ematic Video Background Replacement with Camera-Aware Motion.arXiv preprint arXiv:2606.01399, 2026

    Heyuan Gao, Bangxun Tang, Yiren Song, Guian Fang, Zi- jian He, Jie Yang, and Mike Zheng Shou. PAI-Studio: Cin- ematic Video Background Replacement with Camera-Aware Motion.arXiv preprint arXiv:2606.01399, 2026. 3

    2026 arXiv

  2. [10]

    AnimateDiff: Animate Your Personalized Text-to- Image Diffusion Models without Specific Tuning, 2024

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. AnimateDiff: Animate Your Personalized Text-to- Image Diffusion Models without Specific Tuning, 2024. 3

    2024

  3. [11]

    Denoising Dif- fusion Probabilistic Models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Dif- fusion Probabilistic Models.Advances in neural information processing systems, 33:6840–6851, 2020. 3

    2020

  4. [12]

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video Dif- fusion Models, 2022. 3

    2022

  5. [13]

    Parameter-Efficient Transfer Learning for NLP

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-Efficient Transfer Learning for NLP. InInternational Conference on Machine Learning, pages 2790–2799. PMLR, 2019. 4

    2019

  6. [14]

    LoRA: Low-Rank Adaptation of Large Language Mod- els.Iclr, 1(2):3, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. LoRA: Low-Rank Adaptation of Large Language Mod- els.Iclr, 1(2):3, 2022. 2, 4

    2022

  7. [15]

    Learning High Fi- delity Depths of Dressed Humans by Watching Social Media Dance Videos

    Yasamin Jafarian and Hyun Soo Park. Learning High Fi- delity Depths of Dressed Humans by Watching Social Media Dance Videos. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12753–12762, 2021. 6

    2021

  8. [16]

    Vi- sual Prompt Tuning, 2022

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Vi- sual Prompt Tuning, 2022. 2

    2022

  9. [17]

    V ACE: All-in-one Video Creation and Editing

    Zeyinzi Jiang, Zhen Han, Chaojie Mao, Jingfeng Zhang, Yulin Pan, and Yu Liu. V ACE: All-in-one Video Creation and Editing. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 17191–17202, 2025. 4

    2025

  10. [18]

    Fact: Factor-tuning for Lightweight Adaptation on Vision Transformer

    Shibo Jie and Zhi-Hong Deng. Fact: Factor-tuning for Lightweight Adaptation on Vision Transformer. InProceed- ings of the AAAI Conference on Artificial Intelligence, pages 1060–1068, 2023. 2

    2023

  11. [19]

    Convolutional Bypasses are Better Vision Transformer Adapters

    Shibo Jie, Zhi-Hong Deng, Shixuan Chen, and Zhijuan Jin. Convolutional Bypasses are Better Vision Transformer Adapters. InECAI 2024: 27th European Conference on Ar- tificial Intelligence, 19–24 October 2024, Santiago de Com- postela, Spain–Including 13th Conference on Prestigi...

    2024

  12. [20]

    Openhumanvid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation

    Hui Li, Mingwang Xu, Yun Zhan, Shan Mu, Jiaye Li, Kaihui Cheng, Yuxuan Chen, Tan Chen, Mao Ye, Jingdong Wang, et al. Openhumanvid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation. InPro- ceedings of the Computer Vision and Pattern Recognition Co...

    2025

  13. [21]

    Prefix-tuning: Optimiz- ing Continuous Prompts for Generation

    Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimiz- ing Continuous Prompts for Generation. InProceedings of the 59th Annual Meeting of the Association for Computa- tional Linguistics and the 11th International Joint Confer- ence on Natural Language Processing (Volume 1: Lon...

    2021

  14. [22]

    Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning.Advances in Neural Information Processing Systems, 35:109–123, 2022

    Dongze Lian, Daquan Zhou, Jiashi Feng, and Xinchao Wang. Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning.Advances in Neural Information Processing Systems, 35:109–123, 2022. 2

    2022

  15. [23]

    A Survey on Cache Methods in Dif- fusion Models: Toward Efficient Multi-Modal Generation

    Jiacheng Liu, Xinyu Wang, Yuqi Lin, Zhikai Wang, Peiru Wang, Peiliang Cai, Qinming Zhou, Zhengan Yan, Zexuan Yan, Zhengyi Shi, et al. A Survey on Cache Methods in Dif- fusion Models: Toward Efficient Multi-Modal Generation. arXiv preprint arXiv:2510.19755, 2025. 2

    2025

  16. [24]

    Towards Ef- ficient Visual Adaption via Structural Re-parameterization,

    Gen Luo, Minglang Huang, Yiyi Zhou, Xiaoshuai Sun, Guannan Jiang, Zhiyu Wang, and Rongrong Ji. Towards Ef- ficient Visual Adaption via Structural Re-parameterization,

  17. [25]

    VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation, 2023

    Zhengxiong Luo, Dayou Chen, Yingya Zhang, Yan Huang, Liang Wang, Yujun Shen, Deli Zhao, Jingren Zhou, and Tie- niu Tan. VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation, 2023. 3

    2023

  18. [26]

    Latte: La- tent Diffusion Transformer for Video Generation, 2025

    Xin Ma, Yaohui Wang, Xinyuan Chen, Gengyun Jia, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: La- tent Diffusion Transformer for Video Generation, 2025. 3

    2025

  19. [27]

    Follow Your Pose: Pose- guided Text-to-video Generation Using Pose-free Videos

    Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow Your Pose: Pose- guided Text-to-video Generation Using Pose-free Videos. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 4117–4125, 2024. 2

    2024

  20. [28]

    Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation

    Yue Ma, Hongyu Liu, Hongfa Wang, Heng Pan, Yingqing He, Junkun Yuan, Ailing Zeng, Chengfei Cai, Heung-Yeung Shum, Wei Liu, et al. Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation. InSIGGRAPH Asia 2024 Conference Papers, pages 1–12, 2024. 3

    2024

  21. [29]

    Magicstick: Controllable Video Editing via Control Handle Transforma- tions

    Yue Ma, Xiaodong Cun, Sen Liang, Jinbo Xing, Yingqing He, Chenyang Qi, Siran Chen, and Qifeng Chen. Magicstick: Controllable Video Editing via Control Handle Transforma- tions. In2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 9385–9395. IEEE, 2025. 3

    2025

  22. [30]

    Controllable Video Generation: A Survey.arXiv preprint arXiv:2507.16869,

    Yue Ma, Kunyu Feng, Zhongyuan Hu, Xinyu Wang, Yucheng Wang, Mingzhe Zheng, Xuanhua He, Chenyang Zhu, Hongyu Liu, Yingqing He, et al. Controllable Video Generation: A Survey.arXiv preprint arXiv:2507.16869,

  23. [31]

    Follow-Your-Creation: Empowering 4D Creation through Video Inpainting.arXiv preprint arXiv:2506.04590, 2025

    Yue Ma, Kunyu Feng, Xinhua Zhang, Hongyu Liu, David Junhao Zhang, Jinbo Xing, Yinhan Zhang, Ayden Yang, Zeyu Wang, and Qifeng Chen. Follow-Your-Creation: Empowering 4D Creation through Video Inpainting.arXiv preprint arXiv:2506.04590, 2025. 2

    2025 arXiv

  24. [32]

    Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuning.arXiv preprint arXiv:2506.05207, 2025

    Yue Ma, Yulong Liu, Qiyuan Zhu, Ayden Yang, Kunyu Feng, Xinhua Zhang, Zhifeng Li, Sirui Han, Chenyang Qi, and Qifeng Chen. Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuning.arXiv preprint arXiv:2506.05207, 2025. 2

    2025

  25. [33]

    Follow-your-emoji-faster: To- wards Efficient, Fine-Controllable, and Expressive Freestyle Portrait Animation.arXiv preprint arXiv:2509.16630, 2025

    Yue Ma, Zexuan Yan, Hongyu Liu, Hongfa Wang, Heng Pan, Yingqing He, Junkun Yuan, Ailing Zeng, Chengfei Cai, Heung-Yeung Shum, et al. Follow-your-emoji-faster: To- wards Efficient, Fine-Controllable, and Expressive Freestyle Portrait Animation.arXiv preprint arXiv:2509.16630, 2025. 3

    2025

  26. [34]

    Group Editing: Edit Multiple Im- ages in One Go.arXiv preprint arXiv:2603.22883, 2026

    Yue Ma, Xinyu Wang, Qianli Ma, Qinghe Wang, Mingzhe Zheng, Xiangpeng Yang, Hao Li, Chongbo Zhao, Jixuan Ying, Harry Yang, et al. Group Editing: Edit Multiple Im- ages in One Go.arXiv preprint arXiv:2603.22883, 2026. 2

    2026

  27. [35]

    FastVMT: Eliminat- ing Redundancy in Video Motion Transfer.arXiv preprint arXiv:2602.05551, 2026

    Yue Ma, Zhikai Wang, Tianhao Ren, Mingzhe Zheng, Hongyu Liu, Jiayi Guo, Mark Fong, Yuxuan Xue, Zixi- ang Zhao, Konrad Schindler, et al. FastVMT: Eliminat- ing Redundancy in Video Motion Transfer.arXiv preprint arXiv:2602.05551, 2026. 2

    2026

  28. [36]

    OpenVid-1M: A Large-Scale High-quality Dataset for Text- to-video Generation

    Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhen- heng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. OpenVid-1M: A Large-Scale High-quality Dataset for Text- to-video Generation. InInternational Conference on Learn- ing Representations, pages 1045–1064, 2025. 6

    2025

  29. [37]

    ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning.Advances in Neural Information Process- ing Systems, 35:26462–26477, 2022

    Junting Pan, Ziyi Lin, Xiatian Zhu, Jing Shao, and Hong- sheng Li. ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning.Advances in Neural Information Process- ing Systems, 35:26462–26477, 2022. 2

    2022

  30. [38]

    Scalable Diffusion Models with Transformers

    William Peebles and Saining Xie. Scalable Diffusion Models with Transformers. InProceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4195–4205,

  31. [39]

    Aligning Text-to-Image Diffusion Mod- els with Reward Backpropagation, 2024

    Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, and Ka- terina Fragkiadaki. Aligning Text-to-Image Diffusion Mod- els with Reward Backpropagation, 2024. 6

    2024

  32. [40]

    Jensen, Zhenli Sheng, and Bin Yang

    Xiangfei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, Zhenli Sheng, and Bin Yang. TFB: Towards com- prehensive and fair benchmarking of time series forecasting methods. InProc. VLDB Endow., pages 2363–2377, 2024. 3

    2024

  33. [41]

    DBLoss: Decomposition-based Loss Function for Time Series Fore- casting

    Xiangfei Qiu, Xingjian Wu, Hanyin Cheng, Xvyuan Liu, Chenjuan Guo, Jilin Hu, and Bin Yang. DBLoss: Decomposition-based Loss Function for Time Series Fore- casting. InNeurIPS, 2025

    2025

  34. [42]

    DUET: Dual clustering enhanced mul- tivariate time series forecasting

    Xiangfei Qiu, Xingjian Wu, Yan Lin, Chenjuan Guo, Jilin Hu, and Bin Yang. DUET: Dual clustering enhanced mul- tivariate time series forecasting. InSIGKDD, pages 1185– 1196, 2025. 3

    2025

  35. [43]

    Learning transferable Visual Models from Natural Language Super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable Visual Models from Natural Language Super- vision. InInternational Conference on Machine Learning, p...

    2021

  36. [44]

    Dreambooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22500–...

    2023

  37. [45]

    pytorch-fid: FID Score for PyTorch

    Maximilian Seitzer. pytorch-fid: FID Score for PyTorch. https://github.com/mseitzer/pytorch-fid,

  38. [46]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junx- iao Song, Mingchuan Zhang, Y .K. Li, Y . Wu, and Daya Guo. DeepSeekMath: Pushing the Limits of Mathematical Rea- soning in Open Language Models, 2024. 2

    2024

  39. [47]

    Make-A-Video: Text-to-Video Generation without Text-Video Data, 2022

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taig- man. Make-A-Video: Text-to-Video Generation without Text-Video Data, 2022. 3

    2022

  40. [48]

    Visual Prompt Tuning for Generative Transfer Learning

    Kihyuk Sohn, Huiwen Chang, Jos ´e Lezama, Luisa Polania, Han Zhang, Yuan Hao, Irfan Essa, and Lu Jiang. Visual Prompt Tuning for Generative Transfer Learning. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19840–19851, 2023. 2

    2023

  41. [49]

    Denois- ing Diffusion Implicit Models, 2022

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing Diffusion Implicit Models, 2022. 3

    2022

  42. [50]

    Proces- sPainter: Learning to Draw from Sequence Data

    Yiren Song, Shijie Huang, Chen Yao, Hai Ci, Xiaojun Ye, Jiaming Liu, Yuxuan Zhang, and Mike Zheng Shou. Proces- sPainter: Learning to Draw from Sequence Data. InSIG- GRAPH Asia 2024 Conference Papers, pages 1–10, 2024. 3

    2024

  43. [51]

    StreamingEffect: Real-Time Human-Centric Video Effect Generation.arXiv preprint arXiv:2605.17019, 2026

    Yiren Song, Cheng Liu, Yuxin Jiang, and Mike Zheng Shou. StreamingEffect: Real-Time Human-Centric Video Effect Generation.arXiv preprint arXiv:2605.17019, 2026

    2026 arXiv

  44. [52]

    VISTA: Triplet-Supervised Video Style Transfer with Diffusion Transformers.arXiv preprint arXiv:2605.17312,

    Yiren Song, Wangzi Yao, Haofan Wang, and Mike Zheng Shou. VISTA: Triplet-Supervised Video Style Transfer with Diffusion Transformers.arXiv preprint arXiv:2605.17312,

    arXiv

  45. [53]

    Fedperfix: Towards Partial Model Personaliza- tion of Vision Transformers in Federated Learning

    Guangyu Sun, Matias Mendieta, Jun Luo, Shandong Wu, and Chen Chen. Fedperfix: Towards Partial Model Personaliza- tion of Vision Transformers in Federated Learning. InCon- ferenceProceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 4988–4998, 2023. 2

    2023

  46. [54]

    To- wards Accurate Generative Models of Video: A New Metric & Challenges, 2019

    Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. To- wards Accurate Generative Models of Video: A New Metric & Challenges, 2019. 6

    2019

  47. [55]

    Wan: Open and Advanced Large-Scale Video Generative Models, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianx- iao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jin- gren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, P...

    2025

  48. [56]

    Cove: Unleashing the Diffusion Fea- ture Correspondence for Consistent Video Editing.Advances in Neural Information Processing Systems, 37:96541–96565,

    Jiangshan Wang, Yue Ma, Jiayi Guo, Yicheng Xiao, Gao Huang, and Xiu Li. Cove: Unleashing the Diffusion Fea- ture Correspondence for Consistent Video Editing.Advances in Neural Information Processing Systems, 37:96541–96565,

  49. [57]

    Tam- ing Rectified Flow for Inversion and Editing.arXiv preprint arXiv:2411.04746, 2024

    Jiangshan Wang, Junfu Pu, Zhongang Qi, Jiayi Guo, Yue Ma, Nisha Huang, Yuxin Chen, Xiu Li, and Ying Shan. Tam- ing Rectified Flow for Inversion and Editing.arXiv preprint arXiv:2411.04746, 2024. 3

    2024 arXiv

  50. [58]

    DiffLoRA: Generating Person- alized Low-Rank Adaptation Weights with Diffusion, 2024

    Yujia Wu, Yiming Shi, Jiwei Wei, Chengwei Sun, Yang Yang, and Heng Tao Shen. DiffLoRA: Generating Person- alized Low-Rank Adaptation Weights with Diffusion, 2024. 4

    2024

  51. [59]

    DAPE: Dual-Stage Parameter-Efficient Fine-Tuning for Consistent Video Editing with Diffusion Models, 2025

    Junhao Xia, Chaoyang Zhang, Yecheng Zhang, Chengyang Zhou, Zhichang Wang, Bochun Liu, and Dongshuo Yin. DAPE: Dual-Stage Parameter-Efficient Fine-Tuning for Consistent Video Editing with Diffusion Models, 2025. 2

    2025

  52. [60]

    SimDA: Simple Diffusion Adapter for Efficient Video Generation

    Zhen Xing, Qi Dai, Han Hu, Zuxuan Wu, and Yu-Gang Jiang. SimDA: Simple Diffusion Adapter for Efficient Video Generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7827– 7839, 2024. 2, 4

    2024

  53. [61]

    DanceGRPO: Unleashing GRPO on Visual Generation, 2025

    Zeyue Xue, Jie Wu, Yu Gao, Fangyuan Kong, Lingting Zhu, Mengzhao Chen, Zhiheng Liu, Wei Liu, Qiushan Guo, Weilin Huang, and Ping Luo. DanceGRPO: Unleashing GRPO on Visual Generation, 2025. 2

    2025

  54. [62]

    CogvideoX: Text-to- Video Diffusion Models with an Expert Transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xi- aohan Zhang, Guanyu Feng, et al. CogvideoX: Text-to- Video Diffusion Models with an Expert Transformer. InIn- ternational Conference on Learning Representations, pages 83...

    2025

  55. [63]

    IP- Adapter: Text Compatible Image Prompt Adapter for Text- to-Image Diffusion Models, 2023

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. IP- Adapter: Text Compatible Image Prompt Adapter for Text- to-Image Diffusion Models, 2023. 2

    2023

  56. [64]

    Bruce X. B. Yu, Jianlong Chang, Lingbo Liu, Qi Tian, and Chang Wen Chen. Towards a Unified View on Visual Parameter-Efficient Transfer Learning, 2023. 2

    2023

  57. [65]

    Adding Conditional Control to Text-to-Image Diffusion Models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding Conditional Control to Text-to-Image Diffusion Models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 2

    2023

  58. [66]

    Efros, Eli Shecht- man, and Oliver Wang

    Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shecht- man, and Oliver Wang. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric, 2018. 6

    2018

  59. [67]

    ControlVideo: Training-free Con- trollable Text-to-Video Generation

    Yabo Zhang, Yuxiang Wei, XIAOPENG ZHANG, Wang- meng Zuo, Qi Tian, et al. ControlVideo: Training-free Con- trollable Text-to-Video Generation. InInternational Con- ference on Learning Representations, pages 54441–54461,

  60. [68]

    MagicColor: Multi-Instance Sketch Coloriza- tion

    Yinhan Zhang, Yue Ma, Bingyuan Wang, Qifeng Chen, and Zeyu Wang. MagicColor: Multi-Instance Sketch Coloriza- tion. InProceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 15205–15217, 2025. 3

    2025

  61. [69]

    In- stanceAnimator: Multi-Instance Sketch Video Colorization

    Yinhan Zhang, Yue Ma, Bingyuan Wang, Kunyu Feng, Yey- ing Jin, Qifeng Chen, Anyi Rao, and Zeyu Wang. In- stanceAnimator: Multi-Instance Sketch Video Colorization. arXiv preprint arXiv:2603.25357, 2026

    2026

  62. [70]

    Tea-Adapter: Teacher Adapter for Efficient Conditional Generation

    Yinhan Zhang, Yue Ma, Fangqiu Yi, Chenyang Qi, Chi Zhang, Kunyu Feng, and Zeyu Wang. Tea-Adapter: Teacher Adapter for Efficient Conditional Generation. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4805–4815, 2026. 3

    2026

  63. [71]

    Forecast then Calibrate: Feature Caching as ODE for Efficient Diffusion Transformers

    Shikang Zheng, Liang Feng, Xinyu Wang, Qinming Zhou, Peiliang Cai, Chang Zou, Jiacheng Liu, Yuqi Lin, Junjie Chen, Yue Ma, et al. Forecast then Calibrate: Feature Caching as ODE for Efficient Diffusion Transformers. In Proceedings of the AAAI Conference on Artificial Intelli- ...

    2026

  64. [209]

    Ltd 1 Oliver’s Yard, 55 City Road, London, EC1Y 1SP, 2024

    SAGE Publications Pvt. Ltd 1 Oliver’s Yard, 55 City Road, London, EC1Y 1SP, 2024. 2

    2024

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.