Pith. sign in

REVIEW 3 major objections 6 minor 54 references

Edit as You See: Image-guided Video Editing via Masked Motion Modeling

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read IVEDiff, the first image-guided video editing diffusion model, replaces masked regions with reference-image content while keeping the video temporally smooth.

desk verdict A sensible first image-guided video editing system with two novel components, but the temporal-smoothness claim rests on a partly circular metric; still worth refereeing. read the letter →

arxiv 2501.04325 v1 pith:EFLTXRNP submitted 2025-01-08 cs.CV

Zhi-Lin Huang , Yixuan Liu , Chujun Qin , Zhongdao Wang , Dong Zhou , Dong Li , Emad Barsoum This is my paper · ORCID
classification cs.CV
keywords image-guidedvideoeditingdiffusionmodelmaskedmotionmodelingopticalflowtemporalconsistencyreference-basedmodulebenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that image-guided video editing can be done with a diffusion model built on top of an existing image editor, without retraining the whole model from scratch. The authors introduce IVEDiff, which takes a source video, a reference image, and per-frame masks, and replaces the masked regions with semantically matching content from the reference while preserving temporal smoothness. Its two key ingredients are a motion reference network that uses optical flow from the source video to propagate edited frames, and a masked motion modeling fine-tuning strategy that trains only the newly added motion modules by partially occluding video frames. The paper also builds a benchmark of 236 video-mask-reference triplets and reports that IVEDiff lowers warp error and raises CLIP score relative to frame-wise editing and AnimateDiff-based baselines.

What carries the argument

The two central objects are the optical-flow-guided motion reference network (MotRefNet) and the masked motion modeling (MMM) fine-tuning strategy. MotRefNet is inserted at each layer of the denoising UNet; it takes the latent features of two neighboring edited frames plus the optical flow computed by GMFlow from the corresponding source frames, predicts a two-channel per-pixel offset map, warps the previous edited frame's latents to align with the next frame, and then adds a learnable scaled residual to handle occlusion. MMM is the training scheme: from a video clip, the first frame is used as the reference image and the remaining frames are partially occluded by random grid masks, then the model is trained with the standard denoising objective while only MotRefNet and the motion module weights are updated and the base image editing model stays frozen. Together they let the model learn inter-frame motion dynamics without forgetting the intra-frame semantic editing capability of the base model.

What would settle it

Take a source video of an object moving left-to-right, mask it, and edit it with a reference image of a similar object that should move right-to-left. If the model outputs a video where the new object still moves left-to-right following the source flow, rather than adopting the reference's motion, then the warp prior is overriding the reference information and the claim of robust editing fails. Quantitatively, one could compare the edited video's optical flow inside the mask against both the source flow and the reference's implied motion.

Watch

Extended reading notes

Core claim

The central claim is that IVEDiff is the first image-guided video editing diffusion model and that it can generate temporally smooth edited videos while robustly handling a variety of editing objects at high quality. Given a reference image and per-frame masks, the model replaces the masked regions with content semantically related to the reference, and it does so without requiring text prompts. The paper shows that inflating the MimicBrush image editing model with a motion module and fine-tuning with masked motion modeling yields a better trade-off between inter-frame consistency and per-frame visual quality than frame-wise editing or naive AnimateDiff inflation. Quantitatively, on the object-modification task the full model reduces warp error from 0.683 to 0.553 and raises CLIP score from 23.43 to 25.84 relative to the strongest baseline.

Load-bearing premise

The load-bearing premise is that optical flow computed from the source video remains a valid propagation prior for the edited video, even when the masked region is replaced by semantically unrelated content.

Editorial extensions

If this is right

  • Image-guided video editing becomes a lightweight extension of image editing: inflating an existing image editor with a motion module and fine-tuning with MMM is enough to edit videos, avoiding full video-diffusion training.
  • Users can edit a video by pointing to an object and supplying a reference image, with no text prompt, lowering the interface barrier for localized video editing.
  • The IVE-Benchmark gives the field a shared set of 236 video-mask-reference triplets and five metrics (warp error, temporal consistency, FID, CLIP score) for comparing future methods.
  • Because unedited regions are preserved exactly, the same model can be applied iteratively to edit multiple objects in one video.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same warp-and-fine-tune recipe could plausibly extend to other guided video editing setups, such as text-guided or mask-guided inpainting, since the flow prior comes from the unedited source and does not depend on the modality of the reference.
  • If the source flow assumption is violated enough, the learned two-channel offset in MotRefNet would need to model the full motion difference between old and new content; the paper does not quantify when this correction saturates, so a stress test with motion-opposite references would be a natural next experiment.
  • The benchmark's metrics measure temporal and appearance statistics but not whether the edited object's movement is semantically consistent with the reference; a perceptual or motion-semantics metric would complement the reported numbers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces IVEDiff, a diffusion model for image-guided video editing. Given a source video, per-frame masks, and a reference image, the model edits masked regions by injecting features from a reference UNet and uses a learnable MotRefNet to warp previous-frame latents along source optical flow; an MMM fine-tuning strategy randomly masks spatial regions of training clips while keeping the first frame as reference. The authors construct an IVE-Benchmark of 236 video-mask-reference triplets covering object modification and texture transfer, and compare against per-frame MimicBrush editing, AnimateDiffV3, and a fine-tuned variant on Warp Error, Temporal Consistency, FID, and CLIP Score. The paper claims to be the first image-guided video editing diffusion model and to achieve temporally smooth, high-quality edits.

Significance. If the reported results hold, IVEDiff is a practical contribution: it extends reference-based image editing to video with local control, introduces a sensible MMM objective that aligns fine-tuning with inference, and provides a reusable benchmark for an under-studied task. The ablation structure (Table 2) and consistent direction of improvements across metrics are strengths. However, the central claim of temporal smoothness rests on metrics that reuse the same source optical flow that guides the model, and the benchmark is self-constructed with hyperparameters selected on it; independent verification is therefore needed before the claim is established. The paper does not release code or data, limiting reproducibility.

major comments (3)
  1. [§4.2 and §5.1, Eqs. (3)-(6), Table 1] The temporal-consistency evaluation is partly circular. In §4.2 (Eqs. 3-6), MotRefNet is trained to warp previous edited latents using GMFlow optical flow computed from the source frames. In §5.1, 'Warp Error' and 'Temporal Consistency' are computed by warping edited frames with optical flow obtained from the source video, following references [24,52]. Thus a model that reproduces source motion is rewarded by these metrics even if the edited object's appearance is not faithfully propagated. The paper provides no metric computed on the edited video itself (e.g., flow estimated between edited frames) and no experiment where the edited object moves differently from the source object. This is a load-bearing evidence gap for the abstract and conclusion claim of 'temporally smooth' editing, and it should be addressed with an independent evaluation before the central claim is accepted.
  2. [§5.4, Tables 3-6, Appendix B] The key MMM hyperparameters (mask ratio in Tables 3 and 6, stride in Table 5, mask strategy in Table 4) are selected through ablations on the same IVE-Benchmark that is used to report final numbers. No held-out validation split, cross-validation, or error bars are provided. Because the final comparison in Table 1 and the ablation claims in §5.4 are based on this benchmark, the reported margins (e.g., Warp Error 0.683→0.553 and Temporal Consistency 0.665→0.710 in Table 2) may overfit the evaluation set. The authors should either fix hyperparameters on a validation split or report variance over multiple runs or splits.
  3. [Appendix A.1 and §4.2] For the object-modification setting, Appendix A.1 states that the depth map is zeroed, the reference image is semantically unrelated to the source object, and the mask is a coarse rectangle. In this setting the source optical-flow prior is the only motion signal available to both MotRefNet and the evaluation metrics. The paper does not analyze the failure mode where the new object's motion differs from the source object, even though MotRefNet's warp guidance is then systematically wrong. A concrete test would be to edit objects with known different trajectories and measure flow consistency estimated on the edited video, or to compare with a variant that does not use source flow. Without such a test, the claim of 'robustly dealing with various editing objects' in §6 is not fully supported.
minor comments (6)
  1. [§4.3, Eq. (10)] The text refers to 'the forward diffusion schedule as in Equation (10)', but the forward diffusion formula is Equation (9); Equation (10) is the denoising loss. Please correct the cross-reference.
  2. [Throughout] There are frequent typographical inconsistencies: 'temoprally' in §1, 'M OTREFNET' and 'IVED IFF' vs 'IVEDiff' throughout, and 'Davis90' in §5.1 should be 'DAVIS' with the correct dataset version.
  3. [Tables 3 and 6] Tables 3 and 6 are redundant; Table 6 should be the main table or Table 3 should be removed.
  4. [Figures 4-9] The qualitative comparisons are shown at low resolution without zoomed insets; at the printed size it is difficult for a reader to verify the claimed differences in temporal flicker and texture fidelity.
  5. [Tables 1-6] The paper does not report confidence intervals, standard deviations, or significance tests for any metric; with a benchmark of 236 triplets, several metrics (especially FID) are likely to be noisy.
  6. [Appendix A.2, reference [23]] Appendix A.2 reports that the model is trained on the Pexels dataset, but reference [23] points to Open-Sora-Plan rather than a Pexels data source; please clarify the data source and its license.

Circularity Check

2 steps flagged · score 4.0 of 10

Temporal-smoothness evidence and MMM hyperparameters are coupled to the model's own source-flow input and test benchmark, but the core editing pipeline is externally trained and not derivationally circular.

  1. self definitional [Section 4.2 (Eqs. 3-6) and Section 5.1 (IVE-Benchmark metrics)]
    "In M OTREFNET, we utilize the pre-trained GM-Flow [43], which is a robust optical flow predictor, to capture the complete and confident optical flows across neighboring frames of the source video, and treat it as the information propagation prior between two neighboring frames in the edited video. Additionally, we assess the performance of different methods from 3 perspectives, including the temporal smoothness (Warp Error [24], Temporal Consistency [52])."

    The Warp Error [24] and Temporal Consistency [52] metrics are computed by warping the edited frames according to optical flow estimated on the source video, which is exactly the signal MotRefNet consumes: Eq. (3) extracts of_{i->i+1} from the source frames, Eq. (4) conditions the 2-channel offset on that flow, and Eq. (5) uses it to warp the previous edited latent. If the learned offset reproduces the downsampled source flow, the next edited latent equals the source-flow-warped previous latent, which is precisely what the Warp Error metric scores as correct correspondence. The reported smoothness gains therefore partly measure how well the model echoes its own input flow, rather than an independent property of the edited video.

  2. fitted input called prediction [Section 5.4 and Appendix B (Tabs. 3-6)]
    "We set up a series of experiments to explore the impact of mask settings in MMM on the performance of IVED IFF, including ... the ratio of masked regions ({0, 0.25, 0.5, 0.75} in Tab. 3). The results presented in Tabs. 3 and 4 indicate that by applying masks that occlude different regions to different frames within a video clip and setting a larger proportion of masked regions, the model is required to have a stronger capability for modeling both intra- and inter-frame correlations, thereby effectively enhancing the model's ability to maintain temporal consistency between frames."

    The MMM hyperparameters (mask ratio, stride, and frame-wise versus clip-wise masking) are ablated directly on the IVE-Benchmark whose Table 1 numbers are the paper's main evidence, and no train/validation/test split for the benchmark is described. The final model is therefore presented with the configurations selected by optimizing the same metrics that later appear as the headline comparison. The claim that comprehensive experiments demonstrate a temporally smooth edited video is partly a selection result on the test benchmark, not an independent prediction on held-out data. This inflates the apparent margin over baselines, although it does not by itself invalidate the architecture.

full rationale

No derivation-level circularity is present in the modeling: IVEDiff starts from the external MimicBrush image-editing checkpoint, uses external GMFlow and Depth-Anything priors, and is fine-tuned on the Pexels video dataset with a standard denoising objective, so the edited frame content is not an input to itself. The self-citation patterns in the rubric do not occur; the base models are cited from other groups. The circularity that does exist is evaluative. First, the temporal-smoothness metrics are defined with the same source-video optical flow that MotRefNet explicitly takes as input, so the reported Warp Error and Temporal Consistency gains partly reward the model for reproducing its own conditioning signal. Second, the key MMM hyperparameters are selected by ablations on the same IVE-Benchmark that supplies the final comparison numbers, with no held-out split, making the headline results partly a test-set selection artifact. These are significant evidence-quality concerns but not identity-by-construction: the model could still fail at the object re-appearance task, and the semantic-alignment and FID numbers are not forced by the flow input. Hence score 4.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper relies on several pretrained models and hand-set hyperparameters; none of the introduced components are postulated entities requiring independent falsifiable handles. The key added burden is the assumption that source optical flow correctly guides edited content, and that synthetic grid masking transfers to real user masks. The reported free parameters are hyperparameters tuned on the self-built benchmark.

free parameters (5)
  • mask ratio in MMM = 0.5
    Grid masking over 64x64 latents; selected experimentally in Tabs. 3 and 6. Central to the MMM fine-tuning strategy.
  • temporal stride for clip sampling = 4
    Chosen after ablation in Tab. 5, balancing reference correlation and task difficulty.
  • MMM mask grid size N = 8
    Spatial grid used to generate masks on 64x64 latents; hand-set in Sec. 5.1.
  • fine-tuning video clip length = 7 frames
    First frame as reference plus six training frames; resource-limited choice described in Appendix A.2.
  • MotRefNet occlusion scalar alpha = not reported
    Learned scalar in Eq. 6 for occlusion compensation; no final value is reported, so its contribution is not independently checkable.
assumptions (5)
  • domain assumption Latent diffusion denoising objective is an effective training signal for temporal modules.
    Adopted from AnimateDiff and prior diffusion work; used as the fine-tuning loss in Eq. 10.
  • domain assumption MimicBrush provides a strong image-guided editing prior that can be inflated to video.
    Used as the frozen base model in Sec. 4.1; its semantic correlation capabilities are assumed to transfer to video.
  • domain assumption GMFlow optical flow from source frames is a reliable prior for edited-frame feature propagation.
    Eqs. 3-5 use source optical flow to warp edited latents; no validation is provided for the mapping under semantic replacement.
  • ad hoc to paper Random grid masking on natural videos approximates user-provided object and rectangle masks at inference.
    MMM trains on random 8x8 grid masks with 0.5 ratio, while inference uses SAM2 object masks or rectangles (Sec. 5.1); this is a training-inference distribution shift.
  • ad hoc to paper Depth maps from Depth-Anything remain valid for edited content, and setting depth to zero for object modification is acceptable.
    Depth is concatenated as input (Sec. 4.1) and zeroed for object modification in Appendix A.1; the paper does not analyze the effect.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Edit as You See: Image-guided Video Editing via Masked Motion Modeling." pith.science (2026). https://pith.science/paper/EFLTXRNP

@misc{pith2026250104325,
  author       = {Pith},
  title        = {Pith review of: Edit as You See: Image-guided Video Editing via Masked Motion Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EFLTXRNP}},
  note         = {Machine review of arXiv:2501.04325}
}
read the original abstract

Recent advancements in diffusion models have significantly facilitated text-guided video editing. However, there is a relative scarcity of research on image-guided video editing, a method that empowers users to edit videos by merely indicating a target object in the initial frame and providing an RGB image as reference, without relying on the text prompts. In this paper, we propose a novel Image-guided Video Editing Diffusion model, termed IVEDiff for the image-guided video editing. IVEDiff is built on top of image editing models, and is equipped with learnable motion modules to maintain the temporal consistency of edited video. Inspired by self-supervised learning concepts, we introduce a masked motion modeling fine-tuning strategy that empowers the motion module's capabilities for capturing inter-frame motion dynamics, while preserving the capabilities for intra-frame semantic correlations modeling of the base image editing model. Moreover, an optical-flow-guided motion reference network is proposed to ensure the accurate propagation of information between edited video frames, alleviating the misleading effects of invalid information. We also construct a benchmark to facilitate further research. The comprehensive experiments demonstrate that our method is able to generate temporally smooth edited videos while robustly dealing with various editing objects with high quality.

Figures

Figures reproduced from arXiv: 2501.04325 by the authors.

Figure 1
Figure 1. Based on the given reference image and masks indicating the regions to be edited in the source video, our IVED [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overall pipeline of the proposed masked motion modeling fine-tuning strategy. To align the fine-tuning process with the inference [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Overall framework of the proposed IVEDIFF. spatially downsampled to the resolution of the noised latent. And the reference image which is encoded to the image em￾bedding through the CLIP, is fused into the UNet via the cross-attention mechanism. The noised latents takes charge of the diffusion procedure from initial noises to the output latent codes step by step. Besides, to help the model to bet￾ter capture the fin… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The qualitative comparisons between our methods with baselines. For the texture transfer, we use the fine-grained masks which [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison of main components. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: The exemplar of IVE-Benchmark. For the target contents in the source video, IVE-Benchmark provides two types of mask, the fine- and coarse-grained mask for texture transfer and object modification application, respectively. Moreover, we provide two types of reference i…
Figure 7
Figure 7. Figure 7: Multi-target Editing. [Best viewed with zoom-in.] 14 [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Qualitative comparisons on the object modification application. [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Qualitative comparisons on the texture transfer application. [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 41 canonical work pages

  1. [1]

    Blended diffusion for text-driven editing of natural images

    Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. In IEEE Conf. Comput. Vis. Pattern Recog. , pages 18208–18218,

    work page

  2. [2]

    Text2live: Text-driven layered image and video editing

    Omer Bar-Tal, Dolev Ofri-Amar, Rafail Fridman, Yoni Kas- ten, and Tali Dekel. Text2live: Text-driven layered image and video editing. In Eur. Conf. Comput. Vis., pages 707–

    work page

  3. [3]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. In IEEE Conf. Comput. Vis. Pattern Recog. , pages 18392– 18402, 2023. 3

    work page 2023

  4. [4]

    Pix2video: Video editing using image diffusion

    Duygu Ceylan, Chun-Hao P Huang, and Niloy J Mitra. Pix2video: Video editing using image diffusion. In IEEE Conf. Comput. Vis. Pattern Recog. , pages 23206–23217,

    work page

  5. [5]

    Stable- video: Text-driven consistency-aware diffusion video edit- ing

    Wenhao Chai, Xun Guo, Gaoang Wang, and Yan Lu. Stable- video: Text-driven consistency-aware diffusion video edit- ing. In Int. Conf. Comput. Vis., pages 23040–23050, 2023. 2

    work page 2023

  6. [6]

    Zero-shot image editing with reference imitation

    Xi Chen, Yutong Feng, Mengting Chen, Yiyang Wang, Shi- long Zhang, Yu Liu, Yujun Shen, and Hengshuang Zhao. Zero-shot image editing with reference imitation. arXiv preprint arXiv:2406.07547, 2024. 2, 3, 4, 6, 7, 12

    arXiv 2024

  7. [7]

    Xmem: Long- term video object segmentation with an atkinson-shiffrin memory model

    Ho Kei Cheng and Alexander G Schwing. Xmem: Long- term video object segmentation with an atkinson-shiffrin memory model. In Eur. Conf. Comput. Vis., pages 640–658. Springer, 2022. 6

    work page 2022

  8. [8]

    Compvis/stable-diffusion: A latent text-to- image diffusion model

    CompVis. Compvis/stable-diffusion: A latent text-to- image diffusion model. https://github.com/CompVis/stable- diffusion. 2

    work page

Show all 54 references
  1. [9]

    Diffedit: Diffusion-based seman- tic image editing with mask guidance

    Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. Diffedit: Diffusion-based seman- tic image editing with mask guidance. arXiv preprint arXiv:2210.11427, 2022. 3

    2022 arXiv

  2. [10]

    Videdit: Zero-shot and spatially aware text-driven video editing.IEEE Trans

    Paul Couairon, Cl ´ement Rambour, Jean-Emmanuel Haugeard, and Nicolas Thome. Videdit: Zero-shot and spatially aware text-driven video editing.IEEE Trans. Mach. Learn. Resea., 2023. 2

    2023

  3. [11]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Adv. Neural Inform. Process. Syst., 34:8780–8794, 2021. 2

    2021

  4. [12]

    Editanything: Empower- ing unparalleled flexibility in image editing and generation

    Shanghua Gao, Zhijie Lin, Xingyu Xie, Pan Zhou, Ming- Ming Cheng, and Shuicheng Yan. Editanything: Empower- ing unparalleled flexibility in image editing and generation. In ACMMM, pages 9414–9416, 2023. 3

    2023

  5. [13]

    Tokenflow: Consistent diffusion features for consistent video editing

    Michal Geyer, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Tokenflow: Consistent diffusion features for consistent video editing. arXiv preprint arXiv:2307.10373, 2023. 2

    2023 arXiv

  6. [14]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 2, 3, 6, 7, 12

    2023 arXiv

  7. [15]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In IEEE Conf. Comput. Vis. Pattern Recog., pages 16000–16009, 2022. 5

    2022

  8. [16]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Adv. Neural Inform. Process. Syst., 33:6840–6851, 2020. 2, 6

    2020

  9. [17]

    Gritsenko, William Chan, Mohammad Norouzi, and David J

    Jonathan Ho, Tim Salimans, Alexey A. Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video dif- fusion models. In Adv. Neural Inform. Process. Syst., 2022. 2

    2022

  10. [18]

    Lite- flownet: A lightweight convolutional neural network for op- tical flow estimation

    Tak-Wai Hui, Xiaoou Tang, and Chen Change Loy. Lite- flownet: A lightweight convolutional neural network for op- tical flow estimation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 8981–8989, 2018. 5

    2018

  11. [19]

    Vmc: Video motion customization using temporal attention adap- tion for text-to-video diffusion models

    Hyeonho Jeong, Geon Yeong Park, and Jong Chul Ye. Vmc: Video motion customization using temporal attention adap- tion for text-to-video diffusion models. In IEEE Conf. Com- put. Vis. Pattern Recog., pages 9212–9221, 2024. 2

    2024

  12. [20]

    Imagic: Text-based real image editing with diffusion models

    Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In IEEE Conf. Comput. Vis. Pattern Recog., pages 6007–6017,

  13. [21]

    Dif- fusionclip: Text-guided diffusion models for robust image manipulation

    Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. Dif- fusionclip: Text-guided diffusion models for robust image manipulation. In IEEE Conf. Comput. Vis. Pattern Recog. , pages 2426–2435, 2022. 3

    2022

  14. [22]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Int. Conf. Comput. Vis., pages 4015–4026, 2023. 6, 12 9

    2023

  15. [23]

    Open-sora-plan, 2024

    PKU-Yuan Lab and Tuzhan AI etc. Open-sora-plan, 2024. 6, 12

    2024

  16. [24]

    Learning blind video temporal consistency

    Wei-Sheng Lai, Jia-Bin Huang, Oliver Wang, Eli Shechtman, Ersin Yumer, and Ming-Hsuan Yang. Learning blind video temporal consistency. InEur. Conf. Comput. Vis., pages 170– 185, 2018. 6

    2018

  17. [25]

    Generative image dynamics

    Zhengqi Li, Richard Tucker, Noah Snavely, and Aleksander Holynski. Generative image dynamics. In IEEE Conf. Com- put. Vis. Pattern Recog., pages 24142–24153, 2024. 3

    2024

  18. [26]

    Video-p2p: Video editing with cross-attention control

    Shaoteng Liu, Yuechen Zhang, Wenbo Li, Zhe Lin, and Jiaya Jia. Video-p2p: Video editing with cross-attention control. In IEEE Conf. Comput. Vis. Pattern Recog., pages 8599–8608,

  19. [27]

    Null-text inversion for editing real images using guided diffusion models

    Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In IEEE Conf. Comput. Vis. Pattern Recog., pages 6038–6047, 2023. 3

    2023

  20. [28]

    Dreamix: Video diffusion models are general video editors

    Eyal Molad, Eliahu Horwitz, Dani Valevski, Alex Rav Acha, Yossi Matias, Yael Pritch, Yaniv Leviathan, and Yedid Hoshen. Dreamix: Video diffusion models are general video editors. arXiv preprint arXiv:2302.01329, 2023. 2, 3

    2023 arXiv

  21. [29]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021. 3

    2021 arXiv

  22. [30]

    The best free stock photos, royalty free images & videos shared by creators

    Pexels. The best free stock photos, royalty free images & videos shared by creators. https://www.pexels.com,

  23. [31]

    Fatezero: Fus- ing attentions for zero-shot text-based video editing

    Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. Fatezero: Fus- ing attentions for zero-shot text-based video editing. In Int. Conf. Comput. Vis., pages 15932–15942, 2023. 2

    2023

  24. [32]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In IEEE Conf. Comput. Vis. Pattern Recog., pages 10684–10695, 2022. 2, 3

    2022

  25. [33]

    pytorch-fid: FID Score for PyTorch

    Maximilian Seitzer. pytorch-fid: FID Score for PyTorch. https://github.com/mseitzer/pytorch-fid ,

  26. [34]

    Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling

    Xiaoyu Shi, Zhaoyang Huang, Fu-Yun Wang, Weikang Bian, Dasong Li, Yi Zhang, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, et al. Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling. In ACM SIGGRAPH, pages 1–11, 2024. 2

    2024

  27. [35]

    Dragdiffusion: Harnessing diffusion models for interactive point-based image editing

    Yujun Shi, Chuhui Xue, Jun Hao Liew, Jiachun Pan, Han- shu Yan, Wenqing Zhang, Vincent YF Tan, and Song Bai. Dragdiffusion: Harnessing diffusion models for interactive point-based image editing. In IEEE Conf. Comput. Vis. Pat- tern Recog., pages 8839–8849, 2024. 3

    2024

  28. [36]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 ,

    arXiv

  29. [37]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 6

    2010 arXiv

  30. [38]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. In Int. Conf. Learn. Represent. 2

  31. [39]

    Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training

    Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training. Adv. Neural Inform. Process. Syst., 35:10078–10093, 2022. 5

    2022

  32. [40]

    Zero-shot video editing using off-the-shelf image diffusion models

    Wen Wang, Yan Jiang, Kangyang Xie, Zide Liu, Hao Chen, Yue Cao, Xinlong Wang, and Chunhua Shen. Zero-shot video editing using off-the-shelf image diffusion models. arXiv preprint arXiv:2303.17599, 2023. 2

    2023 arXiv

  33. [41]

    Latent image animator: Learning to ani- mate images via latent space navigation

    Yaohui Wang, Di Yang, Francois Bremond, and Antitza Dantcheva. Latent image animator: Learning to ani- mate images via latent space navigation. arXiv preprint arXiv:2203.09043, 2022. 2, 3

    2022 arXiv

  34. [42]

    Dynamicrafter: Animating open-domain images with video diffusion priors

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Wangbo Yu, Hanyuan Liu, Gongye Liu, Xintao Wang, Ying Shan, and Tien-Tsin Wong. Dynamicrafter: Animating open-domain images with video diffusion priors. In Eur. Conf. Comput. Vis., pages 399–417. Springer, 2025. 2, 3

    2025

  35. [43]

    Gmflow: Learning optical flow via global matching

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, and Dacheng Tao. Gmflow: Learning optical flow via global matching. In IEEE Conf. Comput. Vis. Pattern Recog., pages 8121–8130, 2022. 4, 5

    2022

  36. [44]

    Magicprop: Diffusion-based video editing via motion-aware appearance propagation

    Hanshu Yan, Jun Hao Liew, Long Mai, Shanchuan Lin, and Jiashi Feng. Magicprop: Diffusion-based video editing via motion-aware appearance propagation. arXiv preprint arXiv:2309.00908, 2023. 2

    2023 arXiv

  37. [45]

    Motion-conditioned image animation for video editing

    Wilson Yan, Andrew Brown, Pieter Abbeel, Rohit Girdhar, and Samaneh Azadi. Motion-conditioned image animation for video editing. arXiv preprint arXiv:2311.18827, 2023. 2

    2023 arXiv

  38. [46]

    Paint by example: Exemplar-based image editing with diffusion models

    Binxin Yang, Shuyang Gu, Bo Zhang, Ting Zhang, Xue- jin Chen, Xiaoyan Sun, Dong Chen, and Fang Wen. Paint by example: Exemplar-based image editing with diffusion models. In IEEE Conf. Comput. Vis. Pattern Recog., pages 18381–18391, 2023. 3

    2023

  39. [47]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. InIEEE Conf. Com- put. Vis. Pattern Recog., pages 10371–10381, 2024. 4

    2024

  40. [48]

    Rerender a video: Zero-shot text-guided video-to-video translation

    Shuai Yang, Yifan Zhou, Ziwei Liu, and Chen Change Loy. Rerender a video: Zero-shot text-guided video-to-video translation. In ACM SIGGRAPH Asia, pages 1–11, 2023. 3

    2023

  41. [49]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721,

    arXiv

  42. [50]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Int. Conf. Comput. Vis., pages 3836–3847, 2023. 3

    2023

  43. [51]

    Sine: Single image editing with text-to-image diffusion models

    Zhixing Zhang, Ligong Han, Arnab Ghosh, Dimitris N Metaxas, and Jian Ren. Sine: Single image editing with text-to-image diffusion models. In IEEE Conf. Comput. Vis. Pattern Recog., pages 6027–6037, 2023. 3

    2023

  44. [52]

    Avid: Any-length video inpainting with dif- fusion model

    Zhixing Zhang, Bichen Wu, Xiaoyan Wang, Yaqiao Luo, Luxin Zhang, Yinan Zhao, Peter Vajda, Dimitris Metaxas, 10 and Licheng Yu. Avid: Any-length video inpainting with dif- fusion model. In IEEE Conf. Comput. Vis. Pattern Recog.,, pages 7162–7172, 2024. 3, 6

    2024

  45. [53]

    Motiondirector: Motion customization of text-to-video diffusion models

    Rui Zhao, Yuchao Gu, Jay Zhangjie Wu, David Jun- hao Zhang, Jia-Wei Liu, Weijia Wu, Jussi Keppo, and Mike Zheng Shou. Motiondirector: Motion customization of text-to-video diffusion models. In Eur. Conf. Comput. Vis., pages 273–290. Springer, 2025. 2

    2025

  46. [54]

    clip-score: CLIP Score for Py- Torch

    SUN Zhengwentai. clip-score: CLIP Score for Py- Torch. https : / / github . com / taited / clip - score, 2023. Version 0.1.1. 6 11 A. IVE-Benchmark, Training and Inference Details A.1. IVE-Benchmark. Similar to MimicBrush [6], in the construction of IVE- Benchmark, we select t...

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.