Pith. sign in

REVIEW 3 major objections 6 minor 18 cited by

DiffuEraser: A Diffusion Model for Video Inpainting

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a stable-diffusion video inpainter, seeded with a DDIM-inverted prior from ProPainter and stabilized by staggered clip denoising, beats transformer baselines in detail and temporal consistency on large masks and long…

desk verdict A plausible engineering combination for diffusion video inpainting, but the paper's central claim of outperforming SOTA is unsupported by any quantitative evaluation, ablations, or comparisons beyond a few qualitative figures against ProPainter. read the letter →

arxiv 2501.10018 v1 pith:D2K2NMBK submitted 2025-01-17 cs.CV

Xiaowen Li , Haolan Xue , Peiran Ren , Liefeng Bo This is my paper · ORCID
classification cs.CV
keywords videoinpaintingdiffusionmodeltemporalconsistencyDDIMinversionpriorconditioninglong-sequenceinferencemotionmodulestable
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DiffuEraser tries to establish that a stable-diffusion-based video inpainting model can outperform the transformer-based ProPainter pipeline on large masks and long sequences by decomposing the task into three sub-problems: propagating known pixels, generating unknown pixels, and keeping completed content temporally consistent. It claims that injecting the DDIM-inverted output of a weaker prior model into the diffusion noise latent gives better initialization and weak conditioning, reducing artifacts and hallucinations. It also claims that expanding the temporal receptive field via pre-propagation and pre-inference, plus a staggered denoising schedule at clip boundaries, yields temporally stable long-video inpainting. If true, this would give practical video editing a way to combine flow-based propagation reliability with diffusion-generated detail.

What carries the argument

The central mechanism is prior injection: applying DDIM inversion to the prior model's inpainted frames and adding the inverted latent to the diffusion model's noisy input, which serves both as a better starting point for denoising and as a weak condition that suppresses hallucinated objects. The second mechanism is the staggered denoising schedule that alternates denoising starts between the clip start and the clip midpoint, exploiting the video diffusion model's inherent temporal smoothing to remove cliff-edge jumps between consecutive clips. The third is receptive-field expansion: pre-propagation for the prior model and pre-inference for DiffuEraser on sampled frames, so information from distant frames guides the final pass on every frame.

What would settle it

Run DiffuEraser on identical large-mask video clips in two configurations: with the DDIM-inverted ProPainter prior and with pure noise or a random prior latent. If the no-prior version matches or beats the prior version on a quantitative temporal-consistency metric (e.g., warped error) and FID, the paper's claim that injected priors suppress artifacts fails.

Watch

Extended reading notes

Core claim

On the paper's terms, DiffuEraser is a video inpainting model built from BrushNet with AnimateDiff-style motion modules, trained on Panda-70M clips with random masks. The key inference-time moves are: (1) run ProPainter first, apply DDIM inversion to its output, and add the inverted latent to the noisy latent as initialization and weak conditioning; (2) for long videos, run a pre-propagation and pre-inference on sampled frames to extend the temporal receptive field across the entire video, then guide frame-by-frame inference; (3) at clip intersections, use a staggered denoising schedule where even and odd timesteps start from the clip start and clip midpoint, exploiting the video diffusion model's temporal smoothing to blend boundaries. The paper reports that this outperforms state-of-the-art methods in both content completeness and temporal consistency while keeping inference practical via two-step Phased Consistency Model sampling, processing a 10-second 540p video in about 200 seconds on an L20 GPU.

Load-bearing premise

The method assumes that running DDIM inversion on the prior model's (ProPainter's) output produces a noisy latent that correctly initializes the diffusion denoiser at every timestep; if that inverted latent is inaccurate, the prior could inject errors instead of removing them.

Editorial extensions

If this is right

  • Inpainting results on large masks will contain more fine texture and fewer blur artifacts than transformer-based generation, because diffusion generation replaces the transformer's content-completion stage.
  • Temporal flicker at boundaries between clips in long-sequence inference will be reduced, making the method usable for full-video editing rather than only short clips.
  • The injected-prior technique should suppress the hallucinated objects that pure diffusion inpainting tends to introduce, improving fidelity in object-removal scenarios.
  • By using Phased Consistency Model sampling, the method stays practical: a 10-second 540p video is processed in about 200 seconds on a single L20 GPU.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The DDIM-inversion prior injection is a general recipe: any clip-based video diffusion edit task, such as object replacement or local stylization, could adopt the same 'run a cheap prior, invert it, add to the latent' strategy to stabilize outputs even if the prior itself is blurry.
  • The staggered denoising trick likely transfers to other video diffusion models that suffer from within-clip consistency but cross-clip jumps, since it only requires controlling the denoising start timestep per clip.
  • A testable extension: swap ProPainter for a weaker or faster prior and measure whether final quality stays similar; the paper's claim that prior artifacts get refined away predicts quality should be insensitive to prior blur, which can be quantified.
  • The long-sequence consistency claim rests on qualitative examples; a quantitative temporal-consistency metric over long videos, such as warped error or a learned temporal score, would give a sharper verification than the figures shown.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes DiffuEraser, a video inpainting model built on a Stable Diffusion backbone, combining a BrushNet-style masked-image conditioning branch with a motion module and temporal attention for video generation. To improve long-sequence consistency and reduce diffusion hallucinations, the method injects priors by applying DDIM inversion to the output of a transformer-based prior model (ProPainter) and using the inverted latent as initialization and weak conditioning. It also expands the temporal receptive field via pre-propagation/pre-inference and uses a staggered denoising schedule to smooth clip boundaries. The authors claim their method outperforms state-of-the-art video inpainting techniques in content completeness and temporal consistency while maintaining acceptable efficiency. The experimental section, however, reports only qualitative comparisons against ProPainter on a few illustrative videos, with training details and a single efficiency number, and no quantitative metrics, ablations, or evaluation protocol.

Significance. If the claimed improvements are validated, DiffuEraser would be a practically useful contribution: it combines the generative strength of diffusion models with the known-pixel propagation of an optical-flow based prior, and it addresses long-sequence temporal consistency with inference-time techniques. The paper provides a public code repository, detailed training settings, and an efficiency measurement (about 200 seconds for a 10-second 540p clip on an L20 GPU), which are useful starting points for reproducibility. However, the absence of any quantitative evaluation against competing methods, and the lack of ablations for the three proposed mechanisms, means the central claim of state-of-the-art performance is not currently established. The significance would be much higher if the paper included standard metrics (PSNR, SSIM, LPIPS, VFID, warp error, user study) and comparisons with existing diffusion-based video inpainting methods.

major comments (3)
  1. [Section 4 (Experiments)] The central claim, stated in the abstract and conclusion, that DiffuEraser 'outperforms state-of-the-art techniques in both content completeness and temporal consistency' is not supported by any quantitative evidence. The only comparison is qualitative and against a single baseline, ProPainter (Figures 1, 10-13). No PSNR, SSIM, LPIPS, VFID, warp error, user study, or significance test is reported, and no evaluation protocol or test dataset is described. This makes the empirical superiority claim unfalsifiable from the submitted material. The authors should add a thorough quantitative evaluation on standard video inpainting benchmarks, comparing against several state-of-the-art methods, including diffusion-based baselines such as BIVDiff, AVID, FloED, and FFF-VDI, and report error bars or significance tests where applicable.
  2. [Section 3.2 (Incorporation of Priors)] The paper asserts that 'any blur and mosaic artifacts present in the prior do not adversely affect our model's outputs; instead, they are refined and eliminated.' This is an empirical claim that is not backed by any experiment or ablation. The validity of the DDIM-inversion prior as an initialization for the diffusion denoiser is also not tested; if the inversion is inaccurate, the prior could introduce artifacts or bias the output. An ablation that removes the prior, uses a deliberately degraded prior, and measures output quality would be needed to support the claim and to justify the design choice of using ProPainter as the prior model.
  3. [Section 3.3 (Temporal Consistency)] The proposed temporal-consistency optimizations—prior injection, expanded temporal receptive field via pre-propagation/pre-inference, and the staggered denoising schedule—are not individually ablated. Without per-component ablations, the observed qualitative improvements cannot be attributed to the specific mechanisms proposed. The authors should provide a component-wise ablation study with quantitative metrics on long sequences, including measuring temporal consistency at clip boundaries and across the full video.
minor comments (6)
  1. [Section 4 (Training Details and Metrics)] There is a typo: 'fist stage' should be 'first stage'.
  2. [Throughout] The name 'ProPainter' is written inconsistently as 'Propainter' and 'ProPainter'; please unify the spelling.
  3. [Section 3.2] The statement 'After experimental comparisons, we selected Propainter as our prior model' is not supported by any reported comparison. The authors should describe these comparisons or remove the claim.
  4. [Section 4 (Efficiency)] The efficiency measurement ('about 200 seconds' for a 10-second 540p clip) should specify the hardware, whether it includes the prior model inference and the pre-inference steps, and how it compares with the cost of ProPainter or other baselines.
  5. [Related Work and Experiments] The related work lists several recent diffusion-based video inpainting methods, but the experiments compare only against ProPainter. At least one diffusion-based baseline should be included in the comparison to support the claim of improvement over state-of-the-art in the diffusion category as well.
  6. [Figure 5] The qualitative comparison in Figure 5 would be more informative if the same random seed and inference settings were specified, since diffusion models are stochastic; please state whether the results are cherry-picked or representative.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DiffuEraser builds on externally cited diffusion components and does not derive its results from fitted inputs or self-citations.

full rationale

The paper does not contain a derivation chain that reduces to its own inputs. Its architecture combines externally established components: BrushNet for image inpainting, AnimateDiff-style motion modules, and DDIM inversion of a prior model's output as initialization and weak conditioning. These are cited as prior work and are not self-citations by the present authors. The prior-injection mechanism is an empirical conditioning choice, not a mathematical construction that forces the output to equal the prior: the diffusion denoiser still transforms the inverted latent, and the paper claims the prior suppresses artifacts rather than being copied verbatim. The qualitative comparison against ProPainter is an empirical claim, and even if it is insufficiently supported by quantitative metrics, lack of evidence is a correctness and completeness concern, not circularity. No equation in the paper equates a prediction to a fitted parameter or defines a claimed result in terms of itself. The contribution framing as three sub-problems with corresponding solutions is organizational, not self-referential. No load-bearing self-citation or uniqueness theorem imported from the authors appears. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new conceptual entities such as forces or particles. The free parameters are inference heuristics and hyperparameters chosen by hand. The axioms are domain assumptions inherited from diffusion model literature and ad hoc mechanisms specific to this method.

free parameters (4)
  • clip_length = 22 frames
    Chosen by hand for the temporal receptive field of each inference pass; no ablation justifies this value.
  • pre_inference_sampling_rate
    The paper refers to an 'optimal rate' for sampling frames in pre-inference but does not specify it, making the method incomplete for reproduction.
  • staggered_denoise_schedule = even/odd timesteps
    The heuristic of alternating even and odd timesteps between clip start and midpoint is chosen by hand to smooth clip boundaries; no analysis supports it.
  • PCM_sampling_steps = 2
    Uses Phased Consistency Models for 2-step generation, a design choice that trades quality for speed.
assumptions (4)
  • standard math Stable diffusion and DDIM inversion are valid as described in prior work
    The method relies on the correctness of DDIM inversion to convert ProPainter output into a noisy latent (Section 3.2).
  • domain assumption The motion module from AnimateDiff improves temporal consistency, and the VDM temporal smoothing property can be controlled via staggered denoising
    The paper assumes that the temporal smoothing behavior of the video diffusion model can be leveraged to blend clip intersections (Section 3.3.1).
  • domain assumption ProPainter provides a sufficiently accurate prior for known pixels, and its artifacts are refined by the diffusion model
    The choice of ProPainter as prior is based on 'experimental comparisons' but no quantitative results are shown (Section 3.2).
  • ad hoc to paper The staggered denoising schedule produces seamless clip boundaries without global conditioning
    This mechanism is introduced without formal analysis or ablation, and its effectiveness is only shown qualitatively (Section 3.3.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of DiffuEraser: A Diffusion Model for Video Inpainting." pith.science (2026). https://pith.science/paper/D2K2NMBK

@misc{pith2026250110018,
  author       = {Pith},
  title        = {Pith review of: DiffuEraser: A Diffusion Model for Video Inpainting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D2K2NMBK}},
  note         = {Machine review of arXiv:2501.10018}
}
read the original abstract

Recent video inpainting algorithms integrate flow-based pixel propagation with transformer-based generation to leverage optical flow for restoring textures and objects using information from neighboring frames, while completing masked regions through visual Transformers. However, these approaches often encounter blurring and temporal inconsistencies when dealing with large masks, highlighting the need for models with enhanced generative capabilities. Recently, diffusion models have emerged as a prominent technique in image and video generation due to their impressive performance. In this paper, we introduce DiffuEraser, a video inpainting model based on stable diffusion, designed to fill masked regions with greater details and more coherent structures. We incorporate prior information to provide initialization and weak conditioning,which helps mitigate noisy artifacts and suppress hallucinations. Additionally, to improve temporal consistency during long-sequence inference, we expand the temporal receptive fields of both the prior model and DiffuEraser, and further enhance consistency by leveraging the temporal smoothing property of Video Diffusion Models. Experimental results demonstrate that our proposed method outperforms state-of-the-art techniques in both content completeness and temporal consistency while maintaining acceptable efficiency.

Figures

Figures reproduced from arXiv: 2501.10018 by the authors.

Figure 1
Figure 1. Performance comparison between the proposed model, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed video inpainting model [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Incorporation of priors. We introduce priors during in [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (9 more)
Figure 3
Figure 3. Figure 3: Example of noisy artifacts generated by the model. The [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]
Figure 6
Figure 6. Figure 6: Utilizing the temporal smoothing property of the Video Diffusion Model (VDM) to enhance consistency at the intersections of [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Temporal consistency optimization for long-sequence in [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]
Figure 9
Figure 9. Figure 9: The temporal consistency obtained from pre-propagation [PITH_FULL_IMAGE:figures/full_fig_p006_9.png]
Figure 8
Figure 8. Figure 8: Perform pre-propagation or pre-inference to expand the [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 10
Figure 10. Figure 10: Texture quality comparison between DiffuEraser and Propainter [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]
Figure 11
Figure 11. Figure 11: Texture quality comparison between DiffuEraser and Propainter [PITH_FULL_IMAGE:figures/full_fig_p009_11.png]
Figure 12
Figure 12. Figure 12: Temporal consistency comparison between DiffuEraser and Propainter [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]
Figure 13
Figure 13. Figure 13: Temporal consistency comparison between DiffuEraser and Propainter. [17] Minhyeok Lee, Suhwan Cho, Chajin Shin, Jungho Lee, Sunghun Yang, and Sangyoun Lee. Video diffusion models are strong video inpainter, 2024. [18] Zhen Li, Cheng-Ze Lu, Jianhua Qin, Chun-Le Guo, an…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 18 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Mirror Learning

    cs.LG 2026-07 conditional novelty 7.0 of 10

    Fine-tuning a video diffusion model to perform cross-view perspective transfer, then labeling the generated first-person videos with an inverse dynamics model, yields behavior-cloning data that improves driving policies.

  2. PROVE: A Perceptual RemOVal cohErence Benchmark for Visual Media

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    Removal Coherence (RC) metrics, which compare local feature distributions in masked versus background regions via sliding-window MMD, align with human judgments of object-removal quality substantially better than exis...

  3. SPLIT: Training-Free AI-Generated and Partially Edited Video Detection via Spatial Patch-Level Incoherence and Temporal Roughness

    cs.CV 2026-07 accept novelty 6.5 of 10

    Training-free patch-token signals (TTR + LSMI) detect fully generated and partially edited videos at 0.1% FPR better than supervised and training-free baselines.

  4. EffectLearner: World-Aware Object-Effect Reasoning for Real-World Video Object Removal

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A VLM-guided video eraser removes objects and their induced effects (shadows, reflections, ripples, smoke) and reports gains on self-built benchmarks with mixed external results on ROSE-Bench.

  5. OmniEdit-Bench: A Comprehensive Benchmark for Instruction-based Video Editing

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A new benchmark and evaluation framework for instruction-based video editing that covers spatial, temporal, audio, reference, and reasoning edits, with an accuracy-aware penalty to prevent inflated scores for incorrect edits.

  6. From Draft to Draft-Free: One-Step Video Object Removal via Privileged Distillation and Fast Planting

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A one-step, draft-free video object removal model trained by distilling a ground-truth-conditioned teacher reaches comparable or better quality than multi-step diffusion methods while running in about 1 second.

  7. Glove2Hand: Synthesizing Natural Hand-Object Interaction from Multi-Modal Sensing Gloves

    cs.CV 2026-03 conditional novelty 6.0 of 10

    A 3D-Gaussian-plus-diffusion pipeline translates multi-modal glove HOI videos into photorealistic bare-hand videos, yielding the HandSense dataset that improves contact estimation and occluded tracking.

  8. Under One Sun: Multi-Object Generative Perception of Materials and Illumination

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Factorizing video editing into semantic-token anchoring and motion-restoration pre-training produces strong zero-shot and SOTA open-source instruction-guided video edits without heavy external structural priors.

  9. Learning Stochastic Bridges for Video Object Removal via Video-to-Video Translation

    cs.CV 2026-01 conditional novelty 6.0 of 10

    A stochastic bridge model treats video object removal as video-to-video translation, starting from the source video rather than Gaussian noise, with adaptive mask modulation and a new benchmark.

  10. CustomX: Unified Character, Action, and Scene Customization in Video World Models

    cs.CV 2025-12 conditional novelty 6.0 of 10

    AniX generates controllable videos of a user-supplied character performing typed actions inside a user-supplied 3D scene by fine-tuning a pre-trained video generator on small locomotion datasets.

  11. O-DisCo-Edit: Object Distortion Control for Unified Realistic Video Editing

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A video editor trained on randomly distorted objects, then steered by adaptive noise at inference, is claimed to surpass dedicated and unified editors across eight tasks with far less training.

  12. EraserDiT: Fast Video Inpainting with Diffusion Transformer Model

    cs.CV 2025-06 conditional novelty 6.0 of 10

    EraserDiT fine-tunes LTX-Video into a masked-conditioned video inpainting model and adds a circular position-shift inference strategy, reporting large gains on DAVIS and HQVI.

  13. IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generation

    cs.CV 2025-06 reject novelty 6.0 of 10

    A diffusion video model that jointly uses HDR lighting, relit frames, and 3D point tracks to relight videos from text prompts.

  14. MiniMax-Remover: Taming Bad Noise Helps Video Object Removal

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A two-stage video object remover that removes text conditioning and uses minimax adversarial noise to achieve high-quality removal in 6 sampling steps without classifier-free guidance.

  15. OmniEraser: Remove Objects and Their Effects in Images with Paired Video-Frame Data

    cs.CV 2025-01 conditional novelty 6.0 of 10

    OmniEraser removes objects along with their shadows and reflections by conditioning a FLUX diffusion model on separate object and background latents, trained on a 134,281-sample video-derived dataset.

  16. MagicRoad: Semantic-Aware 3D Road Surface Reconstruction via Obstacle Inpainting

    cs.CV 2025-07 reject novelty 5.0 of 10

    MagicRoad combines video inpainting, semantic color harmonization, and 2D Gaussian surfels to reconstruct clean bird's eye view road surfaces from driving video.

  17. Coherent Video Inpainting Using Optical Flow-Guided Efficient Diffusion

    cs.CV 2024-12 conditional novelty 5.0 of 10

    FloED uses optical flow as extra motion guidance in a diffusion video inpainting model, with flow-warped latent interpolation and attention caching to cut inference cost while improving temporal consistency.

  18. Diff2DGS: Reliable Reconstruction of Occluded Surgical Scenes via 2D Gaussian Splatting

    cs.CV 2026-02 reject novelty 4.0 of 10

    Diff2DGS uses diffusion video inpainting plus 2D Gaussian Splatting to reconstruct occluded deformable surgical scenes, but its geometric superiority claim rests on a circular RAFT-depth evaluation.

Reference graph

Works this paper leans on

47 extracted references · 23 canonical work pages · cited by 18 Pith papers

  1. [1]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. arXiv preprint arXiv:2211.09800, 2022

    arXiv 2022

  2. [2]

    Panda-70m: Captioning 70m videos with multiple cross-modality teachers

    Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, and Sergey Tulyakov. Panda-70m: Captioning 70m videos with multiple cross-modality teachers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

    work page 2024

  3. [3]

    Control-a-video: Controllable text-to-video generation with diffusion models, 2023

    Weifeng Chen, Jie Wu, Pan Xie, Hefeng Wu, Jiashi Li, Xin Xia, Xuefeng Xiao, and Liang Lin. Control-a-video: Controllable text-to-video generation with diffusion models, 2023

    2023

  4. [4]

    Structure and content-guided video synthesis with diffusion models, 2023

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models, 2023

    2023

  5. [5]

    Hierarchical text-conditional image generation with clip latents, 2022

    Aditya Ramesh et al. Hierarchical text-conditional image generation with clip latents, 2022

    work page 2022

  6. [6]

    Bermano, Gal Chechik, and Daniel Cohen-Or

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image gen- eration using textual inversion, 2022

    work page 2022

  7. [7]

    Flow-edge guided video completion

    Chen Gao, Ayush Saraf, Jia-Bin Huang, and Johannes Kopf. Flow-edge guided video completion. In Proc. European Conference on Computer Vision (ECCV), 2020

    work page 2020

  8. [8]

    Preserve your own correlation: A noise prior for video diffusion models, 2024

    Songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming- Yu Liu, and Yogesh Balaji. Preserve your own correlation: A noise prior for video diffusion models, 2024

    work page 2024

Show all 47 references
  1. [9]

    Ad- vanced video inpainting using optical flow-guided efficient diffusion

    Bohai Gu, Hao Luo, Song Guo, and Peiran Dong. Ad- vanced video inpainting using optical flow-guided efficient diffusion. arXiv preprint arXiv:2412.00857, 2024

    2024 arXiv

  2. [10]

    Reuse and diffuse: Iterative denoising for text-to-video generation

    Jiaxi Gu, Shicong Wang, Haoyu Zhao, Tianyi Lu, Xing Zhang, Zuxuan Wu, Songcen Xu, Wei Zhang, Yu-Gang Jiang, and Hang Xu. Reuse and diffuse: Iterative denoising for text-to-video generation

  3. [11]

    Animatediff: Animate your personalized text-to- image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to- image diffusion models without specific tuning. Interna- tional Conference on Learning Representations, 2024

    2024

  4. [12]

    Prompt-to-prompt im- age editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt im- age editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022

    2022 arXiv

  5. [13]

    Kingma, Ben Poole, Mohammad Norouzi, David J

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Fleet, and Tim Sali- mans. Imagen video: High definition video generation with diffusion models, 2022

    2022

  6. [14]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. arXiv preprint arxiv:2006.11239, 2020

    2006 arXiv

  7. [15]

    Video dif- fusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models. arXiv:2204.03458, 2022

    2022 arXiv

  8. [16]

    Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion, 2024

    Xuan Ju, Xian Liu, Xintao Wang, Yuxuan Bian, Ying Shan, and Qiang Xu. Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion, 2024. Figure 10. Texture quality comparison between DiffuEraser and Propainter. Figure 11. Texture quality comparison b...

    2024

  9. [17]

    Video diffusion models are strong video inpainter, 2024

    Minhyeok Lee, Suhwan Cho, Chajin Shin, Jungho Lee, Sunghun Yang, and Sangyoun Lee. Video diffusion models are strong video inpainter, 2024

    2024

  10. [18]

    Towards an end-to-end framework for flow-guided video inpainting

    Zhen Li, Cheng-Ze Lu, Jianhua Qin, Chun-Le Guo, and Ming-Ming Cheng. Towards an end-to-end framework for flow-guided video inpainting. In IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), 2022

    2022

  11. [19]

    Magicedit: High-fidelity and temporally coherent video editing

    Jun Hao Liew, Hanshu Yan, Jianfeng Zhang, Zhongcong Xu, and Jiashi Feng. Magicedit: High-fidelity and temporally coherent video editing. In arXiv, 2023

    2023

  12. [20]

    Fuseformer: Fusing fine-grained information in transformers for video inpainting

    Rui Liu, Hanming Deng, Yangyi Huang, Xiaoyu Shi, Lewei Lu, Wenxiu Sun, Xiaogang Wang, Jifeng Dai, and Hong- sheng Li. Fuseformer: Fusing fine-grained information in transformers for video inpainting. In International Confer- ence on Computer Vision (ICCV), 2021

    2021

  13. [21]

    Video-p2p: Video editing with cross-attention control, 2023

    Shaoteng Liu, Yuechen Zhang, Wenbo Li, Zhe Lin, and Jiaya Jia. Video-p2p: Video editing with cross-attention control, 2023

    2023

  14. [22]

    Null-text inversion for editing real images using guided diffusion models

    Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. arXiv preprint arXiv:2211.09794, 2022

    2022 arXiv

  15. [23]

    Dreamix: Video diffusion models are general video editors, 2023

    Eyal Molad, Eliahu Horwitz, Dani Valevski, Alex Rav Acha, Yossi Matias, Yael Pritch, Yaniv Leviathan, and Yedid Hoshen. Dreamix: Video diffusion models are general video editors, 2023

    2023

  16. [24]

    T2i- adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, Ying Shan, and Xiaohu Qie. T2i- adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. arXiv preprint arXiv:2302.08453, 2023

    2023 arXiv

  17. [25]

    Fatezero: Fusing attentions for zero-shot text-based video editing

    Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. Fatezero: Fusing attentions for zero-shot text-based video editing. arXiv:2303.09535, 2023

    2023 arXiv

  18. [26]

    Deep learning-based image and video inpainting: A survey, 2024

    Weize Quan, Jiaxi Chen, Yanli Liu, Dong-Ming Yan, and Pe- ter Wonka. Deep learning-based image and video inpainting: A survey, 2024

    2024

  19. [27]

    High-resolution image syn- thesis with latent diffusion models, 2021

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2021

    2021

  20. [28]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In arXiv preprint arxiv:2208.12242, 2022

    2022 arXiv

  21. [29]

    Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion mod...

    2022

  22. [30]

    Bivdiff: A training-free framework for general-purpose video synthesis via bridging image and video diffusion models

    Fengyuan Shi, Jiaxi Gu, Hang Xu, Songcen Xu, Wei Zhang, and Limin Wang. Bivdiff: A training-free framework for general-purpose video synthesis via bridging image and video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (C...

    2024

  23. [31]

    Make-a-video: Text-to-video generation without text-video data, 2022

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Make-a-video: Text-to-video generation without text-video data, 2022

    2022

  24. [32]

    Weiss, Niru Mah- eswaranathan, and Surya Ganguli

    Jascha Sohl-Dickstein, Eric A. Weiss, Niru Mah- eswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics, 2015

    2015

  25. [33]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. arXiv:2010.02502, October 2020

    2010 arXiv

  26. [34]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. In International Conference on Learning Represen- tations, 2021

    2021

  27. [35]

    Phased consistency model

    Fu-Yun Wang, Zhaoyang Huang, Alexander William Bergman, Dazhong Shen, Peng Gao, Michael Lingel- bach, Keqiang Sun, Weikang Bian, Guanglu Song, Yu Liu, et al. Phased consistency model. arXiv preprint arXiv:2405.18407, 2024

    2024 arXiv

  28. [36]

    Videocomposer: Compositional video synthesis with motion controllability, 2023

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Ji- uniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jin- gren Zhou. Videocomposer: Compositional video synthesis with motion controllability, 2023

    2023

  29. [37]

    Towards language-driven video inpainting via multimodal large language models

    Jianzong Wu, Xiangtai Li, Chenyang Si, Shangchen Zhou, Jingkang Yang, Jiangning Zhang, Yining Li, Kai Chen, Yun- hai Tong, Ziwei Liu, et al. Towards language-driven video inpainting via multimodal large language models. arXiv preprint arXiv:2401.10226, 2024

    2024 arXiv

  30. [38]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference...

    2023

  31. [39]

    Make-your-video: Customized video generation using textual and structural guidance.arXiv preprint arXiv:2306.00943, 2023

    Jinbo Xing, Menghan Xia, Yuxin Liu, Yuechen Zhang, Yong Zhang, Yingqing He, Hanyuan Liu, Haoxin Chen, Xiaodong Cun, Xintao Wang, et al. Make-your-video: Customized video generation using textual and structural guidance.arXiv preprint arXiv:2306.00943, 2023

    2023 arXiv

  32. [40]

    Learning joint spatial-temporal transformations for video inpainting

    Yanhong Zeng, Jianlong Fu, and Hongyang Chao. Learning joint spatial-temporal transformations for video inpainting. In The Proceedings of the European Conference on Com- puter Vision (ECCV), 2020

    2020

  33. [41]

    Flow-guided transformer for video inpainting

    Kaidong Zhang, Jingjing Fu, and Dong Liu. Flow-guided transformer for video inpainting. In European Conference on Computer Vision, pages 74–90. Springer, 2022

    2022

  34. [42]

    Inertia-guided flow completion and style fusion for video inpainting

    Kaidong Zhang, Jingjing Fu, and Dong Liu. Inertia-guided flow completion and style fusion for video inpainting. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR) , pages 5982–5991, June 2022

    2022

  35. [43]

    Adding conditional control to text-to-image diffusion models, 2023

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models, 2023

    2023

  36. [44]

    Controlvideo: Training-free controllable text-to-video generation

    Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian. Controlvideo: Training-free controllable text-to-video generation. arXiv preprint arXiv:2305.13077, 2023

    2023 arXiv

  37. [45]

    Avid: Any-length video inpainting with diffusion model

    Zhixing Zhang, Bichen Wu, Xiaoyan Wang, Yaqiao Luo, Luxin Zhang, Yinan Zhao, Peter Vajda, Dimitris Metaxas, and Licheng Yu. Avid: Any-length video inpainting with diffusion model. arXiv preprint arXiv:2312.03816, 2023

    2023 arXiv

  38. [46]

    ProPainter: Improving propagation and transformer for video inpainting

    Shangchen Zhou, Chongyi Li, Kelvin C.K Chan, and Chen Change Loy. ProPainter: Improving propagation and transformer for video inpainting. In Proceedings of IEEE International Conference on Computer Vision (ICCV), 2023

    2023

  39. [47]

    Cococo: Improving text-guided video inpainting for bet- ter consistency, controllability and compatibility

    Bojia Zi, Shihao Zhao, Xianbiao Qi, Jianan Wang, Yukai Shi, Qianyu Chen, Bin Liang, Kam-Fai Wong, and Lei Zhang. Cococo: Improving text-guided video inpainting for bet- ter consistency, controllability and compatibility. ArXiv, abs/2403.12035, 2024

    2024 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.