Pith. sign in

REVIEW 4 major objections 5 minor 6 cited by

Coherent Video Inpainting Using Optical Flow-Guided Efficient Diffusion

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Flow-guided diffusion does video inpainting faster, with better coherence

desk verdict FloED is a plausible and genuinely new system for text-guided video inpainting, but its headline claims are not yet supported because the anchor-frame prior alone beats all baselines and the efficiency numbers omit that anchor cost. read the letter →

arxiv 2412.00857 v3 pith:GKR5RKH4 submitted 2024-12-01 cs.CV

Bohai Gu , Hao Luo , Song Guo , Peiran Dong , Qihua Zhou This is my paper · ORCID
classification cs.CV
keywords videoinpaintingdiffusionmodelsopticalflowtemporalconsistencylatentinterpolationobjectremovalbackgroundrestorationefficientsampling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that video inpainting with diffusion models can be made both more temporally coherent and cheaper by explicitly feeding optical flow into the denoising network. It proposes FloED, a dual-branch architecture whose flow branch first completes corrupted motion fields, then injects them through multi-scale adapters into the main inpainting UNet. To cut cost, FloED warps predicted clean latents along the completed flow so that only half the frames are denoised at each early step, plus a cached flow attention that avoids recomputing keys and values. On a new 100-video benchmark spanning background restoration and object removal, FloED reports better PSNR, SSIM, VFID, warping error, temporal consistency, and text alignment than VideoComposer, CoCoCo, and DiffuEraser, with lower per-frame denoising time. If these results hold, text-guided video inpainting can get better coherence and efficiency at the same time.

What carries the argument

The load-bearing mechanism is flow-guided latent interpolation performed in the predicted clean latent space $\hat{z}_0$, not on the noise estimate; at each early step, even-indexed latents are denoised normally and odd-indexed latents are obtained by bi-directional optical-flow warping of the neighboring predicted $\hat{z}_0$, then the roles alternate, halving denoising work per step for the first S steps. The flow attention cache complements this by having the time-agnostic flow branch complete the corrupted flow once at the first step, then storing the keys and values of the multi-scale flow adapters for all later steps, so adding optical flow guidance costs almost nothing at inference.

What would settle it

Measure, per denoising step, the distance between the flow-warped predicted $\hat{z}_0$ and the normally denoised $\hat{z}_0$ on videos with large motion and occlusions; if that distance grows quickly with flow magnitude or if setting S above 5 causes a sharp quality drop on such videos, the load-bearing assumption behind the speed-up is broken.

Watch

Extended reading notes

Core claim

FloED is a text-guided video inpainting framework built on a Stable Diffusion inpainting backbone with motion modules fine-tuned in two stages. Its central discovery is that corrupted optical flow, estimated by RAFT and completed by a time-agnostic flow branch, provides exactly the motion guidance diffusion-based inpainting lacks: multi-scale flow adapters placed between text cross-attention and motion modules adjust the latents so synthesized content follows the scene's movement and lighting. The efficiency claim rests on two observations: adjacent frame latents share similar motion patterns, and early denoising fixes global structure, so in the first S denoising steps a frame's latent can be replaced by flow-warping the predicted clean latent $\hat{z}_0$ of its neighbor, halving the number of latents denoised per step; and since the flow branch is time-agnostic, its keys and values can be computed once and cached. An anchor frame, inpainted by a strong text-to-image model, is concatenated to the noisy latents for extra texture guidance. The paper's reported numbers—PSNR 29.17, SSIM 0.9441, VFID 0.118, Ewarp 2.83, TC 0.994, TA 22.49 on its benchmark—are all better than the three compared open diffusion baselines, and per-frame denoising time is lower at every tested resolution.

Load-bearing premise

The speed-up rests on the assumption that, during the first few denoising steps, a frame's latent can be replaced by the flow-warped predicted clean latent of its neighbor without noticeable quality loss, which works only if adjacent latents really share similar motion and occlusion does not break the warp.

Editorial extensions

If this is right

  • If FloED's numbers hold, a diffusion inpainting model can use optical flow as cheap guidance: the flow branch runs once and the flow adapter's keys and values are cached for the remaining denoising steps.
  • The latent interpolation trick—warping predicted clean latents rather than noise estimates—gives a training-free speed-up that the paper says transfers to other diffusion-based video inpainting models such as CoCoCo.
  • The anchor-frame strategy means video inpainting quality no longer has to wait for video models to catch up with image inpainting; a strong text-to-image model can set the texture for the sequence.
  • Temporal-consistency metrics (Ewarp, TC) improve together with per-frame quality metrics, so coherence is not bought at the expense of per-frame fidelity.
  • Because the flow branch is time-agnostic, the completed flow is stable across sampling steps, which is what makes the cache and the one-time flow completion valid.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One extension the paper leaves implicit: the same flow-guided latent interpolation could be applied to the sampling of any video diffusion model, not only inpainting, whenever dense optical flow between frames is available; the paper only demoes it on its own and CoCoCo.
  • The speed-up depends on the choice S=5, tuned on the paper's own benchmark; on videos with large motion or strong occlusion, the safe number of interpolation steps may be smaller, and the optimal S could be chosen adaptively from flow confidence rather than fixed.
  • A testable prediction: the gap between flow-warped latents and fully denoised latents, measured in latent space after step S, should grow with motion magnitude and occlusion area; if a dataset of such hard videos is built, FloED's efficiency advantage should shrink.
  • The proposed benchmark of 100 real videos paired with prompts is itself a contribution: it gives future diffusion-based video inpainting methods a common test bed, and its PSNR/SSIM numbers could be extended to include confidence intervals across the 50+50 split.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes FloED, a text-guided video inpainting framework that combines a Stable Diffusion inpainting backbone with a time-agnostic optical-flow completion branch, multi-scale flow adapters, an anchor-frame strategy that injects an externally inpainted frame, and two training-free inference accelerators (flow-guided latent interpolation and a flow attention cache). The authors report state-of-the-art results on a self-created 100-video benchmark covering background restoration and object removal, comparing against VideoComposer, CoCoCo, and DiffuEraser on quality metrics (PSNR, SSIM, VFID, warping error, temporal consistency, text alignment) and runtime.

Significance. If the claims are substantiated, FloED would be a meaningful advance for text-guided video inpainting: the dual-branch flow guidance is architecturally plausible, the latent-interpolation and attention-cache mechanisms are training-free and could transfer to other diffusion pipelines, and the paper includes a user study plus component ablations. The main value is in demonstrating that optical-flow priors can be integrated into diffusion video inpainting at low overhead. However, the headline quality and efficiency claims are currently confounded by the anchor-frame prior and by incomplete timing methodology, so the significance is conditional on revising the experimental comparison.

major comments (4)
  1. [§5.2, Table 1 vs. §5.3, Table 2] The quality comparison against baselines is confounded by the anchor-frame strategy. Table 2 shows that adding only the anchor frame (FA=✗, AF=✓) raises PSNR from 21.30 to 25.34, which already exceeds the best baseline in Table 1 (DiffuEraser at 24.23). Since no baseline uses this external T2I inpainting prior, the margins in Table 1 are not attributable solely to the proposed flow adapter, motion-module fine-tuning, or latent interpolation. The authors should either augment all baselines with the same anchor-frame mechanism (with equivalent compute) or report FloED's quality without the anchor, and phrase the SOTA claim accordingly.
  2. [§5.3, Table 4] The efficiency comparison omits the anchor-frame cost: the anchor frame is generated by a separate pretrained image-inpainting model in §4.3, and that inference time is not included in the reported 'average time per frame'. Additionally, Table 4 lists DiffuEraser as slower at 256×256 (1.0336 s) than at 512×512 (0.4176 s), which is implausible under matched settings and suggests that step counts, CFG scales, or batching were not held constant. Please report end-to-end per-video time for all methods, including anchor generation, and clarify the exact inference protocol for each row.
  3. [§4.4, §5.3, Fig. 7] The speed-up step S=5 is selected empirically on the same 100-video benchmark that is used for the final evaluation (Fig. 7 and §5.1). Because S directly trades off quality against denoising time, tuning it on the test set biases the reported quality metrics in FloED's favor. The authors should tune S on a separate validation split and report the held-out quality--time tradeoff, or show the results across a range of S on the benchmark without selecting the best one.
  4. [§5.2, general experimental methodology] All headline metrics come from a self-created benchmark of 100 videos, with no error bars, confidence intervals, or significance tests for the differences in Table 1, and only three diffusion baselines are compared. Given the very large reported margins, it is important to show per-video score distributions or statistical testing to establish that the differences are not driven by a few outliers. At minimum, report the standard deviation or interquartile range for each metric.
minor comments (5)
  1. [§1, introduction] The introduction contains duplicated blocks of text: the paragraph beginning 'we utilize an anchor frame strategy' and the contributions list appear twice in the provided manuscript, which impairs readability and suggests an editing error.
  2. [§5.2, quantitative comparisons] The text says 'For OM tasks, since ground truth data is unavailable' where 'OM' should be 'OR' (object removal); please correct the abbreviation.
  3. [§5.3, Table 3] The table entry '6nd„25th' is a typesetting artifact; it should read '6th–25th'.
  4. [§4.1, network overview] Reference [1] is cited as the pretrained backbone but is listed as 'stable-diffusion-2-depth', while the text says 'Stable Diffusion Inpainting backbone'; please clarify the exact backbone checkpoint and its relationship to the image-inpainting model used for the anchor frame (reference [2]).
  5. [§7, references] Reference [14] (DiffuEraser) is dated 2025, after the initial arXiv submission of this paper; please verify that the version compared is the one described and that no newer version changes the comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper's claims are empirical system comparisons, not derivations, and no output is equivalent to its input by construction.

full rationale

This is a systems/empirical paper: the central claims are that the proposed flow-guided architecture, anchor-frame prior, latent interpolation, and flow-attention cache improve quality and efficiency relative to diffusion baselines. These claims are supported by measured metrics on a newly constructed benchmark, not by a derivation chain in which a quantity is defined in terms of the result it is supposed to predict. The flow completion branch, flow adapters, anchor-frame concatenation, and latent interpolation are all stated as architectural components built on external pretrained backbones (Stable Diffusion, AnimateDiff, RAFT), and the reported gains are measured outcomes. The only self-citation in the paper, reference [5] used for the flow-warping error metric in Sec. 5.2, is a metric reference and is not load-bearing for the central contribution. The tuning of the speed-up step S=5 on the same benchmark and the potential confounding role of the anchor-frame prior are legitimate experimental-design and external-validity concerns, but they do not make the reported numbers equivalent to the inputs by construction. No equation-level circularity, fitted-parameter-renamed-as-prediction, or author-imported uniqueness argument is present, so no circular step is identified.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The listed assumptions are all load-bearing for the central claims. If the flow branch cannot complete corrupted flow reliably, or if flow-warped latent interpolation degrades quality outside the first five steps, the claimed coherence and efficiency advantages would not hold. The free parameters S and lambda are set on the authors' benchmark without sensitivity analysis, so the headline numbers are conditional on these choices.

free parameters (3)
  • Speed-up step S (latent interpolation horizon) = 5
    Empirically chosen as the number of early denoising steps using latent interpolation; Fig. 7 shows performance drops sharply when S exceeds 5, so the value is tuned on the authors' benchmark.
  • Flow loss weight lambda = 0.1
    Set in the second-stage training loss (Eq. 5); no sensitivity analysis is provided.
  • DDIM denoising steps = 25
    Used for all comparisons; not derived and directly affects the reported per-frame times and quality trade-off.
assumptions (6)
  • domain assumption Ground truth flow F used in L_flow is available during training and is computed from clean frames.
    Eq. 5 defines L_flow = ||F_hat - F||_1 and calls F the ground truth flow, but the source of F is not specified in the text.
  • domain assumption Completed optical flow gives reliable motion guidance in latent space for diffusion video inpainting.
    Sec. 1 hypothesizes that motion guidance mitigates temporal disharmony; the flow completion branch and adapters in Sec. 4.2 depend on this premise.
  • domain assumption Adjacent latent features share similar motion patterns, so warping can stand in for denoising a frame.
    Sec. 4.4 cites observation [31] and restricts warping to adjacent frames; this is the basis of the training-free speed-up.
  • domain assumption Early diffusion steps determine global structure, so skipping denoising via warping only in the first S steps preserves quality.
    Sec. 4.4 cites [12] and Fig. 7 shows quality drops when S exceeds 5; the horizon S is chosen empirically.
  • domain assumption A single anchor frame inpainted by a text-to-image model provides useful texture guidance for the whole video sequence.
    Sec. 4.3 introduces the anchor frame strategy; no ablation on anchor choice or on the T2I model is reported.
  • domain assumption The self-created 100-video benchmark is representative enough to support a state-of-the-art claim.
    Sec. 5.1 says the authors manually selected videos for diversity and created masks; no external benchmark is used in Table 1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Coherent Video Inpainting Using Optical Flow-Guided Efficient Diffusion." pith.science (2026). https://pith.science/paper/GKR5RKH4

@misc{pith2026241200857,
  author       = {Pith},
  title        = {Pith review of: Coherent Video Inpainting Using Optical Flow-Guided Efficient Diffusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GKR5RKH4}},
  note         = {Machine review of arXiv:2412.00857}
}
read the original abstract

The text-guided video inpainting technique has significantly improved the performance of content generation applications. A recent family for these improvements uses diffusion models, which have become essential for achieving high-quality video inpainting results, yet they still face performance bottlenecks in temporal consistency and computational efficiency. This motivates us to propose a new video inpainting framework using optical Flow-guided Efficient Diffusion (FloED) for higher video coherence. Specifically, FloED employs a dual-branch architecture, where the time-agnostic flow branch restores corrupted flow first, and the multi-scale flow adapters provide motion guidance to the main inpainting branch. Besides, a training-free latent interpolation method is proposed to accelerate the multi-step denoising process using flow warping. With the flow attention cache mechanism, FLoED efficiently reduces the computational cost of incorporating optical flow. Extensive experiments on background restoration and object removal tasks show that FloED outperforms state-of-the-art diffusion-based methods in both quality and efficiency. Our codes and models will be made publicly available.

Figures

Figures reproduced from arXiv: 2412.00857 by the authors.

Figure 1
Figure 1. Comparison. 079 we utilize an anchor frame strategy to enhance the quality 080 of video inpainting outcomes. 081 Currently, there is no comprehensive benchmark for 082 evaluating diffusion-based generative approaches in video 083 inpainting. This deficiency presents a substantial challenge, 084 as it limits the ability to rigorously assess and compare the 085 efficacy of various inpainting methodologies. To bridge t… view at source ↗
Figure 2
Figure 2. Overview of FloED. FloED employs a dual-branch architecture implemented through a two-stage training approach. In the first [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of flow-guided latent interpolation (left) and warping operation (right) during the denoising process. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative comparisons. We compare FloED against diffusion-based SOTAs on BR and OR tasks. BR OR PSNR ↑ SSIM ↑ VFID ↓ Ewarp↓ TC ↑ TA ↑ VC 22.81 0.8614 0.193 3.43 0.987 21.30 CoCoCo 23.08 0.8694 0.165 3.73 0.991 21.97 DiffuEraser 24.23 0.8583 0.218 2.98 0.984 19.19 Our…
Figure 5
Figure 5. Figure 5: Optical flow related ablation studies. (F) ablation study demonstrates FloED conducts flow warping at noise [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: We conduct reliable User Study with randomized order to assess inpainting outcomes of 4 methods. study in which 15 annotators evaluated the inpainting re￾sults across both BR and OR tasks (100 videos), assessing temporal consistency, text alignment, and context compati…
Figure 7
Figure 7. Figure 7: Speeding steps study. Performance markedly deterio￾rates when the acceleration step surpasses five. Flow Branch Latent Interpolation Flow Cache Average time per frame(s) – – – 0.1287 1 st step – – 0.1491 1 st step 2 nd „ 6 th – 0.1342 (↓ 9.9%) 1 st step 2 nd „ 6 th 6 n…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. From Draft to Draft-Free: One-Step Video Object Removal via Privileged Distillation and Fast Planting

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A one-step, draft-free video object removal model trained by distilling a ground-truth-conditioned teacher reaches comparable or better quality than multi-step diffusion methods while running in about 1 second.

  2. Glove2Hand: Synthesizing Natural Hand-Object Interaction from Multi-Modal Sensing Gloves

    cs.CV 2026-03 conditional novelty 6.0 of 10

    A 3D-Gaussian-plus-diffusion pipeline translates multi-modal glove HOI videos into photorealistic bare-hand videos, yielding the HandSense dataset that improves contact estimation and occluded tracking.

  3. ROSE: Remove Objects with Side Effects in Videos

    cs.CV 2025-08 conditional novelty 6.0 of 10

    A video inpainting model trained on 3D-rendered pairs removes objects together with their shadows, reflections, and other side effects, plus a new benchmark.

  4. MiniMax-Remover: Taming Bad Noise Helps Video Object Removal

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A two-stage video object remover that removes text conditioning and uses minimax adversarial noise to achieve high-quality removal in 6 sampling steps without classifier-free guidance.

  5. Diff2DGS: Reliable Reconstruction of Occluded Surgical Scenes via 2D Gaussian Splatting

    cs.CV 2026-02 reject novelty 4.0 of 10

    Diff2DGS uses diffusion video inpainting plus 2D Gaussian Splatting to reconstruct occluded deformable surgical scenes, but its geometric superiority claim rests on a circular RAFT-depth evaluation.

  6. DiffuEraser: A Diffusion Model for Video Inpainting

    cs.CV 2025-01 reject novelty 4.0 of 10

    DiffuEraser is a stable-diffusion video inpainting model that injects ProPainter priors via DDIM inversion and expands temporal receptive fields for long-sequence consistency.

Reference graph

Works this paper leans on

36 extracted references · 20 canonical work pages · cited by 6 Pith papers

  1. [1]

    https://huggingface

    Stable Diffusion v2 , 2022. https://huggingface. co/stabilityai/stable-diffusion-2-depth . 3, 4

    work page 2022

  2. [2]

    https://github.com/ compvis/stable-diffusion

    Stable Diffusion v1.5 , 2022. https://github.com/ compvis/stable-diffusion. 4

    work page 2022

  3. [3]

    Videocrafter1: Open diffusion models for high-quality video generation

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter1: Open diffusion models for high-quality video generation. CoRR, abs/2310.19512, 2023. 3

    arXiv

  4. [4]

    Video inpainting with short-term windows: Application to object removal and error concealment

    Mounira Ebdelli, Olivier Le Meur, and Christine Guillemot. Video inpainting with short-term windows: Application to object removal and error concealment. IEEE Trans. Image Process., 2015. 2

    work page 2015

  5. [5]

    Two birds, one stone: A unified framework for joint learning of image and video style transfers

    Bohai Gu, Heng Fan, and Libo Zhang. Two birds, one stone: A unified framework for joint learning of image and video style transfers. In IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pages 23488–23497. IEEE, 2023. 7

    work page 2023

  6. [6]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. In The Twelfth International Conference on Learning Representa- tions, ICLR 2024, Vienna, Austria, May 7-11, 2024 . Open- Review.net, 2024. 2, 3, 4

    work page 2024

  7. [7]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV , USA, June 27-30, 2016, pages 770–778. IEEE Computer Society, 2016. 4

    work page 2016

  8. [8]

    Clipscore: A reference-free evaluation met- ric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation met- ric for image captioning. CoRR, abs/2104.08718, 2021. 7

    arXiv 2021

Show all 36 references
  1. [9]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. In NeurIPS, 2020. 2, 3, 4, 6

    2020

  2. [10]

    Brushnet: A plug-and-play image inpaint- ing model with decomposed dual-branch diffusion

    Xuan Ju, Xian Liu, Xintao Wang, Yuxuan Bian, Ying Shan, and Qiang Xu. Brushnet: A plug-and-play image inpaint- ing model with decomposed dual-branch diffusion. CoRR, abs/2403.06976, 2024. 2, 3

    2024 arXiv

  3. [11]

    Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick. Segment anything, 2023. 6

    2023

  4. [12]

    Diffusion models already have A semantic latent space

    Mingi Kwon, Jaeseok Jeong, and Youngjung Uh. Diffusion models already have A semantic latent space. InICLR, 2023. 5, 7

    2023

  5. [13]

    Open-sora-plan, 2024

    PKU-Yuan Lab and Tuzhan AI etc. Open-sora-plan, 2024. 5

    2024

  6. [14]

    Dif- fueraser: A diffusion model for video inpainting

    Xiaowen Li, Haolan Xue, Peiran Ren, and Liefeng Bo. Dif- fueraser: A diffusion model for video inpainting. CoRR, abs/2501.10018, 2025. 2, 3, 6

    2025 arXiv

  7. [15]

    Towards an end-to-end framework for flow- guided video inpainting

    Zhen Li, Chengze Lu, Jianhua Qin, Chun-Le Guo, and Ming- Ming Cheng. Towards an end-to-end framework for flow- guided video inpainting. In CVPR, 2022. 2, 3

    2022

  8. [16]

    Video-chatgpt: Towards detailed video understanding via large vision and language models

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fa- had Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. In Pro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), 2024. 6

    2024

  9. [17]

    Video inpainting of complex scenes

    Alasdair Newson, Andr ´es Almansa, Matthieu Fradet, Yann Gousseau, and Patrick P ´erez. Video inpainting of complex scenes. CoRR, 2015. 2

    2015

  10. [18]

    Gross, and Alexander Sorkine- Hornung

    Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus H. Gross, and Alexander Sorkine- Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In CVPR, 2016. 3

    2016

  11. [19]

    SDXL: improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. SDXL: improving latent diffusion models for high-resolution image synthesis. CoRR, abs/2307.01952,

    arXiv

  12. [20]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2022. 3

    2022

  13. [21]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In ICLR, 2021. 2, 3

    2021

  14. [22]

    RAFT: recurrent all-pairs field transforms for optical flow (extended abstract)

    Zachary Teed and Jia Deng. RAFT: recurrent all-pairs field transforms for optical flow (extended abstract). In IJCAI,

  15. [23]

    Video-to-video synthesis

    Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Nikolai Yakovenko, Andrew Tao, Jan Kautz, and Bryan Catanzaro. Video-to-video synthesis. In NeurIPS, 2018. 7

    2018

  16. [24]

    Videocomposer: Compositional video synthesis with motion controllability

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Ji- uniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jin- gren Zhou. Videocomposer: Compositional video synthesis with motion controllability. CoRR, abs/2306.02018, 2023. 6

    2023 arXiv

  17. [25]

    Bovik, Hamid R

    Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., 2004. 7

    2004

  18. [26]

    Bovik, Hamid R

    Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., 13(4): 600–612, 2004. 7

    2004

  19. [27]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference...

    2023

  20. [28]

    Smartbrush: Text and shape guided object inpainting with diffusion model

    Shaoan Xie, Zhifei Zhang, Zhe Lin, Tobias Hinz, and Kun Zhang. Smartbrush: Text and shape guided object inpainting with diffusion model. In IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, CVPR 2023, Vancou- ver, BC, Canada, June 17-24, 2023 , pages 22428–2243...

    2023

  21. [29]

    Price, Scott Cohen, and Thomas S

    Ning Xu, Linjie Yang, Yuchen Fan, Jianchao Yang, Dingcheng Yue, Yuchen Liang, Brian L. Price, Scott Cohen, and Thomas S. Huang. Youtube-vos: Sequence-to-sequence video object segmentation. In ECCV, 2018. 3

    2018

  22. [30]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. CoRR, abs/2308.06721, 2023. 3, 4

    2023 arXiv

  23. [31]

    Flow-guided transformer for video inpainting

    Kaidong Zhang, Jingjing Fu, and Dong Liu. Flow-guided transformer for video inpainting. In ECCV, 2022. 2, 3, 5

    2022

  24. [32]

    Adding condi- tional control to text-to-image diffusion models

    Lvmin Zhang and Maneesh Agrawala. Adding condi- tional control to text-to-image diffusion models. CoRR, abs/2302.05543, 2023. 3

    2023 arXiv

  25. [33]

    Metaxas, and Licheng Yu

    Zhixing Zhang, Bichen Wu, Xiaoyan Wang, Yaqiao Luo, Luxin Zhang, Yinan Zhao, Peter Vajda, Dimitris N. Metaxas, and Licheng Yu. A VID: any-length video inpainting with diffusion model. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, US...

    2024

  26. [34]

    Shangchen Zhou, Chongyi Li, Kelvin C. K. Chan, and Chen Change Loy. Propainter: Improving propagation and transformer for video inpainting. CoRR, abs/2309.03897,

    arXiv

  27. [35]

    A task is worth one word: Learning with task prompts for high-quality versatile image inpainting

    Junhao Zhuang, Yanhong Zeng, Wenran Liu, Chun Yuan, and Kai Chen. A task is worth one word: Learning with task prompts for high-quality versatile image inpainting. CoRR, abs/2312.03594, 2023. 2, 3

    2023 arXiv

  28. [36]

    Cococo: Improving text-guided video inpainting for bet- ter consistency, controllability and compatibility

    Bojia Zi, Shihao Zhao, Xianbiao Qi, Jianan Wang, Yukai Shi, Qianyu Chen, Bin Liang, Kam-Fai Wong, and Lei Zhang. Cococo: Improving text-guided video inpainting for bet- ter consistency, controllability and compatibility. CoRR, abs/2403.12035, 2024. 2, 3, 6, 8

    2024 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.