Pith. sign in

REVIEW 3 major objections 5 minor 11 cited by

Video object removal can also erase the object's shadows and reflections.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A video inpainting model trained on 3D-rendered pairs removes objects together with their shadows, reflections, and other side effects, plus a new benchmark.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Genuinely useful synthetic dataset, benchmark, and training framework for removing objects with side effects in video; but the real-world generalization claim is unsupported and Table 3 contradicts the abstract's superiority claim. the 3 major comments →

arxiv 2508.18633 v1 pith:BVWMKCYY submitted 2025-08-26 cs.CV cs.AIcs.LG

ROSE: Remove Objects with Side Effects in Videos

Chenxuan Miao , Yutong Feng , Jianshu Zeng , Zixiang Gao , Hantang Liu , Yunfeng Yan , Donglian Qi , Xi Chen
show 2 more authors
Bin Wang Hengshuang Zhao
This is my paper
classification cs.CV cs.AIcs.LG
keywords video object removalside effect removaldiffusion transformersynthetic paired datavideo inpaintingdifference mask supervisionobject shadows and reflectionsROSE-Bench
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that removing an object from a video should mean removing everything the object did to the scene—its shadow, reflection, glow, mirror image, or translucency trace—not just the object pixels. Existing video inpainting models are trained on artificial pairs that hide or paste objects, so they never learn these physical after-effects. ROSE instead trains on thousands of before/after video pairs produced by a 3D rendering engine, where the same scene is rendered once with the object and once without, giving exact masks and pixel-aligned ground truth. On top of a diffusion-transformer inpainting backbone, it feeds the entire original video into the model as reference and adds a head that predicts the binary difference between the two rendered videos, forcing the model to localize every object-correlated region. The authors also build ROSE-Bench, a benchmark with synthetic, realistic-paired, and real-unpaired test sets covering five side-effect categories, and report that ROSE outperforms prior video inpainting baselines and generalizes to real footage.

Core claim

The central claim is that side effects—not just the object—should be part of the target in video object removal, and that a model can learn to erase them from synthetic supervision. ROSE defines five physical side-effect categories (shadow, reflection, light source, mirror, translucent) and builds a pipeline that renders paired videos with the object toggled on and off, with per-frame masks and aligned camera motion. It then uses a diffusion transformer conditioned on the whole input video (rather than the masked remainder) so attention can link object pixels to their environmental traces, and an auxiliary difference-mask predictor supervised by the thresholded L2 difference between the pair

What carries the argument

The engine is the 3D-rendered paired-data pipeline: toggling an object's visibility in the renderer yields pixel-aligned before/after videos and exact masks, converting a task that normally lacks supervision into a supervised one. On this data, ROSE's reference-based erasing feeds the full video, not a masked product, into a diffusion transformer so the same object pixels act as a pointer to correlated regions; the difference-mask predictor, attached to hidden transformer features and trained with MSE on the thresholded pair difference, supplies an explicit localization signal. Mask augmentation (point, bounding box, dilation, erosion) makes the training robust to the imperfect masks users s

Load-bearing premise

The load-bearing premise is that side effects rendered by a 3D engine—how light, mirrors, water, and translucent materials behave—are close enough to real-world physical behavior that a model trained exclusively on such synthetic pairs will erase real shadows, reflections, and lighting changes without additional real paired data.

What would settle it

Take a fixed camera, film a scene with a physical object present, remove the object without changing lighting, and film again; run ROSE on the first video with the object masked. If the model's side-effect removal on these real before/after pairs is no better than a model trained on classical mask-and-inpaint pairs, the synthetic-proxy assumption fails. The same test would also reveal whether the difference-mask supervision transfers to real sensor noise.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A model trained solely on synthetic rendered pairs can erase objects together with shadows, reflections, light-source changes, mirror appearances, and translucency, according to the paper's quantitative and qualitative results.
  • Feeding the whole video, not just the masked remainder, lets the inpainting model use the object region as reference and improves side-effect localization.
  • Explicitly predicting the difference mask between paired videos gives the diffusion model a training signal that encourages it to repaint not only the object mask but all correlated pixels.
  • ROSE-Bench supplies a common protocol, with synthetic paired, realistic paired, and real unpaired subsets, for future comparisons of side-effect removal.
  • The method generalizes to real-world videos on the paper's unpaired benchmark, as measured by VBench-style metrics.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: The same two-element recipe—synthetic before/after pairs plus a difference-mask head—could be applied to other invisible object traces, such as caustics, wetness marks, dust displacement, or dynamic weather, provided the renderer models them.
  • Inference: Since the difference mask is computed purely from frame pairs, any future source of aligned real pairs—for example, stop-motion captures with a physical object removed—could fine-tune the model without architectural change.
  • Inference: The reference-based design suggests that inpainting models in general may benefit from consuming the full video including the object region, which could transfer to image-level 'remove anything' editing and interactive segmentation scenarios.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes ROSE, a video inpainting method aimed at removing not only objects but also their environment side effects (shadows, reflections, lighting changes, translucency, mirrors). Because paired real videos are scarce, the authors synthesize 16,678 paired video clips in Unreal Engine, with automatic camera trajectories and exact masks. The model is built on the Wan2.1 diffusion transformer; it feeds the full original video as reference, applies mask augmentations, and trains an auxiliary difference-mask predictor supervised by thresholded pixel differences between paired videos (Eq. 1). They introduce ROSE-Bench with three test sets: synthetic paired, realistic paired (DAVIS copy-paste), and realistic unpaired (VBench metrics). They report state-of-the-art results on the synthetic benchmark and strong performance on realistic tests.

Significance. If the transfer premise holds, this is a valuable contribution: a large synthetic paired dataset with controllable side effects, a systematic taxonomy of object-environment interactions, and a benchmark that fills a gap in video inpainting evaluation. The reference-based whole-video input and the auxiliary difference-mask supervision are plausible and technically clean changes; I found no derivational circularity in the supervision. The main scientific risk is that all positive quantitative evidence for side-effect removal is generated by the same rendering engine used for training, and the real-world test sets do not isolate the side-effect capability. Thus the paper's headline real-world generalization claim is not yet established.

major comments (3)
  1. [Abstract; §5.2, Table 3] The abstract and §5.2 claim 'superior performance ... as measured by PSNR, SSIM, and LPIPS.' Table 3 on the realistic paired benchmark shows ProPainter achieves higher PSNR (32.81 vs. 31.34) than ROSE; ROSE is better only on SSIM and LPIPS. Similarly, Table 4 shows DiffuEraser has higher Imaging Quality (0.658 vs. 0.630). The claim should be restricted to the metrics/categories where it holds, or additional evidence should be provided.
  2. [§5.1; §3.1; §6] Generalization to real-world videos is the central claim but is not supported by the reported evaluations. The synthetic ROSE-Bench (Table 1) is generated by the same Unreal Engine pipeline used for training (Sec. 3.1/5.1), so it measures in-distribution performance. The realistic paired benchmark (Table 3) is built by copy-pasting objects from DAVIS; this creates artificial composites without genuine scene-integrated side effects, so it does not test the paper's core capability. The unpaired VBench metrics (Table 4) have no ground truth for side-effect removal. Sec. 6 itself states future work is needed to 'further bridge synthetic and real-world domains.' A paired real-world test with actual shadows/reflections/lighting is needed before claiming real-world generalization.
  3. [§5.3, Table 2] Table 2's ablation does not support the claim that all primary designs are effective. Under the natural cumulative reading (Base → +MRG → +MA → +DMP), adding mask augmentation after MRG decreases mean PSNR from 30.84 to 29.89 and hurts every category except Common, e.g., Light Source from 30.37 to 25.80 and Mirror from 28.45 to 27.37. The final +DMP restores the mean, but for Light Source and Mirror the final PSNR remains below the +MRG-only value (30.07 vs. 30.37; 28.35 vs. 28.45). Please clarify the exact order and re-evaluate the contribution of each component, or revise the claims.
minor comments (5)
  1. [§5.1] Typos and inconsistent naming: 'hat are critical' should be 'that are critical'; 'DA VIS' appears instead of 'DAVIS' in the realistic paired benchmark description and elsewhere; 'qualitatitve' in §5.2.
  2. [§4.4, Eq. (3)] The loss weight λ is not specified; please state its value. Also, the notation in Eq. (2) is ambiguous: after introducing the latent-resolution difference mask d_t with size f×h/s×w/s, the full-resolution grid (F,H,W) is not clearly connected to the earlier definitions.
  3. [§5.1] The realistic paired benchmark is described only qualitatively. Please report the number of test videos, the number of source/target pairs, and the object categories used, so the benchmark is reproducible and its coverage is clear.
  4. [§5.1] The synthetic benchmark is said to contain '10 high-quality triplets per category,' but the total number and the sampling procedure are not given. Clarify how these clips were selected from the 16,678 training-style pairs and whether they overlap with training data.
  5. [§5.3, Table 2] The column order Base, +MRG, +MA, +DMP should be explicitly defined in a caption or footnote; the current text does not state whether the columns are cumulative, which is essential for interpreting Table 2.

Circularity Check

0 steps flagged

No circularity: ROSE's training and auxiliary supervision are standard supervised losses on rendered pairs; same-pipeline evaluation is a domain-transfer concern, not a derivational circle.

full rationale

The paper's derivation chain is self-contained and does not reduce to its own inputs. Paired training data are generated by toggling object visibility in Unreal Engine (Sec. 3.1), yielding aligned original/edited/mask triples. The model is trained with the standard diffusion denoising loss plus an auxiliary MSE on a difference mask (Eqs. 1-3); the difference mask is a thresholded pixel difference computed from the paired videos, i.e., a supervised label, not a hidden prediction of the central claim. The synthetic ROSE-Bench subset is produced by the same simulation pipeline as the training set, which makes the synthetic scores in-distribution and leaves the real-world generalization claim under-supported; but that is an evaluation/transfer limitation, not circularity. The paper itself concedes this gap in Sec. 6 ('Future work will explore ... broader environmental effects to further bridge synthetic and real-world domains'). The realistic paired benchmark (Table 3) uses copy-paste from DAVIS and therefore does not create true side effects, and the unpaired VBench metrics lack side-effect ground truth; these are correctness/validation concerns, not circular reasoning. There are no load-bearing self-citations, imported uniqueness theorems, or ansatz-by-citation steps. Accordingly, no circular step can be quoted, and the correct finding is no significant circularity (score 0).

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central result depends on the synthetic-to-real transfer assumption, the definition of the difference mask, and several unreported hyperparameters. No new physical entities are introduced. The biggest burden is the transfer premise: all training and most quantitative evaluation share the same Unreal Engine pipeline.

free parameters (4)
  • difference threshold δ = 0.09
    Hand-chosen threshold in Eq. (1) that defines the ground-truth difference mask from paired videos; affects the auxiliary supervision signal.
  • loss weight λ = not reported
    Balances the diffusion loss and the difference-mask prediction loss in Eq. (3). The paper does not give its value.
  • mask augmentation probabilities = not reported
    The five mask variants (original, point, bounding box, dilated, eroded) are 'randomly sampled' during training but no probabilities are given. This affects robustness and final performance.
  • valid-view filtering threshold = not reported
    Views with insufficient foreground mask coverage are discarded based on an unspecified ratio threshold in Sec. 3.1, changing the training distribution.
axioms (4)
  • domain assumption 3D-rendered synthetic videos from Unreal Engine are a sufficient proxy for real-world object side effects (shadows, reflections, etc.)
    The dataset construction in Sec. 3.1 and the generalization claims in Sec. 6 assume that rendering engines reproduce enough physical light transport for a model trained only on synthetic pairs to work on real videos.
  • domain assumption A thresholded L2 pixel difference between paired videos correctly identifies all object-correlated areas that should be modified
    Eq. (1) defines the difference mask used as auxiliary supervision; if this simple difference misses side effects (e.g., subtle reflections) or includes unrelated changes, the predictor may be misled.
  • domain assumption Copy-and-paste DAVIS-based pairs provide a valid realistic benchmark for object removal
    The realistic paired benchmark in Sec. 5.1 pastes a masked object from one video into another, treating the original as ground truth, but the pasted object may not produce physically plausible scene interactions.
  • domain assumption PSNR, SSIM, and LPIPS on synthetic pairs are meaningful predictors of real-world editing quality
    The paper's quantitative claims in Sec. 5.2 rest on these metrics applied to synthetic data, despite the benchmark being generated by the same pipeline that produced the training set.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of ROSE: Remove Objects with Side Effects in Videos." pith.science (2026). https://pith.science/paper/BVWMKCYY

@misc{pith2026250818633,
  author       = {Pith},
  title        = {Pith review of: ROSE: Remove Objects with Side Effects in Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BVWMKCYY}},
  note         = {Machine review of arXiv:2508.18633}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Video object removal has achieved advanced performance due to the recent success of video generative models. However, when addressing the side effects of objects, e.g., their shadows and reflections, existing works struggle to eliminate these effects for the scarcity of paired video data as supervision. This paper presents ROSE, termed Remove Objects with Side Effects, a framework that systematically studies the object's effects on environment, which can be categorized into five common cases: shadows, reflections, light, translucency and mirror. Given the challenges of curating paired videos exhibiting the aforementioned effects, we leverage a 3D rendering engine for synthetic data generation. We carefully construct a fully-automatic pipeline for data preparation, which simulates a large-scale paired dataset with diverse scenes, objects, shooting angles, and camera trajectories. ROSE is implemented as an video inpainting model built on diffusion transformer. To localize all object-correlated areas, the entire video is fed into the model for reference-based erasing. Moreover, additional supervision is introduced to explicitly predict the areas affected by side effects, which can be revealed through the differential mask between the paired videos. To fully investigate the model performance on various side effect removal, we presents a new benchmark, dubbed ROSE-Bench, incorporating both common scenarios and the five special side effects for comprehensive evaluation. Experimental results demonstrate that ROSE achieves superior performance compared to existing video object erasing models and generalizes well to real-world video scenarios. The project page is https://rose2025-inpaint.github.io/.

Figures

Figures reproduced from arXiv: 2508.18633 by Bin Wang, Chenxuan Miao, Donglian Qi, Hantang Liu, Hengshuang Zhao, Jianshu Zeng, Xi Chen, Yunfeng Yan, Yutong Feng, Zixiang Gao.

Figure 1
Figure 1. Figure 1: Video object removal results generated by ROSE (zoom in for better view). Every two lines are an example where the above is input video with mask and the bottom is inference result. We sequentially show cases of various side effects studied in this paper. with and without the object, where the object’s influence on the environment is correspondingly changed, such as its shadow on the ground. Most works lev… view at source ↗
Figure 2
Figure 2. Figure 2: Paired video preparation pipeline using 3D data, which can be divided into: scene and object sampling, multi-view generation with masks, valid view filtering and video data rendering. Valid View Filtering. To ensure the quality of videos and avoid object-occlusion cases, we further filter out views by calculating the ratio of foreground pixels in the mask. Ratios lower than a threshold suggest videos with … view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of the various side-effect categories studied in the dataset of ROSE. 4 [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The framework of ROSE. We concatenate the noisy latents with the original input video and masks, consumed by a video inpainting model. An additional difference mask predictor is introduced to predict the correlated area in video, automatically computed from the input video pairs. Mirror: Objects reflected in mirrors require spatial reasoning and semantic understanding to inpaint both the object and its mir… view at source ↗
Figure 6
Figure 6. Figure 6: Comparison between the previous paradigm and our reference-based paradigm. attention mechanism is effective for seeking the inter-region correlations in videos. Given the object region as input, the model thereby leverage it prior knowledge to localize the side effect regions, thus outperforms the model with masked video input. Furthermore, the complete video as input serves to enhance the temporal consist… view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative comparison between our method and existing approaches on real-world samples. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. MiVE: Multiscale Vision-language features for reference-guided video Editing

    cs.CV 2026-05 unverdicted novelty 7.0

    MiVE repurposes VLMs as multiscale feature extractors integrated into a unified self-attention Diffusion Transformer, achieving top human preference in reference-guided video editing.

  2. PROVE: A Perceptual RemOVal cohErence Benchmark for Visual Media

    cs.CV 2026-05 conditional novelty 7.0

    Removal Coherence (RC) metrics, which compare local feature distributions in masked versus background regions via sliding-window MMD, align with human judgments of object-removal quality substantially better than exis...

  3. PROVE: A Perceptual RemOVal cohErence Benchmark for Visual Media

    cs.CV 2026-05 unverdicted novelty 7.0

    PROVE proposes RC metrics for perceptual removal coherence and releases PROVE-Bench to better align automatic scores with human judgments on object removal tasks.

  4. Physics-Aware Video Instance Removal Benchmark

    cs.CV 2026-04 unverdicted novelty 7.0

    The PVIR benchmark tests video object removal on physical consistency using 95 annotated videos and shows that existing methods struggle with complex interactions like lingering shadows.

  5. From Draft to Draft-Free: One-Step Video Object Removal via Privileged Distillation and Fast Planting

    cs.CV 2026-07 conditional novelty 6.0

    A one-step, draft-free video object removal model trained by distilling a ground-truth-conditioned teacher reaches comparable or better quality than multi-step diffusion methods while running in about 1 second.

  6. Geometry-Instructed Video Editing

    cs.CV 2026-06 unverdicted novelty 6.0

    GIVE uses paired depth-box and orientation-box geometry streams plus a graphics pipeline to train models for consistent object-level geometric edits like translate, rotate, and scale in videos.

  7. AlbedoEdit: Unified Instance-Level Video Editing with Albedo Guidance

    cs.GR 2026-05 unverdicted novelty 6.0

    AlbedoEdit fine-tunes video foundation models to translate RGB videos into edited versions conditioned on user-edited first-frame albedo maps, trained on a new synthetic paired dataset for insertion, removal, and text...

  8. MiVE: Multiscale Vision-language features for reference-guided video Editing

    cs.CV 2026-05 unverdicted novelty 6.0

    MiVE repurposes VLMs as multiscale feature extractors integrated into a unified self-attention Diffusion Transformer for reference-guided video editing, claiming top human preference scores over prior methods.

  9. VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects

    cs.CV 2026-04 unverdicted novelty 6.0

    VEFX-Bench releases a large human-labeled video editing dataset, a multi-dimensional reward model, and a standardized benchmark that better matches human judgments than generic evaluators.

  10. InsertAnywhere: Geometrically Grounded and Optics-Aware Video Object Insertion

    cs.CV 2025-12 conditional novelty 6.0

    InsertAnywhere inserts a reference object into arbitrary videos by reconstructing 4D geometry to propagate a user-given placement across frames and fine-tuning video diffusion on ROSE++, a removal-to-insertion dataset...

  11. GenEraser: Generalizable Video Object Removal via Balanced Text-Mask Guidance and Decoupled Locator-Preserver

    cs.CV 2026-05 unverdicted novelty 5.0

    GenEraser proposes MC-MoE with bipartite text guidance, LD-CFG fusion, and a decoupled locator-preserver architecture for generalizable video object and effect removal, claiming 2.16 dB and 1.44 dB gains on ROSE and V...

Reference graph

Works this paper leans on

45 extracted references · 30 canonical work pages · cited by 9 Pith papers · 1 internal anchor

  1. [1]

    Magi-1: Autoregressive video generation at scale

    Sand AI. Magi-1: Autoregressive video generation at scale. https://github.com/SandAI-org/ MAGI-1, 2025

    work page 2025

  2. [2]

    Frozen in time: Learning joint representations for vision and text using cross-modal contrastive learning

    Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisserman. Frozen in time: Learning joint representations for vision and text using cross-modal contrastive learning. In NeurIPS, 2021

    work page 2021

  3. [3]

    Videopainter: Any-length video inpainting and editing with plug-and-play context control

    Yuxuan Bian, Zhaoyang Zhang, Xuan Ju, Mingdeng Cao, Liangbin Xie, Ying Shan, and Qiang Xu. Videopainter: Any-length video inpainting and editing with plug-and-play context control. arXiv preprint arXiv:2503.05639, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    Devit: Deformed vision transformers in video inpainting

    Jiayin Cai, Changlin Li, Xin Tao, Chun Yuan, and Yu-Wing Tai. Devit: Deformed vision transformers in video inpainting. In ACM MM, 2022

    work page 2022

  5. [5]

    Free-form video inpainting with 3d gated convolution and temporal patchgan

    Ya-Liang Chang, Zhe Yu Liu, Kuan-Ying Lee, and Winston Hsu. Free-form video inpainting with 3d gated convolution and temporal patchgan. In ICCV, 2019

    work page 2019

  6. [6]

    Epic Games. Fab. https://www.fab.com/, 2024

    work page 2024

  7. [7]

    Unreal engine 5.3

    Epic Games. Unreal engine 5.3. https://www.unrealengine.com/, 2024

    work page 2024

  8. [8]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In ICML, 2024

    2024

  9. [9]

    Advanced video inpainting using optical flow-guided efficient diffusion

    Bohai Gu, Hao Luo, Song Guo, and Peiran Dong. Advanced video inpainting using optical flow-guided efficient diffusion. arXiv preprint arXiv:2412.00857, 2024

    Pith/arXiv arXiv 2024

  10. [10]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020

    2020

  11. [11]

    Image quality metrics: Psnr vs

    Alain Hore and Djemel Ziou. Image quality metrics: Psnr vs. ssim. In ICPR, 2010

    work page 2010

  12. [12]

    Yuan-Ting Hu, Heng Wang, Nicolas Ballas, Kristen Grauman, and Alexander G. Schwing. Proposal-based video completion. In ECCV, 2020

    work page 2020

  13. [13]

    VBench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. In CVPR, 2024

    work page 2024

  14. [14]

    Smarteraser: Remove anything from images using masked-region guidance

    Longtao Jiang, Zhendong Wang, Jianmin Bao, Wengang Zhou, Dongdong Chen, Lei Shi, Dong Chen, and Houqiang Li. Smarteraser: Remove anything from images using masked-region guidance. In CVPR, 2025

    work page 2025

  15. [15]

    Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion

    Xuan Ju, Xian Liu, Xintao Wang, Yuxuan Bian, Ying Shan, and Qiang Xu. Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion. In ECCV, 2024

    work page 2024

  16. [16]

    Deep video inpainting

    Dahun Kim, Sanghyun Woo, Joon-Young Lee, and In So Kweon. Deep video inpainting. In CVPR, 2019

    work page 2019

  17. [17]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In CVPR, 2023

    work page 2023

  18. [18]

    Hunyuanvideo: A systematic framework for large video generative models, 2025

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, Kathrina Wu, Qin Lin, Junkun Yuan, Yanxin Long, Aladdin Wang, Andong Wang, Changlin Li, Duojun Huang, Fang Yang, Hao Tan, Hongmei Wang, Jacob Song, Jiawang Bai, Jianbing Wu, Jinbao Xue, Joey Wang, Kai Wang, Mengyang Liu, Pengyu Li, Shuai Li, ...

    2025

  19. [19]

    Black Forest Labs. Flux. https://github.com/black-forest-labs/flux , 2024

    2024

  20. [20]

    Video Diffusion Models are Strong Video Inpainter

    Minhyeok Lee, Suhwan Cho, Chajin Shin, Jungho Lee, Sunghun Yang, and Sangyoun Lee. Video diffusion models are strong video inpainter. arXiv preprint arXiv:2408.11402, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  21. [21]

    Copy-and-paste networks for deep video inpainting

    Sungho Lee, Seoung Wug Oh, DaeYeun Won, and Seon Joo Kim. Copy-and-paste networks for deep video inpainting. In ICCV, 2019

    work page 2019

  22. [22]

    Short-term and long-term context aggregation network for video inpainting

    Ang Li, Shanshan Zhao, Xingjun Ma, Mingming Gong, Jianzhong Qi, Rui Zhang, Dacheng Tao, and Ramamohanarao Kotagiri. Short-term and long-term context aggregation network for video inpainting. In ECCV, 2020. 11

    work page 2020

  23. [23]

    Diffueraser: A diffusion model for video inpainting

    Xiaowen Li, Haolan Xue, Peiran Ren, and Liefeng Bo. Diffueraser: A diffusion model for video inpainting. 2025

    work page 2025

  24. [24]

    Fuseformer: Fusing fine-grained information in transformers for video inpainting

    Rui Liu, Hanming Deng, Yangyi Huang, Xiaoyu Shi, Lewei Lu, Wenxiu Sun, Xiaogang Wang, Jifeng Dai, and Hongsheng Li. Fuseformer: Fusing fine-grained information in transformers for video inpainting. In ICCV, 2021

    work page 2021

  25. [25]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023

    2023

  26. [26]

    A benchmark dataset and evaluation methodology for video object segmentation

    Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In CVPR, 2016

    work page 2016

  27. [27]

    SDXL: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

    Pith/arXiv arXiv 2023

  28. [28]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022

    2022

  29. [29]

    Bivdiff: A training-free framework for general-purpose video synthesis via bridging image and video diffusion models

    Fengyuan Shi, Jiaxi Gu, Hang Xu, Songcen Xu, Wei Zhang, and Limin Wang. Bivdiff: A training-free framework for general-purpose video synthesis via bridging image and video diffusion models. In CVPR, 2024

    work page 2024

  30. [30]

    Weiss, Niru Maheswaranathan, and Surya Ganguli

    Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. arXiv preprint arXiv:1503.03585, 2015

    Pith/arXiv arXiv 2015

  31. [31]

    Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole

    Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2021

    work page 2021

  32. [32]

    Resolution-robust large mask inpainting with fourier convolutions

    Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution-robust large mask inpainting with fourier convolutions. arXiv preprint arXiv:2109.07161, 2021

    Pith/arXiv arXiv 2021

  33. [33]

    Gomez, Łukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017

    work page 2017

  34. [34]

    Wan: Open and advanced large-scale video generative models

    Wan Video. Wan: Open and advanced large-scale video generative models. https://github.com/ Wan-Video/Wan2.1, 2025

    work page 2025

  35. [35]

    Video inpainting by jointly learning temporal structure and spatial details

    Chuan Wang, Haibin Huang, Xiaoguang Han, and Jue Wang. Video inpainting by jointly learning temporal structure and spatial details. In AAAI, 2019

    work page 2019

  36. [36]

    Zhou Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: From error visibility to structural similarity. TIP, 2004

    work page 2004

  37. [37]

    Omnieraser: Remove objects and their effects in images with paired video-frame data

    Runpu Wei, Zijin Yin, Shuo Zhang, Lanxiang Zhou, Xueyi Wang, Chao Ban, Tianwei Cao, Hao Sun, Zhongjiang He, Kongming Liang, and Zhanyu Ma. Omnieraser: Remove objects and their effects in images with paired video-frame data. arXiv preprint arXiv:2501.07397, 2025

    Pith/arXiv arXiv 2025

  38. [38]

    Towards language-driven video inpainting via multimodal large language models

    Jianzong Wu, Xiangtai Li, Chenyang Si, Shangchen Zhou, Jingkang Yang, Jiangning Zhang, Yining Li, Kai Chen, Yunhai Tong, Ziwei Liu, et al. Towards language-driven video inpainting via multimodal large language models. arXiv preprint arXiv:2401.10226, 2024

    Pith/arXiv arXiv 2024

  39. [39]

    Youtube-vos: Sequence-to-sequence video object segmentation

    Ning Xu, Linjie Yang, Yuchen Fan, Jianchao Yang, Dingcheng Yue, Yuchen Liang, Brian Price, Scott Cohen, and Thomas Huang. Youtube-vos: Sequence-to-sequence video object segmentation. In ECCV, 2018

    2018

  40. [40]

    Video-llava: Learning multimodal video instruction-following

    Yue Xu, Jiabo Ye, Yifan Xu, Hang Zhou, Wayne Wu, and Ziwei Liu. Video-llava: Learning multimodal video instruction-following. In EMNLP, 2023

    work page 2023

  41. [41]

    Flow-guided transformer for video inpainting

    Kaidong Zhang, Jingjing Fu, and Dong Liu. Flow-guided transformer for video inpainting. In ECCV, 2022

    work page 2022

  42. [42]

    Efros, Eli Shechtman, and Oliver Wang

    Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018

    2018

  43. [43]

    Shangchen Zhou, Chongyi Li, Kelvin C. K. Chan, and Chen Change Loy. Propainter: Improving propagation and transformer for video inpainting. In ICCV, 2023. 12

    work page 2023

  44. [44]

    Cococo: Improving text-guided video inpainting for better consistency, controllability and compatibility

    Bojia Zi, Shihao Zhao, Xianbiao Qi, Jianan Wang, Yukai Shi, Qianyu Chen, Bin Liang, Kam-Fai Wong, and Lei Zhang. Cococo: Improving text-guided video inpainting for better consistency, controllability and compatibility. In AAAI, 2025

    work page 2025

  45. [45]

    Progressive temporal feature alignment network for video inpainting

    Xueyan Zou, Linjie Yang, Ding Liu, and Yong Jae Lee. Progressive temporal feature alignment network for video inpainting. In CVPR, 2021. 13

    work page 2021

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.