REVIEW 7 cited by
VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Control
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Video inpainting, which aims to restore corrupted video content, has experienced substantial progress. Despite these advances, existing methods, whether propagating unmasked region pixels through optical flow and receptive field priors, or extending image-inpainting models temporally, face challenges in generating fully masked objects or balancing the competing objectives of background context preservation and foreground generation in one model, respectively. To address these limitations, we propose a novel dual-stream paradigm VideoPainter that incorporates an efficient context encoder (comprising only 6% of the backbone parameters) to process masked videos and inject backbone-aware background contextual cues to any pre-trained video DiT, producing semantically consistent content in a plug-and-play manner. This architectural separation significantly reduces the model's learning complexity while enabling nuanced integration of crucial background context. We also introduce a novel target region ID resampling technique that enables any-length video inpainting, greatly enhancing our practical applicability. Additionally, we establish a scalable dataset pipeline leveraging current vision understanding models, contributing VPData and VPBench to facilitate segmentation-based inpainting training and assessment, the largest video inpainting dataset and benchmark to date with over 390K diverse clips. Using inpainting as a pipeline basis, we also explore downstream applications including video editing and video editing pair data generation, demonstrating competitive performance and significant practical potential. Extensive experiments demonstrate VideoPainter's superior performance in both any-length video inpainting and editing, across eight key metrics, including video quality, mask region preservation, and textual coherence.
Forward citations
Cited by 7 Pith papers
-
VideoCanvas: Unified Video Completion from Arbitrary Spatiotemporal Patches via In-Context Conditioning
A single diffusion model with in-context conditioning and fractional RoPE positions completes videos from arbitrary spatio-temporal image patches.
-
O-DisCo-Edit: Object Distortion Control for Unified Realistic Video Editing
A video editor trained on randomly distorted objects, then steered by adaptive noise at inference, is claimed to surpass dedicated and unified editors across eight tasks with far less training.
-
ROSE: Remove Objects with Side Effects in Videos
A video inpainting model trained on 3D-rendered pairs removes objects together with their shadows, reflections, and other side effects, plus a new benchmark.
-
UNIC: Unified In-Context Video Editing
One diffusion transformer handles ID insert, swap, delete, stylization, propagation, and re-camera control in a single model using in-context token concatenation with task-aware positional encoding and bias.
-
MiniMax-Remover: Taming Bad Noise Helps Video Object Removal
A two-stage video object remover that removes text conditioning and uses minimax adversarial noise to achieve high-quality removal in 6 sampling steps without classifier-free guidance.
-
Follow-Your-Creation: Empowering 4D Creation through Video Inpainting
Follow-Your-Creation fine-tunes the Wan2.1 video inpainting model on composite point-cloud and editing masks so a single monocular video can be converted into editable 4D video with new camera motion.
-
OmniV2V: Versatile Video Generation and Editing via Dynamic Content Manipulation
OmniV2V is one diffusion-transformer model that performs eight video generation and editing tasks by combining mask, pose, image, and text-instruction conditions.
Discussion (0). Sign in to comment.