Pith. sign in

REVIEW 4 major objections 4 minor 47 references

A training-free pipeline replaces video backgrounds while preserving foreground pixels.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A training-free video background replacement pipeline that keeps the foreground pixel-consistent by projecting refined latents through a deterministic reparameterization.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection ANYPORTAL is a clever training-free pipeline with a genuine new trick in RPA, but the evidence base is too thin for the strength of the claims; still, it deserves a serious referee. the 4 major comments →

arxiv 2509.07472 v1 pith:3JZVTRWZ submitted 2025-09-09 cs.CV

ANYPORTAL: Zero-Shot Consistent Video Background Replacement

Wenshuo Gao , Xicheng Lan , Shuai Yang This is my paper
classification cs.CV
keywords video background replacementzero-shot video editingdiffusion modelsforeground consistencyrelightinglatent space projectiontemporal consistencytraining-free pipeline
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AnyPortal aims to make video background replacement work out of the box, with no fine-tuning, by combining two kinds of pre-trained diffusion models: an image model that understands how light should fall on a subject in a new scene, and a video model that knows how frames should move together. The paper claims the hard part is not generating the new background but keeping the foreground identical to the input video. To solve this it introduces a Refinement Projection Algorithm that edits foreground details in pixel space and maps them back into the compact video latent space in a way that, by construction, leaves unedited regions unchanged. If the claim holds, users could 'teleport' a person or object into a new scene with stable lighting, stable geometry, and consistent identity, on a single consumer GPU.

Core claim

On the paper's own terms, the central discovery is that foreground-consistent video background replacement does not require training a video relighting model. AnyPortal demonstrates a modular, zero-shot pipeline in which the image diffusion model IC-Light performs background generation and relighting, and a video diffusion model supplies temporal consistency; the new ingredient is the Refinement Projection Algorithm, which guarantees pixel-level foreground preservation. The algorithm computes a deterministic noise vector from the current unrefined latent, then re-encodes a refined pixel-space video and reuses that same noise vector to sample the new latent. When no refinement is applied, thi

What carries the argument

The Refinement Projection Algorithm (RPA) is the load-bearing mechanism. At each denoising step it takes the noise-free latent prediction x_t^0, decodes it to pixels, separates low- and high-frequency content, injects the input video's high-frequency foreground details, then re-encodes and projects the result back into latent space using the deterministic noise vector epsilon_hat = (x_t^0 - mu)/sigma. The identity property — if no refinement is applied, the projected latent equals the original latent — is what lets the pipeline refine foreground pixels without blurring or distorting the generated background. The two-step IC-Light harmonization with cross-frame attention supplies the relighti

Load-bearing premise

The projection assumes the deterministic noise vector computed from the unrefined latent is still a valid reparameterization after the foreground refinement; if the refinement strongly shifts the VAE's mean or variance in unedited regions, that same noise vector can inject large deviations and degrade the background.

What would settle it

Take a video with fast motion or a low-resolution foreground, run AnyPortal with RPA active and with RPA replaced by naive decode-refine-encode, and compare the background regions pixel-by-pixel against the no-refinement output. If the background distances are similar, the identity guarantee of RPA is not doing the work claimed; alternatively, measure reconstruction error of x_t^0 before and after one full projection step over a grid of refinement strengths to locate where the exact-equality property breaks.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Background replacement becomes an inference-only operation: any improvement in pre-trained image or video diffusion models can be dropped into the modular pipeline without retraining.
  • Foreground identity is preserved at pixel level in the edited region while background regions remain almost identical to the unrefined generation, addressing a failure mode common in text-guided video editing.
  • Both text-prompted and image-prompted backgrounds are supported with the same pipeline by only changing how the first background frame is produced.
  • The method runs on a single 24 GB GPU and produces a 49-frame 480x720 output in about 12 minutes, making the workflow practical outside professional studios.
  • Quantitative and user evaluations on 30 videos report the best temporal consistency, motion preservation, and foreground detail preservation among the zero-shot baselines compared.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The identity property of RPA is really a statement about the VAE's reconstruction, so the same projection trick could plausibly be reused for other pixel-level video edits, such as recolorization, inpainting, or facial attribute editing, whenever the 3D latent space is too compact for direct latent manipulation.
  • A stress test the paper does not run: increasing refinement strength should eventually break the assumption that the deterministic noise vector stays valid; finding that threshold would map the method's operating range.
  • Because RPA only transfers high-frequency foreground detail, the method implicitly depends on the input video having sufficient resolution and sharp edges, a dependency the paper itself flags as a limitation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. ANYPORTAL is a training-free pipeline for video background replacement from an input foreground video plus a text/image background prompt. It consists of three stages: motion-aware background video generation using DAS/ControlNet and ProPainter; a two-step IC-Light illumination harmonization with cross-frame attention; and a consistency-enhancement stage that runs SDEdit denoising with a frozen video diffusion model while injecting high-frequency foreground details through a Refinement Projection Algorithm (RPA). RPA decodes the current latent, refines foreground details in pixel space, re-encodes the refined video, and reparameterizes the refined latent using a deterministic noise computed from the unrefined decode. The authors prove an exact-reconstruction identity when no refinement is applied and claim this keeps background regions almost identical during actual refinement. Experiments compare against IC-Light, TokenFlow, and DAS on 30 self-collected videos, with additional ablations.

Significance. RPA is an elegant attempt to make pixel-level foreground control compatible with compact 3D video latents, and the no-refinement identity is exactly stated and mathematically correct. The modular, fully frozen-model design is practical and clearly described; the limitations section is honest. However, the main technical novelty is only validated qualitatively under the actual refinement operation, and the empirical evidence is small-scale with no statistical significance. If the stability of RPA under refinement is demonstrated with quantitative background-preservation metrics and the code/data are released, this would be a useful contribution to zero-shot video editing. As written, the strengths are not yet matched by the evidence.

major comments (4)
  1. [Sec. 3.2.3 / Algorithm 2] The exact-reconstruction property is shown only for the no-refinement case (tilde I_t^0 = I_t^0). In the actual pipeline, Refine always changes the decoded video, so the reparameterization epsilon_hat=(x_t^0-mu)/sigma is reused with (mu_hat,sigma_hat)=E(Refine(D(x_t^0),I)). If the nonlinear VAE and the inpainted background shift mu_hat/sigma_hat outside the edited foreground, hat x_t^0 = mu_hat + epsilon_hat sigma_hat accumulates drift in every DDIM step. The paper does not analyze this transfer assumption or quantify background drift; Fig. 5(d) and Table 2 are not isolate it. Please add a background-masked distortion comparison (RPA vs. no-RPA, or vs. direct encode-decode) across T1 values and refinement strengths, and state conditions under which the transfer is stable.
  2. [Sec. 4 / Table 1] The evaluation is based on 30 self-collected samples; no standard deviations, confidence intervals, or significance tests are reported. Several differences are small (e.g., Tem-Con 0.993 vs 0.986 for DAS; Mtn-Psrv 0.987 vs 0.985; ID-Psrv 0.313 vs 0.364) and may lie within noise. The user study uses 24 raters but only preference percentages are given. Without per-sample variability or paired tests, the claim that ANYPORTAL outperforms the baselines is not statistically supported. Code and test data are also promised only for publication, preventing independent verification.
  3. [Sec. 3.2.3 / Algorithm 1] The description and the pseudocode disagree on the inpainting target. The text states the background region of tilde I_t^0 is set to 'the inpainted I', while Algorithm 1 line 6 computes I_BG = Inpaint(I_t^0, ForegroundSegmentation(I_t^0)) and line 7 uses it with the mask M_t^0 computed from I. Inpainting the edited decode versus the original input are different operations, and using a mask from one video to select an inpainted region from another can misalign boundaries. Please clarify the intended operation and align the text with the implementation; this choice directly bears on the claim that background areas are preserved by RPA.
  4. [Sec. 4.2 / Table 2] The ablation for RPA is ambiguous. The text compares 'without RPA' (Fig. 5(b)) to 'naive high-frequency detail refinement through decoding and encoding' (Fig. 5(c)), but Table 2 only has a single 'w/o RPA' row. It is unclear whether w/o RPA means no foreground refinement at all, or refinement by direct encode-decode without the deterministic projection. These are different ablations; the former does not isolate RPA and the latter does. Please specify the exact configuration and, ideally, report both variants.
minor comments (4)
  1. [Algorithm 2] Notation is inconsistent: the text uses epsilon_hat while the pseudocode uses \hat\epsilon; also make explicit in Algorithm 2 that x_T1 is obtained by the DDPM forward process from I_L, since only 'Initial noise x_T1' is stated.
  2. [Sec. 3.2.2] The sentence 'for each frame of I\'' refers to an output not yet defined; clarify whether it is I from the input video or the background-generation output.
  3. [Fig. 10] The comparison with Light-A-Video is qualitative only. If quantitative metrics are available for this concurrent work, include them; otherwise, state explicitly that no metric was computed.
  4. [Sec. 6] The limitations section is commendable, but it would be useful to also state the failure mode of RPA when the foreground-background boundary segmentation is inaccurate, since the refinement directly depends on the mask quality.

Circularity Check

0 steps flagged

No significant circularity: RPA's zero-error projection is an algebraic design property, and the framework is tested against external baselines with frozen pretrained models.

full rationale

ANYPORTAL is a modular, training-free pipeline built from frozen pretrained models (CogVideoX, IC-Light, DAS, ProPainter, BiRefNet). The claimed foreground preservation is achieved by construction: Algorithm 1 injects the original video's high-frequency foreground detail into the decoded frame, and Algorithm 2 defines epsilon_hat = (x_t^0 - mu)/sigma so that x_hat_t^0 = mu_hat + epsilon_hat*sigma_hat equals x_t^0 exactly when no refinement is applied. This is an algebraic identity, not a fitted prediction; no parameter is learned from the test data or from the evaluation metrics. The extrapolation that the background remains 'almost identical' under actual refinement is an empirical assumption supported by ablation (Table 2, Fig. 5) and is a robustness concern, not circular reasoning. The only self-citation is reference [35] (Rerender a Video, coauthored by Shuai Yang) for the common practice of operating on noise-free latents and cross-frame attention; it is not load-bearing for the paper's central claim. The method is compared against external baselines (IC-Light, TokenFlow, DAS) and evaluated with standard metrics, so the central claims have independent content.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

All free parameters are hyperparameters, not fitted constants. The method depends on the reliability of external pre-trained models and the stability of the deterministic reparameterization. No new physical or conceptual entities are introduced.

free parameters (4)
  • Total denoising steps T = 20
    Total DDIM steps for both IC-Light and video diffusion; set by hand, a standard hyperparameter.
  • Illumination harmonization noise steps T0 = 0.7T or 0.4T
    Controls the strength of IC-Light relighting; chosen by hand for strong or weak illumination effects.
  • Consistency enhancement noise steps T1 = 0.7T or 0.4T
    Controls how much the video diffusion model refines the coarse video; coupled to T0 in the experiments.
  • Gaussian blur kernel for LF/HF decomposition = not reported
    Algorithm 1 uses GaussianBlur without specifying kernel size or sigma; the split between low and high frequencies is implementation-defined.
axioms (4)
  • domain assumption The pre-trained models IC-Light, CogVideoX, DAS, ProPainter, BiRefNet, and the edge ControlNet behave as documented and are compatible when composed.
    The entire pipeline rests on the correctness and compatibility of these external models, invoked throughout Sections 3.2.1 to 3.2.3.
  • domain assumption The 3D VAE of the video diffusion model has bounded reconstruction error, so the deterministic epsilon projection keeps non-refined regions stable.
    Section 3.2.3 claims that RPA's alignment property keeps the background almost identical, which presumes that the VAE encode/decode statistics do not drift unpredictably.
  • domain assumption High-frequency components of the input foreground can be transferred directly to the generated video without introducing temporal artifacts or breaking the relighting.
    Algorithm 1 copies HF details from I into I_t^0 in the foreground region; this assumes the HF content is semantically compatible with the new background and lighting.
  • domain assumption DDIM denoising with a substituted latent stays on the video latent manifold.
    Both the SDEdit-style refinement and RPA replace the latent at intermediate steps; this presumes the denoiser can accept the projected latent without divergence, as used in Algorithm 2.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of ANYPORTAL: Zero-Shot Consistent Video Background Replacement." pith.science (2026). https://pith.science/paper/3JZVTRWZ

@misc{pith2026250907472,
  author       = {Pith},
  title        = {Pith review of: ANYPORTAL: Zero-Shot Consistent Video Background Replacement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3JZVTRWZ}},
  note         = {Machine review of arXiv:2509.07472}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Despite the rapid advancements in video generation technology, creating high-quality videos that precisely align with user intentions remains a significant challenge. Existing methods often fail to achieve fine-grained control over video details, limiting their practical applicability. We introduce ANYPORTAL, a novel zero-shot framework for video background replacement that leverages pre-trained diffusion models. Our framework collaboratively integrates the temporal prior of video diffusion models with the relighting capabilities of image diffusion models in a zero-shot setting. To address the critical challenge of foreground consistency, we propose a Refinement Projection Algorithm, which enables pixel-level detail manipulation to ensure precise foreground preservation. ANYPORTAL is training-free and overcomes the challenges of achieving foreground consistency and temporally coherent relighting. Experimental results demonstrate that ANYPORTAL achieves high-quality results on consumer-grade GPUs, offering a practical and efficient solution for video content creation and editing.

Figures

Figures reproduced from arXiv: 2509.07472 by Shuai Yang, Wenshuo Gao, Xicheng Lan.

Figure 1
Figure 1. Figure 1: We propose ANYPORTAL, a training-free framework for high-consistency video background replacement and foreground relighting. Given an input foreground video and a text or image prompt of the background, our method produces a video with the target background under harmonious illuminations, while maintaining the foreground video details and intrinsic properties. Abstract Despite the rapid advancements in vid… view at source ↗
Figure 2
Figure 2. Figure 2: Framework of ANYPORTAL. (a) Background Generation: A video diffusion model ϵθ is used to generate a basic background video Ib following the first frame generated by IC-Light model δp and the camera motion of the input video I; (b) Light Harmonization: a two-step pipeline based on IC-Light model δI and δp is proposed to combine the foreground and background video and harmonize its illumination; (c) Consiste… view at source ↗
Figure 3
Figure 3. Figure 3: Motion-aware background generation. The generated background video Ib follows the camera motion of the input video. (a) Input video I. (b) Video output Ib. (c) Inpainted output Ib. The blue dotted lines indicate the inconsistent foreground areas. the corresponding foreground and background under har￾monized illumination. For simplicity, we omit the iterative sampling operations and transformations between … view at source ↗
Figure 5
Figure 5. Figure 5: Consistency enhancement via Refinement Projection Algorithm (RPA). (a) Input video. (b) Video diffusion model strengthens temporal consistency, but creates inconsistent fore￾ground appearance. (c) Foreground refinement without RPA intro￾duces quality degradation in the background. (d) RPA effectively constrains foreground details while preserving other regions. we obtain the image-guided result I ′ img = δ… view at source ↗
Figure 6
Figure 6. Figure 6: Visual comparison on zero-shot video background replacement. Full video results are provided in the supplementary material. Algorithm 2: DDIM Denoising with RPA Input: Initial noise xT1 , input video I, condition c Output: Refined denoised result x0 1 for t = T1 . . . 1 do 2 x t 0 = [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Effect of T0 in illumination harmonization. With in￾creased T0, δp assumes a more prominent role in facilitating inten￾sive light and shadow effect for foreground. (a) (b) [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Effect of Consistency Enhancement. The temporal prior of video diffusion model effectively helps improve temporal consistency, e.g., eliminating the inconsistent appearance of rocks as in the enlarged blue region. 4.2. Ablation Study To validate the contributions of different modules to the overall performance, we systematically deactivate specific modules in our framework. The results are reported in Figs… view at source ↗
Figure 10
Figure 10. Figure 10: Comparison to Light-A-Video. As concurrent work, Light-A-Video only relights backgrounds instead of generating new content [PITH_FULL_IMAGE:figures/full_fig_p008_10.png] view at source ↗
Figure 9
Figure 9. Figure 9: Image-guided background replacement. In addition to text prompts, our method also supports replacing the background based on a user-provided image. ond stage. Without Cst-Enh, Tem-Con drops significantly as in [PITH_FULL_IMAGE:figures/full_fig_p008_9.png] view at source ↗
Figure 11
Figure 11. Figure 11: Limitations. Low-quality input, poor boundary condi￾tion and fast movement degrades the performance of our method. Red boxes indicate the mismatched and blurry boundary region [PITH_FULL_IMAGE:figures/full_fig_p008_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

47 extracted references · 22 canonical work pages

  1. [1]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023. 3

    Pith/arXiv arXiv 2023

  2. [2]

    Align your latents: High-resolution video synthesis with la- tent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with la- tent diffusion models. InCVPR, pages 22563–22575, 2023. 3

    2023

  3. [3]

    MasaCtrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xi- aohu Qie, and Yinqiang Zheng. MasaCtrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing. InICCV, 2023. 2

    work page 2023

  4. [4]

    Arcface: Additive angular margin loss for deep face recognition

    Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. InCVPR, pages 4690–4699, 2019. 7

    work page 2019

  5. [5]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. 2024. 2

    work page 2024

  6. [6]

    Relightvid: Temporal-consistent diffusion model for video relighting.arXiv preprint arXiv:2501.16330, 2025

    Ye Fang, Zeyi Sun, Shangzhan Zhang, Tong Wu, Yinghao Xu, Pan Zhang, Jiaqi Wang, Gordon Wetzstein, and Dahua Lin. Relightvid: Temporal-consistent diffusion model for video relighting.arXiv preprint arXiv:2501.16330, 2025. 3

    Pith/arXiv arXiv 2025

  7. [7]

    Fbsdiff: Plug-and-play fre- quency band substitution of diffusion features for highly con- trollable text-driven image translation

    Xiang Gao and Jiaying Liu. Fbsdiff: Plug-and-play fre- quency band substitution of diffusion features for highly con- trollable text-driven image translation. pages 4101–4109,

    work page

  8. [8]

    Tokenflow: Consistent diffusion features for consistent video editing.arXiv preprint arXiv:2307.10373, 2023

    Michal Geyer, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Tokenflow: Consistent diffusion features for consistent video editing.arXiv preprint arXiv:2307.10373, 2023. 3, 6, 7

    Pith/arXiv arXiv 2023

  9. [9]

    Diffusion as shader: 3d-aware video diffu- sion for versatile video generation control.arXiv preprint arXiv:2501.03847, 2025

    Zekai Gu, Rui Yan, Jiahao Lu, Peng Li, Zhiyang Dou, Chenyang Si, Zhen Dong, Qifeng Liu, Cheng Lin, Ziwei Liu, et al. Diffusion as shader: 3d-aware video diffu- sion for versatile video generation control.arXiv preprint arXiv:2501.03847, 2025. 2, 4, 6, 7

    Pith/arXiv arXiv 2025

  10. [10]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning.arXiv preprint arXiv:2307.04725, 2023

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning.arXiv preprint arXiv:2307.04725, 2023. 2, 3

    Pith/arXiv arXiv 2023

  11. [11]

    Latent video diffusion models for high-fidelity video generation with arbitrary lengths.arXiv preprint arXiv:2211.13221, 2022

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity video generation with arbitrary lengths.arXiv preprint arXiv:2211.13221, 2022. 3

    Pith/arXiv arXiv 2022

  12. [12]

    Prompt-to-prompt image editing with cross-attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-or. Prompt-to-prompt image editing with cross-attention control. InICLR, 2022. 2, 3

    work page 2022

  13. [13]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. InNeurIPS, pages 6840–6851,

  14. [14]

    Imagen video: High definition video generation with diffusion mod- els.arXiv preprint arXiv:2210.02303, 2022

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion mod- els.arXiv preprint arXiv:2210.02303, 2022. 3

    Pith/arXiv arXiv 2022

  15. [15]

    Text2video-zero: Text- to-image diffusion models are zero-shot video generators

    Levon Khachatryan, Andranik Movsisyan, Vahram Tade- vosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text- to-image diffusion models are zero-shot video generators. In ICCV, pages 15954–15964, 2023. 2, 5

    work page 2023

  16. [16]

    Switchlight: Co-design of physics- driven architecture and pre-training framework for human portrait relighting

    Hoon Kim, Minje Jang, Wonjun Yoon, Jisoo Lee, Donghyun Na, and Sanghyun Woo. Switchlight: Co-design of physics- driven architecture and pre-training framework for human portrait relighting. InCVPR, pages 25096–25106, 2024. 3

    work page 2024

  17. [17]

    Repaint: Inpainting using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. InCVPR, pages 11461–11471, 2022. 2

    work page 2022

  18. [18]

    Sdedit: Guided image synthesis and editing with stochastic differential equa- tions.arXiv preprint arXiv:2108.01073, 2021

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jia- jun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equa- tions.arXiv preprint arXiv:2108.01073, 2021. 2, 3, 5

    Pith/arXiv arXiv 2021

  19. [19]

    Null-text inversion for editing real images using guided diffusion models

    Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. InCVPR, pages 6038–6047,

  20. [20]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. InAAAI, pages 4296–4304, 2024. 2

    2024

  21. [21]

    Total relighting: learning to relight portraits for background replacement.ACM TOG, 40 (4):43–1, 2021

    Rohit Pandey, Sergio Orts-Escolano, Chloe Legendre, Chris- tian Haene, Sofien Bouaziz, Christoph Rhemann, Paul E De- bevec, and Sean Ryan Fanello. Total relighting: learning to relight portraits for background replacement.ACM TOG, 40 (4):43–1, 2021. 3

    work page 2021

  22. [22]

    Zero-shot image-to-image translation

    Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu. Zero-shot image-to-image translation. InACM SIGGRAPH Conference Proceedings, pages 1–11, 2023. 2

    work page 2023

  23. [23]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InICCV, pages 4195–4205, 2023. 2, 3

    2023

  24. [24]

    Controlnext: Powerful and effi- cient control for image and video generation.arXiv preprint arXiv:2408.06070, 2024

    Bohao Peng, Jian Wang, Yuechen Zhang, Wenbo Li, Ming- Chang Yang, and Jiaya Jia. Controlnext: Powerful and effi- cient control for image and video generation.arXiv preprint arXiv:2408.06070, 2024. 2

    Pith/arXiv arXiv 2024

  25. [25]

    Fatezero: Fus- ing attentions for zero-shot text-based video editing

    Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. Fatezero: Fus- ing attentions for zero-shot text-based video editing. In ICCV, pages 15932–15942, 2023. 3, 6, 7

    work page 2023

  26. [26]

    Relightful harmonization: Lighting-aware portrait background replacement

    Mengwei Ren, Wei Xiong, Jae Shin Yoon, Zhixin Shu, Jianming Zhang, HyunJoon Jung, Guido Gerig, and He Zhang. Relightful harmonization: Lighting-aware portrait background replacement. InCVPR, pages 6452–6462, 2024. 3

    work page 2024

  27. [27]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. InCVPR, pages 10684– 10695, 2022. 2

    2022

  28. [28]

    FinePhys: Fine-grained hu- man action generation by explicitly incorporating physical laws for effective skeletal guidance

    Dian Shao, Mingfei Shi, Shengda Xu, Haodong Chen, Yon- gle Huang, and Binglu Wang. FinePhys: Fine-grained hu- man action generation by explicitly incorporating physical laws for effective skeletal guidance. InCVPR, pages 1905– 1916, 2025. 2

    work page 1905

  29. [29]

    Make-A-Video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-A-Video: Text-to-video generation without text-video data. InICLR, 2023. 3

    work page 2023

  30. [30]

    Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020. 2, 3

    Pith/arXiv arXiv 2010

  31. [31]

    Cogvideox controlnet extension, 2023

    TheDenk. Cogvideox controlnet extension, 2023. Accessed: 2025-02-12. 2

    work page 2023

  32. [32]

    Plug-and-play diffusion features for text-driven image-to-image translation

    Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image-to-image translation. InCVPR, pages 1921–1930,

    1921

  33. [33]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In ICCV, pages 7623–7633, 2023. 2, 5

    work page 2023

  34. [34]

    Spatialtracker: Tracking any 2d pixels in 3d space

    Yuxi Xiao, Qianqian Wang, Shangzhan Zhang, Nan Xue, Sida Peng, Yujun Shen, and Xiaowei Zhou. Spatialtracker: Tracking any 2d pixels in 3d space. InCVPR, pages 20406– 20417, 2024. 7

    work page 2024

  35. [35]

    Rerender a video: Zero-shot text-guided video-to-video translation

    Shuai Yang, Yifan Zhou, Ziwei Liu, and Chen Change Loy. Rerender a video: Zero-shot text-guided video-to-video translation. InSIGGRAPH Asia Conference, pages 1–11,

    work page

  36. [36]

    Fresco: Spatial-temporal correspondence for zero-shot video translation

    Shuai Yang, Yifan Zhou, Ziwei Liu, and Chen Change Loy. Fresco: Spatial-temporal correspondence for zero-shot video translation. InCVPR, pages 8703–8712, 2024. 3

    work page 2024

  37. [37]

    Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024. 2, 3, 6

    Pith/arXiv arXiv 2024

  38. [38]

    Tfg: Unified training-free guidance for diffusion models

    Haotian Ye, Haowei Lin, Jiaqi Han, Minkai Xu, Sheng Liu, Yitao Liang, Jianzhu Ma, James Zou, and Stefano Ermon. Tfg: Unified training-free guidance for diffusion models. arXiv preprint arXiv:2409.15761, 2024. 2

    Pith/arXiv arXiv 2024

  39. [39]

    Freedom: Training-free energy-guided condi- tional diffusion model

    Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. Freedom: Training-free energy-guided condi- tional diffusion model. InICCV, pages 23174–23184, 2023. 2

    work page 2023

  40. [40]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, pages 3836–3847, 2023. 2

    2023

  41. [41]

    Scaling in-the-wild training for diffusion-based illumination harmo- nization and editing by imposing consistent light transport

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Scaling in-the-wild training for diffusion-based illumination harmo- nization and editing by imposing consistent light transport. InICLR, 2025. 2, 3, 6, 7

    work page 2025

  42. [42]

    Controlvideo: Training-free controllable text-to-video generation.arXiv preprint arXiv:2305.13077, 2023

    Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian. Controlvideo: Training-free controllable text-to-video generation.arXiv preprint arXiv:2305.13077, 2023. 3

    Pith/arXiv arXiv 2023

  43. [43]

    Bilateral refer- ence for high-resolution dichotomous image segmentation

    Peng Zheng, Dehong Gao, Deng-Ping Fan, Li Liu, Jorma Laaksonen, Wanli Ouyang, and Nicu Sebe. Bilateral refer- ence for high-resolution dichotomous image segmentation. arXiv preprint arXiv:2401.03407, 2024. 4

    Pith/arXiv arXiv 2024

  44. [44]

    Open-sora: Democratizing efficient video production for all.arXiv preprint arXiv:2412.20404, 2024

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all.arXiv preprint arXiv:2412.20404, 2024. 2, 3

    Pith/arXiv arXiv 2024

  45. [45]

    Magicvideo: Efficient video generation with latent diffusion models.arXiv preprint arXiv:2211.11018, 2022

    Daquan Zhou, Weimin Wang, Hanshu Yan, Weiwei Lv, Yizhe Zhu, and Jiashi Feng. Magicvideo: Efficient video generation with latent diffusion models.arXiv preprint arXiv:2211.11018, 2022. 3

    Pith/arXiv arXiv 2022

  46. [46]

    Propainter: Improving propagation and transformer for video inpainting

    Shangchen Zhou, Chongyi Li, Kelvin CK Chan, and Chen Change Loy. Propainter: Improving propagation and transformer for video inpainting. InICCV, pages 10477– 10486, 2023. 4, 5

    work page 2023

  47. [47]

    Light-a-video: Training-free video relighting via progressive light fusion.arXiv preprint arXiv:2502.08590, 2025

    Yujie Zhou, Jiazi Bu, Pengyang Ling, Pan Zhang, Tong Wu, Qidong Huang, Jinsong Li, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, et al. Light-a-video: Training-free video relighting via progressive light fusion.arXiv preprint arXiv:2502.08590, 2025. 8

    Pith/arXiv arXiv 2025

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.