Pith. sign in

REVIEW 4 major objections 5 minor 47 references

Zero-to-Hero: Zero-Shot Initialization Empowering Reference-Based Video Appearance Editing

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Reference-based video appearance editing can be made motion-robust by splitting it into a zero-shot correspondence-guided attention transfer and a per-video learned restoration, yielding a 2.6 dB PSNR gain over the best baseline.

desk verdict A genuinely novel two-stage pipeline for reference-based video editing, with convincing ablations, but the headline 2.6 dB PSNR claim rests on an unreleased benchmark. read the letter →

arxiv 2505.23134 v1 pith:DJ3XYONP submitted 2025-05-29 cs.CV cs.AI

Tongtong Su , Chengyu Wang , Jun Huang , Dongming Lu This is my paper
classification cs.CVcs.AI
keywords reference-basedvideoeditingappearancecross-imageattentiondiffusionfeaturecorrespondencezero-shotinitializationrestorationtemporalconsistencyconditionalmodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that reference-based video appearance editing — changing an object's color, texture, or style from a single edited frame — can be made reliable for videos with large motion by splitting the problem in two. First, a zero-shot stage transfers the reference appearance by using semantic correspondences between the original anchor and target frames to guide cross-image attention during diffusion denoising; this preserves structure but introduces a characteristic degradation of over-saturated colors and blurring. Second, a per-video generative restoration model, trained mainly on the anchor-frame pair, repairs that degradation across the whole video, with an optional second condition from the target frame to handle long-range missing regions. If this two-stage scheme holds, users could edit a video by editing just one frame in any image editor, without relying on optical flow or heavy per-video fine-tuning that overfits appearance. The paper reports a 2.6 dB PSNR improvement over the best baseline on deterministic color-editing metrics.

What carries the argument

The central mechanism is correspondence-guided masked cross-image attention: diffusion features from the anchor and target frames are matched by dot-product similarity to build a correspondence map, and the top-k entries of that map are added into the attention score matrix before softmax, forcing each target token to attend almost exclusively to the matching reference token during denoising (Zero-Stage). Setting k=1 gives accurate transfer but compounds imaging degradation, which the paper treats as a shared, learnable distortion pattern. The Hero-Stage then trains a conditional generative model — a diffusion transformer fine-tuned with LoRA — that takes the degraded zero-shot output as one condition and, in Mode 2, the original target frame as a second condition providing a shortcut for unchanged regions; the two conditions are kept informationally balanced so the model restores edited regions from the reference while copying non-edited regions from the target.

What would settle it

Construct a video with large object motion and a background that changes sharply between the anchor and last frame, train the Hero-Stage only on the anchor pair plus the Mode-2 auxiliary pairs, and measure PSNR on the last frame against ground truth; if the anchor-to-last-frame gap substantially exceeds the paper's reported −4.32 dB, the shared-degradation generalization claim fails. A second direct check is to test a video whose later frames exhibit a degradation type never present in the anchor frame, such as occlusion-induced missing patches, and see whether the trained restoration model fails to repair them.

Watch

Extended reading notes

Core claim

The central claim is that accurate correspondence between the original anchor frame and each target frame, extracted from diffusion features, can guide cross-image attention so that the edited reference's appearance is transferred while the target's structure is preserved — and that the resulting distortion, which appears consistently across frames, can be removed by a holistically learned conditional generative model. Training this restoration model on the anchor-frame pair (degraded zero-shot output to reference) is shown to generalize to later frames, and explicitly conditioning on the original target frame as a second branch (Mode 2) closes the gap for long-range frames with missing background color. The claimed result is that the method outperforms the best reference-based baseline by 2.6 dB PSNR on the deterministic Blender-rendered color-editing dataset, with the best or second-best scores on all other metrics.

Load-bearing premise

The load-bearing premise is that the masked-attention degradation pattern looks similar across all frames, so a restoration model trained on the anchor frame can repair every later frame; the paper itself concedes that long-range frames also suffer a color-missing artifact that is absent from the anchor frame.

Editorial extensions

If this is right

  • A user can perform fine-grained appearance edits by editing a single anchor frame in any image editor, and the edit propagates consistently to all other frames without optical flow or per-video I2V inversion.
  • The zero-shot initialization shortens per-video training: the restoration model converges in far fewer optimization steps than direct target-to-reference fine-tuning, which the ablation shows generalizes poorly beyond the anchor frame.
  • The deterministic Blender-rendered evaluation with multiple appearances lets appearance consistency be scored with PSNR/LPIPS/SSIM per frame instead of semantic CLIP-based scores, giving future comparisons a sharper yardstick.
  • Large-motion videos, which break optical-flow-based propagation and overfit memory-heavy I2V fine-tuning, become tractable because correspondence is computed from the original frames rather than from the edited reference.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the shared-degradation observation holds beyond color editing, the same two-stage recipe — zero-shot attention intervention plus a per-video conditional restoration trained only on an anchored pair — could be applied to other attention-based edits such as texture transfer, local inpainting, or object attribute changes, where only one reference frame has ground truth.
  • The Mode-2 two-condition design suggests a general principle for conditional editing: one branch carries the user's edit, the other carries the original structure, and the model should be trained to allocate responsibility between them; this 'information balance' idea could transfer to other conditional generation tasks with paired edited/original data.
  • The reported 2.6 dB gain is measured on Blender-rendered videos; a natural extension is to build a real-video pseudo-ground-truth set with artist-edited anchor frames and check whether the margin persists under natural motion, occlusions, and non-rigid deformation.
  • The ablation showing pixel-swap initialization degenerates into recoloring predicts a testable tradeoff: intermediate Zero-Stage quality (e.g., different k values) should monotonically improve final restoration up to the point where condition information becomes unbalanced, so measuring final PSNR as a function of k would map the accuracy-degradation frontier.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Zero-to-Hero, a two-stage method for reference-based video appearance editing. In Zero-Stage, the user edits one anchor frame; DIFT-based semantic correspondence between the anchor and each target frame is used to build a top-k mask that guides cross-image attention, producing a zero-shot appearance-transferred intermediate result that suffers from over-saturation and blur. In Hero-Stage, a per-video conditional diffusion restoration model with LoRA is trained on pairs built from the anchor frame, optionally using the original target frame as an explicit condition, to holistically restore the intermediate results. The method is evaluated on a colorization task, a self-built Blender-Color-Edit dataset, and a general editing task, reporting a 2.6 dB PSNR improvement over baselines.

Significance. If the reported results hold, the paper makes a useful contribution: it provides a learning-based remedy for attention-intervention degradation in zero-shot appearance transfer, and it avoids reliance on optical flow or heavy temporal modules by using correspondence-guided cross-image attention. The strengths of the paper include a deterministic Blender-based evaluation with ground-truth frames, a careful ablation of training-pair configurations (Table 3), and a comparison with an alternative pixel-swap initialization (Table 4). The central method is plausible, and the ablations support the importance of the Zero-Stage initialization and of the Mode-2 conditional training. However, the headline empirical claim rests on a small, unreleased dataset, and some reported metrics are measured on training frames, so the quantitative contribution is not yet independently verifiable.

major comments (4)
  1. [§4.1.1 and Table 2] The headline claim of a 2.6 dB PSNR improvement is supported only by the self-built Blender-Color-Edit dataset, which contains five objects with two appearances and two camera paths. The dataset, code, checkpoints, and evaluation pipeline are not released, so the reported gap (Ours 26.7640 vs. I2VEdit 24.1103) cannot be reproduced or checked for robustness to scene and camera choices. Because the General-Edit task in the same table has no ground truth beyond the anchor frame, the only strictly quantitative evidence in Table 2 is tied to this unreleased benchmark. Please release the dataset and code, or supplement the evaluation with a public benchmark and report per-scene results.
  2. [§3.3.1 and Table 3] The paper claims that 'training on the anchor frame has the potential to generalize effectively to all subsequent frames,' but the experimental protocol for testing this claim is ambiguous. Mode 1 uses zero-tgt pairs (Table 1c) built from a series of target frames, and if those same frames are then reported as the 'tgt' column in Table 3, the reported target-frame PSNR partly measures memorization rather than generalization. Please clarify exactly which frames participate in training for each row of Table 3, and report results on temporally held-out target frames for Mode 1 and Mode 2.
  3. [§3.2, §3.3, and §4.1.2] Several hyperparameters that are central to the method are either fixed without sensitivity analysis or not fully specified. The mask size k is set to 1 for all experiments with no ablation, and the DIFT feature timestep and U-Net layer used for correspondence are not stated. The Hero-Stage optimization step count is set to t=400 by default, but no convergence or sensitivity analysis is given. Please provide these details and report robustness to k and to the DIFT extraction settings.
  4. [§4.2.2 and Table 2] No error bars, per-scene breakdowns, or significance tests are reported for any of the main quantitative results. The ablations in Table 3 average three runs, but the main comparison in Table 2 does not report variance. Given the small dataset size and the hyperparameter sweeps used for the baselines, the reported superiority could depend on a few favorable scenes. Please report mean and standard deviation of PSNR/LPIPS/SSIM across scenes and seeds for Table 2.
minor comments (5)
  1. [Abstract and §1] The text uses non-standard capitalization such as 'ZERO-shot' and 'vidEo R estOration' in the abstract and introduction; please use normal case or clearly mark the intended wordplay.
  2. [§4.2.1 and Figure 8] The comparison with the commercial Kling service is anecdotal and lacks version information, prompt settings, or quantitative scores; please describe it as a qualitative case study and specify the access date and settings.
  3. [§4.1.2] The term 'zero-shot initialization' may be misleading because the Hero-Stage is a per-video fine-tuned model; consider using 'training-free initialization' or clarifying that 'zero-shot' refers only to the Zero-Stage propagation.
  4. [§4.3.1 and Table 3] The table compares configurations at different optimization step counts (t=600 for one-condition models and t=400 for Mode 2) to equalize compute, but the choice of these specific counts should be justified, and the final Model 2 result should be reported at the default t=400 in the main table with the same convention.
  5. [References] Please add page numbers or venue details where missing, and ensure that all cited datasets and code links are accessible or state clearly which resources will be released.

Circularity Check

1 steps flagged · score 3.0 of 10

General-Edit PSNR† is measured on the anchor frame used to train Hero-Stage, making that one metric a training-fit score; the headline 2.6 dB improvement is from held-out Blender-Color-Edit frames and is not forced.

  1. fitted input called prediction [Section 3.3 (Hero-Stage training) and Section 4.2.2 / Table 2 (General-Edit PSNR†)]
    "When training on the pair {I^anc_M(k), I_ref}, it certainly guarantees fitting on this anchor frame. ... For General-Edit, there is no ground truth available except for the anchor frame. ... calculations can only be performed on the anchor frame (†) using the reference as ground truth."

    Hero-Stage is optimized to map the Zero-Stage output of the anchor frame to I_ref, so this exact pair is a training example regardless of the data-pair mode. The only General-Edit quantitative metric with ground truth, PSNR†, is computed on this same anchor frame against I_ref. Thus the General-Edit anchor PSNR measures reconstruction of the training target, not propagation to unseen frames; a model that merely memorized the anchor pair would score well by construction. Table 3 corroborates the inflation: after Mode 2 training the anchor PSNR is 28.91 while the last target frame is 25.51. Because Table 2 presents this anchor PSNR as a comparative result, one reported 'prediction' reduces to a training fit.

full rationale

The central derivation chain is otherwise self-contained. Hero-Stage is a per-video conditional restoration model trained only on the anchor pair, and the Colorization and Blender-Color-Edit evaluations compute PSNR/LPIPS/SSIM on all frames against independent ground truth, so generalization to non-anchor frames is genuinely tested. The 2.6 dB headline gap (Table 2, Blender-Color-Edit: Ours 26.7640 vs I2VEdit 24.1103) is not circular, although reproducibility is currently limited because the dataset and code are unreleased. The paper also cites its own prior work [16] for the mechanistic explanation of attention-induced over-saturation, but the degradation is directly observed and the method's success is established by ablations, so this self-citation is not load-bearing. The one substantive circular element is the General-Edit anchor-frame PSNR (†), which is a training-set fit; this raises the score above 2 but does not invalidate the central claim.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the quality of DIFT correspondence, the spatial alignment of user references, and the assumption that anchor-frame degradation generalizes across the video. No new physical or mathematical entities are introduced.

free parameters (3)
  • k (top-k correspondence entries for attention mask) = 1
    Set to 1 for all experiments, chosen after qualitative inspection of Figure 3. The paper states it is challenging to find an appropriate k and reports no sensitivity analysis.
  • DIFT feature timestep and U-Net layer = not reported
    Section 3.1 says DIFT at 'certain timesteps and U-Net layers' best represent correspondence, but the specific choices are never disclosed, so correspondence quality depends on undisclosed settings.
  • Hero-Stage optimization steps t = 400 (Mode 2), 600 (single condition)
    Training duration is a practical choice affecting convergence; the ablation compares methods under matched time but does not sweep t.
assumptions (4)
  • domain assumption DIFT features provide accurate semantic correspondence between anchor and target frames under large motion.
    Invoked in Section 3.2 to construct Corr for masked attention; cited to prior work but not re-validated for videos with large motion and dynamic backgrounds.
  • domain assumption The user-provided reference frame is spatially aligned with the anchor frame.
    Section 3 states the anchor is edited 'exclusively at the appearance level (ensuring spatial alignment, e.g., using ComfyUI with Canny ControlNet)', making all subsequent propagation contingent on this alignment.
  • ad hoc to paper Zero-Stage degradation on the anchor frame is representative of degradation on all other frames.
    Section 1: 'Zero-Stage exhibits a similar pattern of degradation across all frames, indicating that training on the anchor frame has the potential to generalize...' The paper later concedes long-range frames add color-missing artifacts, so the assumption only holds approximately.
  • domain assumption Mode 2's explicit target-frame condition provides a shortcut for unchanged regions without leaking appearance into edited regions.
    Section 3.3.2 argues the trained model 'can clearly differentiate their respective roles' based on qualitative results; no quantitative leakage measure is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Zero-to-Hero: Zero-Shot Initialization Empowering Reference-Based Video Appearance Editing." pith.science (2026). https://pith.science/paper/DJ3XYONP

@misc{pith2026250523134,
  author       = {Pith},
  title        = {Pith review of: Zero-to-Hero: Zero-Shot Initialization Empowering Reference-Based Video Appearance Editing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DJ3XYONP}},
  note         = {Machine review of arXiv:2505.23134}
}
read the original abstract

Appearance editing according to user needs is a pivotal task in video editing. Existing text-guided methods often lead to ambiguities regarding user intentions and restrict fine-grained control over editing specific aspects of objects. To overcome these limitations, this paper introduces a novel approach named {Zero-to-Hero}, which focuses on reference-based video editing that disentangles the editing process into two distinct problems. It achieves this by first editing an anchor frame to satisfy user requirements as a reference image and then consistently propagating its appearance across other frames. We leverage correspondence within the original frames to guide the attention mechanism, which is more robust than previously proposed optical flow or temporal modules in memory-friendly video generative models, especially when dealing with objects exhibiting large motions. It offers a solid ZERO-shot initialization that ensures both accuracy and temporal consistency. However, intervention in the attention mechanism results in compounded imaging degradation with over-saturated colors and unknown blurring issues. Starting from Zero-Stage, our Hero-Stage Holistically learns a conditional generative model for vidEo RestOration. To accurately evaluate the consistency of the appearance, we construct a set of videos with multiple appearances using Blender, enabling a fine-grained and deterministic evaluation. Our method outperforms the best-performing baseline with a PSNR improvement of 2.6 dB. The project page is at https://github.com/Tonniia/Zero2Hero.

Figures

Figures reproduced from arXiv: 2505.23134 by the authors.

Figure 1
Figure 1. Left: Our reference-based editing method enables users to precisely edit appearances by incorporating complex layouts of color with arbitrary tools such as Photoshop or ComfyUI to create references, then consistently propagate these edits to subsequent frames. Right: Our approach supports all spatially-aligned appearance editing, including texture and style. Abstract Appearance editing according to user needs is a p… view at source ↗
Figure 2
Figure 2. Our framework. Zero-Stage: Correspondences (Corr) estimated from the anchor and target frames are utilized to guide Cross-image Attention (Attn) between the reference and anchor frames, enabling accurate appearance transfer in a zero-shot manner. Hero-Stage: We learn a conditional generative model by incorporating LoRA to process conditional tokens. There are two modes of condition injection: one condition with one … view at source ↗
Figure 3
Figure 3. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Mode 1 (a+c) can better preserve target structure of car [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Blender-Color-Edit dataset, rendered in Blender. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Qualitative results on General-Edit dataset. Our method maintains the highest consistent fidelity to reference appearance and [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Qualitative results on Blender-Color-Edit dataset. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Qualitative results on General-Edit dataset. [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 29 canonical work pages

  1. [1]

    Self-rectifying diffusion sampling with perturbed-attention guidance

    Donghoon Ahn, Hyoungwon Cho, Jaewon Min, Wooseok Jang, Jungwoo Kim, SeonHwa Kim, Hyun Hee Park, Kyong Hwan Jin, and Seungry- ong Kim. Self-rectifying diffusion sampling with perturbed-attention guidance. InEuropean Confer- ence on Computer Vision, pages 1–17. Springer, 2024. 2

    work page 2024

  2. [2]

    Cross-image attention for zero-shot appearance transfer

    Yuval Alaluf, Daniel Garibi, Or Patashnik, Hadar Averbuch-Elor, and Daniel Cohen-Or. Cross-image attention for zero-shot appearance transfer. InACM SIGGRAPH 2024 Conference Papers, pages 1–12,

    work page 2024

  3. [3]

    black-forest labs. Flux. [Online]https : / / github . com / black - forest - labs / flux,

    work page

  4. [4]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023. 2

    arXiv 2023

  5. [5]

    Pixart- alpha: Fast training of diffusion transformer for pho- torealistic text-to-image synthesis.arXiv preprint arXiv:2310.00426, 2023

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart- alpha: Fast training of diffusion transformer for pho- torealistic text-to-image synthesis.arXiv preprint arXiv:2310.00426, 2023. 3

    arXiv

  6. [6]

    Style injection in diffusion: A training-free approach for adapting large-scale diffusion models for style trans- fer

    Jiwoo Chung, Sangeek Hyun, and Jae-Pil Heo. Style injection in diffusion: A training-free approach for adapting large-scale diffusion models for style trans- fer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8795–8805, 2024. 2, 3

    work page 2024

  7. [7]

    Flat- ten: optical flow-guided attention for consistent text- to-video editing.arXiv preprint arXiv:2310.05922,

    Yuren Cong, Mengmeng Xu, Christian Simon, Shoufa Chen, Jiawei Ren, Yanping Xie, Juan-Manuel Perez- Rua, Bodo Rosenhahn, Tao Xiang, and Sen He. Flat- ten: optical flow-guided attention for consistent text- to-video editing.arXiv preprint arXiv:2310.05922,

    arXiv

  8. [8]

    Diffusion self-guidance for controllable image generation.Advances in Neu- ral Information Processing Systems, 36:16222–16239,

    Dave Epstein, Allan Jabri, Ben Poole, Alexei Efros, and Aleksander Holynski. Diffusion self-guidance for controllable image generation.Advances in Neu- ral Information Processing Systems, 36:16222–16239,

    work page

Show all 47 references
  1. [9]

    Ccedit: Creative and controllable video editing via diffusion models

    Ruoyu Feng, Wenming Weng, Yanhui Wang, Yuhui Yuan, Jianmin Bao, Chong Luo, Zhibo Chen, and Baining Guo. Ccedit: Creative and controllable video editing via diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 6712–6722, 2024. 2

    2024

  2. [10]

    Tokenflow: Consistent diffusion fea- tures for consistent video editing.arXiv preprint arXiv:2307.10373, 2023

    Michal Geyer, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Tokenflow: Consistent diffusion fea- tures for consistent video editing.arXiv preprint arXiv:2307.10373, 2023. 2

    2023 arXiv

  3. [11]

    Prompt- to-prompt image editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aber- man, Yael Pritch, and Daniel Cohen-Or. Prompt- to-prompt image editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022. 3

    2022 arXiv

  4. [12]

    Vbench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pa...

    2024

  5. [13]

    Rave: Ran- domized noise shuffling for fast and consistent video editing with diffusion models

    Ozgur Kara, Bariscan Kurtkaya, Hidir Yesiltepe, James M Rehg, and Pinar Yanardag. Rave: Ran- domized noise shuffling for fast and consistent video editing with diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 6507–6...

    2024

  6. [14]

    Gaussian deblur controlnet

    kohya ss. Gaussian deblur controlnet. [On- line]https://huggingface.co/kohya-ss/ controlnet-lllite, 2023. 3

    2023

  7. [15]

    Anyv2v: A plug-and-play framework for any video-to-video editing tasks.arXiv preprint arXiv:2403.14468, 2024

    Max Ku, Cong Wei, Weiming Ren, Huan Yang, and Wenhu Chen. Anyv2v: A plug-and-play framework for any video-to-video editing tasks.arXiv preprint arXiv:2403.14468, 2024. 2, 3, 7

    2024 arXiv

  8. [16]

    Under- standing attention mechanism in video diffusion mod- els.arXiv preprint arXiv:2504.12027, 2025

    Bingyan Liu, Chengyu Wang, Tongtong Su, Huan Ten, Jun Huang, Kailing Guo, and Kui Jia. Under- standing attention mechanism in video diffusion mod- els.arXiv preprint arXiv:2504.12027, 2025. 2, 5

    2025 arXiv

  9. [17]

    Stablev2v: Stablizing shape con- sistency in video-to-video editing.arXiv preprint arXiv:2411.11045, 2024

    Chang Liu, Rui Li, Kaidong Zhang, Yunwei Lan, and Dong Liu. Stablev2v: Stablizing shape con- sistency in video-to-video editing.arXiv preprint arXiv:2411.11045, 2024. 2, 3

    2024

  10. [18]

    Colorization controlnet

    lllyasviel. Colorization controlnet. [Online] https : / / huggingface . co / lllyasviel / sd _ control _ collection / blob / main / ioclab_sd15_recolor.safetensors, 2023. 3

    2023

  11. [19]

    Tile controlnet

    lllyasviel. Tile controlnet. [Online]https : / / huggingface . co / lllyasviel / ControlNet - v1 - 1 / blob / main / control _ v11f1e_sd15_tile.pth, 2023. 3

    2023

  12. [20]

    Diffusion hyperfea- tures: Searching through time and space for seman- tic correspondence.Advances in Neural Information Processing Systems, 36, 2024

    Grace Luo, Lisa Dunlap, Dong Huk Park, Aleksander Holynski, and Trevor Darrell. Diffusion hyperfea- tures: Searching through time and space for seman- tic correspondence.Advances in Neural Information Processing Systems, 36, 2024. 2, 3, 4

    2024

  13. [21]

    Dragondiffusion: Enabling drag- style manipulation on diffusion models.arXiv preprint arXiv:2307.02421, 2023

    Chong Mou, Xintao Wang, Jiechong Song, Ying Shan, and Jian Zhang. Dragondiffusion: Enabling drag- style manipulation on diffusion models.arXiv preprint arXiv:2307.02421, 2023. 2

    2023 arXiv

  14. [22]

    Diffeditor: Boosting accuracy and flexibility on diffusion-based image editing

    Chong Mou, Xintao Wang, Jiechong Song, Ying Shan, and Jian Zhang. Diffeditor: Boosting accuracy and flexibility on diffusion-based image editing. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8488–8497,

  15. [23]

    T2i- adapter: Learning adapters to dig out more control- lable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i- adapter: Learning adapters to dig out more control- lable ability for text-to-image diffusion models. In Proceedings of the AAAI conference on artificial in- telligence, pages 4296–430...

    2024

  16. [24]

    I2vedit: First-frame-guided video edit- ing via image-to-video diffusion models

    Wenqi Ouyang, Yi Dong, Lei Yang, Jianlou Si, and Xingang Pan. I2vedit: First-frame-guided video edit- ing via image-to-video diffusion models. InSIG- GRAPH Asia 2024 Conference Papers, pages 1–11,

    2024

  17. [25]

    Swapping autoencoder for deep image manipulation

    Taesung Park, Jun-Yan Zhu, Oliver Wang, Jingwan Lu, Eli Shechtman, Alexei Efros, and Richard Zhang. Swapping autoencoder for deep image manipulation. Advances in Neural Information Processing Systems, 33:7198–7211, 2020. 2

    2020

  18. [26]

    Scalable diffu- sion models with transformers

    William Peebles and Saining Xie. Scalable diffu- sion models with transformers. InProceedings of the IEEE/CVF international conference on computer vi- sion, pages 4195–4205, 2023. 3

    2023

  19. [27]

    Fatezero: Fusing attentions for zero-shot text-based video editing

    Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. Fatezero: Fusing attentions for zero-shot text-based video editing. InProceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 15932–15942, 2023. 3

    2023

  20. [28]

    Unicontrol: A unified diffusion model for controllable visual gener- ation in the wild.arXiv preprint arXiv:2305.11147,

    Can Qin, Shu Zhang, Ning Yu, Yihao Feng, Xinyi Yang, Yingbo Zhou, Huan Wang, Juan Carlos Niebles, Caiming Xiong, Silvio Savarese, et al. Unicontrol: A unified diffusion model for controllable visual gener- ation in the wild.arXiv preprint arXiv:2305.11147,

    arXiv

  21. [29]

    High- resolution image synthesis with latent diffusion mod- els

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High- resolution image synthesis with latent diffusion mod- els. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022. 3

    2022

  22. [30]

    Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020. 3

    2010 arXiv

  23. [31]

    Ominicontrol: Minimal and universal control for diffusion transformer.arXiv preprint arXiv:2411.15098, 2024

    Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang. Ominicontrol: Minimal and universal control for diffusion transformer.arXiv preprint arXiv:2411.15098, 2024. 3, 6

    2024 arXiv

  24. [32]

    Ominicontrol2: Effi- cient conditioning for diffusion transformers.arXiv preprint arXiv:2503.08280, 2025

    Zhenxiong Tan, Qiaochu Xue, Xingyi Yang, Songhua Liu, and Xinchao Wang. Ominicontrol2: Effi- cient conditioning for diffusion transformers.arXiv preprint arXiv:2503.08280, 2025. 3

    2025 arXiv

  25. [33]

    Emergent correspondence from image diffusion.Advances in Neural Information Processing Systems, 36: 1363–1389, 2023

    Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. Emergent correspondence from image diffusion.Advances in Neural Information Processing Systems, 36: 1363–1389, 2023. 2, 3, 4

    2023

  26. [34]

    Training- free consistent text-to-image generation.ACM Trans- actions on Graphics (TOG), 43(4):1–18, 2024

    Yoad Tewel, Omri Kaduri, Rinon Gal, Yoni Kasten, Lior Wolf, Gal Chechik, and Yuval Atzmon. Training- free consistent text-to-image generation.ACM Trans- actions on Graphics (TOG), 43(4):1–18, 2024. 2

    2024

  27. [35]

    Splicing vit features for semantic appearance transfer

    Narek Tumanyan, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Splicing vit features for semantic appearance transfer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10748–10757, 2022. 2

    2022

  28. [36]

    Plug-and-play diffusion features for text-driven image-to-image translation

    Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image-to-image translation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 1921–1930, 2023. 3

    1921

  29. [37]

    Cove: Unleashing the diffu- sion feature correspondence for consistent video edit- ing.arXiv preprint arXiv:2406.08850, 2024

    Jiangshan Wang, Yue Ma, Jiayi Guo, Yicheng Xiao, Gao Huang, and Xiu Li. Cove: Unleashing the diffu- sion feature correspondence for consistent video edit- ing.arXiv preprint arXiv:2406.08850, 2024. 1, 2, 4

    2024 arXiv

  30. [38]

    Gmflow: Learning op- tical flow via global matching

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, and Dacheng Tao. Gmflow: Learning op- tical flow via global matching. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8121–8130, 2022. 2

    2022

  31. [39]

    Freetuner: Any subject in any style with training-free diffusion.arXiv preprint arXiv:2405.14201, 2024

    Youcan Xu, Zhen Wang, Jun Xiao, Wei Liu, and Long Chen. Freetuner: Any subject in any style with training-free diffusion.arXiv preprint arXiv:2405.14201, 2024. 3

    2024 arXiv

  32. [40]

    Rerender a video: Zero-shot text-guided video- to-video translation

    Shuai Yang, Yifan Zhou, Ziwei Liu, and Chen Change Loy. Rerender a video: Zero-shot text-guided video- to-video translation. InSIGGRAPH Asia 2023 Con- ference Papers, pages 1–11, 2023. 1, 2, 3

    2023

  33. [41]

    Fresco: Spatial-temporal correspondence for zero-shot video translation

    Shuai Yang, Yifan Zhou, Ziwei Liu, and Chen Change Loy. Fresco: Spatial-temporal correspondence for zero-shot video translation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 8703–8712, 2024. 1, 2, 3, 6

    2024

  34. [42]

    Colormnet: A memory-based deep spatial- temporal feature propagation network for video col- orization

    Yixin Yang, Jiangxin Dong, Jinhui Tang, and Jin- shan Pan. Colormnet: A memory-based deep spatial- temporal feature propagation network for video col- orization. InEuropean Conference on Computer Vi- sion, pages 336–352. Springer, 2024. 3

    2024

  35. [43]

    Bistnet: Seman- tic image prior guided bidirectional temporal feature fusion for deep exemplar-based video colorization

    Yixin Yang, Jinshan Pan, Zhongzheng Peng, Xiaoyu Du, Zhulin Tao, and Jinhui Tang. Bistnet: Seman- tic image prior guided bidirectional temporal feature fusion for deep exemplar-based video colorization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 3

    2024

  36. [44]

    A tale of two features: Stable dif- fusion complements dino for zero-shot semantic cor- respondence.Advances in Neural Information Pro- cessing Systems, 36, 2024

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Polania Cabrera, Varun Jampani, Deqing Sun, and Ming-Hsuan Yang. A tale of two features: Stable dif- fusion complements dino for zero-shot semantic cor- respondence.Advances in Neural Information Pro- cessing Systems, 36, 2024. 2, 3, 4

    2024

  37. [45]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. InProceedings of the IEEE/CVF interna- tional conference on computer vision, pages 3836– 3847, 2023. 2, 3, 6

    2023

  38. [46]

    I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion mod- els.arXiv preprint arXiv:2311.04145, 2023

    Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou. I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion mod- els.arXiv preprint arXiv:2311.04145, 2023. 2, 3

    2023 arXiv

  39. [47]

    Easycontrol: Adding efficient and flexible control for diffusion transformer.arXiv preprint arXiv:2503.07027, 2025

    Yuxuan Zhang, Yirui Yuan, Yiren Song, Haofan Wang, and Jiaming Liu. Easycontrol: Adding efficient and flexible control for diffusion transformer.arXiv preprint arXiv:2503.07027, 2025. 3, 6, 7

    2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.