Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Training-free Geometric Image Editing on Diffusion Models

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Decoupled diffusion editing wins on move, rotate, and resize tasks.

desk verdict A practical training-free geometric editor with a sensible three-step pipeline, but the 'consistent superiority' claim is not supported by the paper's own Table 1. read the letter →

arxiv 2507.23300 v2 pith:6QXIF3F3 submitted 2025-07-31 cs.CV

Hanshen Zhu , Zhen Zhu , Kaile Zhang , Yiming Gong , Yuliang Liu , Xiang Bai This is my paper
classification cs.CV
keywords geometricimageeditingdiffusionmodelstraining-freetemporalcontextualattentionlocalperturbationinpaintingbenchmarkcontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Geometric image editing—moving, rotating, or resizing an object while keeping the scene coherent—fails under large transformations when it is compressed into a single optimization objective. This paper claims the task should be decoupled into three sequential steps: transform the object, inpaint the vacated source region, and refine the target region. The latter two steps are handled by FreeFine, a training-free diffusion sampler that combines a time-varying blend of mask-guided and ordinary self-attention, locally injected noise, and text-conditioned guidance. On GeoBench, a new benchmark of 811 source images and 5,988 editing instructions spanning 2D, 3D, and structural-completion edits, the authors report that this pipeline outperforms specialized editing and inpainting baselines on fidelity and edit precision. If true, large geometric edits become a scheduling problem for an off-the-shelf diffusion model rather than a training problem.

What carries the argument

The central object is Temporal Contextual Attention (TCA), a mask-aware attention schedule for latent diffusion. At denoising step $\tau$ it computes the output as $(1-\alpha_\tau)\,S_t + \alpha_\tau\,[S_o\cdot M_t + S_b\cdot(1-M_t)]$, where $S_t$ is ordinary self-attention on the current latent, $S_o$ and $S_b$ are masked mutual self-attention outputs using source key/value pairs for the object and background, and $\alpha_\tau = (\tau_1-\tau)/(\tau_1-\tau_0)$ decreases linearly from 1 at $\tau_0$ to 0 at the final step $\tau_1$. This gradual hand-off lets early steps copy global structure from the source image while later steps repair fine details, which the paper argues avoids both the artifacts of pure mask-guided attention and the unwanted global changes of pure self-attention. Two supporting mechanisms carry the local changes: Local Perturbation applies DDPM-style stochastic updates only inside a specified mask while keeping DDIM deterministic updates elsewhere, and Content-specified Generation replaces cross-attention keys/values with text-derived ones and applies classifier-free guidance only inside the target mask. The three modules together turn a standard pretrained latent diffusion model into a region-refinement engine with no training.

What would settle it

Run FreeFine on the full GeoBench while sweeping $\tau_0$ for Step 3 over a range such as 1, 5, 13, 25, and 40, keeping all other settings fixed; if the optimal $\tau_0$ changes across move, rotate, resize, and structural-completion subsets, or if per-task tuned $\tau_0$ erases the reported margin over the strongest baseline, then the fixed schedule is not the mechanism the paper claims. A second check: replace the linear $\alpha_\tau$ with a constant early-stop switch at $\tau_0$; the appendix's few-example comparison would need to fail across the whole benchmark for the smooth schedule to be load-bearing.

Watch

Extended reading notes

Core claim

The paper's central claim is that the reason diffusion-based editing fails on large or structurally complex geometric changes is not a lack of model capacity but a conflict of objectives within a single denoising loop, and that separating the edit into object transformation, source inpainting, and target refinement resolves the conflict. The training-free component, FreeFine, rests on a temporal observation: major content changes happen early in denoising and later steps only refine details. Its Temporal Contextual Attention therefore starts from mask-guided mutual self-attention that borrows key/value features from the source image to preserve global structure, then linearly transitions to unconstrained self-attention over the remaining steps, with the blend weight $\alpha_\tau = (\tau_1 - \tau)/(\tau_1 - \tau_0)$. Local Perturbation adds stochastic DDPM-style updates only inside user-defined masks so that regions needing new structure can re-randomize while the rest of the image stays deterministic, and Content-specified Generation restricts cross-attention and classifier-free guidance to those masks so text prompts affect only intended areas. The paper further argues and demonstrates on GeoBench that this decoupled, training-free recipe achieves higher subject and background consistency, lower warp error, and better FID than state-of-the-art editing and inpainting baselines.

Load-bearing premise

The load-bearing premise is that one fixed linear schedule for blending mask-guided attention with ordinary self-attention, using hand-picked start times (1 for inpainting, 13 for structural completion, 25 for general refinement), transfers across all 5,988 GeoBench instructions; if the ideal blend depends on image content or edit size, the method becomes threshold tuning rather than a general mechanism.

Editorial extensions

If this is right

  • A single off-the-shelf diffusion model can perform move, rotate, resize, and 3D viewpoint edits at arbitrary difficulty without any task-specific fine-tuning.
  • The decoupled pipeline allows each step to use the best available tool: affine transforms for 2D edits, depth estimation or single-image 3D lifting for 3D edits, and the same FreeFine sampler for both inpainting and refinement.
  • Editing quality under large transformations is governed by when in the denoising trajectory mask-guided attention is released, not by the capacity of the inpainting network.
  • Because the modules are training-free, improvements or replacements in the base diffusion model, the segmenter, or the depth estimator transfer directly to the editing pipeline.
  • GeoBench's difficulty levels and structural-completion subset provide a common yardstick for comparing geometric editing methods on 5,988 instructions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the fixed schedule is the true mechanism, then making $\tau_0$ content-adaptive—say, inferred from edit size or mask entropy—should improve results further, an untested extension.
  • The same linear blend could apply to other attention-controlled editing tasks such as object removal or appearance transfer, which the appendix already sketches, suggesting that the schedule is a general control knob rather than a geometric-editing-specific fix.
  • The reliance on hand-picked $\tau_0$ values (1, 13, 25) means the method might need recalibration for a different base diffusion model, resolution, or sampling schedule than the one used in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a decoupled three-step pipeline for geometric image editing: (1) coarse geometric transformation of the object, (2) source-region inpainting, and (3) target-region refinement. Steps 2 and 3 are implemented by a training-free diffusion module called FreeFine, which combines Temporally Contextual Attention (TCA), Local Perturbation (LP), and Content-specified Generation (CG). The authors introduce a new benchmark, GeoBench, with 811 source images and 5,988 editing instructions spanning 2D and 3D transformations, and report comparisons against state-of-the-art editing and inpainting methods. The central claim is that FreeFine outperforms existing alternatives in image fidelity and edit precision, especially under demanding transformations.

Significance. If the central claims hold, the paper would make a useful contribution: a training-free, modular approach to geometric editing that decomposes a hard monolithic optimization into simpler, controllable subtasks, with code and benchmark released. The paper also includes qualitative ablations for each proposed component and a user study, which are positive elements. However, the evidence as presented does not support the strongest claims: the quantitative table contradicts 'consistent superiority across all scenarios and metrics,' and the editing-effectiveness metrics are confounded with the geometric warp performed in Step 1. These issues are addressable with additional analyses and revised claims, but they are load-bearing for the paper's stated message.

major comments (4)
  1. [Sec. 4.2, Table 1] The statement 'FreeFine demonstrates consistent superiority across all scenarios and metrics' is contradicted by the paper's own Table 1. On 2D edits, GeoDiffuser achieves better FID (33.89 vs. 34.72) and better DINOv2 (437.75 vs. 478.18), and DragDiffusion achieves a better KD (0.142 vs. 0.144); on structure completion, BrushNet achieves better KD (0.971 vs. 0.982) and better DINOv2 (2516.52 vs. 2526.38). No error bars or significance tests are reported anywhere in the paper, so even the small 3D improvements (e.g., FID 150.89 vs. 152.06) are not established as statistically reliable. The paper should either revise the superiority claim to a more nuanced statement or add uncertainty quantification (e.g., multiple seeds with confidence intervals, paired significance tests).
  2. [Sec. 4.1 (metrics) and Sec. 3.1 (pipeline)] The two metrics on which FreeFine shows the largest gains, Warp Error (WE) and Mean Distance (MD), are defined directly in terms of the same geometric warp used in Step 1. WE is the L1 distance inside the target mask between the output and the warped source object, and MD compares SIFT keypoints transformed by F with their expected positions. Since Step 1 explicitly produces the coarse image Ic via Tθ and Step 3 initializes from the composite of Ic and Ibg, low WE/MD can be largely inherited from the warp itself rather than from FreeFine's attention and perturbation modules. Baselines such as GeoDiffuser and DragDiffusion do not receive this composite initialization. The authors should provide an ablation that separates the contribution of the warp initialization from the contribution of FreeFine's modules (e.g., evaluate the composite before refinement, or initialize baselines with the same composite), otherwise the editing-effectiveness comparisons are confounded.
  3. [Sec. 3.2.1 and Appendix Fig. 10] TCA's temporal schedule alpha_tau = (tau1 - tau)/(tau1 - tau0) relies on hand-selected starting steps tau0 = 1 for Step 2, tau0 = 13 for Step 3 structural completion, and tau0 = 25 for general refinement. The paper claims that TCA is 'free from this concern and thus more robust' relative to early-stopping, but the only evidence (Appendix Fig. 10) is a few qualitative examples. No sensitivity analysis of tau0 is reported across GeoBench, across edit difficulties, or across different object sizes. Without such an analysis, the claim of robustness over threshold tuning is not supported. Please add a quantitative sweep over tau0 for each task and report how FID, SUBC, WE, and MD vary.
  4. [Sec. 4.3, Fig. 7] The ablations for TCA, LP, and CG are presented only qualitatively. Since the central claim is that each of these components contributes to the method's performance, quantitative ablation results (FID, DINOv2, SUBC, WE, MD, or user-study preferences) should be reported for removing or replacing each component. This is especially important because the qualitative figures show subtle differences that may not be representative across the 5,988-instruction benchmark.
minor comments (5)
  1. [Abstract] The phrase 'outperforms state-of-the-art alternatives in image fidelity, and edit precision' has an awkward comma before 'and'; please rephrase for clarity.
  2. [Appendix E.1] There is a typo: 'cgenetated image' should be 'generated image'.
  3. [Appendix Table 3 caption] The caption reads 'V oting statistics' with a spurious space; should be 'Voting statistics'.
  4. [Sec. 4.1 (Datasets)] The sentence 'We randomly sample 2k images from PIE-Bench and Subjects200K as data from the target' is unclear; specify that these images form the reference distribution for FID and clarify whether they overlap with the GeoBench source images.
  5. [Appendix G, Table 4] The user-study table reports absolute vote counts but not the number of participants per condition; adding confidence intervals or a statistical test for the preference percentages would strengthen the conclusions.

Circularity Check

1 steps flagged · score 2.0 of 10

No derivation-level circularity; the WE/MD edit-precision metric is partly self-referential because it scores against the pipeline's own Step-1 warp.

  1. other [Sec. 3.1 (Step 1 and Step 3, coarse composite equation) and Sec. 4.1 (Metrics, WE/MD definition)]
    "Given the target mask Mt and the coarse image Ic from Step 1, and Ibg with clean background in the source object location obtained from Step 2, we can easilyBlend them together to create a composite image ˆIc: ˆIc = Mt · Ic + (1− Mt) · Ibg. ... we employ the same Warp Error (WE) and Mean Distance(MD) as GeoDiffuser [56] to measure editing effectiveness, which warps the source object to the target location and then computes L1 error within masked regions of the generated images."

    WE is computed against Iw, the warped source object, while Step 1 explicitly produces the coarse image Ic by applying the same transformation Tθ to the source object, and Step 3 is initialized from Îc = Mt·Ic + (1−Mt)·Ibg. Thus the metric's ground truth is the pipeline's own Step-1 output: any variant that simply pastes Ic into the target mask would achieve near-zero WE, and MD likewise checks SIFT points against F, the same transformation used to build Ic. The paper's leading claim of superiority in 'edit precision' therefore rests substantially on agreement with its own initialization rather than on an independent check of the TCA/LP/CG modules.

full rationale

The mechanics of FreeFine (DDIM inversion, MMSA blending with a linear schedule, local DDPM perturbation, and masked CFG) are not fitted to or defined by the evaluation numbers; no evaluation target appears in Eqs. (1)-(6), and the τ0/τ1 schedule is a hand-set hyperparameter rather than a fitted parameter. There is no load-bearing self-citation: the method builds on externally published components (MasaCtrl [5], SV3D [71], GeoDiffuser [56], DepthAnything [75]) and no uniqueness argument by the authors is invoked. The only construction-level overlap found is in the edit-precision metrics: WE/MD compare the output to the Step-1 warp that also seeds Step 3, so the abstract's 'edit precision' superiority is partly inherited from the pipeline's own initialization rather than independently established. This is an evaluation-circularity caveat, not a reduction of the method's derivation to its inputs. Separately, and not as circularity, Table 1 contradicts the sentence claiming 'consistent superiority across all scenarios and metrics' (e.g., GeoDiffuser's 2D FID 33.89 vs FreeFine 34.72; BrushNet's SC KD 0.971 vs 0.982), and Appendix G itself concedes that the WE metric 'shows some degree of misalignment' with human preference; these are reporting and benchmark-validity risks that lower confidence in the conclusions but do not change the circularity verdict.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

No new physical entities are introduced. The method uses three hand-designed modulation mechanisms (TCA, LP, CG) and several hand-set hyperparameters. The central empirical claim rests on the pretrained Stable Diffusion model, the assumed depth geometry for 3D edits, and the author-curated GeoBench, rather than on a mathematical derivation.

free parameters (5)
  • TCA start step tau_0 for Step 2 = 1
    Hand-selected in Section 4.1; controls how quickly the method switches from mask-guided MMSA to self-attention for source region inpainting.
  • TCA start step tau_0 for Step 3 structural completion = 13
    Hand-selected in Section 4.1; balances structural completion and detail preservation.
  • TCA start step tau_0 for general refinement = 25
    Hand-selected in Section 4.1; used for fine-grained adjustments without structural completion.
  • CFG guidance scale w = 7.5
    Default guidance scale in Section 4.1, applied in Eq. (3) for content-specified generation.
  • Mask dilation factor for Step 2 inpainting = 30
    Set in Appendix E.10 for all inpainting methods including FreeFine; affects comparability and object remnant removal.
assumptions (6)
  • domain assumption Stable Diffusion v1-5's pretrained prior can synthesize plausible content in masked regions without training.
    FreeFine relies on the generative prior for inpainting and refinement throughout Sections 3.2.2 to 3.2.3.
  • domain assumption DDIM inversion of the source and composite images preserves enough scene context for mask-guided attention editing.
    Section 3.2 states the source latent is inverted once and reused; this is a core premise of the pipeline.
  • domain assumption The affine or 3D transformation function T_theta accurately realizes the user's edit instruction.
    Step 1 in Section 3.1 assumes the instruction can be converted to a transformation function; the appendix notes angle control is imprecise.
  • domain assumption DepthAnything relative depth plus an assumed camera intrinsic matrix K is sufficient for 3D edits.
    Appendix D.1 acknowledges K is unknown or assumed and depth is relative; the main quantitative 3D comparison uses this depth-based setup.
  • domain assumption GeoBench masks, instructions, difficulty levels, and manual completion masks are valid evaluation ground truth.
    Appendix F describes manual mask selection and manual completion masks; no external validation of benchmark annotations is provided.
  • domain assumption Warp Error and Mean Distance using SIFT and DiFT measure true editing precision.
    Appendix E.1 defines these metrics; the user study section admits WE misaligns with human preference for some baselines.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Training-free Geometric Image Editing on Diffusion Models." pith.science (2026). https://pith.science/paper/6QXIF3F3

@misc{pith2026250723300,
  author       = {Pith},
  title        = {Pith review of: Training-free Geometric Image Editing on Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6QXIF3F3}},
  note         = {Machine review of arXiv:2507.23300}
}
read the original abstract

We tackle the task of geometric image editing, where an object within an image is repositioned, reoriented, or reshaped while preserving overall scene coherence. Previous diffusion-based editing methods often attempt to handle all relevant subtasks in a single step, proving difficult when transformations become large or structurally complex. We address this by proposing a decoupled pipeline that separates object transformation, source region inpainting, and target region refinement. Both inpainting and refinement are implemented using a training-free diffusion approach, FreeFine. In experiments on our new GeoBench benchmark, which contains both 2D and 3D editing scenarios, FreeFine outperforms state-of-the-art alternatives in image fidelity, and edit precision, especially under demanding transformations. Code and benchmark are available at: https://github.com/CIawevy/FreeFine

Figures

Figures reproduced from arXiv: 2507.23300 by the authors.

Figure 1
Figure 1. Given an image and an editing instruction, our method precisely performs geometric edits while maintaining high fidelity and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our geometric image editing pipeline. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison of Context Aggregation Methods. This figure illustrates different approaches for context alignment in image editing [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison with state-of-the-art editing approach. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Visualization results of the user study. Participants pre [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison with state-of-the-art inpainting methods. Notably, our method is [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Ablation studies on the impact of removing individual components from [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Qualitative result on Partial Mask editing tasks. Instead [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Qualitative results on Extended Applications [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Comparisons between the proposed TCA and the [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 12
Figure 12. Figure 12: Visualization results of perceptual study in 2D-edits [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Assessment of the Alignment between Metrics from the Main Paper and User Preferences across Three Dimensions. [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Visualization of failure cases. H. Failure Cases and Limitations While our method achieves strong performance across a va￾riety of geometric editing tasks, it still faces certain failure cases and limitations, which are detailed below. H.1. Failure Cases Failure cases…
Figure 15
Figure 15. Figure 15: Qualitative comparison with state-of-the-art editing methods in moving operations. [PITH_FULL_IMAGE:figures/full_fig_p019_15.png]
Figure 16
Figure 16. Figure 16: Qualitative comparison with state-of-the-art editing methods in scaling operation. [PITH_FULL_IMAGE:figures/full_fig_p020_16.png]
Figure 17
Figure 17. Figure 17: Qualitative comparison with state-of-the-art editing methods in rotation operation. [PITH_FULL_IMAGE:figures/full_fig_p021_17.png]
Figure 18
Figure 18. Figure 18: Qualitative comparison with state-of-the-art inpainting methods in 3D-editing scenarios. Two variants of our method are [PITH_FULL_IMAGE:figures/full_fig_p022_18.png]
Figure 19
Figure 19. Figure 19: Qualitative comparison with state-of-the-art inpainting methods in source region inpainting. [PITH_FULL_IMAGE:figures/full_fig_p023_19.png]
Figure 20
Figure 20. Figure 20: Qualitative comparison with state-of-the-art inpainting methods in target region refinement. [PITH_FULL_IMAGE:figures/full_fig_p024_20.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. InsEdit: Towards Instruction-based Visual Editing via Data-Efficient Video Diffusion Models Adaptation

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    InsEdit adapts a video diffusion backbone for text-instruction video editing via Mutual Context Attention, achieving SOTA open-source results with O(100K) data while also supporting image editing.

Reference graph

Works this paper leans on

82 extracted references · 70 canonical work pages · cited by 1 Pith paper

  1. [1]

    Cross-image attention for zero- shot appearance transfer

    Yuval Alaluf, Daniel Garibi, Or Patashnik, Hadar Averbuch- Elor, and Daniel Cohen-Or. Cross-image attention for zero- shot appearance transfer. In ACM SIGGRAPH, pages 1–12,

  2. [2]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023. 1, 2

    2023

  3. [3]

    Sutherland, Michael Arbel, and Arthur Gretton

    Mikolaj Binkowski, Danica J. Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD gans. In ICLR. OpenReview.net, 2018. 5

    work page 2018

  4. [4]

    G. Bradski. The OpenCV Library. Dr. Dobb’s Journal of Software Tools, 2000. 3

    work page 2000

  5. [5]

    Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xi- aohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing. In ICCV, pages 22503–22513, 2023. 3, 4, 5, 7

    work page 2023

  6. [6]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In ICCV, pages 9630–9640, 2021. 4

    work page 2021

  7. [7]

    Sharegpt4v: Improving large multi-modal models with better captions

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. In ECCV, pages 370–387, 2024. 5

    work page 2024

  8. [8]

    Dressing in order: Recurrent person image generation for pose trans- fer, virtual try-on and outfit editing

    Aiyu Cui, Daniel McKee, and Svetlana Lazebnik. Dressing in order: Recurrent person image generation for pose trans- fer, virtual try-on and outfit editing. In ICCV, pages 14618– 14627. IEEE, 2021. 2

    work page 2021

Show all 82 references
  1. [9]

    Stabledrag: Stable dragging for point-based image editing

    Yutao Cui, Xiaotong Zhao, Guozhen Zhang, Shengming Cao, Kai Ma, and Limin Wang. Stabledrag: Stable dragging for point-based image editing. In ECCV, pages 340–356,

  2. [10]

    Deep image homography estimation

    Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabi- novich. Deep image homography estimation. CoRR, abs/1606.03798, 2016. 2

    2016 arXiv

  3. [11]

    Efros, and Aleksander Holynski

    Dave Epstein, Allan Jabri, Ben Poole, Alexei A. Efros, and Aleksander Holynski. Diffusion self-guidance for control- lable image generation. In NeurIPS, 2023. 1, 2, 6, 4

    2023

  4. [12]

    Generative dif- fusion prior for unified image restoration and enhancement

    Ben Fei, Zhaoyang Lyu, Liang Pan, Junzhe Zhang, Weidong Yang, Tianyue Luo, Bo Zhang, and Bo Dai. Generative dif- fusion prior for unified image restoration and enhancement. In CVPR, pages 9935–9946, 2023. 2

    2023

  5. [13]

    Motion guidance: Diffusion-based image editing with differentiable motion es- timators

    Daniel Geng and Andrew Owens. Motion guidance: Diffusion-based image editing with differentiable motion es- timators. In ICLR, 2024. 1, 2, 6, 4

    2024

  6. [14]

    Focus on your instruction: Fine-grained and multi-instruction image editing by atten- tion modulation

    Qin Guo and Tianwei Lin. Focus on your instruction: Fine-grained and multi-instruction image editing by atten- tion modulation. In CVPR, pages 6986–6996, 2024. 4

    2024

  7. [15]

    Xintong Han, Zuxuan Wu, Zhe Wu, Ruichi Yu, and Larry S. Davis. VITON: an image-based virtual try-on network. In CVPR, pages 7543–7552. Computer Vision Foundation / IEEE Computer Society, 2018. 2

    2018

  8. [16]

    Magicman: Generative novel view synthesis of humans with 3d-aware diffusion and iterative refinement

    Xu He, Zhiyong Wu, Xiaoyu Li, Di Kang, Chaopeng Zhang, Jiangnan Ye, Liyang Chen, Xiangjun Gao, Han Zhang, and Haolin Zhuang. Magicman: Generative novel view synthesis of humans with 3d-aware diffusion and iterative refinement. In AAAI, pages 3437–3445, 2025. 3

    2025

  9. [17]

    Prompt-to-prompt image editing with cross-attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross-attention control. In ICLR, 2023. 3, 4

    2023

  10. [18]

    Style aligned image generation via shared atten- tion

    Amir Hertz, Andrey V oynov, Shlomi Fruchter, and Daniel Cohen-Or. Style aligned image generation via shared atten- tion. In CVPR, pages 4775–4785, 2024. 3, 7

    2024

  11. [19]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. In NeurIPS, pages 6626–6637, 2017. 2, 5, 3

    2017

  12. [20]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. CoRR, abs/2207.12598, 2022. 5

    2022 arXiv

  13. [21]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. In NeurIPS, 2020. 2, 3

    2020

  14. [22]

    VBench: Com- prehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Com- prehensive benchmark suite for video generative models....

    2024

  15. [23]

    Spatial transformer networks

    Max Jaderberg, Karen Simonyan, Andrew Zisserman, and koray kavukcuoglu. Spatial transformer networks. In NeurIPS. Curran Associates, Inc., 2015. 2

    2015

  16. [24]

    Designedit: Multi- layered latent decomposition and fusion for unified & accu- rate image editing

    Yueru Jia, Yuhui Yuan, Aosong Cheng, Chuke Wang, Ji Li, Huizhu Jia, and Shanghang Zhang. Designedit: Multi- layered latent decomposition and fusion for unified & accu- rate image editing. CoRR, abs/2403.14487, 2024. 2, 6, 7, 5

    2024 arXiv

  17. [25]

    Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion

    Xuan Ju, Xian Liu, Xintao Wang, Yuxuan Bian, Ying Shan, and Qiang Xu. Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion. In ECCV, pages 150–168, 2024. 2, 6, 5

    2024

  18. [26]

    Pnp inversion: Boosting diffusion-based editing with 3 lines of code

    Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, and Qiang Xu. Pnp inversion: Boosting diffusion-based editing with 3 lines of code. International Conference on Learning Representations (ICLR), 2024. 5

    2024

  19. [27]

    Denoising diffusion restoration models

    Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. In NeurIPS,

  20. [28]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139:1– 139:14, 2023. 2

    2023

  21. [29]

    Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross B

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chlo´e Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross B. Girshick. Segment anything. In ICCV, pages 3992– 4003, 2023. 3, 6, 5 9

    2023

  22. [30]

    Semantic-sam: Segment and recognize anything at any gran- ularity

    Feng Li, Hao Zhang, Peize Sun, Xueyan Zou, Shilong Liu, Jianwei Yang, Chunyuan Li, Lei Zhang, and Jianfeng Gao. Semantic-sam: Segment and recognize anything at any gran- ularity. arXiv preprint arXiv:2307.04767, 2023. 5

    2023 arXiv

  23. [31]

    Dragapart: Learning a part-level motion prior for articulated objects

    Ruining Li, Chuanxia Zheng, Christian Rupprecht, and An- drea Vedaldi. Dragapart: Learning a part-level motion prior for articulated objects. In ECCV, pages 165–183, 2024. 2

    2024

  24. [32]

    MAT: mask-aware transformer for large hole image in- painting

    Wenbo Li, Zhe Lin, Kun Zhou, Lu Qi, Yi Wang, and Jiaya Jia. MAT: mask-aware transformer for large hole image in- painting. In CVPR, pages 10748–10758, 2022. 2, 6, 5

    2022

  25. [33]

    Towards understanding cross and self-attention in stable diffusion for text-guided image editing

    Bingyan Liu, Chengyu Wang, Tingfeng Cao, Kui Jia, and Jun Huang. Towards understanding cross and self-attention in stable diffusion for text-guided image editing. In CVPR, pages 7817–7826. IEEE, 2024. 3

    2024

  26. [34]

    Drag your noise: Interactive point-based editing via diffusion semantic propagation

    Haofeng Liu, Chenshu Xu, Yifei Yang, Lihua Zeng, and Shengfeng He. Drag your noise: Interactive point-based editing via diffusion semantic propagation. In CVPR, pages 6743–6752, 2024. 2

    2024

  27. [35]

    Zero-1-to-3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3d object. In ICCV, pages 9264–

  28. [36]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In ICLR, 2023. 2

    2023

  29. [37]

    Sift-the scale invariant feature transform

    G Lowe. Sift-the scale invariant feature transform. Int. j,

  30. [38]

    Regiondrag: Fast region-based image editing with diffusion models

    Jingyi Lu, Xinghui Li, and Kai Han. Regiondrag: Fast region-based image editing with diffusion models. InECCV, pages 231–246, 2024. 1, 2, 6, 4

    2024

  31. [39]

    Repaint: Inpainting using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andr´es Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In CVPR, pages 11451–11461, 2022. 2

    2022

  32. [40]

    Goldman, and Aleksander Holynski

    Grace Luo, Trevor Darrell, Oliver Wang, Dan B. Goldman, and Aleksander Holynski. Readout guidance: Learning con- trol from diffusion features. In CVPR, pages 8217–8227,

  33. [41]

    Fourier priors-guided diffusion for zero-shot joint low-light enhance- ment and deblurring

    Xiaoqian Lv, Shengping Zhang, Chenyang Wang, Yichen Zheng, Bineng Zhong, Chongyi Li, and Liqiang Nie. Fourier priors-guided diffusion for zero-shot joint low-light enhance- ment and deblurring. In CVPR, pages 25378–25388, 2024. 2

    2024

  34. [42]

    Pose guided person image genera- tion

    Liqian Ma, Xu Jia, Qianru Sun, Bernt Schiele, Tinne Tuyte- laars, and Luc Van Gool. Pose guided person image genera- tion. In NeurIPS, pages 406–416, 2017. 2

    2017

  35. [43]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. In ECCV, pages 405–421. Springer, 2020. 2

    2020

  36. [44]

    Dragondiffusion: Enabling drag-style manipu- lation on diffusion models

    Chong Mou, Xintao Wang, Jiechong Song, Ying Shan, and Jian Zhang. Dragondiffusion: Enabling drag-style manipu- lation on diffusion models. In ICLR, 2024. 1, 2, 6, 7, 4

    2024

  37. [45]

    Ed- itable image elements for controllable synthesis

    Jiteng Mu, Micha ¨el Gharbi, Richard Zhang, Eli Shechtman, Nuno Vasconcelos, Xiaolong Wang, and Taesung Park. Ed- itable image elements for controllable synthesis. In ECCV, pages 39–56, 2024. 2

    2024

  38. [46]

    Drag your GAN: interactive point-based manipulation on the generative image manifold

    Xingang Pan, Ayush Tewari, Thomas Leimk ¨uhler, Lingjie Liu, Abhimitra Meka, and Christian Theobalt. Drag your GAN: interactive point-based manipulation on the generative image manifold. In ACM SIGGRAPH, pages 78:1–78:11,

  39. [47]

    Karran Pandey, Paul Guerrero, Matheus Gadelha, Yannick Hold-Geoffroy, Karan Singh, and Niloy J. Mitra. Diffusion handles enabling 3d edits for diffusion models by lifting ac- tivations to 3d. In CVPR, pages 7695–7704. IEEE, 2024. 2, 6, 4, 5

    2024

  40. [48]

    SDXL: improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. SDXL: improving latent diffusion models for high-resolution image synthesis. In ICLR, 2024. 1

    2024

  41. [49]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In ICML, pages 8748–...

    2021

  42. [50]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In ICML, pages 8821– 8831, 2021. 2

    2021

  43. [51]

    Hierarchical text-conditional image gener- ation with CLIP latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with CLIP latents. CoRR, abs/2204.06125, 2022. 2

    2022 arXiv

  44. [52]

    Move anything with layered scene diffusion

    Jiawei Ren, Mengmeng Xu, Jui-Chieh Wu, Ziwei Liu, Tao Xiang, and Antoine Toisoul. Move anything with layered scene diffusion. In CVPR, pages 6380–6389, 2024. 2

    2024

  45. [53]

    Grounded SAM: assembling open-world models for diverse visual tasks

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kun- chang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded SAM: assembling open-world models for diverse visual tasks. arXi...

    2024 arXiv

  46. [54]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, pages 10674– 10685, 2022. 1, 2, 5, 6

    2022

  47. [55]

    Denton, Seyed Kamyar Seyed Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Seyed Kamyar Seyed Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with de...

    2022

  48. [56]

    Geodiffuser: Geometry-based image editing with diffusion models

    Rahul Sajnani, Jeroen van Baar, Jie Min, Kapil Katyal, and Srinath Sridhar. Geodiffuser: Geometry-based image editing with diffusion models. In WACV, pages 472–482, 2025. 2, 6, 4, 5

    2025

  49. [57]

    pytorch-fid: FID Score for PyTorch

    Maximilian Seitzer. pytorch-fid: FID Score for PyTorch. https://github.com/mseitzer/pytorch-fid ,

  50. [58]

    Yujun Shi, Chuhui Xue, Jun Hao Liew, Jiachun Pan, Han- shu Yan, Wenqing Zhang, Vincent Y . F. Tan, and Song Bai. Dragdiffusion: Harnessing diffusion models for interactive point-based image editing. In CVPR, 2024. 1, 2, 6, 4 10

    2024

  51. [59]

    Instant- drag: Improving interactivity in drag-based image editing

    Joonghyuk Shin, Daehyeon Choi, and Jaesik Park. Instant- drag: Improving interactivity in drag-based image editing. In ACM SIGGRAPH, pages 39:1–39:10, 2024. 2

    2024

  52. [60]

    Deformable gans for pose-based human im- age generation

    Aliaksandr Siarohin, Enver Sangineto, St ´ephane Lathuili`ere, and Nicu Sebe. Deformable gans for pose-based human im- age generation. In CVPR, pages 3408–3416. Computer Vi- sion Foundation / IEEE Computer Society, 2018. 2

    2018

  53. [61]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In ICLR, 2021. 2, 5, 3

    2021

  54. [62]

    Train stable diffusion for inpainting,

    Lorenzo Stacchio. Train stable diffusion for inpainting,

  55. [63]

    Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Leigh Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L

    George Stein, Jesse C. Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Leigh Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L. Caterini, J. Eric T. Taylor, and Gabriel Loaiza- Ganem. Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion m...

    2023

  56. [64]

    Attentive eraser: Unleashing diffusion model’s object re- moval potential via self-attention redirection guidance

    Wenhao Sun, Xue-Mei Dong, Benlei Cui, and Jingqun Tang. Attentive eraser: Unleashing diffusion model’s object re- moval potential via self-attention redirection guidance. In AAAI, pages 20734–20742, 2025. 2

    2025

  57. [65]

    Resolution-robust large mask inpainting with fourier convolutions

    Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution-robust large mask inpainting with fourier convolutions. In WACV, pages 3172–3182, 2022. 2, 6...

    2022

  58. [66]

    Rethinking the inception ar- chitecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception ar- chitecture for computer vision. In CVPR, 2016. 3

    2016

  59. [67]

    Ominicontrol: Minimal and universal control for diffusion transformer

    Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang. Ominicontrol: Minimal and universal control for diffusion transformer. CoRR, abs/2411.15098,

    arXiv

  60. [68]

    Emergent correspondence from image diffusion

    Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. Emergent correspondence from image diffusion. In NeurIPS, 2023. 4

    2023

  61. [69]

    RAFT: recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. RAFT: recurrent all-pairs field transforms for optical flow. In ICCV, pages 402–419, 2020. 6, 4

    2020

  62. [70]

    Training-free consis- tent text-to-image generation

    Yoad Tewel, Omri Kaduri, Rinon Gal, Yoni Kasten, Lior Wolf, Gal Chechik, and Yuval Atzmon. Training-free consis- tent text-to-image generation. ACM Transactions on Graph- ics (TOG), 43(4):1–18, 2024. 3, 7

    2024

  63. [71]

    SV3D: novel multi-view syn- thesis and 3d generation from a single image using latent video diffusion

    Vikram V oleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. SV3D: novel multi-view syn- thesis and 3d generation from a single image using latent video diffusion. In ECCV, pages 439–457, 2024. 2, 3, 4

    2024

  64. [72]

    Toward characteristic- preserving image-based virtual try-on network

    Bochao Wang, Huabin Zheng, Xiaodan Liang, Yimin Chen, Liang Lin, and Meng Yang. Toward characteristic- preserving image-based virtual try-on network. In ECCV, pages 607–623. Springer, 2018. 2

    2018

  65. [73]

    Feng Wang, Jieru Mei, and Alan L. Yuille. SCLIP: rethink- ing self-attention for dense vision-language inference. In ECCV, pages 315–332, 2024. 5

    2024

  66. [74]

    Dreamclean: Restoring clean image using deep diffusion prior

    Jie Xiao, Ruili Feng, Han Zhang, Zhiheng Liu, Zhantao Yang, Yurui Zhu, Xueyang Fu, Kai Zhu, Yu Liu, and Zheng- Jun Zha. Dreamclean: Restoring clean image using deep diffusion prior. In ICLR, 2024. 3

    2024

  67. [75]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In CVPR, pages 10371–10381, 2024. 2, 3, 6, 4

    2024

  68. [76]

    Diffusion models: A comprehensive survey of methods and applications

    Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Run- sheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming- Hsuan Yang. Diffusion models: A comprehensive survey of methods and applications. ACM Comput. Surv., 56(4):105:1– 105:39, 2024. 2

    2024

  69. [77]

    Image sculpting: Precise object editing with 3d geometry control

    Jiraphon Yenphraphai, Xichen Pan, Sainan Liu, Daniele Panozzo, and Saining Xie. Image sculpting: Precise object editing with 3d geometry control. In CVPR, pages 4241– 4251, 2024. 2

    2024

  70. [78]

    Diffusion self guidance implementation

    Shengzhe Zhou. Diffusion self guidance implementation. 4

  71. [79]

    Denoising diffu- sion models for plug-and-play image restoration

    Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bi- han Wen, Radu Timofte, and Luc Van Gool. Denoising diffu- sion models for plug-and-play image restoration. In CVPR, pages 1219–1229, 2023. 2

    2023

  72. [80]

    Progressive pose attention transfer for person image generation

    Zhen Zhu, Tengteng Huang, Baoguang Shi, Miao Yu, Bofei Wang, and Xiang Bai. Progressive pose attention transfer for person image generation. In CVPR, pages 2347–2356, 2019. 2

    2019

  73. [81]

    empty scene

    Zhen Zhu, Tengteng Huang, Mengde Xu, Baoguang Shi, Wenqing Cheng, and Xiang Bai. Progressive and aligned pose attention transfer for person image generation. IEEE Trans. Pattern Anal. Mach. Intell. , 44(8):4306–4320, 2022. 2 11 Training-Free Geometric Image Editing on Diffusio...

    2022

  74. [82]

    Editing Instruction Generation

    An alternative approach, GroundingSAM [53], offers si- multaneous mask and label generation but shares the same limitations regarding segmentation granularity and error ac- cumulation, often resulting in irrelevant or overly coarse masks. Editing Instruction Generation. To ena...

    2004

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.