Pith. sign in

REVIEW 3 major objections 4 minor 5 references

PartEdit: Fine-Grained Image Editing using Pre-Trained Diffusion Models

T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Pith's one-line read PartEdit is a text-based method for editing individual object parts with pre-trained diffusion models: it learns part tokens whose cross-attention maps localize the part, then blends source and edited features through those maps so only…

desk verdict PartEdit is a genuinely useful combination of token optimization and mask-based blending for text-only part editing; the method holds together, but the 'all metrics' claim outruns the evidence because localization is only validated for one part. read the letter →

arxiv 2502.04050 v2 pith:WO7ZTHCQ submitted 2025-02-06 cs.CV

classification cs.CV
keywords fine-grainedimageeditingobjectpartsdiffusionmodelscross-attentiontokenoptimizationfeatureblendingtext-basedpartbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that text-only editing of individual object parts—heads, torsos, hoods, seats—is achievable with a pre-trained diffusion model, without user-supplied masks and without retraining the generation model. The method trains a special part token so that its cross-attention maps reproduce the part's segmentation mask during denoising; those maps then serve as soft blending masks that confine the edit to the part. The authors construct a part-editing benchmark and report that PartEdit beats previous text-editing methods on all measured metrics and is chosen by users 66–90 percent of the time in paired comparisons.

What carries the argument

The load-bearing mechanism is the optimized part token and its cross-attention maps. A trainable embedding, initialized randomly, is supervised with a binary cross-entropy loss against ground-truth part masks at chosen timesteps and UNet layers; the trained token then produces localization maps at every inference step. Those maps are aggregated across layers, min-max normalized, adaptively thresholded so that background is zeroed and a soft transition band remains, and used to blend source-path and edit-path features at each layer and timestep. This single mechanism both localizes the edit and preserves the unedited image.

What would settle it

Train a part token with the paper's 10–20-image protocol, then run edits on a held-out set of object categories, poses, and scenes that were absent from training. Compare the aggregated inference-time masks against manual part annotations: if mean IoU falls toward chance for small or articulated parts, or if background PSNR and SSIM on the edited images drop below existing mask-free baselines, the localization-transfer assumption fails.

Watch

Extended reading notes

Core claim

Pre-trained diffusion models do not reliably know where object parts are: the cross-attention map for "head" can fire on arms, and "hood" can fail to appear at all. PartEdit counters this by expanding the model's part vocabulary with learned tokens while keeping the model frozen. Each part token is optimized with a binary cross-entropy loss against part masks from existing part datasets or a small user-annotated set, over intermediate timesteps and selected UNet decoder layers. At inference the token's aggregated cross-attention maps become a per-step non-binary mask; an adaptive thresholding step plus per-layer, per-timestep feature blending between the source and edited trajectories applies the new attribute exactly where the part lies. The result is that edits such as "spiderman torso" or "destroyed hood" change only the requested region and blend into the original object.

Load-bearing premise

The whole pipeline depends on a token trained on only 10–20 images producing cross-attention maps that correctly find the requested part on unseen images, prompts, objects, and backgrounds; if that localization transfer fails, the feature blending edits the wrong region.

Editorial extensions

If this is right

  • Any object part for which roughly 10–20 annotated images exist can be edited by text alone; masks are needed only during token training, not at edit time.
  • Because the diffusion model stays frozen, the user keeps the full generative knowledge of the base model and can apply arbitrary new attributes or identities to the trained part without retraining.
  • Edits stay localized: unedited regions are preserved better than with prompt-to-prompt and instruction-based editing methods on the new PartEdit benchmark.
  • Multiple part tokens can be combined at inference to edit several regions in one image, by jointly normalizing their attention maps.
  • The approach works for real photographs when paired with an inversion method and an automatic captioner, not only for generated images.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if the localization transfer holds, part tokens effectively convert segmentation annotations into reusable text-level controls, suggesting that cross-attention can serve as a general-purpose bottleneck for part grounding in editing pipelines.
  • Beyond the paper: the authors' observation that transformer-based diffusion models already localize parts more accurately suggests PartEdit's token-learning mechanism could yield even better results on such backbones.
  • Beyond the paper: the benchmark's separate foreground and background metrics give future editors a concrete evaluation recipe: verify the edit landed where requested and that everything else is unchanged.
  • Beyond the paper: the reported failure to mix a new style into a part points to self-attention as the next target, hinting that style and content disentanglement within the edited region is the main open problem.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. PartEdit proposes a text-based image editing method for object parts using a frozen pre-trained diffusion model (SDXL). The key idea is to optimize a small number of textual part tokens (e.g., <head>, <hood>) by supervising their cross-attention maps with binary cross-entropy against part segmentation masks (Eq. 3). At inference, the optimized tokens are used in an edit prompt, their cross-attention maps are aggregated across layers (Eq. 4), thresholded adaptively with an Otsu-based scheme (Eq. 5), and used to blend features between the source and edit diffusion trajectories (Eq. 6). The paper introduces a new part-editing benchmark (PartEdit-Synth and PartEdit-Real), compares against several text-based and mask-based editing methods, reports quantitative metrics and user studies, and releases code and data. The central claims are that learned part tokens provide reliable localization, that the feature-blending pipeline performs seamless localized edits, and that PartEdit outperforms existing methods on all metrics.

Significance. If the claims hold, PartEdit is a practically useful contribution: it enables fine-grained part edits from text alone, without user-supplied masks, while keeping the base diffusion model frozen. The method is clearly specified, the equations for token optimization, mask aggregation, adaptive thresholding, and feature blending are easy to follow, and the ablations (timestep selection, layer selection, thresholding, padding, number of training images) are informative. The released code, benchmark, and evaluation protocol are valuable assets for follow-up work. The main risk is that the entire editing pipeline depends on the cross-attention localization maps transferring reliably to unseen images and prompts, and that risk is not directly quantified for most of the evaluated parts.

major comments (3)
  1. [Section 4.3, Appendices I and J] The central claim that PartEdit outperforms existing methods depends on the learned tokens producing reliable localization masks on unseen images, because the feature blending of Eq. (6) edits whatever region the tokens localize. However, the paper does not report per-part localization accuracy for the tokens used in the main experiments. Table 1 reports averages over 60 synthetic and 13 real images across seven part types, and Appendix J reports mIoU only for the quadruped-head token and only under SD2.1, not for SDXL. This is a load-bearing gap: parts such as car-hood and chair-seat are trained from only 10 custom-annotated images, and if their localization maps are poor, the edit will be applied to the wrong region while the averaged metrics can still look strong. Please add per-part localization metrics (e.g., mIoU against the annotated masks) for all seven tokens on the evaluation sets, report the per-part breakdown of the edit metrics in Table 1, and discuss which parts drive the aggregate gains.
  2. [Table 1 and Section 4.3] The paper repeatedly states that PartEdit 'outperforms existing editing methods on all metrics,' but Table 1 reports single point estimates without confidence intervals, standard deviations, or significance tests. This matters especially for the real-image benchmark, which contains only 13 images, and for the user studies: a 360-response 2AFC study can give a useful aggregate, but reporting a bare percentage such as 66% or 66.92% without an interval makes it impossible to tell whether the preference over that baseline is distinguishable from chance. Please report variability (per-image or bootstrap CIs) and, where feasible, significance tests for the quantitative metrics and user-study preferences, or temper the 'all metrics' claim accordingly.
  3. [Section 3.3, Eq. (6)] The editing pipeline assumes that the attention mask M_t from Eq. (4) is informative at every denoising step used for blending. The paper shows qualitative attention visualizations and ablations of the thresholding scheme, but it does not validate mask quality as a function of timestep or across different prompts and objects for the actual SDXL tokens. Since Eq. (6) is applied for all timesteps in [1, t_e] and t_e=50 in the default synthetic setting, an inaccurate mask at an early or late timestep can directly corrupt the output. Please include a quantitative analysis of mask quality over the blending horizon, or at least per-part failure cases, so that the reader can assess how often localization transfers to the benchmark distribution.
minor comments (4)
  1. [Eq. (4)] The notation says that for timestep t the mask M_t aggregates attention maps from the previous step t-1, but the indexing is not fully specified for t=1; please clarify the boundary case and align the notation with the inference loop described in the text.
  2. [Introduction] There is a typo in the second paragraph: 'LAIOB-5B' should be 'LAION-5B'.
  3. [Figure 3 caption] The caption says 'feature bending' but the method performs feature blending; please correct the caption to match the terminology used in Eq. (6).
  4. [Appendix H] The user-study instructions are reported in a quotation that appears to have an extra unmatched quotation mark; please check the formatting of the quoted instruction text.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: token supervision, inference blending, and evaluation are distinct and the training/evaluation sets are disjoint.

full rationale

The derivation chain is: (i) optimize part tokens with BCE against ground-truth part masks (Eq. 3); (ii) at inference, aggregate attention maps from those tokens into a blending mask (Eq. 4) with adaptive thresholding (Eq. 5); (iii) blend source/edit features with that mask (Eq. 6); (iv) evaluate foreground/background metrics on a separately annotated benchmark. None of these steps reduces to its input by construction: the token optimization supervises attention maps on training images, while the masks used in Eqs. 4-6 at test time come from the same token evaluated on held-out images, not from the training masks themselves. The paper states that "There is no overlap between the training images used in token optimization and evaluation," so the localization output is not fitted to the evaluation targets. No load-bearing self-citation appears in the method or evaluation; the cited token-optimization and diffusion-editing works are external. The only mild concern is that the PartEdit benchmark is authored by the same group and its part taxonomy matches the trained tokens, but that affects benchmark breadth and generalizability, not logical circularity. Similarly, the appendix's limited mIoU reporting (only quadruped head under SD2.1) is an evidence-strength gap, not a circular reduction. Therefore no circularity is present.

Assumptions & free parameters 8 free parameters · 6 assumptions · 1 invented entities

The central pipeline rests mostly on supervised token training rather than undisclosed fitted constants. The free parameters are hyperparameters chosen by ablations on the same benchmark, not parameters fit to the final metric. The main assumption is that small-sample BCE training on cross-attention maps transfers to unseen prompts and images. No new physical entities are postulated; the part tokens are learned artifacts with held-out evaluation.

free parameters (8)
  • adaptive threshold tolerance omega = omega = 3k/2, where k is the Otsu threshold of the aggregated mask
    Appendix C ablates omega from 0.25k to 1.75k and fixes 1.5k as best visual quality; it controls the size of the blended transition region.
  • feature blending horizon t_e = 50 (T) for benchmark edits; user adjustable 30 to 50
    Section 4.5.1 shows t_e controls edit locality; higher t_e preserves more of the original image, lower t_e allows edits to spread.
  • token optimization timestep interval [t_start, t_end] = [30, 20] (intermediate timesteps)
    Section 3.2 and Figure 4 select intermediate timesteps because they localize both large and small parts across inference timesteps.
  • optimized UNet layer subset L = first 8 decoder blocks (indices 24 to 32)
    Appendix E and Figure 16 pick the layer subset with highest per-layer mIoU; using all layers is more expensive.
  • guidance scale = 7.5 for synthetic edits, 12.5 for real-image edits with Ledits++
    Appendix K states larger guidance increases prompt adherence at the cost of PSNR and SSIM; values chosen as a balance.
  • number of training images per part token = 10 to 20 images
    Appendix B shows mIoU plateaus at 10 to 20 samples for 2000 optimization steps; the localization quality of the central claim depends on this small training set.
  • token padding strategy = background token <BG>
    Appendix D compares padding choices and selects <BG> because it produces the cleanest cross-attention maps during inference.
  • diffusion strength during token training = 0.25
    Appendix K lists strength 0.25 and guidance 7.5 for token optimization; it affects the noise level at which attention maps are supervised.
assumptions (6)
  • standard math Cross-attention maps in a frozen text-to-image diffusion model spatially localize text tokens (Eq. 2).
    Foundational assumption inherited from Prompt-to-Prompt and DAAM literature; Section 2 and Figure 2 rely on it to motivate token training.
  • domain assumption BCE supervision of cross-attention maps at selected timesteps and UNet layers is sufficient to make a token localize an object part at inference.
    Core training premise in Section 3.1; if this transfer fails, the localization masks used in Eq. 6 are wrong.
  • domain assumption Annotated part masks from PASCAL-Part, PartImageNet, or a few user-provided images are accurate supervision for the part being edited.
    Section 3.1 and Appendix I use these datasets without quantifying annotation noise; errors would propagate to token localization.
  • domain assumption Real-image inversion methods (Ledits++, EF-DDPM) and BLIP2 captions produce a source trajectory and source prompt that are faithful enough for feature blending.
    Section 3.4 and the real-image benchmark depend on inversion and captioning accuracy, which the paper does not independently measure.
  • domain assumption SDXL encodes style in self-attention, so blending features across paths preserves original style and cannot mix a new style into the edited part.
    Stated in Limitations and Appendix G as the reason style-changing edits fail; the method's seamless-blend claim relies on this architectural assumption.
  • standard math OTSU thresholding of an aggregated attention mask gives a meaningful binary decomposition of the part region.
    Used in Eq. 5; it is a standard histogram threshold, not derived for this task.
invented entities (1)
  • learned part token embeddings, <part-name> independent evidence
    purpose: Represent an object part in text space and produce cross-attention localization masks at every denoising step.
    The tokens are evaluated on held-out benchmark images and user studies, so their localization capability is falsifiable outside the training set; they are learned embeddings, not new physical entities.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PartEdit: Fine-Grained Image Editing using Pre-Trained Diffusion Models." pith.science (2026). https://pith.science/paper/WO7ZTHCQ

@misc{pith2026250204050,
  author       = {Pith},
  title        = {Pith review of: PartEdit: Fine-Grained Image Editing using Pre-Trained Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WO7ZTHCQ}},
  note         = {Machine review of arXiv:2502.04050}
}
read the original abstract

We present the first text-based image editing approach for object parts based on pre-trained diffusion models. Diffusion-based image editing approaches capitalized on the deep understanding of diffusion models of image semantics to perform a variety of edits. However, existing diffusion models lack sufficient understanding of many object parts, hindering fine-grained edits requested by users. To address this, we propose to expand the knowledge of pre-trained diffusion models to allow them to understand various object parts, enabling them to perform fine-grained edits. We achieve this by learning special textual tokens that correspond to different object parts through an efficient token optimization process. These tokens are optimized to produce reliable localization masks at each inference step to localize the editing region. Leveraging these masks, we design feature-blending and adaptive thresholding strategies to execute the edits seamlessly. To evaluate our approach, we establish a benchmark and an evaluation protocol for part editing. Experiments show that our approach outperforms existing editing methods on all metrics and is preferred by users 66-90% of the time in conducted user studies.

Figures

Figures reproduced from arXiv: 2502.04050 by the authors.

Figure 1
Figure 1. Our approach, PartEdit, enables a wide range of fine-grained edits, allowing users to create highly customizable changes. The edits are seamless, precisely localized, and of high visual quality with no leakage into unedited regions. We present the first text-based image editing approach for object parts based on pre-trained diffusion models. Diffusion-based image editing approaches capitalized on the deep understand… view at source ↗
Figure 2
Figure 2. A visualization for the cross-attention maps of SDXL [ [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An overview of our proposed approach for fine-grained part editing. For an object part [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (26 more)
Figure 4
Figure 4. Figure 4: Impact of timestep choice in the token optimization process. Intermediate timesteps achieve reasonable localization for both big and small parts. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison on synthetic images from the PartEdit benchmark. Our method outperforms both iP2P [ [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison against EF-DDPM [ [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Visualization of editing 2 parts at the same time. Note that the attention maps showcase average cross-attention across all time steps. [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 9
Figure 9. Figure 9: Comparison of an edit under different mask binarization strate [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: A comparison on synthetic benchmark against Latent Blending [ [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: Different edits per image on real image editing using our method. We showcase the versatility of our method, as there is no change in the underlying [PITH_FULL_IMAGE:figures/full_fig_p009_11.png]
Figure 12
Figure 12. Figure 12: Challenging multiple subjects edits (more in fig. [PITH_FULL_IMAGE:figures/full_fig_p009_12.png]
Figure 13
Figure 13. Figure 13: The impact of the number of training samples on mIoU. [PITH_FULL_IMAGE:figures/full_fig_p012_13.png]
Figure 14
Figure 14. Figure 14: Applying the edit "with a Joker <head>" with different choices of hyperparameter [PITH_FULL_IMAGE:figures/full_fig_p013_14.png]
Figure 15
Figure 15. Figure 15: Influence of different token padding strategies during inference on the cross-attention maps. [PITH_FULL_IMAGE:figures/full_fig_p013_15.png]
Figure 16
Figure 16. Figure 16: Analysis of how each layer of the UNet performs in terms of mioU. [PITH_FULL_IMAGE:figures/full_fig_p014_16.png]
Figure 17
Figure 17. Figure 17: Applying different identity edits to the same image. We showcase the versatility of our method, as there is no change in the underlying model; we can [PITH_FULL_IMAGE:figures/full_fig_p015_17.png]
Figure 18
Figure 18. Figure 18: Failure cases. Our approach can not perform an edit in a different style (left) or unreasonable edits (right). [PITH_FULL_IMAGE:figures/full_fig_p016_18.png]
Figure 19
Figure 19. Figure 19: Visualization of the user study layout that we conducted. [PITH_FULL_IMAGE:figures/full_fig_p016_19.png]
Figure 20
Figure 20. Figure 20: Example of configuration for random generation of the edits. [PITH_FULL_IMAGE:figures/full_fig_p017_20.png]
Figure 21
Figure 21. Figure 21: InfEdit [Xu et al. 2024] and PnPInversion [Ju et al. 2024b] on real image setting. “formal <torso>” Original Image Grounded SAM MasaCtrl PartEdit (Ours) “afro <hair>” “black <hood>” “cheetah <head>” “ rusty <seat>” “robotic <head>” Edit Edit Original Image Grounded SA…
Figure 22
Figure 22. Figure 22: Qualitative comparison on synthetic images from the PartEdit benchmark, evaluating Grounded SAM [ [PITH_FULL_IMAGE:figures/full_fig_p018_22.png]
Figure 23
Figure 23. Figure 23: Visualization for the cross-attention maps of FLUX [ [PITH_FULL_IMAGE:figures/full_fig_p018_23.png]
Figure 24
Figure 24. Figure 24: Comparison against ReNoise [Garibi et al. 2024] and TurboEdit [Deutch et al. 2024] on real image setting. Prompt: “A muscular man in a white shirt” with a robotic head with an alien head with a robotic torso with an alien torso Prompt: “A red mustang car parked by the…
Figure 25
Figure 25. Figure 25: DiT-based editing results in a synthetic setting using StableFlow [ [PITH_FULL_IMAGE:figures/full_fig_p019_25.png]
Figure 26
Figure 26. Figure 26: Additional challenging examples of multiple subject edits. [PITH_FULL_IMAGE:figures/full_fig_p020_26.png]
Figure 27
Figure 27. Figure 27: Visualization of the generated image, annotated ground truth, and segmentation masks obtained using Grounded SAM [ [PITH_FULL_IMAGE:figures/full_fig_p021_27.png]
Figure 28
Figure 28. Figure 28: An illustration of our user interface. SIGGRAPH Conference Papers ’25, August 10–14, 2025, Vancouver, BC, Canada [PITH_FULL_IMAGE:figures/full_fig_p021_28.png]
Figure 29
Figure 29. Figure 29: Additional visualization of obtained attention maps across all time steps of the qualitative results under the real setting. [PITH_FULL_IMAGE:figures/full_fig_p022_29.png]
Figure 30
Figure 30. Figure 30: Visualization of masks obtained from Segment Anything (huge) model across the 3 heads for the green provided point. The target indicates what we [PITH_FULL_IMAGE:figures/full_fig_p023_30.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

5 extracted references · 2 canonical work pages

  1. [3]

    In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Photomaker: Customizing realistic human photos via stacked id embedding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 8640–8650. Weifeng Lin, Xinyu Wei, Renrui Zhang, Le Zhuo, Shitian Zhao, Siyuan Huang, Huan Teng, Junlin Xie, Yu Qiao, Peng Gao, et al. 2024. Pixwizard: Versatile image-to-image visual assistant wit...

  2. [5]

    young” and “old

    Clic: Concept learning in context. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 6924–6933. Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. 2022. Photorealistic text-to-image diffusion models with ...

  3. [2022]

    In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 10684–10695. Mehdi Safaee, Aryan Mikaeili, Or Patashnik, Daniel Cohen-Or, and Ali Mahdavi-Amiri

  4. [2023]

    In Proceedings of the IEEE/CVF International Conference on Computer Vision

    Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4015–4026. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning . PMLR, 19730–19742. Zhen Li, Mingdeng ...

  5. [2024]

    arXiv:2408.00735 [cs.CV] https://arxiv.org/abs/2408.00735 Ivan Donadello and Luciano Serafini

    TurboEdit: Text-Based Image Editing Using Few-Step Diffusion Models. arXiv:2408.00735 [cs.CV] https://arxiv.org/abs/2408.00735 Ivan Donadello and Luciano Serafini. 2016. Integration of numeric and symbolic information for semantic image interpretation. Intelligenza Artificiale 10, 1 (2016), 33–47. Dave Epstein, Allan Jabri, Ben Poole, Alexei Efros, and Al...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.