{"id":"2f9acd38-a09d-4b61-b09e-af8d8f8d01ff","arxiv_id":"2412.20062","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"MADiff improves fashion image editing by adding a mask-prediction network and an attention-based noise refinement step to a stable diffusion editing pipeline.","lead":"MADiff is a two-stage system for text-guided fashion image editing: MaskNet predicts which part of the garment should be edited, and an attention-based processor strengthens the diffusion model's response to the target description. The paper also contributes Fashion-E, a new dataset of about 28,000 training pairs and 2,639 evaluation images covering color, detail, material, and comprehensive editing tasks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Attention Processor causal premise is untested: no ablation isolates attention-selected noise from random noise, and no post-edit attention measurement confirms the claimed mechanism.","rationale":"The reader's weakest-assumption is in the right neighborhood, but the concern is best stated as a missing causal control rather than a bare semantic assumption. Even if high attention correlates with edit success, the Attention Processor's design has no ablation that isolates attention-selected pixels from randomly selected pixels, and no measurement that the refined noise map actually increases attention in the final edited trajectory. Since the paper's CLIP-T lead over IP2P is only 0.46, a small unresolved mechanism could be the whole difference. I also noted two additional reproducibility issues: Eq. 2 appears inconsistent with standard DDIM sampling, and the N/2-th highest value of a 16x16 attention map is not well-defined when the mask contains more than 256 pixels unless an alignment step is omitted. These are real but secondary; the proposed randomized-selection experiment would settle whether the core attention-enhanced claim is substantive. The paper has strengths: a clear pipeline, an ablation demonstrating that the processor helps on their benchmark, and a new dataset. Thus the appropriate verdict remains CONDITIONAL, unchanged from the reader's assessment.","tokens_in":7337,"tokens_out":19255,"duration_ms":210759,"concrete_test":"With MaskNet and all hyperparameters fixed, rerun the Fashion-E evaluation replacing the Attention Processor's top-attention selection with (a) random selection of N/2 pixels from xno_S and (b) uniform selection over the mask; report CLIP-T, CLIP-I, and LPIPS with bootstrap confidence intervals. Additionally, compute the averaged 16x16 attention map of the final sampling from the refined noise map and compare it with the no-AP baseline. If the random-selection version matches the reported CLIP-T within the interval, or if refined-noise attention is not above baseline, the attention-selection premise is not the cause of the headline improvement.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step in the editing-magnitude claim is the Attention Processor (Section II-B, Eqs. 4-5), which copies the N/2 highest-attention pixels from xno_S into the masked region of xS. The only support is the Fig. 3 correlation that averaged 16x16 attention maps are higher for successful black-to-red/yellow edits on 200 t-shirts. This is a single-task correlation, and it does not show that attention-selected noise, rather than any target-prompt-sampled noise, is responsible for the CLIP-T gain. The Table II ablation removes the whole Attention Processor, so it cannot separate 'attention selection matters' from 'any noise replacement inside the mask matters.' Furthermore, the paper claims the method increases attention map values, but it never computes attention after forming the refined noise map; A is taken from the xno_S trajectory before substitution. If random or uniform replacement of masked noise reproduces the 0.46 CLIP-T gain over IP2P, the central 'attention-enhanced' contribution is unsupported and the method reduces to a mask-guided noise swap. The paper also does not specify how a 16x16 attention map is aligned with a mask of N noise-map pixels, which obstructs reproduction.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MADiff, a training-free diffusion-based pipeline for text-guided fashion image editing. It consists of MaskNet, a lightweight UNet that predicts the editing-region mask from foreground segmentation, DensePose, and a shape-only mask prompt extracted by LLAMA3-8b, and an Attention-Enhanced Diffusion Model in which an Attention Processor selects the highest-attention pixels from a target-prompt DDIM sampling trajectory and copies them into the masked region of the inversion noise map before blended DDIM sampling. The authors also introduce Fashion-E, a fashion-image-editing benchmark with 29,380 training pairs (from Fashion-Gen plus cloth masks) and 2,639 evaluation images across color, detail, material, and comprehensive editing tasks. On this benchmark, MADiff reports the best CLIP-T (29.20), CLIP-I (90.37), and LPIPS (0.137) among six baselines. The central claims are that MaskNet improves editing-region localization and the Attention Processor strengthens editing magnitude.","tokens_in":7581,"tokens_out":2451,"duration_ms":27028,"significance":"If the reported results hold, the paper makes two practically useful contributions: a mask predictor that leverages shape-related language to localize edit regions, and a training-free attention-based noise-swapping mechanism that can be grafted onto Stable Diffusion editing pipelines. The Fashion-E benchmark, with task-specific target texts and per-task annotation, is a potentially valuable resource for the fashion-editing community, and the paper explicitly releases the evaluation protocol. The method is plausible and the qualitative examples suggest genuine improvements in editing magnitude. However, the significance is tempered by the absence of error bars or significance tests in the headline comparison, the lack of an isolated test of the attention-selection mechanism, and several missing implementation details that hinder independent verification. The contribution is incremental rather than paradigm-shifting: it combines known components (mask guidance, DDIM inversion, attention-map manipulation) in a new task-specific configuration, but the fashion-domain adaptation and the benchmark are the main novel elements.","major_comments":[{"comment":"The paper's central claim that the Attention Processor strengthens editing magnitude via attention-weighted noise selection is not directly supported by the ablation. Table II removes the entire Attention Processor, so it cannot distinguish 'attention selection matters' from 'replacing masked noise with any target-prompt-sampled noise matters.' The only evidence for the attention mechanism is the 200-image black-t-shirt probe in Fig. 3, which is a correlation on a single task and does not control for prompt-dependent content. I request an ablation that replaces the top-attention pixels with (a) randomly chosen pixels from xno_S and (b) uniformly selected pixels inside the mask, keeping all other components fixed. If those variants reproduce the CLIP-T gain, the 'attention-enhanced' attribution in the title and abstract is unsupported and the method reduces to a mask-guided noise swap.","section":"§II-B, Eqs. (4)–(5) and Table II"},{"comment":"The mechanism by which attention increases editing magnitude is asserted but never measured. The paper states that high attention-map values correlate with successful edits, yet it never computes attention after forming the refined noise map; the attention map A is taken from the xno_S trajectory before substitution. The claim that the refined noise map 'with higher value of attention map' is obtained (Section II-B) is therefore untested. Please report attention-map values (or another direct measure of attention enhancement) before and after the noise substitution, or explicitly reframe the contribution as noise swapping without an attention-enhancement mechanism.","section":"§II-B, Attention Processor and Eq. (5)"},{"comment":"The quantitative comparison reports single-point metrics with no error bars, no significance tests, and no per-task breakdown across the four Fashion-E task types. The differences that support the headline claims are small in some cases (LPIPS 0.137 vs. 0.139 for PnP), and the claim of 'significantly' outperforming baselines is not backed by statistical evidence. In addition, the evaluation is only on the author-constructed Fashion-E benchmark, so there is no external validation on an established editing benchmark. I ask for per-task results, variance across runs or subsets, and at least a paired test for the main CLIP-T/CLIP-I differences, or a softened claim if such tests are not feasible.","section":"§III-B, Table I"},{"comment":"Several implementation details required for reproduction are missing: the Stable Diffusion version (SD1.4/SD1.5/SD2.x) and whether the same backbone is used for all baselines; the DDIM step counts S and T; how the averaged 16x16 attention map is aligned with a noise map of N pixels, particularly for masks that are not multiples of 16 or when the latent resolution differs from 16x16; and the exact LLAMA3 prompt used to extract the mask prompt. Without these, the Attention Processor cannot be reimplemented and the reported gains cannot be independently verified. Please include these details in a reproducibility appendix.","section":"§II-B and §III-A"}],"minor_comments":[{"comment":"The word 'mothod' in the first sentence of the Conclusion is a typo and should be 'method.'","section":"Section IV"},{"comment":"In the second paragraph, 'FPE [14]' should be 'FPE [17]' to match the reference list; the current citation assigns FPE to the InstructPix2Pix entry.","section":"Section I"},{"comment":"The phrase 'pixels with higher attention values' is underspecified when attention values are tied to 16x16 spatial locations: please clarify whether the selection is per-pixel in the latent noise map or per attention-grid cell, and how ties are broken.","section":"§II-B, Eq. (5)"},{"comment":"The sentence 'If all the pixels in Gpr are used once, the surplus pixels in Ged are replaced by randomly selected ones from Gpr' introduces randomness that is not controlled by a seed; please state whether a fixed seed is used and report variance from this random replacement.","section":"§II-B, Attention Processor"}],"recommendation":"major_revision","confidential_remarks":"The reader's stress-test concern about the Attention Processor is justified: the paper does not isolate attention-based selection from generic noise replacement, and the missing implementation details make the central mechanism hard to verify. That said, the idea is plausible and the Fashion-E benchmark is a useful contribution; with a proper control ablation and additional reproducibility details, the paper could become acceptable. I do not see grounds for rejection, since the issues are fixable within the manuscript's scope and the mask-prediction component is well supported by the provided ablations. No concerns about citation integrity or novelty disclosure beyond the minor reference error noted above."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: this is a serious, clearly written paper with two genuinely useful pieces—MaskNet, whose input combination (foreground map + DensePose + LLM-derived shape mask prompt) I don't see in the cited baselines, and Fashion-E, a new benchmark that the fashion-editing subfield needed. It deserves a serious referee. But the attention-enhanced editing-magnitude story is the softest part of the paper, and the current evidence doesn't separate attention selection from plain masked noise replacement.\n\nThe mask prediction is well designed. Training MaskNet on aligned cloth masks with shape-only prompts makes sense, and the ablations show it improves CLIP-I and LPIPS relative to local blending masks. The Fashion-E construction is described clearly, with separate color/detail/material/comprehensive tasks. That alone is a contribution.\n\nThe concern is the Attention Processor. The paper supports it with a 200-image probe of black t-shirts edited to red/yellow, showing successful edits have higher average 16x16 attention. Then it uses that to justify copying the top half of attention pixels from the target-prompt noise map into every masked pixel of the inversion noise. The ablation removes the whole processor, so it cannot rule out that any target-prompt noise swap inside the mask would give the same CLIP-T gain. And the paper's claim that the method 'increases attention map values' is never tested: attention is measured before substitution, not after. If random noise from the same sampling trajectory reproduces the gain, the 'attention-enhanced' contribution collapses to a mask-guided noise swap. That is a testable, fixable issue, not a fundamental contradiction, but it is load-bearing for the main selling point.\n\nAlso: no error bars or significance tests, evaluation only on self-built Fashion-E, and missing implementation details (SD version, DDIM step counts, how 16x16 attention aligns with N noise pixels). These are reproducibility problems a referee should flag, not grounds for desk rejection.\n\nThe citation pattern is fine and I don't see a circularity problem—MaskNet is supervised by real cloth masks, not by the editing metric.\n\nVerdict: send it to review. Ask for a control ablation that swaps random masked noise, post-edit attention measurement, and code/data release. The paper is a reasonable conditional accept bet.\n\nBest.","headline":"A serious, clearly written fashion-editing paper with a genuinely useful mask predictor and new benchmark, but the attention-enhanced editing-magnitude mechanism needs a cleaner ablation before the main claim is convincing.","tokens_in":8103,"tokens_out":1927,"would_cite":true,"duration_ms":19147,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MADiff uses a learned mask and an attention-based noise swap to make text-guided fashion edits both local and strong.","keywords":["text-guided image editing","fashion image editing","diffusion models","mask prediction","attention maps","DDIM inversion","CLIP score","training-free"],"falsifier":"Take a black t-shirt in front of a busy, high-contrast background and edit it to yellow. If the most-attended pixels fall on the background, the method will copy background detail into the shirt, and a text-alignment score on a batch of such images should drop relative to plain-background cases. That would show the attention premise does not hold for every fashion edit.","tokens_in":7143,"feed_emoji":"👗","tokens_out":9631,"duration_ms":89560,"temperature":0.7,"pith_summary":"This paper addresses two failures of general text-guided image editing applied to fashion: the edit lands on the wrong region, and the change is too weak to match the target text. MADiff attacks both with a two-stage design. First, MaskNet predicts where to edit, using the foreground, a DensePose map, and an LLM-extracted mask prompt. Second, a training-free Attention Processor strengthens the edit by taking the pixels with the highest attention values from a target-prompt noise map and inserting them into the mask region of the inversion noise map. The authors claim that on the new Fashion-E benchmark this yields stronger text alignment and better preservation of the original image than six existing editing methods.","feed_headline":"Mask plus attention swap sharpens fashion image editing","feed_subtitle":"Training-free processor targets the right pixels, beating six baselines on the Fashion-E benchmark.","key_machinery":"The Attention Processor is the load-bearing device: given a mask $m$ with $N$ pixels, it forms the set $G_{ed}$ of inversion-noise pixels inside the mask, identifies the $N/2$ pixels in the target-prompt noise map with the highest attention values, and substitutes $G_{ed}$ with those pixels, recycling the selected pixels at random if there are more masked pixels than selected ones. MaskNet is the second device: a lightweight UNet trained with MSE loss against cloth masks on Fashion-E, taking the foreground region, DensePose map, and an LLM-produced mask prompt as inputs to decide where editing should happen. The two devices split the problem cleanly: the mask says where, the attention map says what noise to put there.","core_discovery":"The central claim is that editing magnitude in diffusion models is controlled by the cross-attention map: in a probe of 200 black t-shirts edited to red and yellow, edits succeeded when attention values were high and failed when they were low. MADiff exploits this by replacing the pixels inside the predicted editing mask of the DDIM inversion noise map with the highest-attention pixels of a separate DDIM sampling noise map conditioned on the target prompt. The refined noise map is then blended with the inversion trajectory through the mask, so the edit is both localized and strong. On Fashion-E the authors report that this combination yields the best text-alignment score, the best content-preservation score, and the lowest perceptual difference among the six baselines considered.","pith_inferences":["A direct test the authors did not run is material and detail edits where the highest-attention pixels may fall on textures or the background; a batch of such edits with text-alignment scores would show whether the attention premise transfers beyond color swaps.","The fixed top-half rule could be replaced by a prompt-dependent or learned threshold; that would be a natural next step the paper does not address.","Because the LLM mask prompt keeps only shape words, texture and material edits may need a different prompt vocabulary; the paper does not evaluate this boundary.","The same swap mechanism could be ported to other masked editing domains, such as faces or vehicles, wherever a parser can supply the mask; that is an extension, not a claim of the paper."],"forward_implications":["Fashion editing becomes a two-step pipeline that can be applied to any diffusion model without retraining, since the Attention Processor only needs masks and attention maps.","Local shape edits such as changing sleeve length or collar shape become tractable, because MaskNet predicts the region the target text describes rather than the whole object.","The Fashion-E evaluation set provides four fixed tasks—color, detail, material, and combined editing—for fair comparison of future fashion editing models.","Mask and attention components can be improved independently: better masks improve preservation, better attention selection improves text alignment."],"supporting_citations":[{"why":"Supplies the DDIM sampling equations used to produce the target-prompt noise map and to denoise the refined map.","marker":"[5]"},{"why":"Supplies the DDIM inversion procedure that encodes the input image into the noise map and records the trajectory used for blending.","marker":"[10]"},{"why":"Provides the cross-attention editing and local-blending ideas that the Attention Processor extends and that serve as the ablation alternative for MaskNet.","marker":"[13]"},{"why":"InstructPix2Pix is a strong text-alignment baseline MADiff compares against, and its design motivates the attention-based editing family.","marker":"[14]"},{"why":"Fashion-Gen supplies the aligned image-text pairs from which the Fashion-E training set and cloth-mask ground truths are built.","marker":"[18]"},{"why":"Graphonomy computes the foreground region that MaskNet takes as input.","marker":"[19]"},{"why":"DensePose computes the body-part map that MaskNet takes as input.","marker":"[20]"},{"why":"LLAMA3-8b generates the mask prompt, keeping only shape-related words, which lets MaskNet locate edits such as sleeve length.","marker":"[21]"},{"why":"Defines the CLIP-based text-alignment and content-preservation metrics used to evaluate the edited images.","marker":"[25]"},{"why":"Defines the LPIPS perceptual similarity metric used to measure how close the edit stays to the original image.","marker":"[26]"}],"fun_headline_variants":["Attention maps drive diffusion edit strength in fashion","Mask plus attention swap refines fashion image editing","MADiff: MaskNet and attention processor for stronger edits","Editing magnitude linked to cross-attention in diffusion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method rests on the premise that the pixels the diffusion model pays most attention to are exactly the pixels that should be copied into the area being edited. The authors tested this only on 200 black t-shirts changed to red and yellow, then applied it to all fashion editing tasks.","fun_headline_variants_meta":{"raw":{"variants":["Attention maps drive diffusion edit strength in fashion","Mask plus attention swap refines fashion image editing","MADiff: MaskNet and attention processor for stronger edits","Editing magnitude linked to cross-attention in diffusion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000728,"raw_usage":{"total_tokens":3241,"prompt_tokens":908,"completion_tokens":2333,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":524,"completion_tokens_details":{"reasoning_tokens":2272}},"tokens_in":524,"tokens_out":2333,"duration_ms":20152,"temperature":1.0,"reasoning_tokens":2272,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:35:56.150324+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a black t-shirt in front of a busy, high-contrast background and edit it to yellow. If the most-attended pixels fall on the background, the method will copy background detail into the shirt, and a text-alignment score on a batch of such images should drop relative to plain-background cases. That would show the attention premise does not hold for every fashion edit.","supporting_citations":[{"cited_title":"Graphonomy: Universal human parsing via graph transfer learning,","cited_arxiv_id":null,"evidence_quote":"Graphonomy computes the foreground region that MaskNet takes as input."},{"cited_title":"Densepose: Dense human pose estimation in the wild,","cited_arxiv_id":null,"evidence_quote":"DensePose computes the body-part map that MaskNet takes as input."},{"cited_title":"Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation,","cited_arxiv_id":null,"evidence_quote":"Defines the CLIP-based text-alignment and content-preservation metrics used to evaluate the edited images."},{"cited_title":"The unreasonable effectiveness of deep features as a perceptual metric,","cited_arxiv_id":null,"evidence_quote":"Defines the LPIPS perceptual similarity metric used to measure how close the edit stays to the original image."},{"cited_title":"Denoising diffusion implicit models,","cited_arxiv_id":null,"evidence_quote":"Supplies the DDIM sampling equations used to produce the target-prompt noise map and to denoise the refined map."},{"cited_title":"Diffusion autoencoders: Toward a meaningful and decodable representation,","cited_arxiv_id":null,"evidence_quote":"Supplies the DDIM inversion procedure that encodes the input image into the noise map and records the trajectory used for blending."},{"cited_title":"Prompt-to-prompt image editing with cross- attention control,","cited_arxiv_id":null,"evidence_quote":"Provides the cross-attention editing and local-blending ideas that the Attention Processor extends and that serve as the ablation alternative for MaskNet."}],"review_version":1}