{"id":"fd41ce33-5100-43f0-ab49-f9dd87418ca9","arxiv_id":"2505.00751","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A new attention-manipulation scheme (SPAA) is used to generate 1.1M synthetic color/material edit pairs, and an InstructPix2Pix-style model trained on them outperforms prior instruction editors in the authors' benchmarks.","lead":"This paper builds a model that edits the color or material of a single object in an image from a text instruction, while keeping the object's shape and the background intact. The authors generate a large synthetic training set by manipulating attention maps inside Stable Diffusion, then fine-tune an instruction-following editing model on it.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Table 2 evaluation appears circular: the material/color target images used as pseudo ground truth are produced by the same SPAA pipeline that generated InstructAttribute's training set, so reported margins may measure imitation of SPAA rather than real editing quality.","rationale":"The reader's weakest-assumption diagnosis is correct and is the single most load-bearing issue. The paper's headline claim is comparative: InstructAttribute achieves a superior balance between attribute modification accuracy and structural preservation. That claim rests almost entirely on Table 2's quantitative metrics. If the evaluation targets are SPAA-generated and filtered by the same LLaVa/DINO criteria used to build the training data, then the metrics cannot distinguish genuine editing ability from imitation of the SPAA output distribution. InstructAttribute is trained on SPAA-filtered pairs, so high DINO/SSIM and low background LPIPS may simply reflect that the model has learned the filtering biases and failure modes of the generator. The SPAA hyperparameters were also tuned on the same selection criterion, compounding the issue. The human preference study is genuinely independent and does provide some support, but 100 examples with no reported agreement statistics is too thin to carry the quantitative comparison alone. My recommendation is unchanged relative to the reader's verdict: the paper should be CONDITIONAL, requiring release of artifacts and an independent or human-annotated evaluation before the superiority claim is accepted. I did not find a more fundamental internal inconsistency in the SPAA mechanism itself; the qualitative figures and ablations are plausible, but they do not address the comparative claim.","tokens_in":16010,"tokens_out":5266,"duration_ms":55465,"concrete_test":"Request the released code/data (or, absent release, an explicit statement of how the Section 6 evaluation targets were generated). Then recompute Table 2 on an independently constructed benchmark: take 200 real photographs with known object masks, create human-verified ground-truth color/material edits using a separate generator or manual annotation, and rerun IP2P, InsDiff, HIVE, MGIE, MagicBrush, HQ-Edit, UltraEdit, and InstructAttribute with identical instructions. If the DINO/SSIM/CS/LPIPS/L1Hue margins over the strongest baselines shrink substantially or reverse, the reported superiority is an artifact of shared SPAA data; if the margins hold, the central claim survives this check.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 6 states that, because no public benchmarks exist, the authors 'developed two specialized datasets' for quantitative evaluation: 239 material source images with targets for all 14 materials, and 659 color source images with targets for all 43 colors. The section does not specify how those target images were produced. The only attribute-variation engine described in the paper is the Section 4 SPAA data pipeline, which includes LLaVa verification, DINO grayscale filtering, and attribute-leakage rejection; Section 5 then trains InstructAttribute on exactly this kind of SPAA-filtered data. With 3,346 plus 28,337 target images, manual construction is implausible, so the natural reading is that the Table 2 targets are SPAA outputs filtered by the same selection criteria. Under that reading, DINO, SSIM, CLIP, LPIPS-bg, and L1-hue on the object measure how closely each editing model imitates SPAA's particular outputs, not whether the edit is correct in an absolute or real-image sense. Moreover, SPAA's amplification ratios (5 for color, 10 for material, with 0.1/0.2 decay per step) were explicitly chosen to maximize success under the same LLaVa/DINO criterion (Table 1 and Section 5), so a model trained on the filtered SPAA distribution has an advantage that is partially baked into the metric. The human preference study (100 random examples, 20 annotators) is the only independent signal and it does support the qualitative direction, but it is small and reports no inter-annotator agreement. The central claim that InstructAttribute outperforms existing instruction-based baselines is therefore not established by the quantitative evidence as reported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SPAA, a training-free method for generating object-level attribute variants by replacing high-resolution self-attention maps with source maps and amplifying the cross-attention value vectors of attribute descriptors under a decaying ratio. Using SPAA, the authors build an automatically curated Attribute Dataset of color and material edits, generate instruction triples with GPT-4o, and fine-tune InstructAttribute from Stable Diffusion via the InstructPix2Pix training objective. The central claim is that InstructAttribute outperforms existing instruction-based image editing methods on fine-grained color and material modification while preserving object structure and background. This claim is supported by qualitative comparisons, quantitative metrics on two newly constructed evaluation sets, and a human preference study.","tokens_in":16330,"tokens_out":6401,"duration_ms":64868,"significance":"If the results hold, the paper would make three contributions: a mechanistic analysis of self-attention and cross-attention roles in attribute editing, a large-scale synthetic dataset for object color and material editing, and a fine-tuned instruction-following model with a favorable structure-attribute trade-off. The ablation in Table 1 and the attention-map visualizations are useful evidence for the SPAA design choices. The scale of the dataset and the inclusion of a human preference study are also positive features. However, the quantitative evaluation appears to be circular because the pseudo-ground-truth targets are likely generated by the same SPAA pipeline that produced the training data, so the reported margins may partly measure agreement with SPAA's output distribution rather than editing quality. The independent human study is too small and too thinly reported to carry the full burden of the superiority claim by itself. These issues are fixable but require additional evidence or a substantial revision of the evaluation.","major_comments":[{"comment":"The quantitative evaluation is circular as described. The paper does not state how the 3,346 material and 28,337 color target images in the evaluation sets were generated, and the only attribute-variation engine described in the manuscript is the SPAA pipeline of Section 4, which is also the source of the training triples in the Attribute Dataset. If the evaluation targets are SPAA outputs, then the DS, SSIM, CS, LPIPS-bg, and L1-hue numbers measure how closely each model reproduces SPAA's filtered output distribution, not how well it performs attribute editing in an independent sense. This is especially problematic because the SPAA hyperparameters (R=5 with lambda=0.1 for color, R=10 with lambda=0.2 for material) were explicitly chosen to maximize success under the same LLaVa/DINO filtering criteria used for dataset construction (Section 5, Table 1). The authors must state explicitly how the evaluation targets were constructed; if SPAA was used, they should add an independent evaluation set with manually verified edits or real-image edits, or justify why similarity to SPAA outputs is the correct notion of editing quality.","section":"Section 6, Table 2"},{"comment":"The amplification schedule is internally inconsistent as written. The text says that R=5 for color and R=10 for material is an initial ratio that decays per denoising step until it reaches 1.0, but the formula V_T*_cross = (1-lambda)*t*R*V_T_cross_i with t decreasing from T to 1 gives an effective initial multiplier of (1-lambda)*T*R (for example, 4500 for color when T=1000) and a final multiplier of (1-lambda)*R. The ratio described as 'initial' in Section 5 is not the ratio at the first denoising step under this formula. Please define t precisely as a normalized timestep, an absolute diffusion timestep, or a step index, and state the exact schedule; this is load-bearing for reproducing the data generation and for interpreting the ablation in Figure 7.","section":"Section 4, Eq. (after Algorithm 1) and Section 5"},{"comment":"The human preference study is the only independent evidence for the main claim, but it is currently underreported. It uses 100 randomly sampled source-target examples and 20 annotators, and Table 3 reports only raw preference percentages without inter-annotator agreement, confidence intervals, or significance tests. Because the quantitative benchmarks are subject to the circularity noted above, this study carries too much weight to be summarized in a single paragraph. Please report the full protocol, the per-example agreement statistics, and a statistical comparison against the baselines.","section":"Section 6, Human Preference Study and Table 3"}],"minor_comments":[{"comment":"There are several typos and inconsistencies in notation, including 'discriptors' for 'descriptors', 'quantifyes' for 'quantifies', 'to to' in the Prompt-to-Prompt discussion, and the heading '5. InstructionAttribute' which does not match the model name 'InstructAttribute' used elsewhere.","section":"Throughout"},{"comment":"The table column 'L1Hue obj' is described in the text as 'L1 loss of HVS of the object area'; please define the acronym and explain exactly how the object-area mask is obtained for source and target images, since the choice of mask has a direct effect on this metric.","section":"Section 6, Table 2"},{"comment":"The baselines cite HIVE as [66] in the related work but as [20] in Section 6 and in Figures 13-15, while reference [20] appears to describe a different model (ACE). Please harmonize the method names and reference entries.","section":"Section 2 and Section 6"},{"comment":"The notation DM(z_S,t,P_S,t) is ambiguous; the denoising function should be given a clear signature indicating which arguments are the latent, the timestep, and the prompt.","section":"Algorithm 1, line 6"},{"comment":"The text and captions alternate between 'InstructDiffusion (InsDiff)' and 'InsDiff'; please use one consistent abbreviation.","section":"Section 6, figures 13-15"},{"comment":"The claim that replacing self-attention maps at resolutions higher than 32x32 preserves structure is supported by qualitative examples but not quantified; adding a small quantitative ablation would strengthen the analysis.","section":"Section 4, Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal, but the main quantitative claim currently rests on a benchmark whose targets appear to be generated by the same engine used to create the training set. I would ask the editor to require an explicit description of the evaluation target construction and an independent validation set. I did not find evidence of misconduct; the abstract and conclusion are, however, stronger than the current evidence supports."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper has a real method and a big new dataset, but I think the reader and the stress-test are right: the central outperformance claim is not established by the numbers as reported. The test targets in Table 2 appear to be SPAA outputs from the same data engine used to build the training set. Section 6 says they \"developed two specialized datasets\" without saying how the target images were made, and the only mechanism described anywhere in the paper for producing color/material variants at that scale is SPAA plus LLaVa/DINO filtering. So the DINO, SSIM, CLIP, LPIPS, and hue-error numbers mostly measure how closely each model imitates SPAA's particular outputs, not how well the edit is actually done. The hyperparameters (R=5, λ=0.1 for color; R=10, λ=0.2 for material) were also tuned to maximize success under the same LLaVa/DINO criteria, so the metric is partly baked in. That is a load-bearing flaw for the quantitative section, and the paper would need either independent annotation or real-image benchmarks to fix it.\n\nWhat is genuinely good: SPAA itself is a simple, plausible combination of known attention control ideas—self-attention injection at high resolution for structure, cross-attention value amplification with a decaying ratio for attribute change—and the ablations (Table 1, Figures 4-7) give some evidence for the design choices. The Attribute Dataset is large in scope (3,300+ subjects, 43 colors, 14 materials, >1M pairs) and, as far as I know, new. The qualitative figures look like the method does work, and the human preference study, though small (100 examples, 20 annotators, no agreement metric), independently supports the direction. I also credit the authors for stating limitations about cross-attention leakage and filtering bias.\n\nMinor issues: Algorithm 1 defines the amplification factor as (1−λ)·t·R, which is ambiguous unless t is normalized; the text says \"gradually reducing\" the ratio, which matches a normalized t or a max(1, R−λk) schedule, but the pseudocode as written is unclear. Also, no code, data, or weights are released; that makes the circularity harder to inspect and the contribution harder to reuse.\n\nBottom line: the method and dataset deserve serious referee time, but the quantitative comparison needs to be redone on independent targets before the paper can be believed. If the authors release artifacts and re-evaluate on non-SPAA data, this could be a solid contribution. As it stands, the qualitative and human-preference evidence hints at real capability, but the headline claims over-reach.","headline":"A plausible training-free attention trick and a large new dataset, but the headline quantitative claim is undercut by an evaluation whose targets appear to come from the same SPAA generator that produced the training set.","tokens_in":16939,"tokens_out":2612,"would_cite":false,"duration_ms":27612,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Object color and material can be edited from a plain sentence by a model trained on pairs generated with a training-free attention method, beating existing instruction-based editors on accuracy and preservation.","keywords":["diffusion models","instruction-based image editing","object attribute editing","color editing","material editing","self-attention manipulation","cross-attention value amplification","synthetic dataset generation"],"falsifier":"Run InstructAttribute and the baselines on a held-out set of real photographs with human-verified masks and target color and material labels; if InstructAttribute no longer leads on attribute accuracy and preservation on that independent benchmark, the paper's central claim is refuted.","tokens_in":15766,"feed_emoji":"🎨","tokens_out":19467,"duration_ms":167090,"temperature":0.7,"pith_summary":"The paper tries to establish that object-level color and material editing can be made precise and instruction-following. It introduces SPAA, a training-free procedure that edits an object's attribute in a text-to-image diffusion model by holding high-resolution self-attention maps fixed to the source image (to preserve structure) and amplifying the cross-attention value vectors of the attribute-describing words with a decaying ratio (to force the color or material change). SPAA is used to generate more than 1.1 million source-target attribute pairs across 43 colors, 14 materials, and 3,300 subjects, which are filtered automatically and paired with generated instruction templates; fine-tuning a diffusion model on this data yields InstructAttribute. If the claim is right, users can edit objects in natural language, for example turning a jacket's material to leather and its color to teal, without damaging shape or background. The paper's evidence is that InstructAttribute scores higher than existing instruction-based editors on structure and attribute metrics and wins a 100-example human preference study.","feed_headline":"InstructAttribute edits color and material from a text command","feed_subtitle":"It beats seven instruction-based editors on color and material accuracy while keeping shape and background intact.","key_machinery":"The load-bearing mechanism is SPAA (Structure Preservation and Attribute Amplification), a training-free intervention with three moving parts. First, the edited image's high-resolution self-attention maps (32x32 and 64x64) are overwritten by the source image's self-attention maps throughout denoising, based on the observation that these maps contain fine structural detail. Second, the cross-attention value vectors $V^{\\text{cross}}$ of the attribute descriptor tokens are scaled as $V^{*} = (1 - \\lambda) \\cdot t \\cdot R \\cdot V$, where $R$ is an initial amplification ratio, $\\lambda$ a decay factor, and $t$ the current timestep; the ablation studies show that early-timestep amplification is necessary and that a decaying ratio avoids over-saturation. Third, an automated filtering and instruction-generation pipeline wraps this generator into a dataset engine, using a vision-language model to confirm the attribute, a semantic-similarity score to reject structurally broken outputs, object detection and segmentation to reject background leakage, and a large language model to write instruction templates. Fine-tuning a latent diffusion model on the resulting triples is what turns the generator into the instruction-following editor InstructAttribute.","core_discovery":"The central claim is that fine-grained object attribute editing can be decomposed into two separable attention interventions inside an existing text-to-image diffusion model. During denoising, the target image's self-attention maps at high resolutions (32x32 and above) are replaced by the source image's self-attention maps, which preserves the object's shape, texture, and background; simultaneously, the cross-attention value vectors assigned to the attribute descriptor tokens are multiplied by a timestep-aware decaying ratio (starting at 5 for color and 10 for material, decreasing by 0.1 or 0.2 per denoising step down to 1.0), which drives the attribute toward the requested color or material without distorting geometry. The paper argues that scaling the value matrix works where scaling the key matrix fails, because values carry attribute-specific feature content while keys govern the attention map layout. This training-free recipe, SPAA, is then used as a data engine: a vision-language model verifies that the target attribute is present, a grayscale semantic-similarity threshold discards structurally broken outputs, and a background-difference check discards edits that leak into the background. The surviving pairs, paired with instruction templates produced from in-context examples, form the Attribute Dataset, and instruction-tuning a latent diffusion model on it produces InstructAttribute. According to the paper's quantitative and human-preference evaluations, InstructAttribute achieves the best balance between attribute modification accuracy and structure/background preservation among all compared instruction-based editors.","pith_inferences":["If the reported numbers hold up, the main lesson is that the data engine matters more than the training objective: an instruction-based editor's success on fine-grained attributes is set by the quality and coverage of the synthetic pairs it is trained on.","A direct way to test that lesson is to train the same recipe on an independently produced set of object-attribute pairs, such as manually edited real photographs; if the performance gap over baselines shrinks, the reported advantage largely reflects the SPAA-generated training data rather than the method itself.","Because SPAA's value-amplification step acts on the attribute descriptor's cross-attention values, the same mechanism should transfer to other attribute types carried by values, such as texture, lighting, or facial expression, and the paper's style-transfer result is a first hint of that generality."],"forward_implications":["Editors trained on the Attribute Dataset can change an object's color to any of 43 listed colors or its material to any of 14 listed materials from a natural-language instruction while keeping structure and background intact.","SPAA can generate attribute-variation pairs at scale, so constructing fine-grained attribute training sets no longer requires manual editing or per-attribute fine-tuning.","The model supports within-hue color adjustments that change brightness and saturation, extending control beyond switching between distinct color names.","The attention-manipulation recipe also transfers to object painting-style transfer, although the paper excludes holistic style from the main study.","The automated filtering chain (attribute verification, similarity threshold, background-leak check) provides a reusable recipe for curating other fine-grained visual attribute datasets."],"supporting_citations":[{"why":"Supplies the instruction-tuning training objective and serves as a primary baseline the paper compares against.","marker":"[3]"},{"why":"Showed that replacing self-attention maps with source maps preserves structure, which SPAA extends to high resolutions across the full denoising process.","marker":"[5]"},{"why":"Provides the grayscale semantic-similarity score used both to filter distorted targets and to measure structural preservation.","marker":"[6]"},{"why":"The training-free editing method SPAA starts from; its failures to preserve structure or change attributes motivate the new attention manipulations.","marker":"[21]"},{"why":"Generates the instruction templates from a small set of in-context examples, creating the diverse text commands used for training.","marker":"[28]"},{"why":"Analyzed the roles of self- and cross-attention in stable diffusion, supporting the structural-preservation component of SPAA.","marker":"[38]"},{"why":"Queries whether the target image shows the requested attribute and filters failed SPAA generations.","marker":"[39]"},{"why":"The pretrained text-to-image model that SPAA intervenes in and from which InstructAttribute is initialized.","marker":"[53]"},{"why":"A manually annotated instruction-guided editing dataset and baseline, contrasting human labels with the paper's synthetic pipeline.","marker":"[64]"},{"why":"A recent fine-grained instruction-editing baseline that the paper compares against.","marker":"[67]"}],"fun_headline_variants":["Split attention: edit color/material while keeping shape","SPAA: training-free attribute editing that preserves structure","Self-attention swap and value scaling give precise attribute edits","Color and material editing without shape distortion: SPAA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim collapses if the synthetic target images used for evaluation are not valid references for correct edits, because those targets come from the same SPAA pipeline that produced the training pairs, so the metric gains could be measuring imitation instead of editing ability.","fun_headline_variants_meta":{"raw":{"variants":["Split attention: edit color/material while keeping shape","SPAA: training-free attribute editing that preserves structure","Self-attention swap and value scaling give precise attribute edits","Color and material editing without shape distortion: SPAA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000317,"raw_usage":{"total_tokens":1850,"prompt_tokens":1057,"completion_tokens":793,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":673,"completion_tokens_details":{"reasoning_tokens":730}},"tokens_in":673,"tokens_out":793,"duration_ms":8880,"temperature":1.0,"reasoning_tokens":730,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:46:48.953227+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run InstructAttribute and the baselines on a held-out set of real photographs with human-verified masks and target color and material labels; if InstructAttribute no longer leads on attribute accuracy and preservation on that independent benchmark, the paper's central claim is refuted.","supporting_citations":[{"cited_title":"In- structpix2pix: Learning to follow image editing instructions","cited_arxiv_id":null,"evidence_quote":"Supplies the instruction-tuning training objective and serves as a primary baseline the paper compares against."},{"cited_title":"Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing","cited_arxiv_id":null,"evidence_quote":"Showed that replacing self-attention maps with source maps preserves structure, which SPAA extends to high resolutions across the full denoising process."},{"cited_title":"Emerg- ing properties in self-supervised vision transformers","cited_arxiv_id":null,"evidence_quote":"Provides the grayscale semantic-similarity score used both to filter distorted targets and to measure structural preservation."},{"cited_title":"Towards understanding cross and self-attention in stable diffusion for text-guided image editing","cited_arxiv_id":null,"evidence_quote":"Analyzed the roles of self- and cross-attention in stable diffusion, supporting the structural-preservation component of SPAA."},{"cited_title":"Visual instruction tuning, 2023","cited_arxiv_id":null,"evidence_quote":"Queries whether the target image shows the requested attribute and filters failed SPAA generations."},{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"The pretrained text-to-image model that SPAA intervenes in and from which InstructAttribute is initialized."},{"cited_title":"Magicbrush: A manually annotated dataset for instruction- guided image editing","cited_arxiv_id":null,"evidence_quote":"A manually annotated instruction-guided editing dataset and baseline, contrasting human labels with the paper's synthetic pipeline."},{"cited_title":"Ultraedit: Instruction-based fine-grained im- age editing at scale, 2024","cited_arxiv_id":null,"evidence_quote":"A recent fine-grained instruction-editing baseline that the paper compares against."}],"review_version":1}