{"id":"d9f492b1-c806-4a53-8242-31e655fc030c","arxiv_id":"2505.19149","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"MIND-Edit combines instruction rewriting with MLLM-derived visual embeddings to guide diffusion-based image editing, but the reported numbers only partly support the claim of state-of-the-art performance.","lead":"MIND-Edit is an image editing system that uses a multimodal large language model to rewrite vague instructions and to generate a visual 'intent' embedding that guides a diffusion model. The paper claims state-of-the-art results, but its own tables show it does not consistently beat the MagicQuill baseline on several metrics.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1 contradicts the central claim: MagicQuill beats MIND-Edit on 5 of 8 metrics (CLIP-I, LPIPS, SSIM on HumanEdit; CLIP-I on Complex), yet the abstract claims state-of-the-art superiority.","rationale":"The reader's verdict is REJECT with moderate confidence, and my independent reading converges on the same conclusion but via a sharper route: the paper's own Table 1 falsifies the abstract's claim. Comparing only MIND-Edit and MagicQuill, MagicQuill wins 3 of 4 metrics on HumanEdit (CLIP-I, LPIPS, SSIM) and CLIP-I on ComplexMultistepImageEditing. MIND-Edit's wins are small and inconsistent, while the CLIP-I deficit is substantial, especially on the complex dataset (0.6340 vs 0.6996). Since Eq. (5) optimizes cosine similarity to the CLIP image embedding, CLIP-I is the metric most aligned with the method's training objective; losing on CLIP-I to MagicQuill undermines the central claim that MLLM-driven visual representations provide superior semantic alignment. The reader's weakest assumption (CLIP embedding sufficiency and IP-Adapter transfer fidelity) is a plausible mechanism-level risk, but the decisive problem is the evidence mismatch: the central claim of SOTA superiority is contradicted by the paper's own quantitative results. The text's own hedge in §4.2 ('performs comparably to MagicQuill') reinforces this inconsistency. A paired re-evaluation with error bars and significance testing would settle whether the SOTA claim can be rescued; without that, the central claim is unsupported. No verdict adjustment is needed because the reader already reached REJECT.","tokens_in":13252,"tokens_out":4893,"duration_ms":43270,"concrete_test":"Re-run the HumanEdit 500-image evaluation and the ComplexMultistepImageEditing dataset with both MIND-Edit and MagicQuill under identical inference settings, and compute per-image CLIP-I, LPIPS, PSNR, and SSIM with paired bootstrap 95% confidence intervals. If MagicQuill's CLIP-I advantage on HumanEdit (0.9381 vs 0.9310) is statistically significant, or if MIND-Edit does not achieve statistically significant wins on a majority of the four metrics on both datasets, then the 'outperforms state-of-the-art' claim is unsupported. A minimal first check is to count the wins in Table 1: no aggregation rule is provided that would make 3-of-8 metric wins equivalent to overall superiority.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Abstract, §4.2, Conclusion) is that MIND-Edit outperforms state-of-the-art image editing methods, especially in complex scenarios. The paper's own evidence, Table 1, contradicts this. On HumanEdit, MagicQuill leads CLIP-I (0.9381 vs 0.9310), LPIPS (0.1162 vs 0.1245), and SSIM (0.8981 vs 0.8517); MIND-Edit wins only PSNR by 0.033 dB (22.2714 vs 22.2380). On ComplexMultistepImageEditing, MagicQuill wins CLIP-I (0.6996 vs 0.6340); MIND-Edit wins LPIPS, PSNR, and SSIM, but the gaps are small (0.0074 LPIPS, 0.1087 dB PSNR, 0.0252 SSIM). Counting only the strongest baseline, MIND-Edit wins 3 of 8 metrics. The text itself in §4.2 concedes MIND-Edit 'performs comparably to MagicQuill' in simple scenarios, which is incompatible with the abstract's 'outperforms state-of-the-art.' Furthermore, the training objective in Eq. (5) directly maximizes cosine similarity to the CLIP image embedding of the target, so CLIP-I is the metric most aligned with the method's optimization; losing on CLIP-I to MagicQuill undercuts the claimed benefit of the visual-representation mechanism. The central claim therefore fails on the paper's own numbers; qualitative figures cannot establish SOTA superiority when the main quantitative table shows the opposite for the strongest baseline.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MIND-Edit, an image-editing framework that couples a multimodal large language model (LLaVA-OneVision-7B) with a frozen Stable Diffusion backbone. The MLLM both rewrites ambiguous user instructions into clearer text and produces a visual embedding v_insight from an intermediate hidden layer; this embedding is injected into the diffusion model through a pretrained IP-Adapter. The two outputs are trained jointly with a cross-entropy text loss and a cosine-similarity loss between v_insight and the CLIP image embedding of the target edited image. Quantitative results are reported on HumanEdit and ComplexMultistepImageEditing against SmartEdit, BrushNet, and MagicQuill, together with ablations that separate instruction optimization, visual guidance, and joint versus independent training.","tokens_in":13626,"tokens_out":5628,"duration_ms":49257,"significance":"The core idea is plausible and potentially useful: instead of relying on the MLLM's text output alone, the paper attempts to exploit the MLLM's internal visual representations to guide a diffusion model, and the plug-and-play use of IP-Adapter is practically attractive. The ablation design is thoughtful in isolating the contributions of the two strategies and of joint training, and the paper includes an explicit Limitations section as well as concrete training details. However, the significance is severely undercut because the paper's headline empirical claim is contradicted by its own quantitative table, and because the training objective in Eq. (5) directly targets the same CLIP embedding space used by the CLIP-I evaluation metric. As presented, the evidence does not support the claimed state-of-the-art performance.","major_comments":[{"comment":"The central claim in the Abstract, §4.2, and Conclusion that MIND-Edit outperforms state-of-the-art methods is contradicted by Table 1. Against MagicQuill, the strongest baseline, MIND-Edit loses CLIP-I on both datasets (0.9310 vs 0.9381 on HumanEdit; 0.6340 vs 0.6996 on ComplexMultistepImageEditing), loses LPIPS and SSIM on HumanEdit (0.1245 vs 0.1162 and 0.8517 vs 0.8981, respectively), and wins only PSNR on HumanEdit by 0.033 dB. On ComplexMultistepImageEditing, MIND-Edit wins LPIPS, PSNR, and SSIM, but by small margins (0.0074, 0.1087 dB, and 0.0252, respectively). The text in §4.2 itself concedes that MIND-Edit \"performs comparably to MagicQuill\" in simple scenarios, which is incompatible with the abstract's \"outperforms\" claim. The paper's own numbers therefore do not support the main conclusion.","section":"§4.2, Table 1"},{"comment":"The training objective ℓ_embed in Eq. (5) maximizes cosine similarity between v_insight and E_CLIP(I_gt), the CLIP image embedding of the target edited image. The evaluation metric CLIP-I in Table 1 is, in standard practice, computed with the same CLIP image encoder, making CLIP-I the metric most directly aligned with the training objective. This creates a circularity concern: CLIP-I cannot serve as an independent measure of semantic alignment for this method. The fact that MIND-Edit still loses CLIP-I to MagicQuill on both datasets is therefore especially damaging to the claim that the visual-representation mechanism provides an advantage. The authors should specify the exact CLIP-I computation (model variant, prompt template, and whether the source or target image is used) and add at least one evaluation that does not rely on the same embedding space, such as a DINO-based metric, a user study, or a different CLIP checkpoint.","section":"§3.5, Eq. (5); §4.1 evaluation metrics"},{"comment":"The ablation study does not fully support the statement in §4.3 that both instruction optimization and visual representation guidance contribute to improved editing performance. In Table 2, row #1 (instruction optimization only, without v_insight) has higher PSNR than the full method in row #6 (22.4354 vs 22.2714), and row #5 (independent text head, jointly trained visual head) has higher PSNR and SSIM than row #6 (22.3914 and 0.8524 vs 22.2714 and 0.8517). Thus the claim of \"consistent improvements across all quantitative metrics\" from joint training is not supported by the reported values. The authors should correct the interpretation, report error bars, or provide significance tests before drawing these conclusions.","section":"Table 2, rows #1 and #6"},{"comment":"Most of the reported differences are very small (e.g., 0.003 in CLIP-I, 0.03 dB in PSNR) and are based on 500 and 120 test images, respectively, with no confidence intervals or significance tests. As reported, the quantitative comparisons cannot reliably distinguish methods from noise, especially for the claim of superiority. The paper should report confidence intervals or paired significance tests, and should clarify whether the baseline numbers are taken from the original papers or re-run under identical experimental protocols.","section":"§4.1, datasets and metrics"}],"minor_comments":[{"comment":"The choice of hidden layer m=21 and the balancing weight λ=2 are not justified and are not varied in any sensitivity analysis; a short study of these choices would make the method more robust.","section":"§3.4 and §4.1"},{"comment":"The model name is written inconsistently as \"LLaV A-OneVision\" and \"LLaV A-OneVision-7B\"; the spacing should be corrected to \"LLaVA-OneVision\".","section":"Throughout"},{"comment":"The caption says \"Our method demonstrates competitive performance,\" which is weaker than and inconsistent with the abstract's \"outperforms state-of-the-art\" claim; the claims should be aligned with what the numbers actually show.","section":"Table 1 caption and §4.2"},{"comment":"The notation z0=E(y) is used but the diffusion loss refers to noisy latents zt; please clarify the conditioning convention and whether c_T is the raw instruction or the optimized instruction during training.","section":"Eq. (1)"},{"comment":"The qualitative comparisons would be easier to assess if they included side-by-side crops and a clear statement of which input mask, if any, is used for each baseline, since mask versus mask-free evaluation affects fairness.","section":"Figures 3, 4, and A"}],"recommendation":"reject","confidential_remarks":"The central claim of state-of-the-art performance is contradicted by the paper's own Table 1, and the main semantic metric, CLIP-I, is directly aligned with the training objective in Eq. (5). These are load-bearing issues that cannot be resolved by minor edits. If the authors substantially rework the method and evaluation, and add independent metrics and significance testing, the underlying idea may merit a fresh submission. I would not recommend acceptance in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Name],\n\nRead MIND-Edit. My take: the method is a defensible extension of the MetaQueries/MetaMorph paradigm to editing, but the paper's own numbers don't support its headline claim. On HumanEdit, MagicQuill beats it on CLIP-I, LPIPS, and SSIM; MIND-Edit only edges out PSNR by 0.03 dB. On the complex dataset the scoreboard is split, and the text in §4.2 admits to performing 'comparably' to MagicQuill in simple scenarios — which contradicts the abstract's 'outperforms state-of-the-art.' The stress-test note is right about that.\n\nWhat actually works: the design — a vision head on intermediate MLLM hidden states regressing a CLIP image embedding, injected via IP-Adapter — is a clean transfer of the generation trick to editing. The instruction optimization component is sensible, and the ablation is honestly run: every component helps, and joint training beats independent training. That part is credible.\n\nThe soft spots are in the evaluation. Eq. (5) trains the visual representation to maximize cosine similarity to the CLIP image embedding of the target; the main reported metric, CLIP-I, is almost certainly computed with the same CLIP image encoder. The paper never states which CLIP model it uses, reports no error bars, and releases no code. That makes the quantitative result look circular and hard to verify. If the visual token is the key contribution, losing CLIP-I to MagicQuill damages the case.\n\nAlso, the baselines are thin: only SmartEdit, BrushNet, and MagicQuill. One of the paper's own references (Step1X-Edit) is a stronger recent method and isn't compared. The limitations section is candid about SD and IP-Adapter constraints — that's honest, but it doesn't fix the gap between claims and evidence.\n\nThis is a paper for someone working on MLLM-guided editing, mainly as a design reference and a cautionary tale about evaluation. It deserves a serious referee, not a desk reject, because the idea is real and the problems are fixable with more careful experiments and a toned-down abstract. But it should not be accepted as presented.\n\nBest,\n\n[You]","headline":"Plausible method, but Table 1 undercuts the SOTA claim and CLIP-I is circular; needs serious revision, not acceptance as is.","tokens_in":14188,"tokens_out":3483,"would_cite":false,"duration_ms":30315,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MIND-Edit claims image editing improves when a multimodal language model predicts a visual embedding of the desired result and injects it into a diffusion model through an image adapter.","keywords":["instruction-based image editing","multimodal large language model","visual representation generation","diffusion model","IP-Adapter","joint training","CLIP embedding","complex multi-step editing"],"falsifier":"Build a test set of edits whose target images have near-identical CLIP embeddings to their sources, such as changing a small object's color or removing one element, and measure whether MIND-Edit still reproduces the edit. If the predicted visual embedding is nearly the source embedding and the edit fails, the assumption that CLIP embeddings encode the needed editing intent is the weak link.","tokens_in":13010,"feed_emoji":"🖼️","tokens_out":4617,"duration_ms":40830,"temperature":0.7,"pith_summary":"Image editing systems that accept natural-language instructions often fail when the instruction is vague, because they treat the user's words as the complete specification of intent. MIND-Edit argues that a multimodal large language model's visual understanding should be used directly, not only its text output. The paper proposes two complementary mechanisms: the MLLM rewrites ambiguous instructions into clearer directives, and, more centrally, it predicts a visual embedding of the desired edited image that is fed into the diffusion model through an existing image adapter. Both outputs are trained jointly so that text and visual reasoning reinforce each other. The paper claims this beats current instruction-based editing methods on standard and complex multi-step benchmarks, with particular gains in semantic alignment.","feed_headline":"A model's visual insight, not just text, steers image edits","feed_subtitle":"MIND-Edit feeds a predicted image embedding into a diffusion model, improving complex instruction-based editing.","key_machinery":"The load-bearing mechanism is the language-vision projection: a lightweight vision head reading the MLLM's $m$-th hidden layer and outputting an embedding whose dimension matches the image adapter's expected clip_embedding input. The adapter then runs a second cross-attention branch parallel to the text-conditioned one, so the visual embedding $v_{\\mathrm{insight}}$ steers the UNet without retraining the diffusion model. The joint loss $\\ell_{\\text{total}} = \\ell_{\\text{text}} + \\lambda \\ell_{\\text{embed}}$ ties the text and visual heads together.","core_discovery":"The central claim is that editing intent can be carried by a learned visual representation $v_{\\mathrm{insight}}$, computed as $v_{\\mathrm{insight}} = F_v(h_m)$ from the MLLM's intermediate hidden states, and that injecting this representation via an image adapter's decoupled cross-attention gives the diffusion model guidance that pure text prompting misses. The vision head $F_v$ is trained against the CLIP image embedding of the ground-truth edited image with a cosine-similarity loss, while the text head is trained with cross-entropy to produce an optimized instruction. A weighted sum of the two losses is trained jointly. The paper reports that ablations removing either strategy degrade performance and that independent training of the two heads is worse than joint training, supporting the claim that the two modalities reinforce each other.","pith_inferences":["A ceiling implied by the design, though not stated in the paper, is that the visual guidance is only as informative as the CLIP embedding space used as supervision; edits that change pixels but not CLIP semantics are likely to be missed, and replacing the regression target with patch-level or dense features would test this directly.","The joint-training result hints at a shared reasoning pathway between the text head and the vision head; one could probe this by ablating the intermediate layer index $m$ and measuring whether the optimal layer shifts with instruction complexity.","A natural extension is to apply the same insight-driven embedding to video editing, where per-frame consistent edits need the same visual intent carried across time."],"forward_implications":["Editorial intent can be transferred as an embedding rather than only as text, which gives diffusion models a direct visual target to reconstruct.","The joint text-plus-embedding training means an editing model can improve both its instruction rewriting and its visual guidance with one objective, rather than maintaining two separate models.","Because the visual representation plugs into an existing image adapter, the approach can be retrofitted to other pretrained diffusion backbones without retraining the adapter.","The reported gains on the complex multi-step dataset suggest the method is most valuable where instructions require inference about layout, object state, or style, not just vocabulary matching."],"supporting_citations":[{"why":"Supplies the HumanEdit training and test sets, including the 5,244 training samples and 500 test images used for the main evaluation.","marker":"[2]"},{"why":"Supplies the complex multi-step editing test set used to demonstrate the method's advantage in challenging scenarios.","marker":"[11]"},{"why":"One of the MLLM-based editing baselines compared against in the quantitative and qualitative experiments.","marker":"[19]"},{"why":"Another MLLM-based editing baseline, providing the comparison that motivates the claim of improved semantic alignment.","marker":"[20]"},{"why":"Provides the base multimodal large language model whose hidden states are fed to the vision head to produce the visual representation.","marker":"[26]"},{"why":"The strongest compared baseline, a dual-branch editing system, used to show competitive or better performance despite fewer parameters.","marker":"[35]"},{"why":"Supplies the CLIP image encoder used both as the ground-truth target for the visual embedding loss and for the CLIP-I metric.","marker":"[43]"},{"why":"Provides the stable diffusion backbone that the framework edits and evaluates on.","marker":"[44]"},{"why":"Provides the prior evidence and motivation that jointly optimizing textual and visual outputs can mutually enhance understanding and generation.","marker":"[52]"},{"why":"Supplies the image adapter through which the predicted visual representation is injected into the diffusion model.","marker":"[62]"}],"fun_headline_variants":["Visual insight, not text, guides precise image edits","MIND-Edit: using MLLM's inner vision for sharper edits","Diffusion editing with MLLM's visual intuition","Beyond text: MLLM visual embeddings for better edits","Image editing that sees what you mean, not just read"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole pipeline leans on the assumption that a single embedding of the target edited image, drawn from a standard image-text model, is a faithful and complete summary of what the edit should look like, and that the frozen image adapter can turn that summary back into pixels.","fun_headline_variants_meta":{"raw":{"variants":["Visual insight, not text, guides precise image edits","MIND-Edit: using MLLM's inner vision for sharper edits","Diffusion editing with MLLM's visual intuition","Beyond text: MLLM visual embeddings for better edits","Image editing that sees what you mean, not just read"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00026,"raw_usage":{"total_tokens":1598,"prompt_tokens":961,"completion_tokens":637,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":577,"completion_tokens_details":{"reasoning_tokens":554}},"tokens_in":577,"tokens_out":637,"duration_ms":5830,"temperature":1.0,"reasoning_tokens":554,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:18:40.464812+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a test set of edits whose target images have near-identical CLIP embeddings to their sources, such as changing a small object's color or removing one element, and measure whether MIND-Edit still reproduces the edit. If the predicted visual embedding is nearly the source embedding and the edit fails, the assumption that CLIP embeddings encode the needed editing intent is the weak link.","supporting_citations":[{"cited_title":"Complex multistep image-editing dataset, 2025","cited_arxiv_id":null,"evidence_quote":"Supplies the complex multi-step editing test set used to demonstrate the method's advantage in challenging scenarios."},{"cited_title":"Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion","cited_arxiv_id":null,"evidence_quote":"Another MLLM-based editing baseline, providing the comparison that motivates the claim of improved semantic alignment."},{"cited_title":"Magicquill: An intelligent interactive image editing system","cited_arxiv_id":null,"evidence_quote":"The strongest compared baseline, a dual-branch editing system, used to show competitive or better performance despite fewer parameters."}],"review_version":1}