{"id":"74517f86-5534-437c-9771-dccaf0a219a4","arxiv_id":"2411.17323","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"InsightEdit uses multimodal language model features in a two-stream adapter to improve complex instruction following and background consistency in image editing.","lead":"This paper introduces InsightEdit, a model that edits images by following complex text instructions while preserving the original background, plus a new 2.5 million pair training dataset. A generalist reader might care because it pushes toward more reliable and controllable AI image editing, a capability with broad applications in design, content creation, and accessibility.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA claim depends on VIEScore, the same MLLM metric used to filter the AdvancedEdit training set; without a human-preference check or a disjoint eval set, the reported advantage may be an artifact of the judge.","rationale":"I read the paper's central claim as SOTA editing performance due to the two-stream bridging and AdvancedEdit data. The experimental support is real: Table 3 shows InsightEdit beating SmartEdit on Reason-Edit, an external benchmark, and multi-metric support (PSNR, SSIM, LPIPS, CLIPScore) is provided. The strongest weakness is not the architecture but the evaluation loop. Because VIEScore is used in Step 5 of data construction and again as the primary semantic metric, the metric is not an independent arbiter; because AdvancedEdit-Eval comes from the same pipeline, the comparison is partly in-domain. The paper itself does not report human evaluation, error bars, or significance tests, and does not release data or code. This does not invalidate the method, but it does mean the SOTA claim is conditional on VIEScore matching human judgment for this task. A human pairwise test would settle the concern. No evidence of fraud or internal inconsistency was found.","tokens_in":12469,"tokens_out":5281,"duration_ms":54495,"concrete_test":"Run a pairwise human preference study on a random 100 prompts from AdvancedEdit-Eval and 100 from Reason-Edit (held-out reasoning split), comparing InsightEdit vs SmartEdit-7B side-by-side; then compute the fraction of cases where human preference agrees with VIEScore ordering. If agreement is not significantly above chance (or if humans prefer SmartEdit on a substantial minority of items), the SOTA claim is not supported. In the same release, verify that no source image or editing pair in AdvancedEdit-Eval overlaps the 202,822 training pairs.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline result (VIEScore 0.831 on AdvancedEdit-Eval; 0.947 on Reason-Edit reasoning) is produced by an MLLM judge that is also part of the data-generation loop. Section 3.1 Step 5 filters training pairs with VIEScore; Section 5.1 then uses VIEScore as the primary semantic metric. The eval set AdvancedEdit-Eval is curated from the same automated pipeline as the training data, and InsightEdit was trained on 202,822 pairs from AdvancedEdit, while the baselines were not. This creates a double advantage: the model is fitted to imitate targets that were selected for high VIEScore, and the judge is the same one used for selection. If VIEScore has any systematic preference—clean inpainted regions, conservative edits, or template-like outputs—it will be baked into both training and evaluation. Reason-Edit is a genuine external check, but VIEScore is still the judge there, and no human validation or error bars are reported. Without release of the dataset/eval split, overlap between AdvancedEdit-Eval and the training subset cannot even be checked.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes InsightEdit, an instruction-based image editing method that introduces a two-stream bridging mechanism to inject both textual and visual features from an MLLM into a diffusion model, along with a new Image Alignment Adapter (IAA) for target-image supervision. The authors also present an automated data construction pipeline that generates the AdvancedEdit dataset (2.5M editing pairs) and a 300-pair AdvancedEdit-Eval benchmark. Experiments report state-of-the-art VIEScore results on AdvancedEdit-Eval and Reason-Edit, with ablations showing gains from IAA and from the AdvancedEdit data.","tokens_in":12694,"tokens_out":6587,"duration_ms":56810,"significance":"If validated, the AdvancedEdit dataset could be a valuable large-scale resource for instruction-based image editing, and the IAA mechanism is a plausible way to inject target-image information explicitly. The two-stream bridging architecture is incremental but reasonable. The main significance, however, is undermined by evaluation issues: the primary metric (VIEScore) is also used as a data filter, the self-created benchmark is from the same pipeline, and the headline comparisons confound the method contribution with additional training data. The paper would be substantially strengthened by a human study, controlled comparisons, and significance testing.","major_comments":[{"comment":"The paper uses VIEScore both to filter the AdvancedEdit training pairs (Section 3.1 Step 5) and as the primary semantic metric on the self-created AdvancedEdit-Eval (Section 5.2). This creates a selection bias: the model is trained to imitate targets that were selected for high VIEScore, and the same judge is then used for evaluation. Since the baselines were not trained on data filtered this way, the reported VIEScore gap (e.g., 0.831 vs. 0.682 for SmartEdit in Table 2) may reflect this bias rather than a genuine editing-quality advantage. The concern extends to the Reason-Edit results, where VIEScore is also the judge. Please add a human preference study or an independent judge (e.g., a different MLLM) and report the correlation with VIEScore.","section":"Section 3.1 Step 5, Section 5.2"},{"comment":"The 'InsightEdit with AdvancedEdit' rows have access to 202,822 additional training pairs from a dataset constructed with the same pipeline as the evaluation set, while the baselines do not. This confounds the method contribution with the data contribution. For example, in Table 3, 'InsightEdit with AdvancedEdit' outperforms SmartEdit by a large margin, but it also uses substantially more training data. To support the architectural claim, the authors should present an architecture-only comparison where InsightEdit and SmartEdit are trained on identical data (the 'InsightEdit' row in Table 3 is one such comparison, but the paper still highlights the AdvancedEdit row as SOTA). The data contribution should be framed separately, not as evidence for the architecture.","section":"Section 5.2, Tables 2 and 3"},{"comment":"All quantitative results are reported as single runs without error bars or significance tests. VIEScore is an MLLM-based, likely stochastic judge; differences such as 0.831 vs. 0.738 (Table 2) or 7.545 vs. 7.307 (Table 4) could be within run-to-run variation. The authors should provide multiple runs or bootstrap confidence intervals for the primary comparisons, and ideally a paired significance test for the head-to-head comparisons.","section":"Section 5.2, Tables 2-5"},{"comment":"The VIEScore scale is inconsistent across tables: Table 2 reports values around 0.3–0.8, while Tables 4 and 5 report values around 7.0–7.5 for the same configuration (e.g., InsightEdit with AdvancedEdit has identical PSNR/SSIM/LPIPS/CLIPScore in Table 2 and Table 4, but VIEScore of 0.831 in Table 2 and 7.545 in Table 4). This inconsistency is unexplained and makes it impossible to compare results across tables. The authors should clarify the normalization procedure and ensure a uniform scale throughout the paper.","section":"Tables 2, 4, and 5"},{"comment":"The paper states that VIEScore 'might better align with human preferences' but provides no human evaluation. Given that the main metric is an MLLM and is also used as a data filter, human validation is essential to support the central claim. A human preference study on a subset of AdvancedEdit-Eval and Reason-Edit (e.g., 100 examples each, comparing InsightEdit vs. SmartEdit) should be added.","section":"Section 5.1, Section 5.2"}],"minor_comments":[{"comment":"Typo: 'qualititive' should be 'qualitative'.","section":"Section 5.2"},{"comment":"The label 'InstructPixPix' should be 'InstructPix2Pix'.","section":"Figure 4"},{"comment":"Some extracted text in the figure has spacing artifacts (e.g., 'v ersion', 'ob j ect'); please check the original figures for clean rendering.","section":"Figure 2"},{"comment":"The check and cross symbols are not defined in the caption; please clarify their meaning.","section":"Table 1"},{"comment":"The prompt used for instruction recaptioning is not provided; including the exact prompt would improve reproducibility.","section":"Section 3.1 Step 4"},{"comment":"The hyperparameters r (number of MM tokens) and lambda in Eq. (6) are not specified; please report their values in the implementation details.","section":"Section 4.2 and 4.3"},{"comment":"The acronyms IAA, BIM, Q-Former, and MM are used in the figure but the caption does not expand them; please add a legend or describe them in the caption.","section":"Figure 3"},{"comment":"The paper references an appendix multiple times (e.g., 'detailed in the Appendix'), but the version under review contains no appendix; please ensure the appendix is included in the final submission.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper has two potentially valuable contributions: the large-scale AdvancedEdit dataset and the IAA/two-stream bridging architecture. However, the evaluation as written is not convincing due to the VIEScore circularity, the confounded comparisons, and the absence of human study or significance testing. The VIEScore scale inconsistency across tables is also concerning and suggests careless reporting. I would encourage the editor to request a major revision with specific demands: human evaluation, controlled method-only comparison, significance tests, clarification of VIEScore normalization, and, ideally, release of the dataset and eval split for external verification."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The two things you should know: this is a solid, well-engineered paper, and its headline number has a built-in advantage that the authors never acknowledge. The AdvancedEdit dataset (2.5M pairs, 1K resolution, complex instructions) and the InsightEdit two-stream conditioning with the IAA adapter are genuinely new. The Reason-Edit results are the most convincing part—InsightEdit beats SmartEdit on an externally constructed benchmark, and the gains are consistent across VIEScore, PSNR, SSIM, and LPIPS. That is real evidence the architecture and data help, not just a self-reported win.\n\nThe soft spot is exactly what the stress-test note flags: VIEScore is used both to filter the training data (Section 3.1 Step 5) and as the primary semantic metric on both eval sets. The eval set AdvancedEdit-Eval is curated from the same pipeline, and baselines were not trained on any AdvancedEdit pairs. That is a double advantage. The fact that PSNR and LPIPS also improve on AdvancedEdit-Eval mitigates this some—those metrics are not VIEScore—but they mostly measure background consistency, not instruction following. CLIPScore against the GT label is weak. The external Reason-Edit benchmark helps, but VIEScore is still the judge there, and no human-preference check or error bars are provided. So the exact magnitude of the SOTA claim is uncertain, even if the direction is probably right.\n\nMinor issues: only 202K of 2.5M pairs were used, which is fine but makes the dataset contribution partly aspirational. No code or data released, so the overlap between AdvancedEdit-Eval and the training subset cannot be checked. The qualitative figures are cherry-picked, as usual. Table 1's \"good background consistency\" for AdvancedEdit rests on the same VIEScore filter.\n\nWho this is for: anyone working on instruction-based editing, especially on dataset construction or MLLM-conditioned diffusion. The architecture is a reasonable extension of SmartEdit with a useful new adapter. If the authors release the dataset and eval split, this becomes a citable resource.\n\nRecommendation: send it to peer review. It deserves a serious referee. The reviewer should push for a human evaluation on a few hundred samples, error bars, and either a disjoint judge or a clear argument that VIEScore bias does not drive the result. Without those, the exact claims should be softened, but the core contribution stands.","headline":"Real engineering, a sizable new dataset, and an external-benchmark win, but the SOTA claim leans on a VIEScore judge that the authors also used to filter training data.","tokens_in":13264,"tokens_out":1268,"would_cite":false,"duration_ms":15082,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that instruction-based image editing improves when the diffusion model conditions on MLLM-reasoned visual features, not just text, and that a new 2.5-million-pair dataset trains this ability.","keywords":["instruction-based image editing","diffusion models","multimodal large language models","visual conditioning","image editing dataset","background consistency","VIEScore","instruction following"],"falsifier":"Run a forced-choice human study on the 300 AdvancedEdit-Eval image pairs: if people do not prefer InsightEdit's edits over SmartEdit's at roughly the rate the VIEScore gap (0.831 vs 0.682) predicts, the claimed state-of-the-art edge is an artifact of the shared metric. A second check would rebuild the dataset with a different human-aligned filter and see whether the advantage survives.","tokens_in":80,"feed_emoji":"🖼️","tokens_out":9986,"duration_ms":146296,"temperature":0.7,"pith_summary":"The paper tries to establish that instruction-based image editing fails today for two fixable reasons: the training datasets are too simple and low-quality, and the models condition on the text instruction while ignoring the visual content of the image. To fix the data side, it builds AdvancedEdit, an automated pipeline producing 2.5 million editing pairs at high resolution by masking objects, generating targets with mask-based editors, recaptioning instructions into simple and complex forms, and filtering with a multimodal-LLM quality score. To fix the conditioning side, it adds a visual stream that carries image features reasoned by a multimodal large language model into the diffusion denoiser, alongside the usual text stream. The resulting model, InsightEdit, is reported to outperform InstructPix2Pix, MagicBrush, InstructDiffusion, MGIE, and SmartEdit on the paper's AdvancedEdit-Eval and on Reason-Edit, with the largest margins on complex instructions and background preservation.","feed_headline":"Visual reasoning stream lifts image editing to state of the art","feed_subtitle":"InsightEdit feeds image features into the diffusion model, not just text, and beats prior editors on complex-instruction benchmarks","key_machinery":"The load-bearing object is the two-stream bridging mechanism. The text branch takes the hidden states of the [MM] tokens that the MLLM is trained to predict, refines them with a Q-Former and a bidirectional information module (BIM), and feeds the result as the text condition. The image branch's Image Alignment Adapter (IAA) maps those same hidden states through an MLP mapper into a CLIP embedding, trained against the CLIP features of the target image, so the condition carries what the edited result should look like. A decoupled cross-attention layer, borrowed from image-prompt adapters, lets the UNet attend to the image stream with an adjustable weight $\\lambda$. The supporting data pipeline uses object extraction, mask generation, mask-based editing, instruction recaptioning, and VIEScore filtering to produce the AdvancedEdit pairs.","core_discovery":"The central claim is that a mask-free editing model can follow complex instructions and keep unedited regions intact only when the diffusion process receives both the text instruction and the visual semantics extracted from the source image, not text alone. InsightEdit implements this with a two-stream bridging module: the text stream processes the hidden states of the [MM] tokens produced by a multimodal large language model through a Q-Former and a bidirectional information module, while the image stream's Image Alignment Adapter (IAA) maps the same hidden states into a CLIP embedding that is supervised by the target image's CLIP features. A decoupled cross-attention layer in the UNet fuses the two streams at a tunable weight. Trained on the AdvancedEdit dataset, the paper reports a VIEScore of 0.831 on AdvancedEdit-Eval and 0.947 on Reason-Edit reasoning scenarios, against SmartEdit's 0.682 and 0.835, with better PSNR, SSIM, LPIPS, and CLIPScore.","pith_inferences":["The paper leaves the text branch unablated: zeroing or masking the text-condition stream at inference would isolate how much of the gain comes from the visual stream alone, which is a direct test the current ablations do not run.","Because VIEScore both filters the training data and reports the headline gains, the numerical advantage could partly reflect the model matching the metric's own preferences; a human preference study on the same 300 pairs would separate the two effects.","The data pipeline's steps (caption, extract objects, generate masks, edit, recaption, filter) are task-agnostic, so the same recipe could be carried to video editing or object-level video manipulation if a mask-based video editor is available; the paper does not explore this.","The paper itself lists its remaining limits as the instruction understanding of the MLLM and the power of the diffusion backbone, so the ceiling of the approach is tied to progress in those two components."],"forward_implications":["Because the image condition enters through a decoupled cross-attention layer with a tunable weight $\\lambda$, the same bridging design can be grafted onto other diffusion backbones without retraining the whole UNet.","The AdvancedEdit pipeline's remove/add/replace pairs give other mask-free editing models a ready-made training resource for complex instructions and background consistency.","Training on advanced recaptioned instructions raises both instruction-following scores and background-consistency metrics, indicating that data complexity and visual conditioning reinforce each other.","The IAA loss ties a hidden-state embedding directly to the target image's CLIP embedding, which is why the edited object matches the intended concept while the unedited scene stays close to the source."],"supporting_citations":[{"why":"Establishes the instruction-editing task and supplies the dataset and baseline the paper builds on.","marker":"[2]"},{"why":"The closest prior method; InsightEdit extends its MLLM text-conditioning with a visual stream and is compared directly against it.","marker":"[14]"},{"why":"Mask-based editor used to produce high-quality target images for the AdvancedEdit pairs.","marker":"[16]"},{"why":"Supplies the MLLM-based metric that filters training pairs during dataset construction and serves as the main evaluation score.","marker":"[20]"},{"why":"The multimodal LLM backbone whose hidden states carry the reasoned textual and visual features used by the bridging module.","marker":"[24]"},{"why":"Grounding and segmentation model that produces the object masks used in editing-pair construction.","marker":"[27]"},{"why":"Supplies the decoupled cross-attention design that fuses image features into the UNet.","marker":"[37]"},{"why":"Mask-based editor used to generate target images, particularly for replacement tasks.","marker":"[44]"}],"fun_headline_variants":["Two-stream visual reasoning elevates image editing","InsightEdit: Visual cues improve instruction following","Image editing accuracy boosted by visual-text fusion","Dual-stream model enhances complex image edits"],"cache_read_input_tokens":15360,"weakest_assumption_plain":"The evaluation assumes the automated VIEScore metric matches human judgment closely enough that selecting training pairs with it and then measuring success with it is a fair test.","fun_headline_variants_meta":{"raw":{"variants":["Two-stream visual reasoning elevates image editing","InsightEdit: Visual cues improve instruction following","Image editing accuracy boosted by visual-text fusion","Dual-stream model enhances complex image edits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000171,"raw_usage":{"total_tokens":1266,"prompt_tokens":935,"completion_tokens":331,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":275}},"tokens_in":551,"tokens_out":331,"duration_ms":4592,"temperature":1.0,"reasoning_tokens":275,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:13:26.013773+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a forced-choice human study on the 300 AdvancedEdit-Eval image pairs: if people do not prefer InsightEdit's edits over SmartEdit's at roughly the rate the VIEScore gap (0.831 vs 0.682) predicts, the claimed state-of-the-art edge is an artifact of the shared metric. A second check would rebuild the dataset with a different human-aligned filter and see whether the advantage survives.","supporting_citations":[{"cited_title":"In- structpix2pix: Learning to follow image editing instructions","cited_arxiv_id":null,"evidence_quote":"Establishes the instruction-editing task and supplies the dataset and baseline the paper builds on."},{"cited_title":"Smartedit: Exploring complex instruction-based image editing with multimodal large lan- guage models","cited_arxiv_id":null,"evidence_quote":"The closest prior method; InsightEdit extends its MLLM text-conditioning with a visual stream and is compared directly against it."}],"review_version":1}