REVIEW 4 major objections 5 minor 8 cited by
RePlan shows that a reasoning-driven region planner plus training-free attention masking lets open-source diffusion editors handle complex, fine-grained image edits with far fewer artifacts and better grounding.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 15:23 UTC pith:ZFN2O7BM
load-bearing objection Clever training-free attention injection and a new benchmark, but the Weighted metric in Table 1 is arithmetically impossible and the evaluation is too self-referential to support the headline claim. the 4 major comments →
RePlan: Reasoning-guided Region Planning for Complex Instruction-based Image Editing
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
On the paper's own terms, the discovery is that a reasoning-driven planner plus attention masking is enough to unlock precise multi-region editing in a pretrained multimodal diffusion transformer. The VLM planner (Qwen2.5-VL 7B, reinforced with GRPO) decomposes an instruction into structured reasoning and a set of bounding-box/hint pairs, including negative hints that tell regions to stay unchanged. The editor then applies a training-free binary attention mask with five rules: text hints are isolated from each other, each image region attends only to its own hint and the global hint, background tokens attend only to the global hint, and image–latent tokens remain fully connected. This keeps
What carries the argument
The attention region injection mask is the central mechanism: a binary mask over the token sequence of an MMDiT (a diffusion transformer that jointly attends to text, image, and noise-latent tokens). The mask enforces five rules — intra-group connection, hint isolation, full image–latent interaction, region constraint (region tokens see only their own hint and the global hint), and background constraint (background tokens see only the global hint). It is training-free and applied at every attention layer, and it is what lets the model execute the planner's region-aligned guidance precisely.
Load-bearing premise
The evaluation premise: Gemini-2.5-Pro's five-point ratings on IV-Edit are a valid, bias-free measure, even though the benchmark's instructions and filters were generated with the same model family and the consistency dimension favors edits that leave regions untouched.
What would settle it
A human-rated study on a random subset of IV-Edit comparing RePlan against Flux.1 Kontext dev on the target and effect dimensions; if humans do not rate RePlan higher on those dimensions, the central claim weakens. A sharper ablation: feed the same region hints to Flux as a global prompt without attention injection — if scores are unchanged, the attention mask is not the cause of the gains.
If this is right
- Open-source MMDiT-based editors can be made substantially more precise on complex grounding tasks without retraining the diffusion model.
- Multiple region edits can be executed in a single pass, avoiding the accumulated errors and extra compute of iterative inpainting.
- A small reinforcement-learned planner (about 1k instruction-only examples) can produce reliable region guidance, reducing dependence on large paired editing datasets.
- Region-level negative prompts are supported directly, preventing unintended changes to semantically similar areas.
- The IV-Edit benchmark provides a public testbed for instruction–visual complexity, with fine-grained grounding and knowledge-intensive edits.
Where Pith is reading between the lines
- If the attention-isolation principle transfers, other diffusion architectures (e.g., cross-attention U-Nets) could gain similar spillover resistance by constraining which text tokens each spatial region attends to.
- The structured region-hint output makes interactive editing practical: a user can directly adjust a bounding box or hint and re-run the same one-pass pipeline.
- One testable extension: swap the planner for a stronger base VLM without RL to isolate how much of the gain comes from the attention mask versus the reinforced reasoning.
- The benchmark's consistency dimension is structurally aligned with the method's design, so future work should report human ratings or an independent judge before generalizing the claimed margin.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes RePlan, a plan-then-execute framework for instruction-based image editing under 'Instruction-Visual Complexity'. A VLM planner (Qwen2.5-VL-7B) is trained via two-stage GRPO with roughly 1k instruction-only examples to output region-aligned guidance (bounding boxes plus per-region hints). The diffusion decoder (Flux.1 Kontext or Qwen-Image-Edit, both MMDiT) executes this guidance through a training-free attention-region injection mechanism governed by five attention rules. The authors introduce IV-Edit, an 800-pair benchmark emphasizing fine-grained grounding and knowledge-intensive edits, and evaluate RePlan against several open and closed baselines using Gemini-2.5-Pro as a judge. They report consistent improvements in Overall and Consistency, along with ablations on planner choice, reasoning, and staged RL, and a bbox-perturbation robustness test.
Significance. If the quantitative claims are valid, the central contribution is notable: a small RL-trained planner plus a training-free attention regime can substantially improve open-source MMDiT editors on hard grounding tasks without requiring paired editing data. The attention-region injection mechanism is clearly specified, and the internal ablations (planner choice in Table 2, reasoning on/off and staged RL in Table 3) support the importance of the planner and the RL training. The bbox perturbation robustness test (Table 5) is a genuine and commendable probe of the method's dependence on region accuracy. However, the evaluation protocol currently prevents acceptance: the Weighted metric used for the headline comparisons is arithmetically inconsistent with the reported numbers, and the benchmark construction/judge share the same model family, with no human evaluation. The claimed edge over baselines is therefore not yet established.
major comments (4)
- [§5.1, Eq. (Weighted) and Table 1] The definition 'Weighted = (1/N) Σ (Target+Quality+Effect+Effect×Consistency)/4' cannot reproduce Table 1. Since all ratings are in [1,5], Effect×Consistency ≥ 1 for every sample. For the Flux baseline, the lower bound of the weighted average is (3.93+3.34+2.73+1)/4 = 2.75, but the table reports 2.49. The implied mean of Effect×Consistency would have to be about −0.04, which is impossible. Because Weighted is the metric introduced to counter consistency inflation, the central comparison (e.g., +0.06 for Flux, +0.29 for Qwen) is unsupported. Please correct the formula or provide per-sample computation and verify all Weighted values in Tables 1, 2, 3, and 5.
- [§4 Evaluation Protocol and Appendix C] The benchmark construction and evaluation are not independent: Gemini-2.5-Pro was used to generate and filter instructions (Appendix C), Qwen2.5-VL-72B annotated ground-truth bounding boxes, and Gemini-2.5-Pro also serves as the grading judge (Section 4). The judge therefore shares a model family with the benchmark generator. More seriously, RePlan's largest measured gains are in Consistency (+0.76 on Flux, +1.45 on Qwen in Table 1), and consistency is directly hard-enforced by attention rules 4 and 5, which restrict background tokens to attend only to the global text. Without a human evaluation or an independently developed judge, the risk that Gemini's ratings reward what the mechanism trivially enforces cannot be dismissed. Provide a human study on a representative sample or use a second, independently built VLM judge, and report agreement between judges.
- [Table 1, Effect column] The Effect dimension regresses on both backbones (Flux 2.73→2.59; Qwen 3.24→3.16), while the largest gains are in Consistency. This contradicts the paper's claim of 'consistently outperforming' and 'improving overall fidelity' in the abstract and introduction. The Weighted metric was designed to handle exactly this trade-off, but it is arithmetically broken (see above). Please provide a per-task breakdown, test whether the Effect regression is statistically significant, and discuss whether the attention rules trade away editing strength for preservation. Without this, the net benefit of RePlan on edit quality remains unclear.
- [§5.1 / all tables] The quantitative claims lack statistical validation. The reported comparisons are aggregate 5-point means over roughly 800 samples, with no error bars, confidence intervals, or significance tests. The headline Weighted improvement for Flux is +0.06 (2.49→2.55), a difference likely below inter-annotator noise. Report bootstrap confidence intervals, per-sample distributions, or paired significance tests (e.g., Wilcoxon) for the main comparisons, and make clear whether the reported numbers are means over all samples or medians.
minor comments (5)
- [Throughout] Typos: 'Specificly' in Section 4, 'anlysis' in §3.2, 'ensuring' in §3.3 ('This ensuring global stylistic coherence'), and inconsistent naming 'Gemini2.5-pro' in Table 2 vs 'Gemini-2.5-Pro' elsewhere. The formula in §5.1 uses 'Weighted=P samples...' due to typesetting; please use proper summation notation.
- [Related Work / References] The citation to Goodfellow et al. (2016) in the context of VLM reasoning abilities is inappropriate (it is a deep-learning textbook). Please replace it with a relevant reference or remove it.
- [§4 / Appendix C] The paper mentions 'manual verification' and 'manual sample-by-sample filtering' for the test set, but provides no details on annotator expertise, number of annotators, or inter-annotator agreement. Add these details for reproducibility.
- [§3.4] The 'Reasoning quality reward' uses the length of <think> content as a proxy, with a cap. This is an ad-hoc axiom; please justify this proxy or provide evidence that longer reasoning correlates with plan quality. Also clarify whether the '1k instruction-only examples' are used for supervised alignment before RL and what that alignment consists of.
- [Table 1, row ordering] Consider marking the second-best open-source model clearly; the underline is easy to miss in the printed table. Also, the closed-source models are included without discussion of their access/cost differences.
Circularity Check
No circular derivation found: RePlan's reported gains come from a stated training-free attention mask and separate RL training, not from fitting the evaluation score.
full rationale
The paper's central chain is plan-then-execute: a VLM planner outputs region–hint pairs, and a diffusion MMDiT decoder executes them under explicit attention rules (§3.2–3.3). These rules are mechanism design, not fitted parameters; the Consistency gain is a direct consequence of rules 4–5, so it is an engineered property being measured rather than a circular prediction. The GRPO training (§3.4) optimizes a reward that resembles the evaluation metric (consistency reweighted by effect), but the model is still evaluated on a separately filtered IV-Edit test set with manual sample-by-sample filtering (Appendix C), so the evaluation is not the training set and does not reduce by construction. The benchmark construction and grading by Gemini-2.5-Pro / Qwen2.5-VL-72B is a legitimate validity concern about judge alignment, but it is not a self-definitional derivation circle in the paper's equations. The §5.1 Weighted formula appears arithmetically inconsistent with Table 1 (the implied mean of Effect×Consistency would be negative), which is a serious correctness/reproducibility issue, not a circularity issue. No load-bearing self-citation or imported uniqueness theorem is used; self-citations in Related Work are non-essential. Therefore the circularity score is 0.
Axiom & Free-Parameter Ledger
free parameters (2)
- λ (format-reliability weight in Stage-2 reward) =
not stated ('a small weight')
- Reasoning-quality reward length cap =
capped maximum, value not stated
axioms (5)
- domain assumption Gemini-2.5-Pro ratings are valid ground-truth edit quality for Target/Effect/Consistency/Quality
- domain assumption IV-Edit's test set is an unbiased sample of IV-Complexity editing tasks
- domain assumption Pixel-space bounding boxes map correctly to VAE patch-grid token groups for attention masking
- domain assumption Full image-latent self-attention preserves global coherence so region-restricted text attention does not degrade out-of-region content
- ad hoc to paper Longer <think> text is a positive proxy for plan reasoning quality
read the original abstract
Instruction-based image editing enables natural-language control over visual modifications, yet existing models falter under Instruction-Visual Complexity (IV-Complexity), where intricate instructions meet cluttered or ambiguous scenes. We introduce RePlan (Region-aligned Planning), a plan-then-execute framework that couples a vision-language planner with a diffusion editor. The planner decomposes instructions via step-by-step reasoning and explicitly grounds them to target regions; the editor then applies changes using a training-free attention-region injection mechanism, enabling precise, parallel multi-region edits without iterative inpainting. To strengthen planning, we apply GRPO-based reinforcement learning using 1K instruction-only examples, yielding substantial gains in reasoning fidelity and format reliability. We further present IV-Edit, a benchmark focused on fine-grained grounding and knowledge-intensive edits. Across IV-Complex settings, RePlan consistently outperforms strong baselines trained on far larger datasets, improving regional precision and overall consistency. Our project page: https://replan-iv-edit.github.io
Figures
Forward citations
Cited by 8 Pith papers
-
GMO-E$^2$DIT: Grounded Multi-Operation Editing for E-Commerce Images
GMO-E²DIT is an agentic editing framework that decouples VLM-based planning from mask-conditioned rendering and uses reflection to execute multi-operation e-commerce image edits with error recovery.
-
GMO-E$^2$DIT: Grounded Multi-Operation Editing for E-Commerce Images
GMO-E²DIT is an agentic framework that decouples VLM-based edit planning from mask-conditioned rendering using reflection loops for reliable multi-operation e-commerce image editing.
-
Rethinking Scribble-Guided Image Editing: Generalization, Instruction Adherence, and Multi-Tasking
Empirical studies reveal instruction-level generalization as the main bottleneck in scribble-guided editing; three strategies (curriculum, multi-task mosaicking, edit-focused loss) achieve SOTA on VIBE benchmark.
-
DDA-Thinker: Decoupled Dual-Atomic Reinforcement Learning for Reasoning-Driven Image Editing
DDA-Thinker decouples planning from generation and applies dual-atomic RL with checklist-based rewards to boost reasoning in image editing, yielding competitive results on RISE-Bench and KRIS-Bench.
-
Making Image Editing Easier via Adaptive Task Reformulation with Agentic Executions
An MLLM agent reformulates image editing tasks into executable operation sequences to improve reliability on challenging cases across existing generative backbones.
-
Making Image Editing Easier via Adaptive Task Reformulation with Agentic Executions
An MLLM agent that profiles, routes, and reformulates image-editing queries into better-conditioned multi-step operations consistently improves existing editors on hard cases.
-
ProductConsistency: Improving Product Identity Preservation in Instruction-Based Image Editing via SFT and RL
Introduces ProductConsistency dataset, benchmark, and Cyclic Consistency reward to fine-tune image editing models, achieving a 5x reduction in character error rate for product identity preservation.
-
OmniVerifier-M1: Multimodal Meta-Verifier with Explicit Structured Recalibration
OmniVerifier-M1 is a generalist visual verifier using symbolic outputs for meta-verification and decoupled RL to outperform joint optimization for robust verification and agentic self-correction.
Reference graph
Works this paper leans on
-
[1]
Directly extract the core action, the target object, and any specific attributes
-
[2]
If the request involves multiple distinct steps, break it down into separate commands, one per line
-
[3]
Keep the commands as short and direct as possible
-
[4]
WAITED 8
Your response must contain ONLY the rephrased command(s). Do not add any explana- tions, apologies, or conversational text. Instruction: E ATTENTIONRULEDISCOVERY When experimenting with different attention rules, we observed several interesting phenomena. First, if we cut off the attention across different regions of an image, very clear boundaries appear...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.