pith. sign in

arxiv: 2605.19639 · v1 · pith:RTDRM53Fnew · submitted 2026-05-19 · 💻 cs.CV

Benchmarking and Evolving Reason-Reflect-Rectify for Reflective Visual Generation

Junjie Wang , Xinghua Lou , Jason Li , Ye Tian , Keyu Chen , Yulin Li , Bin Kang , Jacky Mai
show 3 more authors
Yanwei Li Zhuotao Tian Liqiang Nie
This is my paper

Pith reviewed 2026-05-20 06:22 UTC · model grok-4.3

classification 💻 cs.CV
keywords reflective visual generationR^3 loopR^3-BenchR^3-Refinertext-to-image modelsmultimodal large language modelsiterative refinementpolicy optimization
0
0 comments X

The pith

R^3-Refiner trains models to turn error detection into concrete fixes for complex text-to-image prompts.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to show that single-pass text-to-image models fall short on detailed prompts because they lack a built-in loop for spotting mistakes and issuing useful corrections. It creates R^3-Bench with more than 600 expert-annotated cases to measure how well models reason about an image, reflect on its flaws, and then produce rectification steps. The authors then build R^3-Refiner, a dual-stage system that uses group relative policy optimization and a hierarchical reward signal to teach models to generate actionable fixes rather than vague observations. A reader should care because successful iterative refinement would let existing generators handle prompts that currently produce systematic errors without requiring manual prompt engineering or external human feedback.

Core claim

The paper claims that current multimodal models can identify generation errors but fail to output rectification instructions that actually improve the image; by formalizing the Reason-Reflect-Rectify loop and training R^3-Refiner with Group Relative Policy Optimization plus a Hierarchical Reward Mechanism, the method raises Reflective Verdict Score by 12 percent and Rectification Score by 9 percent on R^3-Bench while also lifting quality on GenEval++ and T2I-CompBench when paired with different MLLMs and T2I backbones.

What carries the argument

R^3-Refiner, a dual-stage framework that first aligns reflective reasoning with rectification instructions via Group Relative Policy Optimization and then applies a Hierarchical Reward Mechanism to score the quality of the resulting fixes.

If this is right

  • T2I models paired with R^3-Refiner produce higher-quality outputs on complex compositional prompts measured by GenEval++ and T2I-CompBench.
  • The same refiner module can be attached to multiple different MLLMs without retraining the underlying image generator.
  • Iterative refinement becomes a measurable and trainable skill rather than an emergent property left to chance.
  • Systems that previously only detected errors now generate concrete correction steps that close the loop back to improved images.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same reward-alignment approach could be tested on video or 3D generation tasks where successive frames or views must be iteratively corrected.
  • R^3-Bench itself could serve as a diagnostic tool to compare how different base models differ in their ability to produce usable rectification instructions.
  • If the dual-stage training generalizes, similar loops might improve agents that act in visual environments by turning internal reflection into executable corrections.

Load-bearing premise

The expert-annotated instances in R^3-Bench provide a stable and unbiased measure of iterative reasoning and rectification quality that generalizes beyond the specific 600-plus cases collected.

What would settle it

Running R^3-Refiner on a fresh collection of 200 expert-annotated prompts drawn from the same distribution but never seen during benchmark construction, then measuring whether the reported 12-point and 9-point gains still appear, would confirm or refute that the improvements reflect genuine capability rather than overfitting to the original set.

Figures

Figures reproduced from arXiv: 2605.19639 by Bin Kang, Jacky Mai, Jason Li, Junjie Wang, Keyu Chen, Liqiang Nie, Xinghua Lou, Yanwei Li, Ye Tian, Yulin Li, Zhuotao Tian.

Figure 1
Figure 1. Figure 1: (a) Existing text-to-image (T2I) models often struggle with compositional prompts, resulting in diverse visual generation errors. (b) Reflective visual generation aims to mitigate these er￾rors, yet current models suffer from a capability misalignment: they accurately diagnose flaws (strong reasoning) but fail to exe￾cute valid corrections (weak rectification). (c) To bridge this gap, we introduce R3 -Benc… view at source ↗
Figure 2
Figure 2. Figure 2: Comparison between existing benchmarks and R3 -Bench. (a) Existing benchmarks predominantly evaluate image generation, editing, and visual verification as isolated tasks. (b) In contrast, R3 -Bench centers on the “Reason-Reflect-Rectify” loop for Reflective Visual Generation (RVG). As illustrated by the “silver spoon” example, the model first employs reasoning to diagnose inconsistency by providing a verdi… view at source ↗
Figure 3
Figure 3. Figure 3: Overview of R3 -Bench. The benchmark covers eight categories sourced from both real-world and model-generated data, comprising 222 aligned and 448 misaligned instances. on rectified images. Benchmark Construction. We construct the benchmark through a multi-stage process designed to ensure clarity and diversity. To form an initial candidate pool, we aggregate data from complementary sources, combining error… view at source ↗
Figure 4
Figure 4. Figure 4: The policy πθ samples N structured trajectories via GRPO. The optimization is driven by a Hierarchical Reward Mechanism (HRM) comprising two stages: Reasoning Alignment (Rreason) and Rectification Alignment (Rrect). as a tuple oi = ⟨vˆi , eˆi , aˆi⟩, corresponding to the Reason, Reflect, and Rectify components, respectively. We imple￾ment the dual-stage optimization of these trajectories via the following … view at source ↗
Figure 5
Figure 5. Figure 5: Effectiveness of HRM. The Reasoning Alignment Re￾ward improves verdict accuracy but induces Illusory Visual Rectifi￾cation. As shown in (b), the policy learns to edit the prompt rather than refining the image. The Rectification Alignment Reward in (c) alleviates this behavior and encourages valid visual rectification. ground its judgments explicitly on accurate visual evidence. By enforcing correctness in … view at source ↗
Figure 6
Figure 6. Figure 6: R 3 -Refiner utilizes the iterative R3 loop to continuously rectify image errors, allowing the final image quality to scale with the number of refinement steps. are maintained as positive instances, while prompts un￾dergo semantic alterations to intentionally contradict vi￾sual content, generating challenging negatives that neces￾sitate precise grounding. Additionally, to simulate realis￾tic application sc… view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative comparison of R3 -Refiner with existing MLLMs, UMMs, and RVG methods [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Hierarchical Distribution of the R3 -Dataset. The inner ring displays data sources, including T2I-R1, BLIP-3O, and PICO-Banana. Moving outward, the middle ring illustrates fine￾grained error categories, such as Spatial, Color, and Numeracy. These categories are distributed evenly to maintain balanced vi￾sual diversity. Finally, the outer ring details the composition of the final preference pairs by indicat… view at source ↗
Figure 9
Figure 9. Figure 9: Overview of the Scalable Paired Data Construction Pipeline. The proposed pipeline is structured into two primary phases. The Multi-Source Synthesis Strategies phase initially lever￾ages Generative Ranking, Counterfactual Rewriting, and Visual Inversion to synthesize diverse candidates with hard negatives. Subsequently, the Cascaded Filtering phase implements a three￾stage verification mechanism comprising … view at source ↗
Figure 10
Figure 10. Figure 10: Qualitative comparison of filtered noise and final data. Top: Low-quality samples rejected by our pipeline due to hallucinations, ambiguity, or visual artifacts. Bottom: High-quality retained preference pairs. Each pair consists of an Aligned image (matching the prompt) and a Misaligned image (containing specific errors), providing the contrastive signal needed for training. ures. Within this scope, we cu… view at source ↗
Figure 11
Figure 11. Figure 11: Visualizations of R3 -Bench. The benchmark spans eight fine-grained categories (e.g., Spatial, Numeracy, Complex), designed to rigorously test visual reasoning and rectification. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Qualitative comparison on Stage I (Verdict). Baselines like Bagel and OmniGen2 often fail to detect semantic mismatches (e.g., identifying a pink toy car as a “pink toaster”). R3 -Refiner correctly issues a “False” verdict based on precise visual evidence. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Qualitative comparison on Stage II (Reflection). Existing methods frequently hallucinate details in their explanations. For instance, ReasonEdit attempts to correct the color of a non-existent object. R3 -Refiner correctly identifies the root cause (e.g., missing object) without fabrication. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Qualitative comparison on Stage III (Rectification). A common failure mode in baselines (e.g., ThinkGen, OmniVerifier) is proposing to edit the text prompt instead of the image. R3 -Refiner generates specific image editing instructions (e.g., “Add a green boat”) to align the visual content with the original prompt. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Visualization of failure cases. We provide several visualizations of failure cases, corresponding to two primary failure types: 1) Dense Numeracy Errors (panels (a) and (b)), 2) Editor Capability Limits (panels (c) and (d)). Change the skateboard color to red, modify the bus color attribute to black and modify the truck color attribute to yellow Change the bus color to black and change the truck color to … view at source ↗
Figure 16
Figure 16. Figure 16: Qualitative visualization of the iterative refinement process. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: The prompt used to train the R3 -Refiner policy. This prompt enforces the iterative R3 loop (Sec. 2.1) by requiring the model to first generate an explicit internal reasoning process (CoT) before producing the structured tuple ⟨vt, et, at⟩. These components are mapped to the JSON fields "answer" (corresponding to vt, Reason), "explanation" (corresponding to et, Reflect), and "edit prompt" (corresponding t… view at source ↗
Figure 18
Figure 18. Figure 18: The exact system prompt used by the LLM-Judge J in Phase I. This judge evaluates whether the generated reflection eˆi is semantically equivalent to the ground truth explanation ei, which is used to compute the Reflective Verdict Score (Sref). 27 [PITH_FULL_IMAGE:figures/full_fig_p027_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: The few-shot prompt (8 examples) used to decompose user prompts into atomic boolean questions. To save space, the JSON outputs in the examples are displayed in a compact format; the actual prompt uses standard JSON indentation. Phase II Evaluation: VQA-based Alignment Function (V) You are tasked with conducting a careful examination of the image. Based on the content of the image, please answer the follow… view at source ↗
Figure 20
Figure 20. Figure 20: The prompt template used for the VQA-based alignment function V in Phase II. This prompt directs the external MLLM to answer the visual question set Qi, producing the scores required to calculate the Rectification Score (Srect). 28 [PITH_FULL_IMAGE:figures/full_fig_p028_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: The system prompt used to generate fine-grained hard negatives via counterfactual rewriting. The model rewrites a correct caption into a misaligned one by altering specific visual attributes. Data Construction: Visual Inversion System Instruction: You are an expert visual analyst capable of reverse-engineering image editing instructions. Input: You are provided with two images: 1. Pre-edit Image (<image 1… view at source ↗
Figure 22
Figure 22. Figure 22: The VLM prompt used for visual inversion. By comparing the pre-edit and post-edit images, the model infers the target prompt P that aligns with the corrected visual state. 29 [PITH_FULL_IMAGE:figures/full_fig_p029_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: The prompts used in the Rationale Verification phase. The Proposer first generates a verdict with explicit reasoning (Step 1), and the Verifier audits the factual grounding of that explanation (Step 2) to filter hallucinations. 30 [PITH_FULL_IMAGE:figures/full_fig_p030_23.png] view at source ↗
read the original abstract

Text-to-Image (T2I) models and Unified Multimodal Models (UMMs) have achieved remarkable progress in visual generation. However, their reliance on a single-pass generation paradigm limits their ability to handle complex prompts requiring iterative refinement. To enable multi-round Reflective Visual Generation (RVG), we formalize the Reason-Reflect-Rectify (R^3) loop as a core framework and introduce R^3-Bench, a benchmark of over 600 expert-annotated instances that quantifies iterative reasoning and rectification capabilities. Evaluation on R^3-Bench reveals a critical gap: while state-of-the-art models can identify generation errors, they fail to generate actionable rectification instructions. To bridge this gap, we propose R^3-Refiner, a dual-stage framework leveraging Group Relative Policy Optimization (GRPO) and a Hierarchical Reward Mechanism (HRM) to better align rectification with reflective reasoning. Experiments show that R^3-Refiner achieves significant improvements on R^3-Bench (+12.0% in Reflective Verdict Score, +9.0% in Rectification Score), and can be seamlessly integrated with various MLLMs to enhance the generation quality of different T2I models on GenEval++ and T2I-CompBench. Code is available at https://github.com/xiaomoguhz/R3-Bench.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript formalizes the Reason-Reflect-Rectify (R^3) loop for multi-round reflective visual generation and introduces R^3-Bench, a collection of over 600 expert-annotated instances that define Reflective Verdict Score and Rectification Score to quantify iterative reasoning and rectification in text-to-image models. It further proposes R^3-Refiner, a dual-stage framework trained via Group Relative Policy Optimization (GRPO) and Hierarchical Reward Mechanism (HRM), reporting +12.0% and +9.0% gains on the two R^3-Bench metrics plus transfer improvements when integrated with MLLMs on GenEval++ and T2I-CompBench. Code is released at the cited repository.

Significance. If the gains prove robust, the work would usefully shift emphasis from single-pass generation toward explicit iterative rectification in multimodal models. The public code release supports reproducibility and is a clear strength. The central quantitative claims, however, rest on a newly constructed expert-annotated benchmark whose stability is not yet demonstrated in the manuscript.

major comments (2)
  1. R^3-Bench section: the Reflective Verdict Score and Rectification Score are computed from expert annotations on >600 instances, yet the manuscript provides neither inter-annotator agreement statistics nor a description of annotation guidelines, adjudication, or quality-control procedures. Because these scores are the primary evidence for the +12.0% and +9.0% improvements, the absence of reproducibility metrics directly weakens attribution of gains to the R^3-Refiner rather than annotation idiosyncrasies.
  2. Experiments section (results on R^3-Bench and transfer benchmarks): percentage improvements are stated without error bars, without ablation isolating GRPO from HRM, and without statistical significance tests. This makes it impossible to determine whether the reported deltas exceed what could arise from prompt engineering or baseline MLLM variability alone.
minor comments (2)
  1. Abstract: the phrase 'seamlessly integrated with various MLLMs' is used without a concise statement of the integration interface; a short description of the input/output format between R^3-Refiner and the host MLLM would improve clarity.
  2. Notation: the definitions of Reflective Verdict Score and Rectification Score should be given explicitly (e.g., as equations) rather than only described in prose, to facilitate future comparisons.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment point by point below and outline the revisions we will make to improve the manuscript's rigor and reproducibility.

read point-by-point responses

  1. Referee: R^3-Bench section: the Reflective Verdict Score and Rectification Score are computed from expert annotations on >600 instances, yet the manuscript provides neither inter-annotator agreement statistics nor a description of annotation guidelines, adjudication, or quality-control procedures. Because these scores are the primary evidence for the +12.0% and +9.0% improvements, the absence of reproducibility metrics directly weakens attribution of gains to the R^3-Refiner rather than annotation idiosyncrasies.

    Authors: We agree that the current manuscript lacks sufficient detail on the annotation process, which is a valid concern for benchmark reliability. In the revised version, we will add a new subsection in the R^3-Bench description that specifies the annotation guidelines given to experts, the multi-stage adjudication procedure for resolving disagreements, and quality-control steps such as spot-checks by senior annotators. We will also report inter-annotator agreement statistics (Fleiss' kappa) calculated on a random subset of 100 instances. These additions will directly support the attribution of the reported gains to the R^3-Refiner. revision: yes

  2. Referee: Experiments section (results on R^3-Bench and transfer benchmarks): percentage improvements are stated without error bars, without ablation isolating GRPO from HRM, and without statistical significance tests. This makes it impossible to determine whether the reported deltas exceed what could arise from prompt engineering or baseline MLLM variability alone.

    Authors: We concur that the absence of error bars, ablations, and significance testing limits the strength of the quantitative claims. We will revise the Experiments section to include error bars (standard deviation over three independent runs) for all R^3-Bench and transfer results. An ablation study isolating GRPO from HRM will be added, along with statistical significance tests (paired t-tests with p-values) comparing R^3-Refiner against baselines. We will also clarify that all compared methods use identical prompting templates to rule out prompt-engineering effects as the sole source of gains. revision: yes

Circularity Check

0 steps flagged

No circularity detected; empirical gains reported on held-out expert annotations and external benchmarks.

full rationale

The paper introduces R^3-Bench as a new collection of expert-annotated instances and reports quantitative improvements from R^3-Refiner (via GRPO + HRM) on both this benchmark and standard external suites (GenEval++, T2I-CompBench). No equations, self-definitions, or self-citation chains are present that reduce the reported deltas (+12% Reflective Verdict, +9% Rectification) to quantities fitted on the identical data used for the central claim. The evaluation protocol is described as using held-out annotations, making the result self-contained against external benchmarks rather than tautological by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 2 invented entities

The central claims rest on the assumption that expert annotations capture true reflective capability and that the GRPO+HRM training produces generalizable rectification behavior.

axioms (1)
  • domain assumption Expert annotations in R^3-Bench accurately quantify iterative reasoning and rectification quality
    Benchmark construction relies on over 600 expert-annotated instances to define success metrics.
invented entities (2)
  • R^3 loop (Reason-Reflect-Rectify) no independent evidence
    purpose: Core framework for multi-round reflective visual generation
    Formalized as the central mechanism the benchmark and refiner are built around.
  • R^3-Refiner no independent evidence
    purpose: Dual-stage framework using GRPO and HRM to align rectification with reflective reasoning
    New proposed model component whose gains are reported in the abstract.

pith-pipeline@v0.9.0 · 5807 in / 1490 out tokens · 47997 ms · 2026-05-20T06:22:53.359697+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

15 extracted references · 15 canonical work pages

  1. [1]

    Simultaneously, Z-Image (Cai et al., 2025) focuses on efficient native gen- eration architectures

    and MMaDA (Yang et al., 2025d) utilize large-scale interleaved multimodal data and exhibit emergent capabili- ties in complex generation and reasoning. Simultaneously, Z-Image (Cai et al., 2025) focuses on efficient native gen- eration architectures. Despite these advancements, these models still struggle with compositional prompts as they operate in an o...

    work page 2025

  2. [2]

    hard negatives

    employ chain-of-thought reasoning to optimize input prompts and guide the image generation and editing pro- cess. ThinkMorph (Gu et al., 2025) investigates interleaved multimodal reasoning to align semantic understanding with visual synthesis. SLD (Wu et al., 2024) and OmniVeri- fier (Zhang et al., 2025b) serve as plug-and-play verifiers that detect and c...

    work page 2025

  3. [3]

    explanation

    Identify the main error(s) and describe them briefly in “explanation”. •Clearly state what the prompt requires vs. what is actually shown in the image. • If there are multiple discrepancies (e.g., object missing, wrong color, wrong position, wrong count), you should mention all of the important ones in a concise way

    work page

  4. [4]

    edit prompt

    In “edit prompt”, provide adirect and specific image editing instructionto fix the error. • Choose the most appropriate action based on the actual error: add / remove / replace / move / change color / change shape / change texture / modify attribute / adjust count / resize / swap positions • The instruction can contain multiple coordinated edits in one se...

    work page

  5. [5]

    yes” or “no

    Each answer should be on a separate line, starting with “yes” or “no”, followed by the reason

    work page

  6. [6]

    The order of answers must correspond exactly to the order of the questions

    work page

  7. [7]

    Each question must have only one answer

    work page

  8. [8]

    Directly return the answers to each question, without any additional content

    work page

  9. [9]

    Each answer must be on its own line!

    work page

  10. [10]

    four”→“two

    Make sure the number of output answers equals the number of questions! Figure 20.The prompt template used for the VQA-based alignment function V in Phase II. This prompt directs the external MLLM to answer the visual question setQ i, producing the scores required to calculate the Rectification Score (Srect). 28 Benchmarking and Evolving Reason-Reflect-Rec...

    work page

  11. [11]

    Only change1 or 2 local details

    Keep theoverall scene, entities, and structuresimilar. Only change1 or 2 local details

    work page

  12. [12]

    The change must bespecific and objectively checkable

    work page

  13. [13]

    false_prompt

    The modification must be big enough to be false, but small enough to be plausible. Output format (JSON): { "false_prompt": "...", // the modified caption that is now false "change_type": "...", // e.g., "numeracy", "color" "changed_detail": "..." // a short explanation of what changed } Examples:Example (numeracy):Original: ”a photo of four coasters” {"fa...

    work page

  14. [14]

    If the teacher model’s answer is “true”: Verify that the explanation correctly describes why the image matches the prompt, and that the described elements actually exist in the image

    work page

  15. [15]

    false”: Verify that the explanation correctly identifies the actual problems, and that these problems truly exist in the image. Output Format: {

    If the teacher model’s answer is “false”: Verify that the explanation correctly identifies the actual problems, and that these problems truly exist in the image. Output Format: { "review_result": "pass" or "fail", "reasoning": "brief explanation..." } User Input:<image>Prompt:{prompt} Teacher Model’s Answer:{model answer} Teacher Model’s Explanation:{expl...

    work page