{"id":"aa91a734-216e-4c95-95a9-646705882a4f","arxiv_id":"2608.04385","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Targeted self-diagnosis, not image re-injection alone, is what makes visual re-examination improve multi-step vision-language reasoning.","lead":"A new training method makes vision-language models pause, decide what they may have misread in an image, and look at the image again before answering. The central result is that this second look only helps if the model is told what to recheck; a generic 'look again' can make things worse.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 5's cue-quality ablation conflates diagnostic accuracy with cue specificity/answer-hint strength; without a wrong-target cue control, the central 'diagnostic quality' claim is underdetermined.","rationale":"The reader's weakest-assumption concern is about the unvalidated 72B judge and the oracle's validity as a diagnostic upper bound. My concern overlaps with that but is more specifically directed at the internal logic of the paper's central claim. Table 5 is the empirical foundation for 'diagnostic quality, not re-examination format alone, is the key variable.' The comparison it uses, generic versus oracle, varies more than one attribute: the oracle cue is both targeted and generated by a stronger model that can solve the task. A generic cue may hurt not because it lacks a diagnosis per se but because it is a vague instruction; an oracle cue may help because it supplies the critical step, which is closer to an answer hint than to a diagnosis. Without a wrong-target cue control, the paper cannot distinguish 'accurate diagnosis matters' from 'any specific target helps' or `stronger-model hints help.' This is load-bearing because the contribution is explicitly framed as diagnostic quality, and the recovery-rate metric (Self vs. Oracle) inherits the same confound. I do not think this invalidates the paper; the central claim is plausible, and the qualitative examples plus the text-only vs. visual ablation provide independent support that re-injection helps. But the sharpest version of the claim needs the missing control. The reader's verdict of CONDITIONAL is therefore appropriate, and my concern does not move it. I mark agreement as partial because the reader identified the oracle/general-capability issue but not the specific wrong-target confound that most directly tests the central claim.","tokens_in":22780,"tokens_out":8903,"duration_ms":100677,"concrete_test":"Run a wrong-target cue control on MathVista and HallusionBench with the same trigger schedule and policy model as Table 5. For the same triggered samples, prompt Qwen2.5-VL-72B to produce cues of identical format and length that direct attention to a salient but task-irrelevant visual element (e.g., background color, a decoration, or an unrelated label), then insert these cues at the same <reground> positions. If wrong-target cues recover more than roughly half of the oracle-vs-generic accuracy gap, the effect is driven by any concrete re-examination target, not by accurate self-diagnosis, and the central claim should be weakened. If wrong-target cues perform near the generic cue or below, diagnostic accuracy is the operative variable and the claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim rests on Table 5, which compares No-ReGround, a generic 'look again' cue, oracle 72B cues, and self-generated cues at fixed trigger positions. The generic cue is untargeted, whereas the oracle cues are produced by Qwen2.5-VL-72B, a model strong enough to solve the benchmark problems. The reported 4.6-8.9 point oracle-vs-generic gap therefore combines at least three variables: (a) presence of any concrete re-examination target, (b) accuracy of that target, and (c) answer-relevant hint strength that may leak the critical reasoning step despite the no-numeric-answer prompt. Since the paper's headline claim is that diagnostic quality ('what to re-examine') is the key variable, the design must separate target accuracy from target specificity and from raw informativeness. The current experiment cannot do this: a wrong-target cue of equal specificity and length is absent. The self-reported limitation in §3.1.1 that the grounding judge's agreement with human annotations is unmeasured further weakens the assertion that the oracle is a diagnostic upper bound rather than a general-capability/answer-hint upper bound. If a large part of the oracle advantage comes simply from having any specific target, the reported 79-90% recovery rates no longer measure learned diagnostic skill.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ReGround, a two-stage, tool-free framework for improving visual grounding in multi-step VLM reasoning. Stage 1 bootstraps training trajectories using a stronger model to generate self-diagnostic cues and re-injects the image, then trains the policy with SFT; Stage 2 uses GRPO with an asymmetric reward to calibrate when re-examination should be triggered. The authors report gains on eight benchmarks across Qwen2.5-VL-7B and Qwen3-VL-8B, provide an attention-entropy analysis of grounding decay and recovery, and claim that diagnostic quality—not the re-examination format alone—is the key variable governing whether visual re-examination helps or hurts self-correction. The main supporting evidence is a cue-quality ablation (Table 5) showing oracle/self-generated targeted cues outperform a generic 'look again' cue, with self-generated cues recovering 79-90% of the oracle's gains.","tokens_in":23039,"tokens_out":7764,"duration_ms":77831,"significance":"If the central claim holds, ReGround is a useful contribution: it identifies a plausible mechanism (targeted self-diagnosis) underlying effective visual re-examination and offers a practical bootstrapping recipe that avoids architectural changes and external tools. The paper is well-structured, ships code, and includes several careful ablations (Tables 3-5, Table 9/10) plus a mechanism-level attention analysis (Table 11) that is explicitly framed as proxy evidence. The empirical documentation of attention entropy decay/recovery across 2,510 samples is a solid observational contribution. However, the headline mechanistic claim about 'diagnostic quality' is not yet cleanly established, and the exploration-preference parameter is tuned directly on the test benchmarks; both issues need to be addressed before the paper's main interpretation can be accepted.","major_comments":[{"comment":"The cue-quality ablation confounds three variables: whether the cue names any concrete re-examination target, whether that target is accurate, and how much answer-relevant information the cue leaks. The generic cue ('Please look at the image again carefully') is targetless, whereas oracle and self cues name specific regions and are prohibited from giving numeric answers but may still reveal the critical reasoning step (e.g., 're-check whether DE is parallel to BC' essentially states the key relation). Thus the 4.6-8.9 point oracle-vs-generic gap could be driven by the mere presence of a specific target—even a wrong one—rather than by diagnostic accuracy. Since the paper's central claim is that diagnostic quality, not the re-examination format, is the key variable, the experiment must include a control with targeted-but-incorrect cues of matched specificity and length, or an analysis measuring the accuracy of self-generated cues against an independent ground truth (e.g., region overlap with the oracle or human-annotated error regions). Without such a control, the reported 79-90% recovery rates do not cleanly measure learned diagnostic skill; they may measure the ability to emit any concrete cue.","section":"Table 5 / §4.3.3"},{"comment":"The exploration-preference (EP) parameter is swept directly on the four test benchmarks, and the 'Ours' results in Table 1 and the recovery rates in Table 5 use the EP value (+0.2) selected on these same test sets. This is a form of test-set tuning that can inflate the reported gains. The manuscript should select EP on a held-out validation split, or at minimum report the full EP sweep in the main text and demonstrate that the qualitative conclusions (ReGround improves over baselines) hold across a range of EP values rather than only at the best test-set value. The current presentation in Table 9 does show gains over baseline for several EP settings, but the headline numbers are in-sample selections.","section":"Appendix C / Table 9"},{"comment":"The Step 2 grounding judge (Qwen2.5-VL-72B-Instruct) is used to construct the Grounding trajectories and also serves as the oracle cue generator in Table 5, yet the manuscript states (in §3.1.1) that its agreement with human annotations remains future work. This is acceptable for the main method-vs-baseline comparison, but it weakens the interpretation of the oracle as a diagnostic upper bound: the oracle's advantage over generic cues may stem from the 72B model's general capability or from answer-hint leakage rather than from superior diagnosis. To support the capability-bootstrapping claim, the authors should provide an independent evaluation of self-generated cue quality (e.g., human-annotated relevance of the cue's focus region) or compare the self cues' focus regions against the oracle's focus regions on a per-sample basis.","section":"§3.1.1 / Table 5"}],"minor_comments":[{"comment":"The statement that the recovery rate 'primarily reflects learned diagnostic reasoning rather than visual perception differences' is too strong, because the 7B and 72B models differ in language-model capacity and instruction following; please soften this claim or add a control cue from an equally capable but non-oracle source.","section":"Table 5 footnote"},{"comment":"No seed variance is reported for the main results or the ablations. Given the stochastic GRPO rollout (temperature 0.7) and the small EP-induced differences in some benchmarks, please report means and standard deviations across at least three seeds for the key tables.","section":"§4.2, Tables 1 and 5"},{"comment":"The attention entropy and SAS analyses use a single layer (Layer 20); please state whether the decay/recovery pattern is stable across layers and heads, or at least acknowledge this limitation in the main text.","section":"Table 11, §4.4"},{"comment":"The phrase 'a swing of several points on key benchmarks' is vague; please quantify it with the actual range (e.g., the −2.1 to +8.6 spread reported in Table 5).","section":"Abstract and §1"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely to be of interest to the multimodal reasoning community, and the core method and ablations are solid. However, the headline mechanistic claim about 'diagnostic quality' is underdetermined by the current cue-quality ablation, and the test-set EP sweep is a methodological concern that affects the interpretation of the reported gains. Both are fixable with additional experiments and re-analysis, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe useful core here is Table 5: holding trigger positions fixed and swapping only the content inside <reground> shows a clean three-tier pattern—generic 'look again' hurts, targeted cues help a lot, and self-learned cues recover most of the oracle gap. That is a new and well-designed isolation of cue quality from re-examination format, and it backs the paper's central claim that diagnostic quality, not re-injection itself, is the operative variable. The supporting ablations are consistent: text-only reflection is clearly inferior to visual re-injection, removing grounding trajectories hurts most, and No-ReGround negative examples keep the trigger rate from exploding. The attention entropy signature (98% decay, 94% recovery) is a useful mechanism-level observation, and the paper is appropriately careful to call it a proxy rather than a grounding metric. The capability bootstrapping story is believable because the 72B model is used only during data construction, not at inference, and the 79–90% recovery rates are plausible.\n\nThe soft spots are real but not disqualifying. First, the stress-test point about Table 5 is valid: the generic cue is untargeted, while the oracle cues come from a 72B model that could leak answer-relevant hints despite the no-numeric-answer prompt. So the generic-vs-targeted gap conflates target specificity with target accuracy and raw informativeness. A wrong-target cue of equal specificity and length would tighten the interpretation. The qualitative example in Figure 8 partially addresses this—it shows a targeted cue that points to specific attributes without giving away the answer—but it is one example. Second, the Step 2 grounding judge (72B) has no measured agreement with human annotations; the paper flags this, but it matters for reading the oracle as a diagnostic upper bound rather than a general-capability bound. Third, the exploration-preference sweep is run directly on test benchmarks, and no seed variance is reported for main results. These are common weaknesses in this line of work; the paper is transparent about the first two.\n\nOverall, this is a solid empirical paper with a genuine new ablation and a clear message. The central claim holds up better than the stress-test note suggests—the qualitative evidence genuinely supports the specificity story—but the quantitative comparison would be sharper with a wrong-target control and a validated judge.\n\nFor a colleague: worth reading if you work on VLM self-correction or multimodal reasoning. I would send it to a serious referee without hesitation; the code and data will help reproducibility.","headline":"Solid empirical paper with a genuinely new cue-quality ablation; the central claim is plausible and worth building on, though the oracle/generic comparison could be tightened.","tokens_in":23594,"tokens_out":2125,"would_cite":true,"duration_ms":21209,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that visual re-examination only improves multi-step VLM reasoning when paired with targeted self-diagnosis, and that diagnostic quality, not re-injection alone, is the key variable.","keywords":["vision-language models","visual grounding","multi-step reasoning","self-diagnosis","visual re-injection","attention entropy","self-correction","capability bootstrapping"],"falsifier":"A human-annotation study of the Step 2 grounding judge: if human raters disagree with the judge's pass/fail decisions on a few hundred correct first-round answers, the data-construction premise is in doubt. Alternatively, if a model from the same family with similar visual ability but weaker instruction-following generates cues that recover the same 79–90%, the recovery could be explained by general capability rather than diagnostic skill.","tokens_in":22544,"feed_emoji":"👁️","tokens_out":5368,"duration_ms":48192,"temperature":0.7,"pith_summary":"Vision-language models (VLMs) progressively lose attention to image evidence as they reason over longer chains, a pattern the paper calls visual grounding decay. The paper argues that simply re-injecting the image for a second look is not enough: without a targeted cue saying what to re-examine, re-checking can even override correct answers. It introduces ReGround, a two-stage training recipe that teaches the model to diagnose its own grounding failures and then revisit the image with a specific cue, without external tools. If the claim holds, the practical lesson is that self-correction in VLMs should be diagnosis-driven rather than format-driven, and that a stronger model's diagnostic skill can be distilled into a smaller one that keeps most of the benefit at inference time.","feed_headline":"Targeted self-diagnosis restores visual grounding; generic retries hurt","feed_subtitle":"A training recipe recovers 79–90% of oracle gains across eight benchmarks at 1.76x latency.","key_machinery":"The load-bearing device is a two-stage training pipeline: a re-examination trigger routes each sample to one of four trajectory modes—corrective, grounding, verification, or no re-grounding—and a visual self-diagnosis module writes a first-person <reground> cue that says what to look at without revealing the answer. In Stage 1, a stronger same-family model provides those cues only during data construction (capability bootstrapping), and the policy learns to produce them autonomously; in Stage 2, GRPO with an asymmetric reward calibrates when to trigger re-examination, rewarding successful recovery, efficient precision, and attempted correction, while penalizing blind failure.","core_discovery":"ReGround claims that effective visual self-correction in multi-step reasoning needs two ingredients: re-injecting the image after reasoning has started, and a targeted self-diagnosis cue that tells the model what to re-check. The central evidence is a cue-quality ablation in which a generic cue ('look again carefully') performs at or below no re-examination on several benchmarks, while oracle cues from a larger same-family model give large gains, and the policy model's self-generated cues recover 79–90% of those gains across four benchmarks. The paper further shows that a balanced triggering policy matters: removing negative examples causes the trigger rate to jump to about 85% and hurts accuracy, while over-triggering produces negative flips. Across 2,510 samples, attention entropy over image tokens narrows in 98.0% of cases and rebroadens after re-injection in 94.4%, which the authors treat as a mechanism signature rather than a direct grounding metric.","pith_inferences":["The paper does not test this, but the attention-entropy dip before re-injection could serve as a cheap online signal for deciding when to trigger re-examination, since the paper already shows entropy correlates with the benefit of re-visiting.","A human-agreement study of the grounding judge would settle whether the 'ungrounded' label is itself reliable; the paper states this agreement has not yet been measured.","For tasks with fine-grained or spatially distributed evidence, the paper's own results suggest combining ReGround with region-level crop or zoom tools could close the remaining gap on benchmarks like VisuLogic and V*Bench.","A natural next experiment is to feed the same diagnostic cue into several models from the same family that share a vision encoder; if gains persist across the whole family, diagnostic-skill transfer is broader than the current two-point comparison shows."],"forward_implications":["A model trained with ReGround can recover 79–90% of the accuracy gain that oracle diagnostic cues provide, without needing the larger model at inference time.","Generic re-examination can actively hurt: it drops accuracy on some benchmarks below the no-re-examination baseline, so any re-injection mechanism should be coupled with diagnostic guidance.","Both image re-injection and targeted diagnosis are necessary; text-only reflection with the same training recipe yields much smaller gains, and even text-only reflection plus RL trails visual re-injection without RL.","A calibrated trigger rate (about 45% on the studied setting) balances positive and negative flips; over-triggering degrades accuracy while conservative triggering forgoes recoverable gains.","The recipe transfers to a second backbone and improves accuracy on all eight benchmarks, with gains concentrated on visually intensive multi-step reasoning tasks and at 1.76x latency relative to baseline."],"supporting_citations":[{"why":"Documents that attention to image tokens decays rapidly after the first decoding steps, supplying the decay signature ReGround targets.","marker":"[5]"},{"why":"Formalizes declining image-text mutual information with generation length, providing a mechanism-level account of grounding decay.","marker":"[8]"},{"why":"Identifies visual attention-sink behavior in which textual context increasingly dominates attention, motivating image re-exposure.","marker":"[15]"},{"why":"Baseline implicit re-focusing method that ReGround contrasts with, since it re-attends only within the original text-dominated context.","marker":"[28]"},{"why":"A multi-stage visual re-examination baseline; ReGround isolates the cue-quality dimension this baseline does not vary.","marker":"[6]"},{"why":"Another visual reflection baseline that uses re-examination, providing context for the format-versus-quality question.","marker":"[13]"},{"why":"Supplies the GRPO reinforcement-learning algorithm used in ReGround's Stage 2 policy optimization.","marker":"[10]"},{"why":"Tool-augmented baseline that achieves strong perception at higher latency, serving as the main efficiency-accuracy comparison.","marker":"[32]"},{"why":"Tool-calling RL baseline whose reported tool-calling collapse motivates ReGround's tool-free design.","marker":"[34]"}],"fun_headline_variants":["Self-diagnosis fixes visual grounding; generic retries backfire","ReGround: targeted re-check restores VLM grounding","Why 'look again' fails: targeted self-diagnosis saves reasoning","ReGround: teaching VLMs to diagnose before re-examining"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach assumes the stronger model's diagnostic cues and the judge that flags 'correct but ungrounded' answers are genuinely identifying visual grounding failures—an assumption the paper states it has not yet checked against human annotations.","fun_headline_variants_meta":{"raw":{"variants":["Self-diagnosis fixes visual grounding; generic retries backfire","ReGround: targeted re-check restores VLM grounding","Why 'look again' fails: targeted self-diagnosis saves reasoning","ReGround: teaching VLMs to diagnose before re-examining"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000218,"raw_usage":{"total_tokens":1481,"prompt_tokens":1030,"completion_tokens":451,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":646,"completion_tokens_details":{"reasoning_tokens":386}},"tokens_in":646,"tokens_out":451,"duration_ms":4820,"temperature":1.0,"reasoning_tokens":386,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T18:31:14.729758+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A human-annotation study of the Step 2 grounding judge: if human raters disagree with the judge's pass/fail decisions on a few hundred correct first-round answers, the data-construction premise is in doubt. Alternatively, if a model from the same family with similar visual ability but weaker instruction-following generates cues that recover the same 79–90%, the recovery could be explained by general capability rather than diagnostic skill.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Formalizes declining image-text mutual information with generation length, providing a mechanism-level account of grounding decay."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Identifies visual attention-sink behavior in which textual context increasingly dominates attention, motivating image re-exposure."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Tool-augmented baseline that achieves strong perception at higher latency, serving as the main efficiency-accuracy comparison."},{"cited_title":"Thinking with Im- ages","cited_arxiv_id":null,"evidence_quote":"Tool-calling RL baseline whose reported tool-calling collapse motivates ReGround's tool-free design."}],"review_version":1}