{"id":"5f30d31f-0b38-406c-8fc0-6dc971c8d0cd","arxiv_id":"2602.23615","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"HART uses a closed-loop 'crop-and-answer' training scheme plus a dynamic-weight GRPO variant to improve LMM grounding and high-resolution reasoning without bounding-box annotations.","lead":"This paper trains vision-language models to first find and crop the important part of a high-resolution image, then answer the question using only that crop, with a modified reinforcement-learning reward to make the model better at choosing the crop. HART reports large accuracy gains on several high-resolution visual reasoning benchmarks without needing human bounding-box annotations.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Prop. 2 (§4.3) is not derivable from Eq. (4)–(6): L is never observed, and the actual gradient difference from vanilla GRPO is Σ kδA∇logπ + βkδ∇KL, not a term removing P(L=0,r=1). The 'direct grounding optimization' guarantee is unsupported.","rationale":"Reader's weakest assumption and my concern coincide. The central claim — annotation-free direct grounding optimization — depends on the correctness of Eq. 7. The algebra shows the AP-GRPO objective actually upweights advantageous answer-reward differences and scales KL, not the unobserved L=0,r=1 term. This is not a disagreement with consensus; it is an internal derivability problem. The empirical results are plausible and the pilot study is informative, but the theoretical guarantee as written is not supported. A conditional acceptance is appropriate: the empirical claim can be checked and the proposition can be corrected or removed. I recommend no change to the reader's verdict, with the condition that the authors resolve or retract Prop. 2 and release code/data for the oracle check.","tokens_in":17771,"tokens_out":7174,"duration_ms":68236,"concrete_test":"Symbolically (or numerically with a toy logistic policy, G=4, one r=1/L=0 sample) compute ∇J_AP-GRPO from Eq. (4)–(6) at θ = θ_old and subtract ∇J_GRPO. If the difference is not of the form −αP(L=0,r=1)E[∇logπ] (specifically it will be Σ[kδA∇logπ + βkδ∇KL]), Prop. 2 fails. Then, to test whether the residual proxy error matters, train AP-GRPO on Visual CoT with access to ground-truth L and explicitly downweight L=0,r=1 samples; if this oracle variant outperforms AP-GRPO, the misspecification is material.","verdict_should_be":"UNCHANGED","load_bearing_attack":"AP-GRPO's claimed theoretical guarantee (Prop. 2, Eq. 7) is the only formal support for the paper's central 'directly optimizes grounding' claim. The proposition asserts the update subtracts exactly the misspecified samples with L=0,r=1. This cannot follow from the defined objective. In §4.3, r_i is answer correctness only; L is never observed. Writing δ_i = r_i − mean({r_i}) and A_i = δ_i/std, the gradient of Eq. (4) at θ = θ_old differs from vanilla GRPO by (1/G)Σ_i[k δ_i A_i ∇_θ log π_θ + β k δ_i ∇_θ KL]. Since δ_i A_i = δ_i²/std ≥ 0, every sample contributes a non-negative policy-gradient increment; there is no αP(L=0,r=1) term and no sign that would remove wrong-crop/correct-answer samples. The residual 21.5% of L=0,r=1 from the authors' own Table 1 means the crops-only reward is still a noisy proxy. What survives is an empirical claim: a crops-only curriculum plus SFT improves grounding and accuracy (Tables 2–6). That may be true, but it is not 'direct optimization of grounding', and Prop. 1's mutual-information framing does not repair Eq. 7. The paper should either re-derive a valid gradient identity or reframe the contribution as an empirical curriculum effect.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HART, a post-training framework for high-resolution LMMs that operates in a closed loop: the model first predicts region-of-interest (ROI) boxes from a downsampled image, then answers the question using only the corresponding high-resolution crops (the original image is withheld), and finally undergoes SFT on full-image data. To train the localization behavior without bounding-box annotations, the authors introduce AP-GRPO, a modified GRPO objective that reweights samples by their answer-correctness advantage (Eqs. 4–6). The paper claims that this directly optimizes visual grounding, supported by Proposition 2 (Eq. 7), and reports large improvements on MME-RealWorld-Lite, TreeBench, V* Bench, HR-Bench-4K/8K, and MMStar, along with grounding-accuracy evaluations on TreeBench and Visual CoT.","tokens_in":18199,"tokens_out":5721,"duration_ms":55166,"significance":"If the empirical results hold, HART is a practically useful annotation-free training recipe for high-resolution visual reasoning, with the valuable feature of evaluating grounding accuracy directly on two labeled benchmarks and reporting IoU curves. The paper also includes ablations over its single hyperparameter k and demonstrates transfer to a second base model (InternVL3-8B). However, the central novelty claimed in the title and abstract is that AP-GRPO 'directly optimizes visual grounding'; this rests on Proposition 2, which is not derivable from the defined objective. Without that theoretical support, the contribution reduces to an empirically motivated curriculum/self-verification effect, and the absence of error bars or significance tests makes the magnitude of the reported gains uncertain. The empirical evaluation is broad and the paper is clearly written, but the central claim needs either a valid derivation or a serious reframing.","major_comments":[{"comment":"Proposition 2 is not a consequence of Eqs. (4)–(6). The AP-GRPO objective depends only on r_i, the generated text o_i, and the group statistics; the localization-correctness variable L is never observed. Taking the gradient at θ=θ_old gives (1/G)Σ_i[(1+kδ_i)A_i∇logπθ(o_i|q) − β(1−kδ_i)∇D_KL(πθ||πref)], with δ_i = r_i − mean({r_i}) and A_i = δ_i/std. The term (1+kδ_i)A_i equals δ_i/std + kδ_i²/std; there is no mechanism that subtracts a contribution proportional to P(L=0,r=1), nor any sign selection that removes those samples. Thus Eq. (7) does not follow from the algorithm as stated. The authors should either derive a valid gradient identity that justifies 'direct grounding optimization' or explicitly reframe the contribution as an empirical curriculum/self-verification effect rather than a provable direct optimization.","section":"§4.3, Eq. (7)"},{"comment":"The paper's own pilot data show that the crops-only reward remains misspecified. For Qwen2.5-VL-7B, even under the HART prompt change, 502/2332 = 21.5% of correct answers still have incorrect grounding; for InternVL3-8B the proportion is 55.9%. This is a large residual fraction, so the assertion in §4.3 that 'rewarding correct answers also encourages faithful grounding' is an empirical assumption, not a guarantee. Proposition 1's mutual-information inequality, even if true, is a correlational statement and does not establish that AP-GRPO directly optimizes L. The paper should acknowledge this limitation explicitly and temper the 'directly optimizes grounding' language throughout, including the abstract.","section":"Table 1 and §4.2"},{"comment":"All experimental results are single-run point estimates without standard deviations, confidence intervals, or significance tests. This is particularly concerning for TreeBench (Table 3), where several subcategories have very small sample sizes (Material n=13, Phy. State n=23, Comparison n=44) and the overall 43.7% versus baselines could be sensitive to a few questions. Given the large claims (+20.1% on MME-RealWorld, +6.7% on TreeBench), the authors should provide multiple seeds (at least 3) or bootstrap confidence intervals for the main comparisons, and ideally for grounding accuracy in Table 6.","section":"§5.1, Tables 2–6"}],"minor_comments":[{"comment":"The statement that 'k is the only hyperparameter in HART' is an overstatement. Equations (4)–(6) also contain β, group size G, and the training pipeline involves the D_RL/D_SFT split, the number of ROIs, and the IoU/cropping thresholds. Please clarify which hyperparameters were fixed identically across baselines and which were tuned.","section":"§4.3"},{"comment":"The small per-category sample sizes make some comparisons fragile. Consider reporting aggregate results with confidence intervals or pooling related categories where possible.","section":"Table 3"},{"comment":"The caption does not specify the metric definition used on the y-axis (presumably Intersection over Ground-Truth coverage rate) or the evaluation protocol. Please state it explicitly and clarify whether the threshold is applied to the area coverage of the ground-truth box.","section":"Figure 3"},{"comment":"The proof of Proposition 1 is deferred to Appendix A but the appendix is not included in the submitted text. Please ensure the proof is self-contained and also state the exact joint distribution used to compute mutual information.","section":"§4.2"},{"comment":"The paper would benefit from a dedicated limitations section discussing the residual reward misspecification, the reliance on answer correctness as a proxy, and the lack of evaluation on diverse base models beyond Qwen2.5-VL-7B and InternVL3-8B.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The empirical study is broad and the grounding evaluation on TreeBench and Visual CoT is a strength. However, the central theoretical claim (Prop. 2) is unsupported by the algorithmic definitions, and the paper's framing as 'direct optimization of grounding' overreaches given its own residual-misspecification data. This is fixable by re-deriving or removing the theoretical guarantee and reframing the contribution as an empirical curriculum/self-verification method, but it is load-bearing for the paper's claimed novelty. I recommend major revision, not rejection, because the empirical results may still be valid and useful once the claims are calibrated and error bars are added."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this paper has a genuinely neat idea — run RL on answers from crops alone, then SFT with the full image — and the reported numbers are striking: +20.1 on MME-RealWorld-Lite, +6.7 on TreeBench, +10.9 on HR-Bench-8K over the base model. The pilot study in Table 1 is well done: it quantifies reward misspecification, shows the prompt change reduces correct-answer-with-wrong-grounding from 36.5% to 21.5%, and the later grounding evaluation on TreeBench and Visual CoT gives real evidence that localization improves. That part deserves credit.\n\nThe soft spots are real, though. The stress-test note is on target: Proposition 2 is not derivable from Eqs. 4–6. AP-GRPO weights samples by r_i − mean(r) only; the latent L is never observed. The actual gradient difference from GRPO is a sum of positive terms like δ_i²·∇logπ plus a KL adjustment, not a subtraction of (L=0, r=1) samples. So the claim that AP-GRPO 'directly optimizes grounding' and removes misspecified samples is unsupported. The residual 21.5% in their own table directly contradicts a clean version of that claim. This is not a minor typo — it is the paper's central theoretical selling point. Either they can derive a valid gradient identity (perhaps under a different formalization) or they should reframe the contribution as an empirical curriculum effect.\n\nSecond, the contribution of AP-GRPO itself is not isolated. The ablations compare HART (closed-loop + AP-GRPO + SFT) to GRPO/MGPO without the closed-loop. To attribute the gain to the preference weighting, they need a GRPO-in-the-same-closed-loop baseline. As it stands, the crops-only setup could be doing most of the work.\n\nThird, no error bars or multiple seeds for large single-run gains. Common in this literature, but it matters when claims are this strong.\n\nMinor: there is a corrupted-glyph artifact after Table 5 in the text, which looks like a production mistake. Not scientific, but sloppy.\n\nVerdict: the empirical claim is plausible and the method is worth engaging with. I would send this to peer review, but with a request for major revision — fix or drop Proposition 2, add error bars, and run the isolated ablation. If those come back clean, it is a solid contribution.\n\nBest,\n\n[Your name]","headline":"HART's empirical recipe — crops-only RL + SFT — looks promising and the reported gains are large, but Proposition 2 does not prove what it claims: AP-GRPO never observes the localization variable, so the 'direct grounding optimization' guarantee is unsupported; what remains is a solid empirical paper that needs reframing, error bars, and an isolated ablation.","tokens_in":18688,"tokens_out":4659,"would_cite":true,"duration_ms":46491,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HART proposes a closed-loop framework where a vision-language model must answer from its own crops, making correct answers a direct signal for correct localization, and uses AP-GRPO to train grounding without annotations.","keywords":["visual grounding","high-resolution LMM","reinforcement learning","GRPO","annotation-free","closed-loop reasoning","region-of-interest cropping","visual reasoning"],"falsifier":"On a held-out set with ground-truth boxes, measure the jointly correct-answer/incorrect-localization rate (L=0, r=1) after AP-GRPO training; if it does not drop materially below the 21.5% pilot figure, the reward remains misspecified and Proposition 2's claimed removal of these samples is not occurring. A complementary check: construct a dataset where the correct answer is obtainable from a deliberately wrong crop (e.g., a text question legible in the wrong region); if HART still receives maximum reward on those items, the method cannot distinguish faithful from unfaithful grounding.","tokens_in":17704,"feed_emoji":"🔍","tokens_out":6150,"duration_ms":49965,"temperature":0.7,"pith_summary":"The paper tries to show that a vision-language model can be taught to find and zoom into the key region of a high-resolution image without human grounding labels. The idea is a feedback loop: have the model propose a crop, hide the original image, and ask it to answer from the crop alone; then a correct answer becomes evidence that the crop was right, so the usual final-answer reward now also measures localization. On top of this, AP-GRPO reweights training samples so that correct, high-advantage responses dominate the gradient update. The authors report gains of +20.1 points on MME-RealWorld, +6.7 on TreeBench, +2.1 on V* Bench, and +10.9 on HR-Bench-8K over strong baselines, and substantially better grounding accuracy on datasets with ground-truth boxes. If true, this would offer a low-cost way to improve high-resolution perception in multimodal models without expensive annotation.","feed_headline":"Answering from crops alone teaches LMMs where to look","feed_subtitle":"A closed-loop RL method, HART, sharpens high-res visual reasoning without human grounding labels.","key_machinery":"Closed-loop visual feedback: the model first outputs bounding-box coordinates for the region(s) of interest from the downsampled image; those regions are cropped from the original high-resolution image, the original is deliberately hidden, and the model must answer from the crops alone. This converts localization quality into a question-answer signal. AP-GRPO then applies group-relative advantages with two dynamic scalar weights (µ1 scales the policy-ratio term, µ2 scales the KL penalty) so that samples with correct, high-advantage answers receive larger updates and a smaller KL penalty, while incorrect samples are down-weighted. The scaling factor k is the only hyperparameter; k=0.6 in the","core_discovery":"Reward misspecification—a positive reward for a correct answer with wrong localization—can be largely removed, the paper argues, by a closed-loop procedure (HART): the model predicts regions of interest, the original image is withheld, and the model must answer from the crops alone, making answer correctness a faithful proxy for grounding quality (pilot data: correct-answer-wrong-localization drops from 36.5% to 21.5% for Qwen2.5-VL-7B). The proposed AP-GRPO up-weights correct, advantageous responses and lowers their KL penalty, directly optimizing localization without annotations. Post-trained on Qwen2.5-VL-7B, HART beats baselines on high-resolution benchmarks, with grounding accuracy risi","pith_inferences":["The residual 21.5% of correct-answer-wrong-localization cases in the pilot data suggests the reward is still partially misspecified; a natural extension is to penalize cases where the crop-only answer is correct but a separate faithfulness check (e.g., answer confidence or answer change when the crop is perturbed) flags the localization as unreliable.","The closed-loop trick is a form of self-supervised grounding: the image is its own label. This could extend beyond cropping to other 'test-time information hiding' schemes, such as masking text regions or occluding objects, to train perception without annotations.","Because AP-GRPO's weighting is a simple function of the group-relative advantage, an ablation is available to readers: replacing µ1 and µ2 with constants should recover vanilla GRPO; the reported difference quantifies the contribution of the dynamic weighting alone versus the closed-loop itself."],"forward_implications":["If HART works as described, any LMM with a bounding-box or region-prediction output head can be post-trained to localize key regions using only final-answer supervision, removing the need for human grounding labels.","Because the reward signal now tracks localization, reinforcement fine-tuning on answer correctness should not merely improve the answer but also the model's pointing/cropping behavior, which the grounding measurements on TreeBench and Visual CoT support.","The two-stage pipeline (AP-GRPO for localization, then SFT with the full image visible) suggests a general recipe: train the model to find, then train it to reason from the found region; performance on MME-RealWorld-Lite and TreeBench depends on both stages.","Improved localization transfers out of distribution, at least from MME-RealWorld training to TreeBench, and the method also transfers to a second base model (InternVL3-8B), indicating the framework is not tied to one architecture."],"fun_headline_variants":["Closed-loop RL teaches LMMs to self-verify high-res detail","No labels: HART sharpens high-res reasoning via crops","Crop-only answers fix LMM grounding without annotations","HART's closed loop halves wrong-location rewards"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The entire method rests on the proxy that a correct answer from the crops alone means the crop contained the necessary visual information; if the model can answer correctly from an incorrect crop, the reward is still misspecified and the proposed gradient correction (Proposition 2) is not directly observable because the training algorithm never sees the true localization label L.","fun_headline_variants_meta":{"raw":{"variants":["Closed-loop RL teaches LMMs to self-verify high-res detail","No labels: HART sharpens high-res reasoning via crops","Crop-only answers fix LMM grounding without annotations","HART's closed loop halves wrong-location rewards"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000169,"raw_usage":{"total_tokens":1114,"prompt_tokens":768,"completion_tokens":346,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":277}},"tokens_in":512,"tokens_out":346,"duration_ms":3827,"temperature":1.0,"reasoning_tokens":277,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T20:13:39.723565+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a held-out set with ground-truth boxes, measure the jointly correct-answer/incorrect-localization rate (L=0, r=1) after AP-GRPO training; if it does not drop materially below the 21.5% pilot figure, the reward remains misspecified and Proposition 2's claimed removal of these samples is not occurring. A complementary check: construct a dataset where the correct answer is obtainable from a deliberately wrong crop (e.g., a text question legible in the wrong region); if HART still receives maximum reward on those items, the method cannot distinguish faithful from unfaithful grounding.","supporting_citations":[],"review_version":1}