{"id":"d84793ea-bb32-4b76-a4d0-5a2c6321668e","arxiv_id":"2505.24025","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"DINO-R1 trains visual-prompt detectors with group-relative query rewards and KL regularization, improving zero-shot and fine-tuned detection over supervised fine-tuning.","lead":"This paper introduces DINO-R1, a training method that uses a reinforcement-style reward to teach an object detection model to find objects matching a visual example rather than a text label. It reports consistent accuracy gains over standard fine-tuning on COCO, LVIS, and real-world detection benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The group-relative reward in Eq. 12 is identically zero when α is uniform because Eq. 9 standardizes over all queries; the method actually depends on an unspecified objectness mask.","rationale":"The reader's weakest assumption was that Eq. (11) is a valid KL-divergence regularization. I agree that Eq. (11) is not the categorical KL divergence of a normalized objectness distribution, and that the ablation labeled 'only KL-Div' should be interpreted with caution. However, that issue is secondary to the reward-term degeneracy. Eq. (9) guarantees Σ_i Â_i = 0, so the reward component of Eq. (12) vanishes whenever α is truly scalar. The paper's only escape is the sentence that sets α to zero for low-objectness queries, but the mask is not described precisely, and the ablations do not separate the mask from the group-relative reward. This directly threatens the central claim that GRQO's query-level relative reward drives the reported generalization gains. The paper does have real strengths: consistent multi-dataset experiments, direct SFT comparison, ablations of reward components and loss weights, and an honest limitations section. Those deserve credit. But without code, a clear specification of the mask, and an ablation that controls for the mask, the reward mechanism cannot be verified. A conditional verdict is appropriate: the empirical numbers are plausible, but the central method needs clarification and code before the claim can be accepted as stated.","tokens_in":12719,"tokens_out":14611,"duration_ms":155849,"concrete_test":"Release code/checkpoints and, on a real O365 batch, numerically verify that Σ_i Â_i = 0 with uniform α. Then train three variants: (a) GRQO with uniform α and no mask, (b) GRQO with the objectness mask as described, and (c) GRQO with a random mask of the same size. Report COCO zero-shot AP for each. If (a) has zero reward gradient and (c) matches (b), the improvement attributed to group-relative reward is actually due to the mask; if (b) clearly beats (c), the mask is not the whole story.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central mechanism is degenerate as written. In Eq. (9), the advantage Â_i = (r_i − μ_r)/σ_r is standardized over all N_q queries, so Σ_i Â_i ≡ 0 for any reward vector. Substituting into Eq. (12), the reward term with the stated scalar weight α is −α/N_q Σ_i Â_i = 0, so it has zero gradient. The only sentence that prevents this is 'we set α to zero for the final set of queries exhibiting low objectness probability,' which makes α per-query and turns the loss into a masked sum of standardized rewards. No threshold, fraction, or source for the mask is specified. Consequently the ablations in Tables 2 and 3 labeled 'reward' do not isolate a group-relative reward; they measure reward plus an objectness-based mask. The reported gains could be driven by the mask (hard-example selection) rather than by GRPO-style relative advantages. This is more load-bearing than the KL-formula issue in Eq. (11): even if Eq. (11) is read as a Bregman penalty rather than a true KL, the reward term is the primary contribution, and its behavior is not what the paper claims.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DINO-R1, a visual-prompt object detector built by adapting Grounding-DINO with a visual prompt encoder and visual-guided query selection (VIS-G-DINO), and a training scheme called GRQO. GRQO defines a query-level reward from the matching cost between each decoder query and ground truth, normalizes rewards within a group, and adds a KL-type penalty on objectness scores to stabilize training. The model is trained on Objects365 and evaluated zero-shot on COCO, LVIS-minival, ODinW13/35, and after fine-tuning on COCO. Across these settings, GRQO-trained models consistently outperform SFT-trained VIS-G-DINO baselines, with the largest gains on ODinW13 (+8.8 AP for the L variant).","tokens_in":13043,"tokens_out":6096,"duration_ms":59512,"significance":"If the empirical results hold, GRQO is a useful training recipe for visual-prompting detectors: the ablations in Tables 2 and 3 show consistent gains from both the reward and the penalty, and the relative form of the reward outperforms the absolute form. The held-out benchmarks are independent of the training objective, so the generalization claim is meaningful, and the layer-wise reward variant gives a clear additional gain. However, the paper does not demonstrate a new 'reasoning capability'; it improves supervised fine-tuning for a specific visual-prompting architecture. The main conceptual contribution is an auxiliary training loss that reuses the detector's own matching costs, so the RL framing should be tempered. No code or checkpoints are mentioned, which limits reproducibility, but the empirical pattern is internally consistent across model sizes and datasets.","major_comments":[{"comment":"As written, the reward term in Eq. (12) is identically zero for any reward vector when α is a scalar: Eq. (9) standardizes Â_i over all N_q queries, so Σ_i Â_i = 0 and the term −(α/N_q) Σ_i Â_i has no gradient. The sentence 'we set α to zero for the final set of queries exhibiting low objectness probability' implicitly makes α per-query, but the threshold, fraction, and how 'final set' is defined are not given, and Eq. (12) still writes α as a scalar. This must be fixed: define the mask explicitly, specify how it is computed, and report its sensitivity, because otherwise the 'only reward' row of Table 2 measures reward plus an objectness-based hard-example selection rather than a pure group-relative reward.","section":"Sec. 3.3, Eqs. (9)-(12)"},{"comment":"Equation (11) is not the KL divergence. The objectness scores in Eq. (10) are max-over-prompt similarities of unnormalized dot products; they are not a probability distribution, and there is no normalization or positivity guarantee, so O_ref/O_θ may be negative or undefined, and O_θ log(O_θ/O_ref) is not the quantity shown. The right-hand side O_ref/O_θ − log(O_ref/O_θ) − 1 is a Bregman penalty on the ratio, not D_KL[O_θ ∥ O_ref]. Please define the exact regularizer used, including whether a softmax or temperature is applied, how the reference indices are computed, and whether reference features are detached, and then re-state what the 'only KL-Div' ablation in Table 2 actually evaluates.","section":"Sec. 3.3, Eqs. (10)-(11)"},{"comment":"The reward reuses the same focal, L1, and GIoU costs as the supervised detection loss, so the 'reward' is partly a reweighted auxiliary detection loss rather than an independent verifiable reward. This does not invalidate the held-out improvements, but it weakens the RL interpretation and the claim of incentivizing a distinct reasoning capability. I ask the authors to state this explicitly and to justify why group-relative normalization plus masking, rather than simply extra gradient pressure on unmatched queries, is the source of the observed gains.","section":"Sec. 3.3, Eq. (7) vs. Eq. (1)"},{"comment":"The paper reports no error bars, multiple seeds, or statistical significance tests. Because RL-style training is typically high-variance and some reported differences are small (e.g., 22.7 vs. 23.5 in Table 3), the consistency claim needs at least a few seeds or a variance estimate. This is especially load-bearing for the fine-tune numbers, where the GRQO-pretrained model is compared against continued SFT.","section":"Sec. 4, Tables 1-5"}],"minor_comments":[{"comment":"The text says the reward weight is swept over 1.0, 10.0, 10e2, 10e3, and 10e4, but the table omits the 10e2 row; please align the text and table.","section":"Sec. 4.3, Table 4"},{"comment":"The sentence 'The remaining stages mirror the G-DINO pipeline' leaves unspecified how final class predictions are obtained from visual prompts; please clarify whether text embeddings are replaced by visual prompt embeddings in the final contrastive classifier.","section":"Sec. 3.2, Eq. (5)"},{"comment":"The reference objectness uses the same top-N_q indices Idx^v_Nq as the current model; please clarify whether these indices come from the current or reference model and whether gradients are blocked through the reference features.","section":"Sec. 3.3, Eq. (10)"},{"comment":"The term 'visual in-context reasoning capability' is used as an invented entity without a formal definition or metric; please replace it with an operational description such as 'zero-shot visual-prompt detection accuracy' or define 'reasoning' precisely.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The empirical contribution is potentially publishable, but the two load-bearing equations need correction and the objectness mask must be specified. I do not see a circularity problem with the held-out evaluation, but the 'reasoning' framing should be tempered, and a comparison to existing visual-prompting methods such as CP-DETR or T-Rex2 would help support the 'first' claim. Please require the authors to clarify the exact implemented loss."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a real attempt to carry GRPO-style training from LLMs to DETR-style detectors, and the empirical pattern is consistent across COCO, LVIS, and ODinW. The idea of computing group-relative rewards over queries rather than sampled generations is genuinely new, and the visual prompt encoder and query selection are reasonable engineering. The gains over SFT are stable and the ablations are thoughtfully laid out.\n\nThe soft spots are concentrated in the method section. The stress-test note is right: Eq. 9 standardizes advantages over all N_q queries, so with the scalar alpha in Eq. 12 the reward term sums to zero identically. The only thing that saves it is the sentence about setting alpha to zero for low-objectness queries, which makes alpha a per-query mask—but no threshold, fraction, or source for that mask is specified. As written, the central mechanism is degenerate; the ablations labeled 'reward' are really measuring reward plus an unspecified hard-example selection. That is more load-bearing than the KL-formula issue in Eq. 11, which is also nonstandard (O_ref/O_theta - log(...) - 1 is not a categorical KL, and O is not normalized). Both need to be fixed and the implemented forms made explicit.\n\nThe rest is what you'd expect: no code, no error bars, no comparison to existing visual-prompt detectors like T-Rex or CP-DETR (the fine-tuning baselines are at least consistent). The 'first such attempt' framing is a bit strong, but the adaptation from probabilistic generation to deterministic query outputs is not a mechanical transfer, so the novelty claim mostly holds.\n\nIf the authors clarify the mask and the KL term and release code, this could be a useful training recipe. As written, the method section doesn't define the method it trains. That is a fixable presentation problem, not a fatal empirical one—the held-out gains are real evidence the pipeline works.\n\nI would send it to review: the idea deserves referee time, and the empirical claim is worth checking. But I would expect major revision on the method specification before it is actionable.","headline":"A promising training recipe for visual-prompt detection, but the central reward term as written is degenerate and the load-bearing mask is left unspecified.","tokens_in":13524,"tokens_out":2320,"would_cite":false,"duration_ms":22339,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that Group Relative Query Optimization (GRQO), a reinforcement-style training objective, makes visual-prompt object detectors generalize better than supervised fine-tuning.","keywords":["visual prompting","object detection","reinforcement learning","group relative query optimization","DETR","open-vocabulary detection","KL regularization","query-level reward"],"falsifier":"Compute Equation 11 for two objectness score vectors and compare the result with the true KL divergence $\\sum_i O_{\\mathrm{ref},i} \\log(O_{\\mathrm{ref},i}/O_{\\theta,i})$; also check whether the objectness vectors sum to 1. If the values differ or the vectors do not sum to 1, the term is not a KL divergence. A second check is to replace Equation 11 with a true KL after softmax normalization and rerun the 'only KL-Div' ablation: a substantially different outcome would show that the reported effect depends on the nonstandard formula.","tokens_in":12552,"feed_emoji":"🎯","tokens_out":4840,"duration_ms":45237,"temperature":0.7,"pith_summary":"The paper sets out to bring R1-style reinforcement training to vision foundation models, focusing on visual-prompting detection, where users specify targets by example instead of text. It argues that supervised fine-tuning gives visual-prompt detectors sparse and unstable supervision, and that its Group Relative Query Optimization (GRQO) provides dense, group-normalized query rewards plus objectness regularization. The payoff, if correct, is a training recipe: a detector trained on Objects365 beats supervised fine-tuning baselines on COCO, LVIS, and ODinW, with the largest variant gaining +8.8 and +4.4 mAP on ODinW13 and ODinW35 and +4.9 mAP on fine-tuned COCO. The authors frame this as incentivizing visual reasoning, but the concrete contribution is a practical optimization strategy for prompt-based detection.","feed_headline":"RL-style query rewards beat supervised fine-tuning for visual prompts","feed_subtitle":"DINO-R1's group-relative rewards lift zero-shot mAP on COCO and ODinW, and in-domain COCO mAP by up to 4.9 points.","key_machinery":"The central mechanism is Group Relative Query Optimization (GRQO), a training objective designed for query-based detectors. For each decoder query, the paper computes a matching cost against ground-truth instances, converts the minimum cost into a query reward, and normalizes rewards within the image so that queries are pushed to beat the dynamic group average rather than an absolute threshold. A KL-style regularization term, Equation 11, anchors the current objectness distribution to a frozen earlier copy of the model. The composite loss combines this group-relative reward, the KL term, contrastive prompt alignment, and the standard focal, L1, and GIoU losses.","core_discovery":"The central claim is that GRQO improves both out-of-domain and in-domain generalization for visual-prompt detection compared with standard supervised fine-tuning. The reward is computed per query as the negative of its minimum matching cost to any ground-truth box, normalized across all queries in the image as a relative advantage $(r_i - \\mu_r)/\\sigma_r$, then used as an auxiliary loss alongside regular detection losses. A KL-style regularization term on the objectness distribution, computed against a frozen reference model, is added to stabilize training. On COCO zero-shot transfer, DINO-R1-T improves over its SFT counterpart by +4.1 mAP; on LVIS rare categories, DINO-R1-B improves by +3.4 APr; and DINO-R1-L exceeds SFT by +8.8 and +4.4 mAP on ODinW13 and ODinW35. In the in-domain COCO fine-tuning setting, GRQO surpasses continued SFT by +4.3 mAP and, when the GRQO-pretrained model is the starting point, by +4.9 mAP.","pith_inferences":["Editorial extension: A natural test outside the paper is whether GRQO also improves text-prompt grounding detectors or other dense prediction heads, since the group-relative reward does not depend on the prompt modality.","Editorial extension: If the objectness scores are not a normalized distribution, the regularization term in Equation 11 is not a true KL divergence; replacing it with an L2 penalty or a true KL after softmax would isolate what the stabilization actually does.","Editorial extension: The authors state the visual prompt encoder was kept deliberately simple; a stronger prompt encoder could further raise results, so architecture gains and optimization gains are entangled in the final system."],"forward_implications":["GRQO can be applied to any DETR-style detector with query-based predictions, not only the visual-prompting variant tested here.","Training with one randomly sampled visual prompt per class outperforms training with many prompts, while inference benefits from using more prompts per class.","Group-relative rewards consistently beat absolute rewards in the ablations, suggesting that inter-query normalization is the key ingredient.","Layer-wise reward propagation improves results, implying that earlier decoder refinement stages also benefit from reinforcement-style supervision.","The best configuration uses a reward weight near 10e3 and a KL weight near 0.04, indicating that the method is sensitive to loss scaling."],"supporting_citations":[{"why":"Source of the group-relative advantage idea that GRQO adapts from language-model RL training.","marker":"[1]"},{"why":"Grounding DINO, the base detector architecture that the visual-prompting model extends.","marker":"[14]"},{"why":"MM-Grounding-DINO, the implementation and strong baseline used throughout the experiments.","marker":"[38]"},{"why":"Visual in-context prompting, the paradigm that motivates the visual-exemplar detection setting.","marker":"[16]"},{"why":"Objects365, the training dataset for the zero-shot transfer experiments.","marker":"[20]"},{"why":"COCO, the dataset for both zero-shot transfer and in-domain fine-tuning evaluation.","marker":"[19]"},{"why":"LVIS, the long-tailed dataset used to evaluate rare-category generalization.","marker":"[58]"},{"why":"Source of the ODinW benchmark used for out-of-domain evaluation in the wild.","marker":"[45]"}],"fun_headline_variants":["Visual prompt detection: RL beats SFT by up to 8.8 mAP","DINO-R1: First RL framework for visual reasoning in foundation models","GRQO: group-relative query rewards lift visual prompting beyond SFT","DINO-R1: RL boosts visual prompt detection, beating SFT"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that the objectness scores in Equation 11 form a probability distribution and that the formula shown is a genuine KL divergence, but objectness is defined as a max-over-prompts similarity that is not normalized, and Equation 11 is not the standard categorical KL formula.","fun_headline_variants_meta":{"raw":{"variants":["Visual prompt detection: RL beats SFT by up to 8.8 mAP","DINO-R1: First RL framework for visual reasoning in foundation models","GRQO: group-relative query rewards lift visual prompting beyond SFT","DINO-R1: RL boosts visual prompt detection, beating SFT"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001682,"raw_usage":{"total_tokens":6711,"prompt_tokens":1030,"completion_tokens":5681,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":646,"completion_tokens_details":{"reasoning_tokens":5598}},"tokens_in":646,"tokens_out":5681,"duration_ms":36218,"temperature":1.0,"reasoning_tokens":5598,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:37:34.681688+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute Equation 11 for two objectness score vectors and compare the result with the true KL divergence $\\sum_i O_{\\mathrm{ref},i} \\log(O_{\\mathrm{ref},i}/O_{\\theta,i})$; also check whether the objectness vectors sum to 1. If the values differ or the vectors do not sum to 1, the term is not a KL divergence. A second check is to replace Equation 11 with a true KL after softmax normalization and rerun the 'only KL-Div' ablation: a substantially different outcome would show that the reported effect depends on the nonstandard formula.","supporting_citations":[{"cited_title":"Visual in-context prompting","cited_arxiv_id":null,"evidence_quote":"Visual in-context prompting, the paradigm that motivates the visual-exemplar detection setting."},{"cited_title":"Lvis: A dataset for large vocabulary instance segmentation","cited_arxiv_id":null,"evidence_quote":"LVIS, the long-tailed dataset used to evaluate rare-category generalization."}],"review_version":1}