{"id":"148a9882-6867-44c4-861e-036ad5913b36","arxiv_id":"2607.07383","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":5,"one_line_summary":"An agentic mRAG framework uses GRPO-trained visual reranking and active rejection to verify retrieved candidate entities, achieving state-of-the-art on three KB-VQA benchmarks.","lead":"MMAgent-R2 is an agentic multimodal RAG framework that adds visual reranking and active rejection to verify retrieved entities before answering visual questions. It matters because retrieval errors are a primary failure mode in knowledge-based VQA, and this mechanism reduces error propagation to the final answer.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"Training data filtered to reachable-entity cases (Supp. D.1) leaves model behavior on unreachable test samples untrained and unanalyzed; on E-VQA this could affect ~62% of test samples (R@15=37.8%).","rationale":"The reader correctly identified the most load-bearing concern: the train-test distribution gap created by the filtering in Supp. D.1. The concern is well-founded because (1) the fraction of affected test samples is substantial, especially on E-VQA where the largest gains are reported, and (2) the paper does not analyze model behavior on unreachable cases. However, I recommend UNCHANGED rather than a harsher verdict because the ablation studies (Tables 5-8) are conducted on unfiltered test data and still demonstrate component-level improvements, which provides partial evidence that the method generalizes beyond the training distribution. The concern is about an unanalyzed gap, not demonstrated incorrectness. The CONDITIONAL verdict with HIGH confidence is appropriate — the paper should address this gap (e.g., by reporting stratified results) but the core method and results remain credible. No code/data release further limits independent verification, consistent with the reader's assessment. I agree with the reader's identification of this concern as the weakest assumption and with the overall CONDITIONAL verdict.","tokens_in":18020,"tokens_out":3176,"duration_ms":190180,"concrete_test":"Stratify the test results on all three benchmarks by entity reachability (reachable within R@15 vs. unreachable) and report MMAgent-R2 accuracy and baseline accuracy on each stratum separately. If MMAgent-R2's gains over baselines are concentrated on the reachable stratum and it underperforms baselines on the unreachable stratum, the filtering creates a train-test gap that inflates the perceived benefit of the rejection mechanism. If gains hold on both strata, the concern does not land. This requires only re-running evaluation with the reachability label (already computable from the retrieval index) and does not need retraining.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Supplementary D.1 states that training samples are retained only when the ground-truth entity appears within the candidate pool accessible through initial retrieval plus at most Rejmax rejections. This is a reasonable design choice for reward signal quality — without filtering, hopeless cases would give degenerate rewards (every Reject gets δ=0.2, every Rerank gets 0). However, the model is never trained on cases where the entity is truly absent from all reachable batches. At test time, evaluation includes all samples. From Table 4, the R@15 ceiling reveals the fraction of unreachable test samples: InfoSeek ~20% (R@15=80.2%), E-VQA ~62% (R@15=37.8%), MMhops Bridging ~22% (R@15=78.4%), MMhops Comparison ~30% (R@15=70.4%). On E-VQA specifically, the majority of test samples have unreachable entities, yet the model's behavior on these cases — whether it wastes its rejection budget, hallucinates an entity, or falls back gracefully to parametric/text-search knowledge — is entirely untrained. The paper does not report accuracy stratified by entity reachability, so we cannot determine whether the SOTA gains come from the reachable subset (where the method is trained) or also extend to unreachable cases (where the method is untrained). The ablations (Tables 5-8) are evaluated on unfiltered test data and still show improvements, which partially mitigates the concern, but without stratified results the contribution of the rejection mechanism on unreachable cases remains opaque. This is load-bearing because the paper's headline gains are largest on E-VQA (+7.2), which also has the highest fraction of unreachable samples.","agreement_with_reader":"agree"},"referee_report":{"model":"glm-5.2","summary":"The paper proposes MMAgent-R2, an agentic multimodal retrieval-augmented generation (mRAG) framework for knowledge-based visual question answering (KB-VQA). The core idea is to equip a VLM agent with two internal verification actions—visual reranking and active rejection—within a multi-turn reasoning loop. Visual reranking allows the model to directly compare query and candidate images to identify the correct entity, while active rejection discards unreliable candidate batches and triggers retrieval of additional candidates. The framework is trained via Group Relative Policy Optimization (GRPO) with a composite reward function that includes an outcome reward, a format reward, and step-level verification rewards for reranking and rejection decisions. Experiments on InfoSeek, E-VQA, and MMhops show state-of-the-art results, with particularly large gains on E-VQA and multi-hop reasoning tasks.","tokens_in":18302,"tokens_out":916,"duration_ms":172099,"significance":"The paper addresses a genuine limitation in existing mRAG pipelines: the reliance on global visual features for initial retrieval, which often surfaces visually similar but factually mismatched candidates. The integration of visual reranking and active rejection as agent actions within a unified RL framework is a well-motivated design. The composite reward function with dense step-level verification signals is a technically sound contribution. The experimental evaluation is comprehensive, spanning three benchmarks with two base models (Qwen2.5-VL-7B and Qwen3-VL-8B) and including ablations isolating rerank vs. reject, batch size, rejection limit, and reward weight. The gains on E-VQA (+7.2 overall) and MMhops (+13.2/+10.4) are substantial. The paper also provides a clear analysis of entity identification accuracy relative to the retriever's recall ceiling (Table 4), which is a useful diagnostic.","major_comments":[{"comment":"Supplementary D.1 describes a training data filtering step: only samples where the ground-truth entity is reachable within the allowed rejection budget (initial retrieval + Rejmax rejections) are retained for training. This is a reasonable design choice for reward signal quality, but it creates a train-test distribution mismatch. At evaluation time, all samples are tested, including those where the correct entity is never retrieved. From Table 4, the R@15 ceiling reveals that a substantial fraction of test samples have unreachable entities (e.g., E-VQA R@15=37.8%, meaning ~62% of test samples have entities outside the reachable candidate pool). The paper does not report accuracy stratified by entity reachability, so it is unclear whether the SOTA gains come from the reachable subset (where the model is trained) or also extend to unreachable cases (where the rejection mechanism is untried","section":null}],"minor_comments":[{"comment":"The paper would benefit from a brief discussion of the train-test distribution mismatch created by the candidate filtering in Supp. D.1, even if only to acknowledge it as a limitation.","section":null},{"comment":"Table 2: The dagger notation for DPRv+t, RORA-VLM, and CoMEM indicates results obtained with a different knowledge base. It would be helpful to briefly state how the knowledge base differs (e.g., size, source) to give readers a sense of comparability.","section":null},{"comment":"Section 4.1.2: The learning rate is stated as 1x10^-6, which seems unusually low for GRPO training. Please verify this is correct.","section":null},{"comment":"Figure 1: The text in the figure is quite small and difficult to read. Consider enlarging key labels or providing a higher-resolution version.","section":null},{"comment":"The paper uses both 'MMAgent-R2-7B' and 'MMAgent-R2-8B' to refer to the two variants. It would be clearer to consistently use the base model size in the variant name (e.g., 'MMAgent-R2 (Qwen2.5-VL-7B)' and 'MMAgent-R2 (Qwen3-VL-8B)').","section":null}],"recommendation":"minor_revision","confidential_remarks":"The train-test distribution mismatch from candidate filtering (Supp. D.1) is the main concern, but it is a localized issue that can be addressed with additional analysis or stratified results. The core method and experimental design are sound. The paper is a strong contribution to the agentic mRAG literature."},"author_rebuttal":{"model":"glm-5.2","summary":"The referee raises one major comment concerning a train-test distribution mismatch: training data is filtered to samples where the ground-truth entity is reachable within the rejection budget, but evaluation includes all samples, including those where the correct entity is never retrieved. The referee asks whether SOTA gains come from the reachable subset or also extend to unreachable cases. We agree this is a valid concern and will add stratified analysis by entity reachability in the revision.","responses":[{"response":"We thank the referee for this insightful comment, which correctly identifies a real gap in our analysis. The referee is right that our training data filtering (retaining only samples where the ground-truth entity is reachable within the rejection budget) creates a distribution difference between training and evaluation, and that the current manuscript does not stratify results by entity reachability. We agree this stratification is important for readers to fully understand where the gains come from. We will add this analysis in the revision. To address the substantive question directly: the training filtering is a design choice to ensure meaningful reward signal quality during GRPO training — without it, the model would receive no positive verification reward on unreachable samples, potentially learning degenerate strategies. However, the capabilities learned on reachable samples (visual comparison for reranking, calibrated rejection when no match exists) are designed to generalize to unreachable cases at test time. Specifically, on unreachable samples, the model should reject all candidate batches and then either answer from parametric knowledge or reason over whatever partial evidence it has gathered. The rejection mechanism is thus not 'untried' on unreachable cases — it is precisely the mechanism that prevents the model from committing to a wrong entity. That said, we acknowledge that the current paper does not explicitly demonstrate this, and the referee's concern about whether gains extend to the unreachable subset is legitimate. In the revision, we will add a stratified breakdown of VQA accuracy and entity identification accuracy by reachability status (reachable vs. unreachable) for all three benchmarks. This will transparently show the model's behavior on both子","revision_made":"no","referee_comment":"Supplementary D.1 describes a training data filtering step: only samples where the ground-truth entity is reachable within the allowed rejection budget are retained for training. This creates a train-test distribution mismatch. At evaluation time, all samples are tested, including those where the correct entity is never retrieved. From Table 4, the R@15 ceiling reveals that a substantial fraction of test samples have unreachable entities (e.g., E-VQA R@15=37.8%, meaning ~62% of test samples have entities outside the reachable candidate pool). The paper does not report accuracy stratified by entity reachability, so it is unclear whether the SOTA gains come from the reachable subset (where the model is trained) or also extend to unreachable cases (where the rejection mechanism is untried)."}],"tokens_in":17697,"tokens_out":579,"duration_ms":89655,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"Short version: MMAgent-R2 adds visual reranking and active rejection as agent actions inside a GRPO-trained mRAG loop, with step-level verification rewards. The method is clean, the results are strong, and the one real concern is a training/evaluation distribution gap that the paper doesn't analyze but should.","headline":"Solid agentic mRAG with visual reranking + rejection; real train-test gap concern on E-VQA that needs stratified results to resolve.","tokens_in":18878,"tokens_out":887,"would_cite":true,"duration_ms":48785,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Visual reranking and active rejection cut VQA errors by 7 points","keywords":[],"falsifier":"If the verification reward (R_ver) is removed and only the outcome reward remains, and the model's entity identification accuracy drops back to near the retriever's Recall@1 level, then the step-level supervision—not the agentic architecture itself—is the load-bearing component. Conversely, if identification accuracy stays high without R_ver, the architecture is what matters and the reward design is incidental.","tokens_in":18266,"feed_emoji":"🔍","tokens_out":1501,"duration_ms":191859,"temperature":0.7,"pith_summary":"MMAgent-R2 argues that the dominant failure mode in knowledge-based visual question answering is not reasoning but entity identification: when a knowledge base contains many visually similar entries, the initial image retriever returns look-alike distractors, and every downstream step is trapped reasoning over the wrong entity. The paper's response is to make the vision-language model itself the verifier. Instead of accepting the retriever's top-K candidates and filtering them at the text level, the model directly compares the query image against each candidate image, reranking by fine-grained visual detail that textual descriptions cannot capture. When no candidate matches, the model actively rejects the entire batch and triggers a new retrieval, breaking out of the fixed candidate pool that conventional pipelines are locked into. These two capabilities—visual reranking and active rejection—are unified as internal verification actions inside a single multi-turn agent loop, alongside external retrieval and answer generation. The entire system is trained end-to-end with Group Relative Policy Optimization (GRPO) using a composite reward function that includes not only final-answer correctness but step-level verification rewards: the model gets credit for correctly identifying the right entity in a candidate batch and for correctly rejecting a batch when the target is absent. This dense intermediate supervision teaches the model when to commit and when to keep searching. On E-VQA, where the retriever's Recall@1 is only 15.4%, MMAgent-R2-8B raises entity identification accuracy to 29.5% and overall VQA accuracy to 54.2%, outperforming the runner-up by 7.2 points. On MMhops, which requires multi-image multi-hop reasoning, the model surpasses the previous best by 13.2 points on Bridging and 10.4 on Comparison. The central claim is that visual-level verification by the reasoning model itself—not external modules, not text-level filtering—is the missing piece that prevents retrieval errors from propagating into final answers.","feed_headline":"Visual reranking and active rejection cut VQA errors by 7 points","feed_subtitle":"A vision-language model trained to verify its own retrievals nearly doubles entity identification on the hardest knowledge base, turning a 7","key_machinery":"The key machinery is a three-part agent action space—External Retrieval (SearchImage, SearchText), Internal Verification (Rerank, Reject), and Answer Generation—trained jointly via GRPO with a composite reward R = R_outcome + R_format + R_ver. The verification reward R_ver provides per-step supervision for every reranking and rejection decision: +1 for correctly selecting the ground-truth entity, +δ (default 0.2) for correctly rejecting a batch that does not contain the target, and 0 otherwise. Rejection triggers non-overlapping candidate batches up to Rejmax=2 times per query image. Environment response tokens are masked during policy gradient computation so only the model's own reasoning","core_discovery":"The paper's central discovery is that a vision-language model, trained with step-level verification rewards via GRPO, can serve as its own visual reranker and rejection mechanism inside an agentic retrieval loop, substantially improving entity identification accuracy beyond what the external retriever achieves alone. The improvement is largest precisely where retrieval is hardest: on E-VQA, where the 2M-page knowledge base yields only 15.4% Recall@1, visual reranking nearly doubles identification accuracy from 15.4% to 29.5%. The ablation shows that visual reranking alone accounts for most of the gain (from 40.2% to 44.3% VQA accuracy without rejection), while active rejection adds a further","pith_inferences":["The training distribution excludes cases where the ground-truth entity is never retrieved, but the evaluation includes them. The model's behavior on these unreachable cases is not separately reported, yet they may constitute a significant fraction of errors—particularly on E-VQA where Recall@15 is only 37.8%. A per-sample breakdown by reachability would clarify whether the gains come from better v","The rejection reward weight δ=0.2 was tuned on InfoSeek and applied to all datasets. Since the optimal δ likely depends on the retriever's recall distribution, datasets with much lower recall (like E-VQA) might benefit from a higher δ to encourage more aggressive candidate exploration.","The paper notes that performance declines at Rejmax=3 due to context length degradation. This suggests that a context-compression mechanism for rejected batches—summarizing visual comparisons rather than retaining full image tokens—could extend the useful rejection budget without the context-length penalty.","The model is trained on three datasets jointly, but the ablations are run on InfoSeek only. Whether the optimal hyperparameters (K=5, Rejmax=2, δ=0.2) transfer to the much harder E-VQA retrieval scenario or the multi-hop MMhops scenario is not directly verified."],"forward_implications":["If visual reranking by the reasoning model is the primary driver of accuracy gains, then improving the base VLM's fine-grained visual discrimination directly improves the entire mRAG pipeline—retrieval quality becomes less of a bottleneck and model perception becomes more of one.","The step-level verification reward structure is transferable: any agentic retrieval system where intermediate decisions can be checked against ground truth (e.g., document retrieval, code search) could adopt the same pattern of rewarding correct acceptance and correct rejection separately.","The training-data filtering that retains only solvable cases means the model is never trained on truly unreachable entities. If deployed at scale, the model's behavior on the unreachable tail—whether it hallucinates, rejects indefinitely, or falls back to parametric knowledge—will determine real-world reliability.","The gap between achieved identification accuracy and the R@15 ceiling (e.g., 29.5% vs 37.8% on E-VQA) quantifies how much room remains for better visual discrimination within the candidate pool, suggesting a concrete benchmark for future VLM perception improvements."],"fun_headline_variants":["VLM that reranks and rejects its own retrievals near-doubles entity ID on hard KB-VQA","Agentic mRAG with self-verification lifts VQA accuracy on 2M-entity knowledge base","Visual reranking plus active rejection cuts VQA errors where retrieval recall is low","Training VLM to verify retrievals via GRPO improves multi-hop VQA over fixed candidates","Self-reranking agent rejects low-confidence retrievals and pulls new candidates for VQA"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The training data is filtered to retain only samples where the ground-truth entity is reachable within the allowed rejection budget, but evaluation includes all samples including those where the correct entity is never retrieved. The model is thus trained exclusively on solvable cases yet must handle unsolvable ones at test time, and the paper does not analyze how this train-eval distribution gap affects the model's rejection or hallucination behavior when the target entityis","fun_headline_variants_meta":{"raw":{"variants":["VLM that reranks and rejects its own retrievals near-doubles entity ID on hard KB-VQA","Agentic mRAG with self-verification lifts VQA accuracy on 2M-entity knowledge base","Visual reranking plus active rejection cuts VQA errors where retrieval recall is low","Training VLM to verify retrievals via GRPO improves multi-hop VQA over fixed candidates","Self-reranking agent rejects low-confidence retrievals and pulls new candidates for VQA","Internal visual reranking nearly doubles Recall@1 on E-VQA's 2M-page knowledge base"]},"model":"glm-5.2","effort":"low","cost_usd":0.0,"raw_usage":{"total_tokens":770,"prompt_tokens":599,"completion_tokens":171,"prompt_tokens_details":null},"tokens_in":599,"tokens_out":171,"duration_ms":14658,"temperature":1.0,"reasoning_tokens":38,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-09T12:24:38.662338+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If the verification reward (R_ver) is removed and only the outcome reward remains, and the model's entity identification accuracy drops back to near the retriever's Recall@1 level, then the step-level supervision—not the agentic architecture itself—is the load-bearing component. Conversely, if identification accuracy stays high without R_ver, the architecture is what matters and the reward design is incidental.","supporting_citations":[],"review_version":1}