{"id":"81f8eb05-5f21-4c60-8099-acf41bef4cad","arxiv_id":"2603.05256","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":5,"one_line_summary":"Controllable retrieval-difficulty curriculum plus reward-propagation sampling lets RL close the pretrain-to-KB-VQA gap and beat prior SOTA on two hard encyclopedic VQA benchmarks.","lead":"Wiki-R1 trains multimodal models for knowledge-based visual QA by generating easier-to-harder retrieval contexts and sampling only the examples that still give useful RL gradients. It reports new state-of-the-art numbers on Encyclopedic-VQA and InfoSeek with far less training data than prior RAG fine-tunes.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Curriculum sampling gains rest on an unvalidated article-similarity proxy for question difficulty under noisy retrieval.","rationale":"The reader correctly isolates the difficulty-proxy assumption as the clearest load-bearing risk. Controllable retrieval modification (gap levels g) is well-motivated by the sparse-reward diagnosis in Fig. 1 and is ablated; the sampling half is more fragile because its only empirical support is the ablation that shows Obs. Prop. is necessary for sampling to help, not that the graph actually ranks difficulty correctly. Absolute gains remain modest and several hyperparameters (τ, α, λ, window) are tuned in the same regime, so CONDITIONAL is still the right call. The concrete correlation-plus-ablation test would settle whether the sampling curriculum is doing real work or is largely decorative. No stronger internal inconsistency appears; the paper’s diagnostics, multi-run stability and data-efficiency claims are otherwise solid.","tokens_in":18974,"tokens_out":582,"duration_ms":8722,"concrete_test":"On the 40k training pool, compute Spearman rank correlation between (a) the final propagated difficulty scores H and (b) empirical per-question success rates obtained by rolling out the fixed base policy 8–16 times under the true inference-time retrieval (g=G, no GT forced). If |ρ| < 0.25, re-run the Table 4 full Wiki-R1 row with a random or accuracy-agnostic sampler of equal budget; if the SOTA numbers drop by more than ~1.5 points absolute, the sampling-curriculum contribution is not load-bearing.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim attributes SOTA gains (37.1% EVQA / 44.1% InfoSeek with 40k examples) to the joint data+sampling curriculum. Table 4 shows that data curriculum alone already lifts DAPO substantially (EVQA overall 31.4\to34.5, InfoSeek 41.5\to43.0); adding sampling without observation propagation hurts, and only the full Obs. Prop. recovers the final numbers. That recovery depends on Appendix A.4 / Algorithms 1–2: sparse binary rewards are propagated over a TF-IDF (or Sentence-Transformer) graph whose edges are defined solely by ground-truth Wikipedia article similarity (top-100 neighbors). The paper never measures whether article-level textual similarity predicts question-level solvability under the actual noisy multi-document retrieval used at training time. If the proxy is weak, the sampling schedule is effectively near-random or cluster-biased, so the headline gains cannot be cleanly credited to the claimed curriculum mechanism rather than to data curriculum + extra RL steps.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"Wiki-R1 addresses sparse-reward RL for knowledge-based VQA by constructing a curriculum of training distributions. Controllable data generation manipulates the retriever via discrete gap levels g (k retrieved candidates and whether ground-truth is forced) so that difficulty rises from pure ground-truth (g=0) to the noisy inference-time distribution (g=G). A curriculum sampling schedule then selects samples whose estimated accuracy is near 0.5; difficulty is obtained by non-parametric label propagation of sparse binary rewards over a graph whose edges are TF-IDF (or Sentence-Transformer) similarities of ground-truth Wikipedia articles. Built on DAPO with Qwen2.5-VL 3B/7B and only 40k entity-balanced examples, the method reports new SOTA numbers: 37.1% on Encyclopedic-VQA (prior 35.5%) and 44.1% on InfoSeek (prior 40.1%), plus strong zero-shot transfer to ViQuAE and competitive oracle-entity upper bounds.","tokens_in":19250,"tokens_out":1077,"duration_ms":10338,"significance":"If the gains hold, the paper supplies a practical, data-efficient recipe for post-training MLLMs under noisy retrieval and a clear distributional gap from pretraining. The controllable gap-level construction is a clean, reusable idea that goes beyond ordinary sample re-weighting; the ablations (Table 4), training-dynamics plots (Figure 3), multi-run stability (Appendix B), and zero-shot transfer results give the empirical claims real weight. The work is therefore of clear interest to the multimodal RAG and RL-for-reasoning communities, even if the precise contribution of the observation-propagation component remains only partially isolated.","major_comments":[{"comment":"Table 4 shows that data curriculum alone already lifts DAPO substantially (EVQA overall 31.4\to34.5, InfoSeek 41.5\to43.0). Adding sampling curriculum without observation propagation hurts, and only the full Obs. Prop. recovers the final numbers. The recovery rests on Appendix A.4 / Algorithms 1–2: sparse binary rewards are propagated over a graph whose edges are defined solely by ground-truth article similarity (top-100 TF-IDF or Sentence-Transformer neighbors). The manuscript never measures whether article-level textual similarity predicts question-level solvability under the actual noisy multi-document retrieval used at training time. Without such a correlation (or an ablation that replaces the graph with random/entity-only edges), the headline gains cannot be cleanly attributed to the claimed sampling curriculum rather than to data curriculum plus extra RL steps.","section":null},{"comment":"Section 3.3 and Algorithm 1 define gap levels g via (k,γ) but never report the empirical distribution of realized difficulties (fraction of non-zero advantages, mean reward) actually observed at each g. Because the upgrade rule is driven by a sliding-window accuracy threshold τ, it is possible that later gap levels remain easier than intended or that the schedule simply re-samples easy clusters. A short diagnostic table or plot of realized reward statistics per gap level would make the “controllable curriculum” claim falsifiable and would strengthen the causal link between the generation mechanism and the final accuracy.","section":null}],"minor_comments":[{"comment":"Figure 1 caption and surrounding text refer to “DAPO on KB-VQA” without stating the exact base model or retrieval configuration used for the preliminary curves; a one-sentence clarification would help readers reproduce the sparse-reward diagnosis.","section":null},{"comment":"Table 1 reports “Avg.” across benchmarks but the column is not defined; clarify whether it is a simple mean of the two overall scores or a weighted average.","section":null},{"comment":"Appendix A.1: the fusion weight λ is chosen on the training set of each benchmark separately (0.985 / 0.997). A short note on sensitivity of final accuracy to λ would be useful, given that the values sit so close to pure visual retrieval.","section":null},{"comment":"Typographical inconsistencies appear in several places (e.g., “V olcengine”, “V .and T.”, “stardenotes”). A light copy-edit pass would improve readability.","section":null},{"comment":"Section 4.3 states that the reward is exact-match binary; it would be helpful to confirm whether any soft matching or BEM-style partial credit is used during training, given that evaluation on EVQA uses BEM.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The core technical contribution (gap-level data generation) is solid and the empirical package is stronger than many concurrent RL-for-MLLM papers. The observation-propagation proxy is the only load-bearing claim that is currently under-supported; requiring a short correlation diagnostic or random-graph ablation would convert the paper into a clean accept without expanding its scope. I see no novelty or citation-pattern concerns."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is that plain DAPO collapses on KB-VQA because of sparse rewards and the pretrain-to-encyclopedic gap; Wiki-R1 fixes most of that by generating a controllable difficulty ladder through the retriever itself (force ground-truth in/out, vary k) and then sampling with reward propagation. That combination is what is actually new relative to ADARFT, DUMP, VL-Cogito, or CL-RAG, which mostly re-weight fixed data.\n\nWhat they do well is the systems work. Ablations cleanly separate data curriculum, sampling, and observation propagation. Training curves show fewer zero-advantage trajectories and stable gains once the curriculum reaches the hardest gap level. They report oracle-entity upper bounds, zero-shot transfer to ViQuAE that beats the semi-oracle RC baseline, multi-run stability, and they do it with only 40k examples versus the millions used by the SFT baselines. The absolute lifts (35.5\to37.1 EVQA, 40.1\to44.1 InfoSeek) are modest but consistent across two hard public benchmarks and a single retrieval system, which is more than most of the prior RAG papers manage.\n\nThe soft spot the stress-test flags is real but secondary. Table 4 already shows that data curriculum alone does most of the heavy lifting; sampling without propagation hurts, and only the full observation-propagation graph recovers the final numbers. That graph is built from TF-IDF (or Sentence-Transformer) similarity of ground-truth Wikipedia articles. They never directly check whether article-level textual similarity predicts question-level solvability under the actual noisy multi-document retrieval used at train time. If the proxy is weak the sampler is partly random or cluster-biased. Still, the paper is honest about the limitation section, the sensitivity plots are stable inside the operating range, and the core claim does not collapse if the sampler is imperfect.\n\nMath is ordinary policy-gradient / DAPO; citations cover the right curriculum-RL and KB-VQA lines without obvious padding. This is for people who post-train VLMs or build tool-using multimodal agents and care about sparse-reward RAG. It deserves a serious referee. I would engage with it and expect to cite the controllable-generation idea.","headline":"Solid, data-efficient curriculum RL for noisy multimodal RAG: the retriever-manipulation ladder is the real novelty, gains are real but modest, and the sampling proxy is the softest joint.","tokens_in":19849,"tokens_out":554,"would_cite":true,"duration_ms":5522,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Wiki-R1 closes the pretraining-to-KB-VQA gap by generating a curriculum of retrieval difficulties and sampling only the samples that still give learning signal.","keywords":["knowledge-based VQA","curriculum learning","reinforcement learning","retrieval-augmented generation","multimodal LLMs","observation propagation","sparse rewards"],"falsifier":"Replace the article-similarity graph with random edges or disable propagation entirely; if the fraction of zero-advantage trajectories and final accuracy remain unchanged, the claimed difficulty-estimation step is inert.","tokens_in":19843,"feed_emoji":"🔍","tokens_out":739,"duration_ms":18938,"temperature":0.7,"pith_summary":"Knowledge-based visual question answering forces a pretrained multimodal model to reason over noisy Wikipedia passages whose style it never saw, so ordinary reinforcement learning mostly produces zero-reward trajectories and stalls. Wiki-R1 solves the problem by deliberately manufacturing a sequence of training distributions: it first hands the model only the gold article, then gradually adds distractors, and finally removes the gold guarantee so the data finally matches real retrieval. A second stage estimates how hard each remaining example is by spreading the sparse observed rewards across a graph of similar knowledge articles, then preferentially samples the ones whose accuracy sits near one-half. On Encyclopedic-VQA and InfoSeek the resulting policy sets new state-of-the-art numbers while training on only forty thousand examples. A reader who cares about teaching large multimodal models to use external knowledge under realistic noise now has a concrete recipe that works.","feed_headline":"Curriculum of easier retrievals lifts knowledge-VQA scores","feed_subtitle":"By first giving models gold Wikipedia pages, then adding noise, RL finally works on hard multimodal RAG.","key_machinery":"Controllable curriculum data generation (discrete gap levels that force ground-truth inclusion and vary the number of retrieved candidates) together with observation-propagation sampling (non-parametric label propagation of binary rewards over a TF-IDF graph of Wikipedia articles).","core_discovery":"The paper shows that a data-generation-based curriculum—controllably altering the retriever’s candidate set and gold-document inclusion, then sampling with difficulty estimates obtained by propagating sparse rewards—systematically incentivizes multimodal reasoning and lifts accuracy past all prior retrieval-augmented systems on two hard knowledge-VQA benchmarks.","pith_inferences":["Observation propagation over shared latent structure could cut wasted trajectories in any sparse-reward RL domain (code problems sharing libraries, math problems sharing lemmas).","If article-level TF-IDF is only a weak proxy, swapping it for question embeddings or a learned difficulty head would further stabilize the sampling schedule.","Controllable generation of intermediate distributions is likely more powerful than pure selection curricula whenever the domain shift itself can be parameterized (noise level, context length, modality drop-out)."],"forward_implications":["Reinforcement learning becomes practical for multimodal RAG settings whose retrieval noise previously produced near-total reward sparsity.","Entity-balanced training sets of only 40 k examples can outperform systems that fine-tune on millions of samples.","The same gap-level schedule transfers to any domain where a retriever can be forced to include or exclude gold documents.","Models trained this way show stronger generalization on unseen questions and zero-shot transfer to related knowledge-VQA sets."],"fun_headline_variants":["Retrieval curriculum with reward propagation lifts multimodal KB-VQA","Wiki-R1 bridges pretrain gap via controlled difficulty for KB-VQA RL","Controllable data generation curriculum enables RL on hard knowledge VQA","Sparse reward sampling advances SOTA on Encyclopedic VQA and InfoSeek","From gold docs to noisy retrieval: curriculum RL incentivizes MLLM reasoning"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The method assumes that the hardness of an unseen question can be reliably guessed from the textual similarity of its Wikipedia article to articles of questions whose rewards have already been observed.","fun_headline_variants_meta":{"raw":{"variants":["Retrieval curriculum with reward propagation lifts multimodal KB-VQA","Wiki-R1 bridges pretrain gap via controlled difficulty for KB-VQA RL","Controllable data generation curriculum enables RL on hard knowledge VQA","Sparse reward sampling advances SOTA on Encyclopedic VQA and InfoSeek","From gold docs to noisy retrieval: curriculum RL incentivizes MLLM reasoning"]},"model":"grok-4.5","effort":"low","cost_usd":0.006066,"raw_usage":{"total_tokens":1627,"prompt_tokens":828,"num_sources_used":0,"completion_tokens":97,"cost_in_usd_ticks":60660000,"prompt_tokens_details":{"text_tokens":828,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":702,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":828,"tokens_out":97,"duration_ms":5906,"temperature":1.0,"reasoning_tokens":702,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-15T14:42:22.250274+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Replace the article-similarity graph with random edges or disable propagation entirely; if the fraction of zero-advantage trajectories and final accuracy remain unchanged, the claimed difficulty-estimation step is inert.","supporting_citations":[],"review_version":1}