{"id":"eabdb81c-4336-4bce-a8f6-713410da9732","arxiv_id":"2506.07905","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"WeThink, a 120K-image QA dataset with AI-generated reasoning paths, combined with hybrid-reward reinforcement learning, improves a 7B vision-language model across 14 benchmarks.","lead":"This paper introduces WeThink, a 120K-question visual reasoning dataset generated by two large AI models, and shows that reinforcement learning on it improves a 7B vision-language model on 14 benchmarks. The work is a step toward general-purpose multimodal reasoning, though the gains on general tasks are modest and some scores decline.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unquantified training/evaluation overlap between WeThink sources (ChartQA, AI2D) and eval benchmarks (AI2D Test, MathVista_MINI) threatens the central generalization claim.","rationale":"The reader's weakest assumption—evaluation contamination from overlapping training images—is also the most load-bearing concern I can identify. The paper's headline is an empirical generalization claim: a 120K dataset plus hybrid-reward GRPO improves a 7B MLLM on both math and general benchmarks. For that claim to hold, the evaluation numbers must reflect reasoning ability rather than memorization of training content. Table 1 explicitly includes AI2D and ChartQA as image sources, and Section C.2 explicitly evaluates on AI2D (Test) and MathVista_MINI, a benchmark with a large chart-derived component. The paper does not report any overlap analysis, deduplication, or exclusion step, so the possibility of inflated scores is concrete rather than hypothetical. This is not a disagreement with the field's consensus; it is an internal evidential gap between the data construction and the evaluation protocol. I do not see a more fundamental issue: the dataset release is a real contribution, the pipeline is clearly described, and the RL setup is standard. If a contamination check came back clean, the conditional verdict could be upgraded. If the check reveals substantial overlap, the central claim would need to be weakened or re-benchmarked on non-overlapping evaluations. The reader's CONDITIONAL verdict is therefore the right call, and no verdict change is needed from this stress-test pass.","tokens_in":23321,"tokens_out":2946,"duration_ms":37293,"concrete_test":"Compute exact and fuzzy overlap between all WeThink training images and every evaluation split used in Tables 5/6 (AI2D Test, MathVista_MINI, MathVerse_MINI_Vision_Only, DynaMath, WeMath, LogicVista, MMMU Val, HallusionBench, etc.), using dataset IDs where available (e.g., ChartQA image indices, AI2D diagram IDs) plus perceptual hashing for near-duplicates. Then re-run the RL checkpoint with all overlapping images or questions removed from training, or evaluate only on the non-overlapping subset, and compare average gains. If the math/general gains drop to near zero or become statistically indistinguishable from the base model, contamination is the likely driver; if gains persist on the overlap-free subset, the generalization claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that RL on WeThink yields broad, generalizable gains (Tables 5/6: +3.9 math average, +1.2 general average)—requires that evaluation benchmarks are not memorized through training-data overlap. Table 1 lists AI2D (12,024 images) and ChartQA (22,865 images) among the 18 WeThink image sources, while Section C.2 evaluates on AI2D (Test) and MathVista_MINI, whose chart questions substantially derive from ChartQA. In addition, Section 3.2's optional contextual constraints allow prior questions from source datasets to guide question generation, so contamination could occur at the question-text level, not only at the image level. The paper provides no deduplication, no exclusion of overlapping images or questions, and no per-benchmark overlap analysis. Notably, the largest reported gains are concentrated in chart/diagram-heavy benchmarks (WeMath +9.9, LogicVista +3.4), exactly where ChartQA/AI2D training data is concentrated. Until this overlap is quantified and removed, the 'general-purpose reasoning' conclusion is not distinguishable from benchmark memorization.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces WeThink, a 120K multimodal QA dataset with reasoning paths generated by a pipeline in which Qwen2.5-VL-72B describes images and DeepSeek-R1 formulates questions through multi-turn refinement; answers are constructed and filtered by DeepSeek-R1, Qwen2.5-VL-72B, and Gemini 2.5 Pro. The authors then apply supervised fine-tuning and GRPO-based reinforcement learning with a hybrid rule/model-based reward to Qwen2-VL-7B and Qwen2.5-VL-7B, reporting improvements across six mathematical and eight general multimodal benchmarks. They also report that direct RL outperforms cold-start SFT followed by RL, that hybrid rewards outperform either reward alone, and that adding more diverse data from the Internet yields further gains.","tokens_in":23455,"tokens_out":4977,"duration_ms":56935,"significance":"If the empirical claims hold, the paper's main contributions are valuable: an open-source dataset with reasoning paths, an automated and potentially scalable synthesis pipeline, and a systematic comparison of SFT versus RL and reward types on a 7B MLLM. The use of VLMEvalKit and the release of the dataset help reproducibility, and the finding that better-conditioned CoT data matters for SFT is clearly demonstrated in Table 2. However, the paper's headline generalization claim is currently under-supported by the uncontrolled overlap between training sources and evaluation benchmarks, by the absence of statistical uncertainty estimates, and by the confounded diversity comparison.","major_comments":[{"comment":"The central claim that RL on WeThink yields generalizable gains (Tables 5 and 6, +3.9 math and +1.2 general) is not distinguishable from benchmark memorization because training and evaluation image sources overlap. Table 1 lists AI2D (12,024 images) and ChartQA (22,865 images) among the 18 WeThink sources, while C.2 evaluates on AI2D (Test) and MathVista_MINI, whose chart questions substantially derive from ChartQA. Since Section 3.2's Optional Contextual Constraints allow prior questions from the source datasets to guide question generation, contamination can occur at the question-text level as well as the image level. The paper reports no deduplication, no exclusion of overlapping images or questions, and no per-benchmark overlap analysis. The largest gains are concentrated in chart/diagram-heavy benchmarks (WeMath +9.9, LogicVista +3.4), which is exactly where ChartQA/AI2D training data is concentrated. Please quantify the overlap (e.g., exact or near-duplicate image hashing and question string matching) and re-report all benchmark numbers after removing overlapping samples, or supplement with evaluations on benchmarks disjoint from the training sources.","section":"Section 3.1 / Table 1 vs Section C.2"},{"comment":"Every reported number is a single run with no error bars, no multiple seeds, and no significance testing. RL training with GRPO at temperature 1.0 is stochastic, and the claimed effects are small relative to benchmark noise: the general-task average gain is +1.2, and individual general benchmarks fluctuate from -0.4 (AI2D) to +4.6 (MMVet); several math gains are under 2 points (MathVision +0.8). Without variance estimates or significance tests, the central quantitative claims (e.g., Table 4's All vs Math comparison and Table 3's Rule+Model vs Rule comparison) are not statistically supported. Please report mean and standard deviation over at least three RL runs, or bootstrap over evaluation samples, for the main comparisons.","section":"Tables 3-6"},{"comment":"The paper uses the comparison 'Math' vs 'All' in Table 4 to conclude that increasing question-domain diversity improves performance (Section 4.3). This comparison is confounded by training-data quantity: the 'All' condition uses the full WeThink dataset while 'Math' uses only math-type questions, so the gains could come from more data rather than from diversity. Since the stated finding is specifically about diversity, please add a matched control that trains on the same number of samples drawn from the math-type subset (or otherwise equalizes sample count and training compute) before attributing the improvement to domain diversity.","section":"Table 4 / Section 4.3"},{"comment":"The hybrid reward claim rests on the model-based reward for descriptive questions, but the paper provides no validation of the DeepSeek-V3 judge. Table 3 shows that mixing model-based reward with rule-based reward improves the average by only 1.2 points over rule alone, while model-only reward degrades performance (-1.1), and the descriptive answers used in RL are generated and verified by the same model family (DeepSeek-R1 answers, DeepSeek-V3 judge, Qwen2.5-VL cross-check). Without a judge-agreement study or a per-question-type breakdown of reward accuracy, it is unclear whether the hybrid reward's benefit comes from the model-based signal or from the extra diversity and number of training questions. Please report judge agreement against a human-annotated sample and ablate the model-based reward while holding question type and sample size fixed.","section":"Appendix B.3 / Table 3"}],"minor_comments":[{"comment":"There are several typos: 'an novel' in the Introduction, 'resoning' in Section 4.3, and 'difficutity' in the Figure 5 caption; these should be corrected.","section":"Introduction and Figure 5"},{"comment":"The Related Work section does not discuss benchmark contamination or data leakage in MLLM training, and the Limitations section does not mention the evaluation-overlap risk identified in this report; adding such a discussion would improve the paper's completeness.","section":"Related Work and Limitations"},{"comment":"The text says 'we perform two rounds of verification' but then describes three stages (Preliminary Question Filtering, Answer Construction and Quality Control, CoT Refinement); please clarify whether the two rounds refer only to the filtering stage.","section":"Section 3.3"},{"comment":"The pipeline description mentions 'Gemini 2.5 Pro' but reference [33] is the Gemini 1.5 technical report; please update the citation to the correct Gemini 2.5 documentation.","section":"References [33] and Section 3.3"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful, honest empirical paper. The WeThink dataset (120K QA pairs with reasoning paths from 18 image sources) is a real resource, and the multi-turn Qwen2.5-VL-72B + DeepSeek-R1 generation pipeline with ability-synergy constraints and refined CoT is a concrete, reproducible construction. The central finding—direct hybrid-reward RL on the full set outperforms cold-start SFT+RL and math-only RL, with gains spilling into general benchmarks—is worth taking seriously. I also credit them for reporting the negative SFT result on Qwen2.5-VL and for the 14-benchmark evaluation.\n\nThe soft spots are real but addressable. First, the contamination risk is not hypothetical. Training samples include 22,865 ChartQA images and 12,024 AI2D images; evaluation uses AI2D Test and MathVista_MINI, which has chart questions drawn from chart QA sources. No dedup or overlap analysis is reported. The biggest gains (WeMath +9.9, LogicVista +3.4) are exactly in chart/diagram-heavy benchmarks. That pattern doesn't prove memorization, but it begs for a per-benchmark overlap check. Second, every table is a single run with no error bars; general gains average around +1.2 to +1.7 points, so run-to-run variance could matter. Third, the model-based reward uses DeepSeek-V3 as judge for answers that DeepSeek-R1 generated; that's a partially self-referential loop, though not a circular benchmark. The authors themselves concede the dataset still contains noise and wrong cases.\n\nNone of this kills the paper. The dataset and the direct-RL-vs-cold-start comparison are solid contributions. But the headline claim of 'significant improvements across general tasks' is conditioned on the overlap question. A serious referee should ask for contamination quantification, multiple seeds, and at least one comparison against an alternative RL/CoT dataset (e.g., LLaVA-CoT or a math-only RL baseline) to ground the diversity story.\n\nWho is it for: people building R1-style multimodal RL pipelines and dataset builders. I'd bring it to a reading group and I'd cite the dataset. It deserves peer review—the empirical claims are externally checkable and the resource is useful.","headline":"Solid dataset-and-recipe paper for R1-style multimodal RL, worth peer review, but the training/eval contamination question needs an answer before the generalization claim is fully credible.","tokens_in":24111,"tokens_out":1864,"would_cite":true,"duration_ms":21326,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 120K diverse QA dataset plus hybrid-reward RL improves a 7B vision-language model across 14 benchmarks.","keywords":["multimodal reasoning","reinforcement learning","visual question answering","dataset synthesis","chain-of-thought","hybrid reward","vision-language models","scalable data pipeline"],"falsifier":"Compute exact and near-duplicate overlap between every evaluation benchmark image and question and the 120K training set; if a substantial fraction of test items are duplicates or near-duplicates of training items, the reported average gains cannot be attributed to general multimodal reasoning.","tokens_in":23037,"feed_emoji":"🧠","tokens_out":6023,"duration_ms":67210,"temperature":0.7,"pith_summary":"The paper tries to establish that general-purpose vision-language reasoning can be trained by reinforcement learning, not just math or perception skills, provided the RL runs on a large, diverse set of reasoning-centric questions with flexible rewards. It builds an automated pipeline that turns images into context-aware question-answer pairs with explicit reasoning paths, releases a dataset of over 120K such pairs from 18 image sources, and trains a 7B vision-language model with a hybrid reward combining exact rule checks on choice and fill-in-the-blank answers with a judge-model assessment of descriptive answers. Across 14 benchmarks the trained model beats its base by an average of +3.9 points on six math benchmarks and +1.2 points on eight general multimodal benchmarks, and adding more diverse web images raises the gains further. The central claim matters because it suggests data diversity plus hybrid-reward RL, rather than domain-specific recipes, is the lever for general multimodal reasoning.","feed_headline":"RL on 120K synthetic questions lifts a 7B vision-language model","feed_subtitle":"Hybrid rewards plus diverse reasoning data beat math-only training across 14 benchmarks.","key_machinery":"The load-bearing mechanism is the Scalable Multimodal QA Synthesis pipeline together with a hybrid reward system. The pipeline pairs two large models in multi-turn dialogue: one gives a coarse image caption and answers follow-up clarification requests, while the other spots missing visual details and formulates questions that must combine reasoning with at least one of recognition, knowledge, OCR, spatial awareness, or math. Answers are kept only when two independent verifiers agree or a third judge confirms them, and chains of thought are shortened by a refinement step. During RL, the reward is a weighted sum of a format reward and an accuracy reward that switches between exact string matching for multiple-choice and fill-in-the-blank questions and a judge-model score for descriptive answers, all optimized with group-relative policy optimization, which normalizes rewards across a group of sampled responses.","core_discovery":"The paper's claim, stated on its own terms, is that direct reinforcement learning on a diverse, reasoning-path-annotated question set is sufficient to improve a strong open 7B multimodal model broadly. It reports that training on all question types in the dataset outperforms training on math questions only, with the math benchmark average rising from 41.4 to 44.1 and the general benchmark average from 70.6 to 71.8. It also reports that direct RL without cold-start supervised fine-tuning beats the cold-start route in this setting, and that adding roughly 20K internet images through the same pipeline raises the math average to 44.7 and the general average to 72.3. The key empirical finding is that question-domain diversity, not just answer verifiability, is what transfers to general multimodal tasks.","pith_inferences":["If the gains survive a clean separation between training and evaluation images, the same pipeline should transfer to other base models and language settings, making data synthesis the main bottleneck for general multimodal reasoning.","The pipeline could be turned into a closed loop: evaluate on held-out diverse benchmarks, mine failures, synthesize questions for those failure images, and retrain; the paper's scalability result implies this would keep improving.","A natural stress test is to train on questions from only one ability family, such as OCR or recognition, and measure whether cross-domain gains come from diversity itself or from a shared reasoning format; the paper does not isolate this.","The hybrid-reward design suggests descriptive-answer RL could extend to open-ended multimodal generation tasks beyond question answering, such as captioning or embodied visual QA."],"forward_implications":["Direct RL on a diverse reasoning dataset can serve as a general post-training recipe for strong multimodal models, reducing the need for cold-start supervised fine-tuning.","Expanding data diversity through an automated pipeline yields further gains, suggesting continuous data collection can keep improving model performance.","Rule-based rewards alone are insufficient for general multimodal reasoning; hybrid rewards with model-based assessment extend RL to subjective and descriptive answers.","Chain-of-thought quality matters: refined, concise reasoning paths support both SFT and RL, while long raw chains can degrade a well-optimized base model.","A 7B open model can reach stronger math and general multimodal benchmark results using RL plus synthetic data, without architectural changes."],"supporting_citations":[{"why":"Supplies the base 7B vision-language model and the 72B captioning model used in data synthesis.","marker":"[1]"},{"why":"Establishes the R1-style reinforcement-learning-with-verifiable-rewards paradigm this paper adapts to multimodal inputs.","marker":"[4]"},{"why":"Acts as a third verifier that re-evaluates disputed answers during quality control.","marker":"[33]"},{"why":"Provides the 18 image source datasets and the chain-of-thought supervised fine-tuning approach this pipeline builds on.","marker":"[48]"},{"why":"Supplies the integrated-ability taxonomy used to require reasoning plus one complementary skill per question.","marker":"[82]"},{"why":"Provides the evaluation toolkit and leaderboard results used for the 14 benchmarks.","marker":"[95]"},{"why":"Supplies the group-relative policy optimization algorithm used for RL training.","marker":"[96]"},{"why":"Supplies the judge model that scores descriptive answers for the model-based reward.","marker":"[97]"}],"fun_headline_variants":["RL on 120K diverse vision-language QAs lifts 7B model","Diverse RL beats math-only for multimodal reasoning","WeThink: 120K synthetic reasoning QAs boost 7B MLLM","Direct RL on diverse QAs improves 14 multimodal benchmarks","Question diversity, not verifiability, drives RL gains"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains are computed on evaluation benchmarks whose images and question types may overlap with the 18 training sources; if that overlap is substantial, the improvements could reflect memorization rather than general reasoning.","fun_headline_variants_meta":{"raw":{"variants":["RL on 120K diverse vision-language QAs lifts 7B model","Diverse RL beats math-only for multimodal reasoning","WeThink: 120K synthetic reasoning QAs boost 7B MLLM","Direct RL on diverse QAs improves 14 multimodal benchmarks","Question diversity, not verifiability, drives RL gains"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000269,"raw_usage":{"total_tokens":1614,"prompt_tokens":929,"completion_tokens":685,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":545,"completion_tokens_details":{"reasoning_tokens":596}},"tokens_in":545,"tokens_out":685,"duration_ms":7033,"temperature":1.0,"reasoning_tokens":596,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:23:25.922467+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute exact and near-duplicate overlap between every evaluation benchmark image and question and the 120K training set; if a substantial fraction of test items are duplicates or near-duplicates of training items, the reported average gains cannot be attributed to general multimodal reasoning.","supporting_citations":[],"review_version":1}