{"id":"bcaba6c6-d4c4-419a-8e1e-00ec57c3147c","arxiv_id":"2507.08306","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"M2-Reasoning-7B reports state-of-the-art average scores across eight multimodal reasoning benchmarks using synthetic spatial data and dynamic multi-task RLVR.","lead":"Researchers built M2-Reasoning-7B, a 7-billion-parameter multimodal model trained with a large filtered dataset of math and spatial questions plus a reinforcement-learning stage, and they report top average scores on eight reasoning benchmarks. The paper matters because it shows how synthetic spatial data and task-specific rewards can be combined with general reasoning training in one model.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"VSI-Bench SOTA claim rests on an incomparable baseline: InternVL3-8B's Table 3 numbers are copied from its report, not evaluated in M2's harness, and the 0.2-point margin is within noise.","rationale":"Good-faith reading: the authors release the model and code, report consistent gains over their own base model, and describe a concrete data pipeline; I do not think the training claim is broken or fraudulent. The immediate stress point is not contamination (which is also untested) but comparability of the spatial benchmark numbers. The general-reasoning leaderboard scores come from OpenCompass and are more likely comparable, but the VSI-Bench column for InternVL3-8B is imported from a different report. Since the abstract's headline is 'SOTA across 8 benchmarks,' and one of the two spatial pillars is a 0.2-point margin on VSI-Bench, this is the most load-bearing unverified comparison. It is fixable by running both models through one harness, so the conditional verdict stands. If the unified-harness test confirms the numbers, I would accept; if it flips them, the paper should be revised to drop the VSI-Bench SOTA claim.","tokens_in":20071,"tokens_out":7195,"duration_ms":92054,"concrete_test":"Use the released M2-Reasoning-7B checkpoint and the official VSI-Bench evaluation code to score both M2-Reasoning-7B and InternVL3-8B under identical conditions: same system prompt (or a neutral prompt for both), same answer extraction, same greedy decoding, and the same metric scripts. Compute per-subtask and average scores, plus bootstrap 95% confidence intervals over benchmark items. If InternVL3-8B's average is ≥42.3 or the intervals overlap 42.3, the claimed VSI-Bench SOTA is not supported. A secondary check: rerun CV-Bench under the same harness to confirm the 82.3 vs 82.0 margin.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The linchpin of the abstract's spatial-SOTA claim is Table 3: M2-Reasoning-7B's VSI-Bench average (42.3) edges InternVL3-8B (42.1). But in the table all InternVL3-8B entries are marked †, i.e. copied from the InternVL3 technical report, while M2-Reasoning-7B is evaluated with its own trained system prompt and output-format requirements (Appendix B.1). VSI-Bench contains eight subtasks with distinct answer schemas (counts, distances, room sizes, route plans, appearance orders), so results are sensitive to prompt wording, answer extraction, and scoring. The paper reports no unified evaluation harness for VSI-Bench and no error bars or item-level bootstrap. A 0.2-point average gap on an eight-subtask benchmark is not beyond evaluation noise, especially when the per-task pattern is lopsided: M2 trails InternVL3-8B on Object Count (41.0 vs 68.1) and Appearance Order (28.8 vs 35.4) but leads on Room Size (55.4 vs 33.6) and Relative Direction (47.3 vs 36.4), exactly the subtasks the spatial RLVR curriculum emphasizes. Additionally, no base-model row is given for VSI-Bench, so the spatial gains cannot be separated from the cold-start SFT stage. The spatial SOTA claim therefore rests on a cross-paper comparison that has not been shown to be apples-to-apples.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces M2-Reasoning-7B, a 7B multimodal LLM built on Qwen2.5-7B and the M2-Omni framework, trained in two stages: a cold-start SFT stage on a 6.2M-sample mixture (with 168K curated CoT samples) and a GRPO-style RLVR stage on 126.2K prompts (100K general, 18.7K spatial image, 7.5K spatial video). The method includes a difficulty-scored curriculum, dynamic advantage weighting, cosine-annealed KL, and a smooth EDNM reward for numeric spatial outputs. Evaluations on six general benchmarks plus CV-Bench and VSI-Bench report a general-reasoning average of 45.0 (up from a 35.5 base), a CV-Bench average of 82.3, and a VSI-Bench average of 42.3. The paper claims these results set a new SOTA across all eight benchmarks.","tokens_in":20471,"tokens_out":7388,"duration_ms":75574,"significance":"If the empirical claims hold under a unified evaluation protocol, the paper is a useful contribution: it releases model weights and code, gives a clearly specified data-synthesis and filtering pipeline, and demonstrates a substantial +9.5 average gain over the base model on the general-reasoning suite. The dynamic multi-task RLVR recipe and the EDNM reward are concrete, reproducible design choices. However, the spatial SOTA claims currently rest on evaluation-rigor weaknesses, especially the cross-paper VSI-Bench comparison and the absence of base-model or ablation rows for spatial benchmarks. The core training idea is plausible, but its significance cannot be fully assessed until those comparisons are made apples-to-apples and the individual contributions are separated.","major_comments":[{"comment":"The abstract and conclusion claim a 'new state-of-the-art across 8 benchmarks,' but the per-benchmark tables do not support that reading. In Table 1, M2-Reasoning-7B is not the best on MathVision (31.5 vs URSA-8B-PS-GRPO's 31.8), LogicVista (50.0 vs WeThink-VL-7B's 51.2), or WeMath (41.8 vs WeThink-VL-7B's 48.0). In Table 3, it is second to Gemini-1.5-pro on the VSI-Bench average (42.3 vs 45.4). If 'SOTA across 8 benchmarks' means the best average on the six-benchmark general suite plus best base-scale CV-Bench average, the authors should state that explicitly and soften the per-benchmark implication.","section":"Abstract; §4.1, Table 1; §4.2, Table 3"},{"comment":"The VSI-Bench comparison is not apples-to-apples. All InternVL3-8B entries are marked †, i.e., copied from the InternVL3 technical report, while M2-Reasoning-7B is evaluated with its own trained system prompt and output-format requirements. VSI-Bench contains eight subtasks with distinct answer schemas, so scores are sensitive to prompt wording, answer extraction, and scoring. No error bars or item-level variance are reported, yet the average gap is only 0.2 points. The per-task pattern is lopsided (M2 trails on Object Count 41.0 vs 68.1 and Appearance Order 28.8 vs 35.4, while leading on Room Size 55.4 vs 33.6 and Relative Direction 47.3 vs 36.4), which is exactly the set of skills the spatial RLVR curriculum emphasizes. The claim that M2 'surpasses' InternVL3-8B on VSI-Bench is therefore not supported; the authors should rerun InternVL3-8B in their own harness or weaken the claim to 'competitive.'","section":"§4.2, Table 3; Appendix B.1"},{"comment":"No base-model or cold-start-only rows are reported for CV-Bench or VSI-Bench. Table 1 separates Base Model, M2-Reasoning-CI-7B, and M2-Reasoning-7B, but the spatial tables report only the final model. Without these rows, the claimed spatial gains cannot be attributed to the RLVR stage or to the task-specific reward; they could come from the cold-start SFT or from the change of backbone. Please add base and cold-start evaluations for both spatial benchmarks.","section":"§4.2, Tables 2 and 3"},{"comment":"The spatial training data are generated from depth maps, segmentation, and simulated videos, while CV-Bench and VSI-Bench contain real images and videos. The paper does not report any overlap check between the training samples and the evaluation benchmarks, nor any measure of the train-to-test domain gap. Given that the reported margins are thin on CV-Bench (82.3 vs 82.0) and VSI-Bench (42.3 vs 42.1), the possibility of reward overfitting or unintentional distribution overlap should be addressed, for example by reporting per-task generalization curves or by evaluating on a held-out set drawn from a different distribution.","section":"§2.2; §4.2"},{"comment":"The EDNM reward is presented as central to spatial reasoning, but the manuscript contains no sensitivity analysis or ablation of this component. The hyperparameters γ=1 and λ=2 are fixed without supporting experiments, and Table 4 does not include a variant with a binary exact-match reward or different λ values. Similarly, the dynamic advantage scale σ=7.2 in Eq. (6) and the per-task epochs are not varied. Adding ablations would substantiate the claim that the task-specific reward and dynamic weighting, rather than the larger cold-start dataset or the backbone change, are responsible for the observed gains.","section":"§3.2, Eq. (9); Table 4"}],"minor_comments":[{"comment":"There are typos and duplicated phrases: 'diffucilty' in Section 1, 'serves as an automated evaluator for response correctness serves as an automated evaluator for response correctness' in Section 2.1.1, 'Genral' in Figure 2, and 'Visulization' in Appendix C.","section":"§1; §2.1.1"},{"comment":"The row labeled 'Base Model' is not defined in the text; please clarify whether it is M2-Omni with the Qwen2.5-7B language model before any reasoning-specific training, and report its evaluation setup.","section":"§4.1, Table 1"},{"comment":"InternVL3-8B* is marked as 'reproduced by us,' but the reproduction settings (prompt, number of samples, answer parsing) are not described; without this information the comparison has the same ambiguity as the VSI-Bench cross-paper comparison, only at a smaller scale.","section":"Table 2"},{"comment":"The curriculum sampling is described qualitatively as 'increasing order of difficulty,' but the operationalization is not given; please specify how the offline difficulty scores are converted into training order (e.g., buckets, thresholds, or full ordering) and how this interacts with the per-task epoch counts in Table 4.","section":"§3.1.2; Appendix B.2"},{"comment":"The paper reports that prompts with 0% or 100% response accuracy are excluded from RLVR, but it does not report the number of prompts removed at each stage or the final distribution of difficulty scores; reporting these flow statistics would strengthen the data-pipeline description.","section":"§2.1.2; §2.3"},{"comment":"The phrase 'surpasses the strong InternVL3-8B baseline (42.1)' in the VSI-Bench discussion should be revised to reflect the cross-paper nature of the comparison, regardless of the outcome of the recommended rerun.","section":"§4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and the authors have released code and weights, which is commendable. I see no indication of misconduct; the concerns are about evaluation rigor and claim calibration. The central issue is that the spatial SOTA claims depend on a 0.2-point cross-paper VSI-Bench margin and on comparisons that are not made in a shared harness. A major revision that adds base/ablation rows, reruns the key baselines under the same protocol, and softens the SOTA wording would make the paper reliable. I would be willing to review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the training recipe is plausible and the data pipeline is the best part, but the paper calls 'SOTA across 8 benchmarks' when the tables show several second places and two margins under a point. The VSI-Bench claim specifically relies on copying InternVL3-8B's numbers from its report rather than running them in the same harness, so the 0.2-point edge is not something I'd hang a headline on.\n\nWhat's genuinely new here is the package, not the pieces: filtering CoT with a quality rubric, generating spatial QA from depth/segmentation on real images plus simulated video, and then running GRPO with a smooth numeric reward (EDNM) instead of a hard match. EDNM is a sensible answer to the sparse-reward problem for absolute distances and sizes, and the description of the filtering criteria (e.g., removing ambiguous object pairs) shows real care. The cold-start plus RLVR progression is standard by now, but the 9.5-point gain over the base model on the general-reasoning suite is consistent with the recipe working. The limitations paragraph about repetition and shallow reasoning is more honest than most.\n\nThe soft spots are the usual ones for this kind of paper, and they're addressable. First, the SOTA wording doesn't survive contact with Table 1: MathVision and LogicVista are second-place results, and the 'average SOTA' is doing all the work. Second, no ablations. The dynamic hyper-parameter adjustment, curriculum sampling, and task-specific rewards are each plausible, but nothing isolates their contribution. Third, no error bars, and the VSI-Bench numbers have no base-model row, so the spatial gains cannot be separated from the cold-start SFT stage. Fourth—and this is the one that matters most—the spatial transfer assumption is unmeasured. The synthetic data come from depth/segmentation on real images and from simulations, and the paper never checks for distribution overlap with CV-Bench or VSI-Bench or tests for leakage. The per-subtask pattern on VSI-Bench (leading on Room Size and Relative Direction, trailing on Object Count and Appearance Order) is exactly what you'd expect from the RLVR curriculum, which is fine in itself, but it doesn't make the cross-paper comparison any stronger.\n\nFor a reader: this is useful for people working on multimodal RLVR and spatial reasoning data. I'd want to see ablations and a leakage check before trusting the method-level claims, but the data pipeline is worth reading now. It deserves peer review, with the expectation of substantial revision.","headline":"Plausible RLVR recipe and a thoughtful data pipeline, but the 'SOTA across 8 benchmarks' label is overstated and the VSI-Bench claim rests on a cross-paper comparison under 0.3 points.","tokens_in":21022,"tokens_out":2894,"would_cite":true,"duration_ms":29133,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"M2-Reasoning-7B reports state-of-the-art scores on eight multimodal benchmarks by pairing a 294.2K-sample data pipeline with dynamic multi-task RLVR and task-specific rewards.","keywords":["multimodal large language models","spatial reasoning","general reasoning","reinforcement learning with verifiable rewards","GRPO","chain-of-thought data synthesis","data curation","curriculum learning"],"falsifier":"Train or run the identical recipe but drop the 26.2K synthetic spatial RLVR samples (or replace them with spatial questions sampled from the benchmarks' own source data), and compare CV-Bench and VSI-Bench scores; if the 82.3 and 42.3 averages do not fall materially, the claimed spatial advantage is not caused by the synthetic data pipeline. Alternatively, test for overlap by asking whether any generated training question shares objects, scenes, or templated wording with CV-Bench or VSI-Bench items.","tokens_in":19887,"feed_emoji":"🧠","tokens_out":7516,"duration_ms":68759,"temperature":0.7,"pith_summary":"The paper claims that a 7-billion-parameter multimodal model, M2-Reasoning-7B, can simultaneously reach state of the art in two domains that are usually trained separately: abstract general reasoning and dynamic spatial reasoning. The recipe is a large, quality-filtered data pipeline — 168K cold-start chain-of-thought samples plus 126.2K reinforcement-learning prompts — followed by a GRPO variant with curriculum sampling, difficulty-based sample weighting, a cosine-annealed KL penalty, and rewards tailored to each task family. The central empirical claim is that this combination raises the model's average on six general-reasoning benchmarks to 45.0, a 9.5-point gain over its base, while setting the best base-scale average on CV-Bench (82.3) and a VSI-Bench average (42.3) that edges past InternVL3-8B. The significance would be that spatial interaction, a known weak point of multimodal reasoners, can be bootstrapped from synthetic annotations without sacrificing general math and logic performance.","feed_headline":"294K curated samples push a 7B MLLM to SOTA on 8 benchmarks","feed_subtitle":"The recipe lifts the general-reasoning average by 9.5 points and tops CV-Bench and VSI-Bench among base-scale models.","key_machinery":"The paper's central machinery is a two-stage training pipeline anchored by two components. First, a data-construction pipeline turns public image-text and simulation data into 168K cold-start chain-of-thought samples (filtered by answer correctness plus a three-axis quality score) and 126.2K RLVR prompts (100K general, 18.7K spatial image, 7.5K spatial video), with RLVR prompts scored for difficulty as $1 - \\text{accuracy}$. Second, a modified GRPO objective adds dynamic advantage weighting ($\\sigma \\cdot \\text{mean\\_acc} \\cdot (1 - \\text{mean\\_acc})$, $\\sigma = 7.2$), a cosine-annealed KL penalty coefficient, curriculum sampling ordered by difficulty, and task-specific accuracy rewards — rule-based exact matching for general questions, and the Exponential Decay Numeric Matching (EDNM) reward $R_{\\text{EDNM}}(x) = \\gamma \\cdot \\exp(-\\lambda |x - x_{gt}| / (|x_{gt}| + \\epsilon))$ with $\\gamma = 1$, $\\lambda = 2$ for spatial numeric answers. The EDNM reward is the named object that lets spatial fill-in-the-blank questions contribute smooth gradients when exact matches are rare.","core_discovery":"The central discovery claimed is that a unified MLLM can be made strong in both general reasoning and spatial reasoning by separating the two-stage reasoning recipe (cold-start SFT then RLVR) from the data and reward design. The authors report that M2-Reasoning-7B sets new state-of-the-art results across the eight evaluated benchmarks: a 45.0 average on six general-reasoning benchmarks (MathVista 75.0, MathVision 31.5, MathVerse 44.7, DynaMath 26.8, WeMath 41.8, LogicVista 50.0), an 82.3 average on CV-Bench, and a 42.3 average on VSI-Bench. They attribute the gains to two things: a data pipeline that synthesizes logically coherent reasoning traces, evaluates them for correctness, structure, cognitive load, and verification richness, and pre-scores RLVR prompts by difficulty; and a dynamic multi-task training strategy where each batch contains one task, steps are distributed uniformly, moderate-difficulty samples receive higher advantage weights, the KL penalty is annealed with cosine scheduling, and spatial numeric questions receive an exponential-decay reward instead of a binary match.","pith_inferences":["If the synthetic-to-real transfer assumption holds, the same depth-and-segmentation-driven QA recipe could extend to other spatial tasks the paper does not test, such as 3D layout estimation, path planning, or motion prediction.","The reported CV-Bench advantage over InternVL3-8B is 0.3 points and the VSI-Bench advantage is 0.2 points; without a leakage or domain-gap check, those margins are within the range where distribution overlap or reward overfitting could explain them.","The EDNM reward is a generic continuous-scoring mechanism; it could in principle be applied to other continuous-output perception tasks such as depth estimation or bounding-box regression, a transfer the paper does not explore.","Because the paper excludes prompts at 0% and 100% response accuracy, the final RLVR set is concentrated on mid-difficulty items; the interaction between this filtering threshold and the dynamic advantage weighting is not ablated."],"forward_implications":["A 7B MLLM can beat larger or dedicated reasoning models on general multimodal reasoning; the paper reports the best average (45.0) on the six-benchmark suite, with top scores on MathVista (75.0) and DynaMath (26.8).","Spatial reasoning in MLLMs can be improved substantially with synthetic annotations: the model reaches 82.3 on CV-Bench (best base-scale average) and 42.3 on VSI-Bench, surpassing InternVL3-8B in both averages.","Task-specific reward design matters: continuous EDNM rewards, rather than exact-match rewards, are what the paper credits with letting spatial numeric questions train effectively.","Dynamic, difficulty-aware GRPO — curriculum sampling, moderate-difficulty advantage weighting, and cosine-annealed KL — contributes the remaining gains over the cold-start model (42.9 to 45.0).","Data quality filtering, not just quantity, drives the result: the 168K cold-start chains were filtered by correctness and a three-axis reasoning-quality score before training."],"supporting_citations":[{"why":"Supplies the base model framework and the non-thinking instruction data the model is initialized from.","marker":"(Guo et al., 2025c)"},{"why":"Provides the cold-start general-reasoning image-text data foundation.","marker":"(Yang et al., 2025d)"},{"why":"Provides the text-only cold-start data used alongside image-text samples.","marker":"(Team, 2025)"},{"why":"Used for difficulty scoring of RLVR prompts and appears as a comparison baseline in the evaluation tables.","marker":"(Bai et al., 2025)"},{"why":"Generates the synthesized chain-of-thought samples and serves as a reasoning-model baseline.","marker":"(Yang et al., 2025b)"},{"why":"Establishes the two-stage SFT plus RLVR paradigm the paper adapts to multimodal reasoning.","marker":"(Guo et al., 2025a)"},{"why":"Provides the base GRPO algorithm that the dynamic multi-task training modifies.","marker":"(Shao et al., 2024)"},{"why":"Motivates the reasoning-quality scoring criteria used to filter the cold-start data.","marker":"(Ye et al., 2025)"}],"fun_headline_variants":["294K samples forge M2-Reasoning-7B, SOTA on 8 benchmarks","M2-Reasoning-7B: dynamic training, unified reasoning, 8-benchmark SOTA","Two-stage recipe (SFT+RLVR) makes M2-Reasoning-7B SOTA on 8","Unified general + spatial reasoning: M2-Reasoning-7B hits SOTA on 8","With 294K curated examples, M2-Reasoning-7B leads 8 benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The spatial gains rest on the assumption that spatial questions synthesized from depth and segmentation on real images and from simulated videos transfer to the real-image and real-video statistics of CV-Bench and VSI-Bench — and that none of those generated questions overlap with the benchmark tests, which the paper does not measure or check.","fun_headline_variants_meta":{"raw":{"variants":["294K samples forge M2-Reasoning-7B, SOTA on 8 benchmarks","M2-Reasoning-7B: dynamic training, unified reasoning, 8-benchmark SOTA","Two-stage recipe (SFT+RLVR) makes M2-Reasoning-7B SOTA on 8","Unified general + spatial reasoning: M2-Reasoning-7B hits SOTA on 8","With 294K curated examples, M2-Reasoning-7B leads 8 benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001257,"raw_usage":{"total_tokens":5174,"prompt_tokens":996,"completion_tokens":4178,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":612,"completion_tokens_details":{"reasoning_tokens":4047}},"tokens_in":612,"tokens_out":4178,"duration_ms":26356,"temperature":1.0,"reasoning_tokens":4047,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:24:02.298920+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train or run the identical recipe but drop the 26.2K synthetic spatial RLVR samples (or replace them with spatial questions sampled from the benchmarks' own source data), and compare CV-Bench and VSI-Bench scores; if the 82.3 and 42.3 averages do not fall materially, the claimed spatial advantage is not caused by the synthetic data pipeline. Alternatively, test for overlap by asking whether any generated training question shares objects, scenes, or templated wording with CV-Bench or VSI-Bench items.","supporting_citations":[],"review_version":1}