{"id":"44833fe5-c91d-4d36-a7e1-b6ccfa7b5758","arxiv_id":"2512.05530","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"MIND improves multimodal reasoning by training on diverse correct and deliberately wrong rationales with two-stage correction and contrastive alignment, reporting SOTA on ScienceQA, A-OKVQA, and M3CoT.","lead":"A training framework for multimodal language models that generates many correct and deliberately wrong reasoning chains and teaches the model to identify and fix errors, reporting state-of-the-art accuracy on three visual QA benchmarks. Worth a read because it is a concrete recipe for improving small models' reasoning, but the headline gains are partly confounded by a stronger base setup.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline 6.98-point ScienceQA gain over Multimodal-CoT is confounded with infrastructure changes (BLIP2 features, Qwen2.5-VL captions, training epochs); Table 4's controlled baseline already beats Multimodal-CoT by 4.98 points, so MIND's own contribution is about 2 points.","rationale":"The paper's contribution is a training framework, and its own ablations are the cleanest evidence. Table 4's baseline controls for visual encoder, captions, and epochs, so the 2.00-point gain (90.29→92.29) is the upper bound on what P2CL+MCA+RAD data add on ScienceQA. Yet the paper leads with a '6.98 point improvement over Multimodal-CoT' and claims SOTA; that number conflates framework contribution with infrastructure upgrades. Because the central claim is specifically about MIND's mechanism realizing large gains, this confound is load-bearing. If the controlled baseline had been reported as the comparison, the headline would be '2 points over a very strong baseline' rather than '6.98 points over Multimodal-CoT.' The concern is not that the framework is unsound — the ablation logic is internally consistent and the qualitative error-correction examples support the intended mechanism. It is that the comparison to prior work does not isolate the proposed components, exactly as the reader's weakest_assumption states. A single matched-baseline rerun of Multimodal-CoT would settle whether the large reported gaps are real or infrastructure-driven. I do not see an additional issue that would change the conditional verdict; no variance reporting is a secondary weakness but not needed to decide.","tokens_in":30899,"tokens_out":6509,"duration_ms":63717,"concrete_test":"Run the Multimodal-CoT training procedure on ScienceQA using exactly MIND's infrastructure — frozen BLIP2-flan-t5-xxl visual features, Qwen2.5-VL-72B captions, 200 epochs, batch size 8, FLAN-Alpaca initialization — with the original single-rationale supervision. If this controlled baseline lands near Table 4's 90.29 rather than 85.31, then the headline Table 1 improvement is mostly infrastructure; the defensible MIND-specific gain is the ~2.00-point difference in Table 4 (and the analogous ~4.7-point differences in Table S1). Report three seeds to establish whether the 0.07-point MCA-only gain is noise.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central SOTA claim is built on comparisons in Tables 1-3 against Multimodal-CoT (85.31 on ScienceQA, 50.6 on A-OKVQA, 44.85 on M3CoT). But Sec. 4.2 changes several variables at once: BLIP2-flan-t5-xxl visual features, Qwen2.5-VL-72B captions, and 200/400 training epochs. The paper's own controlled baseline (Table 4, also 'Original (21K)' in Table 6) uses the same infrastructure and original single-rationale data and reaches 90.29 on ScienceQA — already 4.98 points above the cited Multimodal-CoT number. The MIND components (RAD + P2CL + MCA) add 2.00 points over this baseline (Table 4). Thus the reported '6.98 improvement' is mostly an infrastructure effect, not an effect of MIND's proposed mechanisms. The same pattern appears in the supplementary Table S1: controlled baselines are 65.85 on A-OKVQA and 52.67 on M3CoT, already 15.25 and 7.82 points above the cited Multimodal-CoT baselines. This does not prove MIND is useless: the internal ablation shows a consistent 2-5 point gain over matched baselines and may support a weaker SOTA claim. But the headline improvement numbers and the '1+1>2' synergy claim (Eqs. 3/4/8, Tables 4/S1) are only validated for the matched-baseline increments; no variance/error bars are reported, and the MCA-only gain (0.07) is within the noise that a 3-seed run would likely reveal.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MIND, a training framework for small multimodal encoder-decoder models (T5-based, 223M/738M) that augments single-rationale supervision with multiple positive and negative rationales. RAD constructs multi-rationale datasets via LLM prompts; P2CL-I trains the model to generate answers and rationales from randomly sampled positive rationales; P2CL-II trains it to judge and correct a given positive or negative rationale; MCA adds a margin-based contrastive loss over hard positive and negative rationale embeddings. The paper reports state-of-the-art accuracy on ScienceQA (92.29%), A-OKVQA (70.6%), and M3CoT (57.38% base / 61.56% large), and ablations attributing gains to P2CL and MCA.","tokens_in":1575,"tokens_out":1602,"duration_ms":68831,"significance":"The core idea of training small MLLMs to learn from multiple positive rationales and to actively correct negative rationales is timely and potentially useful. The loss formulations in Eqs. (2)-(4) and (8) are clear, the ablations are structured, and the authors commit to releasing data and code. If the controlled gains (roughly 2-5 points over a matched baseline) hold up, this is a solid engineering contribution to reasoning training recipes. However, the paper's headline SOTA claims are not currently supported: the comparisons against Multimodal-CoT and other baselines change several infrastructure variables at once, and the paper's own matched baseline already accounts for most of the reported gap. The contribution is therefore a promising method with modest, partly confounded evidence, not an established SOTA result as written.","major_comments":[{"comment":"The headline improvements over Multimodal-CoT (6.98 points on ScienceQA, 20.0 on A-OKVQA, 12.53 on M3CoT) are confounded with infrastructure changes. Section 4.2 states that MIND uses BLIP2-flan-t5-xxl visual features, Qwen2.5-VL-72B captions, and 200-400 training epochs, whereas the cited Multimodal-CoT rows in Tables 1-3 do not use these settings. The paper's own controlled baseline in Table 4 (and 'Original (21K)' in Table 6) uses the same infrastructure with single-rationale data and already reaches 90.29 on ScienceQA, 4.98 points above the cited 85.31. Similarly, Table S1 shows controlled baselines of 65.85 on A-OKVQA and 52.67 on M3CoT, already 15.25 and 7.82 points above the cited Multimodal-CoT numbers. Thus the reported improvements in the main tables mostly reflect the changed backbone, captions, and training length, not the MIND components. Please report all baselines under ma","section":"§4.2, Tables 1-3 vs. Table 4/Table S1"},{"comment":"No variance or error bars are reported, and the '1+1>2' synergy claim is not statistically supported. On ScienceQA the MCA-only gain is 0.07% (90.29 to 90.36), which is within the noise expected from a single run; the combined gain is 2.00% versus a sum of individual gains of 1.93%. On A-OKVQA and M3CoT (Table S1), the combined gains (4.72 and 4.71) are actually smaller than the sums of the individual P2CL and MCA gains (6.46 and 6.69), so superadditivity is not observed outside ScienceQA. Please report multiple seeds with standard deviations and, if the synergy claim is retained, provide a statistical test or a consistent pattern across datasets.","section":"§4.4, Table 4 and Table S1"},{"comment":"The inference procedure is underspecified relative to training. P2CL-II is trained on rationales drawn from the RAD pools (gold positive and generated negative rationales), but at inference the input to the second stage is the model's own Phase I generated rationale. This train/inference distribution shift, model-generated rationales are likely noisier and more varied than RAD rationales, is not discussed. Please specify the exact inference protocol and, ideally, include model-generated rationales during training or validate that the RAD-trained correction behavior transfers to the model's own outputs.","section":"§3.3-§3.4 and Figure 1"},{"comment":"The quality of generated rationales is load-bearing for the method, but there is no human or automated audit of the RAD outputs. The paper states that rationales are filtered and cleaned after generation, but it does not report how many are rejected, whether negative rationales were checked to actually fail to derive the answer, or whether positive rationales preserve the original semantics. The downstream accuracy comparison in Table 5 is an indirect, partly circular proxy for rationale quality because the same model's final accuracy is used to validate the data. Please add a sample-quality evaluation (e.g., human annotation on a random subset, or an automated entailment/consistency check) and report agreement numbers.","section":"§3.2, RAD paradigm"}],"minor_comments":[{"comment":"There is a typo: the second equation should select Top-k from the negative similarities {s_j^-}, not from {s_i^+}. As written both lines use {s_i^+}, which makes the hard-negative mining step undefined.","section":"Eq. (7)"},{"comment":"The sentence ends with 'forming a stable.' which is incomplete. Please complete it.","section":"§3.4, last paragraph"},{"comment":"Minor formatting issues: 'LLaMA-Adaper' should be 'LLaMA-Adapter'; category names are inconsistently spaced (e.g., 'M 3CoT'), and the table could be simplified for readability.","section":"Table 3"},{"comment":"The initialization is described as 'FLAN-Alpaca weights' but the citation [7] is to the FLAN paper. If the Alpaca variant is used, please cite the corresponding source or clarify the checkpoint.","section":"§4.2"},{"comment":"The notation 'MINDbase' versus 'MIND base' is used inconsistently; please unify for clarity.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, quick take on the MIND paper. The headline SOTA numbers aren't trustworthy, but the core recipe is real: training a T5-based MLLM on multiple positive and negative rationales, with a two-stage correction objective and a hard-negative contrastive loss, does buy 2–5 points over a properly matched baseline. That part deserves attention.\n\nWhat's new: the RAD paradigm generates diverse positive rationales and semantically inverted negatives in batches, the P2CL strategy separates positive understanding from error correction, and the MCA loss does hard-positive/negative mining with a margin. None of these pieces are individually novel, but the combination—especially feeding negative rationales as inputs and supervising with corrected rationales—is a legitimate new recipe. The paper is honest enough to include the controlled baseline in Table 4: 90.29% on ScienceQA with the same BLIP2 features, Qwen captions, and 200 epochs, using the original single-rationale data. MIND adds 2.00 points on top of that. Same story in Table S1: 2.18/4.28/4.72 on A-OKVQA and 3.02/3.67/4.71 on M3CoT for MCA/P2CL/full. Those are the credible gains.\n\nThe 6.98-point claim against the published Multimodal-CoT number (85.31) is mostly the infrastructure delta: different visual encoder, better captions, 200 epochs instead of the original training schedule. The paper itself shows epochs and captions moving accuracy by several points (Fig. 4). So the SOTA comparisons in Tables 1–3 should be re-run under matched settings before claiming superiority.\n\nOther soft spots: no variance or error bars anywhere. MCA alone moves ScienceQA by 0.07, which is noise-level; the '1+1>2' synergy is 0.14 over P2CL alone, also likely noise. On A-OKVQA and M3CoT the MCA-on-top gain is larger (0.44, 1.04), so MCA may help on harder open-ended tasks, but that needs showing. The RAD negative rationales are machine-generated with no human quality audit; it's plausible that flipped rationales are sometimes not actually misleading or are nonsensical, and the paper's quality validation is just end-task accuracy, which is circular. Also the abstract says 'paradigm evolution from passive imitation to active discriminative reasoning'—too grand for the actual scope, a fine-tuning recipe.\n\nThe paper is well-organized, losses are clearly defined, related work is decent, and code/data are promised. For people working on small-model CoT, distillation, or rationale-based training, the matched-baseline result is useful and reproducible. It deserves a serious referee: not because the SOTA claim is right, but because the underlying approach is plausible and the authors have included enough controls that a careful review can separate wheat from chaff.\n\nMy advice: send it to review, but ask for a revision that (1) re-baselines all comparisons under the same infrastructure, (2) reports variance across seeds, and (3) either shows a human audit of negative rationales or rewrites the claims around the matched-baseline gains.","headline":"The headline SOTA numbers mostly reflect infrastructure changes, but the matched-baseline ablations show a real 2–5 point gain from training on multiple positive/negative rationales.","tokens_in":31865,"tokens_out":2846,"would_cite":true,"duration_ms":29449,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The MIND framework trains multimodal models to reason from many correct and deliberately wrong rationales, and reports state-of-the-art accuracy on three visual question answering benchmarks.","keywords":["multi-rationale reasoning","chain-of-thought","rationale augmentation","contrastive alignment","self-correction","visual question answering","ScienceQA","A-OKVQA"],"falsifier":"Run MIND and the single-rationale CoT baseline under identical conditions — same visual encoder, same image captions, same number of training epochs, same rationale scale — and compare. If the multi-rationale components add little or nothing once infrastructure is matched, the central SOTA claim fails; if the gap persists, the multi-rationale discrimination mechanism is the cause.","tokens_in":30822,"feed_emoji":"🧠","tokens_out":6150,"duration_ms":57897,"temperature":0.7,"pith_summary":"The paper tries to show that a multimodal language model can reason better when it is trained not on a single chain of thought but on many diverse correct rationales plus deliberately wrong ones, and when it is explicitly taught to spot and fix the wrong ones. To that end it introduces MIND, a two-stage training scheme: first learn the common logic across many positively rephrased rationales; then, given either a positive or a negative rationale, produce the correct answer together with a corrected rationale. A contrastive loss with hard-example mining pulls the embeddings of correct rationales together and pushes incorrect ones apart. The paper reports state-of-the-art accuracies on three visual question answering benchmarks — 92.29% on ScienceQA, 70.6% on A-OKVQA, and 57.38% (base) / 61.56% (large) on M3CoT — with the P2CL and MCA components showing a combined '1+1>2' gain. A sympathetic reader would care because this is a step from learning to imitate a single reasoning path toward learning to discriminate and self-correct, which is closer to how humans actually reason.","feed_headline":"Training on good and bad rationales lifts VQA to 92%","feed_subtitle":"Two-stage correction learning plus contrastive alignment lets a 223M model beat far larger baselines on three benchmarks.","key_machinery":"The load-bearing objects are the three components of MIND. RAD (Rationale Augmentation and Discrimination) is a data-construction step: prompted large models rewrite each original rationale into several semantically equivalent positive versions and several semantically inverted negative versions, producing 'Multi-Rationale-Pos' and 'Multi-Rationale-Neg' pools. P2CL (Progressive Two-stage Correction Learning) is the training curriculum: Phase I teaches multi-rationale positive learning; Phase II teaches active logic discrimination and correction by conditioning on positive or negative rationales and supervising with the correct answer plus a positive rationale. MCA (Multi-rationale Contrastiv","core_discovery":"On its own terms, the central claim is that a multi-rationale semantic space, built by RAD, can be shaped by P2CL and MCA so that the model learns both semantic consistency of correct reasoning and the boundary that separates it from incorrect reasoning. The training signal is: in Phase I, randomly sample from a pool of positive rationales and optimize a maximum-likelihood generation loss; in Phase II, take a positive or negative rationale as input and supervise the output [answer + corrected positive rationale]. MCA regularizes the same embeddings with a margin-based contrastive loss over hard positives and hard negatives. The reported result is that this combination outperforms single-rati","pith_inferences":["Editorial inference: Because the paper does not audit the correctness or quality of the RAD-generated rationales, a key open question is how the method behaves when the 'positive' pool contains noisy or actually wrong rationales; the contrastive hard-mining might be sensitive to label noise in the rationale space.","Editorial inference: The reported gains may partly reflect the upgraded infrastructure (stronger visual features, better captions, longer training) rather than the multi-rationale signal itself; the paper does not run MIND under the baseline's exact settings. A cleaner test would control those variables.","Editorial inference: The RAD generation recipe is model-agnostic, so the same prompt-driven positive/negative expansion could be applied to other chain-of-thought datasets beyond VQA, e.g., text-only math or commonsense reasoning, potentially giving similar correction ability.","Editorial inference: If the 'understand-rethink-correct' curriculum is the real driver, one would expect stronger gains on questions where the baseline's rationale is confidently wrong and the image provides a disambiguating cue; this can be tested by stratifying results by rationale correctness."],"forward_implications":["If MIND works as claimed, a dataset with one gold rationale can be turned into a 500–1000x larger training resource through LLM-based paraphrasing and semantic inversion, so multi-rationale supervision becomes a cheap add-on to existing CoT benchmarks.","The two-phase design implies that a model can learn to detect a flawed rationale and then generate a corrected one, which is a path toward verifiable reasoning rather than answer-only mimicry.","The reported 1+1>2 effect for P2CL and MCA implies that contrastive alignment is most useful when it regularizes an already-strong generative rationale-learning objective, not as a standalone method.","The gains at 223M/738M parameters suggest that small models can close part of the gap with much larger zero-shot MLLMs by learning to discriminate among rationales.","The saturation of gains beyond ~1000x rationale expansion suggests a practical ceiling: more rationales help, but with diminishing returns."],"fun_headline_variants":["MIND framework teaches MLLMs to judge and correct rationales","Training on good and bad rationales lifts multimodal reasoning","Two-step correction learning sharpens VQA and other benchmarks","MIND: from imitation to discrimination in multimodal reasoning"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the reported accuracy gaps come from MIND's multi-rationale training rather than from the stronger visual features, richer captions, and 200–400 training epochs used in the MIND runs; the paper's own Table 4 baseline (90.29) is already 4.98 points above the single-rationale baseline it cites (85.31), so the comparison does not isolate the new components.","fun_headline_variants_meta":{"raw":{"variants":["MIND framework teaches MLLMs to judge and correct rationales","Training on good and bad rationales lifts multimodal reasoning","Two-step correction learning sharpens VQA and other benchmarks","MIND: from imitation to discrimination in multimodal reasoning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000244,"raw_usage":{"total_tokens":1368,"prompt_tokens":742,"completion_tokens":626,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":486,"completion_tokens_details":{"reasoning_tokens":568}},"tokens_in":486,"tokens_out":626,"duration_ms":7111,"temperature":1.0,"reasoning_tokens":568,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T18:20:42.201795+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run MIND and the single-rationale CoT baseline under identical conditions — same visual encoder, same image captions, same number of training epochs, same rationale scale — and compare. If the multi-rationale components add little or nothing once infrastructure is matched, the central SOTA claim fails; if the gap persists, the multi-rationale discrimination mechanism is the cause.","supporting_citations":[],"review_version":1}