Pith. sign in

REVIEW 25 cited by

MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2412.05237 v2 pith:C5JEH547 submitted 2024-12-06 cs.CL cs.CV

Jarvis Guo , Tuney Zheng , Yuelin Bai , Bo Li , Yubo Wang , King Zhu , Yizhi Li , Graham Neubig
show 2 more authors
Wenhu Chen Xiang Yue
This is my paper
classification cs.CLcs.CV
keywords datasetmultimodalreasoningdatasetsrationalestasksbenchmarkscapabilities
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Open-source multimodal large language models (MLLMs) have shown significant potential in a broad range of multimodal tasks. However, their reasoning capabilities remain constrained by existing instruction-tuning datasets, which were predominately repurposed from academic datasets such as VQA, AI2D, and ChartQA. These datasets target simplistic tasks, and only provide phrase-level answers without any intermediate rationales. To address these challenges, we introduce a scalable and cost-effective method to construct a large-scale multimodal instruction-tuning dataset with rich intermediate rationales designed to elicit CoT reasoning. Using only open models, we create a dataset containing 12M instruction-response pairs to cover diverse, reasoning-intensive tasks with detailed and faithful rationales. Experiments demonstrate that training MLLMs on this dataset significantly improves reasoning capabilities, achieving state-of-the-art performance on benchmarks such as MathVerse (+8.1%), MMMU-Pro (+7%), and MuirBench (+13.3%). Additionally, the model demonstrates notable improvements of up to 4% on non-reasoning-based benchmarks. Ablation studies further highlight the importance of key components, such as rewriting and self-filtering, in the dataset construction process.

Discussion (0). Sign in to comment.

Forward citations

Cited by 25 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Seeing the Scene Matters: Revealing Forgetting in Video Understanding Models with a Scene-Aware Long-Video Benchmark

    cs.CV 2026-03 unverdicted novelty 7.0 of 10

    SceneBench shows VLMs lose accuracy on scene-level questions in long videos due to forgetting, and Scene-RAG retrieval improves performance by 2.5%.

  2. AIA: Rethinking Architecture Decoupling Strategy In Unified Multimodal Model

    cs.CV 2025-11 unverdicted novelty 7.0 of 10

    AIA loss teaches unified multimodal models task-specific cross-modal attention patterns to reduce conflicts between image understanding and generation without architecture decoupling.

  3. Brevity is the Soul of Inference Efficiency: Inducing Concision in VLMs via Data Curation

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    Curating pretraining data for concision in VLMs produces models with up to 35x lower cost-of-pass at matched accuracy by reducing output token count.

  4. Brevity is the Soul of Inference Efficiency: Inducing Concision in VLMs via Data Curation

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    Curating concise data for VLMs induces brevity, delivering 35x lower Cost-of-Pass at near-identical accuracy and higher matched-length accuracy than uncurated baselines.

  5. HYDRA-X: Native Unified Multimodal Models with Holistic Visual Tokenizers

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    HYDRA-X presents the first unified multimodal model using a single ViT for holistic image-video tokenization, with ablations on attention and compression plus a latent-level editing improvement.

  6. Zamba2-VL Technical Report

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    Zamba2-VL is a family of 1.2B–7B hybrid Mamba2-transformer vision-language models that match leading transformer VLMs on image, reasoning, OCR, grounding and counting benchmarks while delivering roughly 10x lower time...

  7. OProver: A Unified Framework for Agentic Formal Theorem Proving

    cs.CL 2026-05 unverdicted novelty 6.0 of 10

    OProver-32B achieves top Pass@32 scores on MiniF2F, ProverBench, and PutnamBench by combining continued pretraining with iterative agentic proving, retrieval, SFT on repairs, and RL on unresolved cases using a 6.86M-p...

  8. 20/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone

    cs.LG 2026-05 conditional novelty 6.0 of 10

    Data curation alone raises VLM accuracy by more than 11 points on average across many benchmarks while cutting required training compute by up to 87 times.

  9. 20/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone

    cs.LG 2026-05 unverdicted novelty 6.0 of 10

    Data curation alone raises VLM accuracy by 11+ points on average, improves reliability and OOD generalization, and achieves near-frontier results at far lower training and inference cost.

  10. Meta-CoT: Enhancing Granularity and Generalization in Image Editing

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    Meta-CoT uses two-level decomposition of editing operations into meta-tasks and a CoT consistency reward to improve granularity and generalization, reporting 15.8% gains across 21 tasks.

  11. Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Models

    cs.CV 2025-12 conditional novelty 6.0 of 10

    Sparse-LaViDa speeds up masked diffusion LLMs by truncating masked tokens, using register tokens and a step-causal attention mask, reaching ~2-2.8x speedup over LaViDa-O with comparable generation quality.

  12. Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation

    cs.CV 2025-09 conditional novelty 6.0 of 10

    Lavida-O introduces an elastic mixture-of-transformers architecture that brings high-resolution text-to-image generation, object grounding, and image editing into a single masked diffusion model, using planning and se...

  13. LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning

    cs.LG 2025-05 conditional novelty 6.0 of 10

    LLaDA-V is a diffusion-based multimodal large language model that reaches competitive or state-of-the-art results on visual instruction tasks while using a non-autoregressive architecture.

  14. Circle-RoPE: Cone-like Decoupled Rotary Positional Embedding for Large Vision-Language Models

    cs.CV 2025-05 unverdicted novelty 6.0 of 10

    Circle-RoPE achieves cross-modal positional disentanglement in VLMs by mapping 2D image tokens to a cone-like annulus orthogonal to the text axis, with PTD=0 eliminating RoPE geometric bias while preserving intra-imag...

  15. SmolVLM: Redefining small and efficient multimodal models

    cs.AI 2025-04 unverdicted novelty 6.0 of 10

    SmolVLM-256M outperforms a 300-times larger model using under 1 GB GPU memory, while the 2.2B version matches state-of-the-art VLMs at half the memory cost.

  16. OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cycles

    cs.CV 2025-03 conditional novelty 6.0 of 10

    Iterative SFT-RL cycles enable a 7B LVLM to develop sophisticated visual chain-of-thought reasoning and improve performance on math and general reasoning benchmarks.

  17. Anthropogenic Regional Adaptation in Multimodal Vision-Language Model

    cs.AI 2026-04 unverdicted novelty 5.0 of 10

    Anthropogenic Regional Adaptation with GG-EZ improves cultural relevance in multimodal vision-language models for Southeast Asia by 5-15% while retaining over 98% of global performance.

  18. Seeing the Scene Matters: Revealing Forgetting in Video Understanding Models with a Scene-Aware Long-Video Benchmark

    cs.CV 2026-03 unverdicted novelty 5.0 of 10

    SceneBench shows VLMs sharply lose accuracy on scene-level long-video questions, and Scene-RAG only partially mitigates that forgetting (+2.50%).

  19. Emerging Properties in Unified Multimodal Pretraining

    cs.CV 2025-05 unverdicted novelty 5.0 of 10

    BAGEL is a unified decoder-only model that develops emerging complex multimodal reasoning abilities after pretraining on large-scale interleaved data and outperforms prior open-source unified models.

  20. Qwen2.5-VL Technical Report

    cs.CV 2025-02 unverdicted novelty 5.0 of 10

    Qwen2.5-VL reports a vision-language model family using native dynamic-resolution ViT and absolute time encoding that matches GPT-4o on document and diagram tasks while supporting hour-long videos with second-level lo...

  21. ZAYA1-VL-8B Technical Report

    cs.CV 2026-05 unverdicted novelty 4.0 of 10

    ZAYA1-VL-8B is a new MoE vision-language model with vision-specific LoRA adapters and bidirectional image attention that reports competitive performance against several 3B-4B models on image, reasoning, and counting b...

  22. A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models

    cs.CL 2025-09 conditional novelty 4.0 of 10

    A structured literature survey concluding that reasoning capabilities do not automatically make LLMs more trustworthy and can introduce new vulnerabilities in safety, robustness, and privacy.

  23. R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization

    cs.CV 2025-03 unverdicted novelty 4.0 of 10

    R1-Onevision turns images into structured text for multimodal reasoning, trains on a custom dataset with RL, and claims SOTA results on an educational benchmark.

  24. A Survey on Diffusion Language Models

    cs.CL 2025-08 unverdicted novelty 3.0 of 10

    A comprehensive survey of diffusion language models covering taxonomy, training and inference techniques, and comparisons with autoregressive models.

  25. Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey

    cs.CV 2025-03 unverdicted novelty 2.0 of 10

    The paper provides the first comprehensive survey of multimodal chain-of-thought reasoning, including foundational concepts, a taxonomy of methodologies, application analyses, challenges, and future directions.

Pith tools