REVIEW 12 cited by
M$^3$CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Multi-modal Chain-of-Thought (MCoT) requires models to leverage knowledge from both textual and visual modalities for step-by-step reasoning, which gains increasing attention. Nevertheless, the current MCoT benchmark still faces some challenges: (1) absence of visual modal reasoning, (2) single-step visual modal reasoning, and (3) Domain missing, thereby hindering the development of MCoT. Motivated by this, we introduce a novel benchmark (M$^3$CoT) to address the above challenges, advancing the multi-domain, multi-step, and multi-modal CoT. Additionally, we conduct a thorough evaluation involving abundant MCoT approaches on Vision Large Language Models (VLLMs). In addition, we highlight that the current VLLMs still struggle to correctly reason in M$^3$CoT and there remains a large gap between existing VLLMs and human performance in M$^3$CoT, despite their superior results on previous MCoT benchmarks. To our knowledge, we take the first meaningful step toward the multi-domain, multi-step, and multi-modal scenario in MCoT. We hope that M$^3$CoT can serve as a valuable resource, providing a pioneering foundation in multi-domain, multi-step, multi-modal chain-of-thought research.
Forward citations
Cited by 12 Pith papers
-
Multimodal Mathematical Reasoning Embedded in Aerial Vehicle Imagery: Benchmarking, Analysis, and Exploration
Even the strongest tested vision-language model, GPT-4o, answers only about a third of the new AVI-Math aerial-imagery math questions correctly.
-
Evaluating MLLMs with Multimodal Multi-image Reasoning Benchmark
MMRB is the first benchmark combining multi-image inputs with chain-of-thought reasoning annotations, and its evaluation shows open-source MLLMs trail commercial models while multi-image reward models are unstable.
-
ST-Veto: Spatio-Temporal Token Veto for Diffusion MLLMs via Taylor Prediction and Visual Grounding
ST-Veto improves reasoning in diffusion MLLMs by vetoing temporally unstable tokens and tokens with weak image grounding, swapping in safer near-boundary candidates.
-
BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception
BVS combines early-stop attention rollout priors with a scale-aware non-stationary kernel and GP-UCB to locate tiny objects in UHR images more accurately and with fewer MLLM queries than prior visual-search methods.
-
AIM-CoT: Active Information-driven Multimodal Chain-of-Thought for Vision-Language Reasoning
AIM-CoT improves multimodal chain-of-thought by selecting image regions that reduce predictive uncertainty and inserting them when attention shifts toward the visual input.
-
HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenes
A hierarchical benchmark for multimodal models on human-centric visual understanding finds frontier models average under 60% and miss question-uncued visual evidence, with test-time scaling helping only marginally.
-
MMAT-1M: A Large Reasoning Dataset for Multimodal Agent Tuning
A new one-million-sample multimodal agent tuning dataset with GPT-4o-generated rationales, reflection, and tool/RAG calls is shown to improve fine-tuned models, though training/eval benchmark overlap is not addressed.
-
VFaith: Do Large Multimodal Models Really Reason on Seen Images Rather than Previous Memories?
A new benchmark with edited-image question pairs shows that multimodal reasoning models lose accuracy when visual cues change, suggesting their reasoning is often not faithfully tied to the image.
-
GThinker: Towards General Multimodal Reasoning via Cue-Guided Rethinking
A cue-tagging and rethinking training recipe lifts a 7B multimodal model to 81.5% on M3CoT, though the gain may reflect training on the same benchmark.
-
MIND: Multi-rationale INtegrated Discriminative Reasoning Framework for Multi-modal Large Models
MIND improves multimodal reasoning by training on diverse correct and deliberately wrong rationales with two-stage correction and contrastive alignment, reporting SOTA on ScienceQA, A-OKVQA, and M3CoT.
-
VReST: Enhancing Reasoning in Large Vision-Language Models through Tree Search and Self-Reward Mechanism
VReST combines Monte Carlo tree search with a self-reward signal inside a vision-language model to get higher accuracy than CoT, ToT, or voting baselines on MathVista, MathVision, and CharXiv, while spending several t...
-
Mimicking or Reasoning: Rethinking Multi-Modal In-Context Learning in Vision-Language Models
Vision-language models improve little, often not at all, when given demonstrations, even when demonstrations contain explicit reasoning steps.
Discussion (0). Sign in to comment.