REVIEW 26 cited by
VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Visual reasoning is a core component of human intelligence and a critical capability for advanced multimodal models. Yet current reasoning evaluations of multimodal large language models (MLLMs) often rely on text descriptions and allow language-based reasoning shortcuts, failing to measure genuine vision-centric reasoning. To address this, we introduce VisuLogic: a benchmark of 1,000 human-verified problems across six categories (e.g., quantitative shifts, spatial relations, attribute comparisons). These various types of questions can be evaluated to assess the visual reasoning capabilities of MLLMs from multiple perspectives. We evaluate leading MLLMs on this benchmark and analyze their results to identify common failure modes. Most models score below 30% accuracy-only slightly above the 25% random baseline and far below the 51.4% achieved by humans-revealing significant gaps in visual reasoning. Furthermore, we provide a supplementary training dataset and a reinforcement-learning baseline to support further progress.
Forward citations
Cited by 26 Pith papers
-
FeynmanBench: Benchmarking Multimodal LLMs on Diagrammatic Physics Reasoning
FeynmanBench is the first benchmark for evaluating multimodal LLMs on diagrammatic reasoning with Feynman diagrams, revealing systematic failures in enforcing physical constraints and global topology.
-
GraphVerse: A Comprehensive Visual Graph Reasoning Benchmark for Multimodal Large Language Models
GraphVerse is a new visual graph reasoning benchmark with structure-preserving image edits and a process-sensitive score, and it shows current multimodal LLMs lag far behind humans, especially on paired-image and NP-h...
-
PRISM: Priority-aware Rubric Internalization via Structured Multimodal Data Synthesis
PRISM trains MLLMs to act as rubric executors by synthesizing typed, prioritized rules and verification traces, lifting Qwen3-VL-4B from 9.5% to 30.1% Strict accuracy on the authors' PRISM-Eval benchmark.
-
SynthDocBench: Controlled Benchmark for Long-Context Visual Document Understanding
A controlled synthetic long-document VQA benchmark surfaces three VLM failure modes—length degradation, middle-position sensitivity, and long-context chart collapse—that real-document benchmarks cannot isolate.
-
Evaluating MLLMs with Multimodal Multi-image Reasoning Benchmark
MMRB is the first benchmark combining multi-image inputs with chain-of-thought reasoning annotations, and its evaluation shows open-source MLLMs trail commercial models while multi-image reward models are unstable.
-
Towards Spoken Mathematical Reasoning: Benchmarking Speech-based Models over Multi-faceted Math Problems
A new spoken math benchmark, Spoken-MQA, shows that current speech-based AI models reason poorly from spoken math input, especially for arithmetic and knowledge-heavy problems.
-
PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models
Frontier MLLMs remain far from mastering atomic visual perception: none reach 60% on a failure-derived, perception-only benchmark of ten capabilities.
-
Segmentation before Answering: Pixel Grounding for MLLM Visual Reasoning
SegAnswer trains an MLLM to generate segmentation masks instead of bounding boxes when zooming into image regions during visual reasoning, yielding consistent improvements across perception and hallucination benchmarks.
-
BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception
BVS combines early-stop attention rollout priors with a scale-aware non-stationary kernel and GP-UCB to locate tiny objects in UHR images more accurately and with fewer MLLM queries than prior visual-search methods.
-
MolSight: A Graph-Aware Vision-Language Model for Unified Chemical Image Understanding
MolSight integrates a Molecular Topology Module and Molecular Grounding Module into VLMs to enhance molecular image understanding and claims to outperform prior models on chemical visual tasks.
-
H-OPD: Confidence Aware Heterogeneous Multi-Teacher Multimodal On-policy Distillation
Token-level confidence arbitration between VL and text teachers during on-policy distillation improves multimodal student reasoning over single-teacher OPD and GRPO.
-
Test-time Scaling over Perception: Resolving the Grounding Paradox in Thinking with Images
TTSP samples and filters multiple zoom-in exploration traces and iteratively consolidates validated observations into an Evidence Ledger, improving fine-grained multimodal reasoning on V* Bench, HR-Bench, TreeBench, a...
-
Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models
Multimodal reasoning models hallucinate at high-entropy cognitive bifurcation points due to loss of visual semantic anchoring, and the V-STAR training paradigm with HVAR rewards and FRM reflection mitigates this by re...
-
MapTab: A Diagnostic Benchmark for Long-Horizon Multi-Criteria Multimodal Reasoning on Heterogeneous Topological Graphs
MapTab is a new multimodal benchmark with 328 images and nearly 200k queries that shows current MLLMs have substantial difficulty with multi-criteria route planning when visual and tabular information must be combined.
-
MentisOculi: Revealing the Limits of Reasoning with Mental Imagery
Visual thoughts — latent tokens, interleaved images, or video rollouts — do not currently improve multi-step reasoning over text-only baselines in frontier models.
-
Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models
ReAlign corrects the modality gap in unpaired data to let MLLMs learn visual distributions from text alone before instruction tuning, reducing dependence on expensive paired corpora.
-
Kwai Keye-VL 1.5 Technical Report
Keye-VL-1.5 combines similarity-based Slow-Fast video token allocation with progressive context extension and iterative RL, reporting leading video-understanding results among 8B-scale multimodal models.
-
LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Model
RL training on preference-labeled critic data transforms a 7B vision-language model into both a stronger critic and a stronger generative policy, improving average benchmark accuracy by 5.7% and enabling self-critique...
-
R-4B: Incentivizing General-Purpose Auto-Thinking Capability in MLLMs via Bi-Mode Annealing and Reinforce Learning
R-4B uses bi-mode annealing plus a GRPO variant with forced thinking and non-thinking rollouts so a 4B multimodal model adaptively picks when to reason, reportedly matching 16B thinking models on several reasoning benchmarks.
-
Ascending the Infinite Ladder: Benchmarking Spatial Deformation Reasoning in Vision-Language Models
Vision-language models can handle some 2D shape puzzles but nearly all fail at multi-step 3D spatial deformation reasoning.
-
VisualSphinx: Large-Scale Synthetic Vision Logic Puzzles for RL
LLM-derived rules, genetic expansion, and program-drawn images produce 660K visual logic puzzles; GRPO training on them lifts a VLM's accuracy on those puzzles and modestly on MathVista.
-
OCR-Reasoning Benchmark: Unveiling the True Capabilities of MLLMs in Complex Text-Rich Image Reasoning
OCR-Reasoning, a 1,069-question benchmark with reasoning-chain annotations for text-rich images, finds that no evaluated multimodal model surpasses 50% accuracy.
-
DecoupleMix: Decoupled Ratio Search and Convex Allocation for Scalable VLM Data Recipes
Decoupling inter-class ratio search from intra-class convex allocation yields VLM data recipes that beat stacking and transfer from small proxies to larger scales.
-
Skywork-R1V3 Technical Report
A 38B open-source VLM reaches 76.0% on MMMU using RL post-training and connector-only tuning, with a critical-token entropy metric for checkpoint selection.
-
Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
A survey that maps reinforcement learning methods, datasets, benchmarks, and open-source tools across the full training lifecycle of large language models, focusing on verifiable-reward reasoning.
-
Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models
A survey-style position paper claims that reinforcement fine-tuning powers reasoning in multimodal LLMs, summarizing over a hundred recent works and proposing five future research directions.
Discussion (0). Continue with ORCID to comment.