Pith. sign in

REVIEW 10 cited by

GeomVerse: A Systematic Evaluation of Large Models for Geometric Reasoning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.12241 v1 pith:EAGXZFQU submitted 2023-12-19 cs.CV cs.CL

Mehran Kazemi , Hamidreza Alvari , Ankit Anand , Jialin Wu , Xi Chen , Radu Soricut This is my paper · ORCID
classification cs.CVcs.CL
keywords reasoningmodelsproblemsgeometryvlmsalongaxesbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Large language models have shown impressive results for multi-hop mathematical reasoning when the input question is only textual. Many mathematical reasoning problems, however, contain both text and image. With the ever-increasing adoption of vision language models (VLMs), understanding their reasoning abilities for such problems is crucial. In this paper, we evaluate the reasoning capabilities of VLMs along various axes through the lens of geometry problems. We procedurally create a synthetic dataset of geometry questions with controllable difficulty levels along multiple axes, thus enabling a systematic evaluation. The empirical results obtained using our benchmark for state-of-the-art VLMs indicate that these models are not as capable in subjects like geometry (and, by generalization, other topics requiring similar reasoning) as suggested by previous benchmarks. This is made especially clear by the construction of our benchmark at various depth levels, since solving higher-depth problems requires long chains of reasoning rather than additional memorized knowledge. We release the dataset for further research in this area.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Self-Boosting Vision-Language Models with Noisy Student On-Policy Self-Distillation

    cs.LG 2026-07 conditional novelty 6.0 of 10

    NOPD trains a VLM on corrupted images to match its own clean-image predictions, improving accuracy on reasoning benchmarks with no external supervision.

  2. LaRe: Latent Refocusing for Multimodal Reasoning

    cs.CV 2025-11 reject novelty 6.0 of 10

    LaRe performs iterative visual refocusing in latent space and reports accuracy gains with fewer tokens, but its main experiments compare against baselines trained with less data.

  3. Ming-Omni: A Unified Multimodal Model for Perception and Generation

    cs.AI 2025-06 conditional novelty 6.0 of 10

    A single model with modality-specific routing processes image, text, audio, and video inputs and generates text, speech, and images, with public benchmarks reported across all of these abilities.

  4. FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocities

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A 1.5B unified multimodal model trained with discrete flow matching and metric-induced probability paths matches autoregressive baselines of similar size on generation and understanding benchmarks.

  5. Enhancing the Geometric Problem-Solving Ability of Multimodal LLMs via Symbolic-Neural Integration

    cs.CL 2025-04 conditional novelty 6.0 of 10

    A symbolic geometry engine generates step-by-step training data and verifies MLLM reasoning steps, improving accuracy on geometry benchmarks.

  6. GeoSense: Evaluating Identification and Application of Geometric Principles in Multimodal Reasoning

    cs.CL 2025-04 conditional novelty 6.0 of 10

    GeoSense introduces GPI and GPA metrics and a 148-principle hierarchy to jointly measure identification and application of geometric principles in 1,789 bilingual geometry problems.

  7. Chimera: Improving Generalist Model with Domain-Specific Experts

    cs.CV 2024-12 conditional novelty 6.0 of 10

    Chimera fuses frozen domain-expert encoders into a generalist multimodal LLM via routing and a 30% masking of general tokens, lifting InternVL2-8B from 61.6 to 64.9 on MathVista and from 31.3 to 32.4 on MathVerse.

  8. Mixpert: Mitigating Multimodal Learning Conflicts with Efficient Mixture-of-Vision-Experts

    cs.CV 2025-05 conditional novelty 5.0 of 10

    Splitting the vision encoder of an MLLM into domain-specific experts with a lightweight router yields small benchmark improvements at near-zero extra inference cost.

  9. Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search

    cs.CV 2024-12 conditional novelty 5.0 of 10

    Using a group of MLLMs to search reasoning trees and training on the resulting paths improves MLLM reasoning, with Mulberry models beating their base models by up to 7.5 points on average.

  10. Towards Geometry Problem Solving in the Large Model Era: A Survey

    cs.CV 2025-06 conditional novelty 4.0 of 10

    A survey that organizes geometry problem-solving research into benchmark construction, parsing, and reasoning, and proposes a unified parse-then-reason paradigm for the large-model era.

Pith tools