A new multilingual, image-based Kangaroo math benchmark shows Gemini 2.0 Flash, Qwen-VL 2.5 72B, and GPT-4o lead current multimodal LLMs, but all remain far below human accuracy on visual math reasoning.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.AI 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Evaluating Visual Mathematics in Multimodal LLMs: A Multilingual Benchmark Based on the Kangaroo Tests
A new multilingual, image-based Kangaroo math benchmark shows Gemini 2.0 Flash, Qwen-VL 2.5 72B, and GPT-4o lead current multimodal LLMs, but all remain far below human accuracy on visual math reasoning.