A new CAPTCHA benchmark with a reasoning-depth metric shows multimodal LLM agents solve at most 40% of interactive puzzles, far short of the 93% human success rate.
Gemini 2.5 pro: Our most intelligent ai model
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.AI 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Open CaptchaWorld: A Comprehensive Web-based Platform for Testing and Benchmarking Multimodal LLM Agents
A new CAPTCHA benchmark with a reasoning-depth metric shows multimodal LLM agents solve at most 40% of interactive puzzles, far short of the 93% human success rate.