{"paper":{"title":"From Pixels to BFS: High Maze Accuracy Does Not Imply Visual Planning","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Multimodal models solve maze images by converting them to text grids and enumerating paths token by token rather than through visual planning.","cross_cats":["cs.CV"],"primary_cat":"cs.LG","authors_text":"Alberto G. Rodriguez Salgado","submitted_at":"2026-03-27T08:10:05Z","abstract_excerpt":"How do multimodal models solve visual spatial tasks -- through genuine planning, or through brute-force search in token space? We introduce \\textsc{MazeBench}, a benchmark of 110 procedurally generated maze images across nine controlled groups, and evaluate 16 model configurations from OpenAI, Anthropic, Google, and Alibaba. GPT-5.4 solves 91\\% and Gemini 3.1 Pro 79\\%, but these scores are misleading: models typically translate images into text grids and then enumerate paths step by step, consuming 1,710--22,818 tokens per solve for a task humans do quickly. Without added reasoning budgets, al"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"MazeBench therefore shows that high accuracy on visual planning tasks does not imply human-like spatial understanding.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That the two-stage image-to-grid plus token enumeration strategy observed in traces is the dominant mechanism driving performance rather than an artifact of the specific prompting or model configurations tested.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Multimodal models achieve high maze accuracy by translating images to text grids and performing token-level BFS search, not through visual planning.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Multimodal models solve maze images by converting them to text grids and enumerating paths token by token rather than through visual planning.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"d976b97cdf972bb6cafa6e406e76c79ea9bf3feaa696102766660ef5b8dbfcb3"},"source":{"id":"2603.26839","kind":"arxiv","version":2},"verdict":{"id":"bf6d08d8-4812-4267-97ea-c988b94b243a","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-15T00:05:18.200340Z","strongest_claim":"MazeBench therefore shows that high accuracy on visual planning tasks does not imply human-like spatial understanding.","one_line_summary":"Multimodal models achieve high maze accuracy by translating images to text grids and performing token-level BFS search, not through visual planning.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That the two-stage image-to-grid plus token enumeration strategy observed in traces is the dominant mechanism driving performance rather than an artifact of the specific prompting or model configurations tested.","pith_extraction_headline":"Multimodal models solve maze images by converting them to text grids and enumerating paths token by token rather than through visual planning."},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":2,"snapshot_sha256":"2fe155cd7c9677a14573ccbf4acbc8106e30e888fad95d92065dbff341312706"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}