{"paper":{"title":"SceneFunRI: Reasoning the Invisible for Task-Driven Functional Object Localization","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Vision-language models cannot reliably locate invisible functional objects from task instructions and commonsense.","cross_cats":["cs.AI","cs.RO"],"primary_cat":"cs.CV","authors_text":"Gueter Josmy Faure, Hung-Ting Su, Posheng Chen, Powen Cheng, Winston H. Hsu","submitted_at":"2026-05-14T11:21:41Z","abstract_excerpt":"In real-world scenes, target objects may reside in regions that are not visible. While humans can often infer the locations of occluded objects from context and commonsense knowledge, this capability remains a major challenge for vision-language models (VLMs). To address this gap, we introduce SceneFunRI, a benchmark for Reasoning the Invisible. Based on the SceneFun3D dataset, SceneFunRI formulates the task as a 2D spatial reasoning problem via a semi-automatic pipeline and comprises 855 instances. It requires models to infer the locations of invisible functional objects from task instruction"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"The strongest baseline model (Gemini 3 Flash) only achieves an CAcc@75 of 15.20, an mIoU of 0.74, and a Dist of 28.65. These findings indicate that invisible-region reasoning remains an unstable capability in current VLMs.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The semi-automatic pipeline accurately creates 855 instances that genuinely require commonsense and spatial reasoning beyond superficial visual cues, rather than introducing artifacts that explain the low model performance.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"SceneFunRI benchmark shows current VLMs struggle severely with inferring locations of invisible functional objects, with the strongest model (Gemini 3 Flash) reaching only 15.20 CAcc@75.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Vision-language models cannot reliably locate invisible functional objects from task instructions and commonsense.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"350d4c16af96344b115a740becd405acb297eb7087801705eee37fe15c38c56e"},"source":{"id":"2605.14704","kind":"arxiv","version":1},"verdict":{"id":"cc070207-2f90-435d-958a-98bbded92077","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-15T05:00:12.913741Z","strongest_claim":"The strongest baseline model (Gemini 3 Flash) only achieves an CAcc@75 of 15.20, an mIoU of 0.74, and a Dist of 28.65. These findings indicate that invisible-region reasoning remains an unstable capability in current VLMs.","one_line_summary":"SceneFunRI benchmark shows current VLMs struggle severely with inferring locations of invisible functional objects, with the strongest model (Gemini 3 Flash) reaching only 15.20 CAcc@75.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The semi-automatic pipeline accurately creates 855 instances that genuinely require commonsense and spatial reasoning beyond superficial visual cues, rather than introducing artifacts that explain the low model performance.","pith_extraction_headline":"Vision-language models cannot reliably locate invisible functional objects from task instructions and commonsense."},"references":{"count":38,"sample":[{"doi":"","year":2023,"title":"Image amodal completion: A survey.Computer Vision and Image Understanding, 229:103661, 2023","work_id":"5d8af2fe-47c5-4081-9bec-1f7dacd64cb0","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2025,"title":"Open-world amodal appearance completion","work_id":"9543665c-f2f4-42eb-b51f-11d38d50848d","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2024,"title":"It’s not easy being wrong: Large language models struggle with process of elimination reasoning","work_id":"a031f5c6-c8f8-4b2a-b1dd-5578f2641958","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":null,"title":"Scaling spatial intelligence with multimodal foundation models","work_id":"ceb84861-8bdb-4822-864c-8e2d0ae4d322","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2024,"title":"SceneFun3D: Fine-Grained Functionality and Affordance Understanding in 3D Scenes","work_id":"53329c45-0a4d-4522-8b6a-451a787fa9e7","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":38,"snapshot_sha256":"fa2f7fb6a2787cb6480867914701f164a310afa2322f804f54cb0933e767fcbc","internal_anchors":4},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}