{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2025:3C5UYDGHDTNRH22DUZSWATQFCV","short_pith_number":"pith:3C5UYDGH","schema_version":"1.0","canonical_sha256":"d8bb4c0cc71cdb13eb43a665604e05156168d1a34f8eb882eee0e9181e04bdcb","source":{"kind":"arxiv","id":"2502.09696","version":2},"attestation_state":"computed","paper":{"title":"ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":[],"primary_cat":"cs.CV","authors_text":"Aaditya Baranwal, Akash Gupta, Alexander Lo, Alexandru Coca, Anh Totti Nguyen, Ansh Sharma, Brian Pulfer, Charig Yang, David I. Atkinson, Florian Langer, Gyungin Shin, Hanyi Xiong, Ioana Croitoru, Jakob D. Kunz, Jialu Tang, Jingyi Lu, Jonathan Roberts, Kai Han, Kaiqu Liang, Mikah Dang, Mohammad Reza Taesiri, Qiaochu Yang, Sam Purkis, Samuel Albanie, Samuel Roberts, Sebastian Dziadzio, Shiyang Chen, Simion-Vlad Bogolin, Steven Walton, Tianshuo Yan, Vatsal Raina, Vishaal Udandarao, Vyas Raina, Wenye Lin","submitted_at":"2025-02-13T18:59:11Z","abstract_excerpt":"Large Multimodal Models (LMMs) exhibit major shortfalls when interpreting images and, by some measures, have poorer spatial cognition than small children or animals. Despite this, they attain high scores on many popular visual benchmarks, with headroom rapidly eroded by an ongoing surge of model progress. To address this, there is a pressing need for difficult benchmarks that remain relevant for longer. We take this idea to its limit by introducing ZeroBench-a lightweight visual reasoning benchmark that is entirely impossible for contemporary frontier LMMs. Our benchmark consists of 100 manual"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2502.09696","kind":"arxiv","version":2},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.CV","submitted_at":"2025-02-13T18:59:11Z","cross_cats_sorted":[],"title_canon_sha256":"43ea9c05fc894718d681dd0a27beb77ba65363f2fb4f5bc7b21129624f496c91","abstract_canon_sha256":"a8ef3244060670208a131f86a91afc8ca7df76d45a5676eb37afe5f50af14426"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T10:25:17.513445Z","signature_b64":"aVKidnQ18nRmCaDaiC632zG0j8Sx9z1+9uO3fTZk576wRuUSBfyConldabtiCoiZzm8W0kv6+ZJVzsCIiKWBCw==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"d8bb4c0cc71cdb13eb43a665604e05156168d1a34f8eb882eee0e9181e04bdcb","last_reissued_at":"2026-07-05T10:25:17.512804Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T10:25:17.512804Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":[],"primary_cat":"cs.CV","authors_text":"Aaditya Baranwal, Akash Gupta, Alexander Lo, Alexandru Coca, Anh Totti Nguyen, Ansh Sharma, Brian Pulfer, Charig Yang, David I. Atkinson, Florian Langer, Gyungin Shin, Hanyi Xiong, Ioana Croitoru, Jakob D. Kunz, Jialu Tang, Jingyi Lu, Jonathan Roberts, Kai Han, Kaiqu Liang, Mikah Dang, Mohammad Reza Taesiri, Qiaochu Yang, Sam Purkis, Samuel Albanie, Samuel Roberts, Sebastian Dziadzio, Shiyang Chen, Simion-Vlad Bogolin, Steven Walton, Tianshuo Yan, Vatsal Raina, Vishaal Udandarao, Vyas Raina, Wenye Lin","submitted_at":"2025-02-13T18:59:11Z","abstract_excerpt":"Large Multimodal Models (LMMs) exhibit major shortfalls when interpreting images and, by some measures, have poorer spatial cognition than small children or animals. Despite this, they attain high scores on many popular visual benchmarks, with headroom rapidly eroded by an ongoing surge of model progress. To address this, there is a pressing need for difficult benchmarks that remain relevant for longer. We take this idea to its limit by introducing ZeroBench-a lightweight visual reasoning benchmark that is entirely impossible for contemporary frontier LMMs. Our benchmark consists of 100 manual"},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2502.09696","kind":"arxiv","version":2},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2502.09696/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2502.09696","created_at":"2026-07-05T10:25:17.512871+00:00"},{"alias_kind":"arxiv_version","alias_value":"2502.09696v2","created_at":"2026-07-05T10:25:17.512871+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2502.09696","created_at":"2026-07-05T10:25:17.512871+00:00"},{"alias_kind":"pith_short_12","alias_value":"3C5UYDGHDTNR","created_at":"2026-07-05T10:25:17.512871+00:00"},{"alias_kind":"pith_short_16","alias_value":"3C5UYDGHDTNRH22D","created_at":"2026-07-05T10:25:17.512871+00:00"},{"alias_kind":"pith_short_8","alias_value":"3C5UYDGH","created_at":"2026-07-05T10:25:17.512871+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":19,"internal_anchor_count":19,"sample":[{"citing_arxiv_id":"2606.22437","citing_title":"MMGist: A Comprehensive Multimodal Benchmark for 2027","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2606.10651","citing_title":"Kwai Keye-VL-2.0 Technical Report","ref_index":45,"is_internal_anchor":true},{"citing_arxiv_id":"2607.00248","citing_title":"Seed2.0 Model Card: Towards Intelligence Frontier for Real-World Complexity","ref_index":92,"is_internal_anchor":true},{"citing_arxiv_id":"2606.06538","citing_title":"WorldBench: A Challenging and Visually Diverse Multimodal Reasoning Benchmark","ref_index":144,"is_internal_anchor":true},{"citing_arxiv_id":"2606.03087","citing_title":"Learning to Solve, Forgetting to Retain: Correct-Set Turnover in RLVR","ref_index":128,"is_internal_anchor":true},{"citing_arxiv_id":"2606.07639","citing_title":"MOSS-Video-Preview: Toward Real-Time Video Understanding via Cross-Attention","ref_index":37,"is_internal_anchor":true},{"citing_arxiv_id":"2605.30265","citing_title":"LoMo: Local Modality Substitution for Deeper Vision-Language Fusion","ref_index":30,"is_internal_anchor":true},{"citing_arxiv_id":"2505.06698","citing_title":"SCAN: Structured Capability Assessment and Navigation for LLMs","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2507.06261","citing_title":"Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities","ref_index":70,"is_internal_anchor":true},{"citing_arxiv_id":"2503.10615","citing_title":"R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization","ref_index":24,"is_internal_anchor":true},{"citing_arxiv_id":"2603.20633","citing_title":"Seed1.8 Model Card: Towards Generalized Real-World Agency","ref_index":58,"is_internal_anchor":true},{"citing_arxiv_id":"2604.03128","citing_title":"Self-Distilled RLVR","ref_index":17,"is_internal_anchor":true},{"citing_arxiv_id":"2604.27083","citing_title":"Co-Evolving Policy Distillation","ref_index":29,"is_internal_anchor":true},{"citing_arxiv_id":"2604.10219","citing_title":"Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models","ref_index":88,"is_internal_anchor":true},{"citing_arxiv_id":"2505.07062","citing_title":"Seed1.5-VL Technical Report","ref_index":114,"is_internal_anchor":true},{"citing_arxiv_id":"2507.01006","citing_title":"GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning","ref_index":39,"is_internal_anchor":true},{"citing_arxiv_id":"2602.02276","citing_title":"Kimi K2.5: Visual Agentic Intelligence","ref_index":51,"is_internal_anchor":true},{"citing_arxiv_id":"2604.15804","citing_title":"Qwen3.5-Omni Technical Report","ref_index":34,"is_internal_anchor":true},{"citing_arxiv_id":"2604.20733","citing_title":"Near-Future Policy Optimization","ref_index":19,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/3C5UYDGHDTNRH22DUZSWATQFCV","json":"https://pith.science/pith/3C5UYDGHDTNRH22DUZSWATQFCV.json","graph_json":"https://pith.science/api/pith-number/3C5UYDGHDTNRH22DUZSWATQFCV/graph.json","events_json":"https://pith.science/api/pith-number/3C5UYDGHDTNRH22DUZSWATQFCV/events.json","paper":"https://pith.science/paper/3C5UYDGH"},"agent_actions":{"view_html":"https://pith.science/pith/3C5UYDGHDTNRH22DUZSWATQFCV","download_json":"https://pith.science/pith/3C5UYDGHDTNRH22DUZSWATQFCV.json","view_paper":"https://pith.science/paper/3C5UYDGH","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2502.09696&json=true","fetch_graph":"https://pith.science/api/pith-number/3C5UYDGHDTNRH22DUZSWATQFCV/graph.json","fetch_events":"https://pith.science/api/pith-number/3C5UYDGHDTNRH22DUZSWATQFCV/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/3C5UYDGHDTNRH22DUZSWATQFCV/action/timestamp_anchor","attest_storage":"https://pith.science/pith/3C5UYDGHDTNRH22DUZSWATQFCV/action/storage_attestation","attest_author":"https://pith.science/pith/3C5UYDGHDTNRH22DUZSWATQFCV/action/author_attestation","sign_citation":"https://pith.science/pith/3C5UYDGHDTNRH22DUZSWATQFCV/action/citation_signature","submit_replication":"https://pith.science/pith/3C5UYDGHDTNRH22DUZSWATQFCV/action/replication_record"}},"created_at":"2026-07-05T10:25:17.512871+00:00","updated_at":"2026-07-05T10:25:17.512871+00:00"}