{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2024:MWIGUTQR3XUUGLED3V5TOBTUAA","short_pith_number":"pith:MWIGUTQR","schema_version":"1.0","canonical_sha256":"65906a4e11dde9432c83dd7b370674001a3262077ef686554225589f5074ed6f","source":{"kind":"arxiv","id":"2406.12753","version":2},"attestation_state":"computed","paper":{"title":"OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":["cs.AI"],"primary_cat":"cs.CL","authors_text":"Binjie Wang, Dahua Lin, Ethan Chern, Fan Zhou, Haoyang Zou, Jiadi Su, Lyumanshan Ye, Pengfei Liu, Ruijie Xu, Run-Ze Fan, Shaoting Zhang, Shichao Sun, Shijie Xia, Steffi Chern, Ting Wu, Xuefeng Li, Yang Xiao, Yan Ma, Yikai Zhang, Yiwei Qin, Yixin Ye, Yixiu Liu, Yiyuan Li, Yu Qiao, Yuqing Yang, Yuxiang Zheng, Zengzhi Wang, Zhen Huang","submitted_at":"2024-06-18T16:20:53Z","abstract_excerpt":"The evolution of Artificial Intelligence (AI) has been significantly accelerated by advancements in Large Language Models (LLMs) and Large Multimodal Models (LMMs), gradually showcasing potential cognitive reasoning abilities in problem-solving and scientific discovery (i.e., AI4Science) once exclusive to human intellect. To comprehensively evaluate current models' performance in cognitive reasoning abilities, we introduce OlympicArena, which includes 11,163 bilingual problems across both text-only and interleaved text-image modalities. These challenges encompass a wide range of disciplines sp"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2406.12753","kind":"arxiv","version":2},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.CL","submitted_at":"2024-06-18T16:20:53Z","cross_cats_sorted":["cs.AI"],"title_canon_sha256":"61fa7ef14cd970fd5975536f8d51474376ac728d0d9ccf89a9aa3b1a3683306a","abstract_canon_sha256":"d08bf4b5f467d960ceacb10c662498eee1659b92ac06a1a89ca231dd6947a4e2"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T10:25:07.566616Z","signature_b64":"9tmzp7dCaaLvWJeJdfBlI2QmMAiHyipZhHa8Ys5Kw0eY81LaOFjqWm9HwwsXqBt32rHF/XzNFu18SEs7TodJAw==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"65906a4e11dde9432c83dd7b370674001a3262077ef686554225589f5074ed6f","last_reissued_at":"2026-07-05T10:25:07.565734Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T10:25:07.565734Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":["cs.AI"],"primary_cat":"cs.CL","authors_text":"Binjie Wang, Dahua Lin, Ethan Chern, Fan Zhou, Haoyang Zou, Jiadi Su, Lyumanshan Ye, Pengfei Liu, Ruijie Xu, Run-Ze Fan, Shaoting Zhang, Shichao Sun, Shijie Xia, Steffi Chern, Ting Wu, Xuefeng Li, Yang Xiao, Yan Ma, Yikai Zhang, Yiwei Qin, Yixin Ye, Yixiu Liu, Yiyuan Li, Yu Qiao, Yuqing Yang, Yuxiang Zheng, Zengzhi Wang, Zhen Huang","submitted_at":"2024-06-18T16:20:53Z","abstract_excerpt":"The evolution of Artificial Intelligence (AI) has been significantly accelerated by advancements in Large Language Models (LLMs) and Large Multimodal Models (LMMs), gradually showcasing potential cognitive reasoning abilities in problem-solving and scientific discovery (i.e., AI4Science) once exclusive to human intellect. To comprehensively evaluate current models' performance in cognitive reasoning abilities, we introduce OlympicArena, which includes 11,163 bilingual problems across both text-only and interleaved text-image modalities. These challenges encompass a wide range of disciplines sp"},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2406.12753","kind":"arxiv","version":2},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2406.12753/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2406.12753","created_at":"2026-07-05T10:25:07.565836+00:00"},{"alias_kind":"arxiv_version","alias_value":"2406.12753v2","created_at":"2026-07-05T10:25:07.565836+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2406.12753","created_at":"2026-07-05T10:25:07.565836+00:00"},{"alias_kind":"pith_short_12","alias_value":"MWIGUTQR3XUU","created_at":"2026-07-05T10:25:07.565836+00:00"},{"alias_kind":"pith_short_16","alias_value":"MWIGUTQR3XUUGLED","created_at":"2026-07-05T10:25:07.565836+00:00"},{"alias_kind":"pith_short_8","alias_value":"MWIGUTQR","created_at":"2026-07-05T10:25:07.565836+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":7,"internal_anchor_count":0,"sample":[{"citing_arxiv_id":"2606.11470","citing_title":"The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes","ref_index":99,"is_internal_anchor":false},{"citing_arxiv_id":"2606.01145","citing_title":"Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branches","ref_index":114,"is_internal_anchor":false},{"citing_arxiv_id":"2606.01145","citing_title":"Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branches","ref_index":124,"is_internal_anchor":false},{"citing_arxiv_id":"2502.03387","citing_title":"LIMO: Less is More for Reasoning","ref_index":23,"is_internal_anchor":false},{"citing_arxiv_id":"2504.11456","citing_title":"DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning","ref_index":13,"is_internal_anchor":false},{"citing_arxiv_id":"2410.07985","citing_title":"Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models","ref_index":60,"is_internal_anchor":false},{"citing_arxiv_id":"2604.15411","citing_title":"PRL-Bench: A Comprehensive Benchmark Evaluating LLMs' Capabilities in Frontier Physics Research","ref_index":4,"is_internal_anchor":false}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/MWIGUTQR3XUUGLED3V5TOBTUAA","json":"https://pith.science/pith/MWIGUTQR3XUUGLED3V5TOBTUAA.json","graph_json":"https://pith.science/api/pith-number/MWIGUTQR3XUUGLED3V5TOBTUAA/graph.json","events_json":"https://pith.science/api/pith-number/MWIGUTQR3XUUGLED3V5TOBTUAA/events.json","paper":"https://pith.science/paper/MWIGUTQR"},"agent_actions":{"view_html":"https://pith.science/pith/MWIGUTQR3XUUGLED3V5TOBTUAA","download_json":"https://pith.science/pith/MWIGUTQR3XUUGLED3V5TOBTUAA.json","view_paper":"https://pith.science/paper/MWIGUTQR","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2406.12753&json=true","fetch_graph":"https://pith.science/api/pith-number/MWIGUTQR3XUUGLED3V5TOBTUAA/graph.json","fetch_events":"https://pith.science/api/pith-number/MWIGUTQR3XUUGLED3V5TOBTUAA/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/MWIGUTQR3XUUGLED3V5TOBTUAA/action/timestamp_anchor","attest_storage":"https://pith.science/pith/MWIGUTQR3XUUGLED3V5TOBTUAA/action/storage_attestation","attest_author":"https://pith.science/pith/MWIGUTQR3XUUGLED3V5TOBTUAA/action/author_attestation","sign_citation":"https://pith.science/pith/MWIGUTQR3XUUGLED3V5TOBTUAA/action/citation_signature","submit_replication":"https://pith.science/pith/MWIGUTQR3XUUGLED3V5TOBTUAA/action/replication_record"}},"created_at":"2026-07-05T10:25:07.565836+00:00","updated_at":"2026-07-05T10:25:07.565836+00:00"}