{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2025:I55WMNR7KBJM73RPHRW6JRC4IF","short_pith_number":"pith:I55WMNR7","schema_version":"1.0","canonical_sha256":"477b66363f5052cfee2f3c6de4c45c4149ad5d468c03050f2c2aea4c31bd6b5f","source":{"kind":"arxiv","id":"2501.08838","version":1},"attestation_state":"computed","paper":{"title":"ToMATO: Verbalizing the Mental States of Role-Playing LLMs for Benchmarking Theory of Mind","license":"http://creativecommons.org/licenses/by/4.0/","headline":"","cross_cats":["cs.AI"],"primary_cat":"cs.CL","authors_text":"Hiroaki Sugiyama, Kazutoshi Shinoda, Keita Suzuki, Kuniko Saito, Kyosuke Nishida, Nobukatsu Hojo, Ryo Masumura, Saki Mizuno","submitted_at":"2025-01-15T14:47:02Z","abstract_excerpt":"Existing Theory of Mind (ToM) benchmarks diverge from real-world scenarios in three aspects: 1) they assess a limited range of mental states such as beliefs, 2) false beliefs are not comprehensively explored, and 3) the diverse personality traits of characters are overlooked. To address these challenges, we introduce ToMATO, a new ToM benchmark formulated as multiple-choice QA over conversations. ToMATO is generated via LLM-LLM conversations featuring information asymmetry. By employing a prompting method that requires role-playing LLMs to verbalize their thoughts before each utterance, we cap"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2501.08838","kind":"arxiv","version":1},"metadata":{"license":"http://creativecommons.org/licenses/by/4.0/","primary_cat":"cs.CL","submitted_at":"2025-01-15T14:47:02Z","cross_cats_sorted":["cs.AI"],"title_canon_sha256":"fe839408f8d2e05a93110997d55c0f8f13f94fff878954d3bd19df38bcfe48e0","abstract_canon_sha256":"92b73cc77302a3925ed9ddb911352b983202ab239d954a24877169f7eb793cc6"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T10:01:26.910336Z","signature_b64":"WVREH5C38ItsTgoFLqKu9q1Dr81j1I/F138mJ7xopaC/ZLlXki8fWLv9WMg1lYRGdsbGK5Ym5nFmTJHuXDezCw==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"477b66363f5052cfee2f3c6de4c45c4149ad5d468c03050f2c2aea4c31bd6b5f","last_reissued_at":"2026-07-05T10:01:26.909897Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T10:01:26.909897Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"ToMATO: Verbalizing the Mental States of Role-Playing LLMs for Benchmarking Theory of Mind","license":"http://creativecommons.org/licenses/by/4.0/","headline":"","cross_cats":["cs.AI"],"primary_cat":"cs.CL","authors_text":"Hiroaki Sugiyama, Kazutoshi Shinoda, Keita Suzuki, Kuniko Saito, Kyosuke Nishida, Nobukatsu Hojo, Ryo Masumura, Saki Mizuno","submitted_at":"2025-01-15T14:47:02Z","abstract_excerpt":"Existing Theory of Mind (ToM) benchmarks diverge from real-world scenarios in three aspects: 1) they assess a limited range of mental states such as beliefs, 2) false beliefs are not comprehensively explored, and 3) the diverse personality traits of characters are overlooked. To address these challenges, we introduce ToMATO, a new ToM benchmark formulated as multiple-choice QA over conversations. ToMATO is generated via LLM-LLM conversations featuring information asymmetry. By employing a prompting method that requires role-playing LLMs to verbalize their thoughts before each utterance, we cap"},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2501.08838","kind":"arxiv","version":1},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2501.08838/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2501.08838","created_at":"2026-07-05T10:01:26.909952+00:00"},{"alias_kind":"arxiv_version","alias_value":"2501.08838v1","created_at":"2026-07-05T10:01:26.909952+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2501.08838","created_at":"2026-07-05T10:01:26.909952+00:00"},{"alias_kind":"pith_short_12","alias_value":"I55WMNR7KBJM","created_at":"2026-07-05T10:01:26.909952+00:00"},{"alias_kind":"pith_short_16","alias_value":"I55WMNR7KBJM73RP","created_at":"2026-07-05T10:01:26.909952+00:00"},{"alias_kind":"pith_short_8","alias_value":"I55WMNR7","created_at":"2026-07-05T10:01:26.909952+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":3,"internal_anchor_count":0,"sample":[{"citing_arxiv_id":"2606.31916","citing_title":"Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action","ref_index":15,"is_internal_anchor":false},{"citing_arxiv_id":"2506.05425","citing_title":"SIV-Bench: A Video Benchmark for Social Interaction Understanding and Reasoning","ref_index":41,"is_internal_anchor":false},{"citing_arxiv_id":"2604.20443","citing_title":"DialToM: A Theory of Mind Benchmark for Forecasting State-Driven Dialogue Trajectories","ref_index":32,"is_internal_anchor":false}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/I55WMNR7KBJM73RPHRW6JRC4IF","json":"https://pith.science/pith/I55WMNR7KBJM73RPHRW6JRC4IF.json","graph_json":"https://pith.science/api/pith-number/I55WMNR7KBJM73RPHRW6JRC4IF/graph.json","events_json":"https://pith.science/api/pith-number/I55WMNR7KBJM73RPHRW6JRC4IF/events.json","paper":"https://pith.science/paper/I55WMNR7"},"agent_actions":{"view_html":"https://pith.science/pith/I55WMNR7KBJM73RPHRW6JRC4IF","download_json":"https://pith.science/pith/I55WMNR7KBJM73RPHRW6JRC4IF.json","view_paper":"https://pith.science/paper/I55WMNR7","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2501.08838&json=true","fetch_graph":"https://pith.science/api/pith-number/I55WMNR7KBJM73RPHRW6JRC4IF/graph.json","fetch_events":"https://pith.science/api/pith-number/I55WMNR7KBJM73RPHRW6JRC4IF/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/I55WMNR7KBJM73RPHRW6JRC4IF/action/timestamp_anchor","attest_storage":"https://pith.science/pith/I55WMNR7KBJM73RPHRW6JRC4IF/action/storage_attestation","attest_author":"https://pith.science/pith/I55WMNR7KBJM73RPHRW6JRC4IF/action/author_attestation","sign_citation":"https://pith.science/pith/I55WMNR7KBJM73RPHRW6JRC4IF/action/citation_signature","submit_replication":"https://pith.science/pith/I55WMNR7KBJM73RPHRW6JRC4IF/action/replication_record"}},"created_at":"2026-07-05T10:01:26.909952+00:00","updated_at":"2026-07-05T10:01:26.909952+00:00"}