{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2025:QHOILLGG7XVWLVPBO2XCQO5NHL","short_pith_number":"pith:QHOILLGG","schema_version":"1.0","canonical_sha256":"81dc85acc6fdeb65d5e176ae283bad3aca447dbabaea5010da6ca923bbdce96e","source":{"kind":"arxiv","id":"2502.14301","version":2},"attestation_state":"computed","paper":{"title":"SEA-HELM: Southeast Asian Holistic Evaluation of Language Models","license":"http://creativecommons.org/licenses/by/4.0/","headline":"","cross_cats":["cs.AI"],"primary_cat":"cs.CL","authors_text":"Adithya Venkatadri Hulagadri, Hamsawardhini Rengarajan, Jann Railey Montalan, Jian Gang Ngui, Peerat Limkonchotiwat, Weiqi Leong, William Chandra Tjhi, Xian Bin Yong, Yifan Mai, Yosephine Susanto","submitted_at":"2025-02-20T06:32:45Z","abstract_excerpt":"With the rapid emergence of novel capabilities in Large Language Models (LLMs), the need for rigorous multilingual and multicultural benchmarks that are integrated has become more pronounced. Though existing LLM benchmarks are capable of evaluating specific capabilities of LLMs in English as well as in various mid- to low-resource languages, including those in the Southeast Asian (SEA) region, a comprehensive and culturally representative evaluation suite for the SEA languages has not been developed thus far. Here, we present SEA-HELM, a holistic linguistic and cultural LLM evaluation suite th"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2502.14301","kind":"arxiv","version":2},"metadata":{"license":"http://creativecommons.org/licenses/by/4.0/","primary_cat":"cs.CL","submitted_at":"2025-02-20T06:32:45Z","cross_cats_sorted":["cs.AI"],"title_canon_sha256":"0fdeff101869ebc3c591cc9dd6db5dc92a7d410f38c98d53c9f00ad712dc46be","abstract_canon_sha256":"f10f1b82991a6b242a1427d8f4a48d49e0ee1f503298cc6721e3fd08088f915c"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T11:14:04.320506Z","signature_b64":"VdDIQG07wPV8gyXTciWvXY9lJ9BF67dr4EU4yUrwPuGBwqUi87FkFPsy03IJuA7PgsPonwBBRAujPnk11b57AA==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"81dc85acc6fdeb65d5e176ae283bad3aca447dbabaea5010da6ca923bbdce96e","last_reissued_at":"2026-07-05T11:14:04.320026Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T11:14:04.320026Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"SEA-HELM: Southeast Asian Holistic Evaluation of Language Models","license":"http://creativecommons.org/licenses/by/4.0/","headline":"","cross_cats":["cs.AI"],"primary_cat":"cs.CL","authors_text":"Adithya Venkatadri Hulagadri, Hamsawardhini Rengarajan, Jann Railey Montalan, Jian Gang Ngui, Peerat Limkonchotiwat, Weiqi Leong, William Chandra Tjhi, Xian Bin Yong, Yifan Mai, Yosephine Susanto","submitted_at":"2025-02-20T06:32:45Z","abstract_excerpt":"With the rapid emergence of novel capabilities in Large Language Models (LLMs), the need for rigorous multilingual and multicultural benchmarks that are integrated has become more pronounced. Though existing LLM benchmarks are capable of evaluating specific capabilities of LLMs in English as well as in various mid- to low-resource languages, including those in the Southeast Asian (SEA) region, a comprehensive and culturally representative evaluation suite for the SEA languages has not been developed thus far. Here, we present SEA-HELM, a holistic linguistic and cultural LLM evaluation suite th"},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2502.14301","kind":"arxiv","version":2},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2502.14301/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2502.14301","created_at":"2026-07-05T11:14:04.320082+00:00"},{"alias_kind":"arxiv_version","alias_value":"2502.14301v2","created_at":"2026-07-05T11:14:04.320082+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2502.14301","created_at":"2026-07-05T11:14:04.320082+00:00"},{"alias_kind":"pith_short_12","alias_value":"QHOILLGG7XVW","created_at":"2026-07-05T11:14:04.320082+00:00"},{"alias_kind":"pith_short_16","alias_value":"QHOILLGG7XVWLVPB","created_at":"2026-07-05T11:14:04.320082+00:00"},{"alias_kind":"pith_short_8","alias_value":"QHOILLGG","created_at":"2026-07-05T11:14:04.320082+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":4,"internal_anchor_count":0,"sample":[{"citing_arxiv_id":"2605.24414","citing_title":"JT-SAFE-V2: Safety-by-Design Foundation Model with World-Context Data","ref_index":37,"is_internal_anchor":false},{"citing_arxiv_id":"2606.28715","citing_title":"SEATauBench: Adapting Tool-Agent-User Evaluation Into Low-Resource Southeast Asian Languages","ref_index":19,"is_internal_anchor":false},{"citing_arxiv_id":"2605.22203","citing_title":"Evaluation of Chunking Strategies for Effective Text Embedding in Low-Resource Language on Agricultural Documents","ref_index":16,"is_internal_anchor":false},{"citing_arxiv_id":"2601.14958","citing_title":"Script Sensitivity: Benchmarking Language Models on Unicode, Romanized and Mixed-Script Sinhala","ref_index":19,"is_internal_anchor":false}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/QHOILLGG7XVWLVPBO2XCQO5NHL","json":"https://pith.science/pith/QHOILLGG7XVWLVPBO2XCQO5NHL.json","graph_json":"https://pith.science/api/pith-number/QHOILLGG7XVWLVPBO2XCQO5NHL/graph.json","events_json":"https://pith.science/api/pith-number/QHOILLGG7XVWLVPBO2XCQO5NHL/events.json","paper":"https://pith.science/paper/QHOILLGG"},"agent_actions":{"view_html":"https://pith.science/pith/QHOILLGG7XVWLVPBO2XCQO5NHL","download_json":"https://pith.science/pith/QHOILLGG7XVWLVPBO2XCQO5NHL.json","view_paper":"https://pith.science/paper/QHOILLGG","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2502.14301&json=true","fetch_graph":"https://pith.science/api/pith-number/QHOILLGG7XVWLVPBO2XCQO5NHL/graph.json","fetch_events":"https://pith.science/api/pith-number/QHOILLGG7XVWLVPBO2XCQO5NHL/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/QHOILLGG7XVWLVPBO2XCQO5NHL/action/timestamp_anchor","attest_storage":"https://pith.science/pith/QHOILLGG7XVWLVPBO2XCQO5NHL/action/storage_attestation","attest_author":"https://pith.science/pith/QHOILLGG7XVWLVPBO2XCQO5NHL/action/author_attestation","sign_citation":"https://pith.science/pith/QHOILLGG7XVWLVPBO2XCQO5NHL/action/citation_signature","submit_replication":"https://pith.science/pith/QHOILLGG7XVWLVPBO2XCQO5NHL/action/replication_record"}},"created_at":"2026-07-05T11:14:04.320082+00:00","updated_at":"2026-07-05T11:14:04.320082+00:00"}