{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2022:K52ZZPDYVZVFALD35NHYQ5ATOU","short_pith_number":"pith:K52ZZPDY","schema_version":"1.0","canonical_sha256":"57759cbc78ae6a502c7beb4f8874137536d24365114766e066ca54af868b5851","source":{"kind":"arxiv","id":"2204.07705","version":3},"attestation_state":"computed","paper":{"title":"Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks","license":"http://creativecommons.org/licenses/by/4.0/","headline":"","cross_cats":["cs.AI"],"primary_cat":"cs.CL","authors_text":"Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, Chitta Baral, Daniel Khashabi, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Gary Lai, Hannaneh Hajishirzi, Ishani Mondal, Ishan Purohit, Jacob Anderson, Kirby Kuznia, Krima Doshi, Kuntal Kumar Pal, Maitreya Patel, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Noah A. Smith, Pegah Alipoormolabashi, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Savan Doshi, Shailaja Keyur Sampat, Siddhartha Mishra, Sujan Reddy, Sumanta Patro, Swaroop Mishra, Tanay Dixit, Xudong Shen, Yeganeh Kordi, Yejin Choi, Yizhong Wang","submitted_at":"2022-04-16T03:12:30Z","abstract_excerpt":"How well can NLP models generalize to a variety of unseen tasks when provided with task instructions? To address this question, we first introduce Super-NaturalInstructions, a benchmark of 1,616 diverse NLP tasks and their expert-written instructions. Our collection covers 76 distinct task types, including but not limited to classification, extraction, infilling, sequence tagging, text rewriting, and text composition. This large and diverse collection of tasks enables rigorous benchmarking of cross-task generalization under instructions -- training models to follow instructions on a subset of "},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2204.07705","kind":"arxiv","version":3},"metadata":{"license":"http://creativecommons.org/licenses/by/4.0/","primary_cat":"cs.CL","submitted_at":"2022-04-16T03:12:30Z","cross_cats_sorted":["cs.AI"],"title_canon_sha256":"676f1433773a925699670d78af9316f9cb3fb13ff4b5f77313b53214c4ddb600","abstract_canon_sha256":"2d095724060372f6e9cc48c7179f8947d8549fbe6cdcdd4fe38f3a34bce78e72"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T05:09:17.293541Z","signature_b64":"sO2Cc67+lj1LalzHnbcauO+FTRIwwd0s8JrmUAPvdZNP+FHf0kF4TxDPUOfPHG2UxA28MEpCIa6B+5Sl2cz3CA==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"57759cbc78ae6a502c7beb4f8874137536d24365114766e066ca54af868b5851","last_reissued_at":"2026-07-05T05:09:17.293059Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T05:09:17.293059Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks","license":"http://creativecommons.org/licenses/by/4.0/","headline":"","cross_cats":["cs.AI"],"primary_cat":"cs.CL","authors_text":"Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, Chitta Baral, Daniel Khashabi, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Gary Lai, Hannaneh Hajishirzi, Ishani Mondal, Ishan Purohit, Jacob Anderson, Kirby Kuznia, Krima Doshi, Kuntal Kumar Pal, Maitreya Patel, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Noah A. Smith, Pegah Alipoormolabashi, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Savan Doshi, Shailaja Keyur Sampat, Siddhartha Mishra, Sujan Reddy, Sumanta Patro, Swaroop Mishra, Tanay Dixit, Xudong Shen, Yeganeh Kordi, Yejin Choi, Yizhong Wang","submitted_at":"2022-04-16T03:12:30Z","abstract_excerpt":"How well can NLP models generalize to a variety of unseen tasks when provided with task instructions? To address this question, we first introduce Super-NaturalInstructions, a benchmark of 1,616 diverse NLP tasks and their expert-written instructions. Our collection covers 76 distinct task types, including but not limited to classification, extraction, infilling, sequence tagging, text rewriting, and text composition. This large and diverse collection of tasks enables rigorous benchmarking of cross-task generalization under instructions -- training models to follow instructions on a subset of "},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2204.07705","kind":"arxiv","version":3},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2204.07705/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2204.07705","created_at":"2026-07-05T05:09:17.293118+00:00"},{"alias_kind":"arxiv_version","alias_value":"2204.07705v3","created_at":"2026-07-05T05:09:17.293118+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2204.07705","created_at":"2026-07-05T05:09:17.293118+00:00"},{"alias_kind":"pith_short_12","alias_value":"K52ZZPDYVZVF","created_at":"2026-07-05T05:09:17.293118+00:00"},{"alias_kind":"pith_short_16","alias_value":"K52ZZPDYVZVFALD3","created_at":"2026-07-05T05:09:17.293118+00:00"},{"alias_kind":"pith_short_8","alias_value":"K52ZZPDY","created_at":"2026-07-05T05:09:17.293118+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":21,"internal_anchor_count":0,"sample":[{"citing_arxiv_id":"2606.25325","citing_title":"Omni-Perception Policy Optimization for Multimodal Emotion Reasoning","ref_index":157,"is_internal_anchor":false},{"citing_arxiv_id":"2606.09132","citing_title":"Vision Language Model Helps Private Information De-Identification in Vision Data","ref_index":26,"is_internal_anchor":false},{"citing_arxiv_id":"2606.29869","citing_title":"ARKD: Adaptive Reinforcement Learning-Guided Bidirectional KL Divergence Distillation for Text Generation","ref_index":53,"is_internal_anchor":false},{"citing_arxiv_id":"2505.13893","citing_title":"InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model Fusion","ref_index":65,"is_internal_anchor":false},{"citing_arxiv_id":"2410.20791","citing_title":"From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmap","ref_index":109,"is_internal_anchor":false},{"citing_arxiv_id":"2412.02930","citing_title":"TemporalVLM: Video LLMs for Temporal Reasoning in Long Videos","ref_index":48,"is_internal_anchor":false},{"citing_arxiv_id":"2603.22056","citing_title":"Dual-Space Knowledge Distillation with Key-Query Matching for Large Language Models with Vocabulary Mismatch","ref_index":30,"is_internal_anchor":false},{"citing_arxiv_id":"2605.17152","citing_title":"Multilingual and Multimodal LLMs in the Wild: Building for Low-Resource Languages","ref_index":126,"is_internal_anchor":false},{"citing_arxiv_id":"2408.00724","citing_title":"Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models","ref_index":276,"is_internal_anchor":false},{"citing_arxiv_id":"2406.11794","citing_title":"DataComp-LM: In search of the next generation of training sets for language models","ref_index":190,"is_internal_anchor":false},{"citing_arxiv_id":"2601.13262","citing_title":"CURE-Med: Curriculum-Informed Reinforcement Learning for Multilingual Medical Reasoning","ref_index":39,"is_internal_anchor":false},{"citing_arxiv_id":"2401.10020","citing_title":"Self-Rewarding Language Models","ref_index":36,"is_internal_anchor":false},{"citing_arxiv_id":"2212.04089","citing_title":"Editing Models with Task Arithmetic","ref_index":101,"is_internal_anchor":false},{"citing_arxiv_id":"2304.12244","citing_title":"WizardLM: Empowering large pre-trained language models to follow complex instructions","ref_index":42,"is_internal_anchor":false},{"citing_arxiv_id":"2305.10403","citing_title":"PaLM 2 Technical Report","ref_index":194,"is_internal_anchor":false},{"citing_arxiv_id":"2211.05100","citing_title":"BLOOM: A 176B-Parameter Open-Access Multilingual Language Model","ref_index":138,"is_internal_anchor":false},{"citing_arxiv_id":"2604.19015","citing_title":"FedProxy: Federated Fine-Tuning of LLMs via Proxy SLMs and Heterogeneity-Aware Fusion","ref_index":91,"is_internal_anchor":false},{"citing_arxiv_id":"2304.08485","citing_title":"Visual Instruction Tuning","ref_index":52,"is_internal_anchor":false},{"citing_arxiv_id":"2605.07783","citing_title":"Chain-based Distillation for Effective Initialization of Variable-Sized Small Language Models","ref_index":104,"is_internal_anchor":false},{"citing_arxiv_id":"2201.11903","citing_title":"Chain-of-Thought Prompting Elicits Reasoning in Large Language Models","ref_index":70,"is_internal_anchor":false},{"citing_arxiv_id":"2605.02572","citing_title":"On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Length","ref_index":79,"is_internal_anchor":false}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/K52ZZPDYVZVFALD35NHYQ5ATOU","json":"https://pith.science/pith/K52ZZPDYVZVFALD35NHYQ5ATOU.json","graph_json":"https://pith.science/api/pith-number/K52ZZPDYVZVFALD35NHYQ5ATOU/graph.json","events_json":"https://pith.science/api/pith-number/K52ZZPDYVZVFALD35NHYQ5ATOU/events.json","paper":"https://pith.science/paper/K52ZZPDY"},"agent_actions":{"view_html":"https://pith.science/pith/K52ZZPDYVZVFALD35NHYQ5ATOU","download_json":"https://pith.science/pith/K52ZZPDYVZVFALD35NHYQ5ATOU.json","view_paper":"https://pith.science/paper/K52ZZPDY","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2204.07705&json=true","fetch_graph":"https://pith.science/api/pith-number/K52ZZPDYVZVFALD35NHYQ5ATOU/graph.json","fetch_events":"https://pith.science/api/pith-number/K52ZZPDYVZVFALD35NHYQ5ATOU/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/K52ZZPDYVZVFALD35NHYQ5ATOU/action/timestamp_anchor","attest_storage":"https://pith.science/pith/K52ZZPDYVZVFALD35NHYQ5ATOU/action/storage_attestation","attest_author":"https://pith.science/pith/K52ZZPDYVZVFALD35NHYQ5ATOU/action/author_attestation","sign_citation":"https://pith.science/pith/K52ZZPDYVZVFALD35NHYQ5ATOU/action/citation_signature","submit_replication":"https://pith.science/pith/K52ZZPDYVZVFALD35NHYQ5ATOU/action/replication_record"}},"created_at":"2026-07-05T05:09:17.293118+00:00","updated_at":"2026-07-05T05:09:17.293118+00:00"}