{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2021:XAR7ZRDSJJJ7TWLZ4LXLLIT572","short_pith_number":"pith:XAR7ZRDS","schema_version":"1.0","canonical_sha256":"b823fcc4724a53f9d979e2eeb5a27dfe87fba7ddacfa6c5ab32d9e3501f78fd2","source":{"kind":"arxiv","id":"2109.00859","version":1},"attestation_state":"computed","paper":{"title":"CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation","license":"http://creativecommons.org/licenses/by-nc-sa/4.0/","headline":"CodeT5 is a unified encoder-decoder model that pre-trains by distinguishing and recovering developer-assigned identifiers to handle both code understanding and generation.","cross_cats":["cs.PL"],"primary_cat":"cs.CL","authors_text":"Shafiq Joty, Steven C.H. Hoi, Weishi Wang, Yue Wang","submitted_at":"2021-09-02T12:21:06Z","abstract_excerpt":"Pre-trained models for Natural Languages (NL) like BERT and GPT have been recently shown to transfer well to Programming Languages (PL) and largely benefit a broad set of code-related tasks. Despite their success, most current methods either rely on an encoder-only (or decoder-only) pre-training that is suboptimal for generation (resp. understanding) tasks or process the code snippet in the same way as NL, neglecting the special characteristics of PL such as token types. We present CodeT5, a unified pre-trained encoder-decoder Transformer model that better leverages the code semantics conveyed"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":true},"canonical_record":{"source":{"id":"2109.00859","kind":"arxiv","version":1},"metadata":{"license":"http://creativecommons.org/licenses/by-nc-sa/4.0/","primary_cat":"cs.CL","submitted_at":"2021-09-02T12:21:06Z","cross_cats_sorted":["cs.PL"],"title_canon_sha256":"8ad9c618477e458d6c6bbac1583f178e2fd6d63a222bf29a7dd18c5485e4ad34","abstract_canon_sha256":"6a223f9984373fbed5eb2a35710c6cdf294e97eb542a5e7ab4dcf9dea6aa93de"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-17T23:38:52.686919Z","signature_b64":"9AaocKgxGX2aSArlEXbv3acxIACMXuTgf8zo0Nn2eO3Gqb0+VrwHnoucCgB5yXV9PDgbEb0/BwvzwS6ct9r6AA==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"b823fcc4724a53f9d979e2eeb5a27dfe87fba7ddacfa6c5ab32d9e3501f78fd2","last_reissued_at":"2026-05-17T23:38:52.686273Z","signature_status":"signed_v1","first_computed_at":"2026-05-17T23:38:52.686273Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation","license":"http://creativecommons.org/licenses/by-nc-sa/4.0/","headline":"CodeT5 is a unified encoder-decoder model that pre-trains by distinguishing and recovering developer-assigned identifiers to handle both code understanding and generation.","cross_cats":["cs.PL"],"primary_cat":"cs.CL","authors_text":"Shafiq Joty, Steven C.H. Hoi, Weishi Wang, Yue Wang","submitted_at":"2021-09-02T12:21:06Z","abstract_excerpt":"Pre-trained models for Natural Languages (NL) like BERT and GPT have been recently shown to transfer well to Programming Languages (PL) and largely benefit a broad set of code-related tasks. Despite their success, most current methods either rely on an encoder-only (or decoder-only) pre-training that is suboptimal for generation (resp. understanding) tasks or process the code snippet in the same way as NL, neglecting the special characteristics of PL such as token types. We present CodeT5, a unified pre-trained encoder-decoder Transformer model that better leverages the code semantics conveyed"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"CodeT5 significantly outperforms prior methods on understanding tasks such as code defect detection and clone detection, and generation tasks across various directions including PL-NL, NL-PL, and PL-PL.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That the identifier-aware pre-training objective and bimodal dual generation task provide gains that generalize beyond the specific datasets and fine-tuning regimes used in the experiments.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"CodeT5 adds identifier-aware pre-training and bimodal dual generation to a T5-style encoder-decoder, yielding better results on defect detection, clone detection, and code-to-text, text-to-code, and code-to-code tasks than prior encoder-only or decoder-only models.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"CodeT5 is a unified encoder-decoder model that pre-trains by distinguishing and recovering developer-assigned identifiers to handle both code understanding and generation.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"01359dca554300b58db5ce56bb79ae37589921efa0b603c9f70fb54c927668a5"},"source":{"id":"2109.00859","kind":"arxiv","version":1},"verdict":{"id":"476c61c6-91d8-4d3c-b088-066d51e202c6","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-15T11:17:47.127871Z","strongest_claim":"CodeT5 significantly outperforms prior methods on understanding tasks such as code defect detection and clone detection, and generation tasks across various directions including PL-NL, NL-PL, and PL-PL.","one_line_summary":"CodeT5 adds identifier-aware pre-training and bimodal dual generation to a T5-style encoder-decoder, yielding better results on defect detection, clone detection, and code-to-text, text-to-code, and code-to-code tasks than prior encoder-only or decoder-only models.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That the identifier-aware pre-training objective and bimodal dual generation task provide gains that generalize beyond the specific datasets and fine-tuning regimes used in the experiments.","pith_extraction_headline":"CodeT5 is a unified encoder-decoder model that pre-trains by distinguishing and recovering developer-assigned identifiers to handle both code understanding and generation."},"references":{"count":71,"sample":[{"doi":"","year":2021,"title":"Evaluating Large Language Models Trained on Code","work_id":"042493e9-b26f-4b4e-bbde-382072ca9b08","ref_index":2,"cited_arxiv_id":"2107.03374","is_internal_anchor":true},{"doi":"","year":2020,"title":"Le, and Christopher D","work_id":"2ba3b167-5cf6-4376-8f17-c4b4d2000855","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2019,"title":"Alexis Conneau and Guillaume Lample. 2019. https://proceedings.neurips.cc/paper/2019/hash/c04c19c2c2474dbf5f7ac4372c5b9af1-Abstract.html Cross-lingual language model pretraining . In Advances in Neura","work_id":"dafc5cba-d2cd-46b4-b55b-9857ed292628","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2019,"title":"Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://www.aclweb.org/anthology/N19-1423/ BERT: pre-training of deep bidirectional transformers for language understanding . ","work_id":"c8ee5216-3da7-4e76-9e54-2884e94b0843","ref_index":7,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2019,"title":"Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao - Wuen Hon. 2019. https://proceedings.neurips.cc/paper/2019/hash/c20bb2d9a50d5ac1f713f8b34d9aac5a-Ab","work_id":"c1fd755e-24bf-43e0-9d72-0fcc0ecae6d6","ref_index":8,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":71,"snapshot_sha256":"b465d4b142b7947c03cf4816a2a04985a1676afbd3c9a37382ef134846b0026e","internal_anchors":7},"formal_canon":{"evidence_count":2,"snapshot_sha256":"39eb52324363d9af808beca10b7fc45258b263c376b7dbb84d33b61060b0a39c"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2109.00859","created_at":"2026-05-17T23:38:52.686382+00:00"},{"alias_kind":"arxiv_version","alias_value":"2109.00859v1","created_at":"2026-05-17T23:38:52.686382+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2109.00859","created_at":"2026-05-17T23:38:52.686382+00:00"},{"alias_kind":"pith_short_12","alias_value":"XAR7ZRDSJJJ7","created_at":"2026-05-18T12:33:33.725879+00:00"},{"alias_kind":"pith_short_16","alias_value":"XAR7ZRDSJJJ7TWLZ","created_at":"2026-05-18T12:33:33.725879+00:00"},{"alias_kind":"pith_short_8","alias_value":"XAR7ZRDS","created_at":"2026-05-18T12:33:33.725879+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":41,"internal_anchor_count":41,"sample":[{"citing_arxiv_id":"2305.12138","citing_title":"Exploring Code Analysis: Zero-Shot Insights on Syntax and Semantics with LLMs","ref_index":90,"is_internal_anchor":true},{"citing_arxiv_id":"2402.01411","citing_title":"CodePori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology","ref_index":59,"is_internal_anchor":true},{"citing_arxiv_id":"2410.22240","citing_title":"Are Decoder-Only Large Language Models the Silver Bullet for Code Search?","ref_index":34,"is_internal_anchor":true},{"citing_arxiv_id":"2503.12374","citing_title":"Beyond Final Code: A Process-Oriented Error Analysis of Software Development Agents in Real-World GitHub Scenarios","ref_index":68,"is_internal_anchor":true},{"citing_arxiv_id":"2504.16584","citing_title":"Case Study: Fine-tuning Small Language Models for Accurate and Private CWE Detection in Python Code","ref_index":25,"is_internal_anchor":true},{"citing_arxiv_id":"2505.13766","citing_title":"A Blueprint for AI-Driven Software Quality: Integrating LLMs with Established Standards","ref_index":22,"is_internal_anchor":true},{"citing_arxiv_id":"2509.20881","citing_title":"PseudoBridge: Pseudo Code as the Bridge for Better Semantic and Logic Alignment in Code Retrieval","ref_index":48,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16046","citing_title":"XSearch: Explainable Code Search via Concept-to-Code Alignment","ref_index":86,"is_internal_anchor":true},{"citing_arxiv_id":"2605.17458","citing_title":"ClaHF: A Human Feedback-inspired Reinforcement Learning Framework for Improving Classification Tasks","ref_index":38,"is_internal_anchor":true},{"citing_arxiv_id":"2605.18073","citing_title":"A-ProS: Towards Reliable Autonomous Programming Through Multi-Model Feedback","ref_index":73,"is_internal_anchor":true},{"citing_arxiv_id":"2307.06435","citing_title":"A Comprehensive Overview of Large Language Models","ref_index":146,"is_internal_anchor":true},{"citing_arxiv_id":"2507.21954","citing_title":"Fine-Tuning Code Language Models to Detect Cross-Language Bugs","ref_index":65,"is_internal_anchor":true},{"citing_arxiv_id":"2508.20086","citing_title":"Detecting Malicious Intents in Smart Contracts with Pre-trained Programming Language Models","ref_index":23,"is_internal_anchor":true},{"citing_arxiv_id":"2510.04166","citing_title":"Multi Language Models for On-the-Fly Syntax Highlighting","ref_index":10,"is_internal_anchor":true},{"citing_arxiv_id":"2511.05476","citing_title":"A Metamorphic Testing Perspective on Knowledge Distillation for Language Models of Code: Does the Student Deeply Mimic the Teacher?","ref_index":25,"is_internal_anchor":true},{"citing_arxiv_id":"2601.02438","citing_title":"Focus on What Matters: Fisher-Guided Adaptive Multimodal Fusion for Vulnerability Detection","ref_index":35,"is_internal_anchor":true},{"citing_arxiv_id":"2601.02602","citing_title":"SWaRL: Safeguard Code Watermarking via Reinforcement Learning","ref_index":10,"is_internal_anchor":true},{"citing_arxiv_id":"2601.15232","citing_title":"When Agents Fail: A Comprehensive Study of Bugs in LLM Agents with Automated Labeling","ref_index":95,"is_internal_anchor":true},{"citing_arxiv_id":"2605.14055","citing_title":"PEML: Parameter-efficient Multi-Task Learning with Optimized Continuous Prompts","ref_index":76,"is_internal_anchor":true},{"citing_arxiv_id":"2604.02702","citing_title":"TypePro: Boosting LLM-Based Type Inference via Inter-Procedural Slicing","ref_index":33,"is_internal_anchor":true},{"citing_arxiv_id":"2605.04000","citing_title":"Mitigating False Positives in Static Memory Safety Analysis of Rust Programs via Reinforcement Learning","ref_index":59,"is_internal_anchor":true},{"citing_arxiv_id":"2604.26523","citing_title":"RepoDoc: A Knowledge Graph-Based Framework to Automatic Documentation Generation and Incremental Updates","ref_index":32,"is_internal_anchor":true},{"citing_arxiv_id":"2605.00369","citing_title":"InvEvolve: Evolving White-Box Inventory Policies via Large Language Models with Performance Guarantees","ref_index":64,"is_internal_anchor":true},{"citing_arxiv_id":"2604.25599","citing_title":"PLMGH: What Matters in PLM-GNN Hybrids for Code Classification and Vulnerability Detection","ref_index":32,"is_internal_anchor":true},{"citing_arxiv_id":"2605.05949","citing_title":"MAS-Algorithm: A Workflow for Solving Algorithmic Programming Problems with a Multi-Agent System","ref_index":17,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":2,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/XAR7ZRDSJJJ7TWLZ4LXLLIT572","json":"https://pith.science/pith/XAR7ZRDSJJJ7TWLZ4LXLLIT572.json","graph_json":"https://pith.science/api/pith-number/XAR7ZRDSJJJ7TWLZ4LXLLIT572/graph.json","events_json":"https://pith.science/api/pith-number/XAR7ZRDSJJJ7TWLZ4LXLLIT572/events.json","paper":"https://pith.science/paper/XAR7ZRDS"},"agent_actions":{"view_html":"https://pith.science/pith/XAR7ZRDSJJJ7TWLZ4LXLLIT572","download_json":"https://pith.science/pith/XAR7ZRDSJJJ7TWLZ4LXLLIT572.json","view_paper":"https://pith.science/paper/XAR7ZRDS","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2109.00859&json=true","fetch_graph":"https://pith.science/api/pith-number/XAR7ZRDSJJJ7TWLZ4LXLLIT572/graph.json","fetch_events":"https://pith.science/api/pith-number/XAR7ZRDSJJJ7TWLZ4LXLLIT572/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/XAR7ZRDSJJJ7TWLZ4LXLLIT572/action/timestamp_anchor","attest_storage":"https://pith.science/pith/XAR7ZRDSJJJ7TWLZ4LXLLIT572/action/storage_attestation","attest_author":"https://pith.science/pith/XAR7ZRDSJJJ7TWLZ4LXLLIT572/action/author_attestation","sign_citation":"https://pith.science/pith/XAR7ZRDSJJJ7TWLZ4LXLLIT572/action/citation_signature","submit_replication":"https://pith.science/pith/XAR7ZRDSJJJ7TWLZ4LXLLIT572/action/replication_record"}},"created_at":"2026-05-17T23:38:52.686382+00:00","updated_at":"2026-05-17T23:38:52.686382+00:00"}