{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2020:FZMDE3LSPVOF2LQ5KLUU6Z3LD5","short_pith_number":"pith:FZMDE3LS","schema_version":"1.0","canonical_sha256":"2e58326d727d5c5d2e1d52e94f676b1f7d9bb1e34d3edaa3506004c4be861e15","source":{"kind":"arxiv","id":"2005.14165","version":4},"attestation_state":"computed","paper":{"title":"Language Models are Few-Shot Learners","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"Scaling language models to 175 billion parameters enables strong few-shot performance on NLP tasks without any fine-tuning.","cross_cats":[],"primary_cat":"cs.CL","authors_text":"Aditya Ramesh, Alec Radford, Amanda Askell, Ariel Herbert-Voss, Arvind Neelakantan, Benjamin Chess, Benjamin Mann, Christopher Berner, Christopher Hesse, Clemens Winter, Daniel M. Ziegler, Dario Amodei, Eric Sigler, Girish Sastry, Gretchen Krueger, Ilya Sutskever, Jack Clark, Jared Kaplan, Jeffrey Wu, Mark Chen, Mateusz Litwin, Melanie Subbiah, Nick Ryder, Prafulla Dhariwal, Pranav Shyam, Rewon Child, Sam McCandlish, Sandhini Agarwal, Scott Gray, Tom B. Brown, Tom Henighan","submitted_at":"2020-05-28T17:29:03Z","abstract_excerpt":"Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do. Here we show that scaling up language models greatly improves task-agnostic, few-shot performan"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":false},"canonical_record":{"source":{"id":"2005.14165","kind":"arxiv","version":4},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.CL","submitted_at":"2020-05-28T17:29:03Z","cross_cats_sorted":[],"title_canon_sha256":"70b62cc6c025dabeba9b7459e0825b9cc8e3c74057130bb182bdfc6a5cb482f5","abstract_canon_sha256":"dd4f1ea04b346c922e20a115ded0b792e59dd2c63c094228c64c13601df96e76"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T01:21:37.693924Z","signature_b64":"+sQmkhLfV4roIl5pRLXIG6wCmlS1B/S5t//T6mqfBPLdfTqtZatGXqHRXMaEDlyY+Tj0faclvoc5bDEiQ1DUDg==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"2e58326d727d5c5d2e1d52e94f676b1f7d9bb1e34d3edaa3506004c4be861e15","last_reissued_at":"2026-07-05T01:21:37.693472Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T01:21:37.693472Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Language Models are Few-Shot Learners","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"Scaling language models to 175 billion parameters enables strong few-shot performance on NLP tasks without any fine-tuning.","cross_cats":[],"primary_cat":"cs.CL","authors_text":"Aditya Ramesh, Alec Radford, Amanda Askell, Ariel Herbert-Voss, Arvind Neelakantan, Benjamin Chess, Benjamin Mann, Christopher Berner, Christopher Hesse, Clemens Winter, Daniel M. Ziegler, Dario Amodei, Eric Sigler, Girish Sastry, Gretchen Krueger, Ilya Sutskever, Jack Clark, Jared Kaplan, Jeffrey Wu, Mark Chen, Mateusz Litwin, Melanie Subbiah, Nick Ryder, Prafulla Dhariwal, Pranav Shyam, Rewon Child, Sam McCandlish, Sandhini Agarwal, Scott Gray, Tom B. Brown, Tom Henighan","submitted_at":"2020-05-28T17:29:03Z","abstract_excerpt":"Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do. Here we show that scaling up language models greatly improves task-agnostic, few-shot performan"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches. Specifically, we train GPT-3, an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That the few-shot examples placed in the prompt allow genuine generalization rather than the model simply recalling near-duplicates from its web-scale training corpus, and that the chosen evaluation tasks are not contaminated by that corpus.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"GPT-3 shows that scaling an autoregressive language model to 175 billion parameters enables strong few-shot performance across diverse NLP tasks via in-context prompting without fine-tuning.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Scaling language models to 175 billion parameters enables strong few-shot performance on NLP tasks without any fine-tuning.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"ef1c9b14fe1538037924c494bbc2d0e399567a465fa8d1fb94560c7a7d11c863"},"source":{"id":"2005.14165","kind":"arxiv","version":4},"verdict":{"id":"3e5e044a-9f1e-4f8f-8df6-555b07a330b8","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-10T12:00:32.544002Z","strongest_claim":"scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches. Specifically, we train GPT-3, an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting.","one_line_summary":"GPT-3 shows that scaling an autoregressive language model to 175 billion parameters enables strong few-shot performance across diverse NLP tasks via in-context prompting without fine-tuning.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That the few-shot examples placed in the prompt allow genuine generalization rather than the model simply recalling near-duplicates from its web-scale training corpus, and that the chosen evaluation tasks are not contaminated by that corpus.","pith_extraction_headline":"Scaling language models to 175 billion parameters enables strong few-shot performance on NLP tasks without any fine-tuning."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2005.14165/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":86,"sample":[{"doi":"","year":2019,"title":"Massively multilingual neural machine translation","work_id":"2e1f92a4-6faa-47bb-87b0-55c2dd5d65bd","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2005,"title":"arXiv preprint arXiv:2005.14050 , year =","work_id":"df58799e-b670-4f82-bfb3-67cfb3fcceee","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2013,"title":"Semantic parsing on freebase from question-answer pairs","work_id":"9d4212d3-ae64-4b5e-aa57-c1178c245c9b","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2004,"title":"2004.10151 , archiveprefix =","work_id":"8d60e924-b221-49a3-854b-c88413eae1b7","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":1911,"title":"PIQA: Reasoning about Physical Commonsense in Natural Language","work_id":"0d865a62-6376-4606-8d3a-eeb3b6e9ba6d","ref_index":5,"cited_arxiv_id":"1911.11641","is_internal_anchor":true}],"resolved_work":86,"snapshot_sha256":"b7175e4cd3d4dd21e69fa87d77fb6dd6c13e8e9f36cd25fd58b382b3b4792b69","internal_anchors":27},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2005.14165","created_at":"2026-07-05T01:21:37.693536+00:00"},{"alias_kind":"arxiv_version","alias_value":"2005.14165v4","created_at":"2026-07-05T01:21:37.693536+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2005.14165","created_at":"2026-07-05T01:21:37.693536+00:00"},{"alias_kind":"pith_short_12","alias_value":"FZMDE3LSPVOF","created_at":"2026-07-05T01:21:37.693536+00:00"},{"alias_kind":"pith_short_16","alias_value":"FZMDE3LSPVOF2LQ5","created_at":"2026-07-05T01:21:37.693536+00:00"},{"alias_kind":"pith_short_8","alias_value":"FZMDE3LS","created_at":"2026-07-05T01:21:37.693536+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":433,"internal_anchor_count":433,"sample":[{"citing_arxiv_id":"2607.05679","citing_title":"RPAM: A Principled Metric for Evaluating Associations in Language Models with High Predictive Validity in Downstream Outputs","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2607.07916","citing_title":"Persona Cartography: Charting Language Model Personality Traits in Weight Space","ref_index":10,"is_internal_anchor":true},{"citing_arxiv_id":"2607.08028","citing_title":"From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents","ref_index":37,"is_internal_anchor":true},{"citing_arxiv_id":"2607.08071","citing_title":"COBART: Controlled, Optimized, Bidirectional and Auto-Regressive Transformer for Ad Headline Generation","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2607.07619","citing_title":"Rethinking Code Performance Benchmarks for LLMs","ref_index":80,"is_internal_anchor":true},{"citing_arxiv_id":"2607.07669","citing_title":"DiaLLM: An Investigation into the Robustness-Generation Gap in English Dialect Adaptation","ref_index":25,"is_internal_anchor":true},{"citing_arxiv_id":"2604.19679","citing_title":"MMControl: Unified Multi-Modal Control for Joint Audio-Video Generation","ref_index":1,"is_internal_anchor":true},{"citing_arxiv_id":"2606.26025","citing_title":"In-Context World Modeling for Robotic Control","ref_index":10,"is_internal_anchor":true},{"citing_arxiv_id":"2606.25329","citing_title":"State Space Models Meet Remote Sensing: A Survey","ref_index":207,"is_internal_anchor":true},{"citing_arxiv_id":"2606.24320","citing_title":"ZONOS2 Technical Report","ref_index":36,"is_internal_anchor":true},{"citing_arxiv_id":"2606.27100","citing_title":"Pretrained Time-Series Foundation Models for Financial Return Forecasting","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2606.26880","citing_title":"Heterogeneous Neural Predictivity from Language Models During Naturalistic Comprehension","ref_index":9,"is_internal_anchor":true},{"citing_arxiv_id":"2606.26025","citing_title":"In-Context World Modeling for Robotic Control","ref_index":10,"is_internal_anchor":true},{"citing_arxiv_id":"2606.26719","citing_title":"Full spectrum Unlearnable Examples via Spectral Equalization","ref_index":1,"is_internal_anchor":true},{"citing_arxiv_id":"2606.23276","citing_title":"Exposing the Illusion of Erasure in Knowledge Editing for LLMs","ref_index":6,"is_internal_anchor":true},{"citing_arxiv_id":"2606.22627","citing_title":"Orthogonal Representation Editing: Decoupling Semantic Entanglement in Batch Knowledge Editing of LLMs","ref_index":16,"is_internal_anchor":true},{"citing_arxiv_id":"2606.22610","citing_title":"PaperClaw: Harnessing Agents for Autonomous Research and Human-in-the-Loop Refinement","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2606.22394","citing_title":"Curvature-Adaptive Consistency Flow Matching: Autonomous Trajectory Optimization via Reinforcement Learning","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2606.22430","citing_title":"Words as Difference Makers: How Large Language Models Determine Causal Structure in Text","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2606.20272","citing_title":"Efficiently Linking Real Scenes with Synthetic Data Generation for AI-based Cognitive Robotics and Computer Vision Applications","ref_index":15,"is_internal_anchor":true},{"citing_arxiv_id":"2606.20369","citing_title":"CATCH-ME if you RAG: a dataset of Contextually Annotated multi-Turn Counterspeech against Hate and Misinformation Exchanges","ref_index":269,"is_internal_anchor":true},{"citing_arxiv_id":"2606.20005","citing_title":"StreamKL: Fast and Memory-Efficient KL Divergence for Boosting Attention Distillation","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2607.01238","citing_title":"SPARCLE: SPeaker-aware Aligned Representations via Contrastive Language Embeddings","ref_index":10,"is_internal_anchor":true},{"citing_arxiv_id":"2606.03305","citing_title":"The Reliability Gap in Benchmark Auditing: Distribution Shift and Scale as Failure Modes of Contamination Detection","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2605.24956","citing_title":"NITP: Next Implicit Token Prediction for LLM Pre-training","ref_index":31,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/FZMDE3LSPVOF2LQ5KLUU6Z3LD5","json":"https://pith.science/pith/FZMDE3LSPVOF2LQ5KLUU6Z3LD5.json","graph_json":"https://pith.science/api/pith-number/FZMDE3LSPVOF2LQ5KLUU6Z3LD5/graph.json","events_json":"https://pith.science/api/pith-number/FZMDE3LSPVOF2LQ5KLUU6Z3LD5/events.json","paper":"https://pith.science/paper/FZMDE3LS"},"agent_actions":{"view_html":"https://pith.science/pith/FZMDE3LSPVOF2LQ5KLUU6Z3LD5","download_json":"https://pith.science/pith/FZMDE3LSPVOF2LQ5KLUU6Z3LD5.json","view_paper":"https://pith.science/paper/FZMDE3LS","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2005.14165&json=true","fetch_graph":"https://pith.science/api/pith-number/FZMDE3LSPVOF2LQ5KLUU6Z3LD5/graph.json","fetch_events":"https://pith.science/api/pith-number/FZMDE3LSPVOF2LQ5KLUU6Z3LD5/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/FZMDE3LSPVOF2LQ5KLUU6Z3LD5/action/timestamp_anchor","attest_storage":"https://pith.science/pith/FZMDE3LSPVOF2LQ5KLUU6Z3LD5/action/storage_attestation","attest_author":"https://pith.science/pith/FZMDE3LSPVOF2LQ5KLUU6Z3LD5/action/author_attestation","sign_citation":"https://pith.science/pith/FZMDE3LSPVOF2LQ5KLUU6Z3LD5/action/citation_signature","submit_replication":"https://pith.science/pith/FZMDE3LSPVOF2LQ5KLUU6Z3LD5/action/replication_record"}},"created_at":"2026-07-05T01:21:37.693536+00:00","updated_at":"2026-07-05T01:21:37.693536+00:00"}