{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2019:MMMH3ZIPPYIA4PRKN3P3QYSQGI","short_pith_number":"pith:MMMH3ZIP","schema_version":"1.0","canonical_sha256":"63187de50f7e100e3e2a6edfb862503203212d19850bc74f103d3ea21a3fa124","source":{"kind":"arxiv","id":"1904.09751","version":2},"attestation_state":"computed","paper":{"title":"The Curious Case of Neural Text Degeneration","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"Nucleus sampling draws from the dynamic high-probability set to generate more diverse and coherent text than beam search or top-k methods.","cross_cats":[],"primary_cat":"cs.CL","authors_text":"Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, Yejin Choi","submitted_at":"2019-04-22T07:17:18Z","abstract_excerpt":"Despite considerable advancements with deep neural language models, the enigma of neural text degeneration persists when these models are tested as text generators. The counter-intuitive empirical observation is that even though the use of likelihood as training objective leads to high quality models for a broad range of language understanding tasks, using likelihood as a decoding objective leads to text that is bland and strangely repetitive.\n  In this paper, we reveal surprising distributional differences between human text and machine text. In addition, we find that decoding strategies alon"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":true},"canonical_record":{"source":{"id":"1904.09751","kind":"arxiv","version":2},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.CL","submitted_at":"2019-04-22T07:17:18Z","cross_cats_sorted":[],"title_canon_sha256":"129d205975a35cd33688c24c44d313f3ae68ca6cbc7d1714e8be66e73effee3e","abstract_canon_sha256":"0865715a7c2b1b50ff1338569cfaabd8b7b13c833b5130585c2667f391fad78f"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T00:40:54.934801Z","signature_b64":"plEODnP7zyxBm2K0eyJ4BGI6IcT9Xnvw+1rDMSpHtQeu7h2Gxsjj+VrKxoC+21pJLc/oVoclNoP9S2bHkLxECw==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"63187de50f7e100e3e2a6edfb862503203212d19850bc74f103d3ea21a3fa124","last_reissued_at":"2026-07-05T00:40:54.934288Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T00:40:54.934288Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"The Curious Case of Neural Text Degeneration","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"Nucleus sampling draws from the dynamic high-probability set to generate more diverse and coherent text than beam search or top-k methods.","cross_cats":[],"primary_cat":"cs.CL","authors_text":"Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, Yejin Choi","submitted_at":"2019-04-22T07:17:18Z","abstract_excerpt":"Despite considerable advancements with deep neural language models, the enigma of neural text degeneration persists when these models are tested as text generators. The counter-intuitive empirical observation is that even though the use of likelihood as training objective leads to high quality models for a broad range of language understanding tasks, using likelihood as a decoding objective leads to text that is bland and strangely repetitive.\n  In this paper, we reveal surprising distributional differences between human text and machine text. In addition, we find that decoding strategies alon"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"By sampling text from the dynamic nucleus of the probability distribution, which allows for diversity while effectively truncating the less reliable tail of the distribution, the resulting text better demonstrates the quality of human text, yielding enhanced diversity without sacrificing fluency and coherence.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That the model's learned probability distribution is sufficiently well-calibrated that low-probability tokens in the tail are reliably lower quality, so truncating them improves rather than harms the output.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Nucleus sampling draws from the smallest set of tokens whose cumulative probability exceeds threshold p, yielding more human-like diversity and coherence than beam search or full-distribution sampling.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Nucleus sampling draws from the dynamic high-probability set to generate more diverse and coherent text than beam search or top-k methods.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"3b8532c10b46334fa9e442cbb9917dfb0faa1cccd3bb127d09f6208d16be5d6b"},"source":{"id":"1904.09751","kind":"arxiv","version":2},"verdict":{"id":"7fd82606-aaec-4bc9-ba7e-d74e59f4114a","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-12T06:13:35.190353Z","strongest_claim":"By sampling text from the dynamic nucleus of the probability distribution, which allows for diversity while effectively truncating the less reliable tail of the distribution, the resulting text better demonstrates the quality of human text, yielding enhanced diversity without sacrificing fluency and coherence.","one_line_summary":"Nucleus sampling draws from the smallest set of tokens whose cumulative probability exceeds threshold p, yielding more human-like diversity and coherence than beam search or full-distribution sampling.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That the model's learned probability distribution is sufficiently well-calibrated that low-probability tokens in the tail are reliably lower quality, so truncating them improves rather than harms the output.","pith_extraction_headline":"Nucleus sampling draws from the dynamic high-probability set to generate more diverse and coherent text than beam search or top-k methods."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/1904.09751/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":16,"sample":[{"doi":"","year":2015,"title":"Neural machine translation by jointly learning to align and translate","work_id":"68cc9ea9-5ed8-4ad2-9cd8-3ecaba36c66d","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2018,"title":"Language gans falling short","work_id":"7f7fd007-0f9d-4b67-8792-b16d15391b72","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2018,"title":"Language GANs falling short","work_id":"851f20c8-1e5a-4d79-8bac-46b38d0a46aa","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2018,"title":"Elizabeth Clark, Yangfeng Ji, and Noah A. Smith. Neural text generation in stories using entity rep- resentations as context. In Proceedings of the 2018 Conference of the North American Chapter of the","work_id":"eadf665e-7076-4bb7-bd5f-17e33ca5ea81","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2020,"title":"Hierarchical neural story generation","work_id":"493a14d4-4c20-40e4-90a0-6be4b605230e","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":16,"snapshot_sha256":"80b9779dd1ee31b6c918039d8cec74b03e32a904c6d45633339b09e7518b698d","internal_anchors":2},"formal_canon":{"evidence_count":2,"snapshot_sha256":"000a34424d54f510ae578f1e7c406f74e33ea795dfc02fc73bc21b058b807fa1"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"1904.09751","created_at":"2026-07-05T00:40:54.934361+00:00"},{"alias_kind":"arxiv_version","alias_value":"1904.09751v2","created_at":"2026-07-05T00:40:54.934361+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.1904.09751","created_at":"2026-07-05T00:40:54.934361+00:00"},{"alias_kind":"pith_short_12","alias_value":"MMMH3ZIPPYIA","created_at":"2026-07-05T00:40:54.934361+00:00"},{"alias_kind":"pith_short_16","alias_value":"MMMH3ZIPPYIA4PRK","created_at":"2026-07-05T00:40:54.934361+00:00"},{"alias_kind":"pith_short_8","alias_value":"MMMH3ZIP","created_at":"2026-07-05T00:40:54.934361+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":121,"internal_anchor_count":121,"sample":[{"citing_arxiv_id":"2607.05761","citing_title":"Synthetic Consumer Insight Generation with Large Language Models","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2607.07554","citing_title":"RubriQ: Rubric-Guided Group Relative Policy Optimization for Constraint-Aware Quantum Circuit Synthesis","ref_index":42,"is_internal_anchor":true},{"citing_arxiv_id":"2607.06145","citing_title":"Prompting Complexity: Shortest Prompts for Texts and Behaviors in LLMs","ref_index":14,"is_internal_anchor":true},{"citing_arxiv_id":"2606.25476","citing_title":"A Red Teaming Framework for Large Language Models: A Case Study on Faithfulness Evaluation","ref_index":78,"is_internal_anchor":true},{"citing_arxiv_id":"2606.25432","citing_title":"Brevity is the Soul of Inference Efficiency: Inducing Concision in VLMs via Data Curation","ref_index":43,"is_internal_anchor":true},{"citing_arxiv_id":"2606.24063","citing_title":"Selective Capability Unlearning in End-to-End Spoken Language Understanding","ref_index":29,"is_internal_anchor":true},{"citing_arxiv_id":"2606.27359","citing_title":"When are likely answers right? On Sequence Probability and Correctness in LLMs","ref_index":21,"is_internal_anchor":true},{"citing_arxiv_id":"2604.23443","citing_title":"Revisiting Greedy Decoding for Visual Question Answering: A Calibration Perspective","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2606.23489","citing_title":"MeshFlow: Mesh Generation with Equivariant Flow Matching","ref_index":132,"is_internal_anchor":true},{"citing_arxiv_id":"2606.21018","citing_title":"LK Jam: System Architecture and Implementation of a Real-Time Human-AI Interactive Music Generation System using Role-Aware GRU","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2606.19700","citing_title":"TerraMARS: A Domain-Adapted Small-Language-Model Pipeline for Mars Terraforming Literature","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2606.08998","citing_title":"The Token Not Taken: Sampling, State, and the Stochasticity of AI Agents","ref_index":19,"is_internal_anchor":true},{"citing_arxiv_id":"2606.18065","citing_title":"Intelligence Entropy Principle and the ADE Stability Engineering Framework","ref_index":20,"is_internal_anchor":true},{"citing_arxiv_id":"2606.15531","citing_title":"Greedy Coordinate Diffusion: Effective and Semantically Coherent Adversarial Attacks via Diffusion Guidance","ref_index":59,"is_internal_anchor":true},{"citing_arxiv_id":"2606.11805","citing_title":"TextHOI-3D: Text-to-3D Hand-Object Interaction via Discrete Multi-View Generation and Joint Mesh Optimization","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2606.11337","citing_title":"Can AI Agents Synthesize Scientific Conclusions?","ref_index":51,"is_internal_anchor":true},{"citing_arxiv_id":"2606.09587","citing_title":"Seeing the Hivemind: A Consensus-Aware Interaction Technique for Mitigating AI Homogenization","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2606.09145","citing_title":"PrivCode++: Latent-Conditioned Differentially Private Code Generation for Comprehensive Guarantees","ref_index":28,"is_internal_anchor":true},{"citing_arxiv_id":"2606.08998","citing_title":"The Token Not Taken: Sampling, State, and the Stochasticity of AI Agents","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2606.08162","citing_title":"Silent Failure in LLM Agent Systems: The Entropy Principle and the Inevitable Disorder of Autonomous Agents","ref_index":18,"is_internal_anchor":true},{"citing_arxiv_id":"2606.05658","citing_title":"Agent-Orchestrated Adaptive RAG: A Comparative Study on Structured and Multi-Hop Retrieval","ref_index":26,"is_internal_anchor":true},{"citing_arxiv_id":"2606.02093","citing_title":"The Role of Ambiguity in Error Prediction via Uncertainty Quantification","ref_index":19,"is_internal_anchor":true},{"citing_arxiv_id":"2605.23961","citing_title":"Multimodal Alignment and Preference Optimization for Zero-Shot Conditional RNA Generation","ref_index":89,"is_internal_anchor":true},{"citing_arxiv_id":"2605.08455","citing_title":"CUDABeaver: Benchmarking LLM-Based Automated CUDA Debugging","ref_index":10,"is_internal_anchor":true},{"citing_arxiv_id":"2606.25432","citing_title":"Brevity is the Soul of Inference Efficiency: Inducing Concision in VLMs via Data Curation","ref_index":42,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":2,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/MMMH3ZIPPYIA4PRKN3P3QYSQGI","json":"https://pith.science/pith/MMMH3ZIPPYIA4PRKN3P3QYSQGI.json","graph_json":"https://pith.science/api/pith-number/MMMH3ZIPPYIA4PRKN3P3QYSQGI/graph.json","events_json":"https://pith.science/api/pith-number/MMMH3ZIPPYIA4PRKN3P3QYSQGI/events.json","paper":"https://pith.science/paper/MMMH3ZIP"},"agent_actions":{"view_html":"https://pith.science/pith/MMMH3ZIPPYIA4PRKN3P3QYSQGI","download_json":"https://pith.science/pith/MMMH3ZIPPYIA4PRKN3P3QYSQGI.json","view_paper":"https://pith.science/paper/MMMH3ZIP","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=1904.09751&json=true","fetch_graph":"https://pith.science/api/pith-number/MMMH3ZIPPYIA4PRKN3P3QYSQGI/graph.json","fetch_events":"https://pith.science/api/pith-number/MMMH3ZIPPYIA4PRKN3P3QYSQGI/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/MMMH3ZIPPYIA4PRKN3P3QYSQGI/action/timestamp_anchor","attest_storage":"https://pith.science/pith/MMMH3ZIPPYIA4PRKN3P3QYSQGI/action/storage_attestation","attest_author":"https://pith.science/pith/MMMH3ZIPPYIA4PRKN3P3QYSQGI/action/author_attestation","sign_citation":"https://pith.science/pith/MMMH3ZIPPYIA4PRKN3P3QYSQGI/action/citation_signature","submit_replication":"https://pith.science/pith/MMMH3ZIPPYIA4PRKN3P3QYSQGI/action/replication_record"}},"created_at":"2026-07-05T00:40:54.934361+00:00","updated_at":"2026-07-05T00:40:54.934361+00:00"}