{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2024:62JL6FYCFWPEBPXDHYXIJA6K5D","short_pith_number":"pith:62JL6FYC","schema_version":"1.0","canonical_sha256":"f692bf17022d9e40bee33e2e8483cae8f803813ee930430aa4072c035001e7f3","source":{"kind":"arxiv","id":"2404.05090","version":1},"attestation_state":"computed","paper":{"title":"How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse","license":"http://creativecommons.org/licenses/by/4.0/","headline":"","cross_cats":["cs.AI","cs.CL"],"primary_cat":"cs.LG","authors_text":"Merouane Debbah, Mohamed El Amine Seddik, Pierre Youssef, Soufiane Hayou, Suei-Wen Chen","submitted_at":"2024-04-07T22:15:13Z","abstract_excerpt":"The phenomenon of model collapse, introduced in (Shumailov et al., 2023), refers to the deterioration in performance that occurs when new models are trained on synthetic data generated from previously trained models. This recursive training loop makes the tails of the original distribution disappear, thereby making future-generation models forget about the initial (real) distribution. With the aim of rigorously understanding model collapse in language models, we consider in this paper a statistical model that allows us to characterize the impact of various recursive training scenarios. Specifi"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2404.05090","kind":"arxiv","version":1},"metadata":{"license":"http://creativecommons.org/licenses/by/4.0/","primary_cat":"cs.LG","submitted_at":"2024-04-07T22:15:13Z","cross_cats_sorted":["cs.AI","cs.CL"],"title_canon_sha256":"50e8021654e9a9f3296e63e7c76c9a90d373c101a8caa461205a5baf2b9d516f","abstract_canon_sha256":"5ec5d71756a01f2164e61ce77f93cda1b4cc7f815cbe324da7c1108b1fb95009"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T08:05:26.015035Z","signature_b64":"o67C4Z0k0ZFLqMlKn12d/E/djEx0ajbTOQNP1ZfukHmkZ1XC2/q9QAtEMsCUR5ejSZObOUXgnSyvn7saJ3c0BQ==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"f692bf17022d9e40bee33e2e8483cae8f803813ee930430aa4072c035001e7f3","last_reissued_at":"2026-07-05T08:05:26.014561Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T08:05:26.014561Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse","license":"http://creativecommons.org/licenses/by/4.0/","headline":"","cross_cats":["cs.AI","cs.CL"],"primary_cat":"cs.LG","authors_text":"Merouane Debbah, Mohamed El Amine Seddik, Pierre Youssef, Soufiane Hayou, Suei-Wen Chen","submitted_at":"2024-04-07T22:15:13Z","abstract_excerpt":"The phenomenon of model collapse, introduced in (Shumailov et al., 2023), refers to the deterioration in performance that occurs when new models are trained on synthetic data generated from previously trained models. This recursive training loop makes the tails of the original distribution disappear, thereby making future-generation models forget about the initial (real) distribution. With the aim of rigorously understanding model collapse in language models, we consider in this paper a statistical model that allows us to characterize the impact of various recursive training scenarios. Specifi"},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2404.05090","kind":"arxiv","version":1},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2404.05090/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2404.05090","created_at":"2026-07-05T08:05:26.014619+00:00"},{"alias_kind":"arxiv_version","alias_value":"2404.05090v1","created_at":"2026-07-05T08:05:26.014619+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2404.05090","created_at":"2026-07-05T08:05:26.014619+00:00"},{"alias_kind":"pith_short_12","alias_value":"62JL6FYCFWPE","created_at":"2026-07-05T08:05:26.014619+00:00"},{"alias_kind":"pith_short_16","alias_value":"62JL6FYCFWPEBPXD","created_at":"2026-07-05T08:05:26.014619+00:00"},{"alias_kind":"pith_short_8","alias_value":"62JL6FYC","created_at":"2026-07-05T08:05:26.014619+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":7,"internal_anchor_count":0,"sample":[{"citing_arxiv_id":"2606.28438","citing_title":"When AI Reviews Its Own Code: Recursive Self-Training Collapse in Code LLMs","ref_index":151,"is_internal_anchor":false},{"citing_arxiv_id":"2605.28664","citing_title":"Activation Steering for Synthetic Data Generation: The Role of Diversity in Downstream Safety Detection","ref_index":43,"is_internal_anchor":false},{"citing_arxiv_id":"2605.23054","citing_title":"Model Collapse as Cultural Evolution","ref_index":43,"is_internal_anchor":false},{"citing_arxiv_id":"2605.20602","citing_title":"Self-Training Doesn't Flatten Language -- It Restructures It: Surface Markers Amplify While Deep Syntax Dies","ref_index":22,"is_internal_anchor":false},{"citing_arxiv_id":"2604.21853","citing_title":"Generative artificial intelligence reduces social welfare through model collapse","ref_index":36,"is_internal_anchor":false},{"citing_arxiv_id":"2605.01130","citing_title":"Iterative Finetuning is Mostly Idempotent","ref_index":9,"is_internal_anchor":false},{"citing_arxiv_id":"2605.04127","citing_title":"Position: the Stochastic Parrot in the Coal Mine. Model Collapse is a Threat to Low-Resource Communities","ref_index":28,"is_internal_anchor":false}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/62JL6FYCFWPEBPXDHYXIJA6K5D","json":"https://pith.science/pith/62JL6FYCFWPEBPXDHYXIJA6K5D.json","graph_json":"https://pith.science/api/pith-number/62JL6FYCFWPEBPXDHYXIJA6K5D/graph.json","events_json":"https://pith.science/api/pith-number/62JL6FYCFWPEBPXDHYXIJA6K5D/events.json","paper":"https://pith.science/paper/62JL6FYC"},"agent_actions":{"view_html":"https://pith.science/pith/62JL6FYCFWPEBPXDHYXIJA6K5D","download_json":"https://pith.science/pith/62JL6FYCFWPEBPXDHYXIJA6K5D.json","view_paper":"https://pith.science/paper/62JL6FYC","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2404.05090&json=true","fetch_graph":"https://pith.science/api/pith-number/62JL6FYCFWPEBPXDHYXIJA6K5D/graph.json","fetch_events":"https://pith.science/api/pith-number/62JL6FYCFWPEBPXDHYXIJA6K5D/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/62JL6FYCFWPEBPXDHYXIJA6K5D/action/timestamp_anchor","attest_storage":"https://pith.science/pith/62JL6FYCFWPEBPXDHYXIJA6K5D/action/storage_attestation","attest_author":"https://pith.science/pith/62JL6FYCFWPEBPXDHYXIJA6K5D/action/author_attestation","sign_citation":"https://pith.science/pith/62JL6FYCFWPEBPXDHYXIJA6K5D/action/citation_signature","submit_replication":"https://pith.science/pith/62JL6FYCFWPEBPXDHYXIJA6K5D/action/replication_record"}},"created_at":"2026-07-05T08:05:26.014619+00:00","updated_at":"2026-07-05T08:05:26.014619+00:00"}