{"paper":{"title":"Information Theory and Statistical Learning","license":"http://creativecommons.org/licenses/by-nc-nd/4.0/","headline":"Divergence measures unify training objectives across regression, autoencoders, GANs, and diffusion models.","cross_cats":["eess.SP","math.IT","stat.ML"],"primary_cat":"cs.IT","authors_text":"Abbas El Gamal","submitted_at":"2026-05-04T16:52:14Z","abstract_excerpt":"This manuscript contains preprint of a chapter under consideration for inclusion in the forthcoming third edition of {\\em Cover and Thomas's Elements of Information Theory}, posted with permission from Wiley. The table of contents EIT-3 ToC of the new edition can be found at: https://docs.google.com/document/d/1L-m4oQEJw1PJhoxBeMwrrBD8S_HmvzMEkPbYvS24980/edit?usp=sharing . For feedback, please contact abbas@ee.stanford.edu\n  Learning and information theory intersect in both model training and the characterization of fundamental performance limits. This manuscript provides a concise and accessi"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"The treatment of the generative diffusion model provides a more systematic and explicit derivation than is typical in the literature.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The reader possesses only basic background in information theory and statistics at the senior undergraduate or first-year graduate level.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"The chapter gives an accessible overview of divergence measures in statistical learning, covering ELBO, f-divergences, Fisher divergence, and a systematic derivation for diffusion models.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Divergence measures unify training objectives across regression, autoencoders, GANs, and diffusion models.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"b8903959ab3fafe2aa2da904b47f1dfd8bbba49705f6235433e75b34dc593221"},"source":{"id":"2605.02989","kind":"arxiv","version":2},"verdict":{"id":"40a8ed0f-b18b-4f3d-9aa5-d21a6dcff560","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-08T17:43:37.374025Z","strongest_claim":"The treatment of the generative diffusion model provides a more systematic and explicit derivation than is typical in the literature.","one_line_summary":"The chapter gives an accessible overview of divergence measures in statistical learning, covering ELBO, f-divergences, Fisher divergence, and a systematic derivation for diffusion models.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The reader possesses only basic background in information theory and statistics at the senior undergraduate or first-year graduate level.","pith_extraction_headline":"Divergence measures unify training objectives across regression, autoencoders, GANs, and diffusion models."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2605.02989/integrity.json","findings":[],"available":true,"detectors_run":[{"name":"ai_meta_artifact","ran_at":"2026-05-20T15:33:35.768217Z","status":"completed","version":"1.0.0","findings_count":0}],"snapshot_sha256":"5d9e74b8bd1531063ea825e8842f93cca5da44d13b62f6a258d092a46cdfd566"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":3,"snapshot_sha256":"00882553aacae9cc1eee90fc918985e9af82f3270916dcdc0f02d7f9a9712d2f"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}