REVIEW 2 cited by
Tighter Information-Theoretic Generalization Bounds from Supersamples
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In this work, we present a variety of novel information-theoretic generalization bounds for learning algorithms, from the supersample setting of Steinke & Zakynthinou (2020)-the setting of the "conditional mutual information" framework. Our development exploits projecting the loss pair (obtained from a training instance and a testing instance) down to a single number and correlating loss values with a Rademacher sequence (and its shifted variants). The presented bounds include square-root bounds, fast-rate bounds, including those based on variance and sharpness, and bounds for interpolating algorithms etc. We show theoretically or empirically that these bounds are tighter than all information-theoretic bounds known to date on the same supersample setting.
Forward citations
Cited by 2 Pith papers
-
Fairness Overfitting in Machine Learning: An Information-Theoretic Perspective
Claims computable MI/CMI bounds on fairness generalization error, but the core derivation uses an invalid variance-based Hoeffding step.
-
Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis
The authors derive information-theoretic generalization bounds for VAEs and diffusion models that expose a trade-off in the diffusion time T, and propose using the computable bound to select T and regularize training.
Discussion (0). Continue with ORCID to comment.