RECORD
Sample-Conditioned Hypothesis Stability Sharpens Information-Theoretic Generalization Bounds
Machine Learning · Submitted 31 Oct 2023
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Our approach yields sharper bounds that improve upon previous information-theoretic bounds in various learning scenarios.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
We present new information-theoretic generalization guarantees through the a novel construction of the "neighboring-hypothesis" matrix and a new family of stability notions termed sample-conditioned hypothesis (SCH) stability. Our approach yields sharper bounds that improve upon previous information-theoretic bounds in various learning scenarios. Notably, these bounds address the limitations of existing information-theoretic bounds in the context of stochastic convex optimization (SCO) problems, as explored in the recent work by Haghifam et al. (2023).
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | We present new information-theoretic generalization guarantees through the a novel construction of the "neighboring-hypothesis" matrix and a new family of stability notions termed sample-conditioned hypothesis (SCH) stability. | contribution 0.99 | 0.96 | 0.48 | asserted 1.00 |
| s1 | Our approach yields sharper bounds that improve upon previous information-theoretic bounds in various learning scenarios. | result 0.94 | 0.91 | 0.47 | asserted 1.00 |
| s2 | Notably, these bounds address the limitations of existing information-theoretic bounds in the context of stochastic convex optimization (SCO) problems, as explored in the recent work by Haghifam et al. (2023). | gap 0.47 | 0.41 | 0.12 | asserted 0.99 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
We present new information-theoretic generalization guarantees through the a novel construction of the "neighboring-hypothesis" matrix and a new family of stability notions termed sample-conditioned hypothesis (SCH) stability. Our approach yields sharper bounds that improve upon previous information-theoretic bounds in various learning scenarios. Notably, these bounds address the limitations of existing information-theoretic bounds in the context of stochastic convex optimization (SCO) problems, as explored in the recent work by Haghifam et al. (2023).
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 7 sentences without a confident label
Cites as support 1 paper
-
These failures stem from the dimension-dependent nature of current information-theoretic quantities [ref] , appearing an intrinsic barrier for overcoming these limitations.
Information Theoretic Lower Bounds for Information Theoretic Upper Bounds · Introduction · 0.98
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.