Replacing cross-entropy with f-divergence losses in weak-to-strong generalization gives modest accuracy gains and improved label-noise tolerance, though the paper's theoretical equivalence result is constructed after the fact.
Generalization Error of $f$-Divergence Stabilized Algorithms via Duality
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
The solution to empirical risk minimization with $f$-divergence regularization (ERM-$f$DR) is extended to constrained optimization problems, establishing conditions for equivalence between the solution and constraints. A dual formulation of ERM-$f$DR is introduced, providing a computationally efficient method to derive the normalization function of the ERM-$f$DR solution. This dual approach leverages the Legendre-Fenchel transform and the implicit function theorem, enabling explicit characterizations of the generalization error for general algorithms under mild conditions, and another for ERM-$f$DR solutions.
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
On Weak-to-Strong Generalization and f-Divergence
Replacing cross-entropy with f-divergence losses in weak-to-strong generalization gives modest accuracy gains and improved label-noise tolerance, though the paper's theoretical equivalence result is constructed after the fact.