Checkpoint fusion guided by a 'forget' metric, followed by distillation, lets a single model recover test points that were learned and then forgotten during training, improving accuracy, especially under label noise.
Horizontal and Vertical Ensemble with Deep Representation for Classification
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Representation learning, especially which by using deep learning, has been widely applied in classification. However, how to use limited size of labeled data to achieve good classification performance with deep neural network, and how can the learned features further improve classification remain indefinite. In this paper, we propose Horizontal Voting Vertical Voting and Horizontal Stacked Ensemble methods to improve the classification performance of deep neural networks. In the ICML 2013 Black Box Challenge, via using these methods independently, Bing Xu achieved 3rd in public leaderboard, and 7th in private leaderboard; Jingjing Xie achieved 4th in public leaderboard, and 5th in private leaderboard.
citation-role summary
baseline 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
baseline 1polarities
baseline 1representative citing papers
citing papers explorer
-
Forget Me Not: Fighting Local Overfitting with Knowledge Fusion and Distillation
Checkpoint fusion guided by a 'forget' metric, followed by distillation, lets a single model recover test points that were learned and then forgotten during training, improving accuracy, especially under label noise.