RECORD
Robust Loss Functions for Training Decision Trees with Noisy Labels
Machine Learning · Submitted 20 Dec 2023
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Lastly, our experiments on multiple datasets and noise settings validate our theoretical insight and the effectiveness of our adaptive negative exponential loss.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
We consider training decision trees using noisily labeled data, focusing on loss functions that can lead to robust learning algorithms. Our contributions are threefold. First, we offer novel theoretical insights on the robustness of many existing loss functions in the context of decision tree learning. We show that some of the losses belong to a class of what we call conservative losses, and the conservative losses lead to an early stopping behavior during training and noise-tolerant predictions during testing. Second, we introduce a framework for constructing robust loss functions, called distribution losses. These losses apply percentile-based penalties based on an assumed margin distribution, and they naturally allow adapting to different noise rates via a robustness parameter. In particular, we introduce a new loss called the negative exponential loss, which leads to an efficient greedy impurity-reduction learning algorithm. Lastly, our experiments on multiple datasets and noise settings validate our theoretical insight and the effectiveness of our adaptive negative exponential loss.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | We consider training decision trees using noisily labeled data, focusing on loss functions that can lead to robust learning algorithms. | background 0.97 | 0.27 | 0.05 | asserted 0.68 |
| s1 | Our contributions are threefold. | contribution 0.99 | 0.56 | 0.02 | asserted 0.76 |
| s2 | First, we offer novel theoretical insights on the robustness of many existing loss functions in the context of decision tree learning. | contribution 1.00 | 0.88 | 0.07 | asserted 0.98 |
| s3 | We show that some of the losses belong to a class of what we call conservative losses, and the conservative losses lead to an early stopping behavior during training and noise-tolerant predictions during testing. | result 0.64 | 0.90 | 0.23 | asserted 1.00 |
| s4 | Second, we introduce a framework for constructing robust loss functions, called distribution losses. | contribution 0.99 | 0.95 | 0.06 | asserted 1.00 |
| s5 | These losses apply percentile-based penalties based on an assumed margin distribution, and they naturally allow adapting to different noise rates via a robustness parameter. | method 0.69 | 0.81 | 0.07 | asserted 0.99 |
| s6 | In particular, we introduce a new loss called the negative exponential loss, which leads to an efficient greedy impurity-reduction learning algorithm. | contribution 0.92 | 0.94 | 0.09 | asserted 1.00 |
| s7 | Lastly, our experiments on multiple datasets and noise settings validate our theoretical insight and the effectiveness of our adaptive negative exponential loss. | result 0.58 | 0.97 | 0.64 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v2
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
We consider training decision trees using noisily labeled data, focusing on loss functions that can lead to robust learning algorithms. Our contributions are threefold. First, we offer novel theoretical insights on the robustness of many existing loss functions in the context of decision tree learning. We show that some of the losses belong to a class of what we call conservative losses, and the conservative losses lead to an early stopping behavior during training and noise-tolerant predictions during testing. Second, we introduce a framework for constructing robust loss functions, called distribution losses. These losses apply percentile-based penalties based on an assumed margin distribution, and they naturally allow adapting to different noise rates via a robustness parameter. In particular, we introduce a new loss called the negative exponential loss, which leads to an efficient greedy impurity-reduction learning algorithm. Lastly, our experiments on multiple datasets and noise settings validate our theoretical insight and the effectiveness of our adaptive negative exponential loss.
Human review
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.