ODTE builds a bagging ensemble of SVM-based oblique decision trees that handle multiclass problems directly at each node and reports competitive accuracy with more compact trees than alternatives.
Consistency of Oblique Decision Tree and its Boosting and Random Forest
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Classification and Regression Tree (CART), Random Forest (RF) and Gradient Boosting Tree (GBT) are probably the most popular set of statistical learning methods. However, their statistical consistency can only be proved under very restrictive assumptions on the underlying regression function. As an extension to standard CART, the oblique decision tree (ODT), which uses linear combinations of predictors as partitioning variables, has received much attention. ODT tends to perform numerically better than CART and requires fewer partitions. In this paper, we show that ODT is consistent for very general regression functions as long as they are $L^2$ integrable. Then, we prove the consistency of the ODT-based random forest (ODRF), whether fully grown or not. Finally, we propose an ensemble of GBT for regression by borrowing the technique of orthogonal matching pursuit and study its consistency under very mild conditions on the tree structure. After refining existing computer packages according to the established theory, extensive experiments on real data sets show that both our ensemble boosting trees and ODRF have noticeable overall improvements over RF and other forests.
fields
cs.LG 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
ODTE -- An ensemble of multi-class SVM-based oblique decision trees
ODTE builds a bagging ensemble of SVM-based oblique decision trees that handle multiclass problems directly at each node and reports competitive accuracy with more compact trees than alternatives.