{"id":"bfc30831-730a-41dd-a4e2-8d56616839dd","arxiv_id":"2411.13376","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"ODTE builds a bagging ensemble of SVM-based oblique decision trees that handle multiclass problems directly at each node and reports competitive accuracy with more compact trees than alternatives.","lead":"This paper proposes a new ensemble classifier, ODTE, whose decision trees use support vector machines to create diagonal (oblique) splits, and introduces STree as the tree-building algorithm. A reader might care because the method claims better accuracy than several established tree ensembles on 49 tabular datasets, with smaller trees and free code.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Tuned-significance claim rests on an unstated tuning protocol: if ODTET's gridsearch selected hyperparameters on the full datasets before 10x5CV, its rank 1.35 is optimistically biased and the Holm-significant edge may not reflect algorithm quality.","rationale":"The reader's weakest_assumption is real: tuning effort is not matched across algorithms. I partially agree, but the sharper issue is the unstated selection protocol: no nested CV is described, so the tuned result may be double-dipped. This is more concrete than search-quality asymmetry and is checkable from the released code. I do not see a correctness flaw in STree or ODTE; the default comparison is reported honestly, with most differences non-significant, and code and data are available. The compactness result uses averaged normalized size and lacks error bars, but it is a secondary claim. Hence the appropriate verdict remains conditional: the algorithm and default experiments are acceptable, but the tuned significance claim should be verified or softened.","tokens_in":22749,"tokens_out":8756,"duration_ms":88437,"concrete_test":"Inspect the released ODTE/STree code and the per-dataset hyperparameter artifact (https://t.ly/rEZLH): determine whether the gridsearch used the full dataset or only training folds of the 10x5CV. If full-data selection is present, rerun Table 4 with nested GridSearchCV inside each of the 50 training folds for ODTET, and rerun TBRaFT, TBRoFT, and TBRRoFT with the same grid search range and budget over the same folds. If ODTET's Friedman rank moves away from 1.35 or any Holm p-value crosses 0.05, the paper should state that the significant tuned advantage is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim has two parts: default ODTE ranks above competitors, and tuned ODTET significantly beats tuned baselines. The first is honestly reported and mostly not significant in Table 5; the second is the load-bearing result. Section 4.2 says ODTET's hyperparameters were optimized for each dataset using a gridsearch, but it never states whether that gridsearch was nested inside the 10x5CV loop, what the search space or budget was, or how the best configuration was selected. Because Table 4 reports 10x5CV accuracy on the same 49 datasets, any full-data hyperparameter selection leaks test-fold information. The tuned baselines TBRaFT, TBRoFT, and TBRRoFT were not given an identically scoped fresh search; they reuse hyperparameters selected in the original authors' code. Even if those values were themselves tuned, the comparison controls neither search budget nor selection protocol. Table 6's Holm p-values (7.7e-4 to 8.0e-15) therefore cannot be read as evidence that ODTET dominates the baselines as algorithms; they may reflect a more favorable, or more leak-prone, tuning procedure. The default-setting results, compactness claim, and code release remain valuable; the tuned-significance conclusion is what needs a caveat or a rerun.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ODTE, a bagging ensemble of oblique decision trees, and STree, a single oblique tree whose splits are SVM classifiers selected via one-vs-one or one-vs-rest multiclass embeddings. The method is evaluated on 49 datasets against several oblique-tree ensembles and standard classifiers under 10x5 cross-validation. In the default configuration, ODTE obtains the best average accuracy (0.8391) and best Friedman rank (3.88), though Holm post-hoc tests show significance only against XGBoost and TBRoF. In the tuned configuration, ODTET has rank 1.35 and differs significantly from all tuned baselines. The paper also reports that ODTE trees are substantially more compact than competitors, and it releases code and datasets for reproducibility.","tokens_in":23026,"tokens_out":1909,"duration_ms":22160,"significance":"If the empirical claims hold, the contribution is a useful, practically integrated oblique-tree ensemble with a simple multiclass handling scheme and a compact-tree property. The paper deserves credit for a standard evaluation protocol (10x5cv with shared seeds, Friedman plus Holm post hoc), a large benchmark, and public code and data. The default-setting results are reported honestly, including non-significant comparisons. The load-bearing strength of the paper, however, is the tuned comparison, and that part rests on an incompletely specified and asymmetric tuning protocol, which currently prevents the reader from interpreting the Holm-significant results as evidence of algorithmic superiority. The compactness result is interesting but is presented without statistical testing.","major_comments":[{"comment":"The manuscript does not state whether the gridsearch for ODTET hyperparameters was nested inside each 10x5CV fold or performed on the full dataset before cross-validation. Since Table 4 reports 10x5CV accuracy on the same 49 datasets, any full-data hyperparameter selection would leak test-fold information and bias ODTET's accuracy, rank, and Holm p-values. The authors should specify the exact selection protocol; if selection used full datasets, the tuned comparison must be rerun with nested or fold-specific tuning.","section":"Section 4.2 and Table 4"},{"comment":"The tuning effort is asymmetric across algorithms: ODTET receives a fresh gridsearch over STree's hyperparameters per dataset, while TBRaFT, TBRoFT, and TBRRoFT reuse hyperparameter values selected by the original authors in their provided code. No search budget, search space, or model-selection criterion is reported, so the Holm p-values in Table 6 do not establish that ODTET dominates these algorithms as algorithms. A matched tuning protocol, or at least a sensitivity analysis with a common hyperparameter budget, is needed to support the 'significant performance gains when hyperparameters are carefully tuned' claim.","section":"Section 4.2 and Table 6"},{"comment":"In the default setting, ODTE's rank advantage over TBRRoF is 3.88 versus 3.95 with Holm p = 1.0, and ODTE is not significantly better than most baselines; only XGBoost and TBRoF are clearly outperformed. The abstract's phrasing 'ranks consistently above its competitors' is defensible as a rank statement, but the conclusion in Section 5 that ODTE 'emerges as the outstanding algorithm in the comparison' overstates the default-setting evidence. The language should be moderated to reflect that the default advantage is largely not statistically significant.","section":"Section 4.4 and Table 5"}],"minor_comments":[{"comment":"Table 1 lists 'oocytes merluccius nucleus 4d' and similar fishery datasets; a brief reference or description of these datasets beyond the pointer to Ganaie et al. (2020) would improve self-containedness.","section":"Section 4.1 / Table 1"},{"comment":"The pseudocode says 'I(·) is an information theory meassure'; the spelling should be 'measure'.","section":"Algorithm 2, line 9"},{"comment":"The URLs for tuned hyperparameters and supplementary materials are short links; expanding them or placing them in a stable repository reference would improve reproducibility.","section":"Section 4.2"},{"comment":"Table 7 reports normalized tree sizes and training times without variance or statistical testing; a confidence interval or at least a per-dataset breakdown would help assess the stability of the compactness claim.","section":"Section 4.4"},{"comment":"The sentence 'there is no doubt that ODTE emerges as the outstanding algorithm' is too strong given the default-setting significance results; please soften it or qualify it as applying primarily to the tuned scenario after the tuning-protocol issue is resolved.","section":"Section 5"},{"comment":"There are several typographical errors, e.g., 'facilitte' in the introduction, 'bewteen' in Section 4.4, and 'SckitLearn' in Section 1; a careful proofreading pass is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The core methodological innovation (multiclass SVM embedding at each node with impurity-based selection) is reasonable and the default evaluation is honest. The central weakness is the tuned comparison: the protocol is under-specified and asymmetric, which directly affects the strongest claim. I would ask for a rerun or at least a clearly nested tuning protocol and a matched search budget before accepting the significance claim. The paper is otherwise publishable after revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: ODTE is a bagging ensemble of oblique decision trees in which each node trains several SVMs under an OvO or OvR multiclass decomposition and keeps the single SVM with the highest information gain. That node-level selection is the genuinely new piece. The paper is otherwise a conventional empirical comparison: 49 datasets, 10x5CV with shared seeds, Friedman plus Holm post hoc tests. The default-setting result is modest and honestly reported — ODTE has the best Friedman rank (3.88) but is not significantly better than most baselines, and the paper shows that in the p-values it leaves non-boldfaced. That is a fair presentation, not an overclaim.\n\nStrengths: the method is described clearly, the code and data are released, and the benchmark follows a standard protocol. The compactness result (ODTE trees are on average several times smaller than the competitors') is a real point in its favor, even if it comes without error bars and is tangled up with implementation language. The coverage of related work is adequate, and the self-citation of the 2021 STree paper is not a problem.\n\nThe soft spot is the tuned comparison. The abstract and conclusion lean on 'significant performance gains when hyperparameters are carefully tuned,' and Table 6 has ODTET significantly ahead of all tuned baselines. But Section 4.2 says ODTET's hyperparameters were chosen per dataset by a gridsearch, without saying whether that search was nested inside the 10x5CV. The tuned baselines just reuse hyperparameters selected by the original authors in their provided code. That is an asymmetric comparison: the search budget and selection protocol are not matched, and if the gridsearch ran on the full datasets, the test-fold information leaks into the choice. So the Holm-significant ranking in Table 6 should not be read as evidence that ODTET is a better algorithm. It is a claim about a tuning procedure, and the procedure is not described well enough to rule out leakage.\n\nThe default results, the compactness observation, and the released code carry the paper. I would not cite the tuned-significance result as established. For a reader working on oblique trees or tabular classification, the STree construction and the honest default analysis are worth a look.\n\nRecommendation: send it to peer review. A referee should ask for a nested-CV clarification and a matched tuning protocol before the tuned claim can stand. The core algorithm is sound and the paper is largely reproducible, so it is a fixable issue, not a reject.","headline":"Default results are honestly modest; the tuned-significance claim rests on an asymmetric and possibly leaky tuning protocol.","tokens_in":23591,"tokens_out":3007,"would_cite":false,"duration_ms":29530,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ODTE, a bagging ensemble of SVM-based oblique decision trees, claims top accuracy and more compact trees than competing oblique ensembles and state-of-the-art classifiers on 49 datasets.","keywords":["oblique decision trees","support vector machines","ensemble learning","bagging","multiclass classification","one-vs-one","one-vs-rest","hyperparameter tuning"],"falsifier":"Re-run the ten-times five-fold cross-validation with the same ten seeds, giving each tuned baseline a grid search over the same hyperparameter ranges and number of configurations as ODTET. If ODTET no longer holds the best average rank or the adjusted post-hoc p-values climb above $\\alpha = 0.05$, the paper's central superiority claim is falsified.","tokens_in":22553,"feed_emoji":"🌳","tokens_out":6297,"duration_ms":62258,"temperature":0.7,"pith_summary":"The paper claims that a bagging ensemble of oblique decision trees with SVM-learned splits can outrank both specialized oblique-tree ensembles and general-purpose classifiers on tabular data. The base algorithm, STree, handles multi-class problems without clustering labels into artificial binary groups: at each node it trains several one-vs-one or one-vs-rest SVMs and keeps the single SVM whose induced split most reduces class impurity. On 49 datasets, the default ensemble ODTE posts the highest average accuracy (0.8391) and best average rank (3.88) among ten compared methods; the tuned version ODTET (rank 1.35) is statistically significantly better than all three tuned competitors after correction for multiple comparisons. STree's trees are also substantially more compact than the competitors' trees. If the tuned comparison is fair, the method is a strong, easily tunable option for structured-data classification.","feed_headline":"Tuned SVM oblique trees rank first on 49 datasets","feed_subtitle":"A bagging ensemble of SVM-split trees wins the tuned comparison and grows smaller trees than rivals.","key_machinery":"The load-bearing object is STree's node-level model selection. At a node with $k'$ classes, the algorithm enumerates either $r = k'(k'-1)/2$ one-vs-one SVM problems or $k'$ one-vs-rest problems, evaluates each learned SVM by the weighted impurity of the binary partition it induces, $$b^* = \\arg\\min_j \\frac{|D'_+|}{|D'|} I(Y,D'_+) + \\frac{|D'_-|}{|D'|} I(Y,D'_-),$$ and stores only the selected model. This selection turns the multi-class problem into a sequence of binary SVM splits without external clustering; bagging over many such trees produces the ensemble.","core_discovery":"The central discovery is a mechanism for building oblique trees that natively handle multi-class targets. Instead of reducing each node's data to a binary problem by grouping classes, STree fits one SVM for every one-vs-one pair (or every class versus the rest) and chooses the model whose partition minimizes weighted Shannon entropy. The selected hyperplane becomes the node test; recursion continues on the two sides. Wrapped in bagging with 100 trees, this yields ODTE. In the paper's 49-dataset, ten-times five-fold cross-validation study, ODTE has the best default mean accuracy and rank, and after per-dataset grid-search tuning, ODTET's superiority over the three tuned twin-bounded-SVM oblique competitors is statistically significant; normalized average tree size is 1.00 for ODTE versus 2.66 to 11.73 for its oblique competitors.","pith_inferences":["Because the default ODTE is not statistically distinguishable from most default competitors, the strongest claim rests on the tuned comparison; an equal-tuning comparison against RandomForest and XGBoost would reveal whether the margin is intrinsic to the algorithm or driven by tuning procedure.","STree's training cost scales with the number of classes: the one-vs-one strategy trains $O(k'^2)$ SVMs per node, so for datasets with many labels the one-vs-rest variant may be a substantially cheaper alternative that the paper does not systematically explore.","The reported compactness of ODTE trees likely translates into faster inference and a smaller memory footprint, but the paper reports tree size rather than measured inference latency; a latency benchmark is a natural extension.","The node-level selection of a single best SVM can be viewed as a greedy search over a restricted hypothesis space; combining it with random feature subspaces, as in Random Forest, is a testable direction the paper lists for future work."],"forward_implications":["In the tuned scenario, ODTET's rank of 1.35 beats all three tuned oblique competitors, and the adjusted post-hoc comparisons reject equivalence at $\\alpha = 0.05$.","In the default scenario, ODTE has the best average accuracy and rank but is statistically significantly better only than XGBoost and TBRoF; against most other defaults the edge is not significant.","STree handles multi-class targets directly, so practitioners do not need to cluster classes into artificial binary groups before building oblique trees.","ODTE trees are the most compact among the compared oblique-tree methods, with normalized average size 1.00 versus 2.66 to 11.73 for competitors.","Both ODTE and STree are implemented as standard machine-learning library classifiers, so the method can be adopted and tuned with familiar tooling."],"supporting_citations":[{"why":"Provides the 49-dataset benchmark and the TBRaF, TBRoF, and TBRRoF twin-bounded-SVM oblique ensembles that are the main competitors.","marker":"Ganaie et al. (2020)"},{"why":"Supplies the embedded-SVM oblique tree method wrapped as BaggingJ48-SVMODT, a key default-setting competitor.","marker":"Menkovski et al. (2008)"},{"why":"Supplies WODT, wrapped as BaggingWODT and used as a gradient-optimized oblique tree baseline.","marker":"Yang et al. (2019)"},{"why":"Defines Random Forest, the standard bagging ensemble baseline.","marker":"Breiman (2001)"},{"why":"Defines XGBoost, the gradient-boosting baseline.","marker":"Chen and Guestrin (2016)"},{"why":"Establishes the non-parametric comparison and adjusted post-hoc procedure used for the statistical analysis.","marker":"Demšar (2006)"},{"why":"Presents the preliminary STree version that this paper extends into the ODTE ensemble.","marker":"Montañana et al. (2021)"}],"fun_headline_variants":["SVM oblique trees outrank rivals on 49 datasets","ODTE tops benchmarks with compact oblique trees","Multi-class SVM splits give winning tree ensemble","ODTE: smaller trees, top ranks in 49-dataset test","Oblique trees using SVM win accuracy and size"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The tuned comparison assumes that ODTET's grid search and the baselines' author-chosen hyperparameters represent equally thorough tuning; if ODTET received a more favorable search, the statistically significant tuned ranking would reflect tuning asymmetry rather than algorithm quality.","fun_headline_variants_meta":{"raw":{"variants":["SVM oblique trees outrank rivals on 49 datasets","ODTE tops benchmarks with compact oblique trees","Multi-class SVM splits give winning tree ensemble","ODTE: smaller trees, top ranks in 49-dataset test","Oblique trees using SVM win accuracy and size"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000261,"raw_usage":{"total_tokens":1574,"prompt_tokens":904,"completion_tokens":670,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":607}},"tokens_in":520,"tokens_out":670,"duration_ms":7962,"temperature":1.0,"reasoning_tokens":607,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:28:02.764166+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the ten-times five-fold cross-validation with the same ten seeds, giving each tuned baseline a grid search over the same hyperparameter ranges and number of configurations as ODTET. If ODTET no longer holds the best average rank or the adjusted post-hoc p-values climb above $\\alpha = 0.05$, the paper's central superiority claim is falsified.","supporting_citations":[],"review_version":1}