{"id":"12e260dd-eff2-4fd7-b4a0-aa0ccf38482d","arxiv_id":"2411.17003","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A gradient-optimized single oblique regression tree with linear leaf models reports higher average test R2 than random forest on 16 datasets, though the gain is not statistically significant.","lead":"This paper trains a single oblique regression tree with gradient descent, replacing hard splits with a gradually sharpened sigmoid approximation and polishing subtrees. On 16 tabular datasets, the resulting tree (GET-Linear) reports an average test accuracy 2.03 percentage points higher than a tuned random forest, but the statistical evidence for that advantage is weak.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'statistically significant' claim is contradicted by the paper's own test: p=0.127 exceeds the pre-specified α=0.1, and the L1 strength was selected using test-set accuracy, inflating the reported 2.03% advantage.","rationale":"The reader's REJECT verdict is well-founded. I agree that the post hoc L1 regularization choice and the untuned RF baseline are serious weaknesses, but the single most load-bearing concern is more basic: the paper's own significance test fails under its stated threshold. The phrase 'if we accept a tolerance τ>0.127' is not a valid way to establish significance; it is moving the goalposts after seeing the p-value. Since the central claim in the contributions list explicitly says 'demonstrating a statistically significant difference,' and the only reported p-value for that comparison is 0.127, the claim is internally contradicted. The L1 selection compounds this by making the reported 2.03% number a test-set-selected optimum rather than an honest evaluation of a fixed procedure. Even the unregularized 1.30% advantage would require a proper significance test, and the RF comparison would need to include tuning of max_features and min_samples_leaf to ensure the baseline is not artificially weak. Given these issues, the headline claim cannot be accepted as stated; the paper's useful algorithmic contributions do not rescue the central empirical assertion. The reader already reached REJECT, so my recommendation is UNCHANGED.","tokens_in":31263,"tokens_out":4559,"duration_ms":43125,"concrete_test":"Recompute the GET-Linear vs RF comparison under a pre-specified protocol: (1) choose λ on the validation split from {0, 1e-6, 5e-6, 1e-5}; (2) tune RF by cross-validation over n_estimators, depth, max_features ∈ {1/3, 0.5, 0.7, 1.0}, and min_samples_leaf ∈ {1, 5, 10}; (3) fix α=0.1 and run the paired t-test on the 16 test-set differences. If p≥0.1 or the average advantage falls materially below the reported 2.03%, the central claim as stated is unsupported. As an immediate check, the reported p=0.127 already fails the paper's own α=0.1 criterion.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline claim asserts that GET-Linear outperforms random forests by 2.03% with a statistically significant difference. The paper's own paired t-test in Figure 4 yields p=0.127 against RF, while Section 5.1 pre-specifies τ=0.1 as the significance tolerance. The text then says 'if we accept a tolerance τ>0.127, we can reject the null hypothesis'—an ex post facto threshold. Under the stated α=0.1, the null hypothesis is not rejected, so the paper's statistical conclusion contradicts its own evidence. A compounding issue is the selection of the L1 regularization strength: Appendix F.4, Table 10 reports that GET-Linear's test accuracy improves from 83.24% to 83.97% when λ is adjusted only over {0, 1e-5}, and the regularized value is used in Table 1. This is test-set-based hyperparameter selection: the 2.03% gap is a selected maximum, not the result of an unbiased evaluation. Without regularization the advantage is 1.30%, and no significance test is reported for that configuration. Additionally, the RF baseline is only tuned over n_estimators and depth, leaving max_features and min_samples_leaf at sklearn defaults; those defaults may be suboptimal on some of the 16 datasets, so part of the gap could reflect an under-tuned comparator. The load-bearing evidence for the central claim is therefore invalid on two independent grounds.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a gradient-based entire-tree optimization framework for oblique regression trees, using a scaled sigmoid approximation of hard splits, an iterative annealing over the scale factor, and a subtree-polish strategy. The method is instantiated with constant leaf predictions (GET) and linear leaf predictions (GET-Linear). The central empirical claim is that, averaged over 16 UCI/OpenML regression datasets, GET-Linear outperforms a classic random forest by 2.03% in test R2, and that this difference is statistically significant, while GET is statistically comparable to random forest (Section 5.1, Table 1). The paper also reports that GET outperforms several decision-tree baselines, including CART, OC1, HHCART, SoftDT, GradTree, and ORT-LS (Table 2), and provides an analysis of training optimality (Table 3), an ablation of the proposed strategies (Table 4), and a comparison of parameter counts and prediction time (Table 5).","tokens_in":31605,"tokens_out":2597,"duration_ms":26744,"significance":"If the performance claim were fully supported, the result would be a meaningful challenge to the common belief that a single tree cannot reach random-forest-level test accuracy in regression, and it would strengthen the case for interpretable, lightweight alternatives in resource-constrained settings. The paper has clear strengths: the optimization reformulation is concisely presented, the algorithmic contribution (iterative scaled sigmoid plus subtree polishing) is explicit, the code is released, the experiments cover 16 datasets, and the ablation studies isolate the effect of the main components. However, the headline statistical-significance claim is not supported by the paper's own test, and the main accuracy gap relies on test-set-based selection of the L1 regularization strength. These issues are fixable but currently undermine the central conclusion as stated.","major_comments":[{"comment":"The claim that GET-Linear with p=0.127 is 'statistically superior' to random forest contradicts the paper's own pre-specified significance level. Section 5.1 sets the tolerance at τ=0.1 and then, when the paired t-test yields p=0.127, the text says 'if we accept a tolerance τ>0.127, we can reject the null hypothesis.' This is an ex post facto threshold change. Under the stated α=0.1, the null hypothesis is not rejected, so the bullet in Section 1 stating 'demonstrating a statistically significant difference' is not supported. The appropriate conclusion from p=0.127 is that the evidence for superiority is suggestive but not significant at the stated level; for a claim of comparability, a confidence interval or an equivalence test would be needed.","section":"Section 5.1, Figure 4"},{"comment":"The 2.03% average gap in Table 1 is inflated by test-set-based hyperparameter selection. Appendix F.4 reports that GET-Linear's test accuracy improves from 83.24% to 83.97% when the L1 regularization strength λ is chosen from {0, 1e-5}, and the regularized result is used in Table 1. This is selection on the test set: without regularization the gap over RF is 1.30 percentage points, and no significance test is reported for that configuration. To make the central claim valid, the regularization strength should be selected on a validation fold, with the test set used only for final evaluation, and the result should be reported together with the corresponding significance test.","section":"Appendix F.4, Table 10; Section 5.5"},{"comment":"The random forest baseline is only tuned over n_estimators and max_depth, while max_features and min_samples_leaf are left at sklearn defaults. The justification cites general robustness of defaults, but no dataset-specific evidence is given that these defaults are appropriate for the 16 datasets involved. Since the headline claim compares against this baseline, part of the 2.03% gap could reflect an under-tuned comparator. The comparison should either tune these hyperparameters (e.g., with the same cross-validation protocol used for depth) or provide sensitivity results showing that the conclusions are unchanged under reasonable variations of these defaults.","section":"Section 5.1, first paragraph"}],"minor_comments":[{"comment":"The text contains a typo: 'all branch nondes' should read 'all branch nodes.' There are also minor grammatical issues in Section 4.3 ('once a branch node is optimally identified, we can sequentially polish its subtree') and in Section 5.1 ('GET-Linear significantly outperforms RF by 2.03%' overstates the statistical result).","section":"Section 4.2, Algorithm 1"},{"comment":"The text labels in Figure 4 are rendered as unreadable Unicode escape sequences (e.g., 'p/uni...'), which makes the figure impossible to parse. The axis labels and p-values should be plain text.","section":"Figure 4"},{"comment":"The phrase 'less than 20,000 samples' is repeated from the abstract but the numerical claim in Section 5.3 about training time uses averages; please clarify whether the training-time figures in Table 3 are averaged over the 16 datasets or over a subset, and specify the standard deviation or range.","section":"Section 5.5, first paragraph"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal, and the proposed method appears interesting and generally well engineered. The central claim, however, is currently oversold in the abstract and contributions: the paper's own statistical test rejects the significance claim, and the main accuracy gap is selected on the test set. Both issues are fixable through a revised experimental protocol. I do not see circularity or fabrication; the concerns are about evaluation rigour rather than about the method being vacuous. If the authors can re-run with validation-based hyperparameter selection and report the appropriate statistical test, the paper could become publishable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the core method is worth a look, but the headline claim that a single tree outperforms random forest is not backed by the paper's own statistics. The paired t-test gives p=0.127 against a pre-specified α=0.1, so by their own criterion the difference is not significant. The move to \"if we accept a tolerance τ>0.127\" is ex post facto threshold-setting. On top of that, the L1 regularization strength for GET-Linear was picked after seeing test accuracy improve from 83.24% to 83.97% (Appendix F.4, Table 10). That is test-set-based model selection, so the reported 2.03% advantage is a selected maximum, not an unbiased estimate. Without that regularization the gap is 1.30%, and no significance test is reported for that configuration. The RF baseline is also only tuned over n_estimators and max_depth, leaving max_features and min_samples_leaf at defaults; that may be under-tuning the comparator.\n\nWhat the paper does well: the iterative scaled sigmoid approximation with warm starts is a sensible way to handle the approximation-vs-stability tradeoff, and the subtree polish strategy is a reasonable post-hoc refinement. The ablation study (Table 4) shows both strategies help. Hard-split inference with deterministic leaf values is a clean choice. The framework is general and the code is public. The comparisons against other decision tree methods (CART, OC1, HHCART, ORT-LS, etc.) are extensive, and the improvements there are more credible, though those baselines may also be under-tuned.\n\nThe weakness is not in the method; it is in the statistical claim and the evaluation protocol. The paper undercuts its own credibility by overclaiming significance. That said, the idea is not a restatement of earlier work; it builds on known sigmoid approximations but adds the iterative scaling and subtree polish, and reports reproducible experiments.\n\nWho this is for: people working on oblique trees, interpretable ML, or embedded regression. They will get value from the optimization tricks even if they do not believe the RF comparison.\n\nRecommendation: this deserves a serious referee, not a desk reject, but it needs major revision before acceptance. The authors should either drop the \"statistically significant\" language, or pre-register a proper test with validation-based hyperparameter selection and multiple seeds. I would not cite it in its current form.","headline":"The optimization method is solid and worth knowing, but the headline RF-beating claim fails on the paper's own statistics and on test-set-based regularization selection.","tokens_in":32090,"tokens_out":1914,"would_cite":false,"duration_ms":18488,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single decision tree, trained end-to-end with gradient descent and linear leaves, can match—and on average beat—a random forest that pools hundreds of trees.","keywords":["oblique regression tree","gradient-based tree optimization","scaled sigmoid approximation","random forest comparison","hard-split inference","subtree polish","tree interpretability","decision tree vs ensemble"],"falsifier":"Re-run the paper's 16-dataset protocol with a random forest whose max_features and min_samples_leaf are tuned per dataset by cross-validation, and with GET-Linear's L1 strength selected on a validation split instead of by test-set improvement; if the forest then matches or beats GET-Linear in average test $R^2$ and in a paired t-test at the paper's own tolerance, the central claim fails.","tokens_in":31023,"feed_emoji":"🌳","tokens_out":10890,"duration_ms":88207,"temperature":0.7,"pith_summary":"This paper tries to overturn the usual assumption that a single decision tree is inherently less accurate than a random forest. It trains an oblique regression tree as one differentiable optimization problem, replacing non-differentiable split indicators with scaled sigmoid approximations and solving a sequence of increasingly sharp approximations. The result is GET, a hard-split tree with constant leaf predictions, and GET-Linear, whose leaves are linear regressions. Across 16 regression datasets, GET is statistically comparable to the classic random forest (0.17% lower test $R^2$), while GET-Linear outperforms the forest by an average of 2.03% (83.97% versus 81.94% test $R^2$). If the result holds, the accuracy cost of using one interpretable IF-THEN tree instead of a pool of hundreds of trees is much smaller than the field has assumed.","feed_headline":"One trained tree beats a 300-tree forest, on average","feed_subtitle":"End-to-end gradient training plus linear leaves lifts test R2 2.03% above the forest.","key_machinery":"The load-bearing object is the deterministic sample-route formulation coupled with the scaled sigmoid approximation. Each leaf assignment is the product $P_{i,t} = \\prod_{j \\in A^l_t} I_{i,j} \\prod_{j \\in A^r_t} (1 - I_{i,j})$ of branching-test indicators $I_{i,j} = \\mathbf{1}(b_j - a_j^\\top x_i > 0)$, and each indicator is replaced by the scaled sigmoid $\\hat{I}_{i,j} = (1 + e^{-\\alpha(b_j - a_j^\\top x_i)})^{-1}$ so the whole tree is differentiable. Three mechanisms carry the argument: an iterative schedule that solves the loss for small $\\alpha$ and warm-starts each larger-$\\alpha$ problem, keeping the approximation sharp without destabilizing gradients; a subtree polish strategy that re-optimizes each branch node's subtree while holding the rest of the tree fixed, canceling approximation error that accumulates along deep paths; and hard-split inference with deterministically recomputed leaf values, which keeps the final model a genuine IF-THEN tree rather than a soft probabilistic one. Multi-start random initialization supplies diverse scale-factor schedules and better optima.","core_discovery":"The central claim, stated on the paper's own terms, is that an oblique regression tree trained end-to-end by gradient-based optimization can reach—and, with linear leaf predictions, exceed—the test accuracy of a classic random forest. The authors reformulate tree training as an unconstrained differentiable loss over the entire tree, routing each sample deterministically through products of branching-test indicators, and they approximate the indicators with scaled sigmoid functions so that gradients reach both split and leaf parameters. Because a single large scale factor makes the sigmoid nearly an indicator but starves the gradients, they solve a sequence of optimization problems with progressively larger scale factors, warm-starting each from the previous solution, and a subtree polish step re-optimizes each branch node's subtree while the rest of the tree stays fixed. At inference the tree keeps hard splits and recomputes leaf values deterministically as the mean (GET) or a fitted linear regression (GET-Linear) of the training samples that reach each leaf. Across 16 real-world regression datasets the authors report GET-Linear at an average test $R^2$ of 83.97% against 81.94% for a tuned random forest (a 2.03% edge), with GET at 81.77% and a paired t-test p-value of 0.877 for GET versus the forest.","pith_inferences":["Editorial extension: at the conventional 0.05 significance level the reported p = 0.127 for GET-Linear versus the forest would not reject equality; the paper's 'statistically significant' wording depends on its chosen tolerance of τ = 0.1.","Editorial extension: because the forest's max_features and min_samples_leaf are left at defaults, the 2.03% headline is best read as 'single tree versus a lightly tuned forest' until a fully tuned baseline is tested.","Editorial extension: the framework is demonstrated only for regression with an $R^2$ objective; a classification analogue that replaces the squared loss with cross-entropy over leaf distributions is a natural test the paper does not run.","Editorial extension: the authors name forests as future work; whether the iterative scaled-sigmoid and subtree-polish gains survive in an ensemble, where averaging already reduces variance, remains to be shown."],"forward_implications":["A single hard-split tree with roughly 100–300 times fewer parameters can match the test $R^2$ of a forest averaging 309 trees, so the interpretability-accuracy trade-off is looser than the prevailing mindset assumes.","GET-Linear's edge comes from its linear leaves: constant-leaf GET only reaches parity, so the practical win is specifically a piecewise-linear oblique tree.","Prediction is 24–30 times faster than the forest and parameter counts drop by factors of 119–324, which matters for the embedded and real-time control settings the paper motivates.","The parity is bought with training time: the optimized tree is thousands of times slower to train than the forest, though roughly 20 times faster than the prior best oblique-tree heuristic at depth 12."],"supporting_citations":[{"why":"Defines the classic random forest that serves as the central baseline the single tree must match or beat.","marker":"(Breiman, 2001)"},{"why":"Provides the optimal-tree notation and mixed-integer programming formulation that the paper reformulates into unconstrained optimization.","marker":"(Bertsimas & Dunn, 2017)"},{"why":"ORT-LS, the state-of-the-art heuristic oblique regression tree that GET outperforms by 3.76% in test accuracy.","marker":"(Dunn, 2018)"},{"why":"GradTree, the straight-through-estimator gradient tree whose suboptimal performance motivates the sigmoid-approximation route.","marker":"(Marton et al., 2023)"},{"why":"SoftDT, the standard-sigmoid soft tree whose approximation gap and probabilistic predictions motivate scaled sigmoids and hard-split inference.","marker":"(Frosst & Hinton, 2017)"},{"why":"The linear leaf prediction model that GET-Linear's leaves are built from.","marker":"(Quinlan, 1998)"},{"why":"Justifies tuning only the number of trees and depth for the random forest baseline, leaving other hyperparameters at defaults.","marker":"(Probst & Boulesteix, 2018)"},{"why":"OC1, a classic greedy oblique-tree induction method among the baselines GET is compared against.","marker":"(Murthy et al., 1994)"}],"fun_headline_variants":["Single tree beats forest by 2% on average","Gradient-trained tree outperforms random forest","One tree, end-to-end trained, tops forest","Oblique tree with linear leaves exceeds forest","Tree beats 300-tree ensemble in accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The 2.03-point edge reported for GET-Linear rests on the random forest being tuned only for tree count and depth while its other hyperparameters stay at library defaults, and on the L1 penalty strength being picked after observing a 0.73% test improvement rather than chosen on validation data.","fun_headline_variants_meta":{"raw":{"variants":["Single tree beats forest by 2% on average","Gradient-trained tree outperforms random forest","One tree, end-to-end trained, tops forest","Oblique tree with linear leaves exceeds forest","Tree beats 300-tree ensemble in accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000155,"raw_usage":{"total_tokens":1208,"prompt_tokens":936,"completion_tokens":272,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":201}},"tokens_in":552,"tokens_out":272,"duration_ms":3262,"temperature":1.0,"reasoning_tokens":201,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:37:40.919040+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the paper's 16-dataset protocol with a random forest whose max_features and min_samples_leaf are tuned per dataset by cross-validation, and with GET-Linear's L1 strength selected on a validation split instead of by test-set improvement; if the forest then matches or beats GET-Linear in average test $R^2$ and in a paired t-test at the paper's own tolerance, the central claim fails.","supporting_citations":[],"review_version":1}