{"id":"89e9682a-d011-44fb-af4e-ef160bcc35f1","arxiv_id":"2411.08849","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A Bayesian additive regression tree model that grows trees with random oblique hyperplane splits, using a spike-and-slab prior on split directions, matches or beats axis-aligned BART on several benchmarks.","lead":"Oblique BART lets Bayesian regression trees split on random hyperplanes instead of single features. On benchmarks it predicts about as well as standard BART and much better on some rotated or curved decision boundaries.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Leaf-prior precision in Eq. (1)/(B1) is inconsistent with the stated N(0, τ²/M) prior; the grow/prune MH acceptance is off by √M, so the sampler may target a different posterior.","rationale":"After reading the paper in good faith, the central claim is that obliqueBART is a valid Bayesian additive tree model whose posterior predictive means are competitive with axis-aligned BART. For that claim to hold, the MCMC algorithm must sample from the posterior of the model whose prior is defined in Section 2.2. The paper defines the leaf-output prior as N(0, τ²/M), but Eq. (1) and the derived acceptance probabilities in Appendix B use τ⁻² as the precision and τ⁻¹ as the normalizing constant. These expressions correspond to a leaf prior with variance τ², not τ²/M. This is not a cosmetic typo: the precision enters P_ℓ and the normalizing constant enters the grow/prune MH ratio, changing the stationary distribution. With M=200, the missing √M factor makes grow proposals less likely to be accepted than the specified posterior requires, likely yielding shallower trees than intended. The empirical comparisons in Section 4 therefore may not describe the proposed model. A concrete check on the package code and a re-run of the synthetic experiment with the corrected acceptance probability would settle this. I do not agree with the reader's identification of MCMC burn-in as the weakest assumption: while convergence diagnostics are absent, the more fundamental issue is that the target distribution itself appears misspecified in the derivation.","tokens_in":18230,"tokens_out":16019,"duration_ms":151872,"concrete_test":"Re-derive Eq. (1) from the stated priors: with μ_ℓ ∼ N(0, τ²/M), the leaf precision is M/τ². Then inspect the grow/prune MH code in the released obliqueBART R package: does it use M/τ² in P_ℓ and the √M factor in the acceptance probability? If not, rerun the rotated-axes synthetic experiment (n=1000, Δ=4, θ=π/4) with the corrected sampler and compare out-of-sample RMSE to the reported values; if the discrepancy is large, the reported obliqueBART gains are not attributable to the model as defined.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 2.2 states the leaf-output prior is N(0, τ²/M), where M is the number of trees, so the marginal prior for f(x) is N(0, τ²). However, Eq. (1) writes the full conditional with exponent −(1/2)[σ⁻²Σ(r_i−μ_ℓ)² + τ⁻²μ_ℓ²] and normalizing constant τ⁻¹, which is the full conditional for a leaf prior with variance τ², not τ²/M. The factor M is missing from the precision. Consequently the integrated likelihood in Eq. (2) and the MH acceptance probabilities in Eqs. (B1) and (B2) are also incorrect: P_ℓ should be n_ℓσ⁻² + Mτ⁻², and the normalizing ratio for a grow move should be √M/τ instead of τ⁻¹ (and τ/√M instead of τ for a prune). With the recommended M=200, the grow acceptance probability is understated by a factor of √M ≈ 14.1, so the sampler does not target the posterior of the model described in Section 2.2 unless the implementation silently corrects this. Since the central claim concerns the empirical performance of the proposed Bayesian model, the reported comparisons in Section 4 may reflect a different model than the one defined.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes obliqueBART, an extension of Bayesian Additive Regression Trees in which internal nodes use hyperplane splits of the form {ϕ^T x < c}, with a spike-and-slab prior on ϕ and a Beta prior on the sparsity probability θ. The authors derive a Metropolis-within-Gibbs sampler, provide an R package, and compare predictive performance against axis-aligned BART, random-rotation versions of BART/RF/ERT/XGBoost, and other tree ensembles on synthetic and benchmark regression and classification tasks. The central claim is that obliqueBART is competitive with and sometimes substantially better than axis-aligned BART and other tree ensembles, especially when the regression function has boundaries not aligned with coordinate axes.","tokens_in":18428,"tokens_out":9913,"duration_ms":84037,"significance":"If the derivation is corrected, obliqueBART is a conceptually clean and potentially valuable extension of BART. It introduces a coherent generative model for oblique trees, with a sparse direction prior that adapts the expected number of nonzero coefficients, and the empirical study covers a broad set of benchmarks. The paper ships an R package (https://github.com/paulhnguyen/obliqueBART), which is a strength. However, the current manuscript contains a load-bearing error in the MCMC formulas that requires the authors to either correct the paper or confirm that the implementation already implements the corrected forms; and the empirical sections need stronger MCMC diagnostics and more careful statistical inference before the claims can be accepted.","major_comments":[{"comment":"The stated leaf-output prior is N(0, τ²/M), but the full conditional in Eq. (1) uses precision τ^{-2} and normalizing constant τ^{-1}, which is the full conditional for a N(0, τ²) leaf prior. Consequently, Eq. (2) should read P_ℓ = n_ℓ σ^{-2} + M τ^{-2} with an overall factor √M/τ, and the grow/prune acceptance probabilities in Eqs. (B1) and (B2) should contain √M/τ and τ/√M, respectively, instead of τ^{-1} and τ. As written, the sampler targets the posterior of a model in which the marginal prior for f(x) is N(0, M τ²) rather than N(0, τ²); with M=200 this is a 200-fold difference in prior variance, and the missing √M≈14.1 factor materially changes the growth acceptance probability. The authors must correct these formulas and either verify that the released implementation contains the corrected factors or re-run the experiments with the corrected sampler.","section":"Section 2.2 and Eqs. (1)–(3), (B1)–(B2)"},{"comment":"The posterior means are based on a single chain of 2000 iterations with the first 1000 discarded, but the paper reports no convergence diagnostics, no effective sample sizes, and no acceptance rates for the tree proposals. Given that oblique rules dramatically expand the move space and that grow proposals are drawn entirely from the prior, the chain may mix very slowly; 1000 retained samples could yield predictions that are not representative of the posterior. Please report trace plots, ESS, and MH acceptance rates at least for a few representative datasets, and consider increasing the number of iterations or using multiple chains.","section":"Section 4, first paragraph"},{"comment":"The significance claims rely on one-sided paired t-tests at the 5% level applied to 40 datasets (and to many rotation counts), without any multiple-testing correction; the 'number of rotations needed to match' statements in Tables A6–A7 are based on the same unadjusted tests and therefore inherit this problem. Also, average SMSEs and accuracies are reported without standard errors or confidence intervals, making it difficult to assess the strength of the headline comparisons. Please either adjust for multiplicity, or downgrade the language to point estimates, and report measures of uncertainty for the averages.","section":"Section 4.2 and Tables A2–A7"}],"minor_comments":[{"comment":"The text refers to 'the sinusoid function (Figure 2a)', but the sinusoid appears in Figure 2d; please fix the cross-reference.","section":"Section 1.2 and Figure 2"},{"comment":"'fits of other tres' should be 'fits of other trees'.","section":"Section 2.2, first paragraph"},{"comment":"The synthetic predictors are drawn from Uniform([0,1]^2), while the model is defined for predictors in [-1,1]^p; please state explicitly how the data are scaled before fitting.","section":"Section 4.1"},{"comment":"The average SMSE for obliqueBART is reported as 0.296 in the comparison with axis-aligned methods and 0.297 in the random-rotation comparison; clarify whether these averages are computed over the same datasets, given that some rotated methods could not be run on all datasets.","section":"Section 4.2"},{"comment":"The phrase 'data-adaptive decision rule prior' may overstate the case because the rule prior is not itself data-dependent; the sparsity probability θ is updated from the data. Consider rephrasing to avoid confusion.","section":"Section 3.1"},{"comment":"Typos: 'In principal' should be 'In principle'; 'implementating' should be 'implementing'; 'diamons' should be 'diamonds'; 'the the smallest' should be 'the smallest'; and the dataset name 'hepatitis' in Table A3 should be 'hepatitis'.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The missing factor of M in the leaf prior equations appears to be a straightforward typo that the authors can fix; however, it is essential that the authors confirm the released R package implements the corrected formulas rather than the equations as printed. I also note that the paper's contribution is a method plus empirical study, so the empirical reporting standards matter for acceptance. The paper fits the journal's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The oblique rule prior for BART is a legit new construction, and the LP-based cutpoint sampling is a neat way to handle the changing feasible region. But the math in Section 3.2 as written does not match the stated leaf-prior. They say leaf outputs are N(0, τ²/M), yet Eq. (1) uses precision τ⁻² (variance τ²), and the integrated likelihood and MH ratios inherit that. So the sampler described targets a posterior for a model with leaf prior variance τ², not τ²/M. That's not a typo-level issue: with M=200 the grow acceptance is off by √M≈14. Unless the code silently corrects it, the experiments may be for a different model than the one defined. This needs checking and fixing.\n\nWhat's good: the spike-and-slab on ϕ with Beta prior is a natural, well-motivated extension; proposing from the prior and letting MH sort it out is standard BART philosophy; the synthetic experiments are informative and the rotated-axes comparison to random-rotation BART is the right control. The benchmark comparison is broad (18 regression, 22 classification), and they honestly note obliqueBART is not uniformly better. The R package is a real plus.\n\nSoft spots, in order. First, the τ²/M inconsistency above. Second, the empirical section has no error bars or standard errors in the figures/tables, and the many paired t-tests are done at 5% without any multiple-testing correction; some of the \"wins\" will be noise. Third, 1000 post-burn-in draws from a single chain with no convergence diagnostics is thin, especially for a sampler that can have slow mixing in tree space. Fourth, classification results are weaker—they get beaten by ERT on average—so the \"competitive\" claim is fine but the abstract's \"sometimes much better\" is mostly a regression story.\n\nWho's this for: people working on BART extensions or oblique tree ensembles. It deserves a serious referee, but the referee should send it back for a corrected derivation and more careful empirical reporting. I'd want to see the factor M fixed and a small simulation confirming the sampler targets the stated prior.","headline":"A genuinely new oblique BART construction with a real technical bug: the stated leaf prior (N(0, τ²/M)) does not match the full conditional and MH ratios in Section 3.2, which use precision τ⁻².","tokens_in":114,"tokens_out":1691,"would_cite":false,"duration_ms":32383,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F15","62G08","68T05"],"pacs":[],"model":"deepseek-v4-flash","headline":"Hyperplane-splitting BART matches or beats axis-aligned tree ensembles","keywords":["oblique Bayesian additive regression trees","random hyperplanes","spike-and-slab prior","tree ensembles","nonparametric regression","Metropolis-within-Gibbs","tabular regression"],"falsifier":"Re-run the 40 benchmark comparisons with 10 independent chains of 10,000 post-burn-in draws each and compute R-hat and effective sample sizes; if the posterior predictive means change enough to flip which regression datasets show statistically significant SMSE differences, then the single-chain posterior approximation is not valid and the reported performance comparisons are not settled.","tokens_in":17968,"feed_emoji":"🌳","tokens_out":10244,"duration_ms":79965,"temperature":0.7,"pith_summary":"The paper introduces obliqueBART, a Bayesian additive regression tree model in which decision rules are random hyperplanes rather than splits on a single feature at a time. Its central claim is that this added geometric flexibility lets the model match or beat axis-aligned BART and other tree ensembles on tabular regression, with the biggest gains when true decision boundaries are tilted away from the coordinate axes. The claim is supported by synthetic examples and 40 benchmark datasets, where obliqueBART attains the smallest average standardized mean squared error among the non-rotated methods. A practitioner would care because this gives a fully Bayesian, uncertainty-quantifying ensemble that can represent non-axis-aligned structure without needing to pre-rotate the input space.","feed_headline":"Hyperplane-split BART beats axis-aligned tree ensembles","feed_subtitle":"A spike-and-slab prior lets Bayesian regression trees cut along any direction, winning most regression benchmarks.","key_machinery":"The load-bearing object is the oblique decision rule $\\phi^{\\top} x_{\\mathrm{cont}} < c$, with a hierarchical spike-and-slab prior on $\\phi$: a point-mass spike at zero for unused coordinates and a standard normal slab on active coordinates, with the sparsity parameter $\\theta$ given a Beta prior so the number of active features adapts to the data. At a grow move the rule is proposed from this prior, the cutpoint $c$ is drawn uniformly from the interval of valid values of $\\phi^{\\top} x_{\\mathrm{cont}}$ at that node (found by solving two linear programs over the polytope defined by ancestor rules), and the Metropolis-Hastings acceptance probability then simplifies because prior and proposal cancel, leaving a fit-versus-complexity comparison. Categorical predictors are handled by random subset rules. This prior-proposal design is what lets the ensemble explore oblique partitions without hand-tuned rotations.","core_discovery":"The paper's central claim is that replacing axis-aligned splits in BART with oblique rules drawn from a hierarchical spike-and-slab prior yields a valid Bayesian tree ensemble that is competitive with -- and sometimes much better than -- axis-aligned BART and other tree ensembles on regression benchmarks, while remaining competitive on classification. The gains are largest when the true function's discontinuities are not aligned with the coordinate axes, and the model adapts its sparsity automatically: on the rotated-axes synthetic problem it uses axis-aligned rules about 70 percent of the time when the truth is axis-aligned and about 53 percent when the truth is oblique. The paper does not argue for wholesale replacement of axis-aligned BART, but for a practical alternative that is often more accurate and rarely clearly worse.","pith_inferences":["If the posterior approximation is trustworthy, the same spike-and-slab rule prior could be dropped into BART extensions for survival, heteroskedastic regression, or treatment-effect estimation, since only the grow-proposal distribution changes.","A direct test of robustness would be to re-run the benchmark comparison with multiple independent chains and convergence diagnostics; if the posterior means move, the reported nine-dataset advantage over BART may shrink, but the tilted-boundary gains from the synthetic experiments should persist.","For image inputs, constraining the non-zero entries of $\\phi$ to form a connected sub-region of the image would give a Bayesian analog of manifold oblique random forests, an extension the paper itself notes."],"forward_implications":["On 18 regression benchmarks, obliqueBART posts the lowest average standardized mean squared error (0.296) among obliqueBART, BART (0.316), RF (0.330), ERT (0.332), and XGB (0.342).","On the rotated-axes synthetic problem it beats axis-aligned BART at every rotation angle, including $\\theta = 0$, and its rule ensemble shifts from 70.2% axis-aligned splits at $\\theta=0$ to 52.9% at $\\theta=\\pi/4$.","Rotating the input features before fitting axis-aligned methods rarely closes the gap: rotBART cannot match obliqueBART on 19 of the regression datasets even with 200 random rotations, and obliqueBART is roughly 20x faster than rotRF and 2x faster than rotBART at that rotation count.","On 22 classification benchmarks obliqueBART is competitive, with average accuracy 0.846 vs ERT's best 0.866 and differences under 2% on 15 of the 22 datasets."],"supporting_citations":[{"why":"Defines the BART model, its branching-process tree prior, default hyperparameters, and the Metropolis-within-Gibbs sampler that obliqueBART extends; also the primary axis-aligned baseline.","marker":"Chipman et al. (2010)"},{"why":"Introduces random forests and reports that an oblique variant reduces out-of-sample error by 3-8.5%, the key motivating evidence for oblique splits.","marker":"Breiman (2001)"},{"why":"Shows that allowing a variable number of non-zero entries in oblique projection vectors improves prediction, directly motivating the spike-and-slab prior.","marker":"Tomita et al. (2020)"},{"why":"Supplies the categorical decision rules and the argument that proposing rules from the prior avoids deflated MH acceptance probabilities, guiding the grow-proposal design.","marker":"Deshpande (2024)"},{"why":"Provides random rotation ensembles, the comparison class used to test whether pre-rotating features matches obliqueBART's performance.","marker":"Blaser and Fryzlewicz (2016)"},{"why":"Provides the probit data augmentation used to adapt obliqueBART to binary classification.","marker":"Albert and Chib (1993)"},{"why":"Defines extremely randomized trees, one of the four axis-aligned baselines compared in the benchmark experiments.","marker":"Geurts et al. (2006)"},{"why":"Shows that oblique tree ensembles with connected-region projections reduce the gap to convolutional networks on images, the extension obliqueBART targets for future work.","marker":"Li et al. (2023a)"}],"fun_headline_variants":["Oblique BART cuts along any hyperplane, beats axis-aligned rivals","Random hyperplane splits give BART a predictive edge","Slanted decision rules help Bayesian trees win regression tests","When truth isn't axis-aligned, oblique BART excels","BART with random oblique splits adapts sparsity automatically"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The benchmark conclusions assume that 1000 retained draws from a single MCMC chain, without convergence diagnostics, accurately represent obliqueBART's posterior predictive distribution.","fun_headline_variants_meta":{"raw":{"variants":["Oblique BART cuts along any hyperplane, beats axis-aligned rivals","Random hyperplane splits give BART a predictive edge","Slanted decision rules help Bayesian trees win regression tests","When truth isn't axis-aligned, oblique BART excels","BART with random oblique splits adapts sparsity automatically"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000267,"raw_usage":{"total_tokens":1545,"prompt_tokens":807,"completion_tokens":738,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":423,"completion_tokens_details":{"reasoning_tokens":653}},"tokens_in":423,"tokens_out":738,"duration_ms":7476,"temperature":1.0,"reasoning_tokens":653,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T21:15:29.460479+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the 40 benchmark comparisons with 10 independent chains of 10,000 post-burn-in draws each and compute R-hat and effective sample sizes; if the posterior predictive means change enough to flip which regression datasets show statistically significant SMSE differences, then the single-chain posterior approximation is not valid and the reported performance comparisons are not settled.","supporting_citations":[{"cited_title":"A., George, E","cited_arxiv_id":null,"evidence_quote":"Defines the BART model, its branching-process tree prior, default hyperparameters, and the Metropolis-within-Gibbs sampler that obliqueBART extends; also the primary axis-aligned baseline."},{"cited_title":"M., Browne, J., Shen, C., Chung, J., Patsolic, J","cited_arxiv_id":null,"evidence_quote":"Shows that allowing a variable number of non-zero entries in oblique projection vectors improves prediction, directly motivating the spike-and-slab prior."},{"cited_title":"flexBART: Flexible Bayesian regression trees with categorical predictors","cited_arxiv_id":"2211.04459","evidence_quote":"Supplies the categorical decision rules and the argument that proposing rules from the prior avoids deflated MH acceptance probabilities, guiding the grow-proposal design."},{"cited_title":"and Fryzlewicz, P","cited_arxiv_id":null,"evidence_quote":"Provides random rotation ensembles, the comparison class used to test whether pre-rotating features matches obliqueBART's performance."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines extremely randomized trees, one of the four axis-aligned baselines compared in the benchmark experiments."}],"review_version":1}