{"id":"8844dc82-ca23-4991-a377-9f8fcee5df49","arxiv_id":"2412.16534","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"DOFEN, a deep neural network that randomly assembles per-column soft conditions into relaxed oblivious decision trees and ensembles them, reaches state-of-the-art DNN performance on the Tabular Benchmark.","lead":"This paper introduces DOFEN, a neural network architecture for tabular data that builds relaxed decision trees by randomly combining learned column conditions. On the 73-dataset Tabular Benchmark it reports the best results among deep learning models and close competition with gradient boosted trees.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The DNN-SOTA claim rests on a 0.0008 average margin over Trompt with no repeated-seed intervals and no stated compute budget; this is currently unsupported.","rationale":"I read the paper in good faith and take the central claim to be an empirical one: DOFEN is the best DNN on the Tabular Benchmark and competitive with GBDTs. The architecture is clearly presented, the code is public, and the paper includes ablations, stability checks, and additional benchmark results that support the mechanism of the method. The concern I identify is not about the architecture's validity or the existence of the experiments; it is about whether the reported comparison is precise and fair enough to support the quantitative SOTA statement. The reader's weakest assumption already points to evaluation fairness and the absence of repeated-seed intervals. I agree with that assessment. The specific margin between DOFEN and Trompt is very small in the medium classification tables, and the paper gives no confidence interval for the aggregate numbers. Combined with the unstated compute budget and the omission of several baselines in the large-task tables, the headline claim is more fragile than the paper's confident language suggests. However, the available evidence is consistent with DOFEN being a strong and genuinely new tabular DNN, and the concerns do not amount to a demonstrated failure. A conditional verdict remains appropriate, and the concrete test I propose would settle whether the margin is real or an artifact of seed and budget choices.","tokens_in":51824,"tokens_out":6304,"duration_ms":58659,"concrete_test":"Run the official tabular-benchmark harness on the 16 medium numerical classification and 19 medium numerical regression datasets with a fixed wall-clock budget per model (e.g., the same GPU-hour limit for DOFEN and Trompt, applied to both the random-search loop and final training), using at least 10 random seeds for the searched configurations. Report mean ± 2 SE of test accuracy and R2, plus the paired per-dataset win rate between DOFEN and Trompt. If the 95% CI of the DOFEN-minus-Trompt difference includes 0 on either benchmark family, the 'state-of-the-art among DNNs' claim is not established; if the CI excludes 0 and the win rate is consistently above chance, the concern is resolved. Also recompute the large-task rankings with MLP, HGBT, and RandomForest included where they are currently missing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that DOFEN is state-of-the-art among DNNs on the Tabular Benchmark, and the weakest point is that the headline margin is not protected against seed noise or search-budget asymmetry. In the searched medium numerical classification results (Tables 31-32), DOFEN achieves 0.7805 versus Trompt at 0.7797, a 0.0008 advantage, yet no repeated-seed intervals are reported for any main table. Section 4.5 reports seed variance only on four datasets where DOFEN ranks first, under default hyperparameters, not on the searched configurations used in the headline tables. Appendix I.2 gives DOFEN a search space including d up to 8, m up to 64, and Nhead up to 8, while Appendix D shows large gains from increasing m and d on large datasets; Appendix C.3 shows DOFEN's per-configuration training time is roughly twice Trompt's. The paper never states the compute budget per model or per random-search iteration, so it is unclear whether the comparison holds under equal resources. The large-task tables (38-41) also omit MLP, HGBT, and RandomForest in several conditions (Appendix B.2), meaning the aggregate DNN-SOTA claim is effectively decided by medium tasks, where the margin is within plausible run-to-run variation. Additionally, ref [24] (Trompt) shares four authors with this work, which raises the bar for independent reimplementation, although it is not by itself evidence of bias. The load-bearing assumption is that a single-run, budget-uncontrolled comparison reliably estimates model quality; that assumption is currently unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces DOFEN, a deep neural network architecture for tabular data that constructs relaxed oblivious decision trees (rODTs) by randomly combining per-column soft conditions, then ensembles rODTs into multiple forests with a bagging step. The method is evaluated on the Tabular Benchmark (73 datasets) with comparisons against tree-based and deep learning baselines, and the authors claim DOFEN achieves state-of-the-art results among DNNs and is competitive with gradient-boosted trees. The paper also includes an ablation study, an interpretability analysis, a stability analysis, and additional experiments on FT-Transformer and GRANDE benchmark datasets. Code is publicly released.","tokens_in":52123,"tokens_out":4098,"duration_ms":35992,"significance":"If the performance claim is robust, DOFEN is a meaningful contribution to deep learning for tabular data: it is a novel architecture that combines tree-inspired inductive biases with end-to-end neural training, and it reportedly outperforms prior DNNs on a widely used benchmark while remaining interpretable. The paper is thorough in its appendices, providing per-dataset results, computational efficiency analyses, and evaluations on additional benchmarks, and the released code supports reproducibility. However, the headline 'state-of-the-art among DNNs' claim is currently supported by a very small average margin over Trompt (0.0008 accuracy on medium classification), without repeated-seed intervals for the main tables, and the comparison may be affected by search-budget and baseline-coverage asymmetries. These issues bear directly on the central empirical claim.","major_comments":[{"comment":"The claim that DOFEN is state-of-the-art among DNNs rests on an average accuracy margin of 0.0008 over Trompt (0.7805 vs 0.7797 in the optimal-hyperparameter medium classification results of Table 12). The main benchmark tables report a single value per model per dataset, with no confidence intervals or repeated-seed variation. Section 4.5 provides seed statistics only under default hyperparameters on four datasets where DOFEN ranks first; this does not characterize the searched configurations used in the headline comparisons. The authors should report mean and standard deviation (or confidence intervals) over multiple seeds for the main aggregate results, or at least for the comparison against Trompt, to establish that the 0.0008 margin is not within run-to-run noise.","section":"Section 4.2, Tables 31-33, Table 12"},{"comment":"The comparison may be unfair in terms of compute budget. Appendix I.2 gives DOFEN a search space that includes d up to 8, m up to 64, and Nhead up to 8, while Appendix C.3 reports that DOFEN's training time is approximately twice that of Trompt under optimal hyperparameters (1143.8 seconds vs 535.2 seconds on medium datasets). The paper states that it strictly follows the Tabular Benchmark protocols but does not specify the compute budget per model or per random-search iteration, nor the number of iterations used for each baseline. If all models were given the same number of random-search iterations, DOFEN consumed substantially more compute; if they were given equal wall-clock time, the number of iterations differs. The authors should state the exact protocol and, ideally, add an equal-compute or equal-iteration sensitivity analysis.","section":"Appendix I.2, Appendix C.3"},{"comment":"The large-task evaluation omits several baselines: MLP, HGBT, and RandomForest are missing from large classification and/or large regression results because they are absent from the official repository. Since the aggregate 'state-of-the-art among DNNs' claim covers all 73 datasets, the missing MLP baseline in particular means the claim is effectively decided by medium-sized tasks, where the margin over Trompt is very small. The authors should either run these baselines themselves (the code and hardware are available to them) or explicitly qualify the claim as covering only the tasks for which baselines are reported.","section":"Appendix B.2, Tables 38-41"},{"comment":"The regression ablation relies on a post-hoc outlier removal. The cell for 'no forest ensemble' on all regression datasets is 0.3238, marked with an asterisk, and the footnote states that after removing the outlier dataset the average becomes 0.5457 (and 0.6107 for numerical-only regression). This is not a pre-specified outlier criterion. Moreover, the 'remove both' row reports 0.5441, which is higher than the 'no forest ensemble' value of 0.3238; this counterintuitive pattern is unexplained and weakens the claim that the forest ensemble is the key contributor. The authors should either pre-specify an outlier handling rule, report per-dataset ablation results, or both, so readers can assess the robustness of the ablation conclusions.","section":"Table 1 (ablation)"}],"minor_comments":[{"comment":"There is a typo: 'we will futher discuss' should be 'we will further discuss'.","section":"Section 2"},{"comment":"The word 'Baggging' in the caption should be 'Bagging'.","section":"Figure 3 caption"},{"comment":"The sentence 'The result are averaged over various datasets' should be 'The results are averaged over various datasets'.","section":"Figure 1 and Figure 4 captions"},{"comment":"In the regression medium row, the 'Triple' column reports 0.7731, which is higher than the default 0.6796, contradicting the text that 'an increase in num_layers generally results in poorer performance.' Please clarify whether this is a typo or a genuine exception.","section":"Table 22 (Appendix D)"},{"comment":"The sentence 'the workaround strategy of differentiable sparse selection proposed in this study' appears to describe a technique not introduced anywhere in the paper; DOFEN does not use differentiable sparse selection. This sentence should be revised or removed.","section":"Section 6 (Limitation)"}],"recommendation":"major_revision","confidential_remarks":"The key baseline Trompt (ref [24]) shares four authors with this paper. This is not evidence of bias, but it raises the bar for independent verification: the authors should explicitly disclose the relationship and, ideally, re-run Trompt themselves or provide the exact code and configuration used to obtain its numbers. The current manuscript does not state the number of random-search iterations or compute budgets per model, which is a significant omission for a benchmark claim. The paper is otherwise well-structured and the code release is a plus."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The architecture is actually new: it builds relaxed oblivious decision trees by randomly permuting learned per-column conditions, then bagging over randomly sampled rODT forests. That is a real departure from NODE and GRANDE, which learn splits and leaf values, and from Trompt's prompt-style column attention. The two-level ensemble is simple, cheap in parameters, and it works. The paper gives real credit where earned: thorough evaluation on the Tabular Benchmark, ablations, interpretability checks, scalability analysis, and public code. The appendix is unusually transparent about DOFEN's slow inference and the group-convolution bottleneck.\n\nThe central claim—best DNN on the Tabular Benchmark—is consistent with the reported tables and is not obviously an artifact. On medium numerical classification, DOFEN edges Trompt by 0.0008 after search. That margin is tiny, and the paper never reports repeated-seed intervals for the searched results, nor does it state compute budgets per model. Section 4.5's stability numbers are only for default hyperparameters on four datasets where DOFEN already ranks first, so they do not protect the headline comparison. Trompt shares four authors with this paper, and that is not disclosed; it is not evidence of bias, but it raises the bar for independent confirmation. The large-task tables also omit MLP, HGBT, and RandomForest in several conditions, so the aggregate DNN-SOTA claim leans heavily on medium tasks. These are real soft spots, but they are proportionate: none overturns the basic finding, and the ablation shows the forest ensemble matters a lot, which is a clean internal result.\n\nI would send this to peer review. The architecture is worth refereeing, the experiments are extensive enough to check, and the main weakness—missing error bars and budget asymmetry—is fixable in revision. A serious referee should push for repeated-seed results on the searched configurations and a clearer statement of per-model compute. The paper is for people working on tabular deep learning, especially those interested in tree-inspired architectures. It will not change practice for most users, but it is a solid step in a crowded subfield.","headline":"A genuinely new tabular DNN architecture with a plausible but not airtight SOTA claim; worth serious review, but the headline margin over Trompt needs repeated-seed and budget-controlled evidence.","tokens_in":52720,"tokens_out":721,"would_cite":true,"duration_ms":9512,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that DOFEN, a deep network built from randomly assembled relaxed oblivious decision trees, achieves state-of-the-art results among DNNs on the Tabular Benchmark and competes with gradient-boosted trees.","keywords":["tabular data","deep learning","oblivious decision trees","ensemble methods","gradient boosting","Tabular Benchmark","interpretability"],"falsifier":"A re-run of the Tabular Benchmark under the paper's protocol that finds DOFEN's average accuracy or R2 rank among DNNs below Trompt's, or that flips the medium-regression rankings once the missing large-task baselines are included, would settle against the central claim.","tokens_in":1471,"feed_emoji":"🌳","tokens_out":1320,"duration_ms":58351,"temperature":0.7,"pith_summary":"The paper is trying to establish that a deep network can stop lagging behind gradient-boosted decision trees on ordinary tabular data by borrowing the oblivious decision tree's structure. It introduces DOFEN, which generates soft per-column conditions, randomly combines them into relaxed oblivious decision trees (rODTs), groups rODTs into forests, and bags the forests' predictions. On the 73-dataset Tabular Benchmark the authors report that DOFEN ranks first among the DNN models and lands alongside CatBoost and XGBoost, especially on numerical-feature classification and regression tasks. If true, this matters because tabular data is the dominant format in finance, healthcare, and government, where tree ensembles have been the default choice.","feed_headline":"Tree-inspired DNN tops other neural nets on tabular data","feed_subtitle":"DOFEN's relaxed oblivious decision trees close the gap to gradient boosting on the 73-dataset Tabular Benchmark.","key_machinery":"The central object is the relaxed oblivious decision tree (rODT), a decision table whose conditions are soft scores output by a per-column neural sub-network and whose feature/threshold combinations are fixed by a random permutation instead of learned splits. A pool of rODTs is reshaped from the condition matrix; forests are formed by sampling rODT pairs and taking a softmax-weighted sum of paired embedding vectors; and the final prediction is the bagged average over many forests. This machinery replaces the non-differentiable tree-search with random condition combination and differentiable weighting, and the bagging over forests is what the ablation credits for most of the gain.","core_discovery":"The central discovery, on the paper's own terms, is that the non-differentiable tree-building step can be replaced by random condition combination plus differentiable weighting without losing the inductive bias that makes oblivious trees strong on tabular data. DOFEN builds a condition matrix per sample, permutes and reshapes it into a pool of rODTs, learns an importance weight and embedding for each rODT, samples weighted subsets to form forests, and averages forest predictions. The authors report that this design yields the best average accuracy and R2 among all compared DNNs on the benchmark and places DOFEN in the top group with tree-based models, while also supplying feature-importance scores that match tree-based rankings.","pith_inferences":["Editorial extension: the random-condition construction could be used as a frozen feature-extraction front end for semi-supervised or multimodal tabular learning, since the paper's conclusion suggests the rODT pool is a reusable representation rather than a task-specific tree.","Editorial extension: the pruning results imply a cheap testable improvement; keeping only the rODTs whose weights vary most across samples could cut DOFEN's long inference time without retraining.","Editorial extension: the paper's heterogeneous-feature gap suggests a concrete next experiment: applying condition-generation sub-networks with learned categorical thresholds rather than the current shared soft-condition mechanism might recover the missing performance on mixed-feature datasets."],"forward_implications":["On numerical-feature tabular tasks, deep learning can now be reported as the top performer among DNNs and within the top three overall, so the default advice to use gradient-boosted trees loses its blanket force for such datasets.","Tree-inspired architectures that randomize structure rather than learn splits are a viable route to tabular DNNs, and the two-level forest ensemble is the component that prevents overfitting.","DOFEN's feature-importance scores match tree-based importance rankings on mushroom and wine datasets, so the model offers an interpretability story comparable to trees.","Because DOFEN needs no learned split thresholds, it can be trained end-to-end with standard optimizers, making the approach easy to port to new datasets.","The benchmark results suggest that DOFEN is a credible backbone for further tabular research, including self-supervised and multimodal settings."],"supporting_citations":[{"why":"Defines the Tabular Benchmark, its dataset splits, preprocessing, and evaluation protocols that the experiments follow.","marker":"[1]"},{"why":"CatBoost is a main GBDT baseline that DOFEN is compared against.","marker":"[2]"},{"why":"XGBoost provides a strong tree-based baseline, especially in the regression comparisons.","marker":"[3]"},{"why":"Supplies the FT-Transformer and ResNet baseline implementations and the earlier comparison setting that DOFEN extends.","marker":"[4]"},{"why":"Introduces oblivious decision trees, the base model class that DOFEN relaxes.","marker":"[11]"},{"why":"NODE, the prior neural oblivious decision ensemble, is the closest architectural baseline.","marker":"[14]"},{"why":"GRANDE is the tree-inspired DNN baseline with which DOFEN is compared.","marker":"[20]"},{"why":"Trompt is the previous state-of-the-art tabular DNN whose performance DOFEN claims to surpass.","marker":"[24]"}],"fun_headline_variants":["DOFEN: DNN that builds relaxed oblivious forests for tabular wins","Tree-inspired DNN design beats all neural nets on 73 tabular tasks","Random condition mixing gives DNNs a tree's edge on tabular data","New DNN architecture rivals tree models on 73 tabular datasets"],"cache_read_input_tokens":54656,"weakest_assumption_plain":"The load-bearing premise is that the benchmark comparison is fair: DOFEN and every baseline follow the same Tabular Benchmark splits and hyperparameter-search protocol, so the small reported margins reflect the architecture rather than extra tuning, larger compute budgets, or missing baselines.","fun_headline_variants_meta":{"raw":{"variants":["DOFEN: DNN that builds relaxed oblivious forests for tabular wins","Tree-inspired DNN design beats all neural nets on 73 tabular tasks","Random condition mixing gives DNNs a tree's edge on tabular data","New DNN architecture rivals tree models on 73 tabular datasets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000743,"raw_usage":{"total_tokens":3291,"prompt_tokens":900,"completion_tokens":2391,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":516,"completion_tokens_details":{"reasoning_tokens":2310}},"tokens_in":516,"tokens_out":2391,"duration_ms":15903,"temperature":1.0,"reasoning_tokens":2310,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:30:02.853701+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A re-run of the Tabular Benchmark under the paper's protocol that finds DOFEN's average accuracy or R2 rank among DNNs below Trompt's, or that flips the medium-regression rankings once the missing large-task baselines are included, would settle against the central claim.","supporting_citations":[{"cited_title":"Bottom-up induction of oblivious read-once decision graphs","cited_arxiv_id":null,"evidence_quote":"Introduces oblivious decision trees, the base model class that DOFEN relaxes."},{"cited_title":"GRANDE: Gradient-Based Decision Tree Ensembles for Tabular Data","cited_arxiv_id":null,"evidence_quote":"GRANDE is the tree-inspired DNN baseline with which DOFEN is compared."},{"cited_title":"Why do tree-based models still outperform deep learning on typical tabular data? Advances in Neural Information Processing Systems 2022, 35, 507–520","cited_arxiv_id":null,"evidence_quote":"Defines the Tabular Benchmark, its dataset splits, preprocessing, and evaluation protocols that the experiments follow."},{"cited_title":"V .; Gulin, A","cited_arxiv_id":null,"evidence_quote":"CatBoost is a main GBDT baseline that DOFEN is compared against."},{"cited_title":"Xgboost: A scalable tree boosting system","cited_arxiv_id":null,"evidence_quote":"XGBoost provides a strong tree-based baseline, especially in the regression comparisons."},{"cited_title":"Revisiting deep learning models for tabular data","cited_arxiv_id":null,"evidence_quote":"Supplies the FT-Transformer and ResNet baseline implementations and the earlier comparison setting that DOFEN extends."}],"review_version":1}