REVIEW 3 major objections 5 minor 36 references
Tabular classification works better when decision conditions live in hyperbolic space rather than flat Euclidean embeddings.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-14 17:22 UTC pith:XBK3H4IV
load-bearing objection Clean LDN + soft-routing + Poincaré construction that posts top average rank on TALENT-tiny-core; ranking is real but still needs code and variance to be fully secured. the 3 major comments →
Manifold Constrained Tabular Deep Neural Networks
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper shows that abstracting heterogeneous tabular features into unified Latent Decision Nodes and embedding them in the Poincaré ball lets a lightweight neural network capture tree-like decision structure more efficiently than either pure Euclidean tabular DNNs or classical GBDTs, yielding the best average rank on the TALENT-tiny-core classification benchmark.
What carries the argument
Hyperbolic Decision Embeddings (HDEs): Latent Decision Nodes placed in the Poincaré ball; Soft Decision Routing turns continuous numerical values into soft multi-node weights, and Riemannian optimization keeps the embeddings on the manifold while a tangent-space MLP finishes the prediction.
Load-bearing premise
The claim rests on the idea that the Poincaré ball's negative curvature still supplies a useful hierarchical bias even when the data never supply an explicit tree.
What would settle it
Run the same 30-dataset protocol with the hyperbolic embeddings replaced by ordinary Euclidean embeddings of identical dimension and capacity allocation; if average rank does not drop, the geometric inductive bias is not doing the claimed work.
If this is right
- Geometry-aware constraints can close the long-standing performance gap between deep networks and GBDTs on mixed tabular data.
- Numerical and categorical features can share a single value-level semantic unit (the LDN) without heavy tokenization or retrieval.
- A shallow MLP plus hyperbolic embeddings can replace deeper Transformer or retrieval backbones while improving both accuracy and latency.
- Entropy-driven capacity allocation removes the need for a single fixed number of bins or tokens per numerical feature.
Where Pith is reading between the lines
- The same LDN-plus-hyperbolic pattern may transfer to ranking, multi-label, or survival tabular tasks where local rules also dominate.
- If the radial hierarchy observed in the visualizations is stable, one could read off approximate decision depth directly from embedding norms at inference time.
- Soft Decision Routing alone already beats several strong baselines, so the idea of differentiable range nodes could be useful even without hyperbolic geometry.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes HDE-Net, a tabular DNN that unifies heterogeneous features as Latent Decision Nodes (LDNs), embeds them in the Poincaré ball as Hyperbolic Decision Embeddings, and uses Soft Decision Routing plus entropy-aware capacity allocation for numerical features. The central empirical claim is that this construction yields the best average rank (4.1667) on the TALENT-tiny-core classification benchmark of 30 datasets against 31 baselines (Table 2, Fig. 2), with supporting ablations (Table 3), capacity sensitivity (Fig. 4), efficiency numbers (Table 4), and geometric visualizations (Figs. 5–6).
Significance. If the ranking and geometric story hold under independent re-execution, the work supplies a concrete, end-to-end alternative to both GBDTs and Euclidean tabular DNNs that is competitive on a balanced public leaderboard while remaining linear in feature count and independent of training-set size. Strengths include a clear geometric motivation, an external benchmark protocol, ablations that partially isolate routing versus hyperbolic geometry, and qualitative evidence that router weights and radial HDE layouts resemble tree structure. The contribution is therefore of practical interest to tabular representation learning even if the precise source of the gain remains partly entangled.
major comments (3)
- [§5.2, Table 2, Fig. 2] The sole quantitative support for the geometric-mismatch thesis is the best average rank of 4.1667 on TALENT-tiny-core (Table 2, Fig. 2). No per-run standard deviations, seeds, or released code are provided, so the Wilcoxon–Holm ranks cannot be recomputed or checked for sensitivity to the 10-run protocol. Without independent re-execution it is impossible to separate the claimed hyperbolic inductive bias from ordinary engineering advantages (entropy-aware k, hybrid optimizers, lightweight MLP).
- [§5.4, Table 3] Table 3 shows that Soft Decision Routing alone (HDE-noHyp, rank 7.10) already surpasses ModernNCA (8.70), while the hyperbolic manifold supplies a further gain to 4.93. Both gains are reported only as relative ranks inside a changing model pool; absolute accuracy deltas and variance are absent. This leaves open whether the Poincaré geometry itself is load-bearing or whether the routing + capacity design is the dominant factor.
- [Introduction, §§3–4.3, Figs. 5–6] The geometric inductive-bias argument (Introduction, §§3–4.3) rests on the assumption that negative curvature and exponential volume growth remain useful when tabular data supply no explicit tree topology. The visualizations (Figs. 5–6) are qualitative and use XGBoost cover rates as an external proxy; a controlled experiment that isolates curvature (e.g., fixed Euclidean vs. hyperbolic embeddings with identical routing and capacity) with statistical tests would strengthen the claim.
minor comments (5)
- [Table 1] Table 1 caption and body contain a duplicated “1).” and a stray “1).” after the parenthetical rank-shift note; clean the formatting.
- [Running headers] Author placeholder “Trovato et al.” appears in running headers; replace with the actual author list.
- [§4.4, Eq. (4)] Eq. (4) uses a simple 1/k_j scaling; a short justification or ablation against softmax or L2 normalization would clarify the design choice.
- [§5.1] The note on rank inconsistency across sections (§5.1) is helpful but could be moved earlier so readers do not mis-compare absolute ranks between Table 2 and Table 3.
- [§5.6] Footnote on ozone_level treating numerical features as categorical is important; consider elevating it into the main text of §5.6.
Circularity Check
No circularity: empirical architecture + external TALENT benchmark; geometric inductive bias is standard literature, not self-derived by construction.
full rationale
The paper's load-bearing claim is an empirical ranking (best average rank 4.1667 via Wilcoxon–Holm on the public TALENT-tiny-core 30-dataset leaderboard against 31 baselines whose results are taken from the official repository). The architectural construction (LDNs, Soft Decision Routing via linear weights W_j = a_j x̂ + b_j, entropy-aware k_j = min(k_max, ⌊2^H⌋, ⌊log2 N_eff⌋), Poincaré embeddings projected by log_0, hybrid Riemannian/Euclidean optimizers) is a design choice, not a derivation that reduces a claimed prediction to a fitted free parameter. Hyperbolic volume growth as a continuous analogue of trees is cited from independent literature (Nickel & Kiela 2017 and subsequent surveys). Ablations (HDE-noHyp, HDE-noRouter) and visualizations (router weight intersections vs. scikit-learn DT thresholds; MDS of categorical HDEs colored by independent XGBoost cover rates) use external proxies and do not close a definitional loop. No self-citation supplies a uniqueness theorem or ansatz that forces the result; the single overlapping-author reference is ordinary related work. The evaluation protocol is therefore self-contained against an external benchmark and exhibits none of the six circularity patterns.
Axiom & Free-Parameter Ledger
free parameters (4)
- HDE embedding dimension d =
12
- k_min / k_max capacity bounds =
2 / 8
- curvature c
- MLP hidden size / learning rate / weight decay =
TALENT defaults or 384/3e-4/1e-5
axioms (3)
- domain assumption Poincaré-ball volume grows exponentially with radius and therefore supplies a continuous analogue of tree branching
- domain assumption Tabular classification is frequently governed by local, condition-triggered rules rather than smooth global patterns
- ad hoc to paper Histogram Shannon entropy plus log of effective sample size yields a useful capacity for each numeric feature
invented entities (4)
-
Latent Decision Nodes (LDNs)
no independent evidence
-
Soft Decision Routing
no independent evidence
-
Hyperbolic Decision Embeddings (HDEs)
no independent evidence
-
Entropy-aware Capacity Allocation
no independent evidence
read the original abstract
Tabular classification is often governed by local, condition-triggered rules rather than smooth global patterns. However, tabular deep neural networks (DNNs) are typically built upon Euclidean representations that favor smooth variations and semantic locality. This potential geometric mismatch can make it challenging for tabular DNNs to efficiently represent the discrete, rule-partitioned structures often underlying tabular classification. To address this issue, we propose HDE-Net, a manifold-constrained DNN that enables hierarchical decision modeling in hyperbolic space. We first abstract heterogeneous features into unified Latent Decision Nodes (LDNs) and embed them in the Poincar\'e ball, forming a continuous representation that resembles tree-structured reasoning. For numerical features, we introduce a Soft Decision Routing mechanism that approximates range-based local rules in a differentiable manner, bringing their LDN semantics closer to those of categorical features. An entropy-aware capacity allocation algorithm further adapts the number of LDNs per numerical feature to balance expressiveness and complexity. On the TALENT-tiny-core classification benchmark (30 datasets), HDE-Net achieves the \textit{best average rank}, outperforming both industrial GBDTs and recent tabular DNNs while maintaining high efficiency.
Figures
Reference graph
Works this paper leans on
-
[1]
Deepfm: a factorization-machine based neural network for ctr prediction
Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. Deepfm: a factorization-machine based neural network for ctr prediction. InProceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI’17, page 1725–1731. AAAI Press, 2017
2017
-
[2]
A survey of data mining and machine learning methods for cyber security intrusion detection.IEEE Communications surveys & tutorials, 18(2):1153–1176, 2015
Anna L Buczak and Erhan Guven. A survey of data mining and machine learning methods for cyber security intrusion detection.IEEE Communications surveys & tutorials, 18(2):1153–1176, 2015
2015
-
[3]
Early prediction of circulatory failure in the intensive care unit using machine learning.Nature medicine, 26(3):364–373, 2020
Stephanie L Hyland, Martin Faltys, Matthias Hüser, Xinrui Lyu, Thomas Gumb- sch, Cristóbal Esteban, Christian Bock, Max Horn, Michael Moor, Bastian Rieck, et al. Early prediction of circulatory failure in the intensive care unit using machine learning.Nature medicine, 26(3):364–373, 2020
2020
-
[4]
Talent: A tabular analytics and learning toolbox.Journal of Machine Learning Research, 26(226):1–16, 2025
Si-Yang Liu, Hao-Run Cai, Qi-Le Zhou, Huai-Hong Yin, Tao Zhou, Jun-Peng Jiang, and Han-Jia Ye. Talent: A tabular analytics and learning toolbox.Journal of Machine Learning Research, 26(226):1–16, 2025
2025
-
[5]
Xgboost: A scalable tree boosting system.Cornell University, 2016
Tianqi Chen. Xgboost: A scalable tree boosting system.Cornell University, 2016
2016
-
[6]
Catboost: unbiased boosting with categorical features
Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Doro- gush, and Andrey Gulin. Catboost: unbiased boosting with categorical features. Advances in neural information processing systems, 31, 2018
2018
-
[7]
Wide & deep learning for recommender systems
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. Wide & deep learning for recommender systems. InProceedings of the 1st workshop on deep learning for recommender systems, pages 7–10, 2016
2016
-
[8]
Deep & cross network for ad click predictions
Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. Deep & cross network for ad click predictions. InProceedings of the ADKDD’17, pages 1–7. 2017
2017
-
[9]
Tabnet: Attentive interpretable tabular learning
Sercan Ö Arik and Tomas Pfister. Tabnet: Attentive interpretable tabular learning. InProceedings of the AAAI conference on artificial intelligence, volume 35, pages 6679–6687, 2021
2021
-
[10]
On embeddings for numer- ical features in tabular deep learning.Advances in Neural Information Processing Systems, 35:24991–25004, 2022
Yury Gorishniy, Ivan Rubachev, and Artem Babenko. On embeddings for numer- ical features in tabular deep learning.Advances in Neural Information Processing Systems, 35:24991–25004, 2022
2022
-
[11]
Realmlp: Advancing mlps and default parameters for tabular data
David Holzmüller, Leo Grinsztajn, and Ingo Steinwart. Realmlp: Advancing mlps and default parameters for tabular data. InELLIS workshop on Representation Learning and Generative Models for Structured Data, 2025
2025
-
[12]
Autoint: Automatic feature interaction learning via self-attentive neural networks
Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. Autoint: Automatic feature interaction learning via self-attentive neural networks. InProceedings of the 28th ACM international conference on information and knowledge management, pages 1161–1170, 2019
2019
-
[13]
Re- visiting deep learning models for tabular data.Advances in neural information processing systems, 34:18932–18943, 2021
Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Re- visiting deep learning models for tabular data.Advances in neural information processing systems, 34:18932–18943, 2021
2021
-
[14]
Excelformer: A neural network surpassing gbdts on tabular data.arXiv preprint arXiv:2301.02819, 2023
Jintai Chen, Jiahuan Yan, Qiyuan Chen, Danny Ziyi Chen, Jian Wu, and Jimeng Sun. Excelformer: A neural network surpassing gbdts on tabular data.arXiv preprint arXiv:2301.02819, 2023
Pith/arXiv arXiv 2023
-
[15]
Dcn v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems
Ruoxi Wang, Rakesh Shivanna, Derek Cheng, Sagar Jain, Dong Lin, Lichan Hong, and Ed Chi. Dcn v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems. InProceedings of the web conference 2021, pages 1785–1797, 2021
2021
-
[16]
Poincaré embeddings for learning hier- archical representations.Advances in neural information processing systems, 30, 2017
Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hier- archical representations.Advances in neural information processing systems, 30, 2017
2017
-
[17]
Hyperbolic deep neural networks: A survey.IEEE Transactions on pattern analysis and machine intelligence, 44(12):10023–10044, 2021
Wei Peng, Tuomas Varanka, Abdelrahman Mostafa, Henglin Shi, and Guoying Zhao. Hyperbolic deep neural networks: A survey.IEEE Transactions on pattern analysis and machine intelligence, 44(12):10023–10044, 2021
2021
-
[18]
Xin Huang, Ashish Khetan, Milan Cvitkovic, and Zohar Karnin. Tabtrans- former: Tabular data modeling using contextual embeddings.arXiv preprint arXiv:2012.06678, 2020
Pith/arXiv arXiv 2012
-
[19]
Tabm: Advancing tabular deep learning with parameter-efficient ensembling
Yury Gorishniy, Akim Kotelnikov, and Artem Babenko. Tabm: Advancing tabular deep learning with parameter-efficient ensembling. InThe Thirteenth Interna- tional Conference on Learning Representations, 2025
2025
-
[20]
Tabr: Tabular deep learning meets nearest neighbors
Yury Gorishniy, Ivan Rubachev, Nikolay Kartashev, Daniil Shlenskii, Akim Kotel- nikov, and Artem Babenko. Tabr: Tabular deep learning meets nearest neighbors. InThe Twelfth International Conference on Learning Representations, 2024
2024
-
[21]
Revisiting nearest neighbor for tabular data: A deep tabular baseline two decades later
Han-Jia Ye, Huai-Hong Yin, De-Chuan Zhan, and Wei-Lun Chao. Revisiting nearest neighbor for tabular data: A deep tabular baseline two decades later. In The Thirteenth International Conference on Learning Representations, 2025
2025
-
[22]
TabPFN: A transformer that solves small tabular classification problems in a second
Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter. TabPFN: A transformer that solves small tabular classification problems in a second. InNeurIPS 2022 First Table Representation Workshop, 2022
2022
-
[23]
Hyperbolic graph convolutional neural networks.Advances in neural information processing systems, 32, 2019
Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec. Hyperbolic graph convolutional neural networks.Advances in neural information processing systems, 32, 2019
2019
-
[24]
A hyperbolic-to-hyperbolic graph convolutional network
Jindou Dai, Yuwei Wu, Zhi Gao, and Yunde Jia. A hyperbolic-to-hyperbolic graph convolutional network. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 154–163, 2021
2021
-
[25]
Fully hyperbolic convolutional neural networks.Research in the Mathematical Sciences, 9(4):60, 2022
Keegan Lensink, Bas Peters, and Eldad Haber. Fully hyperbolic convolutional neural networks.Research in the Mathematical Sciences, 9(4):60, 2022
2022
-
[26]
Hyperbolic image-text representations
Karan Desai, Maximilian Nickel, Tanmay Rajpurohit, Justin Johnson, and Shan- mukha Ramakrishna Vedantam. Hyperbolic image-text representations. In International Conference on Machine Learning, pages 7694–7731. PMLR, 2023
2023
-
[27]
Fast hyperboloid decision tree algorithms
Philippe Chlenski, Ethan Turok, Antonio Khalil Moretti, and Itsik Pe’er. Fast hyperboloid decision tree algorithms. InThe Twelfth International Conference on Learning Representations, 2024
2024
-
[28]
Euclidean and poincare space ensemble xgboost.Information Fusion, 115:102746, 2025
Ponnuthurai Nagaratnam Suganthan, Lingping Kong, Václav Snášel, Varun Ojha, and Hussein Ahmed Hussein Zaky Aly. Euclidean and poincare space ensemble xgboost.Information Fusion, 115:102746, 2025
2025
-
[29]
World Scientific, 2008
Abraham Albert Ungar.Analytic hyperbolic geometry and Albert Einstein’s special theory of relativity. World Scientific, 2008
2008
-
[30]
Hyperbolic graph neural networks: A tutorial on methods and applications
Min Zhou, Menglin Yang, Bo Xiong, Hui Xiong, and Irwin King. Hyperbolic graph neural networks: A tutorial on methods and applications. InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’23, page 5843–5844, New York, NY, USA, 2023. Association for Computing Machinery
2023
-
[31]
Hyperbolic neural networks.Advances in neural information processing systems, 31, 2018
Octavian Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic neural networks.Advances in neural information processing systems, 31, 2018
2018
-
[32]
Riemannian adaptive optimization methods
Gary Becigneul and Octavian-Eugen Ganea. Riemannian adaptive optimization methods. InInternational Conference on Learning Representations, 2019
2019
-
[33]
Lightgbm: A highly efficient gradient boosting decision tree.Advances in neural information processing systems, 30, 2017
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree.Advances in neural information processing systems, 30, 2017
2017
-
[34]
Better by default: Strong pre-tuned mlps and boosted trees on tabular data.Advances in Neural Information Processing Systems, 37:26577–26658, 2024
David Holzmüller, Léo Grinsztajn, and Ingo Steinwart. Better by default: Strong pre-tuned mlps and boosted trees on tabular data.Advances in Neural Information Processing Systems, 37:26577–26658, 2024
2024
-
[35]
Statistical comparisons of classifiers over multiple data sets
Janez Demšar. Statistical comparisons of classifiers over multiple data sets. Journal of Machine learning research, 7(Jan):1–30, 2006
2006
-
[36]
Hyperbolic neural networks++
Ryohei Shimizu, YUSUKE Mukuta, and Tatsuya Harada. Hyperbolic neural networks++. InInternational Conference on Learning Representations, 2021. Trovato et al. Table 6: Models for Comparison Traditional Models dummy, LogReg, NCM, NaiveBayes, knn, svm, xgboost, catboost, RandomForest, lightgbm Deep Learning Models tabpfn, mlp, resnet, node, switchtab, tabnet...
2021
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.