Basis transformers beat fine-tuned LLMs on 34 multi-task tabular regression datasets while using five times fewer parameters and no data preprocessing.
TabKANet: Tabular Data Modeling with Kolmogorov-Arnold Network and Transformer
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Tabular data is the most common type of data in real-life scenarios. In this study, we propose the TabKANet model for tabular data modeling, which targets the bottlenecks in learning from numerical content. We constructed a Kolmogorov-Arnold Network (KAN) based Numerical Embedding Module and unified numerical and categorical features encoding within a Transformer architecture. TabKANet has demonstrated stable and significantly superior performance compared to Neural Networks (NNs) across multiple public datasets in binary classification, multi-class classification, and regression tasks. Its performance is comparable to or surpasses that of Gradient Boosted Decision Tree models (GBDTs). Our code is publicly available on GitHub: https://github.com/AI-thpremed/TabKANet.
citation-role summary
baseline 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
baseline 1polarities
baseline 1representative citing papers
citing papers explorer
-
Basis Transformers for Multi-Task Tabular Regression
Basis transformers beat fine-tuned LLMs on 34 multi-task tabular regression datasets while using five times fewer parameters and no data preprocessing.