REVIEW 9 cited by
Representation Learning for Tabular Data: A Comprehensive Survey
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Representation Learning for Tabular Data: A Comprehensive Survey
read the original abstract
Tabular data, structured as rows and columns, is among the most prevalent data types in machine learning classification and regression applications. Models for learning from tabular data have continuously evolved, with Deep Neural Networks (DNNs) recently demonstrating promising results through their capability of representation learning. In this survey, we systematically introduce the field of tabular representation learning, covering the background, challenges, and benchmarks, along with the pros and cons of using DNNs. We organize existing methods into three main categories according to their generalization capabilities: specialized, transferable, and general models. Specialized models focus on tasks where training and evaluation occur within the same data distribution. We introduce a hierarchical taxonomy for specialized models based on the key aspects of tabular data -- features, samples, and objectives -- and delve into detailed strategies for obtaining high-quality feature- and sample-level representations. Transferable models are pre-trained on one or more datasets and subsequently fine-tuned on downstream tasks, leveraging knowledge acquired from homogeneous or heterogeneous sources, or even cross-modalities such as vision and language. General models, also known as tabular foundation models, extend this concept further, allowing direct application to downstream tasks without fine-tuning. We group these general models based on the strategies used to adapt across heterogeneous datasets. Additionally, we explore ensemble methods, which integrate the strengths of multiple tabular models. Finally, we discuss representative extensions of tabular learning, including open-environment tabular machine learning, multimodal learning with tabular data, and tabular understanding. More information can be found in the following repository: https://github.com/LAMDA-Tabular/Tabular-Survey.
Forward citations
Cited by 9 Pith papers
-
TabArena: A Living Benchmark for Machine Learning on Tabular Data
TabArena launches a dynamic, updatable benchmarking system for tabular ML that shows boosted trees remain competitive, deep learning matches them under larger budgets with ensembling, foundation models excel on small ...
-
On the Robustness of Tabular Foundation Models: Test-Time Attacks and In-Context Defenses
Tabular foundation models suffer from test-time adversarial vulnerabilities that degrade accuracy and enable transferable attacks, but incremental adversarial in-context learning improves robustness on multiple benchmarks.
-
Foundation Models for Credit Risk Prediction: A Game Changer?
Tabular foundation models, used zero-shot, match or beat tuned gradient boosting on average in credit PD and LGD benchmarks, with a larger edge on small datasets.
-
Accelerometry-Derived Digital Biomarkers for Cardiometabolic Risk: A Population-Representative Tabular Benchmark with Uncertainty Quantification
Introduces NHANES Accelerometry Cardiometabolic Benchmark from 1,381 adults and shows TabPFN v2 outperforms ridge and XGBoost on HbA1c and CRP prediction while conformal intervals show subgroup undercoverage.
-
Foundation Models for Credit Risk Prediction: A Game Changer?
Tabular foundation models outperform standard methods in credit risk PD and LGD tasks, with larger gains on smaller datasets when used out-of-the-box.
-
Evaluating Tabular Representation Learning for Network Intrusion Detection
Tabular representation learning for network intrusion detection exhibits strong dataset-model dependency, with supervised methods outperforming unsupervised anomaly detection and limited but possible cross-dataset gen...
-
End-to-End Compression for Tabular Foundation Models
TACO compresses a training table into a few learned latent rows, cutting repeated-batch inference cost up to ~94x and memory up to ~97% while losing ≤0.005 ROC-AUC against its uncompressed same-architecture baseline.
-
CURED: Creating, Understanding, and Repairing Errors Demonstrator
CURED is a web demo that injects realistic errors into tabular data, cleans them with conformal ML, and detects the error mechanism at work.
-
TabSwift: An Efficient Tabular Foundation Model with Row-Wise Attention
TabSwift introduces an efficient row-wise attention tabular foundation model competitive with TabPFN v2 and TabICL via gated attention stabilization and register tokens, plus adaptive layer-wise early-exit for inference.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.