TabEmbed is the first generalist embedding model for tabular data that unifies classification and retrieval in one space via contrastive learning and outperforms text embedding models on the new TabBench benchmark.
SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption
10 Pith papers cite this work, alongside 8 external citations. Polarity classification is still indexing.
10
Pith papers citing it
8
external citations · Pith
abstract
Self-supervised contrastive representation learning has proved incredibly successful in the vision and natural language domains, enabling state-of-the-art performance with orders of magnitude less labeled data. However, such methods are domain-specific and little has been done to leverage this technique on real-world tabular datasets. We propose SCARF, a simple, widely-applicable technique for contrastive learning, where views are formed by corrupting a random subset of features. When applied to pre-train deep neural networks on the 69 real-world, tabular classification datasets from the OpenML-CC18 benchmark, SCARF not only improves classification accuracy in the fully-supervised setting but does so also in the presence of label noise and in the semi-supervised setting where only a fraction of the available training data is labeled. We show that SCARF complements existing strategies and outperforms alternatives like autoencoders. We conduct comprehensive ablations, detailing the importance of a range of factors.
citation-role summary
background 1
baseline 1
citation-polarity summary
representative citing papers
Tabular encoder choice reorders multimodal rankings, can erase apparent fusion gains, and requires non-vanilla extraction for in-context learning models to avoid train-test representation shift.
RelPrism generates self-supervised pseudo-tasks from three attribute perspectives via multi-granularity clustering to improve representation learning for relational database prediction tasks.
The paper proposes Strategic Prior-data Fitted Network (SPN), an inference-time framework that adapts pretrained tabular foundation models (PFNs) to strategic manipulation by aligning predictions with approximated post-manipulation distributions via strategic in-context examples.
TabICL scales in-context learning to large tabular data via column-then-row attention for row embeddings followed by a transformer, matching TabPFNv2 speed and performance while outperforming it and CatBoost on datasets over 10K samples.
TaskFusion combines AGF feature mapping, cross-task augmentation, and distilled replay for continual anomaly detection on heterogeneous tabular data, reporting gains over baselines on 21 datasets.
TAROT constructs and refines LLM-derived task-adaptive semantic graphs then applies GNN message passing to improve few-shot tabular prediction.
Tabular representation learning for network intrusion detection exhibits strong dataset-model dependency, with supervised methods outperforming unsupervised anomaly detection and limited but possible cross-dataset generalization.
Cooperative coevolution and monolithic evolution achieve similar performance gains over baselines in low-label semi-supervised tabular classification.
On five tabular security datasets at 10% labels, tuning only the classifier with Bayesian optimization recovers a median 86% of the gains from full joint SSL-classifier optimization.
citing papers explorer
-
TabEmbed: Benchmarking and Learning Generalist Embeddings for Tabular Understanding
TabEmbed is the first generalist embedding model for tabular data that unifies classification and retrieval in one space via contrastive learning and outperforms text embedding models on the new TabBench benchmark.
-
The Importance of Encoder Choice:A Tabular-Image Study
Tabular encoder choice reorders multimodal rankings, can erase apparent fusion gains, and requires non-vanilla extraction for in-context learning models to avoid train-test representation shift.
-
RelPrism: A Multi-Faceted Pre-training Framework with Self-Generated Tasks for Relational Databases
RelPrism generates self-supervised pseudo-tasks from three attribute perspectives via multi-granularity clustering to improve representation learning for relational database prediction tasks.
-
When Tabular Foundation Models Meet Strategic Tabular Data: A Prior Alignment Approach
The paper proposes Strategic Prior-data Fitted Network (SPN), an inference-time framework that adapts pretrained tabular foundation models (PFNs) to strategic manipulation by aligning predictions with approximated post-manipulation distributions via strategic in-context examples.
-
TabICL: A Tabular Foundation Model for In-Context Learning on Large Data
TabICL scales in-context learning to large tabular data via column-then-row attention for row embeddings followed by a transformer, matching TabPFNv2 speed and performance while outperforming it and CatBoost on datasets over 10K samples.
-
TaskFusion: Continual Anomaly Detection for Heterogeneous Tabular Data
TaskFusion combines AGF feature mapping, cross-task augmentation, and distilled replay for continual anomaly detection on heterogeneous tabular data, reporting gains over baselines on 21 datasets.
-
TAROT: Task-Adaptive Refinement of LLM-prior Graphs for Few-shot Tabular Learning
TAROT constructs and refines LLM-derived task-adaptive semantic graphs then applies GNN message passing to improve few-shot tabular prediction.
-
Evaluating Tabular Representation Learning for Network Intrusion Detection
Tabular representation learning for network intrusion detection exhibits strong dataset-model dependency, with supervised methods outperforming unsupervised anomaly detection and limited but possible cross-dataset generalization.
-
Cooperative Coevolution versus Monolithic Evolutionary Search for Semi-Supervised Tabular Classification
Cooperative coevolution and monolithic evolution achieve similar performance gains over baselines in low-label semi-supervised tabular classification.
-
SemiScope: Disentangling Classifier Tuning and Joint Optimization in Semi-Supervised Security Classification
On five tabular security datasets at 10% labels, tuning only the classifier with Bayesian optimization recovers a median 86% of the gains from full joint SSL-classifier optimization.