REVIEW 7 cited by
TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
While tabular classification has traditionally relied on from-scratch training, a recent breakthrough called prior-data fitted networks (PFNs) challenges this approach. Similar to large language models, PFNs make use of pretraining and in-context learning to achieve strong performance on new tasks in a single forward pass. However, current PFNs have limitations that prohibit their widespread adoption. Notably, TabPFN achieves very strong performance on small tabular datasets but is not designed to make predictions for datasets of size larger than 1000. In this work, we overcome these limitations and substantially improve the performance of PFNs via context optimization. We introduce TuneTables, a parameter-efficient fine-tuning strategy for PFNs that compresses large datasets into a smaller learned context. We conduct extensive experiments on 19 algorithms over 98 datasets and find that TuneTables achieves the best performance on average, outperforming boosted trees such as CatBoost, while optimizing fewer than 5% of TabPFN's parameters. Furthermore, we show that TuneTables can be used as an interpretability tool and can even be used to mitigate biases by optimizing a fairness objective. We open-source our code and raw results at https://github.com/penfever/TuneTables.
Forward citations
Cited by 7 Pith papers
-
On Finetuning Tabular Foundation Models
Full finetuning of TabPFNv2 outperforms in-context learning and partial finetuning on medium tabular datasets, and its gains come from sharper query-key attention that better reflects target similarity.
-
TabFlex: Scaling Tabular Learning to Millions with Linear Attention
Linear attention lets a TabPFN-style model process millions of tabular samples in seconds with near-identical accuracy on small datasets.
-
Table Foundation Models: on knowledge pre-training for tabular learning
TARTE is a pre-trained transformer that represents table rows using column names and cell strings, and its frozen or fine-tuned embeddings improve tabular prediction with lower compute cost.
-
TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problems
BETA augments TabPFN with encoder fine-tuning and bagging to reduce both bias and variance, achieving SOTA accuracy on 200+ tabular benchmarks while scaling to larger and higher-dimensional data.
-
Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data
Drift-Resilient TabPFN learns to predict under temporal distribution shifts by pre-training on structural causal models whose edge weights drift over time, improving OOD accuracy and calibration on small tabular datasets.
-
Table2Image: Interpretable Tabular Data Classification with Realistic Image Transformations
Table2Image pairs each tabular row with a random same-class MNIST or Fashion-MNIST image, trains an autoencoder plus small CNN, and reports competitive classification with a VIF initialization and a dual SHAP explanat...
-
Position: The Future of Bayesian Prediction Is Prior-Fitted
PFNs, which amortize Bayesian inference by training on datasets sampled from a prior, are likely to supersede MCMC and variational inference for most prediction tasks, the authors argue.
Discussion (0). Continue with ORCID to comment.