PFNs, which amortize Bayesian inference by training on datasets sampled from a prior, are likely to supersede MCMC and variational inference for most prediction tasks, the authors argue.
Fine-Tuning the Retrieval Mechanism for Tabular Deep Learning
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
While interests in tabular deep learning has significantly grown, conventional tree-based models still outperform deep learning methods. To narrow this performance gap, we explore the innovative retrieval mechanism, a methodology that allows neural networks to refer to other data points while making predictions. Our experiments reveal that retrieval-based training, especially when fine-tuning the pretrained TabPFN model, notably surpasses existing methods. Moreover, the extensive pretraining plays a crucial role to enhance the performance of the model. These insights imply that blending the retrieval mechanism with pretraining and transfer learning schemes offers considerable potential for advancing the field of tabular deep learning.
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Position: The Future of Bayesian Prediction Is Prior-Fitted
PFNs, which amortize Bayesian inference by training on datasets sampled from a prior, are likely to supersede MCMC and variational inference for most prediction tasks, the authors argue.