REVIEW 6 cited by
Position: The Future of Bayesian Prediction Is Prior-Fitted
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Training neural networks on randomly generated artificial datasets yields Bayesian models that capture the prior defined by the dataset-generating distribution. Prior-data Fitted Networks (PFNs) are a class of methods designed to leverage this insight. In an era of rapidly increasing computational resources for pre-training and a near stagnation in the generation of new real-world data in many applications, PFNs are poised to play a more important role across a wide range of applications. They enable the efficient allocation of pre-training compute to low-data scenarios. Originally applied to small Bayesian modeling tasks, the field of PFNs has significantly expanded to address more complex domains and larger datasets. This position paper argues that PFNs and other amortized inference approaches represent the future of Bayesian inference, leveraging amortized learning to tackle data-scarce problems. We thus believe they are a fruitful area of research. In this position paper, we explore their potential and directions to address their current limitations.
Forward citations
Cited by 6 Pith papers
-
SurvivalPFN: Amortizing Survival Prediction via In-Context Bayesian Inference
SurvivalPFN amortizes Bayesian survival analysis for right-censored data by pretraining a prior-data fitted network on synthetic identifiable DGPs and then performing in-context inference, achieving competitive result...
-
TabPFN beyond Tabular Data: Calibration and Accuracy on Multimodal Embeddings
TabPFN as a zero-gradient head on frozen multimodal embeddings ranks best on NLL and ECE across 22 820 episodes while matching accuracy in mid-shot, mid-dimension regimes and also fixes miscalibration after fine-tuning.
-
In-Context Learning for Latent Space Bayesian Optimization
Complementing tabular foundation model pretraining with LSBO-specific synthetic tasks and a regularizer yields strong performance on held-out molecular optimization benchmarks.
-
In-Context Learning of Temporal Point Processes with Foundation Inference Models
A pretrained in-context transformer infers Hawkes-style conditional intensities from event histories and transfers zero-shot to real-world event data, roughly matching specialized models after finetuning.
-
TabPFN beyond Tabular Data: Calibration and Accuracy on Multimodal Embeddings
TabPFN as a training-free head on PCA-reduced frozen multimodal embeddings broadly improves calibration (NLL, ECE) over classical heads, with an accuracy edge only for k≥50 shots and d≤32 features.
-
Reinforcement Learning Foundation Models Should Already Be A Thing
A Graph Attention Network pretrained solely on synthetic MDPs solves held-out tabular RL benchmarks in context, outperforming UCB-VI and Q-learning online while matching VI-LCB offline.
Discussion (0). Sign in to comment.