pith. sign in

arxiv: 2603.15928 · v2 · submitted 2026-03-16 · 📊 stat.AP

Prior-Data Fitted Networks for Causal Inference: a Simulation Study with Real-World Scenarios

Francisco Mourao (1 , 2) , David Hajage (1 , 3) , Daria Bystrova (1) , Bertrand Bouvarel (1 , Nathana\"el Lapidus (1 , Fabrice Carrat (1
show 14 more authors
Benjamin Glemain (1 2) ((1) Sorbonne Universit\'e Inserm Institut Pierre-Louis d'\'epid\'emiologie et de sant\'e publique Paris France (2) D\'epartement de sant\'e publique H\^opital Saint-Antoine AP-HP. Sorbonne Universit\'e (3) H\^opital Piti\'e-Salp\^etri\`ere D\'epartement de Sant\'e Publique Centre de Pharmaco\'epid\'emiologie Sorbonne Universit\'e France.)
This is my paper

Pith reviewed 2026-05-15 09:28 UTC · model grok-4.3

classification 📊 stat.AP
keywords Prior-Data Fitted NetworksCausal InferenceAverage Treatment EffectSimulation StudyTabular Data PredictionG-computationInverse Probability WeightingCredible Intervals
0
0 comments X

The pith

CausalPFN estimates average treatment effects quickly but its credible intervals fail to cover the true value adequately.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests Prior-Data Fitted Networks on the task of estimating the average treatment effect of a binary treatment on a binary outcome. It uses simulated clinical data drawn from real-world patterns and compares two approaches: TabPFN paired with standard causal tools such as g-computation and inverse probability weighting, and CausalPFN, which outputs an ATE estimate and credible interval directly. TabPFN methods turn out too slow for regular use because bootstrap resampling is needed to form intervals, and g-computation with TabPFN produces large bias unless separate models are fit for each treatment arm. CausalPFN avoids the speed problem yet its 95 percent credible intervals show poor coverage, stemming from both systematic error in the point estimate and insufficient uncertainty calibration. The authors note that PFNs automate model choice but still require further work before they can be trusted for causal tasks.

Core claim

In simulated clinical scenarios based on real-world data, TabPFN combined with g-computation produced highly biased ATE estimates that were only partly corrected by using a T-learner structure, while the overall computation time remained prohibitive because of the need for bootstrap resampling to obtain confidence intervals; CausalPFN, by contrast, ran efficiently yet delivered 95 percent credible intervals with inadequate coverage of the true ATE due to both estimation bias and weak uncertainty quantification.

What carries the argument

Prior-Data Fitted Networks (PFNs), pre-trained neural networks that perform direct inference on new tabular datasets without retraining, here applied either through TabPFN plus causal wrappers or through the specialized CausalPFN variant that targets the ATE.

If this is right

  • G-computation using TabPFN yields highly biased estimates of the average treatment effect.
  • Fitting separate TabPFN models for each treatment group reduces bias relative to a single pooled model.
  • CausalPFN achieves low computation time but produces credible intervals that fail to achieve nominal coverage.
  • Bootstrap resampling for TabPFN intervals makes the method impractical for routine causal analysis.
  • Further development of PFN variants is required before they can reliably automate causal modeling.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Directly embedding causal estimation inside a pre-trained network may require explicit mechanisms for handling unobserved confounding that current architectures lack.
  • The calibration problems observed could worsen in datasets containing unmeasured variables not present in the simulations.
  • Applying these networks to benchmark datasets from completed randomized trials would provide a stronger test than simulation alone.
  • Hybrid approaches that combine PFN speed with traditional causal sensitivity checks might address the coverage shortfall.

Load-bearing premise

The simulated clinical scenarios based on real-world data accurately capture the bias and uncertainty patterns that would appear in actual observational studies.

What would settle it

Running CausalPFN on a real observational dataset whose true ATE is later revealed by a randomized trial and checking whether the reported credible intervals cover that true value at the claimed 95 percent rate.

Figures

Figures reproduced from arXiv: 2603.15928 by 2), 2) ((1) Sorbonne Universit\'e, (2) D\'epartement de sant\'e publique, 3), (3) H\^opital Piti\'e-Salp\^etri\`ere, AP-HP. Sorbonne Universit\'e, Benjamin Glemain (1, Bertrand Bouvarel (1, Centre de Pharmaco\'epid\'emiologie, Daria Bystrova (1), David Hajage (1, D\'epartement de Sant\'e Publique, Fabrice Carrat (1, France, France.), Francisco Mourao (1, H\^opital Saint-Antoine, Inserm, Institut Pierre-Louis d'\'epid\'emiologie et de sant\'e publique, Nathana\"el Lapidus (1, Paris, Sorbonne Universit\'e.

Figure 1
Figure 1. Figure 1: Comparison of learning paradigms. D1, D2, . . . is a collection of external datasets used to train the Prior-Data Fitted Network (PFN). D0 denotes the dataset from which one aims to develop a predictive algorithm (e.g., a patient cohort). xj,i denotes the vector of covariates of the patient i and dataset j, and ˆyj,i denotes their predicted outcome. The learning phase aims to find a function that minimizes… view at source ↗
Figure 2
Figure 2. Figure 2: Causal graph representing the causal inference task, where [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Confidence interval metrics for the two simulation scenarios. CI: confidence in [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Point estimates metrics for the two simulation scenarios. GLM: logistic regres [PITH_FULL_IMAGE:figures/full_fig_p016_4.png] view at source ↗
read the original abstract

Prior-Data Fitted Networks (PFNs) represent a paradigm shift in tabular data prediction. We present the principles of this new paradigm and evaluate two PFNs for estimating the average treatment effect (ATE) of a binary treatment on a binary outcome, using simulated clinical scenarios based on real-world data. We assessed TabPFN combined with causal inference procedures (g-computation and inverse probability of treatment weighting), and CausalPFN, a PFN that directly provides an ATE estimate with a credible interval. Confidence intervals for the TabPFN-based methods were derived using bootstrap resampling. We found that computation times for TabPFN were prohibitive for routine causal inference, particularly because of the need for bootstrapping to yield confidence intervals. Moreover, g-computation with TabPFN produced a highly biased estimator, partially corrected by fitting separate models for each treatment group (T-learner). CausalPFN, by contrast, was computationally efficient but exhibited poor coverage of its 95% credible interval for the ATE, due to both estimation bias and inadequate uncertainty quantification. Beyond automating model specification, some PFN variants - like CausalPFN - attempt to automate causal modeling. In the settings we evaluated, CausalPFN performed poorly. However, new algorithms of this kind continue to be developed, and their application to causal inference tasks requires further investigation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript evaluates Prior-Data Fitted Networks (PFNs) for average treatment effect (ATE) estimation on binary outcomes in simulated clinical scenarios derived from real-world data. It compares TabPFN combined with g-computation and inverse probability weighting (with bootstrap confidence intervals) against CausalPFN, which directly outputs an ATE point estimate and credible interval. The authors report that TabPFN-based approaches are computationally prohibitive due to bootstrapping, that g-computation yields substantial bias (partially mitigated by a T-learner variant), and that CausalPFN is fast but exhibits poor 95% credible-interval coverage attributable to both bias and miscalibrated uncertainty.

Significance. If the simulation design faithfully reproduces the confounding, selection, and outcome mechanisms of the source observational studies, the results supply concrete empirical evidence on the current limitations of PFN architectures for causal tasks—particularly their uncertainty quantification and scalability. The work also illustrates the practical trade-offs between automated model specification and the need for explicit causal modeling steps, providing a useful benchmark for subsequent PFN variants.

major comments (2)
  1. [Abstract / Results] Abstract and Results section: the central claim that CausalPFN shows 'poor coverage of its 95% credible interval for the ATE, due to both estimation bias and inadequate uncertainty quantification' is unsupported by any reported numerical values (bias, coverage probability, interval width, or effective sample size). Without these quantities the magnitude and practical relevance of the finding cannot be assessed.
  2. [Simulation Setup] Simulation Setup (likely §3): the manuscript states that scenarios are 'based on real-world data' yet provides no quantitative validation—e.g., standardized mean differences before/after weighting, comparison of marginal outcome distributions, or sensitivity checks for unmeasured confounding. This validation is load-bearing for the inference that the observed bias and miscalibration are properties of CausalPFN rather than artifacts of the data-generating process.
minor comments (2)
  1. [Abstract] The abbreviation 'T-learner' is used without definition or citation on first appearance.
  2. [Results] Computation times are described as 'prohibitive' without reporting wall-clock figures or hardware specifications, preventing direct comparison with alternative methods.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and detailed comments on our manuscript. We address each major comment below and describe the revisions we will make to improve clarity and strengthen the supporting evidence for our claims.

read point-by-point responses

  1. Referee: [Abstract / Results] Abstract and Results section: the central claim that CausalPFN shows 'poor coverage of its 95% credible interval for the ATE, due to both estimation bias and inadequate uncertainty quantification' is unsupported by any reported numerical values (bias, coverage probability, interval width, or effective sample size). Without these quantities the magnitude and practical relevance of the finding cannot be assessed.

    Authors: We agree that the abstract would be strengthened by explicitly including the key numerical results already present in the results section and supplementary tables. In the revised version we will add concise quantitative summaries (coverage probabilities, bias magnitudes, and interval widths) directly into the abstract to make the central claim self-contained and allow immediate assessment of practical relevance. revision: yes

  2. Referee: [Simulation Setup] Simulation Setup (likely §3): the manuscript states that scenarios are 'based on real-world data' yet provides no quantitative validation—e.g., standardized mean differences before/after weighting, comparison of marginal outcome distributions, or sensitivity checks for unmeasured confounding. This validation is load-bearing for the inference that the observed bias and miscalibration are properties of CausalPFN rather than artifacts of the data-generating process.

    Authors: We acknowledge that additional quantitative validation of the simulation design would increase transparency and reader confidence. Although the scenarios were generated by fitting parametric models to real observational datasets and then simulating from those fitted distributions, we did not report balance diagnostics or marginal distribution comparisons in the submitted version. We will add these checks (standardized mean differences, propensity score overlap plots, and simulated vs. observed outcome distributions) to the revised methods and results sections. revision: yes

Circularity Check

0 steps flagged

Empirical simulation study with external benchmarks

full rationale

The paper is a simulation-based empirical comparison of PFN variants against known ground-truth ATE values generated from real-world-derived scenarios. No derivation chain, equation, or prediction reduces to its own inputs by construction; all performance claims (bias, coverage, runtime) are evaluated externally via Monte Carlo replication on held-out simulated data. No self-citation is load-bearing for any core result, and the work contains no ansatz, uniqueness theorem, or renaming step.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The evaluation rests on standard causal assumptions (no unmeasured confounding, positivity, consistency) that are invoked implicitly when applying g-computation and IPTW. No new free parameters or invented entities are introduced in the abstract; the simulations themselves contain data-generation parameters that are not detailed here.

axioms (2)
  • domain assumption No unmeasured confounding between treatment and outcome
    Required for g-computation and IPTW to recover the true ATE; invoked when the authors treat the simulated data as ground truth.
  • domain assumption Positivity (every patient has positive probability of receiving either treatment)
    Standard assumption needed for inverse-probability weighting to be well-defined.

pith-pipeline@v0.9.0 · 5703 in / 1430 out tokens · 35185 ms · 2026-05-15T09:28:24.071637+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. IV-ICL: Bounding Causal Effects with Instrumental Variables via In-Context Learning

    cs.LG 2026-05 unverdicted novelty 7.0

    IV-ICL learns the marginal posterior of causal effects via in-context learning to derive bounds as quantiles, recovering the identified set more reliably than variational inference while running 20-500x faster.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages · cited by 1 Pith paper

  1. [1]

    Transformers Can Do Bayesian Inference inInternational Conference on Learning Representations2022

    M¨ uller S, Hollmann N, Arango SP, Grabocka J, Hutter F. Transformers Can Do Bayesian Inference inInternational Conference on Learning Representations2022

    work page

  2. [2]

    Garg S, Tsipras D, Liang P, Valiant G. What can transformers learn in-context? a case study of simple function classes inProceedings of the 36th International Confer- ence on Neural Information Processing SystemsNIPS ’22(Red Hook, NY, USA)Curran Associates Inc. 2022

    work page 2022

  3. [3]

    On the Opportunities and Risks of Founda- tion Models 2022

    Bommasani R, Hudson DA, Adeli E, et al. On the Opportunities and Risks of Founda- tion Models 2022

    work page 2022

  4. [4]

    MotherNet: Fast Training and Inference via Hyper-Network Transformers inThe Thirteenth International Conference on Learning Representations2025

    Mueller AC, Curino CA, Ramakrishnan R. MotherNet: Fast Training and Inference via Hyper-Network Transformers inThe Thirteenth International Conference on Learning Representations2025

    work page

  5. [5]

    Statistical foundations of prior-data fitted networks inProceedings of the 40th International Conference on Machine LearningICML’23JMLR.org 2023

    Nagler T. Statistical foundations of prior-data fitted networks inProceedings of the 40th International Conference on Machine LearningICML’23JMLR.org 2023. 20

    work page 2023

  6. [6]

    Accurate predictions on small data with tabular foundation modelNature.2025;637:319-326

    Hollmann N, Muller S, Purucker L, et al. Accurate predictions on small data with tabular foundation modelNature.2025;637:319-326

    work page 2025

  7. [7]

    Cambridge University Press

    Pearl J.Causality: Models, Reasoning and Inference. Cambridge University Press. 2nd ed. 2009

    work page 2009

  8. [8]

    TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second 2023

    Hollmann N, M¨ uller S, Eggensperger K, Hutter F. TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second 2023

    work page 2023

  9. [9]

    TabPFN: One Model to Rule Them All? 2025

    Zhang Q, Tan YS, Tian Q, Li P. TabPFN: One Model to Rule Them All? 2025

    work page 2025

  10. [10]

    The central role of the propensity score in observational studies for causal effectsBiometrika.1983;70:41-55

    Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effectsBiometrika.1983;70:41-55

    work page 1983

  11. [11]

    Robins J. A new approach to causal inference in mortality studies with a sustained ex- posure period—application to control of the healthy worker survivor effectMathematical Modelling.1986;7:1393-1512

    work page 1986

  12. [12]

    Causal diagrams for empirical researchBiometrika.1995;82:669-688

    Pearl J. Causal diagrams for empirical researchBiometrika.1995;82:669-688

    work page 1995

  13. [13]

    Implementation of G-Computation on a Simu- lated Data Set: Demonstration of a Causal Inference TechniqueAmerican Journal of Epidemiology.2010;173

    Snowden JM, Rose S, Mortimer KM. Implementation of G-Computation on a Simu- lated Data Set: Demonstration of a Causal Inference TechniqueAmerican Journal of Epidemiology.2010;173

    work page 2010

  14. [14]

    The value added of machine learning to causal inference: evidence from revisited studiesThe Econometrics Journal.2024;27:213-234

    Baiardi A, Naghi AA. The value added of machine learning to causal inference: evidence from revisited studiesThe Econometrics Journal.2024;27:213-234

    work page 2024

  15. [15]

    Double/debiased machine learning for treatment and structural parametersThe Econometrics Journal.2018;21:C1-C68

    Chernozhukov V, Chetverikov D, Demirer M, et al. Double/debiased machine learning for treatment and structural parametersThe Econometrics Journal.2018;21:C1-C68

    work page 2018

  16. [16]

    Challenges in Obtaining Valid Causal Effect Estimates With Machine Learning AlgorithmsAmerican Journal of Epidemiology

    Naimi AI, Mishler AE, Kennedy EH. Challenges in Obtaining Valid Causal Effect Estimates With Machine Learning AlgorithmsAmerican Journal of Epidemiology. 2021;192:1536-1544. 21

    work page 2021

  17. [17]

    Doubly Robust Estimation in Missing Data and Causal Inference ModelsBiometrics.2005;61:962-973

    Bang H, Robins JM. Doubly Robust Estimation in Missing Data and Causal Inference ModelsBiometrics.2005;61:962-973

    work page 2005

  18. [18]

    Targeted maximum likelihood learningThe International Journal of Biostatistics.2006;2

    Laan MJ, Rubin D. Targeted maximum likelihood learningThe International Journal of Biostatistics.2006;2

    work page 2006

  19. [19]

    Doubly ro- bust estimation of causal effectsAmerican Journal of Epidemiology.2011;173:761–767

    Funk MJ, Westreich D, Wiesen C, St¨ urmer T, Brookhart MA, Davidian M. Doubly ro- bust estimation of causal effectsAmerican Journal of Epidemiology.2011;173:761–767

    work page 2011

  20. [20]

    Metalearners for estimating heterogeneous treatment effects using machine learningProceedings of the National Academy of Sci- ences.2019;116:4156–4165

    K¨ unzel SR, Sekhon JS, Bickel PJ, Yu B. Metalearners for estimating heterogeneous treatment effects using machine learningProceedings of the National Academy of Sci- ences.2019;116:4156–4165

    work page 2019

  21. [21]

    CausalPFN: Amortized Causal Effect Esti- mation via In-Context Learning 2025

    Balazadeh V, Kamkari H, Thomas V, et al. CausalPFN: Amortized Causal Effect Esti- mation via In-Context Learning 2025

    work page 2025

  22. [22]

    Do-PFN: In-Context Learning for Causal Effect Estimation inThe Thirty- ninth Annual Conference on Neural Information Processing Systems2025

    Robertson Jake, Reuter Arik, Guo Siyuan, Hollmann Noah, Hutter Frank, Sch¨ olkopf Bernhard. Do-PFN: In-Context Learning for Causal Effect Estimation inThe Thirty- ninth Annual Conference on Neural Information Processing Systems2025

    work page

  23. [23]

    R package version 1.7.8.1

    Chen T, He T, Benesty M, et al.xgboost: Extreme Gradient Boosting2024. R package version 1.7.8.1

    work page

  24. [24]

    Taylor and Francis 2024

    Hernan MA, Robins JM.Causal inference: What if. Taylor and Francis 2024

    work page 2024

  25. [25]

    Higgins P.medicaldata: Data Package for Medical Datasets2023

    work page

  26. [26]

    R package version 3.8-3

    Therneau TM.A Package for Survival Analysis in R2024. R package version 3.8-3

    work page

  27. [27]

    Salditt M, Eckes T, Nestler S. A Tutorial Introduction to Heterogeneous Treatment Effect Estimation with Meta-learnersAdministration and Policy in Mental Health and Mental Health Services Research.2024;51:650-673. 22

    work page 2024

  28. [28]

    Kostouraki A, Hajage D, Rachet B, et al. On variance estimation of the inverse probability-of-treatment weighting estimator: A tutorial for different types of propensity score weightsStatistics in Medicine.2024;43:2672–2694

    work page 2024

  29. [29]

    Using simulation studies to evaluate statistical methodsStatistics in Medicine.2019;38:2074–2102

    Morris Tim P., White Ian R., Crowther Michael J.. Using simulation studies to evaluate statistical methodsStatistics in Medicine.2019;38:2074–2102

    work page 2019

  30. [30]

    R Foundation for Statistical ComputingVienna, Austria 2024

    R Core Team .R: A Language and Environment for Statistical Computing. R Foundation for Statistical ComputingVienna, Austria 2024

    work page 2024

  31. [31]

    Scotts Valley, CA: CreateSpace 2009

    Van Rossum G, Drake FL.Python 3 Reference Manual. Scotts Valley, CA: CreateSpace 2009

    work page 2009

  32. [32]

    A Randomized Trial of Rectal In- domethacin to Prevent Post-ERCP PancreatitisNew England Journal of Medicine

    Elmunzer BJ, Scheiman JM, Lehman GA, et al. A Randomized Trial of Rectal In- domethacin to Prevent Post-ERCP PancreatitisNew England Journal of Medicine. 2012;366:1414-1422

    work page 2012

  33. [33]

    External validation of a Cox prognostic model: Principles and methodsBMC Medical Research Methodology.2013;13

    Royston P, Altman DG. External validation of a Cox prognostic model: Principles and methodsBMC Medical Research Methodology.2013;13

    work page 2013

  34. [34]

    Automating the practice of science: Opportunities, challenges, and implicationsProceedings of the National Academy of Sciences.2025;122:e2401238121

    Musslick S, Bartlett LK, Chandramouli SH, et al. Automating the practice of science: Opportunities, challenges, and implicationsProceedings of the National Academy of Sciences.2025;122:e2401238121. 23 Supplementary Material to Prior-Data Fitted Networks for Causal Inference: a Simulation Study with Real-World Scenarios Table of contents A) Empirical Distr...

    work page 2025