Pith. sign in

REVIEW 4 major objections 5 minor 63 references

Realistic Evaluation of TabPFN v2 in Open Environments

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read TabPFN v2, the deep model that beat trees on standard tabular benchmarks, loses that edge once data change: new classes appear, features vanish, distributions shift, or the metric changes.

desk verdict Useful empirical evaluation with a serious unaddressed scale-limit gap and one factual overstatement. read the letter →

arxiv 2505.16226 v1 pith:KNHWJMNG submitted 2025-05-22 cs.LG

Zi-Jian Cheng , Zi-Yi Jia , Zhi Zhou , Yu-Feng Li , Lan-Zhe Guo This is my paper
classification cs.LG
keywords tabulardataopenenvironmentsTabPFNv2tree-basedmodelsdistributionshiftfeaturenewclassdetectionevaluationframework
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether TabPFN v2, a deep learning model that recently overtook tree-based models on standard tabular benchmarks, keeps that edge when real-world conditions change. It builds an evaluation framework covering four open-environment challenges—new classes appearing at test time, features being added or removed, the input distribution shifting, and the evaluation metric changing—and runs it on a range of tabular datasets. Its central claim is that TabPFN v2 is not robust in these settings: it degrades sharply when features are removed, cannot use features added at test time, loses accuracy under concept shift, and shows bias against minority classes. The paper concludes that tree-based models remain the safer default for general tabular tasks, and that TabPFN v2 is only preferable for small-scale, covariate-shifted, class-balanced problems.

What carries the argument

The central object is a unified evaluation framework that instantiates four open-environment challenges—emerging new classes, decremental/incremental features, changing data distributions, and varied learning objectives—as concrete test protocols. New-class detection is run as a leave-one-class-out task scored by ROC-AUC and AUPR. Feature shift is run as random removal of 20% to 100% of features, with a performance-gap metric comparing shifted to original accuracy or RMSE. Distribution shift is measured on nine datasets using covariate-, concept-, and label-shift metrics, including an optimal-transport distance for covariate shift and a Fréchet-based distance for concept shift. Varied learning objectives are scored by accuracy, ROC-AUC, F1, and balanced accuracy. This framework carries the argument because it converts the abstract question of whether TabPFN v2 is robust into comparable numbers across models and tasks.

What would settle it

Run TabPFN v2 against CatBoost, XGBoost, and RandomForest on a large (over 100,000 samples), multi-class, imbalanced tabular task that combines concept shift and newly appearing classes; if TabPFN v2 then ranks first on balanced accuracy and F1 without truncating or ignoring test-time features, the paper's central claim that tree-based models remain optimal for general open-environment tasks would be contradicted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that TabPFN v2's closed-environment superiority does not transfer to open environments. In the new-class detection protocol, TabPFN v2 actually ranked first, showing a real ability to flag unseen classes. But in the feature-shift protocol its performance gap widened faster than most baselines as more features were removed, and it simply discards features that only appear at test time. Under changing distributions, it improved on covariate shift but remained below XGBoost under concept shift. Across four learning objectives it matched tree models on accuracy and ROC-AUC while dropping markedly on F1 and balanced accuracy, a pattern the paper reads as majority-class bias. The overall ranking table places TabPFN v2 behind CatBoost, XGBoost, and RandomForest for general open-environment use.

Load-bearing premise

The whole conclusion rests on the assumption that the selected benchmark datasets and protocols—four small datasets for new-class detection, TabFSBench's random feature-removal tasks, and nine distribution-shift datasets—adequately represent the diversity of real open environments.

Editorial extensions

If this is right

  • Practitioners facing tabular data that will change after deployment should default to CatBoost or XGBoost rather than TabPFN v2 for general tasks.
  • For small datasets with covariate shift and roughly balanced classes, TabPFN v2 is a defensible choice, especially when new classes may appear at test time.
  • Evaluation of tabular foundation models should report robustness metrics such as performance gap and balanced accuracy alongside closed-environment accuracy.
  • Models that cannot incorporate features added at test time, like TabPFN v2 under its fixed input dimension, are at a structural disadvantage in streaming or evolving data pipelines.
  • Benchmark design should cover all four open-environment challenges together, not distribution shift alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's recommendation is limited by its benchmark choices; an independent set of open-environment tabular tasks with different drift types could shift the ranking, so the general claim should be read as describing these protocols until replicated more broadly.
  • A testable extension is to feed TabPFN v2 feature-shift data drawn from its own pre-training distribution; if the sensitivity to feature removal comes from its fixed positional feature encoding, then adapting or retraining that encoding could close the gap to tree-based models.
  • The majority-class bias result implies that AUC-based model selection can hide imbalance failure, so for imbalanced open-environment tasks F1 or balanced accuracy should be the primary metric when comparing models.
  • Because TabPFN v2's design targets datasets smaller than roughly ten thousand rows with at most ten classes, the conclusion that it suits small-scale tasks is partly a consequence of the model's architecture rather than an entirely empirical discovery.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents an empirical evaluation of TabPFN v2 in four open-environment challenges: emerging new classes, decremental/incremental features, changing data distributions, and varied learning objectives. It introduces a unified evaluation framework built on existing benchmarks (SMOOD-style leave-one-class-out protocols, TabFSBench, WhyShift, TableShift), compares TabPFN v2 against tree-based and deep baselines, and concludes that TabPFN v2 is suitable for small-scale, covariate-shifted, and class-balanced tasks, while tree-based models remain preferable for general tabular tasks in open environments. The authors release an evaluation framework and report average ranks across the four challenges.

Significance. If the conclusions hold, the paper would provide a useful practical guide for practitioners choosing models in dynamic tabular settings, where TabPFN v2 is often marketed as a strong general-purpose model. The paper is among the first to evaluate TabPFN v2 across multiple open-environment challenge types rather than a single shift scenario, and it releases a reusable framework, which are concrete contributions. However, the central comparison in the large-scale distribution-shift setting is currently confounded by an unexplained mismatch between TabPFN v2's stated dataset-size limit and the evaluation protocol, and several specific claims in the text are stronger than the tables support. The paper's value is conditional on resolving these points.

major comments (4)
  1. [Appendix A.2 and Section 5.3 / Appendix G.1] The manuscript states in Appendix A.2 that TabPFN v2 has a fundamental constraint of 'dataset size limit < 10^4', yet Section 5.3 and Appendix G.1 describe stratified subsampling 'up to 50,000 instances' for the WhyShift/TableShift datasets, and Tables 9-10 list total sizes of about 50,000. No passage explains how TabPFN v2 was actually run at this scale. If the official implementation was used as-is, it could not process these training sets; if the code instead subsamples further for TabPFN v2 only, then the tree baselines saw much more training data. Either way, the observed 'significant limitations' in changing-data-distribution tasks, and the headline conclusion that tree-based models remain optimal, may be an artifact of unequal effective training-set sizes. This is a load-bearing step between the raw tables and the central claim, and it must be clarified and, if necessary, corrected.
  2. [Table 3 versus Table 2 (Section 5.5 and Section 5.2)] Table 3 reports RandomForest with average rank 1.78 on Decremental/Incremental Features, while Table 2 shows MLP consistently has the smallest performance gaps (e.g., binary classification at 20% shift: MLP gap -0.024 vs RandomForest -0.074; regression at 100% shift: MLP gap +0.250 vs RandomForest +1.421). The rank construction is not described precisely enough to reconcile this discrepancy, and the text of Section 5.2 explicitly identifies MLP and CatBoost as more robust against decremental features. As written, the holistic rank table contradicts the detailed results it is supposed to summarize, which undermines the 'tree-based models remain optimal' conclusion.
  3. [Section 5.1, Table 1] The claim that TabPFN v2 'consistently achieves better AUPR and ROC-AUC in the task of new class detection across all four datasets' is not supported by Table 1. On Wine-Red, MLP attains ROC-AUC 0.700 and AUPR 0.658 against TabPFN v2's 0.533 and 0.521, and on EyeMovement TabPFN v2's 0.511/0.507 is essentially tied with RealMLP's 0.510/0.505. The 'consistently' wording and the conclusion that TabPFN v2 has 'a robust capability for identifying new classes' should be qualified to reflect the dataset-dependent evidence.
  4. [Section 5.4 and Appendix D.1] The text twice uses 'statistically significant' ('statistically significant bias toward majority classes' and 'statistically significant performance deficiencies') without reporting any statistical test, confidence interval, or variance estimate. All results are averages over three random seeds with no error bars, so the word 'statistically significant' is unsupported. The authors should either provide appropriate significance testing or remove the term and describe the observed differences as descriptive.
minor comments (5)
  1. [Section 2.2] The citation for XGBoost in this section is listed as [11], but reference [11] is Chizat et al.; reference [9] is the XGBoost paper. The citation numbering appears to be off.
  2. [Figure 2] The text inside Figure 2 contains garbled sequences such as '/uni00000026/uni00000052/...' that appear to be an encoding artifact; the figure should be regenerated with clean labels.
  3. [Appendix D.1] The heading contains a typo: 'Traning settings' should be 'Training settings'.
  4. [Section 5, first paragraph] The phrase 'as detailed in Section D' refers to an appendix; use 'Appendix D' for consistency.
  5. [Abstract] The abstract says 'We publicly release our evaluation framework at the URL' but the full text replaces the URL with 'https://anonymous.4open.science/r/tabpfn-ood-4E65'; the actual URL should appear in the final version.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the evaluation is empirical against external benchmarks, with one minor self-citation (TabFSBench) that is not load-bearing.

full rationale

This paper is an empirical evaluation, not a derivation. The central claims—that TabPFN v2 underperforms under feature decrement, concept shift, and class-imbalanced objectives while doing well on small-scale, covariate-shift, and class-balanced tasks—are obtained by running established models on public datasets and reporting measured metrics. There is no fitted parameter that is later renamed as a prediction, and no equation defines a claimed output in terms of its own input. The four-challenge taxonomy is attributed to Zhou [61] and used as organizing framing, not as evidence for the empirical results. The only notable self-citation is TabFSBench [10], which is co-authored by the present authors and used as the feature-shift protocol in Section 5.2 and Appendix F; however, the performance gaps are computed from actual model evaluations under that protocol, so the conclusion is not forced by the citation itself. A separate concern that TabPFN v2's stated <10^4 sample limit conflicts with the 50,000-sample evaluation protocol is a potential validity confound, not a circularity, because it concerns unequal effective training-set sizes rather than an argument that reduces to its inputs. Overall, no step in the paper's reasoning is circular by the paper's own definitions.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No mathematical derivation is present; the paper is an empirical study, so the ledger records the experimental assumptions the conclusions depend on. No free parameters were fitted to produce the central claim, and no invented entities are introduced. Hand-chosen experimental settings such as the 50,000-instance subsampling cap and uncertainty intervals are design choices, not fitted parameters.

assumptions (4)
  • domain assumption Four challenges from Zhou (2022) (emerging new classes, decremental/incremental features, changing distributions, varied objectives) are a complete or representative taxonomy of open-environment tabular challenges.
    Introduction and Section 4 adopt this taxonomy as the evaluation scope; if other challenge types dominate real deployments, the conclusions do not transfer.
  • domain assumption Selected benchmarks (TableShift, WhyShift, TabFSBench, and four new-class datasets) are valid proxies for real open-environment scenarios.
    Sections 5.2 to 5.5 build all conclusions on these datasets; selection of only nine distribution-shift datasets and four new-class datasets is not justified by representativeness arguments.
  • domain assumption Leave-one-class-out with maximum softmax confidence thresholds measures a model's ability to detect new classes.
    Section E.2 defines detection via confidence intervals; this operationalizes 'detecting new classes' but is one of many possible definitions.
  • domain assumption Random feature removal at 20% to 100% levels approximates real decremental feature shifts.
    Section 5.2 and Appendix F use random-shift experiments from TabFSBench; real feature shifts may be structured rather than random.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Realistic Evaluation of TabPFN v2 in Open Environments." pith.science (2026). https://pith.science/paper/KNHWJMNG

@misc{pith2026250516226,
  author       = {Pith},
  title        = {Pith review of: Realistic Evaluation of TabPFN v2 in Open Environments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KNHWJMNG}},
  note         = {Machine review of arXiv:2505.16226}
}
read the original abstract

Tabular data, owing to its ubiquitous presence in real-world domains, has garnered significant attention in machine learning research. While tree-based models have long dominated tabular machine learning tasks, the recently proposed deep learning model TabPFN v2 has emerged, demonstrating unparalleled performance and scalability potential. Although extensive research has been conducted on TabPFN v2 to further improve performance, the majority of this research remains confined to closed environments, neglecting the challenges that frequently arise in open environments. This raises the question: Can TabPFN v2 maintain good performance in open environments? To this end, we conduct the first comprehensive evaluation of TabPFN v2's adaptability in open environments. We construct a unified evaluation framework covering various real-world challenges and assess the robustness of TabPFN v2 under open environments scenarios using this framework. Empirical results demonstrate that TabPFN v2 shows significant limitations in open environments but is suitable for small-scale, covariate-shifted, and class-balanced tasks. Tree-based models remain the optimal choice for general tabular tasks in open environments. To facilitate future research on open environments challenges, we advocate for open environments tabular benchmarks, multi-metric evaluation, and universal modules to strengthen model robustness. We publicly release our evaluation framework at https://anonymous.4open.science/r/tabpfn-ood-4E65.

Figures

Figures reproduced from arXiv: 2505.16226 by the authors.

Figure 1
Figure 1. Open environments challenges in tabular data learning, including emerging new classes, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Model performance comparison on accuracy between TabPFN v2 and XGBoost on changing data distributions task. 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 Accuracy ROC-AUC F1-Score Balanced-Accuracy Model performance on Four Learning Objectives of Classification Task CatBoost XGBoost RandomForest MLP RealMLP ModernNCA TabPFN v2 [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 4
Figure 4. Mean Balance Accuracy Across 9 Datasets with Distribution Shifts [PITH_FULL_IMAGE:figures/full_fig_p022_4.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Mean AUC Across 9 Datasets with Distribution Shift [PITH_FULL_IMAGE:figures/full_fig_p022_5.png]
Figure 6
Figure 6. Figure 6: Mean F1-score Across 9 Datasets with Distribution Shift [PITH_FULL_IMAGE:figures/full_fig_p024_6.png]
Figure 7
Figure 7. Figure 7: Mean Balance Accuracy Across 9 Datasets with Distribution Shifts [PITH_FULL_IMAGE:figures/full_fig_p024_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

63 extracted references · 46 canonical work pages

  1. [1]

    Optuna: A next-generation hyperparameter optimization framework

    Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2623–2631, 2019

    2019

  2. [2]

    Tabular data

    Naomi Altman and Martin Krzywinski. Tabular data. Nature Methods, 14(4):329–331, 2017

    work page 2017

  3. [3]

    Geometric dataset distances via optimal transport

    David Alvarez-Melis and Nicolo Fusi. Geometric dataset distances via optimal transport. Advances in Neural Information Processing Systems, pages 21428–21439, 2020

    work page 2020

  4. [4]

    Gradient boosting neural networks: Grownet

    Sarkhan Badirli, Xuanqing Liu, Zhengming Xing, Avradeep Bhowmik, Khoa Doan, and Sathiya Keerthi Keerthi. Gradient boosting neural networks: Grownet. arXiv preprint arXiv:2002.07971, 2020

    arXiv 2002

  5. [5]

    Deep neural networks and tabular data: A survey

    Vadim Borisov, Tobias Leemann, Kathrin Seßler, Johannes Haug, Martin Pawelczyk, and Gjergji Kasneci. Deep neural networks and tabular data: A survey. IEEE Transactions on Neural Networks and Learning Systems, 35(6):7499–7519, 2022

    work page 2022

  6. [6]

    Fine-tuned in-context learning transformers are excellent tabular data classifiers

    Felix den Breejen, Sangmin Bae, Stephen Cha, and Se-Young Yun. Fine-tuned in-context learning transformers are excellent tabular data classifiers. arXiv preprint arXiv:2405.13396, 2024

    arXiv 2024

  7. [7]

    Random forests

    Leo Breiman. Random forests. Machine learning, 45:5–32, 2001

    2001

  8. [8]

    Diagnosing model perfor- mance under distribution shift

    Tiffany Tianhui Cai, Hongseok Namkoong, and Steve Yadlowsky. Diagnosing model perfor- mance under distribution shift. arXiv preprint arXiv:2303.02011, 2023

    arXiv 2023

Show all 63 references
  1. [9]

    XGBoost: A Scalable Tree Boosting System

    Tianqi Chen and Carlos Guestrin. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 785–794, 2016

    2016

  2. [10]

    TabFSBench: Tabular Benchmark for Feature Shifts in Open Environment

    Zi-Jian Cheng, Zi-Yi Jia, Zhi Zhou, Lan-Zhe Guo, and Yu-Feng Li. TabFSBench: Tabular Benchmark for Feature Shifts in Open Environment. arXiv preprint arXiv:2501.18935, 2025

    2025 arXiv

  3. [11]

    Faster Wasserstein distance estimation with the Sinkhorn divergence

    Lenaic Chizat, Pierre Roussillon, Flavien Léger, François-Xavier Vialard, and Gabriel Peyré. Faster Wasserstein distance estimation with the Sinkhorn divergence. Advances in Neural Information Processing Systems, pages 2257–2269, 2020

    2020

  4. [12]

    Cortez, A

    P. Cortez, A. Cerdeira, F. Almeida, T. Matos, and J. Reis. Modeling wine preferences by data mining from physicochemical properties. Decision Support Systems, 47(4):547–553, 1998

    1998

  5. [13]

    Zero-shot generalizable incremental learning for vision-language object detection

    Jieren Deng, Haojian Zhang, Kun Ding, Jianhua Hu, Xingxuan Zhang, and Yunkuan Wang. Zero-shot generalizable incremental learning for vision-language object detection. Advances in Neural Information Processing Systems, pages 136679–136700, 2024

    2024

  6. [14]

    The overlooked elephant of object detection: Open set

    Akshay Dhamija, Manuel Gunther, Jonathan Ventura, and Terrance Boult. The overlooked elephant of object detection: Open set. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1021–1030, 2020

    2020

  7. [15]

    V os: Learning what you don’t know by virtual outlier synthesis

    Xuefeng Du, Zhaoning Wang, Mu Cai, and Yixuan Li. V os: Learning what you don’t know by virtual outlier synthesis. arXiv preprint arXiv:2202.01197, 2022

    2022 arXiv

  8. [16]

    Large language models on tabular data: Prediction, generation, and understanding-a survey

    Xi Fang, Weijie Xu, Fiona Anting Tan, Jiani Zhang, Ziqing Hu, Yanjun Jane Qi, Scott Nick- leach, Diego Socolinsky, Srinivasan Sengamedu, and Christos Faloutsos. Large language models on tabular data: Prediction, generation, and understanding-a survey. arXiv preprint arXiv:2402...

    2024 arXiv

  9. [17]

    J. Gama, I. Zliobaite, and A. Bifet. A survey on concept drift adaptation. ACM Computing Surveys (CSUR), 46(4):44, 2014

    2014

  10. [18]

    Benchmarking distribution shift in tabular data with tableshift

    Josh Gardner, Zoran Popovic, and Ludwig Schmidt. Benchmarking distribution shift in tabular data with tableshift. Advances in Neural Information Processing Systems, pages 53385–53432, 2023. 10

    2023

  11. [19]

    Revisiting deep learning models for tabular data

    Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Revisiting deep learning models for tabular data. Advances in Neural Information Processing Systems, pages 18932–18943, 2021

    2021

  12. [20]

    Why do tree-based models still outperform deep learning on typical tabular data? Advances in Neural Information Processing Systems, pages 507–520, 2022

    Léo Grinsztajn, Edouard Oyallon, and Gaël Varoquaux. Why do tree-based models still outperform deep learning on typical tabular data? Advances in Neural Information Processing Systems, pages 507–520, 2022

    2022

  13. [21]

    Advancing biogeographical ancestry predictions through machine learning

    Carola Sophia Heinzel, Lennart Purucker, Frank Hutter, and Peter Pfaffelhuber. Advancing biogeographical ancestry predictions through machine learning. bioRxiv, pages 1–3, 2025

    2025

  14. [22]

    Drift-resilient TabPFN: In-context learning temporal distribution shifts on tabular data

    Kai Helli, David Schnurr, Noah Hollmann, Samuel Müller, and Frank Hutter. Drift-resilient TabPFN: In-context learning temporal distribution shifts on tabular data. Advances in Neural Information Processing Systems, pages 98742–98781, 2024

    2024

  15. [23]

    A Baseline for Detecting Misclassified and Out-of- Distribution Examples in Neural Networks

    Dan Hendrycks and Kevin Gimpel. A Baseline for Detecting Misclassified and Out-of- Distribution Examples in Neural Networks. In Proceedings of the 5th International Conference on Learning Representations, 2017

    2017

  16. [24]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in Neural Information Processing Systems, pages 6629–6640, 2017

    2017

  17. [25]

    TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second

    Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter. TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second. In Proceedings of the 11th International Conference on Learning Representations, 2023

    2023

  18. [26]

    Accurate predictions on small data with a tabular foundation model

    Noah Hollmann, Samuel Müller, Lennart Purucker, Arjun Krishnakumar, Max Körfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model. Nature, 637:319–326, 2025

    2025

  19. [27]

    Better by default: Strong pre-tuned mlps and boosted trees on tabular data

    David Holzmüller, Léo Grinsztajn, and Ingo Steinwart. Better by default: Strong pre-tuned mlps and boosted trees on tabular data. Advances in Neural Information Processing Systems, pages 26577–26658, 2024

    2024

  20. [28]

    RealMLP: Advancing MLPs and default parameters for tabular data

    David Holzmüller, Leo Grinsztajn, and Ingo Steinwart. RealMLP: Advancing MLPs and default parameters for tabular data. In ELLIS workshop on Representation Learning and Generative Models for Structured Data, 2025

    2025

  21. [30]

    The Tabular Foundation Model TabPFN Outperforms Specialized Time Series Forecasting Models Based on Simple Features

    Shi Bin Hoo, Samuel Müller, David Salinas, and Frank Hutter. The Tabular Foundation Model TabPFN Outperforms Specialized Time Series Forecasting Models Based on Simple Features. arXiv preprint arXiv:2501.02945, 2025

    2025

  22. [31]

    Learning with feature evolvable streams

    Bo-Jian Hou, Lijun Zhang, and Zhi-Hua Zhou. Learning with feature evolvable streams. Advances in Neural Information Processing Systems, page 1416–1426, 2017

    2017

  23. [32]

    Well-tuned simple nets excel on tabular datasets

    Arlind Kadra, Marius Lindauer, Frank Hutter, and Josif Grabocka. Well-tuned simple nets excel on tabular datasets. Advances in Neural Information Processing Systems, pages 23928–23941, 2021

    2021

  24. [33]

    Towards Localization via Data Embedding for TabPFN

    Mykhailo Koshil, Thomas Nagler, Matthias Feurer, and Katharina Eggensperger. Towards Localization via Data Embedding for TabPFN. Advances in Neural Information Processing Systems Table Representation Learning Workshop, 2024

    2024

  25. [34]

    On the need for a language describing distribution shifts: Illustrations on tabular datasets

    Jiashuo Liu, Tianyu Wang, Peng Cui, and Hongseok Namkoong. On the need for a language describing distribution shifts: Illustrations on tabular datasets. Advances in Neural Information Processing Systems, pages 51371–51408, 2023

    2023

  26. [35]

    TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problems

    Si-Yang Liu and Han-Jia Ye. TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problems. arXiv preprint arXiv:2502.02527, 2025. 11

    2025 arXiv

  27. [36]

    Talent: A tabular analytics and learning toolbox

    Si-Yang Liu, Hao-Run Cai, Qi-Le Zhou, and Han-Jia Ye. Talent: A tabular analytics and learning toolbox. arXiv preprint arXiv:2407.04057, 2024

    2024 arXiv

  28. [37]

    Diving into self-evolving training for multimodal reasoning

    Wei Liu, Junlong Li, Xiwen Zhang, Fan Zhou, Yu Cheng, and Junxian He. Diving into self-evolving training for multimodal reasoning. arXiv preprint arXiv:2412.17451, 2024

    2024 arXiv

  29. [38]

    Opening up open world tracking

    Yang Liu, Idil Esen Zulfikar, Jonathon Luiten, Achal Dave, Deva Ramanan, Bastian Leibe, Aljoša Ošep, and Laura Leal-Taixé. Opening up open world tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19045–19055, 2022

    2022

  30. [39]

    When do neural nets outperform boosted trees on tabular data? Advances in Neural Information Processing Systems, pages 34–47, 2023

    Duncan McElfresh, Sujay Khandagale, Jonathan Valverde, Vishak Prasad C., Ganesh Ramakr- ishnan, Micah Goldblum, and Colin White. When do neural nets outperform boosted trees on tabular data? Advances in Neural Information Processing Systems, pages 34–47, 2023

    2023

  31. [40]

    Uncertainty estimation for classification and risk prediction on medical tabular data

    Lotta Meijerink, Giovanni Cinà, and Michele Tonutti. Uncertainty estimation for classification and risk prediction on medical tabular data. arXiv preprint arXiv:2004.05824, 2020

    2004 arXiv

  32. [41]

    Machine learning-based diagnostic prediction of minimal change disease: model development study

    Ryunosuke Noda, Daisuke Ichikawa, and Yugo Shibagaki. Machine learning-based diagnostic prediction of minimal change disease: model development study. Scientific Reports, 14(1): 23460, 2024

    2024

  33. [42]

    Open-world machine learning: applications, challenges, and opportunities

    Jitendra Parmar, Satyendra Chouhan, Vaskar Raychoudhury, and Santosh Rathore. Open-world machine learning: applications, challenges, and opportunities. ACM Computing Surveys, 55 (10):1–37, 2023

    2023

  34. [43]

    Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data

    Sergei Popov, Stanislav Morozov, and Artem Babenko. Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data. In Proceedings of the 8th International Conference on Learning Representations, 2020

    2020

  35. [44]

    CatBoost: unbiased boosting with categorical features

    Liudmila Prokhorenkova, Gleb Gusev, Aleksandr V orobev, Anna Veronika Dorogush, and Andrey Gulin. CatBoost: unbiased boosting with categorical features. Advances in Neural Information Processing Systems, pages 6639–6649, 2018

    2018

  36. [45]

    A TabPFN-based intrusion detection system for the industrial internet of things

    Sergio Ruiz-Villafranca, José Roldán-Gómez, Juan Manuel Castelo Gómez, Javier Carrillo- Mondéjar, and José Luis Martinez. A TabPFN-based intrusion detection system for the industrial internet of things. The Journal of Supercomputing, 80(14):20080–20117, 2024

    2024

  37. [46]

    Explainable artificial intelligence for tabular data: A survey

    Maria Sahakyan, Zeyar Aung, and Talal Rahwan. Explainable artificial intelligence for tabular data: A survey. IEEE access, 9:135392–135422, 2021

    2021

  38. [47]

    Curious exploration via structured world models yields zero-shot object manipulation

    Cansu Sancaktar, Sebastian Blaes, and Georg Martius. Curious exploration via structured world models yields zero-shot object manipulation. Advances in Neural Information Processing Systems, pages 24170–24183, 2022

    2022

  39. [48]

    Tabular data: Deep learning is not all you need

    Ravid Shwartz-Ziv and Amitai Armon. Tabular data: Deep learning is not all you need. Information Fusion, 81:84–90, 2022

    2022

  40. [49]

    Covariate shift adaptation by importance weighted cross validation

    Masashi Sugiyama, Matthias Krauledat, and Klaus-Robert Müller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8(5), 2007

    2007

  41. [50]

    Caterini

    Valentin Thomas, Junwei Ma, Rasa Hosseinzadeh, Keyvan Golestan, Guangwei Yu, Maksims V olkovs, and Anthony L. Caterini. Retrieval & Fine-Tuning for In-Context Tabular Models. Advances in Neural Information Processing Systems, pages 108439–108467, 2024

    2024

  42. [51]

    Predicting dementia in parkinson’s disease on a small tabular dataset using hybrid lightgbm–tabpfn and shap

    Vinh Quang Tran and Haewon Byeon. Predicting dementia in parkinson’s disease on a small tabular dataset using hybrid lightgbm–tabpfn and shap. Digital Health, 10:20–55, 2024

    2024

  43. [52]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, pages 6000–6010, 2017

    2017

  44. [53]

    Dcn v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems

    Ruoxi Wang, Rakesh Shivanna, Derek Cheng, Sagar Jain, Dong Lin, Lichan Hong, and Ed Chi. Dcn v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems. In Proceedings of the Web Conference 2021, pages 1785–1797, 2021. 12

    2021

  45. [54]

    Neural network credit scoring models

    David West. Neural network credit scoring models. Computers & operations research, 27(11): 1131–1152, 2000

    2000

  46. [55]

    Mixture of In-Context Prompters for Tabular PFNs

    Derek Qiang Xu, F Olcay Cirit, Reza Asadi, Yizhou Sun, and Wei Wang. Mixture of In-Context Prompters for Tabular PFNs. In Proceedings of the 13th International Conference on Learning Representations, 2025

    2025

  47. [56]

    Modern Neighborhood Components Analysis: A Deep Tabular Baseline Two Decades Later

    Han-Jia Ye, Huai-Hong Yin, and De-Chuan Zhan. Modern Neighborhood Components Analysis: A Deep Tabular Baseline Two Decades Later. arXiv preprint arXiv:2407.03257, 2024

    2024 arXiv

  48. [57]

    A Closer Look at TabPFN v2: Strength, Limitation, and Extension

    Han-Jia Ye, Si-Yang Liu, and Wei-Lun Chao. A Closer Look at TabPFN v2: Strength, Limitation, and Extension. arXiv preprint arXiv:2502.17361, 2025

    2025 arXiv

  49. [58]

    Gradient boosting decision trees on medical diagnosis over tabular data

    A Yarkın Yıldız and Asli Kalayci. Gradient boosting decision trees on medical diagnosis over tabular data. arXiv preprint arXiv:2410.03705, 2024

    2024 arXiv

  50. [59]

    Domain Generalization with MixStyle

    Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. Domain Generalization with MixStyle. In Proceedings of the 9th International Conference on Learning Representations, 2021

    2021

  51. [60]

    Fully Test-time Adaptation for Tabular Data

    Zhi Zhou, Kun-Yang Yu, Lan-Zhe Guo, and Yu-Feng Li. Fully Test-time Adaptation for Tabular Data. In Proceedings of the 39th AAAI conference on Artificial Intelligence, 2025

    2025

  52. [61]

    Open-environment machine learning

    Zhi-Hua Zhou. Open-environment machine learning. National Science Review, 9(8):nwac123, 2022

    2022

  53. [62]

    Evolutionary learning: Advances in theories and algorithms

    Zhi-Hua Zhou, Yang Yu, and Chao Qian. Evolutionary learning: Advances in theories and algorithms. Springer, 2019

    2019

  54. [63]

    TAT-QA: A question answering benchmark on a hybrid of tabular and textual content in finance

    Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. TAT-QA: A question answering benchmark on a hybrid of tabular and textual content in finance. arXiv preprint arXiv:2105.07624, 2021

    2021 arXiv

  55. [64]

    How to Add New Models

    Marcela Zuluaga, Guillaume Sergent, Andreas Krause, and Markus Püschel. Active learning for multi-objective optimization. In Proceedings of the 29th International Conference on Machine Learning, pages 462–470, 2013. 13 A TabPFN and TabPFN v2 A.1 TabPFN Developed by [ 25], TabP...

    2013

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.