pith. sign in

arxiv: 2604.16123 · v2 · submitted 2026-04-17 · 💻 cs.LG · physics.chem-ph

Tabular foundation models for in-context prediction of molecular properties

Karim K. Ben Hicham , Jan G. Rittig , Martin Grohe , Alexander Mitsos This is my paper

Pith reviewed 2026-05-10 08:39 UTC · model grok-4.3

classification 💻 cs.LG physics.chem-ph
keywords molecularfoundationlearningtfmsin-contextmodelspredictionaccurate
0
0 comments X

The pith

Tabular foundation models achieve high accuracy in molecular property prediction through in-context learning, with up to 100% win rates on MoleculeACE tasks when paired with CheMeleon embeddings.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Predicting properties of molecules is important for making new medicines and chemicals. Traditional machine learning needs training a new model for each property, which is slow and needs lots of data. This work instead uses tabular foundation models that look at a few examples right in the input and make predictions without extra training. They test this on standard drug discovery benchmarks and real chemical engineering data. Different ways to describe molecules are tried, including special embeddings from other models and simple lists of chemical features. The results show good accuracy with much less computing power than fine-tuning approaches. The choice of how molecules are represented turns out to matter a lot for how well the tabular models work.

Core claim

combining TFMs with CheMeleon embeddings yields up to 100% win rates on 30 MoleculeACE tasks, while compact RDKit2d and Mordred descriptors provide strong descriptor-based alternatives.

Load-bearing premise

That in-context learning performance with TFMs generalizes reliably to practical low- to medium-data engineering settings and that molecular representation choice drives the gains without hidden task-specific effects or benchmark overfitting.

read the original abstract

Accurate molecular property prediction is central to drug discovery, catalysis, and process design, yet real-world applications are often limited by small datasets. Molecular foundation models provide a promising direction by learning transferable molecular representations; however, they typically involve task-specific fine-tuning, require machine learning expertise, and often fail to outperform classical baselines. Tabular foundation models (TFMs) offer a fundamentally different paradigm: they perform predictions through in-context learning, enabling inference without task-specific training. Here, we evaluate TFMs in the low- to medium-data regime across both standardized pharmaceutical benchmarks and chemical engineering datasets. We evaluate both frozen molecular foundation model representations, as well as classical descriptors and fingerprints. Across the benchmarks, the approach shows excellent predictive performance while reducing computational cost, compared to fine-tuning, with these advantages also transferring to practical engineering data settings. In particular, combining TFMs with CheMeleon embeddings yields up to 100\% win rates on 30 MoleculeACE tasks, while compact RDKit2d and Mordred descriptors provide strong descriptor-based alternatives. Molecular representation emerges as a key determinant in TFM performance, with molecular foundation model embeddings and 2D descriptor sets both providing substantial gains over classic molecular fingerprints on many tasks. These results suggest that in-context learning with TFMs provides a highly accurate and cost-efficient alternative for property prediction in practical applications.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No free parameters, axioms, or invented entities are described; the work is an empirical comparison of existing models and representations.

pith-pipeline@v0.9.0 · 5551 in / 1033 out tokens · 22645 ms · 2026-05-10T08:39:08.464633+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. TabPFN-3: Technical Report

    cs.LG 2026-05 unverdicted novelty 6.0

    TabPFN-3 delivers state-of-the-art tabular prediction performance on benchmarks up to 1M rows, is up to 20x faster than prior versions, and introduces test-time scaling that beats non-TabPFN models by hundreds of Elo points.

Reference graph

Works this paper leans on

48 extracted references · 48 canonical work pages · cited by 1 Pith paper · 4 internal anchors

  1. [1]

    Yu, M.et al.Deep learning large-scale drug discovery and repurposing.Nature Computational Science4, 600–614 (2024)

    work page 2024

  2. [2]

    A., Khizbullin, D., Nagaraja, S

    Eraqi, B. A., Khizbullin, D., Nagaraja, S. S. & Sarathy, S. M. Molecular prop- erty prediction in the ultra-low data regime.Communications Chemistry8, 201 (2025)

    work page 2025

  3. [3]

    G., Dahmen, M., Grohe, M., Schwaller, P

    Rittig, J. G., Dahmen, M., Grohe, M., Schwaller, P. & Mitsos, A. Molecu- lar machine learning in chemical process design.Current Opinion in Chemical Engineering52, 101239 (2026)

    work page 2026

  4. [4]

    Altae-Tran, H., Ramsundar, B., Pappu, A. S. & Pande, V. Low Data Drug Discovery with One-Shot Learning.ACS Central Science3, 283–293 (2017)

    work page 2017

  5. [5]

    V., van Deursen, R

    Tetko, I. V., van Deursen, R. & Godin, G. Be aware of overfitting by hyperparameter optimization!Journal of Cheminformatics16, 139 (2024)

    work page 2024

  6. [6]

    & Czech, W

    Praski, M., Adamczyk, J. & Czech, W. Benchmarking Pretrained Molec- ular Embedding Models For Molecular Representation Learning (2025). arXiv:2508.06199

    work page arXiv 2025

  7. [7]

    Ross, J.et al.Large-scale chemical language representations capture molecular structure and properties.Nature Machine Intelligence4, 1256–1264 (2022)

    work page 2022

  8. [8]

    & Farimani, A

    Wang, Y., Wang, J., Cao, Z. & Farimani, A. B. Molecular Contrastive Learning of Representations via Graph Neural Networks.Nature Machine Intelligence4, 279–287 (2022)

    work page 2022

  9. [9]

    Descriptor-Based

    Burns, J. W.et al.Deep Learning Foundation Models from Classical Molecular Descriptors (2026). arXiv:2506.15792

    work page arXiv 2026

  10. [10]

    Fine- tuning can distort pretrained features and underperform out-of-distribution.arXiv preprint arXiv:2202.10054, 2022

    Kumar, A., Raghunathan, A., Jones, R., Ma, T. & Liang, P. Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution (2022). arXiv:2202.10054

    work page arXiv 2022

  11. [11]

    & Varoquaux, G

    Grinsztajn, L., Oyallon, E. & Varoquaux, G. Why do tree-based models still out- perform deep learning on typical tabular data?Advances in Neural Information Processing Systems35, 507–520 (2022)

    work page 2022

  12. [12]

    & Grisoni, F

    van Tilborg, D., Alenicheva, A. & Grisoni, F. Exposing the Limitations of Molec- ular Machine Learning with Activity Cliffs.Journal of Chemical Information and Modeling62, 5938–5951 (2022)

    work page 2022

  13. [13]

    Hollmann, N.et al.Accurate predictions on small data with a tabular foundation model.Nature637, 319–326 (2025). 17

    work page 2025

  14. [14]

    TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second

    Hollmann, N., M¨ uller, S., Eggensperger, K. & Hutter, F. TabPFN: A Trans- former That Solves Small Tabular Classification Problems in a Second (2023). arXiv:2207.01848

    work page internal anchor Pith review arXiv 2023

  15. [15]

    TabICLv2: A better, faster, scalable, and open tabular foundation model.arXiv preprint arXiv:2602.11139,

    Qu, J., Holzm¨ uller, D., Varoquaux, G. & Morvan, M. L. TabICLv2: A better, faster, scalable, and open tabular foundation model (2026). arXiv:2602.11139

    work page arXiv 2026

  16. [16]

    TabICL: A Tabular Foundation Model for In-Context Learning on Large Data

    Qu, J., Holzm¨ uller, D., Varoquaux, G. & Morvan, M. L. TabICL: A Tabular Foun- dation Model for In-Context Learning on Large Data (2025). arXiv:2502.05564

    work page internal anchor Pith review arXiv 2025

  17. [17]

    TabArena: A Living Benchmark for Machine Learning on Tabular Data

    Erickson, N.et al.TabArena: A Living Benchmark for Machine Learning on Tabular Data (2025). arXiv:2506.16791

    work page Pith review arXiv 2025

  18. [18]

    Grinsztajn, L.et al.Tabpfn-2.5: Advancing the state of the art in tabular foundation models.arXiv preprint arXiv:2511.08667(2025)

    work page internal anchor Pith review arXiv 2025

  19. [19]

    Chen, W.et al.TabPFN Opens New Avenues for Small-Data Tabular Learning in Drug Discovery.Journal of Chemical Information and Modelingacs.jcim.5c02823 (2026)

    work page 2026

  20. [20]

    Scalfani, Daniel Probst, Kazuya Ujihara, Yakov Pechersky, Jeremy Monat, and Juuso Lehtivarjo

    Landrum, G., Tosco, P., Kelley, B.et al.Rdkit: Open-source cheminformatics (2026). URL https://doi.org/10.5281/zenodo.18797641

    work page doi:10.5281/zenodo.18797641 2026

  21. [21]

    & Takagi, T

    Moriwaki, H., Tian, Y.-S., Kawashita, N. & Takagi, T. Mordred: A molecular descriptor calculator.Journal of Cheminformatics10, 4 (2018)

    work page 2018

  22. [22]

    Soares, E.et al.An open-source family of large encoder-decoder foundation models for chemistry.Communications Chemistry8, 193 (2025)

    work page 2025

  23. [23]

    & Klambauer, G

    Seidl, P., Vall, A., Hochreiter, S. & Klambauer, G. Enhancing Activity Predic- tion Models in Drug Discovery with the Ability to Understand Human Language (2023). arXiv:2303.03363

    work page arXiv 2023

  24. [24]

    Heid, E.et al.Chemprop: a machine learning package for chemical property prediction.Journal of chemical information and modeling64, 9–17 (2023)

    work page 2023

  25. [25]

    $\texttt{MiniMol}$: A Parameter- Efficient Foundation Model for Molecular Learning, April 2024

    Kl¨ aser, K.et al.MiniMol: A Parameter-Efficient Foundation Model for Molecular Learning (2024). arXiv:2404.14986

    work page arXiv 2024

  26. [26]

    P., Wang, W

    Sheridan, R. P., Wang, W. M., Liaw, A., Ma, J. & Gifford, E. M. Extreme Gra- dient Boosting as a Method for Quantitative Structure–Activity Relationships. Journal of Chemical Information and Modeling56, 2353–2360 (2016)

    work page 2016

  27. [27]

    Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A. V. & Gulin, A. Catboost: unbiased boosting with categorical features.Advances in neural information processing systems31(2018). 18

    work page 2018

  28. [28]

    R.et al.Practically Significant Method Comparison Protocols for Machine Learning in Small Molecule Drug Discovery.Journal of Chemical Information and Modeling65, 9398–9411 (2025)

    Ash, J. R.et al.Practically Significant Method Comparison Protocols for Machine Learning in Small Molecule Drug Discovery.Journal of Chemical Information and Modeling65, 9398–9411 (2025)

    work page 2025

  29. [29]

    Therapeutics data commons: Machine learning datasets and tasks for drug discovery and development.arXiv preprint arXiv:2102.09548, 2021

    Huang, K.et al.Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development (2021). arXiv:2102.09548

    work page arXiv 2021

  30. [30]

    M.et al.Graph neural networks for prediction of fuel ignition quality.Energy & fuels34, 11395–11407 (2020)

    Schweidtmann, A. M.et al.Graph neural networks for prediction of fuel ignition quality.Energy & fuels34, 11395–11407 (2020)

    work page 2020

  31. [31]

    & Rittig, J

    bin Javaid, M., Gervens, T., Mitsos, A., Grohe, M. & Rittig, J. G. Exploring data augmentation: Multi-task methods for molecular property prediction.Computers & Chemical Engineering201, 109253 (2025)

    work page 2025

  32. [32]

    & Ramprasad, R

    Kuenneth, C. & Ramprasad, R. polybert: a chemical language model to enable fully machine-driven ultrafast polymer informatics.Nature communications14, 4099 (2023)

    work page 2023

  33. [33]

    & Li, Y.-P

    Liao, H.-C., Lin, Y.-H., Peng, C.-H. & Li, Y.-P. Directed message passing neural networks for accurate prediction of polymer–solvent interaction parameters.ACS Engineering Au5, 530–539 (2025)

    work page 2025

  34. [34]

    Superior molecular representations from intermediate encoder layers

    Pinto, L. Superior molecular representations from intermediate encoder layers. arXiv preprint arXiv:2506.06443(2025)

    work page arXiv 2025

  35. [35]

    Wognum, C.et al.A call for an industry-led initiative to critically assess machine learning for real-world drug discovery.Nature Machine Intelligence6, 1120–1121 (2024)

    work page 2024

  36. [36]

    Zhou, J., Yang, Y., Mroz, A. M. & Jelfs, K. E. Polycl: contrastive learning for polymer representation learning via explicit and implicit augmentations.Digital Discovery4, 149–160 (2025)

    work page 2025

  37. [37]

    & Barati Farimani, A

    Xu, C., Wang, Y. & Barati Farimani, A. Transpolymer: a transformer-based language model for polymer property predictions.npj Computational Materials 9, 64 (2023)

    work page 2023

  38. [38]

    Erickson, N.et al.Autogluon-tabular: Robust and accurate automl for structured data.arXiv preprint arXiv:2003.06505(2020)

    work page internal anchor Pith review arXiv 2003

  39. [39]

    Sinodinos, D.et al.Multitask-informed prior for in-context learning on tabular data: Application to steel property prediction.arXiv preprint arXiv:2603.22738 (2026)

    work page arXiv 2026

  40. [40]

    Sinodinos, D., Wei, J. Y. & Armanfard, N. Multitab: A scalable foundation for multitask learning on tabular data.Proceedings of the AAAI Conference on Artificial Intelligence(2026). 19

    work page 2026

  41. [41]

    T.-Y., Picard, C

    Yu, R. T.-Y., Picard, C. & Ahmed, F. GIT-BO: High-Dimensional Bayesian Optimization with Tabular Foundation Models (2026). arXiv:2505.20685

    work page arXiv 2026

  42. [42]

    Transformers can do bayesian inference.arXiv preprint arXiv:2112.10510,

    M¨ uller, S., Hollmann, N., Arango, S. P., Grabocka, J. & Hutter, F. Transformers Can Do Bayesian Inference (2024). arXiv:2112.10510

    work page arXiv 2024

  43. [43]

    Garg, A.et al.Real-tabpfn: Improving tabular foundation models via continued pre-training with real-world data.arXiv preprint arXiv:2507.03971(2025)

    work page arXiv 2025

  44. [44]

    TabPFN (2026)

    PriorLabs. TabPFN (2026). URL https://github.com/PriorLabs/TabPFN. GitHub repository, accessed 2026-04-09

    work page 2026

  45. [45]

    Mark, J. E. & Mark, J. E.Physical properties of polymers handbookVol. 1076 (Springer, 2007)

    work page 2007

  46. [46]

    K., Rittig, J

    Ben Hicham, K. K., Rittig, J. G., Grohe, M. & Mitsos, A. Tabular foundation models for in-context prediction of molecular properties (2026). URL https:// doi.org/10.5281/zenodo.19605631

    work page doi:10.5281/zenodo.19605631 2026

  47. [47]

    Medina, E. I. S., Linke, S., Stoll, M. & Sundmacher, K. Gibbs–helmholtz graph neural network: capturing the temperature dependency of activity coefficients at infinite dilution.Digital Discovery2, 781–798 (2023)

    work page 2023

  48. [48]

    catalaix

    Brouwer, T., Kersten, S. R., Bargeman, G. & Schuur, B. Trends in solvent impact on infinite dilution activity coefficients of solutes reviewed and visualized using an algorithm to support selection of solvents for greener fluid separations.Separation and Purification Technology272, 118727 (2021). Author Contributions Karim K. Ben Hicham: Conceptualization...

    work page 2021