Pith. sign in

REVIEW 4 major objections 7 minor 49 references

TabLoRA builds neural ensembles for large tables by sharing one backbone and adding only low-rank per-predictor updates.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 00:34 UTC pith:INP644GE

load-bearing objection Solid practical packaging of LoRA into a from-scratch tabular ensemble; the efficiency story holds, the “best rank” claim is thinner than the abstract suggests. the 4 major comments →

arxiv 2607.10077 v1 pith:INP644GE submitted 2026-07-11 cs.LG

TabLoRA: Parameter-Efficient Low-Rank Ensemble Learning for Large-Scale Tabular Data

Jiaqi Luo , Shixin Xu This is my paper
classification cs.LG
keywords large-scale tabular datadeep learninglow-rank adaptationensembleparameter-efficientMLPGPU memory
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Large tabular datasets still push deep models toward high memory cost or weak expressivity, while full neural ensembles scale poorly because every member duplicates the whole network. TabLoRA keeps a single shared multilayer backbone and equips each ensemble member with its own low-rank weight updates plus a cheap feature adapter that produces slightly different inputs. The members are trained jointly from scratch and averaged at inference, so the model behaves like an ensemble without linear growth in parameters. On sixteen large-scale classification and regression benchmarks it records the best average rank among nine methods, never runs out of GPU memory under a fixed budget, and stays competitive with gradient-boosted trees and recent deep tabular baselines. Ablations show that the feature adapters add diversity and that the low-rank design recovers nearly the accuracy of a full independent ensemble at a fraction of the parameter cost. The result is a practical route to trainable neural ensembles when sample size, feature dimension, or class count become large.

Core claim

A shared multilayer backbone plus predictor-specific low-rank adaptations and lightweight feature adapters yields an ensemble that matches most of the accuracy of a full deep ensemble while using far fewer parameters and remaining trainable on large-scale tabular data under fixed GPU memory.

What carries the argument

TabLoRA: each linear layer computes a shared transform plus a rank-r residual (alpha/r) B_k A_k x_k for every predictor k, preceded by a low-rank feature adapter that produces K distinct input views; all members are trained by average loss and averaged at inference.

Load-bearing premise

That a fixed ensemble of 32 members with low ranks chosen by only ten short hyper-parameter trials is enough to capture the diversity and ranking gains the method claims.

What would settle it

Re-run the same sixteen-dataset protocol with a full independent ensemble of identical depth and width, or with substantially larger Optuna budgets and varied K and r; if TabLoRA then loses its rank advantage or its accuracy gap to the full ensemble widens sharply, the central efficiency claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes TabLoRA, a trainable neural ensemble for large-scale tabular learning that shares an MLP backbone across K predictors and adds predictor-specific low-rank (LoRA-style) weight adaptations plus lightweight group feature adapters. The design is trained jointly from scratch (not as PEFT of a frozen backbone) and aggregates predictor outputs by averaging. Complexity analysis (§3.6) shows that ensemble-specific parameter growth scales with rank r rather than full backbone duplication. On 16 large-scale OpenML datasets, under a shared 24 GB GPU / 10-hour budget, TabLoRA reports the best mean rank (3.3) among nine methods, no OOM failures, competitive accuracy/AUC/MSE, and lower average GPU memory multiples than several neural baselines; ablations isolate the feature adapter and the low-rank ensemble parameterization, and a fixed-configuration scaling study quantifies parameter savings versus a full ensemble.

Significance. If the empirical ranking and memory claims hold under more thorough search, the work is a useful systems-level contribution: it makes neural ensembles more feasible on high-dimensional or large-sample tabular data without requiring full backbone replication, and it documents practical OOM behavior of strong baselines (TabPFN, TabM, NCART, some GBDTs). The complexity derivation in §3.6 is clean and falsifiable, the ablations (Figs. 6–8, Table C.5) separate adapter diversity from low-rank specialization, and the protocol reports means±std over five seeds with explicit OOM handling. Novelty is incremental relative to BatchEnsemble/TabM and LoRA, but the combination—trainable low-rank ensembles from scratch plus feature adapters for large-scale tabular data—is a coherent and practically motivated design point.

major comments (4)
  1. The headline ranking (Table 1 mean rank 3.3; Fig. 2) is obtained under a thin HPO budget of only 10 Optuna trials per method (§4.3, Table B.3). For TabLoRA the search still includes depth, width, dropout, and rank, while K is fixed at 32; competing methods (RealMLP, TabM, GBDTs) have comparably large or larger spaces. With so few trials, the reported ordering can be an artifact of under-tuned baselines rather than a stable performance–efficiency advantage. A load-bearing revision is either (i) a substantially larger shared trial budget, or (ii) a controlled re-ranking with fixed architectures / matched budgets that isolates the architectural effect from HPO luck.
  2. Ensemble size K is fixed to 32 for TabLoRA (and TabM) with no sensitivity study (Table B.3; Eqs. (4)–(5)). The central claim that low-rank adaptations preserve “much of the benefit of full ensembles” (§5.3.2, Fig. 7) therefore rests on a single, untested K. Because diversity and capacity both depend on K and r, the paper should report performance and memory as functions of K (and of r at fixed K), including at least one comparison against Full at matched total parameter or matched wall-clock budgets—not only after independent HPO.
  3. Mean-rank aggregation in Table 1 / Fig. 2 does not state how OOM runs are scored. TabPFN has 6 OOM, TabM and NCART 5 each; if OOM is assigned the worst rank, TabLoRA’s average rank is partly driven by competitors’ failures rather than by superior accuracy on completed runs. Please define the ranking rule explicitly, and report a secondary ranking restricted to datasets where all methods finish (or pairwise win rates on completed runs) so the predictive claim is separable from the feasibility claim.
  4. The Full vs TabLoRA ablation (Fig. 7, Table C.5) inherits the same 10-trial HPO and does not control for total parameters or training cost. Full can select different depths/widths than TabLoRA, so equal average rank (1.5) does not by itself show that low-rank parameterization preserves full-ensemble accuracy at reduced cost. A matched-capacity or matched-FLOP comparison on the same backbone depth/width would make this claim load-bearing rather than suggestive.
minor comments (7)
  1. §5.2 refers to “Fig. reff.bubble and Fig. reff.gpu”; these appear to be broken LaTeX references to Figs. 4–5.
  2. Appendix A Table A.2: “CIFAE-100” should be “CIFAR-100”; #Target for regression is described inconsistently as unique values vs continuous targets.
  3. Eq. (5) text: “relu activative function” → “ReLU activation function.”
  4. Scaling factor α appears in Eqs. (4)–(5) but is not listed in the hyperparameter table B.3; state whether it is fixed (and to what value) or tuned.
  5. Related work discusses BatchEnsemble via TabM but does not clearly contrast TabLoRA’s low-rank residual with BatchEnsemble’s rank-1 multiplicative factors; a short paragraph would help position the contribution.
  6. Fig. 3 / relative-improvement metric (Eq. 15): for near-perfect MLP baselines the classification form can amplify tiny absolute gains; consider also reporting absolute metrics in the main plots.
  7. Declaration of generative AI use is present; ensure journal policy compliance on wording and placement.

Circularity Check

0 steps flagged

No significant circularity: empirical rankings, parameter counts, and ablations are self-contained against external OpenML baselines and standard LoRA/adapter constructions.

full rationale

TabLoRA’s load-bearing claims are (i) mean-rank superiority and OOM-free feasibility on 16 public large-scale OpenML datasets under a uniform 10-trial Optuna protocol (Table 1, Figs. 2–5) and (ii) a transparent parameter-complexity reduction obtained by counting a shared backbone plus K low-rank factors (Eqs. 11–14). Both are externally falsifiable comparisons; neither quantity is defined in terms of the other. The relative-improvement metric (Eq. 15) is ordinary gap reduction versus MLP and does not force rankings by construction. Feature-adapter and Single/Full/TabLoRA ablations (§5.3) are likewise empirical under the same HPO budget. Citations to LoRA [46] and adapters [45] supply the known low-rank update form; the authors’ own NCART [11] appears only as one baseline among many and is not invoked as a uniqueness theorem or ansatz justification. No fitted parameter is later re-labeled a prediction, no self-definitional loop exists among the equations, and no renaming of a known empirical pattern is presented as a derivation. The paper is therefore free of the six circularity patterns.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 1 invented entities

The paper is an empirical methods contribution. Its load-bearing content rests on standard neural-network training assumptions, the low-rank hypothesis borrowed from LoRA, and a modest set of free hyper-parameters fixed by light search. No new physical entities are postulated.

free parameters (4)
  • ensemble size K = 32
    Fixed at 32 for all TabLoRA runs; controls diversity–cost trade-off and is not derived.
  • LoRA rank r = UniformInt[4,16] step 2
    Searched in {4,6,...,16}; directly determines parameter count and expressivity of each predictor.
  • hidden dimension and depth
    d_hidden ∈ [64,512], n_blocks ∈ [1,9]; chosen by Optuna and affect both capacity and memory.
  • scaling factor α
    Appears in the low-rank update (Eq. 4); treated as a free hyper-parameter of the LoRA formulation.
axioms (3)
  • domain assumption Low-rank updates of modest rank are sufficient to specialize ensemble members while a shared backbone captures common structure.
    Imported from LoRA literature and used as the central modeling hypothesis in §3.3 without independent theoretical proof for tabular data.
  • domain assumption Averaging logits (classification) or outputs (regression) of the K predictors is a valid ensemble aggregation.
    Standard ensemble practice stated in §3.5; not re-derived.
  • ad hoc to paper 10 Optuna trials and 5 random seeds suffice to rank methods fairly under a fixed 10-hour / 24 GB GPU budget.
    Experimental protocol of §4.3; the ranking claim depends on this budget being representative.
invented entities (1)
  • TabLoRA architecture (shared MLP backbone + per-predictor LoRA + group feature adapters) no independent evidence
    purpose: Realize parameter-efficient neural ensembles for large-scale tabular prediction.
    The specific combination and joint from-scratch training regime are introduced by the paper; independent evidence is limited to the reported benchmarks.

pith-pipeline@v1.1.0-grok45 · 23129 in / 2646 out tokens · 26457 ms · 2026-07-14T00:34:57.268333+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of TabLoRA: Parameter-Efficient Low-Rank Ensemble Learning for Large-Scale Tabular Data." pith.science (2026). https://pith.science/paper/INP644GE

@misc{pith2026260710077,
  author       = {Pith},
  title        = {Pith review of: TabLoRA: Parameter-Efficient Low-Rank Ensemble Learning for Large-Scale Tabular Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/INP644GE}},
  note         = {Machine review of arXiv:2607.10077}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Tabular learning is still dominated by gradient-boosted decision trees (GBDTs), while recent deep learning approaches have become increasingly competitive. However, applying deep tabular models to large-scale datasets remains challenging, as large sample sizes, high feature dimensionality, or many target classes can introduce substantial computational cost. We propose TabLoRA, a parameter-efficient trainable neural ensemble for large-scale tabular learning. Instead of using fully independent ensemble backbones, TabLoRA shares a common backbone across predictors and introduces predictor-specific low-rank adaptations, enabling ensemble-style prediction without full parameter duplication. Across benchmarks, TabLoRA achieves a favorable balance between predictive performance and practical efficiency compared with GBDT methods and recent deep learning baselines under the same resource constraints. Memory analysis and ablation studies further show that the proposed design improves the feasibility of neural ensemble learning while preserving much of the benefit of full ensembles.

Figures

Figures reproduced from arXiv: 2607.10077 by Jiaqi Luo, Shixin Xu.

Figure 1
Figure 1. Figure 1: Overview of TabLoRA. The model combines predictor-specific feature adapta [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Average performance ranks on large-scale datasets. Lower rank indicates better [PITH_FULL_IMAGE:figures/full_fig_p020_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Dataset-level relative improvement over MLP on large-scale datasets. Each [PITH_FULL_IMAGE:figures/full_fig_p021_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Performance–memory trade-off on large-scale datasets. The x-axis reports the [PITH_FULL_IMAGE:figures/full_fig_p022_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Dataset-level GPU memory multiple relative to MLP. Each point represents [PITH_FULL_IMAGE:figures/full_fig_p023_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Dataset-level improvement of TabLoRA over Raw Input. Positive values [PITH_FULL_IMAGE:figures/full_fig_p024_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparison of Single, Full, and TabLoRA. The left panel shows average ranks, [PITH_FULL_IMAGE:figures/full_fig_p025_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Fixed-configuration parameter scaling. The left panel shows trainable parame [PITH_FULL_IMAGE:figures/full_fig_p027_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Performance on 50 small- to medium-scale datasets. The left panel reports [PITH_FULL_IMAGE:figures/full_fig_p028_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 13 linked inside Pith

  1. [1]

    Jiang, S.-Y

    J.-P. Jiang, S.-Y. Liu, H.-R. Cai, Q.-L. Zhou, H.-J. Ye, Representation learning for tabular data: A comprehensive survey, IEEE Transactions on Pattern Analysis and Machine Intelligence (2026)

    2026

  2. [2]

    T. Chen, C. Guestrin, Xgboost: A scalable tree boosting system, in: Proceedings of the 22nd acm sigkdd international conference on knowl- edge discovery and data mining, 2016, pp. 785–794

    2016

  3. [3]

    G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, T.-Y. Liu, Lightgbm: A highly efficient gradient boosting decision tree, Advances in neural information processing systems 30 (2017). 4XGBoost:https://xgboost.readthedocs.io/en/stable/ 5CatBoost:https://catboost.ai/ 6LightGBM:https://lightgbm.readthedocs.io/en/latest/ 7RealMLP:https://github.com/d...

    2017

  4. [4]

    Prokhorenkova, G

    L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, A. Gulin, Catboost: unbiased boosting with categorical features, Advances in neu- ral information processing systems 31 (2018)

    2018

  5. [5]

    Borisov, T

    V. Borisov, T. Leemann, K. Seßler, J. Haug, M. Pawelczyk, G. Kasneci, Deep neural networks and tabular data: A survey, IEEE Transactions on Neural Networks and Learning Systems (2022)

    2022

  6. [6]

    Shwartz-Ziv, A

    R. Shwartz-Ziv, A. Armon, Tabular data: Deep learning is not all you 32 Table C.4: Practical memory usage (MB). Theboldindicates the top result;OOMrep- resents there exists GPU overflow Dataset MLP RealMLP NCART TabM TabPFN TabLoRA credit-g103.26351.6 103.2 3466.4 18229.6 1948.0 road-safety108.41150.0 171.2 3466.0 2541.6 1954.4 Epsilon128.416174.8OOM OOM ...

    arXiv 1948

  7. [7]

    Grinsztajn, E

    L. Grinsztajn, E. Oyallon, G. Varoquaux, Why do tree-based models still outperform deep learning on typical tabular data?, in: Thirty- sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022

    2022

  8. [8]

    Hollmann, S

    N. Hollmann, S. Müller, K. Eggensperger, F. Hutter, Tabpfn: A trans- 33 Table C.5: Mean results of the ablation studies.Rawdenotes the TabLoRA variant without the feature adapter, andImprov.denotes the improvement of TabLoRA over Raw.Singledenotes a single-predictor model, andFulldenotes the full deep ensemble with independent predictors. Dataset TabLoRA ...

    Pith/arXiv arXiv 2022

  9. [9]

    Hollmann, S

    N. Hollmann, S. Müller, L. Purucker, A. Krishnakumar, M. Körfer, S. B. Hoo, R. T. Schirrmeister, F. Hutter, Accurate predictions on small data 34 with a tabular foundation model, Nature 637 (8045) (2025) 319–326

    2025

  10. [10]

    L.Grinsztajn, K.Flöge, O.Key, F.Birkel, P.Jund, B.Roof, M.Manium, S. Bin, M. Bühler, A. Garg, et al., Tabpfn-3: Technical report, arXiv preprint arXiv:2605.13986 (2026)

    Pith/arXiv arXiv 2026

  11. [11]

    J. Luo, S. Xu, Ncart: Neural classification and regression tree for tabular data, Pattern Recognition 154 (2024) 110578

    2024

  12. [12]

    Gorishniy, A

    Y. Gorishniy, A. Kotelnikov, A. Babenko, Tabm: Advancing tabu- lar deep learning with parameter-efficient ensembling, arXiv preprint arXiv:2410.24210 (2024)

    Pith/arXiv arXiv 2024

  13. [13]

    Ye, H.-H

    H.-J. Ye, H.-H. Yin, D.-C. Zhan, Modern neighborhood components analysis: A deep tabular baseline two decades later, arXiv preprint arXiv:2407.03257 (2024)

    Pith/arXiv arXiv 2024

  14. [14]

    Shavitt, E

    I. Shavitt, E. Segal, Regularization learning networks: deep learning for tabular datasets, Advances in Neural Information Processing Systems 31 (2018)

    2018

  15. [15]

    Kadra, M

    A. Kadra, M. Lindauer, F. Hutter, J. Grabocka, Well-tuned simple nets excel on tabular datasets, Advances in neural information processing systems 34 (2021) 23928–23941

    2021

  16. [16]

    Holzmüller, L

    D. Holzmüller, L. Grinsztajn, I. Steinwart, Better by default: Strong pre-tuned mlps and boosted trees on tabular data, Advances in Neural Information Processing Systems 37 (2024) 26577–26658. 35

    2024

  17. [17]

    Gorishniy, I

    Y. Gorishniy, I. Rubachev, A. Babenko, On embeddings for numeri- cal features in tabular deep learning, Advances in Neural Information Processing Systems 35 (2022) 24991–25004

    2022

  18. [18]

    Gorishniy, I

    Y. Gorishniy, I. Rubachev, N. Kartashev, D. Shlenskii, A. Kotelnikov, A. Babenko, Tabr: Tabular deep learning meets nearest neighbors, in: International Conference on Learning Representations, Vol. 2024, 2024, pp. 18209–18249

    2024

  19. [19]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, Advances in neural information processing systems 30 (2017)

    2017

  20. [20]

    Huang, A

    X. Huang, A. Khetan, M. Cvitkovic, Z. Karnin, Tabtransformer: Tabular data modeling using contextual embeddings, arXiv preprint arXiv:2012.06678 (2020)

    Pith/arXiv arXiv 2012

  21. [21]

    Gorishniy, I

    Y. Gorishniy, I. Rubachev, V. Khrulkov, A. Babenko, Revisiting deep learning models for tabular data, Advances in Neural Information Pro- cessing Systems 34 (2021) 18932–18943

    2021

  22. [22]

    S. Ö. Arik, T. Pfister, Tabnet: Attentive interpretable tabular learn- ing, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, 2021, pp. 6679–6687

    2021

  23. [23]

    Somepalli, M

    G. Somepalli, M. Goldblum, A. Schwarzschild, C. B. Bruss, T. Gold- stein, Saint: Improved neural networks for tabular data via row at- 36 tention and contrastive pre-training, arXiv preprint arXiv:2106.01342 (2021)

    Pith/arXiv arXiv 2021

  24. [24]

    Kossen, N

    J. Kossen, N. Band, C. Lyle, A. N. Gomez, T. Rainforth, Y. Gal, Self- attention between datapoints: Going beyond individual input-output pairs in deep learning, Advances in Neural Information Processing Sys- tems 34 (2021) 28742–28756

    2021

  25. [25]

    J. Chen, J. Yan, Q. Chen, D. Z. Chen, J. Wu, J. Sun, Can a deep learning model be a sure bet for tabular prediction?, in: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024, pp. 288–296

    2024

  26. [26]

    J. Yan, J. Chen, Y. Wu, D. Z. Chen, J. Wu, T2g-former: organizing tabular features into relation graphs promotes heterogeneous feature interaction, in: Proceedings of the AAAI Conference on Artificial Intel- ligence, Vol. 37, 2023, pp. 10720–10728

    2023

  27. [27]

    B. Sun, L. Yang, W. Zhang, M. Lin, P. Dong, C. Young, J. Dong, Supertml: Two-dimensional word embedding for the precognition on structured tabular data, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019, pp. 0–0

    2019

  28. [28]

    P. Yin, G. Neubig, W.-t. Yih, S. Riedel, Tabert: Pretraining for joint understanding of textual and tabular data, arXiv preprint arXiv:2005.08314 (2020). 37

    Pith/arXiv arXiv 2005

  29. [29]

    Hegselmann, A

    S. Hegselmann, A. Buendia, H. Lang, M. Agrawal, X. Jiang, D. Sontag, Tabllm: Few-shot classification of tabular data with large language mod- els, in: International Conference on Artificial Intelligence and Statistics, PMLR, 2023, pp. 5549–5581

    2023

  30. [30]

    J. Chen, K. Liao, Y. Wan, D. Z. Chen, J. Wu, Danets: Deep abstract networks for tabular data classification and regression, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36, 2022, pp. 3930–3938

    2022

  31. [31]

    J. Chen, K. Liao, Y. Fang, D. Chen, J. Wu, Tabcaps: A capsule neu- ral network for tabular data classification with bow routing, in: The Eleventh International Conference on Learning Representations, 2022

    2022

  32. [32]

    Popov, S

    S. Popov, S. Morozov, A. Babenko, Neural oblivious decision ensem- bles for deep learning on tabular data, arXiv preprint arXiv:1909.06312 (2019)

    Pith/arXiv arXiv 1909

  33. [33]

    J. Yoon, Y. Zhang, J. Jordon, M. van der Schaar, Vime: Extending the successofself-andsemi-supervisedlearningtotabulardomain, Advances in Neural Information Processing Systems 33 (2020) 11033–11043

    2020

  34. [34]

    T. Ucar, E. Hajiramezanali, L. Edwards, Subtab: Subsetting features of tabular data for self-supervised representation learning, Advances in Neural Information Processing Systems 34 (2021) 18853–18865. 38

    2021

  35. [35]

    Hajiramezanali, N

    E. Hajiramezanali, N. L. Diamant, G. Scalia, M. W. Shen, Stab: Self- supervised learning for tabular data, in: NeurIPS 2022 First Table Rep- resentation Workshop, 2022

    2022

  36. [36]

    Z. Wang, J. Sun, Transtab: Learning transferable tabular transformers across tables, Advances in Neural Information Processing Systems 35 (2022) 2902–2915

    2022

  37. [37]

    Levin, V

    R. Levin, V. Cherepanova, A. Schwarzschild, A. Bansal, C. B. Bruss, T. Goldstein, A. G. Wilson, M. Goldblum, Transfer learning with deep tabular models, arXiv preprint arXiv:2206.15306 (2022)

    Pith/arXiv arXiv 2022

  38. [38]

    Grinsztajn, K

    L. Grinsztajn, K. Flöge, O. Key, F. Birkel, P. Jund, B. Roof, B. Jäger, D. Safaric, S. Alessi, A. Hayler, et al., Tabpfn-2.5: Advancing the state of the art in tabular foundation models, arXiv preprint arXiv:2511.08667 (2025)

    Pith/arXiv arXiv 2025

  39. [39]

    J. Ma, V. Thomas, R. Hosseinzadeh, A. Labach, J. Cresswell, K. Golestan, G. Yu, A. L. Caterini, M. Volkovs, Tabdpt: Scaling tab- ular foundation models on real data, Advances in Neural Information Processing Systems 38 (2026) 172692–172722

    2026

  40. [40]

    J. Qu, D. Holzmüller, G. Varoquaux, M. L. Morvan, Tabicl: A tabular foundation model for in-context learning on large data, arXiv preprint arXiv:2502.05564 (2025). 39

    Pith/arXiv arXiv 2025

  41. [41]

    J. Qu, D. Holzmüller, G. Varoquaux, M. L. Morvan, Tabiclv2: A bet- ter, faster, scalable, and open tabular foundation model, arXiv preprint arXiv:2602.11139 (2026)

    arXiv 2026

  42. [42]

    Zhang, G

    X. Zhang, G. Ren, H. Yu, H. Yuan, H. Wang, J. Li, J. Wu, L. Mo, L. Mao, M. Hao, et al., Limix: Unleashing structured-data modeling capability for generalist intelligence, arXiv preprint arXiv:2509.03505 (2025)

    arXiv 2025

  43. [43]

    Y. Wang, X. Zhang, H. Yu, M. Ming, G. Ren, H. Yuan, L. Mao, Y. Zhang, C. Yuan, P. Cui, Limix-2m: Mitigating low-rank collapse and attention bottlenecks in tabular foundation models, arXiv preprint arXiv:2606.04485 (2026)

    Pith/arXiv arXiv 2026

  44. [44]

    N. Ding, Y. Qin, G. Yang, F. Wei, Z. Yang, Y. Su, S. Hu, Y. Chen, C.- M. Chan, W. Chen, et al., Parameter-efficient fine-tuning of large-scale pre-trained language models, Nature machine intelligence 5 (3) (2023) 220–235

    2023

  45. [45]

    Houlsby, A

    N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, S. Gelly, Parameter-efficient transfer learn- ing for nlp, in: International conference on machine learning, PMLR, 2019, pp. 2790–2799

    2019

  46. [46]

    E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, 40 W. Chen, et al., Lora: Low-rank adaptation of large language models., Iclr 1 (2) (2022) 3

    2022

  47. [47]

    Zhang, M

    Q. Zhang, M. Chen, A. Bukharin, N. Karampatziakis, P. He, Y. Cheng, W. Chen, T. Zhao, Adalora: Adaptive budget allocation for parameter- efficient fine-tuning, arXiv preprint arXiv:2303.10512 (2023)

    Pith/arXiv arXiv 2023

  48. [48]

    Dettmers, A

    T. Dettmers, A. Pagnoni, A. Holtzman, L. Zettlemoyer, Qlora: Efficient finetuning of quantized llms, Advances in neural information processing systems 36 (2023) 10088–10115

    2023

  49. [49]

    Akiba, S

    T. Akiba, S. Sano, T. Yanase, T. Ohta, M. Koyama, Optuna: A next- generation hyperparameter optimization framework, in: Proceedings of the 25th ACM SIGKDD international conference on knowledge discov- ery & data mining, 2019, pp. 2623–2631. 41 Table C.6: Mean results of 9 models on 50 datasets. Theboldindicates the top result. Dataset XGBoost CatBoost Lig...

    arXiv 2019