pith. sign in

arxiv: 2607.05380 · v1 · pith:L7LARWDB · submitted 2026-07-06 · cs.LG

TabPack: Efficient Hyperparameter Ensembles for Tabular Deep Learning

Yury Gorishniy , Akim Kotelnikov , Ivan Rubachev , Artem Babenko This is my paper

pith:L7LARWDBreviewed 2026-07-07 13:39 UTCmodel glm-5.2open to challenge →

classification cs.LG
keywords tabpackhyperparameterefficientmlpsperformancetabulartuningdeep
0
0 comments X

The pith

Tuning-free tabular model matches tuned baselines at 1/20th the compute

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

TabPack replaces hyperparameter tuning with hyperparameter sampling. Instead of searching for the best single configuration of a multilayer perceptron (MLP) for a tabular dataset, TabPack packs dozens of MLPs with randomly sampled hyperparameters into one training run, trains them all in parallel through vectorized operations, and greedily selects the best ensemble members on the fly during training. The central claim is that this single tuning-free run achieves mean rank 2.6 across 17 datasets, matching or beating extensively tuned prior methods like TabM (rank 2.8) and RealMLP (rank 3.6), while requiring 2.4 hours of total compute versus 47.2 and 250.9 hours respectively. The key technical object is the packed hyperparameter ensemble: a single module containing m MLPs that differ in depth, dropout, learning rate, weight decay, and embedding parameters, all trained simultaneously and pruned selectively. The paper argues that the main value of hyperparameter diversity is not improving ensemble quality beyond what tuned same-hyperparameter ensembles achieve, but rather eliminating the need to tune at all.

Core claim

The paper shows that for tabular deep learning, the traditional hyperparameter tuning loop can be replaced by a single run that samples many configurations in parallel and lets greedy ensemble selection find what works. The authors demonstrate this with a packed-ensemble architecture where heterogeneous MLPs are stacked along a new dimension and trained together via batched matrix operations, with ensemble members chosen online based on validation performance. The result is that default TabPack on a MacBook runs faster than tuning some baselines on an A100 GPU, while achieving comparable or better mean rank across 17 medium-to-large datasets. The paper also finds that online ensemble (re-e-e

What carries the argument

Packed hyperparameter ensembling: stacking m MLPs with different hyperparameters into a single tensor module, training them in parallel via batched matrix multiplications, and selecting ensemble members greedily during training using validation-set performance as the ensemble score.

If this is right

  • Practitioners can skip expensive hyperparameter search for tabular MLPs by running TabPack once with default sampling ranges, reducing experiment cycles from days to hours.
  • The packed-ensemble pattern could extend to other parts of the training pipeline — loss functions, data preprocessing, architectural choices — diversifying any vectorizable degree of freedom without separate training runs.
  • The finding that hyperparameter diversity primarily enables tuning-free workflows rather than improving raw ensemble quality suggests that future tabular architectures should be designed with built-in configuration robustness rather than relying on external search.
  • The ability to run competitively on consumer hardware (MacBook M4 Pro) shifts the accessibility boundary for tabular deep learning, potentially broadening adoption beyond GPU-equipped research labs.

Load-bearing premise

The claim that TabPack is tuning-free rests on the assumption that the authors' default hyperparameter sampling ranges (e.g., depth 1-3, width 384, learning rate from a fixed log-uniform interval) are good enough across all tabular datasets. If these ranges are poorly suited to a new domain, performance could degrade with no recourse beyond adjusting the ranges — which is itself a form of tuning. The paper evaluates on 17 datasets but does not test sensitivity to range miss

What would settle it

Run TabPack on a new tabular domain where the optimal hyperparameters fall outside the default sampling ranges (e.g., where depth >3 or learning rates outside the specified log-uniform interval are needed) and check whether mean rank degrades relative to tuned baselines.

Figures

Figures reproduced from arXiv: 2607.05380 by Akim Kotelnikov, Artem Babenko, Ivan Rubachev, Yury Gorishniy.

Figure 1
Figure 1. Figure 1: A summary of the results from [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: A high-level overview of TabPack. (Left) TabPack is constructed by packing multiple base models and optimizers with randomly sampled hyperparameters into a single model–optimizer pair. (Right) Training TabPack consists of training its base models independently in parallel and updating the ensemble once per epoch using intermediate base-model checkpoints. The ensemble is the main output of TabPack training.… view at source ↗
Figure 3
Figure 3. Figure 3: Inference throughput for an MLP of depth 3 and width 384 on one NVIDIA A100 GPU with different batch sizes in three regimes: (1) single-process, (2) multi-process, where multiple processes run in parallel using the same GPU (the process count is tuned to show the full potential of this approach), and (3) packed (64 MLPs are stacked into one and applied to 64 stacked batches in parallel). 3.3. Model Pack As… view at source ↗
Figure 4
Figure 4. Figure 4: Packing multiple heterogeneous MLP backbones into one. (Left) Weights of m = 3 MLP backbones of different widths (d1 < d2 < d3) and depths (2, 2, 1). Biases are omitted for simplicity. Wi j denotes the weights of the j-th layer in the i-th MLP. (Right) Weights of the same m = 3 MLPs as on the left, but now packed in one set of weights. @ = = [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: (Left) A forward pass for a pack of linear layers. The input X consists of m object representations padded with zeros to dmax and stacked along the pack dimension. The remaining notation follows [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: (Left) Task performance of tabular models, on the 17 datasets from [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Performance of tabular models as a function of the time budget, averaged over the 17 datasets from [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Task performance of TabPack† depending on different hyperparameters, averaged over the 17 datasets from [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Results on medium-sized datasets (at least 10K objects in total) from the TabArena-Lite benchmark (Erickson et al., 2025). Linear KNN PerpetualBooster TabSTAR RandomForest FastaiMLP ExtraTrees TorchMLP EBM ModernNCA XGBoost xRFM TabPack CatBoost LightGBM TabM TabDPT RealMLP TabICLv2 RealTabPFN-2.5 TabPFN-2.6 TabPFN-3 800 1000 1200 1400 1600 Elo Default Tuned Tuned + Ensembled [PITH_FULL_IMAGE:figures/full… view at source ↗
Figure 10
Figure 10. Figure 10: Results on small-sized datasets (less than 10K objects in total) from the TabArena-Lite benchmark (Erickson et al., 2025). 12 [PITH_FULL_IMAGE:figures/full_fig_p012_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Inference throughput on NVIDIA A100 on 17 datasets from [PITH_FULL_IMAGE:figures/full_fig_p014_11.png] view at source ↗
read the original abstract

In deep learning for tabular data, efficient ensembles of multilayer perceptrons (MLPs) have recently emerged as effective and practical architectures. Existing methods of this kind use the same hyperparameters for all underlying MLPs, which requires hyperparameter tuning for achieving the best performance. In this work, we introduce TabPack, an efficient MLP ensemble with strong out-of-the-box performance and reduced reliance on traditional tuning. In a single run, TabPack samples and trains many MLPs with different hyperparameters efficiently in parallel and selects ensemble members on the fly during training. Thus, TabPack only requires specifying ranges from which to sample MLP hyperparameter rather than exact hyperparameter values, which naturally demands less precision for good performance. In experiments on medium-to-large public datasets, TabPack with default settings performs on par with extensively tuned prior methods, thus substantially reducing effort and compute resources needed to achieve competitive results on tabular tasks. Notably, running the default TabPack configuration on a modern MacBook took less time than tuning some baselines on an industry-grade GPU.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 7 minor

Summary. The paper introduces TabPack, an efficient ensemble of MLPs with randomly sampled hyperparameters for tabular deep learning. The core idea is to pack many MLPs with diverse hyperparameters (depth, dropout, learning rate, etc.) into a single model, train them in parallel, and select ensemble members online during training. This eliminates the need for per-dataset hyperparameter tuning. Experiments on 17 medium-to-large datasets show that a single tuning-free run of TabPack achieves mean rank 2.6, competitive with extensively tuned TabM (rank 2.8) and RealMLP (rank 3.6), while requiring substantially less compute (2.4h total vs 47.2h and 250.9h respectively). Additional evaluations on TabArena and large-scale datasets (Appendix A.1, A.2) provide further evidence of generalizability.

Significance. The paper makes a practical and well-executed contribution to tabular deep learning. The core technical idea of packing heterogeneous-architecture MLPs into a single trainable module (Model Pack and Optimizer Pack, Section 3.3-3.4) is clean and the implementation is non-trivial. The experimental methodology is thorough: 17 datasets, multiple baselines with 100+ tuning iterations, a conservative multi-seed evaluation protocol, per-dataset results with standard deviations (Table 14), ablations (Table 1, Figure 8), and additional benchmarks (Appendix A.1, A.2). The falsifiable claim that TabPack on a MacBook outperforms tuned baselines on an A100 in terms of total compute is a striking and verifiable result. The analysis in Section 5.1 showing that hyperparameter diversity's main role is enabling the tuning-free workflow rather than improving raw performance is a valuable insight. Reproducible code is provided.

major comments (3)
  1. §3.7 and Table 7: The central 'tuning-free' claim depends on the default hyperparameter sampling ranges (e.g., depth UniformInt[1,3], width 384, Muon LR LogUniform[1e-3,1e-1]). The paper does not describe how these ranges were selected. Given that the benchmark (derived from Gorishniy et al. 2025) and the method share authors, there is a risk that the ranges were refined based on performance on these same 17 datasets, which would make the 'tuning-free' claim partially circular (tuning happened at the range-design level). The independent evaluations in Appendix A.1 and A.2 provide partial mitigation, but the TabArena evaluation (Appendix A.2) switches the ensemble score from validation metric to training loss, which is itself a dataset-specific adjustment. The paper should explicitly state how the ranges in Table 7 were chosen (e.g., based on prior work, held-out datasets, or intuition) —
  2. §3.5 and Appendix A.2 implementation note: The switch from validation metric to training loss as the ensemble score for TabArena is presented as a practical adjustment, but it is a dataset-specific hyperparameter choice that sits in tension with the 'tuning-free' framing. The paper should either (a) run TabArena with the default validation-metric ensemble score and report the resulting performance drop, or (b) explicitly acknowledge this as a limitation and discuss when users should expect to make such adjustments. As stated, the reader cannot assess how much the 'tuning-free' claim is weakened by this adjustment.
  3. §5.1, Table 1: TabPack_SameHP achieves rank 1.4±0.7 vs TabPack's 1.6±0.6, meaning traditional per-dataset tuning of TabPack's architecture actually outperforms the tuning-free default. The paper frames this as 'the tuning slightly improves the performance at the cost of noticeable increase of the total runtime.' However, the rank difference (1.4 vs 1.6) is within one standard deviation, and the paper does not test whether this difference is statistically significant. Given that this comparison directly bears on the claim that hyperparameter diversity is 'a reasonable default strategy' (§5.1), a statistical test or a more careful discussion of whether the difference is meaningful would strengthen the conclusion.
minor comments (7)
  1. Figure 6 (left): The triangle-square-triangle markers for dataset groups are explained in the legend but the specific group assignments per dataset are not immediately clear. Consider adding a table or annotation clarifying which datasets fall into 'worst 25%,' 'middle 50%,' and 'best 25%' groups.
  2. §3.3: The statement 'one can usually use smaller maximum widths and depths than in traditional single models' is not directly supported by the experiments. Table 7 shows width is fixed at 384, which is within the tuning range of baselines (UniformInt[64,1024] for TabM in Table 10). Consider softening this claim or providing evidence.
  3. Appendix A.3: The 'thought experiment' about why TabPack performs well on large regression tasks is speculative. While clearly labeled as such, it could benefit from being shortened or moved to a discussion of limitations, as it does not provide empirical evidence for the proposed mechanisms.
  4. Table 7: The dropout distribution '{0.0, Uniform[0.0,0.5]}' is ambiguous — it is unclear whether this means 50% chance of dropout=0.0 and 50% chance of sampling from Uniform[0,0.5], or something else. Clarify the mixture probability.
  5. §4.2: The conservative evaluation protocol reruns TabPack with only the M_ens selected base models. It would be useful to report what fraction of the full performance is retained in the conservative vs optimistic protocol, to quantify the cost of this conservatism.
  6. Figure 8: The y-axis label 'Rel. improv. over MLP' and the specific values (2.0%-3.0%) are not immediately interpretable. Consider clarifying whether this is the same relative improvement metric used in Figure 6.
  7. The paper introduces 'new periodic embeddings' in Appendix B but does not compare them against prior PLR or PBLD embeddings. A brief comparison or justification for why a new variant was needed would help readers.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive review. The three major comments all raise legitimate points about the strength of the 'tuning-free' claim and the statistical interpretation of our ablation. We address each below and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: §3.7 and Table 7: The central 'tuning-free' claim depends on the default hyperparameter sampling ranges. The paper does not describe how these ranges were selected. Given that the benchmark and the method share authors, there is a risk that the ranges were refined based on performance on these same 17 datasets, which would make the 'tuning-free' claim partially circular.

    Authors: The referee raises a valid concern. We will add an explicit description of how the sampling ranges in Table 7 were chosen. To be transparent: the ranges were designed based on our prior experience with tabular MLPs (Gorishniy et al. 2025) and general intuition about reasonable hyperparameter values for this model family, not through systematic optimization on the 17 benchmark datasets. For example, the depth range UniformInt[1,3] and width 384 reflect typical values from prior work; the Muon learning rate range LogUniform[1e-3,1e-1] matches the tuning space used for TabM (Table 11). We agree that some informal iteration on range design is difficult to fully rule out, and we will acknowledge this explicitly in the revised manuscript. The independent evaluations in Appendix A.1 (large-scale datasets) and A.2 (TabArena) provide partial mitigation, as these datasets are not part of the main benchmark. We will state all of this clearly so readers can assess the claim for themselves. revision: yes

  2. Referee: §3.5 and Appendix A.2 implementation note: The switch from validation metric to training loss as the ensemble score for TabArena is presented as a practical adjustment, but it is a dataset-specific hyperparameter choice that sits in tension with the 'tuning-free' framing. The paper should either (a) run TabArena with the default validation-metric ensemble score and report the resulting performance drop, or (b) explicitly acknowledge this as a limitation and discuss when users should expect to make such adjustments.

    Authors: This is a fair point. We will take option (b): explicitly acknowledge this adjustment as a limitation and discuss when users should expect to make it. Specifically, we will note that on datasets with small validation sets (as is common in TabArena), the validation metric becomes a noisier signal for greedy ensemble selection, and using training loss can be more stable. We will add a brief discussion of this trade-off in the revised manuscript. Running TabArena with the default validation-metric ensemble score and reporting the performance drop (option a) is a reasonable suggestion, and we will attempt it if time permits before the camera-ready deadline, but we commit to at least the explicit acknowledgment as a limitation. revision: partial

  3. Referee: §5.1, Table 1: TabPack_SameHP achieves rank 1.4±0.7 vs TabPack's 1.6±0.6, meaning traditional per-dataset tuning of TabPack's architecture actually outperforms the tuning-free default. The paper frames this as 'the tuning slightly improves the performance at the cost of noticeable increase of the total runtime.' However, the rank difference (1.4 vs 1.6) is within one standard deviation, and the paper does not test whether this difference is statistically significant.

    Authors: The referee is correct that the rank difference between TabPack_SameHP (1.4±0.7) and TabPack (1.6±0.6) is within one standard deviation and that we did not test for statistical significance. We will revise the discussion in §5.1 to be more careful: rather than implying that tuning 'slightly improves' performance, we will state that TabPack_SameHP performs comparably to TabPack, with no statistically significant difference detected, and that the main benefit of the tuning-free default is the dramatic reduction in runtime (2.4h vs 65.5h) rather than any performance sacrifice. We will also add a note that the comparison is underpowered for detecting small differences given the number of datasets. The core claim — that random hyperparameter sampling is a reasonable default strategy — is supported by the fact that it achieves statistically indistinguishable performance at a fraction of the cost, not by claiming it outperforms tuning. revision: yes

Circularity Check

0 steps flagged

No significant circularity; self-citations are for evaluation infrastructure, not for results that define the claim by construction.

full rationale

TabPack's central claim — that a single tuning-free run achieves competitive performance with tuned baselines — is an empirical claim evaluated against external baselines (XGBoost, RealMLP, ModernNCA, TabPFN-3, TabICLv2) from independent groups. The self-citations to Gorishniy et al. (2025) and Rubachev et al. (2025) provide evaluation infrastructure (datasets, preprocessing, metric definitions, experiment protocol) and one baseline (TabM), but none of these citations define TabPack's performance by construction. The default hyperparameter ranges in Table 7 are presented as practical design choices, not as first-principles derivations or predictions. The paper does not claim the ranges are derived from theory; it claims they work empirically. The concern that the ranges might have been informally tuned on the same 17 datasets is a generalizability risk (acknowledged via independent TabArena validation in Appendix A.2 and large-scale evaluation in Appendix A.1), not a circularity in the derivation chain. No step in the paper reduces an output to its input by definition or by self-citation. The one minor issue is that the TabArena evaluation switches the ensemble score from validation metric to training loss (Appendix A.2 implementation note), which is a dataset-specific adjustment, but this is transparently reported and does not constitute a circular argument. Overall, the derivation is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 3 invented entities

TabPack's free parameters are all fixed defaults rather than fitted per-dataset, which is the paper's central design choice. The load-bearing axiom is that the sampling ranges generalize. The invented entities (model pack, optimizer pack, online ensemble) are all concretely described and at least partially ablated, with no untestable postulates.

free parameters (6)
  • m (number of base models) = 64
    Fixed by default, not tuned. Chosen by the authors as a practical default.
  • m_ens (max ensemble size) = 32
    Fixed to match TabM baseline; not tuned per dataset.
  • p_base (base model early stopping patience) = 16
    Fixed default; reduced to 4 for large-scale experiments (Appendix A.1).
  • p_ens (ensemble early stopping patience) = 32
    Fixed default; reduced to 8 for large-scale experiments.
  • Base model width = 384
    Fixed constant width for all base models rather than sampled; justified by Table 1 showing DiverseWidths gives comparable performance at higher cost.
  • Hyperparameter sampling ranges (Table 7) = e.g., depth [1,3], dropout [0,0.5], LR LogUniform[1e-3,1e-1]
    Designed by authors as default ranges. Not tuned per dataset. These ranges are the core 'hyperparameters' of TabPack and their universality is assumed but not tested beyond the 17 datasets.
axioms (5)
  • standard math Greedy forward ensemble selection (Caruana et al. 2004) produces good ensembles from a pool of candidates
    Used as the core ensemble construction primitive in Section 3.5. Well-established prior art.
  • domain assumption Packed-ensemble style vectorization (Laurent et al. 2023) correctly preserves individual model behavior when models are padded to max dimensions
    Section 3.3: narrower models zero-padded, shallower models skip deeper layers. The correctness of this padding strategy is assumed, not formally proven, but is straightforward to verify.
  • ad hoc to paper The default hyperparameter sampling ranges generalize across tabular datasets
    The entire tuning-free claim rests on these ranges being universally good. Tested on 17 datasets but not proven to generalize. This is the load-bearing assumption.
  • domain assumption Validation set performance is a reliable proxy for test set ensemble quality
    Greedy ensemble selection and early stopping both use validation set (Section 3.5-3.6). Authors acknowledge in Appendix C this may fail with small validation sets or distribution shift.
  • domain assumption Muon optimizer (Jordan et al. 2024) is suitable for tabular MLPs
    Adopted as default for all methods based on Gorishniy et al. (2026). Applied uniformly so does not bias comparisons, but is an assumption about optimizer choice.
invented entities (3)
  • Model Pack independent evidence
    purpose: Vectorized container for heterogeneous MLPs with different architectures, enabling parallel forward/backward passes
    The packing mechanism is described concretely (Section 3.3, Figures 4-5) and its throughput advantage is measured (Figure 3). Falsifiable: if packing did not accelerate training, Figure 3 would show no speedup.
  • Optimizer Pack independent evidence
    purpose: Vectorized container for optimizers with different hyperparameters (learning rate, weight decay)
    Described in Section 3.4 and Figure 5 (right). Enables different base models to use different optimizer settings. Falsifiable: if optimizer packs did not correctly implement per-model updates, training would diverge.
  • Online ensemble construction independent evidence
    purpose: Rebuilds the greedy ensemble once per epoch from current base model states plus previous ensemble members
    Section 3.5. Ablated against offline ensemble (Table 1, Figure 6): online (rank 1.6) outperforms offline (rank not directly comparable but shown worse in Figure 6). The ablation provides independent evidence the online approach matters.

pith-pipeline@v1.1.0-glm · 26442 in / 3591 out tokens · 275660 ms · 2026-07-07T13:39:27.036340+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

163 extracted references · 163 canonical work pages · 11 internal anchors

  1. [1]

    KDD , year=

    Optuna: A Next-generation Hyperparameter Optimization Framework , author=. KDD , year=

    work page

  2. [2]

    ASPLOS , year =

    PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation , author =. ASPLOS , year =

    work page

  3. [3]

    ICLR , year=

    Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning , author=. ICLR , year=

    work page

  4. [4]

    NeurIPS , year=

    Depth Uncertainty in Neural Networks , author=. NeurIPS , year=

    work page

  5. [5]

    Arik and Tomas Pfister , journal=

    Sercan O. Arik and Tomas Pfister , journal=. Tab

    work page

  6. [6]

    arXiv , volume=

    Layer Normalization , author=. arXiv , volume=

    work page

  7. [7]

    Keerthi , journal=

    Sarkhan Badirli and Xuanqing Liu and Zhengming Xing and Avradeep Bhowmik and Khoa Doan and Sathiya S. Keerthi , journal=. Gradient Boosting Neural Networks: Grow

    work page

  8. [8]

    Dara Bahri and Heinrich Jiang and Yi Tay and Donald Metzler , booktitle=

    work page

  9. [9]

    Nature Communications , volume=

    Searching for Exotic Particles in High-energy Physics with Deep Learning , author=. Nature Communications , volume=

    work page

  10. [10]

    arXiv , volume=

    Generalization Properties of Retrieval-based Models , author=. arXiv , volume=

    work page

  11. [11]

    Computers and Electronics in Agriculture , volume=

    Comparative Accuracies of Artificial Neural Networks and Discriminant Analysis in Predicting Forest Cover Types from Cartographic Variables , author=. Computers and Electronics in Agriculture , volume=

    work page

  12. [12]

    ICML , year=

    Improving Language Models by Retrieving from Trillions of Tokens , author=. ICML , year=

    work page

  13. [13]

    Neural Computation , volume=

    Local Learning Algorithms , author=. Neural Computation , volume=

    work page

  14. [14]

    Chen, Tianqi and Guestrin, Carlos , booktitle=

    work page

  15. [15]

    arXiv , volume =

    Jintai Chen and Jiahuan Yan and Danny Ziyi Chen and Jian Wu , title =. arXiv , volume =

    work page

  16. [16]

    Trompt: Towards a Better Deep Neural Network for Tabular Data , booktitle =

    Kuan. Trompt: Towards a Better Deep Neural Network for Tabular Data , booktitle =

    work page

  17. [17]

    Nat Struct Mol Biol , year=

    AlphaFold2 and the future of structural biology , author=. Nat Struct Mol Biol , year=

    work page

  18. [18]

    EMNLP , year=

    Case-based Reasoning for Natural Language Queries over Knowledge Bases , author=. EMNLP , year=

    work page

  19. [19]

    ImageNet:

    Jia Deng and Wei Dong and Richard Socher and Li. ImageNet:

    work page

  20. [20]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , author=

    work page

  21. [21]

    arXiv , volume=

    Deep Ensembles: A Loss Landscape Perspective , author=. arXiv , volume=

    work page

  22. [22]

    Advances in Neural Information Processing Systems , year=

    GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration , author=. Advances in Neural Information Processing Systems , year=

    work page

  23. [23]

    Vetrov and Andrew Gordon Wilson , title =

    Timur Garipov and Pavel Izmailov and Dmitrii Podoprikhin and Dmitry P. Vetrov and Andrew Gordon Wilson , title =. NeurIPS , year =

    work page

  24. [24]

    Deep Learning , author=

    work page

  25. [25]

    NeurIPS , year=

    Revisiting Deep Learning Models for Tabular Data , author=. NeurIPS , year=

    work page

  26. [26]

    NeurIPS , year=

    On Embeddings for Numerical Features in Tabular Deep Learning , author=. NeurIPS , year=

    work page

  27. [27]

    Yury Gorishniy and Ivan Rubachev and Nikolay Kartashev and Daniil Shlenskii and Akim Kotelnikov and Artem Babenko , booktitle=. Tab

    work page

  28. [28]

    Yury Gorishniy and Akim Kotelnikov and Artem Babenko , booktitle=. Tab

    work page

  29. [29]

    Benchmarking Optimizers for MLPs in Tabular Deep Learning , author=

    work page

  30. [30]

    Datasets and Benchmarks

    Why do tree-based models still outperform deep learning on typical tabular data? , author=. NeurIPS, the "Datasets and Benchmarks" track , year=

    work page

  31. [31]

    TabPFN-3: Technical Report , author=

    work page

  32. [32]

    ICML , year=

    Retrieval Augmented Language Model Pre-Training , author=. ICML , year=

    work page

  33. [33]

    Training independent subnetworks for robust prediction , author =

    work page

  34. [34]

    ICML , year=

    The tree ensemble layer: Differentiability meets conditional computation , author=. ICML , year=

    work page

  35. [35]

    CVPR , year =

    Deep Residual Learning for Image Recognition , author =. CVPR , year =

    work page

  36. [36]

    category

    Helton Tech, LLC , journal=. category

    work page

  37. [37]

    Noah Hollmann and Samuel Müller and Katharina Eggensperger and Frank Hutter , booktitle=. Tab

    work page

  38. [38]

    arXiv , volume=

    Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Data , author=. arXiv , volume=

    work page

  39. [39]

    Le , title =

    Weizhe Hua and Zihang Dai and Hanxiao Liu and Quoc V. Le , title =

    work page

  40. [40]

    arXiv , volume=

    TabTransformer: Tabular Data Modeling Using Contextual Embeddings , author=. arXiv , volume=

    work page

  41. [41]

    ICML , year =

    Sergey Ioffe and Christian Szegedy , title =. ICML , year =

    work page

  42. [42]

    arXiv , volume=

    A Memory Transformer Network for Incremental Learning , author=. arXiv , volume=

    work page

  43. [43]

    arXiv , volume=

    Few-shot Learning with Retrieval Augmented Language Models , author=. arXiv , volume=

    work page

  44. [44]

    An Introduction to Statistical Learning , author=

    work page

  45. [45]

    NeurIPS , year=

    Joint Training of Deep Ensembles Fails Due to Learner Collusion , author=. NeurIPS , year=

    work page

  46. [46]

    Alan Jeffares and Tennison Liu and Jonathan Crabbé and Fergus Imrie and Mihaela van der Schaar , booktitle=

    work page

  47. [47]

    Rethinking Nearest Neighbors for Visual Classification

    Rethinking nearest neighbors for visual classification , author=. arXiv preprint arXiv:2112.08459 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  48. [48]

    NeurIPS , year=

    Well-tuned Simple Nets Excel on Tabular Datasets , author=. NeurIPS , year=

    work page

  49. [49]

    Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , author=

    work page

  50. [50]

    ICML , year =

    Not All Samples Are Created Equal: Deep Learning with Importance Sampling , author =. ICML , year =

    work page

  51. [51]

    Ke, Guolin and Meng, Qi and Finley, Thomas and Wang, Taifeng and Chen, Wei and Ma, Weidong and Ye, Qiwei and Liu, Tie-Yan , journal=. Light

    work page

  52. [52]

    ICLR , year=

    Generalization through Memorization: Nearest Neighbor Language Models , author=. ICLR , year=

    work page

  53. [53]

    ICLR , year=

    Attentive Neural Processes , author=. ICLR , year=

    work page

  54. [54]

    ICML , year=

    The Lipschitz Constant of Self-Attention , author=. ICML , year=

    work page

  55. [55]

    arXiv , year=

    Myung Jun Kim and Léo Grinsztajn and Ga. arXiv , year=

    work page

  56. [56]

    NIPS , year=

    Self-Normalizing Neural Networks , author=. NIPS , year=

    work page

  57. [57]

    NeurIPS , year=

    Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning , author=. NeurIPS , year=

    work page

  58. [58]

    NeurIPS , year=

    Learning Enhanced Representation for Tabular Data via Neighborhood Propagation , author=. NeurIPS , year=

    work page

  59. [59]

    NeurIPS , year =

    Balaji Lakshminarayanan and Alexander Pritzel and Charles Blundell , title =. NeurIPS , year =

    work page

  60. [60]

    Packed Ensembles for efficient uncertainty estimation , author =

    work page

  61. [61]

    Crandall and Dhruv Batra , title =

    Stefan Lee and Senthil Purushwalkam and Michael Cogswell and David J. Crandall and Dhruv Batra , title =. arXiv , volume =

    work page

  62. [62]

    Patrick S. H. Lewis and Ethan Perez and Aleksandra Piktus and Fabio Petroni and Vladimir Karpukhin and Naman Goyal and Heinrich K. Retrieval-Augmented Generation for Knowledge-Intensive. NeurIPS , year=

    work page

  63. [63]

    NeurIPS , year=

    Pay Attention to MLPs , author=. NeurIPS , year=

    work page

  64. [64]

    A ConvNet for the 2020s , author=

    work page

  65. [65]

    CVPR , year=

    Retrieval Augmented Classification for Long-Tail Visual Recognition , author=. CVPR , year=

    work page

  66. [66]

    ICLR , year=

    Decoupled Weight Decay Regularization , author=. ICLR , year=

    work page

  67. [67]

    Proceedings of the 28th Annual

    Gesture unit segmentation using support vector machines: segmenting gestures from rest positions , author=. Proceedings of the 28th Annual

    work page

  68. [68]

    ArXiv , year=

    Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks , author=. ArXiv , year=

    work page

  69. [69]

    Sascha Marton and Stefan Lüdtke and Christian Bartelt and Heiner Stuckenschmidt , booktitle =

    work page

  70. [70]

    ICML , year=

    DNNR: Differential Nearest Neighbors Regression , author=. ICML , year=

    work page

  71. [71]

    Statistics & Probability Letters , volume=

    Sparse spatial autoregressions , author=. Statistics & Probability Letters , volume=

    work page

  72. [72]

    Adam Paszke and Sam Gross and Francisco Massa and Adam Lerer and James Bradbury and Gregory Chanan and Trevor Killeen and Zeming Lin and Natalia Gimelshein and Luca Antiga and Alban Desmaison and Andreas K. Py. NeurIPS , year=

    work page

  73. [73]

    and Varoquaux, G

    Pedregosa, F. and Varoquaux, G. and Gramfort, A. and Michel, V. and Thirion, B. and Grisel, O. and Blondel, M. and Prettenhofer, P. and Weiss, R. and Dubourg, V. and Vanderplas, J. and Passos, A. and Cournapeau, D. and Brucher, M. and Perrot, M. and Duchesnay, E. , journal=. Scikit-learn: Machine Learning in

    work page

  74. [74]

    ICLR , year=

    Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data , author=. ICLR , year=

    work page

  75. [75]

    Liudmila Prokhorenkova and Gleb Gusev and Aleksandr Vorobev and Anna Veronika Dorogush and Andrey Gulin , booktitle=. Cat

    work page

  76. [76]

    Introducing

    Tao Qin and Tie. Introducing. arXiv , volume=

    work page

  77. [77]

    SIGIR , year=

    Jiarui Qin and Weinan Zhang and Xin Wu and Jiarui Jin and Yuchen Fang and Yong Yu , title=. SIGIR , year=

    work page

  78. [78]

    Retrieval

    Jiarui Qin and Weinan Zhang and Rong Su and Zhirong Liu and Weiwen Liu and Ruiming Tang and Xiuqiang He and Yong Yu , booktitle=. Retrieval

    work page

  79. [79]

    Hopfield Networks is All You Need , author=

    work page

  80. [80]

    NeurIPS , year =

    On Variance Reduction in Stochastic Gradient Descent and it Asynchronous Variants , author =. NeurIPS , year =

    work page

Showing first 80 references.