Pith. sign in

REVIEW 4 major objections 5 minor 55 references

Revisiting Bayesian Model Averaging in the Era of Foundation Models

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Bayesian model averaging, done with frozen features and linear heads, is practical for ensembling foundation models and beats uniform output averaging at a fraction of fine-tuning cost.

desk verdict OMA is a fit to the evaluation set, not a prediction, and the BMA supplement has a wrong log-determinant identity; the framework is clear but the evidence is not. read the letter →

arxiv 2505.21857 v1 pith:J2IXHTRN submitted 2025-05-28 cs.LG stat.ML

Mijung Park This is my paper
classification cs.LGstat.ML MSC 62F1568T07
keywords BayesianmodelaveragingensemblingfoundationmodelsLaplaceapproximationentropyminimizationlinearclassifiersImageNetGLUE
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that Bayesian model averaging, the classical recipe for weighting multiple candidate models by their posterior probability, can be applied to modern foundation models without fine-tuning them. The trick is to freeze each model and learn a thin linear classifier on top of its features, so the intractable posterior over model parameters collapses to an approximate posterior over one linear head per model, computable with a block-diagonal Laplace approximation. This yields model weights that prefer the models most consistent with the data, improving image classification over uniform output averaging. For cases where linear heads are not viable or labels are unavailable, the paper introduces OMA, which learns ensemble weights by directly minimizing the expected entropy of the averaged predictions; the paper reports that OMA beats output averaging on ImageNet out-of-distribution sets and GLUE text tasks at a fraction of the compute of full fine-tuning.

What carries the argument

The machinery is the pairing of a Laplace-approximated posterior over linear classifiers on frozen features with an entropy-based weight optimizer. For BMA, the unnormalized log-posterior is expanded quadratically around the MAP estimate of each linear head; the approximate log marginal likelihood is the MAP log-likelihood minus a prior penalty and a log-determinant term involving the Hessian and prior precision, and the block-diagonal Hessian keeps the computation feasible at ImageNet scale. For OMA, the objective is the average expected entropy of the label predictive mixture, with weights constrained to the simplex via a softplus reparameterization and optimized by gradient descent under a quadratic prior penalty; the zero-shot or MAP predictions of the individual models supply the mixture components.

What would settle it

Fit OMA weights on one half of each benchmark's data (or on an unlabeled sample disjoint from the labels) and report accuracy on the other half; if the margin over output averaging disappears or reverses, the OMA generalization claim is refuted. A second test is to resample the weight-fitting set several times and check whether the reported margins survive on a truly held-out split.

Watch

Extended reading notes

Core claim

The paper's central claim is that BMA is both tractable and beneficial for foundation-model ensembles once the foundation models are frozen and only per-model linear heads are learned. With a Gaussian prior on the head weights and a multiclass logistic likelihood on the frozen features, the marginal likelihood of each model is approximated in closed form via Laplace's method, using a block-diagonal Hessian whose diagonal dominance is demonstrated empirically. The resulting posterior model weights—which concentrate on the features that matter, such as the two strongest OpenCLIP models for ImageNet—drive a posterior predictive that beats uniform output averaging on in-distribution data. The companion OMA scheme replaces posterior computation with direct optimization of the ensemble weights against an expected-entropy objective justified by the lemma that a Bayesian model average has no larger expected surprise than any single model; on out-of-distribution image benchmarks and on MRPC, RTE, CoLA and SST-2, the paper reports consistent gains over output averaging. The paper frames both methods as low-cost alternatives to full fine-tuning, noting that its BMA runs on a single 24 GB GPU.

Load-bearing premise

The reported OMA gains assume that optimizing ensemble weights on a validation set and then computing accuracy on that same set measures generalization; in the ImageNet and GLUE protocols that set is the standard test set, so the gains measure fit rather than independent prediction.

Editorial extensions

If this is right

  • Ensembling foundation models no longer requires access to their training data or gradient updates: the whole BMA pipeline is feature extraction plus linear-head fitting.
  • A model zoo can be treated as a living ensemble: newly released foundation models can be added by computing their features and linear-head posterior, improving classification without retraining existing members.
  • OMA provides a label-free way to adapt ensemble weights to a shifted target distribution, provided the set used for weighting is separate from the set used for evaluation.
  • The posterior weights themselves are interpretable diagnostics, revealing which pretrained features a dataset actually relies on.
  • The computational footprint—a single 24 GB GPU for image experiments and 16 GB for text—makes the approach accessible in academic settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The entropy-minimization objective could double as a test-time adaptation criterion: if OMA weights were optimized on an unlabeled sample from the target distribution and evaluated on a disjoint labeled split, the method might provide a robust alternative to gradient-based adaptation, an extension the paper does not explore.
  • The block-diagonal Hessian approximation is validated only by inspecting a small sub-block; a systematic study of when off-diagonal correlations between class-feature pairs matter would clarify where BMA weights start to degrade.
  • Because the OMA objective is exactly the entropy term in the BMA lemma, one could likely derive a finite-sample bound on the entropy gap between the optimized average and any single model; the paper does not provide such a guarantee.
  • The reported OMA gains on ImageNet variants likely carry an optimistic bias from tuning weights on the evaluation set itself, so a fair comparison against output averaging requires a proper train/validation/test split of the target distribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes two ensembling methods for foundation models: BMA, which trains linear classifiers on frozen features and weights models by a Laplace-approximated marginal likelihood (Eq. 7), and OMA, which directly optimizes ensemble weights on an unlabeled validation set by minimizing expected predictive entropy (Eqs. 10–12). Experiments on ImageNet and GLUE tasks report that both methods improve over simple output averaging while requiring less compute than full fine-tuning. The central claims depend on the empirical comparisons in Tables 2 and 3.

Significance. If the results were valid, the paper would offer a computationally attractive alternative to fine-tuning entire foundation models, and the OMA entropy-minimization idea is a reasonable contribution to the output-ensembling literature. The authors are also transparent about the approximate nature of the Bayesian computation and about the limitations of BMA under distribution shift. However, the empirical support is currently compromised by an evaluation protocol that tunes on the reported test set, and the BMA marginal-likelihood expansion in Eq. (20) contains a mathematical error. These issues are load-bearing, so the significance of the contribution cannot be assessed until they are addressed.

major comments (4)
  1. [Algorithm 2; Section 5.1–5.2] The OMA evaluation protocol is circular: Algorithm 2 Step 1 optimizes β by minimizing Eq. (11) on D_v, and Step 2 reports Eq. (12) on the same D_v. In Sections 5.1 and 5.2, D_v is the standard ImageNet validation set and the GLUE validation sets, respectively, so the reported OMA improvements over output averaging in Tables 2 and 3 are fits to the evaluation set rather than measurements of generalization. A separate held-out split is required before these numbers can support the claimed benefit.
  2. [Supplementary G.1–G.2] The hyperparameters used in the reported results are selected on the test/validation accuracy itself. Supplementary G.1 states that the prior variance α in Eq. (7) is chosen by grid search to maximize test accuracy, and Supplementary G.2 states that the OMA regularization constant λ in Eq. (11) is grid-searched on the same validation/test sets. This test-set tuning applies to both BMA and OMA, so the improvements in Tables 2 and 3 may reflect selection on the evaluation data rather than a generally better ensembling method.
  3. [Supplementary B, Eq. (20)] Equation (20) states ½ log |HS_α + I| = ½ Σ_j (λ_j α + 1). The correct expansion is ½ Σ_j log(λ_j α + 1). This error changes the approximate log marginal likelihood in Eq. (7) and therefore the BMA posterior weights in Eq. (8); as written, the BMA-specific quantitative results are unreliable.
  4. [Table 2, Section 5.1] The claim that BMA outperforms output averaging is contradicted by the paper's own results on ImageNet-R, where BMA is 1.18% worse than output averaging (94.84 vs 95.97). Section 5.1 acknowledges that posterior weights are less useful under large distribution shift, but the abstract and Section 1 state the improvement without this caveat. The claim should be qualified to the setting where the training distribution is close to the evaluation distribution.
minor comments (5)
  1. [Section 3.2, Eq. (10)] Equation (10) uses m as the summation index in the first term but the preceding text defines the validation set as D_v = {x_i^*}_{i=1}^M; the notation should be consistent (e.g., i or m).
  2. [Algorithm 2] In Algorithm 2, the input line reads 'a contatnt λ'; this should be 'a constant λ'. Also, the training steps P are not used in the pseudocode or explained.
  3. [Figure 2] Figure 2's y-axis is labeled 'posterior weight acc'; the reader cannot tell which curve corresponds to accuracy and which to posterior weight. Use separate labeled plots or a legend.
  4. [References] The same paper is cited twice in different forms: [8] and [54] both list Wortsman et al., 'Model soups'; this should be consolidated.
  5. [Supplementary B, Eq. (17)] Equation (17) writes p(x_i)_k = σ(W_map x_i), but W_map is not defined; presumably it is the weight matrix whose columns are the per-class MAP estimates w_k^{map}.

Circularity Check

2 steps flagged · score 6.0 of 10

OMA's headline gains are fit-to-evaluation-set results: Algorithm 2 optimizes beta on D_v and Tables 2-3 score accuracy on the same D_v; hyperparameters alpha and lambda are additionally grid-searched on test accuracy.

  1. fitted input called prediction [Algorithm 2, Steps 1-2; Section 3.2, Eqs. (11)-(12); Tables 2-3]
    "Step 1. Train for the model weights βl: Optimize the weights by minimizing eq. 11. Step 2. Posterior predictive: Compute the predictive distribution given in eq. 12 for Dv ... To answer, we revisit Lemma. 2.1 and propose directly optimizing the entropy of the predictions made by model averaging to find the optimal model weight for a given validation set Dv = {x∗i}M i=1."

    The OMA weights beta are optimized on D_v via Eq. (11), then Eq. (12) computes the predictive distribution and Tables 2-3 report accuracy on the same D_v, which in the experiments is the standard ImageNet or GLUE validation/test set. Thus the reported OMA accuracy measures the ensemble's fit to the exact inputs used to choose beta, not its generalization to new inputs. By the paper's own equations, the claimed improvement over output averaging is a transductive fit to the evaluation set rather than an independent prediction.

  2. fitted input called prediction [Supplementary G.1-G.2]
    "For other datasets, we used the same grid search over α and chose the optimal value that maximizes the test accuracy. ... Similarly, for other datasets, we use the similar grid search to find their optimal λ."

    The BMA prior variance alpha (Eq. 7), which controls the posterior model weights, and the OMA regularization constant lambda (Eq. 11) are both selected by maximizing test accuracy, and the tables then report those test accuracies. The headline numbers are therefore selected on the evaluation metric, so they overstate expected performance on unseen data. This affects both BMA and OMA results, although BMA's posterior weighting itself is a training-data computation.

full rationale

The BMA derivation (Eqs. 3-9) is not circular: posterior weights come from a Laplace-approximated marginal likelihood on training data, and Algorithm 1 evaluates the predictive distribution on D_v. The circularity is concentrated in the OMA evaluation protocol: Algorithm 2 optimizes beta on D_v via Eq. 11 and then reports Eq. 12 on the same D_v, and Section 5.1/5.2 use the standard ImageNet/GLUE validation sets for those accuracies. That makes the OMA gain over output averaging a transductive fit to the evaluation inputs, not an independent generalization prediction. Hyperparameter selection compounds the issue: Supplementary G.1/G.2 choose alpha and lambda by maximizing test accuracy, so even BMA's Table 2 numbers are selected on the reported metric. The missing log in Eq. 20 is a separate algebraic error in the approximate marginal likelihood and affects BMA weights, but it is a correctness risk rather than circularity. No load-bearing self-citation chain appears; the Lemma 2.1 guarantee is an external standard result with an included proof. Overall, partial circularity is present, concentrated in the OMA evaluation protocol and test-set hyperparameter tuning.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims rest on standard Bayesian approximations plus the OMA entropy-minimization assumption and the transductive evaluation protocol. The free parameters alpha and lambda are tuned on the evaluation data, which inflates reported performance.

free parameters (4)
  • Prior variance alpha (BMA) = varies per dataset (e.g., 80 for ImageNet-1K, 10 for Flowers102, 100 for Sun397)
    Hyperparameter in eq. 7 controlling Gaussian prior on linear weights; chosen by grid search to maximize test accuracy (Supp. G.1).
  • Regularization constant lambda (OMA) = varies per dataset (e.g., 1-10000)
    Hyperparameter in eq. 11 controlling deviation from prior beta0; selected per dataset via grid search (Supp. G.2).
  • Prior beta0 for zero-shot models = training log-likelihood under zero-shot weights
    Used as proxy for model prior when no MAP is available (Supp. G.3, eq. 21).
  • Number of feature extractors L = 8 for ImageNet suite, 5 for other image datasets
    Selected from OpenCLIP based on average zero-shot performance and training data diversity (Table 1 and Table 6).
assumptions (5)
  • domain assumption Laplace approximation: the posterior over each linear classifier w_l is a multivariate Gaussian centered at the MAP estimate.
    Invoked in Section 3.1.2, eq. 5-7, to compute the marginal likelihood; assumes the quadratic Taylor expansion is accurate for a million-parameter logistic regression.
  • domain assumption Block-diagonal Hessian: off-diagonal class blocks are negligible for the full Hessian of the logistic likelihood.
    Used in Section 3.1.2 and Supp. B, justified by Fig. 1 and reference [14], but not proven for these features/models.
  • standard math Uniform model prior p(M_l)=1/L over the L candidate models.
    Stated in Section 3.1.3, eq. 8, a standard choice in BMA.
  • ad hoc to paper For OMA, minimizing expected entropy of the ensemble prediction on unlabeled validation data improves classification accuracy.
    Core assumption of Algorithm 2 (Section 3.2, eq. 10); no theoretical guarantee that lower entropy implies higher accuracy, and the evaluation is on the same D_v used for optimization.
  • ad hoc to paper The validation set D_v can serve simultaneously as the weight-tuning set and the evaluation set.
    Assumed implicitly in Algorithm 2 and Section 5; without a separate test set, measured accuracy is training-set performance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Revisiting Bayesian Model Averaging in the Era of Foundation Models." pith.science (2026). https://pith.science/paper/J2IXHTRN

@misc{pith2026250521857,
  author       = {Pith},
  title        = {Pith review of: Revisiting Bayesian Model Averaging in the Era of Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J2IXHTRN}},
  note         = {Machine review of arXiv:2505.21857}
}
read the original abstract

We revisit the classical, full-fledged Bayesian model averaging (BMA) paradigm to ensemble pre-trained and/or lightly-finetuned foundation models to enhance the classification performance on image and text data. To make BMA tractable under foundation models, we introduce trainable linear classifiers that take frozen features from the pre-trained foundation models as inputs. The model posteriors over the linear classifiers tell us which linear heads and frozen features are better suited for a given dataset, resulting in a principled model ensembling method. Furthermore, we propose a computationally cheaper, optimizable model averaging scheme (OMA). In OMA, we directly optimize the model ensemble weights, just like those weights based on model posterior distributions in BMA, by reducing the amount of surprise (expected entropy of the predictions) we get from predictions of ensembled models. With the rapid development of foundation models, these approaches will enable the incorporation of future, possibly significantly better foundation models to enhance the performance of challenging classification tasks.

Figures

Figures reproduced from arXiv: 2505.21857 by the authors.

Figure 1
Figure 1. The diagonal elements are orders of magnitude larger than off-diagonal elements (with the MAP [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Model posterior weights given the Imagenet-1K training data. The first and the fourth feature [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. OMA weights learned by Algorithm 2 and individual model’s performance. For visualization [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The diagonal elements are orders of magnitude larger than off-diagonal elements (with the MAP [PITH_FULL_IMAGE:figures/full_fig_p024_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 37 canonical work pages

  1. [1]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning (ICML), 2021. https://arxiv.org/abs/2103.00020

    arXiv 2021

  2. [2]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248–255. Ieee, 2009

    2009

  3. [3]

    Coca: Contrastive captioners are image-text foundation models

    Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. Transactions on Machine Learning Research, 2022

    2022

  4. [4]

    Eva-02: A visual representation for neon genesis

    Yuxin Fang, Quan Sun, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. Eva-02: A visual representation for neon genesis. Image Vis. Comput. , 149:105171, 2024

    work page 2024

  5. [5]

    Low-rank few-shot adaptation of vision-language models

    Maxime Zanella and Ismail Ben Ayed. Low-rank few-shot adaptation of vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops , pages 1593–1603, 2024

    work page 2024

  6. [6]

    Robust fine-tuning of zero-shot models

    Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo-Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, and Ludwig Schmidt. Robust fine-tuning of zero-shot models. CVPR, 2021. https://arxiv.org/abs/2109.01903

    arXiv 2021

  7. [7]

    The evolution of out-of-distribution robustness throughout fine-tuning

    Anders Johan Andreassen, Yasaman Bahri, Behnam Neyshabur, and Rebecca Roelofs. The evolution of out-of-distribution robustness throughout fine-tuning. Transactions on Machine Learning Research, 2022

    work page 2022

  8. [8]

    Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

    Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba...

    work page 2022

Show all 55 references
  1. [9]

    Openclip, July 2021

    Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip, July 2021. If you use this software, please cite it as below

    2021

  2. [10]

    I. J. Good. Rational decisions. Journal of the Royal Statistical Society. Series B (Methodological) , 14(1):107–114, 1952

    1952

  3. [11]

    David Madigan and Adrian E. Raftery. Model selection and accounting for model uncertainty in graphical models using occam’s window. Journal of the American Statistical Association , 89(428):1535–1546, 1994

    1994

  4. [12]

    Pauler, Jonathan C

    Donna K. Pauler, Jonathan C. Wakefield, and Robert E. Kass. Bayes factors and approximations for variance component models. Journal of the American Statistical Association , 94:1242–1253, 1999

    1999

  5. [13]

    Active learning of neural response functions with gaussian processes

    Mijung Park, Greg Horwitz, and Jonathan Pillow. Active learning of neural response functions with gaussian processes. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems , volume 24. Curran Asso...

    2011

  6. [14]

    Heavy-tailed class imbalance and why adam outperforms gradient descent on language models

    Frederik Kunstner, Alan Milligan, Robin Yadav, Mark Schmidt, and Alberto Bietti. Heavy-tailed class imbalance and why adam outperforms gradient descent on language models. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances i...

    2024

  7. [15]

    What uncertainties do we need in bayesian deep learning for computer vision? In I

    Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 30. Curr...

    2017

  8. [16]

    Zico Kolter

    Vaishnavh Nagarajan and J. Zico Kolter. Uniform convergence may be unable to explain generalization in deep learning. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d 'Alch´ e-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 32...

    2019

  9. [17]

    Linear mode connectivity and the lottery ticket hypothesis

    Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. Linear mode connectivity and the lottery ticket hypothesis. In International Conference on Machine Learning (ICML) , 2020. https://arxiv.org/abs/1912.05671

    2020 arXiv

  10. [18]

    What is being transferred in transfer learning? In Advances in Neural Information Processing Systems (NeurIPS) , 2020

    Behnam Neyshabur, Hanie Sedghi, and Chiyuan Zhang. What is being transferred in transfer learning? In Advances in Neural Information Processing Systems (NeurIPS) , 2020. https://arxiv.org/abs/ 2008.11687

    2020 arXiv

  11. [19]

    Neural networks with late-phase weights

    Johannes Von Oswald, Seijin Kobayashi, Joao Sacramento, Alexander Meulemans, Christian Henning, and Benjamin F Grewe. Neural networks with late-phase weights. ICLR, 2020

    2020

  12. [20]

    Merging models with fisher-weighted averaging, 2021

    Michael Matena and Colin Raffel. Merging models with fisher-weighted averaging, 2021. https: //arxiv.org/abs/2111.09832

    2021 arXiv

  13. [21]

    Ensemble methods in machine learning

    Thomas G Dietterich. Ensemble methods in machine learning. In International workshop on multiple classifier systems , 2000. https://link.springer.com/chapter/10.1007/3-540-45014-9_1

    doi

  14. [22]

    An empirical comparison of voting classification algorithms: Bagging, boosting, and variants

    Eric Bauer and Ron Kohavi. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine learning, 1999. https://link.springer.com/article/10.1023/A: 1007515423169

    1999 doi

  15. [23]

    Bagging predictors

    Leo Breiman. Bagging predictors. Machine learning, 1996. https://link.springer.com/article/10. 1007/BF00058655

    1996

  16. [24]

    The elements of statistical learning

    Jerome Friedman, Trevor Hastie, Robert Tibshirani, et al. The elements of statistical learning . Springer series in statistics New York, 2001. 12

    2001

  17. [25]

    Simple and scalable predictive uncertainty estimation using deep ensembles

    Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems (NeurIPS), 2017. https://arxiv.org/abs/1612.01474

    2017 arXiv

  18. [26]

    A decision-theoretic generalization of on-line learning and an application to boosting

    Yoav Freund and Robert E Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences , 1997. https://www.sciencedirect. com/science/article/pii/S002200009791504X

    1997

  19. [27]

    Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift

    Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua V Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Pro...

    2019 arXiv

  20. [28]

    Deep ensembles for low-data transfer learning, 2020

    Basil Mustafa, Carlos Riquelme, Joan Puigcerver, Andr´ e Susano Pinto, Daniel Keysers, and Neil Houlsby. Deep ensembles for low-data transfer learning, 2020. https://arxiv.org/abs/2010.06866

    2020 arXiv

  21. [29]

    Being bayesian, even just a bit, fixes over- confidence in ReLU networks

    Agustinus Kristiadi, Matthias Hein, and Philipp Hennig. Being bayesian, even just a bit, fixes over- confidence in ReLU networks. In Hal Daum´ e III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Mac...

    2020

  22. [30]

    Variational bayesian last layers

    James Harrison, John Willes, and Jasper Snoek. Variational bayesian last layers. In The Twelfth International Conference on Learning Representations , 2024

    2024

  23. [31]

    Do ImageNet classifiers generalize to ImageNet? In International Conference on Machine Learning (ICML) , 2019

    Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do ImageNet classifiers generalize to ImageNet? In International Conference on Machine Learning (ICML) , 2019. https: //arxiv.org/abs/1902.10811

    2019 arXiv

  24. [32]

    Natural adversarial examples

    Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. Conference on Computer Vision and Pattern Recognition (CVPR) , 2021. https://arxiv. org/abs/1907.07174

    2021 arXiv

  25. [33]

    Learning robust global representations by penalizing local predictive power

    Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. In Advances in Neural Information Processing Systems (NeurIPS) ,

  26. [34]

    Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models

    Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Josh Tenenbaum, and Boris Katz. Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models. In Advances in Neural Information Processing Systems (N...

    2019

  27. [35]

    The many faces of robustness: A critical analysis of out-of-distribution generalization

    Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of out-of-distribution generalization. Inter...

    2021 arXiv

  28. [36]

    P´ eter B´ andi, Oscar Geessink, Quirine Manson, Marcory Van Dijk, Maschenka Balkenhol, Meyke Hermsen, Babak Ehteshami Bejnordi, Byungjae Lee, Kyunghyun Paeng, Aoxiao Zhong, Quanzheng Li, Farhad Ghazvinian Zanjani, Svitlana Zinger, Keisuke Fukuta, Daisuke Komura, Vlado Ovtchar...

    2019

  29. [37]

    Ehinger, Aude Oliva, and Antonio Torralba

    Jianxiong Xiao, James Hays, Krista A. Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition , pages 3485–3492, 2010. 13

    2010

  30. [38]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In Indian Conference on Computer Vision, Graphics and Image Processing , Dec 2008

    2008

  31. [39]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research , 21(140):1–67, 2020

    2020

  32. [40]

    Glue: A multi-task benchmark and analysis platform for natural language understanding

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461, 2018

    2018 arXiv

  33. [41]

    Automatically constructing a corpus of sentential paraphrases

    Bill Dolan and Chris Brockett. Automatically constructing a corpus of sentential paraphrases. In Proc. of IWP , 2005

    2005

  34. [42]

    The pascal recognising textual entailment challenge

    Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entailment challenge. In Machine Learning Challenges Workshop , 2005

    2005

  35. [43]

    The second pascal recognising textual entailment challenge

    Roy Bar-Haim, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, Bernardo Magnini, and Idan Szpektor. The second pascal recognising textual entailment challenge. In Proc. of the II PASCAL challenge, 2006

    2006

  36. [44]

    The third pascal recognizing textual entailment challenge

    Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. The third pascal recognizing textual entailment challenge. In Proc. of the ACL-PASCAL workshop on textual entailment and paraphrasing , 2007

    2007

  37. [45]

    The fifth pascal recognizing textual entailment challenge

    Luisa Bentivogli, Peter Clark, Ido Dagan, and Danilo Giampiccolo. The fifth pascal recognizing textual entailment challenge. In TAC, 2009

    2009

  38. [46]

    Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. Neural network acceptability judgments. TACL, 7:625–641, 2019

    2019

  39. [47]

    Recursive deep models for semantic compositionality over a sentiment treebank

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of EMNLP, 2013

    2013

  40. [48]

    Fine- tuning pretrained language models: Weight initializations, data orders, and early stopping

    Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah Smith. Fine- tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv preprint arXiv:2002.06305, 2020

    2002 arXiv

  41. [49]

    Comparison of the predicted and observed secondary structure of t4 phage lysozyme

    Brian W Matthews. Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure , 1975

    1975

  42. [50]

    BERT: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics (NAACL) , 2019

    2019

  43. [51]

    Zihang Dai, Guokun Lai, Yiming Yang, and Quoc V. Le. Funnel-transformer: filtering out sequential redundancy for efficient language processing. In Proceedings of the 34th International Conference on Neural Information Processing Systems , NIPS ’20, Red Hook, NY, USA, 2020. Cur...

    2020

  44. [52]

    Merging models with fisher-weighted averaging

    Michael S Matena and Colin A Raffel. Merging models with fisher-weighted averaging. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 17703–17716. Curran Associates, Inc., 2022

    2022

  45. [53]

    Christopher M. Bishop. Pattern Recognition and Machine Learning (Information Science and Statistics) . Springer-Verlag, Berlin, Heidelberg, 2006. 14

    2006

  46. [54]

    N M MX n=1 CX c=1 yc n

    Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. Model soups: averaging weights of multiple fine-tuned models improves accuracy without in...

    2022

  47. [2019]

    https://arxiv.org/abs/1905.13549

    1905 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.