Pith. sign in

REVIEW 2 major objections 6 minor 39 references

Beyond Overconfidence: Foundation Models Redefine Calibration in Deep Neural Networks

T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Current-generation image classifiers invert the classic overconfidence pattern: underconfident in-distribution, better calibrated under shift, but post-hoc calibration can backfire under severe shift.

desk verdict Solid, reproducible empirical study with interesting descriptive findings, but the central 'model generation redefines calibration' claim is confounded with pretraining scale and recipe, and the paper's own limitations concede this. read the letter →

arxiv 2506.09593 v1 pith:FVL3JONX submitted 2025-06-11 cs.LG

Achim Hekler , Lukas Kuhn , Florian Buettner This is my paper
classification cs.LG
keywords modelcalibrationexpectederrorunderconfidencedistributionshiftfoundationmodelspost-hocvisiontransformersbiomedicalimaging
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the newest generation of image classification models—ConvNeXt, BEiT, and EVA—reverses the long-standing overconfidence pattern. On ImageNet these models are consistently underconfident in their in-distribution predictions, and as distribution shift becomes more severe their calibration error decreases instead of increasing. That matters because it challenges the assumption that neural network uncertainty always degrades under shift, and because it implies post-hoc calibration, tuned in-distribution, can actually hurt under severe shift. The paper also shows these ImageNet findings do not transfer to biomedical transfer-learning tasks, where convolutional models remain better calibrated than transformers regardless of generation.

What carries the argument

The load-bearing object is the Expected Calibration Error (ECE), measured with 15 equal-mass bins, together with reliability diagrams that separate underconfidence from overconfidence. The mechanism the paper identifies is a confidence shift under distribution shift: shift tends to raise the model's confidence relative to its accuracy. For overconfident traditional models this makes an already bad pattern worse, so ECE grows monotonically; for underconfident current-generation models it partially compensates the initial underconfidence, so ECE shrinks. Post-hoc methods (temperature scaling, ensemble temperature scaling, isotonic regression, splines) are studied as the corrective re-centering of that confidence, which works in-distribution but fails when the shift is severe.

What would settle it

Train a ResNet-50 with ImageNet-22k pretraining and the same augmentation and regularization recipe used for ConvNeXt, then evaluate on ImageNet-C; if it becomes underconfident in-distribution and its ECE decreases with corruption severity, the proposed generation-level difference is a training-recipe effect, not an architectural one.

Watch

Extended reading notes

Core claim

The central discovery is a generation-level reversal in calibration behavior. Current-generation models such as ConvNeXt, BEiT, and EVA show an inverse relationship between accuracy and calibration on ImageNet: higher accuracy comes with higher Expected Calibration Error, driven by systematic underconfidence. Under distribution shift, their ECE falls as severity increases, while traditional models (ResNet, ViT, Swin) follow the classic monotonic degradation. Post-hoc calibration methods work very well for these models in-distribution—temperature scaling cuts ConvNeXt's ECE roughly sixfold—but their benefit shrinks with shift severity and can turn negative on ImageNet-A and at high ImageNet-C severities. On four biomedical datasets, this pattern disappears: convolutional architectures consistently achieve better calibration than transformer-based ones, and the ImageNet-derived robustness under shift is not reproduced, suggesting the effect is tied to training and pretraining methodology rather than architecture alone.

Load-bearing premise

The central narrative assumes that the six selected checkpoints differ by model generation, but the current-generation models also differ in pretraining scale, augmentation, and regularization, so if those training-condition differences rather than architecture drive the observed calibration behavior, the generation-level conclusion does not follow.

Editorial extensions

If this is right

  • On standard ImageNet-style benchmarks, practitioners should expect modern pretrained models to be underconfident rather than overconfident, and a single temperature parameter is enough to fix most of the in-distribution error.
  • The assumption that calibration always degrades under distribution shift is not universal; for current-generation models, ECE can improve under shift, so deployment decisions should measure actual calibration on the target shift rather than assume monotonic decay.
  • Post-hoc calibration tuned on in-distribution data should not be applied blindly under severe shift, because it can raise ECE above the uncalibrated baseline.
  • In biomedical transfer-learning settings, convolutional architectures (ResNet, ConvNeXt) offer better calibration than transformer-based counterparts, independent of model generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves the cause of underconfidence unresolved; an immediate test is whether the same underconfidence appears when a ResNet-50 is pretrained on ImageNet-22k and fine-tuned with the same augmentation and regularization as ConvNeXt, which would attribute the effect to training scale and recipe rather than architecture generation.
  • If underconfidence is a byproduct of modern pretraining and regularization, then training-time objectives could deliberately target calibration, potentially recovering both accuracy and well-calibrated confidence instead of trading them off.
  • The decreasing ECE under shift could partly reflect how accuracy collapse changes bin occupancies; a natural follow-up is to decompose ECE changes into confidence shifts versus accuracy drops and to report proper scoring rules like NLL or Brier under each severity level.
  • The biomedical finding suggests that architecture choice for calibrated deployment is more domain-dependent than the ImageNet story implies; an extension is to evaluate whether the underconfidence pattern returns once biomedical models are also pretrained at the same scale.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper presents an empirical study of the calibration properties of six ImageNet-trained model checkpoints (ResNet-50, ViT-B/16, Swin-S3-B, ConvNeXt-B, BEiT-B/16, EVA-S/14) on in-distribution ImageNet, 95 synthetically corrupted ImageNet-C datasets, two real-world shifts (ImageNet-V2 and ImageNet-A), and four biomedical imaging datasets. It reports that the three models it labels 'current-generation' (ConvNeXt-B, BEiT-B/16, EVA-S/14) exhibit underconfidence in-distribution, show decreasing ECE with increasing distribution shift, and respond well to post-hoc calibration in-distribution but progressively worse under severe shift, sometimes becoming counterproductive. It further reports that convolutional architectures achieve superior calibration over transformers in the biomedical transfer-learning setting, irrespective of model generation. The paper releases code, model outputs, and a benchmark repository.

Significance. If the descriptive measurements hold, the paper challenges the prevailing narrative from Minderer et al. and offers practically relevant observations about post-hoc calibration under shift. The empirical results appear internally consistent, with robustness checks across bin counts and multiple calibration metrics in the appendix. The public release of code, model outputs, and calibration benchmark is a notable strength, as is the honest discussion of limitations. However, the central causal attribution of the observed patterns to 'current-generation' model advancements is confounded with pretraining scale and training recipe, and the paper's own Discussion concedes that disentangling these factors is beyond its scope. As a descriptive study of specific checkpoints, the findings are useful; as an explanation of how model advancements 'redefine' calibration, the evidence is not yet sufficient.

major comments (2)
  1. [Model Selection and Training] The central claim of Contribution 1—that 'current-generation neural networks' exhibit systematic underconfidence in-distribution and reduced calibration error under distribution shift—is not supported by controlled comparisons because the model groups differ jointly in architecture, pretraining data scale, and training recipe. In §'Model Selection and Training', ConvNeXt-B, BEiT-B/16, and EVA-S/14 are all pretrained on ImageNet-22k and fine-tuned on ImageNet-1k, while ResNet-50 and Swin-S3-B are trained only on ImageNet-1k and ViT-B/16 on ImageNet-21k. The observed patterns could therefore be driven by pretraining scale, masked-image-modeling objectives, or modern augmentation/regularization rather than by architectural advancement as a 'generation'. The Discussion explicitly concedes that 'disentangling their individual contributions would require a comprehensive follow-up study,' but the title and abstract nevertheless attribute the effects to model advancements. To make the central claim defensible, the authors should either provide ablation experiments that vary pretraining and recipe within fixed architectures, or reframe the findings as descriptive measurements on these specific checkpoints and soften the generational language throughout.
  2. [Transfer Learning to Biomedical Tasks] The claim in Contribution 3 that 'convolutional architectures consistently achieve superior calibration compared to transformer-based counterparts, irrespective of model generation' is subject to the same confounding as the ImageNet analysis. In the biomedical experiments, the models are initialized with ImageNet-pretrained weights: ResNet-50 (ImageNet-1k), ConvNeXt-B (ImageNet-22k), ViT-B/16 (ImageNet-21k), and BEiT-B/16 (ImageNet-22k). Since ConvNeXt-B is both convolutional and 22k-pretrained, while BEiT-B/16 is transformer-based but also 22k-pretrained, the comparison does not isolate architectural inductive bias from pretraining scale or fine-tuning details. The statement 'irrespective of model generation' is therefore not established. A controlled comparison would require, for example, training the same architecture with different pretraining data, or comparing conv and transformer models with identical pretraining corpora and recipe. As presented, the biomedical conclusion is a useful descriptive observation but not a causal attribution.
minor comments (6)
  1. [Datasets] The text says 'employing a random 90/10 split for testing and post-hoc calibration optimization, respectively' and later 'we utilize the dedicated calibration split from the original in-distribution ImageNet dataset.' Please clarify whether the calibration set is a random 10% subset of the validation set or a separate predefined split, and confirm that no calibration images are included in the ImageNet-C evaluation.
  2. [Throughout] There are several typos and formatting errors: 'wether' (twice), 'Carlibration' in Methods, 'caliration' in Eq. (3), 'pevious' in Model Selection and Training, 'T emperature Scaling' and 'E nsemble Temperature Scaling' in Post-hoc Calibration Methods, 'practioners' in Discussion, 'pretrainind' in Discussion, 'mult-class' in Datasets, and '10 5' instead of '10^5' in the Datasets section.
  3. [Fig. 4] In Fig. 4, panel (a) uses green markers for current-generation models and red for traditional models, but the text for panel (b) refers to 'traditional models (green markers).' Please make the color mapping consistent across all panels and with the figure legend.
  4. [Appendix A1] The definition of Maximum Calibration Error as MCE = max_j (p_j - c_j) omits the absolute value; the standard MCE is max_j |p_j - c_j|. Given that the paper reports underconfidence, the signed difference can be negative, so the definition should be clarified or corrected.
  5. [Appendix A1] The RMSCE formula contains malformed LaTeX ('/radicaltp/radicalvertex/radicalvertex√') and should be typeset correctly; also the square root should cover the entire summation.
  6. [Fig. 6 and caption] In 'Transfer Learning to Biomedical Tasks,' the paper mixes 'dermatological' and 'dermoscopic' in reference to the HAM10000 dataset; please use one consistent term throughout.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's calibration measurements are external checkpoint evaluations against external benchmarks, and the post-hoc calibration fits do not feed the central ECE claims.

full rationale

The paper's central claims are empirical measurements of six externally released timm checkpoints (ConvNeXt-B, BEiT-B/16, EVA-S/14, Swin-S3-B, ViT-B/16, ResNet-50) evaluated on external benchmarks (ImageNet, ImageNet-C, ImageNet-V2, ImageNet-A, and four biomedical datasets). Expected Calibration Error is computed directly from model confidence and ground-truth labels via Eq. (3); no fitted parameter is renamed as a prediction. Post-hoc calibration methods (TS, ETS, IRM, SPL) are optimized only on a held-out in-distribution ImageNet calibration split and then applied unchanged to shifted test sets, so the reported distribution-shift ECE values are not fits to the shifted data. The biomedical fine-tuning hyperparameters are selected on training folds, while calibration methods use the canonical validation sets, and the central in-distribution and shift ECE comparisons rely on test data. Although the interpretation that 'current-generation' architecture causes underconfidence is confounded with pretraining scale and training recipe, the paper itself explicitly flags this in the Discussion as an unexplored attribution question; confounding is a correctness risk, not circularity. The two author-overlapping citations ([15], [39]) support calibration method implementations and estimator consistency, but they are not load-bearing premises from which the empirical results are derived. Therefore no step reduces by construction or by self-citation to its own inputs, and the honest finding is no significant circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claims rest on a small set of empirical assumptions: the ECE estimator captures calibration, the six checkpoints represent model generations, the chosen shifts represent real-world shifts, and the fine-tuning protocol yields comparable biomedical models. No new theoretical entities are introduced, and the only fitted quantities are the standard post-hoc calibration parameters, fit on held-out splits.

free parameters (1)
  • Post-hoc calibration parameters (TS temperature, ETS weights, IRM mapping, SPL spline parameters) = Not reported in text; available in code and Zenodo data
    These are standard calibration method parameters fitted on the held-out 10% in-distribution ImageNet split and on biomedical validation splits. They are not used to construct the central ECE measurements, but the post-hoc effectiveness claim depends on them being fit correctly.
assumptions (4)
  • domain assumption The ECE with 15 equal-mass bins and the top-label calibration criterion are valid measures of calibration.
    The paper defines ECE and uses it as the primary metric, while appendix checks with other bin counts and metrics. The definition is standard but the adequacy of the estimator is assumed.
  • ad hoc to paper The selected timm checkpoints represent their model generations and are directly comparable.
    The paper groups ConvNeXt, BEiT, and EVA as current-generation and ResNet, ViT, and Swin as traditional, but the groups differ in pretraining scale and recipe as well as architecture. This unproven design assumption is central to the causal narrative.
  • domain assumption ImageNet-C, ImageNet-V2, and ImageNet-A are representative of real-world distribution shifts.
    The paper uses these three shift families as proxies for distribution shift and does not test other shift types or domains for the main ImageNet conclusions.
  • domain assumption The two-stage transfer learning protocol with per-architecture hyperparameter tuning produces comparable biomedical models.
    All biomedical models are fine-tuned with the same general protocol, but hyperparameters are tuned per architecture, which could affect final calibration and accuracy in ways not controlled for.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond Overconfidence: Foundation Models Redefine Calibration in Deep Neural Networks." pith.science (2026). https://pith.science/paper/FVL3JONX

@misc{pith2026250609593,
  author       = {Pith},
  title        = {Pith review of: Beyond Overconfidence: Foundation Models Redefine Calibration in Deep Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FVL3JONX}},
  note         = {Machine review of arXiv:2506.09593}
}
read the original abstract

Reliable uncertainty calibration is essential for safely deploying deep neural networks in high-stakes applications. Deep neural networks are known to exhibit systematic overconfidence, especially under distribution shifts. Although foundation models such as ConvNeXt, EVA and BEiT have demonstrated significant improvements in predictive performance, their calibration properties remain underexplored. This paper presents a comprehensive investigation into the calibration behavior of foundation models, revealing insights that challenge established paradigms. Our empirical analysis shows that these models tend to be underconfident in in-distribution predictions, resulting in higher calibration errors, while demonstrating improved calibration under distribution shifts. Furthermore, we demonstrate that foundation models are highly responsive to post-hoc calibration techniques in the in-distribution setting, enabling practitioners to effectively mitigate underconfidence bias. However, these methods become progressively less reliable under severe distribution shifts and can occasionally produce counterproductive results. Our findings highlight the complex, non-monotonic effects of architectural and training innovations on calibration, challenging established narratives of continuous improvement.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 36 canonical work pages

  1. [1]

    Nature Machine Intelligence 1(1), 20–23 (2019)

    Begoli, E., Bhattacharya, T., Kusnezov, D.: The need for uncertainty quantifi- cation in machine-assisted medical decision making. Nature Machine Intelligence 1(1), 20–23 (2019)

    work page 2019

  2. [2]

    NPJ Digital Medicine4(1) (2021)

    Kompa, B., Snoek, J., Beam, A.L.: Second opinion needed: communicating uncertainty in medical machine learning. NPJ Digital Medicine4(1) (2021)

    work page 2021

  3. [3]

    IEEE Transactions on Intelligent Vehicles7(4), 849–862 (2022)

    Tang, X., Yang, K., Wang, H., Wu, J., Qin, Y., Yu, W., Cao, D.: Prediction- uncertainty-aware decision-making for autonomous vehicles. IEEE Transactions on Intelligent Vehicles7(4), 849–862 (2022)

    work page 2022

  4. [4]

    Neurocomputing576(2024)

    Blasco, T., S´ anchez, J.S., Garc´ ıa, V.: A survey on uncertainty quantification in deep learning for financial time series prediction. Neurocomputing576(2024)

    work page 2024

  5. [5]

    Proceedings of the AAAI Conference on Artificial Intelligence29(2015)

    Pakdaman Naeini, M., Cooper, G., Hauskrecht, M.: Obtaining well calibrated probabilities using bayesian binning. Proceedings of the AAAI Conference on Artificial Intelligence29(2015)

    work page 2015

  6. [6]

    International conference on machine learning, 1321–1330 (2017)

    Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q.: On calibration of modern neural networks. International conference on machine learning, 1321–1330 (2017)

    work page 2017

  7. [7]

    Advances in Neural Information Processing Systems32(2019)

    Thulasidasan, S., Chennupati, G., Bilmes, J.A., Bhattacharya, T., Michalak, S.: On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks. Advances in Neural Information Processing Systems32(2019)

    work page 2019

  8. [8]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 15262–15271 (2021)

    Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., Song, D.: Natural adversarial examples. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 15262–15271 (2021)

    work page 2021

Show all 39 references
  1. [9]

    Advances in Neural Information Processing Systems30(2017)

    Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in Neural Information Processing Systems30(2017)

    2017

  2. [10]

    Advances in Neural Information Processing Systems34, 20063–20075 (2021) 20

    Rahaman, R., Thiery, A.: Uncertainty Quantification and Deep Ensembles. Advances in Neural Information Processing Systems34, 20063–20075 (2021) 20

    2021

  3. [11]

    Advances in Neural Information Processing Systems, 13991–14002 (2019)

    Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J., Lakshminarayanan, B., Snoek, J.: Can you trust your model’s uncertainty? Eval- uating predictive uncertainty under dataset shift. Advances in Neural Information Processing Systems, 13991–14002 (2019)

    2019

  4. [12]

    International Conference on Learning Representations (2019)

    Hendrycks, D., Dietterich, T.: Benchmarking neural network robustness to common corruptions and perturbations. International Conference on Learning Representations (2019)

    2019

  5. [13]

    International conference on machine learning, 11117–11128 (2020)

    Zhang, J., Kailkhura, B., Han, T.Y.-J.: Mix-n-match: Ensemble and compo- sitional methods for uncertainty calibration in deep learning. International conference on machine learning, 11117–11128 (2020)

    2020

  6. [14]

    International Conference on Learning Representations (2021)

    Gupta, K., Rahimi, A., Ajanthan, T., Mensink, T., Sminchisescu, C., Hartley, R.: Calibration of neural networks using splines. International Conference on Learning Representations (2021)

    2021

  7. [15]

    European Conference on Computer Vision, 555–569 (2022)

    Tomani, C., Cremers, D., Buettner, F.: Parameterized temperature scaling for boosting the expressive power in post-hoc uncertainty calibration. European Conference on Computer Vision, 555–569 (2022)

    2022

  8. [16]

    Advances in Neural Information Processing Systems34, 15682–15694 (2021)

    Minderer, M., Djolonga, J., Romijnders, R., Hubis, F., Zhai, X., Houlsby, N., Tran, D., Lucic, M.: Revisiting the calibration of modern neural networks. Advances in Neural Information Processing Systems34, 15682–15694 (2021)

    2021

  9. [17]

    Conference on Computer Vision and Pattern Recognition, 11966–11976 (2022)

    Liu, Z., Mao, H., Wu, C., Feichtenhofer, C., Darrell, T., Xie, S.: A convnet for the 2020s. Conference on Computer Vision and Pattern Recognition, 11966–11976 (2022)

    2022

  10. [18]

    International Conference on Learning Representations (2022)

    Bao, H., Dong, L., Piao, S., Wei, F.: BEit: BERT pre-training of image transformers. International Conference on Learning Representations (2022)

    2022

  11. [19]

    Conference on Computer Vision and Pattern Recognition, 19358–19369 (2022)

    Fang, Y., Wang, W., Xie, B., Sun, Q.-S., Wu, L.Y., Wang, X., Huang, T., Wang, X., Cao, Y.: Eva: Exploring the limits of masked visual representation learning at scale. Conference on Computer Vision and Pattern Recognition, 19358–19369 (2022)

    2022

  12. [20]

    International Conference on Computer Vision, 9620–9629 (2021)

    Chen, X., Xie, S., He, K.: An Empirical Study of Training Self-Supervised Vision Transformers. International Conference on Computer Vision, 9620–9629 (2021)

    2021

  13. [21]

    International Conference on Computer Vision, 9630–9640 (2021)

    Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging Properties in Self-Supervised Vision Transformers. International Conference on Computer Vision, 9630–9640 (2021)

    2021

  14. [22]

    arXiv (2021)

    Xie, Z., Lin, Y., Yao, Z., Zhang, Z., Dai, Q., Cao, Y., Hu, H.: Self-Supervised Learning with Swin Transformers. arXiv (2021). https://doi.org/10.48550/arXiv. 2105.04553 21

    2021 doi arXiv

  15. [23]

    Advances in Neural Information Processing Systems33, 18613–18624 (2020)

    Cubuk, E.D., Zoph, B., Shlens, J., Le, Q.: RandAugment: Practical Auto- mated Data Augmentation with a Reduced Search Space. Advances in Neural Information Processing Systems33, 18613–18624 (2020)

    2020

  16. [24]

    International Conference on Computer Vision, 6022–6031 (2019)

    Yun, S., Han, D., Chun, S., Oh, S.J., Yoo, Y., Choe, J.: CutMix: Regulariza- tion Strategy to Train Strong Classifiers With Localizable Features. International Conference on Computer Vision, 6022–6031 (2019)

    2019

  17. [25]

    Proceedings of the AAAI Conference on Artificial Intelligence34(07), 13001–13008 (2020)

    Zhong, Z., Zheng, L., Kang, G., Li, S., Yang, Y.: Random Erasing Data Aug- mentation. Proceedings of the AAAI Conference on Artificial Intelligence34(07), 13001–13008 (2020)

    2020

  18. [26]

    International Conference on Computer Vision, 32–42 (2021)

    Touvron, H., Cord, M., Sablayrolles, A., Synnaeve, G., Jegou, H.: Going deeper with Image Transformers. International Conference on Computer Vision, 32–42 (2021)

    2021

  19. [27]

    Proceedings of the IEEE/CVF International Conference on Computer Vision, 10012–10022 (2021)

    Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, 10012–10022 (2021)

    2021

  20. [28]

    International Conference on Learning Representations (2021)

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. International Conference on Learn...

    2021

  21. [29]

    Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778 (2016)

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778 (2016)

    2016

  22. [30]

    Conference on Computer Vision and Pattern Recognition, 248–255 (2009)

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large- scale hierarchical image database. Conference on Computer Vision and Pattern Recognition, 248–255 (2009)

    2009

  23. [31]

    5389–5400 (2019)

    Recht, B., Roelofs, R., Schmidt, L., Shankar, V.: Do imagenet classifiers gen- eralize to imagenet? In: International Conference on Machine Learning, pp. 5389–5400 (2019). PMLR

    2019

  24. [32]

    Data in brief (2020)

    Al-Dhabyani, W., Gomaa, M., Khaled, H., Fahmy, A.: Dataset of breast ultrasound images. Data in brief (2020)

    2020

  25. [33]

    Scientific data5(1), 1–9 (2018)

    Tschandl, P., Rosendahl, C., Kittler, H.: The HAM10000 dataset, a large col- lection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data5(1), 1–9 (2018)

    2018

  26. [34]

    cell172(5), 1122–1131 (2018)

    Kermany, D.S., Goldbaum, M., Cai, W., Valentim, C.C., Liang, H., Baxter, S.L., McKeown, A., Yang, G., Wu, X., Yan, F.,et al.: Identifying medical diagnoses and 22 treatable diseases by image-based deep learning. cell172(5), 1122–1131 (2018)

    2018

  27. [35]

    Scientific Data10(1) (2023)

    Yang, J., Shi, R., Wei, D., Liu, Z., Zhao, L., Ke, B., Pfister, H., Ni, B.: Medm- nist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data10(1) (2023)

    2023

  28. [36]

    International Conference on Learning Representations (2022)

    Kumar, A., Raghunathan, A., Jones, R.M., Ma, T., Liang, P.: Fine-tuning can distort pretrained features and underperform out-of-distribution. International Conference on Learning Representations (2022)

    2022

  29. [37]

    https://arxiv.org/abs/2406.17536

    Salvo, F.D., Doerrich, S., Ledig, C.: MedMNIST-C: Comprehensive benchmark and improved classifier robustness by simulating realistic image corruptions (2024). https://arxiv.org/abs/2406.17536

    2024 arXiv

  30. [38]

    Journal of Applied Meteorology and Climatology12(4), 595–600 (1973)

    Murphy, A.H.: A new vector partition of the probability score. Journal of Applied Meteorology and Climatology12(4), 595–600 (1973)

    1973

  31. [39]

    International Conference on Artificial Intelligence and Statistics, 3466–3474 (2024) 23 Supplementary Material Appendix A: Further Experimental Results A1

    Popordanoska, T., Gruber, S.G., Tiulpin, A., Buettner, F., Blaschko, M.B.: Consistent and asymptotically unbiased estimation of proper calibration errors. International Conference on Artificial Intelligence and Statistics, 3466–3474 (2024) 23 Supplementary Material Appendix A:...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.