Pith. sign in

REVIEW 4 major objections 5 minor 31 references

MedBayes-Lite: A Clinical Uncertainty Governance Layer for Risk-Aware Medical Decision Support

T0 review · 4 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MedBayes-Lite claims a retraining-free uncertainty layer that cuts clinical LLM overconfidence by 32–48% and flags uncertain cases for human review.

desk verdict The paper's central theoretical claim collapses under the law of total variance, and the experimental reporting has enough internal contradictions that the headline safety numbers can't be trusted. read the letter →

arxiv 2511.16625 v3 pith:MWAOPVGJ submitted 2025-11-20 cs.AI

Elias Hossain , Md Mehedi Hasan Nipu , Maleeha Sheikh , Tasfia Nuzhat , Rajib Rana , Subash Neupane , Björn W. Schuller , Niloofar Yousefi This is my paper · ORCID
classification cs.AI
keywords clinicaluncertaintyquantificationMonteCarlodropoutmedicalquestionansweringcalibrationabstentionoverconfidencelayer-wisevariancedecompositiontransformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a clinical language model can be equipped with uncertainty awareness after training, by adding a lightweight governance layer rather than retraining. The layer runs stochastic forward passes with dropout, downweights unreliable tokens in attention, and abstains when predictive entropy is high. If the empirical claims hold, a drop-in wrapper would cut overconfident wrong answers in medical QA—especially high-severity cases—while adding under 3% parameters and less than 10% inference latency. The paper's theoretical centerpiece is a claimed layer-wise variance decomposition for transformers, which is supposed to make uncertainty propagation interpretable and principled. The authors flag limits in §6.3 (extreme distribution shift, all-noisy inputs, excessive dropout) and note in the abstract that the method does not improve risk-coverage ranking.

What carries the argument

Three coupled mechanisms carry the argument: (1) Bayesian Embedding Calibration—M MC-dropout forward passes produce an embedding mean and variance, treating dropout as a variational posterior; (2) Uncertainty-Weighted Attention—each attention weight α_ij is multiplied by exp(-λU(x_j)) and renormalized so high-variance tokens contribute less; (3) Confidence-Guided Decision Shaping—entropy-based confidence C(p)=1-H(p)/logK gates predictions, deferring anything below threshold τ. The theorem that is meant to unify them is Theorem 5's layer-wise variance decomposition, which is offered as the 'theoretical backbone' and the basis for interpretable, layer-level uncertainty attribution.

What would settle it

Run many stochastic forward passes on a fixed medical QA input and compute the empirical total variance of the prediction. Separately, using the same passes, compute each layer's conditional variance and conditional-mean variance and sum them as in Theorem 5. If the two numbers disagree beyond sampling error, the decomposition is not exact; a similar check should replace the theorem's Gaussian noise with the actual Bernoulli dropout masks used in Algorithm 1.

Watch

Extended reading notes

Core claim

MedBayes-Lite's central claim is that uncertainty can be made a first-class part of transformer inference without retraining: MC dropout converts each token embedding into a distribution, attention weights are renormalized with an uncertainty penalty exp(-λU(x_j)), and predictions are accepted only when normalized confidence C(p)=1-H(p)/logK clears a threshold τ. The paper further claims Theorem 5, the first closed-form hierarchical variance decomposition for transformer architectures, decomposing Var[ŷ] into a sum over layers of per-layer aleatoric and epistemic terms, and thus offering token- and layer-level uncertainty maps. On MedMCQA and MedQA-USMLE, the paper reports ECE reductions of

Load-bearing premise

The load-bearing assumption is that layer-by-layer variance really adds up to the model's total uncertainty; the paper's own implementation adds an extra attention term and uses dropout masks where the proof uses Gaussian noise, so the identity is assumed rather than demonstrated.

Editorial extensions

If this is right

  • If MedBayes-Lite works as described, deploying it requires only inference-time changes: no retraining, no new trainable parameters, under 3% parameter overhead and roughly linear MC-sample latency.
  • Confident high-severity errors, the most dangerous failure mode, fall from about 21% to near zero when moving from MedMCQA to MedQA-USMLE; ECE drops by 0.23–0.33 and calibration drift roughly halves.
  • The Clinical Uncertainty Score (CUS) could serve as a risk-aware calibration audit metric, since the paper reports it is strongly correlated (r≈0.88) with harmful overconfidence.
  • The method reports consistent reliability gains across model families (BERT variants, GPT, LLaMA) and prompt styles, including chain-of-thought, which suggests it is model-agnostic rather than tied to a single architecture.
  • Because the framework abstains rather than guesses, it gives clinical workflows a concrete 'review required' signal, the computational analogue of deferring to human judgment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If Theorem 5's identity survives numerical checking, the same layer-wise variance tracing could be reused as a model-auditing tool independent of MedBayes-Lite—for instance, to locate which layers inject the most uncertainty in any transformer.
  • The most conservative reading is that the empirical gains might persist even if the theorem fails, because MC dropout plus entropy-thresholded abstention are known to improve calibration on their own; the theorem's role would then be explanatory rather than load-bearing.
  • A testable extension is to combine the layer with temperature scaling: the paper reports no improvement in risk-coverage ranking, so a two-stage pipeline—Bayesian abstention for safety, temperature scaling for ranking—could recover both benefits.
  • The reported CUS correlation of 0.88 needs validation outside QA benchmarks, e.g., on real clinical notes with severity labels, before using CUS as a safety monitor in practice.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes MedBayes-Lite, a retraining-free inference-time uncertainty layer for transformer-based clinical language models that combines MC dropout, uncertainty-weighted attention (softmax reweighting by exp(-λU(x_j))), and entropy-based confidence abstention. It claims a 'first closed-form hierarchical variance decomposition for transformer architectures' (Theorem 5), two new clinically motivated metrics (CUS, ZTI), and large empirical gains: 32–48% overconfidence reduction, ECE reductions of 0.23–0.33, and prevention of up to 41% of diagnostic errors on biomedical QA and clinical prediction benchmarks (MedQA, PubMedQA, MIMIC-III in the body; MedMCQA and MedQA-USMLE in the abstract). The framework is evaluated against post-hoc calibration, deep ensembles, and SWAG, with additional cross-dataset and prompt-sensitivity experiments.

Significance. If the central claims were correct, a lightweight, retraining-free uncertainty layer for clinical LLMs would be practically valuable, and the layer-wise variance decomposition would be a meaningful theoretical contribution. The paper also has strengths in scope: it targets an important safety problem, proposes explicit abstention behavior, and includes a computational profiling table. However, the theoretical backbone (Theorem 5) is not a valid variance decomposition, and the empirical reporting is internally inconsistent across the abstract, the main text, and the tables. The new CUS/ZTI metrics are defined in the paper but are not validated against any external ground truth, and their interpretation shifts between sections. These issues are load-bearing: the headline contribution and headline numbers are not established.

major comments (4)
  1. [§3.3.2, Theorem 5 and Algorithm 1] Theorem 5, described as the 'theoretical backbone,' is not a valid layer-wise decomposition of predictive variance. Iterating the law of total variance requires conditioning on the full history of latent states. The expression Σ_l E[Var_e(l)(ŷ|h^(l-1))] + Var[E_e(l)(ŷ|h^(l-1))] is not the recursive decomposition Var[ŷ] = E[Var(ŷ|h^(L))] + Σ_l E[Var(E[ŷ|h^(l)] | h^(l-1))]. As written, each term Var_e(l)(ŷ|h^(l-1)) is the full variance of the final prediction with respect to perturbation at layer l, including all downstream stochastic layers, so the sum double-counts downstream variance. A minimal counterexample with two independent additive Gaussian layers gives the true variance σ1²+σ2² while the theorem yields 2(σ1²+σ2²). Additionally, e(l)~N(0, σ_l² I) is inconsistent with the Bernoulli MC dropout masks used in Algorithm 1, and Algorithm 1 introduces a third term E[Var(ŷ|α)] that is ab
  2. [Abstract vs. §4–§5 and Tables 6–11] The abstract reports evaluation on MedMCQA and MedQA-USMLE with ECE reductions of 0.23–0.33 and a reduction of confident high-severity errors from ~21% to near zero. The full text evaluates PubMedQA, MedQA, and MIMIC-III, and no table reports the headline 'confident high-severity errors' or the specific ECE reductions in the abstract. The CUS/ZTI tables report different quantities, and the ECE improvements shown (e.g., Table 6: 0.182→0.113; Table 8: 0.0698→0.0456) are far smaller than 0.23–0.33. This discrepancy prevents verification of the paper's central empirical claims.
  3. [§3.4 vs. Table 11] Section 3.4 states that with M=10 samples MedBayes-Lite incurs 'under a 10% increase in inference latency,' but Table 11 reports baseline ClinicalBERT latency 32.04 ms and MedBayes-Lite (MC=10) latency 60.13 ms, an ~88% increase. Even MC=5 gives 32.61 ms, only 1.8% overhead, while the later ablation recommends M=10–20. The stated '<10% increase' claim is contradicted by the paper's own efficiency table, and this affects the practical feasibility argument for clinical deployment.
  4. [§4.2.2, §5.3, and abstract] The two headline safety metrics CUS and ZTI are introduced in this paper and are not validated against any external or independently established measure of clinical harm. The severity weights w_clinical(b) in CUS are never specified, and the abstract's claim that CUS 'strongly correlates with harmful overconfidence (r≈0.88)' is not substantiated by any table or analysis in the main text. Moreover, the ablation study in §5.3 describes increasing CUS (with dropout) as 'intentional conservatism,' but CUS is defined as a lower-is-better risk-aware calibration error; the interpretation is thus inconsistent. These issues undermine the safety-oriented conclusions that depend on CUS and ZTI.
minor comments (5)
  1. [Throughout] There are multiple typographical errors and inconsistent labels: 'SW AG' appears in §4.4 and Table 11, 'V AE' in §3.2.1, and 'MMedBayes-Lite' in §5.4.2. These should be corrected.
  2. [§4.3, Table 5] For generative models, temperature is defined as np.random.uniform(0.7, 1.2), which is a random draw rather than a fixed configuration. This makes the experimental setup ambiguous and hampers reproducibility.
  3. [Appendix A.1] The appendix reports total runtime as 'approximately XX GPU hours,' which is a placeholder. This should either be filled with the actual number or removed.
  4. [References] References [14] and [31] are duplicated (both are Guo et al., 'On calibration of modern neural networks'). The reference list should be deduplicated.
  5. [§2.3] The discussion of 'reject-option classifiers and selective prediction frameworks' cites Geifman and El-Yaniv [22] and Jardine et al. [23] but does not connect them to the specific implementation of entropy-based abstention, which would help position the contribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the empirical calibration claims are benchmarked externally, and the theoretical issues with Theorem 5 are validity concerns, not reductions to the paper's own inputs.

full rationale

Walking the claimed derivation chain: Theorem 1 is explicitly a restatement of Gal & Ghahramani's external MC-dropout result; Theorems 2 and 3 are explicit definitions of the attention reweighting and entropy-based abstention rule; Theorem 4 is the standard law-of-total-variance identity. Theorem 5 is presented as an extension whose proof relies only on the law of total variance and not on the paper's own fitted values or self-citations. Even if Theorem 5 is mathematically unsound—e.g., because summing layer-conditional variances of the final prediction double-counts downstream variance, because Gaussian per-layer perturbations do not match the Bernoulli dropout implementation, and because Algorithm 1 introduces an extra E[Var(y|α)] term—that is a correctness or derivation defect, not a circular reduction: the alleged conclusion is not identical, by construction, to its premises. The headline empirical quantities (ECE reductions, NLL, confident high-severity error reduction) are measured against standard external benchmarks. The newly introduced CUS and ZTI metrics are defined scoring rules rather than fitted parameters renamed as predictions; their limited external validation is a generalizability weakness, not equivalence-by-construction. No load-bearing self-citation chain is used: references [4] and [5] are background clinical-LLM citations, and the MC-dropout foundation is an external, code-reproduced result. Therefore no circular step can be substantiated with a quotation and a specific reduction, and the appropriate circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 2 invented entities

The paper's parameters are incomplete (λ, τ, severity weights) or tuned on evaluation data (dropout p, MC samples M). Its theoretical framework relies on an invalid layer-wise variance decomposition and an unverified inference-time attention modification. The CUS and ZTI metrics are author-defined and used as evidence for the authors' own method, with no independent validation.

free parameters (5)
  • λ (uncertainty penalty coefficient)
    Introduced in §3.2.2 for uncertainty-weighted attention; no value or tuning protocol is given.
  • τ (abstention threshold)
    Entropy confidence gate in §3.2.3; threshold is never specified and no sensitivity analysis is reported.
  • MC dropout probability p = 0.3
    Tuned in ablation §5.3; p=0.3 is selected as the best trade-off between stability and uncertainty expressiveness.
  • Number of MC samples M = 10–20
    Tuned in ablation §5.3; M=10–20 chosen as the balance between calibration and runtime.
  • Clinical severity weights w_clinical(b) in CUS
    Defines CUS in §4.2.2; weights assigned to confidence bins are never described, so the metric's output is underdetermined.
assumptions (5)
  • standard math MC dropout approximates variational inference over transformer parameters (Gal & Ghahramani)
    Invoked as Theorem 1 and used to interpret embedding variances as epistemic uncertainty.
  • ad hoc to paper Law of total variance can be applied iteratively to transformer layers to produce a valid variance decomposition
    Used in Theorem 5; not valid as stated because conditioning on single-layer h(l-1) ignores the full stochastic path, and the proof assumes Gaussian per-layer noise inconsistent with dropout.
  • ad hoc to paper Modifying attention scores by exp(-λU(x_j)) at inference without retraining preserves the model's predictive function and yields the reported improvements
    No architectural code or evidence; the paper claims no rewiring but changes attention.
  • domain assumption Entropy-based confidence thresholding C(p)=1-H(p)/logK is an appropriate clinical risk-minimization gate
    Selective prediction is standard, but the specific threshold and its safety properties are asserted without a domain-risk model.
  • domain assumption The reported datasets are processed correctly despite no data code or splits; e.g. MIMIC-III mortality labels and fine-tuning details
    No data processing code or provenance checks; several model/task combinations are described only qualitatively.
invented entities (2)
  • Clinical Uncertainty Score (CUS)
    purpose: Weighted calibration error with clinical severity weights; used as the primary safety metric.
    Defined in §4.2.2; severity weights w_clinical(b) are unspecified and no external clinician validation is provided.
  • Zero-Shot Trustworthiness Index (ZTI)
    purpose: Harmonic mean of coverage and accuracy on the confident set; used to claim reliability improvements.
    Defined in §4.2.2; despite the name, it does not measure zero-shot trustworthiness beyond the authors' threshold and data; no external benchmark.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MedBayes-Lite: A Clinical Uncertainty Governance Layer for Risk-Aware Medical Decision Support." pith.science (2026). https://pith.science/paper/MWAOPVGJ

@misc{pith2026251116625,
  author       = {Pith},
  title        = {Pith review of: MedBayes-Lite: A Clinical Uncertainty Governance Layer for Risk-Aware Medical Decision Support},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MWAOPVGJ}},
  note         = {Machine review of arXiv:2511.16625}
}
read the original abstract

Clinical language models often assign high confidence to incorrect predictions, particularly in high-severity and out-of-distribution cases. We present MedBayes-Lite, a retraining-free uncertainty governance layer for transformer-based clinical predictors. It combines Monte Carlo dropout, predictive calibration, and confidence-guided abstention to defer low-confidence predictions for human review, adding no trainable parameters. Evaluated on MedMCQA and MedQA-USMLE, MedBayes-Lite reduces expected calibration error by 0.23 to 0.33 and drives harmful overconfident errors (confident, incorrect, high-severity predictions) toward zero. Under domain shift from MedMCQA to MedQA-USMLE, it reduces confident high-severity errors from about 21% to near zero while roughly halving calibration drift. We also introduce the Clinical Uncertainty Score (CUS), which strongly correlates with harmful overconfidence (r approximately 0.88). Although the framework does not improve risk-coverage ranking, and temperature scaling or deep ensembles may provide advantages in calibration cost or risk ranking, MedBayes-Lite offers a practical calibration-and-abstention layer that reduces confident high-severity errors in clinical question-answering benchmarks.

Figures

Figures reproduced from arXiv: 2511.16625 by the authors.

Figure 1
Figure 1. Overview of the proposed MedBayes-Lite framework. The model integrates Bayesian reasoning across embedding, attention, and decision layers to estimate both epistemic and aleatoric uncertainty. It combines uncertainty-weighted attention, adaptive evidence scoring, and confidence-guided decision shaping to enable efficient and risk-aware clinical inference. 4 Experimental Settings Having introduced the conceptual desi… view at source ↗
Figure 2
Figure 2. Comparison of calibration metrics (ECE, NLL, CUS, and ZTI) across methods. Each color [PITH_FULL_IMAGE:figures/full_fig_p018_2.png] view at source ↗
Figure 3
Figure 3. Ablation analysis of MC Dropout ratios on ClinicalBERT (MedQA). Increasing dropout [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Effect of varying MC sample counts on calibration and uncertainty metrics for MedBayes-Lite [PITH_FULL_IMAGE:figures/full_fig_p020_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 7 linked inside Pith

  1. [1]

    Large language models in biomedicine and health: Current research landscape and future directions,

    Z. Lu, Y . Peng, T. Cohen, M. Ghassemi, C. Weng, and S. Tian, “Large language models in biomedicine and health: Current research landscape and future directions,”Journal of the American Medical Informatics Association, vol. 31, no. 9, pp. 1801–1811, 2024

    2024

  2. [2]

    Large language models (LLM) and chatgpt: what will the impact on nuclear medicine be?

    I. L. Alberts, L. Mercolli, T. Pyka, G. Prenosil, K. Shi, A. Rominger, and A. Afshar-Oromieh, “Large language models (LLM) and chatgpt: what will the impact on nuclear medicine be?”European Journal of Nuclear Medicine and Molecular Imaging, vol. 50, no. 6, pp. 1549–1552, 2023

    2023

  3. [3]

    Large language models in medicine,

    A. J. Thirunavukarasu, D. S. J. Ting, K. Elangovan, L. Gutierrez, T. F. Tan, and D. S. W. Ting, “Large language models in medicine,”Nature Medicine, vol. 29, no. 8, pp. 1930–1940, 2023

    1930

  4. [4]

    Medinsight: A multi-source context augmentation framework for generating patient-centric medical responses using large language models,

    S. Neupane, S. Mitra, S. Mittal, M. Gaur, N. A. Golilarz, S. Rahimi, and A. Amirlatifi, “Medinsight: A multi-source context augmentation framework for generating patient-centric medical responses using large language models,”ACM Transactions on Computing for Healthcare, vol. 6, no. 2, pp. 1–19, 2025

    2025

  5. [5]

    Clinicsum: Utilizing language models for generating clinical summaries from patient-doctor conversations,

    S. Neupane, H. Tripathi, S. Mitra, S. Bozorgzad, S. Mittal, S. Rahimi, and A. Amirlatifi, “Clinicsum: Utilizing language models for generating clinical summaries from patient-doctor conversations,” in 2024 IEEE International Conference on Big Data (BigData). IEEE, 2024, pp. 5050–5059

    2024

  6. [6]

    Can large language models reason about medical questions?

    V . Li´evin, C. E. Hother, A. G. Motzfeldt, and O. Winther, “Can large language models reason about medical questions?”Patterns, vol. 5, no. 3, 2024

    2024

  7. [7]

    Large language model uncertainty measurement and calibration for medical diagnosis and treatment,

    T. Savage, J. Wang, R. Gallo, A. Boukil, V . Patel, S. A. Ahmad Safavi-Naini, A. Soroush, and J. H. Chen, “Large language model uncertainty measurement and calibration for medical diagnosis and treatment,”medRxiv, pp. 2024–06, 2024

    2024

  8. [8]

    Trustworthy AI guidelines in biomedical decision-making applications: A scoping review,

    M. Mora-Cantallops, E. Garc ´ıa-Barriocanal, and M.- ´A. Sicilia, “Trustworthy AI guidelines in biomedical decision-making applications: A scoping review,”Big Data and Cognitive Computing, vol. 8, no. 7, p. 73, 2024

    2024

Show all 31 references
  1. [9]

    Exploring the use of large language models in risk analysis: Opportunities and challenges,

    Z. ˇSimi´c, “Exploring the use of large language models in risk analysis: Opportunities and challenges,” 2024

    2024

  2. [10]

    Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods,

    E. H¨ullermeier and W. Waegeman, “Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods,”Machine Learning, vol. 110, no. 3, pp. 457–506, 2021

    2021

  3. [11]

    A review of uncertainty quantification in deep learning: Techniques, applications and challenges,

    M. Abdar, F. Pourpanah, S. Hussain, D. Rezazadegan, L. Liu, M. Ghavamzadeh, P. Fieguth, X. Cao, A. Khosravi, U. R. Acharyaet al., “A review of uncertainty quantification in deep learning: Techniques, applications and challenges,”Information Fusion, vol. 76, pp. 243–297, 2021

    2021

  4. [12]

    Explainability and uncertainty: Two sides of the same coin for enhancing the interpretability of deep learning models in healthcare,

    M. Salvi, S. Seoni, A. Campagner, A. Gertych, U. R. Acharya, F. Molinari, and F. Cabitza, “Explainability and uncertainty: Two sides of the same coin for enhancing the interpretability of deep learning models in healthcare,”International Journal of Medical Informatics, p. 105846, 2025

    2025

  5. [13]

    Uncertainty quantification in large language models through convex hull analysis,

    F. O. Catak and M. Kuzlu, “Uncertainty quantification in large language models through convex hull analysis,”Discover Artificial Intelligence, vol. 4, no. 1, pp. 1–14, 2024

    2024

  6. [14]

    On calibration of modern neural networks,

    C. Guo, G. Pleiss, Y . Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” in International Conference on Machine Learning. PMLR, 2017, pp. 1321–1330

    2017

  7. [15]

    Benchmarking LLMs via uncertainty quantification,

    F. Ye, M. Yang, J. Pang, L. Wang, D. Wong, E. Yilmaz, S. Shi, and Z. Tu, “Benchmarking LLMs via uncertainty quantification,”Advances in Neural Information Processing Systems, vol. 37, pp. 15 356–15 385, 2024. 27

    2024

  8. [16]

    Dropout as a bayesian approximation: Representing model uncertainty in deep learning,

    Y . Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” inInternational Conference on Machine Learning. PMLR, 2016, pp. 1050–1059

    2016

  9. [17]

    Uncertainty-aware health diagnostics via class- balanced evidential deep learning,

    T. Xia, T. Dang, J. Han, L. Qendro, and C. Mascolo, “Uncertainty-aware health diagnostics via class- balanced evidential deep learning,”IEEE Journal of Biomedical and Health Informatics, vol. 28, no. 11, pp. 6417–6428, 2024. [Online]. Available: https://doi.org/10.1109/JBHI.20...

    2024

  10. [18]

    Being bayesian about categorical probability,

    A. Kristiadiet al., “Being bayesian about categorical probability,”arXiv preprint arXiv:2002.07965, 2020

    2002 arXiv

  11. [19]

    Uncertainty-aware attention for reliable segmentation and interpretation,

    Y . Heet al., “Uncertainty-aware attention for reliable segmentation and interpretation,”IEEE Transactions on Medical Imaging, vol. 39, no. 2, 2020

    2020

  12. [20]

    Predicting good probabilities with supervised learning,

    A. Niculescu-Mizil and R. Caruana, “Predicting good probabilities with supervised learning,” in Proceedings of the 22nd International Conference on Machine Learning, 2005, pp. 625–632

    2005

  13. [21]

    MAQA: Evaluating uncertainty quantification in LLMs regarding data uncertainty,

    Y . Yang, H. Yoo, and H. Lee, “MAQA: Evaluating uncertainty quantification in LLMs regarding data uncertainty,”arXiv preprint arXiv:2408.06816, 2024

    2024 arXiv

  14. [22]

    Selective classification for deep neural networks,

    Y . Geifman and R. El-Yaniv, “Selective classification for deep neural networks,” inAdvances in Neural Information Processing Systems, vol. 30, 2017

    2017

  15. [23]

    Learning to abstain in healthcare: A survey,

    M. Jardineet al., “Learning to abstain in healthcare: A survey,”Journal of Biomedical Informatics, 2023

    2023

  16. [24]

    Simple and scalable predictive uncertainty estimation using deep ensembles,

    B. Lakshminarayanan, A. Pritzel, and C. Blundell, “Simple and scalable predictive uncertainty estimation using deep ensembles,” inAdvances in Neural Information Processing Systems, vol. 30, 2017

    2017

  17. [25]

    A simple baseline for bayesian uncertainty in deep learning,

    W. Maddox, T. Garipov, P. Izmailov, D. Vetrov, and A. G. Wilson, “A simple baseline for bayesian uncertainty in deep learning,” 2019. [Online]. Available: https://arxiv.org/abs/1902.02476

    2019 arXiv

  18. [26]

    Bayesian BERT: A probabilistic framework for language understanding,

    Z. Xiaoet al., “Bayesian BERT: A probabilistic framework for language understanding,”arXiv preprint arXiv:2106.10314, 2021

    2021 arXiv

  19. [27]

    Pubmedqa: A dataset for biomedical research question answering,

    Q. Jin, B. Dhingra, Z. Liu, W. Cohen, and X. Lu, “Pubmedqa: A dataset for biomedical research question answering,” inProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing...

    2019

  20. [28]

    What disease does this patient have? a large-scale open domain question answering dataset from medical exams,

    D. Jin, E. Pan, N. Oufattole, W.-H. Weng, H. Fang, and P. Szolovits, “What disease does this patient have? a large-scale open domain question answering dataset from medical exams,”arXiv preprint arXiv:2009.13081, 2020

    2009 arXiv

  21. [29]

    MIMIC-III, a freely accessible critical care database,

    A. E. Johnson, T. J. Pollard, L. Shen, L.-w. H. Lehman, M. Feng, M. Ghassemi, B. Moody, P. Szolovits, L. A. Celi, and R. G. Mark, “MIMIC-III, a freely accessible critical care database,” Scientific Data, vol. 3, no. 1, pp. 1–9, 2016

    2016

  22. [30]

    A baseline for detecting misclassified and out-of-distribution examples in neural networks,

    D. Hendrycks and K. Gimpel, “A baseline for detecting misclassified and out-of-distribution examples in neural networks,” 2018. [Online]. Available: https://arxiv.org/abs/1610.02136

    2018 arXiv

  23. [31]

    On calibration of modern neural networks,

    C. Guo, G. Pleiss, Y . Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” 2017. [Online]. Available: https://arxiv.org/abs/1706.04599 28 Appendix A.1 Compute Resource and Complexity Analysis All experiments were performed on Google Colab, a cloud-based comp...

    2017 arXiv

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.