Pith. sign in

REVIEW 4 major objections 5 minor 47 references

Rate-In: Information-Driven Adaptive Dropout Rates for Improved Inference-Time Uncertainty Estimation

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Rate-In dynamically adjusts each layer's dropout rate at inference time by tracking mutual-information loss, yielding better-calibrated and sharper uncertainty estimates without retraining or labels.

desk verdict Per-input adaptive MC dropout is a plausible new idea, but the experiments don't yet separate adaptivity from simply choosing different rates; still worth refereeing. read the letter →

arxiv 2412.07169 v4 pith:Z3ZEWN7K submitted 2024-12-10 cs.LG cs.CVstat.ML

Tal Zeevi , Ravid Shwartz-Ziv , Yann LeCun , Lawrence H. Staib , John A. Onofrey This is my paper · ORCID
classification cs.LGcs.CVstat.ML
keywords MonteCarloDropoutadaptiveratesinference-timeuncertaintyestimationmutualinformationmedicalimagesegmentationcalibrationout-of-distributionrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Rate-In claims that the reason Monte Carlo Dropout gives blurry or mis-calibrated uncertainty is that dropout rates are frozen at training-time values, applied uniformly to every layer and every input. The paper proposes to fix this at inference time by measuring, for each layer, how much mutual information is lost between the pre- and post-dropout feature maps, and then raising or lowering that layer's dropout rate until the loss sits below a threshold. This is done per input and per layer, with no labels and no retraining. The paper reports that the resulting uncertainty estimates are better calibrated and more sharply localized, especially along tissue boundaries in medical segmentation, while predictive accuracy is preserved or improved.

What carries the argument

The machinery is a mutual-information feedback loop between a layer's input and its post-dropout feature maps, using an adaptive-binning estimator to keep mutual information computable in high dimensions. Mutual information plays the role of a per-layer information budget: the information loss, defined as the relative drop in mutual information caused by dropout, is compared to a threshold epsilon, and the dropout rate is stepped until the loss falls within delta of that threshold. This turns dropout-rate selection into a local, label-free, input-specific optimization problem.

What would settle it

Compute Rate-In on a fixed test set, then replace the selected per-layer dropout rates with random rates matched to have the same mean information loss; if calibration and accuracy-rejection metrics remain unchanged, the per-layer mutual information signal is not what drives the gains.

Watch

Extended reading notes

Core claim

The central discovery is that dropout-induced information loss, measured as the relative drop in mutual information between a layer's input and its post-dropout output, can serve as a per-layer control signal for setting inference-time dropout rates. Rate-In runs a feedback loop: apply dropout at the current rate, estimate mutual information with an adaptive binning estimator, compare the loss to a threshold epsilon, and adjust the rate up or down until convergence, then move to the next layer. Because this happens layer-by-layer on the actual test input, the final dropout rates differ across layers and across inputs. The paper argues this converts dropout from uniform impulse noise into task-aware contextual noise, and demonstrates on medical imaging benchmarks that calibration, accuracy-rejection trade-offs, and boundary-focused uncertainty improve without sacrificing segmentation Dice or classification accuracy.

Load-bearing premise

The argument depends on the assumption that the drop in shared information between a layer's input and its post-dropout output is exactly the quantity that must be controlled to keep predictive uncertainty well calibrated, and that a single threshold equal to the baseline dropout rate is the right budget for every layer and input.

Editorial extensions

If this is right

  • MC Dropout becomes input-adaptive: each test sample receives its own per-layer dropout schedule rather than a uniform fixed rate.
  • Uncertainty maps stay clinically interpretable at high dropout rates, with high-uncertainty regions concentrated on anatomical boundaries and error-prone areas rather than diffusing into noise.
  • Pre-trained models that were never trained with a particular dropout policy can still be used for calibrated uncertainty estimation, because the adjustment happens entirely at inference time.
  • The method carries over to corrupted or out-of-distribution inputs, where fixed-rate dropout tends to degrade sharply.
  • The per-layer rates reveal which parts of a network are more information-sensitive, such as U-Net decoder layers, offering a new diagnostic for layer robustness.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the control signal is local and label-free, the same feedback loop could be applied to other stochastic perturbations at inference time, such as feature-map noise or attention dropout, whenever a clean full-activation reference is available.
  • The paper's choice to set the information-loss threshold equal to the baseline dropout rate is a calibration convenience; a natural extension is to search for the threshold on a small validation set and test whether per-layer budgets transfer across architectures and datasets.
  • The reported asymmetry between decoder and encoder layers suggests a testable coarse-grained policy: a single dropout rate for early layers and another for late layers might recover much of Rate-In's benefit at lower computational cost.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Rate-In, an inference-time algorithm that adapts per-layer dropout rates for each input by estimating the mutual-information loss between pre- and post-dropout feature maps and adjusting rates to keep that loss near a threshold. The authors claim this improves Monte Carlo Dropout uncertainty calibration and sharpness without sacrificing predictive accuracy or requiring labels. The claims are supported by experiments on synthetic regression, MedMNIST classification, prostate/liver segmentation, and MedMNIST-C out-of-distribution robustness, with additional runtime and sensitivity analyses in the appendices.

Significance. If the central empirical claim is sustained, Rate-In would be a practical contribution: it is label-free, operates post hoc on pretrained networks, and directly addresses a recognized limitation of fixed-rate MC Dropout. Strengths of the paper include public code, multiple medical-imaging benchmarks, externally defined uncertainty metrics (ECE, AUARC, BUC) that are not optimized by the algorithm, OOD evaluation, and an SSIM-based variant. However, the current experimental protocol does not isolate information-driven adaptivity from the effective dropout-rate distribution, and there is a formal inconsistency in the information-loss definition between the main algorithm and the appendix. These issues must be resolved before the central claim can be accepted.

major comments (4)
  1. [Section 4.1.1, Tables 1-2] The comparison protocol does not isolate adaptivity from the effective dropout rate. Rate-In's information-loss threshold is set equal to the baseline dropout rate p, but Rate-In is free to produce different per-layer and per-input rates, while Constant, Scheduled, and Activation baselines are constrained to exactly p (or a fixed function of p). No control is reported that holds the effective dropout-rate distribution fixed, e.g., a non-adaptive variant that uses the mean optimized per-layer rates as constants, or a variant that applies Rate-In's per-input rates without the MI feedback loop. Without such a control, the reported gains in ECE, AUARC, and BUC could be caused by selecting a different, perhaps more favorable, effective noise schedule rather than by mutual-information-based adaptation. Please add this control and report the average optimized rates per dataset and layer.
  2. [Section 3.3.1 (Algorithm 1) vs. Section 6.2.1] The information-loss definition is internally inconsistent. Algorithm 1 defines Delta I_l = I_full - I(h_in; h_out) and uses the convergence criterion |Delta I_l - epsilon| < delta, while Appendix 6.2.1 defines Delta I_l = (I_drop - I_full) / I_full, a relative change with the opposite sign convention, and states that epsilon = p. These are different mathematical objects with different units and different sign behavior. Concretely, if dropout reduces MI, the Appendix's Delta I_l is negative, so the Algorithm 1 condition Delta I_l > epsilon would be false and the update rule would increase the dropout rate rather than decrease it. The paper must state the exact objective used in every reported experiment and make Algorithm 1 consistent with that objective.
  3. [Section 4.2.3, Table 2] The claim that Rate-In 'consistently outperforms benchmarks in uncertainty metrics' is not supported by the table. At Peripheral MRI with p=0.01, Rate-In has ECE 4.60 versus 4.40 for the best baseline; at Tumor CT with p=0.10, Rate-In has BUC 0.48 versus 0.53 for the best baseline. These are cases where the baseline is better on the stated metric. Because the central claim is about improved calibration and sharpened uncertainty, the paper should report per-condition win/loss counts or paired error bars across seeds, and qualify the 'consistent' language accordingly.
  4. [Section 8.1 (Appendix C), Section 3.3.1] The load-bearing premise that layerwise MI loss is a faithful proxy for preserving calibrated predictive uncertainty is not validated. Appendix C concedes that MI 'may not fully capture vision task nuances' and that the loss objective 'may need empirical validation,' yet all of Rate-In's rate selection is driven by the MI threshold. Table 9 compares MI and SSIM for segmentation, but it does not control for the effective rate distribution and does not directly connect MI threshold values to ECE/AUARC/BUC outcomes. A concrete test would be to vary epsilon monotonically and show that the target uncertainty metrics respond monotonically, or to report a correlation between layerwise MI loss and downstream calibration changes across layers. Without such evidence, the mechanism linking the MI objective to the headline uncertainty improvements remains a premise rather than a demonstrated result.
minor comments (5)
  1. [Table 2 and Table 4] 'Benchmark*' is used in Table 2 but only defined in the Table 4 caption as the baseline with the highest average DSC per task; the main text should define this selection rule explicitly.
  2. [Table 1] The column header 'Accuracy (AUARC)' is ambiguous; the reader cannot tell whether the number before the parentheses is accuracy and the parenthetical is AUARC, or whether both are accuracy-like quantities. Please use separate columns with clear labels.
  3. [Section 6.2.1] The statement that setting epsilon = p with p = 0.2 'targets 80% information preservation' conflates a dropout probability with a fraction of retained mutual information; the mapping between these quantities is not derived and should be clarified or removed.
  4. [Algorithm 1] Algorithm 1 returns 'p' as if it were a single vector, but the rates are adjusted per layer and per input instance; the pseudocode should state whether p is stored per input or aggregated over the dataset.
  5. [Appendix 6.2.2] There is a typo: 'Dropout layers were placed after after each residual block' should read 'after each residual block.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Rate-In's uncertainty and calibration claims are empirically evaluated against external baselines, and no prediction reduces by construction to a fitted parameter or self-citation.

full rationale

No circular step in the derivation chain is exhibited. Rate-In's core mechanism controls the mutual-information loss between pre- and post-dropout feature maps (Algorithm 1, Section 3.3.1), while its success is measured by ECE, AUARC, BUC, accuracy, and interval efficiency, none of which are optimized or defined in terms of the controlled information-loss objective. The threshold epsilon and initial rate p0 are set equal to baseline p (Section 4.1.1 and Appendix 6.2.1); this is a hyperparameter matching choice, not a fitted parameter renamed as a prediction. It may create a comparison-fairness concern because Rate-In's effective per-layer dropout rates are free to diverge from the fixed p applied to baselines and no matched-effective-rate control is reported, but that is a confound and correctness risk, not circularity by construction. The self-citations [36] and [37] are background references on information theory and information bottleneck analysis and are not load-bearing for the algorithm's claimed advantage. The mutual-information estimator used is an external method [7], and no uniqueness theorem from the authors is invoked. Appendix C candidly concedes that MI may not capture all vision-task nuances and that the loss objective needs empirical validation; such admissions support the interpretation that the central claim is empirically contingent rather than definitionally forced. The internal inconsistency between Algorithm 1's absolute information loss and Appendix 6.2.1's relative-change formulation is a reproducibility issue, not a circular derivation. The paper is self-contained against external benchmarks, so the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The ledger is small: no new entities are introduced and no labels are fitted. The main burden sits in axioms about MI as functional information and in free hyperparameters epsilon, p0, delta, and Nmax. The comparison budget epsilon = p is an ad hoc equivalence rather than a derived one.

free parameters (4)
  • Information loss threshold epsilon = 0.01, 0.05, 0.10, 0.20
    Hand-chosen per experiment and set equal to the baseline dropout rate p. This is a normalized relative MI loss, not a dropout rate, so the budget matching is not well-defined.
  • Initial dropout rate p0 = equal to baseline p: 0.01, 0.05, 0.10, 0.20
    Starting point for the per-layer search. The paper provides no principle for choosing it beyond matching the baseline.
  • Convergence tolerance delta and maximum iterations Nmax = delta = 0.01, Nmax = 30
    Set based on preliminary experiments (Section 4.1.1); the algorithm can return non-converged rates, as Appendix B reports convergence failures in deeper layers.
  • Dropout-rate update step = learning rate 0.9 in synthetic experiments
    Main Algorithm 1 does not specify the update rule or step size; a learning rate of 0.9 appears only in the synthetic complexity experiments in Appendix B.
assumptions (4)
  • domain assumption Mutual information between input and layer output under dropout measures functional information loss relevant to predictive uncertainty.
    Invoked in Sections 2.5, 3.2, and 3.4; no derivation is given, and Appendix C concedes MI may not capture vision task nuances.
  • domain assumption Each layer can be treated as a noisy communication channel and dropout as controlled noise injection.
    Conceptual framing in Sections 2.5 and 3.1 used to justify thresholding MI loss; it is an analogy rather than a proven model.
  • ad hoc to paper Sequential greedy per-layer dropout-rate adjustment with a fixed threshold yields a rate policy that improves global uncertainty estimates.
    Algorithm 1 loops layers sequentially; there is no proof of convergence to a global objective, and Appendix B reports convergence failures.
  • ad hoc to paper An information-loss threshold epsilon numerically equal to a baseline dropout rate p is a meaningful matched comparison.
    Section 4.1.1 and Appendix 6.2.1 set epsilon = p; epsilon measures relative MI loss while p is a probability, so the budget matching is not well-defined.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rate-In: Information-Driven Adaptive Dropout Rates for Improved Inference-Time Uncertainty Estimation." pith.science (2026). https://pith.science/paper/Z3ZEWN7K

@misc{pith2026241207169,
  author       = {Pith},
  title        = {Pith review of: Rate-In: Information-Driven Adaptive Dropout Rates for Improved Inference-Time Uncertainty Estimation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z3ZEWN7K}},
  note         = {Machine review of arXiv:2412.07169}
}
read the original abstract

Accurate uncertainty estimation is crucial for deploying neural networks in risk-sensitive applications such as medical diagnosis. Monte Carlo Dropout is a widely used technique for approximating predictive uncertainty by performing stochastic forward passes with dropout during inference. However, using static dropout rates across all layers and inputs can lead to suboptimal uncertainty estimates, as it fails to adapt to the varying characteristics of individual inputs and network layers. Existing approaches optimize dropout rates during training using labeled data, resulting in fixed inference-time parameters that cannot adjust to new data distributions, compromising uncertainty estimates in Monte Carlo simulations. In this paper, we propose Rate-In, an algorithm that dynamically adjusts dropout rates during inference by quantifying the information loss induced by dropout in each layer's feature maps. By treating dropout as controlled noise injection and leveraging information-theoretic principles, Rate-In adapts dropout rates per layer and per input instance without requiring ground truth labels. By quantifying the functional information loss in feature maps, we adaptively tune dropout rates to maintain perceptual quality across diverse medical imaging tasks and architectural configurations. Our extensive empirical study on synthetic data and real-world medical imaging tasks demonstrates that Rate-In improves calibration and sharpens uncertainty estimates compared to fixed or heuristic dropout rates without compromising predictive performance. Rate-In offers a practical, unsupervised, inference-time approach to optimizing dropout for more reliable predictive uncertainty estimation in critical applications.

Figures

Figures reproduced from arXiv: 2412.07169 by the authors.

Figure 1
Figure 1. Rate-In maintains anatomically meaningful uncertainty estimates at high dropout rates, while standard methods degrade into noise. Left: Prostate MRI segmentation showing ground truth (yellow), model predictions (blue), and errors (red). Right: Uncertainty maps (brighter regions indicate higher uncertainty) at increasing dropout rates (1%-16%). While constant dropout produces diffuse uncer￾tainty that obscures anatom… view at source ↗
Figure 2
Figure 2. Rate-In dynamically optimizes dropout rates through an information-theoretic feedback loop: (i) apply a dropout mask to the feature map; (ii) assess information integrity; (iii) cal￾culate the difference from the target information level; (iv) update the dropout rate accordingly. Algorithm 1 Rate-In Algorithm Require: Trained neural network F with L layers, input x, initial dropout rates p (0), information loss thre… view at source ↗
Figure 3
Figure 3. a shows the efficiency of uncertainty estimation across increasing noise levels (σ ∈ {0.1, 0.2, 0.3, 0.4, 0.5}) with a fixed training set size (N = 100). Efficiency is measured as the ratio of interval width to empirical coverage; lower ratios indicate more precise uncertainty estimates without compromising reliability. Rate-In maintains a consistently lower ratio across all noise levels compared to baselines. Fig. … view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: illustrates the impact of dropout rates on infor￾mation loss across network layers. In a pre-trained U-Net model, the decoder layers lose about 40% of information as dropout rates increase from 5% to 20%, while encoder layers remain more stable. This suggests that Rate…
Figure 5
Figure 5. Figure 5: Basic layout of the regression network with dropout [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: A single ResNet layer with dropout layers shown in blue. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Basic layout of the U-Net network with dropout layers [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Rate-In dropout yields narrower, more stable uncertainty intervals under increasing noise levels ( [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Rate-In is more efficient in uncertainty estimation. The ratio of uncertainty interval width to 95% coverage for vary￾ing number of training points at fixed noise levels. Lower ratios indicate more efficient uncertainty estimation. (a) σ = 0.01, (b) σ = 0.10, and (c) σ…
Figure 10
Figure 10. Figure 10: analyzes the convergence behavior of Rate-In across varying initial dropout rates under different informa￾tion loss thresholds (ϵ = 0.10, 0.30, 0.50), with Nmax = 100 and σ = 0.50. The results demonstrate that Rate-In con￾verges to consistent dropout rates, independen…
Figure 12
Figure 12. Figure 12: Information loss patterns across VGG-16, MobileNet, [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: Rate-In execution time analysis for synthetic re￾gression task. Plots show optimization process duration (in sec￾onds) as a function of: (a) initial dropout rate, (b) information loss threshold, (c) number of inference instances, and (d) noise level [PITH_FULL_IMAGE:…
Figure 14
Figure 14. Figure 14: Dropout-induced information loss in the U-Net seg [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 34 canonical work pages

  1. [1]

    Information dropout: Learning optimal representations through noisy computa- tion.IEEE transactions on pattern analysis and machine intelligence, 40(12):2897–2905, 2018

    Alessandro Achille and Stefano Soatto. Information dropout: Learning optimal representations through noisy computa- tion.IEEE transactions on pattern analysis and machine intelligence, 40(12):2897–2905, 2018. 4

    work page 2018

  2. [2]

    The medical segmentation decathlon.Nature communications, 13(1):4128, 2022

    Michela Antonelli, Annika Reinke, Spyridon Bakas, Key- van Farahani, Annette Kopp-Schneider, Bennett A Landman, Geert Litjens, Bjoern Menze, Olaf Ronneberger, Ronald M Summers, et al. The medical segmentation decathlon.Nature communications, 13(1):4128, 2022. 6, 1, 2

    work page 2022

  3. [3]

    Adaptive dropout for training deep neural networks.Advances in neural information pro- cessing systems, 26, 2013

    Jimmy Ba and Brendan Frey. Adaptive dropout for training deep neural networks.Advances in neural information pro- cessing systems, 26, 2013. 2, 6

    work page 2013

  4. [4]

    The need for uncertainty quantification in machine- assisted medical decision making.Nature Machine Intelli- gence, 1(1):20–23, 2019

    Edmon Begoli, Tanmoy Bhattacharya, and Dimitri Kusne- zov. The need for uncertainty quantification in machine- assisted medical decision making.Nature Machine Intelli- gence, 1(1):20–23, 2019. 1, 3

    work page 2019

  5. [5]

    Mine: mutual information neural estimation.arXiv preprint arXiv:1801.04062, 2018

    Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and R Devon Hjelm. Mine: mutual information neural estimation.arXiv preprint arXiv:1801.04062, 2018. 4

    arXiv 2018

  6. [6]

    Weight uncertainty in neural network

    Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. In International conference on machine learning, pages 1613–

  7. [7]

    Adaptive Estimators Show Information Compression in Deep Neural Networks

    Ivan Chelombiev, Conor Houghton, and Cian O’Donnell. Adaptive estimators show information compression in deep neural networks.arXiv preprint arXiv:1902.09037, 2019. 4, 6, 1

    work page Pith review arXiv 1902

  8. [8]

    John Wiley & Sons, 1999

    Thomas M Cover.Elements of information theory. John Wiley & Sons, 1999. 2, 4

    work page 1999

Show all 47 references
  1. [9]

    Medmnist-c: Comprehensive benchmark and im- proved classifier robustness by simulating realistic image corruptions.arXiv preprint arXiv:2406.17536, 2024

    Francesco Di Salvo, Sebastian Doerrich, and Christian Ledig. Medmnist-c: Comprehensive benchmark and im- proved classifier robustness by simulating realistic image corruptions.arXiv preprint arXiv:2406.17536, 2024. 6, 8

    2024 arXiv

  2. [10]

    Masksembles for uncertainty estimation

    Nikita Durasov, Timur Bagautdinov, Pierre Baque, and Pas- cal Fua. Masksembles for uncertainty estimation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13539–13548, 2021. 3

    2021

  3. [11]

    Mc layer normalization for calibrated uncertainty in deep learn- ing.Transactions on Machine Learning Research, 2024

    Thomas Frick, Diego Antognini, Ioana Giurgiu, Benjamin Grewe, Cristiano Malossi, Rong Zhu, and Mattia Rigotti. Mc layer normalization for calibrated uncertainty in deep learn- ing.Transactions on Machine Learning Research, 2024. 3

    2024

  4. [12]

    Dropout as a bayesian approximation: Representing model uncertainty in deep learning

    Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. Ininternational conference on machine learning, pages 1050–1059. PMLR, 2016. 1, 3

    2016

  5. [13]

    Concrete dropout

    Yarin Gal, Jiri Hron, and Alex Kendall. Concrete dropout. Advances in neural information processing systems, 30,

  6. [14]

    Uncertainty in deep learning.phd thesis,

    Yarin Gal et al. Uncertainty in deep learning.phd thesis,

  7. [15]

    Bias- reduced uncertainty estimation for deep neural classifiers

    Yonatan Geifman, Guy Uziel, and Ran El-Yaniv. Bias- reduced uncertainty estimation for deep neural classifiers. arXiv preprint arXiv:1805.08206, 2018. 6, 3

    2018 arXiv

  8. [16]

    On calibration of modern neural networks

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. InInternational conference on machine learning, pages 1321–1330. PMLR,

  9. [17]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 6

    2016

  10. [18]

    Benchmarking neu- ral network robustness to common corruptions and perturba- tions.arXiv preprint arXiv:1903.12261, 2019

    Dan Hendrycks and Thomas Dietterich. Benchmarking neu- ral network robustness to common corruptions and perturba- tions.arXiv preprint arXiv:1903.12261, 2019. 6

    1903 arXiv

  11. [19]

    Imagenet object localization challenge.Kaggle

    Addison Howard, Eunbyung Park, and Wendy Kan. Imagenet object localization challenge.Kaggle. URL: https://kaggle. com/competitions/imagenet-object- localization-challenge, 2018. 4

    2018

  12. [20]

    nnu-net: a self-configuring method for deep learning-based biomedical image segmen- tation.Nature methods, 18(2):203–211, 2021

    Fabian Isensee, Paul F Jaeger, Simon AA Kohl, Jens Pe- tersen, and Klaus H Maier-Hein. nnu-net: a self-configuring method for deep learning-based biomedical image segmen- tation.Nature methods, 18(2):203–211, 2021. 6, 1, 2

    2021

  13. [21]

    What uncertainties do we need in bayesian deep learning for computer vision?Advances in neural information processing systems, 30, 2017

    Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision?Advances in neural information processing systems, 30, 2017. 3

    2017

  14. [22]

    Varia- tional dropout and the local reparameterization trick.Ad- vances in neural information processing systems, 28, 2015

    Durk P Kingma, Tim Salimans, and Max Welling. Varia- tional dropout and the local reparameterization trick.Ad- vances in neural information processing systems, 28, 2015. 2

    2015

  15. [23]

    Improving model calibration with accuracy versus uncertainty optimization

    Ranganath Krishnan and Omesh Tickoo. Improving model calibration with accuracy versus uncertainty optimization. Advances in Neural Information Processing Systems, 33: 18237–18248, 2020. 3

    2020

  16. [24]

    Simple and scalable predictive uncertainty estima- tion using deep ensembles.Advances in neural information processing systems, 30, 2017

    Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estima- tion using deep ensembles.Advances in neural information processing systems, 30, 2017. 3

    2017

  17. [25]

    Well-calibrated model uncertainty with temperature scaling for dropout variational inference.arXiv preprint arXiv:1909.13550, 2019

    Max-Heinrich Laves, Sontje Ihler, Karl-Philipp Kortmann, and Tobias Ortmaier. Well-calibrated model uncertainty with temperature scaling for dropout variational inference.arXiv preprint arXiv:1909.13550, 2019. 3

    1909 arXiv

  18. [26]

    Dropout injection at test time for post hoc uncertainty quantifica- tion in neural networks.Information Sciences, 645:119356,

    Emanuele Ledda, Giorgio Fumera, and Fabio Roli. Dropout injection at test time for post hoc uncertainty quantifica- tion in neural networks.Information Sciences, 645:119356,

  19. [27]

    Boundary-aware uncertainty suppression for semi-supervised medical image segmenta- tion.IEEE Transactions on Artificial Intelligence, 5(8): 4074–4086, 2024

    Congcong Li, Jinshuo Zhang, Dongmei Niu, Xiuyang Zhao, Bo Yang, and Caiming Zhang. Boundary-aware uncertainty suppression for semi-supervised medical image segmenta- tion.IEEE Transactions on Artificial Intelligence, 5(8): 4074–4086, 2024. 3

    2024

  20. [28]

    Confidence calibration and predictive uncertainty estimation for deep medical im- age segmentation.IEEE transactions on medical imaging, 39(12):3868–3878, 2020

    Alireza Mehrtash, William M Wells, Clare M Tempany, Pu- rang Abolmaesumi, and Tina Kapur. Confidence calibration and predictive uncertainty estimation for deep medical im- age segmentation.IEEE transactions on medical imaging, 39(12):3868–3878, 2020. 3

    2020

  21. [29]

    Training-free uncertainty estimation for dense re- gression: Sensitivity as a surrogate

    Lu Mi, Hao Wang, Yonglong Tian, Hao He, and Nir N Shavit. Training-free uncertainty estimation for dense re- gression: Sensitivity as a surrogate. InProceedings of the AAAI Conference on Artificial Intelligence, pages 10042– 10050, 2022. 2, 3

    2022

  22. [30]

    Dropconnect is effective in modeling uncertainty of bayesian deep networks

    Aryan Mobiny, Pengyu Yuan, Supratik K Moulik, Naveen Garg, Carol C Wu, and Hien Van Nguyen. Dropconnect is effective in modeling uncertainty of bayesian deep networks. Scientific reports, 11(1):5458, 2021. 3

    2021

  23. [31]

    Evaluating bayesian deep learning methods for semantic segmentation.arXiv preprint arXiv:1811.12709, 2018

    Jishnu Mukhoti and Yarin Gal. Evaluating bayesian deep learning methods for semantic segmentation.arXiv preprint arXiv:1811.12709, 2018. 3

    2018 arXiv

  24. [32]

    Accuracy-rejection curves (arcs) for com- paring classification methods with a reject option

    Malik Sajjad Ahmed Nadeem, Jean-Daniel Zucker, and Blaise Hanczar. Accuracy-rejection curves (arcs) for com- paring classification methods with a reject option. InMa- chine Learning in Systems Biology, pages 65–81. PMLR,

  25. [33]

    Risk versus uncertainty in deep learning: Bayes, bootstrap and the dangers of dropout

    Ian Osband. Risk versus uncertainty in deep learning: Bayes, bootstrap and the dangers of dropout. InNIPS workshop on bayesian deep learning. MIT Press, 2016. 1, 3

    2016

  26. [34]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zem- ing Lin, Natalia Gimelshein, Luca Antiga, Alban Desmai- son, Andreas K ¨opf, Edward Yang, Zachary DeVito, Mar- tin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Stei...

    2019

  27. [35]

    An- nealed dropout training of deep networks

    Steven J Rennie, Vaibhava Goel, and Samuel Thomas. An- nealed dropout training of deep networks. In2014 IEEE Spo- ken Language Technology Workshop (SLT), pages 159–164. IEEE, 2014. 6, 2

    2014

  28. [36]

    Information flow in deep neural net- works.arXiv preprint arXiv:2202.06749, 2022

    Ravid Shwartz-Ziv. Information flow in deep neural net- works.arXiv preprint arXiv:2202.06749, 2022. 4

    2022 arXiv

  29. [37]

    Opening the black box of deep neural networks via information.arXiv preprint arXiv:1703.00810, 2017

    Ravid Shwartz-Ziv and Naftali Tishby. Opening the black box of deep neural networks via information.arXiv preprint arXiv:1703.00810, 2017. 4

    2017 arXiv

  30. [38]

    Dropout: a simple way to prevent neural networks from overfitting.The journal of machine learning research, 15(1):1929–1958, 2014

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting.The journal of machine learning research, 15(1):1929–1958, 2014. 3, 4, 6, 2

    1929

  31. [39]

    Bayesian uncertainty estimation for batch normalized deep networks

    Mattias Teye, Hossein Azizpour, and Kevin Smith. Bayesian uncertainty estimation for batch normalized deep networks. InInternational conference on machine learning, pages 4907–4916. PMLR, 2018. 3

    2018

  32. [40]

    Regularization of neural networks using drop- connect

    Li Wan, Matthew Zeiler, Sixin Zhang, Yann Le Cun, and Rob Fergus. Regularization of neural networks using drop- connect. InInternational conference on machine learning, pages 1058–1066. PMLR, 2013. 4

    2013

  33. [41]

    Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004. 6, 7

    2004

  34. [42]

    Medm- nist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification.Scientific Data, 10(1):41,

    Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. Medm- nist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification.Scientific Data, 10(1):41,

  35. [43]

    Boundary un- certainty aware network for automated polyp segmentation

    Guanghui Yue, Guibin Zhuo, Weiqing Yan, Tianwei Zhou, Chang Tang, Peng Yang, and Tianfu Wang. Boundary un- certainty aware network for automated polyp segmentation. Neural Networks, 170:390–404, 2024. 6, 3 Rate-In: Information-Driven Adaptive Dropout Rates for Improved Inferen...

    2024

  36. [44]

    Reproducibility Statement The Rate-In algorithm code and implementation examples are available in theGitHub repository 1

    Appendix A: Reproducibility 6.1. Reproducibility Statement The Rate-In algorithm code and implementation examples are available in theGitHub repository 1. •Environment and Data:Dependencies are listed in requirements.txt. Datasets (PathMNIST, BloodMNIST, TissueMNIST) are from ...

  37. [46]

    Synthetic Data Figures 8 and 9 evaluate Rate-In dropout’s performance

    Appendix B: Additional Results 7.1. Synthetic Data Figures 8 and 9 evaluate Rate-In dropout’s performance. Figure 8 compares uncertainty intervals from constant dropout (p= 0.10, red) and Rate-In dropout (ϵ= 0.10, blue) across five noise levels (σ= 0.1−0.5), showing Rate-In’s ...

  38. [47]

    This perspective shift allows us to examine how dropout affects network representations in task-specific contexts

    Appendix C: Further Discussion Dropout, beyond just graph manipulation.Rather than viewing dropout as merely a tool for graph manipulation, Rate-In reinterprets it as a method for controlled noise injection. This perspective shift allows us to examine how dropout affects netwo...

  39. [123]

    Figure 5

    Dropout layers were placed after each hidden layer post-training (Figure 5). Figure 5. Basic layout of the regression network with dropout layers shown in blue. Classification:ResNet-18 networks were employed us- ing dataset-specific pre-trained weights and pre-processing prot...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.