{"id":"dd06cc49-77c3-45e7-adef-c4e3066a5801","arxiv_id":"2412.07169","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Rate-In adapts dropout rates per layer and per input at inference by measuring mutual-information loss, improving calibration and sharpening uncertainty estimates in medical imaging tests.","lead":"The paper introduces Rate-In, an algorithm that changes each network layer's dropout rate during inference to keep information loss below a threshold. It matters because medical imaging needs calibrated uncertainty, and Rate-In does this without labels or retraining.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Per-instance adaptivity may be epiphenomenal: reported gains could come from mismatched effective dropout rates, since epsilon=p and no frozen-rate control is run.","rationale":"The reader's weakest assumption targets the faithfulness of layer-local MI as a proxy for calibrated uncertainty and the arbitrary epsilon=p threshold. I agree that this is a weak point, but the more decisive issue is empirical: the evaluation protocol does not rule out the possibility that Rate-In simply picks a different effective dropout-rate schedule. Since epsilon equals p, the nominal comparison is fair only if the final applied rates equal p, which they generally will not after optimization. The paper reports consistent directional gains, but all tables are single point estimates with no error bars or significance tests, so the risk of confounded comparisons is real. The proposed frozen-rate ablation is a single, concrete experiment that would settle whether per-input adaptation is causal. If the frozen control performs as well as full Rate-In, the method's contribution reduces to choosing better per-layer rates, not to information-driven per-instance adaptation. If full Rate-In clearly outperforms the frozen control, the central claim is strengthened. I therefore keep the verdict conditional, consistent with the reader, and recommend this additional experiment before accepting the empirical claim at face value.","tokens_in":21767,"tokens_out":4643,"duration_ms":57708,"concrete_test":"Run a frozen-rate control on PathMNIST, TissueMNIST, and BloodMNIST: compute Rate-In's optimized per-layer dropout rates on a validation subset, average each layer's rate over inputs, then apply these fixed per-layer rates to every test input with 30 MC forward passes. Compare ECE, AUARC, and accuracy against full Rate-In and against constant dropout at the same mean overall rate. If frozen-mean rates match full Rate-In, the claimed per-input adaptation is not responsible for the gains; if full Rate-In is significantly better across all three datasets, the adaptive mechanism is supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that Rate-In's per-layer, per-input information-driven dropout adjustment improves uncertainty calibration and sharpness. The most load-bearing concern is that the experiments do not isolate adaptivity from the effective noise level. Section 4.1.1 sets Rate-In's information-loss threshold epsilon equal to the baseline dropout rate p, but Rate-In's optimized rates are then free to differ per layer and per input. Constant and scheduled baselines are evaluated at exactly p, so the two conditions are not matched in average or per-layer applied dropout. Any observed improvement could therefore be caused simply by selecting a different, perhaps more favorable, effective dropout schedule rather than by mutual-information-based adaptation. The paper reports no control that holds the effective dropout budget fixed, such as a non-adaptive variant using the mean optimized per-layer rates. This concern is compounded by an internal inconsistency: Algorithm 1 (Section 3.3.1) defines information loss as Delta I = I_full - I(h_in; h_out) with a convergence loop, while Appendix 6.2.1 defines it as the relative change (I_drop - I_full)/I_full. These are different objectives, making the threshold epsilon ambiguous. Appendix C (Section 8.1) itself concedes that MI may not fully capture vision-task nuances and that the loss objective needs empirical validation. Without a control that removes the adaptive component while preserving the same dropout-rate distribution, the causal role of the information-theoretic controller is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Rate-In, an inference-time algorithm that adapts per-layer dropout rates for each input by estimating the mutual-information loss between pre- and post-dropout feature maps and adjusting rates to keep that loss near a threshold. The authors claim this improves Monte Carlo Dropout uncertainty calibration and sharpness without sacrificing predictive accuracy or requiring labels. The claims are supported by experiments on synthetic regression, MedMNIST classification, prostate/liver segmentation, and MedMNIST-C out-of-distribution robustness, with additional runtime and sensitivity analyses in the appendices.","tokens_in":22050,"tokens_out":5666,"duration_ms":62270,"significance":"If the central empirical claim is sustained, Rate-In would be a practical contribution: it is label-free, operates post hoc on pretrained networks, and directly addresses a recognized limitation of fixed-rate MC Dropout. Strengths of the paper include public code, multiple medical-imaging benchmarks, externally defined uncertainty metrics (ECE, AUARC, BUC) that are not optimized by the algorithm, OOD evaluation, and an SSIM-based variant. However, the current experimental protocol does not isolate information-driven adaptivity from the effective dropout-rate distribution, and there is a formal inconsistency in the information-loss definition between the main algorithm and the appendix. These issues must be resolved before the central claim can be accepted.","major_comments":[{"comment":"The comparison protocol does not isolate adaptivity from the effective dropout rate. Rate-In's information-loss threshold is set equal to the baseline dropout rate p, but Rate-In is free to produce different per-layer and per-input rates, while Constant, Scheduled, and Activation baselines are constrained to exactly p (or a fixed function of p). No control is reported that holds the effective dropout-rate distribution fixed, e.g., a non-adaptive variant that uses the mean optimized per-layer rates as constants, or a variant that applies Rate-In's per-input rates without the MI feedback loop. Without such a control, the reported gains in ECE, AUARC, and BUC could be caused by selecting a different, perhaps more favorable, effective noise schedule rather than by mutual-information-based adaptation. Please add this control and report the average optimized rates per dataset and layer.","section":"Section 4.1.1, Tables 1-2"},{"comment":"The information-loss definition is internally inconsistent. Algorithm 1 defines Delta I_l = I_full - I(h_in; h_out) and uses the convergence criterion |Delta I_l - epsilon| < delta, while Appendix 6.2.1 defines Delta I_l = (I_drop - I_full) / I_full, a relative change with the opposite sign convention, and states that epsilon = p. These are different mathematical objects with different units and different sign behavior. Concretely, if dropout reduces MI, the Appendix's Delta I_l is negative, so the Algorithm 1 condition Delta I_l > epsilon would be false and the update rule would increase the dropout rate rather than decrease it. The paper must state the exact objective used in every reported experiment and make Algorithm 1 consistent with that objective.","section":"Section 3.3.1 (Algorithm 1) vs. Section 6.2.1"},{"comment":"The claim that Rate-In 'consistently outperforms benchmarks in uncertainty metrics' is not supported by the table. At Peripheral MRI with p=0.01, Rate-In has ECE 4.60 versus 4.40 for the best baseline; at Tumor CT with p=0.10, Rate-In has BUC 0.48 versus 0.53 for the best baseline. These are cases where the baseline is better on the stated metric. Because the central claim is about improved calibration and sharpened uncertainty, the paper should report per-condition win/loss counts or paired error bars across seeds, and qualify the 'consistent' language accordingly.","section":"Section 4.2.3, Table 2"},{"comment":"The load-bearing premise that layerwise MI loss is a faithful proxy for preserving calibrated predictive uncertainty is not validated. Appendix C concedes that MI 'may not fully capture vision task nuances' and that the loss objective 'may need empirical validation,' yet all of Rate-In's rate selection is driven by the MI threshold. Table 9 compares MI and SSIM for segmentation, but it does not control for the effective rate distribution and does not directly connect MI threshold values to ECE/AUARC/BUC outcomes. A concrete test would be to vary epsilon monotonically and show that the target uncertainty metrics respond monotonically, or to report a correlation between layerwise MI loss and downstream calibration changes across layers. Without such evidence, the mechanism linking the MI objective to the headline uncertainty improvements remains a premise rather than a demonstrated result.","section":"Section 8.1 (Appendix C), Section 3.3.1"}],"minor_comments":[{"comment":"'Benchmark*' is used in Table 2 but only defined in the Table 4 caption as the baseline with the highest average DSC per task; the main text should define this selection rule explicitly.","section":"Table 2 and Table 4"},{"comment":"The column header 'Accuracy (AUARC)' is ambiguous; the reader cannot tell whether the number before the parentheses is accuracy and the parenthetical is AUARC, or whether both are accuracy-like quantities. Please use separate columns with clear labels.","section":"Table 1"},{"comment":"The statement that setting epsilon = p with p = 0.2 'targets 80% information preservation' conflates a dropout probability with a fraction of retained mutual information; the mapping between these quantities is not derived and should be clarified or removed.","section":"Section 6.2.1"},{"comment":"Algorithm 1 returns 'p' as if it were a single vector, but the rates are adjusted per layer and per input instance; the pseudocode should state whether p is stored per input or aggregated over the dataset.","section":"Algorithm 1"},{"comment":"There is a typo: 'Dropout layers were placed after after each residual block' should read 'after each residual block.'","section":"Appendix 6.2.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly motivated and the empirical direction is plausible, but the comparison confound and the formal inconsistency in the information-loss definition are load-bearing for the central claim. I recommend major revision rather than rejection because both issues can be addressed with additional control experiments and a consistent formalization; the manuscript's scope is otherwise appropriate for the venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take on Rate-In: it's a genuine new combination—per-layer, per-instance dropout rates at inference, driven by an information-loss feedback loop, no labels needed. That's not in the earlier test-time dropout work. The empirical spread is good: synthetic regression, three MedMNIST classification tasks, prostate/liver segmentation, OOD corruptions, across ResNet and U-Net. The reported calibration gains (ECE, AUARC, BUC) are consistent and the metrics aren't directly optimized by the algorithm, so there's real signal somewhere. Code is available.\n\nWhere it gets soft: the comparison protocol doesn't isolate adaptivity. They set epsilon = p, the baseline dropout rate, but Rate-In is then free to pick different rates per layer and per input. The constant and scheduled baselines are stuck at p. So the gains could just come from Rate-In choosing a more favorable effective dropout rate, not from the information-theoretic adaptation. There's no control that uses the same optimized rate distribution without the per-input adaptivity—say, a frozen variant with the mean optimized rates per layer. That's a missing experiment and it's load-bearing for the central claim.\n\nAlso, the MI-loss definition is inconsistent between Algorithm 1 (Delta I = I_full - I(h_in;h_out)) and Appendix 6.2.1 (relative change (I_drop - I_full)/I_full). Those are different objectives and make the threshold epsilon ambiguous. No error bars either, so run-to-run variability is unknown.\n\nThe authors themselves admit in Appendix C that MI \"may not fully capture vision task nuances\" and that the loss objective needs empirical validation. That's honest, but it undercuts the theoretical framing more than they acknowledge.\n\nOverall: the idea is worth pursuing and the evidence is suggestive, but the paper doesn't yet prove that the adaptive mechanism—rather than the effective dropout schedule—is what works. It deserves a serious referee, with a request for a frozen-rate control, a reconciled MI definition, and error bars. I'd send it to review rather than desk-reject.","headline":"Per-input adaptive MC dropout is a plausible new idea, but the experiments don't yet separate adaptivity from simply choosing different rates; still worth refereeing.","tokens_in":22573,"tokens_out":1898,"would_cite":false,"duration_ms":20758,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Rate-In dynamically adjusts each layer's dropout rate at inference time by tracking mutual-information loss, yielding better-calibrated and sharper uncertainty estimates without retraining or labels.","keywords":["Monte Carlo Dropout","adaptive dropout rates","inference-time uncertainty estimation","mutual information","medical image segmentation","calibration","out-of-distribution robustness"],"falsifier":"Compute Rate-In on a fixed test set, then replace the selected per-layer dropout rates with random rates matched to have the same mean information loss; if calibration and accuracy-rejection metrics remain unchanged, the per-layer mutual information signal is not what drives the gains.","tokens_in":21552,"feed_emoji":"🩺","tokens_out":5440,"duration_ms":51346,"temperature":0.7,"pith_summary":"Rate-In claims that the reason Monte Carlo Dropout gives blurry or mis-calibrated uncertainty is that dropout rates are frozen at training-time values, applied uniformly to every layer and every input. The paper proposes to fix this at inference time by measuring, for each layer, how much mutual information is lost between the pre- and post-dropout feature maps, and then raising or lowering that layer's dropout rate until the loss sits below a threshold. This is done per input and per layer, with no labels and no retraining. The paper reports that the resulting uncertainty estimates are better calibrated and more sharply localized, especially along tissue boundaries in medical segmentation, while predictive accuracy is preserved or improved.","feed_headline":"Dropout adapts per layer and per image to sharpen uncertainty","feed_subtitle":"Rate-In tunes inference-time dropout by tracking information loss, improving calibration on medical images.","key_machinery":"The machinery is a mutual-information feedback loop between a layer's input and its post-dropout feature maps, using an adaptive-binning estimator to keep mutual information computable in high dimensions. Mutual information plays the role of a per-layer information budget: the information loss, defined as the relative drop in mutual information caused by dropout, is compared to a threshold epsilon, and the dropout rate is stepped until the loss falls within delta of that threshold. This turns dropout-rate selection into a local, label-free, input-specific optimization problem.","core_discovery":"The central discovery is that dropout-induced information loss, measured as the relative drop in mutual information between a layer's input and its post-dropout output, can serve as a per-layer control signal for setting inference-time dropout rates. Rate-In runs a feedback loop: apply dropout at the current rate, estimate mutual information with an adaptive binning estimator, compare the loss to a threshold epsilon, and adjust the rate up or down until convergence, then move to the next layer. Because this happens layer-by-layer on the actual test input, the final dropout rates differ across layers and across inputs. The paper argues this converts dropout from uniform impulse noise into task-aware contextual noise, and demonstrates on medical imaging benchmarks that calibration, accuracy-rejection trade-offs, and boundary-focused uncertainty improve without sacrificing segmentation Dice or classification accuracy.","pith_inferences":["Because the control signal is local and label-free, the same feedback loop could be applied to other stochastic perturbations at inference time, such as feature-map noise or attention dropout, whenever a clean full-activation reference is available.","The paper's choice to set the information-loss threshold equal to the baseline dropout rate is a calibration convenience; a natural extension is to search for the threshold on a small validation set and test whether per-layer budgets transfer across architectures and datasets.","The reported asymmetry between decoder and encoder layers suggests a testable coarse-grained policy: a single dropout rate for early layers and another for late layers might recover much of Rate-In's benefit at lower computational cost."],"forward_implications":["MC Dropout becomes input-adaptive: each test sample receives its own per-layer dropout schedule rather than a uniform fixed rate.","Uncertainty maps stay clinically interpretable at high dropout rates, with high-uncertainty regions concentrated on anatomical boundaries and error-prone areas rather than diffusing into noise.","Pre-trained models that were never trained with a particular dropout policy can still be used for calibrated uncertainty estimation, because the adjustment happens entirely at inference time.","The method carries over to corrupted or out-of-distribution inputs, where fixed-rate dropout tends to degrade sharply.","The per-layer rates reveal which parts of a network are more information-sensitive, such as U-Net decoder layers, offering a new diagnostic for layer robustness."],"supporting_citations":[{"why":"Supplies the MC Dropout posterior-approximation framework that Rate-In adapts at inference time.","marker":"[12]"},{"why":"Provides the adaptive binning mutual information estimator used to compute per-layer information loss.","marker":"[7]"},{"why":"Defines the dropout operation and its noise-injection properties that Rate-In reinterprets as a controlled channel.","marker":"[38]"},{"why":"Gives the information-theoretic channel view of layers and the mutual information definition underlying the loss measure.","marker":"[8]"},{"why":"Establishes the idea of dropout as information-loss control during training, which Rate-In moves to inference time.","marker":"[1]"}],"fun_headline_variants":["Adaptive dropout at inference sharpens uncertainty","Per-layer dropout rates tuned by information loss","Rate-In adjusts dropout to reduce miscalibration","Information-driven dropout improves test-time calibration","Dynamic dropout per input for reliable uncertainty"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument depends on the assumption that the drop in shared information between a layer's input and its post-dropout output is exactly the quantity that must be controlled to keep predictive uncertainty well calibrated, and that a single threshold equal to the baseline dropout rate is the right budget for every layer and input.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive dropout at inference sharpens uncertainty","Per-layer dropout rates tuned by information loss","Rate-In adjusts dropout to reduce miscalibration","Information-driven dropout improves test-time calibration","Dynamic dropout per input for reliable uncertainty"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000246,"raw_usage":{"total_tokens":1534,"prompt_tokens":935,"completion_tokens":599,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":534}},"tokens_in":551,"tokens_out":599,"duration_ms":6599,"temperature":1.0,"reasoning_tokens":534,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:04:24.640322+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute Rate-In on a fixed test set, then replace the selected per-layer dropout rates with random rates matched to have the same mean information loss; if calibration and accuracy-rejection metrics remain unchanged, the per-layer mutual information signal is not what drives the gains.","supporting_citations":[{"cited_title":"Dropout as a bayesian approximation: Representing model uncertainty in deep learning","cited_arxiv_id":null,"evidence_quote":"Supplies the MC Dropout posterior-approximation framework that Rate-In adapts at inference time."},{"cited_title":"Adaptive Estimators Show Information Compression in Deep Neural Networks","cited_arxiv_id":"1902.09037","evidence_quote":"Provides the adaptive binning mutual information estimator used to compute per-layer information loss."},{"cited_title":"Dropout: a simple way to prevent neural networks from overfitting.The journal of machine learning research, 15(1):1929–1958, 2014","cited_arxiv_id":null,"evidence_quote":"Defines the dropout operation and its noise-injection properties that Rate-In reinterprets as a controlled channel."},{"cited_title":"John Wiley & Sons, 1999","cited_arxiv_id":null,"evidence_quote":"Gives the information-theoretic channel view of layers and the mutual information definition underlying the loss measure."},{"cited_title":"Information dropout: Learning optimal representations through noisy computa- tion.IEEE transactions on pattern analysis and machine intelligence, 40(12):2897–2905, 2018","cited_arxiv_id":null,"evidence_quote":"Establishes the idea of dropout as information-loss control during training, which Rate-In moves to inference time."}],"review_version":1}