{"id":"7862e3a8-f50d-46fd-8b0c-80425c87c3cf","arxiv_id":"2501.11258","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Monte-Carlo Frequency Dropout, which randomly removes frequency components in feature maps, yields better-calibrated uncertainty estimates than standard signal dropout for medical image segmentation.","lead":"This paper tests a variant of dropout that randomly silences frequency components inside a neural network instead of switching off individual pixels, and applies it to estimate uncertainty in medical image segmentation. It reports that this frequency-based dropout gives better-calibrated uncertainty maps than standard dropout across MRI, CT, and X-ray segmentation tasks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Selection of 'best' dropout configuration on the test set makes the Frequency-over-Signal calibration advantage unverified; a held-out selection protocol is needed.","rationale":"This is a methodological flaw in the empirical comparison, not a disagreement with the idea. Frequency dropout is a plausible extension, and the public code is a point in the authors' favor. The concern is that the headline comparison is optimized on the same data used to measure it. Given the small cohort sizes and the large config grid, this can easily generate a false positive. I agree with the reader that the paper is conditional, but the most load-bearing weakness is the selection protocol rather than the Fourier-symmetry notation. The symmetry issue is checkable (and likely solved by using rfft) and would not change the conclusion if the code is right; the selection issue directly determines whether the central claim is true. A held-out validation check would settle it.","tokens_in":6424,"tokens_out":3889,"duration_ms":40904,"concrete_test":"Hold out an evaluation portion of each cohort (or use leave-one-out) before any comparison. On the training/selection portion, choose the dropout rate and placement that minimize UCE for each method. Then evaluate the chosen configurations on the held-out portion and compare UCE. Repeat with several random splits. If the Frequency-Dropout advantage disappears or reverses, the current Fig. 2 result is a selection artifact. As a minimal check, also report error bars or bootstrap CIs for the selected UCE values.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim rests on Fig. 2, which compares the best-performing configuration of each method. The search space is 6 dropout rates × 3 layer placements = 18 configurations per method. Because UCE is computed on the same small test cohorts (n=36/37/40) used to pick the best configuration, the reported minimum is a winner's-curse estimate, not a predictive comparison. With noisy per-instance UCE, the minimum over 18 configurations will be substantially lower than a fixed configuration even when methods are equivalent, and comparing two such minima can create a spurious advantage. The paper reports no validation split, no error bars on the Fig. 2 UCE values, and no correction for multiple comparisons. This is more load-bearing than the Hermitian-symmetry issue: even if Eq. (2) is implemented correctly with rfft/irfft, the empirical claim is not established unless configuration selection is independent of the evaluation data. The paper's own Fig. 3 shows strong dependence on rate and placement, so the selection step is exactly where bias enters.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Monte-Carlo Frequency Dropout, a test-time stochastic perturbation method that applies dropout masks in the Fourier domain of feature maps, and evaluates it against standard MC Signal Dropout for uncertainty estimation in semantic segmentation. Experiments cover three public medical imaging tasks: prostate zone segmentation in biparametric MRI, liver and liver-tumor segmentation in contrast-enhanced CT, and lung segmentation in chest X-ray, using pretrained nnU-Net and MedSAM models. For each method, the authors sweep six dropout rates and three layer placements, run R=30 Monte-Carlo repetitions, and report uncertainty calibration error (UCE) and Dice divergence from the full no-dropout model. The headline claim is that the best-performing Frequency Dropout configuration achieves better-calibrated uncertainty estimates than the best-performing Signal Dropout configuration while preserving segmentation performance.","tokens_in":6672,"tokens_out":2874,"duration_ms":29228,"significance":"If the central empirical claim is established, MC-Frequency Dropout would be a simple, practical improvement for uncertainty quantification in medical image segmentation, with potential value for selective prediction and model scrutiny. The paper has concrete strengths: it uses publicly available pretrained models and datasets, it reports computational complexity for the frequency-domain operation, it compares across three imaging modalities, and it makes source code publicly available. However, the current evaluation protocol does not support the headline comparison, and the method definition in Eq. (2) leaves an essential implementation detail unspecified. The significance of the contribution is therefore conditional on correcting the evaluation protocol and clarifying the forward-pass definition.","major_comments":[{"comment":"The central comparison in Fig. 2 is based on the best-performing dropout configuration for each method, selected using UCE computed on the same test cohorts used for evaluation. With 6 dropout rates and 3 placements, this is a minimum over 18 noisy per-instance UCE estimates per method, and minima are systematically biased downward by winner's-curse effects even when the methods are equivalent. The reported test cohort sizes (n=36, 37, 40) make this bias substantial. The paper describes no validation split, no error bars on the Fig. 2 UCE values, and no correction for multiple comparisons. Because Fig. 3 shows strong dependence on rate and placement, the selection step is exactly where the bias enters. The authors should either report the full per-configuration results with confidence intervals and significance tests, or use a held-out selection protocol in which configuration choice is made on data not used for the reported comparison.","section":"Section 4, Fig. 2; Section 3.4"},{"comment":"Eq. (2) defines the frequency-diluted forward pass as C_F_theta(X) = sigma(F^{-1}(F[X * W] ⊙ D) + b), where D is a binary dropout mask. For a real-valued feature map, applying an arbitrary real binary mask to the full complex Fourier transform and then taking the inverse FFT produces a complex output unless the mask preserves Hermitian symmetry (D(k) = conj(D(-k))). The paper never states that D has this symmetry, and it does not specify whether the implementation uses a full FFT with a symmetric mask or an rfft/irfft pair. As written, the forward pass could produce complex feature maps, which would break the CNN. This is a load-bearing implementation detail for the method, and it must be stated explicitly in the paper.","section":"Section 2.2, Eq. (2)"},{"comment":"The DSC divergence values in Table 1 are difficult to interpret. For liver tumors, the baseline DSC is 0.606, yet the reported 'Divergence from baseline DSC' values are -14.53 and -11.72 for Signal Dropout and 1.23 and 3.54 for Frequency Dropout. If these are percentage-point differences they are inconsistent with DSC being in [0,1]; if they are relative percentage changes the units need to be stated. The asterisk annotation is also applied only to some entries. Since the paper claims 'stable segmentation performance' partly on the basis of this table, the metric definition and its units must be clarified, and ideally the underlying DSC values should be reported.","section":"Table 1"}],"minor_comments":[{"comment":"The caption of Fig. 3 refers to 'Expected Calibration Error (ECE)', but Section 3.5 defines the metric as Expected Uncertainty Calibration Error (UCE). These are different quantities, and the notation should be made consistent throughout.","section":"Fig. 3 caption"},{"comment":"The paper should specify whether F and F^{-1} denote the 2D DFT and its inverse with a particular normalization convention, and should state whether the implementation uses rfft/irfft on real feature maps or a full FFT with a Hermitian-symmetric mask.","section":"Section 2.2"},{"comment":"The phrase 'convergence' in the abstract and discussion is not operationalized; no convergence metric or analysis is defined in Section 3.5 or shown in the results. Consider removing or explicitly defining this claim.","section":"Section 5"},{"comment":"There is a minor wording issue: 'should be weighed to determines suitability' should be 'should be weighed to determine suitability'.","section":"Section 5, Discussion"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely within the scope of a medical-image-analysis or computer-vision venue, but the current version does not establish its central empirical claim because of the test-set-based configuration selection. The Hermitian-symmetry issue in Eq. (2) is a separate correctness concern that should be easy to fix by pointing to the implementation and adding an explicit statement. I would be willing to look at a revised version that addresses the evaluation protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: plausible, clearly written extension of frequency dropout to segmentation, with public code and three public datasets. But the headline claim is not actually established by the reported evidence, because the \"best\" dropout configuration for each method is selected on the same test set used for evaluation.\n\nWhat's new: frequency dropout itself is not new, appearing as spectral dropout [11] and in the authors' own ISBI 2024 classification paper [10]. What is new is the systematic evaluation for segmentation: three modalities (MRI, CT, X-ray), three dropout layer placements, six rates, and an analysis of how placement interacts with calibration. That is a legitimate empirical contribution, and the paper is honest about the method's dependence on hyperparameters.\n\nWhat's done well: public code, public datasets, sensible metrics (UCE for calibration, DSC for performance), and a fair comparison protocol with matched MC repetitions and rates. The discussion acknowledges limitations. No sign of overclaiming beyond a standard \"potential to enhance\" closing.\n\nThe main problem: Figure 2 compares the best-performing configuration per method, where \"best\" is found by searching 6 rates × 3 placements = 18 configurations, with UCE computed on the same small test cohorts (n=36, 37, 40). Picking the minimum UCE on the test set is a winner's-curse operation; two such minima can differ even when methods are equivalent. No validation split, no error bars, no multiple-comparison correction is reported. Since Figure 3 shows strong dependence on rate and placement, the selection step is exactly where bias enters. The central claim — that Frequency Dropout yields better-calibrated uncertainties — is therefore unverified as reported. This is more load-bearing than the Hermitian-symmetry question about Eq. (2), which is a real but minor implementation detail (using rfft/irfft with a symmetric mask fixes it; the paper just doesn't say).\n\nThere is also no significance testing for the UCE differences. The DSC table has Bonferroni-corrected p-values, but the calibration comparison does not. Minor, but worth noting.\n\nBottom line: the idea is worth testing and the paper deserves a serious referee, but as submitted the empirical advantage is not established. A revision with a held-out validation split for hyperparameter selection, error bars, and ideally a paired significance test would answer the main concern. I would send this to peer review rather than desk reject; it is a small conference paper with a fixable flaw.\n\nRecommendation: engage with it, and ask for the validation split.","headline":"Useful segmentation extension of frequency dropout, but the central calibration advantage is unverified because the best configurations are chosen on the test set; worth a referee with a requested revision.","tokens_in":7119,"tokens_out":2234,"would_cite":false,"duration_ms":22198,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Frequency dropout, applied during Monte-Carlo inference, produces uncertainty estimates that align more closely with where a pretrained segmentation model actually makes errors than standard signal dropout does, across MRI, CT, and X-ray…","keywords":["uncertainty estimation","semantic segmentation","Monte-Carlo dropout","frequency dropout","calibration","medical imaging","selective prediction"],"falsifier":"Compute the inverse Fourier transform of a real feature map after applying the binary mask as defined in the paper, on any image from the Medical Segmentation Decathlon, and check whether the result has a non-negligible imaginary component; if it does, the forward pass as specified cannot produce real feature maps, and the method depends on an unstated symmetry constraint.","tokens_in":6266,"feed_emoji":"🩺","tokens_out":4346,"duration_ms":38350,"temperature":0.7,"pith_summary":"This paper argues that dropout applied in the Fourier domain, rather than the pixel domain, produces uncertainty estimates that better reflect where a pretrained segmentation model actually fails. Using Monte-Carlo repetitions at inference time, the authors compare frequency-domain dropout with standard signal dropout across prostate MRI, liver CT, and lung X-ray segmentation. In their experiments, the best frequency-dropout configuration converged to better-calibrated uncertainty estimates, with standard deviations across repetitions aligning more closely with full-model segmentation errors. The authors also report that frequency dropout preserved segmentation accuracy, with Dice scores comparable to signal dropout and to the undiluted model.","feed_headline":"Frequency dropout aligns uncertainty maps with segmentation errors","feed_subtitle":"Across MRI, CT, and X-ray segmentation, frequency dropout beats signal dropout at matching model mistakes.","key_machinery":"Frequency Dropout is the operative mechanism: during each forward pass, the feature map is transformed with the Fourier transform, multiplied element-wise by a binary mask drawn from a Bernoulli distribution, and transformed back before the nonlinearity. Because a single deleted Fourier coefficient affects the entire spatial domain, the induced noise is global and coherent rather than local impulse noise, which the paper argues preserves structural dependencies such as object boundaries.","core_discovery":"The central claim is that stochastic attenuation of frequency components in CNN feature maps, sampled during Monte-Carlo inference, yields predictive uncertainty estimates that are better calibrated to true segmentation errors than the impulse noise introduced by standard signal dropout. The authors formalize frequency dropout as $\\mathcal{C}^F_\\theta(X) = \\sigma(\\mathcal{F}^{-1}(\\mathcal{F}[X*W] \\odot D) + b)$, where the binary mask $D$ deletes random Fourier coefficients. Across three modalities, the best-performing frequency-dropout configuration produced lower Expected Uncertainty Calibration Error and uncertainty maps whose spatial pattern matched the error regions of the full model. The method is presented as a drop-in inference-time procedure that works with pretrained models such as nnU-Net and MedSAM with no retraining.","pith_inferences":["Because the method's validity hinges on the Fourier mask preserving Hermitian symmetry, an implementation that enforces this constraint may be essential; if the released code does so, it should be documented.","The frequency-domain perturbation could be combined with test-time augmentation or with heteroscedastic noise to separate epistemic and aleatoric uncertainty.","The same principle might transfer to other global perturbations, such as wavelet-domain dropout, which could offer better spatial localization than Fourier modes.","A direct comparison of uncertainty quality against ensemble methods or MC-dropout with larger $R$ on the same pretrained models would clarify whether the calibration gain persists outside the chosen configurations."],"forward_implications":["MC-Frequency Dropout can produce uncertainty maps that better highlight voxels where a pretrained segmentation model errs, improving selective prediction and human review.","The method works without retraining, so it can be applied to any existing segmentation network first tested with signal dropout.","Best dropout placement is task-dependent: decoder dilution favored frequency dropout in liver and prostate, while encoder and global placements favored signal dropout in some settings.","Lower dropout rates generally yield better-calibrated uncertainty, except in liver tumor segmentation where higher rates were effective.","The extra FFT cost scales as $O(N \\log N)$ per feature map, a trade-off that matters for large feature maps at inference time."],"supporting_citations":[{"why":"Defines dropout in the signal domain and the operation of randomly omitting units; the baseline method the paper compares against.","marker":"[8]"},{"why":"Shows MC dropout approximates Bayesian inference, supplying the theoretical justification for using dropout at inference to obtain predictive distributions.","marker":"[9]"},{"why":"Prior work introducing Monte-Carlo frequency dropout for classification; this paper extends that approach to semantic segmentation.","marker":"[10]"},{"why":"Spectral dropout in the frequency domain; related regularization that motivates attenuating frequencies rather than pixels.","marker":"[11]"},{"why":"Medical Segmentation Decathlon; supplies the prostate MRI and liver CT datasets and the pretrained segmentation model baselines.","marker":"[12]"},{"why":"Chest X-ray lung segmentation dataset with anatomical atlas registration; one of the three evaluation tasks.","marker":"[13]"},{"why":"Chest X-ray tuberculosis screening dataset providing the X-ray images for lung segmentation.","marker":"[14]"},{"why":"nnU-Net, the self-configuring U-Net pipeline used for the prostate and liver segmentation models.","marker":"[15]"},{"why":"MedSAM, the medical-image Segment Anything variant used for the lung X-ray segmentation model.","marker":"[16]"},{"why":"Defines Expected Uncertainty Calibration Error (UCE), the primary metric used to evaluate how well uncertainty estimates match model errors.","marker":"[20]"}],"fun_headline_variants":["Frequency dropout beats signal dropout for segmentation uncertainty","Fourier dropout maps segmentation errors better than signal dropout","MC-Frequency Dropout improves calibration for MRI, CT, X-ray","Uncertainty maps align with errors using frequency dropout","Drop-in Fourier dropout improves segmentation uncertainty across modalities"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The forward pass only works as written if the real binary mask, when applied to the Fourier transform of a real feature map, preserves Hermitian symmetry so that the inverse transform returns a real signal; the paper does not state that the mask has this symmetry.","fun_headline_variants_meta":{"raw":{"variants":["Frequency dropout beats signal dropout for segmentation uncertainty","Fourier dropout maps segmentation errors better than signal dropout","MC-Frequency Dropout improves calibration for MRI, CT, X-ray","Uncertainty maps align with errors using frequency dropout","Drop-in Fourier dropout improves segmentation uncertainty across modalities"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000873,"raw_usage":{"total_tokens":3735,"prompt_tokens":860,"completion_tokens":2875,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":476,"completion_tokens_details":{"reasoning_tokens":2797}},"tokens_in":476,"tokens_out":2875,"duration_ms":17411,"temperature":1.0,"reasoning_tokens":2797,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:28:03.632021+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the inverse Fourier transform of a real feature map after applying the binary mask as defined in the paper, on any image from the Medical Segmentation Decathlon, and check whether the result has a non-negligible imaginary component; if it does, the forward pass as specified cannot produce real feature maps, and the method depends on an unstated symmetry constraint.","supporting_citations":[{"cited_title":"A review of uncertainty quantification in med- ical image analysis: probabilistic and non-probabilistic methods,","cited_arxiv_id":null,"evidence_quote":"Defines dropout in the signal domain and the operation of randomly omitting units; the baseline method the paper compares against."},{"cited_title":"Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks,","cited_arxiv_id":null,"evidence_quote":"Shows MC dropout approximates Bayesian inference, supplying the theoretical justification for using dropout at inference to obtain predictive distributions."},{"cited_title":"Automatic brain tumor segmentation us- ing convolutional neural networks with test-time aug- mentation,","cited_arxiv_id":null,"evidence_quote":"Prior work introducing Monte-Carlo frequency dropout for classification; this paper extends that approach to semantic segmentation."},{"cited_title":"Accurate and ro- bust deep learning-based segmentation of the prostate clinical target volume in ultrasound images,","cited_arxiv_id":null,"evidence_quote":"Spectral dropout in the frequency domain; related regularization that motivates attenuating frequencies rather than pixels."},{"cited_title":"Exploring uncertainty measures in bayesian deep attentive neural networks for prostate zonal segmentation,","cited_arxiv_id":null,"evidence_quote":"Medical Segmentation Decathlon; supplies the prostate MRI and liver CT datasets and the pretrained segmentation model baselines."},{"cited_title":"Exploring uncertainty measures in deep net- works for multiple sclerosis lesion detection and seg- mentation,","cited_arxiv_id":null,"evidence_quote":"Chest X-ray lung segmentation dataset with anatomical atlas registration; one of the three evaluation tasks."},{"cited_title":"Regularization of neural networks using dropconnect,","cited_arxiv_id":null,"evidence_quote":"Chest X-ray tuberculosis screening dataset providing the X-ray images for lung segmentation."},{"cited_title":"Dropout as a Bayesian approximation: Representing model uncer- tainty in deep learning,","cited_arxiv_id":null,"evidence_quote":"MedSAM, the medical-image Segment Anything variant used for the lung X-ray segmentation model."},{"cited_title":"Lung segmentation in chest radiographs using anatomical atlases with nonrigid reg- istration,","cited_arxiv_id":null,"evidence_quote":"Defines Expected Uncertainty Calibration Error (UCE), the primary metric used to evaluate how well uncertainty estimates match model errors."}],"review_version":1}