REVIEW 4 major objections 6 minor 26 references
Frequency-Calibrated Membership Inference Attacks on Medical Image Diffusion Models
T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper claims that membership inference attacks on medical-image diffusion models become far more accurate when the comparison is restricted to a mid-frequency band of the image, and it supports this claim with a new method, FCRE.
desk verdict FCRE is a plausible, useful MIA tweak with a clean win on ChestX-ray8, but the FeTS headline is confounded by institution and thresholds are tuned on the test sets. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the mid-frequency patch mask $L(p)$ built from Laplacian scores of the original image $\mathbf{x}_0$, applied equally to $\mathbf{x}_t$ and $\tilde{\mathbf{x}}_t$ so that both are compared only in mid-frequency regions. The attack score is the sum of an L2 reconstruction error and a structural dissimilarity, Eq. (7), $\text{MIA score} = (1-\text{SSIM}) + \|\tilde{\mathbf{x}}_t^F - \mathbf{x}_t^F\|_2$, computed after reducing both images to their selected patches.
What would settle it
Run FCRE with fixed thresholds on a medical dataset with a different frequency profile, for example CT or ultrasound images where low-frequency background dominates; if the AUC drops to the SecMI baseline or below, the reported advantage is an artifact of threshold selection rather than a general property of mid-frequency reconstruction error.
Extended reading notes
Core claim
The central claim is that frequency calibration resolves the confounding factor of inherent image difficulty in MIAs against diffusion models trained on medical images. FCRE first estimates a mid-frequency patch mask from the original image using Laplacian scores, applying percentile thresholds $L_{\min}$ and $L_{\max}$ (set to 15% and 85% in the main experiments). It then carries out a partial DDIM reverse step from $\mathbf{x}_t$ to $\mathbf{x}_{t+\Delta t}$ and back to $\tilde{\mathbf{x}}_t$, masks both $\mathbf{x}_t$ and $\tilde{\mathbf{x}}_t$ with the same patch mask, and scores membership as $(1-\text{SSIM}) + \|\tilde{\mathbf{x}}_t^F - \mathbf{x}_t^F\|_2$. The paper reports that this score separates member and non-member score distributions much better than SecMI or PIA on two medical datasets, and argues that the mid-frequency band is where the membership signal lives in medical images.
Load-bearing premise
The method assumes that a patch mask fixed by 15% and 85% Laplacian-score thresholds on the candidate image picks out a consistent mid-frequency band in both the noisy and reconstructed images, and that these thresholds, chosen on the same datasets used for evaluation, will hold on new medical data.
Editorial extensions
If this is right
- On FeTS 2022, FCRE (L2+SSIM) reports AUC 0.926 and TPR@FPR1% 0.328, compared with 0.708 and 0.052 for SecMI; on ChestX-ray8 AUC is 0.971 versus 0.907 for SecMI.
- Excluding only high or only low frequencies gives intermediate results, so the full mid-frequency mask is necessary for the reported gains.
- FCRE's advantage shrinks on low-resolution natural images (CIFAR-10), where the authors attribute the gap to limited fine-grained frequency information.
- The method provides a concrete scoring tool for auditing whether a specific medical image was part of a diffusion model's training set.
Reading between the lines
- The fixed 15% and 85% thresholds were selected on the same datasets used for evaluation, so the reported gains may overstate performance on new medical domains; a truly adaptive threshold would test whether the mid-frequency band generalizes.
- Because the Laplacian score is computed on the original candidate image, this attack requires the attacker to possess the exact image, not merely query access to the model, a constraint that bounds its real-world applicability.
- The mid-frequency membership signal might also appear in other high-frequency-rich modalities such as pathology or retinal images, but the paper does not test those domains.
- A testable defensive implication is that training diffusion models to reduce mid-frequency reconstruction asymmetry could lower MIA accuracy, though this is not investigated in the paper.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FCRE, a membership inference attack (MIA) for diffusion models trained on medical images. The method extends the SecMI inversion-symmetry idea by selecting only mid-frequency image patches (defined by Laplacian-score thresholds computed on the original image), computing masked L2 and SSIM reconstruction errors, and thresholding the combined score. Experiments on FeTS 2022, ChestX-ray8, and CIFAR-10 compare against Loss-based, SecMI, and PIA/PIAN baselines, reporting ASR, AUC, and TPR@FPR1%. The authors report large gains on the two medical datasets (e.g., FeTS AUC 0.926 and TPR@FPR1% 0.328 versus 0.708 and 0.052 for SecMI). I evaluate whether those gains are actually supported by the experimental design.
Significance. The problem is timely, and the frequency-calibration idea is a plausible and clearly motivated way to address the hard/easy sample confound in medical image reconstruction. The ChestX-ray8 random-split result is promising, and Table 2 shows that frequency masking improves over no masking. If the experimental confounds are corrected, FCRE could be a useful tool for privacy auditing of medical diffusion models. The paper does not release code or model weights, which limits reproducibility, and the method relies on several free thresholds rather than a parameter-free derivation.
major comments (4)
- [Sec. 3.1, FeTS 2022 split] The FeTS 2022 evaluation confounds membership with institutional origin. Member images are 37,000 slices from 740 cases across 16 institutions, while all 25,600 non-member images come from 511 cases of a single held-out institution, and the diffusion model is trained only on the member institutions. Consequently, the Table 1 FeTS results (AUC 0.926, TPR@FPR1% 0.328) can be explained by scanner, protocol, or anatomical domain shift between the held-out institution and the training institutions, rather than by membership. Please re-run with a patient-level split that spans institutions in both member and non-member sets, or add a control condition in which the held-out institution is included in training, so that membership and institution are not perfectly correlated.
- [Sec. 3.1, ChestX-ray8 split] The ChestX-ray8 partition is described only as independently and randomly selecting over 3,600 images for each set. Because ChestX-ray8 contains multiple images per patient, a random image-level split can place images from the same patient in both the member and non-member sets, which can inflate reconstruction-similarity-based attacks through patient anatomy similarity. Please either perform a patient-level split (all images of a patient in one set) or explicitly justify why image-level splitting is appropriate for the membership definition used.
- [Sec. 3.4, Table 2 vs. Table 1] The main results use thresholds Lmin=15% and Lmax=85%, which are selected using the Table 2 ablation on the same FeTS 2022 and ChestX-ray8 evaluation sets. There is no held-out validation for threshold choice, so the reported TPR@FPR1% and AUC improvements are optimistically biased by selection on the test data. The no-masking row in Table 2 shows that frequency masking helps, but the magnitude of the headline gains in Table 1 should be re-estimated using a nested or validation-split procedure, and fixed default thresholds should be reported as a sensitivity check.
- [Sec. 2.2, Eqs. (3)-(5)] The thresholding mechanism is under-specified. The text says the Laplacian score L(p) is computed and that patches are selected within the mid-frequency range based on thresholds applied to the mean absolute Laplacian score, and Table 2 reports Lmin and Lmax as percentages, but it is not stated whether these are per-image percentiles, dataset-wide percentiles, or absolute values, nor how the binary mask is constructed from L(p). This ambiguity prevents replication of the method and makes the ablation difficult to interpret. Please provide a precise algorithmic description and release code or pseudocode.
minor comments (6)
- [Table 1] The column header 'TPR 1%' should be 'TPR@FPR1%' for consistency, and the table formatting runs numbers together (e.g., '0.3160.810' and '0.8490.291').
- [Sec. 2.2, Eqs. (3)-(5)] The symbol L(p) is used both for the Laplacian score in Eq. (3) and for the binary mask in Eqs. (4)-(5); please rename one of the two to avoid confusion.
- [Sec. 3.3] The claim that FCRE consistently outperforms all baselines across the two medical image datasets is not supported for FeTS until the institution confound is addressed; the wording should be softened or qualified pending re-analysis.
- [Throughout] 'CIF AR-10' should be written as 'CIFAR-10' in the places where the space appears.
- [Sec. 3.4] The statement that the optimal thresholds vary across datasets is in tension with using the same 15%/85% thresholds for both medical datasets; please clarify how the same thresholds were justified if they are not optimal per dataset.
- [Sec. 3] No confidence intervals or standard deviations are reported for any of the AUC, ASR, or TPR values; given the evident sensitivity to thresholds, reporting variability across random splits or model seeds would strengthen the claims.
Circularity Check
The headline FeTS 2022 and ChestX-ray8 numbers are the threshold-fitted configuration from Table 2 reused as Table 1 results, so the leading MIA gains are partly evaluation-set fitted; the core FCRE scoring idea itself is not defined in terms of the target result.
-
fitted input called prediction
[Sec. 3.3, Table 1; Sec. 3.4, Table 2 and threshold discussion]
"Table 2: '15% 85% 0.853 0.926 0.328 0.926 0.971 0.409'; Table 1: 'FCRE (L2+SSIM)0.853 0.926 0.328 0.926 0.971 0.409'; 'the optimal thresholds for Lmin and Lmax vary across datasets. In this work, we did not perform an exhaustive search for the best thresholds for each dataset'"
The ablation in Table 2 is used to select Lmin=15% and Lmax=85%, and that exact configuration appears in Table 1 as the reported FCRE result on the same FeTS 2022 and ChestX-ray8 evaluation sets. The numerical identity between the chosen ablation row and the reported final row means the headline AUC/TPR values are fitted on the very data used to claim superiority, rather than being predictions from a fixed, dataset-independent method. The paper itself states that optimal thresholds vary across datasets, confirming the selection is data-dependent.
full rationale
The derivation chain for FCRE (Eqs. 1-7) is self-contained: it uses standard DDIM inversion, Laplacian patch scores, and SSIM, and no equation defines membership as the output of an earlier fitted quantity. The only circular step I can exhibit is empirical: Table 2's 15%/85% row is numerically identical to Table 1's FCRE L2+SSIM row, and the paper acknowledges that the optimal thresholds vary across datasets, so the reported best numbers are selected on the same evaluation sets used to report them. That is a fitted-input-called-prediction pattern and warrants a 6 under the rubric rather than a 0. I did not count the FeTS held-out-institution design as circularity; it is a validity confound (domain shift), not a reduction of a prediction to its inputs. The self-citation to [8] is motivational and not load-bearing. No other circular steps were found.
Assumptions & free parameters
free parameters (2)
- Lmin, Lmax (Laplacian frequency thresholds) =
15%, 85%
- Patch size for Laplacian score =
8x8
assumptions (3)
- domain assumption DDIM reconstruction error is lower for training members than non-members.
- domain assumption Mid-frequency (Laplacian score) patch selection on x0 is a valid proxy for discriminative frequency content in x_t and \tilde{x}_t.
- domain assumption High and low frequencies are less informative for membership than mid frequencies.
Cite this review
Pith. "Pith review of Frequency-Calibrated Membership Inference Attacks on Medical Image Diffusion Models." pith.science (2026). https://pith.science/paper/6D2NQDXI
@misc{pith2026250614919,
author = {Pith},
title = {Pith review of: Frequency-Calibrated Membership Inference Attacks on Medical Image Diffusion Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/6D2NQDXI}},
note = {Machine review of arXiv:2506.14919}
}
read the original abstract
The increasing use of diffusion models for image generation, especially in sensitive areas like medical imaging, has raised significant privacy concerns. Membership Inference Attack (MIA) has emerged as a potential approach to determine if a specific image was used to train a diffusion model, thus quantifying privacy risks. Existing MIA methods often rely on diffusion reconstruction errors, where member images are expected to have lower reconstruction errors than non-member images. However, applying these methods directly to medical images faces challenges. Reconstruction error is influenced by inherent image difficulty, and diffusion models struggle with high-frequency detail reconstruction. To address these issues, we propose a Frequency-Calibrated Reconstruction Error (FCRE) method for MIAs on medical image diffusion models. By focusing on reconstruction errors within a specific mid-frequency range and excluding both high-frequency (difficult to reconstruct) and low-frequency (less informative) regions, our frequency-selective approach mitigates the confounding factor of inherent image difficulty. Specifically, we analyze the reverse diffusion process, obtain the mid-frequency reconstruction error, and compute the structural similarity index score between the reconstructed and original images. Membership is determined by comparing this score to a threshold. Experiments on several medical image datasets demonstrate that our FCRE method outperforms existing MIA methods.
Figures
Reference graph
Works this paper leans on
-
[1]
Machine Learning: Science and Technology6(1), 015022 (2025)
Akbar, M.U., Wang, W., Eklund, A.: Beware of diffusion models for synthesizing medical images—a comparison with gans in terms of memorizing brain mri and chest x-ray images. Machine Learning: Science and Technology6(1), 015022 (2025)
2025
-
[2]
Conn Health (1), 98–105 (2022)
Ayyoubzadeh, S.M., Ayyoubzadeh, S.M., Esmaeili, M.: Clinical data sharing using generative adversarial networks. Conn Health (1), 98–105 (2022)
work page 2022
-
[3]
In: 2022 IEEE Symposium on Security and Privacy (SP)
Carlini, N., Chien, S., Nasr, M., Song, S., Terzis, A., Tramer, F.: Membership inference attacks from first principles. In: 2022 IEEE Symposium on Security and Privacy (SP). pp. 1897–1914. IEEE (2022)
work page 2022
-
[4]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Chang, Q., Qu, H., Zhang, Y., Sabuncu, M., Chen, C., Zhang, T., Metaxas, D.N.: Synthetic learning: Learn from distributed asynchronized discriminator gan with- out sharing medical image data. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 13856–13866 (2020)
work page 2020
-
[5]
In: MICCAI Workshop on Deep Generative Models
Daum, D., Osuala, R., Riess, A., Kaissis, G., Schnabel, J.A., Di Folco, M.: On differentially private 3d medical image synthesis with controllable latent diffusion models. In: MICCAI Workshop on Deep Generative Models. pp. 139–149. Springer (2024)
work page 2024
-
[6]
Duan, J., Kong, F., Wang, S., Shi, X., Xu, K.: Are diffusion models vulnerable to membership inference attacks? In: International Conference on Machine Learning. pp. 8717–8730. PMLR (2023)
work page 2023
-
[7]
Gonzalez, R.C.: Digital image processing. Pearson education (2009)
work page 2009
-
[8]
In: Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Commu- nications Security
He, Y., Li, B., Wang, Y., Yang, M., Wang, J., Hu, H., Zhao, X.: Is difficulty cal- ibration all we need? towards more practical membership inference attacks. In: Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Commu- nications Security. pp. 1226–1240 (2024)
work page 2024
Show all 26 references
-
[9]
Advances in neural information processing systems33, 6840–6851 (2020)
Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Advances in neural information processing systems33, 6840–6851 (2020)
2020
-
[10]
Medical Image Analysis88, 102846 (2023)
Kazerouni, A., Aghdam, E.K., Heidari, M., Azad, R., Fayyaz, M., Hacihaliloglu, I., Merhof, D.: Diffusion models in medical imaging: A comprehensive survey. Medical Image Analysis88, 102846 (2023)
2023
-
[11]
arXiv preprint arXiv:2305.18355 (2023)
Kong, F., Duan, J., Ma, R., Shen, H., Zhu, X., Shi, X., Xu, K.: An efficient mem- bership inference attack for the diffusion model by proximal initialization. arXiv preprint arXiv:2305.18355 (2023)
2023 arXiv
-
[12]
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
2009
-
[13]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Liu, Z., Zhang, T., He, Y., Zhang, G.: Generating progressive images from patholog- ical transitions via diffusion model. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 308–318. Springer (2024) 10 X. Zhao et al
2024
-
[14]
arXiv preprint arXiv:2402.18607 (2024)
Luo, X., Jiang, Y., Wei, F., Wu, Y., Xiao, X., Ooi, B.C.: Exploring privacy and fairness risks in sharing diffusion models: An adversarial perspective. arXiv preprint arXiv:2402.18607 (2024)
2024 arXiv
-
[15]
In: 2023 IEEE Security and Privacy Workshops (SPW)
Matsumoto, T., Miura, T., Yanai, N.: Membership inference attacks against dif- fusion models. In: 2023 IEEE Security and Privacy Workshops (SPW). pp. 77–83. IEEE (2023)
2023
-
[16]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Mokady, R., Hertz, A., Aberman, K., Pritch, Y., Cohen-Or, D.: Null-text inver- sion for editing real images using guided diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6038– 6047 (2023)
2023
-
[17]
arXiv preprint arXiv:2105.05874 (2021)
Pati, S., Baid, U., Zenk, M., Edwards, B., Sheller, M., Reina, G.A., Foley, P., Gruzdev, A., Martin, J., Albarqouni, S., et al.: The federated tumor segmentation (fets) challenge. arXiv preprint arXiv:2105.05874 (2021)
2021 arXiv
-
[18]
In: 2017 IEEE symposium on security and privacy (SP)
Shokri, R., Stronati, M., Song, C., Shmatikov, V.: Membership inference attacks against machine learning models. In: 2017 IEEE symposium on security and privacy (SP). pp. 3–18. IEEE (2017)
2017
-
[19]
arXiv preprint arXiv:2010.02502 (2020)
Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 (2020)
2020 arXiv
-
[20]
IEEE Journal of Biomedical and Health Informatics (2024)
Wang, S., Zhao, Z., Zhao, Y., Wang, L., Zhang, Y., Wang, J., Qiao, S., Lyu, Z.: A semantic conditional diffusion model for enhanced personal privacy preservation in medical images. IEEE Journal of Biomedical and Health Informatics (2024)
2024
-
[21]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classi- fication and localization of common thorax diseases. In: Proceedings of the IEEE conference on computer vision and patte...
2017
-
[22]
IEEE transactions on image processing 13(4), 600–612 (2004)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13(4), 600–612 (2004)
2004
-
[23]
In: International Conference on Learning Representations (2022)
Watson, L., Guo, C., Cormode, G., Sablayrolles, A.: On the importance of diffi- culty calibration in membership inference attacks. In: International Conference on Learning Representations (2022)
2022
-
[24]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Ye, J., Ni, H., Jin, P., Huang, S.X., Xue, Y.: Synthetic augmentation with large- scale unconditional pre-training. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 754–764. Springer (2023)
2023
-
[25]
In: European Conference on Computer Vision
Zhang, G., Lewis, J.P., Kleijn, W.B.: Exact diffusion inversion via bidirectional integration approximation. In: European Conference on Computer Vision. pp. 19–
-
[26]
arXiv preprint arXiv:2311.12397 (2023)
Zhong, N., Xu, Y., Li, S., Qian, Z., Zhang, X.: Patchcraft: Exploring texture patch for efficient ai-generated image detection. arXiv preprint arXiv:2311.12397 (2023)
2023 arXiv
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.