REVIEW 4 major objections 5 minor 31 references
Harnessing EHRs for Diffusion-based Anomaly Detection on Chest X-rays
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Diff3M integrates structured electronic health records into a diffusion model via image-EHR cross-attention and checkerboard masking, reporting state-of-the-art unsupervised anomaly detection on MIMIC-CXR and CheXpert.
desk verdict EHR-conditioned diffusion UAD is a real first, but the SOTA claim is built on a post-hoc scoring choice; deserves a careful referee, not a desk reject. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism that carries the argument is the Image-EHR Cross Attention (IECA) module. It tokenizes the EHR row into $f$ feature tokens $F \in \mathbb{R}^{f \times d}$, computes token weights $w_r = \mathrm{softmax}(F e^\top / \sqrt{d})$ against the image embedding $e$, and forms the conditional embedding $c_r = w_r F$ that conditions both the noise-prediction (NP) and masked-pixel-generation (MPG) networks. The second mechanism, Pixel-level Checkerboard Masking (PCM), applies two complementary checkerboard masks $M_1, M_2$ to the noised input, reconstructs each masked version, and combines them as $\tilde{x}_t = \mathrm{MPG}(x_t^{m_1}) \odot M_2 + \mathrm{MPG}(x_t^{m_2}) \odot M_1$, forcing the networks to use the EHR-conditioned context to fill in deliberately obscured pixels.
What would settle it
Run the MIMIC comparison several times with different seeds and again with EHR values randomly permuted; if the 0.617-vs-0.610 AUROC gap between the full-EHR and demographics-only variants does not persist beyond run-to-run noise, or if shuffled EHR values give the same lift, then the reported EHR conditioning is not the cause.
Extended reading notes
Core claim
The paper's central claim is that structured EHR data can be turned into a conditioning signal that makes diffusion-based unsupervised anomaly detection more accurate on chest X-rays. Diff3M computes a conditional embedding from the EHR record and the X-ray image, feeds it into both the noise-prediction and masked-pixel-generation networks, and reconstructs masked images into normal-like versions; the anomaly score is the pixel difference between the original and regenerated image. On MIMIC-CXR the full model reports image-level AUROC 0.617 and AUPRC 0.821, against DiAD's 0.597 and 0.803, and on CheXpert it reports AUROC 0.664 and AUPRC 0.931, matching DiAD's AUROC while raising AUPRC. The paper attributes the improvement to EHR features (especially BMI) capturing body shape, which reduces the anatomical variability that pure image-conditioned diffusion models struggle with.
Load-bearing premise
The load-bearing premise is that the chosen EHR fields (BMI, blood pressure, height, weight, plus demographics) carry genuine information about each patient's normal anatomy, so the cross-attention embedding helps reconstruct anomalies into normal-like images rather than simply adding parameters that happen to improve the score.
Editorial extensions
If this is right
- On MIMIC-CXR, Diff3M reports AUROC 0.617 and AUPRC 0.821, exceeding DiAD (0.597/0.803) and the feature-based baselines PatchCore, RD4AD, and MambaAD.
- On CheXpert, Diff3M with demographics-only EHR matches DiAD's AUROC (0.664) while lifting AUPRC from 0.917 to 0.931.
- Ablations attribute part of the gain to the checkerboard masking strategy and part to the IECA conditioning, with the combination performing best on both datasets.
- Attention-weight analysis identifies BMI as the dominant EHR feature, especially among top-decile values, indicating that the injected clinical signal is largely about body shape.
- The choice of anomaly-scoring function matters: MSE suits MIMIC while max-absolute difference suits CheXpert, with disease-class analysis explaining the discrepancy.
Reading between the lines
- If the EHR gain is real, then richer clinical inputs—lab values, medications, prior radiology reports—should push the same mechanism further, since they carry additional anatomical and physiologic context; this is a testable prediction the paper does not make.
- The BMI-dominance result suggests a simpler variant: conditioning only on BMI and demographics might reproduce most of the MIMIC gain, which would clarify whether the extra EHR fields or the cross-attention parameters do the work.
- Because the demographics-only variant ties DiAD on CheXpert, the EHR benefit may be dataset-dependent; reporting multiple seeds and confidence intervals would show whether the 0.007 AUROC gap on MIMIC is a stable effect.
- The checkerboard masking scheme is modality-agnostic and could be evaluated on non-medical anomaly detection benchmarks, where the same masking-forces-reconstruction principle might transfer independently of EHR conditioning.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Diff3M, an unsupervised anomaly-detection method for chest X-rays that conditions a DDPM/DDIM diffusion model on structured electronic health records (EHRs). The two main technical contributions are an Image-EHR Cross Attention (IECA) module that derives an EHR-conditioned embedding from the input image and tabular features, and a Pixel-level Checkerboard Masking (PCM) strategy that masks the noised input and reconstructs it into a normal-like image during the reverse process. Training is performed on normal samples only, and anomalies are scored by the difference between the input and the reconstructed image. The method is evaluated on CheXpert and MIMIC-CXR/IV, reporting image-level AUROC/AUPRC against PatchCore, RD4AD, MambaAD, and DiAD. The authors claim state-of-the-art performance on both datasets, with the MIMIC results attributed mainly to the added EHR features.
Significance. If the empirical claims held, this would be a useful demonstration that structured clinical data can reduce anatomical-variability confounds in diffusion-based anomaly detection, and the released code would support reproducibility. The underlying diffusion machinery is standard, and the ablation structure is sensible. However, the central evidence is currently fragile: the headline comparison relies on per-dataset selection of the anomaly scoring function after seeing test results, the EHR contribution rests on a 0.007 AUROC gap with no error bars, and the CheXpert AUROC ties DiAD. The paper therefore needs substantial revision before its stated conclusions can be accepted.
major comments (4)
- [§4.2 (Tables 1–2)] The reported state-of-the-art results are obtained by selecting different anomaly scoring functions per dataset after inspecting test-set performance: maxabs for CheXpert and MSE for MIMIC. With the alternative scoring, Diff3M falls below DiAD on both datasets (CheXpert MSE AUROC 0.547 vs DiAD 0.664; MIMIC maxabs AUROC 0.568 vs DiAD 0.597). Because the scoring function is part of the detection algorithm and the selection is not pre-registered or validated on a held-out split, the comparison in Table 1 is not a single fixed procedure and the SOTA claim is not yet established.
- [Table 2] The key comparison supporting the EHR contribution is Ours_demo vs Ours_demo,ehr on MIMIC: AUROC 0.610 vs 0.617 and AUPRC 0.818 vs 0.821, a gap of 0.007 and 0.003 respectively. No error bars, confidence intervals, or significance tests are reported anywhere in the paper, so this gap is within plausible run-to-run variability; the conclusion that structured EHR features improve anomaly detection is therefore not supported by the current evidence.
- [Table 1 and Abstract] In Table 1, Diff3M ties DiAD on CheXpert AUROC (0.664 for both), while the Abstract claims the method 'achieves state-of-the-art performance, outperforming existing UAD methods.' The only CheXpert improvement is AUPRC (0.931 vs 0.917). The claim should be qualified to acknowledge the AUROC tie and to specify which metric is improved; as written, it overstates the result.
- [Eq. (1) and Algorithm 1] Equation (1) and Algorithm 1 (line 4) present a DDIM update that does not match the standard DDIM sampling formula in the cited literature. The bracket in Eq. (1) simplifies to (sqrt(1/α_t) − sqrt(1/α_{t+1}))(x_t − ε), which is not the DDIM step used in Song et al. or Wolleb et al.; as written this makes the inference procedure irreproducible and possibly inconsistent with the implementation. Please correct the equations or clarify the exact update used.
minor comments (5)
- [Table 1] The header 'Oursdemo,ehr' is typeset incorrectly; it should be 'Ours_demo,ehr'.
- [§4.2] The CheXpert rows of Table 2 omit the ehr condition; clarify that the additional EHR features are available only for MIMIC and state what this implies for the CheXpert comparison.
- [§4.1] The implementation details do not specify how the baseline methods (PatchCore, RD4AD, MambaAD, DiAD) were configured, in particular which anomaly scoring function DiAD uses; without this information the comparison in Table 1 is not fully reproducible.
- [Eq. (2)] The definition w_r = σ softmax((Fe^T)/√d) is ambiguous because both σ and softmax are used together; please clarify the exact activation and how the final conditional embedding c_r is formed.
- [Eq. (3)] The notation M_{1,2} ← (M_{1,2}×(1−s)+s) is unclear about elementwise operations and whether the masks remain binary after blending; please define the operation precisely.
Circularity Check
No circularity: Diff3M's EHR conditioning and masking are not defined in terms of its reported AUROC/AUPRC targets; results are benchmarked against external baselines.
full rationale
The derivation chain is self-contained. The IECA conditional embedding c_r is computed from tabular EHR features via a feature tokenizer and a similarity-weighted sum with the image embedding (Eq. 2); the PCM masks are static coordinate-based checkerboards scaled by time step (Eq. 3); the training loss compares predicted noise and reconstructed masked pixels against diffusion-process targets (Eq. 4); and the anomaly score is the pixel difference between the input and the DDIM-reconstructed output. None of these quantities is defined in terms of the reported detection metrics, and no network parameter is fitted to the test-set AUROC/AUPRC values. The SOTA claim is supported by comparisons to external baselines (PatchCore, RD4AD, MambaAD, DiAD). The only near-self-citation, RadFusion (ref. [31], which shares a co-author), is used as motivational background for multimodal EHR+image learning, not as a load-bearing premise for Diff3M's design or evaluation. The per-dataset choice between MSE and maxabs scoring in Sec. 4.2 is a post-hoc evaluation-selection concern and a reproducibility risk, but it is not a definitional or fitted-input circularity: the scoring function is chosen after seeing test performance rather than being derived from the method's own outputs. Overall, no step reduces by construction to its own inputs, so the circularity score is minimal.
Assumptions & free parameters
free parameters (5)
- Loss weighting lambda =
0.5
- Masking scale schedule s=t/T =
t/T
- DDIM inference steps =
600 (CheXpert), 400 (MIMIC)
- Anomaly scoring function =
maxabs for CheXpert, MSE for MIMIC
- IECA token count f and embedding dimension d =
Not reported
assumptions (4)
- standard math DDPM/DDIM diffusion processes can represent the distribution of normal chest X-rays and reconstruct anomalies as normal-like images.
- domain assumption CheXpert 'No Finding' cases define the normal class, and all other labels define anomalies.
- domain assumption Paired EHR features (age, sex, AP/PA view, and for MIMIC BMI, blood pressure, height, weight) are predictive of normal anatomical variation relevant to image reconstruction.
- domain assumption MIMIC-IV EHR records matched within three months of the X-ray correspond to the patient state at imaging time.
Cite this review
Pith. "Pith review of Harnessing EHRs for Diffusion-based Anomaly Detection on Chest X-rays." pith.science (2026). https://pith.science/paper/TFUHMVWZ
@misc{pith2026250517311,
author = {Pith},
title = {Pith review of: Harnessing EHRs for Diffusion-based Anomaly Detection on Chest X-rays},
year = {2026},
howpublished = {\url{https://pith.science/paper/TFUHMVWZ}},
note = {Machine review of arXiv:2505.17311}
}
read the original abstract
Unsupervised anomaly detection (UAD) in medical imaging is crucial for identifying pathological abnormalities without requiring extensive labeled data. However, existing diffusion-based UAD models rely solely on imaging features, limiting their ability to distinguish between normal anatomical variations and pathological anomalies. To address this, we propose Diff3M, a multi-modal diffusion-based framework that integrates chest X-rays and structured Electronic Health Records (EHRs) for enhanced anomaly detection. Specifically, we introduce a novel image-EHR cross-attention module to incorporate structured clinical context into the image generation process, improving the model's ability to differentiate normal from abnormal features. Additionally, we develop a static masking strategy to enhance the reconstruction of normal-like images from anomalies. Extensive evaluations on CheXpert and MIMIC-CXR/IV demonstrate that Diff3M achieves state-of-the-art performance, outperforming existing UAD methods in medical imaging. Our code is available at this http URL https://github.com/nth221/Diff3M
Figures
Reference graph
Works this paper leans on
-
[1]
Akcay, S., Atapour-Abarghouei, A., Breckon, T.P.: Ganomaly: Semi-supervised anomaly detection via adversarial training. In: Computer Vision–ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Re- vised Selected Papers, Part III 14. pp. 622–637. Springer (2019)
work page 2019
-
[2]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Bao,J.,Sun,H.,Deng,H.,He,Y.,Zhang,Z.,Li,X.:Bmad:Benchmarksformedical anomaly detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4042–4053 (2024)
2024
-
[3]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Bercea, C.I., Wiestler, B., Rueckert, D., Schnabel, J.A.: Diffusion models with implicit guidance for medical anomaly detection. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 211–220. Springer (2024)
work page 2024
-
[4]
In: International conference on pattern recognition
Defard,T.,Setkov,A.,Loesch,A.,Audigier,R.:Padim:apatchdistributionmodel- ing framework for anomaly detection and localization. In: International conference on pattern recognition. pp. 475–489. Springer (2021)
2021
-
[5]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Deng, H., Li, X.: Anomaly detection via reverse distillation from one-class embed- ding. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 9737–9746 (2022)
2022
-
[6]
Fan, Y., Liao, H., Huang, S., Luo, Y., Fu, H., Qi, H.: A survey of emerging appli- cations of diffusion probabilistic models in mri. Meta-Radiology p. 100082 (2024)
work page 2024
-
[7]
Neurocomputing414, 346– 355 (2020)
Feng, X., Huang, Q., Li, X.: Ultrasound image de-speckling by a hybrid deep network with transferred filtering and structural prior. Neurocomputing414, 346– 355 (2020)
work page 2020
-
[8]
Advances in Neural Information Processing Systems34, 18932–18943 (2021)
Gorishniy, Y., Rubachev, I., Khrulkov, V., Babenko, A.: Revisiting deep learning models for tabular data. Advances in Neural Information Processing Systems34, 18932–18943 (2021)
2021
Show all 31 references
-
[9]
In: International conference on medical image computing and computer- assisted intervention
Han, Z., Wang, Y., Zhou, L., Wang, P., Yan, B., Zhou, J., Wang, Y., Shen, D.: Contrastive diffusion model with auxiliary guidance for coarse-to-fine pet recon- struction. In: International conference on medical image computing and computer- assisted intervention. pp. 239–249. ...
2023
-
[10]
arXiv preprint arXiv:2404.06564 (2024)
He, H., Bai, Y., Zhang, J., He, Q., Chen, H., Gan, Z., Wang, C., Li, X., Tian, G., Xie, L.: Mambaad: Exploring state space models for multi-class unsupervised anomaly detection. arXiv preprint arXiv:2404.06564 (2024)
2024 arXiv
-
[11]
In: Proceedings of the AAAI Conference on Artificial Intelligence
He, H., Zhang, J., Chen, H., Chen, X., Li, Z., Chen, X., Wang, Y., Wang, C., Xie, L.: A diffusion-based framework for multi-class anomaly detection. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 8472–8480 (2024)
2024
-
[12]
Advances in neural information processing systems33, 6840–6851 (2020)
Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Advances in neural information processing systems33, 6840–6851 (2020)
2020
-
[13]
In: International Workshop on Machine Learning in Medical Imaging
Iqbal, H., Khalid, U., Chen, C., Hua, J.: Unsupervised anomaly detection in med- ical images using masked diffusion model. In: International Workshop on Machine Learning in Medical Imaging. pp. 372–381. Springer (2023)
2023
-
[14]
dataset with uncertainty labels and expert comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al.: Chexpert: A large chest radiograph 10 Kim et al. dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI conference on...
2019
-
[15]
physionet (2024)
Johnson, A., Bulgarelli, L., Pollard, T., Gow, B., Moody, B., Horng, S., Celi, L., Mark, R.: Mimic-iv (version 3.1). physionet (2024)
2024
-
[16]
Johnson, A., Pollard, T., Mark, R., Berkowitz, S., Horng, S.: Mimic-cxr database (version 2.0. 0). physionet2, 5 (2019)
2019
-
[17]
PhysioNet
Johnson, A., Bulgarelli, L., Pollard, T., Horng, S., Celi, L.A., Mark, R.: Mimic-iv. PhysioNet. Available online at: https://physionet. org/content/mimiciv/1.0/(accessed August 23, 2021) pp. 49–55 (2020)
2020
-
[18]
IEEE Access10, 78446–78454 (2022)
Lee, S., Lee, S., Song, B.C.: Cfa: Coupled-hypersphere-based feature adaptation for target-oriented anomaly localization. IEEE Access10, 78446–78454 (2022)
2022
-
[19]
arXiv preprint arXiv:2504.06897 (2025)
Mao, J., Wang, Y., Tang, Y., Xu, D., Wang, K., Yang, Y., Zhou, Z., Zhou, Y.: Med- segfactory: Text-guided generation of medical image-mask pairs. arXiv preprint arXiv:2504.06897 (2025)
2025 arXiv
-
[20]
arXiv preprint arXiv:2305.15956 (2023)
Mousakhan, A., Brox, T., Tayyub, J.: Anomaly detection with conditioned denois- ing diffusion models. arXiv preprint arXiv:2305.15956 (2023)
2023 arXiv
-
[21]
Diagnostics15(3), 282 (2025)
Obuchowicz, R., Lasek, J., Wodziński, M., Piórkowski, A., Strzelecki, M., Nurzyn- ska, K.: Artificial intelligence-empowered radiology—current status and critical review. Diagnostics15(3), 282 (2025)
2025
-
[22]
In: Proceedings of the 29th annual conference on Com- puter graphics and interactive techniques
Perlin, K.: Improving noise. In: Proceedings of the 29th annual conference on Com- puter graphics and interactive techniques. pp. 681–682 (2002)
2002
-
[23]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Roth, K., Pemula, L., Zepeda, J., Schölkopf, B., Brox, T., Gehler, P.: Towards total recall in industrial anomaly detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 14318–14328 (2022)
2022
-
[24]
Schlegl,T.,Seeböck,P.,Waldstein,S.M.,Langs,G.,Schmidt-Erfurth,U.:f-anogan: Fastunsupervisedanomalydetectionwithgenerativeadversarialnetworks.Medical image analysis54, 30–44 (2019)
2019
-
[25]
In: International conference on information processing in medical imag- ing
Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., Langs, G.: Unsu- pervised anomaly detection with generative adversarial networks to guide marker discovery. In: International conference on information processing in medical imag- ing. pp. 146–157. Springer (2017)
2017
-
[26]
arXiv preprint arXiv:2010.02502 (2020)
Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 (2020)
2020 arXiv
-
[27]
In: International Conference on Medical image computing and computer-assisted intervention
Wolleb, J., Bieder, F., Sandkühler, R., Cattin, P.C.: Diffusion models for medical anomaly detection. In: International Conference on Medical image computing and computer-assisted intervention. pp. 35–45. Springer (2022)
2022
-
[28]
In: Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition
Wyatt, J., Leach, A., Schmon, S.M., Willcocks, C.G.: Anoddpm: Anomaly detec- tion with denoising diffusion probabilistic models using simplex noise. In: Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 650–656 (2022)
2022
-
[29]
In: Proceedings of the AAAI conference on artificial intelligence
Yao, W., Yin, K., Cheung, W.K., Liu, J., Qin, J.: Drfuse: Learning disentangled representation for clinical multi-modal fusion with missing modality and modal inconsistency. In: Proceedings of the AAAI conference on artificial intelligence. vol. 38, pp. 16416–16424 (2024)
2024
-
[30]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision
Zhang,X.,Li,N.,Li,J.,Dai,T.,Jiang,Y.,Xia,S.T.:Unsupervisedsurfaceanomaly detection with diffusion probabilistic model. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 6782–6791 (2023)
2023
-
[31]
arXiv preprint arXiv:2111.11665 (2021)
Zhou, Y., Huang, S.C., Fries, J.A., Youssef, A., Amrhein, T.J., Chang, M., Baner- jee, I., Rubin, D., Xing, L., Shah, N., et al.: Radfusion: Benchmarking performance and fairness for multimodal pulmonary embolism detection from ct and ehr. arXiv preprint arXiv:2111.11665 (2021)
2021 arXiv
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.