REVIEW 3 major objections 5 minor 1 cited by
CoFE generates counterfactual ECGs that show how waveform changes shift a cardiac AI's predictions, and the changes match clinical knowledge in both test cases.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
CoFE edits ECG signals in a StyleGAN2 latent space so that an AI model changes its prediction, and the resulting feature changes match clinical signs for AF and hyperkalemia.
T0 review reviewed 2026-08-05 challenge →
load-bearing objection A clean demo of the authors' own prior counterfactual-XAI framework, with an AF case study; the physiological-plausibility claim outruns the evidence. the 3 major comments →
CoFE: A Framework Generating Counterfactual ECG for Explainable Cardiac AI-Diagnostics
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
The paper's central claim is that searching for counterfactual ECGs in the latent space of a pretrained generative model, rather than in raw signal space, yields edited ECGs that remain clinically plausible while shifting the AI's prediction toward a chosen target. In the atrial fibrillation case, counterfactuals that moved the classifier toward AF showed a marked drop in P-wave amplitude and a rise in RR-interval variability; in the potassium case, counterfactuals that raised the predicted potassium showed increased T-wave amplitude and prolonged QRS duration. Both feature changes are established ECG signatures of the respective clinical states, so the paper concludes that CoFE produces phy
What carries the argument
The core mechanism is gradient descent on a counterfactual loss inside the latent space of a fixed StyleGAN2 generator (a generative adversarial network architecture for high-fidelity signal synthesis). An encoder E maps a real ECG x into a latent code w0; then CoFE iteratively updates w to minimize cross-entropy between the generator's output G(w) and a user-chosen target class Yt, so that the black-box predictor f, which is only used to compute the loss and never modified, sees the target outcome. Because each update stays on the generator's learned manifold of plausible ECGs, the paper argues the resulting counterfactual remains a realistic clinical waveform rather than a distorted artifa
Load-bearing premise
The framework assumes that nudging a synthetic ECG's internal code to change a prediction always produces a believable tracing that still looks like the same patient's waveform; the paper asserts this in Section 2.2 but does not measure reconstruction fidelity, realism, or morphology preservation directly.
What would settle it
Compute the reconstruction error between an input ECG and its encoded-decoded version G(E(x)) on a held-out set, and run a blinded test in which cardiologists or automated quality metrics distinguish real ECGs from generated and counterfactual ones; if counterfactuals are systematically identifiable as artificial, or if edits meant to change one feature also alter unrelated morphology, the claim of physiologically plausible, morphology-preserving counterfactuals fails.
If this is right
- For atrial fibrillation classifiers, CoFE indicates the model is keying on reduced P-wave amplitude and increased RR-interval variability, so a clinician can see which rhythm features drive the AI's AF decision.
- For potassium-level regression, CoFE shows that the model maps taller T-waves and wider QRS complexes to higher potassium, matching the ECG presentation of hyperkalemia.
- Pairing saliency maps with counterfactuals gives a fuller explanation: where the model looks (saliency) plus what edit would change the outcome (counterfactual).
- Because the predictive model f is a plug-in component, the same framework can be applied to other ECG classification or regression tasks without retraining the generator.
- CoFE can serve as a model-validation tool: if a counterfactual edit pushes a prediction in a clinically implausible direction, that signals the model may be relying on non-physiologic cues.
Where Pith is reading between the lines
- A natural next test would be a blinded clinician study asking whether counterfactual ECGs are indistinguishable from real ECGs and whether the edited features remain diagnostically sensible; the paper reports waveform statistics but no such perceptual validation.
- Because the counterfactual is confined to the generator's latent manifold, the framework's fidelity likely depends on whether the generator was trained on cases resembling the input; rare or unusual morphologies may force edits that alter unrelated features, which the current evaluation does not check.
- The same latent-space optimization could be inverted for hypothesis testing: a clinician prescribes a feature change (e.g., raise the T-wave) and CoFE finds the closest ECG realizing it, then reports how the AI's prediction responds — a mode hinted at by the interactive sliders but not quantitatively evaluated.
- The paper reports average feature shifts but not per-patient consistency; a stronger claim would require showing that counterfactual edits remain stable across repeated runs and across similar inputs.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces CoFE, a framework that generates counterfactual 12-lead ECG signals to explain the decisions of AI-based ECG predictive models. CoFE uses a StyleGAN2 generator trained on MIMIC-IV ECGs, an encoder that maps real ECGs into the generator's latent space, and a pretrained predictive model. Counterfactuals are produced by gradient descent on a cross-entropy loss in latent space to shift the prediction toward a target class. Two case studies are presented: atrial fibrillation classification and potassium level regression. The authors report that the generated counterfactuals show clinically expected changes (reduced P-wave amplitude and increased RR variability for AF; increased T-wave amplitude and prolonged QRS duration for hyperkalemia) and claim that these results demonstrate physiologically plausible counterfactuals that align with established clinical knowledge. The paper also describes an interactive workflow and qualitative comparisons with saliency maps.
Significance. If the central claim is established, CoFE would be a useful addition to ECG XAI, providing a complementary alternative to attribution maps by generating whole-signal counterfactuals that show how a model's prediction would change. The framework's dependence on a pretrained generator and encoder is a sensible design, and the two case studies target clinically meaningful tasks. However, the current evidence does not yet support the strong conclusion of physiological plausibility. The evaluation is limited to four hand-picked features, lacks statistical detail, and does not verify that the generated signals remain realistic ECGs outside the generator's training distribution. The paper's contribution is therefore promising but incompletely validated.
major comments (3)
- [Sec. 2.2, Eq. (1)-(2)] The counterfactual loss contains only the classification term CE(f(G(w)), Y_t). There is no proximity penalty, no realism constraint, and no check that the optimized latent code stays on the generator's learned manifold. The statement that 'the resulting ECG remains a plausible sample from the clinical data distribution' is asserted without any reconstruction error, Fréchet distance, or clinician assessment. Because the optimization is unconstrained, one cannot rule out that the counterfactual is an out-of-distribution artifact. At minimum, the paper should report a distribution-level similarity measure or a human evaluation to support this load-bearing assumption.
- [Table 1 and Sec. 4.1] The quantitative evidence consists of four feature means with p-values, but no standard deviations, sample sizes, confidence intervals, or details of the statistical test. The features are selected a priori as known clinical markers, so a shift in those directions is expected if the model has learned those associations; it does not independently validate physiological plausibility. Moreover, the p-values reported as '<0.05' are insufficient without effect sizes or multiple-testing considerations. The claim that these results 'demonstrate' physiological plausibility overstates what the table can support. I recommend reporting per-sample paired differences with distributions, and ideally comparing against a baseline of random latent perturbations.
- [Sec. 4.1 / Sec. 4.2] The central claim is that the counterfactuals are 'clinically coherent' and 'grounded in physiologically meaningful features.' But the only validation is that the features move in directions consistent with textbook ECG knowledge. This is circular to a degree: the predictive model was trained on labeled data, and the counterfactual optimization pushes the model to the target class; observing that the model's learned associations align with clinical knowledge is a statement about the model, not about the realism or causal validity of the counterfactual. The paper conflates 'the model relies on features that are clinically known' with 'the counterfactual is a physiologically plausible ECG.' An independent check, such as comparing the generated counterfactuals to real ECGs from patients with the target condition or having cardiologists rate plausibility, is needed.
minor comments (5)
- [Abstract / Sec. 1] Typo: 'i,e.' should be 'i.e.' in the abstract and introduction.
- [Table 1 caption] The caption reads 'severed counterfactual ECGs' but should likely be 'severe counterfactual ECGs' (or 'generated counterfactual ECGs').
- [Sec. 2.1] The generator is said to be trained on 300,000 ECGs from MIMIC-IV, but no details are given for the encoder training (optimizer, learning rate, train/validation split). Please specify.
- [Sec. 3] The description says the evaluation sample is 'strictly disjoint' from the potassium regression training set, but is it also disjoint from the generator/encoder training sets? If not, the reconstruction could be optimistically biased. Please clarify.
- [Sec. 4.1] The statistical test used to obtain the p-values is not identified. Please state the test (e.g., paired t-test or Wilcoxon) and the sample size.
Circularity Check
No circularity in the CoFE derivation; the only self-citation is a non-load-bearing component specification.
full rationale
The paper's derivation chain is: (1) obtain a latent code w0 = E(x) from a pretrained encoder; (2) optimize w to minimize CE(f(G(w)), Y_t) with no feature-level constraints; (3) measure standard ECG features on original vs. counterfactual signals and compare them with established clinical knowledge. The counterfactual loss contains no term that directly changes P-wave amplitude, RR variability, T-wave amplitude, or QRS duration, so the observed feature changes are emergent rather than imposed. The evaluation then checks those emergent changes against externally established clinical signs (AF: suppressed P-waves and irregular RR; hyperkalemia: peaked T-waves and widened QRS), which is an empirical validation step, not a definitional equivalence. The assertion in Sec. 2.2 that latent-space optimization 'remains a plausible sample from the clinical data distribution' is an unsupported assumption and a correctness risk, but it is not circular: no equation defines plausibility in terms of the conclusion. The only self-citation is [7], used to specify the potassium regression model's training procedure; this is a component of the demo, and the paper's central claim about clinical alignment is evaluated independently of [7]'s conclusions. Therefore the framework's derivation does not reduce to its inputs, and the minor self-citation does not bear the load of the main claim.
Axiom & Free-Parameter Ledger
free parameters (2)
- learning rate eta in the latent update =
not reported
- iteration count N =
not reported
axioms (3)
- domain assumption The StyleGAN2 generator trained on MIMIC-IV ECGs produces a latent space in which gradient descent on the counterfactual loss yields ECGs that remain realistic and preserve the original morphology.
- domain assumption The predictive models f are sufficiently accurate that their decision boundaries correspond to clinically meaningful ECG features.
- domain assumption The encoder E reconstructs input ECGs well enough that the counterfactual starts from a faithful representation of the original signal.
Cite this review
Pith. "Pith review of CoFE: A Framework Generating Counterfactual ECG for Explainable Cardiac AI-Diagnostics." pith.science (2026). https://pith.science/paper/VIFDQ6ON
@misc{pith2026250816033,
author = {Pith},
title = {Pith review of: CoFE: A Framework Generating Counterfactual ECG for Explainable Cardiac AI-Diagnostics},
year = {2026},
howpublished = {\url{https://pith.science/paper/VIFDQ6ON}},
note = {Machine review of arXiv:2508.16033}
}
read the original abstract
Recognizing the need for explainable AI (XAI) approaches to enable the successful integration of AI-based ECG prediction models (AI-ECG) into clinical practice, we introduce a framework generating \textbf{Co}unter\textbf{F}actual \textbf{E}CGs (i,e., named CoFE) to illustrate how specific features, such as amplitudes and intervals, influence the model's predictive decisions. To demonstrate the applicability of the CoFE, we present two case studies: atrial fibrillation classification and potassium level regression models. The CoFE reveals feature changes in ECG signals that align with the established clinical knowledge. By clarifying both \textbf{where valid features appear} in the ECG and \textbf{how they influence the model's predictions}, we anticipate that our framework will enhance the interpretability of AI-ECG models and support more effective clinical decision-making. Our demonstration video is available at: https://www.youtube.com/watch?v=YoW0bNBPglQ.
Figures
Forward citations
Cited by 1 Pith paper
-
ExECG: An Explainable AI Framework for ECG models
ExECG is a Python framework providing Wrapper, Explainer, and Visualizer stages to unify XAI methods for ECG models and improve reproducibility.
Reference graph
Works this paper leans on
-
[1]
Oguz Akbilgic, Liam Butler, Ibrahim Karabayir, Patricia P Chang, Dalane W Kitzman, Alvaro Alonso, Lin Y Chen, and Elsayed Z Soliman. 2021. ECG-AI: electrocardiographic artificial intelligence model for prediction of heart failure. European Heart Journal-Digital Health (2021)
work page 2021
-
[2]
Zachi I Attia, David M Harmon, Elijah R Behr, and Paul A Friedman. 2021. Application of artificial intelligence to the electrocardiogram. European heart journal (2021)
work page 2021
-
[3]
Andreas Bollmann, Daniela Husser, Luca Mainardi, Federico Lombardi, Philip Langley, Alan Murray, Jose Joaquin Rieta, Jose Millet, S Bertil Olsson, Martin Stridh, et al. 2006. Analysis of surface electrocardiograms in atrial fibrillation: techniques, research, and clinical applications. Europace (2006)
work page 2006
-
[4]
Gari D Clifford, Chengyu Liu, Benjamin Moody, Li-wei H Lehman, Ikaro Silva, Qiao Li, AE Johnson, and Roger G Mark. 2017. AF classification from a short single lead ECG recording: The PhysioNet/computing in cardiology challenge
work page 2017
-
[5]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. Advances in neural information processing systems (2014)
work page 2014
-
[6]
Brian Gow, Tom Pollard, Larry A Nathanson, Alistair Johnson, Benjamin Moody, Chrystinne Fernandes, Nathaniel Greenbaum, Seth Berkowitz, Dana Moukheiber, Parastou Eslami, et al. 2023. Mimic-iv-ecg-diagnostic electrocardiogram matched subset. Type: dataset (2023)
work page 2023
-
[7]
Jong-Hwan Jang, Yong-Yeon Jo, Sora Kang, Jeong Min Son, Hak Seung Lee, Joon- myoung Kwon, and Min Sung Lee. 2024. Unveiling AI-ECG using Generative Counterfactual XAI Framework. medRxiv (2024), 2024–09
work page 2024
-
[8]
Yong-Yeon Jo, Joon-myoung Kwon, Ki-Hyun Jeon, Yong-Hyeon Cho, Jae-Hyun Shin, Yoon-Ji Lee, Min-Seung Jung, Jang-Hyeon Ban, Kyung-Hee Kim, Soo Youn Lee, et al. 2021. Detection and classification of arrhythmia using an explainable deep learning model. Journal of Electrocardiology (2021)
work page 2021
-
[9]
Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. MIMIC-III, A Freely Accessible Critical Care Database. Scientific Data (2016)
work page 2016
-
[10]
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. 2020. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
work page 2020
-
[11]
Ernst Niebur. 2007. Saliency map. Scholarpedia (2007)
work page 2007
-
[12]
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedan- tam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE interna- tional conference on computer vision
work page 2017
-
[13]
Sahil Verma, John Dickerson, and Keegan Hines. 2020. Counterfactual explana- tions for machine learning: A review. arXiv preprint arXiv:2010.10596 (2020)
Pith/arXiv arXiv 2020
-
[2017]
In 2017 computing in cardiology (CinC)
work page 2017
This paper was first reviewed by deepseek-v4-flash on August 5, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.