{"id":"8c89a48c-f72e-4be7-bd77-0a29acb0699d","arxiv_id":"2506.13917","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A structured evaluation framework and scorecard for explainable AI is demonstrated on heatmaps for detecting breast lesions in synthetic mammograms.","lead":"The paper proposes a four-criterion framework for evaluating AI explanations in medical imaging: consistency, plausibility, fidelity, and usefulness. It demonstrates the first three on two heatmap methods for breast-lesion detection in synthetic mammograms and introduces a scorecard for standardized reporting.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fidelity criterion is internally inconsistent: Figure 4B labels a heatmap as higher fidelity precisely when the Single Deletion Method defined in §3.2.3 would call it unfaithful, so the §3.3 screening order cannot yet filter as claimed.","rationale":"The reader's verdict is CONDITIONAL, and my concern reinforces that condition rather than changing it. The reader identified the untreated link between the three quantitative criteria and human usefulness as the weakest assumption; that is a genuine external-validation gap. However, the more fundamental problem is internal: the fidelity criterion itself is operationalized inconsistently. Figure 4B concludes 'higher fidelity' from a large heatmap change with no prediction change, directly contradicting the Single Deletion Method described in §3.2.3, which requires the most important feature to cause the largest output change. Because fidelity is one of the three required screens in the sequential evaluation, an invalid fidelity measure breaks the screening claim even before usefulness is considered. I credit the paper for clearly defining the four criteria, providing a scorecard, and transparently stating that no acceptance thresholds are given. The case study is also reproducible in principle because the synthetic M-SYNTH dataset and two CAM methods are described. But the internal inconsistency in fidelity is a correctness risk that should be addressed before the framework is adopted. My recommendation remains CONDITIONAL: the conditions should include replacing or repairing the fidelity operationalization, validating it against the stated definition, and then testing whether the three quantitative screens predict human usefulness. The reader's CONDITIONAL verdict is therefore unchanged, but the rationale is sharpened: the most load-bearing weakness is not merely that usefulness is unmeasured, but that one of the quantitative measures used to screen for usefulness is currently self-contradictory.","tokens_in":11412,"tokens_out":4205,"duration_ms":46356,"concrete_test":"Recompute the Figure 4B analysis strictly using the Single Deletion criterion from §3.2.3: for each image and each method, remove the highest-activation ROI, then measure the change in model output (e.g., detection score or logit) relative to the unperturbed image. If the output change is near zero for both Eigen CAM and Ablation CAM, both explanations should be labeled low fidelity by the paper's stated criterion, and Figure 4B's 'higher fidelity' label for Ablation CAM would be unsupported. A complementary check: compute Spearman correlation between original explanations and explanations after randomizing model layers; if this correlation does not rank Ablation CAM above Eigen CAM, the fidelity conclusion in Figure 4A also lacks support.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central screening claim in §3.3—that passing consistency, plausibility, and fidelity filters out poor or misleading explanations before human testing—requires that each quantitative criterion validly measures the construct it names. The fidelity criterion fails this requirement as presented. Section 3.2.3 defines fidelity as alignment between the explanation and the model's internal mechanisms and describes the Single Deletion Method: 'A correct explanation should show that removing the most important feature leads to the largest change in the model’s output.' Figure 4B, however, reports input-occlusion runs in which deleting the highest-activation ROI leaves the model prediction unchanged in both the left and right examples. By the paper's own criterion, neither explanation should be considered faithful, because the highlighted region is not causally important to the prediction. The caption instead concludes that the right-hand example shows 'higher fidelity' because the heatmap changed a lot. A heatmap that changes dramatically when an unimportant region is removed is unstable, not faithful. This is an internal contradiction: the operationalization of fidelity in Figure 4 is inconsistent with the definition and method stated in §3.2.3. Consequently, fidelity—one of the three quantitative screens—does not currently distinguish faithful from unfaithful explanations in a principled way. The framework's ability to filter out misleading explanations is therefore unsupported even before considering whether the three screens predict human usefulness. This is not a disagreement with an external standard but an inconsistency within the manuscript's own argument, and it directly undermines the load-bearing screening claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript proposes a framework for evaluating explainability features of AI models, primarily in medical imaging. The framework defines four criteria: consistency (stability of explanations under input perturbations), plausibility (alignment with human/ground-truth annotations), fidelity (alignment with the model's internal mechanisms), and usefulness (impact on clinician task performance). The first three are quantitative and are recommended as screens before expensive human studies; usefulness is to be assessed with human task-based studies. The authors introduce an explainability scorecard for standardized reporting and illustrate the first three criteria by comparing Eigen CAM and Ablation CAM heatmaps for a Faster R-CNN lesion detector on synthetic mammograms from the M-SYNTH dataset. The main claim is that requiring a method to pass consistency, plausibility, and fidelity filters out poor or misleading explanations before human testing.","tokens_in":11639,"tokens_out":7537,"duration_ms":76575,"significance":"If the framework were properly operationalized and validated, it would fill a real gap: standardized evaluation and reporting of XAI in medical devices, potentially reducing reliance on costly user studies and informing regulators and developers. The paper's strengths include a clear separation of four criteria, an explicit evaluation order, a scorecard for reporting, and use of synthetic data with known ground truth. However, the central screening claim is not yet supported because one of the three quantitative criteria (fidelity) is operationalized inconsistently, the plausibility metric mixes model accuracy into an explanation-quality measure, and the case study lacks statistical quantification. With revisions addressing these points, the framework could be a useful contribution.","major_comments":[{"comment":"The fidelity criterion is internally inconsistent. Section 3.2.3 defines fidelity as alignment with the model's internal mechanisms and states, under the Single Deletion Method, that 'a correct explanation should show that removing the most important feature leads to the largest change in the model's output.' In Figure 4B, deleting the highest-activation ROI leaves the model prediction unchanged in both the left and right examples. By the paper's own definition, neither explanation is faithful, because the highlighted region is not causally important to the prediction. The right-hand example is instead labeled 'higher fidelity' because the heatmap changes a lot; a heatmap that changes dramatically when an unimportant region is removed is unstable, not faithful. This contradiction means that fidelity, as demonstrated, does not distinguish faithful from unfaithful explanations, and the screening order in Section 3.3 cannot yet rest on it.","section":"Section 3.2.3, Figure 4B"},{"comment":"Plausibility is defined as alignment between the explanation and human/ground-truth annotations, but the reported 'average plausibility score' is computed from IoU and Model Accuracy (Figure 3A caption and text). Model accuracy is a property of the model, not of the explanation; averaging it into the plausibility score conflates explanation quality with model performance and can make a method look more plausible simply because the model is accurate. The manuscript should either define a pure explanation-ground-truth overlap metric or explicitly justify and sensitivity-analyze the aggregation.","section":"Section 3.2.2, Figure 3A"},{"comment":"The case study claims that Ablation CAM has 'significantly higher plausibility' (Section 3.2.2) and 'consistently exhibits higher fidelity' (Section 3.2.3), but no error bars, confidence intervals, or statistical tests are reported in Figures 2A, 3A, or 4A, and the per-plot sample sizes are not stated. With only 150 images and metric values that fluctuate substantially, the observed differences may be within noise. Add error bars, significance tests or effect-size estimates, and state the number of images and perturbation replicates used for each metric.","section":"Section 3.2.2, Section 3.2.3, Figures 2-4"},{"comment":"The central screening claim—that passing consistency, plausibility, and fidelity 'filters out poor or misleading explanations' before human testing—is asserted rather than tested. Usefulness is defined but never measured, and the paper provides no evidence that the three quantitative criteria predict whether an explanation improves clinician performance or reduces misinformation. If the quantitative screens are imperfect proxies, the recommended order could both waste resources and admit methods that fail in human use. The paper should either soften this claim to a hypothesis or report a validation study linking the quantitative criteria to usefulness outcomes.","section":"Section 3.3"}],"minor_comments":[{"comment":"There is a typo on page 3: 'explinablility' should be 'explainability'.","section":"Introduction"},{"comment":"The axis labels are confusing: the x-axis for dose is given in scientific notation (e.g., 2.04x10^10) without explaining the units or the encoding of radiation dose. Please clarify how the dose values are defined and why these particular numeric values appear.","section":"Figure 2A"},{"comment":"The citation style is inconsistent: the text refers to 'Fuxin et al. 10', and the reference list entry is alphabetized under 'Li Fuxin'. Standardize the citation format and the ordering of the author's name.","section":"Section 3.2.1"},{"comment":"The explainability scorecard is described as belonging to the proposed reporting mechanism but is only mentioned as supplementary material. Since the scorecard is part of the contribution, include at least a summary table or make the supplementary file available for review.","section":"Section 3, scorecard"},{"comment":"The M-SYNTH dataset (reference [26]) is co-authored by two of the present authors, but the manuscript does not include a conflict-of-interest or disclosure statement acknowledging this connection.","section":"Materials and Methods, M-SYNTH dataset"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a framework proposal with an illustrative case study, and its main contribution is conceptual. The lack of statistical validation in the case study and the internal inconsistency in the fidelity criterion are the primary technical concerns. Additionally, the paper would benefit from a clearer statement of how it differs from existing comprehensive evaluation frameworks such as Nauta et al. [22], beyond citing them. If the journal's scope includes framework proposals, the paper is potentially suitable after revision; if it requires empirical validation of central claims, the missing usefulness study should be addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper is a sensible, clearly written framework for evaluating explainability features in medical imaging, but the central screening claim is undercut by an internal inconsistency in its fidelity criterion. Worth a serious referee, but not as is.\n\nWhat's new: the four criteria—consistency, plausibility, fidelity, usefulness—are drawn from prior taxonomies (Nauta et al., Vilone and Longo), which the paper credits. The actual contribution is the fixed evaluation order and the scorecard for reporting. That is a legitimate incremental synthesis, not a breakthrough. The case study on two CAM methods is a reasonable feasibility demonstration, and the authors are transparent about the absence of thresholds and the need for human studies for usefulness.\n\nThe soft spot is real. Section 3.2.3 defines fidelity via the Single Deletion Method: a correct explanation should show that removing the most important feature causes the largest change in the model's output. Figure 4B, however, reports two cases where deleting the highest-activation ROI leaves the model prediction unchanged, and labels the case with the larger heatmap change as 'higher fidelity.' By the paper's own definition, neither explanation is faithful, because the highlighted region is not causally important to the prediction. A heatmap that changes a lot when an unimportant region is removed is unstable, not faithful. This isn't a quibble about an external standard; it's a contradiction inside the manuscript. Since fidelity is one of the three quantitative screens, the claim that the screening order filters out misleading explanations is unsupported as written. The fix is straightforward: redefine the fidelity example, or reconsider the operationalization.\n\nTwo lesser points. Usefulness is defined but never measured, so the paper never demonstrates that passing the three quantitative criteria predicts human usefulness. The authors acknowledge this, but the screening claim rests on it. And the case study has no error bars or significance tests; for a feasibility study that's acceptable, but it would be nice to see the raw numbers.\n\nThe citation pattern is fine; self-citation of M-SYNTH is justified because it's their dataset. Overall, this is a useful framework for clinical AI developers and regulators who need a common reporting format. A serious referee should engage with it, mainly to fix the fidelity inconsistency and to reframe the screening order as a hypothesis rather than a demonstrated filter.","headline":"The framework is worth engaging with, but the fidelity example contradicts the paper's own definition, and the screening claim is unsupported until that's fixed.","tokens_in":12217,"tokens_out":3513,"would_cite":false,"duration_ms":30230,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes that AI explanation quality can be systematically judged by four criteria—consistency, plausibility, fidelity, and usefulness—and that the first three can screen methods before costly human studies.","keywords":["explainable AI","explainability evaluation","consistency","plausibility","fidelity","usefulness","medical imaging","scorecard"],"falsifier":"Run a blinded reader study where radiologists interpret mammograms with and without heatmaps from a method that has passed high thresholds on consistency, plausibility, and fidelity. If the heatmaps do not improve diagnostic accuracy or time-to-decision—or make them worse—the three quantitative criteria have admitted a useless explanation, showing they do not guarantee usefulness.","tokens_in":11196,"feed_emoji":"🩺","tokens_out":6320,"duration_ms":58541,"temperature":0.7,"pith_summary":"The paper argues that the quality of explanations produced by AI medical devices should be judged by four distinct criteria: consistency, plausibility, fidelity, and usefulness. The first three can be computed automatically on existing data, while usefulness requires a human study, and the authors recommend evaluating them in that order so that methods failing early tests are filtered out before expensive reader studies. They demonstrate the framework by scoring two heatmap methods, Ablation CAM and Eigen CAM, on synthetic mammograms for breast lesion detection. They also package the results in an explainability scorecard for standardized reporting. The larger aim is to give developers, evaluators, and clinicians a common language for deciding whether an AI's explanation can be trusted.","feed_headline":"Four criteria filter AI explanations before costly reader studies","feed_subtitle":"Consistency, plausibility, fidelity, and usefulness give medical AI a scorecard for explainability.","key_machinery":"The machinery is an evaluation pipeline with four criteria and a companion scorecard. Consistency is measured by comparing heatmaps under perturbations such as radiation dose changes and rotation, using metrics like SSIM, MSE, and IoU. Plausibility is measured by the overlap between the heatmap's highlighted region and the known ground-truth lesion location. Fidelity is measured by model parameter randomization and deletion tests that compare how the explanation and the model's accuracy change under perturbation. Usefulness is reserved for human task-performance studies. The scorecard combines a descriptive section—method overview, context of use, limitations, and validation setting—with a quantitative section reporting the four criteria, giving stakeholders a standardized report.","core_discovery":"The central claim is that an explainability method can be systematically evaluated through four independent criteria: consistency, which measures how stable the explanation is under small input perturbations; plausibility, which measures overlap with human ground truth; fidelity, which measures how well the explanation tracks the model's actual decision mechanism; and usefulness, which measures whether the explanation improves clinician task performance. The paper establishes that the first three are quantifiable with standard image metrics on the model and data already available, and that evaluation order matters: only methods that pass all three should proceed to human testing. The case study shows Ablation CAM scores higher than Eigen CAM on plausibility and fidelity in this setting, illustrating that a method can be consistent without being trustworthy.","pith_inferences":["If the three quantitative tests turn out to predict usefulness, most reader studies for selecting explanation methods could eventually be replaced by automated benchmarking, but the paper does not provide the validation study that would establish this.","The same four-criteria template could be extended to non-heatmap explanations such as counterfactuals, prototypes, and confidence scores, provided each has a metric for consistency, plausibility, and fidelity.","The framework's fixed order is a resource-saving heuristic; interactions between criteria could make order matter in other ways, for example a highly plausible but low-fidelity method might deserve human testing precisely because it could mislead despite looking sensible.","Setting acceptance thresholds is the natural next step; thresholds could be anchored to known cases where explanations demonstrably degraded reader performance, converting the scorecard from descriptive to regulatory-usable."],"forward_implications":["Evaluation of explanation methods no longer has to begin with expensive user studies; consistency, plausibility, and fidelity can be scored automatically and used to screen candidates.","An explainability scorecard gives developers, regulators, and clinicians a standard format for comparing methods and for documenting context, limitations, and validation setting.","Because plausibility and fidelity are measured separately, an explanation can be clinically sensible while misrepresenting the model, or faithful to the model while clinically misleading; both cases should be reported.","The framework deliberately sets no pass/fail thresholds, so adopters must choose and justify thresholds per task, which may favor consistency for one application and fidelity for another.","Usefulness remains human-only in this framework, so the costliest evaluation cannot be fully automated."],"supporting_citations":[{"why":"Supplies the synthetic mammography dataset with known lesion ground truth, enabling consistency and plausibility calculations against a gold standard.","marker":"26"},{"why":"One of the two heatmap methods evaluated in the case study; provides gradient-free localization via ablation.","marker":"8"},{"why":"The other evaluated method; produces class activation maps from principal components of feature maps.","marker":"21"},{"why":"Introduces the model parameter randomization check used to measure fidelity.","marker":"1"},{"why":"Provides the deletion-based and white-box fidelity evaluation methods adopted in the framework.","marker":"22"},{"why":"States that user studies are the gold standard for assessing explanations, motivating the effort to screen methods before human testing.","marker":"13"},{"why":"Shows that intentionally incorrect AI marks degrade reader performance, grounding the claim that useless or misleading explanations are costly.","marker":"3"},{"why":"The Faster R-CNN object detector whose explanations are evaluated in the case study.","marker":"23"}],"fun_headline_variants":["AI explainability evaluated on four criteria before human testing","Scorecard grades AI explanations: consistency, plausibility, fidelity, usefulness","Four-step filter for trustworthy AI explanations in medical imaging","Ablation CAM beats Eigen CAM in explainability scorecard trial","New framework assesses AI explainability without costly reader studies"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that passing consistency, plausibility, and fidelity screens out explanations that would be useless or misleading for humans, a mapping the paper defines but does not test because usefulness is never measured.","fun_headline_variants_meta":{"raw":{"variants":["AI explainability evaluated on four criteria before human testing","Scorecard grades AI explanations: consistency, plausibility, fidelity, usefulness","Four-step filter for trustworthy AI explanations in medical imaging","Ablation CAM beats Eigen CAM in explainability scorecard trial","New framework assesses AI explainability without costly reader studies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000603,"raw_usage":{"total_tokens":2797,"prompt_tokens":912,"completion_tokens":1885,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":1802}},"tokens_in":528,"tokens_out":1885,"duration_ms":13905,"temperature":1.0,"reasoning_tokens":1802,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:25:53.809588+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a blinded reader study where radiologists interpret mammograms with and without heatmaps from a method that has passed high thresholds on consistency, plausibility, and fidelity. If the heatmaps do not improve diagnostic accuracy or time-to-decision—or make them worse—the three quantitative criteria have admitted a useless explanation, showing they do not guarantee usefulness.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the synthetic mammography dataset with known lesion ground truth, enabling consistency and plausibility calculations against a gold standard."},{"cited_title":"Sanity checks for saliency maps.Advances in neural information processing systems, 31, 2018","cited_arxiv_id":null,"evidence_quote":"Introduces the model parameter randomization check used to measure fidelity."},{"cited_title":"From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable ai.ACM Computing Surveys, 55(13s):1–42, 2023","cited_arxiv_id":null,"evidence_quote":"Provides the deletion-based and white-box fidelity evaluation methods adopted in the framework."},{"cited_title":"Darpa’s explainable ai (xai) program: A retrospective.Applied AI Letters, 2(4):e61, 2021","cited_arxiv_id":null,"evidence_quote":"States that user studies are the gold standard for assessing explanations, motivating the effort to screen methods before human testing."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows that intentionally incorrect AI marks degrade reader performance, grounding the claim that useless or misleading explanations are costly."}],"review_version":1}