{"id":"adf546ce-725b-4187-9a92-d5e9421eff39","arxiv_id":"2608.06351","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A position paper reviewing why evaluating AI explanations is difficult, illustrated by the author's bias-detection and human-survey case studies and by concept-drift explanation methods.","lead":"This workshop paper argues that explanations for AI decisions are rarely evaluated rigorously and illustrates the problem with two case studies and work on evolving data streams. It is a position and experience report, not a new algorithm, dataset, or formal proof.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Survey evidence cannot support the headline preference claim: no correlation coefficients, significance tests, or controls for visualization format; ProtoPNet's win may reflect interface familiarity rather than usefulness.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing concern: the human-grounded survey in Section 3.2 is the paper's central positive evidence and it is unvalidated. My stress-test agrees and sharpens it: (a) no correlation coefficients or significance tests are reported for the 'strong correlations' sentence; (b) no variability is reported for the aggregate ranking points, so the ProtoPNet/ACE/RISE ordering has no stated error bars; (c) the uniform visualization transformation is a methodological confound because it changes the artifacts being compared; (d) the ranking task does not measure a downstream use of explanations, so the claimed preference can be read as an interface-preference result. Any of these is independently sufficient to make Section 3.2 undemonstrated; taken together they are clearly load-bearing because the paper's argument leans on this case study as its human-grounded evaluation example. I do not see a reason to shift the verdict beyond CONDITIONAL: the paper's general position (static-biased evaluation, need for temporal dynamics) is supported by cited literature and its own drift-explanation work, so the failure is in the empirical support, not the overall framework. The concrete test I propose (raw data + non-parametric tests + correlation significance) would settle the concern. I agree with the reader's weak-assumption analysis and recommend keeping the conditional verdict with the condition being release of the survey instrument, data, and significance tests.","tokens_in":9286,"tokens_out":1742,"duration_ms":18750,"concrete_test":"Obtain the raw survey responses and the actual visualization stimuli from the author or the master's thesis [6]; recompute the ranking-point totals with a non-parametric test (e.g., paired Wilcoxon signed-rank or Friedman test followed by Nemenyi post-hoc) on the per-respondent ranking points for ProtoPNet vs ACE vs RISE. Also report the Pearson/Spearman correlation and a permutation-based significance test between each respondent's 'expected features' checklist and the method whose salience map they selected. If the ProtoPNet-vs-ACE difference has p > 0.05 or the correlations are not significant after multiple-testing correction, the headline claim in Section 3.2 is unsupported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is that XAI evaluation practice is too limited and that human-grounded evaluation should be broadened; Section 3.2 is offered as the main empirical illustration. The load-bearing premise is that the 148-respondent survey measures which explanation method is genuinely more useful. That premise is not secured: the paper reports 'strong correlations' between expected animal features and chosen explanations without reporting any correlation coefficients, significance levels, or effect sizes. No statistical test is reported for the headline result that ProtoPNet (1,045 points) beats ACE (763) and RISE (643), so sampling variability alone could reverse or weaken that ordering. The survey is also confounded by presentation: all methods were converted to a 'uniform graphical interface' (custom colored line and highlighting), so the comparison tests the authors' transformation of each method, not the methods as the community would use them. Finally, the task asks respondents to select 0-2 of up to six anonymized visualizations shown side-by-side; this ranking format favors visually salient or intuitive renderings and does not operationalize usefulness for any downstream task. The paper even concedes trust was not measured ('we did not conduct additional analyses to determine whether observing the explanation increased their level of trust'). Because the empirical case study is the only direct evidence for the paper's positive recommendation that human studies should play a larger role, the weakness is load-bearing. If the survey results do not survive reanalysis, the paper still has a reasonable position argument but loses its main illustration and the specific claim that ProtoPNet is preferred overall.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper is a position/talk paper arguing that the evaluation of explainable AI (XAI) methods is insufficiently rigorous: too many new methods are introduced, human-grounded evaluation is neglected, and most existing measures are designed for static settings, whereas real-world systems face evolving data and concept drift. The author supports this position with two case studies: (1) the DetoxAI system, which uses saliency maps and concept unlearning for debiasing image classifiers under quantitative fairness metrics, and (2) a human-grounded survey comparing ACE, ProtoPNet, and RISE on animal image classification, reporting ProtoPNet as the overall preference. The paper then discusses multi-criteria counterfactual selection, prototype-based drift measures, and group counterfactual drift analysis, and closes with open issues for XAI evaluation, including the need for temporal dynamics and more systematic human studies.","tokens_in":9500,"tokens_out":3366,"duration_ms":44445,"significance":"If the central position is accepted, it would push the XAI community toward broader evaluation practices: systematic human studies, multiple complementary metrics, and explicit treatment of temporal dynamics in evolving data. The paper is valuable as a concise synthesis of evaluation taxonomies and as a concrete illustration of why static evaluation is limited. It also gives credit to the author's own prior work on prototype drift measures and group counterfactuals, which are timely contributions. However, the paper's positive empirical evidence is thin and self-referential; the only direct empirical demonstration, the human survey, lacks inferential statistics and controls, so the significance is conditional on the reader accepting the position on general grounds rather than on the presented evidence.","major_comments":[{"comment":"The human-grounded study is the paper's main empirical illustration, but it does not report any inferential statistics: there are no confidence intervals, significance tests, effect sizes, or correlation coefficients for the claimed 'strong correlations' (e.g., ACE with elephant trunk, ProtoPNet with hippo mouth or tiger stripes). The headline ranking (ProtoPNet 1,045 vs ACE 763 vs RISE 643) could plausibly lie within sampling variability, so the claim that ProtoPNet is preferred overall is not supported by the reported evidence. Additionally, because all methods were converted to a custom 'uniform graphical interface' (colored line and highlighting), the comparison evaluates the author's rendering of each method rather than the methods as commonly used by the community; this confound should be discussed as a limitation or the study should be re-analyzed and re-reported with appropriate statistical controls.","section":"Section 3.2, paragraphs beginning 'We decided to develop an online survey' and 'A total of 148 completed surveys were…"},{"comment":"The automated evaluation case study claims that 'most of them led to the elimination of the network's undesirable focus on biases' based on qualitative visual inspection of saliency maps and 'improvement of selected group-level fairness metrics,' but the actual metrics, the number of methods tested, the magnitude of improvements, and the statistical uncertainty are not reported; all details are deferred to reference [28]. Because the paper uses DetoxAI as an example of more rigorous automated evaluation, the absence of these quantitative results makes the case study unverifiable as presented. At minimum, the paper should report the actual fairness metric values and describe how the qualitative claim about saliency maps was assessed.","section":"Section 3.1, DetoxAI case study"},{"comment":"The positive proposals for multi-criteria counterfactual selection, prototype drift measures, and group counterfactual drift analysis are presented only by reference to the author's own prior works; no experimental protocol, parameter settings, datasets, or quantitative results are summarized in this manuscript. Given that the central argument is that XAI evaluation must be broadened and made more rigorous, the reader cannot verify whether these methods meet the standards the paper itself calls for. These claims should be framed as preliminary illustrations, and the key evaluation evidence from [13], [25], and [26] should be summarized in enough detail to allow the reader to judge the strength of the support.","section":"Sections 4 and 5, paragraphs on [25], [13], and [26]"}],"minor_comments":[{"comment":"The footnote in the abstract states 'accepted for a publication ... 2016' while the text refers to IJCAI-ECAI 2026 and the arXiv date is 2026; the year appears to be a typo and should be corrected.","section":"Abstract and footnote 1"},{"comment":"There is a duplicated word 'in' in 'In in the CelebA dataset'; this should be corrected.","section":"Section 3.1, sentence beginning 'In in the CelebA dataset'"},{"comment":"The phrase 'most these methods' is missing the preposition 'of' and should read 'most of these methods'.","section":"Section 5, sentence beginning 'However, most these methods'"},{"comment":"The phrase 'air space exploration' is likely intended to be 'aerospace exploration' or 'space exploration'.","section":"Section 6, final paragraph"},{"comment":"Reference [31] contains the typo 'Proceedins' in the proceedings title; it should read 'Proceedings'.","section":"Reference 31"},{"comment":"Several measurements are referenced only by external papers (e.g., the fairness metrics in [28] and the prototype measures in [13]); adding brief definitions or a short table of the main measures would make the paper more self-contained.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is a workshop-style position paper, and its strength lies in the synthesis of evaluation challenges rather than in novel empirical evidence. The heavy reliance on the author's own prior work for the positive proposals is not, by itself, a reason to reject, but the editor may wish to ask the authors to clarify the status of these self-citations and to provide independent or at least more detailed evidence for the claims in Sections 4 and 5. The main barrier to acceptance is the unsupported statistical and confounded nature of the human study, which is the only direct empirical demonstration of the paper's central recommendation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: read this as a workshop talk, not a research contribution. The central complaint — that XAI methods proliferate while evaluation stays narrow — is already made well in the papers it cites (Moshkovitz et al., Lopes et al.). What the author adds is a set of experiences: a human survey of saliency methods, a debiasing case study, and summaries of prior work on counterfactual ensembles and prototype drift measures.\n\nWhat works: The text is clear and honest. The call to take temporal dynamics seriously in XAI evaluation is a genuinely useful framing, and the drift-explanation measures from earlier papers sound relevant. The survey is a real attempt at human-grounded evaluation: pilot study, careful image selection, a uniform interface. That effort deserves credit even if the reporting falls short.\n\nThe soft spots are real and load-bearing. The survey's headline result — ProtoPNet beats ACE and RISE — has no significance tests, no correlation coefficients despite 'strong correlations' being described, and no effect sizes. Worse, all methods were converted to a custom uniform visualization, so the comparison tests the author's rendering of each method, not the methods as the community runs them. The task format (choose 0–2 of up to six) rewards visual salience, not necessarily usefulness. The paper itself concedes trust wasn't measured. For a claim that human-grounded evaluation is neglected and should be broader, this illustration is too weak to do the work.\n\nThe DetoxAI section is similarly thin: 'most methods led to elimination of bias' is reported without quantitative detail or independent benchmarks, and it leans on several self-citations. The drift-measure sections summarize earlier papers without enough description for a reader to check the claims. None of this is fatal for a position paper, but it makes the paper a synthesis of the author's prior work rather than a new result.\n\nFor whom: people new to the XAI evaluation debate who want a compact overview, or workshop attendees interested in the evolving-data angle. I would not cite it in my own work; I'd cite the underlying papers or the original critiques.\n\nRecommendation: don't desk-reject it out of hand — the author has a defensible position and real experience. But it needs a serious referee only if the venue is a workshop or a position track. For a research journal, it would need the survey data, statistical tests, and independent evaluation of the drift measures. If I were an editor, I'd send it to review for the workshop, with a note that the empirical claims need significant support.","headline":"A coherent workshop position that reinforces existing critiques of XAI evaluation; its own survey is too thinly reported to carry the preference claim.","tokens_in":10091,"tokens_out":2854,"would_cite":false,"duration_ms":33888,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Explainable AI is producing many new explanation methods while evaluating them too narrowly, and this paper argues the field should shift toward systematic human-grounded studies and evaluation that tracks how explanations behave as data…","keywords":["Explainable AI","XAI evaluation","Human-grounded evaluation","Saliency maps","Counterfactual explanations","Concept drift","Evolving data streams","Bias detection"],"falsifier":"Rerun the ranking task with the same three methods but change only the rendering style, such as outline-plus-highlight versus heatmap, while keeping the underlying saliency maps identical; if the ProtoPNet preference of 1045 versus 763 versus 643 points shrinks, reverses, or tracks the style rather than the method, the survey's conclusion that ProtoPNet is the most useful explanation would be an artifact of interface design.","tokens_in":9061,"feed_emoji":"🧪","tokens_out":5475,"duration_ms":64641,"temperature":0.7,"pith_summary":"This paper argues that explainable AI (XAI) is bottlenecked by how it is evaluated, not by how many methods it produces: new explanation methods keep appearing while their usefulness is checked in narrow, mostly automatic ways. It makes the case through two static-image case studies and two evolving-data studies. In a 148-respondent human survey, three saliency-map methods received clearly different preference scores, and preferences shifted with the animal and the respondent's XAI experience, showing that \"which explanation is best\" depends on context. For dynamic settings, the paper claims that static explanations go stale and that concept drift should be explained by watching prototypes and group counterfactuals move over time. If this is right, trustworthy AI deployment in high-stakes fields needs a much broader evaluation practice than current benchmarks provide.","feed_headline":"Explainable AI is overproducing methods and under-testing them","feed_subtitle":"A case-based review argues human studies and temporal checks are the neglected core of trustworthy AI.","key_machinery":"The argument runs on a three-level evaluation taxonomy distinguishing application-grounded, human-grounded, and functionally-grounded evaluation, plus a set of case studies that put the taxonomy to work. The load-bearing mechanisms are: saliency maps and Concept-wise Relevance Propagation for locating bias in image models; a uniform-visualization online ranking survey for comparing explanation methods; an ensemble-of-explainers approach with a Pareto front and Ideal Point Method for choosing counterfactuals; and prototype-displacement measures, feature parallel plots, and cosine similarity between evolving group-counterfactual vectors for detecting and localizing concept drift. Together these mechanisms show how evaluation changes when the audience, the visualization format, and the time dimension are taken seriously.","core_discovery":"The central claim is that XAI's bottleneck is evaluation, not invention. The paper asserts that in XAI there is a trend toward constantly introducing more and more new methods while assessment of their usefulness is too limited, that human-grounded evaluation is neglected, and that existing measures focus on static settings while explanations for evolving data require taking into account longer temporal dynamics and the co-evolution of data, models, and explanations. It supports this with case studies: a bias-detection pipeline using Concept-wise Relevance Propagation saliency maps and concept unlearning, a human survey in which ProtoPNet outscored ACE and RISE overall but not for every animal or every user group, and counterfactual and prototype-based analyses showing that drift can be tracked through moving explanation vectors. The paper also concludes that there is no single definitive best counterfactual method and that evaluation should embrace multiple criteria, including Pareto-style selection.","pith_inferences":["If visualization format drives ranking as much as method identity, then many published saliency-map comparisons may be measuring interface design; a controlled rendering swap would settle this. The paper itself flags format standardization as a challenge but does not test format as an independent variable.","The same temporal machinery of prototype displacement and group-counterfactual cosine similarity could be transferred from stream classifiers to continual learning and online reinforcement learning, where explanations also need to track non-stationarity.","A natural extension is to define 'temporal fidelity' and 'temporal stability' metrics for explanations, analogous to the static proxy measures the paper criticizes, so that drift in explanation quality can be monitored automatically alongside drift in model accuracy.","The multi-criteria/Pareto view of counterfactual quality suggests a general evaluation stance: replace 'best method' competitions with trade-off surfaces, and let the user's task pick the point on the surface."],"forward_implications":["Deployment of XAI in high-stakes settings should treat human-grounded evaluation as a required component, not an optional supplement, because method rankings change with the audience and the task.","No explanation method should be declared best on the basis of automatic proxy metrics alone; the same method can win for one animal or task and lose for another.","Counterfactual evaluation should stop looking for a single winner and instead use ensembles plus Pareto-style multi-criteria selection, which also matches how humans weigh trade-offs.","For evolving data streams, explanations must be evaluated over time: prototype movement and group-counterfactual vector changes can reveal drift location and feature-level causes before model accuracy degrades.","Research should invest in benchmark datasets with annotated ground-truth explanations, because proxy measures can pass while missing the features a ground-truth rule actually uses."],"supporting_citations":[{"why":"Position argument that XAI methods proliferate while usefulness assessment stays too limited, forming the paper's core complaint.","marker":"[20]"},{"why":"Review supplying the statistics on how few XAI papers run user studies and the list of missing evaluation standards.","marker":"[16]"},{"why":"Defines the three-level taxonomy of application-grounded, human-grounded, and functionally-grounded evaluation used throughout.","marker":"[5]"},{"why":"Experimental demonstration that high-fidelity proxy explanations can miss ground-truth rule conditions and feature weights.","marker":"[8]"},{"why":"Survey of counterfactual methods and measures showing that no single definitive best method exists.","marker":"[9]"},{"why":"Source of the ensemble-of-explainers plus Pareto-front and Ideal Point selection approach for counterfactuals.","marker":"[25]"},{"why":"Source of the prototype-based drift explanation and the prototype-comparison measures used on evolving data streams.","marker":"[13]"},{"why":"Source of the group-counterfactual drift analysis with cosine-similarity tracking of evolving vectors.","marker":"[26]"},{"why":"Source of the DetoxAI bias-detection pipeline and saliency-map fairness evaluation metrics.","marker":"[28]"},{"why":"Establishes that explanation methods are mostly static and must be adapted for concept-drift settings.","marker":"[12]"}],"fun_headline_variants":["XAI's real bottleneck: evaluation, not invention","Explainable AI: too many methods, too few tests","Evaluation, not invention, limits explainable AI","Human studies and drift checks: XAI's missing core","Explaining AI fails when testing is an afterthought"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the 148-person online survey, with its fixed set of 10 animal images and a custom uniform visualization, actually measures which explanation method is more useful rather than which visualization style looks more intuitive.","fun_headline_variants_meta":{"raw":{"variants":["XAI's real bottleneck: evaluation, not invention","Explainable AI: too many methods, too few tests","Evaluation, not invention, limits explainable AI","Human studies and drift checks: XAI's missing core","Explaining AI fails when testing is an afterthought"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000156,"raw_usage":{"total_tokens":1172,"prompt_tokens":854,"completion_tokens":318,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":470,"completion_tokens_details":{"reasoning_tokens":241}},"tokens_in":470,"tokens_out":318,"duration_ms":3992,"temperature":1.0,"reasoning_tokens":241,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:36:59.319418+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the ranking task with the same three methods but change only the rendering style, such as outline-plus-highlight versus heatmap, while keeping the underlying saliency maps identical; if the ProtoPNet preference of 1045 versus 763 versus 643 points shrinks, reverses, or tracks the style rather than the method, the survey's conclusion that ProtoPNet is the most useful explanation would be an artifact of interface design.","supporting_citations":[{"cited_title":"In: Proceedings of the 43rd International Conference on Machine Learning (ICML) (2026)","cited_arxiv_id":null,"evidence_quote":"Position argument that XAI methods proliferate while usefulness assessment stays too limited, forming the paper's core complaint."},{"cited_title":"Applied Sciences12(19) (2022)","cited_arxiv_id":null,"evidence_quote":"Review supplying the statistics on how few XAI papers run user studies and the list of missing evaluation standards."},{"cited_title":"Artificial Intelligence291, 103428 (02 2021)","cited_arxiv_id":null,"evidence_quote":"Experimental demonstration that high-fidelity proxy explanations can miss ground-truth rule conditions and feature weights."},{"cited_title":"Data Mining and Knowledge Discovery pp","cited_arxiv_id":null,"evidence_quote":"Survey of counterfactual methods and measures showing that no single definitive best method exists."},{"cited_title":"International Journal of Applied Mathematics and Computer Science34(1), 119– 133 (2024)","cited_arxiv_id":null,"evidence_quote":"Source of the ensemble-of-explainers plus Pareto-front and Ideal Point selection approach for counterfactuals."},{"cited_title":"Information Fusion p","cited_arxiv_id":null,"evidence_quote":"Source of the prototype-based drift explanation and the prototype-comparison measures used on evolving data streams."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the group-counterfactual drift analysis with cosine-similarity tracking of evolving vectors."},{"cited_title":"Investigating the Relationship Between Debiasing and Artifact Removal using Saliency Maps","cited_arxiv_id":"2503.00234","evidence_quote":"Source of the DetoxAI bias-detection pipeline and saliency-map fairness evaluation metrics."},{"cited_title":"Neurocomputing555, 126640 (2023)","cited_arxiv_id":null,"evidence_quote":"Establishes that explanation methods are mostly static and must be adapted for concept-drift settings."}],"review_version":1}