{"id":"c949139f-1eeb-488a-8684-3f93d67cea0e","arxiv_id":"2412.05592","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"XAI faithfulness evaluation can be flipped by changing perturbation type, partition size, or normalization, allowing a chosen explanation method to win, with a ranking-based mitigation called Mean Resilience Rank.","lead":"Explanations from AI systems are often judged by faithfulness scores, but the scores depend heavily on hyperparameters that researchers choose freely. This paper shows that by picking different, still common, hyperparameters, a researcher can make almost any explanation method look like the best, and it proposes a ranking method to reduce this vulnerability.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim overstates generalizability: the feasible hyperparameter set is asserted, not measured, so the demonstrated reversals may not correspond to configurations real faithfulness evaluations actually use.","rationale":"The reader's weakest assumption identifies the same gatekeeping premise: the manipulation is only a threat if the hyperparameters in the feasible set are genuinely acceptable and actually used in the field. The paper gives citations for each option but no quantitative evidence of prevalence, and the missing Appendix A means we cannot even verify which configurations were selected in the reported manipulations. I agree with the reader that this is the most load-bearing condition for the central claim. The paper does provide a credible empirical demonstration that rank reversals occur within its chosen grid, and that is independent support for a weaker sensitivity claim. However, the stronger claim in the abstract and conclusion — that 'seemingly innocent' changes to 'widely used' settings can completely change real faithfulness evaluations — requires the field-distribution premise to hold. The sign inconsistency in Definitions 1-2 is a separate internal weakness that undermines the formal framing, but it is not the primary gatekeeper for the central empirical claim. The paper also honestly acknowledges that MRR depends on the same fragile feasible-set choice and that OOD effects may confound the curves, but these limitations do not supply the missing frequency evidence. Therefore the reader's CONDITIONAL verdict remains appropriate, and no verdict adjustment is needed.","tokens_in":14311,"tokens_out":7026,"duration_ms":84133,"concrete_test":"Conduct a systematic survey of faithfulness evaluation papers (e.g., all papers citing Quantus/OpenXAI plus a keyword search for 'faithfulness' and 'pixel perturbation') and record the exact hyperparameters used. Then recompute Tables 3-7 with the feasible set restricted to configurations that appear in at least 5% of surveyed papers. If the base-to-manipulated reversals disappear or become rare, the 'widely used' premise is unsupported; if they persist, the central claim is strengthened. Independently, rerun the code from the linked repository for one dataset and check that the reported 'towards' tables are reproduced; this would also adjudicate the Definition 2 sign issue.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing premise is in Sec. 7.1 / Table 2: the three hyperparameters (partition size, perturbation function, normalization) and their values are presented as 'common choices in the literature' with citations [7,15,24,26,53], but no frequency data are given. The abstract and Sec. 1 claim manipulation uses 'hyperparameters that are widely used in the literature' and 'seemingly innocent changes'; unless each of the 18 configurations is actually used by a non-negligible share of published faithfulness evaluations, the demonstrated reversals are an artifact of an author-selected grid rather than a realistic threat. The paper references 'Appendix A' for counts of how often each manipulated hyperparameter was selected, but that appendix is missing from this version, so the plausibility of the chosen manipulated settings cannot be checked. A related issue is Definition 2's sign: F is an AUC to be minimized, but the objective maximizes F for the focus method minus the sum for others, which contradicts the reported 'towards LRP' tables. This makes the formal manipulation framing unreliable, but the field-distribution premise is more load-bearing because the empirical ranking reversals could still exist even if the optimization definitions were corrected.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that quantitative faithfulness evaluation of XAI methods is vulnerable to manipulation through hyperparameter choices. The authors formalize two attacks: intra-manipulation, which improves the score of a single explanation method, and inter-manipulation, which reorders the ranking of several methods. They demonstrate across MNIST, FashionMNIST, PneumoniaMNIST, and ImageNet that exhaustive search over 18 hyperparameter configurations (partition size, perturbation function, normalization) can change which of LRP, Saliency, and KernelSHAP is ranked best. They also propose Mean Resilience Rank (MRR), a ranking-based aggregation over the feasible hyperparameter set, as a mitigation. The empirical demonstrations do show ranking reversals, but the formal definitions contain sign errors, the feasible set is asserted rather than measured, and the results are reported without uncertainty quantification.","tokens_in":14498,"tokens_out":8223,"duration_ms":73649,"significance":"If the central result holds, the paper makes a valuable and provocative point: faithfulness evaluations on black-box image classifiers are not a stable basis for comparing XAI methods unless hyperparameter choices are constrained or reported comprehensively. The paper's strengths include a clear and reproducible experimental setup with linked code, a concrete feasible-set definition, and a constructive mitigation proposal in MRR. The implications are substantial because faithfulness metrics are among the most widely used quantitative evaluation tools in XAI. However, the significance is tempered by the fact that the attack's realism depends on the assertion that all 18 configurations are 'common' choices; the paper provides citations but no frequency data. The formal definitions also need correction before the method can be reproduced from the text.","major_comments":[{"comment":"The optimization objectives are inconsistent with the reported results. Because the paper fixes the aggregation as AUC and states that a lower faithfulness score is better (Section 5.1, Section 8 caption), Definition 2's objective 'maximize F(f,x,em,a,b,c) - sum_{m' != m} F(f,x,em',a,b,c)' would select a configuration where the focus method has a high (bad) score and the other methods have low (good) scores, which is the opposite of 'towards LRP' or 'towards Saliency.' The reported tables, e.g. Table 4 for MNIST, show the focus method with the lowest manipulated score (LRP 37.79 vs Saliency 46.23 and KernelSHAP 50.77), indicating that the implementation minimized F(em) relative to the others. The objective should be reversed, e.g. minimize F(em) - sum_{m' != m} F(em') or maximize the sum of the non-focus scores minus the focus score. Definition 1 has the analogous problem: intra-manipulation in Table 3 reduces AUC values, so the direction of optimization should be minimization, not maximization as written. Please correct the sign errors in the definitions and confirm that the code follows the corrected objectives.","section":"Section 4, Definitions 1 and 2"},{"comment":"The feasible set of hyperparameters is presented as 'based on common choices in the literature' with citations [7,15,24,26,53] and [3,40,46] and [10,11,24], but no quantitative evidence is given that each of the 18 configurations is actually used by a non-negligible share of published faithfulness evaluations. The abstract and introduction claim that the manipulation uses 'hyperparameters that are widely used in the literature' and 'seemingly innocent changes'; without frequency data, the demonstrated reversals could be an artifact of an author-selected grid rather than a realistic threat. The paper explicitly references 'Appendix A' for a summary of how often each hyperparameter occurs in the manipulated set, but Appendix A is missing from this version, so the reader cannot check the plausibility of the manipulated settings. Please add the appendix or a small survey of published hyperparameter choices, or temper the generalizability claims accordingly.","section":"Section 7.1, Table 2, and Section 8.2"},{"comment":"All reported scores are point estimates without confidence intervals or standard deviations. Since faithfulness scores are averaged over 100-1000 samples, the observed ranking changes and the 'large changes' claim should be accompanied by bootstrap intervals, per-sample variances, or repeated sampling to show that the differences are not due to sampling noise. This is especially important for ImageNet, which uses only 100 samples, and for the conclusion that the ranking can 'completely change' after manipulation.","section":"Section 8, Tables 3-7"}],"minor_comments":[{"comment":"The MRR formula is ambiguous: R returns integer ranks 0..M-1, but Eq. (5) divides by M, yielding values in [0,(M-1)/M], while Table 7 states 'a rank of 0 is best and 1 is worst.' Either the normalization should be by M-1, or R should return ranks 1..M, or the caption should be corrected to state the actual range.","section":"Section 6, Equation (5)"},{"comment":"The statement 'in some cases as much as a 130% improvement' is unclear; for example, LRP on MNIST changes from 25.20 to 7.86, which is a 69% decrease. Please define the relative change convention used for the 'improvement' percentages.","section":"Section 8.1"},{"comment":"The sentence 'For some tasks, the evaluation outcome can be manipulated such that most of the three methods achieves the best performance' should read 'all three methods' when referring to PneumoniaMNIST, and the subject-verb agreement should be fixed.","section":"Section 8.2"},{"comment":"References [9] and [10] are the same paper (Bhatt, Weller, and Moura, IJCAI 2020) and should be merged or renumbered.","section":"References"},{"comment":"The phrase 'course faithfulness curves' appears to be a typo for 'coarse faithfulness curves'.","section":"Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for the journal and addresses an important problem. The main issues are correctable: the sign errors in Definitions 1 and 2 must be fixed, the missing Appendix A should be supplied because it is cited as support for the central premise, and uncertainty quantification should be added. I would not recommend rejection on the basis of the skeptical concern about the feasible set, because the paper does cite prior work for each choice, but the authors need to either provide the promised frequency summary or substantially soften the 'widely used' claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe empirical result here is real: a small grid search over 18 acceptable-looking hyperparameter settings can flip which of LRP, Saliency, and KernelSHAP wins a faithfulness evaluation. That is worth knowing, and the paper demonstrates it cleanly across MNIST, FashionMNIST, PneumoniaMNIST, and ImageNet. Code is available, so the central demonstration is checkable.\n\nWhat is genuinely new is the framing of evaluation flexibility as an optimization problem—intra-manipulation and inter-manipulation—and the Mean Resilience Rank proposal. Earlier work showed sensitivity; this paper shows deliberate selection to reverse rankings. That is a modest but real step.\n\nNow the soft spots, in proportion.\n\nFirst, the formal definitions have sign errors. Faithfulness AUC is lower-is-better, but Definition 1 says \"maximize F\" and Definition 2 maximizes F(focus) minus the sum of the others. The reported tables show they actually minimized the focus score or minimized the focus minus others. So the definitions misstate the procedure that generated the results. That is fixable with a sign flip or a redefinition of F, but it has to be corrected before anyone relies on the formal framing.\n\nSecond, the claim that the hyperparameters in Table 2 are 'widely used' is asserted, not measured. The paper cites a few references and points to an Appendix A that is not in this version, so I cannot check the frequency of each setting. If the 18 settings are all genuinely common, the manipulation is a real threat. If only a subset are common, the attack is less general. The core caution—evaluations are sensitive to choices within a defensible range—still holds, but the 'seemingly innocent changes' framing would be stronger with a systematic survey of published faithfulness evaluations.\n\nThird, the mitigation, MRR, is not tested against manipulation. It is plausible that averaging ranks across settings reduces vulnerability, but they do not attempt to attack MRR. Also, the faithfulness scores are reported as point estimates without confidence intervals, which makes it harder to judge how stable the reversals are.\n\nNone of these are load-bearing flaws. The main message does not collapse. It needs a revision that fixes the definitions, adds the missing data or tempers the claim, and ideally stress-tests MRR. This paper deserves a serious referee, and I would be happy to see it published after those revisions. I would cite it as a warning about point estimates in faithfulness evaluation.\n\nBest,\n[You]","headline":"The empirical ranking-reversal result is real and useful, but the paper's formal definitions misstate their own objective and the 'widely used' hyperparameter claim lacks the survey to back it, so it needs revision before publication.","tokens_in":15017,"tokens_out":4570,"would_cite":true,"duration_ms":40769,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Faithfulness evaluations of XAI methods can be flipped by changing hyperparameters that papers treat as interchangeable.","keywords":["XAI evaluation","faithfulness","hyperparameter sensitivity","adversarial manipulation","Mean Resilience Rank","explainability","reproducibility","reliability"],"falsifier":"Count how often each hyperparameter combination from the paper's feasible set actually appears in a representative sample of published faithfulness evaluations; if a single configuration is used in the large majority of studies, then the demonstrated manipulation would rarely transfer to real published comparisons, because the claimed flexibility would not exist in practice.","tokens_in":14115,"feed_emoji":"🎭","tokens_out":10462,"duration_ms":91853,"temperature":0.7,"pith_summary":"Quantitative evaluation of explainable AI lacks ground-truth labels, so faithfulness metrics—tests of how well an explanation tracks a model's behavior as input parts are removed—depend on user-specified hyperparameters such as partition size, perturbation function, and normalization. This paper tries to show that those hyperparameters are flexible enough to be an attack surface: switching among settings the literature treats as acceptable can reverse which explanation method scores best. The authors define two manipulation objectives, demonstrate them on four image datasets and several explanation methods, and propose a ranking-based mitigation called Mean Resilience Rank. The stakes are practical: if manipulation is easy, published faithfulness comparisons cannot be taken at face value.","feed_headline":"Small tweaks to evaluation settings flip which AI explanation wins","feed_subtitle":"Changing one accepted setting can reverse the ranking, so single-configuration faithfulness results are untrustworthy.","key_machinery":"The central object is the evaluation function $F(f, x, e, a, b, c) \\to s$, which maps a classifier, an input, an explanation, and three hyperparameters to a faithfulness score, together with a feasible set $A^*_a$ of hyperparameter values regarded as acceptable in the literature. The argument treats the hyperparameters as optimization variables inside $A^*_a$ rather than as a fixed protocol: intra-manipulation maximizes $F$ for a single method, and inter-manipulation maximizes the score gap between a chosen method and the others. The experiments instantiate $F$ by partitioning the input, perturbing blocks in decreasing order of attribution sum, and computing the AUC of the resulting faithfulness curve, so the manipulable choices are concretely partition size, perturbation function, and normalization. Mean Resilience Rank then averages each explanation method's rank across all configurations in $A^*_a$, making it the proposed counterweight to the manipulation.","core_discovery":"On its own terms, the paper's central claim is that faithfulness evaluation of black-box image classifiers is manipulable. With a fixed model, fixed explanations, and a fixed evaluation metric, merely changing the perturbation function from uniform noise to Gaussian blurring can flip the best-performing explanation method, as shown for MNIST where the winner changes from Saliency to LRP while all other settings stay fixed. The paper generalizes this into two optimization problems: intra-manipulation, which chooses a hyperparameter from a feasible set to maximize one method's score, and inter-manipulation, which chooses a hyperparameter to maximize the gap between a focus method and its competitors. Experiments across MNIST, FashionMNIST, PneumoniaMNIST, and ImageNet show score changes large enough to invert rankings or even let every method be made the winner on the same data. The paper concludes that no single-configuration faithfulness score is trustworthy and proposes Mean Resilience Rank, which averages each method's rank across all feasible hyperparameter settings, as a partial remedy.","pith_inferences":["Going beyond the paper: the same flexibility likely extends to other metric families (e.g. randomisation-based sanity checks and robustness metrics) that also expose user-chosen parameters, so the manipulation may be a general property of XAI evaluation rather than a quirk of faithfulness tests.","A second-level inference: if the feasible set itself is selected by the researcher, MRR can be gamed by choosing a feasible set that happens to favor one method, so the mitigation moves the manipulation surface rather than eliminating it.","A testable extension: pre-registering the hyperparameter grid before inspecting any explanations, then reporting the full distribution of scores, should reduce the scope for this type of outcome manipulation in published evaluations."],"forward_implications":["A faithfulness score computed under a single hyperparameter configuration is not a stable measurement of explanation quality; the same methods receive different scores under equally accepted configurations.","Comparative claims such as 'method A beats method B' can be reversed by the evaluator's choice of partition size, perturbation function, or normalization, so papers reporting one configuration may be reporting an artifact of those choices.","The intra- and inter-manipulation definitions give an audit recipe: test whether an evaluation conclusion survives across the full feasible set before trusting it.","Mean Resilience Rank makes the evaluation more robust, but only at the cost of many more model evaluations and with results that still depend on which hyperparameters are included in the feasible set."],"supporting_citations":[{"why":"Establishes prior evidence that XAI evaluation outcomes are sensitive to hyperparameter choices, which is the premise the manipulation exploits.","marker":"[24]"},{"why":"Documents practitioner disagreement about evaluation choices, supporting the claim that multiple hyperparameter settings are accepted in the community.","marker":"[33]"},{"why":"Provides the faithfulness-as-AUC evaluation protocol that the manipulation targets and changes.","marker":"[43]"},{"why":"Shows that attribution baselines are data-dependent and difficult to tune, motivating perturbation-style hyperparameters as a manipulation surface.","marker":"[46]"},{"why":"Supplies a consistent evaluation strategy for attribution methods, used to justify the perturbation and aggregation choices in faithfulness evaluation.","marker":"[42]"}],"fun_headline_variants":["One perturbation change flips which explanation wins","Tweaking a perturbation can flip XAI rankings","Manipulating XAI evaluation with hyperparameter choices","XAI evaluation winners can be chosen by tweaking settings","Flexibility in XAI evaluation becomes manipulation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the hyperparameter values in the paper's feasible set (partition sizes 14/28/56 or 112/224/448, standard-normal/uniform/Gaussian-blur perturbations, normalization on/off) are all genuinely common in published practice, so switching among them looks like a normal choice rather than an unusual one.","fun_headline_variants_meta":{"raw":{"variants":["One perturbation change flips which explanation wins","Tweaking a perturbation can flip XAI rankings","Manipulating XAI evaluation with hyperparameter choices","XAI evaluation winners can be chosen by tweaking settings","Flexibility in XAI evaluation becomes manipulation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001446,"raw_usage":{"total_tokens":5815,"prompt_tokens":927,"completion_tokens":4888,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":4815}},"tokens_in":543,"tokens_out":4888,"duration_ms":30206,"temperature":1.0,"reasoning_tokens":4815,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:33:02.474597+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Count how often each hyperparameter combination from the paper's feasible set actually appears in a representative sample of published faithfulness evaluations; if a single configuration is used in the large majority of studies, then the demonstrated manipulation would rarely transfer to real published comparisons, because the claimed flexibility would not exist in practice.","supporting_citations":[{"cited_title":"Transactions on Machine Learning Research (2023), https://openreview.net/forum?id=j3FK00HyfU","cited_arxiv_id":null,"evidence_quote":"Establishes prior evidence that XAI evaluation outcomes are sensitive to hyperparameter choices, which is the premise the manipulation exploits."},{"cited_title":"IEEE Trans","cited_arxiv_id":null,"evidence_quote":"Provides the faithfulness-as-AUC evaluation protocol that the manipulation targets and changes."},{"cited_title":"In: International Conference on Machine Learning","cited_arxiv_id":null,"evidence_quote":"Supplies a consistent evaluation strategy for attribution methods, used to justify the perturbation and aggregation choices in faithfulness evaluation."}],"review_version":1}