{"id":"cf100353-38d5-4f84-bc9a-6bca738fdbc3","arxiv_id":"2501.11429","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A characteristic function based on weak abductive explanations yields Shapley values that give zero importance to irrelevant features, and a sample-based algorithm makes the approach practical.","lead":"The paper proposes new SHAP-style scores for explainable AI that are based on formal abductive explanations rather than on expected model outputs, claiming these scores avoid known failures of standard SHAP. It introduces a prototype, nuSHAP, that estimates the new scores efficiently from data samples and finds that its feature rankings differ markedly from SHAP's on common datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The zero-score guarantee for νSHAP_T holds only for model-based v_a; the implemented sample-based v_a has no such proof, so practical nuSHAP may assign nonzero scores to irrelevant features.","rationale":"The reader's verdict correctly identifies the sample-based vs model-based gap as the weakest point. The theoretical construction of v_a is sound and Proposition 1 is a straightforward consequence of the definition of irrelevance. However, the paper's actual tool and experiments use a sample-based characteristic function, and no theorem or bound connects the estimated scores to the model-based νSHAP_T scores. The authors' own admission that sbXps are only data-accurate reinforces this. The RBO experiments show only that nuSHAP's sample-based scores and SHAP's scores rank features differently; this does not establish that SHAP is misleading unless the sample-based scores are known to approximate model-based νSHAP_T. Thus the paper's central claim of a practically non-misleading SHAP score is not yet demonstrated. The CONDITIONAL verdict is appropriate: the theory is promising, but the implementation gap and overclaimed experimental interpretation require revision.","tokens_in":114,"tokens_out":5668,"duration_ms":72999,"concrete_test":"Use the binary model y = (x1 ∧ ¬x2) ∨ x3, instance v=(1,0,0,1). Feature 4 is irrelevant (no AXp contains it). Run nuSHAP with the paper's Algorithm 2 on the incomplete sample {(1,0,1,1), (0,0,0,0), (1,0,0,1), (0,1,0,1)}. If the estimated score for feature 4 is nonzero, the sample-based v_a violates the zero-score property of Proposition 1, confirming the implementation gap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Proposition 1 proves that with the model-based characteristic function v_a(S) = 1 iff WAXp(S) over the full feature space (Eq. 9), an irrelevant feature receives Shapley value 0. The nuSHAP implementation, however, replaces WAXp with sbWAXp over a finite sample (Eq. 11, Algorithm 2), and the paper provides no theorem showing that Proposition 1 transfers to this sample-based function. For a finite dataset, an irrelevant feature can appear critical: if every sample row with a different prediction happens to have that feature differing from its value in the instance, then fixing the feature can turn a non-WAXp into an sbWAXp, producing a nonzero Δ_i and hence a nonzero Shapley estimate. Conversely, a relevant feature can appear irrelevant when sampling misses the witnessing point. The paper explicitly concedes sample-based explanations are 'data-accurate, but not model-accurate' (Section 4.3), yet the abstract and Section 4.2 state that the issues with SHAPT 'cannot occur' for the CGT/v_a combination. That claim is only about the theoretical v_a, not the estimated νSHAPE scores used in the experiments. Therefore the headline claim of a non-misleading practical SHAP score is unsupported unless a sample-size or approximation guarantee is supplied.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes replacing the standard SHAP characteristic function v_e (the conditional expected prediction) with a new characteristic function v_a that is 1 exactly when a set of features is a weak abductive explanation (WAXp). Theoretical SHAP values based on v_a are called nuSHAP scores; the paper argues that, unlike SHAPT scores based on v_e, these scores assign zero importance to formally irrelevant features and therefore cannot produce the misleading rankings documented in earlier work. The paper pairs this definition with the Castro et al. (2009) sampling algorithm (CGT) for Shapley-value estimation, and for scalability replaces model-based WAXp checks with sample-based weak abductive explanations (sbWAXp) computed over a dataset (Algorithm 2). Experiments compare the resulting nuSHAP tool with the SHAP tool on tabular and image datasets, reporting rank-biased overlap (RBO) values that are often low and concluding that SHAP's rankings are misleading while nuSHAP's are not.","tokens_in":16864,"tokens_out":6097,"duration_ms":69795,"significance":"If the theoretical and practical claims were both fully established, the paper would make a useful contribution: it would connect feature-attribution scores with logic-based abductive explanations, provide a Shapley-based score with a formal irrelevant-feature guarantee, and give a scalable estimation procedure. The paper also deserves credit for using an estimation algorithm (CGT) with explicit error and probability guarantees, and for being explicit that sample-based explanations are data-accurate rather than model-accurate. However, the central guarantee is proven only for the model-based characteristic function, while the implemented and experimentally evaluated method uses a sample-based function; no theorem transfers the irrelevant-feature zero-score property to the estimated scores. The experimental sections treat nuSHAP's rankings as ground truth, so the comparison with SHAP shows disagreement but does not by itself establish that SHAP is misleading or that nuSHAP is not. These gaps are load-bearing for the paper's headline claims.","major_comments":[{"comment":"Proposition 1 proves the zero-score property only for the model-based characteristic function v_a defined in Eq. (9). The practical nuSHAP implementation defines its characteristic function using sample-based sbWAXps, Eq. (11), decided by Algorithm 2. No theorem shows that Proposition 1 transfers to this sample-based function. For a finite dataset, a model-irrelevant feature can become critical in the sample: if a row with a different prediction differs from v on that feature and on no other feature in the tested set S, then S∪{i} is declared an sbWAXp while S is not, yielding a nonzero sample-based Δ_i and hence a nonzero Shapley estimate. Conversely, a model-relevant feature can be missed if the witnessing sample point is absent. The paper's own statement in Section 4.3 that sample-based explanations are \"data-accurate, but not model-accurate\" directly undercuts the Section 4.2 claim that the issues with SHAPT scores \"cannot occur\" for the CGT/v_a combination, since the experiments use the sample-based v_a, not the theoretical v_a.","section":"Sections 4.2 and 4.3 (Proposition 1, Eq. (9), Eq. (11), Algorithm 2)"},{"comment":"The experiments compare nuSHAP rankings with SHAP rankings using RBO and interpret low RBO as evidence that SHAP is misleading and nuSHAP is \"guaranteed not to mislead.\" This interpretation lacks a ground truth: the RBO comparison only shows that the two methods disagree. It does not establish which features are actually relevant for each model and instance, and it cannot validate the sample-based nuSHAP scores, which may violate the theoretical zero-score property for the reasons given above. To support the claim, the paper would need to compare against formal model-based relevance (e.g., AXp/CXp membership) or use synthetic models with known relevance, or supply a sample-size or coverage guarantee under which the sample-based scores converge to the model-based nuSHAP scores.","section":"Section 5 (Table 4 and surrounding text)"},{"comment":"The paper's headline claim that the proposed SHAP scores \"overcome\" the flaws of SHAPT scores should be scoped carefully. The zero-score property is a direct consequence of defining irrelevance through WAXp membership and then defining v_a as the WAXp indicator: irrelevant features by construction cannot change WAXp status. This is a valid theorem, but it is a definitional property of the theoretical v_a, not an empirical discovery. The abstract and Section 1 present the result as if it were an experimentally confirmed property of the implemented tool. The paper should state clearly that the guarantee is for the model-based theoretical score, and that the experiments are preliminary evidence about a distinct sample-based estimator.","section":"Section 4.1 (Eq. (9)), Section 1 and Abstract"}],"minor_comments":[{"comment":"The claim that deciding an sbWAXp runs in O(m n) is imprecise: the loops are over W and the dataset rows, so the cost is O(|W| n), which is at most O(m n) but should be stated as O(|W| n) for accuracy.","section":"Section 4.3: Algorithm 2, running time"},{"comment":"The paper states that r is estimated from epsilon and alpha but does not give the formula or a reference for the bound; a page-equation pointer to Castro et al. (2009) would help the reader verify the claimed guarantee.","section":"Section 4.2: CGT error parameters"},{"comment":"The RBO results depend on the chosen persistence (0.5) and depth (5); the paper reports only these settings. Since RBO values are sensitive to depth, the conclusions about \"no correlation\" should at least acknowledge this dependence, or report a sensitivity check.","section":"Section 5: RBO parameters"},{"comment":"The phrase \"conclusive practical evidence\" in the introduction and the claim that the experiments \"demonstrate\" that SHAP is misleading are stronger than what a disagreement-based RBO comparison can support; please temper the language to match the evidence presented.","section":"Section 6 / Discussion"},{"comment":"The acronyms SHAPE, nuSHAPE, SHAPT, and nuSHAPT are introduced in Table 1 but the experimental section sometimes refers to \"SHAP scores\" without specifying whether theoretical or estimated values are meant; a consistent qualifier would prevent ambiguity.","section":"General notation"}],"recommendation":"major_revision","confidential_remarks":"The core characteristic function v_a is credited to Letoffe et al. (2025), and the paper is essentially an extended application of that prior definition with a new sampling-based implementation and experiments. The authors are prominent in this subfield and many citations are to their own work, which is normal but should be considered when choosing reviewers and when assessing the novelty claim. The main technical gap, the missing transfer of the zero-score guarantee from model-based to sample-based explanations, is fixable in principle but requires real additional work. I therefore recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the bottom line: the paper's core construction—replacing SHAP's expected-value game with the WAXp-indicator game v_a—is sound, and the CGT estimator provably assigns zero to irrelevant features when v_a is model-based. That matters. But the guarantee does not transfer to what nuSHAP actually computes. The implementation swaps model-based WAXp queries for sample-based sbWAXp checks over a finite dataset (Eq. 11, Algorithm 2), and Section 4.3 concedes these are data-accurate, not model-accurate. No theorem, bound, or sample-size argument closes that gap. So the abstract's claim that the earlier SHAP failures 'cannot occur' is established only for the theoretical νSHAP_T, not for the estimated νSHAP_E scores used in the experiments. The stress-test note is on target; the reader's weakest assumption is exactly right.\n\nCredit where it's due. Algorithm 2 is a clean O(mn) decision procedure for sbWAXp, and the experimental design—reusing SHAP's own samples so both methods see the same data, then comparing top-5 rankings with RBO—is fair and useful. Propositions 1 and 2 are correct for the model-based v_a, and the critical-element view bridges voting power and abductive explanations nicely. The heavy self-citation isn't a sin here; it's honest about where v_a came from, but it also means the novelty is in the estimator and the empirical pipeline, not the definition.\n\nThe soft spots, in order. One: the sample-based gap is load-bearing. With a finite sample, an irrelevant feature can look critical if every sampled counterexample happens to differ on it, and a relevant feature can vanish if the witnessing point isn't sampled. The paper gives no handle on how likely these errors are. Two: the experiments conclude SHAP is misleading almost everywhere, but the only evidence is ranking disagreement with nuSHAP. That presumes nuSHAP's rankings are the correct standard, which is exactly what the missing transfer theorem would need to justify. RBO near zero proves divergence, not deception. Three: the zero-score property is built into v_a, not tested by the experiments; the boolean-function section just illustrates the definition.\n\nWho should read this: anyone working on formal XAI or feature attribution should know the nuSHAP construction and the sbWAXp shortcut; it's a serious attempt to connect abductive explanations to Shapley values. It deserves a serious referee, but with major revision: prove or bound the sample-based transfer, release the code, and soften the 'misleading' language to 'rankings disagree.' I'd send it out, not desk-reject.","headline":"The WAXp-based SHAP fix is real, but the zero-score guarantee is proven only for the model-based v_a, not the sample-based estimator the experiments use.","tokens_in":17392,"tokens_out":2494,"would_cite":true,"duration_ms":26323,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["91A12"],"pacs":[],"model":"deepseek-v4-flash","headline":"Replacing SHAP's averaging rule with a logical sufficiency test makes irrelevant features score zero.","keywords":["Explainable AI","Shapley values","SHAP scores","Abductive explanations","Feature attribution","Feature relevance","Sample-based explanations","Logic-based explainability"],"falsifier":"Feed nuSHAP a model in which feature 2 is irrelevant, for example a binary classifier predicting 1 exactly when $x_1 = 1$, with target instance $((0,0),0)$, and give it a dataset containing only the row $(1,1)$. The sample-based sufficiency test judges $\\{2\\}$ to be a weak abductive explanation while $\\emptyset$ is not, so feature 2 receives a positive estimated score even though it is irrelevant; that would show the sample-based implementation does not inherit the paper's central guarantee, and a reader could run this check with any SHAP-style sampling explainer.","tokens_in":16349,"feed_emoji":"🎲","tokens_out":12615,"duration_ms":119577,"temperature":0.7,"pith_summary":"Standard SHAP scores mislead because their characteristic function averages the model's output over perturbed inputs, so regions far from the instance can distort importance. This paper claims the fix is to redefine the game: a set of features is worth 1 exactly when fixing those features already forces the prediction (a weak abductive explanation), and 0 otherwise. With that function, Shapley values provably give zero importance to every feature irrelevant to the prediction, and positive marginal contributions mean exactly that a feature is critical for making a set sufficient. The paper also shows the scores can be estimated with a sampling algorithm that has formal error guarantees, using data-sample checks of sufficiency for tractability, and reports that the resulting rankings are mostly uncorrelated with the SHAP tool's rankings across tabular and image benchmarks. The upshot, if the paper is right, is that Shapley values need not be abandoned in explainable AI; the characteristic function they are built on is what has to change.","feed_headline":"Irrelevant features score zero under new SHAP definition","feed_subtitle":"Swapping expected values for a logical sufficiency test removes known misleading rankings, the authors argue.","key_machinery":"The load-bearing object is the WAXp-indicator characteristic function $v_a(S) = 1$ if fixing the features in $S$ to the instance's values makes the prediction invariant (so $S$ is a weak abductive explanation, a WAXp), and 0 otherwise. It converts a machine-learning model into a monotone simple game, so a feature's marginal contribution is 1 exactly when the feature is critical: the coalition $S$ is not yet sufficient, but $S \\cup \\{i\\}$ is. Proposition 1, inherited from quantified relevance results, states that an irrelevant feature is never critical, which is what forces its Shapley value to zero. Estimation is carried by the CGT sampling algorithm, which averages marginal contributions over random permutations and comes with $\\epsilon,\\alpha$ guarantees; for scalability, sufficiency is decided by Algorithm 2, a linear-time scan of a dataset that checks whether any sample row with the fixed feature values contradicts the prediction.","core_discovery":"The paper's central discovery is that the known failures of SHAP scores are not a failure of Shapley values but of the expected-value characteristic function $v_e$, and that replacing it with the WAXp-indicator function $v_a(S) = 1$ iff $S$ is a weak abductive explanation for the instance yields theoretical SHAP scores, called $\\nu$SHAPT, immune to those failures. By Proposition 1, an irrelevant feature leaves $v_a$ unchanged for every coalition, so its Shapley value is 0; by Proposition 2 every marginal contribution is 0 or 1, and a 1 occurs exactly when the feature is critical, i.e. when adding it turns a non-sufficient set into a sufficient one. The paper further claims that these scores can be rigorously estimated by a sampling method with probabilistic error bounds, and that in practice the WAXp test can be carried out on a dataset sample in $O(mn)$ time, yielding the prototype nuSHAP. Experimental comparisons show the nuSHAP rankings and the SHAP tool's rankings are mostly uncorrelated, which the paper reads as further evidence that current SHAP scores are misleading while the new ones are not.","pith_inferences":["The sample-based implementation can diverge from the model-based guarantee: a dataset that omits the single row that exposes a feature's irrelevance can make an irrelevant feature look critical, so nuSHAPE is not automatically shielded by Proposition 1.","The low rank correlation with the SHAP tool is descriptive, not a certificate of correctness for nuSHAP; a direct comparison against ground-truth feature relevance, such as planted features or formal AXps, would test whether the new rankings are actually better rather than merely different.","The same $v_a$ game could support other game-theoretic indices, for example the Banzhaf index, or alternative feature-importance measures consistent with formal explanations, since the paper's reduction shows attribution reduces to counting critical coalitions.","A natural next experiment is to feed nuSHAP the exact sample used by the SHAP tool and compare its top-k features against known formal explanations on small models; the paper's theory predicts exact agreement at the model level and sample-induced exceptions in practice."],"forward_implications":["Irrelevant features receive Shapley value 0 under $v_a$, so no ranking can place a formally irrelevant feature above a relevant one, a failure documented for standard SHAP.","Feature attribution and feature selection become unified: a positive marginal contribution is exactly a criticality judgment of the kind used in abductive and contrastive explanations.","Because $v_a$ is monotone with $\\{0,1\\}$ marginal contributions, the CGT estimator keeps its probabilistic accuracy guarantee while remaining polynomial-time per permutation when WAXp tests are efficient.","Data-based sufficiency checks make the method model-agnostic and scalable; the reported rank-overlap comparisons indicate that the SHAP tool's rankings disagree with the new scores on most tested instances.","The practical conclusion is not to abandon Shapley values but to replace the characteristic function used by SHAP-style explainers."],"supporting_citations":[{"why":"Supplies the definition of Shapley values and the axioms of fairness that the paper retains.","marker":"[Shapley 1953]"},{"why":"Defines the SHAP tool and the expected-value characteristic function whose failures motivate the new definition.","marker":"[Lundberg and Lee 2017]"},{"why":"Early XAI instantiation of Shapley values with the expected-value characteristic function, the target of the paper's critique.","marker":"[Strumbelj and Kononenko 2010]"},{"why":"Documents the misleading feature rankings of theoretical SHAP scores that the new scores are designed to eliminate.","marker":"[Marques-Silva and Huang 2024]"},{"why":"Supplies the running example $M_1$ and further case studies of SHAPT flaws.","marker":"[Huang and Marques-Silva 2024]"},{"why":"Proposes the characteristic function $v_a$ and the argument that the problem is the characteristic function, not Shapley values.","marker":"[Letoffe et al. 2025]"},{"why":"Provides the CGT sampling algorithm used to estimate Shapley values with probabilistic error bounds.","marker":"[Castro et al. 2009]"},{"why":"Introduces sample-based abductive explanations, which the implementation uses to decide WAXp status on data.","marker":"[Cooper and Amgoud 2023]"},{"why":"Proposition 3 from this work underlies the paper's Proposition 1 on irrelevant features.","marker":"[Huang et al. 2023]"}],"fun_headline_variants":["Irrelevant features score zero in SHAP after logical fix","Logical sufficiency test replaces expectation in SHAP, zeros out noise","Swapping expectation for logical sufficiency zeros SHAP noise","Fix SHAP by logical test: irrelevant features score zero"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper proves the zero-score property for a characteristic function that queries the model itself, but its tool defines the same function over a dataset sample; it assumes, without proof, that sample-based sufficiency judgments preserve the property, so scores produced by nuSHAP may not be the novel scores the theory describes.","fun_headline_variants_meta":{"raw":{"variants":["Irrelevant features score zero in SHAP after logical fix","Logical sufficiency test replaces expectation in SHAP, zeros out noise","Swapping expectation for logical sufficiency zeros SHAP noise","Fix SHAP by logical test: irrelevant features score zero"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001072,"raw_usage":{"total_tokens":4464,"prompt_tokens":891,"completion_tokens":3573,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":507,"completion_tokens_details":{"reasoning_tokens":3504}},"tokens_in":507,"tokens_out":3573,"duration_ms":25318,"temperature":1.0,"reasoning_tokens":3504,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:16:28.102465+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Feed nuSHAP a model in which feature 2 is irrelevant, for example a binary classifier predicting 1 exactly when $x_1 = 1$, with target instance $((0,0),0)$, and give it a dataset containing only the row $(1,1)$. The sample-based sufficiency test judges $\\{2\\}$ to be a weak abductive explanation while $\\emptyset$ is not, so feature 2 receives a positive estimated score even though it is irrelevant; that would show the sample-based implementation does not inherit the paper's central guarantee, and a reader could run this check with any SHAP-style sampling explainer.","supporting_citations":[],"review_version":1}