{"id":"2827f141-b567-41d7-9537-d3880c6e74bb","arxiv_id":"2607.14271","paper_version":1,"verdict":"ACCEPT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"Local additive feature-attribution methods are only interpretable relative to stated choices about value functions, baselines, paths, perturbation distributions, and conservation rules.","lead":"This survey argues that no feature-attribution method is assumption-free: every explanation of an AI prediction depends on hidden choices about baselines, values, and paths that differ between methods. It offers a common taxonomy, an axiom-by-method matrix, and a ten-item reporting checklist to make those choices explicit and comparable.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Five-choice taxonomy omits essential hyperparameters (e.g., LIME's sparsity penalty, meaningful perturbations' regularizers); checklist R1–R10 would not require reporting them, undermining the central claim's remedy.","rationale":"The reader identified the axiom-by-method matrix (Table IX) as the weakest assumption. That is a reasonable concern about the accuracy of a supporting artifact, but it is not the most load-bearing for the central claim: even if some matrix entries are wrong, the existence of hidden assumptions would still hold. The more fundamental issue is whether the five-choice taxonomy is exhaustive. The paper's strongest claim says every local additive attribution method defines feature importance 'through choices about' exactly these five categories. LIME and meaningful perturbations, both covered in the survey, have additional essential choices (sparsity penalty, regularization coefficients) that do not fit any of the five. The proposed checklist, which operationalizes the claim, contains no item that would require reporting these choices. Therefore, a researcher following R1–R10 could still produce an underdetermined attribution, directly contradicting the paper's assertion that reporting the five choices makes attribution trustworthy. This concern is internal to the paper's own examples, not dependent on external consensus, and is concretely testable by a simple LIME experiment. It warrants a conditional acceptance: the authors should broaden the taxonomy to include surrogate/regularization choices (or explicitly state the five are non-exhaustive) and extend the checklist to capture all method-defining hyperparameters.","tokens_in":45285,"tokens_out":9664,"duration_ms":114107,"concrete_test":"Take a fixed model f and input x. Compute LIME attributions twice: once with sparsity penalty λ=0 and once with λ=1, holding the similarity kernel, mask distribution, and surrogate family fixed. If the top-ranked features differ materially between the two settings, then the sparsity penalty is an independent specification choice that the five-category taxonomy and R1–R10 do not require reporting. This would empirically confirm the concern that the central claim's enumeration is incomplete and the checklist omits a load-bearing assumption.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim (Section I.G) states that every local additive attribution method defines feature importance through choices about value functions, references, paths, perturbation distributions, or conservation rules, and that reporting these makes attribution meaningful. But several surveyed methods have specification choices that fall outside these five categories. In Section VII.B, LIME (Eq. 31) depends on a similarity kernel, a mask distribution, a surrogate family G, and a complexity penalty Ω; the penalty and the choice of G are not captured by any of the five categories. In Section VII.D, meaningful perturbations (Eq. 33) depend on regularization coefficients λ and μ, again not among the five. The proposed checklist (Section XII) asks for R4 (value function/perturbation distribution) and R5 (path/coalition/sampling), but has no item for surrogate family, regularization, or optimization hyperparameters. Thus a LIME user who conscientiously follows R1–R10 could still leave the sparsity penalty unstated, and the attribution would be underdetermined. This directly weakens the central claim's prescription: the five choices are not exhaustive, and the checklist as written is insufficient to guarantee reproducible, interpretable attributions. The paper's own Section XIII.D acknowledges variant-dependence but does not address this incompleteness of the taxonomy itself.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript is a taxonomy-based survey of local additive feature-attribution methods. It proposes that every such method is specified by five mathematical choices—value function, reference/baseline, path, perturbation distribution, and conservation rule—and it compares roughly thirty methods through an axiom-by-method matrix, formalizes common failure modes as consequences of these choices, and proposes a ten-item reporting checklist (R1–R10). The central claim is that there is no assumption-free attribution method and that attribution results are meaningful only when the assumptions under which they were computed are reported.","tokens_in":45429,"tokens_out":5882,"duration_ms":68533,"significance":"If the central claim holds, the paper provides a valuable unifying frame for a fragmented literature and a concrete, actionable reporting standard. The internal mathematics I checked is correct: the Shapley formula (Eq. 6), the IG completeness chain-rule argument (Eqs. 15–17), the Aumann–Shapley/IG identification (Eq. 7), and the running example in Section II.F. The paper also ships per-cell axiom justifications (Appendix A) and a checklist crosswalk (Appendix B), which make the taxonomy substantially reproducible. The main weakness is that the five-choice taxonomy and the R1–R10 checklist are not exhaustive of the specification choices that actually determine several surveyed methods, notably LIME's kernel/surrogate/regularization and meaningful-perturbation regularizers.","major_comments":[{"comment":"The five-choice taxonomy omits specification choices that determine LIME and meaningful-perturbation attributions, and R1–R10 do not require them. In Eq. (31), LIME is defined by a similarity kernel π_x, a mask distribution, a surrogate family G, and a complexity penalty Ω; in Eq. (33), meaningful perturbations depend on λ and μ. None of these is captured by the categories 'value function, reference, path, perturbation distribution, conservation rule', and no checklist item asks for their values. A study could follow R1–R10 and still leave the sparsity penalty, surrogate family, or regularization coefficients unstated, producing an underdetermined attribution—exactly the state that Section I.G says should be avoided. The checklist should be extended (e.g., R4/R5 or a new item) to cover kernel, surrogate family, Ω, λ, μ, and optimization details, and the claim that the five choices are ex","section":"§VII.B, §VII.D, §XII"},{"comment":"The manuscript itself states that Table IX entries depend on the variant/implementation taken as canonical and that the matrix 'encodes the authors' reading'. This variant dependence is load-bearing for checklist item R6, which directs users to report which axioms their method satisfies. The per-cell justifications in Appendix A are helpful, but they do not uniquely identify the canonical variant for every row (e.g., LRP rule, DeepLIFT rule, CAM layer), and the main text does not require R6 users to name their variant. I recommend adding an explicit variant column or per-row canonical statement, and requiring R6 to name the implementation and rule variant. This is a fixable verifiability issue rather than a flaw in the central thesis, but it directly affects the reliability of the central comparison artifact.","section":"§XIII.D–E, Table IX, Appendix A"}],"minor_comments":[{"comment":"The heading 'Result 6' appears twice in the same subsection; the second occurrence should be renumbered or identified as a continuation, since the repeated number is confusing.","section":"§VI.C"},{"comment":"Equations (12) and (18) give the same expected-gradient definition essentially verbatim. Cross-reference one to the other to reduce redundancy.","section":"§V.B, §IV.E"},{"comment":"The supplementary materials are described as including machine-readable CSV files and a scoring sheet, but these are not attached in the arXiv source. Please state explicitly where they are available or include them as appendices.","section":"Reproducibility Statement"},{"comment":"For the Grad-CAM row, the matrix and Appendix A use 'completeness' at different levels (pixel-level vs. layer-level). A one-sentence clarification distinguishing these levels would prevent misreading of the entry.","section":"Table IX / Appendix A"},{"comment":"The corpus-limitation caveat is appropriately transparent. Consider adding the non-exclusive nature of role counts in Table III to the main text next to the table, rather than only in an appendix.","section":"§XIII.D"}],"recommendation":"major_revision","confidential_remarks":"The paper is well organized and makes a genuinely useful contribution. My recommendation of major revision is driven by the checklist omission: the central remedy, R1–R10, does not cover specification choices that are essential for methods the paper itself surveys, so a user following the checklist could still leave an attribution underdetermined. The variant-dependence of Table IX is acknowledged internally but should be tightened because R6 relies on it. These are fixable within the manuscript's scope; once addressed, the paper would be a solid publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this is a solid, unusually self-aware survey of local additive attribution, and its central thesis holds: there is no assumption-free attribution method, and papers should report the assumptions. The five-choice frame (value function, reference, path, perturbation distribution, conservation rule) does organize most of the major families, and the axiom matrix in Table IX with per-cell justifications in Appendix A is a real artifact. The internal math I spot-checked is correct: the Shapley formula, the IG completeness chain-rule argument, the Aumann-Shapley identification, and the running example all check out. The paper also ships reproducible extras (checklist crosswalk, axiom matrix CSV) and is transparent about its own limitations, including variant-dependence of the matrix and the paraphrased status of its numbered results.\n\nThe stress-test concern is legitimate and not manufactured. LIME depends on a similarity kernel, a mask distribution, a surrogate family G, and a complexity penalty Ω; only the first two fall under the five categories. Meaningful perturbations depend on regularization coefficients λ and μ, again outside the frame. The checklist R1–R10 has no item for surrogate family, regularizer, or optimization hyperparameters, so a LIME user who follows R1–R10 could still leave the sparsity penalty unstated and produce an underdetermined attribution. The paper concedes variant-dependence in Section XIII.D but does not address the incompleteness of the taxonomy itself. That is a moderate flaw, not a fatal one: the central claim about assumption-ladenness survives, and the checklist can be extended. But the paper's claim that the checklist supports reproducible reporting is too strong as written.\n\nAlso worth noting: Table IX entries are authorial judgments conditional on canonical variants; the paper flags this, but that means the matrix should be read as a well-structured hypothesis, not a certified map. Table X compiles external empirics without re-testing, which is fine for a survey but should be understood as secondary evidence.\n\nWho gets value: anyone working in XAI, either as a practitioner wanting a reporting standard or as a researcher wanting a compact map of method-axiom relationships. It deserves a serious referee, not a desk reject. I'd send it to review with a request that the authors address the exhaustiveness of the five choices, add checklist items for surrogate/regularization hyperparameters, and soften the reproducibility claim accordingly.","headline":"Useful survey with a real central claim, but the five-choice taxonomy and checklist miss essential hyperparameters for surrogate and perturbation methods; worth serious refereeing with revisions.","tokens_in":46082,"tokens_out":1666,"would_cite":true,"duration_ms":20635,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"No assumption-free feature-attribution method exists: every local additive attribution encodes hidden mathematical choices, and trustworthy use requires reporting them.","keywords":["feature attribution","explainable AI","Shapley values","integrated gradients","axiomatic methods","reporting checklist","value function","local additive attribution"],"falsifier":"Run canonical LRP-epsilon on a bias-free ReLU network where one input feature is a provable dummy (zero weight and bias everywhere); if the dummy ever receives nonzero relevance, the matrix's conditional sensitivity-(b) entry for LRP would need revision. The same check can be repeated for any conditional cell, such as DeepLIFT's implementation invariance under a specific architecture.","tokens_in":45000,"feed_emoji":"📋","tokens_out":4102,"duration_ms":43837,"temperature":0.7,"pith_summary":"The paper argues that local additive feature-attribution methods—SHAP, Integrated Gradients, LIME, LRP, Grad-CAM, and similar—are not interchangeable explainers but are defined by five hidden mathematical choices: how feature absence is encoded, what baseline is compared against, what path is integrated, what perturbation distribution defines locality, and what conservation rule propagates credit. Because methods differ along these axes, their disagreements and failures are traceable to specification choices rather than to implementation bugs. The survey's concrete deliverables are a unified taxonomy, an axiom-by-method matrix showing which guarantees each method keeps, and a ten-item reporting checklist (R1–R10) that asks papers to state model output, feature granularity, baseline, value function, path, axioms, approximation budget, sanity checks, faithfulness metrics, and known failure modes. If the central claim is right, every attribution claim that omits these specifications is underdetermined, and the checklist is the remedy the field needs.","feed_headline":"No attribution is assumption-free — report its assumptions","feed_subtitle":"SHAP, IG, LIME, and LRP hide five specification choices; this paper's ten-item checklist forces them into the open.","key_machinery":"The organizing device is the five-part specification: value function v (what feature absence means), reference x' (compared to what), path γ (along what trajectory), perturbation distribution p_z (which neighbourhood is local), and conservation rule (what quantity propagates). The axiom-by-method matrix (Table IX) turns these choices into a comparison table: each method's row shows which axioms it satisfies unconditionally, conditionally, or not at all. The paper also uses the additive surrogate g(z)=φ0+Σφ_i z_i to show that LIME, SHAP, DeepLIFT, LRP, and Integrated Gradients all speak the same linear-in-mask language and differ only in how coefficients are computed.","core_discovery":"The paper's central claim, stated as a principle, is that there is no assumption-free feature-attribution method: every local additive attribution defines feature importance through choices about value functions, references, paths, perturbation distributions, or conservation rules, and trustworthy use requires reporting the heatmap or ranking together with the assumptions under which it was computed and interpreted. It substantiates this by organizing Shapley, path-based, gradient/backpropagation, perturbation, and CAM-style methods in one notation, cataloguing the axioms of each family, and building an axiom-by-method matrix showing that methods differ precisely in which axioms they satisfy","pith_inferences":["If the paper's thesis is accepted as a discipline norm, the field's central question shifts from 'which attribution method is best?' to 'which specification matches the user's intended question?', and method papers would be required to publish their specification as part of the method definition.","The taxonomy suggests a testable extension: a meta-benchmark that varies the five specification choices systematically while holding model and input fixed could quantify how much of reported method disagreement is actually specification disagreement—a prediction that follows from the paper's Principle 1.","The checklist, if adopted, would make attribution papers more reproducible, but it also exposes a gap the paper itself acknowledges: conditional matrix entries depend on which variant of a method is taken as canonical, so the same method under different propagation rules would need separate matrix rows.","The central claim extends to large-language-model attribution: sub-token, token, span, and generation-target choices are themselves specification choices, so the 'no assumption-free attribution' thesis applies directly to the open problems the paper lists."],"forward_implications":["Any study reporting attributions without stating baseline, value function, path, and perturbation distribution has reported an underdetermined quantity; the checklist gives the minimal information needed to reproduce and interpret an attribution.","Method disagreements should be investigated by holding value function, baseline, path, and perturbation distribution constant before attributing disagreement to the model.","Known failure modes—baseline sensitivity, off-manifold perturbation, adversarial manipulation, sanity-check failure—are consequences of specification choices, so they have structural mitigations rather than being fatal to a method.","Axiom-equivalence results (KernelSHAP equals Shapley in expectation; DeepLIFT-Rescale equals Integrated Gradients in the small-increment limit; LRP-ε equals gradient×input on bias-free ReLU networks) mean empirical differences between methods can be traced to axioms, value functions, and paths rather than to mathematical sophistication.","Attribution studies should report at least one sanity-check and one faithfulness or stability metric; otherwise the attribution cannot be distinguished from a model-ignorant map."],"fun_headline_variants":["Attribution is meaningless without its assumptions","SHAP and LIME hide 5 choices - checklist exposes them","Every feature-attribution method makes assumptions","New taxonomy unifies attribution methods with a checklist","Report your attribution assumptions or lose meaning"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the axiom-by-method matrix correctly assigns axioms to canonical method variants; if even one assignment is wrong, the taxonomy's central comparison and checklist item R6 lose their trustworthiness.","fun_headline_variants_meta":{"raw":{"variants":["Attribution is meaningless without its assumptions","SHAP and LIME hide 5 choices - checklist exposes them","Every feature-attribution method makes assumptions","New taxonomy unifies attribution methods with a checklist","Report your attribution assumptions or lose meaning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000757,"raw_usage":{"total_tokens":3164,"prompt_tokens":670,"completion_tokens":2494,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":414,"completion_tokens_details":{"reasoning_tokens":2425}},"tokens_in":414,"tokens_out":2494,"duration_ms":18121,"temperature":1.0,"reasoning_tokens":2425,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T02:35:38.051110+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run canonical LRP-epsilon on a bias-free ReLU network where one input feature is a provable dummy (zero weight and bias everywhere); if the dummy ever receives nonzero relevance, the matrix's conditional sensitivity-(b) entry for LRP would need revision. The same check can be repeated for any conditional cell, such as DeepLIFT's implementation invariance under a specific architecture.","supporting_citations":[],"review_version":1}