{"id":"e1a62802-2ab2-43bb-b3d3-a677701d6112","arxiv_id":"2508.08966","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Two attention-based explanation methods are introduced: an attention-weighted Shapley decomposition for local attributions and attention-informed concept sensitivity for global explanations.","lead":"This paper proposes two new ways to explain transformer models by folding attention weights into existing explanation frameworks, one based on Shapley values and one based on concept activation vectors. It reports that these attention-based explanations are meaningful on standard benchmarks when compared with established methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"With the full text corrupted, the attention-weighted Shapley construction cannot be checked; the central risk is that the method re-encodes attention rather than explaining model output, so the claim remains unverified.","rationale":"The reader's weakest assumption correctly identifies attention faithfulness as the core premise, and I agree that this is the conceptual soft spot. My stress-test pass cannot go further because the full text is unreadable: I cannot check the derivation of the attention-weighted characteristic function, the implementation of the CAV-based directional derivatives, or the evaluation protocol. In good faith, I do not claim the paper is wrong; I claim that its central construction is currently unverifiable from the available material. The concrete test I propose would settle the most important technical question: whether the Shapley attribution actually satisfies the defining property of the explanation framework (efficiency with respect to the model output) and whether the method is sensitive to output-relevant computation rather than merely to the attention pattern. If the efficiency check fails or the randomized-attention experiment shows large attribution shifts, the central claim of 'meaningful incorporation' would be undermined. Given the lack of inspectable content, the verdict should remain UNVERDICTED, matching the reader's outcome, though my emphasis is on the missing mathematical verification rather than on attention faithfulness per se.","tokens_in":26110,"tokens_out":1665,"duration_ms":20159,"concrete_test":"Obtain a clean copy of the full text and locate the formal definition of the attention-weighted characteristic function (likely in Section 3). Re-derive the Shapley values for a simple three-token transformer and verify the efficiency property: the sum of the computed Shapley values equals f(x) minus the baseline value assigned to the empty coalition. Separately, run a controlled experiment on a trained transformer where the attention weights are replaced by random or shuffled weights while the output logits are approximately preserved by adjusting value/output projections; if the proposed attributions change substantially, the method tracks attention rather than model behavior.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The central claim requires that incorporating attention weights into a Shapley characteristic function and into token-level CAV derivatives yields attributions that reflect actual model behavior. The full text supplied is a corrupted extraction, so the mathematical definitions, derivations, and experimental details cannot be inspected. The most load-bearing concern is therefore not a detected algebraic flaw but an unverified premise: if the characteristic function is built from attention-weighted pairwise interactions, the resulting Shapley values are deterministic functions of attention weights, and the method may simply re-encode attention rather than provide a meaningful explanation of the output. The abstract itself concedes that attention weights 'do not directly determine model outputs,' so faithfulness must be demonstrated, not assumed. Without the clean text, we cannot check whether the characteristic function satisfies basic game-theoretic properties (e.g., efficiency, the treatment of the empty coalition, and whether the computed Shapley values sum to the model output), nor can we assess whether the empirical comparisons to other explanation methods are fair or whether the CAV variant measures output sensitivity rather than attention sensitivity. Thus the paper is unverified rather than refuted, and the reader's UNVERDICTED verdict is appropriate.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two methods for explaining transformer models by integrating attention weights into established XAI frameworks. The first method redefines the Shapley value characteristic function using attention-weighted pairwise token interactions; the second incorporates attention weights into token-level directional derivatives defined through concept activation vectors. The abstract claims these methods produce meaningful local and global explanations for both NLP and vision transformers, supported by empirical evaluations on standard benchmarks. However, the supplied full text is heavily corrupted, so the mathematical definitions, derivations, and experimental details cannot be inspected.","tokens_in":26376,"tokens_out":4803,"duration_ms":45997,"significance":"If the proposed methods work as claimed, the paper would contribute a novel way to leverage attention weights within game-theoretic and concept-based explanation frameworks, potentially enriching transformer explainability. The paper addresses a relevant problem and the idea is plausible. However, the central premise that attention weights faithfully reflect token influence on model outputs is not established, and the current submission provides no verifiable evidence. No code, complete derivations, or experimental tables are accessible, so the significance of the contribution cannot be assessed. The paper's potential lies mainly in its proposal, not in validated results.","major_comments":[{"comment":"The body of the manuscript as supplied is unreadable due to character corruption, with no coherent equations, tables, or experimental descriptions. This prevents verification of the attention-weighted Shapley construction, the CAV derivative definitions, and the empirical claims. Since the central claims depend on these details, the manuscript cannot be evaluated in its current form.","section":"Full text (corrupted)"},{"comment":"The abstract states that attention weights do not directly determine model outputs, yet the methods use attention weights as the defining input to the explanation frameworks. The paper must demonstrate that the resulting attributions reflect model behavior rather than merely re-encoding attention; for instance, by comparing attention-weighted Shapley values against causal perturbation-based attributions and showing they track output changes. Without such evidence, the claim of meaningful incorporation is unsupported.","section":"Abstract"},{"comment":"The proposed Shapley characteristic function is described only as pairwise token interactions via attention weights. The manuscript must specify the exact functional form, including how the empty coalition and full coalition are valued, and verify that the resulting Shapley values satisfy efficiency and other relevant axioms. If the characteristic function is a direct linear function of attention weights, the attributions may be deterministic transformations of attention and not genuine explanations of the output.","section":"Abstract"},{"comment":"For the concept activation vector method, the abstract mentions token-level directional derivatives but does not clarify the differentiation variable. If the derivative is taken with respect to attention weights rather than model outputs, the method would measure attention sensitivity, not concept sensitivity. This distinction is crucial for the claimed global explanation capability and must be clarified.","section":"Abstract"}],"minor_comments":[{"comment":"The abstract is readable, but the rest of the text is corrupted. The authors should ensure the submission is encoded correctly; this is a severe presentation issue that blocks review.","section":"Full text"},{"comment":"The paper should cite and discuss prior work on attention faithfulness, such as Jain and Wallace (2019) and related studies, to position the assumption that attention weights are informative.","section":"Introduction (as evidenced by abstract)"}],"recommendation":"uncertain","confidential_remarks":"The supplied full text is severely corrupted, likely due to an encoding error; I cannot review the technical content. I recommend contacting the authors for a clean version before making a decision. If a clean version is not provided, the paper should be rejected for lack of verifiability. Note also that the abstract promises empirical evaluations but no datasets, metrics, or baselines are visible in the current record."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this paper is easy to under- or over-read because the extracted full text is garbage -- it literally includes hep-lat fragments. The abstract is all we have, and it describes two concrete proposals: redefining the Shapley characteristic function using attention-weighted pairwise token interactions, and incorporating attention weights into token-level directional derivatives for concept activation vectors. Both target transformers, both claim to work for NLP and vision, and the abstract is careful to say attention weights \"do not directly determine model outputs.\" That is the right framing, and it makes the central idea plausible: attention as a complementary signal inside existing XAI frameworks, not a replacement.\n\nThe novelty is incremental but real. Attention-plus-Shapley combinations exist in various forms, and attention-weighted CAV derivatives are close to known ideas, but the specific pair of constructions aimed at local and global explanation respectively is a reasonable contribution -- if the derivations and experiments hold up. The empirical claim is asserted but not visible: no datasets, metrics, baselines, or error bars appear in the accessible record. That is a limitation of the corrupted text, not necessarily a flaw in the paper.\n\nThe load-bearing soft spot is the faithfulness question. If the characteristic function is built directly from attention weights, the resulting Shapley values are deterministic functions of those weights. The method could just re-encode attention rather than explain the model's output. The same concern applies to the CAV variant: it may measure sensitivity of attention rather than sensitivity of the output. The abstract concedes the premise that attention does not directly determine outputs, so the authors need to demonstrate faithfulness, not assume it. I want to see whether the characteristic function satisfies basic game-theoretic properties like efficiency, and whether the empirical comparisons show agreement with model behavior beyond trivial attention correlation. These checks are exactly what a referee would do, and they may turn out fine.\n\nWho is this for: researchers working on transformer explainability, especially those interested in attention plausibility or in extending Shapley/CAV frameworks. If the actual PDF is clean and contains the promised math and benchmarks, this deserves a serious referee. My verdict is unverified rather than negative. Send it to peer review, with a referee asked to focus on the faithfulness question and on whether the comparison study supports the claim that attention is meaningfully incorporated.","headline":"Two plausible attention-aware XAI methods (attention-weighted Shapley and attention-conditioned CAV derivatives), but the supplied text is corrupted so the math and experiments cannot be checked; treat it as unverified, not refuted.","tokens_in":26794,"tokens_out":1611,"would_cite":false,"duration_ms":18956,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Attention weights can be meaningfully incorporated into Shapley and concept explanations","keywords":["attention weights","transformer explainability","Shapley values","concept activation vectors","local explanations","global explanations","natural language processing","computer vision"],"falsifier":"Train a small transformer on a task with a known ground-truth token-influence rule, for example where only one token class decides the label; if the attention-weighted Shapley attributions systematically point to other tokens, or if randomizing the attention weights while freezing the value and output pathway leaves the attributions essentially unchanged, the central claim would be contradicted.","tokens_in":25955,"feed_emoji":"🎯","tokens_out":3430,"duration_ms":35798,"temperature":0.7,"pith_summary":"The paper tries to establish that attention weights in transformers, even though they do not directly determine model outputs, encode token-influence patterns that can be imported into existing explainability frameworks to produce useful explanations. It introduces two methods: a Shapley-based local attribution whose characteristic function is redefined around attention-weighted pairwise token interactions, and a global concept-sensitivity method that folds attention weights into token-level directional derivatives of concept activation vectors. Evaluated on natural language and vision benchmarks, both are claimed to enrich transformer explanations beyond standard gradient or perturbation approaches. A sympathetic reader should care because it offers a principled way to turn the model's own internal weighting signal into explanations without treating attention alone as the explanation.","feed_headline":"Attention weights can power two new transformer explanations","feed_subtitle":"Shapley-style token attributions and concept sensitivity scores built from attention, tested on language and vision tasks.","key_machinery":"The two carrying objects are (1) an attention-weighted characteristic function for Shapley decomposition — the game-theoretic payoff of a set of tokens is redefined as a sum over pairwise interactions weighted by attention, so the Shapley value assigns each token a share of the model output that respects the attention graph — and (2) attention-weighted token-level directional derivatives along concept activation vectors, where concept sensitivity is measured by how much the prediction changes when tokens are moved in the direction of a concept, with attention controlling each token's contribution. The first supplies local, per-instance attributions; the second supplies global, concept-level sensitivity scores.","core_discovery":"On its own terms, the paper claims that attention weights are not merely a visualization aid but can be the building block of formal explanation quantities. For local explanations, it redefines the characteristic function used in Shapley value decomposition so that the value of a coalition of tokens is computed via pairwise interactions weighted by the attention between them; applying Shapley values to this function yields per-token attributions that reflect the attention graph. For global explanations, it defines token-level directional derivatives along concept activation vectors, with attention weights used to weight those derivatives, yielding a concept-sensitivity measure for the whole model. The empirical comparison with established methods is presented as evidence that the resulting attributions are meaningful and complementary.","pith_inferences":["I infer that attention-weighted Shapley attributions could be stress-tested as a diagnostic for spurious attention: if a token receives high attribution but ablating it does not change the output, the attention graph is not faithfully reflecting computation in that case.","An extension the authors do not pursue is using the pairwise-interaction form to recover syntactic dependencies in language transformers, since attention-weighted pairwise Shapley terms can isolate which token pairs jointly drive a prediction.","A direct testable extension would be to compare attention-weighted concept-sensitivity scores against causal interventions on the concept direction, for example removing the concept direction from token embeddings and checking whether the sensitivity score predicts the actual output change."],"forward_implications":["Local explanations for a single text or image become attention-aware Shapley scores, so users can see which tokens the model's attention graph says carry the decision.","Global explanations gain a concept-level question: how sensitive is the model to a concept, measured through attention-weighted token movements, instead of only which inputs matter.","The same two mechanisms run on language and vision transformers, suggesting one explanation recipe across modalities.","Compared with gradient or perturbation baselines, the attention-informed scores provide a different signal, letting explainability users cross-check conclusions."],"supporting_citations":[],"fun_headline_variants":["Attention weights fuel two new transformer explanation methods","Attention powers Shapley and concept explanations for transformers","Attention-based Shapley and concept explanations for transformers","Two attention-integrated explanation methods for transformer models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The methods assume attention weights carry trustworthy information about how much tokens influence the model's output; if attention patterns are not faithful to the computation, both the local and global explanations inherit that unfaithfulness.","fun_headline_variants_meta":{"raw":{"variants":["Attention weights fuel two new transformer explanation methods","Attention powers Shapley and concept explanations for transformers","Attention-based Shapley and concept explanations for transformers","Two attention-integrated explanation methods for transformer models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001421,"raw_usage":{"total_tokens":5701,"prompt_tokens":878,"completion_tokens":4823,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":4763}},"tokens_in":494,"tokens_out":4823,"duration_ms":34610,"temperature":1.0,"reasoning_tokens":4763,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:30:51.476285+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a small transformer on a task with a known ground-truth token-influence rule, for example where only one token class decides the label; if the attention-weighted Shapley attributions systematically point to other tokens, or if randomizing the attention weights while freezing the value and output pathway leaves the attributions essentially unchanged, the central claim would be contradicted.","supporting_citations":[],"review_version":2}