{"id":"fe840621-4d44-4182-843b-fcd09149a7f3","arxiv_id":"2502.04366","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"CT-LRP backpropagates relevance scores through a GNN and text embedding to produce class-specific token-level explanations for rumour detection, adding a contrastive step that filters tokens shared with other classes.","lead":"This paper introduces CT-LRP, a method that highlights the specific words in a social media thread that made a graph-based rumour detection model call it a rumour. The authors report higher fidelity than node-level explanations on three datasets, though the evaluation and algorithm details raise concerns.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 3's token-level LRP rule is invalid for the actual BERT-based ftext: BERT token vectors are contextual, not independent inputs to mean pooling, no LRP rules are provided for its attention layers, and the rule omits the 1/|T_v| pooling weight, breaking conservation.","rationale":"The reader's weakest assumption correctly identifies the same load-bearing weakness: the token-level attribution rule in Eq. 3 is not valid for the BERT-based text embedding function actually used, and no LRP rules for transformer attention layers are supplied. My independent pass confirms this and adds a concrete arithmetical defect: Eq. 3 omits the 1/|T_v| weight from the mean-pooling LRP rule, so conservation fails and token relevance scales with post length. This is a correctness risk rather than a matter of disagreement with consensus, because the paper itself defines ftext as multilingual BERT and defines x_v,d as a mean over token vectors. The central claim cannot be verified or interpreted as stated without either code or an explicit LRP specification for BERT, plus a correction to Eq. 3. These are addressable technical omissions, not a fundamental invalidation of the idea, so I do not see reason to move the verdict to reject; the conditional verdict already requires such a specification and code. I also note the fidelity metric and the contrastive selection step both use perturbation by token removal, which is an additional evaluation concern, but the BERT/Eq. 3 issue is more foundational because it undermines the definition of the explanations themselves.","tokens_in":14478,"tokens_out":10957,"duration_ms":112403,"concrete_test":"Build a minimal two-token reproduction: take a single post, encode it with the same multilingual BERT, mean-pool the output vectors, and compute token attributions three ways: (1) Eq. 3 as printed, (2) Eq. 3 corrected with the 1/|T_v| factor, and (3) a full LRP propagation through BERT's attention layers using a published transformer-LRP rule or an exact gradient decomposition. If the ranking and sum of attributions from (1) differ materially from (3), the paper's token-level explanation is not LRP for the actual ftext; if the corrected rule in (2) changes the removal ranking, the reported fidelity gains must be recomputed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The empirical claim that CT-LRP consistently produces high-fidelity token-level explanations rests on the computation of LRPtext in Algorithm 1. The paper specifies this computation only for a simplified ftext consisting of independent per-token embeddings followed by a pooling function. Eq. 3 is derived from Eq. 1 for mean pooling under that assumption. In the experiments, ftext is a pre-trained multilingual BERT (Section V-B1), so the quantities x_tv,d entering Eq. 3 are contextualised hidden states, not independent token embeddings. Each x_tv,d is a complex function of the entire input sequence through self-attention, and the LRP rule in Eq. 3 ignores the attention, residual, layer-norm, and positional-embedding operations inside BERT. No LRP propagation rules for these layers are given anywhere in the paper, so the computation of LRPtext is unspecified and not reproducible from the text. If Eq. 3 is nevertheless applied directly to BERT output vectors, the resulting z_tv is not a valid input-level token attribution for the actual model; it is a relevance score attached to contextual output vectors and does not account for the dependencies the paper claims to address. A second, independent defect strengthens this concern: even for a genuine per-token embedding followed by mean pooling, Eq. 3 is not the LRP rule implied by Eq. 1. Because x_v,d is defined as 1/|T_v| times the sum of x_tv,d, the numerator in the epsilon-stabilised rule should contain the factor 1/|T_v|. The omission means sum_t r_tv,d approximately equals |T_v| r_v,d, violating LRP conservation and inflating token relevance for longer posts. Since fidelity is computed by removing tokens in order of attribution, this scaling error can change which tokens are removed and thus the reported scores in Tables II-IV.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CT-LRP, a post hoc explainability framework that combines Layerwise Relevance Propagation (LRP) on a GNN with backpropagation through a text embedding function, aiming to produce token-level attributions for GNN-based rumour detection. The authors introduce a contrastive mask intended to keep only tokens that are specific to the predicted class, and they evaluate the method with BiGCN, EBGCN, and ClaHi-GAT on Twitter15/16, Weibo, and PHEME. They report average fidelity improvements of 25.77% over token-level LRP and 66.98% over the next best baseline on fidelity-sparsity. The central claim is that CT-LRP consistently produces high-fidelity, token-level explanations that are more granular than existing node- or edge-level graph explainability methods.","tokens_in":14843,"tokens_out":6736,"duration_ms":68461,"significance":"If the method worked as described, token-level explanations for GNN-based rumour detection would be a useful contribution, since existing graph explainability tools typically stop at node or edge granularity. The paper makes a concrete, falsifiable empirical claim and tests it across three model architectures and three public datasets with cross-validation, reporting standard deviations. It also extends fidelity and sparsity metrics to the token level, which is a sensible direction. However, the current manuscript has load-bearing problems: the token-level LRP rule is not valid for the actual BERT-based text encoder used in the experiments, the contrastive masking pseudocode contradicts the prose, and the fidelity-sparsity comparison mixes token units and node units. These issues prevent the reported numbers from supporting the paper's central claim as written.","major_comments":[{"comment":"The mean-pooling LRP rule in Eq. (3) is not the rule implied by Eq. (1). Since x_v,d is defined as (1/|T_v|) times the sum of x_{t_v,d}, the numerator should contain the same factor 1/|T_v|; omitting it breaks relevance conservation. As written, Eq. (3) is not a valid LRP propagation rule for the pooling operation described in Eq. (2).","section":"Section IV-C, Eq. (3)"},{"comment":"The token-level computation assumes that ftext decomposes into independent per-token embeddings followed by a pooling function, but the experiments use a pre-trained multilingual BERT as ftext. BERT produces contextualized token vectors, so each x_{t_v,d} is a complex function of the entire input sequence through self-attention, residual connections, and layer normalization. The paper gives no LRP propagation rules for these BERT layers, so the quantity LRPtext(ftext, R(c), P) in Algorithm 1 is unspecified and the computation is not reproducible. Applying Eq. (3) directly to BERT output vectors does not yield a valid input-level token attribution; the authors need to either provide complete LRP rules for the transformer stack or use a genuinely non-contextual embedding model.","section":"Section IV-C and Section V-B1"},{"comment":"The masking logic in Algorithm 1 does not implement the described exclusion of negative-attribution tokens. The branch `if z(hat_y) <= 0 then Zmask <- False` is followed by an independent if/else over z(c), and the else branch assigns True regardless, so the False assignment can be overwritten. In addition, the contrastive check is executed per class inside the loop, whereas the prose requires the condition y_hat - y'_hat > y_c - y'_c to hold for all c != hat_y before retaining a token. As written, the pseudocode can retain or discard a token based on only one other class, depending on loop order.","section":"Algorithm 1 and Section IV-D"},{"comment":"The fidelity and fidelity-sparsity comparisons are not unit-consistent. For node-level baselines, sparsity is computed as 1 - m(c)/|V| and fidelity is measured by removing attributed nodes, while for CT-LRP sparsity counts tokens and fidelity is measured by removing attributed tokens. Removing one node is a much larger perturbation than removing one token, so the reported advantages in fidelity, sparsity, and fidelity-sparsity may reflect the different removal units rather than a genuinely better explanation. A controlled comparison is needed, for example by evaluating all methods with the same removal budget measured in embedded-vector mass, or by reporting a token-level baseline under the same token-removal protocol.","section":"Section V-D and Tables II-IV"},{"comment":"The fidelity protocol is described inconsistently. The metric definition says elements with attribution score greater than 0.01 are removed, while the experimental section removes elements in decreasing order of importance until a fixed sparsity level is reached. These are different procedures and could produce different results; the paper should specify exactly which protocol produced Tables II-IV.","section":"Section V-C and Section V-D"}],"minor_comments":[{"comment":"The legend contains a curve labeled 'CT-LRP w/ NSC', but the abbreviation NSC is never defined or discussed in the text.","section":"Figure 3"},{"comment":"The c-EB baseline is described only as 'Contrastive EB' with a reference; the paper should specify how the contrastive variant is implemented and how its probabilistic Winner-Takes-All process is applied to the GNN node features.","section":"Section V-C"},{"comment":"In the contributions list, 'Boarder Impact' should be 'Broader Impact', and the phrase 'the the Russia-Ukraine war' in the introduction contains a duplicated article.","section":"Section I"},{"comment":"The max-pooling rule in Eq. (5) is not an epsilon-stabilized LRP rule; it is a hard argmax mask that allocates all relevance to the maximum neuron. If max pooling is used in any experiment, the paper should justify this rule separately.","section":"Section IV-C, Eq. (5)"},{"comment":"The discussion states that CT-LRP 'represents a significant advancement' and 'achieves a new standard', but these claims go beyond the evidence presented, especially given the unresolved metric and implementation issues; please temper the wording.","section":"Section VI"}],"recommendation":"major_revision","confidential_remarks":"The paper does not provide code or a detailed implementation of the backward pass through BERT, which makes independent verification difficult. The presence of an undefined 'CT-LRP w/ NSC' curve in Figure 3 also suggests the manuscript may be an incomplete version. If the authors can fix the LRP propagation, the masking logic, and the evaluation protocol, the underlying idea is worth reconsidering."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nBottom line: this paper has a reasonable idea—push LRP attribution through the GNN and into the text embedding function to get token-level explanations, with a contrastive filter to focus on class-specific tokens—and the experiments are broad, but as written the central token-attribution computation is not actually defined for the model used, and the evaluation mixes units. It should go to peer review but only with major revision.\n\nWhat's new: combining LRP through a GNN with LRP through the pooling stage of a text embedder is a direct extension, and the class-disambiguation step is a simple but sensible addition. The paper is clearly written and the grid of three GNNs × three datasets is more than many explainability papers do.\n\nWhere it falls down:\n\n1. The token-level attribution is the load-bearing component and it rests on Eq. 3, which is not a valid LRP rule for the actual ftext, a pre-trained multilingual BERT with mean pooling. BERT token vectors are contextual, not independent inputs to pooling, and the paper gives no LRP propagation rules for BERT's attention, residual, layer-norm, or positional operations. Without that, the \"token attributions\" are an informal projection, not a faithful explanation of the model. Even in the simplified mean-pooling case, Eq. 3 omits the 1/|T_v| factor from Eq. 2, so sum_t r_t ≠ r_v, violating LRP conservation and inflating relevance for longer posts.\n\n2. Algorithm 1 has a masking logic bug: the False assigned for negative z(y_hat) is overwritten by the else branch of the subsequent if, so negative-attribution tokens are not actually excluded. That directly contradicts the prose.\n\n3. The fidelity/sparsity comparison is confounded: CT-LRP measures sparsity over tokens, baselines over nodes, so the \"fidelity at fixed sparsity\" and the fidelity-sparsity product are not apples-to-apples.\n\nNotice the paper's own limitations section says it \"implicitly considers graph inductive biases but does not explicitly explain them\"—that's honest, but it doesn't address the BERT gap.\n\nWho this is for: readers working on GNN explainability for text-attributed graphs will find the framing useful, and the contrastive filter is worth thinking about. But the numbers in Tables II–IV shouldn't be taken at face value until the propagation rules and metrics are fixed.\n\nRecommendation: send to peer review with a request for major revision. The idea deserves a referee's time, but the revision must either provide a concrete LRP-through-BERT specification or restrict the claim to a simpler embedder, fix the algorithm, and rerun the comparison on matched units.","headline":"Sensible token-level LRP extension for GNN rumour detection, but the BERT backward pass is unspecified, Algorithm 1 has a masking bug, and the fidelity-sparsity comparison mixes units; fixable, but needs major revision.","tokens_in":15393,"tokens_out":3017,"would_cite":false,"duration_ms":27022,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that CT-LRP, a contrastive token-level LRP framework, produces higher-fidelity, sparser explanations for GNN-based rumour detection than node-level explainability baselines, with an average fidelity gain of 25.77% over…","keywords":["rumour detection","graph neural networks","explainability","layerwise relevance propagation","token-level explanations","contrastive explanations","fidelity-sparsity","misinformation detection"],"falsifier":"Run CT-LRP on a model whose node features come from BERT, then for each token the method highlights as class-specific, remove that token from the post text, re-run the full BERT-plus-GNN pipeline, and compare the predicted-class logit change against the token's attributed relevance rank. If the highest-attribution tokens do not cause the largest logit drops, the Equation (3) decomposition is not faithful. A sharper check is to verify relevance conservation directly: replace BERT with static word vectors plus mean pooling and confirm that the sum of token relevance equals the node-level relevance; with BERT, the contextualized word vectors mean mean-pooling equality generally fails, so the check should reveal the mismatch.","tokens_in":14304,"feed_emoji":"🔍","tokens_out":9827,"duration_ms":80493,"temperature":0.7,"pith_summary":"Rumour detection models built on graph neural networks give a prediction about an event from a tree of posts, but usual explanations say which posts mattered, not which words. This paper claims that pushing layerwise relevance propagation one step further, through the text embedding function and then through a contrastive masking step, produces token-level explanations that are both more faithful and more focused than node-level explanations. The authors test the framework on three GNN rumour detectors and three public datasets, and report that token-level explanations raise fidelity and fidelity-sparsity scores across the board, with an average 25.77% fidelity gain over token-level LRP without the contrastive step. If the claim holds, explainability for rumour detection shifts from 'this post drove the decision' to 'this word in this post drove the decision,' which is the resolution fact-checkers and platform moderators need.","feed_headline":"Token-level LRP beats node-level explanations for GNN rumour checks","feed_subtitle":"A contrastive step filters shared token relevance for finer explanations than node-level methods.","key_machinery":"The load-bearing mechanism is the epsilon-stabilized LRP propagation rule, $r_j = \\sum_k \\frac{a_j w_{jk}}{\\epsilon + \\sum_{j'} a_{j'} w_{j'k}} r_k$, which conserves relevance as it flows backward through the GNN and produces a node-feature attribution map $R \\in \\mathbb{R}^{|V|\\times|D|}$. To turn that into token attributions, the paper assumes the text encoder is a composition of per-token embedding $f_{\\text{embed}}$ followed by a pooling function, and for mean pooling derives the closed-form rule $r_{t_v,d} = \\frac{x_{t_v,d}}{\\epsilon + x_{v,d}} r_{v,d}$; summing over embedding dimensions gives each token's relevance. The contrastive refinement computes attribution maps for all classes and masks tokens that are positively relevant to the predicted class but more influential on another class, using a logit-difference test on a graph where that token's vector has been removed. The mask product yields the final class-specific token explanation.","core_discovery":"The central claim is that the class-disambiguation step is what makes token-level explanations work for GNN rumour detectors: token-level LRP alone already improves fidelity over node-level baselines, and the contrastive filter adds a further, consistent gain. The method computes an LRP attribution map for each output class, then for each token that is positively relevant in both the predicted class and another class it removes that token's vector from the pooled node embedding, re-runs the model, and keeps the token only if removing it drops the predicted-class logit more than it drops every rival-class logit; tokens with negative relevance in the predicted class are masked out as well. The surviving tokens form the explanation. Across BiGCN, EBGCN, and ClaHi-GAT trained on Twitter, Weibo, and PHEME, the framework reports higher fidelity at fixed sparsity than node-level LRP, Grad-CAM, and contrastive excitation backprop, and reports an average 66.98% higher fidelity-sparsity score than the next best baseline.","pith_inferences":["A direct way to isolate the source of the reported gain is to apply the same contrastive logit-difference mask to node-level LRP; if the mask alone explains most of the fidelity improvement, token granularity may be less important than the class-disambiguation step.","Because Equation (3) is exact only for static per-token embeddings plus mean pooling, a natural test is to run CT-LRP with such an encoder and compare with the BERT-based results; a large discrepancy would indicate the reported gains depend on the pooling assumption rather than on genuinely contextualized attributions.","The paper's own limitation note that graph inductive biases are not explained suggests a straightforward extension: combine CT-LRP's token mask with edge-level attributions to test whether propagation structure and lexical cues interact in rumour decisions.","A human evaluation could extend the quantitative claim: ask annotators which words in a source post are most diagnostic of a rumour label, and check whether CT-LRP's retained tokens align better than those of node-level baselines; the paper reports no such user study."],"forward_implications":["If the fidelity results hold, a rumour detector's decision can be drilled down to the exact words in the source post and replies that push it toward a label, rather than only the posts that matter.","Because the contrastive mask filters out tokens shared with rival classes, the remaining explanations are class-specific, which should help distinguish cues for 'false rumour' from 'unverified rumour'.","The fidelity-sparsity gains imply explanations stay informative even when heavily pruned, which matters for large conversation trees where full token lists are unusable.","The framework runs one backward pass per class, so the added interpretability comes at a modest computational cost and can be applied to existing trained GNNs.","The authors argue the same token-level framing can be carried over to fake news detection and other misinformation tasks."],"supporting_citations":[{"why":"Supplies the epsilon-stabilized LRP rule that CT-LRP propagates backward through the GNN and text encoder.","marker":"[24]"},{"why":"Defines BiGCN, one of the three GNN rumour-detection models whose explanations CT-LRP is evaluated on.","marker":"[12]"},{"why":"Defines EBGCN, the second evaluation model, including the edge-consistency architecture CT-LRP must backpropagate through.","marker":"[13]"},{"why":"Defines ClaHi-GAT, the third evaluation model, with attention layers and hierarchical event representation used in the experiments.","marker":"[14]"},{"why":"Provides the Twitter and Weibo rumour datasets and the graph-level rumour-detection task formulation used in the experiments.","marker":"[8]"},{"why":"Provides the PHEME dataset, the third test bed, with multilingual conversations that motivate the choice of multilingual BERT.","marker":"[37]"},{"why":"Supplies the pretrained multilingual BERT model used as the text embedding function, the layer whose pooling assumption Equation (3) relies on.","marker":"[38]"},{"why":"Defines the fidelity and sparsity metrics that the paper adapts to token-level resolution and uses as its primary evaluation.","marker":"[39]"},{"why":"Provides Grad-CAM, a gradient-based baseline that CT-LRP is compared against for node-level explanations.","marker":"[25]"},{"why":"Provides contrastive excitation backprop (c-EB), a decomposition-based baseline that CT-LRP is compared against.","marker":"[26]"}],"fun_headline_variants":["Contrastive token LRP lifts GNN rumour explanation fidelity","Token-level LRP with contrastive step for faithful GNN rumours","CT-LRP explains GNN rumour checks token by token","Contrastive LRP gives finer token explanations for GNN detectors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The token-level attribution rule assumes each word is embedded independently and the post vector is exactly the average of those word vectors, but the actual encoder, BERT, produces word vectors that change with context, so Equation (3) may not describe the model being explained.","fun_headline_variants_meta":{"raw":{"variants":["Contrastive token LRP lifts GNN rumour explanation fidelity","Token-level LRP with contrastive step for faithful GNN rumours","CT-LRP explains GNN rumour checks token by token","Contrastive LRP gives finer token explanations for GNN detectors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000503,"raw_usage":{"total_tokens":2452,"prompt_tokens":937,"completion_tokens":1515,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":553,"completion_tokens_details":{"reasoning_tokens":1441}},"tokens_in":553,"tokens_out":1515,"duration_ms":10370,"temperature":1.0,"reasoning_tokens":1441,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T10:31:15.704316+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run CT-LRP on a model whose node features come from BERT, then for each token the method highlights as class-specific, remove that token from the post text, re-run the full BERT-plus-GNN pipeline, and compare the predicted-class logit change against the token's attributed relevance rank. If the highest-attribution tokens do not cause the largest logit drops, the Equation (3) decomposition is not faithful. A sharper check is to verify relevance conservation directly: replace BERT with static word vectors plus mean pooling and confirm that the sum of token relevance equals the node-level relevance; with BERT, the contextualized word vectors mean mean-pooling equality generally fails, so the check should reveal the mismatch.","supporting_citations":[{"cited_title":"Rumor detection on social media with bi-directional graph convolutional networks,","cited_arxiv_id":null,"evidence_quote":"Defines BiGCN, one of the three GNN rumour-detection models whose explanations CT-LRP is evaluated on."},{"cited_title":"Towards propagation uncertainty: Edge-enhanced Bayesian graph convolutional networks for rumor detection,","cited_arxiv_id":null,"evidence_quote":"Defines EBGCN, the second evaluation model, including the edge-consistency architecture CT-LRP must backpropagate through."},{"cited_title":"Rumor detection on Twitter with claim-guided hierarchical graph attention networks,","cited_arxiv_id":null,"evidence_quote":"Defines ClaHi-GAT, the third evaluation model, with attention layers and hierarchical event representation used in the experiments."},{"cited_title":"Detecting rumors from microblogs with recurrent neural networks,","cited_arxiv_id":null,"evidence_quote":"Provides the Twitter and Weibo rumour datasets and the graph-level rumour-detection task formulation used in the experiments."},{"cited_title":"Analysing how people orient to and spread rumours in social media by looking at conversational threads,","cited_arxiv_id":null,"evidence_quote":"Provides the PHEME dataset, the third test bed, with multilingual conversations that motivate the choice of multilingual BERT."},{"cited_title":"Explainability methods for graph convolutional neural networks,","cited_arxiv_id":null,"evidence_quote":"Defines the fidelity and sparsity metrics that the paper adapts to token-level resolution and uses as its primary evaluation."},{"cited_title":"Top- down neural attention by excitation backprop,","cited_arxiv_id":null,"evidence_quote":"Provides contrastive excitation backprop (c-EB), a decomposition-based baseline that CT-LRP is compared against."}],"review_version":1}