{"id":"38d3f4fe-1a4f-43af-be6a-1aed426764f9","arxiv_id":"2411.18648","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"MADE is a training-set-only graph backdoor defense combining homophily-based poisoned-sample isolation with masked unlearning to drive attack success rate to near zero while keeping accuracy high.","lead":"This paper proposes MADE, a training-time defense that removes backdoor triggers from graph neural networks by first detecting poisoned graphs with homophily and loss signals, then applying learned edge masks that suppress trigger influence. If it works, graph models can be cleaned without any extra clean data, which matters for security-sensitive applications such as drug discovery.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim rests on the assumption that backdoor triggers lower graph homophily; this is demonstrated only for GTA and UGBA, so a homophilous trigger would likely evade MADE's isolation and masking pipeline.","rationale":"The reader's verdict correctly identifies the same load-bearing assumption: that backdoor triggers reliably lower homophily. My stress-test concurs and sharpens the concern by showing that both stages of MADE—isolation via Eq. (4) and masking via Eq. (6)—reduce to detecting low homophily, so a trigger that preserves homophily evades the entire mechanism. This is a genuine correctness risk for the central claim, not merely a missing baseline. The paper's empirical evidence for GTA and UGBA is solid, and the near-zero ASR results are plausible for those attacks, but the claim is stated broadly ('significantly reducing the attack success rate'). Since no experiment addresses homophilous or adaptive triggers, the result is conditionally established. The absence of error bars and code further limits verification, but those are secondary to the homophily dependency. My recommendation is to keep the reader's CONDITIONAL verdict unchanged: the paper should be accepted only if the authors add experiments with homophilous triggers or explicitly scope the claim. I agree with the reader's weakest_assumption and find no other concern more load-bearing than this one.","tokens_in":112,"tokens_out":3387,"duration_ms":94104,"concrete_test":"Construct a homophilous graph backdoor attack by modifying GTA's trigger generation: after generating a trigger subgraph for each poisoned graph, optimize the trigger's node features to maximize the average cosine similarity to the neighboring host nodes, subject to a stealthiness budget, until the overall homophily of the poisoned graph stays within one standard deviation of the clean training-set homophily distribution. Train MADE on AIDS and PROTEINS with this attack at a 10% injection rate, following the exact protocol in Section 4.1, and report the ASR. If the ASR exceeds 20% (compared with near 0% for standard GTA), the homophily assumption is load-bearing and the paper's central claim needs to be scoped to low-homophily triggers.","verdict_should_be":"UNCHANGED","load_bearing_attack":"MADE's effectiveness depends on the premise that backdoor triggers make poisoned graphs less homophilous. Data isolation in Section 3.1 flags graphs whose homophily falls outside one standard deviation of the training-set mean (Eq. 4), and mask generation in Section 3.2.1 scores nodes by the cosine similarity of their projected features to neighbors (Eq. 6). Both mechanisms are built on the same low-homophily assumption. The paper demonstrates this assumption empirically only for GTA (graph classification) and UGBA (node classification), as shown in Table 4. An attacker who crafts a trigger subgraph whose node features are similar to the host graph's local structure can keep the poisoned graph's homophily within the clean distribution. Then Eq. (4) will not isolate the poisoned graphs, and Eq. (6) will assign high natural scores to trigger nodes, so the masks will not remove the trigger. In that case, the adversarial loss in Eq. (10) is applied to samples where the trigger remains active, and the model can still learn the backdoor. Thus the near-zero ASR results in Table 5, obtained against low-homophily attacks, do not support the general claim that MADE significantly reduces the attack success rate of graph backdoors; they only support it for triggers that are heterophilous with the host graph. This is the weakest point in the argument, and it is exactly the precondition that must hold for the claimed mechanism to work.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MADE, a training-time backdoor defense for GNNs that does not require additional clean data. MADE first isolates a poisoned subset using a homophily-based criterion (Eq. 4) plus low-loss selection, then generates edge masks via a learned projection head and natural scores (Eqs. 5-7), and trains with an adversarial loss on the isolated backdoor subset to unlearn triggers while a clean-subset loss preserves utility. The authors evaluate on four graph-classification datasets under the GTA attack and four node-classification datasets under the UGBA attack, reporting near-zero ASR with small accuracy drops, and include ablations on injection rates and hyperparameters.","tokens_in":45,"tokens_out":5490,"duration_ms":89733,"significance":"If the results are reproducible, MADE addresses a practical gap: existing image-based defenses such as ABL and ANP degrade on graphs, and graph-specific defenses are scarce. The method is training-set-only, end-to-end for graph classification, and the ablation in Section 4.2.4 supports the claim that masked unlearning preserves utility better than whole-graph unlearning. The main risk is the strong assumption that graph backdoor triggers reduce homophily, which is only demonstrated for two attack families; the paper should either broaden the attack coverage or temper the general claim.","major_comments":[{"comment":"The defense's isolation and masking mechanisms both assume that backdoor triggers lower graph homophily; this is demonstrated concretely only for GTA (graph classification) and UGBA (node classification) in Table 4. If a trigger subgraph is engineered to be homophilous with the host graph, Eq. (4) will not flag the poisoned graph, and Eq. (6) will assign high natural scores to trigger nodes, so the masks will not remove the trigger. The manuscript's abstract and conclusion claim general effectiveness ('significantly reducing the attack success rate'), which overstates the evidence. Either evaluate against additional attack families that include homophilous triggers, or scope the claim explicitly to low-homophily triggers.","section":"§3.1, §3.2.1, Eq. (4), Eq. (6)"},{"comment":"Section 4.1 states that each experiment is repeated five times and the mean results are recorded, yet Tables 5 and 6 report no standard deviations or other dispersion measures. Because the headline result is near-zero average ASR (e.g., MADE average ASR of 0.00 on PROTEINS in Table 5), the variance across seeds is critical to assessing whether the attack success rate is reliably near zero. Please report standard deviations (or confidence intervals) for both ASR and accuracy.","section":"§4.1, Tables 5 and 6"},{"comment":"The baseline labeled 'GCNJaccard' in Table 5 is not described in Appendix F, which instead presents 'GCN-SVD' with citation [52]. Moreover, reference [46] is titled 'Graph Information Bottleneck,' which is not GCNJaccard. This inconsistency prevents the reader from knowing which baseline was actually compared and makes the experimental comparison unreproducible. Please align the table label, the appendix description, and the references so that the baseline is unambiguously defined and cited.","section":"Table 5, Appendix F, References [46] and [52]"},{"comment":"The robustness study in Section 4.2.2 varies injection rates only for the GTA attack on the AIDS dataset. Combined with the single-attack-per-task main evaluation (GTA for graph classification, UGBA for node classification), the paper does not establish that MADE's performance generalizes across attacks with different trigger designs or different homophily behavior. At a minimum, add a discussion of this limitation; ideally, evaluate one additional attack per task to support the broader claim.","section":"§4.2.2, Figure 6"}],"minor_comments":[{"comment":"In the text after Eq. (4), 'µhomo and σhomo represents the mean and standard variance' should be 'represent the mean and standard deviation'; 'standard variance' is used inaccurately.","section":"§3.1"},{"comment":"The sentence 'To sum up, the problem of backdoor attacks on graphs can be defined as follows:' is duplicated immediately before Problem 1; remove the duplicate.","section":"§2.2"},{"comment":"The caption reads 'PROETINS' but should be 'PROTEINS'.","section":"Figure 1 caption"},{"comment":"The text says 'Graph Convoluation Network' which should be 'Graph Convolutional Network'.","section":"§4.1"},{"comment":"Line 8 selects the top α1 samples with lowest loss as Dl, and line 9 forms Dbad = Dl ∪ Dh; the relationship between the size of Dh and the isolation rate α1 is unclear, since Dh is determined by the homophily threshold. Clarify how the two sets are combined to satisfy the stated isolation rate.","section":"Algorithm 2"},{"comment":"The caption uses 'Lossadv', 'Lossclean', and 'Losssmh' while the text uses Ladv, Lnat, and Lsmh; standardize the notation.","section":"Figure 5 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely problem and the core idea is interesting, but the experimental support is narrower than the claims. The missing standard deviations and the GCNJaccard/GCN-SVD citation inconsistency should be checked by the authors before resubmission; the latter in particular may indicate that the baseline as reported was not actually run as described. I do not see a circularity problem, but the homophily assumption is a real scope limitation that should be stated explicitly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a solid contribution: a training-set-only defense for graph backdoors that combines homophily-based sample isolation with edge-mask unlearning. On the standard GTA and UGBA benchmarks it drives ASR to near zero with modest utility loss, and the analysis of why image defenses (ABL, ANP) fail on graphs is genuinely useful.\n\nWhat's new: the specific mechanism — using homophily outliers to seed a poisoned subset, then a learnable projection mask that removes edge patterns — is not in prior work. The spectral-magnitude motivation in Fig. 4 is a nice observation. The paper also ships ablations (injection rates, architecture choices, isolation precision/recall) that give the results some texture.\n\nMain concern: the whole pipeline leans on the premise that triggers make graphs less homophilous. The paper shows this for GTA and UGBA, but does not test a trigger engineered to preserve homophily. Such a trigger would likely evade Eq. (4) and Eq. (6), and the near-zero ASR would not hold. The authors should state this as a threat-model limitation rather than implying universal protection. That said, this does not refute the demonstrable effectiveness against the attacks they actually tested.\n\nSecond: reporting. The central tables (5 and 6) give means over five seeds but no standard deviations, so I cannot judge whether the 0.00 ASR is stable or driven by one seed. The baselines table lists 'GCNJaccard' while the appendix describes 'GCN-SVD' as the baseline — a discrepancy that needs fixing. No code is released, which makes the work harder to build on.\n\nAlso: only one attack per task (GTA for graph, UGBA for node). I'd want at least one more attack per task, or a clear argument that these are representative.\n\nWho it's for: the graph security subfield, and anyone building training-time defenses. Not a breakthrough that reorganizes the field, but a useful, well-motivated method. I'd send it to reviewers — conditional accept after revisions. It deserves a serious referee. If I were handling it, I'd ask for error bars, the baseline fix, a scoped claim, and ideally an evaluation against a homophilous trigger (even as a negative result).","headline":"A solid training-time graph backdoor defense with a genuine mechanism, worth refereeing — but the authors overclaim generality and need to tighten reporting.","tokens_in":20855,"tokens_out":2230,"would_cite":true,"duration_ms":20063,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MADE is a training-time graph backdoor defense that drives average attack success rates to near zero—below 3% on four graph classification datasets—while keeping accuracy within a few points of vanilla training.","keywords":["graph neural networks","backdoor defense","graph backdoor attacks","masked unlearning","graph homophily","data poisoning","graph classification","node classification"],"falsifier":"Train MADE against a backdoor attack whose trigger subgraph is engineered to be homophilous with its host graph—for example, trigger nodes connect only to feature-similar neighbors—at a 10% injection rate on AIDS. If the attack success rate stays near 100% while accuracy remains high, the homophily assumption is the load-bearing premise that fails.","tokens_in":19752,"feed_emoji":"🛡️","tokens_out":10098,"duration_ms":78114,"temperature":0.7,"pith_summary":"MADE is a training-time defense against backdoor attacks on graph neural networks that works from a poisoned training set alone, without any additional clean data. The paper's central claim is that a two-stage procedure—isolating suspected poisoned graphs with a homophily-based outlier test, then learning edge masks that remove trigger subgraphs while preserving clean structure—can reduce the attack success rate to near zero while keeping classification accuracy close to vanilla training. This matters because graph datasets are small and hard to collect, so defenses that discard poisoned samples wholesale degrade utility, and image-oriented defenses such as unlearning entire poisoned instances transfer poorly to graphs. If correct, MADE gives GNN users a practical way to train on potentially poisoned data and still obtain models that ignore triggers.","feed_headline":"Graph backdoor attacks fall to near zero with masked unlearning","feed_subtitle":"A training-time mask removes trigger subgraphs while preserving clean ones, keeping accuracy close to vanilla.","key_machinery":"The load-bearing mechanism is masked aggregation driven by a homophily-based natural score. A learnable projection head maps node features into a subspace associated with small singular values, and each node's natural score is the average cosine similarity between its projection and its neighbors' projections (Eq. 6); trigger nodes score low because they are dissimilar to their neighbors. Edge masks are then set to the original adjacency weight for edges between the top-$\\beta$ clean nodes and to the projected cosine similarity otherwise, so message passing (Eq. 8) down-weights edges that touch trigger nodes. Homophily outlier detection (Eq. 4) supplies the initial separation of poisoned graphs from clean graphs, and the adversarial loss on the poisoned subset (Eq. 10) together with cross-entropy and smoothing losses on the clean subset (Eq. 13) performs the unlearning and preserves utility.","core_discovery":"On its own terms, the paper claims that backdoor triggers on graphs have a measurable structural signature—they lower a graph's homophily score—and that this signature can be exploited for precise defense. MADE first marks as suspicious any graph whose homophily falls more than one standard deviation from the training-set mean, then adds low-loss samples to form an enriched poisoned subset and high-loss samples as a clean subset. During training, a learnable projection head maps node features into a subspace associated with small singular values, a natural score measures how similar each node is to its neighbors, and edge weights are masked so that trigger-like nodes and edges are suppressed while the top-scoring clean nodes keep their original connections. Adversarial loss on the poisoned subset unlearns the trigger while cross-entropy plus a smoothing loss on the clean subset preserves accuracy. Across AIDS, PROTEINS, PROTEINS_full, and ENZYMES under the GTA attack with a 10% injection rate, the reported average attack success rate is 0.00–2.73% with accuracy within roughly 2–5 points of vanilla training, and the method extends to node classification against UGBA.","pith_inferences":["An attacker who engineers trigger subgraphs to be homophilous with their host graphs—for instance by connecting trigger nodes only to feature-similar neighbors—would likely evade the homophily outlier test; whether the low-loss selection step could still catch such triggers is a testable open question.","The paper's spectral analysis suggests the mask mechanism could generalize to other localized structural anomalies beyond backdoors, such as adversarial edge flips or poisoned subgraphs, but that application is not tested here.","Because mask values are produced at inference time, MADE doubles as a per-node trigger detector; exposing these scores as an audit signal is a natural extension the paper does not pursue.","The comparison with ABL indicates that localized masking rather than whole-sample unlearning is what preserves utility, a principle that could inform defenses for other structured data with sparse malicious perturbations."],"forward_implications":["On the four graph classification datasets under GTA at a 10% injection rate, the reported average attack success rate drops to between 0.00% and 2.73% while average accuracy stays within roughly 2–5 points of vanilla training.","The defense is not architecture-specific: near-zero attack success rates are reported for GCN, GAT, and GraphSAGE on both graph and node classification tasks.","At injection rates from 5% to 20% on AIDS, attack success rate stays below 8% and mostly below 2%, with accuracy degrading by less than 3% even at the highest rate.","On node classification with UGBA, average attack success rate falls below 3.5% on Cora, PubMed, OGBN-Arxiv, and Flickr, showing the same design covers node-level attacks.","Masked unlearning keeps the clean part of poisoned graphs intact, which is why accuracy does not collapse the way it does when whole poisoned graphs are unlearned."],"supporting_citations":[{"why":"Supplies the GTA graph backdoor attack whose poisoned training sets MADE defends against and whose homophily statistics motivate the isolation rule.","marker":"[17]"},{"why":"Provides ABL, the training-time unlearning baseline whose isolation and unlearning design MADE analyzes and improves on for graphs.","marker":"[21]"},{"why":"Establishes that graph adversarial attacks can affect homophily, the observation that grounds the homophily-based outlier detection.","marker":"[36]"},{"why":"Contributes the spectral-signature view of backdoor samples that motivates the singular-value projection head and the natural score.","marker":"[26]"},{"why":"Defines the UGBA node-level backdoor attack used as the second threat model in the node classification experiments.","marker":"[32]"},{"why":"Supplies the homophily scoring formula that MADE adapts to separate poisoned graphs from clean ones.","marker":"[37]"},{"why":"Provides ANP, an image-oriented pruning defense baseline that fails on graphs and contrasts with MADE's topology-aware masking.","marker":"[20]"},{"why":"Provides the GCNJaccard graph-specific defense baseline that MADE outperforms on graph classification.","marker":"[46]"}],"fun_headline_variants":["Masked unlearning erases graph backdoors, preserves accuracy","Graph backdoor defense: mask triggers, unlearn attacks","MADE: Masking edge weights unlearns graph backdoor triggers","Graph backdoors neutralized by masking and unlearning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that injected trigger subgraphs reliably lower graph homophily, so a threshold at one standard deviation from the training-set mean (Eq. 4) marks poisoned graphs, and that trigger nodes are dissimilar to their neighbors so the natural score (Eq. 6) finds them.","fun_headline_variants_meta":{"raw":{"variants":["Masked unlearning erases graph backdoors, preserves accuracy","Graph backdoor defense: mask triggers, unlearn attacks","MADE: Masking edge weights unlearns graph backdoor triggers","Graph backdoors neutralized by masking and unlearning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000996,"raw_usage":{"total_tokens":4247,"prompt_tokens":1006,"completion_tokens":3241,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":622,"completion_tokens_details":{"reasoning_tokens":3171}},"tokens_in":622,"tokens_out":3241,"duration_ms":22811,"temperature":1.0,"reasoning_tokens":3171,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:41:50.715948+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train MADE against a backdoor attack whose trigger subgraph is engineered to be homophilous with its host graph—for example, trigger nodes connect only to feature-similar neighbors—at a 10% injection rate on AIDS. If the attack success rate stays near 100% while accuracy remains high, the homophily assumption is the load-bearing premise that fails.","supporting_citations":[{"cited_title":"Graph backdoor,","cited_arxiv_id":null,"evidence_quote":"Supplies the GTA graph backdoor attack whose poisoned training sets MADE defends against and whose homophily statistics motivate the isolation rule."},{"cited_title":"Anti- backdoor learning: Training clean models on poisoned data,","cited_arxiv_id":null,"evidence_quote":"Provides ABL, the training-time unlearning baseline whose isolation and unlearning design MADE analyzes and improves on for graphs."},{"cited_title":"How does heterophily impact the robustness of graph neural networks? theoretical connections and practical implications,","cited_arxiv_id":null,"evidence_quote":"Establishes that graph adversarial attacks can affect homophily, the observation that grounds the homophily-based outlier detection."},{"cited_title":"Spectral signatures in backdoor attacks,","cited_arxiv_id":null,"evidence_quote":"Contributes the spectral-signature view of backdoor samples that motivates the singular-value projection head and the natural score."},{"cited_title":"Unnotice- able backdoor attacks on graph neural networks,","cited_arxiv_id":null,"evidence_quote":"Defines the UGBA node-level backdoor attack used as the second threat model in the node classification experiments."},{"cited_title":"Finding global homophily in graph neural networks when meeting heterophily,","cited_arxiv_id":null,"evidence_quote":"Supplies the homophily scoring formula that MADE adapts to separate poisoned graphs from clean ones."},{"cited_title":"Adversarial neuron pruning purifies backdoored deep models,","cited_arxiv_id":null,"evidence_quote":"Provides ANP, an image-oriented pruning defense baseline that fails on graphs and contrasts with MADE's topology-aware masking."},{"cited_title":"Graph informa- tion bottleneck,","cited_arxiv_id":null,"evidence_quote":"Provides the GCNJaccard graph-specific defense baseline that MADE outperforms on graph classification."}],"review_version":1}