{"id":"0a3dd579-5222-435c-b983-47f6e46c16ec","arxiv_id":"2505.20074","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"GOOD-MIA combines invariant risk minimization, a graph information bottleneck, and risk extrapolation to run membership inference attacks against graph neural networks across different data domains.","lead":"This paper introduces GOOD-MIA, an attack that guesses whether a node was used to train a graph neural network when the attacker's auxiliary data comes from a different distribution than the target. If the reported results hold, GNN privacy risks extend beyond same-distribution settings and require OOD-aware defenses.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The GIB term in Eq. (10) is never given a computable node-level form and Eq. (7) cannot be executed as written; the reported cross-domain attack gains rest on an unspecified objective.","rationale":"The central claim is that GOOD-MIA achieves superior cross-domain membership inference performance. For this claim to be evaluable, the training objectives must be executable. The IRM and REx components are standard and roughly specified, but the GIB component is used only as notational objective from Eq. (5), inserted directly into the minimization over ω in Eq. (10) and into the node update in Eq. (7), without any variational bound or node-level estimator. This is not a cosmetic omission: the ablation results assert that GIB contributes, but a reader cannot determine which quantity was actually removed. The paper also lacks error bars, uses a single attack baseline, and does not describe the synthetic shift construction, but those would be secondary if the main objective were well defined. The underlying idea is plausible and the reported gains are broadly positive, so the verdict is not a rejection of the research direction; it is a rejection of the current, non-implementable specification. If the authors supply code plus an explicit node-level GIB estimator and the tables reproduce, the paper would merit a different verdict. As submitted, the reader's REJECT is appropriate and unchanged.","tokens_in":12369,"tokens_out":4261,"duration_ms":70367,"concrete_test":"Obtain from the authors or code the exact computation of αGIB in Eq. (10), including the node-level objective and any estimator for I(G;Z) and I(Y;Z); independently reimplement GOOD-MIA with a concrete GIB surrogate (e.g., the variational upper bound in Wu et al., 2020) and rerun Table 2 with the same hyperparameters. If the full-model column is not reproduced, or the GOOD-MIA minus GIB ablation gap changes materially, the central claim depends on an unstated term and is not verifiable as written. The same check should confirm whether Eq. (7) can be executed literally for a single GCN layer on Cora.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The method's central training objective, Eq. (10), contains an αGIB term inside an otherwise standard classification loss, but GIB is defined only abstractly in Eq. (5) as a trade-off of mutual information terms I(Y;Z) and I(G;Z). No node-level variational estimator or surrogate for these mutual information terms is provided for the shadow GNN, so the objective cannot be instantiated. Eq. (7) compounds this: its UPDATE step is written as h_{e,i}^{(l+1)} = UPDATE(z_i^{(l)}, argmin_{h_{e,i}} L_GIB), with an inconsistent layer index and with L_GIB never defined as a function of h_{e,i}. Consequently, the described pipeline is not implementable from the paper alone, and the reported accuracy/AUC gains in Tables 2-4 cannot be traced to the stated objective. This is a correctness issue, not merely missing polish, because the full model versus the w/o GIB ablation is exactly what purports to demonstrate GIB's contribution to cross-domain attack performance.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GOOD-MIA, a membership inference attack against GNNs under a cross-domain setting where the shadow dataset is drawn from a different distribution than the target dataset. The method trains a shadow GNN with invariant risk minimization (IRM), a graph information bottleneck (GIB) term, and risk extrapolation (REx) for the attack classifier. The authors report accuracy, AUC, and recall on three citation networks with synthetic distribution shifts and on two social-network collections, comparing against the TSTS baseline and three ablations.","tokens_in":12580,"tokens_out":4474,"duration_ms":55236,"significance":"The cross-domain membership inference question is timely and practically motivated, and the paper is, to the best of its claims, the first to target the setting where the attacker's auxiliary graph is not identically distributed to the target graph. If the proposed objective were fully specified and the empirical gains were statistically supported, the paper would be a useful contribution to both graph privacy and OOD generalization. However, as written the central training objective is not computable from the manuscript, one of the core update equations is not well formed, and the experimental comparison lacks uncertainty quantification; the claimed contribution is therefore not established in the present text.","major_comments":[{"comment":"The overall shadow-model loss is written as min_ω Ltrain = αGIB + (1−α)RIRM, but GIB is only abstractly defined in Eq. (5) as a trade-off of mutual information terms. No node-level variational estimator, surrogate, or concrete objective is provided for I(Y;Z) and I(G;Z) in a GNN encoder, so Eq. (10) cannot be instantiated or optimized from the paper. This is load-bearing because the ablation GOOD-MIA\\GIB in Table 2 is supposed to isolate the contribution of this term, but the reader cannot tell what was removed or how the reported numbers were produced.","section":"Section 4.2, Eq. (10)"},{"comment":"The layer update h_{e,i}^{(l+1)} = UPDATE(z_i^{(l)}, argmin_{h_{e,i}} L_GIB) is not executable as written: the index on z is inconsistent with the environment subscript e, the variable h_{e,i} is not specified on the right-hand side, and L_GIB is never defined as a function of h_{e,i} or z. This is not a cosmetic typo: Algorithm 1 invokes Eq. (7) as the representation computation on line 5, so the described pipeline cannot be reconstructed from the manuscript.","section":"Section 4.2, Eq. (7)"},{"comment":"All reported numbers are point estimates with no standard deviations, confidence intervals, or significance tests. Several full-model improvements over TSTS are very small; for example, in Table 3 the ENGB/GCN row shows ACC 57.14 versus 57.04 and AUC 57.06 versus 56.99, and in Table 4 the Cornell5/Reed row shows REC 56.83 versus 53.48 with no error bars. Without repeated trials or statistical tests, the claim of 'superior attack performance' is not supported even if the objective were fully specified.","section":"Section 5.2 and Section 5.3, Tables 2-4"},{"comment":"For Cora, Citeseer, and Pubmed, the authors state that they 'synthetically create spurious node features to introduce distribution shifts,' but no protocol is given for how the spurious features are generated, how large the shift is, or how the training and target graphs are split. This prevents reproduction of the synthetic-domain experiments and makes it difficult to interpret the differences between the synthetic results in Table 2 and the real-world results in Tables 3 and 4.","section":"Section 5.1, synthetic data protocol"}],"minor_comments":[{"comment":"The notation in Eq. (8) uses fω(eXe, fAe), but fAe is not introduced; the augmented adjacency matrix is denoted eAe in Eq. (6). The symbols should be made consistent.","section":"Section 4.2, Eq. (8)"},{"comment":"The attack model is defined with parameters (ψ, b) in Eq. (11), but Algorithm 1 says 'Train ω by minimizing Eq. (13)'; the attack update should refer to ψ and b rather than the GNN encoder parameters ω.","section":"Algorithm 1, line 11"},{"comment":"The output of the attack is written as {member, non-membe}; 'non-membe' is a typo for 'non-member'.","section":"Section 3.1, Eq. (1)"},{"comment":"The sentence 'However, the adversary requires all needs a shadow dataset' is grammatically incomplete and should be rephrased for clarity.","section":"Section 2.1"},{"comment":"The complexity expression O(N L d̄²) and the statement that the overall complexity 'has a linear relationship' are not fully quantified; M copies of the encoder are computed, so the dependence on M should be stated explicitly.","section":"Section 4.3, complexity analysis"}],"recommendation":"reject","confidential_remarks":"The central issue is that the GIB term in the main training objective is not given a computable node-level form, and Eq. (7) is not well defined; these are correctness problems rather than presentation issues. The empirical claims further rest on comparisons without error bars or significance tests. I would not oppose a future submission that supplies a fully specified GIB estimator, corrects the update equations, and reports variance-aware results, but the present manuscript cannot be verified as written."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"X, quick take on 2505.20074. The genuinely new thing here is the problem: cross-domain membership inference against GNNs, where the shadow data is drawn from a different distribution than the target. That is a fair and practical extension of the usual same-distribution MIA setting, and the experimental setup on Twitch and Facebook-100 does reflect real domain gaps. The paper also runs ablations on all three borrowed components (IRM, GIB, REx) and consistently reports higher attack accuracy/AUC than the TSTS baseline. So there is a real research direction here.\n\nBut the as-written method has a load-bearing gap that keeps me from trusting the numbers. Eq. (10) defines the shadow-model loss as αGIB + (1-α)RIRM, yet GIB is only given abstractly in Eq. (5) as a trade-off of mutual information terms. No node-level variational estimator or surrogate is ever provided. The paper never says what GIB evaluates to for a node representation under the shadow GNN, so the objective cannot be instantiated. Eq. (7) compounds the problem: the UPDATE step has mismatched layer indices and refers to an argmin over an undefined L_GIB. The ablation that removes GIB is precisely the comparison meant to show GIB's contribution, but without a concrete GIB term that comparison cannot be traced to the stated objective. This is not a minor polish issue; it makes the method unimplementable from the paper alone.\n\nOther soft spots are real but less severe. Only one attack baseline (TSTS) is used; there are no error bars or significance tests; no code or data are released; and the synthetic spurious-feature shift on Cora/Citeseer/Pubmed is described only vaguely, making it hard to know how much the \"cross-domain\" gains on those datasets come from the OOD machinery versus the augmentation. There are also small typos in Algorithm 1 (training ψ but writing ω in the attack step). On the plus side, the citation pattern is honest about the builders (IRM, GIB, REx), even though the overlap with the authors' own prior attack paper (Wei et al., 2025) is not positioned explicitly.\n\nNet: the problem framing deserves attention, but the central result is not verifiable from the text. My own verdict would be reject in current form. I'd still send it to review rather than desk reject, because a serious referee could ask for a concrete GIB bound and code, and the idea is worth one revision cycle.","headline":"Plausible new problem framing, but the central GIB objective is never given a computable form, so the paper's empirical claims are not reproducible as written.","tokens_in":13170,"tokens_out":2914,"would_cite":false,"duration_ms":32711,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Cross-domain graph membership inference attacks are viable: an attacker can infer whether a node trained a target GNN using only a shadow graph from a different distribution, by learning invariant node representations and extrapolating…","keywords":["membership inference attack","graph neural networks","out-of-distribution generalization","invariant risk minimization","graph information bottleneck","risk extrapolation","cross-domain attack","privacy leakage"],"falsifier":"Train GOOD-MIA and TSTS on a shadow graph and test on a target graph from a disjoint domain, using identical backbones and hyperparameters beyond the reported ones, and report whether GOOD-MIA's accuracy and AUC stay above the TSTS baseline and random guessing. Additionally, check whether Eq. (10)'s GIB term accepts a concrete node-level instantiation; if it does not, the paper's claimed mechanism is not fully specified.","tokens_in":12128,"feed_emoji":"🔓","tokens_out":4084,"duration_ms":35851,"temperature":0.7,"pith_summary":"This paper asks whether a membership inference attack on graph neural networks still works when the attacker cannot obtain shadow data drawn from the same distribution as the target. It reframes the distribution gap as an out-of-distribution problem and proposes GOOD-MIA, a two-stage pipeline: a shadow GNN is trained under invariant risk minimization and a graph information bottleneck across data-augmented environments, then a binary attack classifier is trained with risk extrapolation. The paper reports that on citation networks with synthetic distribution shifts and on Twitch and Facebook-100 social graphs, GOOD-MIA beats the TSTS baseline on accuracy and AUC with GCN, GAT, and SGC backbones, and that removing any of the three components degrades attack performance. A sympathetic reader would come away with the claim that cross-domain graph MIAs are viable and stronger than previously assumed.","feed_headline":"Cross-domain graph attacks succeed without matching shadow data","feed_subtitle":"An IRM-plus-information-bottleneck pipeline lets an attacker infer membership from graphs of a different distribution.","key_machinery":"The load-bearing mechanism is the combination of three objectives applied to views of a single shadow graph. IRM enforces that representations predict labels equally well across augmented environments; GIB compresses away environment-specific information; and REx equalizes attack risks across environments to dampen the effect of distribution shift at inference time. Named components: Invariant Risk Minimization (IRM), Graph Information Bottleneck (GIB), and Risk Extrapolation (REx).","core_discovery":"Given a target GNN trained on graph $G_t$, an attacker with only a shadow graph from a different distribution can still infer membership of nodes by: (1) constructing $M$ augmented environments from the shadow graph via feature masking and DropEdge; (2) training the shadow GNN with the joint loss $\\mathcal{L}_{\\text{train}} = \\alpha\\,\\text{GIB} + (1-\\alpha)\\,R_{\\text{IRM}}$ to extract invariant, task-relevant node representations; and (3) training an MLP attack classifier on the shadow model's posteriors, one environment per augmented view, using variance-based risk extrapolation $R_{V-\\text{REx}}$ instead of plain cross-entropy. The paper claims this invariant-plus-bottleneck design is what makes the attack transfer across domains, with ablation results attributing each component a distinct role.","pith_inferences":["If real-world domain shifts resemble the augmented environments tested here, then current privacy evaluations that assume matched shadow data systematically understate leakage risk; evaluations should use cross-domain shadow sets.","The paper leaves the graph information bottleneck term at a symbolic level for node-level representations; making Eq. (10) concrete with a computable node-level mutual-information estimator would either strengthen or weaken the claimed mechanism.","The approach suggests a defense direction: training GNNs to produce representations that differ across domains could frustrate invariant-representation attacks, though the paper does not explore defenses."],"forward_implications":["An attacker does not need identically distributed shadow data; a differently distributed graph suffices to train an effective membership inference model.","Graph model publishers should assume membership status leaks even when an adversary observes the model from outside a black-box API with unrelated data.","The three components are individually necessary: removing IRM, GIB, or REx lowers attack accuracy in the reported citation-network ablations.","Attack performance transfers across GNN backbones (GCN, GAT, SGC), suggesting the pipeline is not tied to a specific encoder."],"supporting_citations":[{"why":"Baseline TSTS: the node-level membership inference attack on GNNs that GOOD-MIA must beat in the cross-domain setting.","marker":"[Olatunji et al., 2021]"},{"why":"Foundational shadow-model MIA formulation that GOOD-MIA extends to the graph and OOD setting.","marker":"[Shokri et al., 2017]"},{"why":"Invariant Risk Minimization, the method used in the shadow model to learn representations invariant across augmented environments.","marker":"[Arjovsky et al., 2019]"},{"why":"Risk Extrapolation, the principle used to train the attack model to be insensitive to distribution shift.","marker":"[Krueger et al., 2021]"},{"why":"Graph Information Bottleneck definition used in the shadow-model training loss.","marker":"[Wu et al., 2020a]"},{"why":"Source of the node feature masking and DropEdge augmentations used to construct multi-environment training graphs.","marker":"[You et al., 2020]"},{"why":"Prior graph membership inference attack that relied on same-distribution shadow data and supplied evaluation metrics.","marker":"[He et al., 2021]"}],"fun_headline_variants":["Cross-domain graph attacks work without matched shadow data","Out-of-distribution graph attack learns to infer membership across domains","No matched shadow set? Graph attack still reveals membership","Graph membership inference attack transfers across domains via OOD training","Invariant graph features let attackers cross domains without data match"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim rests on the assumption that invariant representations learned from augmented views of one shadow graph carry over to a genuinely different target graph, and that the graph information bottleneck term can actually be computed for node-level representations as Eq. (10) requires.","fun_headline_variants_meta":{"raw":{"variants":["Cross-domain graph attacks work without matched shadow data","Out-of-distribution graph attack learns to infer membership across domains","No matched shadow set? Graph attack still reveals membership","Graph membership inference attack transfers across domains via OOD training","Invariant graph features let attackers cross domains without data match"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00034,"raw_usage":{"total_tokens":1873,"prompt_tokens":943,"completion_tokens":930,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":851}},"tokens_in":559,"tokens_out":930,"duration_ms":10450,"temperature":1.0,"reasoning_tokens":851,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:00:18.904985+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train GOOD-MIA and TSTS on a shadow graph and test on a target graph from a disjoint domain, using identical backbones and hyperparameters beyond the reported ones, and report whether GOOD-MIA's accuracy and AUC stay above the TSTS baseline and random guessing. Additionally, check whether Eq. (10)'s GIB term accepts a concrete node-level instantiation; if it does not, the paper's claimed mechanism is not fully specified.","supporting_citations":[{"cited_title":"Membership inference attack on graph neural networks","cited_arxiv_id":null,"evidence_quote":"Baseline TSTS: the node-level membership inference attack on GNNs that GOOD-MIA must beat in the cross-domain setting."},{"cited_title":"Membership infer- ence attacks against machine learning models","cited_arxiv_id":null,"evidence_quote":"Foundational shadow-model MIA formulation that GOOD-MIA extends to the graph and OOD setting."},{"cited_title":"Out-of-distribution generalization via risk extrapolation (rex)","cited_arxiv_id":null,"evidence_quote":"Risk Extrapolation, the principle used to train the attack model to be insensitive to distribution shift."}],"review_version":1}