REVIEW 3 major objections 4 minor 34 references
Impact of Graph Structure on Membership-Inference Risk for Graph Neural Networks
T0 review · 3 major / 4 minor · reviewed 2026-08-03 · deepseek-v4-flash
Pith's one-line read For node-level GNN privacy, the dominant risks are set by how the training graph is sampled and which edges an adversary can query at inference—not by the train-test accuracy gap.
desk verdict The empirical case that graph construction and inference-time edge access shape node-level membership-inference risk is real and worth engaging; the theoretical DP-auditing claim is not proven, and the appendix has a fixable but real duplication error. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the node-neighborhood tuple and the graph view used to compute it: a sample is not just a node's features and label but its L-hop neighborhood within a graph view that may be the training subgraph, the full graph, or no graph. The GNN's prediction is a function of this tuple, so changing only the edges at query time changes the model's output for a fixed node. The theoretical machinery is statistical exchangeability adapted to these tuples: the joint distribution over member and non-member tuples must be invariant under permutations for standard membership-advantage bounds of differentially private models to apply. The paper argues that inductive random and snowball spl
What would settle it
Compute the marginal joint distribution of node-neighborhood tuples for uniform random node sampling by averaging over all possible training subgraphs before evaluating a permutation; if the marginal is exchangeable, Theorem 1's conclusion fails. On the empirical side, freeze one trained GCN on Cora with a fixed 10% snowball split and re-run the membership attack under ORIG/ALLEDGES/NOGRAPH while randomizing only which inter-train-test edges are supplied; if advantage stays flat across regimes, the claim that edge access modulates MI risk is refuted.
Extended reading notes
Core claim
The paper's central claim is that in node-level membership inference against GNNs, the relations between nodes are not a nuisance but a central privacy variable. Each query is formalized as a node-neighborhood tuple—node features, label, and the L-hop neighborhood restricted to a chosen graph view—so both how the training subgraph is built and which edges are visible at query time enter the membership-inference game directly. Across random and snowball training-graph construction and three inference regimes (original split edges only, all graph edges, no edges), the experiments show that snowball sampling often harms generalization through coverage bias, exposing all edges at inference usual
Load-bearing premise
The theoretical non-exchangeability argument assumes that conditioning on the realized training subgraph G′ is the right way to evaluate permutation invariance; the same conditioning would make any i.i.d. sampled training set look non-exchangeable, so the graph-specific failure is not proven at the marginal distribution level.
Editorial extensions
If this is right
- If the generalization gap is not a reliable proxy for membership advantage in GNNs, privacy audits on graphs must record and control the inference-time edge regime separately from accuracy differences.
- Standard DP auditing bounds for node-level GNN tasks are not directly usable in inductive settings; graph-aware lower-bound auditing methods are needed.
- Training-graph construction is a privacy decision: snowball sampling's coverage bias changes both generalization and membership advantage, so choosing a sampling strategy is also choosing a risk profile.
- Adversaries who know the full graph can sometimes obtain higher, sometimes lower, membership advantage than adversaries with no edges, so threat models must specify edge knowledge as a first-class parameter.
- Architectures that preserve the ego node's own signal (GraphSAGE-style concatenation) appear less sensitive to edge access, suggesting model choice interacts with graph-driven privacy risk.
Reading between the lines
- If the non-exchangeability result can be established at the marginal level rather than conditioned on the realized subgraph, it would imply that any DP guarantee for a GNN trained on an induced subgraph cannot be audited by standard membership-inference attacks alone, because attack success would mix genuine leakage with structural correlation artifacts.
- One testable extension is to predict per-node membership advantage from structural features such as degree, centrality, or neighborhood homophily; the paper's finding that edge access dominates in low-homophily Chameleon and near-zero-gap Pubmed cases suggests structural position matters.
- The all-edges regime's tendency to shrink the gap while sometimes increasing membership advantage hints that using more graph context at inference can 'relax' the model toward test nodes while still exposing training-specific aggregation signatures—an effect worth probing with calibrated confidence scores.
- A concrete design consequence: privacy-preserving GNN deployments should specify the adversary's graph view as part of the system threat model, and defenses such as neighborhood truncation or edge perturbation could be evaluated as privacy controls under the paper's framing.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies node-level membership inference (MI) against graph neural networks in inductive settings. It formalizes MI over node–neighborhood tuples and systematically varies two graph-specific factors: training-graph construction (random vs. snowball sampling) and inference-time edge access (original split, all edges, no graph). Experiments on Cora, Chameleon, and PubMed with GCN, GraphSAGE, and GAT report train/test accuracy, performance gaps, and membership advantage. The authors claim that graph structure—not just overfitting—is a dominant determinant of MI risk, that the generalization gap is an insufficient proxy, and that inductive train/test splits break statistical exchangeability, undermining standard DP-based membership-advantage auditing. Code and data are released.
Significance. If fully supported, this would be a valuable contribution: it isolates two graph-specific axes usually absent from MI evaluations, provides a broad benchmark, and raises a legitimate concern about the validity of DP audits for graph models. The release of code and data is a strength. However, the theoretical claim is currently not established: the proof of Theorem 1 conditions on the realized graph in a way that would also invalidate exchangeability for i.i.d. data, and the appendix performance-gap tables contain internal inconsistencies. The qualitative empirical findings (edge access often narrows the gap; membership advantage can diverge from the gap) are plausible but need corrected reporting before the paper can be accepted.
major comments (3)
- [Section VII, Definition 2/Eq. (9), Theorem 1 Cases 1/2] Eq. (9) defines exchangeability as invariance of the unconditional joint distribution over the tuple sequence, but the proof fixes the realized training subgraph G' and asserts Pr(Zσ|G')=0 while Pr(Z|G')>0. This conditional argument applies to any i.i.d. sampling procedure once the realized training set is observed, so it cannot establish a graph-specific failure. The graph-specific failure might be provable from the different graph views in which member and non-member neighborhoods are computed, but that argument is absent. As written, Theorem 1 does not demonstrate that inductive GNN splits break exchangeability, and the conclusion that standard DP-based membership-advantage bounds are inapplicable is unsupported. Please provide a marginal-level proof with an explicit sampling model, or substantially weaken the claim.
- [Appendix Tables IX–XIV; Section VI-B.1] In all performance-gap tables, the snowball columns are identical to the random columns (e.g., Table IX, CORA 10% GCN PERF. GAP = 21.9735±0.7326 for both random and snowball; Table XIII, PUBMED 10% GCN = 10.1968±0.5254 for both). This contradicts the raw accuracies in Tables III–VIII (e.g., CORA 10% GCN ORIG: random 0.9889/0.7716 vs snowball 0.9970/0.7151, gaps ≈22.0% vs ≈28.3%). Thus the reported snowball performance gaps and the qualitative comparisons in VI-B.1 are not supported as presented. The performance-gap values must be recomputed from the raw accuracies and the text/figures updated.
- [Definition 2 and Experiment 1, step 5] The graph view used to construct the non-member tuple is not specified. If N^L_{G'}(v) is computed over the induced training subgraph for all tuples, then any non-member has an empty neighborhood by construction, making non-exchangeability a definitional artifact. If the adversary uses a different graph view (e.g., the full graph), the sampling model should say so. The joint data-generating process must be defined explicitly before claiming a violation of exchangeability.
minor comments (4)
- [Throughout] Typos and grammar: 'gurantees' (II-B), 'excpetions' (VI-B.2), 'manipulataion' (VI-C), and the phrase 'we empirically compute the above advantage by taking the maximum of difference' (III-B) need correction.
- [Section V-D] Cross-reference mismatch: the text says 'As Figure 3 shows' when discussing logit-transformed confidence distributions, but the relevant figure appears to be Figure 4. Also, Figure 3's three panels are labeled Original/Full graph/No edges in the text but the caption is sparse.
- [Section VI-C] The sentence 'When the overall trend holds that membership advantage is positively correlated with high membership advantage' appears to mean 'high performance gap'; please fix.
- [Section III-A] The attack model is trained with true membership labels for 80% of member and non-member instances; this is an unusually strong assumption. A sentence justifying that relative comparisons across edge regimes are invariant to this choice would help readers calibrate the results.
Circularity Check
Theorem 1's non-exchangeability is built into Definition 2, so the paper's central theoretical conclusion is definitional rather than derived.
-
self definitional
[Section VII, Definition 2 (Eq. 9) and Theorem 1, Case 1 proof]
"Let D denote a joint distribution over n member samples {z1, . . . , zn} and a non-member sample z_{n+1}, where each sample zv corresponds to a tuple: zv := (xv, yv, N^L_{G′}(v), {xu : u ∈ N^L_{G′}(v)}), ... The graph G′ may correspond to the full graph G, or to a subgraph induced by the sampled training node set ... Pr(Zσ|G′)=0, while Pr(Z|G′)>0."
Experiment 1 already draws the non-member from V\V_S and defines tuples through the training-induced graph G′; the non-member is thus distinguished from members before any permutation is considered. Conditioning on the realized G′ and observing that swapping in x_{n+1} gives Pr(Zσ|G′)=0 simply restates that x_{n+1} was not in S; the same argument would 'disprove' exchangeability for i.i.d. data conditioned on the realized training set. Therefore Theorem 1 does not establish a graph-specific failure of statistical exchangeability—it builds the member/non-member asymmetry into Definition 2 and then recovers it. The conclusion that DP-based membership-advantage bounds do not transfer to inductive GNNs is not independently derived.
full rationale
The paper's experimental contribution is largely independent: membership advantage is measured on three external datasets (CORA, CHAMELEON, PUBMED) across three GNN families (GCN, GraphSAGE, GAT), with train-graph construction (random vs. snowball) and inference-time edge access (ORIG/ALLEDGES/NOGRAPH) varied; no target quantity is fitted and then renamed as a prediction. The generalization-gap 'insufficient proxy' claim is an empirical observation, not a fitted tautology. The circularity is concentrated in Section VII: Theorem 1's proof conditions on the realized training subgraph G′ and treats a permutation of sample indices as swapping in the node outside S. Because Definition 2 and Experiment 1 already make the non-member a different kind of object (drawn from V\V_S, with tuple evaluated under the training-induced graph view), the claimed violation of Eq. (9) holds by construction for any sampling scheme, not specifically because of graph structure. Thus the headline theoretical conclusion about standard DP-based auditing bounds is already assumed in the definition of the joint distribution. Self-citations [12] and [24] are used for setup, not as load-bearing justifications. Separately, and not as circularity, Appendix Tables IX–XIV show identical snowball and random performance-gap values that contradict the raw accuracies in Tables III–VIII; this is a correctness/consistency concern, not a circularity concern.
Assumptions & free parameters
free parameters (4)
- snowball neighbor expansion parameter k =
3
- snowball seed set size =
10 nodes per class
- attack model labeled training fraction =
80%
- logit clipping epsilon =
1e-6
assumptions (4)
- standard math Binomial node-degree retention formula (Eq. 8) under uniform random sampling.
- domain assumption GNN message-passing aggregation equations (Eqs. 1–7) describe the victim models.
- domain assumption Statistical exchangeability of train/test samples is required for the DP-based membership-advantage bounds of [14,15].
- ad hoc to paper Conditioning on the realized sampled graph G' is a valid test of exchangeability.
Cite this review
Pith. "Pith review of Impact of Graph Structure on Membership-Inference Risk for Graph Neural Networks." pith.science (2026). https://pith.science/paper/GUCISY3U
@misc{pith2026260117130,
author = {Pith},
title = {Pith review of: Impact of Graph Structure on Membership-Inference Risk for Graph Neural Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/GUCISY3U}},
note = {Machine review of arXiv:2601.17130}
}
read the original abstract
Graph neural networks (GNNs) are widely used for tasks such as node classification and link prediction, but their use in sensitive settings raises concerns about training-data leakage. Prior work on privacy leakage in GNNs largely borrows assumptions from non-graph domains, overlooking the role of graph structure. We argue for a graph-specific analysis of privacy risk and study how graph structure affects node-level membership inference. We formalize membership inference (MI) over node-neighborhood tuples and investigate two important dimensions: (i) training-graph construction and (ii) inference-time edge access. We compare snowball sampling, a structure-aware procedure, with uniform random node sampling for constructing training graphs. Our experiments show that snowball sampling often hurts generalization relative to random sampling due to its coverage bias. In contrast, allowing access to inter-train-test edges at inference improves test accuracy, reduces the train-test gap, while also having a strong and setting-dependent effect on membership advantage. These results show that graph structure directly shapes privacy risk. We further show that the generalization gap, measured as the performance difference between training and test nodes, is an incomplete proxy for membership inference risk: membership advantage can rise or fall independently of changes in this gap, with inference-time edge access often playing a crucial role. Theoretically, we show that for node-level tasks, standard privacy-auditing results based on membership inference do not directly carry over to inductive graph settings, because training and test nodes are structurally dependent rather than interchangeable. We release the code and data at https://github.com/PriXAI/GraphStructurePrivacyAnalysis-public.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Utilizing graph machine learning within drug discovery and development,
T. Gaudelet, B. Day, A. R. Jamasb, J. Soman, C. Regep, G. Liu, J. B. Hayter, R. Vickers, C. Roberts, J. Tanget al., “Utilizing graph machine learning within drug discovery and development,”Briefings in bioinformatics, vol. 22, no. 6, p. bbab159, 2021
2021
-
[2]
Integration of multiomics data with graph convolutional networks to identify new can- cer genes and their associated molecular mechanisms,
R. Schulte-Sasse, S. Budach, D. Hnisz, and A. Marsico, “Integration of multiomics data with graph convolutional networks to identify new can- cer genes and their associated molecular mechanisms,”Nature Machine Intelligence, vol. 3, no. 6, pp. 513–526, 2021
2021
-
[3]
Learning to simulate complex physics with graph networks,
A. Sanchez-Gonzalez, J. Godwin, T. Pfaff, R. Ying, J. Leskovec, and P. Battaglia, “Learning to simulate complex physics with graph networks,” inInternational Conference on Machine Learning. PMLR, 2020, pp. 8459–8468
2020
-
[4]
Quantifying privacy leakage in graph embedding,
V . Duddu, A. Boutet, and V . Shejwalkar, “Quantifying privacy leakage in graph embedding,” inMobiQuitous 2020-17th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services, 2020, pp. 76–85
2020
-
[5]
Label-only membership inference attack against node-level graph neural networks,
M. Conti, J. Li, S. Picek, and J. Xu, “Label-only membership inference attack against node-level graph neural networks,” inProceedings of the 15th ACM Workshop on Artificial Intelligence and Security, 2022, pp. 1–12
2022
-
[6]
Stealing links from graph neural networks,
X. He, J. Jia, M. Backes, N. Z. Gong, and Y . Zhang, “Stealing links from graph neural networks,” in30th USENIX security symposium (USENIX security 21), 2021, pp. 2669–2686
2021
-
[7]
How powerful are membership inference attacks on graph neural networks?
A. Jnaini, A. Bettar, and M. A. Koulali, “How powerful are membership inference attacks on graph neural networks?” inProceedings of the 34th International Conference on Scientific and Statistical Database Management, 2022, pp. 1–4
2022
-
[8]
Subgraph structure membership inference attacks against graph neural networks,
X. Wang and W. H. Wang, “Subgraph structure membership inference attacks against graph neural networks,”Proceedings on Privacy Enhanc- ing Technologies, 2024
2024
Show all 34 references
-
[9]
Adapting membership inference attacks to gnn for graph classification: Approaches and implications,
B. Wu, X. Yang, S. Pan, and X. Yuan, “Adapting membership inference attacks to gnn for graph classification: Approaches and implications,” in 2021 IEEE International Conference on Data Mining (ICDM). IEEE, 2021, pp. 1421–1426
2021
-
[10]
A comprehensive survey on trustworthy graph neural networks: Privacy, ro- bustness, fairness, and explainability,
E. Dai, T. Zhao, H. Zhu, J. Xu, Z. Guo, H. Liu, J. Tang, and S. Wang, “A comprehensive survey on trustworthy graph neural networks: Privacy, ro- bustness, fairness, and explainability,”arXiv preprint arXiv:2204.08570, 2022
2022 arXiv
-
[11]
Membership inference attacks on knowledge graphs,
Y . Wang, L. Huang, P. S. Yu, and L. Sun, “Membership inference attacks on knowledge graphs,”arXiv preprint arXiv:2104.08273, 2021
2021 arXiv
-
[12]
Membership inference attack on graph neural networks,
I. E. Olatunji, W. Nejdl, and M. Khosla, “Membership inference attack on graph neural networks,” in2021 Third IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA). IEEE, 2021, pp. 11–20
2021
-
[13]
Membership inference attacks against machine learning models,
R. Shokri, M. Stronati, C. Song, and V . Shmatikov, “Membership inference attacks against machine learning models,” in2017 IEEE symposium on security and privacy (SP). IEEE, 2017, pp. 3–18
2017
-
[14]
Privacy risk in machine learning: Analyzing the connection to overfitting,
S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha, “Privacy risk in machine learning: Analyzing the connection to overfitting,” in2018 IEEE 31st computer security foundations symposium (CSF). IEEE, 2018, pp. 268–282
2018
-
[15]
Investigating membership inference attacks under data dependencies,
T. Humphries, S. Oya, L. Tulloch, M. Rafuse, I. Goldberg, U. Hengart- ner, and F. Kerschbaum, “Investigating membership inference attacks under data dependencies,” in2023 IEEE 36th Computer Security Foun- dations Symposium (CSF). IEEE, 2023, pp. 473–488
2023
-
[16]
Semi-supervised classification with graph convolutional networks,
T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in5th International Conference on Learning Representations, ICLR 2017, 2017
2017
-
[17]
Inductive representation learning on large graphs,
W. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,”Advances in neural information processing systems, vol. 30, 2017
2017
-
[18]
Graph attention networks,
P. Veli ˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y . Ben- gio, “Graph attention networks,”arXiv preprint arXiv:1710.10903, 2017
2017 arXiv
-
[19]
Membership inference attacks from first principles,
N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis, and F. Tramer, “Membership inference attacks from first principles,” in2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022, pp. 1897–1914
2022
-
[20]
Ml-leaks: Model and data independent membership inference at- tacks and defenses on machine learning models,
A. Salem, Y . Zhang, M. Humbert, P. Berrang, M. Fritz, and M. Backes, “Ml-leaks: Model and data independent membership inference at- tacks and defenses on machine learning models,”arXiv preprint arXiv:1806.01246, 2018
2018 arXiv
-
[21]
Membership inference attacks on machine learning: A survey,
H. Hu, Z. Salcic, L. Sun, G. Dobbie, P. S. Yu, and X. Zhang, “Membership inference attacks on machine learning: A survey,”ACM Computing Surveys (CSUR), vol. 54, no. 11s, pp. 1–37, 2022
2022
-
[22]
Calibrating noise to sensitivity in private data analysis,
C. Dwork, F. McSherry, K. Nissim, and A. Smith, “Calibrating noise to sensitivity in private data analysis,” inTheory of cryptography conference. Springer, 2006, pp. 265–284
2006
-
[23]
Scalable private learning with pate,
N. Papernot, S. Song, I. Mironov, A. Raghunathan, K. Talwar, and ´U. Erlingsson, “Scalable private learning with pate,”arXiv preprint arXiv:1802.08908, 2018
2018 arXiv
-
[24]
Releasing graph neural networks with differential privacy guarantees,
I. E. Olatunji, T. Funke, and M. Khosla, “Releasing graph neural networks with differential privacy guarantees,”Transactions on Machine Learning Research, 2023
2023
-
[25]
Deep learning with differential privacy,
M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep learning with differential privacy,” inProceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’16. New York, NY , USA: Association for Computing ...
2016
-
[26]
Preserving node-level privacy in graph neural networks,
Z. Xiang, T. Wang, and D. Wang, “Preserving node-level privacy in graph neural networks,” in2024 IEEE Symposium on Security and Privacy (SP). IEEE, 2024, pp. 4714–4732
2024
-
[27]
Gap: Differentially private graph neural networks with aggregation perturbation,
S. Sajadmanesh, A. S. Shamsabadi, A. Bellet, and D. Gatica-Perez, “Gap: Differentially private graph neural networks with aggregation perturbation,”arXiv preprint arXiv:2203.00949, 2022
2022 arXiv
-
[28]
Locally private graph neural networks,
S. Sajadmanesh and D. Gatica-Perez, “Locally private graph neural networks,” inProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, 2021, pp. 2130–2145
2021
-
[29]
Evaluating differentially private machine learning in practice,
B. Jayaraman and D. Evans, “Evaluating differentially private machine learning in practice,” in28th USENIX security symposium (USENIX security 19), 2019, pp. 1895–1912
2019
-
[30]
Auditing differentially private machine learning: How private is private sgd?
M. Jagielski, J. Ullman, and A. Oprea, “Auditing differentially private machine learning: How private is private sgd?”Advances in Neural Information Processing Systems, vol. 33, pp. 22 205–22 216, 2020
2020
-
[31]
Sampling from large graphs,
J. Leskovec and C. Faloutsos, “Sampling from large graphs,” inProceed- ings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, 2006, pp. 631–636
2006
-
[32]
Node-level membership inference attacks against graph neural networks,
X. He, R. Wen, Y . Wu, M. Backes, Y . Shen, and Y . Zhang, “Node-level membership inference attacks against graph neural networks,”arXiv preprint arXiv:2102.05429, 2021
2021 arXiv
-
[33]
Collective classification in network data,
P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, and T. Eliassi- Rad, “Collective classification in network data,”AI magazine, vol. 29, no. 3, pp. 93–93, 2008
2008
-
[34]
Multi-scale attributed node embedding,
B. Rozemberczki, C. Allen, and R. Sarkar, “Multi-scale attributed node embedding,” 2019. APPENDIX In the following we present detailed performance scores, performance gaps and membership advantage for different models and datasets under different settings. TABLE III: Performan...
2019
Reviewed August 3, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.