{"id":"a3767637-798f-47f4-b8a3-b979e9b3e574","arxiv_id":"2508.19641","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A systematic review that organizes graph-ML IP protection into model-level and data-level attacks and defenses, and ships a benchmark library, PyGIP.","lead":"This preprint surveys attacks and defenses for protecting graph machine learning models and data in machine-learning-as-a-service settings, and proposes a taxonomy plus an open-source library called PyGIP. It is a resource for researchers and practitioners who need to compare watermarking, fingerprinting, differential privacy, and related protection methods on graph data.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Stated black-box-only scope contradicts included white-box attacks/defenses, undermining taxonomy's comprehensiveness claim.","rationale":"The reader's weakest_assumption correctly identifies the restriction to black-box access as a potential gap. My review goes further: the paper is internally inconsistent because it states exclusivity to black-box but then incorporates white-box methods without acknowledging the contradiction. This is more load-bearing than the dataset errors (which are concrete but concentrated in Table 2 and do not invalidate the taxonomy's structure). The taxonomy's claimed comprehensiveness is directly undermined by an undefined and inconsistently applied threat-model boundary. The paper otherwise has merit: it organizes a large body of work, provides a useful structure, and offers an open-source library. The issue is fixable by adding a threat-model dimension or softening the 'exclusively black-box' language, so a conditional acceptance is appropriate rather than rejection. I partially agree with the reader because they identified the black-box restriction as a weak spot, but their concern was about completeness relative to 'practically important' threats, whereas mine is about internal consistency as evidenced by the paper's own citations.","tokens_in":48871,"tokens_out":4618,"duration_ms":53298,"concrete_test":"Audit every method cited in Sections 4 and 5 (attacks and defenses) for its assumed attacker knowledge: black-box (query-only), white-box (parameters/gradients), or gray-box. For each method, look at the original paper's threat model. Count how many cited methods require non-black-box access. If more than 10% of cited attack methods are white-box or gray-box (as appears likely for FedGIG, gradient inversion, and GrOVe), the taxonomy must either add a threat-model dimension or explicitly qualify the 'comprehensive' claim. If the count is near zero, the scope statements are acceptable as a deliberate limitation.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The survey's central claim is that it presents the first comprehensive taxonomy of GML IP protection. That claim rests on the taxonomy being internally consistent and complete. However, the threat-model scope is defined inconsistently. Section 4.1.1 states 'we focus exclusively on this setting in our discussion' after limiting model-level attacks to black-box access, and Section 4.2.2 states 'we consider these attacks exclusively under the black-box setting'. Yet the same section covers FedGIG [137] and gradient-inversion attacks [173], which require access to model gradients or parameters—i.e., white-box access. Section 5.1.2 also includes GrOVe [125], a white-box fingerprinting defense. The taxonomy's four top-level categories (model/data × attack/defense) do not encode any threat-model dimension, so the stated black-box restriction is neither consistently applied nor clearly scoping the taxonomy. If white-box threats are excluded, the taxonomy misses a material class of attacks that the paper itself cites; if they are included, the scope statements are false. Either way, the 'first comprehensive taxonomy' claim is not adequately supported as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This survey reviews intellectual property (IP) protection for graph-based machine learning as a service (GMLaaS). It organizes existing work into a proposed taxonomy with four top-level categories: model-level attacks, model-level defenses, data-level attacks, and data-level defenses. For each category it discusses representative methods, threat assumptions, and evaluation metrics. It also tabulates benchmark datasets, describes real-world application domains, and introduces an open-source library called PyGIP for implementing and comparing attack and defense methods. The paper's central claims are that it provides the first comprehensive taxonomy of GML IP protection, a systematic evaluation framework, a curated set of benchmark datasets, and a reusable software library.","tokens_in":49162,"tokens_out":6053,"duration_ms":65884,"significance":"If the central claims are substantiated, the survey would be a useful reference for a fragmented and growing area: it collects model extraction, model inversion, membership inference, watermarking, fingerprinting, adversarial training, differential privacy, perturbation-based, regularization-based, and knowledge-distillation methods under one framework. The accompanying PyGIP library is a practical contribution that could lower the barrier to reproducing and comparing methods. The evaluation-metric discussion and dataset summary are also potentially valuable. However, the paper currently contains internal contradictions in its stated threat-model scope and in its dataset statistics, and the 'first comprehensive taxonomy' claim is not fully supported as written. These issues are fixable, but they affect the survey's core reliability.","major_comments":[{"comment":"The paper's threat-model scope is internally inconsistent. Section 4.1.1 states 'we focus exclusively on this setting in our discussion' after restricting model-level attacks to black-box access, and Section 4.2.2 states 'we consider these attacks exclusively under the black-box setting.' Yet Section 4.2.2 includes FedGIG [137] and gradient-inversion attacks [173], which require access to model gradients or parameters (white-box), and Section 5.1.2 includes GrOVe [125], described explicitly as a white-box static fingerprinting method. Section 5.2.4 also mentions 'white-box gradient inspection' as part of the regularization-defense procedure. The taxonomy in Fig. 2 has no threat-model dimension, so the stated black-box restriction is not consistently applied. This undermines the 'comprehensive taxonomy' claim as written. The authors should either include an explicit white-box/gray-box dim","section":"§3, §4.1.1, §4.2.2, §5.1.2"},{"comment":"The benchmark dataset statistics are contradictory and contain clear errors. Table 2 lists Reddit as 232,965 nodes and 57,307,946 edges, while Appendix A.2 says 'over 50,000 nodes and over 860,000 edges.' LastFM is 7,083 nodes/25,814 edges in Table 2 but 7,624 nodes/27,806 edges in Appendix A.2. Computers is 245,861 edges in Table 2 but 287,209 edges in Appendix A.4. The column header '(Avg.) #Nodes' mixes aggregate and average counts across rows: MUTAG and NCI1 give averages, while Cora, Citeseer, and Reddit give totals. The AIDS row reports 31,385 nodes and 64,780 edges while Appendix A.3 says the dataset comprises 43,467 compounds (graphs). Because systematic dataset characterization is one of the paper's stated contributions, these inconsistencies must be corrected and the table/appendix aligned.","section":"Table 2 and Appendix A"},{"comment":"The claim of 'the first comprehensive taxonomy' is stronger than what is currently supported. The paper does not state the inclusion/exclusion criteria used to select literature, nor does it compare the proposed four-category partition with existing IP-protection taxonomies for deep learning (e.g., [18], [79], [80]). As noted above, the taxonomy is also applied inconsistently with respect to white-box methods. The authors should either substantiate the 'first/comprehensive' claim through a defined literature search and explicit comparison with prior taxonomies, or soften the claim and describe the taxonomy as a structured organization of the black-box GML IP-protection literature.","section":"Abstract, §1, §3"}],"minor_comments":[{"comment":"The notation is inconsistent: G is defined as (V, A, X) with V ∈ R^n, but V is later called the node set; E is introduced but not used in the graph tuple. Please clarify.","section":"§2.1"},{"comment":"The 'Differential Privacy' entry under Data-Level Defense has a dangling citation '[?]' and no reference number. This should be fixed.","section":"Fig. 2"},{"comment":"The text uses 'NCT1 & NCI109' while Table 2 uses 'NCI1'; also 'Tmall' appears in Table 2 but is not discussed in Section 6.1. Please unify names and descriptions.","section":"§6.1, Table 2"},{"comment":"Several typos remain: 'prortective' (§1), 'acadeMInvA' (§6.1), 'Legismate' (Fig. 4 caption), and 'Distutbances' (Fig. 7 caption). A careful proofread is needed.","section":"§1, §6.1, Fig. 4, Fig. 7"},{"comment":"PyGIP is a highlighted contribution, but the manuscript gives no list of implemented methods, module structure, or compatibility details. A brief appendix or link summary would help readers judge the library's coverage.","section":"§1, §8"}],"recommendation":"major_revision","confidential_remarks":"The survey addresses a timely topic and the PyGIP library is a valuable asset, but the internal contradictions in scope and dataset statistics are serious enough that the paper should not appear in its current form. The 'first comprehensive taxonomy' claim should be moderated or supported by a more rigorous comparison with existing IP taxonomies. I would be willing to review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nHere’s the short version: if you work on graph ML security, this survey is worth having on your desk. It organizes a fragmented literature—model/data attacks and defenses—into a simple four-box taxonomy, and it ships an actual open-source library (PyGIP). That combination is useful, and it is the paper’s real contribution. The taxonomy is not a new theory, but it is a reasonable frame for the field, and the reference list is broad and relevant.\n\nThe best parts: the evaluation-metrics discussion is practical, the dataset overview is a good starting point (caveat below), and the library gives people something to run rather than just read about. Self-citations are present but appropriate; they point to closely related prior work.\n\nNow the soft spots, in order of seriousness. First, the scope claims contradict the content. Section 3 and Section 4.1.1 say the paper considers only black-box attacks, and Section 4.2.2 says 'we consider these attacks exclusively under the black-box setting.' But the same sections cover FedGIG [137] and gradient-inversion attacks [173], which need gradient or parameter access, and Section 5.1.2 includes GrOVe [125], a white-box fingerprinting method. The taxonomy itself does not encode threat model, so the stated restriction is not actually applied. This is fixable—either drop the exclusivity language or add a threat-model dimension—but as written it undercuts the ‘first comprehensive taxonomy’ claim.\n\nSecond, the dataset statistics are unreliable. Table 2 gives Reddit as 232,965 nodes and 57,307,946 edges; Appendix A.2 says over 50,000 nodes and 860,000 edges. Some rows are averages (e.g., MUTAG, NCI1), others are totals (e.g., AIDS, ENZYMES), and the headers don’t tell you which. For a paper that wants to be a standard reference, that’s a real problem. There are also cosmetic issues—a dangling ‘[?]’ reference in Figure 2, ‘NCT1’ for NCI1, ‘acadeMInvA’—that a careful copyedit would catch.\n\nThird, the ‘first taxonomy’ claim is overstated. Being first is hard to establish, and ‘comprehensive’ doesn’t hold given the scope inconsistency. The taxonomy is useful as an organizing device, but it’s not a conceptual breakthrough.\n\nOverall: the central argument—that this taxonomy and library help the GML IP protection community—holds up. The flaws are concentrated in consistency and presentation, not in the core logic. I’d send it to peer review and ask for a revision. The referees would do the field a favor by forcing the scope statements and statistics to be cleaned up.\n\nFor your own work: I’d cite the taxonomy and PyGIP, but I’d double-check any dataset numbers before reusing them. I might bring it to a reading group if we’re discussing how to write surveys that claim comprehensiveness.","headline":"Useful survey of GML IP protection with a workable taxonomy and a real code library; the 'first comprehensive' claim is undercut by black-box/white-box scope contradictions and sloppy dataset stats, but it deserves a serious referee.","tokens_in":49588,"tokens_out":4100,"would_cite":true,"duration_ms":42288,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This survey claims to be the first comprehensive taxonomy of intellectual-property protection for graph machine learning, organizing attacks and defenses into model-level and data-level categories and providing an open-source library, PyGIP","keywords":["intellectual property protection","graph machine learning","model extraction attack","model inversion attack","membership inference attack","watermarking","fingerprinting","GMLaaS"],"falsifier":"A concrete check would be to enumerate the published graph-learning IP protection literature and test whether every established attack or defense fits exactly one of the four taxonomy leaves; a single widely used method that falls outside the partition, or a realistic GMLaaS theft that requires white-box access to succeed, would falsify the comprehensiveness claim. A second check would be to run a representative benchmark pair—say, a model extraction attack against a watermarking defense—through PyGIP and see whether the library reproduces the published success and fidelity numbers.","tokens_in":48804,"feed_emoji":"🛡️","tokens_out":3592,"duration_ms":43770,"temperature":0.7,"pith_summary":"The paper tries to establish that the scattered work on protecting graph machine learning models and data can be organized into a single, first-of-its-kind taxonomy with four branches: model-level attacks, model-level defenses, data-level attacks, and data-level defenses. If that organization holds, researchers and practitioners get a common language for positioning new attack or defense methods, and a shared benchmark library, PyGIP, for comparing them. The survey also argues that existing surveys of graph learning security miss the intellectual-property angle, and that existing DNN IP-protection work does not transfer directly because graph models expose structural as well as attribute information. The practical payoff would be clearer threat modeling for graph-based machine-learning-as-a-service and more reproducible evaluation of watermarking, fingerprinting, differential privacy, and related defenses.","feed_headline":"First taxonomy maps attacks on graph-model intellectual property","feed_subtitle":"Survey sorts model- and data-level threats into four branches and ships an open library, PyGIP, to test them.","key_machinery":"The load-bearing object is the four-branch taxonomy tree: model-level attack, model-level defense, data-level attack, and data-level defense, each with its own representative methods and evaluation metrics. PyGIP is the accompanying implementation layer that turns the taxonomy from a conceptual map into runnable benchmark tooling, letting a user take a taxonomy branch and immediately execute a corresponding attack or defense.","core_discovery":"On the paper's own terms, the central claim is that graph-learning IP protection can be systematically partitioned into model-level attacks (model extraction), model-level defenses (watermarking, fingerprinting, adversarial training), data-level attacks (model inversion and membership inference), and data-level defenses (differential privacy, adversarial training, topology perturbation, regularization, knowledge distillation). The authors assert this is the first taxonomy to cover both threats and defenses at both the model and data levels for graph learning. They pair the taxonomy with an evaluation framework covering attack effectiveness, attack efficiency, task utility, privacy protection","pith_inferences":["The paper's exclusive focus on black-box access means the taxonomy likely underrepresents white-box and gray-box threats, which could become more relevant as open-weight graph models and federated graph training spread.","Because defenses are grouped by mechanism rather than by formal privacy guarantee, methods placed in the same branch (e.g., differential privacy and adversarial training) may have incomparable privacy semantics; a shared quantitative threat model would be needed to truly benchmark them side by side.","The same four-branch structure could extend naturally to graph-based large language models, where the boundary between 'model theft' and 'data leakage' is blurrier than in classic GNNs.","A testable extension would be to use PyGIP as a seed for a community benchmark: if multiple groups can reproduce published attack-success and defense-fidelity numbers on the same datasets, the evaluation framework would gain the credibility the survey argues for."],"forward_implications":["New IP-protection methods can be positioned within a standard four-branch scheme, making it easier to see what a proposal adds and what it competes with.","PyGIP gives the community a common implementation platform, so accuracy, fidelity, robustness, and efficiency claims can be compared under the same codebase and datasets.","The proposed evaluation metrics give authors a checklist—attack effectiveness, attack efficiency, task utility, privacy protection, resource cost—that could become the default reporting standard for the field.","The benchmark dataset survey gives practitioners a map of which graphs (citation, social, molecular, protein, e-commerce, traffic, collaboration) support which kinds of IP-protection experiments.","Future work can use the taxonomy to identify gaps, e.g., classes of graph models or attacks that have no assigned defense branch yet."],"supporting_citations":[{"why":"Supplies the canonical black-box model extraction attack framing and the random-querying baseline that anchors the model-level attack branch.","marker":"[51]"},{"why":"Defines generator-based, data-free model extraction against GNNs, grounding one of the three model-level attack subcategories.","marker":"[53]"},{"why":"Defines model inversion attacks against graph neural networks and provides core datasets (e.g., ENZYMES) used across the survey's evaluations.","marker":"[57]"},{"why":"Introduces membership inference for graph classification and motivates the regularization-based data-level defense branch.","marker":"[127]"},{"why":"First node-level membership inference attack against GNNs, establishing the shadow-model methodology that the data-level attack discussion builds on.","marker":"[134]"},{"why":"Demonstrates watermarking of pre-trained graph encoders, supporting the claim that watermarking can protect models beyond task-specific classifiers.","marker":"[62]"},{"why":"Provides a fingerprinting framework for GNN ownership verification, grounding the fingerprinting defense branch and its evaluation metrics.","marker":"[65]"},{"why":"Systematizes differential privacy on graph-structured data, the formal foundation for the differential-privacy defense category.","marker":"[69]"},{"why":"Represents the prior trustworthy-GNN survey that the paper contrasts with, supporting the claim that existing surveys miss the IP-protection-specific perspective.","marker":"[72]"},{"why":"Provides the earlier DNN IP-protection taxonomy that this survey extends from Euclidean models to graph-structured, non-Euclidean models.","marker":"[79]"}],"fun_headline_variants":["First taxonomy covers graph-model IP attacks and defenses","Graph-model IP: new taxonomy and PyGIP test suite","Survey maps graph IP theft and ships open library PyGIP","Graph IP protection: four attack types, four defense lines","PyGIP library benchmarks graph-ML IP attack defenses"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The taxonomy's claimed comprehensiveness depends on every relevant graph-learning IP protection method fitting into the four model/data attack-defense categories, and on black-box API access being the practically important threat setting.","fun_headline_variants_meta":{"raw":{"variants":["First taxonomy covers graph-model IP attacks and defenses","Graph-model IP: new taxonomy and PyGIP test suite","Survey maps graph IP theft and ships open library PyGIP","Graph IP protection: four attack types, four defense lines","PyGIP library benchmarks graph-ML IP attack defenses"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000993,"raw_usage":{"total_tokens":4079,"prompt_tokens":814,"completion_tokens":3265,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":558,"completion_tokens_details":{"reasoning_tokens":3198}},"tokens_in":558,"tokens_out":3265,"duration_ms":23530,"temperature":1.0,"reasoning_tokens":3198,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T15:35:03.739838+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check would be to enumerate the published graph-learning IP protection literature and test whether every established attack or defense fits exactly one of the four taxonomy leaves; a single widely used method that falls outside the partition, or a realistic GMLaaS theft that requires white-box access to succeed, would falsify the comprehensiveness claim. A second check would be to run a representative benchmark pair—say, a model extraction attack against a watermarking defense—through PyGIP and see whether the library reproduces the published success and fidelity numbers.","supporting_citations":[],"review_version":1}