REVIEW 4 major objections 4 minor 21 references
Exact Certification of (Graph) Neural Networks Against Label Poisoning
T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read The paper proves that label-flipping robustness of sufficiently wide graph neural networks can be certified exactly by solving a single mixed-integer linear program.
desk verdict A genuinely new exact certificate for label poisoning against kernel SVMs / infinite-width NNs, with a fixable big-M bug and an overreaching 'exact for neural networks' claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Neural Tangent Kernel (NTK), the Gram matrix of parameter-gradients of the network, which in the infinite-width limit exactly describes training dynamics. The paper couples it with the known equivalence between wide networks trained with soft-margin loss and a bias-free kernel SVM using the NTK as its kernel. The argument then proceeds in three exact steps: replace the inner training problem α ∈ S(ỹ) by its KKT conditions, valid because Slater's condition holds; model the adversary with binary variables and a 0-norm budget constraint; and linearize every remaining product of a continuous and a binary variable, including a big-M treatment of complementarity slackness. The resulting MILPs P(y) and C(y) carry the proof, so solving them determines robustness exactly.
What would settle it
Run the MILP on a small graph and brute-force enumerate every allowed label flip, retraining the actual finite-width network for each; any flip that changes a prediction the MILP marked robust would refute the exactness claim for that practical setting.
Extended reading notes
Core claim
On the paper's own terms, the core discovery is a reformulation: for an infinitely wide GNN trained with a soft-margin loss, the training process is equivalent to solving the dual of a support-vector machine whose kernel is the GNN's Neural Tangent Kernel. Under that equivalence, label flipping becomes a bilevel optimization—the adversary picks labels, then the model retrains—and the inner retraining step can be replaced by the KKT conditions of the SVM dual, which satisfy Slater's condition for every label vector. After an exact linearization of the resulting products of continuous and binary variables, the whole problem becomes a MILP with 3m binary variables for m labeled nodes. A positive optimal value for the sample-wise MILP means the test node's class cannot be changed by any allowed flip; the collective MILP maximizes the number of test nodes whose predictions can be flipped by a single poisoning. The authors therefore claim the first exact certificate against a poisoning attack for neural networks, with guarantees holding exactly at infinite width and with high probability for wide finite networks.
Load-bearing premise
The certificate is exact only for infinitely wide networks; for a finite-width network the trained model can differ from the certified kernel SVM, so the guarantee holds with high probability and the error only vanishes as width grows.
Editorial extensions
If this is right
- Sample-wise certificates tell, for each test node and budget, whether any flip of up to ⌊εm⌋ training labels can change its prediction; collective certificates state how many test nodes a single poisoning can simultaneously flip.
- For the benchmark graphs studied, robustness rankings of GNN architectures are strongly data-dependent, so no single architecture wins everywhere.
- Linear-activation models such as SGC certify as well as or better than ReLU-based GCNs, while extra depth in skip-connection GNNs generally reduces certified robustness.
- More graph structure—stronger propagation, denser graphs, or higher homophily—improves certified robustness up to intermediate budgets and can hurt at strong budgets.
- Because the MILP only uses the kernel, the certificate applies to any kernelized SVM and to sufficiently wide networks in general, not just to GNNs.
Reading between the lines
- Editorial extension: if the robustness plateau is generic, an attacker gains little from intermediate budgets beyond a saturation point, suggesting defenses can focus on the small set of training nodes whose flip causes the first drop.
- Editorial extension: for finite-width networks used in practice, replacing the infinite-width NTK with the empirical finite-width kernel would turn the same MILP into an approximate certificate whose optimization gap could be measured directly.
- Editorial extension: the collective certificate's plateau may reflect a bottleneck structure in the training set—some labeled nodes are so influential that flipping them dominates the attack—which a graph-theoretic analysis of NTK entries could test.
- Editorial extension: the same exact linearization strategy should transfer to other training-time perturbations, such as feature poisoning or node injection, whenever the retraining step can be written with KKT conditions.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes exact robustness certificates against label-flipping poisoning for graph neural networks (and general neural networks) through an NTK-based reformulation. The authors exploit the equivalence between wide neural networks trained with a soft-margin loss and SVMs with the NTK kernel, then reformulate the resulting bilevel optimization as a mixed-integer linear program (MILP) using KKT conditions and exact linearizations. They present sample-wise certificates (Thm. 1), collective certificates that consider a single perturbation affecting all test nodes (Thm. 2), and a multi-class extension (App. A, Thm. 3). Experiments on several real and synthetic graphs compare GNN architectures and identify a robustness plateau at intermediate perturbation budgets. The code is provided.
Significance. If the technical claims hold, this is a notable advance: it is the first exact (complete) certificate for label-poisoning attacks for the class of infinite-width NTK-SVMs, and it provides both sample-wise and collective guarantees, enabling direct architectural comparisons. The MILP reformulation is original and can be applied to any kernelized SVM. The experimental study is broad, covers multiple GNN architectures and graphs, and reports a new plateauing phenomenon. The paper also clearly states that the exact guarantees apply to infinite-width networks and that finite-width certificates are high-probability with non-computable constants (Sec. 2, App. E). However, the strength of the contribution depends on the validity of the big-M constants and the correctness of the multi-class reformulation, both of which require attention.
major comments (4)
- [App. C, Eqs. (30)-(31)] The big-M constant M_ui is defined as M_ui = sum_j C|Q_ij| - 1, which is non-positive whenever sum_j C|Q_ij| < 1. The theorem statements (Thm. 1, 2, 3) require positive M_ui, and the proof in App. C itself treats the cases U_B > 1 and U_B < 1 separately before stating a single formula. With a non-positive M_ui, the constraint u_i <= M_ui s_i together with u_i >= 0 forces u_i = 0, which may exclude feasible stationarity solutions when sum_j R_ij Q_ij > 1. The authors should replace M_ui by max(0, sum_j C|Q_ij| - 1) (and similarly adjust the tightness claim), or add explicit assumptions that guarantee positivity for all i in the experiments.
- [App. A, Eqs. (15)-(21)] The multi-class exact certificate relies on the assertion that for any optimal dual solution alpha^c in S(y~^c), the prediction value sum_i y~^c_i alpha^c_i Q_ti is the same. This uniqueness claim is not true in general for a PSD but not strictly positive definite kernel matrix; different optimal alpha vectors can yield different decision values at a test point. Since the reformulation M1 -> M2 replaces a maximization over alpha^c with an existential quantification, the resulting MILP in Thm. 3 may fail to represent the true worst-case robustness. The authors need either a proof of uniqueness for the specific NTK matrices used, or a reformulation that explicitly handles the min/max over the optimal-set S(y~^c).
- [Abstract / Conclusion / Title] The claim that the method provides 'the first exact certificate to a poisoning attack ever derived for neural networks' overstates the scope. As the paper itself states in Sec. 2, exact deterministic guarantees hold only for infinite-width networks; for a fixed finite-width network the certificate is incomplete and only a high-probability bound with non-computable constants is available (App. E). The title, abstract, and conclusion should explicitly qualify the claims as applying to infinite-width NTK-based models (or to the associated NTK-SVM), so that the exactness statement is not misleading when read in the context of practical finite-width neural networks.
- [Sec. 3.1, Thm. 1] In the displayed MILP constraints, the stationarity constraint is written as 'forall i, j in [m]' before the sum over j, which is an indexing error; it should be 'forall i in [m]'. The same typo appears in Thm. 4 of App. D. This does not affect the mathematical content but should be corrected for clarity.
minor comments (4)
- [App. D, Thm. 4] The bounds l_t and h_t are defined using Q_ti without absolute values, while Sec. 3.2 correctly uses |Q_ti|. The App. D version would give incorrect bounds when Q_ti has negative entries.
- [Sec. 2, Eq. (2)] The notation S(y) is used for the set of optimal dual solutions, while later S is also used for the graph adjacency/convolution matrix (Sec. 2 and App. G). The overloaded notation is confusing; a different symbol for one of these would improve readability.
- [App. E] The probability statement 'with probability 1 - exp(-Omega(w))' should specify that the probability is over the random parameter initialization, and the dependence on the confidence should be made explicit in the bound.
- [Sec. 4, first paragraph] It is stated that 'All results concern the infinite-width limit'. This is an important clarification and should appear in the abstract as well, to avoid any ambiguity about the setting of the empirical claims.
Circularity Check
No significant circularity: the label-poisoning certificate is a genuine MILP reformulation of the NTK-SVM bilevel problem, with exactness scoped to the infinite-width limit and the finite-width caveat stated explicitly.
full rationale
The derivation chain is self-contained: the bilevel label-poisoning problem (Eq. 3) is replaced, without relaxation, by the KKT conditions of the convex SVM dual (Eqs. 5-6), and all multilinear terms are linearized exactly (Eqs. 8-10). Theorem 1's MILP checks precisely the definition of worst-case robustness; no fitted parameter is renamed as a prediction. The NTK and the wide-network/SVM equivalence are standard results with independent support (Jacot et al. 2018; Arora et al. 2019; Chen et al. 2021; Liu et al. 2020), so the paper's use of self-cited NTK formulas from Gosch et al. (2024) and Sabanayagam et al. (2023) does not make the argument circular: those formulas are parameter-free given the architecture, and the finite-width bound is also attributed to the external Liu et al. (2020). The paper explicitly scopes the guarantee: Section 2 states that exact deterministic guarantees hold for infinite-width GNNs and 'the certificate becomes incomplete for a fixed finite but not sufficiently wide network.' Appendix E admits that 'this theoretical bound is not directly computable unless constants in the derivation are preserved and applied to specific inputs.' These limitations weaken the unqualified 'first exact certificate ... for neural networks' claim, but they are correctness/scoping concerns, not circular reductions. The robustness plateauing is an empirical output of the MILP evaluations, not an input assumption. No step reduces by construction to its own inputs, so the circularity score is 0.
Assumptions & free parameters
free parameters (1)
- Regularization constant C =
0.001 to 10 depending on model/dataset (App G)
assumptions (5)
- domain assumption Infinite-width neural network training with soft-margin loss is exactly equivalent to the SVM problem with the network's NTK kernel.
- standard math The NTK of each considered GNN architecture is correctly computed as in Sabanayagam et al. (2023) and Gosch et al. (2024).
- standard math Slater's condition holds for the SVM dual for any label vector, so KKT conditions are necessary and sufficient.
- ad hoc to paper The big-M values M_ui and M_vi as defined in App C are valid and positive.
- domain assumption Gurobi solves the resulting MILPs to proven optimality within the reported runtimes.
Cite this review
Pith. "Pith review of Exact Certification of (Graph) Neural Networks Against Label Poisoning." pith.science (2026). https://pith.science/paper/5E3UZDEO
@misc{pith2026241200537,
author = {Pith},
title = {Pith review of: Exact Certification of (Graph) Neural Networks Against Label Poisoning},
year = {2026},
howpublished = {\url{https://pith.science/paper/5E3UZDEO}},
note = {Machine review of arXiv:2412.00537}
}
abstract
Machine learning models are highly vulnerable to label flipping, i.e., the adversarial modification (poisoning) of training labels to compromise performance. Thus, deriving robustness certificates is important to guarantee that test predictions remain unaffected and to understand worst-case robustness behavior. However, for Graph Neural Networks (GNNs), the problem of certifying label flipping has so far been unsolved. We change this by introducing an exact certification method, deriving both sample-wise and collective certificates. Our method leverages the Neural Tangent Kernel (NTK) to capture the training dynamics of wide networks enabling us to reformulate the bilevel optimization problem representing label flipping into a Mixed-Integer Linear Program (MILP). We apply our method to certify a broad range of GNN architectures in node classification tasks. Thereby, concerning the worst-case robustness to label flipping: $(i)$ we establish hierarchies of GNNs on different benchmark graphs; $(ii)$ quantify the effect of architectural choices such as activations, depth and skip-connections; and surprisingly, $(iii)$ uncover a novel phenomenon of the robustness plateauing for intermediate perturbation budgets across all investigated datasets and architectures. While we focus on GNNs, our certificates are applicable to sufficiently wide NNs in general through their NTK. Thus, our work presents the first exact certificate to a poisoning attack ever derived for neural networks, which could be of independent interest. The code is available at https://github.com/saper0/qpcert.
Figures
Figures from the paper (13 more)
Reference graph
Works this paper leans on
-
[1]
The political blogosphere and the 2004 us election: divided they blog
Lada A Adamic and Natalie Glance. The political blogosphere and the 2004 us election: divided they blog. In Proceedings of the 3rd international workshop on Link discovery,
work page 2004
-
[2]
We hypothesise that this is due to the fact that the most effective label perturbation may be one that flips the training labels of the predicted class to the runner-up (second highest logit score) class until the budget is exhausted. While the relaxation allows for independent changes to each of the K classifiers until the budget is used up, following th...
work page 2025
-
[4]
(iii) Technical differences: In Gosch et al. (2024), the single-level reformu- lation is a bilinear optimization (product of two continuous variables). As a product between two continuous variables can’t be modeled exactly in a linear way, Gosch et al. (2024) relax the original optimization problem resulting in the incompleteness of their certificate. In ...
work page 2024
-
[7]
Node-aware bi-smoothing: Certified robustness against graph injection attacks
12 Published as a conference paper at ICLR 2025 Yuni Lai, Yulin Zhu, Bailin Pan, and Kai Zhou. Node-aware bi-smoothing: Certified robustness against graph injection attacks. In IEEE Symposium on Security and Privacy (SP),
work page 2025
-
[9]
Attack graph convolutional networks by adding fake nodes
Xiaoyun Wang, Minhao Cheng, Joe Eaton, Cho-Jui Hsieh, and Felix Wu. Attack graph convolutional networks by adding fake nodes. arXiv preprint arXiv:1810.10751,
-
[10]
Simpli- fying graph convolutional networks
13 Published as a conference paper at ICLR 2025 Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Simpli- fying graph convolutional networks. In International Conference on Machine Learning (ICML),
work page 2025
-
[13]
Lastly, the adversaries’ strength can be modeled using mX i=1 (1 − KX c=1 y′c i ˜y′c i ) ≤ ⌊ϵm⌋ (29) 16 Published as a conference paper at ICLR 2025 Thus, we have now successfully shown how to linearly model Eqs. (24), (26) and (27) and can state the following theorem: Theorem 3 (Multiclass MILP) Given the adversary A, positive constants M c ui and M c vi...
work page 2024
-
[14]
Our experiments demonstrate that the certificates are directly applicable to kernels and standard networks, such as fully connected and convolutional networks. Since our primary focus is on the graph node classification problem, convolutional networks were not included in this study, but their inclusion would follow the same methodology. 0 0.05 0.1 0.15 0...
work page 2025
Show all 21 references
-
[15]
First, we define Slater’s condition for the SVM problem: Def
and adapted to the label-flipping case. First, we define Slater’s condition for the SVM problem: Def. 1 (Slater’s condition) A convex optimization problemP1(˜y) fulfills strong Slater’s Constraint Qualification, there exists a point α in the feasible set of P1(˜y) such that no...
2024
-
[18]
For CSBM, we choose S to Srow for GCN, SGC, GCN Skip-α and GCN Skip-PC, Ssym for APPNP with its α = 0.1
G.2 H YPERPARAMETERS We set the hyperparameters based on 4-fold cross-validation, and regarding the regularization pa- rameter C, we choose the smallest one within the standard deviation of the best validation accuracy for simulated datasets and the best one based on the valid...
2025
-
[19]
0 0.05 0.1 0.15 0.2 0.25 0.3 0.5 1 Perturbation budget ϵ 0.0 0.2 0.4 0.6 0.8 1.0 Certified Ratio GCN GCN Skip-α GCN Skip-PC (a) CSBM L = 1 0 0.05 0.1 0.15 0.2 0.25 0.3 0.5 1 Perturbation budget ϵ 0.0 0.2 0.4 0.6 0.8 1.0 Certified Ratio GCN GCN Skip-α GCN Skip-PC (b) CSBM L = 2 0...
2025
-
[20]
The plateauing 27 Published as a conference paper at ICLR 2025 phenomenon is also observed. Fig. 16 shows representative results showing the depth analysis and graph structure analysis also results in the same finding. 0 0.05 0.1 0.15 0.2 0.25 0.3 Perturbation budget ϵ 0.0 0.2...
2025
-
[1977]
Adversarial label-flipping attack and defense for graph neural networks
Mengmei Zhang, Linmei Hu, Chuan Shi, and Xiao Wang. Adversarial label-flipping attack and defense for graph neural networks. In 2020 IEEE International Conference on Data Mining (ICDM). IEEE,
2020
-
[2008]
(p = 3.17%, q = 0.74%), and µ element-wise to Kσ/ 2 √ d with d = ⌊n/ ln2(n)⌋, σ = 1, and K = 1.5, resulting in an interesting classification scheme where both graph structure and features are necessary for good generalization. CBA. Similar to CSBM, we sample nodes in a graph G...
2023
-
[2017]
Label sanitization against label flipping poisoning attacks
Andrea Paudice, Luis Mu˜noz-Gonz´alez, and Emil C Lupu. Label sanitization against label flipping poisoning attacks. In ECML PKDD 2018 Workshops: Nemesis 2018, UrbReas 2018, SoGood 2018, IWAISe 2018, and Green Data Mining 2018, Dublin, Ireland, September 10-14, 2018, Proceedings 18,
2018
-
[2018]
Carlini, M
N. Carlini, M. Jagielski, C. A. Choquette-Choo, D. Paleka, W. Pearce, H. Anderson, A. Terzis, K. Thomas, and F. Tramer. Poisoning web-scale training datasets is practical. In 2024 IEEE Symposium on Security and Privacy (SP),
2024
-
[2019]
2 to multi-class classification, we use a one-vs- all classification approach
14 Published as a conference paper at ICLR 2025 A M ULTI-CLASS LABEL CERTIFICATION To generalize the binary classification setting in Sec. 2 to multi-class classification, we use a one-vs- all classification approach. This means, given K classes, K binary learning problems are...
2025
-
[2020]
Pang Wei Koh, Jacob Steinhardt, and Percy Liang
doi: https://doi.org/10.1287/opre.2019.1944. Pang Wei Koh, Jacob Steinhardt, and Percy Liang. Stronger data poisoning attacks break data sanitization defenses. Machine Learning,
2019
-
[2022]
Adversarial machine learning-industry per- spectives
Ram Shankar Siva Kumar, Magnus Nystr ¨om, John Lambert, Andrew Marshall, Mario Goertzel, Andi Comissoneru, Matt Swann, and Sharon Xia. Adversarial machine learning-industry per- spectives. In 2020 IEEE security and privacy workshops (SPW). IEEE,
2020
-
[2023]
Provable robustness of (graph) neural networks against data poisoning and backdoor attacks
Lukas Gosch, Mahalakshmi Sabanayagam, Debarghya Ghoshdastidar, and Stephan G ¨unnemann. Provable robustness of (graph) neural networks against data poisoning and backdoor attacks. arXiv preprint arXiv:2407.10867,
-
[2024]
Dempe and J
11 Published as a conference paper at ICLR 2025 S. Dempe and J. Dutta. Is bilevel programming a special case of a mathematical program with complementarity constraints? Mathematical Programming,
2025
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.