Pith. sign in

REVIEW 4 major objections 6 minor 15 references

Achieving Network Resilience through Graph Neural Network-enabled Deep Reinforcement Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a GNN-DRL framework, combining GCN-based encrypted-traffic detection with a security-weighted DQN reward, produces routing policies that avoid attack nodes and thereby improve network resilience.

desk verdict The survey half of this paper is solid and worth reading; the case study, however, does not support the 'superiority' claim because the baseline is evaluated on a reward it never optimized and the attack detector is never validated. read the letter →

arxiv 2501.11074 v1 pith:BLGJU4QR submitted 2025-01-19 cs.CR

Xuzeng Li , Tao Zhang , Jian Wang , Zhen Han , Jiqiang Liu , Jiawen Kang , Dusit Niyato , Abbas Jamalipour This is my paper · ORCID
classification cs.CR
keywords networkresiliencegraphneuralnetworksdeepreinforcementlearningencryptedtrafficclassificationroutingoptimizationIoTsecurityCICIoT2022GCN-DQN
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Deep reinforcement learning can optimize network routing, but standard DRL agents are blind to security: they keep using attacked nodes and their policies oscillate under attack. This paper argues that grafting a graph neural network onto the DRL agent closes that gap: a GCN reads the network topology and encrypted traffic, marks attack-prone nodes, and a DQN chooses paths that avoid them. The authors propose a three-phase framework (graph modeling, GNN-DRL decision-making, policy execution) and back it with a case study on CICIoT2022 IoT traffic, where their method converges faster and to a higher average reward than a GNN-DRL baseline that ignores attacks. The value, if the claim holds, is a single data-driven control loop that couples attack detection with defensive routing instead of treating them as separate problems.

What carries the argument

Two components carry the argument. The first is a two-layer GCN detector, designed following TFE-GNN, that classifies encrypted traffic and assigns each node a security weight $w$ from its attack traffic ratio; the second is a DQN whose reward $R = -\sum_{n \in p_1}(w_n + c_n) - \sum_{m \in p_2}(w_m + c_m) - \alpha O$ couples those weights with communication costs $c$ and a penalty $\alpha O$ for overlapping paths $p_1$ and $p_2$. The GCN gives the DRL agent a security-aware state representation, and the reward makes attack avoidance part of the optimization objective, so the learned policy routes around nodes the detector distrusts.

What would settle it

Run the GCN detector on held-out CICIoT2022 samples and measure its precision and recall at the operating point used in the case study; then recompute the final routing policies and check how many selected paths contain ground-truth attack nodes. If the detector misses most attacks, or if the chosen paths do not contain fewer labeled attack nodes than the MPNN-DQN baseline, the resilience claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that security-weighted GNN-DRL can turn encrypted-traffic classification into an actionable routing policy. The framework models the network as a graph, uses a two-layer GCN following TFE-GNN to extract attack features from encrypted traffic, summarizes the environment state, and lets a DQN agent select pairs of routing paths under a reward that penalizes communication cost, node overlap, and attack-node usage. In 20-node and 50-node topologies reconstructed from CICIoT2022, the authors report that this GCN-DQN stabilizes at higher average rewards than MPNN-DQN, which oscillates and degrades when attacks are present. They interpret this as evidence that attack-aware GNN-DRL improves network resilience against flood and brute-force attacks.

Load-bearing premise

Everything hinges on the GCN detector correctly identifying attack nodes, yet the paper reports no detection accuracy, precision, or recall; if the security weights do not track real attacks, the DQN optimizes a reward that looks security-aware without actually avoiding compromised nodes.

Editorial extensions

If this is right

  • Attack-aware routing becomes a byproduct of the same learning loop: once the GCN identifies risky nodes, the DQN adjusts routes without hand-coded avoidance rules.
  • Reported per-episode times (0.031 s for the 20-node network, 0.257 s for the 50-node network) suggest the combined detector and router can operate at control-loop speeds rather than only offline.
  • Because the reward separates node cost and security weight, the framework can be extended to latency-, energy-, or throughput-aware routing by adding terms to the node cost.
  • The three-phase architecture gives a template for adding attack detection to other DRL network-control tasks such as virtual network embedding and resource allocation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not report detection accuracy for the GCN; a direct reader test would measure precision and recall on the CICIoT2022 labels and then count how many chosen paths contain ground-truth attack nodes.
  • A natural stress test is to vary the fraction of attack nodes in the reconstructed topology and check whether the reward gap versus MPNN-DQN grows with attack severity; the paper only uses one attack configuration.
  • Security weighting could also be applied to moving-target defense, where the DQN would randomize routes over a set of low-risk paths; the paper discusses defense broadly but does not instantiate that variant.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a framework that combines graph neural networks (GNNs) with deep reinforcement learning (DRL) to improve network resilience against attacks. After a tutorial-style overview of GNN-DRL methods and their applications in communication networks, the authors present a case study in which a GCN-based attack detector (adapted from TFE-GNN) is integrated with a DQN agent that selects routing paths. The reward function penalizes communication cost, path overlap, and node-level security weights derived from attack traffic. Experiments are performed on two reconstructed topologies (20 and 50 nodes) from the CICIoT2022 dataset, and the proposed GCN-DQN is compared against an MPNN-DQN baseline. The paper claims that the proposed method 'can converge quickly and generate the optimal routing policy' and that the results 'demonstrated the effectiveness and superiority of our framework' (Section IV.C, Abstract).

Significance. If the empirical claims were fully supported, the paper would offer a useful integration of attack detection and defense-oriented routing in a GNN-DRL setting, and its tutorial component provides a structured overview of GNN-DRL for network security. However, the validation as reported is not conclusive. The comparison with the baseline is circular because both methods are evaluated on the same engineered reward that only the proposed method is trained to optimize, and the attack-detection component is never validated with any accuracy metric. The paper ships no code, no formal convergence or optimality analysis, and no statistical confidence measures. Consequently, the central contribution is only partially supported, and the 'superiority' claim should be substantially moderated or backed by additional evidence. The tutorial and taxonomy sections are of interest to readers new to the area, but the case study needs rework before the paper can be accepted.

major comments (4)
  1. [Section IV.B, IV.C, Fig. 6] The main empirical comparison is circular. The proposed GCN-DQN is compared with MPNN-DQN using the average reward R, which is the SAME reward the proposed method is trained to optimize. Section IV.B states that the baseline MPNN-DQN 'considers only communication cost without accounting for attacks,' so the baseline never optimizes the security-weight terms w_n and w_m in R. The higher R of the red curve is therefore expected by construction: it reflects the added penalty term, not better network resilience. To substantiate superiority, the authors should report a direct attack-avoidance metric (e.g., the fraction of selected paths that contain attack nodes) or evaluate both methods under an identical reward function.
  2. [Section IV.A] The security weight w is defined as an integer in [1,200] assigned 'by considering the attack traffic ratio,' but no formula, threshold, or algorithm is provided. This omission is load-bearing because w is the only attack-related signal in the reward R = -Σ_{n∈p1}(w_n+c_n) - Σ_{m∈p2}(w_m+c_m) - α·O. Furthermore, the GCN detector designed following TFE-GNN [2] is never validated: no accuracy, precision, recall, or F1 scores are reported on the CICIoT2022 traffic. Without evidence that w correctly identifies attack nodes, the reward may be misaligned with true network security, and the resilience claim becomes unsupported.
  3. [Section IV.C, Fig. 6] All results are reported as averages over ten experiments with no error bars, standard deviations, or statistical significance tests. The smoothed curves in Fig. 6 cannot support the statements 'can converge quickly' and 'generate the optimal routing policy,' especially because the 20-node red and green curves are numerically close after early episodes (e.g., -130.1 vs -156.5 for episodes 2500-5000, and -50.5 vs -50.3 for episodes 10000-12500). The authors should provide variance or confidence intervals and specify exactly how the table entries were computed from the ten runs.
  4. [Section IV.B] The reconstruction of the two network topologies from CICIoT2022 is described too briefly for reproducibility. The paper does not say how the 20-node and 50-node graphs are derived from the dataset, how edges are assigned, how per-node traffic is 'kept consistent' with the dataset, or how attack labels in CICIoT2022 are mapped to graph nodes. Without these details, the reported reward values are difficult to interpret, and the case study cannot be independently reproduced.
minor comments (6)
  1. [Section II.A.2] The phrase 'conductive learning' should be 'transductive learning' when describing GCNs.
  2. [Section IV.A] The values of the penalty factor α and the communication costs c are not specified beyond the range [1,200]; please state the actual choices used in the experiments.
  3. [Section IV.A] The action-space sizes 464 and 39,038 are mentioned but not explained; please specify how these counts are derived from the path sets for the two routing requirements.
  4. [Section IV.C] The time-consumption numbers (0.49s per 1000 packets, 0.031s and 0.257s per episode) lack context; please report the hardware/software environment and whether these are training or inference times.
  5. [Section II.B] There is a typo: 'has became more and more popular' should be 'has become more and more popular.'
  6. [Section IV.C] The table caption says 'the bolded values representing the optimal ones,' but the text does not explain which values are bolded or why; ensure the table formatting is consistent with the caption.

Circularity Check

1 steps flagged · score 6.0 of 10

The claimed attack-avoidance superiority is measured on the same engineered reward the agent was trained to optimize, while the baseline was optimized under a different reward.

  1. fitted input called prediction [Section IV.A (Parameters setup, reward function) and Section IV.C (Performance evaluation, Fig. 6)]
    "By considering the attack traffic ratio, each node is assigned a security weight w (an integer in the range [1, 200]). Therefore, the reward function R = − P n∈p1 (wn + cn) − P m∈p2 (wm + cm) − α · O. ... The comparison of reward values between our method with MPNN-DQN [15] under different network scale (20 nodes and 50 nodes). ... The comparison of results shows that the MPNN-DQN method demonstrates oscillatory behavior, while our method achieves stable convergence to a higher reward, effectively reducing the impact of attacks."

    The Avg.R plotted in Fig. 6 is exactly the training reward R defined in Section IV.A, including the security weights w that the DQN is trained to minimize. The baseline MPNN-DQN is explicitly described as considering 'only communication cost without accounting for attacks,' so it is evaluated on a metric containing a penalty term it never optimized. Higher Avg.R for the proposed method is therefore a consequence of optimizing that same objective and does not independently demonstrate that routes avoid attack nodes or improve resilience. No separate attack-avoidance metric, no detection accuracy for the GCN supplying w, and no ablation are reported; the 'superiority' claim reduces to training on R and then re-measuring R.

full rationale

The paper is a case study rather than a formal derivation, so its central claim rests on the DQN optimizing a hand-designed reward. The main circularity is in the evaluation: the reported Avg.R is identical to the agent's training reward, including the security weights w, while the baseline was trained without the w-term. As a result, the observed superiority is partly built into the metric. This is compounded by the unspecified derivation of w from the attack traffic ratio and the absence of detection accuracy or direct attack-avoidance metrics. The self-citations in the related-work table are background and not load-bearing; the GCN design follows external TFE-GNN [2], not a same-author uniqueness theorem. The DRL convergence and time-consumption measurements retain some independent content, so the paper is not wholly circular, but the central resilience and superiority claims are not independently supported.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central framework relies on graph modeling of networks, transferability of GCN traffic classification, DRL as a routing optimizer, and a paper-specific reward definition. The main free parameters are the reward-shaping factors and the mapping from detection scores to security weights.

free parameters (3)
  • alpha (path overlap penalty factor) = not reported
    Used in the reward function R to trade off path overlap against cost and security. Its value is not given, making the reward definition incomplete.
  • per-node security weight w = integer in [1,200]
    Assigned to each node based on attack traffic ratio, but the mapping from GCN detection output to w is not specified. This is a hand-designed transformation central to the reward.
  • per-node communication cost c = integer in [1,200]
    Assigned to each node as an input scenario setting. The specific values are not provided, though they affect the reward and the learned policy.
assumptions (4)
  • domain assumption Communication networks can be modeled as undirected graphs with nodes as devices and edges as connections.
    Section III.A.1 assumes static graph representation, which may not capture dynamic or heterogeneous network behavior.
  • domain assumption A two-layer GCN following TFE-GNN can accurately extract attack features from encrypted IoT traffic.
    Section IV.A relies on transferability of TFE-GNN's architecture without reporting detection performance on the reconstructed CICIoT2022 data.
  • domain assumption DRL with GNN-extracted features can generate optimal routing policies.
    The framework assumes that the GNN-DQN agent converges to a policy that maximizes the designed reward, which is standard in RL but not guaranteed.
  • ad hoc to paper The reward function R is a valid proxy for network resilience.
    The reward only includes communication cost, path overlap, and security weights; it does not consider throughput, latency, or actual attack impact, so resilience is defined paper-specifically.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Achieving Network Resilience through Graph Neural Network-enabled Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/BLGJU4QR

@misc{pith2026250111074,
  author       = {Pith},
  title        = {Pith review of: Achieving Network Resilience through Graph Neural Network-enabled Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BLGJU4QR}},
  note         = {Machine review of arXiv:2501.11074}
}
read the original abstract

Deep reinforcement learning (DRL) has been widely used in many important tasks of communication networks. In order to improve the perception ability of DRL on the network, some studies have combined graph neural networks (GNNs) with DRL, which use the GNNs to extract unstructured features of the network. However, as networks continue to evolve and become increasingly complex, existing GNN-DRL methods still face challenges in terms of scalability and robustness. Moreover, these methods are inadequate for addressing network security issues. From the perspective of security and robustness, this paper explores the solution of combining GNNs with DRL to build a resilient network. This article starts with a brief tutorial of GNNs and DRL, and introduces their existing applications in networks. Furthermore, we introduce the network security methods that can be strengthened by GNN-DRL approaches. Then, we designed a framework based on GNN-DRL to defend against attacks and enhance network resilience. Additionally, we conduct a case study using an encrypted traffic dataset collected from real IoT environments, and the results demonstrated the effectiveness and superiority of our framework. Finally, we highlight key open challenges and opportunities for enhancing network resilience with GNN-DRL.

Figures

Figures reproduced from arXiv: 2501.11074 by the authors.

Figure 1
Figure 1. Comparison with existing researches on GNN and DRL in communication networks. We investigate the related works and select articles from the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Analysis of different combinations of GNNs and DRL in communication networks. We selected representative models from GNNs (GCN, GraphSAGE [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The schematic of building a resilient network based on GNN-DRL. Existing GNN-DRL methods for networks are vulnerable to diverse attacks. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The framework of GNN-DRL for building secure and resilient networks. The framework comprises three phases: modeling, decision-making, and [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: GNN-DRL for building a resilient and secure network. We utilize network topology and traffic as inputs, modeling encrypted traffic within the network [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The comparison of reward values between our method with MPNN-DQN [15] under different network scale (20 nodes and 50 nodes). All results were [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 15 canonical work pages

  1. [2]

    Tfe-gnn: A temporal fusion encoder using graph neural networks for fine-grained encrypted traffic classification,

    H. Zhang, L. Yu, X. Xiao, Q. Li, F. Mercaldo, X. Luo, and Q. Liu, “Tfe-gnn: A temporal fusion encoder using graph neural networks for fine-grained encrypted traffic classification,” in Proceedings of the ACM Web Conference, 2023, pp. 2066–2075

    work page 2023

  2. [1]

    G-routing: Graph neural networks-based flexible online routing,

    H. Wei, Y . Zhao, and K. Xu, “G-routing: Graph neural networks-based flexible online routing,” IEEE Network, vol. 37, no. 4, pp. 90–96, 2023

    work page 2023

  3. [3]

    Permutation equivari- ant graph framelets for heterophilous graph learning,

    J. Li, R. Zheng, H. Feng, M. Li, and X. Zhuang, “Permutation equivari- ant graph framelets for heterophilous graph learning,”IEEE Transactions on Neural Networks and Learning Systems , 2024

    work page 2024

  4. [4]

    Graph neural networks for communication networks: Context, use cases and opportunities,

    J. Su ´arez-Varela, P. Almasan, M. Ferriol-Galm ´es, K. Rusek, F. Geyer, X. Cheng, X. Shi, S. Xiao, F. Scarselli, A. Cabellos-Aparicio, and P. Barlet-Ros, “Graph neural networks for communication networks: Context, use cases and opportunities,” IEEE Network , vol. 37, no. 3, pp. 146–153, 2023

    work page 2023

  5. [5]

    Deep graph reinforcement learning for mobile edge computing: Challenges and solutions,

    Y . Wang, H. Wu, and R. Li, “Deep graph reinforcement learning for mobile edge computing: Challenges and solutions,” IEEE Network , vol. 38, no. 5, pp. 314–323, 2024

    work page 2024

  6. [6]

    Graph neural network meets multi-agent reinforcement learning: Fundamentals, applications, and future directions,

    Z. Liu, J. Zhang, E. Shi, Z. Liu, D. Niyato, B. Ai, and X. S. Shen, “Graph neural network meets multi-agent reinforcement learning: Fundamentals, applications, and future directions,” IEEE Wireless Communications, pp. 1–9, 2024

    work page 2024

  7. [7]

    Challenges and opportunities in deep reinforcement learning with graph neural networks: A comprehensive review of algorithms and applications,

    S. Munikoti, D. Agarwal, L. Das, M. Halappanavar, and B. Natarajan, “Challenges and opportunities in deep reinforcement learning with graph neural networks: A comprehensive review of algorithms and applications,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–21, 2023

    work page 2023

  8. [8]

    Dealing with changes: Resilient routing via graph neural networks and multi-agent deep reinforcement learning,

    S. S. Bhavanasi, L. Pappone, and F. Esposito, “Dealing with changes: Resilient routing via graph neural networks and multi-agent deep reinforcement learning,” IEEE Transactions on Network and Service Management, vol. 20, no. 3, pp. 2283–2294, 2023

    work page 2023

Show all 15 references
  1. [9]

    Smarttro: Optimizing topology robustness for internet of things via deep reinforcement learn- ing with graph convolutional networks,

    Y . Peng, C. Liu, S. Liu, Y . Liu, and Y . Wu, “Smarttro: Optimizing topology robustness for internet of things via deep reinforcement learn- ing with graph convolutional networks,” Computer Networks, vol. 218, p. 109385, 2022

    2022

  2. [10]

    Virtualized network function forwarding graph placing in sdn and nfv- enabled iot networks: A graph neural network assisted deep reinforce- ment learning method,

    Y . Xie, L. Huang, Y . Kong, S. Wang, S. Xu, X. Wang, and J. Ren, “Virtualized network function forwarding graph placing in sdn and nfv- enabled iot networks: A graph neural network assisted deep reinforce- ment learning method,” IEEE Transactions on Network and Service Manage...

    2022

  3. [11]

    Graph attention network-based multi-agent reinforcement learning for slicing resource management in dense cellular network,

    Y . Shao, R. Li, B. Hu, Y . Wu, Z. Zhao, and H. Zhang, “Graph attention network-based multi-agent reinforcement learning for slicing resource management in dense cellular network,” IEEE Transactions on Vehicular Technology, vol. 70, no. 10, pp. 10 792–10 803, 2021

    2021

  4. [12]

    Deep-graph-based reinforcement learning for joint cruise control and task offloading for aerial edge internet of things (edgeiot),

    K. Li, W. Ni, X. Yuan, A. Noor, and A. Jamalipour, “Deep-graph-based reinforcement learning for joint cruise control and task offloading for aerial edge internet of things (edgeiot),”IEEE Internet of Things Journal, vol. 9, no. 21, pp. 21 676–21 686, 11 2022

    2022

  5. [13]

    Gtxchain: A secure iot smart blockchain architecture based on graph neural network,

    J. Cai, W. Liang, X. Li, K. Li, Z. Gui, and M. K. Khan, “Gtxchain: A secure iot smart blockchain architecture based on graph neural network,” IEEE Internet of Things Journal , vol. 10, no. 24, pp. 21 502–21 514, 2023

    2023

  6. [14]

    Towards the development of a realistic multi- dimensional iot profiling dataset,

    S. Dadkhah, H. Mahdikhani, P. K. Danso, A. Zohourian, K. A. Truong, and A. A. Ghorbani, “Towards the development of a realistic multi- dimensional iot profiling dataset,” in 2022 19th Annual International Conference on Privacy, Security & Trust (PST) , 2022, pp. 1–11

    2022

  7. [15]

    Deep reinforcement learning meets graph neural networks: Exploring a routing optimization use case,

    P. Almasan, J. Su ´arez-Varela, K. Rusek, P. Barlet-Ros, and A. Cabellos- Aparicio, “Deep reinforcement learning meets graph neural networks: Exploring a routing optimization use case,” Computer Communications, vol. 196, pp. 184–194, 2022

    2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.