arxiv: 2604.04121 · v1 · submitted 2026-04-05 · 💻 cs.CR · cs.AI· cs.NI· cs.PF

Recognition: 1 theorem link

· Lean Theorem

NetSecBed: A Container-Native Testbed for Reproducible Cybersecurity Experimentation

Leonardo Bitzki , Diego Kreutz , Tiago Heinrich , Douglas Fideles , Leandro Bertholdo , Silvio Quincozes , Angelo Diniz

Authors on Pith no claims yet

Pith reviewed 2026-05-13 16:41 UTC · model grok-4.3

classification 💻 cs.CR cs.AIcs.NIcs.PF

keywords cybersecurity testbedreproducible experimentscontainer-nativenetwork traffic generationIoT securitydataset generationattack scenariosmulti-protocol environments

0 comments

The pith

NetSecBed is a container-native testbed that generates reproducible cybersecurity datasets by packaging attacks, services, and traffic generators as single-purpose containers.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper addresses the problem that most public cybersecurity datasets are static and difficult to re-execute or extend under controlled conditions, especially in heterogeneous IoT and multi-protocol networks. It presents NetSecBed as a framework that integrates 60 attack scenarios and 9 target services into declarative container modules, together with benign traffic generators. An automated pipeline then handles parametrized execution, packet capture, log collection, probing, feature extraction, and dataset consolidation. This design aims to deliver repeatable, auditable runs that reduce manual setup bias and enable continuous production of labeled evidence. The result is an extensible testbed suited for controlled experimentation where traceability matters.

Core claim

NetSecBed is a container-native, scenario-oriented testbed for reproducible generation of network traffic evidence and execution artifacts. It packages 60 attack scenarios, 9 target services, and benign traffic generators as single-purpose containers that plug into declarative specifications. The framework automates parametrized execution, full packet capture, log collection, service probing, feature extraction, and dataset consolidation, with particular support for IoT, IIoT, and multi-protocol environments. The central claim is that this architecture produces repeatable, auditable, and extensible cybersecurity experimentation while reducing operational bias.

What carries the argument

NetSecBed, the container-native testbed that orchestrates attack and service modules through declarative scenario specifications and an automated pipeline for execution, capture, and dataset assembly.

If this is right

Experiments become re-executable with identical parameters and full artifact traceability.
New attack or service modules can be added as containers without altering the core pipeline.
Continuous dataset generation becomes feasible as scenarios are extended or parametrized differently.
Operational bias in traffic generation and labeling is reduced through automation.
Support for heterogeneous multi-protocol settings such as IoT networks is provided by the modular container approach.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same declarative container approach could be applied to generate training data for machine-learning-based intrusion detection systems.
Version-controlled scenario files could enable collaborative sharing of exact experimental conditions across research groups.
Integration with orchestration tools might allow scaling the testbed to larger simulated networks while preserving reproducibility.
The framework could serve as a baseline for comparing container-based versus virtual-machine-based cybersecurity testbeds on fidelity metrics.

Load-bearing premise

Containerized execution of attacks and services faithfully reproduces real-world network timing and protocol behavior without introducing container-specific artifacts or isolation failures.

What would settle it

A direct comparison of inter-packet timing distributions and protocol state transitions for identical attack scenarios executed inside the container testbed versus on equivalent bare-metal hosts would reveal whether measurable differences exceed thresholds acceptable for the intended use cases.

Figures

Figures reproduced from arXiv: 2604.04121 by Angelo Diniz, Diego Kreutz, Douglas Fideles, Leandro Bertholdo, Leonardo Bitzki, Silvio Quincozes, Tiago Heinrich.

read the original abstract

Cybersecurity research increasingly depends on reproducible evidence, such as traffic traces, logs, and labeled datasets, yet most public datasets remain static and offer limited support for controlled re-execution and traceability, especially in heterogeneous multi-protocol environments. This paper presents NetSecBed, a container-native, scenario-oriented testbed for reproducible generation of network traffic evidence and execution artifacts under controlled conditions, particularly suitable for IoT, IIoT, and pervasive multi-protocol environments. The framework integrates 60 attack scenarios, 9 target services, and benign traffic generators as single-purpose containers, enabling plug-and-play extensibility and traceability through declarative specifications. Its pipeline automates parametrized execution, packet capture, log collection, service probing, feature extraction, and dataset consolidation. The main contribution is a repeatable, auditable, and extensible framework for cybersecurity experimentation that reduces operational bias and supports continuous dataset generation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

NetSecBed gives a practical container setup for generating fresh IoT cybersecurity datasets with 60 scenarios, but the abstract leaves the key claim about lower bias untested against container artifacts.

read the letter

The paper's core offering is a container-native testbed that packages 60 attack scenarios, 9 target services, and benign generators as single-purpose containers, plus a full pipeline for parametrized runs, packet capture, logging, probing, feature extraction, and dataset output. The declarative specs and plug-and-play design make it straightforward to extend or re-run experiments, which directly tackles the static-dataset problem in cybersecurity work on IoT and IIoT environments. That integration is the concrete new piece: not just another orchestration layer, but an end-to-end flow aimed at traceability and continuous generation. The architecture description is coherent and shows clear engineering thought about how to reduce manual setup errors. Credit is due for shipping a usable artifact rather than just another high-level proposal. The main limitation is the absence of any quantitative checks. No benchmarks compare container-generated traffic timing or packet behavior to bare-metal or physical-device runs, and there is no error analysis for isolation effects in multi-protocol settings. Container networking layers can introduce jitter or alter stack behavior, so the reduction in operational bias remains an assumption until those measurements appear. The stress-test concern about artifacts holds up on the provided text; nothing in the abstract falsifies it. This work is aimed at researchers who need repeatable, labeled traffic for tool validation or ML training in heterogeneous IoT settings. A reader already working on experiment automation or dataset pipelines would find the most immediate value. It deserves peer review because the framework is new, the design is grounded in existing container tools, and the reproducibility gap it targets is real, even though revisions will likely be needed for validation data and fidelity tests.

Referee Report

2 major / 1 minor

Summary. The paper presents NetSecBed, a container-native testbed for reproducible cybersecurity experimentation. It integrates 60 attack scenarios and 9 target services as single-purpose containers with benign traffic generators, using declarative specifications and an automated pipeline for parametrized execution, packet capture, log collection, service probing, feature extraction, and dataset consolidation. The central claim is that this yields a repeatable, auditable, and extensible framework that reduces operational bias in generating network traffic evidence for IoT/IIoT multi-protocol settings.

Significance. If validated, the work would be significant for enabling continuous, traceable dataset generation beyond static public datasets. The plug-and-play container integration and declarative pipeline represent practical strengths in extensibility and auditability for heterogeneous environments.

major comments (2)

[Abstract] Abstract and framework description: the claim that containerization reduces operational bias rests on the untested assumption that attacks/services inside containers faithfully reproduce real-world timing, packet behavior, and multi-protocol interactions; no comparative benchmarks against bare-metal or VM baselines are provided to substantiate this.
[Pipeline description] Pipeline and evaluation sections: no quantitative metrics, error analysis, or isolation-failure tests are reported for the 60 scenarios and 9 targets, leaving the reproducibility and bias-reduction claims without empirical grounding.

minor comments (1)

[Abstract] The abstract would benefit from a brief explicit statement of the container orchestration technology (e.g., Docker Compose or Kubernetes) used for the declarative specifications.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive review. We respond to each major comment below and outline the planned revisions to address the concerns raised.

read point-by-point responses

Referee: [Abstract] Abstract and framework description: the claim that containerization reduces operational bias rests on the untested assumption that attacks/services inside containers faithfully reproduce real-world timing, packet behavior, and multi-protocol interactions; no comparative benchmarks against bare-metal or VM baselines are provided to substantiate this.

Authors: The manuscript emphasizes that containerization, combined with declarative specifications, reduces operational bias by standardizing execution environments and minimizing human-induced variations in setup and configuration. We do not claim that containers perfectly replicate bare-metal timing or interactions, and we acknowledge the lack of direct comparative benchmarks. In the revised version, we will update the abstract and framework description to qualify the bias-reduction claim accordingly and include a limitations subsection noting the need for future fidelity studies. revision: partial
Referee: [Pipeline description] Pipeline and evaluation sections: no quantitative metrics, error analysis, or isolation-failure tests are reported for the 60 scenarios and 9 targets, leaving the reproducibility and bias-reduction claims without empirical grounding.

Authors: We agree that additional empirical data would strengthen the claims. The revised manuscript will expand the evaluation section with quantitative metrics, including execution success rates across the scenarios, resource utilization statistics, and preliminary error analysis. We will also report on basic isolation tests for a representative set of the 60 scenarios and 9 targets to provide grounding for the reproducibility assertions. revision: yes

Circularity Check

0 steps flagged

No circularity: design description integrates existing container technologies without derivations or self-referential reductions

full rationale

The paper presents NetSecBed as a container-native framework integrating 60 attack scenarios, 9 target services, and traffic generators via declarative specifications and an automated pipeline for execution, capture, and dataset generation. No equations, fitted parameters, predictions, or mathematical derivations appear in the provided text. The central claim rests on engineering integration of standard container and networking tools rather than any reduction to prior fitted quantities or self-citation chains. No load-bearing steps match the enumerated circularity patterns; the work is self-contained as a systems description.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The framework depends on standard assumptions about container isolation and network fidelity rather than introducing new fitted parameters or invented entities.

axioms (1)

domain assumption Container technology provides sufficient network isolation and timing fidelity for realistic attack and service execution
Invoked implicitly when claiming that containerized scenarios faithfully reproduce real-world conditions without new artifacts.

pith-pipeline@v0.9.0 · 5476 in / 1178 out tokens · 71665 ms · 2026-05-13T16:41:59.249506+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Foundation/RealityFromDistinction.lean reality_from_one_distinction unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

The main contribution is a repeatable, auditable, and extensible framework for cybersecurity experimentation that reduces operational bias and supports continuous dataset generation.

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

18 extracted references · 18 canonical work pages

[1]

Systematic literature review of cybersecurity testbeds for industrial internet of things

Taiwo Peter Akinremi, Joel Kwesi Appiah, Amir Reza Asadi, Opetunde Ibitoye, Hansinie Madushika Jayathilake, and Hazem Said. Systematic literature review of cybersecurity testbeds for industrial internet of things. In2025 1st International Conference on Secure IoT, Assured and Trusted Computing (SATC), pages 1–7. IEEE, 2025. 2https://github.com/ANONIMIZADO...

work page 2025
[2]

Testbeds and evaluation frameworks for anomaly detection within built environments: A systematic review.ACM Computing Surveys, 57(9):1–36, 2025

Mohammed Alosaimi, Omer Rana, and Charith Perera. Testbeds and evaluation frameworks for anomaly detection within built environments: A systematic review.ACM Computing Surveys, 57(9):1–36, 2025

work page 2025
[3]

Monitoring system reaction in cyber-physical testbed under cyber-attacks.Computers & Electrical Engineering, 59:86–98, 2017

Giuseppe Bernieri, Estefan ´ıa Etchev´es Miciolino, Federica Pascucci, and Roberto Setola. Monitoring system reaction in cyber-physical testbed under cyber-attacks.Computers & Electrical Engineering, 59:86–98, 2017

work page 2017
[4]

Gotham testbed: a reproducible iot testbed for security experiments and dataset generation.IEEE Transactions on Dependable and Secure Computing, 21(1):186–203, 2023

Xabier C ´amara, Jose Luis Flores, Crist´obal Arellano, Aitor Urbieta, and Urko Zurutuza. Gotham testbed: a reproducible iot testbed for security experiments and dataset generation.IEEE Transactions on Dependable and Secure Computing, 21(1):186–203, 2023

work page 2023
[5]

Cybersecurity testbeds for iot: A systematic literature review and taxonomy

Khalil G Queiroz de Santana, Marcos Schwarz, and Michelle Silva Wangham. Cybersecurity testbeds for iot: A systematic literature review and taxonomy. Journal of Internet Services and Applications, 15(1):450–473, 2024

work page 2024
[6]

Efficient detection of intrusions in ton-iot dataset using hybrid feature selection approach.Scientific Reports, 2026

N Dharini, VS Janani, and Jeevaa Katiravan. Efficient detection of intrusions in ton-iot dataset using hybrid feature selection approach.Scientific Reports, 2026

work page 2026
[7]

On the assessment of cyber risks and attack surfaces in a real-time co-simulation cybersecurity testbed for inverter-based microgrids.Energies, 14(16), 2021

Kirti Gupta, Subham Sahoo, Bijaya Ketan Panigrahi, Frede Blaabjerg, and Petar Popovski. On the assessment of cyber risks and attack surfaces in a real-time co-simulation cybersecurity testbed for inverter-based microgrids.Energies, 14(16), 2021

work page 2021
[8]

Lightweight testbed for cybersecurity experiments in scada-based systems

Mohsin Khan, Osama Rehman, Ibrahim MH Rahman, and Saqib Ali. Lightweight testbed for cybersecurity experiments in scada-based systems. In 2020 International Conference on Computing and Information Technology (ICCIT-1441), pages 1–5. IEEE, 2020

work page 2020
[9]

Testbed for lorawan security: Design and validation through man-in-the-middle attacks study.Applied Sciences, 11(16), 2021

Ondrej Pospisil, Radek Fujdiak, Konstantin Mikhaylov, Henri Ruotsalainen, and Jiri Misurec. Testbed for lorawan security: Design and validation through man-in-the-middle attacks study.Applied Sciences, 11(16), 2021

work page 2021
[10]

Next-generation cps testbed-based grid exercise - synthetic grid, attack, and defense modeling

Gelli Ravikumar, Burhan Hyder, and Manimaran Govindarasu. Next-generation cps testbed-based grid exercise - synthetic grid, attack, and defense modeling. pages 92–98, 10 2020

work page 2020
[11]

Sok: The mitre att&ck framework in research and practice.arXiv preprint arXiv:2304.07411, 2023

Shanto Roy, Emmanouil Panaousis, Cameron Noakes, Aron Laszka, Sakshyam Panda, and George Loukas. Sok: The mitre att&ck framework in research and practice.arXiv preprint arXiv:2304.07411, 2023

work page arXiv 2023
[12]

Scapy is a powerful interactive packet manipulation library written in python

Python Scapy. Scapy is a powerful interactive packet manipulation library written in python. scapy is able to forge or decode packets of a wide number of protocols, send them on the wire, capture them, match requests and replies, and much more., 2026

work page 2026
[13]

MohammadMoein Shafi, Arash Habibi Lashkari, and Arousha Haghighian Roudsari. Ntlflowlyzer: Towards generating an intrusion detection dataset and intruders behavior profiling through network and transport layers traffic analysis and pattern extraction.Computers & Security, 148:104160, 2025

work page 2025
[14]

Security testbed for internet-of-things devices.IEEE transactions on reliability, 68(1):23–44, 2018

Shachar Siboni, Vinay Sachidananda, Yair Meidan, Michael Bohadana, Yael Mathov, Suhas Bhairav, Asaf Shabtai, and Yuval Elovici. Security testbed for internet-of-things devices.IEEE transactions on reliability, 68(1):23–44, 2018

work page 2018
[15]

Security testbed for internet-of-things devices.IEEE Transactions on Reliability, 68(1):23–44, 2019

Shachar Siboni, Vinay Sachidananda, Yair Meidan, Michael Bohadana, Yael Mathov, Suhas Bhairav, Asaf Shabtai, and Yuval Elovici. Security testbed for internet-of-things devices.IEEE Transactions on Reliability, 68(1):23–44, 2019

work page 2019
[16]

Wireshark is the world’s foremost network protocol analyzer., 2026

Foundation Wireshark. Wireshark is the world’s foremost network protocol analyzer., 2026

work page 2026
[17]

Man-in-the-middle attacks and defence in a power system cyber-physical testbed.IET Cyber-Physical Systems: Theory & Applications, 6(3):164–177, 2021

Patrick Wlazlo, Abhijeet Sahu, Zeyu Mao, Hao Huang, Ana Goulart, Katherine Davis, and Saman Zonouz. Man-in-the-middle attacks and defence in a power system cyber-physical testbed.IET Cyber-Physical Systems: Theory & Applications, 6(3):164–177, 2021

work page 2021
[18]

Simulation analysis of syn flood and http flood attacks on cloud infrastructure integrity.International Journal of Research and Innovation in Social Science (IJRISS), 9(10), 2025

Caleb Chong Senn Yang, Low Choon Keat, Ng Yen Phing, and Phoon Gar Chi. Simulation analysis of syn flood and http flood attacks on cloud infrastructure integrity.International Journal of Research and Innovation in Social Science (IJRISS), 9(10), 2025

work page 2025