arxiv: 2603.22511 · v3 · submitted 2026-03-23 · 💻 cs.CR

Recognition: no theorem link

CTF as a Service: A reproducible and scalable infrastructure for cybersecurity training

Carlos Jimeno Miguel , Mikel Izal

Authors on Pith no claims yet

Pith reviewed 2026-05-15 00:06 UTC · model grok-4.3

classification 💻 cs.CR

keywords CTFcybersecurity traininginfrastructure as codeCI/CD pipelineDocker Swarmvirtualizationeducation

0 comments

The pith

A CTF as a Service platform automates challenge deployment from Git repositories using infrastructure as code for cybersecurity training

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Capture The Flag events provide hands-on cybersecurity experience but demand substantial infrastructure setup that limits their use in schools. This paper introduces a platform that supports automatic deployment of challenges directly from a Git repository via CI/CD pipelines alongside manual ad-hoc provisioning options. The implementation relies on Proxmox for virtualization, Terraform and Ansible for configuration, Docker Swarm for orchestration, and HAProxy for load balancing to ensure persistent sessions and consistent challenge instances. A sympathetic reader would see this as a way to make realistic security training more practical and repeatable in educational environments.

Core claim

The CaaS platform enables reproducible CTF hosting by integrating a development workflow that pulls challenges from Git and deploys them automatically through CI/CD with a deployment workflow for on-demand infrastructure, while solving issues of session persistence, external routing, and replicability through the chosen virtualization and container stack.

What carries the argument

Dual-workflow automation system that uses Infrastructure as Code to provision and orchestrate virtualized cybersecurity challenge environments.

If this is right

Universities can host CTF competitions with reduced manual infrastructure effort.
Challenges maintain consistency and replicability across different deployment instances.
Session persistence allows users to continue work without interruption in training scenarios.
The platform is ready to expand into a commercial CTF hosting service.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Similar automation could support training in related fields like network administration or software security.
Standardizing the frontend could make the system more user-friendly for educators without technical expertise.
Integration with monitoring tools would help identify performance bottlenecks in real events.

Load-bearing premise

The combination of Proxmox, Terraform, Ansible, Docker Swarm, and HAProxy will achieve session persistence, external routing, and challenge replicability at scale without significant additional custom development.

What would settle it

Running a CTF competition with a large number of participants and checking if all users maintain stable connections and access to identical challenge instances without errors or manual fixes would confirm or refute the platform's capabilities.

Figures

Figures reproduced from arXiv: 2603.22511 by Carlos Jimeno Miguel, Mikel Izal.

**Figure 2.** Figure 2: CI/CD pipeline flow. A post-receive hook in the repository builds a compressed artifact with each push made by the developer and stores it in the artifacts branch. A pipeline.sh script, executed periodically as a cron job on each backend, checks for new artifacts, compares them with deployed versions, and updates the corresponding Swarm services, reporting the result to the repository via a status file… view at source ↗

read the original abstract

Capture The Flag (CTF) competitions have established themselves as a highly effective pedagogical tool in cybersecurity education, offering students hands-on experience in realistic attack and defense scenarios. However, organizing and hosting these events requires considerable infrastructure effort, which frequently limits their adoption in academic settings. This paper presents the design, iterative development, and evaluation of a CTF as a Service (CaaS) platform built on Proxmox virtualization, leveraging Infrastructure as Code (IaC) tools such as Terraform and Ansible, container orchestration via Docker Swarm, and load balancing with HAProxy. The system supports both a development-centered workflow, in which challenges are automatically deployed from a Git repository through a CI/CD pipeline, and a deployment-oriented workflow for ad-hoc infrastructure provisioning. The paper describes the design decisions made, the challenges encountered during development, and the solutions implemented to achieve session persistence, external routing, and challenge replicability. The platform is designed to evolve into a CTF hosting service with commercial potential, and future lines of work are outlined regarding automatic scaling, monitoring integration, and frontend standardization.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper gives a practical blueprint for automating CTF infrastructure with standard IaC tools and two clear workflows, but offers no measurements to show the setup actually scales or stays reliable under load.

read the letter

Hi, the main point is that this describes a CTF-as-a-Service platform built on Proxmox, Terraform, Ansible, Docker Swarm, and HAProxy. It supports a Git-driven CI/CD path for automatic challenge deployment and a separate ad-hoc workflow for quick provisioning. That setup directly tackles the infrastructure overhead that keeps many academic groups from running CTFs often. They walk through the concrete choices made for session persistence, external routing, and challenge replication, plus the problems that came up and how they were handled. This part feels useful because it shows real integration work rather than just listing tools. The description of the two workflows is clear enough that someone could try to replicate the pattern. The main gap is the total absence of any performance data. There are no numbers on concurrent sessions, provisioning times, failure rates under load, or how well replication holds when multiple teams run at once. The claims about scalability and reliability rest on the design narrative alone, so it is impossible to judge whether the stack works as stated without extra custom fixes. This paper is aimed at cybersecurity educators and organizers who want a reproducible way to host events without starting from scratch each time. A reader looking for implementation details on combining these tools for training will find the workflows and design notes worthwhile. It should go to peer review because the problem is practical and the approach is grounded, but the authors will need to add quantitative tests before the scalability claims can be taken as demonstrated.

Referee Report

2 major / 1 minor

Summary. The paper presents the design, iterative development, and evaluation of a CTF as a Service (CaaS) platform built on Proxmox virtualization with Infrastructure as Code tools (Terraform, Ansible), Docker Swarm for container orchestration, and HAProxy for load balancing. It supports a development-centered workflow with automated CI/CD deployment from Git repositories and a deployment-oriented workflow for ad-hoc provisioning, with design decisions aimed at achieving session persistence, external routing, and challenge replicability. The platform is positioned for potential evolution into a commercial CTF hosting service.

Significance. If the described infrastructure reliably meets its scalability and reproducibility goals, the work would provide a practical, open-source-based solution that reduces the infrastructure overhead of hosting CTF events in academic settings. The dual-workflow design and reliance on standard IaC tools represent a useful engineering contribution for reproducible cybersecurity training environments.

major comments (2)

[Evaluation / Results] The abstract states that the platform was evaluated and that solutions were implemented to achieve session persistence, external routing, and challenge replicability, yet no quantitative results (e.g., concurrent user/session counts, provisioning latency distributions, failure rates under load, or cross-challenge reproducibility metrics) are referenced. Without such data, the central claim that the Proxmox+Terraform+Ansible+Docker Swarm+HAProxy stack meets the requirements at scale cannot be assessed.
[System Architecture / Implementation] The description of the CI/CD and ad-hoc workflows assumes that the chosen stack delivers the required properties without substantial custom engineering, but no concrete measurements or failure-mode analysis under concurrent load are provided to support this assumption.

minor comments (1)

[Abstract] The abstract refers to 'evaluation' but the summary focuses exclusively on design decisions and challenges; clarify whether a dedicated results section with metrics exists in the full manuscript.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. The comments highlight the need for stronger empirical support for our claims, and we address each point below with proposed revisions.

read point-by-point responses

Referee: [Evaluation / Results] The abstract states that the platform was evaluated and that solutions were implemented to achieve session persistence, external routing, and challenge replicability, yet no quantitative results (e.g., concurrent user/session counts, provisioning latency distributions, failure rates under load, or cross-challenge reproducibility metrics) are referenced. Without such data, the central claim that the Proxmox+Terraform+Ansible+Docker Swarm+HAProxy stack meets the requirements at scale cannot be assessed.

Authors: We agree that the manuscript lacks quantitative metrics to fully substantiate scalability and reproducibility claims. The evaluation section currently describes the iterative development process and qualitative solutions implemented (e.g., HAProxy for session persistence and Terraform/Ansible for replicability). No performance benchmarks or load-test data were collected or reported. In the revised manuscript we will add a new evaluation subsection with available internal test results, including average provisioning times for sample challenges, observed concurrent session limits in our academic deployments, and reproducibility rates across repeated deployments. This will provide concrete data to support the central claims. revision: yes
Referee: [System Architecture / Implementation] The description of the CI/CD and ad-hoc workflows assumes that the chosen stack delivers the required properties without substantial custom engineering, but no concrete measurements or failure-mode analysis under concurrent load are provided to support this assumption.

Authors: The workflows are presented in Section 4 as relying primarily on standard IaC tooling with limited custom scripting. We acknowledge the absence of failure-mode analysis and load measurements. The revision will incorporate a dedicated analysis subsection drawing from our development logs, documenting encountered issues (such as Terraform state management and Docker Swarm networking) and their resolutions, together with any available deployment success metrics from testing. This will demonstrate that the stack meets the requirements with minimal custom engineering. revision: yes

Circularity Check

0 steps flagged

No circularity; claims rest on standard IaC tooling without self-referential reduction

full rationale

The manuscript describes an infrastructure platform assembled from established open-source components (Proxmox, Terraform, Ansible, Docker Swarm, HAProxy) and two workflows (CI/CD from Git and ad-hoc provisioning). No equations, fitted parameters, predictions, or self-citations appear that would reduce any claimed property (session persistence, replicability, scalability) to a quantity defined by the authors' own prior work or by construction. The text presents design decisions and implemented solutions as engineering choices rather than derivations that loop back to inputs. This is a self-contained systems description whose central assertions remain externally falsifiable via deployment measurements.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The platform description rests on the assumption that standard virtualization and orchestration tools can be composed to meet CTF requirements. No new mathematical entities, fitted parameters, or ad-hoc axioms are introduced in the abstract.

pith-pipeline@v0.9.0 · 5485 in / 1048 out tokens · 30981 ms · 2026-05-15T00:06:44.336804+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

15 extracted references · 15 canonical work pages

[1]

Git-based ctf: A simple and effective approach to organizing in-course attack-and-defense security competition,

S. Wi, J. Choi, and S. K. Cha, “Git-based ctf: A simple and effective approach to organizing in-course attack-and-defense security competition,” in2018 USENIX Workshop on Advances in Security Education (ASE 18). Baltimore, MD, USA: USENIX Association, Aug. 2018. [Online]. Available: https://www.usenix.org/conference/ ase18/presentation/wi

work page 2018
[2]

Ctf: State-of-the-art and building the next generation,

G. M. Taylor and A. Arias, “Ctf: State-of-the-art and building the next generation,” Semantic Scholar, 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:267660094

work page 2024
[3]

Ctfd: The easiest capture the flag platform,

CTFd Development Team, “Ctfd: The easiest capture the flag platform,”

work page
[4]

Available: https://ctfd.io/

[Online]. Available: https://ctfd.io/

work page
[5]

Fbctf: Facebook capture the flag,

Facebook, “Fbctf: Facebook capture the flag,” GitHub repository. [Online]. Available: https://github.com/facebook/fbctf

work page
[6]

An analysis and evaluation of open source capture the flag platforms as cybersecurity e- learning tools,

S. Karagiannis, E. Maragkos-Belmpas, and E. Magkos, “An analysis and evaluation of open source capture the flag platforms as cybersecurity e- learning tools,” inInformation Security Education. Information Security in Action. Cham: Springer International Publishing, 2020, pp. 61–77

work page 2020
[7]

An empirical survey of functions and configurations of open-source capture the flag (ctf) environments,

S. Kucek and M. Leitner, “An empirical survey of functions and configurations of open-source capture the flag (ctf) environments,” Journal of Network and Computer Applications, vol. 157, p. 102419, 2020

work page 2020
[8]

Kypo cyber range: Design and use cases,

J. Vykopal, R. O ˇslejˇsek, P. ˇCeleda, M. Vi ˇzv´ary, and D. Tov ´arn´ak, “Kypo cyber range: Design and use cases,” inProceedings of the 12th International Conference on Software Technologies (ICSOFT 2017), 2017, pp. 310–321

work page 2017
[9]

Hack The Box — hacking training for the best,

Hack The Box Ltd., “Hack The Box — hacking training for the best,” https://www.hackthebox.com, 2025, consultado en 2025

work page 2025
[10]

TryHackMe — learn cyber security,

TryHackMe Ltd., “TryHackMe — learn cyber security,” https:// tryhackme.com, 2025, consultado en 2025

work page 2025
[11]

Proxmox virtual environment

Proxmox Server Solutions GmbH, “Proxmox virtual environment.” [Online]. Available: https://www.proxmox.com/

work page
[12]

Terraform: Infrastructure as code

HashiCorp, “Terraform: Infrastructure as code.” [Online]. Available: https://www.terraform.io/

work page
[13]

Ansible: Automation for everyone

Red Hat, “Ansible: Automation for everyone.” [Online]. Available: https://www.ansible.com/

work page
[14]

Docker swarm mode overview

Docker Inc., “Docker swarm mode overview.” [Online]. Available: https://docs.docker.com/engine/swarm/

work page
[15]

Haproxy: The reliable, high performance tcp/http load balancer

HAProxy Technologies, “Haproxy: The reliable, high performance tcp/http load balancer.” [Online]. Available: https://www.haproxy.org/

work page