Pith. sign in

REVIEW 4 major objections 7 minor 34 references

open5Gcube: A Modular and Usable Framework for Mobile Network Laboratories

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read open5Gcube packages open-source 2G, 4G, and 5G network components into container stacks that a single controller host can build, tear down, and rebuild, and reports which commercial phones and modems actually connect.

desk verdict open5Gcube is a useful, honest systems paper: the repository is a real contribution, but the version-flexibility and reproducibility claims are asserted rather than demonstrated, and the UE sample is small. read the letter →

arxiv 2505.14501 v1 pith:WMWQJX5W submitted 2025-05-20 cs.NI

Thorsten Horstmann , Dominik Brunke , Tobias Kremeyer , Matthias Wilmes , Gunnar Schneider , Julian Sturm , Hartmut König , Michael Rademacher This is my paper · ORCID
classification cs.NI
keywords open5Gcubemobilenetworklaboratoryopen-sourceintegrationCOTSUEcompatibilitycontainerizedstacksexperimentalreproducibility2G/4G/5Gtestbedunifiedorchestration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces open5Gcube, a framework for building small mobile-network laboratories from open-source components. The paper claims that containerization plus a lightweight orchestration layer makes it practical for a research group to combine different radio, core, and subscriber-management implementations, including different software versions, and to switch between complete network scenarios quickly and reproducibly. The authors demonstrate the framework by testing seven commercial off-the-shelf devices (six smartphones and one modem) against a 2G stack, a 4G stack with VoLTE, and several 5G standalone stacks built from different core and radio projects. Basic data connectivity worked broadly, while VoLTE registration and SUCI-based authentication worked only for selected devices. If the claim is right, other laboratories can reproduce these setups from the public repository rather than re-integrating the software by hand.

What carries the argument

The stack abstraction is the object that carries the argument: a complete network scenario condensed into declarative container-composition files, centrally managed environment variables, and configuration templates. Each stack pins fixed IP addresses on dedicated virtual networks so configurable services can rely on static addresses, and a lightweight script-based orchestrator on the controller host starts remote radio containers by having an already-launched container detect the wrong host, disconnect from the stack network, and re-launch the service on the correct host via SSH. Subscriber databases are repopulated from scratch at every start, which makes experiments deterministic. Together these pieces make the framework's central promise—quick switching between network implementations and versions with reproducible state—concrete.

What would settle it

Run the published stacks against a deliberately diverse device set—older phones, mid-range Android devices with MediaTek or older Exynos basebands, and iPhones with SUPI concealment disabled—and record basic data-connectivity success for 2G, 4G, and each 5G stack; a substantial failure rate where the seven tested devices succeeded would refute the 'adequate compatibility' conclusion.

Watch

Extended reading notes

Core claim

The paper's central claim is that the main obstacle to using open-source mobile networks in research is orchestration, not the network software itself, and that a stack-based container abstraction removes that obstacle. In open5Gcube, every complete scenario—core network, radio access, subscriber database, and fixed IP assignments—is packaged as a stack that can be started and stopped from one controller host; radio containers that start on the wrong host detect this, release their network assignments, and relaunch themselves on the designated radio host over SSH. The authors report that all seven devices achieved basic 2G and 4G data connectivity, three registered to the VoLTE/IMS network, and 5G standalone success varied by combination: most devices worked with all core networks, one phone failed to detect the cell broadcast by one of the two radio projects, one dropped its connection without IMS, and one required SUPI concealment that one core does not support. This device-dependent behavior is presented as the kind of experimentally relevant result the framework is built to make easy to obtain.

Load-bearing premise

The conclusion that UE compatibility is adequate for mobile network laboratories rests on seven specific commercial devices, most of them recent high-end phones; if other common devices with different chipsets or Android versions connect far less reliably, the general claim weakens.

Editorial extensions

If this is right

  • A laboratory with the documented hardware should be able to deploy the published 2G, 4G, and 5G stacks from the public repository and observe similar basic-data connectivity across commercial devices.
  • Switching between core-network implementations or radio versions becomes a stack-selection operation instead of a re-integration project, so comparative experiments need much less setup time.
  • Because subscriber databases are rebuilt from scratch on startup and IP assignments are fixed, repeated experiments start from deterministic state rather than from whatever the previous run left behind.
  • Advanced-feature experiments now come with a documented device guide: VoLTE registration worked on only three of seven devices, and one core network lacks SUCI-based authentication, so security and IMS studies can pick equipment accordingly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next step would be to turn the seven-device compatibility table into a community-maintained, machine-readable matrix, since phone models and Android versions change quickly and the current snapshot will age.
  • The framework's ability to hold the radio and device fixed while swapping core networks also enables controlled A/B tests of software-version changes, a use the paper mentions but does not exercise in its evaluation.
  • The pattern of failures—one phone not seeing a radio's cell, one disconnecting without IMS, one blocked by an unsupported authentication mode—suggests device-side behavior, not the open-source core, is often the binding constraint, which could be tested by varying carrier settings and SIM configuration while keeping the stack fixed.
  • If other groups adopt the stacks, the same repository could grow into a standardized benchmark for COTS device compatibility across open-source mobile networks, giving the field a shared reference point instead of private testbeds.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper presents open5Gcube, a container-based framework for integrating open-source mobile network components (RAN, CN, and UE-side tools) into a unified laboratory environment. The framework uses Docker-Compose stacks, a script-based orchestration layer, centralized configuration templates, and a Web-UI to manage stacks across multiple hosts. The authors evaluate the framework by building 2G (Osmocom), 4G VoLTE (srsRAN/Open5GS/Kamailio), and 5G SA (srsRAN/OAI-RAN with Open5GS/OAI-Core/Free5GC) stacks and testing them with seven COTS UEs (five smartphones, one modem, and one software-based UE not actually used in the tests). The paper claims that the framework allows flexible combination of different open-source implementations, including different versions, and that the compatibility results demonstrate its versatility and reproducibility.

Significance. If the claimed version-flexibility and reproducibility properties hold, open5Gcube would address a real gap in the literature: most prior testbed papers provide neither unified management nor publicly available configurations. The paper provides a useful architecture description, and the UE compatibility observations are direct empirical data that other groups could reproduce from the released repository. The framework also deliberately avoids heavyweight orchestration tools like Kubernetes, which is a reasonable design choice for small-scale labs. However, the significance is currently limited by the absence of evidence for the two central differentiators claimed in the abstract: version switching and simplified reproducible setup. The evaluation demonstrates only fixed version combinations and a small, non-representative UE sample, so the broader claims about versatility and reproducibility are not yet supported.

major comments (4)
  1. [Abstract; Section IV, Table IV] The central claim that open5Gcube 'allows researchers to flexibly combine different open-source implementations, including different versions' is not exercised anywhere in the evaluation. Section IV uses exactly one version per project per stack (Table IV), and no experiment changes the version of a RAN or CN component. The configuration-template mechanism described in Section III-B is precisely where version flexibility would be demonstrated, so the absence of a version-switch test leaves the primary differentiator unvalidated. A single failed version change would undercut this advertised feature; this should either be demonstrated or removed from the abstract.
  2. [Section IV, final paragraph; Section V] The conclusion that the results 'underline the versatility and reproducibility' is an overstatement. Versatility is shown only for the fixed combinations in Table IV, not for version flexibility. Reproducibility is asserted from the existence of the repository in [34], but the paper provides no clean-build check, no pinned commit hashes or release tags for the framework or the integrated projects, no explicit base-image versions, and no independent replication. Given that Section III-A names a specific Ubuntu version and kernel (5.15.0-133), a reproducible setup requires more than a GitHub link. The reproducibility claim should be either supported with precise version pinning and a build verification, or explicitly scoped to 'the provided configuration templates and documented hardware'.
  3. [Section IV, Table V and conclusion] The statement that 'the compatibility of the tested UEs is adequate for mobile network laboratories' is a sample-specific judgment, not a general result. The sample consists of seven devices, five of them 2024 flagship smartphones with Qualcomm Snapdragon modems (Table II), and the authors themselves note known problem cases: the Motorola Edge 50 Pro disconnects due to missing IMS, the Google Pixel 9 fails to detect srsRAN, and the iPhone requires SUCI concealment settings. No older, low-end, or non-flagship devices are included, and no justification is given that this sample represents the space of COTS UEs. This conclusion should be restricted to the tested devices or supported by a broader, more diverse sample.
  4. [Section IV (overall); Section I] The framework's usability claim — that it 'simplifies experimental setups through containerization and lightweight orchestration' — is not quantitatively evaluated. There is no measurement of setup time, number of commands, or comparison with a manual deployment of the same stacks without open5Gcube. The orchestration design in Section III-B is plausible, but the paper provides no empirical evidence that the framework reduces effort. Since 'usable' appears in the title, a baseline comparison or at least a qualitative expert assessment of effort reduction would be needed to substantiate this core claim.
minor comments (7)
  1. [Table I] The symbols used in Table I (#, G, blank) are not explained. Please add a legend describing what each symbol and each column means, and complete the final row labeled 'This 2025', which currently contains only the label and empty entries.
  2. [Tables III and V] Tables III and V use checkmarks, circles, dashes, 'G', and blank cells without a legend. The reader cannot unambiguously determine whether 'G' means 'works with restrictions' or something else; please add explicit legends.
  3. [Section IV-A] The text says 'We were able to connect all smartphones shown in Table II', but Table II also includes a Quectel modem (QRM500), which is not a smartphone. Please clarify that QRM500 is a modem and does not affect the smartphone count.
  4. [Section III-A] The hardware description gives qualitative statements about mobility and portability ('fitting into a standard car', 'easily transported') without any quantitative details such as total weight or dimensions. Adding these would strengthen the reproducibility of the hardware setup.
  5. [Section IV-C] The stress test is described as 'a flood-ping was performed for two minutes without any connection loss', but no parameters (packet size, interval, number of parallel pings, or definition of connection loss) are given. Please specify the exact test procedure.
  6. [References] Reference [34] points to a GitHub repository without a commit hash, branch, or release tag. If the paper appears in a camera-ready venue, please cite a specific release or commit to make the artifact auditable.
  7. [Throughout] There are minor typographical issues: 'V oLTE' appears with a spurious space in several places, 'User Equipments' should be lowercase, and the index terms are overly generic ('5G, Testbed, Open-Source'). These should be cleaned up.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: compatibility results are direct empirical observations and the only self-citation is peripheral.

full rationale

This paper makes no derived predictions and fits no parameters. The UE compatibility results in Tables III and V are direct observations of whether phones attach, register, and sustain data sessions; the conclusion that compatibility is 'adequate' is a qualitative judgment, not an output forced by construction. The architecture sections describe design choices (Docker, macvlan, script-based orchestration) without invoking any uniqueness theorem or fitting data. Reference [1] is a prior paper by overlapping authors, but it is cited only as one item among related testbed work in Section II and is not load-bearing: none of the framework's claimed functionality or the compatibility results depends on it. The claim about project versions is asserted from experience ('we found that it was particularly helpful to switch quickly between stacks, to always have a consistent and reproducible setup and to simply change the versions of projects used') rather than demonstrated by a dedicated experiment; this weakens evidential support but is an absence-of-evidence concern, not circularity, because nothing in the paper defines a result in terms of the framework's own outputs.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters are fit to data because the paper makes no quantitative claims requiring fitting. The framework introduces no new theoretical entities; it is an engineering integration. The main trust assumptions are representativeness of the UE sample, acceptable container overhead, and reproducibility of upstream software versions.

assumptions (3)
  • domain assumption The seven tested UEs are a representative sample of COTS UEs for the purpose of the compatibility conclusions.
    Section IV extrapolates from individual pass/fail results to 'broad interoperability' and 'adequate compatibility' without a larger sample or statistical treatment.
  • domain assumption Docker containerization and macvlan networking preserve the real-time behavior needed for SDR-based RAN operation.
    Section III-B introduces containers and macvlan networks as the basis for the stacks; no overhead or jitter measurements are reported, so real-time suitability is assumed from the successful test runs.
  • domain assumption The tested open-source project versions remain stable and reproducible over time.
    Section IV pins versions (e.g., srsRAN 24.10.1, Open5GS 2.7.2, Free5GC 3.4.4) but no commit hashes are given, so exact reproduction depends on upstream availability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of open5Gcube: A Modular and Usable Framework for Mobile Network Laboratories." pith.science (2026). https://pith.science/paper/WMWQJX5W

@misc{pith2026250514501,
  author       = {Pith},
  title        = {Pith review of: open5Gcube: A Modular and Usable Framework for Mobile Network Laboratories},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WMWQJX5W}},
  note         = {Machine review of arXiv:2505.14501}
}
read the original abstract

In mobile network research, the integration of real-world components such as User Equipment (UE) with open-source network infrastructure is essential yet challenging. To address these issues, we introduce open5Gcube, a modular framework designed to integrate popular open-source mobile network projects into a unified management environment. Our publicly available framework allows researchers to flexibly combine different open-source implementations, including different versions, and simplifies experimental setups through containerization and lightweight orchestration. We demonstrate the practical usability of open5Gcube by evaluating its compatibility with various commercial off-the-shelf (COTS) smartphones and modems across multiple mobile generations (2G, 4G, and 5G). The results underline the versatility and reproducibility of our approach, significantly advancing the accessibility of rigorous experimentation in mobile network laboratories.

Figures

Figures reproduced from arXiv: 2505.14501 by the authors.

Figure 1
Figure 1. Representation of our open5Gcube. On the left side, the components in the 19” rack. On the right side, the corresponding [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Logical software organization of the framework. Open [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages

  1. [34]

    https://github.com/open5Gcube/open5Gcube

    work page

  2. [1]

    Implementation of OpenAPI Wireshark Dissectors to Validate SBI Messages of 5G Core Networks,

    L. Schauer, T. Horstmann, S. Druesedow et al. , “Implementation of OpenAPI Wireshark Dissectors to Validate SBI Messages of 5G Core Networks,” in ITG-Fb. 316: Mobilkommunikation – Technologien und Anwendungen. VDE Verlag, Offenbach, 2024

    work page 2024

  3. [2]

    5G DIY: Impact of Different Elements on the Performance of an E2E 5G Standalone Testbed,

    M. Amini, A. El-Ashmawy, C. Rosenberg et al. , “5G DIY: Impact of Different Elements on the Performance of an E2E 5G Standalone Testbed,” in GLOBECOM 2023 - 2023 IEEE Global Communications Conference. IEEE, Dec. 2023, p. 6377–6382

    work page 2023

  4. [3]

    A Comparative Analysis of Open-Source Software in an E2E 5G Standalone Platform,

    M. Amini and C. Rosenberg, “A Comparative Analysis of Open-Source Software in an E2E 5G Standalone Platform,” in 2024 IEEE Wireless Communications and Networking Conference , apr 2024, pp. 1–6

    work page 2024

  5. [4]

    A Versatile 5G Standalone Testbed Based On Commodity Hardware,

    E.-Z. G. Bozis, N. C. Sagias, M. C. Batistatos et al. , “A Versatile 5G Standalone Testbed Based On Commodity Hardware,” in 2024 Panhel- lenic Conference on Electronics & Telecommunications (PACET) . Thessaloniki, Greece: IEEE, mar 2024, pp. 1–4

    work page 2024

  6. [5]

    Enhancing 5G Performance: A Standalone System Platform With Customizable Features,

    ——, “Enhancing 5G Performance: A Standalone System Platform With Customizable Features,” AEU - International Journal of Electronics and Communications, vol. 187, p. 155515, 2024

    work page 2024

  7. [6]

    Evaluation of Open- Source Mobile Network Software Stacks: A Guide to Low-cost De- ployment of 5G Testbeds,

    M. Chepkoech, N. Mombeshora, B. Malila et al., “Evaluation of Open- Source Mobile Network Software Stacks: A Guide to Low-cost De- ployment of 5G Testbeds,” in 2023 18th Wireless On-Demand Network Systems and Services Conference (WONS) . Madonna di Campiglio, Italy: IEEE, jan 2023, pp. 56–63

    work page 2023

  8. [7]

    Performance Evalu- ation of an Open Source Implementation of a 5G Standalone Platform,

    J. E. H ˚akeg˚ard, H. Lundkvist, A. Rauniyar et al., “Performance Evalu- ation of an Open Source Implementation of a 5G Standalone Platform,” IEEE Access, vol. 12, pp. 25 809–25 819, 2024

    work page 2024

Show all 34 references
  1. [8]

    A Reference Deployment of a Minimal Open-Source Private Industry and Campus 5G Standalone (SA) System,

    F. John, J. Schuljak, L. B. V osteen et al., “A Reference Deployment of a Minimal Open-Source Private Industry and Campus 5G Standalone (SA) System,” in 2022 IEEE 10th International Conference on Information, Communication and Networks (ICICN) . Zhangye, China: IEEE, Aug. 2022...

    2022

  2. [9]

    Open Source 5G Core Network Implementations: A Qualitative and Quantitative Analysis,

    R. Reddy, M. Gundall, C. Lipps et al., “Open Source 5G Core Network Implementations: A Qualitative and Quantitative Analysis,” in 2023 IEEE International Black Sea Conference on Communications and Networking (BlackSeaCom). IEEE, jul 2023, pp. 253–258

    2023

  3. [10]

    Deploying a Stable 5G SA Testbed Using srsRAN and Open5GS: UE Integration and Troubleshoot- ing Towards Network Slicing,

    L. Mamushiane, A. Lysko, H. Kobo et al., “Deploying a Stable 5G SA Testbed Using srsRAN and Open5GS: UE Integration and Troubleshoot- ing Towards Network Slicing,” in 2023 International Conference on Artificial Intelligence, Big Data, Computing and Data Communication Systems (...

    2023

  4. [11]

    On the Performance of an Indoor Open-Source 5G Standalone Deployment,

    A. Sahbafard, R. Schmidt, F. Kaltenberger et al., “On the Performance of an Indoor Open-Source 5G Standalone Deployment,” in 2023 IEEE Wireless Communications and Networking Conference , mar 2023

    2023

  5. [12]

    How to Get Away with OpenAirInterface: A practical Guide to 5G RAN Configuration,

    M. Seidel, A. I. Grohmann, P. Sossalla et al., “How to Get Away with OpenAirInterface: A practical Guide to 5G RAN Configuration,” in 2023 3rd International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME) . IEEE, jul 2023, pp. 1–6

    2023

  6. [13]

    Easy to Deploy UA V-based Non-Public Open-Source 5G Network,

    P. Valente, A. Teixeira, M. Lu ´ıs et al. , “Easy to Deploy UA V-based Non-Public Open-Source 5G Network,” in 2024 15th International Conference on Network of the Future . IEEE, oct 2024, pp. 72–80

    2024

  7. [14]

    Implementa- tion of 5G Network Slicing Using Open Source Software,

    P. Vanichchanunt, O. Ritruechai, N. Wuttiananchai et al., “Implementa- tion of 5G Network Slicing Using Open Source Software,” in 2024 12th Int. Electrical Engineering Congress . IEEE, mar 2024, pp. 1–6

    2024

  8. [15]

    COTS UEs - srsRAN Project documentation,

    S. R. Systems, “COTS UEs - srsRAN Project documentation,” https://docs.srsran.com/projects/project/en/latest/knowledge base/ source/cots ues/source/index.html

  9. [16]

    Evaluating open-source 5g sa testbeds: Unveiling performance disparities in ran scenarios,

    M. Rouili, N. Saha, M. Golkarifard et al., “Evaluating open-source 5g sa testbeds: Unveiling performance disparities in ran scenarios,” in NOMS 2024-2024 IEEE Network Operations and Management Symposium . IEEE, May 2024, p. 1–6

    2024

  10. [17]

    Software Radio Systems, https://www.srsran.com/

  11. [18]

    OpenAirInterface Software Alliance, https://openairinterface.org/

  12. [19]

    Ali G ¨ung¨ar, https://github.com/aligungr/UERANSIM

  13. [20]

    https://github.com/osmocom/osmo-bts

  14. [21]

    A review of security attacks on the GSM standard,

    G. Cattaneo, G. De Maio, P. Faruolo et al., “A review of security attacks on the GSM standard,” in Information and Communication Technology- EurAsia Conference. Springer, 2013, pp. 507–512

    2013

  15. [22]

    Linux Foundation, https://free5gc.org/

  16. [23]

    Lee, https://open5gs.org/

    S. Lee, https://open5gs.org/

  17. [24]

    Precision GPS Reference Clock,

    Leo Bodnar, “Precision GPS Reference Clock,” https://www.leobodnar. com/

  18. [25]

    Amarisoft, https://www.amarisoft.com/test-and-measurement/ device-testing/device-products

  19. [26]

    https://docs.docker.com/network/drivers/macvlan/

  20. [27]

    Linux Foundation, https://kubernetes.io/

  21. [28]

    Docker Inc, https://docs.docker.com/engine/swarm/

  22. [29]

    Herle, https://github.com/herlesupreeth/docker open5gs

    S. Herle, https://github.com/herlesupreeth/docker open5gs

  23. [30]

    Kamailio SIP Server Project, https://www.kamailio.org/

  24. [31]

    Sipwise, https://github.com/sipwise/rtpengine

  25. [32]

    Background to the ’V oLTE Mess’,

    N. Jones, “Background to the ’V oLTE Mess’,” dec 2024. [Online]. Available: https://nickvsnetworking.com/background-to-the-volte-mess/

    2024

  26. [33]

    https://github.com/omnt/OpenMobileNetworkToolkit

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.