pith. sign in

arxiv: 2604.04498 · v1 · submitted 2026-04-06 · 💻 cs.DC · cs.PF

An experimental evaluation of satellite constellation emulators

Victor Cionca , Ferenc Szabo , Stanimir Vasilev , Dylan Smyth This is my paper

Pith reviewed 2026-05-10 19:58 UTC · model grok-4.3

classification 💻 cs.DC cs.PF
keywords satellite emulationconstellation emulatorsStarryNetOpenSNCelestialWetLinksperformance evaluationnetwork simulation
0
0 comments X

The pith

Side-by-side runs of StarryNet, OpenSN, and Celestial against real WetLinks data expose shortcomings in current satellite emulation.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Satellite research depends on emulation tools because physical constellations remain out of reach for most labs. The study runs three open-source emulators in parallel, measures their resource costs, and aligns the outputs with actual observations collected in the WetLinks field study. The comparison shows that none of the tools fully reproduces the real-world measurements and that overheads differ markedly. These gaps point to concrete fixes developers can pursue. Readers gain a practical basis for selecting or extending an emulator for their own satellite-network experiments.

Core claim

Executing StarryNet, OpenSN, and Celestial under identical conditions and directly comparing their generated observations to the real-world measurements recorded in the WetLinks study reveals specific misalignments in modeled satellite behavior together with varying computational and setup costs.

What carries the argument

Controlled side-by-side execution of the three emulators followed by quantitative alignment of outputs to the WetLinks real-world dataset.

If this is right

  • Emulators require tighter matching between simulated delay, loss, and throughput values and those observed on actual satellite links.
  • Acceptable resource overhead for experiment setup and runtime remains a necessary condition for any usable tool.
  • Targeted improvements in propagation modeling and resource scaling can address the documented gaps.
  • Researchers gain evidence for choosing one emulator over another depending on the fidelity and cost trade-offs required by their workload.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same comparison protocol could be reused with newer field datasets to track progress in emulator accuracy over time.
  • Findings may generalize to emulation of other large-scale orbital or terrestrial networks that face similar measurement scarcity.
  • Adoption of shared benchmarks such as WetLinks could reduce duplication of validation effort across the satellite-research community.

Load-bearing premise

The WetLinks measurements constitute a valid and representative benchmark against which emulator outputs can be aligned and judged.

What would settle it

Repeating the experiments with fresh real-world traces that all three emulators reproduce within measurement error would show the claimed shortcomings do not exist.

Figures

Figures reproduced from arXiv: 2604.04498 by Dylan Smyth, Ferenc Szabo, Stanimir Vasilev, Victor Cionca.

Figure 1
Figure 1. Figure 1: Architecture for the evaluated satellite emulators [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 4
Figure 4. Figure 4: Delays incurred when updating the constellation char [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
read the original abstract

Satellite emulation software is essential for research due to the lack of access to physical testbeds. To be useful, emulators must generate observations that are well-aligned with real-world ones, and they must have acceptable resource overheads for setting up and running experiments. This study provides an in-depth evaluation of three open-source emulators: StarryNet, OpenSN, and Celestial. Running them side-by-side and comparing them with real-world measurements from the WetLinks study identifies shortcomings of current satellite emulation techniques as well as promising avenues for research and development.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The paper claims to provide an in-depth experimental evaluation of three open-source satellite constellation emulators (StarryNet, OpenSN, and Celestial) by executing them side-by-side and comparing their outputs against real-world measurements collected in the WetLinks study, with the goal of identifying shortcomings in current emulation techniques and highlighting promising research directions.

Significance. If the emulator configurations are verifiably matched to the WetLinks scenario parameters and the comparisons include quantitative metrics with error analysis, the work could offer useful empirical guidance for the satellite networking and distributed systems communities on emulator fidelity and overheads. The direct side-by-side experimental design with external real-world data is a methodological strength.

major comments (1)
  1. Abstract and experimental setup description: The central claim that shortcomings of the emulators have been identified rests on the assumption that each emulator was parameterized identically to the WetLinks study (orbital elements, satellite count, ground-station locations, link frequencies, elevation angles, atmospheric models, and traffic patterns). No description or validation of this one-to-one mapping is supplied, which is load-bearing because any mismatch would make the observed differences inconclusive as to emulator limitations versus setup artifacts.
minor comments (1)
  1. Abstract: The summary outlines the experimental approach but supplies no quantitative results, specific metrics (e.g., latency, throughput, resource usage), error bars, or statistical comparisons, which reduces the abstract's utility as a standalone summary of findings.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the careful and constructive review. The point raised about the experimental setup is important for the validity of our claims, and we address it directly below. We will revise the manuscript to strengthen the documentation of emulator configurations.

read point-by-point responses

  1. Referee: Abstract and experimental setup description: The central claim that shortcomings of the emulators have been identified rests on the assumption that each emulator was parameterized identically to the WetLinks study (orbital elements, satellite count, ground-station locations, link frequencies, elevation angles, atmospheric models, and traffic patterns). No description or validation of this one-to-one mapping is supplied, which is load-bearing because any mismatch would make the observed differences inconclusive as to emulator limitations versus setup artifacts.

    Authors: We agree that explicit documentation of the one-to-one parameterization is essential to attribute observed differences to emulator limitations rather than configuration mismatches. The current manuscript outlines the overall experimental design in Section 3 and references the WetLinks parameters at a high level, but we acknowledge it lacks the granular mapping and validation details. In the revised version, we will add a new subsection (or expanded table) in the experimental setup that lists, for each emulator, the specific values or mappings used for orbital elements, satellite count, ground-station locations, link frequencies, elevation angles, atmospheric models, and traffic patterns. Where the WetLinks study provides sufficient detail, we will include direct comparisons (e.g., computed visibility windows or link durations) to validate the configurations. This addition will also be reflected in a revised abstract to clarify the setup fidelity. revision: yes

Circularity Check

0 steps flagged

No circularity: purely experimental comparison with external benchmark

full rationale

The manuscript contains no derivations, equations, fitted parameters, or predictions. It reports side-by-side execution of three emulators against measurements taken from the independent WetLinks study. All claims rest on direct empirical alignment rather than any self-referential construction, self-citation load-bearing step, or renaming of prior results. The central evaluation therefore cannot reduce to its own inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is an empirical benchmarking study with no mathematical derivations, new postulates, or fitted parameters; it relies on standard experimental comparison methods.

pith-pipeline@v0.9.0 · 5384 in / 943 out tokens · 46108 ms · 2026-05-10T19:58:51.091282+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

21 extracted references · 21 canonical work pages

  1. [1]

    Non-terrestrial networking for 6g: Evolution, opportunities, and future directions,

    F. Wang, S. Zhang, H. Yang, and T. Q. Quek, “Non-terrestrial networking for 6g: Evolution, opportunities, and future directions,”Engineering, 2025. 10https://github.com/containers/crun

    work page 2025

  2. [2]

    {StarryNet}: empowering researchers to evaluate futuristic integrated space and terrestrial networks,

    Z. Lai, H. Li, Y . Deng, Q. Wu, J. Liu, Y . Li, J. Li, L. Liu, W. Liu, and J. Wu, “{StarryNet}: empowering researchers to evaluate futuristic integrated space and terrestrial networks,” in20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23), 2023, pp. 1309–1324

    work page 2023

  3. [3]

    Opensn: An open source library for emulating leo satellite networks,

    W. Lu, Z. Wang, S. Zhang, Q. Meng, and H. Luo, “Opensn: An open source library for emulating leo satellite networks,” inProceedings of the 8th Asia-Pacific Workshop on Networking, 2024, pp. 149–155

    work page 2024

  4. [4]

    Celestial: Virtual software system testbeds for the leo edge,

    T. Pfandzelter and D. Bermbach, “Celestial: Virtual software system testbeds for the leo edge,” inProceedings of the 23rd ACM/IFIP International Middleware Conference, 2022, pp. 69–81

    work page 2022

  5. [5]

    Wetlinks: a large-scale longitudinal starlink dataset with contiguous weather data,

    D. Laniewski, E. Lanfer, B. Meijerink, R. van Rijswijk-Deij, and N. Aschenbruck, “Wetlinks: a large-scale longitudinal starlink dataset with contiguous weather data,”arXiv preprint arXiv:2402.16448, 2024

    work page arXiv 2024

  6. [6]

    Firecracker: Lightweight virtualization for serverless applications,

    A. Agache, M. Brooker, A. Iordache, A. Liguori, R. Neugebauer, P. Piwonka, and D.-M. Popa, “Firecracker: Lightweight virtualization for serverless applications,” in17th USENIX symposium on networked systems design and implementation (NSDI 20), 2020, pp. 419–434

    work page 2020

  7. [7]

    Unveiling beamforming strategies of starlink leo satellites,

    M. Neinavaie and Z. M. Kassas, “Unveiling beamforming strategies of starlink leo satellites,” inProceedings of the 35th International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GNSS+ 2022), 2022, pp. 2525–2531

    work page 2022

  8. [8]

    Making sense of constellations: Methodologies for understanding star- link’s scheduling algorithms,

    H. B. Tanveer, M. Puchol, R. Singh, A. Bianchi, and R. Nithyanand, “Making sense of constellations: Methodologies for understanding star- link’s scheduling algorithms,” inCompanion of the 19th International Conference on Emerging Networking EXperiments and Technologies, 2023, pp. 37–43

    work page 2023

  9. [9]

    The cni open source satellite simulator based on omnet++,

    B. Niehoefer, S. ˇSubik, and C. Wietfeld, “The cni open source satellite simulator based on omnet++,” inProceedings of the 6th International ICST Conference on Simulation Tools and Techniques, 2013, pp. 314– 321

    work page 2013

  10. [10]

    Satellite network simulator 3,

    J. Puttonen, B. Herman, S. Rantanen, F. Laakso, and J. Kurjenniemi, “Satellite network simulator 3,” inWorkshop on Simulation for European Space Programmes (SESP), vol. 24, 2015, p. 26

    work page 2015

  11. [11]

    Exploring the

    S. Kassing, D. Bhattacherjee, A. B. ´Aguas, J. E. Saethre, and A. Singla, “Exploring the” internet from space” with hypatia,” inProceedings of the ACM Internet Measurement conference, 2020, pp. 214–229

    work page 2020

  12. [12]

    Starperf: Characterizing network performance for emerging mega-constellations,

    Z. Lai, H. Li, and J. Li, “Starperf: Characterizing network performance for emerging mega-constellations,” in2020 IEEE 28th International Conference on Network Protocols (ICNP). IEEE, 2020, pp. 1–11

    work page 2020

  13. [13]

    Stardust: A scalable and extensible simulator for the 3d continuum,

    T. Pusztai, J. Hisberger, C. Marcelino, and S. Nastic, “Stardust: A scalable and extensible simulator for the 3d continuum,”arXiv preprint arXiv:2506.01513, 2025

    work page arXiv 2025

  14. [14]

    A comprehensive review of ns-3-based simulation frameworks for leo satellite constellations: Capabilities and limitations,

    P. Manzanares-Lopez, J. P. Mu ˜noz-Gea, and J. Malgosa-Sanahuja, “A comprehensive review of ns-3-based simulation frameworks for leo satellite constellations: Capabilities and limitations,”Software: Practice and Experience, 2025

    work page 2025

  15. [15]

    Splitnn: Single-machine network emulation at scale with minute-level construction of 10k-node virtual networks,

    K. Peng, Y . Li, W. Li, X. Yu, and G. Xie, “Splitnn: Single-machine network emulation at scale with minute-level construction of 10k-node virtual networks,” inProceedings of the 9th Asia-Pacific Workshop on Networking, ser. APNET ’25. New York, NY , USA: Association for Computing Machinery, 2025, p. 128–134. [Online]. Available: https://doi.org/10.1145/37...

    work page doi:10.1145/3735358.3735379 2025

  16. [16]

    Traffic control howto,

    M. A. Brown, “Traffic control howto,” https://tldp.org/HOWTO/ Traffic-Control-HOWTO/index.html, 2006, version 1.0.2

    work page 2006

  17. [17]

    Satcp: Link-layer informed tcp adaptation for highly dynamic leo satellite networks,

    X. Cao and X. Zhang, “Satcp: Link-layer informed tcp adaptation for highly dynamic leo satellite networks,” inIEEE INFOCOM 2023-IEEE Conference on Computer Communications. IEEE, 2023, pp. 1–10

    work page 2023

  18. [18]

    Teaching computer networking with mininet,

    T.-Y . Huang, V . Jeyakumar, B. Lantz, N. Feamster, K. Winstein, and A. Sivaraman, “Teaching computer networking with mininet,” inACM SIGCOMM, 2014

    work page 2014

  19. [19]

    xeoverse: A real-time simulation platform for large leo satellite mega-constellations,

    M. M. Kassem and N. Sastry, “xeoverse: A real-time simulation platform for large leo satellite mega-constellations,” in2024 IFIP Networking Conference (IFIP Networking). IEEE, 2024, pp. 1–9

    work page 2024

  20. [20]

    Emulating space computing networks with{RHONE},

    L. Wang, Q. Li, Y . Zhou, Z. Luo, D. Zhang, S. Wang, X. Liu, and C. Xu, “Emulating space computing networks with{RHONE},” in2025 USENIX Annual Technical Conference (USENIX ATC 25), 2025, pp. 815–831

    work page 2025

  21. [21]

    Mea- surement and analysis of single-hop delay on an ip backbone network,

    K. Papagiannaki, S. Moon, C. Fraleigh, P. Thiran, and C. Diot, “Mea- surement and analysis of single-hop delay on an ip backbone network,” IEEE Journal on Selected Areas in Communications, vol. 21, no. 6, pp. 908–921, 2003

    work page 2003