pith. sign in

arxiv: 2507.04425 · v1 · submitted 2025-07-06 · 💻 cs.NI · cs.SY· eess.IV· eess.SY

TeleSim: A Network-Aware Testbed and Benchmark Dataset for Telerobotic Applications

Zexin Deng (University of Warwick , UK) , Zhenhui Yuan (University of Warwick , Longhao Zou (Pengcheng Laboratory , China) This is my paper

Pith reviewed 2026-05-19 06:08 UTC · model grok-4.3

classification 💻 cs.NI cs.SYeess.IVeess.SY
keywords teleroboticsteleoperationnetwork simulationbenchmark datasetperformance evaluationremote manipulationquality of servicenetwork conditions
0
0 comments X

The pith

Simulated poor networks more than triple completion times and cut success rates by 64 percent for remote robot tasks.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper presents a testbed and dataset that runs three hundred trials of fine manipulation tasks under three levels of simulated network quality. It tracks how changes in bandwidth, latency, jitter, and packet loss alter completion time, success rate, video quality, and service metrics. A sympathetic reader would care because telerobotic systems are used in high-stakes settings where network problems can turn workable control into failure. The results show that the lowest quality tier produces much longer times and sharply lower success, indicating that systems must handle network variation to stay reliable.

Core claim

TeleSim supplies a benchmark of three hundred trials that records how network quality tiers affect telerobotic performance. In the lowest tier, completion time rises by 221.8 percent and success rate falls by 64 percent, accompanied by drops in video quality measures and quality-of-service parameters. The work uses controlled simulation of bandwidth, latency, jitter, and packet loss to generate comparable data across trials and releases the full set of recordings and software to let others test adaptive teleoperation methods under the same conditions.

What carries the argument

A tiered network simulation that defines high, medium, and low communication quality through fixed settings of bandwidth, latency, jitter, and packet loss and records the resulting task performance and video metrics from each trial.

If this is right

  • Network degradation produces longer task times and lower success rates that compound with reduced video quality.
  • Teleoperation designs need to include mechanisms that respond to changing network states.
  • The benchmark supplies uniform conditions for comparing different control approaches.
  • Public release of the trial data enables reproducible tests of new resilient protocols.
  • Worse network tiers reveal clear thresholds where performance becomes unreliable.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar tiered testing could be applied to evaluate remote control of vehicles or drones under variable links.
  • The measured impacts could help set minimum network requirements for safe use in surgery or hazardous environments.
  • Pairing the benchmark with delay-prediction models might produce control methods that compensate before errors accumulate.
  • Direct comparison of the simulation outputs against measurements from live deployed systems would test how well the tiers translate to practice.

Load-bearing premise

The controlled settings of bandwidth, latency, jitter, and packet loss in the simulation accurately represent the communication link experienced in real telerobotic deployments.

What would settle it

Running identical fine manipulation tasks over real physical networks set to the same high, medium, and low parameter values and checking whether the observed increases in completion time and drops in success rate match the reported 221.8 percent and 64 percent figures.

Figures

Figures reproduced from arXiv: 2507.04425 by China), Longhao Zou (Pengcheng Laboratory, UK), Zexin Deng (University of Warwick, Zhenhui Yuan (University of Warwick.

Figure 1
Figure 1. Figure 1: Illustration of the TeleSim testbed at the University of Warwick. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Hardware-in-the-loop network topology of the TeleSim testbed. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Average PSNR and SSIM for the three network tiers. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Example video frames under the three network tiers. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Average completion time and success rate for the three network tiers. [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗
read the original abstract

Telerobotic technologies are becoming increasingly essential in fields such as remote surgery, nuclear decommissioning, and space exploration. Reliable datasets and testbeds are essential for evaluating telerobotic system performance prior to real-world deployment. However, there is a notable lack of datasets that capture the impact of network delays, as well as testbeds that realistically model the communication link between the operator and the robot. This paper introduces TeleSim, a network-aware teleoperation dataset and testbed designed to assess the performance of telerobotic applications under diverse network conditions. TeleSim systematically collects performance data from fine manipulation tasks executed under three predefined network quality tiers: High, Medium, and Low. Each tier is characterized through controlled settings of bandwidth, latency, jitter, and packet loss. Using OMNeT++ for precise network simulation, we record a wide range of metrics, including completion time, success rates, video quality indicators (Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM)), and quality of service (QoS) parameters. TeleSim comprises 300 experimental trials, providing a robust benchmark for evaluating teleoperation systems across heterogeneous network scenarios. In the worst network condition, completion time increases by 221.8% and success rate drops by 64%. Our findings reveal that network degradation leads to compounding negative impacts, notably reduced video quality and prolonged task execution, highlighting the need for adaptive, resilient teleoperation protocols. The full dataset and testbed software are publicly available on our GitHub repository: https://github.com/ConnectedRoboticsLab and YouTube channel: https://youtu.be/Fz_1iOYe104.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The manuscript introduces TeleSim, a network-aware testbed and benchmark dataset for telerobotic applications. It uses OMNeT++ to simulate three predefined network quality tiers (High, Medium, Low) defined by controlled settings of bandwidth, latency, jitter, and packet loss. The authors perform 300 experimental trials on fine manipulation tasks, recording metrics including completion time, success rates, PSNR, SSIM, and QoS parameters. They report that under the Low network condition completion time increases by 221.8% and success rate drops by 64%, and release the full dataset and testbed software publicly via GitHub.

Significance. If the simulated network conditions are representative, TeleSim supplies a publicly available, reproducible benchmark with 300 trials and concrete quantitative outcomes that can support development of adaptive teleoperation protocols. The open release of code and data is a clear strength for the field.

major comments (1)
  1. [Methods (network tier definitions)] Methods (network tier definitions): The three network quality tiers are defined via fixed OMNeT++ parameters for bandwidth, latency, jitter, and packet loss, yet the manuscript provides no references to empirical traces or hardware-in-the-loop calibrations from real telerobotic links (remote surgery, nuclear, or space). This leaves the headline degradations (221.8% time increase, 64% success drop) tied to an unvalidated simulation regime rather than demonstrated heterogeneous scenarios.
minor comments (2)
  1. [Abstract and §4] Abstract and §4: The statement that network degradation leads to 'compounding negative impacts' would be strengthened by reporting any measured correlations or regression results between video quality (PSNR/SSIM) and task metrics.
  2. The manuscript would benefit from a brief table summarizing the exact parameter values used for each of the High/Medium/Low tiers.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their constructive comments on our manuscript. We address the major comment point by point below.

read point-by-point responses

  1. Referee: Methods (network tier definitions): The three network quality tiers are defined via fixed OMNeT++ parameters for bandwidth, latency, jitter, and packet loss, yet the manuscript provides no references to empirical traces or hardware-in-the-loop calibrations from real telerobotic links (remote surgery, nuclear, or space). This leaves the headline degradations (221.8% time increase, 64% success drop) tied to an unvalidated simulation regime rather than demonstrated heterogeneous scenarios.

    Authors: We acknowledge the referee's point that the manuscript does not cite specific empirical network traces or hardware-in-the-loop calibrations from real telerobotic deployments. The High, Medium, and Low tiers were defined using controlled OMNeT++ parameters chosen to span a representative range of conditions for telerobotic tasks, based on typical bandwidth, latency, jitter, and loss values discussed in the broader literature on remote operation. We agree that additional context would strengthen the presentation. In the revised version we will add a dedicated paragraph in the Methods section that (i) states the rationale for the chosen parameter values, (ii) cites representative studies reporting network requirements for remote surgery and similar applications, and (iii) explicitly notes that the reported degradations (221.8 % increase in completion time and 64 % drop in success rate) are observed under these simulated conditions. This revision will clarify the scope of the benchmark without overstating empirical validation. revision: yes

Circularity Check

0 steps flagged

Empirical simulation benchmark with direct measurements and no self-referential derivations

full rationale

The paper generates TeleSim by executing 300 trials of fine manipulation tasks inside OMNeT++ under three fixed network-quality tiers (High/Medium/Low) whose bandwidth, latency, jitter and packet-loss values are set by the authors. Reported headline figures (221.8 % completion-time increase, 64 % success-rate drop) are simply the aggregated outcomes of those runs together with the recorded PSNR, SSIM and QoS values. No equations, fitted parameters, self-citations or uniqueness theorems appear in the derivation; the central claims are therefore direct empirical outputs rather than quantities that reduce to the paper’s own inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the representativeness of the OMNeT++ network model and the chosen fine-manipulation tasks; no free parameters are fitted to data and no new entities are postulated.

axioms (1)
  • domain assumption OMNeT++ provides a precise model of bandwidth, latency, jitter, and packet loss that matches real telerobotic communication links.
    Invoked when the paper defines the three network quality tiers and records QoS parameters.

pith-pipeline@v0.9.0 · 5868 in / 1290 out tokens · 67449 ms · 2026-05-19T06:08:16.147297+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. VISTA: A Benchmark for Real-Time Video Streaming under Network Impairments in Surgical Teleoperation

    eess.IV 2026-05 accept novelty 6.0

    VISTA shows network impairments cut surgical teleoperation success rates from 97% to as low as 12% and extend task times up to 255 seconds under GEO satellite conditions.

Reference graph

Works this paper leans on

12 extracted references · 12 canonical work pages · cited by 1 Pith paper

  1. [1]

    VPLTE "(&DHE ndlkbh xovuks xFQQ|uKNJPRMUVRcb`

    11em plus .33em minus .07em @technote 4000 4000 100 4000 4000 500 `\.=1000 = #1 #1 #1 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEauthorblockAstyle \@IEEEauthordefaulttextstyle \@IEEEauthorblockconfadjspace -0.25em \@IEEEauthorblockNtopspace 0.0ex \@IEEEauthorblockAtopspace 0.0ex \@IEEEauthorblockNinterlinespace 2.6ex \@IEEEauthorblockAinte...

    work page

  2. [2]

    Global surgery 2030: evidence and solutions for achieving health, welfare, and economic development,

    J. G. Meara, A. J. M. Leather, L. Hagander, B. C. Alkire, N. Alonso, E. A. Ameh, et al., “Global surgery 2030: evidence and solutions for achieving health, welfare, and economic development,” The Lancet, vol. 386, no. 9993, pp. 569--624, Aug. 2015

    work page 2030

  3. [3]

    Measuring the impact of haptic feedback in collaborative robotic scenarios,

    F. J. Rodríguez-Sedano, M. Á. Conde, F. J. Rodríguez-Lera, et al., “Measuring the impact of haptic feedback in collaborative robotic scenarios,” Universal Access Inf. Soc., vol. 23, pp. 1031--1049, 2024

    work page 2024

  4. [4]

    Practical approaches towards transparent and stable bilateral teleoperation under time-varying network delay,

    C. Wang, R. Luo, and J. P. Whitney, “Practical approaches towards transparent and stable bilateral teleoperation under time-varying network delay,” Int. J. Social Robot., vol. 17, pp. 409--426, 2025

    work page 2025

  5. [5]

    QoE evaluation of real-time remote operation with network constraints in a system-of-systems,

    C. Schüler, T. Gebauer, M. Patchou, and C. Wietfeld, “QoE evaluation of real-time remote operation with network constraints in a system-of-systems,” in Proc. IEEE Int. Systems Conf. (SysCon), Montreal, QC, Canada, Apr. 2022, pp. 1--8

    work page 2022

  6. [6]

    N. Tian, A. K. Tanwani, K. Goldberg, and S. Sojoudi, ``Mitigating network latency in cloud-based teleoperation using motion segmentation and synthesis,'' in Robotics Research, T. Asfour, E. Yoshida, J. Park, H. Christensen, and O. Khatib, Eds. Cham: Springer International Publishing, 2022, pp. 906--921

    work page 2022

  7. [7]

    Using dVRK teleoperation to facilitate deep learning of automation tasks for an industrial robot,

    J. Liang, J. Mahler, M. Laskey, P. Li, and K. Goldberg, “Using dVRK teleoperation to facilitate deep learning of automation tasks for an industrial robot,” in Proc. 2017 13th IEEE Int. Conf. Autom. Sci. Eng. (CASE), Xi’an, China, Aug. 20--23, 2017, pp. 1--8

    work page 2017

  8. [8]

    Experimental comparison study of control architectures for bilateral teleoperators,

    E. J. Rodríguez-Seda, D. Lee, and M. W. Spong, “Experimental comparison study of control architectures for bilateral teleoperators,” IEEE Trans. Robot., vol. 25, no. 6, pp. 1304--1318, Dec. 2009

    work page 2009

  9. [9]

    OPEN TEACH: A versatile teleoperation system for robotic manipulation,

    A. Iyer, Z. Peng, Y. Dai, I. Guzey, S. Haldar, S. Chintala, and L. Pinto, “OPEN TEACH: A versatile teleoperation system for robotic manipulation,” arXiv preprint arXiv:2403.07870, 2024. [Online]. Available: https://arxiv.org/abs/2403.07870

    work page arXiv 2024

  10. [10]

    A data-set of piercing needle through deformable objects for deep learning from demonstrations,

    H. Hashempour, K. Nazari, F. Zhong, and A. Ghalamzan, “A data-set of piercing needle through deformable objects for deep learning from demonstrations,” arXiv preprint arXiv:2012.02458, 2020. [Online]. Available: https://arxiv.org/abs/2012.02458

    work page arXiv 2012

  11. [11]

    QoS conditions for VoIP and VoD,

    P. Dymarski, S. Kula, and T. N. Huy, “QoS conditions for VoIP and VoD,” Bull. Polish Acad. Sci. Tech. Sci., vol. 60, no. 3, pp. 555--570, 2012

    work page 2012

  12. [12]

    Network performance objectives for IP-based services,

    International Telecommunication Union, “Network performance objectives for IP-based services,” ITU-T Recommendation Y.1541, Geneva, Switzerland, Dec. 2011. [Online]. Available: https://www.itu.int/rec/T-REC-Y.1541

    work page 2011