ros2probe: Non-intrusive, Kernel-selective Observability for Robot Operating System 2 Middleware
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-06-27 13:07 UTCgrok-4.3pith:U5XBB7BJrecord.jsonopen to challenge →
The pith
ROS 2 observer holds discovery graph within 0.5 percent of real system
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
ros2probe reconstructs the full ROS 2 communication state from the domain's discovery packets at no bandwidth cost, then drives an in-kernel filter restricted to the topics the user asks for, lifting only those packets at minimal cost and observing what the real subscriber receives. Its interfaces and recordings match the standard ROS 2 tools. Across three hardware platforms, two DDS implementations, and seven workloads, it holds the discovery graph within 0.5% of an unobserved system while domain-joining tools inflate discovery up to 2.6 times and drop up to 38.5% of messages at saturation.
What carries the argument
Reconstruction of the full ROS 2 communication state from passive domain discovery packets combined with kernel-level selective packet lifting for requested topics.
If this is right
- Discovery graph size stays within 0.5% of an unobserved system on laptop, Jetson, and Raspberry Pi platforms.
- ros2probe reports loss with recall of 1.0 and drops none of the subscriber messages even at saturation.
- Observer CPU usage falls by up to 7 times and memory usage by up to 28 times versus domain-joining tools.
- The method stays practical on embedded robots where joining tools overload the CPU and memory.
Where Pith is reading between the lines
- The same passive reconstruction approach could reduce monitoring overhead in other publish-subscribe systems that use discovery protocols.
- Production robot fleets could run continuous observability without measurable performance cost.
- Kernel filtering might be ported to additional network stacks to achieve low-cost selective capture in other domains.
Load-bearing premise
Domain discovery packets contain sufficient information to reconstruct the complete ROS 2 communication state including all publishers, subscribers, topics, and QoS without loss of semantics.
What would settle it
Run ros2probe alongside a real subscriber in a high-load ROS 2 system and check whether the loss rate reported by ros2probe exactly equals the packets the subscriber actually misses; any difference shows incomplete state reconstruction.
Figures
read the original abstract
Robot Operating System 2 (ROS 2), the de facto standard middleware framework for robots, runs each robot as a graph of nodes communicating over the Data Distribution Service (DDS), a publish/subscribe substrate. Observing this inter-node communication in real time is essential to robot development, yet it has a price. A tool can receive data only by joining the DDS domain as a subscriber that discovery has matched to the publisher, so observing folds the tool into the system it measures and perturbs it. We define this protocol-inherent perturbation as the observer's probe effect. It inflates the discovery plane, adds deserialization cost on the observer, makes the loss it reports diverge from what the subscriber actually received, and near saturation displaces the subscriber's messages. The only escape, capturing all wire traffic passively, discards ROS 2 message semantics and scales with total traffic, not what is observed. We present ros2probe, a non-intrusive observation framework that removes the probe effect. It reconstructs the full ROS 2 communication state from the domain's discovery packets at no bandwidth cost, then drives an in-kernel filter restricted to the topics the user asks for, lifting only those packets at minimal cost and observing what the real subscriber receives. Its interfaces and recordings match the standard ROS 2 tools. Across three hardware platforms (laptop, Jetson, and Raspberry Pi), two DDS implementations, and seven robot-operation workloads, ros2probe holds the discovery graph within 0.5% of an unobserved system, whereas domain-joining tools inflate discovery up to 2.6$\times$ and drop 38.5% of the subscriber's messages at saturation while ros2probe drops none. It reports loss with a recall of 1.0, cuts observer CPU and memory by up to 7$\times$ and 28$\times$, and stays practical on the embedded robots where existing tools overload the system.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents ros2probe, a non-intrusive observability framework for ROS 2 that reconstructs the full communication graph (publishers, subscribers, topics, QoS) solely from passive DDS domain discovery packets, then applies an in-kernel filter to observe only user-specified topics. This is claimed to eliminate the probe effect of domain-joining tools. Experiments across three hardware platforms, two DDS implementations, and seven workloads report that ros2probe keeps discovery graph size within 0.5% of baseline (vs. up to 2.6× inflation for competitors), achieves recall of 1.0 for loss reporting with zero message drops at saturation (vs. 38.5% drops), and reduces observer CPU/memory by up to 7×/28×.
Significance. If the reconstruction from discovery packets is complete and lossless, and the reported performance numbers hold under independent verification, the work would be a meaningful contribution to ROS 2 tooling by enabling accurate, low-overhead monitoring on embedded robots without perturbing the observed system. The multi-platform, multi-DDS experimental design and direct head-to-head comparisons with existing tools are strengths that support practical relevance.
major comments (2)
- [Reconstruction method (methods section)] The central non-intrusive claim rests on the assumption that DDS domain discovery packets contain all information needed to reconstruct the complete ROS 2 state (publishers, subscribers, topics, QoS) without loss of semantics or additional network participation. The manuscript must provide a detailed mapping or algorithm (likely in the methods section describing reconstruction) showing how every required field is extracted, and must address whether this holds across all DDS implementations or if certain dynamic attributes require active queries.
- [§5] §5 (experimental evaluation): the reported 0.5% discovery-graph fidelity, recall=1.0, and zero message loss at saturation are load-bearing for all performance claims, yet the manuscript provides neither raw data nor sufficient methodological detail (e.g., exact packet-capture configuration, how loss is measured against the real subscriber) to allow reproduction or verification of these numbers.
minor comments (2)
- [Implementation section] Clarify the exact kernel-filter mechanism and its interface to user-space ROS 2 tools; the current description leaves open how the lifted packets are presented to match standard ros2 topic echo / bag interfaces.
- [Abstract and §5] The abstract and results tables would benefit from explicit statement of the number of runs per workload and any statistical measures (e.g., standard deviation) on the reported percentages.
Simulated Author's Rebuttal
We thank the referee for the detailed and constructive report, which highlights both the potential contribution of ros2probe and areas where additional clarity is required. We address each major comment below and will incorporate the requested enhancements in a revised manuscript.
read point-by-point responses
-
Referee: [Reconstruction method (methods section)] The central non-intrusive claim rests on the assumption that DDS domain discovery packets contain all information needed to reconstruct the complete ROS 2 state (publishers, subscribers, topics, QoS) without loss of semantics or additional network participation. The manuscript must provide a detailed mapping or algorithm (likely in the methods section describing reconstruction) showing how every required field is extracted, and must address whether this holds across all DDS implementations or if certain dynamic attributes require active queries.
Authors: We agree that a detailed extraction mapping strengthens the non-intrusive claim. The current manuscript describes the reconstruction at a high level but does not include an explicit field-by-field algorithm. In the revision we will add a dedicated subsection (likely §3.2) containing (1) a table mapping each ROS 2 entity attribute to the corresponding DDS ParticipantBuiltinTopicData, PublicationBuiltinTopicData, and SubscriptionBuiltinTopicData fields, (2) the step-by-step reconstruction algorithm, and (3) an explicit statement that the approach relies only on the standardized DDS discovery messages present in both Fast DDS and Cyclone DDS (the two implementations evaluated). We will also note that vendor-specific dynamic attributes outside the standard discovery packets are not reconstructed and would require active queries, which ros2probe deliberately avoids. revision: yes
-
Referee: [§5] §5 (experimental evaluation): the reported 0.5% discovery-graph fidelity, recall=1.0, and zero message loss at saturation are load-bearing for all performance claims, yet the manuscript provides neither raw data nor sufficient methodological detail (e.g., exact packet-capture configuration, how loss is measured against the real subscriber) to allow reproduction or verification of these numbers.
Authors: We concur that the experimental claims require greater methodological transparency for independent verification. The present manuscript reports aggregate metrics but omits the precise capture setup and loss-computation procedure. In the revision we will expand §5 with: (a) the exact tcpdump/libpcap filter expressions and interface configurations used on each platform, (b) the sequence-number comparison method used to compute subscriber loss versus observer-reported loss, and (c) a link to a public repository containing the anonymized packet traces, processing scripts, and raw metric tables that produced the 0.5 % fidelity, recall = 1.0, and zero-drop results. These additions will enable reproduction without altering the reported numbers. revision: yes
Circularity Check
No circularity; claims rest on direct experimental measurement
full rationale
The paper describes an engineering artifact (ros2probe) that passively reconstructs ROS 2 state from DDS discovery packets and evaluates it via side-by-side runtime measurements on three platforms, two DDS stacks, and seven workloads. No equations, fitted parameters, or predictive models appear; the 0.5 % discovery-graph fidelity, zero message drops, recall=1.0, and resource reductions are reported as observed deltas against an unobserved baseline and against domain-joining tools. The reconstruction assumption is stated as a design premise justified by the DDS wire protocol, not derived from or reduced to the tool's own outputs. No self-citation chain, ansatz smuggling, or renaming of known results is present. The evaluation is therefore self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
Forward citations
Cited by 1 Pith paper
-
The Three Dimensions of ROS 2 Middleware
This survey introduces a three-dimensional conceptual framework (Space, Time, State) to analyze structural trade-offs and limits in ROS 2 middleware for distributed robotic systems.
Reference graph
Works this paper leans on
-
[1]
Hazem Abaza, Debayan Roy, Shiqing Fan, Selma Saidi, and Antonios Motakis
-
[2]
In2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)
Trace-enabled timing model synthesis for ROS2-based autonomous ap- plications. In2024 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, Piscataway, NJ, USA, 1–6
-
[3]
Aya Contributors. 2024. Aya: An eBPF Library for the Rust Programming Lan- guage. https://aya-rs.dev
2024
-
[4]
Christophe Bédard, Ingo Lütkebohle, and Michel Dagenais. 2022. ros2_tracing: Multipurpose low-overhead framework for real-time tracing of ROS 2.IEEE Robotics and Automation Letters7, 3 (2022), 6511–6518
2022
-
[5]
bpftrace authors. 2024. bpftrace: High-level Tracing Language for Linux. https: //github.com/bpftrace/bpftrace
2024
-
[6]
Angelo Corsaro, Luca Cominardi, Olivier Hecart, Gabriele Baldoni, Julien Enoch Pierre Avital, Julien Loudet, Carlos Guimares, Michael Ilyin, and Dmitrii Bannov. 2023. Zenoh: Unifying communication, storage and computation from the cloud to the microcontroller. In2023 26th Euromicro Conference on Digital System Design (DSD). IEEE, Piscataway, NJ, USA, 422–428
2023
-
[7]
Yunna Cui, Liwei Shen, Bingkun Sun, Wente Lu, Haojie Zhang, and Xin Peng
-
[8]
InProceedings of the 26th International Middleware Conference
EdgeConnector: Enabling Seamless and Efficient Cross-Cluster Device Access in Edge Environment. InProceedings of the 26th International Middleware Conference. Association for Computing Machinery, New York, NY, USA, 114–125
-
[9]
Eclipse Foundation. 2024. Eclipse Cyclone DDS. https://github.com/eclipse- cyclonedds/cyclonedds
2024
-
[10]
eProsima. 2024. eProsima Fast DDS. https://github.com/eProsima/Fast-DDS
2024
-
[11]
Tânia Esteves, Francisco Neves, Rui Oliveira, and João Paulo. 2021. CAT: Content- aware tracing and analysis for distributed systems. InProceedings of the 22nd International Middleware Conference. Association for Computing Machinery, New York, NY, USA, 223–235
2021
-
[12]
Jason Gait. 1986. A probe effect in concurrent programs.Software: Practice and Experience16, 3 (1986), 225–233
1986
-
[13]
Takahisa Kuboichi, Atsushi Hasegawa, Bo Peng, Keita Miura, Kenji Funaoka, Shinpei Kato, and Takuya Azumi. 2022. CARET: Chain-aware ROS 2 evaluation tool. In2022 IEEE 20th International Conference on Embedded and Ubiquitous Computing (EUC). IEEE, Piscataway, NJ, USA, 1–8
2022
- [14]
- [15]
- [16]
-
[17]
Steven Macenski, Tully Foote, Brian Gerkey, Chris Lalancette, and William Woodall. 2022. Robot operating system 2: Design, architecture, and uses in the wild.Science robotics7, 66 (2022), eabm6074
2022
-
[18]
Steven McCanne, Van Jacobson, et al . 1993. The BSD packet filter: A new architecture for user-level packet capture.. InUSENIX winter, Vol. 46. USENIX Association, Berkeley, CA, USA, 259–270
1993
-
[19]
Tal Mizrahi, Michael Schapira, and Yoram Moses. 2024. The observer effect in computer networks. InProceedings of the 2024 Applied Networking Research Workshop. Association for Computing Machinery, New York, NY, USA, 8–14
2024
-
[20]
2014.The Real-time Publish-Subscribe Protocol (RTPS) DDS Interoperability Wire Protocol Specification, Version 2.2
Object Management Group. 2014.The Real-time Publish-Subscribe Protocol (RTPS) DDS Interoperability Wire Protocol Specification, Version 2.2. Technical Report formal/2014-09-01. Object Management Group (OMG). http://www.omg.org/ spec/DDSI-RTPS/2.2/PDF
2014
-
[21]
Hyung-Seok Park, Sanghoon Lee, Doosik Um, Hyunho Ryu, and Kyung-Joon Park. 2025. An analytical latency model of the data distribution service in ROS
2025
-
[22]
IEEE, Piscataway, NJ, USA, 1–10
InIEEE INFOCOM 2025-IEEE Conference on Computer Communications. IEEE, Piscataway, NJ, USA, 1–10
2025
-
[23]
Real-Time Innovations. 2024. RTI Connext DDS. https://www.rti.com/products/ connext-dds-professional
2024
-
[24]
Dominik Scholz, Daniel Raumer, Paul Emmerich, Alexander Kurtz, Krzysztof Lesiak, and Georg Carle. 2018. Performance Implications of Packet Filtering with Linux eBPF.. InITC (1). IEEE, Piscataway, NJ, USA, 209–217
2018
-
[25]
Alexei Starovoitov and Daniel Borkmann. 2014. extended Berkeley Packet Filter (eBPF). https://docs.kernel.org/bpf/
2014
-
[26]
Marcos AM Vieira, Matheus S Castanho, Racyus DG Pacífico, Elerson RS Santos, Eduardo PM Câmara Júnior, and Luiz FM Vieira. 2020. Fast packet processing with ebpf and xdp: Concepts, code, challenges, and applications.ACM Computing Surveys (CSUR)53, 1 (2020), 1–36
2020
-
[27]
Wireshark Foundation. 2024. Wireshark. https://www.wireshark.org
2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.