Harness Engineering for Physical AI: Robot Middleware Is the Harness Layer
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-06-27 16:13 UTCgrok-4.3pith:M6NFUDETrecord.jsonopen to challenge →
The pith
Robot middleware is the harness layer that enforces AI model outputs on physical robots through mediation of control, computing, and communication.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Robot middleware is the lowest robot-stack layer with mediating abstractions over control, computing, and communication, so it is best positioned to compose the enforcement of an AI model's output. The missing enforcement consists of three functions: Projection gates each output at emission, Isolation bounds the model's execution and transmission slot, and Transfer falls back to a verified baseline when checks fail. Each function already appears as hand-built application code; middleware should host them together because a learned policy's output crosses all three domains at once. The paper proposes realizing this as a ROS 2 Harness Profile deployment artifact that declares an AI model's out
What carries the argument
The Physical AI harness: the robot middleware layer that simultaneously mediates control, computing, and communication boundaries for learned models by composing Projection, Isolation, and Transfer.
If this is right
- AI model outputs become gated at the point of emission using surfaces already present in middleware.
- Model execution and transmission slots become bounded through middleware scheduling and network abstractions.
- Systems gain an automatic fallback to verified baselines when AI checks fail.
- A single deployment artifact can carry an AI model's output region, inference budget, and operating regime for enforcement across ROS 2, DDS, and Zenoh.
Where Pith is reading between the lines
- This role could consolidate safety logic that is now duplicated across separate robot applications.
- Language-agent harness ideas might transfer directly once middleware supplies the equivalent mediation for physical actuators and sensors.
- Standardized profiles could let the same AI model declaration work across different robot hardware stacks without per-platform rewrites.
Load-bearing premise
The three functions of Projection, Isolation, and Transfer can be hosted and composed inside existing middleware abstractions without new hardware, model changes, or additional system layers.
What would settle it
Deploy the three functions inside current middleware such as ROS 2 on a robot running a learned policy and check whether timing violations, unsafe trajectories, or bandwidth overruns still require separate application code or new layers to resolve.
Figures
read the original abstract
Robot middleware faces a new role in the era of Physical AI. Learned policies, planners, and vision-language-action (VLA) models now enter deployed robots as causal participants on the control path, but the layer that integrates them with timing, scheduling, and network has not been named. Recent language-agent work names this layer the harness, the external system that mediates tools, manages state, bounds resources, and records execution. The robotics community has not yet adopted this framing, and we propose that robot middleware is that harness. A Physical AI harness differs from a software harness in where it intervenes. A software harness mediates at tool-call boundaries. A Physical AI harness must mediate at control, computing, and communication simultaneously, because a learned policy's output crosses all three: its commands shift the trajectory, its inference time shifts the schedule, and its payload shifts the bandwidth. Robot middleware is the lowest robot-stack layer with mediating abstractions over all three, so it is best positioned to compose their enforcement. It already provides most of what a harness needs but lacks the enforcement for an AI model. We name this missing enforcement as three functions: Projection gates each output at emission, Isolation bounds the model's execution and transmission slot, and Transfer falls back to a verified baseline when checks fail. Each appears today as hand-built application code in deployed robot systems, built on surfaces robot middleware already provides. Robot middleware should host them not as the best single-axis enforcer but as the layer that composes all three. We sketch this as a ROS 2 Harness Profile, a deployment artifact that carries an AI model's declared output region, inference budget, and operating regime while the middleware enforces them across ROS 2, DDS, and Zenoh.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript claims that robot middleware constitutes the harness layer for Physical AI systems. Learned policies and VLA models act as causal participants on the control path, requiring simultaneous mediation over control, computing, and communication; middleware is the lowest stack layer possessing mediating abstractions over all three, making it the appropriate host for three named enforcement functions—Projection (gating outputs at emission), Isolation (bounding execution and transmission slots), and Transfer (falling back to verified baselines). The paper sketches these functions as extensions of existing middleware surfaces and proposes a ROS 2 Harness Profile as a deployment artifact that carries model-declared constraints for enforcement across ROS 2, DDS, and Zenoh.
Significance. If the architectural recommendation holds, the work supplies a coherent framing that unifies scattered hand-built enforcement practices already present in deployed robot systems and identifies middleware as the natural composition point for AI-model safety constraints. This could influence standardization efforts in robotics middleware and encourage reusable profiles rather than per-application ad-hoc code, though the absence of any empirical validation or formal argument limits immediate impact to conceptual guidance.
major comments (2)
- [Abstract] Abstract: the central recommendation that middleware should host the three functions rests on the assertion that they 'appear today as hand-built application code' on middleware surfaces and can be composed without new layers; no argument or example is supplied showing that Projection, Isolation, and Transfer can be realized solely through existing abstractions (e.g., without changes to scheduling or network primitives), which directly bears on whether middleware is sufficient.
- [Abstract] Abstract: the claim that middleware is 'best positioned to compose their enforcement' because it is the lowest layer with simultaneous mediating abstractions over control, computing, and communication is presented without comparison to alternative layers (e.g., the real-time control layer or a dedicated safety monitor), leaving the optimality argument as a definitional preference rather than a substantiated architectural conclusion.
minor comments (2)
- The three functions are introduced as named entities without formal definitions or pseudocode; adding concise operational specifications would clarify how they map onto concrete middleware primitives.
- The distinction between a 'software harness' (tool-call boundaries) and a 'Physical AI harness' (control/compute/communication) is stated but not illustrated with a side-by-side example; a short comparative table or scenario would aid readability.
Simulated Author's Rebuttal
We thank the referee for the constructive review and for recognizing the conceptual framing of robot middleware as the harness layer. We address the two major comments below and will incorporate clarifications in a revised manuscript to strengthen the architectural arguments.
read point-by-point responses
-
Referee: [Abstract] Abstract: the central recommendation that middleware should host the three functions rests on the assertion that they 'appear today as hand-built application code' on middleware surfaces and can be composed without new layers; no argument or example is supplied showing that Projection, Isolation, and Transfer can be realized solely through existing abstractions (e.g., without changes to scheduling or network primitives), which directly bears on whether middleware is sufficient.
Authors: We agree that the manuscript, being primarily conceptual, does not supply explicit implementation mappings or examples demonstrating realization solely on existing abstractions. The observation that similar patterns exist as hand-built code is noted but not elaborated with concrete primitives. In revision we will add a concise discussion (likely in Section 3 or a new subsection) providing illustrative mappings, such as using ROS 2 QoS policies and executor time budgets for Isolation, message filtering for Projection, and fallback topic remapping for Transfer, to show these can leverage current surfaces without core changes to scheduling or network primitives. revision: yes
-
Referee: [Abstract] Abstract: the claim that middleware is 'best positioned to compose their enforcement' because it is the lowest layer with simultaneous mediating abstractions over control, computing, and communication is presented without comparison to alternative layers (e.g., the real-time control layer or a dedicated safety monitor), leaving the optimality argument as a definitional preference rather than a substantiated architectural conclusion.
Authors: The manuscript's positioning rests on middleware being the lowest layer with simultaneous abstractions over all three domains, but we acknowledge the absence of explicit contrasts with alternatives. In the revision we will expand the relevant paragraph in the abstract and introduction to include brief comparisons, noting for example that real-time control layers typically mediate only control and computation without native communication mediation, while dedicated safety monitors often sit above middleware and lack direct integration into the control path. This will make the rationale more explicit without altering the core claim. revision: yes
Circularity Check
No significant circularity; purely conceptual reframing
full rationale
The manuscript advances an architectural recommendation that robot middleware constitutes the 'harness' layer for Physical AI, naming three enforcement functions (Projection, Isolation, Transfer) and sketching their composition inside existing ROS 2 abstractions. No equations, derivations, fitted parameters, quantitative predictions, or empirical claims appear whose validity depends on untested technical preconditions or self-referential reductions. The central claim is definitional and does not reduce to its inputs by construction, matching the reader's assessment of low circularity burden. No load-bearing self-citations, uniqueness theorems, or ansatzes are invoked.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption A learned policy's output crosses control, computing, and communication simultaneously, requiring mediation at all three levels.
invented entities (3)
-
Projection
no independent evidence
-
Isolation
no independent evidence
-
Transfer
no independent evidence
Forward citations
Cited by 1 Pith paper
-
The Three Dimensions of ROS 2 Middleware
This survey introduces a three-dimensional conceptual framework (Space, Time, State) to analyze structural trade-offs and limits in ROS 2 middleware for distributed robotic systems.
Reference graph
Works this paper leans on
-
[1]
Shubham Agarwal, Subrata Mitra, and Saud Iqbal. 2025. Argus: Quality-Aware High-Throughput Text-to-Image Inference Serving System. InProceedings of the 26th International Middleware Conference. ACM, New York, NY, USA, 13 pages. doi:10.1145/3721462.3770765
-
[2]
Mohammed Alshiekh, Roderick Bloem, Rüdiger Ehlers, Bettina Könighofer, Scott Niekum, and Ufuk Topcu. 2018. Safe Reinforcement Learning via Shielding. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. AAAI Press, Palo Alto, CA, USA, 2669–2678
2018
-
[3]
Ames, Samuel Coogan, Magnus Egerst- edt, Gennaro Notomista, Koushil Sreenath, and Paulo Tabuada
Aaron D. Ames, Samuel Coogan, Magnus Egerstedt, Gennaro Notomista, Koushil Sreenath, and Paulo Tabuada. 2019. Control Barrier Functions: Theory and Applications. In18th European Control Conference (ECC). IEEE, Piscataway, NJ, USA, 3420–3431. doi:10.23919/ECC.2019.8796030
-
[4]
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al
-
[5]
$\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
𝜋0: A Vision-Language-Action Flow Model for General Robot Control. arXiv:2410.24164. Preprint
work page internal anchor Pith review Pith/arXiv arXiv
-
[6]
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. 2023. RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control. InProceedings of the 7th Conference on Robot Learning (CoRL) (Proceedings of Machine Learning Research, Vol. 229)....
2023
-
[7]
Daniel Casini, Tobias Blaß, Ingo Lütkebohle, and Björn B. Brandenburg. 2019. Response-Time Analysis of ROS 2 Processing Chains under Reservation-Based Scheduling. In31st Euromicro Conference on Real-Time Systems (ECRTS 2019) (Leib- niz International Proceedings in Informatics (LIPIcs), Vol. 133). Schloss Dagstuhl, Dagstuhl, Germany, 6:1–6:23. doi:10.4230/...
-
[8]
Jiyeong Chae, Sanghoon Lee, Junhyung Jang, Seohyung Hong, and Kyung-Joon Park. 2023. A Survey and Perspective on Industrial Cyber-Physical Systems (ICPS): From ICPS to AI-Augmented ICPS.IEEE Transactions on Industrial Cyber- Physical Systems1 (2023), 257–272
2023
-
[9]
Angelo Corsaro, Luca Cominardi, Olivier Hécart, Gabriele Baldoni, Julien Enoch, Pierre Avital, Julien Loudet, Carlos Guimarães, Michael Ilyin, and Dmitrii Bannov
-
[10]
In2023 26th Euromicro Conference on Digital System Design (DSD)
Zenoh: Unifying Communication, Storage and Computation from the Cloud to the Microcontroller. In2023 26th Euromicro Conference on Digital System Design (DSD). IEEE, Piscataway, NJ, USA, 422–428. doi:10.1109/DSD60849.2023.00065
-
[11]
Bernhard Dieber, Benjamin Breiling, Sebastian Taurer, Severin Kacianka, Stefan Rass, and Peter Schartner. 2017. Security for the Robot Operating System.Robotics and Autonomous Systems98 (2017), 192–203. doi:10.1016/j.robot.2017.09.017
-
[12]
Daniel Enright, Hyunjong Choi, Yecheng Xiang, and Hyoseung Kim. 2024. PAAM: A Framework for Coordinated and Priority-Driven Accelerator Management in ROS 2. In2024 IEEE 30th Real-Time and Embedded Technology and Applications Symposium (RTAS). IEEE, Piscataway, NJ, USA, 13 pages. doi:10.1109/RTAS61025. 2024.00015
-
[13]
Arpan Gujarati, Reza Karimi, Safya Alzayat, Wei Hao, Antoine Kaufmann, Ymir Vigfusson, and Jonathan Mace. 2020. Serving DNNs like Clockwork: Performance Predictability from the Bottom Up. In14th USENIX Symposium on Operating Systems Design and Implementation (OSDI). USENIX Association, Berkeley, CA, USA, 443–462
2020
-
[14]
Mingcong Han, Rong Chen, Weihang Shen, Han Zhang, Jingyu Yang, and Haibo Chen. 2025. Real-Time, Work-Conserving GPU Scheduling for Concurrent DNN Inference.ACM Transactions on Computer Systems44, 1 (2025), 1–42
2025
-
[15]
Chaoyue He, Xin Zhou, Di Wang, Hong Xu, Wei Liu, and Chunyan Miao. 2026. Harness Engineering for Language Agents: The Harness Layer as Control, Agency, and Runtime. Preprints.org. doi:10.20944/preprints202603.1756
-
[16]
W. P. M. H. Heemels, Karl H. Johansson, and Paulo Tabuada. 2012. An In- troduction to Event-Triggered and Self-Triggered Control. In51st IEEE Con- ference on Decision and Control (CDC). IEEE, Piscataway, NJ, USA, 3270–3285. doi:10.1109/CDC.2012.6425820
-
[17]
Dan Hendrycks and Kevin Gimpel. 2017. A Baseline for Detecting Misclas- sified and Out-of-Distribution Examples in Neural Networks. InInternational Conference on Learning Representations (ICLR). OpenReview.net, Toulon, France, 12 pages
2017
-
[18]
Bishal Jaiswal, Himanshu Tyagi, Aditya Gopalan, and Vishal Sevani. 2023. WiROS: A QoS Software Solution for ros2 in a WiFi Network. In15th International Con- ference on Communication Systems and Networks (COMSNETS). IEEE, Piscataway, NJ, USA, 8 pages. doi:10.1109/COMSNETS56262.2023.10041412
- [19]
-
[20]
Sanghoon Lee, Jiyeong Chae, and Kyung-Joon Park. 2025. Integrating ROS 2 and Physical AI: Architecture and Challenges. In2025 Sixteenth International Conference on Ubiquitous and Future Networks (ICUFN). IEEE, Piscataway, NJ, USA, 9–11. doi:10.1109/ICUFN65838.2025.11170038
- [21]
- [22]
- [23]
-
[24]
Binqi Li, Yuan Zhu, Yingqi Zhao, Ke Cui, Xu Zhong, and Ke Lu. 2026. Fault Tolerance DDS Communications in Time-Sensitive Networking Using IEEE 802.1CB Redundancy Mechanisms.Journal of Systems Architecture175 (2026), 103756. doi:10.1016/j.sysarc.2026.103756
-
[25]
Steve Macenski, Francisco Martín, Ruffin White, and Jonatan Ginés Clavero. 2020. The Marathon 2: A Navigation System. In2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, Piscataway, NJ, USA, 2718–2725. doi:10.1109/IROS45743.2020.9341207
-
[26]
Zanchettin, and Stefano Zanero
Federico Maggi, Davide Quarta, Marcello Pogliani, Mario Polino, Andrea M. Zanchettin, and Stefano Zanero. 2022.A Security Analysis of the Data Distribution Service (DDS) Protocol. Technical Report Trendmicro-2022-DDS. Trend Micro Research, Irving, TX, USA
2022
-
[27]
Yuya Maruyama, Shinpei Kato, and Takuya Azumi. 2016. Exploring the Perfor- mance of ROS2. InProceedings of the 13th International Conference on Embed- ded Software (EMSOFT). ACM, New York, NY, USA, 1–10. doi:10.1145/2968478. 2968502
-
[28]
Qianyu Meng, Yanan Wang, Liyi Chen, Yihang Li, Wei Wu, Wenyuan Jiang, Qimeng Wang, Chengqiang Lu, Yan Gao, Yi Wu, and Yao Hu. 2026. Agent Harness for Large Language Model Agents: A Survey. Preprints.org. doi:10.20944/preprints202604.0428
-
[29]
Zizhao Mo, Huanle Xu, and Wing Cheong Lau. 2024. Optimal Resource Ef- ficiency with Fairness in Heterogeneous GPU Clusters. InProceedings of the 25th International Middleware Conference. ACM, New York, NY, USA, 36–48. doi:10.1145/3652892.3654792
-
[30]
NVIDIA. 2025. GR00T N1: An Open Foundation Model for Generalist Humanoid Robots. arXiv:2503.14734. Preprint
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[31]
2015.Data Distribution Service (DDS), Version 1.4
Object Management Group. 2015.Data Distribution Service (DDS), Version 1.4. Technical Report formal/2015-04-10. OMG, Needham, MA, USA
2015
-
[32]
2020.Extensible and Dynamic Topic Types for DDS (XTypes), Version 1.3
Object Management Group. 2020.Extensible and Dynamic Topic Types for DDS (XTypes), Version 1.3. Technical Report formal/2020-02-04. OMG, Needham, MA, USA
2020
-
[33]
Linyue Pan, Lexiao Zou, Shuo Guo, Jingchen Ni, and Hai-Tao Zheng. 2026. Natural-Language Agent Harnesses. arXiv:2603.25723. Preprint
work page internal anchor Pith review Pith/arXiv arXiv 2026
-
[34]
Hyung-Seok Park, Sanghoon Lee, Doosik Um, Hyunho Ryu, and Kyung-Joon Park. 2025. An Analytical Latency Model of the Data Distribution Service in ROS 2. InIEEE INFOCOM 2025 – IEEE Conference on Computer Communications. IEEE, Piscataway, NJ, USA, 1–10. doi:10.1109/INFOCOM55648.2025.11044454
-
[35]
Manos Pavlidakis, Giorgos Vasiliadis, Stelios Mavridis, Anargyros Argyros, Antony Chazapis, and Angelos Bilas. 2024. Guardian: Safe GPU Sharing in Multi-Tenant Environments. InProceedings of the 25th International Middleware Conference. ACM, New York, NY, USA, 14 pages. doi:10.1145/3652892.3700768
-
[36]
doi: 10.1007/978-3-030-55754-6_6
Dung T. Phan, Radu Grosu, Nils Jansen, Nicola Paoletti, Scott A. Smolka, and Scott D. Stoller. 2020. Neural Simplex Architecture. InNASA Formal Methods (NFM) (Lecture Notes in Computer Science, Vol. 12229). Springer, Cham, Switzer- land, 97–114. doi:10.1007/978-3-030-55754-6_6
-
[37]
Xue Qin, Simin Luan, John See, Cong Yang, and Zhijun Li. 2026. Harness- ing Embodied Agents: Runtime Governance for Policy-Constrained Execution. arXiv:2604.07833. Preprint, cs.RO
work page internal anchor Pith review Pith/arXiv arXiv 2026
- [38]
-
[39]
Lui Sha. 2001. Using Simplicity to Control Complexity.IEEE Software18, 4 (2001), 20–28. doi:10.1109/MS.2001.936213
-
[40]
Steve Vestal. 2007. Preemptive Scheduling of Multi-Criticality Systems with Varying Degrees of Execution Time Assurance. InProceedings of the 28th IEEE Real-Time Systems Symposium (RTSS). IEEE, Piscataway, NJ, USA, 239–243. doi:10. 1109/RTSS.2007.35
2007
-
[41]
Wilson, Abdullah Al Arafat, John Baugh, Ruozhou Yu, and Zhishan Guo
Kurt M. Wilson, Abdullah Al Arafat, John Baugh, Ruozhou Yu, and Zhishan Guo. 2025. Physics-Informed Mixed-Criticality Scheduling for F1Tenth Cars with Preemptable ROS 2 Executors. In2025 IEEE 31st Real-Time and Embedded Technology and Applications Symposium (RTAS). IEEE, Piscataway, NJ, USA, 215–227. doi:10.1109/RTAS65571.2025.00030
-
[42]
Yuxuan Zhao, Weikang Weng, Rob van Nieuwpoort, and Alexandru Uta. 2024. In Serverless, OS Scheduler Choice Costs Money: A Hybrid Scheduling Approach for Cheaper FaaS. InProceedings of the 25th International Middleware Conference. ACM, New York, NY, USA, 13 pages. doi:10.1145/3652892.3700757 8
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.