pith. sign in

arxiv: 2606.26425 · v1 · pith:T5H2CEI2new · submitted 2026-06-24 · 💻 cs.RO

A System for Fast, Resilient, and Adaptable Loco-Manipulation Behaviors on Humanoid Robots

Duncan William Calvert This is my paper

Pith reviewed 2026-06-26 01:13 UTC · model grok-4.3

classification 💻 cs.RO
keywords humanoid robotsloco-manipulationbehavior authoringaffordance templatesbehavior treeswhole-body controlruntime adaptationoperator interface
0
0 comments X

The pith

A behavior system lets humanoid robots adapt loco-manipulation tasks in minutes or hours.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper presents a robot-local system for authoring and running behaviors that coordinate locomotion, whole-body motion, perception, and contact on humanoid robots. It integrates object-centric templates, logic drawn from behavior trees, and runtime-editable perception to keep behaviors observable and directly adjustable by an operator. The goal is to support practical use in human-built spaces by allowing quick adaptation without starting from scratch each time. Experiments on multiple platforms show that existing behaviors can be adapted, extended, and combined into new variants rapidly.

Core claim

The system combines Affordance Templates for object interaction, Behavior Tree-inspired organization for logic, and runtime-editable perception through a behavior scene and primitive scene actions, all built on a whole-body controller with concurrent action layering, which together support a library of more than twenty real-robot task variants and enable adaptation in minutes or hours.

What carries the argument

Integration of object-centric Affordance Templates with Behavior Tree logic and runtime-editable perception via a behavior scene that supports concurrent action layering on a whole-body controller.

Load-bearing premise

That the specific combination of affordance templates, behavior trees, runtime-editable perception, and concurrent layering is what produces the claimed speed of adaptation and resilience.

What would settle it

A side-by-side test in which adapting or combining a behavior with the system takes days rather than minutes or hours, or yields lower reliability than a baseline method.

Figures

Figures reproduced from arXiv: 2606.26425 by Duncan William Calvert.

Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p034_7.png] view at source ↗
Figure 1
Figure 1. Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p041_1.png] view at source ↗
Figure 1.1
Figure 1.1. Figure 1.1: IHMC’s fully electric Alex humanoid robot traversing a right pull door. Alex is the [PITH_FULL_IMAGE:figures/full_fig_p042_1_1.png] view at source ↗
Figure 1
Figure 1. Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p045_1.png] view at source ↗
Figure 1.2
Figure 1.2. Figure 1.2: The Nadia humanoid robot performing a right pull lever handle door traversal using [PITH_FULL_IMAGE:figures/full_fig_p045_1_2.png] view at source ↗
Figure 1
Figure 1. Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p046_1.png] view at source ↗
Figure 1.3
Figure 1.3. Figure 1.3: Distinct real-robot behavior types developed during this work, grouped by category. [PITH_FULL_IMAGE:figures/full_fig_p046_1_3.png] view at source ↗
Figure 3
Figure 3. Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p059_3.png] view at source ↗
Figure 3.1
Figure 3.1. Figure 3.1: Design-space placement of every prior system reviewed in this chapter. [PITH_FULL_IMAGE:figures/full_fig_p060_3_1.png] view at source ↗
Figure 3.2
Figure 3.2. Figure 3.2: “Placing a wheel-turning template in RViz.”, taken from [ [PITH_FULL_IMAGE:figures/full_fig_p062_3_2.png] view at source ↗
Figure 3
Figure 3. Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p062_3.png] view at source ↗
Figure 3.3
Figure 3.3. Figure 3.3: “NASA Valkyrie using an AT to remove a bag from a table”, taken from [ [PITH_FULL_IMAGE:figures/full_fig_p063_3_3.png] view at source ↗
Figure 3
Figure 3. Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p064_3.png] view at source ↗
Figure 3.4
Figure 3.4. Figure 3.4: A sequence node with N children. Algorithm 1: Pseudocode of a Sequence node with 𝑁 children 1 for 𝑖 ← 1 to 𝑁 do 2 childStatus ← Tick (child(𝑖)); 3 if childStatus = Running then 4 return Running; 5 else if childStatus = Failure then 6 return Failure; 7 return Success; A fallback node runs each node in order while they are failing. This is shown in [PITH_FULL_IMAGE:figures/full_fig_p064_3_4.png] view at source ↗
Figure 3.5
Figure 3.5. Figure 3.5: A fallback node with N children. Algorithm 2: Pseudocode of a Fallback node with 𝑁 children 1 for 𝑖 ← 1 to 𝑁 do 2 childStatus ← Tick (child(𝑖)); 3 if childStatus = Running then 4 return Running; 5 else if childStatus = Success then 6 return Success; 7 return Failure; The remaining types of nodes are summarized in [PITH_FULL_IMAGE:figures/full_fig_p065_3_5.png] view at source ↗
Figure 3
Figure 3. Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p065_3.png] view at source ↗
Figure 3.6
Figure 3.6. Figure 3.6: The different types of Behavior Tree nodes. [PITH_FULL_IMAGE:figures/full_fig_p065_3_6.png] view at source ↗
Figure 3
Figure 3. Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p077_3.png] view at source ↗
Figure 4.1
Figure 4.1. Figure 4.1: The IHMC DRC operator interface during the Valve Task in the DARPA Robotics [PITH_FULL_IMAGE:figures/full_fig_p080_4_1.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p080_4.png] view at source ↗
Figure 4.2
Figure 4.2. Figure 4.2: The IHMC DRC operator interface during the Door Task in the DARPA Robotics [PITH_FULL_IMAGE:figures/full_fig_p081_4_2.png] view at source ↗
Figure 4.3
Figure 4.3. Figure 4.3: A still from the MoveIt tutorial [57] showing the red, green, and blue pose gizmo used to move the robot’s end effector. The user can drag the circular planes to reorient the pose and drag the arrows to translate it. few implementations out there such as in RViz [55] and ImGuizmo [56], with a representative MoveIt pose-gizmo example shown in [PITH_FULL_IMAGE:figures/full_fig_p082_4_3.png] view at source ↗
Figure 4.4
Figure 4.4. Figure 4.4: The IHMC DRC operator interface during the Car Egress Task in the DARPA [PITH_FULL_IMAGE:figures/full_fig_p083_4_4.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p083_4.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p084_4.png] view at source ↗
Figure 4.5
Figure 4.5. Figure 4.5: The door traversal behavior hierarchical state machine from the 2019 era. This [PITH_FULL_IMAGE:figures/full_fig_p085_4_5.png] view at source ↗
Figure 4.6
Figure 4.6. Figure 4.6: A rough terrain autonomous exploration behavior that sought out high and low flat [PITH_FULL_IMAGE:figures/full_fig_p086_4_6.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p086_4.png] view at source ↗
Figure 4.7
Figure 4.7. Figure 4.7: A simulation example of the rough terrain up and down exploration behavior. This [PITH_FULL_IMAGE:figures/full_fig_p087_4_7.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p088_4.png] view at source ↗
Figure 4.8
Figure 4.8. Figure 4.8: A simulation of Atlas performing a navigation behavior in the JavaFX-based behavior [PITH_FULL_IMAGE:figures/full_fig_p088_4_8.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p089_4.png] view at source ↗
Figure 4.9
Figure 4.9. Figure 4.9: The Agile Hexapod simulation in Simulation Construction Set. In the upper area, the [PITH_FULL_IMAGE:figures/full_fig_p090_4_9.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p092_4.png] view at source ↗
Figure 4.10
Figure 4.10. Figure 4.10: A screenshot of one of the first Robot Data eXplorer (RDX) applications on January [PITH_FULL_IMAGE:figures/full_fig_p093_4_10.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p093_4.png] view at source ↗
Figure 4.11
Figure 4.11. Figure 4.11: A perceptive locomotion behavior on Atlas in 2021 using RDX for operation. This [PITH_FULL_IMAGE:figures/full_fig_p094_4_11.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p094_4.png] view at source ↗
Figure 4.12
Figure 4.12. Figure 4.12: Development of the 2D canvas behavior tree view began in May 2021. A video is [PITH_FULL_IMAGE:figures/full_fig_p095_4_12.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p095_4.png] view at source ↗
Figure 4.13
Figure 4.13. Figure 4.13: The June 23, 2021 building exploration demo consisting of seven tasks in a hybrid [PITH_FULL_IMAGE:figures/full_fig_p096_4_13.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p096_4.png] view at source ↗
Figure 4.14
Figure 4.14. Figure 4.14: The June 23, 2021 building exploration demo user interface. In the center, a 3D view [PITH_FULL_IMAGE:figures/full_fig_p097_4_14.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p097_4.png] view at source ↗
Figure 4.15
Figure 4.15. Figure 4.15: A person following behavior which used a YOLO [ [PITH_FULL_IMAGE:figures/full_fig_p098_4_15.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p098_4.png] view at source ↗
Figure 4.16
Figure 4.16. Figure 4.16: A Bullet [73] physics simulation of Atlas with a new runtime behavior authoring implementation. A pull door opening behavior is being authored. At this time there were only 4 action types: Walk, Hand Pose, Hand Configuration, and Chest Orientation. A video is available at https://youtu.be/KfUFNM7SWz8. April 6, 2022. supported a linear sequence of actions and they were only executable one at a time. 4.4.… view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p099_4.png] view at source ↗
Figure 4.17
Figure 4.17. Figure 4.17: A presentation of teleoperation features in 2023 for the Nadia humanoid robot in [PITH_FULL_IMAGE:figures/full_fig_p100_4_17.png] view at source ↗
Figure 4.18
Figure 4.18. Figure 4.18: The Milestone 1 tasks from April 2023: rough terrain (top left), push door (top [PITH_FULL_IMAGE:figures/full_fig_p101_4_18.png] view at source ↗
Figure 4.19
Figure 4.19. Figure 4.19: Nadia executing a can of soup pick and place behavior on June 20, 2023. Videos are [PITH_FULL_IMAGE:figures/full_fig_p102_4_19.png] view at source ↗
Figure 4.20
Figure 4.20. Figure 4.20: Nadia executing an autonomous push door behavior in 36 seconds on June 27, 2023. [PITH_FULL_IMAGE:figures/full_fig_p103_4_20.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p103_4.png] view at source ↗
Figure 4.21
Figure 4.21. Figure 4.21: An autonomous push door behavior on Nadia on August 22, 2023. A video is [PITH_FULL_IMAGE:figures/full_fig_p104_4_21.png] view at source ↗
Figure 4.22
Figure 4.22. Figure 4.22: September 1, 2023 screenshot of a new behavior authoring feature. The behavior [PITH_FULL_IMAGE:figures/full_fig_p105_4_22.png] view at source ↗
Figure 4.23
Figure 4.23. Figure 4.23: An architecture diagram for the code structure of behavior nodes that was established [PITH_FULL_IMAGE:figures/full_fig_p106_4_23.png] view at source ↗
Figure 4.24
Figure 4.24. Figure 4.24: An early version of the behavior tree version of the runtime-editable behavior [PITH_FULL_IMAGE:figures/full_fig_p107_4_24.png] view at source ↗
Figure 4.25
Figure 4.25. Figure 4.25: A screenshot from November 14, 2023, showing the introduction of the green arrow [PITH_FULL_IMAGE:figures/full_fig_p107_4_25.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p108_4.png] view at source ↗
Figure 4.26
Figure 4.26. Figure 4.26: A screenshot from November 16, 2023, showing the introduction of a move nodes [PITH_FULL_IMAGE:figures/full_fig_p108_4_26.png] view at source ↗
Figure 4.27
Figure 4.27. Figure 4.27: A screenshot from November 16, 2023, showing how nodes could be converted to [PITH_FULL_IMAGE:figures/full_fig_p109_4_27.png] view at source ↗
Figure 4.28
Figure 4.28. Figure 4.28: A screenshot from December 7, 2023, showing a new screw primitive action type. [PITH_FULL_IMAGE:figures/full_fig_p110_4_28.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p110_4.png] view at source ↗
Figure 4.29
Figure 4.29. Figure 4.29: A behavior for picking up a shoe (top left), walking to the side (top right), placing the [PITH_FULL_IMAGE:figures/full_fig_p111_4_29.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p111_4.png] view at source ↗
Figure 4.30
Figure 4.30. Figure 4.30: A screenshot from January 30, 2024 showing a refreshed user interface for the [PITH_FULL_IMAGE:figures/full_fig_p112_4_30.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p112_4.png] view at source ↗
Figure 4.31
Figure 4.31. Figure 4.31: A screenshot of the February 4, 2024, 17-second push door behavior. A video is [PITH_FULL_IMAGE:figures/full_fig_p113_4_31.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p113_4.png] view at source ↗
Figure 4.32
Figure 4.32. Figure 4.32: A screenshot from February 28, 2024, showing the introduction of the “mark [PITH_FULL_IMAGE:figures/full_fig_p114_4_32.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p114_4.png] view at source ↗
Figure 4.33
Figure 4.33. Figure 4.33: The 14 second continuous walking spring-loaded push bar door traversal on March [PITH_FULL_IMAGE:figures/full_fig_p115_4_33.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p115_4.png] view at source ↗
Figure 4.34
Figure 4.34. Figure 4.34: An illustration of the “execute after” concurrency mechanism. Arrows represent a [PITH_FULL_IMAGE:figures/full_fig_p116_4_34.png] view at source ↗
Figure 4.35
Figure 4.35. Figure 4.35: The April 9, 2024 bimanual box pick and place behavior. A video is available at [PITH_FULL_IMAGE:figures/full_fig_p116_4_35.png] view at source ↗
Figure 4.36
Figure 4.36. Figure 4.36: A 19 second pull door behavior executed on April 12, 2024. A video is available at [PITH_FULL_IMAGE:figures/full_fig_p117_4_36.png] view at source ↗
Figure 4.37
Figure 4.37. Figure 4.37: A pull door behavior is disturbed during the opening phase 5 times. The robot retries [PITH_FULL_IMAGE:figures/full_fig_p118_4_37.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p118_4.png] view at source ↗
Figure 4.38
Figure 4.38. Figure 4.38: Our July 3, 2024 run where we attempted to traverse three doors in a row with [PITH_FULL_IMAGE:figures/full_fig_p120_4_38.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p120_4.png] view at source ↗
Figure 4.39
Figure 4.39. Figure 4.39: The “ONR Demo” run where we entered a mock building through a push door (top), [PITH_FULL_IMAGE:figures/full_fig_p122_4_39.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p122_4.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p123_4.png] view at source ↗
Figure 4.40
Figure 4.40. Figure 4.40: The more complete July 19, 2024 ONR Demo run 2 collage. This run entered [PITH_FULL_IMAGE:figures/full_fig_p124_4_40.png] view at source ↗
Figure 4.41
Figure 4.41. Figure 4.41: The ONR Demo behavior tree (Part 1). 85 [PITH_FULL_IMAGE:figures/full_fig_p125_4_41.png] view at source ↗
Figure 4.42
Figure 4.42. Figure 4.42: The ONR Demo behavior tree (Part 2). On the day of the ONR Demo, we were able to accomplish each task without a failure. However, in the run-up to the demo, we noted an important property of behavior composition. Our full demo was composed of seven main loco-manipulation behaviors: 1. Traverse a push bar door. 2. Move the recycling bin out of the way. 3. Traverse a push knob handle door. 86 [PITH_FULL_… view at source ↗
Figure 4.43
Figure 4.43. Figure 4.43: A demo using the “AI to Robot” (AI2R) node to work with an external LLM process [PITH_FULL_IMAGE:figures/full_fig_p128_4_43.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p128_4.png] view at source ↗
Figure 4.44
Figure 4.44. Figure 4.44: Whiteboards from late 2024 and early 2025 behavior system design meetings. Left: [PITH_FULL_IMAGE:figures/full_fig_p130_4_44.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p130_4.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p131_4.png] view at source ↗
Figure 4.45
Figure 4.45. Figure 4.45: An illustration of fallback and goto node operation within the depth-first action [PITH_FULL_IMAGE:figures/full_fig_p132_4_45.png] view at source ↗
Figure 4.46
Figure 4.46. Figure 4.46: A condition node backed by an LLM. This test on March 6, 2025, replicated a simple [PITH_FULL_IMAGE:figures/full_fig_p133_4_46.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p133_4.png] view at source ↗
Figure 4.47
Figure 4.47. Figure 4.47: An October 21, 2025 behavior design meeting where the behavior scene and the [PITH_FULL_IMAGE:figures/full_fig_p134_4_47.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p135_4.png] view at source ↗
Figure 4.48
Figure 4.48. Figure 4.48: A November 4th, 2025 demo of the new behavior scene. On the right, the behavior [PITH_FULL_IMAGE:figures/full_fig_p136_4_48.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p136_4.png] view at source ↗
Figure 4.49
Figure 4.49. Figure 4.49: A November 10th, 2025 demo where our Unitree H1-2 [ [PITH_FULL_IMAGE:figures/full_fig_p138_4_49.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p138_4.png] view at source ↗
Figure 4.50
Figure 4.50. Figure 4.50: A November 19, 2025 test of our new generalized proximity condition. A video is [PITH_FULL_IMAGE:figures/full_fig_p139_4_50.png] view at source ↗
Figure 4.51
Figure 4.51. Figure 4.51: A November 13th, 2025 development – showing arrows on the right side which [PITH_FULL_IMAGE:figures/full_fig_p140_4_51.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p140_4.png] view at source ↗
Figure 4.52
Figure 4.52. Figure 4.52: A December 10, 2025 screenshot that shows the BehaviorTreeExecutor YoVariable [PITH_FULL_IMAGE:figures/full_fig_p141_4_52.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p142_4.png] view at source ↗
Figure 4.53
Figure 4.53. Figure 4.53: A December 11, 2025 demo where we used FoundationPose [ [PITH_FULL_IMAGE:figures/full_fig_p142_4_53.png] view at source ↗
Figure 4.54
Figure 4.54. Figure 4.54: A screenshot of the hand previewing feature from December 13, 2025. In the center, [PITH_FULL_IMAGE:figures/full_fig_p144_4_54.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p144_4.png] view at source ↗
Figure 4.55
Figure 4.55. Figure 4.55: Unitree H1-2 with Ability Hand performing a pinch grasp on the knob door handle. [PITH_FULL_IMAGE:figures/full_fig_p145_4_55.png] view at source ↗
Figure 4.56
Figure 4.56. Figure 4.56: Our December 23, 2025 demonstration of a knob door handle repeated opening [PITH_FULL_IMAGE:figures/full_fig_p145_4_56.png] view at source ↗
Figure 4.57
Figure 4.57. Figure 4.57: The 32-time pull lever door opening reliability demo run on January 2nd, 2026. A [PITH_FULL_IMAGE:figures/full_fig_p146_4_57.png] view at source ↗
Figure 4.58
Figure 4.58. Figure 4.58: The hand action fallback workaround from January 4th, 2026. The behavior would [PITH_FULL_IMAGE:figures/full_fig_p147_4_58.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p147_4.png] view at source ↗
Figure 4.59
Figure 4.59. Figure 4.59: The scene action settings for setting up a YOLO object. Dropdown combo boxes are [PITH_FULL_IMAGE:figures/full_fig_p148_4_59.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p148_4.png] view at source ↗
Figure 4.60
Figure 4.60. Figure 4.60: Our door panel detector in a screenshot from January 20, 2026. [PITH_FULL_IMAGE:figures/full_fig_p149_4_60.png] view at source ↗
Figure 4.61
Figure 4.61. Figure 4.61: Our first manipulation behavior on the IHMC Alex robot on January 20, 2026. A [PITH_FULL_IMAGE:figures/full_fig_p150_4_61.png] view at source ↗
Figure 4.62
Figure 4.62. Figure 4.62: Alex executing a pull door approach and opening behavior on January 23, 2026 as the [PITH_FULL_IMAGE:figures/full_fig_p151_4_62.png] view at source ↗
Figure 4.63
Figure 4.63. Figure 4.63: Alex executing its first full door traversal, a pull lever door, on January 24, 2026. A [PITH_FULL_IMAGE:figures/full_fig_p152_4_63.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p153_4.png] view at source ↗
Figure 4.64
Figure 4.64. Figure 4.64: The behavior tree used for Alex’s first pull door traversal on January 24, 2026. [PITH_FULL_IMAGE:figures/full_fig_p153_4_64.png] view at source ↗
Figure 4.65
Figure 4.65. Figure 4.65: A demonstration of the “shape contains” condition node type. On the left, a [PITH_FULL_IMAGE:figures/full_fig_p154_4_65.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p154_4.png] view at source ↗
Figure 4.66
Figure 4.66. Figure 4.66: A demonstration of the FREEZE_OBJECT scene action type. On the left, a scene action can be seen in the tree named “Freeze door lever handle”. In the bottom left, the scene action settings area is shown with the FREEZE_OBJECT type selected. In the center, the 3D scene has text overlaid reading “Freezing object: door_lever”. On the right, in the Scene panel, the door_lever object indicates “FROZEN”, meani… view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p155_4.png] view at source ↗
Figure 4.67
Figure 4.67. Figure 4.67: A screenshot from the authoring session on February 22, 2026 for the push door [PITH_FULL_IMAGE:figures/full_fig_p157_4_67.png] view at source ↗
Figure 4.68
Figure 4.68. Figure 4.68: New icons for behavior tree nodes on March 3, 2026. [PITH_FULL_IMAGE:figures/full_fig_p159_4_68.png] view at source ↗
Figure 4.69
Figure 4.69. Figure 4.69: A view-only behavior timeline implementation from March 22, 2026. In the bottom [PITH_FULL_IMAGE:figures/full_fig_p162_4_69.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p162_4.png] view at source ↗
Figure 4.70
Figure 4.70. Figure 4.70: Alex opening the break room door on March 26, 2026. A video is available at [PITH_FULL_IMAGE:figures/full_fig_p164_4_70.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p164_4.png] view at source ↗
Figure 4.71
Figure 4.71. Figure 4.71: Our first ball pick and place behavior on March 31, 2026. A video is available at [PITH_FULL_IMAGE:figures/full_fig_p165_4_71.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p165_4.png] view at source ↗
Figure 4.72
Figure 4.72. Figure 4.72: A reactive and robust ball sorting behavior on April 4, 2026, in which the robot [PITH_FULL_IMAGE:figures/full_fig_p167_4_72.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p167_4.png] view at source ↗
Figure 4.73
Figure 4.73. Figure 4.73: The composite frame feature. On the left, a series of scene actions can be seen: Clear [PITH_FULL_IMAGE:figures/full_fig_p169_4_73.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p170_4.png] view at source ↗
Figure 4.74
Figure 4.74. Figure 4.74: The table approach algorithm visualization. The approach reference frame available [PITH_FULL_IMAGE:figures/full_fig_p171_4_74.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p171_4.png] view at source ↗
Figure 4.75
Figure 4.75. Figure 4.75: A still frame from our open house demo behavior preparation on April 9, 2026, [PITH_FULL_IMAGE:figures/full_fig_p172_4_75.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p172_4.png] view at source ↗
Figure 4.76
Figure 4.76. Figure 4.76: The two-station ball sorting task scenario, as filmed on April 4, 2026. 2 yellow balls, [PITH_FULL_IMAGE:figures/full_fig_p173_4_76.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p173_4.png] view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p176_5.png] view at source ↗
Figure 5.1
Figure 5.1. Figure 5.1: This figure illustrates the domain of our behavior system. Our thesis centers on [PITH_FULL_IMAGE:figures/full_fig_p177_5_1.png] view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p177_5.png] view at source ↗
Figure 5.2
Figure 5.2. Figure 5.2: The runtime structure used in this thesis. Two processes run on the robot, and an [PITH_FULL_IMAGE:figures/full_fig_p178_5_2.png] view at source ↗
Figure 5.3
Figure 5.3. Figure 5.3: Four-part decomposition used for each behavior-tree node. Two process-specific [PITH_FULL_IMAGE:figures/full_fig_p179_5_3.png] view at source ↗
Figure 5.4
Figure 5.4. Figure 5.4: Behavior-tree synchronization between the operator UI and robot executor. Each side [PITH_FULL_IMAGE:figures/full_fig_p180_5_4.png] view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p180_5.png] view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p181_5.png] view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p182_5.png] view at source ↗
Figure 5.5
Figure 5.5. Figure 5.5: Condensed excerpt from a simple behavior JSON file. Each node records its definition [PITH_FULL_IMAGE:figures/full_fig_p182_5_5.png] view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p183_5.png] view at source ↗
Figure 5.6
Figure 5.6. Figure 5.6: IHMC overall whole-body control framework. [PITH_FULL_IMAGE:figures/full_fig_p183_5_6.png] view at source ↗
Figure 5.7
Figure 5.7. Figure 5.7: IHMC humanoid walking controller, highlighting control modules that regulate [PITH_FULL_IMAGE:figures/full_fig_p184_5_7.png] view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p187_5.png] view at source ↗
Figure 5.8
Figure 5.8. Figure 5.8: A representative sample tree. At a glance, the structure looks similar to a behavior tree; [PITH_FULL_IMAGE:figures/full_fig_p188_5_8.png] view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p189_5.png] view at source ↗
Figure 5.9
Figure 5.9. Figure 5.9: Authored tree structure for the top-level door traversal behavior and the expanded [PITH_FULL_IMAGE:figures/full_fig_p190_5_9.png] view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p191_5.png] view at source ↗
Figure 5.10
Figure 5.10. Figure 5.10: Available behavior node library in the RDX behavior editor. The menu is organized [PITH_FULL_IMAGE:figures/full_fig_p191_5_10.png] view at source ↗
Figure 5.11
Figure 5.11. Figure 5.11: The behavior scene consists of a list of active persistent detections and a privileged [PITH_FULL_IMAGE:figures/full_fig_p194_5_11.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p194_4.png] view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p195_5.png] view at source ↗
Figure 5.12
Figure 5.12. Figure 5.12: Door panel object algorithm. Stable YOLO persistent detections for the opening [PITH_FULL_IMAGE:figures/full_fig_p195_5_12.png] view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p196_5.png] view at source ↗
Figure 5.13
Figure 5.13. Figure 5.13: Latch-side frame post search (overhead view). The door panel is shown ajar at [PITH_FULL_IMAGE:figures/full_fig_p196_5_13.png] view at source ↗
Figure 5.14
Figure 5.14. Figure 5.14: Push-pull side detection on a closed door (overhead view). On the push side, the [PITH_FULL_IMAGE:figures/full_fig_p197_5_14.png] view at source ↗
Figure 5.15
Figure 5.15. Figure 5.15: Visual door state estimation across six configurations. A transparent yellow-green [PITH_FULL_IMAGE:figures/full_fig_p197_5_15.png] view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p198_5.png] view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p199_5.png] view at source ↗
Figure 5.16
Figure 5.16. Figure 5.16: Table edge detection (overhead view). Two vertical search capsules start left and [PITH_FULL_IMAGE:figures/full_fig_p200_5_16.png] view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p200_5.png] view at source ↗
Figure 5.17
Figure 5.17. Figure 5.17: The table approach scene object and affordance frame. Left: two vertical capsules [PITH_FULL_IMAGE:figures/full_fig_p201_5_17.png] view at source ↗
Figure 6.1
Figure 6.1. Figure 6.1: The behavior test facilitator on startup. A video of this example is available at [PITH_FULL_IMAGE:figures/full_fig_p206_6_1.png] view at source ↗
Figure 6.2
Figure 6.2. Figure 6.2: An empty behavior with just a root node. A video is available at [PITH_FULL_IMAGE:figures/full_fig_p209_6_2.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p209_6.png] view at source ↗
Figure 6.3
Figure 6.3. Figure 6.3: The root node context menu where we are adding a child node. [PITH_FULL_IMAGE:figures/full_fig_p211_6_3.png] view at source ↗
Figure 6.4
Figure 6.4. Figure 6.4: The node creation menu being used to create an action sequence node. [PITH_FULL_IMAGE:figures/full_fig_p212_6_4.png] view at source ↗
Figure 6.5
Figure 6.5. Figure 6.5: The sequence node has been added and renamed to “Demo Behavior.json”. [PITH_FULL_IMAGE:figures/full_fig_p213_6_5.png] view at source ↗
Figure 6.6
Figure 6.6. Figure 6.6: A right arm action has been created. We then add an arm action by right clicking the sequence node and clicking “Add Child Node...” as before, then click “Right” on the Arm row which instantiates a new arm action node with side set to right. For sided actions, we currently don’t allow changing the side after creation, however, that isn’t an entirely purposeful design choice. Now that the arm action has b… view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p214_6.png] view at source ↗
Figure 6.7
Figure 6.7. Figure 6.7: A right arm action with the 3D pose gizmo activated. [PITH_FULL_IMAGE:figures/full_fig_p215_6_7.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p217_6.png] view at source ↗
Figure 6.8
Figure 6.8. Figure 6.8: A right arm action being adjusted in jointspace. [PITH_FULL_IMAGE:figures/full_fig_p217_6_8.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p218_6.png] view at source ↗
Figure 6.9
Figure 6.9. Figure 6.9: The node mirroring context menu option. Next, we will mirror this action for the left arm using the “Mirror Node” context menu entry on the arm node, as shown in [PITH_FULL_IMAGE:figures/full_fig_p219_6_9.png] view at source ↗
Figure 6.10
Figure 6.10. Figure 6.10: The walk action settings and goal tuning. [PITH_FULL_IMAGE:figures/full_fig_p220_6_10.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p220_6.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p221_6.png] view at source ↗
Figure 6.11
Figure 6.11. Figure 6.11: The walk action executing. In [PITH_FULL_IMAGE:figures/full_fig_p223_6_11.png] view at source ↗
Figure 6.12
Figure 6.12. Figure 6.12: Example behavior definition serialized as JSON (Part 1). [PITH_FULL_IMAGE:figures/full_fig_p224_6_12.png] view at source ↗
Figure 6.14
Figure 6.14. Figure 6.14: The non-concurrent starting behavior for the concurrency example. A video of this [PITH_FULL_IMAGE:figures/full_fig_p224_6_14.png] view at source ↗
Figure 6.13
Figure 6.13. Figure 6.13: Example behavior definition serialized as JSON (Part 2). [PITH_FULL_IMAGE:figures/full_fig_p225_6_13.png] view at source ↗
Figure 6.15
Figure 6.15. Figure 6.15: Adding wait nodes after the walk action. A video of this demonstration is available at [PITH_FULL_IMAGE:figures/full_fig_p228_6_15.png] view at source ↗
Figure 6.16
Figure 6.16. Figure 6.16: Setting the execute after fields of the arm actions to point to the wait actions. A video [PITH_FULL_IMAGE:figures/full_fig_p229_6_16.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p229_6.png] view at source ↗
Figure 6.17
Figure 6.17. Figure 6.17: The concurrently executed result. A video of this demonstration is available at [PITH_FULL_IMAGE:figures/full_fig_p230_6_17.png] view at source ↗
Figure 6.18
Figure 6.18. Figure 6.18: Tuning a screw primitive action. A video of this demonstration is available at [PITH_FULL_IMAGE:figures/full_fig_p231_6_18.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p233_6.png] view at source ↗
Figure 6.19
Figure 6.19. Figure 6.19: The beginning of our behavior scene demonstration. A video of this demonstration is [PITH_FULL_IMAGE:figures/full_fig_p234_6_19.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p235_6.png] view at source ↗
Figure 6.20
Figure 6.20. Figure 6.20: The behavior scene demonstration, after configuring YOLO, we are now configuring [PITH_FULL_IMAGE:figures/full_fig_p237_6_20.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p237_6.png] view at source ↗
Figure 6.21
Figure 6.21. Figure 6.21: Lifecycle of persistent detections in the behavior scene. New instantaneous [PITH_FULL_IMAGE:figures/full_fig_p238_6_21.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p238_6.png] view at source ↗
Figure 6.22
Figure 6.22. Figure 6.22: The behavior scene demonstration, after configuring persistent detections and setting [PITH_FULL_IMAGE:figures/full_fig_p239_6_22.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p239_6.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p240_6.png] view at source ↗
Figure 6.23
Figure 6.23. Figure 6.23: Scene object types supported by the SETUP_OBJECT scene action. Direct objects are based on persistent detections, whereas derived objects are constructed from named frames, paired detections, or depth-based geometric calculations [PITH_FULL_IMAGE:figures/full_fig_p240_6_23.png] view at source ↗
Figure 6.24
Figure 6.24. Figure 6.24: The behavior scene demonstration, after freezing the sports ball object. A video of [PITH_FULL_IMAGE:figures/full_fig_p241_6_24.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p242_6.png] view at source ↗
Figure 6.25
Figure 6.25. Figure 6.25: The behavior scene demonstration, authoring an arm action with respect to a sports [PITH_FULL_IMAGE:figures/full_fig_p243_6_25.png] view at source ↗
Figure 6.26
Figure 6.26. Figure 6.26: A demonstration of re-authoring a footstep plan online. A video of this [PITH_FULL_IMAGE:figures/full_fig_p244_6_26.png] view at source ↗
Figure 6.27
Figure 6.27. Figure 6.27: Adding the first step of a manual footstep plan. A video of this demonstration is [PITH_FULL_IMAGE:figures/full_fig_p245_6_27.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p245_6.png] view at source ↗
Figure 6.28
Figure 6.28. Figure 6.28: A fully authored manual footstep plan that was executed successfully by the robot. A [PITH_FULL_IMAGE:figures/full_fig_p246_6_28.png] view at source ↗
Figure 6.29
Figure 6.29. Figure 6.29: The beginning of our fallback node demonstration. A video of this demonstration is [PITH_FULL_IMAGE:figures/full_fig_p247_6_29.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p248_6.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p248_4.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p249_6.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p249_4.png] view at source ↗
Figure 6.30
Figure 6.30. Figure 6.30: Adding an arm action to the fallback catch. A video of this demonstration is [PITH_FULL_IMAGE:figures/full_fig_p249_6_30.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p250_6.png] view at source ↗
Figure 6.31
Figure 6.31. Figure 6.31: Adding a goto node to implement a while loop. A video of this demonstration is [PITH_FULL_IMAGE:figures/full_fig_p250_6_31.png] view at source ↗
Figure 6.32
Figure 6.32. Figure 6.32: Completing our fallback demonstration by reactively moving the arm to the correct [PITH_FULL_IMAGE:figures/full_fig_p251_6_32.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p251_6.png] view at source ↗
Figure 6.33
Figure 6.33. Figure 6.33: The initial state of the application after we’ve started the robot and the controller and [PITH_FULL_IMAGE:figures/full_fig_p253_6_33.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p253_6.png] view at source ↗
Figure 6.34
Figure 6.34. Figure 6.34: Adding a flat Ability Hand action. A video is available at [PITH_FULL_IMAGE:figures/full_fig_p254_6_34.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p254_6.png] view at source ↗
Figure 6.35
Figure 6.35. Figure 6.35: Specifying finger joint angles for an Ability Hand action. A video is available at [PITH_FULL_IMAGE:figures/full_fig_p255_6_35.png] view at source ↗
Figure 6.36
Figure 6.36. Figure 6.36: Authoring a “left arm ready” action, in preparation for pre-grasp. A video is [PITH_FULL_IMAGE:figures/full_fig_p256_6_36.png] view at source ↗
Figure 6.37
Figure 6.37. Figure 6.37: Locking onto the lever handle and adding a curl fingers action. A video is available [PITH_FULL_IMAGE:figures/full_fig_p257_6_37.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p257_6.png] view at source ↗
Figure 6.38
Figure 6.38. Figure 6.38: Authoring first pre-grasp hand pose. A video is available at [PITH_FULL_IMAGE:figures/full_fig_p258_6_38.png] view at source ↗
Figure 6.39
Figure 6.39. Figure 6.39: Iterative tuning loop for the first pre-grasp hand pose. The pose is guessed with the [PITH_FULL_IMAGE:figures/full_fig_p259_6_39.png] view at source ↗
Figure 6.40
Figure 6.40. Figure 6.40: Authoring second pre-grasp hand pose, which contacts the handle. A video is [PITH_FULL_IMAGE:figures/full_fig_p260_6_40.png] view at source ↗
Figure 6.41
Figure 6.41. Figure 6.41: Authoring the handle turn screw primitive. A video is available at [PITH_FULL_IMAGE:figures/full_fig_p261_6_41.png] view at source ↗
Figure 6.42
Figure 6.42. Figure 6.42: Authoring the open door screw primitive. A video is available at [PITH_FULL_IMAGE:figures/full_fig_p262_6_42.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p262_6.png] view at source ↗
Figure 6.43
Figure 6.43. Figure 6.43: Running the repeated door opening behavior autonomously. A video is available at [PITH_FULL_IMAGE:figures/full_fig_p263_6_43.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p266_7.png] view at source ↗
Figure 7.1
Figure 7.1. Figure 7.1: In-house real robot task durations on a log-second axis with internal phase structure. [PITH_FULL_IMAGE:figures/full_fig_p267_7_1.png] view at source ↗
Figure 7.2
Figure 7.2. Figure 7.2: The Nadia humanoid performing a rough-terrain traversal with the [PITH_FULL_IMAGE:figures/full_fig_p268_7_2.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p269_7.png] view at source ↗
Figure 7.3
Figure 7.3. Figure 7.3: The June 23, 2021 building exploration demo on IHMC Atlas, including the [PITH_FULL_IMAGE:figures/full_fig_p269_7_3.png] view at source ↗
Figure 7.4
Figure 7.4. Figure 7.4: Nadia executing the supervised can-of-soup pick-and-place behavior on June 20, [PITH_FULL_IMAGE:figures/full_fig_p270_7_4.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p271_7.png] view at source ↗
Figure 7.5
Figure 7.5. Figure 7.5: March 15, 2024 Nadia left push-bar traversal on a door with a closer. This [PITH_FULL_IMAGE:figures/full_fig_p271_7_5.png] view at source ↗
Figure 7.6
Figure 7.6. Figure 7.6: The July 3, 2024 Nadia run attempting three consecutive door traversals with [PITH_FULL_IMAGE:figures/full_fig_p272_7_6.png] view at source ↗
Figure 7.7
Figure 7.7. Figure 7.7: July 19, 2024 Nadia right pull-handle traversal with hook hands. This representative [PITH_FULL_IMAGE:figures/full_fig_p273_7_7.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p274_7.png] view at source ↗
Figure 7.8
Figure 7.8. Figure 7.8: July 19, 2024 Nadia ONR mock-building demo run 2 collage. The run entered [PITH_FULL_IMAGE:figures/full_fig_p275_7_8.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p276_7.png] view at source ↗
Figure 7.9
Figure 7.9. Figure 7.9: Current Alex right pull lever-handle traversal key frames from the March 9, 2026 run. [PITH_FULL_IMAGE:figures/full_fig_p277_7_9.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p277_7.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p278_7.png] view at source ↗
Figure 7.10
Figure 7.10. Figure 7.10: Current Alex left push lever-handle traversal key frames from the March 9, 2026 run. [PITH_FULL_IMAGE:figures/full_fig_p278_7_10.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p279_7.png] view at source ↗
Figure 7.11
Figure 7.11. Figure 7.11: Alex reactive single-table ball sorting on April 4, 2026. The run includes human [PITH_FULL_IMAGE:figures/full_fig_p280_7_11.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p281_7.png] view at source ↗
Figure 7.12
Figure 7.12. Figure 7.12: The April 14, 2026 Alex two-table loco-manipulation ball-sorting task. A video is [PITH_FULL_IMAGE:figures/full_fig_p282_7_12.png] view at source ↗
Figure 7.13
Figure 7.13. Figure 7.13: Distance progress through the door frame for four representative 2024 Nadia [PITH_FULL_IMAGE:figures/full_fig_p283_7_13.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p284_7.png] view at source ↗
Figure 7.14
Figure 7.14. Figure 7.14: In-house reliability and resilience evidence at a glance. The repeated run band [PITH_FULL_IMAGE:figures/full_fig_p284_7_14.png] view at source ↗
Figure 7.15
Figure 7.15. Figure 7.15: April 14, 2026 Alex left push door approach and opening reliability test. Key frames [PITH_FULL_IMAGE:figures/full_fig_p285_7_15.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p286_7.png] view at source ↗
Figure 7.16
Figure 7.16. Figure 7.16: April 14, 2026 Alex right pull door approach and opening reliability test. Key frames [PITH_FULL_IMAGE:figures/full_fig_p286_7_16.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p287_7.png] view at source ↗
Figure 7.17
Figure 7.17. Figure 7.17: January 2, 2026 Unitree H1-2 right pull opening reliability test. The left three panels [PITH_FULL_IMAGE:figures/full_fig_p287_7_17.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p288_7.png] view at source ↗
Figure 7.18
Figure 7.18. Figure 7.18: April 12, 2024 Nadia reactive left pull handle opening under human disturbance. [PITH_FULL_IMAGE:figures/full_fig_p288_7_18.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p289_7.png] view at source ↗
Figure 7.19
Figure 7.19. Figure 7.19: March 13, 2026 Alex right pull door reactivity demonstration. The opening retries [PITH_FULL_IMAGE:figures/full_fig_p290_7_19.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p290_7.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p291_7.png] view at source ↗
Figure 7.20
Figure 7.20. Figure 7.20: In-house real-robot from-scratch authoring durations on a log-second axis with [PITH_FULL_IMAGE:figures/full_fig_p292_7_20.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p292_7.png] view at source ↗
Figure 7.21
Figure 7.21. Figure 7.21: Nadia during the 2023 scratch right push door authoring session. A video is [PITH_FULL_IMAGE:figures/full_fig_p293_7_21.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p294_7.png] view at source ↗
Figure 7.22
Figure 7.22. Figure 7.22: Key frames from the Unitree H1-2 scratch authoring session on January 2, 2026, [PITH_FULL_IMAGE:figures/full_fig_p294_7_22.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p295_7.png] view at source ↗
Figure 7.23
Figure 7.23. Figure 7.23: Alex during the January 20–24, 2026 first right pull door bring up, culminating in the [PITH_FULL_IMAGE:figures/full_fig_p295_7_23.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p296_7.png] view at source ↗
Figure 7.24
Figure 7.24. Figure 7.24: Screenshot from the February 22, 2026 scratch left push door authoring session on [PITH_FULL_IMAGE:figures/full_fig_p297_7_24.png] view at source ↗
Figure 7.25
Figure 7.25. Figure 7.25: In-house real-robot adaptation durations on a log-second axis with internal milestone [PITH_FULL_IMAGE:figures/full_fig_p299_7_25.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p299_7.png] view at source ↗
Figure 7.26
Figure 7.26. Figure 7.26: Alex after the first fully autonomous bottle pickup, walk, and door traversal with the [PITH_FULL_IMAGE:figures/full_fig_p300_7_26.png] view at source ↗
Figure 7.27
Figure 7.27. Figure 7.27: Alex opening the break room left pull door on March 26, 2026 after mirrored [PITH_FULL_IMAGE:figures/full_fig_p301_7_27.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p302_7.png] view at source ↗
Figure 7.28
Figure 7.28. Figure 7.28: The two table ball sorting task on April 14, 2026. A video is available at [PITH_FULL_IMAGE:figures/full_fig_p302_7_28.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p303_7.png] view at source ↗
Figure 7.29
Figure 7.29. Figure 7.29: Literature comparison of real-robot door task duration on a log-second axis. [PITH_FULL_IMAGE:figures/full_fig_p304_7_29.png] view at source ↗
Figure 7.30
Figure 7.30. Figure 7.30: Literature comparison of real-robot door-task success rate. [PITH_FULL_IMAGE:figures/full_fig_p306_7_30.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p306_7.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p308_7.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p309_7.png] view at source ↗
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p310_7.png] view at source ↗
Figure 7.32
Figure 7.32. Figure 7.32: Calendar timeline of IHMC real robot behavior milestones since the 2015 DRC [PITH_FULL_IMAGE:figures/full_fig_p311_7_32.png] view at source ↗
Figure 1
Figure 1. Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p311_1.png] view at source ↗
read the original abstract

Humanoid robots could take on physically demanding, hazardous, and repetitive work in spaces built for humans. However, a useful robot for these spaces must coordinate locomotion, whole body motion, perception, contact, and operator supervision. This thesis presents a robot-local, runtime-editable behavior authoring and runtime system. Our system strives to be maximally observable, predictable, and directable following Coactive Design principles developed during the DARPA Robotics Challenge. Our operator interface remains continuously synchronized to the robot for runtime authoring, monitoring, and repair. Our behavior architecture uniquely combines object-centric Affordance Templates, organization and logic inspired by Behavior Trees, and runtime-editable perception through a behavior scene and primitive scene actions. Action primitives build on a whole-body controller that supports moving the arms while walking, and use a concurrent action layering algorithm for speed. The behavior library developed during this work covers more than twenty real-robot task variants, including push and pull doors with knob, push-bar, and lever-handle mechanisms, multi-step exploration sequences, obstacle clearing, and reactive table-to-table manipulation tasks. This behavior system has been deployed on many humanoid robots, such as Boston Dynamics' DRC Atlas, NASA's Valkyrie, IHMC and Boardwalk Robotics' Nadia, Unitree's H1-2, and IHMC's Alex. We evaluate our system across capability, speed, reliability, and speed of behavior creation, adaptation, extension, and combination. Our experiments demonstrate that we can adapt, extend, and combine existing behaviors to create novel loco-manipulation behaviors in minutes or hours. Videos: https://www.youtube.com/playlist?list=PLJK5CTyotYqsfgfnXb-09YNFeBose6uEY.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The paper presents a robot-local, runtime-editable behavior authoring and runtime system for humanoid loco-manipulation that combines object-centric Affordance Templates, Behavior Tree-inspired organization, runtime-editable perception through behavior and primitive scenes, and concurrent action layering atop a whole-body controller supporting arm motion during walking. It describes a library covering more than twenty real-robot task variants (door mechanisms, exploration, obstacle clearing, table-to-table manipulation) deployed on platforms including DRC Atlas, Valkyrie, Nadia, H1-2, and Alex, and evaluates the system on capability, speed, reliability, and speed of behavior creation, adaptation, extension, and combination, claiming that novel behaviors can be created in minutes or hours.

Significance. If supported by quantitative evidence, the work would offer a practical contribution to humanoid deployment by emphasizing coactive design principles for observability and directability, with demonstrated multi-platform applicability and coverage of contact-rich tasks. The explicit integration of perception editing and concurrent layering for speed is a notable architectural choice that could reduce operator burden in real-world settings.

major comments (1)
  1. [Abstract] Abstract: The central claim that experiments demonstrate adaptation, extension, and combination of behaviors to create novel loco-manipulation tasks 'in minutes or hours' is unsupported by any reported quantitative metrics, such as measured adaptation times for specific tasks, number of evaluated adaptations, operator effort data, statistical comparisons, or baselines against prior authoring methods. This directly affects the soundness of the speed-of-adaptation evaluation.
minor comments (1)
  1. [Abstract] The manuscript refers to itself as a 'thesis' in the abstract; clarify whether this is a journal article derived from thesis work and ensure all evaluation details are self-contained.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback. The single major comment concerns the lack of quantitative metrics supporting the abstract's claim on adaptation speed. We address this directly below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The central claim that experiments demonstrate adaptation, extension, and combination of behaviors to create novel loco-manipulation tasks 'in minutes or hours' is unsupported by any reported quantitative metrics, such as measured adaptation times for specific tasks, number of evaluated adaptations, operator effort data, statistical comparisons, or baselines against prior authoring methods. This directly affects the soundness of the speed-of-adaptation evaluation.

    Authors: We agree that the abstract's claim would be strengthened by explicit quantitative metrics. The manuscript reports a library of more than twenty real-robot task variants across multiple platforms and states that novel behaviors were created in minutes or hours, but does not include measured times, counts of evaluated adaptations, or comparisons to baselines. We will revise the abstract and add a dedicated subsection with quantitative adaptation data (e.g., recorded times and operator actions for specific extensions) to support the evaluation. revision: yes

Circularity Check

0 steps flagged

No circularity: descriptive systems paper with no mathematical derivations or self-referential predictions

full rationale

The paper is a systems description of a robot behavior authoring architecture combining Affordance Templates, Behavior Trees, and runtime perception. It reports experimental outcomes on adaptation speed but contains no equations, fitted parameters, uniqueness theorems, or predictions that reduce to prior definitions by construction. The central claim of 'minutes or hours' adaptation is presented as an empirical observation from deployments rather than a derived result from self-cited inputs. No load-bearing self-citations or ansatzes appear in the provided text. This is a normal non-finding for a non-mathematical robotics systems contribution.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is a systems engineering thesis with no mathematical model. No free parameters, axioms, or invented entities are introduced; claims rest on the described software architecture and qualitative experimental deployments.

pith-pipeline@v0.9.1-grok · 5849 in / 1115 out tokens · 19368 ms · 2026-06-26T01:13:54.589335+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

90 extracted references · 34 canonical work pages

  1. [1]

    Team ihmc’s lessons learned from the darpa robotics challenge: Finding data in the rubble,

    M. Johnson et al., “Team ihmc’s lessons learned from the darpa robotics challenge: Finding data in the rubble,”Journal of Field Robotics, vol. 34, no. 2, pp. 241–261, 2017

    2017

  2. [2]

    A behavior architecture for fast humanoid robot door traversals,

    D. Calvert et al., “A behavior architecture for fast humanoid robot door traversals,” Robotics and Autonomous Systems, 2024

    2024

  3. [3]

    Moveit! task constructor for task-level motion planning,

    M. Görner, R. Haschke, H. Ritter, and J. Zhang, “Moveit! task constructor for task-level motion planning,” in2019 International Conference on Robotics and Automation (ICRA), 2019, pp. 190–196.doi: 10.1109/ICRA.2019.8793898

    work page doi:10.1109/icra.2019.8793898 2019

  4. [4]

    Formerly MoveIt Studio., 2021

    PickNik Robotics,Moveit pro, Robotics application platform built on BehaviorTree.CPP and ROS2. Formerly MoveIt Studio., 2021. [Online]. Available: https://picknik.ai/pro/

    2021

  5. [5]

    Faconti and BehaviorTree.CPP Contributors,Behaviortree.cpp, C++17 library for creating Behavior Trees

    D. Faconti and BehaviorTree.CPP Contributors,Behaviortree.cpp, C++17 library for creating Behavior Trees. https://www.behaviortree.dev, 2019. [Online]. Available: https://github.com/BehaviorTree/BehaviorTree.CPP

    2019

  6. [6]

    Faconti and Groot Contributors,Groot 1.0, Graphical editor for BehaviorTree.CPP (v3.8.x)

    D. Faconti and Groot Contributors,Groot 1.0, Graphical editor for BehaviorTree.CPP (v3.8.x). Legacy version in maintenance mode., 2019. [Online]. Available: https://github.com/BehaviorTree/Groot

    2019

  7. [7]

    Faconti and A

    D. Faconti and A. Robotics,Groot2, IDE to create, monitor, and debug Behavior Trees. Compatible with BT.CPP 3 and 4., 2022. [Online]. Available: https://www.behaviortree.dev/groot

    2022

  8. [8]

    Behaviors for robust door opening and doorway traversal with a force-sensing mobile manipulator,

    A. Jain and C. C. Kemp, “Behaviors for robust door opening and doorway traversal with a force-sensing mobile manipulator,” inRSS Manipulation Workshop: Intelligence in Human Environments, Georgia Institute of Technology, Zurich, Jun. 2008. [Online]. Available: http://hdl.handle.net/1853/37399 280

    2008

  9. [9]

    Planning for autonomous door opening with a mobile manipulator,

    S. Chitta, B. Cohen, and M. Likhachev, “Planning for autonomous door opening with a mobile manipulator,” in2010 IEEE International Conference on Robotics and Automation, 2010, pp. 1799–1806.doi: 10.1109/ROBOT.2010.5509475

    work page doi:10.1109/robot.2010.5509475 2010

  10. [10]

    Xiong, R

    H. Xiong, R. Mendonca, K. Shaw, and D. Pathak,Adaptive mobile manipulation for articulated objects in the open world, 2024. arXiv: 2401.14403[cs.RO]

    arXiv 2024

  11. [11]

    A versatile door opening system with mobile manipulator through adaptive position-force control and reinforcement learning,

    G. Kang, H. Seong, D. Lee, and D. H. Shim, “A versatile door opening system with mobile manipulator through adaptive position-force control and reinforcement learning,”Robotics and Autonomous Systems, vol. 180, p. 104760, 2024,issn: 0921-8890.doi: https://doi.org/10.1016/j.robot.2024.104760 [Online]. Available: https://www.sciencedirect.com/science/articl...

    work page doi:10.1016/j.robot.2024.104760 2024

  12. [12]

    Opening the sim-to-real door for humanoid pixel-to-action policy transfer,

    H. Xue et al., “Opening the sim-to-real door for humanoid pixel-to-action policy transfer,” arXiv preprint arXiv:2512.01061, 2025

    arXiv 2025

  13. [13]

    Humanoids 2022 - awards,

    “Humanoids 2022 - awards,” 2022 IEEE-RAS International Conference on Humanoid Robots, Accessed: Apr. 13, 2026. [Online]. Available: https://www.humanoids2022.org/program/awards

    2022

  14. [14]

    Congrats to IHMC robotics for winning bronze in my humanoid olympics: Doors event: Round knob push door with a time of 18 seconds,

    B. Holson. “Congrats to IHMC robotics for winning bronze in my humanoid olympics: Doors event: Round knob push door with a time of 18 seconds,” General Robots, Accessed: Apr. 25, 2026. [Online]. Available: https://generalrobots.substack.com/p/congrats-to-ihmc-robotics-for-winning

    2026

  15. [15]

    A fast, autonomous, bipedal walking behavior over rapid regions,

    D. Calvert, B. Mishra, S. McCrory, S. Bertrand, R. Griffin, and J. Pratt, “A fast, autonomous, bipedal walking behavior over rapid regions,” in2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids), 2022, pp. 24–31.doi: 10.1109/Humanoids53995.2022.10000120

    work page doi:10.1109/humanoids53995.2022.10000120 2022

  16. [16]

    A system for resilient and adaptable loco-manipulation behaviors on humanoid robots,

    D. Calvert et al., “A system for resilient and adaptable loco-manipulation behaviors on humanoid robots,”IEEE Robotics and Automation Letters, 2026, In preparation. 281

    2026

  17. [17]

    Efficient terrain map using planar regions for footstep planning on humanoid robots,

    B. Mishra, D. Calvert, S. Bertrand, J. Pratt, H. E. Sevil, and R. Griffin, “Efficient terrain map using planar regions for footstep planning on humanoid robots,” in2024 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2024

    2024

  18. [18]

    Oikonomou, A

    B. Mishra et al., “Perception engine using a multi-sensor head to enable high-level humanoid robot behaviors,” in2022 International Conference on Robotics and Automation (ICRA), 2022, pp. 9251–9257.doi: 10.1109/ICRA46639.2022.9812178

    work page doi:10.1109/icra46639.2022.9812178 2022

  19. [19]

    Aquavis: A Perception-Aware Autonomous Navigation Framework for Underwater Vehicles,

    B. Mishra, D. Calvert, S. Bertrand, S. McCrory, R. Griffin, and H. E. Sevil, “GPU-accelerated rapid planar region extraction for dynamic behaviors on legged robots,” in2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021, pp. 8493–8499.doi: 10.1109/IROS51168.2021.9636009

    work page doi:10.1109/iros51168.2021.9636009 2021

  20. [20]

    Mixed reality teleoperation assistance for direct control of humanoids,

    L. Penco et al., “Mixed reality teleoperation assistance for direct control of humanoids,” IEEE Robotics and Automation Letters, vol. 9, no. 2, pp. 1937–1944, 2024.doi: 10.1109/LRA.2024.3349904

    work page doi:10.1109/lra.2024.3349904 1937

  21. [21]

    High-speed and impact resilient teleoperation of humanoid robots,

    S. Bertrand et al., “High-speed and impact resilient teleoperation of humanoid robots,” in 2024 IEEE-RAS 23rd International Conference on Humanoid Robots (Humanoids), IEEE, 2024, pp. 189–196

    2024

  22. [22]

    Generating humanoid multi-contact through feasibility visualization,

    S. McCrory, S. Bertrand, A. Mohan, D. Calvert, J. Pratt, and R. Griffin, “Generating humanoid multi-contact through feasibility visualization,” in2023 IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids), IEEE, 2023, pp. 1–8

    2023

  23. [23]

    Detecting usable planar regions for legged robot locomotion,

    S. Bertrand, I. Lee, B. Mishra, D. Calvert, J. Pratt, and R. Griffin, “Detecting usable planar regions for legged robot locomotion,” in2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, pp. 4736–4742

    2020

  24. [24]

    Coactive design: Designing support for interdependence in joint activity,

    M. Johnson, J. M. Bradshaw, P. J. Feltovich, C. M. Jonker, M. B. van Riemsdijk, and M. Sierhuis, “Coactive design: Designing support for interdependence in joint activity,”J. Hum.-Robot Interact., vol. 3, no. 1, 43–69, Feb. 2014. 282

    2014

  25. [25]

    Nielsen,10 usability heuristics for user interface design, https://www.nngroup.com/articles/ten-usability-heuristics/, Accessed: 2024-05-18, 1994

    J. Nielsen,10 usability heuristics for user interface design, https://www.nngroup.com/articles/ten-usability-heuristics/, Accessed: 2024-05-18, 1994

    2024

  26. [26]

    The CLARAty architecture for robotic autonomy,

    R. Volpe, I. Nesnas, T. Estlin, D. Mutz, R. Petras, and H. Das, “The CLARAty architecture for robotic autonomy,” in2001 IEEE Aerospace Conference Proceedings (Cat. No.01TH8542), vol. 1, 2001, pp. 1/121–1/132.doi: 10.1109/AERO.2001.931701

    work page doi:10.1109/aero.2001.931701 2001

  27. [27]

    Affordance templates for shared robot control,

    S. Hart, P. Dinh, and K. A. Hambuchen, “Affordance templates for shared robot control,” in AAAI Fall Symposium on Artificial Intelligence and Human-Robot Interaction, NASA Technical Report JSC-CN-31935, AAAI, Arlington, VA, USA, Nov. 2014. [Online]. Available: https://ntrs.nasa.gov/citations/20140012413

    arXiv 2014

  28. [28]

    The affordance template ROS package for robot task programming,

    S. Hart, P. Dinh, and K. A. Hambuchen, “The affordance template ROS package for robot task programming,” inProceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2015, pp. 6227–6234.doi: 10.1109/ICRA.2015.7140073

    work page doi:10.1109/icra.2015.7140073 2015

  29. [29]

    Iskandar, C

    A. Pettinger, C. Elliott, P. Fan, and M. Pryor, “Reducing the teleoperator’s cognitive burden for complex contact tasks using affordance primitives,” in2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, pp. 11513–11518.doi: 10.1109/IROS45743.2020.9341576

    work page doi:10.1109/iros45743.2020.9341576 2020

  30. [30]

    A versatile affordance modeling framework using screw primitives to increase autonomy during manipulation contact tasks,

    A. Pettinger, F. Alambeigi, and M. Pryor, “A versatile affordance modeling framework using screw primitives to increase autonomy during manipulation contact tasks,”IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 7224–7231, 2022.doi: 10.1109/LRA.2022.3181732

    work page doi:10.1109/lra.2022.3181732 2022

  31. [31]

    Director: A user interface designed for robot operation with shared autonomy,

    P. Marion et al., “Director: A user interface designed for robot operation with shared autonomy,”Journal of Field Robotics, vol. 34, no. 2, pp. 262–280, 2017

    2017

  32. [32]

    Human-robot collaborative high-level control with application to rescue robotics,

    P. Schillinger, S. Kohlbrecher, and O. von Stryk, “Human-robot collaborative high-level control with application to rescue robotics,” in2016 IEEE International Conference on RoboticsandAutomation(ICRA),2016,pp.3898–3905.doi:10.1109/ICRA.2016.7487584 283

    work page doi:10.1109/icra.2016.7487584 2016

  33. [33]

    RAFCON: A graphical tool for engineering complex, robotic tasks,

    S. G. Brunner, F. Steinmetz, R. Belder, and A. Dömel, “RAFCON: A graphical tool for engineering complex, robotic tasks,” in2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016

    2016

  34. [34]

    Task-level authoring for remote robot teleoperation,

    E. Senft et al., “Task-level authoring for remote robot teleoperation,”Frontiers in Robotics and AI, vol. 8, p. 707149, 2021.doi: 10.3389/frobt.2021.707149

    work page doi:10.3389/frobt.2021.707149 2021

  35. [35]

    Autonomous navigation with online replanning and recovery behaviors for wheeled-legged robots using behavior trees,

    A. De Luca, L. Muratore, and N. G. Tsagarakis, “Autonomous navigation with online replanning and recovery behaviors for wheeled-legged robots using behavior trees,”IEEE Robotics and Automation Letters, vol. 8, no. 10, pp. 6803–6810, 2023.doi: 10.1109/LRA.2023.3313052

    work page doi:10.1109/lra.2023.3313052 2023

  36. [36]

    J. Wang, A. Laurenzi, and N. Tsagarakis,Autonomous behavior planning for humanoid loco-manipulation through grounded language model, 2024. arXiv: 2408.08282[cs.RO]

    arXiv 2024

  37. [37]

    J. J. Gibson,The Ecological Approach to Visual Perception. Houghton Mifflin, 1979

    1979

  38. [38]

    Oikonomou, A

    S. Hart, A. H. Quispe, M. W. Lanighan, and S. Gee, “Generalized affordance templates for mobile manipulation,” in2022 International Conference on Robotics and Automation (ICRA), 2022, pp. 6240–6246.doi: 10.1109/ICRA46639.2022.9812082

    work page doi:10.1109/icra46639.2022.9812082 2022

  39. [39]

    Deploying the nasa valkyrie humanoid for ied response: An initial approach and evaluation summary,

    S. J. Jorgensen et al., “Deploying the nasa valkyrie humanoid for ied response: An initial approach and evaluation summary,” in2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids), 2019, pp. 1–8.doi: 10.1109/Humanoids43949.2019.9034993

    work page doi:10.1109/humanoids43949.2019.9034993 2019

  40. [40]

    Colledanchise and P

    M. Colledanchise and P. "Ogren,Behavior Trees in Robotics and AI: An Introduction. CRC Press, Taylor and Francis Group, 2018.doi: 10.1201/9780429489105

    work page doi:10.1201/9780429489105 2018

  41. [41]

    Ogren, and C. Smith, “A survey of behavior trees in robotics and ai,

    M. Iovino, E. Scukins, J. Styrud, P. "Ogren, and C. Smith, “A survey of behavior trees in robotics and ai,”Robotics and Autonomous Systems, vol. 154, p. 104096, 2022,issn: 0921-8890.doi: https://doi.org/10.1016/j.robot.2022.104096 [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0921889022000513 284

    work page doi:10.1016/j.robot.2022.104096 2022

  42. [42]

    Pulling open doors and drawers: Coordinating an omni-directional base and a compliant arm with equilibrium point control,

    A. Jain and C. C. Kemp, “Pulling open doors and drawers: Coordinating an omni-directional base and a compliant arm with equilibrium point control,” in2010 IEEE International Conference on Robotics and Automation, 2010, pp. 1807–1814.doi: 10.1109/ROBOT.2010.5509445

    work page doi:10.1109/robot.2010.5509445 2010

  43. [43]

    Autonomous door opening and traversal,

    B. Axelrod and W. H. Huang, “Autonomous door opening and traversal,” in2015 IEEE InternationalConference onTechnologiesfor PracticalRobotApplications (TePRA),IEEE, 2015.doi: 10.1109/TePRA.2015.7219680

    work page doi:10.1109/tepra.2015.7219680 2015

  44. [44]

    The feel of MEMS barometers: Inexpensive and easily customized tactile array sensors,

    Y. Tenzer, L. P. Jentoft, and R. D. Howe, “The feel of MEMS barometers: Inexpensive and easily customized tactile array sensors,”IEEE Robotics & Automation Magazine, vol. 21, no. 3, pp. 89–95, 2014.doi: 10.1109/MRA.2014.2310152

    work page doi:10.1109/mra.2014.2310152 2014

  45. [45]

    Human-supervised control of the atlas humanoid robot for traversing doors,

    N. Banerjee et al., “Human-supervised control of the atlas humanoid robot for traversing doors,” in2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), 2015, pp. 722–729.doi: 10.1109/HUMANOIDS.2015.7363442

    work page doi:10.1109/humanoids.2015.7363442 2015

  46. [46]

    Motion planning of mobile manipulator for navigation including door traversal,

    K. Jang, S. Kim, and J. Park, “Motion planning of mobile manipulator for navigation including door traversal,”IEEE Robotics and Automation Letters, vol. 8, no. 7, pp. 4147–4154, 2023.doi: 10.1109/LRA.2023.3279612

    work page doi:10.1109/lra.2023.3279612 2023

  47. [47]

    Versatile multi-contact planning and control for legged loco-manipulation,

    J.-P. Sleiman, F. Farshidian, and M. Hutter, “Versatile multi-contact planning and control for legged loco-manipulation,”Science Robotics, 2023.doi: 10.1126/scirobotics.adg5014

    work page doi:10.1126/scirobotics.adg5014 2023

  48. [48]

    Enhancing autonomous door traversal for mobile manipulators using behavior trees,

    K. Thamrongaphichartkul and S. Vongbunyong, “Enhancing autonomous door traversal for mobile manipulators using behavior trees,”IEEE Access, vol. 12, pp. 90317–90330, 2024. doi: 10.1109/ACCESS.2024.3420819

    work page doi:10.1109/access.2024.3420819 2024

  49. [49]

    On realizing autonomous transport services in multi story buildings with doors and elevators,

    P. R. Schulze, S. Müller, T. Müller, and H.-M. Gross, “On realizing autonomous transport services in multi story buildings with doors and elevators,”Frontiers in Robotics and AI, vol. 12, p. 1546894, 2025.doi: 10.3389/frobt.2025.1546894 285

    work page doi:10.3389/frobt.2025.1546894 2025

  50. [50]

    Zhang, Y

    M. Zhang, Y. Ma, T. Miki, and M. Hutter,Learning to open and traverse doors with a legged manipulator, 2024. arXiv: 2409.04882[cs.RO]. [Online]. Available: https://arxiv.org/abs/2409.04882

    arXiv 2024

  51. [51]

    J. Z. Zhang et al.,Sumo: Dynamic and generalizable whole-body loco-manipulation, 2026. arXiv: 2604.08508[cs.RO]

    Pith/arXiv arXiv 2026

  52. [52]

    [Online]

    DARPA,Darpa robotics challenge (drc), Accessed: 2026-05-06, 2026. [Online]. Available: https://www.darpa.mil/research/programs/darpa-robotics-challenge

    2026

  53. [53]

    [Online]

    Wikipedia contributors,Swing (java), Accessed: 2026-05-06, 2026. [Online]. Available: https://en.wikipedia.org/wiki/Swing_(Java)

    2026

  54. [54]

    [Online]

    jMonkeyEngine,Jmonkeyengine, Accessed: 2026-05-06, 2026. [Online]. Available: https://jmonkeyengine.org/

    2026

  55. [55]

    ROS2,Rviz,Accessed:2026-05-06,2026.[Online].Available:https://github.com/ros2/rviz

    2026

  56. [56]

    Guillemet,Imguizmo, Accessed: 2026-05-06, 2026

    C. Guillemet,Imguizmo, Accessed: 2026-05-06, 2026. [Online]. Available: https://github.com/cedricguillemet/imguizmo

    2026

  57. [57]

    [Online]

    MoveIt,Quickstart in rviz tutorial, Accessed: 2026-05-06, 2026. [Online]. Available: https: //moveit.picknik.ai/main/doc/tutorials/quickstart_in_rviz/quickstart_in_rviz_tutorial.html

    2026

  58. [58]

    NASA’s space robotics challenge: Advancing robotics for future exploration missions,

    K. A. Hambuchen et al., “NASA’s space robotics challenge: Advancing robotics for future exploration missions,” inAIAA SPACE and Astronautics Forum and Exposition, 2017.doi: 10.2514/6.2017-5120

    work page doi:10.2514/6.2017-5120 2017

  59. [59]

    Carnegie Robotics,Multisense SL sensor, https://www.carnegierobotics.com/multisense-stereo-cameras, Tri-modal sensor package with stereo cameras and a spindle-mounted lidar, 2024

    2024

  60. [60]

    Hokuyo Automatic Co.,UTM-30LX-EW scanning laser rangefinder, https://www.hokuyo-aut.jp/search/single.php?serial=169, 270-degree scanning lidar used in the MultiSense SL, 2024. 286

    2024

  61. [61]

    A thousand years of productivity: The jrebel story,

    J. Kabanov and V. Vene, “A thousand years of productivity: The jrebel story,”Software: Practice and Experience, vol. 44, no. 1, pp. 105–127, 2014.doi: 10.1002/spe.2158

    work page doi:10.1002/spe.2158 2014

  62. [62]

    Perforce Software,JRebel, https://www.jrebel.com/, Accessed: 2026-05-14, 2026

    2026

  63. [63]

    Footstep planning for autonomous walking over rough terrain,

    R. Griffin, G. Wiedebach, S. McCrory, S. Bertrand, I. Lee, and J. Pratt, “Footstep planning for autonomous walking over rough terrain,” arXiv, 2019. [Online]. Available: https://arxiv.org/abs/1907.08673

    Pith/arXiv arXiv 2019

  64. [64]

    OctoMap: An efficient probabilistic 3D mapping framework based on octrees,

    A. Hornung, K. M. Wurm, M. Bennewitz, C. Stachniss, and W. Burgard, “OctoMap: An efficient probabilistic 3D mapping framework based on octrees,”Autonomous Robots, vol. 34, no. 3, pp. 189–206, 2013.doi: 10.1007/s10514-012-9321-0

    work page doi:10.1007/s10514-012-9321-0 2013

  65. [65]

    Cornut,Dear imgui, 2024

    O. Cornut,Dear imgui, 2024. [Online]. Available: https://www.dearimgui.com/

    2024

  66. [66]

    [Online]

    libGDX Community,Libgdx, 2024. [Online]. Available: https://libgdx.com

    2024

  67. [67]

    [Online]

    Eclipse Foundation,Eclipse ide, 2026. [Online]. Available: https://eclipseide.org/

    2026

  68. [68]

    [Online]

    Blender Foundation,Blender, 2026. [Online]. Available: https://www.blender.org/

    2026

  69. [69]

    [Online]

    Valve Software,Openvr, 2024. [Online]. Available: https://github.com/ValveSoftware/openvr [70]Intel realsense lidar camera l515. [Online]. Available: https://www.intelrealsense.com/lidar-camera-l515/

    2024

  70. [70]

    Ouster,Os0 ultra-wide field-of-view lidar sensor for autonomous vehicles and robotics, https://ouster.com/products/hardware/os0-lidar-sensor, Accessed: 2024-04-27, 2024

    2024

  71. [71]

    You only look once: Unified, real-time object detection,

    J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,”Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 779–788, 2016

    2016

  72. [72]

    Coumans and Y

    E. Coumans and Y. Bai,Pybullet, a python module for physics simulation for games, robotics and machine learning, http://pybullet.org, 2016–2021

    2016

  73. [73]

    SAKE Robotics,Ezgripper™robotic grippers, https://sakerobotics.com/, 2024. 287

    2024

  74. [74]

    Single-stage keypoint-based category-level object pose estimation from an RGB image,

    Y. Lin, J. Tremblay, S. Tyree, P. A. Vela, and S. Birchfield, “Single-stage keypoint-based category-level object pose estimation from an RGB image,” inIEEE International Conference on Robotics and Automation (ICRA), 2022

    2022

  75. [75]

    Unitree Robotics,Unitree H1-2 full-size universal humanoid robot, https://www.unitree.com/h1, 2024

    2024

  76. [76]

    FoundationPose: Unified 6d pose estimation and tracking of novel objects,

    B. Wen, W. Yang, J. Kautz, and S. Birchfield, “FoundationPose: Unified 6d pose estimation and tracking of novel objects,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  77. [77]

    Touch feedback and contact reflexes using the psyonic ability hand,

    A. Akhtar, J. Cornman, J. Austin, and D. Bala, “Touch feedback and contact reflexes using the psyonic ability hand,” inMEC Symposium, 2022.doi: 10.57922/mec.1956

    work page doi:10.57922/mec.1956 2022

  78. [78]

    Rrt-connect: An efficient approach to single-query path planning,

    J. J. Kuffner and S. M. LaValle, “Rrt-connect: An efficient approach to single-query path planning,” inProceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), IEEE, vol. 2, 2000, pp. 995–1001

    2000

  79. [79]

    [Online]

    StereoLabs,Zed x mini stereo camera | stereolabs, Accessed: 2024-04-26, 2024. [Online]. Available: https://www.stereolabs.com/store/products/zed-x-mini-stereo-camera

    2024

  80. [80]

    Design of a momentum-based control framework and application to the humanoid robot atlas,

    T. Koolen et al., “Design of a momentum-based control framework and application to the humanoid robot atlas,”International Journal of Humanoid Robotics, vol. 13, no. 01, p. 1650007, 2016

    2016

Showing first 80 references.