Pith. sign in

REVIEW 4 major objections 7 minor 44 references

Human speech, gestures, and GUI inputs to UAVs must be treated as bounded maneuver requests—not direct commands—and mediated through a safety-checked pipeline before execution.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Operator multi-modal inputs are mediated as constrained motion primitives via a request-evaluate-execute pipeline with preconditions, invariants, and runtime monitors.

T0 review reviewed 2026-07-14 challenge →

load-bearing objection Clear architectural fix for a real multimodal-UAV safety gap, scoped honestly as initial work, but the load-bearing safety predicates are still untested under the uncertainties the authors themselves list. the 4 major comments →

arxiv 2607.11654 v1 pith:6CTT2P7Q submitted 2026-07-13 cs.RO cs.SE

A Model for Mediating Multi-Modal Human Intent into Safe Maneuvers for UAVs

Sofia Nelson , Dalal Alrajeh , Pedro Antonio Alarcon Granadeno , Jane Cleland-Huang This is my paper
classification cs.RO cs.SE
keywords Human-UAV InteractionRuntime AssuranceSafety-Critical SystemsMulti-Modal InteractionRequirements EngineeringManeuver PrimitivesUAV Safety
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Directly turning operator speech, gestures, or screen clicks into drone motion is unsafe: people misjudge distance, conditions change, and commands can hit terrain, other aircraft, or flight limits. This paper claims the fix is to treat every input as a request for a short, bounded motion primitive rather than a free command. Each request is scored for confidence, checked against terrain clearance, inter-UAV separation, local workspace, and flight-envelope limits, then either shortened, rejected, or executed under continuous monitoring. The same rules are written as explicit preconditions, invariants, guards, and postconditions so the system can be verified at runtime and, later, synthesized automatically. Lab prototypes with voice and GUI show that the pipeline can interpret inputs and keep the resulting moves inside those bounds.

Core claim

Multi-modal human intent for autonomous UAVs can be made safe by mediating it through a requirements-governed maneuver-response model: operator inputs are mapped to constrained motion primitives, passed through a request–evaluate–execute pipeline that validates confidence and four families of safety constraints, and formalized with preconditions, invariants, guard conditions, and postconditions that govern admissibility, execution, and emergency handling.

What carries the argument

The Multi-Modal Maneuver Request (M3R) model—a request–evaluate–execute pipeline that maps multi-modal inputs to bounded motion primitives and admits them only when TerrainSafe ∧ SeparationSafe ∧ EnvelopeSafe ∧ ConfidenceAdequate hold—together with per-maneuver contracts of preconditions, invariants, and postconditions.

Load-bearing premise

The central claim assumes that checking terrain, separation, envelope, and confidence constraints plus continuous monitoring is enough to keep maneuvers safe once real wind, GPS drift, latency, and degraded sensing appear—conditions the paper has only tested in controlled lab and simulated settings.

What would settle it

Fly physical UAVs outdoors near real terrain and other aircraft, issue deliberately unsafe voice or gesture commands under measured wind and GPS error, and check whether any accepted or partially constrained maneuver still produces a terrain or separation violation; any such violation falsifies the mediation claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Commands that would breach terrain, separation, workspace, or envelope limits are automatically shortened or rejected instead of flown.
  • Runtime monitors can abort an in-progress maneuver into an EmergencyHold state the moment a safety invariant is predicted to fail.
  • The same maneuver contracts can later feed reactive synthesis of correct-by-construction controllers.
  • Voice, gesture, and GUI become interchangeable once reduced to the same bounded-request abstraction and operator-relative frame.
  • Pairing establishes a local anchor workspace so subsequent moves stay near a known safe point and directions stay operator-relative.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same request-versus-command mediation could transfer to other human-on-the-loop robots whose intent recognizers are noisy.
  • Explicit contracts make it practical to bolt the layer onto existing terrain digital twins and airspace-leasing systems without rewriting the speech or gesture stack.
  • Usability will hinge on whether confidence and freshness thresholds can be set tight enough for safety without making the system reject too many legitimate requests.
  • A requirements-first mediation layer can be audited and certified independently of whichever recognizer is used underneath.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes a Multi-Modal Maneuver Request (M3R) model that mediates speech, gesture, and GUI inputs into safe UAV motion by treating operator inputs as bounded maneuver requests rather than direct commands. Requests are mapped to constrained motion primitives and processed through a request–evaluate–execute pipeline with confidence thresholds, terrain/separation/workspace/flight-envelope checks, and continuous runtime monitoring. The approach is formalized via high-level safety properties, a state machine (Fig. 1), maneuver contracts (e.g., GOFORWARD), and EARS requirements; an initial lab prototype with a simulated drone and voice/GUI (and VLLM-based gesture) interfaces is reported as qualitative evidence that constrained requests can be interpreted and executed safely under controlled conditions.

Significance. If the mediation architecture is sound and the safety predicates remain admissible under realistic sensing and prediction error, the work would provide a useful requirements-governed interface layer between multi-modal HRI and runtime assurance for small UAVs—addressing a gap that prior gesture/speech control work largely leaves open. Strengths include a clear separation of intent recognition from safety enforcement, an explicit Safe(π_cand) decision structure, complementary operationalizations (state machine + contracts + EARS), and an unusually candid Threats section. The contribution is primarily architectural and requirements-oriented rather than a closed-form safety proof or large-scale field evaluation; its value for a journal audience therefore hinges on whether claims about reliable safe execution are scoped and evidenced proportionally to the prototype.

major comments (4)
  1. [Abstract; §V; Fig. 4] Abstract and §V claim that voice- and GUI-based inputs “can be reliably interpreted and safely executed as constrained maneuver requests,” but the reported validation is a qualitative lab prototype with a simulated drone, idealized Environmental Digital Twin, and air-leasing that “guarantees unique airspace under nominal conditions.” No quantitative metrics (recognition accuracy, false-accept/false-reject rates for the Safety Supervisor, constraint-violation rates under injected obstacles, latency, or residual clearance margins) are given. For a journal claim of reliable safe mediation, either add controlled quantitative experiments with clear success/failure criteria or substantially narrow the empirical wording to “feasibility of the pipeline under idealized simulation.”
  2. [§III.E–F; §VI] The load-bearing safety claim rests on Safe(π_cand,x,T,D,C) := TerrainSafe ∧ SeparationSafe ∧ EnvelopeSafe ∧ ConfidenceAdequate (§III.E–F) plus continuous monitoring. These predicates are defined over predicted paths and models T,D,C, yet §VI explicitly lists wind, GPS drift, communication latency, and degraded sensing as unmodeled. There are no residual-error bounds, no analysis of when a predicate can return true while the true future state violates clearance/separation, and no false-negative characterization of the Runtime Monitor. Without at least a threat model and a soundness argument (or simulation stress tests with localization/terrain-model error), the architectural claim that the pipeline yields admissible maneuvers under operational uncertainty remains unestablished.
  3. [§III.A; Table I; §V.C.1; §VI] Gesture support is central to the multi-modal framing (Table I; §V.C.1), but recognition uses a general-purpose VLLM (GPT 5.o) under close-range ideal conditions, with field CV deferred to future NOMAD-trained models. Turn left/right are also left largely unimplemented because operator-relative framing after yaw is unresolved. As written, the multi-modal mediation claim is only partially instantiated; either restrict the validated modalities to voice+GUI with explicit scope, or provide a minimal quantitative gesture evaluation that matches the distances/orientations assumed by the operator-facing design (§III.A).
  4. [§IV; §V opening; §VII] The paper positions the requirements model as supporting “future reactive synthesis” and runtime verification (§IV; Conclusions), but only a manually designed prototype is implemented. That is acceptable if synthesis is clearly future work; however, several passages read as if analyzability for synthesis is already demonstrated. Please separate (i) what is currently executable and checked against EARS requirements from (ii) what would be required for synthesis (assumptions, guarantees, plant model), so the contribution boundary is unambiguous.
minor comments (7)
  1. [Title; Abstract] Title and body inconsistently space “UA V” / “UAV”; abstract also has “UA Vs.” Normalize to “UAV/UAVs” throughout.
  2. [§II–IV headings] Section header “MULTI-MODALMANEUVERREQUESTMODEL” and similar concatenated headings (e.g., RELATEDWORK, REQUIREMENTS-GUIDEDOPERATIONALIZATION) appear to be missing spaces/hyphens from PDF extraction or formatting; fix for readability.
  3. [Table I; §III.E] Table I uses placeholder bounds such as “velocity < x m/s” and “0.5x m/s”; define the symbols or give concrete default values consistent with the free parameters listed in the text (X_step, V_nudger_max, etc.).
  4. [§IV.A] Signal temporal logic example in §IV.A uses □ and ○ but cites “[]” without a proper STL reference; add a standard citation and define the signal alphabet briefly.
  5. [§III; Fig. 1; §V.A] Fig. 1 is described as operationalizing the model, but transition names in the text (e.g., ManeuverRequestArmed vs. ManeuverResponseArmed in §III) are not fully consistent; align mode names across §III, Fig. 1, and the EARS list.
  6. [§V.C.1; §VI] “GPT 5.o” / “VLMM” vs “VLLM” appear inconsistently; correct the model name and acronym.
  7. [§II.D] Related work on runtime assurance / Simplex-style mediation and formal HRI safety could be tightened with a few canonical citations so the novelty of requirements-governed maneuver contracts is clearer against existing runtime assurance literature.

Circularity Check

0 steps flagged

No significant circularity: requirements/design paper whose mediation claim is proposed rather than derived from fitted inputs or self-citation uniqueness; self-cites supply infrastructure only.

full rationale

The paper proposes a request–evaluate–execute mediation pipeline (Section III) that maps multi-modal inputs to bounded motion primitives, then filters them with an explicit safety decision function Safe(π_cand, x, T, D, C) := TerrainSafe ∧ SeparationSafe ∧ EnvelopeSafe ∧ ConfidenceAdequate, plus continuous monitoring and contracts (Section IV). This is a design/requirements contribution, not a derivation that fits parameters to data and then “predicts” them, nor a uniqueness theorem imported from prior author work. Self-citations ([39] NOMAD, [40] terrain twin, [41] air-leasing, [43] MOMUX, [1] MAPE-K extension) appear only as supporting infrastructure for the prototype and future integration; they do not force the mediation claim by construction, and the paper itself scopes validation as “initial lab-based” while listing unmodeled real-world uncertainties (Section VI). No equation reduces to its own input, no fitted quantity is relabeled a prediction, and no load-bearing uniqueness is smuggled via self-citation. Circularity burden is therefore negligible (score 1 for ordinary self-citation of infrastructure).

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 3 invented entities

The central claim is architectural: that treating multi-modal inputs as bounded requests under the stated safety decision function yields safer execution than direct command mapping. It rests on domain assumptions about UAV failure modes and on free parameters (thresholds, workspace radius, step sizes) that are configuration choices, not data fits. Invented entities are the named model and pipeline components; they are engineering constructs with no independent physical evidence required beyond implementation.

free parameters (4)
  • R_max (local workspace radius) = example 5 m
    Example value 5 m; bounds the anchor-centered workspace. Chosen by design, not fitted.
  • confidence thresholds (cf, G_min, C_min^(m), L_min, T_min)
    Gate interpretation and mode transitions; paper states a confidence-threshold is established for experiments but does not report numeric values.
  • maneuver bounds (X_step, Y_step, Z_step, V_nudger_max, Omega_nudger_max, duration) = e.g. ≤1 m, ≤0.5 m
    Table I gives example displacements (≤1 m lateral, ≤0.5 m vertical) and velocity caps; design parameters of the 'nudger' envelope.
  • MinAGL, C_min, S_min, tau_max
    Terrain clearance, separation, and neighbor-state freshness thresholds used by the safety decision function; configuration constants.
axioms (5)
  • domain assumption Operator multi-modal inputs are uncertain and may be unsafe if executed as direct commands (terrain, separation, envelope conflicts).
    Stated in Motivation and Related Work as the problem the model addresses; standard HMT safety premise.
  • ad hoc to paper Safe(π_cand) := TerrainSafe ∧ SeparationSafe ∧ EnvelopeSafe ∧ ConfidenceAdequate is an adequate admissibility predicate for bounded maneuvers.
    Defined in §III.F as the safety decision function; completeness under real dynamics is assumed, not proved.
  • domain assumption Pairing establishes operator-facing orientation so directional commands are interpreted in the operator frame.
    §III.A design choice that reduces heading ambiguity; assumed workable at operational distances.
  • domain assumption Maneuver primitives can be associated with preconditions, invariants, guards, and postconditions that support runtime verification and future reactive synthesis.
    §IV claim; synthesis itself is future work, not demonstrated.
  • standard math Standard UAV kinematics and local ENU-style coordinates suffice for planning short bounded primitives.
    Implicit in Plan/Filter and workspace equation; no novel dynamics claimed.
invented entities (3)
  • M3R (Multi-Modal Maneuver Request) model no independent evidence
    purpose: Name the overall mediation framework and mode set (NormalAutonomy, ManeuverResponseArmed/Active, Blocked, EmergencyHold, ReturnToAnchor).
    Paper-introduced systems construct; evidence is the prototype description only.
  • request-evaluate-execute pipeline with Safety Supervisor and Runtime Monitor no independent evidence
    purpose: Separate intent recognition from constraint checking and continuous monitoring.
    Architectural decomposition in §III and §V; not independently measured outside this design.
  • maneuver-specific behavioral contracts (e.g., GOFORWARD) no independent evidence
    purpose: Encode preconditions/invariants/postconditions per primitive for requirements-guided operationalization.
    §IV.C illustration; engineering artifact without external falsifiable prediction.

reviewed 2026-07-14 · how reviews work

0 comments
Cite this review

Pith. "Pith review of A Model for Mediating Multi-Modal Human Intent into Safe Maneuvers for UAVs." pith.science (2026). https://pith.science/paper/6CTT2P7Q

@misc{pith2026260711654,
  author       = {Pith},
  title        = {Pith review of: A Model for Mediating Multi-Modal Human Intent into Safe Maneuvers for UAVs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6CTT2P7Q}},
  note         = {Machine review of arXiv:2607.11654}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Direct human interaction with autonomous UAV systems can be enabled through modalities such as speech, gestures, and graphical interfaces. However, interpreting such inputs as directly executable commands introduces safety risks in dynamic environments. Operator requests may conflict with terrain constraints, inter-UAV separation requirements, or flight-envelope limitations. In this paper, we present a requirements-governed maneuver-response model that mediates multi-modal human intent into safe UAV maneuvers by treating operator inputs as bounded maneuver requests rather than direct commands. Requested maneuvers are mapped to constrained motion primitives and processed through a structured request-evaluate-execute pipeline. Each request is interpreted with associated confidence, validated against terrain, separation, workspace, and flight-envelope constraints, and either constrained, rejected, or executed under continuous runtime monitoring. We further formalize the approach as a requirements-based specification model in which maneuver primitives are associated with explicit preconditions, invariants, guard conditions, and postconditions governing admissibility, execution safety, and emergency handling. These requirements support runtime verification and future reactive synthesis approaches. We present an initial lab-based validation demonstrating that voice and GUI-based inputs can be reliably interpreted and safely executed as constrained maneuver requests.

Figures

Figures reproduced from arXiv: 2607.11654 by Dalal Alrajeh, Jane Cleland-Huang, Pedro Antonio Alarcon Granadeno, Sofia Nelson.

Figure 1
Figure 1. Figure 1: State machine operationalizing the M3R model, in [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Motion primitives for gesture-based commands. Each command supports “Fly” or “Go” prefix, but are shown here as [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: A sequence of motion commands issued by human [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

44 extracted references · 3 canonical work pages

  1. [1]

    Extending MAPE-K to support human-machine teaming,

    J. Cleland-Huang, A. Agrawal, M. Vierhauser, M. Murphy, and M. Prieto, “Extending MAPE-K to support human-machine teaming,” in International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS 2022, Pittsburgh, PA, USA, May 22-24, 2022, B. R. Schmerl, M. Maggio, and J. C ´amara, Eds. ACM/IEEE, 2022, pp. 120–131. [Online]. Avail...

    work page doi:10.1145/3524844.3528054 2022

  2. [2]

    Motion estimation and hand gesture recognition-based human–UA V interaction approach in real time,

    M. Yoo, Y . Na, H. Song, G. Kim, J. Yun, S. Kim, C. Moon, and K. Jo, “Motion estimation and hand gesture recognition-based human–UA V interaction approach in real time,” Sensors, vol. 22, no. 7, 2022. [Online]. Available: https://www.mdpi.com/1424-8220/22/7/2513

    2022

  3. [3]

    Deep learning based hand gesture recognition and UA V flight controls,

    B. Hu and J. Wang, “Deep learning based hand gesture recognition and UA V flight controls,” International Journal of Automation and Computing, vol. 17, 09 2019

    2019

  4. [4]

    UA V control with vision-based hand gesture recognition over edge-computing,

    S. Abdalla and S. Baidya, “UA V control with vision-based hand gesture recognition over edge-computing,” 05 2025, pp. 481–488

    2025

  5. [5]

    Hgic: A hand gesture based interactive control system for efficient and scalable multi- UA V operations,

    M. Hu, J. Li, R. Jin, C. Shi, L. Xu, and R. Liu, “Hgic: A hand gesture based interactive control system for efficient and scalable multi- UA V operations,” 2024. [Online]. Available: https://arxiv.org/abs/2403.05478

    Pith/arXiv arXiv 2024

  6. [6]

    A real-time hand gesture control of a quadcopter swarm implemented in the gazebo simulation environment,

    R. S. Wijaya, S. Prayoga, R. A. Fatekha, and M. T. Mubarak, “A real-time hand gesture control of a quadcopter swarm implemented in the gazebo simulation environment,” Journal of Applied Informatics and Computing, vol. 9, no. 3, p. 979–988, Jun. 2025. [Online]. Available: https://jurnal.polibatam.ac.id/index.php/JAIC/article/view/9578

    2025

  7. [7]

    Using gesture and speech communication modalities for safe human-drone interaction in construction,

    Z. Zhu, J.-Y . Cheng, I. Jeelani, and M. Gheisari, “Using gesture and speech communication modalities for safe human-drone interaction in construction,” Advanced Engineering Informatics, vol. 62, p. 102827, 2024. [Online]. Available: https://www.sciencedirect.com/ science/article/pii/S1474034624004750

    2024

  8. [8]

    “you two! take off!

    S. Pourmehr, V . M. Monajjemi, R. Vaughan, and G. Mori, ““you two! take off!”: Creating, modifying and commanding groups of robots using face engagement and indirect speech in voice commands,” in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, pp. 137–142

    2013

  9. [9]

    Multimodal speech and visual gesture control interface technique for small unmanned multirotor aircraft,

    A. Abioye, “Multimodal speech and visual gesture control interface technique for small unmanned multirotor aircraft,” Ph.D. dissertation, University of Southampton, July 2023. [Online]. Available: https: //eprints.soton.ac.uk/479472/

    2023

  10. [10]

    Multimodal fusion of voice and gesture data for UA V control,

    X. Xiaojia, Q. Tan, H. Zhou, D. Tang, and J. Lai, “Multimodal fusion of voice and gesture data for UA V control,” Drones, vol. 6, p. 201, 08 2022

    2022

  11. [11]

    The chalearn gesture dataset (cgd 2011),

    I. Guyon, V . Athitsos, P. Jangyodsuk, and H. J. Escalante, “The chalearn gesture dataset (cgd 2011),” Mach. Vision Appl., vol. 25, no. 8, p. 1929–1951, Nov. 2014. [Online]. Available: https://doi.org/10.1007/s00138-014-0596-3

    work page doi:10.1007/s00138-014-0596-3 2011

  12. [12]

    Integrated UA V swarm and human-machine interaction platform for real-time mobile training and evaluation,

    J. Hu, “Integrated UA V swarm and human-machine interaction platform for real-time mobile training and evaluation,” International Journal of Interactive Mobile Technologies (iJIM), vol. 19, pp. 83–97, 12 2025

    2025

  13. [13]

    Design and implementation of voice- command controller for fixed-wing unmanned aerial vehicles using au- tomatic speech recognition and natural language processing techniques,

    C. Villame and S. Guirnaldo, “Design and implementation of voice- command controller for fixed-wing unmanned aerial vehicles using au- tomatic speech recognition and natural language processing techniques,” Sustainable Engineering and Innovation, vol. 6, pp. 199–212, 10 2024

    2024

  14. [14]

    A. Zhou, L. Han, and Y . Meng, Multimodal Control of UA VBased on Gesture, Eye Movement and V oiceInteraction, 01 2023, pp. 3765–3774

    2023

  15. [15]

    Unmanned aerial vehicle control through domain-based automatic speech recognition,

    R. Contreras, A. Ayala, and F. Cruz, “Unmanned aerial vehicle control through domain-based automatic speech recognition,”Computers, vol. 9, no. 3, 2020. [Online]. Available: https://www.mdpi.com/2073-431X/9/ 3/75

    2020

  16. [16]

    In-vehicle speech recognition for voice- driven UA V control in a collaborative environment of MA V and UA V,

    J.-S. Park and N. Geng, “In-vehicle speech recognition for voice- driven UA V control in a collaborative environment of MA V and UA V,” Aerospace, vol. 10, no. 10, 2023. [Online]. Available: https://www.mdpi.com/2226-4310/10/10/841

    2023

  17. [17]

    Leveraging large language models for real-time UA V control,

    K. Choutri, S. Fadloun, A. Khettabi, M. Lagha, S. Meshoul, and R. Fareh, “Leveraging large language models for real-time UA V control,” Electronics, vol. 14, no. 21, 2025. [Online]. Available: https://www.mdpi.com/2079-9292/14/21/4312

    2025

  18. [18]

    Visual recognition of american sign language using hidden markov models,

    T. Starner and M. Group, “Visual recognition of american sign language using hidden markov models,” 05 1995

    1995

  19. [19]

    Deep vision-based real-time hand gesture recognition: a review,

    C. Cui, M. S. Sunar, and G. Su, “Deep vision-based real-time hand gesture recognition: a review,” PeerJ Computer Science, vol. 11, p. e2921, 06 2025

    2025

  20. [20]

    Deep learning model for dynamic hand gesture recognition for natural human-machine interface on end devices,

    T.-P. Chang, H.-M. Chen, S.-Y . Chen, and W.-C. Lin, “Deep learning model for dynamic hand gesture recognition for natural human-machine interface on end devices,” International Journal of Information System Modeling and Design, vol. 13, no. 10, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1947818622000503

    2022

  21. [21]

    Real-time human pose recognition in parts from single depth images,

    J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, and A. Blake, “Real-time human pose recognition in parts from single depth images,” in CVPR 2011, 2011, pp. 1297–1304

    2011

  22. [22]

    Robust part-based hand gesture recognition using kinect sensor,

    Z. Ren, J. Yuan, J. Meng, and Z. Zhang, “Robust part-based hand gesture recognition using kinect sensor,” Multimedia, IEEE Transactions on, vol. 15, pp. 1110–1120, 08 2013

    2013

  23. [23]

    Real time hand pose estimation using depth sensors,

    C. Keskin, F. Kırac ¸, Y . Kara, and L. Akarun, “Real time hand pose estimation using depth sensors,” 11 2011, pp. 1228–1234

    2011

  24. [24]

    Hand gesture recogni- tion with 3d convolutional neural networks,

    P. Molchanov, S. Gupta, K. Kim, and J. Kautz, “Hand gesture recogni- tion with 3d convolutional neural networks,” 06 2015, pp. 1–7

    2015

  25. [25]

    Motion fused frames: Data level fusion strategy for hand gesture recognition,

    O. K ¨op¨ukl¨u, N. K ¨ose, and G. Rigoll, “Motion fused frames: Data level fusion strategy for hand gesture recognition,” 2018. [Online]. Available: https://arxiv.org/abs/1804.07187

    Pith/arXiv arXiv 2018

  26. [26]

    Spatio-temporal lstm with trust gates for 3d human action recognition,

    J. Liu, A. Shahroudy, D. Xu, and G. Wang, “Spatio-temporal lstm with trust gates for 3d human action recognition,” 2016. [Online]. Available: https://arxiv.org/abs/1607.07043

    Pith/arXiv arXiv 2016

  27. [27]

    Vision-based hand- gesture applications,

    J. P. Wachs, M. K ¨olsch, H. Stern, and Y . Edan, “Vision-based hand- gesture applications,” Commun. ACM, vol. 54, no. 2, p. 60–71, Feb

  28. [28]

    Available: https://doi.org/10.1145/1897816.1897838

    [Online]. Available: https://doi.org/10.1145/1897816.1897838

    work page doi:10.1145/1897816.1897838

  29. [29]

    Robust 3d action recognition with random occupancy patterns,

    J. Wang, Z. Liu, J. Chorowski, Z. Chen, and Y . Wu, “Robust 3d action recognition with random occupancy patterns,” inComputer Vision – ECCV 2012, A. Fitzgibbon, S. Lazebnik, P. Perona, Y . Sato, and C. Schmid, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012, pp. 872–885

    2012

  30. [30]

    Real-time face gesture-based robot control using ghostnet in a unity simulation environment,

    Yaseen, “Real-time face gesture-based robot control using ghostnet in a unity simulation environment,” Sensors, vol. 25, p. 6090, 10 2025

    2025

  31. [31]

    Aerokinesis: An iot-based vision-driven gesture control system for quadcopter navigation using deep learning and ros2,

    S. Kondratev, Y . Dyrchenkova, G. Nikitin, L. V oskov, V . Pikalov, and V . Meshcheryakov, “Aerokinesis: An iot-based vision-driven gesture control system for quadcopter navigation using deep learning and ros2,” Technologies, vol. 14, no. 1, 2026. [Online]. Available: https://www.mdpi.com/2227-7080/14/1/69

    2026

  32. [32]

    Gesture based control technology for drones: Challenges and solutions,

    J. Wang, “Gesture based control technology for drones: Challenges and solutions,” Applied and Computational Engineering, vol. 109, pp. 173– 178, 11 2024

    2024

  33. [33]

    How would you gesture navigate a drone?: a user-centered approach to control a drone,

    M. Obaid, F. Kistler, G. Kasparaviciute, E. Yantac, and M. Fjeld, “How would you gesture navigate a drone?: a user-centered approach to control a drone,” 10 2016, pp. 113–121

    2016

  34. [34]

    Emotion encoding in human-drone interaction,

    J. Cauchard, K. Zhai, M. Spadafora, and J. Landay, “Emotion encoding in human-drone interaction,” 03 2016, pp. 263–270

    2016

  35. [35]

    Somatic safety: An embodied approach towards safe human-robot interaction,

    S. Benford, E. Schneiders, J. P. M. Avila, P. Caleb-Solly, P. R. Brundell, S. Castle-Green, F. Zhou, R. Garrett, K. H ¨o¨ok, S. Whatley, K. Marsh, and P. Tennent, “Somatic safety: An embodied approach towards safe human-robot interaction,” 2025. [Online]. Available: https://arxiv.org/abs/2503.16960

    Pith/arXiv arXiv 2025

  36. [36]

    A survey of methods for safe human-robot interaction,

    P. A. Lasota, T. Fong, and J. A. Shah, “A survey of methods for safe human-robot interaction,” Foundations and Trends® in Robotics, vol. 5, no. 4, pp. 261–349, 2017. [Online]. Available: https://doi.org/10.1561/2300000052

    doi:10.1561/2300000052 2017

  37. [37]

    Safety assurances for human-robot interaction via confidence-aware game- theoretic human models,

    R. Tian, L. Sun, A. Bajcsy, M. Tomizuka, and A. D. Dragan, “Safety assurances for human-robot interaction via confidence-aware game- theoretic human models,” in 2022 International Conference on Robotics and Automation (ICRA), 2022, pp. 11 229–11 235

    2022

  38. [38]

    Hazard analysis for human-on-the-loop interactions in suas systems,

    M. Vierhauser, M. N. A. Islam, A. Agrawal, J. Cleland-Huang, and J. Mason, “Hazard analysis for human-on-the-loop interactions in suas systems,” in ESEC/FSE ’21: 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Athens, Greece, August 23-28, 2021, D. Spinellis, G. Gousios, M. Chechik, and M. ...

    doi:10.1145/3468264.3468534 2021

  39. [39]

    Easy ap- proach to requirements syntax (ears),

    A. Mavin, P. Wilkinson, A. Harwood, and M. Novak, “Easy ap- proach to requirements syntax (ears),” in 2009 17th IEEE International Requirements Engineering Conference, 2009, pp. 317–322

    2009

  40. [40]

    NOMAD: A natural, occluded, multi-scale aerial dataset, for emergency response scenarios,

    A. M. R. Bernal, W. J. Scheirer, and J. Cleland-Huang, “NOMAD: A natural, occluded, multi-scale aerial dataset, for emergency response scenarios,” in IEEE/CVF Winter Conference on Applications of Computer Vision, W ACV 2024, Waikoloa, HI, USA, January 3-8, 2024. IEEE, 2024, pp. 8569–8580. [Online]. Available: https://doi.org/10.1109/W ACV57701.2024.00839

    doi:10.1109/w 2024

  41. [41]

    Validating terrain models in digital twins for trustworthy suas operations,

    A. M. R. Bernal, M. Petterson, P. A. Granadeno, M. Murphy, J. Mason, and J. Cleland-Huang, “Validating terrain models in digital twins for trustworthy suas operations,” in 28th ACM/IEEE International Conference on Model Driven Engineering Languages and Systems, MODELS 2025 - Companion, Grand Rapids, MI, USA, October 5-10, 2025. IEEE, 2025, pp. 289–299. [O...

    doi:10.1109/models-c68889.2025.00047 2025

  42. [42]

    Automated on-entry decision-making for utm zones based on reputations and certifications,

    T. P. Chambers, P. Granadeno, U. Gohar, M. C. Hunter, A. M. R. Bernal, W. Tang, M. N. A. Islam, M. Cohen, T. Jung, R. Lutz, and J. Cleland-Huang, “Automated on-entry decision-making for utm zones based on reputations and certifications,” in AIAA A VIATIONFORUM AND ASCEND 2025, 2025, p. 3567

    2025

  43. [43]

    An environmentally complex requirement for safe separation distance between uavs,

    P. A. Granadeno, A. M. R. Bernal, M. N. A. Islam, and J. Cleland- Huang, “An environmentally complex requirement for safe separation distance between uavs,” in 32nd IEEE International Requirements Engineering Conference, RE 2024 - Workshops, Reykjavik, Iceland, June 24-25, 2024. IEEE, 2024, pp. 166–175. [Online]. Available: https://doi.org/10.1109/REW6169...

    doi:10.1109/rew61692.2024.00028 2024

  44. [44]

    Momux: A catalyst for discovering situational awareness and team coordination requirements for human-autonomy teaming,

    J. Olesk, A. M. R. Bernal, and J. Cleland-Huang, “Momux: A catalyst for discovering situational awareness and team coordination requirements for human-autonomy teaming,” in IEEE International Requirements Engineering Conference (RE), Montreal, Canada, 2026

    2026

This paper was first reviewed by grok-4.5 on July 14, 2026.