Pith. sign in

REVIEW 3 major objections 2 minor 24 references

Co-located mixed-reality workspaces improve perceived collaboration and handoff clarity in multi-operator robot control.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-27 22:05 UTC pith:QT62OM4L

load-bearing objection Extends HORUS to two-operator MR control with a study favoring co-located workspaces on subjective measures, but lacks questionnaire validation and stats. the 3 major comments →

arxiv 2606.07013 v1 pith:QT62OM4L submitted 2026-06-05 cs.RO cs.HC

A Multi-Operator Mixed-Reality Interface for Multi-Robot Control and Coordination: Co-Located and Private Workspace Collaboration

Omotoye Shamsudeen Adekoya , Antonio Sgorbissa , Carmine Tommaso Recchiuto This is my paper
classification cs.RO cs.HC
keywords mixed-reality interfacemulti-operator controlmulti-robot coordinationshared workspacehuman-robot interactioncollaborative roboticsworkspace modes
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper extends a mixed-reality interface from single-operator to multi-operator use for controlling robot teams. It implements two modes: a shared physical workspace where operators manipulate the same mini-map together, and independent private workspaces. In a human-subject study, 18 pairs used the system to direct three mobile robots through search tasks. Objective mission performance stayed comparable between modes, yet the co-located shared workspace produced higher ratings for collaboration quality, mutual understanding, and handoff clarity, and participants chose it as the preferred setup.

Core claim

The architecture combines registration-driven scene construction, lightweight shared-session synchronization, and per-robot control leases to enable collaborative monitoring, tasking, and teleoperation while blocking conflicting commands. Across two search environments the objective task results did not differ by mode, but the co-located shared workspace produced statistically higher scores on perceived collaboration, shared understanding, and handoff clarity and was selected as the preferred collaborative mode.

What carries the argument

Two complementary mixed-reality workspace modes (co-located shared mini-map versus independent private workspaces) plus per-robot control leases that serialize commands and prevent simultaneous conflicting interventions.

Load-bearing premise

Subjective questionnaire scores validly measure collaboration quality and the results from 18 pairs with three Nova Carter robots in search tasks extend to other multi-robot missions and operator groups.

What would settle it

A follow-up experiment with different robots, tasks, or participant pools that finds no reliable difference in subjective collaboration or handoff ratings between the two workspace modes.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Both workspace modes support effective execution of multi-robot search missions.
  • Physically co-locating the workspace raises operators' sense of shared understanding and reduces handoff friction.
  • Control leases successfully eliminate conflicting commands without slowing mission progress.
  • Operators consistently prefer the co-located mode when given the choice.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Designers of future multi-operator interfaces may prioritize physical proximity of workspaces even when the underlying control tools stay identical.
  • The same lease mechanism could be tested in domains with higher command frequency, such as simultaneous teleoperation of multiple arms.
  • Combining the two modes dynamically within one session might let teams switch based on task phase.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The manuscript extends the HORUS mixed-reality interface to support collaborative multi-operator control of multi-robot teams via two modes: a co-located shared workspace and a private-workspace mode. The architecture uses registration-driven scene construction, lightweight shared-session synchronization, and per-robot control leases to enable monitoring, tasking, and teleoperation while avoiding command conflicts. A human-subject study with 36 participants (18 pairs) controlling three Nova Carter robots across two search environments reports comparable objective task performance between modes but significantly higher ratings for perceived collaboration, shared understanding, and handoff clarity in the co-located mode, which was also the preferred mode.

Significance. If the results hold after fuller reporting, the work supplies empirical evidence that physical co-location of a shared mixed-reality workspace can improve subjective coordination metrics in multi-robot supervision without degrading objective performance. This has design implications for multi-operator interfaces in search, inspection, or disaster-response scenarios. The paired-operator protocol and dual objective/subjective measurement approach are methodological strengths.

major comments (3)
  1. [Evaluation] Evaluation section: The central claim that the co-located mode produced significantly better perceived collaboration, shared understanding, and handoff clarity rests on subjective questionnaires whose construction, pilot validation, reliability (e.g., Cronbach’s alpha), and correlation with objective proxies (command conflicts, handoff latency) are not reported. Without these, demand characteristics or task-specific factors cannot be ruled out.
  2. [Methods / Results] Methods / Results: No statistical details (exact tests, p-values, effect sizes, power analysis, or multiple-comparison corrections) are supplied for the 18-pair sample to support the reported significance of subjective gains. The abstract states “significantly improved” but the manuscript must demonstrate that the 36-participant design has adequate power for the observed effects.
  3. [Discussion] Discussion: The generalization claim—that results from two search environments with Nova Carter robots support preference for co-located mode across multi-robot tasks and operator populations—is asserted without additional validation tasks, different robot platforms, or cross-validation with other coordination metrics.
minor comments (2)
  1. [Abstract] Abstract: The number of robots (three) and the two search environments are mentioned but their distinguishing features (size, complexity, sensor coverage) are not summarized, making it harder to assess task demands.
  2. [Introduction / System Description] Notation: The terms “registration-driven scene construction” and “per-robot control leases” are introduced without a brief definition or reference to the prior HORUS paper on first use.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive feedback and the recommendation for major revision. We address each major comment below, indicating where revisions will be made to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Evaluation] Evaluation section: The central claim that the co-located mode produced significantly better perceived collaboration, shared understanding, and handoff clarity rests on subjective questionnaires whose construction, pilot validation, reliability (e.g., Cronbach’s alpha), and correlation with objective proxies (command conflicts, handoff latency) are not reported. Without these, demand characteristics or task-specific factors cannot be ruled out.

    Authors: We agree that the manuscript would benefit from greater transparency on the subjective measures. The questionnaires were derived from prior collaboration and shared-awareness instruments in the HRI literature, but details on item construction, any pilot validation, and reliability coefficients were omitted. In the revision we will append the complete questionnaire items, describe pilot testing if performed, report Cronbach’s alpha for each subscale, and explicitly discuss demand characteristics as a potential limitation. Correlations between subjective scores and objective proxies (command conflicts, handoff latency) were not computed in the original study; we will note this as a limitation and an avenue for future analysis rather than claiming such validation exists. revision: yes

  2. Referee: [Methods / Results] Methods / Results: No statistical details (exact tests, p-values, effect sizes, power analysis, or multiple-comparison corrections) are supplied for the 18-pair sample to support the reported significance of subjective gains. The abstract states “significantly improved” but the manuscript must demonstrate that the 36-participant design has adequate power for the observed effects.

    Authors: We will expand the Methods and Results sections to include the precise statistical procedures (e.g., paired t-tests or Wilcoxon signed-rank tests), exact p-values, effect sizes (Cohen’s d), any multiple-comparison corrections applied, and a post-hoc power analysis for the 18-pair sample. The abstract phrasing will be aligned with these reported statistics. Because the raw data remain available, we can compute and present the required power figures in the revision. revision: yes

  3. Referee: [Discussion] Discussion: The generalization claim—that results from two search environments with Nova Carter robots support preference for co-located mode across multi-robot tasks and operator populations—is asserted without additional validation tasks, different robot platforms, or cross-validation with other coordination metrics.

    Authors: We accept that the current wording overstates generalizability. In the revised Discussion we will qualify the claims to reflect the specific conditions tested (two search environments, Nova Carter platforms, 36 participants) and will explicitly frame broader applicability as a hypothesis for future work rather than an established result. No additional validation data exist in the present study, so the revision will remove or soften the overgeneralized statements. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical human-subject evaluation with independent measurements

full rationale

The paper describes a mixed-reality interface extension of prior HORUS work and reports results from a 36-participant (18-pair) user study measuring objective task performance and subjective collaboration metrics across two workspace modes. No equations, fitted parameters, predictions, or derivations appear in the provided text. The central claims rest on direct experimental outcomes (comparable objective performance, significant subjective differences favoring co-located mode) rather than any self-referential reduction, self-citation chain, or ansatz. Self-reference to prior HORUS work is background context only and does not support the evaluation results. This matches the default non-circular case for empirical systems papers.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No free parameters, axioms, or invented entities; the paper is an applied systems and empirical user-study contribution with no mathematical modeling or new postulated constructs.

pith-pipeline@v0.9.1-grok · 5778 in / 1081 out tokens · 26660 ms · 2026-06-27T22:05:04.938637+00:00 · methodology

0 comments
read the original abstract

Multi-operator control of robot teams requires not only access to the same mission information, but also mechanisms for maintaining shared awareness and preventing conflicting interventions. Building on our previous HORUS interface (Holistic Operational Reality for Unified Systems) we present a mixed-reality interface that extends single-operator multi-robot supervision to collaborative multi-operator use. The system supports two complementary modes: a co-located shared workspace, in which operators observe and manipulate the same mini-map in the same physical location, and a private-workspace mode, in which operators work on the same mission through independently placed local workspaces. The architecture combines registration-driven scene construction, lightweight shared-session synchronization, and per-robot control leases to support collaborative monitoring, tasking, and teleoperation while preventing conflicting commands. We evaluated the approach in a human-subject study with 36 participants (18 pairs) controlling three Nova Carter mobile robots in two search environments. The performance of the objective task was comparable across the two modes, indicating that both modes supported effective mission execution. However, the co-located shared workspace significantly improved perceived collaboration, shared understanding, and handoff clarity, and was the preferred collaborative mode. These results indicate that physically co-locating the MR workspace improves how operators coordinate even when the underlying robot-control tools remain unchanged.

Figures

Figures reproduced from arXiv: 2606.07013 by Antonio Sgorbissa, Carmine Tommaso Recchiuto, Omotoye Shamsudeen Adekoya.

Figure 1
Figure 1. Figure 1: HORUS in co-located shared-workspace mode. 3 operators observe [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: System Architecture Diagram [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Workspace-centric interaction in HORUS. The operational [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Experiment environment. where 1 is the hospital environment and [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Composite questionnaire outcomes by condition. The co-located and [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 5
Figure 5. Figure 5: Objective task performance. Left: pair-level AprilTag detections in [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 2 canonical work pages · 1 internal anchor

  1. [1]

    Supervisory con- trol of multiple robots: Human-performance issues and user-interface design,

    J. Y . C. Chen, M. J. Barnes, and M. Harper-Sciarini, “Supervisory con- trol of multiple robots: Human-performance issues and user-interface design,”IEEE Transactions on Systems, Man, and Cybernetics Part C, vol. 41, no. 4, pp. 435–454, 2011

    2011

  2. [2]

    Human interaction with multiple remote robots,

    M. Lewis, “Human interaction with multiple remote robots,”Reviews of Human Factors and Ergonomics, vol. 9, no. 1, pp. 131–174, 2013

    2013

  3. [3]

    Human interaction with robot swarms: A survey,

    A. Kolling, P. Walker, N. Chakraborty, K. Sycara, and M. Lewis, “Human interaction with robot swarms: A survey,”IEEE Transactions on Human-Machine Systems, vol. 46, no. 1, pp. 9–26, 2016

    2016

  4. [4]

    Multi-robot interfaces and operator situational awareness: Study of the impact of immersion and prediction,

    J. J. Rold ´an, E. Pe ˜na Tapia, A. Mart ´ın-Barrio, M. A. Olivares- M´endez, J. Del Cerro, and A. Barrientos, “Multi-robot interfaces and operator situational awareness: Study of the impact of immersion and prediction,”Sensors, vol. 17, no. 8, p. 1720, 2017

    2017

  5. [5]

    Toward mobile mixed-reality interaction with multi-robot systems,

    J. A. Frank, S. P. Krishnamoorthy, and V . Kapila, “Toward mobile mixed-reality interaction with multi-robot systems,”IEEE Robotics and Automation Letters, vol. 2, no. 4, pp. 1901–1908, 2017

    1901

  6. [6]

    Aug- mented reality and robotics: A survey and taxonomy for ar-enhanced human-robot interaction and robotic interfaces,

    R. Suzuki, A. Karim, T. Xia, H. Hedayati, and N. Marquardt, “Aug- mented reality and robotics: A survey and taxonomy for ar-enhanced human-robot interaction and robotic interfaces,” inProceedings of the 2022 CHI Conference on Human Factors in Computing Systems. ACM, 2022, pp. 1–33

    2022

  7. [7]

    Interacting with multi-robot systems via mixed reality,

    F. Kennel-Maushart, R. Poranne, and S. Coros, “Interacting with multi-robot systems via mixed reality,” inProceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 11 633–11 639

    2023

  8. [8]

    A 3d mixed reality interface for human-robot teaming,

    J. Chen, B. Sun, M. Pollefeys, and H. Blum, “A 3d mixed reality interface for human-robot teaming,” in2024 IEEE International Con- ference on Robotics and Automation (ICRA). IEEE, 2024, pp. 11 327– 11 333

    2024

  9. [9]

    HORUS: A Mixed Reality Interface for Managing Teams of Mobile Robots

    O. S. Adekoya, A. Sgorbissa, and C. T. Recchiuto, “Horus: A mixed reality interface for managing teams of mobile robots,” arXiv preprint, 2025, arXiv:2506.02622. [Online]. Available: https://arxiv.org/abs/2506.02622

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  10. [10]

    Designing interfaces for multi-user, multi- robot systems,

    A. Rule and J. Forlizzi, “Designing interfaces for multi-user, multi- robot systems,” inProceedings of the ACM/IEEE International Con- ference on Human-Robot Interaction (HRI), 2012, pp. 97–104

    2012

  11. [11]

    Control sharing in human-robot team interaction,

    S. Musi ´c and S. Hirche, “Control sharing in human-robot team interaction,”Annual Reviews in Control, 2017

    2017

  12. [12]

    Sharing the control of robot swarms among multiple human operators: A user study,

    G. Miyauchi, Y . K. Lopes, and R. Groß, “Sharing the control of robot swarms among multiple human operators: A user study,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2023, pp. 8847–8853

    2023

  13. [13]

    A quest for co-located mixed reality: Aligning and assessing slam tracking for same-space multi-user experiences,

    M. McGillet al., “A quest for co-located mixed reality: Aligning and assessing slam tracking for same-space multi-user experiences,” inProceedings of ACM VRST, 2020

    2020

  14. [14]

    Multi-user augmented reality with communication efficient and spatially consistent virtual objects,

    X. Ranet al., “Multi-user augmented reality with communication efficient and spatially consistent virtual objects,” inProceedings of ACM CoNEXT, 2020

    2020

  15. [15]

    Slam-share: Visual simultaneous localization and mapping for real- time multi-user augmented reality,

    A. Dhakal, X. Ran, Y . Wang, J. Chen, and K. K. Ramakrishnan, “Slam-share: Visual simultaneous localization and mapping for real- time multi-user augmented reality,” inProceedings of ACM CoNEXT, 2022, pp. 293–306

    2022

  16. [16]

    Dynamic authority distribution for cooperative tele- operation,

    E. Noohiet al., “Dynamic authority distribution for cooperative tele- operation,” inProceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015

    2015

  17. [17]

    A dual-user teleoperation system with online authority adjustment for haptic training,

    F. Liuet al., “A dual-user teleoperation system with online authority adjustment for haptic training,” inProceedings of the IEEE Engineer- ing in Medicine and Biology Society (EMBC), 2015

    2015

  18. [18]

    Enhanced transparency dual-user shared control teleoper- ation with multiple adaptive dominance factors,

    Z. Luet al., “Enhanced transparency dual-user shared control teleoper- ation with multiple adaptive dominance factors,”International Journal of Control, Automation and Systems, 2017

    2017

  19. [19]

    Transiting Exoplanet Survey Satellite (TESS) flight dynamics commissioning results and experiences,

    L. Fernet al., “Multi-operator multi-uav (momu) con- trol,” NASA, Tech. Rep., 2018. [Online]. Available: https://ntrs.nasa.gov/citations/20180003961

    work page arXiv 2018

  20. [20]

    Augmented reality user inter- faces for heterogeneous multirobot control,

    R. Chac ´on-Quesada and Y . Demiris, “Augmented reality user inter- faces for heterogeneous multirobot control,” inProceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, pp. 11 439–11 444

    2020

  21. [21]

    Spatial anchor based indoor asset tracking,

    W. He, M. Xi, H. Gardner, B. Swift, and M. Adcock, “Spatial anchor based indoor asset tracking,” inProceedings of IEEE VR, 2021, pp. 255–259

    2021

  22. [22]

    Off-cloud anchor sharing framework for multi- user and multi-platform mixed reality applications,

    A. Vidal-Balea, O. Blanco-Novoa, P. Fraga-Lamas, and T. M. Fern´andez-Caram´es, “Off-cloud anchor sharing framework for multi- user and multi-platform mixed reality applications,”Applied Sciences, vol. 15, no. 13, p. 6959, 2025

    2025

  23. [23]

    Internal consistency and reliability of the networked minds measure of social presence,

    C. Harms and F. Biocca, “Internal consistency and reliability of the networked minds measure of social presence,” inProceedings of the Seventh Annual International Workshop on Presence, 2004

    2004

  24. [24]

    The five-factor perceived shared mental model scale: A consolidation of items across the contemporary literature,

    J. J. van Rensburg, C. Marques Santos, S. B. de Jong, and S. Uit- dewilligen, “The five-factor perceived shared mental model scale: A consolidation of items across the contemporary literature,”Frontiers in Psychology, vol. 12, 2022

    2022