Joint Action is a Framework for Understanding Partnerships Between Humans and Upper Limb Prostheses

Adam S. R. Parker; Ahmed W. Shehata; Craig S. Chapman; Heather E. Williams; Jacqueline S. Hebert; Michael R. Dawson; Patrick M. Pilarski

arxiv: 2212.14124 · v1 · submitted 2022-12-28 · 💻 cs.HC · cs.AI· cs.MA· cs.RO

Joint Action is a Framework for Understanding Partnerships Between Humans and Upper Limb Prostheses

Michael R. Dawson , Adam S. R. Parker , Heather E. Williams , Ahmed W. Shehata , Jacqueline S. Hebert , Craig S. Chapman , Patrick M. Pilarski This is my paper

Pith reviewed 2026-05-24 10:02 UTC · model grok-4.3

classification 💻 cs.HC cs.AIcs.MAcs.RO

keywords joint actionupper limb prosthesesmyoelectric controlhuman-machine interfaceshared autonomypattern recognitioncollaborative systemsadaptive switching

0 comments

The pith

Treating human-prosthesis interfaces as joint action partnerships reframes myoelectric controllers as collaborative agents rather than tools.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes that recent advances in autonomous machine learning controllers for upper limb prostheses make the traditional human-tool model insufficient. It applies the joint action framework, originally developed for coordinated human-human tasks, to evaluate three myoelectric control approaches: proportional electromyography with sequential switching, pattern recognition, and adaptive switching. By assessing each against hallmarks of joint action such as shared goals and coordinated environmental change, the work shows how the systems differ in collaborative qualities and generates recommendations centered on boosting communication between the human user and the prosthesis controller. A reader would care because this shifts design focus from isolated prediction accuracy to mutual understanding in the shared task space.

Core claim

The paper claims that modeling the human-prosthesis interface as a collaborative multi-agent system through the lens of joint action yields a new perspective on how existing myoelectric systems relate to each other, along with concrete recommendations for improvement by increasing the collaborative communication between each partner.

What carries the argument

Hallmarks of joint action drawn from human-human studies, applied as evaluation criteria to compare prosthesis controllers on dimensions of shared environmental change and coordination.

If this is right

Controllers can be ordered along a spectrum according to how many joint action hallmarks they exhibit.
Future controller designs should prioritize explicit signals that increase shared understanding between user and device.
Pattern recognition and adaptive switching already demonstrate more joint action features than basic proportional control with switching.
System improvements follow from adding mechanisms for mutual monitoring and plan adjustment rather than solely from better intent prediction.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same joint action criteria could be used to compare other shared-autonomy devices such as smart wheelchairs or collaborative robots.
If joint action scores predict user acceptance, they could serve as an early filter before full clinical trials of new controllers.
Developers might create explicit communication channels, such as the prosthesis signaling its current plan, to raise the joint action rating of existing pattern recognition systems.

Load-bearing premise

Hallmarks of joint action identified in studies of two humans can be directly and meaningfully applied to evaluate and compare human-machine systems that include autonomous controllers.

What would settle it

An experiment in which controllers scoring higher on joint action hallmarks show no measurable advantage in task completion speed, error rate, or user-reported sense of partnership compared with lower-scoring controllers.

read the original abstract

Recent advances in upper limb prostheses have led to significant improvements in the number of movements provided by the robotic limb. However, the method for controlling multiple degrees of freedom via user-generated signals remains challenging. To address this issue, various machine learning controllers have been developed to better predict movement intent. As these controllers become more intelligent and take on more autonomy in the system, the traditional approach of representing the human-machine interface as a human controlling a tool becomes limiting. One possible approach to improve the understanding of these interfaces is to model them as collaborative, multi-agent systems through the lens of joint action. The field of joint action has been commonly applied to two human partners who are trying to work jointly together to achieve a task, such as singing or moving a table together, by effecting coordinated change in their shared environment. In this work, we compare different prosthesis controllers (proportional electromyography with sequential switching, pattern recognition, and adaptive switching) in terms of how they present the hallmarks of joint action. The results of the comparison lead to a new perspective for understanding how existing myoelectric systems relate to each other, along with recommendations for how to improve these systems by increasing the collaborative communication between each partner.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper maps three myoelectric controller types onto joint-action hallmarks but offers only a conceptual comparison with no new data or tests.

read the letter

This paper applies joint action theory to upper-limb prosthesis control. It treats the human and the device as two agents that must coordinate to move objects, rather than one agent using a passive tool. The main move is to line up three existing controller families—proportional EMG with sequential switching, pattern recognition, and adaptive switching—against standard joint-action features such as shared goals, mutual prediction, and explicit communication channels. From that mapping the authors draw design suggestions for increasing the amount of information that flows both ways between user and controller. That is the only concrete output. The comparison itself is new in the prosthesis literature, and the framing is internally consistent. It gives a tidy way to talk about why pattern-recognition systems feel more “partner-like” than simple proportional ones. The limitation is that the argument never leaves the level of interpretation. No user study, no performance metric, and no falsifiable prediction is supplied to show that the joint-action lens actually improves controller design or user outcomes. The transfer of human-human hallmarks to a human-machine pair is asserted rather than demonstrated. Readers already working on shared-control or adaptive myoelectric systems will find the organizing scheme useful for discussion. Others will see a perspective piece that still needs empirical follow-up. The work is coherent enough to warrant referee time; an editor could reasonably send it out with the expectation that reviewers will ask for at least one concrete test of the proposed recommendations.

Referee Report

1 major / 1 minor

Summary. The paper claims that representing the human-prosthesis interface as a collaborative multi-agent system using the joint action framework provides a new perspective on existing myoelectric controllers. It compares proportional electromyography with sequential switching, pattern recognition, and adaptive switching controllers in terms of their alignment with hallmarks of joint action, leading to recommendations for improving collaborative communication between the human and the prosthesis.

Significance. If valid, this reframing could influence prosthesis controller design by promoting features that enhance shared intentionality and mutual adaptation rather than treating the device as a passive tool. The conceptual approach generates insights without requiring new experiments, but its significance hinges on whether the joint action lens produces actionable and testable design principles.

major comments (1)

[The comparison] The direct transfer of joint action hallmarks from human-human to human-machine contexts is assumed without discussion of potential limitations, such as differences in how autonomy and feedback are handled in prosthetic systems versus human partners. This assumption underpins the comparison and recommendations but lacks supporting argumentation or references to related human-AI joint action studies.

minor comments (1)

Consider adding a table summarizing how each controller aligns with specific hallmarks to improve clarity of the comparison.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their insightful comments. The major comment raises a valid point about the need for explicit discussion of limitations when applying the joint action framework to human-prosthesis systems. We address this below and have revised the manuscript to strengthen the argumentation.

read point-by-point responses

Referee: The direct transfer of joint action hallmarks from human-human to human-machine contexts is assumed without discussion of potential limitations, such as differences in how autonomy and feedback are handled in prosthetic systems versus human partners. This assumption underpins the comparison and recommendations but lacks supporting argumentation or references to related human-AI joint action studies.

Authors: We agree that the original manuscript would be strengthened by an explicit discussion of the assumptions and limitations involved in transferring joint action hallmarks from human-human to human-machine settings. In the revised version, we will add a new subsection (e.g., in the Introduction or a dedicated 'Framework Applicability' section) that addresses differences in autonomy (prosthetic controllers have fixed algorithmic constraints unlike human partners) and feedback (prostheses provide limited sensory channels compared to human partners). We will also incorporate references to relevant human-AI joint action literature, including work on shared autonomy in human-robot collaboration and studies examining joint intentionality in AI systems. This addition will provide the supporting argumentation and contextualize the comparison without altering the core analysis. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper applies joint-action concepts from external human-human literature as a conceptual lens to compare three existing myoelectric controller types. No equations, fitted parameters, derivations, or quantitative predictions are present. The argument consists of mapping controller behaviors onto pre-existing hallmarks drawn from cited psychology sources; this mapping does not reduce to self-definition, self-citation chains, or renaming of the paper's own inputs. The central claim therefore remains an independent reframing rather than a tautological restatement.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the transferability of joint action concepts from human-human to human-machine settings, which is postulated rather than derived from new evidence.

axioms (1)

domain assumption Hallmarks of joint action developed for human-human collaboration apply to human-prosthesis systems with autonomous controllers.
Invoked to enable the comparison of the three controllers.

pith-pipeline@v0.9.0 · 5781 in / 1128 out tokens · 21093 ms · 2026-05-24T10:02:05.306973+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

12 extracted references · 12 canonical work pages

[1]

any form of social interaction whereby two or more individuals coordinate their actions in space and time to bring about a change in the environment

Joint Action is a Framework for Understanding Partnerships Between Humans and Upper Limb Prostheses Michael R. Dawson1,2*, Adam S. R. Parker2,3, Heather E. Williams2,4, Ahmed W. Shehata1, Jacqueline S. Hebert1,4, Craig S. Chapman5, Patrick M. Pilarski1,2 1Division of Physical Medicine and Rehabilitation, Department of Medicine, University of Alberta, Edmo...

work page 2006
[2]

specifically mentioned the continuous improvement of predictions as a type of coordination smoother, which we have also included in our evaluation, due to its pertinence to the HPI setting. 3 Myoelectric Controllers Included in this Analysis The three prosthesis controllers selected for analysis under the joint action architecture are each a representativ...

work page 2004
[3]

to dynamically adapt the order of the switching list (Pilarski et al., 2012; Edwards et al., 2016a). GVF learning is a prediction approach, based on reinforcement learning methods, that can learn expected temporally extended accumulations of signals of interest based on a continuing stream of observations (Sutton et al., 2011; Pilarski et al., 2013), whic...

work page 2012
[4]

For all control schemes, the human partner was considered to reasonably demonstrate all of the hallmarks of joint action related to representations, monitoring, and predictions—humans demonstrating these hallmarks of joint action are supported by studies showing that prosthesis users adapt their internal models to take into account features of their prost...

work page doi:10.1017/9781009029049 2014
[5]

Castellini, C., Artemiadis, P., Wininger, M., Ajoudani, A., Alimusaj, M., Bicchi, A., et al

doi: 10.1038/s41598-018- 24560-3. Castellini, C., Artemiadis, P., Wininger, M., Ajoudani, A., Alimusaj, M., Bicchi, A., et al. (2014). Proceedings of the first workshop on Peripheral Machine Interfaces: going beyond traditional surface electromyography. Front. Neurorobot

work page doi:10.1038/s41598-018- 2014
[6]

Edwards, A

doi: 10.3389/fnbot.2014.00022. Edwards, A. L., Dawson, M. R., Hebert, J. S., Sherstan, C., Sutton, R. S., Chan, K. M., et al. (2016a). Application of real-time machine learning to myoelectric prosthesis control: A case series in adaptive switching. Prosthetics & Orthotics International 40, 573–581. doi: 10.1177/0309364615605373. Edwards, A. L., Hebert, J....

work page doi:10.3389/fnbot.2014.00022 2014
[7]

Powered Upper Limb Prosthetic Practice in Paediatrics,

doi: 10.3389/fnins.2019.00578. Englehart, K., and Hudgins, B. (2003). A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans. Biomed. Eng. 50, 848–854. doi: 10.1109/TBME.2003.813539. Fougner, A., Stavdahl, Ø., Kyberd, P. J., Losier, Y. G., and Parker, P. A. (2012). Control of Upper Limb Prostheses: Terminology and Proportiona...

work page doi:10.3389/fnins.2019.00578 2019
[8]

Marasco, P

doi: 10.1186/1743-0003-11-132. Marasco, P. D., Hebert, J. S., Sensinger, J. W., Shell, C. E., Schofield, J. S., Thumser, Z. C., et al. (2018). Illusory movement perception improves motor control for prosthetic hands. Sci. Transl. Med. 10, eaao6990. doi: 10.1126/scitranslmed.aao6990. Mathewson, K. W., Parker, A. S. R., Sherstan, C., Edwards, A. L., Sutton,...

work page doi:10.1186/1743-0003-11-132 2018
[9]

Pesquita, A., Whitwell, R

doi: 10.1682/JRRD.2010.08.0161. Pesquita, A., Whitwell, R. L., and Enns, J. T. (2018). Predictive joint-action model: A hierarchical predictive approach to human cooperation. Psychon Bull Rev 25, 1751–1769. doi: 10.3758/s13423-017-1393-6. 12 This is a provisional file, not the final typeset article Pilarski, P. M., Dawson, M. R., Degris, T., Carey, J. P.,...

work page doi:10.1682/jrrd.2010.08.0161 2010
[10]

Schofield, J

doi: 10.1682/JRRD.2010.09.0177. Schofield, J. S., Battraw, M. A., Parker, A. S. R., Pilarski, P. M., Sensinger, J. W., and Marasco, P. D. (2021). Embodied Cooperation to Promote Forgiving Interactions With Autonomous Machines. Front. Neurorobot. 15, 661603. doi: 10.3389/fnbot.2021.661603. Scott-Phillips, T. (2015). Speaking Our Minds: Why Human Communicat...

work page doi:10.1682/jrrd.2010.09.0177 2010
[11]

Shehata, A

doi: 10.1186/s12984-018-0417-4. Shehata, A. W., Scheme, E. J., and Sensinger, J. W. (2018b). Audible Feedback Improves Internal Model Strength and Performance of Myoelectric Prosthesis Control. Sci Rep 8,

work page doi:10.1186/s12984-018-0417-4
[12]

13 Shehata, A

doi: 10.1038/s41598-018-26810-w. 13 Shehata, A. W., Scheme, E. J., and Sensinger, J. W. (2018c). Evaluating Internal Model Strength and Performance of Myoelectric Prosthesis Control Strategies. IEEE Trans. Neural Syst. Rehabil. Eng. 26, 1046–1055. doi: 10.1109/TNSRE.2018.2826981. Shehata, A. W., Williams, H. E., Hebert, J. S., and Pilarski, P. M. (2021). ...

work page doi:10.1038/s41598-018-26810-w 2018

[1] [1]

any form of social interaction whereby two or more individuals coordinate their actions in space and time to bring about a change in the environment

Joint Action is a Framework for Understanding Partnerships Between Humans and Upper Limb Prostheses Michael R. Dawson1,2*, Adam S. R. Parker2,3, Heather E. Williams2,4, Ahmed W. Shehata1, Jacqueline S. Hebert1,4, Craig S. Chapman5, Patrick M. Pilarski1,2 1Division of Physical Medicine and Rehabilitation, Department of Medicine, University of Alberta, Edmo...

work page 2006

[2] [2]

specifically mentioned the continuous improvement of predictions as a type of coordination smoother, which we have also included in our evaluation, due to its pertinence to the HPI setting. 3 Myoelectric Controllers Included in this Analysis The three prosthesis controllers selected for analysis under the joint action architecture are each a representativ...

work page 2004

[3] [3]

to dynamically adapt the order of the switching list (Pilarski et al., 2012; Edwards et al., 2016a). GVF learning is a prediction approach, based on reinforcement learning methods, that can learn expected temporally extended accumulations of signals of interest based on a continuing stream of observations (Sutton et al., 2011; Pilarski et al., 2013), whic...

work page 2012

[4] [4]

For all control schemes, the human partner was considered to reasonably demonstrate all of the hallmarks of joint action related to representations, monitoring, and predictions—humans demonstrating these hallmarks of joint action are supported by studies showing that prosthesis users adapt their internal models to take into account features of their prost...

work page doi:10.1017/9781009029049 2014

[5] [5]

Castellini, C., Artemiadis, P., Wininger, M., Ajoudani, A., Alimusaj, M., Bicchi, A., et al

doi: 10.1038/s41598-018- 24560-3. Castellini, C., Artemiadis, P., Wininger, M., Ajoudani, A., Alimusaj, M., Bicchi, A., et al. (2014). Proceedings of the first workshop on Peripheral Machine Interfaces: going beyond traditional surface electromyography. Front. Neurorobot

work page doi:10.1038/s41598-018- 2014

[6] [6]

Edwards, A

doi: 10.3389/fnbot.2014.00022. Edwards, A. L., Dawson, M. R., Hebert, J. S., Sherstan, C., Sutton, R. S., Chan, K. M., et al. (2016a). Application of real-time machine learning to myoelectric prosthesis control: A case series in adaptive switching. Prosthetics & Orthotics International 40, 573–581. doi: 10.1177/0309364615605373. Edwards, A. L., Hebert, J....

work page doi:10.3389/fnbot.2014.00022 2014

[7] [7]

Powered Upper Limb Prosthetic Practice in Paediatrics,

doi: 10.3389/fnins.2019.00578. Englehart, K., and Hudgins, B. (2003). A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans. Biomed. Eng. 50, 848–854. doi: 10.1109/TBME.2003.813539. Fougner, A., Stavdahl, Ø., Kyberd, P. J., Losier, Y. G., and Parker, P. A. (2012). Control of Upper Limb Prostheses: Terminology and Proportiona...

work page doi:10.3389/fnins.2019.00578 2019

[8] [8]

Marasco, P

doi: 10.1186/1743-0003-11-132. Marasco, P. D., Hebert, J. S., Sensinger, J. W., Shell, C. E., Schofield, J. S., Thumser, Z. C., et al. (2018). Illusory movement perception improves motor control for prosthetic hands. Sci. Transl. Med. 10, eaao6990. doi: 10.1126/scitranslmed.aao6990. Mathewson, K. W., Parker, A. S. R., Sherstan, C., Edwards, A. L., Sutton,...

work page doi:10.1186/1743-0003-11-132 2018

[9] [9]

Pesquita, A., Whitwell, R

doi: 10.1682/JRRD.2010.08.0161. Pesquita, A., Whitwell, R. L., and Enns, J. T. (2018). Predictive joint-action model: A hierarchical predictive approach to human cooperation. Psychon Bull Rev 25, 1751–1769. doi: 10.3758/s13423-017-1393-6. 12 This is a provisional file, not the final typeset article Pilarski, P. M., Dawson, M. R., Degris, T., Carey, J. P.,...

work page doi:10.1682/jrrd.2010.08.0161 2010

[10] [10]

Schofield, J

doi: 10.1682/JRRD.2010.09.0177. Schofield, J. S., Battraw, M. A., Parker, A. S. R., Pilarski, P. M., Sensinger, J. W., and Marasco, P. D. (2021). Embodied Cooperation to Promote Forgiving Interactions With Autonomous Machines. Front. Neurorobot. 15, 661603. doi: 10.3389/fnbot.2021.661603. Scott-Phillips, T. (2015). Speaking Our Minds: Why Human Communicat...

work page doi:10.1682/jrrd.2010.09.0177 2010

[11] [11]

Shehata, A

doi: 10.1186/s12984-018-0417-4. Shehata, A. W., Scheme, E. J., and Sensinger, J. W. (2018b). Audible Feedback Improves Internal Model Strength and Performance of Myoelectric Prosthesis Control. Sci Rep 8,

work page doi:10.1186/s12984-018-0417-4

[12] [12]

13 Shehata, A

doi: 10.1038/s41598-018-26810-w. 13 Shehata, A. W., Scheme, E. J., and Sensinger, J. W. (2018c). Evaluating Internal Model Strength and Performance of Myoelectric Prosthesis Control Strategies. IEEE Trans. Neural Syst. Rehabil. Eng. 26, 1046–1055. doi: 10.1109/TNSRE.2018.2826981. Shehata, A. W., Williams, H. E., Hebert, J. S., and Pilarski, P. M. (2021). ...

work page doi:10.1038/s41598-018-26810-w 2018