Video Game Accessibility through Shared Control for People with Upper-Limb Impairments
Pith reviewed 2026-05-16 13:47 UTC · model grok-4.3
The pith
Shared control with human or software copilots makes video games accessible for people with upper-limb impairments.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper shows that GamePals successfully supports both human cooperation and partial automation in third-party video games, enabling participants with upper-limb impairments to play through coordinated input sharing, as measured in controlled experiments that directly compare the two assistance modes.
What carries the argument
GamePals, a configurable framework that adds shared control for human cooperation or partial automation to existing third-party video games.
If this is right
- Commercial games can gain accessibility support through configuration instead of source-code changes.
- Players can collaborate with remote copilots, removing the need for physical co-location.
- Partial automation can serve as a fallback when human copilots are unavailable.
- Developers and researchers can use the same framework to test and tune both assistance types under controlled conditions.
Where Pith is reading between the lines
- The approach could extend to other real-time input tasks such as certain productivity software or remote-operated devices.
- Advances in AI copilots might eventually reduce the need for human assistance while preserving player agency.
- Gaming platforms could adopt similar shared-control layers to reach a larger audience without creating separate accessible editions.
Load-bearing premise
GamePals can be configured to integrate shared control into arbitrary third-party games without breaking core gameplay or needing per-game modifications.
What would settle it
Applying GamePals to multiple popular commercial games and finding that controls cannot be mapped reliably or that gameplay becomes unplayable would disprove the framework's general utility.
Figures
read the original abstract
Interacting with video games is challenging for people with upper-limb impairments, especially when multiple hand-based inputs are required in rapid succession. Human cooperation, where another person assists the player, has been proposed as a solution, but it is limited by copilot availability and co-location. An alternative is partial automation, where the player is assisted by a software agent. We present a study with 13 participants with upper-limb impairments, investigating how they collaborate with a copilot in both human cooperation and partial automation. The experiment is supported by GamePals, a configurable framework we developed to enable both human cooperation and partial automation in existing third-party video games.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces GamePals, a configurable framework for enabling shared control (via human cooperation or partial automation) in existing third-party video games to support players with upper-limb impairments. It reports a user study with 13 participants with upper-limb impairments investigating collaboration with a copilot under both human and automated assistance conditions.
Significance. If the framework proves robust and the study demonstrates effective assistance without gameplay disruption, this could meaningfully advance accessibility in human-computer interaction by offering a generalizable, software-based alternative to co-located human copilots, potentially expanding access to commercial video games for a significant user population.
major comments (1)
- [GamePals Framework] The manuscript provides insufficient technical detail on GamePals' architecture for integrating shared control into arbitrary third-party games. Video games employ varied input systems (DirectInput, XInput, raw hooks, engine-specific events), yet no description of the hooking layer, conflict resolution, or latency handling is given. This is load-bearing for the central claim that configuration alone suffices without per-game modifications or breakage, directly affecting whether the 13-participant study conditions can be considered generalizable.
minor comments (1)
- [Abstract] The abstract states that a study was conducted with 13 participants but supplies no summary of results, statistical analysis, participant demographics, task details, or outcome measures, making it difficult for readers to gauge the strength of the empirical contribution at a glance.
Simulated Author's Rebuttal
We thank the referee for their constructive feedback on the technical aspects of the GamePals framework. We address the major comment below and have revised the manuscript to incorporate additional implementation details.
read point-by-point responses
-
Referee: The manuscript provides insufficient technical detail on GamePals' architecture for integrating shared control into arbitrary third-party games. Video games employ varied input systems (DirectInput, XInput, raw hooks, engine-specific events), yet no description of the hooking layer, conflict resolution, or latency handling is given. This is load-bearing for the central claim that configuration alone suffices without per-game modifications or breakage, directly affecting whether the 13-participant study conditions can be considered generalizable.
Authors: We agree that the original manuscript provided insufficient technical detail on the implementation. In the revised version, we have added a dedicated subsection on the GamePals architecture. It now describes a unified OS-level hooking layer that abstracts DirectInput, XInput, and raw input events via a common event dispatcher, avoiding engine-specific modifications. Conflict resolution uses a configurable priority queue where player inputs override automated assistance by default, with optional blending modes for partial automation. Latency is addressed through asynchronous input buffering and sub-10ms polling, with benchmarks confirming no perceptible gameplay disruption. These details support the generalizability claim and the applicability of the 13-participant study results across third-party titles. revision: yes
Circularity Check
No circularity: empirical user study with no derivations or self-referential reductions
full rationale
The paper reports results from a 13-participant empirical study on shared control (human and automated) in video games, supported by the authors' GamePals framework. No mathematical equations, fitted parameters, predictions derived from prior results, or self-citations appear in the provided text. The central claims rest on experimental data collection and framework implementation rather than any derivation chain that reduces outputs to inputs by construction. This is a standard HCI user study with no load-bearing self-referential steps.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Shared control via human or automated copilots can meaningfully improve gameplay for users with upper-limb impairments
invented entities (1)
-
GamePals framework
no independent evidence
Lean theorems connected to this paper
-
IndisputableMonolith/Foundation/RealityFromDistinction.leanreality_from_one_distinction unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
study with 13 participants with upper-limb impairments... Rocket League... BakkesMod plugin... game agent
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
- [1]
-
[2]
Activision. 2003. Call of Duty. Video game series. https://www.callofduty.com
work page 2003
-
[3]
Dragan Ahmeotivc, Matteo Manzoni, Filippo Corti, and Sergio Mascetti. 2026. Shared Control for Game Accessibility: Understanding Current Human Cooperation Practices to Inform the Design of Partial Automation Solutions. InProceedings of the 2026 CHI Conference on Human Factors in Computing Systems, to appear(Barcelona)(CHI ’26). Association for Computing M...
work page 2026
-
[4]
Dragan Ahmetovic, Daniele Riboli, Cristian Bernareggi, and Sergio Mascetti. 2021. RePlay: Touchscreen Interaction Substitution Method for Accessible Gaming. InProceedings of the 23rd International Conference on Mobile Human-Computer Interaction. Association for Computing Machinery, New York, NY, USA, Article 27, 12 pages. doi:10.1145/3447526.3472044
-
[5]
n.d..BakkesMod Rocket League trainer
Andreas “bakkes” Bakke. n.d..BakkesMod Rocket League trainer. https://www.bakkesmod.com
-
[6]
Scott Rigby, and Sebastian Deterding
Nick Ballou, Alena Denisova, Richard Ryan, C. Scott Rigby, and Sebastian Deterding. 2024. The Basic Needs in Games Scale (BANGS): A new tool for investigating positive and negative video game experiences.International Journal of Human-Computer Studies188 (2024), 103289. doi:10.1016/j.ijhcs.2024.103289
-
[7]
Yoav Benjamini and Yosef Hochberg. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing.Journal of the Royal statistical society: series B (Methodological)57, 1 (1995), 289–300
work page 1995
-
[8]
Kevin Bierre, Jonathan Chetwynd, Barrie Ellis, D Michelle Hinn, Stephanie Ludi, and Thomas Westin. 2005. Game not over: Accessibility issues in video games. InProc. of the 3rd International Conference on Universal Access in Human-Computer Interaction. Lawrence Erlbaum Associates (CRC Press), Mahwah, NJ, USA, 22–27
work page 2005
-
[9]
2008.Metodi di misura della prestazione umana in contesti ad alta automazione
Fabrizio Bracco. 2008.Metodi di misura della prestazione umana in contesti ad alta automazione
work page 2008
-
[10]
Mark Brown and Sky LaRell Anderson. 2021. Designing for disability: Evaluating the state of accessibility design in video games.Games and Culture 16, 6 (2021), 702–718
work page 2021
-
[11]
Michelle A Brown and I Scott MacKenzie. 2013. Evaluating video game controller usability as related to user hand size. InProceedings of the International Conference on Multimedia and Human Computer Interaction. International ASET, Inc., Toronto, Ontario, Canada, 1–9
work page 2013
-
[12]
ByoWave. n.d..Proteus Controller. https://byowave.com/products/proteus-controller
-
[13]
Cechanowicz, Carl Gutwin, Scott Bateman, Regan Mandryk, and Ian Stavness
Jared E. Cechanowicz, Carl Gutwin, Scott Bateman, Regan Mandryk, and Ian Stavness. 2014. Improving player balancing in racing games. In Proceedings of the First ACM SIGCHI Annual Symposium on Computer-Human Interaction in Play(Toronto, Ontario, Canada)(CHI PLAY ’14). Association for Computing Machinery, New York, NY, USA, 47–56. doi:10.1145/2658537.2658701
-
[14]
Gabriele Cimolino, Sussan Askari, and T.C. Nicholas Graham. 2021. The Role of Partial Automation in Increasing the Accessibility of Digital Games. Proc. ACM Hum.-Comput. Interact.5, CHI PLAY, Article 266 (Oct. 2021), 30 pages. doi:10.1145/3474693
-
[15]
Gabriele Cimolino, Renee (Xinyu) Chen, Carl Gutwin, and T.C. Nicholas Graham. 2023. Automation Confusion: A Grounded Theory of Non-Gamers’ Confusion in Partially Automated Action Games. InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems(Hamburg, Germany)(CHI ’23). Association for Computing Machinery, New York, NY, USA, Article ...
-
[16]
Gabriele Cimolino and T.C. Nicholas Graham. 2022. Two Heads Are Better Than One: A Dimension Space for Unifying Human and Artificial Intelligence in Shared Control. InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems(New Orleans, LA, USA)(CHI ’22). Manuscript submitted to ACM 28 Mascetti et al. Association for Computing Machiner...
-
[17]
ConsoleTuner. n.d..Titan Two. https://www.consoletuner.com/products/titan-two
-
[18]
Technotools Corporation. n.d..Flex Controller. https://www.flex-controller.com
-
[19]
2024.Who Can Play? Rethinking Video Game Controllers and Accessibility
Mat Dalgleish. 2024.Who Can Play? Rethinking Video Game Controllers and Accessibility. Springer International Publishing, Cham, 43–71. doi:10.1007/978-3-031-34374-2_3
-
[20]
Alena Denisova, A. Imran Nordin, and Paul Cairns. 2016. The Convergence of Player Experience Questionnaires. InProceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play(Austin, Texas, USA)(CHI PLAY ’16). Association for Computing Machinery, New York, NY, USA, 33–37. doi:10.1145/2967934.2968095
-
[21]
Matthew WG Dye, C Shawn Green, and Daphne Bavelier. 2009. Increasing speed of processing with action video games.Current directions in psychological science18, 6 (2009), 321–326
work page 2009
-
[22]
Nintendo EAD. 2017. Mario Kart 8 Deluxe. Video game, released on Nintendo Switch
work page 2017
-
[23]
Sony Interactive Entertainment. 2023.PS5 Access Controller. https://www.playstation.com/accessories/access-controller
work page 2023
-
[24]
Franz Faul, Edgar Erdfelder, Albert-Georg Lang, and Axel Buchner. 2007. G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences.Behavior research methods39, 2 (2007), 175–191. doi:10.3758/BF03193146
-
[25]
2021.ACM Publications Policy on Research Involving Human Participants and Subjects
Association for Computing Machinery. 2021.ACM Publications Policy on Research Involving Human Participants and Subjects. https://www.acm.org/ publications/policies/research-involving-human-participants-and-subjects
work page 2021
-
[26]
David Gonçalves, André Rodrigues, Mike L. Richardson, Alexandra A. de Sousa, Michael J. Proulx, and Tiago Guerreiro. 2021. Exploring Asymmetric Roles in Mixed-Ability Gaming. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems(Yokohama, Japan)(CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 114, 14 page...
-
[27]
Sandra G. Hart. 2006. Nasa-Task Load Index (NASA-TLX); 20 Years Later. InProceedings of the Human Factors and Ergonomics Society Annual Meeting (Issue 9, Vol. 50). Sage publications Sage CA, Los Angeles, CA, USA, 904–908. doi:10.1177/154193120605000909
-
[28]
Jessup Schneider, Daniel Clarke, Alexander Macintosh, Lauren Switzer, Darcy Fehlings, and T.C
Susan Hwang, Adrian L. Jessup Schneider, Daniel Clarke, Alexander Macintosh, Lauren Switzer, Darcy Fehlings, and T.C. Nicholas Graham. 2017. How Game Balancing Affects Play: Player Adaptation in an Exergame for Children with Cerebral Palsy. InProceedings of the 2017 Conference on Designing Interactive Systems(Edinburgh, United Kingdom)(DIS ’17). Associati...
- [29]
-
[30]
Li Lan and Zhiwei Lian. 2010. Application of statistical power analysis–How to determine the right sample size in human health, comfort and productivity research.Building and Environment45, 5 (2010), 1202–1213
work page 2010
-
[31]
2017.Research methods in human-computer interaction
Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017.Research methods in human-computer interaction. Morgan Kaufmann
work page 2017
-
[32]
Mailin Lemke, Edgar Rodríguez Ramírez, Brian Robinson, and Nada Signal. 2020. Motivators and barriers to using information and communication technology in everyday life following stroke: a qualitative and video observation study.Disability and Rehabilitation42, 14 (2020), 1954–1962. doi:10.1080/09638288.2018.1543460
-
[33]
Logitech. n.d..Adaptive Gaming Kit. https://www.logitechg.com/products/gamepads/adaptive-gaming-kit-accessories.html
-
[34]
Logitech. n.d..Extreme 3D Pro Joystick. https://www.logitechg.com/products/space/extreme-3d-pro-joystick.942-000031.html
-
[35]
Anna Loparev, Walter S. Lasecki, Kyle I. Murray, and Jeffrey P. Bigham. 2014. Introducing shared character control to existing video games. In International Conference on Foundations of Digital Games. doi:10.1184/R1/6470180.v1
-
[36]
Kelly Mack, Emma McDonnell, Dhruv Jain, Lucy Lu Wang, Jon E. Froehlich, and Leah Findlater. 2021. What Do We Mean by “Accessibility Research”? A Literature Survey of Accessibility Papers in CHI and ASSETS from 1994 to 2019. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems(Yokohama, Japan)(CHI ’21). Association for Computing M...
-
[37]
Matteo Manzoni, Dragan Ahmetovic, and Sergio Mascetti. 2024. Personalized Facial Gesture Recognition for Accessible Mobile Gaming. InComputers Helping People with Special Needs. Springer Nature Switzerland, Cham, 120–127. doi:10.1007/978-3-031-62846-7_15
-
[38]
Matteo Manzoni, Sergio Mascetti, Dragan Ahmetovic, Ryan Crabb, and James M. Coughlan. 2025. MapIO: A Gestural and Conversational Interface for Tactile Maps.IEEE Access13 (2025), 84038–84056. doi:10.1109/ACCESS.2025.3566286
-
[39]
Jesse J Martinez, Jon E. Froehlich, and James Fogarty. 2024. Playing on Hard Mode: Accessibility, Difficulty and Joy in Video Game Adoption for Gamers with Disabilities. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems(Honolulu, HI, USA)(CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 524, 17 pages. ...
-
[40]
Lucas Medeiros and Flavio Coutinho. 2015. Developing an Accessible One-Switch Game for Motor Impaired Players.Proceedings of SBGames(2015), 236–239
work page 2015
-
[41]
2017.Xbox Controller Assist (formerly Xbox Copilot)
Microsoft. 2017.Xbox Controller Assist (formerly Xbox Copilot). https://support.xbox.com/en-US/help/account-profile/accessibility/copilot
work page 2017
-
[42]
Microsoft. n.d..Xbox Adaptive Controller. https://www.xbox.com/accessories/controllers/xbox-adaptive-controller
-
[43]
Microsoft. n.d..Xbox Controller. https://support.xbox.com/en-US/help/hardware-network/controller/xbox-one-wireless-controller
-
[44]
Raja Parasuraman and Victor Riley. 1997. Humans and Automation: Use, Misuse, Disuse, Abuse.Human Factors39, 2 (1997), 230–253. doi:10.1518/ 001872097778543886
work page 1997
-
[45]
PEGI. n.d..What do the labels mean?https://pegi.info/what-do-the-labels-mean Manuscript submitted to ACM Game Accessibility Through Shared Control for People With Upper-Limb Impairments 29
-
[46]
PlatinumGames. 2014. Bayonetta. Video game, originally released on Xbox 360, PlayStation 3, and later on Nintendo Wii U, Nintendo Switch, Xbox One, PlayStation 4, and PC
work page 2014
-
[47]
PlatinumGames. 2014. Bayonetta 2. Video game, originally released on Nintendo Wii U and later on Nintendo Switch
work page 2014
- [48]
-
[49]
Nathaniel Poor. 2014. Computer game modders’ motivations and sense of community: A mixed-methods approach.New media & society16, 8 (2014), 1249–1267
work page 2014
-
[50]
Psyonix. 2015. Rocket League. Video game, released on PlayStation 4, Xbox One, Nintendo Switch, and PC
work page 2015
-
[51]
Marco C Rozendaal, Bram AL Braat, and Stephan AG Wensveen. 2010. Exploring sociality and engagement in play through game-control distribution. Ai & Society25 (2010), 193–201
work page 2010
-
[52]
Richard M Ryan, C Scott Rigby, and Andrew Przybylski. 2006. The motivational pull of video games: A self-determination theory approach. Motivation and emotion30, 4 (2006), 344–360
work page 2006
-
[53]
Anastasia Schaadhardt, Alexis Hiniker, and Jacob O. Wobbrock. 2021. Understanding Blind Screen-Reader Users’ Experiences of Digital Artboards. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems(Yokohama, Japan)(CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 270, 19 pages. doi:10.1145/3411764.3445242
-
[54]
SIMA Team, Maria Abi Raad, Arun Ahuja, Catarina Barros, Frederic Besse, Andrew Bolt, Adrian Bolton, Bethanie Brownfield, Gavin Buttimore, Max Cant, Sarah Chakera, Stephanie C. Y. Chan, Jeff Clune, Adrian Collister, Vikki Copeman, Alex Cullum, Ishita Dasgupta, Dario de Cesare, Julia Di Trapani, Yani Donchev, Emma Dunleavy, Martin Engelcke, Ryan Faulkner, F...
-
[55]
Sony. n.d..DualSense Controller. https://www.playstation.com/accessories/dualsense-wireless-controller
-
[56]
Sony. n.d..DUALSHOCK 4 Controller. https://www.playstation.com/accessories/dualshock-4-wireless-controller
-
[57]
Warhorse Studios. 2018. Kingdom Come: Deliverance. Video game, released on Microsoft Windows, PlayStation 4, and Xbox One. https: //www.deepsilver.com/games/kingdom-come-deliverance
work page 2018
-
[58]
Philipp Sykownik, Katharina Emmerich, and Maic Masuch. 2018. Exploring Patterns of Shared Control in Digital Multiplayer Games. InAdvances in Computer Entertainment Technology. Springer International Publishing, Cham, 847–867. doi:10.1007/978-3-319-76270-8_57
-
[59]
Maryam Taeb, Amanda Swearngin, Eldon Schoop, Ruijia Cheng, Yue Jiang, and Jeffrey Nichols. 2024. AXNav: Replaying Accessibility Tests from Natural Language. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems(Honolulu, HI, USA)(CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 962, 16 pages. doi:10.1145/3...
-
[60]
Gareth Terry, Nikki Hayfield, Victoria Clarke, Virginia Braun, et al. 2017. Thematic analysis.The SAGE handbook of qualitative research in psychology 2, 17-37 (2017), 25
work page 2017
-
[61]
2023.Forza Motorsport Accessibility Support
Turn 10 Studios. 2023.Forza Motorsport Accessibility Support. https://support.forzamotorsport.net/hc/articles/20964254277267-Forza-Motorsport- Accessibility-Support
-
[62]
April Tyack and Elisa D. Mekler. 2024. Self-Determination Theory and HCI Games Research: Unfulfilled Promises and Unquestioned Paradigms. ACM Trans. Comput.-Hum. Interact.31, 3, Article 40 (Aug. 2024), 74 pages. doi:10.1145/3673230
- [63]
-
[64]
Frank Wilcoxon. 1945. Individual comparisons by ranking methods.Biometrics bulletin1, 6 (1945), 80–83
work page 1945
-
[65]
Xbox. n.d..Xbox Adaptive Joystick. https://www.xbox.com/accessories/controllers/xbox-adaptive-joystick
-
[66]
Bei Yuan, Eelke Folmer, and Frederick C Harris Jr. 2011. Game accessibility: a survey.Universal Access in the information Society10, 1 (2011), 81–100. AGamePalsFramework Implementation We discuss the main implementation details of theGamePalsframework. In particular, we explain how we transmitted input commandsto the game (Appendix A.1), how we masked use...
work page 2011
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.