Pith. sign in

REVIEW 3 major objections 4 minor 73 references

This paper argues that biomechanical RL simulation can be turned from a days-long expert task into an hour-scale interactive design workflow.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 22:53 UTC pith:DR2QJQ75

load-bearing objection Useful, well-engineered framework with a genuinely credible hour-scale workflow; the headline 98% speedup is confounded by an easier task variant, but the default configuration's ~11x wall-clock gain and the workshop evidence support the central claim. the 3 major comments →

arxiv 2602.15245 v2 pith:DR2QJQ75 submitted 2026-02-16 cs.HC cs.AI

MyoInteract: A Framework for Fast Prototyping of Biomechanical HCI Tasks using Reinforcement Learning

Ankit Bhattarai , Hannah Selder , Florian Fischer , Arthur Fleig , Per Ola Kristensson This is my paper
classification cs.HC cs.AI
keywords deep reinforcement learningbiomechanical modelssimulated usershuman-computer interactionGPU accelerationrapid prototypingmusculoskeletal simulationinteraction design
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper identifies three barriers that keep biomechanical reinforcement learning out of ordinary HCI prototyping: the expertise needed to configure a task, the opacity of the training process, and day-long compute times. It introduces MyoInteract, a framework that combines a GUI-based task composer with massively parallel GPU physics simulation, replacing hand-written XML/Python configuration with sliders and composable pointing/pressing primitives. The central claim is a collapse of these barriers: training the standard 10-target pointing task falls from roughly 6.9 hours to 10 minutes in a tuned replication (98% reduction), and to about 36 minutes in the general default configuration. A workshop with 12 HCI researchers supports the usability claim, with novices successfully setting up, training, and assessing simulated users in a single session, while generated movements still reproduce Fitts's law and bell-shaped velocity profiles.

Core claim

The paper's core claim is that the two barriers that kept biomechanical RL out of ordinary HCI prototyping—specialist setup knowledge and day-long training cycles—can be collapsed simultaneously without sacrificing movement plausibility. MyoInteract does this by running thousands of muscle-actuated simulations in parallel on a GPU, composing tasks from chained pointing and pressing primitives specified through a GUI, and logging per-subtask and per-reward-component diagnostics. On the standard 10-target pointing task, training fell from about 6.9 hours to 10 minutes in the tuned replication and about 36 minutes in the general configuration; the workshop found that novices could set up, train

What carries the argument

The machinery is the combination of a GPU-accelerated physics simulator with thousands of parallel environments, a policy-gradient reinforcement learning algorithm operating end-to-end on the accelerator, and a compositional task model that chains pointing and pressing primitives. Around this sits a GUI with progressive disclosure (simple vs advanced modes), constrained sliders, a 3D environment preview, and a real-time metrics dashboard that decomposes success by subtask and reward by component. The parallelization is what compresses wall-clock training; the composability and GUI are what remove the programming burden; the diagnostics are what make failures interpretable.

Load-bearing premise

The 98% claim assumes the replication's removal of visual inference is not itself the main cause of the step reduction from roughly 43M to 7.4M; if it is, the framework's speed-up is overstated.

What would settle it

Run the original 10-target pointing task with its visual-inference observation pipeline on the same GPU-accelerated training stack; if steps-to-convergence stays near 40M rather than around 7M, the headline training-time reduction is mostly due to a simpler task, not to the framework's acceleration.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If training converges in minutes rather than days, designers can compare several task layouts or reward weightings in a single session instead of committing to one configuration.
  • The workshop result implies that biomechanical RL competence can be separated from RL expertise: novices with a GUI and guided defaults can produce working simulated-user policies.
  • The demonstration scenarios (AR interleaving, public display, mobile typing) suggest the composable primitives cover a useful range of sequential target-acquisition tasks.
  • Maintaining Fitts's law and bell-shaped velocity profiles under the accelerated pipeline implies the speed-up does not obviously degrade the plausibility of the learned movements.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct consequence the authors leave implicit: if per-run cost drops to minutes, parameter sweeps over layouts, target sizes, and effort weights become affordable, turning simulation from a single evaluation into a small-scale experiment generator.
  • Because the replication replaced visual inference with direct target-state information, the cleanest test of the framework's contribution would partition the speed-up: re-run the original visual-inference task on the GPU-accelerated pipeline and measure how many of the saved steps come from parallelization versus task simplification.
  • The paper's decomposition of success by subtask and reward by component is a diagnostic pattern that could transfer to other black-box user models (e.g., cognitive or biomechanical models of typing), suggesting a general principle: when policies are opaque, expose the intermediate objectives.
  • If the 'recommended training duration' heuristic scales with target count and dwell time, it may be the basis for an automatic curriculum or early-stopping rule that future frameworks could incorporate.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents MyoInteract, a framework for fast prototyping of biomechanical HCI tasks using reinforcement learning. It combines a GUI for task configuration, GPU-accelerated physics (MuJoCo MJX/Brax), composable pointing/pressing primitives, real-time logging and visualization, and automatic video generation. The authors claim training-time reductions of up to 98% compared with the User-in-the-Box baseline, report Fitts'-law-consistent movements, and present a workshop study with 12 HCI researchers who were able to configure, train, and assess simulated users within a single session. The central claims are that MyoInteract reduces the Gulf of Execution, the Gulf of Evaluation, and the temporal barrier of biomechanical RL simulation for HCI.

Significance. If the speed and usability claims hold, MyoInteract addresses a genuine bottleneck: biomechanical RL user simulation is currently expert-only and too slow for iterative design. The design goals are well motivated through Norman's action cycle, and the system has concrete, plausible mechanisms—massive GPU parallelization, composable task primitives, progressive disclosure, reward decomposition, and real-time diagnostics. Strengths of the paper include an open-source implementation, a clear design rationale, a benchmarking study (though confounded as discussed below), external validation against Fitts' law, and a hands-on workshop study with the target user group. The usability evidence is qualitative but appropriate for an exploratory tool paper. The main weakness is that the headline quantitative speed claim conflates task simplification with framework acceleration, and the convergence criterion is subjective. These issues are addressable and do not undermine the overall value of the framework, but they must be fixed before the paper can be accepted.

major comments (3)
  1. [§5.1, Table 1; Abstract] The headline 'up to 98%' training-time reduction is based on MyoInteract (Replication), which differs from UitB not only in RL implementation and hyperparameters but also in the observation space: the authors state that the replication uses 'direct provision of information about the target state instead of visual inference.' The step-count drop from 42.9M to 7.4M (~5.8x) cannot be explained by GPU throughput alone, since parallelization changes wall-clock time per step, not sample complexity. The Default configuration requires 34.9M steps—close to UitB's 42.9M—and its ~11x wall-clock speedup is the more defensible measure of the framework's GPU/parallelization benefit. As written, the 98% figure conflates task simplification with framework acceleration. Please provide a controlled comparison, e.g., UitB with direct target-state observations, or MyoInteract (Replication) with visual infer
  2. [§5.1, Table 1] Convergence was determined by 'manual inspection of the reward loggings.' This is a subjective stopping rule and is load-bearing for the reported step counts and speedups. A run stopped slightly before a plateau would exaggerate the speed advantage; a run continued past plateau would understate it. Please specify an a priori convergence criterion (e.g., rolling average success rate ≥95% over a fixed number of evaluation episodes and no improvement over N additional steps), and report the resulting step counts, the number of seeds, and sensitivity to the threshold. This is necessary to make Table 1's numbers reproducible and the 40x claim credible.
  3. [§5.2, Figure 8 and Appendix C] The Fitts'-law validation is reported as R²=0.89 (Default) and R²=0.90 (Replication), but the fit is presumably on mean movement times over a very narrow range of indices of difficulty (roughly 2.8–3.8 bits). The number of targets/IDs, per-ID variability, and the number of independent trials are not reported. Since the speedup claim is only meaningful if the resulting movements are biomechanically plausible, this evidence needs to be more rigorous: report the number of IDs, per-ID mean and SD, the fit on individual movement times rather than only aggregated means, and confidence intervals for the regression coefficients. The current presentation is suggestive but weaker than the claim 'demonstrates comparable movement regularity.'
minor comments (4)
  1. [§4.5.1 vs. Appendix A.3.1] The AR task description says virtual spheres require a 0.5s dwell, while Appendix A.3.1 states all spherical targets have a dwell duration of 25ms. Please reconcile this inconsistency.
  2. [Abstract, §8, Table 1] The abstract and conclusion emphasize an 'up to 98%' training-time reduction. Table 1 shows 91% for the Default configuration, which is the more general and defensible comparison. Please make explicit which configuration supports which percentage, and avoid presenting the Replication number as the primary evidence for the temporal-barrier claim.
  3. [§6] The workshop study is exploratory and the sample is small (n=12). This is acceptable for a usability study, but the paper should avoid over-generalizing; the participants had just received a 20-minute tutorial and were given a structured task with preloaded configurations. A comparison with a text-configuration baseline would strengthen the 'Gulf of Execution' claim.
  4. [§4.2.4] The training-duration heuristic ('adding 1M steps per additional target or additional 0.3s of total dwell time') is presented as a sensible default but is not validated. A sentence acknowledging its heuristic nature and suggesting when users should override it would be helpful.

Circularity Check

0 steps flagged

No significant circularity: central speed and usability claims rest on external measurements, not on the framework's own assumptions.

full rationale

The paper's central quantitative claim—'reducing training times by up to 98%'—is an empirical benchmark against UitB run on the same hardware, not a quantity derived from MyoInteract's own definitions. The replication is admittedly not identical: Section 5.1 lists 'the direct provision of information about the target state instead of visual inference' as one of the differences, so the speedup magnitude is partly confounded with observation/task simplification. A confounded benchmark is a methodological validity concern, however, not a circular derivation. The Fitts' law check (R^2 = 0.89) is an external movement regularity, and the workshop study is an external usability sample; neither is fitted from the framework's inputs and then re-reported as a prediction. The reward function is inherited from the authors' prior guidelines [54,55], and UitB [24] involves overlapping authors, but these self-citations are transparent and not load-bearing: no claimed result reduces by construction to those references, and the UitB baseline was re-run on the same machine rather than taken on faith. No fitted parameter is renamed as a prediction, and no uniqueness or ansatz is imported via self-citation. Hence no circular step can be exhibited.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The central contribution is engineering rather than derivation; the main assumptions are modeling validity and the fairness of the speed comparison. No new physical entities are introduced.

free parameters (5)
  • Reward weights (distance, subtask bonus, completion bonus, effort) = defaults not stated in the paper; exposed in GUI
    Chosen by hand to make training converge and shape movement strategy; the training results in Section 5 depend on them.
  • Motor noise levels k_SDN and k_CN = not specified numerically
    Defined in Eq. (1); they affect movement variability and training difficulty and are exposed as GUI options.
  • Brax PPO hyperparameters (1024 envs, batch size, learning rate, etc.) = Table 3
    Chosen by hand; the claimed parallel speedup and convergence rely on these values.
  • Training-duration heuristic (+1M steps per extra target, +0.3s per extra dwell) = 1M steps/target; 0.3s extra dwell
    Ad hoc recommendation introduced to avoid incomplete runs; it underpins the 'under one hour' workflow claim.
  • Fitts' law fit coefficients a and b = a=0.35, b=0.07; a=0.18, b=0.13
    Fitted to generated movement times to produce the R²=0.89/0.90 plausibility evidence; they are validation outputs, not simulation inputs.
axioms (5)
  • domain assumption MoBL-ARMS model with 26 muscles and fixed fingers is a sufficiently valid model of the human upper extremity for HCI simulation.
    Inherited from UitB [24]; the paper's biomechanical-plausibility claims rest on this model.
  • domain assumption MuJoCo/MJX contact and muscle dynamics are accurate enough that trained policies produce human-like movement.
    The whole framework relies on the physics engine; no real-world motion capture validation is provided beyond Fitts' law and qualitative videos.
  • domain assumption Fitts' law and bell-shaped velocity profiles are appropriate and sufficient validity criteria.
    Section 5.2 uses these as the sole quantitative plausibility checks; they do not validate fatigue, effort, or broader motor behavior.
  • ad hoc to paper Direct target-state information instead of visual inference does not materially alter task difficulty in the speed comparison.
    Section 5.1 lists this as a difference; if false, the 98% speedup is overstated.
  • domain assumption Self-reported ease-of-use by 12 workshop participants is a valid measure of accessibility.
    Workshop conclusions rely on Likert scales and quotes; there are no objective task-success criteria or baseline comparison.

pith-pipeline@v1.3.0-alltime-deepseek · 24708 in / 10619 out tokens · 103627 ms · 2026-08-02T22:53:41.364923+00:00 · methodology

0 comments
read the original abstract

Reinforcement learning (RL)-based biomechanical simulations have the potential to revolutionise HCI research and interaction design, but currently lack usability and interpretability. Using the Human Action Cycle as a design lens, we identify key limitations of biomechanical RL frameworks and develop MyoInteract, a novel framework for fast prototyping of biomechanical HCI tasks. MyoInteract allows designers to setup tasks, user models, and training parameters from an easy-to-use GUI within minutes. It trains and evaluates muscle-actuated simulated users within minutes, reducing training times by up to 98%. A workshop study with 12 interaction designers revealed that MyoInteract allowed novices in biomechanical RL to successfully setup, train, and assess goal-directed user movements within a single session. By transforming biomechanical RL from a days-long expert task into an accessible hour-long workflow, this work significantly lowers barriers to entry and accelerates iteration cycles in HCI biomechanics research.

Figures

Figures reproduced from arXiv: 2602.15245 by Ankit Bhattarai, Arthur Fleig, Florian Fischer, Hannah Selder, Per Ola Kristensson.

Figure 1
Figure 1. Figure 1: The GUI of MyoInteract, a framework that allows HCI researchers to setup, train, monitor, and evaluate biomechanical simulations of interactive user behaviour. Users can (1) choose from a list of pre-defined task configurations for different HCI contexts (Mobile Touch, Public Display, Augmented Reality) and (2) adjust them, or define entirely new interaction tasks that involve sequences of target acquisiti… view at source ↗
Figure 2
Figure 2. Figure 2: Workflow of the biomechanical RL simulations approach analysed using Norman’s seven stages of action [ [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: A built-in 3D renderer allows users to visually inspect [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 3
Figure 3. Figure 3: Box target settings in our GUI. Users can adjust position, size, minimum touch force, and orientation angle via sliders, [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Analysis of subtask completion metrics for each target in the AR task. Note how the success rates of subtasks build [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Analysis of various reward components across training for the AR task. In this example, the individual reward [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Example illustration of an augmented-reality (AR) task where the user has to sequentially complete pointing and [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Example illustrations of the public display interaction and mobile keyboard typing tasks generated with [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Biomechanical validation of the policy trained using MyoInteract (Default). [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Biomechanical validation of the policy trained using MyoInteract (Replication). [PITH_FULL_IMAGE:figures/full_fig_p019_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Simple version of our GUI for configuring reinforcement learning experiments, adjusting task and reward parameters, [PITH_FULL_IMAGE:figures/full_fig_p021_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Advanced version of our GUI for configuring reinforcement learning experiments, adjusting task and reward [PITH_FULL_IMAGE:figures/full_fig_p022_11.png] view at source ↗
Figure 11
Figure 11. Figure 11: Advanced version of our GUI (continued from previous page). [PITH_FULL_IMAGE:figures/full_fig_p023_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

73 extracted references · 7 canonical work pages

  1. [1]

    Abubakar Abid, Ali Abdalla, Ali Abid, Dawood Khan, Abdulrahman Alfozan, and James Y. Zou. 2019. Gradio: Hassle-Free Sharing and Testing of ML Models in the Wild.CoRRabs/1906.02569 (2019). arXiv:1906.02569 http://arxiv.org/abs/ 1906.02569

    Pith/arXiv arXiv 2019

  2. [2]

    Marko Ackermann and Antonie J Van den Bogert. 2010. Optimality principles for model-based prediction of human gait.Journal of biomechanics43, 6 (2010), 1055–1060

    2010

  3. [3]

    Myroslav Bachynskyi, Antti Oulasvirta, Gregorio Palmas, and Tino Weinkauf

  4. [4]

    Myroslav Bachynskyi, Gregorio Palmas, Antti Oulasvirta, Jürgen Steimle, and Tino Weinkauf. 2015. Performance and ergonomics of touch surfaces: A compara- tive study using biomechanical simulation. InProceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1817–1826

    2015

  5. [5]

    Bastien Berret, Enrico Chiovetto, Francesco Nori, and Thierry Pozzo. 2011. Evi- dence for Composite Cost Functions in Arm Movement Planning: An Inverse Op- timal Control Approach.PLoS Computational Biology7, 10 (Oct. 2011), e1002183. doi:10.1371/journal.pcbi.1002183

    work page doi:10.1371/journal.pcbi.1002183 2011

  6. [6]

    2018.JAX: composable transformations of Python+NumPy programs

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. 2018.JAX: composable transformations of Python+NumPy programs. http://github.com/jax-ml/jax

    2018

  7. [7]

    Vittorio Caggiano, Sudeep Dasari, and Vikash Kumar. 2023. MyoDex: a general- izable prior for dexterous manipulation. InProceedings of the 40th International Conference on Machine Learning(Honolulu, Hawaii, USA)(ICML’23). JMLR.org, New York, NY, USA, 20 pages

    2023

  8. [8]

    Vittorio Caggiano, Huawei Wang, Guillaume Durandau, Massimo Sartori, and Vikash Kumar. 2022. MyoSuite: A Contact-rich Simulation Suite for Muscu- loskeletal Motor Control. InProceedings of The 4th Annual Learning for Dynamics and Control Conference (Proceedings of Machine Learning Research, Vol. 168), Roya Firoozi, Negar Mehr, Esen Yel, Rika Antonova, Jea...

    2022

  9. [9]

    Suyog Chandramouli, Danqing Shi, Aini Putkonen, Sebastiaan De Peuter, Shan- shan Zhang, Jussi Jokinen, Andrew Howes, and Antti Oulasvirta. 2024. A Work- flow for Building Computationally Rational Models of Human Behavior.Compu- tational Brain and Behavior7, 3 (Sept. 2024), 399–419. doi:10.1007/s42113-024- 00208-6 Publisher Copyright:©The Author(s) 2024

    work page doi:10.1007/s42113-024- 2024

  10. [10]

    Charaja, Isabell Wochner, Pierre Schumacher, Winfried Ilg, Martin Giese, Christophe Maufroy, Andreas Bulling, Syn Schmitt, Georg Martius, and Daniel F.B

    Jhon P.F. Charaja, Isabell Wochner, Pierre Schumacher, Winfried Ilg, Martin Giese, Christophe Maufroy, Andreas Bulling, Syn Schmitt, Georg Martius, and Daniel F.B. Haeufle. 2024. Generating Realistic Arm Movements in Reinforcement Learning: A Quantitative Comparison of Reward Terms and Task Requirements. In2024 10th IEEE RAS/EMBS International Conference ...

    arXiv 2024

  11. [11]

    Alberto Silvio Chiappa, Pablo Tano, Nisheet Patel, Abigaïl Ingster, Alexandre Pouget, and Alexander Mathis. 2024. Acquiring musculoskeletal skills with curriculum-based reinforcement learning.Neuron112, 23 (2024), 3969–3983

    2024

  12. [12]

    Alberto Silvio Chiappa, Alessandro Marin Vargas, Ann Zixiang Huang, and Alexander Mathis. 2023. Latent exploration for reinforcement learning. InPro- ceedings of the 37th International Conference on Neural Information Processing Systems(New Orleans, LA, USA)(NIPS ’23). Curran Associates Inc., Red Hook, NY, USA, Article 2466, 23 pages

    2023

  13. [13]

    Omar Coser, Christian Tamantini, Paolo Soda, and Loredana Zollo. 2024. AI- based methodologies for exoskeleton-assisted rehabilitation of the lower limb: a review.Frontiers in Robotics and AI11 (2024), 1341580

    2024

  14. [14]

    Joao Marcelo Evangelista Belo, Anna Maria Feit, Tiare Feuchtner, and Kaj Grøn- bæk. 2021. XRgonomics: Facilitating the Creation of Ergonomic 3D Interfaces. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan)(CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 290, 11 pages. doi:10.1145/34...

    arXiv 2021

  15. [15]

    Pouyan Firouzabadi, Wendy Murray, Anton R Sobinov, JD Peiffer, Kunal Shah, Lee E Miller, and R James Cotton. 2024. Biomechanical Arm and Hand Tracking with Multiview Markerless Motion Capture. In2024 10th IEEE RAS/EMBS Inter- national Conference for Biomedical Robotics and Biomechatronics (BioRob). IEEE, 1641–1648

    2024

  16. [16]

    Florian Fischer, Miroslav Bachinski, Markus Klar, Arthur Fleig, and Jörg Müller

  17. [17]

    Florian Fischer, Arthur Fleig, Markus Klar, and Jörg Müller. 2022. Optimal Feedback Control for Modeling Human–Computer Interaction.ACM Transactions on Computer-Human Interaction29, 6 (Dec. 2022), 1–70. doi:10.1145/3524122

    work page doi:10.1145/3524122 2022

  18. [18]

    Florian Fischer, Aleksi Ikkala, Markus Klar, Arthur Fleig, Miroslav Bachinski, Roderick Murray-Smith, Perttu Hämäläinen, Antti Oulasvirta, and Jörg Müller

  19. [19]

    Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem

    C. Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem. 2021.Brax - A Differentiable Physics Engine for Large Scale Rigid Body Simulation. http://github.com/google/brax

    2021

  20. [20]

    Green and M

    T.R.G. Green and M. Petre. 1996. Usability Analysis of Visual Programming En- vironments: A ‘Cognitive Dimensions’ Framework.Journal of Visual Languages & Computing7, 2 (1996), 131–174. doi:10.1006/jvlc.1996.0009

    arXiv 1996

  21. [21]

    Christopher M Harris and Daniel M Wolpert. 1998. Signal-dependent noise determines motor planning.Nature394, 6695 (1998), 780–784. Ankit Bhattarai, Hannah Selder, Florian Fischer, Arthur Fleig, and Per Ola Kristensson

    1998

  22. [22]

    Lorenz Hetzel, John Dudley, Anna Maria Feit, and Per Ola Kristensson. 2021. Complex Interaction as Emergent Behaviour: Simulating Mid-Air Virtual Key- board Typing using Reinforcement Learning.IEEE Transactions on Visualization and Computer Graphics27, 11 (2021), 4140–4149. doi:10.1109/TVCG.2021.3106494

    arXiv 2021

  23. [23]

    Seongwoong Hong and Sukyung Park. 2024. Biomechanical Optimization and Reinforcement Learning Provide Insights into Ankle-to-Hip Strategy Transitions in Human Postural Control.Scientific Reports15 (2024). doi:10.1038/s41598-025- 97637-5

    doi:10.1038/s41598-025- 2024

  24. [24]

    Aleksi Ikkala, Florian Fischer, Markus Klar, Miroslav Bachinski, Arthur Fleig, Andrew Howes, Perttu Hämäläinen, Jörg Müller, Roderick Murray-Smith, and Antti Oulasvirta. 2022. Breathing Life Into Biomechanical User Models. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. ACM, Bend OR USA, 1–14. doi:10.1145/3526113.3545689

    arXiv 2022

  25. [25]

    Markus Klar, Florian Fischer, Arthur Fleig, Miroslav Bachinski, and Jörg Müller

  26. [26]

    Francesco Lacquaniti, Carlo Terzuolo, and Paolo Viviani. 1983. The law relating the kinematic and figural aspects of drawing movements.Acta Psychologica54, 1 (1983), 115–130. doi:10.1016/0001-6918(83)90027-6

    work page doi:10.1016/0001-6918(83)90027-6 1983

  27. [27]

    Thomas Langerak, Sammy Christen, Mert Albaba, Christoph Gebhardt, Christian Holz, and Otmar Hilliges. 2024. MARLUI: Multi-Agent Reinforcement Learning for Adaptive Point-and-Click UIs.Proceedings of the ACM on Human-Computer Interaction8, EICS (June 2024), 1–27. doi:10.1145/3661147

    work page doi:10.1145/3661147 2024

  28. [28]

    Richard L Lewis, Andrew Howes, and Satinder Singh. 2014. Computational ratio- nality: Linking mechanism and behavior through bounded utility maximization. Topics in cognitive science6, 2 (2014), 279–311

    2014

  29. [29]

    Yi Li, Florian Fischer, Tim Dwyer, Barrett Ens, Robert Crowther, Per Ola Kristens- son, and Benjamin Tag. 2025. AlphaPIG: The Nicest Way to Prolong Interactive Gestures in Extended Reality. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 627, 14 pages...

    arXiv 2025

  30. [30]

    Yi Li, Benjamin Tag, Shaozhang Dai, Robert Crowther, Tim Dwyer, Pourang Irani, and Barrett Ens. 2024. Nicer: A new and improved consumed endurance and recovery metric to quantify muscle fatigue of mid-air interactions.ACM Transactions on Graphics (TOG)43, 4 (2024), 1–14

    2024

  31. [31]

    Yi-Chi Liao and Christian Holz. 2025. Redefining Affordance via Computational Rationality. InProceedings of the 30th International Conference on Intelligent User Interfaces (IUI ’25). Association for Computing Machinery, New York, NY, USA, 1188–1202. doi:10.1145/3708359.3712114

    arXiv 2025

  32. [32]

    William Lidwell, Kritina Holden, and Jill Butler. 2010.Universal principles of design, revised and updated: 125 ways to enhance usability, influence perception, increase appeal, make better design decisions, and teach through design. Rockport Pub

    2010

  33. [33]

    Ramona Maas and Sigrid Leyendecker. 2013. Biomechanical optimal control of human arm motion.Proceedings of the Institution of Mechanical Engineers, Part K: Journal of Multi-body Dynamics227, 4 (2013), 375–389

    2013

  34. [34]

    Michał Patryk Miazga and Patrick Ebel. 2025. Increasing Interaction Fidelity: Training Routines for Biomechanical Models in HCI. , Article 99 (2025), 3 pages. doi:10.1145/3746058.3758385

    arXiv 2025

  35. [35]

    Takeshi Miki, Florian Fischer, John J Dudley, and Per Ola Kristensson. 2025. Enhancing XR Accessibility through Anthropometrically Diverse Biomechanical Simulation. In2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). 1186–1187. doi:10.1109/VRW66409.2025.00239

    arXiv 2025

  36. [36]

    Hee-Seung Moon, Yi-Chi Liao, Chenyu Li, Byungjoo Lee, and Antti Oulasvirta

  37. [37]

    Hee-Seung Moon, Antti Oulasvirta, and Byungjoo Lee. 2023. Amortized Inference with User Simulations. InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, Hamburg Germany, 1–20. doi:10.1145/3544548. 3581439

    doi:10.1145/3544548 2023

  38. [38]

    Pietro Morasso. 1981. Spatial control of arm movements.Experimental brain research42, 2 (1981), 223–227

    1981

  39. [39]

    Roderick Murray-Smith, Antti Oulasvirta, Andrew Howes, Jörg Müller, Aleksi Ikkala, Miroslav Bachinski, Arthur Fleig, Florian Fischer, and Markus Klar. 2022. What simulation can do for HCI research.Interactions29, 6 (Nov. 2022), 48–53. doi:10.1145/3564038

    work page doi:10.1145/3564038 2022

  40. [40]

    Williamson, and Sebastian Stein

    Roderick Murray-Smith, John H. Williamson, and Sebastian Stein. 2025. Active Inference and Human–Computer Interaction.ACM Trans. Comput.-Hum. Interact. 32, 6, Article 63 (Dec. 2025), 45 pages. doi:10.1145/3762812

    work page doi:10.1145/3762812 2025

  41. [41]

    Masaki Nakada, Tao Zhou, Honglin Chen, Tomer Weiss, and Demetri Terzopoulos

  42. [42]

    InProceedings of the CHI Conference on Human Factors in Computing Systems

    Real-time 3D Target Inference via Biomechanical Simulation. InProceedings of the CHI Conference on Human Factors in Computing Systems. ACM, Honolulu HI USA, 1–18. doi:10.1145/3613904.3642131

    arXiv

  43. [43]

    Katharine Nowakowski, Karim El Kirat, and Tien-Tuan Dao. 2022. Deep rein- forcement learning coupled with musculoskeletal modelling for a better under- standing of elderly falls.Medical & Biological Engineering & Computing60, 6 (2022), 1745–1761

    2022

  44. [44]

    Antti Oulasvirta, Jussi P. P. Jokinen, and Andrew Howes. 2022. Computational Rationality as a Theory of Interaction. InCHI Conference on Human Factors in Computing Systems. ACM, New Orleans LA USA, 1–14. doi:10.1145/3491102. 3517739

    doi:10.1145/3491102 2022

  45. [45]

    Tianhu Peng, Lingfan Bao, and CHengxu Zhou. 2025. Gait-Conditioned Rein- forcement Learning with Multi-Phase Curriculum for Humanoid Locomotion. arXiv preprint arXiv:2505.20619(2025)

    arXiv 2025

  46. [46]

    Luka Peternel, Cheng Fang, Nikos Tsagarakis, and Arash Ajoudani. 2019. A selective muscle fatigue management approach to ergonomic human-robot co- manipulation.Robotics and Computer-Integrated Manufacturing58 (2019), 69–79

    2019

  47. [47]

    Ivan Poupyrev, Mark Billinghurst, Suzanne Weghorst, and Tadao Ichikawa. 1996. The go-go interaction technique: non-linear mapping for direct manipulation in VR. InProceedings of the 9th annual ACM symposium on User interface software and technology. 79–80

    1996

  48. [48]

    Ilija Radosavovic, Tete Xiao, Bike Zhang, Trevor Darrell, Jitendra Malik, and Koushil Sreenath. 2024. Real-world humanoid locomotion with reinforcement learning.Science Robotics9, 89 (2024), eadi9579

    2024

  49. [49]

    2002.The design of everyday things

    Donald Norman. 2002.The design of everyday things. Basic Books, London, England

    2002

  50. [50]

    Katherine R Saul, Xiao Hu, Craig M Goehler, Meghan E Vidt, Melissa Daly, Anca Velisar, and Wendy M Murray. 2015. Benchmarking of dynamic simulation predictions in two software platforms using an upper limb musculoskeletal model.Computer methods in biomechanics and biomedical engineering18, 13 (2015), 1445–1458

    2015

  51. [51]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov

  52. [52]

    Pierre Schumacher, Thomas Geijtenbeek, Vittorio Caggiano, Vikash Kumar, Syn Schmitt, Georg Martius, and Daniel FB Haeufle. 2025. Emergence of natural and robust bipedal walking by learning from biologically plausible objectives. iScience28, 4 (2025)

    2025

  53. [53]

    Haeufle, Dieter Büchler, Syn Schmitt, and Georg Martius

    Pierre Schumacher, Daniel F.B. Haeufle, Dieter Büchler, Syn Schmitt, and Georg Martius. 2023. DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systems. InProceedings of the Eleventh International Conference on Learning Representations (ICLR). ICLR, Appleton, WI, USA, 1–29. https://openreview.net/forum?id=C-xa_D3oTj6

    2023

  54. [55]

    Hannah Selder, Florian Fischer, Per Ola Kristensson, and Arthur Fleig. 2025. What Makes a Model Breathe? Understanding Reinforcement Learning Reward Function Design in Biomechanical User Simulation. InProceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’25). Association for Computing Machinery, New Yor...

    arXiv 2025

  55. [56]

    Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernes- tus, and Noah Dormann. 2021. Stable-Baselines3: Reliable Reinforcement Learn- ing Implementations.Journal of Machine Learning Research22, 268 (2021), 1–8. http://jmlr.org/papers/v22/20-1364.html

    2021

  56. [57]

    Danqing Shi, Yujun Zhu, Jussi P. P. Jokinen, Aditya Acharya, Aini Putkonen, Shumin Zhai, and Antti Oulasvirta. 2024. CRTypist: Simulating Touchscreen Typing Behavior via Computational Rationality. InProceedings of the CHI Con- ference on Human Factors in Computing Systems. ACM, Honolulu HI USA, 1–17. doi:10.1145/3613904.3642918

    arXiv 2024

  57. [58]

    Merkourios Simos, Alberto Silvio Chiappa, and Alexander Mathis. 2025. Rein- forcement learning-based motion imitation for physiologically plausible muscu- loskeletal motor control.arXiv preprint arXiv:2503.14637(2025)

    arXiv 2025

  58. [59]

    Chun Kwang Tan, Cheryl Wang, Shirui Lyu, Balint K Hodossy, Pierre Schumacher, Elizabeth B Wilson, Vittorio Caggiano, Vikash Kumar, Dario Farina, Letizia Gionfrida, et al. 2025. Myoassist 0.1: Myosuite for Dexterity and Agility in Bionic Humans. In2025 International Conference On Rehabilitation Robotics (ICORR). IEEE, 437–442

    2025

  59. [60]

    William Thibault, William Melek, and Katja Mombaur. 2024. Learning Velocity- Based Humanoid Locomotion: Massively Parallel Learning with Brax and MJX. InClimbing and Walking Robots Conference. Springer, 278–283

    2024

  60. [61]

    Emanuel Todorov, Tom Erez, and Yuval Tassa. 2012. MuJoCo: A physics engine for model-based control. In2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, Piscataway NJ USA, 5026–5033. doi:10.1109/IROS.2012. MyoInteract: A Framework for Fast Prototyping of Biomechanical HCI Tasks using Reinforcement Learning 6386109

    doi:10.1109/iros.2012 2012

  61. [62]

    Robert J van Beers, Patrick Haggard, and Daniel M Wolpert. 2004. The role of execution noise in movement variability.J. Neurophysiol.91, 2 (Feb. 2004), 1050–1063

    2004

  62. [63]

    Rohan Walia, Morgane Billot, Kevin Garzon-Aguirre, Swathika Subramanian, Huiyi Wang, Mohamed Irfan Refai, and Guillaume Durandau. 2025. Myoback: A musculoskeletal model of the human back with integrated exoskeleton.bioRxiv (2025), 2025–03

    2025

  63. [64]

    Bernstein

    Omar Shaikh, Shardul Sapkota, Shan Rizvi, Eric Horvitz, Joon Sung Park, Diyi Yang, and Michael S. Bernstein. 2025. Creating General User Models from Com- puter Use. InProceedings of the 38th Annual ACM Symposium on User Interface Software and Technology (UIST ’25). Association for Computing Machinery, New York, NY, USA, Article 35, 23 pages. doi:10.1145/3...

    arXiv 2025

  64. [65]

    Kevin Zakka, Baruch Tabanpour, Qiayuan Liao, Mustafa Haiderbhai, Samuel Holt, Jing Yuan Luo, Arthur Allshire, Erik Frey, Koushil Sreenath, Lueder A Kahrs, et al. 2025. Mujoco playground.arXiv preprint arXiv:2502.08844(2025)

    Pith/arXiv arXiv 2025

  65. [66]

    Xinnong Zhang, Jiayu Lin, Xinyi Mou, Shiyue Yang, Xiawei Liu, Libo Sun, Hanjia Lyu, Yihang Yang, Weihong Qi, Yue Chen, et al. 2025. Socioverse: A world model for social simulation powered by llm agents and a pool of 10 million real-world users.arXiv preprint arXiv:2504.10157(2025). Ankit Bhattarai, Hannah Selder, Florian Fischer, Arthur Fleig, and Per Ola...

    Pith/arXiv arXiv 2025

  66. [72]

    Huawei Wang, Vittorio Caggiano, Guillaume Durandau, Massimo Sartori, and Vikash Kumar. 2022. MyoSim: Fast and physiologically realistic MuJoCo models for musculoskeletal and exoskeletal studies. In2022 International Conference on Robotics and Automation (ICRA). IEEE, 8104–8111

    2022

  67. [507]

    https://proceedings.mlr.press/v168/caggiano22a.html

  68. [2014]

    InProceedings of the SIGCHI Conference on Human Factors in Computing Systems(Toronto, Ontario, Canada)(CHI ’14)

    Is motion capture-based biomechanical simulation valid for HCI studies? study and implications. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems(Toronto, Ontario, Canada)(CHI ’14). Association for Computing Machinery, New York, NY, USA, 3215–3224. doi:10.1145/2556288. 2557027

    doi:10.1145/2556288

  69. [2017]

    arXiv:1707.06347 http://arxiv.org/abs/1707.06347

    Proximal Policy Optimization Algorithms.CoRRabs/1707.06347 (2017). arXiv:1707.06347 http://arxiv.org/abs/1707.06347

    Pith/arXiv arXiv 2017

  70. [2018]

    2018), 1–15

    Deep learning of biomimetic sensorimotor control for biomechanical human animation.ACM Transactions on Graphics37, 4 (Aug. 2018), 1–15. doi:10. 1145/3197517.3201305 Publisher: Association for Computing Machinery (ACM)

    arXiv 2018

  71. [2021]

    Reinforcement learning control of a biomechanical model of the upper extremity.Scientific Reports11, 1 (2021), 14445

    2021

  72. [2023]

    doi:10.1145/ 3577016

    Simulating Interaction Movements via Model Predictive Control.ACM Transactions on Computer-Human Interaction30, 3 (June 2023), 1–50. doi:10.1145/ 3577016

    2023

  73. [2024]

    InProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology

    SIM2VR: Towards Automated Biomechanical Testing in VR. InProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology. ACM, Pittsburgh PA USA, 1–15. doi:10.1145/3654777.3676452

    arXiv