pith. sign in

arxiv: 2509.08539 · v1 · submitted 2025-09-10 · 💻 cs.HC · cs.LG

Motion-Based User Identification across XR and Metaverse Applications by Deep Classification and Similarity Learning

Lukas Schach , Christian Rack , Ryan P. McMahan , Marc Erich Latoschik This is my paper

Pith reviewed 2026-05-18 18:02 UTC · model grok-4.3

classification 💻 cs.HC cs.LG
keywords motion-based identificationXR applicationsMetaverseuser biometricscross-application generalizationclassification modelssimilarity learning
0
0 comments X

The pith

Motion data identifies users accurately within one XR app but shows limited success across different applications.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tests the ability of motion-based models to identify users even when they switch between different XR and Metaverse applications. The authors built a dataset with motion recordings from 49 users across five apps: four games with varied tasks and one open social environment. They applied two current models, one using direct classification and one using similarity learning, to measure performance. The models performed strongly when identifying users inside a single application yet dropped sharply when tested on data from a new application. These outcomes clarify both the promise and the current boundaries of motion as a biometric signal in varied virtual settings.

Core claim

While the models can accurately identify individuals within the same application, their ability to identify users across different XR applications remains limited. This result holds for both classification and similarity learning models evaluated on a dataset of motion data from 49 users in five distinct XR applications.

What carries the argument

A new dataset of motion data collected from 49 users across five XR applications (four games plus one social app) that serves as the testbed for measuring within-application versus cross-application accuracy of classification and similarity learning models.

If this is right

  • User identification via motion works reliably inside any single XR application.
  • Seamless biometric identification across the full Metaverse remains difficult with current models.
  • The results supply a concrete risk assessment for unwanted tracking in multi-app XR environments.
  • Releasing the dataset publicly allows others to benchmark improvements in cross-application performance.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Privacy risks from motion tracking may be lower when users freely switch between Metaverse apps than when they stay inside one app.
  • Practical systems may need app-specific training data or extra signals to achieve reliable cross-app identification.
  • App designers could deliberately vary motion patterns to increase user anonymity when users move between virtual spaces.

Load-bearing premise

The five selected XR applications are diverse enough to represent typical Metaverse usage patterns when testing cross-application generalization.

What would settle it

Training a model on motion data from one XR application and then observing high identification accuracy when testing it on motion data from a different XR application would challenge the limited cross-application generalization claim.

Figures

Figures reproduced from arXiv: 2509.08539 by Christian Rack, Lukas Schach, Marc Erich Latoschik, Ryan P. McMahan.

Figure 1
Figure 1. Figure 1: Screenshots from all five VR applications, arranged from left to right in the order participants played them: Synth [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Procedure of the dataset collection. for the first time and receive a brief introduction to its use. In this game, users have much freedom and thus can perform many differ￾ent activities through various navigational options.They can shoot, grab objects, solve puzzles through different controller movements, and move around the room directly or via teleportation. 5. Social VR is represented by a diadic multi… view at source ↗
Figure 3
Figure 3. Figure 3: This figure shows the nearest embedding accuracy [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: These figures show heatmaps for different metrics, using one application as reference and another as query. The [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
read the original abstract

This paper examines the generalization capacity of two state-of-the-art classification and similarity learning models in reliably identifying users based on their motions in various Extended Reality (XR) applications. We developed a novel dataset containing a wide range of motion data from 49 users in five different XR applications: four XR games with distinct tasks and action patterns, and an additional social XR application with no predefined task sets. The dataset is used to evaluate the performance and, in particular, the generalization capacity of the two models across applications. Our results indicate that while the models can accurately identify individuals within the same application, their ability to identify users across different XR applications remains limited. Overall, our results provide insight into current models generalization capabilities and suitability as biometric methods for user verification and identification. The results also serve as a much-needed risk assessment of hazardous and unwanted user identification in XR and Metaverse applications. Our cross-application XR motion dataset and code are made available to the public to encourage similar research on the generalization of motion-based user identification in typical Metaverse application use cases.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces a novel dataset of motion data from 49 users across five XR applications (four games with distinct tasks and one untasked social app). It evaluates two deep models (classification and similarity learning) for user identification, reporting high accuracy within individual applications but limited performance when generalizing across applications. The work frames these results as a risk assessment for motion-based biometrics in XR/Metaverse settings and publicly releases the dataset and code.

Significance. If the empirical results hold, the paper supplies concrete evidence that current motion-based identification techniques do not generalize reliably across typical XR use cases, informing privacy and security considerations in the Metaverse. The public dataset release is a clear strength that supports reproducibility and follow-on work on cross-application generalization.

major comments (2)
  1. [Results section] The interpretation that limited cross-application accuracy reflects a fundamental biometric limitation (rather than task or hardware overlap) requires supporting evidence that the five applications induce distinct motion distributions. No inter-app feature divergence statistics, pairwise distance metrics, or cross-application confusion matrices are reported in the results to substantiate this premise.
  2. [§3] §3 (Dataset and Methods): Participant instructions, precise motion feature extraction pipeline, normalization across hardware, and exact train/test split details are insufficiently specified. These omissions directly affect the ability to assess whether the reported within-app vs. cross-app performance gap is robust.
minor comments (2)
  1. [Results] Add explicit per-application-pair performance tables or heatmaps to allow readers to see which application pairs drive the cross-app drop.
  2. [Methods] Clarify the exact architecture and loss functions used for the similarity-learning model in the methods description.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive feedback, which helps clarify the presentation of our results and improves the reproducibility of the work. We address each major comment below and indicate the revisions we will make.

read point-by-point responses

  1. Referee: [Results section] The interpretation that limited cross-application accuracy reflects a fundamental biometric limitation (rather than task or hardware overlap) requires supporting evidence that the five applications induce distinct motion distributions. No inter-app feature divergence statistics, pairwise distance metrics, or cross-application confusion matrices are reported in the results to substantiate this premise.

    Authors: We agree that additional quantitative support would strengthen the claim that the cross-application performance drop reflects a biometric limitation. In the revised manuscript we will add inter-application feature divergence statistics (e.g., mean pairwise Euclidean distances between per-user feature centroids across applications), a summary of cross-application confusion matrices, and, where space permits, a brief t-SNE visualization of motion features colored by application. These additions will help demonstrate that the five applications produce measurably distinct motion distributions beyond simple task or hardware differences. revision: yes

  2. Referee: [§3] §3 (Dataset and Methods): Participant instructions, precise motion feature extraction pipeline, normalization across hardware, and exact train/test split details are insufficiently specified. These omissions directly affect the ability to assess whether the reported within-app vs. cross-app performance gap is robust.

    Authors: We acknowledge that the current description of §3 is too concise. We will expand this section to provide (1) the exact wording of participant instructions, (2) the full motion-feature extraction pipeline with all parameters, (3) the normalization procedure applied to account for differences in tracking hardware, and (4) the precise train/test split strategy (including how users and sessions were partitioned). These details will be added without altering the reported numbers, thereby allowing readers to verify the robustness of the within- versus cross-application gap. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical performance metrics on held-out data

full rationale

The paper reports an empirical machine-learning study: a new dataset of motion recordings from 49 users across five XR applications is collected, two standard deep models (classification and similarity learning) are trained, and identification accuracy is measured on within-app and cross-app held-out splits. All reported numbers are direct experimental outcomes; no equations, predictions, or first-principles claims are derived that reduce by construction to fitted parameters, self-definitions, or self-citation chains. The methodology follows ordinary supervised-learning practice with independent test partitions, rendering the central results self-contained.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The paper rests on standard machine-learning assumptions about data representativeness and the suitability of existing classification/similarity architectures for motion sequences; no new entities or ad-hoc parameters are introduced in the abstract.

pith-pipeline@v0.9.0 · 5724 in / 1064 out tokens · 41044 ms · 2026-05-18T18:02:06.797171+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

  • IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear
    ?
    unclear

    Relation between the paper passage and the cited Recognition theorem.

    We developed a novel dataset containing a wide range of motion data from 49 users in five different XR applications... evaluate the performance and, in particular, the generalization capacity of the two models across applications.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Deep Learning for Virtual Reality User Identification: A Benchmark

    cs.HC 2026-03 unverdicted novelty 4.0

    A benchmark study evaluates standard and emerging deep learning architectures on motion data from 71 VR users, establishing performance baselines for user identification.

Reference graph

Works this paper leans on

30 extracted references · 30 canonical work pages · cited by 1 Pith paper

  1. [1]

    OBS Studio

    2024. OBS Studio. https://obsproject.com/

    work page 2024

  2. [2]

    Sara Baldoni, Salim Benhamadi, Federico Chiariotti, Michele Zorzi, and Federica Battisti. 2025. Movement- and Traffic-based User Identification in Commercial Virtual Reality Applications: Threats and Opportunities. In 2025 IEEE Conference Virtual Reality and 3D User Interfaces (VR) . 72–81. doi:10.1109/VR59515.2025. 00032 arXiv:2501.16326 [cs]

    work page doi:10.1109/vr59515.2025 2025

  3. [3]

    Christopher Bruns. 2025. Pyopenvr. https://github.com/cmbruns/pyopenvr original-date: 2016-05-07T14:10:42Z

    work page 2025

  4. [4]

    Jain, Arun A

    Anil K. Jain, Arun A. Ross, and Karthik Nandakumar. 2011. Introduction to Bio- metrics. Springer US, Boston, MA. doi:10.1007/978-0-387-77326-1

    work page doi:10.1007/978-0-387-77326-1 2011

  5. [5]

    Adam Kendon. 2004. Gesture: Visible Action as Utterance. (01 2004). doi:10. 1017/CBO9780511807572

    work page 2004

  6. [6]

    Sugang Li, Ashwin Ashok, Yanyong Zhang, Chenren Xu, Janne Lindqvist, and Macro Gruteser. 2016. Whose move is it anyway? Authenticating smart wear- able devices using unique head movement patterns. In 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom) . IEEE, Syd- ney, Australia, 1–9. doi:10.1109/PERCOM.2016.7456514

    work page doi:10.1109/percom.2016.7456514 2016

  7. [7]

    Jonathan Liebers, Mark Abdelaziz, Lukas Mecke, Alia Saad, Jonas Auda, Uwe Gruenefeld, Florian Alt, and Stefan Schneegass. 2021. Understanding User Identi- fication in Virtual Reality Through Behavioral Biometrics and the Effect of Body Normalization. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama Japan, 1–11...

    work page doi:10.1145/3411764.3445528 2021

  8. [8]

    Jonathan Liebers, Sascha Brockel, Uwe Gruenefeld, and Stefan Schneegass. 2024. Identifying Users by Their Hand Tracking Data in Augmented and Virtual Re- ality. International Journal of Human–Computer Interaction 40, 2 (Jan. 2024), 409–424. doi:10.1080/10447318.2022.2120845

    work page doi:10.1080/10447318.2022.2120845 2024

  9. [9]

    Jonathan Liebers, Christian Burschik, Uwe Gruenefeld, and Stefan Schneegass

    work page

  10. [10]

    In 29th ACM Symposium on Virtual Reality Software and Tech- nology

    Exploring the Stability of Behavioral Biometrics in Virtual Reality in a Re- mote Field Study: Towards Implicit and Continuous User Identification through Body Movements: Towards Implicit and Continuous User Identification through Body Movements. In 29th ACM Symposium on Virtual Reality Software and Tech- nology. ACM, Christchurch New Zealand, 1–12. doi:1...

    work page doi:10.1145/3611659.3615696

  11. [11]

    David McNeill. 1992. Hand and Mind: What Gestures Reveal about Thought Univ

    work page 1992

  12. [12]

    Landay, and Jeremy N

    Mark Roman Miller, Fernanda Herrera, Hanseul Jun, James A. Landay, and Jeremy N. Bailenson. 2020. Personal identifiability of user tracking data dur- ing observation of 360-degree VR video. Sci Rep 10, 1 (Oct. 2020), 17404. doi:10.1038/s41598-020-74486-y

    work page doi:10.1038/s41598-020-74486-y 2020

  13. [13]

    Robert Miller, Natasha Kholgade Banerjee, and Sean Banerjee. 2020. Within- System and Cross-System Behavior-Based Biometric Authentication in Virtual Reality. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Ab- stracts and Workshops (VRW) . IEEE, Atlanta, GA, USA, 311–316. doi:10.1109/ VRW50115.2020.00070

    work page arXiv 2020

  14. [14]

    Robert Miller, Natasha Kholgade Banerjee, and Sean Banerjee. 2021. Using Siamese Neural Networks to Perform Cross-System Behavioral Authentication in Virtual Reality. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR) . IEEE, Lisboa, Portugal, 140–149. doi:10.1109/VR50410.2021.00035

    work page doi:10.1109/vr50410.2021.00035 2021

  15. [15]

    Moore, Tiffany D

    Alec G. Moore, Tiffany D. Do, Nayan N. Chawla, Antonia Jimenez Iriarte, and Ryan P. McMahan. 2024. The Full-scale Assembly Simulation Testbed (FAST) Dataset. arXiv: 2403.08969 [cs.HC] doi:10.48550/arXiv.2403.08969

    work page doi:10.48550/arxiv.2403.08969 2024

  16. [16]

    Moore, Tiffany D

    Alec G. Moore, Tiffany D. Do, Nicholas Ruozzi, and Ryan P. McMahan. 2023. Identifying Virtual Reality Users Across Domain-Specific Tasks: A Systematic Investigation of Tracked Features for Assembly. In 2023 IEEE International Sym- posium on Mixed and Augmented Reality (ISMAR) . IEEE, Sydney, Australia, 396–

    work page 2023

  17. [17]

    doi:10.1109/ISMAR59233.2023.00054

    work page doi:10.1109/ismar59233.2023.00054 2023

  18. [18]

    Kevin Musgrave. 2025. PyTorch Metric Learning. https://github.com/ KevinMusgrave/pytorch-metric-learning original-date: 2019-10-23T17:20:35Z

    work page 2025

  19. [19]

    O’Brien, Louis Rosenberg, and Dawn Song

    Vivek Nair, Wenbo Guo, Justus Mattern, Rui Wang, James F. O’Brien, Louis Rosenberg, and Dawn Song. 2023. Unique Identification of 50,000+ Virtual Re- ality Users from Head & Hand Motion Data. 895–910. https://www.usenix.org/ conference/usenixsecurity23/presentation/nair-identification

    work page 2023

  20. [20]

    O’Brien, Louis Rosenberg, and Dawn Song

    Vivek Nair, Wenbo Guo, Rui Wang, James F. O’Brien, Louis Rosenberg, and Dawn Song. 2023. Berkeley Open Extended Reality Recordings 2023 (BOXRR- 23): 4.7 Million Motion Capture Recordings from 105,852 Extended Reality De- vice Users. doi:10.1109/TVCG.2024.3372087 arXiv:2310.00430 [cs]

    work page doi:10.1109/tvcg.2024.3372087 2023

  21. [21]

    Vivek Nair, Mark Roman Miller, Rui Wang, Brandon Huang, Christian Rack, Marc Erich Latoschik, and James F. O’Brien. 2024. Effect of Data Degradation on Motion Re-Identification. In 2024 IEEE 25th International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM). IEEE, Perth, Australia, 85–90. doi:10.1109/WoWMoM60985.2024.00026

    work page doi:10.1109/wowmom60985.2024.00026 2024

  22. [22]

    O’Brien, and Dawn Song

    Vivek Nair, Louis Rosenberg, James F. O’Brien, and Dawn Song. 2024. Truth in Motion: The Unprecedented Risks and Opportunities of Extended Reality Mo- tion Data. IEEE Secur. Privacy 22, 1 (Jan. 2024), 24–32. doi:10.1109/MSEC.2023. 3330392 arXiv:2306.06459 [cs]

    work page doi:10.1109/msec.2023 2024

  23. [23]

    Christian Rack, Tamara Fernando, Murat Yalcin, Andreas Hotho, and Marc Erich Latoschik. 2023. Who is Alyx? A new behavioral biometric dataset for user identification in XR. Front. Virtual Real. 4 (Nov. 2023). doi:10.3389/frvir.2023. 1272234 Publisher: Frontiers

    work page doi:10.3389/frvir.2023 2023

  24. [24]

    Christian Rack, Andreas Hotho, and Marc Erich Latoschik. 2022. Compar- ison of Data Encodings and Machine Learning Architectures for User Iden- tification on Arbitrary Motion Sequences. In 2022 IEEE International Confer- ence on Artificial Intelligence and Virtual Reality (AIVR) . IEEE, CA, USA, 11–19. doi:10.1109/AIVR56993.2022.00010

    work page doi:10.1109/aivr56993.2022.00010 2022

  25. [25]

    Christian Rack, Konstantin Kobs, Tamara Fernando, Andreas Hotho, and Marc Erich Latoschik. 2024. Versatile User Identification in Extended Re- ality using Pretrained Similarity-Learning. http://arxiv.org/abs/2302.07517 arXiv:2302.07517 [cs]

    work page arXiv 2024

  26. [26]

    Christian Rack, Vivek Nair, Lukas Schach, Felix Foschum, Marcel Roth, and Marc Erich Latoschik. 2024. Navigating the Kinematic Maze: Analyzing, Stan- dardizing and Unifying XR Motion Datasets. In 2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) . IEEE, Orlando, FL, USA, 507–514. doi:10.1109/VRW62533.2024.00098

    work page doi:10.1109/vrw62533.2024.00098 2024

  27. [27]

    Christian Rack, Lukas Schach, Felix Achter, Yousof Shehada, Jinghuai Lin, and Marc Erich Latoschik. 2024. Motion Passwords. 11 pages. doi:10.1145/3641825. 3687711

    work page doi:10.1145/3641825 2024

  28. [28]

    Rogers, Alexander W

    Cynthia E. Rogers, Alexander W. Witt, Alexander D. Solomon, and Krishna K. Venkatasubramanian. 2015. An approach for user identification for head- mounted displays. In Proceedings of the 2015 ACM International Symposium on Wearable Computers - ISWC ’15. ACM Press, Osaka, Japan, 143–146. doi:10.1145/ 2802083.2808391

    work page arXiv 2015

  29. [29]

    Thiel and Anthony Steed

    Felix J. Thiel and Anthony Steed. 2022. Developing an Accessibility Metric for VR Games Based on Motion Data Captured Under Game Conditions. Front. Virtual Real. 3 (July 2022), 909357. doi:10.3389/frvir.2022.909357

    work page doi:10.3389/frvir.2022.909357 2022

  30. [30]

    Elliott Wen, Chitralekha Gupta, Prasanth Sasikumar, Mark Billinghurst, James Wilmott, Emily Skow, Arindam Dey, and Suranga Nanayakkara. 2024. VR.net: A Real-world Dataset for Virtual Reality Motion Sickness Research. IEEE Trans. Visual. Comput. Graphics 30, 5 (May 2024), 2330–2336. doi:10.1109/TVCG.2024. 3372044 10

    work page doi:10.1109/tvcg.2024 2024