Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Towards spatial computing: recent advances in multimodal natural interaction for XR headsets

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This review of 104 recent XR interaction papers finds gaze and gesture dominating the field and a large-language-model-driven surge in speech interaction from 2024, organized into a three-level taxonomy.

desk verdict A useful survey taxonomy for 2022–2024 XR natural interaction, but the yearly trend claims rest on an informal corpus and should be rewritten as qualitative observations. read the letter →

arxiv 2502.07598 v2 pith:GSKRAV5M submitted 2025-02-11 cs.HC

Zhimin Wang , Maohang Rao , Shanghua Ye , Weitao Song , Feng Lu This is my paper · ORCID
classification cs.HC
keywords extendedrealitymultimodalinteractionnaturaleyegazehandgesturespeechlargelanguagemodelsspatialcomputing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This review argues that recent natural interaction research for wearable XR headsets can be understood through one structured lens: application scenario, operation type, and interaction modality. The authors classify 104 papers from six top venues published in 2022 through 2024 and find that gaze and gesture research dominates, while speech interaction, especially speech powered by large language models, grew sharply in 2024. The framework matters because it turns a scattered set of prototypes into a map of what interaction designs exist, what operations they support, and where the field is heading. A sympathetic reader would take away that multimodal combinations such as gaze selects and hand manipulates are becoming the default recipe for spatial computing input.

What carries the argument

The organizing device is a three-level taxonomy: application scenario (such as drawing, smart assistant, virtual meeting, or navigation), operation type (pointing and selection, creation and editing, translation and transform, locomotion and viewport, typing and querying, no operation, and passive interaction), and interaction modality (gesture-only, gaze-only, speech-only, tactile, and multimodal pairs or triples such as Gaze+Gesture, Gaze+Speech, Gesture+Speech, and Gaze+Gesture+Speech). The taxonomy separates active interactions from passive feedback channels (visual, acoustic, haptic, and hybrid) and adds a no-operation bucket for recognition-accuracy papers. This machinery does the work of turning 104 heterogeneous papers into comparable categories so that yearly bar charts and trend claims can be drawn from the paper's tables and figures.

What would settle it

Run a systematic search with a documented screening protocol over the same six venues and years, counting speech-only and speech-related multimodal papers per year, and check whether the 2024 spike disappears or reverses.

Watch

Extended reading notes

Core claim

The paper's central claim is that the design space of natural XR interaction is concentrated in a few recurring patterns, and those patterns are visible in the statistics of recent top-venue publications. Concretely, it reports that pointing and selection is the most studied operation, that gaze and gesture are the most studied modalities, that speech-only work increased notably in 2024 likely because LLMs lift the old vocabulary restrictions, and that nearly 70 percent of studies still target general scenarios rather than concrete applications. On the interaction side, the review identifies a dominant division of labor, with gaze for fast pointing, hand for manipulation, and voice for commands and queries, and reads the rise of LLM-based assistants as the main new paradigm of the 2022 to 2024 period.

Load-bearing premise

The yearly trend claims rest on the assumption that the 104 papers gathered through informal Google Scholar keyword searches from six venues fairly represent the field, so if the sample skews, reported trends such as the 2024 speech increase could be artifacts of the search rather than real changes.

Editorial extensions

If this is right

  • The taxonomy gives designers a checklist: any new XR interaction can be placed by scenario, operation, and modality, which makes gaps visible, such as the relative absence of studies targeting medicine or education.
  • The reported dominance of pointing and selection suggests the field is still perfecting the basics before harder operations like creation and locomotion mature.
  • If the 2024 speech increase reflects LLM capability, speech-based interaction should keep expanding beyond keyword commands into open-ended queries and programming by voice.
  • The gaze-selects, hand-manipulates pattern is identified as the primary access method for spatial computing, implying that future headset input design will standardize around multimodal combinations.
  • Because 70 percent of studies lack a concrete application, the paper recommends moving interaction design into real-world scenarios and standardizing interactions across apps to lower learning curves.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The yearly trend lines are sensitive to how the 104 papers were sampled, so the sharp 2024 speech rise should be re-tested with a systematic database search before it is treated as a field-level fact.
  • The taxonomy could serve as a coding scheme for a larger automated corpus analysis in which LLM-based annotation classifies papers by the same dimensions, extending the 104-paper sample without manual review.
  • The review's implicit design principle, assign one modality per sub-task rather than duplicating input, predicts that future headset interfaces will increasingly be multimodal by default, with unimodal techniques reserved for accessibility and hands-busy contexts.
  • The reported gap between lab prototypes and applications suggests the next bottleneck for spatial computing may not be sensing accuracy but interaction standardization across applications.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This review paper surveys 104 papers on natural interaction for wearable XR published between 2022 and 2024 in six venues (ACM CHI, UIST, IMWUT, IEEE VR, ISMAR, TVCG). It proposes a three-dimensional taxonomy (application scenario, operation type, interaction modality), classifies active versus passive interaction, and reports descriptive statistics on modality and operation-type trends. The paper also discusses the role of AI and LLMs in emerging interaction paradigms and outlines research challenges and future directions.

Significance. If its statistical and trend claims held up, the paper would provide a useful structured map of recent XR interaction research and a timely synthesis of the LLM-driven shift in speech interaction. The taxonomy itself is plausible and clearly presented, and the paper's strength is the breadth of its organized reference list (104 papers), which will be a helpful entry point for researchers. The venues selected are appropriate, and the distinction between active and passive interaction is a sensible organizing principle. However, the survey's central trend claims are currently not reproducible from the described methodology, which limits the paper's contribution as a 'systematic review'.

major comments (4)
  1. [Section 2.2, Figures 2 and 3] The statistical claims that answer RQ2 and RQ3—e.g., 'a notable increase in research on Speech-only interaction is observed in 2024' and the 'rise in Speech-related multimodal studies, likely driven by recent advancements in LLMs'—are not supported by the described data collection. Section 2.2 reports only a list of Google Scholar keywords with no query strings, no screening or deduplication protocol, no total number of candidate papers, and no inter-rater reliability for the classification. Because the search was finalized on 16 October 2024, the 2024 counts in Figures 2 and 3 are partial-year counts plotted against full years with no normalization. The observed speech/LLM trends could therefore be artifacts of the retrieval window or search procedure rather than genuine field trends. The authors should either provide a reproducible corpus-building protocol (including query strings, inclusion/exclusion decisions, and PRISMA-style flow) or reframe the statistical sections as descriptive of the assembled corpus rather than of the field.
  2. [Section 3.2 and Table 1] There is an internal inconsistency in the central taxonomy. Table 1's caption states the literature is 'categorized based on six operation types,' and the table lists five operation rows plus a 'No Operation' row. Section 3.2, however, states that the paper 'categorizes XR operations into seven main classes' (the five object/view operations plus No Operation plus Passive Interaction). Since the taxonomy is the paper's main contribution, the number of operation types should be stated consistently, and Table 1 should either include Passive Interaction or explicitly explain why it is separated into Table 2 without changing the count.
  3. [Section 2.3 and Table 1] The modality counts in Section 2.3 (Gesture 24, Gaze 13, Speech 7, Tactile 8, and the multimodal counts) are stated as if they were derived from Table 1, but the table's rows include multiple entries per cell and no count column, making it impossible to verify the sums. For example, the 'No Operation' row in Table 1 contains nine references, and it is unclear whether these papers are included in the modality counts for active interaction. The authors should make the correspondence between Table 1 and the reported counts explicit, or the statistical analysis is not auditable.
  4. [Section 5 (Limitations)] The Limitations section acknowledges the restricted venue and year scope, but it does not address the more serious threats to the paper's trend claims: the lack of a documented screening protocol and the partial-year 2024 corpus. Because the paper explicitly presents statistical insights as a contribution, these limitations should be acknowledged in Section 5, and the affected claims should be softened or removed unless the methodology is strengthened.
minor comments (5)
  1. [Section 3.3.3] The heading 'Summery of current research' is a typo for 'Summary of current research.'
  2. [Section 3.2] The sentence 'Fig. gives an illuatration of these operations' is missing the figure number and contains a typo ('illuatration').
  3. [Section 3.1] The text says 'Fig. 3 show the papers with specific scenarios,' but Figure 3 shows operation-type statistics; the intended reference appears to be Figure 4 or Table 3.
  4. [Section 3.1] The pointer 'Further discussion can be found in Section 4.3' appears inaccurate, since Section 4.3 discusses AI and LLMs rather than application scenarios; the relevant discussion appears to be in Section 4.4.
  5. [Table 5] The table formatting for 'Sampling Rate' uses '1 -' in a way that is unclear; the footnote should clarify that '1' is the marker for the footnote and '-' denotes unreported values.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the taxonomy and trend summaries are descriptive of the reviewed corpus, with no fitted parameter, no self-referential derivation, and no load-bearing self-citation.

full rationale

This paper is a literature survey and taxonomy. Its central contributions are the classification of 104 papers into operation types and interaction modalities and the resulting statistical summaries, which are descriptive statements about the collected corpus rather than predictions derived from an input model. There is no fitted parameter, no equation whose output is an input by construction, and no claim that a derived quantity is predicted from a fitted value. The operation-type and modality categories are introduced as an organizing framework and then applied to the same papers that motivate the framework; this is ordinary survey practice and not circular, since the classification neither assumes nor proves any independent empirical result. The trend claims, such as the 2024 increase in speech-only and speech-related multimodal research, are summaries of the papers listed in Table 1 and Figures 2-3, so they are consistent with the data by design rather than by circular reduction. The authors' own prior works appear among the reviewed papers and are cited as examples within the survey, but none of these citations is load-bearing for the survey's claims; the survey does not invoke any uniqueness theorem or prior result to forbid alternatives or force its taxonomy. The main limitations, acknowledged in Section 5, concern venue scope and search procedure, which are validity or representativeness concerns about the corpus, not circularity. No circular step can be exhibited with a specific reduction, so the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters or invented entities are present. The survey rests on sampling assumptions and on the coding reliability of the authors, which is not independently audited.

assumptions (3)
  • domain assumption The six selected venues are representative of the field's top research.
    Section 2.1 criterion 3 restricts the corpus to ACM CHI, UIST, IMWUT, IEEE VR, ISMAR, and TVCG; the trend statistics in Figures 2 and 3 depend on this assumption.
  • domain assumption Papers that do not explicitly mention wearable XR natural interaction can be excluded without distorting the field-level picture.
    Section 2.1 criteria 1 and 2 set the wearable XR and natural interaction filters, and Section 5 acknowledges that non-wearable work might transfer to XR.
  • domain assumption The classification categories are mutually exclusive and jointly exhaustive enough for statistical comparison.
    Section 2.3 and Table 1 rely on operation type and modality categories, but the paper permits multiple assignments and uses a separate 'No Operation' category, which complicates the counts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards spatial computing: recent advances in multimodal natural interaction for XR headsets." pith.science (2026). https://pith.science/paper/GSKRAV5M

@misc{pith2026250207598,
  author       = {Pith},
  title        = {Pith review of: Towards spatial computing: recent advances in multimodal natural interaction for XR headsets},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GSKRAV5M}},
  note         = {Machine review of arXiv:2502.07598}
}
read the original abstract

With the widespread adoption of Extended Reality (XR) headsets, spatial computing technologies are gaining increasing attention. Spatial computing enables interaction with virtual elements through natural input methods such as eye tracking, hand gestures, and voice commands, thus placing natural human-computer interaction at its core. While previous surveys have reviewed conventional XR interaction techniques, recent advancements in natural interaction, particularly driven by artificial intelligence (AI) and large language models (LLMs), have introduced new paradigms and technologies. In this paper, we review research on multimodal natural interaction for wearable XR, focusing on papers published between 2022 and 2024 in six top venues: ACM CHI, UIST, IMWUT (Ubicomp), IEEE VR, ISMAR, and TVCG. We classify and analyze these studies based on application scenarios, operation types, and interaction modalities. This analysis provides a structured framework for understanding how researchers are designing advanced natural interaction techniques in XR. Based on these findings, we discuss the challenges in natural interaction techniques and suggest potential directions for future research. This review provides valuable insights for researchers aiming to design natural and efficient interaction systems for XR, ultimately contributing to the advancement of spatial computing.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multi-Modal Multi-Task Federated Foundation Models for Next-Generation Extended Reality Systems: Towards Privacy-Preserving Distributed Intelligence in AR/VR/MR

    cs.LG 2025-06 conditional novelty 5.0 of 10

    The paper proposes M3T federated foundation models (FedFMs) as a privacy-preserving architecture for XR and codifies the key challenges as the SHIFT dimensions.

Reference graph

Works this paper leans on

143 extracted references · 78 canonical work pages · cited by 1 Pith paper

  1. [1]

    Microsoft hololens 2 for precise, efficient hands- free work, 2024.10.https://www.microsoft.com/en-us/ hololens

    Company M. Microsoft hololens 2 for precise, efficient hands- free work, 2024.10.https://www.microsoft.com/en-us/ hololens

    2024

  2. [2]

    Meta quest 3: New mixed reality vr headset - tech specs, 2024.10.https://www.meta.com/quest/quest-3/#spec s

    Meta . Meta quest 3: New mixed reality vr headset - tech specs, 2024.10.https://www.meta.com/quest/quest-3/#spec s

    2024

  3. [3]

    Using speech to visualise shared gaze cues in MR remote collaboration

    Jing A, Lee G A, Billinghurst M. Using speech to visualise shared gaze cues in MR remote collaboration. In: IEEE Con- ference on Virtual Reality and 3D User Interfaces, VR 2022, Christchurch, New Zealand, March 12-16, 2022. 2022, 250–259

    2022

  4. [4]

    Handynotes: using the hands to create semantic representations of contex- tually aware real-world objects

    Quere C, Menin A, Julien R, Wu H, Winckler M. Handynotes: using the hands to create semantic representations of contex- tually aware real-world objects. In: IEEE Conference Virtual Reality and 3D User Interfaces, VR 2024, Orlando, FL, USA, March 16-21, 2024. 2024, 265–275

    2024

  5. [5]

    Augmented, mixed, and virtual reality-based head- mounted devices for medical education: Systematic review

    Barteit S, Lanfermann L, B ¨arnighausen T, Neuhann F, Beiers- mann C. Augmented, mixed, and virtual reality-based head- mounted devices for medical education: Systematic review. JMIR Serious Games, 2021, 9(3): e29080

    2021

  6. [6]

    Arcosmetics: a real-time augmented reality cosmetics try-on system

    An S, Chen J, Zhu Z, Zhou F, Yang Y, Ma Y, Liu X, Zhu H. Arcosmetics: a real-time augmented reality cosmetics try-on system. Frontiers of Computer Science, 2023, 17(4): 174706

    2023

  7. [7]

    Introducing apple vision pro: Apple’s first spatial computer, 2024.10.https://www.apple.com/newsroom/2 023/06/introducing-apple-vision-pro/

    Apple . Introducing apple vision pro: Apple’s first spatial computer, 2024.10.https://www.apple.com/newsroom/2 023/06/introducing-apple-vision-pro/

    2024

  8. [8]

    Spatial computing: Concept, applications, challenges and future directions, 2024

    Yenduri G, M R, Maddikunta P K R, Gadekallu T R, Jhaveri R H, Bandi A, Chen J, Wang W, Shirawalmath A A, Ravishankar R, Wang W. Spatial computing: Concept, applications, challenges and future directions, 2024

    2024

Show all 143 references
  1. [9]

    Spatial Computing: An AI-Driven Business Revolution

    Hackl C, Cronin I. Spatial Computing: An AI-Driven Business Revolution. John Wiley & Sons, 2024

    2024

  2. [10]

    Exploring 3d interaction with gaze guidance in augmented reality

    Bao Y, Wang J, Wang Z, Lu F. Exploring 3d interaction with gaze guidance in augmented reality. In: 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR). 2023, 22–32

    2023

  3. [11]

    Did ”minority report” get it wrong? superiority of the mouse over 3d input devices in a 3d placement task

    B ´erard F, Ip J, Benovoy M, El-Shimy D, Blum J R, Cooper- stock J R. Did ”minority report” get it wrong? superiority of the mouse over 3d input devices in a 3d placement task. In: Human-Computer Interaction - INTERACT 2009, 12th IFIP TC 13 International Conference, Uppsala, S...

    2009

  4. [12]

    Shape aware haptic retargeting for accurate hand interactions

    Matthews B J, Thomas B H, Itzstein G S V, Smith R T. Shape aware haptic retargeting for accurate hand interactions. In: IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2022, Christchurch, New Zealand, March 12-16, 2022. 2022, 625–634

    2022

  5. [13]

    Dreamcodevr: Towards democratizing behavior design in virtual reality with speech- driven programming

    Giunchi D, Numan N, Gatti E, Steed A. Dreamcodevr: Towards democratizing behavior design in virtual reality with speech- driven programming. In: IEEE Conference Virtual Reality and 3D User Interfaces, VR 2024, Orlando, FL, USA, March 16-21,

    2024

  6. [14]

    Speech-driven facial anima- tion with spectral gathering and temporal attention

    Chai Y, Weng Y, Wang L, Zhou K. Speech-driven facial anima- tion with spectral gathering and temporal attention. Frontiers of Computer Science, 2022, 16(3): 163703

    2022

  7. [15]

    A survey of sketch based modeling systems

    Ding C, Liu L. A survey of sketch based modeling systems. Frontiers of Computer Science, 2016, 10: 985–999

    2016

  8. [16]

    Matching compound prototypes for few-shot action recognition

    Huang Y, Yang L, Chen G, Zhang H, Lu F, Sato Y. Matching compound prototypes for few-shot action recognition. Interna- tional Journal of Computer Vision, 2024, 1–26

    2024

  9. [17]

    DEAMP: dominant-eye-aware foveated rendering with multi-parameter optimization

    Wang Z, Gu X, Lu F. DEAMP: dominant-eye-aware foveated rendering with multi-parameter optimization. In: Bruder G, Olivier A, Cunningham A, Peng Y E, Grubert J, Williams I, eds, IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2023, Sydney, Australia, October ...

    2023

  10. [18]

    An evaluation of bimanual gestures on the microsoft hololens

    Chaconas N, H ¨ollerer T. An evaluation of bimanual gestures on the microsoft hololens. In: Proc. IEEE Conf. Virtual Real. 3D User Interfaces. Mar. 2018, 33–40

    2018

  11. [19]

    Con- sumed endurance: a metric to quantify arm fatigue of mid-air interactions

    Hincapi ´e-Ramos J D, Guo X, Moghadasian P, Irani P. Con- sumed endurance: a metric to quantify arm fatigue of mid-air interactions. In: Proc. Conf. Human Factors Comput. Syst. Apr. 2014, 1063–1072

    2014

  12. [20]

    Elastica: Adap- tive live augmented presentations with elastic mappings across modalities

    Cao Y, Kazi R H, Wei L, Aneja D, Xia H. Elastica: Adap- tive live augmented presentations with elastic mappings across modalities. In: Mueller F F, Kyburz P, Williamson J R, Sas C, Wilson M L, Dugas P O T, Shklovski I, eds, Proceedings of the CHI Conference on Human Factors in...

    2024

  13. [21]

    LLMR: real-time prompting of interactive worlds using large language models

    Torre F D L, Fang C M, Huang H, Banburski-Fahey A, Fernan- dez J A, Lanier J. LLMR: real-time prompting of interactive worlds using large language models. In: Mueller F F, Kyburz P, Williamson J R, Sas C, Wilson M L, Dugas P O T, Shklovski 22 Front. Comput. Sci., 2025, 0(0): 1...

    2025

  14. [22]

    Weighted pointer: Error-aware gaze- based interaction through fallback modalities

    Sidenmark L, Parent M, Wu C, Chan J, Glueck M, Wigdor D, Grossman T, Giordano M. Weighted pointer: Error-aware gaze- based interaction through fallback modalities. IEEE Trans. Vis. Comput. Graph., 2022, 28(11): 3585–3595

    2022

  15. [23]

    Gaze-hand alignment: Combining eye gaze and mid-air pointing for interacting with menus in augmented reality

    Lystbæk M N, Rosenberg P, Pfeuffer K, Grønbæk J E, Gellersen H. Gaze-hand alignment: Combining eye gaze and mid-air pointing for interacting with menus in augmented reality. Proc. ACM Hum.-Comput. Interact., 2022, 6(ETRA)

    2022

  16. [24]

    Omniactions: Predicting digital actions in response to real-world multimodal sensory inputs with llms

    Li J N, Xu Y, Grossman T, Santosa S, Li M. Omniactions: Predicting digital actions in response to real-world multimodal sensory inputs with llms. In: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI ’24. 2024

    2024

  17. [25]

    Hybrid gaze/eeg brain computer interface for robot arm control on a pick and place task

    Wang H, Dong X, Chen Z, Shi B E. Hybrid gaze/eeg brain computer interface for robot arm control on a pick and place task. In: 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015, Milan, Italy, August 25-29, 2015. 2015, 1476–1479

    2015

  18. [26]

    Wearable augmented reality: Research trends and future directions from three major venues

    Tran T T M, Brown S, Weidlich O, Billinghurst M, Parker C. Wearable augmented reality: Research trends and future directions from three major venues. IEEE Trans. Vis. Comput. Graph., 2023, 29(11): 4782–4793

    2023

  19. [27]

    A taxonomy of interaction techniques for immer- sive augmented reality based on an iterative literature review

    Hertel J, Karaosmanoglu S, Schmidt S, Br ¨aker J, Semmann M, Steinicke F. A taxonomy of interaction techniques for immer- sive augmented reality based on an iterative literature review. In: IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2021, Bari, Italy, O...

    2021

  20. [28]

    The potential of 360 virtual reality videos and real vr for education—a literature review

    Pirker J, Dengel A. The potential of 360 virtual reality videos and real vr for education—a literature review. IEEE computer graphics and applications, 2021, 41(4): 76–89

    2021

  21. [29]

    A review of human–computer interaction and virtual reality research fields in cognitive infocommunications

    Katona J. A review of human–computer interaction and virtual reality research fields in cognitive infocommunications. Applied Sciences, 2021, 11(6): 2646

    2021

  22. [30]

    A survey of immersive visualization: Focus on perception and interaction

    Zhang Y, Wang Z, Zhang J, Shan G, Tian D. A survey of immersive visualization: Focus on perception and interaction. Vis. Informatics, 2023, 7(4): 22–35

    2023

  23. [31]

    A review of interaction techniques for immersive environments

    Spittle B, Pascual M F, Creed C, Williams I. A review of interaction techniques for immersive environments. IEEE Trans. Vis. Comput. Graph., 2023, 29(9): 3900–3921

    2023

  24. [32]

    Virtuwander: Enhancing multi-modal interaction for virtual tour guidance through large language models

    Wang Z, Yuan L, Wang L, Jiang B, Zeng W. Virtuwander: Enhancing multi-modal interaction for virtual tour guidance through large language models. In: Mueller F F, Kyburz P, Williamson J R, Sas C, Wilson M L, Dugas P O T, Shklovski I, eds, Proceedings of the CHI Conference on Hu...

    2024

  25. [33]

    AMMA: adaptive multimodal assistants through automated state tracking and user model-directed guid- ance planning

    Yang J J, Qiu L, Corona-Moreno E A, Shi L, Bui H, Lam M S, Landay J A. AMMA: adaptive multimodal assistants through automated state tracking and user model-directed guid- ance planning. In: IEEE Conference Virtual Reality and 3D User Interfaces, VR 2024, Orlando, FL, USA, March 16-21,

    2024

  26. [34]

    What and how together: A taxonomy on 30 years of collaborative human-centered XR tasks

    Ghamandi R, Hmaiti Y, Nguyen T T, Ghasemaghaei A, Kattoju R K, II E M T, LaViola J J. What and how together: A taxonomy on 30 years of collaborative human-centered XR tasks. In: Bruder G, Olivier A, Cunningham A, Peng Y E, Grubert J, Williams I, eds, IEEE International Symposi...

    2023

  27. [35]

    Wriarm: Leveraging wrist movement to design wrist+arm based teleportation in VR

    Chowdhury S, Ullah A K M A, Pelmore N B, Irani P, Hasan K. Wriarm: Leveraging wrist movement to design wrist+arm based teleportation in VR. In: Duh H B L, Williams I, Grubert J, Jones J A, Zheng J, eds, IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2022, S...

    2022

  28. [36]

    Squeezy-feely: Investigating lateral thumb-index pinching as an input modality

    Schmitz M, G¨ unther S, Sch ¨on D, M¨ uller F. Squeezy-feely: Investigating lateral thumb-index pinching as an input modality. In: Barbosa S D J, Lampe C, Appert C, Shamma D A, Drucker S M, Williamson J R, Yatani K, eds, CHI ’22: CHI Conference on Human Factors in Computing Sy...

    2022

  29. [37]

    Wormholes in VR: teleporting hands for flexible passive haptics

    Ban R, Matsumoto K, Narumi T, Kuzuoka H. Wormholes in VR: teleporting hands for flexible passive haptics. In: IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2022, Singapore, October 17-21, 2022. 2022, 748–757

    2022

  30. [38]

    Blending on-body and mid-air interaction in virtual reality

    Yu D, Zhou Q, Dingler T, Velloso E, Gonc ¸alves J. Blending on-body and mid-air interaction in virtual reality. In: Duh H B L, Williams I, Grubert J, Jones J A, Zheng J, eds, IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2022, Singapore, October 17-21, 202...

    2022

  31. [39]

    Vrdoc: Gaze-based interactions for VR reading experience

    Lee G, Healey J, Manocha D. Vrdoc: Gaze-based interactions for VR reading experience. In: Duh H B L, Williams I, Grubert J, Jones J A, Zheng J, eds, IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2022, Singapore, October 17-21, 2022. 2022, 787–796

    2022

  32. [40]

    Kuiper belt: Utilizing the ”out- of-natural angle” region in the eye-gaze interaction for virtual reality

    Choi M, Sakamoto D, Ono T. Kuiper belt: Utilizing the ”out- of-natural angle” region in the eye-gaze interaction for virtual reality. In: Barbosa S D J, Lampe C, Appert C, Shamma D A, Drucker S M, Williamson J R, Yatani K, eds, CHI ’22: CHI Conference on Human Factors in Compu...

    2022

  33. [41]

    Lattice menu: A low-error gaze- based marking menu utilizing target-assisted gaze gestures on a lattice of visual anchors

    Kim T, Ham A, Ahn S, Lee G. Lattice menu: A low-error gaze- based marking menu utilizing target-assisted gaze gestures on a lattice of visual anchors. In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 2022, 1–12

    2022

  34. [42]

    Gazedock: Gaze-only menu selection in virtual reality using auto-triggering peripheral menu

    Yi X, Lu Y, Cai Z, Wu Z, Wang Y, Shi Y. Gazedock: Gaze-only menu selection in virtual reality using auto-triggering peripheral menu. In: IEEE Conference on Virtual Reality and 3D User Zhimin Wang et al. Recent advances in multimodal natural interaction for XR headsets 23 Inter...

    2022

  35. [43]

    Gaze-vergence-controlled see-through vision in augmented reality

    Wang Z, Zhao Y, Lu F. Gaze-vergence-controlled see-through vision in augmented reality. IEEE Trans. Vis. Comput. Graph., 2022, 28(11): 3843–3853

    2022

  36. [44]

    Efring: Enabling thumb-to-index- finger microgesture interaction through electric field sensing using single smart ring

    Chen T, Li T, Yang X, Zhu K. Efring: Enabling thumb-to-index- finger microgesture interaction through electric field sensing using single smart ring. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 2022, 6(4): 161:1–161:31

    2022

  37. [45]

    Detecting input recognition errors and user errors using gaze dynamics in virtual reality

    Sendhilnathan N, Zhang T, Lafreniere B, Grossman T, Jonker T R. Detecting input recognition errors and user errors using gaze dynamics in virtual reality. In: Agrawala M, Wobbrock J O, Adar E, Setlur V, eds, The 35th Annual ACM Symposium on User Interface Software and Technolo...

    2022

  38. [46]

    Realitytalk: Real-time speech-driven augmented presentation for AR live storytelling

    Liao J, Karim A, Jadon S S, Kazi R H, Suzuki R. Realitytalk: Real-time speech-driven augmented presentation for AR live storytelling. In: Agrawala M, Wobbrock J O, Adar E, Setlur V, eds, The 35th Annual ACM Symposium on User Interface Software and Technology, UIST 2022, Bend, ...

    2022

  39. [47]

    DEEP: 3d gaze pointing in virtual reality leveraging eyelid movement

    Yi X, Qiu L, Tang W, Fan Y, Li H, Shi Y. DEEP: 3d gaze pointing in virtual reality leveraging eyelid movement. In: Agrawala M, Wobbrock J O, Adar E, Setlur V, eds, The 35th Annual ACM Symposium on User Interface Software and Technology, UIST 2022, Bend, OR, USA, 29 October 202...

    2022

  40. [48]

    Evaluation of text selection techniques in virtual reality head-mounted displays

    Xu W, Meng X, Yu K, Sarcar S, Liang H. Evaluation of text selection techniques in virtual reality head-mounted displays. In: Duh H B L, Williams I, Grubert J, Jones J A, Zheng J, eds, IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2022, Singapore, October 1...

    2022

  41. [49]

    Clenchclick: Hands-free target selection method leveraging teeth-clench for augmented reality

    Shen X, Yan Y, Yu C, Shi Y. Clenchclick: Hands-free target selection method leveraging teeth-clench for augmented reality. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 2022, 6(3): 139:1–139:26

    2022

  42. [50]

    An exploration of hands-free text selection for virtual reality head-mounted displays

    Meng X, Xu W, Liang H. An exploration of hands-free text selection for virtual reality head-mounted displays. In: Duh H B L, Williams I, Grubert J, Jones J A, Zheng J, eds, IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2022, Singapore, October 17-21, 2022....

    2022

  43. [51]

    Fingerbutton: Enabling controller- free transitions between real and virtual environments

    Das S, Nasser A, Hasan K. Fingerbutton: Enabling controller- free transitions between real and virtual environments. In: Bruder G, Olivier A, Cunningham A, Peng Y E, Grubert J, Williams I, eds, IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2023, Sydney, Au...

    2023

  44. [52]

    Pinchlens: Ap- plying spatial magnification and adaptive control-display gain for precise selection in virtual reality

    Zhu F, Sidenmark L, Sousa M, Grossman T. Pinchlens: Ap- plying spatial magnification and adaptive control-display gain for precise selection in virtual reality. In: Bruder G, Olivier A, Cunningham A, Peng Y E, Grubert J, Williams I, eds, IEEE International Symposium on Mixed a...

    2023

  45. [53]

    Hotgestures: Comple- menting command selection and use with delimiter-free gesture- based shortcuts in virtual reality

    Song Z, Dudley J J, Kristensson P O. Hotgestures: Comple- menting command selection and use with delimiter-free gesture- based shortcuts in virtual reality. IEEE Trans. Vis. Comput. Graph., 2023, 29(11): 4600–4610

    2023

  46. [54]

    Fingermapper: Mapping finger motions onto virtual arms to enable safe virtual reality interaction in confined spaces

    Tseng W, Huron S, Lecolinet E, Gugenheimer J. Fingermapper: Mapping finger motions onto virtual arms to enable safe virtual reality interaction in confined spaces. In: Schmidt A, V¨a¨an¨anen K, Goyal T, Kristensson P O, Peters A, Mueller S, Williamson J R, Wilson M L, eds, Pro...

    2023

  47. [55]

    Leap to the eye: Implicit gaze-based interaction to reveal invisible objects for virtual environment exploration

    Chen Y, Hsieh C, Then M Y J, Han P, Hung Y. Leap to the eye: Implicit gaze-based interaction to reveal invisible objects for virtual environment exploration. In: Bruder G, Olivier A, Cunningham A, Peng Y E, Grubert J, Williams I, eds, IEEE International Symposium on Mixed and ...

    2023

  48. [56]

    Vergence matching: Inferring attention to objects in 3d environments for gaze-assisted selection

    Sidenmark L, Clarke C, Newn J, Lystbæk M N, Pfeuffer K, Gellersen H. Vergence matching: Inferring attention to objects in 3d environments for gaze-assisted selection. In: Schmidt A, V¨a¨an¨anen K, Goyal T, Kristensson P O, Peters A, Mueller S, Williamson J R, Wilson M L, eds, ...

    2023

  49. [57]

    A fitts’ law study of gaze-hand alignment for selection in 3d user interfaces

    Wagner U, Lystbæk M N, Manakhov P, Grønbæk J E S, Pfeuffer K, Gellersen H. A fitts’ law study of gaze-hand alignment for selection in 3d user interfaces. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI ’23. 2023

    2023

  50. [58]

    Exploring gaze- assisted and hand-based region selection in augmented reality

    Shi R, Wei Y, Qin X, Hui P, Liang H N. Exploring gaze- assisted and hand-based region selection in augmented reality. Proc. ACM Hum.-Comput. Interact., 2023, 7(ETRA)

    2023

  51. [59]

    3d selection in mixed reality: Designing a two-phase technique to reduce fatigue

    Caillet A C, Goguey A, Nigay L. 3d selection in mixed reality: Designing a two-phase technique to reduce fatigue. In: Bruder G, Olivier A, Cunningham A, Peng Y E, Grubert J, Williams I, eds, IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2023, Sydney, Austr...

    2023

  52. [60]

    Compass+ring: A multimodal menu to improve interaction performance and comfortability in one-handed scenarios

    Chen X, Guo D, Feng L, Chen B, Liu W. Compass+ring: A multimodal menu to improve interaction performance and comfortability in one-handed scenarios. In: Bruder G, Olivier A, Cunningham A, Peng Y E, Grubert J, Williams I, eds, IEEE International Symposium on Mixed and Augmented...

    2023

  53. [61]

    Predict- 24 Front

    Wei Y, Shi R, Yu D, Wang Y, Li Y, Yu L, Liang H. Predict- 24 Front. Comput. Sci., 2025, 0(0): 1–28 ing gaze-based target selection in augmented reality headsets based on eye and head endpoint distributions. In: Schmidt A, V¨a¨an¨anen K, Goyal T, Kristensson P O, Peters A, Muel...

    2025

  54. [62]

    Classifying head movements to separate head- gaze and head gestures as distinct modes of input

    Hou B J, Newn J, Sidenmark L, Khan A A, Bækgaard P, Gellersen H. Classifying head movements to separate head- gaze and head gestures as distinct modes of input. In: Schmidt A, V¨a¨an¨anen K, Goyal T, Kristensson P O, Peters A, Mueller S, Williamson J R, Wilson M L, eds, Procee...

    2023

  55. [63]

    Conespeech: Exploring directional speech interaction for multi-person remote communication in virtual reality

    Yan Y, Liu H, Shi Y, Wang J, Guo R, Li Z, Xu X, Yu C, Wang Y, Shi Y. Conespeech: Exploring directional speech interaction for multi-person remote communication in virtual reality. IEEE Transactions on Visualization and Computer Graphics, 2023, 29(5): 2647–2657

    2023

  56. [64]

    Exploring bi-manual teleportation in virtual reality

    Sindhupathiraja S R, Ullah A K M A, Delamare W, Hasan K. Exploring bi-manual teleportation in virtual reality. In: IEEE Conference Virtual Reality and 3D User Interfaces, VR 2024, Orlando, FL, USA, March 16-21, 2024. 2024, 754–764

    2024

  57. [65]

    Tripad: Touch input in AR on ordinary surfaces with hand tracking only

    Dupr ´e C, Appert C, Rey S, Saidi H, Pietriga E. Tripad: Touch input in AR on ordinary surfaces with hand tracking only. In: Mueller F F, Kyburz P, Williamson J R, Sas C, Wilson M L, Dugas P O T, Shklovski I, eds, Proceedings of the CHI Con- ference on Human Factors in Computi...

    2024

  58. [66]

    Eye- shadows: Peripheral virtual copies for rapid gaze selection and interaction

    Orlosky J, Liu C, Sakamoto K, Sidenmark L, Mansour A. Eye- shadows: Peripheral virtual copies for rapid gaze selection and interaction. In: IEEE Conference Virtual Reality and 3D User Interfaces, VR 2024, Orlando, FL, USA, March 16-21, 2024. 2024, 681–689

    2024

  59. [67]

    Focusflow: 3d gaze- depth interaction in virtual reality leveraging active visual depth manipulation

    Zhang C, Chen T, Shaffer E, Soltanaghai E. Focusflow: 3d gaze- depth interaction in virtual reality leveraging active visual depth manipulation. In: Mueller F F, Kyburz P, Williamson J R, Sas C, Wilson M L, Dugas P O T, Shklovski I, eds, Proceedings of the CHI Conference on Hu...

    2024

  60. [68]

    Eyeguide & eye- conguide: Gaze-based visual guides to improve 3d sketching systems

    T¨ urkmen R, Gelmez Z E, Batmaz A U, Stuerzlinger W, Asente P, Sarac M, Pfeuffer K, Machuca M D B. Eyeguide & eye- conguide: Gaze-based visual guides to improve 3d sketching systems. In: Mueller F F, Kyburz P, Williamson J R, Sas C, Wilson M L, Dugas P O T, Shklovski I, eds, P...

    2024

  61. [69]

    Beyond the blink: Investigating combined saccadic & blink-suppressed hand redirection in virtual reality

    Zenner A, Karr C, Feick M, Ariza O, Kr¨ uger A. Beyond the blink: Investigating combined saccadic & blink-suppressed hand redirection in virtual reality. In: Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI 2024, Honolulu, HI, USA, May 11-16, 2024. ...

    2024

  62. [70]

    An artists’ perspectives on natural interactions for virtual reality 3d sketching

    Rodriguez R, Sullivan B T, Barrera Machuca M D, Batmaz A U, Tornatzky C, Ortega F R. An artists’ perspectives on natural interactions for virtual reality 3d sketching. In: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI ’24. 2024

    2024

  63. [71]

    Gazepuffer: Hands-free input method leveraging puff cheeks for VR

    Lai Y, Sun M, Li Z. Gazepuffer: Hands-free input method leveraging puff cheeks for VR. In: IEEE Conference Virtual Reality and 3D User Interfaces, VR 2024, Orlando, FL, USA, March 16-21, 2024. 2024, 331–341

    2024

  64. [72]

    Selection performance and reliability of eye and head gaze tracking under varying light conditions

    Marquardt A, Steininger M, Trepkowski C, Weier M, Kruijff E. Selection performance and reliability of eye and head gaze tracking under varying light conditions. In: 2024 IEEE Con- ference Virtual Reality and 3D User Interfaces (VR). 2024, 546–556

    2024

  65. [73]

    Hand interfaces: Using hands to imitate objects in AR/VR for expressive interactions

    Pei S, Chen A, Lee J, Zhang Y. Hand interfaces: Using hands to imitate objects in AR/VR for expressive interactions. In: Barbosa S D J, Lampe C, Appert C, Shamma D A, Drucker S M, Williamson J R, Yatani K, eds, CHI ’22: CHI Conference on Human Factors in Computing Systems, New...

    2022

  66. [74]

    Tangible globes for data visualisation in augmented reality

    Satriadi K A, Smiley J, Ens B, Cordeil M, Czauderna T, Lee B, Yang Y, Dwyer T, Jenny B. Tangible globes for data visualisation in augmented reality. In: Barbosa S D J, Lampe C, Appert C, Shamma D A, Drucker S M, Williamson J R, Yatani K, eds, CHI ’22: CHI Conference on Human F...

    2022

  67. [75]

    Gesturesurface: VR sketching through assembling scaffold surface with non- dominant hand

    Xu X, Zhou Y, Shao B, Feng G, Yu C. Gesturesurface: VR sketching through assembling scaffold surface with non- dominant hand. IEEE Trans. Vis. Comput. Graph., 2023, 29(5): 2499–2507

    2023

  68. [76]

    Toucheditor: Interaction design and evaluation of a flexible touchpad for text editing of head-mounted displays in speech- unfriendly environments

    Zhan L, Xiong T, Zhang H, Guo S, Chen X, Gong J, Lin J, Qin Y. Toucheditor: Interaction design and evaluation of a flexible touchpad for text editing of head-mounted displays in speech- unfriendly environments. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 2023, 7(4):...

    2023

  69. [77]

    The impacts of referent display on gesture and speech elicitation

    Williams A S, Ortega F R. The impacts of referent display on gesture and speech elicitation. IEEE Transactions on Visualiza- tion and Computer Graphics, 2022, 28(11): 3885–3895

    2022

  70. [78]

    Dual-gain mode of head-gaze interaction improves the efficiency of object positioning in a 3d virtual environment

    Deng C, Sun L, Zhou C, Kuai S. Dual-gain mode of head-gaze interaction improves the efficiency of object positioning in a 3d virtual environment. Int. J. Hum. Comput. Interact., 2024, 40(8): 2067–2082

    2024

  71. [79]

    Tell me where to go: Voice-controlled hands-free locomotion for virtual reality systems

    Hombeck J N, Voigt H, Heggemann T, Datta R R, Lawonn K. Tell me where to go: Voice-controlled hands-free locomotion for virtual reality systems. In: IEEE Conference Virtual Reality and 3D User Interfaces, VR 2023, Shanghai, China, March 25-29, 2023. 2023, 123–134 Zhimin Wang e...

    2023

  72. [80]

    Co- bodeck: A large-scale haptic VR system using a collaborative mobile robot

    Mortezapoor S, Vasylevska K, Vonach E, Kaufmann H. Co- bodeck: A large-scale haptic VR system using a collaborative mobile robot. In: IEEE Conference Virtual Reality and 3D User Interfaces, VR 2023, Shanghai, China, March 25-29, 2023. 2023, 297–307

    2023

  73. [81]

    illumotion: An optical-illusion-based VR locomotion technique for long- distance 3d movement

    Sin Z P T, Jia Y, Li R C, Leong H V, Li Q, Ng P H F. illumotion: An optical-illusion-based VR locomotion technique for long- distance 3d movement. In: IEEE Conference Virtual Reality and 3D User Interfaces, VR 2024, Orlando, FL, USA, March 16-21, 2024. 2024, 924–934

    2024

  74. [82]

    The rayhand navigation: A virtual navigation method with relative position between hand and gaze-ray

    Kang S, Jeong J, Lee G A, Kim S H, Yang H J, Kim S. The rayhand navigation: A virtual navigation method with relative position between hand and gaze-ray. In: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI ’24. 2024

    2024

  75. [83]

    Snap, pursuit and gain: Virtual reality viewport control by gaze

    Lee H S, Weidner F, Sidenmark L, Gellersen H. Snap, pursuit and gain: Virtual reality viewport control by gaze. In: Mueller F F, Kyburz P, Williamson J R, Sas C, Wilson M L, Dugas P O T, Shklovski I, eds, Proceedings of the CHI Conference on Human Factors in Computing Systems,...

    2024

  76. [84]

    Shapefindar: Exploring in-situ spatial search for physical artifact retrieval us- ing mixed reality

    Stemasov E, Wagner T, Gugenheimer J, Rukzio E. Shapefindar: Exploring in-situ spatial search for physical artifact retrieval us- ing mixed reality. In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22. 2022

    2022

  77. [85]

    Efficient special char- acter entry on a virtual keyboard by hand gesture-based mode switching

    Song Z, Dudley J J, Kristensson P O. Efficient special char- acter entry on a virtual keyboard by hand gesture-based mode switching. In: IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2022, Singapore, October 17-21,

    2022

  78. [86]

    Tapgazer: Text entry with finger tapping and gaze-directed word selection

    He Z, Lutteroth C, Perlin K. Tapgazer: Text entry with finger tapping and gaze-directed word selection. In: Barbosa S D J, Lampe C, Appert C, Shamma D A, Drucker S M, Williamson J R, Yatani K, eds, CHI ’22: CHI Conference on Human Factors in Computing Systems, New Orleans, LA,...

    2022

  79. [87]

    Fast and robust mid-air gesture typing for AR headsets using 3d trajectory decoding

    Shen J, Dudley J J, Kristensson P O. Fast and robust mid-air gesture typing for AR headsets using 3d trajectory decoding. IEEE Trans. Vis. Comput. Graph., 2023, 29(11): 4622–4632

    2023

  80. [88]

    Gaze speedup: Eye gaze assisted gesture typing in virtual reality

    Zhao M, Pierce A M, Tan R, Zhang T, Wang T, Jonker T R, Benko H, Gupta A. Gaze speedup: Eye gaze assisted gesture typing in virtual reality. In: Proceedings of the 28th Inter- national Conference on Intelligent User Interfaces, IUI 2023, Sydney, NSW, Australia, March 27-31, 20...

    2023

  81. [89]

    Glancewriter: Writing text by glancing over letters with gaze

    Cui W, Liu R, Li Z, Wang Y, Wang A, Zhao X, Rashidian S, Baig F, Ramakrishnan I, Wang F, others . Glancewriter: Writing text by glancing over letters with gaze. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 2023, 1–13

    2023

  82. [90]

    Echospeech: Continuous silent speech recognition on minimally-obtrusive eyewear powered by acoustic sensing

    Zhang R, Li K, Hao Y, Wang Y, Lai Z, Guimbreti `ere F, Zhang C. Echospeech: Continuous silent speech recognition on minimally-obtrusive eyewear powered by acoustic sensing. In: Schmidt A, V ¨a¨an¨anen K, Goyal T, Kristensson P O, Peters A, Mueller S, Williamson J R, Wilson M L...

    2023

  83. [91]

    Ringgesture: A ring-based mid-air gesture typing system powered by a deep-learning word prediction framework

    Shen J, Boldu R, Kalla A, Glueck M, Surale H B, Karlson A. Ringgesture: A ring-based mid-air gesture typing system powered by a deep-learning word prediction framework. IEEE Transactions on Visualization and Computer Graphics, 2024

    2024

  84. [92]

    Skimr: Dwell-free eye typing in mixed reality

    Hu J, Dudley J J, Kristensson P O. Skimr: Dwell-free eye typing in mixed reality. In: IEEE Conference Virtual Reality and 3D User Interfaces, VR 2024, Orlando, FL, USA, March 16-21,

    2024

  85. [93]

    Robust dual-modal speech keyword spotting for XR headsets

    Cai Z, Ma Y, Lu F. Robust dual-modal speech keyword spotting for XR headsets. IEEE Trans. Vis. Comput. Graph., 2024, 30(5): 2507–2516

    2024

  86. [94]

    Eye-hand typing: Eye gaze assisted finger typing via bayesian processes in ar

    Ren Y, Zhang Y, Liu Z, Xie N. Eye-hand typing: Eye gaze assisted finger typing via bayesian processes in ar. IEEE Trans- actions on Visualization and Computer Graphics, 2024, 30(5): 2496–2506

    2024

  87. [95]

    Augmented conversation with embedded speech-driven on-the- fly referencing in AR

    Jadon S S, Faridan M, Mah E, Vaish R, Willett W, Suzuki R. Augmented conversation with embedded speech-driven on-the- fly referencing in AR. CoRR, 2024, abs/2405.18537

    2024 arXiv

  88. [96]

    G-VOILA: gaze-facilitated information querying in daily sce- narios

    Wang Z, Shi Y, Wang Y, Yao Y, Yan K, Wang Y, Ji L, Xu X, Yu C. G-VOILA: gaze-facilitated information querying in daily sce- narios. Proc. ACM Interact. Mob. Wearable Ubiquitous Tech- nol., 2024, 8(2): 78:1–78:33

    2024

  89. [97]

    Gazepointar: A context-aware multimodal voice assistant for pronoun disambiguation in wearable augmented reality

    Lee J, Wang J, Brown E, Chu L, Rodriguez S S, Froehlich J E. Gazepointar: A context-aware multimodal voice assistant for pronoun disambiguation in wearable augmented reality. In: Mueller F F, Kyburz P, Williamson J R, Sas C, Wilson M L, Dugas P O T, Shklovski I, eds, Proceedin...

    2024

  90. [98]

    Exploring visualizations for pre- cisely guiding bare hand gestures in virtual reality

    Wang X, Lafreniere B, Zhao J. Exploring visualizations for pre- cisely guiding bare hand gestures in virtual reality. In: Mueller F F, Kyburz P, Williamson J R, Sas C, Wilson M L, Dugas P O T, Shklovski I, eds, Proceedings of the CHI Conference on Human Factors in Computing Sy...

    2024

  91. [99]

    Gesture spot- ter: A rapid prototyping tool for key gesture spotting in virtual and augmented reality applications

    Shen J, Dudley J J, Mo G B, Kristensson P O. Gesture spot- ter: A rapid prototyping tool for key gesture spotting in virtual and augmented reality applications. IEEE Trans. Vis. Comput. Graph., 2022, 28(11): 3618–3628

    2022

  92. [100]

    Echowrist: Continuous hand pose tracking and hand-object interaction recognition using low-power active acoustic sensing on a wristband

    Lee C, Zhang R, Agarwal D, Yu T C, Gunda V, Lopez O, Kim J, Yin S, Dong B, Li K, Sakashita M, Guimbreti `ere F, Zhang C. Echowrist: Continuous hand pose tracking and hand-object interaction recognition using low-power active acoustic sensing on a wristband. In: Proceedings of ...

    2024

  93. [101]

    Authentication 26 Front

    Rupp D, GrieBer P, B ¨onsch A, Kuhlen T W. Authentication 26 Front. Comput. Sci., 2025, 0(0): 1–28 in immersive virtual environments through gesture-based inter- action with a virtual agent. In: IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops,...

    2025

  94. [102]

    Audiogest: Gesture-based interaction for virtual reality using audio devices

    Liu T, Xiao Y, Hu M, Sha H, Ma S, Gao B, Guo S, Liu Y, Song W. Audiogest: Gesture-based interaction for virtual reality using audio devices. IEEE Transactions on Visualization and Computer Graphics, 2024, 1–13

    2024

  95. [103]

    Ao-finger: Hands-free fine-grained finger gesture recognition via acoustic-optic sensor fusing

    Xu C, Zhou B, Krishnan G, Nayar S K. Ao-finger: Hands-free fine-grained finger gesture recognition via acoustic-optic sensor fusing. In: Schmidt A, V ¨a¨an¨anen K, Goyal T, Kristensson P O, Peters A, Mueller S, Williamson J R, Wilson M L, eds, Proceedings of the 2023 CHI Confe...

    2023

  96. [104]

    Touchlog: Finger micro gesture recognition using photo-reflective sensors

    Kitamura R, Yamamoto T, Sugiura Y. Touchlog: Finger micro gesture recognition using photo-reflective sensors. In: Tentori M, Weibel N, Laerhoven K V, Zhou Z, eds, Proceedings of the 2023 International Symposium on Wearable Computers, ISWC 2023, Cancun, Quintana Roo, Mexico, Oc...

    2023

  97. [105]

    Nailring: An intelligent ring for recognizing micro-gestures in mixed reality

    Li T, Liu Y, Ma S, Hu M, Liu T, Song W. Nailring: An intelligent ring for recognizing micro-gestures in mixed reality. In: Duh H B L, Williams I, Grubert J, Jones J A, Zheng J, eds, IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2022, Singapore, October 17-...

    2022

  98. [106]

    You spin me right round, baby, right round: Examining the impact of multi-sensory self-motion cues on motion sickness during a VR reading task

    P ¨ohlmann K M T, Li G, McGill M, Markoff R, Brewster S A. You spin me right round, baby, right round: Examining the impact of multi-sensory self-motion cues on motion sickness during a VR reading task. In: Schmidt A, V ¨a¨an¨anen K, Goyal T, Kristensson P O, Peters A, Mueller...

    2023

  99. [107]

    Behind the scenes: Adapting cinematography and editing concepts to navigation in virtual reality

    Medlar A, Lehtikari M T, Glowacka D. Behind the scenes: Adapting cinematography and editing concepts to navigation in virtual reality. In: Mueller F F, Kyburz P, Williamson J R, Sas C, Wilson M L, Dugas P O T, Shklovski I, eds, Proceedings of the CHI Conference on Human Factor...

    2024

  100. [108]

    Designing visuo- haptic illusions with proxies in virtual reality: Exploration of grasp, movement trajectory and object mass

    Feick M, Regitz K P, Tang A, Kr¨ uger A. Designing visuo- haptic illusions with proxies in virtual reality: Exploration of grasp, movement trajectory and object mass. In: Barbosa S D J, Lampe C, Appert C, Shamma D A, Drucker S M, Williamson J R, Yatani K, eds, CHI ’22: CHI Con...

    2022

  101. [109]

    Artist: Automated text simplification for task guidance in augmented reality

    Wu G, Qian J, Quispe S C, Chen S, Rulff J, Silva C T. Artist: Automated text simplification for task guidance in augmented reality. In: Mueller F F, Kyburz P, Williamson J R, Sas C, Wilson M L, Dugas P O T, Shklovski I, eds, Proceedings of the CHI Conference on Human Factors i...

    2024

  102. [110]

    SYNC-VR: synchronizing your senses to conquer motion sickness for enriching in-vehicle virtual reality

    Elsharkawy A I A M, Ataya A A S, Yeo D, An E, Hwang S, Kim S. SYNC-VR: synchronizing your senses to conquer motion sickness for enriching in-vehicle virtual reality. In: Mueller F F, Kyburz P, Williamson J R, Sas C, Wilson M L, Dugas P O T, Shklovski I, eds, Proceedings of the...

    2024

  103. [111]

    Dynamic scene adjustment mechanism for manipulating user experience in VR

    Li Y, Liu Z, Yuan L, Tang H, Fan Y, Xie N. Dynamic scene adjustment mechanism for manipulating user experience in VR. In: IEEE Conference Virtual Reality and 3D User Interfaces, VR 2024, Orlando, FL, USA, March 16-21, 2024. 2024, 179– 188

    2024

  104. [112]

    Going, going, gone: Exploring intention communication for multi-user loco- motion in virtual reality

    Rasch J, Rusakov V D, Schmitz M, M¨ uller F. Going, going, gone: Exploring intention communication for multi-user loco- motion in virtual reality. In: Schmidt A, V ¨a¨an¨anen K, Goyal T, Kristensson P O, Peters A, Mueller S, Williamson J R, Wilson M L, eds, Proceedings of the ...

    2023

  105. [113]

    Audioxtend: Assisted reality visual accompaniments for au- diobook storytelling during everyday routine tasks

    Tan F F, Xu P, Ram A, Suen W Z, Zhao S, Huang Y, Hurter C. Audioxtend: Assisted reality visual accompaniments for au- diobook storytelling during everyday routine tasks. In: Mueller F F, Kyburz P, Williamson J R, Sas C, Wilson M L, Dugas P O T, Shklovski I, eds, Proceedings of...

    2024

  106. [114]

    A3RT: attention-aware AR teleconfer- encing with life-size 2.5d video avatars

    Wang X, Zhang W, Fu H. A3RT: attention-aware AR teleconfer- encing with life-size 2.5d video avatars. In: IEEE Conference Virtual Reality and 3D User Interfaces, VR 2024, Orlando, FL, USA, March 16-21, 2024. 2024, 211–221

    2024

  107. [115]

    Integrating real-world distractions into virtual reality

    Tao Y, Lopes P. Integrating real-world distractions into virtual reality. In: Agrawala M, Wobbrock J O, Adar E, Setlur V, eds, The 35th Annual ACM Symposium on User Interface Software and Technology, UIST 2022, Bend, OR, USA, 29 October 2022 - 2 November 2022. 2022, 5:1–5:16

    2022

  108. [116]

    may i speak?

    Lee G, Lee D Y, Su G M, Manocha D. “may i speak?”: Multi- modal attention guidance in social vr group conversations. IEEE Transactions on Visualization and Computer Graphics, 2024, 30(5): 2287–2297

    2024

  109. [117]

    Tasks reflected in the eyes: Egocentric gaze- aware visual task type recognition in virtual reality

    Wang Z, Lu F. Tasks reflected in the eyes: Egocentric gaze- aware visual task type recognition in virtual reality. IEEE Trans- actions on Visualization and Computer Graphics, 2024

    2024

  110. [118]

    Mouth haptics in VR using a headset ultrasound phased array

    Shen V, Shultz C D, Harrison C. Mouth haptics in VR using a headset ultrasound phased array. In: Barbosa S D J, Lampe C, Appert C, Shamma D A, Drucker S M, Williamson J R, Yatani K, eds, CHI ’22: CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 29 Ap...

    2022

  111. [119]

    Airres mask: A precise and robust virtual reality breathing interface utilizing breathing resistance as output modality

    Tatzgern M, Domhardt M, Wolf M, Cenger M, Emsenhuber G, Dinic R, Gerner N, Hartl A. Airres mask: A precise and robust virtual reality breathing interface utilizing breathing resistance as output modality. In: Barbosa S D J, Lampe C, Appert C, Shamma D A, Drucker S M, Williamso...

    2022

  112. [120]

    Big or small, it’s all in your head: Visuo-haptic illusion of size- change using finger-repositioning

    Kim M J, Ofek E, Pahud M, Sinclair M J, Bianchi A. Big or small, it’s all in your head: Visuo-haptic illusion of size- change using finger-repositioning. In: Mueller F F, Kyburz P, Williamson J R, Sas C, Wilson M L, Dugas P O T, Shklovski I, eds, Proceedings of the CHI Confere...

    2024

  113. [121]

    Providing 3d guidance and improving the music-listening experience in virtual reality shooting games using musical vibrotactile feedback

    Yamazaki Y, Hasegawa S. Providing 3d guidance and improving the music-listening experience in virtual reality shooting games using musical vibrotactile feedback. In: IEEE Conference Vir- tual Reality and 3D User Interfaces, VR 2023, Shanghai, China, March 25-29, 2023. 2023, 276–285

    2023

  114. [122]

    Fluid reality: High-resolution, untethered haptic gloves using electroosmotic pump arrays

    Shen V, Rae-Grant T, Mullenbach J, Harrison C, Shultz C D. Fluid reality: High-resolution, untethered haptic gloves using electroosmotic pump arrays. In: Follmer S, Han J, Steimle J, Riche N H, eds, Proceedings of the 36th Annual ACM Sympo- sium on User Interface Software and ...

    2023

  115. [123]

    Double-sided tactile interactions for grasping in virtual reality

    Jingu A, Withana A, Steimle J. Double-sided tactile interactions for grasping in virtual reality. In: Follmer S, Han J, Steimle J, Riche N H, eds, Proceedings of the 36th Annual ACM Sympo- sium on User Interface Software and Technology, UIST 2023, San Francisco, CA, USA, 29 Oc...

    2023

  116. [124]

    Persuasive vibrations: Effects of speech-based vibrations on persuasion, leadership, and co-presence during verbal communication in VR

    Saint-Aubert J, Argelaguet F, Mac ´e M J, Pacchierotti C, Amedi A, L´ecuyer A. Persuasive vibrations: Effects of speech-based vibrations on persuasion, leadership, and co-presence during verbal communication in VR. In: IEEE Conference Virtual Reality and 3D User Interfaces, VR...

    2023

  117. [125]

    Hummer: Text en- try by gaze and hum

    Hedeshy R, Kumar C, Menges R, Staab S. Hummer: Text en- try by gaze and hum. In: Kitamura Y, Quigley A, Isbister K, Igarashi T, Bjørn P, Drucker S M, eds, CHI ’21: CHI Confer- ence on Human Factors in Computing Systems, Virtual Event / Yokohama, Japan, May 8-13, 2021. 2021, 74...

    2021

  118. [126]

    Accuracy evaluation of touch tasks in commodity virtual and augmented reality head-mounted displays

    Schneider D, Biener V, Otte A, Gesslein T, Gagel P, Campos C, Pucihar v K, Kljun M, Ofek E, Pahud M, Kristensson P O, Gru- bert J. Accuracy evaluation of touch tasks in commodity virtual and augmented reality head-mounted displays. In: Proceedings of the 2021 ACM Symposium on ...

    2021

  119. [127]

    Generalizing hand segmentation in ego- centric videos with uncertainty-guided model adaptation

    Cai M, Lu F, Sato Y. Generalizing hand segmentation in ego- centric videos with uncertainty-guided model adaptation. In: Proceedings of the ieee/cvf conference on computer vision and pattern recognition. 2020, 14392–14401

    2020

  120. [128]

    Desktop action recognition from first- person point-of-view

    Cai M, Lu F, Gao Y. Desktop action recognition from first- person point-of-view. IEEE transactions on cybernetics, 2018, 49(5): 1616–1628

    2018

  121. [129]

    Appearance-based gaze estima- tion with deep learning: A review and benchmark

    Cheng Y, Wang H, Bao Y, Lu F. Appearance-based gaze estima- tion with deep learning: A review and benchmark. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 2024

    2024

  122. [130]

    Edge-guided near-eye image analysis for head mounted displays

    Wang Z, Zhao Y, Liu Y, Lu F. Edge-guided near-eye image analysis for head mounted displays. In: IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2021, Bari, Italy, October 4-8, 2021. 2021, 11–20

    2021

  123. [131]

    General theory of remote gaze estimation using the pupil center and corneal reflections

    Guestrin E D, Eizenman M. General theory of remote gaze estimation using the pupil center and corneal reflections. IEEE Trans. Biomed. Eng., 2006, 53(6): 1124–1133

    2006

  124. [132]

    Magiceyes: A large scale eye gaze estimation dataset for mixed reality

    Wu Z, Rajendran S, As v T, Zimmermann J, Badrinarayanan V, Rabinovich A. Magiceyes: A large scale eye gaze estimation dataset for mixed reality. CoRR, 2020, abs/2003.08806

    2020 arXiv

  125. [133]

    Get a grip: slippage- robust and glint-free gaze estimation for real-time pervasive head-mounted eye tracking

    Santini T, Niehorster D C, Kasneci E. Get a grip: slippage- robust and glint-free gaze estimation for real-time pervasive head-mounted eye tracking. In: Krejtz K, Sharif B, eds, Pro- ceedings of the 11th ACM Symposium on Eye Tracking Re- search & Applications, ETRA 2019, Denve...

    2019

  126. [134]

    d J E, Gomes H M

    Narcizo F B, Queiroz R. d J E, Gomes H M. Remote eye tracking systems: Technologies and applications. In: 2013 26th Conference on Graphics, Patterns and Images Tutorials. 2013, 15–22

    2013

  127. [135]

    A fast approach to refraction- aware eye-model fitting and gaze prediction

    Dierkes K, Kassner M, Bulling A. A fast approach to refraction- aware eye-model fitting and gaze prediction. In: Krejtz K, Sharif B, eds, Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications, ETRA 2019, Denver , CO, USA, June 25-28, 2019. 2019, 23:1–23:9

    2019

  128. [136]

    Pinpointing: Precise head- and eye-based target selection for augmented reality

    Kyt ¨o M, Ens B, Piumsomboon T, Lee G A, Billinghurst M. Pinpointing: Precise head- and eye-based target selection for augmented reality. In: Mandryk R L, Hancock M, Perry M, Cox A L, eds, Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018, ...

    2018

  129. [137]

    Watch your mouth: Silent speech recognition with depth sensing

    Wang X, Su Z, Rekimoto J, Zhang Y. Watch your mouth: Silent speech recognition with depth sensing. In: Mueller F F, Kyburz P, Williamson J R, Sas C, Wilson M L, Dugas P O T, Shklovski I, eds, Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI 2024, Ho...

    2024

  130. [138]

    Gazering: En- hancing hand-eye coordination with pressure ring in augmented reality

    Wang Z, Sun J, Hu M, Rao M, Song W, Lu F. Gazering: En- hancing hand-eye coordination with pressure ring in augmented reality. In: Eck U, Sra M, Stefanucci J K, Sugimoto M, Tatzgern M, Williams I, eds, IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2024, Be...

    2024

  131. [139]

    Interaction with gaze, gesture, 28 Front

    Wang Z, Wang H, Yu H, Lu F. Interaction with gaze, gesture, 28 Front. Comput. Sci., 2025, 0(0): 1–28 and speech in a flexibly configurable augmented reality system. IEEE Trans. Hum. Mach. Syst., 2021, 51(5): 524–534

    2025

  132. [140]

    Virtual reality technology (2

    Burdea G C, Coiffet P. Virtual reality technology (2. ed.). Wiley, 2003

    2003

  133. [141]

    Multimodal volume data exploration through mid-air haptics

    Jang J, Frier W, Park J. Multimodal volume data exploration through mid-air haptics. In: Duh H B L, Williams I, Grubert J, Jones J A, Zheng J, eds, IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2022, Singapore, October 17-21, 2022. 2022, 243–251

    2022

  134. [142]

    Ego- centric Action Recognition by Capturing Hand-Object Contact and Object State

    Shiota T, Takagi M, Kumagai K, Seshimo H, Aono Y. Ego- centric Action Recognition by Capturing Hand-Object Contact and Object State . In: 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). January 2024, 6527–6537

    2024

  135. [143]

    Integrating gaze and speech for enabling implicit interactions

    Khan A A, Newn J, Bailey J, Velloso E. Integrating gaze and speech for enabling implicit interactions. In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22. 2022

    2022

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.