REVIEW 4 cited by
VR-GPT: Visual Language Model for Intelligent Virtual Reality Applications
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
The advent of immersive Virtual Reality applications has transformed various domains, yet their integration with advanced artificial intelligence technologies like Visual Language Models remains underexplored. This study introduces a pioneering approach utilizing VLMs within VR environments to enhance user interaction and task efficiency. Leveraging the Unity engine and a custom-developed VLM, our system facilitates real-time, intuitive user interactions through natural language processing, without relying on visual text instructions. The incorporation of speech-to-text and text-to-speech technologies allows for seamless communication between the user and the VLM, enabling the system to guide users through complex tasks effectively. Preliminary experimental results indicate that utilizing VLMs not only reduces task completion times but also improves user comfort and task engagement compared to traditional VR interaction methods.
Forward citations
Cited by 4 Pith papers
-
SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards
Dense scene-graph-grounded rewards let a 7B multimodal LLM trained on 7K synthetic questions beat SFT and sparse-RL baselines and outscore GPT-4o on average across 12 spatial/real-world benchmarks.
-
GesPrompt: Leveraging Co-Speech Gestures to Augment LLM-Based Interaction in Virtual Reality
GesPrompt lets VR users communicate spatial intent to LLM-based assistants through co-speech gestures and speech, reducing reliance on detailed verbal prompts in a small user study.
-
Spatial 3D-LLM: Exploring Spatial Awareness in 3D Vision-Language Models
Spatial 3D-LLM adds a progressive spatial awareness scheme to a 3D vision-language model, improving several 3D understanding and grounding metrics and introducing new distance and layout-editing tasks.
-
Towards user-centered interactive medical image segmentation in VR with an assistive AI agent
SAMIRA lets users speak to a VR AI agent to segment brain and liver tumors, reaching high Dice accuracy and a System Usability Scale score near 90, with controller pointing slightly ahead of head and eye tracking.
Discussion (0). Continue with ORCID to comment.