AR Secretary Agent: Real-time Memory Augmentation via LLM-powered Augmented Reality Glasses

Chun Yu; Ranyi Liu; Rapha\"el A. El Haddad; Yeonsu Shin; Yuntao Wang; Zeyu Wang

arxiv: 2505.11888 · v1 · pith:QPVDCTLVnew · submitted 2025-05-17 · 💻 cs.HC

AR Secretary Agent: Real-time Memory Augmentation via LLM-powered Augmented Reality Glasses

Rapha\"el A. El Haddad , Zeyu Wang , Yeonsu Shin , Ranyi Liu , Yuntao Wang , Chun Yu This is my paper

classification 💻 cs.HC

keywords secretaryagentaugmentedglassesmemoryreal-timerealityadvanced

0 comments

read the original abstract

Interacting with a significant number of individuals on a daily basis is commonplace for many professionals, which can lead to challenges in recalling specific details: Who is this person? What did we talk about last time? The advant of augmented reality (AR) glasses, equipped with visual and auditory data capture capabilities, presents a solution. In our work, we implemented an AR Secretary Agent with advanced Large Language Models (LLMs) and Computer Vision technologies. This system could discreetly provide real-time information to the wearer, identifying who they are conversing with and summarizing previous discussions. To verify AR Secretary, we conducted a user study with 13 participants and showed that our technique can efficiently help users to memorize events by up to 20\% memory enhancement on our study.

This paper has not been read by Pith yet.

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

EgoIntrospect: An Egocentric Dataset and Benchmark for User-Centric Internal State Reasoning
cs.CV 2026-05 unverdicted novelty 8.0

EgoIntrospect provides the first egocentric dataset with self-annotations for internal state tasks and shows multimodal LLMs struggle to infer subjective states from combined signals.