Pith. sign in

REVIEW 3 major objections 4 minor 127 references

AiGet: Transforming Everyday Moments into Hidden Knowledge Discovery with AI Assistance on Smart Glasses

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Wearable AI that reads gaze can turn routine walks into learning opportunities.

desk verdict A credible feasibility study of proactive, gaze-driven incidental learning on smart glasses; read the effectiveness claims as promising, not proven. read the letter →

arxiv 2501.16240 v2 pith:JN27GI3C submitted 2025-01-27 cs.HC

Runze Cai , Nuwan Janaka , Hyeongcheol Kim , Yang Chen , Shengdong Zhao , Yun Huang , David Hsu This is my paper · ORCID
classification cs.HC
keywords smartglassesinformallearningincidentallargelanguagemodelsgazepatternanalysisproactiveAIassistancewearableknowledgediscovery
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Adults often walk past plants, buildings, and products they would genuinely like to know about, but time pressure, inattentional blindness, an illusion of explanatory depth, and discomfort with the unfamiliar keep that knowledge out of reach. AiGet is a smart-glasses assistant built to close that gap: it reads the wearer's gaze pattern, senses the environment, consults a personal profile, and asks a large language model to compose a short, personalized knowledge snippet. The paper reports in-lab ablation results and 40 real-world sessions, including repeated use over several days, in which proactive suggestions were mostly accepted, novel facts were recalled, and users felt more connected to their surroundings. The central claim is that proactive, gaze-aware knowledge delivery can embed informal learning into low-demand daily activities without significantly disrupting the primary task.

What carries the argument

The load-bearing component is the gaze-pattern classifier with three modes: Saccade (random gaze movement, treated as relaxed openness to fun facts), Quick Browse (rapid scanning of familiar related items, treated as medium interest), and Focused (sustained fixation, treated as high interest or decision-making). These modes decide both when the system asks the model for knowledge and what kind of knowledge it should produce, for example comparisons for Focused users and serendipitous facts for Saccade users. A second central mechanism is the knowledge-selection gate, which scores each candidate with Novelty times the sum of alignment, utility, and unexpectedness, keeps items scoring at least 2, filters out content similar to recent history, and caps each delivery at two items.

What would settle it

Run a field study where participants self-report their cognitive state at random moments while wearing the glasses and compare those reports with the system's Saccade/Quick Browse/Focused classifications; near-chance agreement, or a sharp rise in cancellation when users report being deep in thought, would show the trigger mapping is too weak to support the effectiveness claim.

Watch

Extended reading notes

Core claim

The core discovery is that a wearable eye tracker plus an LLM reasoning pipeline can make incidental learning happen during casual walking, shopping, and museum visits. AiGet classifies gaze into three modes—Saccade, Quick Browse, and Focused—and treats those modes as proxies for what the user is open to learning. It overlays gaze on the first-person view, adds location, time, and a user profile, then prompts the model to identify primary and peripheral entities, infer the user's intention, and generate knowledge scored by $\text{Novelty} \times (\text{AlignWithUserPreference} + \text{Utility} + \text{Unexpectedness})$. Only items scoring at least 2 are retained, at most two are shown at once, and output is split across audio, keyword-and-emoji text, and a bounding-box image so the user can keep attending to the environment. In the real-world sessions, AI-initiated suggestions averaged 54.2 per session against 2.3 user-initiated queries, users canceled roughly 4% of the proactive items, and subjective ratings for usefulness, interest, surprise, and relevance all averaged above 5.9 out of 7.

Load-bearing premise

The load-bearing premise is that a person's gaze mode reliably signals their current mental state and openness to learning, so the system can choose the right moment and the right fact; if that mapping is noisy, proactive prompts will arrive at the wrong times and the reported acceptance and enjoyment would not generalize.

Editorial extensions

If this is right

  • If the claim holds, query-free delivery creates learning opportunities that user-initiated visual question answering cannot, because users often do not know what to ask about in familiar places.
  • The 4% cancellation rate suggests proactive knowledge is acceptable during low-demand activities, but the same design may need interruptibility prediction before it can be used in time-sensitive or cognitively demanding tasks.
  • Repeated visits to the same place need not exhaust novelty when the system filters semantically similar content; first- and second-visit novelty ratings stayed close (5.7 vs 5.8 out of 7).
  • Gaze-conditional rules give later systems a concrete recipe: deliver fun facts during Saccade, interest-raising knowledge during Quick Browse, and decision-support comparisons during Focused engagement.
  • Multi-day usage reports indicate the effect can outlast the session, with users noticing environmental details such as a nesting bird even when not wearing the glasses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same gaze-triggered, context-scored pipeline could serve other proactive wearables, from accessibility aids that narrate overlooked hazards to retail assistants that explain unfamiliar products rather than only recommending them.
  • A direct test not reported in the paper would compare the classifier's Saccade/Quick Browse/Focused output with experience-sampling self-reports of mental state in real time, pinpointing where the trigger policy needs per-user calibration.
  • The paper's own numbers also leave room for a caution: with one prototype, one-week usage, and experimenter-present sessions, the 4% cancellation rate could shift under fully unsupervised, long-term deployment.
  • An untested extension follows from the environment-connection result: placing serendipitous knowledge in peripheral vision while keeping requested content central might be a general design principle for attention-maintaining displays.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces AiGet, a proactive AI assistant on AR smart glasses that aims to embed informal learning into low-demand daily activities (e.g., casual walking, shopping, museum visits). The system analyzes real-time gaze patterns, environmental context, and user profiles via multimodal LLMs, then delivers personalized knowledge through audio, text keywords, and images with bounding boxes, with minimal disruption to primary tasks. The authors present a formative study with 12 participants, an in-lab ablation study with the same 12 participants comparing AiGet against two baseline pipelines, and a real-world feasibility study with 18 participants (6 of whom completed additional sessions over up to 7 days). They report that AiGet generated desirable, novel, personalized, and surprising knowledge, was not annoying, and enhanced users' connection with the environment, and they derive design guidelines for future wearable informal-learning assistants.

Significance. If the findings hold, AiGet addresses a relatively underexplored niche: proactive, gaze-context-aware incidental learning in everyday settings, as opposed to reactive query-based systems such as GazePointAR or G-VOILA. The paper contributes a complete system description with detailed prompts, an open-source implementation, and design guidelines that could inform future wearable AI assistants. The real-world deployment, despite its limitations, provides useful feasibility data and user-experience insights for this class of systems.

major comments (3)
  1. [Sec 5.3.2, Sec 4.5.2-4.5.3] The in-lab ablation study reuses the same 12 participants from the formative study (Sec 4.1) to rate knowledge generated from their own recorded activities. Since these participants' stated preferences directly shaped the design goals and the gaze-pattern-to-content rules (Sec 4.5.2, Sec 5.1.3), the evaluation is not independent of the system's design. The observed advantage of AiGet over the two baselines (Sec 5.3.5, Figure 5) could be inflated by participants rating knowledge that matches their own previously articulated desires. To support the central effectiveness claim, the authors should either re-run the ablation with independent raters or provide a post-hoc analysis demonstrating that the results hold for participants who did not contribute to specific rules.
  2. [Sec 5.1.2, Sec 6.6, App C.1] The system's trigger timing and content selection depend on an LLM classification of gaze patterns into Saccade, Quick Browse, and Focused modes. The paper reports no validation of this classification: no accuracy, no confusion matrix, no inter-rater agreement between the LLM and human coders, and no ground-truth recording of user cognitive states in the real-world study. If the classification is noisy or biased, the rules for what knowledge to deliver and when will fire inappropriately, which would undermine the low-disruption and seamless-embedding claims even though the delivered content is individually accurate. The authors should provide at least a small-scale evaluation of the gaze-pattern classifier (e.g., comparing LLM labels against human annotations on a sample of the recorded FPV frames) or explicitly acknowledge and discuss the impact of this uncertainty on the design guidelines.
  3. [Abstract, Sec 6, Sec 7] The abstract and discussion claim that AiGet 'demonstrate[s] effectiveness' in enhancing enjoyment, curiosity, and connection with the environment. The supporting real-world evidence is an uncontrolled, self-reported study with 18 participants (6 of whom had repeated sessions), all in low-demand activities, and the authors themselves state in Sec 6 that 'fully addressing these questions with a high degree of quantitative evidence requires a larger user base and longer periods of daily usage.' The absence of a comparison condition, objective knowledge-retention measures, or behavior-change indicators means the evidence supports feasibility and positive user experience but not causal effectiveness. The claims should be tempered accordingly, or the study design should be extended to include a control group and pre/post measures.
minor comments (4)
  1. [Sec 5.3.5] Multiple Wilcoxon signed-rank tests are conducted across many outcome measures without multiple-comparison correction; the authors should report adjusted p-values or explicitly treat these analyses as exploratory.
  2. [Sec 6.6.1] The cancellation rate is computed as the ratio of means (2.4/54.2), which is not necessarily the mean of per-session ratios; reporting the distribution of cancellation rates across sessions would be more informative.
  3. [App C.1] The operational definitions of the three gaze modes are qualitative; providing quantitative thresholds (e.g., fixation duration, scanpath velocity) would improve reproducibility.
  4. [Sec 6.6.3] Figure 9 shows an upward trend in subjective ratings across days, but no statistical test is reported; please report the test statistics or describe the trend as descriptive.

Circularity Check

1 steps flagged · score 4.0 of 10

In-lab ablation is partly in-sample: the same 12 formative participants whose stated preferences defined AiGet's gaze-mode rules and scoring formula also rate the generated knowledge; the real-world study with independent participants provides separate support.

  1. fitted input called prediction [Section 5.3.2 (Participants), using rules from Sections 4.5.2 and 5.1.3]
    "we invited 12 participants from the formative study to evaluate knowledge generated from their previous daily activity recordings."

    The in-lab ablation evaluates AiGet's knowledge generation with the same 12 participants whose articulated preferences (Sec 4.5.2) were encoded into the system's gaze-mode-to-knowledge rules and the prioritization formula 'Novelty x (AlignUserPreference + Utility + Unexpectedness)' (Sec 5.1.3). The raters are asked to score exactly these dimensions (Novelty, Personalization, Usefulness, Unexpectedness) on the same recorded moments that produced the rules. A pipeline that echoes the participants' own stated desires will therefore score well by construction; the significant p-values quantify preference matching on the training set, not generalization to new users or moments.

full rationale

The central effectiveness claim is not wholly circular because the real-world study (Sec 6) uses an independent participant pool (18 participants, 6 for continued use) and reports feasibility, enjoyment, curiosity, and environment-connection outcomes without relying on the same 12 raters. The in-lab ablation, however, is partially circular: the knowledge-generation rules and scoring criteria were derived from the 12 formative participants' stated preferences, and the same participants rated outputs generated from their own recordings. This is an in-sample fit presented as a comparative prediction. Separately, the gaze-mode classification (Saccade/Quick Browse/Focused) performed by Gemini-1.5-Flash is not independently validated for accuracy, but that is a validity/robustness concern rather than a circularity concern. The paper also cites several prior works by the same authors (e.g., fixation thresholds, gaze-pattern literature), but these citations are not load-bearing in a way that forces the conclusions. Overall, the independent real-world feasibility evidence keeps the paper from being fundamentally circular, but the ablation's same-participant design warrants a score of 4.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claims rest on domain assumptions about gaze as a proxy for interest, LLM reliability, and self-report as evidence of learning. The free parameters are pilot-tuned system thresholds. No new physical or conceptual entities are introduced, so the invented-entities ledger is empty.

free parameters (5)
  • Minimum LLM request interval = 12 seconds
    Chosen through pilot testing to balance information richness and processing time; affects how many proactive prompts users receive.
  • FPV difference trigger threshold = cosine similarity below 0.6 for 80% of 16 frames
    Threshold for triggering LLM requests when the scene changes; tuned in pilot testing.
  • Gaze fixation threshold = within 4.91 degrees for at least 1 second
    Used to detect focused attention; adopted from prior research [25].
  • Knowledge retention score threshold = score >= 2
    Only knowledge items scoring at least 2 under Novelty*(Alignment+Utility+Unexpectedness) are retained; based on pilot results.
  • History similarity filter threshold = >= 0.75 cosine similarity
    Post-generation filter to remove content too similar to prior knowledge; threshold chosen by authors and affects novelty.
assumptions (4)
  • domain assumption Gaze patterns (Saccade, Quick Browse, Focused) reliably indicate user learning desires and interruptibility.
    System triggering and content selection depend on this mapping, introduced in Sec 4.5.2 and operationalized in Sec 5.1.2; no independent validation of the mapping appears in the real-world study.
  • domain assumption LLMs can accurately identify entities and generate factual knowledge with acceptable hallucination rates.
    The entire knowledge pipeline relies on MLLM recognition and generation; the paper acknowledges hallucinations and wrong information (Sec 7.1.3, Sec 8) but does not measure error rates.
  • domain assumption User profiles based on demographics and selected interests predict familiarity and knowledge gaps.
    Personalization uses profile data from Sec 5.1.2; there is no validation that profile items accurately estimate what a user already knows.
  • domain assumption Self-report Likert ratings and interviews are valid evidence for curiosity, enjoyment, learning, and environment connection.
    Main outcome measures are subjective; the objective measures are limited to counts of recalls, cancels, and queries, not long-term learning retention.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AiGet: Transforming Everyday Moments into Hidden Knowledge Discovery with AI Assistance on Smart Glasses." pith.science (2026). https://pith.science/paper/JN27GI3C

@misc{pith2026250116240,
  author       = {Pith},
  title        = {Pith review of: AiGet: Transforming Everyday Moments into Hidden Knowledge Discovery with AI Assistance on Smart Glasses},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JN27GI3C}},
  note         = {Machine review of arXiv:2501.16240}
}
read the original abstract

Unlike the free exploration of childhood, the demands of daily life reduce our motivation to explore our surroundings, leading to missed opportunities for informal learning. Traditional tools for knowledge acquisition are reactive, relying on user initiative and limiting their ability to uncover hidden interests. Through formative studies, we introduce AiGet, a proactive AI assistant integrated with AR smart glasses, designed to seamlessly embed informal learning into low-demand daily activities (e.g., casual walking and shopping). AiGet analyzes real-time user gaze patterns, environmental context, and user profiles, leveraging large language models to deliver personalized, context-aware knowledge with low disruption to primary tasks. In-lab evaluations and real-world testing, including continued use over multiple days, demonstrate AiGet's effectiveness in uncovering overlooked yet surprising interests, enhancing primary task enjoyment, reviving curiosity, and deepening connections with the environment. We further propose design guidelines for AI-assisted informal learning, focused on transforming everyday moments into enriching learning experiences.

Figures

Figures reproduced from arXiv: 2501.16240 by the authors.

Figure 1
Figure 1. A day with AiGet, a wearable knowledge discovery assistant equipped with AR smart glasses and a ring mouse. (1) While casually walking, AiGet actively analyzes the user’s gaze patterns and identifies “interesting” environmental entities. (2) AiGet recognizes the spider lily that the user briefly glanced at as a valuable opportunity for informal learning, considering her interest in plants in the User Profile and lac… view at source ↗
Figure 2
Figure 2. Proposed Framework for Generating Desired Knowledge from Daily Moments. Note: Gray dashed boxes and lines [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. AiGet’s System Processing Pipeline. (1) LLM Request Trigger: Supports mixed-initiative knowledge queries. (2) Context Analysis: Analyzes real-time user attention and long-term interests and infers learning desires. (3) Knowledge Generation & Prioritization: Filters redundant content, prioritizing the most valuable knowledge to avoid overload. (4) Output Transformation: Presents knowledge in a multimodal format to ba… view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: An example of comparative knowledge from three MLLM pipelines. Note: Participants can’t see each pipeline’s name [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Subjective ratings evaluating the desirability of generated knowledge. * indicates significance of [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Different Usage Scenarios with AiGet. 6.3 Study Design In Phase 1, 18 participants were randomly assigned to one of three settings: casual walking on campus, shopping, or visiting a local natural history museum, with 6 participants in each setting. These settings were …
Figure 7
Figure 7. Figure 7: Subjective ratings on the desirability of generated knowledge, its impact on primary tasks, and system usability in [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: An example of AiGet helping users identify ‘un￾known unknowns’ in daily life: two walls on campus are part of the Berlin Wall, with accompanying information intro￾ducing its history. (2) During Shopping & Dining, AiGet was seen as a “Personal￾ized Decision-Making Suppo…
Figure 9
Figure 9. Figure 9: Subjective ratings on the desirability of generated knowledge, its impact on primary tasks, and system usability on [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Context Analysis Prompt of AiGet system. D In-Lab and Real-World Study D.1 Evaluation Measures [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]
Figure 11
Figure 11. Figure 11: Prompt for AiGet System’s Knowledge Generation and Prioritization for different users’ preferences [PITH_FULL_IMAGE:figures/full_fig_p024_11.png]
Figure 12
Figure 12. Figure 12: Prompt for AiGet System’s Output Transformation Role: You are an intelligent assistant equipped to enhance informal learning through real-time, context-sensitive interactions. Utilize input data, including first-person view (FPV) with attention info, ambient audio, cu…
Figure 13
Figure 13. Figure 13: Prompt for Baseline w/o R Role: You are an intelligent assistant equipped to enhance informal learning through real-time, context-sensitive interactions. Utilize input data, including first-person view (FPV) with attention info, ambient audio, current time, and locati…
Figure 14
Figure 14. Figure 14: Prompt for Baseline w/o RP [PITH_FULL_IMAGE:figures/full_fig_p025_14.png]
Figure 15
Figure 15. Figure 15: Bar chart and line chart showing Naturalness in different scenarios and across multiple days [PITH_FULL_IMAGE:figures/full_fig_p026_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

127 extracted references · 30 canonical work pages

  1. [1]

    Fouad Alallah, Ali Neshati, Yumiko Sakamoto, Khalad Hasan, Edward Lank, Andrea Bunt, and Pourang Irani. 2018. Performer vs. observer: whose comfort level should we consider when examining the social acceptability of input modalities for head-worn display?. InProceedings of the 24th ACM Symposium on Virtual Reality Software and Technology (VRST ’18) . Asso...

    arXiv 2018

  2. [2]

    Almusawi, Christopher M

    Hashem A. Almusawi, Christopher M. Durugbo, and Afaf M. Bugawa. 2021. Wearable Technology in Education: A Systematic Review. IEEE Transactions on Learning Technologies 14, 4 (Aug. 2021), 540–554. https://doi.org/10.1109/TLT. 2021.3107459 Conference Name: IEEE Transactions on Learning Technologies

    arXiv 2021

  3. [3]

    Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz

    Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz. 2019. Guidelines for Human- AI Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19) . Association for Computing ...

    arXiv 2019

  4. [4]

    Lawrence Zitnick, and Devi Parikh

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. 2015. VQA: Visual Question Answering. In Proceedings of the IEEE International Conference on Computer Vision (ICCV)

    2015

  5. [5]

    Riku Arakawa, Hiromu Yakura, and Sosuke Kobayashi. 2022. VocabEncounter: NMT-powered Vocabulary Learning by Presenting Computer-Generated Usages of Foreign Words into Users’ Daily Lives. In CHI Conference on Human Factors in Computing Systems. ACM, New Orleans LA USA, 1–21. https://doi.org/10. 1145/3491102.3501839

    arXiv 2022

  6. [6]

    Nicole M Ardoin and Joe E Heimlich. 2021. Environmental learning in everyday life: foundations of meaning and a context for change. Environmental Education Research 27, 12 (2021), 1681–1699

    2021

  7. [7]

    Edited Jill Attewell and Carol Savill-Smith. 2004. Mobile learning anytime everywhere. Mobile learning (2004)

    2004

  8. [8]

    Ronald T Azuma. 1997. A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments 6, 4 (Aug. 1997), 355–385. https://doi.org/10.1162/ pres.1997.6.4.355

    1997

Show all 127 references
  1. [9]

    Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.0...

    2023 arXiv

  2. [10]

    Kortum, and James T

    Aaron Bangor, Philip T. Kortum, and James T. Miller. 2008. An Empirical Evalua- tion of the System Usability Scale. International Journal of Human–Computer In- teraction 24, 6 (July 2008), 574–594. https://doi.org/10.1080/10447310802205776

    2008 doi

  3. [11]

    Jonathan Baron. 2023. Thinking and deciding. Cambridge University Press

    2023

  4. [12]

    Beaudin, Stephen S

    Jennifer S. Beaudin, Stephen S. Intille, Emmanuel Munguia Tapia, Randy Rock- inson, and Margaret E. Morris. 2007. Context-Sensitive Microlearning of Foreign Language Vocabulary on a Mobile Device. In Ambient Intelligence, David Hutchison, Takeo Kanade, Josef Kittler, Jon M. Kl...

    2007

  5. [13]

    2006.Learning in places: The informal education reader

    Zvi Bekerman, Nicholas C Burbules, and Diana Silberman-Keller. 2006.Learning in places: The informal education reader . Vol. 249. Peter Lang

    2006

  6. [14]

    Mark Billinghurst, Adrian Clark, and Gun Lee. 2015. A Survey of Augmented Reality. Foundations and Trends® in Human–Computer Interaction 8, 2-3 (March 2015), 73–272. https://doi.org/10.1561/1100000049

    2015 doi

  7. [15]

    M. Blum, A. Pentland, and G. Troster. 2006. InSense: Interest-Based Life Logging. IEEE MultiMedia 13, 4 (2006), 40–48. https://doi.org/10.1109/MMUL.2006.87

    2006 doi

  8. [16]

    Bagdanov, and Alberto Del Bimbo

    Pietro Bongini, Federico Becattini, Andrew D. Bagdanov, and Alberto Del Bimbo

  9. [17]

    Tayeb Brahimi and Akila Sarirete. 2015. Learning outside the classroom through MOOCs. Computers in Human Behavior 51 (2015), 604–609

    2015

  10. [18]

    Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (Jan. 2006), 77–101. https://doi.org/10. 1191/1478088706qp063oa

    2006

  11. [19]

    John Brooke. 1996. SUS - A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 7

    1996

  12. [20]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffr...

    2020 doi arXiv

  13. [21]

    Brian Bruya. 2010. Effortless attention: A new perspective in the cognitive science of attention and action . MIT Press

    2010

  14. [22]

    Daniela Buzova, Amparo Cervera-Taulet, and Silvia Sanz-Blas. 2020. Explor- ing multisensory place experiences through cruise blog analysis. Psychol- ogy & Marketing 37, 1 (2020), 131–140. https://doi.org/10.1002/mar.21286 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/mar.21286

    2020 doi

  15. [23]

    Arthur Caetano, Alyssa Lawson, Yimeng Liu, and Misha Sra. 2023. ARLang: An Outdoor Augmented Reality Application for Portuguese Vocabulary Learning. In Proceedings of the 2023 ACM Designing Interactive Systems Conference . ACM, Pittsburgh PA USA, 1224–1235. https://doi.org/10....

    2023

  16. [24]

    Cai, Anji Ren, and Robert C

    Carrie J. Cai, Anji Ren, and Robert C. Miller. 2017. WaitSuite: Productive Use of Diverse Waiting Moments. ACM Transactions on Computer-Human Interaction 24, 1 (March 2017), 1–41. https://doi.org/10.1145/3044534

    2017 doi

  17. [25]

    Runze Cai, Nuwan Janaka, Yang Chen, Lucia Wang, Shengdong Zhao, and Can Liu. 2024. PANDALens: Towards AI-Assisted In-Context Writing on OHMD During Travels. In Proceedings of the CHI Conference on Human Fac- tors in Computing Systems (Honolulu, HI, USA) (CHI ’24) . Association...

    2024

  18. [26]

    Runze Cai, Nuwan Janaka, Shengdong Zhao, and Minghui Sun. 2023. Para- GlassMenu: Towards Social-Friendly Subtle Interactions in Conversations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computin...

    2023

  19. [27]

    Ruei-Che Chang, Yuxuan Liu, and Anhong Guo. 2024. WorldScribe: Towards Context-Aware Live Visual Descriptions. InProceedings of the 37th Annual ACM CHI ’25, April 26-May 01, 2025, Yokohama, Japan Cai et al. Symposium on User Interface Software and Technology (Pittsburgh, PA, U...

    2024

  20. [28]

    Jian Chen, Yueqin Zhang, Jingyu Sun, Yongle Chen, Fuping Lin, and Qun Jin

  21. [29]

    Garcia, and Neha Rani

    Sharon Lynn Chu, Brittany M. Garcia, and Neha Rani. 2023. Research on wearable technologies for learning: a systematic review. Frontiers in Education 8 (Nov. 2023). https://doi.org/10.3389/feduc.2023.1270389 Publisher: Frontiers

    2023

  22. [30]

    Randell Cliff. 2005. Wearable Computing: A Review. Wearable Computing: A Review (2005). Publisher: University of Bristol

    2005

  23. [31]

    Thomas J Conlon. 2004. A review of informal learning literature, theory and implications for practice in developing global professional competence. Journal of European industrial training 28, 2/3/4 (2004), 283–295

    2004

  24. [32]

    Jay Cross. 2007. Informal learning: Rediscovering the natural pathways that inspire innovation and performance . Pfeiffer/John Wiley & Sons, San Francisco, CA, US. Pages: xxi, 292

    2007

  25. [33]

    Gonzalez, Karan Ahuja, Ruofei Du, Andrea Colaço, Johnny Lee, Mar Gonzalez-Franco, and David Kim

    Mustafa Doga Dogan, Eric J. Gonzalez, Karan Ahuja, Ruofei Du, Andrea Colaço, Johnny Lee, Mar Gonzalez-Franco, and David Kim. 2024. Augmented Object Intelligence with XR-Objects. (Aug. 2024). https://doi.org/10.1145/3654777. 3676379 arXiv:2404.13274 [cs]

    2024 arXiv

  26. [34]

    Fiona Draxler, Julia Maria Brenner, Manuela Eska, Albrecht Schmidt, and Lewis L Chuang. 2022. Agenda- and Activity-Based Triggers for Microlearning. In 27th International Conference on Intelligent User Interfaces . ACM, Helsinki Finland, 620–632. https://doi.org/10.1145/349009...

    2022

  27. [35]

    Fiona Draxler, Audrey Labrie, Albrecht Schmidt, and Lewis L. Chuang. 2020. Augmented Reality to Enable Users in Learning Case Grammar from Their Real-World Interactions. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20) . Association for...

    2020

  28. [36]

    Darren Edge, Elly Searle, Kevin Chiu, Jing Zhao, and James A. Landay. 2011. MicroMandarin: mobile language learning in context. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems . ACM, Vancouver BC Canada, 3169–3178. https://doi.org/10.1145/1978942.1979413

    2011

  29. [37]

    Jakob Engel, Kiran Somasundaram, Michael Goesele, Albert Sun, Alexander Gamino, Andrew Turner, Arjang Talattof, Arnie Yuan, Bilal Souti, Brighid Mered- ith, Cheng Peng, Chris Sweeney, Cole Wilson, Dan Barnes, Daniel DeTone, David Caruso, Derek Valleroy, Dinesh Ginjupalli, Dunc...

    2023 arXiv

  30. [38]

    Michael Eraut *. 2004. Informal learning in the workplace. Studies in Continuing Education 26, 2 (July 2004), 247–273. https://doi.org/10.1080/ 158037042000225245

    2004

  31. [39]

    Google. 2024. Project Astra. https://deepmind.google/technologies/gemini/ project-astra/

    2024

  32. [40]

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. 2022. Ego4d: Around the world in 3,000 hours of egocentric video. In Pro- ceedings of the IEEE/CVF Conference on Computer...

    2022

  33. [42]

    Sandra G. Hart. 2006. Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50, 9 (2006), 904–908. https://doi.org/10.1177/154193120605000909

    2006 doi

  34. [43]

    Ari Hautasaari, Takeo Hamada, Kuntaro Ishiyama, and Shogo Fukushima. 2020. VocaBura: A Method for Supporting Second Language Vocabulary Learning While Walking. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3, 4 (Sept. 2020), 135:1–135:23. https://doi.org/10.1145/3369824

    2020 doi

  35. [44]

    Caroline Haythornthwaite. 2022. Analytics for informal learning in social media. Handbook of Learning Analytics (2022), 163

    2022

  36. [45]

    Eric Horvitz. 1999. Principles of Mixed-Initiative User Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Pittsburgh, Pennsylvania, USA) (CHI ’99). Association for Computing Machinery, New York, NY, USA, 159–166. https://doi.org/10.1145...

    1999

  37. [46]

    Human Science. [n. d.]. Personal Values. https://humanscience.fandom.com/ wiki/Personal_values. [Accessed: 2024-12-02]

    2024

  38. [47]

    Nuwan Janaka, Runze Cai, Ashwin Ram, Lin Zhu, Shengdong Zhao, and Yong Kai Qi. 2024. PilotAR: Streamlining Pilot Studies with OHMDs from Concept to Insight. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (Sept. 2024). https://doi.org/10.114...

    2024 doi

  39. [48]

    Nuwan Janaka, Jie Gao, Lin Zhu, Shengdong Zhao, Lan Lyu, Peisen Xu, Max- imilian Nabokow, Silang Wang, and Yanch Ong. 2023. GlassMessaging: To- wards Ubiquitous Messaging Using OHMDs. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 3 (Sep...

    2023 doi

  40. [49]

    Nuwan Janaka, Chloe Haigh, Hyeongcheol Kim, Shan Zhang, and Shengdong Zhao. 2022. Paracentral and near-peripheral visualizations: Towards attention- maintaining secondary information presentation on OHMDs during in-person social interactions. In Proceedings of the 2022 CHI Con...

    2022

  41. [50]

    Nuwan Janaka, Xinke Wu, Shan Zhang, Shengdong Zhao, and Petr Slovak. 2022. Visual Behaviors and Mobile Information Acquisition. https://doi.org/10.48550/ arXiv.2202.02748

    2022 doi arXiv

  42. [51]

    Nuwan Janaka, Shengdong Zhao, David Hsu, Sherisse Tan Jing Wen, and Chun Keat Koh. 2024. TOM: A Development Platform For Wearable Intelligent As- sistants. In Companion of the 2024 ACM International Joint Conference on Per- vasive and Ubiquitous Computing Pervasive and Ubiquit...

    2024

  43. [52]

    Nuwan Janaka, Shengdong Zhao, and Shardul Sapkota. 2023. Can Icons Outper- form Text? Understanding the Role of Pictograms in OHMD Notifications. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Compu...

    2023

  44. [53]

    Peter Jarvis. 2009. Learning from everyday life. In The Routledge international handbook of lifelong learning . Routledge, 19–30

    2009

  45. [54]

    Smita Jhajharia, D. S. K. Pal, and Dr Seema Verma. 2014. Wearable Computing and its Application. In International Journal of Computer Science and Information Technologie. https://api.semanticscholar.org/CorpusID:541229

    2014

  46. [55]

    R Kaplan. 1989. The experience of nature: A psychological perspective. Cam- bridge University Perss (1989)

    1989

  47. [56]

    Kelly and Tali Sharot

    Christopher A. Kelly and Tali Sharot. 2021. Individual differences in information- seeking. Nature Communications 12, 1 (Dec. 2021). https://doi.org/10.1038/ s41467-021-27046-5 Publisher: Nature Publishing Group

    2021

  48. [57]

    Minyeong Kim, Jiwook Lee, Youngji Koh, Chanhee Lee, Uichin Lee, and Auk Kim. 2024. Interrupting for Microlearning: Understanding Perceptions and Interruptibility of Proactive Conversational Microlearning Services. In Proceedings of the CHI Conference on Human Factors in Comput...

    2024

  49. [58]

    Denis Kotkov, Alan Medlar, and Dorota Glowacka. 2023. Rethinking Serendipity in Recommender Systems. In Proceedings of the 2023 Conference on Human Information Interaction and Retrieval (Austin, TX, USA) (CHIIR ’23). Association for Computing Machinery, New York, NY, USA, 383–...

    2023

  50. [59]

    Krathwohl

    David R. Krathwohl. 2002. A Revision of Bloom’s Taxonomy: An Overview.The- ory Into Practice 41, 4 (2002), 212–218. https://doi.org/10.1207/s15430421tip4104_ 2

    2002 doi

  51. [60]

    Ranjith Krishna, Pavan Kumar Pagadala, and N

    Nallapaneni Manoj Kumar, P. Ranjith Krishna, Pavan Kumar Pagadala, and N. M. Saravana Kumar. 2018. Use of Smart Glasses in Education-A Study. In 2018 2nd International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC)I-SMAC (IoT in Social, Mobile, Anal...

    2018

  52. [62]

    Yen-Mei Lee. 2023. Mobile microlearning: a systematic literature review and its implications. Interactive Learning Environments 31, 77 (Oct. 2023), 4636–4651. https://doi.org/10.1080/10494820.2021.1977964

    2023

  53. [63]

    Jiahao Nick Li, Yan Xu, Tovi Grossman, Stephanie Santosa, and Michelle Li. 2024. OmniActions: Predicting Digital Actions in Response to Real-World Multimodal Sensory Inputs with LLMs. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI...

    2024

  54. [64]

    Liquin and Alison Gopnik

    Emily G. Liquin and Alison Gopnik. 2022. Children are more exploratory and learn more than adults in an approach-avoid task. Cognition 218 (2022), 104940. AiGet CHI ’25, April 26-May 01, 2025, Yokohama, Japan https://doi.org/10.1016/j.cognition.2021.104940

    2022

  55. [65]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024. Visual instruction tuning. Advances in neural information processing systems 36 (2024)

    2024

  56. [66]

    David W Livingstone. 2001. Adults’ informal learning: Definitions, findings, gaps and future research. (2001)

    2001

  57. [67]

    D. W. Livingstone. 2001. Adults’ Informal Learning: Definitions, Findings, Gaps, and Future Research. NALL Working Paper #21 . Technical Report. NALL: New Approaches to Lifelong Learning, Ontario Institute for Studies in Education, University of Toronto, 252 Bloor Street West,...

    2001

  58. [68]

    Wulf Loh and Catrin Misselhorn. 2019. Augmented learning, smart glasses and knowing how. AI & SOCIETY (March 2019). https://doi.org/10.1007/s00146- 019-00881-3

    2019 doi

  59. [69]

    Arien Mack. 2003. Inattentional blindness: Looking without seeing. Current directions in psychological science 12, 5 (2003), 180–184

    2003

  60. [70]

    Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. 2024. Evaluating very long-term conversational memory of llm agents. arXiv preprint arXiv:2402.17753 (2024)

    2024 arXiv

  61. [71]

    Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi

  62. [72]

    Marsick and Karen E

    Victoria J. Marsick and Karen E. Watkins. 2001. Informal and Incidental Learning. New Directions for Adult and Continuing Education 2001, 89 (2001), 25–34. https: //doi.org/10.1002/ace.5

    2001 doi

  63. [73]

    Richard Mayer (Ed.). 2014. The Cambridge Handbook of Multimedia Learn- ing (2 ed.). Cambridge University Press, Cambridge. https://doi.org/10.1017/ CBO9781139547369

    2014

  64. [74]

    Alireza Modirshanechi, Johanni Brea, and Wulfram Gerstner. 2022. A taxonomy of surprise definitions. Journal of Mathematical Psychology 110 (2022), 102712

    2022

  65. [75]

    R Bruce Money and John C Crotts. 2003. The effect of uncertainty avoidance on information search, planning, and purchases of international travel vacations. Tourism Management 24, 2 (2003), 191–202

    2003

  66. [76]

    Nicolae Moroianu, Silvia-Elena Iacob, and Alexandra Constantin. 2023. Artificial Intelligence in Education: a Systematic Review. 906–921. https://doi.org/10. 2478/9788367405546-084

    2023

  67. [77]

    Donald A. Norman. 2013. The design of everyday things (revised and expanded edition ed.). Basic Books

    2013

  68. [78]

    Anne Oeldorf-Hirsch. 2018. The role of engagement in learning from active and incidental news exposure on social media. Mass communication and society 21, 2 (2018), 225–247

    2018

  69. [79]

    Veljko Pejovic and Mirco Musolesi. 2014. InterruptMe: designing intelligent prompting mechanisms for pervasive applications. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’14). Association for Computing Machinery, ...

    2014

  70. [80]

    Pavan Kartheek Rachabatuni, Filippo Principi, Paolo Mazzanti, and Marco Bertini. 2024. Context-aware chatbot using MLLMs for Cultural Heritage. In Proceedings of the ACM Multimedia Systems Conference 2024 on ZZZ . ACM, Bari Italy, 459–463. https://doi.org/10.1145/3625468.3652193

    2024

  71. [81]

    Luan, Dario Amodei, and Ilya Sutskever

    Alec Radford, Jeff Wu, Rewon Child, D. Luan, Dario Amodei, and Ilya Sutskever

  72. [82]

    Laria Reynolds and Kyle McDonell. 2021. Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI EA ’21). Association for Computing Machinery, New Y...

    2021

  73. [83]

    Chris Richardson, Yao Zhang, Kellen Gillespie, Sudipta Kar, Arshdeep Singh, Zeynab Raeesy, Omar Zia Khan, and Abhinav Sethy. 2023. In- tegrating summarization and retrieval for enhanced personalization via large language models. In CIKM 2023 Workshop Personalized Generative AI...

    2023

  74. [84]

    Leonid Rozenblit and Frank Keil. 2002. The misunderstood limits of folk science: An illusion of explanatory depth. Cognitive science 26, 5 (2002), 521–562

    2002

  75. [85]

    Language Models are Unsupervised Multitask Learners

  76. [86]

    Keith Sawyer (Ed.)

    R. Keith Sawyer (Ed.). 2022. The Cambridge Handbook of the Learning Sci- ences (3 ed.). Cambridge University Press, Cambridge. https://doi.org/10.1017/ 9781108888295

    2022

  77. [87]

    Schwartz

    Shalom H. Schwartz. 1992. Universals in the Content and Structure of Val- ues: Theoretical Advances and Empirical Tests in 20 Countries. Advances in Experimental Social Psychology, Vol. 25. Academic Press, 1–65. https: //doi.org/10.1016/S0065-2601(08)60281-6

    1992 doi

  78. [88]

    Tali Sharot and Cass R Sunstein. 2020. How people decide what they want to know. Nature Human Behaviour 4, 1 (2020), 14–19

    2020

  79. [89]

    Saravanan

    R. Saravanan. 2021. The Rumsfeld Matrix: Degrees of Knowledge. In The Climate Demon: Past, Present, and Future of Climate Prediction , R. Saravanan (Ed.). Cambridge University Press, Cambridge, 193–211. https://doi.org/10. 1017/9781009039604.017

    2021

  80. [90]

    Peter Sommerauer and Oliver Müller. 2014. Augmented Reality in Informal Learning Environments: A Field Experiment in a Mathematics Exhibition. Com- puters & Education in press (Oct. 2014). https://doi.org/10.1016/j.compedu.2014. 07.013

    2014 doi

  81. [91]

    Donggil Song and Curtis J. Bonk. 2016. Motivational factors in self-directed informal learning from online learning resources. Cogent Education 3, 1 (Dec. 2016). https://doi.org/10.1080/2331186X.2016.1205838

    2016

  82. [92]

    Guangzhi Sun, Xiao Zhan, and Jose Such. 2024. Building Better AI Agents: A Provocation on the Utilisation of Persona in LLM-based Conversational Agents. In Proceedings of the 6th ACM Conference on Conversational User Interfaces (CUI ’24) . Association for Computing Machinery, ...

    2024

  83. [93]

    Aleksandar Shtedritski, Christian Rupprecht, and Andrea Vedaldi. 2023. What does CLIP know about a red circle? Visual prompt engineering for VLMs. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) . 11987–11997

    2023

  84. [94]

    Felicia Fang-Yi Tan, Peisen Xu, Ashwin Ram, Wei Zhen Suen, Shengdong Zhao, Yun Huang, and Christophe Hurter. 2024. AudioXtend: Assisted Reality Visual Accompaniments for Audiobook Storytelling During Everyday Routine Tasks. In Proceedings of the CHI Conference on Human Factors...

    2024

  85. [95]

    Kristin Vadas, Kent Lyons, Daniel Ashbrook, Ji Soo Yi, Thad Starner, and Julie Jacko. 2006. Reading on the Go: An Evaluation of Three Mobile Display Tech- nologies. (2006), 16

    2006

  86. [96]

    Kristin Vadas, Nirmal Patel, Kent Lyons, Thad Starner, and Julie Jacko. 2006. Reading on-the-go: a comparison of audio and hand-held displays. InProceedings of the 8th conference on Human-computer interaction with mobile devices and services (MobileHCI ’06) . Association for C...

    2006

  87. [97]

    Aik Lim Tan, Robyn Gillies, and Azilawati Jamaludin. 2021. A Case Study: Using a Neuro-Physiological Measure to Monitor Students’ Interest and Learning during a Micro:Bit Activity. Education Sciences 11, 8 (2021). https://doi.org/10. 3390/educsci11080379

    2021

  88. [98]

    Christian David Vazquez, Afika Ayanda Nyati, Alexander Luh, Megan Fu, Takako Aikawa, and Pattie Maes. 2017. Serendipitous Language Learning in Mixed Reality. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’17) . Assoc...

    2017

  89. [99]

    Fengjie Wang, Yanna Lin, Leni Yang, Haotian Li, Mingyang Gu, Min Zhu, and Huamin Qu. 2024. OutlineSpark: Igniting AI-powered Presentation Slides Cre- ation from Computational Notebooks through Outlines. In Proceedings of the CHI Conference on Human Factors in Computing Systems...

    2024

  90. [100]

    Zeyu Wang, Yuanchun Shi, Yuntao Wang, Yuchen Yao, Kun Yan, Yuhan Wang, Lei Ji, Xuhai Xu, and Chun Yu. 2024. G-VOILA: Gaze-Facilitated Information Querying in Daily Scenarios. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 8, 2, Article 78 (may 2024), 33 pages. https://d...

    2024 doi

  91. [101]

    Jaan Valsiner. 1997. Culture and the development of children’s action: A theory of human development, 2nd ed . John Wiley & Sons Inc, Hoboken, NJ, US. Pages: xx, 364

    1997

  92. [102]

    Maheshya Weerasinghe, Verena Biener, Jens Grubert, Aaron Quigley, Alice Toniolo, Klen Čopič Pucihar, and Matjaž Kljun. 2022. VocabulARy: Learning Vocabulary in AR Supported by Keyword Visualisations. IEEE Transactions on Visualization and Computer Graphics 28, 11 (Nov. 2022), ...

    2022

  93. [103]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits rea- soning in large language models. Advances in Neural Information Processing Systems 35 (2022), 24824–24837

    2022

  94. [104]

    Dick, Fan Yang, Tun Lu, Ning Gu, and Li Shang

    Zhenyu Xu, Hailin Xu, Zhouyang Lu, Yingying Zhao, Rui Zhu, Yujiang Wang, Mingzhi Dong, Yuhu Chang, Qin Lv, Robert P. Dick, Fan Yang, Tun Lu, Ning Gu, and Li Shang. 2024. Can Large Language Models Be Good Companions? An LLM-Based Eyewear System with Conversational Common Ground...

    2024 doi

  95. [105]

    Zhan Wang, Lin-Ping Yuan, Liangwei Wang, Bingchuan Jiang, and Wei Zeng

  96. [106]

    Keiji Yanai, Takuma Maruyama, and Yoshiyuki Kawano. 2014. A Cooking Recipe Recommendation System with Visual Recognition of Food Ingredients. CHI ’25, April 26-May 01, 2025, Yokohama, Japan Cai et al. International Journal of Interactive Mobile Technologies 8, 2 (2014)

    2014

  97. [107]

    Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. 2023. The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision). arXiv:2309.17421 [cs.CV]

    2023 arXiv

  98. [108]

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2023. A Survey on Multimodal Large Language Models. arXiv:2306.13549 [cs.CV]

    2023 arXiv

  99. [109]

    Hamed Zamani, Fernando Diaz, Mostafa Dehghani, Donald Metzler, and Michael Bendersky. 2022. Retrieval-Enhanced Machine Learning. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (Madrid, Spain) (SIGIR ’22). Asso...

    2022 doi

  100. [110]

    Saadiah Yahya, Erny Ahmad, and Kamarularifin Abd Jalil. 2010. The definition and characteristics of ubiquitous learning: A discussion. International Journal of Education and Development using ICT 6, 11 (March 2010). https://www. learntechlib.org/p/188069/

    2010

  101. [111]

    Shengdong Zhao, Felicia Tan, and Katherine Fennedy. 2023. Heads-Up Comput- ing Moving Beyond the Device-Centered Paradigm. Commun. ACM 66, 9 (aug 2023), 56–63. https://doi.org/10.1145/3571722

    2023 doi

  102. [112]

    This apple juice contains natural antioxidants calledpolyphenols, which can help protect your cells fromdamage and may even support heart health

    Wazeer Deen Zulfikar, Samantha Chan, and Pattie Maes. 2024. Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Mem- ory Augmentation. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–18. https://doi.org/10.1145/3613904...

    2024

  103. [115]

    Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. 2023. Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00923 (2023)

    2023 arXiv

  104. [118]

    Align responses with the following user's profile and values, prioritizing valued content but not limiting to it

    Analyze user profile and their current in-situ context: Predict the user's potential learning desires, e.g., gathering interesting facts, making decisions related to current activities, acquiring future skills, or satisfying life values. Align responses with the following user...

  105. [119]

    palm trees play a crucial role in the ecosystem by providing habitat and food for various species

    Knowledge Analysis: Provide interesting knowledge on primary and semantic-related peripheral entities related to the user's current focus. Ensure it enhances interest, expands knowledge, and includes serendipitous information. Avoid repetition of recent topics. Consider follow...

  106. [120]

    Certain green plants, like those found in campus landscapes, can improve air quality by absorbing pollutants and releasing oxygen

    Generate and Evaluate Knowledge: *Generate Knowledge tailored to above user profile (e.g., education level, culture background, etc.)*, context, intention. Then, evaluate each piece using the following criteria considering the user's profile and values: - Novelty (0 or 1): Con...

  107. [121]

    *Score and reason for each piece

    Examination and Decision Making: Examine all generated knowledge for both primary and peripheral entities. *Score and reason for each piece. Retain suggestions with scores ≥ 2.*

  108. [122]

    Response Style

    Format Response: Based on the input's "Response Style" and gaze pattern, format the retained suggestions as follows: Rule no.1-3 for Live Comments, consider "Gaze Mode", from input's "activity":

  109. [123]

    Avoid Repetition content

    Saccade: provide a fun, factual knowledge about the entity (either primary and peripheral are fine, as long as it's the most interesting). Avoid Repetition content

  110. [124]

    Unknown Unknown

    Quick Browse: provide **no more than one** most interesting knowledge about what the user just scanned (i.e., primary in most cases) that they may not know before, i.e., "Unknown Unknown" factual knowledge or useful tips (procedural knowledge) that increase user interest. Avoi...

  111. [125]

    For example, offer comparative information to aid decision-making and align with the user's values (e.g., a healthy lifestyle)

    Focus: a) If decision-making or comparative knowledge is valuable, provide interesting information about both the primary and related peripheral entities (if any) (e.g., blueberry and strawberry) to support the user's intention. For example, offer comparative information to ai...

  112. [126]

    User Comments

    Reply User's questions if received "User Comments" (length equals to 2 danmaku item's length). The suggestion should be conversational style. - Attention! User maybe ask the entity in current environment *or the entity in the previous environment (which you provided knowledge ...

    2025

  113. [128]

    AI Suggestion

    Format Response:Pick one or two most interesting knowledge in AI suggestions. Return null if nothing interested. *Only pick the most relevant and interesting knowledge and make each item short.*In the "AI Suggestion", Concisely mention the entity's location to users in case th...

  114. [129]

    Ensure it enhances interest, expands knowledge, and includes serendipitous information

    Knowledge Analysis:Provide interesting entities knowledge in user environment. Ensure it enhances interest, expands knowledge, and includes serendipitous information. Avoid repetition of recent topics.**All knowledge should be novel and can open user mind. Avoid providing basi...

  115. [130]

    AI Suggestion

    Format Response:Pick one or two most interesting knowledge in AI suggestions. Return null if nothing interested. *Only pick the most relevant and interesting knowledge and make each item short.*In the "AI Suggestion", Concisely mention the entity's location to users in case th...

    2025

  116. [2015]

    In 2015 7th International Conference on Information Technology in Medicine and Education (ITME)

    Personalized micro-learning support based on process mining. In 2015 7th International Conference on Information Technology in Medicine and Education (ITME). IEEE, 511–515

    2015

  117. [2019]

    In Proceedings of the IEEE/cvf conference on computer vision and pattern recognition

    Ok-vqa: A visual question answering benchmark requiring external knowledge. In Proceedings of the IEEE/cvf conference on computer vision and pattern recognition. 3195–3204

  118. [2020]

    IOP Conference Series: Materials Science and Engineering 949, 11 (Nov

    Visual Question Answering for Cultural Heritage. IOP Conference Series: Materials Science and Engineering 949, 11 (Nov. 2020), 012074. https://doi.org/ 10.1088/1757-899X/949/1/012074

    2020 doi

  119. [2024]

    VirtuWander: Enhancing Multi-modal Interaction for Virtual Tour Guid- ance through Large Language Models. (Jan. 2024). https://doi.org/10.1145/ 3613904.3642235 arXiv:2401.11923 [cs]

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.