REVIEW 6 cited by
Understanding User Experience in Large Language Model Interactions
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In the rapidly evolving landscape of large language models (LLMs), most research has primarily viewed them as independent individuals, focusing on assessing their capabilities through standardized benchmarks and enhancing their general intelligence. This perspective, however, tends to overlook the vital role of LLMs as user-centric services in human-AI collaboration. This gap in research becomes increasingly critical as LLMs become more integrated into people's everyday and professional interactions. This study addresses the important need to understand user satisfaction with LLMs by exploring four key aspects: comprehending user intents, scrutinizing user experiences, addressing major user concerns about current LLM services, and charting future research paths to bolster human-AI collaborations. Our study develops a taxonomy of 7 user intents in LLM interactions, grounded in analysis of real-world user interaction logs and human verification. Subsequently, we conduct a user survey to gauge their satisfaction with LLM services, encompassing usage frequency, experiences across intents, and predominant concerns. This survey, compiling 411 anonymous responses, uncovers 11 first-hand insights into the current state of user engagement with LLMs. Based on this empirical analysis, we pinpoint 6 future research directions prioritizing the user perspective in LLM developments. This user-centered approach is essential for crafting LLMs that are not just technologically advanced but also resonate with the intricate realities of human interactions and real-world applications.
Forward citations
Cited by 6 Pith papers
-
What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)
Chat UI and API access to the same chatbot produce different accuracy, consistency, citation, and refusal behaviors on safety benchmarks, and web search changes these patterns further.
-
Understanding Fortunetelling with Large Language Models in China: User Practices, Perceptions, and Impacts on Beliefs and Decisions
Chinese users treat LLM fortunetelling as entertainment and emotional support; it subtly shifts confidence and timing but rarely reverses decisions.
-
Reflections and Recommendations on AI Adoption Practice from a Mixed-Ability Research Group
The five members of one mixed-ability lab interviewed themselves and produced ten AI-use recommendations balancing accessibility, privacy, identity, and experimentation.
-
When Models Meet Users: An Empirical Study of Perceptions of General LLMs and Multimodal LLMs on Hugging Face
Hugging Face discussions show that access barriers, output quality, and setup complexity are the main user concerns for both general and multimodal LLMs.
-
OSS-UAgent: An Agent-based Usability Evaluation Framework for Open Source Software
A framework using LLM agents to simulate developers at four experience levels and automatically evaluate open source platform usability from generated code.
-
PACT: A Contract-Theoretic Framework for Pricing Agentic AI Services Powered by Large Language Models
PACT models agentic AI services as a menu of quality-price contracts and uses contract theory to show incentive-compatible, individually rational pricing, with numerical examples for cybersecurity log analysis.
Discussion (0). Sign in to comment.