An LLM-based UI agent monitors visual analytics interactions, detects when users struggle, infers their intent, and proactively provides context-aware guidance.
DrawTalking: Building Interactive Worlds by Sketching and Speaking
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We introduce DrawTalking, an approach to building and controlling interactive worlds by sketching and speaking while telling stories. It emphasizes user control and flexibility, and gives programming-like capability without requiring code. An early open-ended study with our prototype shows that the mechanics resonate and are applicable to many creative-exploratory use cases, with the potential to inspire and inform research in future natural interfaces for creative exploration and authoring.
citation-role summary
background 1
citation-polarity summary
fields
cs.HC 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
ProactiveVA: Proactive Visual Analytics with LLM-Based UI Agent
An LLM-based UI agent monitors visual analytics interactions, detects when users struggle, infers their intent, and proactively provides context-aware guidance.