Interactive Reasoning: Visualizing and Controlling Chain-of-Thought Reasoning in Large Language Models
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:J35VJDTMrecord.jsonopen to challenge →
read the original abstract
The output quality of large language models (LLMs) can be improved via "reasoning": generating segments of chain-of-thought (CoT) content to further condition the model prior to producing user-facing output. While these chains contain valuable information, they are verbose and lack explicit organization, making them tedious to review. Moreover, they lack opportunities for user feedback, such as to remove unwanted considerations, add desired ones, or clarify unclear assumptions. We introduce Interactive Reasoning, an interaction design that visualizes chain-of-thought outputs as a hierarchy of topics and enables user review and modification. We implement interactive reasoning in Hippo, a prototype for AI-assisted decision making in the face of uncertain trade-offs. In a user study with 16 participants, we find that interactive reasoning in Hippo allows users to quickly identify and interrupt erroneous generations, efficiently steer the model towards customized responses, and better understand both model reasoning and model outputs. Our work contributes to a new paradigm that incorporates user oversight into LLM reasoning processes.
This paper has not been read by Pith yet.
Forward citations
Cited by 5 Pith papers
-
"What Are You Really Trying to Do?": Co-Creating Life Goals from Everyday Computer Use
A co-creation process for inferring and refining personal strivings from computer activity logs yields more representative goals and higher user agency than baselines in a 14-person week-long study.
-
Beyond One Output: Visualizing and Comparing Distributions of Language Model Generations
GROVE visualizes distributions of language model generations as overlapping paths through a text graph, with user studies showing that graph summaries aid structural judgments like diversity assessment while raw outpu...
-
Multi-Turn Neural Transparency: Surfacing Neural Activations Improves User Calibration to LLM Behavioral Drift
Multi-turn neural transparency using behavioral vectors and dynamic visualizations improves user anticipation and evaluation of LLM trait expression while reducing overconfidence, per a randomized study with 246 participants.
-
An Agentic AI Framework with Large Language Models and Chain-of-Thought for UAV-Assisted Logistics Scheduling with Mobile Edge Computing
An agentic AI framework with LLMs generates formulations for coupled UAV product collection and MEC task scheduling, solved by hierarchical PPO that reaches 99.6% collection success and 100% deadline compliance in sim...
-
When the Chain Breaks: Interactive Diagnosis of LLM Chain-of-Thought Reasoning Errors
ReasonDiag combines automated error detection with interactive visualizations to help users identify and diagnose errors in LLM chain-of-thought reasoning traces.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.