REVIEW 6 cited by
Designing a Dashboard for Transparency and Control of Conversational AI
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Conversational LLMs function as black box systems, leaving users guessing about why they see the output they do. This lack of transparency is potentially problematic, especially given concerns around bias and truthfulness. To address this issue, we present an end-to-end prototype-connecting interpretability techniques with user experience design-that seeks to make chatbots more transparent. We begin by showing evidence that a prominent open-source LLM has a "user model": examining the internal state of the system, we can extract data related to a user's age, gender, educational level, and socioeconomic status. Next, we describe the design of a dashboard that accompanies the chatbot interface, displaying this user model in real time. The dashboard can also be used to control the user model and the system's behavior. Finally, we discuss a study in which users conversed with the instrumented system. Our results suggest that users appreciate seeing internal states, which helped them expose biased behavior and increased their sense of control. Participants also made valuable suggestions that point to future directions for both design and machine learning research. The project page and video demo of our TalkTuner system are available at https://bit.ly/talktuner-project-page
Forward citations
Cited by 6 Pith papers
-
HELP: Human-Efficient Large-Scale Robot Post-Training with Rollout Segmentation
VLAC-Cut-guided multi-robot HITL post-training reaches 80–95% success and 1.7–4.2× throughput over the base VLA, outperforming HITL-only under the same human budget.
-
Because we have LLMs, we Can and Should Pursue Agentic Interpretability
Agentic interpretability, using LLMs as proactive conversational teachers that model the user, is offered as a needed complement to black-box interpretability.
-
Robustly Improving LLM Fairness in Realistic Settings via Interpretability
Realistic hiring context reintroduces race and gender bias in LLMs despite anti-bias prompts, and ablating internal demographic directions reduces that bias.
-
Fine-Grained Interpretation of Political Opinions in Large Language Models
Four-dimensional political concept vectors learned from LLM internals can detect and partially steer political leanings better than a single left-right axis.
-
Position: It's Time to Optimize LLMs for Self-Consistency
The paper proposes self-consistency, a mathematical framework that treats relationships between model outputs across related inputs as the primary training target, unifying many existing alignment and robustness methods.
-
Web-Browsing LLMs Can Access Social Media Profiles and Infer User Demographics
Web-browsing LLMs can retrieve X profile content and infer demographics with above-chance accuracy in some cases, but the study's evidence is partly confounded by training-data memorization and a heavily reduced synth...
Discussion (0). Sign in to comment.