REVIEW 8 cited by
CLAM: Selective Clarification for Ambiguous Questions with Generative Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Users often ask dialogue systems ambiguous questions that require clarification. We show that current language models rarely ask users to clarify ambiguous questions and instead provide incorrect answers. To address this, we introduce CLAM: a framework for getting language models to selectively ask for clarification about ambiguous user questions. In particular, we show that we can prompt language models to detect whether a given question is ambiguous, generate an appropriate clarifying question to ask the user, and give a final answer after receiving clarification. We also show that we can simulate users by providing language models with privileged information. This lets us automatically evaluate multi-turn clarification dialogues. Finally, CLAM significantly improves language models' accuracy on mixed ambiguous and unambiguous questions relative to SotA.
Forward citations
Cited by 8 Pith papers
-
Fewer Clarifications, Better Code: Benchmarking Cross-Session Personalized Ambiguity Adaptation in Coding Assistants
Twelve coding LLMs resolve injected user-specific ambiguity more often on the first turn when given same-user session history (average FT-ES +15.6 pp), though shuffled history explains part of the benefit.
-
The Severance Problem: LLMs are Unaware of the Person Beyond the Prompt
Adding a structured list of six unknowable aspects of a user's life to an LLM prompt reduces sycophantic, harmful, and hallucinated advice in synthetic tests across five model families.
-
Can Multiple Responses from an LLM Reveal the Sources of Its Uncertainty?
An auxiliary LLM can diagnose whether an LLM's uncertainty comes from ambiguous questions or missing knowledge by analyzing patterns of disagreement among multiple sampled answers.
-
Beyond Passive Critical Thinking: Fostering Proactive Questioning to Enhance Human-AI Collaboration
A training method using reinforcement learning and answerability heuristics lets small language models actively ask for missing math details and then solve problems, raising accuracy on the new GSM-MC benchmark from 0...
-
SG-CoT: An Ambiguity-Aware Robotic Planning Framework using Scene Graph Representations
SG-CoT grounds an LLM planner's chain-of-thought in a scene graph via iterative API queries, improving ambiguity detection and clarification in simulated manipulation, though its success metric credits any clarifying ...
-
Beyond Solving Math Quiz: Evaluating the Ability of Large Reasoning Models to Ask for Information
Per the abstract, large reasoning models systematically fail to ask for missing information on under-specified math problems, a skill standard benchmarks never test.
-
Demystifying Feature Requests: Leveraging LLMs to Refine Feature Requests in Open-Source Software
GPT-4o with in-context learning can flag ambiguity and incompleteness in GitHub feature requests and draft clarification questions, though moderate annotator agreement and a small sample limit the strength of the evidence.
-
Referential ambiguity and clarification requests: comparing human and LLM behaviour
Humans seldom ask clarification questions for referential ambiguity, while LLMs ask them more often, and reasoning prompts increase LLM question frequency and relevance.
Discussion (0). Sign in to comment.