REVIEW 6 cited by
Know Your Limits: A Survey of Abstention in Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Abstention, the refusal of large language models (LLMs) to provide an answer, is increasingly recognized for its potential to mitigate hallucinations and enhance safety in LLM systems. In this survey, we introduce a framework to examine abstention from three perspectives: the query, the model, and human values. We organize the literature on abstention methods, benchmarks, and evaluation metrics using this framework, and discuss merits and limitations of prior work. We further identify and motivate areas for future research, such as whether abstention can be achieved as a meta-capability that transcends specific tasks or domains, and opportunities to optimize abstention abilities in specific contexts. In doing so, we aim to broaden the scope and impact of abstention methodologies in AI systems.
Forward citations
Cited by 6 Pith papers
-
PhantomFill: When the Form Demands an Answer, Language Models Invent One
Required JSON fields make LLMs invent answers to unanswerable questions 100% of the time in ten of thirteen models, even when an 'insufficient evidence' escape exists.
-
Ground Truth First: A Longitudinal Evaluation Instrument for Agent Memory, and the Tenure Crossover in Memory-Architecture Rankings
A ground-truth-first synthetic memory benchmark shows that agent-memory architecture rankings invert with history length: short-horizon leaders lose at nine weeks.
-
LLM Abstention Can Be a Prompt Artifact, in Addition to Genuine Uncertainty
Adding an extra 'Unknown' option to True/False prompts causes LLMs to abstain on questions they can answer, and random words reproduce the effect, indicating abstention is partly a prompt artifact.
-
Can Video LLMs Refuse to Answer? Alignment for Answerability in Video Large Language Models
Video-LLMs can be trained, via SFT or DPO on a new synthetic dataset UVQA, to refuse questions that cannot be answered from the video content, with modest cost to answerable QA performance.
-
AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
Reasoning fine-tuning makes LLMs more accurate on answerable problems but worse at abstaining on unanswerable ones, across a new 20-dataset benchmark.
-
Measuring Faithfulness and Abstention: An Automated Pipeline for Evaluating LLM-Generated 3-ply Case-Based Legal Arguments
An automated LLM-based evaluator finds that eight LLMs rarely hallucinate factors in legal argument generation but often omit relevant factors and usually fail to abstain when no common ground exists.
Discussion (0). Continue with ORCID to comment.