REVIEW 7 cited by
Evaluating LLMs at Detecting Errors in LLM Responses
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
With Large Language Models (LLMs) being widely used across various tasks, detecting errors in their responses is increasingly crucial. However, little research has been conducted on error detection of LLM responses. Collecting error annotations on LLM responses is challenging due to the subjective nature of many NLP tasks, and thus previous research focuses on tasks of little practical value (e.g., word sorting) or limited error types (e.g., faithfulness in summarization). This work introduces ReaLMistake, the first error detection benchmark consisting of objective, realistic, and diverse errors made by LLMs. ReaLMistake contains three challenging and meaningful tasks that introduce objectively assessable errors in four categories (reasoning correctness, instruction-following, context-faithfulness, and parameterized knowledge), eliciting naturally observed and diverse errors in responses of GPT-4 and Llama 2 70B annotated by experts. We use ReaLMistake to evaluate error detectors based on 12 LLMs. Our findings show: 1) Top LLMs like GPT-4 and Claude 3 detect errors made by LLMs at very low recall, and all LLM-based error detectors perform much worse than humans. 2) Explanations by LLM-based error detectors lack reliability. 3) LLMs-based error detection is sensitive to small changes in prompts but remains challenging to improve. 4) Popular approaches to improving LLMs, including self-consistency and majority vote, do not improve the error detection performance. Our benchmark and code are provided at https://github.com/psunlpgroup/ReaLMistake.
Forward citations
Cited by 7 Pith papers
-
DisarmRAG: Stealthy Retriever-Centric Poisoning to Disable Self-Correction in Retrieval-Augmented Generation (Extended Version)
DisarmRAG compromises the retriever to inject anti-self-correction instructions, achieving over 90% attack success across six LLMs while evading basic detection.
-
From Jack of All Trades to Master of One: Specializing LLM-based Autoraters to a Test Set
Using per-example in-context demonstrations built from historical same-source human MQM ratings makes an LLM judge dramatically better at fine-grained MT evaluation on WMT'23 and WMT'24.
-
Diagnosing Failures in Large Language Models' Answers: Integrating Error Attribution into Evaluation Framework
A new error-attribution dataset and fine-tuned judge model that outputs score, error category, and feedback for LLM responses.
-
(Im)possibility of Automated Hallucination Detection in Large Language Models
Automated hallucination detection is equivalent to identifying the true language from positive data: impossible for most countable collections without negative examples, and always possible with them.
-
Is my Meeting Summary Good? Estimating Quality with a Multi-LLM Evaluator
A multi-agent, self-training LLM framework called MESA evaluates meeting summaries by detecting eight error types and reports higher correlation with human scores than existing automatic metrics.
-
Evaluation of LLMs for mathematical problem solving
A three-model, three-dataset LLM math evaluation using a multi-dimensional reasoning rubric, undermined by contradictory accuracy tables.
-
The Hitchhikers Guide to Production-ready Trustworthy Foundation Model powered Software (FMware)
The paper catalogs the lifecycle stages and production-readiness challenges of software built around foundation models (FMware) and proposes an action plan of engineering practices and research directions.
Discussion (0). Continue with ORCID to comment.