REVIEW 1 cited by
A Gold Standard Methodology for Evaluating Accuracy in Data-To-Text Systems
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Most Natural Language Generation systems need to produce accurate texts. We propose a methodology for high-quality human evaluation of the accuracy of generated texts, which is intended to serve as a gold-standard for accuracy evaluations of data-to-text systems. We use our methodology to evaluate the accuracy of computer generated basketball summaries. We then show how our gold standard evaluation can be used to validate automated metrics
Forward citations
Cited by 1 Pith paper
-
EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Models
Multimodal LLMs frequently accept hallucinated emotion claims on a new adversarial benchmark, with the worst failures on image, audio, and video perception rather than on textbook emotion knowledge.
Discussion (0). Continue with ORCID to comment.