REVIEW 2 cited by
Reference-free Evaluation Metrics for Text Generation: A Survey
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Reference-free Evaluation Metrics for Text Generation: A Survey
read the original abstract
A number of automatic evaluation metrics have been proposed for natural language generation systems. The most common approach to automatic evaluation is the use of a reference-based metric that compares the model's output with gold-standard references written by humans. However, it is expensive to create such references, and for some tasks, such as response generation in dialogue, creating references is not a simple matter. Therefore, various reference-free metrics have been developed in recent years. In this survey, which intends to cover the full breadth of all NLG tasks, we investigate the most commonly used approaches, their application, and their other uses beyond evaluating models. The survey concludes by highlighting some promising directions for future research.
Forward citations
Cited by 2 Pith papers
-
LLM-as-a-Judge for Evaluating System Responses in Conversational Music Recommendation
LLM judges agree moderately with human experts when scoring conversational music recommendation responses, outperform reference-based metrics, but are not reliable enough to replace human evaluation.
-
Fidelity-Diversity Metrics for Text
Optimal-transport fidelity and diversity scores on text embeddings disentangle support mismatch from mass coverage and predict GSM8K finetuning accuracy on synthetic math data.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.