REVIEW 3 cited by
Unsupervised Evaluation of Interactive Dialog with DialoGPT
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
It is important to define meaningful and interpretable automatic evaluation metrics for open-domain dialog research. Standard language generation metrics have been shown to be ineffective for dialog. This paper introduces the FED metric (fine-grained evaluation of dialog), an automatic evaluation metric which uses DialoGPT, without any fine-tuning or supervision. It also introduces the FED dataset which is constructed by annotating a set of human-system and human-human conversations with eighteen fine-grained dialog qualities. The FED metric (1) does not rely on a ground-truth response, (2) does not require training data and (3) measures fine-grained dialog qualities at both the turn and whole dialog levels. FED attains moderate to strong correlation with human judgement at both levels.
Forward citations
Cited by 3 Pith papers
-
Beyond a Joke: Multi-Angle Reasoning for Detecting and Explaining Harmful Humor in Memes
MAR-12 improves humor and hate detection in memes by prompting a VLM through twelve reasoning perspectives, attention-weighting them, and generating explanations from the weighted evidence.
-
Multilingual Prompt Engineering in Large Language Models: A Survey Across NLP Tasks
A survey that categorizes multilingual prompting techniques by NLP task and language family, and designates potential state-of-the-art prompting methods for each dataset.
-
An Explainable Emotion Alignment Framework for LLM-Empowered Agent in Metaverse Service Ecosystem
Adding hand-coded PAD emotions and chain-of-thought rationales to LLM agents made their simulated delivery behavior look more like real rider data, though no quantitative test confirms it.
Discussion (0). Continue with ORCID to comment.