PPLqa, the absolute difference between the perplexity of a question plus answer and the answer alone, ranks LLM responses about as well as GPTScore and G-EVAL on one long-form benchmark, with weak overall rank correlation.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
PPLqa: An Unsupervised Information-Theoretic Quality Metric for Comparing Generative Large Language Models
PPLqa, the absolute difference between the perplexity of a question plus answer and the answer alone, ranks LLM responses about as well as GPTScore and G-EVAL on one long-form benchmark, with weak overall rank correlation.