LLM-generated explanations, paired with a few human labels, produce model judgment distributions as close to human judgment distributions as human explanations do on NLI.
Understanding The Effect Of Temperature On Alignment With Human Opinions
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
With the increasing capabilities of LLMs, recent studies focus on understanding whose opinions are represented by them and how to effectively extract aligned opinion distributions. We conducted an empirical analysis of three straightforward methods for obtaining distributions and evaluated the results across a variety of metrics. Our findings suggest that sampling and log-probability approaches with simple parameter adjustments can return better aligned outputs in subjective tasks compared to direct prompting. Yet, assuming models reflect human opinions may be limiting, highlighting the need for further research on how human subjectivity affects model uncertainty.
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
A Rose by Any Other Name: LLM-Generated Explanations Are Good Proxies for Human Explanations to Collect Label Distributions on NLI
LLM-generated explanations, paired with a few human labels, produce model judgment distributions as close to human judgment distributions as human explanations do on NLI.