LLMs show a quality-dependent position bias, favoring the first option for high-quality choices and later options for low-quality ones, and higher-temperature sampling can reveal the underlying preference.
Measuring the inconsistency of large language models in preferential ranking
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
A survey that organizes LLMs-as-judges research into functionality, methodology, applications, meta-evaluation, and limitations.
citing papers explorer
-
Fragile Preferences: A Deep Dive Into Order Effects in Large Language Models
LLMs show a quality-dependent position bias, favoring the first option for high-quality choices and later options for low-quality ones, and higher-temperature sampling can reveal the underlying preference.
-
LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
A survey that organizes LLMs-as-judges research into functionality, methodology, applications, meta-evaluation, and limitations.