LLM-based raters are less reliable than trained human raters on AP Chinese writing, but hybrid human plus AI composite scoring improves reliability over AI-only scoring.
G study variance and covariance components were estimated for the seven score effects associated with the ๐โขร๐กโร๐โข design
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Exploring LLM Autoscoring Reliability in Large-Scale Writing Assessments Using Generalizability Theory
LLM-based raters are less reliable than trained human raters on AP Chinese writing, but hybrid human plus AI composite scoring improves reliability over AI-only scoring.