TRATES: Trait-Specific Rubric-Assisted Cross-Prompt Essay Scoring
read the original abstract
Research on holistic Automated Essay Scoring (AES) is long-dated; yet, there is a notable lack of attention for assessing essays according to individual traits. In this work, we propose TRATES, a novel trait-specific and rubric-based cross-prompt AES framework that is generic yet specific to the underlying trait. The framework leverages a Large Language Model (LLM) that utilizes the trait grading rubrics to generate trait-specific features (represented by assessment questions), then assesses those features given an essay. The trait-specific features are eventually combined with generic writing-quality and prompt-specific features to train a simple classical regression model that predicts trait scores of essays from an unseen prompt. Experiments show that TRATES achieves a new state-of-the-art performance across all traits on a widely-used dataset, with the generated LLM-based features being the most significant.
This paper has not been read by Pith yet.
Forward citations
Cited by 1 Pith paper
-
LLM Performance on a Real, Double-Marked GCSE Benchmark
Off-the-shelf LLMs match or exceed inter-examiner agreement on a new 32k-response double-marked GCSE dataset spanning five subjects and handwritten scripts.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.