A fine-tuned GPT-4o detects human-annotated LLM-generated short answers at 80% accuracy, outperforming GPTZero, and flagged LLM use is associated with higher posttest MCQ scores.
Human uncertainty makes classification more robust
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
The classification performance of deep neural networks has begun to asymptote at near-perfect levels. However, their ability to generalize outside the training set and their robustness to adversarial attacks have not. In this paper, we make progress on this problem by training with full label distributions that reflect human perceptual uncertainty. We first present a new benchmark dataset which we call CIFAR10H, containing a full distribution of human labels for each image of the CIFAR10 test set. We then show that, while contemporary classifiers fail to exhibit human-like uncertainty on their own, explicit training on our dataset closes this gap, supports improved generalization to increasingly out-of-training-distribution test datasets, and confers robustness to adversarial attacks.
fields
cs.HC 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Detecting LLM-Generated Short Answers and Effects on Learner Performance
A fine-tuned GPT-4o detects human-annotated LLM-generated short answers at 80% accuracy, outperforming GPTZero, and flagged LLM use is associated with higher posttest MCQ scores.