REVIEW 1 cited by
Analyzing Examinee Comments using DistilBERT and Machine Learning to Ensure Quality Control in Exam Content
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
This study explores using Natural Language Processing (NLP) to analyze candidate comments for identifying problematic test items. We developed and validated machine learning models that automatically identify relevant negative feedback, evaluated approaches of incorporating psychometric features enhances model performance, and compared NLP-flagged items with traditionally flagged items. Results demonstrate that candidate feedback provides valuable complementary information to statistical methods, potentially improving test validity while reducing manual review burden. This research offers testing organizations an efficient mechanism to incorporate direct candidate experience into quality assurance processes.
Forward citations
Cited by 1 Pith paper
-
Automated item evaluation: Predicting item acceptance and rejection using LLM-generated critiques
A text-based AI model predicts which standardized test items will be permanently rejected with AUC 0.80 overall and 0.86 for math, though it misses most bias-related rejections.
Discussion (0). Continue with ORCID to comment.