Text embeddings recover 57-63% of the reliable variance in exam-item difficulty, and apparent differences in predictability across IRT parameters are mostly artifacts of calibration noise rather than text signal.
arXiv preprint arXiv:2007.12061 (2020)
7 Pith papers cite this work. Polarity classification is still indexing.
7
Pith papers citing it
abstract
Digital technologies are becoming increasingly prevalent in education, enabling personalized, high quality education resources to be accessible by students across the world. Importantly, among these resources are diagnostic questions: the answers that the students give to these questions reveal key information about the specific nature of misconceptions that the students may hold. Analyzing the massive quantities of data stemming from students' interactions with these diagnostic questions can help us more accurately understand the students' learning status and thus allow us to automate learning curriculum recommendations. In this competition, participants will focus on the students' answer records to these multiple-choice diagnostic questions, with the aim of 1) accurately predicting which answers the students provide; 2) accurately predicting which questions have high quality; and 3) determining a personalized sequence of questions for each student that best predicts the student's answers. These tasks closely mimic the goals of a real-world educational platform and are highly representative of the educational challenges faced today. We provide over 20 million examples of students' answers to mathematics questions from Eedi, a leading educational platform which thousands of students interact with daily around the globe. Participants to this competition have a chance to make a lasting, real-world impact on the quality of personalized education for millions of students across the world.
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
Skill neologisms are optimized soft tokens that enhance specific LLM skills and support zero-shot composition on synthetic and Skill-Mix tasks.
EduEmbed fine-tunes language models in two stages to add semantic information to learner-item embeddings and improve performance on cognitive diagnosis and adaptive testing tasks.
GEMS formulates close-ended human-behavior simulation as link prediction on a heterogeneous graph and matches or exceeds LLM performance with three orders of magnitude fewer parameters across three datasets and three evaluation settings.
A finite-strength logit prior from a weighted knowledge graph lets Bayesian network structure learning recover directed edges under extreme sparsity where data-only methods fail.
A framework using latent class analysis on student data to define personas, LLM simulations of their responses, and ridge regression improves IRT difficulty prediction for MCQs over baselines.
Replication of cold-start analysis in KT models on FoundationalASSIST shows performance varies by practice opportunities and problem types, highlighting reproduction challenges and SafeInsights utility.
citing papers explorer
-
From Text to Parameters: Predicting Item Parameters from Embedding Regularization with Reliability and Design Ceilings
Text embeddings recover 57-63% of the reliable variance in exam-item difficulty, and apparent differences in predictability across IRT parameters are mostly artifacts of calibration noise rather than text signal.
-
Skill Neologisms: Towards Skill-based Continual Learning
Skill neologisms are optimized soft tokens that enhance specific LLM skills and support zero-shot composition on synthetic and Skill-Mix tasks.
-
Embedding Enhancement via Fine-Tuned Language Models for Learner-Item Cognitive Modeling
EduEmbed fine-tunes language models in two stages to add semantic information to learner-item embeddings and improve performance on cognitive diagnosis and adaptive testing tasks.
-
Graph-Based Alternatives to LLMs for Human Simulation
GEMS formulates close-ended human-behavior simulation as link prediction on a heterogeneous graph and matches or exceeds LLM performance with three orders of magnitude fewer parameters across three datasets and three evaluation settings.
-
KG-SoftMAP: Soft Knowledge-Graph Priors for Bayesian Network Structure Learning from Sparse Discrete Data
A finite-strength logit prior from a weighted knowledge graph lets Bayesian network structure learning recover directed edges under extreme sparsity where data-only methods fail.
-
MCQ Difficulty Prediction via Modeling Learner Heterogeneity Using Data-Driven Cognitive Profiling
A framework using latent class analysis on student data to define personas, LLM simulations of their responses, and ridge regression improves IRT difficulty prediction for MCQs over baselines.
-
A Case Study Reexamining the Cold-Start Problem in Knowledge Tracing Models and Implications for SafeInsights, an Education Research Infrastructure
Replication of cold-start analysis in KT models on FoundationalASSIST shows performance varies by practice opportunities and problem types, highlighting reproduction challenges and SafeInsights utility.