Pith. sign in

REVIEW 4 major objections 5 minor 34 references

Can LLMs Reliably Simulate Real Students' Abilities in Mathematics and Reading Comprehension?

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that no LLM–prompt combination reliably matches average student performance in mathematics or reading for grades 4, 8, and 12.

desk verdict A valuable, honest empirical study on LLMs as student proxies, but the IRT measurement model has an unresolved inconsistency that weakens the quantitative claims. read the letter →

arxiv 2507.08232 v1 pith:SQEYJYR3 submitted 2025-07-11 cs.CL cs.AI

KV Aditya Srivatsa , Kaushal Kumar Maurya , Ekaterina Kochmar This is my paper · ORCID
classification cs.CLcs.AI
keywords LLMproxystudentsItemResponseTheoryRaschmodelNAEPgradealignmentpersonapromptingsyntheticexamineeseducationalmeasurement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether large language models can stand in for real students when developing tutoring systems and piloting test questions. Using 489 multiple-choice items from the NAEP mathematics and reading assessments for grades 4, 8, and 12, the authors place 11 LLMs on the same ability scale as real students through item response theory. They find that, without guidance, strong general-purpose models perform far above the average student at every grade, while weaker models may align only by chance. Asking a model to act like an average student shifts performance, but the shift is model- and prompt-specific: no evaluated model–prompt pair aligns with the average student across both subjects and all three grades. If correct, the result means LLM-based proxy students need per-model, per-grade validation before they can be trusted in educational measurement.

What carries the argument

The load-bearing mechanism is the Rasch model from item response theory, a one-parameter model in which the probability of a correct answer depends only on the difference between a test-taker's ability $\theta_i$ and an item's difficulty $b_j$. Item difficulties are fixed from real NAEP students' correct-response proportions via $b_j = \log\left(\frac{1-p_j}{p_j}\right)$, and the population ability scale is pinned by assuming student abilities follow $\mathcal{N}(0,1)$, so the average student sits at $\theta = 0$ and each LLM's percentile rank is $\Phi(\theta_i) \times 100$. This places each LLM as a synthetic examinee on the same scale as real students, with the 50th percentile as the target for an average grade-level student. The second piece of machinery is the set of grade-enforcement prompts (minimal, basic chain-of-thought, and full chain-of-thought) that attempt to steer the model toward that target.

What would settle it

Recompute the analysis with a two-parameter (2PL) or three-parameter (3PL) IRT model fit to the full NAEP item-level response data rather than the logit approximation, and compare each LLM's percentile rank; if models reported as far above the 50th percentile, such as o3-Mini, land near 50 under the alternative model, the alignment failure is a measurement artifact. Alternatively, hold out half of the NAEP items per grade-subject, select the best model-prompt pair on the training half, and test on the held-out half; if the pair's percentile deviation from 50 does not persist, the reported alignment is overfitted to the specific item set.

Watch

Extended reading notes

Core claim

The paper's central discovery is that placing 11 LLMs on the same Rasch ability scale as real NAEP students yields a systematic mismatch: in the unenforced condition, strong general-purpose models such as o3-Mini and Qwen2.5-7B score in the high 90s percentile at every grade and subject, far above the average student's 50th percentile, while weaker or domain-mismatched models land closer to 50 only incidentally. Grade-enforcement prompts that instruct the model to behave like an average student shift percentile ranks significantly in some cases, but the direction and size of the shift are model- and prompt-specific: some models overshoot downward, some barely move, and no single model-prompt pair stays within the targeted alignment across mathematics and reading at grades 4, 8, and 12. The authors conclude that faithful grade-level emulation likely requires explicit alignment objectives in training, not just prompting, and they offer proxy-selection guidelines based on grade alignment, developmental ordering, and prompt stability.

Load-bearing premise

The load-bearing premise is that the Rasch model with logit-estimated item difficulties and a standard-normal student ability distribution puts an LLM's percentile rank on a scale where the 50th percentile truthfully represents the average grade-level student; if that scale is biased, the paper's alignment findings could be artifacts of the measurement model.

Editorial extensions

If this is right

  • Strong general-purpose models score at the 95th percentile or above on unenforced grade-level items, so without explicit constraint they cannot stand in for average students in any grade or subject.
  • Grade-enforcement prompting moves percentile ranks by tens of points in some cases but not in predictable directions, so a prompt that aligns one model on one subject-grade cell cannot be expected to transfer to another.
  • A few model-prompt pairs do align with the average student in specific cells, meaning a proxy student can be selected only cell-by-cell, not globally.
  • Pedagogically fine-tuned models are no better aligned than general models, indicating that domain fine-tuning alone does not produce grade fidelity.
  • Before an LLM is used to pilot test items or train a tutor, it should be checked against the paper's criteria: ability within $\pm 1$ logit of the grade mean, monotonically increasing ability across grades, and stability under grade-enforcement prompts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's IRT percentile check could be reused as a cheap acceptance test for any new LLM proxy: evaluate it on a small NAEP-like multiple-choice set, compute its percentile, and reject it if it falls outside the chosen band, turning the paper's negative result into a practical screening tool.
  • Because the absolute percentiles depend on the normal-ability and logit-difficulty assumptions, the exact numbers may be biased, but the qualitative overshoot pattern would survive re-anchoring because raw accuracies are already far above student means.
  • The results connect to the broader finding that persona prompts like 'you are a 4th grader' change superficial behavior more than calibrated ability; a natural testable extension is to measure whether trial-by-trial confidence or error patterns, not just accuracy, move toward a real student's response distribution.
  • A direct next experiment would hold out a subset of NAEP items per grade-subject, fit the Rasch model on the training subset, and check whether the best model-prompt pair from the paper maintains its alignment on the held-out subset, a generalization check the paper does not report.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper investigates whether LLMs can serve as reliable stand-ins for average K-12 students by comparing 11 LLMs with NAEP students on 489 multiple-choice mathematics and reading items across grades 4, 8, and 12. The authors calibrate item difficulties from real students' aggregate proportion-correct values using a logit approximation, estimate each LLM's ability via a Rasch-model maximum likelihood fit, and convert the resulting abilities to percentile ranks under a standard-normal ability distribution. They compare unenforced and three grade-enforced prompting strategies and report that strong general-purpose models consistently exceed the average student's 50th percentile, while weaker or domain-mismatched models can align incidentally; they conclude that no evaluated model-prompt pair aligns with the average student across all subjects and grades, and they propose screening guidelines for selecting proxy students.

Significance. The paper addresses a timely and practical question, and the dataset release plus the systematic comparison across 11 models and four prompt conditions are useful contributions to the NLP-for-education community. The qualitative pattern—strong models overshoot and weak models align for incidental reasons—is plausible and likely to survive a corrected analysis. However, the paper's quantitative claims are built on a psychometric calibration that is internally inconsistent (Eq. 2 versus the N(0,1) assumption), and the normative bands in the guidelines contain a numerical labeling error. Because the headline conclusion is expressed in percentile units derived from that calibration, the paper needs a substantial statistical revision before the specific alignment claims can be accepted.

major comments (4)
  1. [Section 4.1 (Eq. 2) and Section 5.3] The item difficulty estimator b_j = log((1-p_j)/p_j) is not compatible with the student ability model θ~N(0,1) used for the percentile ranks. Under the Rasch model with a standard-normal ability distribution, the population proportion correct is p_j = ∫ σ(θ-b_j) φ(θ) dθ, and solving this convolution for b_j does not reduce to the inverse logit of p_j. The inverse-logit formula instead makes the θ=0 examinee reproduce each item's population proportion exactly, which systematically shrinks the spacing of item difficulties relative to the normal-ability metric. The LLM abilities θ_i estimated by maximum likelihood are therefore on a different scale, and the reported percentile ranks Φ(θ_i) do not describe the LLM's position in the real student ability distribution. Since the headline conclusion in the abstract and Section 6.3 is stated entirely in percentile units, this issue is load-bearing. Please re-estimate item difficulties under the N(0,1) marginal likelihood (or another explicitly fitted ability distribution), provide item-fit diagnostics, and report whether the conclusions survive.
  2. [Section 5.3 and Figure 2] The normality justification is not about the quantity that matters. The KS tests in Figure 2 are applied to the distribution of item accuracies within each grade and subject, not to student abilities. Thus the text stating that all subsets 'sufficiently follow a normal distribution' cannot be used to support the assumption θ~N(0,1). The N(0,1) assumption is an identification convention, but after using Eq. (2) the item difficulties and the ability distribution are not jointly identified to that convention. The statement that 'the average student has an ability parameter of zero' is therefore an anchor of convenience rather than an empirical benchmark, and the percentile interpretation in Eq. (3) is untested. Please either fit the ability distribution from the available response moments or present the results on the logit scale and add a sensitivity analysis for the assumed ability distribution.
  3. [Section 6.4] The normative band percentiles are numerically wrong. Φ(±1) = [15.87, 84.13], Φ(±1.5) = [6.68, 93.32], and Φ(±2) = [2.28, 97.72]. The paper labels 6.68–93.32 as 'core ±1', 2.28–97.72 as 'extended ±1.5', and 15.87–84.13 as 'outlier ≥±2', which swaps the bands. This also affects the adjacent claim that GPT-3.5-Turbo 'stayed in the core range with appropriate prompts for most grades': under the corrected core band, the grade-8 math PE of 11.7 is below 15.87 and thus not in the core. Please correct the bands and re-check any statements that rely on them.
  4. [Section 5 and Table 2] All model ability estimates are point estimates from a single deterministic run per prompt (temperature=0). No standard errors, confidence intervals, or bootstrap replicates are reported, even though each model answers only 61–106 items per cell. Consequently, cells that are a few percentile points from the alignment band are not statistically distinguishable from aligned cells, and the precise statement that 'no evaluated model–prompt pair fits the bill' is stronger than the evidence supports. Please report uncertainty intervals for θ_i and the percentile ranks, and verify the headline conclusion against a reasonable tolerance interval.
minor comments (5)
  1. [Abstract and Section 6.3] The phrase 'no evaluated model–prompt pair fits the bill across subjects and grades' reads as though no pair is ever aligned, but Table 2 shows several aligned cells (e.g., LLaMA2-13B on grade-8 mathematics). Please state the claim as 'aligned in all six subject–grade combinations' to avoid ambiguity.
  2. [Algorithm 1] Line 10 contains an extra closing parenthesis: 'πi ← Φ(θi) × 100)' should read 'πi ← Φ(θi) × 100'.
  3. [Appendix C] There is a typo: 'HunggingFace' should be 'Hugging Face' or 'HuggingFace'.
  4. [Table 5] Some prompt labels contain spurious spaces (for example, 'G RADEENFORCEDFULLCOT'); please fix the table-generation formatting so the prompt names are rendered consistently.
  5. [Table 2] The Random Choice row is formatted inconsistently with the rest of the table and does not clearly align with the six subject–grade columns; please restructure that row so each value sits under the correct column.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: LLM abilities are measured against external NAEP student response data rather than derived from the target conclusion, so the central claim is an empirical finding.

full rationale

The paper's derivation chain is not circular. Item difficulties are estimated from NAEP student response proportions (Eq. 2), which are external inputs; LLM responses are then queried independently under each prompting condition, and each model's ability is estimated by maximum likelihood from its own response pattern. The 'average student' at percentile 50 is an identification convention (theta ~ N(0,1) in Sec. 5.3), not a fitted parameter, so comparing LLM theta estimates to zero is a substantive empirical comparison rather than a restatement of the input. No model parameter is fitted to the alignment outcome and then renamed as a prediction; the Sec. 6.4 proxy-selection guidelines are post-hoc criteria, not derived predictions. Citations to prior work (Benedetto et al. 2024, Embretson and Reise 2000, Bond and Fox 2015) are used for prompts and standard IRT conventions, not as unverified self-citations carrying the argument. The paper does contain legitimate statistical concerns, such as the inconsistency between Eq. 2's logit approximation and the N(0,1) ability scale used for percentiles, and the numerically swapped normative bands in Sec. 6.4, but these are correctness and validity issues rather than circularity. No equation or claim reduces to its own inputs by construction, so the circularity score is 0.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim depends on the IRT measurement model. We list the item difficulties as fitted parameters (they anchor the scale), the N(0,1) ability assumption and Rasch model as domain assumptions, and the logit item-difficulty approximation as an ad hoc step. No new entities are invented.

free parameters (1)
  • Item difficulty set {b_j} (489 items) = Computed via b_j = log((1-p_j)/p_j) from NAEP p_j
    These item parameters anchor the ability scale for all LLM ability estimates. Any bias in this logit approximation propagates into every theta_i and percentile rank. They are fitted to real student response proportions, so they are the measurement instrument rather than a suspicious free parameter, but they are still estimated from data.
assumptions (3)
  • domain assumption Student abilities are standard normally distributed: theta ~ N(0,1).
    Used to identify the Rasch scale and convert theta_i to percentile rank via Phi(theta_i). The paper states this in Section 5.3 citing Embretson and Reise (2000). If the true NAEP ability distribution is not normal, the reported percentiles are not true student percentiles.
  • domain assumption The Rasch model holds for all NAEP items and the student population.
    Equation (1) assumes the probability of a correct response depends only on theta_i - b_j. No item fit statistics or model diagnostics are reported, so a misfitting item set would bias difficulty estimates and ability comparisons.
  • ad hoc to paper The logit approximation b_j ≈ log((1-p_j)/p_j) is a consistent estimator of item difficulty under the Rasch model with N(0,1) ability.
    This approximation is exact only if the marginal probability p_j equals the Rasch ICC at theta=0, which is not generally true under a normal ability distribution. The paper uses this approximation (Eq 2) without checking its bias.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can LLMs Reliably Simulate Real Students' Abilities in Mathematics and Reading Comprehension?." pith.science (2026). https://pith.science/paper/SQEYJYR3

@misc{pith2026250708232,
  author       = {Pith},
  title        = {Pith review of: Can LLMs Reliably Simulate Real Students' Abilities in Mathematics and Reading Comprehension?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SQEYJYR3}},
  note         = {Machine review of arXiv:2507.08232}
}
read the original abstract

Large Language Models (LLMs) are increasingly used as proxy students in the development of Intelligent Tutoring Systems (ITSs) and in piloting test questions. However, to what extent these proxy students accurately emulate the behavior and characteristics of real students remains an open question. To investigate this, we collected a dataset of 489 items from the National Assessment of Educational Progress (NAEP), covering mathematics and reading comprehension in grades 4, 8, and 12. We then apply an Item Response Theory (IRT) model to position 11 diverse and state-of-the-art LLMs on the same ability scale as real student populations. Our findings reveal that, without guidance, strong general-purpose models consistently outperform the average student at every grade, while weaker or domain-mismatched models may align incidentally. Using grade-enforcement prompts changes models' performance, but whether they align with the average grade-level student remains highly model- and prompt-specific: no evaluated model-prompt pair fits the bill across subjects and grades, underscoring the need for new training and evaluation strategies. We conclude by providing guidelines for the selection of viable proxies based on our findings.

Figures

Figures reproduced from arXiv: 2507.08232 by the authors.

Figure 1
Figure 1. Sample NAEP question from grade 8 mathe [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Distribution of question-level accuracy in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Impact of grade-enforcing prompting We conduct a thorough analysis of 11 diverse mod￾els on mathematics and reading questions from K-12 grades 4, 8, and 12 sourced from the NAEP database. Our IRT-based analysis reveals that in the regular (unenforced) setting, stronger models score far better than the average students of any grade and weaker models may align well inciden￾tally. Though explicit (grade-enforced) promp… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Sample NAEP question from grade 12 read [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: UNENFORCED Prompt Template for Mathe￾matics [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: GRADEENFORCEDMINIMAL Prompt Tem￾plate for Mathematics queried using the Google Cloud (Vertex) API, o3-Mini was queried using the OpenAI API, and LearnLM-1.5-Pro was queried using Google’s AI Studio API. All other models were imported from HunggingFace and queried local…
Figure 10
Figure 10. Figure 10: GRADEENFORCEDMINIMAL Prompt Tem￾plate for Reading [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: GRADEENFORCEDBASICCOT Prompt Tem￾plate for Reading [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 12
Figure 12. Figure 12: GRADEENFORCEDFULLCOT Prompt Tem￾plate for Reading E Analyses Details [PITH_FULL_IMAGE:figures/full_fig_p012_12.png]
Figure 13
Figure 13. Figure 13: Option Extraction Prompt percentile values to 50. LLM Mathematics Reading 4 8 12 4 8 12 LLaMA2-13B 78.05 43.40 41.67 85.15 80.56 77.61 LLaMA2-70B 65.85 59.43 45.00 96.04 91.67 82.09 LLaMA3.1-8B 87.80 77.36 63.33 96.04 93.06 85.07 LLaMA3.1-70B 93.90 91.51 80.00 98.02 9…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 14 canonical work pages

  1. [1]

    Frank B Baker. 2001. The basics of item response theory. ERIC

    work page 2001

  2. [2]

    Anya Belz, Craig Thomson, Ehud Reiter, and Simon Mille. 2023. https://doi.org/10.18653/v1/2023.findings-acl.226 Non-repeatable experiments and non-reproducible results: The reproducibility crisis in human evaluation in NLP . In Findings of the Association for Computational Linguistics: ACL 2023, pages 3676--3687, Toronto, Canada. Association for Computati...

    doi:10.18653/v1/2023.findings-acl.226 2023

  3. [3]

    Luca Benedetto, Giovanni Aradelli, Antonia Donvito, Alberto Lucchetti, Andrea Cappelli, and Paula Buttery. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.663 Using LLM s to simulate students' responses to exam questions . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11351--11368, Miami, Florida, USA. Association f...

    doi:10.18653/v1/2024.findings-emnlp.663 2024

  4. [4]

    Bond and Christine M

    Trevor G. Bond and Christine M. Fox. 2015. https://doi.org/10.4324/9781315814698 Applying the Rasch Model: Fundamental Measurement in the Human Sciences , 3 edition. Routledge, New York

    doi:10.4324/9781315814698 2015

  5. [5]

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, and 1 others. 2024. A survey on evaluation of large language models. ACM transactions on intelligent systems and technology, 15(3):1--45

    2024

  6. [6]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

  7. [7]

    Embretson and Steven P

    Susan E. Embretson and Steven P. Reise. 2000. https://doi.org/10.4324/9781410605269 Item Response Theory for Psychologists . Multivariate Applications Series. Lawrence Erlbaum Associates, Mahwah, NJ

    work page doi:10.4324/9781410605269 2000

  8. [8]

    Michael Grohs, Luka Abb, Nourhan Elsayed, and Jana - Rebecca Rehse. 2024. Large language models can accomplish business process management tasks. In Proceedings of the International Conference on Business Process Management. Extended version available as arXiv:2307.09923

    arXiv 2024

Show all 34 references
  1. [9]

    Domingue, Emma Brunskill, and Noah D

    Joy He-Yueya, Wanjing Anya Ma, Kanishk Gandhi, Benjamin W. Domingue, Emma Brunskill, and Noah D. Goodman. 2024. https://arxiv.org/abs/2407.15645 Psychometric alignment: Capturing human knowledge distributions via language models . Preprint, arXiv:2407.15645

    2024 arXiv

  2. [10]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300

    2020 arXiv

  3. [11]

    Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, and 1 others

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, and 1 others. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  4. [12]

    Junseok Kim, Nakyeong Yang, and Kyomin Jung. 2024. https://doi.org/10.48550/arXiv.2408.08631 Persona is a double-edged sword: Mitigating the negative impact of role-playing prompts in zero-shot reasoning tasks . arXiv preprint

    2024 doi arXiv

  5. [13]

    Jiayu Liu, Zhenya Huang, Tong Xiao, Jing Sha, Jinze Wu, Qi Liu, Shijin Wang, and Enhong Chen. 2024. Socraticlm: Exploring socratic personalized teaching with large language models. In Advances in Neural Information Processing Systems (NeurIPS) 2024

    2024

  6. [14]

    Yunting Liu, Shreya Bhandari, and Zachary A. Pardos. 2025. https://doi.org/10.1111/bjet.13570 Leveraging llm respondents for item evaluation: A psychometric analysis . British Journal of Educational Technology, 56:1028--1052

    2025 doi

  7. [15]

    Jakub Macina, Nico Daheim, Sankalan Chowdhury, Tanmay Sinha, Manu Kapur, Iryna Gurevych, and Mrinmaya Sachan. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.372 M ath D ial: A dialogue tutoring dataset with rich pedagogical properties grounded in math reasoning problems...

    2023 doi

  8. [16]

    Noboru Matsuda, Dan Lv, and Guoliang Zheng. 2023. https://doi.org/10.1007/s40593-022-00306-1 Teaching how to teach promotes learning by teaching . International Journal of Artificial Intelligence in Education, 33(3):720--751

    2023 doi

  9. [17]

    Abhinit Modi and the LearnLM Team. 2024. Learnlm: Improving gemini for learning. arXiv preprint arXiv:2412.16429

    2024 arXiv

  10. [18]

    Mollick, Lilach Mollick, Natalie Bach, L

    Ethan R. Mollick, Lilach Mollick, Natalie Bach, L. J. Ciccarelli, Ben Przystanski, and Daniel Ravipinto. 2024. https://doi.org/10.48550/arXiv.2407.12796 Ai agents and education: Simulated practice at scale . arXiv preprint

    2024 doi arXiv

  11. [19]

    National Center for Education Statistics . 2022. The nation's report card: 2022 naep reading and mathematics assessments. https://nces.ed.gov/nationsreportcard/. Accessed: 2025-04-20

    2022

  12. [20]

    OpenAI. 2023. GPT-3.5-Turbo [large language model]. https://platform.openai.com/docs/models/gpt-3-5-turbo. Accessed: 2025-04-25

    2023

  13. [21]

    OpenAI. 2025. OpenAI o3-mini [large language model]. https://platform.openai.com/docs/models/o3-mini. Accessed: 2025-04-25

    2025

  14. [22]

    Georg Rasch. 1960. Probabilistic Models for Some Intelligence and Attainment Tests. Danish Institute for Educational Research, Copenhagen. Reprinted by University of Chicago Press (1980) and MESA Press (1992)

    1980

  15. [23]

    MacLellan

    Glen Smith, Adit Gupta, and Christopher J. MacLellan. 2024. https://arxiv.org/abs/2404.07883 Apprentice tutor builder: A platform for users to create and personalize intelligent tutors . arXiv preprint

    2024 arXiv

  16. [24]

    Shashank Sonkar, Xinghe Chen, Naiming Liu, Richard G Baraniuk, and Mrinmaya Sachan. 2024. Llm-based cognitive models of students with misconceptions. arXiv preprint arXiv:2410.12294

    2024 arXiv

  17. [25]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  18. [26]

    Quan Tu, Shilong Fan, Zihang Tian, Tianhao Shen, Shuo Shang, Xin Gao, and Rui Yan. 2024. https://doi.org/10.18653/v1/2024.acl-long.638 C haracter E val: A C hinese benchmark for role-playing conversational agent evaluation . In Proceedings of the 62nd Annual Meeting of the Ass...

    2024 doi

  19. [27]

    UNESCO . 2023. Global education monitoring report 2023: Technology in education. https://unesdoc.unesco.org/ark:/48223/pf0000385723

    2023

  20. [28]

    Department of Education

    U.S. Department of Education . 2023. Artificial intelligence and the future of teaching and learning: Insights and recommendations. https://www.ed.gov/sites/ed/files/documents/ai-report/ai-report.pdf

    2023

  21. [29]

    Shen Wang, Tianlong Xu, Hang Li, Chaoli Zhang, Joleen Liang, Jiliang Tang, Philip S Yu, and Qingsong Wen. 2024. Large language models for education: A survey and outlook. arXiv preprint arXiv:2403.18105

    2024 arXiv

  22. [30]

    Joel Williams. 2003. The Skills for Life survey: A national needs and impact survey of literacy, numeracy and ICT skills. 490. The Stationery Office

    2003

  23. [31]

    Beverly Woolf, Ivon Arroyo, and 1 others. 2013. Intelligent tutoring systems by and for the developing world: A review of trends and opportunities. International Journal of Artificial Intelligence in Education, 24(3):331--367

    2013

  24. [32]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, and 1 others. 2024. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115

    2024 arXiv

  25. [33]

    Eric Zelikman, Wanjing Ma, Jasmine Tran, Diyi Yang, Jason Yeatman, and Nick Haber. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.135 Generating and evaluating tests for k-12 students with language model simulations: A case study on sentence reading efficiency . In Proceedi...

    2023 doi

  26. [34]

    Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024. Explainability for large language models: A survey. ACM Transactions on Intelligent Systems and Technology, 15(2):1--38

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.