Pith. sign in

REVIEW 3 major objections 6 minor 50 references

EducationQ: Evaluating LLMs' Teaching Capabilities Through Multi-Agent Dialogue Framework

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper reports that LLM teaching ability is a distinct, measurable capability that does not track model scale or reasoning benchmarks, with a 70B open model outperforming larger commercial models in simulated student dialogues.

desk verdict A genuinely new framework for evaluating LLMs as teachers, with a central empirical claim that is not yet proven because the post-test lacks a no-teacher control; still worth refereeing, with heavy revision expected. read the letter →

arxiv 2504.14928 v3 pith:CRJGZJRV submitted 2025-04-21 cs.AI cs.CEcs.CLcs.CYcs.HC

Yao Shi , Rongkeng Liang , Yong Xu This is my paper · ORCID
classification cs.AIcs.CEcs.CLcs.CYcs.HC
keywords LLMevaluationteachingcapabilitymulti-agentdialogueformativeassessmentlearninggainLLM-as-teacherGPQAMMLU-Pro
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EducationQ proposes a way to test how well large language models teach, separate from how much they know. It stages a five-round dialogue between a fixed simulated student and the model being evaluated, on 1,498 graduate-level questions, and scores the teacher by the student's accuracy gain from pre-test to post-test. The paper reports that this teaching ranking does not follow model scale or general reasoning scores: Llama 3.1 70B Instruct tops the list with an average 11.01% gain, ahead of Gemini 1.5 Pro 002 at 7.48%, Llama 3.1 405B Instruct at 6.14%, and GPT-4o-mini at 2.44%. Expert teachers agreed with the automated evaluator's choice in 78% of anonymised comparisons, and the authors use this to argue the qualitative assessment is meaningful. If the finding holds, selecting and building educational LLMs requires measuring interactive pedagogy, not just benchmark accuracy.

What carries the argument

The load-bearing mechanism is the multi-agent formative-assessment loop: a teacher agent, a fixed student agent (Llama 3.1 70B Instruct), and a GPT-4o evaluator agent. The teacher is given the student's pre-test reasoning and correctness, but never the answer options, and conducts five rounds of questioning; between pre-test and post-test the same questions are readministered. Teaching effectiveness is the ALG metric, $\mathrm{ALG} = \mathrm{ACC}_{\mathrm{post}} - \mathrm{ACC}_{\mathrm{pre}}$; stability and uniqueness metrics (PNIR, CSS, UIC) support the ranking, and a 17-dimension evaluator rubric turns dialogues into qualitative scores that correlate with human judgments. The critical design choice is the enforced information boundary: teachers cannot see answer options, so gains are attributed to pedagogical dialogue rather than answer leakage.

What would settle it

Run the identical pre/post protocol with a no-teacher control and with a post-test made of new but matched questions; if the no-teacher control yields a similar ALG, or if gains disappear on held-out questions, the EducationQ ranking is measuring test familiarity rather than teaching.

Watch

Extended reading notes

Core claim

The paper's central claim is that teaching is a measurable, separable capability in LLMs, and that its ranking cannot be inferred from the usual benchmarks. EducationQ's triadic setup—teacher, student, evaluator—produces a quantitative teaching score per model: Absolute Learning Gain (ALG), the percentage-point improvement in a fixed student agent's accuracy from pre-test to post-test after five teacher turns. On 1,498 questions spanning 13 disciplines and 10 difficulty levels, the best teacher is Llama 3.1 70B Instruct (ALG 11.01%), followed by Gemini 1.5 Pro 002 (7.48%), with Llama 3.1 405B Instruct at 6.14%, OpenAI o1-mini at 5.84%, and GPT-4o-mini at 2.44%. The authors report model-specific teaching styles—progressive questioning and scaffolding for Llama 3.1 70B, targeted feedback for Gemini 1.5 Pro 002, and reasoning-heavy support for o1-mini—and note that no expert reviewer observed a teacher revealing answers. They take these results to challenge the assumption that larger scale or higher general intelligence directly improves teaching.

Load-bearing premise

The load-bearing premise is that the post-test gain is caused by the teacher's dialogue, but since the post-test repeats the same questions and includes the dialogue in the student's context, part of the gain could come from re-exposure or hint-following rather than durable teaching.

Editorial extensions

If this is right

  • Educational model selection should treat teaching ability as an independent axis, not as a corollary of reasoning benchmarks.
  • Dialogue-based simulated evaluation can replace some human panels for ranking teaching quality, with the 78% agreement supporting scaled qualitative review.
  • Models have complementary teaching styles, so a tutoring system could route students to the teacher model best matched to the subject and the student's state.
  • Scaling parameter count alone is not a route to better teaching; optimising questioning and feedback behaviour is.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper has no no-teacher control: the same gain could partly come from re-reading the question or following hints embedded in the dialogue, so a no-teacher and a hint-only baseline would reveal how much of the 11% is teaching.
  • Because the post-test reuses the identical pre-test questions and includes the dialogue in the student's context, the measured gains may reflect short-term answer reshaping rather than durable understanding; a held-out, same-topic post-test would distinguish these.
  • Since the student is a single 70B model, the ranking may be specific to that simulated learner; using novice or grade-school student personas could reorder the teachers.
  • An extension the authors do not pursue is to use the same evaluator loop to give formative feedback to teacher models, turning the benchmark into a training signal.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces EducationQ, a multi-agent dialogue framework for evaluating LLMs' teaching capabilities. The framework pairs a fixed student agent (Llama 3.1 70B Instruct) with a teacher agent under evaluation across five dialogue rounds per question, using a pre-test/post-test design on 1,498 questions from GPQA Diamond and a newly constructed MMLU-Pro Stratified set. The main outcome metric is Absolute Learning Gain (ALG), defined as the difference between post-test and pre-test accuracy. Testing 14 LLMs, the authors report that Llama 3.1 70B Instruct achieves the highest ALG (11.01%), followed by Gemini 1.5 Pro 002 (7.48%), while larger commercial models such as GPT-4o-mini score much lower (2.44%). They supplement this with qualitative evaluator-agent analyses, expert case studies, and a human-alignment study reporting 78% agreement. The central claim is that teaching effectiveness does not correlate linearly with model scale or general reasoning ability.

Significance. If the measurement of ALG were robust, this would be a valuable contribution: it provides a large-scale, reproducible, multi-agent methodology for an important but under-benchmarked capability (LLM-as-teacher), with a substantial released corpus of teacher-student dialogues and an ablation study across student models. The external human-expert alignment check (78% on 50 pairs) is a genuine strength, as are the open-source commitment and the explicit content-boundary controls preventing direct answer disclosure. However, the main quantitative claim rests on ALG in a pre/post design with identical questions and a post-test prompt that includes the full teaching dialogue; without a no-teacher control, the framework cannot separate genuine learning from re-exposure effects and hint-following. The ranking of models in Table 4 is therefore not yet established as a ranking of teaching quality. The finding that small open models can beat large commercial ones is interesting and plausible, but the current evidence does not yet support it at the level of statistical confidence required for the paper's headline claim.

major comments (3)
  1. [Section 4.7 and Eq. (1)] The post-test protocol reuses the exact same questions as the pre-test and, per Section 4.7, includes both the student's pre-test reasoning and the full teacher-student dialogue in the prompt. Consequently, a positive ALG can result from re-exposure to the question text or from answer-relevant cues embedded in the dialogue, rather than from durable learning. The framework has no no-teacher control condition (e.g., a post-test with the same question but no teaching dialogue), so the ranking in Table 4 may reflect how well each teacher happens to steer the student toward the correct option within the dialogue context. The case studies in Appendix F.1 and F.2 illustrate this: the teacher dialogues converge on the correct option, and the post-test answers closely echo the reasoning scaffolded by the teacher. This is not acknowledged in the Limitations section. I recommend adding a no-teacher control (post-test with only the question and the student's own pre-test reasoning) and reporting ALG relative to that baseline, ideally accompanied by a retention test on non-overlapping questions to assess durable learning.
  2. [Table 4 and Section 7.1] The overall ALG values in Table 4 are reported as point estimates with no confidence intervals, standard errors, or significance tests. Several adjacent rankings differ by fractions of a percentage point (e.g., Hermes 3 Llama 3.1 70B at 4.14%, Mistral Nemo at 3.94%, Claude 3.5 Sonnet at 3.81%). The stability study in Table 3 shows run-to-run variance for three models on GPQA-main, but these variances are not applied to the main results; for example, the reported variance of 0.01246 for Llama 3.1 405B corresponds to a standard deviation of about 0.11 percentage points, which is comparable to some of the gaps in Table 4. Without uncertainty quantification or pairwise significance tests, the specific ordering of models in Table 4 is not statistically supported. Please provide bootstrap confidence intervals over questions for each ALG and report which pairwise differences are significant after appropriate multiple-comparison correction, or explicitly state which differences are within noise.
  3. [Section 8.2] The 78% human-expert agreement validates the evaluator agent's comparative judgments of teaching behaviors on 50 anonymized dialogue pairs, which is a useful external check. However, it does not validate that ALG measures durable learning. The human experts were asked to rate teaching behaviors and to check for direct answer disclosure; they were not asked whether the student's post-test performance reflects understanding that would transfer to new questions. Given that the post-test prompt includes the dialogue content (Section 4.7), the experts' confirmation of 'no direct answer disclosure' does not rule out that students are following embedded hints rather than genuinely learning. Please either add a transfer test (post-test on previously unseen but related questions) or explicitly scope the paper's claims from 'teaching quality' to 'context-conditioned answering in the presence of a dialogue.'
minor comments (6)
  1. [Conclusion] The word 'fudamental' in the Conclusion should be 'fundamental.'
  2. [Model Content Limitations] The phrase 'protential impacts' should be 'potential impacts.'
  3. [Appendix A.3.3] The heading 'Dimensons' is misspelled; it should be 'Dimensions.'
  4. [References] The reference to 'Rob Wass and Clinton Golding and. 2014' contains a stray 'and' and should be cleaned up.
  5. [Appendix F.1] The case study in Table 22 uses inconsistent teacher labels: the header refers to Teacher 1 and Teacher 2, while the evaluator analysis refers to Teacher A and Teacher B, making it difficult to map the verdicts back to the models. Please align the labeling throughout.
  6. [Section 4.1] The student model is referred to as 'Mistral Nemo 12b' in Section 4.1 but as 'Mistral Nemo' in other tables; please standardize the model naming.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ALG is a directly measured pre/post accuracy difference, and the qualitative evaluation is externally validated against human experts.

full rationale

EducationQ's central quantity is ALG = ACCpost - ACCpre (Eq. 1), computed by running the same student agent on pre- and post-tests around a five-round teacher-student dialogue. No parameter is fitted to the outcome and then renamed a prediction: the reported ALG values (Table 4) are direct accuracy differences on GPQA Diamond and MMLU-Pro Stratified, and the headline ranking is an observed ordering, not the output of a model calibrated to itself. The qualitative evaluator (GPT-4o) is validated against seven human educators with 78% agreement on 50 anonymized pairs, which is an external check rather than a self-citation. There are no load-bearing self-citations by the present authors; the cited work (MMLU-Pro, GPQA, MT-Bench, etc.) is external. The only self-reference is practical: the student agent is Llama 3.1 70B Instruct, which is also one of the teacher models, and the evaluator is GPT-4o while GPT-4o-mini is ranked. This shared-model setup could interact with the measurements, but it does not make ALG equal to an input by construction. A separate validity concern—the post-test reuses the same questions and includes the dialogue and the student's pre-test reasoning, without a no-teacher control (Section 4.7)—could mean ALG partly captures re-exposure or hint-following rather than durable learning. That is a substantive experimental-design criticism, not circularity: the paper's equations do not reduce to themselves, and the claim would remain a measurement claim even if the measurement's construct validity were disputed. No circular step was exhibited.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The framework's rankings rest on a small number of protocol choices (student model, dialogue length, token limits, difficulty stratification) and on the assumption that a fixed LLM's accuracy gain on repeated questions is a valid proxy for real learning. None of these are fitted values in a statistical sense, but they were tuned by the authors' ablations and they materially affect the measured gains. No new physical or mathematical entities are introduced.

free parameters (4)
  • dialogue rounds = 5
    Chosen via ablations in Section 4.9; doubling to 10 rounds did not improve ALG, and shorter teacher turns degraded performance. The central rankings depend on this protocol choice.
  • teacher token limit = 150 tokens
    Ablation-selected in Section 4.9; affects how much scaffolding each teacher can express per turn and therefore the measured gains.
  • student agent model = Meta-Llama-3.1-70B-Instruct
    A fixed student model is assumed to be a neutral measuring instrument; ablations with Qwen 2.5 72B and Mistral Nemo show stable rank order (Table 2), but the choice still shapes all ALG values.
  • difficulty stratum boundaries = 10 percentage-point bins based on top-10 MMLU-Pro model accuracies
    Used to build MMLU-Pro Stratified (Section 3); the distribution of easy and hard items shapes ALG by subject and difficulty level.
assumptions (4)
  • domain assumption ALG on identical pre/post questions measures teaching effectiveness
    Section 4.4 defines ALG as the teacher's effectiveness measure; no no-teacher retest control is reported, so re-exposure effects are not separated from teaching effects.
  • domain assumption A fixed LLM student simulates a real learner well enough for ranking teacher models
    Section 4.1 and the Limitations section acknowledge that more sophisticated generative student models were not used; if the student model is not learner-like, rankings may not transfer to humans.
  • domain assumption Withholding answer options from the teacher prevents answer leakage
    Section 4.8 enforces content boundaries; human experts found no direct disclosure (Section 8.2), but indirect leakage through correctness judgments and question text remains possible.
  • domain assumption The fixed teacher system prompt is a fair test of each model's teaching capability
    Appendix A.2.2 uses the same prompt for all models with temperature 0 and no few-shot examples; models may differ in instruction following for this specific prompt, so the ranking is prompt-conditioned.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EducationQ: Evaluating LLMs' Teaching Capabilities Through Multi-Agent Dialogue Framework." pith.science (2026). https://pith.science/paper/CRJGZJRV

@misc{pith2026250414928,
  author       = {Pith},
  title        = {Pith review of: EducationQ: Evaluating LLMs' Teaching Capabilities Through Multi-Agent Dialogue Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CRJGZJRV}},
  note         = {Machine review of arXiv:2504.14928}
}
read the original abstract

Large language models (LLMs) increasingly serve as educational tools, yet evaluating their teaching capabilities remains challenging due to the resource-intensive, context-dependent, and methodologically complex nature of teacher-student interactions. We introduce EducationQ, a multi-agent dialogue framework that efficiently assesses teaching capabilities through simulated dynamic educational scenarios, featuring specialized agents for teaching, learning, and evaluation. Testing 14 LLMs across major AI Organizations (OpenAI, Meta, Google, Anthropic, and others) on 1,498 questions spanning 13 disciplines and 10 difficulty levels reveals that teaching effectiveness does not correlate linearly with model scale or general reasoning capabilities - with some smaller open-source models outperforming larger commercial counterparts in teaching contexts. This finding highlights a critical gap in current evaluations that prioritize knowledge recall over interactive pedagogy. Our mixed-methods evaluation, combining quantitative metrics with qualitative analysis and expert case studies, identifies distinct pedagogical strengths employed by top-performing models (e.g., sophisticated questioning strategies, adaptive feedback mechanisms). Human expert evaluations show 78% agreement with our automated qualitative analysis of effective teaching behaviors, validating our methodology. EducationQ demonstrates that LLMs-as-teachers require specialized optimization beyond simple scaling, suggesting next-generation educational AI prioritize targeted enhancement of specific pedagogical effectiveness.

Figures

Figures reproduced from arXiv: 2504.14928 by the authors.

Figure 1
Figure 1. The evolution of LLMs in education: from individual single-turn tasks to dynamic educational scenarios [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Dataset distribution across 13 academic dis [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The formative assessment interaction flow in the EducationQ framework, detailing the multi-agent [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 19 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Linda Allal and Greta Pelgrims Ducrey . 2000. https://doi.org/10.1016/S0959-4752(99)00025-0 Assessment of—or in—the zone of proximal development . Learning and Instruction, 10(2):137--152

    work page doi:10.1016/s0959-4752(99)00025-0 2000

  4. [4]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian...

    arXiv 2021

  5. [5]

    Yuyan Chen, Chenwei Wu, Songzhou Yan, Panjun Liu, Haoyu Zhou, and Yanghua Xiao. 2024. https://arxiv.org/abs/2408.10947 Dr.academy: A benchmark for evaluating questioning capability in education for large language models . Preprint, arXiv:2408.10947

    arXiv 2024

  6. [6]

    Clayton Cohn, Nicole Hutchins, Tuan Le, and Gautam Biswas. 2024. https://doi.org/10.1609/aaai.v38i21.30364 A chain-of-thought prompting approach with llms for evaluating students’ formative assessment responses in science . Proceedings of the AAAI Conference on Artificial Intelligence, 38(21):23182--23190

    doi:10.1609/aaai.v38i21.30364 2024

  7. [7]

    Duschl and Drew H

    Richard A. Duschl and Drew H. Gitomer. 1997. https://doi.org/10.1207/s15326977ea0401\_2 Strategies and challenges to changing the focus of assessment and instruction in science classrooms . Educational Assessment, 4(1):37--73

    work page doi:10.1207/s15326977ea0401 1997

  8. [8]

    Gitomer and Richard A

    Drew H. Gitomer and Richard A. Duschl. 2007. https://doi.org/10.1111/j.1744-7984.2007.00106.x chapter 12 establishing multilevel coherence in assessment . Yearbook of the National Society for the Study of Education, 106(1):288--320

    work page arXiv 2007

Show all 50 references
  1. [9]

    Thomas R Guskey. 2005. https://files.eric.ed.gov/fulltext/ED490412.pdf Formative classroom assessment and benjamin s. bloom: Theory, research, and implications . Online submission

    2005

  2. [10]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021 a . https://arxiv.org/abs/2009.03300 Measuring massive multitask language understanding . Preprint, arXiv:2009.03300

    2021 arXiv

  3. [11]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021 b . https://arxiv.org/abs/2103.03874 Measuring mathematical problem solving with the math dataset . Preprint, arXiv:2103.03874

    2021 arXiv

  4. [12]

    Cindy E Hmelo-Silver and Howard S Barrows. 2006. https://doi.org/10.7771/1541-5015.1004 Goals and strategies of a problem-based learning facilitator . Interdisciplinary Journal of Problem-Based Learning, 1(1):4

    2006

  5. [13]

    Hyoungwook Jin, Minju Yoo, Jeongeon Park, Yokyung Lee, Xu Wang, and Juho Kim. 2025. https://arxiv.org/abs/2410.04078 Teachtune: Reviewing pedagogical agents against diverse student profiles with simulated students . Preprint, arXiv:2410.04078

    2025 arXiv

  6. [14]

    Takeshi Kojima, Shixiang (Shane) Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. https://proceedings.neurips.cc/paper_files/paper/2022/file/8bb0d291acd4acf06ef112099c16f326-Paper-Conference.pdf Large language models are zero-shot reasoners . In Advances in Neural Inf...

    2022

  7. [15]

    Krathwohl

    David R. Krathwohl. 2002. https://doi.org/10.1207/s15430421tip4104\_2 A revision of bloom's taxonomy: An overview . Theory Into Practice, 41(4):212--218

    2002 doi

  8. [16]

    Zhenwen Liang, Dian Yu, Wenhao Yu, Wenlin Yao, Zhihan Zhang, Xiangliang Zhang, and Dong Yu. 2024. https://arxiv.org/abs/2405.19444 Mathchat: Benchmarking mathematical reasoning and instruction following in multi-turn interactions . Preprint, arXiv:2405.19444

    2024 arXiv

  9. [17]

    Jiawen Liu, Yuanyuan Yao, Pengcheng An, and Qi Wang. 2024. https://doi.org/10.1145/3613905.3651008 Peergpt: Probing the roles of llm-based peer agents as team moderators and participants in children's collaborative learning . In Extended Abstracts of the CHI Conference on Huma...

    2024

  10. [18]

    Janet Looney. 2011. https://doi.org/10.1787/5kghx3kbl734-en Integrating formative and summative assessment: Progress toward a seamless system? Technical Report 58, OECD Publishing, Paris

    2011 doi

  11. [19]

    Xinyi Lu and Xu Wang. 2024. https://doi.org/10.1145/3657604.3662031 Generative students: Using llm-simulated student profiles to support question item evaluation . In Proceedings of the Eleventh ACM Conference on Learning @ Scale, L@S '24, page 16–27, New York, NY, USA. Associ...

    2024

  12. [20]

    Markel, Steven G

    Julia M. Markel, Steven G. Opferman, James A. Landay, and Chris Piech. 2023. https://doi.org/10.1145/3573051.3593393 Gpteach: Interactive ta training with gpt-based students . In Proceedings of the Tenth ACM Conference on Learning @ Scale, L@S '23, page 226–236, New York, NY, ...

    2023

  13. [21]

    Cecile Hoareau McGrath, Benoit Guerin, Emma Harte, Michael Frearson, and Catriona Manville. 2015. https://doi.org/10.7249/RR996 Learning gain in higher education . RAND Corporation, Santa Monica, CA

    2015 doi

  14. [22]

    Bahar Memarian and Tenzin Doleck. 2023. https://doi.org/10.1016/j.chbah.2023.100022 Chatgpt in education: Methods, potentials, and limitations . Computers in Human Behavior: Artificial Humans, 1(2):100022

    2023

  15. [23]

    Neil Mercer and Karen Littleton. 2007. https://doi.org/10.4324/9780203946657 Dialogue and the Development of Children's Thinking: A Sociocultural Approach , 1st edition. Routledge, London

    2007 doi

  16. [24]

    Fengchun Miao, Wayne Holmes, Ronghuai Huang, Hui Zhang, et al. 2021. https://doi.org/10.54675/PCSP7350 AI and education: A guidance for policymakers . UNESCO Publishing

    2021 doi

  17. [25]

    Benjamin D Nye, Dillon Mee, and Mark G Core. 2023. https://ceur-ws.org/Vol-3487/paper4.pdf Generative large language models for dialog-based tutoring: An early consideration of opportunities and concerns . In LLM@AIED, pages 78--88

    2023

  18. [26]

    Andrew Olney. 2023. https://par.nsf.gov/biblio/10447593 Generating multiple choice questions from a textbook: Llms match human performance on most metrics . In Workshop on Empowering Education with LLMs - the Next-Gen Interface and Content Generation at the AIED'23 Conference

    2023

  19. [27]

    Sullivan Palincsar

    A. Sullivan Palincsar. 1998. https://doi.org/10.1146/annurev.psych.49.1.345 Social constructivist perspectives on teaching and learning . Annual Review of Psychology, 49(Volume 49, 1998):345--375

    1998 doi

  20. [28]

    Abelardo Pardo, Jelena Jovanovic, Shane Dawson, Dragan Gašević, and Negin Mirriahi. 2019. https://doi.org/10.1111/bjet.12592 Using learning analytics to scale the provision of personalised feedback . British Journal of Educational Technology, 50(1):128--138

    2019 doi

  21. [29]

    Bernstein

    Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. https://doi.org/10.1145/3586183.3606763 Generative agents: Interactive simulacra of human behavior . In Proceedings of the 36th Annual ACM Symposium on User Int...

    2023

  22. [30]

    Rajlaxmi Patil, Aditya Ashutosh Kulkarni, Ruturaj Ghatage, Sharvi Endait, Geetanjali Kale, and Raviraj Joshi. 2024. https://arxiv.org/abs/2409.15749 Automated assessment of multimodal answer sheets in the stem domain . Preprint, arXiv:2409.15749

    2024 arXiv

  23. [31]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2023. https://arxiv.org/abs/2311.12022 Gpqa: A graduate-level google-proof q&a benchmark . Preprint, arXiv:2311.12022

    2023 arXiv

  24. [32]

    Maria Araceli Ruiz-Primo and Erin Marie Furtak. 2007. https://doi.org/10.1002/tea.20163 Exploring teachers' informal formative assessment practices and students' understanding in the context of scientific inquiry . Journal of Research in Science Teaching, 44(1):57--84

    2007 doi

  25. [33]

    Asli Sezen-Barrie and Gregory J. Kelly. 2017. https://doi.org/10.1080/09500693.2016.1274921 From the teacher’s eyes: facilitating teachers noticings on informal formative assessments (ifas) and exploring the challenges to effective implementation . International Journal of Sci...

    2017

  26. [34]

    Kumar Shridhar, Jakub Macina, Mennatallah El-Assady, Tanmay Sinha, Manu Kapur, and Mrinmaya Sachan. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.277 Automatic generation of socratic subquestions for teaching math word problems . In Proceedings of the 2022 Conference on Em...

    2022 doi

  27. [35]

    Department of Education, Office of Educational Technology

    U.S. Department of Education, Office of Educational Technology . 2023. https://www.ed.gov/sites/ed/files/documents/ai-report/ai-report.pdf Artificial intelligence and future of teaching and learning: Insights and recommendations . Technical report, U.S. Department of Education...

    2023

  28. [36]

    L. S. Vygotsky. 1978. https://doi.org/10.2307/j.ctvjf9vz4 Mind in Society: Development of Higher Psychological Processes . Harvard University Press, Cambridge, MA

    1978 doi

  29. [37]

    Siyuan Wang, Zhuohan Long, Zhihao Fan, Zhongyu Wei, and Xuanjing Huang. 2024 a . https://arxiv.org/abs/2402.11443 Benchmark self-evolving: A multi-agent framework for dynamic llm evaluation . Preprint, arXiv:2402.11443

    2024 arXiv

  30. [38]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. https://doi.org/10.18653/v1/2023.acl-long.754 Self-instruct: Aligning language models with self-generated instructions . In Proceedings of the 61st Annual Mee...

    2023 doi

  31. [39]

    Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuzni...

    2022

  32. [40]

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. 2024 b . https://arxiv.org/abs/2406.01574 Mmlu-pro: A more robust ...

    2024 arXiv

  33. [41]

    Rob Wass and Clinton Golding and. 2014. https://doi.org/10.1080/13562517.2014.901958 Sharpening a tool for teaching: the zone of proximal development . Teaching in Higher Education, 19(6):671--684

    2014

  34. [42]

    J. Wei, M. Bosma, V. Zhao, K. Guu, A.W. Yu, B. Lester, N. Du, A.M. Dai, and Q.V. Le. 2022. https://arxiv.org/abs/2109.01652 Finetuned language models are zero-shot learners . Preprint, arXiv:2109.01652

    2022 arXiv

  35. [43]

    Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.167 Large language models are better reasoners with self-verification . In Findings of the Association for Computational L...

    2023 doi

  36. [44]

    William W Wilen. 1987. https://files.eric.ed.gov/fulltext/ED310102.pdf Questions, questioning techniques, and effective teaching . National Education Association, Washington, DC

    1987

  37. [45]

    Dylan William. 2011. https://doi.org/10.1016/j.stueduc.2011.03.001 What is assessment for learning? Studies in Educational Evaluation, 37(1):3--14

    2011 doi

  38. [46]

    Bruner, and Gail Ross

    David Wood, Jerome S. Bruner, and Gail Ross. 1976. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x The role of tutoring in problem solving . Journal of Child Psychology and Psychiatry, 17(2):89--100

    1976

  39. [47]

    Songlin Xu and Xinyu Zhang. 2023. https://arxiv.org/abs/2310.19206 Leveraging generative artificial intelligence to simulate student learning behavior . Preprint, arXiv:2310.19206

    2023 arXiv

  40. [48]

    Siyu Zha, Yujia Liu, Chengbo Zheng, Jiaqi XU, Fuze Yu, Jiangtao Gong, and Yingqing XU. 2024. https://arxiv.org/abs/2409.14228 Mentigo: An intelligent agent for mentoring students in the creative problem solving process . Preprint, arXiv:2409.14228

    2024 arXiv

  41. [49]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. https://arxiv.org/abs/2306.05685 Judging llm-as-a-judge with mt-bench and chatbot arena . P...

    2023 arXiv

  42. [50]

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. https://arxiv.org/abs/2311.07911 Instruction-following evaluation for large language models . Preprint, arXiv:2311.07911

    2023 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.