Pith. sign in

REVIEW 3 major objections 4 minor 60 references

A Novel Approach to Scalable and Automatic Topic-Controlled Question Generation in Education

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A small, open question generator can produce education questions that stick to a prescribed topic.

desk verdict Solid, modest contribution to small-model topic-controlled question generation, but the headline metric may partly reward superficial topic mention because it shares its entity linker with the topic definition. read the letter →

arxiv 2501.05220 v1 pith:2UN5B3LY submitted 2025-01-09 cs.CY cs.AIcs.CLcs.IR

Ziqing Li , Mutlu Cukurova , Sahan Bulathwela This is my paper · ORCID
classification cs.CYcs.AIcs.CLcs.IR
keywords topic-controlledquestiongenerationeducationalsmalllanguagemodelsT5fine-tuningcontrastivedataaugmentationWikipediasemanticrelatednessmodelquantisationscalableAIineducation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a small, open language model can be fine-tuned to generate educational questions that track a prescribed topic, without the cost or privacy concerns of calling a large proprietary model. The authors build contrastive training datasets from SQuAD and KhanQ, linking each context-question pair to a Wikipedia concept that serves as the topic, and train T5-small variants on them. They also propose a Wikipedia-based semantic relatedness metric, WikiSemRel (Jaccard), and validate it against human judgments. On the MixKhanQ evaluation set, their best model, TopicQG2X, shows the largest gap between questions generated on the prescribed topic and questions generated on an alternative topic, which they interpret as superior topic control. If the result holds, teachers and tutoring systems could generate topic-specific questions with a model small enough to run on a phone.

What carries the argument

The load-bearing mechanism is the contrastive data-construction method. Starting from SQuAD and KhanQ pairs, the authors wikify each context and question, keep the top five Wikipedia concepts by PageRank, and select the most authoritative concept in the question as the target topic; they then concatenate two randomly chosen contexts into one shared context and ask the model to generate a question for a given topic from the combined paragraph, creating MixSQuAD and, with both orderings, MixSQuAD2X. This forces the model to learn which concepts in a long context the topic points to. The other central object is the evaluation metric, WikiSemRel with the Jaccard measure, which computes relatedness from outward Wikipedia links and, per the paper's human annotation study, agrees with human judgments better than BERTScore or the w2v variant.

What would settle it

Take a fresh set of, say, 200 Khan Academy question contexts, generate questions with the prescribed and an alternative topic using TopicQG2X, and have teachers judge which set is more on topic; if teacher judgments do not reproduce the reported WikiSemRel (Jaccard) gap, the central claim fails.

Watch

Extended reading notes

Core claim

The central discovery is that fine-tuning a T5-small model (about 60M parameters) on contrastively mixed contexts makes its questions follow a specified topic more closely. On the MixKhanQ evaluation set, the best model, TopicQG2X, achieves a WikiSemRel (Jaccard) difference of 0.680 between questions generated with the prescribed topic and questions generated with an alternative topic, against 0.545 for the baseline, and it surpasses all other models on this indicator. The paper also reports that the TopicQG model improves over the baseline on most linguistic-quality metrics, that 8-bit and 4-bit quantisation cut memory from about 230MB to around 110MB and 94MB with only small metric drops, and that further pre-training on scientific text (TopicQGedu) does not yield a statistically significant improvement. These results are offered as evidence that topic-controlled educational question generation is feasible with a small, locally deployable model rather than requiring proprietary large language models.

Load-bearing premise

The entire topic-control result rests on a semantic-similarity metric validated with only 30 human-rated question pairs, so if that metric misses what teachers actually mean by 'on topic,' the headline improvement may be an artifact.

Editorial extensions

If this is right

  • Teachers and tutoring systems could generate topic-specific questions from paragraph-level materials using an open model small enough to deploy on a phone.
  • Data augmentation by reversing context order is a cheap way to strengthen topical control without new human annotation.
  • Quantised versions of the model cut memory by roughly half while keeping most of the quality gains, lowering infrastructure and energy costs.
  • The WikiSemRel (Jaccard) metric can serve as an automated substitute for human ratings of topical relatedness in educational question generation.
  • Educational institutions can avoid depending on externally hosted proprietary models for this task, addressing privacy and governance concerns.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit is that the same contrastive pairing recipe could be extended to control other generation properties, such as question difficulty, length, or cognitive load, by replacing the topical-pairing criterion with those attributes.
  • Because the topic signal is whatever Wikipedia concept has the highest PageRank in the reference question, the model's 'topic' is an annotation artifact; whether it matches the curriculum topic a teacher intends is a question the paper does not yet test.
  • The metric validation rests on 30 question pairs, so a larger human study is the natural next check: if WikiSemRel (Jaccard) fails to track teacher judgments at scale, the headline topic-control gains would need re-interpretation.
  • One could test the approach on other educational domains and languages; the paper's evidence is confined to English STEM content from Khan Academy and SQuAD-derived Wikipedia articles.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper proposes Topic-Controlled Question Generation (T-CQG) by fine-tuning T5-small. The authors create SQuAD+, MixSQuAD, and MixSQuAD2X from SQuAD by wikifying contexts and questions, filtering on concept overlap, and concatenating context pairs to create contrastive examples. They evaluate a Baseline, TopicQG, TopicQGedu, quantised variants, and TopicQG2X on MixKhanQ, using lexical and embedding metrics plus Wikipedia-based semantic relatedness (WikiSemRel, w2v and Jaccard) to measure topical alignment. A small human annotation study (30 pairs, 4 annotators) is used to compare BERTScore and WikiSemRel. The headline finding is that TopicQG2X achieves the largest topic-vs-alternative difference on WikiSemRel (Jaccard), which is presented as evidence of superior topic control with a small open-source model. The paper also reports memory savings from 8-bit and 4-bit quantisation and concludes that the approach can reduce reliance on proprietary LLMs in educational question generation.

Significance. If the headline result holds, the paper offers a useful, low-cost alternative to proprietary LLMs for topic-focused educational question generation, and the dataset-construction and evaluation methodology could inform future work. The authors are transparent about limitations, and the approach is in principle reproducible using public datasets, an open model, and a linked code repository. However, the evidence for the central claim is currently weakened by the lack of inferential statistics for the key comparisons and by the dependence of both the control signal and the main evaluation metric on the same Wikipedia annotation pipeline. The paper also provides a useful human-evaluation baseline for semantic relatedness metrics, although the sample is small.

major comments (3)
  1. [Section 4.3, Table 4] The central claim that TopicQG2X 'surpasses all other models' and the RQ2/RQ5 comparisons rest on differences in WikiSemRel (Jaccard) that are reported without confidence intervals or significance tests. The only inferential test in the paper (paired t-test, p=0.083, in Section 4.2) is applied to a comparison that is not central to the headline result. Given that Table 2 reports MAE 0.23 for the Jaccard metric on the 30-pair human validation, the observed differences between models (e.g., 0.595 vs 0.545 for TopicQG vs Baseline, 0.680 vs 0.595 for TopicQG2X vs TopicQG) are not obviously outside the metric's noise level. Please report paired significance tests or bootstrap confidence intervals for all model comparisons in Tables 3 and 4.
  2. [Sections 3.2.1 and 3.6.2, with Section 3.4 and Table 2] The target topic t is defined as the most authoritative WAT Wikipedia concept in the reference question, and the WikiSemRel (Jaccard) evaluation score is computed from WAT concepts and their outward links in the generated and reference questions. The control signal and the evaluation instrument therefore share the same entity linker, creating a partial circularity: a model that echoes the topic entity string can raise the Jaccard score without producing a question with genuine topical depth. The human validation does not break this circularity because the four annotators were only asked to choose which of two generated questions was closer to a reference, not to judge whether the metric rewards superficial topic mention. Please add a validation study or a quantitative analysis (e.g., comparing scores of outputs with and without the surface topic string) that directly tests what the metric rewards.
  3. [Sections 3.2.3 and 4.3] The RQ5 conclusion that data augmentation improves topical relevance is confounded with dataset size. MixSQuAD2X is twice as large as MixSQuAD because two orderings of each context pair are included, so TopicQG2X is trained on strictly more examples than TopicQG. Setting aside the qualitative difference between the two orderings, the gain in Table 4 could be due to more training data rather than to the augmentation strategy itself. An ablation that trains TopicQG on the same number of examples, or that otherwise controls for training steps or data volume, is needed to support the claim that the reversal augmentation is the cause of the improvement.
minor comments (4)
  1. [Section 4.2] The sentence 'all newly proposed models except the 4bit quantised TopicQG model outperforms the baseline' is contradicted by Table 4, where TopicQGedu has a difference of 0.536, below the baseline's 0.545; please correct this factual error and adjust the surrounding RQ3 discussion.
  2. [Section 3.6.2 and Tables 2-4] The metric name is inconsistent: 'WikiSemRel' appears in Section 3.6.2 and the abstract, while Tables 2-4 and Section 4.1 use 'WikiSimRel'; please standardise the terminology throughout.
  3. [Section 4.2] The paired t-test is described only as comparing 'performance scores' of TopicQGedu and TopicQG; please specify which metric was tested, the sample size, and whether correction for multiple comparisons was considered, since the paper reports many metrics.
  4. [Throughout] There are several typos and minor wording issues, including 'BERTSCore' in Section 3.6.1, 'MATEOR' in Section 4.3, and the unexplained notation in equation (2); a careful copyedit would improve readability.

Circularity Check

1 steps flagged · score 4.0 of 10

Topic-definition and topic-evaluation metric share the same WAT/Wikification pipeline, creating partial self-reference in the headline topic-alignment claim.

  1. self definitional [Section 3.2.1, Section 3.6.2, Table 4 (Section 4.3)]
    "We select the concept with the highest PageRank score in the question (most authoritative) as the target topic 𝑡. ... WikiSemRel. quantifies the semantic relatedness between the Wikipedia-based concepts extracted from the reference question 𝑞𝑡 and the generated question ˆ𝑞𝑡 . We employ the WAT API [45] service ... Jaccard-based measure, that uses the outward links to other Wikipedia pages to calculate similarity [47]. We Wikify the generated question to compute the WikiSemRel score."

    The 'target topic' t is not an independent educational label; it is the WAT-linked concept with the highest PageRank in the reference question q_t (Section 3.2.1). The WikiSemRel (Jaccard) metric that produces Table 4's headline 'Difference' is computed by Wikifying generated and reference questions with the same WAT API and measuring outward-link overlap (Section 3.6.2). The control signal and evaluation instrument are therefore the same annotation pipeline: a generated question that echoes the topic string will be WAT-linked to the same entity that defines t, inflating Jaccard overlap with q_t. The TopicQG2X advantage (0.680 vs 0.545) may thus reflect surface topic-token reproduction rather than independent topicality.

full rationale

The paper's core training/evaluation split is genuine: models are trained on SQuAD-derived MixSQuAD/MixSQuAD2X and evaluated on KhanQ-derived MixKhanQ, with quality metrics (BLEU, ROUGE, METEOR, perplexity) that do not share machinery with the training labels. The self-citations in Sections 2 and 4 ([7], [25], [26]) support auxiliary claims (grammar post-processing, pre-training expectations) and are not load-bearing; RQ3's actual result (no significant gain) undercuts reliance on [7]. The only substantial circularity is the confluence of the topic-control signal and the topic-alignment metric: both use WAT/Wikification on the same reference questions. Because the 'topic' is the highest-PageRank WAT concept in q_t, and WikiSemRel (Jaccard) scores the WAT-link overlap between q̂_t and q_t, Table 4's difference partially measures whether the model reproduced the topic entity string. Section 3.4's human validation validates the metric as a proxy for human 'closeness to the reference question', not as an independent check of educational topical focus, so it does not eliminate this loop. The paper's acknowledgement of a small human sample (Section 6) is a separate limitation and does not address the shared-WAT issue. Overall: partial self-reference in the headline topic-alignment evidence, but independent quality metrics and a held-out evaluation set prevent the central claim from being fully forced by definition; no load-bearing self-citation chain.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim leans on standard pre-trained model assumptions and the Wikipedia annotation infrastructure for both topic labels and evaluation. While no exotic free parameters are tuned to the outcome, the top-5 concept cutoff, overlap threshold, dataset size, and hyperparameters are hand-picked. The invented_entities list is empty because no theoretical constructs are introduced.

free parameters (5)
  • Top-5 Wikipedia concepts retained per text = 5
    Hand-chosen cutoff in SQuAD+ construction (Section 3.2.1); the number of concepts retained determines which topic candidates are considered.
  • Concept overlap threshold |T_c ∩ T_q| >= 1 = 1
    Required intersection between context and question concepts to keep an example; filters the training set.
  • MixSQuAD dataset size = 10000
    Number of mixed entries generated by random pairing; affects training exposure.
  • Fine-tuning hyperparameters = batch=64, lr=1e-3, eps=1e-8, max 50 epochs
    Chosen without a reported sweep; these govern optimization and could affect the reported improvements.
  • MixSQuAD2X augmentation factor = 2
    Reversing context order doubles the data; the effect of this factor is central to RQ5.
assumptions (5)
  • domain assumption T5-small pre-trained weights from HuggingFace are a suitable base for educational question generation.
    The entire method builds on this foundation without re-validating it for the target task.
  • domain assumption Wikification via WAT identifies salient and correct Wikipedia concepts for educational texts.
    Used to define topics in SQuAD+ and to compute WikiSemRel; no error analysis of the annotator is provided.
  • ad hoc to paper The target topic can be represented as a single Wikipedia concept and used as a textual input to the model.
    Equation (1) defines p(q_t|c,t) with t as a topic token; the paper supplies t from the reference question's most authoritative concept.
  • domain assumption KhanQ reference questions are valid ground-truth educational questions for the given contexts.
    Used as references to compute BLEU, METEOR, ROUGE, and as the basis of MixKhanQ.
  • domain assumption Human judgments on 30 pairs are a reliable gold standard for semantic relatedness.
    Small sample, but high Fleiss kappa; the paper acknowledges limited statistical power.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Novel Approach to Scalable and Automatic Topic-Controlled Question Generation in Education." pith.science (2026). https://pith.science/paper/2UN5B3LY

@misc{pith2026250105220,
  author       = {Pith},
  title        = {Pith review of: A Novel Approach to Scalable and Automatic Topic-Controlled Question Generation in Education},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2UN5B3LY}},
  note         = {Machine review of arXiv:2501.05220}
}
read the original abstract

The development of Automatic Question Generation (QG) models has the potential to significantly improve educational practices by reducing the teacher workload associated with creating educational content. This paper introduces a novel approach to educational question generation that controls the topical focus of questions. The proposed Topic-Controlled Question Generation (T-CQG) method enhances the relevance and effectiveness of the generated content for educational purposes. Our approach uses fine-tuning on a pre-trained T5-small model, employing specially created datasets tailored to educational needs. The research further explores the impacts of pre-training strategies, quantisation, and data augmentation on the model's performance. We specifically address the challenge of generating semantically aligned questions with paragraph-level contexts, thereby improving the topic specificity of the generated questions. In addition, we introduce and explore novel evaluation methods to assess the topical relatedness of the generated questions. Our results, validated through rigorous offline and human-backed evaluations, demonstrate that the proposed models effectively generate high-quality, topic-focused questions. These models have the potential to reduce teacher workload and support personalised tutoring systems by serving as bespoke question generators. With its relatively small number of parameters, the proposals not only advance the capabilities of question generation models for handling specific educational topics but also offer a scalable solution that reduces infrastructure costs. This scalability makes them feasible for widespread use in education without reliance on proprietary large language models like ChatGPT.

Figures

Figures reproduced from arXiv: 2501.05220 by the authors.

Figure 1
Figure 1. Methodology for generating the different training datasets proposed in the model from the contexts [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Methodology for training and evaluating the Baseline model (black), TopicQGedu model (green, RQ3), TopicQG model (dark [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 40 canonical work pages

  1. [1]

    Derek Adamson, Deepak Bhartiya, Baljeet Gujral, Ritu Kedia, Ankit Singh, and Carolyn P. Rose. 2013. Automatically Generating Discussion Questions. In Proceedings of the International Conference on Artificial Intelligence in Education (AIED)

    work page 2013

  2. [2]

    Aida Amini, Saadia Gabriel, Peter Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. 2019. Mathqa: Towards interpretable math word problem solving with operation-based formalisms. arXiv preprint arXiv:1905.13319 (2019)

    arXiv 2019

  3. [3]

    H. P. Bahrick, L. E. Bahrick, A. S. Bahrick, and P. E. Bahrick. 1993. Maintenance of foreign language vocabulary and the spacing effect. Psychological Science 4, 5 (1993), 316–321

    work page 1993

  4. [4]

    Banerjee and A

    S. Banerjee and A. Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization . 65–72

    work page 2005

  5. [5]

    Ariel Blobstein, Daniel Izmaylov, Tal Yifat, Michal Levy, and Avi Segal. 2023. Angel: A New Generation Tool for Learning Material based Questions and Answers. In Proc. of the NeurIPS Workshop on Generative AI for Education (GAIED)

    work page 2023

  6. [6]

    Janez Brank, Gregor Leban, and Marko Grobelnik. 2017. Annotating Documents with Relevant Wikipedia Concepts. In Proc. of Slovenian KDD Conference on Data Mining and Data Warehouses (SiKDD) . Manuscript submitted to ACM A Novel Approach to Scalable and Automatic Topic-Controlled Question Generation in Education 15

    work page 2017

  7. [7]

    Sahan Bulathwela, Hamze Muse, and Emine Yilmaz. 2023. Scalable educational question generation with pre-trained language models. InInternational Conference on Artificial Intelligence in Education . Springer, 327–339

    work page 2023

  8. [8]

    Sahan Bulathwela, María Pérez-Ortiz, Catherine Holloway, Mutlu Cukurova, and John Shawe-Taylor. 2024. Artificial intelligence alone will not democratise education: On educational inequality, techno-solutionism and inclusive tools. Sustainability 16, 2 (2024), 781

    work page 2024

Show all 60 references
  1. [9]

    Sahan Bulathwela, María Pérez-Ortiz, Emine Yilmaz, and John Shawe-Taylor. 2021. Semantic TrueLearn: using semantic knowledge graphs in recommendation systems. arXiv preprint arXiv:2112.04368 (2021)

    2021 arXiv

  2. [10]

    Andrew Caines, Luca Benedetto, Shiva Taslimipoor, Christopher Davis, et al . 2023. On the application of large language models for language teaching and assessment technology. arXiv preprint arXiv:2307.08393 (2023)

    2023 arXiv

  3. [11]

    Feng Chen, Jiayuan Xie, Yi Cai, Tao Wang, and Qing Li. 2021. Difficulty-Controllable Visual Question Generation. In Proc. Web and Big Data: International Joint Conference. Springer-Verlag, 332–347

    2021

  4. [12]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 (2021)

    2021 arXiv

  5. [13]

    Teodor-Mihai Cotet, Stefan Ruseti, and Mihai Dascalu. 2020. Neural grammatical error correction for romanian. In 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI) . IEEE, 625–631

    2020

  6. [14]

    Jiaxi Cui, Zongjian Li, Yang Yan, Bohua Chen, and Li Yuan. 2023. Chatlaw: Open-source legal large language model with integrated external knowledge bases. arXiv preprint arXiv:2306.16092 (2023)

    2023 arXiv

  7. [15]

    Mutlu Cukurova, Xin Miao, and Richard Brooker. 2023. Adoption of artificial intelligence in schools: unveiling factors influencing teachers’ engagement. In International conference on artificial intelligence in education . Springer, 151–163

    2023

  8. [16]

    Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and Play Language Models: A Simple Approach to Controlled Text Generation. In International Conference on Learning Representations . https://openrevi...

    2020

  9. [17]

    Heffernan, Tanja Käser, Steven Moore, Anna N

    Paul Denny, Sumit Gulwani, Neil T. Heffernan, Tanja Käser, Steven Moore, Anna N. Rafferty, and Adish Singla. 2024. Generative AI for Education (GAIED): Advances, Opportunities, and Challenges. arXiv:2402.01580 [cs.CY] https://arxiv.org/abs/2402.01580

    2024 arXiv

  10. [18]

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale. arXiv preprint arXiv:2208.07339 (2022)

    2022 arXiv

  11. [19]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLora: Efficient Fine-Tuning of Quantized LLMs. arXiv preprint arXiv:2305.14314 (2023)

    2023 arXiv

  12. [20]

    Tung Do Viet and Konstantin Markov. 2023. Using Large Language Models for Bug Localization and Fixing. In 2023 12th International Conference on A wareness Science and Technology (iCAST). IEEE, 192–197

    2023

  13. [21]

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2022. A survey on in-context learning. arXiv preprint arXiv:2301.00234 (2022)

    2022 arXiv

  14. [22]

    Xinya Du, Junru Shao, and Claire Cardie. 2017. Learning to Ask: Neural Question Generation for Reading Comprehension. In Proc. Annual Meeting of the Association for Computational Linguistics . 1342–1352

    2017

  15. [23]

    Sabina Elkins, Ekaterina Kochmar, Jackie CK Cheung, and Iulian Serban. 2024. How Teachers Can Use Large Language Models and Bloom’s Taxonomy to Create Educational Quizzes. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 23084–23091

    2024

  16. [24]

    Said Al Faraby, Ade Romadhony, and Adiwijaya. 2024. Analysis of LLMs for educational question classification and generation. Computers and Education: Artificial Intelligence 7 (2024), 100298. https://doi.org/10.1016/j.caeai.2024.100298

    2024

  17. [25]

    Fares Fawzi, Sadie Amini, and Sahan Bulathwela. [n. d.]. Small Generative Language Models for Educational Question Generation. In Proc. of the NeurIPS Workshop on Generative AI for Education (GAIED)

  18. [26]

    Fawzi, S

    F. Fawzi, S. Balan, M. Cukurova, E. Yilmaz, and S. Bulathwela. 2024. Towards Human-Like Educational Question Generation with Small Language Models. In Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Trac...

    2024

  19. [27]

    Paolo Ferragina and Ugo Scaiella. 2010. TAGME: on-the-fly annotation of short text fragments (by wikipedia entities). In Proceedings of the 19th ACM International Conference on Information and Knowledge Management (Toronto, ON, Canada) (CIKM ’10). Association for Computing Mac...

    2010

  20. [28]

    Joseph L. Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological Bulletin 76, 5 (1971), 378–382

    1971

  21. [29]

    Department for Education. 2024. Use Cases for Generative AI in Education: User Research Report . Technical Report. Department for Education, UK Government. https://www.gov.uk/government/publications/generative-ai-in-education-user-research-and-technical-report Accessed: 2024-09-21

    2024

  22. [30]

    Michail Giannakos, Roger Azevedo, Peter Brusilovsky, Mutlu Cukurova, Yannis Dimitriadis, Davinia Hernandez-Leo, Sanna Järvelä, Manolis Mavrikis, and Bart Rienties. 2024. The promise and challenges of generative AI in education. Behaviour & Information Technology (2024), 1–27

    2024

  23. [31]

    Huanli Gong and Hengchang Pan, Liangming andHu. 2022. KHANQ: A Dataset for Generating Deep Questions in Education. In Proceedings of the 29th International Conference on Computational Linguistics

    2022

  24. [32]

    Lasse Hansen, Ludvig Renbo Olsen, and Kenneth Enevoldsen. 2023. TextDescriptives: A Python package for calculating a large variety of metrics from text. Journal of Open Source Software 8, 84 (April 2023), 5153. https://doi.org/10.21105/joss.05153

    2023 doi

  25. [33]

    Michael Heilman and Noah A. Smith. 2010. Good question! Statistical ranking for question generation. In Proceedings of the Human Language Technology Conference and the North American Chapter of the Association for Computational Linguistics (HLT-NAACL) . Manuscript submitted to...

    2010

  26. [34]

    Wenpeng Hu, Bing Liu, Rui Yan, Dongyan Zhao, and Jinwen Ma. 2018. Topic-Based Question Generation. In International Conference on Learning Representations. ICLR 2018 Conference Blind Submission. Invite to Workshop Track

    2018

  27. [35]

    Muhammad Khalifa, Hady Elsahar, and Marc Dymetman. 2021. A Distributional Approach to Controlled Text Generation. InInternational Conference on Learning Representations. https://openreview.net/forum?id=jWkw45-9AbL

    2021

  28. [36]

    Bor-Chen Kuo, Frederic TY Chang, and Zong-En Bai. 2023. Leveraging LLMs for Adaptive Testing and Learning in Taiwan Adaptive Learning Platform (TALP).. In Workshop on Empowering Education with LLMs at AIED . 101–110

    2023

  29. [37]

    Hai Li, Chenglu Li, Wanli Xing, Sami Baral, and Neil Heffernan. 2024. Automated Feedback for Student Math Responses Based on Multi-Modality and Fine-Tuning. In Proceedings of the 14th Learning Analytics and Knowledge Conference . 763–770

    2024

  30. [38]

    C. Y. Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Workshop on Text Summarization Branches Out

    2004

  31. [39]

    L. E. Lopez, D. K. Cruz, J. C. B. Cruz, and C. Cheng. 2021. Simplifying Paragraph-level Question Generation via Transformer Language Models. In Proceedings of the PRICAI 2021: Trends in Artificial Intelligence (8–12 November 2021). Hanoi, Vietnam

    2021

  32. [40]

    Rosemary Luckin and Mutlu Cukurova. 2019. Designing educational technologies in the age of AI: A learning sciences-driven approach. British Journal of Educational Technology 50, 6 (2019), 2824–2838

    2019

  33. [41]

    Kamil Malinka, Martin Peresíni, Anton Firc, Ondrej Hujnák, and Filip Janus. 2023. On the educational impact of chatgpt: Is artificial intelligence ready to obtain a university degree?. InProceedings of the 2023 Conference on Innovation and Technology in Computer Science Educat...

    2023

  34. [42]

    Louis Martin, Éric Villemonte de La Clergerie, Benoît Sagot, and Antoine Bordes. 2020. Controllable Sentence Simplification. In LREC 2020 - 12th Language Resources and Evaluation Conference . Marseille, France. https://inria.hal.science/hal-02678214

    2020

  35. [43]

    Papineni, S

    K. Papineni, S. Roukos, T. Ward, and W.J. Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. InProceedings of the 40th annual meeting on association for computational linguistics . Association for Computational Linguistics, 311–318

    2002

  36. [44]

    Zachary A Pardos and Shreya Bhandari. 2023. Learning gain differences between ChatGPT and human tutor generated algebra hints. arXiv preprint arXiv:2302.06871 (2023)

    2023 arXiv

  37. [45]

    Francesco Piccinno and Paolo Ferragina. 2014. From TagME to WAT: a new entity annotator. InProceedings of the First International Workshop on Entity Recognition & Disambiguation (ERD ’14) . Association for Computing Machinery, 55–62. https://doi.org/10.1145/2633211.2634350

    2014

  38. [46]

    Gustavo Pinto, Isadora Cardoso-Pereira, Danilo Monteiro, Danilo Lucena, Alberto Souza, and Kiev Gama. 2023. Large language models for education: Grading open-ended questions using chatgpt. In Proceedings of the XXXVII Brazilian Symposium on Software Engineering . 293–302

    2023

  39. [47]

    Marco Ponza, Paolo Ferragina, and Soumen Chakrabarti. 2020. On Computing Entity Relatedness in Wikipedia, with Applications. Knowledge-Based Systems 188 (2020)

    2020

  40. [48]

    Yuxiang Qiu, Karim Djemili, Denis Elezi, Aaneel Shalman Srazali, María Pérez-Ortiz, Emine Yilmaz, John Shawe-Taylor, and Sahan Bulathwela. 2024. A Toolbox for Modelling Engagement with Educational Videos. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. ...

    2024

  41. [49]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2022. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.The Journal of Machine Learning Research 21, 1 (2022), 5485–5551

    2022

  42. [50]

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ Questions for Machine Comprehension of Text. arXiv:1606.05250 [cs.CL] https://arxiv.org/abs/1606.05250

    2016 arXiv

  43. [51]

    Luca Soldaini and Kyle Lo. 2023. peS2o (Pretraining Efficiently on S2ORC) Dataset . Technical Report. Allen Institute for AI. ODC-By, https: //github.com/allenai/pes2o

    2023

  44. [52]

    2024.Global Report on Teachers: Addressing Teacher Shortages and Transforming the Profession

    UNESCO and International Task Force on Teachers for Education 2030. 2024.Global Report on Teachers: Addressing Teacher Shortages and Transforming the Profession. UNESCO, Paris. 187 pages. https://doi.org/10.54675/FIGU8035 CC BY-SA 3.0 IGO

    2024 doi

  45. [53]

    Kristiyan Vachev, Momchil Hardalov, Georgi Karadzhov, Georgi Georgiev, Ivan Koychev, and Preslav Nakov. 2022. Leaf: Multiple-choice question generation. In Proc. of the European Conf. on Information Retrieval

    2022

  46. [54]

    Yu, and Qingsong Wen

    Shen Wang, Tianlong Xu, Hang Li, Chaoli Zhang, Joleen Liang, Jiliang Tang, Philip S. Yu, and Qingsong Wen. 2024. Large Language Models for Education: A Survey and Outlook. arXiv:2403.18105 [cs.CL] https://arxiv.org/abs/2403.18105

    2024 arXiv

  47. [55]

    Z. Wang, A. S. Lan, W. Nie, A. E. Waters, P. J. Grimaldi, and R. G. Baraniuk. 2018. QG-Net: A Data-Driven Question Generation Model for Educational Content. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale (26–28 June 2018). London, UK

    2018

  48. [56]

    Gautam Yadav, Ying-Jui Tseng, and Xiaolin Ni. 2023. Contextualizing problems to student interests at scale in intelligent tutoring system using large language models. arXiv preprint arXiv:2306.00190 (2023)

    2023 arXiv

  49. [57]

    Yudelson, Kenneth R

    Michael V. Yudelson, Kenneth R. Koedinger, and Geoffrey J. Gordon. 2013. Individualized Bayesian Knowledge Tracing Models. In Proc. of Artificial Intelligence in Education, H. Chad Lane, Kalina Yacef, Jack Mostow, and Philip Pavlik (Eds.)

    2013

  50. [58]

    Zhang and A

    L. Zhang and A. Rettinger. 2014. Final Ontological Word-Sense Disambiguation Prototype. Deliverable D3.2.3. xLike Project

    2014

  51. [59]

    Ruqing Zhang, Jiafeng Guo, Lu Chen, Yixing Fan, and Xueqi Cheng. 2021. A Review on Question Generation from Natural Language Text. ACM Trans. Inf. Syst. 40, 1, Article 14 (sep 2021), 43 pages. https://doi.org/10.1145/3468889

    2021 doi

  52. [60]

    Weinberger, and Yoav Artzi

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating Text Generation with BERT. arXiv:1904.09675 [cs.CL] https://arxiv.org/abs/1904.09675 Received Not Available; revised Not Available; accepted Not Available Manuscript submi...

    2020 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.