Pith. sign in

REVIEW 4 major objections 5 minor 42 references

How Good is ChatGPT in Giving Adaptive Guidance Using Knowledge Graphs in E-Learning Environments?

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Knowledge graph context lets ChatGPT-4 generate adaptive, expert-approved feedback.

desk verdict Honest, clearly written preliminary study of KG-augmented prompt engineering for ChatGPT feedback, but the central efficacy claim is not supported because the knowledge graph's contribution is never isolated. read the letter →

arxiv 2412.03856 v1 pith:7JK4TZN3 submitted 2024-12-05 cs.AI cs.ET

Patrick Ocheja , Brendan Flanagan , Yiling Dai , Hiroaki Ogata This is my paper · ORCID
classification cs.AIcs.ET
keywords knowledgegraphlargelanguagemodelChatGPT-4adaptivefeedbackpersonalizedlearningstudentimpasseintelligenttutoringsystemROUGEevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that an e-learning assistant can generate genuinely adaptive guidance by feeding a large language model not just the student's question but also their knowledge state, as traced through a knowledge graph of prerequisite concepts. Concretely, it claims that ChatGPT-4, when given a question, its standard solution, an expert-identified impasse, and ranked prerequisites, produces feedback that expert evaluators rate as correct and mostly precise across easy, moderate, and hard questions. The paper also finds that the model alone cannot estimate what a student is stuck on, and that some generated feedback contains errors, so human oversight remains necessary. A sympathetic reader would care because this is a concrete test of whether knowledge-graph context can move LLM tutoring from generic answers toward personalization, and it identifies where the pipeline still depends on humans.

What carries the argument

The load-bearing mechanism is the prompt template P1, which injects four elements into every ChatGPT-4 query: the question, the correct and standard solution, the student's impasse, and ranked prerequisite concepts retrieved from a knowledge graph built from the textbook's 'GO for Help' indicators. That knowledge graph also defines the tiers of support: students with poor mastery of prerequisites receive foundational reviews, average students receive in-depth prerequisite explanations, and strong students receive advanced assistance. The ROUGE metric and expert ratings are the measurement machinery used to decide whether the feedback is personalized, with high ROUGE overlap between student types read as insufficient personalization for harder questions.

What would settle it

A controlled study in which actual students who failed problems from the easy, moderate, and hard levels receive the generated feedback and are asked whether it addresses their own stuck point; if many students report the feedback misses their confusion or learning outcomes do not improve, the claim that this pipeline gives adaptive guidance would be weakened.

Watch

Extended reading notes

Core claim

The central claim is that augmenting a prompt with the student's impasse and ranked prerequisite concepts from a knowledge graph makes ChatGPT-4's guidance adaptive: for easy questions all three simulated student types receive similar, correct feedback, while as difficulty increases the feedback diverges to match each type's sticking point. The paper reports expert ratings of correctness at or near the top of the scale across all three question types, with precision ratings ranging from 'precise' to 'very precise' in most cases and only occasional imprecision, plus rare hallucinations. The authors explicitly state that when ChatGPT-4 was asked to generate its own impasses, the results were too generic or similar across student profiles, so experts supplied the impasses; and when the standard solution was omitted from the prompt, the generated feedback was wrong. The discovery, on the paper's own terms, is that LLMs can deliver personalized feedback well when the hard part of diagnosis is done for them by knowledge-graph-based tracing and expert input, and that they are not yet reliable enough to run unsupervised.

Load-bearing premise

The expert-written impasses for the three simulated student types accurately represent the ways real students get stuck, so feedback judged correct and precise for those impasses would also help real students.

Editorial extensions

If this is right

  • An e-learning system can use a static knowledge graph and a prompt template to tier LLM feedback without retraining the model.
  • Feedback personalization increases with question difficulty, so harder problems benefit most from knowledge-graph context.
  • Deploying such a system in classrooms requires a separate, reliable impasse-detection component; asking the LLM to diagnose impasses itself does not work.
  • Human validation of LLM feedback remains mandatory because the model occasionally produces incorrect or imprecise guidance.
  • The same prompt pattern can now be evaluated on more subjects and languages to test how far the approach generalizes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to replace expert-written impasses with impasses detected from real student solution traces, then compare learning gains between a ChatGPT-guided group and a control group on a post-test.
  • Because the knowledge graph supplies prerequisite rankings, the same pipeline likely transfers to any subject whose curriculum can be mapped to prerequisite relations, not only Algebra 2, though impasse estimation would need domain-specific data.
  • The ROUGE-based personalization measure could be sharpened by testing whether lower overlap between student types actually predicts better learning outcomes in a real classroom, which would separate wording variation from true adaptation.
  • If the system were scaled, the human oversight that the authors find necessary could be structured as a teacher dashboard that flags low-confidence or hallucination-prone responses rather than reviewing every output.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes AI-sensei, a system that integrates a knowledge graph of prerequisite relations with ChatGPT-4 to generate personalized feedback for mathematics questions. The authors construct a knowledge graph from an Algebra 2 textbook, select one easy, one moderate, and one hard question, define three simulated student types (S1, S2, S3) with expert-estimated impasses, and use prompt P1 (question, correct solution, impasse, ranked prerequisites) to generate feedback. They evaluate the outputs using ROUGE scores and ratings from three experts on correctness, precision, hallucination, and variability, and report a six-participant pilot user study. The abstract claims that the tiered support could enhance comprehension and improve task outcomes.

Significance. The integration of knowledge graphs with LLMs for adaptive feedback is timely and potentially useful, and the paper is transparent in publishing the exact prompt, full ROUGE tables, and expert rating statistics. It also honestly reports failure cases when the correct solution is not provided (Section VI). However, the evidence presented does not support the paper's central claim: the KG-derived prerequisite list is never isolated from the expert-authored scaffolding, the impasses are simulated rather than observed, and the pilot study measures self-reported perceptions rather than learning outcomes. These findings are suggestive for a feasibility study, but the claims of enhanced comprehension and improved task outcomes are premature.

major comments (4)
  1. [Section III-D, prompt P1] The prompt contains the question, the correct and standard solution, the impasse, and the ranked prerequisites; only the ranked prerequisites are KG-derived. Section VI reports that when the correct solution was omitted, ChatGPT produced wrong answers, and when ChatGPT was asked to estimate impasses it performed poorly. The high correctness and precision scores in Table III therefore reflect mostly the correctness of the provided scaffolding rather than the contribution of the knowledge graph. A baseline that removes or randomizes the ranked-prerequisite field while keeping the question, solution, and impasse fixed is necessary to support the claim that the KG-based knowledge state is the operative component.
  2. [Section III-C and Section VI] The three student impasses are expert estimates written from the standard solution, not from actual student work. The limitation paragraph in Section VI acknowledges "the absence of real students in our experiment led to not capturing the true diversity and unpredictability of student impasse(s)." Because the prompt's personalization depends entirely on the impasse string, the expert ratings of precision show only how well ChatGPT follows a synthetic scenario. The paper should either validate the simulated impasses against real students' stuck points or conduct the evaluation with real students who actually experience those impasses.
  3. [Section V and Table III] The ROUGE results are interpreted inconsistently. For easy questions, high overlap with the standard solution is taken as consistency, while for hard questions low overlap is taken as personalization, but no criterion or threshold is provided for what level of ROUGE indicates appropriate adaptivity, and there is no evidence that n-gram overlap measures pedagogical quality. In addition, Table III shows substantial rater disagreement (e.g., SD = 1.73 for S2 precision on the easy question) and the Cohen's Kappa for the hard question is only 0.30 (fair), so the measurements are too noisy to support the conclusion that feedback is meaningfully differentiated across student types.
  4. [Section IV-B and Section V-E] The pilot user study has only six participants, no control group, and no learning-outcome measure; the abstract's claim about enhanced comprehension and improved task outcomes is not evidenced by the reported data. The post-test improvement in perception (µ from 3.0 to 3.67) is a self-report difference without statistical testing and says nothing about actual comprehension. The conclusions should either be limited to perceived usefulness or the study should add pre/post learning assessments.
minor comments (5)
  1. [Section V-C] The text "For hard (B) type questions" should read "hard (C) type questions" to match the A/B/C difficulty labels used elsewhere in the paper.
  2. [Section V-A1] The text states that "the feedback for S1 edges out slightly with the highest F1-score for ROUGE-1," but Table I shows S vs S2 has F1 = 0.52 while S vs S1 has F1 = 0.51; this description is inconsistent with the table.
  3. [Section V-E] The text says "Figure 5 shows the results from the pre-test survey," but the pre-test figure is captioned as Figure 4 and the post-test figure is Figure 5; the cross-references are swapped.
  4. [Section IV-A] The phrase "We evaluation precision of generated feedback" should be "We evaluated the precision of the generated feedback"; similar grammatical errors appear elsewhere, e.g., "show in Figure 1" in Section III-A should be "shown in Figure 1."
  5. [Throughout] The paper inconsistently writes "ChatGPT4" and "ChatGPT-4"; please use one convention consistently.

Circularity Check

2 steps flagged · score 8.0 of 10

Correctness and adaptivity ratings are largely forced by the prompt's inclusion of the standard solution and expert-written impasse; the KG/LLM integration is not isolated.

  1. fitted input called prediction [Section III-D (prompt P1); Section VI (Discussions and Limitations)]
    "P1: Solve this question: {question}. The correct and standard solution is {answer}. Your solution should include detailed explanation to help this impasse:: {impasse}. This impasse exists because: {ranked_prerequisites}?"

    The paper's headline correctness result is evaluated on outputs generated by a prompt that already contains the exact standard solution. Since P1 instructs the model to solve the question with {answer} provided, the expert correctness ratings (mostly 5.00 in Table III) are largely a test of restating or explaining a supplied solution, not of independently generating correct adaptive guidance. Section VI confirms the dependency: 'when we did not provide ChatGPT with the correct standard solution, the answers or feedback generated were wrong.' Thus the 'correctness' prediction is forced by the input by construction.

  2. self definitional [Section III-C and Section III-D; Section IV (ROUGE evaluation)]
    "The difference between our proposed method and direct prompt to ChatGPT is that we factor in the student’s current knowledge state as traced from the knowledge graph and estimated by experts. Given this knowledge state, it is expected that the LLM generates a tailored feedback or answer to address the student’s specific challenges."

    The adaptive signal is inserted into the prompt by the experimenters: Section III-C says 'due to lack of real students in this preliminary study, we asked experts to review the standard solution to each question and estimate the likely impasse for different types of students.' P1 then feeds that impasse and the ranked prerequisites directly to the LLM. The ROUGE comparisons across S1/S2/S3 therefore mostly quantify differences between the expert-written impasse strings, and the expert 'precision' ratings test whether the LLM followed the prompt's own impasse. Section VI concedes ChatGPT could not produce the impasses itself ('the impasses generated were either too generic or similar for different student profiles').

full rationale

The paper's central evaluation is not protected by external benchmarks or ablations. Prompt P1 (Section III-D) already contains the correct standard solution, the expert-written impasse, and the ranked prerequisites, so the high expert correctness/precision scores (Table III) and the ROUGE-based variability results (Tables II, V, VII) are largely forced by the prompt's own contents. Section VI confirms this reading: without the supplied solution ChatGPT generated wrong answers, and when asked to estimate impasses itself it produced generic, profile-similar text. The knowledge graph contributes only the ranked-prerequisite list, and no condition removes that list, so the claimed KG/LLM integration is not isolated from the expert-authored scaffolding. The self-citations present are not load-bearing for the circularity; the issue is prompt construction, not citation chains. The pilot user study (n=6, no control) provides only weak external evidence and does not rescue the main derivation. Score 8 reflects that the main 'correct and adaptive feedback' result is largely defined by the input prompt rather than by an independent prediction.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No fitted numerical parameters appear in the study; the only hand-chosen settings are design choices such as ChatGPT temperature 0.2 and the mapping of difficulty to node position. The assumptions listed above are the unverified premises that the evaluation rests on, especially the simulation of student impasses by experts and the validity of ROUGE as a personalization metric. The AI-sensei system is a software artifact, not a newly invented theoretical entity.

assumptions (4)
  • domain assumption The knowledge graph built from the textbook's 'GO for Help' indicators correctly represents prerequisite relationships between concepts.
    All descendant-node retrieval and prerequisite ranking in Sections III-A and III-B depend on this structure being accurate.
  • domain assumption Expert-estimated impasses for the three student profiles represent the difficulties real students would encounter.
    Section III-C states that impasses were estimated by experts due to lack of real students; the entire evaluation of personalization rests on this proxy.
  • domain assumption ROUGE n-gram overlap between feedback texts is a valid measure of personalization, with high overlap interpreted as low personalization.
    Section IV-A explicitly assumes this, and the paper itself notes ROUGE's limitations.
  • ad hoc to paper Question difficulty can be mapped from node position in the knowledge graph, with leaf or basic concepts counted as easy and advanced concepts as hard.
    Section III-B states 'This assumption has been made for simplicity of analysis.'

how reviews work

0 comments
Cite this review

Pith. "Pith review of How Good is ChatGPT in Giving Adaptive Guidance Using Knowledge Graphs in E-Learning Environments?." pith.science (2026). https://pith.science/paper/7JK4TZN3

@misc{pith2026241203856,
  author       = {Pith},
  title        = {Pith review of: How Good is ChatGPT in Giving Adaptive Guidance Using Knowledge Graphs in E-Learning Environments?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7JK4TZN3}},
  note         = {Machine review of arXiv:2412.03856}
}
read the original abstract

E-learning environments are increasingly harnessing large language models (LLMs) like GPT-3.5 and GPT-4 for tailored educational support. This study introduces an approach that integrates dynamic knowledge graphs with LLMs to offer nuanced student assistance. By evaluating past and ongoing student interactions, the system identifies and appends the most salient learning context to prompts directed at the LLM. Central to this method is the knowledge graph's role in assessing a student's comprehension of topic prerequisites. Depending on the categorized understanding (good, average, or poor), the LLM adjusts its guidance, offering advanced assistance, foundational reviews, or in-depth prerequisite explanations, respectively. Preliminary findings suggest students could benefit from this tiered support, achieving enhanced comprehension and improved task outcomes. However, several issues related to potential errors arising from LLMs were identified, which can potentially mislead students. This highlights the need for human intervention to mitigate these risks. This research aims to advance AI-driven personalized learning while acknowledging the limitations and potential pitfalls, thus guiding future research in technology and data-driven education.

Figures

Figures reproduced from arXiv: 2412.03856 by the authors.

Figure 1
Figure 1. System Architecture. Large Language Models (LLMs) such as GPT-3 and GPT￾4 represent a transformative shift in the capability of AI to provide personalized educational support [26]. Unlike ear￾lier systems, LLMs can generate nuanced and contextually relevant feedback by understanding the broader context of a student’s work. They are capable of recognizing diverse and non-standard solutions due to their training on ex… view at source ↗
Figure 2
Figure 2. Knowledge graph constructed from the textbook Math Algebra 2 by Prentice Hall [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Experiment setup. generated by ChatGPT4 for student S1 vs S2, S1 vs S3 and S2 vs S3? Similar to the measure of precision, we use the variability metric to validate the precision of the feedback and how it applies to each student. Variability is graded on the following scale: 1 - Very Low, 2 - Low, 3 - Moderate, 4 - High, 5 - Very High. To check the inter-rater reliability, we calculate Cohen’s Kappa coefficient [38]… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Participants’ pre-test perspective on use of AI [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Participants’ post-test perspective on use of AI sensei [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 33 canonical work pages

  1. [1]

    Educational data mining and learning analytics,

    R. S. Baker, T. Martin, and L. M. Rossi, “Educational data mining and learning analytics,” The Wiley handbook of cognition and assessment: Frameworks, methodologies, and applications , pp. 379–396, 2016

    work page 2016

  2. [2]

    Multimodal learning analytics and educa- tion data mining: Using computational technologies to measure complex learning tasks,

    P. Blikstein and M. Worsley, “Multimodal learning analytics and educa- tion data mining: Using computational technologies to measure complex learning tasks,” Journal of Learning Analytics , vol. 3, no. 2, pp. 220– 238, 2016

    work page 2016

  3. [3]

    Visualization of education blockchain data: trends and challenges,

    P. Ocheja, B. Flanagan, H. Ogata, and S. S. Oyelere, “Visualization of education blockchain data: trends and challenges,” Interactive Learning Environments, vol. 31, no. 9, pp. 5970–5994, 2023

    work page 2023

  4. [4]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019

    2019

  5. [5]

    Handbook of automated essay evalua- tion: Current applications and new directions,

    M. D. Shermis and J. Burstein, “Handbook of automated essay evalua- tion: Current applications and new directions,” 2013

    work page 2013

  6. [6]

    A conceptual map model for developing intelligent tutoring systems,

    G.-J. Hwang, “A conceptual map model for developing intelligent tutoring systems,” Computers & Education, vol. 40, no. 3, pp. 217–235, 2003. 10

    work page 2003

  7. [7]

    Adaptive intelligent tutoring systems for e-learning systems,

    P. Phobun and J. Vicheanpanya, “Adaptive intelligent tutoring systems for e-learning systems,” Procedia-Social and Behavioral Sciences, vol. 2, no. 2, pp. 4064–4069, 2010

    work page 2010

  8. [8]

    Delivering hints in a dialogue-based intelligent tutoring system

    Y . Zhou, R. Freedman, M. Glass, J. A. Michael, A. A. Rovick, and M. W. Evens, “Delivering hints in a dialogue-based intelligent tutoring system.” in AAAI/IAAI, 1999, pp. 128–134

    work page 1999

Show all 42 references
  1. [9]

    Knowledge map creation for modeling learning behaviors in digital learning environments,

    B. Flanagan, R. Majumdar, G. Akçapınar, J. Wang, and H. Ogata, “Knowledge map creation for modeling learning behaviors in digital learning environments,” in Companion Proceedings of the 9th Inter- national Conference on Learning Analytics and Knowledge (LAK’19) . Society for L...

    2019

  2. [10]

    A prototype framework for a distributed lifelong learner model,

    P. Ocheja, B. Flanagan, S. S. Oyelere, L. Lecailliez, and H. Ogata, “A prototype framework for a distributed lifelong learner model,” in 28th International Conference on Computers in Education Conference Proceedings, vol. 1. Asia-Pacific Society for Computers in Education (APS...

    2020

  3. [11]

    Rouge: A package for automatic evaluation of summaries,

    C.-Y . Lin, “Rouge: A package for automatic evaluation of summaries,” in Text summarization branches out , 2004, pp. 74–81

    2004

  4. [12]

    Ethics of large language models in medicine and medical research,

    H. Li, J. T. Moon, S. Purkayastha, L. A. Celi, H. Trivedi, and J. W. Gichoya, “Ethics of large language models in medicine and medical research,” The Lancet Digital Health, vol. 5, no. 6, pp. e333–e335, 2023

    2023

  5. [13]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017

    2017

  6. [14]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  7. [15]

    Chatbots put to the test in math and logic problems: A comparison and assessment of chatgpt-3.5, chatgpt-4, and google bard,

    V . Plevris, G. Papazafeiropoulos, and A. Jiménez Rios, “Chatbots put to the test in math and logic problems: A comparison and assessment of chatgpt-3.5, chatgpt-4, and google bard,” AI, vol. 4, no. 4, pp. 949–969, 2023

    2023

  8. [16]

    “so what if chatgpt wrote it?

    Y . K. Dwivedi, N. Kshetri, L. Hughes, E. L. Slade, A. Jeyaraj, A. K. Kar, A. M. Baabdullah, A. Koohang, V . Raghavan, M. Ahujaet al., ““so what if chatgpt wrote it?” multidisciplinary perspectives on opportunities, challenges and implications of generative conversational ai f...

    2023

  9. [17]

    Ai bot chatgpt writes smart essays-should academics worry?

    C. Stokel-Walker, “Ai bot chatgpt writes smart essays-should academics worry?” Nature, 2022

    2022

  10. [18]

    Penetrating the fog: Analytics in learning and education

    G. Siemens and P. Long, “Penetrating the fog: Analytics in learning and education.” EDUCAUSE review, vol. 46, no. 5, p. 30, 2011

    2011

  11. [19]

    State of the art and practice in ai in education,

    W. Holmes and I. Tuomi, “State of the art and practice in ai in education,” European Journal of Education , vol. 57, no. 4, pp. 542– 570, 2022

    2022

  12. [20]

    Artificial intelligence in education: a panoramic review,

    K. Ahmad, J. Qadir, A. Al-Fuqaha, W. Iqbal, A. El-Hassan, D. Benhad- dou, and M. Ayyash, “Artificial intelligence in education: a panoramic review,” DOI: https://doi. org/10.35542/osf. io/zvu2n , 2020

    2020 doi

  13. [21]

    An application of chinese dialogue-based intelligent tutoring system in remedial instruc- tion for mathematics learning,

    K.-C. Pai, B.-C. Kuo, C.-H. Liao, and Y .-M. Liu, “An application of chinese dialogue-based intelligent tutoring system in remedial instruc- tion for mathematics learning,” Educational Psychology, vol. 41, no. 2, pp. 137–152, 2021

    2021

  14. [22]

    Intelligent tutoring systems: an overview,

    H. S. Nwana, “Intelligent tutoring systems: an overview,” Artificial Intelligence Review, vol. 4, no. 4, pp. 251–277, 1990

    1990

  15. [23]

    Designing a multi-strategic intelligent tutoring system for training in industry,

    C. Frasson and E. Aïmeur, “Designing a multi-strategic intelligent tutoring system for training in industry,” Computers in industry, vol. 37, no. 2, pp. 153–167, 1998

    1998

  16. [24]

    Automated feedback generation for introductory programming assignments,

    R. Singh, S. Gulwani, and A. Solar-Lezama, “Automated feedback generation for introductory programming assignments,” in Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation, 2013, pp. 15–26

    2013

  17. [25]

    Stupid tutoring systems, intelligent humans,

    R. S. Baker, “Stupid tutoring systems, intelligent humans,” International Journal of Artificial Intelligence in Education , vol. 26, pp. 600–614, 2016

    2016

  18. [26]

    Practical and ethical challenges of large language models in education: A systematic scoping review,

    L. Yan, L. Sha, L. Zhao, Y . Li, R. Martinez-Maldonado, G. Chen, X. Li, Y . Jin, and D. Gaševi´c, “Practical and ethical challenges of large language models in education: A systematic scoping review,” British Journal of Educational Technology , vol. 55, no. 1, pp. 90–112, 2024

    2024

  19. [27]

    Generating high-precision feedback for program- ming syntax errors using large language models,

    T. Phung, J. Cambronero, S. Gulwani, T. Kohn, R. Majumdar, A. Singla, and G. Soares, “Generating high-precision feedback for program- ming syntax errors using large language models,” arXiv preprint arXiv:2302.04662, 2023

    2023 arXiv

  20. [28]

    Engineering education in the era of chatgpt: Promise and pitfalls of generative ai for education,

    J. Qadir, “Engineering education in the era of chatgpt: Promise and pitfalls of generative ai for education,” in2023 IEEE Global Engineering Education Conference (EDUCON) . IEEE, 2023, pp. 1–9

    2023

  21. [29]

    Knowledge graph quality management: a compre- hensive survey,

    B. Xue and L. Zou, “Knowledge graph quality management: a compre- hensive survey,” IEEE Transactions on Knowledge and Data Engineer- ing, vol. 35, no. 5, pp. 4969–4988, 2022

    2022

  22. [30]

    Defining a knowledge graph devel- opment process through a systematic review,

    G. Tamašauskait ˙e and P. Groth, “Defining a knowledge graph devel- opment process through a systematic review,” ACM Transactions on Software Engineering and Methodology , vol. 32, no. 1, pp. 1–40, 2023

    2023

  23. [31]

    Knowledge graph identification,

    J. Pujara, H. Miao, L. Getoor, and W. Cohen, “Knowledge graph identification,” in The Semantic Web–ISWC 2013: 12th International Semantic Web Conference, Sydney, NSW, Australia, October 21-25, 2013, Proceedings, Part I 12 . Springer, 2013, pp. 542–557

    2013

  24. [32]

    Third-year high school mathematics curriculum: Effects of content organization and curriculum implementation,

    Ó. Chávez, J. E. Tarr, D. A. Grouws, and V . M. Soria, “Third-year high school mathematics curriculum: Effects of content organization and curriculum implementation,” International Journal of Science and Mathematics Education, vol. 13, pp. 97–120, 2015

    2015

  25. [33]

    Identifying students’ stuck points using self-explanations and pen stroke data in a mathematics quiz,

    R. Nakamoto, B. Flanagan, K. Takam, Y . Dai, and H. Ogata, “Identifying students’ stuck points using self-explanations and pen stroke data in a mathematics quiz,” in ICCE, vol. 2021, 2021, pp. 22–26

    2021

  26. [34]

    Robotgpt: Robot manipulation learning from chatgpt,

    Y . Jin, D. Li, A. Yong, J. Shi, P. Hao, F. Sun, J. Zhang, and B. Fang, “Robotgpt: Robot manipulation learning from chatgpt,” IEEE Robotics and Automation Letters , 2024

    2024

  27. [35]

    Autogen: Enabling next-gen llm applications via multi-agent conversation framework,

    Q. Wu, G. Bansal, J. Zhang, Y . Wu, S. Zhang, E. Zhu, B. Li, L. Jiang, X. Zhang, and C. Wang, “Autogen: Enabling next-gen llm applications via multi-agent conversation framework,” arXiv e-prints , pp. arXiv– 2308, 2023

    2023

  28. [36]

    Analytics 2.0 for precision education,

    J.-Y . Wu, C. C. Yang, C.-H. Liao, and M.-W. Nian, “Analytics 2.0 for precision education,” Educational Technology & Society, vol. 24, no. 1, pp. 267–279, 2021

    2021

  29. [37]

    A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity,

    Y . Bang, S. Cahyawijaya, N. Lee, W. Dai, D. Su, B. Wilie, H. Lovenia, Z. Ji, T. Yu, W. Chung et al. , “A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity,” arXiv preprint arXiv:2302.04023 , 2023

    2023 arXiv

  30. [38]

    Cohen’s kappa coefficient as a performance measure for feature selection,

    S. M. Vieira, U. Kaymak, and J. M. Sousa, “Cohen’s kappa coefficient as a performance measure for feature selection,” inInternational conference on fuzzy systems . IEEE, 2010, pp. 1–8

    2010

  31. [39]

    Stakeholders’ insights on learning analytics: Perspectives of students and staff,

    G. Gray, A. E. Schalk, G. Cooke, P. Murnion, P. Rooney, and K. O’Rourke, “Stakeholders’ insights on learning analytics: Perspectives of students and staff,” Computers & Education , vol. 187, p. 104550, 2022

    2022

  32. [40]

    Large language models in education: A focus on the complementary relationship between human teachers and chatgpt,

    J. Jeon and S. Lee, “Large language models in education: A focus on the complementary relationship between human teachers and chatgpt,” Education and Information Technologies , vol. 28, no. 12, pp. 15 873– 15 892, 2023

    2023

  33. [41]

    Chatgpt for good? on opportunities and challenges of large language models for education,

    E. Kasneci, K. Seßler, S. Küchemann, M. Bannert, D. Dementieva, F. Fischer, U. Gasser, G. Groh, S. Günnemann, E. Hüllermeier et al. , “Chatgpt for good? on opportunities and challenges of large language models for education,” Learning and individual differences , vol. 103, p. ...

    2023

  34. [2019]

    Patrick received his Ph.D

    Dr. Patrick received his Ph.D. in Informatics from Kyoto University in 2022 and he is currently a Japan Society for the Promotion of Science (JSPS) Postdoctoral Fellow at Kyoto University, Kyoto, Japan. Outside academia, from 2014 to 2016, he worked as a software engineer at G...

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.