Pith. sign in

REVIEW 5 major objections 6 minor 39 references

Assessing Personalized AI Mentoring with Large Language Models in the Computing Field

T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GPT-4 gives more personalized AI mentoring than LLaMA 3 or PaLM 2

desk verdict The paper's own semantic-similarity metric says LLaMA 3 is more personalized, so the headline claim for GPT-4o rests on an unvalidated word-cloud override. read the letter →

arxiv 2412.08430 v1 pith:CXD3PQGD submitted 2024-12-11 cs.CL cs.AI

Xiao Luo , Sean O'Connell , Shamima Mithun This is my paper · ORCID
classification cs.CLcs.AI
keywords largelanguagemodelsAImentoringpersonalizedcomputingeducationcareerplanningsemanticsimilarityhumanevaluationzero-shotprompting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether large language models can act as personalized career mentors for computing students, tailoring advice to who a student is—their race, gender, major, and year in college. It evaluates GPT-4, LLaMA 3, and PaLM 2 by giving each model 15 career questions from three fictional student profiles, then measures how much the answers differ across profiles. Both an automatic word-and-similarity analysis and human expert ratings point the same way: GPT-4's answers are more personalized than the other two models' answers. The authors conclude that LLM-based mentoring tools are feasible but should keep human mentors involved, and that fine-tuning on educational data could improve personalization further.

What carries the argument

The load-bearing setup is a paired comparison: three student profiles (African-American male junior computer-science major; white male freshman undecided major; Hispanic female freshman computer-science major) are combined with 15 career-planning questions, and each LLM answers every question in zero-shot mode through a system prompt that instructs it to act as an AI mentor and a user message carrying the student's background. Personalization is measured in two ways: the NLP pipeline converts sentences to embeddings and flags sentence pairs whose cosine similarity $\cos(a,b) = \frac{a \cdot b}{\|a\| \, \|b\|}$ exceeds 0.7 as shared content, so a lower overlap percentage indicates more profile-specific answers; word clouds then show which words each model uses repeatedly for each profile. The human survey, built from earlier academic-advising and mentor-effectiveness instruments, provides the second, independent read on whether the profile-specific differences are actually better mentoring.

What would settle it

A controlled version of the same experiment—keeping the 15 questions and prompt format, but changing only one attribute at a time between otherwise identical profiles—would settle the claim: if GPT-4's overlap scores and word signatures barely move when race alone changes, then the personalization attributed to social background is actually driven by the other co-varying attributes, or by prompt noise.

Watch

Extended reading notes

Core claim

The paper's central claim is that GPT-4 outperforms LLaMA 3 and PaLM 2 at personalized career mentoring for computing students: its responses vary more distinctly with a student's social and educational background, and human evaluators rate them as more accurate and useful. The quantitative evidence is that GPT-4's answers to the same question for different profiles share fewer semantically similar sentences (measured by cosine similarity above a 0.7 threshold in sentence-embedding space) and display profile-specific vocabulary such as 'Hispanic,' 'woman,' or 'professional network' in word clouds. The qualitative evidence is a 13-question Likert survey on which GPT-4 received the highest scores for most items, while PaLM 2 scored lowest. The paper also reports that all three models give weak help on setting concrete goals, and it attributes that to the absence of goal-oriented questions in the test set rather than to a general model limitation.

Load-bearing premise

The load-bearing assumption is that comparing three profiles that differ at once in gender, race, college year, and major allows the study to attribute differences in the LLMs' answers to those background factors; because the profiles vary on several dimensions together, the contribution of any single dimension is not isolated.

Editorial extensions

If this is right

  • If GPT-4's advantage is stable across new questions and profiles, it is the natural default backend for AI career-mentoring assistants in computing education.
  • Because overlap measurement is automatic, the same pipeline can serve as a cheap screening test for personalization before a human evaluation is run.
  • Since all three models scored low on goal-setting, a practical mentoring tool would need a dedicated goal-setting module or additional prompt structure, not just a stronger LLM.
  • The authors' conclusion that human involvement is essential means near-term deployments should be human-in-the-loop, with the LLM drafting advice and a mentor checking it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test which background dimension drives the personalization: because profile changes are bundled, the GPT-4 advantage cannot yet be assigned specifically to race, gender, or experience level.
  • A natural extension is to run the same pipeline in interactive chat mode, where the model discovers the student's background through conversation; the current one-shot setup may understate how personalized a dialogue-based mentor could be.
  • The word-frequency and overlap method could transfer to other mentoring domains, such as engineering or business, and to non-English settings by swapping the sentence-embedding model.
  • A larger and more diverse evaluator panel, with results split by evaluator background, would test whether the human preference for GPT-4 is stable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This manuscript compares GPT-4o, LLaMA 3, and PaLM 2 as zero-shot personalized career mentors for computing students. Three hand-constructed student profiles (Table I) and 15 mentoring questions (Table II) are used. A semantic-similarity NLP pipeline (Section III.C) measures the percentage of sentences with similarity ≥0.7 between answers across profiles, and word clouds identify profile-specific vocabulary. A human evaluation (Section IV.B) uses 13 Likert items rated by two experts. The paper claims GPT-4 offers the most personalized mentoring.

Significance. The research question is timely, and the study is a useful template for evaluating LLM-based mentoring: it compares three LLMs under a fixed zero-shot prompt, makes the operationalization of "personalized response" explicit, and combines quantitative and human assessment. If the central claim were supported, the work would inform choices of LLM backends for career-mentoring tools. However, the current evidence does not support the headline claim, because the paper's own quantitative criterion ranks LLaMA 3 as more distinct/personalized than GPT-4o, and the human evaluation lacks statistical grounding. The study therefore currently serves more as a cautionary example than as a basis for selecting GPT-4.

major comments (5)
  1. [Section III.C and Table III] The paper's stated operationalization of personalization is the low percentage of sentences with semantic similarity ≥ θ between responses to different profiles (Section III.C). Under this definition, the Overall Average row of Table III shows LLaMA 3 has the lowest overlap for every profile (0.327, 0.142, 0.328 versus GPT-4o's 0.352, 0.197, 0.421), meaning LLaMA 3 produces the most distinct answers. The abstract and conclusion nevertheless attribute the most personalized mentoring to GPT-4. This is a direct internal contradiction, and the quantitative basis for the central claim is missing.
  2. [Section IV.A, Figures 4–6] The word-cloud analysis that is used to override Table III is purely descriptive. It reports salient tokens for each profile without a baseline corpus, without frequency counts, and without any statistical or threshold-based criterion for what counts as "more personalized". The qualitative observations about "African American", "Hispanic", and "woman" cannot by themselves establish a comparative ranking of GPT-4o over LLaMA 3.
  3. [Section IV.B and Figure 3] The human evaluation is reported without numeric results: Figure 3 shows questionnaire scores only as an image, no table of per-item means is given, and no inferential statistics are computed. Only two raters are mentioned, with no inter-rater reliability (e.g., Cohen's κ or Cronbach's α). Consequently, the claim that "GPT-4 outperformed the other two LLMs" in the survey is not statistically supported.
  4. [Section III.A and Table I] The three student profiles differ simultaneously in gender, race/ethnicity, college year, and major (Table I). RQ1 asks whether LLMs account for social background (race/ethnicity), and RQ2 asks whether they account for educational background (year/major). With this profile set, any observed difference in responses between profiles could be due to any of the four attributes, or their interaction; it is impossible to attribute differences to race/ethnicity or educational level separately. This confound undermines both research questions.
  5. [Section III.C, Eq. (1)] The similarity threshold θ=0.7 is introduced without justification or sensitivity analysis. Since the comparative ranking of models depends on this threshold, and the threshold is not validated against human judgments of "same answer," the main quantitative result is not robustly established.
minor comments (6)
  1. [Section IV.A] The sentence "Table IV show the results" should refer to Table III, which is the table that actually appears.
  2. [Throughout] The model name is written inconsistently as "PaLM 2" in Section III.B and "Palm 2" throughout Section IV; please standardize.
  3. [References] Reference [5] is an anonymized placeholder ("Anonymous. Anonymous. In 2024 ASEE Annual Conference & Exposition, 2024") and must be completed or removed.
  4. [Figure 3] The caption of Figure 3 does not identify whether the plotted values are mean scores or individual ratings, and the image is too small to read the survey items; a companion table would improve clarity.
  5. [Figures 4–6] The word clouds use different visual scales and fonts across models, making cross-model comparison unreliable; adding frequency tables would improve clarity.
  6. [Section III.C] The phrase "does not provider personalized answers" should be "does not provide personalized answers."

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the empirical comparison is self-contained, with only a minor non-load-bearing self-citation.

full rationale

The paper is an empirical evaluation, not a derivation: it prompts three LLMs with fixed student profiles, measures response overlap with a sentence-transformer cosine threshold, and adds a human survey. The central claim that GPT-4 is more personalized is grounded in word-frequency inspection and human ratings, not in a fitted model. No parameter is fitted to a subset of the data and then renamed as a prediction; the threshold of 0.7 is fixed and stated. The only self-citation ([5], the authors' prior survey) motivates the selection of the 15 mentoring questions, but it does not constrain which model is judged more personalized, so it is not load-bearing. Concerns raised by the data—such as LLaMA 3 having lower semantic-overlap percentages than GPT-4o on the paper's own metric, and the three profiles differing simultaneously on gender, race, year, and major—are validity or interpretation issues, not circularity. There is no equation whose output is its own input by construction.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or formal entities. The evaluation pipeline and profiles are design choices, not invented entities. The main ledger entries are the unvalidated similarity threshold and the confounded profile design.

free parameters (1)
  • Semantic similarity threshold theta = 0.7
    Chosen without validation. Sentences with cosine similarity above 0.7 are counted as repeated content; the threshold drives the main quantitative table and is not justified by any sensitivity analysis.
assumptions (4)
  • domain assumption Cosine similarity of sentence embeddings measures semantic overlap between mentoring answers.
    Invoked in Equation 1, Section III.C; no evidence that this metric aligns with human judgments of personalization.
  • ad hoc to paper The three constructed student profiles isolate the effect of each social and educational characteristic.
    Section III.A and Table I: profiles differ simultaneously in gender, race, college year, and major, so differences cannot be attributed to any single dimension.
  • domain assumption Word-cloud cumulative word frequencies reflect the degree of personalization.
    Section IV.A: conclusions about GPT-4o's personalization are based on descriptive word clouds, without quantitative measures or inter-rater agreement.
  • domain assumption Human raters' scores are reliable indicators of mentoring quality.
    Section IV.B: only two evaluators are mentioned, with no inter-rater reliability, rater training, or statistical analysis reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Assessing Personalized AI Mentoring with Large Language Models in the Computing Field." pith.science (2026). https://pith.science/paper/CXD3PQGD

@misc{pith2026241208430,
  author       = {Pith},
  title        = {Pith review of: Assessing Personalized AI Mentoring with Large Language Models in the Computing Field},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CXD3PQGD}},
  note         = {Machine review of arXiv:2412.08430}
}
read the original abstract

This paper provides an in-depth evaluation of three state-of-the-art Large Language Models (LLMs) for personalized career mentoring in the computing field, using three distinct student profiles that consider gender, race, and professional levels. We evaluated the performance of GPT-4, LLaMA 3, and Palm 2 using a zero-shot learning approach without human intervention. A quantitative evaluation was conducted through a custom natural language processing analytics pipeline to highlight the uniqueness of the responses and to identify words reflecting each student's profile, including race, gender, or professional level. The analysis of frequently used words in the responses indicates that GPT-4 offers more personalized mentoring compared to the other two LLMs. Additionally, a qualitative evaluation was performed to see if human experts reached similar conclusions. The analysis of survey responses shows that GPT-4 outperformed the other two LLMs in delivering more accurate and useful mentoring while addressing specific challenges with encouragement languages. Our work establishes a foundation for developing personalized mentoring tools based on LLMs, incorporating human mentors in the process to deliver a more impactful and tailored mentoring experience.

Figures

Figures reproduced from arXiv: 2412.08430 by the authors.

Figure 1
Figure 1. Overview of the AI mentoring and analysis using LLMs [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Design of the prompt for AI mentoring C. NLP Analysis Pipeline The NLP analysis pipeline designed in this research is to objectively evaluate whether and how the LLMs consider each student’s social and educational background when answering the questions. The pipeline first analyzes the similarity between the an￾swers for each question using each LLM. The hypothesis is that the percentage of sentences in the answers … view at source ↗
Figure 3
Figure 3. Human Evaluation of the LLMs towards Mentoring [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Word Clouds of All question answers for profile M-AA-J-CS, M-W-F-U, and F-H-F-CS using GPT-4o [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Word Clouds of All question answers for profile M-AA-J-CS, M-W-F-U, and F-H-F-CS using LLaMA 3 [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Word Clouds of All question answers for profile M-AA-J-CS, M-W-F-U, and F-H-F-CS using Palm 2 [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 35 canonical work pages

  1. [1]

    Large language models in medical education: opportunities, challenges, and future directions

    Alaa Abd-Alrazaq, Rawan AlSaad, Dari Alhuwail, Arfan Ahmed, Padraig Mark Healy, Syed Latifi, Sarah Aziz, Rafat Damseh, Sadam Al- abed Alrazak, Javaid Sheikh, et al. Large language models in medical education: opportunities, challenges, and future directions. JMIR Medi- cal Education, 9(1):e48291, 2023

    work page 2023

  2. [2]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [3]

    Ai-supported academic advis- ing: Exploring chatgpt’s current state and future potential toward student empowerment

    Daisuke Akiba and Michelle C Fraboni. Ai-supported academic advis- ing: Exploring chatgpt’s current state and future potential toward student empowerment. Education Sciences, 13(9):885, 2023

    work page 2023

  4. [4]

    Palm 2 technical report

    Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. Palm 2 technical report. arXiv preprint arXiv:2305.10403, 2023

    arXiv 2023

  5. [5]

    Anonymous

    Anonymous. Anonymous. In 2024 ASEE Annual Conference & Exposition, 2024

    work page 2024

  6. [6]

    Designing an AI-Powered Mentorship Platform for Professional Development: Opportunities and Challenges

    Rahul Bagai and Vaishali Mane. Designing an ai-powered mentorship platform for professional development: opportunities and challenges. arXiv preprint arXiv:2407.20233 , 2024

    work page Pith review arXiv 2024

  7. [7]

    Harnessing the potential of large language models in medical education: promise and pitfalls

    Trista M Ben ´ıtez, Yueyuan Xu, J Donald Boudreau, Alfred Wei Chieh Kow, Fernando Bello, Le Van Phuoc, Xiaofei Wang, Xiaodong Sun, Gilberto Ka-Kit Leung, Yanyan Lan, et al. Harnessing the potential of large language models in medical education: promise and pitfalls. Journal of the American Medical Informatics Association , 31(3):776– 783, 2024

    work page 2024

  8. [8]

    Measuring the effectiveness of faculty mentoring relationships

    Ronald A Berk, Janet Berg, Rosemary Mortimer, Benita Walton-Moss, and Theresa P Yeo. Measuring the effectiveness of faculty mentoring relationships. Academic medicine, 80(1):66–71, 2005

    work page 2005

Show all 39 references
  1. [9]

    Role of chatgpt in computer programming

    Som Biswas. Role of chatgpt in computer programming. Mesopotamian Journal of Computer Science , 2023:9–15, 2023

    2023

  2. [10]

    Academic writing with gpt-3.5 (chatgpt): reflections on practices, efficacy and transparency

    O ˘guz’Oz’ Buruk. Academic writing with gpt-3.5 (chatgpt): reflections on practices, efficacy and transparency. In Proceedings of the 26th International Academic Mindtrek Conference , pages 144–153, 2023

    2023

  3. [11]

    Exploring the role of chatgpt as a peer coach for developing research proposals: Feedback quality, prompts, and student reflection

    Johannes Cronj ´e. Exploring the role of chatgpt as a peer coach for developing research proposals: Feedback quality, prompts, and student reflection. Electronic Journal of e-Learning , 22(2):1–15, 2023

    2023

  4. [12]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  5. [13]

    Beyond traditional assessment: Exploring the impact of large language models on grading practices

    O Fagbohun, NP Iduwe, M Abdullahi, A Ifaturoti, and OM Nwanna. Beyond traditional assessment: Exploring the impact of large language models on grading practices. Journal of Artifical Intelligence and Machine Learning & Data Science , 2(1):1–8, 2024

    2024

  6. [14]

    Ai-tutoring in software engineering education

    Eduard Frankford, Clemens Sauerwein, Patrick Bassner, Stephan Kr- usche, and Ruth Breu. Ai-tutoring in software engineering education. In Proceedings of the 46th International Conference on Software Engi- neering: Software Engineering Education and Training, pages 309–319, 2024

    2024

  7. [15]

    Let’s not forget: Learning analytics are about learning

    Dragan Ga ˇsevi´c, Shane Dawson, and George Siemens. Let’s not forget: Learning analytics are about learning. TechTrends, 59:64–71, 2015

    2015

  8. [16]

    Accuracy and reliability of chatbot responses to physician questions

    Rachel S Goodman, J Randall Patrinely, Cosby A Stone, Eli Zimmer- man, Rebecca R Donald, Sam S Chang, Sean T Berkowitz, Avni P Finn, Eiman Jahangir, Elizabeth A Scoville, et al. Accuracy and reliability of chatbot responses to physician questions. JAMA network open , 6(10):e23...

    2023

  9. [17]

    Students’ experiences of us- ing chatgpt in an undergraduate programming course

    Philipp Haindl and Gerald Weinberger. Students’ experiences of us- ing chatgpt in an undergraduate programming course. IEEE Access , 12:43519–43529, 2024

    2024

  10. [18]

    Artificial intelligence supporting independent student learning: An evaluative case study of chatgpt and learning to code

    Kendall Hartley, Merav Hayak, and Un Hyeok Ko. Artificial intelligence supporting independent student learning: An evaluative case study of chatgpt and learning to code. Education Sciences, 14(2):120, 2024

    2024

  11. [19]

    The classroom as a dashboard: Co-designing wearable cognitive augmentation for k-12 teachers

    Kenneth Holstein, Gena Hong, Mera Tegene, Bruce M McLaren, and Vincent Aleven. The classroom as a dashboard: Co-designing wearable cognitive augmentation for k-12 teachers. In Proceedings of the 8th international conference on learning Analytics and knowledge , pages 79–88, 2018

    2018

  12. [20]

    Lecturer’s apprentice: A chatbot for assisting novice programmers

    Mohammed Ismail and Abejide Ade-Ibijola. Lecturer’s apprentice: A chatbot for assisting novice programmers. In 2019 international multidisciplinary information technology and engineering conference (IMITEC), pages 1–8. IEEE, 2019

    2019

  13. [21]

    Computer science education in chatgpt era: Experiences from an exper- iment in a programming course for novice programmers

    Toma ˇz Kosar, Dragana Ostoji ´c, Yu David Liu, and Marjan Mernik. Computer science education in chatgpt era: Experiences from an exper- iment in a programming course for novice programmers. Mathematics, 12(5):629, 2024

    2024

  14. [22]

    Architecture for building conversa- tional agents that support collaborative learning

    Rohit Kumar and Carolyn P Rose. Architecture for building conversa- tional agents that support collaborative learning. IEEE Transactions on Learning Technologies, 4(1):21–34, 2010

    2010

  15. [23]

    Growing enrollments require us to do more: Perspectives on broadening participation during an undergraduate computing enrollment boom

    Kathleen J Lehman, Julia Rose Karpicz, Veronika Rozhenkova, Jamelia Harris, and Tomoko M Nakajima. Growing enrollments require us to do more: Perspectives on broadening participation during an undergraduate computing enrollment boom. In Proceedings of the 52nd ACM Technical Sy...

    2021

  16. [24]

    Personalized tutoring: Chatgpt as a virtual tutor for personalized learning experiences

    Fernando Antonio Flores Limo, David Raul Hurtado Tiza, Mari- bel Mamani Roque, Edward Espinoza Herrera, Jos ´e Patricio Mu ˜noz Murillo, Jorge Jinchu ˜na Huallpa, Victor Andre Ariza Flores, Alejandro Guadalupe Rinc´on Castillo, Percy Fritz Puga Pe ˜na, Christian Paolo Mar- tel...

    2023

  17. [25]

    Revision-bot: A chatbot for studying past questions in introductory programming

    Chinedu Wilfred Okonkwo and Abejide Ade-Ibijola. Revision-bot: A chatbot for studying past questions in introductory programming. IAENG International Journal of Computer Science , 49(3), 2022

    2022

  18. [26]

    Large language models (gpt) for automating feedback on programming assignments

    Maciej Pankiewicz and Ryan S Baker. Large language models (gpt) for automating feedback on programming assignments. arXiv e-prints, pages arXiv–2307, 2023

    2023

  19. [27]

    Nitin Rane. Enhancing the quality of teaching and learning through gemini, chatgpt, and similar generative artificial intelligence: Challenges, future prospects, and ethical considerations in education. TESOL and Technology Studies, 5(1):1–6, 2024

    2024

  20. [28]

    Evolution and revolution in artificial intel- ligence in education

    Ido Roll and Ruth Wylie. Evolution and revolution in artificial intel- ligence in education. International journal of artificial intelligence in education, 26:582–599, 2016

    2016

  21. [29]

    Chatgpt: bridging the gap on mentorship for international medical graduates in low enrolment specialties

    Frank Sanya-Isijola and Fok-Han Leung. Chatgpt: bridging the gap on mentorship for international medical graduates in low enrolment specialties. Canadian Medical Education Journal , 15(2):93, 2024

    2024

  22. [30]

    Exploring the future of mathematics teaching: Insight with chatgpt

    Edi Supriyadi and KS Kuncoro. Exploring the future of mathematics teaching: Insight with chatgpt. Union: Jurnal Ilmiah Pendidikan Matem- atika, 11(2):305–316, 2023

    2023

  23. [31]

    Using generative ai to provide feedback to adult tutors in training and assess real-life performance

    Danielle R Thomas, Erin Gatz, Shivang Gupta, Jionghao Lin, Cindy Tipper, and Kenneth Koedinger. Using generative ai to provide feedback to adult tutors in training and assess real-life performance

  24. [32]

    Ex- ploring the role of ai assistants in computer science education: Methods, implications, and instructor perspectives

    Tianjia Wang, Daniel Vargas D ´ıaz, Chris Brown, and Yan Chen. Ex- ploring the role of ai assistants in computer science education: Methods, implications, and instructor perspectives. In 2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), pages 92–

    2023

  25. [33]

    Chatgpt: A revolutionary tool for teaching and learning math- ematics

    Yousef Wardat, Mohammad A Tashtoush, Rommel AlAli, and Adeeb M Jarrah. Chatgpt: A revolutionary tool for teaching and learning math- ematics. Eurasia Journal of Mathematics, Science and Technology Education, 19(7):em2286, 2023

    2023

  26. [34]

    Large language models for education: A survey

    Hanyi Xu, Wensheng Gan, Zhenlian Qi, Jiayang Wu, and Philip S Yu. Large language models for education: A survey. arXiv preprint arXiv:2405.13001, 2024

    2024 arXiv

  27. [35]

    Does chatgpt help with introductory programming? an experiment of students using chatgpt in cs1

    Yuankai Xue, Hanlin Chen, Gina R Bai, Robert Tairas, and Yu Huang. Does chatgpt help with introductory programming? an experiment of students using chatgpt in cs1. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and T...

    2024

  28. [36]

    Augmented intelligence in programming learning: Examining student views on the use of chatgpt for programming learning

    Ramazan Yilmaz and Fatma Gizem Karaoglan Yilmaz. Augmented intelligence in programming learning: Examining student views on the use of chatgpt for programming learning. Computers in Human Behavior: Artificial Humans , 1(2):100005, 2023

    2023

  29. [37]

    The effect of generative artificial intelligence (ai)-based tool use on students’ computational thinking skills, programming self-efficacy and motivation

    Ramazan Yilmaz and Fatma Gizem Karaoglan Yilmaz. The effect of generative artificial intelligence (ai)-based tool use on students’ computational thinking skills, programming self-efficacy and motivation. Computers and Education: Artificial Intelligence , 4:100147, 2023

    2023

  30. [38]

    A new mentor evaluation tool: Evidence of validity

    Michi Yukawa, Stuart A Gansky, Patricia O’Sullivan, Arianne Teherani, and Mitchell D Feldman. A new mentor evaluation tool: Evidence of validity. PloS one, 15(6):e0234345, 2020

    2020

  31. [39]

    Chatgpt in academic writing and publishing: A comprehensive guide

    Medhat Zohery. Chatgpt in academic writing and publishing: A comprehensive guide. Artificial intelligence in academia, research and science: ChatGPT as a case study , pages 10–61, 2023

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.