Pith. sign in

REVIEW 3 major objections 4 minor 103 references

Towards Reliable Generative AI-Driven Scaffolding: Reducing Hallucinations and Enhancing Quality in Self-Regulated Learning Support

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims a multi-agent evaluator can catch hallucinated AI-generated learning scaffolds almost as well as human experts, and that both proposed evaluation approaches reduce hallucinations in the generated content.

desk verdict Potentially useful evaluation pipeline for SRL scaffolds, but the central alignment claim is unverifiable in the delivered text. read the letter →

arxiv 2508.05929 v2 pith:FUVCIBA6 submitted 2025-08-08 cs.CY

Keyang Qian , Shiqi Liu , Tongguang Li , Mladen Raković , Xinyu Li , Rui Guan , Inge Molenaar , Sadia Nawaz
show 3 more authors
Zachari Swiecki Lixiang Yan Dragan Gašević
This is my paper
classification cs.CY
keywords hallucinationslargelanguagemodelsself-regulatedlearningscaffoldingmulti-agentevaluationLLM-as-a-judgereliabilitygenerativeAIineducation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that hallucinations in large-language-model-generated scaffolds for self-regulated learning can be reliably detected and reduced before students ever see them. It proposes two evaluation approaches: a multi-agent system that assesses whether a scaffold actually targets the intended SRL process, and an LLM-as-a-Judge technique that rates the scaffold's helpfulness. On self-constructed evaluation datasets, the multi-agent reliability approach reportedly outperforms single-agent LLMs and machine-learning baselines, showing near-perfect agreement with human expert judgments. Both approaches, the paper argues, can be built into GenAI-powered scaffolding systems to filter low-quality or fabricated content. The paper also identifies bias limitations in the LLM-as-a-Judge approach, tempering its use as a standalone quality gate.

What carries the argument

The central mechanism is the multi-agent system for reliability evaluation, in which multiple LLM-based agents collectively assess whether a generated scaffold is on-target for a specified SRL process, and the 'LLM-as-a-Judge' technique, which rates scaffold helpfulness. The multi-agent design is what produces the near-expert alignment, and the evaluation outputs are what allow hallucinated or off-target scaffolds to be filtered before reaching students.

What would settle it

Apply the multi-agent reliability evaluator to a fresh set of scaffolds from an unseen domain or learner population, have human experts independently label hallucinations in them, and compute agreement. If agreement falls materially below the reported near-perfect level, the claim of generalizable expert-level reliability is falsified. A complementary test: adversarially insert fabricated but plausible-sounding SRL guidance and check how often the evaluator flags it.

Watch

Extended reading notes

Core claim

The central claim is that a multi-agent reliability evaluation approach can assess whether LLM-generated scaffolds accurately target relevant self-regulated learning processes, and that this evaluation shows almost perfect alignment with human experts' evaluations, outperforming single-agent LLM systems and machine-learning baselines. The second proposed approach, LLM-as-a-Judge, evaluates scaffolds for helpfulness and also contributes to reducing hallucinations. Together, the findings support integrating these evaluation methods into GenAI-powered personalised SRL scaffolding systems to mitigate hallucination issues and improve scaffolding quality. The authors additionally report and discus

Load-bearing premise

The claim of near-perfect alignment with human experts rests on the assumption that the self-constructed evaluation datasets and the human expert labels are a valid, representative ground truth for hallucinations in SRL scaffolds.

Editorial extensions

If this is right

  • If the multi-agent reliability check works as reported, it can be inserted as an automatic pre-filter before AI-generated scaffolds are shown to students, reducing exposure to fabricated or irrelevant learning guidance.
  • Both evaluation approaches could be combined: one to check SRL-process targeting and one to judge helpfulness, catching different failure modes of generative models.
  • The reported reduction in hallucinations suggests a viable path toward safer deployment of generative AI in educational technology without requiring a human reviewer for every generated scaffold.
  • The identified bias limitations of LLM-as-a-Judge imply that LLM-based quality judgments should be validated against human ratings in the specific educational context before being relied on.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 'almost perfect alignment with human experts' is measured against self-constructed datasets and expert labels; whether that alignment persists across varied subjects, student populations, and scaffolding styles is an open empirical question the paper does not resolve.
  • A testable extension of the paper's approach is to run the multi-agent evaluator on scaffolds designed to contain subtle, context-specific hallucinations (e.g., wrong prerequisite knowledge for a given learner model) and measure its catch rate relative to expert review.
  • If the evaluator is deployed in a live tutoring system, one would expect observable effects on downstream learning outcomes; a controlled study comparing filtered versus unfiltered scaffolding would provide a stronger test of practical value than agreement metrics alone.
  • The paper's bias analysis of LLM-as-a-Judge hints that a single-judge setup may be systematically skewed by model preferences; a multi-agent judging ensemble could be a natural follow-up to reduce that bias.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes and evaluates two GenAI-based approaches for reducing hallucinations and improving quality in automatically generated self-regulated learning (SRL) scaffolds. The first is a multi-agent reliability evaluation system that judges whether a scaffold targets the intended SRL processes; the second uses an LLM-as-a-judge technique to assess helpfulness. The authors report constructing evaluation datasets, comparing against single-agent LLM and machine-learning baselines, and claim that the reliability evaluation approach 'shows almost perfect alignment with human experts' evaluations' and that both approaches reduce hallucinations. The abstract also discloses bias limitations of the LLM-as-a-judge technique. The full text supplied for review, however, is unreadable due to character-encoding corruption, so the methodology, prompts, statistics, and tables cannot be independently checked.

Significance. If the claims hold, the paper contributes a practical pipeline for filtering hallucinated or off-target SRL scaffolds before they reach students, with a multi-agent evaluator that approaches expert-level agreement. The topic is timely, the two-pronged evaluation design is sensible, and the explicit discussion of LLM-as-a-judge bias is a strength. However, the central empirical claim—'almost perfect alignment with human experts'—is currently unverifiable because the manuscript text is garbled and the abstract omits the required quantitative support (agreement metric, sample size, annotator details, confidence intervals, baseline strength). The broader contribution therefore rests on evidence that the paper does not presently make accessible.

major comments (3)
  1. [Abstract and Tables 1–6] The headline claim of 'almost perfect alignment with human experts' is unsupported as reported: no agreement statistic (e.g., Cohen's kappa, ICC, accuracy), no sample size, no number of annotators, no confidence interval, and no baseline performance values are given. The full text is encoded garbage, so these cannot be recovered from the tables. Because the production recommendation depends on this alignment, the authors must provide the full evaluation protocol, raw agreement scores, and dataset sizes.
  2. [Reliability evaluation method (multi-agent LLM judge)] The reliability evaluation uses LLM agents to decide whether LLM-generated scaffolds target SRL processes. The paper itself acknowledges bias limitations of LLM-as-a-judge, but it does not rule out a circularity concern: if the judge rubric, the generator prompt, and the human label instructions all derive from the same SRL framework, high agreement with human labels may be inflated. Please report how human expert labels were collected independently, whether labelers saw the generator's rubric, and provide per-process agreement and disagreement examples.
  3. [Full text (entire manuscript)] The provided manuscript body is corrupted by character-encoding errors, rendering all methodology, prompts, dataset descriptions, and result tables unreadable. This is not a minor presentation issue: it blocks verification of every load-bearing result, including the claimed hallucination reduction and the baseline comparisons. The authors should resubmit a readable source and include the full prompts and dataset construction details in an appendix.
minor comments (4)
  1. [Abstract] Specify the agreement metric and its numerical range for 'almost perfect alignment' so readers can interpret the claim without accessing the body.
  2. [Tables 1–3] Add explicit sample sizes and class balance to each dataset table; without these, the percentage-based hallucination-reduction results are difficult to interpret.
  3. [Notation] The SRL process categories should be defined consistently in one place; the current garbled Table 1 appears to mix process names and evaluation categories.
  4. [Source file] Fix the character encoding in the arXiv source; the non-ASCII sections are rendered as mojibake throughout.

Circularity Check

0 steps flagged · score 0.0 of 10

No demonstrated circularity: the central evaluation claim is empirical and no self-referential reduction is exhibited in the readable text.

full rationale

The only readable portion of the manuscript is the abstract; the supplied full text is encoding-corrupted and contains no parseable equations, method statements, or reference list. The abstract reports an empirical evaluation: the authors 'constructed evaluation datasets, and compared our results with single-agent LLM systems and machine learning approach baselines,' then found that the reliability evaluation approach 'shows almost perfect alignment with human experts' evaluations.' This is a comparative empirical claim, and constructing one's own evaluation dataset with human expert labels is a standard practice, not by itself a circular reduction. Nothing in the abstract defines the multi-agent reliability score in terms of the human-expert labels or fits the evaluator to the labels it is then said to predict; the disclosed 'bias limitations of the LLM-as-a-Judge technique' also speak to a genuine methodological concern rather than a definitional identity. No self-citation is visible in the readable text, and no equation or construction can be quoted to exhibit X defined in terms of Y. Under the rule that circularity must be demonstrated by quotation and specific reduction, the paper receives 0. The unverifiability of the full text is a transparency/correctness concern, not evidence of circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

Abstract-only review. The only identifiable upstream commitments are domain assumptions about ground truth and construct validity. No free parameters are stated in the abstract, and no new entities are invented.

assumptions (3)
  • domain assumption Human expert evaluations are a valid ground truth for scaffold quality and hallucination.
    Abstract says alignment with human experts is the success criterion. Whether expert labels are noise-free and consistent is not addressed.
  • domain assumption The constructed evaluation datasets are representative of the scaffold distribution the system will face.
    Abstract reports constructed datasets but no sampling strategy or coverage analysis; generalization to unseen scaffolds is assumed.
  • domain assumption The multi-agent reliability judgment and the LLM-as-a-judge scores measure the intended constructs: SRL process targeting and helpfulness.
    Construct validity is asserted (accurate targeting, helpfulness) but no evidence of disentanglement from prompt-following is given in the abstract.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Reliable Generative AI-Driven Scaffolding: Reducing Hallucinations and Enhancing Quality in Self-Regulated Learning Support." pith.science (2026). https://pith.science/paper/FUVCIBA6

@misc{pith2026250805929,
  author       = {Pith},
  title        = {Pith review of: Towards Reliable Generative AI-Driven Scaffolding: Reducing Hallucinations and Enhancing Quality in Self-Regulated Learning Support},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FUVCIBA6}},
  note         = {Machine review of arXiv:2508.05929}
}
read the original abstract

Generative Artificial Intelligence (GenAI) holds a potential to advance existing educational technologies with capabilities to automatically generate personalised scaffolds that support students' self-regulated learning (SRL). While advancements in large language models (LLMs) promise improvements in the adaptability and quality of educational technologies for SRL, there remain concerns about the hallucinations in content generated by LLMs, which can compromise both the learning experience and ethical standards. To address these challenges, we proposed GenAI-enabled approaches for evaluating personalised SRL scaffolds before they are presented to students, aiming for reducing hallucinations and improving the overall quality of LLM-generated personalised scaffolds. Specifically, two approaches are investigated. The first approach involved developing a multi-agent system approach for reliability evaluation to assess the extent to which LLM-generated scaffolds accurately target relevant SRL processes. The second approach utilised the "LLM-as-a-Judge" technique for quality evaluation that evaluates LLM-generated scaffolds for their helpfulness in supporting students. We constructed evaluation datasets, and compared our results with single-agent LLM systems and machine learning approach baselines. Our findings indicate that the reliability evaluation approach is highly effective and outperforms the baselines, showing almost perfect alignment with human experts' evaluations. Moreover, both proposed evaluation approaches can be harnessed to effectively reduce hallucinations. Additionally, we identified and discussed bias limitations of the "LLM-as-a-Judge" technique in evaluating LLM-generated scaffolds. We suggest incorporating these approaches into GenAI-powered personalised SRL scaffolding systems to mitigate hallucination issues and improve the overall scaffolding quality.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

103 extracted references · 63 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in ":" * " " * FUNCTION f...

  2. [2]

    , author Wiratunga, N

    author Abeyratne, R. , author Wiratunga, N. , author Martin, K. , author Nkisi-Orji, I. , author Jayawardena, L. , year 2025 . title Alignllm: Alignment-based evaluation using ensemble of llms-as-judges for q&a , in: booktitle International Conference on Case-Based Reasoning , organization Springer . pp. pages 21--36

    2025

  3. [3]

    , author Echeverria, V

    author Alfredo, R. , author Echeverria, V. , author Jin, Y. , author Yan, L. , author Swiecki, Z. , author Ga s evi \'c , D. , author Martinez-Maldonado, R. , year 2024 . title Human-centred learning analytics and ai in education: A systematic literature review . journal Computers and Education: Artificial Intelligence , pages 100215

    2024

  4. [4]

    , author Jivet, I

    author Alvarez, R.P. , author Jivet, I. , author P \'e rez-Sanagustin, M. , author Scheffel, M. , author Verbert, K. , year 2022 . title Tools designed to support self-regulated learning in online learning environments: A systematic review . journal IEEE Transactions on Learning Technologies volume 15 , pages 508--522

    2022

  5. [5]

    , author Hadwin, A.F

    author Azevedo, R. , author Hadwin, A.F. , year 2005 . title Scaffolding self-regulated learning and metacognition--implications for the design of computer-based scaffolds . journal Instructional science volume 33 , pages 367--379

    2005

  6. [6]

    , author Jeske, D

    author Backhaus, J. , author Jeske, D. , author Poinstingl, H. , author Koenig, S. , year 2017 . title Assessing efficiency of prompts based on learner characteristics . journal Computers volume 6 , pages 7

    2017

  7. [7]

    , author Agarwal, A

    author Banerjee, S. , author Agarwal, A. , author Singla, S. , year 2024 . title Llms will always hallucinate, and we need to live with this . journal arXiv preprint arXiv:2409.05746

    arXiv 2024

  8. [8]

    , year 2023

    author Bang, F. , year 2023 . title Gptcache: An open-source semantic cache for llm applications enabling faster answers and cost savings , in: booktitle Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023) , pp. pages 212--218

    2023

Show all 103 references
  1. [9]

    , year 2009

    author Bannert, M. , year 2009 . title Promoting self-regulated learning through prompts . journal Zeitschrift f \"u r P \"a dagogische Psychologie volume 23 , pages 139--145

    2009

  2. [10]

    , author Sonnenberg, C

    author Bannert, M. , author Sonnenberg, C. , author Mengelkamp, C. , author Pieger, E. , year 2015 . title Short-and long-term effects of students’ self-directed metacognitive prompts on navigation behavior and learning performance . journal Computers in Human Behavior volume ...

    2015

  3. [11]

    , author Bjork, R.A

    author Bjork, E.L. , author Bjork, R.A. , year 2023 . title Introducing desirable difficulties into practice and instruction: Obstacles and opportunities. , in: editor Overson, C.E. , editor Hakala, C.M. , editor Kordonowy, L.L. , editor Benassi, V.A. (Eds.), booktitle In thei...

    2023

  4. [12]

    , year 2022

    author Gutierrez de Blume, A.P. , year 2022 . title Calibrating calibration: A meta-analysis of learning strategy instruction interventions to improve metacognitive monitoring accuracy. journal Journal of Educational Psychology volume 114 , pages 681

    2022

  5. [13]

    , year 1997

    author Boekaerts, M. , year 1997 . title Self-regulated learning: A new concept embraced by researchers, policy makers, educators, teachers, and students . journal Learning and instruction volume 7 , pages 161--186

    1997

  6. [14]

    , author Chandrasekaran, V

    author Bubeck, S. , author Chandrasekaran, V. , author Eldan, R. , author Gehrke, J. , author Horvitz, E. , author Kamar, E. , author Lee, P. , author Lee, Y.T. , author Li, Y. , author Lundberg, S. , et al., year 2023 . title Sparks of artificial general intelligence: Early e...

    2023 arXiv

  7. [15]

    , author Lin, M.P.C

    author Chang, D.H. , author Lin, M.P.C. , author Hajian, S. , author Wang, Q.Q. , year 2023 . title Educational design principles of using ai chatbot that supports self-regulated learning in education: Goal setting, feedback, and personalization . journal Sustainability volume...

    2023

  8. [16]

    , author Chen, S

    author Chen, G.H. , author Chen, S. , author Liu, Z. , author Jiang, F. , author Wang, B. , year 2024 . title Humans or llms as the judge? a study on judgement biases . journal arXiv preprint arXiv:2402.10669

    2024 arXiv

  9. [17]

    , author Maladry, A

    author De Langhe, L. , author Maladry, A. , author Vanroy, B. , author De Bruyne, L. , author Singh, P. , author Lefever, E. , author De Clercq, O. , year 2024 . title Benchmarking zero-shot text classification for dutch . journal Computational Linguistics in the Netherlands J...

    2024

  10. [18]

    , author G \"u nther, F

    author Dentella, V. , author G \"u nther, F. , author Murphy, E. , author Marcus, G. , author Leivada, E. , year 2024 . title Testing ai on language comprehension tasks reveals insensitivity to underlying meaning . journal Scientific Reports volume 14 , pages 28083

    2024

  11. [19]

    , author Nietfeld, J.L

    author DiFrancesca, D. , author Nietfeld, J.L. , author Cao, L. , year 2016 . title A comparison of high and low achieving students on self-regulated learning variables . journal Learning and Individual Differences volume 45 , pages 228--236

    2016

  12. [20]

    , author B \"u ttner, G

    author Dignath, C. , author B \"u ttner, G. , year 2008 . title Components of fostering self-regulated learning among students. a meta-analysis on intervention studies at primary and secondary school level . journal Metacognition and learning volume 3 , pages 231--264

    2008

  13. [21]

    , author Lin, C.J

    author Fan, R.E. , author Lin, C.J. , year 2007 . title A study on threshold selection for multi-label classification . journal Department of Computer Science, National Taiwan University , pages 1--23

    2007

  14. [22]

    , author van der Graaf, J

    author Fan, Y. , author van der Graaf, J. , author Lim, L. , author Rakovi \'c , M. , author Singh, S. , author Kilgour, J. , author Moore, J. , author Molenaar, I. , author Bannert, M. , author Ga s evi \'c , D. , year 2022 . title Towards investigating the validity of measur...

    2022

  15. [23]

    , author Tang, L

    author Fan, Y. , author Tang, L. , author Le, H. , author Shen, K. , author Tan, S. , author Zhao, Y. , author Shen, Y. , author Li, X. , author Ga s evi \'c , D. , year 2024 . title Beware of metacognitive laziness: Effects of generative artificial intelligence on learning mo...

    2024

  16. [24]

    , author Junus, K

    author Fariani, R.I. , author Junus, K. , author Santoso, H.B. , year 2023 . title A systematic literature review on personalised learning in the higher education context . journal Technology, Knowledge and Learning volume 28 , pages 449--476

    2023

  17. [25]

    , year 2016

    author Flasi \'n ski, M. , year 2016 . title Introduction to artificial intelligence . publisher Springer

    2016

  18. [26]

    , author Garrido-Merch \'a n, E.C

    author Gozalo-Brizuela, R. , author Garrido-Merch \'a n, E.C. , year 2023 . title A survey of generative ai applications . journal arXiv preprint arXiv:2306.02781

    2023 arXiv

  19. [27]

    , author Lim, L

    author van der Graaf, J. , author Lim, L. , author Fan, Y. , author Kilgour, J. , author Moore, J. , author Bannert, M. , author Gasevic, D. , author Molenaar, I. , year 2021 . title Do instrumentation tools capture self-regulated learning? , in: booktitle LAK21: 11th Internat...

    2021

  20. [28]

    , author Rakovi \'c , M

    author van der Graaf, J. , author Rakovi \'c , M. , author Fan, Y. , author Lim, L. , author Singh, S. , author Bannert, M. , author Ga s evi \'c , D. , author Molenaar, I. , year 2023 a. title How to design and evaluate personalized scaffolds for self-regulated learning . jou...

    2023

  21. [29]

    , author Taub, M

    author van der Graaf, J. , author Taub, M. , author Fan, Y. , year 2023 b. title Introduction to special issue on facilitating self-regulated learning with scaffolds: Recent advances and future directions . journal Metacognition and Learning volume 18 , pages 623--629

    2023

  22. [30]

    , author Land, S.M

    author Greene, B.A. , author Land, S.M. , year 2000 . title A qualitative analysis of scaffolding use in a resource-based learning environment involving the world wide web . journal Journal of Educational Computing Research volume 23 , pages 151--179

    2000

  23. [31]

    , author Dellinger, K.R

    author Greene, J.A. , author Dellinger, K.R. , author T \"u ys \"u zo g lu, B.B. , author Costa, L.J. , year 2013 . title A two-tiered approach to analyzing self-regulated learning data to inform the design of hypermedia learning environments . journal International handbook o...

    2013

  24. [32]

    , author Bishop, P

    author Gunawardena, M. , author Bishop, P. , author Aviruppola, K. , year 2024 . title Personalized learning: The simple, the complicated, the complex and the chaotic . journal Teaching and Teacher Education volume 139 , pages 104429

    2024

  25. [33]

    , year 2022

    author Guo, L. , year 2022 . title Using metacognitive prompts to enhance self-regulated learning and learning outcomes: A meta-analysis of experimental studies in computer-based learning environments . journal Journal of Computer Assisted Learning volume 38 , pages 811--832

    2022

  26. [34]

    , author Zhang, Q

    author Han, S. , author Zhang, Q. , author Yao, Y. , author Jin, W. , author Xu, Z. , author He, C. , year 2024 . title Llm multi-agent systems: Challenges and open problems . journal arXiv preprint arXiv:2402.03578

    2024 arXiv

  27. [35]

    , author Timperley, H

    author Hattie, J. , author Timperley, H. , year 2007 . title The power of feedback . journal Review of educational research volume 77 , pages 81--112

    2007

  28. [36]

    , author Qu, Y

    author Huang, H. , author Qu, Y. , author Liu, J. , author Yang, M. , author Zhao, T. , year 2024 . title An empirical study of llm-as-a-judge for llm evaluation: Fine-tuned judge models are task-specific classifiers . journal arXiv preprint arXiv:2403.02839

    2024 arXiv

  29. [37]

    title Tracking the pricing trajectory for large language models like gpt-4 and chatgpt

    author Inworld AI , year 2023 . title Tracking the pricing trajectory for large language models like gpt-4 and chatgpt . https://inworld.ai/blog/tracking-the-pricing-trajectory-for-large-language-models-like-gpt-4-and-chatgpt. note accessed: 2024-06-04

    2023

  30. [38]

    , author Ga s evi \'c , D

    author Jovanovi \'c , J. , author Ga s evi \'c , D. , author Dawson, S. , author Pardo, A. , author Mirriahi, N. , year 2017 . title Learning analytics to unveil learning strategies in a flipped classroom . journal The Internet and Higher Education volume 33 , pages 74--85

    2017

  31. [39]

    , year 2017

    author Karlen, Y. , year 2017 . title The development of a new instrument to assess metacognitive strategy knowledge about academic writing and its relation to self-regulated writing and writing performance . journal Journal of Writing Research volume 9 , pages 61--86

    2017

  32. [40]

    , author Retnawati, H

    author Kesuma, A.T. , author Retnawati, H. , author Putranta, H. , year 2021 . title Analysis of self-regulated learning skills in senior high school students: A phenomenological study. journal TEM Journal volume 10

    2021

  33. [41]

    , year 2023

    author Kikalishvili, S. , year 2023 . title Unlocking the potential of gpt-3 in education: Opportunities, limitations, and recommendations for effective integration . journal Interactive Learning Environments , pages 1--13

    2023

  34. [42]

    , author Downs, C.T

    author Kirby, N.F. , author Downs, C.T. , year 2007 . title Self-assessment and the disadvantaged student: Potential for encouraging self-regulated learning? journal Assessment & Evaluation in Higher Education volume 32 , pages 475--494

    2007

  35. [43]

    , author Dainese, N

    author Koutcheme, C. , author Dainese, N. , author Sarsa, S. , author Hellas, A. , author Leinonen, J. , author Denny, P. , year 2024 . title Open source language models can provide feedback: Evaluating llms' ability to help students using gpt-4-as-a-judge , in: booktitle Proc...

    2024

  36. [44]

    , author Khosravi, H

    author Lahza, H. , author Khosravi, H. , author Demartini, G. , author Gasevic, D. , year 2022 . title Effects of technological interventions for self-regulation: A control experiment in learnersourcing , in: booktitle LAK22: 12th international learning analytics and knowledge...

    2022

  37. [45]

    title Guide to hallucinations in large language models

    author Lakera AI , year 2024 . title Guide to hallucinations in large language models . https://www.lakera.ai/blog/guide-to-hallucinations-in-large-language-models. note accessed: 2024-06-04

    2024

  38. [46]

    , author Hakkarainen, K

    author Lehtinen, E. , author Hakkarainen, K. , author Lipponen, L. , author Rahikainen, M. , author Muukkonen, H. , year 1999 . title Computer supported collaborative learning: A review . journal The JHGI Giesbers reports on education volume 10 , pages 1999

    1999

  39. [47]

    , author Fan, Y

    author Li, T. , author Fan, Y. , author Tan, Y. , author Wang, Y. , author Singh, S. , author Li, X. , author Rakovi \'c , M. , author Van Der Graaf, J. , author Lim, L. , author Yang, B. , et al., year 2023 . title Analytics of self-regulated learning scaffolding: effects on ...

    2023

  40. [48]

    , author Nath, D

    author Li, T. , author Nath, D. , author Cheng, Y. , author Fan, Y. , author Li, X. , author Raković, M. , author Khosravi, H. , author Swiecki, Z. , author Tsai, Y.S. , author Gašević, D. , year 2025 . title Turning real-time analytics into adaptive scaffolds for self-regulat...

    2025

  41. [49]

    , author Bannert, M

    author Lim, L. , author Bannert, M. , author van der Graaf, J. , author Fan, Y. , author Rakovic, M. , author Singh, S. , author Molenaar, I. , author Ga s evi \'c , D. , year 2024 . title How do students learn with real-time personalized scaffolds? journal British Journal of ...

    2024

  42. [50]

    , author Bannert, M

    author Lim, L. , author Bannert, M. , author van der Graaf, J. , author Singh, S. , author Fan, Y. , author Surendrannair, S. , author Rakovic, M. , author Molenaar, I. , author Moore, J. , author Ga s evi \'c , D. , year 2023 . title Effects of real-time analytics-based perso...

    2023

  43. [51]

    , author Chang, D

    author Lin, M.P.C. , author Chang, D. , year 2023 . title Chat-acts: A pedagogical framework for personalized chatbot to enhance active learning and self-regulated learning . journal Computers and Education: Artificial Intelligence volume 5 , pages 100167

    2023

  44. [52]

    , author Huang, Z

    author Liu, J. , author Huang, Z. , author Xiao, T. , author Sha, J. , author Wu, J. , author Liu, Q. , author Wang, S. , author Chen, E. , year 2024 a. title Socraticlm: Exploring socratic personalized teaching with large language models , in: booktitle The Thirty-eighth Annu...

    2024

  45. [53]

    , author Hwang, G.J

    author Liu, Z.M. , author Hwang, G.J. , author Chen, C.Q. , author Chen, X.D. , author Ye, X.D. , year 2024 b. title Integrating large language models into efl writing instruction: effects on performance, self-regulated learning strategies, and motivation . journal Computer As...

    2024

  46. [54]

    , author Ngoc, H.N

    author Long, D.X. , author Ngoc, H.N. , author Sim, T. , author Dao, H. , author Joty, S. , author Kawaguchi, K. , author Chen, N.F. , author Kan, M.Y. , year 2024 . title Llms are biased towards output formats! systematically evaluating and mitigating output format bias of ll...

    2024 arXiv

  47. [55]

    , author Ga s evi \'c , D

    author Miliki \'c , N. , author Ga s evi \'c , D. , author Jovanovi \'c , J. , year 2018 . title Measuring effects of technology-enabled mirroring scaffolds on self-regulated learning . journal IEEE Transactions on Learning Technologies volume 13 , pages 150--163

    2018

  48. [56]

    , author Biswas, G

    author Munshi, A. , author Biswas, G. , author Baker, R. , author Ocumpaugh, J. , author Hutt, S. , author Paquette, L. , year 2023 . title Analysing adaptive scaffolds that help students develop self-regulated learning behaviours . journal Journal of Computer Assisted Learnin...

    2023

  49. [57]

    , year 1990

    author Nelson, T.O. , year 1990 . title Metamemory: A theoretical framework and new findings , in: booktitle Psychology of learning and motivation . publisher Elsevier . volume volume 26 , pp. pages 125--173

    1990

  50. [58]

    , year 2017

    author Panadero, E. , year 2017 . title A Review of Self -regulated Learning : Six Models and Four Directions for Research . journal Frontiers in Psychology volume 8

    2017

  51. [59]

    , author Jovanovic, J

    author Pardo, A. , author Jovanovic, J. , author Dawson, S. , author Ga s evi \'c , D. , author Mirriahi, N. , year 2019 . title Using learning analytics to scale the provision of personalised feedback . journal British Journal of Educational Technology volume 50 , pages 128--138

    2019

  52. [60]

    , year 2018

    author Pea, R.D. , year 2018 . title The social and technological dimensions of scaffolding and related theoretical concepts for learning, education, and human activity , in: booktitle Scaffolding . publisher Psychology Press , pp. pages 423--451

    2018

  53. [61]

    , year 2002

    author Prechelt, L. , year 2002 . title Early stopping-but when? , in: booktitle Neural Networks: Tricks of the trade . publisher Springer , pp. pages 55--69

    2002

  54. [62]

    , author Zhang, L.J

    author Qin, L. , author Zhang, L.J. , year 2019 . title English as a foreign language writers’ metacognitive strategy knowledge of writing and their writing performance in multimedia environments . journal Journal of Writing Research volume 11 , pages 393--413

    2019

  55. [63]

    , author Fan, Y

    author Rakovic, M. , author Fan, Y. , author Van Der Graaf, J. , author Singh, S. , author Kilgour, J. , author Lim, L. , author Moore, J. , author Bannert, M. , author Molenaar, I. , author Gasevic, D. , year 2022 . title Using learner trace data to understand metacognitive p...

    2022

  56. [64]

    , author Sheth, A

    author Rawte, V. , author Sheth, A. , author Das, A. , year 2023 . title A survey of hallucination in large foundation models . journal arXiv preprint arXiv:2309.05922

    2023 arXiv

  57. [65]

    , author Kovaleva, O

    author Rogers, A. , author Kovaleva, O. , author Rumshisky, A. , year 2021 . title A primer in bertology: What we know about how bert works . journal Transactions of the Association for Computational Linguistics volume 8 , pages 842--866

    2021

  58. [66]

    , author Tan, S

    author Rudolph, J. , author Tan, S. , author Tan, S. , year 2023 . title Chatgpt: Bullshit spewer or the end of traditional assessments in higher education? journal Journal of applied learning and teaching volume 6 , pages 342--363

    2023

  59. [67]

    , author Henderson, M

    author Ryan, T. , author Henderson, M. , author Ryan, K. , author Kennedy, G. , year 2023 . title Identifying the components of effective learner-centred feedback information . journal Teaching in Higher Education volume 28 , pages 1565--1582

    2023

  60. [68]

    , year 2023

    author Sangaré, M.J.P. , year 2023 . title Chatgpt is doomed . https://www.linkedin.com/pulse/chatgpt-doomed-mohamed-jean-philippe-sangar note accessed: 2024-06-12

    2023

  61. [69]

    , year 2021

    author Schiff, D. , year 2021 . title Out of the laboratory and into the classroom: the future of artificial intelligence in education . journal AI & society volume 36 , pages 331--348

    2021

  62. [70]

    , author Ifenthaler, D

    author Schumacher, C. , author Ifenthaler, D. , year 2021 . title Investigating prompts for supporting students' self-regulation--a remaining challenge for learning analytics approaches? journal The Internet and higher education volume 49 , pages 100791

    2021

  63. [71]

    , year 2023

    author Schunk, D.H. , year 2023 . title Self-regulation of self-efficacy and attributions in academic settings , in: booktitle Self-regulation of learning and performance . publisher Routledge , pp. pages 75--99

    2023

  64. [72]

    , author Greene, J.A

    author Schunk, D.H. , author Greene, J.A. , year 2017 . title Historical, contemporary, and future perspectives on self-regulated learning and performance , in: booktitle Handbook of self-regulation of learning and performance . publisher Routledge , pp. pages 1--15

    2017

  65. [73]

    , author Zimmerman, B

    author Schunk, D.H. , author Zimmerman, B. , year 2011 . title Handbook of self-regulation of learning and performance . publisher Taylor & Francis

    2011

  66. [74]

    , author Ma, W

    author Shi, L. , author Ma, W. , author Vosoughi, S. , year 2024 . title Judging the judges: A systematic investigation of position bias in pairwise comparative assessments by llms . journal arXiv preprint arXiv:2406.07791

    2024

  67. [75]

    , author Chen, H.C

    author Shih, K.P. , author Chen, H.C. , author Chang, C.Y. , author Kao, T.C. , year 2010 . title The development and implementation of scaffolding-based self-regulated learning system for e/m-learning . journal Journal of Educational Technology & Society volume 13 , pages 80--93

    2010

  68. [76]

    , author Ga s evi \'c , D

    author Siadaty, M. , author Ga s evi \'c , D. , author Hatala, M. , year 2016 a. title Associations between technological scaffolding and micro-level processes of self-regulated learning: A workplace study . journal Computers in Human Behavior volume 55 , pages 1007--1019

    2016

  69. [77]

    , author Ga s evi \'c , D

    author Siadaty, M. , author Ga s evi \'c , D. , author Hatala, M. , year 2016 b. title Measuring the impact of technological scaffolding interventions on micro-level processes of self-regulated workplace learning . journal Computers in Human Behavior volume 59 , pages 469--482

    2016

  70. [78]

    , author LMSYS ,

    author SkyLab, U.B. , author LMSYS , . title Chatbot arena . https://lmarena.ai/. note accessed: 2023-12-19

    2023

  71. [79]

    , author Ananthakrishnan, S

    author Soltan, S. , author Ananthakrishnan, S. , author FitzGerald, J. , author Gupta, R. , author Hamza, W. , author Khan, H. , author Peris, C. , author Rawls, S. , author Rosenbaum, A. , author Rumshisky, A. , et al., year 2022 . title Alexatm 20b: Few-shot learning using a...

    2022 arXiv

  72. [80]

    , author Bannert, M

    author Sonnenberg, C. , author Bannert, M. , year 2019 . title Using process mining to examine the sustainability of instructional support: How stable are the effects of metacognitive prompting on self-regulatory behavior? journal Computers in Human Behavior volume 96 , pages 259--272

    2019

  73. [81]

    , author Nadiri, A

    author Talebirad, Y. , author Nadiri, A. , year 2023 . title Multi-agent collaboration: Harnessing the power of intelligent llm agents . journal arXiv preprint arXiv:2306.03314

    2023 arXiv

  74. [82]

    , year 2020

    author Vosniadou, S. , year 2020 . title Bridging secondary and higher education. the importance of self-regulated learning . journal European Review volume 28 , pages S94--S103

    2020

  75. [83]

    , author Rose, C.P

    author Wang, Q. , author Rose, C.P. , author Ma, N. , author Jiang, S. , author Bao, H. , author Li, Y. , year 2022 . title Design and application of automatic feedback scaffolding in forums to promote learning . journal IEEE Transactions on Learning Technologies volume 15 , p...

    2022

  76. [84]

    , author Kamanda, A

    author Weyssow, M. , author Kamanda, A. , author Sahraoui, H. , year 2024 . title Codeultrafeedback: An llm-as-a-judge dataset for aligning large language models to coding preferences . journal arXiv preprint arXiv:2403.09032

    2024 arXiv

  77. [85]

    , author Hadwin, A

    author Winne, P. , author Hadwin, A. , year 1998 . title Studying as self-regulated learning. in, dj hacker, j. dunlosky, & ac graesser . journal Metacognition in educational theory and practice , pages 277--304

    1998

  78. [86]

    , year 1996

    author Winne, P.H. , year 1996 . title A metacognitive view of individual differences in self-regulated learning . journal Learning and individual differences volume 8 , pages 327--353

    1996

  79. [87]

    , year 2017

    author Winne, P.H. , year 2017 . title Cognition and metacognition within self-regulated learning , in: booktitle Handbook of self-regulation of learning and performance . publisher Routledge , pp. pages 36--48

    2017

  80. [88]

    , year 2023

    author Winne, P.H. , year 2023 . title Roles for information in trace data used to model self-regulated learning , in: booktitle Unobtrusive Observations of Learning in Digital Environments: Examining Behavior, Cognition, Emotion, Metacognition and Social Processes Using Learn...

    2023

  81. [89]

    , author Debut, L

    author Wolf, T. , author Debut, L. , author Sanh, V. , author Chaumond, J. , author Delangue, C. , author Moi, A. , author Cistac, P. , author Rault, T. , author Louf, R. , author Funtowicz, M. , author Davison, J. , author Shleifer, S. , author von Platen, P. , author Ma, C. ...

    2020

  82. [90]

    , author Baars, M

    author Wong, J. , author Baars, M. , author de Koning, B.B. , author Paas, F. , year 2021 . title Examining the use of prompts to facilitate self-regulated learning in massive open online courses . journal Computers in Human Behavior volume 115 , pages 106596

    2021

  83. [91]

    , author Bansal, G

    author Wu, Q. , author Bansal, G. , author Zhang, J. , author Wu, Y. , author Zhang, S. , author Zhu, E. , author Li, B. , author Jiang, L. , author Zhang, X. , author Wang, C. , year 2023 . title Autogen: Enabling next-gen llm applications via multi-agent conversation framewo...

    2023 arXiv

  84. [92]

    , author Chen, M

    author Xu, H. , author Chen, M. , author Huang, L. , author Vucetic, S. , author Yin, W. , year 2024 a. title X-shot: A unified system to handle frequent, few-shot and zero-shot learning simultaneously in classification . journal arXiv preprint arXiv:2403.03863

    2024 arXiv

  85. [93]

    , author Jain, S

    author Xu, Z. , author Jain, S. , author Kankanhalli, M. , year 2024 b. title Hallucination is inevitable: An innate limitation of large language models . journal arXiv preprint arXiv:2401.11817

    2024 arXiv

  86. [94]

    , author Greiff, S

    author Yan, L. , author Greiff, S. , author Teuber, Z. , author Ga s evi \'c , D. , year 2024 . title Promises and challenges of generative artificial intelligence for human learning . journal Nature Human Behaviour volume 8 , pages 1839--1850

    2024

  87. [95]

    , author Jin, H

    author Yang, J. , author Jin, H. , author Tang, R. , author Han, X. , author Feng, Q. , author Jiang, H. , author Zhong, S. , author Yin, B. , author Hu, X. , year 2024 . title Harnessing the power of llms in practice: A survey on chatgpt and beyond . journal ACM Transactions ...

    2024

  88. [96]

    , author Yu, C

    author Yang, K. , author Yu, C. , author Fung, Y.R. , author Li, M. , author Ji, H. , year 2023 . title Adept: A debiasing prompt framework , in: booktitle Proceedings of the AAAI conference on artificial intelligence , pp. pages 10780--10788

    2023

  89. [97]

    , year 2001

    author Yang, Y. , year 2001 . title A study of thresholding strategies for text categorization , in: booktitle Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval , pp. pages 137--145

    2001

  90. [98]

    , author Press, O

    author Zhang, M. , author Press, O. , author Merrill, W. , author Liu, A. , author Smith, N.A. , year 2023 a. title How language model hallucinations can snowball . journal arXiv preprint arXiv:2305.13534

    2023 arXiv

  91. [99]

    , author Li, C

    author Zhang, X. , author Li, C. , author Zong, Y. , author Ying, Z. , author He, L. , author Qiu, X. , year 2023 b. title Evaluating the performance of large language models on gaokao benchmark . journal arXiv preprint arXiv:2305.12474

    2023 arXiv

  92. [100]

    , author Li, Y

    author Zhang, Y. , author Li, Y. , author Cui, L. , author Cai, D. , author Liu, L. , author Fu, T. , author Huang, X. , author Zhao, E. , author Zhang, Y. , author Chen, Y. , et al., year 2023 c. title Siren's song in the ai ocean: a survey on hallucination in large language ...

    2023 arXiv

  93. [101]

    , author Wang, M

    author Zhang, Y. , author Wang, M. , author Ren, C. , author Li, Q. , author Tiwari, P. , author Wang, B. , author Qin, J. , year 2024 . title Pushing the limit of llm capacity for text classification . journal arXiv preprint arXiv:2402.07470

    2024 arXiv

  94. [102]

    , author Zhou, K

    author Zhao, W.X. , author Zhou, K. , author Li, J. , author Tang, T. , author Wang, X. , author Hou, Y. , author Min, Y. , author Zhang, B. , author Zhang, J. , author Dong, Z. , et al., year 2023 . title A survey of large language models . journal arXiv preprint arXiv:2303.18223

    2023 arXiv

  95. [103]

    , author Chiang, W.L

    author Zheng, L. , author Chiang, W.L. , author Sheng, Y. , author Zhuang, S. , author Wu, Z. , author Zhuang, Y. , author Lin, Z. , author Li, Z. , author Li, D. , author Xing, E. , et al., year 2024 . title Judging llm-as-a-judge with mt-bench and chatbot arena . journal Adv...

    2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.