Pith. sign in

REVIEW 4 major objections 3 minor 1 cited by

From First Draft to Final Insight: A Multi-Agent Approach for Feedback Generation

T0 review · 4 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A multi-agent generate-evaluate-regenerate loop can bring LLM-generated student feedback to near-uniform high quality, regardless of the first-round prompt method.

desk verdict A useful first-round comparison of six feedback prompt methods, but the headline improvement claims rest on an asymmetric evaluation where GPT-4o scores its own regenerated output. read the letter →

arxiv 2505.04869 v1 pith:XOTSV3DQ submitted 2025-05-08 cs.HC

Jie Cao , Chloe Qianhui Zhao , Xian Chen , Shuman Wang , Christian Schunn , Kenneth R. Koedinger , Jionghao Lin This is my paper · ORCID
classification cs.HC
keywords feedbackgenerationlargelanguagemodelsmulti-agentsystemslearner-centeredretrieval-augmentedchain-of-thoughtpromptingautomatedevaluationeducationaltechnology
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to show that a multi-agent 'generation, evaluation, and regeneration' (G-E-RG) process can turn first-round LLM feedback of varying quality into second-round feedback that nearly always contains the four learner-centered components regarded as effective. Across six combinations of prompt strategy and feedback framework, the share of feedback containing all four components rose from an average of 27.72% to an average of 98.49%, and evaluation accuracy improved by 3.36 to 12.98 percentage points, with most differences significant at p<0.001. If the claim holds, it gives instructors a practical route to high-quality, personalized feedback at scale without hand-editing every comment. The paper also positions the loop as a general quality-control mechanism that reduces the instability of single-round LLM generation.

What carries the argument

The load-bearing mechanism is a three-agent cycle: an LLM generates first-round feedback from a prompt that combines one of two prompting strategies (zero-shot or retrieval-augmented generation with chain-of-thought) with one of three frameworks (none, learner-centered, or knowledge-transmission); the same model class then evaluates that feedback using a rubric of reliability, four effectiveness components, five features, and word count, prompted with few-shot and chain-of-thought examples; finally, a third pass regenerates the feedback by direct instruction, feeding the evaluation-derived suggestions back in along with the question, student response, and first-round feedback. The rubric, grounded in a learner-centered feedback framework, is what turns regeneration from a generic rewrite into a targeted correction of specific missing components. A notable fixed point is that retrieved slides are decided before the second round, so retrieval errors persist into the regenerated feedback.

What would settle it

Have two independent trained annotators blindly code the second-round feedback with the same rubric used for the first round, then compare those human-coded scores to the human-coded first-round scores. If the share of feedback containing all four effective components does not rise to near 98%, or if the improvement shrinks dramatically when a different evaluator model or a human judge is substituted, the G-E-RG claim would be falsified.

Watch

Extended reading notes

Core claim

The central discovery is that an iterative pipeline in which one LLM writes feedback, a second evaluates it against a structured rubric, and a third rewrites it using the evaluation results yields feedback that is substantially and consistently better than the initial draft. The authors report that, regardless of whether the first round used zero-shot prompting or retrieval-augmented generation with chain-of-thought, and regardless of which feedback framework was embedded, the regenerated feedback achieved high coverage of all four effective components (critiques, strengths, actionable advice, and encouragement of agency), with 96.63%–99.52% of second-round samples containing all four. They further report that automatic evaluation accuracy rose by 3.36 to 12.98 percentage points for the six methods, that feature scores improved on most sub-dimensions, and that longer feedback was condensed for several methods. The paper concludes that this loop transforms even weak initial feedback into high-quality feedback in the second round, though it notes that some sub-dimensions (e.g., strengthening the teacher-student relationship, usability, and independence) still need further work.

Load-bearing premise

The load-bearing premise is that second-round quality measured by the same model that produced the regeneration suggestions, checked only lightly by research assistants, is comparable to first-round quality that was fully coded by trained human annotators; if the model evaluator is biased toward its own revised output, the reported gains are inflated.

Editorial extensions

If this is right

  • Instructors could deploy a fully automated pipeline that starts from almost any first-round feedback method and still ends with feedback that reliably contains all four learner-centered components.
  • Human effort can shift from writing or rewriting each comment to spot-checking edge cases, because the loop supplies its own quality control.
  • Because the gains appeared across baseline and RAG_CoT methods, the regeneration phase may matter more than the initial prompt design, which would simplify deployment.
  • The pipeline costs three LLM calls per student response, so its practical value depends on whether the measured quality gain justifies the added compute.
  • Some sub-dimensions such as teacher-student relationship, usability, and independence remain below ceiling, so the method is not yet a complete substitute for human feedback on those aspects.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the gains are real rather than evaluator self-preference, the same generate-evaluate-regenerate loop could be applied to other rubric-scorable LLM writing tasks—personalized explanations, peer-review comments, or clinical notes—with a similar expectation of reduced variance across prompt strategies.
  • A direct test is to replace the second-round evaluator with a different model or blinded human coders; if the quality jump largely persists, that isolates the regeneration step as the cause, and if it vanishes, the reported effect is an evaluation artifact.
  • The paper's own observation that retrieved slides are fixed in the second round suggests a natural extension: make retrieval part of the regeneration loop so retrieval errors do not propagate.
  • The cost-quality frontier is worth measuring: a single evaluation-regeneration pass might be the practical recipe, with additional passes yielding diminishing returns.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper proposes a multi-agent "generation, evaluation, regeneration" (G-E-RG) pipeline built on GPT-4o to improve automated feedback quality. First-round feedback is generated with six combinations of prompt strategy (zero-shot vs. RAG_CoT) and feedback framework (none, learner-centered, knowledge-transmission) for 208 student responses from a graduate course; 1,248 feedback items are human-coded. A GPT-4o evaluator then scores the feedback, its scores are converted into regeneration suggestions, and a third GPT-4o agent regenerates the feedback. The paper claims that the second round significantly improves evaluation accuracy (by 3.36% to 12.98%), raises the proportion of feedback containing all four effective components from 27.72% to 98.49% on average, improves most feature scores, and in some conditions reduces verbosity. The Discussion concludes that, regardless of initial feedback quality, the G-E-RG process can transform feedback into high-quality feedback.

Significance. The problem is well motivated: scaling high-quality, learner-centered feedback is practically important, and the paper compares several prompt/framework combinations in a realistic course dataset. The first-round human-coding procedure is a genuine strength, including iterative coder training and reported Cohen's kappa values, and Table 4 provides useful calibration data on how GPT-4o compares with human coders. The comparative first-round analysis (RQ1) and the LLM-evaluation-accuracy analysis (RQ2) could be informative if presented carefully. However, the central claim—that regeneration improves feedback quality—is not supported by the measurements as reported, because the second round is evaluated by the same model family that produced the regeneration suggestions, and the abstract and Table 5 contain specific errors. If the measurement asymmetry and arithmetic issues were resolved, this could become a useful empirical contribution; in its current form, the paper does not establish RQ3.

major comments (4)
  1. [§3.3 (Feedback Re-Generation in the Second Round)] The central comparison is invalid as reported because first-round quality is measured by two trained human coders (Cohen's kappa 0.751–0.962), while second-round quality is measured by GPT-4o, the same model family that produced the regeneration suggestions; the 'research assistants check' is mentioned without any counts, correction statistics, or reliability estimates. Table 4 shows that GPT-4o's agreement with human coders on the first-round data is only F1=0.75 for F2-usable and F1=0.73 for F5-independence, and the Discussion itself concedes that automated evaluation is less reliable for F2-usable. Consequently, the headline gains (e.g., all-four-components rising from 27.72% to 98.49% in Section 4.3/Fig. 4) compare a human-coded baseline with a self-assessed LLM outcome and are not a valid support for RQ3.
  2. [Abstract and §4.3] The abstract states that evaluation accuracy increased by 3.36% to 12.98% with p<0.001 for six methods, but §4.3 reports that the RAG_CoT_knowledge improvement of 3.36% is non-significant (p=0.07). The significance claim must be corrected, and the abstract should not imply that all six methods improved significantly.
  3. [Table 5] Table 5 reports RAG_CoT_none as improving from 89.42% to 97.12% with an increase of +2.89%, but the arithmetic difference is +7.70 percentage points (the +2.89% value appears to be a copy of the RAG_CoT_knowledge row). This error affects the reported range and must be fixed before any quantitative conclusion is drawn from the table.
  4. [Section 4.3 (component and feature comparisons)] Even apart from the human/LLM asymmetry, the regeneration prompt is constructed from the same rubric that GPT-4o uses to score the second-round output, so the near-ceiling component coverage (99.04%–100% and 96.63%–99.52% all-four-components) is largely a measure of instruction-following with respect to the evaluator's own rubric, not independent evidence of pedagogical quality. A human-coded evaluation of the second-round feedback, or at least a validation of the LLM judge against human codes on second-round data, is necessary to support the claim that the G-E-RG process 'can be transformed into high-quality feedback.'
minor comments (3)
  1. [Table 6] The column labeled 'Mean' is not defined; from the 'Mean change' values it appears to be the second-round mean word count, but the first-round means are not shown, making the comparison hard to verify. Also, 'Pair-t test' should be 'paired t-test'.
  2. [§3.3 and throughout] The manuscript repeatedly refers to a 'Digital Appendix' (rubrics, prompts, detailed statistics), but no appendix or supplementary material is included in the arXiv deposit; without it, the methods are not fully reproducible.
  3. [§4.3] The p-value notation in the components paragraph ('p < 0.001 ; p = 0 .014∗') omits which methods each p-value applies to and has inconsistent spacing; present a table or label each method explicitly.

Circularity Check

2 steps flagged · score 7.0 of 10

Second-round feedback is evaluated by the same GPT-4o model whose rubric-based suggestions drove regeneration, so the headline improvement is largely a self-evaluation artifact.

  1. self definitional [Section 3.3, 'Feedback Re-Generation in the Second Round'; Abstract and Section 4.3]
    "After evaluating the feedback generated in the first round, this study decoded the evaluation results into suggestions, then put suggestions, questions, students' responses, and first-round feedback into another GPT-4o model to re-generate feedback. ... Finally, after feedback re-generation, to assess the quality of feedback in the second round, this study employed the GPT-4o model to automatically evaluate feedback, and let research assistants check the results."

    The second-round output is produced from the evaluator's own rubric-derived suggestions and then scored by the same model family using the same rubric, while the first-round baseline was fully human-coded. The claimed jump in all-four-components feedback from 27.72% to 98.49% therefore mostly measures the regenerator's compliance with the evaluator's suggestions, not an independent quality gain. The paper itself reports low LLM-human agreement on F2-usable (F1=0.75) and F5-independence (F1=0.73) and concedes automated evaluation is less reliable on such features, yet uses that same automated evaluation for the round-two feature claims.

  2. self definitional [Section 3.3, 'Feedback Rubric and Evaluation'; Section 5 Discussion]
    "we adopted four components and five features to evaluate the effectiveness of feedback, following the learner-centered feedback framework proposed by [26] ... feedback generated using a learner-centered approach exhibits more comprehensive quality in the first round. This advantage likely stems from our evaluation rubric being rooted in learner-centered principles."

    The generation frameworks compared include the learner-centered framework, and the evaluation rubric is an operationalization of that same learner-centered framework. Thus the finding that learner-centered feedback is 'more comprehensive' is partly guaranteed by the measurement instrument: the rubric rewards exactly what the learner-centered prompt is designed to produce. The paper explicitly attributes the advantage to the rubric's roots in learner-centered principles, making this a secondary input-output circularity rather than an external benchmark result.

full rationale

The central G-E-RG improvement claim rests on asymmetric measurement. First-round feedback (1,248 items) was coded by two trained human annotators with Cohen's kappa 0.751-0.962, while second-round feedback was automatically evaluated by GPT-4o, the same model family whose evaluation suggestions were fed into the regenerator, with only an unquantified 'research assistants check' of the results. The abstract's headline gain in all-four-components coverage (27.72% to 98.49%) therefore substantially measures the model's compliance with its own rubric, and the feature-level gains rest on the same automated evaluator despite the paper's own concession that automated evaluation is less reliable on features such as F2-usable. This is partial circularity, not total: the first-round method comparisons are human-coded, simplicity is computed objectively by word count, and a future independent human coding of round two could validate the pipeline. A secondary self-definitional step is the learner-framework comparison, since the evaluation rubric is derived from the learner-centered framework used as a generation condition and the paper attributes that condition's advantage to the rubric itself. No load-bearing self-citation chain is present; the numeric inconsistencies (e.g., abstract says p<0.001 for all methods while Section 4.3 reports p=0.07 for RAG_CoT_knowledge; Table 5's +2.89% does not match 89.42% to 97.12%) are correctness concerns rather than circularity evidence.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no fitted numerical parameters; the analysis compares six fixed prompt/framework conditions. The quality rubric is imported from prior educational theory, which is a domain assumption rather than a measured quantity. The main hidden dependence is on the validity of the GPT-4o evaluator for second-round feedback and on the equation of word count with simplicity.

assumptions (4)
  • domain assumption The learner-centered feedback framework from Ryan et al. (2021) defines high-quality feedback.
    The rubric's components and features are adopted from this framework, and all quality claims are cast in its terms (Section 3.2, Table 1).
  • domain assumption GPT-4o can judge feedback quality according to the rubric well enough to serve as the second-round evaluator.
    Second-round quality is measured by the LLM itself; the authors acknowledge imperfect reliability for F2 and F5 in Section 4.2 and Section 5.
  • domain assumption Word count is an adequate proxy for simplicity.
    Simplicity is defined solely by word count in Table 1 and Section 3.3, with lower word counts treated as more concise.
  • domain assumption Retrieved slides from the course are the correct knowledge source and can remain fixed across rounds.
    Regeneration reuses the same retrieved slides; the paper notes this prevents improving retrieved-slide accuracy (Section 3.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of From First Draft to Final Insight: A Multi-Agent Approach for Feedback Generation." pith.science (2026). https://pith.science/paper/XOTSV3DQ

@misc{pith2026250504869,
  author       = {Pith},
  title        = {Pith review of: From First Draft to Final Insight: A Multi-Agent Approach for Feedback Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XOTSV3DQ}},
  note         = {Machine review of arXiv:2505.04869}
}
read the original abstract

Producing large volumes of high-quality, timely feedback poses significant challenges to instructors. To address this issue, automation technologies-particularly Large Language Models (LLMs)-show great potential. However, current LLM-based research still shows room for improvement in terms of feedback quality. Our study proposed a multi-agent approach performing "generation, evaluation, and regeneration" (G-E-RG) to further enhance feedback quality. In the first-generation phase, six methods were adopted, combining three feedback theoretical frameworks and two prompt methods: zero-shot and retrieval-augmented generation with chain-of-thought (RAG_CoT). The results indicated that, compared to first-round feedback, G-E-RG significantly improved final feedback across six methods for most dimensions. Specifically:(1) Evaluation accuracy for six methods increased by 3.36% to 12.98% (p<0.001); (2) The proportion of feedback containing four effective components rose from an average of 27.72% to an average of 98.49% among six methods, sub-dimensions of providing critiques, highlighting strengths, encouraging agency, and cultivating dialogue also showed great enhancement (p<0.001); (3) There was a significant improvement in most of the feature values (p<0.001), although some sub-dimensions (e.g., strengthening the teacher-student relationship) still require further enhancement; (4) The simplicity of feedback was effectively enhanced (p<0.001) for three methods.

Figures

Figures reproduced from arXiv: 2505.04869 by the authors.

Figure 1
Figure 1. The “G-E-RG” framework for feedback generation Feedback Generation in the First Round Our study used a GPT-4o model to generate feedback in the first round. We put the quiz questions and corresponding students’ responses from the course into the system along with prompts, which were combined with different prompt strategies: Zero-shot (as the baseline) and RAG_CoT and different feedback frameworks: no framework, lea… view at source ↗
Figure 2
Figure 2. The average value of five features of feedback generated by different methods and Baseline_learner method emerge as the more balanced method for generat￾ing feedback that keeps effective features, but there’s also a need to strengthen the other four features, in addition to the F4-dialog. Regarding simplicity, as [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. The word count (simplicity) of feedback generated by different methods shown in [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Comparison of the percentage of components in feedback generated in the 1st and 2nd rounds In terms of the comparison of components dimension, [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Count and percentage distribution of the value increases (by 1/2) for five fea￾tures in the 2nd-round feedback under three methods [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Automatic Large Language Models Creation of Interactive Learning Lessons

    cs.CY 2025-06 conditional novelty 6.0 of 10

    GPT-4o with retrieval-augmented generation produces higher-rated tutor training lessons when lesson creation is split into three segments rather than one step, though references remain unreliable.

Reference graph

Works this paper leans on

35 extracted references · 23 canonical work pages · cited by 1 Pith paper

  1. [1]

    Studies in Higher Education pp

    Carless, D., Young, S.: Towards an idealized model of feedback uptake: longitudinal development of feedback literacy from a social constructivist perspective. Studies in Higher Education pp. 1–13 (2024)

    work page 2024

  2. [2]

    Computers and Education: Artificial Intelligence2, 100027 (2021)

    Cavalcanti, A.P., Barbosa, A., Carvalho, R., Freitas, F., Tsai, Y.S., Gašević, D., Mello, R.F.: Automatic feedback in online learning environments: A systematic lit- erature review. Computers and Education: Artificial Intelligence2, 100027 (2021)

    2021

  3. [3]

    In: Olney, A.M., Chounta, I.A., Liu, Z., Santos, O.C., Bittencourt, I.I

    Chen, S., Lan, Y., Yuan, Z.: A multi-task automated assessment system for essay scoring. In: Olney, A.M., Chounta, I.A., Liu, Z., Santos, O.C., Bittencourt, I.I. (eds.) AIED2024. pp. 276–283. Springer Nature Switzerland, Cham (2024) A Multi-Agent Approach for Feedback Generation 13

    work page 2024

  4. [4]

    Computers and Education: Artificial Intelligence7, 100299 (2024)

    Dai, W., Tsai, Y.S., Lin, J., Aldino, A., Jin, H., Li, T., Gašević, D., Chen, G.: As- sessing the proficiency of large language models in automatic feedback generation: An evaluation study. Computers and Education: Artificial Intelligence7, 100299 (2024)

    2024

  5. [5]

    The Leadership Quarterly33(6), 101534 (2022)

    Drouvelis, M., Paiardini, P.: Feedback quality and performance in organisations. The Leadership Quarterly33(6), 101534 (2022)

    work page 2022

  6. [6]

    Teaching and Teacher Education110, 103578 (2022)

    Dunbar, K., Yadav, A.: Shifting to student-centered learning: Influences of teaching a summer service learning program. Teaching and Teacher Education110, 103578 (2022)

    work page 2022

  7. [7]

    Education and Information Technologies pp

    Fokides, E., Peristeraki, E.: Comparing chatgpt’s correction and feedback com- ments with that of educators in the context of primary students’ short essays written in english and greek. Education and Information Technologies pp. 1–45 (2024)

    work page 2024

  8. [8]

    In: Proceedings of the Eleventh ACM Conference on Learning@ Scale

    Fung, S.C.E., Wong, M.F., Tan, C.W.: Automatic feedback generation on k-12 students’ data science education by prompting cloud-based large language models. In: Proceedings of the Eleventh ACM Conference on Learning@ Scale. pp. 255–258 (2024)

    work page 2024

Show all 35 references
  1. [9]

    Gabbay, H., Cohen, A.: Combining llm-generated and test-based feedback in a moocforprogramming.In:ProceedingsoftheEleventhACMConferenceonLearn- ing@ Scale. pp. 177–187 (2024)

    2024

  2. [10]

    Teaching and Learning in Nursing19(2), e358–e367 (2024)

    Goktas, P., Kucukkaya, A., Karacay, P.: Utilizing gpt 4.0 with prompt learning in nursing education: A case study approach based on benner’s theory. Teaching and Learning in Nursing19(2), e358–e367 (2024)

    2024

  3. [11]

    Sports Economics Review8, 100045 (2024)

    Goller, D., Späth, M.: ‘good job!’the impact of positive and negative feedback on performance. Sports Economics Review8, 100045 (2024)

    2024

  4. [12]

    Education and Information Technologies 29(7), 8435–8463 (2024)

    Guo, K., Wang, D.: To resist it or to embrace it? examining chatgpt’s potential to support teacher feedback in efl writing. Education and Information Technologies 29(7), 8435–8463 (2024)

    2024

  5. [13]

    Review of educational research 77(1), 81–112 (2007)

    Hattie, J., Timperley, H.: The power of feedback. Review of educational research 77(1), 81–112 (2007)

    2007

  6. [14]

    JMIR Medical Education 10(1), e59213 (2024)

    Holderried, F., Stegemann-Philipps, C., Herrmann-Werner, A., Festl-Wietek, T., Holderried, M., Eickhoff, C., Mahling, M., et al.: A language model–powered simu- lated patient with automated feedback for history taking: Prospective study. JMIR Medical Education 10(1), e59213 (2024)

    2024

  7. [15]

    arXiv preprint arXiv:2312.13010 (2023)

    Huang, D., Bu, Q., Zhang, J.M., Luck, M., Cui, H.: Agentcoder: Multi-agent- based code generation with iterative testing and optimisation. arXiv preprint arXiv:2312.13010 (2023)

    2023 arXiv

  8. [16]

    Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., Liu, T.: A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions43(2) (Jan 2025)

    2025

  9. [17]

    Journal of Educational Technology & Society26(2) (2023)

    Hwang,G.J.,Chen,N.S.:Exploringthepotentialofgenerativeartificialintelligence in education: applications, challenges, and future research directions. Journal of Educational Technology & Society26(2) (2023)

    2023

  10. [18]

    Psychological bulletin 119(2), 254 (1996)

    Kluger, A.N., DeNisi, A.: The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological bulletin 119(2), 254 (1996)

    1996

  11. [19]

    In: Proceedings of the 2024 on Innovation and Tech- nology in Computer Science Education V

    Koutcheme, C., Dainese, N., Sarsa, S., Hellas, A., Leinonen, J., Denny, P.: Open source language models can provide feedback: Evaluating llms’ ability to help stu- dents using gpt-4-as-a-judge. In: Proceedings of the 2024 on Innovation and Tech- nology in Computer Science Educ...

    2024

  12. [20]

    Advances in Neural Information Processing Systems 33, 9459–9474 (2020)

    Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.t., Rocktäschel, T., et al.: Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems 33, 9459–9474 (2020)

    2020

  13. [21]

    In: Olney, A.M., Chounta, I.A., Liu, Z., Santos, O.C., Bittencourt, I.I

    Liang, Z., Sha, L., Tsai, Y.S., Gašević, D., Chen, G.: Towards the automated gen- eration of readily applicable personalised feedback in education. In: Olney, A.M., Chounta, I.A., Liu, Z., Santos, O.C., Bittencourt, I.I. (eds.) AIED2024. pp. 75–88. Springer Nature Switzerland,...

    2024

  14. [22]

    Nguyen, H., Stott, N., Allan, V.: Comparing feedback from large language models andinstructors:Teachingcomputerscienceatscale.In:ProceedingsoftheEleventh ACM Conference on Learning@ Scale. pp. 335–339 (2024)

    2024

  15. [23]

    arXiv preprint arXiv:2302.04662 (2023)

    Phung, T., Cambronero, J., Gulwani, S., Kohn, T., Majumdar, R., Singla, A., Soares, G.: Generating high-precision feedback for programming syntax errors us- ing large language models. arXiv preprint arXiv:2302.04662 (2023)

    2023 arXiv

  16. [24]

    Assessment & Evaluation in Higher Education33(2), 143–154 (2008)

    Poulos, A., Mahony, M.J.: Effectiveness of feedback: The students’ perspective. Assessment & Evaluation in Higher Education33(2), 143–154 (2008)

    2008

  17. [25]

    Behavioral science 28(1), 4–13 (1983)

    Ramaprasad, A.: On the definition of feedback. Behavioral science 28(1), 4–13 (1983)

    1983

  18. [26]

    Assessment & Evaluation in Higher Education46(6), 894–912 (2021)

    Ryan, T., Henderson, M., Ryan, K., Kennedy, G.: Designing learner-centred text- based feedback: a rapid review and qualitative synthesis. Assessment & Evaluation in Higher Education46(6), 894–912 (2021)

    2021

  19. [27]

    arXiv preprint arXiv:2402.07927 (2024)

    Sahoo, P., Singh, A.K., Saha, S., Jain, V., Mondal, S., Chadha, A.: A systematic survey of prompt engineering in large language models: Techniques and applica- tions. arXiv preprint arXiv:2402.07927 (2024)

    2024 arXiv

  20. [28]

    arXiv preprint arXiv:2310.10076 (2023)

    Saito, K., Wachi, A., Wataoka, K., Akimoto, Y.: Verbosity bias in preference la- beling by large language models. arXiv preprint arXiv:2310.10076 (2023)

    2023 arXiv

  21. [29]

    In: ICAART (2)

    da Silva, D.C.A., de Mello, C.E., Garcia, A.C.B.: Analysis of the effectiveness of large language models in assessing argumentative writing and generating feedback. In: ICAART (2). pp. 573–582 (2024)

    2024

  22. [30]

    chatgpt is the companion, not enemies

    Teng, M.F.: “chatgpt is the companion, not enemies”: Efl learners’ perceptions and experiences in using chatgpt for feedback in writing. Computers and Education: Artificial Intelligence 7, 100270 (2024)

    2024

  23. [31]

    Thinking Skills and Creativity51, 101440 (2024)

    Wang, L., Chen, X., Wang, C., Xu, L., Shadiev, R., Li, Y.: Chatgpt’s capabilities in providing feedback on undergraduate students’ argumentation: A case study. Thinking Skills and Creativity51, 101440 (2024)

    2024

  24. [32]

    Advances in neural information processing systems35, 24824–24837 (2022)

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V., Zhou, D., et al.: Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems35, 24824–24837 (2022)

    2022

  25. [33]

    Frontiers in psychology 10, 487662 (2020)

    Wisniewski, B., Zierer, K., Hattie, J.: The power of feedback revisited: A meta- analysis of educational feedback research. Frontiers in psychology 10, 487662 (2020)

    2020

  26. [34]

    arXiv preprint arXiv:2308.08155 (2023)

    Wu, Q., Bansal, G., Zhang, J., Wu, Y., Zhang, S., Zhu, E., Li, B., Jiang, L., Zhang, X., Wang, C.: Autogen: Enabling next-gen llm applications via multi-agent conversation framework. arXiv preprint arXiv:2308.08155 (2023)

    2023 arXiv

  27. [35]

    arXiv preprint arXiv:2409.03512 (2024)

    Yu, J., Zhang, Z., Zhang-li, D., Tu, S., Hao, Z., Li, R.M., Li, H., Wang, Y., Li, H., Gong, L., et al.: From mooc to maic: Reshaping online teaching and learning through llm-driven agents. arXiv preprint arXiv:2409.03512 (2024)

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.