Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Fine-tuning small language models on student-written feedback makes automated programming feedback shorter, more peer-like, and more often correct than prompt engineering alone.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Fine-tuning small open language models on student-written feedback produces shorter and more accurate feedback than prompt engineering on the same 30 programs, but the test set overlaps the training data.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection The dataset and flipped-role task are genuinely useful, but the central accuracy claim is built on an evaluation that overlaps the fine-tuning data, so generalization is not yet shown. the 3 major comments →

arxiv 2509.10647 v1 pith:NJ4TCBOS submitted 2025-09-12 cs.CY

Humanizing Automated Programming Feedback: Fine-Tuning Generative Models with Student-Written Feedback

Victor-Alexandru P\u{a}durean , Tung Phung , Nachiket Kotalwar , Michael Liut , Juho Leinonen , Paul Denny , Adish Singla This is my paper
classification cs.CY
keywords learnersourcingfine-tuningprogramming feedbackgenerative AIflipped-role taskstudent-written feedbackLlama3Phi3
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that automated programming feedback can be made more human by training models on feedback that students themselves wrote, instead of relying on carefully engineered prompts. The authors collected about 1,900 tutor-style comments from introductory students, fine-tuned two open-access language models (Llama3-8B and Phi3-3.8B) on those comments, and compared the output with the same models prompted in the standard way. Their central claim is that fine-tuning makes the feedback shorter and closer in style to what peers write, and at the same time more often correct—even though some of the student comments used for training were wrong. If that holds, it offers a low-cost path to personalized, peer-like feedback at scale.

Core claim

The paper's central finding is that supervised fine-tuning on learnersourced feedback simultaneously improves two things that are usually traded off: style alignment and correctness. In their experiments, the fine-tuned Llama3-8B produced correct feedback on 86.7% of the evaluated buggy programs, compared with 71.7% for the same model using an expert-engineered prompt, and the fine-tuned Phi3-3.8B reached 88.3% versus 80.0%. The generated feedback also moved toward the student profile—around 46 words and 2–3 sentences, with fixes suggested roughly half the time and bugs located by variable name rather than line number. The authors interpret this as evidence that learnersourced data can repla

What carries the argument

The mechanism is a flipped-role learnersourcing pipeline. Students are given a buggy program, must first identify a failing test case and see a corrected version, and then write feedback as if they were tutors; those comments form the fine-tuning corpus. The style and correctness of both student and AI feedback are then measured with a five-attribute rubric: correctness, word count, sentence count, whether a fix is given, and whether variables or line numbers are mentioned. The argument is carried by the comparison between fine-tuned and prompted outputs on the same 30 buggy programs, with the fine-tuned models matching the student profile while exceeding the engineered-prompt baseline in co

Load-bearing premise

The load-bearing premise is that the higher correctness measured on the 30 buggy programs reflects generalizable feedback skill rather than memorization of those same programs, which also supplied the fine-tuning data.

What would settle it

Evaluate the fine-tuned models on buggy programs that were held out from fine-tuning—either new bugs for the same three problems or entirely new problems—and compare correctness against the engineered-prompt baseline. If the fine-tuned advantage disappears or reverses, the reported gain is task-specific memorization.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Fine-tuned small models can generate student-style feedback without hand-crafted prompts, lowering the engineering cost of automated tutoring systems.
  • Because accuracy improved despite incorrect training labels, learnersourced corpora do not need to be expert-filtered to be useful for fine-tuning.
  • The method provides a way to collect large feedback datasets that reflect the language of a specific student population, enabling course- or context-specific models.
  • Feedback produced this way tends to name variables rather than line numbers, matching how students localize bugs and potentially making hints more digestible to novices.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment the paper does not run is filtering the training corpus to only correct student feedback; the current design cannot separate the benefit of student style from the effect of removing incorrect labels.
  • Because the evaluation programs are the same 30 tasks whose feedback formed the training set, the reported accuracy advantage should be read as in-distribution; a held-out-program test would clarify whether the gain transfers to new bugs.
  • The flipped-role collection protocol—requiring a failing test case before writing feedback—could generalize to other subjects where students are asked to explain errors, and might itself improve the quality of learnersourced explanations.
  • If the accuracy gain comes mostly from shorter, simpler output, then fine-tuning on any concise feedback corpus—not necessarily student-written—might reproduce part of the effect; the paper does not test this alternative.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces a learnersourcing pipeline for programming feedback: students act as tutors and write feedback on buggy C programs; the collected texts are used to fine-tune open-weight LLMs (Llama3-8B, Phi3-3.8B). The authors analyze 300 student-written feedback instances on three attributes (correctness, length, bug localization style), then compare prompt-engineered baselines with fine-tuned models on the same set of buggy programs. They claim that fine-tuning improves both stylistic alignment with student feedback and correctness relative to prompt engineering alone. The paper releases its fine-tuning code and uses real student data, which are strengths. However, the central accuracy claim is not supported as stated because the evaluation uses the very same 30 buggy programs on which the models were fine-tuned, and the sample is too small for the reported percentage differences to be statistically meaningful.

Significance. If the accuracy improvement survived a held-out evaluation, this would be a valuable contribution to computing education: it would show that modest amounts of learnersourced text can replace expert prompt engineering for feedback generation, while producing more student-like output. The flipped-role collection design, the public release of the fine-tuning implementation, and the use of open-access models are concrete assets. The style-alignment results are suggestive, and the descriptive analysis of student feedback is interesting. But the current evidence does not establish the headline claim of improved accuracy, and the style claims are also evaluated in-distribution. With a held-out evaluation and appropriate statistical reporting, the paper could be a solid empirical contribution.

major comments (3)
  1. [§3.2 and §4.2, Table 2] The central accuracy claim is compromised by train/evaluation overlap. Section 3.2 states that the fine-tuning corpus (1,903 instances) was collected on exactly 30 tasks: 3 problems × 10 buggy programs. Section 4.2 and Table 2 evaluate all models on the same 10 buggy programs per problem. Thus the fine-tuned Llama3-8B and Phi3-3.8B have seen multiple student-written feedback examples for each exact (problem, buggy program) pair during training, while the prompt-engineered baselines have not. The reported gains (86.7% and 88.3% vs. 71.7% and 80.0%) can therefore be explained by memorization or near-copying of training targets. The abstract's claim that fine-tuning 'improves accuracy' is not established for unseen buggy programs. A held-out set of buggy programs, or task-level cross-validation, is required before this claim can be made.
  2. [§4.2, Table 2] The evaluation uses only 30 outputs per model (10 programs × 3 problems), with no confidence intervals, error bars, or significance tests. For binary correctness, a 15-percentage-point gap based on 30 instances (e.g., 26/30 vs. 21/30) has widely overlapping confidence intervals, so the observed differences are statistically fragile. Report exact binomial confidence intervals, bootstrap resampling, or formal tests, and interpret the results accordingly. In addition, Section 3.4 says two experts annotated each AI-generated instance, but no inter-rater reliability (kappa) is reported for that annotation; this should be provided.
  3. [§3.3, §4.2, Table 2] The style-alignment conclusion is also evaluated on the training distribution. The fine-tuned models are trained to imitate student-written feedback and are then compared to student-written feedback on the same 30 tasks. While the style metrics (length, sentence count, mentions of variables/lines) are more robust to memorization than correctness is, the claim that fine-tuned feedback 'better matches' student style would be stronger with held-out tasks or at least with a clear statement that the reported numbers are in-distribution. As written, the comparison partly measures how well the model reproduces its training texts rather than a general stylistic shift.
minor comments (4)
  1. [Figure 1 caption] The caption asserts that the engineered-prompt feedback 'contains redundant information that may confuse the student.' This is a subjective interpretation; the caption should be neutral and let the example speak for itself.
  2. [Table 1] The differences between Understanding=1 and Understanding=0 rows are discussed as substantive findings, but no significance tests are reported. Given the modest sample sizes per cell (e.g., 26, 47, 51), the authors should avoid causal language without statistical support.
  3. [Footnote 1] The footnote explains that the same fixed code was used for all models to enable comparison. It would help to state explicitly whether the fixed code is the same across models but different across the 30 tasks, and whether it was identical to the fixed code shown to students during data collection.
  4. [Section 5] The limitations paragraph acknowledges the lack of classroom deployment and the fixed-code condition, but it does not acknowledge the train/eval overlap, which is the most serious threat to the accuracy claim. This should be added.

Circularity Check

1 steps flagged

Fine-tuned models are evaluated on the same 30 buggy programs used to collect their training feedback, so the reported accuracy gain over prompt engineering is an in-sample result, not a held-out prediction.

specific steps
  1. fitted input called prediction [Section 3.2 (dataset), Section 3.3 (fine-tuning), Section 4.2 / Table 2 (evaluation)]
    "To prepare tasks, we started with 3 programming problems and 10 buggy programs for each. ... In total, we obtained 30 feedback-writing tasks. ... We assessed the models for 10 buggy programs for each of the 3 problems."

    The 1,903 fine-tuning instances are student feedback on these exact 30 tasks. Table 2 evaluates fine-tuned and baseline models on the same 10 buggy programs per problem, so the fine-tuned models have seen training targets for every evaluation input. The reported correctness gain (e.g., Llama3-8B-fine-tuned 86.7% vs. engineered 71.7%) is a resubstitution comparison, not a held-out prediction; memorizing the correct fix for each input would suffice. The style-alignment claim is similarly measured on the training distribution. This makes the central 'improves accuracy' claim, as stated, an in-sample result rather than evidence of generalization.

full rationale

The central empirical claim—fine-tuning on learnersourced data improves accuracy—is supported only by Table 2, whose evaluation set is exactly the set of tasks used to build the fine-tuning corpus. Section 3.2 defines 30 feedback-writing tasks (3 problems × 10 buggy programs); Section 3.3 filters 1,920 collected instances to 1,903 and fine-tunes on them; Table 2 evaluates on the same 10 buggy programs per problem. Thus the fine-tuned models have seen multiple student-written feedback examples for each exact (problem, buggy program) pair, while the prompt-engineered baselines have not seen any. The accuracy gap can be explained by memorization or near-copying of training targets, and does not demonstrate a general ability to produce correct feedback. The style-alignment results are subject to the same overlap: matching statistics of the training texts on the training tasks is close to the training objective. No external or held-out benchmark is used. The findings may still be valid as an in-distribution demonstration, but the abstract's general claim overreaches. No self-citation chain or uniqueness theorem is load-bearing, so the circularity is partial and localized to the evaluation design.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

The paper's central claims are empirical and do not depend on fitted mathematical constants, but they rest on assumptions about dataset representativeness and annotation reliability. No new entities are postulated.

axioms (3)
  • domain assumption Expert rubric annotations are a valid measure of feedback correctness and style
    Two experts annotated with Cohen's kappa 0.63 on 30 instances, then one expert annotated the remaining 270 student samples; the AI feedback was annotated by two experts without a reported agreement statistic.
  • domain assumption The 10 buggy programs per problem are representative of the target distribution of programming bugs
    Evaluation and fine-tuning both use these 30 tasks; no held-out buggy programs or problems are used, so generalization to other programs is assumed.
  • domain assumption Providing failing test cases and fixed code to both students and models creates comparable conditions
    Section 3.3 gives the same fixed code and failing test case to all models; Section 3.2 shows students the fixed code after the pre-feedback activity.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Humanizing Automated Programming Feedback: Fine-Tuning Generative Models with Student-Written Feedback." pith.science (2026). https://pith.science/paper/NJ4TCBOS

@misc{pith2026250910647,
  author       = {Pith},
  title        = {Pith review of: Humanizing Automated Programming Feedback: Fine-Tuning Generative Models with Student-Written Feedback},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NJ4TCBOS}},
  note         = {Machine review of arXiv:2509.10647}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

The growing need for automated and personalized feedback in programming education has led to recent interest in leveraging generative AI for feedback generation. However, current approaches tend to rely on prompt engineering techniques in which predefined prompts guide the AI to generate feedback. This can result in rigid and constrained responses that fail to accommodate the diverse needs of students and do not reflect the style of human-written feedback from tutors or peers. In this study, we explore learnersourcing as a means to fine-tune language models for generating feedback that is more similar to that written by humans, particularly peer students. Specifically, we asked students to act in the flipped role of a tutor and write feedback on programs containing bugs. We collected approximately 1,900 instances of student-written feedback on multiple programming problems and buggy programs. To establish a baseline for comparison, we analyzed a sample of 300 instances based on correctness, length, and how the bugs are described. Using this data, we fine-tuned open-access generative models, specifically Llama3 and Phi3. Our findings indicate that fine-tuning models on learnersourced data not only produces feedback that better matches the style of feedback written by students, but also improves accuracy compared to feedback generated through prompt engineering alone, even though some student-written feedback is incorrect. This surprising finding highlights the potential of student-centered fine-tuning to improve automated feedback systems in programming education.

Figures

Figures reproduced from arXiv: 2509.10647 by Adish Singla, Juho Leinonen, Michael Liut, Nachiket Kotalwar, Paul Denny, Tung Phung, Victor-Alexandru P\u{a}durean.

Figure 1
Figure 1. Figure 1: Examples of feedback by Llama3-8B with different settings for a buggy program attempting the “Sum Positive [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: (a) shows the scenario including the problem description and the buggy code. (b) shows the instruction given to the student asking them to act in the flipped role as a tutor and give feedback and a genuine example of student-written feedback. Problem and Buggy Program « @q Student’s Understanding about Bug(s) ✓ [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Feedback-writing task (Sections 3.1 and 3.2) [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Data analysis workflow (Sections 3.3 and 3.4) [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of prompts using two kinds of instruc [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. A Comparative Study of Student Perspectives on Technical Writing Feedback Quality: Evaluating LLMs, SLMs, and Humans in Computer Science Topics

    cs.HC 2025-12 conditional novelty 5.0

    Students rated locally-run Llama-3.1 feedback roughly on par with GPT-4 and often above human TAs in technical courses, but preferred human feedback for specialized writing.

Reference graph

Works this paper leans on

44 extracted references · 4 linked inside Pith · cited by 1 Pith paper

  1. [1]

    M. I. Abdin et al. Phi-3 Technical Report: A Highly Ca- pable Language Model Locally on Your Phone.CoRR, abs/2404.14219, 2024

    Pith/arXiv arXiv 2024

  2. [2]

    Al-batlaa, M

    A. Al-batlaa, M. Abdullah-Al-Wadud, and M. A. Hos- sain. A Review on Recommending Solutions for Bugs Using Crowdsourcing. InSaudi Computer Society Na- tional Computer Conference (NCC), 2018

    2018

  3. [3]

    K. M. Ala-Mutka. A Survey of Automated Assessment Approaches for Programming Assignments.Computer Science Education, 2005

    2005

  4. [4]

    B. A. Becker. An Effective Approach to Enhancing Compiler Error Messages. InProc. of the Techni- cal Symp. on Computer Science Education (SIGCSE), 2016

    2016

  5. [5]

    J. Cohen. A Coefficient of Agreement for Nominal Scales.Educationalf and Psychological Measurement, 1960

    1960

  6. [6]

    Denny, S

    P. Denny, S. Gulwani, N. T. Heffernan, T. K ¨aser, S. Moore, A. N. Rafferty, and A. Singla. Generative AI for Education (GAIED): Advances, Opportunities, and Challenges.CoRR, abs/2402.01580, 2024

    Pith/arXiv arXiv 2024

  7. [7]

    Frazier, K

    M. Frazier, K. Damevski, and L. Pollock. Customizing ChatGPT to Help Computer Science Principles Stu- dents Learn Through Conversation. InProceedings of the Conference on Innovation and Technology in Com- puter Science Education (ITiCSE), 2024

    2024

  8. [8]

    Hartmann, D

    B. Hartmann, D. MacDougall, J. Brandt, and S. R. Klemmer. What Would Other Programmers Do: Sug- gesting Solutions to Error Messages. InProceedings of the International Conference on Human Factors in Computing Systems (CHI), 2010

    2010

  9. [9]

    A. Head, E. L. Glassman, G. Soares, R. Suzuki, L. Figueredo, L. D’Antoni, and B. Hartmann. Writing Reusable Code Feedback at Scale with Mixed-Initiative Program Synthesis. InProceedings of the Conference on Learning @ Scale (L@S), 2017

    2017

  10. [10]

    Hellas, J

    A. Hellas, J. Leinonen, S. Sarsa, C. Koutcheme, L. Ku- janp¨a¨a, and J. Sorva. Exploring the Responses of Large Language Models to Beginner Programmers’ Help Re- quests. InProceedings of the Conference on Interna- tional Computing Education Research (ICER), 2023

    2023

  11. [11]

    Ihantola, T

    P. Ihantola, T. Ahoniemi, V. Karavirta, and O. Sep- p¨al¨a. Review of Recent Systems for Automatic Assess- ment of Programming Assignments. InProceedings of the Koli Calling International Conference on Comput- ing Education Research, 2010

    2010

  12. [12]

    M. A. Islam et al. MapCoder: Multi-Agent Code Gener- ation for Competitive Problem Solving. InProceedings of the Annual Meeting of the Association for Computa- tional Linguistics (ACL), 2024

    2024

  13. [13]

    Jiang et al

    H. Jiang et al. Toward Better Summarizing Bug Re- ports with Crowdsourcing Elicited Attributes.IEEE Transactions on Reliability, 2018

    2018

  14. [14]

    Khosravi, P

    H. Khosravi, P. Denny, S. Moore, and J. Stamper. Learnersourcing in the Age of AI: Student, Educator and Machine Partnerships for Content Creation.Com- puters and Education: Artificial Intelligence, 2023

    2023

  15. [15]

    Kohn and B

    T. Kohn and B. Manaris. Tell Me What’s Wrong: A Python IDE with Error Messages. InProceedings of the Technical Symposium on Computer Science Education (SIGCSE), 2020

    2020

  16. [16]

    Kotalwar, A

    N. Kotalwar, A. Gotovos, and A. Singla. Hints-In- Browser: Benchmarking Language models for Program- ming Feedback Generation. InProceedings of the An- nual Conference on Neural Information Processing Sys- tems (NeurIPS) Track on Datasets and Benchmarks, 2024

    2024

  17. [17]

    Koutcheme, N

    C. Koutcheme, N. Dainese, S. Sarsa, A. Hellas, J. Leinonen, and P. Denny. Open source language mod- els can provide feedback: Evaluating llms’ ability to help students using gpt-4-as-a-judge. InProceedings of the Conference on Innovation and Technology in Com- puter Science Education (ITiCSE), 2024

    2024

  18. [18]

    Kumar, I

    H. Kumar, I. Musabirov, M. Reza, J. Shi, A. Kuzminykh, J. J. Williams, and M. Liut. Im- pact of Guidance and Interaction Strategies for LLM Use on Learner Performance and Perception.CoRR, abs/2310.13712, 2023

    Pith/arXiv arXiv 2023

  19. [19]

    Lahza et al

    H. Lahza et al. Analytics of Learning Tactics and Strategies in an Online Learnersourcing Environment. Journal of Computer Assisted Learning, 2023

    2023

  20. [20]

    Leinonen, A

    J. Leinonen, A. Hellas, S. Sarsa, B. N. Reeves, P. Denny, J. Prather, and B. A. Becker. Using Large Language Models to Enhance Programming Error Messages. In Proceedings of the Technical Symposium on Computer Science Education (SIGCSE), 2023

    2023

  21. [21]

    R. Liu, C. Zenke, C. Liu, A. Holmes, P. Thornton, and D. J. Malan. Teaching CS50 with AI: Leveraging Gen- erative Artificial Intelligence in Computer Science Ed- ucation. InProceedings of the Technical Symposium on Computer Science Education (SIGCSE), 2024

    2024

  22. [22]

    S. Liu, Z. Yu, F. Huang, Y. Bulbulia, A. Bergen, and M. Liut. Can Small Language Models With Retrieval- Augmented Generation Replace Large Language Mod- els When Learning Computer Science? InProceedings of Innovation and Technology in Computer Science Ed- ucation (ITiCSE), 2024

    2024

  23. [23]

    Llama-3.https://ai.meta.com/blog/ meta-llama-3/, 2024

    Meta. Llama-3.https://ai.meta.com/blog/ meta-llama-3/, 2024

    2024

  24. [24]

    Mujumdar, M

    D. Mujumdar, M. Kallenbach, B. Liu, and B. Hart- mann. Crowdsourcing Suggestions to Programming Problems for Dynamic Web Development Languages. InExtended Abstracts on Human Factors in Comput- ing Systems (CHI), 2011

    2011

  25. [25]

    ChatGPT.https://openai.com/blog/ chatgpt, 2023

    OpenAI. ChatGPT.https://openai.com/blog/ chatgpt, 2023

    2023

  26. [26]

    GPT-4 Technical Report.CoRR, abs/2303.08774, 2023

    OpenAI. GPT-4 Technical Report.CoRR, abs/2303.08774, 2023

    Pith/arXiv arXiv 2023

  27. [27]

    OpenAI Platform Models.https:// platform.openai.com/docs/models, 2024

    OpenAI. OpenAI Platform Models.https:// platform.openai.com/docs/models, 2024

    2024

  28. [28]

    Otaki and O

    B. Otaki and O. Lindwall. Generative AI and the Human Touch: Investigating the Changing Landscape of Feedback in Higher Education. InProceedings of the International Conference of the Learning Sciences (ICLS), 2024

    2024

  29. [29]

    Padurean, P

    V. Padurean, P. Denny, and A. Singla. BugSpotter: Au- tomated Generation of Code Debugging Exercises. In Proceedings of the Technical Symposium on Computer Science Education (SIGCSE), 2025

    2025

  30. [30]

    Paiva et al

    J. Paiva et al. Automated Assessment in Computer Science Education: A State-of-the-Art Review.Trans- actions on Computing Education (TOCE), 2022

    2022

  31. [31]

    Phung, J

    T. Phung, J. Cambronero, S. Gulwani, T. Kohn, R. Ma- jumdar, A. Singla, and G. Soares. Generating High- Precision Feedback for Programming Syntax Errors us- ing Large Language Models. InProceedings of the International Conference on Educational Data Mining (EDM), 2023

    2023

  32. [32]

    Phung, V

    T. Phung, V. P˘ adurean, J. Cambronero, S. Gulwani, T. Kohn, R. Majumdar, A. Singla, and G. Soares. Gen- erative AI for Programming Education: Benchmarking Chatgpt, GPT-4, and Human Tutors. InProceedings of the Conference on International Computing Education Research (ICER) - Volume 2, 2023

    2023

  33. [33]

    Phung, V

    T. Phung, V. Padurean, A. Singh, C. Brooks, J. Cam- bronero, S. Gulwani, A. Singla, and G. Soares. Au- tomating Human Tutor-Style Programming Feedback: Leveraging GPT-4 Tutor Model for Hint Generation and GPT-3.5 Student Model for Hint Validation. In Proceedings of the International Learning Analytics and Knowledge Conference (LAK), 2024

    2024

  34. [34]

    Piech, J

    C. Piech, J. Huang, A. Nguyen, M. Phulsuksombati, M. Sahami, and L. J. Guibas. Learning Program Em- beddings to Propagate Feedback on Student Code. In Proceedings of the International Conference on Ma- chine Learning (ICML), 2015

    2015

  35. [35]

    Pirttinen, P

    N. Pirttinen, P. Denny, A. Hellas, and J. Leinonen. Lessons Learned from Four Computing Education Crowdsourcing Systems.IEEE Access, 2023

    2023

  36. [36]

    Pirttinen and J

    N. Pirttinen and J. Leinonen. Can Students Review Their Peers? Comparison of Peer and Instructor Re- views. InProceedings of the Annual Conference on In- novation and Technology in Computer Science Educa- tion (ITiCSE), 2022

    2022

  37. [37]

    Singh, C

    A. Singh, C. Brooks, and S. Doroudi. Learnersourcing in Theory and Practice: Synthesizing the Literature and Charting the Future. InProceedings of the Confer- ence on Learning @ Scale (L@S), 2022

    2022

  38. [38]

    Singh, C

    A. Singh, C. Brooks, Y. Lin, and W. Li. What’s in It for the Learners? Evidence from a Randomized Field Experiment on Learnersourcing Questions in a MOOC. InProceedings of the Conference on Learning @ Scale (L@S), 2021

    2021

  39. [39]

    Singh, C

    A. Singh, C. Brooks, X. Wang, W. Li, J. Kim, and D. Wilson. Bridging Learnersourcing and AI: Explor- ing the Dynamics of Student-AI Collaborative Feedback Generation. InProceedings of the International Learn- ing Analytics and Knowledge Conference (LAK), 2024

    2024

  40. [40]

    Singh et al

    R. Singh et al. Automated Feedback Generation for Introductory Programming Assignments. InProceed- ings of the Conference on Programming Language De- sign and Implementation (PLDI), 2013

    2013

  41. [41]

    S. Wang, J. C. Mitchell, and C. Piech. A Large Scale RCT on Effective Error Messages in CS1. InProceed- ings of the Technical Symposium on Computer Science Education (SIGCSE), 2024

    2024

  42. [42]

    Zamfirescu-Pereira, L

    J. Zamfirescu-Pereira, L. Qi, B. Hartmann, J. DeN- ero, and N. Norouzi. Conversational Programming with LLM-Powered Interactive Support in an Introductory Computer Science Course.NeurIPS’23 Workshop on Generative AI for Education (GAIED), 2023

    2023

  43. [43]

    Zheng, R

    Y. Zheng, R. Zhang, J. Zhang, Y. Ye, Z. Luo, and Y. Ma. LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models. InProceedings of the Annual Meeting of the Association for Computational Linguis- tics (ACL), 2024

    2024

  44. [44]

    Zhou et al

    C. Zhou et al. LIMA: Less Is More for Alignment. In Proceedings of the Annual Conference on Neural Infor- mation Processing Systems (NeurIPS), 2023

    2023

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.