REVIEW 4 major objections 6 minor 40 references
Who Introduces and Who Fixes? Analyzing Code Quality in Collaborative Student's Projects
T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read In student pair projects, the heavier code contributor tends to introduce more issues, and 64.37% of issues are fixed by their original author; partner fixes take significantly more commits and days.
desk verdict Useful educational case study, but the 'who fixes' numbers rest on an attribution rule the authors themselves concede is unreliable, and the abstract overstates RQ1. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is commit-level issue attribution. The authors run cppcheck and a custom embedded-systems checker on every commit of every cloned repository, store each detected issue in a database, and use git blame to assign the introduction and removal of each issue to a specific student. This attribution is what turns raw counts into the 'who introduces, who fixes' stories, and it is paired with a clustering of pairs into balanced versus dominant-contributor groups and with Mann-Whitney U tests to compare fix-effort distributions.
What would settle it
Take a random sample of resolved issues, open the commits that removed them, and classify each removal as a deliberate correction, an incidental removal during unrelated refactoring, or a deletion of the offending code; if a substantial share are incidental, or if git blame's assigned author differs from the actual commit author on meaningful share, the central claims would not survive.
Extended reading notes
Core claim
The paper's central discovery is a behavioral asymmetry in student pairs: code-quality issues tend to be fixed by the student who created them, and that self-fix is faster than waiting for a partner. On its own terms, the paper shows that 64.37% of issues were removed by the same student who introduced them, with a mean of 3.77 commits and 4.60 days to fix, compared with 6.05 commits and 8.45 days when a different student made the fix. Issues inherited from template code take even longer, averaging 12 commits before correction, which the authors attribute to the absence of ownership. The paper also finds no correlation between a student's issue counts in individual labs and their project issues (r = -0.07), but a moderate correlation between the same student's issue counts across the two projects (r = 0.41), and interprets this as evidence that individual work habits, not lab performance, predict project code quality.
Load-bearing premise
The results rest on git blame correctly assigning each issue's introduction and removal to a specific student, and on a later commit that makes the issue disappear counting as that student's deliberate fix; if blame misattributes authorship or if issues vanish as side effects, the same-author share and fix-latency gaps could be artifacts.
Editorial extensions
If this is right
- If most issues are self-fixed, then pair members see little of each other's code; courses that want cross-learning should assign explicit code-review tasks rather than assume collaboration happens.
- Because cross-student fixes take more commits and days, shared and template code is a quality bottleneck; assigning explicit ownership of starter code could shorten that latency.
- The clustering of issue introduction and fixing near the deadline suggests that acceptance-gating feedback mostly shapes final cleanup, not ongoing development.
- The near-zero lab-to-project issue correlation means an individual's lab performance is not a reliable predictor of their code-quality behavior in team projects.
- The faster fixing of critical issues indicates students prioritize functionality, so blocking non-critical issues at submission is a plausible lever to raise quality.
Reading between the lines
- The authors' own caveat that issues may 'simply disappear due to other modifications' is testable: a diff-level audit of resolved issues would separate deliberate fixes from incidental ones, and could change the 64.37% figure.
- The same pipeline applied to larger teams or professional repositories would show whether the same-author fixing bias is specific to two-student projects or generalizes wherever code ownership is clear.
- The r = 0.41 cross-project correlation hints at a stable, measurable individual tendency to introduce issues; a short diagnostic task could test whether this tendency is a usable predictor before team formation.
- Since template code was fixed slowest, an intervention that assigns explicit ownership or pre-fixes starter code would provide a clean experiment for the ownership explanation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper investigates how code quality issues are introduced and resolved in student pair projects in an embedded systems course. The authors collected git histories from eight individual labs and two group projects for 34 students, ran cppcheck and a custom embedded-code analysis tool on every commit, and used git blame to attribute issue introduction and resolution to individual students. The study addresses three research questions: the impact of group formation on code quality, how students interact in fixing issues, and when issues are introduced and fixed. The main reported findings are that the most active contributors tend to introduce the most issues, that most issues are fixed by the same student who introduced them (64.37%), that cross-student fixes take significantly longer, and that issue resolution is often postponed until late in the project.
Significance. The paper targets a genuinely understudied area: code quality dynamics in student team projects, rather than individual programming assignments. Its strengths include the use of complete commit histories, the integration of course-specific embedded-systems quality rules, and the explicit reporting of effect sizes and p-values for several comparisons. The descriptive timing results—issues introduced in the middle-to-late phases and many fixed near the deadline—are plausible and useful for educators. However, the paper's headline claims are not all supported by its own statistics. The abstract's statement that the most active contributors introduce the most issues is contradicted by the non-significant Mann-Whitney test in Section VI.A (p = 0.974). Moreover, the 'who fixes' analysis rests on an unvalidated assumption that the first commit in which an issue disappears is a deliberate fix by that commit's author; the authors themselves acknowledge in Section VI.J that issues may simply disappear due to unrelated modifications. This attribution problem is load-bearing for the 64.37% figure and for the latency comparisons.
major comments (4)
- [Abstract and Section VI.A] The abstract's first finding, that 'the most active contributors often introduce the most issues,' is not supported by the paper's own statistical analysis. Section VI.A reports a Mann-Whitney U test between balanced (Cluster 0) and dominant-contribution (Cluster 1) groups with p = 0.974, and states that within-cluster differences between the higher- and lower-contribution students were not statistically significant. The descriptive means in Cluster 0 (7.14 vs. 1.90 issues) go in the expected direction, but the paper explicitly concludes in Section VI.E that no significant difference was found. This claim must be removed or rewritten as a non-significant descriptive trend.
- [Section VI (RQ2) and Section VI.J] The fixer analysis identifies the first commit in which a previously detected issue no longer appears and credits that commit's author as the fixer. Section VI.J concedes that 'it is unclear whether issues were genuinely corrected by the students or if they simply disappeared due to other modifications in the code.' This is not a minor edge case: a code block that is deleted, renamed, moved, or rewritten for functional reasons will cause the issue to vanish from the analyzer output without any quality-fix intent. Because the original author is often the one who later edits or deletes their own code, this can create a systematic same-author bias. The 64.37% same-student rate and the commit/day latency differences (p = 0.0057 and p = 0.0002) are therefore not interpretable as evidence about 'who fixes' unless the authors validate that the disappearance events correspond to intentional corrections. A manual inspection of a random sample of fixed issues, or an explicit sensitivity analysis excluding deletions and moves, is needed.
- [Section VI.H and Table II] The finding that 'critical issues are fixed quickly' depends on an ad-hoc classification of which issue types are critical. Table II has a 'critical' column, but only five of the listed issue types are marked, and the criteria for this designation are not defined anywhere in the paper. The statement in the abstract that 'Critical issues are fixed quickly, but non-critical ones may be ignored' is a central takeaway, so the label set must be justified before this claim can be evaluated. If the critical labels were chosen after observing the fix times, this is a circularity that should be disclosed.
- [Section VI.A and RQ1] The clustering into balanced versus dominant groups uses arbitrary cutoffs (50–70% and 70–100% of LOC for the higher contributor) with no sensitivity analysis. With only 14 and 12 groups in the two projects, the Mann-Whitney test has low power, so the absence of a significant difference should be reported with a confidence interval or effect size rather than as a definitive null result. The current text moves from 'not statistically significant' to 'no impact' in Section VI.E, which overstates what can be concluded.
minor comments (6)
- [Title and author line] The title's possessive 'Student's Projects' should be 'Students' Projects,' and '3nd' should be '3rd' in the author affiliation line.
- [Section VI headings] The second research-question section is headed 'Q2: How do students interact...' and should be 'RQ2' for consistency with RQ1 and RQ3.
- [Section IV.C and Section VI.H] The rule name is written 'slowIRS' in Section IV.C and 'slowIRS' elsewhere, but the text also refers to similar names with different capitalization; the spelling should be unified (the code-level name should be consistently rendered, e.g., 'slowISR' or 'slowIRS').
- [Table I] The cppcheck row 'invalidPrintfArgType' appears twice with different counts; if these are separate rule variants they should be renamed or merged, and if this is a typo it should be corrected.
- [Section V and Section VI] The study is described as 'qualitative' in the abstract and introduction, but the methodology and results are overwhelmingly quantitative; the label should be changed to 'quantitative' or 'mixed-methods.'
- [Various places] There are numerous typos that should be cleaned up, including 'assesment' for 'assessment,' 'inseted' for 'inserted,' 'nornalized' for 'normalized,' 'fallowing' for 'following,' and 'Fall semester of 2023 semesters.'
Circularity Check
No significant circularity: the empirical derivation chain is self-contained, and the sole self-citation (embedded-check tool) functions as a measurement apparatus rather than a load-bearing premise.
full rationale
I walked the derivation chain for RQ1-RQ3. The behavior claims (dominant student, same-author fixes, issue timing, critical-issue latency) are all estimated from commit histories, cppcheck output, and the course-specific embedded-check rules stated in Section IV; none of the reported statistics is fitted to a subset of data and then presented as a prediction of the same data, and no quantity is defined in terms of the quantity it is meant to explain. The only self-citation is reference [21] for the embedded-check tool, used as the operationalization of the embedded-issues rules; citing one's own tool is not circular unless the tool's output is the conclusion being argued, which is not the case here, since the conclusions concern student behavior and fix latency rather than the tool's correctness. Section VI.J does flag a genuine validity threat: "it is unclear whether issues were genuinely corrected by the students or if they simply disappeared due to other modifications in the code." That caveat means the 64.37% same-author fix rate and the commit/day latency differences are vulnerable to a measurement artifact, but this is a threats-to-validity issue, not an input-output equivalence: the paper does not define 'fixed' in a way that makes the same-author result true by construction, and the authors do not reuse that result as an input to any further derivation. I therefore find no circular step and assign score 0.
Assumptions & free parameters
free parameters (2)
- LOC dominance threshold for Cluster 1 =
70%
- Critical issue labels =
4 issue types marked critical
assumptions (3)
- domain assumption Git blame maps each issue to the student who actually introduced or fixed it.
- domain assumption Students read and acted on the automated code-quality feedback.
- domain assumption The custom embedded-check tool's rule set correctly operationalizes embedded code quality.
Cite this review
Pith. "Pith review of Who Introduces and Who Fixes? Analyzing Code Quality in Collaborative Student's Projects." pith.science (2026). https://pith.science/paper/4GNOPCWP
@misc{pith2026250514315,
author = {Pith},
title = {Pith review of: Who Introduces and Who Fixes? Analyzing Code Quality in Collaborative Student's Projects},
year = {2026},
howpublished = {\url{https://pith.science/paper/4GNOPCWP}},
note = {Machine review of arXiv:2505.14315}
}
read the original abstract
This paper investigates code quality education by analyzing how errors are introduced and corrected in group projects within an embedded systems course. We identify who introduces errors, who fixes them, and when these actions occur. Students learn code quality rules for C and embedded systems. We address three questions: RQ1: What is the impact of group formation on code quality? RQ2: How do students interact to fix code issues? RQ3: When are issues introduced and resolved? We analyzed data from eight individual labs and two group projects involving 34 students. The course provides continuous, automated feedback on code quality. Findings show that the most active contributors often introduce the most issues. Many issues are fixed late in the project. Individual labs tend to have fewer issues due to their structured nature. Most problems are fixed by the original author, while cross-student fixes take longer, especially in shared code. Critical issues are fixed quickly, but non-critical ones may be ignored, showing a focus on functionality over quality.
Figures
Reference graph
Works this paper leans on
-
[1]
A Systematic Mapping Study of Code Quality in Education,
H. Keuning, J. Jeuring, and B. Heeren, “A Systematic Mapping Study of Code Quality in Education,” in Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1. Turku Finland: ACM, 2023, pp. 5–11
work page 2023
-
[2]
P. Louridas, “Static code analysis,” IEEE Software, vol. 23, no. 4, pp. 58–61, Jul. 2006
work page 2006
-
[3]
An overview on the Static Code Analysis approach in Software Development,
I. Gomes, P. Morgado, T. Gomes, and R. Moreira, “An overview on the Static Code Analysis approach in Software Development,” 2009
work page 2009
-
[4]
The Cost of Poor Software Quality in the US: A 2020 Report,
H. Krasner, “The Cost of Poor Software Quality in the US: A 2020 Report,” 2020
work page 2020
-
[5]
Software design analysis and technical debt management based on design rule theory,
Y . Cai and R. Kazman, “Software design analysis and technical debt management based on design rule theory,” Information and Software Technology, vol. 164, p. 107322, Dec. 2023
work page 2023
-
[6]
Technical Debt: From Metaphor to Theory and Practice,
P. Kruchten, R. L. Nord, and I. Ozkaya, “Technical Debt: From Metaphor to Theory and Practice,” IEEE Software, vol. 29, no. 6, pp. 18–21, Nov. 2012
work page 2012
-
[7]
S. Berrezueta-Guzman, I. Parmacli, M. K. Habib, S. Krusche, and S. Wagner. Assessing Teamwork Dynamics in Software Development Projects. [Online]. Available: http://arxiv.org/abs/2501.11965
-
[8]
What is embedded systems and how should it be taught?—results from a didactic analysis,
M. Grimheden and M. T ¨orngren, “What is embedded systems and how should it be taught?—results from a didactic analysis,” ACM Transactions on Embedded Computing Systems , vol. 4, no. 3, pp. 633– 651, Aug. 2005
work page 2005
Show all 40 references
-
[9]
Embedded Systems Education: Experiences With Application-Driven Pedagogy,
S. Pasricha, “Embedded Systems Education: Experiences With Application-Driven Pedagogy,” IEEE Embedded Systems Letters , vol. 14, no. 4, pp. 167–170, Dec. 2022
2022
-
[10]
Faults-slip-through—a concept for measuring the efficiency of the test process,
L.-O. Damm, L. Lundberg, and C. Wohlin, “Faults-slip-through—a concept for measuring the efficiency of the test process,” Software Process: Improvement and Practice , vol. 11, no. 1, pp. 47–59, 2006
2006
-
[11]
Evaluating the cost reduction of static code analysis for software security,
D. Baca, B. Carlsson, and L. Lundberg, “Evaluating the cost reduction of static code analysis for software security,” in Proceedings of the Third ACM SIGPLAN Workshop on Programming Languages and Analysis for Security , ser. PLAS ’08. New York, NY , USA: Association for Comput...
2008
-
[12]
What we have learned about fighting defects,
F. Shull, V . Basili, B. Boehm, A. Brown, P. Costa, M. Lindvall, D. Port, I. Rus, R. Tesoriero, and M. Zelkowitz, “What we have learned about fighting defects,” in Proceedings Eighth IEEE Symposium on Software Metrics, Jun. 2002, pp. 249–258
2002
-
[13]
The Google FindBugs fixit,
N. Ayewah and W. Pugh, “The Google FindBugs fixit,” in Proceedings of the 19th International Symposium on Software Testing and Analysis . Trento Italy: ACM, Jul. 2010, pp. 241–252
2010
-
[14]
Using Static Analysis to Find Bugs,
N. Ayewah, W. Pugh, D. Hovemeyer, J. D. Morgenthaler, and J. Penix, “Using Static Analysis to Find Bugs,” IEEE Software , vol. 25, no. 5, pp. 22–29, Sep. 2008
2008
-
[15]
Prioritizing Warning Categories by Analyzing Software History,
S. Kim and M. D. Ernst, “Prioritizing Warning Categories by Analyzing Software History,” in Fourth International Workshop on Mining Software Repositories (MSR’07:ICSE Workshops 2007) , May 2007, pp. 27–27
2007
-
[16]
Reliability in the Assessment of Program Quality by Teaching Assistants During Code Reviews,
M. J. Scott and G. Ghinea, “Reliability in the Assessment of Program Quality by Teaching Assistants During Code Reviews,” in Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education, ser. ITiCSE ’15. New York, NY , USA: Association for ...
2015
-
[17]
Investigating novice programming mistakes: Educator beliefs vs. student data,
N. C. Brown and A. Altadmri, “Investigating novice programming mistakes: Educator beliefs vs. student data,” in Proceedings of the Tenth Annual Conference on International Computing Education Research, ser. ICER ’14. New York, NY , USA: Association for Computing Machinery, Jul...
2014
-
[18]
Teachers’ Beliefs and Practices on the Naming of Variables in Introductory Python Programming Courses,
V . Van Der Werf, A. Swidan, F. Hermans, M. Specht, and E. Aival- oglou, “Teachers’ Beliefs and Practices on the Naming of Variables in Introductory Python Programming Courses,” in Proceedings of the 46th International Conference on Software Engineering: Software Engineer- ing...
2024
-
[19]
Improving TA Feedback on In-Class Coding Assignments for Introductory Computer Science,
A. Cook, V . Phan, and A. Windsor, “Improving TA Feedback on In-Class Coding Assignments for Introductory Computer Science,” in Proceed- ings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1 . Dublin Ireland: ACM, Jul. 2022, pp. 421–427
2022
-
[20]
Teaching Quality in Programming Education: The Effect of Teachers’ Background Characteristics and Self-efficacy,
X. Jia and F. Hermans, “Teaching Quality in Programming Education: The Effect of Teachers’ Background Characteristics and Self-efficacy,” in Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 1. Lugano and Virtual Event Switzerland: A...
2022
-
[21]
Embedded-check a Code Quality Tool for Automatic Firmware Verification,
R. Corsi Ferr ˜ao, I. Dos Santos Montagner, M. Silva, C. Zilles, and R. Azevedo, “Embedded-check a Code Quality Tool for Automatic Firmware Verification,” in Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1 . Milan Italy: ACM, Jul. 2024, ...
2024
-
[22]
Overcoming Barriers in Scaling Computing Education Research Programming Tools: A Developer’s Perspective,
K. Tran, J. Bacher, Y . Shi, J. Skripchuk, and T. Price, “Overcoming Barriers in Scaling Computing Education Research Programming Tools: A Developer’s Perspective,” inProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 1. Melbourne VIC...
2024
-
[23]
Understanding semantic style by analysing student code,
G. De Ruvo, E. Tempero, A. Luxton-Reilly, G. B. Rowe, and N. Gi- acaman, “Understanding semantic style by analysing student code,” in Proceedings of the 20th Australasian Computing Education Conference , ser. ACE ’18. New York, NY , USA: Association for Computing Machinery, Ja...
2018
-
[24]
Catalog of Code Quality Defects in Introductory Programming,
A. ˇRecht´aˇckov´a, R. Pel ´anek, and T. Effenberger, “Catalog of Code Quality Defects in Introductory Programming,” in Proceedings of the 2024 on Innovation and Technology in Computer Science Education V
2024
-
[25]
2024, pp
Milan Italy: ACM, Jul. 2024, pp. 59–65
2024
-
[26]
GiraffeReversed/edulint,
A. ˇRecht´aˇckov´a, “GiraffeReversed/edulint,” Apr. 2024
2024
-
[27]
Software Quality as a Subsidy for Teaching Programming,
P. H. Gomes, R. E. Garcia, D. M. Eler, R. C. Correia, and C. O. Junior, “Software Quality as a Subsidy for Teaching Programming,” in 2021 IEEE Frontiers in Education Conference (FIE) , Oct. 2021, pp. 1–9
2021
-
[28]
”In the Beginning, I Couldn’t Necessarily Do Anything With It
M. Dornbusch and J. Vahrenhold, “”In the Beginning, I Couldn’t Necessarily Do Anything With It”: Links Between Compiler Error Messages and Sense of Belonging,” in Proceedings of the 2024 ACM Conference on International Computing Education Research - Volume
2024
-
[29]
2024, pp
Melbourne VIC Australia: ACM, Aug. 2024, pp. 14–26
2024
-
[30]
Forming Groups for Collaborative Learning in Introductory Computer Programming Courses Based on Students’ Programming Styles: An Empirical Study,
E. Jose De Faria, J. Adan-coello, and K. Yamanaka, “Forming Groups for Collaborative Learning in Introductory Computer Programming Courses Based on Students’ Programming Styles: An Empirical Study,” in Proceedings. Frontiers in Education. 36th Annual Conference . San Diego, CA...
2006
-
[31]
Practice Patterns to Improve the Quality of Design Model in Embedded Software Development,
D.-H. Kim, J.-P. Kim, and J.-E. Hong, “Practice Patterns to Improve the Quality of Design Model in Embedded Software Development,” in 2009 Ninth International Conference on Quality Software , Aug. 2009, pp. 179–184
2009
-
[32]
An Investigation on Quality Models and Quality Attributes for Embedded Systems,
L. B. R. Oliveira, M. Galster, M. Guessi, F. Oquendo, D. Feitosa, C. Manteuffel, and E. Y . Nakagawa, “An Investigation on Quality Models and Quality Attributes for Embedded Systems,” 2013
2013
-
[33]
Empirical investigation in embedded systems: Quality attributes in general, maintainability in particular,
S. Motogna, A. Vescan, and C. S ¸erban, “Empirical investigation in embedded systems: Quality attributes in general, maintainability in particular,” Journal of Systems and Software , vol. 201, p. 111678, Jul. 2023
2023
-
[34]
Interrelations between Software Quality Met- rics, Performance and Energy Consumption in Embedded Applications,
L. Papadopoulos, C. Marantos, G. Digkas, A. Ampatzoglou, A. Chatzi- georgiou, and D. Soudris, “Interrelations between Software Quality Met- rics, Performance and Energy Consumption in Embedded Applications,” in Proceedings of the 21st International Workshop on Software and Com...
2018
-
[35]
Investi- gating Quality Trade-offs in Open Source Critical Embedded Systems,
D. Feitosa, A. Ampatzoglou, P. Avgeriou, and E. Y . Nakagawa, “Investi- gating Quality Trade-offs in Open Source Critical Embedded Systems,” in Proceedings of the 11th International ACM SIGSOFT Conference on Quality of Software Architectures . Montr ´eal QC Canada: ACM, May 20...
2015
-
[36]
A tool for IoT Firmware Certification,
G. M. Bianco, L. Ardito, and M. Valsesia, “A tool for IoT Firmware Certification,” in Proceedings of the 19th International Conference on Availability, Reliability and Security. Vienna Austria: ACM, Jul. 2024, pp. 1–7
2024
-
[37]
Definition, Implementation and Validation of Energy Code Smells: An Exploratory Study on an Embedded System,
A. Vetro’, L. Ardito, G. Procaccianti, and M. Morisio, “Definition, Implementation and Validation of Energy Code Smells: An Exploratory Study on an Embedded System,” 2013
2013
-
[38]
On the Use of Open-Source C/C++ Static Analysis Tools in Large Projects,
J. D’Abruzzo Pereira and M. Vieira, “On the Use of Open-Source C/C++ Static Analysis Tools in Large Projects,” in 2020 16th European Dependable Computing Conference (EDCC) , Sep. 2020, pp. 97–102
2020
-
[39]
Koopman, Better Embedded System Software
P. Koopman, Better Embedded System Software . Drumnadrochit Education Pittsburgh, 2010
2010
-
[40]
Toyota Sudden Unintended Acceleration,
S. Kane, E. Liberman, T. DiViesti, and F. Click, “Toyota Sudden Unintended Acceleration,” 2010
2010
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.