Pith. sign in

REVIEW 5 major objections 6 minor 28 references

The Impact of Team Diversity in Agile Development Education

T0 review · 5 major / 6 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper reports that gender diversity, measured as the number of gender categories in a team, shows a moderate positive correlation with the final project score of agile student teams, while nationality diversity has no meaningful negati

desk verdict Good-faith exploratory dataset and a new co-presence angle, but the headline gender-diversity result fails the paper's own α=0.01 and collapses when one team is removed; §5.2's 'still significant' is wrong. read the letter →

arxiv 2509.08389 v1 pith:HY5UQT25 submitted 2025-09-10 cs.SE cs.CY

Marco Torchiano , Riccardo Coppola , Antonio Vetro' , Xhoi Musaj This is my paper · ORCID
classification cs.SEcs.CY
keywords teamdiversitygendernationalityagilesoftwaredevelopmentengineeringeducationperformanceindexesempiricalstudy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether team diversity helps or hurts student teams in an agile software engineering project course, and answers with data from 51 teams over three academic years. It finds one statistically significant relationship by its own analysis: teams containing more than one gender category tend to score moderately higher on the final project (Spearman ρ = 0.307, p = 0.028). Nationality diversity and the joint gender–nationality 'co-presence' measure show negative but negligible and non-significant correlations, which the paper interprets as evidence that promoting diversity does not harm performance. The actionable conclusion for educators is that deliberately forming diverse teams carries no measurable grade penalty and may carry a small gender-diversity benefit.

What carries the argument

The analysis is carried by three standard diversity indexes—Richness (number of distinct categories in a team), the Blau index (probability that two randomly chosen members are in different categories), and the Shannon entropy index (Σ pᵢ ln pᵢ)—each applied to four team-level factors: gender, nationality, the co-presence class of gender×nationality, and career (Erasmus/international status). The paper's headline result comes specifically from the Gender Richness measure. Two statistical instruments do the work: Spearman rank correlations for individual factor effects, and a linear regression model with ANOVA to weigh the factors jointly alongside team size, self-reported skill, and academic

What would settle it

Remove the single team containing a member who preferred not to disclose gender from the 51-team sample and recompute the Spearman correlation between Gender Richness and normalized team score; the paper's own numbers put the resulting p-value near 0.099, above conventional significance. A second check: pool the next two editions of the course and see whether the correlation replicates with a similar effect size.

Watch

Extended reading notes

Core claim

The paper's central claim is that gender diversity is positively associated with team performance in agile development education. Measured with the Richness index—the count of distinct gender categories self-declared in a team—gender diversity correlates at ρ=0.307 with the normalized final team score (p=0.028), a moderate effect. Nationality diversity shows a negative but negligible correlation, and the combined gender×nationality factor is negative in the joint linear model; neither is statistically significant. The paper reads the absence of significant negative effects as the main practical result: diversifying teams by gender and nationality does not lower student project outcomes, and

Load-bearing premise

The claim of a statistically significant gender-diversity effect depends on the inclusion of one specific team, the only team containing a student who preferred not to disclose gender; dropping that team changes the p-value to 0.099, and the paper's stated α=0.01 threshold is actually stricter than the 0.028 p-value it reports as significant.

Editorial extensions

If this is right

  • Educators can assign students to maximize gender and nationality diversity without expecting lower project scores; nationality diversity's negative effect is small and non-significant.
  • If the gender-richness correlation is causal, team-formation policies that mix genders would produce moderately better project outcomes on average.
  • The negative coefficient for gender–nationality co-presence, even though non-significant, points to a need for communication support in teams that combine both dimensions.
  • Because the three diversity indexes correlate very strongly (0.96–0.99), simple category-richness counts are a practical proxy for more elaborate diversity measures in classroom monitoring.
  • Academic year and team size also enter the model, so future studies should include context co-factors rather than looking at diversity alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The headline correlation is fragile: the one team containing a student who preferred not to disclose gender supplies much of the effect, and excluding it pushes p to about 0.099. Treating the gender–performance link as established would require replication with more teams and more gender categories.
  • The proposed communication-barrier explanation for the negative co-presence effect is testable: collect measures of meeting quality, conflict frequency, or peer evaluations and see whether they mediate the diversity–score relationship.
  • The same design could be moved from the classroom to industrial agile teams, where team composition is less controlled and the performance stakes differ; the direction of the gender effect might not carry over.
  • An intersectional reading (age, disability, first-generation status) is the paper's acknowledged next step; its nationality measure is only a proxy for ethnicity, so the co-presence result should not be generalized to ethnic diversity.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper reports an observational study of 51 student teams in an agile software engineering capstone course over three academic years. The authors compute Richness, Blau, and Shannon diversity indexes for gender, nationality, gender-by-nationality co-presence, and career, and relate them to normalized team project scores using Spearman correlations and a linear regression model. The paper claims a moderate, statistically significant positive correlation between gender diversity and project success, a negligible negative effect of nationality diversity, and a negative effect of gender-nationality co-presence, and it offers recommendations for educators.

Significance. The topic is important and understudied: quantitative evidence on the effects of team diversity in SE education is scarce, and the authors are transparent about many limitations. The use of three standard diversity indexes and the reporting of team-level data are strength. However, the central positive claim is not supported by the paper's own statistical decision rule, and the robustness check reported by the authors removes the evidential basis for the headline result. As it stands, the paper's main empirical contribution is not established, although the secondary null results and the dataset may be of interest to the community.

major comments (5)
  1. [§3.4 / §4.3 / Table 4] The paper sets α=0.01 as the significance threshold in §3.4, yet §4.3 and Table 4 treat the Gender Richness correlation (p=0.028, ρ=0.307) as statistically significant and mark it with an asterisk. Section 5.1 later refers to a '0.05 selected threshold' for nationality. These decision rules are mutually inconsistent. Under the stated α=0.01, no correlation in Table 4 is significant; even under α=0.05, no correction is applied for the 12 correlations tested. The abstract's 'statistically significant' therefore rests on an unstable criterion.
  2. [§5.2] This section contains the decisive robustness problem. It states that the p=0.028 result is 'mainly related to the presence of a single person ... when removing the relative team the resulting p-value (0,099) would still be significant.' That statement is false: p=0.099 is not significant at α=0.05 or α=0.01. The only correlation supporting the headline claim is therefore driven by one team containing a student who preferred not to disclose gender. The paper's own limitation text removes the evidential basis for the abstract's positive claim.
  3. [§3.4 / §5.2] The claim in §5.2 that 'we adopted a fairly permissive value of α since our is an exploratory study' directly contradicts §3.4's 'customary α=0.01.' Exploratory status does not justify a post hoc relaxation of the threshold, and it should in fact motivate multiplicity control. The reader cannot determine which α was applied to which test; this ambiguity is load-bearing because the sole positive result lies exactly in the disputed region.
  4. [§4.4 / Table 5] The linear model reports Gender Richness with p=0.039, and the text calls this a 'confirmation' of the significant correlation. This p-value also fails the stated α=0.01. The model-building procedure selects one diversity index per factor based on the strength of individual bivariate correlations, introducing selection bias, and the model estimates 8 coefficients from only 51 observations. The 'confirmation' language is therefore not supported by the reported analysis.
  5. [Abstract / §4.4 / §5.1] The abstract asserts that 'gender and nationality combined had a negative impact.' In Table 5, the CoPresence coefficient is negative but non-significant (p=0.215), and all CoPresence correlations in Table 4 are non-significant. A non-significant negative coefficient is not evidence of an impact. The same overstatement appears in the conclusion that 'promoting diversity in teams does not negatively impact their performance,' which treats null results as evidence of absence despite low statistical power.
minor comments (6)
  1. [§3.4] Typo: 'we e consider' should be 'we consider.'
  2. [§4.3 / Figure 5] The text says Spearman correlations are used, but the Figure 5 caption says 'Pearson ρ coefficients.' Please reconcile the notation.
  3. [Table 4] The asterisk on Gender Richness is not explained in the caption. Given the α inconsistency, it is essential to state the threshold that the asterisk represents.
  4. [§5.2] The decimal comma in '0,099' should be a decimal point: 0.099.
  5. [§6] The actionable bullet says 'inter-team diversity has no statistically significant negative effect,' but the analysis concerns intra-team diversity. The wording should be corrected to avoid confusion.
  6. [§1] The claim that this is 'the first quantitative investigation' of DEI effects on team performance in SE courses is stronger than the cited literature search supports. Please soften to 'to our knowledge' or provide a systematic search.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical observational study with standard diversity indexes; the sensitivity/alpha concerns in §5.2 are statistical robustness issues, not circular steps.

full rationale

This is an observational empirical study, not a derivation. The diversity indexes (Richness, Blau, Shannon) are standard formulas applied to self-reported gender and nationality; they are not fitted to the outcome, and the dependent variable (normalized team score) comes from independent grading. The correlations and regression in §4.3–4.4 are statistical summaries of the data, not predictions derived from fitted values, and no parameter is calibrated to a subset and then 'predicted' on a closely related quantity. The reference list contains no prior work by the present authors, so no load-bearing self-citation chain exists. RQ0’s correlations among the three indexes are partly mathematical consequences of their shared definitions, but the paper presents them descriptively, not as a discovery that reduces to its inputs. The one passage worth explicit flagging is §5.2 (Threats to Validity), where the authors admit that the Gender Richness correlation (p=0.028) is mainly driven by a single person who preferred not to disclose gender and that removing that team yields p=0.099, which they incorrectly call 'still significant.' That is a legitimate sensitivity and threshold-consistency limitation (the paper states α=0.01 in §3.4 yet sometimes refers to 0.05), but it does not make the claim circular: the correlation is still an empirical measurement of the data as analyzed, not an input redefined as an output. No circular step is present.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central claim rests on standard diversity indexes and a fitted regression model. The main free parameters are the sample correlation and regression coefficient that drive the positive gender result. The load-bearing data assumption is the inclusion of the one team with a non-disclosed-gender student, and the invented CoPresence construct is used to test the interaction.

free parameters (3)
  • Gender Richness Spearman correlation = ρ=0.307
    Computed from the 51-team sample; it is the basis for the paper's headline positive gender-diversity result.
  • Gender Richness regression coefficient = 11.185 (SE 5.248)
    Estimated in the combined linear model (Table 5); the paper uses it to claim gender diversity positively affects performance when other factors are held constant, with p=0.039.
  • Significance threshold α = 0.01 stated, applied as ~0.05
    The choice of threshold determines whether p=0.028 counts as significant; the paper states α=0.01 but treats 0.028 as significant, making the effective threshold load-bearing.
assumptions (4)
  • domain assumption Self-declared gender and official nationality accurately reflect team diversity.
    Diversity indexes are computed from student self-reports and enrollment records (§3.3). Misreporting would distort the measured factors.
  • domain assumption Normalized team scores from three academic years are comparable.
    Scores are normalized to a 100 top grade per year (§3.3), and the pooled correlation assumes the year-specific grading scales are linearly comparable.
  • domain assumption Inclusion of the team with a student who did not disclose gender is valid.
    The robustness check in §5.2 shows p moves from 0.028 to 0.099 if this team is removed; the headline significance depends on this inclusion.
  • standard math OLS linear regression on the bounded 0-100 team score is appropriate.
    The paper uses a linear model with a normalized bounded outcome (§3.4) without discussing ceiling/floor effects or residual diagnostics.
invented entities (1)
  • CoPresence diversity factor (Gender × Nationality joint category)
    purpose: To test whether the intersection of gender and nationality diversity has an effect on team performance beyond each factor alone (§3.3, Figure 1).
    The construct is defined entirely within this paper by crossing two demographic variables; the negative regression coefficient is fit on the same 51-team dataset and has no external benchmark.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Impact of Team Diversity in Agile Development Education." pith.science (2026). https://pith.science/paper/HY5UQT25

@misc{pith2026250908389,
  author       = {Pith},
  title        = {Pith review of: The Impact of Team Diversity in Agile Development Education},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HY5UQT25}},
  note         = {Machine review of arXiv:2509.08389}
}
read the original abstract

Software Engineering is mostly a male-dominated sector, where gender diversity is a key feature for improving equality of opportunities, productivity, and innovation. Other diversity aspects, including but not limited to nationality and ethnicity, are often understudied.In this work we aim to assess the impact of team diversity, focusing mainly on gender and nationality, in the context of an agile software development project-based course. We analyzed 51 teams over three academic years, measuring three different Diversity indexes - regarding Gender, Nationality and their co-presence - to examine how different aspects of diversity impact the quality of team project outcomes.Statistical analysis revealed a moderate, statistically significant correlation between gender diversity and project success, aligning with existing literature. Diversity in nationality showed a negative but negligible effect on project results, indicating that promoting these aspects does not harm students' performance. Analyzing their co-presence within a team, gender and nationality combined had a negative impact, likely due to increased communication barriers and differing cultural norms.This study underscores the importance of considering multiple diversity dimensions and their interactions in educational settings. Our findings, overall, show that promoting diversity in teams does not negatively impact their performance and achievement of educational goals.

Figures

Figures reproduced from arXiv: 2509.08389 by the authors.

Figure 1
Figure 1. figure 1. This section details the context of our analyses, the research [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 1
Figure 1. Overview of study design. • the Co-present (Gender and Nationality) diversity, mea￾sured as the joint class of Gender and Nationality, • the Career diversity, measured on the basis of the student being an Erasmus or International student, i.e. students who do not share a common background with the other students, since they typically come from foreign university for a single semester. All diversity factors have been… view at source ↗
Figure 2
Figure 2. Team score distribution Blau Richness Shannon Career CoPresence Gender Nationality 0.0 0.2 0.4 0.6 0.8 1 2 3 4 5 0.0 0.5 1.0 1.5 0 10 20 30 40 0 5 10 15 20 0 10 20 30 40 50 0 10 20 Diversity count [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Cross correlations of the three Diversity Indexes [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 3
Figure 3. Figure 3: Overview of DI distributions the three years, while the proportion of foreign students doubled (from 14% to 28%). The average normalized team scores ranged from 93 to 90. Fig￾ure 2 reports the detailed distribution of team scores in the three considered academic years.…
Figure 5
Figure 5. Figure 5: Correlations of Diversity and Team Score for differ [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 26 canonical work pages

  1. [1]

    American Psychological Association and others. 2015. Guidelines for psycho- logical practice with transgender and gender nonconforming people.American psychologist70, 9 (2015), 832–864

    work page 2015

  2. [2]

    Neil Anderson, Aidan McGowan, Leo Galway, Matthew Collins, and Philip Hanna

    work page

  3. [3]

    Peter M. Blau. 1977.Inequality and heterogeneity: A primitive theory of social structure. Free Press

    work page 1977

  4. [4]

    Bhaskar Chakravorti. 2020. To Increase Diversity, U.S. Tech Companies Need to Follow the Talent.Harvard Business Review(2020). https://hbr.org/2020/12/to- increase-diversity-u-s-tech-companies-need-to-follow-the-talent

    work page 2020

  5. [5]

    Kimberlé Crenshaw. 2013. Demarginalizing the intersection of race and sex: A black feminist critique of antidiscrimination doctrine, feminist theory and antiracist politics. InFeminist legal theories. Routledge, 23–51

    2013

  6. [6]

    Claudia Maria Cutrupi, Irene Zanardi, and Letizia Jaccheri. 2024. Draw a Software Engineer Test-Preliminary Attempts to Investigate University Students’ Percep- tions of Software Engineering Professions. InProceedings of the 5th ACM/IEEE Workshop on Gender Equality, Diversity, and Inclusion in Software Engineering. 45–46

    work page 2024

  7. [7]

    Kiev Gama and Reydne Santos. 2024. It’s not all about gender: A Multi- dimensional Course Perspective on Diversity and Inclusion in Software En- gineering Education. InSimpósio Brasileiro de Engenharia de Software (SBES). SBC, 487–498

    work page 2024

  8. [8]

    Emitzá Guzmán, Ricarda Anna-Lena Fischer, and Janey Kok. 2023. Mind the gap: gender, micro-inequities and barriers in software development.Empirical Software Engineering29, 1 (Dec. 2023). doi:10.1007/s10664-023-10379-8

    work page doi:10.1007/s10664-023-10379-8 2023

Show all 28 references
  1. [9]

    Lucia Happe, Kai Marquardt, Ricarda Trumpf, and Ingo Wagner. 2024. Decoding the Gap: A Retrospective Analysis of Women’s Experiences in Software Engineer- ing. InProceedings of the 5th ACM/IEEE Workshop on Gender Equality, Diversity, and Inclusion in Software Engineering. 27–28

    2024

  2. [10]

    Sonja M Hyrynsalmi. 2023. How Diversity and Inclusion Are Approached in Software Engineering University-level Teaching. In2023 IEEE/ACM 4th Workshop on Gender Equity, Diversity, and Inclusion in Software Engineering (GEICSE). IEEE, 17–24

    2023

  3. [11]

    Sonja M Hyrynsalmi. 2024. Challenges and opportunities: Implementing diversity and inclusion in software engineering university level education in Finland. Journal of Systems and Software(2024), 112239

    2024

  4. [12]

    Sonja M Hyrynsalmi, Ella Peltonen, Fanny Vainionpää, and Sami Hyrynsalmi

  5. [13]

    Yekaterina Kovaleva, Ari Happonen, and Eneli Kindsiko. 2022. Designing gender- neutral software engineering program. stereotypes, social pressure, and current attitudes based on recent studies. InProceedings of the Third Workshop on Gender Equality, Diversity, and Inclusion in...

    2022

  6. [14]

    In Proceedings of the 5th ACM/IEEE Workshop on Gender Equality, Diversity, and Inclusion in Software Engineering

    The Second Round: Diverse Paths Towards Software Engineering. In Proceedings of the 5th ACM/IEEE Workshop on Gender Equality, Diversity, and Inclusion in Software Engineering. 29–36

  7. [15]

    Anh Nguyen-Duc and Letizia Jaccheri. 2023. Gender Equality in Software En- gineering Education – A Study of Female Participation in Customer-Driven Projects. InSustainability in Software Engineering and Business Information Man- agement, Varun Gupta, Luis Rubalcaba, Chetna Gup...

    2023

  8. [16]

    Christian Murphy, Anya Mushakevich, and Yunha Park. 2021. Incorporating readings on diversity and inclusion into a traditional software engineering course. In2021 Conference on Research in Equitable and Sustained Participation in Engi- neering, Computing, and Technology (RESPE...

    2021

  9. [17]

    Manuela Andreea Petrescu, Simona Motogna, and Liviu Berciu. 2023. Women in Scrum Master Role: Challenges and Opportunities. In2023 IEEE/ACM 4th Workshop on Gender Equity, Diversity, and Inclusion in Software Engineering (GEICSE). IEEE, 49–55

    2023

  10. [18]

    Marco Ortu, Giuseppe Destefanis, Steve Counsell, Stephen Swift, Roberto Tonelli, and Michele Marchesi. 2017. How diverse is your team? Investigating gender and nationality diversity in GitHub teams.Journal of Software Engineering Research and Development5, 1 (2017), 9. doi:10....

    2017 doi

  11. [19]

    Gema Rodríguez-Pérez, Reza Nadri, and Meiyappan Nagappan. 2021. Perceived diversity in software engineering: a systematic literature review.Empirical Software Engineering26 (2021), 1–38

    2021

  12. [20]

    2024.Diversità è Cambiamento - Bilancio di Genere 2023

    Politecnico di Torino. 2024.Diversità è Cambiamento - Bilancio di Genere 2023. https://www.polito.it/sites/default/files/2024-01/BDG_2023_DEF.pdf

    2024

  13. [21]

    Spellerberg and Peter J

    Ian F. Spellerberg and Peter J. Fedor. 2003. A tribute to Claude Shan- non (1916–2001) and a plea for more rigorous use of species richness, species diversity and the ‘Shannon–Wiener’ Index.Global Ecology and Biogeography12, 3 (2003), 177–179. doi:10.1046/j.1466-822X.2003.0001...

    2003 arXiv

  14. [22]

    Mary Sánchez-Gordón and Ricardo Colomo-Palacios. 2024. On the Intersectional- ity of Software Practitioners and Role Models. InProceedings of the 5th ACM/IEEE Workshop on Gender Equality, Diversity, and Inclusion in Software Engineering (Lisbon, Portugal)(GE@ICSE ’24). Associa...

    2024

  15. [23]

    Anna Szlavi, Marit Fredrikke Hansen, Sandra Helen Husnes, and Tayana Uchôa Conte. 2023. Intersectionality in Computer Science: A Systematic Literature Review. In2023 IEEE/ACM 4th Workshop on Gender Equity, Diversity, and Inclusion in Software Engineering (GEICSE). 9–16. doi:10...

    2023

  16. [24]

    Günter K Stahl, Martha L Maznevski, Andreas Voigt, and Karsten Jonsen. 2010. Unraveling the effects of cultural diversity in teams: A meta-analysis of research on multicultural work groups.Journal of International Business Studies41, 4 (2010), 690–709. doi:10.1057/jibs.2009.85

    2010 doi

  17. [25]

    Shujian Wu. 2012. Overview of communication in global software development process. InProceedings of 2012 IEEE International Conference on Service Operations and Logistics, and Informatics. IEEE, 474–478

    2012

  18. [26]

    Bogdan Vasilescu, Daryl Posnett, Baishakhi Ray, Mark GJ van den Brand, Alexan- der Serebrenik, Premkumar Devanbu, and Vladimir Filkov. 2015. Gender and tenure diversity in GitHub teams. InProceedings of the 33rd annual ACM confer- ence on human factors in computing systems. 3789–3798

    2015

  19. [28]

    Xin Zhao and Riley Young. 2023. Workplace Discrimination in Software Engineer- ing: Where We Stand Today. In2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS). IEEE, 188–193

    2023

  20. [2024]

    InProceedings of the 5th ACM/IEEE Workshop on Gender Equality, Diversity, and Inclusion in Software Engineering

    Learning to Improve Gender Equality: An Analysis of Software Engineering Education in a UK University. InProceedings of the 5th ACM/IEEE Workshop on Gender Equality, Diversity, and Inclusion in Software Engineering. 37–44

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.