Recognition: unknown
Dead Code Doesn't Talk: Authentic Requirements Elicitation in Introductory Software Engineering
Pith reviewed 2026-05-10 16:59 UTC · model grok-4.3
The pith
Students improve requirements elicitation by practicing on games they built and meeting campus researchers as clients.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
By anchoring requirements elicitation practice to a student-authored artifact and engaging campus researchers as proxy clients, the intervention produced measurable gains in self-assessed soft skills and generated concrete outputs including 203 requirements and SRS documents averaging 6.79 out of 10. The four-phase structure—preparation, client meeting, requirements elaboration, and prototype sprint—enabled students to experience the trade-offs between stakeholder desires and implementation constraints firsthand. Thematic analysis of reflective reports identified this tension as the dominant learning theme. The authors conclude that the student-owned artifact reduces cognitive barriers while
What carries the argument
The four-phase elicitation activity anchored to the students' own prior Java 2D game, with campus researchers serving as proxy clients.
If this is right
- Programs without industry partnerships can still deliver authentic elicitation practice using internal researchers.
- Students experience the central professional tension between client wishes and technical feasibility as the most relevant outcome.
- Self-reported confidence rises across stakeholder empathy, negotiation, and related soft skills after the four-phase sequence.
- The method produces usable SRS documents and working prototypes within a standard course unit.
Where Pith is reading between the lines
- The same anchoring principle could be tested on other communication-intensive activities such as design reviews or acceptance testing.
- Longitudinal tracking of whether the skill gains persist into later projects or industry internships would clarify durability.
- Varying the familiarity of the student artifact across courses could isolate how much prior ownership drives the reported barrier reduction.
Load-bearing premise
The statistically significant self-assessment gains reflect genuine skill development caused by the intervention rather than response bias or other course elements, and campus researchers supply enough authenticity to generalize beyond this single course.
What would settle it
A controlled experiment that measures objective requirements quality or observed client-interaction performance in a group using this anchored method versus a traditional lecture-based approach, with no difference in outcomes.
Figures
read the original abstract
Requirements elicitation is among the most communication-intensive activities in software engineering, yet it receives limited explicit treatment in undergraduate curricula. This paper presents a case study of an Introduction to Software Engineering course in which 20 student teams applied requirements elicitation practices to a Java-based 2D game they had built in a prior programming course, engaging 18 campus doctoral and postdoctoral researchers as authentic clients. Structured across four phases--preparation, client meeting, requirements elaboration, and a prototype sprint--the activity produced 203 elicited requirements, SRS documents with a mean quality score of $6.79 \pm 1.08$ out of 10, and prototype demonstrations scoring $7.21 \pm 1.15$. A pre/post self-assessment survey revealed statistically significant improvements across all eight measured soft-skill dimensions, with the largest gains in Stakeholder Empathy ($\Delta = +1.33$) and Negotiation ($\Delta = +1.13$). Thematic analysis of reflective reports identified four dominant learning themes, with the tension between client wishes and technical feasibility cited as the most professionally relevant experience. Our findings suggest that anchoring elicitation practice to a student-authored artifact lowers cognitive barriers while increasing authenticity, and that campus researchers serve as an accessible and effective proxy client for programs without established industry partnerships.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a case study of an introductory software engineering course in which 20 student teams elicited requirements for a Java-based 2D game they had built previously, using 18 campus doctoral and postdoctoral researchers as clients. Structured in four phases (preparation, client meeting, requirements elaboration, prototype sprint), the activity produced 203 requirements, SRS documents with mean quality 6.79/10, and prototypes scoring 7.21/10. Pre/post self-assessments showed statistically significant gains across eight soft-skill dimensions (largest in Stakeholder Empathy Δ=+1.33 and Negotiation Δ=+1.13), while thematic analysis of reflections identified four learning themes, with client wishes vs. technical feasibility as the most relevant. The authors conclude that anchoring elicitation to student-authored artifacts lowers cognitive barriers and increases authenticity, and that campus researchers are accessible effective proxy clients for programs without industry partnerships.
Significance. If the results hold, this work is significant for software engineering education by offering a practical, scalable approach to authentic requirements elicitation practice using readily available student artifacts and academic clients. The concrete metrics (203 requirements, specific quality scores) and identified themes provide useful benchmarks and insights for curriculum designers seeking to address the limited treatment of elicitation in undergraduate programs, particularly where industry partnerships are unavailable.
major comments (3)
- Results section, pre/post self-assessment paragraph: The manuscript reports statistically significant improvements in all eight soft-skill dimensions with specific deltas, but provides no details on survey instrument validation, handling of demand characteristics or response bias, inter-rater reliability, or objective measures (e.g., blinded rubric scoring of requirements quality). This is load-bearing for the central claim that the intervention lowers cognitive barriers, as the gains could reflect maturation, general course effects, or bias rather than the specific anchoring to the student artifact and real clients.
- Methods/Study Design section: No control condition is described (such as elicitation on a non-student artifact or with simulated clients), and the 203 requirements and SRS/prototype scores are post-intervention only. Without this, alternative explanations cannot be ruled out, weakening the causal suggestion that the student-authored artifact and campus researchers specifically increase authenticity and reduce barriers.
- Thematic Analysis section: The identification of four dominant themes from reflective reports, including the tension between client wishes and feasibility, lacks reporting on the analysis process, number of coders, inter-rater reliability, or saturation criteria. This affects the robustness of the qualitative evidence supporting the professional relevance and authenticity claims.
minor comments (3)
- Abstract: The abstract states 'statistically significant improvements' and reports means with standard deviations but omits the specific statistical test, p-values, and sample size for the surveys; including these would improve transparency and allow better assessment of the findings.
- Results section: Clarify how the SRS quality scores (6.79 ± 1.08) and prototype scores (7.21 ± 1.15) were obtained, including the rubric used, who performed the scoring, and whether it was blinded or independent.
- Discussion: Consider adding effect sizes (e.g., Cohen's d) for the reported deltas to better interpret the practical magnitude of the skill gains beyond statistical significance.
Simulated Author's Rebuttal
We thank the referee for their constructive comments, which identify key areas for strengthening the methodological transparency of our case study. We address each point below and have revised the manuscript accordingly to clarify the study design, add limitations, and expand reporting on qualitative methods while preserving the case-study framing.
read point-by-point responses
-
Referee: Results section, pre/post self-assessment paragraph: The manuscript reports statistically significant improvements in all eight soft-skill dimensions with specific deltas, but provides no details on survey instrument validation, handling of demand characteristics or response bias, inter-rater reliability, or objective measures (e.g., blinded rubric scoring of requirements quality). This is load-bearing for the central claim that the intervention lowers cognitive barriers, as the gains could reflect maturation, general course effects, or bias rather than the specific anchoring to the student artifact and real clients.
Authors: We agree that the self-assessment data are self-reported and that the survey instrument received only face validation through pilot testing rather than formal psychometric validation. No objective measures or blinded scoring were used for the soft-skill gains. In the revised manuscript we have added an explicit 'Limitations and Threats to Validity' subsection that discusses response bias, demand characteristics, and the possibility of maturation or general course effects. We now state that the reported deltas should be interpreted as student perceptions rather than objective skill acquisition, while noting that the quantitative gains converge with the qualitative themes and the concrete artifact outcomes (203 requirements, SRS and prototype scores). revision: yes
-
Referee: Methods/Study Design section: No control condition is described (such as elicitation on a non-student artifact or with simulated clients), and the 203 requirements and SRS/prototype scores are post-intervention only. Without this, alternative explanations cannot be ruled out, weakening the causal suggestion that the student-authored artifact and campus researchers specifically increase authenticity and reduce barriers.
Authors: Our work is presented as a case study of a pedagogical intervention in a required course, not a controlled experiment; therefore no parallel control arm (e.g., non-student artifact or simulated clients) was included. We have revised the Methods section to state this design choice explicitly and to explain the practical constraints (course logistics and ethics of withholding a learning activity from enrolled students). A new 'Threats to Validity' paragraph now enumerates alternative explanations such as general course maturation and discusses why post-only artifact metrics are appropriate given that the activity constituted the students' first formal elicitation experience. We retain the claim that anchoring to student-authored code lowers cognitive barriers on the basis of student reflections, but we no longer imply strict causality. revision: yes
-
Referee: Thematic Analysis section: The identification of four dominant themes from reflective reports, including the tension between client wishes and feasibility, lacks reporting on the analysis process, number of coders, inter-rater reliability, or saturation criteria. This affects the robustness of the qualitative evidence supporting the professional relevance and authenticity claims.
Authors: The thematic analysis was conducted inductively by the first author with iterative review by the second author; disagreements were resolved through discussion until consensus. Saturation was reached after coding 15 of the 20 reports. We have now expanded the 'Thematic Analysis' subsection to describe this process, the use of open then axial coding, and the criterion for saturation. Formal inter-rater reliability statistics were not computed because the workflow was single-coder with consensus review rather than independent parallel coding; we acknowledge this as a limitation in the revised text and note that the four themes are presented as descriptive rather than as statistically validated categories. revision: yes
Circularity Check
No significant circularity; empirical case study with independent data collection
full rationale
The paper reports an educational case study involving pre/post self-assessment surveys, thematic analysis of student reflections, and post-intervention metrics (SRS quality scores, prototype scores, and 203 elicited requirements). No mathematical derivations, fitted parameters, predictions, or first-principles claims appear that could reduce outcomes to inputs by construction. The central suggestions about lowered cognitive barriers and campus researchers as proxy clients are interpretive conclusions drawn from the collected data rather than tautological re-statements of any prior definitions or self-citations. Self-citations, if present, are not load-bearing for the reported findings, which rest on direct observation and analysis within this study.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption Pre- and post-intervention self-assessments validly and reliably measure changes in soft skills such as stakeholder empathy and negotiation
- domain assumption Thematic analysis of student reflective reports can reliably surface the most professionally relevant learning experiences
Reference graph
Works this paper leans on
-
[1]
Perceived causes of software project failures–an analysis of their relationships,
T. O. Lehtinen, M. V . M ¨antyl¨a, J. Vanhanen, J. Itkonen, and C. Lasse- nius, “Perceived causes of software project failures–an analysis of their relationships,”Information and Software Technology, vol. 56, no. 6, pp. 623–643, 2014
2014
-
[2]
A systematic literature review of requirements engineering education,
M. Daun, A. M. Grubb, V . Stenkova, and B. Tenbergen, “A systematic literature review of requirements engineering education,”Requirements Engineering, vol. 28, no. 2, pp. 145–175, 2023
2023
-
[3]
Requirements engineering education: a systematic mapping study,
S. Ouhbi, A. Idri, J. L. Fern ´andez-Alem´an, and A. Toval, “Requirements engineering education: a systematic mapping study,”Requirements En- gineering, vol. 20, no. 2, pp. 119–138, 2015
2015
-
[4]
Gaps between industry expectations and the abilities of graduates,
A. Radermacher and G. Walia, “Gaps between industry expectations and the abilities of graduates,” inProceeding of the 44th ACM technical symposium on Computer science education, pp. 525–530, 2013
2013
-
[5]
Game-based learning in requirements engi- neering: An overview,
M. T. Soo and H. Aris, “Game-based learning in requirements engi- neering: An overview,” in2018 IEEE Conference on e-Learning, e- Management and e-Services (IC3e), pp. 46–51, IEEE, 2018
2018
-
[6]
Group projects using clients versus not using clients: Do students perceive any differences?,
A. L. Parsons and E. Lepkowska-White, “Group projects using clients versus not using clients: Do students perceive any differences?,”Journal of Marketing Education, vol. 31, no. 2, pp. 154–159, 2009
2009
-
[7]
Involving external stakeholders in project courses,
J.-P. Stegh ¨ofer, H. Burden, R. Hebig, G. Calikli, R. Feldt, I. Hammouda, J. Horkoff, E. Knauss, and G. Liebel, “Involving external stakeholders in project courses,”ACM Transactions on Computing Education (TOCE), vol. 18, no. 2, pp. 1–32, 2018
2018
-
[8]
Devel- opment of soft skills in computer science bachelors in the project learning process,
O. Glazunova, V . Korolchuk, T. V oloshyna, and T. Vakaliuk, “Devel- opment of soft skills in computer science bachelors in the project learning process,”Information Technologies and Learning Tools, vol. 92, pp. 111–123, 12 2022
2022
-
[9]
Ideas for adding soft skills education to service learning and capstone courses for computer science students,
L. Carter, “Ideas for adding soft skills education to service learning and capstone courses for computer science students,” inProceedings of the 42nd ACM Technical Symposium on Computer Science Education, SIGCSE ’11, (New York, NY , USA), p. 517–522, Association for Computing Machinery, 2011
2011
-
[10]
Long-term effects of promoting communication and soft skills in higher computing education,
R. Motschnig, D. Dolezal, V . ˇSv´abensk`y, K. Palubov ´a, and M. Silber, “Long-term effects of promoting communication and soft skills in higher computing education,”ACM Transactions on Computing Education, 2026
2026
-
[11]
Trends and challenges of project-based learning in computer science and engineering education,
S. U. Rehman, “Trends and challenges of project-based learning in computer science and engineering education,” inProceedings of the 15th International Conference on Education Technology and Computers, pp. 397–403, 2023
2023
-
[12]
Gamified project-based learning in vocational education and training computer science courses,
B. D ´ıaz-Lauzurica and D. Moreno-Salinas, “Gamified project-based learning in vocational education and training computer science courses,” Computers, vol. 15, no. 2, p. 82, 2026
2026
-
[13]
Teaching agile software development: A case study,
V . Devedˇzi´cet al., “Teaching agile software development: A case study,” IEEE transactions on Education, vol. 54, no. 2, pp. 273–278, 2010
2010
-
[14]
Bridg- ing the skills gap through agile methodologies in vocational software development education,
U. Sa’adah, U. L. Yuhana, S. Rochimah, and M. B. A. Rasyid, “Bridg- ing the skills gap through agile methodologies in vocational software development education,”Array, p. 100614, 2025
2025
-
[15]
S. Berrezueta-Guzman, V . Metaj, and S. Wagner, “Beyond the code: A multi-modal assessment strategy for fostering professional com- petencies via introductory programming projects,”arXiv preprint arXiv:2603.18741, 2026
-
[16]
Software requirements testing approaches: a systematic literature review,
J. dos Santos, L. E. G. Martins, V . A. de Santiago J ´unior, L. V . Povoa, and L. B. R. dos Santos, “Software requirements testing approaches: a systematic literature review,”Requirements Engineering, vol. 25, no. 3, pp. 317–337, 2020
2020
-
[17]
How soft skills shape first-year success in higher education,
K. Andree, S. Berrezueta-Guzman, S. Krusche, L. Pufahl, and S. Wag- ner, “How soft skills shape first-year success in higher education,” in 2025 IEEE 4th German Education Conference (GECon), pp. 1–7, IEEE, 2025
2025
-
[18]
Using thematic analysis in psychology,
V . Braun and V . Clarke, “Using thematic analysis in psychology,” Qualitative research in psychology, vol. 3, no. 2, pp. 77–101, 2006
2006
-
[19]
A unified model of requirements elicitation,
A. M. Hickey and A. M. Davis, “A unified model of requirements elicitation,”Journal of management information systems, vol. 20, no. 4, pp. 65–84, 2004
2004
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.