Small language models with a targeted prompting scheme detect seven test-smell types in natural-language Ubuntu manual tests, with pass@2 scores of 90-97% across three models.
NALABS: Detecting Bad Smells in Natural Language Requirements and Test Specifications
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
In large-scale embedded system development, requirement and test specifications are often expressed in natural language. In the context of developing such products, requirement review is performed in many cases manually using these specifications as a basis for quality assurance. Low-quality specifications can have expensive consequences during the requirement engineering process. Especially, if feedback loops during requirement engineering are long, leading to artifacts that are not easily maintainable, are hard to understand, and are inefficient to port to other system variants. We use the idea of smells to specifications expressed in natural language, defining a set of specifications for bad smells. We developed a tool called NALABS (NAtural LAnguage Bad Smells), available on https://github.com/eduardenoiu/NALABS and used for automatically checking specifications. We discuss some of the decisions made for its implementation, and future work.
citation-role summary
citation-polarity summary
fields
cs.SE 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
Investigating the Performance of Small Language Models in Detecting Test Smells in Manual Test Cases
Small language models with a targeted prompting scheme detect seven test-smell types in natural-language Ubuntu manual tests, with pass@2 scores of 90-97% across three models.