A feature-model-driven LLM pipeline that generates synthetic requirements data improves defect classification when combined with real data, but the headline gains rest on a 40-sample test set with high variance.
Which AI Technique Is Better to Classify Requirements? An Experiment with SVM, LSTM, and ChatGPT
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Recently, Large Language Models like ChatGPT have demonstrated remarkable proficiency in various Natural Language Processing tasks. Their application in Requirements Engineering, especially in requirements classification, has gained increasing interest. This paper reports an extensive empirical evaluation of two ChatGPT models, specifically gpt-3.5-turbo, and gpt-4 in both zero-shot and few-shot settings for requirements classification. The question arises as to how these models compare to traditional classification methods, specifically Support Vector Machine and Long Short-Term Memory. Based on five different datasets, our results show that there is no single best technique for all types of requirement classes. Interestingly, the few-shot setting has been found to be beneficial primarily in scenarios where zero-shot results are significantly low.
fields
cs.SE 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Synthline: A Product Line Approach for Synthetic Requirements Engineering Data Generation using Large Language Models
A feature-model-driven LLM pipeline that generates synthetic requirements data improves defect classification when combined with real data, but the headline gains rest on a 40-sample test set with high variance.