Pith. sign in

REVIEW 2 cited by

Which AI Technique Is Better to Classify Requirements? An Experiment with SVM, LSTM, and ChatGPT

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2311.11547 v2 pith:AQP56QZP submitted 2023-11-20 cs.AI cs.SE

Abdelkarim El-Hajjami , Nicolas Fafin , Camille Salinesi This is my paper · ORCID
classification cs.AIcs.SE
keywords requirementschatgptclassificationmodelsfew-shotlanguageresultsspecifically
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Recently, Large Language Models like ChatGPT have demonstrated remarkable proficiency in various Natural Language Processing tasks. Their application in Requirements Engineering, especially in requirements classification, has gained increasing interest. This paper reports an extensive empirical evaluation of two ChatGPT models, specifically gpt-3.5-turbo, and gpt-4 in both zero-shot and few-shot settings for requirements classification. The question arises as to how these models compare to traditional classification methods, specifically Support Vector Machine and Long Short-Term Memory. Based on five different datasets, our results show that there is no single best technique for all types of requirement classes. Interestingly, the few-shot setting has been found to be beneficial primarily in scenarios where zero-shot results are significantly low.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The Few-shot Dilemma: Over-prompting Large Language Models

    cs.CL 2025-09 conditional novelty 5.0 of 10

    Across seven LLMs on two requirements datasets, F1 scores rise then fall as more few-shot examples are added, and TF-IDF-selected examples at small counts match or beat larger prompts, including a 1% gain over prior SOTA.

  2. Synthline: A Product Line Approach for Synthetic Requirements Engineering Data Generation using Large Language Models

    cs.SE 2025-05 conditional novelty 5.0 of 10

    A feature-model-driven LLM pipeline that generates synthetic requirements data improves defect classification when combined with real data, but the headline gains rest on a 40-sample test set with high variance.

Pith tools