BERT embeddings and meta-classifiers trained on 172 expert-annotated defeaters from two assurance cases achieve F1≈0.84 in predicting quality ratings, outperforming the low inter-rater agreement (κ<0.442) between human reviewers.
Using ChatGPT for Thematic Analysis
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
The utilisation of AI-driven tools, notably ChatGPT, within academic research is increasingly debated from several perspectives including ease of implementation, and potential enhancements in research efficiency, as against ethical concerns and risks such as biases and unexplained AI operations. This paper explores the use of the GPT model for initial coding in qualitative thematic analysis using a sample of UN policy documents. The primary aim of this study is to contribute to the methodological discussion regarding the integration of AI tools, offering a practical guide to validation for using GPT as a collaborative research assistant. The paper outlines the advantages and limitations of this methodology and suggests strategies to mitigate risks. Emphasising the importance of transparency and reliability in employing GPT within research methodologies, this paper argues for a balanced use of AI in supported thematic analysis, highlighting its potential to elevate research efficacy and outcomes.
fields
cs.SE 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Automating Quality Assessment with NLP of LLM-Generated Defeaters
BERT embeddings and meta-classifiers trained on 172 expert-annotated defeaters from two assurance cases achieve F1≈0.84 in predicting quality ratings, outperforming the low inter-rater agreement (κ<0.442) between human reviewers.