REVIEW 5 cited by
Using Large Language Models to Support Thematic Analysis in Empirical Legal Studies
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Thematic analysis and other variants of inductive coding are widely used qualitative analytic methods within empirical legal studies (ELS). We propose a novel framework facilitating effective collaboration of a legal expert with a large language model (LLM) for generating initial codes (phase 2 of thematic analysis), searching for themes (phase 3), and classifying the data in terms of the themes (to kick-start phase 4). We employed the framework for an analysis of a dataset (n=785) of facts descriptions from criminal court opinions regarding thefts. The goal of the analysis was to discover classes of typical thefts. Our results show that the LLM, namely OpenAI's GPT-4, generated reasonable initial codes, and it was capable of improving the quality of the codes based on expert feedback. They also suggest that the model performed well in zero-shot classification of facts descriptions in terms of the themes. Finally, the themes autonomously discovered by the LLM appear to map fairly well to the themes arrived at by legal experts. These findings can be leveraged by legal researchers to guide their decisions in integrating LLMs into their thematic analyses, as well as other inductive coding projects.
Forward citations
Cited by 5 Pith papers
-
Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Models
Occ-LLM tokenizes 4D occupancy with a motion/static separation VAE and uses Llama-2 to forecast occupancy, plan ego motion, and answer scene questions, reporting state-of-the-art results on nuScenes.
-
LLM-TA: An LLM-Enhanced Thematic Analysis Pipeline for Transcripts from Parents of Children with Congenital Heart Disease
A chunked LLM prompting pipeline for inductive thematic analysis outperforms existing LLM-assisted methods on nine AAOCA parent transcripts but does not yet reach human-level theme quality.
-
Human vs. LLM-Based Thematic Analysis for Digital Mental Health Research: Proof-of-Concept Comparative Study
GPT-4o with RISEN prompts can perform thematic analysis faster and cheaper, but humans still excel at child-code development, excerpt coding, and theme synthesis.
-
From Assistance to Autonomy -- A Researcher Study on the Potential of AI Support for Qualitative Data Analysis
Interviews with 15 HCI researchers show openness to AI in qualitative data analysis under conditions of privacy, control, and reliability, leading to a framework of AI involvement levels from minimal to high.
-
Analyzing Images of Legal Documents: Toward Multi-Modal LLMs for Access to Justice
A pilot study finds GPT-4o extracts 73% of fields from photos of a lease form, with accuracy dropping from 98% on typed copies to 60% on low-quality handwritten photos.
Discussion (0). Continue with ORCID to comment.