REVIEW 5 cited by
Language Models for Text Classification: Is In-Context Learning Enough?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent foundational language models have shown state-of-the-art performance in many NLP tasks in zero- and few-shot settings. An advantage of these models over more standard approaches based on fine-tuning is the ability to understand instructions written in natural language (prompts), which helps them generalise better to different tasks and domains without the need for specific training data. This makes them suitable for addressing text classification problems for domains with limited amounts of annotated instances. However, existing research is limited in scale and lacks understanding of how text generation models combined with prompting techniques compare to more established methods for text classification such as fine-tuning masked language models. In this paper, we address this research gap by performing a large-scale evaluation study for 16 text classification datasets covering binary, multiclass, and multilabel problems. In particular, we compare zero- and few-shot approaches of large language models to fine-tuning smaller language models. We also analyse the results by prompt, classification type, domain, and number of labels. In general, the results show how fine-tuning smaller and more efficient language models can still outperform few-shot approaches of larger language models, which have room for improvement when it comes to text classification.
Forward citations
Cited by 5 Pith papers
-
From scratch to silver: Creating trustworthy training data for patent-SDG classification using Large Language Models
A weak-supervision pipeline using LLM-extracted concepts and rank fusion creates silver-standard patent-to-SDG labels that recover known citation-derived associations and show high network modularity.
-
Measuring Cognitive Engagement in Collaborative Discourse with an Extended ICAP Framework: Comparing Human Annotation, In-Context Learning, and Reflective LLM Agents
Trained human coders agree more with each other than LLM labelers do on a 7-point ICAP cognitive-engagement scale, and human-refined coding rules do not transfer to LLM prompting.
-
A Multi-Stage Large Language Model Framework for Extracting Suicide-Related Social Determinants of Health
A multi-stage LLM pipeline improves extraction of infrequent suicide-related social determinants from death narratives, but some evaluation results are compromised by using the test set to tune the system.
-
Poodle: Seamlessly Scaling Down Large Language Models with Just-in-Time Model Replacement
Poodle shows that LLMs can be automatically replaced with cheaper models for recurring tasks to save significant cost and energy without extra user effort.
-
From Domain Documents to Requirements: Retrieval-Augmented Generation in the Space Industry
A RAG-based pipeline with neural label classification generates draft space requirements from mission documents, shown in a single qualitative case study.
Discussion (0). Sign in to comment.