Pith. sign in

REVIEW 26 cited by

Fine-Tuned 'Small' LLMs (Still) Significantly Outperform Zero-Shot Generative AI Models in Text Classification

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.08660 v2 pith:6UWFNICG submitted 2024-06-12 cs.CL cs.AI

Martin Juan José Bucher , Marco Martini This is my paper · ORCID
classification cs.CLcs.AI
keywords classificationllmstextfine-tunedgenerativemodelstrainingchatgpt
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Generative AI offers a simple, prompt-based alternative to fine-tuning smaller BERT-style LLMs for text classification tasks. This promises to eliminate the need for manually labeled training data and task-specific model training. However, it remains an open question whether tools like ChatGPT can deliver on this promise. In this paper, we show that smaller, fine-tuned LLMs (still) consistently and significantly outperform larger, zero-shot prompted models in text classification. We compare three major generative AI models (ChatGPT with GPT-3.5/GPT-4 and Claude Opus) with several fine-tuned LLMs across a diverse set of classification tasks (sentiment, approval/disapproval, emotions, party positions) and text categories (news, tweets, speeches). We find that fine-tuning with application-specific training data achieves superior performance in all cases. To make this approach more accessible to a broader audience, we provide an easy-to-use toolkit alongside this paper. Our toolkit, accompanied by non-technical step-by-step guidance, enables users to select and fine-tune BERT-like LLMs for any classification task with minimal technical and computational effort.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 26 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Truthful Text Sanitization Guided by Inference Attacks

    cs.CL 2024-12 conditional novelty 7.0 of 10

    INTACT generates abstraction-sorted replacement candidates for sensitive spans and selects the most specific candidate that resists LLM-based inference attacks, achieving a strong privacy-utility trade-off on the Text...

  2. Enhancing Few-Shot Vision-Language Classification with Large Multimodal Model Features

    cs.CV 2024-11 conditional novelty 7.0 of 10

    SAVs extract a sparse set of attention head outputs from a frozen large multimodal model and use them as nearest-centroid features, achieving state-of-the-art few-shot vision-language classification without finetuning.

  3. Fine-Tuning Large Language Models for Codebook-Guided Coding of Students' Mathematics Metaphor Responses

    cs.HC 2026-08 conditional novelty 6.0 of 10

    LoRA fine-tuning makes compact open-weight LLMs competitive with, and often better than, prompt-only proprietary GPT models for codebook-guided coding of students' math metaphors, while improving run-to-run reliability.

  4. DLT-Corpus: A Large-Scale Text Collection for the Distributed Ledger Technology Domain

    cs.CL 2026-02 conditional novelty 6.0 of 10

    DLT-Corpus is a 2.98B-token scientific/patent/Twitter corpus for blockchain NLP, plus LedgerBERT (+23% NER vs BERT), a sentiment dataset, and cross-domain innovation-diffusion analyses.

  5. PerSoMed: A Large-Scale Balanced Dataset for Persian Social Media Text Classification

    cs.CL 2026-02 conditional novelty 6.0 of 10

    PerSoMed, a balanced nine-class dataset of 36,000 Persian social media posts, with benchmark results showing TookaBERT-Large at F1 0.962.

  6. Few-Shot Vision-Language Reasoning for Satellite Imagery via Verifiable Rewards

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A few-shot RLVR method using only rule-based rewards lifts a 2B vision-language model's remote sensing accuracy by double digits, with 128 examples rivaling thousands.

  7. Evaluating Apple Intelligence's Writing Tools for Privacy Against Large Language Model-Based Inference Attacks: Insights from Early Datasets

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Apple Intelligence's Friendly and Professional text rewrites can substantially reduce LLM-based emotion inference accuracy in small early datasets.

  8. ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editing

    cs.CV 2025-06 conditional novelty 6.0 of 10

    ReSpace is an autoregressive LLM framework for text-driven 3D indoor scene editing and synthesis, using a structured JSON scene representation and a voxelization-based layout metric.

  9. Analyzing Political Bias in LLMs via Target-Oriented Sentiment Classification

    cs.CL 2025-05 conditional novelty 6.0 of 10

    LLMs show systematic target-dependent sentiment inconsistency that is politically biased: left and center politicians rated more positively, far-right politicians more negatively, with stronger effects in larger model...

  10. Adapting Pretrained Language Models for Citation Classification via Self-Supervised Contrastive Learning

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Citss combines sentence-level cropping and keyphrase perturbation with contrastive learning to fine-tune both encoder and decoder language models for citation classification.

  11. Extracting Participation in Collective Action from Social Media

    cs.SI 2025-01 conditional novelty 6.0 of 10

    A new classifier suite detects and levels expressions of collective action participation in Reddit comments, reaching weighted F1=0.71 for binary detection.

  12. Polish-English medical knowledge transfer: A new benchmark and results

    cs.CL 2024-11 conditional novelty 6.0 of 10

    A new Polish-English medical exam benchmark shows GPT-4o answering at or above average human level, with smaller and medical-specific models lagging and persistent cross-lingual gaps.

  13. A Supervised Machine Learning Approach for Assessing Grant Peer Review Reports

    econ.EM 2024-11 conditional novelty 6.0 of 10

    Fine-tuned transformer models classify Swiss National Science Foundation grant peer review sentences into twelve content categories with an average macro F1 of 0.85, enabling large-scale analysis of review reports.

  14. Multi-Lingual Implicit Discourse Relation Recognition with Multi-Label Hierarchical Learning

    cs.CL 2025-08 conditional novelty 5.0 of 10

    HArch, a hierarchical multi-task model, is the first to recognize implied discourse relations with multi-label sense distributions in four languages, and it outperforms few-shot LLM prompting.

  15. A Multi-Stage Large Language Model Framework for Extracting Suicide-Related Social Determinants of Health

    cs.CL 2025-08 conditional novelty 5.0 of 10

    A multi-stage LLM pipeline improves extraction of infrequent suicide-related social determinants from death narratives, but some evaluation results are compromised by using the test set to tune the system.

  16. How and Where to Translate? The Impact of Translation Strategies in Cross-lingual LLM Prompting

    cs.CL 2025-07 conditional novelty 5.0 of 10

    For multilingual RAG intent classification, the best translation strategy depends on the model and language; translating instructions into the user's language helps some models, while making the model answer in low-re...

  17. Rethinking Hate Speech Detection on Social Media: Can LLMs Replace Traditional Models?

    cs.CL 2025-06 conditional novelty 5.0 of 10

    On three hate speech datasets, including a new code-mixed IndoHateMix benchmark, fine-tuned LLMs such as LLaMA-3.1 beat multilingual BERT models, with the largest gains on code-mixed Indian text.

  18. Data Quality Enhancement on the Basis of Diversity with Large Language Models for Text Classification: Uncovered, Difficult, and Noisy

    cs.CL 2024-12 conditional novelty 5.0 of 10

    DQE selects about half of a training set via greedy sampling plus similarity-based categorization of uncovered, difficult, and noisy samples, and reports improved text classification accuracy over full-data fine-tuning.

  19. AIx4Soccer: A Unified Platform Architecture for Football Club Management and Structured Athlete Development

    cs.CY 2026-07 conditional novelty 4.0 of 10

    A conceptual multi-tenant football-club SaaS design embeds a PDI development cycle and a 75/25 certified video-analyst marketplace on a proposed event-sourced knowledge-graph substrate.

  20. A Scalable and Efficient Signal Integration System for Job Matching

    cs.LG 2025-07 conditional novelty 4.0 of 10

    STAR integrates fine-tuned LLM embeddings as node features into a large-scale GNN, improving job matching metrics across three LinkedIn products.

  21. DS@GT at CheckThat! 2025: Ensemble Methods for Detection of Scientific Discourse on Social Media

    cs.CL 2025-07 conditional novelty 4.0 of 10

    The DS@GT system achieved 0.8611 macro-F1 on the CheckThat! 2025 Task 4a development set by combining a fine-tuned DeBERTa model with GPT-4o few-shot prompting, outperforming the DeBERTaV3 baseline of 0.8375.

  22. FinSage: A Multi-aspect RAG System for Financial Filings Question Answering

    cs.IR 2025-04 reject novelty 4.0 of 10

    FinSage reports a 92.51% retrieval recall and a 49.66% financial QA accuracy, but the main private-dataset metrics are in-sample and no code or data is released.

  23. AdaPhish: AI-Powered Adaptive Defense and Education Resource Against Deceptive Emails

    cs.CR 2025-02 conditional novelty 4.0 of 10

    An LLM-based phish bowl that automatically anonymizes reported phishing emails and combines nearest-neighbor retrieval with GPT-4o classification to detect and track new phishing campaigns.

  24. Advancing Single and Multi-task Text Classification through Large Language Model Fine-tuning

    cs.CL 2024-12 conditional novelty 4.0 of 10

    Fully fine-tuned Llama3-70B beats RoBERTa-large and other LLMs on 20 Newsgroups and MASSIVE, and a single consolidated multi-task LLM matches separated dual models.

  25. Practical Considerations for Agentic LLM Systems

    cs.AI 2024-12 conditional novelty 3.0 of 10

    This paper is a practical survey that organizes research on LLM-based agents into design considerations for planning, memory, tools, and control flow.

  26. Towards Robust Few-Shot Text Classification Using Transformer Architectures and Dual Loss Strategies

    cs.CL 2025-05 reject novelty 2.0 of 10

    Combining cross-entropy with contrastive loss and L2 regularization reportedly raises few-shot text classification accuracy on FewRel 2.0, but the method is standard and the evaluation is under-specified.

Pith tools