Pith. sign in

REVIEW 3 cited by

A Survey of Small Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.20011 v1 pith:N4JWG25L submitted 2024-10-25 cs.CL

Chien Van Nguyen , Xuan Shen , Ryan Aponte , Yu Xia , Samyadeep Basu , Zhengmian Hu , Jian Chen , Mihir Parmar
show 20 more authors
Sasidhar Kunapuli Joe Barrow Junda Wu Ashish Singh Yu Wang Jiuxiang Gu Franck Dernoncourt Nesreen K. Ahmed Nedim Lipka Ruiyi Zhang Xiang Chen Tong Yu Sungchul Kim Hanieh Deilamsalehy Namyong Park Mike Rimer Zhehao Zhang Huanrui Yang Ryan A. Rossi Thien Huu Nguyen
This is my paper
classification cs.CL
keywords languageslmsmodelssmallsurveytechniquescompressionincluding
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Small Language Models (SLMs) have become increasingly important due to their efficiency and performance to perform various language tasks with minimal computational resources, making them ideal for various settings including on-device, mobile, edge devices, among many others. In this article, we present a comprehensive survey on SLMs, focusing on their architectures, training techniques, and model compression techniques. We propose a novel taxonomy for categorizing the methods used to optimize SLMs, including model compression, pruning, and quantization techniques. We summarize the benchmark datasets that are useful for benchmarking SLMs along with the evaluation metrics commonly used. Additionally, we highlight key open challenges that remain to be addressed. Our survey aims to serve as a valuable resource for researchers and practitioners interested in developing and deploying small yet efficient language models.

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Punching Above Their Weight: Classification-Head Fine-Tuning of Tiny Language Models (TLMs) for Verifiable Multiple-Choice Tasks

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Classification-head LoRA fine-tuning of sub-3B Qwen3 models outperforms label-generation SFT by 2–3% on HellaSwag, WinoGrande and PIQA and yields SOTA numbers competitive with GPT-3/PaLM/GPT-4.

  2. Investigating the Performance of Small Language Models in Detecting Test Smells in Manual Test Cases

    cs.SE 2025-07 conditional novelty 5.0 of 10

    Small language models with a targeted prompting scheme detect seven test-smell types in natural-language Ubuntu manual tests, with pass@2 scores of 90-97% across three models.

  3. Advancing SLM Tool-Use Capability using Reinforcement Learning

    cs.CL 2025-09 reject novelty 4.0 of 10

    GRPO with a strict reward for structured JSON output improves tool-call accuracy on small models, from 0.98%-6.1% to 22%-71% on the xLAM benchmark, though without baselines or variance.

Pith tools