Pith. sign in

REVIEW 5 cited by

TinyGSM: achieving >80% on GSM8k with small language models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.09241 v1 pith:RKJAHZER submitted 2023-12-14 cs.LG cs.CL

Bingbin Liu , Sebastien Bubeck , Ronen Eldan , Janardhan Kulkarni , Yuanzhi Li , Anh Nguyen , Rachel Ward , Yi Zhang This is my paper
classification cs.LGcs.CL
keywords modelmodelstinygsmtextttdatasetgeneratedgpt-3grade
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Small-scale models offer various computational advantages, and yet to which extent size is critical for problem-solving abilities remains an open question. Specifically for solving grade school math, the smallest model size so far required to break the 80\% barrier on the GSM8K benchmark remains to be 34B. Our work studies how high-quality datasets may be the key for small language models to acquire mathematical reasoning. We introduce \texttt{TinyGSM}, a synthetic dataset of 12.3M grade school math problems paired with Python solutions, generated fully by GPT-3.5. After finetuning on \texttt{TinyGSM}, we find that a duo of a 1.3B generation model and a 1.3B verifier model can achieve 81.5\% accuracy, outperforming existing models that are orders of magnitude larger. This also rivals the performance of the GPT-3.5 ``teacher'' model (77.4\%), from which our model's training data is generated. Our approach is simple and has two key components: 1) the high-quality dataset \texttt{TinyGSM}, 2) the use of a verifier, which selects the final outputs from multiple candidate generations.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Training with (Swap) Regret Loss in a Single-Layer Self-Attention Model: A Case Study on the Probability Simplex

    cs.LG 2026-07 conditional novelty 7.0 of 10

    Training single-layer attention with squared regret loss has stationary points that implement smoothed fictitious play (external regret) and, via a new swap-regret loss, the Blum–Mansour no-swap-regret algorithm.

  2. Make me an Expert: Distilling from Generalist Black-Box Models into Specialized Models for Semantic Segmentation

    cs.CV 2025-08 conditional novelty 6.0 of 10

    ATGC selects the best input scale for a black-box open-vocabulary segmentation API, using DINOv2 attention entropy, improving one-hot-label distillation on Cityscapes and ACDC.

  3. Tag-Evol: Achieving Efficient Instruction Evolving via Tag Injection

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Tag-Evol generates harder, more diverse instruction data by injecting sampled knowledge tags into seed instructions, improving downstream SFT accuracy across math, code, and general benchmarks.

  4. SLM-Bench: A Comprehensive Benchmark of Small Language Models on Environmental Impacts--Extended Version

    cs.CL 2025-08 conditional novelty 5.0 of 10

    A new benchmark of 15 small language models across 23 datasets and 11 metrics shows clear accuracy-versus-energy trade-offs, with no single model dominating.

  5. A Survey on Large Language Models for Mathematical Reasoning

    cs.AI 2025-06 conditional novelty 1.0 of 10

    Recent advances in LLM mathematical reasoning are organized into comprehension and generation phases, covering methods from prompting to test-time scaling.

Pith tools