REVIEW 10 cited by
Specializing Smaller Language Models towards Multi-Step Reasoning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
abstract
The surprising ability of Large Language Models (LLMs) to perform well on complex reasoning with only few-shot chain-of-thought prompts is believed to emerge only in very large-scale models (100+ billion parameters). We show that such abilities can, in fact, be distilled down from GPT-3.5 ($\ge$ 175B) to T5 variants ($\le$ 11B). We propose model specialization, to specialize the model's ability towards a target task. The hypothesis is that large models (commonly viewed as larger than 100B) have strong modeling power, but are spread on a large spectrum of tasks. Small models (commonly viewed as smaller than 10B) have limited model capacity, but if we concentrate their capacity on a specific target task, the model can achieve a decent improved performance. We use multi-step math reasoning as our testbed because it is a very typical emergent ability. We show two important aspects of model abilities: (1). there exists a very complex balance/ tradeoff between language models' multi-dimensional abilities; (2). by paying the price of decreased generic ability, we can clearly lift up the scaling curve of models smaller than 10B towards a specialized multi-step math reasoning ability. We further give comprehensive discussions about important design choices for better generalization, including the tuning data format, the start model checkpoint, and a new model selection method. We hope our practice and discoveries can serve as an important attempt towards specialized smaller models in the new research paradigm set by LLMs.
Forward citations
Cited by 10 Pith papers
-
Rationales Are Not Silver Bullets: Measuring the Impact of Rationales on Model Performance and Reliability
Rationale-augmented finetuning can hurt accuracy while improving calibration, with the sizes of both effects tied linearly to task difficulty.
-
LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks
Training a language model by distilling a coach's written experiential knowledge beats training on a scalar rubric score for open-ended tasks, with better out-of-distribution transfer.
-
MaskSearch: A Universal Pre-Training Framework to Enhance Agentic Search Capability
A pre-training task called RAMP, where models practice searching to fill masked text spans, improves downstream agentic open-domain QA performance across Qwen and LLaMA models.
-
Enhancing LLMs' Reasoning-Intensive Multimedia Search Capabilities through Fine-Tuning and Reinforcement Learning
A two-stage fine-tuning and reinforcement-learning method makes LLMs generate token-efficient natural-language search plans, reporting strong accuracy gains on financial and news search benchmarks.
-
CDS: Knowledge Component-Driven Data Synthesis Guided by Cognitive Diagnosis Theory
A knowledge-component diagnostic pipeline, inspired by cognitive diagnosis theory, generates weakness-targeted synthetic data that improves small LLMs on math, code, and exam benchmarks by up to 13.1 percentage points.
-
RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarios
RuleArena evaluates LLMs on realistic rule-guided reasoning and finds that even o1-preview solves only about half of the easiest problems and near zero of the hardest.
-
Honey, I Shrunk the Language Model: Impact of Knowledge Distillation Methods on Performance and Explainability
A small language model distilled with a combination of multitask and counterfactual training on critique-revised explanations produced human-rated higher-quality explanations, while multitask training alone gave the b...
-
Enhancing Generalization in Chain of Thought Reasoning for Smaller Models
PRADA combines P-Tuning and domain-adversarial training with CoT distillation and claims improved cross-domain reasoning in small models, though the evaluation is confounded by target-data access.
-
LLMQuoter: Enhancing RAG Capabilities Through Efficient Quote Extraction From Large Contexts
LLMQuoter uses a distilled 3B model to extract quotes for RAG; the paper shows gold quotes greatly improve QA, but does not test its own model's quotes end-to-end.
-
Enhancing the Reasoning Capabilities of Small Language Models via Solution Guidance Fine-Tuning
SGFT fine-tunes a small model to produce calculation-free solution plans and uses a second model to answer from them, outperforming CoT fine-tuning with roughly 3% of the training data.
Discussion (0). Continue with ORCID to comment.