Pith. sign in

REVIEW 5 cited by

The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2305.14045 v2 pith:IFFK623L submitted 2023-05-23 cs.CL cs.AIcs.LG

Seungone Kim , Se June Joo , Doyoung Kim , Joel Jang , Seonghyeon Ye , Jamin Shin , Minjoon Seo This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords collectionflan-t5taskscapabilitieschain-of-thoughtfew-shotfine-tuningimprovement
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Language models (LMs) with less than 100B parameters are known to perform poorly on chain-of-thought (CoT) reasoning in contrast to large LMs when solving unseen tasks. In this work, we aim to equip smaller LMs with the step-by-step reasoning capability by instruction tuning with CoT rationales. In order to achieve this goal, we first introduce a new instruction-tuning dataset called the CoT Collection, which augments the existing Flan Collection (including only 9 CoT tasks) with additional 1.84 million rationales across 1,060 tasks. We show that CoT fine-tuning Flan-T5 (3B & 11B) with CoT Collection enables smaller LMs to have better CoT capabilities on unseen tasks. On the BIG-Bench-Hard (BBH) benchmark, we report an average improvement of +4.34% (Flan-T5 3B) and +2.60% (Flan-T5 11B), in terms of zero-shot task accuracy. Furthermore, we show that instruction tuning with CoT Collection allows LMs to possess stronger few-shot learning capabilities on 4 domain-specific tasks, resulting in an improvement of +2.24% (Flan-T5 3B) and +2.37% (Flan-T5 11B), even outperforming ChatGPT utilizing demonstrations until the max length by a +13.98% margin. Our code, the CoT Collection data, and model checkpoints are publicly available.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. One Token to Fool LLM-as-a-Judge

    cs.LG 2025-07 conditional novelty 6.0 of 10

    LLM reward models falsely accept empty 'master key' responses such as ':' or 'Thought process:' across many models, and a fine-tuning defense reduces these false positives to near zero.

  2. The System Description of CPS Team for Track on Driving with Language of CVPR 2024 Autonomous Grand Challenge

    cs.CV 2025-09 conditional novelty 4.0 of 10

    A competition report showing that LLaVA fine-tuned with LoRA, enhanced with depth cues and answer fusion, scored 0.7799 on the DriveLM-nuScenes validation set.

  3. AI Reasoning for Wireless Communications and Networking: A Survey and Perspectives

    cs.NI 2025-09 conditional novelty 4.0 of 10

    A survey that organizes LLM and AI reasoning methods into a taxonomy and maps them onto the physical, link, network, transport, and application layers of wireless networks.

  4. Investigating the Effects of Cognitive Biases in Prompts on Large Language Model Outputs

    cs.CL 2025-06 conditional novelty 4.0 of 10

    Injecting explicit suggestions or biased recollections into prompts reduces LLM accuracy on multiple-choice QA tasks, and attention weights shift toward the suggested answer.

  5. MASTER: Enhancing Large Language Model via Multi-Agent Simulated Teaching

    cs.CL 2025-06 conditional novelty 4.0 of 10

    A multi-agent simulated teaching pipeline creates BOOST-QA, and fine-tuning on it lifts reported LLM benchmark scores by up to 31 points over the original data.

Pith tools