Pith. sign in

REVIEW 10 cited by

How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.05492 v4 pith:POXFCKP4 submitted 2023-10-09 cs.CL cs.AIcs.LG

Guanting Dong , Hongyi Yuan , Keming Lu , Chengpeng Li , Mingfeng Xue , Dayiheng Liu , Wei Wang , Zheng Yuan
show 2 more authors
Chang Zhou Jingren Zhou
This is my paper
classification cs.CLcs.AIcs.LG
keywords dataabilitiescompositionamountperformancecodefine-tuninggeneration
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models (LLMs) with enormous pre-training tokens and parameters emerge diverse abilities, including math reasoning, code generation, and instruction following. These abilities are further enhanced by supervised fine-tuning (SFT). While the open-source community has explored ad-hoc SFT for enhancing individual capabilities, proprietary LLMs exhibit versatility across various skills. Therefore, understanding the facilitation of multiple abilities via SFT is paramount. In this study, we specifically focuses on the interplay of data composition between mathematical reasoning, code generation, and general human-aligning abilities during SFT. We propose four intriguing research questions to explore the association between model performance and various factors including data amount, composition ratio, model size and SFT strategies. Our experiments reveal that distinct capabilities scale differently and larger models generally show superior performance with same amount of data. Mathematical reasoning and code generation consistently improve with increasing data amount, whereas general abilities plateau after roughly a thousand samples. Moreover, we observe data composition appears to enhance various abilities under limited data conditions, yet can lead to performance conflicts when data is plentiful. Our findings also suggest the amount of composition data influences performance more than the composition ratio. In analysis of SFT strategies, we find that sequentially learning multiple skills risks catastrophic forgetting. Our proposed Dual-stage Mixed Fine-tuning (DMT) strategy offers a promising solution to learn multiple abilities with different scaling patterns.

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The Gold Medals in an Empty Room: Diagnosing Metalinguistic Reasoning in LLMs with Camlang

    cs.CL 2025-08 conditional novelty 7.0 of 10

    A novel constructed language with explicit grammar and dictionary reveals a large gap between human metalinguistic learning (87%) and the best LLM (47%) on translated CommonsenseQA.

  2. OPERA: Online Data Pruning for Efficient Retrieval Model Adaptation

    cs.IR 2026-03 conditional novelty 6.0 of 10

    Dynamic hierarchical data pruning improves NDCG@10 and Recall@20 for dense retrievers while reaching full performance in half the iterations.

  3. UrbanLLaVA: A Multi-modal Large Language Model for Urban Intelligence with Spatial Reasoning and Understanding

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A fine-tuned small multimodal LLM outperforms much larger general models on urban tasks in a new benchmark, with caveats about benchmark overlap with training data.

  4. Surgery-R1: Advancing Surgical-VQLA with Reasoning Multimodal Large Language Model via Reinforcement Learning

    cs.CV 2025-06 conditional novelty 6.0 of 10

    Surgery-R1 uses supervised fine-tuning and reinforcement fine-tuning to give a surgical visual question answering model chain-of-thought reasoning, improving accuracy and localization on two EndoVis benchmarks.

  5. SOS-LoRA: Static Orthogonal-Subspace Low-Rank Adaptation with Fixed Multi-Scale Scaling

    cs.LG 2026-06 conditional novelty 5.0 of 10

    A LoRA update split into several fixed, differently-scaled low-rank experts with orthogonal input directions improves fine-tuning accuracy at the same parameter count.

  6. SafeLLM: Unlearning Harmful Outputs from Large Language Models against Jailbreak Attacks

    cs.LG 2025-08 unverdicted novelty 5.0 of 10

    SafeLLM detects unsafe outputs, traces them to specific feedforward-network components, and applies constrained optimization to unlearn harmful generation while preserving general capability.

  7. Rewrite-to-Rank: Optimizing Ad Visibility via Retrieval-Aware Text Rewriting

    cs.CL 2025-07 conditional novelty 5.0 of 10

    PPO-trained rewriting of ads improves retrieval rank and LLM inclusion on a custom ad dataset, measured by two new delta metrics.

  8. Probabilistic Concept-Aware Steering for Trustworthy LLM Inference

    cs.AI 2026-05 reject novelty 4.0 of 10

    PCS improves steering direction accuracy by adaptively sampling the intervention coefficient from a cosine-similarity-conditioned Gaussian, but its evaluation is partly circular because the optimal coefficient is chos...

  9. A Novel Self-Evolution Framework for Large Language Models

    cs.CL 2025-07 reject novelty 4.0 of 10

    A dual-phase framework that uses a Censor satisfaction scorer to expand training data and then applies SFT plus frequency-weighted DPO, reporting benchmark gains over SFT, PO, and memory baselines.

  10. Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges

    cs.AI 2025-07 reject novelty 1.0 of 10

    A broad survey of LLM alignment that catalogs objectives, benchmarks, SFT/RLHF/DPO methods, and safety challenges, without contributing new experimental or theoretical results.

Pith tools