Pith. sign in

REVIEW 12 cited by

Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2505.03469 v2 pith:LU64E74F submitted 2025-05-06 cs.CL

Bin Yu , Hang Yuan , Haotian Li , Xueyin Xu , Yuliang Wei , Bailing Wang , Weizhen Qi , Kai Chen This is my paper
classification cs.CL
keywords modelsreasoningfine-tuningsupervisedchain-of-thoughtlargeapproachcapabilities
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recent advances in large language models have demonstrated that Supervised Fine-Tuning (SFT) with Chain-of-Thought (CoT) reasoning data distilled from large reasoning models (e.g., DeepSeek R1) can effectively transfer reasoning capabilities to non-reasoning models. However, models fine-tuned with this approach inherit the "overthinking" problem from teacher models, producing verbose and redundant reasoning chains during inference. To address this challenge, we propose Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning (LS-Mixture SFT), which combines long CoT reasoning dataset with their short counterparts obtained through structure-preserved rewriting. Our experiments demonstrate that models trained using the LS-Mixture SFT method, compared to those trained with direct SFT, achieved an average accuracy improvement of 2.3% across various benchmarks while substantially reducing model response length by approximately 47.61%. This work offers an approach to endow non-reasoning models with reasoning capabilities through supervised fine-tuning while avoiding the inherent overthinking problems inherited from teacher models, thereby enabling efficient reasoning in the fine-tuned models.

Discussion (0). Sign in to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Post Reasoning: Improving the Performance of Non-Thinking Models at No Cost

    cs.AI 2026-05 conditional novelty 7.0 of 10

    Post-Reasoning boosts LLM accuracy by reversing the usual answer-after-reasoning order, delivering mean relative gains of 17.37% across 117 model-benchmark pairs with zero extra cost.

  2. ReasonAlloc: Hierarchical Decoding-Time KV Cache Budget Allocation for Reasoning Models

    cs.AI 2026-06 unverdicted novelty 6.0 of 10

    ReasonAlloc introduces a hierarchical decoding-time KV cache budget allocation framework that outperforms uniform and other baselines on math reasoning tasks at small cache budgets.

  3. Data Difficulty and the Generalization--Extrapolation Tradeoff in LLM Fine-Tuning

    cs.LG 2026-05 unverdicted novelty 6.0 of 10

    For a fixed data budget in LLM supervised fine-tuning, optimal data difficulty shifts toward harder examples as the budget grows because of the tradeoff between in-distribution generalization gap and extrapolation gap.

  4. Compute Where it Counts: Self Optimizing Language Models

    cs.LG 2026-05 unverdicted novelty 6.0 of 10

    SOL trains a policy to dynamically control multiple efficiency mechanisms per token via group-relative policy optimization on teacher-forced episodes, yielding better quality at matched average budget than static or r...

  5. ConPress: Learning Efficient Reasoning from Multi-Question Contextual Pressure

    cs.CL 2026-02 conditional novelty 6.0 of 10

    Multi-question prompts elicit shorter chain-of-thought traces, and fine-tuning on those traces transfers the compression to single-question reasoning.

  6. SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD

    cs.CL 2026-07 conditional novelty 5.0 of 10

    An Ascend-NPU training stack reaches 34.22% MFU on DeepSeek-V4-Pro, and a solver-verified CPT+SFT recipe raises OR benchmark averages to 71.81% (Flash) and 77.33% (Pro).

  7. D$^2$Evo: Dual Difficulty-Aware Self-Evolution for Data-Efficient Reinforcement Learning

    cs.LG 2026-05 unverdicted novelty 5.0 of 10

    D²Evo mines medium-difficulty anchors from the current model, trains a Questioner to generate matching questions, and jointly optimizes Solver and Questioner for progressive gains, outperforming baselines on math reas...

  8. Data Difficulty and the Generalization--Extrapolation Tradeoff in LLM Fine-Tuning

    cs.LG 2026-05 unverdicted novelty 5.0 of 10

    Optimal data difficulty for LLM supervised fine-tuning shifts toward harder examples as data budget increases due to the generalization-extrapolation tradeoff.

  9. Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs

    cs.AI 2025-07 conditional novelty 5.0 of 10

    The survey's L1/L2 taxonomy and benchmark show that current reasoning models waste compute on easy problems and underthink hard ones, motivating more adaptive inference.

  10. Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models

    cs.CL 2025-03 accept novelty 5.0 of 10

    A survey organizing techniques to achieve efficient reasoning in LLMs by shortening chain-of-thought outputs.

  11. From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI

    cs.AI 2026-06 conditional novelty 4.0 of 10

    Autonomous AI becomes dependable when tool use is embedded in persistent workspaces with reusable skills, shifting evaluation from answers to task closure.

  12. Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.

Pith tools