Pith. sign in

REVIEW 10 cited by

COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.18058 v2 pith:ESGZ4K6W submitted 2024-03-26 cs.CL cs.AI

Yuelin Bai , Xinrun Du , Yiming Liang , Yonggang Jin , Junting Zhou , Ziqiang Liu , Feiteng Fang , Mingshan Chang
show 14 more authors
Tianyu Zheng Xincheng Zhang Nuo Ma Zekun Wang Ruibin Yuan Haihong Wu Hongquan Lin Wenhao Huang Jiajun Zhang Chenghua Lin Jie Fu Min Yang Shiwen Ni Ge Zhang
This is my paper · ORCID
classification cs.CLcs.AI
keywords chinesecoig-cqiadatasetsinstructionmodelstuningdatasetllms
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Remarkable progress on English instruction tuning has facilitated the efficacy and reliability of large language models (LLMs). However, there remains a noticeable gap in instruction tuning for Chinese, where the complex linguistic features pose significant challenges. Existing datasets, generally distilled from English-centric LLMs, are not well-aligned with Chinese users' interaction patterns. To bridge this gap, we introduce COIG-CQIA, a new Chinese instruction tuning dataset derived from various real-world resources and undergoing rigorous human verification. We conduct extensive experiments on COIG-CQIA, and compare them with strong baseline models and datasets. The experimental results show that models trained on COIG-CQIA achieve highly competitive performance in diverse benchmarks. Additionally, our findings offer several insights for designing effective Chinese instruction-tuning datasets and data-mixing strategies. Our dataset are available at https://huggingface.co/datasets/m-a-p/COIG-CQIA.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Chumor 2.0: Towards Benchmarking Chinese Humor Understanding

    cs.CL 2024-12 conditional novelty 7.0 of 10

    Chumor is a new Chinese humor explanation benchmark where LLMs perform near chance (best accuracy 60.3%) and well below human accuracy (78.3%).

  2. Improving Natural Language Understanding for LLMs via Large-Scale Instruction Synthesis

    cs.CL 2025-02 conditional novelty 6.0 of 10

    A large synthetic instruction corpus with guidelines, preference rules, and format variants improves LLM performance on five NLU benchmarks by an average of 3.1%.

  3. Mono-InternVL-1.5: Towards Cheaper and Faster Monolithic Multimodal Large Language Models

    cs.CV 2025-07 conditional novelty 5.5 of 10

    A monolithic multimodal LLM that cuts pre-training data by 58% and first-token latency by up to 69% while matching or beating its predecessor on 15 benchmarks.

  4. CCI4.0: A Bilingual Pretraining Dataset for Enhancing Reasoning in Large Language Models

    cs.CL 2025-06 reject novelty 5.0 of 10

    A new 35 TB bilingual pretraining dataset with 4.5 billion chain-of-thought templates is described, but the evidence for its benefits is marginal, confounded, and contradicted by the paper's own tables.

  5. Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Training

    cs.AI 2025-05 conditional novelty 5.0 of 10

    Reinforcing the two experts most correlated with thinking tokens improves reasoning accuracy and efficiency in MoE large reasoning models, with gains of up to 10 points on AIME benchmarks.

  6. A Plug-and-Play Method for Improving Imperceptibility and Capacity in Practical Generative Text Steganography

    cs.CR 2024-12 conditional novelty 5.0 of 10

    FreStega adjusts language model token probabilities at decoding time, using entropy-based temperature and target-domain token frequency alignment, improving steganalysis resistance and increasing embedding rate by abo...

  7. Small Language Model as Data Prospector for Large Language Model

    cs.CL 2024-12 conditional novelty 5.0 of 10

    SuperNUGGETS uses small language models and a refined 100-example test set to pick instruction data, matching the quality of the LLM-based NUGGETS at far lower cost.

  8. GMAI-VL & GMAI-VL-5.5M: A Large Vision-Language Model and A Comprehensive Multimodal Dataset Towards General Medical AI

    cs.CV 2024-11 reject novelty 5.0 of 10

    GMAI-VL-5.5M is a new 5.5M-sample medical image-text dataset built from 219 datasets via GPT-4o annotation-guided generation, and GMAI-VL is a three-stage LLaVA-style model reporting SOTA numbers, though the evaluatio...

  9. Educational-Psychological Dialogue Robot Based on Multi-Agent Collaboration

    cs.CL 2024-12 reject novelty 4.0 of 10

    A multi-agent dialogue system for education and psychological counseling, with benchmark results only for the educational component.

  10. Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models

    cs.CL 2025-06 reject novelty 3.0 of 10

    A survey of LLM hallucination research that formalizes hallucination types and argues, via incompleteness and undecidability arguments, that hallucinations cannot be fully eliminated.

Pith tools