UltraChat supplies 1.5 million high-quality multi-turn dialogues that, when used to fine-tune LLaMA, produce UltraLLaMA, which outperforms prior open-source chat models including Vicuna.
Exploring the impact of instruction data scaling on large language models: An empirical study on real-world use cases.arXiv preprint arXiv:2303.14742
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
citation-role summary
dataset 1
citation-polarity summary
fields
cs.CL 3roles
dataset 1polarities
use dataset 1representative citing papers
Embedding-based QLoRA fine-tuning of causal LLMs matches BERT on single-label patent classification with 10–30x fewer trainable parameters, while instruction-tuning wins on multi-label classification only with ≥100M trainable parameters.
Yi models are 6B and 34B open foundation models pretrained on 3.1T curated tokens that achieve strong benchmark results through data quality and targeted extensions like long context and vision alignment.
citing papers explorer
-
Enhancing Chat Language Models by Scaling High-quality Instructional Conversations
UltraChat supplies 1.5 million high-quality multi-turn dialogues that, when used to fine-tune LLaMA, produce UltraLLaMA, which outperforms prior open-source chat models including Vicuna.
-
Fine-Tuning Causal LLMs for Text Classification: Embedding-Based vs. Instruction-Based Approaches
Embedding-based QLoRA fine-tuning of causal LLMs matches BERT on single-label patent classification with 10–30x fewer trainable parameters, while instruction-tuning wins on multi-label classification only with ≥100M trainable parameters.
-
Yi: Open Foundation Models by 01.AI
Yi models are 6B and 34B open foundation models pretrained on 3.1T curated tokens that achieve strong benchmark results through data quality and targeted extensions like long context and vision alignment.