Adding lightweight multi-token auxiliary heads with a weighted future-token loss improves supervised fine-tuning of Llama-3-8B-Instruct across five diverse tasks.
L+M-24: Building a Dataset for Language + Molecules @ ACL 2024
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Language-molecule models have emerged as an exciting direction for molecular discovery and understanding. However, training these models is challenging due to the scarcity of molecule-language pair datasets. At this point, datasets have been released which are 1) small and scraped from existing databases, 2) large but noisy and constructed by performing entity linking on the scientific literature, and 3) built by converting property prediction datasets to natural language using templates. In this document, we detail the $\textit{L+M-24}$ dataset, which has been created for the Language + Molecules Workshop shared task at ACL 2024. In particular, $\textit{L+M-24}$ is designed to focus on three key benefits of natural language in molecule design: compositionality, functionality, and abstraction.
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Improving Large Language Models with Concept-Aware Fine-Tuning
Adding lightweight multi-token auxiliary heads with a weighted future-token loss improves supervised fine-tuning of Llama-3-8B-Instruct across five diverse tasks.