A continually pretrained 7B Japanese pharmaceutical LLM outperforms open medical models on new Japanese pharma benchmarks, while all models, including GPT-4o, fail at cross-sentence consistency checks.
70B-parameter large language models in Japanese medical question-answering
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Since the rise of large language models (LLMs), the domain adaptation has been one of the hot topics in various domains. Many medical LLMs trained with English medical dataset have made public recently. However, Japanese LLMs in medical domain still lack its research. Here we utilize multiple 70B-parameter LLMs for the first time and show that instruction tuning using Japanese medical question-answering dataset significantly improves the ability of Japanese LLMs to solve Japanese medical license exams, surpassing 50\% in accuracy. In particular, the Japanese-centric models exhibit a more significant leap in improvement through instruction tuning compared to their English-centric counterparts. This underscores the importance of continual pretraining and the adjustment of the tokenizer in our local language. We also examine two slightly different prompt formats, resulting in non-negligible performance improvement.
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
A Japanese Language Model and Three New Evaluation Benchmarks for Pharmaceutical NLP
A continually pretrained 7B Japanese pharmaceutical LLM outperforms open medical models on new Japanese pharma benchmarks, while all models, including GPT-4o, fail at cross-sentence consistency checks.