REVIEW 8 cited by
BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicine
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Foundation models (FMs) have exhibited remarkable performance across a wide range of downstream tasks in many domains. Nevertheless, general-purpose FMs often face challenges when confronted with domain-specific problems, due to their limited access to the proprietary training data in a particular domain. In biomedicine, there are various biological modalities, such as molecules, proteins, and cells, which are encoded by the language of life and exhibit significant modality gaps with human natural language. In this paper, we introduce BioMedGPT, an open multimodal generative pre-trained transformer (GPT) for biomedicine, to bridge the gap between the language of life and human natural language. BioMedGPT allows users to easily ``communicate'' with diverse biological modalities through free text, which is the first of its kind. BioMedGPT aligns different biological modalities with natural language via a large generative language model, namely, BioMedGPT-LM. We publish BioMedGPT-10B, which unifies the feature spaces of molecules, proteins, and natural language via encoding and alignment. Through fine-tuning, BioMedGPT-10B outperforms or is on par with human and significantly larger general-purpose foundation models on the biomedical QA task. It also demonstrates promising performance in the molecule QA and protein QA tasks, which could greatly accelerate the discovery of new drugs and therapeutic targets. In addition, BioMedGPT-LM-7B is the first large generative language model based on Llama2 in the biomedical domain, therefore is commercial friendly. Both BioMedGPT-10B and BioMedGPT-LM-7B are open-sourced to the research community. In addition, we publish the datasets that are meticulously curated for the alignment of multi-modalities, i.e., PubChemQA and UniProtQA. All the models, codes, and datasets are available at \url{https://github.com/PharMolix/OpenBioMed}.
Forward citations
Cited by 8 Pith papers
-
Hypothesis-and-Refinement Learning of Organic Structures from Multimodal Spectroscopic Data
A two-stage AI pipeline — spectral hypothesis generation followed by mass-constrained molecular refinement — reconstructs organic structures from multimodal spectra, with 93.8% top-1 accuracy on simulated QM9 data and...
-
Interpreting Chest X-rays Like a Radiologist: A Benchmark with Clinical Reasoning
A new 8-stage chest X-ray VQA benchmark and a context-aware model trained on it.
-
Toward Scientific Reasoning in LLMs: Training from Expert Discussions via Reinforcement Learning
Reinforcement learning on questions extracted from CRISPR expert forums improves LLM accuracy on a new benchmark (Genome-Bench) by over 15 percentage points.
-
X-MAS: Towards Building Multi-Agent Systems with Heterogeneous LLMs
A benchmark-guided selection of different LLMs for different agent roles consistently improves multi-agent system accuracy over using one model everywhere.
-
Self-Consistency of the Internal Reward Models Improves Self-Rewarding Language Models
Forcing a self-rewarding LLM's generative judge and implicit DPO reward to agree on preference labels substantially improves its AlpacaEval 2.0 alignment score.
-
The Philosophy and Physics of Duality
A philosophical monograph that surveys dualities across physics and proposes a 'geometric view of theories' for theoretical equivalence, realism, and explanation.
-
ICH-Qwen: A Large Language Model Towards Chinese Intangible Cultural Heritage
They fine-tuned Qwen2.5-7B on Chinese intangible cultural heritage texts to build ICH-Qwen, and report n-gram metric wins over general LLMs on 100-sample ICH QA tasks.
-
BioPars: A Pretrained Biomedical Large Language Model for Persian Biomedical Text Mining
A proposed Persian biomedical LLM, BioPars, is evaluated on medical QA datasets and reported to beat GPT-4 on a self-built Persian QA benchmark, but the training setup is not described.
Discussion (0). Continue with ORCID to comment.