RECORD
Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning
Computation and Language · Submitted 11 Apr 2022
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Our method achieves state-of-the-art results on several biomedical EL tasks without candidate selection which displays the effectiveness of proposed pre-training and fine-tuning strategies.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Entities lie in the heart of biomedical natural language understanding, and the biomedical entity linking (EL) task remains challenging due to the fine-grained and diversiform concept names. Generative methods achieve remarkable performances in general domain EL with less memory usage while requiring expensive pre-training. Previous biomedical EL methods leverage synonyms from knowledge bases (KB) which is not trivial to inject into a generative method. In this work, we use a generative approach to model biomedical EL and propose to inject synonyms knowledge in it. We propose KB-guided pre-training by constructing synthetic samples with synonyms and definitions from KB and require the model to recover concept names. We also propose synonyms-aware fine-tuning to select concept names for training, and propose decoder prompt and multi-synonyms constrained prefix tree for inference. Our method achieves state-of-the-art results on several biomedical EL tasks without candidate selection which displays the effectiveness of proposed pre-training and fine-tuning strategies.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Entities lie in the heart of biomedical natural language understanding, and the biomedical entity linking (EL) task remains challenging due to the fine-grained and diversiform concept names. | background 1.00 | 0.05 | 0.02 | asserted 1.00 |
| s1 | Generative methods achieve remarkable performances in general domain EL with less memory usage while requiring expensive pre-training. | background 0.98 | 0.07 | 0.04 | asserted 1.00 |
| s2 | Previous biomedical EL methods leverage synonyms from knowledge bases (KB) which is not trivial to inject into a generative method. | background 0.51 | 0.04 | 0.02 | asserted 1.00 |
| s3 | In this work, we use a generative approach to model biomedical EL and propose to inject synonyms knowledge in it. | contribution 0.99 | 0.90 | 0.06 | asserted 0.95 |
| s4 | We propose KB-guided pre-training by constructing synthetic samples with synonyms and definitions from KB and require the model to recover concept names. | method 0.83 | 0.94 | 0.05 | asserted 0.97 |
| s5 | We also propose synonyms-aware fine-tuning to select concept names for training, and propose decoder prompt and multi-synonyms constrained prefix tree for inference. | method 0.86 | 0.93 | 0.06 | asserted 0.96 |
| s6 | Our method achieves state-of-the-art results on several biomedical EL tasks without candidate selection which displays the effectiveness of proposed pre-training and fine-tuning strategies. | result 0.98 | 0.96 | 0.90 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v3
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Entities lie in the heart of biomedical natural language understanding, and the biomedical entity linking (EL) task remains challenging due to the fine-grained and diversiform concept names. Generative methods achieve remarkable performances in general domain EL with less memory usage while requiring expensive pre-training. Previous biomedical EL methods leverage synonyms from knowledge bases (KB) which is not trivial to inject into a generative method. In this work, we use a generative approach to model biomedical EL and propose to inject synonyms knowledge in it. We propose KB-guided pre-training by constructing synthetic samples with synonyms and definitions from KB and require the model to recover concept names. We also propose synonyms-aware fine-tuning to select concept names for training, and propose decoder prompt and multi-synonyms constrained prefix tree for inference. Our method achieves state-of-the-art results on several biomedical EL tasks without candidate selection which displays the effectiveness of proposed pre-training and fine-tuning strategies.
Human review
References
How the literature uses it
7 arXiv papers cite this in 8 sentences
Papers citing it · 4 sentences without a confident label
Compares with it 1 paper
-
We think the key to the success of our method versus approaches that either use LLMs to generate knowledge graph sourced definitions only [ref] , definition and synonyms without mentions [ref] or use self-supervision [ref] is in part due to the limitations of existing knowledge resources like UMLS.
Disease Entity Recognition and Normalization is Improved with Large Language Model Derived Synthetic Normalized Mentions · Discussion · 0.98
Cites it as background 3 papers
-
Some choose to use a synonym to construct sentences by simply using templates and conducting Mask-Word-Predication task [ref] .
Sem4SAP: Synonymous Expression Mining From Open Knowledge Graph For Language Model Synonym-Aware Pretraining · Synonym Augmentation · 0.80
-
In the past years, generative language models have also been explored for the task of BEL [ref] .
Biomedical Entity Linking for Dutch: Fine-tuning a Self-alignment BERT Model on an Automatically Generated Wikipedia Corpus · Related Work · 0.99
-
Other studies have focused on Generative Entity Linking (GEL) [ref] .
Optimal Transport Guided Correlation Assignment for Multimodal Entity Linking · Related Works · 0.95
What it cites · 5 sentences without a confident label
Cites as background 3 papers
-
Recent methods in biomedical EL mainly used neural networks to encode mentions and each concept name into the same dense space, then linked mentions to corresponding concepts depending on embedding similarities [ref] .
Fast and Effective Biomedical Entity Linking Using a Dual Encoder · Introduction · 0.94
-
Recent methods in biomedical EL mainly used neural networks to encode mentions and each concept name into the same dense space, then linked mentions to corresponding concepts depending on embedding similarities [ref] .
Biomedical Entity Linking with Contrastive Context Matching · Introduction · 0.94
-
Recent methods in biomedical EL mainly used neural networks to encode mentions and each concept name into the same dense space, then linked mentions to corresponding concepts depending on embedding similarities [ref] .
Entity Linking and Discovery via Arborescence-based Supervised Clustering · Introduction · 0.94
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.