REVIEW 8 cited by
LLMs in Biomedicine: A study on clinical Named Entity Recognition
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Large Language Models (LLMs) demonstrate remarkable versatility in various NLP tasks but encounter distinct challenges in biomedical due to the complexities of language and data scarcity. This paper investigates LLMs application in the biomedical domain by exploring strategies to enhance their performance for the NER task. Our study reveals the importance of meticulously designed prompts in the biomedical. Strategic selection of in-context examples yields a marked improvement, offering ~15-20\% increase in F1 score across all benchmark datasets for biomedical few-shot NER. Additionally, our results indicate that integrating external biomedical knowledge via prompting strategies can enhance the proficiency of general-purpose LLMs to meet the specialized needs of biomedical NER. Leveraging a medical knowledge base, our proposed method, DiRAG, inspired by Retrieval-Augmented Generation (RAG), can boost the zero-shot F1 score of LLMs for biomedical NER. Code is released at \url{https://github.com/masoud-monajati/LLM_Bio_NER}
Forward citations
Cited by 8 Pith papers
-
From Text to Actionable Intelligence: Automating STIX Entity and Relationship Extraction
Fine-tuned LLMs extract STIX entities and relationships from threat reports with per-module F1 scores of 84.4%, 88.5%, 95.5%, and 84.6%, backed by a new 4,011-entity annotated dataset.
-
RAMIE: Retrieval-Augmented Multi-task Information Extraction with Large Language Models on Dietary Supplements
RAMIE, a retrieval-augmented multi-task instruction-tuned framework, improves LLM information extraction for dietary supplements from clinical records, with RAG recovering accuracy lost in multi-task training.
-
ChiMed 2.0: Advancing Chinese Medical Dataset in Facilitating Large Language Modeling
ChiMed 2.0 is a 204.4M-character Chinese medical dataset spanning pretraining, SFT, and preference data that yields small gains on CMMLU and CEval medical subsets.
-
STAYKATE: Hybrid In-Context Example Selection Combining Representativeness Sampling and Retrieval-based Approach -- A Case Study on Science Domains
STAYKATE blends static representativeness sampling with kNN retrieval to pick in-context examples for LLM scientific NER, reporting modest F1 improvements over KATE and random baselines.
-
Distilling Large Language Models for Efficient Clinical Information Extraction
Small BERT models distilled from LLM and ontology labels match the teachers on medication and disease extraction at a fraction of cost, but trail on symptoms.
-
Unveiling Performance Challenges of Large Language Models in Low-Resource Healthcare: A Demographic Fairness Perspective
An evaluation of GPT-4, Claude-3, and LLaMA-3 on six healthcare tasks finds low accuracy and demographic unfairness, with less favorable predictions for African American patients.
-
Cohort Retrieval using Dense Passage Retrieval
A DPR-based retrieval model trained on echocardiography reports outperforms BM25 and off-the-shelf embeddings on common cohort queries, but underperforms on rare-condition (out-of-distribution) searches.
-
GerPS-Compare: Comparing NER methods for legal norm analysis
On a German legal norm corpus, a fine-tuned XLM-RoBERTa outperforms a rule-based system and a prompted LLM (LeoLM) in macro F1 for ten annotation classes.
Discussion (0). Continue with ORCID to comment.