Pith. sign in

REVIEW 2 major objections 5 minor 59 references

The paper introduces MedPath, a dataset that unifies nine biomedical entity-linking corpora under one UMLS backbone and adds hierarchical paths in 11 vocabularies to enable semantically aware evaluation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 22:19 UTC pith:2X3E5F5W

load-bearing objection MedPath is a genuinely useful dataset artifact, but the paper's own experiments don't yet demonstrate its value because the EL evaluation appears to train and test on overlapping mentions. the 2 major comments →

arxiv 2511.10887 v2 pith:2X3E5F5W submitted 2025-11-14 cs.CL cs.DB

MedPath: Multi-Domain Cross-Vocabulary Hierarchical Paths for Biomedical Entity Linking

Nishant Mishra , Wilker Aziz , Iacer Calixto This is my paper
classification cs.CL cs.DB
keywords biomedical entity linkingUMLS normalizationhierarchical pathscross-vocabulary mappingentity linking evaluationsemantic typesmulti-domain datasetclinical NLP
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents MedPath, a new entity-linking dataset that folds nine existing expert-annotated corpora—spanning clinical notes, scientific abstracts, drug labels, and social media—into one unified resource. Every mention is normalized to a canonical UMLS concept, and each concept is linked to parallel codes in up to 62 biomedical vocabularies and given full root-to-leaf hierarchical paths in up to 11 of them. The authors argue that this combination of scale, vocabulary integration, and hierarchy has not existed before, and that it allows entity-linking systems to be trained and evaluated in ways that respect semantic proximity rather than all-or-nothing accuracy. They support the claim with experiments showing that a reranker trained on the unified data outperforms the same model trained on individual datasets or single domains.

Core claim

MedPath is a large-scale, multi-domain biomedical entity-linking dataset that normalizes all entities to UMLS 2025AA, maps each CUI to parallel codes in up to 62 vocabularies, and enriches each concept with full hierarchical paths in up to 11 vocabularies. In total it comprises 513,218 mentions, 44,259 unique concepts, and 573,786 distinct hierarchical paths. The paper's central claim is that no existing benchmark simultaneously provides both cross-vocabulary integration/standardization and explicit hierarchical path annotations, and that MedPath therefore fills the gap needed for hierarchy-aware evaluation and interpretable biomedical NLP. Initial experiments illustrate the resource's value

What carries the argument

The central object is the MedPath dataset itself, a harmonized JSON schema in which each mention carries its original span, its native vocabulary ID, and a canonical UMLS CUI; each CUI is then attached to a UMLS semantic type (TUI), to parallel codes in up to 62 vocabularies, and to complete ancestor chains (root-to-leaf paths) for 11 vocabularies that expose an accessible hierarchy. The pipeline's load-bearing components are UMLS-based canonicalization, with a fallback to exact-match and substring-containment heuristics when native-to-CUI mapping is missing, and bespoke per-vocabulary path extractors that traverse parent relations to the root, capturing multiple inheritance when present. Th

Load-bearing premise

The load-bearing premise is that the automated pipeline—especially the fallback exact-match and substring-containment mapping to UMLS CUIs and the bespoke path extractors—produces correct annotations without expert validation; if these automated steps make frequent errors, the dataset's ground truth for semantic and hierarchical evaluation is compromised, a limitation the paper itself acknowledges in Section 3.4 and in the Limitations.

What would settle it

Take a random sample of mentions that were mapped via the exact-match or substring-containment fallback (they are labelled in the dataset) and have clinical experts judge the assigned UMLS CUI; if a substantial fraction (e.g., more than 5-10%) are wrong, the resource's core value degrades. Similarly, for a random sample of concepts in each of the 11 vocabularies, recompute the root-to-leaf paths independently using the official parent relationships and compare them to MedPath's paths; any mismatch would indicate a systematic extractor error that propagates to every descendant.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Entity-linking evaluation can move beyond flat metrics: a prediction that lands on an ancestor, descendant, sibling, or cousin of the gold concept can receive partial credit, and the relative sizes of those error families can be measured.
  • Models can be trained to output codes in multiple vocabularies at once, since every CUI carries parallel codes in up to 62 vocabularies, making cross-ontology interoperability an explicit target rather than an afterthought.
  • MedPath provides direct supervision for inherently explainable EL: a model can be trained to predict a mention's full hierarchical path from root to leaf, exposing the reasoning behind a link.
  • Pooling supervision from nine datasets through the UMLS backbone improves EL and NER performance over training on any single dataset or domain, according to the paper's preliminary experiments.
  • The 573,786 path annotations make MedPath a testbed for designing and comparing hierarchy-aware metrics, which the paper argues are needed because flat metrics treat a heart-disease error and an influenza error as equally wrong.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the automated mappings and path extractions are accurate, the same pipeline could be re-run on updated versions of UMLS, SNOMED CT, and other vocabularies to keep the resource current, turning MedPath from a single snapshot into a living benchmark (an extension the authors leave implicit in their update strategy).
  • The observed split of errors into roughly 20% ancestor and 20% descendant suggests that many entity-linking failures are granularity problems—models identify the right neighborhood but the wrong specificity. If this pattern generalizes, hierarchy-aware metrics may be more informative than flat accuracy for ranking models in clinical settings.
  • A natural testable extension is to use MedPath's cross-vocabulary codes to train a single model that predicts the same mention's codes in several vocabularies simultaneously (e.g., SNOMED CT, MeSH, and ICD-10), which would provide a concrete measure of multi-vocabulary fluency.
  • Because the noisy fallback mappings are explicitly labelled, MedPath also allows quantifying how much annotation noise affects entity-linking model performance by comparing models trained with and without those examples—a controlled experiment the paper does not run.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces MedPath, a large-scale biomedical entity linking (EL) resource built by merging nine existing expert-annotated datasets. All mentions are normalized to UMLS 2025AA CUIs, each CUI is mapped to codes in up to 62 vocabularies, and 11 vocabularies are enriched with full hierarchical paths, yielding 513,218 mentions, 44,259 unique CUIs, and 573,786 distinct paths. The paper also proposes hierarchy-aware evaluation metrics and reports preliminary EL experiments with TF-IDF and SapBERT retrievers plus a reranker, including an ablation comparing in-dataset, in-domain, and overall (MedPath) training regimes. The central claims are that MedPath provides a unique multi-domain, cross-vocabulary, hierarchy-annotated resource and that training on it improves EL performance.

Significance. If the resource is constructed as described, MedPath would fill a genuine gap: existing benchmarks do not combine cross-vocabulary normalization with explicit hierarchical paths in multiple biomedical ontologies. The authors ship the pipeline, disclose the automated mapping heuristics, quantify the fallback cases (2.5% of mentions), and state clearly that no expert clinical validation was performed on the new annotation layers. These are notable strengths. The dataset itself is likely to be useful even if the preliminary experiments are imperfect. However, the empirical validation of MedPath's utility currently rests on an under-specified evaluation protocol, which is load-bearing for the claim that training on MedPath outperforms in-dataset and in-domain training.

major comments (2)
  1. [Section 5.1 (Evaluation; Reranking)] The test set is described as 'all unique mentions with ground-truth CUIs across datasets,' while the reranker is trained on 'top-32 generated candidates plus the gold entity' for each mention. If the same mentions or CUIs appear in both training and this test set, the reported Acc@1=80.84% and MRR=0.857 are memorization estimates, not generalization results. The ablation definitions (in-dataset, in-domain, overall) imply splits exist, but the paper never states whether Table 4 uses the 'overall' regime, whether the test set is disjoint from training in that regime, or how mention-level overlap is prevented. This directly affects the central experimental claim. Please specify the exact split construction, deduplicate mentions/CUIs across train/test, and report results on a genuinely held-out set; if the current numbers are already on held-out sets, state this explicitly.
  2. [Section 5.1, Table 4] The hierarchy-aware metrics are not fully specified. 'Hierarchy@k' is defined as any top-k predicted CUI that is an ancestor, descendant, or has 'any hierarchy overlap' with the gold CUI, after 'skipping top-3 levels from the root.' This is very permissive and can count sibling or cousin matches. The relationship between Hierarchy@1=85.40% and Acc@1=80.84% is not derived. Please provide the exact algorithm (how ancestors are determined, how the top-3 skip interacts with the lowest common ancestor, how multi-parent DAGs are handled) and report what fraction of Hierarchy@1 hits are exact, ancestor, descendant, or collateral. Without this, the hierarchy-aware evaluation cannot be interpreted.
minor comments (5)
  1. [Section 3.2 and 3.4] The paper states that fallback exact-match and substring-containment mappings are 'clearly labelled' so users can filter them out and have a 'noise-free dataset.' This overstates the guarantee: exact-match on surface forms is still error-prone for ambiguous mentions, and native-ID-to-CUI mappings can also contain errors. A small sample-based error analysis of the fallback mappings would strengthen the data-quality section.
  2. [Section 3.2, Stage 4 / Appendix D] The abstract and Table 1 claim mappings to 'up to 62 vocabularies,' but Appendix D reports only the top 25 vocabularies considered for path extraction. Please clarify whether the full 62-vocabulary list is available in the released code or provide it in the appendix, and distinguish 'vocabularies with flat code mappings' from 'vocabularies with full hierarchical paths.'
  3. [Section 5.1, Reranking] The text says the cross-encoder is trained on 'top-32 generated candidates plus the gold entity (in case the gold entity is not in the top-32).' It should be stated clearly that this gold-entity insertion is used only for constructing training examples and is not applied at inference time, otherwise the evaluation protocol appears to leak the gold label into the candidate set.
  4. [References] There are duplicate reference entries: Huang et al. (2024) and Huang et al. (2019) are listed separately in the text but the reference list has two different papers with the same first author and no disambiguating labels; Gu et al. (2021a) and (2021b) appear to be the same paper; Kartchner et al. (2023a) and (2023b) are also the same item. Please deduplicate.
  5. [Figure 5] The radar/bar chart for EL performance in the three data settings lacks axis labels and a legend in the printed caption. It is hard to read which metric is being plotted beyond 'acc@16 per semantic class.' Please add a clear caption and axis descriptions.

Circularity Check

0 steps flagged

No circularity found: MedPath's vocabulary mappings and hierarchical paths are imported from external UMLS/native ontologies, and the EL/NER experiments are empirical evaluations, not derivations from the dataset's own outputs.

full rationale

MedPath is a dataset-integration paper: its central products (CUI normalization, parallel codes in up to 62 vocabularies, hierarchical paths in 11 vocabularies) are imported from external resources — UMLS 2025AA, SNOMED CT, MeSH, HPO, MedDRA, GO, etc. — and the paper makes no claim to have derived those ontologies or their mappings from its own outputs. The one stage that introduces paper-specific inference (Stage 2's fallback to exact-match/substring-containment CUI mapping) is a data-quality caveat, not a self-definitional loop: the fallback is explicitly acknowledged, itemized (2.13% + 0.37% of unique mentions), and labeled in the released data so users can filter it (Sections 3.2, 3.4). The preliminary EL/NER experiments (Tables 4–6; Figures 5, 15–17) are empirical benchmarks, not predictions that reduce to their inputs by construction; the hand-picked evaluation thresholds (top-32 candidates; skipping top-3 hierarchy levels) do not force the reported accuracies or MRRs. The paper contains no self-citations or author-imported uniqueness theorems; the EL model is adapted from an external toolkit (X-MEN, Borchert et al. 2024) and retrieval uses TF-IDF/SapBERT. Two reviewer concerns are flagged but are not circularity under the rubric: (i) Section 5.1 defines the test set as 'all unique mentions with ground-truth CUIs across datasets' without fully specifying how the overall regime's training splits are kept disjoint, which is a potential leakage/methodology risk for the reported MedPath training advantage; and (ii) the Limitations section concedes 'we did not perform expert, clinical validation of the new layers of annotation and mappings added,' which is an acknowledged quality limitation on the resource claims. Neither concern exhibits an equation-level reduction of a prediction to its input, and the ontology content is externally grounded, so the honest circularity verdict is none.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The dataset's correctness rests on the source annotations, UMLS mappings, and hierarchy extraction; all are inherited or automated, with no expert validation of the new layers. The hand-chosen parameters (fallback heuristics, top-k, skip levels) shape the reported benchmark numbers but are not fitted to the central claims.

free parameters (5)
  • semantic_containment_fallback = bidirectional substring containment with token-overlap and length-similarity scoring
    Stage 2 maps source codes to CUIs when native ID lookup fails; affects 0.37% of mentions and 0.35% of concepts, and may inject noise.
  • top_k_hierarchy_skip = 3 levels from root
    Hierarchy@k metrics skip the top-3 most general levels so that trivial root-level matches are not counted; hand-chosen threshold.
  • candidate_k_rerank = 32
    Cross-encoder reranker is trained on top-32 candidates plus the gold entity; this value shapes retrieval and reranking results.
  • ner_semantic_classes = 11 high-level UMLS Semantic Group classes
    NER labels are collapsed to 11 semantic classes, changing the granularity and comparability of NER results.
  • ner_chunking = 512-character chunks with 128-character stride
    Documents are split into overlapping chunks for NER; this hyperparameter affects boundary detection.
axioms (4)
  • domain assumption UMLS 2025AA native-ID-to-CUI dictionary correctly maps source vocabulary codes to concepts.
    Stage 2 canonicalization relies on UMLS as the authoritative backbone; incorrect UMLS entries propagate to all downstream vocabulary and path annotations.
  • domain assumption The nine source datasets' expert-annotated mention spans and codes are high-quality ground truth.
    The curation rationale selects these datasets, and the paper performs no re-adjudication of the original annotations.
  • domain assumption Parent/child relations extracted from each of the 11 vocabularies (is-a relations, tree numbers, subsumption) accurately represent semantic generality.
    Hierarchical paths are used both as dataset annotations and as evaluation signal, so errors in those relations distort the claimed explainability benefit.
  • ad hoc to paper String exact-match and substring-containment can recover the intended UMLS CUI when the native ID is missing.
    The fallback in Stage 2 injects 2.5% of mentions that may be mislabeled; the paper labels them but includes them in the default release.

pith-pipeline@v1.3.0-alltime-deepseek · 22053 in / 11396 out tokens · 107645 ms · 2026-08-03T22:19:40.154224+00:00 · methodology

0 comments
read the original abstract

Progress in biomedical Named Entity Recognition (NER) and Entity Linking (EL) is currently hindered by a fragmented data landscape, a lack of resources for building explainable models, and the limitations of semantically-blind evaluation metrics. To address these challenges, we present MedPath, a large-scale and multi-domain biomedical EL dataset that builds upon nine existing expert-annotated EL datasets. In MedPath, all entities are 1) normalized using the latest version of the Unified Medical Language System (UMLS), 2) augmented with mappings to 62 other biomedical vocabularies and, crucially, 3) enriched with full ontological paths -- i.e., from general to specific -- in up to 11 biomedical vocabularies. MedPath directly enables new research frontiers in biomedical NLP, facilitating training and evaluation of semantic-rich and interpretable EL systems, and the development of the next generation of interoperable and explainable clinical NLP models.

Figures

Figures reproduced from arXiv: 2511.10887 by Iacer Calixto, Nishant Mishra, Wilker Aziz.

Figure 1
Figure 1. Figure 1: MedPath creation process. For illustration purposes, we show one example from two different datasets, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Semantic type distribution in MedPath. 4.1 Size and Genre Balance The final harmonized corpus comprises over 5 mil￾lion tokens and 513k expert-annotated mentions ( [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Vocabulary overlap heat map. Datasets’ anno [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Histogram of lengths of entity hierarchical [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Figure showing EL performance in the three [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: An example showing the schema of the proposed dataset, which shows the four components (clockwise): [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The mean and median document length for each dataset, shown in terms of BERT tokens. B Additional Data Analysis In this section, we present further analyses. B.1 Document length As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Contribution of each source dataset to concepts belonging to 15 major semantic type categories. [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: Path length distribution per vocabulary. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png] view at source ↗
Figure 12
Figure 12. Figure 12: Performance of models across the four main [PITH_FULL_IMAGE:figures/full_fig_p017_12.png] view at source ↗
Figure 11
Figure 11. Figure 11: Performance of fine-tuned models in full [PITH_FULL_IMAGE:figures/full_fig_p017_11.png] view at source ↗
Figure 13
Figure 13. Figure 13: Figure showing the macro average per￾formance over semantic types of the NER model in￾domain, in-dataset, and a full mix setting by domain for all models, including the zero-shot GliNER-BioMed. Ablations The radar chart in [PITH_FULL_IMAGE:figures/full_fig_p017_13.png] view at source ↗
Figure 16
Figure 16. Figure 16: Performance of TF-IDF and SapBERT Can￾didate Generation across domains [PITH_FULL_IMAGE:figures/full_fig_p018_16.png] view at source ↗
Figure 15
Figure 15. Figure 15: Performance of TF-IDF and SapBERT Can￾didate Generation across datasets that SapBERT consistently outperforms TF-IDF by small margins. Mantra-GSC, ShaRE/CLEF, and MedMentions show the best performance, which follows logically from the fact that their original ground truth annotations were in UMLS ( [PITH_FULL_IMAGE:figures/full_fig_p018_15.png] view at source ↗
Figure 17
Figure 17. Figure 17: Entity linking performance across metrics (rows: Acc@1, Acc@5, Acc@32, MRR@32), averaged over [PITH_FULL_IMAGE:figures/full_fig_p019_17.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

59 extracted references · 6 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Enrique Amig \'o and Agust \'i n Daniel Delgado. 2022. https://api.semanticscholar.org/CorpusID:248779945 Evaluating extreme hierarchical multi-label classification . In Annual Meeting of the Association for Computational Linguistics

    2022

  4. [4]

    Alan R Aronson and Fran c ois-Michel Lang. 2010. An overview of metamap: historical perspective and recent advances. Journal of the American medical informatics association, 17(3):229--236

    2010

  5. [5]

    Marco Basaldella, Fangyu Liu, Ehsan Shareghi, and Nigel Collier. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.253 COMETA : A corpus for medical entity linking in the social media . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3122--3137, Online. Association for Computational Linguistics

    work page doi:10.18653/v1/2020.emnlp-main.253 2020

  6. [6]

    Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. Scibert: A pretrained language model for scientific text. arXiv preprint arXiv:1903.10676

    Pith/arXiv arXiv 2019

  7. [7]

    Olivier Bodenreider. 2004. https://api.semanticscholar.org/CorpusID:205228801 The unified medical language system (umls): integrating biomedical terminology . Nucleic acids research, 32 Database issue:D267--70

    2004

  8. [8]

    Florian Borchert, Ignacio Llorca, Roland Roller, Bert Arnrich, and Matthieu-P Schapranow. 2024. https://doi.org/10.1093/jamiaopen/ooae147 xmen: a modular toolkit for cross-lingual medical entity normalization . JAMIA Open, 8(1):ooae147

    work page doi:10.1093/jamiaopen/ooae147 2024

  9. [9]

    Nicol \`o Cesa-Bianchi, Claudio Gentile, and Luca Zaniboni. 2006. Incremental algorithms for hierarchical classification. Journal of Machine Learning Research, 7:31--54

    2006

  10. [10]

    Rory Davidson, Will Hardman, Guy Amit, Yonatan Bilu, Vincenzo Della Mea, Aleksandr Galaida, Irena Girshovitz, Mikhail Kulyabin, Mihai Horia Popescu, Kevin Roitero, Gleb Sokolov, and Chen Yanover. 2025. https://api.semanticscholar.org/CorpusID:280015103 Snomed ct entity linking challenge. Journal of the American Medical Informatics Association : JAMIA

    2025

  11. [11]

    Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. 2020. Autoregressive entity retrieval. arXiv preprint arXiv:2010.00904

    Pith/arXiv arXiv 2020

  12. [12]

    Rezarta Islamaj Do g an, Robert Leaman, and Zhiyong Lu. 2014. NCBI disease corpus: a resource for disease name recognition and concept normalization. J Biomed Inform, 47:1--10

    2014

  13. [13]

    Mat \'u s Falis, Hang Dong, Alexandra Birch, and Beatrice Alex. 2021. Cophe: A count-preserving hierarchical evaluation metric in large-scale multi-label text classification. arXiv preprint arXiv:2109.04853

    Pith/arXiv arXiv 2021

  14. [14]

    Jason Fries, Leon Weber, Natasha Seelam, Gabriel Altay, Debajyoti Datta, Samuele Garda, Sunny Kang, Rosaline Su, Wojciech Kusa, Samuel Cahyawijaya, and 1 others. 2022. Bigbio: A framework for data-centric biomedical natural language processing. Advances in Neural Information Processing Systems, 35:25792--25806

    2022

  15. [15]

    Samuele Garda, Leon Weber-Genzel, Robert Martin, and Ulf Leser. 2023. Belb: a biomedical entity linking benchmark. Bioinformatics, 39(11):btad698

    2023

  16. [17]

    Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021 b . https://doi.org/10.1145/3458754 Domain-specific language model pretraining for biomedical natural language processing . ACM Trans. Comput. Healthcare, 3(1)

    doi:10.1145/3458754 2021

  17. [18]

    Wenhan Han, Meng Fang, Zihan Zhang, Yu Yin, Zirui Song, Ling Chen, Mykola Pechenizkiy, and Qingyu Chen. 2024. Medinst: Meta dataset of biomedical instructions. arXiv preprint arXiv:2410.13458

    Pith/arXiv arXiv 2024

  18. [19]

    Zexue He, Yu Wang, An Yan, Yao Liu, Eric Y Chang, Amilcare Gentili, Julian McAuley, and Chun-Nan Hsu. 2023. Medeval: A multi-level, multi-task, and multi-domain medical benchmark for language model evaluation. arXiv preprint arXiv:2310.14088

    Pith/arXiv arXiv 2023

  19. [20]

    Guangming Huang, Yingya Li, Shoaib Jameel, Yunfei Long, and Giorgos Papanastasiou. 2024. From explainable to interpretable deep learning for natural language processing in healthcare: How far from reality? Computational and structural biotechnology journal, 24:362--373

    2024

  20. [21]

    Kexin Huang, Jaan Altosaar, and Rajesh Ranganath. 2019. Clinicalbert: Modeling clinical notes and predicting hospital readmission. arXiv preprint arXiv:1904.05342

    Pith/arXiv arXiv 2019

  21. [22]

    Mohamed Ben Jannet, Martine Adda-Decker, Olivier Galibert, Juliette Kahn, and Sophie Rosset. 2014. Eter: a new metric for the evaluation of hierarchical named entity recognition. In Ninth International Conference on Language Resources and Evaluation (LREC'14), pages 3987--3994

    2014

  22. [23]

    Alistair E. W. Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J. Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, Li-wei H. Lehman, Leo A. Celi, and Roger G. Mark. 2023. https://doi.org/10.1038/s41597-022-01899-x Mimic-iv, a freely accessible electronic health record dataset . Scientific Data, 10(1):1

    doi:10.1038/s41597-022-01899-x 2023

  23. [24]

    Sarvnaz Karimi, Alejandro Metke-Jimenez, Madonna Kemp, and Chen Wang. 2015. Cadec: A corpus of adverse drug event annotations. J Biomed Inform, 55:73--81

    2015

  24. [25]

    Mitchell

    David Kartchner, Jennifer Deng, Shubham Lohiya, Tejasri Kopparthi, Prasanth Bathala, Daniel Domingo-Fern \'a ndez, and Cassie S. Mitchell. 2023 a . https://api.semanticscholar.org/CorpusID:266164127 A comprehensive evaluation of biomedical entity linking models . Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference...

    2023

  25. [26]

    Mitchell

    David Kartchner, Jennifer Deng, Shubham Lohiya, Tejasri Kopparthi, Prasanth Bathala, Daniel Domingo-Fern \'a ndez, and Cassie S. Mitchell. 2023 b . https://doi.org/10.18653/v1/2023.emnlp-main.893 A comprehensive evaluation of biomedical entity linking models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP...

    doi:10.18653/v1/2023.emnlp-main.893 2023

  26. [27]

    Fazel Famili

    Svetlana Kiritchenko, Stan Matwin, and A. Fazel Famili. 2005. Functional annotation of genes using hierarchical text categorization. In Proceedings of the ACL Workshop on Linking Biological Literature, Ontologies and Databases: Mining Biological Semantics (BioLINK)

    2005

  27. [28]

    Fazel Famili

    Svetlana Kiritchenko, Stan Matwin, Richard Nock, and A. Fazel Famili. 2006. Learning and evaluation in the presence of class hierarchies: Application to text categorization. In Advances in Artificial Intelligence, 19th Conference of the Canadian Society for Computational Studies of Intelligence, Canadian AI 2006, pages 395--406, Québec City, Québec, Canad...

    2006

  28. [29]

    Jan A Kors, Simon Clematide, Saber A Akhondi, Erik M van Mulligen, and Dietrich Rebholz-Schuhmann. 2015. https://doi.org/10.1093/jamia/ocv037 A multilingual gold-standard corpus for biomedical concept recognition: the mantra gsc . Journal of the American Medical Informatics Association, 22(5):948--956

    work page doi:10.1093/jamia/ocv037 2015

  29. [30]

    Aris Kosmopoulos, Ioannis Partalas, Eric Gaussier, Georgios Paliouras, and Ion Androutsopoulos. 2015. https://doi.org/10.1007/s10618-014-0382-x Evaluation measures for hierarchical classification: a unified view and novel approaches . Data Mining and Knowledge Discovery, 29(3):820--865

    work page doi:10.1007/s10618-014-0382-x 2015

  30. [31]

    Robert Leaman and Zhiyong Lu. 2016. https://www.ncbi.nlm.nih.gov/pubmed/27283952 Taggerone: joint named entity recognition and normalization with semi-markov models . Bioinformatics, 32 18:2839--46

    arXiv 2016

  31. [32]

    Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2019. https://doi.org/10.1093/bioinformatics/btz682 BioBERT : a pre-trained biomedical language representation model for biomedical text mining . Bioinformatics, 36(4):1234--1240. \_eprint: https://academic.oup.com/bioinformatics/article-pdf/36/4/1234/48983216/...

    doi:10.1093/bioinformatics/btz682 2019

  32. [33]

    Johnson, Daniela Sciaky, Chih - Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J

    Jiao Li, Yueping Sun, Robin J. Johnson, Daniela Sciaky, Chih - Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J. Mattingly, Thomas C. Wiegers, and Zhiyong Lu. 2016. https://doi.org/10.1093/database/baw068 Biocreative V CDR task corpus: a resource for chemical disease relation extraction . Database J. Biol. Databases Curation, 2016

    doi:10.1093/database/baw068 2016

  33. [34]

    Fangyu Liu, Ehsan Shareghi, Zaiqiao Meng, Marco Basaldella, and Nigel Collier. 2020. Self-alignment pretraining for biomedical entity representations. arXiv preprint arXiv:2010.11784

    Pith/arXiv arXiv 2020

  34. [35]

    Fangyu Liu, Ehsan Shareghi, Zaiqiao Meng, Marco Basaldella, and Nigel Collier. 2021. Self-alignment pretraining for biomedical entity representations. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4228--4238

    2021

  35. [36]

    Sunil Mohan and Donghui Li. 2019. https://arxiv.org/abs/1902.09476 Medmentions: A large biomedical corpus annotated with umls concepts . Preprint, arXiv:1902.09476

    Pith/arXiv arXiv 2019

  36. [37]

    Diego Moussallem, Ricardo Usbeck, Michael R\" o eder, and Axel-Cyrille Ngonga Ngomo. 2017. https://doi.org/10.1145/3148011.3148024 Mag: A multilingual, knowledge-base agnostic and deterministic entity linking approach . In Proceedings of the 9th Knowledge Capture Conference, K-CAP '17, New York, NY, USA. Association for Computing Machinery

    arXiv 2017

  37. [38]

    David Nadeau and Satoshi Sekine. 2007. https://api.semanticscholar.org/CorpusID:8310135 A survey of named entity recognition and classification . Lingvisticae Investigationes, 30:3--26

    2007

  38. [39]

    Mark Neumann, Daniel King, Iz Beltagy, and Waleed Ammar. 2019. Scispacy: fast and robust models for biomedical natural language processing. arXiv preprint arXiv:1902.07669

    Pith/arXiv arXiv 2019

  39. [40]

    Yifan Peng, Shankai Yan, and Zhiyong Lu. 2019. Transfer learning in biomedical natural language processing: An evaluation of bert and elmo on ten benchmarking datasets. In Proceedings of the 2019 Workshop on Biomedical Natural Language Processing (BioNLP 2019), pages 58--65

    2019

  40. [41]

    Catia Pesquita, Daniel Faria, Andr \'e O Falc \ a o, Phillip Lord, and Francisco M Couto. 2009. https://doi.org/10.1371/journal.pcbi.1000443 Semantic similarity in biomedical ontologies . PLoS Computational Biology, 5(7):e1000443

    work page doi:10.1371/journal.pcbi.1000443 2009

  41. [42]

    Roman Plaud, Matthieu Labeau, Antoine Saillenfest, and Thomas Bonald. 2024. https://api.semanticscholar.org/CorpusID:273026126 Revisiting hierarchical text classification: Inference and metrics . ArXiv, abs/2410.01305

    Pith/arXiv arXiv 2024

  42. [43]

    PubMed. 2025. https://pubmed.ncbi.nlm.nih.gov/ Pubmed . [Online; accessed 28-July-2025]

    2025

  43. [44]

    Kirk Roberts, Dina Demner-Fushman, and Joseph M Tonning. 2017 a . Overview of the tac 2017 adverse reaction extraction from drug labels track. In TAC

    2017

  44. [45]

    Kirk Roberts, Dina Demner-Fushman, and Joseph M Tonning. 2017 b . Overview of the tac 2017 adverse reaction extraction from drug labels track. In TAC

    2017

  45. [46]

    Michael R \"o der, Ricardo Usbeck, and Axel-Cyrille Ngonga Ngomo. 2018. Gerbil--benchmarking named entity recognition and linking consistently. Semantic Web, 9(5):605--625

    2018

  46. [47]

    Rouhizadeh, Irina Nikishina, A

    H. Rouhizadeh, Irina Nikishina, A. Yazdani, A. Bornet, Boya Zhang, Julien Ehrsam, C. Gaudet-Blavignac, Nona Naderi, and Douglas Teodoro. 2024. https://api.semanticscholar.org/CorpusId:269587330 A dataset for evaluating contextualized representation of biomedical concepts in language models . Scientific Data, 11

    2024

  47. [48]

    Mujeen Sung, Hwisang Jeon, Jinhyuk Lee, and Jaewoo Kang. 2020. Biomedical entity representations with synonym marginalization. arXiv preprint arXiv:2005.00239

    Pith/arXiv arXiv 2020

  48. [49]

    Chapman, Guergana Savova, Noemie Elhadad, Sameer Pradhan, Brett R

    Hanna Suominen, Sanna Salanter \"a , Sumithra Velupillai, Wendy W. Chapman, Guergana Savova, Noemie Elhadad, Sameer Pradhan, Brett R. South, Danielle L. Mowery, Gareth J. F. Jones, Johannes Leveling, Liadh Kelly, Lorraine Goeuriot, David Martinez, and Guido Zuccon. 2013 a . Overview of the share/clef ehealth evaluation lab 2013. In Information Access Eval...

    2013

  49. [50]

    Hanna Suominen, Sanna Salantera, Sumithra Velupillai, Wendy W Chapman, Guergana Savova, Angus Roberts, Liadh Kelly, Lorraine Goeuriot, Diego Martinez, Guido Zuccon, and 1 others. 2013 b . Overview of the share/clef ehealth evaluation lab 2013. In CLEF (Working Notes)

    2013

  50. [51]

    Ehtesham Ullah, Anil Parwani, Mirza Mansoor Baig, and Rajeev Singh. 2024. https://doi.org/10.1186/s13000-024-01464-7 Challenges and barriers of using large language models (llm) such as chatgpt for diagnostic medicine with a focus on digital pathology: a recent scoping review . Diagnostic Pathology, 19(1):43

    doi:10.1186/s13000-024-01464-7 2024

  51. [52]

    o der, Axel-Cyrille Ngonga Ngomo, Ciro Baron, Andreas Both, Martin Br \

    Ricardo Usbeck, Michael R \"o der, Axel-Cyrille Ngonga Ngomo, Ciro Baron, Andreas Both, Martin Br \"u mmer, Diego Ceccarelli, Marco Cornolti, Didier Cherix, Bernd Eickmann, and 1 others. 2015. Gerbil: general entity annotator benchmarking framework. In Proceedings of the 24th international conference on World Wide Web, pages 1133--1143

    2015

  52. [53]

    Ozlem Uzuner, Brett R South, Shuying Shen, and Scott L DuVall. 2011. i2b2/va 2010 nlp challenge. In AMIA Annual Symposium Proceedings

    2011

  53. [54]

    David Wadden, Ulme Wennberg, Yi Luan, and Hannaneh Hajishirzi. 2019. Entity, relation, and event extraction with contextualized span representations. arXiv preprint arXiv:1909.03546

    Pith/arXiv arXiv 2019

  54. [55]

    Zilin Xiao, Ming Gong, Jie Wu, Xingyao Zhang, Linjun Shou, Jian Pei, and Daxin Jiang. 2023. Instructed language models with retrievers are powerful entity linkers. arXiv preprint arXiv:2311.03250

    Pith/arXiv arXiv 2023

  55. [56]

    Xi Yang, Aokun Chen, Nima PourNejatian, Hoo Chang Shin, Kaleb E Smith, Christopher Parisien, Colin Compas, Cheryl Martin, Mona G Flores, Ying Zhang, and 1 others. 2022. Gatortron: A large clinical language model to unlock patient information from unstructured electronic health records. arXiv preprint arXiv:2203.03540

    Pith/arXiv arXiv 2022

  56. [57]

    Anthony Yazdani, Ihor Stepanov, and Douglas Teodoro. 2025. Gliner-biomed: A suite of efficient models for open biomedical named entity recognition. arXiv preprint arXiv:2504.00676

    Pith/arXiv arXiv 2025

  57. [58]

    Hongyi Yuan, Zheng Yuan, and Sheng Yu. 2022. https://api.semanticscholar.org/CorpusId:248084945 Generative biomedical entity linking via knowledge base-guided pre-training and synonyms-aware fine-tuning . ArXiv, abs/2204.05164

    Pith/arXiv arXiv 2022

  58. [59]

    Lizheng Zu, Lin Lin, Song Fu, Jie Liu, Shiwei Suo, Wenhui He, Jinlei Wu, and Yancheng Lv. 2024. Pathel: A novel collective entity linking method based on relationship paths in heterogeneous information networks. Information Systems, 126:102433

    2024

  59. [60]

    Stefan Zwicklbauer, Christin Seifert, and Michael Granitzer. 2016. https://doi.org/10.1007/978-3-319-34129-3_12 Doser - a knowledge-base-agnostic framework for entity disambiguation using semantic embeddings . pages 182--198

    work page doi:10.1007/978-3-319-34129-3_12 2016