FRACCO: A gold-standard annotated corpus of oncological entities with ICD-O-3.1 normalisation

Amandine Stettler; Christian Lovis; Christophe Gaudet-Blavignac; Fanny Amrein; Jamil Zaghir; Jean-Philippe Goldman; Johann Pignat; Jonatan Bonjour; Milena Vucetic; Mina Bjelogrlic

REVIEW 4 major objections 6 minor 18 references

FRACCO: A gold-standard annotated corpus of oncological entities with ICD-O-3.1 normalisation

T0 review · 4 major / 6 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FRACCO gives French oncology NLP a gold-standard ICD-O corpus

desk verdict FRACCO is a real and useful resource for French oncology NLP, but the gold-standard label is not supported by the reported validation, especially the 51.5% concordance on manually assigned normalisations. read the letter →

arxiv 2510.13873 v1 pith:ML5EB3VL submitted 2025-10-13 cs.CL cs.AI

Johann Pignat , Milena Vucetic , Christophe Gaudet-Blavignac , Jamil Zaghir , Amandine Stettler , Fanny Amrein , Jonatan Bonjour , Jean-Philippe Goldman

show 3 more authors

Olivier Michielin Christian Lovis Mina Bjelogrlic

This is my paper · ORCID

classification cs.CLcs.AI

keywords FrenchclinicalNLPoncologycorpusICD-Onormalisationnamedentityrecognitionconceptcompositeexpressionssyntheticcasesannotationbenchmarking

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces FRACCO, a corpus of 1,301 synthetic French clinical case reports carrying 71,127 ICD-O normalisations across morphology, topography, differentiation, and a composite expression layer. The authors argue that this expert-reviewed dataset fills a gap in French-language oncology resources and provides a reference standard for training and evaluating named entity recognition and concept normalisation tools. The work matters because most French clinical NLP corpora lack oncology-specific annotations and standardised cancer terminology.

What carries the argument

The central object is the expression_CIM layer, a composite annotation that binds a morphology code, an optional topography code, and an optional differentiation grade into a single unified clinical concept (for example, 'carcinome pulmonaire indifférencié' maps to topography C34.9, morphology 8010/3, and grade 4). This layer is what allows the corpus to represent whole oncological expressions rather than isolated terms, and it is carried by a semi-automated workflow: dictionary-based matching for straightforward spans and expert-guided reconstruction for complex or ambiguous phrases.

What would settle it

Take a random sample of the manually normalised expressions and have them re-coded independently by a fresh panel of oncology experts; if agreement with the published codes falls to the level of the 51.5% baseline or below, the gold-standard claim for those codes is not supported.

Watch

Extended reading notes

Core claim

FRACCO is presented as a gold-standard French oncology corpus built by translating a Spanish oncology corpus, projecting its annotations, manually correcting spans, and extending the annotation scheme with topography codes, differentiation grades, and a new expression-level layer. All entities were normalised to ICD-O-3.1 using automatic dictionary matching for roughly 78% of cases and expert consensus review for the remaining complex expressions. The result is 399 unique morphology codes, 272 topography codes, and 2,043 unique composite expression codes, claimed to be suitable as a benchmark for French oncology NLP.

Load-bearing premise

The corpus is called gold-standard because expert consensus resolved normalisation disagreements, but manually assigned codes agreed only 51.5% between two validating experts before discussion, so the correctness of the final codes rests on the assumption that consensus produces accurate ICD-O codes rather than merely averaging judgment.

Editorial extensions

If this is right

French oncology NER and normalisation models can be trained and compared on a shared benchmark with over 70,000 annotated entities.
The composite expression layer enables tasks beyond span detection, such as relation extraction and document-level classification of cancer phenotypes.
The corpus can support cross-lingual research by aligning with the Spanish source corpus, allowing direct comparison of French and Spanish clinical NLP systems.
The detailed coverage statistics (399 morphology, 272 topography codes) give a concrete picture of which oncological concepts are most represented in synthetic clinical case language.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the texts are translations of synthetic case reports, performance on real French electronic health records may be lower; the corpus should ideally be supplemented with authentic clinical text before being used as a sole training resource.
The low pre-consensus inter-annotator agreement on manually assigned codes (51.5%) suggests that some gold-standard codes encode consensus judgment rather than unambiguous ontology mappings, so downstream users may want to audit edge-case codes for their own applications.
The translation-and-projection workflow could be reused to create oncology corpora in other low-resource languages, though the manual correction burden appears substantial.
The expression_CIM layer could serve as training data for sequence-to-sequence normalisation models that map raw clinical phrases directly to composite ICD-O codes.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The FRACCO corpus fills a genuine gap: there is no other French oncology-specific corpus with this breadth of ICD-O annotations. The authors took the Spanish CANTEMIST texts, translated them, and added topography, differentiation, and a novel expression-level layer (expression_CIM) that combines multiple ICD-O components into unified concepts. That layer is the most original part of the resource, and the scale — over 71k normalisations across 1,301 documents — is substantial. The data and tools are publicly available, and the paper is honest enough to report the low agreement on the manual subset rather than bury it. For those working on French clinical NER or normalisation, this is a potentially valuable benchmark.

The soft spots are real, and they cluster around the "gold-standard" claim. The biggest issue is the normalisation validation. The paper reports 80.65% concordance on the automatically matched subset but only 51.5% on the manually assigned codes. That is the set that contains most of the novel expression_CIM annotations, so the reliability is weakest exactly where the added value is highest. The authors say disagreements were resolved by consensus and propagated, but there is no independent adjudication, no kappa, and no post-consensus check against an external standard. Consensus between two people with that level of starting agreement can easily bake in a shared bias. There is also a concrete data problem: the abstract says 399 morphology codes and 272 topography codes, while the Data Record says 350 and 300, and the methods text mentions 375 topography codes. These inconsistencies need to be reconciled before the corpus is trustworthy as a reference standard.

The NER experiments are fine as a learnability check, but they do not validate correctness — training and evaluating on the same corpus only shows the labels are internally coherent, which is expected. The synthetic nature of the text also means generalisation to real EHRs is untested. These are not fatal flaws, but they should be stated more carefully.

My overall sense: this deserves a serious referee, not a desk rejection. The resource is useful and the work is honest, but the presentation oversells it. I would recommend acceptance after major revision — the authors should either provide some form of external validation on the manual codes (e.g., a third annotator or a second panel on a sample), release per-item confidence or review status, and fix the count inconsistencies. They should also soften the "gold-standard" wording to something like "expert-annotated reference corpus" unless the validation can back it up.

Referee Report

4 major / 6 minor

Summary. FRACCO is a French-language corpus of 1,301 synthetic clinical oncology case reports, translated from CANTEMIST via FRASIMED, with three entity layers (morphologie, topographie, différenciation) and a fourth composite layer (expression_CIM). A total of 71,127 ICD-O-3.1 normalisations are claimed, produced by automated dictionary matching plus manual review. The paper reports span-level inter-annotator agreement (soft F1 0.82–0.90, hard F1 0.70–0.90, with différenciation at 0.48), normalisation concordance (80.65% on an automatic sample, 51.5% on a manual sample), and NER baselines (weighted F1 ≈ 89%). The authors position the resource as a gold standard for French oncology NER and normalisation.

Significance. FRACCO addresses a genuine gap: there are few French oncology corpora with ICD-O normalisation and no existing resource with expression-level composite codes. The dataset is publicly released on Zenodo with code on GitHub, and the paper includes explicit error analyses and benchmark results. If the annotation quality can be substantiated, this would be a valuable resource for French clinical NLP and for cross-lingual studies building on CANTEMIST. The main weakness is that the most novel layer — manually assigned ICD-O codes for complex expressions — rests on a consensus step after only 51.5% inter-annotator concordance, with no external validation; this weakens the 'gold-standard' designation.

major comments (4)

[Technical Validation, ICD-O Normalisation Validation] The 51.5% concordance on the manual normalisation sample is a load-bearing result. The manual subset consists of the complex expressions that are the paper's most novel contribution (2,043 unique expression_CIM codes). With only two annotators, a raw concordance of 51.5%, no kappa statistic, and a consensus step whose protocol is not described, the claim that the final codes are a gold standard is not supported. Consensus can average errors rather than resolve them. I request either (a) independent adjudication by a second panel or comparison against an external gold standard on a sample of the manual codes, with kappa and per-category error analysis; or (b) a substantial softening of the gold-standard claim, describing the resource as 'expert-annotated with moderate agreement and consensus-based normalisation'.
[Abstract vs. Methods vs. Data Record] Unique-code counts are inconsistent across sections: the Abstract reports 399 morphology and 272 topography codes; the Methods reports 399 morphology and 375 topography codes; the Data Record reports 350 morphology and 300 topography codes. The document count also wavers between 1,301 (Abstract, Methods) and 1,300 (Technical Validation). Total annotations are given as 71,127, 71,065, and 'over 70,000'. These discrepancies must be reconciled; they currently prevent a precise assessment of the dataset's coverage.
[Technical Validation, NER model fine-tuning] The fine-tuning results show that the annotation scheme is learnable by BERT-based models, but they do not validate the correctness of the ICD-O codes. Because the test set is a random split of the same corpus, annotated using the same consensus decisions, the high F1 scores only demonstrate internal consistency. The text says these procedures 'confirm both the internal quality of the annotations and the usability of the dataset', which overstates the evidence. Please explicitly state that the NER results are not evidence for the external validity of the normalisation codes.
[Technical Validation, Span Annotation Validation] The reconciliation rule that 'unresolved items default to the version provided by one annotator' can inject a systematic bias, especially for the différenciation category with hard F1 = 0.48. No post-reconciliation agreement is reported. Please report final agreement after reconciliation (e.g., on a held-out sample) and describe how the default choice was made.

minor comments (6)

[Throughout] Typographical errors: 'ditributions' (Figure 1), 'disribution' (Figure 5), 'Precisioin' (Affiliations). Also, terminology is mixed between 'histologic differentiation', 'différenciation', and 'differentiation'; please unify.
[Table 1] The composite code 'C34.9 8010/34' is likely a typo; it should probably be 'C34.9 8010/3 4' or an explicitly defined concatenation. Please clarify the exact string representation of composite codes.
[Figure 5] The code '0* (grade 0)' is marked non-standard. Please explain why this non-standard code is included in the corpus and how it is distinguished from standard differentiation codes.
[Methods, ICD-O normalisation] The phrase 'were flagged for manual and normalisation' appears to be missing a word; should be 'manual normalisation'.
[Usage Notes] The example filename 'case001.txt/case001.ann' is inconsistent with the actual naming convention described earlier ('cc_onco859.txt'). Please use a consistent example.
[References] Reference 15 (Stenetorp et al., brat) lacks full bibliographic details including publication venue and year.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: FRACCO's annotations are anchored to external CANTEMIST/ICD-O resources; internal validations are quality checks, not predictions derived from fitted inputs.

full rationale

Walking the derivation chain, FRACCO's core claim is corpus construction, not a prediction derived from a fitted model. The texts and morphology annotations originate from the external CANTEMIST corpus (ref 13) and are projected onto French translations, then manually corrected and extended (Methods: Annotation projection; Refinement and extension). The ICD-O normalisation is anchored to an external ontology through dictionary matching against an ICD-O terminology lexicon, with discrepancies and unmatched terms sent to manual review (Methods: ICD-O normalisation). The expression_CIM layer is a compositional annotation combining morphology, topography, and differentiation codes already present in the corpus, not a result that is used to define those components. The validation sections report inter-annotator agreement and NER fine-tuning on held-out splits. The low 51.5% concordance on manual normalisations is a legitimate accuracy/robustness concern, but it is not circularity: consensus adjudication is a corpus construction procedure, not a prediction that is forced by the same data. The NER experiments demonstrate learnability and internal coherence, and the authors do not use the model scores as proof of external gold-standard correctness. The only self-citation, FRASIMED (ref 14), is used for provenance of the translated texts and overlaps with the current authors, but it is not load-bearing for the dataset's annotation quality, which is independently grounded in ICD-O and expert review. No step in the paper reduces by construction to its own inputs, so no circular step is identified.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two domain assumptions: the fidelity of annotation projection after machine translation, and the correctness of consensus coding given low pre-consensus agreement. A further assumption is that dictionary coverage is adequate and that NER performance validates annotation quality. No free parameters or invented entities are introduced.

assumptions (4)

domain assumption Projected annotations from CANTEMIST transfer semantic equivalence to French after machine translation via character-level alignment.
Used in Methods/Annotation projection; if false, span and code errors propagate despite manual review.
domain assumption Consensus resolution among annotators yields correct ICD-O codes for ambiguous or composite expressions.
The low 51.5% pre-consensus IAA on manual normalisations makes this assumption load-bearing for the gold-standard claim.
domain assumption The ICD-O dictionary and regular-expression matching cover the French clinical expressions in the corpus.
Dictionary matching is the main automated normalisation pathway; coverage gaps affect completeness even though unmatched terms are flagged.
domain assumption Fine-tuning NER models on the corpus demonstrates annotation coherence and usability.
The technical validation treats model learnability as evidence of quality, but this only shows internal consistency, not external correctness.

how reviews work

0 comments

Cite this review

Pith. "Pith review of FRACCO: A gold-standard annotated corpus of oncological entities with ICD-O-3.1 normalisation." pith.science (2026). https://pith.science/paper/ML5EB3VL

@misc{pith2026251013873,
  author       = {Pith},
  title        = {Pith review of: FRACCO: A gold-standard annotated corpus of oncological entities with ICD-O-3.1 normalisation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ML5EB3VL}},
  note         = {Machine review of arXiv:2510.13873}
}

read the original abstract

Developing natural language processing tools for clinical text requires annotated datasets, yet French oncology resources remain scarce. We present FRACCO (FRench Annotated Corpus for Clinical Oncology) an expert-annotated corpus of 1301 synthetic French clinical cases, initially translated from the Spanish CANTEMIST corpus as part of the FRASIMED initiative. Each document is annotated with terms related to morphology, topography, and histologic differentiation, using the International Classification of Diseases for Oncology (ICD-O) as reference. An additional annotation layer captures composite expression-level normalisations that combine multiple ICD-O elements into unified clinical concepts. Annotation quality was ensured through expert review: 1301 texts were manually annotated for entity spans by two domain experts. A total of 71127 ICD-O normalisations were produced through a combination of automated matching and manual validation by a team of five annotators. The final dataset representing 399 unique morphology codes (from 2549 different expressions), 272 topography codes (from 3143 different expressions), and 2043 unique composite expressions (from 11144 different expressions). This dataset provides a reference standard for named entity recognition and concept normalisation in French oncology texts.

Figures

Figures reproduced from arXiv: 2510.13873 by the authors.

**Figure 2.** most common expression_CIM codes: 8000/1 (Neoplasm, uncertain whether benign or malignant), 8000/6 (Neoplasm, metastatic), 8000/3 (Neoplasm, malignant), C34.9 (Lung, NOS), C77.9 (Lymph node, NOS), C22.0 (Liver), C77.1 (Intrathoracic lymph nodes), C49.9 (Connective, subcutaneous and other soft tissues, NOS), C77.0 (Lymph nodes of head, face and neck). NOS: « not otherwise specified » Each annotated entity was also no… view at source ↗

**Figure 3.** most common morphologie codes: 8000/1 (Neoplasm, uncertain whether benign or malignant), 8000/6 (Neoplasm, metastatic), 8000/3 (Neoplasm, malignant), 8140/3 (Adenocarcinoma, NOS), 8010/3 (Carcinoma, NOS), 8500/3 (Infiltrating [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (7 more)

**Figure 4.** Figure 4: most common topographie codes: C34.9 (Lung, NOS), C22.0 (Liver), C50.9 (Breast, NOS), C76.1 (Thorax, NOS), C41.9 (Bone, NOS), C76.0 (Head, face or neck, NOS), C49.9 (Connective, subcutaneous and other soft tissues, NOS), C38.3 (Mediastinum, NOS), C71.0 (Cerebrum), C64.…

**Figure 5.** Figure 5: disribution of differenciation codes: 4 (Undifferentiated, grade IV), 3 (Poorly differentiated, grade III), 2 (Moderately differentiated, grade II), 1 (Well differentiated, grade I), 0* (“grade 0”). *Non- standard [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]

**Figure 6.** Figure 6: example of annotations labels These annotations capture clinically meaningful multi-attribute expressions (e.g., “carcinome épidermoïde bien différencié du poumon”) and are each linked to their corresponding ICD-O codes. The creation of this layer enables higher-level …

**Figure 7.** Figure 7: inter-annotator agreement (partial agreement included) [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

**Figure 8.** Figure 8: inter-annotator agreement (perfect agreement only) [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

**Figure 9.** Figure 9: Test set F1 scores (micro, macro and weighted) for five pretrained models fine-tuned on the corpus. [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]

**Figure 10.** Figure 10: Per-entity F1 scores (morphologie, topographie, différenciation) on the test set for five pretrained models fine [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 1 linked inside Pith

[1]

Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review

Koleck TA, Dreisbach C, Bourne PE, Bakken S. Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review. Journal of the American Medical Informatics Association. 2019 Apr 1;26(4):364–79

2019
[2]

Analysis of free text in electronic health records for identification of cancer patient trajectories

Jensen K, Soguero-Ruiz C, Oyvind Mikalsen K, Lindsetmo RO, Kouskoumvekaki I, Girolami M, et al. Analysis of free text in electronic health records for identification of cancer patient trajectories. Sci Rep. 2017 Apr 7;7(1):46226

2017
[3]

Clinical Text Data in Machine Learning: Systematic Review

Spasic I, Nenadic G. Clinical Text Data in Machine Learning: Systematic Review. JMIR Medical Informatics. 2020 Mar 31;8(3):e17984

2020
[4]

Building a semantically annotated corpus of clinical texts

Roberts A, Gaizauskas R, Hepple M, Demetriou G, Guo Y , Roberts I, et al. Building a semantically annotated corpus of clinical texts. Journal of Biomedical Informatics. 2009 Oct 1;42(5):950–66

2009
[5]

Temporal Annotation in the Clinical Domain

Styler WF IV , Bethard S, Finan S, Palmer M, Pradhan S, de Groen PC, et al. Temporal Annotation in the Clinical Domain. Transactions of the Association for Computational Linguistics. 2014 Apr 1;2:143–54

2014
[6]

Task definition, annotated dataset, and supervised natural language processing models for symptom extraction from unstructured clinical notes

Steinkamp JM, Bala W, Sharma A, Kantrowitz JJ. Task definition, annotated dataset, and supervised natural language processing models for symptom extraction from unstructured clinical notes. Journal of Biomedical Informatics. 2020 Feb 1;102:103354

2020
[7]

Synthetic data for annotation and extraction of family history information from clinical text

Brekke PH, Rama T, Pilán I, Nytrø Ø, Øvrelid L. Synthetic data for annotation and extraction of family history information from clinical text. J Biomed Semant. 2021 Jul 14;12(1):11

2021
[8]

Annotation of a Large Clinical Entity Corpus

Patel P, Davey D, Panchal V , Pathak P. Annotation of a Large Clinical Entity Corpus. In: Riloff E, Chiang D, Hockenmaier J, Tsujii J, editors. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing [Internet]. Brussels, Belgium: Association for Computational Linguistics; 2018 [cited 2025 Jul 9]. p. 2033–42. Available from:...

2018

Show all 18 references

[9]

Use of the Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) for Processing Free Text in Health Care: Systematic Scoping Review

Gaudet-Blavignac C, Foufi V , Bjelogrlic M, Lovis C. Use of the Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) for Processing Free Text in Health Care: Systematic Scoping Review. Journal of Medical Internet Research. 2021 Jan 26;23(1):e24594

2021
[10]

A French clinical corpus with comprehensive semantic annotations: development of the Medical Entity and Relation LIMSI annOtated Text corpus (MERLOT)

Campillos L, Deléger L, Grouin C, Hamon T, Ligozat AL, Névéol A. A French clinical corpus with comprehensive semantic annotations: development of the Medical Entity and Relation LIMSI annOtated Text corpus (MERLOT). Lang Resources & Evaluation. 2018 Jun 1;52(2):571–601

2018
[11]

[cited 2025 May 21]

Papers with Code - The QUAERO French Medical Corpus Dataset [Internet]. [cited 2025 May 21]. Available from: https://paperswithcode.com/dataset/the-quaero-french-medical-corpus

2025
[12]

SIFR annotator: ontology-based semantic annotation of French biomedical text and clinical notes

Tchechmedjiev A, Abdaoui A, Emonet V , Zevio S, Jonquet C. SIFR annotator: ontology-based semantic annotation of French biomedical text and clinical notes. BMC Bioinformatics. 2018 Nov 6;19(1):405

2018
[13]

Cantemist corpus: gold standard of oncology clinical cases annotated with CIE-O 3 terminology [Internet]

Miranda-Escalada A, Farré E, Krallinger M. Cantemist corpus: gold standard of oncology clinical cases annotated with CIE-O 3 terminology [Internet]. Zenodo; 2020 [cited 2024 Dec 12]. Available from: https://zenodo.org/record/3773228

2020
[14]

FRASIMED: a Clinical French Annotated Resource Produced through Crosslingual BERT-Based Annotation Projection [Internet]

Zaghir J, Bjelogrlic M, Goldman JP, Aananou S, Gaudet-Blavignac C, Lovis C. FRASIMED: a Clinical French Annotated Resource Produced through Crosslingual BERT-Based Annotation Projection [Internet]. arXiv; 2023 [cited 2024 Nov 28]. Available from: http://arxiv.org/abs/2309.10770

2023 arXiv
[15]

brat: a Web-based Tool for NLP- Assisted Text Annotation

Stenetorp P, Pyysalo S, Topic G, Ohta T, Ananiadou S, Tsujii J. brat: a Web-based Tool for NLP- Assisted Text Annotation
[16]

Developing and Validating an Automatic Support System for Tumor Coding in Pathology Reports in Spanish

Villena F, Báez P, Peñafiel S, Rojas M, Paredes I, Dunstan J. Developing and Validating an Automatic Support System for Tumor Coding in Pathology Reports in Spanish. JCO Clin Cancer Inform. 2025 Feb;(9):e2400124

2025
[17]

Natural language processing for structuring clinical text data on depression using UK-CRIS

Vaci N, Liu Q, Kormilitzin A, De Crescenzo F, Kurtulmus A, Harvey J, et al. Natural language processing for structuring clinical text data on depression using UK-CRIS. Evidence-Based Mental Health. 2020 Feb 11;23:21–6

2020
[18]

Enhancing medical coding efficiency through domain- specific fine-tuned large language models

Hou Z, Liu H, Bian J, He X, Zhuang Y . Enhancing medical coding efficiency through domain- specific fine-tuned large language models. npj Health Syst. 2025 May 1;2(1):14

2025

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.

[1] [1]

Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review

Koleck TA, Dreisbach C, Bourne PE, Bakken S. Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review. Journal of the American Medical Informatics Association. 2019 Apr 1;26(4):364–79

2019

[2] [2]

Analysis of free text in electronic health records for identification of cancer patient trajectories

Jensen K, Soguero-Ruiz C, Oyvind Mikalsen K, Lindsetmo RO, Kouskoumvekaki I, Girolami M, et al. Analysis of free text in electronic health records for identification of cancer patient trajectories. Sci Rep. 2017 Apr 7;7(1):46226

2017

[3] [3]

Clinical Text Data in Machine Learning: Systematic Review

Spasic I, Nenadic G. Clinical Text Data in Machine Learning: Systematic Review. JMIR Medical Informatics. 2020 Mar 31;8(3):e17984

2020

[4] [4]

Building a semantically annotated corpus of clinical texts

Roberts A, Gaizauskas R, Hepple M, Demetriou G, Guo Y , Roberts I, et al. Building a semantically annotated corpus of clinical texts. Journal of Biomedical Informatics. 2009 Oct 1;42(5):950–66

2009

[5] [5]

Temporal Annotation in the Clinical Domain

Styler WF IV , Bethard S, Finan S, Palmer M, Pradhan S, de Groen PC, et al. Temporal Annotation in the Clinical Domain. Transactions of the Association for Computational Linguistics. 2014 Apr 1;2:143–54

2014

[6] [6]

Task definition, annotated dataset, and supervised natural language processing models for symptom extraction from unstructured clinical notes

Steinkamp JM, Bala W, Sharma A, Kantrowitz JJ. Task definition, annotated dataset, and supervised natural language processing models for symptom extraction from unstructured clinical notes. Journal of Biomedical Informatics. 2020 Feb 1;102:103354

2020

[7] [7]

Synthetic data for annotation and extraction of family history information from clinical text

Brekke PH, Rama T, Pilán I, Nytrø Ø, Øvrelid L. Synthetic data for annotation and extraction of family history information from clinical text. J Biomed Semant. 2021 Jul 14;12(1):11

2021

[8] [8]

Annotation of a Large Clinical Entity Corpus

Patel P, Davey D, Panchal V , Pathak P. Annotation of a Large Clinical Entity Corpus. In: Riloff E, Chiang D, Hockenmaier J, Tsujii J, editors. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing [Internet]. Brussels, Belgium: Association for Computational Linguistics; 2018 [cited 2025 Jul 9]. p. 2033–42. Available from:...

2018