Pith. sign in

REVIEW 3 major objections 5 minor 69 references

Ontology- and LLM-based Data Harmonization for Federated Learning in Healthcare

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A two-step pipeline that retrieves candidate ontology terms by embedding similarity or ontology cross-references and then lets a large language model accept only equivalent-or-more-generic matches aligns heterogeneous EHR text to…

desk verdict The headline 78–92% precision number is actually overall agreement; recomputing accepted-pair precision from their own tables gives as low as ~32% for the SNOMED generator. read the letter →

arxiv 2505.20020 v1 pith:CRTVOND6 submitted 2025-05-26 cs.LG cs.SE

Natallia Kokash , Lei Wang , Thomas H. Gillespie , Adam Belloum , Paola Grosso , Sara Quinney , Lang Li , Bernard de Bono This is my paper · ORCID
classification cs.LGcs.SE
keywords dataharmonizationfederatedlearninglargelanguagemodelsbiomedicalontologieselectronichealthrecordsretrieval-augmentedgenerationSNOMEDCTICD-10
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a two-step, mostly automated pipeline can harmonize heterogeneous electronic health record data well enough for federated learning, without a team of human curators. The first step produces candidate matches by embedding-based retrieval or by chaining ontology cross-references; the second step asks a large language model to accept a candidate only when the target term is equivalent to, or more generic than, the source description. The authors test this on real maternal/pediatric drug-safety data, mapping short free-text pregnancy outcomes to MONDO and HPO terms and mapping ICD-10-coded outcomes to those same ontologies. Measured against one physician's judgments, the LLM agreed on 92% of decisions in the free-text round and on 78% and 91% in the two ICD-10 rounds, which the authors summarize as 78–92% mapping precision with little or no human expertise needed.

What carries the argument

The load-bearing mechanism is the acceptance criterion plus the LLM's adjudication of candidate pairs. The pipeline builds an embedding index of target ontology labels and synonyms with their identifiers, retrieves the top few nearest terms for each input record, and forms candidate pairs; it then prompts the LLM to decide whether the two descriptions denote the same condition, accepting the pair only if the target term is equivalent or more generic. A second generator chains ICD-10 to SNOMED CT and then to MONDO/HPO through existing cross-references, producing a larger, noisier candidate pool that the same LLM filter then prunes. This criterion converts an open-ended mapping task into a binary semantic judgment, which is what makes the LLM's decisions comparable to a human expert's.

What would settle it

Take the released evaluation pairs and have several independent clinicians, blinded to the LLM's answers, apply the same same-or-more-generic criterion; then compute pairwise agreement among clinicians and against the LLM. If clinician–clinician agreement is no higher than clinician–LLM agreement, the 78–92% figures reflect a lenient or idiosyncratic gold standard rather than a reliable precision estimate.

Watch

Extended reading notes

Core claim

The central discovery is that the difficult part of EHR-to-ontology conversion is not finding plausible target terms but deciding which plausible term is right in context, and that an LLM can perform that decision when given an explicit acceptance rule. The paper shows that raw embedding similarity alone is not reliable enough; judged by a physician, many top-retrieved candidates are too specific, too narrow, or otherwise mismatched. Adding a second LLM step that accepts only pairs where the target is the same disease or a more generic description raises the usable precision to the reported 78–92% agreement band across the three evaluation rounds. The pipeline also covers the hard case of ICD-10-to-MONDO/HPO translation where identifier cross-references are sparse, using SNOMED CT as an intermediate bridge and embeddings as an alternative generator.

Load-bearing premise

The reported precision numbers assume that one physician's yes/no decisions, made after a criteria clarification, are the correct ground truth for whether two descriptions refer to the same or a more generic condition.

Editorial extensions

If this is right

  • Short, abbreviation-heavy EHR entries that defeat conventional ontology search can still be mapped when embeddings propose candidates and the LLM filters them.
  • Ontology translation can proceed even when direct identifier cross-references are missing, as in ICD-10 to MONDO/HPO, by using SNOMED CT as an intermediate or by embedding retrieval.
  • LLM validation materially improves precision over raw vector-similarity matching, while the candidate generator determines recall; the two roles should be tuned separately.
  • Combining the RAG and SNOMED-based generators improves coverage, since they agree on only 475 of 1162 ICD-10 codes and relying on one generator leaves gaps.
  • As a reusable data-alignment function inside a programmable federated workflow platform, the pipeline lowers the data-preparation barrier that currently blocks FL deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the same-or-more-generic criterion is made precise, it can be checked automatically against ontology hierarchy edges, so future work could replace human review with a formal subsumption oracle in the overlap between source and target ontologies.
  • The top-3 retrieval limit likely underestimates recall: with top-10 retrieval, 98% of ICD-10 codes would have a relevant candidate available by the SNOMED route, so an LLM filter over a larger pool should be tested.
  • Because the current pipeline calls an LLM over a network, a practical deployment inside a healthcare federation would need a locally hosted model or privacy-preserving decoding before the approach can be used on identifiable records, a constraint the paper itself raises.
  • Because agreement was measured against a single physician, a multi-site study with several clinicians and per-site acceptance criteria would give a more transferable estimate of precision.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper describes a two-step pipeline for harmonizing heterogeneous EHR data against biomedical ontologies (MONDO, HPO) in a federated learning context. In the first step, candidate ontology terms are generated either by embedding-based retrieval (RAG) or by cross-referencing ICD-10 through SNOMED CT; in the second step, an LLM (ChatGPT-4o) applies a stated acceptance criterion (equivalent or more generic target) to accept or reject each candidate. The authors evaluate the pipeline on two real-world datasets: an unannotated set of pregnancy/neonatal outcome descriptions and a set of ICD-10-coded outcomes. The reported conclusion is that the approach achieves mapping precision of 78%–92% and can align data with very limited human expertise.

Significance. The problem of data harmonization in federated health research is timely and important, and the proposed two-step design (retrieval plus LLM adjudication) is a practical, clearly described contribution. The manuscript is notably transparent: it provides raw contingency counts in Figures 6 and 9, discusses the human expert's criteria clarification in Section IV.A, and releases evaluation datasets and scripts (Zenodo). These strengths make the work reproducible and the numbers checkable. However, the headline quantitative claim is not supported by the paper's own data: the reported 78%–92% figures are overall agreement rates, not precision, and the true precision of LLM-accepted mappings drops to as low as 31.7% for the SNOMED-based generator. In addition, the evaluation uses a single human expert whose judgments were guided by the same acceptance criteria given to the LLM, so the agreement metric does not measure correctness against an independent gold standard. The core idea is defensible, but the evidence as presented substantially overstates the claim that the pipeline can replace or drastically reduce human expertise.

major comments (3)
  1. [Section VI and Figures 6, 9] The concluding claim that 'the mapping precision in our experiments ranged from 78% to 92%' is not precision but overall agreement (accuracy). Using the paper's own definitions in Section IV.A (precision = proportion of predicted ontology terms that are correct) and the contingency tables: in the first dataset (Figure 6, revised, yes-yes=352, no-yes=98), precision is 352/(352+98)=78.2%, while the 92% figure is overall agreement (352+944)/1401=92.5%. In the second dataset, RAG-based generator (Figure 9a, yes-yes=155, no-yes=153) gives precision 155/(155+153)=50.3%, and the SNOMED-based generator (Figure 9b, yes-yes=38, no-yes=82) gives precision 38/(38+82)=31.7%. Thus the range of precision is approximately 32%–78%, not 78%–92%. This is load-bearing because the conclusion that the pipeline aligns data 'effectively ... without or with very limited involvement of human expertise' depends on high precision of the accepted mappings; the reported numbers show that a large fraction of LLM-accepted mappings would be rejected by the human expert, especially for the SNOMED-based generator. The manuscript must correct the presentation of these metrics and re-evaluate the stated conclusion accordingly.
  2. [Sections IV.A and IV.B] The evaluation uses a single human expert (MD) as the reference standard, and that expert's criteria were explicitly aligned with the LLM acceptance criterion: Section IV.A reports that after clarifying the requirements 'similarly to the LLM prompt, asking to accept the mapping only if the target mapping is the same or more generic', the expert retracted 11 of 18 initially positive decisions. This creates a non-independent standard: the comparison measures agreement between two systems that were deliberately given the same decision rule, not correctness against an external gold standard. As a result, the reported agreement rates do not establish that the LLM's accept/reject decisions are correct. To support the central claim, the paper should either obtain an independent gold standard (e.g., a second expert annotating without exposure to the prompt, or a curated reference set) or at minimum report inter-annotator agreement and discuss how the alignment of criteria affects the interpretation. The random subsets (728 and 915 out of 3129 and 7787 pairs) are reasonable in size but inherit this limitation.
  3. [Section IV.B and Figure 9] The operational claim that the pipeline requires 'very limited involvement of human expertise' is not supported by the data for the SNOMED-based generator, where only 31.7% of LLM-accepted mappings are also accepted by the human expert (TP=38, FP=82). This implies that, for this generator, roughly two-thirds of the mappings that would flow into a downstream federated analysis are false positives. Even for the RAG-based generator, precision is 50.3%. The paper discusses candidate generation and acceptance rates but does not translate these results into a concrete human-review workload or a recall estimate. Without reporting how many true mappings are missed (recall) and what the expected number of false positives per input code is, the conclusion that the pipeline reduces human effort 'effectively' is premature. The authors should either add a human-in-the-loop workflow analysis or temper the claim.
minor comments (5)
  1. [Figure 9] The caption of Figure 9(b) appears garbled: '38793 82' should likely read '38 793 82' to match the bar labels; please fix.
  2. [Figure 1 caption] The caption contains 'V ANTAGE6' with an extra space; correct to 'Vantage6'.
  3. [Section IV.A, Table III] The table row for '5-minutes Apgar<7' is missing the explanation column entry; please complete the cell or remove the row if the explanation is intentionally omitted.
  4. [Section IV.B] The text states 'For more relaxed acceptance criteria, i.e., whether both descriptions refer to the same disease or to the related but more general or more specific condition, the acceptance ratios were 71% and 80%, respectively.' It would be helpful to state explicitly which acceptance ratio corresponds to which generator and to include these numbers in Figure 9 or a table.
  5. [Section III and Figure 4] The description of the embedding step (e.g., which embedding model is used, dimensionality, similarity metric) is absent; adding these details would improve reproducibility, though the code repository partially addresses this.

Circularity Check

2 steps flagged · score 4.0 of 10

The evaluation gold standard is partly defined by the LLM's own acceptance criterion, and the headline 'precision 78-92%' relabels overall agreement as precision; the empirical pipeline itself is transparently evaluated and not fitted.

  1. self definitional [Section IV.A, evaluation of the unannotated dataset (paragraph discussing Figure 6)]
    "We clarified the requirements for the assessment of related outcomes similarly to the LLM prompt, asking to accept the mapping only if the target mapping is the same or more generic. The human expert retracted 11 out of 18 initially accepted mappings that he considered acceptable for the study context but that formally did not match the aforementioned relation."

    The ground truth used to validate the LLM (the human expert's accept/reject labels) is produced by instructing the expert with the LLM's own acceptance rule ('similarly to the LLM prompt'). After this recalibration, LLM-vs-human agreement measures how consistently two raters implement the same decision rule rather than whether the LLM's mappings are correct by an independent standard. The conclusion that data can be aligned 'without or with very limited involvement of human expertise' therefore rests partly on agreement with an evaluator whose judgment was aligned to the system's criterion.

  2. other [Section VI, Conclusions]
    "We showed that the two-step LLM-assisted conversion of data can be used effectively to align heterogeneous datasets to standardized vocabulary (the mapping precision in our experiments ranged from 78% to 92%), without or with very limited involvement of human expertise."

    Section IV.A defines precision as 'the proportion of predicted ontology terms that are correct,' i.e., TP/(TP+FP). The 78-92% figures quoted in the conclusion are not that quantity: 92% is overall agreement on the first dataset, (352+933)/1401, and 78% and 91% are the agreement rates in Figures 9a and 9b, i.e., (155+413)/727 and (38+793)/914. Recomputing precision as accepted-pair precision from the authors' own contingency tables gives 352/(352+98)=78.2%, 155/(155+153)=50.3%, and 38/(38+82)=31.7%.

full rationale

This paper's derivation chain is empirical rather than mathematical: the pipeline (embedding- or SNOMED-based candidate generation plus LLM adjudication) is evaluated against a human expert's accept/reject judgments, with all raw contingency counts reported in Figures 6 and 9. There is no fitted parameter later renamed as a prediction, and no load-bearing self-citation: the Brane/EPI references ([61], [6], [26]) supply architectural context in Sections II and do not justify the mapping results. The main circularity concern is the evaluation standard: when the human expert's assessment disagreed with the LLM, the authors 'clarified the requirements for the assessment of related outcomes similarly to the LLM prompt,' i.e., the gold standard was set by the system's own acceptance rule. Agreement measured after this calibration partly reduces to agreement between two raters applying the same criterion, which is self-referential; it is only partially circular because the criterion (target mapping equivalent or more generic) is an externally meaningful ontological relation, and even the pre-clarification agreement was 91.7% with 98 of 1401 LLM approvals still rejected. A second load-bearing issue is numerical rather than circular: the conclusion's 'mapping precision ... 78% to 92%' is overall agreement relabeled as precision; recomputing precision as TP/(TP+FP) from the authors' own tables gives 78.2%, 50.3%, and 31.7%, so the claim that data can be aligned 'without or with very limited involvement of human expertise' is overstated. Because the central pipeline is evaluated transparently against an independent (if partially recalibrated) human judge and the claimed numbers are checkable from the paper's own tables, the overall circularity score is moderate rather than high.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new ontology, force, model, or entity is introduced. The pipeline is assembled from existing components. The main hidden costs are the hand-chosen retrieval depth (k=3), the tuned prompt criterion, the single-expert ground truth, and the SNOMED cross-reference assumption. These should be read as the price of the empirical claims.

free parameters (3)
  • top-k retrieval limit for RAG generator = 3 (with the paper noting 10 would improve recall)
    Section IV.A and IV.B: the pipeline retrieves at most 3 candidate terms per record. The authors explicitly state that retrieving 10 would cover 98% of entries instead of 54%, so the chosen k=3 limits measured recall and affects all agreement numbers.
  • acceptance criterion wording in the LLM prompt = equivalent or more generic target only
    Section IV.A: the prompt asks the LLM to answer Yes only if the second description is the same or broader. The paper shows that a stricter or looser criterion (related condition accepted) changes agreement rates from 92% to 71% or 80%, so the criterion is a tuned knob, not an outcome.
  • random subset sizes for expert evaluation = 728 and 915 records for the two ICD-10 pipelines
    Section IV.B: the human expert reviewed only random subsets of the candidate pairs. The reported 78% and 91% agreement are statistics on these subsets, and no confidence intervals are provided.
assumptions (4)
  • domain assumption MONDO and HPO cross-references to SNOMED CT are reliable enough to generate candidate mappings.
    Section IV.B: the SNOMED-based generator pairs all ICD-10 codes with MONDO/HPO codes sharing a SNOMED identifier. The paper acknowledges the known imperfections of SNOMED-to-ICD-10 mapping but relies on the cross-reference paths as the candidate source.
  • domain assumption Vector-semantic similarity between embedded clinical phrases and ontology labels is a useful proxy for relatedness.
    Section IV.A and Figure 5: the retriever returns the top-3 most relevant terms by embedding distance. The assumption is that the embedding space organizes the concise, sometimes abbreviated clinical phrases near their correct ontology terms.
  • domain assumption One medical doctor's expert judgement, after being re-briefed with the LLM's criteria, is a valid ground truth for mapping correctness.
    Sections IV.A and IV.B: all agreement percentages are measured against this single expert. The expert even retracted 11 of 18 initial acceptances after criteria clarification, showing the ground truth depends on instruction framing.
  • standard math LLM outputs at the time of the experiment are representative of the pipeline's behavior.
    The evaluation uses ChatGPT-4o through an API with no reported version pin or temperature. The paper does not address the non-determinism of LLM outputs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Ontology- and LLM-based Data Harmonization for Federated Learning in Healthcare." pith.science (2026). https://pith.science/paper/CRTVOND6

@misc{pith2026250520020,
  author       = {Pith},
  title        = {Pith review of: Ontology- and LLM-based Data Harmonization for Federated Learning in Healthcare},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CRTVOND6}},
  note         = {Machine review of arXiv:2505.20020}
}
read the original abstract

The rise of electronic health records (EHRs) has unlocked new opportunities for medical research, but privacy regulations and data heterogeneity remain key barriers to large-scale machine learning. Federated learning (FL) enables collaborative modeling without sharing raw data, yet faces challenges in harmonizing diverse clinical datasets. This paper presents a two-step data alignment strategy integrating ontologies and large language models (LLMs) to support secure, privacy-preserving FL in healthcare, demonstrating its effectiveness in a real-world project involving semantic mapping of EHR data.

Figures

Figures reproduced from arXiv: 2505.20020 by the authors.

Figure 1
Figure 1. VANTAGE6 [44] The server loads its configuration parameters and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Brane’s approach to the distributed workflow implementation via [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. shows the main components of the framework, which are: • the orchestrators (both at the application level and infras￾tructural level); • the policy management system; • the components required to be present at the participating institutions: the resource provisional and the authorizers [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: LLM-based pipeline to align data with target vocabulary. (A) Prepare target mapping space; (B) Find best matching targets for input data; (C) Define [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: LLM-based pipeline to annotate patient outcomes with MONDO and/or HPO ontology terms. (A) Extract labels and synonyms from target databases, [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Number of data record mappings approved and rejected by a human [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: ICD-10-to-MONDO/HPO conversion via SNOMED, candidate pair [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: ICD-10-to-MONDO/HPO conversion via SNOMED, number of inputs [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 53 canonical work pages

  1. [1]

    BMC Medical Informatics and Decision Making23(1), 8 (2023)

    Abbasizanjani, H., Torabi, F., Bedston, e.a.: Harmonising electronic health records for reproducible research: challenges, solutions and recommendations from a UK-wide COVID-19 research collaboration. BMC Medical Informatics and Decision Making23(1), 8 (2023). DOI 10.1186/s12911-022-02093-0

    work page doi:10.1186/s12911-022-02093-0 2023

  2. [2]

    JMIR Medical Education10, e51391 (2024)

    Abdullahi, T., Singh, R., Eickhoff, C.: Learning to Make Rare and Complex Diagnoses With Generative AI Assistance: Qualitative Study of Popular Large Language Models. JMIR Medical Education10, e51391 (2024). DOI 10.2196/51391

    work page doi:10.2196/51391 2024

  3. [3]

    International Journal of Population Data Science5(1) (2020)

    Adhikari, K., Patten, S., Patel, A., Premji, S., Tough, S., Letourneau, N., Giesbrecht, G., Metcalfe, A.: Data harmonization and data pooling from cohort studies: a practical approach for data management. International Journal of Population Data Science5(1) (2020)

    work page 2020

  4. [4]

    Analytics Vidhya, unpublished (2023)

    Ajay: A Deep Dive into Qdrant, the Rust-Based Vector Database. Analytics Vidhya, unpublished (2023). URL https://www.analyticsvidhya.com/blog/2023/11/a-deep-dive-into-qdrant- the-rust-based-vector-database/

    work page 2023

  5. [5]

    Information Fusion96, 156–191 (2023)

    Albahri, A., Duhaim, A.M., Fadhel, M.A., Alnoor, A., Baqer, N.S., Alzubaidi, L., Albahri, O., Alamoodi, A., Bai, J., Salhi, A., Santamar ´ıa, J., Ouyang, C., Gupta, A., Gu, Y ., Deveci, M.: A systematic review of trustworthy and explainable artificial intelligence in healthcare: As- sessment of quality, bias risk, and data fusion. Information Fusion96, 15...

    doi:10.1016/j.inffus.2023.03.008 2023

  6. [6]

    Alsayed Kassem, J., Allaart, C., Amiri, S., Kebede, M., M ¨uller, T., Turner, R., Belloum, A., van Binsbergen, L.T., Grunwald, P., van Halteren, A., Grosso, P., de Laat, C., Klous, S.: Building a Digital Health Twin for Personalized Intervention: The EPI Project. In: B.R. Haverkort, A. de Jongste, P. van Kuilenburg, R.D. Vromans (eds.) Commit2Data, Open A...

    2024

  7. [7]

    Scientific Reports14(1), 29881 (2024)

    Bhanbhro, J., Nistic `o, S., Palopoli, L.: Issues in federated learning: some experiments and preliminary results. Scientific Reports14(1), 29881 (2024). DOI 10.1038/s41598-024-81732-0

    work page doi:10.1038/s41598-024-81732-0 2024

  8. [8]

    In: Proceedings of the 19th ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences, GPCE 2020, p

    van Binsbergen, L.T., Liu, L.C., van Doesburg, R., van Engers, T.: eflint: a domain-specific language for executable norm specifications. In: Proceedings of the 19th ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences, GPCE 2020, p. 124–136. Association for Computing Machinery, New York, NY , USA (2020). DOI 10.1145/34...

    work page arXiv 2020

Show all 69 references
  1. [9]

    Drug Safety20(2), 109–117 (1999)

    Brown, E.G., Wood, L., Wood, S.: The medical dictionary for regulatory activities (meddra). Drug Safety20(2), 109–117 (1999). DOI 10.2165/00002018-199920020-00002

    1999 doi

  2. [10]

    In: Proceedings of the 10th USENIX Conference on Operating Systems Design and Implemen- tation (OSDI), pp

    Burns, B., Grant, B., Oppenheimer, D., Brewer, E., Wilkes, J.: Borg, omega, and kubernetes: Lessons learned from three container- management systems over a decade. In: Proceedings of the 10th USENIX Conference on Operating Systems Design and Implemen- tation (OSDI), pp. 51–66....

    2016

  3. [11]

    Patterns (N Y)5(12), 101098 (2024)

    Chen, C., Feng, X., Li, Y ., Lyu, L., Zhou, J., Zheng, X., Yin, J.: Integration of large language models and federated learning. Patterns (N Y)5(12), 101098 (2024). DOI 10.1016/j.patter.2024.101098

    2024

  4. [12]

    An- alytics India Magazine (2020)

    Dhooper, G.: Understanding agnostic approach in machine learning. An- alytics India Magazine (2020). URL https://analyticsindiamag.com/ai- features/understanding-agnostic-approach-in-machine-learning/

    2020

  5. [13]

    Morgan Kaufmann (2012)

    Doan, A., Halevy, A.Y ., Ives, Z.G.: Principles of Data Integration. Morgan Kaufmann (2012)

    2012

  6. [14]

    Studies in Health Technology and Informatics121, 279–290 (2006)

    Donnelly, K.: Snomed-ct: The advanced terminology and coding system for ehealth. Studies in Health Technology and Informatics121, 279–290 (2006). DOI 10.3233/978-1-58603-622-8-279

    2006 doi

  7. [15]

    StatPearls [Internet]

    Edemekong, P.F., Annamaraju, P., Afzal, M., Haydel, M.J.: Health Insurance Portability and Accountability Act (HIPAA) Compliance. StatPearls [Internet]. StatPearls Publishing, Treasure Island, FL (2024). PMID: 29763195

    2024

  8. [16]

    DOI 10.48550/arXiv.2502.00138

    Esterhuyse, C.A., M ¨uller, T., van Binsbergen, L.T.: Justact+: Justified and accountable actions in policy-regulated, multi-domain data process- ing (2025). DOI 10.48550/arXiv.2502.00138

    2025 doi arXiv

  9. [17]

    Fernandez, R.C., Elmore, A.J., Franklin, M.J., Krishnan, S., Tan, C.: How large language models will disrupt data management. Proc. VLDB Endow.16(11), 3302–3309 (2023). DOI 10.14778/3611479.3611527

    2023

  10. [18]

    URL https://www.gartner.com/reviews/market/enterprise-low-code- application-platform

    Gartner, Inc.: Enterprise Low-Code Application Platforms Reviews and Ratings (2025). URL https://www.gartner.com/reviews/market/enterprise-low-code- application-platform. Accessed: 2025-04-11

    2025

  11. [19]

    Journal of the American Medical Informatics Association28(7), 1507–1517 (2021)

    Gibson, T.B., Nguyen, M.D., Burrell, T., Yoon, F., Wong, J., Dharmara- jan, S., Ouellet-Hellstrom, R., Hua, W., Ma, Y ., Baro, E., Bloemers, S., Pack, C., Kennedy, A., Toh, S., Ball, R.: Electronic phenotyping of health outcomes of interest using a linked claims-electronic hea...

    2021 doi

  12. [20]

    URL https://www.grandviewresearch.com/industry- analysis/low-code-application-development-platform-market

    Grand View Research, Inc.: Low-code application development platform market size, share & trends analysis report by com- ponent (platform, services), by application, by deployment, by organization size, by region, and segment forecasts, 2024 - 2030 (2024). URL https://www.gran...

    2024

  13. [21]

    New England Journal of Medicine379(15), 1452–1462 (2018)

    Haendel, M.A., Chute, C.G., Robinson, P.N.: Classification, ontology, and precision medicine. New England Journal of Medicine379(15), 1452–1462 (2018). DOI 10.1056/NEJMra1615014

    2018 doi

  14. [22]

    In: International Conference on Extending Database Technology (2025)

    Hellenberg, J.E., Mahling, F.D., Laskowski, L., Naumann, F., Paganelli, M., Panse, F.: Prisma: A privacy-preserving schema matcher using functional dependencies. In: International Conference on Extending Database Technology (2025). URL https://api.semanticscholar.org/CorpusID:...

    2025

  15. [23]

    https://www.docker.com/ (2024)

    Inc., D.: Docker: Enterprise container platform. https://www.docker.com/ (2024). Accessed: 2025-04-10

    2024

  16. [24]

    Computers in Biology and Medicine171, 108189 (2024)

    Jahan, I., Laskar, M.T.R., Peng, C., Huang, J.X.: A comprehensive evaluation of large language models on benchmark biomedical text processing tasks. Computers in Biology and Medicine171, 108189 (2024). DOI https://doi.org/10.1016/j.compbiomed.2024.108189

    2024

  17. [25]

    URL https://doi.org/10.13026/C2XW26

    Johnson, A.E.W., Pollard, T.J., Mark, R.G.: MIMIC-III clinical database (version 1.4) (2016). URL https://doi.org/10.13026/C2XW26. Accessed: 2025-04-10

    2016 doi

  18. [27]

    https://kidsfirstdrc.org/

    Kids First Data Resource Center. https://kidsfirstdrc.org/. Accessed: 2025-04-11

    2025

  19. [28]

    DOI 10.5281/zenodo.15411810

    Kokash, N., de Bono, B.: Ontology- and LLM-based Data Alignment Evaluation: Mapping Patient Outcomes and ICD-10 codes to MONDO and HPO ontologies (2025). DOI 10.5281/zenodo.15411810. Version 1.0.0

    2025 doi

  20. [29]

    PLOS ONE12(5), e0177459 (2017)

    Kurtzer, G.M., Sochat, V ., Bauer, M.W.: Singularity: Scientific contain- ers for mobility of compute. PLOS ONE12(5), e0177459 (2017). DOI 10.1371/journal.pone.0177459

    2017 doi

  21. [30]

    Nucleic Acids Research49(D1), D1207–D1217 (2021)

    K ¨ohler, S., et al.: The human phenotype ontology in 2021. Nucleic Acids Research49(D1), D1207–D1217 (2021). DOI 10.1093/nar/gkaa1043

    2021 doi

  22. [31]

    Journal of Biomedical Informatics46(1), 87–96 (2013)

    Lee, D., Cornet, R., Lau, F., de Keizer, N.: A survey of snomed ct implementations. Journal of Biomedical Informatics46(1), 87–96 (2013). DOI https://doi.org/10.1016/j.jbi.2012.09.006

    2013 doi

  23. [32]

    In: Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20

    Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V ., Goyal, N., K¨uttler, H., Lewis, M., Yih, W.t., Rockt ¨aschel, T., Riedel, S., Kiela, D.: Retrieval-augmented generation for knowledge-intensive NLP tasks. In: Proceedings of the 34th International Conference on Neu...

    2020

  24. [33]

    IEEE Signal Processing Magazine37(3), 50–60 (2020)

    Li, T., Sahu, A.K., Talwalkar, A., Smith, V .: Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine37(3), 50–60 (2020). DOI 10.1109/MSP.2020.2975749

    2020

  25. [34]

    arXiv preprint, unpublished (2023)

    Li, Y ., Wang, Z., Wang, Y ., Rasmy, L., Nguyen, P., Wang, Y ., Zhang, Y ., Liu, H.: Enhancing Health Data Interoperability with Large Language Models: A FHIR Study. arXiv preprint, unpublished (2023). URL https://arxiv.org/abs/2310.12989

    2023 arXiv

  26. [35]

    Master’s thesis, University of Amsterdam and Vrije Universiteit Amsterdam, Amsterdam, The Netherlands (2021)

    Liu, L.: Implementing a federated machine learning scenario using vantage6 and brane. Master’s thesis, University of Amsterdam and Vrije Universiteit Amsterdam, Amsterdam, The Netherlands (2021). URL https://staff.fnwi.uva.nl/a.s.z.belloum/MSctheses/MSchesis Leyu- Liu.pdf. Joi...

    2021

  27. [36]

    Journal of Systems and Software 220, 112278 (2024)

    Liu, Y ., Lo, S.K., Lu, Q., Zhu, L., Zhao, D., Xu, X., Harrer, S., Whittle, J.: Agent design pattern catalogue: A collection of architectural patterns for foundation model based agents. Journal of Systems and Software 220, 112278 (2024). DOI 10.1016/j.jss.2024.112278

    2024

  28. [37]

    Journal of Systems and Software192, 111357 (2022)

    Lo, S.K., Lu, Q., Zhu, L., Paik, H.y., Xu, X., Wang, C.: Architectural patterns for the design of federated learning systems. Journal of Systems and Software192, 111357 (2022). DOI 10.1016/j.jss.2022.111357

    2022

  29. [38]

    arXiv preprint arXiv:2205.13621 (2022)

    Majmudar, J., Dupuy, C., Peris, C., Smaili, S., Gupta, R., Zemel, R.: Differentially private decoding in large language models. arXiv preprint arXiv:2205.13621 (2022). URL https://arxiv.org/abs/2205.13621

    2022 arXiv

  30. [39]

    Cambridge University Press (2008)

    Manning, C.D., Raghavan, P., Sch ¨utze, H.: Introduction to Information Retrieval. Cambridge University Press (2008)

    2008

  31. [40]

    arXiv preprint, unpublished (2024)

    Matos, J., Hughes, M.C., Lehman, L.w.H., Szolovits, P., Chen, I.Y .: EHRmonize: A Framework for Medical Concept Abstraction from Elec- tronic Health Records using Large Language Models. arXiv preprint, unpublished (2024). URL https://arxiv.org/abs/2407.00242

    2024 arXiv

  32. [41]

    Yearbook of Medical Informatics 17(01), 128–144 (2008)

    Meystre, S.M., Savova, G.K., Kipper-Schuler, K.C., Hurdle, J.F.: Ex- tracting information from textual documents in the electronic health record: A review of recent research. Yearbook of Medical Informatics 17(01), 128–144 (2008)

    2008

  33. [42]

    CoRRabs/1301.3781(2013)

    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. CoRRabs/1301.3781(2013)

    2013 arXiv

  34. [43]

    Journal of the American Medical Informatics Association26(6), 491–495 (2019)

    Miller, R.A., Wright, A., Payne, T.H., Sittig, D.F., Ash, J.S.: Using snomed ct-encoded problems to improve icd-10-cm coding. Journal of the American Medical Informatics Association26(6), 491–495 (2019). DOI 10.1093/jamia/ocz017

    2019 doi

  35. [44]

    AMIA Annual Symposium Proceedings2020, 870–877 (2021)

    Moncada-Torres, A., Martin, F., Sieswerda, M., Van Soest, J., Gelei- jnse, G.: V ANTAGE6: an open source priV Acy preserviNg federaTed leArninG infrastructurE for Secure Insight eXchange. AMIA Annual Symposium Proceedings2020, 870–877 (2021)

    2021

  36. [45]

    Database2022, baac041 (2022)

    Mungall, C.J., et al.: Mondo: Unifying disease ontology representation through logical definitions. Database2022, baac041 (2022). DOI 10.1093/database/baac041

    2022 doi

  37. [46]

    Information Fusion82, 99–122 (2022)

    Nan, Y ., Ser, J.D., Walsh, S., et al.: Data harmonisation for information fusion in digital healthcare: A state-of-the-art systematic review, meta- analysis and future research directions. Information Fusion82, 99–122 (2022). DOI https://doi.org/10.1016/j.inffus.2022.01.001

    2022 doi

  38. [47]

    Information Fusion108, 102412 (2024)

    Nasarian, E., Alizadehsani, R., Acharya, U., Tsui, K.L.: Design- ing interpretable ML system to enhance trust in healthcare: A systematic review to proposed responsible clinician-AI-collaboration framework. Information Fusion108, 102412 (2024). DOI https://doi.org/10.1016/j.in...

    2024

  39. [48]

    Journal of the American Medical Informatics Association18(4), 441–448 (2011)

    Nelson, S.J., et al.: Rxnorm: A standardized nomenclature for clinical drugs. Journal of the American Medical Informatics Association18(4), 441–448 (2011). DOI 10.1136/amiajnl-2011-000149

    2011 doi

  40. [49]

    https://opencontainers.org/ (2015)

    Open Container Initiative: Open container initiative (oci). https://opencontainers.org/ (2015). Accessed: 2025-04-11

    2015

  41. [50]

    Kairouz H

    P. Kairouz H. Brendan McMahan, e.a.: Advances and Open Problems in Federated Learning. Foundations and Trends in Machine Learning 14(2021)

    2021

  42. [51]

    https://github.com/Y AMWD/PyTorch interface for Brane (2024)

    Pytorch interface for Brane. https://github.com/Y AMWD/PyTorch interface for Brane (2024). Accessed: 2025-04-16

    2024

  43. [52]

    Pharmacotherapy: The Journal of Human Pharmacol- ogy and Drug Therapy43(1), e14–e25 (2023)

    Ren, Z., Bremer, A.A., Pawlyk, A.C., et al.: The mprint hub data, model, knowledge and research coordination center: Bridging the gap in maternal–pediatric therapeutics research through data integration and pharmacometrics. Pharmacotherapy: The Journal of Human Pharmacol- ogy ...

    2023 doi

  44. [53]

    npj Digital Medicine3(1), 119 (2020)

    Rieke, N., Hancox, J., Li, W., Milletari, e.a.: The future of digital health with federated learning. npj Digital Medicine3(1), 119 (2020). DOI 10.1038/s41746-020-00323-1

    2020 doi

  45. [54]

    American journal of epidemiology182(12), 1033–1038 (2015)

    Rolland, B., Smith, B.R., Potter, J.D., Mbah, A.K., Rutter, C.M.: Toward rigorous data harmonization in cancer epidemiology research: one approach. American journal of epidemiology182(12), 1033–1038 (2015)

    2015

  46. [55]

    arXiv preprint, unpublished (2025)

    Santos, A., Garcia, A., Strubell, E., Rekatsinas, T., Li, F.T., Altman, R.B.: Interactive Data Harmonization with LLM Agents. arXiv preprint, unpublished (2025). URL https://arxiv.org/abs/2502.07132

    2025 arXiv

  47. [56]

    BMC Medical Informatics and Decision Making20(1), 222 (2020)

    Schmidt, B., Colvin, C., Hohlfeld, A., Leon, N.: Definitions, components and processes of data harmonisation in healthcare: a scoping review. BMC Medical Informatics and Decision Making20(1), 222 (2020). DOI 10.1186/s12911-020-01218-7

    2020 doi

  48. [57]

    arXiv preprint, unpublished (2024)

    Sett, A., Bhooshan, S., Xie, B., Pfohl, S., Marotta, T., Rajkomar, A.: Speaking the Same Language: Leveraging LLMs in Standardizing Clinical Data for AI. arXiv preprint, unpublished (2024). URL https://arxiv.org/abs/2408.11861

    2024 arXiv

  49. [58]

    IEEE Data Engineering Bulletin (2018)

    Stonebraker, M., Ilyas, I.: Data integration: The current status and the way forward. IEEE Data Engineering Bulletin (2018)

    2018

  50. [59]

    JCO Clinical Cancer Informatics2, 1–10 (2018)

    Topaloglu, U., Palchuk, M.B.: Using a federated network of real- world data to optimize clinical trials operations. JCO Clinical Cancer Informatics2, 1–10 (2018). DOI 10.1200/CCI.17.00067. PMID: 30652541

    2018 doi

  51. [60]

    Official Journal of the European Union (2016)

    Union, E.: General Data Protection Regulation (GDPR). Official Journal of the European Union (2016)

    2016

  52. [61]

    In: 2021 IEEE 17th International Conference on eScience (eScience), pp

    Valkering, O., Cushing, R., Belloum, A.: Brane: A framework for programmable orchestration of multi-site applications. In: 2021 IEEE 17th International Conference on eScience (eScience), pp. 277–282 (2021). DOI 10.1109/eScience51609.2021.00056

    2021

  53. [62]

    J Med Internet Res26, e22769 (2024)

    Wang, L., Wan, Z., Ni, C., Song, Q., Li, Y ., Clayton, E., Malin, B., Yin, Z.: Applications and Concerns of ChatGPT and Other Conversational Large Language Models in Health Care: Systematic Review. J Med Internet Res26, e22769 (2024). DOI 10.2196/22769

    2024 doi

  54. [63]

    Studies in Health Technology and Informatics192, 682–686 (2013)

    Wang, Y ., Bodenreider, O.: Synergism between the mapping projects from SNOMED CT to ICD-10 and ICD-10-CM. Studies in Health Technology and Informatics192, 682–686 (2013). URL https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3540534/

    2013

  55. [64]

    Journal of Biomedical Informatics77, 34–49 (2018)

    Wang, Y ., Wang, L., Rastegar-Mojarad, M., Liu, S., Shen, F., Liu, H.: Clinical information extraction applications: A literature review. Journal of Biomedical Informatics77, 34–49 (2018)

    2018

  56. [65]

    Nature594(7862), 265—270 (2021)

    Warnat-Herresthal, S., Schultze, H., Shastry, e.a.: Swarm learning for de- centralized and confidential clinical machine learning. Nature594(7862), 265—270 (2021). DOI 10.1038/s41586-021-03583-3

    2021 doi

  57. [66]

    World Health Organization (2004)

    World Health Organization: International Statistical Classification of Diseases and Related Health Problems, 10th Revision (ICD-10). World Health Organization (2004)

    2004

  58. [67]

    WHO Technical Report Series1030 (2021)

    World Health Organization: The Anatomical Therapeutic Chemical (ATC) Classification System. WHO Technical Report Series1030 (2021). URL https://www.who.int/tools/atc-ddd-toolkit

    2021

  59. [68]

    Journal of Medical Internet Research26, e56614 (2024)

    Yoon, D., Jeong, J., Kim, S., Song, M., Lee, S., Cho, I., Kim, H.Y ., Oh, J.Y ., Kim, H.H., Kim, G.H., et al.: Redefining health care data interoperability: Empirical exploration of large language models in information exchange. Journal of Medical Internet Research26, e56614 (...

    2024 doi

  60. [69]

    Nature Communications16(1), 2852 (2025)

    Zhang, F., Zhai, D., Bai, G., Jiang, J., Ye, Q., Ji, X., Liu, X.: Towards fairness-aware and privacy-preserving enhanced collaborative learning for healthcare. Nature Communications16(1), 2852 (2025). DOI 10.1038/s41467-025-58055-3

    2025 doi

  61. [70]

    Journal of Biomedical Informatics46(6), 1088–1098 (2013)

    Zhang, R., Elhadad, N.: Unsupervised biomedical named entity recog- nition: Experiments with clinical and biological texts. Journal of Biomedical Informatics46(6), 1088–1098 (2013)

    2013

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.