H2: A Dual Hybrid Semantic Data Lake Architecture for Medical Data Harmonization with Human-In-the-Loop verified, LLM Driven Metadata Annotation System

Agapi Davradou; Anastasios Doulamis; Dimitrios Kalogeras; Georgia Kapetadimitri; Ioannis N. Tzortzis; Ioannis Rallis; Nefeli Kousta; Nikolaos Bakalos; Nikolaos Doulamis

REVIEW 5 major objections 6 minor 54 references

H2: A Dual Hybrid Semantic Data Lake Architecture for Medical Data Harmonization with Human-In-the-Loop verified, LLM Driven Metadata Annotation System

T0 review · 5 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that H2, a dual hybrid semantic data lake architecture, can keep schema-less medical data queryable by pairing rule-based RDF triples with human-verified, LLM-generated operation tags, and that Gemma 3:4B delivers the…

desk verdict Solid engineering write-up, but the evaluation leaks ground truth and the soft metrics are circular, so the headline recall ranking is not yet supported. read the letter →

arxiv 2608.08056 v1 pith:KXZ5VM5E submitted 2026-08-08 cs.AI

Ioannis N. Tzortzis , Georgia Kapetadimitri , Agapi Davradou , Nefeli Kousta , Nikolaos Bakalos , Ioannis Rallis , Dimitrios Kalogeras , Nikolaos Doulamis

show 1 more author

Anastasios Doulamis

This is my paper · ORCID

classification cs.AI

keywords semanticdatalakeknowledgegraphmetadataannotationlargelanguagemodelshuman-in-the-loopmedicalharmonizationconstrainedgenerationmulti-labelclassification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that medical data lakes, which store heterogeneous data without a fixed schema, can be kept out of the 'data swamp' with a dual hybrid architecture. The idea is to let rigid, rule-built metadata indexing coexist with a flexible knowledge graph, then have a language model annotate each dataset with the machine-learning operations it is suitable for, under strict constraints and with a human checking the output. If true, schema-less collections of medical datasets and models become automatically queryable by ML applicability, without manual tagging. The paper also claims that among seven tested models, Gemma 3:4B gives the best combination of recall and speed, and was therefore chosen for deployment.

What carries the argument

H2-TGM, the Triple Generation Mechanism, builds the knowledge graph in two passes. First, rule-based generation converts required metadata fields such as id, owner, category, and tags into RDF triples with fixed predicates. Second, an LLM-assisted generator takes the title, description, and existing tags and outputs one to three task or technology tags from a constrained target list; those tags become triples using the 'isSuitableFor' relation for datasets and the 'usesTechStack' relation for models. The companion H2-MMS, the Metadata Modelling Schema, stores the raw metadata in a document-oriented database while the graph provides the schema-on-read layer. A simulated human-in-the-loop LLM sees both the original tags and the generator output and keeps only matching tags, giving the 'soft' metrics that the paper reports.

What would settle it

Rerun the same experiments after removing every occurrence of target-list terms from the title, description, and tags of all three metadata collections; if Gemma 3:4B's recall drops toward chance or another model surpasses it, the claimed annotation reliability is an artifact of lexical leakage.

Watch

Extended reading notes

Core claim

The central claim is that a dual hybrid design, a document database for rigid metadata plus a knowledge graph built from RDF triples, can harmonize heterogeneous medical data, and that the graph's extension through LLM-generated tags is reliable enough to support ML-readiness queries. The paper frames the annotation as constrained multi-label classification: the LLM must pick from a fixed target list of operation tags, which prevents hallucinated labels, and the pipeline adds a simulated human-in-the-loop LLM that filters proposed tags. Evaluated on three metadata collections drawn from public data-science hubs, including 500 medical datasets, 500 cancer datasets, and 140 AI models, the recall of the best small model, Gemma 3:4B, reaches 0.788, 0.801, and 0.572 respectively. The paper concludes that this model is the best trade-off of recall, speed, and resource use across these datasets, with the exception of the model-technology collection where Llama 3.1:8B leads.

Load-bearing premise

The evaluation assumes that removing the ground-truth words from the tags field means the LLM can no longer see them, but dataset titles and descriptions frequently contain those same target words, so the reported recall could reflect word copying rather than semantic annotation.

Editorial extensions

If this is right

Schema-less medical datasets can be indexed into a knowledge graph whose edges state which ML operations each dataset is suitable for, without manual curation.
A small 4-billion-parameter model can match or exceed models an order of magnitude larger on metadata annotation while running faster and on less hardware.
Constraining the LLM to a fixed target list prevents hallucinated labels, so the generated triples stay within the set of known ML techniques.
The human-in-the-loop gate, simulated here by a second LLM, can discard mismatched tags before they become permanent graph triples.
The same pipeline applies to model entries via the usesTechStack relation, so both datasets and trained models can share one harmonized metadata layer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, a direct test would strip target words from every metadata field rather than only the tags field; if recall collapses, the reported ranking measures lexical overlap, not semantic annotation.
The constrained-tag pattern should transfer to other controlled vocabularies, such as data-quality grades, privacy classes, or imaging modalities, because the architecture only needs a fixed target list and a verification gate.
Substituting a panel of human annotators for the simulated HIL-LLM would reveal how much of the soft recall comes from genuine verification rather than from a second LLM that shares the first model's blind spots.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. The paper is a competent engineering write-up of a medical-data-lake architecture, and its seven-LLM comparison is new. But the headline result—that Gemma3:4B is the best annotator—is not supported by the evaluation as designed. The recall figures are inflated by lexical leakage, and the soft metrics are close to circular.

What is genuinely there: the H2-MMS/H2-TGM split is a clean way to separate rigid document metadata from a flexible KG layer. Constrained prompting with a fixed target list and output rules is a sensible way to get structured tags out of an LLM. The rule-based triple generation is standard, and the paper says so. The comparison across three model tiers and three Kaggle collections produces measurements that I have not seen in the cited literature. The authors also disclose the proof-of-concept nature and the HIL simulation clearly, which counts for something.

The problems are in the evaluation. Section III-A1 says ground-truth words were filtered from the tags field only; titles and descriptions were passed verbatim. Listing 1 shows "Binary Classification Prediction for type of Breast Cancer" in the description, so a model can reach high recall by copying task words from free text rather than by semantic annotation. The HIL-LLM in Figure 3 receives the full unfiltered tag list, and Equations 8–9 build the extended ground truth by unioning the HIL approvals with the ground truth, then scoring the generator against that same union. The soft metrics therefore measure how often the HIL-LLM approves the generator, not how often a human would. There is no keyword-matching baseline, which would be the natural sanity check. The datasets are small (500/500/140 Kaggle metadata entries), and the GitHub link has no commit hash, so the experiments cannot be reproduced or even verified from the paper. Table 4 also rows llama3-70B under "Low," a copy-paste slip that suggests rushed table production.

Proportionately: the architecture is plausible and the writing is clear. The flaw is in the measurement, not in the concept. A re-run with leakage-free test sets, a simple baseline, and real human validation of the HIL simulation would make the empirical claim credible.

Who gets value: engineers building metadata annotation pipelines for data lakes, and anyone teaching evaluation pitfalls in LLM-as-annotator work. I would not cite the recall numbers, but I would send it to review—the architecture and the comparison are worth referee time, and the evaluation problems are fixable.

Referee Report

5 major / 6 minor

Summary. The paper proposes H2, a hybrid semantic data lake architecture for medical metadata that combines a MongoDB-based metadata schema with an RDF knowledge graph, rule-based triple generation, and LLM-assisted operation-oriented tag annotation. The constrained-generation problem is framed as multi-label classification, and seven LLMs from three families are evaluated on three Kaggle-derived collections (two dataset collections and one model collection) in terms of recall, precision, 'soft' metrics from a simulated LLM-based human-in-the-loop reviewer, and generation speed. The paper concludes that Gemma 3:4B offers the best recall/speed trade-off and is therefore selected for deployment. The architectural ideas and the constrained-generation prompting scheme are coherent, but the quantitative evidence for model selection and for the claimed human-in-the-loop verification is undermined by lexical leakage, circular 'soft' metrics, and the absence of non-LLM baselines.

Significance. The paper addresses a real problem—automated metadata annotation for medical data lakes—and has several strengths: the dual MongoDB/KG design is sensible, the constrained output format is a reasonable way to reduce hallucination, the comparison spans multiple model families and size tiers, and the authors provide a public GitHub repository for reproducibility. If the evaluation were sound, the result would support the modest but useful claim that a small open-weight LLM can annotate operation-oriented metadata with acceptable recall at low latency. As it stands, however, the headline empirical claims are not yet supported: the reported recall values may largely reflect lexical copying from titles and descriptions, and the 'soft' metrics are constructed so that the simulated reviewer's approvals become true positives by definition. I therefore view the contribution as a promising system paper whose central quantitative evaluation needs substantial rework before the conclusions can be accepted.

major comments (5)

[Section III-A1, Fig. 2, Tables 2-4] The reported recall in Tables 2-4 is likely inflated by lexical leakage. Section III-A1 states that 'tags given to the generator were filtered so as to be absent from the target list,' but Fig. 2 shows that the generator prompt still receives the title and description verbatim. Kaggle metadata frequently contains the target terms in those free-text fields; for example, Listing 1 has description 'Binary Classification Prediction for type of Breast Cancer' while the original tags include classification and binary_classification. After filtering, the phrase 'Binary Classification' still appears in the description, so a model that copies task words from the free text can achieve high recall without performing semantic annotation. Because model selection, including the choice of Gemma3:4B, is driven by these recall values, the central empirical claim is not yet supported. Please re-run the evaluation with target terms also removed from titles and descriptions (or with a held-out target list) and include a deterministic substring or keyword-matching baseline for comparison.
[Section III-A2, Eqs. (8)-(9)] The 'soft' metrics are circular by construction. The HIL-LLM prompt (Fig. 3) supplies the full original tag list, including the ground-truth labels, and Eq. (8) defines the extended ground truth as Y_ext_j = Y_hat^a_j union Y_j. Consequently, any proposed tag approved by the auxiliary LLM becomes a true positive in Eq. (9), regardless of whether a human would actually accept it. The high soft-recall values (e.g., 0.907 for gemma3-4b in Table 2) therefore measure the rate at which the same model family approves its own generator's outputs, not verification quality. Please evaluate the HIL-LLM against real human accept/reject decisions on a sample of cases, or remove the soft metrics from the headline claims.
[Section V-B] There is no non-LLM baseline in the experimental evaluation. The recall values in Tables 2-4 are only interpretable relative to a simple extractive method; a TF-IDF, BM25, or substring-match tagger would likely reproduce a large fraction of the reported recall given the leakage described above. Please add such a baseline and report the increment that each LLM provides over it, since the paper's core claim is that LLM-based annotation adds value over basic keyword matching.
[Section III-A2] The assumption that the auxiliary LLM simulates a human-in-the-loop is unvalidated. No evidence is provided that the HIL-LLM's accept/reject decisions agree with human annotators on this task. Since the integrity and verification claims of the architecture rest on this assumption, please provide a human-agreement experiment on a subset of the data, or temper the claim to 'LLM-assisted review' rather than human-in-the-loop verification.
[Section IV-A vs. Fig. 2] There is a discrepancy in the output cardinality constraint. Section IV-A states that the predicted set satisfies |Y_hat_j| <= 6, while the generator prompt in Fig. 2 instructs the model to output 1-3 tags. Precision and recall are sensitive to this constraint, so the exact value used in the experiments must be specified and the prompt and methodology must be aligned.

minor comments (6)

[Table 4] The row for llama3-70B is labeled 'Low' although it is the high-tier model; this is inconsistent with the tier definitions in Section V-A and should be corrected.
[Fig. 8 and Listings 1-2] The caption of Fig. 8 describes orange edges as signifying the 'isRelatedTo' relation, but Listings 1-2 and the surrounding text describe LLM-generated ML-task edges as 'isSuitableFor'. Please align the terminology.
[Tables 2-4] The definition of 'Mean LLM time' is missing; please specify whether it is end-to-end latency per entry, the number of runs used to compute the mean and standard deviation, and the hardware or API environment.
[Section V-A] The full target list L is only described with examples; please include the complete list (or a pointer to the repository file) so that the filtering procedure and the potential for target leakage can be audited.
[Fig. 1] Figure 1 contains placeholder 'Lorem ipsum' text under several collection headings; this should be replaced with actual example labels.
[Throughout] There are several typos and grammatical issues (e.g., 'constists' in Section II-B, 'avalable' in Section V-A, 'lead' in Section I); please run a careful proofreading pass.

Circularity Check

1 steps flagged · score 6.0 of 10

Soft recall and soft precision are circular by construction: the HIL-LLM's own approvals define the extended ground truth and are then counted as true positives, so the high soft scores in Tables 2-4 are forced rather than independent evidence.

self definitional [Section IV-B, Evaluation Metrics, Equations 8-9]
"It outputs a list of accepted tags, which is then concatenated with the ground-truth tags to construct an extended ground-truth list. Formally, let ˆY a j ⊆ ˆYj be the set of proposed tags approved by the HIL-LLM and let Y ext represent the extended ground-truth list. The ’soft’ metrics are then calculated as shown in Equations 6, 7 by using the new TPs total in Equations 8 - 9. Y ext j = ˆY a j ∪ Y j (8) TPs total = Σ |Y ext j ∩ ˆYj| (9)"

By Equation 8, the extended ground truth Y_ext includes every HIL-LLM-approved proposed tag, and by Equation 9 the soft true-positive count is |Y_ext ∩ Ŷ|. Since Ŷa_j ⊆ Ŷ_j by definition, every approved proposed tag is counted as a true positive by construction. Soft recall and soft precision therefore measure agreement between the generator and the HIL-LLM, not agreement with an independent ground truth; the HIL-LLM simultaneously defines the scoring target and is scored against it. Section III-A2 confirms the HIL-LLM receives the full unfiltered tag list, so its approvals can be string-identity matches, making the high soft recall and soft precision values in Tables 2-4 partly forced by the evaluation design rather than independent semantic verification.

full rationale

The main hard recall metric is not circular: ground truth is defined as Y_j = T_j ∩ L from original dataset tags (Section IV-A), independently of the LLM's output, and the reported recall scores in Tables 2-4 are genuine measurements of that definition. The central model-selection claim (Gemma 3:4B as best recall/speed trade-off) therefore has independent content, although it may be affected by a separate lexical-leakage validity issue: Section III-A1 removes target-list words only from the tags field while titles and descriptions still contain them, so a keyword-copying model could achieve high recall without semantic annotation. That is an experimental validity concern, not a circularity step. The clear circularity is in the soft metrics: Equations 8-9 define the extended ground truth using the HIL-LLM's own approved set, guaranteeing that every approved proposed tag is a true positive. The paper explicitly expects these metrics to be high ('both soft recall and soft precision are expected to be high'), and the HIL-LLM is prompted with the unfiltered tag list, making its approvals potentially lexical matches. No load-bearing self-citations or imported uniqueness theorems were found; the model citations [53], [54] are external technical reports. Overall, one claimed validation contribution reduces by construction while the headline recall comparison retains independent but leakage-affected content, giving partial circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no free parameters fitted to data. Its central evaluation rests on several unvalidated domain assumptions: the reliability of Kaggle tags as ground truth, the fidelity of an LLM as a stand-in for a human reviewer, and the absence of answer leakage through titles and descriptions. The H2-MMS and H2-TGM modules are software components, not newly postulated physical or ontological entities.

assumptions (4)

domain assumption Kaggle user-provided tags are a valid ground truth for the ML-suitability of datasets and models.
Section V-A uses the Kaggle tags after intersection with the target list as ground truth, with no independent validation of their correctness or completeness.
ad hoc to paper The auxiliary HIL-LLM behaves like a real human annotator in accepting or rejecting proposed tags.
Section III-A2 and Section IV-B use the simulated HIL to define soft metrics and to support the integrity claim, but no human study is reported to validate this proxy.
domain assumption Filtering target-list words from the tags field prevents the LLM from seeing the answer in titles or descriptions.
Section III-A1 describes filtering only the tags field; examples in the paper show descriptions containing the target terms, so this assumption is likely false for many entries.
domain assumption The LLM outputs always parse as valid Python lists matching the target list.
Section III-A1 imposes output rules, but no parsing failure rate or manual correction count is reported.

how reviews work

0 comments

Cite this review

Pith. "Pith review of H2: A Dual Hybrid Semantic Data Lake Architecture for Medical Data Harmonization with Human-In-the-Loop verified, LLM Driven Metadata Annotation System." pith.science (2026). https://pith.science/paper/KXZ5VM5E

@misc{pith2026260808056,
  author       = {Pith},
  title        = {Pith review of: H2: A Dual Hybrid Semantic Data Lake Architecture for Medical Data Harmonization with Human-In-the-Loop verified, LLM Driven Metadata Annotation System},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KXZ5VM5E}},
  note         = {Machine review of arXiv:2608.08056}
}

read the original abstract

Medical data, by its nature, exhibit a high degree of heterogeneity on multiple levels ranging from (a) different modalities like images, text and time series, (b) diverse tabular schemata introduced by institutions and (c) completely unstructured textual information data provided by healthcare professionals. Data lakes are often used in medical data storage to consolidate all heterogeneous diverse data in a single, central location, where it can be saved "as is", without the need to impose a schema like a data warehouse does. Despite their flexibility, though, data lakes are notorious for the "data swamp" failure. Thus, providing a reliable data harmonization mechanism through metadata, without compromising integrity or flexibility, is a real challenge. To this end, knowledge graphs have attracted attention since they provide a dynamic way to depict relationships without a rigid schema-on-write approach. Additionally, another rigorous task relies on the interoperability of data: application of appropriate ML techniques on such a diverse nature of data is not an easy task, as a domain expert must decide the efficacy of a method to a specific data type or dataset. Metadata annotation can aid by tagging applicable operations, however this requires manual intervention, not to mention the plethora of existing datasets which lack such information. To tackle both challenges, in this paper, we propose a semantic data lake architecture that promotes data harmonization and incorporates a generative annotation process (i.e. LLMs) of non-labeled metadata collections to support the application of meaningful ML techniques. Building on top of this approach, we create a higher level of knowledge, identifying suitability of data with respect to applicable ML operations based on their data nature...

Figures

Figures reproduced from arXiv: 2608.08056 by the authors.

**Figure 2.** FIGURE 2 [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** FIGURE 3 [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: FIGURE 4 [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: FIGURE 5 [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 7.** Figure 7: FIGURE 7 [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

**Figure 6.** Figure 6: FIGURE 6 [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

**Figure 8.** Figure 8: FIGURE 8 [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]

**Figure 9.** Figure 9: FIGURE 9 [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

**Figure 10.** Figure 10: FIGURE 10 [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]

**Figure 11.** Figure 11: FIGURE 11 [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 50 canonical work pages

[1]

Knezevic Ivanovski, S

T. Knezevic Ivanovski, S. Honap, R. Matic, S. Markovic, and L. Peyrin- Biroulet, ‘‘Building a healthcare data warehouse: considerations, oppor- tunities, and challenges,’’Frontiers in Digital Health, vol. 7, p. 1691142, 2025

work page 2025
[2]

R. Hai, C. Koutras, C. Quix, and M. Jarke, ‘‘Data lakes: A survey of functions and systems,’’IEEE Transactions on Knowledge and Data En- gineering, vol. 35, no. 12, pp. 12 571–12 590, 2023

work page 2023
[3]

Guntupalli, ‘‘Data lake vs

B. Guntupalli, ‘‘Data lake vs. data warehouse: Choosing the right architec- ture,’’International Journal of Artificial Intelligence, Data Science, and Machine Learning, vol. 4, no. 4, pp. 54–64, 2023

work page 2023
[4]

Karami, A

M. Karami, A. Rahimi, and A. H. Shahmirzadi, ‘‘Clinical data warehouse: an effective tool to create intelligence in disease management,’’The health care manager, vol. 36, no. 4, pp. 380–384, 2017

work page 2017
[5]

Sebaa, F

A. Sebaa, F. Chikh, A. Nouicer, and A. Tari, ‘‘Medical big data ware- house: architecture and system design, a case study: improving healthcare resources distribution,’’Journal of medical systems, vol. 42, no. 4, p. 59, 2018

work page 2018
[6]

S. R. Sukumar, R. Natarajan, and R. K. Ferrell, ‘‘Quality of big data in health care,’’International journal of health care quality assurance, vol. 28, no. 6, pp. 621–634, 2015

work page 2015
[7]

P . C. Sukhwal, V . Rajan, and A. Kankanhalli, ‘‘A joint llm-kg system for disease q&a,’’IEEE Journal of Biomedical and Health Informatics, vol. 29, no. 3, pp. 2257–2270, 2024

work page 2024
[8]

S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, and Z. Ives, ‘‘Dbpedia: A nucleus for a web of open data,’’ ininternational semantic web conference. Springer, 2007, pp. 722–735

work page 2007

Show all 54 references

[9]

F. M. Suchanek, G. Kasneci, and G. Weikum, ‘‘Y ago: A large ontology from wikipedia and wordnet,’’Journal of Web Semantics, vol. 6, no. 3, pp. 203–217, 2008

2008
[10]

Bollacker, C

K. Bollacker, C. Evans, P . Paritosh, T. Sturge, and J. Taylor, ‘‘Freebase: a collaboratively created graph database for structuring human knowledge,’’ inProceedings of the 2008 ACM SIGMOD international conference on Management of data, 2008, pp. 1247–1250

2008
[11]

Vrandečić and M

D. Vrandečić and M. Krötzsch, ‘‘Wikidata: a free collaborative knowledge- base,’’Communications of the ACM, vol. 57, no. 10, pp. 78–85, 2014

2014
[12]

G. A. Miller, ‘‘Wordnet: a lexical database for english,’’Communications of the ACM, vol. 38, no. 11, pp. 39–41, 1995

1995
[13]

Navigli and S

R. Navigli and S. P . Ponzetto, ‘‘Babelnet: The automatic construction, eval- uation and application of a wide-coverage multilingual semantic network,’’ Artificial intelligence, vol. 193, pp. 217–250, 2012

2012
[14]

Mitchell, W

T. Mitchell, W. Cohen, E. Hruschka, P . Talukdar, B. Y ang, J. Betteridge, A. Carlson, B. Dalvi, M. Gardner, B. Kisielet al., ‘‘Never-ending learning,’’ Communications of the ACM, vol. 61, no. 5, pp. 103–115, 2018

2018
[15]

X. Dong, E. Gabrilovich, G. Heitz, W. Horn, N. Lao, K. Murphy, T. Strohmann, S. Sun, and W. Zhang, ‘‘Knowledge vault: A web-scale approach to probabilistic knowledge fusion,’’ inProceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining,...

2014
[16]

Speer, J

R. Speer, J. Chin, and C. Havasi, ‘‘Conceptnet 5.5: An open multilingual graph of general knowledge,’’ inProceedings of the AAAI conference on artificial intelligence, vol. 31, no. 1, 2017

2017
[17]

J. D. Hwang, C. Bhagavatula, R. Le Bras, J. Da, K. Sakaguchi, A. Bosselut, and Y . Choi, ‘‘(comet-) atomic 2020: On symbolic and neural common- sense knowledge graphs,’’ inProceedings of the AAAI conference on artificial intelligence, vol. 35, no. 7, 2021, pp. 6384–6392

2020
[18]

S. Auer, J. Lehmann, and S. Hellmann, ‘‘Linkedgeodata: Adding a spatial dimension to the web of data,’’ inInternational Semantic Web Conference. Springer, 2009, pp. 731–746

2009
[19]

Dsouza, N

A. Dsouza, N. Tempelmeier, S. Gottschalk, R. Y u, and E. Demidova, ‘‘Worldkg: World-scale completion of geographic information,’’ inV olun- teered Geographic Information: Interpretation, Visualization and Social Context. Springer, 2023, pp. 3–19

2023
[20]

Bodenreider, ‘‘The unified medical language system (umls): integrating biomedical terminology,’’Nucleic acids research, vol

O. Bodenreider, ‘‘The unified medical language system (umls): integrating biomedical terminology,’’Nucleic acids research, vol. 32, no. suppl_1, pp. D267–D270, 2004

2004
[21]

L. Li, P . Wang, J. Y an, Y . Wang, S. Li, J. Jiang, Z. Sun, B. Tang, T.- H. Chang, S. Wanget al., ‘‘Real-world data medical knowledge graph: construction and applications,’’Artificial intelligence in medicine, vol. 103, p. 101817, 2020

2020
[22]

Kertkeidkachorn, R

N. Kertkeidkachorn, R. Nararatwong, Z. Xu, and R. Ichise, ‘‘Finkg: a core financial knowledge graph for financial analysis,’’ in2023 IEEE 17th International Conference on Semantic Computing (ICSC). IEEE, 2023, pp. 90–93

2023
[23]

F.-L. Li, H. Chen, G. Xu, T. Qiu, F. Ji, J. Zhang, and H. Chen, ‘‘Alimekg: Domain knowledge graph construction and application in e-commerce,’’ in Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 2020, pp. 2581–2588

2020
[24]

L. D. Dang, U. T. Phan, and N. T. Nguyen, ‘‘Gena: A knowledge graph for nutrition and mental health,’’Journal of Biomedical Informatics, vol. 145, p. 104460, 2023

2023
[25]

Y ang, H

P . Y ang, H. Wang, Y . Huang, S. Y ang, Y . Zhang, L. Huang, Y . Zhang, G. Wang, S. Y ang, L. Heet al., ‘‘Lmkg: A large-scale and multi- source medical knowledge graph for intelligent medicine applications,’’ Knowledge-Based Systems, vol. 284, p. 111323, 2024

2024
[26]

M. Wang, H. Wang, G. Qi, and Q. Zheng, ‘‘Richpedia: a large-scale, com- prehensive multi-modal knowledge graph,’’Big Data Research, vol. 22, p. 100159, 2020

2020
[27]

D. Li, Y . Liang, S. Qian, H. Wu, W. Jia, Y . Fu, and Y . Sun, ‘‘A review of background, methods, limitations and opportunities of knowledge graph completion,’’Computer Science Review, vol. 58, p. 100809, 2025

2025
[28]

C. Peng, F. Xia, M. Naseriparsa, and F. Osborne, ‘‘Knowledge graphs: Op- portunities and challenges,’’Artificial Intelligence Review, vol. 56, no. 11, pp. 13 071–13 102, 2023

2023
[29]

L. Cai, Y . Kang, C. Y u, Y . Fu, H. Zhang, and Y . Zhao, ‘‘Bringing two worlds together: The convergence of large language models and knowledge graphs,’’ in2024 3rd International Conference on Automation, Robotics and Computer Engineering (ICARCE). IEEE, 2024, pp. 207–216

2024
[30]

Y . Zhu, X. Wang, J. Chen, S. Qiao, Y . Ou, Y . Y ao, S. Deng, H. Chen, and N. Zhang, ‘‘Llms for knowledge graph construction and reasoning: Recent capabilities and future opportunities,’’World Wide Web, vol. 27, no. 5, p. 58, 2024

2024
[31]

S. Pan, L. Luo, Y . Wang, C. Chen, J. Wang, and X. Wu, ‘‘Unifying large language models and knowledge graphs: A roadmap,’’IEEE Transactions on Knowledge and Data Engineering, vol. 36, no. 7, pp. 3580–3599, 2024

2024
[32]

Azzabi, Z

S. Azzabi, Z. Alfughi, and A. Ouda, ‘‘Data lakes: A survey of concepts and architectures,’’Computers, vol. 13, no. 7, p. 163, 2024

2024
[33]

V aswani, N

A. V aswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, ‘‘Attention is all you need,’’Advances in neural information processing systems, vol. 30, 2017

2017
[34]

Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, ‘‘Roberta: A robustly optimized bert pretraining approach,’’arXiv preprint arXiv:1907.11692, 2019

1907 arXiv
[35]

S. Wang, X. Sun, X. Li, R. Ouyang, F. Wu, T. Zhang, J. Li, G. Wang, and C. Guo, ‘‘Gpt-ner: Named entity recognition via large language models,’’ inFindings of the association for computational linguistics: NAACL 2025, 2025, pp. 4257–4275

2025
[36]

L. Sun, P . Zhang, F. Gao, Y . An, Z. Li, and Y . Zhao, ‘‘Sf-gpt: A training- free method to enhance capabilities for knowledge graph construction in llms,’’Neurocomputing, vol. 613, p. 128726, 2025

2025
[37]

Y . Chen, Q. Fan, X. Y uan, Q. Zhang, and Y . Dong, ‘‘Pgd-gp: A chinese named entity recognition model for constructing food safety standard knowledge graph,’’IEEE Transactions on Multimedia, vol. 27, pp. 2836– 2847, 2025

2025
[38]

F. Y an, P . Y u, and X. Chen, ‘‘Ltner: Large language model tagging for named entity recognition with contextualized entity marking,’’ inInterna- tional Conference on Pattern Recognition. Springer, 2024, pp. 399–411

2024
[39]

Zaratiana, N

U. Zaratiana, N. Tomeh, P . Holat, and T. Charnois, ‘‘Gliner: Generalist model for named entity recognition using bidirectional transformer,’’ in Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language T...

2024
[40]

J. Lu, Z. Y ang, Y . Wang, X. Liu, B. Mac Namee, and C. Huang, ‘‘Padellm-ner: Parallel decoding in large language models for named entity recognition,’’ inAdvances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, an...

2024
[41]

Zhang, Z

H. Zhang, Z. Wei, G. Liu, R. Wang, R. Mu, C. Liu, A. Y uan, G. Cao, and N. Hu, ‘‘Mkeah: Multimodal knowledge extraction and accumulation based on hyperplane embedding for knowledge-based visual question an- swering,’’Virtual Reality & Intelligent Hardware, vol. 6, no. 4, pp. 2...

2024
[42]

M. Li, A. Zareian, Y . Lin, X. Pan, S. Whitehead, B. Chen, B. Wu, H. Ji, S.-F. Chang, C. V osset al., ‘‘Gaia: A fine-grained multimedia knowledge extraction system,’’ inProceedings of the 58th annual meeting of the 14 VOLUME 11, 2023 Tzortziset al.:H 2: A Dual Hybrid semantic ...

2023
[43]

S. Hao, B. Tan, K. Tang, B. Ni, X. Shao, H. Zhang, E. Xing, and Z. Hu, ‘‘Bertnet: Harvesting knowledge graphs with arbitrary relations from pre- trained language models,’’ inFindings of the association for computational linguistics: ACL 2023, 2023, pp. 5000–5015

2023
[44]

W. Zhou, K. Huang, T. Ma, and J. Huang, ‘‘Document-level relation extraction with adaptive thresholding and localized context pooling,’’ in Proceedings of the AAAI conference on artificial intelligence, vol. 35, no. 16, 2021, pp. 14 612–14 620

2021
[45]

Y . Ma, A. Wang, and N. Okazaki, ‘‘Dreeam: Guiding attention with evi- dence for improving document-level relation extraction,’’ inProceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, 2023, pp. 1971–1983

2023
[46]

X. Li, K. Chen, Y . Long, and M. Zhang, ‘‘Llm with relation classifier for document-level relation extraction,’’IEEE Transactions on Big Data, 2026

2026
[47]

Zhong, X

H. Zhong, X. Wei, and H. Zhang, ‘‘Leveraging llm for enhancing document-level relation extraction with correction and completion,’’ in 2025 8th International Conference on Advanced Algorithms and Control Engineering (ICAACE). IEEE, 2025, pp. 2364–2369

2025
[48]

S. Broscheit, ‘‘Investigating entity knowledge in bert with simple neural end-to-end entity linking,’’ inProceedings of the 23rd conference on computational natural language learning (CoNLL), 2019, pp. 677–685

2019
[49]

Ayoola, S

T. Ayoola, S. Tyagi, J. Fisher, C. Christodoulopoulos, and A. Pierleoni, ‘‘Refined: An efficient zero-shot-capable approach to end-to-end entity linking,’’ inProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Huma...

2022
[50]

De Cao, G

N. De Cao, G. Izacard, S. Riedel, and F. Petroni, ‘‘Autoregressive entity retrieval,’’arXiv preprint arXiv:2010.00904, 2020

2010 arXiv
[51]

De Cao, L

N. De Cao, L. Wu, K. Popat, M. Artetxe, N. Goyal, M. Plekhanov, L. Zettle- moyer, N. Cancedda, S. Riedel, and F. Petroni, ‘‘Multilingual autoregres- sive entity linking,’’Transactions of the Association for Computational Linguistics, vol. 10, pp. 274–290, 2022

2022
[52]

Q. Liu, Y . He, T. Xu, D. Lian, C. Liu, Z. Zheng, and E. Chen, ‘‘Unimel: A unified framework for multimodal entity linking with large language models,’’ inProceedings of the 33rd ACM International Conference on Information and Knowledge Management, 2024, pp. 1909–1919

2024
[53]

G. D. Gemma Team, ‘‘Gemma 3 technical report,’’ https://arxiv.org/abs/2503.19786, 2025

2025 arXiv
[54]

Grattafiori, A

A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. V aughanet al., ‘‘The llama 3 herd of models,’’arXiv preprint arXiv:2407.21783, 2024. IOANNIS N. TZORTZISreceived his Diploma in Electronics and Computer Engineering f...

1996 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Knezevic Ivanovski, S

T. Knezevic Ivanovski, S. Honap, R. Matic, S. Markovic, and L. Peyrin- Biroulet, ‘‘Building a healthcare data warehouse: considerations, oppor- tunities, and challenges,’’Frontiers in Digital Health, vol. 7, p. 1691142, 2025

work page 2025

[2] [2]

R. Hai, C. Koutras, C. Quix, and M. Jarke, ‘‘Data lakes: A survey of functions and systems,’’IEEE Transactions on Knowledge and Data En- gineering, vol. 35, no. 12, pp. 12 571–12 590, 2023

work page 2023

[3] [3]

Guntupalli, ‘‘Data lake vs

B. Guntupalli, ‘‘Data lake vs. data warehouse: Choosing the right architec- ture,’’International Journal of Artificial Intelligence, Data Science, and Machine Learning, vol. 4, no. 4, pp. 54–64, 2023

work page 2023

[4] [4]

Karami, A

M. Karami, A. Rahimi, and A. H. Shahmirzadi, ‘‘Clinical data warehouse: an effective tool to create intelligence in disease management,’’The health care manager, vol. 36, no. 4, pp. 380–384, 2017

work page 2017

[5] [5]

Sebaa, F

A. Sebaa, F. Chikh, A. Nouicer, and A. Tari, ‘‘Medical big data ware- house: architecture and system design, a case study: improving healthcare resources distribution,’’Journal of medical systems, vol. 42, no. 4, p. 59, 2018

work page 2018

[6] [6]

S. R. Sukumar, R. Natarajan, and R. K. Ferrell, ‘‘Quality of big data in health care,’’International journal of health care quality assurance, vol. 28, no. 6, pp. 621–634, 2015

work page 2015

[7] [7]

P . C. Sukhwal, V . Rajan, and A. Kankanhalli, ‘‘A joint llm-kg system for disease q&a,’’IEEE Journal of Biomedical and Health Informatics, vol. 29, no. 3, pp. 2257–2270, 2024

work page 2024

[8] [8]

S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, and Z. Ives, ‘‘Dbpedia: A nucleus for a web of open data,’’ ininternational semantic web conference. Springer, 2007, pp. 722–735

work page 2007