Pith. sign in

REVIEW 4 major objections 6 minor 76 references

PatTree: a novel approach for automated creation of multimodal, graph-based patient representations for medical classification tasks

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read PatTree converts raw, unharmonized clinical records into a per-patient tree that a graph attention network classifies directly, reaching 98.5% balanced accuracy in a three-way Alzheimer's task.

desk verdict Novel automated patient-graph pipeline whose headline 98.5% is a test-set max over 72 configurations rather than a valid state-of-the-art estimate, but the core idea deserves serious referee engagement. read the letter →

arxiv 2608.02692 v1 pith:VCZVKCMV submitted 2026-08-03 cs.LG

classification cs.LG
keywords multimodaldatapatientrepresentationknowledgegraphneuralnetworkmedicalclassificationAlzheimer'sdiseaseharmonizationattentionmechanism
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PatTree is proposed as a way to turn heterogeneous, multimodal clinical records into a per-patient graph automatically, without first mapping the data into a common model or running named-entity recognition. The central claim is that this representation can be used directly for classification: on 763 subjects with clinical assessments and structural MRI scans, a graph attention classifier achieved 98.5% balanced accuracy and an F1 score of 0.987 on a held-out test set for the three-way distinction between Alzheimer's disease, mild cognitive impairment, and cognitively normal individuals. That result beats the paper's own table-transformer baseline (88.7% balanced accuracy) and matches it in the easier binary setting. The practical significance, if the claim holds, is that the expensive and error-prone data harmonization step could be bypassed in clinical AI pipelines, with the natural structure of patient journeys and table schemas serving as the scaffolding for learning.

What carries the argument

The load-bearing object is PatTree itself: a per-patient, directed acyclic graph that is a tree once edge direction and labels are ignored, with the patient as root, visits/tables/measurements as internal nodes, and data values as leaves, all at depth at most four. The message-passing scheme is a customized graph attention network that activates one depth level at a time, updating nodes from depth four up to the root with attention-weighted sum aggregation, so the root vector after four layers summarizes every leaf. Node property vectors are built from pretrained sentence-transformer embeddings of names and category terms, combined with z-score-normalized values; image information enters either as unsupervised brain-region volumes or as supervised prototype-similarity scores. The only structural assumption is automated table-type inference from row counts, and the resulting tree natively represents missing values and multiple visits rather than requiring imputation or pooling.

What would settle it

A decisive check is to pre-register a single PatTree configuration before touching the test set, use only unsupervised image features such as brain-region volumes, and measure three-class balanced accuracy; if the result lands near the paper's own ROI-volume-only peak (92.2%) rather than 98.5%, the headline performance depends on the supervised prototype features or on selecting the best of 72 test-set-evaluated configurations.

Watch

Extended reading notes

Core claim

The paper's discovery claim is that an assumption-free, automated structuring of multimodal patient data can support state-of-the-art classification, and that the structuring itself does the work. PatTree is built from the only structural fact the method assumes: whether each data table's rows describe patients, visits, or measurements. From that, each patient becomes a depth-at-most-four tree whose root is the patient, whose leaves are data values, and whose internal nodes are visits, tables, and measurements; sentence-transformer embeddings of node names and category terms, together with normalized numeric values, give every node a vector. A four-layer graph attention network passes messages from leaves to root and classifies from the root vector, achieving the reported test-set scores. The intended consequence is that early integration of all available data, including images, can be done at scale without harmonization.

Load-bearing premise

The headline three-class score is the best result among 72 configurations evaluated on the held-out test set, and the winning image features are similarities to prototypes learned by a supervised model trained on the same training labels; if that peak reporting or the label-informed features is what produces the high score, the state-of-the-art claim collapses.

Editorial extensions

If this is right

  • In the three-class AD/MCI/CN task, direct classification on PatTree reaches 98.5% balanced accuracy and 0.987 F1 on the held-out test set, compared with 88.7% and 0.868 for the table-transformer baseline.
  • In the binary AD-versus-CN task, PatTree and the table baseline both reach perfect performance, so the more informative comparison is the three-class case.
  • PatTree handles missing values and multiple visits natively, whereas the baseline requires mean-pooling across visits and imputation of missing entries.
  • Including image features improves PatTree's three-class performance over no-image features, while adding image features to the baseline reduces its performance.
  • Construction choices (sentence transformer, data-node vector strategy, structure-node vectors, image-feature type) behave like tunable hyperparameters rather than make-or-break decisions, since all examined options reached high performance in the three-class setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the gap between the peak 98.5% configuration and the best ROI-volume-only configuration (92.2%) suggests that the supervised prototype-similarity features, not the tree structure alone, may carry much of the three-class signal; swapping in an unsupervised image encoder while freezing everything else would isolate that contribution.
  • Editorial inference: because only the best test-set result among 72 configurations is reported, a realistic expectation for a practitioner is lower than the headline; reporting the median or full distribution across configurations would give a truer estimate of typical PatTree performance.
  • Editorial inference: the depth-staged attention coefficients at event, table, and measurement nodes could be read out as a built-in explanation mechanism, showing which branches of the patient tree drove a classification without additional interpretability machinery.
  • Editorial inference: PatTree's automation still depends on two implicit inputs, a human-readable textual description for every retained feature and a sentence transformer that embeds those descriptions, so 'assumption-free' refers to data-structure assumptions rather than to zero human curation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes PatTree, a graph-based patient representation constructed automatically from heterogeneous clinical data tables without prior data harmonization, using event/table/measurement structure and sentence-transformer embeddings of node names and category terms. Patient classification is performed by a customized GATv2 message-passing model that aggregates leaf-to-root information into the central patient node. The method is evaluated on ADNI-1 (n=763) for binary and three-class AD/MCI/CN classification, reporting a peak balanced accuracy of 98.5% and F1 of 0.987 in the three-class task on the held-out test set, which the authors describe as state-of-the-art. The paper also presents ablation studies over sentence transformers, node-vector construction strategies, structure-node vectors, and image-feature types, plus runtime analyses.

Significance. If the reported results were properly supported, PatTree would be a valuable contribution: it offers a fully automated pipeline from raw clinical tables to a semantically structured patient graph, bypassing resource-intensive harmonization, and it demonstrates that GNN-based classification can operate directly on such representations. The method description is detailed and the ablation design is systematic; the runtime analysis is a useful practical addition. The authors also explicitly acknowledge the potential leakage of the supervised prototype-similarity features, which is commendable. However, the current experimental protocol invalidates the headline classification claim, so the study's main evidence for the method's utility is not established.

major comments (4)
  1. [§4.2 and §4.5] The test set is used as a model-selection criterion, contradicting the claim that it was held out from model selection. Section 4.2 states that the authors 'training and testing 144 PatTree-based GNN models' and 'restricted our analysis to the respective peak performance overall and in individual ablation studies,' while Section 4.5 asserts the held-out test set 'was not included in any training or model selection process.' Taking the maximum over 72 configurations evaluated on the same test set is model selection on that test set. The reported 98.5% balanced accuracy is therefore an extreme-order statistic, not a valid estimate of PatTree's generalization performance. The abstract's and conclusion's 'state-of-the-art' claims rest on this invalid estimate. The authors should either select configurations on the validation set and then evaluate the selected model once on the test set, or report the full distribution of results across all 72 configurations.
  2. [§5.3.4 and §4.3.1] The winning three-class configuration uses prototype-similarity image features extracted by PIPNet3D, which is fine-tuned on the ADNI training labels for the same classification task. Because the feature extractor is supervised by the target labels, the comparison against the no-image (96.5%) and ROI-volume (92.2%) ablations is confounded: the 98.5% result may largely reflect the discriminative power of the label-trained feature extractor rather than the PatTree representation. The caveat in Section 5.3.4 that this 'could in principle leak class information' is insufficient. A concrete remedy is to evaluate PatTree with a feature extractor trained without task labels (e.g., self-supervised or unsupervised radiomics) and to perform a nested split in which the feature extractor is trained only on the training fold and the configuration is selected on validation.
  3. [§4.5 and §5.2.2] No repeated runs or confidence intervals are reported; all 72 GNNs are single runs with fixed hyperparameters. With 150 test patients, the difference between the peak 98.5% and the 96.5% no-image peak is only a few patients, and the reported maximum could easily reflect run-to-run variation. The paper should report mean and standard deviation over multiple seeds for the selected configuration and for the baselines, and should use a statistically valid comparison (e.g., paired bootstrap or McNemar's test) before claiming superiority over the 88.7% baseline.
  4. [§11] The code and data availability statement is a placeholder: it says the source code 'is publicly available at: GitLab project PatTree' and that 'This link points to the exact commit,' but no URL, repository path, or commit hash is provided. Given the complexity of the pipeline (FastSurfer, PIPNet3D, three sentence transformers, custom GATv2), a working repository with the exact experimental configuration is essential for reproducibility and for verifying the reported peak performance.
minor comments (6)
  1. [§4.1] Typo: 'methodolgy' should be 'methodology.'
  2. [§4.5] Typo: 'held-out test test' should be 'held-out test set.'
  3. [§6] Grammatical error: 'Second, the we could aim for improved PatTree classification performance' should read 'Second, we could aim for improved PatTree classification performance.'
  4. [§10] Typo: 'accesing' should be 'accessing.'
  5. [Tables 12–15] The baseline 'peak performance' also appears to be selected across sentence transformers and image-feature sets on the test set; the paper should state explicitly how baseline configurations were chosen and whether the same test-set-selection concern applies to the baseline comparison.
  6. [§4.3.1] The description of PIPNet3D hyperparameters is vague ('phase- and ROI-specific hyperparameters determined by hyperparameter optimization and recommendations by Nauta et al.'); providing the actual values or a supplementary table would improve reproducibility.

Circularity Check

2 steps flagged · score 6.0 of 10

The reported 98.5% peak is the maximum over 72 configurations evaluated on the held-out test set and uses prototype-similarity image features trained on the same ADNI labels; the central SOTA claim therefore partly reduces to these selection and leakage choices.

  1. fitted input called prediction [Section 4.2 (Experimental Design) and Section 4.5 (PatTree Classification)]
    "Exhaustively testing all options and combinations of options across the four ablation studies in two classification settings (binary and three-class), we ended up training and testing 144 PatTree-based GNN models. Given the resulting scale, we restricted our analysis to the respective peak performance overall and in individual ablation studies. ... All trained models were assessed on both the training dataset, to monitor learning and detect overfitting, and the held-out test set, which was not included in any training or model selection process."

    The reported three-class test balanced accuracy of 98.5% is the maximum of the 72 configurations evaluated on that same held-out test set. Choosing the configuration with the best test performance is model selection on the test set, so the test set is not independent of the final model. The headline number is therefore an extreme-order statistic of the configuration search rather than a prediction of a fixed PatTree pipeline, directly contradicting the statement that the test set was 'not included in any ... model selection process'.

  2. fitted input called prediction [Section 3.1.1 (Image Feature Extraction), Section 4.3.1 (Image Feature Extraction), and Section 5.3.4 (Ablation Study 4)]
    "The second, supervised approach utilizes a CNN to learn prototypes, i.e. prototypical image patches from the training data, and determines similarities of MRI scans to these learned prototypes. ... In a pretraining step, the parameters of each ResNet were finetuned to the ADNI training data while the linear layer was frozen. ... One caveat is that PrototypeSimilarities are derived through supervised learning and could in principle leak class information; however, the baseline results on individual tables suggest this leakage, if present, is limited."

    The prototype-similarity features are produced by a CNN fine-tuned on the ADNI training labels, and the winning three-class configuration uses exactly these prototype similarities as image-feature node values (Section 5.2.2). The downstream GNN is then trained on the same AD/MCI/CN labels, so the input features already encode label information from the training set. The reported 98.5% peak is thus partly manufactured by a feature extractor fitted to the target labels, even though the paper acknowledges the leakage risk in the ablation caveat.

full rationale

The core PatTree construction itself is not circular: the tree is built from clinical table structure, node-name embeddings, and feature values, and the GATv2 message-passing derivation is self-contained. There is also independent evidence that the representation works without the contested components, since the three-class test peak drops to 96.5% with no image features and to 92.2% with unsupervised ROI volumes. However, the headline state-of-the-art claim is specifically attached to two non-independent choices: (1) the test set was used to select the best of 72 configurations, making the reported number a selected maximum rather than a valid held-out estimate, and (2) the winning configuration uses prototype-similarity features derived from a supervised model trained on the same class labels, which is a form of label leakage through the input representation. These two issues jointly mean the central '98.5% state-of-the-art' claim partially reduces to the model-selection and feature-construction protocol, warranting a score of 6 rather than a lower non-circularity score.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claim depends mainly on the table-type inference heuristic, the semantic embeddings, and the leak-free use of supervised image features. These are all domain assumptions rather than standard mathematical axioms; the leak-free assumption is the most delicate.

free parameters (7)
  • hidden_dimension = 32
    GNN hidden size chosen by hand; controls expressiveness.
  • learning_rate = 0.01
    Adam learning rate for GNN training.
  • epochs = 50
    Training duration, fixed across runs.
  • batch_size = 16
    Batch size for GNN training.
  • num_layers = 4
    Equal to tree depth; architecture design choice.
  • PIPNet3D_hyperparameters = not reported in detail
    Phase- and ROI-specific hyperparameters from HPO, not fully specified.
  • weighted_cross_entropy_weights = not specified
    Class imbalance weighting used in loss; values not given.
assumptions (5)
  • domain assumption Rows of each clinical data table can be classified as patient-, visit-, or measurement-centered based on row count characteristics (Table 1).
    The only assumption in PatTree construction (Section 3.2); if false, graph structure is incorrect.
  • domain assumption Clinical data tables across sources share the property that each row represents exactly one patient, one visit/event, or one measurement instance.
    Needed for automatic table type inference and tree construction (Section 3.2).
  • domain assumption Sentence transformer embeddings of node names and category terms preserve enough semantic information for the downstream GNN classification.
    Node property vectors are based on these embeddings (Sections 3.1.2, 4.4.2).
  • standard math z-score normalization computed on the training set can be applied to validation and test sets.
    Standard preprocessing assumption; if distribution shifts, features are distorted.
  • ad hoc to paper Supervised prototype-similarity features trained on the training labels do not leak label information into the test evaluation.
    The winning configuration uses these features; Section 5.3.4 acknowledges the risk but assumes leakage is limited.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PatTree: a novel approach for automated creation of multimodal, graph-based patient representations for medical classification tasks." pith.science (2026). https://pith.science/paper/VCZVKCMV

@misc{pith2026260802692,
  author       = {Pith},
  title        = {Pith review of: PatTree: a novel approach for automated creation of multimodal, graph-based patient representations for medical classification tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VCZVKCMV}},
  note         = {Machine review of arXiv:2608.02692}
}
abstract

Access to holistic, multimodal data improves the performance of Artificial Intelligence (AI) in medical classification tasks compared to utilizing single modalities or data sources. However, the inherent heterogeneity and complexity of clinical real-world data pose significant challenges to structured data analysis and AI application. This heterogeneity includes missing values, multiple time points, diverse modalities, and inconsistent formats and semantics. Data harmonization prior to data integration tackles this challenge but remains resource-intensive and error-prone, limiting the scalability and reproducibility of holistic, AI-driven decision support on clinical real-world data. We therefore propose PatTree, a graph-based, holistic representation of patients that can be derived from real-world clinical data through the automated structuring of multimodal clinical data. PatTree enables early-stage data integration without relying on pre-standardized inputs. While representing heterogeneous clinical data within a unified knowledge graph, PatTree preserves the semantic relationships between data elements across modalities and data sources, facilitating interoperability and machine-interpretable data access. Using a subset of the ADNI-1 cohort (n = 763), we demonstrate that classification of patients is directly feasible on PatTree reaching state-of-the-art classification performance. In the three-class classification task distinguishing Alzheimer's disease, mild cognitive impairment, and cognitively normal individuals, we achieve a balanced accuracy of 98.5% and an F$_1$ score of 0.987 on the held-out test set. Our results show that assumption-free, automated structuring of multimodal medical data can serve as a scalable foundation for clinical AI pipelines bypassing tedious data preparation and standardization.

Figures

Figures reproduced from arXiv: 2608.02692 by the authors.

Figure 1
Figure 1. Embedding of node names and category terms as 𝑑-dimensional embedding vectors using various sentence transformers. The embedding dimension 𝑑 is determined by the sentence transformer used. 3.2.2. Adding Data Nodes The second step of the PatTree construction process cre￾ates a data node for each value or extracted feature available. Based on the type of the table a data element stems from, we differentiate between pa… view at source ↗
Figure 2
Figure 2. KG Schema behind PatTree construction with node types as indicated in [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Three-stage GNN architecture applied to classify the PatTree patient representations as described in Section 3.3. a) Projection of initial node property vectors 𝑥𝑖𝑛𝑖𝑡 to hidden dimension 𝑑ℎ𝑖𝑑𝑑𝑒𝑛 (c.f. section 3.3.1). b) GATv2-based message passing from leaf nodes to root in 4 GNN layers (c.f. sections 3.3.2, 3.3.3, and 3.3.4). c) Linear classification layer applied on final root node property vector for graph-level … view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: KG Schema behind PatTree construction from [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Processes applied for creating initial node property vectors from heterogeneous data features and metadata as described in Section 4.4.2. a) Process for creation of initial node property vectors using strategy "pre_emb" for combination of node names and data values. b)…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

76 extracted references · 47 canonical work pages

  1. [1]

    What prevents us from reusing medical real-world data in research.Sci

    JuliaGehrmann,EditHerczog,StefanDecker,andOyaBeyan. What prevents us from reusing medical real-world data in research.Sci. Data, 10(1):459, 2023.doi:10.1038/s41597-023-02361-2

  2. [2]

    A new model for learning in graph domains

    Marco Gori, Gabriele Monfardini, and Franco Scarselli. A new model for learning in graph domains. InProceedings. 2005 IEEE international joint conference on neural networks, 2005., volume 2, pages 729–734. IEEE, 2005.doi:10.1109/IJCNN.2005.1555942

  3. [3]

    Graph neural networks in alzheimer’s disease diagnosis: a review of unimodalandmultimodaladvances.Front.inNeurosci.,19:1623141, 2025.doi:10.3389/fnins.2025.1623141

    Shahzad Ali, Michele Piana, Matteo Pardini, and Sara Garbarino. Graph neural networks in alzheimer’s disease diagnosis: a review of unimodalandmultimodaladvances.Front.inNeurosci.,19:1623141, 2025.doi:10.3389/fnins.2025.1623141

  4. [4]

    Fhir-ontop-omop: Building clinical knowledge graphs in fhir rdf with the omop common data model.J

    Guohui Xiao, Emily Pfaff, Eric Prud’hommeaux, David Booth, DeepakKSharma,NanHuo,YueYu,NansuZong,KathrynJRuddy, Christopher G Chute, et al. Fhir-ontop-omop: Building clinical knowledge graphs in fhir rdf with the omop common data model.J. Biomed. Inform., 134:104201, 2022.doi:10.1016/j.jbi.2022.104201

  5. [5]

    Medax:Aknowledgegraphonfhir

    Ilya Mazein, Tom Gebhardt, Felix Zinkewitz, Lea Michaelis, Sarah Braun, Dagmar Waltemath, Ron Henkel, and Judith AH Wodke. Medax:Aknowledgegraphonfhir. InDigitalHealthandInformatics Innovations for Sustainable Health Care Systems: Proceedings of MIE 2024, pages 367–371. SAGE Publications 1 Oliver’s Yard, 55 City Road, London, EC1Y 1SP, 2024.doi:10.3233/SHTI240423

  6. [6]

    InDigitalHealthandInformaticsInnovationsforSustainableHealth Care Systems, pages 1406–1410

    AchilleasChytas,NickBassileiades,andPantelisNatsiavas.Mapping omop-cdmtordf:bringingreal-world-datatothesemanticwebrealm. InDigitalHealthandInformaticsInnovationsforSustainableHealth Care Systems, pages 1406–1410. IOS Press, 2024.doi:10.3233/ SHTI240674

  7. [7]

    Evolution of a graph model for the omop common data model.Appl

    Mengjia Kang, Jose A Alvarado-Guzman, Luke V Rasmussen, and Justin B Starren. Evolution of a graph model for the omop common data model.Appl. Clin. Inform., 15(05):1056–1065, 2024.doi: 10.1055/s-0044-1791487. J. Gehrmann et al.:Preprint submitted to ElsevierPage 19 of 22 PatTree

  8. [8]

    Connecting elec- tronichealthrecordstoabiomedicalknowledgegraphtolinkclinical phenotypes and molecular endotypes in atopic dermatitis.Sci

    Francesca Frau, Paul Loustalot, Margaux Törnqvist, Nina Temam, Jean Cupe, Martin Montmerle, and Franck Augé. Connecting elec- tronichealthrecordstoabiomedicalknowledgegraphtolinkclinical phenotypes and molecular endotypes in atopic dermatitis.Sci. Rep., 15(1):3082, 2025.doi:10.1038/s41598-024-78794-5

Show all 76 references
  1. [9]

    Using knowledge graph struc- tures for semantic interoperability in electronic health records data exchanges.Inf., 13(2):52, 2022.doi:10.3390/info13020052

    Shelly Sachdeva and Subhash Bhalla. Using knowledge graph struc- tures for semantic interoperability in electronic health records data exchanges.Inf., 13(2):52, 2022.doi:10.3390/info13020052

  2. [10]

    The challenges in data integration–heterogeneity and complexity in clinical trials and patient registries of systemic lupus erythematosus.BMC Med

    Helen Le Sueur, Ian N Bruce, Nophar Geifman, and Masterplans Consortium. The challenges in data integration–heterogeneity and complexity in clinical trials and patient registries of systemic lupus erythematosus.BMC Med. Res. Methodol., 20(1):164, 2020.doi: 10.1186/s12874-020-01057-0

  3. [11]

    Nandita Goyal and Navdeep Singh. Named entity recognition and relationship extraction for biomedical text: A comprehensive survey, recent advancements, and future research directions.Neurocomput- ing, 618:129171, 2025.doi:10.1016/j.neucom.2024.129171

  4. [12]

    Validation of a common data model for active safety surveillance research.J

    J Marc Overhage, Patrick B Ryan, Christian G Reich, Abraham G Hartzema, and Paul E Stang. Validation of a common data model for active safety surveillance research.J. Am. Med. Inform. Assoc., 19(1):54–60, 2012.doi:10.1136/amiajnl-2011-000376

  5. [13]

    Observationalhealth datasciencesandinformatics(ohdsi):opportunitiesforobservational researchers.Stud

    George Hripcsak, Jon D Duke, Nigam H Shah, Christian G Reich, Vojtech Huser, Martijn J Schuemie, Marc A Suchard, Rae Woong Park,IanChiKeiWong,PeterRRijnbeek,etal. Observationalhealth datasciencesandinformatics(ohdsi):opportunitiesforobservational researchers.Stud. Health Techn...

  6. [14]

    Web services fordatawarehouses:Omopandpcornetoni2b2.J.Am.Med.Inform

    Jeffrey G Klann, Lori C Phillips, Christopher Herrick, Matthew AH Joss, Kavishwar B Wagholikar, and Shawn N Murphy. Web services fordatawarehouses:Omopandpcornetoni2b2.J.Am.Med.Inform. Assoc., 25(10):1331–1338, 2018.doi:10.1093/jamia/ocy093

  7. [15]

    Muhammad Ayaz, Muhammad F Pasha, Mohammed Y Alzahrani, Rahmat Budiarto, and Deris Stiawan. The fast health interoperability resources(fhir)standard:systematicliteraturereviewofimplementa- tions,applications,challengesandopportunities.JMIRMed.Inform., 9(7):e21929, 2021.doi:10.2...

  8. [16]

    Common data models and data standards for tabular health data: a systematic review.BMC Med

    Melissa Finster, Markus Wenzel, and Elham Taghizadeh. Common data models and data standards for tabular health data: a systematic review.BMC Med. Inform. Decis. Mak., 25(1):422, 2025.doi: 10.1186/s12911-025-03267-2

  9. [17]

    A scoping review on multimodal deep learning in biomedical images and texts.J

    Zhaoyi Sun, Mingquan Lin, Qingqing Zhu, Qianqian Xie, Fei Wang, Zhiyong Lu, and Yifan Peng. A scoping review on multimodal deep learning in biomedical images and texts.J. Biomed. Inform., 146:104482, 2023.doi:10.1016/j.jbi.2023.104482

  10. [18]

    Gram: graph-based attention model for healthcare representation learning

    EdwardChoi,MohammadTahaBahadori,LeSong,WalterFStewart, and Jimeng Sun. Gram: graph-based attention model for healthcare representation learning. InProceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 787–795, 2017.doi:10.114...

  11. [19]

    Learningthegraphicalstructure of electronic health records with graph convolutional transformer

    Edward Choi, Zhen Xu, Yujia Li, Michael Dusenberry, Gerardo Flores,EmilyXue,andAndrewDai. Learningthegraphicalstructure of electronic health records with graph convolutional transformer. InProceedings of the AAAI conference on artificial intelligence, volume 34, pages 606–613,...

  12. [20]

    Comorbidity-based framework for alzheimer’s disease classification using graph neural networks.Sci

    FerialAbuhantash,MohdKhalilAbuHantash,andAamnaAlShehhi. Comorbidity-based framework for alzheimer’s disease classification using graph neural networks.Sci. Rep., 14(1):21061, 2024.doi: 10.1038/s41598-024-72321-2

  13. [21]

    A scoping review of self-supervised representation learning for clini- cal decision making using ehr categorical data.NPJ Digit

    Yuanyuan Zheng, Adel Bensahla, Mina Bjelogrlic, Jamil Zaghir, Hugues Turbe, Lydie Bednarczyk, Christophe Gaudet-Blavignac, Julien Ehrsam, Stéphane Marchand-Maillet, and Christian Lovis. A scoping review of self-supervised representation learning for clini- cal decision making ...

  14. [22]

    Medicalbigdata:promiseand challenges.Kidney Res

    ChoongHoLeeandHyung-JinYoon. Medicalbigdata:promiseand challenges.Kidney Res. Clin. Pract., 36(1):3, 2017.doi:10.23876/j. krcp.2017.36.1.3

  15. [23]

    An overview of deep learning methods for multimodal medical data mining.Expert Syst

    Fatemeh Behrad and Mohammad Saniee Abadeh. An overview of deep learning methods for multimodal medical data mining.Expert Syst. Appl., 200:117006, 2022.doi:10.1016/j.eswa.2022.117006

  16. [24]

    Multimodal machine learning in precision health: A scoping review

    Adrienne Kline, Hanyin Wang, Yikuan Li, Saya Dennis, Meghan Hutch, Zhenxing Xu, Fei Wang, Feixiong Cheng, and Yuan Luo. Multimodal machine learning in precision health: A scoping review. NPJ Digit. Med., 5(1):171, 2022.doi:10.1038/s41746-022-00712-8

  17. [25]

    A cross-disciplinary comparison of multimodal data fusion approaches and applications: Acceleratinglearningthroughtrans-disciplinaryinformationsharing

    RohitBokade,AlfredNavato,RuilinOuyang,XiaoningJin,Chun-An Chou, Sarah Ostadabbas, and Amy V Mueller. A cross-disciplinary comparison of multimodal data fusion approaches and applications: Acceleratinglearningthroughtrans-disciplinaryinformationsharing. Expert Syst. Appl., 165:...

  18. [26]

    Harnessing multimodal data integration to advance precision oncology.Nat

    KevinMBoehm,PegahKhosravi,RamiVanguri,JianjiongGao,and Sohrab P Shah. Harnessing multimodal data integration to advance precision oncology.Nat. Rev. Cancer, 22(2):114–126, 2022.doi: 10.1038/s41568-021-00408-3

  19. [27]

    Image feature extraction tech- niques:Acomprehensivereview.FranklinOpen,page100366,2025

    Sudhakar Hallur and Anil Gavade. Image feature extraction tech- niques:Acomprehensivereview.FranklinOpen,page100366,2025. doi:10.1016/j.fraope.2025.100366

  20. [28]

    A comparative study of statistical, radiomics, and deep learning feature extraction techniques for medical image classification in optical and radiological modalities.Comput

    Pegah Dehbozorgi, Oleg Ryabchykov, and Thomas W Bocklitz. A comparative study of statistical, radiomics, and deep learning feature extraction techniques for medical image classification in optical and radiological modalities.Comput. Biol. Med., 187:109768, 2025. doi:10.1016/j....

  21. [29]

    The deep learning revolution in neuroimaging: Insights from a bibliometric analysis (2014–2024).Neuroinformatics, 24(2):16, 2026.doi:10.1007/ s12021-026-09775-4

    Jyotismita Chaki and Gopikrishna Deshpande. The deep learning revolution in neuroimaging: Insights from a bibliometric analysis (2014–2024).Neuroinformatics, 24(2):16, 2026.doi:10.1007/ s12021-026-09775-4

  22. [30]

    Radiomicsandartificialintelligenceforthediagnosisandmonitoring of alzheimer’s disease: a systematic review of studies in the field.J

    RobertaBevilacqua,FedericoBarbarossa,LorenzoFantechi,Daniela Fornarelli, Enrico Paci, Silvia Bolognini, Cinzia Giammarchi, Fab- rizia Lattanzio, Lucia Paciaroni, Giovanni Renato Riccardi, et al. Radiomicsandartificialintelligenceforthediagnosisandmonitoring of alzheimer’s dise...

  23. [31]

    Introduction to radiomics.J

    Marius E Mayerhoefer, Andrzej Materka, Georg Langs, Ida Häg- gström,PiotrSzczypiński,PeterGibbs,andGaryCook. Introduction to radiomics.J. Nucl. Med., 61(4):488–495, 2020.doi:10.2967/ jnumed.118.222893

  24. [32]

    A comparative study of one-hot, tf-idf, and word2vec for classifying illegal advertising texts

    Yuedan Zhang, Lingmin He, Yunpeng Zhang, Panzhi Zhao, Bole Zhang, and Fang Cheng. A comparative study of one-hot, tf-idf, and word2vec for classifying illegal advertising texts. InProceed- ings of the 2024 8th International Conference on Natural Language Processing and Informa...

  25. [33]

    Efficient estimation of word representations in vector space.arXiv preprint arXiv:1301.3781, 2013.doi:10.48550/arXiv.1301.3781

    TomasMikolov,KaiChen,GregCorrado,andJeffreyDean. Efficient estimation of word representations in vector space.arXiv preprint arXiv:1301.3781, 2013.doi:10.48550/arXiv.1301.3781

  26. [34]

    At- tention is all you need.Adv

    AshishVaswani,NoamShazeer,NikiParmar,JakobUszkoreit,Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. At- tention is all you need.Adv. Neural Inf. Process. Syst., 30, 2017. doi:10.48550/arXiv.1706.03762

  27. [35]

    Transformer models in natural language processing: A comprehensive review and prospects for future development

    Maha Salem, Azza Mohamed, and Khaled Shaalan. Transformer models in natural language processing: A comprehensive review and prospects for future development. InInternational Conference on Advanced Intelligent Systems and Informatics, pages 463–472. Springer, 2025.doi:10.1007/9...

  28. [36]

    Aidan Hogan, Claudio Gutierrez, Michael Cochez, Gerard De Melo, Sabrina Kirrane, Axel Polleres, Roberto Navigli, Axel- Cyrille Ngonga Ngomo, Sabbir M. Rashid, Lukas Schmelzeisen, Steffen Staab, Eva Blomqvist, Claudia d’Amato, José Emilio Labra Gayo, Sebastian Neumaier, Anisa R...

  29. [37]

    The scalable precision J

    John H Morris, Karthik Soman, Rabia E Akbas, Xiaoyuan Zhou, Brett Smith, Elaine C Meng, Conrad C Huang, Gabriel Cerono, Gundolf Schenk, Angela Rizk-Jackson, et al. The scalable precision J. Gehrmann et al.:Preprint submitted to ElsevierPage 20 of 22 PatTree medicineopenknowled...

  30. [38]

    Building a knowledge graph to enable precision medicine.Sci

    Payal Chandak, Kexin Huang, and Marinka Zitnik. Building a knowledge graph to enable precision medicine.Sci. Data, 10(1):67, 2023.doi:10.1038/s41597-023-01960-3

  31. [39]

    Capturingsemanticrelationshipsinelectronichealthrecords using knowledge graphs: An implementation using mimic iii dataset and graphdb

    Bader Aldughayfiq, Farzeen Ashfaq, NZ Jhanjhi, and Mamoona Hu- mayun. Capturingsemanticrelationshipsinelectronichealthrecords using knowledge graphs: An implementation using mimic iii dataset and graphdb. InHealthcare, volume 11, page 1762. MDPI, 2023. doi:10.3390/healthcare11121762

  32. [40]

    Demo- graphic aware probabilistic medical knowledge graph embeddings of electronic medical records

    Aynur Guluzade, Endri Kacupaj, and Maria Maleshkova. Demo- graphic aware probabilistic medical knowledge graph embeddings of electronic medical records. InInternational Conference on Artificial Intelligence in Medicine, pages 408–417. Springer, 2021.doi:10. 1007/978-3-030-77211-6_48

  33. [41]

    Towards a personal health knowledge graph framework for patient monitoring.arXiv preprint arXiv:2311.06524, 2023.doi:10.48550/arXiv.2311.06524

    Daniel Bloor, Nnamdi Ugwuoke, David Taylor, Keir Lewis, Luis Mur, and Chuan Lu. Towards a personal health knowledge graph framework for patient monitoring.arXiv preprint arXiv:2311.06524, 2023.doi:10.48550/arXiv.2311.06524

  34. [42]

    Personal health knowledge graph for clinically relevant diet recommendations.arXiv preprint arXiv:2110.10131, 2021.doi:10.48550/arXiv.2110.10131

    Oshani Seneviratne, Jonathan Harris, Ching-Hua Chen, and Debo- rah L McGuinness. Personal health knowledge graph for clinically relevant diet recommendations.arXiv preprint arXiv:2110.10131, 2021.doi:10.48550/arXiv.2110.10131

  35. [43]

    Real-world data medical knowledge graph: construction and applications.Artif

    Linfeng Li, Peng Wang, Jun Yan, Yao Wang, Simin Li, Jinpeng Jiang, Zhe Sun, Buzhou Tang, Tsung-Hui Chang, Shenghui Wang, et al. Real-world data medical knowledge graph: construction and applications.Artif. Intell. Med., 103:101817, 2020.doi:10.1016/j. artmed.2020.101817

  36. [44]

    Medaka: Construction of biomedical knowledge graphs using large language models.arXiv preprint arXiv:2509.26128, 2025.doi:10.48550/arXiv.2509.26128

    Asmita Sengupta, David Antony Selby, Sebastian Josef Vollmer, and Gerrit Großmann. Medaka: Construction of biomedical knowledge graphs using large language models.arXiv preprint arXiv:2509.26128, 2025.doi:10.48550/arXiv.2509.26128

  37. [45]

    Hao Yang, Jiaxi Li, Chi Zhang, Alejandro Pazos Sierra, and Bairong Shen. Large language model–driven knowledge graph construction in sepsis care using multicenter clinical databases: Development and usabilitystudy.J.Med.InternetRes.,27:e65537,2025.doi:10.2196/ 65537

  38. [46]

    Early multimodal data integration for data-driven medical research–a scoping review.Ger

    Julia Gehrmann and Oya Beyan. Early multimodal data integration for data-driven medical research–a scoping review.Ger. Med. Data Sci. 2024, pages 49–58, 2024.doi:10.3233/SHTI240837

  39. [47]

    word2vec, node2vec, graph2vec, x2vec: Towards a theory of vector embeddings of structured data

    Martin Grohe. word2vec, node2vec, graph2vec, x2vec: Towards a theory of vector embeddings of structured data. Inproceedings of the 39th ACM SIGMOD-SIGACT-SIGAI symposium on principles of database systems, pages 1–16, 2020.doi:10.1145/3375395.3387641

  40. [48]

    Graph neural networks: Graph classification

    Christopher Morris. Graph neural networks: Graph classification. In Graph Neural Networks: Foundations, Frontiers, and Applications, pages 179–193. Springer, 2022.doi:10.1007/978-981-16-6054-2_9

  41. [49]

    Graph attention networks

    Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio, et al. Graph attention networks. InInternational conference on learning representations, volume 6. Ithaca, 2018.doi:10.17863/CAM.48429

  42. [50]

    How attentive are graph attention networks?arXiv preprint arXiv:2105.14491, 2021.doi: 10.48550/arXiv.2105.14491

    Shaked Brody, Uri Alon, and Eran Yahav. How attentive are graph attention networks?arXiv preprint arXiv:2105.14491, 2021.doi: 10.48550/arXiv.2105.14491

  43. [51]

    Deep learning for alzheimer’s disease: advances in classification, segmentation, subtyping, and explainability.Biomed

    MohammedRizwanShaikh,AndrewJeyabose,andRVijayaArjunan. Deep learning for alzheimer’s disease: advances in classification, segmentation, subtyping, and explainability.Biomed. Eng. Online, 24(1):150, 2025.doi:10.1186/s12938-025-01482-6

  44. [52]

    Convolutional neural networks for alzheimer’s disease detection on mri images.J

    Amir Ebrahimi, Suhuai Luo, and for the Alzheimer’s Disease Neu- roimaging Initiative. Convolutional neural networks for alzheimer’s disease detection on mri images.J. Med. Imaging, 8(2):024503– 024503, 2021.doi:10.1117/1.JMI.8.2.024503

  45. [53]

    Con- volutional neural networks for classification of alzheimer’s disease: Overviewandreproducibleevaluation.Med.ImageAnal.,63:101694, 2020.doi:10.1016/j.media.2020.101694

    Junhao Wen, Elina Thibeau-Sutre, Mauricio Diaz-Melo, Jorge Samper-González, Alexandre Routier, Simona Bottani, Didier Dor- mont, Stanley Durrleman, Ninon Burgos, Olivier Colliot, et al. Con- volutional neural networks for classification of alzheimer’s disease: Overviewandrepro...

  46. [54]

    A deep learning model for early di- agnosis of alzheimer’s disease combined with 3d cnn and video swin transformer.Sci

    Juan Zhou, Yiming Wei, Xiong Li, Weiqiang Zhou, Ruiyang Tao, Yi Hua, and Hongwei Liu. A deep learning model for early di- agnosis of alzheimer’s disease combined with 3d cnn and video swin transformer.Sci. Rep., 15(1):23311, 2025.doi:10.1038/ s41598-025-05568-y

  47. [55]

    Alzformer: Multi-modal framework for alzheimer’sclassificationusingmriandgraph-embeddeddemograph- ics guided by adaptive attention gating.Comput

    Sayyed Shahid Hussain, Xu Degang, Pir Masoom Shah, Hikmat Khan, and Adnan Zeb. Alzformer: Multi-modal framework for alzheimer’sclassificationusingmriandgraph-embeddeddemograph- ics guided by adaptive attention gating.Comput. Med. Imaging Graph., page 102638, 2025.doi:10.1016/j...

  48. [56]

    AnovelmultimodalMRIanalysisforalzheimer’s disease based on convolutional neural network

    Yan Wang, Yanwu Yang, Xin Guo, Chenfei Ye, Na Gao, Yuan Fang, andHeatherT.Ma. AnovelmultimodalMRIanalysisforalzheimer’s disease based on convolutional neural network. In2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pa...

  49. [57]

    Shangran Qiu, Matthew I Miller, Prajakta S Joshi, Joyce C Lee, Chonghua Xue, Yunruo Ni, Yuwei Wang, Ileana De Anda-Duran, PhillipHHwang,JustinACramer,etal.Multimodaldeeplearningfor alzheimer’sdiseasedementiaassessment.Nat.Commun.,13(1):3404, 2022.doi:10.1038/s41467-022-31037-5

  50. [58]

    Inclusionofneuropsycho- logicalscoresinatrophymodelsimprovesdiagnosticclassificationof alzheimer’s disease and mild cognitive impairment.Comput

    Mohammed Goryawala, Qi Zhou, Warren Barker, David A Loewen- stein,RanjanDuara,andMalekAdjouadi. Inclusionofneuropsycho- logicalscoresinatrophymodelsimprovesdiagnosticclassificationof alzheimer’s disease and mild cognitive impairment.Comput. Intell. Neurosci., 2015(1):865265, 2...

  51. [59]

    Marthe Mieling, Mushfa Yousuf, and Nico Bunzeck. Predicting the progression of mci and alzheimer’s disease on structural brain integrity and other features with machine learning.GeroScience, 48(1):463–487, 2026.doi:10.1007/s11357-025-01626-5

  52. [60]

    Early diagnosis of alzheimer’s disease using machine learning: a multi-diagnostic, gen- eralizable approach.Alzheimer’s Res

    Vasco Sá Diogo, Hugo Alexandre Ferreira, Diana Prata, and Alzheimer’s Disease Neuroimaging Initiative. Early diagnosis of alzheimer’s disease using machine learning: a multi-diagnostic, gen- eralizable approach.Alzheimer’s Res. Ther., 14(1):107, 2022.doi: 10.1186/s13195-022-01047-y

  53. [61]

    Machine learning techniques for the diagnosis of alzheimer’s disease: A review.ACM Trans

    Muhammad Tanveer, Bharat Richhariya, Riyaj Uddin Khan, Ashraf Haroon Rashid, Pritee Khanna, Mukesh Prasad, and Chin-Teng Lin. Machine learning techniques for the diagnosis of alzheimer’s disease: A review.ACM Trans. Multimed. Comput. Commun. Appl., 16(1s):1–35, 2020.doi:10.114...

  54. [62]

    Machine learning of neuroimaging for assisted diagnosis of cognitiveimpairmentanddementia:asystematicreview.Alzheimer’s Dement.: Diagn

    Enrico Pellegrini, Lucia Ballerini, Maria del C Valdes Hernandez, FrancescaMChappell,VictorGonzález-Castro,DevasudaAnblagan, Samuel Danso, Susana Muñoz-Maniega, Dominic Job, Cyril Pernet, et al. Machine learning of neuroimaging for assisted diagnosis of cognitiveimpairmentandd...

  55. [63]

    Springer, 2002.doi:10.1007/978-3-030-81885-2

    GabrielValiente.Algorithmsontreesandgraphs,volume112,chap- ter 1.1, pages 3–19. Springer, 2002.doi:10.1007/978-3-030-81885-2

  56. [64]

    2005.06.003

    Susanne G Mueller, Michael W Weiner, Leon J Thal, Ronald C Petersen, Clifford R Jack, William Jagust, John Q Trojanowski, ArthurWToga,andLaurelBeckett.Waystowardanearlydiagnosisin alzheimer’s disease: the alzheimer’s disease neuroimaging initiative (adni).Alzheimer’s Dement., ...

  57. [65]

    Fastsurfer-a fast and accurate deep learning based neuroimaging pipeline.NeuroImage, 219:117012, 2020.doi:10.1016/j.neuroimage.2020.117012

    Leonie Henschel, Sailesh Conjeti, Santiago Estrada, Kersten Diers, Bruce Fischl, and Martin Reuter. Fastsurfer-a fast and accurate deep learning based neuroimaging pipeline.NeuroImage, 219:117012, 2020.doi:10.1016/j.neuroimage.2020.117012

  58. [66]

    Comparison of automated volume extraction with freesurfer and fastsurfer for early alzheimer’s disease detection with machine learning

    Louise Bloch and Christoph M Friedrich. Comparison of automated volume extraction with freesurfer and fastsurfer for early alzheimer’s disease detection with machine learning. In2021 IEEE 34th Inter- national Symposium on Computer-Based Medical Systems (CBMS), pages 113–118. I...

  59. [67]

    Lisa Anita De Santi, Jörg Schlötterer, Michael Scheschenja, Joel Wessendorf, Meike Nauta, Vincenzo Positano, and Christin Seifert. J. Gehrmann et al.:Preprint submitted to ElsevierPage 21 of 22 PatTree Pipnet3d: Interpretable detection of alzheimer in mri scans. InInter- natio...

  60. [68]

    Pip-net: Patch-based intuitive prototypes for interpretable image classification

    Meike Nauta, Jörg Schlötterer, Maurice Van Keulen, and Christin Seifert. Pip-net: Patch-based intuitive prototypes for interpretable image classification. InProceedings of the IEEE/CVF conference oncomputervisionandpatternrecognition,pages2744–2753,2023. doi:10.1109/CVPR52729....

  61. [69]

    Domain-specific language model pretraining for biomedical natural languageprocessing.ACMTrans.Comput.Healthc.,3(1):1–23,2021

    Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. Domain-specific language model pretraining for biomedical natural languageprocessing.ACMTrans.Comput.Healthc.,3(1):1–23,2021. doi:10.1145/3458754

  62. [70]

    BioLORD- 2023: semantic textual representations fusing large language models and clinical knowledge graph insights.J

    FrançoisRemy,KrisDemuynck,andThomasDemeester. BioLORD- 2023: semantic textual representations fusing large language models and clinical knowledge graph insights.J. Am. Med. Inform. Assoc., page ocae029, 02 2024.doi:10.1093/jamia/ocae029

  63. [71]

    Multilingual e5 text embeddings: A technical report.arXiv preprint arXiv:2402.05672, 2024.doi:10

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Multilingual e5 text embeddings: A technical report.arXiv preprint arXiv:2402.05672, 2024.doi:10. 48550/arXiv.2402.05672

  64. [72]

    A systematic analysis of per- formance measures for classification tasks.Inf

    Marina Sokolova and Guy Lapalme. A systematic analysis of per- formance measures for classification tasks.Inf. Process. Manag., 45(4):427–437, 2009.doi:10.1016/j.ipm.2009.03.002

  65. [73]

    Revisiting deep learning models for tabular data.Adv

    Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Revisiting deep learning models for tabular data.Adv. Neural Inf. Pocess. Syst., 34:18932–18943, 2021.doi:10.48550/ arXiv.2106.11959

  66. [74]

    Jiayuan Xu and Fumie Costen. Machine learning-based multiclass classification of cognitive stages using plasma biomarkers, clinical assessments,andgeneticfeatures:Arepeated,nestedcross-validation study in adni with external evaluation in cntn.Diagnostics, 16(12), 2026.doi:10.3...

  67. [75]

    A gen- eralizable 3d framework and model for self-supervised learning in medical imaging.NPJ Digit

    Tony Xu, Sepehr Hosseini, Chris Anderson, Anthony Rinaldi, Rahul G Krishnan, Anne L Martel, and Maged Goubran. A gen- eralizable 3d framework and model for self-supervised learning in medical imaging.NPJ Digit. Med., 8(1):639, 2025.doi:10.1038/ s41746-025-02035-w

  68. [76]

    ThomasBarba,BryceABagley,SandraSteyaert,FranciscoCarrillo- Perez, Christoph Sadée, Michael Iv, and Olivier Gevaert. Dune: a versatile neuroimaging encoder captures brain complexity across 3 major diseases: cancer, dementia, and schizophrenia.GigaScience, 14:giaf116, 2025.doi:1...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.