Pith. sign in

REVIEW 5 major objections 10 minor 46 references

Multimodal Contrastive Representation Learning in Augmented Biomedical Knowledge Graphs

T0 review · 5 major / 10 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Pretrained multimodal embeddings improve biomedical link prediction, including for unseen nodes, by combining language-model vectors with graph contrastive learning.

desk verdict PrimeKG++ is a genuinely useful resource and the pipeline is plausible, but the main comparison is compromised by an unaddressed GCL test-edge leakage risk and overclaimed 'unseen nodes' generality. read the letter →

arxiv 2501.01644 v2 pith:HNDV3B62 submitted 2025-01-03 cs.CL cs.LG

Tien Dang , Viet Thanh Duy Nguyen , Minh Tuan Le , Truong-Son Hy This is my paper · ORCID
classification cs.CLcs.LG
keywords biomedicalknowledgegraphslinkpredictiongraphcontrastivelearninglanguagemodelembeddingsmultimodalfusionPrimeKG++drug-targetinteractiondrugrepurposing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that initializing a biomedical link-prediction model with pretrained node embeddings, rather than random vectors or plain language-model vectors, makes prediction measurably better. The embeddings are built by taking modality-specific language-model representations of each entity (protein sequences, gene sequences, SMILES strings, and text descriptions), fusing them, and refining them with graph contrastive learning before supervised link prediction. If true, this gives a practical recipe: pretrain once on a rich graph like the new PrimeKG++ resource, then reuse the embeddings to get higher average precision and F1 on biomedical link prediction, including on entities not seen during supervised training. The paper also contributes PrimeKG++, an augmented version of PrimeKG with added sequences and descriptions for genes, proteins, and drugs.

What carries the argument

The load-bearing mechanism is a two-stage representation pipeline. First, each entity's modalities are encoded by frozen domain-specific language models—protein sequences, nucleotide sequences, SMILES strings, and text descriptions—and combined by an attention-based or relation-guided dual adaptive fusion module into one embedding per node. Second, graph contrastive learning, a self-supervised technique that maximizes agreement between two augmented views of the same graph, aligns embeddings of nodes of the same type; the paper instantiates this with DGI, GGD, and GRACE. The resulting vectors initialize a relational graph convolutional network whose DistMult scoring function is trained for link prediction. PrimeKG++ supplies the multimodal node attributes that make this pipeline possible.

What would settle it

Run the same GRACE/ReDAF and RGCN/DistMult pipeline with validation and test edges explicitly removed from the graph used for GCL pretraining; if PrimeKG++ AP and F1 fall to the direct-LM baseline levels, the reported gain is leakage rather than representation quality.

Watch

Extended reading notes

Core claim

The central claim is that the fusion of LM-derived semantic embeddings with graph contrastive learning produces node representations that outperform both random initialization and direct LM embeddings when used as input to an RGCN encoder with a DistMult decoder for link prediction. On PrimeKG++ the best configuration (GRACE with ReDAF) reaches 0.996 AP and 0.983 F1 at a 1:1 negative sampling ratio, and the gap over baselines widens at harder 1:3 and 1:5 ratios. On the DrugBank drug-target interaction dataset the same pretrained representations generalize to a different graph with novel nodes, again beating random initialization and generally matching or exceeding direct LM embeddings. The paper further claims that the combined semantic-plus-relational embeddings are accurate for unseen nodes, a property the authors attribute to graph contrastive learning capturing topology that standalone LM vectors lack.

Load-bearing premise

The evaluation assumes that the graph-contrastive pretraining stage never trains on the edges that later appear in validation and test splits, so those edges cannot leak into the pretrained node embeddings.

Editorial extensions

If this is right

  • Users of biomedical link prediction can replace random or LM-only initializations with these pretrained embeddings and expect higher AP and F1 at no change to the downstream RGCN/DistMult model.
  • The benefit is largest when negative sampling is hard at 1:3 and 1:5, where the direct-LM baseline degrades more sharply than the pretrained configurations.
  • The same pretrained embeddings transfer to a second dataset, DrugBank, whose nodes and relations differ from PrimeKG++, supporting the paper's claim of generalization to unseen nodes.
  • PrimeKG++ becomes a public resource for evaluating multimodal biomedical knowledge graph methods, since it adds sequences and text to PrimeKG's existing structure.
  • The framework is easily extended to other relation types such as drug-disease or protein-disease interactions, according to the authors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not state whether the graph-contrastive pretraining stage masks validation and test edges; if it does not, the pretrained embeddings could have seen test structure, so an explicit masking protocol is the first check on the reported gains.
  • The claim of generalization to unseen nodes rests on DrugBank sharing entity identities with PrimeKG++; a sharper test would remove all overlapping entities and measure whether the performance advantage survives.
  • Because the language models are frozen and the contrastive module adds only about 164K parameters, the recipe suggests that cheap self-supervised pretraining on a rich graph can substitute for training embeddings from scratch in low-resource biomedical settings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 10 minor

Summary. The manuscript proposes a multimodal pretraining framework for biomedical knowledge graph link prediction. For each entity type, modality-specific language models (ProtBERT, DNABERT, MolFormer, BioBERT) produce attribute embeddings; a fusion module (attention or ReDAF) combines them; graph contrastive learning (DGI, GGD, or GRACE) pretrains representations on within-type subgraphs; and an RGCN encoder with a DistMult decoder performs supervised link prediction. The authors introduce PrimeKG++, a version of PrimeKG enriched with biological sequences and textual descriptions, and evaluate the pipeline on PrimeKG++ and the DrugBank drug-target interaction dataset, comparing random initialization, direct LM-derived embeddings, and nine GCL/fusion configurations at negative sampling ratios of 1:1, 1:3, and 1:5.

Significance. If the experimental comparison is leakage-free, the paper offers a useful and reproducible recipe: code, pretrained models, and the PrimeKG++ dataset are promised, and the configuration study (three GCL objectives, two learned fusion modules, multiple negative ratios, per-relation precision, embedding-size ablation, and a downstream XGBoost probe) is more thorough than is typical for a short empirical paper. The main contribution is not a new architecture but a pretrained-initialization strategy, and the paper is honest that it is not targeting state-of-the-art link prediction. However, the central claim rests on an unstated procedural premise: that GCL pretraining does not see validation or test edges. The text does not state this, and Section 5.3 says that PrimeKG++ was used during GCL pretraining. The 'unseen nodes' claim is also not established by the random edge split, and the DrugBank summary overstates Table 3. These issues make the current evidence conditional.

major comments (5)
  1. [§4.4 and §5.1] Section 4.4 defines the GCL module on the graph adjacency A and describes only random node/edge masking as augmentation; Section 5.1 describes a 60/20/20 random split of edges for the supervised link prediction model but never states that the same split is applied to the GCL pretraining stage. Since DGI, GGD, and GRACE all consume the adjacency structure, the default implementation would use all PrimeKG++ edges, including those that later appear in validation and test. The pretrained embeddings z_i would then encode test-edge topology before RGCN/DistMult is trained, biasing Table 2 in favor of the proposed method and making the comparison with random and LM-derived initialization misleading. The authors must state explicitly whether GCL pretraining masks validation/test edges; if it does not, the experiments need to be rerun with GCL pretraining on the training-only subgraph, and the abstract and conclusion claims revised accordingly.
  2. [§5.1 and §5.2.2] Section 5.1 randomly splits edges into 60/20/20, so every node present in a test edge is also present in the training graph and in GCL pretraining; no node is held out. Section 5.2.2 nevertheless claims the DrugBank experiments evaluate 'handling unseen nodes.' With a random edge split, nodes in the test set are seen at both the GCL and KGE stages, and only individual edges are held out, so the claim is not supported. To substantiate generalizability to unseen nodes, the authors need either to define 'unseen' precisely or to run an inductive split in which all edges incident to held-out nodes are removed from training and to report performance separately for those held-out nodes.
  3. [Abstract, §5.2.2, Table 3] The abstract and Section 6 state that the approach consistently outperforms baseline methods on DrugBank, but Table 3 contradicts this. At 1:1 negative sampling, Direct LM-derived embeddings reach AP 0.994 and F1 0.957, while GGD with None, Attention, and ReDAF reach only AP 0.985, 0.9862, and 0.9865 with F1 0.948, 0.951, and 0.954, respectively, and all DGI configurations have lower AP and F1. The text in Section 5.2.2 acknowledges that some configurations do not surpass LM-derived embeddings, so the abstract and conclusion should be revised to identify GRACE as the configuration family that consistently matches or exceeds the LM baseline, or should report an aggregate result with a significance test.
  4. [Tables 2–3] All results in Tables 2 and 3 are reported as single point estimates, with no standard deviations, confidence intervals, number of seeds, or significance tests. The headline improvements at 1:1 are small, for example GRACE with ReDAF reaches AP 0.996 and F1 0.983 versus Direct LM AP 0.993 and F1 0.975, and configuration-to-configuration differences are even smaller. Without repeated runs, the claim of significant performance improvements is not established. Please report means and standard deviations over at least three random seeds, ideally with paired significance tests across configurations.
  5. [§2.3 and §5.2] Section 2.3 cites several contrastive learning methods for knowledge graphs, including KGCL, KE-GCL, and MCLEA, and the paper claims novelty in applying multimodal contrastive learning to biomedical knowledge graphs, but Section 5.2 evaluates only random and LM initialization baselines under a single RGCN-DistMult backbone. The contribution would be better contextualized by comparing against at least one existing contrastive knowledge graph representation learning method, or a strong graph-pretraining baseline, using the same downstream protocol, or by narrowing the novelty claim to the specific pretraining-initialization setting. This is a scope issue rather than a correctness error, but it affects how a reader can interpret the novel framing.
minor comments (10)
  1. [Table 1] The caption says 'DPI benchmark' while the text and Section 5.2.2 say 'DTI'; correct the acronym.
  2. [§4.3] The fused embedding is denoted u_i in the equation and then immediately written as u_i in R^D, while Table 5 defines h_i as the unified embedding; use one symbol consistently.
  3. [§4.5] The regularization term Lreg uses X for the encoder output and Z for relation embeddings, but Table 5 defines Z as the latent node representation; disambiguate the notation.
  4. [References] Reference [18] is cited for BioGPT, but the listed paper is a different one by Lewis et al.; please provide the correct BioGPT citation or revise the sentence.
  5. [References] References [14] and [15] both cite DrugBank 6.0; merge them or clearly distinguish the benchmark dataset from the knowledgebase.
  6. [Appendix C.2] The equation for omega_m uses V, tanh, and zeta_r, but the following sentence introduces X_i, S_i, W_t, W_r, and C that do not appear in the displayed equations; align the notation.
  7. [Appendix D.2] The method is introduced as GGD but is later referred to as GD; use GGD consistently.
  8. [Figure 3] The caption refers to left and right panels, but the figure as included does not show panel labels; add (a) and (b) labels.
  9. [Introduction] The Introduction says PrimeKG has more than 4 million relationships, while Table 1 reports 3,527,861 triples for PrimeKG++; clarify whether the difference is due to filtering, entity coverage, or counting of triples.
  10. [§5.2.1] The text says the most significant gains appear over baseline methods, but at 1:1 the gain over Direct LM is at most 0.003 in AP and 0.008 in F1; temper the wording or support it with significance evidence.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: the GCL-pretraining pipeline is an empirical comparison, not a derivation that reduces to its inputs; the only self-citation is background, and the main risk is a possible train/test leakage that is procedural rather than circular.

full rationale

The paper's central claim is empirical: LM+GCL pretrained embeddings, when used as initialization, improve link prediction relative to random or direct LM initialization while the downstream RGCN/DistMult model is held fixed (Tables 2 and 3). The contrastive step z_i = C(u_i) in Section 4.4 is a pretraining map on node attributes and graph augmented views, and the supervised objective in Section 4.5 is the separate BCE-plus-regularizer loss; neither equation defines the target link prediction in terms of the pretrained embeddings by construction. No fitted constant is later renamed as a prediction, and no uniqueness theorem or ansatz is imported from the authors' prior work. The only self-citation I found is reference [23] (Truong-Son Hy is an author), cited in the Introduction as general background for why link prediction matters; it is not load-bearing. The external DrugBank benchmark and the XGBoost downstream check provide independent evaluation surfaces. The most serious concern is procedural, not circular: Section 5.1 describes a 60/20/20 split of edges for the link prediction task but never states whether the GCL pretraining stage masks validation and test edges; if pretraining consumed the full PrimeKG++ adjacency, the Table 2 gains over LM baselines could reflect topology leakage. That would be an evaluation-validity flaw, not a reduction of the result to its own inputs, so it does not raise the circularity score above the low range.

Assumptions & free parameters 6 free parameters · 5 assumptions · 1 invented entities

The ledger counts one invented artifact (PrimeKG++), several hand-chosen hyperparameters, and one unstated procedural assumption about masking test edges during GCL pretraining. The method itself introduces no new physical or mathematical entities, but its empirical validity rests on public LM encoders, standard contrastive losses, and the assumption that the pretraining graph is not contaminated with test edges.

free parameters (6)
  • Embedding dimension after fusion = 128
    Table 4 fixes the fused embedding dimension at 128; Appendix E.1 reports that increasing to 256 improves AP from 0.994 to 0.996, so the chosen value is a tunable hyperparameter that affects the headline numbers.
  • GCN hidden dimension and depth = 128, 2 layers
    Table 4 sets hidden dimension 128 and two hidden layers; these control GCL encoder capacity.
  • Regularization weight (alpha) = 0.01
    Tables 4 and 5 set the weight on the squared-norm regularizer in the final loss; no sensitivity analysis is provided.
  • Dropout rate = 0.2
    Table 4; applied in the network and not varied.
  • Negative sampling ratio = 1:1, 1:3, 1:5
    Main results are reported at three ratios; metrics drop substantially at 1:5, so this choice materially affects the reported performance.
  • Random vector for missing modalities = not specified
    Appendix C.2 replaces missing attribute values with a random vector in the same space; distribution and scale are not given, yet many nodes may lack one or more modalities.
assumptions (5)
  • domain assumption Frozen pretrained language models (ProtBERT, DNABERT, MolFormer, BioBERT) produce informative embeddings for biomedical sequences and text.
    Section 4.2 relies on these encoders as the only semantic signal; if these embeddings are weak, the whole pipeline loses its advantage over random initialization.
  • standard math Contrastive learning between augmented views maximizes useful mutual information for nodes of the same type.
    Section 4.4 adopts DGI, GGD, and GRACE based on this premise; the paper does not prove the objective improves link prediction, it assumes the standard GCL theory transfers to BKGs.
  • domain assumption RGCN with DistMult is a strong enough link prediction backbone that the comparison isolates embedding quality.
    Section 4.5 uses only this KGE model; without baselines from ComplEx or RotatE, the reported improvements could be specific to RGCN/DistMult initialization behavior.
  • domain assumption Randomly sampled negative edges are true negatives.
    Section 4.5 samples negative edges uniformly; in biomedical graphs, some unobserved edges are unknown positives, which biases AP/F1 upward, though this is a standard practice.
  • ad hoc to paper GCL pretraining does not see validation/test edges.
    Section 5.1 describes the edge split for link prediction but never states the GCL pretraining stage masks those edges; all test metrics depend on this unstated procedural assumption.
invented entities (1)
  • PrimeKG++ multimodal knowledge graph independent evidence
    purpose: Adds amino acid sequences, nucleotide sequences, SMILES strings, and text descriptions to PrimeKG entities so that LM encoders can produce node attributes for link prediction.
    The authors state the dataset is publicly available at the GitHub repository, giving an external falsifiable handle; however, the paper provides no persistent version or DOI or commit hash, and the scraping-based construction (Appendix B) is hard to verify independently.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multimodal Contrastive Representation Learning in Augmented Biomedical Knowledge Graphs." pith.science (2026). https://pith.science/paper/HNDV3B62

@misc{pith2026250101644,
  author       = {Pith},
  title        = {Pith review of: Multimodal Contrastive Representation Learning in Augmented Biomedical Knowledge Graphs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HNDV3B62}},
  note         = {Machine review of arXiv:2501.01644}
}
read the original abstract

Biomedical Knowledge Graphs (BKGs) integrate diverse datasets to elucidate complex relationships within the biomedical field. Effective link prediction on these graphs can uncover valuable connections, such as potential novel drug-disease relations. We introduce a novel multimodal approach that unifies embeddings from specialized Language Models (LMs) with Graph Contrastive Learning (GCL) to enhance intra-entity relationships while employing a Knowledge Graph Embedding (KGE) model to capture inter-entity relationships for effective link prediction. To address limitations in existing BKGs, we present PrimeKG++, an enriched knowledge graph incorporating multimodal data, including biological sequences and textual descriptions for each entity type. By combining semantic and relational information in a unified representation, our approach demonstrates strong generalizability, enabling accurate link predictions even for unseen nodes. Experimental results on PrimeKG++ and the DrugBank drug-target interaction dataset demonstrate the effectiveness and robustness of our method across diverse biomedical datasets. Our source code, pre-trained models, and data are publicly available at https://github.com/HySonLab/BioMedKG

Figures

Figures reproduced from arXiv: 2501.01644 by the authors.

Figure 1
Figure 1. The subgraph illustrates the interactions surround￾ing the Parathyroid hormone receptor and its connections to related drugs and diseases. Different entity types are color￾coded: red nodes represent drugs, blue nodes indicate genes or proteins, and yellow nodes denote diseases. Black arrows depict drug-treatment relationships with diseases, while or￾ange arrows represent drug-receptor interactions. This sub￾graph is… view at source ↗
Figure 2
Figure 2. Overview of our proposed framework. A. Modality Embedding: Creating node attribute embeddings through domain-specific LMs. B. Contrastive Learning: Enhancement of LM-derived embeddings for specific node attributes of the same type through Fusion Module and Contrastive Learning. C. Link Prediction on KG Embedding: Utilizing the enhanced embeddings to perform link prediction tasks through a Knowledge Graph Embedding (… view at source ↗
Figure 3
Figure 3. t-SNE visualization of drug embeddings for a single protein with the highest number of interactions in the PrimeKG++ dataset. The left panel displays embeddings derived solely from the Language Model (LM), while the right panel shows embeddings generated using our proposed approach (GRACE + ReDAF). Drugs interacting with the selected protein are labeled in red, and non-interacting drugs are labeled in blue. This com… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 39 canonical work pages

  1. [1]

    ProteinBERT: a universal deep-learning model of protein se- quence and function

    Nadav Brandes, Dan Ofer, Yam Peleg, Nadav Rappoport, and Michal Linial. ProteinBERT: a universal deep-learning model of protein se- quence and function. Bioinformatics, 38(8):2102–2110, 02 2022

    work page 2022

  2. [2]

    Building a knowl- edge graph to enable precision medicine

    Payal Chandak, Kexin Huang, and Marinka Zitnik. Building a knowl- edge graph to enable precision medicine. Scientific Data, 10(1):67, 2023

    work page 2023

  3. [3]

    Large knowledge model: Perspectives and challenges, 2023

    Huajun Chen. Large knowledge model: Perspectives and challenges, 2023

    work page 2023

  4. [4]

    Bartsmiles: Generative masked language models for molecular representations, 2022

    Gayane Chilingaryan, Hovhannes Tamoyan, Ani Tevosyan, Nelly Babayan, Lusine Khondkaryan, Karen Hambardzumyan, Zaven Navoyan, Hrant Khachatrian, and Armen Aghajanyan. Bartsmiles: Generative masked language models for molecular representations, 2022

    work page 2022

  5. [5]

    de Almeida, Hassan Sirelkhatim, Guillaume Richard, Marcin Skwark, Karim Beguir, Marie Lopez, and Thomas Pierrot

    Hugo Dalla-Torre, Liam Gonzalez, Javier Mendoza-Revilla, Nico- las Lopez Carranza, Adam Henryk Grzywaczewski, Francesco Oteri, Christian Dallago, Evan Trop, Bernardo P. de Almeida, Hassan Sirelkhatim, Guillaume Richard, Marcin Skwark, Karim Beguir, Marie Lopez, and Thomas Pierrot. The nucleotide transformer: Building and evaluating robust foundation model...

    work page 2023

  6. [6]

    Bioblp: a modular framework for learning on multimodal biomedical knowledge graphs

    Daniel Daza, Dimitrios Alivanistos, Payal Mitra, Thom Pijnenburg, Michael Cochez, and Paul Groth. Bioblp: a modular framework for learning on multimodal biomedical knowledge graphs. Journal of Biomedical Semantics, 14(1):20, 2023

    work page 2023

  7. [7]

    BERT: Pre-training of deep bidirectional transformers for language understanding, June 2019

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding, June 2019. Multimodal Contrastive Representation Learning in Augmented Biomedical Knowledge Graphs Figure 3. t-SNE visualization of drug embeddings for a single protein with the highest number of interactions ...

    work page 2019

  8. [8]

    MVGCN: data integration through multi-view graph convolutional network for predicting links in biomedical bipartite networks

    Haitao Fu, Feng Huang, Xuan Liu, Yang Qiu, and Wen Zhang. MVGCN: data integration through multi-view graph convolutional network for predicting links in biomedical bipartite networks. Bioinformatics, 38(2):426–434, 09 2021

    work page 2021

Show all 46 references
  1. [9]

    Dudgeon, Kei-Hoi Cheung, Thomas J.S

    Katrin H"ansel, Stephen N. Dudgeon, Kei-Hoi Cheung, Thomas J.S. Durant, and Wolfgang L. Schulz. From data to wisdom: Biomedical knowledge graphs for real-world data insights. Journal of Medical Systems, 47(1):65, May 2023

    2023

  2. [10]

    Contrastive multi-view representation learning on graphs, 2020

    Kaveh Hassani and Amir Hosein Khasahmadi. Contrastive multi-view representation learning on graphs, 2020

    2020

  3. [11]

    Can language models act as knowledge bases at scale?, 2024

    Qiyuan He, Yizhong Wang, and Wenya Wang. Can language models act as knowledge bases at scale?, 2024

    2024

  4. [12]

    DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome

    Yanrong Ji, Zhihan Zhou, Han Liu, and Ramana V Davuluri. DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome. Bioinformatics, 37(15):2112–2120, 02 2021

    2021

  5. [13]

    Instruction-tuned language models are better knowledge learners, 2024

    Zhengbao Jiang, Zhiqing Sun, Weijia Shi, Pedro Rodriguez, Chunting Zhou, Graham Neubig, Xi Victoria Lin, Wen tau Yih, and Srinivasan Iyer. Instruction-tuned language models are better knowledge learners, 2024

    2024

  6. [14]

    Drugbank 6.0: the drugbank knowledgebase for

    Craig Knox, Mike Wilson, Christen M Klinger, Mark Franklin, Eponine Oler, Alex Wilson, Allison Pon, Jordan Cox, Na Eun Chin, Seth A Strawbridge, et al. Drugbank 6.0: the drugbank knowledgebase for

  7. [15]

    DrugBank 6.0: The DrugBank knowledgebase for 2024

    Craig Knox, Mike Wilson, Christen M Klinger, Mark Franklin, Eponine Oler, Alex Wilson, Allison Pon, Jordan Cox, Na Eun Lucy Chin, Seth A Strawbridge, Marysol Garcia-Patino, Ray Kruger, Aadhavya Sivaku- maran, Selena Sanford, Rahil Doshi, Nitya Khetarpal, Omolola Fatokun, Daphn...

    2024

  8. [16]

    Otter-knowledge: bench- marks of multimodal knowledge graph representation learning from different sources for drug discovery

    Hoang Thanh Lam, Marco Luca Sbodio, Marcos Martinez Gallindo, Mykhaylo Zayats, Raul Fernandez-Diaz, Victor Valls, Gabriele Picco, Cesar Berrospi Ramis, and Vanessa Lopez. Otter-knowledge: bench- marks of multimodal knowledge graph representation learning from different sources...

    2023 arXiv

  9. [17]

    Biobert: a pre-trained biomedical language representation model for biomedical text mining

    Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinfor- matics, 36(4):1234–1240, 2020

    2020

  10. [18]

    Pretrained language models for biomedical and clinical tasks: Understanding and extending the state-of-the-art

    Patrick Lewis, Myle Ott, Jingfei Du, and Veselin Stoyanov. Pretrained language models for biomedical and clinical tasks: Understanding and extending the state-of-the-art. In Anna Rumshisky, Kirk Roberts, Steven Bethard, and Tristan Naumann, editors, Proceedings of the 3rd Clin...

    2020

  11. [19]

    Evolutionary-scale prediction of atomic-level protein structure with a language model

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Went- ing Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Salva- tore Candido, and Alexander Rives. Evolutionary-scale prediction of atomic-lev...

    2023

  12. [20]

    Multi-modal contrastive representation learning for entity alignment, October 2022

    Zhenxi Lin, Ziheng Zhang, Meng Wang, Yinghui Shi, Xian Wu, and Yefeng Zheng. Multi-modal contrastive representation learning for entity alignment, October 2022

    2022

  13. [21]

    En- trez gene: gene-centered information at ncbi

    Donna Maglott, Jim Ostell, Kim D Pruitt, and Tatiana Tatusova. En- trez gene: gene-centered information at ncbi. Nucleic acids research, 39(suppl_1):D52–D57, 2010

    2010

  14. [22]

    Link prediction via matrix factorization

    Aditya Krishna Menon and Charles Elkan. Link prediction via matrix factorization. In Dimitrios Gunopulos, Thomas Hofmann, Donato Malerba, and Michalis Vazirgiannis, editors, Machine Learning and Knowledge Discovery in Databases , pages 437–452, Berlin, Heidelberg,

  15. [23]

    Predicting drug- drug interactions using deep generative models on graphs

    Khang Nhat Ngo, Truong Son Hy, and Risi Kondor. Predicting drug- drug interactions using deep generative models on graphs. In NeurIPS 2022 AI for Science: Progress and Promises , 2022. Dang and Nguyen et al

    2022

  16. [24]

    Nicholson and Casey S

    David N. Nicholson and Casey S. Greene. Constructing knowledge graphs and their biomedical applications. Computational and Struc- tural Biotechnology Journal, 18:1414–1428, 2020

    2020

  17. [25]

    Graph Representation Learning via Graphical Mutual Information Maximization, 2020

    Zhen Peng, Wenbing Huang, Minnan Luo, Qinghua Zheng, Yu Rong, Tingyang Xu, and Junzhou Huang. Graph Representation Learning via Graphical Mutual Information Maximization, 2020

    2020

  18. [26]

    Language models as knowledge bases? arXiv preprint arXiv:1909.01066, 2019

    Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. Language models as knowledge bases? arXiv preprint arXiv:1909.01066, 2019

    1909 arXiv

  19. [27]

    Large-scale chemical language repre- sentations capture molecular structure and properties.Nature Machine Intelligence, 4(12):1256–1264, 2022

    Jerret Ross, Brian Belgodere, Vijil Chenthamarakshan, Inkit Padhi, Youssef Mroueh, and Payel Das. Large-scale chemical language repre- sentations capture molecular structure and properties.Nature Machine Intelligence, 4(12):1256–1264, 2022

    2022

  20. [28]

    Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling

    Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks, 2017

    2017

  21. [29]

    Rotate: Knowledge graph embedding by relational rotation in complex space

    Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197, 2019

    1902 arXiv

  22. [30]

    Complex embeddings for simple link predic- tion

    Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link predic- tion. In International conference on machine learning, pages 2071–2080. PMLR, 2016

    2016

  23. [31]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023

    2023

  24. [32]

    Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm

    Petar Veličković, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax, 2018

    2018

  25. [33]

    Biokg: A knowl- edge graph for relational learning on biological data

    Brian Walsh, Sameh K Mohamed, and Vít Nováček. Biokg: A knowl- edge graph for relational learning on biological data. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 3173–3180, 2020

    2020

  26. [34]

    Pre-trained language models in biomedical domain: A systematic survey

    Benyou Wang, Qianqian Xie, Jiahuan Pei, Zhihong Chen, Prayag Tiwari, Zhao Li, and Jie Fu. Pre-trained language models in biomedical domain: A systematic survey. ACM Computing Surveys, 56(3):1–52, 2023

    2023

  27. [35]

    Drug-drug interaction predictions via knowledge graph and text embedding: Instrument validation study

    Mengqi Wang, Haonan Wang, Xingyu Liu, Xinhe Ma, and Baoy- ing Wang. Drug-drug interaction predictions via knowledge graph and text embedding: Instrument validation study. JMIR Med Inform, 9(6):e28277, Jun 24 2021

    2021

  28. [36]

    Embedding entities and relations for learning and inference in knowl- edge bases, 2015

    Bishan Yang, Wen tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowl- edge bases, 2015

    2015

  29. [37]

    Knowledge graph contrastive learning for recommendation, 2022

    Yuhao Yang, Chao Huang, Lianghao Xia, and Chenliang Li. Knowledge graph contrastive learning for recommendation, 2022

    2022

  30. [38]

    Graphsaint: Graph sampling based inductive learning method, 2020

    Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan, and Viktor Prasanna. Graphsaint: Graph sampling based inductive learning method, 2020

    2020

  31. [39]

    KE-GCL: Knowledge enhanced graph contrastive learning for commonsense question answering, December 2022

    Lihui Zhang and Ruifan Li. KE-GCL: Knowledge enhanced graph contrastive learning for commonsense question answering, December 2022

    2022

  32. [40]

    Native: Multi-modal knowledge graph completion in the wild

    Yichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu, Binbin Hu, Ziqi Liu, Wen Zhang, and Huajun Chen. Native: Multi-modal knowledge graph completion in the wild. Authorea Preprints, 2024

    2024

  33. [41]

    Llama beyond english: An empirical study on language capability transfer, 2024

    Jun Zhao, Zhihao Zhang, Luhui Gao, Qi Zhang, Tao Gui, and Xuan- jing Huang. Llama beyond english: An empirical study on language capability transfer, 2024

    2024

  34. [42]

    Yizhen Zheng, Shirui Pan, Vincent Cs Lee, Yu Zheng, and Philip S. Yu. Rethinking and scaling up graph contrastive learning: An extremely efficient approach with group discrimination, 2022

    2022

  35. [43]

    Deep graph contrastive representation learning, 2020

    Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Deep graph contrastive representation learning, 2020

    2020

  36. [44]

    Modeling polypharmacy side effects with graph convolutional networks

    Marinka Zitnik, Monica Agrawal, and Jure Leskovec. Modeling polypharmacy side effects with graph convolutional networks. Bioin- formatics, 34(13):i457–i466, 06 2018. Multimodal Contrastive Representation Learning in Augmented Biomedical Knowledge Graphs Appendix A Parameters a...

    2018

  37. [2011]

    Springer Berlin Heidelberg

  38. [2024]

    Nucleic acids research, 52(D1):D1265–D1275, 2024

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.