Pith. sign in

REVIEW 4 major objections 4 minor 3 cited by

Towards Structure-aware Model for Multi-modal Knowledge Graph Completion

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Structure-anchored contrastive learning lifts multi-modal KG completion by up to 7.4% MRR.

desk verdict TSAM is a sensible, incremental extension of MyGO with plausible SOTA results on two datasets, but the missing-modality handling is unspecified and the evaluation needs more rigor before the gains are trusted. read the letter →

arxiv 2505.21973 v1 pith:VQ4VUYX4 submitted 2025-05-28 cs.MM

Linyu Li , Zhi Jin , Yichi Zhang , Dongming Jin , Chengfeng Dou , Yuanpeng He , Xuan Zhang , Haiyan Zhao This is my paper
classification cs.MM
keywords multi-modalknowledgegraphcompletionfine-grainedmodalityfusionstructure-awarecontrastivelearningembeddinglinkpredictionvisual-textualtokenizationattentionlossalignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that multi-modal knowledge graph completion works best when graph structure, not images or text, is the anchor of entity representations. To that end it proposes TSAM, which fuses image and text information at the level of individual tokens and applies two contrastive losses that pull visual and textual embeddings toward the structural embedding of each entity. On the DB15K, MKG-W, and MKG-Y benchmarks, TSAM reports mean reciprocal rank gains of 7.37%, 7.43%, and 1.5% over the strongest baseline, with larger gains on Hits@1 than on Hits@10. The practical point is that modality noise can be tamed by explicitly subordinating auxiliary modalities to graph structure.

What carries the argument

The two load-bearing mechanisms are FgMAF and SaCL. FgMAF is a fine-grained modality fusion pipeline: visual and textual tokenizers turn each entity's image and description into discrete token sequences, linear projections map both into one space, a Transformer encoder pools them into embeddings, and a softmax attention over structural, visual, and textual embeddings produces a fused entity representation. SaCL is the structure-anchoring component: it computes two symmetric contrastive losses, one between structural and visual embeddings and one between structural and textual embeddings, with in-batch negatives, so the fused representation is regularized to stay close to the graph-structure embedding. A knowledge graph embedding scoring function (TuckER, TransE, or RotatE) supplies structural embeddings and scores candidate triples during training.

What would settle it

Run TSAM on MKG-W and MKG-Y with all entities that lack an image or a text description removed from the evaluation set, and also with those entities replaced by a learned placeholder token; if the MRR gap over MyGO shrinks or disappears under either variant, the reported gains come from missing-modality handling rather than from structure-aware contrastive learning.

Watch

Extended reading notes

Core claim

TSAM claims that the two obstacles to multimodal knowledge graph completion are coarse modality fusion and neglect of structural dominance. Its Fine-grained Modality Awareness Fusion (FgMAF) tokenizes images with BEIT-V2 and descriptions with BERT, linearly projects both token sequences into a shared space, encodes them with a Transformer, and then weights the resulting visual, textual, and structural embeddings with an attention vector before decoding a tail entity. Its Structure-aware Contrastive Learning (SaCL) adds symmetric contrastive losses between structural and visual embeddings and between structural and textual embeddings, so that auxiliary modalities are brought closer to the structure modality rather than allowed to introduce noise. With these components, TSAM reports the best MRR, Hits@1, Hits@3, and Hits@10 on DB15K and MKG-W and the best MRR on MKG-Y, with ablation showing that removing SaCL costs more than removing FgMAF.

Load-bearing premise

The load-bearing premise is that every entity can be given a meaningful visual or textual token sequence, but the paper never states what is fed into the model for the roughly 4 to 6 percent of entities on MKG-W and MKG-Y that lack an image or a description, so the reported comparison could hinge on that unspecified handling.

Editorial extensions

If this is right

  • If TSAM's central claim is right, future multimodal knowledge graph completion models should treat graph structure as the reference modality and align images and text to it, rather than fusing all modalities symmetrically.
  • Fine-grained token-level interaction between image and text tokens is worth more than whole-embedding concatenation, because removing FgMAF lowers MRR on both main benchmarks.
  • Structure-aware contrastive alignment improves ranking quality most at the top of the list, since Hits@1 gains exceed Hits@10 gains on DB15K and MKG-W.
  • The choice of structural scoring function matters: TuckER gives the best fused result on MKG-W, ahead of RotatE and TransE, so the structure encoder remains a significant source of performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit is that the same structure-anchored contrastive loss could be dropped into other fusion architectures, potentially improving their robustness to noisy or irrelevant images and text without retraining their tokenizers.
  • The paper does not describe how entities lacking an image or a description are handled; if missing modalities are zero-padded, a natural test is to compare against masked or learned-placeholder alternatives before attributing gains to contrastive alignment.
  • A testable extension is to replace the fixed attention weights with per-entity modality credibility scores, which the paper lists as future work but does not evaluate; this could matter on datasets with more missing or corrupted modalities than DB15K and MKG-W.
  • Because TSAM centers contrastive learning on structure, it suggests that incremental structure-aware contrastive updates, rather than full retraining, may let multimodal knowledge graph completion models adapt to dynamic knowledge graphs; the paper names this direction but does not test it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes TSAM, a multi-modal knowledge graph completion model combining (i) Fine-grained Modality Awareness Fusion (FgMAF), which tokenizes visual and textual entity data with BEIT-V2 and BERT, encodes the tokens with transformer encoders, and fuses the resulting embeddings with structural KGE embeddings under a learned attention mechanism, and (ii) Structure-aware Contrastive Learning (SaCL), which adds two contrastive losses that align visual and textual embeddings with structural embeddings. The model is trained with a cross-entropy link-prediction loss plus the two contrastive losses. Experiments on DB15K, MKG-W, and MKG-Y report improvements over published baselines, with ablations attributing the gains mainly to SaCL and FgMAF.

Significance. The central claim, that structure-anchored contrastive learning improves MMKGC by reducing modality noise while preserving graph structure, is plausible and potentially useful for the field. The paper provides a motivating demonstration in Fig. 2 that existing models degrade sharply without structural modality knowledge, and the ablations in Table III consistently show that SaCL contributes to the reported performance. The reported MRR improvements on DB15K and MKG-W are sizable. The paper also ships code, includes a parameter sensitivity study, and reports experiments with different decoders and token counts. However, the empirical significance is currently qualified by an underspecified treatment of entities with missing modalities, an unconnected decoder output in Section III.E, an ambiguous fusion operation in Eq. (10), and a baseline comparison that mixes reproduced and source-paper numbers without variance reporting.

major comments (4)
  1. [§III-C/III-D and Table I] The model is underspecified for entities that lack visual or textual data. Table I shows that MKG-W has 14,463 of 15,000 entities with visual data and 14,123 with text, and MKG-Y has similar gaps, yet Eqs. (1)-(2) define tokenizers only for images and text descriptions that exist, Eqs. (10)-(11) require e_vis and e_txt for every entity, and Eqs. (13)-(19) form positive contrastive pairs (S_i,V_i) and (S_i,T_i) for every batch element. The paper must specify what token sequence or embedding is used for entities with missing modalities, how the contrastive losses treat such pairs, and whether evaluation candidates are restricted to entities with all modalities. Without this, the reported gains over SNAG, NativE, and MyGO on MKG-W/Y could be inflated by removing or down-weighting the hardest modality-incomplete entities from ranking or contrastive training.
  2. [§III.E, Eqs. (20)-(22)] The decoder output t_p is never connected to the scoring function used in the prediction loss. Eq. (20) defines t_p = g_d([CLS], h_f, r), but Eqs. (21)-(22) define the loss through Theta(h,r,t_n) = sigmoid(Score(h,r,t_n)), with Score taken from the KGE scoring functions in Eqs. (7)-(9). No equation states whether Score is computed with the fused representation e_f, the structural embedding e_str, or t_p, nor how t_p is compared with candidate entities to produce a distribution over |E| tails. Please specify the exact training and inference scoring pipeline, including how t_p is converted into scores over candidate entities.
  3. [§III.C, Eqs. (10)-(11)] The fusion operation is not well-defined. Eq. (10) says e_f = stack(alpha_s e_str, alpha_v e_vis, alpha_t e_txt), while Eq. (11) computes the attention weights from dot products involving alpha and the three embeddings. It is unclear whether e_f is a concatenation of three weighted vectors, a weighted sum, or some other tensor operation, and how the attention vector alpha is learned. This matters because the proposed modality-awareness mechanism could amount to no more than per-modality scaling if the operation is not specified. Please define the exact tensor operation and dimensions, and clarify how the attention weights are trained.
  4. [Table II and §IV.B] The state-of-the-art comparison mixes reproduced results (only SNAG and NativE are marked with a reproduced symbol) with numbers taken from source papers and prior reports, and no variance, standard deviation, or repeated-seed statistics are reported. Since the central claim is an empirical improvement of 7.37% MRR on DB15K and 7.43% on MKG-W, the paper should report mean and variance over multiple seeds, specify the exact evaluation protocol including candidate filtering and handling of missing modalities, and ideally re-run all baselines under the same protocol or justify why cross-paper numbers are directly comparable.
minor comments (4)
  1. [Eq. (24)] The MRR formula uses |E| and then |N| inconsistently in the same displayed equation; the denominator should be the number of test triples, not the entity set size, and the notation should be unified.
  2. [Algorithm 1, line 19] Line 19 says the prediction loss is L_ST, but L_ST is later defined as the structure-text contrastive loss; the prediction loss is L_p in Eq. (23). Please correct this label.
  3. [§III.C, Eq. (4)] The textual token projection in Eq. (4) writes the result as {t'_1,...,t'_n}, but Eq. (2) defines m textual tokens; the index should be m. Please also state the padding and truncation strategy for token sequences, which interacts with the missing-modality issue.
  4. [§IV and Table II] The paper sometimes refers to two datasets and sometimes to three, and dataset statistics are called 'two datasets' in Section IV.A. In Table II, the SNAG row for MKG-Y has '41.0846.70' without separation, and several model names (MyGO/MyGo, SNAG/SnAg, NativE/Native) are used inconsistently. Please standardize these textual details.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is an empirical SOTA comparison; the only self-citation (MyGO tokenizer/baseline) is not load-bearing, and the missing-modality ambiguity is a validity gap, not a circular reduction.

full rationale

The paper's central claim is an empirical performance comparison on DB15K, MKG-W, and MKG-Y against published baselines, not a derivation from a target result. The two proposed components, FgMAF and SaCL, are defined by equations (5), (6), (10), (11), and (13)-(19), and the total loss in Eq. (23) combines a prediction loss with contrastive losses; the ablations in Table III withhold components and measure test-set MRR, so the reported gains are not forced by construction. No equation defines a predicted quantity in terms of the evaluation metric, and no parameter is fitted to a subset and then reported as a prediction of a closely related quantity. The only self-citation is to MyGO [25], used to justify the BEIT-V2 and BERT tokenizers and as a baseline; since these tokenizers are public external models and the MyGO baseline numbers are published independently, this citation is not load-bearing. A genuine validity gap exists but is not circular: Table I reports that MKG-W has 14,463/15,000 entities with visual data and 14,123/15,000 with text, and MKG-Y has similar gaps, while Section III-C defines tokenizers only for the image and text paragraph 'corresponding to each entity' and Section III-D applies contrastive losses to positive pairs (S_i,V_i) and (S_i,T_i) for every entity in the batch, with no placeholder, exclusion rule, or missing-modality embedding specified. That ambiguity could distort the comparison if incomplete entities are handled differently from baselines, but it is an implementation/evaluation concern rather than a circular derivation. The premise that the structural modality should dominate is encoded in the SaCL loss design, but it is tested by an ablation that removes the loss and observes a drop in MRR, which is a legitimate controlled experiment rather than a self-fulfilling definition.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities. Its free parameters are standard hyperparameters, though the token count and missing-modality handling are not fully disclosed. The main axioms are domain assumptions about pretrained encoders and KGE scoring, plus the paper-specific premise that structure should dominate, which is embedded in the loss design rather than independently demonstrated.

free parameters (4)
  • temperature tau = 0.02
    Contrastive temperature tuned on validation; Fig. 5 shows best at 0.02 on DB15K and MKG-W.
  • number of negative samples K = 16
    Negative samples per positive in SaCL; Table V shows K=16 gives near-best MRR/Hits@1 with moderate memory.
  • number of modality tokens = not reported (varied 2-24)
    Fig. 6 varies visual and textual token counts; the paper does not state the final setting used in Table II.
  • attention vector alpha = learned
    Modality attention weights in Eq. (11) are learned parameters, not fixed; included for completeness.
assumptions (5)
  • domain assumption Pre-trained BEIT-V2 and BERT tokenizers provide semantically meaningful fine-grained tokens for images and text.
    Used in Section III-C to define Tokens_visual(e) and Tokens_textual(e); if these tokens lose information, the fine-grained fusion premise fails.
  • domain assumption KGE scoring functions (TuckER, TransE, RotatE) capture structural semantics.
    Invoked in Section III-C to obtain structural embeddings; standard in KGC literature.
  • ad hoc to paper The graph structure is the dominant modality for MMKGC.
    This is the paper's stated premise (Section I, Fig. 2) and is built into SaCL via losses that align other modalities to structure; it is asserted rather than independently established.
  • ad hoc to paper Contrastive alignment to structure reduces noise without discarding useful modality-specific information.
    Motivates SaCL (Section III-D); not proven, only supported by the model's own ablations.
  • standard math Cross-entropy with sigmoid over KGE scores is a valid training objective for link prediction.
    Section III-E, Eqs. (21)-(22); standard practice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Structure-aware Model for Multi-modal Knowledge Graph Completion." pith.science (2026). https://pith.science/paper/VQ4VUYX4

@misc{pith2026250521973,
  author       = {Pith},
  title        = {Pith review of: Towards Structure-aware Model for Multi-modal Knowledge Graph Completion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VQ4VUYX4}},
  note         = {Machine review of arXiv:2505.21973}
}
read the original abstract

Knowledge graphs (KGs) play a key role in promoting various multimedia and AI applications. However, with the explosive growth of multi-modal information, traditional knowledge graph completion (KGC) models cannot be directly applied. This has attracted a large number of researchers to study multi-modal knowledge graph completion (MMKGC). Since MMKG extends KG to the visual and textual domains, MMKGC faces two main challenges: (1) how to deal with the fine-grained modality information interaction and awareness; (2) how to ensure the dominant role of graph structure in multi-modal knowledge fusion and deal with the noise generated by other modalities during modality fusion. To address these challenges, this paper proposes a novel MMKGC model named TSAM, which integrates fine-grained modality interaction and dominant graph structure to form a high-performance MMKGC framework. Specifically, to solve the challenges, TSAM proposes the Fine-grained Modality Awareness Fusion method (FgMAF), which uses pre-trained language models to better capture fine-grained semantic information interaction of different modalities and employs an attention mechanism to achieve fine-grained modality awareness and fusion. Additionally, TSAM presents the Structure-aware Contrastive Learning method (SaCL), which utilizes two contrastive learning approaches to align other modalities more closely with the structured modality. Extensive experiments show that the proposed TSAM model significantly outperforms existing MMKGC models on widely used multi-modal datasets.

Figures

Figures reproduced from arXiv: 2505.21973 by the authors.

Figure 1
Figure 1. A simple example of MMKGC, which includes not only the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Performance of the MyGO [25] and OTKGE [42] models in terms [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The Architecture of the TSAM Model. TSAM incorporates the FgMAF method to better fuse and perceive various modalities knowledge in MMKG, [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The experimental results of the TSAM model using Bert-base/large, RoBERTa- large, and DeBERTa-base/large as decoders on the DB15K and [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Parameter sensitivity experiment of the number of Temperature [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: The MRR performance of the TSAM model on the DB15K and [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Use t-SNE [62] to visualize the dimensionality reduction of triplet [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Adaptive Fuzzy Time Series Forecasting via Partially Asymmetric Convolution and Sub-Sliding Window Fusion

    cs.AI 2025-07 reject novelty 4.0 of 10

    A fuzzy sliding-window plus partially asymmetric convolutional model reports state-of-the-art MAE/RMSE on most of 43 benchmark time series datasets.

  2. Complementarity-driven Representation Learning for Multi-modal Knowledge Graph Completion

    cs.AI 2025-07 reject novelty 4.0 of 10

    MoCME combines expert-network fusion weighted by estimated mutual information and entropy-based negative sampling, and reports state-of-the-art multi-modal knowledge graph completion on five benchmarks.

  3. Co-Evidential Fusion with Information Volume for Medical Image Segmentation

    cs.CV 2025-06 conditional novelty 3.0 of 10

    A semi-supervised segmentation method using evidential fusion and information-volume weighting reports better Dice on four medical benchmarks, but its novelty relative to the authors' own prior papers is unclear.

Reference graph

Works this paper leans on

83 extracted references · 50 canonical work pages · cited by 3 Pith papers

  1. [1]

    A survey of knowledge graph reasoning on graph types: Static, dynamic, and multi-modal,

    K. Liang, L. Meng, M. Liu, Y . Liu, W. Tu, S. Wang, S. Zhou, X. Liu, F. Sun, and K. He, “A survey of knowledge graph reasoning on graph types: Static, dynamic, and multi-modal,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  2. [2]

    A survey on knowledge graphs: Representation, acquisition, and applications,

    S. Ji, S. Pan, E. Cambria, P. Marttinen, and S. Y . Philip, “A survey on knowledge graphs: Representation, acquisition, and applications,”IEEE transactions on neural networks and learning systems, vol. 33, no. 2, pp. 494–514, 2021

    2021

  3. [3]

    Multi-modal variational graph auto-encoder for recommendation systems,

    J. Yi and Z. Chen, “Multi-modal variational graph auto-encoder for recommendation systems,”IEEE Transactions on Multimedia, vol. 24, pp. 1067–1079, 2021

    2021

  4. [4]

    Adapting large language models by integrating collaborative semantics for recommendation,

    B. Zheng, Y . Hou, H. Lu, Y . Chen, W. X. Zhao, M. Chen, and J.-R. Wen, “Adapting large language models by integrating collaborative semantics for recommendation,” in2024 IEEE 40th International Conference on Data Engineering (ICDE). IEEE, 2024, pp. 1435–1448

    2024

  5. [5]

    Graph neural networks in recommender systems: a survey,

    S. Wu, F. Sun, W. Zhang, X. Xie, and B. Cui, “Graph neural networks in recommender systems: a survey,”ACM Computing Surveys, vol. 55, no. 5, pp. 1–37, 2022

    2022

  6. [6]

    Cross-modal knowledge graph contrastive learning for machine learning method rec- ommendation,

    X. Cao, Y . Shi, J. Wang, H. Yu, X. Wang, and Z. Yan, “Cross-modal knowledge graph contrastive learning for machine learning method rec- ommendation,” inProceedings of the 30th ACM international conference on multimedia, 2022, pp. 3694–3702

    work page 2022

  7. [7]

    Knowledge graph self- supervised rationalization for recommendation,

    Y . Yang, C. Huang, L. Xia, and C. Huang, “Knowledge graph self- supervised rationalization for recommendation,” inProceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining, 2023, pp. 3046–3056

    work page 2023

  8. [8]

    Building and using personal knowledge graph to improve suicidal ideation detection on social media,

    L. Cao, H. Zhang, and L. Feng, “Building and using personal knowledge graph to improve suicidal ideation detection on social media,”IEEE Transactions on Multimedia, vol. 24, pp. 87–102, 2022

    work page 2022

Show all 83 references
  1. [9]

    Context matters: Distilling knowledge graph for enhanced object detection,

    A. Yang, S. Lin, C.-H. Yeh, M. Shu, Y . Yang, and X. Chang, “Context matters: Distilling knowledge graph for enhanced object detection,” IEEE Transactions on Multimedia, vol. 26, pp. 487–500, 2023

    2023

  2. [10]

    Unifying large language models and knowledge graphs: A roadmap,

    S. Pan, L. Luo, Y . Wang, C. Chen, J. Wang, and X. Wu, “Unifying large language models and knowledge graphs: A roadmap,”IEEE Transactions on Knowledge and Data Engineering, 2024

    2024

  3. [11]

    Knowledge graphs meet multi-modal learning: A comprehensive survey,

    Z. Chen, Y . Zhang, Y . Fang, Y . Geng, L. Guo, X. Chen, Q. Li, W. Zhang, J. Chen, Y . Zhuet al., “Knowledge graphs meet multi-modal learning: A comprehensive survey,”arXiv preprint arXiv:2402.05391, 2024

    2024 arXiv

  4. [12]

    Psnea: Pseudo- siamese network for entity alignment between multi-modal knowledge graphs,

    W. Ni, Q. Xu, Y . Jiang, Z. Cao, X. Cao, and Q. Huang, “Psnea: Pseudo- siamese network for entity alignment between multi-modal knowledge graphs,” inProceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 3489–3497

    2023

  5. [13]

    Multi-modal knowledge graph construction and application: A survey,

    X. Zhu, Z. Li, X. Wang, X. Jiang, P. Sun, X. Wang, Y . Xiao, and N. J. Yuan, “Multi-modal knowledge graph construction and application: A survey,”IEEE Transactions on Knowledge and Data Engineering, vol. 36, no. 2, pp. 715–735, 2022

    2022

  6. [14]

    Translating embeddings for modeling multi-relational data,

    A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, and O. Yakhnenko, “Translating embeddings for modeling multi-relational data,”Advances in neural information processing systems, vol. 26, 2013

    2013

  7. [15]

    Rotate: Knowledge graph embedding by relational rotation in complex space,

    Z. Sun, Z.-H. Deng, J.-Y . Nie, and J. Tang, “Rotate: Knowledge graph embedding by relational rotation in complex space,”arXiv preprint arXiv:1902.10197, 2019

    1902 arXiv

  8. [16]

    Are missing links predictable? an inferential benchmark for knowledge graph completion,

    Y . Cao, X. Ji, X. Lv, J. Li, Y . Wen, and H. Zhang, “Are missing links predictable? an inferential benchmark for knowledge graph completion,” C. Zong, F. Xia, W. Li, and R. Navigli, Eds. Online: Association for Computational Linguistics, Aug. 2021, pp. 6855–6865. [Online]. Av...

    2021

  9. [17]

    Simkgc: Simple contrastive knowledge graph completion with pre-trained language models,

    L. Wang, W. Zhao, Z. Wei, and J. Liu, “Simkgc: Simple contrastive knowledge graph completion with pre-trained language models,”arXiv preprint arXiv:2203.02167, 2022

    2022 arXiv

  10. [18]

    Matching knowledge graphs in entity embedding spaces: an experimental study,

    W. Zeng, X. Zhao, Z. Tan, J. Tang, and X. Cheng, “Matching knowledge graphs in entity embedding spaces: an experimental study,”IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 12, pp. 12 770–12 784, 2023

    2023

  11. [19]

    Multi- modal siamese network for few-shot knowledge graph completion,

    Y . Wei, W. Chen, X. Zhang, P. Zhao, J. Qu, and L. Zhao, “Multi- modal siamese network for few-shot knowledge graph completion,” in 2024 IEEE 40th International Conference on Data Engineering (ICDE). IEEE, 2024, pp. 719–732. 12

    2024

  12. [20]

    Mgksite: Multi-modal knowledge-driven site selection via intra and inter-modal graph fusion,

    K. Liang, L. Meng, H. Li, M. Liu, S. Wang, S. Zhou, X. Liu, and K. He, “Mgksite: Multi-modal knowledge-driven site selection via intra and inter-modal graph fusion,”IEEE Transactions on Multimedia, 2024

    2024

  13. [21]

    Mmkgr: Multi-hop multi-modal knowledge graph reasoning,

    S. Zheng, W. Wang, J. Qu, H. Yin, W. Chen, and L. Zhao, “Mmkgr: Multi-hop multi-modal knowledge graph reasoning,” in2023 IEEE 39th International Conference on Data Engineering (ICDE). IEEE, 2023, pp. 96–109

    2023

  14. [22]

    Multi- modal analogical reasoning over knowledge graphs,

    N. Zhang, L. Li, X. Chen, X. Liang, S. Deng, and H. Chen, “Multi- modal analogical reasoning over knowledge graphs,” inThe Eleventh International Conference on Learning Representations, 2022

    2022

  15. [23]

    Native: Multi-modal knowledge graph completion in the wild,

    Y . Zhang, Z. Chen, L. Guo, Y . Xu, B. Hu, Z. Liu, W. Zhang, and H. Chen, “Native: Multi-modal knowledge graph completion in the wild,” inProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2024, pp. 91– 101

    2024

  16. [24]

    Con- trast then memorize: Semantic neighbor retrieval-enhanced inductive multimodal knowledge graph completion,

    Y . Zhao, Y . Zhang, B. Zhou, X. Qian, K. Song, and X. Cai, “Con- trast then memorize: Semantic neighbor retrieval-enhanced inductive multimodal knowledge graph completion,” inProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information...

    2024

  17. [25]

    Mygo: Discrete modality information as fine-grained to- kens for multi-modal knowledge graph completion,

    Y . Zhang, Z. Chen, L. Guo, Y . Xu, B. Hu, Z. Liu, H. Chen, and W. Zhang, “Mygo: Discrete modality information as fine-grained to- kens for multi-modal knowledge graph completion,”arXiv preprint arXiv:2404.09468, 2024

    2024 arXiv

  18. [26]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” inInternational conference on machine learning. PMLR, 2021, pp. 8748–8763

    2021

  19. [27]

    Beit v2: Masked image modeling with vector-quantized visual tokenizers,

    Z. Peng, L. Dong, H. Bao, Q. Ye, and F. Wei, “Beit v2: Masked image modeling with vector-quantized visual tokenizers,”arXiv preprint arXiv:2208.06366, 2022

    2022 arXiv

  20. [28]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models,

    D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “Minigpt-4: Enhancing vision-language understanding with advanced large language models,”arXiv preprint arXiv:2304.10592, 2023

    2023 arXiv

  21. [29]

    Beit: Bert pre-training of image transformers,

    H. Bao, L. Dong, S. Piao, and F. Wei, “Beit: Bert pre-training of image transformers,”arXiv preprint arXiv:2106.08254, 2021

    2021 arXiv

  22. [30]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, “Bert: Pre-training of deep bidirectional transformers for language understanding,”arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  23. [31]

    Roberta: A robustly optimized bert pretraining approach,

    Y . Liu, “Roberta: A robustly optimized bert pretraining approach,”arXiv preprint arXiv:1907.11692, vol. 364, 2019

    1907 arXiv

  24. [32]

    Deberta: Decoding-enhanced bert with disentangled attention,

    P. He, X. Liu, J. Gao, and W. Chen, “Deberta: Decoding-enhanced bert with disentangled attention,”arXiv preprint arXiv:2006.03654, 2020

    2006 arXiv

  25. [33]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,”Advances in Neural Information Processing Systems, 2017

    2017

  26. [34]

    Tucker: Tensor factoriza- tion for knowledge graph completion,

    I. Bala ˇzevi´c, C. Allen, and T. M. Hospedales, “Tucker: Tensor factoriza- tion for knowledge graph completion,”arXiv preprint arXiv:1901.09590, 2019

    1901 arXiv

  27. [35]

    Structure- augmented text representation learning for efficient knowledge graph completion,

    B. Wang, T. Shen, G. Long, T. Zhou, Y . Wang, and Y . Chang, “Structure- augmented text representation learning for efficient knowledge graph completion,” inProceedings of the Web Conference 2021, 2021, pp. 1737–1748

    2021

  28. [36]

    Kg-bert: Bert for knowledge graph completion,

    L. Yao, C. Mao, and Y . Luo, “Kg-bert: Bert for knowledge graph completion,”arXiv preprint arXiv:1909.03193, 2019

    1909 arXiv

  29. [37]

    Neural bellman-ford networks: A general graph neural network framework for link predic- tion,

    Z. Zhu, Z. Zhang, L.-P. Xhonneux, and J. Tang, “Neural bellman-ford networks: A general graph neural network framework for link predic- tion,”Advances in Neural Information Processing Systems, vol. 34, pp. 29 476–29 490, 2021

    2021

  30. [38]

    Composition- based multi-relational graph convolutional networks,

    S. Vashishth, S. Sanyal, V . Nitin, and P. Talukdar, “Composition- based multi-relational graph convolutional networks,”arXiv preprint arXiv:1911.03082, 2019

    1911 arXiv

  31. [39]

    A knowledge graph completion model based on contrastive learning and relation enhancement method,

    L. Li, X. Zhang, Y . Ma, C. Gao, J. Wang, Y . Yu, Z. Yuan, and Q. Ma, “A knowledge graph completion model based on contrastive learning and relation enhancement method,”Knowledge-Based Systems, vol. 256, p. 109889, 2022

    2022

  32. [40]

    Relational message passing for fully inductive knowledge graph completion,

    Y . Geng, J. Chen, J. Z. Pan, M. Chen, S. Jiang, W. Zhang, and H. Chen, “Relational message passing for fully inductive knowledge graph completion,” in2023 IEEE 39th International Conference on Data Engineering (ICDE). IEEE, 2023, pp. 1221–1233

    2023

  33. [41]

    A survey of multi-modal knowledge graphs: Technologies and trends,

    W. Liang, P. D. Meo, Y . Tang, and J. Zhu, “A survey of multi-modal knowledge graphs: Technologies and trends,”ACM Computing Surveys, vol. 56, no. 11, pp. 1–41, 2024

    2024

  34. [42]

    Otkge: Multi- modal knowledge graph embeddings via optimal transport,

    Z. Cao, Q. Xu, Z. Yang, Y . He, X. Cao, and Q. Huang, “Otkge: Multi- modal knowledge graph embeddings via optimal transport,”Advances in Neural Information Processing Systems, vol. 35, pp. 39 090–39 102, 2022

    2022

  35. [43]

    Lafa: Multimodal knowledge graph completion with link aware fusion and aggregation,

    B. Shang, Y . Zhao, J. Liu, and D. Wang, “Lafa: Multimodal knowledge graph completion with link aware fusion and aggregation,” inProceed- ings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 8, 2024, pp. 8957–8965

    2024

  36. [44]

    Multimodal reasoning with multimodal knowledge graph,

    J. Lee, Y . Wang, J. Li, and M. Zhang, “Multimodal reasoning with multimodal knowledge graph,”arXiv preprint arXiv:2406.02030, 2024

    2024 arXiv

  37. [45]

    Imf: interactive multimodal fusion model for link prediction,

    X. Li, X. Zhao, J. Xu, Y . Zhang, and C. Xing, “Imf: interactive multimodal fusion model for link prediction,” inProceedings of the ACM Web Conference 2023, 2023, pp. 2572–2580

    2023

  38. [46]

    Rethinking graph convolutional networks in knowledge graph completion,

    Z. Zhang, J. Wang, J. Ye, and F. Wu, “Rethinking graph convolutional networks in knowledge graph completion,” inProceedings of the ACM Web Conference 2022, 2022, pp. 798–807

    2022

  39. [47]

    Image-embodied knowledge representation learning,

    R. Xie, Z. Liu, H. Luan, and M. Sun, “Image-embodied knowledge representation learning,”arXiv preprint arXiv:1609.07028, 2016

    2016 arXiv

  40. [48]

    Unleashing the power of imbalanced modality information for multi-modal knowledge graph completion,

    Y . Zhang, Z. Chen, L. Liang, H. Chen, and W. Zhang, “Unleashing the power of imbalanced modality information for multi-modal knowledge graph completion,”arXiv preprint arXiv:2402.15444, 2024

    2024 arXiv

  41. [49]

    Vista: Visual-textual knowledge graph representation learning,

    J. Lee, C. Chung, H. Lee, S. Jo, and J. Whang, “Vista: Visual-textual knowledge graph representation learning,” inFindings of the Association for Computational Linguistics: EMNLP 2023, 2023, pp. 7314–7328

    2023

  42. [50]

    Tiva-kg: A multimodal knowledge graph with text, image, video and audio,

    X. Wang, B. Meng, H. Chen, Y . Meng, K. Lv, and W. Zhu, “Tiva-kg: A multimodal knowledge graph with text, image, video and audio,” in Proceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 2391–2399

    2023

  43. [51]

    Relation-enhanced negative sampling for multimodal knowledge graph completion,

    D. Xu, T. Xu, S. Wu, J. Zhou, and E. Chen, “Relation-enhanced negative sampling for multimodal knowledge graph completion,” inProceedings of the 30th ACM international conference on multimedia, 2022, pp. 3857–3866

    2022

  44. [52]

    Pytorch: An imperative style, high-performance deep learning library,

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antigaet al., “Pytorch: An imperative style, high-performance deep learning library,”Advances in neural information processing systems, vol. 32, 2019

    2019

  45. [53]

    Adam: A method for stochastic optimization,

    D. P. Kingma, “Adam: A method for stochastic optimization,”arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  46. [54]

    A simple framework for contrastive learning of visual representations,

    T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” inInternational conference on machine learning. PMLR, 2020, pp. 1597–1607

    2020

  47. [55]

    Mocov1: Momentum contrast for unsupervised visual representation learning,

    K. He, H. Fan, Y . Wu, S. Xie, and R. Girshick, “Mocov1: Momentum contrast for unsupervised visual representation learning,” 2020

    2020

  48. [56]

    Simcse: Simple contrastive learning of sentence embeddings,

    T. Gao, X. Yao, and D. Chen, “Simcse: Simple contrastive learning of sentence embeddings,”arXiv preprint arXiv:2104.08821, 2021

    2021 arXiv

  49. [57]

    Mmkg: multi-modal knowledge graphs,

    Y . Liu, H. Li, A. Garcia-Duran, M. Niepert, D. Onoro-Rubio, and D. S. Rosenblum, “Mmkg: multi-modal knowledge graphs,” inThe Semantic Web: 16th International Conference, ESWC 2019, Portoro ˇz, Slovenia, June 2–6, 2019, Proceedings 16. Springer, 2019, pp. 459–474

    2019

  50. [58]

    Multimodal deep learning,

    J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Y . Ng, “Multimodal deep learning,” inProceedings of the 28th international conference on machine learning (ICML-11), 2011, pp. 689–696

    2011

  51. [59]

    Embedding entities and relations for learning and inference in knowledge bases,

    B. Yang, S. W.-t. Yih, X. He, J. Gao, and L. Deng, “Embedding entities and relations for learning and inference in knowledge bases,” inPro- ceedings of the International Conference on Learning Representations (ICLR) 2015, 2015

    2015

  52. [60]

    Learning entity and relation embeddings for knowledge graph completion,

    Y . Lin, Z. Liu, M. Sun, Y . Liu, and X. Zhu, “Learning entity and relation embeddings for knowledge graph completion,” inProceedings of the AAAI conference on artificial intelligence, vol. 29, no. 1, 2015

    2015

  53. [61]

    Embracing unimodal aleatoric uncertainty for robust multimodal fusion,

    Z. Gao, X. Jiang, X. Xu, F. Shen, Y . Li, and H. T. Shen, “Embracing unimodal aleatoric uncertainty for robust multimodal fusion,” inPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 26 876–26 885

    2024

  54. [62]

    Visualizing data using t-sne

    L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.”Journal of machine learning research, vol. 9, no. 11, 2008

    2008

  55. [63]

    Understanding the behaviour of contrastive loss,

    F. Wang and H. Liu, “Understanding the behaviour of contrastive loss,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 2495–2504

    2021

  56. [64]

    The power of noise: Toward a unified multi-modal knowledge graph representation framework,

    Z. Chen, Y . Fang, Y . Zhang, L. Guo, J. Chen, H. Chen, and W. Zhang, “The power of noise: Toward a unified multi-modal knowledge graph representation framework,”arXiv preprint arXiv:2403.06832, 2024

    2024 arXiv

  57. [65]

    Learning hierarchy-aware knowledge graph embeddings for link prediction,

    Z. Zhang, J. Cai, Y . Zhang, and J. Wang, “Learning hierarchy-aware knowledge graph embeddings for link prediction,” inProceedings of the AAAI conference on artificial intelligence, vol. 34, no. 03, 2020, pp. 3065–3072

    2020

  58. [66]

    Knowledge graph completion method based on quantum embedding and quaternion interaction enhancement,

    L. Li, X. Zhang, Z. Jin, C. Gao, R. Zhu, Y . Liang, and Y . Ma, “Knowledge graph completion method based on quantum embedding and quaternion interaction enhancement,”Information Sciences, vol. 648, p. 119548, 2023. 13

    2023

  59. [67]

    Relation-aware multi-positive contrastive knowledge graph completion with embedding dimension scaling,

    B. Shang, Y . Zhao, D. Wang, and J. Liu, “Relation-aware multi-positive contrastive knowledge graph completion with embedding dimension scaling,” inProceedings of the 46th International ACM SIGIR Con- ference on Research and Development in Information Retrieval, 2023, pp. 878–888

    2023

  60. [68]

    Weighted knowledge graph embedding,

    Z. Zhang, Z. Guan, F. Zhang, F. Zhuang, Z. An, F. Wang, and Y . Xu, “Weighted knowledge graph embedding,” inProceedings of the 46th international ACM SIGIR conference on research and development in information retrieval, 2023, pp. 867–877

    2023

  61. [69]

    Sphere: Expressive and interpretable knowl- edge graph embedding for set retrieval,

    Z. Li, Y . Ao, and J. He, “Sphere: Expressive and interpretable knowl- edge graph embedding for set retrieval,” inProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2024, pp. 2629–2634

    2024

  62. [70]

    Multi-view riemannian manifolds fusion enhancement for knowledge graph completion,

    L. Li, Z. Jin, X. Zhang, H. Duan, J. Wang, Z. Tao, H. Zhao, and X. Zhu, “Multi-view riemannian manifolds fusion enhancement for knowledge graph completion,”IEEE Transactions on Knowledge and Data Engineering, 2025

    2025

  63. [71]

    Clustering then propagation: Select better anchors for knowledge graph embedding,

    K. Liang, Y . Liu, H. Li, L. Meng, S. Liu, S. Wang, X. Liuet al., “Clustering then propagation: Select better anchors for knowledge graph embedding,” inThe Thirty-eighth Annual Conference on Neural Information Processing Systems

  64. [72]

    Expressive: A spatio-functional embed- ding for knowledge graph completion,

    A. Pavlovi ´c and E. Sallinger, “Expressive: A spatio-functional embed- ding for knowledge graph completion,” inThe Eleventh International Conference on Learning Representations

  65. [73]

    Dipping plms sauce: Bridging structure and text for effective knowledge graph completion via conditional soft prompting,

    C. Chen, Y . Wang, A. Sun, B. Li, and K.-Y . Lam, “Dipping plms sauce: Bridging structure and text for effective knowledge graph completion via conditional soft prompting,”arXiv preprint arXiv:2307.01709, 2023

    2023 arXiv

  66. [74]

    InGram: Inductive knowledge graph embedding via relation graphs,

    J. Lee, C. Chung, and J. J. Whang, “InGram: Inductive knowledge graph embedding via relation graphs,” inProceedings of the 40th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabat...

    2023

  67. [75]

    Mixed geometry message and trainable convolutional attention network for knowledge graph comple- tion,

    B. Shang, Y . Zhao, J. Liu, and D. Wang, “Mixed geometry message and trainable convolutional attention network for knowledge graph comple- tion,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 8, 2024, pp. 8966–8974

    2024

  68. [76]

    Geometry interaction knowledge graph embeddings,

    Z. Cao, Q. Xu, Z. Yang, X. Cao, and Q. Huang, “Geometry interaction knowledge graph embeddings,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 5, 2022, pp. 5521–5529

    2022

  69. [77]

    Hybrid transformer with multi-level fusion for multimodal knowledge graph completion,

    X. Chen, N. Zhang, L. Li, S. Deng, C. Tan, C. Xu, F. Huang, L. Si, and H. Chen, “Hybrid transformer with multi-level fusion for multimodal knowledge graph completion,” inProceedings of the 45th international ACM SIGIR conference on research and development in information retri...

    2022

  70. [78]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azharet al., “Llama: Open and efficient foundation language models,”arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  71. [79]

    Is visual context really helpful for knowledge graph? a representation learning perspective,

    M. Wang, S. Wang, H. Yang, Z. Zhang, X. Chen, and G. Qi, “Is visual context really helpful for knowledge graph? a representation learning perspective,” inProceedings of the 29th ACM International Conference on Multimedia, 2021, pp. 2735–2743

    2021

  72. [80]

    Knowledge graph contrastive learning based on relation-symmetrical structure,

    K. Liang, Y . Liu, S. Zhou, W. Tu, Y . Wen, X. Yang, X. Dong, and X. Liu, “Knowledge graph contrastive learning based on relation-symmetrical structure,”IEEE Transactions on Knowledge and Data Engineering, vol. 36, no. 1, pp. 226–238, 2023

    2023

  73. [81]

    Simple yet effective: Structure guided pre- trained transformer for multi-modal knowledge graph reasoning,

    K. Liang, L. Meng, Y . Liu, M. Liu, W. Wei, S. Liu, W. Tu, S. Wang, S. Zhou, and X. Liu, “Simple yet effective: Structure guided pre- trained transformer for multi-modal knowledge graph reasoning,” in Proceedings of the 32nd ACM International Conference on Multimedia, 2024, pp...

    2024

  74. [82]

    Dysarl: Dynamic structure-aware representation learning for multimodal knowledge graph reasoning,

    K. Liu, F. Zhao, Y . Yang, and G. Xu, “Dysarl: Dynamic structure-aware representation learning for multimodal knowledge graph reasoning,” in Proceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 8247–8256

    2024

  75. [83]

    A mul- timodal translation-based approach for knowledge graph representation learning,

    H. Mousselly-Sergieh, T. Botschen, I. Gurevych, and S. Roth, “A mul- timodal translation-based approach for knowledge graph representation learning,” inProceedings of the Seventh Joint Conference on Lexical and Computational Semantics, 2018, pp. 225–234

    2018

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.