REVIEW 4 major objections 5 minor 41 references
Diffusion-based Hierarchical Negative Sampling for Multimodal Knowledge Graph Completion
T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper proposes DHNS, a diffusion-based scheme that generates negative triples for multimodal knowledge graph completion, and reports it outperforms state-of-the-art baselines on three benchmarks.
desk verdict A useful new combination—diffusion-based negative sampling for multimodal KGC—but the experimental reporting is too thin to fully support the central claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is DiffHEG (Diffusion-based Hierarchical Embedding Generation), a denoising diffusion probabilistic model that reverses a forward noising Markov chain to produce synthetic entity embeddings. Conditioning is multimodal, combining structural, visual, and textual features of the head entity and the relation through one of three interaction mechanisms (Hadamard, bilinear, or additive) matched to the KGE model. Hardness is controlled by the reverse-diffusion time step $t$, with hardness level $\mathrm{HL}(\hat{x}_t) \propto 1/t$, so early steps give near-positive "hard" negatives. These generated negatives are consumed by NTAT (Negative Triple-Adaptive Training), whose Hardness-Adaptive Loss uses a margin $\gamma_t$ that shrinks as the negative gets harder, and whose multimodal joint scoring averages the KGE score over the three modality-specific generated embeddings.
What would settle it
One decisive check is to score every generated negative triple against the KG: if a large share of generated negatives at small time steps are either already true triples or lie far outside the convex hull of real entity embeddings, the training signal is corrupted and the claimed advantage should shrink or vanish.
Extended reading notes
Core claim
The central claim is that negative triples for MMKGC should be generated, not sampled: a conditional denoising diffusion model (DiffHEG) starts from pure Gaussian noise and, guided by the head entity and relation embeddings plus structural, textual, and visual features, produces synthetic tail-entity embeddings. Time steps $T/20$, $T/10$, $T/5$, and $T/2$ yield negatives of increasing easiness, with hardness inversely proportional to the time step. These synthesized negatives are scored by the KGE model's own score function through multimodal joint scoring, and a Hardness-Adaptive Loss assigns smaller margins to harder negatives. The paper argues that direct generation combined with adaptive margins gives a more robust training signal than sampling-based approaches, and reports that DHNS integrated with RotatE achieves the best or second-best MRR, Hits@1, Hits@3, and Hits@10 across DB15K, MKG-W, and MKG-Y.
Load-bearing premise
The whole scheme rests on the assumption that a random vector denoised under the condition of a head entity and relation lands on a plausible entity embedding that the scoring model can treat as a sensible negative, even though nothing explicitly checks that the generated vector resembles a real entity or is not a true triple.
Editorial extensions
If this is right
- DHNS can be plugged into existing KGE models such as TransE, DistMult, and RotatE to improve their MMKGC accuracy without altering their score functions.
- Because the diffusion process can generate negatives of any hardness by choosing the time step, training can be balanced across easy and hard negatives instead of relying on a single fixed margin.
- The margin adaptation in the Hardness-Adaptive Loss implies that the same KGE model can be trained more effectively on a mixture of negative difficulties, reducing sensitivity to the global margin hyperparameter.
- The ablation results indicate that the DiffHEG generation module, rather than the training strategy, is the main source of gain; removing it causes the largest performance drops across all three datasets.
Reading between the lines
- A testable extension would be to filter generated negatives that already exist as true triples in the KG, or to project generated embeddings onto the nearest real entity embedding; the paper does not report whether such false negatives occur and how often.
- Because hardness is tied to the diffusion time step, the same generator could support curriculum learning, starting training with easy negatives and gradually increasing difficulty, which the paper does not explore.
- The conditioning mechanisms are matched to the algebraic structure of the base KGE model, suggesting a general recipe: the interaction used for conditioning should align with the score function's composition rule, a principle the paper illustrates but does not state explicitly.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes DHNS, a diffusion-based negative sampling scheme for multimodal knowledge graph completion. DHNS trains a conditional diffusion model (DiffHEG) to generate tail (or head) entity embeddings from Gaussian noise conditioned on the counterpart entity, relation, and multimodal features; hardness of the generated negative triples is controlled by stopping the reverse diffusion at different time steps. The generated negatives are then used in a Negative Triple-Adaptive Training (NTAT) loss with hardness-adaptive margins. Experiments on DB15K, MKG-W, and MKG-Y report improvements over existing unimodal, multimodal, and negative-sampling baselines, with ablations showing the contribution of each module.
Significance. If the claims are substantiated, DHNS would be a novel and useful contribution to MMKGC: it is, to the best of the paper's claim, the first diffusion-based negative embedding generation approach specific to multimodal knowledge graph completion, and the idea of controlling hardness via diffusion time steps is intuitive. The paper also proposes a promising adaptive-margin training scheme. However, the current manuscript leaves key definitions underspecified and lacks empirical validation under controlled comparisons, so the significance of the work is not yet demonstrated.
major comments (4)
- [3.2, Eqs. (4)-(6)] The reverse diffusion update rule in Eqs. (4)-(6) appears to use β_t in the denominator, but β_t is defined as 1 - α_t in Eq. (1). For small t, α_t is typically near 1, so β_t is near 0, making the coefficient 1/√β_t in Eq. (5) extremely large and the update numerically unstable. In the standard DDPM posterior, the coefficient is 1/√α_t or 1/√(1-β_t), where the noise variance is small for small t. Please re-derive the reverse process from the forward process in Eqs. (1)-(2) and correct the equations, or clarify if β_t is intended to denote the noise variance rather than 1 - α_t.
- [3.3, Eq. (13)] The hardness-adaptive margin γ_t and the negative weights w(negt) in Eq. (13) are never explicitly defined. The text states that γ_t is adaptive to the hardness level HL(t) and that weights are higher for embeddings closer to the halfway point of the diffusion process, but no functional form, schedule, or update rule is provided. This makes the proposed NTAT loss non-reproducible and prevents independent verification of the adaptive-margin mechanism. Please specify γ_t and w(negt) in closed form or with an exact algorithmic description.
- [3.2-3.3, Eqs. (5)-(6), (12)-(13)] The framework never checks whether the generated negative embeddings lie near the manifold of real entity embeddings or whether the composed triples (h, r, neg_t) are already true triples. The generated negatives are arbitrary points in the embedding space, and at small t they are explicitly described as being closer to positives, increasing the risk of false negatives. Consequently, the loss in Eq. (13) may train the KGE model to assign low scores to off-manifold vectors, while inference only ranks real entities, so the improvements reported in Tables 2-4 may not transfer to the actual ranking task. Please provide quantitative evidence about the relation of generated embeddings to real entities (e.g., nearest-neighbor statistics) and report the false-negative rate among generated negatives.
- [4.1, Tables 2-4] Baseline results are copied from prior publications ([38] for Tables 2-3 and [34] for Table 4) rather than re-run in the same experimental framework, and no error bars or significance tests are reported anywhere. The claimed consistent improvements over baselines therefore rest on potentially incompatible setups and are not statistically validated. Please re-run the comparisons under identical conditions, or at least report multiple-run statistics (mean and standard deviation) and significance tests for the reported metrics.
minor comments (5)
- [Eq. (13)] The symbol t is used both for the tail entity in a triple and for the diffusion time step, causing ambiguity in the loss definition; using e_t or a distinct subscript for the tail entity would clarify the equation.
- [3.3] In the opening of Section 3.3, "HTAT" appears to be a typo for "NTAT", the negative triple-adaptive training mechanism.
- [3.2, Eq. (7)] The word "Hardmard" should be "Hadamard" in the description of the multiplication operation in Eq. (7).
- [Contributions, Section 1] The phrase "As we can be concerned" is awkward and should be rephrased, for example as "To the best of our knowledge".
- [4.2, Table 4] The claim that DHNS "consistently and significantly outperforms" other NS strategies is not fully supported by Table 4: for TransE on MKG-W, the H10 of DHNS (46.85) is lower than that of MMRNS (47.11), and several other differences are small. Please either soften the claim or discuss the discrepancies in the text.
Circularity Check
No circular derivation: DiffHEG is a trained generative model, not a re-fit of the target metric. One minor self-citation for baseline numbers does not make the central claim circular.
full rationale
The paper's core claim is that DHNS improves MMKGC training by generating hard negative entity embeddings. The generation path is not circular: DiffHEG is trained by the denoising loss in Eq. 11 to reconstruct true tail-entity embeddings from noised versions conditioned on head and relation, and negatives are produced by the reverse process in Eqs. 5-6 at chosen timesteps. Hardness is defined by the timestep (HL ∝ 1/t) rather than by the downstream KGE score, so the hardness-adaptive margin in Eq. 13 is a design choice, not a self-fulfilling prediction. The only self-citation of note is the sourcing of baseline results in Tables 2-3 from [38], a prior paper co-authored by W. Zhang; this is a reporting-fairness caveat, not a circular derivation, because those numbers are published external results and the DHNS method itself is independently specified. The absence of a check that generated embeddings lie on the entity manifold is a soundness/correctness risk, not a circularity.
Assumptions & free parameters
free parameters (6)
- Total diffusion time steps T =
not reported (tuned from {20, 50, 70, 100})
- DiffHEG learning rate =
not reported (tuned from {2e-3, 1e-4, 5e-4})
- Loss trade-off lambda =
not reported
- Hardness-adaptive margin gamma_t =
undefined
- Negative weights w(negt) =
undefined
- Hardness time-step set {T/20, T/10, T/5, T/2} =
chosen by hand
assumptions (5)
- standard math DDPM forward and reverse processes with Gaussian noise are valid for continuous entity embeddings.
- domain assumption Conditional denoising C(x_e, x_r) via Hadamard, bilinear, or addition captures relational semantics for generation.
- ad hoc to paper Generated embeddings at different diffusion timesteps correspond to semantically meaningful hardness levels, with hardness proportional to 1/t.
- domain assumption The KGE score function E(.) can be applied to generated embeddings as if they were real entity embeddings.
- domain assumption Fixed BEiT and SBERT features are sufficient and informative for the conditional generation.
Cite this review
Pith. "Pith review of Diffusion-based Hierarchical Negative Sampling for Multimodal Knowledge Graph Completion." pith.science (2026). https://pith.science/paper/4N6ZAPSW
@misc{pith2026250115393,
author = {Pith},
title = {Pith review of: Diffusion-based Hierarchical Negative Sampling for Multimodal Knowledge Graph Completion},
year = {2026},
howpublished = {\url{https://pith.science/paper/4N6ZAPSW}},
note = {Machine review of arXiv:2501.15393}
}
read the original abstract
Multimodal Knowledge Graph Completion (MMKGC) aims to address the critical issue of missing knowledge in multimodal knowledge graphs (MMKGs) for their better applications. However, both the previous MMGKC and negative sampling (NS) approaches ignore the employment of multimodal information to generate diverse and high-quality negative triples from various semantic levels and hardness levels, thereby limiting the effectiveness of training MMKGC models. Thus, we propose a novel Diffusion-based Hierarchical Negative Sampling (DHNS) scheme tailored for MMKGC tasks, which tackles the challenge of generating high-quality negative triples by leveraging a Diffusion-based Hierarchical Embedding Generation (DiffHEG) that progressively conditions on entities and relations as well as multimodal semantics. Furthermore, we develop a Negative Triple-Adaptive Training (NTAT) strategy that dynamically adjusts training margins associated with the hardness level of the synthesized negative triples, facilitating a more robust and effective learning procedure to distinguish between positive and negative triples. Extensive experiments on three MMKGC benchmark datasets demonstrate that our framework outperforms several state-of-the-art MMKGC models and negative sampling techniques, illustrating the effectiveness of our DHNS for training MMKGC models. The source codes and datasets of this paper are available at https://github.com/ngl567/DHNS.
Figures
Reference graph
Works this paper leans on
-
[38]
Zhang, Y., Chen, Z., Liang, L., Chen, H., Zhang, W.: Unleashing the power of imbalanced modality information for multi-modal knowledge graph completion. In: LREC-COLING. pp. 17120–17130 (2024)
work page 2024
-
[34]
Xu, D., Xu, T., Wu, S., Zhou, J., Chen, E.: Relation-enhanced negative sampling for multimodal knowledge graph completion. In: ACM MM. p. 3857–3866 (2022)
work page 2022
- [1]
-
[2]
Bao, H., Dong, L., Piao, S., Wei, F.: BEiT: BERT pre-training of image transform- ers. In: ICLR (2022)
work page 2022
-
[3]
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: NeurIPS. pp. 2787–2795 (2013)
work page 2013
- [4]
-
[5]
Cao, J., Fang, J., Meng, Z., Liang, S.: Knowledge graph embedding: A survey from the perspective of representation spaces. ACM Comput. Surv.56(6) (2024)
work page 2024
-
[6]
Cao, Z., Xu, Q., Yang, Z., He, Y., Cao, X., Huang, Q.: Otkge: multi-modal knowl- edge graph embeddings via optimal transport. In: NeurIPS (2024)
work page 2024
Show all 41 references
-
[7]
In: ACL-IJCNLP
Chao, L., He, J., Wang, T., Chu, W.: PairRE: Knowledge graph embeddings via paired relation vectors. In: ACL-IJCNLP. pp. 4360–4369 (2021)
2021
-
[8]
F. M. Suchanek, G. Kasneci, G.W.: Yago: A core of semantic knowledge. In: Web Conference. pp. 697–706 (2007)
2007
-
[9]
In: NeurIPS (2020)
Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In: NeurIPS (2020)
2020
-
[10]
In: ACL-IJCNLP
Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: ACL-IJCNLP. pp. 687–696 (2015)
2015
-
[11]
In: Findings of EMNLP
Lee, J., Chung, C., Lee, H., Jo, S., Whang, J.: VISTA: Visual-textual knowledge graph representation learning. In: Findings of EMNLP. pp. 7314–7328 (2023)
2023
-
[12]
Lee, J., Wang, Y., Li, J., Zhang, M.: Multimodal reasoning with multimodal knowl- edge graph. In: ACL. pp. 10767–10782 (2024)
2024
-
[13]
In: Web Conference
Li, Z., Ji, J., Fu, Z., Ge, Y., Xu, S., Chen, C., Zhang, Y.: Efficient non-sampling knowledge graph embedding. In: Web Conference. p. 1727–1736 (2021)
2021
-
[14]
ACM Comput
Liang, W., Meo, P.D., Tang, Y., Zhu, J.: A survey of multi-modal knowledge graphs: Technologies and trends. ACM Comput. Surv.56(11) (2024)
2024
-
[15]
In: Hitzler, P., Fernández, M., Janowicz, K., Zaveri, A., Gray, A.J., Lopez, V., Haller, A., Hammar, K
Liu, Y., Li, H., Garcia-Duran, A., Niepert, M., Onoro-Rubio, D., Rosenblum, D.S.: Mmkg: Multi-modal knowledge graphs. In: Hitzler, P., Fernández, M., Janowicz, K., Zaveri, A., Gray, A.J., Lopez, V., Haller, A., Hammar, K. (eds.) The Semantic Web. pp. 459–474 (2019)
2019
-
[16]
In: AAAI (2024)
Long, X., Zhuang, L., Li, A., Wei, J., Li, H., Wang, S.: Kgdm: A diffusion model to capture multiple relation semantics for knowledge graph embedding. In: AAAI (2024)
2024
-
[17]
Applied Intelligence52, 7480–7497 (2021)
Lu, X., Wang, L., Jiang, Z., He, S., Liu, S.: Mmkrl: A robust embedding approach for multi-modal knowledge graph representation learning. Applied Intelligence52, 7480–7497 (2021)
2021
-
[18]
Applied Intelligence52, 7480 – 7497 (2021)
Lu, X., Wang, L., Jiang, Z., He, S., Liu, S.: Mmkrl: A robust embedding approach for multi-modal knowledge graph representation learning. Applied Intelligence52, 7480 – 7497 (2021)
2021
-
[19]
arXiv preprint arXiv:2402.19195 (2024)
Madushanka, T., Ichise, R.: Negative sampling in knowledge graph representation learning: A review. arXiv preprint arXiv:2402.19195 (2024)
2024 arXiv
-
[20]
Mousselly-Sergieh, H., Botschen, T., Gurevych, I., Roth, S.: A multimodal translation-based approach for knowledge graph representation learning. In: SEM. pp. 225–234 (2018)
2018
-
[21]
In: Web Conference
Nguyen, T.K., Fang, Y.: Diffusion-based negative sampling on graphs for link pre- diction. In: Web Conference. p. 948–958 (2024) 16 G. Niu et al
2024
-
[22]
In: ICML
Nickel, M., Tresp, V., Kriegel, H.P.: A three-way model for collective learning on multi-relational data. In: ICML. pp. 809–816 (2011)
2011
-
[23]
In: EMNLP-IJCNLP
Reimers, N., Gurevych, I.: Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In: EMNLP-IJCNLP. pp. 3982–3992 (2019)
2019
-
[24]
In: AAAI
Shang, B., Zhao, Y., Liu, J., Wang, D.: Lafa: Multimodal knowledge graph com- pletion with link aware fusion and aggregation. In: AAAI. pp. 8957–8965 (2024)
2024
-
[25]
In: ICLR (2019)
Sun, Z., Deng, Z.H., Nie, J.Y., Tang, J.: RotatE: Knowledge graph embedding by relational rotation in complex space. In: ICLR (2019)
2019
-
[26]
Tang, Y., Huang, J., Wang, G., He, X., Zhou, B.: Orthogonal relation transforms with graph context modeling for knowledge graph embedding. In: ACL. pp. 2713– 2722 (2020)
2020
-
[27]
In: ICML
Trouillon, T., Welbl, J., Riedel, S., Éric Gaussier, Bouchard, G.: Complex embed- dings for simple link prediction. In: ICML. pp. 2071–2080 (2016)
2016
-
[28]
Com- mun
Vrandečić, D., Krötzsch, M.: Wikidata: a free collaborative knowledgebase. Com- mun. ACM 57(10), 78–85 (2014)
2014
-
[29]
In: ACM MM
Wang, M., Wang, S., Yang, H., Zhang, Z., Chen, X., Qi, G.: Is visual context really helpful for knowledge graph? a representation learning perspective. In: ACM MM. pp. 2735–2743 (2021)
2021
-
[30]
In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (2018)
Wang, P., Li, S., Pan, R.: Incorporating gan for negative sampling in knowledge representation learning. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (2018)
2018
-
[31]
In: AAAI
Wang,Z.,Zhang,J.,Feng,J.,Chen,Z.:Knowledgegraphembeddingbytranslating on hyperplanes. In: AAAI. pp. 1112–1119 (2014)
2014
-
[32]
In: IJCNN
Wang, Z., Li, L., Li, Q., Zeng, D.: Multimodal data enhanced representation learn- ing for knowledge graphs. In: IJCNN. pp. 1–8 (2019)
2019
-
[33]
In: IJCAI
Xie, R., Liu, Z., Luan, H., Sun, M.: Image-embodied knowledge representation learning. In: IJCAI. pp. 3140–3146 (2017)
2017
-
[35]
In: ICLR (2015)
Yang, B., Yih, W., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. In: ICLR (2015)
2015
-
[36]
In: NeurIPS
Zhang, S., Tay, Y., Yao, L., Liu, Q.: Quaternion knowledge graph embeddings. In: NeurIPS. pp. 2731–2741 (2019)
2019
-
[37]
In: IJCNN
Zhang, Y., Chen, M., Zhang, W.: Modality-aware negative sampling for multi- modal knowledge graph embedding. In: IJCNN. pp. 1–8 (2023)
2023
-
[39]
ArXiv (2022)
Zhang, Y., Zhang, W.: Knowledge graph completion with pre-trained multimodal transformer and twins negative sampling. ArXiv (2022)
2022
-
[40]
In: ICDE
Zhang, Y., Yao, Q., Shao, Y., Chen, L.: Nscaching: Simple and efficient negative sampling for knowledge graph embedding. In: ICDE. pp. 614–625 (2019)
2019
-
[41]
Zhu, X., Li, Z., Wang, X., Jiang, X., Sun, P., Wang, X., Xiao, Y., Yuan, N.J.: Multi-modalknowledgegraphconstructionandapplication:Asurvey.IEEETKDE 36(2), 715–735 (2024)
2024
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.