Pith. sign in

REVIEW 4 major objections 5 minor 41 references

Diffusion-based Hierarchical Negative Sampling for Multimodal Knowledge Graph Completion

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read This paper proposes DHNS, a diffusion-based scheme that generates negative triples for multimodal knowledge graph completion, and reports it outperforms state-of-the-art baselines on three benchmarks.

desk verdict A useful new combination—diffusion-based negative sampling for multimodal KGC—but the experimental reporting is too thin to fully support the central claim. read the letter →

arxiv 2501.15393 v1 pith:4N6ZAPSW submitted 2025-01-26 cs.AI cs.CL

classification cs.AIcs.CL
keywords multimodalknowledgegraphcompletionnegativesamplingdiffusionmodeldenoisingprobabilistichierarchicalembeddinggenerationhardness-adaptivelossMMKGC
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes DHNS, a negative-sampling scheme for multimodal knowledge graph completion (MMKGC) that replaces the usual step of sampling a corrupted entity with a diffusion model that generates new entity embeddings conditioned on the head entity, relation, and visual, textual, and structural features. The generated negatives are organized by hardness level, controlled by the diffusion time step, and the training loss adapts its margin to that hardness. The authors claim that this yields higher-quality and more diverse negative triples than random, adversarial, and prior multimodal negative-sampling strategies, and report consistent improvements over state-of-the-art baselines on DB15K, MKG-W, and MKG-Y. If true, DHNS is a pluggable training component that could improve any MMKGC model without changing its score function.

What carries the argument

The load-bearing machinery is DiffHEG (Diffusion-based Hierarchical Embedding Generation), a denoising diffusion probabilistic model that reverses a forward noising Markov chain to produce synthetic entity embeddings. Conditioning is multimodal, combining structural, visual, and textual features of the head entity and the relation through one of three interaction mechanisms (Hadamard, bilinear, or additive) matched to the KGE model. Hardness is controlled by the reverse-diffusion time step $t$, with hardness level $\mathrm{HL}(\hat{x}_t) \propto 1/t$, so early steps give near-positive "hard" negatives. These generated negatives are consumed by NTAT (Negative Triple-Adaptive Training), whose Hardness-Adaptive Loss uses a margin $\gamma_t$ that shrinks as the negative gets harder, and whose multimodal joint scoring averages the KGE score over the three modality-specific generated embeddings.

What would settle it

One decisive check is to score every generated negative triple against the KG: if a large share of generated negatives at small time steps are either already true triples or lie far outside the convex hull of real entity embeddings, the training signal is corrupted and the claimed advantage should shrink or vanish.

Watch

Extended reading notes

Core claim

The central claim is that negative triples for MMKGC should be generated, not sampled: a conditional denoising diffusion model (DiffHEG) starts from pure Gaussian noise and, guided by the head entity and relation embeddings plus structural, textual, and visual features, produces synthetic tail-entity embeddings. Time steps $T/20$, $T/10$, $T/5$, and $T/2$ yield negatives of increasing easiness, with hardness inversely proportional to the time step. These synthesized negatives are scored by the KGE model's own score function through multimodal joint scoring, and a Hardness-Adaptive Loss assigns smaller margins to harder negatives. The paper argues that direct generation combined with adaptive margins gives a more robust training signal than sampling-based approaches, and reports that DHNS integrated with RotatE achieves the best or second-best MRR, Hits@1, Hits@3, and Hits@10 across DB15K, MKG-W, and MKG-Y.

Load-bearing premise

The whole scheme rests on the assumption that a random vector denoised under the condition of a head entity and relation lands on a plausible entity embedding that the scoring model can treat as a sensible negative, even though nothing explicitly checks that the generated vector resembles a real entity or is not a true triple.

Editorial extensions

If this is right

  • DHNS can be plugged into existing KGE models such as TransE, DistMult, and RotatE to improve their MMKGC accuracy without altering their score functions.
  • Because the diffusion process can generate negatives of any hardness by choosing the time step, training can be balanced across easy and hard negatives instead of relying on a single fixed margin.
  • The margin adaptation in the Hardness-Adaptive Loss implies that the same KGE model can be trained more effectively on a mixture of negative difficulties, reducing sensitivity to the global margin hyperparameter.
  • The ablation results indicate that the DiffHEG generation module, rather than the training strategy, is the main source of gain; removing it causes the largest performance drops across all three datasets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension would be to filter generated negatives that already exist as true triples in the KG, or to project generated embeddings onto the nearest real entity embedding; the paper does not report whether such false negatives occur and how often.
  • Because hardness is tied to the diffusion time step, the same generator could support curriculum learning, starting training with easy negatives and gradually increasing difficulty, which the paper does not explore.
  • The conditioning mechanisms are matched to the algebraic structure of the base KGE model, suggesting a general recipe: the interaction used for conditioning should align with the score function's composition rule, a principle the paper illustrates but does not state explicitly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes DHNS, a diffusion-based negative sampling scheme for multimodal knowledge graph completion. DHNS trains a conditional diffusion model (DiffHEG) to generate tail (or head) entity embeddings from Gaussian noise conditioned on the counterpart entity, relation, and multimodal features; hardness of the generated negative triples is controlled by stopping the reverse diffusion at different time steps. The generated negatives are then used in a Negative Triple-Adaptive Training (NTAT) loss with hardness-adaptive margins. Experiments on DB15K, MKG-W, and MKG-Y report improvements over existing unimodal, multimodal, and negative-sampling baselines, with ablations showing the contribution of each module.

Significance. If the claims are substantiated, DHNS would be a novel and useful contribution to MMKGC: it is, to the best of the paper's claim, the first diffusion-based negative embedding generation approach specific to multimodal knowledge graph completion, and the idea of controlling hardness via diffusion time steps is intuitive. The paper also proposes a promising adaptive-margin training scheme. However, the current manuscript leaves key definitions underspecified and lacks empirical validation under controlled comparisons, so the significance of the work is not yet demonstrated.

major comments (4)
  1. [3.2, Eqs. (4)-(6)] The reverse diffusion update rule in Eqs. (4)-(6) appears to use β_t in the denominator, but β_t is defined as 1 - α_t in Eq. (1). For small t, α_t is typically near 1, so β_t is near 0, making the coefficient 1/√β_t in Eq. (5) extremely large and the update numerically unstable. In the standard DDPM posterior, the coefficient is 1/√α_t or 1/√(1-β_t), where the noise variance is small for small t. Please re-derive the reverse process from the forward process in Eqs. (1)-(2) and correct the equations, or clarify if β_t is intended to denote the noise variance rather than 1 - α_t.
  2. [3.3, Eq. (13)] The hardness-adaptive margin γ_t and the negative weights w(negt) in Eq. (13) are never explicitly defined. The text states that γ_t is adaptive to the hardness level HL(t) and that weights are higher for embeddings closer to the halfway point of the diffusion process, but no functional form, schedule, or update rule is provided. This makes the proposed NTAT loss non-reproducible and prevents independent verification of the adaptive-margin mechanism. Please specify γ_t and w(negt) in closed form or with an exact algorithmic description.
  3. [3.2-3.3, Eqs. (5)-(6), (12)-(13)] The framework never checks whether the generated negative embeddings lie near the manifold of real entity embeddings or whether the composed triples (h, r, neg_t) are already true triples. The generated negatives are arbitrary points in the embedding space, and at small t they are explicitly described as being closer to positives, increasing the risk of false negatives. Consequently, the loss in Eq. (13) may train the KGE model to assign low scores to off-manifold vectors, while inference only ranks real entities, so the improvements reported in Tables 2-4 may not transfer to the actual ranking task. Please provide quantitative evidence about the relation of generated embeddings to real entities (e.g., nearest-neighbor statistics) and report the false-negative rate among generated negatives.
  4. [4.1, Tables 2-4] Baseline results are copied from prior publications ([38] for Tables 2-3 and [34] for Table 4) rather than re-run in the same experimental framework, and no error bars or significance tests are reported anywhere. The claimed consistent improvements over baselines therefore rest on potentially incompatible setups and are not statistically validated. Please re-run the comparisons under identical conditions, or at least report multiple-run statistics (mean and standard deviation) and significance tests for the reported metrics.
minor comments (5)
  1. [Eq. (13)] The symbol t is used both for the tail entity in a triple and for the diffusion time step, causing ambiguity in the loss definition; using e_t or a distinct subscript for the tail entity would clarify the equation.
  2. [3.3] In the opening of Section 3.3, "HTAT" appears to be a typo for "NTAT", the negative triple-adaptive training mechanism.
  3. [3.2, Eq. (7)] The word "Hardmard" should be "Hadamard" in the description of the multiplication operation in Eq. (7).
  4. [Contributions, Section 1] The phrase "As we can be concerned" is awkward and should be rephrased, for example as "To the best of our knowledge".
  5. [4.2, Table 4] The claim that DHNS "consistently and significantly outperforms" other NS strategies is not fully supported by Table 4: for TransE on MKG-W, the H10 of DHNS (46.85) is lower than that of MMRNS (47.11), and several other differences are small. Please either soften the claim or discuss the discrepancies in the text.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: DiffHEG is a trained generative model, not a re-fit of the target metric. One minor self-citation for baseline numbers does not make the central claim circular.

full rationale

The paper's core claim is that DHNS improves MMKGC training by generating hard negative entity embeddings. The generation path is not circular: DiffHEG is trained by the denoising loss in Eq. 11 to reconstruct true tail-entity embeddings from noised versions conditioned on head and relation, and negatives are produced by the reverse process in Eqs. 5-6 at chosen timesteps. Hardness is defined by the timestep (HL ∝ 1/t) rather than by the downstream KGE score, so the hardness-adaptive margin in Eq. 13 is a design choice, not a self-fulfilling prediction. The only self-citation of note is the sourcing of baseline results in Tables 2-3 from [38], a prior paper co-authored by W. Zhang; this is a reporting-fairness caveat, not a circular derivation, because those numbers are published external results and the DHNS method itself is independently specified. The absence of a check that generated embeddings lie on the entity manifold is a soundness/correctness risk, not a circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central method rests on several hand-chosen or undefined quantities (gamma_t, w(negt), lambda, T, time-step set), plus assumptions that denoised embeddings are semantically meaningful entities and that the adaptive margin mapping exists and helps. No new physical entities are introduced.

free parameters (6)
  • Total diffusion time steps T = not reported (tuned from {20, 50, 70, 100})
    Chosen by tuning; affects hardness levels T/20, T/10, T/5, T/2.
  • DiffHEG learning rate = not reported (tuned from {2e-3, 1e-4, 5e-4})
    Tuned for diffusion module; no final value given.
  • Loss trade-off lambda = not reported
    Weight combining L_KGC and L_HA in Eq. 15; value not specified.
  • Hardness-adaptive margin gamma_t = undefined
    Key to HAL loss Eq. 13; described only as adaptive to HL(t) with no functional form.
  • Negative weights w(negt) = undefined
    Used in Eq. 13; only qualitative statement that halfway embeddings get higher weights.
  • Hardness time-step set {T/20, T/10, T/5, T/2} = chosen by hand
    Selected to represent multiple hardness levels; no sensitivity analysis.
assumptions (5)
  • standard math DDPM forward and reverse processes with Gaussian noise are valid for continuous entity embeddings.
    Invoked in Sec. 3.2, Eqs. 1-6.
  • domain assumption Conditional denoising C(x_e, x_r) via Hadamard, bilinear, or addition captures relational semantics for generation.
    Sec. 3.2 Eqs. 7-9; no justification beyond compatibility with KGE score functions.
  • ad hoc to paper Generated embeddings at different diffusion timesteps correspond to semantically meaningful hardness levels, with hardness proportional to 1/t.
    Stated in Sec. 3.2 without empirical or theoretical support.
  • domain assumption The KGE score function E(.) can be applied to generated embeddings as if they were real entity embeddings.
    Used in Eq. 12; no manifold alignment or validation.
  • domain assumption Fixed BEiT and SBERT features are sufficient and informative for the conditional generation.
    Preprocessing inherited from AdaMF and MMRNS (Sec. 4.1), not re-evaluated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Diffusion-based Hierarchical Negative Sampling for Multimodal Knowledge Graph Completion." pith.science (2026). https://pith.science/paper/4N6ZAPSW

@misc{pith2026250115393,
  author       = {Pith},
  title        = {Pith review of: Diffusion-based Hierarchical Negative Sampling for Multimodal Knowledge Graph Completion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4N6ZAPSW}},
  note         = {Machine review of arXiv:2501.15393}
}
read the original abstract

Multimodal Knowledge Graph Completion (MMKGC) aims to address the critical issue of missing knowledge in multimodal knowledge graphs (MMKGs) for their better applications. However, both the previous MMGKC and negative sampling (NS) approaches ignore the employment of multimodal information to generate diverse and high-quality negative triples from various semantic levels and hardness levels, thereby limiting the effectiveness of training MMKGC models. Thus, we propose a novel Diffusion-based Hierarchical Negative Sampling (DHNS) scheme tailored for MMKGC tasks, which tackles the challenge of generating high-quality negative triples by leveraging a Diffusion-based Hierarchical Embedding Generation (DiffHEG) that progressively conditions on entities and relations as well as multimodal semantics. Furthermore, we develop a Negative Triple-Adaptive Training (NTAT) strategy that dynamically adjusts training margins associated with the hardness level of the synthesized negative triples, facilitating a more robust and effective learning procedure to distinguish between positive and negative triples. Extensive experiments on three MMKGC benchmark datasets demonstrate that our framework outperforms several state-of-the-art MMKGC models and negative sampling techniques, illustrating the effectiveness of our DHNS for training MMKGC models. The source codes and datasets of this paper are available at https://github.com/ngl567/DHNS.

Figures

Figures reproduced from arXiv: 2501.15393 by the authors.

Figure 1
Figure 1. The whole framework of our DHNS. MHLD means multiple hardness-level denoising. x0:T and xˆ0:T are the noised and the denoised embeddings in the range of time steps [0, T] corresponding to an entity. x struc T /20 , x text T /20 and x vis T /20 are three modality￾specific (structural/textual/visual) denoised embeddings at the time step T /20. γT /20 denotes the margin adaptive to the negative triples with the hardnes… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 39 canonical work pages

  1. [38]

    In: LREC-COLING

    Zhang, Y., Chen, Z., Liang, L., Chen, H., Zhang, W.: Unleashing the power of imbalanced modality information for multi-modal knowledge graph completion. In: LREC-COLING. pp. 17120–17130 (2024)

  2. [34]

    In: ACM MM

    Xu, D., Xu, T., Wu, S., Zhou, J., Chen, E.: Relation-enhanced negative sampling for multimodal knowledge graph completion. In: ACM MM. p. 3857–3866 (2022)

  3. [1]

    In: EMNLP

    Ahrabian, K., Feizi, A., Salehi, Y., Hamilton, W.L., Bose, A.J.: Structure aware negative sampling in knowledge graphs. In: EMNLP. pp. 6093–6101 (2020)

  4. [2]

    In: ICLR (2022)

    Bao, H., Dong, L., Piao, S., Wei, F.: BEiT: BERT pre-training of image transform- ers. In: ICLR (2022)

  5. [3]

    In: NeurIPS

    Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: NeurIPS. pp. 2787–2795 (2013)

  6. [4]

    In: NAACL

    Cai, L., Wang, W.Y.: KBGAN: Adversarial learning for knowledge graph embed- dings. In: NAACL. pp. 1470–1480 (2018)

  7. [5]

    ACM Comput

    Cao, J., Fang, J., Meng, Z., Liang, S.: Knowledge graph embedding: A survey from the perspective of representation spaces. ACM Comput. Surv.56(6) (2024)

  8. [6]

    In: NeurIPS (2024)

    Cao, Z., Xu, Q., Yang, Z., He, Y., Cao, X., Huang, Q.: Otkge: multi-modal knowl- edge graph embeddings via optimal transport. In: NeurIPS (2024)

Show all 41 references
  1. [7]

    In: ACL-IJCNLP

    Chao, L., He, J., Wang, T., Chu, W.: PairRE: Knowledge graph embeddings via paired relation vectors. In: ACL-IJCNLP. pp. 4360–4369 (2021)

  2. [8]

    F. M. Suchanek, G. Kasneci, G.W.: Yago: A core of semantic knowledge. In: Web Conference. pp. 697–706 (2007)

  3. [9]

    In: NeurIPS (2020)

    Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In: NeurIPS (2020)

  4. [10]

    In: ACL-IJCNLP

    Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: ACL-IJCNLP. pp. 687–696 (2015)

  5. [11]

    In: Findings of EMNLP

    Lee, J., Chung, C., Lee, H., Jo, S., Whang, J.: VISTA: Visual-textual knowledge graph representation learning. In: Findings of EMNLP. pp. 7314–7328 (2023)

  6. [12]

    Lee, J., Wang, Y., Li, J., Zhang, M.: Multimodal reasoning with multimodal knowl- edge graph. In: ACL. pp. 10767–10782 (2024)

  7. [13]

    In: Web Conference

    Li, Z., Ji, J., Fu, Z., Ge, Y., Xu, S., Chen, C., Zhang, Y.: Efficient non-sampling knowledge graph embedding. In: Web Conference. p. 1727–1736 (2021)

  8. [14]

    ACM Comput

    Liang, W., Meo, P.D., Tang, Y., Zhu, J.: A survey of multi-modal knowledge graphs: Technologies and trends. ACM Comput. Surv.56(11) (2024)

  9. [15]

    In: Hitzler, P., Fernández, M., Janowicz, K., Zaveri, A., Gray, A.J., Lopez, V., Haller, A., Hammar, K

    Liu, Y., Li, H., Garcia-Duran, A., Niepert, M., Onoro-Rubio, D., Rosenblum, D.S.: Mmkg: Multi-modal knowledge graphs. In: Hitzler, P., Fernández, M., Janowicz, K., Zaveri, A., Gray, A.J., Lopez, V., Haller, A., Hammar, K. (eds.) The Semantic Web. pp. 459–474 (2019)

  10. [16]

    In: AAAI (2024)

    Long, X., Zhuang, L., Li, A., Wei, J., Li, H., Wang, S.: Kgdm: A diffusion model to capture multiple relation semantics for knowledge graph embedding. In: AAAI (2024)

  11. [17]

    Applied Intelligence52, 7480–7497 (2021)

    Lu, X., Wang, L., Jiang, Z., He, S., Liu, S.: Mmkrl: A robust embedding approach for multi-modal knowledge graph representation learning. Applied Intelligence52, 7480–7497 (2021)

  12. [18]

    Applied Intelligence52, 7480 – 7497 (2021)

    Lu, X., Wang, L., Jiang, Z., He, S., Liu, S.: Mmkrl: A robust embedding approach for multi-modal knowledge graph representation learning. Applied Intelligence52, 7480 – 7497 (2021)

  13. [19]

    arXiv preprint arXiv:2402.19195 (2024)

    Madushanka, T., Ichise, R.: Negative sampling in knowledge graph representation learning: A review. arXiv preprint arXiv:2402.19195 (2024)

  14. [20]

    Mousselly-Sergieh, H., Botschen, T., Gurevych, I., Roth, S.: A multimodal translation-based approach for knowledge graph representation learning. In: SEM. pp. 225–234 (2018)

  15. [21]

    In: Web Conference

    Nguyen, T.K., Fang, Y.: Diffusion-based negative sampling on graphs for link pre- diction. In: Web Conference. p. 948–958 (2024) 16 G. Niu et al

  16. [22]

    In: ICML

    Nickel, M., Tresp, V., Kriegel, H.P.: A three-way model for collective learning on multi-relational data. In: ICML. pp. 809–816 (2011)

  17. [23]

    In: EMNLP-IJCNLP

    Reimers, N., Gurevych, I.: Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In: EMNLP-IJCNLP. pp. 3982–3992 (2019)

  18. [24]

    In: AAAI

    Shang, B., Zhao, Y., Liu, J., Wang, D.: Lafa: Multimodal knowledge graph com- pletion with link aware fusion and aggregation. In: AAAI. pp. 8957–8965 (2024)

  19. [25]

    In: ICLR (2019)

    Sun, Z., Deng, Z.H., Nie, J.Y., Tang, J.: RotatE: Knowledge graph embedding by relational rotation in complex space. In: ICLR (2019)

  20. [26]

    Tang, Y., Huang, J., Wang, G., He, X., Zhou, B.: Orthogonal relation transforms with graph context modeling for knowledge graph embedding. In: ACL. pp. 2713– 2722 (2020)

  21. [27]

    In: ICML

    Trouillon, T., Welbl, J., Riedel, S., Éric Gaussier, Bouchard, G.: Complex embed- dings for simple link prediction. In: ICML. pp. 2071–2080 (2016)

  22. [28]

    Com- mun

    Vrandečić, D., Krötzsch, M.: Wikidata: a free collaborative knowledgebase. Com- mun. ACM 57(10), 78–85 (2014)

  23. [29]

    In: ACM MM

    Wang, M., Wang, S., Yang, H., Zhang, Z., Chen, X., Qi, G.: Is visual context really helpful for knowledge graph? a representation learning perspective. In: ACM MM. pp. 2735–2743 (2021)

  24. [30]

    In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (2018)

    Wang, P., Li, S., Pan, R.: Incorporating gan for negative sampling in knowledge representation learning. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (2018)

  25. [31]

    In: AAAI

    Wang,Z.,Zhang,J.,Feng,J.,Chen,Z.:Knowledgegraphembeddingbytranslating on hyperplanes. In: AAAI. pp. 1112–1119 (2014)

  26. [32]

    In: IJCNN

    Wang, Z., Li, L., Li, Q., Zeng, D.: Multimodal data enhanced representation learn- ing for knowledge graphs. In: IJCNN. pp. 1–8 (2019)

  27. [33]

    In: IJCAI

    Xie, R., Liu, Z., Luan, H., Sun, M.: Image-embodied knowledge representation learning. In: IJCAI. pp. 3140–3146 (2017)

  28. [35]

    In: ICLR (2015)

    Yang, B., Yih, W., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. In: ICLR (2015)

  29. [36]

    In: NeurIPS

    Zhang, S., Tay, Y., Yao, L., Liu, Q.: Quaternion knowledge graph embeddings. In: NeurIPS. pp. 2731–2741 (2019)

  30. [37]

    In: IJCNN

    Zhang, Y., Chen, M., Zhang, W.: Modality-aware negative sampling for multi- modal knowledge graph embedding. In: IJCNN. pp. 1–8 (2023)

  31. [39]

    ArXiv (2022)

    Zhang, Y., Zhang, W.: Knowledge graph completion with pre-trained multimodal transformer and twins negative sampling. ArXiv (2022)

  32. [40]

    In: ICDE

    Zhang, Y., Yao, Q., Shao, Y., Chen, L.: Nscaching: Simple and efficient negative sampling for knowledge graph embedding. In: ICDE. pp. 614–625 (2019)

  33. [41]

    Zhu, X., Li, Z., Wang, X., Jiang, X., Sun, P., Wang, X., Xiao, Y., Yuan, N.J.: Multi-modalknowledgegraphconstructionandapplication:Asurvey.IEEETKDE 36(2), 715–735 (2024)

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.