Pith. sign in

REVIEW 5 major objections 8 minor 100 references

KG-BiLM: Knowledge Graph Embedding via Bidirectional Language Models

T0 review · 5 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read KG-BiLM claims a bidirectional graph-aware decoder achieves the best link prediction on Wikidata5M and FB15k-237N among the comparisons it reports.

desk verdict A coherent KG-LM hybrid with plausible SOTA numbers, but the evaluation protocol—triple scoring, filtered/raw ranking, zero-shot encoding—is unspecified, so the headline margins rest on an assumption. read the letter →

arxiv 2506.03576 v1 pith:YQZYEBB3 submitted 2025-06-04 cs.CL cs.AI

Zirui Chen , Xin Wang , Zhao Li , Wenbin Guo , Dongxiao He This is my paper
classification cs.CLcs.AI
keywords knowledgegraphembeddingbidirectionalattentionlinkpredictionzero-shotreasoningcontrastivelearningmaskedlanguagemodelingWikidata5Mdecoder-onlytransformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

KG-BiLM argues that the gap between knowledge-graph reasoning and language-model semantics can be closed by turning a decoder-only transformer into a bidirectional, graph-aware reader. Instead of the usual causal mask, it uses a mask that lets any token attend to any other token inside a local window or connected in the knowledge graph within two hops. On top of that it adds masked prediction of entities and text tokens, with the loss shifted to the preceding position, plus a contrastive loss that aligns two corrupted views of the same sub-graph. The paper reports that this combination achieves the best link-prediction numbers it compares against on Wikidata5M and FB15k-237N, including a strong zero-shot result on Wikidata5M, while remaining competitive on structure-only WN18RR.

What carries the argument

The load-bearing object is the graph-aware attention mask $\mathbf{M}_{\mathrm{BKA}}$ inside a decoder transformer. For each ordered token pair, the mask permits attention when the tokens are within a small positional window or when their parent entities are connected by a path of at most two hops, and sets every other pair to negative infinity. This one mechanism carries most of the argument: it gives the masked-prediction objective genuine bidirectional context, and it provides the contrastive module with graph-consistent views of the same sub-graph.

What would settle it

Run the Wikidata5M evaluation with an explicitly written scoring rule, such as ranking each candidate tail by the softmax probability of the masked tail token from the preceding hidden state under filtered ranking, and check whether KG-BiLM still reports MRR 40.3 and Hits@1 39.7; separately, verify that in the zero-shot split unseen entities receive embeddings from their descriptions alone and not from a learned lookup table.

Watch

Extended reading notes

Core claim

The central claim is that a decoder-only transformer can encode knowledge-graph structure and textual semantics at the same time by changing what attention is allowed to see. Specifically, the Bidirectional Knowledge Attention mask sets an attention entry to zero when tokens are textually local or when their parent entities lie within a bounded hop distance, so hidden states at every layer mix past, future, and graph-neighbour information. Knowledge-Masked Prediction then trains the model to recover masked entity and word tokens from the preceding hidden state, forcing inference about the current position to flow through context that includes later tokens and linked entities. Contrastive Graph Semantic Aggregation adds an InfoNCE term over dropout-corrupted views of the same sub-graph, which the paper says preserves cluster structure for rare entities and sharpens embedding boundaries. On the reported numbers, the model reaches MRR 40.3 and Hits@1 39.7 on Wikidata5M versus 39.6 and 37.3 for ReSKGC, leads FB15k-237N with MRR 37.8, and achieves MRR 74.8 in the Wikidata5M zero-shot split; on WN18RR its MRR of 68.2 is the best among the listed baselines.

Load-bearing premise

The reported quality comparisons assume that KG-BiLM is scored under the same evaluation protocol as the baselines, but the paper never specifies how token-level embeddings are turned into a triple score, whether ranks are filtered, or how an unseen entity is embedded in the zero-shot split.

Editorial extensions

If this is right

  • If the claims hold, knowledge-graph completion can be built on a single decoder backbone rather than a symbolic KGE encoder plus a separate text model, with the graph-aware mask as the main structural change.
  • Unseen entities can be scored from their textual descriptions alone, which is exactly what the reported zero-shot Wikidata5M result is testing.
  • The ablation pattern indicates the graph-aware mask is the largest contributor to the gain, since removing it costs 2.0 MRR on Wikidata5M.
  • The same framework can extend to generation and reasoning tasks, provided the paper's assertion that bidirectional knowledge attention preserves autoregressive decoding is verified.
  • On structure-only graphs with many relations, the paper itself flags that gains over specialized path-based models remain limited, so the approach is most clearly useful when text and graph are both available.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit is that a two-hop mask in every layer behaves like a shallow graph propagation folded into each transformer layer, so information can travel along very long paths over 24 layers; sweeping the hop threshold would directly test whether deeper or wider masks add more multi-hop reasoning power.
  • The zero-shot result deserves a closer look because the paper does not say how an unseen entity token is embedded at test time; a clean extension would compare description-only initialization against oracle entity embeddings to isolate true generalization.
  • Because the paper's reported margin over the previous Wikidata5M state of the art is about 0.7 MRR points, the comparison is only meaningful under an identical ranking protocol; reproducing the baseline's filtered protocol is the cheapest way to confirm the ordering.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. KG-BiLM proposes a decoder-style transformer for knowledge graph embeddings that replaces the causal attention mask with a graph-aware bidirectional mask (BKA), trains masked prediction of entities and tokens with a position-shifted loss (KMP), and adds a contrastive objective over sampled graph/text views (CGSA). The paper reports state-of-the-art link prediction results on Wikidata5M (MRR 40.3, Hits@1 39.7), FB15k-237N (MRR 37.8), and zero-shot Wikidata5M (MRR 74.8), while being competitive on WN18RR and on par with recent transformer baselines on FB15k-237. The main contribution is empirical; the three modules are described algorithmically and an open-source code link is provided.

Significance. If the reported numbers are correct under protocols comparable to the baselines, the paper would make a useful empirical contribution: it demonstrates that a decoder-only transformer can be adapted to bidirectional graph reasoning with relatively simple mask and objective changes, and the ablation study in Table 4 gives each component credit. The zero-shot result is particularly interesting, since it claims strong generalization to unseen entities. However, the paper is entirely empirical and the evaluation protocol is not fully specified, so the headline SOTA margins of 0.7 MRR on Wikidata5M and 3.4 MRR in zero-shot cannot currently be verified. The strengths are the clear architecture description, the inclusion of ablations, and the availability of source code; the weakness is that the missing scoring/ranking specification and the unclear zero-shot encoding procedure block verification of the central claim.

major comments (5)
  1. [§3.4, §4.1] The triple scoring function and ranking protocol are never specified. Equation (10) defines a per-position softmax over V∪E for masked prediction, but link prediction requires a scalar score for a triple (h,r,t) to rank candidate tails, and the paper does not state whether this score is a masked-token logit, a pooled embedding dot product, or another function. It is also not stated whether ranks are filtered (removing other true triples) or raw, nor how the negative candidate set is constructed (all 4.5M entities versus a sampled subset). Since the claimed Wikidata5M margin over ReSKGC is only 0.7 MRR, a protocol difference of this kind could account for the reported improvement. Please specify the exact procedure and confirm that it matches the protocol used for the baselines.
  2. [§4.5, §3.5] The zero-shot evaluation procedure is not defined. The output projection in Equation (9) is over V∪E, so an entity unseen during training has no output symbol; Section 4.5 says that every zero-shot test triple contains an unseen entity but does not describe how such an entity is encoded from its description at inference time. CGSA, as described in Section 3.5, is a training-time contrastive objective, not an inference-time encoder for novel entities. Please provide the exact inference-time mechanism (for example, encoding the description and scoring the candidate by similarity) and state whether the same mechanism is used for the zero-shot baselines.
  3. [§4.3, §4.4] Table 4 is labelled "Ablation on validation split," yet its Full-model row for Wikidata5M (MRR 40.3, Hits@10 45.2) is identical to the corresponding row in Table 3, which is presented as the benchmark result. If Table 3 reports test-set numbers, the equality needs explanation; if it actually reports validation numbers, then the comparison against published baselines is not on the standard test split. Please clarify which split is used in Tables 3 and 5 and align the protocol with the baseline papers.
  4. [Appendix E, §4.1] The initialization of the 24-layer decoder is not disclosed. It is not stated whether the model is initialized from a pretrained checkpoint (and which one) or trained from scratch. This materially affects both reproducibility and interpretation, since a pretrained language model would contribute substantial semantic knowledge independent of the proposed KG-specific modules.
  5. [Checklist Q7, §4] No error bars or multiple-seed variance are reported for any result, as admitted in the NeurIPS checklist (Question 7). Given that the main SOTA margins are 0.7 MRR points on Wikidata5M and 3.4 MRR points in the zero-shot setting, run-to-run variation may be comparable to the reported gains. Please report at least three seeds, or confidence intervals, for the core comparisons in Tables 3 and 5.
minor comments (8)
  1. [Abstract] The abstract contains a typo: "KG-BiLM2" should read "KG-BiLM."
  2. [§4.4] The text refers to "Ablation results (Section 4.5)", but the ablation study is in Section 4.4; the reference should be corrected.
  3. [Appendix C.1.3] The claim that BKA attention coefficients "can be shown to approximate a truncated power series of the KG adjacency matrix" is stated without proof or a precise statement; it should either be proven in an appendix or softened to an intuition.
  4. [§3.3, Appendix C.1.2] The CanInteract helper depends on a local window parameter δ, but δ is not defined in the main text or listed in the hyperparameter settings in Appendix E; please add it to the implementation details.
  5. [Appendix D, Tables 7 and 8] The RoBERTa baseline is inconsistent across tables: Table 7 reports MRR 7.4, Hits@1 0.7 on Wikidata5M, while Table 8 reports MRR 0.1, Hits@1 0, Hits@3 0.1 for the same model and dataset. Please correct or explain the discrepancy.
  6. [§4.6, Figure 3] The qualitative claims about t-SNE clusters and silhouette scores are not accompanied by quantitative values or a description of how the figure was produced; please report the silhouette score and specify the subset of entities shown.
  7. [Figure 1] The caption includes "Fréchet Inception Distance" as a label, which appears unrelated to the knowledge-representation failure modes being illustrated; this label should be removed or replaced.
  8. [NeurIPS Checklist Q4 and Q5] The checklist answers "Yes" to full disclosure of experimental information and reproducibility, but Sections 4 and Appendix E omit the scoring/ranking protocol and the zero-shot encoding procedure; the checklist answers should be revised or the missing details added.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: KG-BiLM is an empirical architecture paper whose claims are judged against external baselines and module ablations, not against its own fitted inputs.

full rationale

I walked the derivation chain of KG-BiLM and found no step in which a claimed prediction is defined in terms of the quantity it is supposed to predict, nor any load-bearing self-citation. The three technical components are architectural and objective-level proposals: BKA (Eqs. 2-3) changes the attention mask using graph adjacency; KMP (Eqs. 8-10) is a masked-prediction training loss; CGSA (Eqs. 11-12) is a standard InfoNCE contrastive loss. None of these is fitted to the reported MRR/Hits@K numbers and then renamed as a prediction. The main claims are empirical comparisons against independently published baselines (SimKGC, ReSKGC, CSPromp-KG, CD, etc.), and Table 4 ablates each proposed module against the full model, which is non-circular practice. I also checked the self-citation patterns enumerated in the rubric: the reference list contains no prior work by the present authors that is invoked as a uniqueness theorem or as justification for the core architecture, so there is no self-citation chain to import authority. The paper's own limitation statement (Appendix F) concedes that FB15k-237 performance trails NBFNet, which is an honest external check rather than a circular justification. The skeptic's concern that the triple scoring function, filtered/raw ranking, candidate generation, and zero-shot entity encoding protocol are unspecified is a reproducibility and correctness risk, not a circularity reduction: an undefined evaluation protocol can make numbers unverifiable, but it does not make the derivation equivalent to its inputs by construction. The NeurIPS checklist's 'Yes' reproducibility claims are likewise evidence-quality concerns that do not constitute the specific Eq.-to-Eq. or parameter-to-prediction reduction required to flag circularity. Therefore the appropriate finding is no significant circularity, score 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The model's central claims rest on hand-chosen hyperparameters (hop threshold, masking ratio, temperature, and the unstated local window delta), on the accuracy and protocol-compatibility of transcribed baselines, and on the assumption that textual descriptions alone can embed zero-shot entities. No theoretical derivation is given, so the ledger is dominated by evaluation-protocol assumptions rather than free mathematical constants.

free parameters (4)
  • hop threshold h for BKA mask = 2
    Chosen by hand; controls which entity pairs may attend to each other, which is the core structural mechanism of the model (Eq. 2, Section 3.3, Appendix E).
  • masking ratio gamma for KMP = reported as '15' in Appendix E, likely a typo for 0.15
    Controls the difficulty of the masked-prediction task; the stated value is illegible as written and the sentence is cut off.
  • temperature tau for CGSA InfoNCE = 0.07
    Standard contrastive temperature, chosen by hand; affects cluster sharpness in Eq. 12.
  • local window delta in CanInteract = unspecified
    Appendix C.1.2 defines condition (1) |i-j| <= delta for local syntax attention, but the value of delta is never given.
assumptions (5)
  • domain assumption Entity descriptions in Wikidata5M and FB15k-237N contain sufficient linguistic signal to embed unseen entities purely from text in the zero-shot setting
    Section 3.4's 'Link to Zero-Shot Capability' and Section 4.5 assume the KMP-trained encoder produces usable embeddings for unseen entities from descriptions; no failure analysis is given.
  • domain assumption Transcribed baseline results are accurate and use the same evaluation protocol as the reported KG-BiLM numbers
    All SOTA claims in Tables 2 and 3 rely on cross-paper comparisons, but the paper never states its filtering or candidate-corruption protocol, so protocol alignment is assumed.
  • domain assumption BKA with hop threshold 2 preserves autoregressive decoding capability
    Appendix C.1.3 asserts the model can still generate text because only KG neighbors are visible at inference, but the bidirectional training mask and the generation-time mask are never reconciled in a single formulation.
  • ad hoc to paper BKA self-attention approximates a truncated power series of the KG adjacency matrix
    Section C.1.3 asserts this 'can be shown' without a proof or citation; it is used to justify multi-hop reasoning capability.
  • standard math InfoNCE loss maximizes a lower bound on mutual information between augmented views
    Appendix C.3 invokes the standard InfoNCE property from reference [36]; this is accepted background.

how reviews work

0 comments
Cite this review

Pith. "Pith review of KG-BiLM: Knowledge Graph Embedding via Bidirectional Language Models." pith.science (2026). https://pith.science/paper/YQZYEBB3

@misc{pith2026250603576,
  author       = {Pith},
  title        = {Pith review of: KG-BiLM: Knowledge Graph Embedding via Bidirectional Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YQZYEBB3}},
  note         = {Machine review of arXiv:2506.03576}
}
read the original abstract

Recent advances in knowledge representation learning (KRL) highlight the urgent necessity to unify symbolic knowledge graphs (KGs) with language models (LMs) for richer semantic understanding. However, existing approaches typically prioritize either graph structure or textual semantics, leaving a gap: a unified framework that simultaneously captures global KG connectivity, nuanced linguistic context, and discriminative reasoning semantics. To bridge this gap, we introduce KG-BiLM, a bidirectional LM framework that fuses structural cues from KGs with the semantic expressiveness of generative transformers. KG-BiLM incorporates three key components: (i) Bidirectional Knowledge Attention, which removes the causal mask to enable full interaction among all tokens and entities; (ii) Knowledge-Masked Prediction, which encourages the model to leverage both local semantic contexts and global graph connectivity; and (iii) Contrastive Graph Semantic Aggregation, which preserves KG structure via contrastive alignment of sampled sub-graph representations. Extensive experiments on standard benchmarks demonstrate that KG-BiLM outperforms strong baselines in link prediction, especially on large-scale graphs with complex multi-hop relations - validating its effectiveness in unifying structural information and textual semantics.

Figures

Figures reproduced from arXiv: 2506.03576 by the authors.

Figure 1
Figure 1. Illustration of complementary failure modes in KRL models. (a) Translation-based [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the KG-BiLM Architecture. employ fully bidirectional attention but often lack the generative expressivity required for open￾ended inference. Decoder-based transformers, by contrast, use a causal mask that blocks access to future tokens, making it difficult to capture symmetric relations or long-range entity co-occurrences. Consequently, a refined attention scheme is required to bridge these limitations. … view at source ↗
Figure 3
Figure 3. Results of Entity Embedding Clusters and Knowledge-Attention Heatmap. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

100 extracted references · 45 canonical work pages

  1. [1]

    Jeff Z. Pan, S. Razniewski, and Jan-Christoph Kalo et al. Large language models and knowledge graphs: Opportunities and challenges. TGDK, 2023. doi: 10.48550/arXiv.2308.06374

    arXiv doi:10.48550/arxiv.2308.06374 2023

  2. [2]

    Advancing graph representation learning with large language models: A comprehensive survey of techniques

    Qiheng Mao, Zemin Liu, and Chenghao Liu et al. Advancing graph representation learning with large language models: A comprehensive survey of techniques. ArXiv, 2024. doi: 10.48550/arXiv.2402.05952

    arXiv doi:10.48550/arxiv.2402.05952 2024

  3. [3]

    Hwang, Chandra Bhagavatula, and Ronan Le Bras et al

    Jena D. Hwang, Chandra Bhagavatula, and Ronan Le Bras et al. Comet-atomic 2020: On symbolic and neural commonsense knowledge graphs. AAAI, 2021. doi: 10.1609/aaai.v35i7. 16792

    doi:10.1609/aaai.v35i7 2020

  4. [4]

    Cambria, Shaoxiong Ji, and Shirui Pan et al

    E. Cambria, Shaoxiong Ji, and Shirui Pan et al. Knowledge graph representation and reasoning. Neurocomputing, 2021. doi: 10.1016/j.neucom.2021.05.101

    work page doi:10.1016/j.neucom.2021.05.101 2021

  5. [5]

    Cambria et al

    Shaoxiong Ji, Shirui Pan, and E. Cambria et al. A survey on knowledge graphs: Representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems,

  6. [6]

    Improving knowledge graph representation learning by structure contextual pre-training

    Ganqiang Ye, Wen Zhang, and Zhen Bi et al. Improving knowledge graph representation learning by structure contextual pre-training. IJCNN Proceedings, 2021. doi: 10.1145/3502223.3502244

    arXiv 2021

  7. [7]

    Trajanoska, Riste Stojanov, and D

    M. Trajanoska, Riste Stojanov, and D. Trajanov. Enhancing knowledge graph construction using large language models. ArXiv, 2023. doi: 10.48550/arXiv.2305.04676

    arXiv doi:10.48550/arxiv.2305.04676 2023

  8. [8]

    Unifying large language models and knowledge graphs: A roadmap

    Shirui Pan, Linhao Luo, and Yufei Wang et al. Unifying large language models and knowledge graphs: A roadmap. ArXiv, 2023. doi: 10.48550/arXiv.2306.08302

    arXiv doi:10.48550/arxiv.2306.08302 2023

Show all 100 references
  1. [9]

    A survey on knowledge graph embedding

    Mohan Li Guanqun Qu Yang Xiao Qi Yan, Jiaxin Fan. A survey on knowledge graph embedding. IEEE International Conference on Data Science in Cyberspace , pages 576–583, 2022. doi: 10.1109/DSC55868.2022.00086

    2022

  2. [11]

    Mem-kgc: Masked entity model for knowledge graph completion with pre-trained language model

    Youngjoong Ko Bonggeun Choi, Daesik Jang. Mem-kgc: Masked entity model for knowledge graph completion with pre-trained language model. IEEE Access, 9:132025–132032, 2021. doi: 10.1109/ACCESS.2021.3113329

    2021

  3. [12]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  4. [14]

    Translating embeddings for modeling multi-relational data

    Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. Advances in neural information processing systems, 26, 2013

    2013

  5. [16]

    Translating embeddings for modeling multi-relational data

    Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, Ad- vances in Neural Information Processing...

  6. [17]

    Embedding entities and relations for learning and inference in knowledge bases, 2015

    Bishan Yang, Wen tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases, 2015. URL https://arxiv.org/ abs/1412.6575

    2015 arXiv

  7. [18]

    Learning entity and relation embeddings for knowledge graph completion

    Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation embeddings for knowledge graph completion. Proceedings of the AAAI Conference on Artificial Intelligence , 29(1), Feb. 2015. doi: 10.1609/aaai.v29i1.9491. URL https: //ojs.aaai.org/index...

    2015 doi

  8. [19]

    Complex embeddings for simple link prediction

    Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ICML’16, page 2071–2080. JML...

    2016

  9. [20]

    Holographic embeddings of knowledge graphs

    Maximilian Nickel, Lorenzo Rosasco, and Tomaso Poggio. Holographic embeddings of knowledge graphs. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16, page 1955–1961. AAAI Press, 2016

    1955

  10. [21]

    Representing text for joint embedding of text and knowledge bases

    Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. Representing text for joint embedding of text and knowledge bases. In Lluís Màrquez, Chris Callison-Burch, and Jian Su, editors, Proceedings of the 2015 Conference on Empirical ...

    2015

  11. [22]

    Knowledge graph representation with jointly structural and textual encoding

    Jiacheng Xu, Xipeng Qiu, Kan Chen, and Xuanjing Huang. Knowledge graph representation with jointly structural and textual encoding. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI’17, page 1318–1324. AAAI Press, 2017. ISBN 9780999241103

    2017

  12. [23]

    Representation learning with entity topics for knowledge graphs

    Xin Ouyang, Yan Yang, Liang He, Qin Chen, and Jiacheng Zhang. Representation learning with entity topics for knowledge graphs. In Gang Li, Yong Ge, Zili Zhang, Zhi Jin, and Michael Blumenstein, editors, Knowledge Science, Engineering and Management , pages 534–542, Cham, 2017....

    2017

  13. [25]

    BERT: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of...

    2019 doi

  14. [26]

    Roberta: A robustly optimized bert pretraining approach, 2019

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach, 2019. URL http://arxiv.org/abs/1907.11692. cite arxiv:1907.11692

    2019 arXiv

  15. [27]

    Kg-bert: Bert for knowledge graph completion,

    Liang Yao, Chengsheng Mao, and Yuan Luo. Kg-bert: Bert for knowledge graph completion,

  16. [28]

    Kepler: A unified model for knowledge embedding and pre-trained language representation

    Xiaozhi Wang, Tianyu Gao, Zhaocheng Zhu, Zhengyan Zhang, Zhiyuan Liu, Juanzi Li, and Jian Tang. Kepler: A unified model for knowledge embedding and pre-trained language representation. Transactions of the Association for Computational Linguistics , 9:176–194, 2021

    2021

  17. [29]

    Reasoning through memorization: Nearest neighbor knowledge graph embeddings

    Peng Wang, Xin Xie, Xiaohan Wang, and Ninyu Zhang. Reasoning through memorization: Nearest neighbor knowledge graph embeddings. In CCF International Conference on Natural Language Processing and Chinese Computing, pages 111–122. Springer, 2023. 11

    2023

  18. [30]

    Gpt-4 technical report

    OpenAI, Josh Achiam, Steven Adler, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2024

    2024 arXiv

  19. [31]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, andet al. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, 2020

    2020

  20. [32]

    LLaMA: Open and efficient founda- tion language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, and et al. LLaMA: Open and efficient founda- tion language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  21. [33]

    Exploring large language models for knowledge graph completion

    Liang Yao, Jiazhen Peng, Chengsheng Mao, and Yuan Luo. Exploring large language models for knowledge graph completion. arXiv preprint arXiv:2308.13916, 2023

    2023 arXiv

  22. [34]

    Contextualization distillation from large language model for knowledge graph completion

    Dawei Li, Zhen Tan, Tianlong Chen, and Huan Liu. Contextualization distillation from large language model for knowledge graph completion. arXiv preprint arXiv:2402.01729, 2024

    2024 arXiv

  23. [35]

    Enhancing text-based knowledge graph completion with zero-shot large language models: A focus on semantic enhancement,

    Rui Yang, Jiahao Zhu, Jianping Man, Li Fang, and Yi Zhou. Enhancing text-based knowledge graph completion with zero-shot large language models: A focus on semantic enhancement,

  24. [36]

    A. v. d. Oord, Y . Li, and O. Vinyals. Representation learning with contrastive predictive coding. In arXiv preprint arXiv:1807.03748, 2018

    2018 arXiv

  25. [37]

    Complex embeddings for simple link prediction

    Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In International conference on machine learning, pages 2071–2080. PMLR, 2016

    2016

  26. [38]

    Convolutional 2d knowledge graph embeddings

    Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018

    2018

  27. [39]

    Tucker: Tensor factorization for knowledge graph completion

    Ivana Balaževi´c, Carl Allen, and Timothy M Hospedales. Tucker: Tensor factorization for knowledge graph completion. arXiv preprint arXiv:1901.09590, 2019

    1901 arXiv

  28. [40]

    Composition-based multi-relational graph convolutional networks

    Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha Talukdar. Composition-based multi-relational graph convolutional networks. arXiv preprint arXiv:1911.03082, 2019

    1911 arXiv

  29. [41]

    Quatde: Dynamic quaternion embedding for knowledge graph completion

    Haipeng Gao, Kun Yang, Yuxue Yang, Rufai Yusuf Zakari, Jim Wilson Owusu, and Ke Qin. Quatde: Dynamic quaternion embedding for knowledge graph completion. arXiv preprint arXiv:2105.09002, 2021

    2021 arXiv

  30. [42]

    Neural bellman-ford networks: A general graph neural network framework for link prediction

    Zhaocheng Zhu, Zuobai Zhang, Louis-Pascal Xhonneux, and Jian Tang. Neural bellman-ford networks: A general graph neural network framework for link prediction. Advances in Neural Information Processing Systems, 34:29476–29490, 2021

    2021

  31. [43]

    Pretrain-kge: learning knowledge representation from pretrained language models

    Zhiyuan Zhang, Xiaoqian Liu, Yi Zhang, Qi Su, Xu Sun, and Bin He. Pretrain-kge: learning knowledge representation from pretrained language models. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 259–266, 2020

    2020

  32. [44]

    Joint language semantic and structure embedding for knowledge graph completion

    Jianhao Shen, Chenguang Wang, Linyuan Gong, and Dawn Song. Joint language semantic and structure embedding for knowledge graph completion. arXiv preprint arXiv:2209.08721, 2022

    2022 arXiv

  33. [45]

    Simkgc: Simple contrastive knowledge graph completion with pre-trained language models

    Liang Wang, Wei Zhao, Zhuoyu Wei, and Jingming Liu. Simkgc: Simple contrastive knowledge graph completion with pre-trained language models. arXiv preprint arXiv:2203.02167, 2022

    2022 arXiv

  34. [46]

    Knowledge is flat: A seq2seq generative framework for various knowledge graph completion

    Chen Chen, Yufei Wang, Bing Li, and Kwok-Yan Lam. Knowledge is flat: A seq2seq generative framework for various knowledge graph completion. arXiv preprint arXiv:2209.07299, 2022

    2022 arXiv

  35. [47]

    Bridging structure and text for effective knowledge graph completion via conditional soft prompting

    Chen Chen et al. Bridging structure and text for effective knowledge graph completion via conditional soft prompting. arXiv preprint arXiv:2307.01709, 2023

    2023 arXiv

  36. [48]

    Kicgpt: Large language model with knowledge in context for kg completion

    Yanbin Wei et al. Kicgpt: Large language model with knowledge in context for kg completion. arXiv preprint arXiv:2402.02389, 2024

    2024 arXiv

  37. [49]

    Rotate: Knowledge graph embed- ding by relational rotation in complex space

    Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. Rotate: Knowledge graph embed- ding by relational rotation in complex space. arXiv preprint arXiv:1902.10197, 2019. 12

    1902 arXiv

  38. [50]

    Quaternion knowledge graph embeddings.Advances in neural information processing systems, 32, 2019

    Shuai Zhang, Yi Tay, Lina Yao, and Qi Liu. Quaternion knowledge graph embeddings.Advances in neural information processing systems, 32, 2019

    2019

  39. [51]

    Retrieval-enhanced generative model for large-scale knowledge graph completion

    Donghan Yu and Yiming Yang. Retrieval-enhanced generative model for large-scale knowledge graph completion. In Proceedings of SIGIR, pages 2334–2338, 2023

    2023

  40. [52]

    Representation learning of knowledge graphs with entity descriptions

    Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, and Maosong Sun. Representation learning of knowledge graphs with entity descriptions. In Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016

    2016

  41. [53]

    Expressive: A spatio-functional embedding for knowledge graph completion

    Marko Pavlovi´c, Yuta Wang, and Xia Li. Expressive: A spatio-functional embedding for knowledge graph completion. In International Conference on Learning Representations (ICLR), 2023

    2023

  42. [54]

    An inductive knowledge graph embedding via combination of subgraph and type information

    Hongbo Liu, Yifan Zhang, and Lixin Zhou. An inductive knowledge graph embedding via combination of subgraph and type information. Scientific Reports, 13(1):12345, 2023

    2023

  43. [55]

    A type-augmented knowledge graph embedding framework for knowledge graph completion

    Peng He, Jie Sun, and Rui Chen. A type-augmented knowledge graph embedding framework for knowledge graph completion. Scientific Reports, 13(2):23456, 2023

    2023

  44. [56]

    Multi-task learning for knowl- edge graph completion with pre-trained language models

    Bosung Kim, Taesuk Hong, Youngjoong Ko, and Jungyun Seo. Multi-task learning for knowl- edge graph completion with pre-trained language models. In Proceedings of COLING, pages 1737–1743, 2020

    2020

  45. [57]

    K-bert: Enabling language representation with knowledge graph

    Weijie Liu et al. K-bert: Enabling language representation with knowledge graph. In Proceed- ings of AAAI, pages 2901–2908, 2020

    2020

  46. [58]

    Mlmlm: Link prediction with mean likelihood masked language model

    Louis Clouatre, Philippe Trempe, Amal Zouaq, and Sarath Chandar. Mlmlm: Link prediction with mean likelihood masked language model. arXiv preprint arXiv:2009.07058, 2020

    2009 arXiv

  47. [59]

    Do pre-trained models benefit knowledge graph completion? a reliable evaluation and a reasonable approach

    Xin Lv et al. Do pre-trained models benefit knowledge graph completion? a reliable evaluation and a reasonable approach. In Proceedings of ACL, 2022

    2022

  48. [60]

    Structure-augmented text representation learning for efficient knowledge graph completion

    Bo Wang et al. Structure-augmented text representation learning for efficient knowledge graph completion. In Proceedings of WWW, pages 1737–1748, 2021

    2021

  49. [61]

    Multi-task pre-training language model for semantic network completion

    Da Li et al. Multi-task pre-training language model for semantic network completion. ACM Trans. Asian Low-Resour. Lang. Inf. Process., 22(11), 2023

    2023

  50. [62]

    Robust knowledge graph completion with stacked convolutions and a student re-ranking network

    Justin Lovelace et al. Robust knowledge graph completion with stacked convolutions and a student re-ranking network. In Proceedings of ACL, pages 1016–1029, 2021

    2021

  51. [63]

    Inductive entity representations from text via link prediction

    Daniel Daza, Michael Cochez, and Paul Groth. Inductive entity representations from text via link prediction. In Proceedings of WWW, pages 798–808, 2021

    2021

  52. [64]

    Making large language models perform better in knowledge graph completion

    Yichi Zhang, Zhuo Chen, Wen Zhang, and Huajun Chen. Making large language models perform better in knowledge graph completion. arXiv preprint arXiv:2310.06671, 2023

    2023 arXiv

  53. [65]

    Contextual language models for knowledge graph completion

    Russa Biswas, Radina Sofronova, Mehwish Alam, and Harald Sack. Contextual language models for knowledge graph completion. In Proceedings of MLSMKG@ECML-PKDD, pages 13–24, 2021

    2021

  54. [66]

    Knowledge graph embedding by translating on hyperplanes

    Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the AAAI conference on artificial intelligence, volume 28, 2014

    2014

  55. [67]

    Learning entity and relation embeddings for knowledge graph completion

    Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the AAAI conference on artificial intelligence, volume 29, 2015

    2015

  56. [68]

    Knowledge graph embedding via dynamic mapping matrix

    Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing...

    2015

  57. [69]

    A convolutional neural network-based model for knowledge base completion and its application to search personalization

    Dai Quoc Nguyen, Dat Quoc Nguyen, Tu Dinh Nguyen, and Dinh Phung. A convolutional neural network-based model for knowledge base completion and its application to search personalization. Semantic Web, 10(5):947–960, 2019

    2019

  58. [70]

    Modeling relational data with graph convolutional networks

    Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In The semantic web: 15th international conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, proceedings...

    2018

  59. [71]

    Kbgan: Adversarial learning for knowledge graph embed- dings

    Liwei Cai and William Yang Wang. Kbgan: Adversarial learning for knowledge graph embed- dings. arXiv preprint arXiv:1711.04071, 2017

    2017 arXiv

  60. [72]

    Dense: An enhanced non-abelian group representation for knowledge graph embedding

    Haonan Lu and Hailin Hu. Dense: An enhanced non-abelian group representation for knowledge graph embedding. arXiv preprint arXiv:2008.04548, 2020

    2008 arXiv

  61. [73]

    Lineare: Simple but powerful knowledge graph embedding for link prediction

    Yanhui Peng and Jing Zhang. Lineare: Simple but powerful knowledge graph embedding for link prediction. In 2020 IEEE international conference on data mining (ICDM), pages 422–431. IEEE, 2020

    2020

  62. [74]

    Duality-induced regularizer for tensor factorization based knowledge graph completion

    Zhanqiu Zhang, Jianyu Cai, and Jie Wang. Duality-induced regularizer for tensor factorization based knowledge graph completion. Advances in Neural Information Processing Systems, 33: 21604–21615, 2020

    2020

  63. [75]

    Modeling heterogeneous hierarchies with relation-specific hyperbolic cones

    Yushi Bai, Zhitao Ying, Hongyu Ren, and Jure Leskovec. Modeling heterogeneous hierarchies with relation-specific hyperbolic cones. Advances in Neural Information Processing Systems, 34:12316–12327, 2021

    2021

  64. [76]

    Rot-pro: Modeling transitivity by projection in knowledge graph embedding

    Tengwei Song, Jie Luo, and Lei Huang. Rot-pro: Modeling transitivity by projection in knowledge graph embedding. Advances in Neural Information Processing Systems, 34:24695– 24706, 2021

    2021

  65. [77]

    Mem-kgc: Masked entity model for knowledge graph completion with pre-trained language model

    Bonggeun Choi, Daesik Jang, and Youngjoong Ko. Mem-kgc: Masked entity model for knowledge graph completion with pre-trained language model. IEEE Access, 9:132025–132032, 2021

    2021

  66. [78]

    Sequence-to-sequence knowledge graph completion and question answering

    Apoorv Saxena, Adrian Kochsiek, and Rainer Gemulla. Sequence-to-sequence knowledge graph completion and question answering. arXiv preprint arXiv:2203.10321, 2022

    2022 arXiv

  67. [79]

    Knowledge graph extension with a pre-trained language model via unified learning method

    Bonggeun Choi and Youngjoong Ko. Knowledge graph extension with a pre-trained language model via unified learning method. Knowledge-Based Systems, 262:110245, 2023

    2023

  68. [80]

    Language models as knowledge embeddings

    Xintao Wang, Qianyu He, Jiaqing Liang, and Yanghua Xiao. Language models as knowledge embeddings. arXiv preprint arXiv:2206.12617, 2022

    2022 arXiv

  69. [81]

    From discrimination to generation: Knowledge graph completion with generative transformer

    Xin Xie et al. From discrimination to generation: Knowledge graph completion with generative transformer. In Proceedings of WWW Companion, pages 162–165, 2022. 14 NeurIPS Paper Checklist

    2022

  70. [87]

    Guidelines: • The answer NA means that the abstract and introduction do not include the claims made in the paper

    Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? Answer: [Yes] Justification: Yes, the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope. Gui...

  71. [88]

    Limitations

    Limitations Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: Yes, we discuss the limitations in Appendix F. Guidelines: • The answer NA means that the paper has no limitation while the answer No means that the ...

  72. [89]

    Guidelines: • The answer NA means that the paper does not include theoretical results

    Theory assumptions and proofs Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA] 15 Justification: This paper does not include theoretical results. Guidelines: • The answer NA means that th...

  73. [90]

    We provide more details of the module implementation in Appendix C

    Experimental result reproducibility Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and da...

  74. [91]

    All datasets used for evaluation are open-sourced

    Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material? 16 Answer: [Yes] Justification: Yes, our source code is ...

  75. [92]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experimental setting/details Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results? Answer: [Yes] Justification: Yes, we provide detailed desc...

  76. [93]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experiment statistical significance Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [No] Justification: Although we do not report error bars, we provide so...

  77. [94]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experiments compute resources Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [Yes] Justification: Yes, we report the deta...

  78. [95]

    Guidelines: • The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics

    Code of ethics Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? Answer: [Yes] Justification: Yes, we have reviewed the NeurIPS Code of Ethics and ensured full compliance th...

  79. [96]

    Guidelines: • The answer NA means that there is no societal impact of the work performed

    Broader impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: Yes, we discuss the potential positive societal impacts and negative societal impacts in Appendix G. Guid...

  80. [97]

    Guidelines: • The answer NA means that the paper poses no such risks

    Safeguards Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: This paper pose...

  81. [98]

    Guidelines: • The answer NA means that the paper does not use existing assets

    Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: Yes, we cite the...

  82. [99]

    Our source code is anonymous

    New assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: Yes, we communicate the details of the code as part of our submission. Our source code is anonymous. Guidelines: • T...

  83. [100]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Crowdsourcing and research with human subjects Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? A...

  84. [101]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals...

  85. [102]

    Answer: [Yes] Justification: LLMs were employed to support the understanding and interpretation of complex technical concepts

    Declaration of LLM usage Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the ...

    2025

  86. [103]

    Describe the relationship between h and t given that r holds

    attaches a special token to each entity description, maps relations to trainable vectors, and jointly optimises a translational distance loss and Wikipedia-based MLM. This dual training scheme mitigates the frequency imbalance problem of long-tail entities. Two notable extensi...

  87. [104]

    Is it true that Paris is-the-capital-of France?

    follows a similar philosophy but emphasises constrained prompts that prevent factual drift. The authors design slot-filled templates whose lexical space is restricted by ontology types, dramatically reducing hallucinated entities. By regenerating or extending existing descript...

  88. [2013]

    URL https://proceedings.neurips.cc/paper_files/paper/2013/file/ 1cecc7a77928ca8133fa24680a88d2f9-Paper.pdf. 10

    2013

  89. [2015]

    doi: 10.18653/v1/D15-1174

    Association for Computational Linguistics. doi: 10.18653/v1/D15-1174. URL https: //aclanthology.org/D15-1174/

    doi

  90. [2019]

    URL https://arxiv.org/abs/1909.03193

    1909 arXiv

  91. [2020]

    doi: 10.1109/TNNLS.2021.3070843

    2021

  92. [2024]

    URL https://arxiv.org/abs/2310.08279

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.