Pith. sign in

REVIEW 5 major objections 6 minor 44 references

Graph Collaborative Attention Network for Link Prediction in Knowledge Graphs

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that an attention network that aggregates both entities and relations in local graph neighborhoods outperforms rule-based link prediction and matches or exceeds neural embedding baselines on four benchmark knowledge graphs.

desk verdict A renamed KBGAT with internally impossible evaluation tables; the central claim collapses under inspection. read the letter →

arxiv 2507.03947 v2 pith:NZA7VBEF submitted 2025-07-05 cs.LG

Thanh Hoang-Minh This is my paper
classification cs.LG
keywords knowledgegraphlinkpredictionattentionnetworkmulti-headTransEConvKBKBGATcompletion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper seeks to show that an attention-based graph neural network, GCAT, improves link prediction in knowledge graphs by aggregating information from both neighboring entities and the relations that connect them. The authors build on the KBGAT encoder, refine its context aggregation and interaction steps, and decode with ConvKB. On FB15k, FB15k-237, WN18, and WN18RR they report that GCAT consistently beats rule-based systems and is competitive with or superior to neural embedding baselines such as TransE and ComplEx. If the reported results hold, attention-based aggregation of heterogeneous neighborhoods is a practical path to knowledge graph completion without hand-crafted rules.

What carries the argument

The central object is the GCAT encoder: multi-head graph attention whose coefficient for a neighbor $j$ of entity $i$ is computed from a triple embedding $t_{ijk} = W_1[e_i \,||\, e_j \,||\, r_k]$ (head, tail, relation), normalized with softmax over all neighbors and relations; the updated entity embedding is the attention-weighted sum of these triple vectors, concatenated across heads, with a residual connection $H = W_E E + E''$. This mechanism carries the argument because it lets relation identity participate in attention instead of attending only over entity features, so context aggregation distinguishes different relational roles of the same entity.

What would settle it

Recompute the baseline rows on WN18 under the same filtered query set used for GCAT; if ComplEx's published H@1 of 94.53 and MRR of 0.349 cannot be reproduced together, the table mixes evaluation protocols and the comparison collapses.

Watch

Extended reading notes

Core claim

The central claim is that jointly attending over entity and relation embeddings in a local neighborhood yields link-prediction embeddings that generalize better than rule-based and earlier neural approaches. GCAT concatenates head, relation, and tail embeddings into a triple vector, computes normalized attention coefficients per neighbor and relation, aggregates over multiple heads, and adds a residual projection of the initial TransE embedding to avoid vanishing gradients; ConvKB scores candidate triples. The paper reports H@10 of 91.64 on FB15k and 58.32 on FB15k-237 with MRR 0.4353, and on WN18 H@1 of 35.12, H@10 57.01, MR 1974, and MRR 0.4301, which it reads as evidence that attention-based context aggregation captures relational diversity better than rule mining.

Load-bearing premise

The load-bearing premise is that the numbers in Tables 2 and 3 come from the same evaluation protocol, so the GCAT row and the baseline rows are directly comparable.

Editorial extensions

If this is right

  • On filtered benchmarks without inverse-relation leakage, attention-based embedding models hold their advantage over rule mining.
  • GCAT's encoder-decoder pipeline can serve as a reproducible baseline for future knowledge-graph completion work.
  • Incorporating relation-aware attention plus a residual TransE initialization stabilizes training enough for multi-hop neighborhood aggregation.
  • The ConvKB decoder converts learned embeddings into ranked candidate triples, matching the output format used in recommendation-style link prediction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the FB15k-237 gain (MRR 0.4353 versus 0.352 for TuckER) reproduces, relation-aware collaborative attention appears to pay off most on datasets where inverse relations have been removed, so testing on other filtered benchmarks such as YAGO3-10 would sharpen the claim.
  • Because the model relies on TransE initialization, a natural extension the paper does not develop is replacing that initialization with a more expressive pretrained embedding; the residual connection could preserve the new starting point.
  • The attention coefficients over entity-relation pairs could be inspected as explanations for individual predictions, a use the paper does not explore.
  • The paper's future-work direction of temporal attention could be tested by adding time-stamped relation embeddings to the same triple-attention formula.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This manuscript surveys knowledge-graph completion, describes the KBGAT architecture (TransE initialization, multi-head attention encoder over relation-aware triples, and a ConvKB decoder), and proposes GCAT as a refined variant. It reports Hits@1, Hits@10, Mean Rank, and MRR on FB15k, FB15k-237, WN18, and WN18RR, and claims that GCAT consistently outperforms rule-based methods and is competitive or superior to neural embedding baselines. The paper also states that training code and model results are publicly available.

Significance. If the reported results were reliable, the paper would offer a modest architectural variant of KBGAT plus a broad benchmark comparison, and the open-code statement is a positive for reproducibility. However, the central empirical claim is load-bearing and is contradicted by the paper's own Table 3: the WN18 ComplEx baseline row is internally impossible, the GCAT WN18 row is below every baseline, and no WN18RR or rule-based results are reported. Moreover, the GCAT architecture is presented entirely through KBGAT equations, so the claimed contribution is not crisply defined. As it stands, the manuscript does not establish its main claims.

major comments (5)
  1. [Section 4.4, Table 3 (WN18 row)] In the WN18 ComplEx row, H@1 = 94.53 and MRR = 0.349 are mutually inconsistent on the same query set: if H@1 is the fraction of queries whose correct entity is ranked first, each such query contributes 1 to the MRR sum, so MRR must be at least 0.9453. The reported MRR of 0.349 is therefore impossible, which indicates that this baseline row is not a valid measured result and undermines all comparisons in Table 3.
  2. [Section 4.4, Table 3 (WN18RR row)] The WN18RR TransE row reports H@1 = 2.79, H@10 = 94.87, MRR = 0.646. These numbers are jointly impossible: even under the most favorable assignment (2.79% of queries at rank 1, 92.08% at rank 2, and the remaining 5.13% at rank 11), the MRR ceiling is about 0.493, below the reported 0.646. The identical TransE row appearing verbatim in both the WN18 and WN18RR blocks compounds the concern that Table 3 was assembled from mismatched sources.
  3. [Section 4.4, Table 3; Section 5] GCAT's WN18 results (H@1 = 35.12, H@10 = 57.01, MRR = 0.4301) are far below every baseline in the same table, and no GCAT entry is shown for WN18RR even though Section 5 claims strong performance 'particularly on complex and filtered datasets such as FB15k-237 and WN18RR'. This directly contradicts the abstract's assertion that GCAT achieves competitive or superior performance.
  4. [Section 3.4, Eqs. (14)-(19)] The encoder equations presented for GCAT are explicitly attributed to KBGAT ('All equations presented here are based on those in the work of Nathani, Deepak[26]'), and the manuscript never states an equation, algorithm, or architectural change that distinguishes GCAT from KBGAT. Without a specification of the proposed model, the core contribution cannot be evaluated or reproduced.
  5. [Section 4.4, Tables 2 and 3] No rule-based baseline results are reported, although the abstract claims GCAT 'consistently outperforms rule-based methods'. Methods such as AMIE, RuleN, and RuDiK are discussed in Sections 1-2 but never evaluated; a direct comparison is essential for supporting that half of the central claim.
minor comments (6)
  1. [Section 4.2] The subsection is titled 'Mean Reciprocal Rank (MMR)' and the text later writes 'M RR'; the acronym should be MRR, and the formula should be typeset consistently.
  2. [Table 1 and Section 4.1] The text says the evaluation uses four datasets, but Table 1 and the accompanying bar chart include YAGO3-10; this inconsistency should be resolved.
  3. [Section 4.4] The sentence 'As previously mentioned, our rule-based model can be fully executed on a standard laptop' refers to a rule-based model that is not described anywhere in the paper.
  4. [Section 3.4.3, Eq. (23)] The scoring-function notation mixes h_j with e_j and uses the symbol labeled as a sum over filters ambiguously; all symbols in Eq. (23) should be defined consistently.
  5. [References] References [9] and [10] are duplicate entries for the same Dettmers et al. paper; citation and reference-list details should be corrected.
  6. [Section 4.3.1] Auxiliary relations are said to be added to enrich sparse graphs, but their construction is never defined; if they affect the reported results, this is a reproducibility gap.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the central claim is an empirical benchmark comparison; the invalid Table 3 is a data-integrity problem, not a derivation that reduces to its inputs.

full rationale

The paper's central claim ('GCAT not only consistently outperforms rule-based methods but also achieves competitive or superior performance compared to existing neural embedding models') is supported, in principle, by benchmark numbers in Tables 2 and 3. Those numbers are external observations relative to the model equations, so no prediction is defined in terms of the result, and no fitted parameter is renamed as a prediction. The method sections closely follow KBGAT: Section 3.4 states 'All equations presented here are based on those in the work of Nathani, Deepak[26],' and the GCAT equations (14)-(24) are essentially KBGAT's encoder-decoder. That is a novelty/attribution concern, not circular reasoning, because the empirical claim is not implied by adopting those equations. Likewise, the Table 3 contradiction (WN18 ComplEx H@1 94.53 with MRR 0.349, and WN18RR TransE H@1 2.79 with MRR 0.646) is an internal-validity or fabrication concern; MRR is defined as an average of reciprocal ranks and cannot be below H@1/100 on the same query set, so the table could not have been produced by the stated evaluation. This is a correctness failure, not self-referential derivation. There are no self-citations, and no load-bearing appeal to an author-derived uniqueness theorem. Therefore the circularity score is 0.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim depends on standard attention math, the reliability of the benchmark splits, the comparability of survey baseline numbers to the new runs, and an unstated auxiliary-relations design choice. The model's hyperparameter set is largely unreported, and the GCAT architecture is indistinguishable from KBGAT without an unstated modification.

free parameters (6)
  • embedding dimension = 200
    Set to 200 for entities and relations; no sensitivity analysis reported (Section 4.3.1).
  • learning rate = 0.001
    Adam optimizer learning rate; no schedule described (Section 4.3.1).
  • margin gamma = not reported
    Margin in the ranking loss of Equation 21; the value is absent from the text.
  • number of attention heads = not reported
    Number of heads in the multi-head attention encoder is never stated.
  • nhop neighborhood size = not reported
    Attention aggregation radius (nhop hops) is mentioned but not specified.
  • ConvKB hyperparameters = not reported
    Number of convolution filters and the regularization weight lambda in Equation 24 are not given.
assumptions (5)
  • standard math Softmax normalization and LeakyReLU attention coefficients as in GAT and KBGAT equations.
    The paper adopts standard attention machinery without proof; treated as background knowledge.
  • domain assumption Benchmark datasets FB15k, FB15k-237, WN18, and WN18RR with their standard splits measure link prediction performance.
    The paper relies on these splits without leakage analysis or a re-derived evaluation protocol.
  • domain assumption Baseline numbers in Tables 2 and 3, taken from the survey Rossi et al. [32], are directly comparable to GCAT runs.
    No identical setup, hardware, or code is provided; cross-paper comparison is assumed.
  • ad hoc to paper Auxiliary relations are added to enrich sparse graphs without defining what they are or how they were constructed.
    Section 4.3.1 mentions auxiliary relations; this modeling choice is unstated and could affect results.
  • ad hoc to paper GCAT is treated as a distinct model while all equations are attributed to KBGAT [26].
    Section 3.4 states equations are based on Nathani et al.; no new equation defines GCAT, so the new model claim depends on an unstated modification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Graph Collaborative Attention Network for Link Prediction in Knowledge Graphs." pith.science (2026). https://pith.science/paper/NZA7VBEF

@misc{pith2026250703947,
  author       = {Pith},
  title        = {Pith review of: Graph Collaborative Attention Network for Link Prediction in Knowledge Graphs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NZA7VBEF}},
  note         = {Machine review of arXiv:2507.03947}
}
read the original abstract

Knowledge graphs offer a structured representation of real-world entities and their relationships, enabling a wide range of applications from information retrieval to automated reasoning. In this paper, we conduct a systematic comparison between traditional rule-based approaches and modern deep learning methods for link prediction. We focus on KBGAT, a graph neural network model that leverages multi-head attention to jointly encode both entity and relation features within local neighborhood structures. To advance this line of research, we introduce \textbf{GCAT} (Graph Collaborative Attention Network), a refined model that enhances context aggregation and interaction between heterogeneous nodes. Experimental results on four widely-used benchmark datasets demonstrate that GCAT not only consistently outperforms rule-based methods but also achieves competitive or superior performance compared to existing neural embedding models. Our findings highlight the advantages of attention-based architectures in capturing complex relational patterns for knowledge graph completion tasks.

Figures

Figures reproduced from arXiv: 2507.03947 by the authors.

Figure 1
Figure 1. Knowledge graph and normalized attention coefficients of the entity [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. A taxonomy of research areas in knowledge graphs [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Example of an input graph Graph embedding is the process of transforming graph features into vectors or sets of low-dimensional vectors. The more effective the embedding, the higher the accuracy in subsequent graph mining and analysis tasks. The biggest challenge in graph embedding depends on the problem set￾ting, which includes both the embedding input and output, as illustrated in [PITH_FULL_IMAGE:figures/full_fi… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Graph Embedding Techniques Based on the embedding input, we categorize the sur￾veyed methods in [5] as follows: Homogeneous Graph, Het￾erogeneous Graph, Graph with Auxiliary Information, and Graph Constructed from Non-relational Data. Different types of embedding input…
Figure 5
Figure 5. Figure 5: Graph embedding methods. Second, some methods embed a pair of nodes as a vector to compare it with other node pairs or to predict the existence of a link between the two nodes. Edge embedding benefits graph analyses that involve edges (node pairs), such as link predict…
Figure 6
Figure 6. Figure 6: Graph Embedding Techniques embedding into the attention mechanism. These methods will be discussed in detail in the subsequent sections. • Other methods There are also other approaches, such as autoencoder￾based techniques like Structural Deep Network Embed￾ding (SDNE)…
Figure 7
Figure 7. Figure 7: Illustration of embedding vectors in the TransE model [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: The TransE graph embedding vector with E ∈ R Ne×Din and E′′ ∈ R Ne×D′′ . Simultaneously, it transforms the relation embedding ma￾trix R = n −→r1 , −→r2 , ..., −→rNr o −→ R′′ = n−→ r ′′ 1 , −→ r ′′ 2 , ..., −→ r ′′ Nr o , with R ∈ R Nr×Pin and R′′ ∈ R Nr×P ′′ . Similar …
Figure 9
Figure 9. Figure 9: Illustration of the encoder layers in the GCAT model [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Illustration of the decoder layers of the ConvKB model [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 28 canonical work pages

  1. [26]

    Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs

    Deepak Nathani, Jatin Chauhan, Charu Sharma, and Manohar Kaul. Learning attention-based embeddings for rela- tion prediction in knowledge graphs. arXiv preprint arXiv:1906.01195, 2019. 1, 3, 7, 11

    work page Pith review arXiv 1906

  2. [1]

    Neural machine translation by jointly learning to align and translate

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014. 9

    arXiv 2014

  3. [2]

    Video suggestion and discovery for youtube: taking random walks through the view graph

    Shumeet Baluja, Rohan Seth, Dharshi Sivakumar, Yushi Jing, Jay Yagnik, Shankar Kumar, Deepak Ravichandran, and Mo- hamed Aly. Video suggestion and discovery for youtube: taking random walks through the view graph. In Proceed- ings of the 17th international conference on World Wide Web, pages 895–904, 2008. 1

    work page 2008

  4. [3]

    Netgan: Generating graphs via random walks

    Aleksandar Bojchevski, Oleksandr Shchur, Daniel Z ¨ugner, and Stephan G ¨unnemann. Netgan: Generating graphs via random walks. arXiv preprint arXiv:1803.00816, 2018. 7

    arXiv 2018

  5. [4]

    Translating embeddings for modeling multi-relational data

    Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Ja- son Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In Advances in neural information processing systems, pages 2787–2795, 2013. 1, 11, 12, 13, 15, 16

    work page 2013

  6. [5]

    A comprehensive survey of graph embedding: Prob- lems, techniques, and applications

    Hongyun Cai, Vincent W Zheng, and Kevin Chen-Chuan Chang. A comprehensive survey of graph embedding: Prob- lems, techniques, and applications. IEEE Transactions on Knowledge and Data Engineering, 30(9):1616–1637, 2018. 2, 5, 9

    work page 2018

  7. [6]

    Grarep: Learning graph representations with global structural information

    Shaosheng Cao, Wei Lu, and Qiongkai Xu. Grarep: Learning graph representations with global structural information. In Proceedings of the 24th ACM international on conference on information and knowledge management, pages 891–900,

    work page

  8. [7]

    On the relationship between self-attention and convolutional layers

    Jean-Baptiste Cordonnier, Andreas Loukas, and Martin Jaggi. On the relationship between self-attention and convolutional layers. arXiv preprint arXiv:1911.03584, 2019. 3, 9

    arXiv 1911

Show all 44 references
  1. [8]

    Multi-head attention: Collaborate instead of concatenate

    Jean-Baptiste Cordonnier, Andreas Loukas, and Martin Jaggi. Multi-head attention: Collaborate instead of concatenate. arXiv preprint arXiv:2006.16362, 2020. 3

    2006 arXiv

  2. [9]

    Convolutional 2d knowledge graph embed- dings

    Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embed- dings. arXiv preprint arXiv:1707.01476, 2017. 1, 7, 16

    2017 arXiv

  3. [10]

    Convolutional 2d knowledge graph em- beddings

    Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph em- beddings. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018. 15

    2018

  4. [11]

    Fast rule mining in ontological knowledge bases with amie

    Luis Gal´arraga, Christina Teflioudi, Katja Hose, and Fabian M Suchanek. Fast rule mining in ontological knowledge bases with amie. The VLDB Journal, 24(6):707–730, 2015. 1, 2

    2015

  5. [12]

    Introducing the Knowledge Graph: things, not strings, 2020 (accessed on August 27, 2020)

    Google. Introducing the Knowledge Graph: things, not strings, 2020 (accessed on August 27, 2020). 1

    2020

  6. [13]

    Graph embedding tech- niques, applications, and performance: A survey

    Palash Goyal and Emilio Ferrara. Graph embedding tech- niques, applications, and performance: A survey. Knowledge- Based Systems, 151:78–94, 2018. 2

    2018

  7. [14]

    node2vec: Scalable feature learning for networks

    Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pages 855–864, 2016. 7

    2016

  8. [15]

    Learning to exploit long-term relational dependencies in knowledge graphs.arXiv preprint arXiv:1905.04914, 2019

    Lingbing Guo, Zequn Sun, and Wei Hu. Learning to exploit long-term relational dependencies in knowledge graphs.arXiv preprint arXiv:1905.04914, 2019. 7

    1905 arXiv

  9. [16]

    Hopfield network

    John J Hopfield. Hopfield network. Scholarpedia, 2(5):1977,

    1977

  10. [17]

    Anonymous walk em- beddings

    Sergey Ivanov and Evgeny Burnaev. Anonymous walk em- beddings. arXiv preprint arXiv:1805.11921, 2018. 7

    2018 arXiv

  11. [18]

    A survey on knowledge graphs: Rep- resentation, acquisition and applications

    Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, and Philip S Yu. A survey on knowledge graphs: Rep- resentation, acquisition and applications. arXiv preprint arXiv:2002.00388, 2020. 1, 3

    2002 arXiv

  12. [19]

    Semi-supervised classi- fication with graph convolutional networks

    Thomas N Kipf and Max Welling. Semi-supervised classi- fication with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016. 10, 11

    2016 arXiv

  13. [20]

    Object recognition with gradient-based learning

    Yann LeCun, Patrick Haffner, L´eon Bottou, and Yoshua Ben- gio. Object recognition with gradient-based learning. In Shape, contour and grouping in computer vision, pages 319–

  14. [21]

    Makowsky

    Johann A. Makowsky. Why horn formulas matter in computer science: Initial structures and generic examples. Journal of Computer and System Sciences, 34(2-3):266–292, 1987. 2

    1987

  15. [22]

    Fine- grained evaluation of rule-and embedding-based systems for knowledge graph completion

    Christian Meilicke, Manuel Fink, Yanjie Wang, Daniel Ruffinelli, Rainer Gemulla, and Heiner Stuckenschmidt. Fine- grained evaluation of rule-and embedding-based systems for knowledge graph completion. In International Semantic Web Conference, pages 3–20. Springer, 2018. 1, 2

    2018

  16. [23]

    Anytime Bottom- Up Rule Learning for Knowledge Graph Completion, 2019

    Christian Meilicke, Melisachew Wudage Chekol, Daniel Ruffinelli, and Heiner Stuckenschmidt. Anytime Bottom- Up Rule Learning for Knowledge Graph Completion, 2019. 2

    2019

  17. [24]

    Efficient estimation of word representations in vector space

    Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013. 11

    2013 arXiv

  18. [25]

    Hierarchical graph embedding in vector space by graph pyramid

    Seyedeh Fatemeh Mousavi, Mehran Safayani, Abdolreza Mirzaei, and Hoda Bahonar. Hierarchical graph embedding in vector space by graph pyramid. Pattern Recognition, 61: 245–254, 2017. 6

    2017

  19. [27]

    A novel embedding model for knowledge base completion based on convolutional neural network

    Dai Quoc Nguyen, Tu Dinh Nguyen, Dat Quoc Nguyen, and Dinh Phung. A novel embedding model for knowledge base completion based on convolutional neural network. arXiv preprint arXiv:1712.02121, 2017. 7, 15

    2017 arXiv

  20. [28]

    Memory and Attention, 2020

    Ministry of Health of Vietnam. Memory and Attention, 2020. (Accessed on August 26, 2020). 9

    2020

  21. [29]

    Robust discovery of positive and negative rules in knowledge bases

    Stefano Ortona, Venkata Vamsikrishna Meduri, and Paolo Papotti. Robust discovery of positive and negative rules in knowledge bases. In 2018 IEEE 34th International Confer- ence on Data Engineering (ICDE), pages 1168–1179. IEEE,

    2018

  22. [30]

    Deepwalk: Online learning of social representations

    Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowl- edge discovery and data mining, pages 701–710, 2014. 7

    2014

  23. [31]

    Stand- alone self-attention in vision models

    Prajit Ramachandran, Niki Parmar, Ashish Vaswani, Irwan Bello, Anselm Levskaya, and Jonathon Shlens. Stand- alone self-attention in vision models. arXiv preprint arXiv:1906.05909, 2019. 10

    1906 arXiv

  24. [32]

    Knowledge graph em- bedding for link prediction: A comparative analysis

    Andrea Rossi, Donatella Firmani, Antonio Matinata, Paolo Merialdo, and Denilson Barbosa. Knowledge graph em- bedding for link prediction: A comparative analysis. arXiv preprint arXiv:2002.00819, 2020. 7, 16

    2002 arXiv

  25. [33]

    Dy- namic routing between capsules

    Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. Dy- namic routing between capsules. In Advances in neural infor- mation processing systems, pages 3856–3866, 2017. 2

    2017

  26. [34]

    Megatron-lm: Training multi-billion parameter language models using gpu model parallelism

    Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-lm: Training multi-billion parameter language models using gpu model parallelism. arXiv preprint arXiv:1909.08053, 2019. 3

    1909 arXiv

  27. [35]

    Line: Large-scale information network em- bedding

    Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. Line: Large-scale information network em- bedding. In Proceedings of the 24th international conference on world wide web, pages 1067–1077, 2015. 7

    2015

  28. [36]

    Hierarchi- cal multi-scale attention for semantic segmentation

    Andrew Tao, Karan Sapra, and Bryan Catanzaro. Hierarchi- cal multi-scale attention for semantic segmentation. arXiv preprint arXiv:2005.10821, 2020. 3

    2005 arXiv

  29. [37]

    Observed versus latent features for knowledge base and text inference

    Kristina Toutanova and Danqi Chen. Observed versus latent features for knowledge base and text inference. In Proceed- ings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality, pages 57–66, 2015. 15, 16

    2015

  30. [38]

    Complex embeddings for simple link prediction

    Th´eo Trouillon, Johannes Welbl, Sebastian Riedel, ´Eric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In Proceedings of the 33rd In- ternational Conference on Machine Learning (ICML), 2016. 1

    2016

  31. [39]

    The anatomy of the facebook social graph

    Johan Ugander, Brian Karrer, Lars Backstrom, and Cameron Marlow. The anatomy of the facebook social graph. arXiv preprint arXiv:1111.4503, 2011. 1

    2011 arXiv

  32. [40]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008, 2017. 1, 7, 9

    2017

  33. [41]

    Graph attention networks

    Petar Veliˇckovi´c, Guillem Cucurull, Arantxa Casanova, Adri- ana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017. 1, 3, 7, 10

    2017 arXiv

  34. [42]

    A capsule network-based embedding model for knowl- edge graph completion and search personalization

    Thanh Vu, Tu Dinh Nguyen, Dat Quoc Nguyen, Dinh Phung, et al. A capsule network-based embedding model for knowl- edge graph completion and search personalization. In Pro- ceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Lingu...

    2019

  35. [43]

    Structural deep net- work embedding

    Daixin Wang, Peng Cui, and Wenwu Zhu. Structural deep net- work embedding. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1225–1234, 2016. 8

    2016

  36. [44]

    Xlnet: Generalized autoregressive pretraining for language understanding

    Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems , pages 5753–5763, 2019. 2

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.