Pith. sign in

REVIEW 4 major objections 5 minor 14 references

The authors claim that a plain Transformer reading only sampled local subgraphs can match specialized link predictors without hand-coded heuristics or node-ID embeddings.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 05:36 UTC pith:4NWAIE6I

load-bearing objection A useful Transformer+GNN hybrid overclaimed as a plain Transformer; the empirical work is solid and the theory is fixable, but the headline claim does not survive contact with the architecture. the 4 major comments →

arxiv 2602.01553 v3 pith:4NWAIE6I submitted 2026-02-02 cs.LG cs.AI

Plain Transformers are Surprisingly Powerful Link Predictors

Quang Truong , Yu Song , Donald Loveland , Mingxuan Ju , Tong Zhao , Neil Shah , Jiliang Tang This is my paper
classification cs.LG cs.AI
keywords link predictionplain Transformergraph Transformersubgraph samplingstructural heuristicsparameter efficiencypairwise heuristicsnode-adjacency tokenization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

PENCIL is an argument that the machinery people add to link predictors—structural heuristics, node-ID embeddings, spectral or random-walk positional encodings—may be optional. The paper proposes a plain BERT-style Transformer that consumes a fixed-size random sample of the local subgraph around each candidate link, and shows it matches or beats specialized models on several benchmarks while using 22–146× fewer parameters. A theoretical analysis identifies why: with endpoint-pinned randomized tokens, the architecture can degenerate to classical path-based scores and to unbiased common-neighbor estimators, and its expressivity is at least that of subgraph-based predictors like SEAL. The practical stakes: if true, large-scale link prediction can be done with standard hardware-optimized Transformer stacks, no global preprocessing, no per-node identity table, and no hand-crafted features.

Core claim

PENCIL's central claim is that a standard encoder-only Transformer—the kind used for text, without positional encodings, structural encodings, or per-node ID embeddings—can be a top-tier link predictor if each candidate link is represented as a fixed-budget sampled subgraph with the two endpoints pinned as dedicated tokens. The model's input per token concatenates a one-hot index, the token's adjacency row within the sampled subgraph, and a role flag; random reindexing of non-query nodes makes the predictor permutation invariant in distribution. On seven benchmarks it outperforms or matches heuristic-informed GNNs and ID-embedding models, and the authors prove that with suitable weights the

What carries the argument

The load-bearing mechanism is link-centric subgraph tokenization plus a 'multiplicative residual': each sampled node is encoded as a concatenation of a one-hot slot identifier, its adjacency row inside the sampled subgraph, and a role flag, with the two query endpoints pinned to slots 0 and 1; a stack of bidirectional self-attention blocks is followed by a projection of the reconstructed adjacency matrix applied to the block output. That residual lets the plain Transformer reduce to a source-conditioned message-passing network and to sum-aggregation message passing with quasi-orthogonal token vectors, so path-based heuristics, common-neighbor counts, and SEAL-level expressivity are achieved

Load-bearing premise

The load-bearing premise, stated in Section 3.1, is that a small fixed random sample of the local neighborhood around each candidate link contains the topology needed to decide that link; on datasets where ground-truth links are determined by structure outside that budget, PENCIL's scores would drop regardless of Transformer capacity.

What would settle it

Train and evaluate PENCIL on a synthetic graph family where positive links are deliberately placed only between pairs whose local neighborhoods within the sampling budget contain no overlap—for example, cross-community edges with no short paths or common neighbors within Nmax hops. If ranking accuracy stays near chance while a full-graph or larger-budget model succeeds, the paper's central claim is falsified; if PENCIL still ranks those links correctly, the fixed-budget locality assumption is not the bottleneck.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • PENCIL reports state-of-the-art results on cora and ogbl-ppa in the original setting and top HeaRT scores on ogbl-ppa and ogbl-ddi, using 22–146× fewer learnable parameters than the next-best competitors.
  • It converges in roughly 0.5–15 epochs on ogbl-citation2, ogbl-ddi, and ogbl-ppa, versus 20–100+ epochs reported for pure GNNs.
  • Theoretically, PENCIL can degenerate to a source-conditioned message-passing model that reproduces path-based scores (Katz index, Personalized PageRank, shortest-path distance, widest and most reliable path) and, under random or orthogonal token vectors, unbiased estimators of common-neighbor-type heuristics; with relational pooling it is at least as expressive as SEAL under the same sampling budg
  • Depth helps: unlike MPNNs, PENCIL's performance improves with more layers on cora, pubmed, and ogbl-collab, with only mild saturation at greater depths.
  • On several benchmarks structure alone is competitive with or better than using node features, suggesting node features can be weakly informative relative to local topology.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: if the coherence argument transfers, Nmax is not just a compute knob but a correctness lever—for fixed hidden dimension d, keeping Nmax ≤ d preserves the possibility of exactly orthogonal token vectors, and pushing past d should degrade the implicit heuristic estimators; a sweep of Nmax against d on a heuristic-recovery task would isolate that effect.
  • Inference: PENCIL's distributional permutation invariance provides a general template for canonicalizing query pairs in other dyadic prediction tasks (knowledge-graph completion, pair classification) without bespoke positional encodings.
  • Inference: the result suggests that much of the observed gap between GNNs and ID-based methods on link prediction may be attributable to how node identity is presented to the model—a bounded set of sampled-token vectors versus a global embedding table—rather than to the need for hand-designed structural features.
  • Inference: the fast convergence on ogbl-citation2 and ogbl-ppa and slower learning on Planetoid point to a testable prediction: pretraining PENCIL's input projection on structural heuristic-regression tasks should reduce the small-data gap; the paper leaves this untested.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes PENCIL, a Transformer-encoder architecture for link prediction. For each candidate pair (u,v), it samples a fixed-budget local subgraph, tokenizes the nodes with one-hot subgraph indices, adjacency rows, and role flags, and appends two task tokens for the endpoints. The model stacks BERT-style blocks; after each block, Eq. (2) adds a multiplicative residual P_k(Ã Z^(k)) built from the adjacency matrix reconstructed from the input tokenization. The paper reports strong results on Planetoid and OGB benchmarks under both the original and HeaRT protocols, claims state-of-the-art performance on several datasets, and provides a theoretical analysis connecting PENCIL to NBFNet, MPLP, heuristic estimation, and SEAL. A stated limitation in Sec. 7 is that a tight characterization of full-attention Transformer link predictors is still lacking, which is relevant when evaluating the theoretical claims.

Significance. If the empirical results hold, PENCIL is a meaningful contribution: it shows that a relatively simple architecture — a Transformer encoder over sampled subgraphs plus an explicit adjacency residual — can match or beat more complex heuristic- and ID-based link predictors on several large benchmarks, with far fewer parameters and good convergence behavior. The code release, multi-benchmark evaluation including HeaRT, and the small reported variances are strengths. However, the central 'plain Transformer' claim is not supported by the architecture as presented, and the theoretical reductions contain load-bearing mistakes. The work is therefore significant as a Transformer-plus-minimal-structural-prior link predictor, but the contribution as stated in the title and abstract is narrower and needs to be reframed.

major comments (4)
  1. [§3.2, Eq. (2); Appendix C.3, Table 9] The paper's central claim that PENCIL is a 'plain Transformer' that 'replaces hand-crafted priors' is not supported. Eq. (2) defines H^(k) = Z^(k) + P_k(Ã Z^(k)), an explicit adjacency-propagation branch that is a GCN-style message-passing operation, not a standard Transformer component. The ablation in Table 9 shows that removing this multiplicative residual degrades performance sharply (e.g., pubmed MRR drops from 38.28 to 21.49; ogbl-collab Hits@20 from 53.47 to 49.52), and Appendix C.3 states that 'an explicit structural prior is necessary for every layer.' The title and abstract therefore overstate the architecture. The authors should either reframe the contribution as a Transformer augmented with a multiplicative adjacency residual, or present a version without the residual that supports the 'plain Transformer' claim.
  2. [Appendix B.3, B.6; Propositions 4.2, 4.5] The proofs of Propositions 4.2 and 4.5 set T_k to map tokens to zeroes. Under Eq. (2), this makes Z^(k)=0 and hence H^(k)=P_k(÷0)=0, so the layerwise update does not reduce to any nontrivial source-conditioned MPNN. The intended choice appears to be T_k=Id, which is the setting used in the proof of Theorem 4.9 in Appendix B.8. Additionally, B.6 claims e_i can be chosen orthonormal when d≥|V|, but PENCIL operates on at most Nmax sampled tokens, with Nmax≪|V|; the relevant condition is d≥Nmax, not d≥|V|. These existence claims do not follow as written and must be corrected.
  3. [Appendix C.2; Table 2] For ogbl-ppa under HeaRT, the text says the authors used the optimal checkpoint identified in the original benchmark setting for the final HeaRT evaluation. Because HeaRT changes the validation negatives, selecting a checkpoint on the original validation split is not equivalent to selecting on the HeaRT validation split and can bias the reported MRR of 45.43. This matters because the paper's claim of top HeaRT performance on ogbl-ppa rests on this number. The authors should either rerun with HeaRT-appropriate checkpoint selection or provide evidence that the difference has no effect.
  4. [Appendix B.8; Theorem 4.9] The proof of Theorem 4.9 equates the sampled subgraph with the full m-hop enclosing neighborhood N_m(u,v). But PENCIL's fixed-budget sampler (e.g., (1,150) on ogbl-ppa) is not guaranteed to contain all nodes at distance ≤m; it is a random subsample. Consequently, the kϕ-kρ-m parameters m and the set over which the 1-|N_m|-WL guarantee is claimed need to be defined in terms of the actual sampled set, and the comparison to SEAL under the 'same sampling constraint' does not follow as written. The paper should also make explicit that this result concerns an LRP-augmented variant of PENCIL, not the model evaluated in Sec. 5.
minor comments (5)
  1. [§3.2, Fig. 3] The text says 'We omit the default sequential PE,' but the input includes one-hot node identifiers and role flags. Please clarify that these are not sequential positional encodings but token-identity and role signals.
  2. [Tables 5, 6] The entry '# Epochs 0.5' for ogbl-citation2 is confusing; report optimization steps or early-stopping checkpoints instead.
  3. [Figure 1] The caption contains placeholder star glyphs ('⋆⋆⋆...'); the figure should be re-rendered with the intended marker.
  4. [Appendix C.3] The sentence 'an explicit structural prior is necessary for every layer' should be reconciled with the abstract; as written, it confirms that the architecture is not plain.
  5. [Sec. 7] The conclusion correctly notes that a tighter theoretical characterization of full-attention Transformer link predictors is lacking; consider moving this caveat earlier, since the theoretical claims in Sec. 4 are mostly existence results.

Circularity Check

0 steps flagged

No significant circularity: PENCIL's empirical results are benchmark-based and the theoretical expressivity claims are transparent containment proofs relying on external theorems.

full rationale

The paper's central empirical claim is supported by direct comparisons on standard benchmarks (Tables 1-2) against external baselines; no fitted constant or trained parameter is subsequently relabeled as a prediction. The pairwise-heuristic study (Sec. 3.3) is explicitly a supervised regression probe with full-graph targets, not a claimed prediction of unseen links. The theoretical chain is also non-circular. Prop. 4.2 shows that by zeroing the attention branch, Eq. (2) degenerates to a source-conditioned MPNN; Cor. 4.3 then imports the generalized Bellman-Ford theorem from Zhu et al. (2021); Prop. 4.5 imports MPLP's unbiased-estimator results (Dong et al., 2024); Thm. 4.9 imports Lachi et al.'s kϕ-kρ-m framework and explicitly states it is a lower-bound/degeneration result, with the full-attention characterization left as future work. These cited theorems are external, parameter-free, and their assumptions do not include PENCIL's benchmark outcomes, so they count as independent support. The paper's self-citations (LPFormer, HeaRT, ContextGNN, GiGL) appear as baselines/context, not as the load-bearing justification for the main result. The mismatch between the 'plain Transformer' label and the multiplicative residual in Eq. (2), including the ablation showing the residual is necessary (Table 9), is a substantive correctness/novelty concern but not circularity: no derived quantity is equal to an input by construction. Therefore, no significant circularity is found.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 2 invented entities

The central method depends on a small number of tuned sampling and model hyperparameters rather than on fitted physical constants. The main assumed inputs are the sufficiency of local subgraphs, the quasi-orthogonality of random token vectors, and the transfer of external expressivity results. The invented entities are architectural components with direct empirical support.

free parameters (3)
  • Sampling configuration (depth, neighbors) = (2,20) on cora/citeseer, (1,150) on ogbl-ppa, (1,350) on ogbl-ddi, etc.
    Chosen per dataset to maximize validation performance; determines the subgraph budget and thus the information available to the model.
  • Model hyperparameters (layers, hidden size, heads, batch size, LR, epochs) = Varies by dataset; see Tables 5 and 6
    Tuned per dataset, though the paper describes tuning as 'minimal'; Figure 5 shows performance is sensitive to depth.
  • Katz damping beta and PageRank alpha in heuristic regression = beta=0.005, alpha=0.85
    Fixed target-generation choices for the heuristic-estimation proxy; they do not affect the main link-prediction results.
axioms (5)
  • domain assumption Sampled subgraphs of budget Nmax are sufficient for the link prediction tasks
    Core to the method; if links depend on global structure beyond the sampled neighborhood, the fixed-budget design fails. Invoked throughout Sections 3 and 5.
  • domain assumption The input projection's random vectors for the Nmax tokens are sufficiently well-separated (small mutual coherence) to preserve structural identity
    The Welch-bound argument in Section 4.2 assumes token vectors act like quasi-orthogonal signatures; if coherence is high, the common-neighbor estimation guarantee weakens.
  • standard math Known expressivity results of NBFNet, MPLP, and relational pooling transfer to PENCIL's encoding
    Corollary 4.3 and Theorem 4.9 rest on results from Zhu et al. 2021, Dong et al. 2024, and Zhou et al. 2023.
  • ad hoc to paper There exists a parameter setting with T_k = identity (or zero, as written) and a chosen input projection that makes PENCIL behave as an MPNN
    Propositions 4.2 and 4.5 require this construction; the paper is internally inconsistent about whether T_k is zero or identity, and the setting is not the trained model.
  • domain assumption HeaRT and original splits, and quoted baseline numbers, are comparable to PENCIL's runs
    Baselines are taken from Li et al. 2023 or original papers; any differences in tuning or hardware could affect comparisons.
invented entities (2)
  • Multiplicative residual connection P_k(ÃZ(k)) independent evidence
    purpose: Injects one-hop structural propagation into each Transformer layer
    Ablation in Table 9 shows large performance drops without it; it is an architectural component, not an unobserved entity.
  • Random token vectors induced by the frozen input projection W0 independent evidence
    purpose: Provides symmetry-breaking labels for sampled subgraph nodes
    Supported by the initialization study in Figure 7 and the Welch-bound analysis; not an unobserved entity.

pith-pipeline@v1.3.0-alltime-deepseek · 30167 in / 15621 out tokens · 163784 ms · 2026-08-03T05:36:58.731476+00:00 · methodology

0 comments
read the original abstract

Link prediction is a core challenge in graph machine learning, demanding models that capture rich and complex topological dependencies. While Graph Neural Networks (GNNs) are the standard solution, state-of-the-art pipelines often rely on explicit structural heuristics or memory-intensive node embeddings -- approaches that struggle to generalize or scale to massive graphs. Emerging Graph Transformers (GTs) offer a potential alternative but often incur significant overhead due to complex structural encodings, hindering their applications to large-scale link prediction. We challenge these sophisticated paradigms with PENCIL, an encoder-only plain Transformer that replaces hand-crafted priors with attention over sampled local subgraphs, retaining the scalability and hardware efficiency of standard Transformers. Through experimental and theoretical analysis, we show that PENCIL extracts richer structural signals than GNNs, implicitly generalizing a broad class of heuristics and subgraph-based expressivity. Empirically, PENCIL outperforms heuristic-informed GNNs and is far more parameter-efficient than ID-embedding--based alternatives, while remaining competitive across diverse benchmarks -- even without node features. Our results challenge the prevailing reliance on complex engineering techniques, demonstrating that simple design choices are potentially sufficient to achieve the same capabilities. Our code is publicly available at https://github.com/quang-truong/pencil.

Figures

Figures reproduced from arXiv: 2602.01553 by Donald Loveland, Jiliang Tang, Mingxuan Ju, Neil Shah, Quang Truong, Tong Zhao, Yu Song.

Figure 1
Figure 1. Figure 1: Parameter efficiency vs. performance on ogbl-ppa. PENCIL (⋆) achieves state-of-the-art performance without node IDs or handcrafted heuristics, using orders of magnitude fewer parameters than leading ID-based methods. Given its centrality in graph learning, Graph Neural Net￾works (GNNs) have become the de facto standard for this challenge. However, standard Message-Passing Neural Net￾works (MPNNs)—the prima… view at source ↗
Figure 2
Figure 2. Figure 2: Visualization of the input encoding scheme for a sampled subgraph revolving around a query link (dashed), with the number of nodes N = 5 and the sampling budget Nmax = 6. Blue block corresponds to the sampled nodes (the context set). Green block contains one-hot identifiers of nodes, yellow block contains nodes’ adjacency row, purple block contains role flags, and red block contains task tokens. Context-no… view at source ↗
Figure 3
Figure 3. Figure 3: PENCIL architecture. Node features can be optionally used as discussed in Section 3.2, but we omit them from the figure for clarity. tion h that, given the observed graph G, maps a candidate node pair (u, v) ∈ V × V to a real-valued score. Candidate pairs are then ranked by this score to estimate the likelihood that an edge exists between u and v. Common pairwise heuristics are summarized in Appendix C.1. … view at source ↗
Figure 4
Figure 4. Figure 4: reports RMSE on cora; additional setup details and citeseer results are provided in Appendix C.1. Based on [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Effect of model depth on different performance metrics across the cora, pubmed, and ogbl-collab datasets. 6. Related Work Link Prediction with GNNs. A long line of link predic￾tors augment a base MPNN with auxiliary structural signals. These include overlap-driven/local structure modules (e.g., NCN, Neo-GNN, BUDDY) (Wang et al., 2024; Yun et al., 2021; Chamberlain et al., 2023), explicit pairwise encodings… view at source ↗
Figure 6
Figure 6. Figure 6: RMSE of PENCIL and other GNNs for estimating pairwise heuristics on the citeseer dataset [PITH_FULL_IMAGE:figures/full_fig_p020_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: MRR averaged over 5 runs on the citeseer, cora, and pubmed datasets using different input embeddings’ initializations. C.5.2. TRAINING/INFERENCE TIME We benchmark per-mini-batch training and inference time for PENCIL and GAT on pubmed and ogbl-collab under identical sampling and model configurations. Using the same sampling configuration isolates how runtime depends on graph sparsity: ogbl-collab exhibits … view at source ↗
Figure 8
Figure 8. Figure 8: Batching time and memory comparison of Transformers and GNNs on the ogbl-collab dataset. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

14 extracted references · 2 canonical work pages

  1. [2]

    URL https: //doi.org/10.24963/ijcai.2021/291

    doi: 10.24963/IJCAI.2021/291. URL https: //doi.org/10.24963/ijcai.2021/291. Adamic, L. A. and Adar, E. Friends and neighbors on the web.Soc. Networks, 25:211–230, 2003. doi: 10.1016/ S0378-8733(03)00009-1. Alon, U. and Yahav, E. On the bottleneck of graph neural networks and its practical implications. InIn- ternational Conference on Learning Representations,

    work page doi:10.24963/ijcai.2021/291 2021

  2. [3]

    Baras, J

    URL https://openreview.net/forum? id=i80OPhOCVH2. Baras, J. S. and Theodorakopoulos, G. Path problems in net- works.Synthesis Lectures on Communication Networks, 3:1–77, 2010. Bodnar, C., Frasca, F., Wang, Y ., Otter, N., Montufar, G. F., Li´o, P., and Bronstein, M. Weisfeiler and Lehman go topological: Message passing simplicial networks. In Meila, M. an...

    Pith/arXiv arXiv 2010

  3. [4]

    Coupette, C., Wayland, J., Simons, E., and Rieck, B

    URL https://openreview.net/forum? id=Ua6zuk0WRH. Coupette, C., Wayland, J., Simons, E., and Rieck, B. No metric to rule them all: Toward principled evaluations of graph-learning datasets. InForty-second International Conference on Machine Learning, 2025. URL https: //openreview.net/forum?id=XbmBNwrfG5. Dao, T., Fu, D. Y ., Ermon, S., Rudra, A., and R´e, C...

    Pith/arXiv arXiv 2025

  4. [9]

    M¨uller, L., Galkin, M., Morris, C., and Ramp ´aˇsek, L

    URL http://proceedings.mlr.press/ v97/murphy19a.html. M¨uller, L., Galkin, M., Morris, C., and Ramp ´aˇsek, L. Attending to Graph Transformers.ArXiv preprint, abs/2302.04181, 2023. URL https://arxiv.org/ abs/2302.04181. Newman, M. E. J. Clustering and preferential at- tachment in growing networks.Phys. Rev. E, 64: 025102, Jul 2001. doi: 10.1103/PhysRevE.6...

    Pith/arXiv arXiv 2023

  5. [2009]

    Here, N(u) denotes the set of one-hop neighbors of node u, and deg(w) :=|N(w)| denotes the degree of node w

    discount common neighbors w with large degree, thereby emphasizing rarer shared neighbors. Here, N(u) denotes the set of one-hop neighbors of node u, and deg(w) :=|N(w)| denotes the degree of node w. The summations range over the shared neighbors w∈ N(u)∩ N(v). Since these scores are non-negative and typically right-skewed, a log transformation s7→log(1 +...

    1953

  6. [2017]

    Kreuzer, D., Beaini, D., Hamilton, W

    URL https://openreview.net/forum? id=SJU4ayYgl. Kreuzer, D., Beaini, D., Hamilton, W. L., L ´etourneau, V ., and Tossou, P. Rethinking graph transformers with spec- tral attention. InAdvances in Neural Information Process- ing Systems 34: Annual Conference on Neural Informa- tion Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pp. 216...

    Pith/arXiv arXiv 2021

  7. [2018]

    Wang, X., Yang, H., and Zhang, M

    URL https://openreview.net/forum? id=rJXMpikCZ. Wang, X., Yang, H., and Zhang, M. Neural common neigh- bor with completion for link prediction. InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024, 2024. Welch, L. Lower bounds on the maximum cross correlation of signals (corresp.).IEEE Transactions...

    Pith/arXiv arXiv 2024

  8. [2019]

    Murphy, R

    URL https://openreview.net/forum? id=Syx72jC9tm. Murphy, R. L., Srinivasan, B., Rao, V . A., and Ribeiro, B. Relational pooling for graph representations. In Chaudhuri, K. and Salakhutdinov, R. (eds.),Proceed- ings of the 36th International Conference on Ma- chine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 ofProceedings of...

    2019

  9. [2020]

    URL https://openreview.net/forum? id=S1ldO2EFPr. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., K¨opf, A., Yang, E., DeVito, Z., Rai- son, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S.PyTorch: an imperative 11 Plain Transformers ar...

    arXiv 2019

  10. [2021]

    Abboud, R., Ceylan, ˙I

    doi: 10.1186/S12859-021-04082-Y. Abboud, R., Ceylan, ˙I. ˙I., Grohe, M., and Lukasiewicz, T. The surprising power of graph neural networks with ran- dom node initialization. In Zhou, Z. (ed.),Proceedings of the Thirtieth International Joint Conference on Artifi- cial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, pp. 2112–2...

    work page doi:10.1186/s12859-021-04082-y 2021

  11. [2022]

    Truong, Q

    URL https://openreview.net/forum? id=7UmjRGzp-A. Truong, Q. and Chin, P. Weisfeiler and lehman go paths: Learning topological features via path complexes. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 38, pp. 15382–15391, 2024. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosu...

    2024

  12. [2023]

    Zhou, T., L¨u, L., and Zhang, Y .-C

    URL https://proceedings.mlr.press/ v202/zhou23n.html. Zhou, T., L¨u, L., and Zhang, Y .-C. Predicting missing links via local information.The European Physical Journal B - Condensed Matter and Complex Systems, 71:623–630, 10 2009. doi: 10.1140/epjb/e2009-00335-8. Zhu, Z., Zhang, Z., Xhonneux, L. A. C., and Tang, J. Neural bellman-ford networks: A general ...

    doi:10.1140/epjb/e2009-00335-8 2009

  13. [2024]

    Fey, M., Sunil, J., Nitta, A., Puri, R., Shah, M., Stojanoviˇc, B., Bendias, R., Barghi, A., Kocijan, V ., Zhang, Z., He, X., Lenssen, J

    URL https://proceedings.mlr.press/ v235/fey24a.html. Fey, M., Sunil, J., Nitta, A., Puri, R., Shah, M., Stojanoviˇc, B., Bendias, R., Barghi, A., Kocijan, V ., Zhang, Z., He, X., Lenssen, J. E., and Leskovec, J. Pyg 2.0: Scal- able learning on real world graphs. InTemporal Graph Learning Workshop @ KDD 2025, 2025. URL https: //openreview.net/forum?id=DHHL...

    Pith/arXiv arXiv 2025

  14. [8856]

    Ma, L., Lin, C., Lim, D., Romero-Soriano, A., Dokania, P

    URL https://openreview.net/forum? id=pHCdMat0gI. Ma, L., Lin, C., Lim, D., Romero-Soriano, A., Dokania, P. K., Coates, M., Torr, P. H. S., and Lim, S. Graph inductive biases in transformers without message passing. InInternational Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202, pp. 23321–23337, 2023. Ma, L., ...

    arXiv 2023