Toward a Unified Statistical Theory of Unsupervised Pretraining and Supervised Neural Knowledge Graph Learning

Jifan Zhang; Miklos Racz; Suqi Liu

REVIEW 3 major objections 4 minor 34 references

Toward a Unified Statistical Theory of Unsupervised Pretraining and Supervised Neural Knowledge Graph Learning

T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proves an end-to-end oracle inequality that decomposes downstream knowledge-graph prediction risk into four additive terms — neural approximation, supervised estimation, pretraining, and optimization — and shows that unsupervised

desk verdict A solid theory paper with a clean end-to-end bound; the main theorem is conditional on strong unverified structural assumptions, and the real-data evidence is thinner than the 'subsumes' claim suggests. read the letter →

arxiv 2607.26346 v1 pith:3HGWSRP7 submitted 2026-07-28 math.ST cs.SIstat.MEstat.MLstat.TH

Jifan Zhang , Miklos Racz , Suqi Liu This is my paper

classification math.STcs.SIstat.MEstat.MLstat.TH MSC 62H2562G2068T07

keywords knowledgegraphlearningunsupervisedpretrainingkernelPCAmulti-viewrepresentationoracleinequalityriskdecompositionsubspacerecoveryReLUnetworks

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the value of unsupervised pretraining for knowledge-graph prediction can be pinned down by a single four-term risk bound. In its two-stage framework, entity coordinates are first extracted from multiple unlabeled side-information views by weighted kernel PCA, then relation-wise ReLU networks are trained on labeled triples. The end-to-end oracle inequality splits downstream squared prediction risk into neural approximation, supervised estimation, subspace-recovery, and optimization error, and shows that pretraining reduces label complexity exactly when the subspace-recovery term is smaller than the balanced approximation–estimation rate. A sympathetic reader would care because this turns a long-empirical practice — augmenting sparse knowledge graphs with text-encoder embeddings — into a testable quantitative condition, including a view-weighting rule and a per-relation sample-complexity factor K.

What carries the argument

The load-bearing object is the weighted multi-view entity-kernel matrix K_obs(w)=Σ_s w_s K_obs_s, whose top-d eigenvectors are the frozen entity coordinates. It is analyzed under a shared-signal-eigenspace condition: every view's pure-signal kernel is assumed to be Z_sig Λ_s Z_sig^T plus a residual confined to the orthogonal complement. This reduces heterogeneous unlabeled views to a single kernel-PCA problem, and the paper's Theorem 1 is a nonasymptotic sin-theta perturbation bound on the recovered subspace under three kernel regimes (linear, Gaussian distance, nonlinear inner product). A second alignment condition states that this shared eigenspace equals the true latent coordinate span up

What would settle it

Take two views whose pure-signal Gram matrices have top-d eigenspaces with sin-theta distance bounded away from zero (for instance, one view's signal lives in only half the latent dimensions), or locate real encoder-induced kernels on a benchmark KG with a large principal-angle distance. If the subspace-recovery error then fails to follow the O(m^{-1/2}) rate of Theorem 1, or if downstream risk stops improving as unlabeled sample size m grows while the bound says it should, the shared-eigenspace assumption is violated and the end-to-end inequality does not apply.

Watch

Extended reading notes

Core claim

The central discovery is Theorem 2 (eq. 4.4): if all side-information views share a common rank-d entity eigenspace and that eigenspace aligns with the true latent coordinates of the scoring function, then the squared prediction risk of a pretrained neural knowledge-graph model is bounded by the sum of four terms — a ReLU approximation term (W/logW)^{-β/d}, a supervised estimation term of order K·D_net·W·log W / n, a pretraining term proportional to the squared sin-theta subspace-recovery error from Theorem 1, and an optimization term δ_opt. The pretraining term inherits the inverse effective information Σ_s g_s^2/v_s^2 and the O(m^{-1}+N^{-1}) rate of weighted multi-view KPCA. Consequently,

Load-bearing premise

The theory stands or falls on the assumption that all unlabeled side-information views share exactly one rank-d latent subspace and that this subspace is exactly the span of the true entity coordinates used by the relation-scoring function — real text-derived kernels need not satisfy either.

Editorial extensions

If this is right

Pretraining is provably not the bottleneck whenever the unlabeled sample is large enough to bring the subspace-recovery square error below the balanced approximation–estimation rate — the regime of abundant text, scarce labels.
The factor K multiplying the supervised estimation term makes precise a 'curse of relations': with many relation types, even perfect pretraining leaves a labeled-sample bottleneck unless the heads share parameters.
The signal-adjusted inverse-variance weighting rule w_s ∝ g_s/v_s^2 attains the aggregate information rate and reduces to sample-size weighting when views are comparable; the analysis licenses validation-based weight selection when view quality is heterogeneous.
The bound's three non-optimization terms can be probed independently by sweeping labeled sample size, pretraining dimension, and network width; the paper's simulations confirm the predicted m^{-1/2} subspace-recovery slope and the downstream bias-variance trade-off in width.
Under the alignment assumptions, multi-view pretraining strictly dominates graph-only learners in sample complexity, giving a formal justification for text-augmented knowledge-graph embeddings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Inference: A practical diagnostic follows — measure the principal-angle distance between the top eigenspaces of encoder-induced Gram matrices and a subspace recovered from a small labeled probe; when the distance is large, the bound predicts that more unlabeled data will not help, so pretraining efforts should shift to view selection or alignment.
Inference: The theory's zero-bias regime may mislead exactly in the complementary-signals case the paper flags: if views encode different aspects of the entity (one text, one ontology) rather than noisy copies of one subspace, the shared-eigenspace bias is nonzero and the pretraining term needs an additive misspecification term that the current bound omits.
Inference: A direct extension would allow the subspace-recovery event in Theorem 2 to hold with a tolerance term that scales with the distance from the nearest common eigenspace, converting the assumption into a bias-variance trade-off; the paper's own limitations section suggests this is the next step.
Inference: The same four-term decomposition likely transfers to other relational tasks with heterogeneous unlabeled side information (drug-target interaction, cross-lingual entity alignment), where the labeled-sample bottleneck and the K factor would play the same role.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take on arXiv:2607.26346. The real contribution is Theorem 2: an end-to-end oracle inequality that decomposes KG prediction risk into approximation, estimation, pretraining, and optimization terms. That is new, and the proof skeleton is coherent: Theorem 1 gives subspace recovery for weighted multi-view KPCA, Lemma 10 transfers coordinate alignment, and Yarotsky handles approximation. The synthetic experiments actually test the rates they claim, and the paper is unusually honest about its limitations — the Discussion explicitly notes frozen coordinates, approximate shared-subspace, and unknown weighting parameters.

The soft spots are concentrated in the gap between the theory and the real-data evaluation. Theorem 2 is stated under exact shared-eigenspace (Assumption 3) and latent-span alignment (Assumption 8). The synthetic experiments build those assumptions in by construction. The WordNet/PrimeKG sections never probe whether text-kernel eigenspaces actually share a low-dimensional subspace or whether that subspace lines up with the scoring function's latent coordinates. Without such a diagnostic, the paper's claim that pretraining helps 'exactly when' the subspace-recovery term is small is more a conditional statement than a measured one. The real-data gains over single views are consistently small (AUROC +0.003 to +0.009), and on PrimeKG overall RotatE is the best method; so the abstract's 'subsumes existing methods' is stronger than the evidence supports.

A lesser but notable issue: Lemma 7 imports Theorem 1 and Lemma 3 from Liu et al. [2024] — a paper this group co-authored — without proving them. That is not circular, but it makes the end-to-end bound depend on the correctness of a separate preprint. A referee should ask for a proof or a clear published reference.

None of this dissolves the contribution. The decomposition is real, the rates are consistent, and the appendix gives enough detail to verify the core inequalities. The right fix is to add a real-data check of the structural assumptions (e.g., eigenvalue gap behavior or alignment of top subspaces across views) and to soften the 'subsumes' language. I'd send it to a serious referee; the theory is worth the attention. I'd cite it if I were doing KG theory, and it would make an interesting reading-group session on what 'pretraining helps' means.

Referee Report

3 major / 4 minor

Summary. The paper proposes a two-stage 'Pretrained Neural Knowledge Graph' (PNKG) framework: first, multi-view entity side information is converted into kernel matrices, aggregated with view weights, and reduced by weighted KPCA to d-dimensional frozen entity coordinates; second, relation-wise bounded ReLU networks are trained on labeled triples using those coordinates. The main theoretical contribution is Theorem 2 (Eq. 4.4), an end-to-end oracle inequality that decomposes downstream squared prediction risk into four additive terms: ReLU approximation error, supervised estimation error, pretraining (subspace-recovery) error, and optimization error. Theorem 1 supplies the pretraining component, a nonasymptotic Davis--Kahan-type bound for weighted multi-view KPCA under information-plus-noise models, with explicit rates for linear, Gaussian-distance, and nonlinear inner-product kernels. Simulations on synthetic data check the predicted m^{-1/2} subspace-recovery rate, the behavior of inverse-variance view weighting, and the qualitative n-/m-/W-trends of the risk decomposition. Experiments on WordNet and PrimeKG report that fixed pretrained multi-view text-derived embeddings improve over graph-only baselines, with validation-selected view weights on PrimeKG.

Significance. If the main theorem is correct, the paper gives a clean and modular account of when unsupervised pretraining helps knowledge-graph prediction: pretraining is not the bottleneck once its error term falls below the balanced approximation--estimation rate, and the linear dependence on K quantifies the price of separate relation heads. The subspace-recovery result of Theorem 1 is of independent interest, as it extends weighted PCA perturbation theory to heterogeneous kernel views and provides a principled inverse-variance weighting principle. The paper also ships reproducible code and its simulations use ground-truth generators that directly test the stated rates, which is a strength. The main caveat is that the end-to-end transfer guarantee is conditional on exact shared-eigenspace and alignment assumptions that are acknowledged to hold only approximately in practice; the real-data evaluation does not verify these conditions.

major comments (3)

[Appendix A.2, Lemma 7] Lemma 7 is load-bearing: it supplies the supervised estimation term and the probability bound in Theorem 2. However, its proof is not self-contained; it invokes 'Theorem 1 of Liu et al. [2024]' without stating that theorem's hypotheses or verifying them for the relation-wise ReLU class F_{\hat Z}. Since the cited paper is a preprint with overlapping authorship, this is not an independent check. Please either provide a complete proof of Lemma 7 or state the exact conditions of the cited theorem and verify them for the present class, including the boundedness, noise, and entropy/pseudo-dimension requirements.
[Section 4.2, Assumption 8 and Eq. (4.4)] The transfer term in Theorem 2 relies on exact alignment: Assumption 3 requires all view signal kernels to share a common rank-d eigenspace, and Assumption 8 requires that eigenspace to coincide with the true latent span up to an invertible linear map. The Discussion explicitly acknowledges that (3.4) may hold only approximately when views encode complementary signals, but Theorem 2 has no misspecification bias term. The real-data experiments on WordNet and PrimeKG provide no diagnostic for either condition, so the empirical support for the 'pretraining is not the bottleneck' regime is indirect. I request one of two additions: (i) a robustness version of Lemma 10/Theorem 2 with an explicit alignment-bias term, or (ii) an empirical diagnostic on the real datasets, e.g., the distance between leading view eigenspaces or a validation-based test of whether the learned coordinates predict held
[Theorem 1 and Lemma 3, d_s scale] The proof of Lemma 3 derives \|D^{\rm diag}\|_F \le L_1\nu\sigma_\infty^2 / \sqrt{N}, i.e., the diagonal-correction scale d_{s,N,m_s} decays as N^{-1/2}. However, the display in Theorem 1 (and the Lemma 3 statement as printed) appears to show d_s \propto \sqrt{N}, which would be inconsistent with the proof. Please correct the displayed formula and ensure the notation is unambiguous; this affects the interpretation of the O(N^{-1}) rate in the text.

minor comments (4)

[Section 5.2.1, view-weight selection] The text says validation selects the weight vector (8:1:1) from a fixed candidate set for PrimeKG, but the candidate set and selection criterion are not specified. Please list the candidates and the validation metric used; otherwise the reported weights may be seen as validation-set overfitting.
[Section 3.2 / Assumption 7 / Lemma 10] The proof of Lemma 10 chooses R_* 'large enough' to contain both the true and transformed KPCA coordinates, but R_* is part of Assumption 7. Since the transformed coordinates depend on \hat Z, this should be stated as a condition on R_*, e.g., R_* \ge \max(\sup_j \|z^*_j\|, \kappa_A), before Theorem 2 is stated.
[Section 4.1, rate discussion] The statement 'the squared bias is of lower order than the variance term' is correct for the given scales, but the sentence conflates the unweighted per-view bias with the weighted sum. Making the bounded-parameter and B_{s,N}=O(1) assumptions explicit in one displayed display would help the reader.
[Section 5.2.3] The WordNet gain of multi-view over the best single view is numerically modest (AUROC +0.0028). I suggest adding a small statistical significance test or reporting confidence intervals over the 10 seeds, so the reader can judge whether the improvement is stable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Theorem 2 is a conditional oracle inequality whose components are either proved in this paper or imported from independent theorems with stated assumptions.

full rationale

Theorem 2 is a conditional statement: under Assumptions 1–8, the risk bound (4.4) is assembled from four independent ingredients—Theorem 1 (proved in Appendix A.1 via kernel-decomposition lemmas and the Davis–Kahan theorem), Lemma 11 (Yarotsky approximation), Lemma 10 (transfer of the oracle bias using Assumption 8), and Lemma 7 (a fixed-embedding oracle inequality). Lemma 7 cites Theorem 1 of Liu et al. [2024], a paper overlapping with author Suqi Liu; however, the imported result is a supervised-only oracle inequality with its own stated assumptions and does not contain the pretraining term, the shared-eigenspace condition, or the end-to-end claim of this paper. It is therefore independent support rather than a self-referential reduction. The pretraining term in (4.4) is exactly the subspace-recovery bound from Theorem 1, but this is a compositional step: the term measures ∥sin Θ(bZ, Zsig)∥²F, and Assumption 8 only aligns Zsig with Z⋆ in the hypothesis, not in the derivation. The real-data view weights are selected by validation, and the paper explicitly acknowledges that the weighting rules depend on unknown kernel parameters and that cross-validation with guarantees is future work; this is a stated limitation, not a fitted input renamed as a prediction. The paper also acknowledges that the shared-subspace condition may hold only approximately, which limits applicability but is not circular. No equation in the paper reduces to its own input by construction, and no fitted parameter is presented as a prediction. Hence no significant circularity.

Assumptions & free parameters 3 free parameters · 10 assumptions · 0 invented entities

The theoretical results are honest: all structural conditions are stated as assumptions, and no parameter is secretly fitted to make Theorem 1 or 2 hold. The free parameters (view weights, dimension, relation reweighting) affect the experimental pipeline but not the validity of the theorems. The main epistemic debt is the shared-subspace/alignment assumption, which is exactly where the theory could break on real data.

free parameters (3)

view weights w = uniform on WordNet; (8:1:1) for PubMedBERT:BioBERT:SciBERT on PrimeKG
Weights in the aggregated kernel K_obs(w) are selected by validation performance, not derived from data-free theory; the theory's inverse-variance rule depends on unknown kernel parameters.
embedding dimension d = 128 in real experiments; varies in simulations (d=4)
Choice of the number of leading eigenvectors; treated as a tuning parameter in Remark 2.
relation-balanced weights ω_r = inverse-frequency weights per relation
Used in the training objective (eq. 5.2); depends on label frequencies, so it is fit to the data.

assumptions (10)

domain assumption Assumption 1: entity vectors decompose into latent signal plus scaled noise
All theory depends on the information-plus-noise model (3.1).
domain assumption Assumption 2: admissible noise regimes (Lipschitz concentration or Gaussian)
The kernel perturbation bounds fail without these noise conditions.
domain assumption Assumption 3: shared effective signal eigenspace span(Z_sig)
Central structural condition (eq. 3.4); unrealistic for complementary views.
domain assumption Assumption 4: positive signal separation γ_s > 0 in every view
Needed for a non-vanishing eigengap in the Davis–Kahan argument.
domain assumption Assumption 5: independence across views and supervised sample
Used to sum per-view fluctuations in the subspace-recovery bound.
domain assumption Assumption 7: target is a Sobolev-smooth function of true latent coordinates
Required for the ReLU network approximation term in Lemma 11.
domain assumption Assumption 8: KPCA subspace aligns with the true latent span via a well-conditioned A_⋆
Ensures the pretraining representation transfers to the downstream task; untestable on real data.
standard math Davis–Kahan sin-theta theorem (Yu et al. 2015)
Converts the kernel perturbation bound into a subspace-distance bound.
standard math Yarotsky ReLU approximation theorem (Yarotsky 2017)
Provides the approximation error rate for the relation surfaces.
standard math Gaussian hypercontractivity (Nourdin et al. 2010)
Bounds higher moments of quadratic noise terms in the Gaussian kernel regime.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Toward a Unified Statistical Theory of Unsupervised Pretraining and Supervised Neural Knowledge Graph Learning." pith.science (2026). https://pith.science/paper/3HGWSRP7

@misc{pith2026260726346,
  author       = {Pith},
  title        = {Pith review of: Toward a Unified Statistical Theory of Unsupervised Pretraining and Supervised Neural Knowledge Graph Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3HGWSRP7}},
  note         = {Machine review of arXiv:2607.26346}
}

read the original abstract

Knowledge graph learning provides a powerful framework for representing and inferring structured knowledge, with broad practical applications. However, the scarcity of relation-specific labeled triples per entity hinders the training of expressive models, and the ad hoc design of scoring functions limits generalizability and lacks theoretical grounding. We address both issues with a theoretically grounded, end-to-end training framework that extends and subsumes existing methods. Our framework is a two-stage procedure: unsupervised pretraining over heterogeneous corpora followed by supervised learning with multiple relation types. We establish a nonasymptotic risk bound that disentangles pretraining representation error from labeled-sample complexity, formally quantifying the benefit of large-scale unlabeled data for downstream knowledge prediction. Synthetic experiments validate each theoretical component, and real-world experiments confirm the effectiveness of our approach on large-scale knowledge graph benchmarks.

Figures

Figures reproduced from arXiv: 2607.26346 by the authors.

**Figure 2.** End-to-end risk decomposition (Experiment 3). The [PITH_FULL_IMAGE:figures/full_fig_p018_2.png] view at source ↗

**Figure 3.** Random edge-split results on WordNet with relation-balanced training/evaluation and [PITH_FULL_IMAGE:figures/full_fig_p021_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Hard-relation-focused results on PrimeKG with relation-balanced training/evaluation [PITH_FULL_IMAGE:figures/full_fig_p021_4.png]

**Figure 5.** Figure 5: Overall results on PrimeKG with relation-balanced training/evaluation and fixed entity [PITH_FULL_IMAGE:figures/full_fig_p022_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 4 linked inside Pith

[1]

A review of biomedical datasets relating to drug discovery: a knowledge graph perspective

Stephen Bonner, Ian P Barrett, Cheng Ye, Rowan Swiers, Ola Engkvist, Andreas Bender, Charles Tapley Hoyt, and William L Hamilton. A review of biomedical datasets relating to drug discovery: a knowledge graph perspective. Briefings in Bioinformatics, 23 0 (6): 0 bbac404, 2022

2022
[2]

Translating embeddings for modeling multi-relational data

Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. Advances in Neural Information Processing Systems, 26, 2013

2013
[3]

Building a knowledge graph to enable precision medicine

Payal Chandak, Kexin Huang, and Marinka Zitnik. Building a knowledge graph to enable precision medicine. Scientific Data, 10 0 (1): 0 67, 2023

2023
[4]

Kernel methods for deep learning

Youngmin Cho and Lawrence Saul. Kernel methods for deep learning. Advances in neural information processing systems, 22, 2009

2009
[5]

Inductive entity representations from text via link prediction

Daniel Daza, Michael Cochez, and Paul Groth. Inductive entity representations from text via link prediction. In Proceedings of The Web Conference, pages 798--808, 2021

2021
[6]

The spectrum of kernel random matrices

Noureddine El Karoui. The spectrum of kernel random matrices. The Annals of Statistics, 38 0 (1): 0 1--50, 2010

2010
[7]

Distributed estimation of principal eigenspaces

Jianqing Fan, Dong Wang, Kaizheng Wang, and Ziwei Zhu. Distributed estimation of principal eigenspaces. Annals of statistics, 47 0 (6): 0 3009, 2019

2019
[8]

Remainder formulae in taylor's theorem

William J Firey. Remainder formulae in taylor's theorem. The American Mathematical Monthly, 67 0 (9): 0 903--905, 1960

1960

Show all 34 references

[9]

On the provable advantage of unsupervised pretraining

Jiawei Ge, Shange Tang, Jianqing Fan, and Chi Jin. On the provable advantage of unsupervised pretraining. In The Twelfth International Conference on Learning Representations, 2024

2024
[10]

HaoChen, Colin Wei, Adrien Gaidon, and Tengyu Ma

Jeff Z. HaoChen, Colin Wei, Adrien Gaidon, and Tengyu Ma. Provable guarantees for self-supervised deep learning with spectral contrastive loss. In Advances in Neural Information Processing Systems, volume 34, pages 5000--5011, 2021

2021
[11]

Hoff, Adrian E

Peter D. Hoff, Adrian E. Raftery, and Mark S. Handcock. Latent space approaches to social network analysis. Journal of the American Statistical Association, 97 0 (460): 0 1090--1098, 2002

2002
[12]

Knowledge graph embedding based question answering

Xiao Huang, Jingyuan Zhang, Dingcheng Li, and Ping Li. Knowledge graph embedding based question answering. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining , pages 105--113, 2019

2019
[13]

A survey on knowledge graphs: Representation, acquisition, and applications

Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, and Philip S Yu. A survey on knowledge graphs: Representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems , 33 0 (2): 0 494--514, 2021

2021
[14]

Lee, Qi Lei, Nikunj Saunshi, and Jiacheng Zhuo

Jason D. Lee, Qi Lei, Nikunj Saunshi, and Jiacheng Zhuo. Predicting what you already know helps: Provable self-supervised learning. In Advances in Neural Information Processing Systems, volume 34, pages 309--323, 2021

2021
[15]

Representation-enhanced neural knowledge integration with application to large-scale medical ontology learning

Suqi Liu, Tianxi Cai, and Xiaoou Li. Representation-enhanced neural knowledge integration with application to large-scale medical ontology learning. arXiv preprint arXiv :2410.07454 , 2024

2024 arXiv
[16]

MacDonald, Elizaveta Levina, and Ji Zhu

Peter W. MacDonald, Elizaveta Levina, and Ji Zhu. Latent space models for multiplex networks with shared structure. Biometrika, 109 0 (3): 0 683--706, 2022

2022
[17]

WordNet : a lexical database for english

George A Miller. WordNet : a lexical database for english. Communications of the ACM , 38 0 (11): 0 39--41, 1995

1995
[18]

A review of relational machine learning for knowledge graphs

Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. A review of relational machine learning for knowledge graphs. Proceedings of the IEEE , 104 0 (1): 0 11--33, 2015

2015
[19]

Invariance principles for homogeneous sums: universality of gaussian wiener chaos

Ivan Nourdin, Giovanni Peccati, and Gesine Reinert. Invariance principles for homogeneous sums: universality of gaussian wiener chaos. The Annals of Probability, 38 0 (5): 0 1947--1985, 2010. doi:10.1214/10-AOP531

1947 doi
[20]

A theoretical analysis of contrastive unsupervised representation learning

Nikunj Saunshi, Orestis Plevrakis, Sanjeev Arora, Mikhail Khodak, and Hrishikesh Khandeparkar. A theoretical analysis of contrastive unsupervised representation learning. In International Conference on Machine Learning, pages 5628--5637. PMLR , 2019

2019
[21]

RotatE : Knowledge graph embedding by relational rotation in complex space

Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. RotatE : Knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv :1902.10197 , 2019

1902 arXiv
[22]

Complex embeddings for simple link prediction

Th \'e o Trouillon, Johannes Welbl, Sebastian Riedel, \'E ric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In International Conference on Machine Learning, pages 2071--2080. PMLR , 2016
[23]

Structure-augmented text representation learning for efficient knowledge graph completion

Bo Wang, Tao Shen, Guodong Long, Tianyi Zhou, Ying Wang, and Yi Chang. Structure-augmented text representation learning for efficient knowledge graph completion. In Proceedings of The Web Conference, pages 1737--1748, 2021 a

2021
[24]

SimKGC : Simple contrastive knowledge graph completion with pre-trained language models

Liang Wang, Wei Zhao, Zhuoyu Wei, and Jingming Liu. SimKGC : Simple contrastive knowledge graph completion with pre-trained language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4281--4294, 2022

2022
[25]

KEPLER : A unified model for knowledge embedding and pre-trained language representation

Xiaozhi Wang, Tianyu Gao, Zhaocheng Zhu, Zhengyan Zhang, Zhiyuan Liu, Juanzi Li, and Jian Tang. KEPLER : A unified model for knowledge embedding and pre-trained language representation. Transactions of the Association for Computational Linguistics, 9: 0 176--194, 2021 b

2021
[26]

Deep kernel learning

Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov, and Eric P Xing. Deep kernel learning. In Artificial intelligence and statistics, pages 370--378. PMLR, 2016

2016
[27]

Representation learning of knowledge graphs with entity descriptions

Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, and Maosong Sun. Representation learning of knowledge graphs with entity descriptions. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 30, 2016

2016
[28]

Inference for heteroskedastic pca with missing data

Yuling Yan, Yuxin Chen, and Jianqing Fan. Inference for heteroskedastic pca with missing data. The Annals of Statistics, 52 0 (2): 0 729--756, 2024

2024
[29]

Embedding entities and relations for learning and inference in knowledge bases

Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv :1412.6575 , 2014

2014 arXiv
[30]

A survey of information extraction based on deep learning

Yang Yang, Zhilei Wu, Yuexiang Yang, Shuangshuang Lian, Fengjie Guo, and Zhiwei Wang. A survey of information extraction based on deep learning. Applied Sciences, 12 0 (19): 0 9691, 2022

2022
[31]

KG-BERT : BERT for knowledge graph completion

Liang Yao, Chengsheng Mao, and Yuan Luo. KG-BERT : BERT for knowledge graph completion. arXiv preprint arXiv :1909.03193 , 2019

1909 arXiv
[32]

Error bounds for approximations with deep ReLU networks

Dmitry Yarotsky. Error bounds for approximations with deep ReLU networks. Neural Networks, 94: 0 103--114, 2017

2017
[33]

A useful variant of the Davis--Kahan theorem for statisticians

Yi Yu, Tengyao Wang, and Richard J Samworth. A useful variant of the Davis--Kahan theorem for statisticians. Biometrika, 102 0 (2): 0 315--323, 2015

2015
[34]

Collaborative knowledge base embedding for recommender systems

Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei-Ying Ma. Collaborative knowledge base embedding for recommender systems. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 353--362, 2016

2016

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [1]

A review of biomedical datasets relating to drug discovery: a knowledge graph perspective

Stephen Bonner, Ian P Barrett, Cheng Ye, Rowan Swiers, Ola Engkvist, Andreas Bender, Charles Tapley Hoyt, and William L Hamilton. A review of biomedical datasets relating to drug discovery: a knowledge graph perspective. Briefings in Bioinformatics, 23 0 (6): 0 bbac404, 2022

2022

[2] [2]

Translating embeddings for modeling multi-relational data

Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. Advances in Neural Information Processing Systems, 26, 2013

2013

[3] [3]

Building a knowledge graph to enable precision medicine

Payal Chandak, Kexin Huang, and Marinka Zitnik. Building a knowledge graph to enable precision medicine. Scientific Data, 10 0 (1): 0 67, 2023

2023

[4] [4]

Kernel methods for deep learning

Youngmin Cho and Lawrence Saul. Kernel methods for deep learning. Advances in neural information processing systems, 22, 2009

2009

[5] [5]

Inductive entity representations from text via link prediction

Daniel Daza, Michael Cochez, and Paul Groth. Inductive entity representations from text via link prediction. In Proceedings of The Web Conference, pages 798--808, 2021

2021

[6] [6]

The spectrum of kernel random matrices

Noureddine El Karoui. The spectrum of kernel random matrices. The Annals of Statistics, 38 0 (1): 0 1--50, 2010

2010

[7] [7]

Distributed estimation of principal eigenspaces

Jianqing Fan, Dong Wang, Kaizheng Wang, and Ziwei Zhu. Distributed estimation of principal eigenspaces. Annals of statistics, 47 0 (6): 0 3009, 2019

2019

[8] [8]

Remainder formulae in taylor's theorem

William J Firey. Remainder formulae in taylor's theorem. The American Mathematical Monthly, 67 0 (9): 0 903--905, 1960

1960