REVIEW 3 major objections 4 minor 29 references
Knowledge Graph Completion by Intermediate Variables Regularization
T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper proposes a regularizer that bounds tensor rank for knowledge graph completion models and shows empirical gains.
desk verdict New regularizer with real empirical gains, but the proof of the trace-norm bound is false for α>2, and that's the regime used in the main experiments. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The key object is the overlapped trace norm L(X;α) = ‖X(1)‖_∗^{α/2} + ‖X(2)‖_∗^{α/2} + ‖X(3)‖_∗^{α/2}, the sum of trace norms of the three mode-unfoldings of the predicted tensor, which is used as a convex surrogate for tensor rank. The proof machinery is Lemma 1, an identity for the α-th power of the trace norm of a matrix Z as a minimum, over factorizations Z = UV^T, of a balanced combination of the 2α-th powers of the Frobenius norms of U and V. Applying that identity to each mode-unfolding of X turns the overlapped trace norm into sums of Frobenius norms of exactly the intermediate variables that IVR penalizes, which is what lets the paper claim that IVR is an upper bound on L(X;α).
What would settle it
A concrete counterexample with X ∈ $R^{{1×1×1}}$, the value X = 2, and a CP decomposition with two components (D = 2) at α = 4 gives a left-hand side of 24 against a right-hand side of 12 in the key inequality, contradicting Proposition 1; equally, computing L(X;α) and the IVR penalty for a randomly generated tensor at α = 3.0 and checking whether the claimed inequality holds would settle the matter.
Extended reading notes
Core claim
The central claim is that minimizing the norms of intermediate variables in the many ways of computing the tensor decomposition output — the embeddings themselves, their pairwise Kronecker products, the core tensor applied to one mode, and the core tensor applied to two modes — provides a tractable uniform regularizer for TDB models. The theoretical support is Proposition 1: for any decomposition of the predicted tensor, the overlapped trace norm L(X;α) is upper-bounded, up to a constant, by the IVR terms that pair embedding norms with norms of the two-mode products W×2R×3T, W×3T×1H, W×1H×2R. The regularizer also includes the complementary pair of terms (products of embedding norms with one-mode products), giving the full IVR objective. Empirically, IVR improves mean reciprocal rank over the F2, N3, and DURA baselines for the tested models, with the largest gains on models (SimplE, ANALOGY, QuatE, TuckER) that previously had no strong specialized regularizer.
Load-bearing premise
The theoretical guarantee depends on the inequality that the α/2-th power of a trace norm of a sum of matrices is bounded by the sum of the α/2-th powers of the summands, which fails when α > 2 and the bounds used in the main experiments with α > 2 are therefore not covered by the proof as written.
Editorial extensions
If this is right
- IVR gives the first uniform regularizer that applies to most tensor-decomposition KGC models, not only CP-derived models like N3 and DURA.
- If the theoretical bound holds, minimizing IVR minimizes a rank surrogate of the predicted tensor, yielding a principled and not merely heuristic form of overfitting control.
- The ablation shows that the bound in Proposition 1 (IVR-1) alone nearly matches the full IVR, so users can trade a single regularization coefficient against slightly lower performance in cases where hyperparameter budget is tight.
- The framework suggests new TDB models could be designed by choosing the partition P and core tensor W while inheriting IVR as a ready-made regularizer.
Reading between the lines
- A testable extension: compute L(X;α) directly for a trained model and compare the value against the IVR penalty; the paper does this only on the small Kinship dataset, and doing it on WN18RR with randomized subsampling of unfoldings could check the bound's practical tightness at scale.
- The regularizer's correlation narrative suggests it should most help datasets with many symmetric or inverse relation pairs and well-clustered entities; an experiment that correlates the MRR gain of IVR with the prevalence of such relation patterns across datasets could sharpen when to use it.
- If Proposition 1 fails for α > 2 as a counterexample indicates, the practical gains from IVR may still be real but would need a different theoretical justification, for instance that the penalty favors small norms rather than strictly small rank.
- The general-form unification in Eq. (2) could be used to instantiate new models with custom core tensors, with IVR as a ready-made regularizer.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Intermediate Variables Regularization (IVR), a regularizer for tensor-decomposition-based (TDB) knowledge graph completion models. The authors unify CP, ComplEx, SimplE, ANALOGY, QuatE, and TuckER under a general block-term form (Eq. 2), and define IVR as a weighted sum of Frobenius-norm powers over the intermediate variables that arise in different ways of computing the predicted tensor. The central theoretical claim is that IVR upper-bounds the overlapped trace norm L(X; α) (Propositions 1 and 2), thereby promoting low tensor rank and reducing overfitting. Experiments on WN18RR, FB15k-237, and YAGO3-10 show consistent MRR improvements over F2, N3, and DURA baselines, and a Kinship experiment measures reductions in L(X; α).
Significance. If the theoretical claim were sound, IVR would be a valuable and broadly applicable regularizer, since it covers several model families and outperforms existing regularizers in Table 1. The paper also contributes a unified view of TDB models (Eq. 2) and a logical-rule analysis (Appendix A), which are useful organizational results. The code release and the consistent empirical gains are strengths. However, the main advertised contribution is the theoretical justification, and the proof of the central upper bound is invalid in the experimental regime that the paper actually uses. The empirical results alone, while suggestive, do not compensate for a false theorem at the core of the paper.
major comments (3)
- [Appendix B, proof of Proposition 1 (Eq. 7)] The proof uses the step 2L(X;α) ≤ Σ_d ||H:d:(W(1)(T:d:⊗R:d:)^T)||_*^{α/2} + ... , which relies on the inequality (Σ_d a_d)^p ≤ Σ_d a_d^p for p = α/2. This inequality holds only for p ≤ 1; for p > 1 the direction is reversed. The main experiments use α = 3.0 or 2.25 for every model except TuckER (Table 6), so p > 1 throughout the reported experimental regime. The proposition is false as stated: for the 1×1×1 tensor X = 2 with a CP decomposition (W = 1, D = 2, H:d: = R:d: = T:d: = [1] for d = 1,2), α = 4, and λ1 = λ4 = 1, the left side of Eq. (5) is 2√(λ1λ4)L(X;4) = 2(4+4+4) = 24, while the right side is 6 + 6 = 12, giving 24 ≤ 12. The same power inequality appears in the proof of Proposition 2 (after Eq. (8)), so both upper-bound results are unsupported for α > 2.
- [Appendix B, tightness construction in Proposition 1] The claimed equality construction is dimensionally inconsistent with the model class in Eq. (2). The proof sets H:1: = (λ1/λ4)^{-1/α} U1√Σ1 with U1 ∈ R^{n1×r1}, and analogously R:1:, T:1:, and W ∈ R^{r1×r2×r3}. In the general form Eq. (2), however, the factors must satisfy H:d: ∈ R^{n1×P}, R:d: ∈ R^{n2×P}, T:d: ∈ R^{n3×P}, and W ∈ R^{P×P×P}, where P is the number of parts. The proof states 'Let P = D', but the n-ranks r1, r2, r3 are determined by the tensor X and are not equal to the embedding dimension D in general. Thus the proposed decomposition does not belong to the model class, and the claimed tightness of Eq. (5) is not established.
- [Section 4.4 and Table 3] The empirical verification of the upper bound on Kinship uses TuckER only, and Table 6 reports α = 2.0 for TuckER. At α = 2 the power inequality is an equality, so this experiment tests the theorem only in the regime where it is valid. It does not provide any evidence for the α > 2 settings used for CP, ComplEx, SimplE, ANALOGY, and QuatE in Table 1, where the theorem is false. The sentence in Section 4.4 that the results 'confirm the reliability of our theoretical analysis' is therefore not supported for the main experimental configuration.
minor comments (4)
- [Section 3.3] In the paragraph after Proposition 2, the text says 'the r.h.s. of Eq.(4) is an upper bound'; the reference should be to Eq.(6), since Eq.(4) is the weighted regularizer, not the bound in Proposition 2.
- [Table 4 and Section 4.1] The dataset name is written inconsistently as 'YGAO3-10' in Table 1 and Table 4 and as 'YAGO3-10' in the text of Section 4.1; please unify the spelling.
- [Appendix B, Lemma 1] In the proof of Lemma 1, the expression '∥Z∥α∗ = Tr(Σ)α' should be parenthesized as '(Tr(Σ))^α' to avoid confusion with Tr(Σ^α).
- [Section 3.2] The notation H:d: is used for the d-th slice of the reshaped embedding, but in Eq. (4) the slices are written as Hid:, Rjd:, Tkd: for a single triplet; the relationship between these notations could be stated explicitly for readability.
Circularity Check
No significant circularity: the IVR upper-bound theorem is derived from an external matrix-factorization lemma, and the experimental check measures the target trace norm independently.
full rationale
The paper's central derivation is Proposition 1 (Eq. 5), which claims that the IVR regularizer (Eq. 3) upper-bounds the overlapped trace norm L(X; alpha). The proof in Appendix B reduces this to Lemma 1, credited to Ciliberto et al. (2017) and re-proved in the appendix using SVD and norm inequalities. This lemma is an external, parameter-free factorization result; it is not defined in terms of L(X; alpha) or the IVR regularizer, and the proposition does not fit any parameter to the trace norm. The equality half of Proposition 1 is an explicit existence construction using compact SVDs, not a fitted input disguised as a prediction. The Kinship experiments (Table 3) compute L(X; alpha) directly from the trained predicted tensor, and this quantity is not part of the training loss; it is a post-hoc measurement, so the result is not statistically forced by construction. The paper contains no load-bearing self-citations: the proof relies on Ciliberto et al. (2017) and standard tensor literature, not on the present authors' prior work. The general form (Eq. 2) is presented as a unifying summary of known TDB models, but the regularization and the upper-bound analysis do not depend on renaming a known result as a new prediction. A separate mathematical concern exists: the step in Appendix B replacing (sum of trace norms)^(alpha/2) by the sum of the alpha/2-th powers is invalid for alpha/2 > 1, and Table 6 uses alpha > 2 for most models. That is a correctness or soundness issue in the proof, not a circularity: the claimed bound does not reduce by definition to its own inputs, and the counterexample does not show that any fitted parameter was renamed as a prediction. Therefore, under the circularity definition used here, the paper is self-contained against external benchmarks and receives a score of 0.
Assumptions & free parameters
free parameters (4)
- alpha (power coefficient) =
3.0 on WN18RR for CP, ComplEx, SimplE, ANALOGY, QuatE; 2.25 on FB15k-237; 2.0 for TuckER; see Table 6
- lambda1 = lambda3 =
e.g., 0.05 for CP on WN18RR, 0.005 for ComplEx on FB15k-237, 0.01 for TuckER on WN18RR; see Table 6
- lambda2 = lambda4 =
e.g., 0.07 for CP on WN18RR, 0.05 for ComplEx on FB15k-237, 0.03 for TuckER on WN18RR; see Table 6
- embedding dimension D and number of parts P =
D=2000/4000/256, P=1/2/4/256 depending on model and dataset; see Table 5
assumptions (4)
- standard math Lemma 1 from Ciliberto et al. (2017): ||Z||_*^alpha = min_{Z=UV^T} 1/2 (lambda||U||_F^{2alpha} + (1/lambda)||V||_F^{2alpha}).
- ad hoc to paper The paper implicitly assumes ||sum_d A_d||_*^{alpha/2} <= sum_d ||A_d||_*^{alpha/2} for alpha/2 >= 1.
- ad hoc to paper In Proposition 1's tightness construction, H:1: is set to an n1 x r1 SVD factor while the general form requires H:d: to be n1 x P, and W is set to r1 x r2 x r3 while it should be P x P x P.
- domain assumption Low overlapped trace norm L(X;alpha) corresponds to high entity and relation correlation and reduces overfitting.
Cite this review
Pith. "Pith review of Knowledge Graph Completion by Intermediate Variables Regularization." pith.science (2026). https://pith.science/paper/ST2MFSZ5
@misc{pith2026250602749,
author = {Pith},
title = {Pith review of: Knowledge Graph Completion by Intermediate Variables Regularization},
year = {2026},
howpublished = {\url{https://pith.science/paper/ST2MFSZ5}},
note = {Machine review of arXiv:2506.02749}
}
read the original abstract
Knowledge graph completion (KGC) can be framed as a 3-order binary tensor completion task. Tensor decomposition-based (TDB) models have demonstrated strong performance in KGC. In this paper, we provide a summary of existing TDB models and derive a general form for them, serving as a foundation for further exploration of TDB models. Despite the expressiveness of TDB models, they are prone to overfitting. Existing regularization methods merely minimize the norms of embeddings to regularize the model, leading to suboptimal performance. Therefore, we propose a novel regularization method for TDB models that addresses this limitation. The regularization is applicable to most TDB models and ensures tractable computation. Our method minimizes the norms of intermediate variables involved in the different ways of computing the predicted tensor. To support our regularization method, we provide a theoretical analysis that proves its effect in promoting low trace norm of the predicted tensor to reduce overfitting. Finally, we conduct experiments to verify the effectiveness of our regularization technique as well as the reliability of our theoretical analysis. The code is available at https://github.com/changyi7231/IVR.
Figures
Reference graph
Works this paper leans on
-
[1]
Templates for the solution of algebraic eigenvalue problems: a practical guide
Zhaojun Bai, James Demmel, Jack Dongarra, Axel Ruhe, and Henk van der Vorst. Templates for the solution of algebraic eigenvalue problems: a practical guide. SIAM, 2000
work page 2000
-
[2]
Tucker: Tensor factorization for knowledge graph completion
Ivana Bala z evi \'c , Carl Allen, and Timothy Hospedales. Tucker: Tensor factorization for knowledge graph completion. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5185--5194, 2019
work page 2019
-
[3]
Algorithms for hyper-parameter optimization
James Bergstra, R \'e mi Bardenet, Yoshua Bengio, and Bal \'a zs K \'e gl. Algorithms for hyper-parameter optimization. Advances in neural information processing systems, 24, 2011
2011
-
[4]
Translating embeddings for modeling multi-relational data
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. Advances in neural information processing systems, 26, 2013
work page 2013
-
[5]
Reexamining Low Rank Matrix Factorization for Trace Norm Regularization
Carlo Ciliberto, Dimitris Stamos, and Massimiliano Pontil. Reexamining low rank matrix factorization for trace norm regularization. arXiv preprint arXiv:1706.08934, 2017
work page Pith review arXiv 2017
-
[6]
Decompositions of a higher-order tensor in block terms—part i: Lemmas for partitioned matrices
Lieven De Lathauwer. Decompositions of a higher-order tensor in block terms—part i: Lemmas for partitioned matrices. SIAM Journal on Matrix Analysis and Applications, 30 0 (3): 0 1022--1032, 2008
work page 2008
-
[7]
Convolutional 2d knowledge graph embeddings
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. In Thirty-second AAAI conference on artificial intelligence, 2018
work page 2018
-
[8]
Adaptive subgradient methods for online learning and stochastic optimization
John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research, 12 0 (7), 2011
2011
Show all 29 references
-
[9]
Robust low-rank tensor recovery: Models and algorithms
Donald Goldfarb and Zhiwei Qin. Robust low-rank tensor recovery: Models and algorithms. SIAM Journal on Matrix Analysis and Applications, 35 0 (1): 0 225--253, 2014
2014
-
[10]
The expression of a tensor or a polyadic as a sum of products
Frank L Hitchcock. The expression of a tensor or a polyadic as a sum of products. Journal of Mathematics and Physics, 6 0 (1-4): 0 164--189, 1927
1927
-
[11]
A survey on knowledge graphs: Representation, acquisition, and applications
Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, and S Yu Philip. A survey on knowledge graphs: Representation, acquisition, and applications. IEEE transactions on neural networks and learning systems, 33 0 (2): 0 494--514, 2021
2021
-
[12]
Simple embedding for link prediction in knowledge graphs
Seyed Mehran Kazemi and David Poole. Simple embedding for link prediction in knowledge graphs. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pages 4289--4300, 2018
2018
-
[13]
Statistical predicate invention
Stanley Kok and Pedro Domingos. Statistical predicate invention. In Proceedings of the 24th international conference on Machine learning, pages 433--440, 2007
2007
-
[14]
Tensor decompositions and applications
Tamara G Kolda and Brett W Bader. Tensor decompositions and applications. SIAM review, 51 0 (3): 0 455--500, 2009
2009
-
[15]
Canonical tensor decomposition for knowledge base completion
Timoth \'e e Lacroix, Nicolas Usunier, and Guillaume Obozinski. Canonical tensor decomposition for knowledge base completion. In International Conference on Machine Learning, pages 2863--2872. PMLR, 2018
2018
-
[16]
Analogical inference for multi-relational embeddings
Hanxiao Liu, Yuexin Wu, and Yiming Yang. Analogical inference for multi-relational embeddings. In International conference on machine learning, pages 2168--2178. PMLR, 2017
2017
-
[17]
Tensor robust principal component analysis: Exact recovery of corrupted low-rank tensors via convex optimization
Canyi Lu, Jiashi Feng, Yudong Chen, Wei Liu, Zhouchen Lin, and Shuicheng Yan. Tensor robust principal component analysis: Exact recovery of corrupted low-rank tensors via convex optimization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pag...
2016
-
[18]
Square deal: Lower bounds and improved relaxations for tensor recovery
Cun Mu, Bo Huang, John Wright, and Donald Goldfarb. Square deal: Lower bounds and improved relaxations for tensor recovery. In International conference on machine learning, pages 73--81. PMLR, 2014
2014
-
[19]
A three-way model for collective learning on multi-relational data
Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. A three-way model for collective learning on multi-relational data. In Proceedings of the 28th International Conference on International Conference on Machine Learning, pages 809--816, 2011
2011
-
[20]
Holographic embeddings of knowledge graphs
Maximilian Nickel, Lorenzo Rosasco, and Tomaso Poggio. Holographic embeddings of knowledge graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016
2016
-
[21]
Tensor completion algorithms in big data analytics
Qingquan Song, Hancheng Ge, James Caverlee, and Xia Hu. Tensor completion algorithms in big data analytics. ACM Transactions on Knowledge Discovery from Data (TKDD), 13 0 (1): 0 1--48, 2019
2019
-
[22]
Statistical performance of convex tensor decomposition
Ryota Tomioka, Taiji Suzuki, Kohei Hayashi, and Hisashi Kashima. Statistical performance of convex tensor decomposition. Advances in neural information processing systems, 24, 2011
2011
-
[23]
Representing text for joint embedding of text and knowledge bases
Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. Representing text for joint embedding of text and knowledge bases. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 1499--1509, 2015
2015
-
[24]
Knowledge graph completion via complex tensor factorization
Th \'e o Trouillon, Christopher R Dance, \'E ric Gaussier, Johannes Welbl, Sebastian Riedel, and Guillaume Bouchard. Knowledge graph completion via complex tensor factorization. Journal of Machine Learning Research, 18: 0 1--38, 2017
2017
-
[25]
Some mathematical notes on three-mode factor analysis
Ledyard R Tucker. Some mathematical notes on three-mode factor analysis. Psychometrika, 31 0 (3): 0 279--311, 1966
1966
-
[26]
Embedding entities and relations for learning and inference in knowledge bases
Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. arXiv e-prints, pages arXiv--1412, 2014
2014
-
[27]
Neural, symbolic and neural-symbolic reasoning on knowledge graphs
Jing Zhang, Bo Chen, Lingxi Zhang, Xirui Ke, and Haipeng Ding. Neural, symbolic and neural-symbolic reasoning on knowledge graphs. AI Open, 2: 0 14--35, 2021
2021
-
[28]
Quaternion knowledge graph embeddings
Shuai Zhang, Yi Tay, Lina Yao, and Qi Liu. Quaternion knowledge graph embeddings. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pages 2735--2745, 2019
2019
-
[29]
Duality-induced regularizer for tensor factorization based knowledge graph completion
Zhanqiu Zhang, Jianyu Cai, and Jie Wang. Duality-induced regularizer for tensor factorization based knowledge graph completion. Advances in Neural Information Processing Systems, 33, 2020
2020
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.