REVIEW 4 major objections 5 minor 46 references
Probabilistic Circuits for Knowledge Graph Completion with Reduced Rule Sets
T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read A learned distribution over rule contexts lets knowledge graph completion keep 91% of peak accuracy with 70-96% fewer rules.
desk verdict A genuinely new PC-over-contexts method for rule-set reduction in KG completion, with sound formal results but an under-specified and unvalidated context-discovery step; deserves review but needs major experimental and algorithmic additions. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a 'rule context': a mutually exclusive mode in which some learned first-order rules activate and others do not, encoded with activation atoms $\mu_r$ and context atoms $\nu_c$. A probabilistic circuit learns $P_\theta$ over these contexts; marginals over rule sets are obtained by summing context probabilities, with Propositions 1-4 giving lower bounds, exact probabilities, and upper bounds without any independence assumption.
What would settle it
Take one benchmark dataset, split the training triples into two halves, build contexts and learn the circuit on each half, and compare the top-K rule subsets selected by marginal probability. If the two halves select substantially different rule subsets, or if subsets selected from one half lose most of the claimed 91% accuracy when evaluated on the other half's queries, then the discovered contexts are not carrying the generalization.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that a probabilistic circuit learned over rule contexts captures rule interactions that confidence scores miss, so the marginal probability of a rule set is a better guide for selecting rules than individual rule confidence. Formally, for a learned distribution $P_\theta$ over mutually exclusive contexts, the marginal of a query $q$ equals the sum of $P_\theta(c)$ over contexts that entail $q$; the paper proves that this equals $1 - P_\theta(\bigwedge_{c: \Pi_c \models q} \neg\nu_c)$, giving an exact query probability, and derives lower and upper bounds when only subsets of rules are available. Experiments on eight benchmark knowledge graphs
Load-bearing premise
The load-bearing premise is that the rule contexts produced from training triples by the abductive engine are the meaningful groups of rules and that the circuit's learned distribution over them continues to hold on unseen test queries; if the contexts are arbitrary or training-biased, the reduced rule sets will not transfer.
Editorial extensions
If this is right
- Rule-based knowledge graph completion can be made explainable at scale: each prediction is supported by a handful of rules rather than thousands.
- Selection by learned marginal probability over contexts outperforms confidence-based rule selection at equal rule counts, often by large multiples.
- Because no independence assumption is used and the semantics align with Nilsson's probabilistic logic, the framework offers a general way to shrink rule sets in any rule-based reasoner.
- The same machinery can be applied beyond knowledge graphs, for example to inductive logic programming or proof search, where rule explosion is also a bottleneck.
Reading between the lines
- The 91% preservation is measured against the confidence-based baseline's full rule set; a sterner test would compare against embedding-based completers or against the same circuit using all rules at an equal inference budget.
- Because contexts are discovered from training triples by an abductive engine, the method's transfer depends on whether those contexts are stable across training splits; one could test this by learning contexts on subsamples and checking how much the selected rule subsets change.
- The framework could be turned into a rule-learning guide, using the learned distribution over contexts to propose which candidate rules to replace rather than only which ones to keep.
- The circuit's structure over contexts may itself be inspected to explain why a cluster of rules is predictive, for example by identifying shared body atoms that drive high context probability.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a probabilistic-circuit-guided framework for knowledge graph completion. It introduces 'rule contexts' (subsets of learned rules that work together), learns a PC over those contexts, and uses marginal probabilities to rank and select small rule sets. Three inference methods are presented: PC1 (lower-bound approximation with singleton rule sets), PC2 (exact query probability), and PC3 (lower-bound approximation with greedy-walk rule sets). The authors provide formal propositions about these marginals and a semantics connection to Nilsson probabilistic logic. Experiments on eight KG benchmarks claim a 70–96% reduction in rules, a 31-fold performance improvement over the baseline at equal small rule counts, and preservation of about 91% of peak baseline performance.
Significance. If the empirical claims hold, the paper offers a promising way to make rule-based KG completion more explainable by using a PC to select far fewer rules, without assuming rule independence, and with tractable exact/lower-bound inference. The inclusion of formal semantics and the release of code/datasets in the supplementary material are strengths. However, the central context-discovery step is not described or validated, several formal statements are incorrect as written, and the experimental protocol is too thin to support the headline quantitative claims. The contribution is therefore interesting but not yet established.
major comments (4)
- [Technical Preliminaries, Proposition 1] As written, Proposition 1 is false. Take R={p←, q←} and Πc={p←} with Pθ(c)>0, and suppose no context entails q←. Then Πc ⊆ R, but Πc does not entail R, so the left-hand side Pθ(R) can be 0 while the right-hand side is at least Pθ(c)>0. The proof's statement 'if Πc is contained in R then R must be a logical consequence' is not a valid inference. The intended statement may be with R ⊆ Πc, but as written this is a load-bearing formal error.
- [Technical Preliminaries, Proposition 4] Proposition 4 is also false as stated. Let Rj=∅. For any non-tautological query q, Rj does not entail q, and Pθ(Rj)=1 because every program entails the empty set. Then the claimed upper bound becomes Pθ(q) ≤ 0, which fails whenever Pθ(q)>0. The correct upper-bound condition should ensure disjointness, e.g., Rj |= ¬q (or a similar condition). This matters because the paper advertises Propositions 1–4 as a suite of formal guarantees, and a reader cannot verify which results are actually used by the proposed inference methods.
- [Rule Context and Experimental Setup] The rule-context discovery step is the unvalidated crux of the framework. The paper only says 'We use training triples to establish contexts' and 'We have used PyClause's abductive reasoning engine to determine rule-context associations,' with no definition of a context, no algorithm or configuration, no number of resulting contexts, and no consistency check. Since Eq. (1) and all PC marginals are conditioned on this association matrix, the claimed 70–96% rule reduction and performance preservation could be artifacts of this step. Please provide a precise description and a sensitivity/robustness analysis of the context-generation procedure.
- [Experimental Setup and Results and Discussion] The experimental protocol is too thin to support the quantitative claims. Seven of the eight datasets are run once (only Family has 5 runs), and hyperparameters such as the confidence threshold and number of EM iterations are chosen per dataset. The headline comparisons ('31-fold improvement', '91% preservation', 'minimal equivalent number of rules') are not formally defined, and the WN18 baseline has zero performance at 500 rules, which makes fold-ratio comparisons unbounded. Please report per-dataset raw values, define the aggregation precisely, and provide multiple seeds or error bars.
minor comments (5)
- [Formal Probabilistic Semantics, Theorem 1] The theorem statement uses R on the right-hand side ('Σ_{c s.t. Πc|=R} Pθ(c)') but the theorem is about a query q. This should be q, not R.
- [Proposition 2 proof] The proof writes '¬c′″' where it should write '¬νc′″'; the helper atom symbol is missing.
- [Appendix A, Algorithm 1] The indentation of Algorithm 1 appears incorrect: lines 8–9 (updating S and R_remaining) are inside the inner while loop, and 'return S' appears before the outer while loop terminates. Please fix the loop structure.
- [Appendix B, Figure 11] The caption says 'UML dataset' but the dataset name in Table 1 is UMLS. Please correct the typo.
- [Hyperparameter Selection] For WN18, WN18RR, and Family, the 'confidence threshold' is reported as 0%. Since AnyBURL's confidence is a ratio, a 0% threshold typically means no filtering; please clarify whether this means all mined rules were used.
Circularity Check
No significant circularity: formal results are definitional identities and empirical claims are held-out test measurements.
full rationale
The paper's derivation chain is self-contained. Propositions 1-4 and Theorem 1 are proven from the definition of Pθ(R)=Σ_{c:Πc|=R}Pθ(c) and the mutual-exclusivity of contexts; they do not presuppose the empirical rule-reduction claim. The central empirical claims (70-96% rule reduction, ~91% performance preservation) compare PC-selected rule subsets against a confidence-based baseline on held-out test triples. The PC marginals are learned from training triples only (via EM on contexts discovered by PyClause), while Hits@k/MRR are computed on the test set with AnyBURL's evaluation engine—so the reported performance is a separate measurement, not a restatement of the fitted marginals. Self-citations (Choi et al. 2020 for PC definitions, Shakarian et al. 2012 for Nilsson-style semantics, Aditya et al. 2023 for first-order syntax) are standard references used for background, not to justify the novelty. The underspecified PyClause context-generation step is a reproducibility/validity limitation, not a circular reduction; no equation in the paper forces the test results from the training-fit quantities.
Assumptions & free parameters
free parameters (5)
- Confidence threshold for input rules =
Per dataset: 50% (CODEX-S/Kinship/UMLS), 60% (FB15K-237), 70% (Nations), 0% (WN18, WN18RR, Family)
- Number of EM iterations for PC learning =
100 (Family/WN18/WN18RR), 50 (Kinship), 10 (others)
- AnyBURL learning time =
10 seconds
- AnyBURL minimum support threshold =
>= 10
- Greedy walk threshold delta for PC3 =
Not specified in the paper
assumptions (4)
- domain assumption Contexts are mutually exclusive and exhaustive: every world with non-zero probability satisfies exactly one context atom νc.
- domain assumption For a context c, if Πc entails q, then q is true in that context with probability 1.
- domain assumption The PyClause abductive reasoning engine correctly identifies the rule-context associations from training data.
- standard math Standard first-order logic semantics with worlds as subsets of ground atoms.
invented entities (2)
-
Rule contexts
-
Activation atoms μr and context atoms νc
Cite this review
Pith. "Pith review of Probabilistic Circuits for Knowledge Graph Completion with Reduced Rule Sets." pith.science (2026). https://pith.science/paper/L2IXOYGO
@misc{pith2026250806706,
author = {Pith},
title = {Pith review of: Probabilistic Circuits for Knowledge Graph Completion with Reduced Rule Sets},
year = {2026},
howpublished = {\url{https://pith.science/paper/L2IXOYGO}},
note = {Machine review of arXiv:2508.06706}
}
abstract
Rule-based methods for knowledge graph completion provide explainable results but often require a significantly large number of rules to achieve competitive performance. This can hinder explainability due to overwhelmingly large rule sets. We discover rule contexts (meaningful subsets of rules that work together) from training data and use learned probability distribution (i.e. probabilistic circuits) over these rule contexts to more rapidly achieve performance of the full rule set. Our approach achieves a 70-96% reduction in number of rules used while outperforming baseline by up to 31$\times$ when using equivalent minimal number of rules and preserves 91% of peak baseline performance even when comparing our minimal rule sets against baseline's full rule sets. We show that our framework is grounded in well-known semantics of probabilistic logic, does not require independence assumptions, and that our tractable inference procedure provides both approximate lower bounds and exact probability of a given query. The efficacy of our method is validated by empirical studies on 8 standard benchmark datasets where we show competitive performance by using only a fraction of the rules required by AnyBURL's standard inference method, the current state-of-the-art for rule-based knowledge graph completion. This work may have further implications for general probabilistic reasoning over learned sets of rules.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
Aditya, D.; Mukherji, K.; Balasubramanian, S.; Chaudhary, A.; and Shakarian, P. 2023. Pyreason: Software for open world temporal logic. arXiv preprint arXiv:2302.13482
arXiv 2023
-
[2]
Ahmed, K.; Chang, K.-W.; and Van den Broeck, G. 2024. Controllable generation via locally constrained resampling. arXiv preprint arXiv:2410.13111
arXiv 2024
-
[3]
Ahmed, K.; Teso, S.; Chang, K.-W.; Van den Broeck, G.; and Vergari, A. 2022 a . Semantic Probabilistic Layers for Neuro-Symbolic Learning. In Koyejo, S.; Mohamed, S.; Agarwal, A.; Belgrave, D.; Cho, K.; and Oh, A., eds., Advances in Neural Information Processing Systems, volume 35, 29944--29959. Curran Associates, Inc
work page 2022
-
[4]
Ahmed, K.; Wang, E.; Chang, K.-W.; and Van den Broeck, G. 2022 b . Neuro-symbolic entropy regularization. In Uncertainty in Artificial Intelligence, 43--53. PMLR
work page 2022
-
[5]
d.; Serafini, L.; and Spranger, M
Badreddine, S.; Garcez, A. d.; Serafini, L.; and Spranger, M. 2022. Logic tensor networks. Artificial Intelligence, 303: 103649
work page 2022
-
[6]
Betz, P.; Galarraga, L.; Ott, S.; Meilicke, C.; Suchanek, F. M.; and Stuckenschmidt, H. 2024. PyClause-Simple and Efficient Rule Handling for Knowledge Graphs. In IJCAI, demo track. Ijcai.org
work page 2024
-
[7]
Bordes, A.; Usunier, N.; Garcia-Duran, A.; Weston, J.; and Yakhnenko, O. 2013 a . Translating Embeddings for Modeling Multi-relational Data. In Burges, C.; Bottou, L.; Welling, M.; Ghahramani, Z.; and Weinberger, K., eds., Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc
work page 2013
-
[8]
Bordes, A.; Usunier, N.; Garcia-Duran, A.; Weston, J.; and Yakhnenko, O. 2013 b . Translating embeddings for modeling multi-relational data. Advances in neural information processing systems, 26
work page 2013
Show all 46 references
-
[9]
Cheng, K.; Ahmed, N.; and Sun, Y. 2023 a . Neural Compositional Rule Learning for Knowledge Graph Reasoning. In The Eleventh International Conference on Learning Representations
2023
-
[10]
K.; and Sun, Y
Cheng, K.; Ahmed, N. K.; and Sun, Y. 2023 b . Neural compositional rule learning for knowledge graph reasoning. arXiv preprint arXiv:2303.03581
2023 arXiv
-
[11]
Choi, Y.; Vergari, A.; and Van den Broeck, G. 2020. Probabilistic circuits: A unifying framework for tractable probabilistic models. UCLA. URL: http://starai. cs. ucla. edu/papers/ProbCirc20. pdf, 6
2020
-
[12]
C \^o rte-Real, J.; Dries, A.; Dutra, I.; and Rocha, R. 2018. Improving candidate quality of probabilistic logic models. In Technical Communications of the 34th International Conference on Logic Programming (ICLP 2018), 6--1. Schloss Dagstuhl--Leibniz-Zentrum f \"u r Informatik
2018
-
[13]
Dang, M.; Khosravi, P.; Liang, Y.; Vergari, A.; and Van den Broeck, G. 2021. Juice: A julia package for logic and probabilistic circuits. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 16020--16023
2021
-
[14]
De Raedt, L.; Dries, A.; Thon, I.; Van den Broeck, G.; and Verbeke, M. 2015. Inducing probabilistic relational rules from probabilistic examples. In Proceedings of 24th international joint conference on artificial intelligence (IJCAI), volume 2015, 1835--1842. IJCAI-INT JOINT ...
2015
-
[15]
Dettmers, T.; Minervini, P.; Stenetorp, P.; and Riedel, S. 2018 a . Convolutional 2d knowledge graph embeddings. In Proceedings of the AAAI conference on artificial intelligence, volume 32
2018
-
[16]
Dettmers, T.; Minervini, P.; Stenetorp, P.; and Riedel, S. 2018 b . Convolutional 2D knowledge graph embeddings. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eight...
2018
-
[17]
Feldstein, J.; Phillips, D.; and Tsamoura, E. 2024. Efficiently learning probabilistic logical models by cheaply ranking mined rules. arXiv preprint arXiv:2409.16238
2024
-
[18]
Gal \'a rraga, L.; Teflioudi, C.; Hose, K.; and Suchanek, F. M. 2015. Fast rule mining in ontological knowledge bases with AMIE + . The VLDB Journal, 24(6): 707--730
2015
-
[19]
A.; Teflioudi, C.; Hose, K.; and Suchanek, F
Gal \'a rraga, L. A.; Teflioudi, C.; Hose, K.; and Suchanek, F. 2013. AMIE: association rule mining under incomplete evidence in ontological knowledge bases. In Proceedings of the 22nd international conference on World Wide Web, 413--422
2013
-
[20]
Giunchiglia, E.; and Lukasiewicz, T. 2020. Coherent hierarchical multi-label classification networks. Advances in neural information processing systems, 33: 9662--9673
2020
-
[21]
Hayes, P. J. 1977. In defence of logic. In Proc. IJCAI-77, 559--565
1977
-
[22]
Jain, A.; Friedman, T.; Kuzelka, O.; Van den Broeck, G.; and De Raedt, L. 2019. Scalable rule learning in probabilistic knowledge bases. Automated Knowledge Base Construction
2019
-
[23]
Jiang, D.; Fonseca, M.; and Cohen, S. B. 2024. Leanreasoner: Boosting complex logical reasoning with lean. arXiv preprint arXiv:2403.13312
2024 arXiv
-
[24]
V.; Nau, D.; Sliva, A.; Simari, G
Khuller, S.; Martinez, M. V.; Nau, D.; Sliva, A.; Simari, G. I.; and Subrahmanian, V. S. 2007. Computing most probable worlds of action probabilistic logic programs: scalable estimation for 10\^ 30,000 worlds. Annals of Mathematics and Artificial Intelligence, 51(2-4): 295--331
2007
-
[25]
Kok, S.; and Domingos, P. 2007. Statistical predicate invention. In Proceedings of the 24th International Conference on Machine Learning, ICML '07, 433–440. New York, NY, USA: Association for Computing Machinery. ISBN 9781595937933
2007
-
[26]
Liu, A.; and Van den Broeck, G. 2021. Tractable regularization of probabilistic circuits. Advances in Neural Information Processing Systems, 34: 3558--3570
2021
-
[27]
Liu, C.; Wei, Z.; and Zhou, L. 2025. Contrastive Predictive Embedding for learning and inference in knowledge graph. Knowledge-Based Systems, 307: 112730
2025
-
[28]
Loconte, L.; Di Mauro, N.; Peharz, R.; and Vergari, A. 2023. How to turn your knowledge graph embeddings into generative models. Advances in Neural Information Processing Systems, 36: 77713--77744
2023
-
[29]
Manhaeve, R.; Dumancic, S.; Kimmig, A.; Demeester, T.; and De Raedt, L. 2018. DeepProbLog: Neural Probabilistic Logic Programming. In Bengio, S.; Wallach, H.; Larochelle, H.; Grauman, K.; Cesa-Bianchi, N.; and Garnett, R., eds., Advances in Neural Information Processing System...
2018
-
[30]
W.; Fink, M.; and Stuckenschmidt, H
Meilicke, C.; Chekol, M. W.; Fink, M.; and Stuckenschmidt, H. 2020. Reinforced anytime bottom up rule learning for knowledge graph completion. arXiv preprint arXiv:2004.04412
2020 arXiv
-
[31]
Q.; Nguyen, T
Nguyen, D. Q.; Nguyen, T. D.; Nguyen, D. Q.; and Phung, D. 2017. A novel embedding model for knowledge base completion based on convolutional neural network. arXiv preprint arXiv:1712.02121
2017 arXiv
-
[32]
Nilsson, N. J. 1986. Probabilistic Logic. Artificial Intelligence, 28(1): 71--87
1986
-
[33]
G.; Wang, K.; and Wang, Z
Omran, P. G.; Wang, K.; and Wang, Z. 2018. Scalable rule learning via learning representation. In IJCAI-18. International Joint Conferences on Artificial Intelligence
2018
-
[34]
Ott, S.; Meilicke, C.; and Samwald, M. 2021. SAFRAN : An interpretable, rule-based link prediction method outperforming embedding models. In 3rd Conference on Automated Knowledge Base Construction
2021
-
[35]
Reiter, R. 1980. A logic for default reasoning. Artificial intelligence, 13(1-2): 81--132
1980
-
[36]
Safavi, T.; and Koutra, D. 2020. C o DE x: A C omprehensive K nowledge G raph C ompletion B enchmark. In Webber, B.; Cohn, T.; He, Y.; and Liu, Y., eds., Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 8328--8350. Online: Associa...
2020
-
[37]
I.; and Subrahmanian, V
Shakarian, P.; Simari, G. I.; and Subrahmanian, V. S. 2012. Annotated Probabilistic Temporal Logic: Approximate Fixpoint Implementation. ACM Transactions on Computational Logic, 13(2): 13:1--13:33
2012
-
[38]
S.; and Kersting, K
Skryagin, A.; Stammer, W.; Ochs, D.; Dhami, D. S.; and Kersting, K. 2021. SLASH: embracing probabilistic circuits into neural answer set programming. arXiv preprint arXiv:2110.03395
2021 arXiv
-
[39]
Sun, Z.; Deng, Z.-H.; Nie, J.-Y.; and Tang, J. 2019. Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197
2019 arXiv
-
[40]
Toutanova, K.; Chen, D.; Pantel, P.; Poon, H.; Choudhury, P.; and Gamon, M. 2015. Representing text for joint embedding of text and knowledge bases. In Proceedings of the 2015 conference on empirical methods in natural language processing, 1499--1509
2015
-
[41]
Trouillon, T.; Welbl, J.; Riedel, S.; Gaussier, \'E .; and Bouchard, G. 2016. Complex embeddings for simple link prediction. In International conference on machine learning, 2071--2080. PMLR
2016
-
[42]
u nchmeyer, J.; Leser, U.; and Rockt \
Weber, L.; Minervini, P.; M \"u nchmeyer, J.; Leser, U.; and Rockt \"a schel, T. 2019. Nlprolog: Reasoning with weak unification for question answering in natural language. arXiv preprint arXiv:1906.06187
2019 arXiv
-
[43]
Xiong, B.; Nayyeri, M.; Luo, L.; Wang, Z.; Pan, S.; and Staab, S. 2024. NestE: modeling nested relational structures for knowledge graph reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 9205--9213
2024
-
[44]
Xu, J.; Zhang, Z.; Friedman, T.; Liang, Y.; and Van den Broeck, G. 2018. A semantic loss function for deep learning with symbolic knowledge. In International conference on machine learning, 5502--5511. PMLR
2018
-
[45]
Yang, F.; Yang, Z.; and Cohen, W. W. 2017. Differentiable learning of logical rules for knowledge base reasoning. Advances in neural information processing systems, 30
2017
-
[46]
Zhang, R.; Mao, Y.; and Zhao, W. 2020. Knowledge graphs completion via probabilistic reasoning. Information Sciences, 521: 144--159
2020
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.