REVIEW 2 major objections 4 minor 35 references
Embedding Symbolic Knowledge into Deep Networks
T0 review · 2 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper claims that projecting propositional formulas, especially d-DNNF circuits, into vector embeddings and using them as a logic regularizer improves deep models on entailment checking and visual relation prediction.
desk verdict A solid neural-symbolic embedding paper whose synthetic experiments are convincing, but the VRP logic loss needs more scrutiny before the real-world claim is taken at face value. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is a logic graph embedder with semantic regularization. Each formula or assignment is represented as a graph whose nodes are propositions and logical operators, augmented with a global node; a multi-layer graph convolutional network computes node embeddings, with distinct weight matrices per node type to respect heterogeneity. The semantic regularizer encodes d-DNNF structural conditions directly into the embedding: AND children are encouraged to be mutually orthogonal, and OR children are encouraged to sum to the unit vector. Training uses a triplet loss so satisfying assignments lie closer to their formula than unsatisfying ones, and the trained embedder is then frozen and used to define a logic loss that penalizes the distance between the embedding of the known formula and the embedding of the network's predicted conjunction. This embedding space is what transfers symbolic knowledge into the downstream network.
What would settle it
A direct test would replace the true formula embedding in the logic loss with a fixed random vector or with an embedding of a contradictory set of rules; if visual relation accuracy remains unchanged rather than degrading, the loss is not conveying the symbolic constraints.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that propositional formulas, and in particular d-DNNF circuits, can be projected into a shared embedding space in which logical satisfaction is reflected by Euclidean distance, and that this space can serve as a reusable regularizer. The embedder is a graph convolutional network over logic graphs, modified so that each node type (proposition, conjunction, disjunction, implication, global) has its own propagation weights, and trained with a triplet loss that pulls satisfying assignments toward their formula and pushes unsatisfying assignments away. Two semantic regularizers enforce structural meaning: children of an AND node are embedded orthogonally, reflecting decomposability, and children of an OR node are embedded so they sum to a unit vector, reflecting determinism. Once the embedder is fixed, the target network's probability-weighted relation predictions are formed into a conjunction graph, embedded, and penalized by squared distance from the input's formula embedding. The paper reports that this logic loss raises top-5 visual relation prediction accuracy from 84.30 percent for the baseline to 92.77 percent with LENSR using d-DNNF.
Load-bearing premise
The load-bearing assumption is that squared distance in the learned embedding space reliably tracks whether a set of predicted relations is consistent with the logical rules, so pushing the network's weighted predictions toward the formula embedding is a valid training signal.
Editorial extensions
If this is right
- Entailment checking can be performed approximately by thresholding distance in the embedding space, giving a fast surrogate for satisfaction tests on propositional formulas.
- Deep models trained for visual relation prediction can absorb spatial and existence rules without a dedicated label for each rule, improving over the same architecture trained only on annotated relations.
- d-DNNF, despite often larger size, is a better input representation for learned embeddings than CNF, so knowledge compilation to tractable forms pays off in downstream learning, not only in query efficiency.
- The same frozen logic embedder can be attached to different target networks, since the embedder and the classifier are trained separately.
Reading between the lines
- The paper's separate training of embedder and target network suggests a testable extension: jointly fine-tuning the embedder on the downstream task could yield further gains, since the logic loss would then adapt the embedding geometry to the task's prediction distribution.
- If the embeddable-demanding ordering generalizes, one would expect other tractable compilation targets, such as sentential decision diagrams or ordered binary decision diagrams, to be easier to embed than CNF, which could be checked by repeating the synthetic entailment experiment on those languages.
- The distance-based logic loss could also be used at inference time as a consistency check, flagging predictions whose embedded conjunction is far from the known rules, without retraining.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes LENSR, a graph-embedding framework that maps propositional formulae in general form, CNF, and d-DNNF, together with assignments, into a shared vector space using a GCN with node-type-specific parameters and semantic regularizers derived from d-DNNF structure (decomposability and determinism). The embedder is trained with a triplet loss over (formula, satisfying assignment, unsatisfying assignment) triples obtained from a SAT solver. The learned embeddings are then used to define a logic loss that regularizes a target network on visual relation prediction. Experiments on a synthetic entailment-checking task show that d-DNNF embeddings improve accuracy, especially on moderate-complexity formulas, and a VRP experiment on VRD reports a top-5 accuracy increase over baselines. The paper also introduces embeddable-demanding, a complexity-theoretic notion, and proves that CNF is at least as embeddable-demanding as d-DNNF unless P=PP.
Significance. If the empirical claims hold, the paper makes a useful contribution by showing a practical way to inject prior symbolic knowledge into deep networks and, more originally, by connecting knowledge compilation (d-DNNF) to embedding quality. The synthetic study has clear strengths: standard errors over 10 runs are reported, labels come from a SAT solver, code is released, and comparisons include semantic loss and TreeLSTM baselines. The embeddable-demanding theorem is a crisp, falsifiable complexity statement. However, the real-world VRP result currently rests on an underspecified logic loss and on a table without statistical support, so the central application-level claim is not yet fully established.
major comments (2)
- [§3.3, Eq. (7)] The VRP improvement claim rests on the assumption that L_logic = ||q(f) - q(∧_i p_i)||^2 is a faithful measure of whether the predicted relation set satisfies the logical constraints. However, q is trained with a triplet loss on pairs of the form (formula, satisfying assignment) and (formula, unsatisfying assignment), where assignments are conjunctions of propositions. In Eq. (7) the second argument is a graph whose leaves are probability-weighted averages of relationship embeddings, which is neither a Boolean assignment nor a well-formed formula of the kind used in embedder training. Moreover, f contains existence constraints, spatial constraints, and many relation clauses, while ∧_i p_i contains only the predicted relations, so the distance may be dominated by graph size or node counts rather than by semantic consistency. The paper provides no ablation or analysis of Eq. (7), and Figure 3b only validates formula-assignment distances, not formula-versus-soft-conjunction distances. I ask the authors to supply either (i) an experiment showing that L_logic separates correct relation sets from incorrect ones on held-out images, or (ii) an ablation replacing q(h(x)) with a comparably informative graph (for example, the unweighted conjunction of top-k predicted relations, or a randomly sampled soft conjunction) to demonstrate that the improvement is due to logical semantics and not to a generic graph statistic. Without this, the VRP result in Table 2 cannot be attributed to the symbolic knowledge being injected.
- [§5.2, Table 2; §A.4] Table 2 reports a single top-5 accuracy value per configuration, with no standard errors, confidence intervals, or significance tests, whereas Table 1 provides standard errors over 10 runs. The claim that LENSR 'significantly outperforms' baselines is therefore not backed by statistical evidence. In addition, the hyperparameters (m, λ_r, λ) are selected by grid search 'across all experiments' (Section A.4) without a described validation split, leaving it unclear whether the reported numbers were selected on the test set. The authors should report means and variances over multiple runs and state the exact split used for hyperparameter selection.
minor comments (4)
- [Eq. (2)] The element-wise sum and the '1' vector in the semantic regularizer are not fully specified; please state the embedding dimension and whether embeddings are normalized before the regularizer is applied.
- [Fig. 3 (a)] The caption says 'Prediction loss (on the training set)', but the text refers to training progress; please make the caption consistent with what is plotted.
- [§6, Definition 1 and Appendix A.2] Definition 1(iii) requires only a bijection between models, while Proof 1 asserts a polynomial-time computable mapping; the definition should either require the bijection to be polynomial-time computable or the proof should not claim it.
- [§5.1] Equation (7) uses the notation p_i for both a predicted relationship and a proposition; please introduce separate symbols for the relation prediction and the corresponding logical proposition to avoid ambiguity.
Circularity Check
No significant circularity: LENSR's embeddings are trained from SAT assignments and evaluated against independent baselines.
full rationale
The paper's central derivation is self-contained rather than circular. The logic embedder q is trained with a triplet loss (Eq. 3) using satisfying and unsatisfying assignments obtained from an external SAT solver (python-sat), and the target network h is trained separately with a logic loss (Eq. 7) that measures embedding distance between a formula f and the embedded predicted conjunction. No fitted parameter from the target task is used to define the embedder's training signal, and the VRP gain is measured against a no-logic baseline and other embedders using the same input information. The d-DNNF advantage is an empirical finding on held-out test sets, not an identity forced by the semantic regularizer: the regularizer (Eq. 2) encodes standard d-DNNF structural properties (decomposability and determinism), but the paper does not define d-DNNF accuracy in terms of those constraints. The embeddable-demanding notion (Def. 1, Thm. 1) is explicitly introduced after the empirical observation and the paper states that establishing a complete formal connection is beyond its scope; it is therefore not used as a load-bearing derivation of the empirical results. The Eq. 7 construction q(∧_i p_i) may raise questions about semantic faithfulness, but that is a correctness/robustness concern, not circularity: nothing in the equations forces the reported improvement to equal a fitted quantity. Self-citations in the reference list (e.g., [6], [31], [32]) are incidental and do not support any central claim. Overall, no circular step can be exhibited from the paper's own equations or load-bearing citations.
Assumptions & free parameters
free parameters (4)
- triplet loss margin m =
1.0
- semantic regularizer weight lambda_r =
0.1
- logic loss weight lambda =
0.1
- network architecture sizes =
3 GCN layers, 50 hidden units, 100-dim embedding; classifier 2x150; VRP MLP 2x512
assumptions (5)
- domain assumption The d-DNNF decomposability and determinism conditions can be approximately encoded in the embedding space by making AND children orthogonal and OR children sum to a unit vector.
- domain assumption Distance in the learned embedding space between a formula and an assignment is a reliable proxy for entailment, and the same distance transfers to a logic loss over predicted relations.
- standard math The Tseytin encoding of a d-DNNF circuit into CNF gives a polynomial-time computable formula with a bijection between models of the original and encoded formula.
- standard math Standard complexity-theoretic collapses, including PH subset of PP, are accepted background for the P equals PP consequence.
- domain assumption The c2d compiler produces correct d-DNNF representations of the generated CNF formulas in practical time.
invented entities (1)
-
Global graph node connected to every node in the logic graph
Cite this review
Pith. "Pith review of Embedding Symbolic Knowledge into Deep Networks." pith.science (2026). https://pith.science/paper/QIDUKAND
@misc{pith2026190901161,
author = {Pith},
title = {Pith review of: Embedding Symbolic Knowledge into Deep Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/QIDUKAND}},
note = {Machine review of arXiv:1909.01161}
}
read the original abstract
In this work, we aim to leverage prior symbolic knowledge to improve the performance of deep models. We propose a graph embedding network that projects propositional formulae (and assignments) onto a manifold via an augmented Graph Convolutional Network (GCN). To generate semantically-faithful embeddings, we develop techniques to recognize node heterogeneity, and semantic regularization that incorporate structural constraints into the embedding. Experiments show that our approach improves the performance of models trained to perform entailment checking and visual relation prediction. Interestingly, we observe a connection between the tractability of the propositional theory representation and the ease of embedding. Future exploration of this connection may elucidate the relationship between knowledge compilation and vector representation learning.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Y . LeCun, Y . Bengio, and G. E. Hinton, “Deep learning,”Nature, vol. 521, no. 7553, pp. 436– 444, 2015
work page 2015
-
[2]
A survey on deep learning in medical image analysis,
G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. van der Laak, B. van Ginneken, and C. I. Sánchez, “A survey on deep learning in medical image analysis,” Medical Image Analysis, vol. 42, pp. 60 – 88, 2017
work page 2017
-
[3]
Mastering the game of go without human knowledge,
D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. R. Baker, M. Lai, A. Bolton, Y . Chen, T. P. Lillicrap, F. Hui, L. Sifre, G. van den Driessche, T. Graepel, and D. Hassabis, “Mastering the game of go without human knowledge,”Nature, vol. 550, pp. 354–359, 2017
work page 2017
-
[4]
Neural-symbolic learning and reasoning: A survey and interpretation,
T. R. Besold, A. d. Garcez, S. Bader, H. Bowman, P. Domingos, P. Hitzler, K.-U. Kühnberger, L. C. Lamb, D. Lowd, P. M. V . Lima,et al., “Neural-symbolic learning and reasoning: A survey and interpretation,”arXiv preprint arXiv:1711.03902, 2017
arXiv 2017
-
[5]
Harnessing deep neural networks with logic rules,
Z. Hu, X. Ma, Z. Liu, E. H. Hovy, and E. P. Xing, “Harnessing deep neural networks with logic rules,”CoRR, vol. abs/1603.06318, 2016
arXiv 2016
-
[6]
Hyperprior induced unsupervised disentanglement of latent representa- tions,
A. F. Ansari and H. Soh, “Hyperprior induced unsupervised disentanglement of latent representa- tions,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3175–3182, 2019
work page 2019
-
[7]
A semantic loss function for deep learning with symbolic knowledge,
J. Xu, Z. Zhang, T. Friedman, Y . Liang, and G. Van den Broeck, “A semantic loss function for deep learning with symbolic knowledge,” inICML, 2018
work page 2018
-
[8]
Character-aware neural language models,
Y . Kim, Y . Jernite, D. Sontag, and A. M. Rush, “Character-aware neural language models,” in AAAI, 2016
work page 2016
Show all 35 references
-
[9]
Learning continuous semantic representations of symbolic expressions,
M. Allamanis, P. Chanthirasegaran, P. Kohli, and C. A. Sutton, “Learning continuous semantic representations of symbolic expressions,” inICML, 2017
2017
-
[10]
Can neural networks understand logical entailment?,
R. Evans, D. Saxton, D. Amos, P. Kohli, and E. Grefenstette, “Can neural networks understand logical entailment?,” inICLR, 2018
2018
-
[11]
Long short-term memory over recursive structures,
X.-D. Zhu, P. Sobhani, and H. Guo, “Long short-term memory over recursive structures,” in ICML, 2015
2015
-
[12]
Improved semantic representations from tree- structured long short-term memory networks,
K. S. Tai, R. Socher, and C. D. Manning, “Improved semantic representations from tree- structured long short-term memory networks,” inACL, 2015
2015
-
[13]
On the tractability of counting theory models and its application to belief revision and truth maintenance,
A. Darwiche, “On the tractability of counting theory models and its application to belief revision and truth maintenance,”Journal of Applied Non-classical Logics - JANCL, vol. 11, 01 2001
2001
-
[14]
A knowledge compilation map,
A. Darwiche and P. Marquis, “A knowledge compilation map,” J. Artif. Int. Res. , vol. 17, pp. 229–264, Sept. 2002
2002
-
[15]
Semi-supervised classification with graph convolutional networks,
T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in ICLR, 2017
2017
-
[16]
Decomposable negation normal form,
A. Darwiche, “Decomposable negation normal form,”J. ACM, vol. 48, pp. 608–647, July 2001
2001
-
[17]
New advances in compiling CNF into decomposable negation normal form,
A. Darwiche, “New advances in compiling CNF into decomposable negation normal form,” in ECAI, 2004
2004
-
[18]
Learning logistic circuits,
Y . Liang and G. Van den Broeck, “Learning logistic circuits,” inAAAI, 2019
2019
-
[19]
Logic tensor networks for semantic image interpretation,
I. Donadello, L. Serafini, and A. S. d’Avila Garcez, “Logic tensor networks for semantic image interpretation,” inIJCAI, 2017
2017
-
[20]
Logic tensor networks: Deep learning and logical reasoning from data and knowledge,
L. Serafini and A. S. d’Avila Garcez, “Logic tensor networks: Deep learning and logical reasoning from data and knowledge,” inNeSy@HLAI, 2016
2016
-
[21]
Label-free supervision of neural networks with physics and domain knowledge,
R. Stewart and S. Ermon, “Label-free supervision of neural networks with physics and domain knowledge,” inAAAI, 2017
2017
-
[22]
Injecting logical background knowledge into embed- dings for relation extraction,
T. Rocktäschel, S. Singh, and S. Riedel, “Injecting logical background knowledge into embed- dings for relation extraction,” inHLT-NAACL, 2015
2015
-
[23]
Lifted rule injection for relation embeddings,
T. Demeester, T. Rocktäschel, and S. Riedel, “Lifted rule injection for relation embeddings,” in EMNLP, 2016. 10
2016
-
[24]
Compositional distributional semantics with long short term memory,
P. Le and W. Zuidema, “Compositional distributional semantics with long short term memory,” in *SEM@NAACL-HLT, 2015
2015
-
[25]
Wavelets on graphs via spectral graph theory,
D. K. Hammond, P. Vandergheynst, and R. Gribonval, “Wavelets on graphs via spectral graph theory,”Applied and Computational Harmonic Analysis, vol. 30, no. 2, pp. 129 – 150, 2011
2011
-
[26]
Convolutional neural networks on graphs with fast localized spectral filtering,
M. Defferrard, X. Bresson, and P. Vandergheynst, “Convolutional neural networks on graphs with fast localized spectral filtering,” in NIPS, 2016
2016
-
[27]
PySAT: A Python toolkit for prototyping with SAT oracles,
A. Ignatiev, A. Morgado, and J. Marques-Silva, “PySAT: A Python toolkit for prototyping with SAT oracles,” inSAT, pp. 428–437, 2018
2018
-
[28]
Visual relationship detection with language priors,
C. Lu, R. Krishna, M. Bernstein, and L. Fei-Fei, “Visual relationship detection with language priors,” inECCV, pp. 852–869, Springer, 2016
2016
-
[29]
Glove: Global vectors for word representation,
J. Pennington, R. Socher, and C. Manning, “Glove: Global vectors for word representation,” in EMNLP, 2014
2014
-
[30]
Adam: A method for stochastic optimization,
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” CoRR, vol. abs/1412.6980, 2015
2015 arXiv
-
[31]
Learning assistance by demonstration: Smart mobility with shared control and paired haptic controllers,
H. Soh and Y . Demiris, “Learning assistance by demonstration: Smart mobility with shared control and paired haptic controllers,”J. Hum.-Robot Interact., vol. 4, pp. 76–100, Dec. 2015
2015
-
[32]
Towards effective tactile identification of textures using a hybrid touch approach,
T. Taunyazov, H. F. Koh, Y . Wu, C. Cai, and H. Soh, “Towards effective tactile identification of textures using a hybrid touch approach,” in2019 International Conference on Robotics and Automation (ICRA), pp. 4269–4275, IEEE, 2019
2019
-
[33]
Arora and B
S. Arora and B. Barak, Computational complexity: a modern approach. Cambridge University Press, 2009
2009
-
[34]
G. S. Tseytin, On the Complexity of Derivation in Propositional Calculus, pp. 466–483. Berlin, Heidelberg: Springer Berlin Heidelberg, 1983
1983
-
[35]
Pp is as hard as the polynomial-time hierarchy,
S. Toda, “Pp is as hard as the polynomial-time hierarchy,”SIAM Journal on Computing, vol. 20, no. 5, pp. 865–877, 1991. 11 A Supplementary Material A.1 Embedder Training Algorithm The algorithm for training the embedder is summarized in Algorithm 1. Algorithm 1 trainEmbedder I...
1991
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.