REVIEW 1 major objections 3 minor 19 references
Conditions for Unnecessary Logical Constraints in Kernel Machines
T0 review · 1 major / 3 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A logical constraint is unnecessary for a kernel-machine problem exactly when its Lagrange multipliers can be zeroed, provided the optimum is unique.
desk verdict The algebraic redundancy criterion is a useful idea, but Corollary 1's sufficiency claim is false as stated because unregularized biases break uniqueness. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the linear system called Problem 2, $M\lambda = \alpha^*$, together with the nullspace of $M$. The matrix $M$ encodes, for every affine piece of every Lukasiewicz formula, how that piece weights the groundings of each predicate; $\alpha^*$ is the coefficient vector of an optimal solution in the kernel expansion. Any $\lambda$ solving this system yields the same optimal solution, and $\mathrm{Ker}(M)$ parametrizes alternative multiplier assignments. A constraint is removable when the affine subspace $\lambda^* + \mathrm{Ker}(M)$ intersects the coordinate subspace in which that constraint's multiplier block is zero. This algebraic criterion is the mechanism that turns support-vector intuition into a testable condition on Lagrange multipliers.
What would settle it
Solve a convex quadratic program with a positive-semidefinite Gram matrix and two logical constraints, and exhibit a KKT solution in which one constraint's multipliers are all zero while removing that constraint adds new optimal solutions; such an example would show why Corollary 1 needs uniqueness, while failure to find one would suggest the sufficiency result can be strengthened.
Extended reading notes
Core claim
The paper's central claim is that redundancy of a hard constraint is captured by the existence of an alternative set of Lagrange multipliers that reproduce the same optimal solution while setting the target constraint's multiplier block to zero. Concretely, with $\alpha^*$ denoting the coefficients of an optimal solution in the kernel expansion and $M$ the matrix whose rows are the affine pieces of the logical constraints, the paper studies solutions of $M\lambda = \alpha^*$. If a solution has $\lambda_{\bar h,i}=0$ for every affine component $i$ of constraint $\phi_{\bar h}$, then that constraint's contribution can be transferred to the other constraints. The paper proves this condition is necessary when $\phi_{\bar h}$ is unnecessary (Proposition 2), sufficient for the original optima to survive removal (Proposition 3), and equivalent to redundancy when the optimal solution is unique (Corollary 1). It also proves that logical consequence $\{\phi_h : h\neq \bar h\}\models \phi_{\bar h}$ implies redundancy (Proposition 1).
Load-bearing premise
All the redundancy criteria assume the learning problem satisfies the KKT conditions, meaning a constraint-qualification condition and full convexity of the constraints; if that assumption fails, the zero-multiplier test no longer certifies redundancy.
Editorial extensions
If this is right
- Redundant logical rules can be dropped before solving the quadratic program, shrinking the constraint matrix without changing the answer, whenever the criterion finds a zero-block KKT solution and uniqueness holds.
- In the convex Lukasiewicz fragment, any constraint that is a logical consequence of the remaining knowledge base is certifiably removable, giving a direct link between deductive logic and optimization.
- The criterion applies uniformly to pointwise, consistency, and logical constraints, because the first two can be written as logical formulas; this is demonstrated by removing a supervised label constraint in Example 4.
- With a positive-definite Gram matrix, detecting an unnecessary constraint becomes a linear feasibility problem in the nullspace of $M$.
Reading between the lines
- Editorial inference: because Proposition 3 gives only inclusion of the optimal sets when the optimum is not unique, a user who removes a constraint on the strength of a zero-multiplier KKT solution may enlarge the set of optima; checking uniqueness of the solution is therefore part of applying the criterion safely.
- Editorial inference: the paper states that the convex fragment uses only the operators $\wedge$ and $\oplus$, yet all worked examples use implication; this suggests either the fragment is wider than stated or the examples rely on rewriting implications in terms of the convex operators, and an explicit rewriting would pin down the exact boundary.
- Editorial inference: the nullspace computation could be performed symbolically before training once the optimal coefficient vector is known, turning the criterion into a preprocessing pass; the paper does not discuss the algorithmic complexity or large-scale implementation of such a step.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies learning-from-constraints problems in kernel machines where logical constraints are expressed in Łukasiewicz logic. It extends the notion of support vector to support constraint and defines a constraint to be unnecessary when removing it from the optimization problem leaves the set of optimal solutions unchanged. The main results are: (i) Proposition 1, that a logical consequence relation among formulas implies redundancy; (ii) Propositions 2 and 3, which relate redundancy to the existence of a Lagrange-multiplier solution of a linear system (Problem 2) in which the multipliers of the candidate constraint vanish; and (iii) Corollary 1, which claims that under uniqueness of the optimal solution this multiplier condition is also sufficient. The paper illustrates the criteria on transitivity and pointwise-constraint examples solved in MATLAB. The overall goal is to prune unnecessary constraints before or during optimization while preserving the optimal solutions.
Significance. If the results were correct, the paper would give a practically useful algebraic criterion for detecting redundant constraints in kernel machines with logical constraints, and it would clarify the relationship between logical deducibility and constraint redundancy. The formal definition of unnecessary constraint and the clean statement of Proposition 1 are valuable, and the examples are helpful for understanding the intended method. However, the central sufficiency result (Corollary 1) is false as stated because the objective is not strictly convex in the bias terms, and the manuscript leaves important regularity and well-posedness assumptions implicit. The positive contributions are therefore conditional on a substantial revision of the main theorems and their hypotheses.
major comments (1)
- [Sections 3.1 and 3.2] The phrase 'KKT-solution of Problem 2' is used throughout Section 3, but Problem 2 is only a linear system and has no KKT conditions of its own. The proofs require that the constructed multiplier vector additionally satisfies the KKT conditions of the original learning problem, but this extra condition is not stated in Problem 2 or in the surrounding definitions. Adding a formal definition of this object would remove a significant source of ambiguity.
minor comments (3)
- [Table 1] The name of the logic is typeset inconsistently ('Lukasiewicz' without the diacritic in the table caption); this is a presentation issue and should be made uniform.
- [Definition 1 and Section 2.1] Definition 1 writes the loss as Loss(alpha)=sum alpha'_j K_j alpha_j and does not indicate where the bias terms b_j appear in the constraints of HP and HP; since Problem 1 explicitly includes b_j in p_j(x), the relationship between the two formulations should be stated explicitly.
- [Abstract and Section 1] The phrase 'support constraint' is introduced casually and never given a formal definition separate from the definition of unnecessary constraint; aligning the two notions in one place would improve readability.
Circularity Check
No circularity: the multiplier-removal criterion is an algebraic consistency check, not a fitted prediction or self-citation chain.
full rationale
The paper's derivation chain is self-contained with respect to its stated assumptions. The central criterion (Problem 2) is an algebraic consistency condition on an already-computed optimal multiplier vector: given alpha*, it asks whether M*lambda = alpha* admits a solution with lambda_{\bar h,i} = 0 for all i. This is not a fitted parameter later relabeled as a prediction; it is a linear algebra check applied to the KKT solution of the original problem. Proposition 1 is a direct feasible-set inclusion argument. Propositions 2 and 3 are KKT-based implications that rely on convexity and the existence of Lagrange multipliers, and Corollary 1 follows from those propositions together with uniqueness of the optimum. The only self-citation, to [7,8] for the convex Lukasiewicz fragment, is an input assumption about convexity of the constraint set rather than the conclusion being derived; it is not invoked to forbid alternatives or to supply the uniqueness theorem. No definition is circular in the sense of defining X in terms of Y, and no known empirical result is renamed as a new contribution. Even if the uniqueness argument in Corollary 1 is mathematically questionable because the unregularized bias terms prevent strict convexity in (omega,b), that is a correctness concern, not circularity. Accordingly, no circular step is identified.
Assumptions & free parameters
assumptions (5)
- domain assumption Formulas in the knowledge base belong to the convex Lukasiewicz fragment, so each 1 - f_h is convex and piecewise linear.
- domain assumption KKT conditions are necessary and sufficient for Problem 1, and the feasible set is nonempty.
- standard math Each p_j is an RKHS function and the optimal solution has the kernel expansion of eq. (2) and (3).
- domain assumption The Lukasiewicz logic semantics in Table 1 correctly represents the intended logical constraints.
- standard math The Gram matrix K is positive-definite when uniqueness of the solution is required.
Cite this review
Pith. "Pith review of Conditions for Unnecessary Logical Constraints in Kernel Machines." pith.science (2026). https://pith.science/paper/ZBKH7YRG
@misc{pith2026190900216,
author = {Pith},
title = {Pith review of: Conditions for Unnecessary Logical Constraints in Kernel Machines},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZBKH7YRG}},
note = {Machine review of arXiv:1909.00216}
}
read the original abstract
A main property of support vector machines consists in the fact that only a small portion of the training data is significant to determine the maximum margin separating hyperplane in the feature space, the so called support vectors. In a similar way, in the general scheme of learning from constraints, where possibly several constraints are considered, some of them may turn out to be unnecessary with respect to the learning optimization, even if they are active for a given optimal solution. In this paper we extend the definition of support vector to support constraint and we provide some criteria to determine which constraints can be removed from the learning problem still yielding the same optimal solutions. In particular, we discuss the case of logical constraints expressed by Lukasiewicz logic, where both inferential and algebraic arguments can be considered. Some theoretical results that characterize the concept of unnecessary constraint are proved and explained by means of examples.
Figures
Reference graph
Works this paper leans on
-
[1]
In: Proceedings of the fifth annual workshop on Computational learning theory
Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: Proceedings of the fifth annual workshop on Computational learning theory. pp. 144–152. ACM (1992)
work page 1992
-
[2]
Cambridge university press (2004)
Boyd, S., Vandenberghe, L.: Convex optimization. Cambridge university press (2004)
2004
-
[3]
Machine Learning 20(3), 273– 297 (Sep 1995)
Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20(3), 273– 297 (Sep 1995). https://doi.org/10.1023/A:1022627411411, https://doi.org/10. 1023/A:1022627411411
-
[4]
In: Proceedings of the 20th International Conference on Machine Learning (ICML-03)
Cumby, C.M., Roth, D.: On kernel methods for relational learning. In: Proceedings of the 20th International Conference on Machine Learning (ICML-03). pp. 107–114 (2003) Conditions for Unnecessary Logical Constraints in Kernel Machines 13
work page 2003
-
[5]
Machine learning 86(1), 57–88 (2012)
Diligenti, M., Gori, M., Maggini, M., Rigutini, L.: Bridging logic and kernel ma- chines. Machine learning 86(1), 57–88 (2012)
work page 2012
-
[6]
Diligenti, M., Gori, M., Sacc` a, C.: Semantic-based regularization for learning and inference. Artificial Intelligence (2015)
work page 2015
-
[7]
In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases
Giannini, F., Diligenti, M., Gori, M., Maggini, M.: Learning lukasiewicz logic frag- ments by quadratic programming. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases. pp. 410–426. Springer (2017)
work page 2017
-
[8]
IEEE Transactions on Fuzzy Systems (2018)
Giannini, F., Diligenti, M., Gori, M., Maggini, M.: On a convex logic fragment for learning and reasoning. IEEE Transactions on Fuzzy Systems (2018)
2018
Show all 19 references
-
[9]
Neural computation 27(2), 388–480 (2015)
Gnecco, G., Gori, M., Melacci, S., Sanguineti, M.: Foundations of support con- straint machines. Neural computation 27(2), 388–480 (2015)
2015
-
[10]
In: Lu, B.L., Zhang, L., Kwok, J
Gori, M., Melacci, S.: Support constraint machines. In: Lu, B.L., Zhang, L., Kwok, J. (eds.) Neural Information Processing. pp. 28–37. Springer Berlin Heidelberg, Berlin, Heidelberg (2011)
2011
-
[11]
IEEE transac- tions on neural networks and learning systems 24(5), 825–831 (2013)
Gori, M., Melacci, S.: Constraint verification with kernel machines. IEEE transac- tions on neural networks and learning systems 24(5), 825–831 (2013)
2013
-
[12]
H´ ajek, P.: Metamathematics of fuzzy logic, vol. 4. Springer Science & Business Media (1998)
1998
-
[13]
arXiv preprint arXiv:1603.06318 (2016)
Hu, Z., Ma, X., Liu, Z., Hovy, E., Xing, E.: Harnessing deep neural networks with logic rules. arXiv preprint arXiv:1603.06318 (2016)
2016 arXiv
-
[14]
Computational Optimization and Applications 51(1), 125–157 (2012)
Jung, J.H., OLeary, D.P., Tits, A.L.: Adaptive constraint reduction for convex quadratic programming. Computational Optimization and Applications 51(1), 125–157 (2012)
2012
-
[15]
In: Discovery science
Muggleton, S., Lodhi, H., Amini, A., Sternberg, M.J.: Support vector inductive logic programming. In: Discovery science. vol. 3735, pp. 163–175. Springer (2005)
2005
-
[16]
Paulsen, V.I., Raghupathi, M.: An introduction to the theory of reproducing kernel Hilbert spaces, vol. 152. Cambridge University Press (2016)
2016
-
[17]
Rockafellar, R.T., Wets, R.J.B.: Variational analysis, vol. 317. Springer Science & Business Media (2009)
2009
-
[18]
arXiv preprint arXiv:1606.04422 (2016)
Serafini, L., Garcez, A.d.: Logic tensor networks: Deep learning and logical reason- ing from data and knowledge. arXiv preprint arXiv:1606.04422 (2016)
2016 arXiv
-
[19]
In: AI* IA
Serafini, L., Garcez, A.S.d.: Learning and reasoning with logic tensor networks. In: AI* IA. pp. 334–348 (2016)
2016
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.