{"id":"2ec26025-5f26-40d5-ab87-e062ef46030c","arxiv_id":"1909.00216","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The paper defines an 'unnecessary constraint' in kernel-based learning from logical constraints and shows it can be detected by logical consequence or by finding alternative Lagrange multipliers that preserve the optimal kernel expansion.","lead":"Support vector machines keep only a few training points, the support vectors, that define the learned boundary. This paper studies which logical rules used to train a kernel machine can be dropped without changing the learned answer, and gives criteria for spotting such unnecessary rules.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Corollary 1 is false as stated: with an unregularized bias, a positive-definite Gram matrix does not guarantee uniqueness of the reduced problem, so a zero-multiplier KKT solution can coexist with the constraint being necessary.","rationale":"The reader's weakest assumption concerned KKT qualifications and the boundary of the convex Lukasiewicz fragment. Those are legitimate presentation issues, but they are not the decisive failure. The more load-bearing problem is internal: Corollary 1's sufficiency theorem is false as stated. The proof conflates uniqueness of the kernel-expansion coefficients alpha with uniqueness of the full learning problem. Because bias terms b_j are unregularized, a positive-definite Gram matrix does not exclude a continuum of optimal bias shifts once a constraint is removed. The counterexample above uses only constructs explicitly present in the paper: one predicate, consistency constraints, the p->0 logical encoding from Section 4.1, and a positive-definite kernel Gram matrix. It satisfies Corollary 1's premise (the original problem has a unique optimal function) while violating its conclusion (removing the constraint enlarges the set of optimal functions). This is an internal inconsistency, not a disagreement with external consensus. Propositions 1-3 may still be salvageable and the paper could be repaired by requiring uniqueness of the reduced problem, or by regularizing the bias, but as written the central sufficiency claim is false. I therefore recommend REJECT rather than the reader's CONDITIONAL, while noting that the failure is localized and fixable.","tokens_in":10515,"tokens_out":23112,"duration_ms":242985,"concrete_test":"Run the one-predicate, one-sample problem with linear kernel on S={1}: minimize 1/2 omega^2 subject to 0<=omega+b<=1 and the logical constraint omega+b<=0. Verify that (omega,b)=(0,0) is the unique original optimum and that zero is a valid KKT multiplier for the logical constraint. Then remove the logical constraint and solve the reduced problem, recording that every (0,b) with b in [0,1] is optimal. If both hold, Corollary 1's sufficiency claim is refuted within the paper's own formal setup.","verdict_should_be":"REJECT","load_bearing_attack":"Corollary 1 claims that if either HP or HP has a unique solution, then the existence of a KKT solution to Problem 2 with zero multipliers for phi_hbar is sufficient for phi_hbar to be unnecessary. The proof asserts that uniqueness follows from positive-definiteness of K, but the primal problem also contains unregularized bias terms b_j. The objective is therefore not strictly convex in (omega,b), and uniqueness of the original and reduced problems are not equivalent. A concrete counterexample fits the paper's own setting. Let J=1, S_1={1}, use the linear kernel, and set p(x)=omega x+b with loss 1/2 omega^2. Add consistency 0<=p(1)<=1 and the logical constraint p(1)<=0, i.e. the p->0 encoding used in Section 4.1. The original problem forces omega+b=0 and has the unique optimal solution (omega,b)=(0,0), p=0. At this point the loss gradient is zero, so all KKT multipliers can be zero, yielding a KKT solution of Problem 2 with lambda=0 for the logical constraint. Removing the logical constraint leaves the feasible set 0<=omega+b<=1, whose optimizers are omega=0, b in [0,1], a continuum of distinct functions. By Definition 1 the constraint is necessary. Thus the 'unique solution' branch of Corollary 1 is invalid: the algebraic criterion can certify removal of a constraint that changes the optimal solution set.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies learning-from-constraints problems in kernel machines where logical constraints are expressed in Łukasiewicz logic. It extends the notion of support vector to support constraint and defines a constraint to be unnecessary when removing it from the optimization problem leaves the set of optimal solutions unchanged. The main results are: (i) Proposition 1, that a logical consequence relation among formulas implies redundancy; (ii) Propositions 2 and 3, which relate redundancy to the existence of a Lagrange-multiplier solution of a linear system (Problem 2) in which the multipliers of the candidate constraint vanish; and (iii) Corollary 1, which claims that under uniqueness of the optimal solution this multiplier condition is also sufficient. The paper illustrates the criteria on transitivity and pointwise-constraint examples solved in MATLAB. The overall goal is to prune unnecessary constraints before or during optimization while preserving the optimal solutions.","tokens_in":10832,"tokens_out":11780,"duration_ms":114636,"significance":"If the results were correct, the paper would give a practically useful algebraic criterion for detecting redundant constraints in kernel machines with logical constraints, and it would clarify the relationship between logical deducibility and constraint redundancy. The formal definition of unnecessary constraint and the clean statement of Proposition 1 are valuable, and the examples are helpful for understanding the intended method. However, the central sufficiency result (Corollary 1) is false as stated because the objective is not strictly convex in the bias terms, and the manuscript leaves important regularity and well-posedness assumptions implicit. The positive contributions are therefore conditional on a substantial revision of the main theorems and their hypotheses.","major_comments":[{"comment":"The phrase 'KKT-solution of Problem 2' is used throughout Section 3, but Problem 2 is only a linear system and has no KKT conditions of its own. The proofs require that the constructed multiplier vector additionally satisfies the KKT conditions of the original learning problem, but this extra condition is not stated in Problem 2 or in the surrounding definitions. Adding a formal definition of this object would remove a significant source of ambiguity.","section":"Sections 3.1 and 3.2"}],"minor_comments":[{"comment":"The name of the logic is typeset inconsistently ('Lukasiewicz' without the diacritic in the table caption); this is a presentation issue and should be made uniform.","section":"Table 1"},{"comment":"Definition 1 writes the loss as Loss(alpha)=sum alpha'_j K_j alpha_j and does not indicate where the bias terms b_j appear in the constraints of HP and HP; since Problem 1 explicitly includes b_j in p_j(x), the relationship between the two formulations should be stated explicitly.","section":"Definition 1 and Section 2.1"},{"comment":"The phrase 'support constraint' is introduced casually and never given a formal definition separate from the definition of unnecessary constraint; aligning the two notions in one place would improve readability.","section":"Abstract and Section 1"}],"recommendation":"major_revision","confidential_remarks":"The bias-related counterexample to Corollary 1 is decisive in my reading. The authors can likely repair the paper by adding a positive regularization on b_j (or by explicitly restricting to bias-free expansions) and by stating Slater-type conditions under which the KKT system is necessary and sufficient. The fragment/example mismatch also needs a consistent treatment. These are substantial but local fixes; I would not reject the paper outright, but the current version should not be accepted without revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nShort version: the paper has a genuinely useful idea — detect unnecessary logical constraints by looking for alternate KKT multiplier vectors that zero out one constraint's block — but its main sufficiency theorem, Corollary 1, has a real gap that I think is a false claim, not just a proof omission.\n\nWhat is new: Definition 1 (unnecessary constraint) and the multiplier-null-space criterion in Problem 2 are a clean formalization of the support-constraint idea, and Propositions 2 and 3 give a nice necessary/sufficient pair in the convex case. Proposition 1 (logical consequence implies redundancy) is standard but appropriately applied here. The examples are small but do show the mechanics. The self-cited convex fragment result is an input assumption, not the target of the proof, so I don't see a circularity problem.\n\nThe soft spots: the paper conflates the full kernel-expansion coefficients α* with the logical part α*(L) in Problem 2 — M·λ = α* only makes sense if α* is the logical contribution, and that is never said explicitly. The convex fragment claim is also inconsistent with the examples, which all use implication. But the serious problem is Corollary 1. The proof says uniqueness follows from a positive-definite Gram matrix, but the primal has unregularized bias terms b_j, so the objective is not strictly convex in (omega,b). The stress-test counterexample is right: with one sample, linear kernel, loss 1/2 omega^2, and constraints 0<=omega+b<=1 and omega+b<=0, the original problem has unique solution (0,0), all logical multipliers can be zero, and removing the logical constraint leaves omega=0, b in [0,1] as optimizers. So the constraint is necessary by Definition 1, yet the criterion says removable. This kills the 'HP has a unique solution' branch of Corollary 1. The other branch may survive, but the paper uses both interchangeably.\n\nAlso, KKT existence is just assumed ('assuming to satisfy the associated KKT-conditions') without stating a constraint qualification; for a convex piecewise-linear feasible set you typically need Slater or a linear-constraint qualification.\n\nVerdict: the core criterion and the necessary direction (Prop 2) are worth keeping, and Prop 3's inclusion result is fine under its premises. But the sufficiency claim needs either regularization of the bias, an explicit assumption that the bias is fixed, or a different argument. This is fixable. I'd send it to a referee who knows convex optimization and ask them to re-check the uniqueness argument carefully. The paper deserves a serious referee despite the flaw, because the algebraic redundancy idea is nontrivial and the flaw is concrete and repairable.","headline":"The algebraic redundancy criterion is a useful idea, but Corollary 1's sufficiency claim is false as stated because unregularized biases break uniqueness.","tokens_in":11284,"tokens_out":8670,"would_cite":false,"duration_ms":111284,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T05","68T27","03B50","90C20"],"pacs":[],"model":"deepseek-v4-flash","headline":"A logical constraint is unnecessary for a kernel-machine problem exactly when its Lagrange multipliers can be zeroed, provided the optimum is unique.","keywords":["support vectors","support constraints","kernel machines","learning from constraints","Lukasiewicz logic","unnecessary constraints","Lagrange multipliers","quadratic programming"],"falsifier":"Solve a convex quadratic program with a positive-semidefinite Gram matrix and two logical constraints, and exhibit a KKT solution in which one constraint's multipliers are all zero while removing that constraint adds new optimal solutions; such an example would show why Corollary 1 needs uniqueness, while failure to find one would suggest the sufficiency result can be strengthened.","tokens_in":10345,"feed_emoji":"✂️","tokens_out":7867,"duration_ms":68923,"temperature":0.7,"pith_summary":"This paper asks when a logical constraint in kernel-machine learning can be removed without changing the set of optimal solutions, extending the SVM idea that only support vectors matter. It establishes an algebraic criterion: a constraint $\\phi_{\\bar h}$ is unnecessary for the optimization if there is a KKT solution of the linear system $M\\lambda = \\alpha^*$ in which all multipliers attached to $\\phi_{\\bar h}$ vanish. That condition is necessary for redundancy in general and becomes sufficient when the optimal solution is unique, as happens with positive-definite Gram matrices; otherwise it certifies only that the original optima remain optimal after removal. For constraints expressed in the convex Lukasiewicz fragment, logical entailment of $\\phi_{\\bar h}$ from the other formulas is a sufficient reason for redundancy, but the multiplier criterion is broader because it is local to the dataset. The practical point is a way to detect redundant rules before or during quadratic programming.","feed_headline":"One multiplier test finds redundant constraints in kernel learners","feed_subtitle":"Droppable logical rules are those whose Lagrange multipliers can be set to zero, when the optimum is unique.","key_machinery":"The load-bearing object is the linear system called Problem 2, $M\\lambda = \\alpha^*$, together with the nullspace of $M$. The matrix $M$ encodes, for every affine piece of every Lukasiewicz formula, how that piece weights the groundings of each predicate; $\\alpha^*$ is the coefficient vector of an optimal solution in the kernel expansion. Any $\\lambda$ solving this system yields the same optimal solution, and $\\mathrm{Ker}(M)$ parametrizes alternative multiplier assignments. A constraint is removable when the affine subspace $\\lambda^* + \\mathrm{Ker}(M)$ intersects the coordinate subspace in which that constraint's multiplier block is zero. This algebraic criterion is the mechanism that turns support-vector intuition into a testable condition on Lagrange multipliers.","core_discovery":"The paper's central claim is that redundancy of a hard constraint is captured by the existence of an alternative set of Lagrange multipliers that reproduce the same optimal solution while setting the target constraint's multiplier block to zero. Concretely, with $\\alpha^*$ denoting the coefficients of an optimal solution in the kernel expansion and $M$ the matrix whose rows are the affine pieces of the logical constraints, the paper studies solutions of $M\\lambda = \\alpha^*$. If a solution has $\\lambda_{\\bar h,i}=0$ for every affine component $i$ of constraint $\\phi_{\\bar h}$, then that constraint's contribution can be transferred to the other constraints. The paper proves this condition is necessary when $\\phi_{\\bar h}$ is unnecessary (Proposition 2), sufficient for the original optima to survive removal (Proposition 3), and equivalent to redundancy when the optimal solution is unique (Corollary 1). It also proves that logical consequence $\\{\\phi_h : h\\neq \\bar h\\}\\models \\phi_{\\bar h}$ implies redundancy (Proposition 1).","pith_inferences":["Editorial inference: because Proposition 3 gives only inclusion of the optimal sets when the optimum is not unique, a user who removes a constraint on the strength of a zero-multiplier KKT solution may enlarge the set of optima; checking uniqueness of the solution is therefore part of applying the criterion safely.","Editorial inference: the paper states that the convex fragment uses only the operators $\\wedge$ and $\\oplus$, yet all worked examples use implication; this suggests either the fragment is wider than stated or the examples rely on rewriting implications in terms of the convex operators, and an explicit rewriting would pin down the exact boundary.","Editorial inference: the nullspace computation could be performed symbolically before training once the optimal coefficient vector is known, turning the criterion into a preprocessing pass; the paper does not discuss the algorithmic complexity or large-scale implementation of such a step."],"forward_implications":["Redundant logical rules can be dropped before solving the quadratic program, shrinking the constraint matrix without changing the answer, whenever the criterion finds a zero-block KKT solution and uniqueness holds.","In the convex Lukasiewicz fragment, any constraint that is a logical consequence of the remaining knowledge base is certifiably removable, giving a direct link between deductive logic and optimization.","The criterion applies uniformly to pointwise, consistency, and logical constraints, because the first two can be written as logical formulas; this is demonstrated by removing a supervised label constraint in Example 4.","With a positive-definite Gram matrix, detecting an unnecessary constraint becomes a linear feasibility problem in the nullspace of $M$."],"supporting_citations":[{"why":"Supplies the convex Lukasiewicz fragment that makes the logical constraints convex piecewise linear and the KKT conditions applicable.","marker":"[8]"},{"why":"Introduces the notion of support constraints that this paper refines and redefines as unnecessary constraints.","marker":"[10]"},{"why":"Provides the convex-optimization background on KKT sufficiency and uniqueness of solutions underpinning Propositions 2 and 3 and Corollary 1.","marker":"[2]"},{"why":"Defines support vectors in SVMs, the analogy that motivates extending the concept to constraints.","marker":"[3]"},{"why":"Supplies the max-of-affine representation theorem used to rewrite each Lukasiewicz formula as a set of linear inequalities.","marker":"[17]"},{"why":"Establishes the learning-from-constraints setting with kernel machines on which the paper builds.","marker":"[9]"}],"fun_headline_variants":["Zero-multiplier test spots removable logical constraints","Drop needless rules: kernel learner's Lagrange multiplier check","Which logical constraints are unnecessary? A multiplier criterion","Kernel machines: identify redundant constraints via multipliers","Support constraints: how to know which ones you can ignore"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All the redundancy criteria assume the learning problem satisfies the KKT conditions, meaning a constraint-qualification condition and full convexity of the constraints; if that assumption fails, the zero-multiplier test no longer certifies redundancy.","fun_headline_variants_meta":{"raw":{"variants":["Zero-multiplier test spots removable logical constraints","Drop needless rules: kernel learner's Lagrange multiplier check","Which logical constraints are unnecessary? A multiplier criterion","Kernel machines: identify redundant constraints via multipliers","Support constraints: how to know which ones you can ignore"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00021,"raw_usage":{"total_tokens":1382,"prompt_tokens":890,"completion_tokens":492,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":506,"completion_tokens_details":{"reasoning_tokens":416}},"tokens_in":506,"tokens_out":492,"duration_ms":5723,"temperature":1.0,"reasoning_tokens":416,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T05:59:04.308910+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Solve a convex quadratic program with a positive-semidefinite Gram matrix and two logical constraints, and exhibit a KKT solution in which one constraint's multipliers are all zero while removing that constraint adds new optimal solutions; such an example would show why Corollary 1 needs uniqueness, while failure to find one would suggest the sufficiency result can be strengthened.","supporting_citations":[{"cited_title":"In: Lu, B.L., Zhang, L., Kwok, J","cited_arxiv_id":null,"evidence_quote":"Introduces the notion of support constraints that this paper refines and redefines as unnecessary constraints."},{"cited_title":"Neural computation 27(2), 388–480 (2015)","cited_arxiv_id":null,"evidence_quote":"Establishes the learning-from-constraints setting with kernel machines on which the paper builds."}],"review_version":1}