REVIEW 4 major objections 5 minor 15 references
NeurOWL claims a candidate subsumption can be verified against an incomplete ontology and explained by the missing axioms that would entail it, without preselecting hypotheses.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
NeurOWL explains plausible subsumptions in incomplete OWL ontologies by finding missing bridge axioms with embedding ranking and LLM validation.
T0 review reviewed 2026-08-03 challenge →
load-bearing objection NeurOWL is a genuinely useful pipeline for missing-axiom suggestion and subsumption plausibility, but its recovery guarantee (Theorem 1) is not actually proven for the implemented system. the 4 major comments →
NeurOWL: An LLM-Based Neural-symbolic Framework for Incomplete OWL Ontology Reasoning
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
On the paper's own terms, the central discovery is that a non-entailed subsumption query can be handled end-to-end by decomposing it into a search for bridge concepts: first look at logical children of B and parents of A (Stages 2a/2b), then use embedding scores to retrieve top-k intermediate concepts C such that A⊑C⊑B (Stage 3a), and if no bridge is found, treat the subsumption itself as a candidate missing axiom and validate it directly with an LLM (Stage 3b). The system returns True with an explanation E iff the LLM validates a bridge, where E is the new axioms along with a justification for the entailed part. Complementing the design, Theorem 1 states that for normalized EL ontologies, i
What carries the argument
The load-bearing object is the "bridge concept" — an intermediate concept C that can connect the left-hand side A to the right-hand side B of a candidate subsumption A⊑B. NeurOWL's pipeline is built around finding such C in three complementary ways: logical bridging over direct children of B and direct parents of A; embedding-based bidirectional retrieval that scores candidate C by the average of s(A⊑C) and s(C⊑B); and a direct LLM check when no bridge exists. The embedding model provides a ranked candidate set, the LLM performs semantic verification, and a reasoner supplies justifications for the non-missing part, so the final explanation is a logically sound combination of new and existing
Load-bearing premise
The recovery guarantee in Theorem 1 assumes that NeurOWL's neural components actually identify the correct missing axioms at each step; the theorem's conditions only ensure the candidate concept appears in the Stage 3 set, not that it survives top-k retrieval and the LLM's YES/NO test.
What would settle it
Feed NeurOWL a normalized EL ontology with a known minimal solution S, remove one axiom, and query the resulting non-entailed subsumption. The theorem predicts iterative NeurOWL returns exactly S; if repeated runs on such constructed cases return a different or incomplete set while the correct bridge is in the candidate space, the recovery claim is false. A sharper test: pick a case where the ground-truth bridge is in the top-k but the LLM rejects it while accepting a plausible wrong bridge; systematic occurrence of this pattern would falsify the explanation-accuracy claims even with perfect r
If this is right
- Users can query arbitrary subsumptions against an incomplete ontology and receive both a YES/NO plausibility judgment and, on YES, a set of missing axioms that entail the query.
- The approach removes the need for a predefined hypothesis space of candidate missing axioms, which prior TBox abduction methods required.
- Iterative application lets the system recover a minimal set of missing axioms, not just a single ad hoc patch, for normalized EL ontologies (Theorem 1).
- The framework works with a training-free embedding model (SBERT) as well as a fine-tuned ontology embedding (OnT), with the fine-tuned version handling complex existential-restriction bridges better.
- On hard negative samples (plausible-looking but false subsumptions), NeurOWL improves F1 by up to 0.13 over the strongest baseline in the reported experiments.
Where Pith is reading between the lines
- If LLM verification is as reliable as these results suggest, the pipeline could be pointed the other way: given an ontology with known entailments, identify where the LLM disagrees with the reasoner to surface candidate missing axioms for expert review.
- The bridge-concept decomposition is not specific to EL; the same retrieve-then-verify skeleton could be adapted to richer description logics, though the recovery guarantee would need a new proof.
- The method's dependence on top-k retrieval means explanation quality is capped by retrieval recall; in the complex ∃r.B setting, retrieval H@100 is only 0.577 even with fine-tuning, so the bottleneck has already moved from reasoning to embedding quality.
- A natural testable extension is to treat the LLM's YES/NO margin (e.g., via logprobs) as a confidence score and let users trade off precision and recall per domain.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes NeurOWL, a neuro-symbolic framework for reasoning over incomplete OWL ontologies. Given an incomplete ontology O' and a candidate subsumption A⊑B that is not entailed, NeurOWL decides whether the subsumption is semantically plausible and, if so, returns an explanation E consisting of at most two missing axioms such that O' ∪ E ⊢ A⊑B. The pipeline combines a DL reasoner (Stage 1), logical bridging through parents/children (Stage 2), embedding-based candidate retrieval with LLM validation (Stage 3a), and a direct LLM plausibility check (Stage 3b). The authors report experiments on FoodOn, SNOMED CT, and GO under random and hard negative sampling, with F1 scores up to 0.970, and also prove a theorem (Theorem 1) claiming that iterative applications of NeurOWL can recover every axiom in a subset-and-entailment-minimal solution for normalized EL ontologies.
Significance. If the claims are correct, NeurOWL addresses a genuinely useful generalization of TBox abduction: it does not require a predefined hypothesis space and also handles the case where the input subsumption is false. The empirical methodology is reasonable, including ablations, a complex setting with existential restrictions, and case studies. The main theoretical contribution, however, is not established for the implemented system as described, and the empirical support would benefit from statistical rigor. The paper's strengths are its clear system design, the use of real-world ontologies, and the inclusion of explanations in the evaluation metrics (X-F1, X-F1*). The central recovery guarantee is the load-bearing element, and it currently rests on unstated oracle assumptions.
major comments (4)
- [Theorem 1 / Supplementary 'Poof of Theorem 1'] Theorem 1 is not a theorem about the implemented NeurOWL. The theorem's hypotheses only require the Stage 3 candidate set to contain certain concepts; they say nothing about the embedding ranker placing the correct bridge within top-k, nor about the LLM accepting it. Yet the proof repeatedly asserts 'NeurOWL can directly identify the missing axioms' and 'we can assume NeurOWL identify ∃r.B2 ... in Stage 3a'. These are extra oracle assumptions. In the actual Stage 3a, candidates are ranked by s̄(C) and only top-k are sent to the LLM; a true bridge can be ranked outside top-k or rejected by the LLM. The theorem therefore describes an idealized system with a perfect retriever and perfect LLM, not the one evaluated. This directly undermines the abstract's and introduction's claim that iterative applications of NeurOWL 'recover any subset-and-entailment-minimal solutions'. Please either state
- [Table 1 / Experimental Setting] All reported F1, X-F1, and X-F1* values are point estimates without error bars, multiple seeds, or significance tests. Several comparisons are close (e.g., FoodOnA random: NeurOWL OnT 0.960 vs OnT baseline 0.910; SnomedA random: 0.968 vs 0.943). The claim of 'strong and robust performance across different domains' is not fully supported without variance information or at least a repeated-seed analysis. This is a load-bearing issue for the empirical contribution, as the differences could be within noise, especially on the smaller or harder datasets.
- [Table 2 and Table 1 on Snomed∃] There is a notable tension between the retrieval results and the reported F1 scores for Snomed∃. In Table 2, H@1 for Stage 3a is 0.001 (OnT) and H@100 is 0.577; with the default top-k=10, the correct existential bridge is essentially never retrieved. Yet Table 1 reports F1=0.924 for NeurOWL OnT on Snomed∃ random. This implies that the F1 score is dominated by Stage 3b's direct LLM check, not by the bridging mechanism that supposedly provides explanations. The X-F1* of 0.193 confirms that explanations are rarely correct. The paper should explicitly discuss this: the 'success' on F1 does not validate the explanation-generation component, and the reader should not infer that NeurOWL is solving the task as stated for complex bridges.
- [Dataset Construction / Positive samples] The construction of positive samples for the standard setting may introduce a bias toward easy cases. Positive samples are randomly sampled from all (A,B) such that O⊨A⊑B and O'⊭A⊑B. With only 5% pruning, many such entailments in O are lost due to a single missing axiom, and the bridging concept is often a direct neighbor. The hard-negative setting is more challenging, but the random-negative setting seems to overrepresent simple one-hop cases. Please report the distribution of positive samples by the number of required missing axioms and by the distance between A and B in the taxonomy, so readers can assess the difficulty of the benchmark.
minor comments (5)
- [Supplementary, Proof header] The proof section is titled 'Poof of Theorem 1'; should be 'Proof'. Also, 'obatined' typo in the same section, and 'Snd∃' in Table 2 is inconsistent with 'Snomed∃'.
- [Figure 2] The labels 'INOR (OnT)' and 'INOR (SBERT)' appear to be typos; presumably should be 'NeurOWL'. This makes the figure confusing.
- [Evaluation, footnote 2] The definition of direct child has a typo: 'A is a direct children' should be 'A is a direct child'. Also, the direct-child condition is defined relative to O (the original ontology), while the pruning is on O'; this should be clarified because directness in O' may differ.
- [Related Work] The related work is thorough, but the discussion of Zhao et al. (2026) is very brief; since that work is described as close in using LLMs for plausibility with human verification, a few sentences on the difference would help contextualize NeurOWL.
- [System description, Stage 3a] The scoring function is defined as the average of s(A⊑C) and s(C⊑B), but it is not clarified whether s is the same as the subsumption score used for classification in the baselines. This should be stated explicitly for reproducibility.
Circularity Check
Theorem 1's recovery guarantee is assumed in its own proof; the empirical pipeline itself is not circular.
specific steps
-
other
[Supplementary material, 'Poof of Theorem 1', Base Case and Induction step (b)]
"In each case, NeurOWL can directly identify the missing axioms in S by identifying A1 in Stage 2a, Stage 2b, or Stage 3a for the first, second, or third case, respectively. ... In the first round of applying NeurOWL with input query α, we can assume NeurOWL identify ∃r.B2 as an intermediate concept in Stage 3a."
The theorem's conclusion is that iterative applications of NeurOWL recover every axiom in S. The proof's base case and induction step simply assert that NeurOWL can directly identify the necessary missing axioms/bridge concepts, which is exactly the capability to be established. The stated hypotheses only ensure the concept is in the Stage 3 candidate set; they do not imply it is ranked within top-k or accepted by the LLM, both of which the implemented system requires before returning an explanation. Thus the formal recovery claim is not derived from the assumptions—it is assumed as a premise of the proof.
full rationale
The main NeurOWL verification/abduction pipeline is not circular: OnT and SBERT are not fitted to the target subsumption labels, the LLM judges plausibility from external knowledge, and the positive/negative samples and explanation ground truths are defined from the complete ontologies after pruning, independent of NeurOWL's outputs. The only significant circular step is in the supplementary proof of Theorem 1, where the recovery guarantee is assumed as an identification ability rather than proved from the algorithm's stated behavior. Because Theorem 1 is a stated contribution but the empirical results and the core stage-based framework retain independent content, the circularity is partial, not total.
Axiom & Free-Parameter Ledger
free parameters (5)
- pruning_rate r =
5%
- candidate_top_k =
10 (default); 5/20 in ablations
- direct_parents/children restriction (k=1) =
direct only
- OnT fine-tuning hyperparameters =
20 epochs, lr 1e-5, batch 256, margins 3.0/0.5
- baseline embedding thresholds =
max F1 on 100-sample validation drawn from test set
axioms (6)
- standard math EL normalization plus completeness of the inference rules in Table 4 (adapted from Baader et al. 2005; Arif et al. 2016).
- domain assumption Textual labels of concepts carry reusable semantics aligned with formal subsumption.
- domain assumption LLM YES/NO answers about subsumption are a valid estimator of semantic plausibility.
- ad hoc to paper Any true missing axiom can be witnessed by a bridge concept of the form A, ∃r.A, or A⊓B from the ontology vocabulary.
- domain assumption Randomly removing 5% of TBox axioms produces an 'incomplete ontology' representative of real incompleteness.
- ad hoc to paper The neural verifier identifies the correct bridge concept whenever it exists.
Cite this review
Pith. "Pith review of NeurOWL: An LLM-Based Neural-symbolic Framework for Incomplete OWL Ontology Reasoning." pith.science (2026). https://pith.science/paper/GYTE2YBZ
@misc{pith2026260715776,
author = {Pith},
title = {Pith review of: NeurOWL: An LLM-Based Neural-symbolic Framework for Incomplete OWL Ontology Reasoning},
year = {2026},
howpublished = {\url{https://pith.science/paper/GYTE2YBZ}},
note = {Machine review of arXiv:2607.15776}
}
read the original abstract
OWL ontologies provide a formal knowledge representation framework that enables semantic reasoning, and have been widely adopted across domains such as healthcare and bioinformatics. In practice, however, real-world ontologies are often incomplete, which pose challenges for reasoning. In this work, we focus on a fundamental subsumption reasoning problem: given an incomplete ontology and a candidate (non-entailed) subsumption, determine whether the subsumption is semantically plausible and, if so, providing a logically sound explanation containing potential missing axioms. This task unifies subsumption verification with ontology abduction, and generalizes the latter by removing the need for a predefined candidate set of missing axioms. To address this subsumption reasoning problem, we propose NeurOWL, an end-to-end neuro-symbolic framework that jointly performs verification and abduction, leveraging both formally defined semantics and textual semantics through Large Language Models and ontology embeddings. We evaluate NeurOWL on real-world ontologies across multiple domains, demonstrating strong and robust performance across different domains.
Figures
Reference graph
Works this paper leans on
-
[1]
In Stage 2, we find atomic conceptsAsuch that O′ |=C⊑A orO ′ |=A⊑D
When the input axiom contains complex concepts, for example,whenitisoftheformC⊑D,wecanstillapply NeurOWL. In Stage 2, we find atomic conceptsAsuch that O′ |=C⊑A orO ′ |=A⊑D. Similarly, in Stage 3, we can check every conceptC′ in the given candidates setCto determine whether C⊑C ′ ⊑D holds
-
[2]
, C⊑Dn as inputs, rather than using the single conjunctive axiom directly
If we obtain an intermediate result of the form C⊑D 1 ⊓ · · · ⊓Dn, then, in the next iteration, we use thendifferent axioms C⊑D 1, . . . , C⊑Dn as inputs, rather than using the single conjunctive axiom directly
-
[3]
If we obtain the intermediate result of the form ∃r.C⊑ ∃r.D then,inthenextiteration,wewillinputboth∃r.C⊑ ∃r.D andC⊑D. Specifically, in the normalizedELcase, it sufficestoinputonlyC⊑D,asshowninCase(b)ofthe induction step in the following proof. The Proof Our proof is based on the following theorem, adapted from Arif et al. (2016); Baader, Brandt, and Lutz ...
arXiv 2016
-
[10]
{A}" a subclass of
Is "{A}" a subclass of "{candidate_1}"?
-
[11]
{A}" a subclass of
Is "{A}" a subclass of "{candidate_2}"? . . . Answer format (one per line):
-
[12]
A” for Stage 2b, or all atomic concepts for Stage 3a. The questions ask whether the candidates are subclasses of “B
YES/NO Figure5:PrompttemplateusedforStage2a.ThepromptsforStage2b/3afollowasimilarprocedure,butwithdifferentcandidate sets: candidates are parents of “A” for Stage 2b, or all atomic concepts for Stage 3a. The questions ask whether the candidates are subclasses of “B” for Stage 2a, or both directions for Stage 3a. LLM Prompt You are an ontology expert analy...
-
[13]
"{A}" plausibly has this role-filler property
-
[14]
This filler captures the essential characteristic that defines membership in "{B}" {numbered list of candidates} Respond in the format:
-
[15]
r some B
YES/NO ... Only output the numbered YES/NO answers, nothing else. Figure 6: Prompt template for Stage 3a with relation restriction concepts∃r.Bas candidates.{A}and{B}are replaced with concept labels;{n}with the number of candidates; the numbered list enumerates each candidate as “r some B”. LLM Prompt You are an ontology expert in the biomedical and food ...
-
[2019]
InProceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, 6103–6109
EL Embeddings: Geometric Construction of Mod- els for the Description Logic EL++. InProceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, 6103–6109. International Joint Conferences on Artificial Intelligence Organization. ISBN 978-0-9992411- 4-1. Kulmanov, M.; Smaili, F. Z.; Gao, X.; and Hoehndorf, R
-
[2020]
InProceedings of the International Conference on PrinciplesofKnowledgeRepresentationandReasoning,vol- ume 17, 592–602
Signature-based abduction for expressive description logics. InProceedings of the International Conference on PrinciplesofKnowledgeRepresentationandReasoning,vol- ume 17, 592–602. Kulmanov, M.; Liu-Wei, W.; Yan, Y.; and Hoehndorf, R
-
[2021]
Semantic similarity and machine learning with on- tologies.Briefings in bioinformatics, 22(4): bbaa199. Lo,A.;Jiang,A.Q.;Li,W.;andJamnik,M.2024.End-to-end ontology learning with large language models.Advances in Neural Information Processing Systems, 37: 87184–87225. Peñaloza, R. 2020. Axiom Pinpointing.Applications and Practices in Ontology Design, Extra...
arXiv 2024
-
[2022]
InInternational Joint Conference on Automated Reasoning, 188–207
Connection-minimal abduction in EL via translation to FOL. InInternational Joint Conference on Automated Reasoning, 188–207. Springer. Halland, K.; and Britz, K. 2012. ABox abduction in ALC using a DL tableau. InProceedings of the South African in- stitute for computer scientists and information technologists conference, 51–58. He,Y.;Chen,J.;Jiménez-Ruiz,...
2012
-
[2023]
In Rogers, A.; Boyd-Graber, J
Language ModelAnalysis for OntologySubsumption Inference. In Rogers, A.; Boyd-Graber, J. L.; and Okazaki, N., eds.,Findings of the Association for Computational Lin- guistics:ACL2023,Toronto,Canada,July9-14,2023,3439–
2023
-
[3453]
He,Y.;Yuan,M.;Chen,J.;andHorrocks,I.2024
Association for Computational Linguistics. He,Y.;Yuan,M.;Chen,J.;andHorrocks,I.2024. Language Models as Hierarchy Encoders. In Globersons, A.; Mackey, L.; Belgrave, D.; Fan, A.; Paquet, U.; Tomczak, J. M.; and Zhang, C., eds.,Advances in Neural Information Process- ing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 20...
2024
This paper was first reviewed by deepseek-v4-flash on August 3, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.