{"id":"a59f5828-445b-41c2-972a-fa9bcbe351a1","arxiv_id":"2507.14334","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"OnT combines pretrained language models with hyperbolic embeddings and role rotations to encode EL ontologies, and reports better axiom prediction and inference than existing methods.","lead":"This paper introduces OnT, a method that teaches a language model to embed medical ontologies as hyperbolic geometry while keeping text and logical structure together. If it holds up, it could make tools for maintaining large medical vocabularies faster and less manual.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proposition 1 does not actually connect the role-transformed score to the verbalized existential embedding, so the logical-preservation claim rests on an unproven alignment.","rationale":"I read the strongest claim as two-part: (i) OnT preserves EL logical structure, and (ii) it achieves SOTA prediction and inference. The empirical part is plausible; the reported numbers are consistently better, though the duplicated Table 3 row and missing error bars noted by the reader should be fixed. The theoretical part is where the argument is least secure. Proposition 1 is the only formal support for 'preservation', and it has a scope gap: it compares fr(x_C) with fr(x_D), not the verbalized existential embeddings that the score function uses. Since the role-transformed vectors are only a training-time auxiliary representation, the proof never touches the actual inference-time geometry of existential concepts. The reader's verbalization concern is related but not identical; even with good verbalization, this alignment gap remains. A targeted monotonicity probe would settle whether the claimed logical preservation actually holds in the trained model. The conditional verdict stands, with the additional condition that the authors either tighten the proposition or provide this empirical evidence.","tokens_in":15194,"tokens_out":8679,"duration_ms":88500,"concrete_test":"On the trained GALEN and GO models, build a monotonicity probe: sample 1,000 true atomic subsumptions A⊑B from held-out axioms, choose roles r occurring in NF3/NF4 axioms, and rank ∃r.A⊑∃r.B against 50 negative existential axioms ∃r.A⊑∃r.E. Report Hits@10 and MRR, and compare with OnT(w/o r). Also record the distribution of learned kr and the average alignment distance dκ(x_{∃r.D}, fr(x_D)) over the validation set. If the probe is near chance, if alignment distances are large, or if kr values deviate substantially from 1, then Proposition 1 does not transfer to the actual embeddings and the logical-preservation claim should be weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim that OnT 'preserves logical relationships of EL' and thereby augments axiom inference is supported by Proposition 1 (Section 4.3). That proposition, however, only proves a rotation-invariance statement: with kr=1, s(fr(xC) ≺ fr(xD)) = s(xC ≺ xD). It never relates fr(x_D) to the verbalization-based embedding x_{∃r.D}, which is what the scoring function Eq. 8 actually uses for existential concepts. The alignment loss Lr (Eq. 6) is a pair of opposite hierarchy losses with margins α, β and random negatives; it can be zero without exact equality, so x_{∃r.D} and fr(x_D) may be far apart. Moreover, kr is a learned parameter and is not constrained to 1, so even the proved equality does not hold in practice. Consequently, the monotonicity pattern A⊑B ⇒ ∃r.A⊑∃r.B is not guaranteed by the model, and the 'preservation' claim outruns the proof. The strong inference results on NF1 axioms could be explained by the centripetal norm-ordering term in Eq. 8 alone, independent of the role and conjunction losses. This is an internal gap, not merely an untested verbalization assumption.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces OnT, an approach to embedding EL ontologies that combines pretrained-language-model text representations with hyperbolic geometric constraints. Atomic concepts and roles are verbalized and encoded by a sentence transformer; complex concepts are embedded via compositional verbalization; roles are modeled as rotation-scaling transformations; and training uses a hierarchy loss together with role-alignment and conjunction losses. The method is evaluated on axiom prediction (held-out split) and axiom inference (entailed NF1 axioms) over GALEN, GO, and Uberon, reporting state-of-the-art results against geometric and text-based baselines, plus transfer-learning and SNOMED case studies.","tokens_in":15469,"tokens_out":7087,"duration_ms":76172,"significance":"If the empirical results hold, OnT is a useful engineering contribution: it combines textual semantics from PLMs with hyperbolic geometry for ontologies, and the released code and data should facilitate replication. The held-out prediction task is a clean evaluation of inductive ontology completion, and the reported gains, especially in mean rank, are substantial. However, the paper's central claim that OnT 'preserves logical relationships of EL' is not fully supported: Proposition 1 is an invariance statement about the role transformation, not a proof about the verbalized existential embeddings actually used in scoring, and the inference benchmark may be largely solvable from the norm-ordering term in the loss. The empirical comparisons would also be more convincing with error bars and validation of the re-implemented baselines.","major_comments":[{"comment":"Proposition 1 only shows that the score s is invariant under the role transformation fr when kr=1. It does not relate fr(x_D) to the verbalization embedding x_{∃r.D}, which is the representation actually scored by Eq. (8) for existential concepts. The alignment loss in Eq. (6) is a pair of hierarchical losses with margins; it can be satisfied without exact equality between fr(x_D) and x_{∃r.D}, and kr is a learned parameter not constrained to 1. Consequently, the claimed preservation of the pattern A⊑B ⇒ ∃r.A⊑∃r.B is not guaranteed by the proof. Please either prove a bound under the actual training objective, report empirical distances between fr(x_D) and x_{∃r.D}, or reframe Proposition 1 as a design motivation rather than a preservation guarantee.","section":"Section 4.3, Proposition 1, Eq. (4) and Eq. (6)"},{"comment":"The centripetal term in the hierarchy loss explicitly enforces ‖D‖κ < ‖C‖κ for every asserted subsumption C⊑D, and the scoring function in Eq. (8) contains the same norm-difference term. Along a chain C⊑D⊑E, transitivity follows from the norm ordering alone, so the strong inference results on entailed NF1 axioms may be largely driven by this term rather than by the role-alignment or conjunction losses. Table 3 is consistent with this concern: OnT(w/o r) is close to OnT on several inference metrics (e.g., GO H@100 39 vs 40, ANATOMY MR 499 vs 458). The authors should isolate the contribution of the role and conjunction losses to transitive inference, for example by ablating the norm-difference term in Eq. (8) or by measuring whether Eq. (6) actually brings fr(x_D) close to x_{∃r.D}.","section":"Section 4.3, Eq. (5) and Eq. (8); Section 5.3"},{"comment":"The experiments report no variance: each result appears to come from a single 80/10/10 split and a single run, with no error bars or significance tests. Given the large gains claimed in Table 2 (e.g., MR 1,121 vs 7,092 on GO), the absence of repeated-seed statistics makes it impossible to judge whether the state-of-the-art claim is robust. Please report means and standard deviations over multiple random splits and seeds, and ideally paired significance tests for the main comparisons.","section":"Section 5.1"},{"comment":"The geometric baselines (BoxEL, TransBox, ELBE, ELEM) were re-implemented rather than run from original code, and no validation against published numbers is provided. While the paper explains that dataset modifications require this, the re-implementations should be validated (e.g., by reproducing a previously reported score on a standard setting) and the re-implementation code should be clearly documented; otherwise the baselines may be unintentionally weaker than the original methods.","section":"Section 5.1"},{"comment":"All complex-concept embeddings are derived from the compositional verbalization templates V(C⊓D) = 'V(C) and V(D)' and V(∃r.C) = 'something that V(r) some V(C)', but the paper does not evaluate verbalization quality or sensitivity to alternative templates. The conclusion explicitly lists 'exploring the impact of verbalization quality' as future work, yet this is a load-bearing component of the method: any misrepresentation of the EL semantics in these strings propagates to all downstream scores. Please provide at least a qualitative evaluation of verbalization correctness or an ablation over template variants.","section":"Section 4.1 and Section 6"}],"minor_comments":[{"comment":"In the ANATOMY column, the Box2EL row (0/1/7, 1, 11,801) is identical to the GO row, which is presumably a copy-paste error; please correct and re-verify all entries.","section":"Table 3"},{"comment":"The sentence 'incorporating role embeddings and losses for logical constraints allows OnT(w/o r) to achieve even better performance than both OnT and HiT' is contradictory; it should say 'omitting role embeddings' or similar.","section":"Section 5.3"},{"comment":"There are several typos, including 'on the other hand, in in Figure 2b' and 'where Dneg represents a randomly sampled concept' without specifying whether negatives are sampled from the ontology vocabulary or from all candidates; please clarify.","section":"Section 4.3"},{"comment":"The inference task evaluates only entailed NF1 axioms; the paper should justify this restriction and state whether the same conclusions would be expected for NF2–NF4 inferences, especially since those axiom types are present in the training data.","section":"Section 5.3"}],"recommendation":"major_revision","confidential_remarks":"The empirical contribution is potentially publishable, but the framing of Proposition 1 as a logical-preservation guarantee is not defensible and the inference benchmark may be solvable from the norm-ordering term alone. A careful rewrite, plus experiments that isolate the role and conjunction losses and add repeated-seed statistics, would make the paper suitable. The copy-paste errors in Table 3 and the contradictory ablation sentence should also be fixed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: OnT is a genuine empirical step forward for embedding EL ontologies with text, but the paper overclaims the logical-preservation guarantee. Proposition 1 only shows rotation-invariance of the score when kr=1; it never bridges from fr(x_D) to the verbalized embedding x_{∃r.D} that the score actually uses. So the monotonicity A⊑B ⇒ ∃r.A⊑∃r.B is not actually proved. That is not a small technicality: it is the main advertised mechanism. The alignment loss L_r encourages fr(x_D) and x_{∃r.D} to be close, but it can plateau without exact equality, and kr is trained, not fixed to 1. Also, the centripetal loss alone enforces parent-child norm ordering, which plausibly explains a large chunk of the inference results on NF1 axioms. The paper never isolates how much the role and conjunction losses add beyond the centripetal term for inference; the ablation OnT vs OnT(w/o r) is on prediction, not on inference, if I read Table 4 correctly. That should be addressed.\n\nWhat is genuinely good: combining a sentence-transformer verbalization with Poincaré embeddings is a natural extension of HiT and it works—the empirical tables are consistently better than the geometric and language-model baselines, and they include transfer learning and a SNOMED case study. They ship code and data, and they re-implemented all the geometric baselines on the same normalized corpora, which is more than many papers do. I believe the empirical SOTA claim, modulo the lack of error bars and a few copy-paste errors in Table 3 (Box2EL's ANATOMY row is a duplicate of GO).\n\nSoft spots beyond the proof gap: no multiple seeds or confidence intervals; the normalization pipeline is their own, which is reasonable but makes cross-paper comparison harder; verbalization quality is untested, and they admit this in the conclusion. These are fixable.\n\nBottom line: the paper deserves a serious referee round, but the authors should be pushed to narrow the claim to 'empirically effective' and either prove a real alignment guarantee or present the role and conjunction losses as heuristics. As an empirical tool for biomedical ontology completion, OnT is likely useful.\n\nI'd bring it to a reading group, cite it if I work on ontology embeddings, and send it to review with a request for major revision.","headline":"OnT is an empirically strong ontology embedding method, but the logical-preservation claim outruns the proof.","tokens_in":15996,"tokens_out":2986,"would_cite":true,"duration_ms":517251,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"OnT embeds EL ontologies by tuning a pretrained language model in hyperbolic space, preserving logical structure and beating prior methods on axiom prediction and inference.","keywords":["ontology embedding","description logic EL","hyperbolic space","pretrained language model","axiom prediction","axiom inference","Poincaré ball","role embedding"],"falsifier":"Replace the compositional verbalizations with shuffled word sequences (same vocabulary, no grammatical relation) and rerun prediction and inference on GALEN, GO, and Uberon; if the metrics do not drop substantially, the specific semantics of the verbalization are not load-bearing for the reported gains.","tokens_in":14995,"feed_emoji":"🧠","tokens_out":6138,"duration_ms":65545,"temperature":0.7,"pith_summary":"This paper proposes OnT, an ontology embedding method that tunes a pretrained language model inside a hyperbolic (Poincaré ball) space so that textual labels and Description Logic EL structure are encoded together. The authors argue that prior geometric methods ignore text, while language-model methods fail to preserve logical relationships such as subsumption transitivity and existential-restriction monotonicity. OnT verbalizes complex concepts, represents roles as rotation-and-scaling functions, and trains with hierarchy, role, and conjunction losses. On GALEN, Gene Ontology, and Uberon, it reports state-of-the-art results on both axiom prediction and axiom inference, along with transfer-learning success and a real SNOMED CT case. The reason to care is that ontology embeddings that keep logic in the vector space could support approximate reasoning and ontology completion in biomedicine and other structured domains.","feed_headline":"Hyperbolic language models beat prior ontology embeddings","feed_subtitle":"Tuning a language model in hyperbolic space keeps ontology logic while using text, boosting axiom prediction and inference.","key_machinery":"The load-bearing mechanism is a set of geometric-training losses in the Poincaré ball model of hyperbolic space: a hierarchy loss $\\mathcal{L}_{\\prec}$ that treats axioms $C \\sqsubseteq D$ as a partial order (contrastive plus centripetal terms), a role loss $\\mathcal{L}_r$ that identifies $x_{\\exists r.D}$ with $f_r(x_D)$ through two opposite hierarchy losses, and a conjunction loss $\\mathcal{L}_{\\sqcap}$ that enforces $C \\sqcap D \\sqsubseteq C$ and $C \\sqcap D \\sqsubseteq D$. The role transformer $f_r(v) = k_r \\odot (R(\\Theta_r) \\cdot v)$ is a rotation-scaling operation, and the axiom score $s(C \\sqsubseteq D) = -(d_\\kappa(x_C, x_D) + \\lambda(\\|x_D\\|_\\kappa - \\|x_C\\|_\\kappa))$ combines hyperbolic distance with a norm-based penalty. These pieces together are what let the PLM embeddings carry both textual and formal-logical semantics.","core_discovery":"The central claim is that a pretrained language model can act as an ontology encoder for EL when its outputs are placed in a Poincaré ball and trained with losses that mirror the logic. Complex concepts are first verbalized compositionally ($C \\sqcap D$ becomes \"$C$ and $D$\"; $\\exists r.C$ becomes \"something that $V(r)$ some $V(C)$\"), then embedded by the PLM; subsumption axioms $C \\sqsubseteq D$ are treated as a partial order on hyperbolic points and learned with a hierarchical contrastive plus centripetal loss. Roles are embedded as hyperbolic transformations $f_r(v) = k_r \\odot (R(\\Theta_r) \\cdot v)$, and two extra losses align $f_r(x_D)$ with $x_{\\exists r.D}$ and enforce $C \\sqcap D \\sqsubseteq C$, $C \\sqcap D \\sqsubseteq D$. Proposition 1 shows that when $k_r = 1$, rotations leave hyperbolic distances and norms invariant, which makes the score for $\\exists r.C \\sqsubseteq \\exists r.D$ equal to that for $C \\sqsubseteq D$ and thereby captures the deductive pattern $A \\sqsubseteq B \\Rightarrow \\exists r.A \\sqsubseteq \\exists r.B$. The paper reports consistent state-of-the-art performance in axiom prediction and inference on GALEN, GO, and Uberon, with up to a sevenfold improvement in Mean Rank over prior methods on GO prediction.","pith_inferences":["The paper does not measure verbalization quality; if the English template were replaced by a semantically empty one, such as random word shuffles, with little change in results, that would suggest the PLM is mainly exploiting lexical overlap rather than compositional meaning.","The choice of hyperbolic space may matter more than the PLM size: the ablation shows large gains over Euclidean-style geometric baselines but modest differences among three sentence-transformer models.","The rotation-and-scaling role representation is untested on role inclusion axioms ($r \\sqsubseteq s$) or role chains; extending the loss design to those patterns is a natural next step.","The reported GO prediction gains (about a sevenfold Mean Rank improvement) may depend on the evaluation protocol of ranking over all atomic concepts; direct comparison with other papers should be done carefully."],"forward_implications":["Plausible new axioms, including inferred ones, can be ranked directly in vector space, supporting ontology completion and approximate reasoning without a separate classifier.","Unseen entities can be embedded from their textual descriptions alone, which is what enables the reported transfer-learning behaviour across GALEN, GO, and Uberon.","The rotation-based role model captures monotonicity of existential restrictions (if $A \\sqsubseteq B$ then $\\exists r.A \\sqsubseteq \\exists r.B$), a deductive pattern earlier LM-based embeddings lacked.","The method can flag missing and erroneous direct subsumptions during real ontology construction, as demonstrated on SNOMED CT.","Because OnT(w/o r) also outperforms most baselines, even the core hierarchy-plus-verbalization combination is a competitive option when role information is unavailable."],"supporting_citations":[{"why":"Supplies the hyperbolic hierarchy-training approach that OnT extends with role and conjunction losses.","marker":"[17]"},{"why":"Provides the Poincaré embedding model and hierarchical loss that underpin the hyperbolic space.","marker":"[26]"},{"why":"Baseline geometric-box method and the framework OnT re-implements for comparisons; also a reference for ontology normalization.","marker":"[18]"},{"why":"Geometric ball-based baseline and a normalization reference for EL++ ontologies.","marker":"[19]"},{"why":"Language-model-based baseline using Word2Vec and random forest classification for ontology embeddings.","marker":"[8]"},{"why":"Biomedical word-embedding baseline that combines formal ontology content with informal text.","marker":"[29]"},{"why":"Gene Ontology, one of the three benchmark ontologies used in evaluation.","marker":"[2]"},{"why":"GALEN, a benchmark biomedical ontology used in evaluation.","marker":"[28]"},{"why":"Uberon, the anatomy benchmark ontology used in evaluation.","marker":"[25]"},{"why":"Provides the EL normalization procedure that transforms arbitrary axioms into the four normalized forms NF1–NF4.","marker":"[3]"}],"fun_headline_variants":["Hyperbolic language models keep ontology logic and beat baselines","PLMs tuned in hyperbolic space improve axiom prediction","Poincaré ball tuning strengthens language model ontology encoding","Language models in hyperbolic space reason over ontologies better","OnT: Hyperbolic PLM tuning for ontology reasoning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the automatically generated English sentences for complex concepts—such as \"something that is parent of some person\"—preserve enough of the EL semantics that a pretrained language model can embed them faithfully; the paper does not test how sensitive results are to verbalization quality.","fun_headline_variants_meta":{"raw":{"variants":["Hyperbolic language models keep ontology logic and beat baselines","PLMs tuned in hyperbolic space improve axiom prediction","Poincaré ball tuning strengthens language model ontology encoding","Language models in hyperbolic space reason over ontologies better","OnT: Hyperbolic PLM tuning for ontology reasoning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00054,"raw_usage":{"total_tokens":2640,"prompt_tokens":1049,"completion_tokens":1591,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":665,"completion_tokens_details":{"reasoning_tokens":1514}},"tokens_in":665,"tokens_out":1591,"duration_ms":18640,"temperature":1.0,"reasoning_tokens":1514,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:59:17.414020+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the compositional verbalizations with shuffled word sequences (same vocabulary, no grammatical relation) and rerun prediction and inference on GALEN, GO, and Uberon; if the metrics do not drop substantially, the specific semantics of the verbalization are not load-bearing for the reported gains.","supporting_citations":[{"cited_title":"In: Globersons, A., Mackey, L., Belgrave, D., Fan, A., Pa- quet, U., Tomczak, J.M., Zhang, C","cited_arxiv_id":null,"evidence_quote":"Supplies the hyperbolic hierarchy-training approach that OnT extends with role and conjunction losses."},{"cited_title":"In: Guyon, I., von Luxburg, U., Bengio, S., Wallach, H.M., Fergus, R., Vishwanathan, S.V.N., Garnett, R","cited_arxiv_id":null,"evidence_quote":"Provides the Poincaré embedding model and hierarchical loss that underpin the hyperbolic space."},{"cited_title":"In: Chua, T., Ngo, C., Kumar, R., Lauw, H.W., Lee, R.K","cited_arxiv_id":null,"evidence_quote":"Baseline geometric-box method and the framework OnT re-implements for comparisons; also a reference for ontology normalization."},{"cited_title":"Nature genetics25(1), 25–29 (2000)","cited_arxiv_id":null,"evidence_quote":"Gene Ontology, one of the three benchmark ontologies used in evaluation."},{"cited_title":"In: Medical Informatics Europe’96, pp","cited_arxiv_id":null,"evidence_quote":"GALEN, a benchmark biomedical ontology used in evaluation."},{"cited_title":"Genome biology13, 1–20 (2012)","cited_arxiv_id":null,"evidence_quote":"Uberon, the anatomy benchmark ontology used in evaluation."},{"cited_title":"In: Kaelbling, L.P., Saffiotti, A","cited_arxiv_id":null,"evidence_quote":"Provides the EL normalization procedure that transforms arbitrary axioms into the four normalized forms NF1–NF4."}],"review_version":1}