Pith. sign in

REVIEW 4 major objections 5 minor 38 references

HyDRA: A Hybrid-Driven Reasoning Architecture for Verifiable Knowledge Graphs

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read HyDRA aims to make automated knowledge-graph construction verifiable by chaining design-by-contract specifications from competency questions to final triplets.

desk verdict Plausible architecture, honest negative result, but the central 'global structural integrity' claim is not backed by the implementation — worth reviewing, needs major revision. read the letter →

arxiv 2507.15917 v2 pith:NXF5WYJF submitted 2025-07-21 cs.LG

Adrian Kaiser , Claudiu Leoveanu-Condrei , Ryan Gold , Marius-Constantin Dinu , Markus Hofmarcher This is my paper
classification cs.LG
keywords knowledgegraphconstructiondesignbycontractneurosymbolicAIlargelanguagemodelscompetencyquestionsontologygenerationverificationMedExQA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

HyDRA is a proposed architecture that automates knowledge-graph construction by treating every stage—competency questions, ontology schema, and instance triplets—as a stage governed by machine-checkable contracts. If a generated artifact violates a contract's postconditions, such as graph disconnectedness or class-instance conflation, the system re-prompts the LLM until the artifact passes. The paper's experiments on a medical question-answering benchmark found that an ontology-free baseline outperformed HyDRA on simple questions; the authors argue the benchmark rewards single-hop retrieval rather than the multi-hop reasoning where ontology-guided structure should help. The contribution is a blueprint for self-correcting, auditable KG pipelines and a proposal for evaluating graphs by translating questions into graph-database queries.

What carries the argument

The central mechanism is the verifiable contract, defined as a quadruple of preconditions, postconditions, invariants, and a verification function. Contracts are chained: the postcondition of scope and competency-question generation becomes the precondition of ontology generation, and the ontology contract becomes the precondition of KG population. The verification function is realized through neurosymbolic operations that treat LLMs as semantic parsers with validation and retry layers, using constraints such as a strict single-root class hierarchy, no circular inheritance, and no class-instance conflation.

What would settle it

Take a multi-hop QA benchmark with a gold-standard ontology for the domain, construct both a HyDRA graph and an unconstrained graph from the same documents, and translate the benchmark questions into graph-database queries; if the unconstrained graph answers the multi-hop queries at least as accurately as the contract-compliant HyDRA graph, the claim that contract-driven structural integrity is required for functional correctness is refuted.

Watch

Extended reading notes

Core claim

The paper claims that applying design-by-contract to LLM-driven KG construction yields graphs that are structurally sound and verifiably compliant: competency questions serve as the root scope contract, the ontology built from them as a schema contract, and the extracted triplets as a population contract, with each phase's postcondition becoming the next phase's precondition. Violations detected by a symbolic verification layer trigger corrective re-prompting until compliance, giving end-to-end traceability from requirements to data. The paper reports that on the MedExQA biomedical engineering benchmark, the unconstrained baseline answered 95 to 98 percent of questions versus HyDRA's 42 to 62 percent, and attributes this gap to the benchmark's single-hop simplicity rather than to a failure of the ontology-guided method.

Load-bearing premise

HyDRA assumes that the LLM-generated personas, scope documents, competency questions, and ontology correctly and completely capture the target domain, so that passing the contracts means the graph is correct rather than merely self-consistent.

Editorial extensions

If this is right

  • If HyDRA's core claim holds, automated KG construction can become self-correcting: every artifact carries a contract lineage back to a competency question, so failures are localized and repaired without manual curation.
  • Structural invariants such as absence of isolated components, a single class hierarchy, and prohibition of class-instance conflation become enforceable properties rather than hopes, making generated graphs directly loadable into graph databases like Neo4j.
  • The reported performance drop on single-hop QA is a predicted trade-off: the ontology restricts exactness to gain consistency, so benchmarks must measure multi-hop reasoning to reflect the method's intended advantage.
  • Replacing LLM-based evaluation with executable graph queries removes the confound of the evaluator's parametric knowledge, giving a direct measure of graph quality.
  • Deciding when re-prompting is wasteful, the paper identifies deterministic repairs, such as swapping reversed subject-object triplets, that can short-circuit the loop and reduce cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own logic implies that contract verification is a consistency check, not a truth check: if the generated competency questions misstate the domain, the repair loop can converge on a graph that is internally coherent but factually wrong, so external gold standards would be needed to certify correctness.
  • The proposed Neo4j-based functional evaluation could be turned into a reusable benchmark by publishing a corpus of natural-language questions paired with hand-verified Cypher queries and required traversal lengths.
  • Because HyDRA allows class narrowing but not reassignment, the architecture encodes a monotonicity assumption about information arriving across documents; testing that assumption on corpora with contradictory or evolving entities would be a natural next step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. HyDRA is a proposed architecture for automated ontology and knowledge-graph construction that combines LLM-driven generation with Design-by-Contract-style validation. The pipeline starts from a user domain description, generates stakeholder groups and personas, produces scope documents and competency questions, builds an OWL-like ontology, and then extracts instance triplets from input documents. The paper's central claim is that a contract-driven verification loop, with postconditions such as absence of isolated components and prevention of class-instance conflation, enforces global structural integrity and makes the resulting KG 'verifiably compliant.' The experimental section evaluates HyDRA on the MedExQA Biomedical Engineering subset, where the ontology-guided KG achieves 42--62% accuracy while an ontology-free baseline reaches 95--98%. The authors attribute this to benchmark simplicity and propose, but do not complete, a graph-database-based functional evaluation using Neo4j and Cypher queries. The code is publicly available.

Significance. If HyDRA's central claims were fully demonstrated, the work would be a useful step toward self-correcting, auditable KG construction: the contract lineage from competency questions to ontology to instance data is a genuinely appealing idea, and the shift from LLM-context evaluation to direct graph-database queries addresses a real evaluation gap. The paper is also transparent about its limitations and ships a public codebase, which is commendable. However, the central differentiator named in Section 2.1---global structural integrity enforced by postcondition contracts---is not actually implemented in the provided algorithms, the only completed experiment shows HyDRA underperforming a baseline, and the evaluation is complicated by the fact that the domain description and competency questions were derived from the same benchmark used for testing. The significance is therefore conditional on substantial additional implementation and evaluation work.

major comments (4)
  1. [§2.1, §4.5.1, Algorithm 3] The claim that HyDRA enforces global structural integrity via postcondition contracts such as the absence of isolated components is not supported by the implementation. In Algorithm 3, FindViolations is applied only to the newly generated triplets Tj, and the checks listed in Section 4.5.1 are local: duplicate entities, naming conventions, property validity, class-assignment validity, and class narrowing. No check of global graph connectivity, disconnected components, or class-instance conflation appears in any algorithm or validation description. The single-root and single-inheritance constraints in Section 4.4.1 apply to the ontology TBox, not to the ABox instance graph. As written, the repair loop can terminate on a graph that satisfies every local check but still contains the isolated islands the paper motivates against, so the primary differentiation of HyDRA is not demonstrated.
  2. [§5.2, §5.3, §6.4] The evaluation is circular. The domain description used to generate the ontology was 'derived from the Biomedical Engineering subset of MedExQA' (Section 5.2), and the same benchmark's question-answer pairs are used to evaluate the resulting KG (Section 5.3). Section 6.4 explicitly concedes that the ontology can fully represent the relevant information 'only if ... the competency questions precisely match the evaluation questions, which introduces significant evaluation bias.' Therefore the reported 42--62% accuracy values in Table 1 are not an independent measure of KG quality, and the paper's interpretation that the performance gap is due to benchmark simplicity is not established. An evaluation on a held-out domain, or at least with competency questions generated independently of the test questions, is needed.
  3. [§6.1, Table 1; §6.3] The central empirical support for HyDRA's claimed benefit is missing. The only completed experiment shows HyDRA underperforming the baseline (42--62% vs. 95--98% accuracy in Table 1). The rebuttal that the benchmark only tests simple single-hop retrieval is supported only by three illustrative Cypher queries in Section 6.3, with no aggregate results, and the proposed graph-database evaluation is described as 'still a work in progress.' Thus the paper does not demonstrate that HyDRA produces KGs that are verifiably compliant and functionally useful for multi-hop reasoning. The authors should either run the planned evaluation and report aggregate numbers, or explicitly restrict the paper's claims to the architecture and its qualitative properties.
  4. [§4.3, §4.4.1, §6.3] The architecture assumes that LLM-generated artifacts---personas, scope documents, competency questions, and the ontology---constitute a correct and complete domain specification, so that the deterministic validation constraints encode genuine correctness rather than self-consistency. As the paper notes in Section 6.3, the ontology-based method can fully represent relevant information only if the ontology is 'infinitely detailed' or the competency questions precisely match the evaluation questions. Without an external yardstick for specification correctness, the repair loop can converge to a graph that is internally consistent but factually wrong. This is a load-bearing assumption for the 'verifiable' claim and should be stated as an explicit limitation or addressed with external gold ontologies.
minor comments (5)
  1. [§4.2, Algorithm 1] The text says scope documents are merged 'in batches of size n (with k = 6 as the default hyperparameter),' while Algorithm 1 line 8 uses batch size k; moreover n already denotes the number of persona groups. Use distinct names for the number of groups and the batch size, and make the text consistent with the pseudocode.
  2. [§4.3, §4.4, §3.2] There are several typos and grammatical errors: 'Competency questions are natural language queries are used' in Section 4.3; 'fomally' in Section 4.4; 'anologous' in Section 3.2. These should be corrected.
  3. [§5.1, §6.2] The references to 'Table 6.1' should be 'Table 1,' and the caption of Table 1 should clarify that Gemini-2.5-Flash, o4-mini, and o3 are the LLMs used for KG construction and evaluation.
  4. [§5.3] The evaluation contract uses a fuzzy matching algorithm to determine answer correctness, but the matching threshold and the exact implementation of the 'best guess' contract are not specified in the paper. Reporting these details would improve reproducibility, even if the code is public.
  5. [§6.3] The term 'cypher' should be capitalized as 'Cypher' consistently, both in prose and in the figure captions.

Circularity Check

2 steps flagged · score 6.0 of 10

Evaluation is self-referential: the CQs that define the ontology are also translated into the queries that measure 'functional correctness', and the domain description is derived from the same MedExQA benchmark used for evaluation.

  1. self definitional [Section 2.4 (Ontology and Knowledge-Graph Evaluation); cf. Section 4.3 and Algorithms 2-3]
    "HyDRA's evaluation framework represents the logical conclusion of this trend, creating a perfectly symmetrical and closed-loop process. The very CQs that serve as the initial requirement contract are programmatically translated into executable queries for the final artifact. Specifically, we query the generated graph via verifiable operators using neurosymbolic design principles. The success rate of these queries provides a direct measure of the KG's functional correctness."

    The CQs are the specification: Section 4.3 says they 'define the requirements and intended capabilities of an ontology', Algorithm 2 constructs the ontology from them, and Algorithm 3 uses that ontology to constrain KG extraction. The evaluation framework then translates 'the very CQs that serve as the initial requirement contract' into queries and calls the success rate a measure of 'functional correctness'. The test set is therefore identical to the requirements that generated the artifact. A graph that satisfies its own self-imposed CQs is declared correct by construction; the measurement reduces to checking the system against its own definition, with no external ground truth or independent benchmark involved.

  2. other [Sections 5.1-5.2 and 6.3; concession in Section 7 (Lessons learned: Evaluation)]
    "For the benchmark, we created a concise domain-specific description derived from the Biomedical Engineering subset of MedExQA to generate an ontology. This ontology, combined with the structured question-answer data, was then used to construct a KG. ... Then, the questions of the benchmark dataset are automatically translated into graph database queries."

    The domain description that drives CQ generation and ontology construction is derived from the same MedExQA benchmark whose question-answer pairs are later used for evaluation. The pipeline can therefore tailor its specification to the exact benchmark before the benchmark is used as the yardstick. The paper itself concedes in Section 7: 'if the competency questions precisely match the evaluation questions, which introduces significant evaluation bias.' This is not a fitted parameter renamed as a prediction, but it is circular evaluation: the input specification is a function of the test set, so the reported success rate is at least partly a self-comparison against a benchmark-derived specification.

full rationale

The main circularity is in the verification/evaluation loop. HyDRA builds the ontology and KG from LLM-generated competency questions that 'define the requirements', and then evaluates the KG by translating those very CQs into queries; this makes the success measure a self-consistency check, not a test against an independent oracle. The circularity is compounded by deriving the domain description from MedExQA and then evaluating on MedExQA, with the paper itself admitting the resulting bias in Section 7. I do not treat the SymbolicAI self-citation as load-bearing: it is a published framework used as an implementation substrate, and the paper's central claims do not reduce to a SymbolicAI theorem. The skeptic's point about 'global structural integrity' is a correctness/completeness gap rather than a circularity step: Algorithm 3 validates each new chunk's triplets locally and no global connectivity contract is shown, but that is a missing implementation, not an input-output equivalence. Overall, the evaluation methodology is partially circular, but the paper also contains independent deterministic constraints and an honest negative empirical result, so the derivation is not entirely forced. Score 6 reflects one or more evaluation claims that reduce by construction to self-generated specifications, without full self-citation-chain circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on no fitted physical constants. The free parameters are default hyperparameters (l=4, k=6) and unspecified epoch and fuzzy-threshold choices. The main assumptions are domain assumptions about LLM reliability and the sufficiency of hand-coded constraints. HyDRA introduces no new physical or formal entities; contracts, personas, and CQs are organizational constructs, not entities with independent falsifiable handles.

free parameters (4)
  • persona group size l = 4
    Default hyperparameter in Section 4.1; splits personas into heterogeneous groups, influencing scope document diversity.
  • scope merge batch size k = 6
    Default hyperparameter in Section 4.2 and Algorithm 1; controls how many scope documents are merged per step.
  • KG generation epochs = not specified
    Algorithm 3 loops over epochs, but no default or range is given; affects graph density and runtime.
  • fuzzy matching threshold = not specified
    Evaluation uses fuzzy matching to detect correct answers, but the threshold is not reported in Section 5.3.
assumptions (4)
  • domain assumption LLM-generated competency questions correctly scope the ontology to the domain.
    Section 4.3 generates CQs from scope documents and uses them as preconditions for ontology generation. If CQs miss key requirements, the ontology cannot represent them.
  • domain assumption The hand-coded validation constraints (unique names, single root, no circular inheritance, etc.) are sufficient to guarantee structural integrity and semantic quality.
    Section 4.4.1 lists simple and complex constraints, but no theorem or empirical study shows these constraints prevent the harmful failure modes, such as isolated islands or class-instance conflation, that motivate the paper.
  • domain assumption Re-prompting the LLM with violation messages will yield corrected, compliant output.
    Algorithms 2 and 3 loop on violations until none remain. Section 7 reports that models sometimes drop violating triplets instead of fixing them, so repair success is not guaranteed.
  • domain assumption The evaluation benchmark's single-hop questions are representative of KG utility, or the proposed multi-hop graph-database evaluation can be automated reliably.
    Section 5.3 and 6.3 admit the QA benchmark is biased and that Neo4j query translation is work in progress. The paper's argument for ontology value therefore rests on an evaluation that is not yet delivered.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HyDRA: A Hybrid-Driven Reasoning Architecture for Verifiable Knowledge Graphs." pith.science (2026). https://pith.science/paper/NXF5WYJF

@misc{pith2026250715917,
  author       = {Pith},
  title        = {Pith review of: HyDRA: A Hybrid-Driven Reasoning Architecture for Verifiable Knowledge Graphs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NXF5WYJF}},
  note         = {Machine review of arXiv:2507.15917}
}
abstract

The synergy between symbolic knowledge, often represented by Knowledge Graphs (KGs), and the generative capabilities of neural networks is central to advancing neurosymbolic AI. A primary bottleneck in realizing this potential is the difficulty of automating KG construction, which faces challenges related to output reliability, consistency, and verifiability. These issues can manifest as structural inconsistencies within the generated graphs, such as the formation of disconnected $\textit{isolated islands}$ of data or the inaccurate conflation of abstract classes with specific instances. To address these challenges, we propose HyDRA, a $\textbf{Hy}$brid-$\textbf{D}$riven $\textbf{R}$easoning $\textbf{A}$rchitecture designed for verifiable KG automation. Given a domain or an initial set of documents, HyDRA first constructs an ontology via a panel of collaborative neurosymbolic agents. These agents collaboratively agree on a set of competency questions (CQs) that define the scope and requirements the ontology must be able to answer. Given these CQs, we build an ontology graph that subsequently guides the automated extraction of triplets for KG generation from arbitrary documents. Inspired by design-by-contracts (DbC) principles, our method leverages verifiable contracts as the primary control mechanism to steer the generative process of Large Language Models (LLMs). To verify the output of our approach, we extend beyond standard benchmarks and propose an evaluation framework that assesses the functional correctness of the resulting KG by leveraging symbolic verifications as described by the neurosymbolic AI framework, $\textit{SymbolicAI}$. This work contributes a hybrid-driven architecture for improving the reliability of automated KG construction and the exploration of evaluation methods for measuring the functional integrity of its output. The code is publicly available.

Figures

Figures reproduced from arXiv: 2507.15917 by the authors.

Figure 1
Figure 1. Overview of the HyDRA architecture. The ontology generation process begins with a user-defined domain description. HyDRA first generates stakeholder groups, which define the constraints for generating personas (yellow). It then creates scope documents and competency questions (blue), which are used to construct an ontology (green). If input documents are supplied, HyDRA can, as an optional final step, generate a kno… view at source ↗
Figure 1
Figure 1. The process begins with a user-defined domain description [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. HyDRA-generated, ontology-conditioned KG inserted into Neo4j as one-to-one mapping (partial graph is pictured). 4.1 Persona Generation Instead of directly generating personas, we first prompt the LLM to produce a prioritized list of stakeholder groups relevant to the do￾main. For each group, distinct personas are generated, with the num￾ber of personas per group increasing with the group’s priority. This [PITH_FULL… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: We prompt OpenAI’s o4-mini with the natural language version of the cypher query, given the same knowledge graph. The model answers that the first relationship exists, but cannot validate the second condition. 6.3 Discussion These results highlight a clear trade-off be…
Figure 3
Figure 3. Figure 3: We use the generated Neo4j Cypher query to directly query the knowledge graph that has been inserted into a Neo4j database. The chain of relationships exists, so we obtain a boolean true value. This Cypher query translates to the natural language question: “Is the skel…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 30 canonical work pages

  1. [1]

    Al Machot, M

    F. Al Machot, M. T. Horsch, and H. Ullah. Designing the conceptual landscape for a xair validation infrastructure. In Building Trustworthy AI: Transparent AI Systems via Language Models, Ontologies, and Log- ical Reasoning (TranspNet), volume 1375 ofLecture Notes in Networks and Systems, pages 25–34. Springer Nature, 2024. First Online: 16 May 2025

    work page 2024

  2. [2]

    S. Auer, V . Kovtun, M. Prinz, A. Kasprzik, M. Stocker, and M. E. Vi- dal. Towards a knowledge graph for science. In Proceedings of the 8th International Conference on Web Intelligence, Mining and Semantics , pages 1–6, 2018

    work page 2018

  3. [3]

    Baldwin, J

    T. Baldwin, J. H. Lau, and K. Verspoor. OLLM: End-to-End Ontology Construction with Regularised LLMs. In Findings of the Association for Computational Linguistics: ACL 2024, 2024

    work page 2024

  4. [4]

    Assessing LLMs Suitability for Knowledge Graph Completion

    A. Bayer and H. Paulheim. Assessing llms suitability for knowledge graph completion. arXiv preprint arXiv:2405.17249, 2024

    work page Pith review arXiv 2024

  5. [5]

    T. R. Besold, A. d’Avila Garcez, S. Bader, H. Bowman, P. Domingos, P. Hitzler, K.-U. Kuehnberger, L. C. Lamb, D. Lowd, P. Machado Vieira Lima, L. de Penning, G. Pinkas, H. Poon, and G. Zaverucha. Neural- symbolic learning and reasoning: A survey and interpretation. 2017. arXiv preprint arXiv:1711.03902

    arXiv 2017

  6. [6]

    X. Bian, A. Tor-Shell, and P. Tabuada. Pacti: Assume-Guarantee Con- tracts at Scale. In Proceedings of the International Conference on Em- bedded Software (EMSOFT), 2023

    work page 2023

  7. [7]

    P. A. Bonatti, S. Decker, A. Polleres, and V . Presutti. Knowledge graphs: New directions for knowledge representation on the semantic web (dagstuhl seminar 18371). Dagstuhl Reports, 8(9):29–111, 2019. doi: 10.4230/DagRep.8.9.29

    doi:10.4230/dagrep.8.9.29 2019

  8. [8]

    Brank, M

    J. Brank, M. Grobelnik, and D. Mladeni ´c. A survey of ontology evalu- ation techniques. Journal on Data Semantics, IV:132–157, 2005

    work page 2005

Show all 38 references
  1. [9]

    Carta, A

    S. Carta, A. Giuliani, L. Piano, A. S. Podda, L. Pompianu, and S. G. Tiddia. Iterative zero-shot llm prompting for knowledge graph con- struction. 2023. arXiv preprint arXiv:2307.01128

    2023 arXiv

  2. [10]

    B. C. Colelough and W. Regli. Neuro-symbolic ai in 2024: A systematic review. 2025. arXiv preprint arXiv:2501.05435

    2024 arXiv

  3. [11]

    D. Daza, M. Cochez, and G. de Melo. GraphRAFT: Fine-Tuning LLMs for Cypher Query Writing. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies (NAACL), 2024

    2024

  4. [12]

    M.-C. Dinu, C. Leoveanu-Condrei, W. Zellinger, M. Holzleitner, and S. Hochreiter. Symbolicai: A framework for logic-based ap- proaches combining generative models and solvers. In Proceedings of the 3rd Conference on Lifelong Learning Agents (CoLLAs) , 2024. arXiv:2402.00854

    2024 arXiv

  5. [13]

    A. Funk, T. Kliegr, T. Soru, A. Primpeli, T. Pellegrini, and A. Angioni. LLMs4OL: A Framework for Large-Scale Ontology Learning. In Pro- ceedings of the ISWC 2023 Posters, Demos and Industry Tracks: From Novel Ideas to Industrial Applications co-located with 22nd Interna- tion...

    2023

  6. [14]

    Gangemi, V

    A. Gangemi, V . Presutti, D. R. Recupero, A. G. Nuzzolese, F. Draic- chio, and M. Mongiovì. FRED: Frame-Based Event Recognition and Description. Semantic Web, 8(6):821–832, 2017

    2017

  7. [15]

    Ghosh, R

    A. Ghosh, R. Li, Z. Zhang, and O. Bastani. ML-Guard: Runtime Contract Enforcement for ML Pipelines. In Proceedings of the 2023 USENIX Annual Technical Conference (USENIX ATC ’23), 2023

    2023

  8. [16]

    Gruninger and M

    M. Gruninger and M. S. Fox. The role of competency questions in enterprise engineering. In Proceedings of the IFIP WG5.7 Workshop on Benchmarking - Theory and Practice, pages 22–29, 1995

    1995

  9. [17]

    C. A. R. Hoare. An axiomatic basis for computer programming. Com- munications of the ACM , 12(10):576–580, Oct. 1969. doi: 10.1145/ 363235.363259

    1969

  10. [18]

    C. M. Keet. What We Still Don’t Evaluate in KGs. Semantic Web Journal, 13(9):1237–1250, 2022

    2022

  11. [19]

    Y . Kim, J. Wu, Y . Abdulle, and H. Wu. Medexqa: Medical question answering benchmark with multiple explanations, 2024

    2024

  12. [20]

    Knublauch and D

    H. Knublauch and D. Kontokostas. Shapes Constraint Language (SHACL). W3C Recommendation, July 2017. URL https://www.w3. org/TR/shacl/

    2017

  13. [21]

    L. Lamport. Proving the correctness of multiprocess programs. IEEE Transactions on Software Engineering , SE-3(2):125–143, Mar. 1977. doi: 10.1109/TSE.1977.229904

    1977

  14. [22]

    S. Li, Z. Wang, X. Zhang, et al. Knowledge graphs, large lan- guage models, and hallucinations: An nlp perspective. arXiv preprint arXiv:2411.14258, 2024

    2024 arXiv

  15. [23]

    G. F. Marcus. The Algebraic Mind: Integrating Connectionism and Cognitive Science . The MIT Press, Cambridge, MA, 2001. ISBN 9780262279086. doi: 10.7551/mitpress/1187.001.0001. Bradford Books imprint

    2001 doi

  16. [24]

    design by contract

    B. Meyer. Applying "design by contract". IEEE Computer, 25(10): 40–51, 1992. doi: 10.1109/2.161279

    1992 doi

  17. [25]

    B. Meyer. Object-Oriented Software Construction. Prentice-Hall, Up- per Saddle River, NJ, USA, 2nd edition, 1997

    1997

  18. [26]

    Mohajerani, I

    S. Mohajerani, I. Hasuo, K. Komorida, and Y . Sato. A theory of proba- bilistic contracts. In Proceedings of the 26th International Symposium on Formal Methods (FM), 2024. Forthcoming

    2024

  19. [27]

    H. B. Newcombe, J. M. Kennedy, S. J. Axford, and A. P. James. Auto- matic linkage of vital records. Science, 130(3381):954–959, 1959. doi: 10.1126/science.130.3381.954

    1959 doi

  20. [28]

    NeMo Guardrails

    NVIDIA. NeMo Guardrails. https://github.com/NVIDIA/ NeMo-Guardrails, 2023. Accessed: 2025-06-11

    2023

  21. [29]

    Odense and A

    S. Odense and A. d’Avila Garcez. A semantic framework for neuro- symbolic computing. 2024. arXiv preprint arXiv:2212.12050

    2024 arXiv

  22. [30]

    OpenAI. GPT-4.1. Large language model, Apr. 2025. URL https: //platform.openai.com/. Released April 14, 2025; accessed via OpenAI API

    2025

  23. [31]

    Z. Pan, K. Zhao, and Y . Luo. Retrieval-Augmented CQ Generation for Scientific Domains. In Proceedings of the Extended Semantic Web Conference (ESWC), 2024

    2024

  24. [32]

    Paulheim

    H. Paulheim. Knowledge graph refinement: A survey of approaches and evaluation methods. pages 489–508. SAGE Publications, 2016. Published in the journal Semantic Web, V ol. 8, No. 3

    2016

  25. [33]

    Ribeiro, G

    L. Ribeiro, G. Costa, and G. de Melo. KGrEaT: KG Quality via Down- stream Tasks. InProceedings of the 30th ACM International Conference on Information & Knowledge Management (CIKM ’21) , pages 1569– 1578, 2021

    2021

  26. [34]

    Robinson, S

    P. Robinson, S. Singh, and J. Morris. GraphRAG: Augmenting LLMS with Knowledge Graphs, 2023

    2023

  27. [35]

    H. Wu, Y . Chen, Y . Wang, and J. Li. Multi-persona prompting for di- verse cqs, 2024

    2024

  28. [36]

    Zhang and H

    B. Zhang and H. Soh. Extract, define, canonicalize: An llm-based framework for knowledge graph construction. 2024. arXiv preprint arXiv:2404.03868

    2024 arXiv

  29. [37]

    Zhang, X

    R. Zhang, X. Chen, L. Gui, Y . Chen, Y . Tang, and Y . Zhang. AdaKGC: Schema-Adaptable KG Construction. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1–15, 2023

    2023

  30. [38]

    Zhong, J

    L. Zhong, J. Wu, Q. Li, H. Peng, and X. Wu. A comprehensive survey on automatic knowledge graph construction. Association for Comput- ing Machinery, 2023. Published in ACM Computing Surveys, V ol. 56, No. 4, Article 94. DOI: 10.1145/3618295

    2023 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.