Pith. sign in

REVIEW 4 major objections 5 minor 53 references

Knowledge Conceptualization Impacts RAG Efficacy

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A knowledge graph's schema complexity and the format in which it is shown to a language model materially change whether an AI agent can translate competency questions into correct SPARQL queries, so schema design is an active variable in…

desk verdict A genuinely new but exploratory comparison of schema complexity and representation for LLM SPARQL generation, whose central causal claim about complexity is not isolated from size and coverage confounds. read the letter →

arxiv 2507.09389 v1 pith:3UKN7EW6 submitted 2025-07-12 cs.AI cs.CYcs.IR

Chris Davis Jaldi , Anmol Saini , Elham Ghiasi , O. Divine Eziolise , Cogan Shimizu This is my paper
classification cs.AIcs.CYcs.IR
keywords agenticRAGknowledgegraphsSPARQLgenerationontologycomplexityschemarepresentationreificationlargelanguagemodelscompetencyquestions
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the schema of a knowledge graph—how complex it is and how it is presented to a large language model—is a first-order determinant of whether an agentic retrieval-augmented generation (RAG) system can translate natural-language competency questions into correct SPARQL queries. The authors compare two real-world knowledge graphs, each in a fuller and a simplified form, using two representation formats (plain node-edge-node triples and formal axioms in Manchester Syntax) across three levels of question complexity. They find that simpler schemas generally help, but representation format interacts with schema size: axiomatization helped the smaller schemas and hurt the larger ones. The paper concludes that query generation is strongly influenced by both complexity and representation, in ways that relate to the particular competency question. If right, this means ontology design choices should be treated as active variables when building LLM-driven query systems, not as passive background.

What carries the argument

The experiment itself is the load-bearing mechanism. Each knowledge source supplies a pair of schemas chosen to differ in conceptual complexity: KnowWhereGraph versus the simplified KnowWhereGraph-Lite, and the Enslaved ontology versus its Wikibase-based alternative. Each schema is presented to the language model in two formats—node-edge-node triples describing connections between entities, and axiomatic statements in Manchester Syntax capturing classes and constraints—paired with competency questions classified as simple, moderate, or complex. The LLM (GPT-4o, chosen after a three-model comparison) produces a SPARQL query, which is scored on a three-point scale of syntactic and semantic validity with qualitative annotations; the distribution of those scores across schema, format, and question complexity is the evidence for the hypotheses.

What would settle it

Build two schemas over identical data and identical class and property coverage, differing only in how much reification (intermediate node classes) they use, and compare SPARQL accuracy; if accuracy is unchanged, the complexity effect in H1 is an artifact of size and coverage.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central finding is that both the internal complexity of an ontology (its reification structure) and the way that ontology is serialized into the prompt measurably change an LLM's ability to generate syntactically and semantically valid SPARQL queries. In the KnowWhereGraph family, the simplified KWG-Lite schema outperformed the full schema under both representations, and the plain triple-based representation outperformed the axiomatic one across all competency-question levels, supporting the complexity hypothesis and part of the representation hypothesis. In the Enslaved family, the smaller Wikibase schema beat the larger Enslaved ontology, but the axiomatic format was the best performer there, which runs against the simple expectation that simpler representations always help. Hallucinated classes and relationships were the dominant failure mode, appearing most when a competency question had no direct mapping to terms in the provided schema. The paper reads these results as evidence that schema complexity alone does not determine performance; the interaction between complexity and representation does.

Load-bearing premise

The comparisons assume the paired knowledge graphs differ mainly in schema complexity, but they also differ in size and which topics they cover, so a simpler schema's better query results could be due to being smaller rather than being less complex.

Editorial extensions

If this is right

  • Ontology engineers building knowledge graphs for LLM access should treat flat, smaller schemas as the default, since both the KnowWhereGraph and Enslaved families showed better query generation on the simplified variants.
  • Representation format should be tuned to schema size: axiomatic (Manchester Syntax) presentation helped the small Enslaved Wikibase schema but sharply degraded the large KnowWhereGraph schema, where plain triples were more effective.
  • Agentic RAG systems should avoid injecting full schemas into prompts; the size-driven performance drops point to dynamic or partial schema injection, such as selecting only the fragments relevant to the competency question.
  • Hallucination in generated SPARQL is concentrated where competency questions lack direct term mappings to the schema, suggesting that grounding constraints and schema-validation feedback could reduce the dominant failure mode.
  • These results are specific to SPARQL and GPT-4o, but they imply that schema design should be treated as a measured variable in any evaluation of neurosymbolic or knowledge-grounded generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • By the same logic the paper applies to SPARQL, other schema-grounded generation tasks—text-to-SQL, API selection, and tool use—should inherit the same sensitivity to ontology complexity and serialization format; that is an extension the paper gestures at but does not test.
  • The size/coverage confound is testable: two schemas over identical data with identical coverage but different reification levels would isolate the complexity effect that H1 claims; the paper calls for ablations, and this is the cleanest one.
  • The reversal in the Enslaved family suggests a ceiling-and-floor effect: representation differences matter most at intermediate schema sizes, and very small schemas may benefit from axioms while very large ones need triples or selective injection.
  • A default mitigation direction implied by the failure data is to constrain decoding to only URIs appearing in the injected schema, directly attacking the dominant hallucination pattern the authors catalogue.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper investigates how the conceptual structure of a knowledge graph schema and the way that schema is represented to an LLM affect the ability of an agentic RAG system to generate correct SPARQL queries from competency questions. Four schemas are used: KnowWhereGraph (KWG) and its simplified variant KWG-Lite, and Enslaved.org and its Wikibase variant. Each schema is presented either as node-edge-node (NEN) triples or as Manchester-Syntax axioms, and GPT-4o is prompted to answer CQs of three complexity levels. Queries are scored on a -1/0/1 scale. The paper reports that KWG-Lite outperforms KWG, that the axiomatized Enslaved Wikibase representation outperforms the original Enslaved schema, and concludes that both schema complexity and representation affect RAG efficacy, with hallucination occurring when CQs lack direct mappings to schema terms.

Significance. The question addressed is timely: as RAG systems increasingly consume structured knowledge, understanding how schema design and serialization choices affect LLM query generation has practical value for ontology engineering and neurosymbolic system design. The paper contributes a concrete experimental setup with publicly available data and code, and it examines two domains and three CQ complexity levels. However, the significance is conditional: the reported effects are plausible but the experimental design does not cleanly isolate the factors named in the hypotheses, so the paper currently supports a weaker claim about schema characteristics mattering rather than specifically reification-based complexity or representation format. If the authors can either eliminate the confounds or reframe the claims accordingly, the study would be a useful empirical contribution to an underexplored area.

major comments (4)
  1. [Section 3.2 / Section 4.2 / Table 2] The evidence for H1 is confounded: the paired schemas differ not only in reification-based complexity but also in size, class/property counts, content coverage, and required hop length. KWG-Lite is introduced as 'a simplified version ... with fewer and less complex relationships' and as requiring fewer hops (Section 3.2), while the Enslaved schema was deliberately reduced to the portion present in the Wikibase alternative 'to reduce bias' (Section 3.2). No quantitative reification metric is reported for any schema. The paper itself later concedes that 'schema size significantly impacts LLM performance' (Section 4.2). Consequently, large performance gaps such as KWG axioms (µ=-0.79) vs. KWG-Lite axioms (µ=0.50) in Table 2 cannot be attributed specifically to reification complexity; they are equally consistent with a size or prompt-length effect. To support H1, the authors need either controlled variants that hold size and coverage fixed while varying reification, or a reframing of H1 as a combined effect of schema characteristics with reification as one component rather than the primary driver.
  2. [Section 3.2] The representation comparison (H2) is undermined by non-independence of the two representation conditions. For schemas where axioms were not readily available, the authors 'programmatically generate[d] the scoped domain and range based on the corresponding NEN representations.' This means the axiomatic representation is derived from, and therefore not independent of, the NEN representation; any performance difference between the conditions could reflect the derivation process rather than the representation form. The acknowledged information loss when saving OWL/TTL in Manchester Syntax further complicates the comparison. The paper should either use independently authored axiom sets, or clearly qualify H2 as a comparison of two specific serialization pipelines rather than of representation formalisms.
  3. [Section 4 / Table 2] The central quantitative claims are not supported by inferential statistics. The results are summarized with means and standard deviations over small per-cell samples (n=6 to 11), yet the paper concludes that query generation is 'strongly influenced' by both complexity and representation. Several reported differences are smaller than the associated standard deviations (e.g., Enslaved Wikibase Simple Axioms 0.34±0.75 vs. Moderate Axioms 0.00±0.82), so without significance tests, confidence intervals, or effect sizes, the strength of the claims is not established. The authors should report per-CQ paired comparisons between conditions (since the same CQs are used across schemas/representations) and provide effect-size estimates; otherwise the conclusion should be softened to describe descriptive trends.
  4. [Section 4.2 / Section 5] The treatment of H2 is internally inconsistent and needs clarification. In Section 4.2, the better performance of axiomatized Enslaved Wikibase is described as 'rejecting H2,' but H2 as stated in the introduction predicts that 'when a simpler representation is used, the model will generally perform better.' If the axiomatic representation is considered more complex or more expressive, then the result is contrary to H2, not merely a rejection; if the axiomatic representation is considered simpler in some sense, the authors should define that sense and explain why H2 is rejected rather than supported. The abstract's generic conclusion that 'there are impacts from both approaches' sidesteps the directional nature of the hypotheses, and the discussion should reconcile these statements.
minor comments (5)
  1. [Abstract / Section 1] The abstract opens with 'Explainability and interpretability are cornerstones...', but the paper does not measure explainability or interpretability; the framing should be aligned with the actual focus on schema structure and query generation.
  2. [Section 3.5 / Table 2] The scoring rubric uses -1, 0, 1 with descriptions in the text, while Table 2's note labels the classes as 'not accurate', 'partially accurate', and 'accurate'; please harmonize the terminology to avoid confusion.
  3. [Section 4.1] The paper states that GPT-4o was chosen after analysis of three models, but no quantitative comparison of Mistral Large, Gemini Flash 2.0, and GPT-4o is provided; including a brief table or summary would strengthen reproducibility.
  4. [Figure 3] The prompt templates contain placeholder tokens 'Insert_CQ_here' and 'Insert_schema_here' with underscores; using a consistent placeholder convention or a footnote would improve readability.
  5. [Section 3.3] The description of CQ generalization says the CQs were 'generalized' to limit bias, but the generalization procedure is not described; a short explanation of how generalization was performed and who did it would help readers assess potential bias in the question set.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the paper's empirical claims are not equivalent to its inputs by construction.

full rationale

This paper is an empirical study comparing LLM-generated SPARQL queries across schemas and representation formats. It contains no derivation chain in which a predicted quantity reduces to an input by construction: there is no fitted parameter later relabeled as a prediction, no result that is definitionally identical to a measurement, and no load-bearing uniqueness theorem or ansatz imported from prior work by the authors. The schemas and competency questions come from the authors' earlier projects, but that is provenance for experimental inputs, not circular evidence: the paper does not cite its own prior work to justify the truth of its conclusions. The programmatic generation of axiomatic representations from NEN representations, disclosed in Section 3.2, weakens the independence of the representation manipulation, and the concession in Section 4.2 that schema size significantly impacts performance reveals a confound with the complexity construct. These are methodological validity threats, not circularity under the criteria used here, which require an exhibited reduction of a claimed result to its own input. Accordingly, no circular step is identified.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The study is empirical; its central claim rests on hand-chosen experimental parameters (temperature, complexity definitions, CQ selection, schema reduction, model choice, scoring rubric) and domain assumptions about the validity of these choices. No new theoretical entities are introduced.

free parameters (6)
  • LLM temperature = 0.8
    Chosen after iterative pilot experiments to balance creativity and consistency (Section 3.4).
  • CQ complexity thresholds = 1 hop / 3 hops
    Simple defined as single hop, moderate as 2-3 hops, complex as >3 hops (Section 3.3).
  • CQ set and generalizations = Not fully specified
    CQs curated from existing KWG-Lite and Enslaved sources and generalized to limit bias (Section 3.3); the exact selection is not enumerated in the paper.
  • Enslaved schema reduction = Subset shared with Wikibase
    Original Enslaved schema reduced to a portion present in the alternative representation to reduce bias (Section 3.2).
  • Model choice = GPT-4o
    Selected after unreported analysis of three LLMs for superior semantic accuracy in generated queries (Section 3.4).
  • Scoring rubric thresholds = -1/0/1 Likert
    Hand-defined 3-point scale; missing namespaces not penalized; no inter-annotator agreement check (Section 3.5).
assumptions (5)
  • domain assumption LLM-generated SPARQL queries are a valid proxy for RAG efficacy
    The paper equates query correctness with RAG system efficacy in RQ1 and the evaluation; this is asserted rather than validated.
  • domain assumption The Likert scoring is a reliable and unbiased measure of query correctness
    All conclusions rest on these scores; no inter-annotator reliability analysis is reported (Section 3.5).
  • domain assumption The selected competency questions are representative and correctly classified by hop count
    CQs are taken from existing sources and generalized; representativeness is not validated (Section 3.3).
  • domain assumption The schema representations accurately capture the source ontologies
    For some schemas the axiomatic form was generated from NEN, and Manchester Syntax cannot express all OWL constructs, so semantic fidelity is limited (Section 3.2).
  • domain assumption Schema pairs differ primarily in complexity rather than size or content
    KWG vs KWG-Lite and Enslaved vs Wikibase differ in size and coverage; Section 4.2 concedes schema size impacts performance, confounding H1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Knowledge Conceptualization Impacts RAG Efficacy." pith.science (2026). https://pith.science/paper/3UKN7EW6

@misc{pith2026250709389,
  author       = {Pith},
  title        = {Pith review of: Knowledge Conceptualization Impacts RAG Efficacy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3UKN7EW6}},
  note         = {Machine review of arXiv:2507.09389}
}
read the original abstract

Explainability and interpretability are cornerstones of frontier and next-generation artificial intelligence (AI) systems. This is especially true in recent systems, such as large language models (LLMs), and more broadly, generative AI. On the other hand, adaptability to new domains, contexts, or scenarios is also an important aspect for a successful system. As such, we are particularly interested in how we can merge these two efforts, that is, investigating the design of transferable and interpretable neurosymbolic AI systems. Specifically, we focus on a class of systems referred to as ''Agentic Retrieval-Augmented Generation'' systems, which actively select, interpret, and query knowledge sources in response to natural language prompts. In this paper, we systematically evaluate how different conceptualizations and representations of knowledge, particularly the structure and complexity, impact an AI agent (in this case, an LLM) in effectively querying a triplestore. We report our results, which show that there are impacts from both approaches, and we discuss their impact and implications.

Figures

Figures reproduced from arXiv: 2507.09389 by the authors.

Figure 1
Figure 1. This figure overall demonstrates the positioning of our work in agentic [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. This figure shows a graphical depiction of our methodology. We have taken [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. System and User Prompts 4 Results We evaluated a set of CQs with the same schema across different LLMs – so as to minimize confounding factors – to identify the best model in understanding, interpreting, and generating valid SPARQL queries. OpenAI GPT-4o performed better than Gemini Flash 2.0 and Mistral Large in constructing structurally sound SPARQL queries from natural language. As such, we proceeded with GPT-4o … view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 37 canonical work pages

  1. [1]

    Foundations of Artificial Intelligence3, 135–179 (2008)

    Baader, F., Horrocks, I., Sattler, U.: Description logics. Foundations of Artificial Intelligence3, 135–179 (2008)

    work page 2008

  2. [2]

    arXiv preprint arXiv:2409.05746 (2024)

    Banerjee, S., Agarwal, A., Singla, S.: Llms will always hallucinate, and we need to live with this. arXiv preprint arXiv:2409.05746 (2024)

    arXiv 2024

  3. [3]

    arXiv preprint arXiv:2405.13012 (2024) 16 Jaldi, C

    Bellemare-Pepin, A., Lespinasse, F., Th¨ olke, P., Harel, Y., Mathewson, K., Olson, J.A., Bengio, Y., Jerbi, K.: Divergent creativity in humans and large language models. arXiv preprint arXiv:2405.13012 (2024) 16 Jaldi, C. D., et al

    arXiv 2024

  4. [4]

    In: 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelli- gence (WI) and Intelligent Agent Technologies (IAT)

    Bezerra, C., Freitas, F., Santana, F.: Evaluating ontologies with competency ques- tions. In: 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelli- gence (WI) and Intelligent Agent Technologies (IAT). vol. 3, pp. 284–285. IEEE (2013)

    work page 2013

  5. [5]

    for Biomedical Informatics Research, S.C.: Prot´ eg´ e ontology editor and framework (2025),https://protege.stanford.edu/, accessed: 2025-05-11

    work page 2025

  6. [6]

    arXiv preprint arXiv:2404.15923 (2024)

    Boylan, J., Mangla, S., Thorn, D., Ghalandari, D.G., Ghaffari, P., Hokamp, C.: Kg- validator: A framework for automatic validation of knowledge graph construction. arXiv preprint arXiv:2404.15923 (2024)

    arXiv 2024

  7. [7]

    Christou, A., Dave, B., Shimizu, C.: Experiments in graph structure and knowledge graph embeddings

    work page

  8. [8]

    In: International Conference on Neural- Symbolic Learning and Reasoning

    Dave, B., Christou, A., Shimizu, C.: Towards understanding the impact of graph structure on knowledge graph embeddings. In: International Conference on Neural- Symbolic Learning and Reasoning. pp. 41–50. Springer (2024)

    work page 2024

Show all 53 references
  1. [9]

    In: International Semantic Intelligence Conference

    Dave, B., Shimizu, C.: Towards understanding the impact of schema on knowledge graph embeddings. In: International Semantic Intelligence Conference. pp. 3–10. Springer (2023)

    2023

  2. [10]

    arXiv preprint arXiv:2410.06062 (2024)

    Emonet, V., Bolleman, J., Duvaud, S., de Farias, T.M., Sima, A.C.: Llm-based sparql query generation from natural language over federated knowledge graphs. arXiv preprint arXiv:2410.06062 (2024)

    2024 arXiv

  3. [11]

    Enslaved: Peoples of the Historical Slave Trade.https://enslaved.org/, accessed: 2025-05-11

    2025

  4. [12]

    In: Description Logics (2023)

    Gehrke, B., Mossakowski, T.: Extending owl2 manchester syntax to include missing features from owl2 abstract syntax. In: Description Logics (2023)

    2023

  5. [13]

    Gemini Flash 2.0.https://cloud.google.com/vertex-ai/generative-ai/docs/ models/gemini/2-0-flash, accessed: 2025-05-11

    2025

  6. [14]

    Knowl- edge acquisition5(2), 199–220 (1993)

    Gruber, T.R.: A translation approach to portable ontology specifications. Knowl- edge acquisition5(2), 199–220 (1993)

    1993

  7. [15]

    Hawthorne, W., Williams, D., Niven, S.J., McClelland, K.: Enslaved.org compe- tency questions,https://docs.enslaved.org/competencyQuestions/, accessed: 2025-05-11

    2025

  8. [16]

    Communications of the ACM64(2), 76–83 (2021)

    Hitzler, P.: A review of the semantic web field. Communications of the ACM64(2), 76–83 (2021)

    2021

  9. [17]

    Hitzler, P., Sarker, M.K.: Neuro-symbolic artificial intelligence: The state of the art (2022)

    2022

  10. [18]

    ACM Computing Surveys (Csur)54(4), 1–37 (2021)

    Hogan, A., Blomqvist, E., Cochez, M., d’Amato, C., Melo, G.D., Gutierrez, C., Kirrane, S., Gayo, J.E.L., Navigli, R., Neumaier, S., et al.: Knowledge graphs. ACM Computing Surveys (Csur)54(4), 1–37 (2021)

    2021

  11. [19]

    In: OWLed

    Horridge, M., Drummond, N., Goodwin, J., Rector, A.L., Stevens, R., Wang, H.: The manchester owl syntax. In: OWLed. vol. 216 (2006)

    2006

  12. [20]

    Accessed: 2025-05-11

    Horridge, M., Patel-Schneider, P.F.: Owl 2 web ontology language: Manchester syn- tax (second edition) (2012),https://www.w3.org/TR/owl2-manchester-syntax/, w3C Working Group Note. Accessed: 2025-05-11

    2012

  13. [21]

    arXiv preprint arXiv:2212.10403 (2022)

    Huang, J., Chang, K.C.C.: Towards reasoning in large language models: A survey. arXiv preprint arXiv:2212.10403 (2022)

    2022 arXiv

  14. [22]

    Knowledge-Conceptualization-GraphRAG.https://github.com/kastle-lab/ knowledge-conceptualization-graphrag, accessed: 2025-05-11

    2025

  15. [23]

    https://knowwheregraph.org/

    KnowWhereGraph. https://knowwheregraph.org/

  16. [24]

    In: SEMANTiCS (Posters & Demos) (2023) Knowledge Conceptualization Impacts RAG Efficacy 17

    Kovriguina, L., Teucher, R., Radyush, D., Mouromtsev, D.: Sparqlgen: One-shot prompt-based approach for sparql query generation. In: SEMANTiCS (Posters & Demos) (2023) Knowledge Conceptualization Impacts RAG Efficacy 17

    2023

  17. [25]

    Lab, K.: Nl-to-sparql using llm (2025),https://github.com/kastle-lab/ NL-to-SPARQL-using-llm, accessed: 2025-05-11

    2025

  18. [26]

    Advances in neural information processing systems 33, 9459–9474 (2020)

    Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., K¨ uttler, H., Lewis, M., Yih, W.t., Rockt¨ aschel, T., et al.: Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems 33, 9459–9474 (2020)

    2020

  19. [27]

    org/2016/05/ontolex/, accessed: 2025-05-11

    OntoLex: TLexicon Model for Ontologies - Community Report.https://www.w3. org/2016/05/ontolex/, accessed: 2025-05-11

    2016

  20. [28]

    ACM computing surveys (CSUR)51(3), 1–34 (2018)

    Liu, Y., Safavi, T., Dighe, A., Koutra, D.: Graph summarization methods and applications: A survey. ACM computing surveys (CSUR)51(3), 1–34 (2018)

    2018

  21. [29]

    arXiv preprint arXiv:2302.09185 (2023)

    Lu, A., Zhang, H., Zhang, Y., Wang, X., Yang, D.: Bounding the capabilities of large language models in open text generation with prompt constraints. arXiv preprint arXiv:2302.09185 (2023)

    2023 arXiv

  22. [30]

    Advances in Neural Information Processing Systems36, 46534–46594 (2023)

    Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al.: Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems36, 46534–46594 (2023)

    2023

  23. [31]

    Big Data and Cognitive Computing8(9), 115 (2024)

    Mansurova, A., Mansurova, A., Nugumanova, A.: Qa-rag: Exploring llm reliance on external knowledge. Big Data and Cognitive Computing8(9), 115 (2024)

    2024

  24. [32]

    arXiv preprint arXiv:2409.05925 (2024)

    Meyer, L.P., Frey, J., Brei, F., Arndt, N.: Assessing sparql capabilities of large language models. arXiv preprint arXiv:2409.05925 (2024)

    2024 arXiv

  25. [33]

    Mistral Large.https://mistral.ai/news/mistral-large-2407, accessed: 2025- 05-11

    2025

  26. [34]

    In: Handbook on Neurosymbolic AI and Knowledge Graphs, pp

    Norouzi, S.S., Barua, A., Christou, A., Gautam, N., Eells, A., Hitzler, P., Shimizu, C.: Ontology population using llms. In: Handbook on Neurosymbolic AI and Knowledge Graphs, pp. 421–438. IOS Press (2025)

    2025

  27. [35]

    Noy, N.F., Gao, Y., Jain, A., Narayanan, A., Patterson, A., Taylor, J.: Industry- scale knowledge graphs: lessons and challenges. Commun. ACM62(8), 36–43 (2019). https://doi.org/10.1145/3331166,https://doi.org/10.1145/3331166

    2019 doi

  28. [36]

    ChatGPT.https://openai.com/index/hello-gpt-4o/, accessed: 2025-05-11

    2025

  29. [37]

    Journal of information science50(4), 1019–1029 (2024)

    Rajabi, E., Etminani, K.: Knowledge-graph-based explainable ai: A systematic review. Journal of information science50(4), 1019–1029 (2024)

    2024

  30. [38]

    In: Proceedings of the 7th Joint Workshop on Graph Data Management Experiences & Systems (GRADES) and Network Data Analytics (NDA)

    Sequeda, J., Allemang, D., Jacob, B.: A benchmark to understand the role of knowledge graphs on large language model’s accuracy for question answering on enterprise sql databases. In: Proceedings of the 7th Joint Workshop on Graph Data Management Experiences & Systems (GRADES)...

    2024

  31. [39]

    Journal of Web Semantics82, 100823 (2024)

    Shimizu, C., Eells, A., Gonzalez, S., Zhou, L., Hitzler, P., Sheill, A., Foley, C., Rehberger, D.: Ontology design facilitating wikibase integration—and a worked example for historical data. Journal of Web Semantics82, 100823 (2024)

    2024

  32. [40]

    Journal of Web Semantics p

    Shimizu, C., Hitzler, P.: Accelerating knowledge graph and ontology engineering with large language models. Journal of Web Semantics p. 100862 (2025)

    2025

  33. [41]

    Journal of Web Semantics63, 100567 (2020)

    Shimizu, C., Hitzler, P., Hirt, Q., Rehberger, D., Estrecha, S.G., Foley, C., Sheill, A.M., Hawthorne, W., Mixter, J., Watrall, E., et al.: The enslaved ontology: Peoples of the historic slave trade. Journal of Web Semantics63, 100567 (2020)

    2020

  34. [42]

    Shimizu, C., Hitzler, P., Hirt, Q., Shiell, A., Gonzalez, S., Foley, C., Rehberger, D., Watrall, E., Hawthorne, W., Tarr, D., Carty, R., Mixter, J.: The enslaved ontology 1.0: People of the historic slave trade. Tech. rep., Michigan State University, East Lansing, Michigan (Ap...

    2019

  35. [43]

    Shimizu, C., Stephen, S., Christou, A., Currier, K., Mahdavinejad, M.S., Norouzi, S.S., Dalal, A., Barua, A., Fisher, C., D’Onofrio, A., Thelen, T., Janowicz, K., Rehberger, D., Schildhauer, M., Hitzler, P.: Knowwheregraph- lite: Sparql construct queries (2023),https://github....

    2023

  36. [44]

    In: Iberoamerican Knowledge Graphs and Semantic Web Conference

    Shimizu, C., Stephen, S., Christou, A., Currier, K., Mahdavinejad, M.S., Norouzi, S.S., Dalal, A., Barua, A., Fisher, C.K., D’Onofrio, A., et al.: Knowwheregraph- lite: a perspective of the knowwheregraph. In: Iberoamerican Knowledge Graphs and Semantic Web Conference. pp. 199...

    2023

  37. [45]

    arXiv preprint arXiv:2503.00735 (2025)

    Simonds, T., Yoshiyama, A.: Ladder: Self-improving llms through recursive prob- lem decomposition. arXiv preprint arXiv:2503.00735 (2025)

    2025 arXiv

  38. [46]

    Bioinformatics 40(9), btae560 (2024)

    Soman, K., Rose, P.W., Morris, J.H., Akbas, R.E., Smith, B., Peetoom, B., Villouta-Reyes, C., Cerono, G., Shi, Y., Rizk-Jackson, A., et al.: Biomedical knowl- edge graph-optimized prompt generation for large language models. Bioinformatics 40(9), btae560 (2024)

    2024

  39. [47]

    Song, Z., Yan, B., Liu, Y., Fang, M., Li, M., Yan, R., Chen, X.: Injecting domain- specific knowledge into large language models: A comprehensive survey (2025), https://arxiv.org/abs/2502.10708

    2025 arXiv

  40. [48]

    arXiv preprint arXiv:2408.02442 (2024)

    Tam, Z.R., Wu, C.K., Tsai, Y.L., Lin, C.Y., Lee, H.y., Chen, Y.N.: Let me speak freely? a study on the impact of format restrictions on performance of large lan- guage models. arXiv preprint arXiv:2408.02442 (2024)

    2024 arXiv

  41. [49]

    Advances in neural information pro- cessing systems30(2017)

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. Advances in neural information pro- cessing systems30(2017)

    2017

  42. [50]

    In: China Conference on Knowledge Graph and Semantic Computing

    Yang, S., Teng, M., Dong, X., Bo, F.: Llm-based sparql generation with selected schema from large scale knowledge base. In: China Conference on Knowledge Graph and Semantic Computing. pp. 304–316. Springer (2023)

    2023

  43. [51]

    arXiv preprint arXiv:2501.13958 (2025)

    Zhang, Q., Chen, S., Bei, Y., Yuan, Z., Zhou, H., Hong, Z., Dong, J., Chen, H., Chang, Y., Huang, X.: A survey of graph retrieval-augmented generation for cus- tomized large language models. arXiv preprint arXiv:2501.13958 (2025)

    2025

  44. [52]

    arXiv preprint arXiv:2409.14924 (2024)

    Zhao, S., Yang, Y., Wang, Z., He, Z., Qiu, L.K., Qiu, L.: Retrieval augmented generation (rag) and beyond: A comprehensive survey on how to make your llms use external data more wisely. arXiv preprint arXiv:2409.14924 (2024)

    2024 arXiv

  45. [53]

    arXiv preprint arXiv:2502.13874 (2025)

    Zhu, R., Shimizu, C., Stephen, S., Fisher, C.K., Thelen, T., Currier, K., Janowicz, K., Hitzler, P., Schildhauer, M., Li, W., et al.: The knowwheregraph: A large-scale geo-knowledge graph for interdisciplinary knowledge discovery and geo-enrichment. arXiv preprint arXiv:2502.1...

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.