Pith. sign in

REVIEW 3 major objections 6 minor 30 references

LLM-KG-Bench 3.0: A Compass for SemanticTechnology Capabilities in the Ocean of LLMs

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A public benchmark automatically scores 30+ LLMs on RDF and SPARQL skills.

desk verdict A genuinely useful open benchmark for LLM+KGE with a clear soft spot: the automated scoring metrics are never checked against human judgment, so the headline comparisons rest on unvalidated proxies. read the letter →

arxiv 2505.13098 v1 pith:DSVI3YFQ submitted 2025-05-19 cs.AI cs.CLcs.DB

Lars-Peter Meyer , Johannes Frey , Desiree Heim , Felix Brei , Claus Stadler , Kurt Junghanns , Michael Martin This is my paper · ORCID
classification cs.AIcs.CLcs.DB
keywords LLMevaluationRDFSPARQLknowledgegraphengineeringsemantictechnologiesbenchmarkframeworkTurtleJSON-LD
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to establish that LLM-KG-Bench 3.0 is a working, extensible benchmark for automatically scoring how well large language models handle knowledge-graph engineering tasks: generating and repairing RDF in multiple serializations, answering questions over small graphs, and writing SPARQL queries. If that claim holds, researchers can compare dozens of LLMs on semantic-technology skills without manually reading answers, and can track capability changes as models are updated. The paper backs the claim with a released framework, a dataset of prompts, answers, and scores for more than 30 open and proprietary models, and exemplary model cards. A sympathetic reader would treat the contribution as a public measurement resource and a method for further knowledge-graph engineering benchmarking.

What carries the argument

The prompt-answer-evaluate loop, orchestrated by the task API, is the load-bearing mechanism. Each task supplies the prompt and the evaluator; the model connector returns an answer; evaluation determines whether another correction round starts. For syntax-repair tasks, scoring is composite: parsableSyntax with weight 0.2, contentF1 on normalized triples with weight 0.7, string similarity with weight 0.1, plus a brevity score for extraneous text, with contentF1 computed only when the document parses. The capability compass spider plot aggregates these scores into per-model cards.

What would settle it

Have human semantic-web experts score a random sample of the published model answers on the same tasks, then compare their scores to the framework's automated scores; if the two disagree systematically, the reported rankings would not survive.

Watch

Extended reading notes

Core claim

The paper claims that the LLM-KG-Bench framework, in Version 3.0, provides an extensible, automated evaluation pipeline for LLM capabilities in semantic technologies. Applying this pipeline to more than 30 open and proprietary LLMs yields a public dataset of prompts, answers, and evaluation scores, and aggregate scores can be visualized as capability compasses across dimensions such as RDF syntax, RDF analytics, SPARQL syntax, SPARQL semantics, and brevity. The framework's design enables dialogue-based correction cycles: when a generated RDF or SPARQL answer fails parsing or content checks, the LLM is asked to revise it for up to three rounds, and final scores reflect the corrected answer.

Load-bearing premise

The automated scoring is assumed to reflect genuine knowledge-graph engineering ability, yet the paper never checks those scores against human expert judgment; if an automated evaluator is biased, every model comparison inherits the bias.

Editorial extensions

If this is right

  • New LLMs can be plugged in through existing connectors and compared on the same tasks without manual answer checking.
  • The published dataset allows others to rerun evaluations and re-evaluate stored answers with updated scoring code, without new LLM calls.
  • Serialization preferences between Turtle and JSON-LD can be detected statistically for individual models using paired t-tests.
  • Encrypted task data reduces the risk that benchmark prompts leak into training corpora.
  • Capability compass plots provide compact, comparable model cards for RDF and SPARQL proficiency.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The weighted combined score places 70% of its weight on contentF1, so a model that produces correct triples in an equivalent but differently ordered or named form may be penalized unless normalization captures that equivalence.
  • The same task API could be extended to other semantic-technology artifacts, such as SHACL shapes, OWL axioms, or R2RML mappings, though the paper does not propose those tasks.
  • The observed format-preference differences between models could inform serialization-specific prompting strategies, but the paper only reports the statistical differences, not how to exploit them.
  • Because the automated scores are not validated against expert human judgment, the rankings should be read as framework-internal scores until agreement with human assessment is demonstrated.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper describes LLM-KG-Bench 3.0, an open-source framework for automated evaluation of LLM capabilities on knowledge graph engineering tasks, including RDF serialization (Turtle, JSON-LD, N-Triples, RDF/XML) and SPARQL syntax and semantics. It introduces a revised Task API, an RDF-syntax repair task with dialogue-based correction cycles, encrypted task data to avoid test-set leakage, a vLLM connector, and capability compass visualizations. The authors apply the framework to evaluate more than 30 open and proprietary LLMs and publish the resulting prompts, answers, and evaluation scores, along with exemplary model cards and pairwise t-tests comparing Turtle versus JSON-LD performance.

Significance. If the automated scores are a valid proxy for human-judged knowledge graph engineering capability, this is a valuable community resource: it is the first benchmark framework that combines automatic evaluation with a broad set of KGE topics and many LLMs, and the released code and data (with DOIs) are concrete and reproducible. The framework's extensibility, the public dataset of model interactions, and the machine-checked evaluation pipeline are strong assets. However, the central validity claim for the automated evaluator is currently unsubstantiated, so the significance is conditional on the missing validation evidence.

major comments (3)
  1. [3.4] The automated evaluation scores defined in Section 3.4 (parsableSyntax, contentF1 computed on normalized triples, strSimilarity, and the combined score 0.1*strSimilarity + 0.2*parsableSyntax + 0.7*contentF1) are used to produce the capability compass plots and the Turtle-versus-JSON-LD t-test results in Table 3, but the paper provides no validation that these scores agree with human expert judgment. A systematic evaluator bias, for example a normalizer that penalizes semantically equivalent triple reorderings or alternative blank-node labels, would propagate into every model comparison derived from these scores. Please add a human-judgment validation study, an inter-annotator agreement analysis, or at least a qualitative error analysis of the automated evaluation, and discuss the sensitivity of the hand-weighted combined score.
  2. [4.2, Table 3] Table 3 reports two-sided t-test preferences between Turtle and JSON-LD, but the paper does not state which score was used as the dependent variable (e.g., contentF1, combined, or per-task scores), whether the tests were paired or independent, or how multiple comparisons across 33 models and six tasks were corrected. Without this information, the format-preference results are not reproducible from the description in the paper. Please specify the exact test procedure, the score, the sample sizes per model and task, and the multiple-testing correction.
  3. [4.1] The selection of 20 iterations for proprietary models and 50 iterations for open models is presented as a trade-off, but the paper does not show how this asymmetry affects the statistical power of the comparisons or the reported preference tests in Table 3. Since the dataset is itself a central contribution, the paper should include a comparison of effect sizes or confidence intervals for at least the headline models, to demonstrate that the iteration counts are sufficient for the claims made.
minor comments (6)
  1. [Title] The title on the arXiv page reads "SemanticTechnology" without a space; it should be "Semantic Technology".
  2. [3.4] In the combined score formula, "contentF 1" contains a stray space; it should read "contentF1".
  3. [4.2, Fig. 4] The capability compass plots are too small to read the dimension labels clearly; please enlarge them or provide a tabular summary of the aggregated scores.
  4. [2, Table 1] The row for LLM-KG-Bench 3 reads "many several automatic"; this should be clarified, for example by giving the number of models and tasks covered.
  5. [3.4] The description of RdfSyntaxFixList states that there are five task case entries for each serialization format; please also state the total number of task cases per format and how they are selected across iterations.
  6. [4.1, Table 3] The date notation in Table 3 (e.g., "2024/01") is not explained; clarifying that it denotes the model snapshot would help reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the benchmark scores are computed outputs from fixed task data, and self-citations document provenance rather than supplying load-bearing justification.

full rationale

This is a benchmark-resource paper, not a derivation. The central claims are that the LLM-KG-Bench framework provides automated evaluation tasks and that the published dataset enables capability comparisons. These are supported by the open-source code, the released prompts/answers/evaluations, and the explicit task and scoring definitions in Section 3.4. The scores (parsableSyntax, contentF1, strSimilarity, brevity, and the weighted combined score) are defined directly from LLM outputs and fixed gold data; they are not fitted parameters, and no equation in the paper reduces a reported result to an input. The t-tests in Table 3 compare measured score distributions, so they inherit any metric-validity limitations but are not circular. Citations to the authors' prior framework papers [8, 9, 19, 20] document the framework's history and task provenance; they are not invoked as a uniqueness theorem or as a substitute for the presented implementation and data. The absence of human validation of contentF1 is a measurement-validity concern, not a circular-dependency concern, because the paper does not claim the score is derived from human judgment. No specific self-definitional reduction, fitted-input-called-prediction, or imported-uniqueness step can be quoted from the text. Accordingly, the honest finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on benchmark design choices and domain assumptions rather than on fitted physical parameters or invented entities. The most important assumptions are that the automated evaluator is a valid measure of KGE capability and that task prompts and data are fair and correct. The paper ships data that allow others to test these assumptions, but it does not validate them internally.

free parameters (3)
  • Combined score weights = 0.1, 0.2, 0.7
    Hand-chosen weights for strSimilarity, parsableSyntax, and contentF1 in the RdfSyntaxFixList score; they determine the reported combined score and therefore influence model rankings.
  • Evaluation iterations per model = 20 for proprietary LLMs, 50 for open LLMs
    Chosen as a resource and confidence trade-off; the difference affects variance and the power of the reported t-tests.
  • t-test confidence thresholds = 95% and 99%
    Thresholds for declaring Turtle versus JSON-LD preference in Table 3; conventional but arbitrary significance levels.
assumptions (5)
  • domain assumption Automated scoring (parsableSyntax, contentF1, strSimilarity, brevity, combined) validly measures knowledge graph engineering capability.
    No human validation or inter-annotator agreement is reported; the scores are assumed to reflect semantic correctness even though only syntactic and triple-level F1 checks are described in Section 3.4.
  • domain assumption Task prompts are unambiguous and fair across different LLMs.
    The authors state they avoid LLM-specific prompt optimization, but no pilot study or robustness check is presented to support cross-model fairness.
  • domain assumption Encrypted task data prevents test data leakage into LLM training data.
    Section 3.1 lists data security as a feature, but the paper provides no empirical check for memorization or contamination.
  • domain assumption The task datasets contain correct expected answers and representative examples.
    The tasks use existing datasets such as LC-QuAD 2.0 and the authors' earlier ChatGPT experiments; errors or ambiguities in those datasets would cap reported model scores.
  • standard math Two-sided t-tests are appropriate for the aggregated per-iteration scores.
    Table 3 uses t-tests on repeated iterations; independence and normality of the score distributions are not discussed, which could affect the reported preference direction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLM-KG-Bench 3.0: A Compass for SemanticTechnology Capabilities in the Ocean of LLMs." pith.science (2026). https://pith.science/paper/DSVI3YFQ

@misc{pith2026250513098,
  author       = {Pith},
  title        = {Pith review of: LLM-KG-Bench 3.0: A Compass for SemanticTechnology Capabilities in the Ocean of LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DSVI3YFQ}},
  note         = {Machine review of arXiv:2505.13098}
}
read the original abstract

Current Large Language Models (LLMs) can assist developing program code beside many other things, but can they support working with Knowledge Graphs (KGs) as well? Which LLM is offering the best capabilities in the field of Semantic Web and Knowledge Graph Engineering (KGE)? Is this possible to determine without checking many answers manually? The LLM-KG-Bench framework in Version 3.0 is designed to answer these questions. It consists of an extensible set of tasks for automated evaluation of LLM answers and covers different aspects of working with semantic technologies. In this paper the LLM-KG-Bench framework is presented in Version 3 along with a dataset of prompts, answers and evaluations generated with it and several state-of-the-art LLMs. Significant enhancements have been made to the framework since its initial release, including an updated task API that offers greater flexibility in handling evaluation tasks, revised tasks, and extended support for various open models through the vllm library, among other improvements. A comprehensive dataset has been generated using more than 30 contemporary open and proprietary LLMs, enabling the creation of exemplary model cards that demonstrate the models' capabilities in working with RDF and SPARQL, as well as comparing their performance on Turtle and JSON-LD RDF serialization tasks.

Figures

Figures reproduced from arXiv: 2505.13098 by the authors.

Figure 3
Figure 3. fig. 3 [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 1
Figure 1. Overview of the evaluation workflow and execution scopes. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. UML class diagram of the Task API and its reference by some example [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figures from the paper (2 more)
Figure 3
Figure 3. Figure 3: Example dialogue for the RdfSyntaxFixList task with a missing dot in [PITH_FULL_IMAGE:figures/full_fig_p010_3.png]
Figure 4
Figure 4. Figure 4: Examples of capability compasses generated with the framework on the [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 15 canonical work pages

  1. [1]

    Abdin, M., et al.: Phi-3 technical report: A highly capable language model locally on your phone (2024),https://arxiv.org/abs/2404.14219

    arXiv 2024

  2. [2]

    In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Bai, G., Liu, J., Bu, X., He, Y., Liu, J., Zhou, Z., Lin, Z., Su, W., Ge, T., Zheng, B., Ouyang, W.: Mt-bench-101: A fine-grained benchmark for evaluat- ing large language models in multi-turn dialogues. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). p. 7421–7454. Association for Comput...

    doi:10.18653/v1/2024.acl-long.401 2024

  3. [3]

    In: Holze, J., Tramp, S., Martin, M., Auer, S., Usbeck, R., Krdzavac, N

    Brei, F., Frey, J., Meyer, L.P.: Leveraging small language models for text2sparqltasks to improve the resilience of ai assistance. In: Holze, J., Tramp, S., Martin, M., Auer, S., Usbeck, R., Krdzavac, N. (eds.) Proceedings of the Third International Workshop on Linked Data-driven Resilience Research 2024 (D2R2’24), colocated with ESWC 2024. CEUR Workshop ...

    work page 2024

  4. [4]

    it - Information Technology (Mar 2025).https://doi.org/10

    Brei, F., Meyer, L.P., Martin, M.: Queryfy: from knowledge graphs to questions using open large language models: Enabling finetuning by question generation on given knowledge. it - Information Technology (Mar 2025).https://doi.org/10. 1515/itit-2024-0079

    work page 2025

  5. [5]

    In: Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., Berkenkamp, F

    Chiang, W.L., Zheng, L., Sheng, Y., Angelopoulos, A.N., Li, T., Li, D., Zhu, B., Zhang, H., Jordan, M., Gonzalez, J.E., Stoica, I.: Chatbot arena: An open plat- form for evaluating LLMs by human preference. In: Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., Berkenkamp, F. (eds.) Proceedings of the 41st International Confe...

    work page 2024

  6. [6]

    In: Proceedings of the 18th International Semantic Web Conference (ISWC)

    Dubey, M., Banerjee, D., Abdelkawi, A., Lehmann, J.: Lc-quad 2.0: A large dataset for complex question answering over wikidata and dbpedia. In: Proceedings of the 18th International Semantic Web Conference (ISWC). Springer (2019). https: //doi.org/10.1007/978-3-030-30796-7_5

    doi:10.1007/978-3-030-30796-7_5 2019

  7. [7]

    https://huggingface.co/spaces/open-llm-leaderboard/open_llm_ leaderboard (2024)

    Fourrier, C., Habib, N., Lozovskaya, A., Szafer, K., Wolf, T.: Open llm leader- board v2. https://huggingface.co/spaces/open-llm-leaderboard/open_llm_ leaderboard (2024)

    work page 2024

  8. [8]

    Frey, J., Meyer, L.P., Arndt, N., Brei, F., Bulert, K.: Benchmarking the abilities of large language models for RDF knowledge graph creation and comprehension: How well do llms speak turtle? In: Alam, M., Cochez, M. (eds.) Proceedings of the Workshop on Deep Learning for Knowledge Graphs (DL4KG 2023) co-located with the 21th International Semantic Web Con...

    work page 2023

Show all 30 references
  1. [9]

    In: The Semantic Web: ESWC 2024 Satellite Events

    Frey, J., Meyer, L.P., Brei, F., Gruender, S., Martin, M.: Assessing the evolution of LLM capabilities for knowledge graph engineering in 2023. In: The Semantic Web: ESWC 2024 Satellite Events. pp. 51–60. Springer Nature Switzerland (2025). https://doi.org/10.1007/978-3-031-78952-6_5

    2025 doi

  2. [10]

    Grattafiori, A., et al.: The llama 3 herd of models (2024),https://arxiv.org/ abs/2407.21783

    2024 arXiv

  3. [11]

    Guo, D., Zhu, Q., Yang, D., Xie, Z., Dong, K., Zhang, W., Chen, G., Bi, X., Wu, Y., Li, Y.K., Luo, F., Xiong, Y., Liang, W.: Deepseek-coder: When the large language model meets programming – the rise of code intelligence (2024),https: //arxiv.org/abs/2401.14196

    2024 arXiv

  4. [12]

    CONCLUSION AND FUTURE WORK 17

  5. [13]

    In: Fifth International Workshop on Knowledge Graph Construction @ ESWC2024

    Hofer, M., Frey, J., Rahm, E.: Towards self-configuring knowledge graph construc- tion pipelines using llms - a case study with rml. In: Fifth International Workshop on Knowledge Graph Construction @ ESWC2024. CEUR Workshop Proceedings, vol. 3718. CEUR-WS.org (2024),https://ce...

    2024

  6. [14]

    Huang, S., Cheng, T., Liu, J.K., Hao, J., Song, L., Xu, Y., Yang, J., Liu, J.H., Zhang, C., Chai, L., Yuan, R., Zhang, Z., Fu, J., Liu, Q., Zhang, G., Wang, Z., Qi, Y., Xu, Y., Chu, W.: Opencoder: The open cookbook for top-tier code large language models (2024),https://arxiv.o...

    2024 arXiv

  7. [15]

    Hui, B., et al.: Qwen2.5-coder technical report (2024),https://arxiv.org/abs/ 2409.12186

    2024 arXiv

  8. [16]

    In: International Conference on Semantic Systems

    Kovriguina, L., Teucher, R., Radyush, D., Mouromtsev, D.: Sparqlgen: One-shot prompt-based approach for sparql query generation. In: International Conference on Semantic Systems. CEUR Workshop Proceedings, vol. 3526. CEUR-WS.org (2023), https://ceur-ws.org/Vol-3526/paper-08.pdf

    2023

  9. [17]

    In: Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles (2023)

    Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C.H., Gonzalez, J.E., Zhang, H., Stoica, I.: Efficient memory management for large language model serv- ing with PagedAttention. In: Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles (2023)

    2023

  10. [18]

    Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., Newman, B., Yuan, B., Yan, B., Zhang, C., Cosgrove, C., Manning, C.D., Ré, C., Acosta-Navas, D., Hudson, D.A., Zelikman, E., Durmus, E., Ladhak, F., Rong, F., ...

    2023 arXiv

  11. [19]

    Advances in Neural Information Processing Systems36 (2024)

    Liu, J., Xia, C.S., Wang, Y., Zhang, L.: Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems36 (2024)

    2024

  12. [20]

    In: Vakaj, E., Iranmanesh, S., Stamartina, R., Mihindukula- sooriya, N., Tiwari, S., Ortiz-Rodríguez, F., Mcgranaghan, R

    Meyer, L.P., Frey, J., Brei, F., Arndt, N.: Assessing SPARQL capabilities of large language models. In: Vakaj, E., Iranmanesh, S., Stamartina, R., Mihindukula- sooriya, N., Tiwari, S., Ortiz-Rodríguez, F., Mcgranaghan, R. (eds.) Proceedings of the 3rd International Workshop on...

    2024

  13. [21]

    In: Keshan, N., Neumaier, S., Gentile, A.L., Vahdati, S

    Meyer, L.P., Frey, J., Junghanns, K., Brei, F., Bulert, K., Gründer-Fahrer, S., Mar- tin, M.: Developing a scalable benchmark for assessing large language models in knowledge graph engineering. In: Keshan, N., Neumaier, S., Gentile, A.L., Vahdati, S. (eds.) Proceedings of the ...

    2023

  14. [22]

    In: Zinke-Wehlmann, C., Friedrich, J

    Meyer, L.P., Stadler, C., Frey, J., Radtke, N., Junghanns, K., Meissner, R., Dziwis, G., Bulert, K., Martin, M.: LLM-assisted knowledge graph engineering: Experi- ments with ChatGPT. In: Zinke-Wehlmann, C., Friedrich, J. (eds.) First Work- ing Conference on Artificial Intellig...

    2023 doi

  15. [23]

    In: Payne, T.R., Presutti, V., Qi, G., Poveda-Villalón, M., Stoilos, G., Hollink, L., Kaoudi, Z., Cheng, G., Li, J

    Mihindukulasooriya, N., Tiwari, S., Enguix, C.F., Lata, K.: Text2KGBench: A benchmark for ontology-driven knowledge graph generation from text. In: Payne, T.R., Presutti, V., Qi, G., Poveda-Villalón, M., Stoilos, G., Hollink, L., Kaoudi, Z., Cheng, G., Li, J. (eds.) The Semant...

    2023 doi

  16. [24]

    IEEE Transactions on Knowledge and Data Engineering (TKDE) (2024)

    Pan, S., Luo, L., Wang, Y., Chen, C., Wang, J., Wu, X.: Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering (TKDE) (2024). https://doi.org/10.1109/TKDE.2024. 3352100

    2024 doi

  17. [25]

    Shorten, C., Pierse, C., Smith, T.B., Cardenas, E., Sharma, A., Trengrove, J., van Luijt, B.: Structuredrag: Json response formatting with large language models (2024), https://arxiv.org/abs/2408.11061

    2024 arXiv

  18. [26]

    Transactions on Machine Learning Research (2023)

    Srivastava, A., et al.: Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on Machine Learning Research (2023)

    2023

  19. [27]

    Seman- tic Web10(2), 293–304 (Jan 2019).https://doi.org/10.3233/sw-180312

    Usbeck, R., Röder, M., Hoffmann, M., Conrads, F., Huthmann, J., Ngonga-Ngomo, A.C., Demmler, C., Unger, C.: Benchmarking question answering systems. Seman- tic Web10(2), 293–304 (Jan 2019).https://doi.org/10.3233/sw-180312

    2019 doi

  20. [28]

    Yang, A., et al.: Qwen2 technical report (2024),https://arxiv.org/abs/2407. 10671

    2024

  21. [29]

    In: SEMANTiCS 2024: Knowledge Graphs in the Age of Language Models and Neuro-Symbolic AI

    Zahera, H.M., Ali, M., Sherif, M.A., Moussallem, D., Ngonga Ngomo, A.C.: Generating sparql from natural language using chain-of-thoughts prompting. In: SEMANTiCS 2024: Knowledge Graphs in the Age of Language Models and Neuro-Symbolic AI. Amsterdam, Netherlands (2024).https://d...

    2024

  22. [30]

    World Wide Web27(5) (Aug 2024)

    Zhu, Y., Wang, X., Chen, J., Qiao, S., Ou, Y., Yao, Y., Deng, S., Chen, H., Zhang, N.: Llms for knowledge graph construction and reasoning: recent capabilities and future opportunities. World Wide Web27(5) (Aug 2024). https://doi.org/10. 1007/s11280-024-01297-w Metadata for th...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.