Pith. sign in

REVIEW 3 major objections 6 minor 44 references

The benefits of query-based KGQA systems for complex and temporal questions in LLM era

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A modular KGQA pipeline that turns questions into SPARQL queries beats direct GPT-4o answering on multi-hop and temporal Wikidata benchmarks.

desk verdict A solid multi-stage Wikidata KGQA system with a genuinely useful rejection study, but the headline direct-vs-pipeline comparison is scored on mismatched metrics and needs fixing. read the letter →

arxiv 2507.11954 v1 pith:RZBNPH46 submitted 2025-07-16 cs.CL cs.LG

Artem Alekseev , Mikhail Chaichuk , Miron Butko , Alexander Panchenko , Elena Tutubalina , Oleg Somov This is my paper · ORCID
classification cs.CLcs.LG
keywords questionansweringknowledgegraphsSPARQLWikidataentitylinkingtemporalreasoningmulti-hopchain-of-thought
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a query-based KGQA system, built from small fine-tuned language models and explicit retrieval stages, outperforms direct large-language-model question answering on benchmarks dominated by multi-hop and temporal questions. On LC-QuAD 2.0 and PAT-Questions, the full pipeline achieves higher F1 than GPT-4o prompted to answer directly, and it also beats a few-shot GPT-4o text-to-SPARQL setup. The authors interpret this as evidence that structured, modular pipelines remain a viable path for complex structured QA, not merely a fallback when proprietary APIs are unavailable. The paper additionally introduces a retrieval-plus-chain-of-thought entity linking and predicate matching method, and a rejection mechanism that filters invalid queries before they reach the user.

What carries the argument

The load-bearing components are: a BM25 index over Wikidata entities and predicates that proposes candidates; a chain-of-thought disambiguation step that asks a reasoning LLM to select the correct entity or predicate from the candidates; a fine-tuned Qwen2.5 text-to-SPARQL generator that turns the question plus linked candidates into an executable query; and two rejection filters—an entity-to-predicate compatibility check using the knowledge graph's actual relation structure, and execution-based filtering that drops queries that error out or return empty. The central object is the full pipeline treated as one system from natural language to entity-set answers, with the text-to-SPARQL model as the core and the linking stages as guardrails.

What would settle it

Run GPT-4o direct QA on LC-QuAD 2.0 and PAT, map each generated answer text to Wikidata entity IDs, and compute entity-set F1 using the same gold labels and scoring function as the pipeline; if the direct-model F1 rises to or above the reported pipeline values, the central advantage claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that an explicit pipeline—BM25 candidate retrieval, LLM-based disambiguation of entities and predicates, a fine-tuned text-to-SPARQL model, and ontology-aware filtering—achieves strong results on complex Wikidata QA, and that its advantage is largest precisely on datasets requiring multiple hops or time-sensitive answers. The best reported setup (Qwen-7B with ReFinED) reaches F1 of 31.0 on LC-QuAD 2.0 and 39.0 on PAT, versus 21.1 and 18.3 for GPT-4o direct QA, under the paper's evaluation protocols. The paper also shows that the text-to-SPARQL component generalizes across datasets with only a modest drop, and that combining entity-to-predicate compatibility checks with query execution filtering rejects a large share of incorrect query generations.

Load-bearing premise

The comparison with GPT-4o direct QA assumes that exact-match text scoring is on the same footing as entity-set F1 for the pipeline; if direct answers were first mapped to Wikidata entity IDs and scored the same way, the reported lead on LC-QuAD and PAT could shrink or disappear.

Editorial extensions

If this is right

  • On datasets dominated by multi-hop and temporal questions, query-based KGQA composed of small fine-tuned models can outperform direct proprietary LLM answering, which is the paper's headline claim from Table 4.
  • The proposed RetReason entity and predicate linking via BM25 plus chain-of-thought is competitive with a strong fine-tuned linker like ReFinED and can be used through an LLM API without additional training, per Tables 2 and 3.
  • The rejection mechanism combining ontology-based entity-predicate filtering with query execution detects substantially more incorrect queries than LLM self-rejection alone, as reported in Table 6.
  • Cross-dataset validation shows the text-to-SPARQL model generalizes to unseen datasets with a modest performance gap relative to in-dataset fine-tuning, as shown in Figure 3 and Table 5.
  • Error propagation across successive components is acknowledged as a limiting factor, yet system-level gains on LC-QuAD and PAT remain the paper's main evidence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reported advantage over GPT-4o is robust, an even stronger test would be to map direct GPT-4o answers to Wikidata entity IDs and score them with the same entity-set F1 used for the pipeline; any shrinkage of the gap would indicate part of the lead is an artifact of scoring protocol rather than true QA ability.
  • The rejection mechanism could be repurposed as a confidence signal for deciding when to fall back to direct LLM answering, yielding a hybrid that keeps the pipeline's precision on complex questions and the LLM's strength on simple ones.
  • The BM25-plus-chain-of-thought retrieval pattern may transfer to other structured generation tasks, such as text-to-SQL over schemas with many relations, where candidate generation followed by reasoning-based selection has the same shape.
  • For temporal questions in PAT, a future test would re-run the pipeline on updated versions of the benchmark to see whether the auto-updating answers remain correct without retraining.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a multi-stage pipeline for Wikidata knowledge graph question answering (KGQA). The pipeline consists of entity linking (ReFinED or the proposed RetReason BM25+CoT approach), predicate matching (same RetReason), a fine-tuned Qwen2.5 text-to-SPARQL model (0.5B–7B), and an entity-predicate compatibility filter. The system is evaluated on LC-QuAD 2.0, RuBQ 2.0, QALD-10, and PAT-Questions, against GPT-4o direct QA, few-shot GPT-4o with pipeline components for SPARQL generation, and published baselines. The authors report that the pipeline significantly outperforms direct GPT-4o and baselines on LC-QuAD 2.0 and PAT, which are dominated by multi-hop and temporal questions. They also present a cross-dataset generalization study and a rejection study for filtering invalid queries.

Significance. If the results hold, the paper provides a valuable empirical data point that structured pipelines based on small fine-tuned models can outperform proprietary LLM direct answering on complex and temporal Wikidata QA, with public code and reproducibility materials. The use of four standard benchmarks and the inclusion of a few-shot GPT-4o SPARQL baseline are strengths. The few-shot GPT-4o + ReFineD baseline is scored with the same entity-set F1 as the pipeline, and the pipeline wins on LC-QuAD 2.0 and PAT by substantial margins. However, the headline comparison against GPT-4o direct QA is currently not commensurate because of different scoring schemes, and the per-dataset BM25 tuning raises a leakage risk; both issues need to be addressed before the central claim can be accepted as stated.

major comments (3)
  1. [§5, Table 4] The comparison between GPT-4o direct QA and the pipeline is scored on mismatched metrics. Direct answers are evaluated by exact text match against gold labels, while the pipeline and the few-shot GPT-4o + ReFineD row are evaluated with F1 over executed answer entity sets. Free-form text answers that mention the correct entity via an alias or an indirect phrasing are counted as wrong, while the pipeline receives partial credit. The paper does not describe any normalization of GPT-4o outputs to Wikidata Q-IDs. This is a load-bearing issue because the abstract and Section 5 use this comparison to claim that the pipeline 'significantly outperforms' ChatGPT on LC-QuAD 2.0 and PAT. Please re-score GPT-4o direct answers under the same entity-set F1 metric (e.g., by mapping predicted text to Wikidata entities before scoring) or remove/qualify the claim. Note that the few-shot GPT-4o + ReFineD baseline already provides a fair entity-set comparison and the pipeline still wins on LC-QuAD 2.0 (31.0 vs 24.4) and PAT (39.0 vs 17.7), so the central finding is likely recoverable; the current presentation, however, asserts it on the basis of an invalid comparison.
  2. [§4.1, §6, Appendix B] The BM25 hyperparameters (k1, b) and the candidate retrieval depth are reported as optimized 'for each dataset' in Appendix B, but the manuscript does not state on which split this optimization is performed. If the test split is used to select k1, b, and the entity-index popularity threshold, then all reported end-to-end numbers are optimistically biased, and the cross-dataset generalization study in Section 6 is compromised because the target test set would have influenced retrieval parameters. Please state explicitly whether these values are chosen on training/validation splits only; if they are, provide the split definitions and the selected values for each dataset. If they are chosen on test, the experiments should be re-run with a fixed configuration or a proper nested validation.
  3. [§6, Table 5] The generalization experiment shows a large drop on PAT: the full system trained on the other three datasets reaches 12.5 F1, compared with 38.9 F1 when fine-tuned on PAT (Qwen-1.5B + ReFineD in Table 4), a gap of 26.4 points. The text claims that 'the performance gap remains relatively small' and that in certain cases the model exhibits 'comparable generalization,' but the PAT result is neither small nor comparable. Since PAT is one of the two datasets highlighted in the headline claim, this discrepancy needs to be discussed explicitly; the abstract's wording that the system is 'robust' across multi-hop and temporal datasets is stronger than the evidence in Table 5.
minor comments (6)
  1. [§5, Table 4] The row labeled 'Few-shot GPT-4o + ReFineD' does not match the description in the text, which states that the prompt includes entities from ReFineD and predicates from RetReason. Please rename the row to reflect the actual configuration.
  2. [Throughout] The paper alternates between 'ChatGPT' and 'GPT-4o' for the same model family (e.g., abstract vs Section 5). Use one consistent name, and specify the exact model version and access date for reproducibility.
  3. [§3, Eq. (1)] Acc@1 is defined as a full set match between the predicted and gold entity sets; this is not the standard top-1 accuracy. Consider renaming it to 'exact set match' or clarifying the definition to avoid confusion.
  4. [§7, Table 6] The rejection study reports the 'proportion of identified incorrect SPARQL queries' without defining how the set of incorrect queries is determined (e.g., queries whose execution differs from the gold answer set) or what the denominator is. Please provide the evaluation protocol and report rejection precision and recall as well.
  5. [§5, Table 4] For the published baselines (SPINACH, Konstruktor, Text2Graph, Review-Then-Refine), please specify whether the scores are copied from the original papers or re-computed with the same evaluation script; different protocols for entity-set comparison could affect the comparison.
  6. [§4.1, §6] Minor typographical issues include 'T able' in Table 1, 'P A T' in Table 1, 'an multi-stage' in Section 2, and 'ReFined' in Section 5. The appendix cross-references are otherwise fine.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical system evaluated on held-out splits of external benchmarks, with no fitted quantity renamed as a prediction.

full rationale

The paper's central claims are empirical: a multi-stage pipeline (entity linking, predicate matching, text-to-SPARQL generation) is trained on supervised splits of four external benchmarks and evaluated on held-out test splits, with end-to-end F1/Acc@1 computed by comparing predicted answer entity IDs to gold entity sets using the explicit set-intersection equations in Section 3. No quantity is derived by fitting the answer it aims to predict. The only per-dataset tuning is BM25 hyperparameters (k1, b) for candidate retrieval (Appendix B), which is a potential optimization/leakage concern rather than circular reasoning, and it affects a component rather than the final correctness criterion. The direct-QA comparison with GPT-4o uses exact text matching while the pipeline is scored by entity-set F1; this is a metric-comparability weakness, not an equivalence-by-construction circularity. Self-citations ([9], [25], [35], [36]) appear as baselines and prior work, but the headline comparison also rests on external baselines (SPINACH, SPARQL-QA, Review-Then-Refine) and held-out benchmark splits, so the self-citations are not load-bearing in the sense of uniquely forcing the result. No circular step can be exhibited from the paper's own equations or definitions.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim depends on a handful of tuned retrieval parameters (BM25 k1/b, retrieval depth, index filter, noise augmentation) and on the correctness of dataset gold queries. No new scientific entities are introduced. The most consequential assumption is the comparability of the two evaluation metrics used for the ChatGPT comparison.

free parameters (5)
  • BM25 k1 (entity and predicate indexes) = per dataset, e.g., QALD-10: 2.95 entity / 5.18 predicate
    Optimized per dataset favoring recall (Appendix B, Table 7). No validation split is reported, so tuning may leak test-set information into retrieval.
  • BM25 b (entity and predicate indexes) = per dataset, e.g., QALD-10: 0.2 entity / 0.01 predicate
    Same tuning concern as k1; values differ by dataset in Table 7.
  • Candidate retrieval depth = top-10 and top-100
    Chosen by hand (Section 4.1); both reported, no ablation of other depths.
  • Entity index popularity filter = entities with fewer than 10 predicates removed
    Ad hoc threshold to keep index at 15M entities (Section 4.1); affects recall for rare entities.
  • Fine-tuning noise entities/relations = top-5 BM25-similar incorrect items added to gold set
    Augmentation choice in Appendix C to emulate noisy inference; value 5 chosen without ablation.
assumptions (4)
  • domain assumption Gold SPARQL queries and entity annotations in all four datasets are correct and complete.
    The system is trained and evaluated against these gold queries; any errors propagate into reported F1 (Section 3).
  • domain assumption All answerable questions can be answered by executing a SPARQL query over Wikidata.
    Central to query-based KGQA; temporal data in Wikidata is assumed sufficient for PAT answers (Section 3).
  • domain assumption BM25 over entity/predicate labels and descriptions retrieves candidates containing the gold item with high recall.
    Used in RetReason (Section 4.1). Recall@100 reaches 0.88-1.0 but is not perfect, so failures cascade.
  • ad hoc to paper Exact-match scoring of direct QA text is a valid comparison basis with entity-set F1.
    The headline claim rests on comparing these two metrics; this is a contested assumption (Section 5 vs Section 3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of The benefits of query-based KGQA systems for complex and temporal questions in LLM era." pith.science (2026). https://pith.science/paper/RZBNPH46

@misc{pith2026250711954,
  author       = {Pith},
  title        = {Pith review of: The benefits of query-based KGQA systems for complex and temporal questions in LLM era},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RZBNPH46}},
  note         = {Machine review of arXiv:2507.11954}
}
read the original abstract

Large language models excel in question-answering (QA) yet still struggle with multi-hop reasoning and temporal questions. Query-based knowledge graph QA (KGQA) offers a modular alternative by generating executable queries instead of direct answers. We explore multi-stage query-based framework for WikiData QA, proposing multi-stage approach that enhances performance on challenging multi-hop and temporal benchmarks. Through generalization and rejection studies, we evaluate robustness across multi-hop and temporal QA datasets. Additionally, we introduce a novel entity linking and predicate matching method using CoT reasoning. Our results demonstrate the potential of query-based multi-stage KGQA framework for improving multi-hop and temporal QA with small language models. Code and data: https://github.com/ar2max/NLDB-KGQA-System

Figures

Figures reproduced from arXiv: 2507.11954 by the authors.

Figure 1
Figure 1. The proposed KGQA system processes natural language question and trans￾lates it into SPARQL for retrieving information from Wikidata. It follows a struc￾tured pipeline where entities and predicates are first retrieved and disambiguated using Wikidata indexes. A filtering step then ensures correct entity-predicate mappings be￾fore passing it to a text-to-SPARQL model. The generated query undergoes execution match fil… view at source ↗
Figure 2
Figure 2. The models comparison across different sizes (0.5B, 1.5B, 3B, 7B) of Qwen2.5 models and training approaches (LoRA & SFT) with F1 score. entity disambiguation for CoT reasoning is provided in the Appendix A. The final metrics for predicate disambiguation presented in [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. The generalization evaluation across different sizes (0.5B, 1.5B, 3B, 7B) of Qwen2.5 models and training approaches (LoRA & SFT) with F1 score [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 39 canonical work pages

  1. [1]

    Allen-Zhu, Z., Li, Y.: Physics of language models: Part 3.1, knowledge storage and extraction (2023)

    work page 2023

  2. [2]

    In: international semantic web conference

    Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., Ives, Z.: Dbpedia: A nucleus for a web of open data. In: international semantic web conference. pp. 722–735. Springer (2007)

    work page 2007

  3. [3]

    In: Proceed- ings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

    Ayoola, T., Tyagi, S., Fisher, J., Christodoulopoulos, C., Pierleoni, A.: ReFinED: An efficient zero-shot-capable approach to end-to-end entity linking. In: Proceed- ings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pp. 209–220 (2022) 12 A. Alekseev et al

    work page 2022

  4. [4]

    In: International Conference on Advanced Infor- mation Systems Engineering (2024)

    Baazouzi, W., Kachroudi, M., Faiz, S.: Sweeping knowledge graphs with sparql queries to palliate q/a problems. In: International Conference on Advanced Infor- mation Systems Engineering (2024)

    work page 2024

  5. [5]

    In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

    Banerjee, D., Nair, P.A., Kaur, J.N., Usbeck, R., Biemann, C.: Modern baselines for sparql semantic parsing. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. pp. 2260–2265 (2022)

    work page 2022

  6. [6]

    a is b” fail to learn “b is a

    Berglund, L., Tong, M., Kaufmann, M., Balesni, M., Stickland, A.C., Korbak, T., Evans, O.: The reversal curse: LLMs trained on “a is b” fail to learn “b is a”. In: The Twelfth International Conference on Learning Representations (2024)

    work page 2024

  7. [7]

    In: Proceedings of the 7th Natural Language Interfaces for the Web of Data (NLI- WoD) co-located with the 19th European Semantic Web Conference, Hersonissos, Greece

    Borroto, M., Ricca, F., Cuteri, B., Barbara, V.: Sparql-qa enters the qald challenge. In: Proceedings of the 7th Natural Language Interfaces for the Web of Data (NLI- WoD) co-located with the 19th European Semantic Web Conference, Hersonissos, Greece. vol. 3196, pp. 25–31 (2022)

    work page 2022

  8. [8]

    Brei, F., Frey, J., Meyer, L.P.: Leveraging small language models for text2sparql tasks to improve the resilience of ai assistance (2024)

    work page 2024

Show all 44 references
  1. [9]

    In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop

    Chekalina, V., Razzhigaev, A., Sayapin, A., Frolov, E., Panchenko, A.: Meker: Memory efficient knowledge embedding representation for link prediction and ques- tion answering. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student ...

    2022

  2. [10]

    arXiv:2412.15101 (2024)

    Chen, X., Hu, X., Tang, N.: Then-refine: A dynamic framework for multi-hop question answering with temporal adaptability. arXiv:2412.15101 (2024)

    2024 arXiv

  3. [11]

    IEEE Transactions on Knowledge and Data Engineering (2024)

    Chong, Y., Lee, C., Muhd-Yassin, S., Lim, K.: Transkgqa: Enhanced knowl- edge graph question answering with sentence transformers. IEEE Transactions on Knowledge and Data Engineering (2024)

    2024

  4. [12]

    In: The Semantic Web– ISWC2019:18thInternationalSemanticWebConference,Auckland,NewZealand, October 26–30, 2019, Proceedings, Part II 18

    Dubey, M., Banerjee, D., Abdelkawi, A., Lehmann, J.: Lc-quad 2.0: A large dataset for complex question answering over wikidata and dbpedia. In: The Semantic Web– ISWC2019:18thInternationalSemanticWebConference,Auckland,NewZealand, October 26–30, 2019, Proceedings, Part II 18. ...

    2019

  5. [13]

    Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, H.: Retrieval-augmented generation for large language models: A survey

  6. [14]

    Procesamiento del lenguaje natural 73, 271–281 (2024)

    Ghajari, A., Ros, S., Pérez, Á.: Querying the depths: Unveiling the strengths and struggles of large language models in sparql generation. Procesamiento del lenguaje natural 73, 271–281 (2024)

    2024

  7. [15]

    In: Proceedings of the Web Conference 2021

    Gu, Y., Kase, S., Vanni, M., Sadler, B., Liang, P., Yan, X., Su, Y.: Beyond iid: three levels of generalization for question answering on knowledge bases. In: Proceedings of the Web Conference 2021. pp. 3477–3488 (2021)

    2021

  8. [16]

    World Wide Web26(5), 2855–2886 (2023)

    Hu, N., Wu, Y., Qi, G., Min, D., Chen, J., Pan, J.Z., Ali, Z.: An empirical study of pre-trained language models in simple knowledge graph question answering. World Wide Web26(5), 2855–2886 (2023)

    2023

  9. [17]

    Jiang, L., Usbeck, R.: Knowledge graph question answering datasets and their generalizability: Are they enough for future research? In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Informa- tion Retrieval. pp. 3209–3218 (2022)

    2022

  10. [18]

    Transactions of the Association for Computational Linguistics9, 962–977 (2021)

    Jiang, Z., Araki, J., Ding, H., Neubig, G.: How can we know when language models know? on the calibration of language models for question answering. Transactions of the Association for Computational Linguistics9, 962–977 (2021)

    2021

  11. [19]

    In: Webber, B., Cohn, T., He, Y., Liu, Y

    Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., Yih, W.t.: Dense passage retrieval for open-domain question answering. In: Webber, B., Cohn, T., He, Y., Liu, Y. (eds.) Proceedings of the EMNLP. pp. 6769–6781. Online (2020) Query-based KGQA systems fo...

    2020

  12. [20]

    Journal of Physics: Conference Series1740, 012050 (01 2021)

    Kostenetskiy, P., Chulkevich, R., Kozyrev, V.: Hpc resources of the higher school of economics. Journal of Physics: Conference Series1740, 012050 (01 2021)

    2021

  13. [21]

    Advances in neural information processing systems 33, 9459–9474 (2020)

    Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.t., Rocktäschel, T., et al.: Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems 33, 9459–9474 (2020)

    2020

  14. [22]

    In: Findings of the Association for Computational Linguistics: EMNLP 2024

    Li, Y., Song, D., Zhou, C., Tian, Y., Wang, H., Yang, Z., Zhang, S.: A framework of knowledge graph-enhanced large language model based on question decomposition and atomic retrieval. In: Findings of the Association for Computational Linguistics: EMNLP 2024. pp. 11472–11485 (2024)

    2024

  15. [23]

    In: Findings of the Association for Computational Linguistics: EMNLP 2024

    Liu, S., Semnani, S., Triedman, H., Xu, J., Zhao, I., Lam, M.: Spinach: Sparql- based information navigation for challenging real-world questions. In: Findings of the Association for Computational Linguistics: EMNLP 2024. pp. 15977–16001 (2024)

    2024

  16. [24]

    In: Proceedings of the 2024 ACM International Conference on Web Search and Data Mining (2024)

    Longwell, J., Alavi, M.A.A., Zarrinkalam, F.: Triple augmented generative lan- guage models for sparql query generation from natural language questions. In: Proceedings of the 2024 ACM International Conference on Web Search and Data Mining (2024)

    2024

  17. [25]

    In: International Confer- ence on Applications of Natural Language to Information Systems

    Lysyuk, M., Salnikov, M., Braslavski, P., Panchenko, A.: Konstruktor: A strong baseline for simple knowledge graph question answering. In: International Confer- ence on Applications of Natural Language to Information Systems. pp. 107–118. Springer (2024)

    2024

  18. [26]

    In: Findings of the Association for Computational Linguistics ACL 2024

    Meem, J., Rashid, M., Dong, Y., Hristidis, V.: Pat-questions: A self-updating benchmark for present-anchored temporal question-answering. In: Findings of the Association for Computational Linguistics ACL 2024. pp. 13129–13148 (2024)

    2024

  19. [27]

    In: Walker, M., Ji, H., Stent, A

    Mohammed, S., Shi, P., Lin, J.: Strong baselines for simple question answering over knowledge graphs with and without neural networks. In: Walker, M., Ji, H., Stent, A. (eds.) Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational...

    2018

  20. [28]

    arxiv 2022

    Muennighoff, N.: Sgpt: Gpt sentence embeddings for semantic search. arxiv 2022

    2022

  21. [29]

    Applied Sciences14(4), 1521 (2024)

    Qi, J., Su, C., Guo, Z., Wu, L., Shen, Z., Fu, L., Wang, X., Zhou, C.: Enhancing sparql query generation for knowledge base question answering systems by learning to correct triplets. Applied Sciences14(4), 1521 (2024)

    2024

  22. [30]

    Rangel, J.C., de Farias, T.M., Sima, A.C., Kobayashi, N.: Sparql generation: an analysis on fine-tuning openllama for question answering over a life science knowl- edge graph (2024)

    2024

  23. [31]

    In: The Semantic Web: 18th International Con- ference, ESWC 2021, Virtual Event, June 6–10, 2021, Proceedings 18

    Rybin, I., Korablinov, V., Efimov, P., Braslavski, P.: Rubq 2.0: An innovated rus- sian question answering dataset. In: The Semantic Web: 18th International Con- ference, ESWC 2021, Virtual Event, June 6–10, 2021, Proceedings 18. pp. 532–547. Springer (2021)

    2021

  24. [32]

    In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management

    Sakor, A., Singh, K., Patel, A., Vidal, M.E.: Falcon 2.0: An entity and relation link- ing tool over wikidata. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. p. 3141–3148. CIKM ’20, Association for Computing Machinery, New York,...

    2020

  25. [33]

    Schwabe, T., Siebel, L., Valach, P., Acosta, M.: Q-nl verifier: Leveraging synthetic data for robust knowledge graph question answering (2025)

    2025

  26. [34]

    Advances in Neural Information Processing Systems 36, 8634–8652 (2023) 14 A

    Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., Yao, S.: Reflexion: Lan- guage agents with verbal reinforcement learning. Advances in Neural Information Processing Systems 36, 8634–8652 (2023) 14 A. Alekseev et al

    2023

  27. [35]

    In: Proceedings of the 1st GenBench Workshop on (Benchmarking) Generalisation in NLP

    Somov, O., Tutubalina, E.: Shifted PAUQ: Distribution shift in text-to-SQL. In: Proceedings of the 1st GenBench Workshop on (Benchmarking) Generalisation in NLP. pp. 214–220 (2023)

    2023

  28. [36]

    Proceedings of the AAAI Conference on Artificial Intelligence39(23), 25137–25145 (Apr 2025)

    Somov, O., Tutubalina, E.: Confidence estimation for error detection in text-to- sql systems. Proceedings of the AAAI Conference on Artificial Intelligence39(23), 25137–25145 (Apr 2025)

    2025

  29. [37]

    Electronics (2024)

    Sun, S., Hou, K., Li, J., Liu, Y.: Kg-egv: A framework for question answering with integrated knowledge graphs and large language models. Electronics (2024)

    2024

  30. [38]

    In: International Semantic Web Conference

    Tan, Y., Min, D., Li, Y., Li, W., Hu, N., Chen, Y., Qi, G.: Can chatgpt replace tra- ditional kbqa models? an in-depth analysis of the question answering performance of the gpt llm family. In: International Semantic Web Conference. pp. 348–367. Springer (2023)

    2023

  31. [39]

    arXiv preprint arXiv:2307.09288 (2023)

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bash- lykov, N., Batra, S., Bhargava, P., Bhosale, S., et al.: Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023)

    2023 arXiv

  32. [40]

    Semantic Web15(6), 2193–2207 (2024)

    Usbeck, R., Yan, X., Perevalov, A., Jiang, L., Schulz, J., Kraft, A., Möller, C., Huang, J., Reineke, J., Ngonga Ngomo, A.C., et al.: Qald-10–the 10th challenge on question answering over linked data: Shifting from dbpedia to wikidata as a kg for kgqa. Semantic Web15(6), 2193–...

    2024

  33. [41]

    Com- munications of the ACM57(10), 78–85 (2014)

    Vrandečić, D., Krötzsch, M.: Wikidata: a free collaborative knowledgebase. Com- munications of the ACM57(10), 78–85 (2014)

    2014

  34. [42]

    In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Xiong, G., Bao, J., Zhao, W.: Interactive-kbqa: Multi-turn interactions for knowl- edge base question answering with large language models. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 10561–10582 (2024)

    2024

  35. [43]

    In: The Eleventh International Conference on Learning Representations (2022)

    Yu, D., Zhang, S., Ng, P., Zhu, H., Li, A.H., Wang, J., Hu, Y., Wang, W.Y., Wang, Z., Xiang, B.: Decaf: Joint decoding of answers and logical forms for question answering over knowledge bases. In: The Eleventh International Conference on Learning Representations (2022)

    2022

  36. [44]

    disambiguation

    Zahera, H., Sherif, M., Moussallem, D.: Generating sparql from natural language using chain-of-thoughts prompting. Semantic Web (2024) A Entity Disambiguation and Predicate Matching with Reasoning The prompt and evaluation procedure is specified in the solution repository fold...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.