Pith. sign in

REVIEW 4 major objections 4 minor 79 references

The paper claims that SEAL, a two-stage agentic semantic parser with self-evolving memory, lets an LLM answer conversational knowledge-graph questions without retraining, reaching state-of-the-art unsupervised results on the SPICE benchmark

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 18:28 UTC pith:SJAXPESC

load-bearing objection The SOTA claim in the abstract doesn't survive the paper's own Table 2, but the underlying pipeline is a real, novel combination that deserves a careful revision rather than a desk reject. the 4 major comments →

arxiv 2512.04868 v2 pith:SJAXPESC submitted 2025-12-04 cs.CL cs.AI

SEAL: Self-Evolving Agentic Learning for Conversational Question Answering over Knowledge Graphs

Hao Wang , Jialun Zhong , Changcheng Wang , Zhujun Nie , Zheng Li , Shunyu Yao , Yanzeng Li , Xinchi Li This is my paper
classification cs.CL cs.AI
keywords conversational question answeringknowledge graphsemantic parsingS-expressionlarge language modelsself-evolving memoryagentic learningcoreference resolution
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

SEAL is a semantic-parsing framework for conversational question answering over knowledge graphs. It decomposes logical-form generation into two stages: first, an LLM writes a minimal S-expression core—the essential query structure—which an agent then calibrates against the knowledge graph; second, a question-type classifier selects a template that the LLM fills with the calibrated cores. The system also keeps local and global memories plus a reflection module, so every turn can update global memory with validated logical forms, letting the agent improve over a conversation without any retraining. On the SPICE benchmark, SEAL claims state-of-the-art performance for unsupervised methods, particularly in multi-hop reasoning, comparison, and aggregation, coming close to supervised models while issuing far fewer SPARQL queries. A sympathetic reader would take this as evidence that careful decomposition plus a self-evolving memory can substitute for labeled training data in complex KG-based dialogue.

Core claim

The central claim is that a decomposition strategy makes LLMs reliable semantic parsers for conversational KBQA. Rather than asking the LLM to emit a full S-expression end-to-end, SEAL has it extract a minimal core containing only JOIN, R, AND, VALUES, and IS_TRUE operations; an agent then corrects syntax and grounds each surface name to the most similar entity or relation (top-1 linking), retaining only variants whose execution returns non-empty results. The second stage predicts the question type, selects a template, and replaces placeholders with calibrated cores, functions, and constants to form an executable S-expression. The self-evolving loop—reflection plus local/global memory—writes

What carries the argument

The S-expression core is the central object: a minimal subset of S-expressions built from JOIN, R, AND, VALUES, and IS_TRUE, capturing the WHERE-clause structure of a SPARQL query. Agentic calibration is the mechanism that carries the argument: it parses and corrects the core's syntax, links each entity/relation mention to the single most similar KG element by cosine similarity of embeddings, and keeps only calibrated variants that execute to non-empty results. Template-based completion then predicts the question type and fills placeholders in a predefined template with the calibrated cores. Local and global memory with a reflection module form the self-evolving loop that stores validated lo

Load-bearing premise

The system's calibration and reflection treat a logical form as correct only if it executes to a non-empty result, so any question whose true answer is the empty set is assumed to be a parsing error and gets discarded.

What would settle it

On a hand-built set of valid zero-answer questions (e.g., 'Which countries have no rivers?' over Wikidata), run SEAL's calibration: if the correct empty-returning S-expression is systematically filtered out, the correctness signal is broken and the reported accuracy on non-empty-biased SPICE subsets is inflated. Alternatively, re-score SEAL on the official SPICE test set after removing all questions whose gold SPARQL returns an empty result, and check whether the accuracy gap to supervised models shrinks or vanishes.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If SEAL's design is correct, unsupervised conversational KBQA can match supervised performance on complex reasoning: SEAL's m-F1 on logical reasoning is 73.08 versus 46.85 for KB-Binder, approaching the supervised LLMGT's 89.61 while using no labeled data.
  • The two-stage decomposition plus single-candidate linking reduces SPARQL query counts compared to candidate-space approaches, because core-level subtasks shrink the exponential search base.
  • The self-evolving mechanism should yield growing accuracy as dialog turns accumulate; the paper reports SEAL's F1 rising from around 0.85 in early turns to 0.862 in the 9-12 turn range while the memory-ablated version lags, and a 56% advantage in the 60-80% cumulative-context interval.
  • Structural accuracy improves: structure overlap 54.8% versus 32.2% and parse success 88.1% versus 66.0% compared to KB-Binder, suggesting fewer syntactically invalid logical forms.
  • Accuracy on verification and count tasks also exceeds baselines (85.97 and 70.12 accuracy), suggesting the framework generalizes beyond the highly complex question types.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • One implication left implicit is that treating non-empty execution as the sole correctness signal biases calibration against legitimate empty-answer questions; the authors themselves acknowledge this in the conclusion, so reported numbers likely overstate precision on any dataset slice with zero-answer queries.
  • A testable extension would be to evaluate SEAL on a dataset with a controlled proportion of empty-answer questions (e.g., 'Which countries have no rivers?') and measure how often calibration discards the correct empty logical form; if many are dropped, the calibration rule would need to accept empty results as valid.
  • The single-candidate top-1 linking strategy trades recall for precision; the paper's own ablation shows removing entity-candidate generation affects precision, so making the candidate count adaptive to confidence could improve recall on ambiguous mentions without exploding query counts.
  • Since the template library is written for SPICE-style questions, the claimed state of the art may not transfer to other knowledge graphs or question distributions; the paper itself notes the S-expression-to-SPARQL transformation is constrained by specific syntactic patterns, so the self-evolving memory's value on out-of-distribution queries remains unknown.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes SEAL, a two-stage semantic parsing framework for conversational QA over knowledge graphs. Stage one extracts a minimal S-expression core and calibrates it against the KG; stage two uses question-type prediction and template completion to produce executable S-expressions. A self-evolving mechanism with local/global memory and reflection is claimed to improve performance over dialogue turns without retraining. Experiments on SPICE are reported, with the abstract and Section 5 claiming state-of-the-art results, especially in multi-hop reasoning, comparison, and aggregation.

Significance. The architectural decomposition and the memory/reflection loop are interesting and potentially useful for low-resource KBQA. However, the empirical claims are not supported by the paper's own tables: the headline SOTA statement contradicts the reported baseline numbers. The non-empty-result filter and the template library hand-built from the SPICE training corpus further undermine the validity of the evaluation as a measure of generalization. If the stated limitations were addressed with a re-designed evaluation, the framework could merit renewed attention, but the current evidence is not reliable.

major comments (4)
  1. [5.2.1 and Table 2] Section 5.2.1 claims 'in Logical Reasoning, SEAL obtains a m-F1 of 73.08, surpassing all baselines.' Table 2 explicitly reports LLMGT at 89.61 on that same row. The Abstract's claim of 'state-of-the-art performance, especially in multi-hop reasoning, comparison, and aggregation tasks' is similarly contradicted: in Comparative Reasoning, DCG GL reaches 79.86 vs SEAL's 41.06; in Quantitative Reasoning, DCG GL reaches 89.67 vs SEAL's 64.45. The aggregate AC (66.83) beats LLMGT (65.65), but the component-wise claims are false as stated.
  2. [4.2.2.2 and 6] Calibration retains 'candidate variants whose query executions return non-empty results,' and the reflection module (Sec 4.5) treats non-null execution as a validation signal. The Conclusion admits this 'may inadvertently revise valid empty queries, reducing precision.' This is a load-bearing limitation: if SPICE contains legitimate empty-answer questions, SEAL systematically discards correct logical forms, and the reported m-F1/AC are biased toward non-empty queries. The paper should report performance on empty-answer instances separately or use a filter that preserves semantically valid empty results.
  3. [4.3.2 and 5.1] The template library is 'grounded in a detailed analysis of the training corpus' (Sec 4.3.2), and the question taxonomy is 'following the annotation schema of the SPICE dataset' (Sec 5.1). Because both are hand-built from the same benchmark on which SEAL is evaluated, the method is partly tuned to the test distribution. This does not by itself invalidate the system, but the claimed generalization and 'self-evolving without retraining' behavior would require evaluation on a held-out benchmark or a template-source ablation. As it stands, the design choice is a correctness-risk concern for the generalization claim.
  4. [Table 4 and Table 5] The row 'w/o core extraction' in Table 4 (F1 41.34, AC 35.39, Overall 39.36) is numerically identical to 'SEALbase (few-shot)' in Table 5. If this is not a typo, the ablation does not isolate core extraction, and Section 5.5's conclusion about its 'greatest influence' is unsupported. Additionally, Section 5.6 states 'KB-Binder lacks data in the zero-shot setting,' while Table 5 reports KB-Binder (zero-shot) with F1 21.67, AC 15.96, Overall 19.76; the text and table are contradictory.
minor comments (4)
  1. [5.1] Type: 'marco-F1' should be 'macro-F1'; Section 4.2.2 title has 'Calibaration'; Section 5.6 title has 'Senario'.
  2. [Figure 4] The three panels lack confidence intervals and statistical tests. The early-turn F1 of 1.0 for both SEAL and the ablation suggests ceiling or trivial samples; the baselines for the ablation curves are not fully specified.
  3. [References] Reference [24] lacks a venue, and references [35] and [36] are the same paper (duplicate). Several other references are incomplete or have inconsistent formatting.
  4. [Appendix D] The paper notes an error in the SPICE dataset (missing FILTER components for equality) and that templates were designed to compensate. The number of affected questions is not quantified; this transparency is good but should be part of the main evaluation discussion.

Circularity Check

0 steps flagged

No circular derivation: SEAL's pipeline is self-contained against the SPICE benchmark; the flagged concerns are benchmark-tuning and correctness issues, not input-output circularity.

full rationale

Walking the derivation chain (Eq. 2 core generation, Eq. 4 cosine linking, Eq. 8 template-plan selection, and the memory/reflection loop), no predicted quantity is defined in terms of the target or fitted to it by construction. The LLM generates cores and instantiates templates; the templates are hand-built from the SPICE training corpus (Sec. 4.3.2: "The design of S-expression templates is grounded in a detailed analysis of the training corpus") and the taxonomy follows the SPICE annotation schema (Implementation Details). This weakens the 'requiring no labeled data' claim and makes the system benchmark-tuned, but it is not an equation-level reduction: the final S-expression still requires entity/relation linking, type prediction, and placeholder instantiation, and accuracy is measured against gold SPICE answers. The non-empty-result filter (Sec. 4.2.2.2: "The final set Calibrated_Core_j retains the candidate variants whose query executions return non-empty results") uses execution success as an internal validation signal and is acknowledged in the Conclusion to discard valid empty queries, a precision/recall bias rather than a self-fulfilling prediction; reported F1/AC are computed against gold S-expressions, not against the filter. The abstract's SOTA claim is internally contradicted by Table 2 (LLMGT 89.61 vs SEAL 73.08 on Logical Reasoning), but that is an empirical correctness issue, not circularity. No load-bearing self-citation or author-imported uniqueness theorem appears.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 3 invented entities

The central claims rest on several assumptions specific to the SPICE benchmark: lossless SPARQL-to-S-expression conversion, the sufficiency of top-1 embedding linking, the correctness of the non-empty-result filter, and the transferability of hand-built templates from training to test. The non-empty filter is the most consequential, as it directly shapes the reported accuracy by discarding empty-answer logical forms.

free parameters (5)
  • candidate variant retention count = 1 or 3 (unclear for main table)
    Section 4.2.2.2 keeps either the first non-empty variant or the top three; Section 5.1 explores both, but Table 2 does not state which is used.
  • linking top-k = 1 or 3
    Section 5.1 compares top-1 vs. top-3 entity/relation candidates per surface form; the main results do not specify the setting.
  • in-context example counts = 3 per question type; 4 per template
    Section 5.1 states these counts; they are hand-chosen and affect question-type prediction and template selection.
  • non-empty result filter = N/A
    Calibration discards variants with empty SPARQL results; this is an ad hoc success criterion that biases the evaluation.
  • embedding model = unspecified
    Equation 4 uses an embedding model for cosine-similarity linking, but no concrete model is named; the representation is a free choice.
axioms (5)
  • domain assumption All SPICE SPARQL queries can be converted losslessly to S-expressions and back.
    Section 4.2.2.1 asserts 'Testing confirms...' without showing the conversion or edge cases; the whole pipeline depends on this equivalence.
  • ad hoc to paper A correct logical form must execute to a non-empty result.
    Section 4.2.2.2 keeps only variants with non-empty SPARQL results; the conclusion admits this can 'revise valid empty queries'.
  • domain assumption Top-1 cosine-similarity linking is sufficient for entity and relation grounding.
    Section 4.2.2.1 claims single candidate suffices because LLM cores are semantically precise; only internal comparisons support this.
  • domain assumption The question-type taxonomy and template library derived from the SPICE training corpus transfer to the test distribution.
    Section 4.3.2 says templates are grounded in a detailed analysis of the training corpus; the method is evaluated on the same benchmark.
  • domain assumption LLMs (DeepSeek-V3, Qwen2.5-32B-Instruct) reliably perform coreference resolution, core generation, type prediction, template selection, and plan generation with few-shot prompts.
    Section 5.1 implementation details assume this reliability; no oracle analysis or per-component accuracy is reported.
invented entities (3)
  • S-expression core no independent evidence
    purpose: Minimal substructure using JOIN/R/AND/VALUES/IS_TRUE that captures query semantics before template completion.
    New abstraction defined by the paper; no independent benchmark shows it captures 'essential semantics' beyond the paper's structural-overlap metric.
  • Global memory as dynamically updateable knowledge graph no independent evidence
    purpose: Stores validated S-expressions and question-type/surface-form pairs for reuse in later dialogs.
    Architectural construct; its benefit is shown only via in-house ablation with no external replication.
  • New S-expression functions marked * (VALUES, IS_TRUE, OR, DISTINCT, GROUP_COUNT, GROUP_SUM, ALL, EQ) no independent evidence
    purpose: Extend the S-expression grammar to cover SPICE query types.
    Introduced for this benchmark; no evidence they generalize to other KBCQA datasets.

pith-pipeline@v1.3.0-alltime-deepseek · 22951 in / 11196 out tokens · 99686 ms · 2026-08-03T18:28:44.539750+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of SEAL: Self-Evolving Agentic Learning for Conversational Question Answering over Knowledge Graphs." pith.science (2026). https://pith.science/paper/SJAXPESC

@misc{pith2026251204868,
  author       = {Pith},
  title        = {Pith review of: SEAL: Self-Evolving Agentic Learning for Conversational Question Answering over Knowledge Graphs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SJAXPESC}},
  note         = {Machine review of arXiv:2512.04868}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Knowledge-based conversational question answering (KBCQA) confronts persistent challenges in resolving coreference, modeling contextual dependencies, and executing complex logical reasoning. Existing approaches often suffer from inaccuracies and prohibitive computational costs, particularly when processing intricate queries over large knowledge graphs. Specifically, large language models (LLMs) tend to generate syntactically invalid or semantically misaligned logical forms for complex multi-hop or aggregation queries, while conventional entity-relation linking methods face an exponentially growing candidate space. To address these limitations, we introduce SEAL, a novel two-stage semantic parsing framework grounded in self-evolving agentic learning. In the first stage, an LLM extracts a minimal S-expression core capturing the essential semantics, which is then refined by an agentic calibration module to correct syntactic inconsistencies and align entities and relations with the knowledge graph. The second stage employs template-based completion guided by question-type prediction to construct a fully executable S-expression. Crucially, SEAL incorporates a self-evolving mechanism integrating local and global memory with a reflection module, enabling continuous adaptation from dialog history and execution feedback without explicit retraining. Extensive experiments on the SPICE benchmark demonstrate that SEAL achieves state-of-the-art performance in multi-hop reasoning, comparison, and aggregation tasks, validating notable gains in both structural accuracy and computational efficiency.

Figures

Figures reproduced from arXiv: 2512.04868 by Changcheng Wang, Hao Wang, Jialun Zhong, Shunyu Yao, Xinchi Li, Yanzeng Li, Zheng Li, Zhujun Nie.

Figure 1
Figure 1. Figure 1: The challenges of leveraging LLMs in KBCQA. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The framework of our method. 4.2. Reasoning Module The core extraction phase, the initial critical step of the proposed method, focuses on deriving the S-expression core that encapsulates the essential se￾mantics of natural language questions. This phase comprises two key steps: • S-expression Core Generation: LLM analyzes the question text to identify independent query objects, employing five fundamental … view at source ↗
Figure 3
Figure 3. Figure 3: SPARQL query count for each S-expression core extraction parameter setting [PITH_FULL_IMAGE:figures/full_fig_p021_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Impact of different factors on F1 under the Self-Evolving mechanism. (a) S [PITH_FULL_IMAGE:figures/full_fig_p024_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of SEAL and KB-Binder on a multi-turn QA example from the [PITH_FULL_IMAGE:figures/full_fig_p027_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

79 extracted references · 10 linked inside Pith

  1. [1]

    A survey on knowledge graphs: Representation, acquisition, and appli- cations | IEEE journals & magazine | IEEE xplore

  2. [2]

    A survey on application of knowledge graph.Journal of Physics: Conference Series, 1487(1):012016, 2020

    Xiaohan Zou. A survey on application of knowledge graph.Journal of Physics: Conference Series, 1487(1):012016, 2020

    2020

  3. [3]

    A knowledge graph for industry 4.0

    Sebastian R Bader, Irlan Grangel-Gonzalez, Priyanka Nanjappa, Maria- Esther Vidal, and Maria Maleshkova. A knowledge graph for industry 4.0. InEuropean Semantic Web Conference, pages 465–480. Springer, 2020

    2020

  4. [4]

    Complex knowledge base question answering: A survey

    Yunshi Lan, Gaole He, Jinhao Jiang, Jing Jiang, Wayne Xin Zhao, and Ji-Rong Wen. Complex knowledge base question answering: A survey. IEEE Transactions on Knowledge and Data Engineering, 35(11):11196– 11215, 2022

    2022

  5. [5]

    Core techniques of question answering systems over knowledge bases: a survey.Knowledge and Information systems, 55(3):529–569, 2018

    Dennis Diefenbach, Vanessa Lopez, Kamal Singh, and Pierre Maret. Core techniques of question answering systems over knowledge bases: a survey.Knowledge and Information systems, 55(3):529–569, 2018

    2018

  6. [6]

    Rgr-kbqa: Generatinglogicalformsforques- tion answering using knowledge-graph-enhanced large language model

    TengfeiFengandLiangHe. Rgr-kbqa: Generatinglogicalformsforques- tion answering using knowledge-graph-enhanced large language model. 29 InProceedings of the 31st International Conference on Computational Linguistics, pages 3057–3070, 2025

    2025

  7. [7]

    Semantic parsing with candi- date expressions for knowledge base question answering.arXiv preprint arXiv:2410.00414, 2024

    Daehwan Nam and Gary Geunbae Lee. Semantic parsing with candi- date expressions for knowledge base question answering.arXiv preprint arXiv:2410.00414, 2024

    arXiv 2024

  8. [8]

    A brief sur- vey on recent advances in coreference resolution.Artificial Intelligence Review, 56(12):14439–14481, 2023

    Ruicheng Liu, Rui Mao, Anh Tuan Luu, and Erik Cambria. A brief sur- vey on recent advances in coreference resolution.Artificial Intelligence Review, 56(12):14439–14481, 2023

    2023

  9. [9]

    End-to-end neural coreference resolution

    Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. End-to-end neural coreference resolution. In Martha Palmer, Rebecca Hwa, and Se- bastian Riedel, editors,Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark, 2017. Association for Computational Linguistics

    2017

  10. [10]

    Large language models: A survey.arXiv preprint arXiv:2402.06196, 2024

    Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. Large language models: A survey.arXiv preprint arXiv:2402.06196, 2024

    Pith/arXiv arXiv 2024

  11. [11]

    A survey of large language models.arXiv preprint arXiv:2303.18223, 1(2), 2023

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models.arXiv preprint arXiv:2303.18223, 1(2), 2023

    Pith/arXiv arXiv 2023

  12. [12]

    Two is better than one: Answering complex questions by multiple knowledge sources with generalized links.arXiv preprint arXiv:2309.05201, 2023

    Minhao Zhang, Yongliang Ma, Yanzeng Li, Ruoyu Zhang, Lei Zou, and Ming Zhou. Two is better than one: Answering complex questions by multiple knowledge sources with generalized links.arXiv preprint arXiv:2309.05201, 2023

    Pith/arXiv arXiv 2023

  13. [13]

    A progressive question answering framework adaptable to multiple knowledge sources

    Yirui Zhan, Yanzeng Li, Minhao Zhang, and Lei Zou. A progressive question answering framework adaptable to multiple knowledge sources. InAsia-Pacific Web (APWeb) and Web-Age Information Management (WAIM) Joint International Conference on Web and Big Data, pages 362–367. Springer, 2024

    2024

  14. [14]

    Question answering system based on the combination of large language model and knowledge graph

    Jihong Wang, Yichen Zhang, and Wei Liu. Question answering system based on the combination of large language model and knowledge graph. Applied Intelligence, 55(15):1000, 2025. 30

    2025

  15. [15]

    An inter- pretable logic kbqa method based on open-source large language models

    Bicheng Xua, Rong Penga, Yongchang Dinga, and Lin Fanga. An inter- pretable logic kbqa method based on open-source large language models

  16. [16]

    A survey on semantic parsing

    Aishwarya Kamath and Rajarshi Das. A survey on semantic parsing. arXiv preprint arXiv:1812.00978, 2018

    Pith/arXiv arXiv 2018

  17. [17]

    Semantic pars- ing for question answering over knowledge graphs.arXiv preprint arXiv:2401.06772, 2023

    Sijia Wei, Wenwen Zhang, Qisong Li, and Jiang Zhao. Semantic pars- ing for question answering over knowledge graphs.arXiv preprint arXiv:2401.06772, 2023

    Pith/arXiv arXiv 2023

  18. [18]

    Cogkr: Cognitive graph for multi-hop knowledge reasoning.IEEE Transactions on Knowledge and Data Engineering, 35(2):1283–1295, 2021

    Zhengxiao Du, Chang Zhou, Jiangchao Yao, Teng Tu, Letian Cheng, Hongxia Yang, Jingren Zhou, and Jie Tang. Cogkr: Cognitive graph for multi-hop knowledge reasoning.IEEE Transactions on Knowledge and Data Engineering, 35(2):1283–1295, 2021

    2021

  19. [19]

    Clear-kgqa: Clarification-enhanced ambiguity resolution for knowledge graph question answering.arXiv preprint arXiv:2504.09665, 2025

    Liqiang Wen, Guanming Xiong, Tong Mo, Bing Li, Weiping Li, and Wen Zhao. Clear-kgqa: Clarification-enhanced ambiguity resolution for knowledge graph question answering.arXiv preprint arXiv:2504.09665, 2025

    Pith/arXiv arXiv 2025

  20. [20]

    Cord: a three-stage coarse-to-fine framework for relation detection in knowledge base ques- tion answering

    Yanzeng Li, Sen Hu, Wenjuan Han, and Lei Zou. Cord: a three-stage coarse-to-fine framework for relation detection in knowledge base ques- tion answering. InProceedings of the 32nd ACM International Con- ference on Information and Knowledge Management, pages 4069–4073, 2023

    2023

  21. [21]

    Se- mantic parsing via staged query graph generation: Question answering with knowledge base

    Wen-tau Yih, Ming-Wei Chang, Xiaodong He, and Jianfeng Gao. Se- mantic parsing via staged query graph generation: Question answering with knowledge base. In Chengqing Zong and Michael Strube, editors, Proceedings of the 53rd Annual Meeting of the Association for Computa- tional Linguistics and the 7th International Joint Conference on Natural Language Pro...

    2015

  22. [22]

    Constraint-based question answering with knowledge graph

    Junwei Bao, Nan Duan, Zhao Yan, Ming Zhou, and Tiejun Zhao. Constraint-based question answering with knowledge graph. In Yuji Matsumoto and Rashmi Prasad, editors,Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Tech- nical Papers, pages 2503–2514, Osaka, Japan, 2016. The COLING 2016 Organizing Committee. 31

    2016

  23. [23]

    A state-transition framework to answer complex questions over knowledge base

    Sen Hu, Lei Zou, and Xinbo Zhang. A state-transition framework to answer complex questions over knowledge base. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii, editors,Proceedings of the 2018 Conference on Empirical Methods in Natural Language Process- ing, pages 2098–2108, Brussels, Belgium, 2018. Association for Compu- tational Li...

    2018

  24. [24]

    Few-shot in-context learning for knowledge base question answering

    Tianle Li, Xueguang Ma, Alex Zhuang, Yu Gu, Yu Su, and Wenhu Chen. Few-shot in-context learning for knowledge base question answering

  25. [25]

    Code- style in-context learning for knowledge-based question answering

    Zhijie Nie, Richong Zhang, Zhongyuan Wang, and Xudong Liu. Code- style in-context learning for knowledge-based question answering. In Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan, editors, Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelli- gence, IA...

    2024

  26. [26]

    ChatKBQA: A generate-then-retrieve framework for knowledge base question answering with fine-tuned large language models

    Haoran Luo, Haihong E, Zichen Tang, Shiyao Peng, Yikai Guo, Wentai Zhang, Chenghao Ma, Guanting Dong, Meina Song, Wei Lin, Yifan Zhu, and Luu Anh Tuan. ChatKBQA: A generate-then-retrieve framework for knowledge base question answering with fine-tuned large language models. InFindings of the Association for Computational Linguistics ACL 2024, pages 2039–2056

    2024

  27. [27]

    Harness- ing large language models for knowledge graph question answering via adaptive multi-aspect retrieval-augmentation

    Derong Xu, Xinhang Li, Ziheng Zhang, Zhenxi Lin, Zhihong Zhu, Zhi Zheng, Xian Wu, Xiangyu Zhao, Tong Xu, and Enhong Chen. Harness- ing large language models for knowledge graph question answering via adaptive multi-aspect retrieval-augmentation

  28. [29]

    Interactive-KBQA: 32 Multi-turninteractionsforknowledgebasequestionansweringwithlarge language models

    Guanming Xiong, Junwei Bao, and Wen Zhao. Interactive-KBQA: 32 Multi-turninteractionsforknowledgebasequestionansweringwithlarge language models. InProc. of ACL, pages 10561–10582

  29. [30]

    ODA: Observation-driven agent for integrating LLMs and knowledge graphs

    Lei Sun, Zhengwei Tao, Youdi Li, and Hiroshi Arakawa. ODA: Observation-driven agent for integrating LLMs and knowledge graphs

  30. [31]

    Rea- soning on graphs: Faithful and interpretable large language model rea- soning

    Linhao Luo, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. Rea- soning on graphs: Faithful and interpretable large language model rea- soning. InThe Twelfth International Conference on Learning Represen- tations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024

    2024

  31. [32]

    Conversational question answering sys- tems: A comprehensive literature review

    Vaishali Vadhavana, Krishna Patel, Brinda Patel, Bansari Patel, Naina Parmar, and Vaibhavi Patel. Conversational question answering sys- tems: A comprehensive literature review. In2024 International Confer- ence on Inventive Computation Technologies (ICICT), pages 1088–1095. ISSN: 2767-7788

  32. [33]

    Khapra, Karthik Sankara- narayanan, and Sarath Chandar

    Amrita Saha, Vardaan Pahuja, Mitesh M. Khapra, Karthik Sankara- narayanan, and Sarath Chandar. Complex sequential question answer- ing: Towards learning to converse over linked question answer pairs with a knowledge graph. In Sheila A. McIlraith and Kilian Q. Weinberger, ed- itors,Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence...

    2018

  33. [34]

    Con- versational question answering on heterogeneous sources

    Philipp Christmann, Rishiraj Saha Roy, and Gerhard Weikum. Con- versational question answering on heterogeneous sources. In Enrique Amigó, Pablo Castells, Julio Gonzalo, Ben Carterette, J. Shane Culpep- per, and Gabriella Kazai, editors,SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spa...

    2022

  34. [35]

    Semantic parsing for conversational question answering over knowledge graphs

    Laura Perez-Beltrachini, Parag Jain, Emilio Monti, and Mirella Lapata. Semantic parsing for conversational question answering over knowledge graphs. In Andreas Vlachos and Isabelle Augenstein, editors,Proceed- ings of the 17th Conference of the European Chapter of the Association 33 for Computational Linguistics, pages 2507–2522, Dubrovnik, Croatia,

  35. [36]

    Semantic parsing for conversational question answering over knowledge graphs

    Laura Perez-Beltrachini, Parag Jain, Emilio Monti, and Mirella Lapata. Semantic parsing for conversational question answering over knowledge graphs. In Andreas Vlachos and Isabelle Augenstein, editors,Proceed- ings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2507–2522, Dubrovnik, Croatia,

  36. [37]

    Sadler, Percy Liang, Xifeng Yan, andYuSu

    Yu Gu, Sue Kase, Michelle Vanni, Brian M. Sadler, Percy Liang, Xifeng Yan, andYuSu. BeyondI.I.D.: threelevelsofgeneralizationforquestion answeringonknowledgebases. InJureLeskovec, MarkoGrobelnik, Marc Najork, Jie Tang, and Leila Zia, editors,WWW ’21: The Web Confer- ence 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021, pages 3477–3488. ACM / ...

    2021

  37. [38]

    Association for Computational Linguistics

  38. [39]

    A brief survey and comparative study of recent development of pronoun coreference resolu- tion in English

    Hongming Zhang, Xinran Zhao, and Yangqiu Song. A brief survey and comparative study of recent development of pronoun coreference resolu- tion in English. In Maciej Ogrodniczuk, Sameer Pradhan, Massimo Poe- sio, Yulia Grishina, and Vincent Ng, editors,Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coref- erence, pages...

    2021

  39. [40]

    Don’t generate, discriminate: A pro- posal for grounding language models to real-world environments

    Yu Gu, Xiang Deng, and Yu Su. Don’t generate, discriminate: A pro- posal for grounding language models to real-world environments. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors,Pro- ceedings of the 61st Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), pages 4928–4949, Toronto, Canada, 2023. Assoc...

    2023

  40. [41]

    A qualitative comparison of CoQA, SQuAD 2.0 and QuAC

    Mark Yatskar. A qualitative comparison of CoQA, SQuAD 2.0 and QuAC. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of 34 the Association for Computational Linguistics: Human Language Tech- nologies, Volume 1 (Long and Short Papers), pages 2318–2323, Min- neapolis, Minnesota, 2...

    2019

  41. [42]

    Disambiguation in conversational question answering in the era of LLM: A survey

  42. [43]

    Reinforced question rewriting for conversational ques- tion answering

    Zhiyu Chen, Jie Zhao, Anjie Fang, Besnik Fetahu, Oleg Rokhlenko, and Shervin Malmasi. Reinforced question rewriting for conversational ques- tion answering. In Yunyao Li and Angeliki Lazaridou, editors,Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 357–370, Abu Dhabi, UAE, 2022. Association ...

    2022

  43. [44]

    Question rewriting for conversational question answering

    Svitlana Vakulenko, Shayne Longpre, Zhucheng Tu, and Raviteja Anan- tha. Question rewriting for conversational question answering

  44. [45]

    Robust training for conversational question answering models with reinforced reformulation generation

    Magdalena Kaiser, Rishiraj Saha Roy, and Gerhard Weikum. Robust training for conversational question answering models with reinforced reformulation generation

  45. [46]

    Ques- tion rewriting? assessing its importance for conversational question an- swering

    Gonçalo Raposo, Rui Ribeiro, Bruno Martins, and Luísa Coheur. Ques- tion rewriting? assessing its importance for conversational question an- swering. volume 13186, pages 199–206

  46. [48]

    Con- versational question answering with language models generated refor- mulations over knowledge graph

    Lihui Liu, Blaine Hill, Boxin Du, Fei Wang, and Hanghang Tong. Con- versational question answering with language models generated refor- mulations over knowledge graph. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Findings of the Association for Computational Linguistics: ACL 2024, pages 839–850. Association for Computational Linguistics

    2024

  47. [49]

    Integrating large language models with graph-based reasoning for conversational question answering

    Parag Jain and Mirella Lapata. Integrating large language models with graph-based reasoning for conversational question answering

  48. [50]

    Conversational semantic parsing using dynamic context graphs

    Parag Jain and Mirella Lapata. Conversational semantic parsing using dynamic context graphs. In Houda Bouamor, Juan Pino, and Kalika 35 Bali, editors,Proceedings of the 2023 Conference on Empirical Meth- ods in Natural Language Processing, pages 8667–8679, Singapore, 2023. Association for Computational Linguistics

    2023

  49. [51]

    Manoj Madushanka Perera, Adnan Mahmood, Kasun Eranda Wijethi- lake, and Quan Z. Sheng. Towards adaptive context management for in- telligent conversational question answering. In Quan Z. Sheng, Gill Dob- bie, Jing Jiang, Xuyun Zhang, Wei Emma Zhang, Yannis Manolopoulos, Jia Wu, Wathiq Mansoor, and Congbo Ma, editors,Advanced Data Min- ing and Applications...

  50. [52]

    Reinforcement learning from constraints and focal entity shifting in conversational KGQA

    Xirong Xu, Tao Xu, Ziming Wang, Haochen Li, Li Zhu, and Xiaopeng Wei. Reinforcement learning from constraints and focal entity shifting in conversational KGQA. 36(4):2015–2028

    2015

  51. [53]

    Dialog-to- action: Conversational question answering over a large-scale knowledge base

    Daya Guo, Duyu Tang, Nan Duan, Ming Zhou, and Jian Yin. Dialog-to- action: Conversational question answering over a large-scale knowledge base. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors,Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Pro...

    2018

  52. [54]

    KaFSP: Knowledge-aware fuzzy semantic parsing for conversational question answering over a large-scale knowl- edge base

    Junzhuo Li and Deyi Xiong. KaFSP: Knowledge-aware fuzzy semantic parsing for conversational question answering over a large-scale knowl- edge base. In Smaranda Muresan, Preslav Nakov, and Aline Villavicen- cio, editors,Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 461–473, Dublin, Ir...

    2022

  53. [55]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wain- wright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Chris- tiano, Jan Leike, and Ryan Lowe. Training language models to fol- low instructions with hum...

    2022

  54. [56]

    A survey of large language models, 2025

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yu- peng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yi- fanDu, ChenYang, YushuoChen, ZhipengChen, JinhaoJiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. A survey of large language models, 2025. 36

    2025

  55. [57]

    Miao, Eduardo Rodriguez Almaraz, Mad- humita Sushil, Atul J

    Nikita Mehandru, Brenda Y. Miao, Eduardo Rodriguez Almaraz, Mad- humita Sushil, Atul J. Butte, and Ahmed M. Alaa. Evaluating large language models as agents in the clinic.npj Digit. Medicine, 7(1), 2024

    2024

  56. [58]

    Yu, and Ming Zhang

    Junyu Luo, Weizhi Zhang, Ye Yuan, Yusheng Zhao, Junwei Yang, Yiyang Gu, Bohan Wu, Binqi Chen, Ziyue Qiao, Qingqing Long, Rongcheng Tu, Xiao Luo, Wei Ju, Zhiping Xiao, Yifan Wang, Meng Xiao, Chenwu Liu, Jingyang Yuan, Shichang Zhang, Yiqiao Jin, Fan Zhang, Xian Wu, Hanqing Zhao, Dacheng Tao, Philip S. Yu, and Ming Zhang. Large language model agent: A surve...

    Pith/arXiv arXiv 2025

  57. [59]

    Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence. Palm-e: An embodied ...

    2023

  58. [60]

    Yu, and Qingsong Wen

    Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jinheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S. Yu, and Qingsong Wen. LLM agents for education: Advances and applications.CoRR, abs/2503.11733, 2025

    arXiv 2025

  59. [61]

    Under- standing the planning of LLM agents: A survey.CoRR, abs/2402.02716, 2024

    Xu Huang, Weiwen Liu, Xiaolong Chen, Xingmei Wang, Hao Wang, Defu Lian, Yasheng Wang, Ruiming Tang, and Enhong Chen. Under- standing the planning of LLM agents: A survey.CoRR, abs/2402.02716, 2024

    Pith/arXiv arXiv 2024

  60. [62]

    Steve-eye: Equipping llm-based embodied agents with visual perception in open worlds

    Sipeng Zheng, Jiazheng Liu, Yicheng Feng, and Zongqing Lu. Steve-eye: Equipping llm-based embodied agents with visual perception in open worlds. InThe Twelfth International Conference on Learning Represen- tations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024

    2024

  61. [63]

    Kim, Christine P

    Callie Y. Kim, Christine P. Lee, and Bilge Mutlu. Understanding large- language model (llm)-powered human-robot interaction. In Dan Groll- man, Elizabeth Broadbent, Wendy Ju, Harold Soh, and Tom Williams, editors,Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, HRI 2024, Boulder, CO, USA, March 11- 15, 2024, pages 371–...

    2024

  62. [64]

    Plangenllms: A modern survey of LLM planning capabilities

    Hui Wei, Zihao Zhang, Shenghua He, Tian Xia, Shijia Pan, and Fei Liu. Plangenllms: A modern survey of LLM planning capabilities. In Wanx- iang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors,Proceedings of the 63rd Annual Meeting of the Asso- ciation for Computational Linguistics (Volume 1: Long Papers), ACL 2025, Vienna, Austr...

    2025

  63. [65]

    Agentbench: Evaluating llms as agents

    Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, 38 Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. Agentbench: Evaluating llms as agents. InThe Twelfth Inter- national Conference on Learn...

    2024

  64. [66]

    Em- bodied agent interface: Benchmarking llms for embodied decision mak- ing

    Manling Li, Shiyu Zhao, Qineng Wang, Kangrui Wang, Yu Zhou, San- jana Srivastava, Cem Gokmen, Tony Lee, Li Erran Li, Ruohan Zhang, Weiyu Liu, Percy Liang, Li Fei-Fei, Jiayuan Mao, and Jiajun Wu. Em- bodied agent interface: Benchmarking llms for embodied decision mak- ing. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jak...

    2024

  65. [67]

    Think-in-memory: Recalling and post-thinking enable llms with long-term memory.CoRR, abs/2311.08719, 2023

    LeiLiu, XiaoyanYang, YueShen, BinbinHu, ZhiqiangZhang, JinjieGu, and Guannan Zhang. Think-in-memory: Recalling and post-thinking enable llms with long-term memory.CoRR, abs/2311.08719, 2023

    Pith/arXiv arXiv 2023

  66. [68]

    Chatdb: Augmenting llms with databases as their symbolic memory.CoRR, abs/2306.03901, 2023

    Chenxu Hu, Jie Fu, Chenzhuang Du, Simian Luo, Junbo Zhao, and Hang Zhao. Chatdb: Augmenting llms with databases as their symbolic memory.CoRR, abs/2306.03901, 2023

    Pith/arXiv arXiv 2023

  67. [69]

    Wikidata: a new platform for collaborative data col- lection

    Denny Vrandecic. Wikidata: a new platform for collaborative data col- lection. In Alain Mille, Fabien Gandon, Jacques Misselis, Michael Rabi- novich, and Steffen Staab, editors,Proceedings of the 21st World Wide Web Conference, WWW 2012, Lyon, France, April 16-20, 2012 (Com- panion Volume), pages 1063–1064. ACM, 2012

    2012

  68. [70]

    Voyager: Anopen-ended embodied agent with large language models.Trans

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, YukeZhu, LinxiFan, andAnimaAnandkumar. Voyager: Anopen-ended embodied agent with large language models.Trans. Mach. Learn. Res., 2024, 2024

    2024

  69. [71]

    Evaluating large language models in semantic parsing for conversational question answering over knowledge graphs

    Phillip Schneider, Manuel Klettner, Kristiina Jokinen, Elena Simperl, and Florian Matthes. Evaluating large language models in semantic parsing for conversational question answering over knowledge graphs. In Ana Paula Rocha, Luc Steels, and H. Jaap van den Herik, editors,Pro- ceedings of the 16th International Conference on Agents and Artificial Intellige...

    2024

  70. [72]

    Conversational semantic parsing using dynamic context graphs

    Parag Jain and Mirella Lapata. Conversational semantic parsing using dynamic context graphs. In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Proceedings of the 2023 Conference on Empirical Meth- ods in Natural Language Processing, pages 8667–8679, Singapore, 2023. Association for Computational Linguistics

    2023

  71. [74]

    (AND (JOIN x x) (JOIN x x))

  72. [75]

    (AND (JOIN x (VALUES x ...)) (JOIN x x))

  73. [76]

    (AND (JOIN (R x) x) (JOIN x x))

  74. [77]

    (AND (JOIN (R x) (VALUES x ...)) (JOIN x x))

  75. [78]

    (AND (JOIN x (JOIN x x)) (JOIN x x))

  76. [79]

    (AND (JOIN (R x) (JOIN x x)) (JOIN x x))

  77. [80]

    (AND (JOIN x x) (JOIN x x) (JOIN x x))

  78. [81]

    (AND (JOIN (R x) x) (JOIN (R x) x) (JOIN x x))

  79. [82]

    selection,

    (AND (JOIN (R x) x) (JOIN x x) (JOIN (R x) x)) Appendix D. During the organizing of S expression templates, we identified an error in the SPICE dataset. For questions involving equality relations, the corre- sponding SPARQL queries often lack the necessary FILTER component to screen for equality, resulting in incomplete S-expressions. For example, the cor...