Pith. sign in

REVIEW 4 major objections 8 minor 58 references

A single policy, rewarded only for executed answers, beats search-based agents on KBQA with 90% fewer LLM calls.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 17:18 UTC pith:CBNKJPG5

load-bearing objection Credible RL-for-KBQA recipe with a real baseline-comparability problem that must be fixed before the SOTA claim is trusted. the 4 major comments →

arxiv 2512.10999 v4 pith:CBNKJPG5 submitted 2025-12-10 cs.CL

KBQA-R1: Reinforcing Large Language Models for Knowledge Base Question Answering

Xin Sun , Zhongqi Chen , Xing Zheng , Qiang Liu , Shu Wu , Bowen Song , Zilei Wang , Weiqiang Wang
show 1 more author
Liang Wang
This is my paper
classification cs.CL
keywords knowledge base question answeringreinforcement learningGRPOlogical formsagentic reasoningrelation retrievalrejection samplingFreebase
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that knowledge-base question answering is best learned as a multi-turn interaction policy rather than as text imitation or test-time search. The authors define a compact action space—find relation, merge, order, compare, count, time-constrain—and train a single policy with GRPO, rewarding only executed answers (F1 against gold) plus a small format bonus when the answer is correct. To escape the cold-start problem, they introduce Referenced Rejection Sampling, which generates warm-start traces by conditioning a strong model on the gold action sequence and then strips those hints. Reported results on WebQSP (83.4 F1), GrailQA (86.1), and GraphQuestions (53.8) surpass prior agent-based methods, with per-question LLM calls dropping from roughly 29-78 to about 3. If correct, the work shows that outcome-based reinforcement learning on verifiable actions generalizes better than heavy test-time search.

Core claim

The central claim is that execution feedback, not trace imitation, is what makes an LLM reason well over knowledge graphs. KBQA-R1 rephrases KBQA as a Markov decision process: at each turn the model picks an atomic action, the action is grounded as an S-expression and executed against Freebase, and the resulting entity sets and diagnostics become the next observation. The policy is optimized by GRPO with an outcome reward that is the F1 of the final answer against all gold variants, plus a format reward applied only when the outcome is positive. A relation-validation layer (RRCG) prevents hallucinated relations by scoring the model's proposed relation against the entity's actual schema neigh

What carries the argument

The load-bearing machinery is the closed think-action-observe loop with a typed action space and a schema-gating validator. Actions (Find_relation, Merge, Order, Compare, Time_constraint, Count) map to S-Expression fragments, so any single error is localized instead of invalidating a whole program. The RRCG module computes dense-retrieval similarity between the model's proposed relation and the neighboring relations of the current entity, routing the action into three tiers: auto-validate, tentative acceptance (returned with top-k candidate scores so the model can verify), or rejection (returned with the full list of neighbor relations and scores). Referenced Rejection Sampling warms up the

Load-bearing premise

The headline SOTA claim rests on the baseline being fairly reproduced: if KBQA-o1 was not run with the same entity linking, Freebase backend, and evaluation protocol, its reported WebQSP F1 of 57.8—18 points below MCTS-KBQA with the same backbone—makes the +25.6 improvement an artifact, and the central claim of beating agent-based methods collapses.

What would settle it

Run KBQA-o1's released code under KBQA-R1's exact entity-linking and evaluation pipeline on WebQSP and GrailQA; if WebQSP F1 lands near 76.0 instead of 57.8, the reported margins over that baseline are mostly a weak-baseline effect. A second check: retrain KBQA-R1 with standard rejection sampling instead of RRS and compare final F1; if the gap is small, the contribution attributed to referenced warm-start data is not as claimed.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A single trained policy can replace test-time Monte Carlo Tree Search in agentic KBQA, cutting per-question LLM calls by roughly 90% (from 28.8-78 to 2.7-3.2) while raising F1.
  • Execution-grounded RL helps most where generalization is hardest: the biggest margins over the MCTS baseline are in GrailQA's zero-shot and compositional splits, not just the i.i.d. split.
  • Schema gating fixes relation hallucination: removing RRCG drops overall F1 by about 18 points on average, and removing multi-turn iteration drops it by about 25 points.
  • The same hyperparameters work across three datasets, suggesting the RRS warm-start plus GRPO recipe is a stable, transferable training pipeline rather than a per-benchmark hack.
  • Because the trained policy needs no separate reward model or search at inference, KBQA-R1 improves throughput from about 5.9 to 155.6 questions per minute on GrailQA under the same GPU setup.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Our reading: the same interaction-optimization recipe—typed, validated actions plus outcome-only rewards—should transfer to other structured environments with cheap verification, such as SQL generation over relational schemas or API tool use with typed signatures; RRCG is effectively a general 'typed tool validator' that could be ported.
  • Our reading: the reported comparison to KBQA-o1 may be inflated by a weak baseline—KBQA-o1's WebQSP F1 (57.8) is 18 points below MCTS-KBQA running the same backbone (76.0) in the paper's own table—so the true margin over a well-tuned agent could be much smaller than the headline +25.6 improvement.
  • Our reading: RRS leaks the gold action sequence into training-data generation; a natural follow-up experiment would measure how much of the final gain survives when RL runs on top of plain-SFT data, isolating the value of RL from the value of the warmer start.
  • Our reading: the evaluation assumes topic entities are given, so the approach is tested with perfect linking; integrating an entity linker would show whether the learned policy stays robust under realistic linking errors.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper proposes KBQA-R1, a two-stage framework for KBQA with a Llama-3.1-8B policy. In Stage 1, Referenced Rejection Sampling (RRS) extracts ground-truth action sequences from gold S-expressions, uses Qwen-2.5-72B-Instruct to generate reasoning traces conditioned on those references, filters for correctness, and strips the references before SFT. In Stage 2, GRPO optimizes the policy with an outcome F1 reward plus a format reward gated on correct outcomes, while interactions are mediated by a relation-retrieval/confidence-gating (RRCG) module and a typed multi-turn action space. The paper reports state-of-the-art F1 on WebQSP (83.4), GrailQA dev (86.1 overall), and GraphQuestions (53.8), with far fewer inference-time LLM calls than an MCTS-based baseline. The central claim is that execution-grounded RL, warm-started by RRS, produces a single policy that generalizes better than MCTS-based agents under the same backbone.

Significance. If the reported numbers hold, this is a valuable result: it would show that outcome-based RL with a structured action space can beat an MCTS-based agentic KBQA pipeline under the same backbone while using far fewer inference calls, and that the RRS warm-start is substantially more sample-efficient than standard rejection sampling. The ablation study (Table VI) demonstrates nontrivial contributions from RRCG, multi-turn interaction, RRS, and GRPO; Table VII's acceptance-rate gains for RRS are a concrete strength. The paper also states that code and data will be released. I do not see a circularity problem: RRS uses gold logical forms only as warm-start hints, and the GRPO objective is outcome-based. However, the empirical SOTA claim is not currently supported because the baseline comparisons are not demonstrably apples-to-apples, and one key baseline row is internally inconsistent (see major comments). The paper emphasizes that prompt-based baselines share KBQA-R1's annotation constraints, which is also not accurate as written.

major comments (4)
  1. [Table IV; §V.E; §V.A.2] The central SOTA claim rests on an unexplained baseline inconsistency. Table IV lists KBQA-o1 at 57.8 F1 and MCTS-KBQA at 76.0 F1 on WebQSP, both with Llama-3.1-8B. §V.E defines MCTS as “the MCTS-optimized variant in KBQA-o1 without incremental Fine-tuning.” If that is correct, adding incremental fine-tuning would lower F1 by about 18 points, which is counterintuitive and unexplained. The paper never specifies the entity linker, Freebase dump version, or evaluation protocol for each row; §III assumes topic entities are given as input, so if KBQA-R1 uses gold topic entities while baselines perform their own linking, the comparison is unfair. The row also cites [54], which is the “Don’t generate, discriminate” paper and does not describe an MCTS method. These issues affect the headline +25.6% F1 gain and must be resolved by rerunning or fully documenting the baseline settings.
  2. [§V.A.2; §IV-B] The statement that prompt-based baselines “operate under the same limited annotation constraint as KBQA-R1” is not accurate as written. KBQA-R1 is trained on the full training sets and, in RRS, uses gold S-expressions to derive reference action sequences (Step 1 of §IV-B), whereas KB-BINDER, KB-Coder, and ARG-KBQA are few-shot prompting methods with no such training signal. If “limited annotation” means something specific (e.g., no additional human rationales), that must be defined; otherwise the comparison to prompting baselines conflates supervision regimes. Because the paper claims superiority over both end-to-end and agent-based baselines, this distinction is load-bearing and should be corrected or the claims restricted.
  3. [§IV.A.3; Table VI; §V.F] RRCG is the largest single component by the ablation: removing it drops F1 by about 18 points on average (Table VI). Yet the paper only says it is “implemented using dense retrieval techniques [38],[39].” The retriever model, its training data, and the candidate relation set are not specified, so the core system is not reproducible. Moreover, because RRCG replaces the proposed relation with the best schema relation above tau_high, the policy may not be the component performing most schema grounding. The §V.F claim that GRPO “effectively guides the agent toward more accurate relation selection” should be supported by reporting relation accuracy before RRCG gating, not only after it, to attribute the improvement correctly.
  4. [§V.B; Table III; Table IV] Several stated margins are not accurately represented. §V.B says KBQA-R1 outperforms “the best prompting baseline by over 20 percentage points” on WebQSP, but Table IV’s best prompting baseline is Interactive-KBQA at 71.2 F1, giving a margin of 12.2 points; the >20-point margin holds only against ARG-KBQA (58.8). Additionally, Table III’s “Improv.” column mixes absolute and relative changes (e.g., i.i.d. EM +12.8 points vs. i.i.d. F1 +7.0% relative), and Table IV’s “+25.6%” is actually absolute percentage points. These should be reported with a consistent, explicitly labeled convention.
minor comments (8)
  1. [§V.A.4] “All experiments use the same base model architecture” is ambiguous: Tables III–V include T5-large, BERT, and GPT-3.5/4-turbo baselines. Clarify that the same-backbone statement applies only to the Llama-3.1-8B comparisons and the KBQA-R1 ablations.
  2. [§V.F; Fig. 5] Figure 5 is labeled “Critic Mean Reward,” but GRPO as described in §IV.C does not train a critic. Rename the axis to “mean reward” or explain the critic.
  3. [Table VII] The denominators (e.g., 2,929 for WebQSP, 43,851 for GrailQA, 2,332 for GraphQ) differ from the training set sizes in §V.A.1 (3,098, 44,337, 2,508). Explain the filtering. Also, the caption says “SFT Init F1 reports dev-set F1” while §V.C says “initial test-set F1”; reconcile the two.
  4. [§IV.B; Table VII] Step 3 of RRS uses an unspecified threshold τ for trajectory acceptance; Table VII later uses F1>0.9. Specify τ in the method description so the filtering criterion is unambiguous.
  5. [Table V; Fig. 3] The dataset is introduced as GraphQuestions, but Table V is titled “Results on the test set of GraphQ” and Figure 3 uses “GraphQ.” Define the abbreviation or use the full name consistently.
  6. [§IV.B; §V.D] The RRS teacher is Qwen-2.5-72B-Instruct, and the final policy is Llama-3.1-8B-Instruct. This training cost should be stated in the abstract or contributions; the “far fewer LLM calls” claim is an inference-time claim and should be labeled as such.
  7. [§V.D] The LLM-call efficiency comparison in Table VIII reports averages on “200 sampled examples” per dataset. Specify which split the samples come from and whether the same questions were used for both methods.
  8. [§IV.A.3] The RRCG tentative-acceptance branch returns a top-k candidate set, but k is not specified. Also, no sensitivity analysis is given for tau_high=0.95 and tau_low=0.3, which are set across all datasets. A brief sensitivity table would strengthen the reproducibility of the method.

Circularity Check

0 steps flagged

No significant circularity: the KBQA-R1 pipeline is self-contained and its reported results are not forced by construction.

full rationale

KBQA-R1's derivation chain is empirical rather than definitional. RRS (Section IV-B) conditions generation on ground-truth action sequences extracted from gold S-Expressions, filters by executed F1, strips the references before SFT, and then GRPO (Section IV-C) optimizes an outcome+format reward defined against gold answers. None of the paper's equations (Eq. 1-4) or definitions makes the held-out test-set F1 in Tables III-V equal to a fitted parameter or training input; the results are measured against external benchmarks on the Freebase backend. The RRS-versus-RS acceptance comparison in Table VII and the relation-similarity diagnostic in Figure 4 both use gold actions by construction, but they are presented as pipeline diagnostics rather than as independent predictions, so they do not constitute circular derivation. The only serious concern is an external-validity issue, not circularity: Section V.E states that 'MCTS corresponds to the MCTS-optimized variant in KBQA-o1 [6] without incremental Fine-tuning,' which is difficult to reconcile with Table IV showing KBQA-o1 at 57.8 and MCTS-KBQA at 76.0 on the same Llama-3.1-8B backbone. That inconsistency undermines baseline comparability of the headline SOTA claim, but it is a reproducibility/evaluation-protocol risk, not a case of the result reducing to its own inputs.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The method depends on standard supervised data (gold logical forms), pre-supplied entity linking, and a dense relation-retrieval matcher. No new entities are invented.

free parameters (5)
  • tau_high = 0.95
    Confidence threshold above which RRCG auto-validates a proposed relation and replaces it with the best-matching schema relation (Section IV-A3).
  • tau_low = 0.3
    Below this similarity the proposed relation is rejected and the full candidate list is returned (Section IV-A3).
  • lambda_format = 0.1
    Weight of format reward in composite reward, Eq. (2).
  • lambda_outcome = 1.0
    Weight of outcome reward, Eq. (2).
  • RRS filter threshold = F1 > 0.9
    Trajectories accepted for SFT only if final answer F1 > 0.9, per Table VII (not explicitly defined in text).
axioms (4)
  • domain assumption Gold S-Expressions (logical forms) are available for the training questions.
    RRS extracts ground-truth action sequences from gold S-Expressions (Section IV-B, Step 1).
  • domain assumption Entity mentions in questions are already linked to KB entities and topic entities are given as input.
    Section III: 'Following prior work [6], we assume that entity mentions in q are already linked...'
  • domain assumption Dense retrieval similarity can correctly rank candidate KB schema relations for a natural-language relation phrase.
    RRCG relies on Sim(.,.) to map agent-proposed relations to schema relations (Section IV-A3).
  • domain assumption The KB executor and Freebase backend return exact, noiseless results.
    Environment observations are treated as ground truth in rewards and training traces.

pith-pipeline@v1.3.0-alltime-deepseek · 19700 in / 13907 out tokens · 131172 ms · 2026-08-03T17:18:49.041076+00:00 · methodology

0 comments
read the original abstract

Knowledge Base Question Answering (KBQA) challenges models to bridge the gap between natural language and strict knowledge graph schemas by generating executable logical forms. While Large Language Models (LLMs) have advanced this field, current approaches often struggle with a dichotomy of failure: they either generate hallucinated queries without verifying schema existence or exhibit rigid, template-based reasoning that mimics synthesized traces without true comprehension of the environment. To address these limitations, we present \textbf{KBQA-R1}, a framework that shifts the paradigm from text imitation to interaction optimization via Reinforcement Learning. Treating KBQA as a multi-turn decision process, our model learns to navigate the knowledge base using a list of actions, leveraging Group Relative Policy Optimization (GRPO) to refine its strategies based on concrete execution feedback rather than static supervision. Furthermore, we introduce \textbf{Referenced Rejection Sampling (RRS)}, a data synthesis method that resolves cold-start challenges by strictly aligning reasoning traces with ground-truth action sequences. Extensive experiments on WebQSP, GrailQA, and GraphQuestions demonstrate that KBQA-R1 achieves state-of-the-art performance, effectively grounding LLM reasoning in verifiable execution.

Figures

Figures reproduced from arXiv: 2512.10999 by Bowen Song, Liang Wang, Qiang Liu, Shu Wu, Weiqiang Wang, Xing Zheng, Xin Sun, Zhongqi Chen, Zilei Wang.

Figure 1
Figure 1. Figure 1: Overview of the KBQA-R1 multi-turn reasoning framework. Given a natural language question, the LLM-based agent [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The two-stage training pipeline of KBQA-R1. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Relation similarity analysis of dataset WebQSP during [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Training reward of dataset GrailQA curve during RL. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

58 extracted references · 13 linked inside Pith

  1. [1]

    Few-shot in- context learning on knowledge base question answering,

    T. Li, X. Ma, A. Zhuang, Y . Gu, Y . Su, and W. Chen, “Few-shot in- context learning on knowledge base question answering,” inProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)(A. Rogers, J. Boyd-Graber, and N. Okazaki, eds.), (Toronto, Canada), pp. 6966–6980, Association for Computational Lingu...

    2023

  2. [2]

    Code-style in-context learning for knowledge-based question answering,

    Z. Nie, R. Zhang, Z. Wang, and X. Liu, “Code-style in-context learning for knowledge-based question answering,”Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 18833–18841, Mar. 2024

    2024

  3. [3]

    ChatKBQA: A generate- then-retrieve framework for knowledge base question answering with fine-tuned large language models,

    H. Luo, H. E, Z. Tang, S. Peng, Y . Guo, W. Zhang, C. Ma, G. Dong, M. Song, W. Lin, Y . Zhu, and A. T. Luu, “ChatKBQA: A generate- then-retrieve framework for knowledge base question answering with fine-tuned large language models,” inFindings of the Association for Computational Linguistics ACL 2024(L.-W. Ku, A. Martins, and V . Srikumar, eds.), (Bangkok...

    2024

  4. [4]

    Think-on-graph: Deep and responsible reasoning of large language model on knowledge graph,

    J. Sun, C. Xu, L. Tang, S. Wang, C. Lin, Y . Gong, H.-Y . Shum, and J. Guo, “Think-on-graph: Deep and responsible reasoning of large language model on knowledge graph,”arXiv preprint arXiv:2307.07697, 2023

    Pith/arXiv arXiv 2023

  5. [5]

    Reasoning on graphs: Faith- ful and interpretable large language model reasoning,

    L. Luo, Y .-F. Li, G. Haffari, and S. Pan, “Reasoning on graphs: Faith- ful and interpretable large language model reasoning,”arXiv preprint arXiv:2310.01061, 2023

    Pith/arXiv arXiv 2023

  6. [6]

    Kbqa-o1: Agentic knowledge base question answering with monte carlo tree search,

    H. Luo, Y . Guo, Q. Lin, X. Wu, X. Mu, W. Liu, M. Song, Y . Zhu, L. A. Tuan,et al., “Kbqa-o1: Agentic knowledge base question answering with monte carlo tree search,”arXiv preprint arXiv:2501.18922, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models,

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . Li, Y . Wu,et al., “Deepseekmath: Pushing the limits of mathematical reasoning in open language models,”arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  8. [8]

    Open domain question answering using early fusion of knowledge bases and text,

    H. Sun, B. Dhingra, M. Zaheer, K. Mazaitis, R. Salakhutdinov, and W. Cohen, “Open domain question answering using early fusion of knowledge bases and text,” inProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, (Brussels, Belgium), pp. 4231–4242, Association for Computational Linguistics, Oct.-Nov. 2018

    2018

  9. [9]

    PullNet: Open domain question answering with iterative retrieval on knowledge bases and text,

    H. Sun, T. Bedrax-Weiss, and W. Cohen, “PullNet: Open domain question answering with iterative retrieval on knowledge bases and text,” inProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), (Hong Kong, China), pp. 2380–2390, Associat...

    2019

  10. [10]

    Subgraph retrieval enhanced model for multi-hop knowledge base question answering,

    J. Zhang, X. Zhang, J. Yu, J. Tang, J. Tang, C. Li, and H. Chen, “Subgraph retrieval enhanced model for multi-hop knowledge base question answering,” inProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), (Dublin, Ireland), pp. 5773–5784, Association for Computational Lin- guistics, May 2022

    2022

  11. [11]

    Improving multi-hop knowledge base question answering by learning intermediate supervision signals,

    G. He, Y . Lan, J. Jiang, W. X. Zhao, and J.-R. Wen, “Improving multi-hop knowledge base question answering by learning intermediate supervision signals,” inProceedings of the 14th ACM International Conference on Web Search and Data Mining, WSDM ’21, (New York, NY , USA), p. 553–561, Association for Computing Machinery, 2021

    2021

  12. [12]

    Improving multi-hop question answering over knowledge graphs using knowledge base embeddings,

    A. Saxena, A. Tripathi, and P. Talukdar, “Improving multi-hop question answering over knowledge graphs using knowledge base embeddings,” inProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, (Online), pp. 4498–4507, Association for Computational Linguistics, July 2020. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGU...

    2020

  13. [13]

    RNG-KBQA: Generation augmented iterative ranking for knowledge base question answering,

    X. Ye, S. Yavuz, K. Hashimoto, Y . Zhou, and C. Xiong, “RNG-KBQA: Generation augmented iterative ranking for knowledge base question answering,” inProceedings of the 60th Annual Meeting of the Associ- ation for Computational Linguistics (Volume 1: Long Papers), (Dublin, Ireland), pp. 6032–6043, Association for Computational Linguistics, May 2022

    2022

  14. [14]

    TIARA: Multi-grained retrieval for robust question answering over large knowledge base,

    Y . Shu, Z. Yu, Y . Li, B. Karlsson, T. Ma, Y . Qu, and C.-Y . Lin, “TIARA: Multi-grained retrieval for robust question answering over large knowledge base,” inProceedings of the 2022 Conference on Empir- ical Methods in Natural Language Processing, (Abu Dhabi, United Arab Emirates), pp. 8108–8121, Association for Computational Linguistics, Dec. 2022

    2022

  15. [15]

    ArcaneQA: Dynamic program induction and contextu- alized encoding for knowledge base question answering,

    Y . Gu and Y . Su, “ArcaneQA: Dynamic program induction and contextu- alized encoding for knowledge base question answering,” inProceedings of the 29th International Conference on Computational Linguistics (N. Calzolari, C.-R. Huang, H. Kim, J. Pustejovsky, L. Wanner, K.- S. Choi, P.-M. Ryu, H.-H. Chen, L. Donatelli, H. Ji, S. Kurohashi, P. Paggio, N. Xue...

    2022

  16. [16]

    FC-KBQA: A fine-to-coarse composition framework for knowledge base question answering,

    L. Zhang, J. Zhang, Y . Wang, S. Cao, X. Huang, C. Li, H. Chen, and J. Li, “FC-KBQA: A fine-to-coarse composition framework for knowledge base question answering,” inProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), (Toronto, Canada), pp. 1002–1017, Association for Computational Linguistics, July 2023

    2023

  17. [17]

    StructGPT: A general framework for large language model to reason over structured data,

    J. Jiang, K. Zhou, Z. Dong, K. Ye, X. Zhao, and J.-R. Wen, “StructGPT: A general framework for large language model to reason over structured data,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing(H. Bouamor, J. Pino, and K. Bali, eds.), (Singapore), pp. 9237–9251, Association for Computational Linguistics, Dec. 2023

    2023

  18. [18]

    Don’t generate, discriminate: A proposal for grounding language models to real-world environments,

    Y . Gu, X. Deng, and Y . Su, “Don’t generate, discriminate: A proposal for grounding language models to real-world environments,” inProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)(A. Rogers, J. Boyd-Graber, and N. Okazaki, eds.), (Toronto, Canada), pp. 4928–4949, Association for Computational...

    2023

  19. [19]

    QueryAgent: A reliable and efficient reasoning framework with environmental feedback based self-correction,

    X. Huang, S. Cheng, S. Huang, J. Shen, Y . Xu, C. Zhang, and Y . Qu, “QueryAgent: A reliable and efficient reasoning framework with environmental feedback based self-correction,” inProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)(L.-W. Ku, A. Martins, and V . Srikumar, eds.), (Bangkok, Thailan...

    2024

  20. [20]

    Interactive-KBQA: Multi-turn inter- actions for knowledge base question answering with large language models,

    G. Xiong, J. Bao, and W. Zhao, “Interactive-KBQA: Multi-turn inter- actions for knowledge base question answering with large language models,” inProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)(L.-W. Ku, A. Martins, and V . Srikumar, eds.), (Bangkok, Thailand), pp. 10561– 10582, Association fo...

    2024

  21. [21]

    Plan-on-graph: Self-correcting adaptive planning of large language model on knowledge graphs,

    L. Chen, P. Liu, Z. Wang, Y . Xiao, X. Wang, J. Zhao, C. Wang, K. Zhang, and J. Wang, “Plan-on-graph: Self-correcting adaptive planning of large language model on knowledge graphs,” inAdvances in Neural Information Processing Systems, vol. 37, 2024

    2024

  22. [22]

    Kg-agent: An efficient autonomous agent framework for complex reasoning over knowledge graph,

    J. Jiang, K. Zhou, W. X. Zhao, Y . Song, C. Zhu, H. Zhu, and J.-R. Wen, “Kg-agent: An efficient autonomous agent framework for complex reasoning over knowledge graph,” 2024

    2024

  23. [23]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, b. ichter, F. Xia, E. Chi, Q. V . Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” inAdvances in Neural Information Processing Systems(S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, eds.), vol. 35, pp. 24824–24837, Curran Associates, Inc., 2022

    2022

  24. [24]

    React: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y . Cao, “React: Synergizing reasoning and acting in language models,”arXiv preprint arXiv:2210.03629, 2022

    Pith/arXiv arXiv 2022

  25. [25]

    Reasoning with language model is planning with world model,

    S. Hao, Y . Gu, H. Ma, J. Hong, Z. Wang, D. Wang, and Z. Hu, “Reasoning with language model is planning with world model,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing(H. Bouamor, J. Pino, and K. Bali, eds.), (Singa- pore), pp. 8154–8173, Association for Computational Linguistics, Dec. 2023

    2023

  26. [26]

    Tree of thoughts: Deliberate problem solving with large language models,

    S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y . Cao, and K. Narasimhan, “Tree of thoughts: Deliberate problem solving with large language models,” inAdvances in Neural Information Processing Systems(A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, eds.), vol. 36, pp. 11809–11822, Curran Associates, Inc., 2023

    2023

  27. [27]

    Retrieval-augmented generation for knowledge-intensive nlp tasks,

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive nlp tasks,” in Advances in Neural Information Processing Systems(H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, eds.), vol. 33, pp. 9459– 9474, ...

    2020

  28. [28]

    Interleav- ing retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions,

    H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal, “Interleav- ing retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions,”arXiv preprint arXiv:2212.10509, 2022

    Pith/arXiv arXiv 2022

  29. [29]

    Toolformer: Language models can teach themselves to use tools,

    T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom, “Toolformer: Language models can teach themselves to use tools,”Advances in Neural Infor- mation Processing Systems, vol. 36, pp. 68539–68551, 2023

    2023

  30. [30]

    Search-o1: Agentic search-enhanced large reasoning models,

    X. Li, G. Dong, J. Jin, Y . Zhang, Y . Zhou, Y . Zhu, P. Zhang, and Z. Dou, “Search-o1: Agentic search-enhanced large reasoning models,”arXiv preprint arXiv:2501.05366, 2025

    Pith/arXiv arXiv 2025

  31. [31]

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning,

    B. Jin, H. Zeng, Z. Yue, J. Yoon, S. Arik, D. Wang, H. Zamani, and J. Han, “Search-r1: Training llms to reason and leverage search engines with reinforcement learning,”arXiv preprint arXiv:2503.09516, 2025

    Pith/arXiv arXiv 2025

  32. [32]

    G-retriever: Retrieval-augmented generation for textual graph understanding and question answering,

    X. He, Y . Tian, Y . Sun, N. V . Chawla, T. Laurent, Y . LeCun, X. Bresson, and B. Hooi, “G-retriever: Retrieval-augmented generation for textual graph understanding and question answering,” inAdvances in Neural Information Processing Systems, vol. 37, 2024

    2024

  33. [33]

    Think-on-graph 2.0: Deep and faithful large language model reasoning with knowledge-guided retrieval augmented genera- tion,

    S. Maet al., “Think-on-graph 2.0: Deep and faithful large language model reasoning with knowledge-guided retrieval augmented genera- tion,” inThe Thirteenth International Conference on Learning Repre- sentations, 2025

    2025

  34. [34]

    Beyond i.i.d.: Three levels of generalization for question answering on knowledge bases,

    Y . Gu, S. Kase, M. Vanni, B. Sadler, P. Liang, X. Yan, and Y . Su, “Beyond i.i.d.: Three levels of generalization for question answering on knowledge bases,” inProceedings of the Web Conference 2021, WWW ’21, (New York, NY , USA), p. 3477–3488, Association for Computing Machinery, 2021

    2021

  35. [35]

    Active retrieval augmented generation,

    Z. Jiang, F. F. Xu, L. Gao, Z. Sun, Q. Liu, J. Dwivedi-Yu, Y . Yang, J. Callan, and G. Neubig, “Active retrieval augmented generation,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 7969–7992, 2023

    2023

  36. [36]

    Semantics and complexity of sparql,

    J. Pérez, M. Arenas, and C. Gutierrez, “Semantics and complexity of sparql,”ACM Trans. Database Syst., vol. 34, Sept. 2009

    2009

  37. [37]

    Siren’s song in the ai ocean: A survey on hallucination in large language models,

    Y . Zhang, Y . Li, L. Cui, D. Cai, L. Liu, T. Fu, X. Huang, E. Zhao, Y . Zhang, Y . Chen, L. Wang, A. T. Luu, W. Bi, F. Shi, and S. Shi, “Siren’s song in the ai ocean: A survey on hallucination in large language models,” 2023

    2023

  38. [38]

    Dense passage retrieval for open-domain question answering.,

    V . Karpukhin, B. Oguz, S. Min, P. S. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih, “Dense passage retrieval for open-domain question answering.,” inEMNLP (1), pp. 6769–6781, 2020

    2020

  39. [39]

    Dense text retrieval based on pretrained language models: A survey,

    W. X. Zhao, J. Liu, R. Ren, and J.-R. Wen, “Dense text retrieval based on pretrained language models: A survey,”ACM Transactions on Information Systems, vol. 42, no. 4, pp. 1–60, 2024

    2024

  40. [40]

    Scaling relationship on learning mathematical reasoning with large language models,

    Z. Yuan, H. Yuan, C. Li, G. Dong, K. Lu, C. Tan, C. Zhou, and J. Zhou, “Scaling relationship on learning mathematical reasoning with large language models,”arXiv preprint arXiv:2308.01825, 2023

    Pith/arXiv arXiv 2023

  41. [41]

    Star: Bootstrapping reasoning with reasoning,

    E. Zelikman, Y . Wu, J. Mu, and N. D. Goodman, “Star: Bootstrapping reasoning with reasoning,” inAdvances in Neural Information Process- ing Systems, vol. 35, pp. 15476–15488, 2022

    2022

  42. [42]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray,et al., “Training language models to follow instructions with human feedback,”Advances in neural information processing systems, vol. 35, pp. 27730–27744, 2022

    2022

  43. [43]

    Reinforcement learning: A survey,

    L. P. Kaelbling, M. L. Littman, and A. W. Moore, “Reinforcement learning: A survey,”Journal of artificial intelligence research, vol. 4, pp. 237–285, 1996

    1996

  44. [44]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  45. [45]

    High- dimensional continuous control using generalized advantage estimation,

    J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel, “High- dimensional continuous control using generalized advantage estimation,” arXiv preprint arXiv:1506.02438, 2015

    Pith/arXiv arXiv 2015

  46. [46]

    Simple statistical gradient-following algorithms for con- nectionist reinforcement learning,

    R. J. Williams, “Simple statistical gradient-following algorithms for con- nectionist reinforcement learning,”Machine learning, vol. 8, pp. 229– 256, 1992

    1992

  47. [47]

    Efficient memory management for large language model serving with pagedattention,

    W. Kwon, Z. Li, S. Zhuang, Y . Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for large language model serving with pagedattention,” inProceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, pp. 611–626, 2023. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 12

    2023

  48. [48]

    Free- base: a collaboratively created graph database for structuring human knowledge,

    K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor, “Free- base: a collaboratively created graph database for structuring human knowledge,” inProceedings of the 2008 ACM SIGMOD International Conference on Management of Data, SIGMOD ’08, (New York, NY , USA), p. 1247–1250, Association for Computing Machinery, 2008

    2008

  49. [49]

    The value of semantic parse labeling for knowledge base question answer- ing,

    W.-t. Yih, M. Richardson, C. Meek, M.-W. Chang, and J. Suh, “The value of semantic parse labeling for knowledge base question answer- ing,” inProceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), (Berlin, Germany), pp. 201–206, Association for Computational Linguistics, Aug. 2016

    2016

  50. [50]

    On generating characteristic-rich question sets for QA evaluation,

    Y . Su, H. Sun, B. Sadler, M. Srivatsa, I. Gür, Z. Yan, and X. Yan, “On generating characteristic-rich question sets for QA evaluation,” in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing(J. Su, K. Duh, and X. Carreras, eds.), (Austin, Texas), pp. 562–572, Association for Computational Linguistics, Nov. 2016

    2016

  51. [51]

    Decaf: Joint decoding of answers and logical forms for question answering over knowledge bases,

    D. Yu, S. Zhang, P. Ng, H. Zhu, A. H. Li, J. Wang, Y . Hu, W. Wang, Z. Wang, and D. Hakkani-Tur, “Decaf: Joint decoding of answers and logical forms for question answering over knowledge bases,”arXiv preprint arXiv:2210.00063, 2022

    Pith/arXiv arXiv 2022

  52. [52]

    Sparqa: Skeleton-based seman- tic parsing for complex questions over knowledge bases,

    Y . Sun, L. Zhang, G. Cheng, and Y . Qu, “Sparqa: Skeleton-based seman- tic parsing for complex questions over knowledge bases,”Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 8952–8959, Apr. 2020

    2020

  53. [53]

    Augmenting reasoning capabilities of LLMs with graph structures in knowledge base question answering,

    Y . Tian, D. Song, Z. Wu, C. Zhou, H. Wang, J. Yang, J. Xu, R. Cao, and H. Wang, “Augmenting reasoning capabilities of LLMs with graph structures in knowledge base question answering,” inFindings of the As- sociation for Computational Linguistics: EMNLP 2024(Y . Al-Onaizan, M. Bansal, and Y .-N. Chen, eds.), (Miami, Florida, USA), pp. 11967– 11977, Associ...

    2024

  54. [54]

    Don’t generate, discriminate: A proposal for grounding language models to real-world environments,

    W. Xu, Y . Sun, X. Huang, Y . Cai, S. Liu, S. Liu, and M. Sun, “Don’t generate, discriminate: A proposal for grounding language models to real-world environments,” inProceedings of the 61st Annual Meeting of the Association for Computational Linguistics, pp. 3726–3741, 2023

    2023

  55. [55]

    Chain-of-thought rea- soning with knowledge retrieval,

    D. Yu, K. Narasimhan, C. Wang, and W. Xiong, “Chain-of-thought rea- soning with knowledge retrieval,” inProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 4523–4538, 2024

    2024

  56. [56]

    Dapo: An open-source llm reinforcement learning system at scale,

    Q. Yu, Z. Sun, Y . Shen, R. Xu, Y . Li, J. Lin, B. Xiao, Y . Zhang, H. Zeng, Z. Wang,et al., “Dapo: An open-source llm reinforcement learning system at scale,”arXiv preprint arXiv:2503.14476, 2025

    Pith/arXiv arXiv 2025

  57. [57]

    Pytorch fsdp: Experiences on scaling fully sharded data parallel,

    Y . Zhao, A. Gu, R. Varma, L. Luo, C.-C. Huang, M. Xu, L. Wright, H. Shojanazeri, M. Ott, S. Shleifer,et al., “Pytorch fsdp: Experiences on scaling fully sharded data parallel,”Proceedings of the VLDB Endowment, vol. 16, no. 12, pp. 3848–3860, 2023

    2023

  58. [58]

    Virtuoso: Rdf support in a native rdbms,

    O. Erling and I. Mikhailov, “Virtuoso: Rdf support in a native rdbms,” inSemantic Web Information Management: A Model-Based Perspective, pp. 501–519, Springer, 2010. VII. BIOGRAPHYSECTION Xin Sunis a joint Ph.D. candidate from Univer- sity of Science and Technology of China(USTC) and Institute of Automation, Chinese Academy of Sciences(CASIA). He received...

    2010