{"id":"b9770058-bdc7-4ea1-a8c3-42c1c6da6958","arxiv_id":"2512.10999","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"KBQA-R1 achieves state-of-the-art KBQA results by combining GRPO reinforcement learning with an action-based multi-turn agent and reference-conditioned warm-start data.","lead":"KBQA-R1 trains an LLM to answer knowledge-base questions step-by-step, using reinforcement learning on execution feedback. It reports state-of-the-art results on three KBQA benchmarks, but a key baseline comparison looks potentially uneven.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Baseline comparability for KBQA-o1 is not established: WebQSP 57.8 vs MCTS-KBQA 76.0 on the same backbone is unexplained; the claimed SOTA margin may be an artifact of differing entity linking or evaluation protocols.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the comparison to KBQA-o1 assumes identical setup, and the paper's own numbers create an anomaly. The WebQSP gap between KBQA-o1 (57.8) and MCTS-KBQA (76.0) on the same backbone is the sharpest evidence that something is inconsistent. My read of the full text confirms this: Section V.E explicitly equates MCTS-KBQA with a KBQA-o1 variant, making the 18-point drop from adding incremental fine-tuning implausible without an explanation. The paper also omits entity-linker and Freebase-version details, which are precisely the variables that can change KBQA scores by double digits. I do not see a more load-bearing concern. The methodological pieces (RRS, GRPO, RRCG) are coherent and the ablations are suggestive, but the central claim is empirical SOTA, and that claim cannot be evaluated until the baseline anomaly is resolved. I therefore agree with the reader's conditional verdict and recommend no change.","tokens_in":19980,"tokens_out":7084,"duration_ms":73247,"concrete_test":"Run the official KBQA-o1 code with the cited checkpoint on the exact WebQSP test split used by KBQA-R1, using KBQA-R1's entity linker and Freebase backend, and compute F1 with KBQA-R1's evaluation script. If KBQA-o1 still scores about 57.8 F1, the comparison is valid; if it scores near 76.0 or higher, the claimed margin over KBQA-o1 is overstated and the SOTA claim must be re-evaluated. Also run KBQA-R1 with the entity linker used by KBQA-o1 if the linkers differ.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central SOTA claim depends on apples-to-apples baseline numbers, and the paper contains an internal contradiction that makes the KBQA-o1 comparison suspect. Table IV lists KBQA-o1 at 57.8 F1 on WebQSP and MCTS-KBQA at 76.0 F1, both labeled Llama-3.1-8B. Section V.E then says MCTS-KBQA is 'the MCTS-optimized variant in KBQA-o1 without incremental Fine-tuning.' If that is correct, adding KBQA-o1's incremental fine-tuning would lower F1 by about 18 points, which is unexplained and counterintuitive. More likely, the two numbers come from different entity linkers, Freebase dumps, evaluation scripts, or training constraints. The paper never specifies the entity linker used, the Freebase version, or how the 'same limited annotation constraint' is enforced for the prompting baselines. Since the headline result is a +25.6% F1 gain over KBQA-o1 on WebQSP and a broad superiority claim across all three datasets, an unverified baseline makes the central empirical claim unreliable. This is not a disagreement with the RL/GRPO methodology; it is a correctness risk in the evidence supporting the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes KBQA-R1, a two-stage framework for KBQA with a Llama-3.1-8B policy. In Stage 1, Referenced Rejection Sampling (RRS) extracts ground-truth action sequences from gold S-expressions, uses Qwen-2.5-72B-Instruct to generate reasoning traces conditioned on those references, filters for correctness, and strips the references before SFT. In Stage 2, GRPO optimizes the policy with an outcome F1 reward plus a format reward gated on correct outcomes, while interactions are mediated by a relation-retrieval/confidence-gating (RRCG) module and a typed multi-turn action space. The paper reports state-of-the-art F1 on WebQSP (83.4), GrailQA dev (86.1 overall), and GraphQuestions (53.8), with far fewer inference-time LLM calls than an MCTS-based baseline. The central claim is that execution-grounded RL, warm-started by RRS, produces a single policy that generalizes better than MCTS-based agents under the same backbone.","tokens_in":20278,"tokens_out":21040,"duration_ms":200379,"significance":"If the reported numbers hold, this is a valuable result: it would show that outcome-based RL with a structured action space can beat an MCTS-based agentic KBQA pipeline under the same backbone while using far fewer inference calls, and that the RRS warm-start is substantially more sample-efficient than standard rejection sampling. The ablation study (Table VI) demonstrates nontrivial contributions from RRCG, multi-turn interaction, RRS, and GRPO; Table VII's acceptance-rate gains for RRS are a concrete strength. The paper also states that code and data will be released. I do not see a circularity problem: RRS uses gold logical forms only as warm-start hints, and the GRPO objective is outcome-based. However, the empirical SOTA claim is not currently supported because the baseline comparisons are not demonstrably apples-to-apples, and one key baseline row is internally inconsistent (see major comments). The paper emphasizes that prompt-based baselines share KBQA-R1's annotation constraints, which is also not accurate as written.","major_comments":[{"comment":"The central SOTA claim rests on an unexplained baseline inconsistency. Table IV lists KBQA-o1 at 57.8 F1 and MCTS-KBQA at 76.0 F1 on WebQSP, both with Llama-3.1-8B. §V.E defines MCTS as “the MCTS-optimized variant in KBQA-o1 without incremental Fine-tuning.” If that is correct, adding incremental fine-tuning would lower F1 by about 18 points, which is counterintuitive and unexplained. The paper never specifies the entity linker, Freebase dump version, or evaluation protocol for each row; §III assumes topic entities are given as input, so if KBQA-R1 uses gold topic entities while baselines perform their own linking, the comparison is unfair. The row also cites [54], which is the “Don’t generate, discriminate” paper and does not describe an MCTS method. These issues affect the headline +25.6% F1 gain and must be resolved by rerunning or fully documenting the baseline settings.","section":"Table IV; §V.E; §V.A.2"},{"comment":"The statement that prompt-based baselines “operate under the same limited annotation constraint as KBQA-R1” is not accurate as written. KBQA-R1 is trained on the full training sets and, in RRS, uses gold S-expressions to derive reference action sequences (Step 1 of §IV-B), whereas KB-BINDER, KB-Coder, and ARG-KBQA are few-shot prompting methods with no such training signal. If “limited annotation” means something specific (e.g., no additional human rationales), that must be defined; otherwise the comparison to prompting baselines conflates supervision regimes. Because the paper claims superiority over both end-to-end and agent-based baselines, this distinction is load-bearing and should be corrected or the claims restricted.","section":"§V.A.2; §IV-B"},{"comment":"RRCG is the largest single component by the ablation: removing it drops F1 by about 18 points on average (Table VI). Yet the paper only says it is “implemented using dense retrieval techniques [38],[39].” The retriever model, its training data, and the candidate relation set are not specified, so the core system is not reproducible. Moreover, because RRCG replaces the proposed relation with the best schema relation above tau_high, the policy may not be the component performing most schema grounding. The §V.F claim that GRPO “effectively guides the agent toward more accurate relation selection” should be supported by reporting relation accuracy before RRCG gating, not only after it, to attribute the improvement correctly.","section":"§IV.A.3; Table VI; §V.F"},{"comment":"Several stated margins are not accurately represented. §V.B says KBQA-R1 outperforms “the best prompting baseline by over 20 percentage points” on WebQSP, but Table IV’s best prompting baseline is Interactive-KBQA at 71.2 F1, giving a margin of 12.2 points; the >20-point margin holds only against ARG-KBQA (58.8). Additionally, Table III’s “Improv.” column mixes absolute and relative changes (e.g., i.i.d. EM +12.8 points vs. i.i.d. F1 +7.0% relative), and Table IV’s “+25.6%” is actually absolute percentage points. These should be reported with a consistent, explicitly labeled convention.","section":"§V.B; Table III; Table IV"}],"minor_comments":[{"comment":"“All experiments use the same base model architecture” is ambiguous: Tables III–V include T5-large, BERT, and GPT-3.5/4-turbo baselines. Clarify that the same-backbone statement applies only to the Llama-3.1-8B comparisons and the KBQA-R1 ablations.","section":"§V.A.4"},{"comment":"Figure 5 is labeled “Critic Mean Reward,” but GRPO as described in §IV.C does not train a critic. Rename the axis to “mean reward” or explain the critic.","section":"§V.F; Fig. 5"},{"comment":"The denominators (e.g., 2,929 for WebQSP, 43,851 for GrailQA, 2,332 for GraphQ) differ from the training set sizes in §V.A.1 (3,098, 44,337, 2,508). Explain the filtering. Also, the caption says “SFT Init F1 reports dev-set F1” while §V.C says “initial test-set F1”; reconcile the two.","section":"Table VII"},{"comment":"Step 3 of RRS uses an unspecified threshold τ for trajectory acceptance; Table VII later uses F1>0.9. Specify τ in the method description so the filtering criterion is unambiguous.","section":"§IV.B; Table VII"},{"comment":"The dataset is introduced as GraphQuestions, but Table V is titled “Results on the test set of GraphQ” and Figure 3 uses “GraphQ.” Define the abbreviation or use the full name consistently.","section":"Table V; Fig. 3"},{"comment":"The RRS teacher is Qwen-2.5-72B-Instruct, and the final policy is Llama-3.1-8B-Instruct. This training cost should be stated in the abstract or contributions; the “far fewer LLM calls” claim is an inference-time claim and should be labeled as such.","section":"§IV.B; §V.D"},{"comment":"The LLM-call efficiency comparison in Table VIII reports averages on “200 sampled examples” per dataset. Specify which split the samples come from and whether the same questions were used for both methods.","section":"§V.D"},{"comment":"The RRCG tentative-acceptance branch returns a top-k candidate set, but k is not specified. Also, no sensitivity analysis is given for tau_high=0.95 and tau_low=0.3, which are set across all datasets. A brief sensitivity table would strengthen the reproducibility of the method.","section":"§IV.A.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is potentially interesting, and I found no circularity or deliberate misreporting. The main barrier is empirical: the WebQSP KBQA-o1 row (57.8 F1) versus MCTS-KBQA (76.0 F1) with the same backbone is internally inconsistent with §V.E, and the entity-linking/evaluation protocol is unspecified. I recommend requiring the authors to rerun or fully document the baselines, correct the annotation-constraint claim, and supply RRCG implementation details before the SOTA claim can be accepted. The reference misattribution for MCTS-KBQA ([54]) should also be fixed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe short version: this is a credible RL-for-KBQA recipe with a real baseline-comparability problem that needs fixing before the SOTA claim is taken at face value. The method is not a huge conceptual leap — GRPO on a compact ReAct-style action space, warm-started with a rejection-sampling variant — but it is cleanly engineered and the public code and data are a plus.\n\nWhat is new: RRS, conditioning generation on gold action sequences from the S-expression and then stripping them after filtering, is a small but useful twist on STaR/rejection sampling, and the paper shows it improves acceptance rates and SFT initialization across three datasets (Table VII). The application of GRPO to KBQA is an expected extension of Search-R1-style tool-use RL, but this is one of the first papers to do it with a typed action space and a live Freebase executor.\n\nThe good stuff: the ablation study is honest and fairly comprehensive. Removing RRCG costs ~18 F1 points on average, which tells you the dense retrieval module is doing much of the schema grounding — that is important context for interpreting the 'RL yields genuine reasoning' narrative. The efficiency results are impressive: ~2.6 LLM calls per question vs ~28 for KBQA-o1, and 155 vs 5.9 questions per minute on GrailQA. If the accuracy numbers hold, that is a real practical contribution.\n\nThe soft spots: the central SOTA claim rests on a suspicious baseline number. Table IV lists KBQA-o1 at 57.8 F1 on WebQSP and MCTS-KBQA at 76.0, both Llama-3.1-8B, and Section V.E identifies MCTS as the MCTS variant of KBQA-o1 without incremental fine-tuning. That implies adding incremental fine-tuning lowers F1 by 18 points, which is not credible unless the two numbers come from different settings. The paper never specifies the entity linker, Freebase version, or evaluation protocol for the baselines, and it says prompting baselines share the same annotation constraint as KBQA-R1 when KBQA-R1 uses full training data with gold logical forms. Those are load-bearing omissions, not nitpicks.\n\nAlso, the claim that RL alone produces 'genuine reasoning' is overstated. The RRCG module is doing much of the relation grounding, and the similarity-score analysis in Figure 4 measures agreement with the gold action list — a proxy that is partly circular given RRS used those same action lists for training.\n\nWho this is for: researchers working on KBQA or RL for tool use. They will get useful engineering detail and a new training recipe. The baseline issue is fixable; the authors should release a clear baseline-reproduction appendix and either correct the KBQA-o1 number or drop that comparison.\n\nRecommendation: send to peer review. It deserves a serious referee, but not acceptance until the comparability question is resolved.","headline":"Credible RL-for-KBQA recipe with a real baseline-comparability problem that must be fixed before the SOTA claim is trusted.","tokens_in":20793,"tokens_out":3137,"would_cite":true,"duration_ms":29524,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single policy, rewarded only for executed answers, beats search-based agents on KBQA with 90% fewer LLM calls.","keywords":["knowledge base question answering","reinforcement learning","GRPO","logical forms","agentic reasoning","relation retrieval","rejection sampling","Freebase"],"falsifier":"Run KBQA-o1's released code under KBQA-R1's exact entity-linking and evaluation pipeline on WebQSP and GrailQA; if WebQSP F1 lands near 76.0 instead of 57.8, the reported margins over that baseline are mostly a weak-baseline effect. A second check: retrain KBQA-R1 with standard rejection sampling instead of RRS and compare final F1; if the gap is small, the contribution attributed to referenced warm-start data is not as claimed.","tokens_in":19846,"feed_emoji":"🤖","tokens_out":8987,"duration_ms":81614,"temperature":0.7,"pith_summary":"The paper argues that knowledge-base question answering is best learned as a multi-turn interaction policy rather than as text imitation or test-time search. The authors define a compact action space—find relation, merge, order, compare, count, time-constrain—and train a single policy with GRPO, rewarding only executed answers (F1 against gold) plus a small format bonus when the answer is correct. To escape the cold-start problem, they introduce Referenced Rejection Sampling, which generates warm-start traces by conditioning a strong model on the gold action sequence and then strips those hints. Reported results on WebQSP (83.4 F1), GrailQA (86.1), and GraphQuestions (53.8) surpass prior agent-based methods, with per-question LLM calls dropping from roughly 29-78 to about 3. If correct, the work shows that outcome-based reinforcement learning on verifiable actions generalizes better than heavy test-time search.","feed_headline":"One RL policy beats MCTS agents, using 90% fewer LLM calls","feed_subtitle":"Trained only on executed answers, the policy tops 83 F1 on WebQSP and 86 on GrailQA while eliminating test-time search.","key_machinery":"The load-bearing machinery is the closed think-action-observe loop with a typed action space and a schema-gating validator. Actions (Find_relation, Merge, Order, Compare, Time_constraint, Count) map to S-Expression fragments, so any single error is localized instead of invalidating a whole program. The RRCG module computes dense-retrieval similarity between the model's proposed relation and the neighboring relations of the current entity, routing the action into three tiers: auto-validate, tentative acceptance (returned with top-k candidate scores so the model can verify), or rejection (returned with the full list of neighbor relations and scores). Referenced Rejection Sampling warms up the","core_discovery":"The central claim is that execution feedback, not trace imitation, is what makes an LLM reason well over knowledge graphs. KBQA-R1 rephrases KBQA as a Markov decision process: at each turn the model picks an atomic action, the action is grounded as an S-expression and executed against Freebase, and the resulting entity sets and diagnostics become the next observation. The policy is optimized by GRPO with an outcome reward that is the F1 of the final answer against all gold variants, plus a format reward applied only when the outcome is positive. A relation-validation layer (RRCG) prevents hallucinated relations by scoring the model's proposed relation against the entity's actual schema neigh","pith_inferences":["Our reading: the same interaction-optimization recipe—typed, validated actions plus outcome-only rewards—should transfer to other structured environments with cheap verification, such as SQL generation over relational schemas or API tool use with typed signatures; RRCG is effectively a general 'typed tool validator' that could be ported.","Our reading: the reported comparison to KBQA-o1 may be inflated by a weak baseline—KBQA-o1's WebQSP F1 (57.8) is 18 points below MCTS-KBQA running the same backbone (76.0) in the paper's own table—so the true margin over a well-tuned agent could be much smaller than the headline +25.6 improvement.","Our reading: RRS leaks the gold action sequence into training-data generation; a natural follow-up experiment would measure how much of the final gain survives when RL runs on top of plain-SFT data, isolating the value of RL from the value of the warmer start.","Our reading: the evaluation assumes topic entities are given, so the approach is tested with perfect linking; integrating an entity linker would show whether the learned policy stays robust under realistic linking errors."],"forward_implications":["A single trained policy can replace test-time Monte Carlo Tree Search in agentic KBQA, cutting per-question LLM calls by roughly 90% (from 28.8-78 to 2.7-3.2) while raising F1.","Execution-grounded RL helps most where generalization is hardest: the biggest margins over the MCTS baseline are in GrailQA's zero-shot and compositional splits, not just the i.i.d. split.","Schema gating fixes relation hallucination: removing RRCG drops overall F1 by about 18 points on average, and removing multi-turn iteration drops it by about 25 points.","The same hyperparameters work across three datasets, suggesting the RRS warm-start plus GRPO recipe is a stable, transferable training pipeline rather than a per-benchmark hack.","Because the trained policy needs no separate reward model or search at inference, KBQA-R1 improves throughput from about 5.9 to 155.6 questions per minute on GrailQA under the same GPU setup."],"fun_headline_variants":["RL over execution feedback makes LLMs reason better on knowledge graphs","KBQA-R1: Learn by doing, not by mimicking query traces","Execution-grounded RL lifts LLM KBQA to SOTA without test-time search","RRS + GRPO: LLMs learn KBQA by acting and seeing results","RL interaction focus makes LLMs reason over knowledge graphs without hallucination"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The headline SOTA claim rests on the baseline being fairly reproduced: if KBQA-o1 was not run with the same entity linking, Freebase backend, and evaluation protocol, its reported WebQSP F1 of 57.8—18 points below MCTS-KBQA with the same backbone—makes the +25.6 improvement an artifact, and the central claim of beating agent-based methods collapses.","fun_headline_variants_meta":{"raw":{"variants":["RL over execution feedback makes LLMs reason better on knowledge graphs","KBQA-R1: Learn by doing, not by mimicking query traces","Execution-grounded RL lifts LLM KBQA to SOTA without test-time search","RRS + GRPO: LLMs learn KBQA by acting and seeing results","RL interaction focus makes LLMs reason over knowledge graphs without hallucination"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000858,"raw_usage":{"total_tokens":3562,"prompt_tokens":747,"completion_tokens":2815,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":491,"completion_tokens_details":{"reasoning_tokens":2718}},"tokens_in":491,"tokens_out":2815,"duration_ms":22797,"temperature":1.0,"reasoning_tokens":2718,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T17:18:49.041076+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run KBQA-o1's released code under KBQA-R1's exact entity-linking and evaluation pipeline on WebQSP and GrailQA; if WebQSP F1 lands near 76.0 instead of 57.8, the reported margins over that baseline are mostly a weak-baseline effect. A second check: retrain KBQA-R1 with standard rejection sampling instead of RRS and compare final F1; if the gap is small, the contribution attributed to referenced warm-start data is not as claimed.","supporting_citations":[],"review_version":1}