{"id":"1875d544-2d30-4e9f-92b1-b7d24a2ed0a4","arxiv_id":"2605.01482","paper_version":3,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"An SCM-GRPO framework grounds multi-hop reasoning in structural dependency graphs and optimizes chain length via rule-based RL, outperforming baselines on HoVer and EX-FEVER.","lead":"This paper introduces an SCM-inspired framework that models multi-hop fact verification as building explicit directed dependency graphs and uses Group Relative Policy Optimization to balance reasoning depth against accuracy. A smart generalist might read it to see how reinforcement learning can reduce hallucinations in AI systems that check complex claims.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Core claim rests on treating directed dependency graphs as sufficient 'SCM grounding' despite explicit omission of interventions/counterfactuals; performance gains may stem only from rule-based GRPO rather than structural causal properties.","rationale":"Reader's weakest_assumption directly identifies the same non-causal reduction; full-text access does not resolve it because the abstract already flags the deliberate omission of interventions. This makes the SCM label and the 'grounding' claim the least secure link to the empirical headline.","tokens_in":1654,"tokens_out":326,"duration_ms":23074,"concrete_test":"Add an ablation replacing the SCM-derived dependency graphs with randomly generated or purely syntactic dependency graphs of matched length distribution; retrain GRPO under identical hyperparameters and compare F1/accuracy on HoVer dev set. If the gap to the original SCM-GRPO shrinks below 2 points, the causal-graph component is not load-bearing.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The abstract and framing state the approach is 'SCM-inspired' yet 'rather than full causal inference with interventions or counterfactual semantics,' reducing it to constructive graph building. For the outperformance claim on HoVer/EX-FEVER to be attributable to the SCM component (vs. standard RL or CoT), the graphs must demonstrably encode causal dependencies that improve traceability or accuracy beyond what non-causal DAGs achieve. The inverted-U correlation is cited as motivation for GRPO, but without evidence that GRPO's rule-based updates actually exploit causal structure (as opposed to length penalties), the central attribution is insecure.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces an SCM-inspired framework (SCM-GRPO) for multi-hop fact verification that grounds reasoning in explicit directed dependency graphs, treating verification as constructive structural reasoning rather than full causal inference with interventions or counterfactuals. It reports identifying an inverted U-shaped correlation between reasoning-chain length and accuracy, proposes rule-based Group Relative Policy Optimization to optimize the depth-conciseness trade-off, and claims outperformance over strong baselines on HoVer and EX-FEVER while yielding more traceable structures.","tokens_in":1811,"tokens_out":465,"duration_ms":14308,"significance":"If the empirical claims hold under rigorous controls and ablations, the work could offer a practical method for improving traceability in LLM-based verification via graph-structured RL. However, the explicit reduction to non-causal graph construction limits its connection to Structural Causal Model literature and risks the gains being attributable solely to the RL component rather than any causal grounding.","major_comments":[{"comment":"Abstract: the central claim of outperformance on HoVer and EX-FEVER is stated without any metrics, baseline details, statistical tests, ablation results, or error bars, preventing evaluation of whether gains derive from the SCM-inspired graphs versus the GRPO rule-based updates alone.","section":"Abstract"},{"comment":"Abstract: the framework is titled and framed as 'Grounding ... in Structural Causal Models' yet explicitly states it avoids 'full causal inference with interventions or counterfactual semantics,' reducing to directed graph construction; this creates a mismatch that undermines attribution of improved traceability or accuracy to causal properties rather than standard graph-based RL.","section":"Abstract"},{"comment":"Abstract: the inverted U-shaped correlation between chain length and accuracy is cited as motivation for GRPO, but no evidence is supplied that the rule-based updates exploit causal dependencies in the graphs (as opposed to simple length penalties), leaving the central attribution insecure.","section":"Abstract"}],"minor_comments":[{"comment":"The abstract supplies no dataset statistics, model sizes, or implementation details for the claimed experiments, which should be added for reproducibility.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We appreciate the referee's thorough review and constructive criticism of our work. We respond to each major comment in turn, indicating planned revisions where appropriate to enhance the clarity and rigor of the manuscript.","responses":[{"response":"We agree that the abstract, as a high-level summary, omits specific quantitative details. The full manuscript reports these metrics, baselines, ablations, and error bars in the experimental sections. To address the concern directly, we will revise the abstract to incorporate key performance figures, baseline comparisons, and a note on statistical significance where space permits.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the central claim of outperformance on HoVer and EX-FEVER is stated without any metrics, baseline details, statistical tests, ablation results, or error bars, preventing evaluation of whether gains derive from the SCM-inspired graphs versus the GRPO rule-based updates alone."},{"response":"The work is explicitly SCM-inspired, using the structural equations and directed dependency graphs from SCMs to model evidence-claim relations for constructive reasoning, while deliberately avoiding interventions and counterfactuals as stated in the text. This distinguishes the approach from generic graph-based RL by grounding optimization in dependency structures. We will revise the abstract to more precisely phrase the contribution as 'SCM-inspired structural dependency graphs' to reduce any framing ambiguity.","revision_made":"partial","referee_comment":"[Abstract] Abstract: the framework is titled and framed as 'Grounding ... in Structural Causal Models' yet explicitly states it avoids 'full causal inference with interventions or counterfactual semantics,' reducing to directed graph construction; this creates a mismatch that undermines attribution of improved traceability or accuracy to causal properties rather than standard graph-based RL."},{"response":"The observed inverted U-shaped correlation motivates the depth-conciseness trade-off. The rule-based GRPO incorporates rewards derived from the full structural properties of the dependency graphs (including path dependencies and verification consistency), not length alone. We will expand the method description in revision to explicitly detail the rule components and how they reference graph dependencies.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the inverted U-shaped correlation between chain length and accuracy is cited as motivation for GRPO, but no evidence is supplied that the rule-based updates exploit causal dependencies in the graphs (as opposed to simple length penalties), leaving the central attribution insecure."}],"tokens_in":1340,"tokens_out":521,"duration_ms":27076,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main thing here is a framework that builds directed dependency graphs for fact verification and then applies Group Relative Policy Optimization to control chain length. They note an inverted-U pattern where longer chains hurt accuracy, and they use rule-based RL to balance depth against conciseness. Experiments on HoVer and EX-FEVER are said to beat baselines while producing clearer structures.\n\nWhat is new is the specific combination of graph construction with this GRPO variant for the verification setting, plus the empirical observation on chain length. The approach does produce more explicit reasoning traces than plain CoT, which is a practical step for people who need to audit LLM outputs.\n\nThe soft spot is the causal framing. The abstract states outright that the method skips interventions and counterfactuals, so the graphs are dependency structures rather than causal models. Without ablations that isolate the graph structure from the RL rules, or comparisons to ordinary DAGs, it is unclear whether the reported gains come from anything causal or just from the length penalty and rule-based updates. The abstract also gives no numbers, baseline details, or statistical tests, which leaves the central claim hard to judge from the summary alone.\n\nThis is aimed at researchers working on reliable multi-hop reasoning in LLMs. It has enough concrete experiments and a clear optimization trick to deserve a serious referee, though the causal claims will need tightening and more controls in revision.","headline":"The paper gives a workable graph-plus-RL method for traceable multi-hop verification but the SCM label is mostly decorative since interventions are left out.","tokens_in":2327,"tokens_out":351,"would_cite":false,"duration_ms":14571,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Grounding multi-hop fact verification in directed dependency graphs and optimizing them with group relative policy reinforcement learning improves accuracy and traceability.","keywords":["multi-hop fact verification","structural causal models","group relative policy optimization","directed dependency graphs","reasoning chain optimization","large language models","fact verification"],"falsifier":"If the SCM-GRPO method applied to HoVer and EX-FEVER shows no accuracy gain over strong baselines and no improvement in traceability of reasoning structures, the central claim would be falsified.","tokens_in":2572,"feed_emoji":"","tokens_out":605,"duration_ms":24552,"temperature":0.7,"pith_summary":"The paper aims to show that multi-hop fact verification improves when treated as the explicit construction of directed dependency graphs inspired by structural causal models rather than relying on implicit chain-of-thought steps in large language models. It reports an inverted U-shaped pattern where medium-length reasoning chains perform best and introduces a rule-based Group Relative Policy Optimization method to learn the right depth dynamically. This addresses hallucinations and broken logic chains that arise when models must combine disparate evidence. If the approach holds, verification systems could produce outputs that are both more accurate and easier to inspect. The experiments focus on showing gains on the HoVer and EX-FEVER benchmarks.","feed_headline":"Graphs plus RL improve multi-hop fact verification","feed_subtitle":"By building explicit evidence-claim dependency structures and tuning chain length, the method reduces errors on complex verification benchma","key_machinery":"Directed dependency graphs for modeling evidence-claim relations, optimized by rule-based Group Relative Policy Optimization to control reasoning chain length.","core_discovery":"The central claim is that an SCM-inspired framework representing verification as construction of directed dependency graphs, paired with Group Relative Policy Optimization to balance structural depth against conciseness according to the observed inverted U-shaped accuracy curve, produces higher performance and more traceable reasoning than standard baselines on multi-hop fact verification tasks.","pith_inferences":["The explicit graph representation could allow human inspection or editing of intermediate reasoning steps in deployed systems.","The optimization principle might transfer to other multi-step tasks such as multi-document summarization or legal argument construction.","If the inverted U pattern appears in additional domains, prompting strategies for language models could be redesigned around length control rather than pure depth."],"forward_implications":["The framework outperforms strong baselines on the HoVer and EX-FEVER datasets.","It generates more traceable reasoning structures for complex fact verification.","It mitigates hallucinations and fractured logical chains in large language models.","The inverted U-shaped correlation between chain length and accuracy can be dynamically optimized via the reinforcement learning rule."],"fun_headline_variants":["SCM graphs with GRPO improve multi-hop fact verification","GRPO balances depth in evidence dependency graphs","Directed graphs ground multi-hop fact verification via GRPO","Structural causal models use GRPO to tune reasoning chains"],"cache_read_input_tokens":64,"weakest_assumption_plain":"That fact verification reduces to constructive building of directed dependency graphs without needing interventions or counterfactual causal reasoning.","fun_headline_variants_meta":{"raw":{"variants":["SCM graphs with GRPO improve multi-hop fact verification","GRPO balances depth in evidence dependency graphs","Directed graphs ground multi-hop fact verification via GRPO","Structural causal models use GRPO to tune reasoning chains"]},"model":"grok-4.3","cost_usd":0.009715,"raw_usage":{"total_tokens":4288,"prompt_tokens":589,"num_sources_used":0,"completion_tokens":59,"cost_in_usd_ticks":97149500,"prompt_tokens_details":{"text_tokens":589,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3640,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":589,"tokens_out":59,"duration_ms":29331,"temperature":1.0,"reasoning_tokens":3640,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-01T00:19:00.279250+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"If the SCM-GRPO method applied to HoVer and EX-FEVER shows no accuracy gain over strong baselines and no improvement in traceability of reasoning structures, the central claim would be falsified.","supporting_citations":[],"review_version":3}