{"id":"a0604100-73c4-457f-9bac-fba5e894f2f5","arxiv_id":"2605.27858","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"DecomposeRL uses GRPO-based RL on a curated 5K-claim subset to train a decomposition policy that achieves 86.3 in-domain and 69.8 out-of-domain balanced accuracy on 11 benchmarks, supporting semi-supervised learning.","lead":"DecomposeRL trains a 7B language model with reinforcement learning to decompose claims into questions, producing traceable verification steps while matching larger models on accuracy. This could enable more transparent fact-checking systems that work with limited labeled data across domains like medicine and politics.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Data-curation funnel from 115K to 5K claims may embed benchmark-specific selection bias that inflates both in-domain and out-of-domain numbers.","rationale":"The reader's weakest_assumption directly names the same point; the full paper would need to supply the missing curation details and an ablation against random selection to move the verdict.","tokens_in":1782,"tokens_out":349,"duration_ms":11942,"concrete_test":"Release the exact curation script and criteria (or at minimum the list of 5K selected claims with their provenance). Then retrain the 7B policy on a random 5K subset drawn from the original 115K and re-evaluate on the same 11 benchmarks; a drop >5 points on the OOD average would indicate that the funnel itself carries the generalization signal.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The headline result rests on training a 7B policy exclusively on a distilled 5K subset whose construction is described only at high level (a 'learning-signal-dense' funnel). For the 86.3 / 69.8 balanced-accuracy figures and the claim of matching 32B and GPT-4.1-mini models to be credible, two conditions must hold: (1) the funnel must not have used any information derived from the 11 evaluation benchmarks, and (2) the retained 5K examples must preserve the diversity needed for the observed OOD transfer. Neither condition is demonstrated by the abstract; if the funnel implicitly favors claims whose decomposition patterns or evidence styles match the test sets, the performance numbers become circular. The multi-faceted reward ensemble could further amplify any such alignment.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes DecomposeRL, which frames claim decomposition for verification as an RL policy trained via GRPO with a multi-faceted reward ensemble. It introduces a data-curation funnel to distill 115K claims into a compact 5K subset for efficient training, enabling both fully supervised and semi-supervised modes. A DecomposeRL-7B model trained on the 5K subset reports 86.3 in-domain and 69.8 out-of-domain balanced accuracy across 11 benchmarks (biomedical, political, scientific, general), matching 32B baselines and GPT-4.1-mini while outperforming in semi-supervised settings with 10% labeled data. Code, data, and models are released.","tokens_in":1940,"tokens_out":408,"duration_ms":20646,"significance":"If the central performance claims hold, the work would be significant for showing that compact models can achieve competitive traceable claim verification via RL and curated data, bridging the gap between accurate but opaque end-to-end classifiers and inspectable but weaker decomposition methods. The code and model release is a clear strength that supports reproducibility and further research.","major_comments":[{"comment":"Abstract: the headline result (86.3/69.8 balanced accuracy on 11 benchmarks) rests on the data-curation funnel reducing 115K claims to 5K; the manuscript provides only a high-level description of this funnel and does not demonstrate that its selection criteria avoid any information derived from the 11 evaluation benchmarks or that the retained examples preserve sufficient diversity for the reported OOD transfer. This is load-bearing for the claim that the numbers are non-circular.","section":"Abstract"}],"minor_comments":[{"comment":"Abstract: minor grammatical issue ('DecomposeRL an accurate claim-verifier that produce inspectable traces') should be corrected for clarity.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the thoughtful review and for highlighting the importance of rigorously documenting the data-curation process. We address the concern below and commit to a substantive revision.","responses":[{"response":"We agree that the current manuscript provides only a high-level description of the curation funnel and does not include explicit verification that the selection criteria are independent of the 11 evaluation benchmarks or quantitative evidence of retained diversity. In the revision we will (1) expand the methods section with the precise, reproducible criteria used to distill the 115K claims (including all filtering, scoring, and selection steps), (2) add an explicit statement and supporting table confirming that the curation pipeline operated exclusively on the 115K pool with no access to or leakage from any of the 11 held-out benchmarks, and (3) report diversity statistics (e.g., claim-topic distribution, length, source coverage) for the final 5K subset relative to both the original pool and the OOD evaluation sets. These additions will directly substantiate the non-circular nature of the reported numbers.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the headline result (86.3/69.8 balanced accuracy on 11 benchmarks) rests on the data-curation funnel reducing 115K claims to 5K; the manuscript provides only a high-level description of this funnel and does not demonstrate that its selection criteria avoid any information derived from the 11 evaluation benchmarks or that the retained examples preserve sufficient diversity for the reported OOD transfer. This is load-bearing for the claim that the numbers are non-circular."}],"tokens_in":1390,"tokens_out":348,"duration_ms":24934,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that this paper trains a decomposition policy with GRPO and a reward ensemble, then uses a curation funnel to shrink 115K claims down to 5K for efficient training. A 7B model reaches 86.3 in-domain and 69.8 out-of-domain balanced accuracy on 11 benchmarks spanning biomedical, political, and other domains, matching 32B baselines and GPT-4.1-mini while also improving in the semi-supervised case with 10% labels. Code and models are released.\n\nWhat is new is the explicit RL framing of decomposition combined with the multi-faceted rewards and the funnel that makes full supervision on a small set feasible. It directly targets the accuracy-traceability gap without obvious performance loss.\n\nThe curation funnel is the soft spot worth checking. Reducing to 5K is central to the result, yet the abstract gives only a high-level description. If the funnel implicitly selects for patterns that align with the evaluation benchmarks, both the in-domain and OOD numbers could be overstated. The reward ensemble might reinforce that alignment. The paper reports no circular fitting in the accuracies themselves, and the empirical setup looks standard.\n\nThis is for NLP groups working on explainable claim verification or RL for decomposition tasks. Readers who care about traceable outputs on fact-checking benchmarks will get concrete numbers and a working method.\n\nIt deserves peer review because the performance claims are specific, the approach is distinct from prior decomposition work, and the code release supports checking. Reviewers should focus on the funnel details and any overlap with test data.","headline":"DecomposeRL gets competitive traceable verification numbers from a 7B RL policy on a 5K curated set, but the funnel's construction is the unexamined load-bearing step.","tokens_in":2441,"tokens_out":400,"would_cite":false,"duration_ms":20324,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A 7B DecomposeRL policy trained on 5K curated claims matches 32B baselines and GPT-4.1-mini on claim verification while producing inspectable traces.","keywords":["claim verification","reinforcement learning","question decomposition","semi-supervised learning","traceable reasoning","GRPO","data curation"],"falsifier":"Measuring balanced accuracy of the released 7B model on a fresh collection of claims drawn from domains absent from the original 11 benchmarks.","tokens_in":2669,"feed_emoji":"🔍","tokens_out":692,"duration_ms":21422,"temperature":0.7,"pith_summary":"The paper introduces DecomposeRL to close the gap between accurate but opaque end-to-end claim verifiers and traceable but weaker decomposition methods. It casts decomposition as an RL policy optimized via GRPO together with a multi-faceted reward that rewards useful, informative, and diverse questions. A curation funnel compresses 115K claims into a 5K subset that supports both full supervision and semi-supervised training from unlabeled data. The resulting 7B model reaches 86.3 in-domain and 69.8 out-of-domain balanced accuracy across 11 biomedical, political, scientific, and general benchmarks.","feed_headline":"7B RL model matches 32B baselines on claim verification","feed_subtitle":"DecomposeRL trains a policy to break claims into inspectable questions using only 5K curated examples and reaches 86.3 in-domain accuracy.","key_machinery":"GRPO-trained RL policy with multi-faceted reward ensemble that learns to generate useful, informative, and diverse questions for decomposing claims.","core_discovery":"DecomposeRL treats claim decomposition as an RL policy trained with GRPO and a multi-faceted reward ensemble; a data-curation funnel reduces 115K fact-verification claims to a 5K subset; the resulting 7B policy achieves 86.3 in-domain and 69.8 out-of-domain balanced accuracy on 11 benchmarks, matching 32B and GPT-4.1-mini models while also outperforming baselines in a semi-supervised regime that uses only 10 percent labeled data.","pith_inferences":["The same RL decomposition approach could be applied to other multi-step reasoning tasks that benefit from inspectable intermediate steps.","Generated question traces could be used directly by human fact-checkers to audit model decisions.","Varying the curation funnel's selection criteria might allow further reduction in the number of required labeled examples.","Combining the policy with larger base models could produce additional accuracy gains without retraining the full pipeline."],"forward_implications":["Traceable decomposition becomes possible at the accuracy level of end-to-end classifiers.","Semi-supervised training works with only 10 percent labeled claims.","Model size can be reduced to 7B while still matching 32B and GPT-4.1-mini performance.","A compact curated dataset of roughly 5K examples suffices for strong generalization."],"fun_headline_variants":["DecomposeRL 7B matches 32B baselines with 5K examples","RL policy DecomposeRL achieves 69.8 out-of-domain accuracy","5K claims suffice for DecomposeRL 7B to match GPT-4.1-mini","Curated 5K dataset trains 7B DecomposeRL to 86.3 in-domain accuracy"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The curation funnel that reduces 115K claims to 5K claims retains sufficient learning signal for both in-domain and out-of-domain generalization without overfitting to the curation criteria or the benchmarks.","fun_headline_variants_meta":{"raw":{"variants":["DecomposeRL 7B matches 32B baselines with 5K examples","RL policy DecomposeRL achieves 69.8 out-of-domain accuracy","5K claims suffice for DecomposeRL 7B to match GPT-4.1-mini","Curated 5K dataset trains 7B DecomposeRL to 86.3 in-domain accuracy"]},"model":"grok-4.3","cost_usd":0.004999,"raw_usage":{"total_tokens":2462,"prompt_tokens":710,"num_sources_used":0,"completion_tokens":91,"cost_in_usd_ticks":49987000,"prompt_tokens_details":{"text_tokens":710,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1661,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":710,"tokens_out":91,"duration_ms":21787,"temperature":1.0,"reasoning_tokens":1661,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T13:36:50.335589+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Measuring balanced accuracy of the released 7B model on a fresh collection of claims drawn from domains absent from the original 11 benchmarks.","supporting_citations":[],"review_version":1}