Pith

open record

sign in

arxiv: 2506.14965 · v1 · pith:SK4MJSGS · submitted 2025-06-17 · cs.LG · cs.AI· cs.CL

Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective

Zhoujun Cheng , Shibo Hao , Tianyang Liu , Fan Zhou , Yutao Xie , Feng Yao , Yuexin Bian , Yonghao Zhuang
show 16 more authors
Nilabjo Dey Yuheng Zha Yi Gu Kun Zhou Yuqi Wang Yuan Li Richard Fan Jianshu She Chengqian Gao Abulhair Saparov Haonan Li Taylor W. Killian Mikhail Yurochkin Zhengzhong Liu Eric P. Xing Zhiting Hu
This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:SK4MJSGSrecord.jsonopen to challenge →

classification cs.LG cs.AIcs.CL
keywords reasoningmodelsdomainscodetrainingacrossdataperformance
0
0 comments X
read the original abstract

Reinforcement learning (RL) has emerged as a promising approach to improve large language model (LLM) reasoning, yet most open efforts focus narrowly on math and code, limiting our understanding of its broader applicability to general reasoning. A key challenge lies in the lack of reliable, scalable RL reward signals across diverse reasoning domains. We introduce Guru, a curated RL reasoning corpus of 92K verifiable examples spanning six reasoning domains--Math, Code, Science, Logic, Simulation, and Tabular--each built through domain-specific reward design, deduplication, and filtering to ensure reliability and effectiveness for RL training. Based on Guru, we systematically revisit established findings in RL for LLM reasoning and observe significant variation across domains. For example, while prior work suggests that RL primarily elicits existing knowledge from pretrained models, our results reveal a more nuanced pattern: domains frequently seen during pretraining (Math, Code, Science) easily benefit from cross-domain RL training, while domains with limited pretraining exposure (Logic, Simulation, and Tabular) require in-domain training to achieve meaningful performance gains, suggesting that RL is likely to facilitate genuine skill acquisition. Finally, we present Guru-7B and Guru-32B, two models that achieve state-of-the-art performance among open models RL-trained with publicly available data, outperforming best baselines by 7.9% and 6.7% on our 17-task evaluation suite across six reasoning domains. We also show that our models effectively improve the Pass@k performance of their base models, particularly on complex tasks less likely to appear in pretraining data. We release data, models, training and evaluation code to facilitate general-purpose reasoning at: https://github.com/LLM360/Reasoning360

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Backdoors in RLVR: Jailbreak Backdoors in LLMs From Verifiable Reward

    cs.CR 2026-04 accept novelty 7.0

    RLVR can be backdoored with under 2% poisoned data using an asymmetric reward trigger, implanting jailbreaks that cut safety performance by 73% on average without harming benign tasks.

  2. Blending Supervised and Reinforcement Fine-Tuning with Prefix Sampling

    cs.LG 2025-07 unverdicted novelty 7.0

    Prefix-RFT blends SFT and RFT via prefix sampling from demonstrations to outperform standalone SFT, RFT, and mixed-policy baselines on math reasoning problems.

  3. From Reasoning Traces to Reusable Modules: Understanding Compositional Generalization in Language Model Reasoning

    cs.LG 2026-06 unverdicted novelty 6.0

    Introduces a hierarchical latent selection model showing SFT supplies raw module materials in compound traces while RL decomposes them to identify atomic modules and enable recombination for new reasoning configurations.

  4. VIMPO: Value-Implicit Policy Optimization for LLMs

    cs.LG 2026-06 unverdicted novelty 5.0

    VIMPO derives a policy-implied value function from optimality conditions for critic-free RL in LLMs and shows gains over GRPO on math benchmarks.

  5. CARE-RL: Capability-Aware Reinforcement Learning for Mitigating Cross-Domain Conflicts

    cs.LG 2026-05 unverdicted novelty 5.0

    CARE-RL combines PA-GRM for task-adaptive rewards on open-ended tasks and DACSP for modulating RL updates using historical capability directions, reporting higher total average scores than baselines on Qwen models.

  6. Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models

    cs.AI 2025-03 unverdicted novelty 5.0

    The paper unifies perspectives on Long CoT in reasoning LLMs by introducing a taxonomy, detailing characteristics of deep reasoning and reflection, and discussing emergence phenomena and future directions.

  7. A Survey of Reinforcement Learning for Large Reasoning Models

    cs.CL 2025-09 accept novelty 3.0

    A survey compiling RL methods, challenges, data resources, and applications for enhancing reasoning in large language models and large reasoning models since DeepSeek-R1.