REVIEW 5 cited by
VerIF: Verification Engineering for Reinforcement Learning in Instruction Following
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
VerIF: Verification Engineering for Reinforcement Learning in Instruction Following
read the original abstract
Reinforcement learning with verifiable rewards (RLVR) has become a key technique for enhancing large language models (LLMs), with verification engineering playing a central role. However, best practices for RL in instruction following remain underexplored. In this work, we explore the verification challenge in RL for instruction following and propose VerIF, a verification method that combines rule-based code verification with LLM-based verification from a large reasoning model (e.g., QwQ-32B). To support this approach, we construct a high-quality instruction-following dataset, VerInstruct, containing approximately 22,000 instances with associated verification signals. We apply RL training with VerIF to two models, achieving significant improvements across several representative instruction-following benchmarks. The trained models reach state-of-the-art performance among models of comparable size and generalize well to unseen constraints. We further observe that their general capabilities remain unaffected, suggesting that RL with VerIF can be integrated into existing RL recipes to enhance overall model performance. We have released our datasets, codes, and models to facilitate future research at https://github.com/THU-KEG/VerIF.
Forward citations
Cited by 5 Pith papers
-
LsrIF: Enhancing Logic-Structured Instruction Following of Large Language Models
Logic-structured rewards—averaging parallel constraints, decaying rewards after sequential failures, rewarding only the active conditional branch—improve instruction-following and transfer to reasoning.
-
Reasoning Up the Instruction Ladder for Controllable Language Models
RLVR on ~7K verifiable system/user conflict examples teaches LLMs to prioritize higher-priority instructions, improving instruction-hierarchy and safety benchmarks.
-
Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models
A dynamic replay and reweighting scheduler (RECAP) preserves general capabilities during RLVR while keeping reasoning performance at least as good as reasoning-only finetuning.
-
ComplexConstraints and Beyond: Expert Rubrics for RLVR
Expert atomic rubrics used as RL rewards lift a 4B model by +15.5 pp on ComplexConstraints and transfer gains to AdvancedIF, MultiChallenge, and agentic tool benchmarks.
-
ComplexConstraints and Beyond: Expert Rubrics for RLVR
Expert-curated rubrics in the new ComplexConstraints dataset improve LLM instruction following by 12-15% when used as RL training signals, with gains transferring to out-of-distribution agentic benchmarks.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.