{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2024:NQJTNYMIKV2COGVMGH3T2L434C","short_pith_number":"pith:NQJTNYMI","schema_version":"1.0","canonical_sha256":"6c1336e1885574271aac31f73d2f9be0be7973e1d05b2fed82916d8eb20ec3db","source":{"kind":"arxiv","id":"2407.16216","version":4},"attestation_state":"computed","paper":{"title":"Reinforcement Learning for LLM Post-Training: A Survey","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":[],"primary_cat":"cs.CL","authors_text":"Bin Bi, Kiran Ramnath, Na (Claire) Cheng, Shiva Kumar Pentyala, Shubham Mehrotra, Sitaram Asur, Sougata Chaudhuri, Xiang-Bo Mao, Zhichao Wang, Zixu (James) Zhu","submitted_at":"2024-07-23T06:45:52Z","abstract_excerpt":"Large language models (LLMs) trained via pretraining and supervised fine-tuning (SFT) can still produce harmful and misaligned outputs, or struggle in domains like math and coding. Reinforcement learning (RL)-based post-training methods, including Reinforcement Learning from Human Feedback (RLHF) methods like Direct Preference Optimization (DPO) and Reinforcement Learning with Verifiable Rewards (RLVR) approaches like PPO and GRPO, have made remarkable gains to alleviate these issues. Yet, no existing work offers a technically detailed comparison of the various methods driving this progress. I"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2407.16216","kind":"arxiv","version":4},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.CL","submitted_at":"2024-07-23T06:45:52Z","cross_cats_sorted":[],"title_canon_sha256":"bfec645c6a4739bb121af3462e0aafbe5a72d5ae773b578fbab24860c6d8d48c","abstract_canon_sha256":"75f29f80c71bf34cb9ade4953861a2bc9667cce7a8189fa8eb6a8cc0e6c3edf2"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-20T00:02:46.591068Z","signature_b64":"NvAjAEZTPIwEJk74RBzanemryi6cyykdmIrRHuicKUAixz+IgM+sFBzoqhIQYbl41jxrdDoGISvnJHh+3bOJDA==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"6c1336e1885574271aac31f73d2f9be0be7973e1d05b2fed82916d8eb20ec3db","last_reissued_at":"2026-05-20T00:02:46.590504Z","signature_status":"signed_v1","first_computed_at":"2026-05-20T00:02:46.590504Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Reinforcement Learning for LLM Post-Training: A Survey","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":[],"primary_cat":"cs.CL","authors_text":"Bin Bi, Kiran Ramnath, Na (Claire) Cheng, Shiva Kumar Pentyala, Shubham Mehrotra, Sitaram Asur, Sougata Chaudhuri, Xiang-Bo Mao, Zhichao Wang, Zixu (James) Zhu","submitted_at":"2024-07-23T06:45:52Z","abstract_excerpt":"Large language models (LLMs) trained via pretraining and supervised fine-tuning (SFT) can still produce harmful and misaligned outputs, or struggle in domains like math and coding. Reinforcement learning (RL)-based post-training methods, including Reinforcement Learning from Human Feedback (RLHF) methods like Direct Preference Optimization (DPO) and Reinforcement Learning with Verifiable Rewards (RLVR) approaches like PPO and GRPO, have made remarkable gains to alleviate these issues. Yet, no existing work offers a technically detailed comparison of the various methods driving this progress. I"},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2407.16216","kind":"arxiv","version":4},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2407.16216/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2407.16216","created_at":"2026-05-20T00:02:46.590599+00:00"},{"alias_kind":"arxiv_version","alias_value":"2407.16216v4","created_at":"2026-05-20T00:02:46.590599+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2407.16216","created_at":"2026-05-20T00:02:46.590599+00:00"},{"alias_kind":"pith_short_12","alias_value":"NQJTNYMIKV2C","created_at":"2026-05-20T00:02:46.590599+00:00"},{"alias_kind":"pith_short_16","alias_value":"NQJTNYMIKV2COGVM","created_at":"2026-05-20T00:02:46.590599+00:00"},{"alias_kind":"pith_short_8","alias_value":"NQJTNYMI","created_at":"2026-05-20T00:02:46.590599+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":23,"internal_anchor_count":23,"sample":[{"citing_arxiv_id":"2510.08141","citing_title":"SCOPE-RL: Stable and Quantitative Control of Policy Entropy in RL Post-Training","ref_index":19,"is_internal_anchor":true},{"citing_arxiv_id":"2605.18799","citing_title":"ReCrit: Transition-Aware Reinforcement Learning for Scientific Critic Reasoning","ref_index":37,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16776","citing_title":"Distinguishable Deletion: Unifying Knowledge Erasure and Refusal for Large Language Model Unlearning","ref_index":73,"is_internal_anchor":true},{"citing_arxiv_id":"2604.27859","citing_title":"Rethinking Agentic Reinforcement Learning In Large Language Models","ref_index":95,"is_internal_anchor":true},{"citing_arxiv_id":"2506.01770","citing_title":"ReGA: Model-Based Safeguard for LLMs via Representation-Guided Abstraction","ref_index":41,"is_internal_anchor":true},{"citing_arxiv_id":"2506.12382","citing_title":"Exploring the Secondary Risks of Large Language Models","ref_index":47,"is_internal_anchor":true},{"citing_arxiv_id":"2508.16771","citing_title":"EyeMulator: Improving Code Language Models by Mimicking Human Visual Attention","ref_index":65,"is_internal_anchor":true},{"citing_arxiv_id":"2508.20697","citing_title":"Token Buncher: Shielding LLMs from Harmful Reinforcement Learning Fine-Tuning","ref_index":52,"is_internal_anchor":true},{"citing_arxiv_id":"2509.02547","citing_title":"The Landscape of Agentic Reinforcement Learning for LLMs: A Survey","ref_index":14,"is_internal_anchor":true},{"citing_arxiv_id":"2509.07794","citing_title":"Query Expansion in the Age of Pre-trained and Large Language Models: A Comprehensive Survey","ref_index":110,"is_internal_anchor":true},{"citing_arxiv_id":"2602.02280","citing_title":"RACC: Representation-Aware Coverage Criteria for LLM Safety Testing","ref_index":51,"is_internal_anchor":true},{"citing_arxiv_id":"2602.07892","citing_title":"Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projection","ref_index":22,"is_internal_anchor":true},{"citing_arxiv_id":"2603.18113","citing_title":"VC-Soup: Value-Consistency Guided Multi-Value Alignment for Large Language Models","ref_index":36,"is_internal_anchor":true},{"citing_arxiv_id":"2605.11516","citing_title":"Agents Should Replace Narrow Predictive AI as the Orchestrator in 6G AI-RAN","ref_index":45,"is_internal_anchor":true},{"citing_arxiv_id":"2605.11549","citing_title":"UNIPO: Unified Interactive Visual Explanation for RL Fine-Tuning Policy Optimization","ref_index":32,"is_internal_anchor":true},{"citing_arxiv_id":"2604.27859","citing_title":"Rethinking Agentic Reinforcement Learning In Large Language Models","ref_index":95,"is_internal_anchor":true},{"citing_arxiv_id":"2604.27859","citing_title":"Rethinking Agentic Reinforcement Learning In Large Language Models","ref_index":95,"is_internal_anchor":true},{"citing_arxiv_id":"2604.24536","citing_title":"Generating Place-Based Compromises Between Two Points of View","ref_index":73,"is_internal_anchor":true},{"citing_arxiv_id":"2604.23543","citing_title":"Pref-CTRL: Preference Driven LLM Alignment using Representation Editing","ref_index":34,"is_internal_anchor":true},{"citing_arxiv_id":"2605.00553","citing_title":"Stable-GFlowNet: Toward Diverse and Robust LLM Red-Teaming via Contrastive Trajectory Balance","ref_index":9,"is_internal_anchor":true},{"citing_arxiv_id":"2604.19083","citing_title":"ProjLens: Unveiling the Role of Projectors in Multimodal Model Safety","ref_index":37,"is_internal_anchor":true},{"citing_arxiv_id":"2604.11259","citing_title":"Mobile GUI Agent Privacy Personalization with Trajectory Induced Preference Optimization","ref_index":33,"is_internal_anchor":true},{"citing_arxiv_id":"2604.07754","citing_title":"The Art of (Mis)alignment: How Fine-Tuning Methods Effectively Misalign and Realign LLMs in Post-Training","ref_index":61,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/NQJTNYMIKV2COGVMGH3T2L434C","json":"https://pith.science/pith/NQJTNYMIKV2COGVMGH3T2L434C.json","graph_json":"https://pith.science/api/pith-number/NQJTNYMIKV2COGVMGH3T2L434C/graph.json","events_json":"https://pith.science/api/pith-number/NQJTNYMIKV2COGVMGH3T2L434C/events.json","paper":"https://pith.science/paper/NQJTNYMI"},"agent_actions":{"view_html":"https://pith.science/pith/NQJTNYMIKV2COGVMGH3T2L434C","download_json":"https://pith.science/pith/NQJTNYMIKV2COGVMGH3T2L434C.json","view_paper":"https://pith.science/paper/NQJTNYMI","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2407.16216&json=true","fetch_graph":"https://pith.science/api/pith-number/NQJTNYMIKV2COGVMGH3T2L434C/graph.json","fetch_events":"https://pith.science/api/pith-number/NQJTNYMIKV2COGVMGH3T2L434C/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/NQJTNYMIKV2COGVMGH3T2L434C/action/timestamp_anchor","attest_storage":"https://pith.science/pith/NQJTNYMIKV2COGVMGH3T2L434C/action/storage_attestation","attest_author":"https://pith.science/pith/NQJTNYMIKV2COGVMGH3T2L434C/action/author_attestation","sign_citation":"https://pith.science/pith/NQJTNYMIKV2COGVMGH3T2L434C/action/citation_signature","submit_replication":"https://pith.science/pith/NQJTNYMIKV2COGVMGH3T2L434C/action/replication_record"}},"created_at":"2026-05-20T00:02:46.590599+00:00","updated_at":"2026-05-20T00:02:46.590599+00:00"}