{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2025:DUAN2XXI3DPW6Q2HDAISXP3W5T","short_pith_number":"pith:DUAN2XXI","schema_version":"1.0","canonical_sha256":"1d00dd5ee8d8df6f434718112bbf76ecc8455f71c08167e98509fd90e8c6f544","source":{"kind":"arxiv","id":"2506.15050","version":1},"attestation_state":"computed","paper":{"title":"Truncated Proximal Policy Optimization","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":[],"primary_cat":"cs.AI","authors_text":"Bole Ma, Chengyi Wang, Chi Zhang, Cong Xie, Gaohong Liu, Haotian Zhou, Jiaze Chen, Lingjun Liu, Lin Yan, Mingxuan Wang, Mofan Zhang, Qiying Yu, Ruidong Zhu, Ruofei Zhu, Ru Zhang, Tiantian Fan, Xiaochen Zuo, Xin Liu, Yonghui Wu, Yufeng Yuan, Yu Yue, Zhiqi Lin, Zhi Zhang","submitted_at":"2025-06-18T01:21:38Z","abstract_excerpt":"Recently, test-time scaling Large Language Models (LLMs) have demonstrated exceptional reasoning capabilities across scientific and professional tasks by generating long chains-of-thought (CoT). As a crucial component for developing these reasoning models, reinforcement learning (RL), exemplified by Proximal Policy Optimization (PPO) and its variants, allows models to learn through trial and error. However, PPO can be time-consuming due to its inherent on-policy nature, which is further exacerbated by increasing response lengths. In this work, we propose Truncated Proximal Policy Optimization "},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2506.15050","kind":"arxiv","version":1},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.AI","submitted_at":"2025-06-18T01:21:38Z","cross_cats_sorted":[],"title_canon_sha256":"24556348c2a59c43a1b5bab0c0b269cd890d6e6ba8916a8b597381bd1587a3aa","abstract_canon_sha256":"c947681d076feccf8b67590d8936d0eecd2db400b695417732fe8c6c3c9a5d71"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T11:23:36.233672Z","signature_b64":"2MCt2ZiA5E0QaFsZA4Lm+dT0GaOmEUw5Qt1RuPfw0GdoXAdh1tbhmFnlCFeEKMTCnmGY7JmMKDqBFoKlk4MIBg==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"1d00dd5ee8d8df6f434718112bbf76ecc8455f71c08167e98509fd90e8c6f544","last_reissued_at":"2026-07-05T11:23:36.233100Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T11:23:36.233100Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Truncated Proximal Policy Optimization","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":[],"primary_cat":"cs.AI","authors_text":"Bole Ma, Chengyi Wang, Chi Zhang, Cong Xie, Gaohong Liu, Haotian Zhou, Jiaze Chen, Lingjun Liu, Lin Yan, Mingxuan Wang, Mofan Zhang, Qiying Yu, Ruidong Zhu, Ruofei Zhu, Ru Zhang, Tiantian Fan, Xiaochen Zuo, Xin Liu, Yonghui Wu, Yufeng Yuan, Yu Yue, Zhiqi Lin, Zhi Zhang","submitted_at":"2025-06-18T01:21:38Z","abstract_excerpt":"Recently, test-time scaling Large Language Models (LLMs) have demonstrated exceptional reasoning capabilities across scientific and professional tasks by generating long chains-of-thought (CoT). As a crucial component for developing these reasoning models, reinforcement learning (RL), exemplified by Proximal Policy Optimization (PPO) and its variants, allows models to learn through trial and error. However, PPO can be time-consuming due to its inherent on-policy nature, which is further exacerbated by increasing response lengths. In this work, we propose Truncated Proximal Policy Optimization "},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2506.15050","kind":"arxiv","version":1},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2506.15050/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2506.15050","created_at":"2026-07-05T11:23:36.233168+00:00"},{"alias_kind":"arxiv_version","alias_value":"2506.15050v1","created_at":"2026-07-05T11:23:36.233168+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2506.15050","created_at":"2026-07-05T11:23:36.233168+00:00"},{"alias_kind":"pith_short_12","alias_value":"DUAN2XXI3DPW","created_at":"2026-07-05T11:23:36.233168+00:00"},{"alias_kind":"pith_short_16","alias_value":"DUAN2XXI3DPW6Q2H","created_at":"2026-07-05T11:23:36.233168+00:00"},{"alias_kind":"pith_short_8","alias_value":"DUAN2XXI","created_at":"2026-07-05T11:23:36.233168+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":8,"internal_anchor_count":0,"sample":[{"citing_arxiv_id":"2606.21943","citing_title":"Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learning","ref_index":44,"is_internal_anchor":false},{"citing_arxiv_id":"2606.09471","citing_title":"Escaping the KL Agreement Trap in On-Policy Distillation","ref_index":25,"is_internal_anchor":false},{"citing_arxiv_id":"2606.01249","citing_title":"Trust Region On-Policy Distillation","ref_index":249,"is_internal_anchor":false},{"citing_arxiv_id":"2606.27580","citing_title":"Retroactive Advantage Correction: Closed-Form V-Trace Bias Correction for Delay-Aware RLHF","ref_index":6,"is_internal_anchor":false},{"citing_arxiv_id":"2605.27736","citing_title":"Explicit Critic Guidance for Aligning Diffusion Models","ref_index":15,"is_internal_anchor":false},{"citing_arxiv_id":"2509.08827","citing_title":"A Survey of Reinforcement Learning for Large Reasoning Models","ref_index":131,"is_internal_anchor":false},{"citing_arxiv_id":"2503.09567","citing_title":"Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models","ref_index":177,"is_internal_anchor":false},{"citing_arxiv_id":"2604.10701","citing_title":"Bringing Value Models Back: Generative Critics for Value Modeling in LLM Reinforcement Learning","ref_index":18,"is_internal_anchor":false}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/DUAN2XXI3DPW6Q2HDAISXP3W5T","json":"https://pith.science/pith/DUAN2XXI3DPW6Q2HDAISXP3W5T.json","graph_json":"https://pith.science/api/pith-number/DUAN2XXI3DPW6Q2HDAISXP3W5T/graph.json","events_json":"https://pith.science/api/pith-number/DUAN2XXI3DPW6Q2HDAISXP3W5T/events.json","paper":"https://pith.science/paper/DUAN2XXI"},"agent_actions":{"view_html":"https://pith.science/pith/DUAN2XXI3DPW6Q2HDAISXP3W5T","download_json":"https://pith.science/pith/DUAN2XXI3DPW6Q2HDAISXP3W5T.json","view_paper":"https://pith.science/paper/DUAN2XXI","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2506.15050&json=true","fetch_graph":"https://pith.science/api/pith-number/DUAN2XXI3DPW6Q2HDAISXP3W5T/graph.json","fetch_events":"https://pith.science/api/pith-number/DUAN2XXI3DPW6Q2HDAISXP3W5T/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/DUAN2XXI3DPW6Q2HDAISXP3W5T/action/timestamp_anchor","attest_storage":"https://pith.science/pith/DUAN2XXI3DPW6Q2HDAISXP3W5T/action/storage_attestation","attest_author":"https://pith.science/pith/DUAN2XXI3DPW6Q2HDAISXP3W5T/action/author_attestation","sign_citation":"https://pith.science/pith/DUAN2XXI3DPW6Q2HDAISXP3W5T/action/citation_signature","submit_replication":"https://pith.science/pith/DUAN2XXI3DPW6Q2HDAISXP3W5T/action/replication_record"}},"created_at":"2026-07-05T11:23:36.233168+00:00","updated_at":"2026-07-05T11:23:36.233168+00:00"}