{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2024:BQ4XRC6RV7IPENIK4YADW3PVC6","short_pith_number":"pith:BQ4XRC6R","schema_version":"1.0","canonical_sha256":"0c39788bd1afd0f2350ae6003b6df517b7e1007385e08f26ee503c587ab163fb","source":{"kind":"arxiv","id":"2410.10148","version":4},"attestation_state":"computed","paper":{"title":"AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":["cs.AI","cs.CL"],"primary_cat":"cs.LG","authors_text":"Bolin Ding, Jiancan Wu, Jinyang Gao, Junkang Wu, Xiangnan He, Xiang Wang, Xue Wang, Zhengyi Yang","submitted_at":"2024-10-14T04:29:57Z","abstract_excerpt":"Aligning large language models (LLMs) with human values and intentions is crucial for their utility, honesty, and safety. Reinforcement learning from human feedback (RLHF) is a popular approach to achieve this alignment, but it faces challenges in computational efficiency and training stability. Recent methods like Direct Preference Optimization (DPO) and Simple Preference Optimization (SimPO) have proposed offline alternatives to RLHF, simplifying the process by reparameterizing the reward function. However, DPO depends on a potentially suboptimal reference model, and SimPO's assumption of a "},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2410.10148","kind":"arxiv","version":4},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.LG","submitted_at":"2024-10-14T04:29:57Z","cross_cats_sorted":["cs.AI","cs.CL"],"title_canon_sha256":"afa32b7c7afdfb2c877608433d8f4fd86b1cd5b727cfea990d1e1ed920c98643","abstract_canon_sha256":"543767d8aa00ed680c13b35805cec6763b8f523970804daeb0bb121bd0d9e89d"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T11:39:44.057376Z","signature_b64":"ouZVzBicqxEHem9/jXZU7LvZbamJ1oDXhvwcXXGW1P91tzn/kxR/sfSW3eFn4ubspbtYhY+qcuRh7eBCS4ofBg==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"0c39788bd1afd0f2350ae6003b6df517b7e1007385e08f26ee503c587ab163fb","last_reissued_at":"2026-07-05T11:39:44.056820Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T11:39:44.056820Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":["cs.AI","cs.CL"],"primary_cat":"cs.LG","authors_text":"Bolin Ding, Jiancan Wu, Jinyang Gao, Junkang Wu, Xiangnan He, Xiang Wang, Xue Wang, Zhengyi Yang","submitted_at":"2024-10-14T04:29:57Z","abstract_excerpt":"Aligning large language models (LLMs) with human values and intentions is crucial for their utility, honesty, and safety. Reinforcement learning from human feedback (RLHF) is a popular approach to achieve this alignment, but it faces challenges in computational efficiency and training stability. Recent methods like Direct Preference Optimization (DPO) and Simple Preference Optimization (SimPO) have proposed offline alternatives to RLHF, simplifying the process by reparameterizing the reward function. However, DPO depends on a potentially suboptimal reference model, and SimPO's assumption of a "},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2410.10148","kind":"arxiv","version":4},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2410.10148/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2410.10148","created_at":"2026-07-05T11:39:44.056875+00:00"},{"alias_kind":"arxiv_version","alias_value":"2410.10148v4","created_at":"2026-07-05T11:39:44.056875+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2410.10148","created_at":"2026-07-05T11:39:44.056875+00:00"},{"alias_kind":"pith_short_12","alias_value":"BQ4XRC6RV7IP","created_at":"2026-07-05T11:39:44.056875+00:00"},{"alias_kind":"pith_short_16","alias_value":"BQ4XRC6RV7IPENIK","created_at":"2026-07-05T11:39:44.056875+00:00"},{"alias_kind":"pith_short_8","alias_value":"BQ4XRC6R","created_at":"2026-07-05T11:39:44.056875+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":9,"internal_anchor_count":0,"sample":[{"citing_arxiv_id":"2606.12505","citing_title":"Boosting Direct Preference Optimization with Penalization","ref_index":6,"is_internal_anchor":false},{"citing_arxiv_id":"2605.18721","citing_title":"General Preference Reinforcement Learning","ref_index":10,"is_internal_anchor":false},{"citing_arxiv_id":"2602.06239","citing_title":"Provably avoiding over-optimization in Direct Preference Optimization without knowing the data distribution","ref_index":32,"is_internal_anchor":false},{"citing_arxiv_id":"2605.18721","citing_title":"General Preference Reinforcement Learning","ref_index":10,"is_internal_anchor":false},{"citing_arxiv_id":"2605.18721","citing_title":"General Preference Reinforcement Learning","ref_index":10,"is_internal_anchor":false},{"citing_arxiv_id":"2602.06239","citing_title":"Provably avoiding over-optimization in Direct Preference Optimization without knowing the data distribution","ref_index":32,"is_internal_anchor":false},{"citing_arxiv_id":"2605.11679","citing_title":"Explaining and Breaking the Safety-Helpfulness Ceiling via Preference Dimensional Expansion","ref_index":80,"is_internal_anchor":false},{"citing_arxiv_id":"2605.11679","citing_title":"Explaining and Breaking the Safety-Helpfulness Ceiling via Preference Dimensional Expansion","ref_index":80,"is_internal_anchor":false},{"citing_arxiv_id":"2605.01462","citing_title":"LocalAlign: Enabling Generalizable Prompt Injection Defense via Generation of Near-Target Adversarial Examples for Alignment Training","ref_index":42,"is_internal_anchor":false}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/BQ4XRC6RV7IPENIK4YADW3PVC6","json":"https://pith.science/pith/BQ4XRC6RV7IPENIK4YADW3PVC6.json","graph_json":"https://pith.science/api/pith-number/BQ4XRC6RV7IPENIK4YADW3PVC6/graph.json","events_json":"https://pith.science/api/pith-number/BQ4XRC6RV7IPENIK4YADW3PVC6/events.json","paper":"https://pith.science/paper/BQ4XRC6R"},"agent_actions":{"view_html":"https://pith.science/pith/BQ4XRC6RV7IPENIK4YADW3PVC6","download_json":"https://pith.science/pith/BQ4XRC6RV7IPENIK4YADW3PVC6.json","view_paper":"https://pith.science/paper/BQ4XRC6R","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2410.10148&json=true","fetch_graph":"https://pith.science/api/pith-number/BQ4XRC6RV7IPENIK4YADW3PVC6/graph.json","fetch_events":"https://pith.science/api/pith-number/BQ4XRC6RV7IPENIK4YADW3PVC6/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/BQ4XRC6RV7IPENIK4YADW3PVC6/action/timestamp_anchor","attest_storage":"https://pith.science/pith/BQ4XRC6RV7IPENIK4YADW3PVC6/action/storage_attestation","attest_author":"https://pith.science/pith/BQ4XRC6RV7IPENIK4YADW3PVC6/action/author_attestation","sign_citation":"https://pith.science/pith/BQ4XRC6RV7IPENIK4YADW3PVC6/action/citation_signature","submit_replication":"https://pith.science/pith/BQ4XRC6RV7IPENIK4YADW3PVC6/action/replication_record"}},"created_at":"2026-07-05T11:39:44.056875+00:00","updated_at":"2026-07-05T11:39:44.056875+00:00"}