{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2026:7DDKR4XUTVPMLMVNBPZAA2FTN5","short_pith_number":"pith:7DDKR4XU","schema_version":"1.0","canonical_sha256":"f8c6a8f2f49d5ec5b2ad0bf20068b36f4e0483f4f460a5f7fd10d6ada7f80dcb","source":{"kind":"arxiv","id":"2605.13435","version":1},"attestation_state":"computed","paper":{"title":"Q-Flow: Stable and Expressive Reinforcement Learning with Flow-Based Policy","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Q-Flow stabilizes training of expressive flow-based policies in reinforcement learning by propagating terminal values backward along deterministic flow paths.","cross_cats":["cs.AI"],"primary_cat":"cs.LG","authors_text":"Byeongguk Jeon, JaeHyeok Doo, Kimin Lee, Minjoon Seo, Seonghyeon Ye","submitted_at":"2026-05-13T12:31:02Z","abstract_excerpt":"There is growing interest in utilizing flow-based models as decision-making policies in reinforcement learning due to their high expressive capacity. However, effectively leveraging this expressivity for value maximization remains challenging, as naive gradient-based optimization requires backpropagating through numerical solvers and often leads to instability. Existing approaches typically address this issue by restricting the expressive capacity of flow-based policies, resulting in a trade-off between optimization stability and representational flexibility. To resolve this, we introduce Q-Fl"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":false},"canonical_record":{"source":{"id":"2605.13435","kind":"arxiv","version":1},"metadata":{"license":"http://creativecommons.org/licenses/by/4.0/","primary_cat":"cs.LG","submitted_at":"2026-05-13T12:31:02Z","cross_cats_sorted":["cs.AI"],"title_canon_sha256":"5d8746ad1b114d8b8f642d3fc3e2b0905a72d645b9d317e72b75b91c473bb228","abstract_canon_sha256":"d3f3b4c1a13b3c5ce7999a8811fedb2756df7e952a1be1811ec7abc65d2dd2fc"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-18T02:44:47.114672Z","signature_b64":"JNIbH6dE3sDvlwhGVxXqOW8MXGK1a+x3ylQC2y2JUcapgV3cWSmk2glPp1DYoEUPhWV3a94Y3jB1WycwpAMIDQ==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"f8c6a8f2f49d5ec5b2ad0bf20068b36f4e0483f4f460a5f7fd10d6ada7f80dcb","last_reissued_at":"2026-05-18T02:44:47.114249Z","signature_status":"signed_v1","first_computed_at":"2026-05-18T02:44:47.114249Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Q-Flow: Stable and Expressive Reinforcement Learning with Flow-Based Policy","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Q-Flow stabilizes training of expressive flow-based policies in reinforcement learning by propagating terminal values backward along deterministic flow paths.","cross_cats":["cs.AI"],"primary_cat":"cs.LG","authors_text":"Byeongguk Jeon, JaeHyeok Doo, Kimin Lee, Minjoon Seo, Seonghyeon Ye","submitted_at":"2026-05-13T12:31:02Z","abstract_excerpt":"There is growing interest in utilizing flow-based models as decision-making policies in reinforcement learning due to their high expressive capacity. However, effectively leveraging this expressivity for value maximization remains challenging, as naive gradient-based optimization requires backpropagating through numerical solvers and often leads to instability. Existing approaches typically address this issue by restricting the expressive capacity of flow-based policies, resulting in a trade-off between optimization stability and representational flexibility. To resolve this, we introduce Q-Fl"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"Q-Flow leverages the deterministic nature of flow dynamics to explicitly propagate terminal trajectory value to intermediate latent states along the policy-induced flow, enabling stable policy optimization using intermediate value gradients without unrolling the numerical solver.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The assumption that propagating terminal trajectory value to intermediate latent states along the flow provides reliable gradients for policy optimization without introducing bias or instability from the flow matching process itself.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Q-Flow enables stable optimization of expressive flow-based policies in RL by propagating terminal values along deterministic flow dynamics to intermediate states for gradient updates without solver unrolling.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Q-Flow stabilizes training of expressive flow-based policies in reinforcement learning by propagating terminal values backward along deterministic flow paths.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"37cf3ed8a08477a8eb5c6b3e93179c95c43ca8af44a7f5ece25319303346ee1b"},"source":{"id":"2605.13435","kind":"arxiv","version":1},"verdict":{"id":"cfa19716-0d98-4d27-b0f7-20e4d038bf5a","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-14T19:22:59.970072Z","strongest_claim":"Q-Flow leverages the deterministic nature of flow dynamics to explicitly propagate terminal trajectory value to intermediate latent states along the policy-induced flow, enabling stable policy optimization using intermediate value gradients without unrolling the numerical solver.","one_line_summary":"Q-Flow enables stable optimization of expressive flow-based policies in RL by propagating terminal values along deterministic flow dynamics to intermediate states for gradient updates without solver unrolling.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The assumption that propagating terminal trajectory value to intermediate latent states along the flow provides reliable gradients for policy optimization without introducing bias or instability from the flow matching process itself.","pith_extraction_headline":"Q-Flow stabilizes training of expressive flow-based policies in reinforcement learning by propagating terminal values backward along deterministic flow paths."},"references":{"count":26,"sample":[{"doi":"","year":null,"title":"Diffusion guidance is a controllable policy improvement operator.arXiv preprint arXiv:2505.23458","work_id":"85c1b3dc-42d2-4b34-9b85-9da92334b592","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2004,"title":"D4RL: Datasets for Deep Data-Driven Reinforcement Learning","work_id":"47082e4e-a4a5-418b-bf4f-4667355065fc","ref_index":2,"cited_arxiv_id":"2004.07219","is_internal_anchor":true},{"doi":"","year":null,"title":"IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies","work_id":"913326e6-9ea4-4974-b2d7-ff53984b387f","ref_index":3,"cited_arxiv_id":"2304.10573","is_internal_anchor":true},{"doi":"","year":null,"title":"AlignIQL: Policy alignment in implicit q-learning through constrained optimization","work_id":"53133ba5-a064-47f1-90f9-aae89fe61013","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":null,"title":"Gaussian Error Linear Units (GELUs)","work_id":"0466fd22-03a1-4a61-af0a-a900e77bb023","ref_index":5,"cited_arxiv_id":"1606.08415","is_internal_anchor":true}],"resolved_work":26,"snapshot_sha256":"f84d1fa840b0d5c471e83f8ba7b2a728281f1c250fe82409ed74770d39453e9c","internal_anchors":8},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2605.13435","created_at":"2026-05-18T02:44:47.114323+00:00"},{"alias_kind":"arxiv_version","alias_value":"2605.13435v1","created_at":"2026-05-18T02:44:47.114323+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2605.13435","created_at":"2026-05-18T02:44:47.114323+00:00"},{"alias_kind":"pith_short_12","alias_value":"7DDKR4XUTVPM","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_16","alias_value":"7DDKR4XUTVPMLMVN","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_8","alias_value":"7DDKR4XU","created_at":"2026-05-18T12:33:37.589309+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":0,"internal_anchor_count":0,"sample":[]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/7DDKR4XUTVPMLMVNBPZAA2FTN5","json":"https://pith.science/pith/7DDKR4XUTVPMLMVNBPZAA2FTN5.json","graph_json":"https://pith.science/api/pith-number/7DDKR4XUTVPMLMVNBPZAA2FTN5/graph.json","events_json":"https://pith.science/api/pith-number/7DDKR4XUTVPMLMVNBPZAA2FTN5/events.json","paper":"https://pith.science/paper/7DDKR4XU"},"agent_actions":{"view_html":"https://pith.science/pith/7DDKR4XUTVPMLMVNBPZAA2FTN5","download_json":"https://pith.science/pith/7DDKR4XUTVPMLMVNBPZAA2FTN5.json","view_paper":"https://pith.science/paper/7DDKR4XU","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2605.13435&json=true","fetch_graph":"https://pith.science/api/pith-number/7DDKR4XUTVPMLMVNBPZAA2FTN5/graph.json","fetch_events":"https://pith.science/api/pith-number/7DDKR4XUTVPMLMVNBPZAA2FTN5/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/7DDKR4XUTVPMLMVNBPZAA2FTN5/action/timestamp_anchor","attest_storage":"https://pith.science/pith/7DDKR4XUTVPMLMVNBPZAA2FTN5/action/storage_attestation","attest_author":"https://pith.science/pith/7DDKR4XUTVPMLMVNBPZAA2FTN5/action/author_attestation","sign_citation":"https://pith.science/pith/7DDKR4XUTVPMLMVNBPZAA2FTN5/action/citation_signature","submit_replication":"https://pith.science/pith/7DDKR4XUTVPMLMVNBPZAA2FTN5/action/replication_record"}},"created_at":"2026-05-18T02:44:47.114323+00:00","updated_at":"2026-05-18T02:44:47.114323+00:00"}