{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2020:BHQFDYQ2RTFWVTYAKRI7G74UWI","short_pith_number":"pith:BHQFDYQ2","schema_version":"1.0","canonical_sha256":"09e051e21a8ccb6acf005451f37f94b213a0d6416acab0368a6b78956a12c20d","source":{"kind":"arxiv","id":"2006.09359","version":6},"attestation_state":"computed","paper":{"title":"AWAC: Accelerating Online Reinforcement Learning with Offline Datasets","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"AWAC combines offline data with online reinforcement learning to accelerate policy improvement for robotic control.","cross_cats":["cs.RO","stat.ML"],"primary_cat":"cs.LG","authors_text":"Abhishek Gupta, Ashvin Nair, Murtaza Dalal, Sergey Levine","submitted_at":"2020-06-16T17:54:41Z","abstract_excerpt":"Reinforcement learning (RL) provides an appealing formalism for learning control policies from experience. However, the classic active formulation of RL necessitates a lengthy active exploration process for each behavior, making it difficult to apply in real-world settings such as robotic control. If we can instead allow RL algorithms to effectively use previously collected data to aid the online learning process, such applications could be made substantially more practical: the prior data would provide a starting point that mitigates challenges due to exploration and sample complexity, while "},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":true},"canonical_record":{"source":{"id":"2006.09359","kind":"arxiv","version":6},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.LG","submitted_at":"2020-06-16T17:54:41Z","cross_cats_sorted":["cs.RO","stat.ML"],"title_canon_sha256":"38086df4404bc2a7ae68618bd5ae7153e26e52005051f55df074ef5d29987275","abstract_canon_sha256":"7edf47029291ba308a8ace796f21d0a8e98d5b5aad6934bfaaa22dee2edff7c1"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T02:34:40.161882Z","signature_b64":"iqZyKHR0lODin9DZxejjz4PDfZM8SzPXIb4UiWSOMRuC30WuBcwRWEEGsJTWZ76XrgrhhPoer8v2WEoOGc+CBA==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"09e051e21a8ccb6acf005451f37f94b213a0d6416acab0368a6b78956a12c20d","last_reissued_at":"2026-07-05T02:34:40.161435Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T02:34:40.161435Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"AWAC: Accelerating Online Reinforcement Learning with Offline Datasets","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"AWAC combines offline data with online reinforcement learning to accelerate policy improvement for robotic control.","cross_cats":["cs.RO","stat.ML"],"primary_cat":"cs.LG","authors_text":"Abhishek Gupta, Ashvin Nair, Murtaza Dalal, Sergey Levine","submitted_at":"2020-06-16T17:54:41Z","abstract_excerpt":"Reinforcement learning (RL) provides an appealing formalism for learning control policies from experience. However, the classic active formulation of RL necessitates a lengthy active exploration process for each behavior, making it difficult to apply in real-world settings such as robotic control. If we can instead allow RL algorithms to effectively use previously collected data to aid the online learning process, such applications could be made substantially more practical: the prior data would provide a starting point that mitigates challenges due to exploration and sample complexity, while "},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"Our method, advantage weighted actor critic (AWAC), enables rapid learning of skills with a combination of prior demonstration data and online experience.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That offline data (expert or sub-optimal) can be leveraged via AWAC to bootstrap online RL without the typical difficulties in transitioning from offline to online training remaining insurmountable.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"AWAC combines offline data with online RL via advantage-weighted actor-critic updates to enable faster acquisition of robotic skills such as dexterous manipulation.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"AWAC combines offline data with online reinforcement learning to accelerate policy improvement for robotic control.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"e6c6f9aea8f84eddca047d174c1f67e477db374f3ebb05a4541e9d6356e17014"},"source":{"id":"2006.09359","kind":"arxiv","version":6},"verdict":{"id":"1463b969-c79f-42f6-bb2e-086f13c19d3f","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-13T03:28:40.431855Z","strongest_claim":"Our method, advantage weighted actor critic (AWAC), enables rapid learning of skills with a combination of prior demonstration data and online experience.","one_line_summary":"AWAC combines offline data with online RL via advantage-weighted actor-critic updates to enable faster acquisition of robotic skills such as dexterous manipulation.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That offline data (expert or sub-optimal) can be leveraged via AWAC to bootstrap online RL without the typical difficulties in transitioning from offline to online training remaining insurmountable.","pith_extraction_headline":"AWAC combines offline data with online reinforcement learning to accelerate policy improvement for robotic control."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2006.09359/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":68,"sample":[{"doi":"","year":2004,"title":"Apprenticeship learning via inverse reinforcement learning","work_id":"55f5d70f-5791-410a-a5b1-939094feb422","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2018,"title":"Maximum a Posteriori Policy Optimisation","work_id":"9317bc09-2729-4016-870c-ed365cd5acfc","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2019,"title":"An Optimistic Perspective on Offline Reinforce- ment Learning","work_id":"189fbeac-cf4a-4029-8af1-8f6d322576bf","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2019,"title":"ROBEL: Robotics Benchmarks for Learning with Low- Cost Robots","work_id":"3747b14e-b759-4712-b12e-754105ca6191","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":1997,"title":"Robot Learning From Demonstration","work_id":"0f19df2c-c43a-475e-b93e-bda79a70b173","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":68,"snapshot_sha256":"14c42ef49f296fe18b1a5fe7559f5e58d15f9680f32b88eb914d1075354688f3","internal_anchors":1},"formal_canon":{"evidence_count":2,"snapshot_sha256":"ce12733f1ca213ba25889a848a3babd844807356c3da06d368173def46f2c7a8"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2006.09359","created_at":"2026-07-05T02:34:40.161500+00:00"},{"alias_kind":"arxiv_version","alias_value":"2006.09359v6","created_at":"2026-07-05T02:34:40.161500+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2006.09359","created_at":"2026-07-05T02:34:40.161500+00:00"},{"alias_kind":"pith_short_12","alias_value":"BHQFDYQ2RTFW","created_at":"2026-07-05T02:34:40.161500+00:00"},{"alias_kind":"pith_short_16","alias_value":"BHQFDYQ2RTFWVTYA","created_at":"2026-07-05T02:34:40.161500+00:00"},{"alias_kind":"pith_short_8","alias_value":"BHQFDYQ2","created_at":"2026-07-05T02:34:40.161500+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":70,"internal_anchor_count":70,"sample":[{"citing_arxiv_id":"2606.25527","citing_title":"Beyond One-Size-Fits-All: Diagnosis-Driven Online Reinforcement Learning with Offline Priors","ref_index":49,"is_internal_anchor":true},{"citing_arxiv_id":"2606.24633","citing_title":"Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot Imitation","ref_index":35,"is_internal_anchor":true},{"citing_arxiv_id":"2606.27163","citing_title":"Learning to Fold: prizewinning solution at LeHome Challenge 2026 (1st place online, 2nd offline)","ref_index":9,"is_internal_anchor":true},{"citing_arxiv_id":"2606.19656","citing_title":"DF-ExpEnse: Diffusion Filtered Exploration for Sample Efficient Finetuning","ref_index":9,"is_internal_anchor":true},{"citing_arxiv_id":"2606.17551","citing_title":"Reversal Q-Learning","ref_index":16,"is_internal_anchor":true},{"citing_arxiv_id":"2606.12406","citing_title":"FACTR 2: Learning External Force Sensing for Commodity Robot Arms Improves Policy Learning","ref_index":30,"is_internal_anchor":true},{"citing_arxiv_id":"2606.11982","citing_title":"PAWS: Preference Learning with Advantage-Weighted Segments","ref_index":16,"is_internal_anchor":true},{"citing_arxiv_id":"2606.11087","citing_title":"Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning","ref_index":44,"is_internal_anchor":true},{"citing_arxiv_id":"2606.10705","citing_title":"Event-Driven Reinforcement Learning Enables Long-Horizon Control in Semiconductor Fabrication","ref_index":46,"is_internal_anchor":true},{"citing_arxiv_id":"2605.13435","citing_title":"Q-Flow: Stable and Expressive Reinforcement Learning with Flow-Based Policy","ref_index":9,"is_internal_anchor":true},{"citing_arxiv_id":"2605.14779","citing_title":"Peng's Q($\\lambda$) for Conservative Value Estimation in Offline Reinforcement Learning","ref_index":10,"is_internal_anchor":true},{"citing_arxiv_id":"2605.00416","citing_title":"Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot Policies","ref_index":43,"is_internal_anchor":true},{"citing_arxiv_id":"2605.26552","citing_title":"Aligning Few-Step Generative Models by Amortizing Sample-based Variational Inference","ref_index":66,"is_internal_anchor":true},{"citing_arxiv_id":"2605.27877","citing_title":"SPAR: Support-Preserving Action Rectification","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2605.28409","citing_title":"Efficient Post-training of LLMs for Code Generation With Offline Reinforcement Learning","ref_index":9,"is_internal_anchor":true},{"citing_arxiv_id":"2605.31455","citing_title":"DRIFT: Decoupled Rollouts and Importance-Weighted Fine-Tuning for Efficient Multi-Turn Optimization","ref_index":15,"is_internal_anchor":true},{"citing_arxiv_id":"2605.30749","citing_title":"FLAG: Flow Policy MaxEnt-RL by Latent Augmented Guidance","ref_index":30,"is_internal_anchor":true},{"citing_arxiv_id":"2605.23551","citing_title":"Goal-Conditioned Agents that Learn Everything All at Once","ref_index":25,"is_internal_anchor":true},{"citing_arxiv_id":"2503.13934","citing_title":"COLSON: Controllable Learning-Based Social Navigation via Diffusion-Based Reinforcement Learning","ref_index":35,"is_internal_anchor":true},{"citing_arxiv_id":"2511.03828","citing_title":"From Static Constraints to Dynamic Adaptation: Sample-Level Constraint Relaxation for Offline-to-Online Reinforcement Learning","ref_index":9,"is_internal_anchor":true},{"citing_arxiv_id":"2602.22801","citing_title":"Unleashing the Potential of Diffusion Models for End-to-End Autonomous Driving","ref_index":38,"is_internal_anchor":true},{"citing_arxiv_id":"2605.05863","citing_title":"SOPE: Stabilizing Off-Policy Evaluation for Online RL with Prior Data","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2605.11151","citing_title":"RankQ: Offline-to-Online Reinforcement Learning via Self-Supervised Action Ranking","ref_index":12,"is_internal_anchor":true},{"citing_arxiv_id":"2605.18675","citing_title":"COOPO: Cyclic Offline-Online Policy Optimization Algorithm","ref_index":24,"is_internal_anchor":true},{"citing_arxiv_id":"2605.18580","citing_title":"When Outcome Looks Right But Discipline Fails: Trace-Based Evaluation Under Hidden Competitor State","ref_index":10,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":2,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/BHQFDYQ2RTFWVTYAKRI7G74UWI","json":"https://pith.science/pith/BHQFDYQ2RTFWVTYAKRI7G74UWI.json","graph_json":"https://pith.science/api/pith-number/BHQFDYQ2RTFWVTYAKRI7G74UWI/graph.json","events_json":"https://pith.science/api/pith-number/BHQFDYQ2RTFWVTYAKRI7G74UWI/events.json","paper":"https://pith.science/paper/BHQFDYQ2"},"agent_actions":{"view_html":"https://pith.science/pith/BHQFDYQ2RTFWVTYAKRI7G74UWI","download_json":"https://pith.science/pith/BHQFDYQ2RTFWVTYAKRI7G74UWI.json","view_paper":"https://pith.science/paper/BHQFDYQ2","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2006.09359&json=true","fetch_graph":"https://pith.science/api/pith-number/BHQFDYQ2RTFWVTYAKRI7G74UWI/graph.json","fetch_events":"https://pith.science/api/pith-number/BHQFDYQ2RTFWVTYAKRI7G74UWI/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/BHQFDYQ2RTFWVTYAKRI7G74UWI/action/timestamp_anchor","attest_storage":"https://pith.science/pith/BHQFDYQ2RTFWVTYAKRI7G74UWI/action/storage_attestation","attest_author":"https://pith.science/pith/BHQFDYQ2RTFWVTYAKRI7G74UWI/action/author_attestation","sign_citation":"https://pith.science/pith/BHQFDYQ2RTFWVTYAKRI7G74UWI/action/citation_signature","submit_replication":"https://pith.science/pith/BHQFDYQ2RTFWVTYAKRI7G74UWI/action/replication_record"}},"created_at":"2026-07-05T02:34:40.161500+00:00","updated_at":"2026-07-05T02:34:40.161500+00:00"}