{"paper":{"title":"Reinforcement Learning for Real-Time Vision-Language-Action Policies","arxiv_id":"2609.18207","primary_cat":"cs.RO","submitted_at":"2026-09-16T06:38:36+00:00","version_compiled":"v1"},"notice":"Every quoted sentence is the authors' text from the arXiv listing abstract. Labels and probabilities are the outputs of classifier-1.13.0 on the question bank abstract-p1-20260923, exactly as returned. No sentence in this record was written by a language model.","schema":"pith.compiled-record/v0","status":"compiled","abstract":{"source":"arXiv listing abstract v1","sentences":[{"i":0,"p":1.0,"id":"s0","own":false,"role":"background","text":"Reinforcement learning fine-tuning on top of large, pretrained Vision-Language-Action (VLA) models offers promise for highly reliable robot deployment.","hedge":{"p":0.9,"label":"hedged_possibility"},"own_p":0.08,"main_claim_p":0.07,"quantitative":false,"quantitative_p":0.03,"role_probabilities":{"gap":0.0,"method":0.0,"result":0.0,"background":1.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":1,"p":0.93,"id":"s1","own":false,"role":"background","text":"However, because of their scale, modern VLA models suffer from high inference latency, so the observation used to select an action is often stale by execution time, creating a distribution shift that can substantially degrade reliability and performance.","hedge":{"p":0.98,"label":"asserted"},"own_p":0.12,"main_claim_p":0.03,"quantitative":false,"quantitative_p":0.08,"role_probabilities":{"gap":0.06,"method":0.0,"result":0.0,"background":0.93,"limitation":0.01,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":2,"p":0.77,"id":"s2","own":false,"role":"gap","text":"Prior work has explored asynchronous policy execution to reduce the effect of latency, but these methods are mostly built on imitation learning and offer no mechanism for moving beyond the training distribution toward higher reliability.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.03,"main_claim_p":0.03,"quantitative":false,"quantitative_p":0.14,"role_probabilities":{"gap":0.77,"method":0.0,"result":0.0,"background":0.23,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":3,"p":0.93,"id":"s3","own":true,"role":"contribution","text":"We close this gap by enabling RL fine-tuning that meets the real-time control requirements of dynamic real-world manipulation.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.77,"main_claim_p":0.3,"quantitative":false,"quantitative_p":0.03,"role_probabilities":{"gap":0.07,"method":0.0,"result":0.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.93,"interpretation":0.0}},{"i":4,"p":0.95,"id":"s4","own":true,"role":"method","text":"Our approach builds on EXPO-FT, a framework for sample-efficient, reliable VLA fine-tuning with reinforcement learning, and decouples slow, expressive action generation from fast, reactive action edits: a large pretrained VLA proposes action chunks using its strong behavior prior, while a lightweight edit policy performs fast, reactive decision-making by editing actions in response to changes in state, conditioned on the latest observation.","hedge":{"p":0.99,"label":"asserted"},"own_p":0.85,"main_claim_p":0.14,"quantitative":false,"quantitative_p":0.07,"role_probabilities":{"gap":0.0,"method":0.95,"result":0.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.05,"interpretation":0.0}},{"i":5,"p":0.71,"id":"s5","own":true,"role":"contribution","text":"We instantiate this as Real-Time EXPO-FT, an RL framework for finetuning real-time VLA policies.","hedge":{"p":0.98,"label":"asserted"},"own_p":0.81,"main_claim_p":0.07,"quantitative":false,"quantitative_p":0.04,"role_probabilities":{"gap":0.0,"method":0.29,"result":0.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.71,"interpretation":0.0}},{"i":6,"p":1.0,"id":"s6","own":true,"role":"result","text":"On the Kinetix benchmark, Real-Time EXPO-FT enables a delayed policy to achieve the best performance among delayed and non-delayed methods in 10 out of 10 environments.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.95,"main_claim_p":0.23,"quantitative":true,"quantitative_p":0.99,"role_probabilities":{"gap":0.0,"method":0.0,"result":1.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":7,"p":1.0,"id":"s7","own":true,"role":"result","text":"On four dynamic real-world tasks, robot object passing, ball balancing, table soccer kicking, and dynamic object picking, with online robot data capped at 10 minutes, Real-Time EXPO-FT improves average policy performance from 42% to 97%, all without human intervention, demonstrating rapid, sample-efficient adaptation to challenging real-world dynamics.","hedge":{"p":1.0,"label":"asserted"},"own_p":0.96,"main_claim_p":0.42,"quantitative":true,"quantitative_p":0.99,"role_probabilities":{"gap":0.0,"method":0.0,"result":1.0,"background":0.0,"limitation":0.0,"future_work":0.0,"contribution":0.0,"interpretation":0.0}},{"i":8,"p":0.33,"id":"s8","own":false,"role":"future_work","text":"Website: https://pd-perry.github.io/real-time-expo-ft","hedge":{"p":0.97,"label":"unclear"},"own_p":0.13,"main_claim_p":0.02,"quantitative":false,"quantitative_p":0.03,"role_probabilities":{"gap":0.0,"method":0.04,"result":0.07,"background":0.26,"limitation":0.0,"future_work":0.33,"contribution":0.29,"interpretation":0.01}}]},"compiled":{"main_claim":{"p":0.68,"text":"On four dynamic real-world tasks, robot object passing, ball balancing, table soccer kicking, and dynamic object picking, with online robot data capped at 10 minutes, Real-Time EXPO-FT improves average policy performance from 42% to 97%, all without human intervention, demonstrating rapid, sample-efficient adaptation to challenging real-world dynamics.","location":"abstract s7","hedge_strength":{"p":1.0,"label":"asserted"}},"provenance":{"route":{"via":"TypeSafe API","model":"classifier-1.13.0","endpoint":"https://api.typesafe.ai/v1/systemone","provider":"TypeSafe"},"packing":"one request per paper","requests":1,"input_tokens":7169,"cost_usd_reported":0.0},"compiled_at":"2026-09-23","bank_version":"abstract-p1-20260923","model_version":"classifier-1.13.0","abstract_states":{"claims_first":0.18,"states_limitation":0.14,"human_participants":0.08,"controlled_comparison":0.84,"randomized_assignment":0.01,"states_quantitative_result":0.99,"compares_with_prior_results":0.95,"mentions_code_or_data_availability":0.39},"evidential_basis":{"p":0.99,"value":"observation or experiment","probabilities":{"argument":0.0,"literature":0.0,"proof_or_derivation":0.0,"analysis_of_existing_data":0.0,"observation_or_experiment":0.99,"simulation_or_computation":0.01}},"contribution_type":{"p":1.0,"value":"method or tool","probabilities":{"review":0.0,"method_or_tool":1.0,"empirical_study":0.0,"simulation_study":0.0,"theoretical_result":0.0,"dataset_or_benchmark":0.0,"position_or_commentary":0.0}},"contribution_statement":{"p":0.83,"text":"Our approach builds on EXPO-FT, a framework for sample-efficient, reliable VLA fine-tuning with reinforcement learning, and decouples slow, expressive action generation from fast, reactive action edits: a large pretrained VLA proposes action chunks using its strong behavior prior, while a lightweight edit policy performs fast, reactive decision-making by editing actions in response to changes in state, conditioned on the latest observation.","location":"abstract s4"}},"coverage":{"tier":"abstract","levels_run":["P abstract"],"not_examined":["full text","figures and tables","references and citing sentences","relations to other papers"],"sources_read":["arXiv listing abstract v1"],"levels_not_run":["S sections","T body sentences","C citations","R cross-paper relations"],"sentences_examined":9,"sentences_in_abstract":9},"measured_accuracy":{"url":"https://pith.science/accuracy","measured_on":"2026-10-02","method":"two independent model families labelled a fresh random sample blind; the figure is how often the published label matches their consensus","sentence_role_agreement":0.92,"main_claim_agreement":0.908,"typed_citation_agreement":0.933,"typed_citation_agreement_95_interval":[0.917,0.945],"review_comments_accurate":0.758,"review_comments_accurate_95_interval":[0.684,0.82],"review_caught_admitted_error_found_or_related":0.448,"review_comments_judged":{"judged_by":["gpt-5.6-luna","grok-4.7"],"judged_against":"Pith's own text copy of each paper: text extracted from the PDF, the first 90,000 characters","measured_on":"2026-10-04","accurate":113,"decided":149,"sampled_reviews":200,"split_by_lane":false,"by_lane":"https://pith.science/accuracy#by-lane"}}}