{"id":"e6c1b82b-1714-48e8-8adc-fd91635f12e8","arxiv_id":"2611.15491","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"The key to VLA reactivity in dynamic manipulation lies as much in how the policy is executed as in the policy itself, with the best prediction horizon about one second and the best execution horizon depending on the checkpoint.","lead":"This paper studies what makes a Vision-Language-Action model reactive when objects move during execution. It finds that the execution pipeline, including prediction horizons, inference scheduling, and chunk continuity, matters as much as the policy itself for dynamic manipulation tasks like conveyor belt picking.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"No significant objection identified.","rationale":"The reader identified the weakest assumption as generalizability and sample size limitations. The paper itself acknowledges these limitations (b104, b105). My review finds no additional load-bearing concerns beyond what the authors have already qualified. The internal evidence is consistent, the methods are clearly described, and the conclusions are appropriately cautious. Therefore, no change to the verdict is warranted.","tokens_in":13409,"tokens_out":1965,"duration_ms":16511,"concrete_test":"Replicate the main comparisons (Fig. 2b, Fig. 3c, Fig. 4a) using a different base VLA model (e.g., Octo or RT-2) on the same conveyor-belt task. If the relative ranking of methods (e.g., ABPolicy > RTC ≈ FASTER, short horizons help more asynchronously) changes qualitatively, then the design guidelines are model-specific; if the ranking holds, generalizability is strengthened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that reactivity depends on the execution pipeline as much as on the policy—is supported by a thorough empirical study within the stated scope (pi0.5-based adaptations on a limited set of tasks). The authors explicitly acknowledge the main limitation: findings characterize pi0.5 adaptations and may not generalize to other VLA architectures (b104). Real-robot sample sizes are small (15–20 trials) with manually induced motion, and the authors caution against over-interpretation (b105). No internal inconsistency, methodological flaw, or unsupported claim was found. The weakest assumption identified by the reader—generalizability—is already stated by the authors. The paper's design guidelines are appropriately qualified, and the evidence presented internally supports the conclusions drawn.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"This paper empirically investigates how the execution pipeline affects the reactivity of pretrained Vision-Language-Action (VLA) models, using π0.5 as the base policy. The authors systematically vary prediction and execution horizons, compare synchronous and asynchronous inference, test methods that improve trajectory continuity (RTC, ABPolicy) or reduce latency (FASTER, DynamicVLA), and evaluate motion cues (PUMA) and latent-space reinforcement learning (DSRL). Experiments on two simulation benchmarks (DOM, DOMINO) and three real-robot tasks (moving-cup placement, rolling-can grasping, conveyor-belt picking) show that (i) the best prediction horizon is roughly one second, (ii) the best execution horizon depends on the checkpoint, (iii) asynchronous execution and trajectory continuity matter more than raw latency, and (iv) policy adaptation can help but a simple change of execution horizon often achieves comparable gains. The paper concludes that reactivity depends on the execution pipeline as much as on the policy, so both should be evaluated together.","tokens_in":13696,"tokens_out":1490,"duration_ms":12444,"significance":"If the findings hold, the paper provides concrete design guidelines for deploying VLA models in dynamic manipulation: prediction horizons should be chosen in seconds, execution horizons must be tuned per checkpoint, and preserving trajectory continuity is more important than minimizing latency. The study is thorough within its scope (π0.5-based methods on a limited set of tasks) and the authors honestly acknowledge limitations in generalizability and sample size. The work is likely to influence how the robotics community evaluates and deploys chunk-based policies, making execution strategy a first-class design dimension.","major_comments":[],"minor_comments":[{"comment":"Several plots report overall success percentages without error bars or confidence intervals. Given the small trial counts (15–20 per condition on real robots), adding bootstrapped 95% confidence intervals would help readers assess the reliability of reported differences, even if the authors caution against over-interpretation.","section":"Figures 2 and 3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Figure 3c is a numeric table embedded as a figure; converting it to a proper table in the main text would improve readability and reduce potential formatting issues in publication.","section":"Table 3 / Figure 3c","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The notation for Adaptive Action Chunking (AAC) uses symbols 'α' and 'ξ' that are introduced only in the appendix (B.2). A brief inline definition or a forward reference to the appendix would help the main-text reader.","section":"Section 2, paragraph after Table 1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The phrase 'the best execution horizon depends on the checkpoint, not on replanning more often' is supported by the 1k vs 5k checkpoint comparison, but the explanation about 'the 5k policy barely moves during the first steps' is qualitative. A quantitative measure (e.g., average displacement magnitude in the first 5 steps) would strengthen the claim.","section":"Section 3, Table 2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The DSRL results on the conveyor belt (Fig. 4b) mix success counts and percentages. The explanation is clear, but standardizing to percentages throughout would avoid potential reader confusion.","section":"Section 3, Table 4","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The description of Cartesian ABPolicy adaptation mentions reference quaternions and rotation vector mapping. A brief justification of why this representation is preferred over direct quaternion fitting would be helpful.","section":"Appendix B.6","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"minor_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"This paper is exactly what it advertises: a systematic empirical comparison of execution strategies (horizons, scheduling, continuity, policy adaptation) for a single VLA model (pi0.5) on dynamic manipulation. The central finding—that reactivity depends as much on how you execute the policy as on the policy itself—is well supported by the data. The paper earns its keep with genuinely novel observations that interactions matter: the optimal execution horizon is checkpoint-dependent (Fig. 2b), short horizons help more with async execution than sync (Fig. 3a), and trajectory continuity methods (ABPolicy) outperform latency-only methods (FASTER) on dynamic tasks. These are concrete, actionable design rules that the VLA deployment community will use.\n\nThe experimental design is sound, the research questions are clear, and the authors consistently state their limitations (b104, b105). There are no hidden flaws—the acknowledged scope (pi0.5, 15–20 real-robot trials, manually induced motion) is the real scope. The paper has no circularity and its claims are observational, not derived from assumed premises.\n\nThe main soft spot is the obvious one: generalizability. We don't know if these findings transfer to other VLA architectures (e.g., Octo, RT-2, or diffusion policies with different chunking). The authors say so themselves, but the reader should not assume the design rules hold universally until replicated. The trial counts are small and the manual motion in the can/cup tasks adds uncontrolled variance—the authors caution against over-interpretation, and I agree. Also, the adaptive horizon method (AAC) fails convincingly, but that failure may be specific to their entropy-based trigger; other adaptive approaches (e.g., learned horizon prediction) might give different results.\n\nWho is this for? Researchers and engineers deploying VLA models in dynamic scenes. It is a useful empirical contribution that raises a legitimate flag: don't optimize the policy alone, optimize the whole execution pipeline. It deserves a serious referee slot. I would accept it, perhaps with a suggestion to add a short paragraph contrasting their negative AAC result with other adaptive methods that use learned value functions (e.g., ACH, DEHP) to help the reader judge generality.\n\nI would not cite this in my own work unless I were working directly on VLA execution, but I would recommend it to students entering the area.","headline":"Solid empirical paper showing execution pipeline matters as much as the policy for VLA reactivity, though results are specific to pi0.5 and small task sets.","tokens_in":14147,"tokens_out":1387,"duration_ms":11680},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"VLA reactivity depends on execution strategy more than on policy alone.","keywords":["vision-language-action models","dynamic manipulation","execution horizon","asynchronous inference","trajectory continuity","π0.5","reactive control","action chunking"],"falsifier":"If another VLA model (e.g., Octo or RT-2) were evaluated under the same protocol and the optimal execution horizon did not vary with checkpoint, or if continuity methods showed no advantage over pure latency reduction, the central claim that execution pipeline dominates reactivity would be weakened.","tokens_in":14968,"feed_emoji":"🤖","tokens_out":3977,"duration_ms":30970,"temperature":0.7,"pith_summary":"This paper argues that for vision-language-action (VLA) models used in dynamic manipulation, reactivity is governed as much by the execution pipeline as by the policy itself. Using the π0.5 model on simulation and real robots, the authors vary prediction and execution horizons, compare synchronous and asynchronous inference, and test trajectory continuity methods. They find that the best execution horizon is checkpoint-dependent, that short horizons only help when combined with asynchronous execution and continuity, and that policy adaptation yields targeted gains but often no more than changing the execution horizon. The conclusion is that execution strategy and policy must be evaluated together to fairly compare VLA reactivity.","feed_headline":"VLA reactivity hinges on execution, not just policy","feed_subtitle":"Study shows horizon, continuity, and async scheduling matter as much as the policy.","key_machinery":"The key machinery is the execution pipeline comprising the execution horizon (how many predicted actions are executed before replanning), inference scheduling (synchronous versus asynchronous), and trajectory continuity mechanisms (e.g., spline refitting with ABPolicy, action inpainting with RTC). These components are varied systematically with a fixed policy (π0.5) to isolate their effect on task success under dynamic object motion.","core_discovery":"The central discovery is that the reactivity of a pretrained VLA model is not solely a property of the policy but emerges from the interplay of prediction horizon (best at about one second), execution horizon (best value depends on policy checkpoint), inference scheduling (asynchronous execution critically amplifies short-horizon benefits), and trajectory continuity (methods like ABPolicy that preserve continuity outperform low-latency methods like FASTER). Action rate also matters: raising it from 20 to 30 Hz turned a near-zero success method into a 93% success method on the fast conveyor belt. Policy adaptation via motion cues (PUMA) or latent steering (DSRL) can fix targeted failures, but changing the execution horizon on the conveyor task gave equivalent improvement without any training. Therefore, comparing VLA methods requires controlling for the execution pipeline.","pith_inferences":["The finding that execution horizon changes often match or exceed gains from policy adaptation suggests that optimizing the execution pipeline is a cheaper and faster path to improved reactivity than retraining the policy.","These results imply that many published comparisons of VLA methods may be confounded by uncontrolled execution details, so the field should adopt standardized reporting of pipeline parameters.","The checkpoint-dependent optimal horizon hints that a lightweight online search or learned horizon selector could be a practical add-on to any VLA deployment."],"forward_implications":["Researchers evaluating VLA models for dynamic manipulation should report prediction and execution horizons, inference scheduling, action rate, and chunk continuity alongside success rates.","Execution horizon should not be fixed across policy checkpoints; the optimal horizon depends on the policy’s learned behavior, particularly its initial action dynamics.","Methods that maintain trajectory continuity are likely to outperform methods that only reduce latency in settings where object motion is fast and unpredictable.","Raising the action rate can dramatically improve performance, suggesting that hardware and control frequency are first-order design variables for reactive manipulation.","Adaptive horizon strategies should consider not only action uncertainty but also inference overhead and continuity costs."],"supporting_citations":[{"why":"Provides the π0.5 model, the base policy whose behavior is analyzed throughout the study.","marker":"[5]"},{"why":"Introduces RTC, a continuity-preserving method that bridges chunks via inference-time inpainting, used as a baseline.","marker":"[1]"},{"why":"Introduces FASTER, a latency-reduction method that serves as a baseline for the discontinuity versus continuity comparison.","marker":"[2]"},{"why":"Introduces DynamicVLA, a method with asynchronous execution and latency-aware action skipping, adapted here.","marker":"[3]"},{"why":"Introduces ABPolicy, the best-performing continuity method with B-spline control points and constrained refitting.","marker":"[8]"}],"fun_headline_variants":["VLA reactivity is shaped by execution, not just policy","Best VLA prediction horizon? About one second","Async scheduling and continuity drive VLA reactivity","Exec horizon can match policy adaptation for VLA"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The results are obtained with a single VLA model (π0.5) on a small set of tasks with limited trials (15–20 per setting for real-robot tasks), so generalizability to other architectures or larger-scale deployments is untested, as the authors acknowledge.","fun_headline_variants_meta":{"raw":{"variants":["VLA reactivity is shaped by execution, not just policy","Best VLA prediction horizon? About one second","Async scheduling and continuity drive VLA reactivity","Exec horizon can match policy adaptation for VLA"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000202,"raw_usage":{"cost":0.000202328,"is_byok":false,"cost_details":{"upstream_inference_cost":0.000202328,"upstream_inference_prompt_cost":9.0888e-05,"upstream_inference_completions_cost":0.00011144},"total_tokens":1252,"prompt_tokens":854,"completion_tokens":398,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":256,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":337}},"tokens_in":854,"tokens_out":398,"cost_source":"openrouter_usage","duration_ms":4468,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":337,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-11-13T19:21:27.931291+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"If another VLA model (e.g., Octo or RT-2) were evaluated under the same protocol and the optimal execution horizon did not vary with checkpoint, or if continuity methods showed no advantage over pure latency reduction, the central claim that execution pipeline dominates reactivity would be weakened.","supporting_citations":[{"cited_title":"$\\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization","cited_arxiv_id":"2504.16054","evidence_quote":"Provides the π0.5 model, the base policy whose behavior is analyzed throughout the study."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces RTC, a continuity-preserving method that bridges chunks via inference-time inpainting, used as a baseline."},{"cited_title":"FASTER: Rethinking Real-Time Flow VLAs","cited_arxiv_id":"2603.19199","evidence_quote":"Introduces FASTER, a latency-reduction method that serves as a baseline for the discontinuity versus continuity comparison."},{"cited_title":"DynamicVLA: A Vision-Language-Action Model for Dynamic Object Manipulation","cited_arxiv_id":"2601.22153","evidence_quote":"Introduces DynamicVLA, a method with asynchronous execution and latency-aware action skipping, adapted here."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces ABPolicy, the best-performing continuity method with B-spline control points and constrained refitting."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}