Pith.

Robotics · arXiv:2611.03103 v1 · submitted 2 Nov 2026 · 39 references

VLALagBench: Understanding When and Why VLA Inference Latency Matters

Qi Wen, Borui Xu, Hussein Amro, Hongshi Tan, Hanmin Lin, Chaoyi Ruan, Shanghao Liu, Gang Yang

show 6 more authors

Yuejiang Liu, Jiafei Duan, Bingsheng He, Jason Clemons, Christos Kozyrakis, Wenqi Jiang

The paper says

“Vision-language-action (VLA) models offer a promising path toward general-purpose robot manipulation. While inference latency can affect task success in a constantly evolving physical world, the relationship between latency and task success rate remains poorly understood.”

more of the abstract

In this paper, we introduce VLALagBench, a comprehensive benchmark for studying VLA inference latency sensitivity across task types, object speeds, execution modes (synchronous and asynchronous), and VLA policies. Using VLALagBench, we find that even a modest inference latency of 100 ms can nearly halve task success on moving targets, with sensitivity varying substantially across tasks and policies. These differences reflect task-specific requirements for precision and continuous motion. By overlapping inference with action execution, asynchronous execution can recover some of the lost success at the cost of more frequent inference requests. Building on these findings, we develop an empirical model that predicts task success from limited calibration data, achieving root-mean-square prediction errors of only 5.7-7.8% across various VLA policies. Given an inference system's latency and per-robot throughput, these predictions guide the selection of system-specific VLA algorithm configurations to maximize task success. We believe VLALagBench and these findings provide a solid foundation for future robotics algorithm-system co-design.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 3 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. Well-measured latency benchmark with a genuinely predictive empirical model; the core findings are solid, and the main weakness is that the deployment-selection claim rests on a single validation cell. This line is wrong

2 major objections · 4 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 39 references say.

The paper says, at §5.2 / Table 3

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

The deployment-selection contribution is validated on exactly one policy-task-speed cell. In §5.2, the model is calibrated for π0.5 on set-down at 0.075 m/s from one complete continuous-100 ms curve plus two 200 ms points, and Systems A–D differ only in injected latency and per-robot rate limit over the same MuJoCo success surface.

more of objection 1

System A's selected configuration is itself the calibration curve, so its row in Table 3 is not an independent validation point. This demonstrates that Eq. (1) can rank four latency/throughput operating points for one cell, but not the abstract's general claim that the predictions 'guide the selection of system-specific VLA algorithm configurations.' Because a and b are refitted for every policy-task pair and the reference curve is measured per task, cross-task and cross-policy generalization does not follow automatically. Please add at least one additional policy-task cell to the §5.2 validation, ideally under a rate-limited configuration, or explicitly restrict the deployment-selection claim to the demonstrated cell.

The paper says, at §5.1 / Eq. (1), Table 2, Appendix D

The review did not quote this passage.

Pith says: Objection 2 of 2 · major

The headline accuracy numbers (5.7–7.8 points RMS) are reported for π0.5 and MolmoAct2, while GR00T N1.7's limited-data calibration has a median RMS error of 11.6 points and fitted coefficients whose signs contradict the proposed staleness and idle-time mechanisms on set-down and grasping (Table 2).

more of objection 2

Appendix D also documents two systematic departures that Eq. (1) cannot represent: at 250 ms, lazy and continuous execution lose near-static success on release and set-down, and on the fastest targets synchronous execution beats asynchronous execution despite the model's prediction that synchronous execution should be worse. These limitations are honestly acknowledged, but because Eq. (1) is the foundation of the §5.2 selection claim, the model's validity domain should be stated as a scope condition in the abstract and Section 5 rather than only in appendix caveats. A concrete test would be a deployment-selection validation on a policy-task pair where Eq. (1) actually fits, together with a statement of which of the paper's conclusions depend on the model rather than on the direct measurements.

4 minor objections

The paper says, at Throughout

Pith says: minor objection 1 of 4

The benchmark name is spelled both 'VLALagBench' and 'VLALAGBENCH'; please standardize the spelling in the abstract, main text, and figures.

The paper says, at §3.5 / Figure 5

Pith says: minor objection 2 of 4

The statement that synchronous and lazy asynchronous execution have identical average observation staleness is central to the pause-vs-staleness attribution; a one-line algebraic derivation in the text would make this easier to verify than reading it off Figure 5.

show 2 more

The paper says, at Table 3 / Appendix D

Pith says: minor objection 3 of 4

The latencies for Systems B–D combine measured GPU latencies with network latency estimates taken from VLA-Perf rather than measured in the deployment scenario; a short sensitivity analysis over the network estimate would strengthen the recommendation.

The paper says, at Appendix F

Pith says: minor objection 4 of 4

The real-robot results use five rollouts per condition, which is appropriate for a pilot but should be explicitly labeled as a pilot verification in the main text rather than appearing only in the appendix.

What would settle it

Run the exact Table 3 calibration protocol — one complete continuous-100 ms curve plus one synchronous and one continuous point at 200 ms — for π0.5 on release and grasping and for MolmoAct2 on set-down, at both 0.075 and 0.1 m/s, and compare the predicted best (mode, H) against the measured best under the same four latency/throughput constraints as Systems A–D. Report the match rate over these cells; if the model selects the measured best configuration in fewer than 75% of cells, or misses by more than 5 success-rate points in more than two cells, the system-selection claim should be downgraded to a single-case demonstration.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 3 Nov 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic partly agrees 4 min 25 s · 32,495 tokens read, 11,054 written
  3. Editor wrote the verdict above 48 s · 33,444 tokens read, 8,269 written
  4. Referee major revision: 2 major objections, 4 minor 50 s · 32,877 tokens read, 6,392 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Two things to know up front. The measurements are well built: 200 rollouts per condition, Wilson intervals, latency injected as control steps with the action budget extended so the policy isn't starved, and a stated sampling-noise floor that the model is judged against. The headline — 100 ms of latency can nearly halve success on moving targets, with sensitivity varying by task and policy — is direct measurement, not a fitted artifact.

What's new: ReflexBench already varied latency but on one policy and two tasks; MoveBench and DOM don't sweep latency at all. This paper is the first joint controlled sweep of task type, target speed, execution mode, and three generalist policies, and it turns the vague intuition about latency into two measurable mechanisms: stale observations hurt precision-sensitive tasks, robot pauses hurt continuous-motion tasks. The tolerance-zone analysis (Fig. 7) and stage pass rates (Table 8) genuinely support that decomposition. The Eq. (1) model is also useful: held-out RMS of 5.7 points for π0.5 against a 4.0-point noise floor is real predictive power, and the authors report its failures (GR00T's negative fitted coefficients, near-static async at 250 ms) rather than burying them.

Soft spots, in proportion. The stress-test note lands: the §5.2 deployment-selection claim is validated on exactly one cell — π0.5, set-down, 0.075 m/s. The four 'systems' differ only in injected latency and throughput cap over the same MuJoCo surface, so cross-task and cross-policy generality of the selection procedure is unestablished. This is the biggest gap between the abstract's 'guide the selection' claim and the evidence. Not fatal: the model itself is validated across four tasks and held-out latencies for π0.5 and MolmoAct2, so the mechanism isn't fragile, just the system-selection demonstration is thin. Note System A in Table 3 is its own calibration curve; the held-out evidence in that table is Systems B–D, plus the 133-point validation in Appendix D.

Second, the benchmark artifacts aren't released; for a benchmark paper that's a concrete omission. Third, the abstract's 5.7–7.8% range covers two of the three policies; GR00T's error under the same protocol isn't reported in the main table (its limited-calibration median in Appendix D is 11.6), so 'across various VLA policies' mildly overstates coverage. Fourth, the real-robot confirmation is five rollouts per condition on one policy — fine as a sanity check, thin as evidence. GR00T's sensitivity to initial target positions (Table 1) is a real confound for its fitted coefficients; the paper acknowledges it, to its credit, but it means the speed-scaling story should be read as π0.5/MolmoAct2-first.

This paper deserves a serious referee. The revision that would make it solid: release the code, data, and waypoints; temper the abstract; report GR00T's held-out error in the main table; and either extend §5.2 to a second task or policy or state plainly that selection is demonstrated for one cell.

The strongest objection

The paper's third contribution — using the §5.1 model to select execution mode and horizon for a given inference system — is supported by a single validation cell. Table 3 calibrates the model on π0.5, set-down, target speed 0.075 m/s, using one complete continuous-100 ms curve plus two calibration points, and then evaluates Systems A–D on that same task, policy, and speed. The differences between the four systems are only the injected latency and the per-robot throughput cap; the underlying success surface is the same MuJoCo set-down surface. The experiment therefore demonstrates that Eq. (1) can rank four latency/rate operating points for one policy-task-speed combination, not that it selects configurations across tasks, policies, or speeds. Because a and b in Eq. (1) are refit for every policy-task pair and the reference curve is measured per task, the abstract's general 'guide the selection' claim needs cross-task and cross-policy evidence. The acknowledged failures for GR00T N1.7 (Table 2) and near-static async at 250 ms (Appendix D) are honestly reported and are not the basis of this concern; they make the lack of replication in the flagship deployment validation more consequential.

The direct empirical claim — that 100 ms of inference latency can nearly halve success on moving targets — is well supported by 200-rollout conditions with Wilson intervals, cross-simulator checks in Isaac Sim, and real-robot verification, so I do not object to Finding 1. The empirical status of Eq. (1) is not by itself the decisive problem: the paper reports held-out validation, compares RMS error to a 4.0-point sampling-noise floor, and explicitly lists the GR00T N1.7 and near-static async departures. The unresolved gap is that the model's advertised use for system-specific configuration selection is demonstrated in only one policy-task-speed cell, with the four 'systems' differing only by latency and rate limit over the same simulator surface. My proposed check would settle whether the selection protocol generalizes beyond that cell. This reinforces, rather than moves, the reader's CONDITIONAL verdict: the benchmark measurements are credible, but the model-guided selection contribution needs broader validation and the release of artifacts before the abstract-level claim is fully supported.

The referee's report

If the findings hold, the benchmark fills a real gap: existing dynamic-manipulation benchmarks do not systematically isolate inference latency while varying task type, speed, execution mode, and policy. The strengths are substantial: controlled latency injection, 200 rollouts per condition, 95% Wilson intervals, held-out latency predictions whose RMS error is reported against an estimated sampling-noise floor, and honest, clearly located discussion of the model's failures for GR00T N1.7 and near-static asynchronous behavior at 250 ms. The paper also verifies the main findings in a second simulator and on a real robot. The main weakness is that the third contribution, system-specific configuration selection, is validated on exactly one policy-task-speed cell, while the abstract states the guidance claim in general terms. The empirical model itself is an ansatz rather than a derivation, but it is evaluated on held-out latencies; the concern is scope, not circularity of the core measurements.

Pith's reading of the paper (a machine's summary, not the authors')

This paper tries to establish a quantitative answer to a question robotics system builders currently answer by guesswork: how many points of task success does VLA inference latency cost, and why. Working from four moving-target manipulation tasks in the LIBERO/MuJoCo simulator, three open VLA policies, and a controlled sweep of latencies, execution modes, and target speeds, it finds that even 100 ms of inference latency can nearly halve success on a precision task -- set-down drops from 71.5% to 40.5% at 0.1 m/s -- while a more tolerant grasping task keeps 86% of its zero-latency success. The paper attributes the loss to two separable mechanisms, stale observations and robot pauses, and shows that asynchronous execution recovers part of the damage at the price of more frequent inference requests. On top of these findings it claims that the whole latency effect can be summarized by reading a measured reference success curve at a rescaled target speed, with a two-coefficient adjustment linear in staleness and idle time; this model predicts held-out latencies within 5.7-7.8 points RMS and selects the measured-best execution mode and horizon on all four deployment systems it considers. If the claim holds, a few calibration measurements on one system could replace an expensive configuration sweep on every new GPU, network, or cloud endpoint.

What carries the argument. The load-bearing object is Eq. (1), a speed-scaling identity: the success of configuration $X$ is the reference curve $S_0$ read at a rescaled target speed $\lambda_X v$, where $\lambda_X$ is linear in the relative change of mean observation staleness and in idle steps per executed action. Its function is to compress an expensive configuration search into a curve lookup: once $S_0$ and the two coefficients are measured, every combination of latency, execution mode, and execution horizon maps to one point on one curve, and the coefficients themselves become an interpretable diagnostic -- a large $a$ means the task is hurt by stale observations, a large $b$ means it is hurt by pauses. The identity also supports calibration without zero-latency data, because $S_X(v) \approx S_R((\lambda_X/\lambda_R)v)$ lets any measured curve serve as the reference, provided one calibrated configuration is synchronous at nonzero latency so that idle time can be separated from staleness.

Core claim. On its own terms, the paper's central claim is that inference latency damages VLA task success through exactly two channels -- the observation the policy acts on is stale by the time the action executes, and the robot sits idle while inference runs -- and that their combined effect is equivalent, for the purpose of predicting success, to making the target move faster. That equivalence is formalized as $S_X(v) \approx S_0(\lambda_X v)$, with $\lambda_X = 1 + a\,(\bar{s}_X - \bar{s}_0)/\bar{s}_0 + b\,I_X/H_X$, where $S_0$ is a measured reference success curve and the two fitted coefficients $a$ and $b$ quantify a policy-task pair's sensitivity to staleness and idle time respectively. Fitted on zero-, 100-, and 200-ms runs, the model predicts held-out 50-, 150-, and 250-ms curves with 5.7 points RMS error on $\pi 0.5$ and 7.8 on MolmoAct2, close to the 4-point sampling noise of the measurements themselves; with only one complete curve plus two points it holds a median 7.7-point error. Applied to four deployment systems differing in GPU latency and per-robot throughput, it predicts the measured best execution mode and horizon in every case. The paper is explicit about where the equivalence stops: GR00T N1.7 fits negative coefficients on some tasks, partly because changing initial target positions already costs that policy 30-40 points at zero latency, and near-static asynchronous runs at 250 ms lose success the model cannot see.

The premise it rests on. Everything rests on the equivalence between latency damage and a faster target: success under any configuration is read off one measured curve at a speed rescaled linearly by staleness and idle time, an equivalence the paper itself shows already breaks for GR00T N1.7 and for near-static asynchronous execution at 250 ms.

If the paper is right (Pith's inferences, not the authors')

For a given deployment, the best execution mode and horizon can be chosen from three or four measurements: the model matched the measured best configuration on all four representative systems, from onboard GPU to slow cloud.

Continuous asynchronous execution at short latency can beat zero-latency synchronous execution on precision tasks because it freshens observations: at 50 ms it raised set-down success at 0.125 m/s from 51.5% to 70.0%.

Latency sensitivity is not a single number per policy: it factors into task precision tolerance and motion-continuity demands, so the question of how fast inference must be has to be answered per task, per target speed, and per policy.

The two fitted coefficients per policy-task pair give a cheap screening test for which failure channel dominates, predicting whether asynchronous execution will help before a full sweep is run.

Held-out prediction error on unseen latencies (5.7-7.8 points RMS) is within about two points of the measurements' own 4-point sampling noise, so further accuracy gains would need more rollouts or better measurement rather than a richer model.

If the speed-scaling identity holds beyond the tested grid, it can be inverted into a design specification: for a target success rate and task, $S_0$ and $\lambda$ give the maximum tolerated latency and idle-time budget, letting a systems team state a hardware requirement ('no more than X ms end-to-end') in task-success terms instead of frames-per-second terms.

The two-channel decomposition suggests a direct evaluation recipe for latency-robust training: policies trained with deliberately stale observations or inserted idle gaps should show smaller fitted $a$ and $b$, so the benchmark's coefficients could serve as the metric for such methods.

The model's acknowledged failures -- GR00T's negative coefficients and the 250 ms near-static asynchronous drop -- point to a missing third mechanism, plausibly chunk-boundary discontinuities or sensitivity to initial target position; the residuals at extreme latencies are where such a term would be identified and tested.

Because the best policy depends on task and speed as well as latency, single-number leaderboard rankings of VLA models are under-specified for real-time robotics; a latency-by-speed success matrix is closer to what a deployment decision actually needs.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Vision-language-action (VLA) models offer a promising path toward general-purpose robot manipulation. While inference latency can affect task success in a constantly evolving physical world, the relationship between latency and task success rate remains poorly understood. In this paper, we introduce VLALagBench, a comprehensive benchmark for studying VLA inference latency sensitivity across task types, object speeds, execution modes (synchronous and asynchronous), and VLA policies. Using VLALagBench, we find that even a modest inference latency of 100 ms can nearly halve task success on moving targets, with sensitivity varying substantially across tasks and policies. These differences reflect task-specific requirements for precision and continuous motion. By overlapping inference with action execution, asynchronous execution can recover some of the lost success at the cost of more frequent inference requests. Building on these findings, we develop an empirical model that predicts task success from limited calibration data, achieving root-mean-square prediction errors of only 5.7-7.8% across various VLA policies. Given an inference system's latency and per-robot throughput, these predictions guide the selection of system-specific VLA algorithm configurations to maximize task success. We believe VLALagBench and these findings provide a solid foundation for future robotics algorithm-system co-design.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-04

The authors' main claim, as the classifier picked it out 0.90

“Using VLALagBench, we find that even a modest inference latency of 100 ms can nearly halve task success on moving targets, with sensitivity varying substantially across tasks and policies.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.09 · states limitation 0.28 · human participants 0.05 · controlled comparison 0.33 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.23 · mentions code or data availability 0.06

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Vision-language-action (VLA) models offer a promising path toward general-purpose robot manipulation. background 1.00 0.04 0.02 hedged possibility 0.58
While inference latency can affect task success in a constantly evolving physical world, the relationship between latency and task success rate remains poorly understood. gap 0.99 0.06 0.03 asserted 0.58
In this paper, we introduce VLALagBench, a comprehensive benchmark for studying VLA inference latency sensitivity across task types, object speeds, execution modes (synchronous and asynchronous), and VLA policies. contribution 0.94 0.89 0.05 asserted 0.99
Using VLALagBench, we find that even a modest inference latency of 100 ms can nearly halve task success on moving targets, with sensitivity varying substantially across tasks and policies. result 1.00 0.97 0.63 asserted 1.00
These differences reflect task-specific requirements for precision and continuous motion. interpretation 1.00 0.58 0.08 asserted 0.97
By overlapping inference with action execution, asynchronous execution can recover some of the lost success at the cost of more frequent inference requests. result 0.92 0.86 0.08 asserted 0.94
Building on these findings, we develop an empirical model that predicts task success from limited calibration data, achieving root-mean-square prediction errors of only 5.7-7.8% across various VLA policies. result 0.44 0.98 0.14 asserted 1.00
Given an inference system's latency and per-robot throughput, these predictions guide the selection of system-specific VLA algorithm configurations to maximize task success. contribution 0.44 0.76 0.25 asserted 0.97
We believe VLALagBench and these findings provide a solid foundation for future robotics algorithm-system co-design. future work 0.71 0.34 0.18 hedged possibility 0.62

What kind of paper the classifier reports this is: dataset or benchmark 0.99 · evidence from observation or experiment 0.99

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-04: 1 request, 6,876 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (27)
Figure 1 of the paper
Figure 1. VLALAGBENCH links VLA inference latency to task success through observation staleness and idle time, enabling a compact model to predict success at unseen latencies and choose the execution mode for a given deployment. The benchmark varies five factors—task type, target speed, inference latency, execution mode, and policy—to answer Q1–Q4. synchronous and various asynchronous execution methods. (5) Inference latency: controlled latency levels. Beyond our primary benchmark built on the MuJoCo simulator (Todorov et al., 2012), we also conduct experiments in Isaac Sim (an￾other…
Figure 2 of the paper
Figure 2. A VLA model combines a VLM backbone with an action expert that predicts a sequence of robot actions. 2.1 Vision-Language-Action Models Vision-language-action (VLA) models represent an impor￾tant step toward physical AI by using visual observations and natural-language instructions to generate robot actions for continuous control (Black et al., 2025b;a; NVIDIA et al., 2025). As shown in
Figure 4 of the paper
Figure 4. The four primary tasks in VLALAGBENCH cover releasing, setting down, grasping, and articulated-object manipulation. Each target, highlighted by a cyan box, moves continuously along the closed path shown in pink without colliding with background objects. 3 BENCHMARK AND EXECUTION MODES We present VLALAGBENCH, a benchmark for systemati￾cally studying how VLA inference latency affects task suc￾cess. As summarized in
Figure 5 of the paper
Figure 5. Synchronous and two types of asynchronous execution, assuming 100 ms inference latency and 50 ms per action. 3.5 Execution Modes We compare synchronous execution with two representative asynchronous schedules: lazy and continuous execution (
Figure 6 of the paper
Figure 6. Task success across target speeds and inference latencies using π0.5 with synchronous execution. Different tasks exhibit distinct latency sensitivities. Q2. What factors determine a task’s sensitivity to latency? Finding 2. A task’s latency sensitivity depends on its requirements for action precision and continuous motion. (2.1) Tasks requiring high spatial precision suffer from stale observations, whereas (2.2) tasks requiring sustained mo￾tion suffer from robot pauses. (2.1) Precision. Stale observations make precise manipula￾tion harder because the target moves between image…
Figure 7 of the paper
Figure 7. Hit rate of π0.5’s attempts against their horizontal offset from the target center when the object leaves the gripper (release), first touches the surface (set-down), or the fingertips descend below the object’s rim (grasping); the tolerance zone is the range of offsets where at least half of the attempts hit, and the photos show one hit inside it above one miss outside it. Set-down requires greater precision than the other tasks. to both. 4.3 Synchronous versus asynchronous inference Q3. What are the trade-offs between synchronous and asyn￾chronous execution as inference latency…
Figure 8 of the paper
Figure 8. Task success using π0.5 under synchronous and asyn￾chronous execution at 50 and 200 ms inference latency. line at both 50 and 200 ms. In particular, synchronous and lazy asynchronous execution have the same average obser￾vation staleness at the same inference latency (§3.5), so the recovery under lazy execution indicates that removing pauses accounts for much of the improvement. (3.2) Reducing observation staleness. For set-down and grasping, the two tasks more sensitive to stale observations (§4.2), continuous asynchronous execution can improve suc￾cess by updating actions more…
Figure 9 of the paper
Figure 9. Task success across VLA policies at 200 ms latency. The best-performing policy depends on the task and target speed. its observation staleness approaches that of lazy execution; at 200 ms, neither asynchronous mode consistently recovers zero-latency success on these two tasks (
Figure 10 of the paper
Figure 10. Model predictions track π0.5’s measured success at in￾ference latencies excluded from calibration. Dashed black curves show predictions; markers show measurements. idle time: λX = 1 + a s¯X − s¯0 s¯0 + b IX HX , (1) where s¯X and s¯0 are the mean observation staleness under configuration X and the reference configuration, respec￾tively; HX is the execution horizon, and IX is the number of idle steps per execution horizon. The ratio (¯sX − s¯0)/s¯0 measures the relative change in observation staleness, while IX/HX measures the number of idle control steps per ex￾ecuted action…
Figure 11 of the paper
Figure 11. π0.5 moves its hand at a nearly constant pace regardless of target speed, which asynchronous execution preserves and synchronous pauses reduce. Dotted lines mark hand speed equal to target speed. recorded once per 50 ms control step, where two bodies are in contact when their contact distance is ≤ 0. A con￾firmed pickup starts the first uninterrupted run of at least three samples with both fingers in contact with the object in which the object is at least once 2 cm above its post-warm￾up height. A release attempt begins a run of at least two samples without finger contact, after…
Figure 12 of the paper
Figure 12. At 100 and 200 ms latencies, lazy execution eliminates most synchronous losses on release and articulated-object manipulation, whereas at 200 ms set-down loses success on moving targets in all three modes. Curves show π0.5’s measured success at the calibration latencies. 17
Figure 13 of the paper
Figure 13. Continuous execution outperforms zero-latency execution on set-down for π0.5 at 50 ms before converging toward lazy execution as latency approaches lazy execution’s horizon, while 250 ms asynchronous execution loses success even near static. 18
Figure 14 of the paper
Figure 14. Policy rankings invert across tasks and speeds: GR00T N1.7 trails near static on three tasks but surpasses π0.5 on fast set-down targets, while MolmoAct2 leads on moving articulated targets. 19
Figure 15 of the paper
Figure 15. On articulated-object manipulation at 0.05 m/s, lazy execution at 200 ms raises GR00T N1.7’s success from 8.5% under synchronous execution to 34.0% (27.5% at zero latency), but on release at 0.1 m/s only from 30.0% to 32.0%; on grasping at 200 ms, synchronous execution beats lazy execution at six of eight moving-target speeds. 20
Figure 16 of the paper
Figure 16. On MolmoAct2, continuous execution at 50 ms exceeds zero-latency success on set-down from 0.075 m/s, by 9.0–27.5 points, and zero-latency success is lower at 0.001 m/s than at 0.025 m/s on release and set-down. 21
Figure 17 of the paper
Figure 17. Speed-scaling predictions fitted on 0, 100, and 200 ms runs track π0.5’s held-out 50, 150, and 250 ms curves in three representative settings; over every task and mode, the held-out error grows with latency (4.2, 5.5, and 7.0 points RMS). Dashed black curves show model predictions beside measured markers. 23
Figure 18 of the paper
Figure 18. Held-out predictions on MolmoAct2 capture speed scaling across latencies and carry the zero-latency dip of near-static set-down, with larger residuals at 250 ms. Dashed black curves show predictions from 0, 100, and 200 ms fits beside measured markers
Figure 19 of the paper
Figure 19. On set-down, the model predicts the highest π0.5 suc￾cess for continuous execution at nearly every moving-target speed up to 200 ms latency and for lazy and continuous execution, tied, at 250 ms; synchronous execution does not lead at any speed from 0.025 to 0.2 m/s. Cells show predicted success across target speeds and latencies from calibrating on one complete curve (continuous execution at 100 ms) plus one point on each of two other curves (synchronous and continuous execution at 200 ms). 4 5 10 20 Granted inference rate (Hz) 50 100 150 200 250 Latency (ms) Lazy 80% Cont. 84%…
Figure 20 of the paper
Figure 20. On set-down at 0.075 m/s, continuous execution gives π0.5 the highest predicted success up to 200 ms at 5 Hz or more, and lazy execution at 4 Hz and at 250 ms. Under a rate below one inference per latency, continuous execution requests once per granted period and switches to each chunk on arrival. 25
Figure 21 of the paper
Figure 21. compares the precision the two tasks require: grasping requires closing the fingers within 3 cm of the can’s center, whereas release tolerates drops within 12.5 cm of the box’s center. At a target speed of 0.1 m/s, 240 ms of additional observation staleness allows the target to move 2.4 cm, four-fifths of grasping’s tolerance zone. Finding 3 holds, except on slow release targets. Asyn￾chronous execution recovers task success, and continuous asynchronous execution recovers the most. At 240 ms, lazy execution keeps grasping within 7.5 points of zero latency at every speed, but raises…
Figure 22 of the paper
Figure 22. Our Isaac Sim verification replicates release and grasping with DynamicVLA along closed trajectories in robot-base coordinates. A cyan box marks the moving object and pink traces its path; an unrendered rail carrier moves the grasping can. 0 25 50 75 100 Synchronous (a) Release (b) Grasping 0 25 50 75 100 Lazy (c) Release (d) Grasping 0 0.2 0.4 0 25 50 75 100 Continuous (e) Release 0 0.1 0.2 (f) Grasping Target speed (m/s) Success rate (%) Sync. 120 ms Lazy 120 ms Cont. 120 ms 0 ms Sync. 240 ms Lazy 240 ms Cont. 240 ms
Figure 23 of the paper
Figure 23. On DynamicVLA in Isaac Sim, near-static grasping stays at 93.5% or higher in every setting, synchronous latency moves grasping success to lower speeds, and asynchronous grasping never falls more than 7.5 points below its zero-latency curve, while 120 ms continuous execution exceeds it from 0.075 m/s. 0 25 50 75 100 120 ms latency (a) Release (b) Grasping 0 0.2 0.4 0 25 50 75 100 240 ms latency (c) Release 0 0.1 0.2 (d) Grasping Target speed (m/s) Success rate (%) Sync. 120 ms Lazy 120 ms Cont. 120 ms 0 ms Sync. 240 ms Lazy 240 ms Cont. 240 ms
Figure 24 of the paper
Figure 24. Lazy execution keeps DynamicVLA’s grasp￾ing within 7.5 points of its zero-latency curve at 240 ms, while 120 ms continuous execution raises success on fast targets and lowers it on slow release targets. 27
Figure 25 of the paper
Figure 25. The four real-robot tasks pair grasping and release with a small and a big target, shown from the front camera at the start of a rollout and as the robot interacts with the moving target. Each moving target, highlighted by a cyan box, travels once along a straight path in the direction of the pink arrow. Static 0.02 0.05 0 1 2 3 4 5 (a) Grasping small Static 0.02 0.05 (b) Grasping big Static 0.02 0.05 (c) Release small Static 0.02 0.05 (d) Release big Target speed (m/s) Successes (of 5) Synchronous 267 ms Synchronous 367 ms Synchronous 467 ms
Figure 26 of the paper
Figure 26. Task success on the SO-101 arm across target speeds and inference latencies using MolmoAct2 with synchronous execution. Static targets succeed in every rollout, while moving targets lose success as latency increases. 0.02 0.05 0 1 2 3 4 5 (a) 267 ms latency 0.02 0.05 (b) 367 ms latency 0.02 0.05 (c) 467 ms latency Target speed (m/s) Successes (of 5) Sync. 267 ms Lazy 267 ms Cont. 267 ms Sync. 367 ms Lazy 367 ms Sync. 467 ms Lazy 467 ms
Figure 27 of the paper
Figure 27. Task success on release small using MolmoAct2 under synchronous and asynchronous execution at each inference latency. Continuous execution, and lazy execution at shorter latencies, recover success on the faster target. 29

Showing 26 of the paper's 27 figures. Figures are the authors', reproduced from the paper.

References (39)
  1. [1] Akinola, I., Xu, J., Song, S., and Allen, P. K. Dynamic grasping with reachability and motion awareness. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.\ 9422--9429, 2021. doi:10.1109/IROS51168.2021.9636057. URL https://arxiv.org/abs/2103.10562 on PithPith's reviewarXiv
  2. [2] Arachchige, N. R., Chen, Z., Jung, W., Shin, W. C., Bansal, R., Barroso, P., He, Y. H., Lin, Y. C., Joffe, B., Kousik, S., and Xu, D. SAIL : Faster-than-demonstration execution of imitation learning policies. In Proceedings of The 9th Conference on Robot Learning, volume 305 o... on Pith
  3. [3] Black, K., Brown, N., Darpinian, J., Dhabalia, K., Driess, D., Esmail, A., Equi, M. R., Finn, C., Fusai, N., Galliker, M. Y., Ghosh, D., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., LeBlanc, D., Levine, S., Li-Bell, A., Mothukuri, M., Nair, S., Pertsch... on Pith
  4. [4] Black, K., Brown, N., Driess, D., Esmail, A., Equi, M. R., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., Levine, S., Li-Bell, A., Mothukuri, M., Nair, S., Pertsch, K., Shi, L. X., Smith, L., Tanner, J., Vuong, Q., Walling, A., Wang,... on Pithdoi:10.15607/rss.2025.xxi.010
  5. [5] Black, K., Galliker, M. Y., and Levine, S. Real-time execution of action chunking flow policies. In Advances in Neural Information Processing Systems, volume 38, pp.\ 33383--33407, 2025 c . doi:10.52202/085713-1122. URL https://papers.nips.cc/paper_files/paper/2025/hash/300ccb... on Pithdoi:10.52202/085713-1122
  6. [6] Chen, T., Chen, Z., Chen, B., Cai, Z., Liu, Y., Li, Z., Liang, Q., Lin, X., Ge, Y., Gu, Z., Deng, W., Guo, Y., Nian, T., Xie, X., Chen, Q., Su, K., Xu, T., Liu, G., Hu, M., Gao, H.-a., Wang, K., Liang, Z., Qin, Y., Yang, X., Luo, P., and Mu, Y. RoboTwin 2.0 : A scalable data g... on Pith
  7. [7] Chen, Y., Zhang, W., and Li, X. Reflex : Enabling fast and predictive vision-language-action models for reaction-critical manipulation. arXiv preprint arXiv:2608.14379, 2026 b . URL https://arxiv.org/html/2608.14379v1 on PithPith's reviewarXiv
  8. [8] Chi, C., Feng, S., Du, Y., Xu, Z., Cousineau, E., Burchfiel, B. C., and Song, S. Diffusion policy: Visuomotor policy learning via action diffusion. In Proceedings of Robotics: Science and Systems, 2023. doi:10.15607/RSS.2023.XIX.026. URL https://roboticsproceedings.org/rss19/p026.html on Pithdoi:10.15607/rss.2023.xix.026
  9. [9] Fang, H., Duan, J., Clay, D., Wang, S., Liu, S., Huang, W., Fan, X., Tsai, W.-C., Chen, S., Wang, Y. R., Xing, S., Cho, J., Park, J. S., Eftekhar, A., Sushko, P., Farley, K., Wadhwa, A., Harrison, C., Han, W., Lee, Y.-C., VanderBilt, E., Hendrix, R., Ellawela, S., Ngoo, L., Ch... on PithPith's reviewarXiv
  10. [10] Guo, L., Xue, Z., Xu, Z., and Xu, H. DemoSpeedup : Accelerating visuomotor policies via entropy-guided demonstration acceleration. In Proceedings of The 9th Conference on Robot Learning, volume 305 of Proceedings of Machine Learning Research, pp.\ 599--609. PMLR, 2025. URL htt... on Pith
  11. [11] Jiang, W., Clemons, J., Sankaralingam, K., and Kozyrakis, C. How fast can I run my VLA ? Demystifying VLA inference performance with VLA-Perf . arXiv preprint arXiv:2602.18397, 2026. URL https://arxiv.org/html/2602.18397v1 on PitharXiv
  12. [12] Jing, D., Nie, J., Zhang, T., Liu, J., Yao, H., Lu, Z., and Ding, M. TempoVLA : Learning speed-controllable vision-language-action policies. arXiv preprint arXiv:2606.06491, 2026. URL https://arxiv.org/abs/2606.06491 on PithPith's reviewarXiv
  13. [13] Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E. P., Sanketi, P. R., Vuong, Q., Kollar, T., Burchfiel, B., Tedrake, R., Sadigh, D., Levine, S., Liang, P., and Finn, C. OpenVLA : An open-source vision-language-action model. ... on Pith
  14. [14] Lee, J., Duan, J., Fang, H., Deng, Y., Liu, S., Li, B., Fang, B., Zhang, J., Wang, Y. R., Lee, S., Han, W., Pumacay, W., Wu, A., Hendrix, R., Farley, K., Vanderbilt, E., Farhadi, A., Fox, D., and Krishna, R. MolmoAct : Action reasoning models that can reason in space. In 2026 ... on Pithdoi:10.1109/icra57385.2026.11696546
  15. [15] Li, X., Hsu, K., Gu, J., Mees, O., Pertsch, K., Walke, H. R., Fu, C., Lunawat, I., Sieh, I., Kirmani, S., Levine, S., Wu, J., Finn, C., Su, H., Vuong, Q., and Xiao, T. Evaluating real-world robot manipulation policies in simulation. In Proceedings of The 8th Conference on Robo... on Pith
  16. [16] Liu, B., Zhu, Y., Gao, C., Feng, Y., Liu, Q., Zhu, Y., and Stone, P. LIBERO : Benchmarking knowledge transfer for lifelong robot learning. In Advances in Neural Information Processing Systems, volume 36, pp.\ 44776--44791, 2023. doi:10.52202/075280-1939. URL https://proceeding... on Pithdoi:10.52202/075280-1939
  17. [17] Ma, Y., Zhou, Y., Yang, Y., Wang, T., and Fan, H. Running VLA s at real-time speed. arXiv preprint arXiv:2510.26742, 2025. URL https://arxiv.org/abs/2510.26742 on PitharXiv
  18. [18] Matthews, M., Beukman, M., Lu, C., and Foerster, J. Kinetix : Investigating the training of general agents through open-ended physics-based control tasks. In International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=zCxGCdzreM on Pith
  19. [19] Nasiriany, S., Maddukuri, A., Zhang, L., Parikh, A., Lo, A., Joshi, A., Mandlekar, A., and Zhu, Y. RoboCasa : Large-scale simulation of everyday tasks for generalist robots. In Proceedings of Robotics: Science and Systems, 2024. doi:10.15607/RSS.2024.XX.050. URL https://www.ro... on Pithdoi:10.15607/rss.2024.xx.050
  20. [20] Nasiriany, S., Nasiriany, S., Maddukuri, A., and Zhu, Y. RoboCasa365 : A large-scale simulation framework for training and benchmarking generalist robots. In International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=tQJYKwc3n4 on Pith
show 19 more
  1. [21] Niu, J., Gu, K., Zhao, Y., Liang, S., Wang, T., Hu, X., Wang, Y., and Li, H. Realtime-VLA FLASH : Speculative inference framework for diffusion-based VLA s. In Advances in Neural Information Processing Systems, 2026. URL https://arxiv.org/abs/2605.13778. To appear on PithPith's reviewarXiv
  2. [22] NVIDIA . GR00T N1.7 LIBERO : Model card and checkpoints. Hugging Face model repository, 2026 a . URL https://huggingface.co/nvidia/GR00T-N1.7-LIBERO. Evaluated revision: 2ea293aa20ba7cf5bbf3ba17a5fbcb1a01cbfe21. LIBERO results: Isaac-GR00T examples/LIBERO/README.md, commit 1a1... on Pith
  3. [23] NVIDIA . GR00T N1.7 SimplerEnv evaluation. GitHub repository, 2026 b . URL https://github.com/NVIDIA/Isaac-GR00T/blob/1a1837f20538b7d7e21f977a11a5aee14f99803c/examples/SimplerEnv/README.md. examples/SimplerEnv/README.md, commit 1a1837f20538b7d7e21f977a11a5aee14f99803c on Pith
  4. [24] NVIDIA . Isaac Sim . Robotics simulation software and documentation, n.d. URL https://docs.isaacsim.omniverse.nvidia.com/latest/index.html. Accessed October 19, 2026 on Pith
  5. [25] NVIDIA , Bjorck, J., Casta \ n eda, F., Cherniadev, N., Da, X., Ding, R., Fan, L. J., Fang, Y., Fox, D., Hu, F., Huang, S., Jang, J., Jiang, Z., Kautz, J., Kundalia, K., Lao, L., Li, Z., Lin, Z., Lin, K., Liu, G., Llontop, E., Magne, L., Mandlekar, A., Narayan, A., Nasiriany, ... on PithPith's reviewarXiv
  6. [26] Physical Intelligence . openpi: LIBERO benchmark example and results. GitHub repository, 2025. URL https://github.com/Physical-Intelligence/openpi/blob/2d70d966582e711128ad8358d8dbf23d2cc3d658/examples/libero/README.md. examples/libero/README.md, commit 2d70d966582e711128ad835... on Pith
  7. [27] Shukor, M., Aubakirova, D., Capuano, F., Kooijmans, P., Palma, S., Zouitine, A., Aractingi, M., Pascal, C., Russi, M., Marafioti, A., Alibert, S., Cord, M., Wolf, T., and Cadene, R. SmolVLA : A vision-language-action model for affordable and efficient robotics. arXiv preprint ... on PithPith's reviewarXiv
  8. [28] Su, L. Execution-state capsules: Graph-bound execution-state checkpoint and restore for low-latency, small-batch, on-device physical- AI serving. arXiv preprint arXiv:2606.20537, 2026. doi:10.48550/arXiv.2606.20537. URL https://arxiv.org/abs/2606.20537 on PithPith's reviewarXivdoi:10.48550/arxiv.2606.20537
  9. [29] Tang, J., Sun, Y., Zhao, Y., Yang, S., Lin, Y., Zhang, Z., Hou, J., Lu, Y., Liu, Z., and Han, S. VLASH : Real-time VLAs via future-state-aware asynchronous inference. In 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2026. URL https://arxiv.or... on PithPith's reviewarXiv
  10. [30] TheRobotStudio . SO-ARM100 : Standard open arm 100 ( SO-100 and SO-101 ). GitHub repository, 2025. URL https://github.com/TheRobotStudio/SO-ARM100 on Pith
  11. [31] Todorov, E., Erez, T., and Tassa, Y. MuJoCo : A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp.\ 5026--5033. IEEE, 2012. doi:10.1109/IROS.2012.6386109. URL https://homes.cs.washington.edu/ todorov/papers/... on Pithdoi:10.1109/iros.2012.6386109
  12. [32] Xie, H., Wen, B., Zheng, J., Chen, Z., Hong, F., Diao, H., and Liu, Z. DynamicVLA : A vision-language-action model for dynamic object manipulation. In Advances in Neural Information Processing Systems, 2026. URL https://arxiv.org/abs/2601.22153. To appear on PithPith's reviewarXiv
  13. [33] Xu, B., Tan, H., Jiang, W., He, B., Wu, M., Zeng, J., and Heinrich, M. VLA-HWBench : Where the time goes when VLA models run on edge hardware. arXiv preprint arXiv:2610.34717, 2026. URL https://arxiv.org/abs/2610.34717 on PithPith's reviewarXiv
  14. [34] Yang, C., Hu, Y., Ma, Y., Yang, Y., Tan, J., and Fan, H. Realtime-VLA V2 : Learning to run VLA s fast, smooth, and accurate. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.\ 4501--4509, 2026 a . URL https://openaccess.... on Pith
  15. [35] Yang, X., Dagli, R., Zook, A., Hadfield, H., Goyal, A., Birchfield, S., Ramos, F., and Tremblay, J. RoboLab : A high-fidelity simulation benchmark for analysis of task generalist policies. In Proceedings of Robotics: Science and Systems, 2026 b . doi:10.15607/RSS.2026.XXII.096... on Pithdoi:10.15607/rss.2026.xxii.096
  16. [36] Yu, T., Quillen, D., He, Z., Julian, R., Hausman, K., Finn, C., and Levine, S. Meta-World : A benchmark and evaluation for multi-task and meta reinforcement learning. In Proceedings of the Conference on Robot Learning, volume 100 of Proceedings of Machine Learning Research, pp... on Pith
  17. [37] Zhang, Y., Song, C., Singh, V., Li, X., Ye, K., Hu, Z., Pu, Z., Yin, Y., and Chaudhary, V. Overcoming dynamics-blindness: Training-free pace-and-path correction for VLA models. arXiv preprint arXiv:2605.11459, 2026. URL https://arxiv.org/html/2605.11459v1 on PithPith's reviewarXiv
  18. [38] Zhao, T. Z., Kumar, V., Levine, S., and Finn, C. Learning fine-grained bimanual manipulation with low-cost hardware. In Proceedings of Robotics: Science and Systems, 2023. doi:10.15607/RSS.2023.XIX.016. URL https://roboticsproceedings.org/rss19/p016.html on Pithdoi:10.15607/rss.2023.xix.016
  19. [39] Zhou, X., Xu, Y., Tie, G., Chen, Y., Zhang, G., Chu, D., Zhou, P., and Sun, L. LIBERO-PRO : Towards robust and fair evaluation of vision-language-action models beyond memorization. arXiv preprint arXiv:2510.03827, 2025. URL https://arxiv.org/abs/2510.03827 on PithPith's reviewarXiv

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment