Robotics · arXiv:2611.03103 v1 · submitted 2 Nov 2026 · 39 references
VLALagBench: Understanding When and Why VLA Inference Latency Matters
The paper says
“Vision-language-action (VLA) models offer a promising path toward general-purpose robot manipulation. While inference latency can affect task success in a constantly evolving physical world, the relationship between latency and task success rate remains poorly understood.”
more of the abstract
In this paper, we introduce VLALagBench, a comprehensive benchmark for studying VLA inference latency sensitivity across task types, object speeds, execution modes (synchronous and asynchronous), and VLA policies. Using VLALagBench, we find that even a modest inference latency of 100 ms can nearly halve task success on moving targets, with sensitivity varying substantially across tasks and policies. These differences reflect task-specific requirements for precision and continuous motion. By overlapping inference with action execution, asynchronous execution can recover some of the lost success at the cost of more frequent inference requests. Building on these findings, we develop an empirical model that predicts task success from limited calibration data, achieving root-mean-square prediction errors of only 5.7-7.8% across various VLA policies. Given an inference system's latency and per-robot throughput, these predictions guide the selection of system-specific VLA algorithm configurations to maximize task success. We believe VLALagBench and these findings provide a solid foundation for future robotics algorithm-system co-design.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 3 Nov 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. Well-measured latency benchmark with a genuinely predictive empirical model; the core findings are solid, and the main weakness is that the deployment-selection claim rests on a single validation cell. This line is wrong
2 major objections · 4 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 39 references say.
The paper says, at §5.2 / Table 3
The review did not quote this passage.
Pith says: Objection 1 of 2 · major
The deployment-selection contribution is validated on exactly one policy-task-speed cell. In §5.2, the model is calibrated for π0.5 on set-down at 0.075 m/s from one complete continuous-100 ms curve plus two 200 ms points, and Systems A–D differ only in injected latency and per-robot rate limit over the same MuJoCo success surface.
more of objection 1
System A's selected configuration is itself the calibration curve, so its row in Table 3 is not an independent validation point. This demonstrates that Eq. (1) can rank four latency/throughput operating points for one cell, but not the abstract's general claim that the predictions 'guide the selection of system-specific VLA algorithm configurations.' Because a and b are refitted for every policy-task pair and the reference curve is measured per task, cross-task and cross-policy generalization does not follow automatically. Please add at least one additional policy-task cell to the §5.2 validation, ideally under a rate-limited configuration, or explicitly restrict the deployment-selection claim to the demonstrated cell.
The paper says, at §5.1 / Eq. (1), Table 2, Appendix D
The review did not quote this passage.
Pith says: Objection 2 of 2 · major
The headline accuracy numbers (5.7–7.8 points RMS) are reported for π0.5 and MolmoAct2, while GR00T N1.7's limited-data calibration has a median RMS error of 11.6 points and fitted coefficients whose signs contradict the proposed staleness and idle-time mechanisms on set-down and grasping (Table 2).
more of objection 2
Appendix D also documents two systematic departures that Eq. (1) cannot represent: at 250 ms, lazy and continuous execution lose near-static success on release and set-down, and on the fastest targets synchronous execution beats asynchronous execution despite the model's prediction that synchronous execution should be worse. These limitations are honestly acknowledged, but because Eq. (1) is the foundation of the §5.2 selection claim, the model's validity domain should be stated as a scope condition in the abstract and Section 5 rather than only in appendix caveats. A concrete test would be a deployment-selection validation on a policy-task pair where Eq. (1) actually fits, together with a statement of which of the paper's conclusions depend on the model rather than on the direct measurements.
4 minor objections
The paper says, at Throughout
Pith says: minor objection 1 of 4
The benchmark name is spelled both 'VLALagBench' and 'VLALAGBENCH'; please standardize the spelling in the abstract, main text, and figures.
The paper says, at §3.5 / Figure 5
Pith says: minor objection 2 of 4
The statement that synchronous and lazy asynchronous execution have identical average observation staleness is central to the pause-vs-staleness attribution; a one-line algebraic derivation in the text would make this easier to verify than reading it off Figure 5.
show 2 more
The paper says, at Table 3 / Appendix D
Pith says: minor objection 3 of 4
The latencies for Systems B–D combine measured GPU latencies with network latency estimates taken from VLA-Perf rather than measured in the deployment scenario; a short sensitivity analysis over the network estimate would strengthen the recommendation.
The paper says, at Appendix F
Pith says: minor objection 4 of 4
The real-robot results use five rollouts per condition, which is appropriate for a pilot but should be explicitly labeled as a pilot verification in the main text rather than appearing only in the appendix.
What would settle it
Run the exact Table 3 calibration protocol — one complete continuous-100 ms curve plus one synchronous and one continuous point at 200 ms — for π0.5 on release and grasping and for MolmoAct2 on set-down, at both 0.075 and 0.1 m/s, and compare the predicted best (mode, H) against the measured best under the same four latency/throughput constraints as Systems A–D. Report the match rate over these cells; if the model selects the measured best configuration in fewer than 75% of cells, or misses by more than 5 success-rate points in more than two cells, the system-selection claim should be downgraded to a single-case demonstration.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 3 Nov 2026
- Reader holds up, with conditions
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 2 major objections, 4 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Two things to know up front. The measurements are well built: 200 rollouts per condition, Wilson intervals, latency injected as control steps with the action budget extended so the policy isn't starved, and a stated sampling-noise floor that the model is judged against. The headline — 100 ms of latency can nearly halve success on moving targets, with sensitivity varying by task and policy — is direct measurement, not a fitted artifact.
What's new: ReflexBench already varied latency but on one policy and two tasks; MoveBench and DOM don't sweep latency at all. This paper is the first joint controlled sweep of task type, target speed, execution mode, and three generalist policies, and it turns the vague intuition about latency into two measurable mechanisms: stale observations hurt precision-sensitive tasks, robot pauses hurt continuous-motion tasks. The tolerance-zone analysis (Fig. 7) and stage pass rates (Table 8) genuinely support that decomposition. The Eq. (1) model is also useful: held-out RMS of 5.7 points for π0.5 against a 4.0-point noise floor is real predictive power, and the authors report its failures (GR00T's negative fitted coefficients, near-static async at 250 ms) rather than burying them.
Soft spots, in proportion. The stress-test note lands: the §5.2 deployment-selection claim is validated on exactly one cell — π0.5, set-down, 0.075 m/s. The four 'systems' differ only in injected latency and throughput cap over the same MuJoCo surface, so cross-task and cross-policy generality of the selection procedure is unestablished. This is the biggest gap between the abstract's 'guide the selection' claim and the evidence. Not fatal: the model itself is validated across four tasks and held-out latencies for π0.5 and MolmoAct2, so the mechanism isn't fragile, just the system-selection demonstration is thin. Note System A in Table 3 is its own calibration curve; the held-out evidence in that table is Systems B–D, plus the 133-point validation in Appendix D.
Second, the benchmark artifacts aren't released; for a benchmark paper that's a concrete omission. Third, the abstract's 5.7–7.8% range covers two of the three policies; GR00T's error under the same protocol isn't reported in the main table (its limited-calibration median in Appendix D is 11.6), so 'across various VLA policies' mildly overstates coverage. Fourth, the real-robot confirmation is five rollouts per condition on one policy — fine as a sanity check, thin as evidence. GR00T's sensitivity to initial target positions (Table 1) is a real confound for its fitted coefficients; the paper acknowledges it, to its credit, but it means the speed-scaling story should be read as π0.5/MolmoAct2-first.
This paper deserves a serious referee. The revision that would make it solid: release the code, data, and waypoints; temper the abstract; report GR00T's held-out error in the main table; and either extend §5.2 to a second task or policy or state plainly that selection is demonstrated for one cell.
The strongest objection
The paper's third contribution — using the §5.1 model to select execution mode and horizon for a given inference system — is supported by a single validation cell. Table 3 calibrates the model on π0.5, set-down, target speed 0.075 m/s, using one complete continuous-100 ms curve plus two calibration points, and then evaluates Systems A–D on that same task, policy, and speed. The differences between the four systems are only the injected latency and the per-robot throughput cap; the underlying success surface is the same MuJoCo set-down surface. The experiment therefore demonstrates that Eq. (1) can rank four latency/rate operating points for one policy-task-speed combination, not that it selects configurations across tasks, policies, or speeds. Because a and b in Eq. (1) are refit for every policy-task pair and the reference curve is measured per task, the abstract's general 'guide the selection' claim needs cross-task and cross-policy evidence. The acknowledged failures for GR00T N1.7 (Table 2) and near-static async at 250 ms (Appendix D) are honestly reported and are not the basis of this concern; they make the lack of replication in the flagship deployment validation more consequential.
The direct empirical claim — that 100 ms of inference latency can nearly halve success on moving targets — is well supported by 200-rollout conditions with Wilson intervals, cross-simulator checks in Isaac Sim, and real-robot verification, so I do not object to Finding 1. The empirical status of Eq. (1) is not by itself the decisive problem: the paper reports held-out validation, compares RMS error to a 4.0-point sampling-noise floor, and explicitly lists the GR00T N1.7 and near-static async departures. The unresolved gap is that the model's advertised use for system-specific configuration selection is demonstrated in only one policy-task-speed cell, with the four 'systems' differing only by latency and rate limit over the same simulator surface. My proposed check would settle whether the selection protocol generalizes beyond that cell. This reinforces, rather than moves, the reader's CONDITIONAL verdict: the benchmark measurements are credible, but the model-guided selection contribution needs broader validation and the release of artifacts before the abstract-level claim is fully supported.
The referee's report
If the findings hold, the benchmark fills a real gap: existing dynamic-manipulation benchmarks do not systematically isolate inference latency while varying task type, speed, execution mode, and policy. The strengths are substantial: controlled latency injection, 200 rollouts per condition, 95% Wilson intervals, held-out latency predictions whose RMS error is reported against an estimated sampling-noise floor, and honest, clearly located discussion of the model's failures for GR00T N1.7 and near-static asynchronous behavior at 250 ms. The paper also verifies the main findings in a second simulator and on a real robot. The main weakness is that the third contribution, system-specific configuration selection, is validated on exactly one policy-task-speed cell, while the abstract states the guidance claim in general terms. The empirical model itself is an ansatz rather than a derivation, but it is evaluated on held-out latencies; the concern is scope, not circularity of the core measurements.
Pith's reading of the paper (a machine's summary, not the authors')
This paper tries to establish a quantitative answer to a question robotics system builders currently answer by guesswork: how many points of task success does VLA inference latency cost, and why. Working from four moving-target manipulation tasks in the LIBERO/MuJoCo simulator, three open VLA policies, and a controlled sweep of latencies, execution modes, and target speeds, it finds that even 100 ms of inference latency can nearly halve success on a precision task -- set-down drops from 71.5% to 40.5% at 0.1 m/s -- while a more tolerant grasping task keeps 86% of its zero-latency success. The paper attributes the loss to two separable mechanisms, stale observations and robot pauses, and shows that asynchronous execution recovers part of the damage at the price of more frequent inference requests. On top of these findings it claims that the whole latency effect can be summarized by reading a measured reference success curve at a rescaled target speed, with a two-coefficient adjustment linear in staleness and idle time; this model predicts held-out latencies within 5.7-7.8 points RMS and selects the measured-best execution mode and horizon on all four deployment systems it considers. If the claim holds, a few calibration measurements on one system could replace an expensive configuration sweep on every new GPU, network, or cloud endpoint.
What carries the argument. The load-bearing object is Eq. (1), a speed-scaling identity: the success of configuration $X$ is the reference curve $S_0$ read at a rescaled target speed $\lambda_X v$, where $\lambda_X$ is linear in the relative change of mean observation staleness and in idle steps per executed action. Its function is to compress an expensive configuration search into a curve lookup: once $S_0$ and the two coefficients are measured, every combination of latency, execution mode, and execution horizon maps to one point on one curve, and the coefficients themselves become an interpretable diagnostic -- a large $a$ means the task is hurt by stale observations, a large $b$ means it is hurt by pauses. The identity also supports calibration without zero-latency data, because $S_X(v) \approx S_R((\lambda_X/\lambda_R)v)$ lets any measured curve serve as the reference, provided one calibrated configuration is synchronous at nonzero latency so that idle time can be separated from staleness.
Core claim. On its own terms, the paper's central claim is that inference latency damages VLA task success through exactly two channels -- the observation the policy acts on is stale by the time the action executes, and the robot sits idle while inference runs -- and that their combined effect is equivalent, for the purpose of predicting success, to making the target move faster. That equivalence is formalized as $S_X(v) \approx S_0(\lambda_X v)$, with $\lambda_X = 1 + a\,(\bar{s}_X - \bar{s}_0)/\bar{s}_0 + b\,I_X/H_X$, where $S_0$ is a measured reference success curve and the two fitted coefficients $a$ and $b$ quantify a policy-task pair's sensitivity to staleness and idle time respectively. Fitted on zero-, 100-, and 200-ms runs, the model predicts held-out 50-, 150-, and 250-ms curves with 5.7 points RMS error on $\pi 0.5$ and 7.8 on MolmoAct2, close to the 4-point sampling noise of the measurements themselves; with only one complete curve plus two points it holds a median 7.7-point error. Applied to four deployment systems differing in GPU latency and per-robot throughput, it predicts the measured best execution mode and horizon in every case. The paper is explicit about where the equivalence stops: GR00T N1.7 fits negative coefficients on some tasks, partly because changing initial target positions already costs that policy 30-40 points at zero latency, and near-static asynchronous runs at 250 ms lose success the model cannot see.
The premise it rests on. Everything rests on the equivalence between latency damage and a faster target: success under any configuration is read off one measured curve at a speed rescaled linearly by staleness and idle time, an equivalence the paper itself shows already breaks for GR00T N1.7 and for near-static asynchronous execution at 250 ms.
If the paper is right (Pith's inferences, not the authors')
For a given deployment, the best execution mode and horizon can be chosen from three or four measurements: the model matched the measured best configuration on all four representative systems, from onboard GPU to slow cloud.
Continuous asynchronous execution at short latency can beat zero-latency synchronous execution on precision tasks because it freshens observations: at 50 ms it raised set-down success at 0.125 m/s from 51.5% to 70.0%.
Latency sensitivity is not a single number per policy: it factors into task precision tolerance and motion-continuity demands, so the question of how fast inference must be has to be answered per task, per target speed, and per policy.
The two fitted coefficients per policy-task pair give a cheap screening test for which failure channel dominates, predicting whether asynchronous execution will help before a full sweep is run.
Held-out prediction error on unseen latencies (5.7-7.8 points RMS) is within about two points of the measurements' own 4-point sampling noise, so further accuracy gains would need more rollouts or better measurement rather than a richer model.
If the speed-scaling identity holds beyond the tested grid, it can be inverted into a design specification: for a target success rate and task, $S_0$ and $\lambda$ give the maximum tolerated latency and idle-time budget, letting a systems team state a hardware requirement ('no more than X ms end-to-end') in task-success terms instead of frames-per-second terms.
The two-channel decomposition suggests a direct evaluation recipe for latency-robust training: policies trained with deliberately stale observations or inserted idle gaps should show smaller fitted $a$ and $b$, so the benchmark's coefficients could serve as the metric for such methods.
The model's acknowledged failures -- GR00T's negative coefficients and the 250 ms near-static asynchronous drop -- point to a missing third mechanism, plausibly chunk-boundary discontinuities or sensitivity to initial target position; the residuals at extreme latencies are where such a term would be identified and tested.
Because the best policy depends on task and speed as well as latency, single-number leaderboard rankings of VLA models are under-specified for real-time robotics; a latency-by-speed success matrix is closer to what a deployment decision actually needs.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Vision-language-action (VLA) models offer a promising path toward general-purpose robot manipulation. While inference latency can affect task success in a constantly evolving physical world, the relationship between latency and task success rate remains poorly understood. In this paper, we introduce VLALagBench, a comprehensive benchmark for studying VLA inference latency sensitivity across task types, object speeds, execution modes (synchronous and asynchronous), and VLA policies. Using VLALagBench, we find that even a modest inference latency of 100 ms can nearly halve task success on moving targets, with sensitivity varying substantially across tasks and policies. These differences reflect task-specific requirements for precision and continuous motion. By overlapping inference with action execution, asynchronous execution can recover some of the lost success at the cost of more frequent inference requests. Building on these findings, we develop an empirical model that predicts task success from limited calibration data, achieving root-mean-square prediction errors of only 5.7-7.8% across various VLA policies. Given an inference system's latency and per-robot throughput, these predictions guide the selection of system-specific VLA algorithm configurations to maximize task success. We believe VLALagBench and these findings provide a solid foundation for future robotics algorithm-system co-design.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-04
The authors' main claim, as the classifier picked it out 0.90
“Using VLALagBench, we find that even a modest inference latency of 100 ms can nearly halve task success on moving targets, with sensitivity varying substantially across tasks and policies.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.09 · states limitation 0.28 · human participants 0.05 · controlled comparison 0.33 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.23 · mentions code or data availability 0.06
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Vision-language-action (VLA) models offer a promising path toward general-purpose robot manipulation. | background 1.00 | 0.04 | 0.02 | hedged possibility 0.58 |
| While inference latency can affect task success in a constantly evolving physical world, the relationship between latency and task success rate remains poorly understood. | gap 0.99 | 0.06 | 0.03 | asserted 0.58 |
| In this paper, we introduce VLALagBench, a comprehensive benchmark for studying VLA inference latency sensitivity across task types, object speeds, execution modes (synchronous and asynchronous), and VLA policies. | contribution 0.94 | 0.89 | 0.05 | asserted 0.99 |
| Using VLALagBench, we find that even a modest inference latency of 100 ms can nearly halve task success on moving targets, with sensitivity varying substantially across tasks and policies. | result 1.00 | 0.97 | 0.63 | asserted 1.00 |
| These differences reflect task-specific requirements for precision and continuous motion. | interpretation 1.00 | 0.58 | 0.08 | asserted 0.97 |
| By overlapping inference with action execution, asynchronous execution can recover some of the lost success at the cost of more frequent inference requests. | result 0.92 | 0.86 | 0.08 | asserted 0.94 |
| Building on these findings, we develop an empirical model that predicts task success from limited calibration data, achieving root-mean-square prediction errors of only 5.7-7.8% across various VLA policies. | result 0.44 | 0.98 | 0.14 | asserted 1.00 |
| Given an inference system's latency and per-robot throughput, these predictions guide the selection of system-specific VLA algorithm configurations to maximize task success. | contribution 0.44 | 0.76 | 0.25 | asserted 0.97 |
| We believe VLALagBench and these findings provide a solid foundation for future robotics algorithm-system co-design. | future work 0.71 | 0.34 | 0.18 | hedged possibility 0.62 |
What kind of paper the classifier reports this is: dataset or benchmark 0.99 · evidence from observation or experiment 0.99
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-04: 1 request, 6,876 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (27)
Showing 26 of the paper's 27 figures. Figures are the authors', reproduced from the paper.
References (39)
- [1] Akinola, I., Xu, J., Song, S., and Allen, P. K. Dynamic grasping with reachability and motion awareness. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.\ 9422--9429, 2021. doi:10.1109/IROS51168.2021.9636057. URL https://arxiv.org/abs/2103.10562 on PithPith's reviewarXiv
- [2] Arachchige, N. R., Chen, Z., Jung, W., Shin, W. C., Bansal, R., Barroso, P., He, Y. H., Lin, Y. C., Joffe, B., Kousik, S., and Xu, D. SAIL : Faster-than-demonstration execution of imitation learning policies. In Proceedings of The 9th Conference on Robot Learning, volume 305 o... on Pith
- [3] Black, K., Brown, N., Darpinian, J., Dhabalia, K., Driess, D., Esmail, A., Equi, M. R., Finn, C., Fusai, N., Galliker, M. Y., Ghosh, D., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., LeBlanc, D., Levine, S., Li-Bell, A., Mothukuri, M., Nair, S., Pertsch... on Pith
- [4] Black, K., Brown, N., Driess, D., Esmail, A., Equi, M. R., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., Levine, S., Li-Bell, A., Mothukuri, M., Nair, S., Pertsch, K., Shi, L. X., Smith, L., Tanner, J., Vuong, Q., Walling, A., Wang,... on Pithdoi:10.15607/rss.2025.xxi.010
- [5] Black, K., Galliker, M. Y., and Levine, S. Real-time execution of action chunking flow policies. In Advances in Neural Information Processing Systems, volume 38, pp.\ 33383--33407, 2025 c . doi:10.52202/085713-1122. URL https://papers.nips.cc/paper_files/paper/2025/hash/300ccb... on Pithdoi:10.52202/085713-1122
- [6] Chen, T., Chen, Z., Chen, B., Cai, Z., Liu, Y., Li, Z., Liang, Q., Lin, X., Ge, Y., Gu, Z., Deng, W., Guo, Y., Nian, T., Xie, X., Chen, Q., Su, K., Xu, T., Liu, G., Hu, M., Gao, H.-a., Wang, K., Liang, Z., Qin, Y., Yang, X., Luo, P., and Mu, Y. RoboTwin 2.0 : A scalable data g... on Pith
- [7] Chen, Y., Zhang, W., and Li, X. Reflex : Enabling fast and predictive vision-language-action models for reaction-critical manipulation. arXiv preprint arXiv:2608.14379, 2026 b . URL https://arxiv.org/html/2608.14379v1 on PithPith's reviewarXiv
- [8] Chi, C., Feng, S., Du, Y., Xu, Z., Cousineau, E., Burchfiel, B. C., and Song, S. Diffusion policy: Visuomotor policy learning via action diffusion. In Proceedings of Robotics: Science and Systems, 2023. doi:10.15607/RSS.2023.XIX.026. URL https://roboticsproceedings.org/rss19/p026.html on Pithdoi:10.15607/rss.2023.xix.026
- [9] Fang, H., Duan, J., Clay, D., Wang, S., Liu, S., Huang, W., Fan, X., Tsai, W.-C., Chen, S., Wang, Y. R., Xing, S., Cho, J., Park, J. S., Eftekhar, A., Sushko, P., Farley, K., Wadhwa, A., Harrison, C., Han, W., Lee, Y.-C., VanderBilt, E., Hendrix, R., Ellawela, S., Ngoo, L., Ch... on PithPith's reviewarXiv
- [10] Guo, L., Xue, Z., Xu, Z., and Xu, H. DemoSpeedup : Accelerating visuomotor policies via entropy-guided demonstration acceleration. In Proceedings of The 9th Conference on Robot Learning, volume 305 of Proceedings of Machine Learning Research, pp.\ 599--609. PMLR, 2025. URL htt... on Pith
- [11] Jiang, W., Clemons, J., Sankaralingam, K., and Kozyrakis, C. How fast can I run my VLA ? Demystifying VLA inference performance with VLA-Perf . arXiv preprint arXiv:2602.18397, 2026. URL https://arxiv.org/html/2602.18397v1 on PitharXiv
- [12] Jing, D., Nie, J., Zhang, T., Liu, J., Yao, H., Lu, Z., and Ding, M. TempoVLA : Learning speed-controllable vision-language-action policies. arXiv preprint arXiv:2606.06491, 2026. URL https://arxiv.org/abs/2606.06491 on PithPith's reviewarXiv
- [13] Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E. P., Sanketi, P. R., Vuong, Q., Kollar, T., Burchfiel, B., Tedrake, R., Sadigh, D., Levine, S., Liang, P., and Finn, C. OpenVLA : An open-source vision-language-action model. ... on Pith
- [14] Lee, J., Duan, J., Fang, H., Deng, Y., Liu, S., Li, B., Fang, B., Zhang, J., Wang, Y. R., Lee, S., Han, W., Pumacay, W., Wu, A., Hendrix, R., Farley, K., Vanderbilt, E., Farhadi, A., Fox, D., and Krishna, R. MolmoAct : Action reasoning models that can reason in space. In 2026 ... on Pithdoi:10.1109/icra57385.2026.11696546
- [15] Li, X., Hsu, K., Gu, J., Mees, O., Pertsch, K., Walke, H. R., Fu, C., Lunawat, I., Sieh, I., Kirmani, S., Levine, S., Wu, J., Finn, C., Su, H., Vuong, Q., and Xiao, T. Evaluating real-world robot manipulation policies in simulation. In Proceedings of The 8th Conference on Robo... on Pith
- [16] Liu, B., Zhu, Y., Gao, C., Feng, Y., Liu, Q., Zhu, Y., and Stone, P. LIBERO : Benchmarking knowledge transfer for lifelong robot learning. In Advances in Neural Information Processing Systems, volume 36, pp.\ 44776--44791, 2023. doi:10.52202/075280-1939. URL https://proceeding... on Pithdoi:10.52202/075280-1939
- [17] Ma, Y., Zhou, Y., Yang, Y., Wang, T., and Fan, H. Running VLA s at real-time speed. arXiv preprint arXiv:2510.26742, 2025. URL https://arxiv.org/abs/2510.26742 on PitharXiv
- [18] Matthews, M., Beukman, M., Lu, C., and Foerster, J. Kinetix : Investigating the training of general agents through open-ended physics-based control tasks. In International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=zCxGCdzreM on Pith
- [19] Nasiriany, S., Maddukuri, A., Zhang, L., Parikh, A., Lo, A., Joshi, A., Mandlekar, A., and Zhu, Y. RoboCasa : Large-scale simulation of everyday tasks for generalist robots. In Proceedings of Robotics: Science and Systems, 2024. doi:10.15607/RSS.2024.XX.050. URL https://www.ro... on Pithdoi:10.15607/rss.2024.xx.050
- [20] Nasiriany, S., Nasiriany, S., Maddukuri, A., and Zhu, Y. RoboCasa365 : A large-scale simulation framework for training and benchmarking generalist robots. In International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=tQJYKwc3n4 on Pith
show 19 more
- [21] Niu, J., Gu, K., Zhao, Y., Liang, S., Wang, T., Hu, X., Wang, Y., and Li, H. Realtime-VLA FLASH : Speculative inference framework for diffusion-based VLA s. In Advances in Neural Information Processing Systems, 2026. URL https://arxiv.org/abs/2605.13778. To appear on PithPith's reviewarXiv
- [22] NVIDIA . GR00T N1.7 LIBERO : Model card and checkpoints. Hugging Face model repository, 2026 a . URL https://huggingface.co/nvidia/GR00T-N1.7-LIBERO. Evaluated revision: 2ea293aa20ba7cf5bbf3ba17a5fbcb1a01cbfe21. LIBERO results: Isaac-GR00T examples/LIBERO/README.md, commit 1a1... on Pith
- [23] NVIDIA . GR00T N1.7 SimplerEnv evaluation. GitHub repository, 2026 b . URL https://github.com/NVIDIA/Isaac-GR00T/blob/1a1837f20538b7d7e21f977a11a5aee14f99803c/examples/SimplerEnv/README.md. examples/SimplerEnv/README.md, commit 1a1837f20538b7d7e21f977a11a5aee14f99803c on Pith
- [24] NVIDIA . Isaac Sim . Robotics simulation software and documentation, n.d. URL https://docs.isaacsim.omniverse.nvidia.com/latest/index.html. Accessed October 19, 2026 on Pith
- [25] NVIDIA , Bjorck, J., Casta \ n eda, F., Cherniadev, N., Da, X., Ding, R., Fan, L. J., Fang, Y., Fox, D., Hu, F., Huang, S., Jang, J., Jiang, Z., Kautz, J., Kundalia, K., Lao, L., Li, Z., Lin, Z., Lin, K., Liu, G., Llontop, E., Magne, L., Mandlekar, A., Narayan, A., Nasiriany, ... on PithPith's reviewarXiv
- [26] Physical Intelligence . openpi: LIBERO benchmark example and results. GitHub repository, 2025. URL https://github.com/Physical-Intelligence/openpi/blob/2d70d966582e711128ad8358d8dbf23d2cc3d658/examples/libero/README.md. examples/libero/README.md, commit 2d70d966582e711128ad835... on Pith
- [27] Shukor, M., Aubakirova, D., Capuano, F., Kooijmans, P., Palma, S., Zouitine, A., Aractingi, M., Pascal, C., Russi, M., Marafioti, A., Alibert, S., Cord, M., Wolf, T., and Cadene, R. SmolVLA : A vision-language-action model for affordable and efficient robotics. arXiv preprint ... on PithPith's reviewarXiv
- [28] Su, L. Execution-state capsules: Graph-bound execution-state checkpoint and restore for low-latency, small-batch, on-device physical- AI serving. arXiv preprint arXiv:2606.20537, 2026. doi:10.48550/arXiv.2606.20537. URL https://arxiv.org/abs/2606.20537 on PithPith's reviewarXivdoi:10.48550/arxiv.2606.20537
- [29] Tang, J., Sun, Y., Zhao, Y., Yang, S., Lin, Y., Zhang, Z., Hou, J., Lu, Y., Liu, Z., and Han, S. VLASH : Real-time VLAs via future-state-aware asynchronous inference. In 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2026. URL https://arxiv.or... on PithPith's reviewarXiv
- [30] TheRobotStudio . SO-ARM100 : Standard open arm 100 ( SO-100 and SO-101 ). GitHub repository, 2025. URL https://github.com/TheRobotStudio/SO-ARM100 on Pith
- [31] Todorov, E., Erez, T., and Tassa, Y. MuJoCo : A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp.\ 5026--5033. IEEE, 2012. doi:10.1109/IROS.2012.6386109. URL https://homes.cs.washington.edu/ todorov/papers/... on Pithdoi:10.1109/iros.2012.6386109
- [32] Xie, H., Wen, B., Zheng, J., Chen, Z., Hong, F., Diao, H., and Liu, Z. DynamicVLA : A vision-language-action model for dynamic object manipulation. In Advances in Neural Information Processing Systems, 2026. URL https://arxiv.org/abs/2601.22153. To appear on PithPith's reviewarXiv
- [33] Xu, B., Tan, H., Jiang, W., He, B., Wu, M., Zeng, J., and Heinrich, M. VLA-HWBench : Where the time goes when VLA models run on edge hardware. arXiv preprint arXiv:2610.34717, 2026. URL https://arxiv.org/abs/2610.34717 on PithPith's reviewarXiv
- [34] Yang, C., Hu, Y., Ma, Y., Yang, Y., Tan, J., and Fan, H. Realtime-VLA V2 : Learning to run VLA s fast, smooth, and accurate. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.\ 4501--4509, 2026 a . URL https://openaccess.... on Pith
- [35] Yang, X., Dagli, R., Zook, A., Hadfield, H., Goyal, A., Birchfield, S., Ramos, F., and Tremblay, J. RoboLab : A high-fidelity simulation benchmark for analysis of task generalist policies. In Proceedings of Robotics: Science and Systems, 2026 b . doi:10.15607/RSS.2026.XXII.096... on Pithdoi:10.15607/rss.2026.xxii.096
- [36] Yu, T., Quillen, D., He, Z., Julian, R., Hausman, K., Finn, C., and Levine, S. Meta-World : A benchmark and evaluation for multi-task and meta reinforcement learning. In Proceedings of the Conference on Robot Learning, volume 100 of Proceedings of Machine Learning Research, pp... on Pith
- [37] Zhang, Y., Song, C., Singh, V., Li, X., Ye, K., Hu, Z., Pu, Z., Yin, Y., and Chaudhary, V. Overcoming dynamics-blindness: Training-free pace-and-path correction for VLA models. arXiv preprint arXiv:2605.11459, 2026. URL https://arxiv.org/html/2605.11459v1 on PithPith's reviewarXiv
- [38] Zhao, T. Z., Kumar, V., Levine, S., and Finn, C. Learning fine-grained bimanual manipulation with low-cost hardware. In Proceedings of Robotics: Science and Systems, 2023. doi:10.15607/RSS.2023.XIX.016. URL https://roboticsproceedings.org/rss19/p016.html on Pithdoi:10.15607/rss.2023.xix.016
- [39] Zhou, X., Xu, Y., Tie, G., Chen, Y., Zhang, G., Chu, D., Zhou, P., and Sun, L. LIBERO-PRO : Towards robust and fair evaluation of vision-language-action models beyond memorization. arXiv preprint arXiv:2510.03827, 2025. URL https://arxiv.org/abs/2510.03827 on PithPith's reviewarXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment