Pith.

Robotics · arXiv:2610.34717 v2 · submitted 26 Oct 2026 · 15 references

VLA-HWBench: Where the Time Goes When VLA Models Run on Edge Hardware

Borui Xu, Hongshi Tan, Wenqi Jiang, Bingsheng He, Ming Wu, Jason Zeng, Michael Heinrich

The paper says

“Vision-language-action (VLA) models increasingly run on edge devices aboard robots, where inference latency constrains closed-loop robot control.”

more of the abstract

However, it remains unclear how to design VLA models, their inference software and edge hardware so that robots can run VLA inference in real time. Most VLA models report only task success, while prior studies cover limited action-generation strategies, divide inference into at most three stages, and lack comparisons of inference optimizations on the same models. Therefore, we introduce VLA-HWBench, a benchmark that measures 18 open VLA models on NVIDIA Jetson AGX Thor and AMD Ryzen AI Max+ PRO 395, with a seven-stage inference breakdown. Our study reveals three insights. First, action-generation strategy matters more than model size. It dominates 14 of 19 Jetson configurations, latency varies by 82 × on Jetson and 157 × on AMD, and parameter count explains at most 15% of latency variance. Models that generate long action chunks in a few flow-matching steps keep up with control, whereas token-by-token decoding does not. Second, kernel fusion with 4-bit weights helps most, running π0.5 and GR00T N1.7 2.6-2.9× faster than PyTorch with torch.compile, but it exists for only these two models and only on the Jetson. Optimization also moves the time to the stages it shortens least, as CPU preprocessing grows from 12% to 39% of the GR00T N1.7 call. Third, after optimization, memory bandwidth limits only the stages that process few tokens on the Jetson, such as the denoising steps, whereas software limits all other stages, from unfused kernels to serial preprocessing on the host CPU. Overall, low-latency edge VLAs should use few action-generation iterations, inference software should extend kernel fusion and low-bit weights to more models and platforms, and edge hardware needs high memory bandwidth and a fast host CPU.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 28 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A genuinely useful measurement paper with a robust first two insights, a fragile third insight that rests on unmeasured byte counts, and one missing action-equivalence check on its flashiest number. This line is wrong

2 major objections · 4 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 15 references say.

The paper says, at Section 5.3, Insight 3; Appendix G; Table 15

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

The memory-bound versus software-limited classification of the FlashRT Jetson stages rests on derived, not measured, per-step memory traffic. Appendix G states that F and W are taken from GEMM shapes recorded in an eager run and converted into each optimized format, and that a FlashRT denoising step reads only the weights of its GEMMs because modulation and KV context are cached once per call.

more of objection 1

No profiler measurement of FlashRT's actual DRAM bytes is reported. This is load-bearing: if FlashRT's fused kernels read fewer bytes per step than the converted eager shapes imply, the 1.6x (pi0.5 action), 1.7x (GR00T prefill), and 1.8x (GR00T action) distances to the W/B bound would grow; a 40% reduction in W would put them at roughly 2.7-3.0x above the bound, in the same range as the 2.8-3.3x AMD values that the paper labels software-limited. Since the hardware recommendation (bandwidth and host CPU over peak compute) follows from this distinction, I ask for direct validation of FlashRT's memory traffic, for example with hardware memory-throughput counters, or for an explicitly conditional formulation of Insight 3.

The paper says, at Section 5.1, Insight 1; Tables 8-9

The review did not quote this passage.

Pith says: Objection 2 of 2 · major

As printed, the stage tables do not reproduce the claim that action generation is the largest stage in 14 of the 19 configurations. Under the arrow convention stated in Appendix C, X-VLA shows S3=51.7 ms and S4=0.1 ms, and RIPT-VLA shows S3=223.6 ms and S4=99.5 ms; taken together with VLA-Adapter, SimpleVLA-RL, OpenVLA-OFT, and Evo-1, these rows would make action generation the largest stage in 13 configurations rather than 14.

more of objection 2

If X-VLA's flow-matching action generation is executed inside the call timed as prefill and is therefore counted in S3, the table and the text should state this explicitly. Otherwise the count or the table needs correction before the paper's first central claim can be checked by a reader.

4 minor objections

The paper says, at Tables 8-9

Pith says: minor objection 1 of 4

The arrow notation indicating fused stages is not self-explanatory: a reader cannot tell whether the arrowed source time is included in the target stage's total or listed separately. Please define the notation in the caption and show the resulting column sums explicitly.

The paper says, at Tables 8-9 and Section 4.4

Pith says: minor objection 2 of 4

Several rows contain negative S6 residuals (for example, -3.0 ms and -1.1 ms). Since the paper states that stage sums are checked against end-to-end time, a brief explanation of these negative residuals as timing jitter or rounding would remove a confusing discrepancy.

show 2 more

The paper says, at Figure 4a

Pith says: minor objection 3 of 4

The AMD-platform circles do not show the 5th-95th percentile spreads even though Table 7 reports them; adding whiskers or a note explaining why they are omitted would improve comparability with the Jetson bars.

The paper says, at Table 2 and Figure 1

Pith says: minor objection 4 of 4

The paper says "one point per checkpoint" for the parameter-count regression, but MolmoAct2 contributes two reference configurations; please clarify whether the regression uses 18 or 19 points and whether the two MolmoAct2 points are treated as independent.

What would settle it

On the Jetson, run the fastest FlashRT NVFP4 configurations of π0.5 and GR00T N1.7 under CUPTI and read the actual DRAM bytes transferred per denoising step (ncu as a fallback, mindful of its replay effects). Compare measured per-step bytes against Table 15's W columns (322 MB for π0.5 action, 489 MB for GR00T) and measured per-step kernel time against the 1.92 ms and 3.30 ms used there. Separately, re-sweep the GR00T N1.7 FlashRT step count with more points: Table 10 reports R² = 0.868 for that sweep, far below every other sweep in the paper. If measured per-step bytes are within ±20% of W and the re-swept slope within ±20% of the reported step time, the 1.6-1.8x distance to the memory bound and the 'memory-bound / hardware-limited' label for action generation stand.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 28 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 4 min 39 s · 25,279 tokens read, 32,368 written
  3. Editor wrote the verdict above 38 s · 26,382 tokens read, 2,960 written
  4. Referee major revision: 2 major objections, 4 minor 3 min 20 s · 25,715 tokens read, 25,153 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Short version: this is a careful, useful benchmark paper and it deserves a serious referee. The first two insights hold up; the third one is shakier than the prose lets on.

What is genuinely new: 18 open VLAs, both edge platforms, all four action-generation forms, and a seven-stage breakdown that finally times CPU preprocessing. Prior studies covered at most five models and two forms. The methodology is unusually disciplined: stage sums are checked against end-to-end time, step costs come from swept linear fits with R2 above 0.99 almost everywhere, latency is reported with 5th–95th percentiles, and the paper discloses that compilation changes actions, including a flipped gripper command for pi0.5. That level of transparency earns real credit. Insight 1 — action-generation iterations, not parameter count, set latency — survives reading: the 82x and 157x spreads are measured numbers, and the log-parameter-count fit explains at most 15 percent of variance.

The soft spots, in proportion. Insight 3 is the fragile one. Appendix G states the assumption honestly: F and W come from GEMM shapes recorded in an eager run, converted into each optimized format, and a FlashRT denoising step is assumed to read exactly those W bytes, with modulation and KV context cached once per call. But no DRAM traffic is measured. The stress-test arithmetic lands: a roughly 40 percent overestimate of W moves the Jetson action stages from 1.6–1.8x to about 2.7–3.0x above the memory bound, which erases the boundary Insight 3 draws between memory-bound and software-limited. The time side of that ratio is also the weakest fit in the paper: the GR00T N1.7 FlashRT step sweep has R2 = 0.868, far below the others. The hardware recommendation — prioritize bandwidth and a fast host CPU over peak compute — is plausible but unverified. I would not build a bandwidth-first edge board on it yet.

Second: no benchmark artifacts are released or linked. No code, data, or traces. For a benchmark whose stated purpose is to be extended, that is a real gap. Third: the LeRobot-to-FlashRT swap is the largest single gain in the paper (5.6x over eager, 2.15x over compiled), and it is the one optimization missing from the action-agreement table. The authors clearly know how to do this check; its absence for the flashiest number is a hole.

Who this is for: edge-inference and robot-systems people will get real value, and the benchmark itself is a contribution even if Insight 3 needs tightening. Send it to a serious referee, and push for artifacts, the missing action check, and either a measured byte count or a softened Insight 3.

The strongest objection

Insight 3 and its hardware corollary — after optimization only the few-token stages are memory-bound, so edge hardware should prioritize bandwidth and a fast host CPU over peak compute — rest entirely on the Appendix G / Table 15 accounting: F and W come from GEMM shapes recorded in an eager run, converted into each optimized format, and a FlashRT denoising step is assumed to read exactly those W bytes once per step, with modulation and KV context cached once per call. The per-step byte count is derived, not measured; FlashRT's actual DRAM traffic is never observed. The error direction is not neutral. If FlashRT's fused kernels merge, skip, or sparsify weight blocks, or its NVFP4 path loads fewer bytes than the converted eager shapes imply, then W/B shrinks, the measured step time sits farther above the roofline than the reported 1.6-1.8x, and the 'hardware-limited' label flips toward 'software-limited'. Quantitatively, a ~40% overestimate of W pushes the three Jetson action and prefill stages from 1.6-1.8x to ~2.7-3.0x above the bound, erasing the boundary Insight 3 draws and removing the direct evidence that adding memory bandwidth accelerates the bottleneck stage. This is exactly the direction the protocol cannot see: Appendix G states the once-per-step caching assumption explicitly (credit where due), but neither the eager-shape accounting nor the kernel-order graph attribution would reveal fewer actual bytes. Two stresses compound this: the GR00T N1.7 FlashRT step sweep has R² = 0.868 (Table 10), far below every other sweep in the paper, so the time side of the ratio is itself the least certain fitted quantity; and the graph-stage attribution splits captured CUDA graphs by kernel order and repetition counts, plausible but unverified for FlashRT's fused graphs. The opposite error (more bytes, e.g., re-reading KV per step) would only tighten the memory-bound label; the uncontrolled direction is under-reading of W.

This is a careful measurement paper with genuine internal discipline: stage sums checked against totals, per-step sweeps with R² ≥ 0.986 for every reference configuration, disclosed action changes under compilation (Table 14), frame-to-frame variation reported, and the FlashRT analysis explicitly scoped to two models on one platform. Insights 1 and 2 have multiple independent supports: the same-weights MolmoAct2 head comparison, the iteration sweeps, the R² ≤ 15% parameter regression, and the cosine checks on the precision steps. The load-bearing seam is Insight 3, which is also the piece with the longest evidence chain: F and W derived from eager-run GEMM shapes in optimized formats, an assumption about a third-party runtime's per-step byte reads (KV and modulation cached once per call), and per-step times from sweeps and graph-kernel attribution. The decisive weakness is directional asymmetry: if FlashRT reads more bytes than assumed the memory-bound label tightens; if it reads fewer (fused or sparsified kernels, format conversion effects), the stages move away from the roofline and the label flips — a ~40% byte overestimate is enough to push the 1.6-1.8x stages to ~2.7-3.0x and erase the boundary Insight 3 draws. The protocol cannot detect this because it never measures FlashRT's actual traffic. I agree with the reader's weakest_assumption; my refinement is that the decisive quantity is measured traffic versus derived W, and that the GR00T FlashRT step sweep (R² = 0.868) makes the time side of the ratio the paper's least certain fitted quantity. Verdict UNCHANGED: the concern is genuine, concrete, and cheaply testable, but it is an addressable verification gap rather than a demonstrated error; Insights 1 and 2 stand independently, and even a large byte error leaves the few-token stages the most bandwidth-proximal stages on the Jetson. The paper's explicit disclosure of the assumption (Appendix G) and its honest scoping support keeping the conditional acceptance posture.

The referee's report

If correct, this is a valuable benchmark for the edge-VLA community: it is substantially broader in model coverage and stage granularity than prior studies, and it gives concrete, falsifiable guidance for model, runtime, and hardware co-design. The methodology is unusually careful in several respects: end-to-end latencies are reported with 5th-95th percentile spreads, stage times are checked against end-to-end time, per-step costs come from swept linear fits with mostly very high R2, and the paper discloses action changes under compilation and the existence of fast reasoning modes that change task behavior. The two-platform design also usefully separates host, GPU, and memory effects. The main risk is that the bottleneck classification underlying Insight 3 rests on derived, rather than measured, memory traffic for the FlashRT configurations, and the central stage-count claim in Insight 1 needs to be reconciled with the printed stage tables.

Pith's reading of the paper (a machine's summary, not the authors')

The paper sets out to show where inference time actually goes when vision-language-action (VLA) robot models run on edge hardware, and which levers — model design, inference software, or hardware — actually reduce it. It builds VLA-HWBench, which splits one inference call into seven timed stages and measures 18 open VLA models on two architecturally different edge platforms, an NVIDIA Jetson AGX Thor and an AMD Ryzen AI Max+ PRO 395. The central result is that latency follows the number of sequential iterations a model spends generating actions — denoising steps, decoded tokens, reasoning tokens — rather than its size: latency spans $82\times$ on the Jetson and $157\times$ on AMD, and parameter count explains at most 15% of the variance. The paper also shows that fused-kernel, 4-bit-weight backends give the largest single speedup, 2.6–2.9$\times$ over PyTorch with torch.compile, but exist for only two models on one platform, and that after optimization memory bandwidth limits only the few-token stages such as denoising, while vision encoding, prefill, and CPU preprocessing are software-limited. If these claims hold, edge VLA designers should cut action-generation iterations before shrinking models, inference engines should adopt fused low-bit backends, and edge hardware should buy memory bandwidth and host CPU speed ahead of peak compute.

What carries the argument. The load-bearing machinery is a seven-stage decomposition of one observation-to-action call, $T = \sum_{k=0}^{6} T_k$, whose stages are CPU preprocessing (S0), host-to-device copy (S1), vision encoding (S2), language-model prefill (S3), action generation (S4), postprocessing (S5), and an S6 residual; stages are timed at the boundaries of each release's own code and split further by kernels inside CUDA graphs. The central sub-identity is $T_4 \approx N t$, which says action-generation time equals the number of iterations $N$ — denoising steps for flow-matching heads, one pass for parallel decoders, generated tokens for autoregressive heads, reasoning tokens plus action iterations for reasoning models — times the per-iteration cost $t$. To classify what limits each stage, the paper applies the roofline model, computing each stage's arithmetic intensity $F/W$ against the ridge point $P/B$ and comparing achieved time with the compute bound $F/P$ and the memory bound $W/B$; a stage within a small factor of its bound is hardware-limited and one far from it is software-limited. The two platforms are chosen so that peak compute differs 4.4$\times$ while memory bandwidth differs only 1.1$\times$, which separates host, GPU, and memory effects.

Core claim. Stated in the paper's own terms, the discovery is that a VLA's end-to-end inference latency on edge hardware is governed jointly by its action-generation method, its runtime backend, and the platform, not by its parameter count. Across 19 reference configurations of 18 released models, action generation is the largest of the seven stages in 14 of 19 cases on the Jetson, taking 42–99% of the call, and median latency ranges from 77 ms to 6.3 s on the Jetson ($82\times$) and from 114 ms to 17.9 s on AMD ($157\times$); a log-log fit of latency against loaded parameters explains at most 15% of the variance, and the gap is accounted for by the identity $T_4 \approx N t$, in which $N$ is the number of action-generation iterations. The paper further reports that stacking compilation, a specialized fused backend, and 8-bit or 4-bit weights makes $\pi_{0.5}$ 7.2$\times$ faster and GR00T N1.7 2.6$\times$ faster on the Jetson, with the fused 4-bit backend alone worth 2.6–2.9$\times$ over PyTorch with torch.compile, and that this optimization pushes time into the stages it shortens least, so CPU preprocessing grows from 12% to 39% of the GR00T N1.7 call. A roofline analysis of the fastest configurations then finds the few-token action steps memory-bound on the Jetson, running 1.6–1.8$\times$ above the weight-read bound, while vision encoding, prefill, and host preprocessing sit far from their hardware limits and are therefore software-limited.

The premise it rests on. The memory-bound versus software-bound classification rests on an accounting that assumes FlashRT's kernels read only their GEMM weights once per denoising step, with modulation and key/value context computed once per call; if the real kernels move different byte counts per step, the distance-to-bound figures and the hardware recommendations built on them would shift.

If the paper is right (Pith's inferences, not the authors')

VLA design for edge robots should cut action-generation iterations — fewer denoising steps, parallel decoding, or longer action chunks — before shrinking parameters, because only 10 of 19 Jetson configurations keep up with LIBERO's 20 Hz control rate and the $82$–$157\times$ latency spread tracks iteration counts, not size.

Inference software should port fused-kernel, low-bit-weight backends beyond $\pi_{0.5}$ and GR00T N1.7 and beyond the Jetson, since that single change is worth 2.6–2.9$\times$ over torch.compile, and model code should compile as one graph so the whole call is captured.

Edge hardware should be specified for memory bandwidth and host CPU throughput ahead of peak matrix compute, because after optimization the few-token action stages are memory-bound and CPU preprocessing is the largest untouched stage, at 39% of the GR00T N1.7 call.

Latency engineering must treat host preprocessing and GPU kernels as one system, because every optimization shortens GPU stages unevenly and simply moves the remaining time into the stages it shortens least.

Extension the authors leave implicit: the same recipe — count a model's sequential action-generation iterations and time one iteration — could serve as a cheap pre-deployment screen for any new VLA, because it predicts the latency ordering of the 19 configurations without building new hardware or software.

The paper's roofline tables suggest a portability test it did not run: whether FlashRT-style fused kernels on the AMD platform would close the 2.8–3.3$\times$ gap its action steps show below the memory roof, with Isaac-GR00T's per-step recomputation of keys and values the first suspect.

If the memory-bound diagnosis of denoising steps is right, the next generation of action heads should be designed to shrink weight bytes per step — lower-bit weights, shared modulation — rather than to raise compute throughput, a bias that would show up as speedups scaling with byte reduction.

A testable prediction beyond the paper: on a board with the same compute but double the memory bandwidth, the optimized Jetson action stages should shrink toward the memory bound while vision encoding and prefill stay roughly flat, cleanly separating the hardware and software limits the paper labels.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper · Review history (2 versions)

The paper

Abstract

Vision-language-action (VLA) models increasingly run on edge devices aboard robots, where inference latency constrains closed-loop robot control. However, it remains unclear how to design VLA models, their inference software and edge hardware so that robots can run VLA inference in real time. Most VLA models report only task success, while prior studies cover limited action-generation strategies, divide inference into at most three stages, and lack comparisons of inference optimizations on the same models. Therefore, we introduce VLA-HWBench, a benchmark that measures 18 open VLA models on NVIDIA Jetson AGX Thor and AMD Ryzen AI Max+ PRO 395, with a seven-stage inference breakdown. Our study reveals three insights. First, action-generation strategy matters more than model size. It dominates 14 of 19 Jetson configurations, latency varies by 82 × on Jetson and 157 × on AMD, and parameter count explains at most 15% of latency variance. Models that generate long action chunks in a few flow-matching steps keep up with control, whereas token-by-token decoding does not. Second, kernel fusion with 4-bit weights helps most, running π0.5 and GR00T N1.7 2.6-2.9× faster than PyTorch with torch.compile, but it exists for only these two models and only on the Jetson. Optimization also moves the time to the stages it shortens least, as CPU preprocessing grows from 12% to 39% of the GR00T N1.7 call. Third, after optimization, memory bandwidth limits only the stages that process few tokens on the Jetson, such as the denoising steps, whereas software limits all other stages, from unfused kernels to serial preprocessing on the host CPU. Overall, low-latency edge VLAs should use few action-generation iterations, inference software should extend kernel fusion and low-bit weights to more models and platforms, and edge hardware needs high memory bandwidth and a fast host CPU.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-27

The authors' main claim, as the classifier picked it out 0.99

“Overall, low-latency edge VLAs should use few action-generation iterations, inference software should extend kernel fusion and low-bit weights to more models and platforms, and edge hardware needs high memory bandwidth and a fast host CPU.”

How the main claim is stated: asserted 0.76

What the abstract itself states, as probabilities: claims first 0.17 · states limitation 0.70 · human participants 0.03 · controlled comparison 0.66 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.91 · mentions code or data availability 0.05

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Vision--language--action (VLA) models increasingly run on edge devices aboard robots, where inference latency constrains closed-loop robot control. background 1.00 0.05 0.02 asserted 0.99
However, it remains unclear how to design VLA models, their inference software and edge hardware so that robots can run VLA inference in real time. gap 0.98 0.04 0.02 asserted 0.72
Most VLA models report only task success, while prior studies cover limited action-generation strategies, divide inference into at most three stages, and lack comparisons of inference optimizations on the same models.benchmarking 18 open VLA models across released and alternative backends on NVIDIA Jetson AGX Thor and AMD Ryzen AI Max+ PRO 395, with a seven-stage inference breakdown. contribution 0.66 0.84 0.04 asserted 0.92
Our study reveals three insights. contribution 0.57 0.51 0.05 asserted 0.65
First, action-generation strategy matters more than model size. result 0.89 0.89 0.31 asserted 1.00
It dominates 14 of 19 Jetson configurations, latency varies by $82\times$ on Jetson and $157\times$ on AMD, and parameter count explains at most 15% of latency variance. result 0.99 0.94 0.19 asserted 1.00
Models that generate long action chunks in a few flow-matching steps keep up with control, whereas token-by-token decoding does not. result 0.91 0.89 0.16 asserted 1.00
Second, kernel fusion with 4-bit weights helps most, running $\pi_{0.5}$ and GR00T N1.7 2.6--2.9$\times$ faster than PyTorch with \texttt{torch.compile}, but it exists for only these two models and only on the Jetson. result 1.00 0.93 0.06 asserted 1.00
Optimization also moves the time to the stages it shortens least, as CPU preprocessing grows from 12\% to 39\% of the GR00T N1.7 call. result 0.97 0.94 0.04 asserted 1.00
Third, after optimization, memory bandwidth limits only the stages that process few tokens on the Jetson, such as the denoising steps, whereas software limits all other stages, from unfused kernels to serial preprocessing on the host CPU. result 0.87 0.93 0.10 asserted 1.00
Overall, low-latency edge VLAs should use few action-generation iterations, inference software should extend kernel fusion and low-bit weights to more models and platforms, and edge hardware needs high memory bandwidth and a fast host CPU. interpretation 0.55 0.61 0.86 asserted 0.76

What kind of paper the classifier reports this is: empirical study 0.72 · evidence from observation or experiment 1.00

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-27: 1 request, 8,525 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (7)
Figure 1 of the paper
Figure 1. End-to-end latency versus loaded parameter count on the Jetson AGX Thor (left) and the Ryzen AI Max+ PRO 395 (right), with one point per reference configuration. of a single architecture (Vishwanathan et al., 2026) and analytical performance models (Jiang et al., 2026), but three gaps remain. First, they cover few of the ways in which recent VLAs generate actions. Recent models differ in architecture and training and denoise action chunks by flow matching, decode them in parallel, generate them token by token or reason before acting. These choices change how much sequential work a…
Figure 2 of the paper
Figure 2. The seven stages of one inference call. S0 and S5 run on the CPU and S2–S4 on the GPU
Figure 3 of the paper
Figure 3. Stage breakdown of the 19 reference configurations on the Jetson (a) and the AMD platform (b), with a fused stage counted in the stage that contains it. 4.4 MEASUREMENT PROTOCOL Input and timing. Every configuration processes the same real LIBERO-10 observation (Liu et al., 2023), two 256 × 256 camera images, the robot state and the task instruction, adapted only where a release’s input contract requires it. Each configuration runs in its own process with 20 warm-up calls and up to 100 timed calls, trimmed to fit 90 s but never below 20, each bracketed by device synchronization, and…
Figure 4 of the paper
Figure 4. Latency of the 19 reference configurations. (a) Median latency on the Jetson (bars, 5th–95th percentile) and the AMD platform (circles). (b) Latency over the chunk size H. Where the time goes
Figure 5 of the paper
Figure 5. Optimizations applied in stages to π0.5 and GR00T N1.7. (a, b) Latency of the fastest configuration with the optimizations up to each stage, labeled with the speed-up of the stage. (c, d) Their stage times on the Jetson. Jetson and 1.8× on the AMD platform, which has no TF32. Isaac-GR00T’s recipe compiles only the action head, which speeds that stage up by 1.4× but slows the uncompiled vision encoding and prefill, so the call gains only 1.02× on the Jetson. Compilation also changes π0.5’s actions and can flip a gripper command, as Appendix F details. Backend replacement. FlashRT…
Figure 6 of the paper
Figure 6. What limits the fastest configurations. (a) Stage shares, with the stages inside a captured graph separated by its kernels. (b) CPU preprocessing of GR00T N1.7 and its sub-steps with PyTorch’s 14 CPU threads and with one thread. (c, d) Rooflines, with the peak memory bandwidth as the sloped line, the peak of each number format as a flat line and each GPU stage at its arithmetic intensity F/W and achieved throughput. Prefill length decides its bound. The 574–712 prefix tokens of π0.5 put its prefill near or above the ridge. On the AMD platform it is compute-bound, but it reaches only…
Figure 7 of the paper
Figure 7. Time of every GPU stage of the fastest configurations over the larger of its two bounds,

Figures are the authors', reproduced from the paper.

References (15)
  1. [5] 11 VLA-HWBench Wenqi Jiang, Jason Clemons, Karu Sankaralingam, and Christos Kozyrakis. How fast can I run my VLA? Demystifying VLA inference performance with VLA-Perf.arXiv preprint arXiv:2602.18397, on PitharXiv
  2. [6] Haoxiang Ma, Junhao Cai, Xiaoxu Xu, Hao Li, Yuyin Yang, Yang Tian, Jiafei Cao, Hongrui Zhu, et al. InternVLA-A1.5: Unifying understanding, latent foresight, and action for compositional generalization.arXiv preprint arXiv:2607.04988, on PithPith's reviewarXiv
  3. [10] Manoj Vishwanathan, Suvinay Subramanian, and Anand Raghunathan. Characterizing VLA mod- els: Identifying the action generation bottleneck for edge AI architectures.arXiv preprint arXiv:2603.02271, on PitharXiv
  4. [12] Shuai Yang, Hao Li, Bin Wang, Yilun Chen, Yang Tian, Tai Wang, Hanqing Wang, Feng Zhao, et al. Vision-Language-Action instruction tuning: From understanding to manipulation. InInternational Conference on Learning Representations (ICLR), pp. 152896–152943, 2026a. Zebin Yang, Qi... on PithPith's reviewarXiv
  5. [13] Yuke Zhu, Josiah Wong, Ajay Mandlekar, Roberto Mart ´ın-Mart´ın, Abhishek Joshi, Kevin Lin, Abhiram Maddukuri, Soroush Nasiriany, and Yifeng Zhu. robosuite: A modular simulation framework and benchmark for robot learning.arXiv preprint arXiv:2009.12293, on PithPith's reviewarXiv
  6. [395] Model S0 S1 S2 S3 S4 S5 S6 Total X-VLA 2.8 0.4→51.5 154.3 0.1 0.1 209 VLA-Adapter 5.9 0.3 60.2 29.7 15.8 1.8 0.1 114 GR00T N1.7 2.4 0.9 98.6 49.6 145.3 0.1 0.7 298 SimpleVLA-RL 3.3 0.3 32.7 408.4→1.0 -0.0 446 SmolVLA 0.7→40.8 10.9 87.5 0.1 0.0 140 InstructVLA 3.8→49.5 71.8 224... on Pith
  7. [2009] Ling Xu, Borui Li, Hao Wu, Chuyu Han, Xiangyu Li, Mohan Hua, Shiqi Jiang, Ting Cao, Chuanyou Li, Sheng Zhong, and Shuai Wang. Embodied.cpp: A portable inference runtime of embodied AI models on heterogeneous robots.arXiv preprint arXiv:2607.02501, on PithPith's reviewarXiv
  8. [2018] Khanh D. Nguyen, Hung T. Ho, Chinh T. Nguyen, Thanh Q. Duong, Linh D. Le, Duy M. H. Nguyen, Vien A. Ngo, and An T. Le. vla.cpp: A unified inference runtime for vision-language-action models.arXiv preprint arXiv:2606.08094, on PithPith's reviewarXiv
  9. [2019] Haoquan Fang, Jiafei Duan, Donovan Clay, Sam Wang, Shuo Liu, Weikai Huang, Xiang Fan, Wei- Chuan Tsai, et al. MolmoAct2: Action reasoning models for real-world deployment.arXiv preprint arXiv:2605.02881, on PithPith's reviewarXiv
  10. [2020] 13 VLA-HWBench A ADDITIONALBENCHMARKDETAILS A.1 MODELCONFIGURATIONS Table 4 complements Table 2 with the software, the number of camera views, and the chunk sizeH and the number of actions He that the release’s own LIBERO evaluation executes per call, for every one of the 19 r... on Pith
  11. [2022] Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, et al. SmolVLA: A vision-language-action model for affordable and efficient robotics. arXiv preprint arXiv:2506.01844, on PithPith's reviewarXiv
  12. [2023] Ankit Goyal, Hugo Hadfield, Xuning Yang, Valts Blukis, and Fabio Ramos. VLA-0: Building State-of-the-Art VLAs with zero modification.arXiv preprint arXiv:2510.13054, on PitharXiv
  13. [2024] Johan Bjorck, Fernando Casta ˜neda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, et al. GR00T N1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, on PithPith's reviewarXiv
  14. [2025] Chia-Yu Hung, Navonil Majumder, Haoyuan Deng, Renhang Liu, Yankang Ang, Amir Zadeh, Chuan Li, Dorien Herremans, et al. NORA-1.5: A Vision-Language-Action model trained using world model- and action-based preference rewards.arXiv preprint arXiv:2511.14659, on PitharXiv
  15. [2026] Shuhan Tan, Kairan Dou, Yue Zhao, and Philipp Kr¨ahenb¨uhl. Interactive Post-Training for Vision- Language-Action models.arXiv preprint arXiv:2505.17016, on PithPith's reviewarXiv
Cited by 1 Pith paper

Reviewed papers on Pith that cite this one. The full citation record

  1. VLALagBench: Understanding When and Why VLA Inference Latency Matters cs.RO · Nov 2026

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment