Pith.

Robotics · arXiv:2611.06710 v1 · submitted 4 Nov 2026 · 53 references

Where Is VLA Inference Time Spent? Bottlenecks and Optimization Interactions Across GPUs

Muhammad Danish Waseem, Ahmed Ali-Eldin Hassan

The paper says

“Vision-language-action (VLA) models map camera observations and language instructions to robot actions. However, their large backbones and repeated action generation can make inference take longer than the robot's real-time requirements for its next action.”

more of the abstract

To reduce the load of VLA computations, researchers have proposed multiple optimization approaches, including quantization and pruning. These optimizations can be combined with other general optimizations such as better kernel compilation or using CUDA graphs instead of the eager mode. However, most of these optimization results are usually reported on a single GPU type, typically for a few VLA models, with no analysis of what the real bottlenecks are for a model on a particular hardware or how these optimizations would generalize beyond the reported results. In this paper, we provide a comprehensive measurement study of seven families of inference optimization methods, from CUDA graphs and compilation to token pruning and quantization, on seven state-of-the-art VLAs evaluated on four GPU families ranging from a datacenter H100 to an embedded Jetson Thor. We analyse how CPU launch overhead, memory bandwidth, and kernel efficiency jointly shape optimization gains, and how applying one method changes the benefit of another. From this analysis, we derive VLA-Triage, guidelines for stacking optimizations under latency budgets. The guidelines identify which stages can provide the required savings and which combinations need testing in the deployed execution mode.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 5 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A careful, honest measurement study whose real contribution is the negative result — separately measured optimization savings neither compose nor transfer — undercut mainly by shipping no code, data, or raw traces. This line is wrong

3 major objections · 8 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 53 references say.

The paper says, at Section 1 (Introduction), versus Section 5.3, Appendix D and Table 7

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.88):

“Host CPUs and other software com- ponents still differ, so these comparisons do not isolate GPU architecture.”

Pith says: Objection 1 of 3 · major

The two headline examples in Section 1 state the claim in hardware terms: 'compiling GR00T with CUDA graphs makes it faster on the H100 and slower on the Thor, and storing CogACT's weights in 8 bits makes it faster on the A10G and slower on the H100.'

more of objection 1

The abstract and the bullet 'includes cases where a method helps on one platform and hurts on another' carry the same framing. The body, however, localizes the GR00T-on-Thor slowdown to a compiler outcome: Appendix D reports that kernels identified as copies or data-type conversions take 70.4 ms eager versus 136.8 ms compiled, with 93.9 ms of the compiled figure in compiler-generated Triton kernels, and Table 7 shows Thor on JetPack R39 with PyTorch 2.10 throughout. Table 11 rules out a power-state or launch artifact but not a codegen artifact, and no experiment varies the compiler stack at fixed hardware. Section 5.3 does state the caveat ('Host CPUs and other software components still differ, so these comparisons do not isolate GPU architecture'), and Appendix B notes that CUDA-graph latencies under the two software stacks agree within 4 ms, which is relevant evidence that deserves to be brought into the main text. As written, a reader of the introduction will attribute a toolchain effect to device architecture. Please either restate the headline claim in platform terms throughout (which the deployment-oriented framing of Section 6 already supports), or add one control at fixed hardware that varies the compiled-path configuration for GR00T on Thor (for example reduce-overhead versus max-autotune, or disabling the fusion that produces the copy/convert kernels), and report whether the sign of the effect survives. Table 13 is the right model for how such a matched-software control should be presented.

The paper says, at Section 1, with Table 8 and Table 3 (lower block)

The review did not quote this passage.

Pith says: Objection 2 of 3 · major

The CogACT quantization example is the second of the two headline sign-flip claims and should be tied explicitly to a table entry. In the copy of Table 8 available to me the listed rows cover OpenVLA-OFT and OpenVLA, and the only CogACT quantization rows I can match are the lower block of Table 3, where CogACT on H100 uses FP8 static and CogACT on A10G uses int8 compute.

more of objection 2

Those are different schemes with different baselines, so a comparison between them cannot by itself support the sentence 'storing CogACT's weights in 8 bits makes it faster on the A10G and slower on the H100.' Section 3.1 also states that the fastest available library implementation is selected per platform (Table 9 shows Machete at 0.79x prefill on H100 versus Gemlite at 2.13x decode on A10G), so a scheme held fixed can still be instantiated as different kernels on different devices. Please point the sentence to the specific table row and state whether the compared scheme is identical on both platforms; if it is not, reword the example to say that the same quantization scheme is instantiated by different kernel libraries and that the sign of its whole-inference effect therefore differs. The scheme-matched, matched-software CogACT pruning comparison in Table 13 already demonstrates the pattern the sentence wants to convey, and could carry it.

The paper says, at Section 6, step 4 and Table 17, with Section 4.2 and Table 5

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

VLA-Triage's pruning recommendations rest on latencies measured with random token selection. Section 4.2 says so ('We use random token selection to measure the latency effect of token count'), and Appendix H confirms that pruning in Table 17 uses random selection.

more of objection 3

Deployed selectors are not free: Table 5 shows FastV with unfused attention takes 33.2 ms on H100 against 30.6 ms for random selection with fused attention from a 41.8 ms unpruned fused baseline, which is the gap the Section 1 example (21.4% versus 6.9%) is about. Section 5.1 Case 3 partially accounts for this, reporting that after updating the affected-stage share the measured latency is still up to about 1.385 times the implied value in selector-retaining comparisons, and Section 5.3 repeats that selectors need measurement. But step 4's instruction to pick 'initial levels whose implied latency, T1T2/T0, is at most (1+delta)B_model' and the budget tables in Table 17 contain no term for the selector's own cost, and a reader who follows the guideline with FastV will land above the budget in the cases where the selector forces an unfused backend. Please state explicitly in step 4 that recorded pruning levels are latency-equivalent at a fixed retained token count and must be re-measured with the deployed selector and attention backend, or add the measured selector overhead as a term in the screen.

8 minor objections

The paper says, at Section 5.1, flow-model paragraph

Pith says: minor objection 1 of 8

The 'up to 227%' flow-model discrepancy is reported without the fact, stated in Appendix G, that all 32 flow combinations meet all three tested budgets, so their discrepancies change none of the 96 budget decisions. The magnitude is real and supports the non-composability claim, but the paragraph should tell the reader that the practical consequence in this grid is ranking rather than feasibility.

more of objection 1

Relatedly, the count of 21 comparisons with 'zero or negative latency' under millisecond reuse is a degenerate case in which the assumption predicts a physically impossible value; report those separately from the comparisons that exceed the 5.3% cutoff so the two kinds of evidence are not conflated.

The paper says, at Section 4, opening paragraph of Section 4

Pith says: minor objection 2 of 8

The sentence 'each stage of inference is limited by either the GPU compute power, its memory bandwidth, or the CPU that launches the GPU kernels' is stated more strongly than the evidence the paper itself presents. Section 4 immediately qualifies it ('position in Figure 3 alone does not establish the bottleneck'), and Appendix C reports that the attribution error against measured whole-inference saving has a median of 0.4 and a maximum of 4.1 percentage points.

more of objection 2

Please hedge the sentence to match, for example by saying which bottleneck dominates the reference cost rather than which one limits the stage.

show 6 more

The paper says, at Section 4.2, with Table 5

Pith says: minor objection 3 of 8

Table 5 contains a row that materially changes the lesson of the FastV example and is not discussed in the main text: 'FastV scored by a fused kernel' gives 30.6 ms on H100, 136.9 ms on A10G, 113.5 ms on Thor and 68.5 ms on RTX 4090, essentially matching random selection with fused attention.

more of objection 3

The 21.4% versus 6.9% contrast therefore reflects an implementation choice inside the selector as much as an inherent property of attention-based selection. A sentence in Section 4.2 would let readers take the practical conclusion ('keep the selector on the fused path') as well as the cautionary one.

The paper says, at Section 3.1 and Appendix B, kernel selection rule

Pith says: minor objection 4 of 8

The rule 'we time the available library implementations and use the fastest one whose output error stays within 1.25 times the smallest error among them' introduces a free parameter that the paper does not motivate. Please state where 1.25 comes from, how sensitive the selected implementation is to it, and how often the fastest kernel was also the most accurate. Table 9 shows several selections where no implementation beats bf16, which makes the criterion's consequences visible and worth a sentence.

The paper says, at Section 6, step 4, and Appendix H

Pith says: minor objection 5 of 8

The delta = 5.3% allowance is validated on the same 81 recorded cases that motivated it, and Appendix H says as much ('a heuristic evaluated on this grid, not a bound for unseen settings'). The main text should carry that qualifier rather than leaving it in the appendix, since step 4 is written as a procedure for new deployments.

The paper says, at Figure 1 and its caption

Pith says: minor objection 6 of 8

The caption says 'Points are measured latencies and dashed lines the budgets B of Equation (1)', with the vertical axis described as the resulting slowdown 1/(1-q_idle). It is not stated whether the mapped slowdown values come from the scheduler replay of Appendix A or are computed analytically from the measured latencies, nor how a point beyond budget maps to a specific vertical position. Please make the mapping explicit in the caption.

The paper says, at Table 1 and Table 8 headers

Pith says: minor objection 7 of 8

Table 1's header renders the chunk fields as 'vis. tok.A A r B(ms) kernelsT(ms)', and Table 8's header collapses the model and quantization columns into one line, making both tables hard to read in print. The same applies to the missing column separation in Table 17's 'prune quantisation pair' fields.

The paper says, at Throughout

Pith says: minor objection 8 of 8

There are a number of typographical errors that should be cleaned up: 'we perfrom' (Section 5.1), 'our study of 25 published paper' (Section 1), 'Engieering' and 'Chalmer University' in the affiliation footnote, a sentence beginning with lowercase 'quantization should be evaluated at the retained token count' in Section 5.1, and the recurring spacing of 'LLaV A-VLA'. Section 6 step 4 also renders 'A pruninglevelis'.

What would settle it

Re-run GR00T compiled on Thor in two compiler environments at otherwise identical settings: (a) the shipped JetPack R39 / PyTorch 2.10 stack used in Appendix D, and (b) a second torch build, or the same build with the Triton copy/convert path avoided (for example max-autotune instead of reduce-overhead, or disabling the fusion that emits the 93.9 ms of copy and conversion kernels). Also run GR00T compiled on H100 under the Thor's torch version. Report eager versus compiled latency for each cell. If compiled Thor drops below eager in (b), or if the H100/Thor ordering flips with the torch version alone, the sign flip is a toolchain effect and the abstract's 'faster on one GPU, slower on another' should be restated as a toolchain-version effect. If compiled Thor stays slower than eager across both environments and both compiler modes, the hardware attribution stands.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 5 Nov 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic partly agrees 5 min 50 s · 34,339 tokens read, 12,497 written
  3. Editor wrote the verdict above 16 s · 35,444 tokens read, 1,606 written
  4. Referee minor revision: 3 major objections, 8 minor 1 min 42 s · 34,667 tokens read, 10,814 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Short version: the useful part of this paper is not the roofline, it's the demonstration that optimization savings measured one at a time neither stack nor carry across platforms, and the 714 four-latency comparisons behind that claim look real.

What's new and earned. The reuse census is the contribution: for each method pair they measure T0, T1, T2, T12 and check whether the implied latency matches. That is the right experiment and almost nobody runs it. The 29-of-32 flow-model result, where percentage reuse overestimates the gain in most cases and millisecond reuse produces zero or negative latencies in 21 of them, is the kind of thing that quietly invalidates a lot of deployment arithmetic. The budget-flip examples (CogACT on A10G at 46.4 ms measured versus a 48.5 ms estimate) are concrete and checkable. The FastV-versus-FlashAttention-2 point is the cheapest and most immediately useful thing in the paper: 21.4% against unfused attention, 6.9% against FA-2, because the selector forces the slow kernel. Anyone citing a pruning speedup should read Table 5. The literature audit finding — one of 25 papers states its execution mode — is a fair hit and the authors correctly disclaim it as non-systematic (b592).

Soft spots. The big one: for a paper whose product is a measurement protocol, no code, no traces, no raw numbers beyond tables. The reader is right to condition on this. Second, the cross-platform attribution. I think the stress-test overstates this. Appendix D localizes GR00T-on-Thor to compiler-generated Triton copy/convert kernels (93.9 ms), and the paper still reports the sign flip — correctly, because the claim is scoped to "GPU-host platform" (b273). That's an honest deployment finding, not a misattribution. What does deserve a flag is the CogACT 8-bit example in the intro (b34): Table 8 only tabulates OpenVLA and OpenVLA-OFT, so the headline "faster on A10G, slower on H100" has no supporting table in the body. Third, minor: the 5.3% threshold and the δ allowance are heuristics validated on the same grid that motivated them, which the authors state themselves (b653, b688). Fine as a heuristic; don't let it read as a bound.

Who it's for. Systems and deployment people deciding what to run on a Jetson versus an H100. Practitioners get real value; roofline researchers less so.

Recommendation: send it out, but make artifact release a condition of acceptance rather than a suggestion. With the code and traces, this becomes a reference; without them it's an assertion.

The strongest objection

The cross-platform claim needs one thing to hold: that a given optimization is instantiated comparably on each platform, so measured differences track hardware. Two of the marquee examples do not satisfy that, and the paper does not test whether they would survive a toolchain change.

(i) GR00T compiled on Thor. Appendix D localizes the slowdown in the paper's own traces: kernels identified as copies or data-type conversions take 136.8 ms per inference compiled versus 70.4 ms eager, and 93.9 ms of the compiled figure sits in compiler-generated Triton kernels. That is a torch.compile codegen outcome for the Thor backend on JetPack R39 / PyTorch 2.10 (Table 7), not a demonstrated property of the device. The authors show the slowdown persists under pruning and in every power mode (Table 11), which rules out a launch or power-state artifact but not a codegen artifact; no experiment varies the compiler stack at fixed hardware.

(ii) Quantization. Table 9 deliberately selects the fastest available kernel per platform, so 'int8 weights' is Machete on H100 (0.79x prefill, i.e. slower than bf16) and Gemlite on A10G (2.13x decode). Table 8 additionally compares different schemes across platforms, since FP8 tensor cores are absent on A10G and NVFP4 exists only on Thor. A sign change between platforms can therefore reflect kernel-library availability rather than a hardware property.

The scope note (b273) concedes that PyTorch, CUDA and torchao versions differ, and the matched-version control in Section 5.3 (0 of 14 work-removing, 3 of 13 kernel-replacing sign flips on H100 versus A10G) is genuine evidence that some flips survive version matching. But it does not cover Thor, does not match torchao or the selected kernel libraries, and does not cover the CogACT 8-bit example, whose per-platform numbers do not appear in Table 8 (only OpenVLA and OpenVLA-OFT are tabulated there).

The paper is a careful empirical census and I found no internal inconsistency while checking it. I verified that Equation (1)'s budgets reproduce Table 1 exactly for all seven models (AR = floor((A-1)/2) gives 3, 2, 4, 7, 24 and the single-action 50 ms cases), and that the comparison counts in Table 14 sum to the stated 714 (134 + 156 + 378 + 14 + 32). The interaction claim in Section 5 rests on 714 four-latency comparisons with a disclosed timing protocol, a stated profiler cross-check, and same-mode baselines, and the paper is unusually candid about the limits of its own instruments (b152, b154, b464, b688). The composition findings (Cases 1-3) are the load-bearing part of the contribution and they look well supported; the A10G CogACT pruning-under-compilation numbers in Table 13 are internally consistent with the story told in the body.

The soft spot is narrower than the reader's framing. The reader treats the risk as version confounding across PyTorch/CUDA/torchao. The sharper issue is that the treatment itself is instantiated differently per platform: Table 9 picks a different kernel library for the same nominal scheme on each device, and Appendix D shows the GR00T-on-Thor slowdown comes from compiler-generated Triton copy/convert kernels rather than from the device. Neither point is acknowledged, even though b273 acknowledges the version contrast. Against that, the paper does report 3 of 13 kernel-replacing comparisons flipping sign even with matched PyTorch versions on H100 and A10G, which is honest evidence that not all flips are version artifacts, and the guideline 'measure on the physical target' survives regardless of how the attribution resolves.

I mark agreement partial because the reader identified the right family of concern but framed it as version confounding, whereas the operative issue is per-platform kernel instantiation plus the paper's own localization of the Thor slowdown to codegen. I leave the verdict UNCHANGED: this concern lands in the same place as the reader's weakest assumption, which the CONDITIONAL already prices in, and it does not threaten the composition results, the measurement protocol, or the practical guidance.

The referee's report

If the results hold, the contribution is a well-instrumented set of conditional and negative results plus a usable procedure, and the paper earns credit for several things a referee should name. The measurements are direct rather than derived, so there is no fitting-to-prediction circularity in the central claims; the only self-referential step is the delta = 5.3% allowance, which the authors disclose as a heuristic. The protocol is unusually explicit, with pinned CPU cores, warmups, medians over 200+ inferences, reported repeatability, and an independent Nsight cross-check of CUPTI kernel-busy time (at most 0.12% on A10G and 2.1% under graphs and compilation on H100, Appendix B). The Thor power-mode sweep (Table 11), the p99 recheck that changes no decision (Appendix H), the matched-PyTorch comparison on H100 and A10G (Table 13), and the recomputation of every interaction example over all baseline/optimized round pairings (Appendix B) are all falsification attempts in the right direction. The audit of 25 VLA acceleration papers (Table 12) is a genuinely useful side contribution, reported with appropriate scope limits.

Pith's reading of the paper (a machine's summary, not the authors')

The paper asks a practical question for anyone deploying a vision-language-action model on a robot: when you speed up inference, where does the time actually go, and does a saving measured on one machine survive on another? The authors sweep seven families of optimization — CUDA graphs, compilation, fused attention, batching, fewer denoising steps, visual token pruning, and quantization — over seven VLA models with four different action-head designs, on four GPU-host systems ranging from a datacenter H100 down to an embedded Jetson Thor. The answer is that gains are not portable: the same method can speed a model up on one platform and slow it down on another, and a method applied second acts only on the time left over by the first, so separately measured savings do not compose. The failure is concrete enough to change deployment decisions — a budget that looks met when you multiply percentage savings turns out to be missed in several measured combinations. The paper turns this into VLA-Triage, a five-step workflow that selects an execution mode on the target, screens candidates with lower bounds before measuring, and requires that stacked optimizations be measured in the mode they will run in.

What carries the argument. Two pieces of machinery carry the argument. The first is a stage-share accounting identity: if an optimization touches only a fraction $\sigma$ of baseline latency and speeds that stage up by a factor $s$, the overall saving is $g = \sigma(1 - 1/s)$. Every interaction result is read through this identity — an earlier optimization changes either $\sigma$ (compilation shortens the action loop, leaving fewer denoising steps less time to remove) or $s$ (pruning changes matrix sizes and therefore how much quantized kernels gain), or both. The second is a four-latency reuse test that compares measured $T_{12}$ against the two naive composition rules and classifies a rule as optimistic when it predicts a latency below what is measured. Around these sit the budget model $B = A_r \Delta$ for asynchronous chunked policies, a roofline reference $L_j = \sum_o \max(F_o/C, W_o/\mathrm{BW})$ that locates each stage relative to a platform's compute and bandwidth, and a weight-read lower bound $L_w$ that screens out deployment candidates whose mandatory weight traffic alone cannot fit the budget.

Core claim. The central claim is that a VLA optimization's benefit is a property of the model, the GPU-host platform, and the set of methods already applied — not of the optimization method alone. Gains differ across GPUs "in magnitude, in sign and in rank": compiling GR00T's denoising loop makes it faster on the H100 and slower on the Thor, and storing CogACT's weights in 8 bits makes it faster on the A10G and slower on the H100. The same effect appears in the choice of baseline: FastV at 50% pruning saves 21.4% of OpenVLA-OFT's inference time on an H100 when measured against unfused attention, but only 6.9% against FlashAttention-2, because the token selector needs attention weights and therefore forces the slower attention kernel. The authors then test composition directly, comparing four latencies per setting — baseline $T_0$, each method alone $T_1$ and $T_2$, and both together $T_{12}$ — against what percentage reuse ($T_1 T_2/T_0$) and millisecond reuse ($T_1 - (T_0 - T_2)$) would predict. For flow models, 29 of 32 compilation-plus-fewer-steps comparisons come out worse than the percentage assumption implies, by up to 227%; among 194 compiled pruning-then-quantization configurations, 52 are worse by more than the paper's 5.3% remeasurement threshold, the largest by 49%. Because these gaps straddle real deadlines — CogACT with 75% pruning on an A10G is predicted at 48.5 ms against a 50 ms budget and measures 59.3 ms — the paper argues that composing published savings can flip a budget decision in both directions.

The premise it rests on. The claim that a method helps on one GPU and hurts on another rests on cross-platform comparisons in which the software stacks also differ, so the sign flips are attributed to the GPU-host platform rather than to a specific hardware architecture.

If the paper is right (Pith's inferences, not the authors')

Published VLA speed-up percentages should not be reused to predict a deployed combination; pruning and quantization savings must be measured at the retained token count and in the chosen execution mode.

Because a token-pruning method that needs attention weights forces an unfused attention kernel, its real saving is measured against the fused baseline a production stack already uses, roughly a third of the advertised figure in the measured case.

A candidate can be screened before benchmarking: if the mandatory vision-language-model weight reads at 4 bits already exceed the deployment budget, no pruning level or quantization scheme can rescue that model on that platform.

Execution modes carry resource costs beyond latency — each captured CUDA graph reserves a memory pool sized like eager activation memory, so keeping four pruning-level graphs at batch eight exceeds 24 GB on an OpenVLA-OFT or LLaVA-VLA deployment.

The audit of 25 VLA acceleration papers finds that almost none report the execution mode of baseline and method, so readers currently cannot tell whether a reported saving applies to their deployment.

The four-latency reuse test is a cheap general instrument: it could be applied to any accelerator claim (speculative decoding, KV-cache reuse, mixture-of-experts routing) by measuring the pair and comparing against both composition rules, rather than being tied to VLA models.

If the sign flips are genuinely architectural rather than toolchain artifacts, embedded robot platforms would need per-device optimization ports rather than a single tuned recipe — an expensive proposition that argues for the screening bounds in the workflow over brute-force tuning.

The paper's own screening bounds are optimistic in a checkable way: the weight-read bound assumes each weight is read from DRAM once per pass, so a deployment that caches weights across passes would violate the premise of the exclusion, which the authors flag as a condition to verify before using the bound.

The stated 5.3% discrepancy threshold is a remeasurement noise figure rather than a calibrated statistical bound, so future work could replace it with a per-platform variance estimate and turn the qualitative interaction cases into a quantified error budget.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Vision-language-action (VLA) models map camera observations and language instructions to robot actions. However, their large backbones and repeated action generation can make inference take longer than the robot's real-time requirements for its next action. To reduce the load of VLA computations, researchers have proposed multiple optimization approaches, including quantization and pruning. These optimizations can be combined with other general optimizations such as better kernel compilation or using CUDA graphs instead of the eager mode. However, most of these optimization results are usually reported on a single GPU type, typically for a few VLA models, with no analysis of what the real bottlenecks are for a model on a particular hardware or how these optimizations would generalize beyond the reported results. In this paper, we provide a comprehensive measurement study of seven families of inference optimization methods, from CUDA graphs and compilation to token pruning and quantization, on seven state-of-the-art VLAs evaluated on four GPU families ranging from a datacenter H100 to an embedded Jetson Thor. We analyse how CPU launch overhead, memory bandwidth, and kernel efficiency jointly shape optimization gains, and how applying one method changes the benefit of another. From this analysis, we derive VLA-Triage, guidelines for stacking optimizations under latency budgets. The guidelines identify which stages can provide the required savings and which combinations need testing in the deployed execution mode.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-05

The authors' main claim, as the classifier picked it out 0.94

“From this analysis, we derive VLA-Triage, guidelines for stacking optimizations under latency budgets.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.10 · states limitation 0.22 · human participants 0.03 · controlled comparison 0.26 · randomized assignment 0.01 · states quantitative result 0.50 · compares with prior results 0.44 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Vision-language-action (VLA) models map camera observations and language instructions to robot actions. background 1.00 0.04 0.01 asserted 1.00
However, their large backbones and repeated action generation can make inference take longer than the robot's real-time requirements for its next action. background 1.00 0.06 0.02 hedged possibility 0.83
To reduce the load of VLA computations, researchers have proposed multiple optimization approaches, including quantization and pruning. background 1.00 0.03 0.02 asserted 0.99
These optimizations can be combined with other general optimizations such as better kernel compilation or using CUDA graphs instead of the eager mode. background 1.00 0.05 0.02 asserted 0.60
However, most of these optimization results are usually reported on a single GPU type, typically for a few VLA models, with no analysis of what the real bottlenecks are for a model on a particular hardware or how these optimizations would generalize beyond the reported results. gap 1.00 0.05 0.03 asserted 0.97
In this paper, we provide a comprehensive measurement study of seven families of inference optimization methods, from CUDA graphs and compilation to token pruning and quantization, on seven state-of-the-art VLAs evaluated on four GPU families ranging from a datacenter H100 to an embedded Jetson Thor. contribution 0.92 0.97 0.06 asserted 1.00
We analyse how CPU launch overhead, memory bandwidth, and kernel efficiency jointly shape optimization gains, and how applying one method changes the benefit of another. method 0.56 0.92 0.08 asserted 0.99
From this analysis, we derive VLA-Triage, guidelines for stacking optimizations under latency budgets. contribution 0.77 0.96 0.66 asserted 1.00
The guidelines identify which stages can provide the required savings and which combinations need testing in the deployed execution mode. contribution 0.59 0.92 0.26 asserted 1.00

What kind of paper the classifier reports this is: empirical study 1.00 · evidence from observation or experiment 1.00

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-05: 1 request, 6,956 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (14)
Figure 1 of the paper
Figure 1. Inference latency against the robot’s budget. Replay of asynchronous action queue. Points are measured latencies and dashed lines the budgets B of Equation (1). Beyond B the queue finishes before the next action chunk arrives; the vertical axis is the resulting slowdown. method’s benefit may not extend beyond the platform it is tested on. In addition, optimization methods can be combined, or stacked. Hence, running multiple optimizations in sequence means that later VLA optimizations act on the inference stages that are left by previous optimizations. Hence, the total gains from a
Figure 2 of the paper
Figure 2. Where the optimizations act in VLA inference. Pruning reduces tokens only in LLM Prefill. VLA quantization methods usually quantise the vision encoder and language model and keep a separate action head in full precision, because it is most sensitive to quantization error (Xu et al., 2026b; Akbari et al., 2026). The hatched band marks autoregressive decoding, which runs the lan￾guage model. inference when Ar actions remain in the robot’s execution queue. For comparison, in our experiments, we request the next chunk when roughly half the current chunk remains: Ar = ⌊(A − 1)/2⌋ for…
Figure 3 of the paper
Figure 3. Stage throughput against each platform’s roofline reference, unpruned bf16 at batch one. Colour denotes the model and shape denotes the stage. Arithmetic intensity uses the operator traffic of Equation (2). H100 A10G Thor RTX 4090 1.0 1.2 1.4 speed-up over CUDA graphs 1.9 1.9 2.2 2.3 2.0 1.9 2.7 2.8 2.1 2.1 2.2 2.2 1.9 1.9 2.7 2.8 (a) compilation beyond graphs OFT CogACT OpenVLA LLaVA-VLA without graphs OFT CogACT OpenVLA LLaVA 10 −1 10 0 10 1 10 2 GiB above the weights (b) memory above the weights, batch 1 and 8 eager compiled CUDA graphs graphs, 4 pruning levels batch 8
Figure 4 of the paper
Figure 4. Latency benefits and memory costs of accuracy-neutral methods. (a) Speed-up of reduce-overhead compilation over CUDA graphs alone for the four Llama-based models. The RTX 4090 shows compilation without graphs. Bar labels give the eager kernel count divided by the compiled count. (b) GPU memory beyond the model weights at batch sizes one and eight, medians over the GPUs where the configuration fits. The CUDA-graphs bars hold one graph. The four-level bars hold one graph per pruning level, each with its own memory pool, and at batch eight OpenVLA-OFT and LLaVA-VLA then exceed the 24GB…
Figure 5 of the paper
Figure 5. Testing whether a separately measured percentage saving carries over to a combination. Each point compares one measured combination T12 with the latency implied by the separate measurements, T1T2/T0, for one model, platform and setting, such as a pruning level, quantization scheme or step count. Zero means the percentage saving is unchanged. Positive values mean less saving than assumed, and negative values mean more. Appendix Tables 14–15 give the counts and millisecond-reuse results. saves less time. GR00T on Thor shows the reverse: compi￾lation lengthens its steps (Section 4.2).…
Figure 6 of the paper
Figure 6. Measured model latency relative to the nominal budget, on a logarithmic axis. Circles show eager execution and squares the selected execution mode. Orange diamonds show one further pruning or quantization setting from Appendix
Figure 7 of the paper
Figure 7. Eager inference by stage across models and platforms. Stacked bars show stage shares; labels give whole-inference latency in milliseconds. Dark bars show latency under CUDA graphs, including compilation for π0.5, relative to the eager total
Figure 8 of the paper
Figure 8. Stage-time interpretation across seven VLAs and four platforms. Gray shows measured CUDA graph savings; hatched gray shows estimated removable time for flow models. Orange and blue are compute and memory reference costs; yellow is unexplained time. Allocation and clipping are described in the text. These portions do not establish measured bottlenecks
Figure 9 of the paper
Figure 9. Execution-mode savings versus estimated eager time out￾side kernels, I/T. Circles use CUDA graphs alone; squares com￾bine graphs with compilation. The dashed line marks equality. Time outside kernels combines unprofiled latency with a separate CUPTI trace. ence time per kernel divides total unprofiled time by kernel count for models with estimated time outside kernels above 40%. This is not a direct CPU-time measurement. Quantization implementations
Figure 11 of the paper
Figure 11. Int8 prefill speed-up on A10G at matching pruning lev￾els under reduce-overhead compilation. Earlier layers retain the original token count. Lines connect bf16/int8 comparisons
Figure 12 of the paper
Figure 12. GPU memory by model, platform and execution mode at batches one and eight. Stacks separate weights, inference memory, setup-only memory and the outside-allocator estimate. Signed labels give GiB above eager execution. An asterisk marks CUDA-graph modes that include compilation for flow models
Figure 13 of the paper
Figure 13. Inference latency versus denoising-step count. Colors identify platforms. Dashed lines and open markers show eager execution; solid lines and filled markers show compilation with CUDA graphs. Lines connect measurements. Panel titles give default step counts
Figure 14 of the paper
Figure 14. Unpruned batch latency Tb relative to the nominal inference budget B. Colors and markers identify models; both axes are logarithmic. The dashed line marks Tb = B. Curves use the execution modes stated in the text and do not estimate robot capacity

Showing 13 of the paper's 14 figures. Figures are the authors', reproduced from the paper.

References (53)
  1. [1] Openvla: An open-source vision-language-action model , author=. arXiv preprint arXiv:2406.09246 , year= on PithPith's reviewarXiv
  2. [2] Intelligence, Physical and Black, Kevin and Brown, Noah and Darpinian, James and Dhabalia, Karan and Driess, Danny and Esmail, Adnan and Equi, Michael and Finn, Chelsea and Fusai, Niccolo and others , journal=. pi\_ on Pith
  3. [3] Brianna Zitkovich and others , booktitle=. 2023 , url= on Pith
  4. [4] How fast can i run my vla? demystifying vla inference performance with vla-perf , author=. arXiv preprint arXiv:2602.18397 , year= on PitharXiv
  5. [5] Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment , author=. arXiv preprint arXiv:2604.24447 , year= on PithPith's reviewarXiv
  6. [6] An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models , author=. European Conference on Computer Vision , pages=. 2024 , organization= on Pith
  7. [7] Flashattention-2: Faster attention with better parallelism and work partitioning , author=. International Conference on Learning Representations , volume= on Pith
  8. [8] Roofline: an insightful visual performance model for multicore architectures , author=. Communications of the ACM , volume=. 2009 , publisher= on Pith
  9. [9] Llm inference unveiled: Survey and roofline model insights , author=. arXiv preprint arXiv:2402.16363 , year= on PithPith's reviewarXiv
  10. [10] Time-based roofline for deep learning performance analysis , author=. 2020 IEEE/ACM Fourth Workshop on Deep Learning on Supercomputers (DLS) , pages=. 2020 , organization= on Pith
  11. [11] Beyond LLM Serving: Characterizing Vision-Language-Action Workloads for Embodied AI System Design , author=. arXiv preprint arXiv:2610.05062 , year= on PithPith's reviewarXiv
  12. [12] Rethinking the practicality of vision-language-action model: A comprehensive benchmark and an improved baseline , author=. arXiv preprint arXiv:2602.22663 , year= on PitharXiv
  13. [13] Fine-tuning vision-language-action models: Optimizing speed and success , author=. arXiv preprint arXiv:2502.19645 , year= on PithPith's reviewarXiv
  14. [14] Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation , author=. arXiv preprint arXiv:2411.19650 , year= on PithPith's reviewarXiv
  15. [15] Gr00t n1: An open foundation model for generalist humanoid robots , author=. arXiv preprint arXiv:2503.14734 , year= on PithPith's reviewarXiv
  16. [16] Smolvla: A vision-language-action model for affordable and efficient robotics , author=. arXiv preprint arXiv:2506.01844 , year= on PithPith's reviewarXiv
  17. [17] Scalable diffusion models with transformers , author=. 2023 IEEE/CVF International Conference on Computer Vision (ICCV) , pages=. 2023 , organization= on Pith
  18. [18] Denoising diffusion implicit models , author=. arXiv preprint arXiv:2010.02502 , year= on PithPith's reviewarXiv
  19. [19] Flow matching for generative modeling , author=. arXiv preprint arXiv:2210.02747 , year= on PithPith's reviewarXiv
  20. [20] 2: Faster Machine Learning through Dynamic Python Bytecode Transformation and Graph Compilation; 2024 , author=. Google Scholar , pages= on Pith
show 33 more
  1. [21] Snapflow: One-step action generation for flow-matching vlas via progressive self-distillation , author=. arXiv preprint arXiv:2604.05656 , year= on PithPith's reviewarXiv
  2. [22] Fewer Steps, Better Actions: Rethinking Flow-Matching Inference for VLA Policies , author=. arXiv preprint arXiv:2609.21216 , year= on PithPith's reviewarXiv
  3. [23] Vla-cache: Efficient vision-language-action manipulation via adaptive token caching , author=. Advances in Neural Information Processing Systems , volume= on Pith
  4. [24] Bridging the Semantic-Action Gap in Visual Token Pruning for Efficient VLA Inference , author=. arXiv preprint arXiv:2511.16449 , year= on PithPith's reviewarXiv
  5. [25] Real-time execution of action chunking flow policies , author=. Advances in Neural Information Processing Systems , volume= on Pith
  6. [26] Lerobot: An open-source library for end-to-end robot learning , author=. International Conference on Learning Representations , volume= on Pith
  7. [27] 2026 , howpublished= on Pith
  8. [28] Quantization-aware imitation-learning for resource-efficient robotic control , author=. arXiv preprint arXiv:2412.01034 , year= on PithPith's reviewarXiv
  9. [29] Nn-meter: Towards accurate latency prediction of deep-learning model inference on diverse edge devices , author=. Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services , pages= on Pith
  10. [30] Hat: Hardware-aware transformers for efficient natural language processing , author=. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages= on Pith
  11. [31] \ TVM \ : An automated \ End-to-End \ optimizing compiler for deep learning , author=. 13th USENIX symposium on operating systems design and implementation (OSDI 18) , pages= on Pith
  12. [32] Ansor: Generating \ High-Performance \ tensor programs for deep learning , author=. 14th USENIX symposium on operating systems design and implementation (OSDI 20) , pages= on Pith
  13. [33] QVLA: Not All Channels Are Equal in Vision-Language-Action Model's Quantization , author=. arXiv preprint arXiv:2602.03782 , year= on PitharXiv
  14. [34] ActQuant: Sub-4-bit Action-Guided Quantization for Vision-Language-Action Models , author=. arXiv preprint arXiv:2605.24011 , year= on PithPith's reviewarXiv
  15. [35] Marlin: Mixed-precision auto-regressive parallel inference on large language models , author=. Proceedings of the 30th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming , pages= on Pith
  16. [36] Efficientvla: Training-free acceleration and compression for vision-language-action models , author=. Advances in Neural Information Processing Systems , volume= on Pith
  17. [37] Specprune-vla: Accelerating vision-language-action models via action-aware self-speculative pruning , author=. arXiv preprint arXiv:2509.05614 , year= on PithPith's reviewarXiv
  18. [38] Sp-vla: A joint model scheduling and token pruning approach for vla model acceleration , author=. The Fourteenth International Conference on Learning Representations , year= on Pith
  19. [39] Think twice, act once: Token-aware compression and action reuse for efficient inference in vision-language-action models , author=. arXiv preprint arXiv:2505.21200 , year= on PithPith's reviewarXiv
  20. [40] The better you learn, the smarter you prune: Towards efficient vision-language-action models via differentiable token pruning , author=. 2026 IEEE International Conference on Robotics and Automation (ICRA) , pages=. 2026 , organization= on Pith
  21. [41] Action-aware dynamic pruning for efficient vision-language-action manipulation , author=. International Conference on Learning Representations , volume= on Pith
  22. [42] CogVLA: Cognition-aligned vision-language-action models via instruction-driven routing & sparsification , author=. Advances in neural information processing systems , volume= on Pith
  23. [43] Token expand-merge: Training-free token compression for vision-language-action models , author=. IEEE Robotics and Automation Letters , year= on Pith
  24. [44] DepthCache: Depth-Guided Training-Free Visual Token Merging for Vision-Language-Action Model Inference , author=. arXiv preprint arXiv:2603.10469 , year= on PitharXiv
  25. [45] Training-Free Interaction-Aligned Visual Token Pruning for Efficient Embodied Manipulation , author=. arXiv preprint arXiv:2603.22991 , year= on PithPith's reviewarXiv
  26. [46] Bitvla: 1-bit vision-language-action models for robotics manipulation , author=. arXiv preprint arXiv:2506.07530 , year= on PitharXiv
  27. [47] Sqap-vla: A synergistic quantization-aware pruning framework for high-performance vision-language-action models , author=. arXiv preprint arXiv:2509.09090 , year= on PithPith's reviewarXiv
  28. [48] Deer-vla: Dynamic inference of multimodal large language models for efficient robot execution , author=. Advances in Neural Information Processing Systems , volume= on Pith
  29. [49] Mole-vla: Dynamic layer-skipping vision language action model via mixture-of-layers for efficient robot manipulation , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume= on Pith
  30. [50] Pd-vla: Accelerating vision-language-action model integrated with action chunking via parallel decoding , author=. 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages=. 2025 , organization= on Pith
  31. [51] Spec-vla: speculative decoding for vision-language-action models with relaxed acceptance , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages= on Pith
  32. [52] TinyVLA: toward fast, data-efficient vision-language-action models for robotic manipulation , author=. IEEE Robotics and Automation Letters , volume=. 2025 , publisher= on Pith
  33. [53] Vla-adapter: An effective paradigm for tiny-scale vision-language-action model , author=. Proceedings of the AAAI conference on artificial intelligence , volume= on Pith

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment