Pith.

Robotics · arXiv:2610.37091 v1 · submitted 27 Oct 2026 · 56 references

GAWM: A Compact Latent World-Action Model for Multi-Task Robot Manipulation

Yang Liu, Xiaoli Liu, Liuzhao Li, Yuduo Song, Muyuan Liu, Xiang Gao

The paper says

“World-action models (WAMs) incorporate the prediction of future observations into robot policy learning. However, most WAMs build on billion-parameter vision-language or video-generation backbones, which incur high training cost and inference latency.”

more of the abstract

In this letter, we present GAWM (Geometry-Aware World Model), a compact multi-task WAM that predicts future observations in the latent space of a frozen DINOv3 encoder. GAWM compresses the features of each camera view into a small set of tokens. A lightweight predictor, conditioned on the task rather than on actions, forecasts these tokens at two future horizons. The forecast is supervised with the dense patch features of future frames, which requires it to encode the spatial layout of the future scene. An action decoder directly regresses a chunk of future actions from the current and predicted tokens. With fewer than 100M parameters and no robot or video pretraining, GAWM outperforms billion-parameter WAMs on RoboTwin 2.0 and performs comparably to billion-parameter VLAs on LIBERO. On a dual-arm robot, it achieves lower action-prediction error on held-out demonstrations than a fine-tuned billion-parameter VLA, with substantially lower inference latency. Ablation studies show that the latent world model substantially improves performance when training data or training time is limited. Code is available at https://github.com/gaoxiang12/ga-wm.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 28 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. GAWM is a genuine new design point with well-controlled internal evidence, but the headline superiority over billion-parameter WAMs rests on cross-paper baselines from a slightly different simulator build—treat the ~1-point margins as provisional. This line is wrong

3 major objections · 6 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 56 references say.

The paper says, at Section IV-A and IV-B, Table I

The review did not quote this passage.

Pith says: Objection 1 of 3 · major

The central claim that GAWM-S outperforms billion-parameter WAMs on RoboTwin 2.0 is a cross-paper comparison made on a non-identical evaluation protocol. Section IV-A states: 'Our simulator build fixes the renderer's denoiser and inverse-kinematics checks without changing the task logic.' A renderer-denoiser change alters the pixel observations every policy receives, and a change in inverse-kinematics checks alters how proposed joint commands …

more of objection 1

… are validated or executed; both can affect different policies differently, so 'task logic unchanged' does not establish equivalence with the build on which LaWAM (92.64/89.80), VLAct-PI (93.0/88.8), LingBot-VA (92.93/91.55) and Fast-WAM (91.88/91.78) were evaluated. The margins are roughly one point in clean (93.94 vs 93.0 for VLAct-PI) and 1.3 points in randomized (93.10 vs 91.78 for Fast-WAM), so a protocol shift of this size could rearrange the ranking. The statement in Section IV-B that the four strongest baselines are 'below the lower bounds of these intervals' covers only GAWM-S's own sampling error over its 5,000 episodes; the baseline scores are single point estimates without intervals, and cross-protocol variance is unquantified. Because the code is released, I request that the leading baselines with public code be re-run under the GAWM build and protocol (same cameras, action space, outlier list, 100 episodes per task), or that the superiority claim be reworded as build-relative and presented with the protocol caveats attached to every quoted margin.

The paper says, at Section IV-B and Table I

The review did not quote this passage.

Pith says: Objection 2 of 3 · major

The claim that GAWM-S 'attains 95.6% ... above π0 (3.2B, 94.2%)' on LIBERO compares GAWM-S trained on the Extended data (6,585 trajectories, including 89 rollouts of a StarVLA Qwen3-VL-OFT policy) with baselines trained on the 1,693-demonstration Standard set. The † marker and footnote in Table I disclose this, and the bold/underline rule excludes the † row, but the abstract, Fig. …

more of objection 2

… 1 and Section IV-B present the comparison without the caveat, and Fig. 1 plots GAWM-S on the same curve as the Standard-data baselines. The paper's own Table II shows that the world-model gain on the Extended data is only 2.5 points with CI [-0.8, 6.1], so LIBERO numbers are sensitive to data quantity; a like-for-like run of GAWM-S on the Standard data, or a comparison restricted to the same data, is needed before 'above π0' can be claimed.

The paper says, at Section IV-D and Table III

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

The real-robot GAWM-S is 'initialized from the 765k-step RoboTwin 2.0 model and fine-tuned for 40k steps', meaning it inherits the 1,749 GPU-hours of simulated-robot training reported in Section IV-B; the abstract's 'no robot or video pretraining' phrasing does not convey this for the real-robot experiment.

more of objection 3

The offline error comparison also spans different action protocols: GAWM-S regresses 16-step absolute joint targets while π0.5 uses 20-step flow-matched chunks under its own action normalization, so the per-step MAE difference (0.072 vs 0.092–0.098 rad) may reflect protocol rather than prediction accuracy; the closed-loop evidence is limited to a single task with 50 trials. I recommend stating the RoboTwin initialization wherever the real-robot result is summarized, and either matching the action protocol for the offline error comparison or presenting closed-loop success on additional tasks as the primary evidence.

6 minor objections

The paper says, at Section IV-C, Table II

Pith says: minor objection 1 of 6

Per-suite bootstrap intervals are not reported, and the Object suite shows a negative effect (−3.6 points) from the world model; per-suite intervals would clarify whether this is noise, given that the aggregate quarter-data CI is [5.2, 14.2].

The paper says, at Section IV-C, forecast quality

Pith says: minor objection 2 of 6

The statement that the forecast similarity 'approaches 0.983, the bound of reconstructing the current frame from 64 tokens' asserts a bound without saying how it is computed; specify the derivation or replace the word 'bound'.

show 4 more

The paper says, at Section IV-B, model size and cost

Pith says: minor objection 3 of 6

The ViT-L comparison on RoboTwin is made against the 765k-step GAWM-S checkpoint (94.36/93.48) rather than the final fine-tuned model (93.94/93.10); since the fine-tuning stage shifts success by −0.4 points, state explicitly which checkpoint is used so the reader can align the numbers.

The paper says, at Section IV-A, Extended LIBERO data

Pith says: minor objection 4 of 6

The 89 rollouts of StarVLA Qwen3-VL-OFT are used as training data for a method that is also compared against StarVLA in Table I; report the rollout success rate and the conditions under which those rollouts were collected.

The paper says, at Fig. 1 and Fig. 3

Pith says: minor objection 5 of 6

The LIBERO panel of Fig. 1 should carry the † marker or an equivalent caption note, since the † appears only in Table I and the abstract-level claim is stated without it; the latency comparison would also be more informative as a control-rate or per-executed-step figure, because GAWM-S executes 16 of its 32 predicted steps.

The paper says, at Section IV-C, ablation definition

Pith says: minor objection 6 of 6

The 'without world model' variant removes both the forecast from the decoder memory and all feature losses (Lfut, Lcur, Ldiv, Lvar, Ltmp) simultaneously, so the ablation conflates test-time future conditioning with the auxiliary losses' shaping of the adapter; an additional variant that keeps the feature losses but omits the forecast from the decoder memory would isolate the mechanism behind the large RoboTwin gap (90.7% vs 18.8%).

What would settle it

One build-controlled A/B test settles this. Using the released code, (1) run GAWM-S with released weights on the unmodified upstream RoboTwin 2.0 build under the same 100 episodes per task and the same 14-D action space; and (2) run one officially released baseline with the closest protocol (LiLa-WAM or TurboVLA) on GAWM's fixed build using the same episode list and evaluation harness. Recompute the clean and randomized success rates with Wilson intervals. If the reported margins (e.g., +0.94 clean versus VLAct-PI, +1.32 randomized versus Fast-WAM) persist, the headline stands; if they shrink below the interval width or reverse, the claim must be qualified to GAWM's build. A secondary check is to report GAWM-S on the Standard LIBERO set (1,693 demonstrations) so the 'above π0' comparison uses equivalent training data.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 28 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 1 min 7 s · 17,320 tokens read, 7,695 written
  3. Editor wrote the verdict above 47 s · 18,312 tokens read, 6,255 written
  4. Referee major revision: 3 major objections, 6 minor 2 min 17 s · 17,791 tokens read, 13,648 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Quick take: GAWM is a legitimate new design point—a task-conditioned, action-free latent predictor that forecasts two horizons in parallel and feeds the forecast into a reactive ACT-style decoder—and the internal evidence is better controlled than most of this literature. The headline "under-100M WAM beats billion-parameter WAMs" is plausible but not yet settled, because the margin over the strongest baselines is a point or two and the baseline numbers come from other papers running on a slightly different simulator build.

What's new is the configuration. The pieces are borrowed—frozen DINOv3 features and dense patch targets from LiLa-WAM, ACT/TurboVLA decoder, JEPA-style latent prediction from LeWM/DINO-WM—but the combination is new: task-conditioned rather than action-conditioned forecasting, no search, no goal image, one forward pass. The paper argues that design rationale clearly.

What it does well: the ablations are properly isolated. Removing the world model (forecast dropped from decoder memory, all feature losses off) costs 9.5 points [5.2, 14.2] on quarter-data LIBERO and collapses RoboTwin from 90.7% to 18.8% at 191k steps. That is real evidence the latent forecast carries control-relevant signal when data is scarce. The forecast-quality analysis is concrete: per-patch cosine to true future features beats a copy of the current frame on 91% of anchors at +32 frames. Latency of 8.8 ms per chunk is a genuine engineering result. Statistical practice is above average—paired per-task bootstrap intervals, Wilson intervals, parameter-count conventions disclosed in the table footnote—and they even report the task-code fine-tuning stage changing success by −0.4 points with intervals including zero.

Soft spots, in proportion:

(1) The RoboTwin headline depends on cross-paper numbers. Section IV-A discloses that GAWM's simulator build fixes the renderer's denoiser and inverse-kinematics checks. Disclosure is honest but not equivalence: a denoiser change alters the observations, an IK-check change alters command validation. GAWM's clean margin over the strongest baselines is roughly one point (93.94 vs 93.0 for VLAct-PI, 92.93 LingBot-VA, 92.64 LaWAM). A protocol shift of one point rearranges the ranking. This is the one thing I'd want resolved before the superiority claim goes from plausible to established.

(2) The LIBERO 95.6% uses the Extended data set, disclosed by the † and composed in detail in Section IV-A, while most baselines use Standard. The abstract and intro invite the π0 comparison without stating that asymmetry. Presentation issue, easy to fix.

(3) The real-robot evidence: the headline metric is action-prediction error on held-out demonstrations, a proxy. Closed-loop success is 46/50 vs 39/50 on one task. Direction agrees with the offline signal and the claim is stated carefully, but it is one task.

Who this is for: anyone building compact manipulation policies, latent world models, or benchmark protocols. It deserves a serious referee. My recommendation: send it to review; ask for reruns of the strongest baselines on GAWM's build or a softened superiority claim, and move the Extended-data condition into the abstract.

The strongest objection

GAWM's headline result on RoboTwin 2.0 is a cross-paper comparison. The paper is transparent about its protocol, but its own Section IV-A introduces a protocol difference: 'Our simulator build fixes the renderer's denoiser and inverse-kinematics checks without changing the task logic.' A renderer denoiser change alters the observations the policy sees, and an IK-check change alters how a proposed joint command is validated or executed. Both can affect different policies differently, so 'task logic unchanged' does not make the evaluation equivalent to the build on which LaWAM, LingBot-VA, Fast-WAM and VLAct-PI scores were reported. The margins are small: clean is 93.94 vs 93.0 for VLAct-PI, 92.93 for LingBot-VA and 92.64 for LaWAM; randomized is 93.10 vs 91.78 for Fast-WAM, 91.55 for LingBot-VA and 89.80 for LaWAM. A protocol shift of a point or two could rearrange the ranking. Table I also records other non-equivalences: LiLa-WAM uses one third-person camera and fewer evaluation episodes, TurboVLA uses three cameras and unseen instructions, and GAWM uses 14-D absolute joint targets. These are disclosed, but disclosure does not establish that the quoted baseline scores were obtained under a protocol equivalent to GAWM's. The LIBERO 'above π0' claim has a separate data-set mismatch (Extended vs Standard), also disclosed by the † marker; it reinforces the same need for protocol-controlled reruns rather than being the primary issue.

I read the paper as an empirical claim that a very small latent world-action model can match or beat much larger WAMs on fixed task sets. The internal evidence is strong where it is controlled: the world-model ablation in Table II is properly paired, the latency measurements use official implementations on the same GPU, the randomized-versus-clean gap on RoboTwin 2.0 is plausible, and the forecast and attention analyses support the proposed mechanism. The weakest point is not the method but the headline comparison. The RoboTwin 2.0 numbers in Table I come from other papers, while GAWM is evaluated on a modified simulator build. The authors disclose this exactly in Section IV-A, so there is no hidden claim, but disclosure is not equivalence. A renderer denoiser fix and an IK-check fix can plausibly change success rates by more than the one-to-three-point margins involved, and the effect may be policy-dependent. That makes the central comparative claim conditional on a rerun rather than settled. I agree with the reader's weakest_assumption and with the CONDITIONAL verdict; this does not require moving to REJECT because the issue is empirically resolvable and the paper's own ablations are sound. I would not, however, let the abstract claim 'outperforms billion-parameter WAMs' stand unqualified until such a rerun is done. The LIBERO Extended-data point is a genuine secondary qualifier, but the paper marks it with † and in the figure legend, so I weight it less than the simulator-build issue.

The referee's report

If the headline comparisons survive protocol-matched evaluation, the paper makes a solid contribution: it shows that test-time latent foresight can be supplied by a few million parameters on top of frozen features, without a VLM, video generator, or robot/video pretraining, and that the benefit is largest when data and compute are scarce. The internal evidence is well controlled: the world-model ablation uses paired per-task bootstrap intervals (Table II), success rates carry Wilson intervals, RoboTwin evaluation uses 100 episodes per task and 5,000 episodes per setting, forecast quality is quantified per-patch against a frozen target, and latency is measured with official implementations on the same GPU. The released code makes the main experiments reproducible. The weaknesses are in the external comparisons: the RoboTwin superiority claim rests on published numbers obtained under a different simulator build, camera setup, action space and instruction protocol, and the LIBERO 'above π0' claim compares different training data.

Pith's reading of the paper (a machine's summary, not the authors')

This paper tries to establish that foresight — predicting what the scene will look like — does not require a billion-parameter video generator or vision-language backbone. Its model, GAWM, stays under 100 million parameters by compressing each camera view into a small set of visual tokens from a frozen encoder and forecasting those tokens at two future horizons with a predictor conditioned on the task identity rather than on actions. The claim is that this compact design reports higher success than billion-parameter world-action models on RoboTwin 2.0 (93.94% clean, 93.10% randomized with 77M parameters), competes with billion-parameter vision-language-action models on LIBERO (95.6% with 38M parameters, using extra in-domain data), and answers each query in milliseconds rather than seconds. The authors argue this matters because most deployed robots cannot host billion-parameter models, and their ablations show the latent future prediction is exactly the component that pays off when training data or training time is scarce.

What carries the argument. The load-bearing mechanism is the geometry-aware latent forecast built around a frozen encoder. A frozen DINOv3 ViT-S/16 plays both roles at once: its fused multi-layer patch features are the input to a query adapter that compresses each camera view into 64 tokens, and its patch features from the recorded future frames are the supervised target, compared per patch by a cosine loss through a training-only patch decoder. Because the target comes from a frozen encoder, it cannot collapse, which removes the need for the explicit regularizers that end-to-end JEPA models require. The predictor itself is action-free and task-conditioned: it forecasts $\hat{z}_{t+h_1}$ and $\hat{z}_{t+h_2}$ in parallel, as residual updates of the current tokens $z_t$, with an output projection initialized to zero so the model starts from a static-scene forecast and learns change as a correction ($h_1 = H/2$, $h_2 = H$, the midpoint and end of the action chunk). This design makes world modeling a small subroutine inside a reactive policy: the two forecasts enter the decoder memory directly, so no action search, goal image, or iterative denoising is needed at inference, and the patch decoder is discarded after training. The two-horizon scheme is what lets the decoder's attention align each action step with the temporally nearest forecast group, which the attention analysis documents.

Core claim. The central discovery is that a task-conditioned latent predictor can give a reactive policy the same kind of foresight that existing world-action models buy from video generators or vision-language backbones. GAWM-S encodes each camera with a frozen DINOv3 ViT-S encoder, compresses the fused features into 64 tokens per view, and lets a 4-6M-parameter predictor forecast these tokens at two future horizons in parallel — the midpoint and the end of the action chunk — as residual updates of the current tokens, conditioned on a task code and on nothing else about the action. The forecast is supervised against the frozen encoder's dense patch features of the recorded future frames, which forces the tokens to encode where objects and the gripper will be, so the model learns the spatial layout of the future scene without depth or 3D supervision. An ACT-style action decoder then regresses a whole chunk of future actions from the current tokens, the two forecasts, the robot state and the task code in a single forward pass. On RoboTwin 2.0 the 77M-parameter model reports 93.94% clean and 93.10% randomized success (95% intervals [93.2, 94.6] and [92.4, 93.8]), above the quoted scores of LaWAM (2.1B, 92.64/89.80), LingBot-VA (10.9B, 92.93/91.55) and Fast-WAM (11.9B, 91.88/91.78), at 8.8 ms per query; on LIBERO it reports 95.6% with 38M parameters when given extra in-domain data, above the quoted 94.2% of the 3.2B-parameter pi0. On a real dual-arm handover task its closed-loop success is 92% (46/50) versus 78% (39/50) for a fine-tuned pi0.5, with lower action-prediction error. Two analyses support the interpretation: the decoded forecast is closer to the true future patch features than the current frame is, and each action step attends mostly to the forecast group nearest in time. The authors state plainly that the model targets a closed set of tasks identified by demonstration-derived codes and does not interpret language.

The premise it rests on. The headline margin over billion-parameter world-action models assumes the quoted baseline scores come from the same protocol as GAWM's own runs, including the same RoboTwin 2.0 simulator build — the paper says its own build 'fixes the renderer's denoiser and inverse-kinematics checks without changing the task logic' — so if the published baselines were evaluated on the original build, or with different cameras or action spaces, the reported advantage could shrink or reverse.

If the paper is right (Pith's inferences, not the authors')

On closed task sets, foresight can be decoupled from language and from video generation: a task-identifier-conditioned latent predictor is enough to match or beat world-action models hundreds of times larger, so a small controller can sit under a language layer that only picks the task.

World-action models become deployable at control rates: 7-9 ms per action chunk against 114-3,763 ms for video-based WAMs means the same foresight fits on onboard hardware where billion-parameter backbones cannot.

The world-model component pays for itself most when data or compute is scarce: removing it costs 9.5 points on a quarter of the LIBERO demonstrations and drops RoboTwin 2.0 success from 90.7% to 18.8% under a shortened schedule, so latent future prediction acts as a strong regularizer for low-data imitation.

Because the forecast is supervised with location-specific patch features, the model learns the spatial layout of the future scene without depth or 3D supervision, making latent-token prediction a cheap substitute for explicit geometric reasoning in manipulation.

A single predictor shared across 50 bimanual tasks and 40 LIBERO tasks transfers across a closed task set at under 100M total parameters, keeping per-task training cost low (45 GPU-hours on LIBERO).

Because the predictor is conditioned on the task and not on actions, GAWM cannot answer counterfactual questions such as what would happen if the arm pushed left; a natural extension is to condition the same latent predictor on a compact action embedding as well, keeping the search-free decoder while gaining action-conditioned futures.

The LIBERO-Object ablation, where removing the world model actually improves success by 3.6 points, suggests the dense future target can be a distraction when tasks differ mainly in object identity; weighting the future loss per task is a testable refinement the paper does not explore.

The zero-initialized residual forecast implies GAWM begins training as a plain ACT-style policy and learns the future as a correction; a testable prediction is that early checkpoints behave like the no-world-model ablation, with foresight becoming behaviorally relevant only once the forecast separates from the static scene.

The attention pattern in which each action step favors the temporally nearest forecast group suggests the two horizons act as a temporal index for the chunk; a direct test is whether adding intermediate horizons improves the long-horizon LIBERO-Long suite, where the world model's measured gain was largest (21.0 points).

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

World-action models (WAMs) incorporate the prediction of future observations into robot policy learning. However, most WAMs build on billion-parameter vision-language or video-generation backbones, which incur high training cost and inference latency. In this letter, we present GAWM (Geometry-Aware World Model), a compact multi-task WAM that predicts future observations in the latent space of a frozen DINOv3 encoder. GAWM compresses the features of each camera view into a small set of tokens. A lightweight predictor, conditioned on the task rather than on actions, forecasts these tokens at two future horizons. The forecast is supervised with the dense patch features of future frames, which requires it to encode the spatial layout of the future scene. An action decoder directly regresses a chunk of future actions from the current and predicted tokens. With fewer than 100M parameters and no robot or video pretraining, GAWM outperforms billion-parameter WAMs on RoboTwin 2.0 and performs comparably to billion-parameter VLAs on LIBERO. On a dual-arm robot, it achieves lower action-prediction error on held-out demonstrations than a fine-tuned billion-parameter VLA, with substantially lower inference latency. Ablation studies show that the latent world model substantially improves performance when training data or training time is limited. Code is available at https://github.com/gaoxiang12/ga-wm.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-28

The authors' main claim, as the classifier picked it out 1.00

“With fewer than 100M parameters and no robot or video pretraining, GAWM outperforms billion-parameter WAMs on RoboTwin 2.0 and performs comparably to billion-parameter VLAs on LIBERO.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.13 · human participants 0.03 · controlled comparison 0.94 · randomized assignment 0.01 · states quantitative result 0.93 · compares with prior results 0.98 · mentions code or data availability 0.99

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
World-action models (WAMs) incorporate the prediction of future observations into robot policy learning. background 1.00 0.06 0.02 asserted 1.00
However, most WAMs build on billion-parameter vision-language or video-generation backbones, which incur high training cost and inference latency. background 0.63 0.07 0.03 asserted 0.99
In this letter, we present GAWM (Geometry-Aware World Model), a compact multi-task WAM that predicts future observations in the latent space of a frozen DINOv3 encoder. contribution 0.94 0.90 0.10 asserted 1.00
GAWM compresses the features of each camera view into a small set of tokens. method 0.99 0.88 0.03 asserted 1.00
A lightweight predictor, conditioned on the task rather than on actions, forecasts these tokens at two future horizons. method 0.99 0.88 0.03 asserted 1.00
The forecast is supervised with the dense patch features of future frames, which requires it to encode the spatial layout of the future scene. method 0.99 0.78 0.03 asserted 0.98
An action decoder directly regresses a chunk of future actions from the current and predicted tokens. method 1.00 0.85 0.03 asserted 1.00
With fewer than 100M parameters and no robot or video pretraining, GAWM outperforms billion-parameter WAMs on RoboTwin 2.0 and performs comparably to billion-parameter VLAs on LIBERO. result 1.00 0.95 0.73 asserted 1.00
On a dual-arm robot, it achieves lower action-prediction error on held-out demonstrations than a fine-tuned billion-parameter VLA, with substantially lower inference latency. result 1.00 0.95 0.21 asserted 1.00
Ablation studies show that the latent world model substantially improves performance when training data or training time is limited. result 0.95 0.95 0.11 asserted 1.00
Code is available at https://github.com/gaoxiang12/ga-wm. contribution 0.74 0.50 0.02 asserted 0.90

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.88

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-28: 1 request, 8,171 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (6)
Figure 1 of the paper
Figure 1. Success rate versus model size on LIBERO (left; GAWM-S with additional in-domain data) and on the clean (middle) and randomized (right) settings of RoboTwin 2.0. Top: success rate versus size. Bottom: methods ranked by success rate (SR) with size (as in Table I, or as reported; N/R: not reported). Baselines (references in Table I and [15]–[28]): blue, as reported by TurboVLA [29]; gray, other sources. On RoboTwin 2.0, GAWM-S is the smallest and best-performing of the plotted methods. With 77M parameters and no robot or video pretrain￾ing, GAWM-S, the ViT-S variant, achieves success…
Figure 2 of the paper
Figure 2. Overview of GAWM. A frozen DINOv3 encoder and a query adapter encode each camera view into 64 visual tokens, stacked into zt. The latent predictor, conditioned on the task code gk, forecasts these tokens at two future horizons, and the action decoder directly regresses a chunk of future actions from the current tokens, the forecast, the robot state and the task code. Dashed panel: training only; a patch decoder maps the forecast to the encoder’s patch features of the recorded future frames (thumbnails as in
Figure 3 of the paper
Figure 3. Inference latency per action-chunk query. Official implementa￾tions on the same GPU model (NVIDIA RTX PRO 5000) with batch size 1 and default precision and denoising steps. Latency covers the modules counted in Table I, from preprocessed inputs to the action chunk (median of 200 queries). †: LIBERO configuration; all others use RoboTwin 2.0. Parentheses: parameters as in Table I. Baselines are randomly initialized, which does not change the computation. ters (Table I; 95% intervals [93.2, 94.6] and [92.4, 93.8]). The four strongest baselines, LaWAM (2.1B), VLAct-PI (≈4B), LingBot-VA…
Figure 4 of the paper
Figure 4. Decoded forecasts. GAWM-S on RoboTwin 2.0, head camera. Left: frames t, t+16 and t+32. Right: first three principal components of the encoder patch features of the true future frames (True) and of the decoded forecast (Pred.), projected on a PCA basis shared within each example. Labels: mean per-patch cosine similarity to the true features for the forecast and for frame t. Examples: first clean demonstrations, at the frame with the largest 32-frame feature change within 20–80% of the episode. 1 8 16 24 32 action step in the chunk 0 20 40 attention (%) current tokens zt forecast ẑ t…
Figure 5 of the paper
Figure 5. Attention of GAWM-S on RoboTwin 2.0. Top: head-camera frame t of the examples of
Figure 6 of the paper
Figure 6. Real-robot task. Head-camera frames of a demonstration of the green-bottle task. The four tasks differ only in the target bottle. TABLE III EVALUATION ON THE REAL ROBOT GAWM-S π0.5 (40k) π0.5 (60k) Joint error, first step 0.072 0.098 0.092 Joint error, full chunk 0.088a 0.106b 0.101b Gripper error, first step 0.020 0.028 0.023 Closed-loop success, yellow bottle 46/50c – 39/50 Latency, median / 95th pct. (ms) 9.3 / 9.6 63.8 / 63.9 Errors: mean absolute errors on the 81 test demonstrations with respect to the recorded joint targets of both arms (rad) and gripper commands. Success:…

Figures are the authors', reproduced from the paper.

References (56)
  1. [1] B. Zitkovich et al., “RT-2: Vision-language-action models transfer web knowledge to robotic control,” in Proc. Conf. Robot Learn. (CoRL) , 2023 on Pith
  2. [2] M. J. Kim et al. , “OpenVLA: An open-source vision-language-action model,” in Proc. Conf. Robot Learn. (CoRL) , 2024 on Pith
  3. [3] K. Black et al. , “ π0: A vision-language-action flow model for general robot control,” in Proc. Robot.: Sci. Syst. (RSS) , 2025 on Pith
  4. [4] ——, “ π0.5: A vision-language-action model with open-world gener- alization,” in Proc. Conf. Robot Learn. (CoRL) , 2025 on Pith
  5. [5] Y . Du et al. , “Learning universal policies via text-guided video gen- eration,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS) , 2023 on Pith
  6. [6] H. Wu et al. , “Unleashing large-scale video generative pre-training for visual robot manipulation,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2024 on Pith
  7. [7] Y . Tianet al., “Predictive inverse dynamics models are scalable learn- ers for robotic manipulation,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2025 on Pith
  8. [8] L. Li et al. , “Causal world modeling for robot control,” arXiv:2601.21998, 2026 on PithPith's reviewarXiv
  9. [9] T. Yuan, Z. Dong, Y . Liu, and H. Zhao, “Fast-W AM: Do world action models need test-time future imagination?” arXiv:2603.16666, 2026 on PithPith's reviewarXiv
  10. [10] J. Chen et al. , “LaW AM: Latent world action models for efficient dynamics-aware robot policies,” in Proc. Conf. Robot Learn. (CoRL) , 2026, to be published on Pith
  11. [11] F. Y anget al., “LiLa-W AM: Lightweight latent reasoning world-action model for robotic manipulation,” arXiv:2608.03701, 2026 on PithPith's reviewarXiv
  12. [12] Q. Gu et al. , “The right future for action: Learning action-relevant predictive states in world action models,” arXiv:2609.23369, 2026 on PithPith's reviewarXiv
  13. [13] W . Zhao et al. , “Faster-W AM: Efficient inference-time future condi- tioning for robust world action models,” arXiv:2608.04404, 2026 on PithPith's reviewarXiv
  14. [14] Y . Liuet al., “PLaW-VLA: Predictive latent world modeling for vision- language-action policies,” in Proc. Conf. Robot Learn. (CoRL) , 2026, to be published on Pith
  15. [15] C. Chi et al., “Diffusion Policy: Visuomotor policy learning via action diffusion,” in Proc. Robot.: Sci. Syst. (RSS) , 2023 on Pith
  16. [16] K. Pertsch et al. , “FAST: Efficient action tokenization for vision- language-action models,” in Proc. Robot.: Sci. Syst. (RSS) , 2025 on Pith
  17. [17] Q. Bu et al. , “Learning to act anywhere with task-centric latent ac- tions,” in Proc. Robot.: Sci. Syst. (RSS) , 2025 on Pith
  18. [18] W . Li, R. Zhang, R. Shao, J. He, and L. Nie, “CogVLA: Cognition- aligned vision-language-action models via instruction-driven routing & sparsification,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS) , 2025 on Pith
  19. [19] Y . Y anget al., “Mantis: A versatile vision-language-action model with disentangled visual foresight,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2026 on Pith
  20. [20] H. Liang et al., “MM-ACT: Learn from multimodal parallel generation to act,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2026 on Pith
show 36 more
  1. [21] J. Sun et al. , “VLA-JEPA: Enhancing vision-language-action model with latent world model,” in Proc. Eur. Conf. Comput. Vis. (ECCV) , 2026 on Pith
  2. [22] X. Wu et al. , “Generation models know space: Unleashing implicit 3D priors for scene understanding,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2026 on Pith
  3. [23] Z. Liang et al. , “Discrete Diffusion VLA: Bringing discrete diffusion to action decoding in vision-language-action policies,” in Proc. Int. Conf. Mach. Learn. (ICML) , 2026 on Pith
  4. [24] W. Zhang et al. , “DreamVLA: A vision-language-action model dreamed with comprehensive world knowledge,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS) , 2025 on Pith
  5. [25] W. Wu et al. , “A pragmatic VLA foundation model,” arXiv:2601.18692, 2026 on PithPith's reviewarXiv
  6. [26] A. Y e et al. , “GigaWorld-Policy: An efficient action-centered world– action model,” arXiv:2603.17240, 2026 on PitharXiv
  7. [27] Y . Y anget al. , “ABot-M0: Vla foundation model for robotic manipu- lation with action manifold learning,” arXiv:2602.11236, 2026 on PithPith's reviewarXiv
  8. [28] Q. Wang et al. , “Qwen-VLA: Unifying vision-language-action modeling across tasks, environments, and robot embodiments,” arXiv:2605.30280, 2026 on PithPith's reviewarXiv
  9. [29] H. Xie et al. , “TurboVLA: Real-time vision-language-action model at 32 Hz on an RTX 4090 with <1 GB VRAM,” arXiv:2607.27205, 2026 on PithPith's reviewarXiv
  10. [30] Wan Team et al. , “Wan: Open and advanced large-scale video gener- ative models,” arXiv:2503.20314, 2025 on PithPith's reviewarXiv
  11. [31] H. Bi et al. , “Motus: A unified latent action world model,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2026 on Pith
  12. [32] B. Ichter et al. , “Do as I can, not as I say: Grounding language in robotic affordances,” in Proc. Conf. Robot Learn. (CoRL) , 2022 on Pith
  13. [33] L. X. Shi et al. , “Hi Robot: Open-ended instruction following with hierarchical vision-language-action models,” in Proc. Int. Conf. Mach. Learn. (ICML) , 2025 on Pith
  14. [34] M. Assran et al. , “Self-supervised learning from images with a joint- embedding predictive architecture,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2023 on Pith
  15. [35] A. Bardes et al. , “Revisiting feature prediction for learning visual representations from video,” Trans. Mach. Learn. Res. , 2024 on Pith
  16. [36] L. Maes, Q. L. Lidec, D. Scieur, Y . LeCun, and R. Balestriero, “LeWorldModel: Stable end-to-end joint-embedding predictive archi- tecture from pixels,” arXiv:2603.19312, 2026 on PithPith's reviewarXiv
  17. [37] M. Liu, Y . Huang, Z. Liang, and X. Gao, “Toward physically grounded JEPA world models for goal-conditioned robotic planning,” arXiv:2609.03565, 2026 on PithPith's reviewarXiv
  18. [38] O. Siméoni et al. , “DINOv3,” Trans. Mach. Learn. Res. , 2026 on Pith
  19. [39] T. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” in Proc. Robot.: Sci. Syst. (RSS) , 2023 on Pith
  20. [40] R. Balestriero and Y . LeCun, “LeJEPA: Provable and scalable self- supervised learning without the heuristics,” arXiv:2511.08544, 2025 on PithPith's reviewarXiv
  21. [41] T. Chen et al. , “RoboT win 2.0: A scalable data generator and bench- mark with strong domain randomization for robust bimanual robotic manipulation,” in Proc. Int. Conf. Mach. Learn. (ICML) , 2026 on Pith
  22. [42] B. Liu et al., “LIBERO: Benchmarking knowledge transfer for lifelong robot learning,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS) , 2023 on Pith
  23. [43] M. J. Kim, C. Finn, and P . Liang, “Fine-tuning vision-language-action models: Optimizing speed and success,” in Proc. Robot.: Sci. Syst. (RSS), 2025 on Pith
  24. [44] X. Zhou et al. , “LIBERO-PRO: Towards robust and fair eval- uation of vision-language-action models beyond memorization,” arXiv:2510.03827, 2025 on PithPith's reviewarXiv
  25. [45] M. Assran et al. , “V-JEPA 2: Self-supervised video models enable understanding, prediction and planning,” arXiv:2506.09985, 2025 on PithPith's reviewarXiv
  26. [46] G. Zhou, H. Pan, Y . LeCun, and L. Pinto, “DINO-WM: World models on pre-trained visual features enable zero-shot planning,” in Proc. Int. Conf. Mach. Learn. (ICML) , 2025 on Pith
  27. [47] J. Xu et al., “JEPA Policy: Diffusion-free imitation learning via paired action and future representation prediction,” arXiv:2609.09630, 2026 on PithPith's reviewarXiv
  28. [48] A. Bardes, J. Ponce, and Y . LeCun, “VICReg: Variance-invariance- covariance regularization for self-supervised learning,” in Proc. Int. Conf. Learn. Represent. (ICLR) , 2022 on Pith
  29. [49] K. Sendai, T. Matsushima, and Y . Iwasawa, “MINERV A: How small can a manipulation policy be and still solve LIBERO?” arXiv:2609.03715, 2026 on PithPith's reviewarXiv
  30. [50] Y . Wang et al. , “VLA-Adapter: An effective paradigm for tiny-scale vision-language-action model,” in Proc. AAAI Conf. Artif. Intell. , 2026 on Pith
  31. [51] StarVLA Community, “StarVLA: A Lego-like codebase for vision- language-action model developing,” arXiv:2604.05014, 2026 on PithPith's reviewarXiv
  32. [52] S. Y anget al., “Beyond data scaling: Representation-centric continued pre-training for vision-language-action models,” arXiv:2608.27550, 2026 on PithPith's reviewarXiv
  33. [53] D. Ghosh et al. , “Octo: An open-source generalist robot policy,” in Proc. Robot.: Sci. Syst. (RSS) , 2024 on Pith
  34. [54] M. Shukor et al. , “SmolVLA: A vision-language-action model for affordable and efficient robotics,” arXiv:2506.01844, 2025 on PithPith's reviewarXiv
  35. [55] T. Lin et al. , “Evo-1: Lightweight vision-language-action model with preserved semantic alignment,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2026 on Pith
  36. [56] J. Zheng et al., “X-VLA: Soft-prompted transformer as scalable cross- embodiment vision-language-action model,” in Proc. Int. Conf. Learn. Represent. (ICLR) , 2026 on Pith

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment