Pith.

Computer Vision and Pattern Recognition · arXiv:2611.02474 v2 · submitted 2 Nov 2026 · 95 references

Gnomon4D: Learning Where and When to Focus for Geometric Motion-Grounded World-Action Modeling

Lingwei Dang, Yan Zhou, Juntong Li, Zhenhao Zhang, Yiteng Chen, Xiaoqiang Liu, Liang An, Hongwen Zhang

show 5 more authors

Wei Min, Guoxin Zhang, Pengfei Wan, Yebin Liu, Qingyao Wu

The paper says

“World-action models (WAMs) predict scene changes to guide robot actions, yet their RGB futures entangle state changes with appearance and lack 3D geometry.”

more of the abstract

Recent WAMs add geometry as a training signal or a control interface, or build native 3D models, but decide what geometry to model rather than how the action should read it. Intuitively, control-relevant cues lie at embodiment–object interactions and their transitions, yet probing existing WAMs reveals that their action attention underweights these interactions in favor of the rest of the embodiment. To focus the action on these interactions, we present Gnomon4D, a WAM that jointly generates future video, robot actions, and GeoDelta, a video-aligned 4D geometric motion field recording the image-plane displacement and depth change of embodiment and object points. With the embodiment and objects in one field, their relative motion, which marks an interaction, can be read off. Guided by this motion and the task, Interaction-Salient Attention (InSAttn) selects where the action reads, and Transition-Anchored Aggregation (TransAgg) selects when, around grasp and release. As GeoDelta encodes surface motion rather than embodiment-specific controls, it serves as a human–robot quasi-action that links action-free pretraining on EgoManip-5000H, which we curate, to robot mid-training on Robot-30H. Gnomon4D leads on RoboTwin 2.0, VLABench, and RoboDojo, and outperforms real-robot baselines, especially under visual shifts. Project page: https://droliven.github.io/Gnomon4D.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 5 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A well-built WAM paper whose GeoDelta representation and real-robot protocol are the real news, but its biggest benchmark margins are confounded by privileged test-time inputs, so the mechanism's true share is unquantified. This line is wrong

2 major objections · 5 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 95 references say.

The paper says, at Sec. 3.4, 4.2, E.1

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

The benchmark margins credited to GeoDelta are not isolated from the model's test-time condition inputs. In Sec. 3.4 the Action expert cross-attends directly to the clean tokens V_obs and G_cond (current depth and foreground mask), so a geometric input channel is available even if the predicted GeoDelta were never read.

more of objection 1

On RoboDojo (E.1), G_cond is a robot-only depth render recomputed at test time from live joint states and camera parameters — the strongest possible geometric prior — yet Sec. 4.2 credits the 23.86 vs 11.92 SR margin and the 28.24 vs 20.42 Precision gain to GeoDelta encoding 'exact positions in both the image plane and depth.' Similarly, the VLABench claim that the margin over OpenWAM-Alpha grows from 0.6 to 7.4 points on unseen textures 'as GeoDelta encodes motion rather than appearance' is not established by a comparison against an RGB-only baseline that lacks the appearance-invariant depth+mask conditions. Table 4 is the right kind of evidence — it holds the conditions fixed and still shows 2.6 points for InSAttn+TransAgg and 3.4 points for full GeoDelta over RGB — but it is restricted to 10 RoboTwin 2.0 tasks. Please add an input-control experiment on VLABench and RoboDojo (for example, ablating G_cond, or providing the baselines with the same rendered depth/mask conditions), or reword the Sec. 4.2 attributions so the margins are described as joint effects of the condition inputs and the GeoDelta readout.

The paper says, at Eq. (8), Sec. E.1, Sec. 4.2 (RoboDojo)

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.71):

“Limitations and future work.Without simulator state, robot GeoDelta is estimated from measured or monocular depth, masks, and optical flow, whose errors on thin, reflec- tive, or transparent surfaces mainly perturb ∆d.”

Pith says: Objection 2 of 2 · major

The RoboDojo Precision claim rests on a depth-change channel whose supervision is estimated, and the magnitude of the resulting risk is unquantified. Object GeoDelta on RoboDojo uses DA3 monocular depth rescaled by a single per-episode factor s fitted against rendered robot depth (Eq. 8); if s is biased for objects at depths different from the arm, or if the DA3/RAFT …

more of objection 2

… correspondences drift, the Δd channel — precisely the channel invoked to explain the 28.24% Precision result — is trained on a biased signal. The Limitations paragraph acknowledges that estimation errors 'mainly perturb Δd,' and the real-robot leg is indeed not the main exposure since it uses measured sensor depth (Sec. F.2, Table 7). The RoboDojo leg is where the risk is largest, and no sensitivity analysis is provided. A concrete fix would be to validate s on episodes with known object geometry, or to retrain the RoboDojo policy with Δd zeroed or corrupted and show that the Precision and overall gains depend on the estimated depth change rather than on the image-plane motion and the rendered conditions. Without such an analysis, the RoboDojo results remain compatible with a label-artifact explanation.

5 minor objections

The paper says, at Sec. 3.3, Eq. (2)

Pith says: minor objection 1 of 5

The depth-change scale ω_d is only described as set from the depth-change statistics of the training data; please report the value per domain (the real-robot section uses ω_d = 0.10 m in F.2) and add a brief sensitivity check for ω_d and the soft thresholds (0.5 px and 2 mm), since the Δd channel is the paper's core differentiator.

The paper says, at Sec. 4.3, Tables 10–11

Pith says: minor objection 2 of 5

With 40 trials per task, per-task margins of two to five successes are within binomial sampling noise, and the aggregate ID margin (165 vs 151 of 200) is roughly 1.7 standard errors, whereas the OOD margin (156 vs 122) is highly significant; please report binomial confidence intervals and either soften the 'ranks first on every task' claim or anchor the strong claim to the OOD aggregate.

show 3 more

The paper says, at Sec. 4.2, E.1

Pith says: minor objection 3 of 5

The main text should state up front that on RoboDojo the model's test-time conditions are robot-only renders computed from live joint states, since the current wording leaves this input-privilege detail in the supplementary only.

The paper says, at Sec. A (Supplementary)

Pith says: minor objection 4 of 5

The attention probe compares Gnomon4D against baselines but does not include a Gnomon4D variant without InSAttn and TransAgg, so the probe difference could in principle reflect pretraining or the GeoDelta supervision rather than the readout; adding such a variant would make the probe diagnostic for the proposed mechanism.

The paper says, at Sec. 3.6, D.1

Pith says: minor objection 5 of 5

EgoManip-5000H is mined from an industrial-scale proprietary video library; please state whether the pretraining corpus, the Robot-30H processing, or the trained checkpoints will be released, since this directly determines the reproducibility of the pretraining-transfer claim.

What would settle it

Run the Table-4-style cross on RoboDojo: train the 2×2 {joint-state depth+mask condition inputs on/off} × {generic-query dense readout / InSAttn+TransAgg} with pretraining fixed, and fine-tune OpenWAM-Alpha on the same depth+mask conditions. If the readout's margin over generic queries collapses when the conditions are removed, or if OpenWAM-Alpha with those conditions approaches the 23.86% SR, then the RoboDojo margin and its 'precision via depth' attribution reflect test-time geometric conditions, not the GeoDelta mechanism.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 5 Nov 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic partly agrees 3 min 12 s · 34,903 tokens read, 26,003 written
  3. Editor wrote the verdict above 58 s · 36,170 tokens read, 9,696 written
  4. Referee major revision: 2 major objections, 5 minor 1 min 45 s · 35,287 tokens read, 15,030 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Two things to know before you read Gnomon4D. The novel piece is real: GeoDelta, a jointly generated 4D motion field (Δu, Δv, Δd) that puts embodiment and object points on one pixel grid so relative motion marks an interaction, and the InSAttn/TransAgg readout that reads it at interaction regions and transition frames. I checked the paper's own taxonomy and the cited work, and no baseline gives the action online access to joint embodiment–object 3D motion. The real-robot evaluation is also better than most of this subfield: shared initial configurations, alternating trials, and full per-trial counts (165/200 ID and 156/200 OOD, vs 124/200 for the best baseline). That 16-point OOD gap is the strongest result in the paper.

Credit where it is due: the ablations are genuinely diagnostic. With conditions fixed, full GeoDelta beats depth by 2.5 points and flow by 0.9, object motion drives the gain, and InSAttn + TransAgg adds 2.6 over generic queries. The readout mechanism is not vacuous, and the probe's baseline side — released WAMs giving 1.8–2.3× attention to the non-interaction embodiment — is a legitimate motivation, even if Gnomon4D's own side partly restates its design.

The soft spots, in order of size. First, the headline margins on RoboDojo and VLABench do not isolate the mechanism from the model's inputs. The Action expert cross-attends directly to clean depth + mask conditions, and on RoboDojo those conditions are kinematically exact robot renders recomputed from live joint states at test time. The 2× RoboDojo margin and the +7.4 VLABench texture gain both admit input-level explanations: appearance-invariant handed-in geometry can produce texture robustness without any motion field. The paper never crosses condition-input presence with readout type, so the mechanism's share of its biggest numbers is unquantified. To be fair, the handed-in geometry on RoboDojo and VLABench covers the robot, not the objects, and the RoboTwin leg partially controls (X-WAM is RGB-D and trails by 4.11), but a referee should demand the missing cross experiment.

Second, the reader's DA3 worry is half off target. The real robot uses measured RealSense RGB-D depth, not DA3 monocular depth; DA3 enters only RoboDojo object GeoDelta with one fitted scale per episode (Eq. 8) and the Robot-30H mid-training. That estimation risk is real, and the paper discloses it in Limitations, but it should not be loaded onto the flagship real-robot numbers.

Third, minor: the RoboTwin margins are under a point with no error bars, and the full pipeline is unauditable — no code or checkpoints, and EgoManip-5000H is proprietary, so the pretraining contribution in Table 4c cannot be independently verified.

This deserves a serious referee — anyone working on world-action models or geometry-conditioned policies should engage with it. The right outcome is major revision asking for an input-matched control; if that lands, this is a solid contribution, and even as it stands the GeoDelta representation and the real-robot protocol are worth citing.

The strongest objection

The least-secure condition for the paper's central claim — that reading a jointly predicted GeoDelta at interaction regions and transition frames drives the benchmark wins — is isolation of that mechanism from the model's test-time inputs. In Sec. 3.4, the Action expert cross-attends directly to the clean condition tokens V_obs and G_cond (current depth + foreground mask); it does not need the predicted motion field to see geometry. The headline comparisons against RGB baselines therefore bundle three differences: (i) privileged condition inputs, (ii) the predicted GeoDelta plus focused readout, and (iii) pretraining. On RoboDojo, the conditions are kinematically exact robot renders recomputed at test time from live joint states (Sec. 4.1, E.1) — the strongest possible geometric prior — and the headline is a 2× success-rate gain (23.86 vs 11.92) whose Precision story (28.24 vs 20.42) is credited to GeoDelta encoding 'exact positions in both the image plane and depth.' On VLABench, the +7.4 unseen-texture margin is credited to 'GeoDelta encodes motion rather than appearance,' but the simulator-rendered depth+mask conditions are equally appearance-invariant, so that robustness admits a direct input-level explanation. The reader's weakest assumption is partly off target: the real-robot leg uses measured RGB-D depth, not DA3 (Sec. 4.1, F.2); DA3 enters only RoboDojo object GeoDelta (Eq. 8, one fitted scale per episode) and Robot-30H mid-training, and the paper discloses that estimation risk in its Limitations. The RoboTwin leg is partially controlled — X-WAM is RGB-D yet trails by 4.11 — and Table 4b shows InSAttn+TransAgg adds +2.6 over generic queries with conditions fixed, so the mechanism is not vacuous. What is missing is any experiment crossing condition-input presence with readout type; without it, the share of the flagship RoboDojo and VLABench margins creditable to reading predicted GeoDelta, rather than to handed-in geometry, is unquantified.

I read the paper's central claim as causal: GeoDelta jointly predicted with video and actions, read where and when interactions occur, and pretrained in a shared human-robot motion space, is the recipe behind the SOTA numbers. The three legs of evidence are internally consistent — every headline rate reproduces from the supplement's success counts (165/200 ID, 156/200 OOD; Table 10), the ablation increments are arithmetically coherent, and the real-robot protocol (shared initial configurations, alternating trials, per-task and per-shift counts) is among the more careful of its kind. The paper also deserves credit for disclosing the label-estimation pipeline in its Limitations and for reporting the condition inputs factually in Secs. 3.4 and 4.1. The load-bearing unmet condition is attribution: the Action expert attends directly to current depth+mask condition tokens, so the benchmark comparisons conflate 'reading predicted GeoDelta' with 'being handed clean geometry at each replan.' The two results the paper most leans on — the RoboDojo 2× margin and the VLABench unseen-texture robustness — are exactly the legs with no geometry-conditioned baseline, and on RoboDojo the conditions are rendered from live joint states. The reader's chosen weakest assumption (DA3 on the real robot) does not survive contact with Sec. 4.1 and F.2, which specify measured RGB-D depth for that leg; the DA3 rescaling concern applies to RoboDojo object GeoDelta (Eq. 8) and Robot-30H mid-training. Because Table 4b gives the readout +2.6 over generic queries with inputs held fixed, the mechanism has genuine internal support; the right conclusion is not rejection but a demand for an isolation experiment, which is precisely the conditionality the reader's verdict already encodes. I therefore keep the verdict at CONDITIONAL (UNCHANGED) while shifting the emphasized risk from label estimation to input isolation.

The referee's report

If the results hold, the paper's recipe — jointly predicting a 4D embodiment-object motion field and gating the action readout on interaction-salient regions and transition frames, then pretraining on human video in that shared motion space — is a genuinely new organizational principle for WAM design, and the real-robot OOD margin (78.0% vs 62.0% for the best baseline, Table 11) is a practically substantial result. The paper is unusually careful on several axes that often sink this literature: the real-robot protocol uses shared initial configurations, alternating trials, and complete success counts (Tables 10–11); GeoDelta is predicted rather than oracle-fed at inference; the Table 4 ablations are diagnostic (InSAttn adds 1.7 points over generic queries under identical conditions, a contrast the mechanism's construction does not guarantee); and the attention probe reports bootstrap intervals. I do not share the reader's concern about the real-robot depth channel, since Sec. F.2 documents measured sensor depth; the estimation concern is specific to RoboDojo object GeoDelta and to Robot-30H mid-training.

Pith's reading of the paper (a machine's summary, not the authors')

Gnomon4D claims that robot world-action models fail at precise manipulation not because they lack geometry, but because their action module reads the predicted future at the wrong place and time. The paper's answer is GeoDelta, a video-aligned 4D motion field that records, for every foreground pixel of the robot and the manipulated object, the image-plane displacement and depth change of the same physical point. Because embodiment and objects share one field, their relative motion marks where an interaction happens and when it transitions at grasp and release, and two attention mechanisms use that cue to focus action readout. The paper reports state-of-the-art success rates on RoboTwin 2.0, VLABench, and RoboDojo, and a real-robot edge that grows under visual shifts, from 82.5% in-distribution to 78.0% out-of-distribution while the best baseline falls from 75.5% to 62.0%. A reader would care because the paper's claim identifies a mechanism, motion-guided readout, rather than more data or bigger models, as the source of control accuracy and visual robustness.

What carries the argument. The load-bearing object is GeoDelta: a video-aligned 4D geometric motion field that stores at each foreground pixel the displacement $(\Delta u,\Delta v)$ and depth change $\Delta d$ of the same physical point between adjacent sampled frames, measured in the source camera so camera motion cancels, and encoded with signed $\mu$-law companding ($\mu=31$) into three RGB channels of the frozen video VAE. From GeoDelta features, lightweight heads predict per-token motion $\delta$ and embodiment/object role probabilities, and the relative-motion field $c = \|\delta - \bar\delta_e\|/(\|\delta\| + \|\bar\delta_e\|)$ measures how far a token departs from its local embodiment neighborhood, so it is near 1 at interactions and near 0 when things move together, with a frame-to-frame change that spikes at grasp and release. Interaction-Salient Attention (InSAttn) converts $c$ into a task-conditioned saliency bias for action tokens, Transition-Anchored Aggregation (TransAgg) scores frames by saliency-weighted change of $c$ and pools around transitions, and the resulting interaction tokens enter the Action expert residually. A shared denoising clock, with $\sigma_V(u)=1-u$ and $\sigma_G(u)=\sigma_A(u)=\max(1-2u,0)$, finishes GeoDelta and actions before video refinement, so the action reads the predicted motion as it forms.

Core claim. The paper's central claim is that the usefulness of a geometric future for robot control depends on how the action reads it, not only on what the future encodes. Probing three released world-action models on RoboTwin 2.0 shows their action attention allocates 1.82-2.30 times uniform attention to the non-interacting rest of the embodiment versus 1.18-1.52 times to the interaction region; Gnomon4D reverses this to 0.74 times and 2.42 times by generating GeoDelta, the per-pixel $(\Delta u,\Delta v,\Delta d)$ motion of embodiment and object points on one grid, and gating the action readout with the predicted embodiment-object relative-motion field. The paper argues, and reports evidence, that this mechanism together with GeoDelta as an embodiment-agnostic quasi-action that enables pretraining on 5,000+ hours of human egocentric video yields the best average result on all three simulation benchmarks, with 94.33% on RoboTwin 2.0, 62.5% on VLABench, and 23.86% on RoboDojo, and a real-robot 82.5% in-distribution and 78.0% out-of-distribution average. Ablations attribute the gain mainly to the object-motion component of GeoDelta and to the guided readout over generic queries, supporting the paper's thesis that a predicted future serves control through both what it encodes and how the action reads it.

The premise it rests on. The load-bearing premise is that estimated labels stand in for true 4D motion: on the real robot and in RoboDojo, GeoDelta is computed from monocular depth, segmentation masks, and optical flow rather than measured ground truth, so if those estimates are biased, the depth-change signal that drives the reported precision and out-of-distribution gains teaches the policy a wrong motion, an estimation issue the paper acknowledges in its Limitations section.

If the paper is right (Pith's inferences, not the authors')

If the central claim holds, any world-action model that keeps a video prior can gain precision and visual robustness by generating an embodiment-object 4D motion field at inference and reading it at interaction-salient regions and transition frames.

Human egocentric video becomes a usable pretraining source for robot policies without hand-pose retargeting, because GeoDelta is a shared motion space for hands and grippers; the paper reports a 2.1-point gain from EgoManip-5000H pretraining and a further 2.7 points from Robot-30H mid-training.

Robustness to unseen appearance follows from the mechanism: GeoDelta encodes motion and depth change rather than texture, and the paper's largest real-robot out-of-distribution margins, 18 points under unseen object appearance and added distractors, occur exactly where texture-invariant motion is unchanged.

Precision manipulation gains the most: on RoboDojo's Precision dimension the paper reports 28.24% success versus at most 20.42% for the baselines, consistent with GeoDelta encoding image-plane and depth motion jointly.

Action attention can be deliberately steered rather than left to emerge: probes show released WAMs underweight the interaction region, and Gnomon4D reverses that allocation, implying that attention probing is a usable diagnostic for where world-action models look.

Editorial inference: the where-and-when readout could transfer beyond tabletop manipulation to contact-rich assembly, locomotion with ground contact, or human-robot handover, wherever a foreground relative-motion field marks an interaction; the paper does not test these settings.

Editorial inference: the paper's ablations replace GeoDelta wholesale with depth or 2D flow but never decompose the $\Delta d$ channel within GeoDelta itself, so a sharper experiment that zeroes or corrupts $\Delta d$ while keeping $(\Delta u,\Delta v)$ would separate the contribution of line-of-sight motion from image-plane motion.

Editorial inference: because InSAttn and TransAgg act as a residual readout on predicted motion, they could plausibly be stacked with end-effector-distance biases or interaction-aware training objectives without conflict, a combination the paper does not explore.

Editorial inference: the real-robot and RoboDojo legs of the results rest on estimated labels, so a skeptic would want the real-robot protocol repeated with ground-truth depth or motion capture; the simulator legs already use exact rendered GeoDelta and isolate the mechanism in simulation.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper · Review history (2 versions)

The paper

Abstract

World-action models (WAMs) predict scene changes to guide robot actions, yet their RGB futures entangle state changes with appearance and lack 3D geometry. Recent WAMs add geometry as a training signal or a control interface, or build native 3D models, but decide what geometry to model rather than how the action should read it. Intuitively, control-relevant cues lie at embodiment–object interactions and their transitions, yet probing existing WAMs reveals that their action attention underweights these interactions in favor of the rest of the embodiment. To focus the action on these interactions, we present Gnomon4D, a WAM that jointly generates future video, robot actions, and GeoDelta, a video-aligned 4D geometric motion field recording the image-plane displacement and depth change of embodiment and object points. With the embodiment and objects in one field, their relative motion, which marks an interaction, can be read off. Guided by this motion and the task, Interaction-Salient Attention (InSAttn) selects where the action reads, and Transition-Anchored Aggregation (TransAgg) selects when, around grasp and release. As GeoDelta encodes surface motion rather than embodiment-specific controls, it serves as a human–robot quasi-action that links action-free pretraining on EgoManip-5000H, which we curate, to robot mid-training on Robot-30H. Gnomon4D leads on RoboTwin 2.0, VLABench, and RoboDojo, and outperforms real-robot baselines, especially under visual shifts. Project page: https://droliven.github.io/Gnomon4D.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-03

The authors' main claim, as the classifier picked it out 1.00

“Gnomon4D leads on RoboTwin~2.0, VLABench, and RoboDojo, and outperforms the strongest real-robot baseline by 7.0 and 16.0 percentage points under in-distribution and out-of-distribution settings, respectively.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.11 · states limitation 0.10 · human participants 0.11 · controlled comparison 0.94 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.97 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
World-action models (WAMs) use video-generation priors to jointly predict future observations and robot actions. background 1.00 0.04 0.02 asserted 1.00
Yet RGB futures entangle physical state changes with appearance, while frame-wise depth and point maps lack temporal correspondence. gap 0.73 0.10 0.02 asserted 1.00
Existing 4D approaches either keep motion implicit in visual features or translate predicted traces into actions through a separate interface. gap 0.82 0.04 0.02 asserted 1.00
We present Gnomon4D, a WAM that jointly generates future video, robot actions, and GeoDelta, a video-aligned 4D geometric motion field encoding the cross-frame image-plane displacement and depth change of embodiment and object points. contribution 0.96 0.95 0.14 asserted 1.00
By placing embodiment motion and object responses in a shared space, GeoDelta exposes their motion relations to the action expert at inference. method 0.83 0.81 0.07 asserted 1.00
Interaction-Salient Attention (InSAttn) uses these relations to focus spatial readout on task-relevant interaction regions, while Transition-Anchored Aggregation (TransAgg) aggregates evidence around changes in their relative motion, such as grasp and release. method 0.97 0.86 0.05 asserted 1.00
GeoDelta describes surface motion independently of embodiment-specific controls, providing a shared human--robot quasi-action for action-free pretraining on EgoManip-5000H followed by robot mid-training on Robot-30H. method 0.98 0.90 0.05 asserted 1.00
Gnomon4D leads on RoboTwin~2.0, VLABench, and RoboDojo, and outperforms the strongest real-robot baseline by 7.0 and 16.0 percentage points under in-distribution and out-of-distribution settings, respectively. result 1.00 0.96 0.78 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.87

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-03: 1 request, 6,470 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (17)
Figure 1 of the paper
Figure 1. Gnomon4D is a world-action model that jointly generates future video, robot actions, and 4D geometric motion (GeoDelta),
Figure 2 of the paper
Figure 2. How 4D motion reaches the action. Existing methods keep 4D motion implicit (a) or decouple it from con￾trol (b). Gnomon4D generates GeoDelta jointly with the action and, through InSAttn and TransAgg, uses it to decide where and when the action should focus (c). the action, and place embodiment and object motion in a shared spatiotemporal space. Accordingly, we introduce Gnomon4D, a WAM built on a Mixture-of-Transformers architecture [33] that jointly generates future video, robot actions, and a video-aligned 4D geometric motion, i.e., GeoDelta (Figs. 1 and 2c). For each foreground pi
Figure 3 of the paper
Figure 3. Overview of Gnomon4D. (a) Video, GeoDelta, and Action experts jointly generate future video, GeoDelta, and actions through joint attention, and an interaction readout is added residually to the Action expert. (b) GeoDelta and Action share a denoising clock that finishes ahead of Video, so actions are ready before the video is refined. (c) Embodiment–object motion relations give task-conditioned saliency, with which InSAttn selects interaction regions and TransAgg aggregates transition frames. Image tiles show RoboTwin examples. flow-matching path [37] x m σm = (1 − σm)x m + σmϵ m…
Figure 4 of the paper
Figure 4. GeoDelta supervision in the four domains. Each row shows four head-camera frames of one training demonstration with its
Figure 5 of the paper
Figure 5. Real-robot success rate. (a) SR per task and its average
Figure 6 of the paper
Figure 6. Where does action attention go? Attention enrichment over 50 event windows, one per RoboTwin 2.0 task, on the interac￾tion region, formed by the manipulated object and the embodiment within 3 cm of contact, and on the rest of the embodiment. Enrich￾ment is the attention mass on a region divided by its area fraction; the dashed line marks uniform attention (1×). The baselines favor the embodiment as a whole, whereas Gnomon4D concentrates on the interaction region. Error bars are 95% bootstrap intervals over tasks. A.2. Results and Visualization The baselines favor the embodiment as a…
Figure 7 of the paper
Figure 7. Action-conditioned spatial readout. Three selected interactions from the 50-window probe; population statistics are given in the text. All models are shown on the future latent block containing the event; Fast-WAM is omitted because its action reads only the observed frame. Gnomon4D shows the interaction rollout of the benchmark checkpoint. Reference regions are pooled within that block. Cyan contours mark contact-overlap tokens, and dashed white contours mark motion-associated objects. Colors show attention relative to uniform within each displayed frame on one piecewise log scale…
Figure 8 of the paper
Figure 8. Attention visibility of the joint softmax. Rows are queries and columns are keys. Colored cells are visible, gray cells are masked, Itti ℓ Action expert ttn Action Chunk UMT5 0 0.5 1 0
Figure 9 of the paper
Figure 9. Intuition behind relation-guided interaction readout. Left: relative motion decreases when the embodiment and object move together. Middle: InSAttn converts this relation into task-conditioned spatial saliency. Right: TransAgg emphasizes frames where the relation changes. Here ρ = ρ e + ρ o denotes foreground probability. The motion and role heads act on GeoDelta features, su￾pervised by L1 loss on mean token displacement and cross￾entropy on role labels. All geometric quantities are com￾puted for every future frame n and token i. The weights wij form a fixed 5×5 Gaussian…
Figure 10 of the paper
Figure 10. Curation of EgoManip-5000H. A sub-pool of the library passes real-scene filtering and an egocentric-manipulation classifier,
Figure 11 of the paper
Figure 11. Composition of EgoManip-5000H. Shares of Kimi-K2.6 scene, action, and attribute labels among labeled clips of the full set
Figure 12 of the paper
Figure 12. Training windows of EgoManip-5000H across scenes. Each example shows frames
Figure 13 of the paper
Figure 13. Robot-30H training windows from different scenes. Each example shows frames
Figure 14 of the paper
Figure 14. Robot-30H overview. (a) Hours of the 10,500 episodes
Figure 15 of the paper
Figure 15. Real-robot platform. A UR5e arm with a 1-DoF paral
Figure 16 of the paper
Figure 16. Real-robot tasks and visual OOD conditions. Each row is labeled with its task instruction, and the first five frames show
Figure 17 of the paper
Figure 17. Geometric supervision on Drawer Storage, Color Sorting, and Block Stacking. From left to right: RGB, recorded camera

Figures are the authors', reproduced from the paper.

References (95)
  1. [1] Awomo-PhysicalRSI Team, Danjiao Ma, Enhui Ma, Haohan Liu, Heng Jia, Hui Shan, Jianhua Xu, Jiahuan Zhang, Jiangdi Xu, Kaiwen Guo, Kaicheng Yu, Linwei Zhang, Liyang Jin, Maochun Luo, Pengyao Niu, Shiwen Li, Shuangyu Feng, Tong Zhang, Tianheng Wang, Xin Wang, Xiangru Huang, Yongq... on PithPith's reviewarXiv
  2. [2] Sami Azirar, Enrico Pallotta, Jan Nogga, J¨urgen Gall, Sven Behnke, and Hermann Blum. ContactFlow: A video ac- tion conditioning that transfers across embodiments.arXiv preprint arXiv:2607.26579, 2026. 3 on PithPith's reviewarXiv
  3. [3] Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhao- hai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Jun... on PithPith's reviewarXiv
  4. [4] Jai Bardhan, Josef Sivic, and Vladimir Petrik. DepthWorld: 3D world model for robot manipulation.arXiv preprint arXiv:2610.08780, 2026. 2, 3 on PithPith's reviewarXiv
  5. [5] Hongzhe Bi, Hengkai Tan, Shenghao Xie, Zeyuan Wang, Shuhe Huang, Haitian Liu, Ruowen Zhao, Yao Feng, Chen- dong Xiang, Yinze Rong, Hongyan Zhao, Hanyu Liu, Zhizhong Su, Lei Ma, Hang Su, and Jun Zhu. Motus: A unified latent action world model. InCVPR, pages 35101– 35113, 2026. 7, 3 on Pith
  6. [6] Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Robert Equi, Chelsea Finn, Niccolo Fusai, Manuel Y . Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey L... on Pith
  7. [7] Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoub- hik Debnath, Ronghang Hu, Didac Suris Coll-Vinent, Chai- tanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, An- drew Huang, Jie Lei, Tengyu Ma, Baishan Guo, Arpit Kalla, Markus Marks, Joseph Greer, Meng Wang, Peize Sun, Rom... on Pith
  8. [8] Ying Chai, Litao Deng, Ruizhi Shao, Jiajun Zhang, Kangchen Lv, Liangjun Xing, Xiang Li, Hongwen Zhang, and Yebin Liu. GAF: Gaussian action field as a 4D representation for dynamic world modeling in robotic manipulation. InICRA, on Pith
  9. [9] Jialei Chen, Kai Wang, Kang Chen, Shuaihang Chen, Feng Gao, Wenhao Tang, Zhiyuan Li, Weilin Liu, Zhuyu Yao, Boxun Li, Yuanbo Xu, and Chao Yu. LaW AM: Latent world action models for efficient dynamics-aware robot policies. arXiv preprint arXiv:2606.15768, 2026. 3 on PithPith's reviewarXiv
  10. [10] Tianxing Chen, Yue Chen, Zixuan Li, Junyuan Tang, Kailun Su, Haoran Lu, Weijie Wan, Baijun Chen, Songling Liu, Haowen Yan, Honghao Su, Zhiyang Dou, Kaixuan Wang, Dandan Zhang, Yunze Liu, Yan Qin, Qiwei Liang, Qiwei Wu, Zijian Lin, Wenwei Lin, Yuran Wang, Minghua He, Tianshu Wu... on PithPith's reviewarXiv
  11. [11] Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, Yiheng Ge, Zhenyu Gu, Weiliang Deng, Yubin Guo, Tian Nian, Xuan- bing Xie, Qiangyu Chen, Kailun Su, Tianling Xu, Guodong Liu, Mengkang Hu, Huan-Ang Gao, Kaixuan Wang, Zhix- u... on Pith
  12. [12] Yixiang Chen, Peiyan Li, Yuan Xu, Qisen Ma, Jiabing Yang, Kai Wang, Jianhua Yang, Dong An, He Guan, Gaoteng Liu, Jianlou Si, Jun Huang, Jing Liu, Nianfeng Liu, Yan Huang, and Liang Wang. FlowWAM: Optical flow as a unified ac- tion representation for world action models.arXiv p... on PithPith's reviewarXiv
  13. [13] Jintao Cheng, Weibin Li, Haozhe Wang, Gang Wang, Yipu Zhang, Xiaoyu Tang, Jin Wu, Xieyuanli Chen, Yunhui Liu, and Wei Zhang. Training-free interaction-aligned visual token pruning for efficient embodied manipulation.arXiv preprint arXiv:2603.22991, 2026. 3 on PithPith's reviewarXiv
  14. [14] Karthik Dharmarajan, Wenlong Huang, Jiajun Wu, Li Fei-Fei, and Ruohan Zhang. Dream2Flow: Bridging video generation and open-world manipulation with 3D object flow. InICRA, on Pith
  15. [15] Dyna Robotics. Dyna-2: A 1-million-hour scaling law for world-action models. https://www.dyna.co/dyna- 2, 2026. Technical report. 3 on Pith
  16. [16] Qiwen Gu, Jifan Li, Bingjie Gao, Rui Chen, Jing Tang, Xi- angxiang Chu, and Junqiao Zhao. The right future for action: Learning action-relevant predictive states in world action mod- els.arXiv preprint arXiv:2609.23369, 2026. 3, 7 on PithPith's reviewarXiv
  17. [17] Jun Guo, Qiwei Li, Peiyan Li, Zilong Chen, Nan Sun, Yifei Su, Heyun Wang, Yuan Zhang, Xinghang Li, and Huaping Liu. Unified 4D world action modeling from video priors with asynchronous denoising.arXiv preprint arXiv:2604.26694, on PithPith's reviewarXiv
  18. [18] Yanjiang Guo, Haodong Yan, Zhide Zhong, Zhongru Zhang, Qingyuan Yang, Qingzhou Lu, Xiaoyu Chen, Yen-Jen Wang, Shuying Deng, Chenghan Yang, Puzhen Yuan, Chenxin Liu, Tun Ban, Xiang Zhu, Yichen Liu, Kun Feng, Haoang Li, and Jianyu Chen. Video Prediction Policy 2: Predict better,... on PithPith's reviewarXiv
  19. [19] Jisang Han, Seonghu Jeon, Jaewoo Jung, Ren ´e Zurbr ¨ugg, Honggyu An, Tifanny Portela, Marco Hutter, Marc Polle- feys, Seungryong Kim, and Sunghwan Hong. Geometric Action Model for robot policy learning.arXiv preprint arXiv:2606.17046, 2026. 2, 3 on PithPith's reviewarXiv
  20. [20] Yifan Han, Jianxiang Liu, Haoyu Zhang, Yuqi Gu, Yun- han Guo, and Wenzhao Lian. BridgeACT: Bridging human demonstrations to robot actions via unified tool-target affor- dances.arXiv preprint arXiv:2604.23249, 2026. 3 on PithPith's reviewarXiv
show 75 more
  1. [21] Yifan Han, Litao Liu, Yuqi Gu, Ye Lu, Hanqing Wang, Sid- ney Wai, Ishaan Myrie, Qi Zhang, Jingjin Yu, and Gen Li. A4A: Cross-embodiment transfer of action-oriented 4D affordances from human demonstrations.arXiv preprint arXiv:2609.05892, 2026. 3 on PithPith's reviewarXiv
  2. [22] Wenlong Huang, Yu-Wei Chao, Arsalan Mousavian, Ming-Yu Liu, Dieter Fox, Kaichun Mo, and Li Fei-Fei. PointWorld: Scaling 3D world models for in-the-wild robotic manipulation. InCVPR, pages 20765–20779, 2026. 3 on Pith
  3. [23] Adam Hung, Bardienus P. Duisterhof, Deva Ramanan, and Jeffrey Ichnowski. Modality-autoregressive world-action models.arXiv preprint arXiv:2609.17524, 2026. 3 on PithPith's reviewarXiv
  4. [24] Physical Intelligence, Bo Ai, Ali Amin, Raichelle Aniceto, Ashwin Balakrishna, Greg Balke, Kevin Black, George Bokin- sky, Shihao Cao, Thomas Charbonnier, Vedant Choudhary, Foster Collins, Ken Conley, Grace Connors, James Darpinian, Karan Dhabalia, Maitrayee Dhaka, Jared DiCar... on PithPith's reviewarXiv
  5. [25] Haoyi Jiang, Liu Liu, Xinjiang Wang, Zhihao Sun, Zequn Chen, Sen Wang, Xinjie Wang, Xia Chen, Jingfeng Yao, Weiheng Zhao, Shanglin Yuan, Zhizhong Su, Wei Sui, Wenyu Liu, and Xinggang Wang. Rethinking representations for world-action modeling.arXiv preprint arXiv:2609.38163, on PithPith's reviewarXiv
  6. [26] Moo Jin Kim, Yihuai Gao, Tsung-Yi Lin, Yen-Chen Lin, Yunhao Ge, Grace Lam, Percy Liang, Shuran Song, Ming-Yu Liu, Chelsea Finn, and Jinwei Gu. Cosmos Policy: Fine- tuning video models for visuomotor control and planning. In ICLR, 2026. 2, 3 on Pith
  7. [27] Kimi Team. Kimi K2.6. https://huggingface.co/ moonshotai/Kimi-K2.6, 2026. 5 on Pith
  8. [28] Seungjae Lee, Yoonkyo Jung, Inkook Chun, Yao-Chih Lee, Zikui Cai, Hongjia Huang, Aayush Talreja, Tan Dao, Yongyuan Liang, Jia-Bin Huang, and Furong Huang. Trace- Gen: World modeling in 3D trace space enables learning from cross-embodiment videos. InCVPR, pages 20721–20731, on Pith
  9. [29] Seungjae Lee, Yoonkyo Jung, Jusuk Lee, Jonghun Shin, Amir Hossein Shahidzadeh, Yao-Chih Lee, H. Jin Kim, Jia- Bin Huang, and Furong Huang. µ0: A scalable 3D interaction- trace world model.arXiv preprint arXiv:2606.13769, 2026. 2, 3 on PithPith's reviewarXiv
  10. [30] Baoyu Li, Xinchen Yin, Mengying Lin, Yixin Zhang, and Danfei Xu. EgoWAM: World action models beyond pix- els with in-the-wild egocentric human data.arXiv preprint arXiv:2607.08436, 2026. 3 on PithPith's reviewarXiv
  11. [31] Mingxin Li, Yanzhao Zhang, Dingkun Long, Keqin Chen, Sibo Song, Shuai Bai, Zhibo Yang, Pengjun Xie, An Yang, Dayiheng Liu, Jingren Zhou, and Junyang Lin. Qwen3-VL- Embedding and Qwen3-VL-Reranker: A unified framework for state-of-the-art multimodal retrieval and ranking.arXiv ... on PithPith's reviewarXiv
  12. [32] Xiang Li, Yupeng Zheng, Songen Gu, Huailiang Ma, Feng Yu, Yuhang Zheng, Xian Nie, Shanshuai Yuan, Yujie Zang, Weize Li, Shuai Tian, Moyang Liu, Ya-Qin Zhang, and Wen- chao Ding. Latent action as intention enables efficient fu- ture imagination for world action models.arXiv pre... on PithPith's reviewarXiv
  13. [33] Ying Li, Xiaobao Wei, Jiajun Cao, Hao Wang, Xiaowei Chi, Chengyu Bai, Qianpu Sun, Jiajun Li, Xiaojie Zhang, Peidong Jia, Jian Tang, Sirui Han, and Shanghang Zhang. WAM4D: Fast 4D world action model via spatial register tokens.arXiv preprint arXiv:2606.14048, 2026. 2, 3, 7 on PithPith's reviewarXiv
  14. [34] Zhiyi Li, Peilin Wu, Xiaoshen Han, Ruojin Cai, and Yilun Du. Structured 4D latent predictive model for robot planning. InICML, pages 71216–71230, 2026. 3 on Pith
  15. [35] Weixin Liang, Lili Yu, Liang Luo, Srini Iyer, Ning Dong, Chunting Zhou, Gargi Ghosh, Mike Lewis, Wen tau Yih, Luke Zettlemoyer, and Xi Victoria Lin. Mixture-of-Transformers: A sparse and scalable architecture for multi-modal foundation models.TMLR, 2025. 2, 3, 4 on Pith
  16. [36] Haotong Lin, Sili Chen, Jun Hao Liew, Donny Y . Chen, Zhenyu Li, Yang Zhao, Sida Peng, Hengkai Guo, Xiaowei Zhou, Guang Shi, Jiashi Feng, and Bingyi Kang. Depth Any- thing 3: Recovering the visual space from any views. InICLR, pages 141261–141285, 2026. 6, 5 on Pith
  17. [37] Jianman Lin, Shailesh Shailesh, Zhongyi Luo, and Jiafei Duan. Breaking the vision-action shortcut: Latent interface training for generalizable robotics foundation models.arXiv preprint arXiv:2609.12641, 2026. 2 on PithPith's reviewarXiv
  18. [38] Yihan Lin, Jiawei He, Shifeng Bao, Chen Zhao, Yang Li, Xiaobo Wang, Yan Wang, Cheng Chi, and Jing Zhang. JEPA-WAM: Learning vision-language-action poli- cies with joint-embedding world modeling.arXiv preprint arXiv:2608.09381, 2026. 3 on PithPith's reviewarXiv
  19. [39] Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maxi- milian Nickel, and Matt Le. Flow matching for generative modeling. InICLR, 2023. 3 on Pith
  20. [40] Yicheng Liu, Zibin Dong, Baijun Ye, Tianyuan Yuan, Tao Jiang, Anqi Yang, Shicheng Cao, Haonan Liu, Yue Sun, Zihan Guo, Xiao Liu, Dong Ke, Changxun Pan, Chenru Wu, Tailai Cheng, Xiaoshu Ren, Xinlei Zhang, Jianning Cui, Zijie Zhao, Haoyu Zhang, Kaiming Xu, Haodong Yang, Bowen Zh... on PithPith's reviewarXiv
  21. [41] Yuanzhe Liu, Jingyuan Zhu, Yuchen Mo, Gen Li, Xu Cao, Jin Jin, Yifan Shen, Zhengyuan Li, Tianjiao Yu, Wenzhen Yuan, Fangqiang Ding, and Ismini Lourentzou. PALM: Progress-aware policy learning via affordance reasoning for long-horizon robotic manipulation. InCVPR, pages 28096– ... on Pith
  22. [42] Yunfan Lou, Xiaowei Chi, Xiaojie Zhang, Zezhong Qian, Chengxuan Li, Rongyu Zhang, Yaoxu Lyu, Guoyu Song, Chuyao Fu, Haoxuan Xu, Pengwei Wang, and Shanghang Zhang. Predicting what matters: Robust generalist robot policy learning via future semantic mask. InICML, pages 80599–806... on Pith
  23. [43] Hao Luo, Wanpeng Zhang, Yicheng Feng, Sipeng Zheng, Haiweng Xu, Chaoyi Xu, Ziheng Xi, Yuhui Fu, and Zongqing Lu. Being-H0.7: A latent world-action model from egocentric videos.arXiv preprint arXiv:2605.00078, 2026. 3 on PithPith's reviewarXiv
  24. [44] Haoxiang Ma, Junhao Cai, Xiaoxu Xu, Hao Li, Yuyin Yang, Yang Tian, Jiafei Cao, Hongrui Zhu, Zherui Qiu, Zhax- izhuoma, Yuqiang Yang, Jiaqi Peng, Xueyuan Wei, Yangkun Zhu, Jiahao Jiang, Xing Gao, Hanqing Wang, Feng Yuan, Kailin Li, Xueyue Zhu, Tai Wang, Yan Ding, Jiangmiao Pang... on PithPith's reviewarXiv
  25. [45] Juncheng Ma, Jianxin Bi, Yufan Deng, Xuanran Zhai, Kewei Zhang, Ye Huang, Bo Liang, Shukai Gong, Jiankai Tu, Xi- aotian Tang, Jiaxin Li, Kaiqi Chen, Duomin Wang, Yuqi Wang, Bingyi Kang, Eric Huang, Zhiyang Dou, Zhen Dong, Enze Xie, Wojciech Matusik, Tat-Seng Chua, and Daquan Z... on PithPith's reviewarXiv
  26. [46] Xingyu Miao, Zizun Li, Baole Fang, Kaiwen Song, Tenghui Wang, Hanxue Zhang, Yating Wang, Xudong Li, Yuping He, Xueyuan Wei, Chao Gao, Xijie Yang, Yingxiang Xu, Kerui Ren, Wenqi Guo, Jianjun Zhou, Xinzhe Wang, Weiguang Zhao, Ni Yang, Zetao Cai, Yufei Xue, Hengjie Li, Zeyu He, Y... on PithPith's reviewarXiv
  27. [47] MMLab & PhysicalRSI contributors. Physical RSI 1.0: Re- cursive self-harness for scaling embodied skills. https: //mmlab.hk/research/PhysicalRSI , 2026. Pre- view release. 2 on Pith
  28. [48] Chaojun Ni, Xinyu Zhou, Yukun Zhou, Jingyu Liu, Xiaofeng Wang, Zheng Zhu, Yang Wang, Qiuping Deng, Yun Ye, Hao Li, Zhichao Liu, Jindi Lv, Boyuan Wang, Guosheng Zhao, Guan Huang, Min Cao, and Wenjun Mei. GigaWorld-Policy: An efficient action-centered world–action model. InECCV,... on Pith
  29. [49] NVIDIA, Johan Bjorck, Fernando Casta ˜neda, Nikita Cher- niadev, Xingye Da, Runyu Ding, Linxi ”Jim” Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, Joel Jang, Zhenyu Jiang, Jan Kautz, Kaushil Kundalia, Lawrence Lao, Zhiqi Li, Zongyu Lin, Kevin Lin, Guilin Liu, Edith Llon... on PithPith's reviewarXiv
  30. [50] Chunghyun Park, Beomjun Kim, Seungcheol Park, Heese- ung Kwon, Yashu Shukla, Seunghoon Sim, Jinwoo Shin, and Minsu Cho. PointW AM: 3D world action modeling for dexter- ous robotic manipulation.arXiv preprint arXiv:2610.02840, on PithPith's reviewarXiv
  31. [51] Rolandos Alexandros Potamias, Jinglei Zhang, Jiankang Deng, and Stefanos Zafeiriou. WiLoR: End-to-end 3D hand localization and reconstruction in-the-wild. InCVPR, pages 12242–12254, 2025. 5 on Pith
  32. [52] Jingjing Qian, Boyao Han, Chen Shi, Lei Xiao, Long Yang, Shaoshuai Shi, and Li Jiang. GeoPredict: Leveraging predic- tive kinematics and 3D Gaussian geometry for precise VLA manipulation. InCVPR, pages 13529–13539, 2026. 3 on Pith
  33. [53] Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Jiayuan Gu, Zhigang Wang, Yan Ding, Bin Zhao, Dong Wang, and Xuelong Li. SpatialVLA: Exploring spatial repre- sentations for visual-language-action models. InRSS, 2025. 2 on Pith
  34. [54] Javier Romero, Dimitrios Tzionas, and Michael J. Black. Embodied hands: Modeling and capturing hands and bodies together.ACM TOG, 36(6):245:1–245:17, 2017. 5 on Pith
  35. [55] Xinyu Shao, Keru Zhou, Guowei Huang, Yajun Gao, Tong- tong Cao, and Xiu Li. KAM-WM: Kinematic affordance maps from latent world models for robot manipulation.arXiv preprint arXiv:2607.04652, 2026. 3 on PithPith's reviewarXiv
  36. [56] Yichao Shen, Fangyun Wei, Zhiying Du, Yaobo Liang, Yan Lu, Jiaolong Yang, Nanning Zheng, and Baining Guo. VideoVLA: Video generators can be generalizable robot ma- nipulators. InNeurIPS, pages 95597–95621, 2025. 2, 3 on Pith
  37. [57] Zijian Song, Qichang Li, Jiawei Zhou, Zhenlong Yuan, Tian- shui Chen, Liang Lin, and Guangrun Wang. Robotic ma- nipulation is vision-to-geometry mapping: Vision-geometry backbones over language and video models.arXiv preprint arXiv:2604.12908, 2026. 2, 3 on PithPith's reviewarXiv
  38. [58] Ziying Song, Yuchen Liu, Zhuoran Xu, Ziyang Liu, Jian Jin, Jiangtao Su, Haibao Yu, Lei Yang, and Yuanpei Chen. KineWorld: Action-induced transport fields for embodied world modeling.arXiv preprint arXiv:2610.06349, 2026. 3 on PithPith's reviewarXiv
  39. [59] Rongze Tang, Jianjie Fang, Zhaolu Wang, Ziyou Wang, Xvyuan Liu, Haisheng Su, Xin Zhang, Wei Wu, Chen Gao, Yong Li, and Zhibo Chen. IMPACT: Attention is the interac- tion map for scalable interaction-aware world model training. arXiv preprint arXiv:2609.00161, 2026. 3 on PithPith's reviewarXiv
  40. [60] GigaWorld Team, Angen Ye, Angyuan Ma, Boyuan Wang, Chaojun Ni, Fangzheng Ye, Guan Huang, Guo Li, Guosheng Zhao, Haodong Yan, Hengtao Li, Jiwen Lu, Kai Wang, Ming- ming Yu, Qitang Hu, Qiuping Deng, Songling Liu, Xiaoyu Tian, Xiaofeng Wang, Xinyu Zhou, Xiuwei Xu, Xinze Chen, Yan... on PithPith's reviewarXiv
  41. [61] Xiaomi Robotics Team, Jun Guo, Piaopiao Jin, Jason Li, Peiyan Li, Yingyan Li, Futeng Liu, Wanli Peng, Optimus Qin, Yifei Su, Nan Sun, Qiao Sun, Runze Suo, Heyun Wang, Yunhong Wang, Rujie Wu, Caoyu Xia, Lina Zhang, Jack Zhao, Guoliang Chen, Wenlong Chen, Xinze He, Bin Li, Qing ... on PithPith's reviewarXiv
  42. [62] Zachary Teed and Jia Deng. RAFT: Recurrent all-pairs field transforms for optical flow. InECCV, pages 402–419, 2020. 6, 5, 8 on Pith
  43. [63] Yuxuan Tian, Yurun Jin, Bin Yu, Yukun Shi, Hao Wu, Chi Harold Liu, Kai Chen, and Cong Huang. STARRY: Spatial-temporal action-centric world modeling for robotic manipulation.arXiv preprint arXiv:2604.26848, 2026. 3, 7 on PithPith's reviewarXiv
  44. [64] Mutian Tong, Han Jiang, Qiao Feng, Lingjie Liu, and Jiatao Gu. PointAction: 3D points as universal action represen- tations for robot control.arXiv preprint arXiv:2606.03943, on PithPith's reviewarXiv
  45. [65] Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianx- iao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jin- gren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, P... on PithPith's reviewarXiv
  46. [66] Alex Wang, Zhiwei Dong, Qicheng Bai, Chenshi Zhang, Yujie Yi, Guang Dai, Yong Liu, and Mengmeng Wang. DynBridge: Bridging imagination and control through interaction dynam- ics for robot manipulation. InCVPR, pages 22423–22432, on Pith
  47. [67] Chenyi Wang, Xinkai Wang, Bokai Lin, Jialin Tian, Fucheng Zhang, Cewu Lu, and Lixin Yang. Track4Action: Distilling world-centric 3D tracker into vision-language-action policies. arXiv preprint arXiv:2608.03727, 2026. 3 on PithPith's reviewarXiv
  48. [68] Jianyuan Wang, Minghao Chen, Shangzhan Zhang, Nikita Karaev, Johannes Sch ¨onberger, Patrick Labatut, Piotr Bo- janowski, David Novotny, Andrea Vedaldi, and Christian Rupprecht. VGGT-Ω. InCVPR, pages 21486–21499, 2026. 5 on Pith
  49. [69] Ruixiang Wang, Yongyi Su, Wenlve Zhou, Bo Yue, Hengyan Liu, Dekun Lu, Yuxin Tian, Yihan Fang, Zerui Wu, Xing Hu, Jietao Chen, Yong Guo, Ziyan He, Junbin Yuan, Guiliang Liu, Xiaofen Xing, and Kui Jia. UNITAS: A 3D-native world action model for embodied manipulation.arXiv prepri... on PithPith's reviewarXiv
  50. [70] Yuran Wang, Siqiao Huang, Mingleyang Li, Chenhao Zhang, Jiaqi Liang, Weiyang Jin, Yue Chen, Xuemin Chi, Donghao Zhou, Qize Yu, Yu-Kai Wang, Yuhan Rui, Shenzhe Yao, Zhen Yuan, Zhenhao Shen, Kefei Zhu, Zijie Zhu, Ning Gao, Xi- aowei Chi, Guanqi He, Shanghang Zhang, Hao Dong, Lin... on PithPith's reviewarXiv
  51. [71] Lei Xiao, Jifeng Li, Juntao Gao, Feiyang Ye, Yan Jin, Jingjing Qian, Jing Zhang, Yong Wu, and Xiaoyuan Yu. A V A-VLA: Improving vision-language-action models with active visual attention. InCVPR, pages 13453–13463, 2026. 3 on Pith
  52. [72] Ziyin Xiong, Nikos Gkanatsios, Moritz Reuss, and Katerina Fragkiadaki. GeomVLA: Unifying scene, motion, and action in 3D.arXiv preprint arXiv:2609.13812, 2026. 3 on PithPith's reviewarXiv
  53. [73] Chongyang Xu, Haipeng Li, Shen Cheng, Haoqiang Fan, Zil- iang Feng, and Shuaicheng Liu. Action-geometry prediction with 3D geometric prior for bimanual manipulation. InCVPR, pages 35036–35046, 2026. 3 on Pith
  54. [74] Fuxiang Yang, Donglin Di, Lulu Tang, Xuancheng Zhang, Lei Fan, Hao Li, Wei Chen, Tonghua Su, and Baorui Ma. Chain of World: World model thinking in latent motion. In CVPR, pages 6675–6684, 2026. 3 on Pith
  55. [75] Fan Yang, Yuting Su, Xiaobo Wang, Yuncheng You, Fugui Fan, Yuting Wu, Minghui Wu, Chenxu Zhao, JiaHong Ning, and Peiguang Jing. LiLa-W AM: Lightweight latent reasoning world-action model for robotic manipulation.arXiv preprint arXiv:2608.03701, 2026. 3 on PithPith's reviewarXiv
  56. [76] Jiarui Yang, Jiale Zhange, Jiawei Li, Hang Guo, Wen Huang, Jinpeng Wang, Peidong Liu, and Shu-Tao Xia. LAWM-3D: Learning 3D-aware latent actions from human videos for generalizable robot world models.arXiv preprint arXiv:2608.05706, 2026. 3 on PithPith's reviewarXiv
  57. [77] Lishan Yang, Wenxuan Song, Xi Wang, Pingyue Sheng, Zheng Fang, Ziyang Zhou, Junjie He, Haodong Yan, Ji- ayi Chen, Nan Sun, Qiao Sun, Pengwei Wang, Lingqiao Liu, Yan Wang, Yuxiang Gao, Feras Dayoub, and Haoang Li. 4D-W AM: Infusing spatiotemporal awareness into world action mod... on PithPith's reviewarXiv
  58. [78] Yi Yang, Xueqi Li, Yiyang Chen, Jin Song, Yihan Wang, Zipeng Xiao, Jiadi Su, You Qiaoben, Pengfei Liu, and Zhijie Deng. Mantis: A versatile vision-language-action model with disentangled visual foresight. InCVPR, pages 42505–42515, on Pith
  59. [79] Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, Shenyuan Gao, Sihyun Yu, George Kurian, Suneel Indupuru, You Liang Tan, Chuning Zhu, Jiannan Xiang, Ayaan Malik, Kyungmin Lee, William Liang, Nadun Ranawaka, Jiasheng Gu, Yinzhen Xu, Guanzhi Wang, Fengyuan Hu, Avnish Narayan, Johan Bjor... on PithPith's reviewarXiv
  60. [80] Jiadi You, Qize Yu, Yue Chen, Minghong Cai, Zhide Zhong, Yuran Wang, Bowen Ping, Jiaqi Liang, Zhenhao Shen, Haodong Yan, Yinchuan Li, Ruihai Wu, Xiaojuan Qi, and Yingcong Chen. AffordanceWAM: Affordance-aware joint world-action modeling for robot manipulation.arXiv preprint ar... on PithPith's reviewarXiv
  61. [81] Haoqi Yuan, Zhixuan Liang, Anzhe Chen, Ye Wang, Haoyang Li, Pei Lin, Yiyang Huang, Zixing Lei, Tong Zhang, Jiazhao Zhang, Jie Zhang, Jingyang Fan, Gengze Zhou, Qihang Peng, Chenxu Lv, Xiaoyue Chen, An Yang, Fei Huang, Junyang Lin, Dayiheng Liu, Jingren Zhou, Chenfei Wu, and Xi... on PithPith's reviewarXiv
  62. [82] Shanglin Yuan, Weiheng Zhao, Xin Shi, Haoyi Jiang, Xianda Guo, Liu Liu, Wenyu Liu, Wei Sui, and Xinggang Wang. DreamW AM: Beyond RGB future prediction for world action models.arXiv preprint arXiv:2608.04996, 2026. 2 on PithPith's reviewarXiv
  63. [83] Tianyuan Yuan, Zibin Dong, Yicheng Liu, and Hang Zhao. Fast-WAM: Do world action models need test-time future imagination?arXiv preprint arXiv:2603.16666, 2026. 2, 3, 6, 7, 10 on PithPith's reviewarXiv
  64. [84] Gehao Zhang, Weikai Huang, Shailesh Shailesh, Yiyan Peng, Jiafei Duan, and Ranjay Krishna. Grounded Action Model: 3D grounding as a foundation for robotics.arXiv preprint arXiv:2609.23863, 2026. 3 on PithPith's reviewarXiv
  65. [85] Jianjun Zhang, Jian Zhu, Taiyi Su, Chong Ma, Zitai Huang, Yi Xu, and Hanli Wang. Learning 4D geometric priors for inference-efficient world action models.arXiv preprint arXiv:2607.05468, 2026. 2, 3, 7 on PithPith's reviewarXiv
  66. [86] Qihang Zhang, Lin Li, Luyao Zhang, Shuai Yang, Yim- ing Luo, Shuaiting Li, Ruilin Wang, Junke Wang, Jiahao Shao, Gangwei Xu, Jiaming Zhou, Yishu Shen, Yudong Jin, Fangyi Xu, Shuailei Ma, Jiaqi Liao, Guanxing Lu, Zifan Shi, Yongkun Wen, Yujie Zhao, Weixuan Tang, Xinyang Wang, C... on PithPith's reviewarXiv
  67. [87] Shiduo Zhang, Zhe Xu, Peiju Liu, Xiaopeng Yu, Yuan Li, Qinghui Gao, Zhaoye Fei, Zhangyue Yin, Zuxuan Wu, Yu- Gang Jiang, and Xipeng Qiu. VLABench: A large-scale benchmark for language-conditioned robotics manipulation with long-horizon reasoning tasks. InICCV, pages 11142– 111... on Pith
  68. [88] Haoyu Zhao, Xingyue Zhao, Siteng Huang, Xin Li, Deli Zhao, and Zhongyu Li. RynnWorld-4D: 4D embodied world models for robotic manipulation.arXiv preprint arXiv:2607.06559, on PithPith's reviewarXiv
  69. [89] Ruijie Zheng, Dantong Niu, Yuqi Xie, Jing Wang, Mengda Xu, Yunfan Jiang, Fernando Casta ˜neda, Fengyuan Hu, You Liang Tan, Letian Fu, Trevor Darrell, Furong Huang, Yuke Zhu, Danfei Xu, and Linxi Fan. EgoScale: Scaling dexterous manipulation with diverse egocentric human data. ... on PitharXiv
  70. [90] Kaichen Zhou, Yuzhen Chen, Fangneng Zhan, Hang Hua, Grace Chen, Xinhai Chang, Ao Qu, Yilun Du, Zhuang Liu, Paul Pu Liang, and Mengyu Wang. GEM-4D: Geometry- enhanced video world models for robot manipulation. In ECCV, pages 130–147, 2026. 2, 3 on Pith
  71. [91] Xunzhe Zhou, Yiyang Cai, Fengyi Wang, Ran Ju, Hanxiang Ren, Ruizhe Liu, Yu Zhang, Qian Luo, Feng Chen, Pei Zhou, Yi Ma, and Yanchao Yang. The Imitator Game: Benchmark- ing robot imitative ability beyond action prediction.arXiv preprint arXiv:2608.22301, 2026. 5 on PithPith's reviewarXiv
  72. [92] Yinghua Zhou, Junjie Ye, Yiqi Zhao, Hao Dong, Celina Shiyu Wang, Ruohai Ge, Tingyi Yang, Basile Van Hoorick, Gaurav Sukhatme, Vitor Guizilini, and Yue Wang. Rolling-WAM: World action models with rolling imagination.arXiv preprint arXiv:2609.30247, 2026. 3 on PithPith's reviewarXiv
  73. [93] Chuning Zhu, Raymond Yu, Siyuan Feng, Benjamin Burch- fiel, Paarth Shah, and Abhishek Gupta. Unified world models: Coupling video and action diffusion for pretraining on large robotic datasets. InRSS, 2025. 2, 3, 4 Gnomon4D: Learning Where and When to Focus for Geometric Motio... on Pith
  74. [95] The controller executes the first 16 of the 32 predicted actions, then acquires a new observation and replans. The remaining denoising steps refine Video only when a visual prediction is requested. F.4. Evaluation and Results Each method runs 40 ID and 40 OOD trials per task (... on Pith
  75. [2026] 2, 3, 4, 5, 6, 7, 10 on Pith

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment