DexFuture: Hierarchical Future-State Visuomotor Targeting for Bimanual Dexterous Tool Use

Dwait Bhatt; Keito Suzuki; Ki Myung Brian Lee; Kuang-Ting Tu; Nikola Raicevic; Nikolay Atanasov; Runfa Blark Li; Truong Nguyen; Xinshuang Liu

arxiv: 2606.05699 · v1 · pith:ZUPL7YIP · submitted 2026-06-04 · cs.RO

DexFuture: Hierarchical Future-State Visuomotor Targeting for Bimanual Dexterous Tool Use

Runfa Blark Li , Kuang-Ting Tu , Nikola Raicevic , Dwait Bhatt , Xinshuang Liu , Keito Suzuki , Ki Myung Brian Lee , Nikolay Atanasov

show 1 more author

Truong Nguyen

This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-06-28 01:31 UTCgrok-4.3pith:ZUPL7YIP record.json open to challenge →

classification cs.RO

keywords bimanual dexterous manipulationvisuomotor policieshierarchical controlfuture state predictiontool userobot learning

0 comments

The pith

A two-level visuomotor hierarchy generates future targets from camera history and tracks them to reach near-oracle bimanual tool performance at real-time speed.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces DexFuture to solve the problem of generating consistent future references for dexterous bimanual tool use without relying on demonstration states or slow planning. It uses a high-level predictor to create multi-step target trajectories from visual and sensor history, paired with a low-level policy that follows those targets. This separation allows the system to run at 60 Hz while achieving 90 percent of the performance of an oracle policy that has privileged information. A no-reference policy only gets 7 percent on the same tasks. The approach matters because it could enable robots to perform complex manipulation tasks efficiently in dynamic settings.

Core claim

DexFuture is a hierarchical system consisting of a high-level Future-State Visuomotor Target Predictor that builds structured hand-tool-object embeddings from egocentric RGB, proprioceptive and geometric history and uses a horizon-conditioned transformer to output multi-step future target trajectories, coupled with a low-level Target-Conditioned Structured Dexterous Policy that uses a target-conditioned per-link transformer to track those targets. On OakInk2 bimanual tool-use tasks this yields 90 percent of privileged-oracle performance at 60 Hz, which is about 250 times faster than CEM planning with a future action-conditioned world model.

What carries the argument

The hierarchical coupling of the Future-State Visuomotor Target Predictor for coarse multi-step reference generation and the Target-Conditioned Structured Dexterous Policy for fine-grained tracking, which decouples semantic prediction from high-frequency control.

If this is right

The system performs bimanual dexterous tasks near oracle level without access to privileged demonstration states.
Control runs at 60 Hz, enabling real-time execution far faster than planning-based methods.
Future target trajectories generated from history alone suffice for dynamic consistency in tool use.
Structured visuomotor embeddings support prediction of hand-tool-object interactions over multiple steps.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Similar hierarchies could apply to other high-dimensional control problems where future references are costly to obtain.
Combining this with learned world models might further improve long-horizon consistency without increasing computation.
Validation on physical hardware would test whether the predicted targets transfer beyond simulation.

Load-bearing premise

The targets predicted by the high-level component stay close enough to feasible dynamics that the low-level policy can follow them without causing instability or failure.

What would settle it

Running the low-level policy on targets generated by the predictor and measuring whether success rate drops significantly below the oracle on held-out OakInk2 sequences.

Figures

Figures reproduced from arXiv: 2606.05699 by Dwait Bhatt, Keito Suzuki, Ki Myung Brian Lee, Kuang-Ting Tu, Nikola Raicevic, Nikolay Atanasov, Runfa Blark Li, Truong Nguyen, Xinshuang Liu.

**Figure 1.** Figure 1: DexFuture is a hierarchical system for bimanual dexterous tool use that couples a highlevel Future-State Visuomotor Target Predictor with a low-level Target-Conditioned Structured Dexterous Policy. It removes the need for privileged future demonstration targets by predicting a coarse future-state target trajectory from visuomotor history. The predicted targets provide long-horizon guidance for the low-le… view at source ↗

**Figure 2.** Figure 2: DexFuture overview. DexFuture is a hierarchical system that separates bimanual dexterous manipulation into slow future target generation and fast action-level control. A. Given recent egocentric RGB observations and proprioceptive/geometric states, we construct structured visuomotor tokens instead of passing dense image patches to the predictor. Hand-link embeddings are obtained by projecting each link i… view at source ↗

**Figure 3.** Figure 3: Qualitative comparison on bimanual dexterous tool-use tasks. From top to bottom: 1. chop knife cutting bread (083f7@0). 2. fruite knife cutting apple (9fc3e@0). 3. shear paper with scissors (9bb17@5). 4. wipe whiteboard with brush (fc88d@0). From left to right: 1. ManipTrans with ground-truth target. 2. PhysGraph with ground-truth target 3. PhysGraph without target. 4. Our DexFuture with predicted target. … view at source ↗

**Figure 4.** Figure 4: Qualitative results of DexWM-style CEM-based Planning. From left to right: Finetune + Horizon 1; No-finetune + Horizon 1; Finetune + Horizon 16; No-finetune + Horizon 16 [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗

read the original abstract

Bimanual dexterous tool use remains challenging for robots due to high-dimensional hand configurations and complex hand-tool-object dynamics and contact. Most existing control policies depend on future configuration references provided from demonstrations, while future action-conditioned world models require slow online planning over high-dimensional action sequences. A significant challenge is generating a dynamically consistent future reference trajectory without relying on privileged states from demonstrations or slow counterfactual planning. We propose DexFuture, a hierarchical system that couples a high-level Future-State Visuomotor Target Predictor with a low-level Target-Conditioned Structured Dexterous Policy. Conditioned on egocentric RGB, proprioceptive and geometric history, the high-level predictor constructs structured hand-tool-object visuomotor embeddings and uses a horizon-conditioned transformer to generate a multi-step future target trajectory. Then, the low-level policy tracks them with a target-conditioned per-link transformer. This hierarchy decouples coarse future reference generation from fine-grained action control, and slow long-horizon semantic prediction from high-frequency execution. On OakInk2 bimanual tool-use tasks, DexFuture achieves 90% of the privileged-oracle performance, compared to 7% for a no-reference policy. DexFuture operates at 60 Hz, approximately 250 times faster than DexWM-style Cross-Entropy Method (CEM) planning with a future action-conditioned world model.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

DexFuture's hierarchical target predictor plus tracker gets close to oracle on OakInk2 bimanual tasks at 60 Hz, but the abstract leaves the dynamic consistency of those targets unverified.

read the letter

The core idea is a two-level setup: a horizon-conditioned transformer that turns egocentric RGB plus proprioceptive and geometric history into structured multi-step hand-tool-object target trajectories, then a target-conditioned per-link transformer policy that executes them. On the OakInk2 bimanual tool-use suite this reaches 90% of privileged-oracle success while running at 60 Hz, roughly 250 times faster than the CEM planning baseline with a future-action world model, and well above the 7% no-reference policy.

The architecture choice itself is the main novelty. It separates coarse future-reference generation from fine action control and avoids both privileged demonstration states and online counterfactual search. That separation is a reasonable response to the speed and dimensionality problems in dexterous tool use.

The reported gains rest entirely on the claim that the high-level predictor produces targets the low-level policy can actually track without extra planning. The abstract supplies no trajectory-error numbers, no penetration or velocity-limit checks, and no ablations that isolate predictor quality from low-level robustness. Without those, the performance gap could come from task leniency or policy compensation rather than accurate future targets. The stress-test note correctly flags this missing validation.

The work is aimed at researchers building real-time dexterous controllers who already have access to similar datasets. It is worth sending to peer review so the methods, training details, and any feasibility metrics can be examined; the empirical headline is strong enough to justify referee time even if the current evidence is still thin.

Referee Report

2 major / 1 minor

Summary. The manuscript proposes DexFuture, a hierarchical visuomotor architecture for bimanual dexterous tool use. A high-level Future-State Visuomotor Target Predictor, conditioned on egocentric RGB, proprioceptive and geometric history, employs a horizon-conditioned transformer to output multi-step hand-tool-object target trajectories. These are tracked by a low-level Target-Conditioned Structured Dexterous Policy using a per-link transformer. On OakInk2 bimanual tool-use tasks the system is reported to reach 90% of privileged-oracle performance (versus 7% for a no-reference baseline) while executing at 60 Hz, approximately 250 times faster than DexWM-style CEM planning with a future action-conditioned world model.

Significance. If the reported performance holds under rigorous validation, the work provides a concrete demonstration that decoupling coarse future-reference generation from high-frequency tracking can deliver near-oracle dexterous manipulation at real-time rates without privileged states or online counterfactual planning. This architectural separation addresses a central tension in high-DoF control and supplies an empirical baseline that subsequent methods can be measured against.

major comments (2)

[Abstract] Abstract: the central claim that the high-level predictor produces dynamically consistent trajectories rests on the 90%-of-oracle result, yet no trajectory-level metrics (penetration, velocity-limit violations, contact maintenance) or ablations isolating the predictor’s contribution are referenced; without these the performance gap versus the 7% no-reference policy cannot be attributed to predictor quality rather than low-level compensation or task leniency.
[Abstract] The 250× speed advantage over DexWM-style CEM is load-bearing for the real-time claim, but the abstract supplies neither the precise CEM implementation details (horizon, number of samples, world-model architecture) nor confirmation that the same low-level policy is used in the baseline, preventing direct attribution of the speedup to the hierarchical design.

minor comments (1)

[Abstract] The term “DexWM-style” appears without an accompanying citation or definition; a reference to the original DexWM work should be supplied on first use.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on the abstract. We address each major comment below and will revise the manuscript accordingly to strengthen the presentation of results.

read point-by-point responses

Referee: [Abstract] Abstract: the central claim that the high-level predictor produces dynamically consistent trajectories rests on the 90%-of-oracle result, yet no trajectory-level metrics (penetration, velocity-limit violations, contact maintenance) or ablations isolating the predictor’s contribution are referenced; without these the performance gap versus the 7% no-reference policy cannot be attributed to predictor quality rather than low-level compensation or task leniency.

Authors: We agree that trajectory-level metrics and an explicit ablation would more directly attribute the performance gap to the predictor. The current 90% vs. 7% comparison already uses an identical low-level policy for both the full system and the no-reference baseline, which provides some isolation. To fully address the concern we will add trajectory quality metrics (penetration, velocity violations, contact maintenance) and a dedicated ablation on the predictor in the revised experiments section, and update the abstract to reference these supporting results. revision: yes
Referee: [Abstract] The 250× speed advantage over DexWM-style CEM is load-bearing for the real-time claim, but the abstract supplies neither the precise CEM implementation details (horizon, number of samples, world-model architecture) nor confirmation that the same low-level policy is used in the baseline, preventing direct attribution of the speedup to the hierarchical design.

Authors: The manuscript already specifies in Section 3.4 that the CEM baseline uses the identical low-level policy and the same future action-conditioned world model architecture. To improve clarity we will revise the abstract to include the key CEM parameters (horizon, sample count) and explicitly state that the low-level policy is shared, thereby making the attribution to the hierarchical design direct. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical results on OakInk2 are measured outcomes, not quantities derived by construction

full rationale

The paper's central claims consist of an architectural hierarchy (high-level horizon-conditioned transformer predictor + low-level per-link tracker) and measured task success rates (90% of oracle vs. 7% baseline) obtained by training and evaluating on OakInk2 data. No equations, predictions, or uniqueness arguments in the abstract reduce a claimed output to its own fitted inputs or to a self-citation chain. The performance numbers are external benchmarks rather than tautological re-statements of model parameters or prior self-work. The hierarchy is presented as an explicit design choice whose validity is tested empirically, not asserted by definition.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The central performance claim rests on learned neural network weights fitted to OakInk2 data and on the unverified assumption that the predicted targets remain dynamically feasible for the low-level tracker.

free parameters (1)

transformer architecture and training hyperparameters
Model sizes, learning rates, and horizon length are chosen to produce the reported results but are not enumerated in the abstract.

axioms (1)

domain assumption Predicted future targets are dynamically consistent with robot and object physics
The high-level predictor is assumed to output trajectories the low-level policy can track without explicit dynamics constraints or privileged states.

pith-pipeline@v0.9.1-grok · 5808 in / 1417 out tokens · 42370 ms · 2026-06-28T01:31:13.280083+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

30 extracted references · 6 canonical work pages · 2 internal anchors

[1]

DexMachina: Functional Retargeting for Bimanual Dexterous Manipulation

Z. Mandi, Y . Hou, D. Fox, Y . Narang, A. Mandlekar, and S. Song. Dexmachina: Functional retargeting for bimanual dexterous manipulation, 2025. URLhttps://arxiv.org/abs/ 2505.24853

work page Pith review arXiv 2025
[2]

X. Liu, J. Adalibieke, Q. Han, Y . Qin, and L. Yi. DexTrack: Towards generalizable neural tracking control for dexterous manipulation from human references. InICLR, 2025

2025
[3]

Z. Luo, J. Cao, S. Christen, A. Winkler, K. Kitani, and W. Xu. Omnigrasp: Grasping diverse objects with simulated humanoids. InNeuRIPS, 2024

2024
[4]

K. Li, P. Li, T. Liu, Y . Li, and S. Huang. ManipTrans: Efficient dexterous bimanual manipula- tion transfer via residual learning. 2025

2025
[5]

R. B. Li, D. Kim, X. Liu, K. Suzuki, D. Bhatt, N. Raicevic, X. Lin, K. M. B. Lee, N. Atanasov, and T. Nguyen. Physgraph: Physically-grounded graph-transformer policies for bimanual dex- terous hand-tool-object manipulation, 2026. URLhttps://arxiv.org/abs/2603.01436

work page arXiv 2026
[6]

R. G. Goswami, A. Bar, D. Fan, T.-Y . Yang, G. Zhou, P. Krishnamurthy, M. Rabbat, F. Khor- rami, and Y . LeCun. World models for learning dexterous hand-object interactions from human videos, 2026. URLhttps://arxiv.org/abs/2512.13644

work page arXiv 2026
[7]

X. Zhan, L. Yang, Y . Zhao, K. Mao, H. Xu, Z. Lin, K. Li, and C. Lu. OAKINK2: A dataset of bimanual hands-object manipulation in complex task completion. InCVPR, pages 445–456, June 2024

2024
[8]

Handa, K

A. Handa, K. Van Wyk, W. Yang, J. Liang, Y .-W. Chao, Q. Wan, S. Birchfield, N. D. Ratliff, and D. Fox. Dexpilot: Vision-based teleoperation of dexterous robotic hand-arm system. In IEEE International Conference on Robotics and Automation (ICRA), pages 9164–9170, 2020

2020
[9]

Qin, Y .-H

Y . Qin, Y .-H. Wu, S. Liu, H. Jiang, R. Yang, Y . Fu, and X. Wang. Dexmv: Imitation learning for dexterous manipulation from human videos.European Conference on Computer Vision (ECCV), 2022

2022
[10]

Y . Chen, T. Wu, S. Wang, X. Feng, J. Jiang, S. McAleer, Y . Geng, H. Dong, Z. Lu, S.-C. Zhu, and Y . Yang. Towards human-level bimanual dexterous manipulation with reinforcement learning.NeuRIPS, 2022

2022
[11]

Y . Qin, B. Huang, Z.-H. Yin, H. Su, and X. Wang. Dexpoint: Generalizable point cloud reinforcement learning for sim-to-real dexterous manipulation.Conference on Robot Learning (CoRL), 2022

2022
[12]

Y . Xu, W. Wan, J. Zhang, H. Liu, Z. Shan, H. Shen, R. Wang, H. Geng, Y . Weng, J. Chen, et al. Unidexgrasp: Universal robotic dexterous grasping via learning diverse proposal generation and goal-conditioned policy.CVPR, 2023

2023
[13]

Jiang, Y

Z. Jiang, Y . Xie, K. Lin, Z. Xu, W. Wan, A. Mandlekar, L. Fan, and Y . Zhu. Dexmimicgen: Automated data generation for bimanual dexterous manipulation via imitation learning. In 2025 IEEE International Conference on Robotics and Automation (ICRA), 2025

2025
[14]

Y .-W. Chao, W. Yang, Y . Xiang, P. Molchanov, A. Handa, J. Tremblay, Y . S. Narang, K. Van Wyk, U. Iqbal, S. Birchfield, J. Kautz, and D. Fox. DexYCB: A benchmark for capturing hand grasping of objects. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021

2021
[15]

Z. Fan, O. Taheri, D. Tzionas, M. Kocabas, M. Kaufmann, M. J. Black, and O. Hilliges. ARC- TIC: A dataset for dexterous bimanual hand-object manipulation. InProceedings IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), 2023. 9

2023
[16]

Taheri, N

O. Taheri, N. Ghorbani, M. J. Black, and D. Tzionas. GRAB: A dataset of whole-body human grasping of objects. InEuropean Conference on Computer Vision (ECCV), 2020

2020
[17]

Y . Liu, Y . Liu, C. Jiang, K. Lyu, W. Wan, H. Shen, B. Liang, Z. Fu, H. Wang, and L. Yi. Hoi4d: A 4d egocentric dataset for category-level human-object interaction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 21013–21022, June 2022

2022
[18]

EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video

R. Hoque, P. Huang, D. J. Yoon, M. Sivapurapu, and J. Zhang. Egodex: Learning dexter- ous manipulation from large-scale egocentric video, 2025. URLhttps://arxiv.org/abs/ 2505.11709

work page internal anchor Pith review Pith/arXiv arXiv 2025
[19]

Hafner, J

D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap. Mastering diverse control tasks through world models.Nature, pages 1–7, 2025

2025
[20]

Hansen, X

N. Hansen, X. Wang, and H. Su. Temporal difference learning for model predictive control. In International Conference on Machine Learning (ICML), 2022

2022
[21]

Hansen, H

N. Hansen, H. Su, and X. Wang. Td-mpc2: Scalable, robust world models for continuous control. InInternational Conference on Learning Representations (ICLR), 2024

2024
[22]

Y . Seo, D. Hafner, H. Liu, F. Liu, S. James, K. Lee, and P. Abbeel. Masked world models for visual control. InConference on Robot Learning (CoRL), 2022

2022
[23]

Y . Seo, J. Kim, S. James, K. Lee, J. Shin, and P. Abbeel. Multi-view masked world models for visual robotic manipulation.ICML, 2023

2023
[24]

Assran, Q

M. Assran, Q. Duval, I. Misra, P. Bojanowski, P. Vincent, M. Rabbat, Y . LeCun, and N. Bal- las. Self-supervised learning from images with a joint-embedding predictive architecture. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

2023
[25]

Revisiting Feature Prediction for Learning Visual Representations from Video

A. Bardes, Q. Garrido, J. Ponce, X. Chen, M. Rabbat, Y . LeCun, M. Assran, and N. Ballas. Revisiting feature prediction for learning visual representations from video.arXiv preprint arXiv:2404.08471, 2024

work page internal anchor Pith review Pith/arXiv arXiv 2024
[26]

A. Bar, G. Zhou, D. Tran, T. Darrell, and Y . LeCun. Navigation world models.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

2025
[27]

Peebles and S

W. Peebles and S. Xie. Scalable diffusion models with transformers. InIEEE/CVF Interna- tional Conference on Computer Vision (ICCV), 2023

2023
[28]

R. Y . Rubinstein. The cross-entropy method for combinatorial and continuous optimization. Methodology and Computing in Applied Probability, pages 127–190, 1999

1999
[29]

A. Bar, G. Zhou, D. Tran, T. Darrell, and Y . LeCun. Navigation world models.arXiv preprint arXiv:2412.03572, 2024

work page arXiv 2024
[30]

Schulman, F

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms, 2017. 10 7 Appendix 7.1 Comparison with Action-Conditioned Planning In this section, we complete the comparison in Section 4.5. We compare DexFuture against an action-conditioned world-model planning baseline following DexWM [6]. The original DexWM pla...

2017

[1] [1]

DexMachina: Functional Retargeting for Bimanual Dexterous Manipulation

Z. Mandi, Y . Hou, D. Fox, Y . Narang, A. Mandlekar, and S. Song. Dexmachina: Functional retargeting for bimanual dexterous manipulation, 2025. URLhttps://arxiv.org/abs/ 2505.24853

work page Pith review arXiv 2025

[2] [2]

X. Liu, J. Adalibieke, Q. Han, Y . Qin, and L. Yi. DexTrack: Towards generalizable neural tracking control for dexterous manipulation from human references. InICLR, 2025

2025

[3] [3]

Z. Luo, J. Cao, S. Christen, A. Winkler, K. Kitani, and W. Xu. Omnigrasp: Grasping diverse objects with simulated humanoids. InNeuRIPS, 2024

2024

[4] [4]

K. Li, P. Li, T. Liu, Y . Li, and S. Huang. ManipTrans: Efficient dexterous bimanual manipula- tion transfer via residual learning. 2025

2025

[5] [5]

R. B. Li, D. Kim, X. Liu, K. Suzuki, D. Bhatt, N. Raicevic, X. Lin, K. M. B. Lee, N. Atanasov, and T. Nguyen. Physgraph: Physically-grounded graph-transformer policies for bimanual dex- terous hand-tool-object manipulation, 2026. URLhttps://arxiv.org/abs/2603.01436

work page arXiv 2026

[6] [6]

R. G. Goswami, A. Bar, D. Fan, T.-Y . Yang, G. Zhou, P. Krishnamurthy, M. Rabbat, F. Khor- rami, and Y . LeCun. World models for learning dexterous hand-object interactions from human videos, 2026. URLhttps://arxiv.org/abs/2512.13644

work page arXiv 2026

[7] [7]

X. Zhan, L. Yang, Y . Zhao, K. Mao, H. Xu, Z. Lin, K. Li, and C. Lu. OAKINK2: A dataset of bimanual hands-object manipulation in complex task completion. InCVPR, pages 445–456, June 2024

2024

[8] [8]

Handa, K

A. Handa, K. Van Wyk, W. Yang, J. Liang, Y .-W. Chao, Q. Wan, S. Birchfield, N. D. Ratliff, and D. Fox. Dexpilot: Vision-based teleoperation of dexterous robotic hand-arm system. In IEEE International Conference on Robotics and Automation (ICRA), pages 9164–9170, 2020

2020

[9] [9]

Qin, Y .-H

Y . Qin, Y .-H. Wu, S. Liu, H. Jiang, R. Yang, Y . Fu, and X. Wang. Dexmv: Imitation learning for dexterous manipulation from human videos.European Conference on Computer Vision (ECCV), 2022

2022

[10] [10]

Y . Chen, T. Wu, S. Wang, X. Feng, J. Jiang, S. McAleer, Y . Geng, H. Dong, Z. Lu, S.-C. Zhu, and Y . Yang. Towards human-level bimanual dexterous manipulation with reinforcement learning.NeuRIPS, 2022

2022

[11] [11]

Y . Qin, B. Huang, Z.-H. Yin, H. Su, and X. Wang. Dexpoint: Generalizable point cloud reinforcement learning for sim-to-real dexterous manipulation.Conference on Robot Learning (CoRL), 2022

2022

[12] [12]

Y . Xu, W. Wan, J. Zhang, H. Liu, Z. Shan, H. Shen, R. Wang, H. Geng, Y . Weng, J. Chen, et al. Unidexgrasp: Universal robotic dexterous grasping via learning diverse proposal generation and goal-conditioned policy.CVPR, 2023

2023

[13] [13]

Jiang, Y

Z. Jiang, Y . Xie, K. Lin, Z. Xu, W. Wan, A. Mandlekar, L. Fan, and Y . Zhu. Dexmimicgen: Automated data generation for bimanual dexterous manipulation via imitation learning. In 2025 IEEE International Conference on Robotics and Automation (ICRA), 2025

2025

[14] [14]

Y .-W. Chao, W. Yang, Y . Xiang, P. Molchanov, A. Handa, J. Tremblay, Y . S. Narang, K. Van Wyk, U. Iqbal, S. Birchfield, J. Kautz, and D. Fox. DexYCB: A benchmark for capturing hand grasping of objects. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021

2021

[15] [15]

Z. Fan, O. Taheri, D. Tzionas, M. Kocabas, M. Kaufmann, M. J. Black, and O. Hilliges. ARC- TIC: A dataset for dexterous bimanual hand-object manipulation. InProceedings IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), 2023. 9

2023

[16] [16]

Taheri, N

O. Taheri, N. Ghorbani, M. J. Black, and D. Tzionas. GRAB: A dataset of whole-body human grasping of objects. InEuropean Conference on Computer Vision (ECCV), 2020

2020

[17] [17]

Y . Liu, Y . Liu, C. Jiang, K. Lyu, W. Wan, H. Shen, B. Liang, Z. Fu, H. Wang, and L. Yi. Hoi4d: A 4d egocentric dataset for category-level human-object interaction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 21013–21022, June 2022

2022

[18] [18]

EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video

R. Hoque, P. Huang, D. J. Yoon, M. Sivapurapu, and J. Zhang. Egodex: Learning dexter- ous manipulation from large-scale egocentric video, 2025. URLhttps://arxiv.org/abs/ 2505.11709

work page internal anchor Pith review Pith/arXiv arXiv 2025

[19] [19]

Hafner, J

D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap. Mastering diverse control tasks through world models.Nature, pages 1–7, 2025

2025

[20] [20]

Hansen, X

N. Hansen, X. Wang, and H. Su. Temporal difference learning for model predictive control. In International Conference on Machine Learning (ICML), 2022

2022

[21] [21]

Hansen, H

N. Hansen, H. Su, and X. Wang. Td-mpc2: Scalable, robust world models for continuous control. InInternational Conference on Learning Representations (ICLR), 2024

2024

[22] [22]

Y . Seo, D. Hafner, H. Liu, F. Liu, S. James, K. Lee, and P. Abbeel. Masked world models for visual control. InConference on Robot Learning (CoRL), 2022

2022

[23] [23]

Y . Seo, J. Kim, S. James, K. Lee, J. Shin, and P. Abbeel. Multi-view masked world models for visual robotic manipulation.ICML, 2023

2023

[24] [24]

Assran, Q

M. Assran, Q. Duval, I. Misra, P. Bojanowski, P. Vincent, M. Rabbat, Y . LeCun, and N. Bal- las. Self-supervised learning from images with a joint-embedding predictive architecture. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

2023

[25] [25]

Revisiting Feature Prediction for Learning Visual Representations from Video

A. Bardes, Q. Garrido, J. Ponce, X. Chen, M. Rabbat, Y . LeCun, M. Assran, and N. Ballas. Revisiting feature prediction for learning visual representations from video.arXiv preprint arXiv:2404.08471, 2024

work page internal anchor Pith review Pith/arXiv arXiv 2024

[26] [26]

A. Bar, G. Zhou, D. Tran, T. Darrell, and Y . LeCun. Navigation world models.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

2025

[27] [27]

Peebles and S

W. Peebles and S. Xie. Scalable diffusion models with transformers. InIEEE/CVF Interna- tional Conference on Computer Vision (ICCV), 2023

2023

[28] [28]

R. Y . Rubinstein. The cross-entropy method for combinatorial and continuous optimization. Methodology and Computing in Applied Probability, pages 127–190, 1999

1999

[29] [29]

A. Bar, G. Zhou, D. Tran, T. Darrell, and Y . LeCun. Navigation world models.arXiv preprint arXiv:2412.03572, 2024

work page arXiv 2024

[30] [30]

Schulman, F

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms, 2017. 10 7 Appendix 7.1 Comparison with Action-Conditioned Planning In this section, we complete the comparison in Section 4.5. We compare DexFuture against an action-conditioned world-model planning baseline following DexWM [6]. The original DexWM pla...

2017