Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

Scalable Multi-Task Reinforcement Learning for Generalizable Spatial Intelligence in Visuomotor Agents

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RL post-training on 100,000 auto-generated Minecraft tasks quadruples a visuomotor agent's interaction success and transfers that spatial skill zero-shot to unseen worlds, including a real robot.

desk verdict RL post-training at 100k-task scale looks real and useful, but the zero-shot transfer claim hinges on an undocumented mask-input question. read the letter →

arxiv 2507.23698 v1 pith:5IH6CBLM submitted 2025-07-31 cs.RO cs.AI

Shaofei Cai , Zhancun Mu , Haiwen Xia , Bowei Zhang , Anji Liu , Yitao Liang This is my paper
classification cs.ROcs.AI
keywords reinforcementlearningpost-trainingcross-viewgoalspecificationmulti-taskzero-shotgeneralizationvisuomotorpolicyMinecraftspatialreasoningautomatedtasksynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that reinforcement learning post-training, applied to a visuomotor policy that was first imitation-pretrained, can sharpen generalizable spatial reasoning the way RL post-training sharpens reasoning in language models. The agent trains on roughly 100,000 automatically generated Minecraft tasks in which a target object is specified only by a segmentation mask in a second, third-person camera view; RL raises average interaction success from 7% to 28%, a fourfold gain. The same policy, with no further training, then performs cross-view search-and-approach tasks in DeepMind Lab, Unreal Engine, and a real robot car through a simple rule-based action mapping. If correct, this shows RL can act as a general post-training stage for embodied agents, and that a cross-view goal representation makes large-scale task definition practical. The paper also reports that a language-prompted baseline stays near zero success under the same pipeline, evidence that the visual cross-view representation, not the RL method alone, is what makes the tasks learnable.

What carries the argument

Cross-view goal specification (CVGS): a task is specified by an initial egocentric view, a goal image from a distinct third-person camera, a segmentation mask of the target object inside that goal image, and an interaction event such as break, use, pick up, or approach. Its work in the argument is to serve as a unified task space that is open (any viewable object can be a goal), unambiguous (the mask pins the exact instance), scalable (rewards derive automatically from voxel-change detection), and curriculum-friendly (the distance between the two views sets the difficulty). Because every task reduces to one operation — aligning the current view with the masked goal view — the policy learns a single transferable skill, cross-view spatial reasoning, rather than a collection of environment-specific behaviors.

What would settle it

Retrain the pipeline with a corrupted reward that fires on any voxel change within a small radius of the target rather than a change to the target's own voxels, then test zero-shot transfer on the DMLab fruit task and the real robot; if transfer success is unchanged, the policy learned a proximity shortcut and the 4× gain does not demonstrate spatial reasoning, whereas a collapse would confirm the reward tracks true alignment with the masked goal view.

Watch

Extended reading notes

Core claim

The paper's central claim is that RL post-training enhances the spatial reasoning of a pretrained visuomotor policy, and that the enhanced capability generalizes zero-shot across 3D environments. Tasks are synthesized by sampling a Minecraft world seed, terrain, a camera distance of 20, 40, or 60 blocks, and a target object; the goal is specified as a third-person image with a SAM-generated segmentation mask of that object. A binary outcome reward is produced automatically by detecting voxel changes in the simulator, so 100,000 tasks require no manual reward design. PPO with a KL constraint toward the pretrained policy raises average success from 7% to 28% across Approach, Break, Interact, Melee Hunt, and Archery, with Archery jumping from under 1% to 28%. The same policy, using only the action mapping in Table 4, improves DMLab30 fruit collection, Unreal Zoo casualty rescue, and real-robot ball finding, including cases where the target is invisible in the egocentric view and visible only in the goal image; the visibility and centroid prediction heads also keep working after RL even though only the action head was trained. The authors conclude that the cross-view goal space is what unifies multi-task RL and enables the transfer.

Load-bearing premise

The load-bearing premise is that the automatic reward, which fires when the target object's voxels change in Minecraft, truly measures cross-view spatial reasoning, and that the rule-based action mapping in Table 4 preserves each task's meaning when the policy is deployed in DMLab, Unreal, and the real robot.

Editorial extensions

If this is right

  • Average interaction success on the Minecraft task suite rises from 7% to 28% across all five skill types, with hard tasks improving as well, which the authors read as the model acquiring exploration ability.
  • Archery goes from under 1% after pre-training to 28% after RL, showing that RL can unlock abilities that imitation learning left latent.
  • The same policy, with no environment-specific training, transfers to DMLab30 fruit collection, Unreal Zoo rescue, and a real robot car, with real-world ball-finding success up by as much as 41%.
  • A mixed-difficulty curriculum that samples easy, medium, and hard tasks uniformly trains faster and reaches higher value-function explained variance than hard-only training.
  • The KL constraint toward the pretrained policy prevents the performance collapse seen in KL-free runs, and policies trained from scratch fail entirely, so RL post-training only works on top of a strong imitation-learned foundation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism holds, the pretrain-then-RL-post-train recipe from language modeling transfers to embodied agents: one simulator rich enough for procedural task generation could replace per-environment fine-tuning wherever action spaces can be mapped.
  • The near-zero performance of the language-conditioned baseline suggests the representation, not RL itself, carries the generalization; a clean test would train RL on identical voxel-change rewards under both goal representations and compare transfer curves.
  • The paper's documented failures — long visually homogeneous corridors, low robot camera heights, and absent skills such as door-opening — point to the next lever: scaling the diversity of training worlds, not just the number of tasks.
  • A testable extension is running the same pipeline in a photorealistic simulator; if viewpoint alignment is the true abstraction, gains should reach the real robot with a smaller visual-texture gap than Minecraft training leaves.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies whether RL post-training can improve a pretrained visuomotor policy's cross-view spatial reasoning in 3D worlds. The method defines tasks in the Cross-View Goal Specification (CVGS) space T = <O1, Og, Mg, E>, synthesizes roughly 100,000 Minecraft tasks by sampling the camera view, distance, target object, and interaction type, and fine-tunes a 0.3B ROCKET-2 policy with PPO plus a KL constraint against the pretrained policy. The authors report a 4x improvement in Minecraft interaction success rate (from 7% to 28% on average), ablation evidence favoring the KL constraint and mixed-difficulty curriculum, and zero-shot deployment to DMLab, Unreal Engine, and a real Mecanum-wheeled robot via a manually defined action mapping. The paper also contributes a distributed RL framework with fragment-based storage for long-sequence Transformer policies.

Significance. If the central claim holds, the paper is a meaningful step toward making RL a viable post-training stage for visuomotor policies, analogous to RL fine-tuning for LLMs. The empirical scope is unusually broad for a single paper: multiple independent simulated environments and a physical robot, with informative ablations (with/without KL, with/without curriculum, no-pretraining RL, language-based STEVE-1). The authors also release code and an RL framework, and the supplementary failure analysis is candid. However, the strongest evidence for the headline contribution—zero-shot transfer to unseen domains—depends on the deployment-time task specification actually being CVGS, and the current manuscript does not document how the required segmentation mask Mg is obtained in DMLab, Unreal, or the real robot. Without that documentation, the transfer results could reflect a different input format, and the central interpretation would be unsupported. This is a documentation gap that can be settled by inspection of the release, not an internal contradiction.

major comments (3)
  1. [Section 4 (Task Formulation) and Supplementary D] The paper defines CVGS as T = <O1, Og, Mg, E> and the architecture in Supplementary D encodes Mg separately with a trainable ViT-tiny and fuses it with the goal image before SpatialFusion; the policy therefore does not receive a goal image alone at training time. The three transfer protocols in Supplementary C.2 describe only a goal image or a language prompt: Unreal uses "images of the injured person," DMLab uses "human-generated prompts curated from live gameplay," and the real robot is given a top-down goal image. No statement explains how Mg is computed or supplied in these domains. If mask-free goal inputs are used at deployment, the zero-shot evaluation is not an evaluation of the CVGS policy, and the measured gains cannot be attributed to cross-view spatial reasoning. Please document mask generation for each target domain, or restrict the generalization claim to the actual deployment input format.
  2. [Section 5.2 and Figure 3f/g; Supplementary C.2] The central quantitative claims are reported as point estimates without appropriate uncertainty: the headline improvement from 7% to 28% is a single average, and the real-world comparison is based on 12 trials per condition (3/12 vs. 8/12 in the hard variant). DMLab and Unreal results are given without episode counts, raw trial numbers, confidence intervals, or multiple seeds. Given that the abstract and introduction describe a "remarkable 4x" and "compelling" zero-shot generalization, the paper should report variances and sample sizes; otherwise the strength of the central claim is overstated.
  3. [Table 4] The hand-mapped action table appears inconsistent in the transverse channel: Minecraft "right=1" maps to Unreal "velocity = -100," while Minecraft "left=1" maps to Unreal "/" (masked), and the same Minecraft actions map to "0.1 meters left/right" for the robot car. Without signed coordinate conventions for each environment and validation that the mapping preserves task semantics, Table 4 does not by itself establish that the transferred policy is solving the same task. Please add coordinate definitions and per-action validation results in each target environment.
minor comments (5)
  1. [Abstract and Section 5.2] The text should define whether the "4x" and "up to 41%" figures are relative improvements or absolute percentage-point increases; currently "up to 41%" is ambiguous.
  2. [Section 5.4 and Supplementary C.2] The real-world robot operates at 2 Hz while Minecraft runs at 15+ Hz; this temporal control mismatch is a confound for the real-world transfer results and should be discussed quantitatively, not only mentioned in passing.
  3. [Figure 3(d)] The caption and text refer to "current SOTA goal-conditioned agents," but the table mixes single-task RL agents, language-conditioned agents, and multi-task baselines with different problem setups; please add a column specifying task space, number of tasks, and evaluation conditions.
  4. [Supplementary E] The failure analysis is a useful addition, but several explanations (e.g., the marble-hallway hypothesis) are presented after the outcome; please label them explicitly as post hoc hypotheses rather than measured causes.
  5. [Equation (5)] The rotation-matrix convention for yaw and pitch is not stated; please specify the axis order and sign conventions used in the coordinate transformation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central RL gains are measured as a before/after delta and as zero-shot transfer to external worlds, and the cited prior work is only a building block, not a load-bearing self-referential proof.

full rationale

The derivation chain is an empirical claim: start from a pre-trained ROCKET-2 policy, run PPO post-training on automatically synthesized CVGS tasks in Minecraft, and measure success. Each link is independently checkable: the 7% to 28% Minecraft figure is a before/after comparison of the same policy family on the same task distribution; the DMLab, Unreal, and real-robot results are out-of-distribution evaluations that require no fitted parameter and are not constructed from the training objective. The CVGS task space is indeed cited to Cai et al. 2025, also by the authors, but the citation is used as a prior building block (the pretrained checkpoint and goal representation), not as the argument for the paper's conclusion; the experimental design compares ROCKET-2 before and after RL and against other baselines, so the conclusion does not reduce to the self-citation. The voxel-change outcome reward is a simulator ground-truth success signal, and using the same signal for evaluation is a standard measurement choice, not a fitted input renamed as a prediction. No equation in the paper defines a predicted quantity in terms of a fitted coefficient, no uniqueness theorem is imported from the authors' prior work to force a choice, and no ansatz is smuggled in via citation. The reviewer-flagged absence of documented segmentation masks in DMLab/Unreal/real goal inputs is a potential construct-validity and reporting gap, but it is a documentation issue, not circularity, because it would weaken the transfer claim rather than make it tautological. Therefore no circular step is exhibitable under the required reduction standard.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim leans on the CVGS task space, a voxel-based outcome reward, the pretrained ROCKET-2 checkpoint, and a hand-defined action mapping; none of these is derived from first principles, and all are inherited from prior work or engineering choices.

free parameters (3)
  • KL divergence coefficient (rho) = 0.2
    Chosen by hand; the paper shows omitting KL leads to collapse, so this value materially affects the stability of the central result.
  • PPO clip ratio = 0.2
    Standard PPO hyperparameter, manually set; not fitted to evaluation data.
  • Curriculum difficulty distances = 20/40/60 blocks
    Hand-selected Manhattan distances defining easy, medium, and hard synthetic tasks; the curriculum comparison depends on these.
assumptions (4)
  • domain assumption Cross-view goal specification (segmentation mask in a third-person view) is a sufficient and unambiguous goal representation for learning generalizable spatial reasoning.
    Adopted from ROCKET-2 (Cai et al. 2025) without independent justification; the paper's entire task space and transfer claims rest on it.
  • domain assumption The voxel-change outcome reward in Minecraft is a faithful proxy for task success and rewards the intended cross-view spatial reasoning.
    Section 4: 'Rewards are then automatically generated by detecting changes in the object's voxels.' If this reward can be gamed by simulator shortcuts, the central result would not measure spatial reasoning.
  • ad hoc to paper The pretrained ROCKET-2 policy contains latent spatial reasoning that RL can unlock; the paper shows no-pretraining RL fails, making this load-bearing.
    Section 5.2 reports that policies without pretraining fail in multi-task RL, so the main positive result depends on the specific pretrained checkpoint from the authors' prior work.
  • domain assumption The action-space mapping in Table 4 preserves task semantics across Minecraft, DMLab, Unreal, and the real robot.
    Zero-shot transfer is evaluated only through these mappings; if the mapping changes the task, the transfer claim is weakened.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scalable Multi-Task Reinforcement Learning for Generalizable Spatial Intelligence in Visuomotor Agents." pith.science (2026). https://pith.science/paper/5IH6CBLM

@misc{pith2026250723698,
  author       = {Pith},
  title        = {Pith review of: Scalable Multi-Task Reinforcement Learning for Generalizable Spatial Intelligence in Visuomotor Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5IH6CBLM}},
  note         = {Machine review of arXiv:2507.23698}
}
abstract

While Reinforcement Learning (RL) has achieved remarkable success in language modeling, its triumph hasn't yet fully translated to visuomotor agents. A primary challenge in RL models is their tendency to overfit specific tasks or environments, thereby hindering the acquisition of generalizable behaviors across diverse settings. This paper provides a preliminary answer to this challenge by demonstrating that RL-finetuned visuomotor agents in Minecraft can achieve zero-shot generalization to unseen worlds. Specifically, we explore RL's potential to enhance generalizable spatial reasoning and interaction capabilities in 3D worlds. To address challenges in multi-task RL representation, we analyze and establish cross-view goal specification as a unified multi-task goal space for visuomotor policies. Furthermore, to overcome the significant bottleneck of manual task design, we propose automated task synthesis within the highly customizable Minecraft environment for large-scale multi-task RL training, and we construct an efficient distributed RL framework to support this. Experimental results show RL significantly boosts interaction success rates by $4\times$ and enables zero-shot generalization of spatial reasoning across diverse environments, including real-world settings. Our findings underscore the immense potential of RL training in 3D simulated environments, especially those amenable to large-scale task generation, for significantly advancing visuomotor agents' spatial reasoning.

Figures

Figures reproduced from arXiv: 2507.23698 by the authors.

Figure 1
Figure 1. The Post-Training Pipeline. We synthesize large-scale, mixed-difficulty cross-view interaction tasks in an open￾world environment by randomly sampling terrain, distances, target objects, and camera views. The foundational policy is fine-tuned using our distributed RL framework and then deployed in unseen 3D worlds via a simple action space mapping. scale trajectories collected via backward trajectory relabel￾ing. We… view at source ↗
Figure 2
Figure 2. Trajectory Storage Comparison. open-source our RL training framework to foster further RL research in complex environments. Details about the RL framework can be found in supplementary materials. 5 Experiments 5.1 Environments As shown in [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. RL Post-Training Boosts Generalizable Spatial Reasoning and Open-World Interaction Capabilities. (a) RL training curves for five skills in the Minecraft environment. This panel shows simultaneous performance gains across all skills. It also highlights the policy’s performance collapse in later training stages without a KL divergence constraint. (b) Sample target viewpoints for each skill during training, encompassin… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Our fragment-based storage strategy. Our rollout workers only save the initial latent states (K-V caches) at the beginning of each contiguous fragment. Latent states within the fragments are computed on the fly during tBPTT. C Evaluation Protocols C.1 Minecraft Evaluat…
Figure 5
Figure 5. Figure 5: The zero shot setting for real world environ￾ments. The goal would be blocked by the paper box if the car naively rotates towards the direction [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: The easy and hard variant of cross-view ap￾proach setting. or a set of actions (e.g., yaw, pitch, forward)—the actions are executed sequentially. After completion, an onboard cam￾era image of 640 × 360 resolution is sent back to the server for the next inference step. …
Figure 8
Figure 8. Figure 8: Different goal captures. Goals from phone cam￾eras does not deteriorate the performance of our method. Additional Variants We also evaluated several other set￾tings, including an alternative goal image( [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: (a) Gallery of task synthesis results. (b) Minecraft Demonstrations. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Zero-Shot Environment Showcases. We evaluated both the pre-trained ROCKET-2 and our agent in Unreal (Zhong et al. 2024), DMLab30 (Beattie et al. 2016), and real-world environments. Experimental results demonstrate that this reinforce￾ment learning approach can signifi…

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agents

    cs.LG 2026-05 conditional novelty 6.0 of 10

    GROW decomposes trajectories into state-action samples for GRPO training of VLM agents and reports state-of-the-art results on over 800 Minecraft tasks.

  2. GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agents

    cs.LG 2026-05 unverdicted novelty 5.0 of 10

    GROW decomposes trajectories into state-action samples to enable GRPO for multi-turn VLM agents and reports state-of-the-art results on more than 800 Minecraft tasks.

Reference graph

Works this paper leans on

46 extracted references · 15 canonical work pages · cited by 1 Pith paper

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  4. [4]

    J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F

    Achiam, O. J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; et al. 2023. GPT-4 Technical Report

    work page 2023

  5. [5]

    Baker, B.; Akkaya, I.; Zhokhov, P.; Huizinga, J.; Tang, J.; Ecoffet, A.; Houghton, B.; Sampedro, R.; and Clune, J. 2022. Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos. ArXiv, abs/2206.11795

    arXiv 2022

  6. [6]

    Z.; Teplyashin, D.; Kaufmann, T

    Beattie, C.; Leibo, J. Z.; Teplyashin, D.; Kaufmann, T. K.; Siddharth, N.; Clark, A.; Phillps, L.; Hughes, E.; Lamb, A.; Kelly, A.; Rowland, D. J.; Merel, J.; Wayne, G.; Porcel, N.; Noury, S.; Clark, S.; Babuschkin, I.; and Botvinick, M. 2016. DeepMind Lab . arXiv preprint arXiv:1612.03801

    arXiv 2016

  7. [7]

    Brohan, A.; Brown, N.; Carbajal, J.; Chebotar, Y.; Chen, X.; Choromanski, K.; Ding, T.; Driess, D.; Dubey, A.; Finn, C.; et al. 2023. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818

    arXiv 2023

  8. [8]

    J.; Julian, R

    Brohan, A.; Brown, N.; Carbajal, J.; Chebotar, Y.; Dabis, J.; Finn, C.; Gopalakrishnan, K.; Hausman, K.; Herzog, A.; Hsu, J.; Ibarz, J.; Ichter, B.; Irpan, A.; Jackson, T.; Jesmonth, S.; Joshi, N. J.; Julian, R. C.; Kalashnikov, D.; Kuang, Y.; Leal, I.; Lee, K.-H.; Levine, S.; Lu, Y.; Malla, U.; Manjunath, D.; Mordatch, I.; Nachum, O.; Parada, C.; Peralta...

    arXiv 2022

Show all 46 references
  1. [9]

    B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; Agarwal, S.; Herbert-Voss, A.; Krueger, G.; Henighan, T

    Brown, T. B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; Agarwal, S.; Herbert-Voss, A.; Krueger, G.; Henighan, T. J.; Child, R.; Ramesh, A.; Ziegler, D. M.; Wu, J.; Winter, C.; Hesse, C.; Chen, M.; Sigler, E....

    2020 arXiv

  2. [10]

    Cai, S.; Mu, Z.; Liu, A.; and Liang, Y. 2025. ROCKET-2: Steering Visuomotor Policy via Cross-View Goal Alignment. arXiv preprint arXiv:2503.02505

    2025 arXiv

  3. [11]

    Cai, S.; Wang, Z.; Lian, K.; Mu, Z.; Ma, X.; Liu, A.; and Liang, Y. 2024. ROCKET-1: Master Open-World Interaction with Visual-Temporal Context Prompting. arXiv preprint arXiv:2410.17856

    2024 arXiv

  4. [12]

    Cai, S.; Wang, Z.; Ma, X.; Liu, A.; and Liang, Y. 2023 a . Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 13734--13744

    2023

  5. [13]

    Cai, S.; Zhang, B.; Wang, Z.; Ma, X.; Liu, A.; and Liang, Y. 2023 b . GROOT: Learning to Follow Instructions by Watching Gameplay Videos. In The Twelfth International Conference on Learning Representations

    2023

  6. [14]

    Caron, M.; Touvron, H.; Misra, I.; J\'egou, H.; Mairal, J.; Bojanowski, P.; and Joulin, A. 2021. Emerging Properties in Self-Supervised Vision Transformers. In Proceedings of the International Conference on Computer Vision (ICCV)

    2021

  7. [15]

    DeepSeek-AI; Guo, D.; Yang, D.; Zhang, H.; Song, J.-M.; Zhang, R.; Xu, R.; Zhu, Q.; Ma, S.; Wang, P.; Bi, X.; Zhang, X.; Yu, X.; Wu, Y.; Wu, Z. F.; Gou, Z.; Shao, Z.; Li, Z.; Gao, Z.; Liu, A.; Xue, B.; Wang, B.-L.; Wu, B.; Feng, B.; Lu, C.; Zhao, C.; Deng, C.; Zhang, C.; Ruan,...

    2025 arXiv

  8. [16]

    Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; and Houlsby, N. 2020. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. ArXiv, abs/2010.11929

    2020 arXiv

  9. [17]

    J.; Wang, G.; Jiang, Y.; Mandlekar, A.; Yang, Y.; Zhu, H.; Tang, A.; Huang, D.-A.; Zhu, Y.; and Anandkumar, A

    Fan, L. J.; Wang, G.; Jiang, Y.; Mandlekar, A.; Yang, Y.; Zhu, H.; Tang, A.; Huang, D.-A.; Zhu, Y.; and Anandkumar, A. 2022. MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge. ArXiv, abs/2206.08853

    2022 arXiv

  10. [18]

    H.; Houghton, B.; Topin, N.; Wang, P.; Codel, C.; Veloso, M

    Guss, W. H.; Houghton, B.; Topin, N.; Wang, P.; Codel, C.; Veloso, M. M.; and Salakhutdinov, R. 2019. MineRL: A Large-Scale Dataset of Minecraft Demonstrations. In International Joint Conference on Artificial Intelligence

    2019

  11. [19]

    Ilon, B. E. 1975. Wheeled vehicle

    1975

  12. [20]

    Krantz, J.; Gervet, T.; Yadav, K.; Wang, A.; Paxton, C.; Mottaghi, R.; Batra, D.; Malik, J.; Lee, S.; and Chaplot, D. S. 2023. Navigating to objects specified by images. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 10916--10925

    2023

  13. [21]

    Li, J.; Wang, Q.; Wang, Y.; Jin, X.; Li, Y.; Zeng, W.; and Yang, X. 2025. Open-World Reinforcement Learning over Long Short-Term Imagination. In ICLR

    2025

  14. [22]

    E.; Jordan, M

    Liang, E.; Liaw, R.; Nishihara, R.; Moritz, P.; Fox, R.; Goldberg, K.; Gonzalez, J. E.; Jordan, M. I.; and Stoica, I. 2017. RLlib: Abstractions for Distributed Reinforcement Learning. In International Conference on Machine Learning

    2017

  15. [23]

    Lifshitz, S.; Paster, K.; Chan, H.; Ba, J.; and McIlraith, S. A. 2023. STEVE-1: A Generative Model for Text-to-Behavior in Minecraft. ArXiv, abs/2306.00937

    2023 arXiv

  16. [24]

    Lin, H.; Wang, Z.; Ma, J.; and Liang, Y. 2023. Mcu: A task-centric framework for open-ended agent evaluation in minecraft. arXiv preprint arXiv:2310.08367

    2023 arXiv

  17. [25]

    Liu, S.; Yuan, H.; Hu, M.; Li, Y.; Chen, Y.; Liu, S.; Lu, Z.; and Jia, J. 2024. RL-GPT : Integrating Reinforcement Learning and Code-as-policy. arXiv preprint arXiv:2402.19299

    2024 arXiv

  18. [26]

    A.; Veness, J.; Bellemare, M

    Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A. A.; Veness, J.; Bellemare, M. G.; Graves, A.; Riedmiller, M. A.; Fidjeland, A. K.; Ostrovski, G.; Petersen, S.; Beattie, C.; Sadik, A.; Antonoglou, I.; King, H.; Kumaran, D.; Wierstra, D.; Legg, S.; and Hassabis, D. 2015. Human-l...

    2015

  19. [27]

    I.; and Stoica, I

    Moritz, P.; Nishihara, R.; Wang, S.; Tumanov, A.; Liaw, R.; Liang, E.; Paul, W.; Jordan, M. I.; and Stoica, I. 2017. Ray: A Distributed Framework for Emerging AI Applications. ArXiv, abs/1712.05889

    2017 arXiv

  20. [28]

    Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 27730--27744

    2022

  21. [29]

    Padalkar, A.; Pooley, A.; Jain, A.; Bewley, A.; Herzog, A.; Irpan, A.; Khazatsky, A.; Rai, A.; Singh, A.; Brohan, A.; et al. 2023. Open x-embodiment: Robotic learning datasets and rt-x models. arXiv preprint arXiv:2310.08864

    2023 arXiv

  22. [30]

    Pomerleau, D. A. 1988. Alvinn: An autonomous land vehicle in a neural network. Advances in neural information processing systems, 1

    1988

  23. [31]

    Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; Sutskever, I.; et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8): 9

    2019

  24. [32]

    V.; Carion, N.; Wu, C.-Y.; Girshick, R.; Doll \'a r, P.; and Feichtenhofer, C

    Ravi, N.; Gabeur, V.; Hu, Y.-T.; Hu, R.; Ryali, C.; Ma, T.; Khedr, H.; R \"a dle, R.; Rolland, C.; Gustafson, L.; Mintun, E.; Pan, J.; Alwala, K. V.; Carion, N.; Wu, C.-Y.; Girshick, R.; Doll \'a r, P.; and Feichtenhofer, C. 2024. SAM 2: Segment Anything in Images and Videos. ...

    2024 arXiv

  25. [33]

    Ross, S.; Gordon, G.; and Bagnell, D. 2011. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, 627--635. JMLR Workshop and Conference Proceedings

    2011

  26. [34]

    Schulman, J.; Moritz, P.; Levine, S.; Jordan, M.; and Abbeel, P. 2015. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438

    2015 arXiv

  27. [35]

    Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017. Proximal Policy Optimization Algorithms. ArXiv, abs/1707.06347

    2017 arXiv

  28. [36]

    Silver, D.; Huang, A.; Maddison, C. J.; Guez, A.; Sifre, L.; van den Driessche, G.; Schrittwieser, J.; Antonoglou, I.; Panneershelvam, V.; Lanctot, M.; Dieleman, S.; Grewe, D.; Nham, J.; Kalchbrenner, N.; Sutskever, I.; Lillicrap, T. P.; Leach, M.; Kavukcuoglu, K.; Graepel, T....

    2016

  29. [37]

    E.; and Stone, P

    Taylor, M. E.; and Stone, P. 2011. An introduction to intertask transfer for reinforcement learning. Ai Magazine, 32(1): 15--15

    2011

  30. [38]

    Vithayathil Varghese, N.; and Mahmoud, Q. H. 2020 a . A survey of multi-task deep reinforcement learning. Electronics, 9(9): 1363

    2020

  31. [39]

    Vithayathil Varghese, N.; and Mahmoud, Q. H. 2020 b . A Survey of Multi-Task Deep Reinforcement Learning. Electronics, 9(9)

    2020

  32. [40]

    Ye, D.; Liu, Z.; Sun, M.; Shi, B.; Zhao, P.; Wu, H.; Yu, H.; Yang, S.; Wu, X.; Guo, Q.; et al. 2020. Mastering complex control in moba games with deep reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence, volume 34, 6672--6679

    2020

  33. [41]

    Yuan, H.; Mu, Z.; Xie, F.; and Lu, Z. 2024. Pre-training goal-based models for sample-efficient reinforcement learning. In The Twelfth International Conference on Learning Representations

    2024

  34. [42]

    Ze, Y.; Hansen, N.; Chen, Y.; Jain, M.; and Wang, X. 2023. Visual reinforcement learning with self-supervised 3d representations. IEEE Robotics and Automation Letters, 8(5): 2890--2897

    2023

  35. [43]

    Zhong, F.; Wu, K.; Wang, C.; Chen, H.; Ci, H.; Li, Z.; and Wang, Y. 2024. Unrealzoo: Enriching photo-realistic virtual worlds for embodied ai. arXiv preprint arXiv:2412.20977

    2024 arXiv

  36. [44]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  37. [45]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  38. [46]

    Pattern Anal

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    2021 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.