Pith. sign in

REVIEW 4 major objections 6 minor 48 references

FOUNDER: Grounding Foundation Models in World Models for Open-Ended Embodied Decision Making

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Grounding foundation-model embeddings into world-model goal states, with temporal-distance rewards, lets a reward-free offline agent solve text- and video-specified tasks across embodiments.

desk verdict Strong empirical integration of VLM and world models with good ablations, but the load-bearing claim of cross-domain grounding is not established, and the reward-consistency claim in the abstract is contradicted by Table 8. read the letter →

arxiv 2507.12496 v1 pith:HSYI7HOK submitted 2025-07-15 cs.RO cs.AIcs.LG

Yucen Wang , Rui Yu , Shenghua Wan , Le Gan , De-Chuan Zhan This is my paper
classification cs.ROcs.AIcs.LG
keywords foundationmodelsworldgoal-conditionedreinforcementlearningofflinereward-freeRLtemporaldistancevision-languagecross-embodimentgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FOUNDER tries to show that a single learned map from foundation-model embeddings into a world model's latent state space lets an agent solve open-ended tasks that are specified only by a text prompt or a short video, without any reward labels. The authors claim this works even when the prompt comes from another embodiment or camera viewpoint, a setting where prior methods align surface visuals and fail. They care because a reward-free offline dataset could then serve any number of user-defined goals, and task understanding becomes a matter of grounding semantics into physical states rather than hand-designing rewards. On DMC and Franka Kitchen language tasks the authors report the best normalized score on 14 of 19 setups, and on cross-embodiment video tasks on 11 of 12, with consistency checks showing the learned reward tracks the ground-truth reward.

What carries the argument

The load-bearing piece is the map Qψ: VLM embedding e → latent world-model goal state ẑ, trained to minimize the KL divergence between the mapped state distribution and the world-model posterior at matching timesteps, plus an autoencoder reconstruction loss for e. Around it sit a DreamerV3-style world model without a reward head, and a temporal-distance predictor Dθ trained on offline state pairs so that Dθ(z_t, z_{t+c}) predicts c/T and negative pairs predict 1. At decision time a prompt embedding e_g is pushed through Qψ to become goal z_g, imagined rollouts are rewarded by r_D(z_t, z_g) = −Dθ(z_t, z_g), and an actor-critic policy is trained inside the world model. The temporal-distance reward is what prevents the reward hacking that cosine similarity to a single goal state produces in dynamic tasks.

What would settle it

Run the paper's own reward-consistency protocol on cross-embodiment prompts: rank imagined trajectories by FOUNDER's pseudo-return and correlate with ground-truth returns separately for prompts whose VLM embeddings are far from the offline training embeddings. The central claim predicts high correlation in both near and far cases; if correlation drops to chance on the far-from-support prompts, the mapping is not transferring the physical goal but is memorizing nearby states.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that grounding FM representations into the world model's state space, rather than aligning them with visual observations step by step, is what makes open-ended multimodal task solving work. A mapping Qψ trained only on paired data from the target environment's offline trajectories—VLM embeddings of short observation clips paired with world-model states—transfers to out-of-distribution prompts, so a task video of one robot maps to the correct physical goal state of another. Behavior learning then runs as goal-conditioned RL in imagination, with reward given by negative predicted temporal distance to that goal. The authors argue this combination captures task semantics deeper than visual appearance, and that this is why FOUNDER succeeds on cross-embodiment, cross-view, and static-manipulation tasks where GenRL, WM-CLIP, and cosine-reward variants fail.

Load-bearing premise

The load-bearing premise is that the mapping Qψ, trained only on paired VLM embeddings and states from the target environment's own offline trajectories, transfers to prompt embeddings from other embodiments and viewpoints, so a video of a Walker running lands on a goal state that means 'run' in a Cheetah world model, and the temporal-distance model rewards progress toward that state.

Editorial extensions

If this is right

  • A single pretrained world model plus one mapping module can answer many prompt-defined tasks from the same reward-free offline dataset, since tasks become goal states rather than separate reward functions.
  • Temporal distance as reward makes dynamic tasks like Run and Flip solvable where fixed-goal cosine similarity collapses through reward hacking.
  • Cross-embodiment and cross-view prompts, including videos of a different agent or from a different camera, can be grounded as goals, so one-shot third-person imitation becomes a goal-conditioned RL problem.
  • The learned reward function is consistent enough with ground-truth reward to rank trajectories, with rank correlation of 0.54 and regret 0.07 averaged over seven DMC tasks.
  • In Minecraft, the same recipe matches or exceeds an oracle MineCLIP-IQL baseline on three of five tasks without any internet-scale FM fine-tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Our inference: the transfer claim rests on an unexamined geometric assumption about VLM embeddings—that an out-of-domain prompt embedding falls in a neighborhood where Qψ was trained; a direct test would be to measure nearest-neighbor distances between prompt embeddings and the offline trajectory embeddings, with low support predicting mapping failure.
  • Our inference: because Dθ is trained from offline trajectory pairs, the reward implicitly encodes the dataset's behavioral distribution, so tasks whose successful behavior is rare or absent in the offline data will be unreachable no matter how well the prompt maps; this makes FOUNDER's ceiling the coverage of the offline dataset, a limitation the authors acknowledge.
  • Our inference: a natural extension not tested in the paper is to use the temporal-distance reward as a dense signal in online learning or to widen VLM embedding coverage through prompt-conditioned data augmentation, since the paper's ablations show the mapping generalizes but give no recipe for when it will fail.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes FOUNDER, an offline, reward-free framework for open-ended embodied decision making. The method trains a DreamerV3-style world model on observation-action data, learns a mapping Qψ from VLM (InternVideo2) embeddings to world-model latent states, and trains a temporal-distance predictor Dθ. At task time, a text or video prompt is embedded, mapped to a goal state zg, and a goal-conditioned policy is trained inside the world model using the reward rD(zt,zg) = -Dθ(zt,zg). The paper evaluates FOUNDER on DMC locomotion tasks, Franka Kitchen, and Minecraft, including 19 language tasks, 12 cross-embodiment video tasks, cross-viewpoint tasks, and real-world video prompts, and reports that FOUNDER outperforms GenRL, WM-CLIP, and model-free baselines on most tasks. The abstract also claims that consistency of the learned reward with ground-truth reward is empirically validated.

Significance. If the central mechanism holds, FOUNDER is a valuable contribution: it integrates foundation-model semantics with world-model dynamics in a reward-free offline setting, and the reported gains over GenRL on cross-domain and cross-embodiment tasks are substantial. The paper is admirably thorough on the empirical side: multiple seeds, several benchmarks, ablations (FOUNDER w/o TempD, GenRL-TempD), additional model-free baselines, efficiency comparisons, and documented failure cases of cosine-similarity rewards in Appendix E.3. The extension to Minecraft and the comparison with MineCLIP-IQL as an oracle-style baseline are also strengths. However, the headline cross-domain capability rests on an unverified assumption about the transfer of the mapping Qψ to out-of-distribution prompt embeddings, and the paper's own reward-consistency tables contain per-task results that undercut the abstract's consistency claim. These are load-bearing issues that require additional analysis or a more measured claim.

major comments (4)
  1. [Section 4.2, Eq. (4)-(5); Section 5.2, Figure 2; Section 6] The cross-embodiment results in Figure 2 rest on the claim that Qψ, trained only on paired data (et, zt) from the target domain, generalizes to prompt embeddings from other embodiments (e.g., a Walker running video mapped to a Cheetah goal state). The paper asserts in Section 4.2 that "the generalization power of the VLM enables the mapping to extend to data from other domains," but no analysis of InternVideo2 embedding geometry, nearest-neighbor structure, or failure conditions is provided. Section 6 explicitly concedes that "prompt elements not represented in the dataset may not be properly grounded," and Walker/Stickman prompts are not represented in the Cheetah dataset, which appears to conflict with the claimed cross-domain transfer. Because the 11/12 cross-embodiment result is the headline "domain gaps" contribution, this transfer mechanism needs direct evidence: for example, nearest-neighbor analysis of prompt embeddings against target-state VLM embeddings, or a study of Qψ's mapped goals for held-out domain prompts. Without such evidence, an alternative explanation—that the TempD reward drives task-relevant behavior regardless of goal identity—cannot be excluded.
  2. [Section 5.3, Table 8; Abstract] The abstract states that "the consistency of our learned reward function with the ground-truth reward is also empirically validated," and Table 2 reports averaged consistency metrics over seven tasks. However, Table 8 shows that on "walker stand" FOUNDER achieves rank correlation -0.11 and F1 score 0.04, while Table 1 shows a normalized task score of 1.01 for this same task. The per-task consistency is therefore not merely imperfect but essentially uncorrelated in a case where the policy performs well. This is a direct counterexample to the abstract's general consistency claim. The paper should either report per-task results in the main text, qualify the claim as holding on a subset of tasks, or explain why reward consistency fails on this particular task despite high policy performance.
  3. [Section 5.3, Appendix C.4] The reward-consistency evaluation is computed on the agent's own rollouts from language-task behavior learning (Section 5.1), not on the cross-domain or cross-embodiment tasks that motivate the framework. Consequently, the evaluation does not actually test whether the mapping Qψ grounds out-of-distribution prompts; it only tests whether the pseudo-reward ranks trajectories within a single-domain language-task buffer. Moreover, Table 8 demonstrates that high normalized policy scores can coexist with near-zero reward consistency (walker stand row), so cross-domain policy performance in Figure 2 cannot by itself be read as evidence of correct goal grounding. A separate grounding metric applied to cross-domain prompts—for example, reward consistency on cross-domain rollouts, or an analysis of whether the mapped goal states are semantically interpretable—is needed to support the claim that FOUNDER captures deep-level task semantics.
  4. [Section 4.3, Eq. (7)-(8); Appendix E.3] The temporal-distance predictor Dθ is trained on offline trajectory pairs and negative pairs from different trajectories, but during behavior learning it is applied to states imagined by a policy that may visit regions of the latent space outside the offline training distribution. Appendix E.3 documents a catastrophic reward-hacking failure for the cosine-similarity variant, and Figure 8 reports consistency only for three dynamic tasks. The paper does not report a calibration analysis of Dθ on imagined out-of-distribution states, such as predicted temporal distance versus actual steps-to-go under the learned policy. Since rD is the only learning signal in Eq. (8), this calibration is load-bearing for the method's reliability; at minimum, the paper should discuss the distribution shift between Dθ's training pairs and the imagined rollouts it scores.
minor comments (6)
  1. [Section 4.2, Eq. (5)] The reconstruction loss -ln Pψ(et | ẑt) is described as an autoencoder loss, but the distribution family of Pψ over the VLM embedding space is not specified; please state whether a Gaussian, Laplacian, or other density is used and how the loss is normalized across the high-dimensional embedding.
  2. [Figure 2 caption] The caption reads "Each row corresponds to one of theRun or Flip tasks," with a missing space after "the"; also, the subplot axis labels are dense and hard to read at print size—please enlarge or restructure.
  3. [Table 1, Kitchen Burner / Table 3] Table 3 lists the prompt "the burner becomes red" but the dataset composition in Table 4 lists "kitchen bottom burner"; please align the task naming between tables and appendix.
  4. [Section 5.3, Table 2] The header "Regret↓" is used without defining the value of k; Appendix C.4 defines Regret@k, but the main text should state which k is used in Table 2.
  5. [Appendix C.1] The choice to use the mean of the goal distribution for DMC/Kitchen but to sample goals for Minecraft is stated without justification; please add a brief explanation, since it affects the interpretation of the Minecraft results.
  6. [Section 5.1, GenRL-TempD discussion] The claim that "GenRL-TempD does not outperform GenRL, indicating that temporal distance alone does not contribute significantly" is weakened by the fact that GenRL-TempD uses GenRL's visual-tokenizer-only states, as described in Appendix C.2, so it is not a clean evaluation of temporal distance independent of the representation; please acknowledge this confound.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the mapping, temporal-distance reward, and goal-conditioned policy are trained from reward-free offline data, and the claimed results are tested against external environment rewards.

full rationale

The central derivation chain is self-contained. The mapping function Qψ is trained by Equations (4)-(5) on paired data (e_t, z_t) obtained from offline trajectories and the pre-trained world model; the goal state for a prompt is produced by Equation (6) from the VLM embedding. The cross-embodiment and cross-view results in Section 5.2 and Figure 2 are evaluated with ground-truth environment rewards that were not used in training any component of FOUNDER. The temporal-distance predictor Dθ is trained by Equation (7) to predict normalized timestep distances c/T and 1 from offline trajectory pairs, without access to ground-truth rewards, and the reward rD = -Dθ in Equation (8) is the policy objective. The claimed consistency with ground-truth rewards (Tables 2 and 8) is measured on the agent's own rollouts, but this is an empirical evaluation rather than a construction: the pseudo-reward is not defined in terms of the ground-truth reward, and Table 8 contains clear failures (e.g., walker stand Corr -0.11, F1 0.04) that show the metrics are not forced. The assumption that Qψ transfers to out-of-distribution prompt embeddings is stated explicitly and tested empirically; while it is a correctness risk with limited geometric analysis, it is not a definitional reduction. Self-citations (Wan et al. 2023/2024, Huang et al. 2024) are background references for imitation settings and MOPO-style penalties, and they are not load-bearing for the paper's central claim. Thus, no circular step is exhibited.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

FOUNDER introduces no new physical entities; its constructs are neural modules (mapping Qψ, temporal distance Dθ) and WM latent goal states, all learned from offline data. The free-parameter burden is moderate: loss balancing and several hyperparameters (ensemble size, goal re-sampling period, negative-pair targets) are unablated, and the reward scale is normalized without a stated scheme. The load-bearing assumptions are data coverage, WM fidelity, VLM embedding transferability, and the validity of temporal distance as a reachability proxy.

free parameters (6)
  • KL vs reconstruction loss weighting (Eq. 4 vs Eq. 5) = not specified
    The mapping objective combines DKL alignment and autoencoder reconstruction without stated coefficients; this balance materially affects whether mapped goal states preserve VLM semantics or WM dynamics, yet the paper gives no values.
  • Mapping ensemble size = 5
    Appendix C.1: an ensemble of 5 stochastic-state output heads is used, following Plan2Explore; the choice is not ablated.
  • Goal re-sampling period = 8 steps
    Appendix C.1: the goal state is re-sampled every 8 steps during policy rollouts for 'short-period decision-making consistency'; not ablated.
  • Temporal distance normalization and negative-pair target = c/T for positive pairs; 1 for negative pairs
    Eq. 7: distances are normalized by sequence length T and negative pairs are assigned target 1 (max distance), a modeling choice inherited from LEXA that shapes the reward scale.
  • VLM frame window k = 8 frames
    Section 4.2 and 5.2: k is the maximum number of frames InternVideo2 can process; the prompt is therefore a single 8-frame clip.
  • TempD reward normalization = unspecified
    Appendix C.1 states normalization is applied on temporal-distance rewards for efficient policy learning, but the scheme is not described; this affects the RL objective.
assumptions (4)
  • domain assumption The offline dataset contains trajectories covering the evaluated behaviors
    Table 4 shows task-specific RL buffers (e.g., cheetah run 820K, walker run/walk/stand 500K each) seed the data with the same behaviors evaluated in Table 1; without this coverage the goals cannot be grounded (Section 6 concedes the upper bound).
  • domain assumption The DreamerV3 world model, trained without rewards, yields accurate imagination rollouts for policy learning and goal distances
    Section 4.1 and 4.3: behavior learning runs entirely inside the WM; no MOPO-style penalty for unreliable transitions is used (Appendix C.1), so model error is assumed small.
  • domain assumption InternVideo2 embeddings are semantically aligned across embodiments, viewpoints, and text/visual modalities
    Section 4.2: the mapping Qψ is trained on in-distribution pairs and is assumed to generalize to out-of-distribution prompt embeddings via 'the generalization power of the VLM'; this assumption carries the cross-domain results of Section 5.2.
  • domain assumption Temporal distance learned on offline trajectory pairs is a valid reachability proxy on imagined states
    Eq. 7-8: Dθ is trained on offline pairs (zt, zt+c) and negative pairs, but deployed on imagined states near goals; Section E.3 shows the alternative cosine reward fails by reward hacking, so the reward design's validity is an empirical rather than guaranteed property.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FOUNDER: Grounding Foundation Models in World Models for Open-Ended Embodied Decision Making." pith.science (2026). https://pith.science/paper/HSYI7HOK

@misc{pith2026250712496,
  author       = {Pith},
  title        = {Pith review of: FOUNDER: Grounding Foundation Models in World Models for Open-Ended Embodied Decision Making},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HSYI7HOK}},
  note         = {Machine review of arXiv:2507.12496}
}
read the original abstract

Foundation Models (FMs) and World Models (WMs) offer complementary strengths in task generalization at different levels. In this work, we propose FOUNDER, a framework that integrates the generalizable knowledge embedded in FMs with the dynamic modeling capabilities of WMs to enable open-ended task solving in embodied environments in a reward-free manner. We learn a mapping function that grounds FM representations in the WM state space, effectively inferring the agent's physical states in the world simulator from external observations. This mapping enables the learning of a goal-conditioned policy through imagination during behavior learning, with the mapped task serving as the goal state. Our method leverages the predicted temporal distance to the goal state as an informative reward signal. FOUNDER demonstrates superior performance on various multi-task offline visual control benchmarks, excelling in capturing the deep-level semantics of tasks specified by text or videos, particularly in scenarios involving complex observations or domain gaps where prior methods struggle. The consistency of our learned reward function with the ground-truth reward is also empirically validated. Our project website is https://sites.google.com/view/founder-rl.

Figures

Figures reproduced from arXiv: 2507.12496 by the authors.

Figure 1
Figure 1. FOUNDER encompasses pretraining phase and behavior learning phase. During pretraining, we first train a Dreamer-style WM, then use it to learn a mapping function and a temporal distance predictor. During behavior learning, the VLM representation of a task prompt is mapped to a goal state within the WM and a goal-conditioned policy is learned through imagination, where the predicted temporal distance to the goal stat… view at source ↗
Figure 2
Figure 2. Normalized evaluation performance on cross-embodiment tasks built upon DMC. Each row corresponds to one of the Run or Flip tasks, while each column represents the domain in which the agent is evaluated. Each subplot presents the results of respectively using videos from the remaining two domains as task prompts. This yields 6 domain combinations: (Cheetah, Walker), (Cheetah, Stickman), (Walker, Cheetah), (Walker, St… view at source ↗
Figure 3
Figure 3. Performance of FOUNDER and baselines over 4 seeds on two tasks in Kitchen. The agent’s observations is captured from the Main viewpoint, while task video prompts are provided from the Right or Top viewpoints, yielding 4 cross-view tasks. as is demonstrated by the low performance of FOUNDER w/o TempD. Moreover, when studying the failure cases of GenRL and FOUNDER w/o TempD, we find that their failures can be attribut… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Performance evaluation of FOUNDER and baselines over 4 seeds across 5 cross-viewpoint video tasks in DMC. Visualizations of video prompts indicating the task semantics and the agent’s observation in the corresponding environment is presented at first row. ios where the…
Figure 5
Figure 5. Figure 5: Performance of FOUNDER and baselines over 3 seeds across 5 tasks in Minecraft during behavior learning. Each task is specified in a text prompt. The solid curves and the shaded region indicate the average episodic success rates and the 95% confidence intervals across d…
Figure 6
Figure 6. Figure 6: Demonstration of the Minecraft environment, where the agent performs tasks such as chopping trees, hunting cows, milking cows, hunting sheep, and shearing sheep. A. Environments and Tasks Minecraft environment. The Minecraft environment is based on a popular sandbox ga…
Figure 7
Figure 7. Figure 7: Testing returns of the FOUNDER w/o TempD agent during behavior learning stage for Walker Run, Cheetah Run and Stickman Walk in terms of real performance and pseudo return. As the agent maximizes pseudo-returns derived from cosine similarity (blue curves), the agent’s p…
Figure 8
Figure 8. Figure 8: Testing returns of the FOUNDER agent during behavior learning stage for Walker Run, Cheetah Run and Stickman Walk in terms of real performance and pseudo return, when using TempD-based rewards. The trends of the two return curves are now generally consistent. to the ag…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 26 canonical work pages

  1. [1]

    Vision-language models as a source of rewards

    Baumli, K., Baveja, S., Behbahani, F., Chan, H., Comanici, G., Flennerhag, S., Gazeau, M., Holsheimer, K., Horgan, D., Laskin, M., et al. Vision-language models as a source of rewards. arXiv preprint arXiv:2312.09187, 2023

    arXiv 2023

  2. [2]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Chen, X., Choromanski, K., Ding, T., Driess, D., Dubey, A., Finn, C., et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818, 2023 a

    arXiv 2023

  3. [3]

    Do as i can, not as i say: Grounding language in robotic affordances

    Brohan, A., Chebotar, Y., Finn, C., Hausman, K., Herzog, A., Ho, D., Ibarz, J., Irpan, A., Jang, E., Julian, R., et al. Do as i can, not as i say: Grounding language in robotic affordances. In Conference on robot learning, pp.\ 287--318. PMLR, 2023 b

    2023

  4. [4]

    Groot: Learning to follow instructions by watching gameplay videos

    Cai, S., Zhang, B., Wang, Z., Ma, X., Liu, A., and Liang, Y. Groot: Learning to follow instructions by watching gameplay videos. arXiv preprint arXiv:2310.08235, 2023

    arXiv 2023

  5. [5]

    Cross-modal domain adaptation for cost-efficient visual reinforcement learning

    Chen, X.-H., Jiang, S., Xu, F., Zhang, Z., and Yu, Y. Cross-modal domain adaptation for cost-efficient visual reinforcement learning. Advances in Neural Information Processing Systems, 34: 0 12520--12532, 2021

    work page 2021

  6. [6]

    Can foundation models perform zero-shot task specification for robot manipulation? In Learning for dynamics and control conference, pp.\ 893--905

    Cui, Y., Niekum, S., Gupta, A., Kumar, V., and Rajeswaran, A. Can foundation models perform zero-shot task specification for robot manipulation? In Learning for dynamics and control conference, pp.\ 893--905. PMLR, 2022

    work page 2022

  7. [7]

    Guiding pretraining in reinforcement learning with large language models

    Du, Y., Watkins, O., Wang, Z., Colas, C., Darrell, T., Abbeel, P., Gupta, A., and Andreas, J. Guiding pretraining in reinforcement learning with large language models. In International Conference on Machine Learning, pp.\ 8657--8677. PMLR, 2023

    work page 2023

  8. [8]

    Minedojo: Building open-ended embodied agents with internet-scale knowledge

    Fan, L., Wang, G., Jiang, Y., Mandlekar, A., Yang, Y., Zhu, H., Tang, A., Huang, D.-A., Zhu, Y., and Anandkumar, A. Minedojo: Building open-ended embodied agents with internet-scale knowledge. Advances in Neural Information Processing Systems, 35: 0 18343--18362, 2022

    work page 2022

Show all 48 references
  1. [9]

    and Gu, S

    Fujimoto, S. and Gu, S. S. A minimalist approach to offline reinforcement learning. Advances in neural information processing systems, 34: 0 20132--20145, 2021

    2021

  2. [10]

    and Kong, R

    Gao, C.-X. and Kong, R. OfflineRL-Lib: Benchmarked Implementations of Offline RL Algorithms , February 2023. URL https://github.com/typoverflow/OfflineRL-Lib

    2023

  3. [11]

    Relay policy learning: Solving long-horizon tasks via imitation and reinforcement learning

    Gupta, A., Kumar, V., Lynch, C., Levine, S., and Hausman, K. Relay policy learning: Solving long-horizon tasks via imitation and reinforcement learning. arXiv preprint arXiv:1910.11956, 2019

    1910 arXiv

  4. [12]

    Learning latent dynamics for planning from pixels

    Hafner, D., Lillicrap, T., Fischer, I., Villegas, R., Ha, D., Lee, H., and Davidson, J. Learning latent dynamics for planning from pixels. In International conference on machine learning, pp.\ 2555--2565. PMLR, 2019

    2019

  5. [13]

    Deep hierarchical planning from pixels

    Hafner, D., Lee, K.-H., Fischer, I., and Abbeel, P. Deep hierarchical planning from pixels. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems, volume 35, pp.\ 26091--26104. Curran Associates, Inc., 2022

    2022

  6. [14]

    Mastering diverse domains through world models

    Hafner, D., Pasukonis, J., Ba, J., and Lillicrap, T. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104, 2023

    2023 arXiv

  7. [15]

    Dynamical distance learning for semi-supervised and unsupervised skill discovery

    Hartikainen, K., Geng, X., Haarnoja, T., and Levine, S. Dynamical distance learning for semi-supervised and unsupervised skill discovery. arXiv preprint arXiv:1907.08225, 2019

    1907 arXiv

  8. [16]

    and Sadigh, D

    Hu, H. and Sadigh, D. Language instructed reinforcement learning for human-ai coordination. In International Conference on Machine Learning, pp.\ 13584--13598. PMLR, 2023

    2023

  9. [17]

    Sensor: Imitate third-person expert's behaviors via active sensoring

    Huang, K., Shao, M., Wan, S., Sun, H.-H., Feng, S., Gan, L., and Zhan, D.-C. Sensor: Imitate third-person expert's behaviors via active sensoring. arXiv preprint arXiv:2404.03386, 2024

    2024 arXiv

  10. [18]

    P., Littman, M

    Kaelbling, L. P., Littman, M. L., and Cassandra, A. R. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101 0 (1-2): 0 99--134, 1998

    1998

  11. [19]

    Motif: Intrinsic motivation from artificial intelligence feedback

    Klissarov, M., D'Oro, P., Sodhani, S., Raileanu, R., Bacon, P.-L., Vincent, P., Zhang, A., and Henaff, M. Motif: Intrinsic motivation from artificial intelligence feedback. arXiv preprint arXiv:2310.00166, 2023

    2023 arXiv

  12. [21]

    Offline reinforcement learning with implicit q-learning

    Kostrikov, I., Nair, A., and Levine, S. Offline reinforcement learning with implicit q-learning. arXiv preprint arXiv:2110.06169, 2021 b

    2021 arXiv

  13. [22]

    M., Bullard, K., and Sadigh, D

    Kwon, M., Xie, S. M., Bullard, K., and Sadigh, D. Reward design with language models. arXiv preprint arXiv:2303.00001, 2023

    2023 arXiv

  14. [23]

    R., Mesnard, T., Ferret, J., Bishop, C., Hall, E., Carbune, V., and Rastogi, A

    Lee, H., Phatale, S., Mansoor, H., Lu, K. R., Mesnard, T., Ferret, J., Bishop, C., Hall, E., Carbune, V., and Rastogi, A. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. 2023

    2023

  15. [24]

    Rl-gpt: Integrating reinforcement learning and code-as-policy

    Liu, S., Yuan, H., Hu, M., Li, Y., Chen, Y., Liu, S., Lu, Z., and Jia, J. Rl-gpt: Integrating reinforcement learning and code-as-policy. arXiv preprint arXiv:2402.19299, 2024

    2024 arXiv

  16. [25]

    J., Rudner, T

    Lu, C., Ball, P. J., Rudner, T. G., Parker-Holder, J., Osborne, M. A., and Teh, Y. W. Challenges and opportunities in offline reinforcement learning from visual observations. arXiv preprint arXiv:2206.04779, 2022

    2022 arXiv

  17. [26]

    A survey on model-based reinforcement learning

    Luo, F.-M., Xu, T., Lai, H., Chen, X.-H., Zhang, W., and Yu, Y. A survey on model-based reinforcement learning. arXiv preprint arXiv:2206.09328, 2022

    2022 arXiv

  18. [27]

    Harmonydream: Task harmonization inside world models

    Ma, H., Wu, J., Feng, N., Xiao, C., Li, D., Hao, J., Wang, J., and Long, M. Harmonydream: Task harmonization inside world models. arXiv preprint arXiv:2310.00344, 2023

    2023 arXiv

  19. [28]

    Choreographer: Learning and adapting skills in imagination

    Mazzaglia, P., Verbelen, T., Dhoedt, B., Lacoste, A., and Rajeswar, S. Choreographer: Learning and adapting skills in imagination. arXiv preprint arXiv:2211.13350, 2022

    2022 arXiv

  20. [29]

    Genrl: Multimodal-foundation world models for generalization in embodied agents

    Mazzaglia, P., Verbelen, T., Dhoedt, B., Courville, A., and Rajeswar, S. Genrl: Multimodal-foundation world models for generalization in embodied agents. arXiv preprint arXiv:2406.18043, 2024

    2024 arXiv

  21. [30]

    Discovering and achieving goals via world models

    Mendonca, R., Rybkin, O., Daniilidis, K., Hafner, D., and Pathak, D. Discovering and achieving goals via world models. Advances in Neural Information Processing Systems, 34: 0 24379--24391, 2021

    2021

  22. [31]

    M., Broekens, J., Plaat, A., Jonker, C

    Moerland, T. M., Broekens, J., Plaat, A., Jonker, C. M., et al. Model-based reinforcement learning: A survey. Foundations and Trends in Machine Learning , 16 0 (1): 0 1--118, 2023

    2023

  23. [32]

    A., Maccarini, M., Forgione, M., Piga, D., Spahiu, B., and Roveda, L

    Moroncelli, A., Soni, V., Shahid, A. A., Maccarini, M., Forgione, M., Piga, D., Spahiu, B., and Roveda, L. Integrating reinforcement learning with foundation models for autonomous robotics: Methods and perspectives. arXiv preprint arXiv:2410.16411, 2024

  24. [33]

    Foundation policies with hilbert representations

    Park, S., Kreiman, T., and Levine, S. Foundation policies with hilbert representations. In Proceedings of the 41st International Conference on Machine Learning, pp.\ 39737--39761, 2024

    2024

  25. [34]

    Mp5: A multi-modal open-ended embodied system in minecraft via active perception

    Qin, Y., Zhou, E., Liu, Q., Yin, Z., Sheng, L., Zhang, R., Qiao, Y., and Shao, J. Mp5: A multi-modal open-ended embodied system in minecraft via active perception. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 16307--16316. IEEE, 2024

    2024

  26. [35]

    Vision-language models are zero-shot reward models for reinforcement learning

    Rocamonde, J., Montesinos, V., Nava, E., Perez, E., and Lindner, D. Vision-language models are zero-shot reward models for reinforcement learning. arXiv preprint arXiv:2310.12921, 2023

    2023 arXiv

  27. [36]

    Planning to explore via self-supervised world models

    Sekar, R., Rybkin, O., Daniilidis, K., Abbeel, P., Hafner, D., and Pathak, D. Planning to explore via self-supervised world models. In International conference on machine learning, pp.\ 8583--8592. PMLR, 2020

    2020

  28. [37]

    Roboclip: One demonstration is enough to learn robot policies

    Sontakke, S., Zhang, J., Arnold, S., Pertsch, K., B y k, E., Sadigh, D., Finn, C., and Itti, L. Roboclip: One demonstration is enough to learn robot policies. Advances in Neural Information Processing Systems, 36, 2024

    2024

  29. [38]

    C., Abbeel, P., and Sutskever, I

    Stadie, B. C., Abbeel, P., and Sutskever, I. Third-person imitation learning. arXiv preprint arXiv:1703.01703, 2017

    2017 arXiv

  30. [39]

    Tassa, Y., Doron, Y., Muldal, A., Erez, T., Li, Y., Casas, D. d. L., Budden, D., Abdolmaleki, A., Merel, J., Lefrancq, A., et al. Deepmind control suite. arXiv preprint arXiv:1801.00690, 2018

    2018 arXiv

  31. [40]

    Team, O. E. L., Stooke, A., Mahajan, A., Barros, C., Deck, C., Bauer, J., Sygnowski, J., Trebacz, M., Jaderberg, M., Mathieu, M., et al. Open-ended learning leads to generally capable agents. arXiv preprint arXiv:2107.12808, 2021

    2021 arXiv

  32. [41]

    Semail: eliminating distractors in visual imitation via separated models

    Wan, S., Wang, Y., Shao, M., Chen, R., and Zhan, D.-C. Semail: eliminating distractors in visual imitation via separated models. In International Conference on Machine Learning, pp.\ 35426--35443. PMLR, 2023

    2023

  33. [42]

    Semopo: Learning high-quality model and policy from low-quality offline visual datasets

    Wan, S., Chen, Z., Gan, L., Feng, S., and Zhan, D.-C. Semopo: Learning high-quality model and policy from low-quality offline visual datasets. arXiv preprint arXiv:2406.09486, 2024

    2024 arXiv

  34. [43]

    Internvideo2: Scaling foundation models for multimodal video understanding

    Wang, Y., Li, K., Li, X., Yu, J., He, Y., Chen, G., Pei, B., Zheng, R., Wang, Z., Shi, Y., et al. Internvideo2: Scaling foundation models for multimodal video understanding. In European Conference on Computer Vision, pp.\ 396--416. Springer, 2025

    2025

  35. [44]

    Daydreamer: World models for physical robot learning, 2022

    Wu, P., Escontrela, A., Hafner, D., Goldberg, K., and Abbeel, P. Daydreamer: World models for physical robot learning, 2022. URL https://arxiv.org/abs/2206.14176

    2022 arXiv

  36. [45]

    Y., Levine, S., Finn, C., and Ma, T

    Yu, T., Thomas, G., Yu, L., Ermon, S., Zou, J. Y., Levine, S., Finn, C., and Ma, T. Mopo: Model-based offline policy optimization. Advances in Neural Information Processing Systems, 33: 0 14129--14142, 2020

    2020

  37. [46]

    Whale-x: Learning scalable embodied world models with enhanced generalizability

    Zhang, Z., Chen, R., Ye, J., Sun, Y., Wang, P., Pang, J.-C., Li, K., Liu, T.-S., Lin, H., and Yu, Y. Whale-x: Learning scalable embodied world models with enhanced generalizability. 2024

    2024

  38. [47]

    Learning from visual observation via offline pretrained state-to-go transformer

    Zhou, B., Li, K., Jiang, J., and Lu, Z. Learning from visual observation via offline pretrained state-to-go transformer. Advances in Neural Information Processing Systems, 36, 2024

    2024

  39. [48]

    and Liu, X.-Y

    Zhou, Z.-H. and Liu, X.-Y. Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Transactions on knowledge and data engineering, 18 0 (1): 0 63--77, 2005

    2005

  40. [49]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.