Pith. sign in

REVIEW 3 major objections 5 minor 50 references

Sample Efficient Reinforcement Learning via Large Vision Language Model Distillation

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Large vision-language model advice cuts reinforcement learning sample use by about 2.5x

desk verdict A clean, modest empirical result—LVLM soft-action distillation cuts MiniGrid sample counts 2-3x—but the missing teacher-informativeness control leaves the causal role of the LVLM's knowledge unproven. read the letter →

arxiv 2505.11221 v1 pith:MWRAIUQU submitted 2025-05-16 cs.LG

Donghoon Lee , Tung M. Luu , Younghwan Lee , Chang D. Yoo This is my paper · ORCID
classification cs.LG
keywords reinforcementlearningsampleefficiencyknowledgedistillationlargevision-languagemodelslanguageMiniGridsoftactionprobabilitiespolicy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LVLM2P is a training framework that uses a large vision-language model as a teacher during reinforcement learning. As the student policy collects trajectories, the teacher looks at images of the environment and returns soft probabilities over actions, and the student is trained to imitate those suggestions while also optimizing reward. The paper claims this reduces the number of environment interactions needed to reach a given return by an average of 2.56x for PPO and 2.86x for A2C across four MiniGrid tasks. The practical point is that a small student policy can inherit visual understanding from a much larger model and then run without the teacher at deployment time, avoiding both wasteful exploration and the cost of online large-model inference.

What carries the argument

The load-bearing object is the combined training loss. The student updates by maximizing the RL objective plus a distillation term, $$ \mathcal{L}(\pi_\$\theta$) = \mathcal{L}_{\mathrm{RL}}(\pi_\$\theta$) + \$\lambda$ \, D_{\mathrm{KL}}\big(\pi_T(\cdot|o,\ell_{\mathrm{in}})\,\big\|\,\pi_\$\theta$(\cdot|s)\big), $$ where $\pi_T$ is the LVLM teacher policy, $\ell_{\mathrm{in}}$ is the textual prompt, $o$ is the visual observation, and $s$ is the student's state. The two-stage prompt converts an image into a soft distribution over actions, and the KL term transfers the teacher's relative confidence across all actions; the RL term keeps the student reward-driven. The teacher is queried online on batches sampled from the student's own trajectory buffer, so the process needs no pretrained task-specific teacher and no textual state description.

What would settle it

Train LVLM2P and vanilla PPO/A2C on a held-out task where the teacher keeps a top-down full map but the student sees only an egocentric, partially observable view, and compare environment steps to a fixed success threshold; if the speedup vanishes, the reported gains depend on the teacher's privileged viewpoint rather than on distillation itself.

Watch

Extended reading notes

Core claim

The central claim is that prompting a pretrained LVLM with an image and a two-stage text prompt—first a scene-analysis question, then an action-inference question with few-shot examples—produces a teacher policy whose soft action probabilities accelerate RL when added as a KL-divergence term to PPO/A2C objectives. The paper reports a teacher success rate of about 0.96 and student sample-efficiency gains averaging 2.56x for PPO and 2.86x for A2C over vanilla baselines. Because the student keeps optimizing the RL objective, it can override teacher mistakes, and because the teacher reads the image directly, no handcrafted textual state descriptors are needed.

Load-bearing premise

The speedup assumes that actions suggested by a large model looking at a full image of the scene remain sensible for a student that perceives the environment through its own state representation; the paper checks this on only four MiniGrid tasks, so the 2–3x gain may not transfer when that alignment breaks.

Editorial extensions

If this is right

  • PPO and A2C integrated with LVLM2P reach target returns with roughly 2.1–2.9 times fewer environment steps on the four MiniGrid tasks than the vanilla algorithms.
  • The distilled student policy operates without the LVLM at test time, so deployment cost stays that of a small policy network.
  • Soft teacher probabilities outperform hard one-hot labels in the LavaGap ablation, and a small distillation weight $\lambda=0.01$ gives the best success rate; larger weights hurt performance.
  • Because the teacher consumes the raw image, the method removes the need for handcrafted textual descriptions and can be layered onto either PPO or A2C.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The distillation objective needs only a teacher distribution over the student's action space, so porting LVLM2P to off-policy algorithms (for example DQN) or continuous-action policies is a direct next step not tested in this paper.
  • A cost-saving variant would anneal or disable LVLM queries once the student's own return approaches the teacher's level, reducing the number of large-model calls during training; this paper keeps the teacher active for all training steps.
  • Environments with stronger partial observability or action vocabularies that differ from the teacher prompt would test whether the 2.5x speedup is a property of the distillation mechanism or of the four MiniGrid tasks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper proposes LVLM2P, a framework that distills soft action probabilities from a pretrained vision-language model into a compact reinforcement-learning student. During training, a Gemini-1.5-Flash teacher is prompted with the agent's visual observation and a task-specific analysis/action-inference prompt, and the student optimizes L_RL + lambda * D_KL(pi_T || pi_theta). Experiments on four MiniGrid tasks compare PPO and A2C with and without LVLM2P. The paper reports average sample-efficiency improvements of 2.56x for PPO and 2.86x for A2C, and ablations show that soft targets outperform hard targets and that lambda = 0.01 gives the best success rate. The claimed contribution is that the framework reduces sample complexity without requiring the large teacher at test time and without hand-crafted textual state descriptors.

Significance. If the attribution is supported, the framework would be a practical way to transfer internet-scale visual-linguistic knowledge into small deployable policies without test-time access to the teacher. The paper provides code, evaluates on environment reward rather than teacher reward, and includes ablations of the distillation coefficient and target encoding; these are genuine strengths. However, the central causal claim that LVLM knowledge, rather than the KL regularizer or the teacher's privileged full-observation coordinates, drives the speedup is not yet established, and the statistical basis for the headline ratios is thin. The contribution is promising but requires additional controls and experiments before the central mechanism can be accepted.

major comments (3)
  1. [Section V, Fig. 4] The ablation study in Fig. 4 varies only the distillation coefficient lambda and hard versus soft targets; there is no control in which the teacher is replaced by an uninformative smooth distribution (e.g., uniform over actions) or by a scripted non-LVLM planner given the same fully-observable image. Without such a control, the 2.56x/2.86x speedups in Eq. (1) could be attributed to the KL term acting as label smoothing or entropy regularization, which is known to help early exploration, or to the privileged global coordinates in the teacher's input, rather than to LVLM reasoning. This control is load-bearing for the paper's central mechanism and should be added before the claim that LVLM distillation reduces sample complexity can be evaluated.
  2. [Section V, Fig. 3] The headline sample-efficiency ratios are computed from the curves in the bottom row of Fig. 3, but the bottom row reports only mean values without error bars, confidence intervals, or a statistical test on the ratios. The caption states that three seeds were used, so the top-row standard-deviation ranges exist; the same error propagation should be applied to the sample-efficiency estimates. Reporting the per-seed number of samples needed to reach each threshold would make it possible to judge whether 2.56x and 2.86x are robust effects or artifacts of the chosen thresholds.
  3. [Section V setup and Fig. 2] The teacher receives a fully-observable view of the environment (Section V setup), while the student policy trains on its own state representation, and Dynamic Obstacles is described as partially observable. The paper does not test whether the teacher's privileged observations remain aligned with the student's state when observability differs. Moreover, the claim that manual textual descriptors are eliminated (Abstract, Section III) is overstated: the prompt in Fig. 2 is hand-crafted per task with object names, colors, coordinate questions, and few-shot examples. The authors should either quantify the prompt-engineering cost or soften the claim, and should discuss or experimentally test transfer to other observation spaces.
minor comments (5)
  1. [Section I] There are several typos, including 'proprosed' in the first paragraph and 'intergrated' in Section V; these should be corrected.
  2. [Fig. 2] The action list in the prompt contains 'Go Forward' twice and does not include 'Turn left', while the example output includes 'Turn left'; this inconsistency should be fixed.
  3. [Section V, Fig. 4] The lambda ablation is shown only for PPO, not for A2C; the text should state whether the same lambda value was used for A2C in the main experiments.
  4. [References] References [24] and [25] have malformed author fields ('J. at al' and 'V. et al') and should be completed.
  5. [Section V] The paper does not report the wall-clock time, API cost, or number of teacher queries required during training; since the introduction emphasizes resource-constrained deployment, the authors should at least clarify that the reported sample-efficiency metric counts environment steps only and not teacher-query cost.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the sample-efficiency claim is an empirical comparison against environment reward, not a construction-level reduction.

full rationale

The paper's central claim is an empirical speedup measurement. The student objective L(pi_theta) = L_RL(pi_theta) + lambda D_KL(pi_T(·|o,l_in) || pi_theta(·|s)) (Eq. 1) is optimized during training, and sample efficiency is measured directly from MiniGrid success-rate and mean-return curves against vanilla PPO and A2C baselines. The teacher pi_T is a pretrained Gemini-1.5-Flash model queried with an image and a hand-written prompt; it is external to the student's parameters and is not fitted to the reported speedup figures. The 2.56x (PPO) and 2.86x (A2C) improvements are computed from environment-step thresholds in Fig. 3, not derived algebraically from the distillation loss or from any fitted identity. The hyperparameter lambda is tuned in an ablation (Fig. 4), but that is a standard hyperparameter choice, not a prediction that reduces to its own input by construction. The authors' self-citations [5]-[7] support background statements about sample inefficiency and are not load-bearing for the main result. No uniqueness theorem or privileged ansatz is imported from the authors' prior work, and the method is not a renaming of a known result: it is an empirical combination of an external LVLM teacher with a standard RL objective. The absence of a teacher-informativeness control (for example, replacing the LVLM with a random or scripted privileged planner) is a legitimate experimental concern about causal attribution, but it is not circularity under the specified definitions: the paper's 'derivation' is an empirical comparison against environment reward, and its conclusion is not already contained in its premises.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The method introduces no new physical entities. It depends on two hand-tuned items (distillation weight lambda and per-task prompt templates) and three domain assumptions about information alignment, teacher usefulness, and task representativeness. These are standard for an empirical RL paper but should be stated when assessing transfer.

free parameters (2)
  • Distillation coefficient lambda = 0.01 (chosen on LavaGap)
    Controls the trade-off between reward and distillation; optimal value is task and hyperparameter specific and selected from a small ablation grid on a single environment.
  • Task-specific prompt templates = Hand-crafted per task (e.g., analysis questions and few-shot examples)
    The teacher's behavior is controlled by these prompts; different prompts would yield different soft labels and different speedups, making them a manually chosen input to the method.
assumptions (3)
  • domain assumption The student policy's state representation contains enough information to imitate the teacher's image-based action distribution.
    The teacher sees a fully-observable view while the student acts on its own state; if the student's observations are partial, the teacher's suggested actions may not be groundable. The paper does not quantify this alignment. See Sections III and V.
  • domain assumption The LVLM teacher's soft probabilities provide a useful learning signal beyond the environment reward.
    The entire speedup relies on the teacher being informative; the paper reports a teacher success rate around 0.96 on these tasks but does not demonstrate robustness to a weaker or misaligned teacher.
  • domain assumption MiniGrid sparse-reward tasks are representative of target decision-making domains.
    The conclusion about practical applicability is extrapolated from four grid-world tasks; real-world vision, continuous actions, and long-horizon tasks are assumed to behave similarly.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sample Efficient Reinforcement Learning via Large Vision Language Model Distillation." pith.science (2026). https://pith.science/paper/MWRAIUQU

@misc{pith2026250511221,
  author       = {Pith},
  title        = {Pith review of: Sample Efficient Reinforcement Learning via Large Vision Language Model Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MWRAIUQU}},
  note         = {Machine review of arXiv:2505.11221}
}
read the original abstract

Recent research highlights the potential of multimodal foundation models in tackling complex decision-making challenges. However, their large parameters make real-world deployment resource-intensive and often impractical for constrained systems. Reinforcement learning (RL) shows promise for task-specific agents but suffers from high sample complexity, limiting practical applications. To address these challenges, we introduce LVLM to Policy (LVLM2P), a novel framework that distills knowledge from large vision-language models (LVLM) into more efficient RL agents. Our approach leverages the LVLM as a teacher, providing instructional actions based on trajectories collected by the RL agent, which helps reduce less meaningful exploration in the early stages of learning, thereby significantly accelerating the agent's learning progress. Additionally, by leveraging the LVLM to suggest actions directly from visual observations, we eliminate the need for manual textual descriptors of the environment, enhancing applicability across diverse tasks. Experiments show that LVLM2P significantly enhances the sample efficiency of baseline RL algorithms.

Figures

Figures reproduced from arXiv: 2505.11221 by the authors.

Figure 1
Figure 1. Given trajectories sampled by the student agent, we query the LVLM teacher during the RL update. The LVLM [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An example of a prompt in GoToDoor: In the first [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Top: The mean success rate, along with the standard deviation range, across all four environments (the higher the better). Bottom: Number of training samples needed with respect to mean return for all four environment (the lower the better). Each method was evaluated using three random seeds. agents to locate and approach the correct door as indicated by a textual cue, assessing their navigation skills in response t… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Ablation study of LVLM2P : Comparison of hard and [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 26 canonical work pages

  1. [48]

    Kickstarting deep reinforcement learning,

    S. Schmitt, J. J. Hudson, A. Zidek, S. Osindero, C. Doersch, W. M. Czarnecki, J. Z. Leibo, H. Kuttler, A. Zisserman, K. Simonyan et al. , “Kickstarting deep reinforcement learning,” arXiv:1803.03835, 2018

    arXiv 2018

  2. [1]

    Mastering the game of go with deep neural networks and tree search,

    D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V . Panneershelvam, M. Lanctot et al., “Mastering the game of go with deep neural networks and tree search,” Nature, 2016

    2016

  3. [2]

    Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning,

    C. You, J. Lu, D. Filev, and P. Tsiotras, “Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning,” RAS, 2019

    work page 2019

  4. [3]

    Qt-opt: Scalable deep reinforcement learning for vision-based robotic manipu- lation,

    D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V . Vanhoucke et al. , “Qt-opt: Scalable deep reinforcement learning for vision-based robotic manipu- lation,” CoRL, 2018

    work page 2018

  5. [4]

    Mastering atari, go, chess and shogi by planning with a learned model,

    J. Schrittwieser, I. Antonoglou, T. Hubert, K. Simonyan, L. Sifre, S. Schmitt, A. Guez, E. Lockhart, D. Hassabis, T. Graepel et al. , “Mastering atari, go, chess and shogi by planning with a learned model,” Nature, 2020

    work page 2020

  6. [5]

    Hindsight goal ranking on replay buffer for sparse reward environment,

    T. M. Luu and C. D. Yoo, “Hindsight goal ranking on replay buffer for sparse reward environment,” IEEE Access , 2021

    work page 2021

  7. [6]

    Utilizing skipped frames in action repeats for improving sample efficiency in reinforcement learning,

    T. M. Luu, T. Nguyen, T. Vu, and C. D. Yoo, “Utilizing skipped frames in action repeats for improving sample efficiency in reinforcement learning,” IEEE Access , 2022

    work page 2022

  8. [7]

    Predictive coding for decision transformer,

    T. M. Luu, D. Lee, and C. D. Yoo, “Predictive coding for decision transformer,” 2024

    work page 2024

Show all 50 references
  1. [8]

    Dota 2 with large scale deep reinforcement learning,

    C. Berner, G. Brockman, B. Chan, V . Cheung, P. Debiak, C. Dennison, D. Farhi, Q. Fischer, S. Hashme, C. Hesse et al. , “Dota 2 with large scale deep reinforcement learning,” arXiv:1912.06680, 2019

    1912 arXiv

  2. [9]

    A comprehensive survey on safe reinforce- ment learning,

    J. Garcıa and F. Fern ´andez, “A comprehensive survey on safe reinforce- ment learning,” JMLR, 2015

    2015

  3. [10]

    No falls, no resets: Reliable humanoid behavior in the darpa robotics challenge,

    C. G. Atkeson, B. P. W. Babu, N. Banerjee, D. Berenson, C. P. Bove, X. Cui, M. DeDonato, R. Du, S. Feng, P. Franklin et al. , “No falls, no resets: Reliable humanoid behavior in the darpa robotics challenge,” in IEEE-RAS, 2015

    2015

  4. [11]

    Deep reinforcement learning: A brief survey,

    K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, “Deep reinforcement learning: A brief survey,” IEEE Signal Processing Magazine, 2017

    2017

  5. [12]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al. , “Language models are unsupervised multitask learners,” OpenAI blog , 2019

    2019

  6. [13]

    Language models are few-shot learners,

    T. B. Brown, “Language models are few-shot learners,” arXiv:2005.14165, 2020

    2005 arXiv

  7. [14]

    On the opportunities and risks of foundation models,

    R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill et al., “On the opportunities and risks of foundation models,” arXiv:2108.07258, 2021

    2021 arXiv

  8. [15]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,

    M. Reid, N. Savinov, D. Teplyashin, D. Lepikhin, T. Lillicrap, J.-b. Alayrac, R. Soricut, A. Lazaridou, O. Firat, J. Schrittwieser et al. , “Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,” arXiv:2403.05530, 2024

    2024 arXiv

  9. [16]

    Openai. gpt-4v,

    “Openai. gpt-4v,” 2023. [Online]. Available: https://openai.com/index/ gpt-4v-system-card/

    2023

  10. [17]

    Do as i can, not as i say: Grounding language in robotic affordances,

    M. Ahn, A. Brohan, N. Brown, Y . Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausman et al., “Do as i can, not as i say: Grounding language in robotic affordances,” arXiv:2204.01691, 2022

    2022 arXiv

  11. [18]

    Language models as zero-shot planners: Extracting actionable knowledge for embodied agents,

    W. Huang, P. Abbeel, D. Pathak, and I. Mordatch, “Language models as zero-shot planners: Extracting actionable knowledge for embodied agents,” in ICML, 2022

    2022

  12. [19]

    Large language models as generalizable policies for embodied tasks,

    A. Szot, M. Schwarzer, H. Agrawal, B. Mazoure, R. Metcalf, W. Talbott, N. Mackraz, R. D. Hjelm, and A. T. Toshev, “Large language models as generalizable policies for embodied tasks,” in ICLR, 2023

    2023

  13. [20]

    Fine-tuning large vision-language models as decision-making agents via reinforcement learning,

    Y . Zhai, H. Bai, Z. Lin, J. Pan, S. Tong, Y . Zhou, A. Suhr, S. Xie, Y . LeCun, Y . Maet al. , “Fine-tuning large vision-language models as decision-making agents via reinforcement learning,” arXiv:2405.10292, 2024

    2024 arXiv

  14. [21]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar et al. , “Llama: Open and efficient foundation language models,” arXiv:2302.13971, 2023

    2023 arXiv

  15. [22]

    Palm: Scaling language modeling with pathways,

    A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann et al., “Palm: Scaling language modeling with pathways,” JMLR, 2023

    2023

  16. [23]

    Distilling the knowledge in a neural network,

    G. Hinton, “Distilling the knowledge in a neural network,” arXiv:1503.02531, 2015

    2015 arXiv

  17. [24]

    Proximal policy optimization algorithms,

    J. S. at al, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  18. [25]

    Asynchronous methods for deep reinforcement learning,

    V . M. et al, “Asynchronous methods for deep reinforcement learning,” in ICML, 2016

    2016

  19. [26]

    Guiding pretraining in reinforcement learning with large language models,

    Y . Du, O. Watkins, Z. Wang, C. Colas, T. Darrell, P. Abbeel, A. Gupta, and J. Andreas, “Guiding pretraining in reinforcement learning with large language models,” in ICML, 2023

    2023

  20. [27]

    Progprompt: Generating situated robot task plans using large language models,

    I. Singh, V . Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. Garg, “Progprompt: Generating situated robot task plans using large language models,” in ICRA, 2023

    2023

  21. [28]

    Inner mono- logue: Embodied reasoning through planning with language models,

    W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y . Chebotar et al. , “Inner mono- logue: Embodied reasoning through planning with language models,” arXiv:2207.05608, 2022

    2022 arXiv

  22. [29]

    Enabling intelligent interactions between an agent and an llm: A reinforcement learning approach,

    B. Hu, C. Zhao, P. Zhang, Z. Zhou, Y . Yang, Z. Xu, and B. Liu, “Enabling intelligent interactions between an agent and an llm: A reinforcement learning approach,” RLC, 2024

    2024

  23. [30]

    Grounding large language models in interactive environments with online reinforcement learning,

    T. Carta, C. Romac, T. Wolf, S. Lamprier, O. Sigaud, and P.-Y . Oudeyer, “Grounding large language models in interactive environments with online reinforcement learning,” in ICML, 2023

    2023

  24. [31]

    Introducing gemini: our largest and most capable ai model,

    “Introducing gemini: our largest and most capable ai model,” 2023. [Online]. Available: https://blog.google/technology/ai/google-gemini-ai/

    2023

  25. [32]

    Qwen2. 5 technical report,

    A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei et al. , “Qwen2. 5 technical report,” arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  26. [33]

    The claude 3 model family: Opus, sonnet, haiku

    “The claude 3 model family: Opus, sonnet, haiku.” [Online]. Available: https://api.semanticscholar.org/CorpusID:268232499

  27. [34]

    Plan-seq- learn: Language model guided rl for solving long horizon robotics tasks,

    M. Dalal, T. Chiruvolu, D. Chaplot, and R. Salakhutdinov, “Plan-seq- learn: Language model guided rl for solving long horizon robotics tasks,” arXiv:2405.01534, 2024

    2024 arXiv

  28. [35]

    Reward design with language models,

    M. Kwon, S. M. Xie, K. Bullard, and D. Sadigh, “Reward design with language models,” arXiv:2303.00001, 2023

    2023 arXiv

  29. [36]

    Vision-language models are zero-shot reward models for reinforcement learning,

    J. Rocamonde, V . Montesinos, E. Nava, E. Perez, and D. Lindner, “Vision-language models are zero-shot reward models for reinforcement learning,” arXiv:2310.12921, 2023

    2023 arXiv

  30. [37]

    Rl-vlm-f: Reinforcement learning from vision language foundation model feedback,

    Y . Wang, Z. Sun, J. Zhang, Z. Xian, E. Biyik, D. Held, and Z. Erickson, “Rl-vlm-f: Reinforcement learning from vision language foundation model feedback,” arXiv:2402.03681, 2024

    2024 arXiv

  31. [38]

    V oyager: An open-ended embodied agent with large language models,

    G. Wang, Y . Xie, Y . Jiang, A. Mandlekar, C. Xiao, Y . Zhu, L. Fan, and A. Anandkumar, “V oyager: An open-ended embodied agent with large language models,” arXiv:2305.16291, 2023

    2023 arXiv

  32. [39]

    Bootstrap your own skills: Learning to solve new tasks with large language model guidance,

    J. Zhang, J. Zhang, K. Pertsch, Z. Liu, X. Ren, M. Chang, S.-H. Sun, and J. J. Lim, “Bootstrap your own skills: Learning to solve new tasks with large language model guidance,” arXiv:2310.10021, 2023

    2023 arXiv

  33. [40]

    Embodiedgpt: Vision-language pre-training via embodied chain of thought,

    Y . Mu, Q. Zhang, M. Hu, W. Wang, M. Ding, J. Jin, B. Wang, J. Dai, Y . Qiao, and P. Luo, “Embodiedgpt: Vision-language pre-training via embodied chain of thought,” NeurIPS, 2024

    2024

  34. [41]

    Vision-language mod- els provide promptable representations for reinforcement learning,

    W. Chen, O. Mees, A. Kumar, and S. Levine, “Vision-language mod- els provide promptable representations for reinforcement learning,” arXiv:2402.02651, 2024

    2024 arXiv

  35. [42]

    Policy distillation,

    A. A. Rusu, S. G. Colmenarejo, C ¸ aglar G ¨ulc ¸ehre, G. Desjardins, J. Kirkpatrick, R. Pascanu, V . Mnih, K. Kavukcuoglu, and R. Hadsell, “Policy distillation,” CoRR, 2015

    2015

  36. [43]

    Actor-mimic: Deep multitask and transfer reinforcement learning,

    E. Parisotto, J. L. Ba, and R. Salakhutdinov, “Actor-mimic: Deep multitask and transfer reinforcement learning,” arXiv:1511.06342, 2015

    2015 arXiv

  37. [44]

    Guided policy search,

    S. Levine and V . Koltun, “Guided policy search,” in ICML, 2013

    2013

  38. [45]

    Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning,

    A. Nagabandi, G. Kahn, R. S. Fearing, and S. Levine, “Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning,” in ICRA, 2018

    2018

  39. [46]

    Agents teaching agents: a survey on inter-agent transfer learning,

    F. L. Da Silva, G. Warnell, A. H. R. Costa, and P. Stone, “Agents teaching agents: a survey on inter-agent transfer learning,” Autonomous Agents and Multi-Agent Systems , 2020

    2020

  40. [47]

    Reincarnating reinforcement learning: Reusing prior computation to accelerate progress,

    R. Agarwal, M. Schwarzer, P. S. Castro, A. C. Courville, and M. Belle- mare, “Reincarnating reinforcement learning: Reusing prior computation to accelerate progress,” NeurIPS, 2022

    2022

  41. [49]

    When does label smoothing help?

    R. Muller, S. Kornblith, and G. E. Hinton, “When does label smoothing help?” NeurIPS, 2019

    2019

  42. [50]

    Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal- oriented tasks,

    M. Chevalier-Boisvert, B. Dai, M. Towers, R. Perez-Vicente, L. Willems, S. Lahlou, S. Pal, P. S. Castro, and J. Terry, “Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal- oriented tasks,” NeurIPS, 2024

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.