Pith. sign in

REVIEW 3 major objections 7 minor 62 references

Policy Learning from Large Vision-Language Model Feedback without Reward Modeling

T0 review · 3 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper shows that offline robot policies can be trained directly from a VLM's pairwise trajectory preferences, with no reward model, improving average success by 13.5 percent over the previous state of the art.

desk verdict PLARE is a useful integration—VLM preference labels plus CPL, no reward model—and the evidence mostly backs it, but per-task dropout tuning and max-over-checkpoint selection inflate the headline gains. read the letter →

arxiv 2507.23391 v1 pith:QDHBP43T submitted 2025-07-31 cs.LG cs.RO

Tung M. Luu , Donghoon Lee , Younghwan Lee , Chang D. Yoo This is my paper
classification cs.LGcs.RO
keywords offlinereinforcementlearningpreference-basedRLvision-languagemodelscontrastivepreferencereward-freepolicyroboticmanipulationnoisylabelsVLMfeedback
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a robotic policy can be learned from an unlabeled, suboptimal dataset using only a natural-language task description, by asking a large vision-language model (VLM) which of two short trajectory video segments makes more progress toward the goal. The resulting pairwise preferences are fed directly into a supervised contrastive preference learning objective, so no explicit reward model is trained. The claim matters because reward design is a major bottleneck in offline reinforcement learning: if a VLM can supply the signal, new tasks become trainable from language plus unlabeled video. On four simulated manipulation tasks the method reports an average success rate of 70.0 percent, a relative improvement of 13.5 percent over the previous state-of-the-art VLM-reward approach, and it also trains a real robot on two simple tabletop tasks.

What carries the argument

The load-bearing object is the contrastive preference learning (CPL) objective, a supervised loss that treats each segment's discounted action log-likelihood as the segment's return and uses a softmax over the two compared segments to push probability mass toward the preferred one. The VLM labeling pipeline is the second piece: a two-stage prompt first asks the model to describe each three-frame sequence and then asks which sequence better achieves the language-specified goal. Dropout acts as the noise-handling mechanism, and dropping equal-preference labels removes ambiguous supervision.

What would settle it

Evaluate PLARE on a task with known ground-truth preferences; if VLM label accuracy drops below roughly 70 percent and final policy success falls to the level of behavior cloning, the claim that VLM preferences suffice for policy learning would fail.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that pairwise preference labels from a large vision-language model, obtained by showing the model three frames (first, middle, and last) from each of two trajectory segments, are a sufficient supervisory signal to train a manipulation policy offline. The policy is optimized with a supervised contrastive preference learning objective that compares segment-level action log-likelihoods, so no reward function is ever fit. The paper reports an average success rate of 70.0 percent across four simulated manipulation tasks, a relative improvement of 13.5 percent over the previous state-of-the-art VLM-reward baseline, and successful real-robot policies on two simple tabletop tasks. The paper also finds that excluding labels where the VLM says the two segments are equal, and tuning dropout as a regularizer, are both important for handling the 20 to 30 percent label noise the VLM produces.

Load-bearing premise

The load-bearing premise is that a VLM's pairwise comparisons of three frames per segment are reliable enough to guide a manipulation policy, even though the paper measures only 70 to 80 percent agreement with ground-truth preferences.

Editorial extensions

If this is right

  • New manipulation tasks could be specified by a sentence and an unlabeled video collection, removing reward engineering entirely.
  • Without a reward model, training is cheaper and has fewer components to tune; the paper reports 0.28 million parameters and 0.4 hours of training versus 1.42 million parameters and 1.3 hours for the reward-model baseline.
  • Filtering out equal-preference labels and tuning dropout appear to be simple, transferable ways to cope with noisy VLM labels.
  • If the result holds, the same recipe could apply to other embodied domains where a vision-language model can judge trajectory progress from language instructions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own comparison with an oracle trained on ground-truth labels shows the learning objective is not the bottleneck: the VLM's 70 to 80 percent label accuracy is. This suggests the next gains will come from better questions or better frame selection, not from a stronger loss.
  • Because the simulated policies use low-dimensional state vectors, the VLM preference signal may matter less when the state already encodes goal progress; a purely visual-control version would test whether the preference signal alone carries the information.
  • The two-stage prompting scheme is one design choice among many, and the reported gains could be sensitive to prompt wording; ablating the prompt across tasks would reveal how much of the improvement comes from the learning objective versus the query format.
  • The real-robot tasks are deliberately simple, so the method's ceiling on long-horizon, multi-stage manipulation remains untested; a natural extension is to evaluate whether three-frame segment comparisons can judge progress in tasks where early states look visually similar.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper introduces PLARE, a method for offline policy learning that replaces hand-designed rewards and learned reward models with preference labels generated by a large vision-language model (VLM). Given an unlabeled dataset and a natural-language task description, PLARE samples pairs of trajectory segments, represents each segment by three uniformly spaced frames (first, middle, last), and queries Gemini for a pairwise preference. The policy is then trained directly with the contrastive preference learning (CPL) objective, without an intermediate reward model. Experiments cover four MetaWorld manipulation tasks and two real-robot tasks with a 7-DOF Sawyer arm. The paper reports an average success rate of 70.0 on MetaWorld, a 13.5% relative improvement over the RL-VLM-F baseline, and ablations on equal-preference filtering and dropout as a regularizer for noisy labels.

Significance. If the claims are sustained, PLARE offers a practical approach to training robotic policies from language descriptions and unlabeled video, avoiding both manual reward engineering and learned reward models. The use of CPL is a sound and well-motivated choice, and the comparison against RL-VLM-F uses the same VLM-generated preference dataset, which isolates the effect of skipping reward modeling. The paper also includes informative ablations on label noise. The main limitations are the evaluation protocol (per-task dropout selection and max-over-checkpoint reporting), the lack of analysis of VLM label reliability as a function of frame count and task complexity, and the statistically weak real-robot results. The code is promised to be public, which supports reproducibility.

major comments (3)
  1. [Table III and Table I] The main results in Table I are obtained by selecting, per task, the dropout probability that yields the best success rate in Table III (e.g., 0.4 for Drawer Open and Plate Slide, 0.25 for Sweep Into and Door Open), and success rates are the maximum average over an 8-checkpoint, 200-episode evaluation window. Because the same performance metric is used both for hyperparameter selection and for final evaluation, the reported 13.5% relative improvement over RL-VLM-F may be inflated by selection bias. The authors should use a validation split for dropout and checkpoint selection and report performance on a held-out split, or at least report results across all dropout values without selecting the best.
  2. [Section IV.A and Table III] The choice of three uniformly spaced frames per segment is justified only by 'preliminary experiments' that are not reported, and Table III shows VLM label accuracy of only 69.4-80.7% against ground-truth preferences, meaning that 20-30% of training labels are incorrect. The paper does not analyze how label accuracy or downstream policy success varies with the number of frames or with task complexity, and the four MetaWorld tasks are short-horizon with success often visible in the final frame. The current evidence therefore does not establish that the pipeline works for temporally extended or multi-stage tasks. The authors should report the preliminary experiments and an analysis of label quality as a function of frame count, and ideally include a task that requires reasoning about intermediate states rather than just the outcome in the last frame.
  3. [Section V.E and Figure 4] The real-robot comparison is based on only 10 trials per method, and no error bars, confidence intervals, or significance tests are reported. The statement that 'PLARE achieves the highest success rates' is therefore not statistically substantiated. The authors should report per-trial results, standard errors or confidence intervals, and details of the human assessment protocol (e.g., number of raters and whether they were blind to the method).
minor comments (7)
  1. [Section I] There is a typo in the introduction: 'large Vison-Language Models' should be 'large Vision-Language Models'.
  2. [Algorithm 1 and Section V.A] The segment length L is an input to Algorithm 1, but its value is never specified for either the MetaWorld or the real-world experiments; please state the segment length used.
  3. [Section IV.A and Figure 2] The VLM prompt in Figure 2 instructs the model to reply '-1' when unsure or when there is no difference, but the preference dataset and the CPL objective use a label of 0.5 for equal preference; the mapping from the raw VLM response to the stored label should be described explicitly.
  4. [Section IV.B] The dropout used as a regularizer is not described in terms of where it is applied (e.g., in the policy MLP) or how it interacts with the CPL objective; this implementation detail is needed for reproducibility.
  5. [Equation (2)] The term γtα log πθ(at|st) is ambiguous: it is unclear whether α multiplies the log-probability only or also the discount factor, and the roles of α and λ are not defined in the text.
  6. [Table III] The row labeled 'Accuracy' in Table III is not explained in the caption; please state that it measures the agreement between VLM-labeled preferences and the ground-truth preferences from the CPL dataset.
  7. [Section V.A] In MetaWorld, all policies are trained from 35-dimensional low-dimensional states, while VLM preferences are generated from rendered images; the paper does not discuss this modality mismatch, which limits the strength of the claim that the simulation results demonstrate vision-based policy learning.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: VLM preference labels are external inputs and the CPL objective is imported from prior work; only minor non-load-bearing self-citations appear.

full rationale

The paper's derivation chain is not circular. PLARE's preference labels are produced by Gemini, an external vision-language model, based on three uniformly spaced frames per trajectory segment (Section IV.A), and these labels are stored in a preference dataset. The policy is then optimized with the contrastive preference learning objective of Hejna et al. [35], which is prior external work, not a re-derivation of the paper's own claim. Nothing in the objective is fitted to the target success rates; the only task-specific choices are the dropout probability and the exclusion of equal-preference labels, which are hyperparameter/ablation decisions rather than quantities used to define the prediction. The reported 70.0 average and 13.5% relative improvement are computed directly from Table I against the RL-VLM-F baseline, so they are arithmetic, not constructional. The VLM label accuracy values in Table III (70-80%) are an independent measurement of input quality and do not define the output. Self-citations appear only as background or related-work references ([6], [39], [43], [52], [53], [57]) and are not load-bearing; there is no uniqueness theorem, ansatz, or fitted input imported from the authors' own prior work. The core claim, that policies can be trained directly from external VLM feedback without an explicit reward model, is self-contained and externally benchmarked against the datasets and CPL framework of prior work. Accordingly, no circular step is identified, and the low score reflects only the presence of minor non-load-bearing self-citations.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the validity of the CPL objective, the reliability of VLM preference labels, and the sufficiency of the offline dataset. The paper provides some support for the VLM label reliability (accuracy 70-80%), but the other assumptions are taken from prior work or the experimental setup without independent verification.

free parameters (3)
  • dropout probability p_drop = 0.25 or 0.40 depending on task (0.40 for Drawer Open and Plate Slide)
    Tuned per task in Table III using success rate on the evaluation tasks; the best value is used in the main results.
  • number of frames per segment = 3
    Chosen from preliminary experiments as a trade-off between query time and temporal context; not validated on held-out tasks.
  • equal-preference exclusion = True (excluded)
    Decision made after observing in ablations that excluding equal labels improves performance (Table III top).
assumptions (5)
  • standard math The CPL objective (Eq. 1) correctly produces a policy that maximizes the probability of preferred behavior and minimizes non-preferred behavior.
    Adopted from Hejna et al. [35] without modification; its correctness is assumed from the published result.
  • domain assumption VLM pairwise preferences are correlated with task success.
    The paper measures 70-80 percent agreement with ground-truth preference labels (Table III), which supports but does not fully establish this assumption; hallucination is acknowledged.
  • domain assumption The offline dataset has sufficient coverage and quality (about 50-60 percent successful trajectories) for the four MetaWorld tasks and two real-world tasks.
    Dataset properties are taken from [35] and teleoperation; the paper does not analyze coverage or diversity.
  • domain assumption R3M image features plus low-dimensional state are a sufficient representation for the policy.
    Used as a fixed representation; no comparison with other encoders or end-to-end pixels.
  • domain assumption Gemini VLM responses are consistent and reproducible across queries.
    No version, temperature, or sampling parameters are specified, and no repeated-query variance analysis is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Policy Learning from Large Vision-Language Model Feedback without Reward Modeling." pith.science (2026). https://pith.science/paper/QDHBP43T

@misc{pith2026250723391,
  author       = {Pith},
  title        = {Pith review of: Policy Learning from Large Vision-Language Model Feedback without Reward Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QDHBP43T}},
  note         = {Machine review of arXiv:2507.23391}
}
read the original abstract

Offline reinforcement learning (RL) provides a powerful framework for training robotic agents using pre-collected, suboptimal datasets, eliminating the need for costly, time-consuming, and potentially hazardous online interactions. This is particularly useful in safety-critical real-world applications, where online data collection is expensive and impractical. However, existing offline RL algorithms typically require reward labeled data, which introduces an additional bottleneck: reward function design is itself costly, labor-intensive, and requires significant domain expertise. In this paper, we introduce PLARE, a novel approach that leverages large vision-language models (VLMs) to provide guidance signals for agent training. Instead of relying on manually designed reward functions, PLARE queries a VLM for preference labels on pairs of visual trajectory segments based on a language task description. The policy is then trained directly from these preference labels using a supervised contrastive preference learning objective, bypassing the need to learn explicit reward models. Through extensive experiments on robotic manipulation tasks from the MetaWorld, PLARE achieves performance on par with or surpassing existing state-of-the-art VLM-based reward generation methods. Furthermore, we demonstrate the effectiveness of PLARE in real-world manipulation tasks with a physical robot, further validating its practical applicability.

Figures

Figures reproduced from arXiv: 2507.23391 by the authors.

Figure 1
Figure 1. PLARE utilizes a VLM to generate preference labels [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of PLARE: Given an unlabeled, reward-free dataset, PLARE samples two trajectory segments and queries a VLM teacher for a preference label based on the language task description. Notably, only visual observations from each segment are used for querying the VLM. The selected segments, along with their assigned preference labels, are then stored in the preference dataset. Finally, the policy is learned from th… view at source ↗
Figure 3
Figure 3. Workspace setup for our 7-Dof Sawyer Robot arm [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Success rates on the 2 real-world robot manipulation [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 57 canonical work pages

  1. [35]

    Contrastive preference learning: learning from human feedback without rl,

    J. Hejna, R. Rafailov, H. Sikchi, C. Finn, S. Niekum, W. B. Knox, and D. Sadigh, “Contrastive preference learning: learning from human feedback without rl,” International Conference on Learning Represen- tations, 2024

    work page 2024

  2. [22]

    Rl-vlm-f: Reinforcement learning from vision language foundation model feedback,

    Y . Wang, Z. Sun, J. Zhang, Z. Xian, E. Biyik, D. Held, and Z. Erickson, “Rl-vlm-f: Reinforcement learning from vision language foundation model feedback,” International Conference on Machine Learning , 2024

    work page 2024

  3. [1]

    Grandmaster level in starcraft ii using multi-agent reinforcement learning,

    O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev, et al. , “Grandmaster level in starcraft ii using multi-agent reinforcement learning,” Nature, 2019

    work page 2019

  4. [2]

    Mastering the game of stratego with model-free multiagent reinforcement learning,

    J. Perolat, B. De Vylder, D. Hennes, E. Tarassov, F. Strub, V . de Boer, P. Muller, J. T. Connor, N. Burch, T. Anthony, et al., “Mastering the game of stratego with model-free multiagent reinforcement learning,” Science, 2022. Fig. 4: Success rates on the 2 real-world robot manipulation tasks in table-top environment

    work page 2022

  5. [3]

    Autonomous navigation of stratospheric balloons using reinforcement learning,

    M. G. Bellemare, S. Candido, P. S. Castro, J. Gong, M. C. Machado, S. Moitra, S. S. Ponda, and Z. Wang, “Autonomous navigation of stratospheric balloons using reinforcement learning,” Nature, 2020

    work page 2020

  6. [4]

    Champion-level drone racing using deep reinforce- ment learning,

    E. Kaufmann, L. Bauersfeld, A. Loquercio, M. M ¨uller, V . Koltun, and D. Scaramuzza, “Champion-level drone racing using deep reinforce- ment learning,” Nature, 2023

    work page 2023

  7. [5]

    Scalable deep reinforcement learning for vision-based robotic manipulation,

    D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V . Vanhoucke,et al., “Scalable deep reinforcement learning for vision-based robotic manipulation,” in Conference on Robot Learning , 2018

    work page 2018

  8. [6]

    Hindsight goal ranking on replay buffer for sparse reward environment,

    T. M. Luu and C. D. Yoo, “Hindsight goal ranking on replay buffer for sparse reward environment,” IEEE Access, 2021

    work page 2021

Show all 62 references
  1. [7]

    Towards human-level bimanual dexterous manipulation with reinforcement learning,

    Y . Chen, T. Wu, S. Wang, X. Feng, J. Jiang, Z. Lu, S. McAleer, H. Dong, S.-C. Zhu, and Y . Yang, “Towards human-level bimanual dexterous manipulation with reinforcement learning,” in Conference on Neural Information Processing Systems , 2022

    2022

  2. [8]

    Inverse reward design,

    D. Hadfield-Menell, S. Milli, P. Abbeel, S. J. Russell, and A. Dragan, “Inverse reward design,” Conference on Neural Information Process- ing Systems, 2017

    2017

  3. [9]

    Defining and characterizing reward gaming,

    J. Skalse, N. Howe, D. Krasheninnikov, and D. Krueger, “Defining and characterizing reward gaming,” in Conference on Neural Information Processing Systems, 2022

    2022

  4. [10]

    Minigpt-4: En- hancing vision-language understanding with advanced large language models,

    D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “Minigpt-4: En- hancing vision-language understanding with advanced large language models,” arXiv:2304.10592, 2023

    2023 arXiv

  5. [11]

    Hear: Hearing enhanced audio response for video-grounded dialogue,

    S. Yoon, D. Kim, E. Yoon, H. S. Yoon, J. Kim, and C. D. Yoo, “Hear: Hearing enhanced audio response for video-grounded dialogue,” Em- pirical Methods in Natural Language Processing , 2023

    2023

  6. [12]

    Autobench-v: Can large vision-language models benchmark themselves?,

    H. Bao, Y . Huang, Y . Wang, J. Ye, X. Wang, X. Chen, M. Elhoseiny, and X. Zhang, “Autobench-v: Can large vision-language models benchmark themselves?,” arXiv:2410.21259, 2024

    2024 arXiv

  7. [13]

    Openvla: An open- source vision-language-action model,

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al., “Openvla: An open- source vision-language-action model,” Conference on Robot Learning, 2024

    2024

  8. [14]

    Code as reward: Empowering reinforcement learning with vlms,

    D. Venuto, S. N. Islam, M. Klissarov, D. Precup, S. Yang, and A. Anand, “Code as reward: Empowering reinforcement learning with vlms,” International Conference on Machine Learning , 2024

    2024

  9. [15]

    Guiding pretraining in reinforcement learning with large language models,

    Y . Du, O. Watkins, Z. Wang, C. Colas, T. Darrell, P. Abbeel, A. Gupta, and J. Andreas, “Guiding pretraining in reinforcement learning with large language models,” in International Conference on Machine Learning, 2023

    2023

  10. [16]

    Bootstrap your own skills: Learning to solve new tasks with large language model guidance,

    J. Zhang, J. Zhang, K. Pertsch, Z. Liu, X. Ren, M. Chang, S.-H. Sun, and J. J. Lim, “Bootstrap your own skills: Learning to solve new tasks with large language model guidance,” Conference on Robot Learning , 2023

    2023

  11. [17]

    Code as policies: Language model programs for embodied control,

    J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Flo- rence, and A. Zeng, “Code as policies: Language model programs for embodied control,” in International Conference on Robotics and Automation, 2023

    2023

  12. [18]

    Progprompt: Generating situ- ated robot task plans using large language models,

    I. Singh, V . Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. Garg, “Progprompt: Generating situ- ated robot task plans using large language models,” in International Conference on Robotics and Automation , 2023

    2023

  13. [19]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark,et al., “Learning transferable visual models from natural language supervision,” in International Conference on Machine Learning , 2021

    2021

  14. [20]

    Roboclip: One demonstration is enough to learn robot policies,

    S. Sontakke, J. Zhang, S. Arnold, K. Pertsch, E. Bıyık, D. Sadigh, C. Finn, and L. Itti, “Roboclip: One demonstration is enough to learn robot policies,” in Conference on Neural Information Processing Systems, 2024

    2024

  15. [21]

    Vision-language models are zero-shot reward models for reinforce- ment learning,

    J. Rocamonde, V . Montesinos, E. Nava, E. Perez, and D. Lindner, “Vision-language models are zero-shot reward models for reinforce- ment learning,” International Conference on Learning Representa- tions, 2024

    2024

  16. [23]

    Language to rewards for robotic skill synthesis,

    W. Yu, N. Gileadi, C. Fu, S. Kirmani, K.-H. Lee, M. G. Arenas, H.- T. L. Chiang, T. Erez, L. Hasenclever, J. Humplik, et al., “Language to rewards for robotic skill synthesis,” Conference on Robot Learning, 2023

    2023

  17. [24]

    Text2reward: Automated dense reward function generation for reinforcement learning,

    T. Xie, S. Zhao, C. H. Wu, Y . Liu, Q. Luo, V . Zhong, Y . Yang, and T. Yu, “Text2reward: Automated dense reward function generation for reinforcement learning,” in International Conference on Learning Representations, 2024

    2024

  18. [25]

    Real- world offline reinforcement learning from vision language model feedback,

    S. Venkataraman, Y . Wang, Z. Wang, Z. Erickson, and D. Held, “Real- world offline reinforcement learning from vision language model feedback,” CoRL-W, 2024

    2024

  19. [26]

    Deep reinforcement learning from human preferences,

    P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” in Conference on Neural Information Processing Systems , 2017

    2017

  20. [27]

    B-pref: Benchmark- ing preference-based reinforcement learning,

    K. Lee, L. Smith, A. Dragan, and P. Abbeel, “B-pref: Benchmark- ing preference-based reinforcement learning,” Conference on Neural Information Processing Systems , 2021

    2021

  21. [28]

    Inverse preference learning: Preference- based rl without a reward function,

    J. Hejna and D. Sadigh, “Inverse preference learning: Preference- based rl without a reward function,” Conference on Neural Information Processing Systems, 2023

    2023

  22. [29]

    Information- theoretic text hallucination reduction for video-grounded dialogue,

    S. Yoon, E. Yoon, H. S. Yoon, J. Kim, and C. D. Yoo, “Information- theoretic text hallucination reduction for video-grounded dialogue,” Empirical Methods in Natural Language Processing , 2022

    2022

  23. [30]

    Siren’s song in the ai ocean: a survey on hallucination in large language models,

    Y . Zhang, Y . Li, L. Cui, D. Cai, L. Liu, T. Fu, X. Huang, E. Zhao, Y . Zhang, Y . Chen,et al., “Siren’s song in the ai ocean: a survey on hallucination in large language models,” arXiv:2309.01219, 2023

    2023 arXiv

  24. [31]

    ” task success

    L. Guan, Y . Zhou, D. Liu, Y . Zha, H. B. Amor, and S. Kambhampati, “” task success” is not enough: Investigating the use of video-language models as behavior critics for catching undesirable agent behaviors,” Conference on Language Modeling , 2024

    2024

  25. [32]

    Non-markovian reward modelling from trajectory labels via interpretable multiple instance learning,

    J. Early, T. Bewley, C. Evers, and S. Ramchurn, “Non-markovian reward modelling from trajectory labels via interpretable multiple instance learning,” Conference on Neural Information Processing Systems, 2022

    2022

  26. [33]

    Preference transformer: Modeling human preferences using transformers for rl,

    C. Kim, J. Park, J. Shin, H. Lee, P. Abbeel, and K. Lee, “Preference transformer: Modeling human preferences using transformers for rl,” International Conference on Learning Representations , 2023

    2023

  27. [34]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,

    M. Reid, N. Savinov, D. Teplyashin, D. Lepikhin, T. Lillicrap, J.-b. Alayrac, R. Soricut, A. Lazaridou, O. Firat, J. Schrittwieser, et al. , “Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,” arXiv:2403.05530, 2024

    2024 arXiv

  28. [36]

    Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning,

    T. Yu, D. Quillen, Z. He, R. Julian, K. Hausman, C. Finn, and S. Levine, “Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning,” in Conference on Robot Learning , 2020

    2020

  29. [37]

    Minedojo: Building open- ended embodied agents with internet-scale knowledge,

    L. Fan, G. Wang, Y . Jiang, A. Mandlekar, Y . Yang, H. Zhu, A. Tang, D.-A. Huang, Y . Zhu, and A. Anandkumar, “Minedojo: Building open- ended embodied agents with internet-scale knowledge,” Conference on Neural Information Processing Systems , 2022

    2022

  30. [38]

    Liv: Language-image representations and rewards for robotic control,

    Y . J. Ma, V . Kumar, A. Zhang, O. Bastani, and D. Jayaraman, “Liv: Language-image representations and rewards for robotic control,” in International Conference on Machine Learning , 2023

    2023

  31. [39]

    Enhancing rating-based reinforcement learning to effectively leverage feedback from large vision-language models,

    T. M. Luu, Y . Lee, D. Lee, S. Kim, M. J. Kim, and C. D. Yoo, “Enhancing rating-based reinforcement learning to effectively leverage feedback from large vision-language models,” in Forty-second Inter- national Conference on Machine Learning , 2025

    2025

  32. [40]

    Alvinn: An autonomous land vehicle in a neural network,

    D. A. Pomerleau, “Alvinn: An autonomous land vehicle in a neural network,” Conference on Neural Information Processing Systems , 1988

    1988

  33. [41]

    Rt-1: Robotics transformer for real-world control at scale,

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu, et al. , “Rt-1: Robotics transformer for real-world control at scale,” arXiv:2212.06817, 2022

    2022 arXiv

  34. [42]

    Interactive language: Talking to robots in real time,

    C. Lynch, A. Wahid, J. Tompson, T. Ding, J. Betker, R. Baruch, T. Armstrong, and P. Florence, “Interactive language: Talking to robots in real time,” RA-L, 2023

    2023

  35. [43]

    Predictive coding for decision transformer,

    T. M. Luu, D. Lee, and C. D. Yoo, “Predictive coding for decision transformer,” in International Conference on Intelligent Robots and Systems, 2024

    2024

  36. [44]

    Algorithms for inverse reinforcement learning.,

    A. Y . Ng, S. Russell, et al. , “Algorithms for inverse reinforcement learning.,” in International Conference on Machine Learning , 2000

    2000

  37. [45]

    Generative adversarial imitation learning,

    J. Ho and S. Ermon, “Generative adversarial imitation learning,” Conference on Neural Information Processing Systems , 2016

    2016

  38. [46]

    Nonlinear inverse reinforcement learning with gaussian processes,

    S. Levine, Z. Popovic, and V . Koltun, “Nonlinear inverse reinforcement learning with gaussian processes,” Conference on Neural Information Processing Systems, 2011

    2011

  39. [47]

    Guided cost learning: Deep inverse optimal control via policy optimization,

    C. Finn, S. Levine, and P. Abbeel, “Guided cost learning: Deep inverse optimal control via policy optimization,” in International Conference on Machine Learning , 2016

    2016

  40. [48]

    Learning robust rewards with adversarial inverse reinforcement learning,

    J. Fu, K. Luo, and S. Levine, “Learning robust rewards with adversarial inverse reinforcement learning,” International Conference on Learning Representations, 2018

    2018

  41. [49]

    Confidence-aware imitation learning from demonstrations with varying optimality,

    S. Zhang, Z. Cao, D. Sadigh, and Y . Sui, “Confidence-aware imitation learning from demonstrations with varying optimality,” Conference on Neural Information Processing Systems , 2021

    2021

  42. [50]

    Extrapolating beyond suboptimal demonstrations via inverse reinforcement learning from observations,

    D. Brown, W. Goo, P. Nagarajan, and S. Niekum, “Extrapolating beyond suboptimal demonstrations via inverse reinforcement learning from observations,” in International Conference on Machine Learning, 2019

    2019

  43. [51]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Conference on Neural Information Processing Systems, 2020

    2020

  44. [52]

    Mdsgen: Fast and efficient masked diffusion temporal-aware transformers for open-domain sound generation,

    T. X. Pham, T. Ton, and C. D. Yoo, “Mdsgen: Fast and efficient masked diffusion temporal-aware transformers for open-domain sound generation,” International Conference on Learning Representations , 2025

    2025

  45. [53]

    Taro: Timestep-adaptive repre- sentation alignment with onset-aware conditioning for synchronized video-to-audio synthesis,

    T. Ton, J. W. Hong, and C. D. Yoo, “Taro: Timestep-adaptive repre- sentation alignment with onset-aware conditioning for synchronized video-to-audio synthesis,” International Conference on Computer Vi- sion, 2025

    2025

  46. [54]

    Diffusion reward: Learning rewards via conditional video diffusion,

    T. Huang, G. Jiang, Y . Ze, and H. Xu, “Diffusion reward: Learning rewards via conditional video diffusion,” in European Conference on Computer Vision, 2024

    2024

  47. [55]

    Di- rect preference-based policy optimization without reward modeling,

    G. An, J. Lee, X. Zuo, N. Kosaka, K.-M. Kim, and H. O. Song, “Di- rect preference-based policy optimization without reward modeling,” Conference on Neural Information Processing Systems , 2023

    2023

  48. [56]

    Atari-head: Atari human eye- tracking and demonstration dataset,

    R. Zhang, C. Walshe, Z. Liu, L. Guan, K. Muller, J. Whritner, L. Zhang, M. Hayhoe, and D. Ballard, “Atari-head: Atari human eye- tracking and demonstration dataset,” in AAAI, 2020

    2020

  49. [57]

    Scanet: Scene complexity aware network for weakly-supervised video moment retrieval,

    S. Yoon, G. Koo, D. Kim, and C. D. Yoo, “Scanet: Scene complexity aware network for weakly-supervised video moment retrieval,” in International Conference on Computer Vision , 2023

    2023

  50. [58]

    Learning deep networks from noisy labels with dropout regularization,

    I. Jindal, M. Nokleby, and X. Chen, “Learning deep networks from noisy labels with dropout regularization,” in ICDM, 2017

    2017

  51. [59]

    Compressing features for learning with noisy labels,

    Y . Chen, S. X. Hu, X. Shen, C. Ai, and J. A. Suykens, “Compressing features for learning with noisy labels,” TNNLS, 2022

    2022

  52. [60]

    R3m: A universal visual representation for robot manipulation,

    S. Nair, A. Rajeswaran, V . Kumar, C. Finn, and A. Gupta, “R3m: A universal visual representation for robot manipulation,” Conference on Robot Learning, 2022

    2022

  53. [61]

    Offline reinforcement learning with implicit q-learning,

    I. Kostrikov, A. Nair, and S. Levine, “Offline reinforcement learning with implicit q-learning,” International Conference on Learning Rep- resentations, 2022

    2022

  54. [62]

    Rime: Robust preference-based reinforcement learning with noisy preferences,

    J. Cheng, G. Xiong, X. Dai, Q. Miao, Y . Lv, and F.-Y . Wang, “Rime: Robust preference-based reinforcement learning with noisy preferences,” International Conference on Machine Learning , 2024

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.