Pith. sign in

REVIEW 5 major objections 5 minor 35 references

CAREL: Instruction-guided reinforcement learning with cross-modal auxiliary objectives

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read CAREL claims that an X-CLIP-style cross-modal contrastive auxiliary loss plus instruction masking gives instruction-following RL agents a substantial boost in sample efficiency and systematic generalization, with reported gains like 73%…

desk verdict A solid empirical paper with a genuinely new tracking heuristic; the sample-efficiency claims are plausible but the reporting of key hyperparameters needs to be tighter. read the letter →

arxiv 2411.19787 v2 pith:EUZ32MFB submitted 2024-11-29 cs.LG cs.AI

Armin Saghafian , Amirmohammad Izadi , Negin Hashemi Dijujin , Mahdieh Soleymani Baghshah This is my paper · ORCID
classification cs.LGcs.AI
keywords reinforcementlearninginstructionfollowingcross-modalalignmentcontrastiveX-CLIPsampleefficiencysystematicgeneralizationgoal-conditionedRL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that instruction-following reinforcement learning agents learn faster and generalize better when the model is trained, in addition to the reward signal, to align its observations with the textual instruction through a contrastive cross-modal auxiliary loss. The key claim is that borrowing a multi-grained alignment objective from video-text retrieval, where the instruction plays the role of the text and the observation sequence the role of the video, gives the agent a grounding signal that reward alone fails to provide. Combined with a tracking mechanism that masks completed parts of the instruction, the framework is reported to raise success rates substantially at fixed frame budgets, for instance from 41% to 73% on a sequenced goal-reaching task and from 24% to 79% on an ordered door-opening task. If correct, this is a training-time recipe that can be bolted onto existing dual-encoder agents to make them more sample-efficient without changing the environment or the reward structure.

What carries the argument

The engine of CAREL is a contrastive cross-modal loss adapted from X-CLIP, a video-text retrieval method. For a batch of successful episodes, it computes four similarity matrices between local and global representations of the observation sequence and the instruction: episode-instruction, episode-word, observation-instruction, and observation-word. These are aggregated with softmax attention, and the final score feeds a symmetric InfoNCE-style loss that pulls each episode's representation close to its own instruction and repels it from other instructions in the batch. The second mechanism, instruction tracking, builds on the same similarity scores: for each subtask parsed from the instruction, it keeps a moving average of the episode-word similarity, and when the current score spikes above $k$ times the running average (Eq. 14), the subtask's tokens are masked from the instruction with a probability that ramps up with training progress. The auxiliary loss is what makes the similarity scores trustworthy enough to drive masking, so the two mechanisms are designed to reinforce each other.

What would settle it

Train CAREL on GoToSeqS5R2 with the success threshold deliberately set to 0% of maximum reward (all trajectories become positives) and separately to 100%; if the success-rate curves remain close to Table 1, the threshold is not what drives the gains, while if they collapse, the unspecified threshold is a critical hyperparameter. A second falsifier targets instruction tracking: replace the moving-average spike rule of Eq. 14 with random masking of one subtask per episode under the same probability schedule; if random masking matches the CAREL+IT curve, the similarity-based tracking is not the source of the improvement.

Watch

Extended reading notes

Core claim

The paper proposes CAREL, a framework that adds a cross-modal contrastive auxiliary loss to the RL objective of an instruction-following agent. The loss is the X-CLIP objective, which computes similarity between an episode (a sequence of observations with action embeddings) and an instruction at four granularities: episode-instruction, episode-word, observation-instruction, and observation-word, and then combines them into a single per-pair score. During training, only trajectories that achieve a high enough reward are treated as positive pairs; the loss pulls matched episode-instruction pairs together in embedding space and pushes unmatched pairs apart. On top of the loss, CAREL's instruction tracking uses the per-word similarity scores produced during the forward pass to detect when a subtask has been completed, and masks the corresponding tokens, along with their connecting conjunctions, out of the instruction given to the policy for the remainder of the episode. The reported experiments, on navigation and manipulation tasks with procedurally generated instructions and unseen color and type combinations, show higher success rates at the same frame budgets than the BabyAI baseline and, when applied to the SHELM baseline, comparable or better mean rewards with faster convergence. The authors frame the contribution as a way to inject grounding into the representations of a dual-encoder RL agent without altering its architecture.

Load-bearing premise

The auxiliary loss labels a trajectory as a positive example only if it crosses a success threshold defined as a fraction of the maximum achievable reward, a number the paper never specifies; the whole contrastive signal assumes this threshold cleanly separates episodes that genuinely match the instruction from those that do not, and the paper offers no evidence the same threshold works across tasks without tuning.

Editorial extensions

If this is right

  • On the BabyAI benchmark, equipping the baseline with CAREL lifts the success rate at fixed frame budgets from 41% to 73% on GoToSeqS5R2 (10M frames) and from 24% to 79% on OpenDoorsOrderN4 (3M frames); similar gains appear across all five tested tasks.
  • Adding instruction tracking on top gives a further boost on the sequential tasks, with the full method reaching 93% versus 76% for the baseline on GoToSeqS5R2 and 97% versus 62% on OpenDoorsOrderN4 at convergence.
  • The framework transfers to a different base model: applying CAREL to SHELM improves sample efficiency on MiniGrid-GoToDoor (mean reward 0.83 versus 0.50 at 1.5M frames) and on PutNear.
  • CAREL matches or beats the imitation-learning baseline LISA in success rate on the two tasks LISA was designed for.
  • The implementation adds only a modest compute overhead (about 420 versus 669 frames per second on average in BabyAI), so the sample-efficiency gains are not bought by drastically slower updates.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the auxiliary loss labels episodes by reward threshold and not by instruction semantics, the same recipe might apply to any goal-conditioned RL agent that receives a textual description and has a dual-encoder architecture, not just the BabyAI and SHELM settings tested here; the main barrier would be calibrating the success threshold per task.
  • The instruction-tracking masking rule has a natural connection to curriculum learning: as the masking probability grows with training progress, the agent sees progressively shorter instructions, which may implicitly encourage it to schedule its own subgoals; a testable extension would be to measure whether the learned policy becomes better at decomposing long instructions in zero-shot generalizati
  • The X-CLIP loss is symmetric in the sense that it aligns both directions, episode-to-instruction and instruction-to-episode; one could exploit the same scores to provide dense reward shaping during the episode, which the paper does not do.
  • If the success threshold in Section 2.1 is miscalibrated, the contrastive positives are mislabeled; a robustness study sweeping that threshold would clarify whether the method's gains are sensitive to this otherwise unspecified hyperparameter.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The manuscript introduces CAREL, a training-time framework for instruction-following reinforcement learning. It augments a base RL agent with a multi-grained X-CLIP-style contrastive loss that aligns successful episode-observation sequences with their textual instructions (Eq. 12), and an 'instruction tracking' mechanism that masks completed sub-tasks from the instruction once their similarity to the partial episode spikes (Eq. 14). The paper reports experiments on BabyAI with the BabyAI baseline, on MiniGrid with a modified SHELM baseline, and comparisons with LISA, and claims superior sample efficiency and systematic generalization. An appendix provides ablations for the auxiliary-loss coefficient, action embeddings, similarity-score choice, and the interaction between CAREL and instruction tracking.

Significance. Language grounding in RL is an important and active problem, and CAREL is a simple, architecture-agnostic proposal: a video-text-retrieval-style auxiliary loss plus a goal-masking heuristic. The manuscript is self-contained in its derivation of the auxiliary loss, gives algorithmic pseudocode for both components, and includes several relevant ablations (lambda_C, action embeddings, choice of similarity score, and CAREL-plus-IT interaction). The reported gains, such as 73% vs 41% at 10M frames on GoToSeqS5R2, are substantial. However, the central empirical claim is currently supported by point estimates from only two seeds for the main BabyAI results, an unreported success-reward threshold, and an undefined sampling-budget convention. The significance of the contribution is therefore conditional on the requested revisions.

major comments (5)
  1. [§2.1] The success-reward threshold used to select episodes for the auxiliary loss is never quantified. The text states only that a trajectory is successful if its aggregated reward exceeds 'a fraction of the maximum achievable reward', but the fraction is not reported for any task. This is load-bearing because the contrastive positives in Eq. (12) are exactly these successful episodes: a permissive threshold includes poorly aligned trajectories and corrupts the contrastive signal, while a strict threshold shrinks the positive batch and can eliminate the auxiliary gradient early in training. Please report the threshold used for every environment, state whether it was tuned per task, and provide a sensitivity analysis (e.g., sweeping the fraction) or an ablation that removes the threshold dependence.
  2. [§3.1, Table 1] The main quantitative results in Table 1 are point estimates over two random seeds with no error bars or per-seed values, whereas Table 2 reports standard deviations. The reported differences are large, but with two seeds one cannot assess whether the ordering is stable. Please report all individual runs, add at least five seeds per condition, and include standard errors or confidence intervals; if computational cost is a concern, provide the raw curves in an appendix.
  3. [§3.1.1] The sample budgets in Table 1 are said to be 'roughly the halfway point of the samples required for model convergence', but no convergence criterion is defined. This is not reproducible and, if chosen post hoc, can exaggerate the method's advantage. The same issue appears in the SHELM comparison ('the midpoint of convergence') and in Table 3. Please define the convergence rule (e.g., a threshold on the moving average of success rate over a fixed window), report the resulting convergence points for each method, and show the full learning curves with variance bands.
  4. [§3.2, Table 3] The LISA comparison in Table 3 is not an equivalent sample-efficiency comparison. LISA is an imitation-learning method that consumes 100k expert trajectories (reported as 7M and 1.5M frames), while CAREL is trained online from environment interaction. Reporting both in the same 'Frames' column conflates environment steps with demonstration data and obscures differences in supervision. Please compare on a common data axis (e.g., number of trajectories), make explicit the supervision source for LISA, and, if possible, include a supervised or offline baseline trained on the same number of expert demonstrations.
  5. [§3.1.2] The full CAREL method in Figure 3 differs from the vanilla CAREL baseline by both instruction tracking and the addition of action embeddings to the observation representations. Consequently, the improvement attributed to instruction tracking is confounded with a representation change. The ablation in Appendix A.2.2 separates action embeddings only for CAREL+IT, so please also report CAREL with action embeddings but without IT, and BabyAI+IT with and without action embeddings, so the contribution of each component can be isolated.
minor comments (5)
  1. [Algorithm 2] Line 6 of Algorithm 2 writes S_t_ci <- -mean(V_ci x~_t) with a minus sign, but Eq. (14) and the surrounding text describe a positive spike in similarity; the sign should be fixed.
  2. [Abstract] The abstract says 'Our code base is available here', but the full text contains no URL or repository identifier; please provide a working link.
  3. [Figures 2-4] The text states that all results are smoothed before plotting, but the smoothing window is not specified; please report it or provide the unsmoothed curves.
  4. [Eqs. (5)-(9)] The slice notation [.,1] and [1,.] is used without definition; it should be defined explicitly, for example as selecting the column or row of the similarity matrix.
  5. [§3.1] The evaluation protocol for 'systematic generalization' is not described in detail; please state how the unseen-task split is constructed and which color/type combinations are held out.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: CAREL's reported sample-efficiency gains are measured against the external BabyAI reward signal, and the auxiliary and tracking losses are training heuristics rather than derived predictions.

full rationale

CAREL's central claim is an empirical sample-efficiency comparison (Table 1, Figures 2-3) against the BabyAI environment's external reward/success signal. The auxiliary X-CLIP loss (Eq. 12) is an off-the-shelf contrastive objective (Ma et al., 2022) applied to episodes filtered by a reward-based success criterion; the criterion is a training-data selection heuristic, not a fitted quantity later reported as a prediction. The instruction-tracking condition (Eq. 14) does compare a sub-task's similarity score to its own running average, which is self-referential as a heuristic calibration, but the paper does not derive sample-efficiency gains from this equation; the gains are measured externally. The unspecified success-reward fraction and the subjective 'midpoint of convergence' frame counts are reproducibility and robustness concerns, and the abstract promises a code link that does not appear in the full text, but none of these makes a prediction equivalent to an input by construction. No load-bearing self-citations or imported uniqueness claims appear; X-CLIP and BabyAI are external, citable baselines, and the evaluation metric is not produced by the method's own outputs.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the reward-based success filter, the similarity-spike detector, and joint training stability. All three are domain assumptions with limited validation; the free parameters are tuned on the benchmark tasks themselves, so the reported improvements are not parameter-free predictions.

free parameters (5)
  • lambda_C = 0.01
    Coefficient of the auxiliary loss in Eq. 13; ablation in Appendix A.2.1 shows performance drops for values outside [0.001, 0.01]. Chosen by tuning on the target tasks.
  • tau = 1
    Softmax temperature in Eq. 7 for similarity aggregation; set without sensitivity analysis.
  • k = 2
    Multiplicative threshold in Eq. 14 for detecting subtask completion; fixed without a reported sensitivity sweep.
  • success_threshold_fraction = unspecified
    Section 2.1 defines successful trajectories as those exceeding 'a fraction of the maximum achievable reward', but the fraction is never reported; it determines which episodes feed the contrastive loss.
  • masking_probability_schedule = tanh(current_frame/max_frames)
    Masking probability grows with training steps, but max_frames is not specified, so the schedule shape across tasks is unknown.
assumptions (3)
  • domain assumption Successful trajectories are semantically aligned with their instructions; unsuccessful ones are not.
    Section 2.1 uses the reward threshold to build positive and negative pairs for the X-CLIP loss. This is a modeling assumption about the correlation between reward and language-observation alignment.
  • domain assumption A sharp increase in subtask-token similarity relative to its running average marks completion of that subtask.
    Equation 14 operationalizes completion as S_t_ci >= k * running average; the paper states this but does not validate it against ground-truth subtask completion labels.
  • domain assumption Adding the auxiliary loss and sharing encoders with the PPO base model does not destabilize policy optimization.
    The paper trains PPO and the auxiliary loss jointly with shared encoders; it assumes gradient interference is benign. The ablations provide indirect support but no theoretical or measured interference analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CAREL: Instruction-guided reinforcement learning with cross-modal auxiliary objectives." pith.science (2026). https://pith.science/paper/EUZ32MFB

@misc{pith2026241119787,
  author       = {Pith},
  title        = {Pith review of: CAREL: Instruction-guided reinforcement learning with cross-modal auxiliary objectives},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EUZ32MFB}},
  note         = {Machine review of arXiv:2411.19787}
}
read the original abstract

Grounding the instruction in the environment is a key step in solving language-guided goal-reaching reinforcement learning problems. In automated reinforcement learning, a key concern is to enhance the model's ability to generalize across various tasks and environments. In goal-reaching scenarios, the agent must comprehend the different parts of the instructions within the environmental context in order to complete the overall task successfully. In this work, we propose CAREL (Cross-modal Auxiliary REinforcement Learning) as a new framework to solve this problem using auxiliary loss functions inspired by video-text retrieval literature and a novel method called instruction tracking, which automatically keeps track of progress in an environment. The results of our experiments suggest superior sample efficiency and systematic generalization for this framework in multi-modal reinforcement learning problems. Our code base is available here.

Figures

Figures reproduced from arXiv: 2411.19787 by the authors.

Figure 1
Figure 1. Overall view of CAREL. In this figure, we showcase CAREL over a candidate baseline model from Chevalier-Boisvert et al. (2018). (Left) The blue box handles the (masked) instruction and its lo￾cal/global representations, while the pink box contains the components related to observation. (Right) The purple box shows the calculation steps for the X-CLIP loss and tracks scores for instruction masking. observations with … view at source ↗
Figure 2
Figure 2. Test SRs indicating the overall effect of Vanilla CAREL on BabyAI. (The results are smoothed [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Test SRs indicating the effect of CAREL + instruction tracking on the baseline models. (The [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Sample efficiency comparison after applying the CAREL framework to the SHELM baseline. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Experiment on the impact of the coefficient for our auxiliary loss. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Effect of adding action embeddings to the observations in Instruction Tracking. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Results for using different embeddings for calculating the similarity score in Instruction Tracking [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Ablation study on the impact of combining CAREL with Instruction Tracking. [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 16 canonical work pages

  1. [1]

    Grounding language to autonomously-acquired skills via goal generation

    Ahmed Akakzia, C \'e dric Colas, Pierre-Yves Oudeyer, Mohamed Chetouani, and Olivier Sigaud. Grounding language to autonomously-acquired skills via goal generation. arXiv preprint arXiv:2006.07185, 2020

    arXiv 2006

  2. [2]

    Modular multitask reinforcement learning with policy sketches

    Jacob Andreas, Dan Klein, and Sergey Levine. Modular multitask reinforcement learning with policy sketches. In International conference on machine learning, pp.\ 166--175. PMLR, 2017

    work page 2017

  3. [3]

    Babyai: A platform to study the sample efficiency of grounded language learning

    Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, and Yoshua Bengio. Babyai: A platform to study the sample efficiency of grounded language learning. arXiv preprint arXiv:1810.08272, 2018

    arXiv

  4. [4]

    Evolving graphical planner: Contextual global planning for vision-and-language navigation

    Zhiwei Deng, Karthik Narasimhan, and Olga Russakovsky. Evolving graphical planner: Contextual global planning for vision-and-language navigation. Advances in Neural Information Processing Systems, 33: 0 20660--20672, 2020

    work page 2020

  5. [5]

    Contrastive learning as goal-conditioned reinforcement learning

    Benjamin Eysenbach, Tianjun Zhang, Sergey Levine, and Russ R Salakhutdinov. Contrastive learning as goal-conditioned reinforcement learning. Advances in Neural Information Processing Systems, 35: 0 35603--35620, 2022

    2022

  6. [6]

    LISA: Learning Interpretable Skill Abstractions from Language

    Divyansh Garg, Skanda Vaidyanath, Kuno Kim, Jiaming Song, and Stefano Ermon. Lisa: Learning interpretable skill abstractions from language, 2022. URL https://arxiv.org/abs/2203.00054

    work page Pith review arXiv 2022

  7. [7]

    Target languages (vs

    Hector Geffner. Target languages (vs. inductive biases) for learning to act and plan. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp.\ 12326--12333, 2022

    work page 2022

  8. [8]

    Using natural language for reward shaping in reinforcement learning

    Prasoon Goyal, Scott Niekum, and Raymond J Mooney. Using natural language for reward shaping in reinforcement learning. arXiv preprint arXiv:1903.02020, 2019

    arXiv 1903

Show all 35 references
  1. [9]

    Grounding language to entities and dynamics for generalization in reinforcement learning

    Austin W Hanjie, Victor Y Zhong, and Karthik Narasimhan. Grounding language to entities and dynamics for generalization in reinforcement learning. In International Conference on Machine Learning, pp.\ 4051--4062. PMLR, 2021

    2021

  2. [10]

    Improving long-horizon imitation through language prediction

    Donald Joseph Hejna III, Pieter Abbeel, and Lerrel Pinto. Improving long-horizon imitation through language prediction. 2021

    2021

  3. [11]

    Language as an abstraction for hierarchical deep reinforcement learning

    Yiding Jiang, Shixiang Shane Gu, Kevin P Murphy, and Chelsea Finn. Language as an abstraction for hierarchical deep reinforcement learning. Advances in Neural Information Processing Systems, 32, 2019

    2019

  4. [12]

    Exploration in deep reinforcement learning: A survey

    Pawel Ladosz, Lilian Weng, Minwoo Kim, and Hyondong Oh. Exploration in deep reinforcement learning: A survey. Information Fusion, 85: 0 1–22, September 2022. ISSN 1566-2535. doi:10.1016/j.inffus.2022.03.003. URL http://dx.doi.org/10.1016/j.inffus.2022.03.003

    2022 doi

  5. [13]

    Understanding the world through action

    Sergey Levine. Understanding the world through action. In Conference on Robot Learning, pp.\ 1752--1757. PMLR, 2022

    2022

  6. [14]

    Offline reinforcement learning: Tutorial, review, and perspectives on open problems

    Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643, 2020

    2005 arXiv

  7. [15]

    Fine-grained semantically aligned vision-language pre-training

    Juncheng Li, Xin He, Longhui Wei, Long Qian, Linchao Zhu, Lingxi Xie, Yueting Zhuang, Qi Tian, and Siliang Tang. Fine-grained semantically aligned vision-language pre-training. Advances in neural information processing systems, 35: 0 7290--7303, 2022

    2022

  8. [16]

    Goal-conditioned reinforcement learning: Problems and solutions

    Minghuan Liu, Menghui Zhu, and Weinan Zhang. Goal-conditioned reinforcement learning: Problems and solutions. arXiv preprint arXiv:2201.08299, 2022

    2022 arXiv

  9. [17]

    A survey of reinforcement learning informed by natural language

    Jelena Luketina, Nantas Nardelli, Gregory Farquhar, Jakob Foerster, Jacob Andreas, Edward Grefenstette, Shimon Whiteson, and Tim Rockt \"a schel. A survey of reinforcement learning informed by natural language. arXiv preprint arXiv:1906.03926, 2019

    1906 arXiv

  10. [18]

    X-clip: End-to-end multi-grained contrastive learning for video-text retrieval

    Yiwei Ma, Guohai Xu, Xiaoshuai Sun, Ming Yan, Ji Zhang, and Rongrong Ji. X-clip: End-to-end multi-grained contrastive learning for video-text retrieval. In Proceedings of the 30th ACM International Conference on Multimedia, pp.\ 638--647, 2022

    2022

  11. [19]

    Fast and slow learning of recurrent independent mechanisms

    Kanika Madan, Nan Rosemary Ke, Anirudh Goyal, Bernhard Sch \"o lkopf, and Yoshua Bengio. Fast and slow learning of recurrent independent mechanisms. arXiv preprint arXiv:2105.08710, 2021

    2021 arXiv

  12. [20]

    Ella: Exploration through learned language abstraction

    Suvir Mirchandani, Siddharth Karamcheti, and Dorsa Sadigh. Ella: Exploration through learned language abstraction. Advances in Neural Information Processing Systems, 34: 0 29529--29540, 2021

    2021

  13. [21]

    Semantic helm: An interpretable memory for reinforcement learning

    Fabian Paischer, Thomas Adler, Markus Hofmarcher, and Sepp Hochreiter. Semantic helm: An interpretable memory for reinforcement learning. arXiv preprint arXiv:2306.09312, 2023

    2023 arXiv

  14. [22]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp...

    2021

  15. [23]

    o der, Ozan \

    Frank R \"o der, Ozan \"O zdemir, Phuong DH Nguyen, Stefan Wermter, and Manfred Eppe. The embodied crossmodal self forms language and interaction: a computational cognitive review. Frontiers in psychology, 12: 0 716671, 2021

    2021

  16. [24]

    Where do goals come from? a generic approach to autonomous goal-system development

    Matthias Rolf and Minoru Asada. Where do goals come from? a generic approach to autonomous goal-system development. arXiv preprint arXiv:1410.5557, 2014

    2014 arXiv

  17. [25]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  18. [26]

    Decoupling representation learning from reinforcement learning

    Adam Stooke, Kimin Lee, Pieter Abbeel, and Michael Laskin. Decoupling representation learning from reinforcement learning. In International Conference on Machine Learning, pp.\ 9870--9879. PMLR, 2021

    2021

  19. [27]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  20. [28]

    Constrained contrastive reinforcement learning

    Haoyu Wang, Xinrui Yang, Yuhang Wang, and Lan Xuguang. Constrained contrastive reinforcement learning. In Asian Conference on Machine Learning, pp.\ 1070--1084. PMLR, 2023

    2023

  21. [29]

    Filip: Fine-grained interactive language-image pre-training

    Lewei Yao, Runhui Huang, Lu Hou, Guansong Lu, Minzhe Niu, Hang Xu, Xiaodan Liang, Zhenguo Li, Xin Jiang, and Chunjing Xu. Filip: Fine-grained interactive language-image pre-training. arXiv preprint arXiv:2111.07783, 2021

    2021 arXiv

  22. [30]

    Coca: Contrastive captioners are image-text foundation models

    Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917, 2022

    2022 arXiv

  23. [31]

    Towards sample efficient reinforcement learning

    Yang Yu. Towards sample efficient reinforcement learning. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18 , pp.\ 5739--5743. International Joint Conferences on Artificial Intelligence Organization, 7 2018. doi:10.24963/i...

    2018 doi

  24. [32]

    Taco: Temporal latent action-driven contrastive loss for visual reinforcement learning

    Ruijie Zheng, Xiyao Wang, Yanchao Sun, Shuang Ma, Jieyu Zhao, Huazhe Xu, Hal Daum \'e III, and Furong Huang. Taco: Temporal latent action-driven contrastive loss for visual reinforcement learning. arXiv preprint arXiv:2306.13229, 2023

    2023 arXiv

  25. [33]

    Rtfm: Generalising to novel environment dynamics via reading

    Victor Zhong, Tim Rockt \"a schel, and Edward Grefenstette. Rtfm: Generalising to novel environment dynamics via reading. arXiv preprint arXiv:1910.08210, 2019

    1910 arXiv

  26. [34]

    Deep learning for video-text retrieval: a review

    Cunjuan Zhu, Qi Jia, Wei Chen, Yanming Guo, and Yu Liu. Deep learning for video-text retrieval: a review. International Journal of Multimedia Information Retrieval, 12 0 (1): 0 3, 2023

    2023

  27. [35]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.