Pith. sign in

REVIEW 3 major objections 5 minor 39 references

Towards Effective Context for Meta-Reinforcement Learning: an Approach based on Contrastive Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-27 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Contrastive learning sharpens the context that lets meta-RL agents adapt fast.

desk verdict Useful empirical recipe for contrastive context encoders in meta-RL, but the exploration reward's theoretical derivation is overstated and needs substantial framing changes. read the letter →

arxiv 2009.13891 v3 pith:AGAXZFJB submitted 2020-09-29 cs.LG cs.AIstat.ML

Haotian Fu , Hongyao Tang , Jianye Hao , Chen Chen , Xidong Feng , Dong Li , Wulong Liu This is my paper · ORCID
classification cs.LGcs.AIstat.ML
keywords meta-reinforcementlearningcontext-basedmeta-RLcontrastiveInfoNCElossinformationgainexplorationpolicyintrinsicrewardlatentcontext
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that context-based meta-reinforcement learning fails for two reasons: the context encoder is trained with noisy or overfitting objectives, and the exploration policy does not deliberately collect informative trajectories. The authors propose CCM, a framework that trains the encoder with a contrastive loss over trajectories from the same versus different tasks, and that trains a separate exploration agent with an intrinsic reward derived from a lower bound on the information gain of a new transition. They show that both components improve adaptation performance on continuous-control benchmarks, especially sparse-reward environments, and that CCM's latent context clusters by task more cleanly than PEARL's.

What carries the argument

The key machinery is the information-gain intrinsic reward $r_{\text{aux}} = L_{\text{upper}} - L_{\text{lower}}$, where $L_{\text{lower}}$ is the standard InfoNCE contrastive loss and $L_{\text{upper}}$ is a modified loss over negative keys only. The paper proves that this difference is a lower bound on the mutual information $I(z|\tau_{1:i-1}; \tau_i)$, the information gained about the latent task belief $z$ from collecting transition $\tau_i$. This reward drives a separate exploration policy to collect trajectories that most increase the confidence of task inference.

What would settle it

If one could compute or estimate the true mutual information $I(z|\tau_{1:i-1}; \tau_i)$ directly (e.g., via a learned discriminative classifier) and show that the derived reward $L_{\text{upper}} - L_{\text{lower}}$ does not positively correlate with it across a range of tasks, the claim that the intrinsic reward estimates information gain would be falsified. Alternatively, a controlled experiment where the encoder is forced to be non-sufficient (e.g., by removing task-relevant information) should produce worse adaptation if the derivation's assumption is load-bearing.

Watch

Extended reading notes

Core claim

The central claim is that the quality of latent context in context-based Meta-RL can be improved by (1) training the context encoder with a contrastive loss that pulls together trajectory embeddings from the same task and pushes apart those from different tasks, and (2) training a separate exploration policy to maximize a tractable lower bound on the information gain about the task belief, which is expressed as the difference of two InfoNCE-style losses. The paper derives this lower bound theoretically and uses it as an intrinsic reward.

Load-bearing premise

The derivation of the intrinsic reward assumes the context encoder is 'context-sufficient' (Definition 4.1: $I(b_1,b_2) = I(e(b_1),b_2)$) and that the latent belief $z$ can be approximated by the embedding of a positive transition batch; the paper does not verify this sufficiency condition empirically.

Editorial extensions

If this is right

  • If the contrastive context encoder is effective, existing context-based Meta-RL algorithms can be upgraded by simply swapping in this encoder, without changing the RL objective.
  • If the information-gain exploration reward works as claimed, meta-RL agents will adapt faster in sparse-reward environments where random or task-agnostic exploration yields little signal.
  • The decoupling of exploration and execution policies could generalize to other meta-learning settings where a separate agent is tasked with collecting informative data.
  • The theoretical connection between contrastive loss and information gain suggests that similar intrinsic rewards can be derived for any contrastive representation learning framework.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The contrastive encoder's positive pairs come from the same task's replay buffer, which means the encoder must implicitly solve a task-inference problem during meta-training; this could serve as a stronger training signal than value-function reconstruction in sparse-reward regimes.
  • The derivation assumes the context encoder is 'context-sufficient', a condition that is not empirically verified in the paper. If the encoder is not sufficient, the intrinsic reward may not be a true lower bound on the information gain, though it might still be a useful heuristic.
  • The intrinsic reward's second term in Equation (13) acts as a regularizer that discourages exploring transitions that increase similarity to all tasks; this resembles an exploration-exploitation trade-off in the embedding space and could be connected to count-based or curiosity-driven exploration.
  • The method's reliance on a learned similarity function $f$ may introduce a non-stationary reward for the exploration policy during training, which could be a practical concern for on-policy exploration.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Request a human review

A listed scientist reviews the paper for a fee and the review publishes here regardless of verdict. See the reviewers or get listed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes CCM (Contrastive learning augmented Context-based Meta-RL), a context-based meta-reinforcement learning method with two contributions: a contrastively trained context encoder that pulls embeddings of same-task transition batches together and pushes different-task batches apart, and a separate exploration policy whose intrinsic reward is claimed to be a theoretically derived lower bound on the information gain I(z|τ_{1:i-1};τ_i). The empirical evaluation compares the contrastive encoder against value-recovery and dynamics-prediction encoders on MuJoCo benchmarks, and compares full CCM against PEARL, PEARL-CL, MAML, ProMP, and VariBAD on sparse-reward environments, with ablations for the intrinsic reward scale, context update frequency, and the regularization term in the exploration objective.

Significance. The empirical part of the paper is credible: the evaluation uses standard benchmarks, reports error bars, and includes ablations that isolate the influence of the contrastive encoder and of the exploration reward. The contrastive context encoder is a reasonable and potentially reusable idea. However, the paper's advertised theoretical contribution, namely that Eq. (11) is a derived lower bound on the information gain, is not supported by the derivation as written. If the information-gain objective were properly justified, it would be a valuable contribution to meta-RL exploration; with the current gaps, the exploration reward should be treated as a heuristic, and the central claim must be revised accordingly. The overall contribution is still of interest, but the theory needs substantial reworking.

major comments (3)
  1. [§4, Eq. (6)] The reduction of the information gain to the difference I(c_pos;c_{1:i}) - I(c_pos;c_{1:i-1}) is not established. Equation (6) requires both I(z;τ)=I(z;e(τ)) and the approximation z≈e(b_pos)=c_pos. Neither identity follows from Definition 4.1, which only asserts I(b1,b2)=I(e(b1),b2) for two same-task batches. The latent task belief z is a posterior over tasks given all collected transitions, whereas e(b_pos) is the embedding of an independent same-task batch; nothing in the contrastive training objective forces these to coincide. As a result, Eq. (11) lower-bounds a difference of mutual informations involving c_pos, not the intended information gain about the task belief z. Please prove the required identities under explicit assumptions, verify them empirically, or revise the claim to a heuristic.
  2. [Appendix D.1, Proposition 4.1] The proof of Proposition 4.1 assumes that the critic f equals the optimal density ratio p(c_pos|c_{1:i-1})/p(c_pos). However, f is trained to minimize L_lower in Eq. (8), not L_upper in Eq. (10), and no optimality condition for the upper-bound loss is verified. In addition, the step replacing the sum over negative keys by (W−1) times its expectation is an uncontrolled approximation, not an inequality; the expectation E_{c_neg}[p(c_j|c_{1:i-1})/p(c_j)] equals 1 only under a marginal-distribution condition that is not established for the learned embeddings. As written, Proposition 4.1 does not provide a valid upper bound for the f actually used in the algorithm, and the derivation of Eq. (11) is therefore not a proof.
  3. [§4, Eq. (5) and surrounding text] The notation I(z|τ_{1:i-1};τ_i) is nonstandard, and the identity in Eq. (5) actually expresses the conditional mutual information I(z;τ_i|τ_{1:i-1}) = I(z;τ_{1:i}) - I(z;τ_{1:i-1}). This is not by itself an error, but the subsequent text conflates the task-belief variable z with the contrastive embedding c_pos. Since the intrinsic reward in Eq. (12) is a difference of contrastive losses involving c_pos, the statement that the exploration policy maximizes the information gain about the task belief is unsupported. The authors should either supply a rigorous chain of equalities and inequalities connecting the two quantities or explicitly reframe the objective as a heuristic information-gain surrogate.
minor comments (5)
  1. [§3, Eq. (3)] The notation f(z_q, z_k_n) is under-specified; please state whether f is a bilinear score, how the momentum encoder is used, and how the query and key encoders differ.
  2. [§2.2, Eq. (2)] The counting convention for K is ambiguous: Eq. (1) uses K keys, while Eq. (2) writes log(K); later W denotes the number of tasks. Please state the relationship between the number of negatives and W so that the lower bound in Eq. (7) is clear.
  3. [Algorithm 1, line 14] The update step 'Update θ_enc and θ_exe with RL loss and contrastive loss' does not specify which loss terms contribute gradients to which parameters; please make the objective explicit, including the roles of L_contrastive, the SAC losses, and any KL term.
  4. [Appendix B] The hyperparameter tables are useful, but the text should state the number of random seeds used for the reported curves and how the standard-deviation error bars are computed; this information is only implied by 'meta-train 3 models'.
  5. [Throughout] There are several typographical and wording issues, e.g., 'conprehensively' in Section 6.1, 'does not consider' in Section 5.1, and the inconsistent use of 'z' versus 'c' for latent context. A careful proofreading pass is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Eq. (11) is a bound manipulation built on standard InfoNCE bounds, not a fitted prediction; the z≈cpos step is an unverified assumption rather than a circular reduction.

full rationale

The derivation of the intrinsic reward r_aux = Lupper - Llower in Eq. (11) is not circular. It starts from the definition of information gain (Eqs. 4-5), rewrites it under the stated context-sufficiency condition and the explicit approximation z≈e(bpos)=cpos (Eq. 6), and then combines the standard InfoNCE lower bound (Eq. 7, from van den Oord et al.) with a proposed upper bound (Prop. 4.1) to form a lower bound on the mutual-information difference. The same contrastive encoder/critic f appears both in the representation learning loss and in the intrinsic reward, but that is a deliberate self-supervised coupling, not the fitting of a parameter to a target and then renaming it a prediction. The main weaknesses are correctness risks rather than circularity: Definition 4.1 does not by itself imply z≈cpos, and the Appendix D.1 proof of Prop. 4.1 assumes the optimal density-ratio form of f even though f is trained with L_lower; these gaps mean the theoretical claim is under-supported, but they do not make the output equivalent to the input by construction. The only author-overlap citation (Fu et al., 2019, MGHRL) appears in the related-work survey and is not load-bearing for any derivation or empirical claim.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central method relies on standard RL/Meta-RL machinery plus one strong unverified assumption (context sufficiency). Several hyperparameters are tuned per environment; alpha and the contrastive loss scale are the most load-bearing for the exploration and encoder objectives.

free parameters (4)
  • intrinsic reward scale alpha = 1 (cheetah-sparse), 2 (walker-sparse, hard-point-robot)
    Selected from {0.1, 1, 10} per environment; the ablation in Appendix C.2 shows performance is sensitive to it.
  • KL divergence weight beta = chosen from {0.01, 0.1, 1}
    Standard PEARL-style objective weight, tuned per environment.
  • contrastive loss scale = chosen from {1, 5, 10}
    Weight on the contrastive term when combined with RV or DP losses.
  • number of exploration episodes K = 2 (cheetah-sparse, walker-sparse), 4 (hard-point-robot)
    Hyperparameter controlling how much data is used to form context during meta-testing.
assumptions (4)
  • standard math InfoNCE loss lower-bounds mutual information (Eq. 2)
    Taken from van den Oord et al. 2018; used as the basis for both the contrastive encoder and the intrinsic reward.
  • domain assumption The context encoder is context-sufficient (Def 4.1)
    Stated in Section 4 before Eq. (6); unverified, and the derivation of the intrinsic reward depends on it.
  • ad hoc to paper Negative context samples are drawn from the marginal distribution so that expectation over c_neg of p(c_j|c)/p(c_j) equals 1
    Used in the proof of Proposition 4.1 in Appendix D.1 to replace the sum over negatives with (W-1) times an expectation equal to 1.
  • domain assumption Tasks share structure and differ in dynamics or reward
    Standard Meta-RL assumption from Section 2.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Effective Context for Meta-Reinforcement Learning: an Approach based on Contrastive Learning." pith.science (2026). https://pith.science/paper/AGAXZFJB

@misc{pith2026200913891,
  author       = {Pith},
  title        = {Pith review of: Towards Effective Context for Meta-Reinforcement Learning: an Approach based on Contrastive Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AGAXZFJB}},
  note         = {Machine review of arXiv:2009.13891}
}
read the original abstract

Context, the embedding of previous collected trajectories, is a powerful construct for Meta-Reinforcement Learning (Meta-RL) algorithms. By conditioning on an effective context, Meta-RL policies can easily generalize to new tasks within a few adaptation steps. We argue that improving the quality of context involves answering two questions: 1. How to train a compact and sufficient encoder that can embed the task-specific information contained in prior trajectories? 2. How to collect informative trajectories of which the corresponding context reflects the specification of tasks? To this end, we propose a novel Meta-RL framework called CCM (Contrastive learning augmented Context-based Meta-RL). We first focus on the contrastive nature behind different tasks and leverage it to train a compact and sufficient context encoder. Further, we train a separate exploration policy and theoretically derive a new information-gain-based objective which aims to collect informative trajectories in a few steps. Empirically, we evaluate our approaches on common benchmarks as well as several complex sparse-reward environments. The experimental results show that CCM outperforms state-of-the-art algorithms by addressing previously mentioned problems respectively.

Figures

Figures reproduced from arXiv: 2009.13891 by the authors.

Figure 1
Figure 1. Contrastive Context Encoder Exploration Replay Buffer Execution Replay Buffer Execution Agent Exploration Agent Context Encoder Contrastive Loss RL Loss RL Loss 𝑏𝑒𝑥𝑝 𝑏𝑒𝑥𝑒 𝑏𝑒𝑛𝑐 𝑧𝑒𝑥𝑝 𝑧𝑒𝑥𝑝 [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 3
Figure 3. Comparison for different context encoder training strategies. Our methods CCM + DP and [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. CCM’s overall performance compared with state-of-the-art Meta-RL methods on complex [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (6 more)
Figure 5
Figure 5. Figure 5: Visualization of context in embedding space. Different color represents context from [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Meta-RL tasks: left-to-right: humanoid, ant, half cheetah, and walker. B Implementation Details For each environment, we meta-train 3 models, and meta-test each of them. The evaluation results reflect the average return in the last episode over a test rollout. We show …
Figure 7
Figure 7. Figure 7: CCM adaptation procedure. Environment Meta-train tasks Meta-test tasks Meta batch size humanoid-dir 100 30 20 cheetah-mass 30 5 16 cheetah-mass-OOD 30 5 16 cheetah-vel-OOD 50 5 24 ant-mass 50 5 24 [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Contrastive context encoder without combining other training strategies [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: (a) Context updating frequency comparison; (b) Intrinsic reward scale comparison [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Influence of the regularization term D Additional Details D.1 Derivation of Proposition 1. Proof. As defined in van den Oord, Li, and Vinyals (2018), the optimal value for exp(f(c1:i−1, cpos)) is given by p(cpos|c1:i−1) p(cpos) , inserting this back into Equation (10)…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 36 canonical work pages

  1. [1]

    Anand, A.; Racah, E.; Ozair, S.; Bengio, Y.; C \^ o t \' e , M.; and Hjelm, R. D. 2019. Unsupervised State Representation Learning in Atari. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, 8766--8779

    work page 2019

  2. [2]

    Brockman, G.; Cheung, V.; Pettersson, L.; Schneider, J.; Schulman, J.; Tang, J.; and Zaremba, W. 2016. OpenAI Gym. CoRR abs/1606.01540

    work page Pith review arXiv 2016

  3. [3]

    Chen, T.; Kornblith, S.; Norouzi, M.; and Hinton, G. E. 2020. A Simple Framework for Contrastive Learning of Visual Representations. CoRR abs/2002.05709

    work page Pith review arXiv 2020

  4. [4]

    RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning

    Duan, Y.; Schulman, J.; Chen, X.; Bartlett, P. L.; Sutskever, I.; and Abbeel, P. 2016. RL \^ 2 : Fast Reinforcement Learning via Slow Reinforcement Learning. CoRR abs/1611.02779

    work page Pith review arXiv 2016

  5. [5]

    Dwibedi, D.; Tompson, J.; Lynch, C.; and Sermanet, P. 2018. Learning Actionable Representations from Visual Observations. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018 , 1577--1584. IEEE

    work page 2018

  6. [6]

    Fakoor, R.; Chaudhari, P.; Soatto, S.; and Smola, A. J. 2020. Meta-Q-Learning. In 8th International Conference on Learning Representations, ICLR 2020 . OpenReview.net

    work page 2020

  7. [7]

    Finn, C.; Abbeel, P.; and Levine, S. 2017. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017 , 1126--1135

    work page 2017

  8. [8]

    Fu, H.; Tang, H.; Hao, J.; Liu, W.; and Chen, C. 2019. MGHRL: Meta Goal-generation for Hierarchical Reinforcement Learning. ArXiv abs/1909.13607

    work page Pith review arXiv 2019

Show all 39 references
  1. [9]

    Gupta, A.; Mendonca, R.; Liu, Y.; Abbeel, P.; and Levine, S. 2018. Meta-Reinforcement Learning of Structured Exploration Strategies. In NeurIPS

    2018

  2. [10]

    Haarnoja, T.; Zhou, A.; Abbeel, P.; and Levine, S. 2018. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018 , 1856--1865

    2018

  3. [11]

    He, K.; Fan, H.; Wu, Y.; Xie, S.; and Girshick, R. B. 2020. Momentum Contrast for Unsupervised Visual Representation Learning. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020 , 9726--9735. IEEE

    2020

  4. [12]

    J.; Srinivas, A.; Fauw, J

    H \' e naff, O. J.; Srinivas, A.; Fauw, J. D.; Razavi, A.; Doersch, C.; Eslami, S. M. A.; and van den Oord, A. 2019. Data-Efficient Image Recognition with Contrastive Predictive Coding. CoRR abs/1905.09272

    2019 arXiv

  5. [13]

    Lee, K.; Seo, Y.; Lee, S.; Lee, H.; and Shin, J. 2020. Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learning. CoRR abs/2005.06800

    2020 arXiv

  6. [14]

    P.; Hunt, J

    Lillicrap, T. P.; Hunt, J. J.; Pritzel, A.; Heess, N.; Erez, T.; Tassa, Y.; Silver, D.; and Wierstra, D. 2016. Continuous control with deep reinforcement learning. In 4th International Conference on Learning Representations, ICLR 2016

    2016

  7. [15]

    Linsker, R. 1988. Self-Organization in a Perceptual Network. Computer 21(3): 105--117

    1988

  8. [16]

    Z.; Raghunathan, A.; Liang, P.; and Finn, C

    Liu, E. Z.; Raghunathan, A.; Liang, P.; and Finn, C. 2020. Explore then Execute: Adapting without Rewards via Factorized Meta-Reinforcement Learning. CoRR abs/2008.02790

    2020 arXiv

  9. [17]

    Liu, H.; Socher, R.; and Xiong, C. 2019. Taming MAML: Efficient unbiased meta-reinforcement learning. In ICML

    2019

  10. [18]

    Maaten, L. V. D.; and Hinton, G. E. 2008. Visualizing Data using t-SNE. Journal of Machine Learning Research 9: 2579--2605

    2008

  11. [19]

    A.; Veness, J.; Bellemare, M

    Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A. A.; Veness, J.; Bellemare, M. G.; Graves, A.; Riedmiller, M. A.; Fidjeland, A.; Ostrovski, G.; Petersen, S.; Beattie, C.; Sadik, A.; Antonoglou, I.; King, H.; Kumaran, D.; Wierstra, D.; Legg, S.; and Hassabis, D. 2015. Human-leve...

    2015 doi

  12. [20]

    K.; and Mammone, R

    Naik, D. K.; and Mammone, R. 1992. Meta-neural networks that learn by learning. [Proceedings 1992] IJCNN International Joint Conference on Neural Networks 1: 437--442 vol.1

    1992

  13. [21]

    A.; and Darrell, T

    Pathak, D.; Agrawal, P.; Efros, A. A.; and Darrell, T. 2017. Curiosity-driven Exploration by Self-supervised Prediction. In Precup, D.; and Teh, Y. W., eds., Proceedings of the 34th International Conference on Machine Learning, ICML 2017 , volume 70 of Proceedings of Machine L...

    2017

  14. [22]

    Poole, B.; Ozair, S.; van den Oord, A.; Alemi, A.; and Tucker, G. 2019. On Variational Bounds of Mutual Information. In Chaudhuri, K.; and Salakhutdinov, R., eds., Proceedings of the 36th International Conference on Machine Learning, ICML 2019 , volume 97 of Proceedings of Mac...

    2019

  15. [23]

    Rakelly, K.; Zhou, A.; Finn, C.; Levine, S.; and Quillen, D. 2019. Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019 , 5331--5340

    2019

  16. [24]

    Rothfuss, J.; Lee, D.; Clavera, I.; Asfour, T.; and Abbeel, P. 2019. ProMP: Proximal Meta-Policy Search. In 7th International Conference on Learning Representations, ICLR 2019

    2019

  17. [25]

    Schmidhuber, J. 1987. Evolutionary principles in self-referential learning

    1987

  18. [26]

    I.; and Moritz, P

    Schulman, J.; Levine, S.; Abbeel, P.; Jordan, M. I.; and Moritz, P. 2015. Trust Region Policy Optimization. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015 , 1889--1897

    2015

  19. [27]

    Sermanet, P.; Lynch, C.; Chebotar, Y.; Hsu, J.; Jang, E.; Schaal, S.; and Levine, S. 2018. Time-Contrastive Networks: Self-Supervised Learning from Video. In 2018 IEEE International Conference on Robotics and Automation, ICRA 2018 , 1134--1141. IEEE

    2018

  20. [28]

    Srinivas, A.; Laskin, M.; and Abbeel, P. 2020. CURL: Contrastive Unsupervised Representations for Reinforcement Learning. CoRR abs/2004.04136

    2020 arXiv

  21. [29]

    Thrun, S.; and Pratt, L. Y. 1998. Learning to Learn. In Springer US

    1998

  22. [30]

    Todorov, E.; Erez, T.; and Tassa, Y. 2012. MuJoCo: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2012 , 5026--5033

    2012

  23. [31]

    van den Oord, A.; Li, Y.; and Vinyals, O. 2018. Representation Learning with Contrastive Predictive Coding. CoRR abs/1807.03748

    2018 arXiv

  24. [32]

    X.; Kurth - Nelson, Z.; Tirumala, D.; Soyer, H.; Leibo, J

    Wang, J. X.; Kurth - Nelson, Z.; Tirumala, D.; Soyer, H.; Leibo, J. Z.; Munos, R.; Blundell, C.; Kumaran, D.; and Botvinick, M. 2016. Learning to reinforcement learn. CoRR abs/1611.05763

    2016 arXiv

  25. [33]

    Wu, Z.; Xiong, Y.; Yu, S.; and Lin, D. 2018. Unsupervised Feature Learning via Non-parametric Instance Discrimination. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 3733--3742

    2018

  26. [34]

    Zhang, J.; Wang, J.; Hu, H.; Chen, Y.; Fan, C.; and Zhang, C. 2020. Learn to Effectively Explore in Context-Based Meta-RL. CoRR abs/2006.08170

    2020 arXiv

  27. [35]

    Zhou, W.; Pinto, L.; and Gupta, A. 2019. Environment Probing Interaction Policies. In 7th International Conference on Learning Representations, ICLR 2019 . OpenReview.net

    2019

  28. [36]

    M.; Shiarlis, K.; Igl, M.; Schulze, S.; Gal, Y.; Hofmann, K.; and Whiteson, S

    Zintgraf, L. M.; Shiarlis, K.; Igl, M.; Schulze, S.; Gal, Y.; Hofmann, K.; and Whiteson, S. 2020. VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning. In 8th International Conference on Learning Representations, ICLR 2020 . OpenReview.net

    2020

  29. [37]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution isbn issn journal key month note number organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sen...

  30. [38]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  31. [39]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 27, 2026 · model on record in the stance chip above.