Pith. sign in

REVIEW 2 major objections 5 minor 2 cited by

A Survey of State Representation Learning for Deep Reinforcement Learning

T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This survey argues that the scattered literature on state representation learning for deep reinforcement learning can be organized into six method families, and that matching family to regime is the practical way to choose a method.

desk verdict A genuinely useful taxonomy of SRL methods, with a practical guide that outruns its evidence; worth reading and worth refereeing, but Table 4 should be read as a heuristic, not a result. read the letter →

arxiv 2506.17518 v1 pith:RZ2QLXYF submitted 2025-06-20 cs.LG cs.AIstat.ML

Ayoub Echchahed , Pablo Samuel Castro This is my paper
classification cs.LGcs.AIstat.ML
keywords staterepresentationlearningdeepreinforcementtaxonomybisimulationmetriccontrastiveself-predictivedataaugmentationevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is a survey, not a new algorithm. It claims that nearly all recent state representation learning (SRL) methods for model-free online deep RL fall into six families distinguished by the mechanism that shapes the representation: metric-based, auxiliary-task, data augmentation, contrastive, non-contrastive, and attention-based. For each family it specifies what the method does, what benefits are typical, and what failure modes appear in practice. Its practical payoff is a selection guide: metric-based methods for known dynamics with dense rewards, auxiliary tasks for sparse rewards, augmentation for visual distractors, contrastive methods when ample negatives exist, non-contrastive methods in low-batch regimes, and attention when interpretability is central.

What carries the argument

The organizing device is the six-class taxonomy itself plus the comparison table that pairs each class with a use case, a benefit, and a cost. The load-bearing theoretical objects inside the taxonomy are the bisimulation metric (a fixed point that measures behavioral distance between states via rewards and transition distributions) and the contrastive InfoNCE loss, with the non-contrastive collapse-prevention mechanisms (covariance regularization, predictor asymmetry, stop-gradient) and the attention mask as the remaining structural anchors. These are the mechanisms that determine what each class can and cannot do, and the taxonomy's claim is that mechanism, not benchmark, is the right lens for comparing methods.

What would settle it

Run one representative method from each of the six classes on the same three suites — dense-reward known dynamics, sparse rewards, and visual distractor tasks — with matched environment interactions, seeds, and compute, and record total return plus probing scores. If the best method per suite consistently disagrees with Table 4's use cases, or if a hybrid method outperforms its declared class on every regime, the proposed guide is falsified.

Watch

Extended reading notes

Core claim

The central claim is that the SRL literature is not an unstructured pile of tricks: each method can be understood through the principle it uses to impose structure on the latent space, and that principle determines where the method helps and where it breaks. Metric-based methods shape the space so that task-relevant distances between embeddings match a behavioral metric such as bisimulation, giving provable value-function bounds. Auxiliary-task methods add prediction heads whose loss gradients enrich the shared encoder. Data augmentation methods enforce invariance to geometric and photometric transformations without directly building a representation. Contrastive methods pull positive pairs together and push negatives apart, non-contrastive methods only pull positives together while guarding against collapse, and attention-based methods highlight task-relevant input regions. The paper argues that these six mechanisms, together with the evaluation tools it reviews, form a practical guide for selecting SRL methods by regime.

Load-bearing premise

The load-bearing premise is that the cross-class comparisons and the Table 4 guidelines, which the paper admits are largely theoretical or drawn from prior studies, carry over reliably to a practitioner's setting; if they do not, the taxonomy still organizes the field but the selection guide can mislead.

Editorial extensions

If this is right

  • A practitioner facing sparse rewards can go directly to auxiliary-task methods, while dense-reward settings with known dynamics point to metric-based methods, per the paper's comparison table.
  • Visual distractors and invariance needs point to data augmentation, and contrastive methods when large batches of negatives are available; low-batch regimes point to non-contrastive methods.
  • Evaluation can be cheaper and more diagnostic by using linear probing and nearest-neighbor geometry alongside total return, rather than relying on end performance alone.
  • The taxonomy's failure-mode catalogue (embedding collapse and explosion, negative-mining difficulty, dimensional collapse, attention overfitting) gives a way to debug a method by class.
  • The paper identifies future directions: multi-task sharing, offline pre-training, pre-trained visual representations, zero-shot RL, language-model priors, and multimodal SRL.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: Because the paper concedes methods can be hybrid, Table 4's one-line use-case advice is better read as a heuristic; a method that combines, say, contrastive and auxiliary-task ideas may inherit both the strengths and the failure modes.
  • Editorial: A concrete way to test the taxonomy beyond the paper's scope would be a unified benchmark running one representative of each class on the same distractor, sparse-reward, and dense-reward suites with matched budgets and seeds.
  • Editorial: The review's separation of auxiliary tasks (prediction heads) from auxiliary losses could become a useful vocabulary for describing methods precisely.
  • Editorial: If linear probing correlates strongly with downstream control as the cited work suggests, probing could serve as a screening tool before expensive RL runs, even in settings the paper does not evaluate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This survey organizes state representation learning (SRL) for model-free online deep reinforcement learning into six classes—metric-based, auxiliary tasks, data augmentation, contrastive, non-contrastive, and attention-based—and additionally discusses spectral and information-bottleneck approaches. For each class it describes mechanisms, representative methods, benefits, and limitations, then reviews evaluation techniques and future directions. The paper's central value claim is that the proposed taxonomy and the practical guidelines in Section 3.9 and Table 4 can serve as a guide for selecting an SRL method according to the learning regime (e.g., known dynamics, sparse rewards, visual distractors).

Significance. If its descriptive content is accurate, this is a useful and broadly up-to-date map of a fast-moving area. The survey's main strengths are its breadth, its explicit scope restriction to model-free online RL, and its balanced treatment of strengths and limitations within each class; the evaluation section is also practically helpful. The paper does not run a unified comparison, and the Limitations section candidly states that cross-class comparisons are 'largely theoretical or rely on previous studies.' That honesty is a strength, but it also means the selection-guidance claims in Table 4 are not directly validated. The taxonomy itself is a reasonable contribution; the prescriptive use-case table needs qualification.

major comments (2)
  1. [Section 3.9 / Table 4] The selection guide is the load-bearing applied contribution, but its prescriptive 'Use Case' column is not supported by the evidence in the survey and is internally contradicted in at least one row. The 'Metric-based — Known dynamics' row conflicts with Section 3.2, where DBC (Zhang et al., 2021) and MICo (Castro et al., 2021) are described as model-free methods that estimate state similarity from sampled transitions and do not require known dynamics. The same row's 'Sparse-reward failure' is too blanket, since Section 3.2 explicitly discusses Kemertas & Aumentado-Armstrong (2021) and Chen et al. (2024b) as addressing sparse-reward settings. Given that the survey's stated purpose is to guide new researchers, the table should either be reframed as 'commonly reported strengths' with a prominent caveat that no controlled cross-class comparison was performed, or it should be revised to avoid one-line prescriptive claims that the cited literature does not support.
  2. [Section 3.2, Eq. (2)] The sentence following Eq. (2) is garbled: 'In practice, it is common to set c=γ, which corresponds to the discount factor in RL, without using (1−c).' With c=γ, the displayed update rule uses both (1−c) and c, so the text as written is mathematically inconsistent with the equation. If the intended statement is that some practical implementations drop the (1−c) reward-difference term, that is a different update rule and should be stated explicitly; if not, the sentence should be corrected. Because Eq. (2) is the formal definition of bisimulation for the metric-based class, this is a load-bearing technical description and should be fixed.
minor comments (5)
  1. [Section 5 (Table 4)] The table in Section 5 is also numbered 'Table 4', duplicating the numbering of the comparison table in Section 3.9; renumber the later table.
  2. [References] The in-text citation 'McInroe et al.' lacks a year, and the corresponding reference entry also has no year or volume; please complete the bibliographic information.
  3. [References] The references for van den Oord et al. (2018a) and (2018b) appear to be the same CPC paper with identical titles and URLs; consolidate or clearly distinguish them.
  4. [Section 3.8] The abstract promises six principal classes, but Section 3.8 introduces spectral and information-bottleneck approaches as 'two additional classes'; adding a sentence that explicitly situates these within or outside the six-class taxonomy would improve coherence.
  5. [Table 3] The 'Angular distance' row divides by π, while the 'Cosine distance' row lists a similarity score ranging from -1 to 1; the table would be clearer if it distinguished distance conventions (e.g., 1 − cosine similarity) from similarity conventions.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the survey's taxonomy and guidelines are independent content, and the authors' self-citations are descriptive examples rather than load-bearing arguments.

full rationale

This is a survey paper, not a derivation, so there is no claimed chain from first principles to a predicted result. Its product is a six-class taxonomy, descriptions of method mechanisms, and practical selection guidelines. The equations presented (e.g., Eq. 1 for metric-based objectives, Eq. 4 for InfoNCE) are restatements of existing objectives from the cited literature, not new results derived from the survey's own definitions. The benefits and limitations of each class are attributed to prior studies and are not used to prove the taxonomy itself. The paper's self-citations (Castro 2020; Castro et al. 2021; Le Lan et al. 2021; Agarwal et al. 2021a; Farebrother et al. 2023) appear in method descriptions and in narrow supporting claims, such as 'some metric-based methods avoid training extra parameters (Castro et al., 2021)'. These are descriptive references to specific methods, not circular justifications of the survey's organizing claims, so they do not make the taxonomy or the guidelines depend on the authors' prior work. The Limitations section explicitly states: 'The comparisons between classes are also largely theoretical or rely on previous studies. Future work could include experimental evaluations to compare approaches on multiple aspects.' This is an admission that the practical comparisons lack unified experimental support, but it is a statement of evidential weakness, not circularity: the survey does not fit a parameter to data and then rename that fit as a prediction, and it does not import a uniqueness theorem from the authors' prior papers to force its taxonomy. Author overlap could create selection bias in which methods are highlighted, but that is not circular reasoning under the review rules. Overall, the central content is self-contained as a literature organization and no specific reduction of a claimed result to its own inputs was found.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

This survey introduces no fitted parameters, no free constants, and no new postulated entities. Its contributions are organizational: a six-class taxonomy, precision notes (e.g., the distinction between auxiliary tasks and auxiliary losses in Section 3.3), and evaluation guidance. Everything it describes is drawn from the cited literature, and the authors disclose that comparisons are theoretical or borrowed from prior studies (Limitations section). The axioms listed capture the scope restrictions (model-free online RL) and the assumed validity of the formal background (MDP/POMDP and bisimulation metric convergence) on which the descriptions rely.

assumptions (3)
  • domain assumption The six-class partition (metric-based, auxiliary tasks, augmentation, contrastive, non-contrastive, attention) is a meaningful and mostly disjoint organization of the SRL literature.
    Section 3.1 introduces the taxonomy; the paper admits some methods are hybrid and coverage is not exhaustive, so the partition's completeness and disjointness are assumed, not proven.
  • standard math The MDP/POMDP formalism (Sections 2.1-2.2) and the bisimulation metric update rule (Section 3.2a, Eq. 2) are used correctly as background, with convergence of the iterative bisimulation update borrowed from cited works (Ferns et al. 2012; Castro et al. 2021).
    The survey relies on these formal tools for its descriptions but does not prove them; the 'without using (1-c)' sentence near Eq. 2 is a point of technical confusion.
  • domain assumption The scope restriction to model-free online RL (with offline pretraining discussed in Section 5) means conclusions and guidelines do not necessarily transfer to model-based or offline RL.
    Stated in Section 2.4 and in the Limitations section; the survey's guidance is scoped to this setting, and the authors flag this as a limitation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Survey of State Representation Learning for Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/RZ2QLXYF

@misc{pith2026250617518,
  author       = {Pith},
  title        = {Pith review of: A Survey of State Representation Learning for Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RZ2QLXYF}},
  note         = {Machine review of arXiv:2506.17518}
}
read the original abstract

Representation learning methods are an important tool for addressing the challenges posed by complex observations spaces in sequential decision making problems. Recently, many methods have used a wide variety of types of approaches for learning meaningful state representations in reinforcement learning, allowing better sample efficiency, generalization, and performance. This survey aims to provide a broad categorization of these methods within a model-free online setting, exploring how they tackle the learning of state representations differently. We categorize the methods into six main classes, detailing their mechanisms, benefits, and limitations. Through this taxonomy, our aim is to enhance the understanding of this field and provide a guide for new researchers. We also discuss techniques for assessing the quality of representations, and detail relevant future directions.

Figures

Figures reproduced from arXiv: 2506.17518 by the authors.

Figure 1
Figure 1. Simplified intuition for SRL: Raw sensory inputs from a busy environment are distilled into compressed, task-relevant representations, enabling improved decision-making. While the color of buildings or the type of roadside trees might be detectable, these details are irrelevant to effective navigation. Instead, elements such as the positions and velocities of vehicles, traffic light statuses, road signs, and pedestr… view at source ↗
Figure 2
Figure 2. Comparison of End-to-End RL (left) and SRL+RL (right). End-to-end directly maps high-dimensional inputs to actions, while SRL separates representation learning and policy learning. Motivation. In recent years, there has been a growth in methods that integrate improved representation learning into DRL, using various approaches. However, many works present inconsistent structuring and categorization of these approache… view at source ↗
Figure 3
Figure 3. Illustration of State Representation Learning (SRL) for RL, where a parametrized transformation ϕ is learned, mapping sequences of observations to representations. Two configurations are presented depending if a value-based RL approach is used (right) or a policy-based one (left). However, not all representation functions are useful to obtain; the goal is to learn an encoder ϕ that captures the essential characteris… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Illustration of some optimal state representation properties. The left section demonstrates invariance to noise, distractions, and masking in the representation space, preserving task-relevant information. The right section illustrates disentanglement, where changes in…
Figure 5
Figure 5. Figure 5: Metric-based methods shape the representation space to capture task-relevant information. Representa￾tions with similar outcomes (e.g., good basketball trajectory) have minimal distance d1, while those with different outcomes (e.g., wrong basketball trajectory) have a …
Figure 6
Figure 6. Figure 6: The representation xt of an RL agent is used to make additional predictions on auxiliary task function(s). These predictions are used to improve the representation itself. 11 [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Reconstruction as an auxiliary task: The encoder learns compact latent representations by ensuring that the original observation can be reconstructed from the representation. 12 [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Dynamics modeling as an auxiliary task: Forward Dynamics Models predict future representation(s) based on the current representation and action, capturing environment dynamics. Inverse Dynamics Models predict action that caused transitions between representations, emph…
Figure 9
Figure 9. Figure 9: Implicit DA (left) augments observations directly used to train the policy and/or value network, promoting robustness through diversity without explicit constraints. Explicit DA (right) augments observations supplemented by regularization penalties that enforce Q/π inv…
Figure 10
Figure 10. Figure 10: Common observation augmentations in RL. Geometric transformations alter spatial properties like cropping or flipping, while photometric transformations alter visual features such as lighting and color. Other augmentations also exist, such as masking or noise injection…
Figure 11
Figure 11. Figure 11: Two contrastive learning frameworks: (1) Instance-discriminative contrastive learning with data aug￾mentation (left), and (2) Temporal contrastive learning (right). Categorization: Contrastive methods can be categorized by how they generate positive and negative obser…
Figure 12
Figure 12. Figure 12: Distinction between contrastive and non-contrastive approaches. Contrastive methods rely on both positive and negative pairs to structure the representation space, maximizing similarity within positive pairs and minimizing it for negatives. Non-contrastive methods, wh…
Figure 13
Figure 13. Figure 13: Top: A self-attention module operates on high-level feature maps extracted from observations, creating masks that reweight features via element-wise multiplication. Bottom: An attention bottleneck is applied directly to observations, where an attention module selectiv…

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Unified Causal-Origin Taxonomy of Distributional Shifts in Reinforcement Learning

    cs.LG 2026-06 conditional novelty 6.5 of 10

    Distributional shift in RL is classified by which POMDP generative component changes (internal agent vs external environment) and by whether the time boundary is explicit, implicit, or hybrid.

  2. Interpret Policies in Deep Reinforcement Learning using SILVER with RL-Guided Labeling: A Model-level Approach to High-dimensional and Multi-action Environments

    cs.LG 2025-10 reject novelty 4.0 of 10

    SILVER with RL-guided labeling: SHAP plus clustering plus policy-query labels plus decision trees or regression to interpret multi-action Atari policies.

Reference graph

Works this paper leans on

157 extracted references · 35 canonical work pages · cited by 2 Pith papers

  1. [1]

    A theory of abstraction in reinforcement learning

    David Abel. A theory of abstraction in reinforcement learning. arXiv preprint arXiv:2203.00397, 2022

    arXiv 2022

  2. [2]

    Machado, Pablo Samuel Castro, and Marc G Bellemare

    Rishabh Agarwal, Marlos C. Machado, Pablo Samuel Castro, and Marc G Bellemare. Contrastive behavioral similarity embeddings for generalization in reinforcement learning. In International Conference on Learning Representations, 2021 a . URL https://openreview.net/forum?id=qda7-sVg84

    2021

  3. [3]

    Bellemare

    Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron Courville, and Marc G. Bellemare. Deep reinforcement learning at the edge of the statistical precipice. In Advances in Neural Information Processing Systems, volume 34, pp.\ 29314--29327, 2021 b

    2021

  4. [4]

    Proto successor measure: Representing the space of all possible solutions of reinforcement learning, 2024

    Siddhant Agarwal, Harshit Sikchi, Peter Stone, and Amy Zhang. Proto successor measure: Representing the space of all possible solutions of reinforcement learning, 2024. URL https://arxiv.org/abs/2411.19418

    arXiv 2024

  5. [5]

    Alemi, Ian Fischer, Joshua V

    Alexander A. Alemi, Ian Fischer, Joshua V. Dillon, and Kevin Murphy. Deep variational information bottleneck. In Proceedings of the 5th International Conference on Learning Representations (ICLR), 2017

    2017

  6. [6]

    Learning markov state abstractions for deep reinforcement learning

    Cameron Allen, Neev Parikh, Omer Gottesman, and George Konidaris. Learning markov state abstractions for deep reinforcement learning. Advances in Neural Information Processing Systems, 34: 0 8229--8241, 2021

    2021

  7. [7]

    A recipe for unbounded data augmentation in visual reinforcement learning

    Abdulaziz Almuzairee, Nicklas Hansen, and Henrik I Christensen. A recipe for unbounded data augmentation in visual reinforcement learning. Reinforcement Learning Journal, 1: 0 130--157, 2024

    2024

  8. [8]

    Unsupervised State Representation Learning in Atari

    Ankesh Anand, Evan Racah, Sherjil Ozair, Yoshua Bengio, Marc - Alexandre C \^ o t \' e , and R. Devon Hjelm. Unsupervised state representation learning in atari. CoRR, abs/1906.08226, 2019. URL http://arxiv.org/abs/1906.08226

    work page Pith review arXiv 1906

Show all 157 references
  1. [9]

    Hindsight experience replay

    Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, OpenAI Pieter Abbeel, and Wojciech Zaremba. Hindsight experience replay. Advances in neural information processing systems, 30, 2017

    2017

  2. [10]

    Self-supervised learning from images with a joint-embedding predictive architecture

    Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and P...

    2023

  3. [11]

    Neural machine translation by jointly learning to align and translate

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , 2015. URL http:/...

    2015 arXiv

  4. [12]

    Vicreg: Variance-invariance-covariance regularization for self-supervised learning

    Adrien Bardes, Jean Ponce, and Yann LeCun. Vicreg: Variance-invariance-covariance regularization for self-supervised learning. arXiv preprint arXiv:2105.04906, 2021

    2021 arXiv

  5. [13]

    Combining reconstruction and contrastive methods for multimodal representations in rl, 2024

    Philipp Becker, Sebastian Mossburger, Fabian Otto, and Gerhard Neumann. Combining reconstruction and contrastive methods for multimodal representations in rl, 2024. URL https://arxiv.org/abs/2302.05342

    2024 arXiv

  6. [14]

    The arcade learning environment: An evaluation platform for general agents

    Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47: 0 253--279, 2013

    2013

  7. [15]

    Courville, and Pascal Vincent

    Yoshua Bengio, Aaron C. Courville, and Pascal Vincent. Unsupervised feature learning and deep learning: A review and new perspectives. CoRR, abs/1206.5538, 2012. URL http://arxiv.org/abs/1206.5538

    2012 arXiv

  8. [16]

    Look where you look! saliency-guided q-networks for generalization in visual reinforcement learning

    David Bertoin, Adil Zouitine, Mehdi Zouitine, and Emmanuel Rachelson. Look where you look! saliency-guided q-networks for generalization in visual reinforcement learning. Advances in Neural Information Processing Systems, 35: 0 30693--30706, 2022

    2022

  9. [17]

    Riedmiller, and Klaus Obermayer

    Wendelin B \"o hmer, Jost Tobias Springenberg, Joschka Boedecker, Martin A. Riedmiller, and Klaus Obermayer. Autonomous learning of state representations for control: An emerging field aims to autonomously learn state representations for reinforcement learning agents from thei...

    2015

  10. [18]

    Unsupervised representation learning in deep reinforcement learning: A review

    Nicol \`o Botteghi, Mannes Poel, and Christoph Brune. Unsupervised representation learning in deep reinforcement learning: A review. arXiv preprint arXiv:2208.14226, 2022

    2022 arXiv

  11. [19]

    Barlowrl: Barlow twins for data-efficient reinforcement learning, 2023

    Omer Veysel Cagatan and Baris Akgun. Barlowrl: Barlow twins for data-efficient reinforcement learning, 2023

    2023

  12. [20]

    Scalable methods for computing state similarity in deterministic markov decision processes

    Pablo Samuel Castro. Scalable methods for computing state similarity in deterministic markov decision processes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp.\ 10069--10076, 2020

    2020

  13. [21]

    MIC o: Improved representations via sampling-based state similarity for markov decision processes

    Pablo Samuel Castro, Tyler Kastner, Prakash Panangaden, and Mark Rowland. MIC o: Improved representations via sampling-based state similarity for markov decision processes. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information P...

    2021

  14. [22]

    Learning action representations for reinforcement learning

    Yash Chandak, Georgios Theocharous, James Kostas, Scott Jordan, and Philip Thomas. Learning action representations for reinforcement learning. In International conference on machine learning, pp.\ 941--950. PMLR, 2019

    2019

  15. [23]

    Why do we need large batchsizes in contrastive learning? a gradient-bias perspective

    Changyou Chen, Jianyi Zhang, Yi Xu, Liqun Chen, Jiali Duan, Yiran Chen, Son Tran, Belinda Zeng, and Trishul Chilimbi. Why do we need large batchsizes in contrastive learning? a gradient-bias perspective. Advances in Neural Information Processing Systems, 35: 0 33860--33875, 2022

    2022

  16. [24]

    Focus-then-decide: Segmentation-assisted reinforcement learning

    Chao Chen, Jiacheng Xu, Weijian Liao, Hao Ding, Zongzhang Zhang, Yang Yu, and Rui Zhao. Focus-then-decide: Segmentation-assisted reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 11240--11248, 2024 a

    2024

  17. [25]

    Learning representations via a robust behavioral metric for deep reinforcement learning

    Jianda Chen and Sinno Pan. Learning representations via a robust behavioral metric for deep reinforcement learning. Advances in Neural Information Processing Systems, 35: 0 36654--36666, 2022

    2022

  18. [26]

    State chrono representation for enhancing generalization in reinforcement learning

    Jianda Chen, Wen zheng terence Ng, Zichen Chen, Sinno Jialin Pan, and Tianwei Zhang. State chrono representation for enhancing generalization in reinforcement learning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024 b . URL https://openre...

    2024

  19. [27]

    Vision-language models provide promptable representations for reinforcement learning, 2024 c

    William Chen, Oier Mees, Aviral Kumar, and Sergey Levine. Vision-language models provide promptable representations for reinforcement learning, 2024 c . URL https://arxiv.org/abs/2402.02651

    2024 arXiv

  20. [28]

    Exploring simple siamese representation learning

    Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 15750--15758, 2021

    2021

  21. [29]

    Mudalige, Katharina Muelling, and John M

    Yilun Chen, Chiyu Dong, Praveen Palanisamy, Priyantha W. Mudalige, Katharina Muelling, and John M. Dolan. Attention-based hierarchical deep reinforcement learning for lane change behaviors in autonomous driving. 2019 IEEE/RSJ International Conference on Intelligent Robots and ...

    2019

  22. [30]

    Provable benefit of multitask representation learning in reinforcement learning

    Yuan Cheng, Songtao Feng, Jing Yang, Hong Zhang, and Yingbin Liang. Provable benefit of multitask representation learning in reinforcement learning. Advances in Neural Information Processing Systems, 35: 0 31741--31754, 2022

    2022

  23. [31]

    Improving generalisation for temporal difference learning: The successor representation

    Peter Dayan. Improving generalisation for temporal difference learning: The successor representation. Neural Computation, 5 0 (4): 0 613--624, 1993

    1993

  24. [32]

    Integrating state representation learning into deep reinforcement learning

    Tim de Bruin, Jens Kober, Karl Tuyls, and Robert Babuška. Integrating state representation learning into deep reinforcement learning. IEEE Robotics and Automation Letters, 3 0 (3): 0 1394--1401, 2018. doi:10.1109/LRA.2018.2800101

    2018

  25. [33]

    The hidden pitfalls of the cosine similarity loss

    Andrew Draganov, Sharvaree Vadgama, and Erik J Bekkers. The hidden pitfalls of the cosine similarity loss. arXiv preprint arXiv:2406.16468, 2024

    2024 arXiv

  26. [34]

    Provably efficient rl with rich observations via latent state decoding

    Simon Du, Akshay Krishnamurthy, Nan Jiang, Alekh Agarwal, Miroslav Dudik, and John Langford. Provably efficient rl with rich observations via latent state decoding. In International Conference on Machine Learning, pp.\ 1665--1674. PMLR, 2019

    2019

  27. [35]

    Czarnecki, Siddhant M

    Yunshu Du, Wojciech M. Czarnecki, Siddhant M. Jayakumar, Mehrdad Farajtabar, Razvan Pascanu, and Balaji Lakshminarayanan. Adapting auxiliary losses using gradient similarity, 2020. URL https://arxiv.org/abs/1812.02224

    2020 arXiv

  28. [36]

    Multi-view disentanglement for reinforcement learning with multiple cameras

    Mhairi Dunion and Stefano V Albrecht. Multi-view disentanglement for reinforcement learning with multiple cameras. Reinforcement Learning Journal, 2: 0 498--515, 2024

    2024

  29. [37]

    Hanna, and Stefano V Albrecht

    Mhairi Dunion, Trevor McInroe, Kevin Sebastian Luck, Josiah P. Hanna, and Stefano V Albrecht. Temporal disentanglement of representations for improved generalisation in reinforcement learning. In The Eleventh International Conference on Learning Representations, 2023. URL http...

    2023

  30. [38]

    Conditional mutual information for disentangled representations in reinforcement learning

    Mhairi Dunion, Trevor McInroe, Kevin Sebastian Luck, Josiah Hanna, and Stefano Albrecht. Conditional mutual information for disentangled representations in reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024

    2024

  31. [39]

    Provable benefits of representational transfer in reinforcement learning

    Yonathan Efroni, Sham Kakade, Tengyang Xie Ma, and Lin F Yang. Provable benefits of representational transfer in reinforcement learning. In Advances in Neural Information Processing Systems, volume 35, pp.\ 29998--30010, 2022

    2022

  32. [40]

    Dribo: Robust deep reinforcement learning via multi-view information bottleneck

    Jiameng Fan and Wenchao Li. Dribo: Robust deep reinforcement learning via multi-view information bottleneck. In Proceedings of the 39th International Conference on Machine Learning (ICML). PMLR, 2022

    2022

  33. [41]

    Bellemare

    Jesse Farebrother, Joshua Greaves, Rishabh Agarwal, Charline Le Lan, Ross Goroshin, Pablo Samuel Castro, and Marc G. Bellemare. Proto-value networks: Scaling representation learning with auxiliary tasks, 2023. URL https://arxiv.org/abs/2304.12567

    2023 arXiv

  34. [42]

    Hyperbolic discounting and learning over multiple horizons

    William Fedus, Carles Gelada, Yoshua Bengio, Marc G Bellemare, and Hugo Larochelle. Hyperbolic discounting and learning over multiple horizons. arXiv preprint arXiv:1902.06865, 2019

    1902 arXiv

  35. [43]

    Metrics for finite markov decision processes

    Norman Ferns, Prakash Panangaden, and Doina Precup. Metrics for finite markov decision processes. CoRR, abs/1207.4114, 2012. URL http://arxiv.org/abs/1207.4114

    2012 arXiv

  36. [44]

    Riedmiller

    Carlos Florensa, Jonas Degrave, Nicolas Heess, Jost Tobias Springenberg, and Martin A. Riedmiller. Self-supervised learning of image embedding for continuous control. CoRR, abs/1901.00943, 2019. URL http://arxiv.org/abs/1901.00943

    1901 arXiv

  37. [45]

    For sale: State-action representation learning for deep reinforcement learning

    Scott Fujimoto, Wei-Di Chang, Edward Smith, Shixiang Shane Gu, Doina Precup, and David Meger. For sale: State-action representation learning for deep reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024

    2024

  38. [46]

    Towards general-purpose model-free reinforcement learning, 2025

    Scott Fujimoto, Pierluca D'Oro, Amy Zhang, Yuandong Tian, and Michael Rabbat. Towards general-purpose model-free reinforcement learning, 2025. URL https://arxiv.org/abs/2501.16142

    2025 arXiv

  39. [47]

    Rankme: Assessing the downstream performance of pretrained self-supervised representations by their rank

    Quentin Garrido, Randall Balestriero, Laurent Najman, and Yann Lecun. Rankme: Assessing the downstream performance of pretrained self-supervised representations by their rank. In International conference on machine learning, pp.\ 10929--10974. PMLR, 2023

    2023

  40. [48]

    Learning and leveraging world models in visual representation learning

    Quentin Garrido, Mahmoud Assran, Nicolas Ballas, Adrien Bardes, Laurent Najman, and Yann LeCun. Learning and leveraging world models in visual representation learning. arXiv preprint arXiv:2403.00504, 2024

    2024 arXiv

  41. [49]

    Diego Gomez, Michael Bowling, and Marlos C. Machado. Proper laplacian representation learning. In International Conference on Learning Representations (ICLR), 2024

    2024

  42. [50]

    Visualizing and understanding atari agents

    Samuel Greydanus, Anurag Koul, Jonathan Dodge, and Alan Fern. Visualizing and understanding atari agents. In International conference on machine learning, pp.\ 1792--1801. PMLR, 2018

    2018

  43. [51]

    Bootstrap your own latent-a new approach to self-supervised learning

    Jean-Bastien Grill, Florian Strub, Florent Altch \'e , Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neur...

    2020

  44. [52]

    Bootstrap latent-predictive representations for multitask reinforcement learning

    Zhaohan Daniel Guo, Bernardo \' A vila Pires, Bilal Piot, Jean - Bastien Grill, Florent Altch \' e , R \' e mi Munos, and Mohammad Gheshlaghi Azar. Bootstrap latent-predictive representations for multitask reinforcement learning. CoRR, abs/2004.14646, 2020. URL https://arxiv.o...

    2004 arXiv

  45. [53]

    Stabilizing deep q-learning with convnets and vision transformers under data augmentation

    Nicklas Hansen, Hao Su, and Xiaolong Wang. Stabilizing deep q-learning with convnets and vision transformers under data augmentation. Advances in neural information processing systems, 34: 0 3680--3693, 2021

    2021

  46. [54]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll \'a r, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 16000--16009, 2022

    2022

  47. [55]

    Rainbow: Combining improvements in deep reinforcement learning

    Matteo Hessel, Joseph Modayil, Hado Van Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Dan Horgan, Bilal Piot, Mohammad Azar, and David Silver. Rainbow: Combining improvements in deep reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence,...

    2018

  48. [56]

    Multi-task deep reinforcement learning with popart

    Matteo Hessel, Hubert Soyer, Lasse Espeholt, Wojciech Czarnecki, Simon Schmitt, and Hado Van Hasselt. Multi-task deep reinforcement learning with popart. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp.\ 3796--3803, 2019

    2019

  49. [57]

    Burgess, Xavier Glorot, Matthew M

    Irina Higgins, Lo \"i c Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew M. Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representa...

    2016

  50. [58]

    Darla: Improving zero-shot transfer in reinforcement learning

    Irina Higgins, Arka Pal, Andrei Rusu, Loic Matthey, Christopher Burgess, Alexander Pritzel, Matthew Botvinick, Charles Blundell, and Alexander Lerchner. Darla: Improving zero-shot transfer in reinforcement learning. In International Conference on Machine Learning, pp.\ 1480--1...

    2017

  51. [59]

    Learning deep representations by mutual information estimation and maximization

    R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. arXiv preprint arXiv:1808.06670, 2018

    2018 arXiv

  52. [60]

    Revisiting data augmentation in deep reinforcement learning, 2024

    Jianshu Hu, Yunpeng Jiang, and Paul Weng. Revisiting data augmentation in deep reinforcement learning, 2024

    2024

  53. [61]

    Spectrum random masking for generalization in image-based reinforcement learning

    Yangru Huang, Peixi Peng, Yifan Zhao, Guangyao Chen, and Yonghong Tian. Spectrum random masking for generalization in image-based reinforcement learning. Advances in Neural Information Processing Systems, 35: 0 20393--20406, 2022

    2022

  54. [62]

    Generalization in reinforcement learning with selective noise injection and information bottleneck

    Maximilian Igl, Kamil Ciosek, Yingzhen Li, Sebastian Tschiatschek, Cheng Zhang, Sam Devlin, and Katja Hofmann. Generalization in reinforcement learning with selective noise injection and information bottleneck. In Advances in Neural Information Processing Systems (NeurIPS), 2019

    2019

  55. [63]

    Zero-shot reinforcement learning via function encoders

    Tyler Ingebrand, Amy Zhang, and Ufuk Topcu. Zero-shot reinforcement learning via function encoders. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Confer...

    2024

  56. [64]

    Offline multitask representation learning for reinforcement learning

    Haque Ishfaq, Thanh Nguyen-Tang, Songtao Feng, Raman Arora, Mengdi Wang, Ming Yin, and Doina Precup. Offline multitask representation learning for reinforcement learning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openrev...

    2024

  57. [65]

    Principled offline rl in the presence of rich exogenous information

    Riashat Islam, Manan Tomar, Alex Lamb, Yonathan Efroni, Hongyu Zang, Aniket Didolkar, Dipendra Misra, Xin Li, Harm Van Seijen, Remi Tachet Des Combes, and John Langford. Principled offline rl in the presence of rich exogenous information. In Proceedings of the 40th Internation...

    2023

  58. [66]

    Representation learning in deep rl via discrete information bottleneck

    Riashat Islam, Hongyu Zang, Manan Tomar, Aniket Didolkar, et al. Representation learning in deep rl via discrete information bottleneck. In Proceedings of the 26th International Conference on Artificial Intelligence and Statistics (AISTATS). PMLR, 2023 b

    2023

  59. [67]

    Zero-shot reinforcement learning from low quality data

    Scott Jeen, Tom Bewley, and Jonathan Cullen. Zero-shot reinforcement learning from low quality data. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=79eWvkLjib

    2024

  60. [68]

    Contextual decision processes with low bellman rank are pac-learnable

    Nan Jiang, Akshay Krishnamurthy, Alekh Agarwal, John Langford, and Robert E Schapire. Contextual decision processes with low bellman rank are pac-learnable. In International Conference on Machine Learning, pp.\ 1704--1713. PMLR, 2017

    2017

  61. [69]

    Information-bottleneck-based behavior representation learning for multi-agent reinforcement learning

    Yue Jin, Shuangqing Wei, Jian Yuan, and Xudong Zhang. Information-bottleneck-based behavior representation learning for multi-agent reinforcement learning. arXiv preprint arXiv:2109.14188, 2021

    2021 arXiv

  62. [70]

    Riedmiller

    Rico Jonschkowski, Roland Hafner, Jonathan Scholz, and Martin A. Riedmiller. Pves: Position-velocity encoders for unsupervised learning of structured state representations. CoRR, abs/1705.09805, 2017. URL http://arxiv.org/abs/1705.09805

    2017 arXiv

  63. [71]

    Rishabh Kabra, Daniel Zoran, Goker Erdogan, Loic Matthey, Antonia Creswell, Matthew Botvinick, Alexander Lerchner, and Christopher P. Burgess. SIMON e: View-invariant, temporally-abstracted object representations via unsupervised video decomposition. In A. Beygelzimer, Y. Daup...

    2021

  64. [72]

    Scaling up multi-task robotic reinforcement learning

    Dmitry Kalashnikov, Jake Varley, Yevgen Chebotar, Benjamin Swanson, Rico Jonschkowski, Chelsea Finn, Sergey Levine, and Karol Hausman. Scaling up multi-task robotic reinforcement learning. In Conference on Robot Learning, pp.\ 557--575. PMLR, 2022

    2022

  65. [73]

    Terminal prediction as an auxiliary task for deep reinforcement learning

    Bilal Kartal, Pablo Hernandez-Leal, and Matthew E Taylor. Terminal prediction as an auxiliary task for deep reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, volume 15, pp.\ 38--44, 2019

    2019

  66. [74]

    Towards robust bisimulation metric learning

    Mete Kemertas and Tristan Aumentado - Armstrong. Towards robust bisimulation metric learning. CoRR, abs/2110.14096, 2021. URL https://arxiv.org/abs/2110.14096

    2021 arXiv

  67. [75]

    A unifying framework for action-conditional self-predictive reinforcement learning, 2024

    Khimya Khetarpal, Zhaohan Daniel Guo, Bernardo Avila Pires, Yunhao Tang, Clare Lyle, Mark Rowland, Nicolas Heess, Diana Borsa, Arthur Guez, and Will Dabney. A unifying framework for action-conditional self-predictive reinforcement learning, 2024. URL https://arxiv.org/abs/2406.02035

    2024 arXiv

  68. [76]

    Investigating pre-training objectives for generalization in vision-based reinforcement learning, 2024

    Donghu Kim, Hojoon Lee, Kyungmin Lee, Dongyoon Hwang, and Jaegul Choo. Investigating pre-training objectives for generalization in vision-based reinforcement learning, 2024. URL https://arxiv.org/abs/2406.06037

    2024 arXiv

  69. [77]

    Auto-encoding variational bayes

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In Proceedings of the 2nd International Conference on Learning Representations (ICLR), 2014. URL https://arxiv.org/abs/1312.6114

    2014 arXiv

  70. [78]

    Image augmentation is all you need: Regularizing deep reinforcement learning from pixels

    Ilya Kostrikov, Denis Yarats, and Rob Fergus. Image augmentation is all you need: Regularizing deep reinforcement learning from pixels. CoRR, abs/2004.13649, 2020. URL https://arxiv.org/abs/2004.13649

    2004 arXiv

  71. [79]

    Pac reinforcement learning with rich observations

    Akshay Krishnamurthy, Alekh Agarwal, and John Langford. Pac reinforcement learning with rich observations. Advances in Neural Information Processing Systems, 29, 2016

    2016

  72. [80]

    Guaranteed discovery of control-endogenous latent states with multi-step inverse models

    Alex Lamb, Riashat Islam, Yonathan Efroni, Aniket Rajiv Didolkar, Dipendra Misra, Dylan J Foster, Lekan P Molu, Rajan Chari, Akshay Krishnamurthy, and John Langford. Guaranteed discovery of control-endogenous latent states with multi-step inverse models. Transactions on Machin...

    2023

  73. [81]

    Reinforcement learning with augmented data

    Michael Laskin, Kimin Lee, Adam Stooke, Lerrel Pinto, Pieter Abbeel, and Aravind Srinivas. Reinforcement learning with augmented data. CoRR, abs/2004.14990, 2020. URL https://arxiv.org/abs/2004.14990

    2004 arXiv

  74. [82]

    Metrics and continuity in reinforcement learning

    Charline Le Lan, Marc G Bellemare, and Pablo Samuel Castro. Metrics and continuity in reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.\ 8261--8269, 2021

    2021

  75. [83]

    A path towards autonomous machine intelligence version 0.9

    Yann LeCun. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review, 62 0 (1): 0 1--62, 2022

    2022

  76. [85]

    Unsupervised state representation learning with robotic priors: a robustness benchmark

    Timoth \' e e Lesort, Mathieu Seurin, Xinrui Li, Natalia D \' az Rodr \' guez, and David Filliat. Unsupervised state representation learning with robotic priors: a robustness benchmark. CoRR, abs/1709.05185, 2017 b . URL http://arxiv.org/abs/1709.05185

    2017 arXiv

  77. [86]

    State representation learning for control: An overview

    Timoth \' e e Lesort, Natalia D \' az Rodr \' guez, Jean - Fran c ois Goudou, and David Filliat. State representation learning for control: An overview. CoRR, abs/1802.04181, 2018. URL http://arxiv.org/abs/1802.04181

    2018 arXiv

  78. [87]

    Normalization enhances generalization in visual reinforcement learning

    Lu Li, Jiafei Lyu, Guozheng Ma, Zilin Wang, Zhenjie Yang, Xiu Li, and Zhiheng Li. Normalization enhances generalization in visual reinforcement learning. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems, AAMAS '24, pp.\ 1137–1146,...

    2024

  79. [88]

    Provable general function class representation learning in multitask bandits and mdp

    Rui Lu, Andrew Zhao, Simon S Du, and Gao Huang. Provable general function class representation learning in multitask bandits and mdp. Advances in Neural Information Processing Systems, 35: 0 11507--11519, 2022

    2022

  80. [89]

    On the effect of auxiliary tasks on representation dynamics

    Clare Lyle, Mark Rowland, Georg Ostrovski, and Will Dabney. On the effect of auxiliary tasks on representation dynamics. CoRR, abs/2102.13089, 2021. URL https://arxiv.org/abs/2102.13089

    2021 arXiv

  81. [90]

    A comprehensive survey of data augmentation in visual reinforcement learning, 2022

    Guozheng Ma, Zhen Wang, Zhecheng Yuan, Xueqian Wang, Bo Yuan, and Dacheng Tao. A comprehensive survey of data augmentation in visual reinforcement learning, 2022. URL https://arxiv.org/abs/2210.04561

    2022 arXiv

  82. [91]

    Revisiting plasticity in visual reinforcement learning: Data, modules and training stages

    Guozheng Ma, Lu Li, Sen Zhang, Zixuan Liu, Zhen Wang, Yixin Chen, Li Shen, Xueqian Wang, and Dacheng Tao. Revisiting plasticity in visual reinforcement learning: Data, modules and training stages. In The Twelfth International Conference on Learning Representations, 2024. URL h...

    2024

  83. [92]

    Vip: Towards universal visual reward and representation via value-implicit pre-training, 2023

    Yecheng Jason Ma, Shagun Sodhani, Dinesh Jayaraman, Osbert Bastani, Vikash Kumar, and Amy Zhang. Vip: Towards universal visual reward and representation via value-implicit pre-training, 2023. URL https://arxiv.org/abs/2210.00030

    2023 arXiv

  84. [93]

    Where are we in the search for an artificial visual cortex for embodied intelligence? Advances in Neural Information Processing Systems, 36: 0 655--677, 2023

    Arjun Majumdar, Karmesh Yadav, Sergio Arnaud, Jason Ma, Claire Chen, Sneha Silwal, Aryan Jain, Vincent-Pierre Berges, Tingfan Wu, Jay Vakil, et al. Where are we in the search for an artificial visual cortex for embodied intelligence? Advances in Neural Information Processing S...

    2023

  85. [94]

    Deep reinforcement and infomax learning

    Bogdan Mazoure, Remi Tachet des Combes, Thang Long Doan, Philip Bachman, and R Devon Hjelm. Deep reinforcement and infomax learning. Advances in Neural Information Processing Systems, 33: 0 3686--3698, 2020

    2020

  86. [95]

    Multi-horizon representations with hierarchical forward models for reinforcement learning

    Trevor McInroe, Lukas Sch \"a fer, and Stefano V Albrecht. Multi-horizon representations with hierarchical forward models for reinforcement learning. Transactions on Machine Learning Research

  87. [96]

    Towards principled representation learning from videos for reinforcement learning, 2024

    Dipendra Misra, Akanksha Saran, Tengyang Xie, Alex Lamb, and John Langford. Towards principled representation learning from videos for reinforcement learning, 2024. URL https://arxiv.org/abs/2403.13765

    2024 arXiv

  88. [97]

    Riedmiller

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin A. Riedmiller. Playing atari with deep reinforcement learning. CoRR, abs/1312.5602, 2013. URL http://arxiv.org/abs/1312.5602

    2013 arXiv

  89. [98]

    Alex Mott, Daniel Zoran, Mike Chrzanowski, Daan Wierstra, and Danilo J. Rezende. Towards interpretable reinforcement learning using attention augmented agents. CoRR, abs/1906.02500, 2019. URL http://arxiv.org/abs/1906.02500

    1906 arXiv

  90. [99]

    R3m: A universal visual representation for robot manipulation, 2022

    Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3m: A universal visual representation for robot manipulation, 2022

    2022

  91. [100]

    Bridging state and history representations: Understanding self-predictive rl, 2024

    Tianwei Ni, Benjamin Eysenbach, Erfan Seyedsalehi, Michel Ma, Clement Gehring, Aditya Mahajan, and Pierre-Luc Bacon. Bridging state and history representations: Understanding self-predictive rl, 2024

    2024

  92. [101]

    Foundation policies with H ilbert representations

    Seohong Park, Tobias Kreiman, and Sergey Levine. Foundation policies with H ilbert representations. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Confer...

    2024

  93. [102]

    Foundation policies with hilbert representations, 2024 b

    Seohong Park, Tobias Kreiman, and Sergey Levine. Foundation policies with hilbert representations, 2024 b . URL https://arxiv.org/abs/2402.15567

    2024 arXiv

  94. [103]

    What makes useful auxiliary tasks in reinforcement learning: investigating the effect of the target policy, 2022

    Banafsheh Rafiee, Jun Jin, Jun Luo, and Adam White. What makes useful auxiliary tasks in reinforcement learning: investigating the effect of the target policy, 2022. URL https://arxiv.org/abs/2204.00565

    2022 arXiv

  95. [104]

    Natural language-based state representation in deep reinforcement learning

    Md Masudur Rahman and Yexiang Xue. Natural language-based state representation in deep reinforcement learning. In Findings of the Association for Computational Linguistics: NAACL 2024, pp.\ 1310--1319, 2024

    2024

  96. [105]

    Automatic data augmentation for generalization in reinforcement learning

    Roberta Raileanu, Maxwell Goldstein, Denis Yarats, Ilya Kostrikov, and Rob Fergus. Automatic data augmentation for generalization in reinforcement learning. In Neural Information Processing Systems, 2021. URL https://api.semanticscholar.org/CorpusID:221094237

    2021

  97. [106]

    Gonzalez, Dale Schuurmans, and Bo Dai

    Tongzheng Ren, Tianjun Zhang, Lisa Lee, Joseph E. Gonzalez, Dale Schuurmans, and Bo Dai. Spectral decomposition representation for reinforcement learning. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=FBMLeaXpZN

    2023

  98. [107]

    Continuous MDP homomorphisms and homomorphic policy gradient

    Sahand Rezaei-Shoshtari, Rosie Zhao, Prakash Panangaden, David Meger, and Doina Precup. Continuous MDP homomorphisms and homomorphic policy gradient. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, ...

    2022

  99. [108]

    Learning action-based representations using invariance, 2024

    Max Rudolph, Caleb Chuck, Kevin Black, Misha Lvovsky, Scott Niekum, and Amy Zhang. Learning action-based representations using invariance, 2024. URL https://arxiv.org/abs/2403.16369

    2024 arXiv

  100. [109]

    Learning to act without actions

    Dominik Schmidt and Minqi Jiang. Learning to act without actions. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=rvUq3cxpDF

    2024

  101. [110]

    The surprising ineffectiveness of pre-trained visual representations for model-based reinforcement learning

    Moritz Schneider, Robert Krug, Narunas Vaskevicius, Luigi Palmieri, and Joschka Boedecker. The surprising ineffectiveness of pre-trained visual representations for model-based reinforcement learning. In The Thirty-eighth Annual Conference on Neural Information Processing Syste...

    2024

  102. [111]

    Data-efficient reinforcement learning with self-predictive representations

    Max Schwarzer, Ankesh Anand, Rishab Goel, R Devon Hjelm, Aaron Courville, and Philip Bachman. Data-efficient reinforcement learning with self-predictive representations. arXiv preprint arXiv:2007.05929, 2020

    2007 arXiv

  103. [112]

    Devon Hjelm, Philip Bachman, and Aaron C

    Max Schwarzer, Nitarshan Rajkumar, Michael Noukhovitch, Ankesh Anand, Laurent Charlin, R. Devon Hjelm, Philip Bachman, and Aaron C. Courville. Pretraining representations for data-efficient reinforcement learning. CoRR, abs/2106.04799, 2021. URL https://arxiv.org/abs/2106.04799

    2021 arXiv

  104. [113]

    Time-contrastive networks: Self-supervised learning from video

    Pierre Sermanet, Corey Lynch, Yevgen Chebotar, Jasmine Hsu, Eric Jang, Stefan Schaal, Sergey Levine, and Google Brain. Time-contrastive networks: Self-supervised learning from video. In 2018 IEEE international conference on robotics and automation (ICRA), pp.\ 1134--1141. IEEE, 2018

    2018

  105. [114]

    Diffusion spectral representation for reinforcement learning

    Dmitry Shribak, Chen-Xiao Gao, Yitong Li, Chenjun Xiao, and Bo Dai. Diffusion spectral representation for reinforcement learning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=C3tEX45hJX

    2024

  106. [115]

    Multi-task reinforcement learning with context-based representations

    Shagun Sodhani, Amy Zhang, and Joelle Pineau. Multi-task reinforcement learning with context-based representations. In Proceedings of the 38th International Conference on Machine Learning, pp.\ 9767--9779. PMLR, 2021 a

    2021

  107. [116]

    Multi-task reinforcement learning with context-based representations

    Shagun Sodhani, Amy Zhang, and Joelle Pineau. Multi-task reinforcement learning with context-based representations. CoRR, abs/2102.06177, 2021 b . URL https://arxiv.org/abs/2102.06177

    2021 arXiv

  108. [117]

    CURL: contrastive unsupervised representations for reinforcement learning

    Aravind Srinivas, Michael Laskin, and Pieter Abbeel. CURL: contrastive unsupervised representations for reinforcement learning. CoRR, abs/2004.04136, 2020. URL https://arxiv.org/abs/2004.04136

    2004 arXiv

  109. [118]

    Decoupling representation learning from reinforcement learning

    Adam Stooke, Kimin Lee, Pieter Abbeel, and Michael Laskin. Decoupling representation learning from reinforcement learning. CoRR, abs/2009.08319, 2020. URL https://arxiv.org/abs/2009.08319

    2009 arXiv

  110. [119]

    Neuroevolution of self-interpretable agents

    Yujin Tang, Duong Nguyen, and David Ha. Neuroevolution of self-interpretable agents. CoRR, abs/2003.08165, 2020. URL https://arxiv.org/abs/2003.08165

    2003 arXiv

  111. [120]

    Understanding self-predictive learning for reinforcement learning

    Yunhao Tang, Zhaohan Daniel Guo, Pierre Harvey Richemond, Bernardo Avila Pires, Yash Chandak, R \'e mi Munos, Mark Rowland, Mohammad Gheshlaghi Azar, Charline Le Lan, Clare Lyle, et al. Understanding self-predictive learning for reinforcement learning. In International Confere...

    2023

  112. [121]

    Bounding performance loss in approximate mdp homomorphisms

    Jonathan Taylor, Doina Precup, and Prakash Panangaden. Bounding performance loss in approximate mdp homomorphisms. pp.\ 1649--1656, 01 2008

    2008

  113. [122]

    Disentangling the independently controllable factors of variation by interacting with the world

    Valentin Thomas, Emmanuel Bengio, William Fedus, Jules Pondard, Philippe Beaudoin, Hugo Larochelle, Joelle Pineau, Doina Precup, and Yoshua Bengio. Disentangling the independently controllable factors of variation by interacting with the world. arXiv preprint arXiv:1802.09484, 2018

    2018 arXiv

  114. [123]

    The information bottleneck method

    Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. arXiv preprint arXiv:physics/0004057, 2000

    2000 arXiv

  115. [124]

    Mishra, Amy Zhang, and Matthew E

    Manan Tomar, Utkarsh A. Mishra, Amy Zhang, and Matthew E. Taylor. Learning representations for pixel-based control: What matters and why? CoRR, abs/2111.07775, 2021. URL https://arxiv.org/abs/2111.07775

    2021 arXiv

  116. [125]

    Learning one representation to optimize all rewards

    Ahmed Touati and Yann Ollivier. Learning one representation to optimize all rewards. CoRR, abs/2103.07945, 2021. URL https://arxiv.org/abs/2103.07945

    2021 arXiv

  117. [126]

    Does zero-shot reinforcement learning exist?, 2023

    Ahmed Touati, Jérémy Rapin, and Yann Ollivier. Does zero-shot reinforcement learning exist?, 2023. URL https://arxiv.org/abs/2209.14935

    2023 arXiv

  118. [127]

    Evaluating learned state representations for atari

    Adam Tupper and Kourosh Neshatian. Evaluating learned state representations for atari. In 2020 35th International Conference on Image and Vision Computing New Zealand (IVCNZ), pp.\ 1--6, 2020. doi:10.1109/IVCNZ51579.2020.9290609

    2020

  119. [129]

    Representation learning with contrastive predictive coding

    A \" a ron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. CoRR, abs/1807.03748, 2018 b . URL http://arxiv.org/abs/1807.03748

    2018 arXiv

  120. [130]

    Optimal transport: Old and new

    C \'e dric Villani. Optimal transport: Old and new. 2008. URL https://api.semanticscholar.org/CorpusID:118347220

    2008

  121. [131]

    When does self-prediction help? understanding auxiliary tasks in reinforcement learning

    Claas A Voelcker, Tyler Kastner, Igor Gilitschenski, and Amir-massoud Farahmand. When does self-prediction help? understanding auxiliary tasks in reinforcement learning. Reinforcement Learning Journal, 4: 0 1567--1597, 2024

    2024

  122. [132]

    LLM -empowered state representation for reinforcement learning

    Boyuan Wang, Yun Qu, Yuhang Jiang, Jianzhun Shao, Chang Liu, Wenming Yang, and Xiangyang Ji. LLM -empowered state representation for reinforcement learning. In Forty-first International Conference on Machine Learning, 2024 a . URL https://openreview.net/forum?id=xJMZbdiQnf

    2024

  123. [133]

    Investigating the properties of neural network representations in reinforcement learning

    Han Wang, Erfan Miahi, Martha White, Marlos C Machado, Zaheer Abbas, Raksha Kumaraswamy, Vincent Liu, and Adam White. Investigating the properties of neural network representations in reinforcement learning. Artificial Intelligence, pp.\ 104100, 2024 b

    2024

  124. [134]

    Reachability-aware laplacian representation in reinforcement learning

    Kaixin Wang, Kuangqi Zhou, Jiashi Feng, Bryan Hooi, and Xinchao Wang. Reachability-aware laplacian representation in reinforcement learning. In Proceedings of the 40th International Conference on Machine Learning (ICML). PMLR, 2023

    2023

  125. [135]

    Contrastive learning with consistent representations

    Zihu Wang, Yu Wang, Zhuotong Chen, Hanbin Hu, and Peng Li. Contrastive learning with consistent representations. Transactions on Machine Learning Research, 2024 c . ISSN 2835-8856. URL https://openreview.net/forum?id=gKeSI8w63Z

    2024

  126. [136]

    Visual rationalizations in deep reinforcement learning for atari games

    Laurens Weitkamp, Elise van der Pol, and Zeynep Akata. Visual rationalizations in deep reinforcement learning for atari games. In Artificial Intelligence: 30th Benelux Conference, BNAIC 2018,‘s-Hertogenbosch, The Netherlands, November 8--9, 2018, Revised Selected Papers 30, pp...

    2018

  127. [137]

    Williams

    Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn., 8 0 (3–4): 0 229–256, may 1992. ISSN 0885-6125. doi:10.1007/BF00992696. URL https://doi.org/10.1007/BF00992696

    1992 doi

  128. [138]

    Self-supervised attention-aware reinforcement learning

    Haiping Wu, Khimya Khetarpal, and Doina Precup. Self-supervised attention-aware reinforcement learning. In AAAI Conference on Artificial Intelligence, 2021. URL https://api.semanticscholar.org/CorpusID:235349100

    2021

  129. [139]

    The laplacian in rl: Learning representations with efficient approximations

    Yifan Wu, George Tucker, and Ofir Nachum. The laplacian in rl: Learning representations with efficient approximations. In International Conference on Learning Representations (ICLR), 2019

    2019

  130. [140]

    Masked visual pre-training for motor control

    Tete Xiao, Ilija Radosavovic, Trevor Darrell, and Jitendra Malik. Masked visual pre-training for motor control. arXiv preprint arXiv:2203.06173, 2022

    2022 arXiv

  131. [141]

    Representation matters: Offline pretraining for sequential decision making

    Mengjiao Yang and Ofir Nachum. Representation matters: Offline pretraining for sequential decision making. CoRR, abs/2102.05815, 2021. URL https://arxiv.org/abs/2102.05815

    2021 arXiv

  132. [142]

    Learning task-relevant representations for generalization via characteristic functions of reward sequence distributions

    Rui Yang, Jie Wang, Zijie Geng, Mingxuan Ye, Shuiwang Ji, Bin Li, and Feng Wu. Learning task-relevant representations for generalization via characteristic functions of reward sequence distributions. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and D...

    2022

  133. [143]

    Mastering visual continuous control: Improved data-augmented reinforcement learning

    Denis Yarats, Rob Fergus, Alessandro Lazaric, and Lerrel Pinto. Mastering visual continuous control: Improved data-augmented reinforcement learning. CoRR, abs/2107.09645, 2021. URL https://arxiv.org/abs/2107.09645

    2021 arXiv

  134. [144]

    Multimodal information bottleneck for deep reinforcement learning with multiple sensors

    Bang You and Huaping Liu. Multimodal information bottleneck for deep reinforcement learning with multiple sensors. Neural Networks, 2024. Accepted for publication

    2024

  135. [145]

    Playvirtual: Augmenting cycle-consistent virtual trajectories for reinforcement learning

    Tao Yu, Cuiling Lan, Wenjun Zeng, Mingxiao Feng, Zhizheng Zhang, and Zhibo Chen. Playvirtual: Augmenting cycle-consistent virtual trajectories for reinforcement learning. Advances in Neural Information Processing Systems, 34: 0 5276--5289, 2021

    2021

  136. [146]

    Mask-based latent reconstruction for reinforcement learning

    Tao Yu, Zhizheng Zhang, Cuiling Lan, Yan Lu, and Zhibo Chen. Mask-based latent reconstruction for reinforcement learning. Advances in Neural Information Processing Systems, 35: 0 25117--25131, 2022

    2022

  137. [147]

    Don’t touch what matters: Task-aware lipschitz data augmentation for visual reinforcement learning

    Zhecheng Yuan, Guozheng Ma, Yao Mu, Bo Xia, Bo Yuan, Xueqian Wang, Ping Luo, and Huazhe Xu. Don’t touch what matters: Task-aware lipschitz data augmentation for visual reinforcement learning. In Lud De Raedt (ed.), Proceedings of the Thirty-First International Joint Conference...

    2022 doi

  138. [148]

    Pre-trained image encoder for generalizable visual reinforcement learning

    Zhecheng Yuan, Zhengrong Xue, Bo Yuan, Xueqian Wang, Yi Wu, Yang Gao, and Huazhe Xu. Pre-trained image encoder for generalizable visual reinforcement learning. Advances in Neural Information Processing Systems, 35: 0 13022--13037, 2022 b

    2022

  139. [149]

    Barlow twins: Self-supervised learning via redundancy reduction

    Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and St \'e phane Deny. Barlow twins: Self-supervised learning via redundancy reduction. In International conference on machine learning, pp.\ 12310--12320. PMLR, 2021

    2021

  140. [150]

    Learning invariant representations for reinforcement learning without reconstruction

    Amy Zhang, Rowan Thomas McAllister, Roberto Calandra, Yarin Gal, and Sergey Levine. Learning invariant representations for reinforcement learning without reconstruction. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=-2FCwDKRREu

    2021

  141. [151]

    A new embedding quality assessment method for manifold learning

    Peng Zhang, Yuanyuan Ren, and Bo Zhang. A new embedding quality assessment method for manifold learning. Neurocomputing, 97: 0 251--266, 2012

    2012

  142. [152]

    Light-weight probing of unsupervised representations for reinforcement learning

    Wancong Zhang, Anthony GX-Chen, Vlad Sobal, Yann LeCun, and Nicolas Carion. Light-weight probing of unsupervised representations for reinforcement learning. Reinforcement Learning Journal, 4: 0 1924--1949, 2024 a

    1924

  143. [153]

    Light-weight probing of unsupervised representations for reinforcement learning, 2024 b

    Wancong Zhang, Anthony GX-Chen, Vlad Sobal, Yann LeCun, and Nicolas Carion. Light-weight probing of unsupervised representations for reinforcement learning, 2024 b . URL https://arxiv.org/abs/2208.12345

    2024 arXiv

  144. [154]

    What makes representation learning from videos hard for control

    Tony Zhao, Siddharth Karamcheti, Kollar Thomas, and Chelsea Finn. What makes representation learning from videos hard for control. In RSS Workshop on Scaling Robot Learning, 2022

    2022

  145. [155]

    Contrastive difference predictive coding, 2024 a

    Chongyi Zheng, Ruslan Salakhutdinov, and Benjamin Eysenbach. Contrastive difference predictive coding, 2024 a

    2024

  146. [156]

    Taco: Temporal latent action-driven contrastive loss for visual reinforcement learning

    Ruijie Zheng, Xiyao Wang, Yanchao Sun, Shuang Ma, Jieyu Zhao, Huazhe Xu, Hal Daum \'e III, and Furong Huang. Taco: Temporal latent action-driven contrastive loss for visual reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024 b

    2024

  147. [157]

    Lewis, and Satinder Singh

    Zeyu Zheng, Vivek Veeriah, Risto Vuorio, Richard L. Lewis, and Satinder Singh. Learning state representations from random deep action-conditional predictions. CoRR, abs/2102.04897, 2021. URL https://arxiv.org/abs/2102.04897

    2021 arXiv

  148. [158]

    Learning robust representation for reinforcement learning with distractions by reward sequence prediction

    Qi Zhou, Jie Wang, Qiyuan Liu, Yufei Kuang, Wengang Zhou, and Houqiang Li. Learning robust representation for reinforcement learning with distractions by reward sequence prediction. In Uncertainty in Artificial Intelligence, pp.\ 2551--2562. PMLR, 2023

    2023

  149. [159]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.