REVIEW 8 cited by
Loss is its own Reward: Self-Supervision for Reinforcement Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Reinforcement learning optimizes policies for expected cumulative reward. Need the supervision be so narrow? Reward is delayed and sparse for many tasks, making it a difficult and impoverished signal for end-to-end optimization. To augment reward, we consider a range of self-supervised tasks that incorporate states, actions, and successors to provide auxiliary losses. These losses offer ubiquitous and instantaneous supervision for representation learning even in the absence of reward. While current results show that learning from reward alone is feasible, pure reinforcement learning methods are constrained by computational and data efficiency issues that can be remedied by auxiliary losses. Self-supervised pre-training and joint optimization improve the data efficiency and policy returns of end-to-end reinforcement learning.
Forward citations
Cited by 8 Pith papers
-
TaskSense: Focusing on What Matters in World Models
TaskSense filters visual observations with latent-conditioned stochastic spatial attention before encoding, improving world-model control under visual distractions relative to DreamerV3.
-
Exploration by Running Away from the Past
RAMP maximizes the divergence between an agent's current and past state distributions, a proxy for Shannon entropy, and outperforms several prior exploration methods on continuous-control benchmarks.
-
Self-Supervised Goal-Reaching Results in Multi-Agent Cooperation and Exploration
Self-supervised multi-agent goal-reaching, where each agent independently learns a contrastive critic of its own observations, achieves cooperation and exploration in sparse-reward MARL tasks where standard baselines fail.
-
Supervise Thyself: Examining Self-Supervised Representations in Interactive Environments
Empirical comparison finds that self-supervised representations vary in capturing agent state and generalizing to new levels or textures depending on environment visuals and dynamics.
-
Mask-based Predictive Representations for Reinforcement Learning
Mask-based predictive representations (MPR) as an auxiliary self-supervised task improve sample efficiency of vision-based RL over prior SOTA on continuous and discrete control benchmarks.
-
Effective Reward Specification in Deep Reinforcement Learning
A thesis presenting four methods (ASAF, TeamReg, CoachReg, constrained RL, goal-conditioned GFlowNets) that improve reward specification for deep RL through demonstrations, policy regularization, behavior constraints,...
-
A survey on intrinsic motivation in reinforcement learning
A survey that classifies intrinsic motivation methods in deep RL as knowledge acquisition or skill learning and proposes their unification through information compression.
-
Reward Prediction Error Prioritisation in Experience Replay: The RPE-PER Method
RPE-PER prioritises replay buffer samples by the absolute error between a learned reward model and the actual reward, and reports improved continuous-control RL performance in MuJoCo benchmarks.
Discussion (0). Continue with ORCID to comment.