REVIEW 5 cited by
C-Learning: Learning to Achieve Goals via Recursive Classification
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We study the problem of predicting and controlling the future state distribution of an autonomous agent. This problem, which can be viewed as a reframing of goal-conditioned reinforcement learning (RL), is centered around learning a conditional probability density function over future states. Instead of directly estimating this density function, we indirectly estimate this density function by training a classifier to predict whether an observation comes from the future. Via Bayes' rule, predictions from our classifier can be transformed into predictions over future states. Importantly, an off-policy variant of our algorithm allows us to predict the future state distribution of a new policy, without collecting new experience. This variant allows us to optimize functionals of a policy's future state distribution, such as the density of reaching a particular goal state. While conceptually similar to Q-learning, our work lays a principled foundation for goal-conditioned RL as density estimation, providing justification for goal-conditioned methods used in prior work. This foundation makes hypotheses about Q-learning, including the optimal goal-sampling ratio, which we confirm experimentally. Moreover, our proposed method is competitive with prior goal-conditioned RL methods.
Forward citations
Cited by 5 Pith papers
-
Equivariant Goal Conditioned Contrastive Reinforcement Learning
Equivariant Contrastive RL imposes C8 rotation symmetry on the critic and actor, improving sample efficiency and goal generalization in simulated manipulation.
-
Efficient Skill Discovery via Regret-Aware Optimization
A regret-aware skill discovery algorithm, RSD, improves sample efficiency and zero-shot goal-reaching in high-dimensional continuous control by focusing exploration on unmastered skills.
-
Reachability Weighted Offline Goal-conditioned Resampling
RWS trains a positive-unlabeled reachability classifier on goal-conditioned Q-values and uses it to re-weight goal sampling, improving offline goal-conditioned RL performance on robotic manipulation benchmarks.
-
GCHR : Goal-Conditioned Hindsight Regularization for Sample-Efficient Reinforcement Learning
GCHR combines hindsight self-imitation with a hindsight-goal KL prior in off-policy RL, reporting large sample-efficiency gains on Fetch and Shadow Hand tasks.
-
Closing the Gap between TD Learning and Supervised Learning with $Q$-Conditioned Maximization
GCReinSL adds Q-conditioned maximization to supervised offline RL, using normalizing flows to estimate goal-reaching probabilities and expectile regression to condition actions on the best in-distribution value, impro...
Discussion (0). Sign in to comment.