REVIEW 6 cited by
Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Making
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Temporal distances lie at the heart of many algorithms for planning, control, and reinforcement learning that involve reaching goals, allowing one to estimate the transit time between two states. However, prior attempts to define such temporal distances in stochastic settings have been stymied by an important limitation: these prior approaches do not satisfy the triangle inequality. This is not merely a definitional concern, but translates to an inability to generalize and find shortest paths. In this paper, we build on prior work in contrastive learning and quasimetrics to show how successor features learned by contrastive learning (after a change of variables) form a temporal distance that does satisfy the triangle inequality, even in stochastic settings. Importantly, this temporal distance is computationally efficient to estimate, even in high-dimensional and stochastic settings. Experiments in controlled settings and benchmark suites demonstrate that an RL algorithm based on these new temporal distances exhibits combinatorial generalization (i.e., "stitching") and can sometimes learn more quickly than prior methods, including those based on quasimetrics.
Forward citations
Cited by 6 Pith papers
-
Understanding Behavioral Metric Learning: A Large-Scale Study on Distracting Reinforcement Learning Environments
Across noisy DeepMind Control tasks, explicit bisimulation-metric losses add little denoising benefit beyond plain self-prediction and feature normalization, which dominate performance.
-
NFTR: From Provable Mode-Averaging to Geodesic Subgoal Selection in Offline Goal-Conditioned RL
Normalizing-flow subgoal policies plus triangle-slack reweighting provably avoid Gaussian mode-averaging and filter lucky transitions in offline hierarchical GCRL.
-
Good Rankers, Bad Objectives: Bilinear Contrastive Critics under Expressive Policy Search
Bilinear contrastive critics remain good compatibility rankers but are unsafe to maximize for action selection; cosine bounding does not fix value decalibration, while Bellman TD-Q does.
-
CLARIFY: Contrastive Preference Reinforcement Learning for Untangling Ambiguous Queries
CLARIFY uses contrastive learning on preference data to embed trajectories, then rejection-samples queries that humans can distinguish clearly, improving offline preference-based RL.
-
Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Following
A temporal alignment auxiliary loss on goal and language representations improves zero-shot compositional generalization in robot instruction following.
-
Closing the Gap between TD Learning and Supervised Learning with $Q$-Conditioned Maximization
GCReinSL adds Q-conditioned maximization to supervised offline RL, using normalizing flows to estimate goal-reaching probabilities and expectile regression to condition actions on the best in-distribution value, impro...
Discussion (0). Continue with ORCID to comment.