REVIEW 4 cited by
Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In many sequential decision-making tasks, the agent is not able to model the full complexity of the world, which consists of multitudes of relevant and irrelevant information. For example, a person walking along a city street who tries to model all aspects of the world would quickly be overwhelmed by a multitude of shops, cars, and people moving in and out of view, each following their own complex and inscrutable dynamics. Is it possible to turn the agent's firehose of sensory information into a minimal latent state that is both necessary and sufficient for an agent to successfully act in the world? We formulate this question concretely, and propose the Agent Control-Endogenous State Discovery algorithm (AC-State), which has theoretical guarantees and is practically demonstrated to discover the minimal control-endogenous latent state which contains all of the information necessary for controlling the agent, while fully discarding all irrelevant information. This algorithm consists of a multi-step inverse model (predicting actions from distant observations) with an information bottleneck. AC-State enables localization, exploration, and navigation without reward or demonstrations. We demonstrate the discovery of the control-endogenous latent state in three domains: localizing a robot arm with distractions (e.g., changing lighting conditions and background), exploring a maze alongside other agents, and navigating in the Matterport house simulator.
Forward citations
Cited by 4 Pith papers
-
Minimal Markovization via Stable Quotients in Holonomy-Cover Decision Processes
In holonomy-cover POMDPs, the observation plus its stable quotient class is the minimal exact finite Markov state, recoverable under diagnostics and usable by standard RL.
-
TaskSense: Focusing on What Matters in World Models
TaskSense filters visual observations with latent-conditioned stochastic spatial attention before encoding, improving world-model control under visual distractions relative to DreamerV3.
-
How Well Do Latent World Models Understand Partially Observable Safety Constraints?
RGB-only latent safety filters fail to prevent overheating because safety-relevant temperature is unobservable in the image; a mutual-information metric predicts this failure, and multimodal supervision during world-m...
-
Latent Action Learning Requires Supervision in the Presence of Distractors
Latent action models need at least a small amount of action supervision to learn useful actions when observations contain distractors, as shown on the Distracting Control Suite.
Discussion (0). Continue with ORCID to comment.