Reinforcement Learning through Active Inference
read the original abstract
The central tenet of reinforcement learning (RL) is that agents seek to maximize the sum of cumulative rewards. In contrast, active inference, an emerging framework within cognitive and computational neuroscience, proposes that agents act to maximize the evidence for a biased generative model. Here, we illustrate how ideas from active inference can augment traditional RL approaches by (i) furnishing an inherent balance of exploration and exploitation, and (ii) providing a more flexible conceptualization of reward. Inspired by active inference, we develop and implement a novel objective for decision making, which we term the free energy of the expected future. We demonstrate that the resulting algorithm successfully balances exploration and exploitation, simultaneously achieving robust performance on several challenging RL benchmarks with sparse, well-shaped, and no rewards.
This paper has not been read by Pith yet.
Forward citations
Cited by 4 Pith papers
-
Prediction and Empowerment: A Theory of Agency through Bridge Interfaces
In deterministic partially observable worlds, perfect prediction requires either identifying the relevant hidden quotient or achieving overwrite control, while high empowerment alone is insufficient.
-
Multi-Agent Digital Twins for Strategic Decision-Making using Active Inference
Multi-agent active inference digital twins with contextual inference and streaming ML integration, illustrated via a Cournot competition socio-economic example.
-
Exploratory Experience Shapes the Geometry of Predictive Representations
Exploratory experience produces more spatially organized and transition-preserving predictive representations in maze navigation for both agents and mice.
-
Active Inference: A method for Phenotyping Agency in AI systems?
Active inference offers a variational way to phenotype agency in AI systems by measuring empowerment in generative models via a T-maze paradigm.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.