Pith. sign in

REVIEW 4 major objections 5 minor 59 references

AXIOM: Learning to Play Games in Minutes with Expanding Object-Centric Models

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AXIOM, a gradient-free object-centric agent, masters ten pixel-based games within 10,000 interaction steps and matches or beats deep RL baselines on each.

desk verdict A serious object-centric active inference paper whose headline claim is undercut by its own Table 1; the architecture and benchmark are worth engaging, but the claims need scaling back. read the letter →

arxiv 2505.24784 v1 pith:HIOBPTLA submitted 2025-05-30 cs.AI cs.LGstat.ML

Conor Heins , Toon Van de Maele , Alexander Tschantz , Hampus Linander , Dimitrije Markovic , Tommaso Salvatori , Corrado Pezzato , Ozan Catal
show 6 more authors
Ran Wei Magnus Koudahl Marco Perin Karl Friston Tim Verbelen Christopher Buckley
This is my paper · ORCID
classification cs.AIcs.LGstat.ML
keywords activeinferenceobject-centricworldmodelssample-efficientreinforcementlearningmixtureBayesianmodelreductionlow-dataRLpixel-basedcontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AXIOM is an attempt to close the data-efficiency gap between deep reinforcement learning and human learning by giving an active-inference agent a minimal set of object priors: scenes are made of objects, motion is piecewise linear, and interactions are sparse. The agent parses pixels into object slots, learns a small set of shared motion modes and interaction clusters, and grows or prunes its model online, all without gradients or replay buffers. The paper's central claim is empirical: within 10,000 interaction steps on a new suite of ten pixel-based games, AXIOM matches or beats two popular deep RL methods (BBF and DreamerV3) on every game while using far fewer parameters. A sympathetic reader should care because the result suggests that sample efficiency can come from structured Bayesian inference rather than model scale.

What carries the argument

The central machinery is an object-centric hidden Markov model assembled from four mixture models: the slot mixture model (sMM) parses pixels into object slots; the identity mixture model (iMM) assigns object types by color and shape; the transition mixture model (tMM) describes each slot's motion as piecewise-linear systems; and the recurrent mixture model (rMM) learns how switch states, actions, rewards, and identities co-occur. A truncated stick-breaking prior lets each mixture grow when new data are poorly explained, and Bayesian model reduction every 500 frames merges redundant rMM components to improve generalization. Planning rolls out imagined trajectories and selects actions by minimizing expected free energy, which combines predicted reward with information gain about the world model.

What would settle it

Run AXIOM with unchanged hyperparameters on a pixel-based game where objects are textured or the relevant interaction is not determined by proximity; if the low-data advantage disappears, the advantage rests on the engineered priors rather than on the learning algorithm.

Watch

Extended reading notes

Core claim

The discovery is that a fully Bayesian, object-centric generative model can be the entire controller for pixel-based games: the slot mixture model extracts object latents, the identity mixture model assigns types by color and shape, the transition mixture model captures each object's motion with a shared set of linear modes, and the recurrent mixture model ties switch states to actions, rewards, identities, and relative distances. Learning is online variational Bayes with natural-gradient parameter updates, and structure learning both adds mixture components when new events cannot be explained and merges components via Bayesian model reduction to generalize. Planning selects actions that minimize expected free energy, balancing predicted reward against information gain. On ten Gameworld 10k games, this gradient-free architecture attains higher or comparable cumulative reward than BBF and DreamerV3 within 10k steps, with 0.3–1.6 million parameters, and it recovers from color and shape perturbations by re-mapping object identities.

Load-bearing premise

The central assumption is that Gameworld 10k fairly represents low-data RL tasks, even though the games use deliberately simplified monochromatic objects and proximity-based interactions that match the hand-engineered priors AXIOM is given.

Editorial extensions

If this is right

  • In the low-data regime, a Bayesian agent that explicitly represents objects can reach competitive scores without replay buffers or gradient computations, so sample efficiency need not come from model scale.
  • Convergence is fast: most of AXIOM's final reward is reached within the first 5,000 steps, whereas the deep RL baselines improve over nearly the full 10k budget.
  • Because the world model is built from human-readable components, behavior can be diagnosed: reward clusters, identity slots, and linear motion modes can all be inspected directly.
  • Structure learning automatically grows and prunes the model, so the final model size (0.3–1.6 million parameters) is tuned to the task rather than fixed in advance.
  • Ablations indicate that Bayesian model reduction and information-seeking exploration are not uniformly beneficial: BMR helps spatial generalization in games like Gold and Hunt but can hurt on Cross, and information gain can slow reward acquisition when interactions are punishing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct testable extension: run AXIOM with unchanged hyperparameters on a pixel-based benchmark with textured or occluded objects; keeping the same 10k budget would show whether the object priors generalize beyond simple polygons or need to be learned from data.
  • The fixed-distance ablation suggests a predictable scaling: when the interaction radius is known in advance, AXIOM's reward improves on most games; a learned interaction-radius estimate should therefore dominate the default online version on tasks with a single dominant interaction range.
  • If the same expanding-mixture machinery were applied to infer object types from dynamics as well as shape and color, the color-remapping robustness shown in the perturbation experiments could extend to tasks where shape alone cannot distinguish objects, such as obstacles and rewards with identical geometry.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. AXIOM is a gradient-free, object-centric active inference agent that learns from raw pixel frames by growing and pruning four mixture models (sMM, iMM, tMM, rMM) via variational updates, Bayesian model reduction, and expected-free-energy planning. The paper introduces the Gameworld 10k benchmark of ten simple arcade-style games and reports 10,000-step cumulative rewards against BBF and DreamerV3, along with ablations (fixed interaction distance, no BMR, no information gain) and perturbation experiments. The central claims are that AXIOM 'masters' these games within 10k steps and outperforms deep RL baselines in the low-data regime.

Significance. The architecture is interesting and the paper is technically detailed: the model is fully Bayesian, online, interpretable, and does not use gradient-based optimization; the appendices provide full derivations, algorithms, hyperparameters, and ablation analyses, and the code is released. If the sample-efficiency claims could be supported with a fair comparison, this would be a meaningful contribution to sample-efficient pixel-based control. However, the significance is limited by the fact that the benchmark is author-created and deliberately aligned with AXIOM's hand-engineered priors, and by the internal mismatch between the headline claim and the reported Table 1 numbers. The paper currently reads as a promising proof-of-concept rather than a demonstration that Bayesian object-centric priors beat deep RL in general low-data regimes.

major comments (4)
  1. [Abstract; Section 3; Table 1] The headline claim that AXIOM 'outperforms popular reinforcement learning models in the low-data regime' is contradicted by Table 1. On Cross, AXIOM's cumulative reward is -68±36, the worst of the three methods (BBF: -48±07, DreamerV3: -27±08); on Drive, AXIOM (-49±04) is worse than BBF (-37±06); and on Aviate AXIOM ties BBF (-90±19 vs -90±05). The Section 3 sentence stating that AXIOM 'attains higher, or on par, average cumulative reward than BBF and DreamerV3 in every Gameworld environment' is therefore inaccurate as written. The claims need to be revised to the actual win/tie/loss counts, and the comparison should be accompanied by a statistical analysis over the 10 seeds rather than inspection of means and standard deviations.
  2. [Section 3 'Benchmark'; Appendix D; Limitations] The Gameworld 10k benchmark is author-created and its games are specifically designed around the same object-centric, smooth-trajectory, sparse-interaction priors that AXIOM hard-codes: monochromatic polygon sprites, continuous trajectories, and proximity-based interactions. The deep RL baselines do not receive these priors, so the comparison largely measures how well the benchmark matches AXIOM's inductive bias. The Limitations paragraph acknowledges that the core priors are engineered, but the abstract and conclusion still generalize the result to 'various games' and pixel-based control. To make the central claim load-bearing, the authors should either evaluate on an established low-data pixel-control benchmark (e.g., Atari 100k) or explicitly restrict the claims to the proposed benchmark and frame the result as a proof-of-concept.
  3. [Section 3 'Baselines'; Table 2] The baseline comparison is not fully controlled: AXIOM operates on full 210×160 frames, whereas BBF and DreamerV3 receive downscaled 84×84 and 96×96 inputs. Since pixel resolution is a known factor in sample-efficient RL, this gives AXIOM an information advantage and weakens the claim that its sample efficiency is due to the model architecture. The authors should run resolution-matched baselines or explicitly justify why the resolution difference is expected to be neutral. In addition, no significance tests are reported; several Table 1 entries have overlapping standard deviations across methods.
  4. [Table 1; Section 3 'Reward'; Appendix E.2] The full AXIOM model is not the best-performing variant in its own ablations: the fixed-distance variant achieves higher cumulative reward in 9 of 10 games, and the no-IG variant is better in 6 of 10 games (Aviate, Cross, Drive, Gold, Fruits, Jump). The paper discusses cases where IG or BMR hurt performance, but the presentation still centers the full model as the main result. Please clarify the selection protocol for the full model and discuss whether the full configuration is the recommended one, or whether the claim should be attached to the best variant per game rather than a single method.
minor comments (5)
  1. [Appendix D] The environment is called 'Driver' in the appendix but 'Drive' in Table 1 and Figure 2; please unify the name.
  2. [Section 2] The text says 'a object-centric state space model'; it should be 'an object-centric state space model'.
  3. [Equation (1); Appendix A.1] The term p(xt–1,|zt,ΘiMM) contains a misplaced comma; it should read p(xt–1 | zt, ΘiMM).
  4. [Appendix B] In the fixed-distance ablation, both groups of games are assigned the same value rmin = 1.25; if a different value was intended for the second group, please correct this.
  5. [Table 3; Appendix B] Table 3 lists rmin = 0.075 for the main model, while Appendix B uses rmin = 1.25 for the fixed-distance ablations; please state explicitly how rmin is set for each configuration and whether it was tuned per game.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; AXIOM's empirical claims are externally anchored to game scores, and the internal consistency issues are correctness concerns, not circular derivation.

full rationale

AXIOM's central empirical claim is anchored to external game scores: Table 1 reports cumulative reward against BBF and DreamerV3 on Gameworld 10k, and the baselines are standard published agents run under their own published settings, with only preprocessing modifications described in Section 3. The internal mechanisms the paper invokes—BMR scoring using ancestrally sampled data from the model, planning with the learned rMM reward model, and information gain computed from the agent's own posteriors—are self-referential by design, but none of them is a fitted constant that the headline result reduces to; the ablations show mixed effects (e.g., BMR helps Gold and Hunt but hurts Cross; information gain hurts several games), so performance is not forced by construction. The engineered object-centric priors and the author-created benchmark aligned to those priors are an acknowledged limitation (Section 4, 'Limitations and future work'), but this is an external-validity concern, not circularity, because the benchmark's reward signal is independent of AXIOM's model. Many citations are to the authors' own prior work (e.g., [15], [20], [23], [24]), but the load-bearing algorithms (expansion, BMR, planning) are specified in the paper with equations and pseudocode, so those citations are not load-bearing. Separately, the claim that AXIOM 'outperforms' baselines in every environment is not supported by Table 1 (e.g., Cross: AXIOM -68±36 vs BBF -48±07 and DreamerV3 -27±08; Drive: AXIOM -49±04 vs BBF -37±06), but this is an internal-consistency and correctness issue, not a circularity of derivation.

Assumptions & free parameters 8 free parameters · 7 assumptions · 0 invented entities

No new physical entities are introduced; the object slots, identity types, transition modes, and recurrent mixture components are latent variables internal to AXIOM's learned model, not claims about the world. The free parameters listed above are hand-chosen hyperparameters that the reported performance depends on. The axioms include the domain assumptions (objects, piecewise linear dynamics, sparse interactions) that are engineered into the model and the author-created benchmark.

free parameters (8)
  • Expansion thresholds tau_smm, tau_imm, tau_rmm, tau_tmm = 5.7, -1e2, -1e1, -1e-5
    Hand-set thresholds that determine when new mixture components are created; directly control the model's structure growth rate and are not learned.
  • Interaction radius rmin = 0.075 (default); 1.25 in fixed_distance variant
    Defines the neighborhood for object interactions; the fixed_distance variant tunes it per game (Appendix B), and performance is sensitive to it.
  • Information gain weight lambda_IG = 0.1
    Scales the exploration term in the expected free energy planning objective; ablations show it can hurt performance on some games (Gold, Cross).
  • Component growth priors alpha_0,smm, alpha_0,imm, alpha_0,tmm, alpha_0,rmm = 1, 1e-4, 0.1, 0.1
    Dirichlet pseudocounts controlling the propensity to add new components in each mixture module.
  • Moving and presence latent parameters lambda, beta, zeta = 0.99, 0.01, 0.01
    Hyperparameters shaping the moving and presence Markov chains and the damping of the presence EP update.
  • Unused counter parameters xi, gamma_u, nu_u = 1, infinity, 0.05
    Parameters of the exponentially decaying presence likelihood for respawning objects; the infinite gamma_u collapses the update to a deterministic counter.
  • Planning hyperparameters H, P, S, gamma_discount, topk_ratio, temperature, alpha_smooth = 32, 512, 3, 0.99, 0.1, 10.0, 1.0
    Planning horizon, number of rollouts, samples per rollout, discount, proposal update hyperparameters; a grid search was performed on rollouts and samples (Appendix E.2).
  • Natural-gradient learning rate schedule rho_t
    Used in Eq. (39) for parameter updates but not listed in Table 3; exact schedule is unspecified, hampering direct reproduction.
assumptions (7)
  • domain assumption The world is a POMDP with Markov dynamics and rewards that depend only on the current state and action.
    Section 2 states the POMDP formulation; the agent's model assumes Markovian dynamics, which the Gameworld environments satisfy by construction.
  • domain assumption Scenes factorize into objects with position, color, shape, velocity, and sparse pairwise interactions.
    Core priors introduced in Section 1 and formalized in the sMM and interaction features (Appendices A.2, A.4). This matches the author-designed Gameworld games but may not hold in natural scenes.
  • domain assumption Object dynamics are piecewise linear (switching linear dynamical systems).
    tMM (Section 2 and A.7) models each object's trajectory as a mixture of linear maps; valid for the simple game physics presented.
  • standard math Variational mean-field factorization and conjugate exponential-family priors yield closed-form updates.
    The variational inference in Appendix A.9 relies on conjugacy of Gaussian, Gamma, Dirichlet, and NIW distributions; this is standard Bayesian machinery.
  • ad hoc to paper Hard assignment in the growing heuristic approximates the true posterior and is treated as exact for parameter updates.
    The expansion algorithm (Algorithm 1) hard-assigns datapoints to components and uses their sufficient statistics in M-steps; this is a MAP or CRP heuristic, not a variational approximation of the full posterior.
  • ad hoc to paper Truncated stick-breaking priors bound the number of components (K <= 32, L <= 500, M <= 5000).
    Finite caps on mixture components are needed for tractability and are set by hand (Table 3).
  • ad hoc to paper The unused counter dynamics assume a fixed deterministic reset and increment rule.
    The infinite-gamma limit in Appendix A.5 replaces the exponential likelihood with a deterministic update (Eq. 47), an approximation chosen for simplicity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AXIOM: Learning to Play Games in Minutes with Expanding Object-Centric Models." pith.science (2026). https://pith.science/paper/HIOBPTLA

@misc{pith2026250524784,
  author       = {Pith},
  title        = {Pith review of: AXIOM: Learning to Play Games in Minutes with Expanding Object-Centric Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HIOBPTLA}},
  note         = {Machine review of arXiv:2505.24784}
}
read the original abstract

Current deep reinforcement learning (DRL) approaches achieve state-of-the-art performance in various domains, but struggle with data efficiency compared to human learning, which leverages core priors about objects and their interactions. Active inference offers a principled framework for integrating sensory information with prior knowledge to learn a world model and quantify the uncertainty of its own beliefs and predictions. However, active inference models are usually crafted for a single task with bespoke knowledge, so they lack the domain flexibility typical of DRL approaches. To bridge this gap, we propose a novel architecture that integrates a minimal yet expressive set of core priors about object-centric dynamics and interactions to accelerate learning in low-data regimes. The resulting approach, which we call AXIOM, combines the usual data efficiency and interpretability of Bayesian approaches with the across-task generalization usually associated with DRL. AXIOM represents scenes as compositions of objects, whose dynamics are modeled as piecewise linear trajectories that capture sparse object-object interactions. The structure of the generative model is expanded online by growing and learning mixture models from single events and periodically refined through Bayesian model reduction to induce generalization. AXIOM masters various games within only 10,000 interaction steps, with both a small number of parameters compared to DRL, and without the computational expense of gradient-based optimization.

Figures

Figures reproduced from arXiv: 2505.24784 by the authors.

Figure 1
Figure 1. Inference and prediction flow using AXIOM: The sMM extracts object-centric repre￾sentations from pixel inputs. For each object latent and its closest interacting counterpart, a discrete identity token is inferred using the iMM and passed to the rMM, along with the distance and the action, to predict the next reward and the tMM switch. The object latents are then updated using the tMM and the predicted switch to gene… view at source ↗
Figure 2
Figure 2. Gameworld10k: Visual impression of the 10 games in the Gameworld 10k suite. Se￾quences of ten frames are overlayed with increasing opacity to showcase the game dynamics. structure of the game, e.g., the shape and color of game objects. On this benchmark, our agent outperforms popular reinforcement learning models in the low-data regime (10,000 interaction steps) without relying on any kind of gradient-based optimiza… view at source ↗
Figure 3
Figure 3. Online learning performance. Moving average (1k steps) reward per step during training for AXIOM, BBF and DreamerV3 on Gameworld 10k environments. Mean and standard deviation over 10 parameter seeds per model and environment. 7 [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Tracking AXIOM’s Behavior. (a) Example frame from Impact at time t (left); imagined trajectory in latent space conditioned on the observation at time t and 32 timesteps into the future, conditioned on an action sequence with high predicted reward (middle); and rMM clus…
Figure 5
Figure 5. Figure 5: Computational costs. Scaling of planning time as a function of the number of policies (left), and model inference time as a function of the number of sMM components (right). All times measured on a single A100 GPU. 30 [PITH_FULL_IMAGE:figures/full_fig_p030_5.png]
Figure 6
Figure 6. Figure 6: 100K performance on Gold & Hunt. 31 [PITH_FULL_IMAGE:figures/full_fig_p031_6.png]
Figure 7
Figure 7. Figure 7: Performance of AXIOM ablations. Average reward over the final 1,000 frames across 10 Gameworld 10K environments for three AXIOM variants: the full AXIOM model, a version without Bayesian Model Reduction (AXIOM (no BMR)), and a version excluding information gain during …
Figure 8
Figure 8. Figure 8: Visualizations of the rMM clusters on Cross for information gain and BMR ablations. Each Gaussian cluster depicts a particular dynamics for a particular object type, colored by the object color, and the edge color of a nearby “interacting” object. (a) AXIOM has various…
Figure 9
Figure 9. Figure 9: Ablation on the amount of sampled policies. The label indicates the number of policies × number of samples for that policy (a) 1 Sample (b) 3 Samples (c) 5 Samples 34 [PITH_FULL_IMAGE:figures/full_fig_p034_9.png]
Figure 10
Figure 10. Figure 10: Perturbations. Sample frames from each of the ten environments under (a) no perturbation, (b) a color perturbation, and (c) a shape perturbation. type inference. In practice, what this means is that slots that changed colors don’t rapidly get assigned new identities, …
Figure 11
Figure 11. Figure 11: Impact of perturbations on average reward. Smoothed 1k-step average rewards for Axiom, BBF, and Dreamer across ten games under (a) no perturbation, (b) color perturbation, (c) shape perturbation, and (d) Axiom’s color perturbation with and without remapping. 36 [PITH…
Figure 12
Figure 12. Figure 12: iMM identity slots on Explode. (a) On Explode, the iMM constructs a slot for the player, bomber and bomb respectively. (b) When color is perturbed, novel slots are created for the blue (player) and the pink bomb, and the yellow enemy is mapped onto the old player slot…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 41 canonical work pages

  1. [1]

    Deep reinforcement learning: An overview,

    Y . Li, “Deep reinforcement learning: An overview,”arXiv preprint arXiv:1701.07274, 2017

    arXiv 2017

  2. [2]

    Core knowledge,

    E. S. Spelke and K. D. Kinzler, “Core knowledge,”Developmental science, vol. 10, no. 1, pp. 89–96, 2007

    work page 2007

  3. [3]

    Building machines that learn and think like people,

    B. M. Lake, T. D. Ullman, J. B. Tenenbaum, and S. J. Gershman, “Building machines that learn and think like people,”Behavioral and Brain Sciences, vol. 40, p. e253, 2017

    work page 2017

  4. [4]

    Human-level concept learning through probabilistic program induction,

    B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum, “Human-level concept learning through probabilistic program induction,”Science, vol. 350, no. 6266, pp. 1332–1338, 2015

    2015

  5. [5]

    Pure reasoning in 12-month-old infants as probabilistic inference,

    E. Téglás, E. Vul, V . Girotto, M. Gonzalez, J. B. Tenenbaum, and L. L. Bonatti, “Pure reasoning in 12-month-old infants as probabilistic inference,”science, vol. 332, no. 6033, pp. 1054–1059, 2011

    work page 2011

  6. [6]

    Spatiotemporal continuity, smooth- ness of motion and object identity in infancy,

    E. S. Spelke, R. Kestenbaum, D. J. Simons, and D. Wein, “Spatiotemporal continuity, smooth- ness of motion and object identity in infancy,”British journal of developmental psychology, vol. 13, no. 2, pp. 113–142, 1995

    work page 1995

  7. [7]

    Principles of object perception,

    E. S. Spelke, “Principles of object perception,”Cognitive science, vol. 14, no. 1, pp. 29–56, 1990

    work page 1990

  8. [8]

    Do six-month-old infants perceive causality?,

    A. M. Leslie and S. Keeble, “Do six-month-old infants perceive causality?,”Cognition, vol. 25, no. 3, pp. 265–288, 1987

    work page 1987

Show all 59 references
  1. [9]

    Provable composi- tional generalization for object-centric learning,

    T. Wiedemer, J. Brady, A. Panfilov, A. Juhos, M. Bethge, and W. Brendel, “Provable composi- tional generalization for object-centric learning,”arXiv preprint arXiv:2310.05327, 2023

    2023 arXiv

  2. [10]

    Object-centric representations generalize better compositionally with less compute,

    F. Kapl, A. M. K. Mamaghan, M. Horn, C. Marr, S. Bauer, and A. Dittadi, “Object-centric representations generalize better compositionally with less compute,” inICLR 2025 Workshop on World Models: Understanding, Modelling and Scaling, 2025

    2025

  3. [11]

    Unsupervised object-level deep reinforcement learning,

    W. Agnew and P. Domingos, “Unsupervised object-level deep reinforcement learning,” in NeurIPS workshop on deep RL, 2018

    2018

  4. [12]

    Neural relational inference for interacting systems,

    T. Kipf, E. Fetaya, K.-C. Wang, M. Welling, and R. Zemel, “Neural relational inference for interacting systems,” inInternational conference on machine learning, pp. 2688–2697, Pmlr, 2018

    2018

  5. [13]

    Spartan: A sparse transformer learning local causation,

    A. Lei, B. Schölkopf, and I. Posner, “Spartan: A sparse transformer learning local causation,” arXiv preprint arXiv:2411.06890, 2024

    2024

  6. [14]

    Objects matter: object-centric world models improve reinforcement learning in visually complex environments,

    W. Zhang, A. Jelley, T. McInroe, and A. Storkey, “Objects matter: object-centric world models improve reinforcement learning in visually complex environments,”arXiv preprint arXiv:2501.16443, 2025

    2025

  7. [15]

    T. Parr, G. Pezzulo, and K. J. Friston,Active inference: the free energy principle in mind, brain, and behavior. MIT Press, 2022. 10

    2022

  8. [16]

    The free-energy principle: a unified brain theory?,

    K. Friston, “The free-energy principle: a unified brain theory?,”Nature reviews neuroscience, vol. 11, no. 2, pp. 127–138, 2010

    2010

  9. [17]

    The bayesian brain: the role of uncertainty in neural coding and computation,

    D. C. Knill and A. Pouget, “The bayesian brain: the role of uncertainty in neural coding and computation,”TRENDS in Neurosciences, vol. 27, no. 12, pp. 712–719, 2004

    2004

  10. [18]

    Object-centric learning with slot attention,

    F. Locatello, D. Weissenborn, and O. Unsupervised, “Object-centric learning with slot attention,” inAdvances in Neural Information Processing Systems, vol. 33, pp. 1821–1834, 2020

    2020

  11. [19]

    Recurrent switching linear dynamical systems,

    S. W. Linderman, A. C. Miller, R. P. Adams, D. M. Blei, L. Paninski, and M. J. Johnson, “Recurrent switching linear dynamical systems,”arXiv preprint arXiv:1610.08466, 2016

    2016 arXiv

  12. [20]

    Gradient-free variational learning with conditional mixture networks,

    C. Heins, H. Wu, D. Markovic, A. Tschantz, J. Beck, and C. Buckley, “Gradient-free variational learning with conditional mixture networks,”arXiv preprint arXiv:2408.16429, 2024

    2024 arXiv

  13. [21]

    Bayesian model reduction and empirical bayes for group (dcm) studies,

    K. J. Friston, V . Litvak, A. Oswal, A. Razi, K. E. Stephan, B. C. Van Wijk, G. Ziegler, and P. Zeidman, “Bayesian model reduction and empirical bayes for group (dcm) studies,” Neuroimage, vol. 128, pp. 413–431, 2016

    2016

  14. [22]

    Bayesian model reduction,

    K. Friston, T. Parr, and P. Zeidman, “Bayesian model reduction,”arXiv preprint arXiv:1805.07092, 2018

    2018 arXiv

  15. [23]

    From pixels to planning: scale-free active inference,

    K. Friston, C. Heins, T. Verbelen, L. Da Costa, T. Salvatori, D. Markovic, A. Tschantz, M. Koudahl, C. Buckley, and T. Parr, “From pixels to planning: scale-free active inference,” arXiv preprint arXiv:2407.20292, 2024

    2024 arXiv

  16. [24]

    Supervised structure learning,

    K. J. Friston, L. Da Costa, A. Tschantz, A. Kiefer, T. Salvatori, V . Neacsu, M. Koudahl, C. Heins, N. Sajid, D. Markovic,et al., “Supervised structure learning,”Biological Psychology, vol. 193, p. 108891, 2024

    2024

  17. [25]

    The arcade learning environment: An evaluation platform for general agents,

    M. G. Bellemare, Y . Naddaf, J. Veness, and M. Bowling, “The arcade learning environment: An evaluation platform for general agents,”Journal of Artificial Intelligence Research, vol. 47, pp. 253–279, jun 2013

    2013

  18. [26]

    Mujoco: A physics engine for model-based control,

    E. Todorov, T. Erez, and Y . Tassa, “Mujoco: A physics engine for model-based control,” in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5026–5033, IEEE, 2012

    2012

  19. [27]

    Gibbs sampling methods for stick-breaking priors,

    H. Ishwaran and L. F. James, “Gibbs sampling methods for stick-breaking priors,”Journal of the American statistical Association, vol. 96, no. 453, pp. 161–173, 2001

    2001

  20. [28]

    Dynamic markov blanket detection for macroscopic physics discovery,

    J. Beck and M. J. Ramstead, “Dynamic markov blanket detection for macroscopic physics discovery,”arXiv preprint arXiv:2502.21217, 2025

    2025 arXiv

  21. [29]

    Switching state-space models,

    Z. Ghahramani and G. E. Hinton, “Switching state-space models,”University of Toronto Technical Report CRG-TR-96-3, Department of Computer Science, 1996

    1996

  22. [30]

    Generative or discriminative? getting the best of both worlds,

    C. Bishop and J. Lasserre, “Generative or discriminative? getting the best of both worlds,” Bayesian statistics, vol. 8, no. 3, pp. 3–24, 2007

    2007

  23. [31]

    Graphical models, exponential families, and variational inference,

    M. J. Wainwright, M. I. Jordan,et al., “Graphical models, exponential families, and variational inference,”Foundations and Trends® in Machine Learning, vol. 1, no. 1–2, pp. 1–305, 2008

    2008

  24. [32]

    Stochastic variational inference,

    M. D. Hoffman, D. M. Blei, C. Wang, and J. Paisley, “Stochastic variational inference,”the Journal of machine Learning research, vol. 14, no. 1, pp. 1303–1347, 2013

    2013

  25. [33]

    Active inference: a process theory,

    K. Friston, T. FitzGerald, F. Rigoli, P. Schwartenbeck, and G. Pezzulo, “Active inference: a process theory,”Neural computation, vol. 29, no. 1, pp. 1–49, 2017

    2017

  26. [34]

    Big- ger, better, faster: Human-level atari with human-level efficiency,

    M. Schwarzer, J. S. O. Ceron, A. Courville, M. G. Bellemare, R. Agarwal, and P. S. Castro, “Big- ger, better, faster: Human-level atari with human-level efficiency,” inInternational Conference on Machine Learning, pp. 30365–30380, PMLR, 2023. 11

    2023

  27. [35]

    Sample- efficient reinforcement learning by breaking the replay ratio barrier,

    P. D’Oro, M. Schwarzer, E. Nikishin, P.-L. Bacon, M. G. Bellemare, and A. Courville, “Sample- efficient reinforcement learning by breaking the replay ratio barrier,” inDeep Reinforcement Learning Workshop NeurIPS 2022, 2022

    2022

  28. [36]

    Mastering diverse control tasks through world models,

    D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap, “Mastering diverse control tasks through world models,”Nature, pp. 1–7, 2025

    2025

  29. [37]

    Expectation propagation for approximate bayesian inference,

    T. P. Minka, “Expectation propagation for approximate bayesian inference,”arXiv preprint arXiv:1301.2294, 2013

    2013 arXiv

  30. [38]

    Variational inference mpc for bayesian model-based reinforcement learning,

    M. Okada and T. Taniguchi, “Variational inference mpc for bayesian model-based reinforcement learning,” inConference on Robot Learning, 2019

    2019

  31. [39]

    Playing atari with deep reinforcement learning,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,”arXiv preprint arXiv:1312.5602, 2013

    2013 arXiv

  32. [40]

    Mastering atari games with limited data,

    W. Ye, S. Liu, T. Kurutach, P. Abbeel, and Y . Gao, “Mastering atari games with limited data,” Advances in neural information processing systems, vol. 34, pp. 25476–25488, 2021

    2021

  33. [41]

    Efficientzero v2: Mastering discrete and continuous control with limited data,

    S. Wang, S. Liu, W. Ye, J. You, and Y . Gao, “Efficientzero v2: Mastering discrete and continuous control with limited data,”arXiv preprint arXiv:2403.00564, 2024

    2024 arXiv

  34. [42]

    Dream to control: Learning behaviors by latent imagination,

    D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi, “Dream to control: Learning behaviors by latent imagination,”arXiv preprint arXiv:1912.01603, 2019

    1912 arXiv

  35. [43]

    Mastering atari with discrete world models,

    D. Hafner, T. Lillicrap, M. Norouzi, and J. Ba, “Mastering atari with discrete world models,” arXiv preprint arXiv:2010.02193, 2020

    2010 arXiv

  36. [44]

    Multi-object representation learning with iterative variational inference,

    K. Greff, R. L. Kaufman, R. Kabra, N. Watters, C. Burgess, D. Zoran, L. Matthey, M. Botvinick, and A. Lerchner, “Multi-object representation learning with iterative variational inference,” in International conference on machine learning, pp. 2424–2433, PMLR, 2019

    2019

  37. [45]

    Object-centric learning with slot attention,

    F. Locatello, D. Weissenborn, T. Unterthiner, A. Mahendran, G. Heigold, J. Uszkoreit, A. Doso- vitskiy, and T. Kipf, “Object-centric learning with slot attention,”Advances in neural information processing systems, vol. 33, pp. 11525–11538, 2020

    2020

  38. [46]

    Attention as implicit structural inference,

    R. Singh and C. L. Buckley, “Attention as implicit structural inference,”Advances in Neural Information Processing Systems, vol. 36, pp. 24929–24946, 2023

    2023

  39. [47]

    Object-centric learning with slot mixture module,

    D. Kirilenko, V . V orobyov, A. K. Kovalev, and A. I. Panov, “Object-centric learning with slot mixture module,”arXiv preprint arXiv:2311.04640, 2023

    2023 arXiv

  40. [48]

    Slot state space models,

    J. Jiang, F. Deng, G. Singh, M. Lee, and S. Ahn, “Slot state space models,”arXiv preprint arXiv:2406.12272, 2024

    2024 arXiv

  41. [49]

    Focus: Object-centric world models for robotic manipulation,

    S. Ferraro, P. Mazzaglia, T. Verbelen, and B. Dhoedt, “Focus: Object-centric world models for robotic manipulation,”Frontiers in Neurorobotics, vol. 19, p. 1585386, 2025

    2025

  42. [50]

    Slot structured world models,

    J. Collu, R. Majellaro, A. Plaat, and T. M. Moerland, “Slot structured world models,”arXiv preprint arXiv:2402.03326, 2024

    2024 arXiv

  43. [51]

    The infinite gaussian mixture model,

    C. Rasmussen, “The infinite gaussian mixture model,”Advances in neural information process- ing systems, vol. 12, 1999

    1999

  44. [52]

    Structure learning with temporal gaussian mixture for model-based reinforcement learning,

    T. Champion, M. Grze´s, and H. Bowman, “Structure learning with temporal gaussian mixture for model-based reinforcement learning,”arXiv preprint arXiv:2411.11511, 2024

    2024 arXiv

  45. [53]

    Variational learning for switching state-space models,

    Z. Ghahramani and G. E. Hinton, “Variational learning for switching state-space models,” Neural computation, vol. 12, no. 4, pp. 831–864, 2000

    2000

  46. [54]

    Parsing neural dynamics with infinite recurrent switching linear dynamical systems,

    V . Geadah, J. W. Pillow,et al., “Parsing neural dynamics with infinite recurrent switching linear dynamical systems,” inThe Twelfth International Conference on Learning Representations, 2024

    2024

  47. [55]

    Dependent multinomial models made easy: Stick-breaking with the pólya-gamma augmentation,

    S. Linderman, M. J. Johnson, and R. P. Adams, “Dependent multinomial models made easy: Stick-breaking with the pólya-gamma augmentation,”Advances in neural information process- ing systems, vol. 28, 2015. 12 A Full Model Details AXIOM’s world model is a hidden Markov model wit...

    2015

  48. [56]

    First, we compute theexpected sufficient statistics bTµ using the current posteriors over the latent variablesq(Z t)obtained from thet-th E-step: bTµ =E q(Zt) T Θµ,Z t .(38) These expected statistics are then combined with prior natural parameters ηµ,0 to form the target natur...

  49. [57]

    Second, we update the current natural parameters η(t−1) µ using anatural-gradientstep, which acts as a stochastic update blending the previous parameters with the new target parameters, controlled by a learning rate scheduleρ t: η(t) µ ←(1−ρ t)η (t−1) µ +ρ t bηµ,where0< ρ t ≤1...

  50. [58]

    The pixel responsibilities rn t,k, representing q(z n t,k,sMM = 1), are updated using the standard mixture model update: rn t,k = exp Eq[logπ k,smm] +E q[logN(y n t ;Ax (k) t ,Σ (k))] PK j=1 exp Eq[logπ j,smm] +E q[logN(y n t ;Ax (j) t ,Σ (j))] .(40) 21 The per-slot observatio...

  51. [59]

    expand-or-stop

    The parameters of the Gaussian posterior q(x(k) t ) are updated by incorporating evidence from the pixels assigned to slot k. This involves updating its natural parameters (precision Λ(k) t and precision-adjusted meanh (k) t ): Λ(k) t = Λ(k) t|t–1 + NX n=1 rn t,k A⊤ Σ(k) −1 A,...

    2000

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.