Pith. sign in

REVIEW 4 major objections 7 minor 1 cited by

Structure learning with Temporal Gaussian Mixture for model-based Reinforcement Learning

T0 review · 4 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A temporal Gaussian mixture with a Dirichlet-categorical transition model and belief-scaled Q-learning can discover the number of maze states, learn transition probabilities, and navigate from start to exit.

desk verdict A transparent but partial structure-learning result: the method works on small mazes, and the paper's own figures show the component-drift problem that limits the central claim. read the letter →

arxiv 2411.11511 v1 pith:HW7PSKUM submitted 2024-11-18 cs.LG cs.AIstat.ML

Théophile Champion , Marek Grześ , Howard Bowman This is my paper · ORCID
classification cs.LGcs.AIstat.ML
keywords structurelearningtemporalGaussianmixturevariationalmodel-basedreinforcementbelief-basedQ-learningDirichlet-categoricaltransitionmodelcontinuousobservationmazenavigation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a model-based reinforcement learner that builds its own map of a continuous environment. The core claim is that a temporal Gaussian mixture—a variational Gaussian mixture for perception plus a Dirichlet-categorical model for transitions—can discover how many states a maze has, learn the transition probabilities between those states, and then plan with a Q-learning rule that works on belief states rather than observed states. The authors demonstrate the claim in several small continuous mazes: the learned components line up with maze cells, the learned transition tables match the environment's dynamics, and the agent navigates from start to the exit. The reason to care is that the number of latent states is not given in advance, so the same machinery could apply where an expert model is unavailable.

What carries the argument

The carrying object is the temporal Gaussian mixture (TGM), which splits into two conjugate-friendly models. The perception model is a variational Gaussian mixture (VGM): each latent state is a multivariate Gaussian component, mean-shift clustering initializes the components, and variational updates compute responsibilities $\hat r_{nk}$, with unused components pruned and new components added as data accumulate. The transition model is a Dirichlet-categorical network whose concentration tensor $\hat b[a]_{kj}$ counts, probabilistically, transitions from component $k$ to component $j$ under action $a$; conjugacy makes the updates simple accumulations $\hat b[a]_{kj} = \bar b[a]_{kj} + \sum_{n\in M''} [a=a^0_n]\hat r^0_{nk}\hat r^1_{nj}$. The forgetting mechanism is what keeps the method online: observations associated with fixed components are folded into an empirical prior (same functional form as the original prior) and then discarded, while flexible components keep their data. Structure learning is driven by the fixed/flexible distinction: every 100 iterations the model matches components by KL divergence below 0.5, and a component becomes fixed only after persisting four consecutive checks. The Q-learning variant then uses the learned $P(z_{t+1}|z_t,a_t)$ and the posterior belief $Q(z_t)$ to update values: $q(a_t,z_t)\leftarrow q(a_t,z_t)+\alpha Q(z_t)[r_{t+1}+\gamma\sum_{z_{t+1}}P(z_{t+1}|z_t,a_t)\max_{a_{t+1}}q(a_{t+1},z_{t+1})-q(a_t,z_t)]$.

What would settle it

Record the maze cell occupied by the agent and the Gaussian component with highest responsibility at every 100-step checkpoint. If the same physical cell is ever assigned to two different components that satisfy the $D_{\mathrm{KL}} < 0.5$ matching criterion, or if a fixed component's mean shifts from one cell to a neighbor while staying fixed, the identity assumption fails and the learned transition probabilities and Q-values describe a different environment; the paper's Figure 12 already shows unlearned components in mazes 12(b) and 12(f).

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that structure learning and planning can be combined without a fixed state count. The perception model is a variational Gaussian mixture initialized by mean-shift clustering: it prunes Gaussian components that no data support and adds new components whenever the agent encounters a new cluster, so the number of components (states) is learned rather than specified. A separate categorical-Dirichlet transition model records, for each action, the probabilistic counts of moving from one component to the next, and conjugate updates give the transition probabilities $P(z_{t+1}|z_t,a_t)$. To keep memory bounded, the model folds forgettable observations into empirical priors, but only for components that are deemed fixed by repeated KL-based matching across 100-step checks. Finally, decision making uses a variant of Q-learning in which the temporal-difference term is scaled by the posterior belief $Q(z_t)$ over states; the paper shows this reduces to standard Q-learning when the belief is a point mass. Empirically, the agent solved four of six mazes, and the authors report that it discovered the number of states and the transition probabilities.

Load-bearing premise

The load-bearing premise is that a Gaussian component which looks the same across consecutive 100-step checks really is the same physical state forever; the forgetting rules and the transition counts all assume this identity is stable, and the paper's own experiments show it fails as the number of maze states grows.

Editorial extensions

If this is right

  • A model-based RL agent can operate in continuous state spaces without a pre-specified number of states: the perception model adds and prunes Gaussian components as the agent explores.
  • The learned transition tensor $B[a]_{kj}$ gives an interpretable description of the environment's dynamics, so the resulting behavior is inspectable rather than a black-box policy.
  • Belief-scaled Q-learning is a viable planning rule under state uncertainty; when the posterior is concentrated on one state, it coincides with standard Q-learning.
  • On the tested mazes, the method is competitive with DQN and A2C: it outperforms A2C on all solvable mazes, matches or beats DQN on some, and learns faster on one maze.
  • The structure-learning loop is the bottleneck: with more states, components become unstable and fuse or disappear (orange and red cells in Figure 12), and lack of exploration prevents solving a long-corridor maze.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the KL-threshold matching rule treats component identity as a local, pairwise judgment; an alternative is to track identity by posterior predictive probability over a fixed set of cells, which would let the model detect that a 'fixed' component has drifted to a neighboring cell.
  • Editorial extension: the forgetting rule keeps only observations attached to fixed components, so memory is automatically concentrated on stable parts of the environment; one could test whether making the KL threshold and persistence count scale with maze size stabilizes larger mazes.
  • Editorial extension: belief-scaled Q-learning suggests a natural exploration bonus—update Q-values for states in proportion to current belief, then choose actions by uncertainty in the transition model; the paper does not implement this, but its own exploration failure in the corridor maze points to it.
  • Editorial extension: the paper intentionally trains perception before transitions because joint learning performs worse; a natural test is to re-tune both models together after an initial structure-learning phase, to see whether the VGM's component competition is disrupted by transition information.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes a temporal Gaussian mixture model (TGM) for model-based reinforcement learning in continuous-observation environments. The perception model is a variational Gaussian mixture (VGM) initialized by mean-shift clustering, with an empirical-prior mechanism that lets the model forget part of the data while integrating its information into the prior. The transition model is a Dirichlet-categorical model that accumulates transition counts between inferred Gaussian components under each action. A belief-scaled Q-learning variant is introduced for planning from posterior beliefs over latent states. The experimental section evaluates the model on six small maze environments, comparing against DQN and A2C, and reports that the TGM learns the maze structure and solves a subset of the mazes. Extensive variational derivations are provided in Appendices B--F.

Significance. If the central claims are fully supported, the paper would make a useful contribution: it combines a tractable variational Gaussian mixture with a forgetting mechanism and a belief-based Q-learning update, providing an interpretable model-based alternative to model-free deep RL on small continuous-state tasks. The variational derivations in the appendices are detailed and mostly standard, which is a strength: the update equations for the perception and transition models are given explicitly and are checkable. However, the paper's central empirical claim—that the model 'discovered the number of states and the transition probabilities'—is currently supported only by qualitative inspection of small mazes, and the paper's own results show that component identity drift causes failures as the number of states grows. The incremental contribution over existing variational GMM and active-inference structure-learning work is real but modest, and the lack of quantitative structure-learning metrics, convergence guarantees for the belief Q-learning update, and code release limits the paper's impact.

major comments (4)
  1. [Section 4.2, Eq. (57)] The component-persistence and fixing mechanism is described only procedurally, with ad-hoc thresholds θkl=0.5 and θcounts=4. The paper does not specify how component identities are tracked when the number of components K grows or shrinks, nor how the KL matching is performed when components split or merge. Since forgetting (Section 4.3) and transition-count accumulation (Eqs. (55)--(56)) are restricted to fixed components, any matching failure propagates directly into the learned transition tensor B. The paper's own Figure 12 and Section 7 concede that components become unstable, take over neighboring cells, and fuse together. Therefore the abstract's claim that the model discovered the number of states and transition probabilities is not supported for general mazes. Please provide a quantitative, ground-truth-based evaluation of structure discovery (e.g., component-to-cell confusion matrices, component count over time, number of identity switches) for every maze, and constrain the abstract's claims accordingly.
  2. [Section 3.1, Eqs. (52) and surrounding text] In the transition model, the variational factors Q(Z0) and Q(Z1) are initialized from the perception model and are explicitly kept fixed; no update equations for Z0 and Z1 are derived for the transition model. This means the transition model never uses action or transition information to refine the state posterior, so the model is not a single jointly inferred temporal model but two independently fitted components. The empirical statement in the text that separate learning 'works better' is not quantified. Please state this as an explicit approximation, justify it with an ablation, or derive the missing update equations.
  3. [Section 5.2, Eq. (71)] The belief-scaled Q-learning update is introduced without convergence analysis. Standard Q-learning convergence requires repeated updates of each state-action pair under a Robbins-Monro step-size schedule, but here the update is scaled by the posterior Q(zt), the transition kernel P(zt+1|zt,at) is itself estimated and changing over time, and the state set K grows and shrinks. The paper does not provide a fixed-point or contraction argument, and the experiments do not isolate the contribution of the belief weighting. Please add a convergence or consistency analysis, or at least an ablation comparing Eq. (71) to standard Q-learning with a hard assignment to the maximum-responsibility state.
  4. [Section 6, Figure 12 and Figure 13] The evaluation of structure learning relies on manual inspection of learned components ('By manual inspection of the learned components and transition matrices...'), and the unsolved mazes are attributed to exploration and component instability without quantitative support. The number of independent runs, hyperparameter settings, and the exact state representation used for DQN and A2C are not reported. There is also no model-based baseline. Please report quantitative structure-learning metrics (e.g., adjusted Rand index against the true cell partition, learned state count over time), standard errors over seeds, and the full hyperparameter configurations for all algorithms.
minor comments (7)
  1. [Section 4.2, Eq. (57)] The KL divergence between two Gaussian distributions is asymmetric, but the direction is not specified. Please state explicitly whether the divergence is DKL[N(µ1,Λ1)||N(µ2,Λ2)] or the reverse, and whether the comparison is symmetric in practice.
  2. [Section 2.2.4, after Eq. (41)] The text says that when a component's responsibilities become zero, 'the posterior parameters of this component revert to their prior counterparts,' but this reversion is not reflected in the update equations. Please specify the condition and the exact parameters that are reverted.
  3. [Section 3.1, Eqs. (55)--(56) and Eq. (77)] The index order in the transition-count updates (k for z0 and j for z1) appears inconsistent with the categorical likelihood in Eq. (77), where the parameters B[a] are indexed with zτ first and zτ+1 second. Please check and unify the index conventions.
  4. [Section 2.2.1] The hyperparameters dk, βk, and vk are set to values proportional to K, but K changes over time as components are added or removed. The paper does not discuss how the priors should be rescaled or whether this dependence is intentional; a brief comment would help.
  5. [Figure 7 caption] The caption states that state seven is above state six but not displayed because the agent did not visit it in the last 100 steps. This is confusing: if the state is never visited, it is unclear how it was learned; please clarify the visitation condition and its effect on the displayed transition matrices.
  6. [Section 6, Figure 13] In Figure 13(b) the text says all three agents performed the same, and only DQN is visible. Please state explicitly whether the TGM and A2C curves are exactly zero or whether they were omitted for legibility, and clarify the episode-reward scale used for the comparison.
  7. [Appendix F, Eq. (179)] The conditional expectation Eπ[rt+1 + γGt+1 | st+1] omits the conditioning on st and at. As written it is a notational shortcut that may mislead readers about the derivation; please write the full conditioning.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the variational derivation and maze-solving check are independent of the model's own fitted quantities.

full rationale

The paper's derivation chain is self-contained in the relevant sense. The perception model uses standard conjugate variational updates (Eqs. 24-41) derived in Appendices C-D from the general variational update Eq. (23); no target behavioral result is used to set these parameters. The transition model's Dirichlet-categorical updates (Eqs. 55-56) are exact conjugacy counts over the responsibility products from the perception model, and the belief-scaled Q-learning rule (Eqs. 70-71) is introduced as a new algorithmic proposal rather than as a fitted predictor. The maze-solving outcomes in Section 6 are external behavioral checks: the learned transition matrices and Q-values are used to navigate, and success or failure is not fed back into the model parameters. The cited prior work by the authors (e.g., Champion et al. 2021 for variational message passing) supports standard update mechanics, but the paper re-derives those updates in its appendices, so the self-citations are not load-bearing. The equality of responsibilities in the empirical prior and posterior (Section 2.2.2) is a modeling approximation, not a prediction; the paper explicitly notes that responsibilities cannot be used in the generative model. Component instability and unlearning (Figure 12, Section 7) are correctness limitations of the learned representation, not circularities in the derivation. The structure-learning claim is evaluated by manual inspection of learned components against maze cells, which is an external comparison. No step reduces by construction to its inputs.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central contribution rests on several hand-set thresholds and on modeling assumptions that are not independently validated. The most consequential are the belief-based Q-learning convergence assumption and the component-persistence assumption, both of which are load-bearing for the reported navigation and structure-learning results.

free parameters (6)
  • Mean-shift bandwidth θb = not reported
    Controls the number of initial Gaussian components found by mean-shift clustering in Section 2.1; no sensitivity analysis is provided.
  • KL component-matching threshold θkl = 0.5
    Used in Eq. (57) to decide whether two Gaussian components are the same across time in Section 4.2.
  • Component persistence count θcounts = 4
    Number of consecutive 100-step survivals required for a component to become fixed in Section 4.2.
  • VGM prior strengths dk and βk = 2K
    Hand-chosen initial concentration and precision-scaling parameters in Section 2.2.1.
  • Wishart degrees of freedom vk = 2K + O - 0.99
    Hand-chosen initialization in Section 2.2.1, where O is the observation dimensionality.
  • Q-learning learning rate α = not reported
    Defined in Eq. (69) but its value and schedule are not stated in the experiments.
assumptions (5)
  • domain assumption Observations are drawn from a mixture of multivariate Gaussians, one component per latent state.
    Stated in Section 1 and central to the perception model; if observations within a state are multimodal or non-Gaussian, the component-to-state mapping breaks.
  • domain assumption The structured mean-field factorization in Eqs. (9) and (14) is accurate enough for structure learning.
    Variational inference is approximate, and the paper does not measure the approximation error or its effect on component discovery.
  • ad hoc to paper Fixed responsibilities from the perception model can be used as exact posterior inputs to the transition model.
    Section 3.1 states that learning the likelihood first and then the transition works better than joint inference, but no ablation or theoretical justification is given.
  • ad hoc to paper The belief-based Q-learning update in Eq. (71) converges to useful Q-values.
    The update scales temporal differences by posterior beliefs; no convergence proof or comparison to belief-MDP planning is provided.
  • domain assumption Gaussian components can be matched across time using the KL threshold, preserving their identity.
    Section 4.2 relies on this matching, but the paper's own experiments show component fusion and unlearning when the number of states grows.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Structure learning with Temporal Gaussian Mixture for model-based Reinforcement Learning." pith.science (2026). https://pith.science/paper/HW7PSKUM

@misc{pith2026241111511,
  author       = {Pith},
  title        = {Pith review of: Structure learning with Temporal Gaussian Mixture for model-based Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HW7PSKUM}},
  note         = {Machine review of arXiv:2411.11511}
}
read the original abstract

Model-based reinforcement learning refers to a set of approaches capable of sample-efficient decision making, which create an explicit model of the environment. This model can subsequently be used for learning optimal policies. In this paper, we propose a temporal Gaussian Mixture Model composed of a perception model and a transition model. The perception model extracts discrete (latent) states from continuous observations using a variational Gaussian mixture likelihood. Importantly, our model constantly monitors the collected data searching for new Gaussian components, i.e., the perception model performs a form of structure learning (Smith et al., 2020; Friston et al., 2018; Neacsu et al., 2022) as it learns the number of Gaussian components in the mixture. Additionally, the transition model learns the temporal transition between consecutive time steps by taking advantage of the Dirichlet-categorical conjugacy. Both the perception and transition models are able to forget part of the data points, while integrating the information they provide within the prior, which ensure fast variational inference. Finally, decision making is performed with a variant of Q-learning which is able to learn Q-values from beliefs over states. Empirically, we have demonstrated the model's ability to learn the structure of several mazes: the model discovered the number of states and the transition probabilities between these states. Moreover, using its learned Q-values, the agent was able to successfully navigate from the starting position to the maze's exit.

Figures

Figures reproduced from arXiv: 2411.11511 by the authors.

Figure 1
Figure 1. (a) shows how the centroids are initialized and updated, while (b) illustrates how the data points are clustered (different colors indicate different clusters). 2.2 Variational Gaussian mixture While the mean-shift algorithm can identify clusters of data points, this algorithm does not scale well with the number of data points, i.e., for each additional data point Equation (1) must be iterated until convergence. Thu… view at source ↗
Figure 2
Figure 2. This figure illustrates the graphical model of the variational Gaussian mixture. The latent variables are represented by a white circle with the variable’s name at the center, and the observed variables are depicted similarly but with a gray background. Finally, arrows connect the predictor variables to their target variable [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. This figure illustrates the variational Gaussian mixture prior, when its parameters are initialized using the mean shift algorithm. 2.2.2 Empirical prior and variational distribution Now that the generative model has been laid out, we would like to take into account the information provided by the available data points. As mentioned previously, we want to forget part of the data points as time passes, thus, we assum… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: The variational Gaussian mixture: (a) before the optimization process, (b) after 5 steps of optimization, (c) after 7 steps of optimization, and (d) after 10 steps of optimization. 3. Transition Model In this section, we explain how to learn the transition mapping betw…
Figure 5
Figure 5. Figure 5: This figure illustrates the graphical model of (a) a partially observable Markov decision process with variational Gaussian mixture likelihood, and (b) a temporal model based on a categorical-Dirichlet model. 10 [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: This figure illustrates: (a) the partially observable Markov decision process from [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: This figure depicts the transition matrices learnt from 5.000 data points, i.e., each matrix corresponds to a different action. Note, the “eat” action allows the mouse to eat the cheese; if the mouse is not on the maze’s terminal cell, this action is equivalent to idle…
Figure 8
Figure 8. Figure 8: This figure depicts the Gaussian components discovered every 100 training steps. The number associated to each component counts how many times a component has been ob￾served. When a component has been observed five times in a row or more, this component becomes fixed a…
Figure 9
Figure 9. Figure 9: illustrates the data available for a trial composed of seven actions and eight observations. For each observation, the variational Gaussian mixture can be used to compute the associated re￾sponsibilities. An observation xt can be forgotten if the previous xt−1, current…
Figure 10
Figure 10. Figure 10: This figure shows the dataset after removing the forgettable data points in Figure 9. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: This figure illustrates the computation of the updated Q-values. More precisely, the row of temporal difference corresponding to the action selected is multiplied element-wise with the posterior beliefs. The resulting product is multiplied with the learning rate and a…
Figure 12
Figure 12. Figure 12: This figure illustrates the mazes used throughout the experiment section. The starting position is represented by a mouse, while the goal state is represented by the cheese. Red cells correspond to positions for which no Gaussian components were learnt. In contrast, t…
Figure 13
Figure 13. Figure 13: This figure illustrates the average episodic reward gathered by the TGM, DQN and A2C agents on the corresponding mazes in [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AXIOM: Learning to Play Games in Minutes with Expanding Object-Centric Models

    cs.AI 2025-05 conditional novelty 6.0 of 10

    AXIOM, a gradient-free active inference agent with growing and pruning object-centric mixture models, achieves better or similar reward than BBF and DreamerV3 after 10,000 interactions on the custom Gameworld 10k suite.

Reference graph

Works this paper leans on

35 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gaussian mixture variational autoencoder with contrastive learning for multi-label classification

    Junwen Bai, Shufeng Kong, and Carla P Gomes. Gaussian mixture variational autoencoder with contrastive learning for multi-label classification. In international conference on machine learning, pages 1383--1398. PMLR, 2022

    work page 2022

  2. [2]

    Pattern recognition and machine learning, volume 4

    Christopher M Bishop and Nasser M Nasrabadi. Pattern recognition and machine learning, volume 4. Springer, 2006

    2006

  3. [3]

    Variational inference: A review for statisticians

    David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: A review for statisticians. Journal of the American statistical Association, 112 0 (518): 0 859--877, 2017

    2017

  4. [4]

    A survey of Monte Carlo tree search methods

    Cameron B Browne, Edward Powley, Daniel Whitehouse, Simon M Lucas, Peter I Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. A survey of Monte Carlo tree search methods. IEEE Transactions on Computational Intelligence and AI in games, 4 0 (1): 0 1--43, 2012

    2012

  5. [5]

    A review of mean-shift algorithms for clustering

    Miguel A Carreira-Perpin \'a n. A review of mean-shift algorithms for clustering. arXiv preprint arXiv:1503.00687, 2015

    arXiv 2015

  6. [6]

    Learning perception and planning with deep active inference

    Ozan C atal, Tim Verbelen, Johannes Nauta, Cedric De Boom, and Bart Dhoedt. Learning perception and planning with deep active inference. In 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2020, Barcelona, Spain, May 4-8, 2020 , pages 3952--3956. IEEE , 2020. doi:10.1109/ICASSP40776.2020.9054364. URL https://doi.org/10...

    arXiv 2020

  7. [7]

    Realizing Active Inference in Variational Message Passing: The Outcome-Blind Certainty Seeker

    Théophile Champion, Marek Grześ, and Howard Bowman. Realizing Active Inference in Variational Message Passing: The Outcome-Blind Certainty Seeker . Neural Computation, 33 0 (10): 0 2762--2826, 09 2021. ISSN 0899-7667. doi:10.1162/neco_a_01422. URL https://doi.org/10.1162/neco\_a\_01422

    work page doi:10.1162/neco_a_01422 2021

  8. [8]

    Branching time active inference: Empirical study and complexity class analysis

    Théophile Champion, Howard Bowman, and Marek Grześ. Branching time active inference: Empirical study and complexity class analysis. Neural Networks, 152: 0 450--466, 2022 a . ISSN 0893-6080. doi:https://doi.org/10.1016/j.neunet.2022.05.010. URL https://www.sciencedirect.com/science/article/pii/S0893608022001824

    work page doi:10.1016/j.neunet.2022.05.010 2022

Show all 35 references
  1. [9]

    Branching time active inference: The theory and its generality

    Théophile Champion, Lancelot Da Costa , Howard Bowman, and Marek Grześ. Branching time active inference: The theory and its generality. Neural Networks, 151: 0 295--316, 2022 b . ISSN 0893-6080. doi:https://doi.org/10.1016/j.neunet.2022.03.036. URL https://www.sciencedirect.co...

    2022 doi

  2. [10]

    Multi-modal and multi-factor branching time active inference, 2022 c

    Théophile Champion, Marek Grześ, and Howard Bowman. Multi-modal and multi-factor branching time active inference, 2022 c . URL https://arxiv.org/abs/2206.12503

    2022 arXiv

  3. [11]

    Branching Time Active Inference with Bayesian Filtering

    Théophile Champion, Marek Grześ, and Howard Bowman. Branching Time Active Inference with Bayesian Filtering . Neural Computation, 34 0 (10): 0 2132--2144, 09 2022 d . ISSN 0899-7667. doi:10.1162/neco_a_01529. URL https://doi.org/10.1162/neco\_a\_01529

    2022 doi

  4. [12]

    Deconstructing deep active inference, 2023

    Théophile Champion, Marek Grześ, Lisa Bonheme, and Howard Bowman. Deconstructing deep active inference, 2023. URL https://arxiv.org/abs/2303.01618

    2023 arXiv

  5. [13]

    Deep unsupervised clustering with gaussian mixture variational autoencoders

    Nat Dilokthanakul, Pedro AM Mediano, Marta Garnelo, Matthew CH Lee, Hugh Salimbeni, Kai Arulkumaran, and Murray Shanahan. Deep unsupervised clustering with gaussian mixture variational autoencoders. arXiv preprint arXiv:1611.02648, 2016

    2016 arXiv

  6. [14]

    Tutorial on variational autoencoders, 2016

    Carl Doersch. Tutorial on variational autoencoders, 2016. URL https://arxiv.org/abs/1606.05908

    2016 arXiv

  7. [15]

    Zafeirios Fountas, Noor Sajid, Pedro A. M. Mediano, and Karl J. Friston. Deep active inference agents using Monte-Carlo methods. In Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria - Florina Balcan, and Hsuan - Tien Lin, editors, Advances in Neural Information Proces...

    2020

  8. [16]

    Fox and Stephen J

    Charles W. Fox and Stephen J. Roberts. A tutorial on variational Bayesian inference. Artificial Intelligence Review, 38 0 (2): 0 85--95, Aug 2012. ISSN 1573-7462. doi:10.1007/s10462-011-9236-8. URL https://doi.org/10.1007/s10462-011-9236-8

    2012 doi

  9. [17]

    Bayesian model reduction

    Karl Friston, Thomas Parr, and Peter Zeidman. Bayesian model reduction. arXiv preprint arXiv:1805.07092, 2018

    2018 arXiv

  10. [18]

    beta- VAE : Learning basic visual concepts with a constrained variational framework

    Irina Higgins, Lo \" c Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta- VAE : Learning basic visual concepts with a constrained variational framework. In 5th International Conference on Learning Representat...

    2017

  11. [19]

    An exponential family of probability distributions for directed graphs

    Paul W Holland and Samuel Leinhardt. An exponential family of probability distributions for directed graphs. Journal of the american Statistical association, 76 0 (373): 0 33--50, 1981

    1981

  12. [20]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-Encoding Variational Bayes . In International Conference on Learning Representations, volume 2, Banff, Canada, 2014. URL http://arxiv.org/abs/1312.6114

    2014 arXiv

  13. [21]

    On information and sufficiency

    Solomon Kullback and Richard A Leibler. On information and sufficiency. The annals of mathematical statistics, 22 0 (1): 0 79--86, 1951

    1951

  14. [22]

    Robot self/other distinction: active inference meets neural networks learning in a mirror

    Pablo Lanillos, Gordon Cheng, et al. Robot self/other distinction: active inference meets neural networks learning in a mirror. arXiv preprint arXiv:2004.05473, 2020

    2004 arXiv

  15. [23]

    The stability-plasticity dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effects

    Martial Mermillod, Aurélia Bugaiska, and Patrick BONIN. The stability-plasticity dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effects. Frontiers in Psychology, 4, 2013. ISSN 1664-1078. doi:10.3389/fpsyg.2013.00504. URL https://www.f...

    2013

  16. [24]

    Deep active inference as variational policy gradients

    Beren Millidge. Deep active inference as variational policy gradients. Journal of Mathematical Psychology, 96: 0 102348, 2020. ISSN 0022-2496. doi:https://doi.org/10.1016/j.jmp.2020.102348. URL http://www.sciencedirect.com/science/article/pii/S0022249620300298

    2020

  17. [25]

    Rusu, Joel Veness, Marc G

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, ...

    2015 doi

  18. [26]

    Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu

    Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning, 2016

    2016

  19. [27]

    Structure learning enhances concept formation in synthetic active inference agents

    Victorita Neacsu, M Berk Mirza, Rick A Adams, and Karl J Friston. Structure learning enhances concept formation in synthetic active inference agents. Plos one, 17 0 (11): 0 e0277199, 2022

    2022

  20. [28]

    Active inference body perception and action for humanoid robots

    Guillermo Oliver, Pablo Lanillos, and Gordon Cheng. Active inference body perception and action for humanoid robots. arXiv preprint arXiv:1906.03022, 2019

    1906 arXiv

  21. [29]

    The infinite Gaussian mixture model

    Carl Rasmussen. The infinite Gaussian mixture model. Advances in neural information processing systems, 12, 1999

    1999

  22. [30]

    Stochastic Backpropagation and Approximate Inference in Deep Generative Models

    Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic Backpropagation and Approximate Inference in Deep Generative Models . In Eric P Xing and Tony Jebara, editors, Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings o...

    2014

  23. [31]

    Cansu Sancaktar, Marcel A. J. van Gerven, and Pablo Lanillos. End-to-end pixel-based deep active inference for body perception and action. In Joint IEEE 10th International Conference on Development and Learning and Epigenetic Robotics, ICDL-EpiRob 2020, Valparaiso, Chile, Octo...

    2020

  24. [32]

    Ryan Smith, Philipp Schwartenbeck, Thomas Parr, and Karl J. Friston. An active inference approach to modeling structure learning: Concept learning as an example case. Frontiers in Computational Neuroscience, 14, 2020. ISSN 1662-5188. doi:10.3389/fncom.2020.00041. URL https://w...

    2020

  25. [33]

    Reinforcement learning: An introduction

    Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018

    2018

  26. [34]

    Deep active inference for partially observable MDPs

    Otto van der Himst and Pablo Lanillos. Deep active inference for partially observable MDPs . CoRR, abs/2009.03622, 2020. URL https://arxiv.org/abs/2009.03622

    2009 arXiv

  27. [35]

    Variational message passing

    John Winn and Christopher Bishop. Variational message passing. Journal of Machine Learning Research, 6: 0 661--694, 2005

    2005

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.