Pith. sign in

REVIEW 4 major objections 5 minor 64 references

Deep Active Inference Agents for Delayed and Long-Horizon Environments

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An active-inference agent replaces exhaustive planning with one gradient step per horizon.

desk verdict A genuinely new AIF architecture with a policy-conditional latent transition, but overclaims on handcrafted rewards and leaves the policy representation under-justified. read the letter →

arxiv 2505.19867 v1 pith:L6WG5P3Z submitted 2025-05-26 cs.LG cs.AI

Yavar Taheri Yeganeh , Mohsen Jafari , Andrea Matta This is my paper · ORCID
classification cs.LGcs.AI
keywords activeinferenceexpectedfreeenergyworldmodelsgenerativepolicymulti-steplatenttransitiondelayedenvironmentslong-horizonplanningenergy-efficientmanufacturingcontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that an active-inference agent can act effectively in delayed, long-horizon tasks without tree search or per-step planning. The agent's generative model uses a multi-step latent transition $P_{\theta_s}(s_{t+H} \mid \tilde{s}_t, \hat{\pi}(\phi_a))$ to predict an entire horizon in a single look-ahead, conditioned on the policy network itself rather than on sampled action sequences. Gradients of the expected free energy with respect to the policy parameters $\phi_a$ are computed from that prediction, so one gradient update every $H$ steps is the whole planning loop. This matters because practical active-inference agents have relied on accurate immediate predictions and exhaustive planning, which breaks down when consequences of actions appear only after tens to hundreds of steps, as in industrial control. On a simulated automotive workstation with stochastic arrivals, failures, and repairs, the paper reports that the agent improves energy efficiency per production unit by $10.21\% \pm 0.14\%$ over a model-free DQN baseline while keeping production loss low.

What carries the argument

The load-bearing object is the multi-step latent transition $P_{\theta_s}(s_{t+H} \mid \tilde{s}_t, \hat{\pi}(\phi_a))$, a neural network that maps the current latent state together with an embedding of the actor's parameters to a distribution over the latent state $H$ steps later. The actor embedding $\hat{\pi}(\phi_a) = \Pi(Q_{\phi_a})$ is formed by concatenating the observation input, the first hidden layer, and the output values of the policy network, a simplified neural implicit representation of the policy function. Because the transition is conditioned on the policy rather than on a sampled action sequence, the expected free energy of the predicted observation is a differentiable function of $\phi_a$, and backpropagating that gradient is the planning step. An encoder $Q_{\phi_s}$ and decoder $P_{\theta_o}$ provide latent states and observation predictions, the variational free energy trains the generative model, and a preference mapping $\Psi: P_{\theta}(o_\tau) \to [0,1]$ converts predictions into the continuous preference signal used by the extrinsic value term.

What would settle it

Re-train the agent on the same workstation but change a system parameter after training, such as raising the arrival rate $\lambda$ or shrinking the buffer capacity $K$; then compare one-gradient-step-per-$H$ planning against a planner that re-evaluates the expected free energy at every decision. If the single-step updates fail to track the moved optimum while re-planning succeeds, the multi-step transition is not carrying the planning signal the paper claims.

Watch

Extended reading notes

Core claim

The central claim is that the world model can absorb the policy: with actor parameters $\phi_a$ embedded as $\hat{\pi}(\phi_a)$, the transition $P_{\theta_s}(s_{t+H} \mid \tilde{s}_t, \hat{\pi}(\phi_a))$ predicts the latent state $H$ steps ahead in one step, and the decoder $P_{\theta_o}(o_{t+H} \mid \tilde{s}_{t+H})$ turns that latent state into a future observation. The expected free energy $G_{\theta}(\tilde{o}, \phi_a)$ of the predicted observation is then differentiated with respect to $\phi_a$, and a single gradient update is taken every $H$ steps; no Monte Carlo tree search and no sequential latent rollouts enter the control loop. In the industrial test, the trained agent is evaluated over one month of simulated interaction on ten independently initialized environments, with gradient updates every $H$ steps during planning. The paper reports higher preference scores and falling expected free energy during training, and a final result in which the DAIF agent reaches $10.21\% \pm 0.14\%$ higher energy efficiency per production unit than the model-free DQN baseline while keeping production loss lower than the nearest DQN variants. The paper also reports that lengthening the overshooting horizon from $H=300$ to $H=1000$ does not break the learned policy, which supports the claim that the single multi-step transition, rather than deeper search, carries the long-horizon planning.

Load-bearing premise

The argument rests on the learned $H$-step transition being accurate enough that gradients of the expected free energy with respect to the simplified policy embedding point the policy in a useful direction; if that relationship is biased or too coarse, a single gradient step can steer the policy the wrong way.

Editorial extensions

If this is right

  • Active-inference agents no longer need a tree search or per-step rollouts to handle delayed feedback: a single EFE gradient every $H$ steps is claimed to be sufficient for control after training.
  • Because the policy is optimized by gradient descent on the expected free energy, the architecture scales to continuous action spaces that discrete action-selection AIF implementations do not handle.
  • The same generative-policy loop can be applied where handcrafted rewards are expensive, since the agent is driven by a preference mapping over raw window-based performance observations rather than a cumulative reward function.
  • The world model can keep improving after the policy has stabilized, suggesting that full reconstruction of future observations is not required for control and that effort can be redirected to policy-relevant prediction.
  • Performance is reported to remain robust when the overshooting horizon is extended from 300 to 1000 steps, indicating that the method degrades gracefully with the horizon length.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: the comparison is to a single DQN baseline family on one industrial scenario, so 'surpasses model-free RL' is a claim about this benchmark and should not be read as a general result.
  • Editorial: the actor embedding, a concatenation of input, first hidden layer, and outputs, is a coarse stand-in for the policy function; if a richer embedding or an aggregator over time steps is used, the single-look-ahead transition could be trained more cheaply, but the paper only sketches this direction.
  • Editorial: a natural testable extension is to measure how the EFE gradient direction tracks true performance as $H$ grows and as transition error accumulates; the paper reports performance at several horizons but does not dissect this relationship.
  • Editorial: the authors note that recurrent or set-based actor representations could replace the simplified concatenation, which would allow the horizon to be segmented down to single steps while preserving EFE gradient flow; this is a forward direction rather than an established result.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes Deep Active Inference (DAIF), a generative-policy architecture in which a multi-step latent transition predicts the observation H steps ahead, conditioned on a representation of the actor network, and the actor parameters are updated by gradients of the expected free energy. The agent is evaluated on a stochastic discrete-event simulation of an energy-efficient control problem for parallel identical machines, and the results are compared with a DQN baseline taken from a prior paper. The paper claims that the approach eliminates exhaustive planning from the control loop with a single gradient update every H steps, and that it achieves effective decision making 'without handcrafted rewards.' The architecture consists of an encoder, a transition conditioned on a pointwise actor representation, a decoder, and an alternating training scheme that updates the generative model via VFE and the policy via EFE gradients.

Significance. If the approach holds up, it would offer a practical way to apply active inference to long-horizon, delayed-feedback control, where per-step planning is infeasible. The paper includes a public code repository and uses a realistic, validated industrial environment, which are strengths. The central idea of conditioning a multi-step transition on the policy itself and optimizing the policy through EFE gradients is novel and could be of interest to the world-model and active-inference communities. However, the significance is currently constrained by the lack of a rigorous justification for the pointwise actor representation, the absence of model-based or prior-AIF baselines, and a baseline comparison that uses a different reward formulation.

major comments (4)
  1. [§3.1, §4.2, Appendix B.1] The H-step transition Pθs(s_{t+H}|s_t, π̂(ϕa)) is conditioned on π̂ = Π(Qϕa), which Appendix B.1 defines as the concatenation of a single observation, the first hidden-layer values, and the output values of the actor network. This representation does not determine the policy's behavior at states visited later within the horizon: distinct parameter settings that agree at one observation can produce different action distributions at subsequent observations, so identical conditioning inputs can correspond to different H-step outcome distributions. No injectivity, Lipschitz, or identifiability argument is provided for the claim in Section 4.2 that this mapping 'effectively serves as an approximation of the actor's neural function.' Because the EFE gradient in Section 3.2 and the policy update in Algorithm 1 backpropagate through this transition, the resulting gradient can be optimal only for an averaged policy rather than the actual one. The Limitations section itself notes that an operator aggregating the sequence of actor representations is needed, corroborating the insufficiency of the pointwise representation.
  2. [§4.2, Table 1] The comparison with DQN is not apples-to-apples. The DQN results are taken from Loffredo et al. [57], which uses a different reward function (linear combination with parameter ϕ, Eq. 7) and possibly a different evaluation protocol, while the DAIF agent uses a different preference function (Eq. 8) and window-based performance metric. The reported '10.21% ± 0.14%' improvement in energy efficiency per production unit is not derivable from Table 1: DAIF's EN Saving of 12.49% versus the best DQN row of 10.87% gives a relative improvement of about 14.9%, and other definitions of efficiency yield different values. The paper should state the exact formula for the 10.21% figure, report its standard deviation across the 10 replications, and ideally re-train DQN under the same preference function and evaluation protocol. Additionally, no model-based baseline (e.g., Dreamer, PlaNet, or the AIF agent of Fountas et al. [12]) is included, so the contribution of the proposed architecture relative to existing world-model methods is not isolated.
  3. [Abstract, §3.2, §4.1, Appendix A.3] The abstract and Section 4.1 claim the agent works 'without handcrafted rewards' and 'operating directly on raw performance signals rather than handcrafted rewards.' However, Appendix A.3 defines a handcrafted preference mapping R = R_production · σ(c_r · R_energy) with a tunable scalar c_r, and Section 3.2 states that Ψ is an 'analytical mapping' that transforms the prediction distribution into a preference. This preference is used in the extrinsic-value term (Eq. 4a) and is part of the observation, so it functions as a reward-like signal. The claim of no handcrafted rewards is therefore not supported; the paper should either qualify the claim or show that the preference can be learned or derived.
  4. [Algorithm 1, Appendix B.1] Algorithm 1 is underspecified in several places that are essential for reproducibility. Line 13 computes DKL(Qϕs(s_{t'+H}) || N(μ, σ²)) using the transition prediction, while line 15 adds β·DKL(Qϕs(s_{t'+H}) || N(μ̃, σ̃²)) without defining μ̃ and σ̃; line 17 updates 'ϕo' where the context indicates 'ϕs'; line 2 'Randomly initialize E' is unclear since E denotes the environment. In the policy-optimization loop, line 24 conditions the transition on π̂t, but it is not specified whether π̂t is the stored representation from the replay tuple or is recomputed at the current observation with the current actor parameters. These ambiguities prevent exact reproduction of the training procedure from the paper alone, despite the availability of code.
minor comments (5)
  1. [Figure 4] Please report the number of seeds used to compute the 95% confidence intervals and state whether the same random seeds were used across the different horizon values H.
  2. [§4.2] Please clarify whether Table 1 reports results from the single best-performing validation instance or averaged over the 10 test replications, since Section C.2 describes both a selection step and an averaging step.
  3. [Appendix B.1.1] The approximation of continuous preference outputs as Bernoulli-like parameters for the entropy computation should be empirically or analytically compared against a continuous-entropy alternative, since the EFE estimate in Eq. (4b)–(4c) depends on these entropies.
  4. [Eq. (10)] The reconstruction-loss weights (2/7, 1/7, 4/7) in Eq. (10) are introduced without motivation; a brief justification or a sensitivity analysis would improve reproducibility.
  5. [Algorithm 1, line 17] There is a typo in Algorithm 1: line 17 updates 'ϕo' instead of 'ϕs'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the EFE formalism is adopted from external prior work and the empirical claims are checked against a simulator with separate validation.

full rationale

The paper's derivation chain is not circular. The EFE equations (Eqs. 2-4) are taken from Fountas et al. [12], an external source, and are used as a modeling framework rather than re-derived from the paper's own outputs. The multi-step transition and window-based preference metric build on the authors' prior work [11], but the central contribution—the integrated policy representation Π and EFE-gradient policy update—is evaluated empirically against a separate DQN baseline in a high-fidelity simulator, with validation on independently initialized environments. The handcrafted preference function Ψ (Appendix A.3) is a design choice analogous to reward shaping; it defines the control objective and is not a fitted parameter disguised as a prediction. No uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via self-citation. The skeptic's concern about the under-specified actor representation (concatenating observation, hidden layer, and output) is a modeling limitation that the authors explicitly acknowledge in the Limitations section; it affects correctness and identifiability, but it does not make any prediction equivalent to its inputs by construction. Therefore, no circular step is present.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the assumed active-inference/EFE formalism, the DTMC/Poisson environment model, and a VAE-style architecture. The free parameters are mostly hand-chosen hyperparameters, especially the preference sensitivity c_r, which effectively defines the reward the agent optimizes.

free parameters (6)
  • c_r (preference sensitivity) = not explicitly given
    Eq. 8: sensitivity of the preference score to energy savings; determines the trade-off the agent optimizes.
  • β (KL regularization weight) = not specified
    Eq. 10: β-VAE regularization, chosen by hand; affects latent space structure.
  • λ_s (variance scale) = 1.5
    Appendix B.1.2: scales the sigmoid output for variances; set to stay in non-saturated domain.
  • Horizon H = 300 (main), 200-1000 tested
    Overshooting horizon; central to the single-lookahead claim; results vary with H.
  • Reconstruction loss weights = 2/7, 1/7, 4/7
    Eq. 10: manual weighting of buffer, machine state, and preference reconstruction terms.
  • Learning rates ξ, γ, η, α = not reported
    Algorithm 1 lists separate learning rates for model and actor but the values are not given in the paper.
assumptions (3)
  • domain assumption The EFE decomposition in Eq. 4 from Fountas et al. (2020) is assumed valid for this generative model.
    Section 2.1: The paper adopts the EFE formulation without re-derivation.
  • domain assumption The industrial workstation is accurately modeled as a DTMC with Poisson arrival, processing, failure, and repair processes.
    Appendix A.2: parameters and stochastic models taken from Loffredo et al. [55].
  • domain assumption A VAE-style encoder/decoder with diagonal Gaussian latent states and Bernoulli observation likelihoods can represent the relevant dynamics.
    Section 3.1: the generative model is instantiated as an autoencoder; this is an architectural choice, not derived.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deep Active Inference Agents for Delayed and Long-Horizon Environments." pith.science (2026). https://pith.science/paper/L6WG5P3Z

@misc{pith2026250519867,
  author       = {Pith},
  title        = {Pith review of: Deep Active Inference Agents for Delayed and Long-Horizon Environments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L6WG5P3Z}},
  note         = {Machine review of arXiv:2505.19867}
}
read the original abstract

With the recent success of world-model agents, which extend the core idea of model-based reinforcement learning by learning a differentiable model for sample-efficient control across diverse tasks, active inference (AIF) offers a complementary, neuroscience-grounded paradigm that unifies perception, learning, and action within a single probabilistic framework powered by a generative model. Despite this promise, practical AIF agents still rely on accurate immediate predictions and exhaustive planning, a limitation that is exacerbated in delayed environments requiring plans over long horizons, tens to hundreds of steps. Moreover, most existing agents are evaluated on robotic or vision benchmarks which, while natural for biological agents, fall short of real-world industrial complexity. We address these limitations with a generative-policy architecture featuring (i) a multi-step latent transition that lets the generative model predict an entire horizon in a single look-ahead, (ii) an integrated policy network that enables the transition and receives gradients of the expected free energy, (iii) an alternating optimization scheme that updates model and policy from a replay buffer, and (iv) a single gradient step that plans over long horizons, eliminating exhaustive planning from the control loop. We evaluate our agent in an environment that mimics a realistic industrial scenario with delayed and long-horizon settings. The empirical results confirm the effectiveness of the proposed approach, demonstrating the coupled world-model with the AIF formalism yields an end-to-end probabilistic controller capable of effective decision making in delayed, long-horizon settings without handcrafted rewards or expensive planning.

Figures

Figures reproduced from arXiv: 2505.19867 by the authors.

Figure 1
Figure 1. Two perspectives of the AIF framework: general steps (left) and core elements (right). [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The Deep AIF agent architecture illustrates its interaction with the environment. The actor [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. The performance of the agent with H = 300 on the real industrial system. 9 [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Performance of the agents versus overshooting horizon [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Layout of parallel, identical machines in the workstation [55]. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 34 canonical work pages

  1. [57]

    Reinforcement learning for sustainability enhancement of production lines

    Alberto Loffredo, Marvin Carl May, Andrea Matta, and Gisela Lanza. Reinforcement learning for sustainability enhancement of production lines. Journal of Intelligent Manufacturing, pages 1–17, 2023

    work page 2023

  2. [12]

    Fountas, Noor Sajid, Pedro A

    Z. Fountas, Noor Sajid, Pedro A. M. Mediano, and Karl J. Friston. Deep active inference agents using monte-carlo methods. ArXiv, abs/2006.04176, 2020. 11

    arXiv 2006

  3. [1]

    Reinforcement learning: An introduction

    Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018

    2018

  4. [2]

    Deep learning.Nature, 521(7553):436–444, 2015

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning.Nature, 521(7553):436–444, 2015

    2015

  5. [3]

    Bishop and Hugh Bishop

    Christopher M. Bishop and Hugh Bishop. Deep Learning: Foundations and Concepts. Springer International Publishing, 2024

    work page 2024

  6. [4]

    Human-level control through deep reinforcement learning

    V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature, 518(7540):529–533, 2015

    2015

  7. [5]

    Moerland, Joost Broekens, Aske Plaat, and Catholijn M

    Thomas M. Moerland, Joost Broekens, Aske Plaat, and Catholijn M. Jonker. Model-based reinforcement learning: A survey. Foundations and Trends® in Machine Learning, 16(1):1–118, 2023

    2023

  8. [6]

    Mastering diverse control tasks through world models

    Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse control tasks through world models. Nature, 640:647–653, 2025

    work page 2025

Show all 64 references
  1. [7]

    Moran, Yukie Nagai, Tadahiro Taniguchi, Hiroaki Gomi, and Joshua B

    Karl Friston, Rosalyn J. Moran, Yukie Nagai, Tadahiro Taniguchi, Hiroaki Gomi, and Joshua B. Tenenbaum. World model learning and inference. Neural Networks, 144:573–590, 2021

    2021

  2. [8]

    Active inference: a process theory

    Karl Friston, Thomas FitzGerald, Francesco Rigoli, Philipp Schwartenbeck, and Giovanni Pezzulo. Active inference: a process theory. Neural computation, 29(1):1–49, 2017

    2017

  3. [9]

    Active inference: the free energy principle in mind, brain, and behavior

    Thomas Parr, Giovanni Pezzulo, and Karl J Friston. Active inference: the free energy principle in mind, brain, and behavior. MIT Press, 2022

    2022

  4. [10]

    The free-energy principle: a unified brain theory? Nature reviews neuroscience, 11(2):127–138, 2010

    Karl Friston. The free-energy principle: a unified brain theory? Nature reviews neuroscience, 11(2):127–138, 2010

    2010

  5. [11]

    Active inference meeting energy- efficient control of parallel and identical machines

    Yavar Taheri Yeganeh, Mohsen Jafari, and Andrea Matta. Active inference meeting energy- efficient control of parallel and identical machines. In International Conference on Machine Learning, Optimization, and Data Science, pages 479–493. Springer, 2024

    2024

  6. [13]

    Dream to control: Learning behaviors by latent imagination

    Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination. In International Conference on Learning Representa- tions, 2020

    2020

  7. [14]

    R-aif: Solving sparse-reward robotic tasks from pixels with active inference and world models

    Viet Dung Nguyen, Zhizhuo Yang, Christopher L Buckley, and Alexander Ororbia. R-aif: Solving sparse-reward robotic tasks from pixels with active inference and world models. arXiv preprint arXiv:2409.14216, 2024

    2024 arXiv

  8. [15]

    Model-based reinforcement learning for biological sequence design

    Christof Angermueller, David Dohan, David Belanger, Ramya Deshpande, Kevin Murphy, and Lucy Colwell. Model-based reinforcement learning for biological sequence design. In International conference on learning representations, 2019

    2019

  9. [16]

    Fine-tuning discrete diffusion models via reward optimization with applications to dna and protein design

    Chenyu Wang, Masatoshi Uehara, Yichun He, Amy Wang, Tommaso Biancalani, Avantika Lal, Tommi Jaakkola, Sergey Levine, Hanchen Wang, and Aviv Regev. Fine-tuning discrete diffusion models via reward optimization with applications to dna and protein design. arXiv preprint arXiv:24...

    arXiv

  10. [17]

    Deep Generative Modeling

    Jakub M Tomczak. Deep Generative Modeling. Springer Cham, 2024

    2024

  11. [18]

    Ballard, Joshua Bambrick, Sebastian W

    Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J. Ballard, Joshua Bambrick, Sebastian W. Bodenstein, David A. Evans, Chia Chun Hung, Michael O’Neill, David Reiman, Kathryn Tunyasuvunakool, Zachar...

    2024

  12. [19]

    World models

    David Ha and Jürgen Schmidhuber. World models. arXiv preprint arXiv:1803.10122, 2018

    2018 arXiv

  13. [20]

    Auto-encoding variational bayes

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  14. [21]

    Learning latent dynamics for planning from pixels

    Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. InInternational conference on machine learning, pages 2555–2565. PMLR, 2019

    2019

  15. [22]

    Predictive coding: towards a future of deep learning beyond backpropagation? arXiv preprint arXiv:2202.09467, 2022

    Beren Millidge, Tommaso Salvatori, Yuhang Song, Rafal Bogacz, and Thomas Lukasiewicz. Predictive coding: towards a future of deep learning beyond backpropagation? arXiv preprint arXiv:2202.09467, 2022

    2022 arXiv

  16. [23]

    Action and behavior: A free-energy formulation

    Karl Friston, Francesco Rigoli, Dimitri Ognibene, Christoph Mathys, Thomas Fitzgerald, and Giovanni Pezzulo. Action and behavior: A free-energy formulation. Biological Cybernetics, 102(3):227–260, 2010

    2010

  17. [24]

    Applications of the free energy principle to machine learning and neuroscience

    Beren Millidge. Applications of the free energy principle to machine learning and neuroscience. arXiv preprint arXiv:2107.00140, 2021

    2021 arXiv

  18. [25]

    Experimental validation of the free-energy principle with in vitro neural networks

    Takuya Isomura, Kiyoshi Kotani, Yasuhiko Jimbo, and Karl J Friston. Experimental validation of the free-energy principle with in vitro neural networks. Nature Communications, 14(1):4547, 2023

    2023

  19. [26]

    Active inference and behavior trees for reactive action planning and execution in robotics

    Corrado Pezzato, Carlos Hernández Corbato, Stefan Bonhof, and Martijn Wisse. Active inference and behavior trees for reactive action planning and execution in robotics. IEEE Transactions on Robotics, 39(2):1050–1069, 2023

    2023

  20. [27]

    Active exploration for robotic manipulation

    Tim Schneider, Boris Belousov, Georgia Chalvatzaki, Diego Romeres, Devesh K Jha, and Jan Peters. Active exploration for robotic manipulation. In2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 9355–9362. IEEE, 2022. 12

    2022

  21. [28]

    Navigating autonomous vehicle on unmarked roads with diffusion-based motion prediction and active inference

    Yufei Huang, Yulin Li, Andrea Matta, and Mohsen Jafari. Navigating autonomous vehicle on unmarked roads with diffusion-based motion prediction and active inference. arXiv preprint arXiv:2406.00211, 2024

    2024 arXiv

  22. [29]

    How active inference could help revolutionise robotics

    Lancelot Da Costa, Pablo Lanillos, Noor Sajid, Karl Friston, and Shujhat Khan. How active inference could help revolutionise robotics. Entropy, 24(3):361, 2022

    2022

  23. [30]

    Reward maximiza- tion through discrete active inference

    Lancelot Da Costa, Noor Sajid, Thomas Parr, Karl Friston, and Ryan Smith. Reward maximiza- tion through discrete active inference. Neural Computation, 35(5):807–852, 2023

    2023

  24. [31]

    Planning and acting in partially observable stochastic domains

    Leslie Pack Kaelbling, Michael L Littman, and Anthony R Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1-2):99–134, 1998

    1998

  25. [32]

    On efficient computation in active inference

    Aswin Paul, Noor Sajid, Lancelot Da Costa, and Adeel Razi. On efficient computation in active inference. arXiv preprint arXiv:2307.00504, 2023

    2023 arXiv

  26. [33]

    Variational inference: A review for statisticians

    David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518):859–877, 2017

    2017

  27. [34]

    Bayesian brains and the rényi divergence

    Noor Sajid, Francesco Faccio, Lancelot Da Costa, Thomas Parr, Jürgen Schmidhuber, and Karl Friston. Bayesian brains and the rényi divergence. Neural Computation, 34(4):829–855, 2022

    2022

  28. [35]

    Computational mechanisms of curiosity and goal-directed exploration

    Philipp Schwartenbeck, Johannes Passecker, Tobias U Hauser, Thomas HB FitzGerald, Martin Kronbichler, and Karl J Friston. Computational mechanisms of curiosity and goal-directed exploration. elife, 8:e41703, 2019

    2019

  29. [36]

    Curiosity-driven exploration by self-supervised prediction

    Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In International conference on machine learning, pages 2778–

  30. [37]

    Exploration by random network distillation

    Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation. arXiv preprint arXiv:1810.12894, 2018

    2018 arXiv

  31. [38]

    Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pages 1861–1870. Pmlr, 2018

    2018

  32. [39]

    Efficient selectivity and backup operators in monte-carlo tree search

    Rémi Coulom. Efficient selectivity and backup operators in monte-carlo tree search. In International conference on computers and games, pages 72–83. Springer, 2006

    2006

  33. [40]

    Mastering the game of go without human knowledge

    David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature, 550(7676):354–359, 2017

    2017

  34. [41]

    Mastering atari, go, chess and shogi by planning with a learned model

    Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Si- mon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, et al. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 588(7839):604–609, 2020

    2020

  35. [42]

    Probabilistic planning with sequential monte carlo methods

    Alexandre Piché, Valentin Thomas, Cyril Ibrahim, Yoshua Bengio, and Chris Pal. Probabilistic planning with sequential monte carlo methods. In International Conference on Learning Representations, 2018

    2018

  36. [43]

    Iterative amortized inference

    Joe Marino, Yisong Yue, and Stephan Mandt. Iterative amortized inference. In International Conference on Machine Learning, pages 3403–3412. PMLR, 2018

    2018

  37. [44]

    Control as hybrid inference

    Alexander Tschantz, Beren Millidge, Anil K Seth, and Christopher L Buckley. Control as hybrid inference. arXiv preprint arXiv:2007.05838, 2020

    2007 arXiv

  38. [45]

    Information processing in decision-making systems

    Matthijs Van Der Meer, Zeb Kurth-Nelson, and A David Redish. Information processing in decision-making systems. The Neuroscientist, 18(4):342–359, 2012

    2012

  39. [46]

    Mastering atari games with limited data

    Weirui Ye, Shaohuai Liu, Thanard Kurutach, Pieter Abbeel, and Yang Gao. Mastering atari games with limited data. Advances in neural information processing systems, 34:25476–25488, 2021. 13

    2021

  40. [47]

    Amortized inference in probabilistic reasoning

    Samuel Gershman and Noah Goodman. Amortized inference in probabilistic reasoning. In Proceedings of the annual meeting of the cognitive science society, volume 36, 2014

    2014

  41. [48]

    Amortized variational inference: When and why? arXiv preprint arXiv:2307.11018, 2023

    Charles C Margossian and David M Blei. Amortized variational inference: When and why? arXiv preprint arXiv:2307.11018, 2023

    2023 arXiv

  42. [49]

    From data to functa: Your data point is a function and you can treat it like one

    Emilien Dupont, Hyunjik Kim, SM Eslami, Danilo Rezende, and Dan Rosenbaum. From data to functa: Your data point is a function and you can treat it like one. arXiv preprint arXiv:2201.12204, 2022

    2022 arXiv

  43. [50]

    Graph neural networks for learning equivariant representations of neural networks

    Miltiadis Kofinas, Boris Knyazev, Yan Zhang, Yunlu Chen, Gertjan J Burghouts, Efstratios Gavves, Cees GM Snoek, and David W Zhang. Graph neural networks for learning equivariant representations of neural networks. arXiv preprint arXiv:2403.12143, 2024

    2024 arXiv

  44. [51]

    Dropout as a bayesian approximation: Representing model uncertainty in deep learning

    Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. Ininternational conference on machine learning, pages 1050–1059. PMLR, 2016

    2016

  45. [52]

    beta-vae: Learning basic visual concepts with a constrained variational framework

    Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. In International conference on learning representations, 2016

    2016

  46. [53]

    The animal-ai olympics

    Matthew Crosby, Benjamin Beyret, and Marta Halina. The animal-ai olympics. Nature Machine Intelligence, 1(5):257–257, 2019

    2019

  47. [54]

    Active inference in robotics and artificial agents: Survey and challenges

    Pablo Lanillos, Cristian Meo, Corrado Pezzato, Ajith Anil Meera, Mohamed Baioumy, Wataru Ohata, Alexander Tschantz, Beren Millidge, Martijn Wisse, Christopher L Buckley, et al. Active inference in robotics and artificial agents: Survey and challenges. arXiv preprint arXiv:2112...

    2021 arXiv

  48. [55]

    Reinforce- ment learning for energy-efficient control of parallel and identical machines

    Alberto Loffredo, Marvin Carl May, Louis Schäfer, Andrea Matta, and Gisela Lanza. Reinforce- ment learning for energy-efficient control of parallel and identical machines. CIRP Journal of Manufacturing Science and Technology, 44:91–103, 2023

    2023

  49. [56]

    Energy-efficient control in multi-stage production lines with parallel machine workstations and production constraints

    Alberto Loffredo, Nicla Frigerio, Ettore Lanzarone, and Andrea Matta. Energy-efficient control in multi-stage production lines with parallel machine workstations and production constraints. IISE Transactions, 56(1):69–83, 2024

    2024

  50. [58]

    Introduction to probability models

    Sheldon M Ross. Introduction to probability models. Academic press, 2014

    2014

  51. [59]

    Deep sets

    Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep sets. Advances in neural information processing systems, 30, 2017

    2017

  52. [60]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  53. [61]

    Fourier neural operator for parametric partial differen- tial equations

    Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differen- tial equations. arXiv preprint arXiv:2010.08895, 2020

    2010 arXiv

  54. [62]

    Learning nonlinear operators via deeponet based on the universal approximation theorem of operators

    Lu Lu, Pengzhan Jin, Giovanni Pang, Zhiping Zhang, and George Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218–229, 2021

    2021

  55. [63]

    A literature review of energy efficiency and sustainability in manufacturing systems

    Paolo Renna and Sergio Materi. A literature review of energy efficiency and sustainability in manufacturing systems. Applied Sciences, 11(16):7366, 2021

    2021

  56. [64]

    Poisson processes, volume 3

    John Frank Charles Kingman. Poisson processes, volume 3. Clarendon Press, 1992. 14 A Application A.1 Energy-Efficiency Control Energy-Efficiency Control (EEC) is attracting growing attention in both academia and industrial research within manufacturing systems. Acting at the c...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.