Pith. sign in

REVIEW 1 major objections 6 minor 29 references

Sophisticated Policies from Epistemic Priors

T0 review · 1 major / 6 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that the advantage of Sophisticated Inference comes from closed-loop posterior dependencies between future states and actions, not from tree search itself.

desk verdict A plausible, useful argument that Sophisticated Inference's advantage comes from closed-loop control plus epistemic drive, but the key ablation changes the objective it claims to hold fixed. read the letter →

arxiv 2607.19518 v1 pith:SCPERKI3 submitted 2026-07-21 cs.AI

Wouter W. L. Nuijten , Bert de Vries This is my paper · ORCID
classification cs.AI
keywords activeinferenceexpectedfreeenergyepistemicpriorsplanningasclosed-loopcontrolvariationalSophisticatedtreesearch
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the behavioral advantage of Sophisticated Inference, an active-inference planning method usually linked to recursive tree search, comes from something more basic: combining an epistemic objective that values information with a posterior in which future actions depend on future states. The authors show this combination can be implemented as variational free energy minimization over a joint posterior, with no explicit tree needed. In a stochastic maze benchmark, agents with only the epistemic objective or only the joint posterior fail, while both the tree-search method and the variational method succeed at similar rates. A sympathetic reader would care because it relocates the source of 'sophisticated' behavior from a specific algorithm to a representational capacity that other inference schemes can supply.

What carries the argument

The central mechanism is the pair of an epistemic-prior variational free energy objective (a variational objective augmented with priors that reward information-seeking, ambiguity reduction, and novelty) and a joint posterior over future states and actions that keeps dependencies such as q(u_t | x_{t-1}, θ), making planning closed-loop inside the horizon. The one-step policy q(u_t | x_{t-1}) is read out from the optimized joint posterior by conditioning, so no separate tree search is needed. The factorized variant q(x)q(u) removes exactly this dependency and, with the same objective, destroys the ability to exploit the cue. This pair is what carries the argument: the objective supplies the w

What would settle it

Redefine the epistemic priors in the factorized condition using a fixed reference posterior so the objective is literally identical across posterior families; if the factorized agent then reaches the goal at rates comparable to the full-joint agent, the paper's conclusion that state-action dependencies are the deciding factor would be refuted. Alternatively, find a stochastic environment where an open-loop planner with epistemic value matches the success rate of a closed-loop planner; that would undermine the necessity claim.

Watch

Extended reading notes

Core claim

The paper's central claim is a representational equivalence: Sophisticated Inference's advantage is not unique to recursive tree search; it arises from closed-loop active inference, where future actions depend on future states. Epistemic priors in the variational free energy supply the active-inference objective, and a joint posterior over future states and actions supplies the state-contingent control structure. In the Reactivity Maze, a factorized active objective visits the informative cue every episode but reaches the goal only 1% of the time, while the full-joint active objective reaches the goal 98% of the time, statistically indistinguishable from Sophisticated Inference's 95%. Both i

Load-bearing premise

The experiments assume that the 'active' objective is genuinely unchanged when the posterior family is altered, so the comparison isolates the effect of state-action dependencies rather than a change in the effective objective.

Editorial extensions

If this is right

  • If the claim holds, any planner using an epistemic objective and a non-factorized state-action posterior should reproduce the qualitative behavior of Sophisticated Inference in similar stochastic environments.
  • Information-seeking is only useful if the posterior can propagate the gathered information back into later action selection; otherwise cue-visiting behavior produces no goal-reaching.
  • Inner-horizon closed-loop control is a distinct requirement from outer-loop replanning; the benchmark shows that replanning after observations does not compensate for missing state-action dependencies inside the rollout.
  • The advantage of Sophisticated Inference is representational rather than algorithmic, so tree search and variational inference are interchangeable realizations of the same planning structure.
  • Epistemic-prior VFE minimization over a joint posterior can match tree-search performance on a stochastic planning task without a dedicated branching procedure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's decisive ablation may be stronger than its design supports: the epistemic priors are defined through the variational posterior itself, so switching the posterior family to the factorized form could change the effective objective even when the formula is kept fixed. A cleaner test would freeze the prior densities before varying the family.
  • If the decomposition generalizes, closed-loop 'sophistication' could be added to any variational planner simply by relaxing action-state factorization, which may scale better than explicit tree search in continuous or high-dimensional settings.
  • The Reactivity Maze offers a safe-sink fallback that rewards risk aversion; in environments without such a stable suboptimal option, the gap between factorized and joint methods might shrink or grow, making the necessity claim testable across task families.
  • The paper deliberately leaves computational scaling open; a natural next step is to translate the joint-posterior objective into message-passing updates and measure whether the behavioral advantage survives in larger state spaces and longer horizons.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 6 minor

Summary. The paper argues that the computational advantage of Sophisticated Inference (SI) in active inference is not due to tree search per se, but to the closed-loop form of planning: future actions conditioned on future states/observations within the planning horizon. The authors show that this structure can be represented in epistemic-prior variational free energy minimization over a joint posterior over states and actions. They introduce the Reactivity Maze benchmark, designed to separate epistemic incentive (information seeking) from inner-horizon closed-loop control. They compare three variational objectives with the same state-action joint posterior (Fmarginal, Fplanning, Factive), an action-state factorized active variant, SI, and standard EFE planning. Results show that Factive and SI achieve high optimal rates (98% and 95%), while methods lacking epistemic drive or with factorized action-state posteriors perform poorly. The paper concludes that SI's advantage is reproduced by combining epistemic priors with posterior dependencies, without dedicated tree search.

Significance. If the central claim is established, it offers a conceptually important reframing of Sophisticated Inference: the key ingredient is closed-loop contingency, not the tree-search procedure. This could simplify algorithmic implementations of sophisticated behavior. The paper's strengths include the use of an external baseline library (pymdp), exact Clopper-Pearson confidence intervals, a clear conceptual distinction between inner-horizon closed-loop control and outer replanning, and the availability of code. The theoretical mapping of epistemic priors to active inference objectives in Section 2.5 is concise and plausible. However, the decisive empirical support is weakened by a confound in the factorized-active ablation, which is the main load-bearing evidence for the claim that closed-loop dependencies, rather than the epistemic objective, are necessary for the observed behavior. The paper's contribution would be solid after this is addressed.

major comments (1)
  1. [§4.1–4.2 (Eqs. (4a), (12), (13))] The decisive ablation F_u⊥x_active is confounded. The epistemic prior ~p(u) in Eq. (4a) is defined as a functional of the variational posterior: ~p(u) ∝ exp(H[q(x|u)]). In the factorized family (13), q(x|u) = q(x) because actions are independent of states and context, so ~p(u) becomes a constant independent of u, i.e., a uniform action prior. Thus F_active evaluated in the factorized family is not the same objective as F_active evaluated in the joint family (9). The paper's statement in §4.2 that 'for the factorized active condition, the active objective is held fixed while the posterior constraint in (13) is imposed' is therefore inaccurate unless the priors were frozen to a reference posterior q0, and no such procedure is described. The observed drop in optimal rate from 98% to 1% (Table 1) could be due to the removal of action-level epistemic pressure rather than the removal of closed
minor comments (6)
  1. [§5, Table 1] The text says F_u⊥x_active 'performs similarly to Standard EFE planning', but the 95% CIs for optimal rate are non-overlapping (1% [0,5] vs 12% [6,20]). Please either revise the wording or explicitly qualify the sense in which they are similar.
  2. [Appendix A] The convergence diagnostics cover Fmarginal, Fplanning, and Factive only. The factorized active variant is not included, so the claim that its failure is not an optimization artifact is not directly supported. Please add diagnostics for that condition or qualify the statement.
  3. [Eq. (13) and Table 1] The subscript 'u⊥x' is used in Table 1 but not defined in the main text. Define it at first use, e.g., 'action-state factorized'.
  4. [§4.3] The description of pymdp baselines says they use 'utility' but does not specify the utility matrix or how preferences are encoded. Please clarify.
  5. [§4, environment definition] The cue action prior 'p(at = 7) ∝ 1/ϵ with ϵ = 1.1' is ambiguous: does the unnormalized weight apply to all non-cue actions equally? Please state the normalization explicitly.
  6. [Abstract and §4.1] The abstract uses 'full-joint epistemic-prior active inference', but the posterior used in experiments is temporally factorized (Eq. (9)) and only retains selected dependencies. Please use consistent terminology, e.g., 'state-action joint'.

Circularity Check

1 steps flagged · score 6.0 of 10

The decisive F_u⊥x_active ablation does not hold the active objective fixed: because ~p(u) in Eq. (4a) is a functional of q, imposing the factorization (13) makes it uniform by construction.

  1. self definitional [Section 4.1, Eq. (4a) and Eq. (13); Section 4.2; Section 6]
    "This variant keeps the active objective and its epistemic priors, but constrains the posterior so that future actions are independent of future states and the latent context: q⊥(y,x,u,θ)=q(y,x,θ)∏_{t}q(ut). ... for the factorized active condition, the active objective is held fixed while the posterior constraint in (13) is imposed."

    The paper's own Eq. (4a) defines the action prior as ~p(u)∝exp(H[q(x|u)]), a functional of the variational posterior q. Under the factorized family (13), u is independent of x and θ, so q(x|u)=q(x) and H[q(x|u)]=H[q(x)] is constant in u; hence ~p(u) becomes uniform. Minimizing F_active over q⊥ is therefore not the same objective as minimizing it over the joint family (9): the action-level epistemic pressure is removed by construction. The claim that 'the active objective is held fixed' while imposing (13) is false unless the priors are frozen to a reference posterior, and no such freezing is described. The decisive ablation thus conflates the loss of closed-loop dependencies with a change in the objective, weakening the central empirical conclusion.

full rationale

Most of the paper's derivation is self-contained: the epistemic-prior objective is written out in Eqs. (3)–(5), and the main comparison includes external baselines (pymdp SI and Standard EFE), so the self-citations to the authors' earlier epistemic-prior work are background rather than load-bearing. The main circularity is localized to the decisive ablation. Eq. (4a) defines the action prior through the variational posterior q, so changing the posterior family from (9) to (13) changes the objective itself: under q⊥, q(x|u)=q(x), making ~p(u) uniform. The paper repeatedly claims the active objective is held fixed across this ablation, but that is not true by its own equations. The observed failure of the factorized variant could therefore be caused by the loss of action-level epistemic pressure instead of the absence of state-contingent control. This is a definitional confound in the paper's central empirical argument, though it does not invalidate the theoretical mapping or the comparison with Sophisticated Inference. Score 6 reflects one decisive 'prediction' reduced by construction rather than by the tested mechanism.

Assumptions & free parameters 6 free parameters · 7 assumptions · 0 invented entities

The central claim rests on the epistemic-prior framework inherited from the authors' own prior work, the custom-designed Reactivity Maze, and a variational posterior family that can represent state-conditioned actions. No new theoretical entities are introduced; the 'reactivity knob' and 'cue' are benchmark elements, not postulates.

free parameters (6)
  • cue action prior penalty ε = 1.1
    The prior over the cue action is p(a_t=7) ∝ 1/ε, making information-gathering costly; used in all experiments.
  • planning horizon T = 3
    All planning methods use T=3; results may depend on horizon length.
  • reward magnitudes = +1.0, -1.0, +0.33, -0.33
    Terminal rewards in Eq (8) shape the preference prior and define what 'optimal' means; hand-chosen to make the safe sink fallback viable.
  • transition stochasticity constant = 1/4 scaling in Eq (7)
    Knob values map to move probability; the environment's randomness is controlled by this scaling.
  • pymdp temperature γ = 16
    Used in the pymdp baselines; standard but hand-set.
  • optimization steps = 1000
    VFE objectives minimized for 1000 Adam steps; sensitivity analysis in Appendix A shows qualitative stability.
assumptions (7)
  • standard math Generative model p(y,x,u,θ) factorizes as in Eq (1).
    Basic state-space model assumed for planning-as-inference.
  • domain assumption Preference prior p̂(x) encodes preferred outcomes and acts as soft goal.
    Standard in active inference; not derived.
  • domain assumption Epistemic priors from Eq (4) yield expected free energy as in Eq (5).
    Taken from the authors' prior work (refs 8,19,20); used as given.
  • ad hoc to paper The F_planning correction (Eq 11) accounts for non-manipulability of environment noise.
    Quoted from ref [17] without derivation; unweighted conditional entropy is added to F_marginal.
  • standard math The temporal factorization (Eq 9) is sufficient to represent state- and context-contingent policies.
    This is a property of the variational family; necessary for the central claim.
  • ad hoc to paper Reactivity Maze dynamics (Eq 7) and reward (Eq 8) instantiate the intended separation of epistemic incentive and closed-loop control.
    The environment is custom-built for this study; its design choices are not externally validated.
  • ad hoc to paper The self-referential epistemic priors can be optimized by standard gradient methods.
    The paper assumes Adam minimization of F_active (with q-dependent priors) converges to the intended optimum; no fixed-point procedure is described.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sophisticated Policies from Epistemic Priors." pith.science (2026). https://pith.science/paper/SCPERKI3

@misc{pith2026260719518,
  author       = {Pith},
  title        = {Pith review of: Sophisticated Policies from Epistemic Priors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SCPERKI3}},
  note         = {Machine review of arXiv:2607.19518}
}
read the original abstract

Sophisticated Inference is a variant of active inference often associated with recursive belief modeling and tree search. We argue that its central computational role is simpler: within a planning horizon, it makes active inference closed-loop by allowing future actions to depend on future states and observations. This closed-loop structure can be represented in the epistemic-prior variational free energy framework. Epistemic priors supply the active-inference objective, while a joint posterior over future states and actions supplies the state-contingent control structure. We evaluate this decomposition in the Reactivity Maze, a stochastic benchmark designed to separate epistemic incentive from inner-horizon closed-loop control. The comparison includes three variational objectives with the same state-action posterior family, an action-state factorized active inference objective, Sophisticated Inference, and standard Expected Free Energy planning. The results show that neither ingredient is sufficient on its own. Methods without an epistemic component do not seek information, while methods that prevent future actions from depending on future states cannot turn information into reliable goal-reaching. By contrast, both Sophisticated Inference and full-joint epistemic-prior active inference solve the environment by combining epistemic drive with closed-loop inference. These results show that the advantage associated with Sophisticated Inference need not be specific to tree search itself. It arises from the closed-loop form of active inference, and this form can be represented in epistemic-prior variational inference when the posterior keeps future actions dependent on future states.

Figures

Figures reproduced from arXiv: 2607.19518 by the authors.

Figure 1
Figure 1. Schematic of the Reactivity Maze. Navigation states [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Optimization diagnostics for the three VFE objectives. Panel (a) shows [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗
Figure 3
Figure 3. Loss curves for four representative planning scenarios. The qualitative [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 1 canonical work pages

  1. [17]

    (eds.) Advances in Neural Information Processing Systems

    Lázaro-Gredilla, M., Ku, L.Y., Murphy, K.P., George, D.: What type of inference is planning? In: Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., Zhang, C. (eds.) Advances in Neural Information Processing Systems. vol. 37, pp. 116705–116742. Curran Associates, Inc. (2024).https://doi.org/10. 52202/079017-3705 14 W. W. L. Nuijten ...

    2024

  2. [1]

    In: International Workshop on Artificial Intelligence and Statistics

    Attias, H.: Planning by probabilistic inference. In: International Workshop on Artificial Intelligence and Statistics. pp. 9–16. PMLR (2003),https://proceedings. mlr.press/r4/attias03a.html Sophisticated Policies from Epistemic Priors 13

    2003

  3. [2]

    Bertsekas, D.: Dynamic Programming and Optimal Control: Volume I, vol. 1. Athena Scientific, Belmont, MA, 4 edn. (2017), https://books.google.com/ books?hl=en&lr=&id=qVBEEAAAQBAJ&oi=fnd&pg=PR1&dq=Dynamic+Programming+ and+Optimal+Control&ots=x0bAav0O5n&sig=s3UxthkdnzR2UpqCUsUsQ7zKgLc

    2017

  4. [3]

    com/jax-ml/jax

    Bradbury, J., Frostig, R., Hawkins, P., Johnson, M.J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., VanderPlas, J., Wanderman-Milne, S., Zhang, Q.: JAX: Composable transformations of Python+NumPy programs (2018),http://github. com/jax-ml/jax

    2018

  5. [4]

    Journal of Mathematical Psychology 81, 55–79 (Dec 2017).https://doi.org/10.1016/j.jmp.2017.09.004

    Buckley, C.L., Kim, C.S., McGregor, S., Seth, A.K.: The free energy principle for action and perception: A mathematical review. Journal of Mathematical Psychology 81, 55–79 (Dec 2017).https://doi.org/10.1016/j.jmp.2017.09.004

    doi:10.1016/j.jmp.2017.09.004 2017

  6. [5]

    Neural Networks151, 295–316 (Jul 2022)

    Champion, T., Da Costa, L., Bowman, H., Grześ, M.: Branching Time Active Inference: The theory and its generality. Neural Networks151, 295–316 (Jul 2022). https://doi.org/10.1016/j.neunet.2022.03.036

    doi:10.1016/j.neunet.2022.03.036 2022

  7. [6]

    Journal of Mathematical Psychology 99, 102447 (Dec 2020).https://doi.org/10.1016/j.jmp.2020.102447

    Da Costa, L., Parr, T., Sajid, N., Veselic, S., Neacsu, V., Friston, K.: Active inference on discrete state-spaces: A synthesis. Journal of Mathematical Psychology 99, 102447 (Dec 2020).https://doi.org/10.1016/j.jmp.2020.102447

    arXiv 2020

  8. [7]

    Da Costa, L., Tenka, S., Zhao, D., Sajid, N.: Active Inference as a Model of Agency (Jan 2024).https://doi.org/10.48550/arXiv.2401.12917

    arXiv doi:10.48550/arxiv.2401.12917 2024

Show all 29 references
  1. [8]

    De Vries, B., Nuijten, W., van de Laar, T., Kouw, W., Adamiat, S., Nisslbeck, T., Lukashchuk, M., Nguyen, H.M.H., Araya, M.H., Tresor, R., Jenneskens, T., Nikoloska, I., Subramanian, R.G., van Erp, B., Bagaev, D., Podusenko, A.: Expected Free Energy-based Planning as Variation...

    2025 doi

  2. [9]

    Friston, K.: The free-energy principle: A unified brain theory? Nature Reviews Neuroscience11(2), 127–138 (Feb 2010).https://doi.org/10.1038/nrn2787

    2010 doi

  3. [10]

    Neural Computation33(3), 713–763 (Mar 2021).https://doi.org/10.1162/neco_ a_01351

    Friston, K., Da Costa, L., Hafner, D., Hesp, C., Parr, T.: Sophisticated Inference. Neural Computation33(3), 713–763 (Mar 2021).https://doi.org/10.1162/neco_ a_01351

    2021 doi

  4. [11]

    Cognitive Neuroscience6(4), 187–214 (Oct 2015)

    Friston, K., Rigoli, F., Ognibene, D., Mathys, C., Fitzgerald, T., Pezzulo, G.: Active inference and epistemic value. Cognitive Neuroscience6(4), 187–214 (Oct 2015). https://doi.org/10.1080/17588928.2015.1020053

    2015

  5. [12]

    Biological Cybernetics102(3), 227–260 (Mar 2010).https: //doi.org/10.1007/s00422-010-0364-z

    Friston, K.J., Daunizeau, J., Kilner, J., Kiebel, S.J.: Action and behavior: A free- energy formulation. Biological Cybernetics102(3), 227–260 (Mar 2010).https: //doi.org/10.1007/s00422-010-0364-z

    2010 doi

  6. [13]

    Journal of Open Source Software7(73), 4098 (May 2022)

    Heins, C., Millidge, B., Demekas, D., Klein, B., Friston, K., Couzin, I.D., Tschantz, A.: Pymdp: A Python library for active inference in discrete state spaces. Journal of Open Source Software7(73), 4098 (May 2022). https://doi.org/10.21105/ joss.04098

    2022

  7. [14]

    In: International Conference on Learning Representations (ICLR)

    Kingma, D., Ba, J.: Adam: A Method for Stochastic Optimization. In: International Conference on Learning Representations (ICLR). San Diego, CA, USA (2015)

    2015

  8. [15]

    Koudahl, M., van de Laar, T., de Vries, B.: Realising Synthetic Active Inference Agents, Part I: Epistemic Objectives and Graphical Specification Language (Jun 2023).https://doi.org/10.48550/arXiv.2306.08014

    2023 doi arXiv

  9. [16]

    Cambridge university press (2006)

    LaValle, S.M.: Planning Algorithms. Cambridge university press (2006)

    2006

  10. [18]

    Levine, S.: Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review (May 2018).https://doi.org/10.48550/arXiv.1805.00909

    2018 doi arXiv

  11. [19]

    In: Albarracin, M., Benrimoh, D., Buckley, C.L., Lanillos, P., Pitliya, R.J., Shimazaki, H., Stoianov, I.P., Verbelen, T., Wisse, M

    Nuijten, W.W.L., Lukashchuk, M., van de Laar, T., de Vries, B.: A Message Passing Realization of Expected Free Energy Minimization. In: Albarracin, M., Benrimoh, D., Buckley, C.L., Lanillos, P., Pitliya, R.J., Shimazaki, H., Stoianov, I.P., Verbelen, T., Wisse, M. (eds.) Activ...

    2026 doi

  12. [20]

    Transactions on Machine Learning Research (2026),https://openreview.net/forum?id=Kzm8I1oS1s

    Nuijten, W.W.L., van de Laar, T., de Vries, B.: Expected Free Energy-based Planning as Variational Inference. Transactions on Machine Learning Research (2026),https://openreview.net/forum?id=Kzm8I1oS1s

    2026

  13. [21]

    IEEE Access10, 15193–15216 (2022)

    Palmieri, F.A.N., Pattipati, K.R., Gennaro, G.D., Fioretti, G., Verolla, F., Buo- nanno, A.: A Unifying View of Estimation and Control Using Belief Propaga- tion With Application to Path Planning. IEEE Access10, 15193–15216 (2022). https://doi.org/10.1109/ACCESS.2022.3148127

    2022

  14. [22]

    Bio- logical Cybernetics113(5), 495–513 (Dec 2019)

    Parr, T., Friston, K.J.: Generalised free energy and active inference. Bio- logical Cybernetics113(5), 495–513 (Dec 2019). https://doi.org/10.1007/ s00422-019-00805-w

    2019

  15. [23]

    The MIT Press (Mar 2022).https://doi.org/10

    Parr, T., Pezzulo, G., Friston, K.J.: Active Inference: The Free Energy Principle in Mind, Brain, and Behavior. The MIT Press (Mar 2022).https://doi.org/10. 7551/mitpress/12441.001.0001

    2022

  16. [24]

    Entropy26(6), 484 (Jun 2024).https://doi.org/10.3390/ e26060484

    Paul, A., Isomura, T., Razi, A.: On Predictive Planning and Counterfactual Learning in Active Inference. Entropy26(6), 484 (Jun 2024).https://doi.org/10.3390/ e26060484

    2024

  17. [25]

    Expert Systems with Applications253, 124315 (Nov 2024)

    Paul, A., Sajid, N., Costa, L.D., Razi, A.: On efficient computation in active inference. Expert Systems with Applications253, 124315 (Nov 2024). https: //doi.org/10.1016/j.eswa.2024.124315

    2024

  18. [26]

    Rawlik, K., Toussaint, M., Vijayakumar, S.: On stochastic optimal control and reinforcement learning by approximate inference. Proceedings of Robotics: Science and Systems VIII (2012),https://books.google.com/books?hl=en&lr= &id=NOrxCwAAQBAJ&oi=fnd&pg=PA353&dq=On+stochastic+op...

    2012

  19. [27]

    In: Ad- vances in Neural Information Processing Systems

    Todorov, E.: Linearly-solvable Markov decision problems. In: Ad- vances in Neural Information Processing Systems. vol. 19. MIT Press (2006), https://proceedings.neurips.cc/paper_files/paper/2006/hash/ d806ca13ca3449af72a1ea5aedbed26a-Abstract.html

    2006

  20. [28]

    In: Proceedings of the 23rd International Conference on Machine Learning

    Toussaint, M., Storkey, A.: Probabilistic inference for solving discrete and continu- ous state Markov Decision Processes. In: Proceedings of the 23rd International Conference on Machine Learning. pp. 945–952 (2006), http://delivery.acm. org/10.1145/1150000/1143963/p945-toussa...

    2006 doi

  21. [29]

    van de Laar, T., Koudahl, M., de Vries, B.: Realizing Synthetic Active Inference Agents, Part II: Variational Message Updates. Neural Computation37(1), 38–75 (Jan 2025).https://doi.org/10.1162/neco_a_01713 Sophisticated Policies from Epistemic Priors 15 A Convergence Diagnosti...

    2025 doi

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.