Pith. sign in

REVIEW 3 major objections 4 minor 87 references

A reinforcement-learned meta-policy can decide when to plan by watching a cheap uncertainty signal, and it beats fixed strategies that always react or always plan.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 20:58 UTC pith:DTZBFXYX

load-bearing objection Competent empirical study of RL-based arbitration between reactive control and planning; the mechanism claim is plausible but the uncertainty signal needs calibration before the joint-training result can be taken at face value. the 3 major comments →

arxiv 2607.16421 v1 pith:DTZBFXYX submitted 2026-07-17 cs.AI

When to Plan: Learning to Select Between Reactive Control and Deliberative Planning

Adam Labiosa , Josiah P. Hanna This is my paper
classification cs.AI
keywords meta-reasoningadaptive computationreinforcement learninguncertainty estimationensemble disagreementplanningreactive controlcompute allocation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks when an agent should act instantly from a learned reactive policy rather than spend time planning, and claims an RL-trained meta-policy can answer that question by watching a cheap uncertainty signal. The meta-policy conditions on an ensemble-disagreement score from the reactive policy, the distance to the goal, and a short history, then chooses between one reactive step and plans of different horizons. Across five motion-planning and navigation domains, the learned controller achieves higher average return than fixed baselines that always react or always plan, measured by time to reach the goal. Ablations show that the uncertainty score and the observation history are the load-bearing inputs. The paper also demonstrates that when the reactive policy improves during joint training, the meta-policy tracks that improvement and shifts toward fully reactive control.

Core claim

The central claim is that a model-free RL meta-policy can learn adaptive computation allocation—deciding at each moment whether to emit a fast reactive action or invoke a slower planner—using only a task-agnostic uncertainty signal from the reactive policy together with distance-to-goal, previous action, and a short history. The uncertainty signal is the disagreement of an ensemble of behavior-cloned networks: variance of predicted action means for continuous control, entropy of the averaged action distribution for discrete control. On all five environments the learned meta-policy achieves higher average return than always-reactive, always-short-plan, always-medium-plan, and always-long-plan

What carries the argument

The carrying mechanism is the meta-MDP with options: a meta-policy selects among a one-step reactive option and three planning options of fixed horizon, each converted to a common time-based reward (a cost of -1 per action, plus -αk for a plan of length k). The meta-observation is built from the reactive policy's ensemble-disagreement score u(s)—variance across an ensemble of M behavior-cloned networks for continuous actions, or entropy of the averaged categorical distribution for discrete actions—plus distance to goal, previous meta-action, and a history of recent observations. This uncertainty score is what lets the meta-policy estimate reactive competence without search and without the ra

Load-bearing premise

The load-bearing premise is that the ensemble-disagreement score u(s) is a valid, task-agnostic proxy for when the reactive policy is incompetent; the paper does not calibrate it against actual reactive value loss, so if high disagreement does not align with out-of-distribution states where planning helps, the meta-policy's advantage should not transfer.

What would settle it

Compute, on held-out tasks in any of the five environments, the rank correlation between the ensemble-disagreement score and the suboptimality gap V*(s) - V^{πr}(s). If the correlation is weak or negative, the mechanism the paper attributes to uncertainty is not operating. Alternatively, replacing u(s) with a random scalar in the meta-observation and showing performance barely changes would falsify the claim that the uncertainty signal carries the adaptive behavior.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim holds, autonomous systems can deploy a fast learned policy most of the time and reserve expensive planning for the states where it matters, reducing average time-to-goal without sacrificing reliability.
  • Because the meta-observation is computed without search, the method imposes only a small overhead on the reactive baseline.
  • The joint-training result implies that planning can be used as a data generator to improve the reactive policy, and the meta-policy will automatically reallocate computation as competence grows.
  • The same framing applies wherever a cheap policy and an expensive deliberative routine coexist, including language-model reasoning and cloud-robot control, since the tradeoff is time versus quality.
  • The ablations imply that a history of uncertainty and progress is needed, not just instantaneous values, so meta-decisions are made from short trends rather than a single snapshot.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the method assumes the planner is reliable everywhere; with a learned world model, the planner's own uncertainty would need to enter the meta-observation, and the same ensemble trick could be applied to the world model.
  • Editorial inference: because the uncertainty score is only a proxy, tasks with high aleatoric noise but good reactive actions might waste compute planning; adding a calibration step against actual reactive returns would test whether the score can be trusted beyond these five domains.
  • Editorial inference: the joint-training loop suggests a self-improvement curriculum—planning in out-of-distribution states generates the very data that makes planning unnecessary—which could be studied as a general mechanism for reducing deliberation over an agent's lifetime.
  • Editorial inference: the paper leaves open whether the meta-policy can track a worsening reactive policy; capacity limits or catastrophic forgetting are plausible failure modes that would require the meta-policy to relearn when to plan.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies meta-reasoning in goal-reaching tasks: a PPO-trained meta-policy chooses at each meta-step between a fast reactive policy and several planning options of different horizons, with planning incurring a time cost. The meta-observation is the ensemble-disagreement uncertainty of the reactive policy, distance to goal, previous meta-action, and a short history of these features. Experiments on five motion-planning/navigation domains compare against fixed-compute baselines (always react, always short/medium/long plan), ablate observation components, vary planning cost, in-distribution proportion, and stochasticity, and finally study joint training in which the reactive policy is trained online from planner rollouts. The paper reports that the meta-policy outperforms all fixed baselines in all five domains, that uncertainty and history are the most critical observation components, and that an uncertainty-conditioned meta-policy shifts toward fully reactive control as the reactive policy improves.

Significance. If the results hold, this is a useful empirical contribution to adaptive computation allocation. The options-MDP formulation with an explicit time-based planning cost is clean, and the experimental protocol is solid: 30 seeds, 300 evaluation episodes per seed, bootstrap confidence intervals, ablations, and sensitivity sweep. The central novelty—an RL meta-policy conditioned on an ensemble-disagreement uncertainty score—is timely and relevant to the broader compute-allocation literature. The principal limitations are the unvalidated link between ensemble disagreement and reactive-policy incompetence, and the use of a perfect world model; both are acknowledged or implicit in the text but need to be addressed before the mechanism claims can be accepted at face value.

major comments (3)
  1. [Section 5, Eqs. for u(s); Section 6.6] The load-bearing premise is that the ensemble-disagreement score u(s) is a valid, task-agnostic proxy for reactive-policy incompetence. The paper asserts that 'high uncertainty corresponds to out-of-distribution inputs where planning may be beneficial,' but it never calibrates u(s) against the actual advantage of planning, e.g., V*(s)-V^{pi_r}(s) or the expected return difference between reactive and planning options. This is especially important because the central claim attributes the meta-policy's success to conditioning on this signal. Please add a calibration analysis (correlation or ranking of u(s) against measured planning advantage on held-out ID/OOD states) and, ideally, an oracle-ablation where the meta-policy receives the true competence signal, to establish that the uncertainty score is adding the intended information rather than a generic novelty signal.
  2. [Section 6.4, Table 3] The summary 'the two most important components are history and uncertainty scores' is not supported by the reported numbers. In Doorkey, removing distance causes the largest degradation (-34.3 vs -25.1), while removing history has almost no effect (-26.1). In Box Push, removing distance improves return (-20.9 vs -24.3), contradicting the idea that distance is a helpful component. In Maze, removing uncertainty degrades from -19.3 to -25.0, which is substantial, yet the text says there is no substantial drop-off without uncertainty in the 'other three environments.' Please re-analyze Table 3 with per-environment pairwise comparisons and correct the text; the current selective summary misrepresents which observation components matter and in which environments.
  3. [Section 6.6, Figure 4] The joint-training experiment is confounded with the training procedure. The reactive policy is trained only on state-action pairs collected from planning options, so ensemble disagreement is mechanically reduced as those specific states are fit, regardless of whether the reactive policy has actually become competent in a broader sense. The paper concludes that the uncertainty signal 'reflects the strength of the reactive policy,' but the experiment as reported cannot separate genuine competence improvement from distribution fitting. Please add a control condition—for example, training the same reactive policy on an equal number of ID-only or random states—and report the reactive policy's actual success/value on held-out ID and OOD states. This would clarify whether the meta-policy is tracking true competence or merely a decreasing ensemble-variance artifact.
minor comments (4)
  1. [Section 6.1/Table 4] The text says planning horizons are 15%, 30%, and 50% of the task horizon, but the table values correspond to approximately 16-17%, 32-33%, and 50% for most environments. Please say 'approximately' or reconcile the fractions.
  2. [Table 1 caption] The caption states that bold entries overlap in CI with the best performer, but no boldface is visible in the provided text. Please render the bold in the final version.
  3. [Section 6.4] The phrase 'model-free uncertainty' is confusing; the uncertainty score comes from an ensemble, not from model-free RL. Consider renaming to 'reactive-policy ensemble uncertainty'.
  4. [Section 6.5] In the stochasticity experiment, the text says the ratio of plan lengths remains roughly constant as alpha increases, but the accompanying figure shows some shifts; a quantitative statement or per-environment figure would help.

Circularity Check

0 steps flagged

No significant circularity: results are empirical comparisons against external fixed-compute baselines; the uncertainty proxy is an assumption, not a circular reduction.

full rationale

The paper's central claims are established by direct RL training and evaluation against fixed-compute baselines (Table 1), not by fitting parameters to the target and then 'predicting' it. The uncertainty score u(s) is an input feature computed from an ensemble of reactive networks; the load-bearing premise that high ensemble disagreement indicates states where planning helps is an explicit design assumption (Section 5), supported by ablations (Table 3), not a circular reduction. In the joint-training experiment (Section 6.6), the reactive policy is trained on planner rollouts, so a drop in ensemble disagreement is expected; however, the meta-policy's shift toward reactive control is a learned behavior under the RL reward, not an identity forced by the definition of u(s). The only self-citation (Hanna & Corrado 2025) appears in Related Work and is not load-bearing. Limitations such as the perfect world model assumption are external-validity concerns, not circularity. No equation-level reduction between input and output was found.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central claims rest on hand-picked cost and horizon parameters, a perfect world model, an engineered in-distribution/out-of-distribution competence gap, and an uncalibrated ensemble-disagreement uncertainty signal. No new physical or conceptual entities are introduced beyond standard constructs (meta-policy, options, uncertainty score). The reader must grant these assumptions for the results to transfer beyond the five simulated environments.

free parameters (4)
  • Plan cost scaling alpha = 0.5 (default), swept over {0.1, 0.5, 1, 5}
    Chosen by hand in Section 6.2 to make planning neither trivially cheap nor prohibitively expensive. The learned reactive/planning balance is directly controlled by this parameter.
  • Planning horizons as fractions of task horizon = 15% / 30% / 50% of task horizon (e.g., 5/10/15 for Box Push)
    Hand-chosen in Section 6.2 to span low, medium, and high compute. The authors explicitly leave sensitivity to these values for future work, yet the meta-policy behavior depends on them.
  • Ensemble size M = 4
    All experiments use M=4 reactive networks; the uncertainty score is computed from this ensemble and is a core input to the meta-policy.
  • Meta-observation history length n = 4
    Set in Section 6.2. The ablation shows history is one of the most critical components, so the choice of n affects the central results.
axioms (4)
  • domain assumption A perfect, accurate world model is available for planning.
    Section 6.2: 'Planning is performed using a given accurate world model which isolates the compute allocation problem from the model learning problem.' Acknowledged in Section 7 as unrealistic in most real-world settings.
  • domain assumption The reactive policy is near-optimal on in-distribution states and poor on out-of-distribution states.
    This competence gap is engineered by behavior cloning from planner trajectories on S_ID (Section 4). The entire value of planning in the setup depends on this gap.
  • domain assumption Ensemble disagreement (variance or entropy) is a valid proxy for reactive-policy incompetence.
    Section 5 defines u(s) from deep-ensemble disagreement and asserts that high uncertainty corresponds to out-of-distribution inputs where planning may be beneficial. The paper does not calibrate u(s) against reactive value loss.
  • domain assumption A* with a fixed search-depth limit returns a near-optimal action sequence for the chosen horizon.
    Section 6.2 uses depth-limited A* as the planner. The meta-policy's planning options are only as good as these approximations, and no error analysis of the depth-limited plans is provided.

pith-pipeline@v1.3.0-alltime-deepseek · 18785 in / 15048 out tokens · 132691 ms · 2026-08-01T20:58:47.134836+00:00 · methodology

0 comments
read the original abstract

It has long been recognized that humans have the ability to switch between fast, reactive decision-making and slower, deliberative planning. In this paper, we study the question of how to learn this ability, known as meta-reasoning, in artificial agents. We model reactive decision-making as a policy that directly maps state observations to actions. Such policies can be trained with reinforcement learning (RL) or imitation learning, but may generalize poorly outside of their training distribution. Alternatively, model-based decision-time planning is more likely to produce good actions across a broader set of states but requires additional computation time, which delays acting. In this work, we introduce an RL method for training a meta-reasoning policy that allocates computation by conditioning on a reactive-policy uncertainty score. This score enables it to predict when the reactive policy is likely to perform poorly and when planning is needed. We conduct an empirical study on motion planning and navigation environments, showing that this design enables the meta-reasoning policy to learn when the reactive policy provides a good-enough action versus when decision-time planning is needed. Additionally, we show that our design enables the meta-agent to shift toward fully reactive control as the reactive policy improves.

Figures

Figures reproduced from arXiv: 2607.16421 by Adam Labiosa, Josiah P. Hanna.

Figure 1
Figure 1. Figure 1: Example images of our environment suite. Environments vary in state space type (discrete [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The meta reasoning agent. The reactive policy (ensemble of neural networks), outputs an [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Average meta-action taken after training across all environments for different environment [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Average return, reactive action percent and reactive actor uncertainty value as the meta [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Reactive policy in-distribution percentage experiment. Individual environment results when varying the percent of in-distribution tasks for the reactive policy. 20 [PITH_FULL_IMAGE:figures/full_fig_p021_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Plan cost experiment. Individual environment results when varying the planning cost α. 21 [PITH_FULL_IMAGE:figures/full_fig_p022_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Action stochasticity experiments. Individual environment results when varying the en￾vironment stochasticity. 22 [PITH_FULL_IMAGE:figures/full_fig_p023_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

87 extracted references · 18 canonical work pages · 5 internal anchors

  1. [1]

    Philosophical Transactions of the Royal Society B: Biological Sciences , author =

    Cortical mechanisms of action selection: the affordance competition hypothesis , volume =. Philosophical Transactions of the Royal Society B: Biological Sciences , author =. 2007 , pages =. doi:10.1098/rstb.2007.2054 , abstract =

    arXiv 2007

  2. [2]

    Selecting

    Hay, Nicholas and Russell, Stuart and Tolpin, David and Shimony, Solomon Eyal , month = aug, year =. Selecting. doi:10.48550/arXiv.1408.2048 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.1408.2048 2048

  3. [3]

    Simple and

    Lakshminarayanan, Balaji and Pritzel, Alexander and Blundell, Charles , month = nov, year =. Simple and. doi:10.48550/arXiv.1612.01474 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.1612.01474

  4. [4]

    Thinking Fast and Slow in AI

    Booch, Grady and Fabiano, Francesco and Horesh, Lior and Kate, Kiran and Lenchner, Jon and Linck, Nick and Loreggia, Andrea and Murugesan, Keerthiram and Mattei, Nicholas and Rossi, Francesca and Srivastava, Biplav , month = dec, year =. Thinking. doi:10.48550/arXiv.2010.06002 , abstract =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2010.06002 2010

  5. [5]

    Li, Zhong-Zhi and Zhang, Duzhen and Zhang, Ming-Liang and Zhang, Jiaxin and Liu, Zengyan and Yao, Yuxuan and Xu, Haotian and Zheng, Junhao and Wang, Pei-Jie and Chen, Xiuyi and Zhang, Yingying and Yin, Fei and Dong, Jiahua and Li, Zhiwei and Bi, Bao-Long and Mei, Ling-Rui and Fang, Junfeng and Liang, Xiao and Guo, Zhijiang and Song, Le and Liu, Cheng-Lin ...

    Pith/arXiv arXiv doi:10.48550/arxiv.2502.17419

  6. [6]

    , month = feb, year =

    Lee, Sang Wan and Shimojo, Shinsuke and O’Doherty, John P. , month = feb, year =. Neural. Neuron , publisher =. doi:10.1016/j.neuron.2013.11.028 , language =

    doi:10.1016/j.neuron.2013.11.028 2013

  7. [7]

    2011 , publisher=

    Thinking, fast and slow , author=. 2011 , publisher=

    2011

  8. [8]

    arXiv preprint arXiv:2302.04840 , year=

    What are the mechanisms underlying metacognitive learning? , author=. arXiv preprint arXiv:2302.04840 , year=

    Pith/arXiv arXiv

  9. [9]

    Rodrigo de Lazcano and Kallinteris Andreas and Jun Jet Tai and Seungjae Ryan Lee and Jordan Terry , title =

  10. [10]

    CoRR , volume =

    Maxime Chevalier-Boisvert and Bolun Dai and Mark Towers and Rodrigo de Lazcano and Lucas Willems and Salem Lahlou and Suman Pal and Pablo Samuel Castro and Jordan Terry , title =. CoRR , volume =

  11. [11]

    Journal of Machine Learning Research , year =

    Antonin Raffin and Ashley Hill and Adam Gleave and Anssi Kanervisto and Maximilian Ernestus and Noah Dormann , title =. Journal of Machine Learning Research , year =

  12. [12]

    The quarterly journal of economics , pages=

    A behavioral model of rational choice , author=. The quarterly journal of economics , pages=. 1955 , publisher=

    1955

  13. [13]

    British Journal of Psychology , volume=

    Heuristic and analytic processes in reasoning , author=. British Journal of Psychology , volume=. 1984 , publisher=

    1984

  14. [14]

    Learning

    Paglieri, Davide and Cupiał, Bartłomiej and Cook, Jonathan and Piterbarg, Ulyana and Tuyls, Jens and Grefenstette, Edward and Foerster, Jakob Nicolaus and Parker-Holder, Jack and Rocktäschel, Tim , month = sep, year =. Learning. doi:10.48550/arXiv.2509.03581 , abstract =

    doi:10.48550/arxiv.2509.03581

  15. [15]

    Current Opinion in Behavioral Sciences , author =

    The structure of reinforcement-learning mechanisms in the human brain , volume =. Current Opinion in Behavioral Sciences , author =. 2015 , pages =. doi:10.1016/j.cobeha.2014.10.004 , abstract =

    work page doi:10.1016/j.cobeha.2014.10.004 2015

  16. [16]

    Deployable Reinforcement Learning with Variable Control Rate

    Wang, Dong and Beltrame, Giovanni , month = apr, year =. Deployable. doi:10.48550/arXiv.2401.09286 , abstract =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2401.09286

  17. [17]

    and Corrado, Nicholas E

    Hanna, Josiah P. and Corrado, Nicholas E. , month = oct, year =. When. doi:10.48550/arXiv.2506.17124 , abstract =

    doi:10.48550/arxiv.2506.17124

  18. [18]

    , month = jul, year =

    Sheikhnezhad Fard, Farzaneh and Trappenberg, Thomas P. , month = jul, year =. A. Frontiers in Neurorobotics , publisher =. doi:10.3389/fnbot.2019.00052 , abstract =

    arXiv 2019

  19. [19]

    Psychological Science , author =

    The. Psychological Science , author =. 2013 , pages =. doi:10.1177/0956797612463080 , abstract =

    doi:10.1177/0956797612463080 2013

  20. [20]

    and Trappenberg, Thomas P

    Fard, Farzaneh S. and Trappenberg, Thomas P. , month = jul, year =. Mixing. 2018. doi:10.1109/IJCNN.2018.8489122 , abstract =

    arXiv 2018

  21. [21]

    Surprisal-Triggered Conditional Computation with Neural Networks

    Lugosch, Loren and Nowrouzezahrai, Derek and Meyer, Brett H. , month = jun, year =. Surprisal-. doi:10.48550/arXiv.2006.01659 , abstract =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2006.01659 2006

  22. [22]

    Journal of Cognitive Neuroscience , author =

    Planning. Journal of Cognitive Neuroscience , author =. 2018 , keywords =. doi:10.1162/jocn_a_01263 , abstract =

    work page doi:10.1162/jocn_a_01263 2018

  23. [23]

    Liao, Baohao and Xu, Yuhui and Dong, Hanze and Li, Junnan and Monz, Christof and Savarese, Silvio and Sahoo, Doyen and Xiong, Caiming , month = jun, year =. Reward-. doi:10.48550/arXiv.2501.19324 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2501.19324

  24. [24]

    Thinkless:

    Fang, Gongfan and Ma, Xinyin and Wang, Xinchao , month = jun, year =. Thinkless:. doi:10.48550/arXiv.2505.13379 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.13379

  25. [25]

    and Kadous, M

    Ong, Isaac and Almahairi, Amjad and Wu, Vincent and Chiang, Wei-Lin and Wu, Tianhao and Gonzalez, Joseph E. and Kadous, M. Waleed and Stoica, Ion , month = feb, year =. doi:10.48550/arXiv.2406.18665 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2406.18665

  26. [26]

    and Johansson, Karl H

    Yoo, Jaehyun and Molin, Adam and Jafarian, Matin and Esen, Hasan and Dimarogonas, Dimos V. and Johansson, Karl H. , month = dec, year =. Event-triggered model predictive control with machine learning for compensation of model uncertainties , url =. 2017. doi:10.1109/CDC.2017.8264468 , abstract =

    arXiv 2017

  27. [27]

    Ocean Engineering , author =

    Predictive compensator based event-triggered model predictive control with nonlinear disturbance observer for unmanned surface vehicle under cyber-attacks , volume =. Ocean Engineering , author =. 2022 , keywords =. doi:10.1016/j.oceaneng.2022.111868 , abstract =

    arXiv 2022

  28. [28]

    IEEE Transactions on Cybernetics , author =

    Learning-. IEEE Transactions on Cybernetics , author =. 2025 , keywords =. doi:10.1109/TCYB.2025.3536606 , abstract =

    arXiv 2025

  29. [29]

    Automatica , author =

    Event-based model predictive control for nonlinear systems with dynamic disturbance , volume =. Automatica , author =. 2022 , keywords =. doi:10.1016/j.automatica.2022.110533 , abstract =

    arXiv 2022

  30. [30]

    IEEE/ASME Transactions on Mechatronics , author =

    Tracking of. IEEE/ASME Transactions on Mechatronics , author =. 2020 , keywords =. doi:10.1109/TMECH.2019.2962099 , abstract =

    arXiv 2020

  31. [31]

    Reinforcement

    Chen, Jun and Meng, Xiangyu and Li, Zhaojian , month = jun, year =. Reinforcement. 2022. doi:10.23919/ACC53348.2022.9867347 , abstract =

    arXiv 2022

  32. [32]

    IEEE Transactions on Intelligent Vehicles , author =

    Event-. IEEE Transactions on Intelligent Vehicles , author =. 2024 , keywords =. doi:10.1109/TIV.2023.3329785 , abstract =

    arXiv 2024

  33. [33]

    and Griffiths, Thomas L

    Callaway, Frederick and Gul, Sayan and Krueger, Paul M. and Griffiths, Thomas L. and Lieder, Falk , month = aug, year =. Learning to select computations , url =. doi:10.48550/arXiv.1711.06892 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.1711.06892

  34. [34]

    and Xia, Jerry and Chung, Stephen and Greenwald, Amy , month = apr, year =

    Wang, Kevin A. and Xia, Jerry and Chung, Stephen and Greenwald, Amy , month = apr, year =. Dynamic

  35. [35]

    Manvi, Rohin and Hong, Joey and Seyde, Tim and Labonne, Maxime and Lechner, Mathias and Levine, Sergey , month = dec, year =. Zero-. doi:10.48550/arXiv.2512.01457 , abstract =

    doi:10.48550/arxiv.2512.01457

  36. [36]

    PLoS computational biology , author =

    Speed/accuracy trade-off between the habitual and the goal-directed processes , volume =. PLoS computational biology , author =. 2011 , keywords =. doi:10.1371/journal.pcbi.1002055 , abstract =

    work page doi:10.1371/journal.pcbi.1002055 2011

  37. [37]

    Proceedings of the AAAI Conference on Artificial Intelligence , author =

    Stop!. Proceedings of the AAAI Conference on Artificial Intelligence , author =. 2024 , keywords =. doi:10.1609/aaai.v38i18.29983 , abstract =

    work page doi:10.1609/aaai.v38i18.29983 2024

  38. [38]

    The European Journal of Neuroscience , author =

    Adaptive. The European Journal of Neuroscience , author =. 2025 , pages =. doi:10.1111/ejn.70024 , abstract =

    work page doi:10.1111/ejn.70024 2025

  39. [39]

    , month = aug, year =

    Ackerman, Rakefet and Thompson, Valerie A. , month = aug, year =. Meta-. Trends in Cognitive Sciences , publisher =. doi:10.1016/j.tics.2017.05.004 , language =

    doi:10.1016/j.tics.2017.05.004 2017

  40. [40]

    , month = may, year =

    Gläscher, Jan and Daw, Nathaniel and Dayan, Peter and O'Doherty, John P. , month = may, year =. States versus. Neuron , publisher =. doi:10.1016/j.neuron.2010.04.016 , language =

    doi:10.1016/j.neuron.2010.04.016 2010

  41. [41]

    Inference

    Wu, Yangzhen and Sun, Zhiqing and Li, Shanda and Welleck, Sean and Yang, Yiming , month = mar, year =. Inference. doi:10.48550/arXiv.2408.00724 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2408.00724

  42. [42]

    Snell, Charlie and Lee, Jaehoon and Xu, Kelvin and Kumar, Aviral , month = aug, year =. Scaling. doi:10.48550/arXiv.2408.03314 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2408.03314

  43. [43]

    Heuristic and analytic processes in reasoning* -

  44. [44]

    and Chess, Benjamin and Child, Rewon and Gray, Scott and Radford, Alec and Wu, Jeffrey and Amodei, Dario , month = jan, year =

    Kaplan, Jared and McCandlish, Sam and Henighan, Tom and Brown, Tom B. and Chess, Benjamin and Child, Rewon and Gray, Scott and Radford, Alec and Wu, Jeffrey and Amodei, Dario , month = jan, year =. Scaling. doi:10.48550/arXiv.2001.08361 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2001.08361 2001

  45. [45]

    OpenAI and Jaech, Aaron and Kalai, Adam and Lerer, Adam and Richardson, Adam and El-Kishky, Ahmed and Low, Aiden and Helyar, Alec and Madry, Aleksander and Beutel, Alex and Carney, Alex and Iftimie, Alex and Karpenko, Alex and Passos, Alex Tachard and Neitz, Alexander and Prokofiev, Alexander and Wei, Alexander and Tam, Allison and Bennett, Ally and Kumar...

    Pith/arXiv arXiv doi:10.48550/arxiv.2412.16720

  46. [46]

    2025 , note =

    Nature , author =. 2025 , note =. doi:10.1038/s41586-025-09422-z , abstract =

    doi:10.1038/s41586-025-09422-z 2025

  47. [47]

    Sensors (Basel, Switzerland) , author =

    A. Sensors (Basel, Switzerland) , author =. 2024 , pages =. doi:10.3390/s24154808 , abstract =

    work page doi:10.3390/s24154808 2024

  48. [48]

    Baniodeh, Mustafa and Goel, Kratarth and Ettinger, Scott and Fuertes, Carlos and Seff, Ari and Shen, Tim and Gulino, Cole and Yang, Chenjie and Jerfel, Ghassen and Choe, Dokook and Wang, Rui and Charrow, Benjamin and Kallem, Vinutha and Casas, Sergio and Al-Rfou, Rami and Sapp, Benjamin and Anguelov, Dragomir , month = sep, year =. Scaling. doi:10.48550/a...

    Pith/arXiv arXiv doi:10.48550/arxiv.2506.08228

  49. [49]

    doi:10.48550/arXiv.1812.03079 , abstract =

    Bansal, Mayank and Krizhevsky, Alex and Ogale, Abhijit , month = dec, year =. doi:10.48550/arXiv.1812.03079 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.1812.03079

  50. [50]

    and Tirumala, Dhruva and Wulfmeier, Markus and Humplik, Jan and Tunyasuvunakool, Saran and Siegel, Noah Y

    Haarnoja, Tuomas and Moran, Ben and Lever, Guy and Huang, Sandy H. and Tirumala, Dhruva and Wulfmeier, Markus and Humplik, Jan and Tunyasuvunakool, Saran and Siegel, Noah Y. and Hafner, Roland and Bloesch, Michael and Hartikainen, Kristian and Byravan, Arunkumar and Hasenclever, Leonard and Tassa, Yuval and Sadeghi, Fereshteh and Batchelor, Nathan and Cas...

  51. [51]

    Transportation Research Part F: Traffic Psychology and Behaviour , author =

    Driving on familiar roads:. Transportation Research Part F: Traffic Psychology and Behaviour , author =. 2013 , keywords =. doi:10.1016/j.trf.2013.03.008 , abstract =

    work page doi:10.1016/j.trf.2013.03.008 2013

  52. [52]

    Proximal

    Schulman, John and Wolski, Filip and Dhariwal, Prafulla and Radford, Alec and Klimov, Oleg , month = aug, year =. Proximal. doi:10.48550/arXiv.1707.06347 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.1707.06347

  53. [53]

    Botea, Adi and Müller, Martin and Schaeffer, Jonathan , editor =. Using. Computers and. 2003 , pages =. doi:10.1007/978-3-540-40031-8_24 , abstract =

    work page doi:10.1007/978-3-540-40031-8_24 2003

  54. [54]

    Nicolò De and Sumers, Theodore R

    Sabbata, C. Nicolò De and Sumers, Theodore R. and AlKhamissi, Badr and Bosselut, Antoine and Griffiths, Thomas L. , month = jun, year =. Rational. doi:10.48550/arXiv.2410.05563 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2410.05563

  55. [55]

    Biological Cybernetics , author =

    Path planning versus cue responding: a bio-inspired model of switching between navigation strategies , volume =. Biological Cybernetics , author =. 2010 , keywords =. doi:10.1007/s00422-010-0400-z , abstract =

    work page doi:10.1007/s00422-010-0400-z 2010

  56. [56]

    Hafez, Muhammad Burhan and Weber, Cornelius and Kerzel, Matthias and Wermter, Stefan , month = jul, year =. Curious. 2019. doi:10.1109/IJCNN.2019.8852254 , abstract =

    arXiv 2019

  57. [57]

    Kästner, Linh and Cox, Johannes and Buiyan, Teham and Lambrecht, Jens , month = sep, year =. All-in-. doi:10.48550/arXiv.2109.11636 , abstract =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2109.11636

  58. [60]

    Qu, Xiaoye and Li, Yafu and Su, Zhaochen and Sun, Weigao and Yan, Jianhao and Liu, Dongrui and Cui, Ganqu and Liu, Daizong and Liang, Shuxian and He, Junxian and Li, Peng and Wei, Wei and Shao, Jing and Lu, Chaochao and Zhang, Yue and Hua, Xian-Sheng and Zhou, Bowen and Cheng, Yu , month = mar, year =. A. doi:10.48550/arXiv.2503.21614 , abstract =

    doi:10.48550/arxiv.2503.21614

  59. [61]

    Cognition , author =

    The best of both worlds:. Cognition , author =. 2022 , keywords =. doi:10.1016/j.cognition.2022.105118 , abstract =

    arXiv 2022

  60. [62]

    Current Opinion in Behavioral Sciences , author =

    Doing more with less: meta-reasoning and meta-learning in humans and machines , volume =. Current Opinion in Behavioral Sciences , author =. 2019 , pages =. doi:10.1016/j.cobeha.2019.01.005 , language =

    doi:10.1016/j.cobeha.2019.01.005 2019

  61. [63]

    , year =

    Lieder, Falk and Griffiths, Thomas L. , year =. Strategy selection as rational metareasoning , volume =. Psychological Review , publisher =. doi:10.1037/rev0000075 , abstract =

    work page doi:10.1037/rev0000075

  62. [64]

    and Markman, Arthur B

    Gershman, Samuel J. and Markman, Arthur B. and Otto, A. Ross , year =. Retrospective revaluation in sequential decision making:. Journal of Experimental Psychology: General , publisher =. doi:10.1037/a0030844 , abstract =

    work page doi:10.1037/a0030844

  63. [65]

    Artificial Intelligence , author =

    Between. Artificial Intelligence , author =. 1999 , keywords =. doi:10.1016/S0004-3702(99)00052-1 , abstract =

    doi:10.1016/s0004-3702(99)00052-1 1999

  64. [66]

    Javed, Khurram and Sutton, Richard S , year=. The

  65. [67]

    Young, Kenny and Ramesh, Aditya and Kirsch, Louis and Schmidhuber, Jürgen , month = jul, year =. The. doi:10.48550/arXiv.2211.02222 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2211.02222

  66. [68]

    Artificial Intelligence Review , author =

    Monte. Artificial Intelligence Review , author =. 2023 , keywords =. doi:10.1007/s10462-022-10228-y , abstract =

    doi:10.1007/s10462-022-10228-y 2023

  67. [69]

    Computation and action under bounded resources / , abstract =

    Horvitz, Eric , year =. Computation and action under bounded resources / , abstract =

  68. [70]

    Artificial Intelligence , author =

    Principles of metareasoning , volume =. Artificial Intelligence , author =. 1991 , pages =. doi:10.1016/0004-3702(91)90015-C , abstract =

    doi:10.1016/0004-3702(91)90015-c 1991

  69. [71]

    He, Ruiqi and Jain, Yash Raj and Lieder, Falk , month = jan, year =. Have. doi:10.48550/arXiv.2201.00764 , abstract =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2201.00764

  70. [72]

    Measuring and modelling how people learn how to plan and how people adapt their planning strategies to the structure of the environment , abstract =

  71. [73]

    Rupam and Jagersand, Martin and Tosatto, Samuele , month = oct, year =

    Karimi, Amirmohammad and Jin, Jun and Luo, Jun and Mahmood, A. Rupam and Jagersand, Martin and Tosatto, Samuele , month = oct, year =. Dynamic. 2023. doi:10.1109/IROS55552.2023.10342408 , abstract =

    arXiv 2023

  72. [74]

    Honda, Kohei and Yonetani, Ryo and Nishimura, Mai and Kozuno, Tadashi , month = may, year =. When to. 2024. doi:10.1109/ICRA57147.2024.10611474 , abstract =

    arXiv 2024

  73. [75]

    Sharma, Vishnu Dutt and Lee, Jeongran and Andrews, Matthew and Hadžic, Ilija , year =. Hybrid

  74. [76]

    Proceedings of the Annual Meeting of the Cognitive Science Society , author =

    Enhancing metacognitive reinforcement learningusing reward structures and feedback , volume =. Proceedings of the Annual Meeting of the Cognitive Science Society , author =

  75. [77]

    Rational metareasoning and the plasticity of cognitive control , doi =

    Lieder, Falk and Shenhav, Amitai and Musslick, Sebastian and Griffiths, Thomas , month = feb, year =. Rational metareasoning and the plasticity of cognitive control , doi =

  76. [78]

    How do people learn how to plan? , url =

    Jain, Yash Raj and Gupta, Sanit and Rakesh, Vasundhara and Dayan, Peter and Callaway, Frederick and Lieder, Falk , year =. How do people learn how to plan? , url =. 2019. doi:10.32470/CCN.2019.1313-0 , abstract =

    arXiv 2019

  77. [79]

    Thinker:

    Chung, Stephen and Du, Wenyu and Fu, Jie , month = may, year =. Thinker:. doi:10.48550/arXiv.2505.21097 , abstract =

    work page doi:10.48550/arxiv.2505.21097

  78. [80]

    Proceedings of the AAAI Conference on Artificial Intelligence , author =

    Dynamic. Proceedings of the AAAI Conference on Artificial Intelligence , author =. 2017 , keywords =. doi:10.1609/aaai.v31i1.10918 , abstract =

    work page doi:10.1609/aaai.v31i1.10918 2017

  79. [81]

    Metelli, Alberto Maria and Mazzolini, Flavio and Bisi, Lorenzo and Sabbioni, Luca and Restelli, Marcello , month = nov, year =. Control. Proceedings of the 37th

  80. [82]

    Orenstein, Adrian and Chen, Jessica and Santos, Gwyneth Anne Delos and Sapara, Bayley and Bowling, Michael , month = oct, year =. Toward. doi:10.48550/arXiv.2510.22833 , abstract =

    doi:10.48550/arxiv.2510.22833

Showing first 80 references.