The reviewed record of science sign in
Pith

arxiv: 2607.05359 · v1 · pith:445BXH2U · submitted 2026-07-06 · cs.AI

Graph Sparse Sampling: Breaking the Curse of the Horizon in Continuous MDP Planning

Idan Lev-Yehudi , Vadim Indelman This is my paper

Reviewed by Pith2026-07-07 14:58 UTCglm-5.2pith:445BXH2Uopen to challenge →

classification cs.AI
keywords planningsamplingcontinuousgraphactionhorizonsearchsparse
0
0 comments X

The pith

Shared-future graph planner breaks exponential horizon barrier

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes Graph Sparse Sampling (GSS), an online planning algorithm for continuous Markov Decision Processes that replaces the branching tree structure of traditional sparse sampling and MCTS with a layered graph where successor states are sampled once per depth and shared across all candidate actions. In a tree-based planner, each state-action pair gets its own set of sampled successors, so the number of simulator queries grows exponentially with the planning horizon. GSS instead draws a common next-state layer at each depth and uses self-normalized importance sampling (SNIS) to reweight each state-action pair's evaluation against that shared layer. The central theoretical result (Corollary 4.3) shows that under a bounded density-ratio overlap condition between the true transition distributions and the state proposal distributions, the number of simulator queries scales as Õ(T^5/ε²) — polynomial in the horizon T rather than exponential. This formalizes precisely when sharing sampled futures can avoid the curse of the horizon that afflicts tree-shaped sparse sampling. The paper extends the framework to low-rank generative simulators (where transition densities are singular) via kernel-smoothed SNIS backups with controllable bias, and to continuous action spaces via sampled action sets with coverage guarantees. Experiments on three continuous-control benchmarks show GSS matching or outperforming tree-based planners (DPW, VPW) particularly at longer horizons, while leveraging GPU-friendly fixed-shape batch operations to reach far higher sample counts.

Core claim

The key insight is that the exponential horizon dependence of tree-based sparse sampling is not fundamental to the planning problem itself but is an artifact of the tree data structure, which forces each state-action pair to sample its own independent successors. By sharing a common successor-state layer across all candidate actions at each depth and using self-normalized importance sampling to correct for the mismatch between the proposal distribution and each action's true transition distribution, the sample complexity becomes polynomial in the horizon — provided the proposal distribution overlaps sufficiently with all transition distributions (bounded Rényi divergence). The mechanism that

What carries the argument

The central object is the shared-layer planning graph: at each depth t, a single set of C_t successor states is drawn from a proposal distribution q_t^s, and every candidate (state, action) pair at depth t is evaluated against this same set via SNIS weights ρ = p_t(s'|s,a) / q_t^s(s'). The backward pass propagates value estimates through these weighted backups. The theoretical analysis rests on three pillars: (1) a graph backup concentration inequality (Theorem 1) that recursively bounds the sup-norm error of action-value estimates across the entire graph, combining local SNIS concentration with a Lipschitz stability property of normalized-weight backups; (2) action-side coverage via small-b

If this is right

  • If the overlap condition can be verified or enforced in specific problem classes (e.g., LQR-style systems with Gaussian transitions), GSS provides a principled polynomial-horizon online planner with provable guarantees, filling a gap between exact dynamic programming and heuristic tree search.
  • The GPU-friendly fixed-shape batch structure of GSS suggests that the practical throughput advantage over tree-based planners may grow as hardware parallelism increases, potentially shifting the regime where graph-based planners dominate tree-based ones.
  • The framework extends to partially observable settings if the state proposal and transition model are replaced with belief-space analogues, though the overlap condition may become harder to satisfy in belief space.
  • Adaptive multi-pass variants — where proposals are refined based on earlier value estimates — could relax the proposal-quality sensitivity that currently limits GSS, potentially combining the throughput advantages of graph planning with the adaptivity of tree search.

Load-bearing premise

The polynomial sample complexity bound requires that the Rényi divergence between each action's true transition distribution and the shared state proposal distribution remains uniformly bounded across all states, actions, and depths. If this overlap degrades — for instance, exponentially in the state dimension — the bound becomes vacuous, and the paper does not provide general mechanisms ensuring the overlap stays bounded in high-dimensional problems.

What would settle it

Find a natural continuous-control problem class where the density-ratio overlap d_∞(p_t(·|s,a) || q_t^s) necessarily grows exponentially with state dimension or horizon, making the polynomial guarantee vacuous despite the problem being otherwise well-behaved (smooth dynamics, bounded rewards).

Figures

Figures reproduced from arXiv: 2607.05359 by Idan Lev-Yehudi, Vadim Indelman.

Figure 1
Figure 1. Figure 1: GSS builds a layered planning graph from the root state s0. Sampled states are shown as circles and actions as rectangles. In the forward pass, we draw candidate actions for each state using the (1) action proposal; the next shared state layer is then drawn by the (2) state proposal. At depth T, nodes are evaluated by a (3) tail value. The backward pass applies the (4) graph backup to each node-action pair… view at source ↗
Figure 2
Figure 2. Figure 2: Error bars show ±2 standard errors. We report mean planner performance for displayed domains, over discounted returns for Lunar Lander, or over undiscounted returns for the other domains. In Rotating DDI, the x-axis is the rotation parameter α, which controls the mismatch of the rollout guide and the difficulty of the problem. DPW/VPW curves span wall-clock budgets 0.01, 0.0316, 0.1, 0.316, and 1.0 seconds… view at source ↗
read the original abstract

Planning under uncertainty in continuous domains is essential for autonomous systems, yet computationally demanding. Tree-based search methods such as Monte Carlo Tree Search (MCTS) remain popular, but their branching structure can require sampling budgets that grow exponentially with lookahead depth in the worst case. From a tree perspective, continuous state or action spaces become especially challenging, since the planner must decide where to search in an infinite branching hierarchy. We propose Graph Sparse Sampling (GSS), an online planning algorithm that shares sampled futures across many candidate decisions, rather than sampling separate successors for each candidate action. This branch-free graph exposes large GPU-friendly batches, while using heuristics to focus computation. We prove finite-sample performance guarantees for GSS covering full-rank or low-rank generative simulators via smoothed backups, and discrete or sampled continuous action spaces. Under suitable overlap, regularity, and action-coverage conditions, these bounds have polynomial dependence on the planning horizon, formalizing when shared futures can avoid the exponential horizon dependence of tree-shaped sparse sampling. We demonstrate continuous-control simulations where GSS substantially outperforms tree-based planners on long horizons or achieves near-optimal performance, supporting no-branching graph planning as a complementary design principle for online control.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 5 minor

Summary. The paper proposes Graph Sparse Sampling (GSS), an online planning algorithm for continuous MDPs that replaces per-action sampled subtrees with shared successor-state layers evaluated via self-normalized importance sampling (SNIS) backups. The algorithm is designed for GPU-friendly batched computation. The main theoretical result (Theorem 1) is a recursive concentration inequality bounding the root-action suboptimality in terms of local backup error, action-coverage slack, and tail-value error. Corollary 4.3 instantiates this with SNIS backups under a bounded d_infinity overlap assumption, yielding a simulator-query complexity of eO(R_max^2 (d_max_infinity)^2 T^5 / epsilon^2), which is polynomial in the horizon T when d_max_infinity is itself polynomial. Extensions to low-rank (singular) transition models via kernel smoothing (Claim 1, Corollary 4.2) and to sampled continuous action spaces (Claim 2, Corollary 4.1) are provided. Experiments on three continuous-control benchmarks (Rotating DDI, Lunar Lander, Reacher) compare GSS against tree-based planners (DPW, VPW) and domain-specific baselines.

Significance. The paper makes a genuine algorithmic and theoretical contribution to the literature on continuous-domain online planning. The core idea of sharing sampled successor layers across candidate actions, backed by SNIS graph backups, is a clean departure from tree-shaped sparse sampling and is well-motivated by modern GPU batched computation. The finite-sample guarantees (Theorem 1, Corollary 4.3) are derived from standard concentration inequalities (SNIS bounds from Lim et al. 2023) and are not fitted to data; the sample complexity is a genuine consequence of the assumed overlap, regularity, and action-coverage conditions. The extension to low-rank generative simulators via smoothed backups (Claim 1, Corollary 4.2) addresses a practically important case. The experiments demonstrate that the approach can outperform tree-based planners in specific regimes, particularly with degraded rollout guides. The paper is honest about its limitations, including the proposal-limited nature of GSS.

major comments (1)
  1. §4.2, Corollary 4.3 and Remark 1: The headline claim of 'breaking the curse of the horizon' is conditional on d_max_infinity = sup_{t,s,a} d_infinity(p_t(.|s,a) || q^s_t) being polynomial in T (or constant). The paper acknowledges this in Remark 1 ('If d_max_infinity = O(T^n)...'), but provides no mechanism ensuring this holds in general continuous state spaces. The d_infinity divergence is the essential supremum of the density ratio, which in n_s-dimensional state spaces typically grows exponentially with n_s for any fixed proposal q^s_t that does not adapt to local transition geometry. The FitProposal routine (Algorithm 1, line 5) is heuristic and no proof is given that it controls d_max_infinity. This is the central gap between the theoretical guarantee and the headline claim: the bound is correct given its assumptions, but the assumptions may themselves reintroduce exponential (in n_
minor comments (5)
  1. §3.1, Eq. (1.A)-(1.D): The notation for the backup variants is introduced somewhat abruptly. A brief sentence clarifying that (1.A)-(1.D) are four alternative instantiations of the abstract backup B_t would help the reader.
  2. §5, Figure 2a: The x-axis label 'rotation' is ambiguous; it would benefit from the symbol alpha used in the text.
  3. §5, Figure 2b: The y-axis label 'discounted return' is clear, but the tree-based planner curves show declining performance with increasing time budget, which is counterintuitive. A brief discussion of why this occurs (e.g., overfitting to rollout noise, exploration-exploitation imbalance at higher budgets) would strengthen the presentation.
  4. Appendix B, Table 2: The GSS 'xs' configuration has a very large standard error (30.763 +/- 20.144) compared to other configurations. This should be flagged or discussed.
  5. §2: The citation 'Barenboim and Indelman, 2026' has a future publication year; this may be a typo.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for a careful and constructive report. The referee correctly identifies the central theoretical gap: the headline claim of 'breaking the curse of the horizon' is conditional on d_max_infinity being polynomial in T, and the paper does not provide a mechanism guaranteeing this in general continuous state spaces. We agree this gap between the conditional guarantee and the headline claim must be addressed more carefully in the revision. We propose targeted revisions to the title, abstract, and Remark 1 to make the conditional nature of the result fully transparent, to add an explicit discussion of when d_max_infinity can and cannot be controlled, and to reframe the FitProposal routine's role honestly as heuristic without theoretical guarantee. We do not claim to close the gap entirely—no general mechanism exists—but we will make the scope and limitations unambiguous.

read point-by-point responses

  1. Referee: §4.2, Corollary 4.3 and Remark 1: The headline claim of 'breaking the curse of the horizon' is conditional on d_max_infinity being polynomial in T (or constant). The paper acknowledges this in Remark 1 but provides no mechanism ensuring this holds in general continuous state spaces. The d_infinity divergence typically grows exponentially with n_s for any fixed proposal q^s_t that does not adapt to local transition geometry. The FitProposal routine is heuristic and no proof is given that it controls d_max_infinity. This is the central gap between the theoretical guarantee and the headline claim.

    Authors: The referee is correct on all counts. We acknowledge the following: (1) The bound in Corollary 4.3 is conditional on d_max_infinity being polynomial in T, and no general mechanism in arbitrary continuous state spaces guarantees this. (2) For a fixed, non-adaptive proposal q^s_t in an n_s-dimensional state space, d_infinity can indeed grow exponentially in n_s, potentially reintroducing exponential dependence. (3) The FitProposal routine (Algorithm 1, line 5) is presented as a heuristic component of the algorithm, and we provide no proof that it controls d_max_infinity in general. This is a genuine gap between the conditional theoretical guarantee and the headline framing. We will make the following revisions: First, we will reframe the title and abstract to state that GSS achieves polynomial horizon dependence *under suitable overlap conditions* rather than unconditionally 'breaking the curse of the horizon.' The phrase 'breaking the curse of the horizon' will be qualified or replaced with more precise language such as 'polynomial horizon sample complexity under density overlap conditions.' Second, we will expand Remark 1 to explicitly discuss the scenarios in which d_max_infinity can remain polynomial: (a) when the transition model has bounded support or light tails and the proposal is chosen to cover the effective support, (b) when the state space is compact and the transition densities are uniformly bounded above and below (as in some controlled diffusion models), and (c) when the proposal is adapted to approximate the transition mixture over the relevant state-action pairs, as our transition-moment mixture proposals attempt in experiments. We will also explicitly state the scenarios where d_max_infinity is expected to grow exponentially, namely high-dimensional full revision: no

  2. Referee: Continuation of the above: the bound is correct given its assumptions, but the assumptions may themselves reintroduce exponential (in n_s) dependence, undermining the headline claim.

    Authors: We agree that this is a fair characterization. The bound is correct given its assumptions, and the assumptions may reintroduce exponential dependence in the state dimension n_s. Our contribution is identifying the *structural* mechanism—shared successor layers with SNIS backups—that *can* yield polynomial horizon dependence when the overlap condition holds, in contrast to tree-based sparse sampling where exponential horizon dependence is fundamental and cannot be avoided by any overlap condition on a shared proposal. We will make this contrast explicit in the revision: GSS does not claim to break the curse of the horizon in all continuous MDPs; it identifies the conditions under which shared-future graph planning avoids the exponential horizon dependence that is inherent to tree-shaped sparse sampling. The state-dimensional dependence enters through d_max_infinity, and we will state this transparently. We believe this conditional result is still a genuine contribution—it characterizes precisely when the algorithmic innovation of shared successor layers yields a theoretical advantage over tree-based methods—but we agree the headline must not overstate the scope. revision: no

standing simulated objections not resolved
  • No general mechanism exists to guarantee d_max_infinity is polynomial in T for arbitrary continuous MDPs. This is a fundamental limitation of importance-sampling-based approaches in high-dimensional spaces, and we cannot fully close this gap. The revision will make the conditional nature of the result transparent rather than claiming to solve it.

Circularity Check

0 steps flagged

No circularity found. The derivation chain is self-contained, built on external concentration inequalities, and the self-citations are non-load-bearing.

full rationale

The paper's main result (Corollary 4.3) is derived through a transparent chain: external SNIS concentration theorems (Lim et al. 2023, Dau 2022 — both independent groups) supply the local backup bound (Assumption 3); a Lipschitz stability argument (Assumption 4) supplies recursive error propagation (Claim 3); Assumptions 1–2 supply action-coverage bounds (Claim 2); and backward induction with union bounds yields Theorem 1. Corollary 4.3 then substitutes the SNIS parameters into Theorem 1's framework. No step reduces to its inputs by construction. The self-citations (Lev-Yehudi et al. 2025, Barenboim and Indelman 2026) appear only in related work and are not invoked in any proof. The d∞_max overlap parameter is an explicitly stated assumption about the problem class, not a fitted parameter renamed as a prediction, and the paper openly acknowledges in Remark 1 that the polynomial bound is conditional on d∞_max remaining polynomial. This is a correctness/applicability concern, not circularity. The sample complexity bound is not fitted to data; it is a parameter-free derivation from stated assumptions. No circularity patterns (self-definitional, fitted-input-as-prediction, self-citation load-bearing, uniqueness-imported, ansatz-smuggled, or renaming) are present.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The paper does not invent new physical entities or mathematical objects outside of the algorithm itself. The free parameters are standard algorithmic budgets and the smoothing bandwidth. The axioms are a mix of standard concentration assumptions and domain regularity conditions, with the overlap condition being the most load-bearing and potentially restrictive.

free parameters (4)
  • d_max_infinity
    The uniform bound on the exponentiated Renyi divergence between the true transition and the state proposal. This is an assumed property of the domain and proposal, not a fitted parameter, but it determines the sample complexity.
  • tau_t (smoothing bandwidth) = e.g., 0.02 in Lunar Lander
    Chosen per-domain to control smoothing bias in low-rank simulator settings.
  • C_t (state layer widths) = e.g., 512, 2048
    Algorithmic budget parameters chosen empirically or via theoretical schedules.
  • K_t (action budget) = e.g., 24, 64
    Number of candidate actions sampled per node, chosen empirically.
axioms (5)
  • domain assumption Assumption 1 (Proposal small-ball coverage): The action proposal has sufficient mass near optimal actions.
    Required for Claim 2 to bound the action-side approximation error.
  • domain assumption Assumption 2 (Action-side Holder regularity): Q* is Holder continuous in actions.
    Required for Claim 2 to bound the action-side approximation error.
  • standard math Assumption 3 (Graph backup bound): The backup estimator concentrates around the true value.
    Standard concentration inequality assumption, satisfied by SNIS under overlap.
  • standard math Assumption 4 (Graph backup stability): The backup is Lipschitz in child value estimates.
    Required for recursive error propagation in Claim 3.
  • domain assumption Bounded d_infinity overlap: sup d_infinity(p_t || q^s_t) < infinity.
    The central assumption for Corollary 4.3, ensuring SNIS variance is controlled. Its practicality is the weakest link.

pith-pipeline@v1.1.0-glm · 29779 in / 2380 out tokens · 332659 ms · 2026-07-07T14:58:33.441910+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 33 canonical work pages · 1 internal anchor

  1. [1]

    Continuous upper confidence trees with polynomial exploration--consistency

    David Auger, Adrien Couetoux, and Olivier Teytaud. Continuous upper confidence trees with polynomial exploration--consistency. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2013, Prague, Czech Republic, September 23-27, 2013, Proceedings, Part I 13, pages 194--209. Springer, 2013

    work page 2013

  2. [2]

    Barenboim and V

    M. Barenboim and V. Indelman. Online POMDP planning with anytime deterministic optimality guarantees. Artificial Intelligence, 350: 0 104442, 2026. doi:https://doi.org/10.1016/j.artint.2025.104442

    work page doi:10.1016/j.artint.2025.104442 2026

  3. [3]

    Semitractability of optimal stopping problems via a weighted stochastic mesh algorithm

    Denis Belomestny, Maxim Kaledin, and John Schoenmakers. Semitractability of optimal stopping problems via a weighted stochastic mesh algorithm. Mathematical Finance, 30 0 (4): 0 1591--1616, 2020

    work page 2020

  4. [4]

    Weighted mesh algorithms for general markov decision processes: Convergence and tractability

    Denis Belomestny, John Schoenmakers, and Veronika Zorina. Weighted mesh algorithms for general markov decision processes: Convergence and tractability. Journal of Complexity, 88: 0 101932, 2025

    work page 2025

  5. [5]

    JAX : composable transformations of P ython+ N um P y programs, 2018

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Yash Katariya, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake Vander P las, Skye Wanderman- M ilne, and Qiao Zhang. JAX : composable transformations of P ython+ N um P y programs, 2018. URL http://github.com/jax-ml/jax

    work page 2018

  6. [6]

    A stochastic mesh method for pricing high-dimensional american options

    Mark Broadie, Paul Glasserman, et al. A stochastic mesh method for pricing high-dimensional american options. Journal of Computational Finance, 7: 0 35--72, 2004

    work page 2004

  7. [7]

    A survey of Monte Carlo tree search methods

    Cameron B Browne, Edward Powley, Daniel Whitehouse, Simon M Lucas, Peter I Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. A survey of Monte Carlo tree search methods. IEEE Transactions on Computational Intelligence and AI in games, 4 0 (1): 0 1--43, 2012

    work page 2012

  8. [8]

    Continuous upper confidence trees

    Adrien Cou \"e toux, Jean-Baptiste Hoock, Nataliya Sokolovska, Olivier Teytaud, and Nicolas Bonnard. Continuous upper confidence trees. In International conference on learning and intelligent optimization, pages 433--445. Springer, 2011

    work page 2011

  9. [9]

    Sequential Bayesian Computation

    Hai Dang Dau. Sequential Bayesian Computation . PhD thesis, Institut Polytechnique de Paris , September 2022. URL https://theses.hal.science/tel-03848268

    work page 2022

  10. [10]

    POMDPs.jl : A framework for sequential decision making under uncertainty

    Maxim Egorov, Zachary N Sunberg, Edward Balaban, Tim A Wheeler, Jayesh K Gupta, and Mykel J Kochenderfer. POMDPs.jl : A framework for sequential decision making under uncertainty. The Journal of Machine Learning Research, 18 0 (1): 0 831--835, 2017

    work page 2017

  11. [11]

    Julier, Jeffrey, and K

    Simon J. Julier, Jeffrey, and K. Uhlmann. Unscented filtering and nonlinear estimation. In Proceedings of the IEEE, pages 401--422, 2004

    work page 2004

  12. [12]

    A sparse sampling algorithm for near-optimal planning in large M arkov decision processes

    Michael Kearns, Yishay Mansour, and Andrew Y Ng. A sparse sampling algorithm for near-optimal planning in large M arkov decision processes. Machine learning, 49 0 (2): 0 193--208, 2002

    work page 2002

  13. [13]

    Monte Carlo tree search in continuous spaces using Voronoi optimistic optimization with regret bounds

    Beomjoon Kim, Kyungjae Lee, Sungbin Lim, Leslie Kaelbling, and Tom \'a s Lozano-P \'e rez. Monte Carlo tree search in continuous spaces using Voronoi optimistic optimization with regret bounds. In AAAI Conf. on Artificial Intelligence, volume 34, pages 9916--9924, 2020

    work page 2020

  14. [14]

    Bayesian estimates of equation system parameters: an application of integration by Monte Carlo

    Teun Kloek and Herman K Van Dijk. Bayesian estimates of equation system parameters: an application of integration by Monte Carlo . Econometrica: Journal of the Econometric Society, pages 1--19, 1978

    work page 1978

  15. [15]

    Bandit based M onte- C arlo planning

    Levente Kocsis and Csaba Szepesv \'a ri. Bandit based M onte- C arlo planning. In European conference on machine learning, pages 282--293. Springer, 2006

    work page 2006

  16. [16]

    Continuous Monte Carlo graph search

    Kalle Kujanp \"a \"a , Amin Babadi, Yi Zhao, Juho Kannala, Alexander Ilin, and Joni Pajarinen. Continuous Monte Carlo graph search. In Intl. Conf. on Autonomous Agents and Multiagent Systems (AAMAS), pages 1047--1056, 2024

    work page 2024

  17. [17]

    Monte-Carlo tree search in continuous action spaces with value gradients

    Jongmin Lee, Wonseok Jeon, Geon-Hyeong Kim, and Kee-Eung Kim. Monte-Carlo tree search in continuous action spaces with value gradients. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 4561--4568, 2020

    work page 2020

  18. [18]

    Monte-Carlo graph search: the value of merging similar states

    Edouard Leurent and Odalric-Ambrym Maillard. Monte-Carlo graph search: the value of merging similar states. In Sinno Jialin Pan and Masashi Sugiyama, editors, Asian Conference on Machine Learning (ACML 2020), pages 577 -- 592, Bangkok, Thailand, November 18-20 2020

    work page 2020

  19. [19]

    Value gradients with action adaptive search trees in continuous (po)mdps

    Idan Lev-Yehudi, Michael Novitsky, Moran Barenboim, Ron Benchetrit, and Vadim Indelman. Value gradients with action adaptive search trees in continuous (po)mdps. 2025

    work page 2025

  20. [20]

    Voronoi progressive widening: efficient online solvers for continuous state, action, and observation POMDPs

    Michael H Lim, Claire J Tomlin, and Zachary N Sunberg. Voronoi progressive widening: efficient online solvers for continuous state, action, and observation POMDPs . In 2021 60th IEEE conference on decision and control (CDC), pages 4493--4500. IEEE, 2021

    work page 2021

  21. [21]

    Optimality guarantees for particle belief approximation of POMDPs

    Michael H Lim, Tyler J Becker, Mykel J Kochenderfer, Claire J Tomlin, and Zachary N Sunberg. Optimality guarantees for particle belief approximation of POMDPs . Journal of Artificial Intelligence Research, 77: 0 1591--1636, 2023

    work page 2023

  22. [22]

    Sample-based planning for continuous action markov decision processes

    Chris Mansley, Ari Weinstein, and Michael Littman. Sample-based planning for continuous action markov decision processes. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 21, pages 335--338, 2011

    work page 2011

  23. [23]

    Poly-hoot: Monte-carlo planning in continuous space mdps with non-asymptotic analysis

    Weichao Mao, Kaiqing Zhang, Qiaomin Xie, and Tamer Basar. Poly-hoot: Monte-carlo planning in continuous space mdps with non-asymptotic analysis. volume 33, pages 4549--4559, 2020

    work page 2020

  24. [24]

    Bayesian optimized Monte Carlo planning

    John Mern, Anil Yildiz, Zachary Sunberg, Tapan Mukerji, and Mykel J Kochenderfer. Bayesian optimized Monte Carlo planning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11880--11887, 2021

    work page 2021

  25. [25]

    Art B. Owen. Monte Carlo theory, methods and examples. https://artowen.su.domains/mc/, 2013

    work page 2013

  26. [26]

    Papadimitriou and John N

    Christos H. Papadimitriou and John N. Tsitsiklis. The complexity of Markov decision processes. Mathematics of operations research, 12 0 (3): 0 441--450, 1987

    work page 1987

  27. [27]

    Nonasymptotic analysis of Monte Carlo Tree Search

    Devavrat Shah, Qiaomin Xie, and Zhi Xu. Nonasymptotic analysis of Monte Carlo Tree Search . Operations Research, 70 0 (6): 0 3234--3260, 2022

    work page 2022

  28. [28]

    MuJoCo : A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. MuJoCo : A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026--5033. IEEE, 2012. doi:10.1109/IROS.2012.6386109

    work page doi:10.1109/iros.2012.6386109 2012

  29. [29]

    Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U. Balis, Gianluca De Cola, Tristan Deleu, Manuel Goul \ a o, Andreas Kallinteris, Markus Krimmel, Arjun KG, Rodrigo Perez-Vicente, Andrea Pierr \'e , Sander Schulhoff, Jun Jet Tai, Hannah Tan, and Omar G. Younis. Gymnasium: A standard interface for reinforcement learning environments. arXiv preprint arXi...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  30. [30]

    R \'e nyi divergence and kullback-leibler divergence

    Tim Van Erven and Peter Harremos. R \'e nyi divergence and kullback-leibler divergence. IEEE Transactions on Information Theory, 60 0 (7): 0 3797--3820, 2014

    work page 2014

  31. [31]

    Kernel smoothing

    Matt P Wand and M Chris Jones. Kernel smoothing. CRC press, 1994

    work page 1994

  32. [32]

    Theodorou

    Grady Williams, Andrew Aldrich, and Evangelos A. Theodorou. Model predictive path integral control: From theory to parallel computation. Journal of Guidance, Control, and Dynamics, 40 0 (2): 0 344--357, 2017. doi:10.2514/1.G001921

    work page doi:10.2514/1.g001921 2017

  33. [33]

    Monte Carlo Tree Search in continuous action spaces with execution uncertainty

    Timothy Yee, Viliam Lis \'y , and Michael Bowling. Monte Carlo Tree Search in continuous action spaces with execution uncertainty. In Subbarao Kambhampati, editor, IJCAI, pages 690--697, 2016

    work page 2016