Resolving Multiple-Dynamic Model Uncertainty in Hypothesis-Driven Belief-MDPs

Ofer Dagan; Tyler Becker; Zachary N. Sunberg

REVIEW 3 major objections 5 minor 22 references

Resolving Multiple-Dynamic Model Uncertainty in Hypothesis-Driven Belief-MDPs

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A sparse reward that pays one point when the agent commits to a hypothesis by a deadline lets a POMDP agent resolve model uncertainty without wrecking the original task's performance.

desk verdict Useful multi-model planning idea and clean experiments, but the belief-MDP formalization has a load-bearing flaw: the sparse reward depends on time and a resolved flag that are not part of the state. read the letter →

arxiv 2411.14404 v1 pith:DWLXFZN2 submitted 2024-11-21 cs.AI cs.RO

Ofer Dagan , Tyler Becker , Zachary N. Sunberg This is my paper · ORCID

classification cs.AIcs.RO

keywords hypothesis-drivenplanningbelief-MDPPOMDPmultiple-modelestimationsparserewardMonteCarlotreesearchresolutiontimehybridbelief

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The author is trying to establish that an autonomous agent facing several possible dynamic models of a system can plan actions that both identify the right model and keep doing its original job. To do this, the paper wraps an existing POMDP in a belief-MDP whose belief is a mixture over hypotheses, and adds a reward that fires only when the agent commits to a hypothesis with high confidence before a deadline. Simulation experiments on a tracking problem and a space-domain-awareness problem show this sparse resolution-time reward outperforms an entropy-based reward, giving higher rates of correct in-time decisions and less damage to the base task's reward. The broader claim is that information gathering can be explicitly scheduled and terminated, not just encouraged by a generic information measure.

What carries the argument

The MDH-BMDP tuple $(\bar{B}, A, \bar{T}, \bar{\rho}, \gamma)$ and its generative model: the joint belief is a mixture $\bar{b}(s_x, s_H) = b_H(s_H)\, b_x(s_x \mid s_H)$, each hypothesis carries its own conditional filter (a particle filter for VDPTrack, an unscented Kalman filter for SDA), and a multiple-model update renormalizes hypothesis probabilities through model likelihoods as in Algorithm 3. The new reward $\rho_H$ of Equation (12) is a sparse, terminal, time-bounded indicator that pays only when $\max(b_H) \ge 1-\epsilon$ within $\tau$ steps and no decision has been made yet; it is added with weight $w$ to the base belief reward, making the trade-off between resolution and base performance tunable. This reward is what turns 'reduce uncertainty' into 'decide by a deadline'.

What would settle it

Run the same MDH-BMDP planner on a variant of the SDA scenario where the object's true dynamics is a weighted average of two of the hypothesized models, or an unlisted third model with a similar observation signature. If the resolution-time reward leads the agent to a confident, wrong decision on most runs, while an entropy-based reward keeps uncertainty higher, the claim that this reward reliably resolves model uncertainty would be falsified. A simpler check is to measure the reward collected on episodes where the committed hypothesis is not the true one; the paper reports success rates but not the reward accumulated on wrong decisions.

Watch

Extended reading notes

Core claim

The central claim is that the MDH-BMDP formulation—a belief MDP over the joint space $\bar{B} = B_x \times B_H$, where $B_H$ is a categorical belief over $n_H$ transition-model hypotheses—correctly balances resolving which dynamic model drives the system with performing well in the underlying POMDP. The load-bearing new element is the sparse resolution-time reward of Equation (12), which pays $1.0$ exactly when $\max(b_H) \ge 1-\epsilon$, the time step is no later than a deadline $\tau$, and a decision has not yet been made; this is contrasted with negative entropy, which rewards continued uncertainty reduction without any stopping condition. In the VDPTrack and SDA simulations, the resolution-time reward achieves higher in-time success rates (84–90% versus 76–80%) and higher base reward ($51.3$ vs $46$ and $-8.2$ vs $-13.6$) than the entropy reward, as reported in Table 1. The paper argues that the framework can be instantiated by augmenting an existing POMDP with an array of hypothesis-conditioned POMDPs and multiple-model belief updaters, so existing sparse tree search solvers apply unchanged.

Load-bearing premise

The whole scheme assumes the true system behaves according to exactly one of the listed hypotheses, with no switching between models and no other dynamics; if the truth falls outside the list, the belief will concentrate on the least-wrong option and the reward pushes the agent to commit to it confidently.

Editorial extensions

If this is right

A human operator supervising a cyber-physical system can get a planner that actively probes which fault model is active, and the planner knows when to stop probing and declare a decision.
The same underlying POMDP can be augmented with an arbitrary set of dynamic-model hypotheses without rewriting the solver; only the belief updater and the reward change.
Sparse in-time resolution rewards outperform entropy-based rewards on both a nonlinear tracking problem and a space-domain awareness task, in both decision success and base-task reward.
The weight $w$ provides a calibration knob connecting the base reward scale to the urgency of hypothesis resolution.
The architecture supports hypothesis types beyond dynamics, such as measurement-ambiguity hypotheses, within the same belief-MDP structure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The key mechanism is the deadline: any belief-dependent reward of the form 'pay when confidence crosses a threshold by time $\tau$' should behave similarly, so the improvement over entropy likely comes from encoding a stopping rule rather than from a superior information measure; this is testable by comparing against other thresholded information rewards.
The closed-world assumption has a concrete failure mode: if the true dynamics lies outside the hypothesis set, the sparse reward incentivizes confident commitment to the least-wrong model; a natural extension is adding a 'none of the above' hypothesis with an explicit cost for false commitment, which the paper does not explore.
The framework extends naturally to active model discrimination in safety-critical control, where wrong commitment is costly; the reward could be generalized to asymmetric costs for wrong versus timely decisions.
The entropy reward's continued probing after a decision, visible in the SDA traces, suggests a potential benefit: it reconsiders when new evidence contradicts the decision. A hybrid reward that resumes information gathering when evidence disagrees with the committed hypothesis would test whether the resolution-time reward's stability is always desirable.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The idea is solid: when an operator maintains several candidate dynamics models, augment the POMDP with a hypothesis belief and reward in-time resolution of the right model. The sparse reward in Eq. 12 (credit only when max(b_H) ≥ 1−ε within τ, and only once) is genuinely new relative to the entropy-based rewards in earlier multi-model planning work. The experiments on the VDP-track and space-domain-awareness examples are clear, and the resolution-time reward consistently beats entropy on both success rate and base reward. That is a real empirical result worth a look.

The soft spots, in order. First, the formal core has a gap. Equation 6 defines the belief state as (b_x, b_H), but Equation 12's reward also depends on the time step t and a !resolved flag. Neither is part of the belief state. So as written, the MDH-BMDP is not a well-defined belief-MDP: the reward at a belief-action node is not a function of that node alone. The authors even include !resolved explicitly to avoid double-counting, which makes the reward history-dependent. The generative model in Algorithm 2 has no slot for t or resolved, so either the implementation tracks them behind the scenes (and the formal model is mis-specified) or it does not (and the sparse reward could be collected repeatedly, inflating the value of information-gathering). This is load-bearing because the paper's main contribution is the formulation itself. The fix is straightforward—add time and a resolved flag to the state, or formulate it as a finite-horizon MDP with a termination condition—but the paper as written is internally inconsistent.

Second, the reward weight w is swept on the same test problems used for the headline comparison (Figure 3), and no statistical significance tests are reported. The differences look consistent, but the reader cannot tell how much of the gap is tuning. Third, the true dynamics are assumed to be exactly one of the hypotheses; the paper states this but does not analyze what happens when the real model lies outside the set. That is a practical limitation for operators.

The reader's summary mostly hits these points but misses the formal issue; the stress-test note is right.

Bottom line: this deserves a serious referee—the empirical contribution is useful and the idea is worth building on—but the formal definition must be fixed before acceptance. I would send it to review with a request for major revision, and I would not cite it in its current form.

Referee Report

3 major / 5 minor

Summary. The paper introduces a 'multiple-dynamic hypothesis belief-MDP' (MDH-BMDP) that augments a base POMDP with a set of candidate transition models (hypotheses). The agent maintains a joint belief over the underlying state and over which hypothesis is correct, and plans with sparse tree search. The main proposed reward (Eq. 12) gives a bonus of 1.0 when the hypothesis belief reaches a confidence threshold within a time limit and has not already been collected, added to the base reward with a tunable weight w. The framework is evaluated on a Van Der Pol tracking problem and a space-domain-awareness problem, comparing the resolution-time reward with a negative-entropy reward.

Significance. If the formal issue with the reward is repaired, the paper addresses a genuinely practical and underexplored problem: explicitly planning to resolve model uncertainty while preserving performance on an underlying POMDP task. The framework is built on standard components (POMDPs.jl, MCTS, particle-filter and Gaussian-mixture updaters), and the sparse-resolution reward is a simple, interpretable alternative to entropy-based information rewards. The two simulation studies offer a first demonstration, but the reported quantitative gains currently lack statistical support and are weakened by the tuning procedure.

major comments (3)

[§4.4, Eq. (12) and Algorithm 2] The sparse hypothesis reward in Eq. (12) is not a function of the belief state as defined by Eqs. (6) and (8). It depends on the time index t and on a 'resolved' flag, neither of which appears in b̄ = (b_x, b_H), and the paper explicitly introduces '!resolved' to prevent the reward from being collected more than once. This makes the reward history-dependent, so the object being planned over is not a belief-MDP and the MCTS value backups do not correspond to the stated optimization objective. Please revise the formalization by augmenting the state with a time index and a resolved indicator (with corresponding transitions in Algorithm 2 and Algorithm 3), or by defining a different Markovian reward. This is a load-bearing issue for the paper's central claim, since the MDH-BMDP formulation is itself a contribution.
[§5.3, Figure 3 and Table 1] The weight w is selected by a discrete search over the same test problems that are then used to produce Table 1, and the reported success rates are not accompanied by significance tests or confidence intervals. In VDPTrack the in-time success rates are 84% versus 76% (resolution-time vs entropy) over 50 simulations, a difference that may be within sampling error. Please report confidence intervals or hypothesis tests, or use a separate tuning set, before claiming that the resolution-time reward outperforms the entropy reward on base reward and success rate.
[§3 and §5] The problem statement assumes the true dynamics is exactly one of the n_H hypotheses ('with no transitions between the models'), but the manuscript does not analyze the failure mode when the true dynamics lies outside the hypothesis set. In that case the hypothesis belief will concentrate on the closest model, and the resolution reward in Eq. (12) can drive the agent to commit to a wrong explanation with high confidence. Please add an explicit discussion of this limitation and, ideally, a small experiment with a deliberately misspecified hypothesis set.

minor comments (5)

[Algorithm 3, lines 5-7] In Algorithm 3, line 6 writes to the output field b̄'.b'_x[i] while line 7 evaluates the likelihood using the local variable b'_x[i]; please make explicit that the hypothesis-weight update uses the predicted belief before correction, to avoid ambiguity about a potential double-counting of the observation.
[§4.4] There is a typo in 'We noe focus our attention' ('noe' should be 'now'), and the same paragraph would benefit from a brief explanation of why the entropy reward lacks a mechanism to stop information gathering after a decision is made.
[Figure 1 caption] The caption contains the misspelling 'Plannig'; it should read 'Planning'.
[Table 1] For the SDA baseline (w=0), the base reward of 0 and success rate of 0 are not directly comparable with the other rows because the baseline never attempts hypothesis resolution; please clarify this in the text.
[§5.1] The sentence describing the VDPTrack state space as '9-dimensional ... with the 2D position of the objects and 3 hypothesis states' is ambiguous; the dimension should count the object positions clearly (e.g., three objects times 2D position, plus the hypothesis index).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the MDH-BMDP formulation and sparse-reward comparison are evaluated by forward simulation and do not reduce to their inputs.

full rationale

The paper's central claims are (i) the MDH-BMDP tuple augments a base POMDP with a categorical hypothesis belief, and (ii) the sparse reward in Eq. 12, when combined with the base reward in Eq. 10, yields higher in-time correct-decision rates and base reward than the entropy reward in Eq. 11. Neither claim is obtained by substituting a fitted parameter or a self-citation into the result. The belief update in Algorithm 3 is a standard multiple-model Bayesian update whose likelihoods come from the hypothesis dynamics, not from the quantity being predicted. Eq. 12 is an explicit objective, not a derived prediction; the success metric is measured from independent Monte-Carlo rollouts and includes correctness of the chosen hypothesis, which Eq. 12 does not itself check. The only self-citations (POMDPs.jl, VDP tag) are implementation tools and a benchmark domain; they do not carry the argument. The formal issue that Eq. 12 references t and !resolved, which are not components of the belief state defined in Eq. 8, is a well-posedness/correctness limitation rather than a circular reduction, since the empirical comparison does not assume the reward equals the reported success metric by construction.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The main free parameter is the reward weight w, which is tuned on the same simulation domains used for the final comparison, plus a hand-chosen particle count. The framework assumes the true dynamics is among the hypotheses and that the belief factorizes as a mixture. No new physical entities are introduced.

free parameters (2)

hypothesis reward weight w = VDPTrack resolution-time: w=50 and 75; VDPTrack entropy: w=50; SDA: w=150
Selected by discrete search on the test problems (Fig. 3), then used to produce Table 1. The reported performance advantage depends on this tuning.
number of particles per hypothesis in VDPTrack = 250
The particle filter approximation uses 250 particles per hypothesis; this hand-chosen value affects the quality of the belief update and the probability estimates.

assumptions (5)

domain assumption The true system dynamics is exactly one of the n_H hypotheses, with no transitions between models.
Invoked in the problem statement (Section 3) when defining the hypothesis set; the belief update in Algorithm 3 has no mechanism for a model outside the set, so the agent will eventually commit to the closest model even if none is correct.
domain assumption The joint belief over underlying state and hypothesis factorizes as b(s_x, s_H) = b_H(s_H) * b_x(s_x|s_H).
Used in Equation (8) and in the generative model (Algorithm 2, lines 4-5); this is a structural assumption about the belief representation, not a consequence of the POMDP.
domain assumption Each conditional belief b_x(s_x|s_H) is represented by the same type of filter across hypotheses.
Stated in Section 4.3 as 'without loss of generality'; it enables a shared update implementation but restricts the belief family.
domain assumption Sparse tree search (MCTS) with a generative model provides an adequate approximation of the optimal policy for the MDH-BMDP.
The paper relies on existing MCTS solvers (POMDPs.jl) without optimality guarantees; the sparse reward in Eq. 12 may be hard to explore, though the experiments indicate it works in these domains.
ad hoc to paper The sparse resolution reward in Equation (12) is a faithful operationalization of the user's decision objective.
This reward is introduced by the authors (Section 4.4); its parameters (epsilon, tau) are user-chosen and its adequacy is not derived from first principles.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Resolving Multiple-Dynamic Model Uncertainty in Hypothesis-Driven Belief-MDPs." pith.science (2026). https://pith.science/paper/DWLXFZN2

@misc{pith2026241114404,
  author       = {Pith},
  title        = {Pith review of: Resolving Multiple-Dynamic Model Uncertainty in Hypothesis-Driven Belief-MDPs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DWLXFZN2}},
  note         = {Machine review of arXiv:2411.14404}
}

read the original abstract

When human operators of cyber-physical systems encounter surprising behavior, they often consider multiple hypotheses that might explain it. In some cases, taking information-gathering actions such as additional measurements or control inputs given to the system can help resolve uncertainty and determine the most accurate hypothesis. The task of optimizing these actions can be formulated as a belief-space Markov decision process that we call a hypothesis-driven belief MDP. Unfortunately, this problem suffers from the curse of history similar to a partially observable Markov decision process (POMDP). To plan in continuous domains, an agent needs to reason over countlessly many possible action-observation histories, each resulting in a different belief over the unknown state. The problem is exacerbated in the hypothesis-driven context because each action-observation pair spawns a different belief for each hypothesis, leading to additional branching. This paper considers the case in which each hypothesis corresponds to a different dynamic model in an underlying POMDP. We present a new belief MDP formulation that: (i) enables reasoning over multiple hypotheses, (ii) balances the goals of determining the (most likely) correct hypothesis and performing well in the underlying POMDP, and (iii) can be solved with sparse tree search.

Figures

Figures reproduced from arXiv: 2411.14404 by the authors.

**Figure 2.** MDH-BMDP framework diagram Algorithm 2 Belief Generative Model 1: Input: ¯𝑏, 𝑎, P¯ = { (B¯, A, T¯, 𝜌,𝛾 ¯ ), 𝑃, ® Updaters} 2: 𝜌¯ = P¯.𝜌¯ 3: 𝑏𝑥 = ¯𝑏.𝑏𝑥 , 𝑏H = ¯𝑏.𝑏H 4: 𝑠H ∼ 𝑏H 5: 𝑠𝑥 ∼ 𝑏𝑥 [𝑠H ] 6: 𝑜 ← 𝐺 (𝑃®[𝑠H ], 𝑠𝑥, 𝑎) ⊲ Algorithm 1 7: ¯𝑏 ′ ← update(P¯, ¯𝑏, 𝑎, 𝑜 ) ⊲ Algorithm 3 8: 𝑟 ← 𝜌¯( ¯𝑏, 𝑎, ¯𝑏 ′ , 𝑜 ) 9: return (𝑏 ′ , 𝑜, 𝑟) in the next section, this allows for the belief update of the multihypothesis belief ¯𝑏.… view at source ↗

**Figure 3.** Hypothesis probability results for the VDPTrack [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Probability traces for the VDPTrack (top row), and [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 13 canonical work pages

[1]

Mauricio Araya, Olivier Buffet, Vincent Thomas, and Françcois Charpil- let. 2010. A POMDP Extension with Belief-dependent Rewards. In Advances in Neural Information Processing Systems , Vol. 23. Curran Associates, Inc. https://papers.nips.cc/paper_files/paper/2010/hash/ 68053af2923e00204c3ca7c6a3150cf7-Abstract.html

work page 2010
[2]

Yaakov Bar-Shalom, Fred Daum, and Jim Huang. 2009. The probabilistic data association filter. IEEE Control Systems Magazine 29, 6 (Dec. 2009), 82–100. https: //doi.org/10.1109/MCS.2009.934469 Conference Name: IEEE Control Systems Magazine

work page arXiv 2009
[3]

Yaakov Bar-Shalom, X Rong Li, and Thiagalingam kirubarajan. 2001. Adaptive Estimation and Maneuvering Targets. In Estimation with Applications to Tracking and Navigation. John Wiley & Sons, Ltd, 421–490. http://onlinelibrary.wiley. com/doi/abs/10.1002/0471221279.ch11

doi:10.1002/0471221279.ch11 2001
[4]

Moran Barenboim, Moshe Shienman, and Vadim Indelman. 2023. Monte Carlo Planning in Hybrid Belief POMDPs. IEEE Robotics and Automation Letters 8, 8 (Aug. 2023), 4410–4417. https://doi.org/10.1109/LRA.2023.3282773 Conference Name: IEEE Robotics and Automation Letters

arXiv 2023
[5]

Blackman and Robert Popoli

Samuel S. Blackman and Robert Popoli. 1999. Design and Analysis of Modern Tracking Systems. Artech House. Google-Books-ID: lTIfAQAAIAAJ

work page 1999
[6]

Browne, Edward Powley, Daniel Whitehouse, Simon M

Cameron B. Browne, Edward Powley, Daniel Whitehouse, Simon M. Lucas, Pe- ter I. Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. 2012. A Survey of Monte Carlo Tree Search Methods. IEEE Transactions on Computational Intelligence and AI in Games 4, 1 (March 2012), 1–43. https://doi.org/10.1109/TCIAIG.2012.2186810

arXiv 2012
[7]

Emma Brunskill, Leslie Kaelbling, Tomas Lozano-Perez, and Nicholas Roy. 2008. Continuous-State POMDPs with Hybrid Dynamics. In ISAIM

work page 2008
[8]

Sunberg, Edward Balaban, Tim A

Maxim Egorov, Zachary N. Sunberg, Edward Balaban, Tim A. Wheeler, Jayesh K. Gupta, and Mykel J. Kochenderfer. 2017. POMDPs.jl: A Framework for Sequential Decision Making under Uncertainty. Journal of Machine Learning Research 18, 26 (2017), 1–5. http://jmlr.org/papers/v18/16-300.html

2017

Show all 22 references

[9]

Samuel Fedeler, Marcus Holzinger, and William Whitacre. 2022. Sensor tasking in the cislunar regime using Monte Carlo Tree Search. Advances in Space Research 70, 3 (Aug. 2022), 792–811. https://doi.org/10.1016/j.asr.2022.05.003

2022 doi
[10]

Johannes Fischer and Ömer Sahin Tas. 2020. Information Particle Filter Tree: An Online Algorithm for POMDPs with Belief-Based Rewards on Continuous Domains. In Proceedings of the 37th International Conference on Machine Learning . PMLR, 3177–3187. https://proceedings.mlr.press...

2020
[11]

Ming Hsiao and Michael Kaess. 2019. MH-iSAM2: Multi-hypothesis iSAM using Bayes Tree and Hypo-tree. In 2019 International Conference on Robotics and Automation (ICRA). 1274–1280. https://doi.org/10.1109/ICRA.2019.8793854 ISSN: 2577-087X

2019
[12]

Mangelson, Sudharshan Suresh, Christian Debrunner, and Michael Kaess

Ming Hsiao, Joshua G. Mangelson, Sudharshan Suresh, Christian Debrunner, and Michael Kaess. 2020. ARAS: Ambiguity-aware Robust Active SLAM based on Multi-hypothesis State and Map Estimations. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . ...

2020
[13]

Ajinkya Jain and Scott Niekum. 2017. Belief Space Planning under Approximate Hybrid Dynamics. In Robotics: Science and Systems (RSS) Workshop on POMDPs in Robotics

2017
[14]

A. D. Jaunzemis, M. J. Holzinger, M. W. Chan, and P. P. Shenoy. 2019. Evidence gathering for hypothesis resolution using judicial evidential reasoning. Informa- tion Fusion 49 (Sept. 2019), 26–45. https://doi.org/10.1016/j.inffus.2018.09.010

2019 doi
[15]

S. J. Julier, J. K. Uhlmann, and H. F. Durrant-Whyte. 1995. A new approach for filtering nonlinear systems. In Proceedings of 1995 American Control Conference - ACC’95, Vol. 3. 1628–1632 vol.3. https://doi.org/10.1109/ACC.1995.529783

1995
[16]

Littman, and Anthony R

Leslie Pack Kaelbling, Michael L. Littman, and Anthony R. Cassandra. 1998. Plan- ning and acting in partially observable stochastic domains. Artificial Intelligence 101, 1 (May 1998), 99–134. https://doi.org/10.1016/S0004-3702(98)00023-X

1998 doi
[17]

Kochenderfer, Tim A

Mykel J. Kochenderfer, Tim A. Wheeler, and Kyle H. Wray. 2022. Algorithms for Decision Making. MIT Press

2022
[18]

Zakariya Laouar, Qi Heng Ho, Rayan Mazouz, Tyler Becker, and Zachary N. Sunberg. 2024. Feasibility-Guided Safety-Aware Model Predictive Control for Jump Markov Linear Systems. http://arxiv.org/abs/2310.14116 arXiv:2310.14116 [cs, eess]

2024 arXiv
[19]

Shashank Pathak, Antony Thomas, and Vadim Indelman. 2018. A unified framework for data association aware robust belief space planning and percep- tion. The International Journal of Robotics Research 37, 2-3 (Feb. 2018), 287–315. https://doi.org/10.1177/0278364918759606 Publish...

2018 doi
[20]

Branko Ristic, Sanjeev Arulampalam, and Neil Gordon. 2003. Beyond the Kalman Filter: Particle Filters for Tracking Applications . Artech House

2003
[21]

Moshe Shienman and Vadim Indelman. 2022. D2A-BSP: Distilled Data Association Belief Space Planning with Performance Guarantees Under Budget Constraints. https://doi.org/10.48550/arXiv.2202.04954 arXiv:2202.04954 [cs]

2022 doi arXiv
[22]

Zachary Sunberg and Mykel Kochenderfer. 2018. Online Algorithms for POMDPs with Continuous State, Action, and Observation Spaces. In Proceedings of the International Conference on Automated Planning and Scheduling , Vol. 28. 259–263. https://doi.org/10.1609/icaps.v28i1.13882

2018 doi

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Mauricio Araya, Olivier Buffet, Vincent Thomas, and Françcois Charpil- let. 2010. A POMDP Extension with Belief-dependent Rewards. In Advances in Neural Information Processing Systems , Vol. 23. Curran Associates, Inc. https://papers.nips.cc/paper_files/paper/2010/hash/ 68053af2923e00204c3ca7c6a3150cf7-Abstract.html

work page 2010

[2] [2]

Yaakov Bar-Shalom, Fred Daum, and Jim Huang. 2009. The probabilistic data association filter. IEEE Control Systems Magazine 29, 6 (Dec. 2009), 82–100. https: //doi.org/10.1109/MCS.2009.934469 Conference Name: IEEE Control Systems Magazine

work page arXiv 2009

[3] [3]

Yaakov Bar-Shalom, X Rong Li, and Thiagalingam kirubarajan. 2001. Adaptive Estimation and Maneuvering Targets. In Estimation with Applications to Tracking and Navigation. John Wiley & Sons, Ltd, 421–490. http://onlinelibrary.wiley. com/doi/abs/10.1002/0471221279.ch11

doi:10.1002/0471221279.ch11 2001

[4] [4]

Moran Barenboim, Moshe Shienman, and Vadim Indelman. 2023. Monte Carlo Planning in Hybrid Belief POMDPs. IEEE Robotics and Automation Letters 8, 8 (Aug. 2023), 4410–4417. https://doi.org/10.1109/LRA.2023.3282773 Conference Name: IEEE Robotics and Automation Letters

arXiv 2023

[5] [5]

Blackman and Robert Popoli

Samuel S. Blackman and Robert Popoli. 1999. Design and Analysis of Modern Tracking Systems. Artech House. Google-Books-ID: lTIfAQAAIAAJ

work page 1999

[6] [6]

Browne, Edward Powley, Daniel Whitehouse, Simon M

Cameron B. Browne, Edward Powley, Daniel Whitehouse, Simon M. Lucas, Pe- ter I. Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. 2012. A Survey of Monte Carlo Tree Search Methods. IEEE Transactions on Computational Intelligence and AI in Games 4, 1 (March 2012), 1–43. https://doi.org/10.1109/TCIAIG.2012.2186810

arXiv 2012

[7] [7]

Emma Brunskill, Leslie Kaelbling, Tomas Lozano-Perez, and Nicholas Roy. 2008. Continuous-State POMDPs with Hybrid Dynamics. In ISAIM

work page 2008

[8] [8]

Sunberg, Edward Balaban, Tim A

Maxim Egorov, Zachary N. Sunberg, Edward Balaban, Tim A. Wheeler, Jayesh K. Gupta, and Mykel J. Kochenderfer. 2017. POMDPs.jl: A Framework for Sequential Decision Making under Uncertainty. Journal of Machine Learning Research 18, 26 (2017), 1–5. http://jmlr.org/papers/v18/16-300.html

2017