Sequential Bayesian optimal experimental design via approximate dynamic programming
read the original abstract
The design of multiple experiments is commonly undertaken via suboptimal strategies, such as batch (open-loop) design that omits feedback or greedy (myopic) design that does not account for future effects. This paper introduces new strategies for the optimal design of sequential experiments. First, we rigorously formulate the general sequential optimal experimental design (sOED) problem as a dynamic program. Batch and greedy designs are shown to result from special cases of this formulation. We then focus on sOED for parameter inference, adopting a Bayesian formulation with an information theoretic design objective. To make the problem tractable, we develop new numerical approaches for nonlinear design with continuous parameter, design, and observation spaces. We approximate the optimal policy by using backward induction with regression to construct and refine value function approximations in the dynamic program. The proposed algorithm iteratively generates trajectories via exploration and exploitation to improve approximation accuracy in frequently visited regions of the state space. Numerical results are verified against analytical solutions in a linear-Gaussian setting. Advantages over batch and greedy design are then demonstrated on a nonlinear source inversion problem where we seek an optimal policy for sequential sensing.
This paper has not been read by Pith yet.
Forward citations
Cited by 6 Pith papers
-
Subspace accelerated measure transport methods for fast and scalable sequential experimental design, with application to photoacoustic imaging
A derivative-based upper bound on iEIG combined with likelihood-informed subspace projectors and conditional measure transport maps yields a scalable unified framework for sOED and amortized inference in high- and inf...
-
Variational Sequential Optimal Experimental Design using Reinforcement Learning
vsOED uses a variational one-point reward and RL policy optimization to provide a lower bound on expected information gain for sequential experimental design, supporting nuisance parameters, implicit likelihoods, and ...
-
Robust Sequential Experimental Design for A/B Testing
A unified robust framework for sequential A/B testing bounds the worst-case mean squared error of treatment effect estimates under model misspecification in both contextual bandit and dynamic regimes.
-
Adaptive Sensing beyond Non-Adaptive Information Limits: End-to-End Co-Design of Geometry, Policy, and Inference
Joint dynamic programming co-optimizes continuous hardware geometry and Bellman-optimal adaptive policies, yielding large gains over baselines in radar POMDPs, qubit sensors, and 90k-pixel photonic metasensors.
-
Sensor Placement for Tsunami Early Warning via Large-Scale Bayesian Optimal Experimental Design
A reformulation of Bayesian OED as dense matrix subset selection plus a pipelined Schur-complement greedy algorithm on hundreds of GPUs enables optimization of 175-sensor networks for billion-degree-of-freedom tsunami...
-
Optimal experimental design: Formulations and computations
A systematic survey of optimal experimental design covering criteria formulations, estimation and optimization methods, and emerging sequential design policies.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.