Markovian Continuity of the MMSE

Minimum mean square error (MMSE) estimation is widely used in signal processing, information theory, and related fields. Despite its practical robustness, the MMSE can be discontinuous under standard notions of stochastic convergence. To bridge this gap, we review classical counterexamples to the continuity of the MMSE and observe that they share a common pathology: along the approximating sequence, the observation is strictly more informative about the limit estimand than the limit observation is. Motivated by practical acquisition mechanisms, we study MMSE continuity under two natural constraints: (1) continuity of the second moment, and (2) a degradedness (Markov) restriction ensuring that each approximating observation is no more informative than the limit observation is about the limit estimand. Under these conditions, we establish continuity of the MMSE and of the MMSE estimator. We provide complementary semicontinuity results and continuity guarantees in related settings and establish continuity under linear estimation. We further extend the analysis to the families of Bregman divergences and continuous metric cost functions, including the Kullback-Leibler and Jensen-Shannon divergences as special cases.