Dynamic Programming for controlled Markov families: abstractly and over Martingale Measures

We describe an abstract control-theoretic framework in which the validity of the dynamic programming principle can be established in continuous time by a verification of a small number of structural properties. As an application we treat several cases of interest, most notably the lower-hedging and utility-maximization problems of financial mathematics both of which are naturally posed over ``sets of martingale measures''.