Mapping class groups of medium distance Heegaard splittings

We show that if the Hempel distance of a Heegaard splitting is larger than three then the mapping class group of the Heegaard splitting is isomorphic to a subgroup of the mapping class group of the ambient 3-manifold. This implies that given two handlebody sets in the curve complex for a surface that are distance at least four apart, the group of automorphisms of the curve complex that preserve both handlebody sets is finite.