TDCOSMO III: Dark matter substructure meets dark energy -- the effects of (sub)halos on strong-lensing measurements of $H_0$

Time delay cosmography uses the arrival time delays between images in strong gravitational lenses to measure cosmological parameters, in particular the Hubble constant $H_0$. The lens models used in time delay cosmography omit dark matter subhalos and line-of-sight halos because their effects are assumed to be negligible. We explicitly quantify this assumption by analyzing mock lens systems that include full populations of dark matter subhalos and line-of-sight halos, applying the same modeling assumptions used in the literature to infer $H_0$. We base the mock lenses on six quadruply-imaged quasars that have delivered measurements of the Hubble constant, and quantify the additional uncertainties and/or bias on a lens-by-lens basis. We show that omitting dark substructure does not bias inferences of $H_0$. However, perturbations from substructure contribute an additional source of random uncertainty in the inferred value of $H_0$ that scales as the square root of the lensing volume divided by the longest time delay. This additional source of uncertainty, for which we provide a fitting function, ranges from $0.7 - 2.4\%$. It may need to be incorporated in the error budget as the precision of cosmographic inferences from single lenses improves, and sets a precision limit on inferences from single lenses.