On Galactic density modeling in the presence of dust extinction

Inferences about the spatial density or phase-space structure of stellar populations in the Milky Way require a precise determination of the effective survey volume. The volume observed by surveys such as Gaia or near-infrared spectroscopic surveys, which have good coverage of the Galactic mid-plane region, is highly complex because of the abundant small-scale structure in the three-dimensional interstellar dust extinction. We introduce a novel framework for analyzing the importance of small-scale structure in the extinction. This formalism demonstrates that the spatially-complex effect of extinction on the selection function of a pencil-beam or contiguous sky survey is equivalent to a low-pass filtering of the extinction-affected selection function with the smooth density field. We find that the angular resolution of current 3D extinction maps is sufficient for analyzing Gaia sub-samples of millions of stars. However, the current distance resolution is inadequate and needs to be improved by an order of magnitude, especially in the inner Galaxy. We also present a practical and efficient method for properly taking the effect of extinction into account in analyses of Galactic structure through an effective selection function. We illustrate its use with the selection function of red-clump stars in APOGEE using and comparing a variety of current 3D extinction maps.