The impact of extragalactic foregrounds on internal delensing of CMB B-mode polarization

The search for primordial $B$-mode polarization of the CMB is limited by the sample variance of $B$-modes produced at later times by gravitational lensing. Constraints can be improved by `delensing': using some proxy of the matter distribution to partially remove the lensing-induced $B$-modes. Current and soon-upcoming experiments will infer a matter map -- at least in part -- from the temperature anisotropies of the CMB. These reconstructions are contaminated by extragalactic foregrounds: radio-emitting galaxies, the cosmic infrared background, or the Sunyaev--Zel'dovich effects. Using the Websky simulations, we show that the foregrounds add spurious power to the angular auto-spectrum of delensed $B$-modes via non-Gaussian higher-point functions, biasing constraints on the tensor-to-scalar ratio, $r$. We consider an idealized experiment similar to the Simons Observatory, with no Galactic or atmospheric foregrounds. After removing point sources detectable at 143 GHz and reconstructing lensing from CMB temperature modes $l<3500$ using a Hu-Okamoto quadratic estimator (QE), we infer a value of $r$ that is $1.5\,\sigma$ higher than the true $r=0$. Reconstructing instead from a minimum-variance ILC map only exacerbates the problem, bringing the bias above $3\,\sigma$. When the $TT$ estimator is co-added with other QEs or with external matter tracers, new couplings ensue which partially cancel the diluted bias from $TT$. We provide a simple and effective prescription to model these effects. In addition, we demonstrate that the point-source-hardened or shear-only QEs can not only mitigate the biases to acceptable levels, but also lead to lower power than the Hu-Okamoto QE after delensing. Thus, temperature-based reconstructions remain powerful tools in the quest to measure $r$.