Impurity-potential-induced gap at the Dirac point of topological insulators with in-plane magnetization

The quantum anomalous Hall effect (QAHE), characterized by dissipationless quantized edge transport, relies crucially on a non-trivial topology of the electronic bulk bandstructure and a robust ferromagnetic order that breaks time-reversal symmetry. Magnetically-doped topological insulators (TIs) satisfy both these criteria, and are the most promising quantum materials for realizing the QAHE. Because the spin of the surface electrons aligns along the direction of magnetic-impurity exchange field, only magnetic TIs with an out-of-plane magnetization are thought to open a gap at the Dirac point (DP) of the surface states, resulting in the QAHE. Using a continuum model supported by atomistic tight-binding and first-principles calculations of transition-metal doped Bi$_2$Se$_3$, we show that a surface-impurity potential generates an additional effective magnetic field which spin-polarizes the surface electrons along the direction perpendicular to the surface. The predicted gap-opening mechanism results from the interplay of this additional field and the in-plane magnetization that shifts the position of the DP away from the $\Gamma$ point. This effect is similar to the one originating from the hexagonal warping correction of the bandstructure but is one order of magnitude stronger. Our calculations show that in a doped TI with in-plane magnetization the impurity-potential-induced gap at the DP is comparable to the one opened by an out-of-plane magnetization.