Sharp heat kernel estimates for relativistic stable processes in open sets

In this paper, we establish sharp two-sided estimates for the transition densities of relativistic stable processes [i.e., for the heat kernels of the operators $m-(m^{2/\alpha}-\Delta)^{\alpha/2}$] in $C^{1,1}$ open sets. Here $m>0$ and $\alpha\in(0,2)$. The estimates are uniform in $m\in(0,M]$ for each fixed $M>0$. Letting $m\downarrow0$, we recover the Dirichlet heat kernel estimates for $\Delta^{\alpha/2}:=-(-\Delta)^{\alpha/2}$ in $C^{1,1}$ open sets obtained in [14]. Sharp two-sided estimates are also obtained for Green functions of relativistic stable processes in bounded $C^{1,1}$ open sets.