The A Priori Estimate and Existence of the Positive Solution for A Nonlinear System Involving the Fractional Laplacian

In the paper, we consider the fractional elliptic system \begin{equation*}\left\{\begin{array}{ll} (- \Delta)^{\frac{\alpha_1}{2}}u(x)+\sum\limits^n_{i=1}b_i(x)\frac{\partial u}{\partial x_i}+B(x)u(x)=f(x,u,v),& \mbox { in } \Omega,\\ (- \Delta)^{\frac{\alpha_2}{2}}v(x)+\sum\limits^n_{i=1}c_i(x)\frac{\partial v}{\partial x_i}+C(x)v(x)=g(x,u,v),& \mbox { in } \Omega,\\ u=v=0, & \mbox { in } \mathbb{R}^n\setminus\Omega, \end{array} \right.\label{a-1.2} \end{equation*} where $\Omega$ is a bounded domain with $C^2$ boundary in $\mathbb{R}^n$ and $n>\max\{\alpha_1,\alpha_2\}$. We first utilize the blowing-up and re-scaling method to derive the a priori estimate for positive solutions when $1<\alpha_1,\alpha_2 <2$. Then for $0<\alpha_1,\alpha_2 <1$, we obtain the regularity estimate of positive solutions. On top of this, using the topological degree theory we prove the existence of positive solutions.