Evidence for competing magnetic instabilities in underdoped rm YBa₂Cu₃O_{6+x}

We report a polarized neutron scattering study of the orbital-like magnetic order in strongly underdoped ${\rm YBa_2Cu_3O_{6.45}}$ and ${\rm YBa_2(Cu_{0.98}Zn_{0.02})_3O_{6.6}}$. Their hole doping levels are located on both sides of the critical doping $p_{MI}$ of a metal-insulator transition inferred from transport measurements. Our study reveals a drop down of the orbital-like order slightly below $p_{MI}$ with a steep decrease of both the ordering temperature $T_{mag}$ and the ordered moment. Above $p_{MI}$, substitution of quantum impurities does not change $T_{mag}$, whereas it lowers significantly the bulk ordered moment. The modifications of the orbital-like magnetic order are interpreted in terms of a competition with electronic liquid crystal phases around $p_{MI}$. This competition gives rise to a mixed magnetic state in ${\rm YBa_2Cu_3O_{6.45}}$ and a phase separation in ${\rm YBa_2(Cu_{0.98}Zn_{0.02})_3O_{6.6}}$.