New insight into tuning magnetic phases of $R$Mn$_6$Sn$_6$ kagome metals

Predicting magnetic ordering in kagome compounds offers the possibility of harnessing topological or flat-band physical properties through tuning of the magnetism. Here, we examine the magnetic interactions and phases of ErMn$_6$Sn$_6$ which belongs to a family of $R$Mn$_6$Sn$_6$, $R=$ Sc, Y, Gd--Lu, compounds with magnetic kagome Mn layers, triangular $R$ layers, and signatures of topological properties. Using results from single-crystal neutron diffraction and mean-field analysis, we find that ErMn$_6$Sn$_6$ sits close to the critical boundary separating the spiral-magnetic and ferrimagnetic ordered states typical for nonmagnetic versus magnetic $R$ layers, respectively. Finding interlayer magnetic interactions and easy-plane Mn magnetic anisotropy consistent with other members of the family, we predict the existence of a number of temperature and field dependent collinear, noncollinear, and noncoplanar magnetic phases. We show that thermal fluctuations of the Er magnetic moment, which act to weaken the Mn-Er interlayer magnetic interaction and quench the Er magnetic anisotropy, dictate magnetic phase stability. Our results provide a starting point and outline a multitude of possibilities for studying the behavior of Dirac fermions in $R$Mn$_6$Sn$_6$ compounds with control of the Mn spin orientation and real-space spin chirality.