First-Order Transition to Incommensurate Phase with Broken Lattice Rotation Symmetry in Frustrated Heisenberg Model

We study a finite-temperature phase transition in the two-dimensional classical Heisenberg model on a triangular lattice with a ferromagnetic nearest-neighbor interaction $J_1$ and an antiferromagnetic third-nearest-neighbor interaction $J_3$ using a Monte Carlo method. Apart from a trivial degeneracy corresponding to O(3) spin rotations,the ground state for $J_3 \neq 0$ has a threefold degeneracy corresponding to 120 degree lattice rotations. We find that this model exhibits a first-order phase transition with the breaking of the threefold symmetry when the interaction ratio is $J_3/J_1=-3$.