Incommensurate magnetic order and phase separation in the two-dimensional Hubbard model with nearest and next-nearest neighbor hopping

We consider the ground state magnetic phase diagram of the two-dimensional Hubbard model with nearest and next-nearest neighbor hopping in terms of electronic density and interaction. We treat commensurate ferro- and antiferromagnetic, as well as incommensurate (spiral) magnetic phases. The first-order magnetic transitions with changing chemical potential, resulting in a phase separation (PS) in terms of density, are found between ferromagnetic, antiferromagnetic and spiral magnetic phases. We argue that the account of PS has a dramatic influence on the phase diagram in the vicinity of half-filling. The results imply possible interpretation of the unusual behavior of magnetic properties of one-layer cuprates in terms of PS between collinear and spiral magnetic phases. The relation of the results obtained to the magnetic properties of ruthenates is also discussed.