Competition between antiferromagnetic instability and frustrations in Fe-Se

Using linear-response density-functional theory, we obtain the magnetic interactions in the several iron pnictides. The ground state has been found to be non-collinear in FeSe, with a large continuum of nearly degenerate states lying very close to the magnetic "striped" structure. The presence of non-collinearity also seems to be a generic feature of iron pnictides when the Fe moment is small. At small R_{Fe-Se} the system is itinerant: strong frustration give rise to excess of spin entropy, long ranged interactions create incommensurate orderings and strong biquadratic (or ring) couplings violate the applicability of Heisenberg model. There is a smooth transition to more localized behavior as R_{Fe-Se} increases: stable magnetic orbital order develops which favor long range AFM stripe ordering with strongly anisotropic in-plane exchange couplings. The stabilization of the stripe magnetic order is accompanied by the inversion of the exchange coupling.