Observation of the in-plane magnetic field-induced phase transitions in FeSe

We investigate the thermodynamic properties of FeSe under the in-plane magnetic fields using torque magnetometry, specific heat, magnetocaloric measurements. Below the upper critical field Hc2, we observed the field-induced anomalies at H1 ~ 15 T and H2 ~ 22 T near H//ab and below a characteristic temperature T* ~ 2 K. The transition magnetic fields H1 and H2 exhibit negligible dependence on both temperature and field orientation. This contrasts with the strong temperature and angle dependence of Hc2, suggesting that these anomalies are attributed to the field-induced phase transitions, originating from the inherent spin-density-wave instability of quasiparticles near the superconducting gap minima or possible Flude-Ferrell-Larkin-Ovchinnikov state in the highly spin-polarized Fermi surfaces. Our observations imply that FeSe, an atypical multiband superconductor with extremely small Fermi energies, represents a unique model system for stabilizing unusual superconducting orders beyond the Pauli limit.