Topological phase transition in a two-species fermion system: Effects of a rotating trap potential or a synthetic gauge field

We numerically investigate the quantum phases and phase transition in a system made of two species of fermionic atoms that interact with each other via $s$-wave Feshbach resonance, and are subject to rotation or a synthetic gauge field that puts the fermions at Landau level filling factor $\nu_f = 2$. We show that the system undergoes a continuous quantum phase transition from a $\nu_f = 2$ fermionic integer quantum Hall state formed by atoms, to a $\nu_b = 1/2$ bosonic fractional quantum Hall state formed by bosonic diatomic molecules. In the disk geometry we use, these two different topological phases are distinguished by their different gapless edge excitation spectra, and the quantum phase transition between them is signaled by the closing of the energy gap in the bulk. Comparisons will be made with field theoretical predictions, and the case of $p$-wave pairing.