U(1) spin liquids and valence bond solids in a large-N three-dimensional Heisenberg model

We study possible quantum ground states of the Sp(N) generalized Heisenberg model on a cubic lattice with nearest-neighbor and next-nearest-neighbor exchange interactions. The phase diagram is obtained in the large-N limit and fluctuation effects are considered via appropriate gauge theories. In particular, we find three U(1) spin liquid phases with different short-range magnetic correlations. These phases are characterized by deconfined gapped spinons, gapped monopoles, and gapless ``photons''. As N becomes smaller, a confinement transition from these phases to valence bond solids (VBS) may occur. This transition is studied by using duality and analyzing the resulting theory of monopoles coupled to a non-compact dual gauge field; the condensation of the monopoles leads to VBS phases. We determine the resulting VBS phases emerging from two of the three spin liquid states. On the other hand, the spin liquid state near J_1 \approx J_2 appears to be more stable against monopole condensation and could be a promising candidate for a spin liquid state in real systems.