Jahn-Teller driven quadrupolar ordering and spin-orbital dimer formation in GaNb₄Se₈

The lacunar spinel GaNb$_4$Se$_8$ is a tetrahedral cluster Mott insulator where spin-orbit coupling on molecular orbitals and Jahn-Teller energy scales are competitive. GaNb$_4$Se$_8$ undergoes a structural and anti-polar ordering transition at T$_Q$ = 50 K that corresponds to a quadrupolar ordering of molecular orbitals on Nb$_4$ clusters. A second transition occurs at T$_M$ = 29 K, where local distortions on the Nb$_4$ clusters rearrange. We present a single crystal x-ray diffraction investigation these phase transitions and solve the crystal structure in the intermediate T$_M$ < T < T$_Q$ and low T < T$_M$ temperature phases. The intermediate phase is a primitive cubic P2$_1$3 structure with a staggered arrangement of Nb4 cluster distortions. A symmetry mode analysis reveals that the transition at TQ is continuous and described by a single Jahn-Teller active amplitude mode. In the low temperature phase, the symmetry of Nb$_4$ clusters is further reduced and the unit cell doubles into an orthorhombic P2$_1$2$_1$2$_1$ space group. Nb$_4$ clusters rearrange through this transition to form a staggered arrangement of intercluster dimers, suggesting a valence bond solid magnetic state.