Magnetic sensitivity beyond the projection noise limit by spin squeezing

We report the generation of spin squeezing and entanglement in a magnetically-sensitive atomic ensemble, and entanglement-enhanced field measurements with this system. A maximal Raman coherence is prepared in an ensemble of 8.5x10^5 laser-cooled Rb-87 atoms in the f=1 hyperfine ground state, and the collective spin is squeezed by synthesized optical quantum non-demolition measurement. This prepares a state with large spin alignment and noise below the projection-noise level in a mixed alignment-orientation variable. 3.2dB of noise reduction is observed and 2.0dB of squeezing by the Wineland criterion, implying both entanglement and metrological advantage. Enhanced sensitivity is demonstrated in field measurements using alignment-to-orientation conversion.