Search for New Physics in Rare Top Decays: t bar t Spin Correlations and Other Observables

In this paper we study new-physics contributions to the top-quark decay $t \to b \bar b c$. We search for ways of detecting such new physics via measurements at the LHC. As top quarks are mainly produced at the LHC in $t \bar t$ production via gluon fusion, we analyze the process $gg \to t \bar t \to (b \bar b c) (\bar b \ell \bar \nu)$. We find six observables that can be used to reveal the presence of new physics in $t \to b \bar b c$. Three are invariant mass-squared distributions involving two of the final-state particles in the top decay, and three are angular correlations between the final-state quarks coming from the $t$ decay and the $\ell^-$ coming from the $\bar t$ decay. The angular correlations are related to the $t \bar t$ spin correlation.