Group theory method for extracting order parameters from scanning tunneling microscopy data

Scanning tunneling microscopy (STM) is a powerful local probe of correlated electronic states. Here we present a group theoretical framework for the analysis of STM data, filtering STM images into components which provide a real space mapping of the local symmetry properties of the underlying density of states. Using this formalism, we show that certain kinds of symmetry breaking are impossible to resolve in the first Brillouin zone, due to symmetry restrictions we term ``Bragg peak extinctions'' in analogy with related ideas in x-ray crystallography. We show extinct patterns of symmetry breaking can be resolved using sub-unit cell structure, and develop methodological details for the accurate extraction of this symmetry information. We illustrate our results on synthetic STM data for $2\times 2$ charge density waves on the kagome lattice, and on topographic data for kagome metal ScV$_6$Sn$_6$. Our results provide a powerful method for extracting symmetry insights from STM data, and provide constraints on when and how certain ground states are experimentally observable.