Learning Broken Symmetries with Resimulation and Encouraged Invariance

Recognizing symmetries in data allows for significant boosts in neural network training. In many cases, however, the underlying symmetry is present only in an idealized dataset, and is broken in the training data, due to effects such as arbitrary and/or non-uniform detector bin edges. Standard approaches, such as data augmentation or equivariant networks fail to represent the nature of the full, broken symmetry. We introduce a novel data-augmentation scheme that respects the true underlying symmetry and avoids artifacts by augmenting the training set with transformed pre-detector examples whose detector response is then resimulated. In addition, we encourage the network to treat the augmented copies identically, allowing it to learn the broken symmetry. While the technique can be extended to other symmetries, we demonstrate its application on rotational symmetry in particle physics calorimeter images. We find that neural networks trained with pre-detector rotations converge to a solution more quickly than networks trained with standard post-detector augmentation, and that networks modified to encourage similar internal treatment of augmentations of the same input converge even faster.