Length dependence of waveform mismatch: a caveat on waveform accuracy

The Simulating eXtreme Spacetimes Collaboration's code SpEC can now routinely simulate binary black hole mergers undergoing $\sim25$ orbits, with the longest simulations undergoing nearly $\sim180$ orbits. While this sounds impressive, the mismatch between the highest resolutions for this long simulation is $\mathcal{O}(10^{-1})$. Meanwhile, the mismatch between resolutions for the more typical simulations tends to be $\mathcal{O}(10^{-4})$, despite the resolutions being similar to the long simulations'. In this note, we explain why mismatch alone gives an incomplete picture of code -- and waveform -- quality, especially in the context of providing waveform templates for LISA and 3G detectors, which require templates with $\mathcal{O}(10^{3}) - \mathcal{O}(10^{5})$ orbits. We argue that to ready the GW community for the sensitivity of future detectors, numerical relativity groups must be aware of this caveat, and also run future simulations with at least three resolutions to properly assess waveform accuracy.