Comparison of algorithms for determination of rotation measure and Faraday structure I. 1100 - 1400 MHz

(abridged) We run a Faraday structure determination data challenge to benchmark the currently available algorithms including Faraday synthesis (previously called RM synthesis in the literature), wavelet, compressive sampling and $QU$-fitting. The frequency set is similar to POSSUM/GALFACTS with a 300 MHz bandwidth from 1.1 to 1.4 GHz. We define three figures of merit motivated by the underlying science: a) an average RM weighted by polarized intensity, RMwtd, b) the separation $\Delta\phi$ of two Faraday components and c) the reduced chi-squared. Based on the current test data of signal to noise ratio of about 32, we find that: (1) When only one Faraday thin component is present, most methods perform as expected, with occasional failures where two components are incorrectly found; (2) For two Faraday thin components, QU-fitting routines perform the best, with errors close to the theoretical ones for RMwtd, but with significantly higher errors for $\Delta\phi$. All other methods including standard Faraday synthesis frequently identify only one component when $\Delta\phi$ is below or near the width of the Faraday point spread function; (3) No methods, as currently implemented, work well for Faraday thick components due to the narrow bandwidth; (4) There exist combinations of two Faraday components which produce a large range of acceptable fits and hence large uncertainties in the derived single RMs; in these cases, different RMs lead to the same Q, U behavior, so no method can recover a unique input model.