Robust direction-dependent gain-calibration of beam-modelling errors far from the target field

Many astronomical questions require deep, wide-field observations at low radio frequencies. Phased arrays like LOFAR and SKA-low are designed for this, but have inherently unstable element gains, leading to time, frequency and direction-dependent gain errors. Precise direction-dependent calibration of observations is therefore key to reaching the highest possible dynamic range. Many tools for direction-dependent calibration utilise sky and beam models to infer gains. However, these calibration tools struggle with precision calibration for relatively bright (e.g. A-team) sources far from the beam centre. Therefore, the point-spread-function of these sources can potentially obscure a faint signal of interest. We show that, and why, the assumption of a smooth gain solution per station fails for realistic radio interferometers, and how this affects gain-calibration results. Subsequently, we introduce an improvement for smooth spectral gain constraints for direction-dependent gain-calibration algorithms, in which the level of regularisation is weighted by the expected station response to the sky model. We test this method using direction-dependent calibration method DDECal and physically-motivated beam modelling errors for LOFAR-HBA stations. The new method outperforms the standard method for various calibration settings near nulls in the beam, and matches the standard inverse-variance-weighted method's performance for the remainder of the data. The proposed method is especially effective for short baselines, both in visibility and image space. Improved direction-dependent gain-calibration is critical for future high-precision SKA-low observations, where higher sensitivity, increased antenna beam complexity, and mutual coupling call for better off-axis source subtraction, which may not be achieved through improved beam models alone.