Higher order derivative moments of CUE characteristic polynomials and the Riemann zeta function

We use random matrix theory for the Circular Unitary Ensemble (CUE) to study moments of derivatives of the Riemann zeta function shifted a small distance from the critical line. The corresponding CUE moments are studied in the limit of large matrix size in two regimes: when the spectral parameter is (1) suitably far inside the unit disc, and (2) at a small distance from the unit circle. In case (1), we obtain an asymptotic formula as a combinatorial sum over contingency tables, while in case (2) we obtain a sum over certain determinants with multiplicative coefficients given by Kostka numbers. The latter result is also valid exactly on the unit circle. Then, we consider the analogous problem for mean values of derivatives of the zeta function with suitable shifts. Assuming the Lindel\"of hypothesis, we show that this mean value gives rise to the same sum over contingency tables obtained in the CUE. For sufficiently low-order moments, we establish this result unconditionally.