The chi-y genera of relative Hilbert schemes for linear systems on Abelian and K3 surfaces

For an ample line bundle on an Abelian or K3 surface, minimal with respect to the polarization, the relative Hilbert scheme of points on the complete linear system is known to be smooth. We give an explicit expression in quasi-Jacobi forms for the chi-y genus of the restriction of the Hilbert scheme to a general linear subsystem. This generalizes a result of Yoshioka and Kawai for the complete linear system on the K3 surface, a result of Maulik, Pandharipande, and Thomas on the Euler characteristics of linear subsystems on the K3 surface, and a conjecture of the authors.