Invariants of nested Hilbert and Quot schemes on surfaces

Let $(S,p)$ be a smooth pointed surface. In the first part of this paper we study motivic invariants of punctual nested Hilbert schemes attached to $(S,p)$ using the Hilbert-Samuel stratification. We compute two infinite families of motivic classes of punctual nested Hilbert schemes, corresponding to nestings of the form $(2,n)$ and $(3,n)$. As a consequence, we are able to give a lower bound for the number of irreducible components of $S_p^{[2,n]}$ and $S_p^{[3,n]}$. In the second part of this paper we characterise completely the generating series of Euler characteristics of all nested Hilbert and Quot schemes. This is achieved via a novel technique, involving differential operators modelled on the enumerative problem, which we introduce. From this analysis, we deduce that in the Hilbert scheme case the generating series is the product of a rational function by the celebrated Euler's product formula counting integer partitions. In higher rank, we derive functional equations relating the nested Quot scheme generating series to the rank one series, corresponding to nested Hilbert schemes.