Modular vector bundles with and without moduli

If $X\subset\operatorname{Gr}(2,6)$ is the Fano variety of lines of a smooth cubic fourfold, then we show that the restriction to $X$ of any Schur functor of the tautological quotient bundle is modular and slope polystable. Moreover it is atomic if and only if it is rigid, in which case it is also slope stable. We further compute the Ext-groups of such bundles in infinitely many cases, showing in particular the existence of new modular vector bundles on manifolds of type $\operatorname{K3}^{[2]}$ that are slope stable and whose $\operatorname{Ext}^1$-group is 40-dimensional.