Rest-frame UV--Optically Selected Galaxies at 2.3<z<3.5: Searching for Dusty Star-forming and Passively-Evolving Galaxies

A new set of color selection criteria (VJL) analogous with the BzK method is designed to select both star-forming galaxies (SFGs) and passively-evolving galaxies (PEGs) at 2.3<z<3.5 by using rest-frame UV--optical (V-J vs. J-L) colors. The criteria are thoroughly tested with theoretical stellar population synthesis models and real galaxies with spectroscopic redshifts to evaluate their efficiency and contamination. We apply the well-tested VJL criteria to the HST/WFC3 Early Release Science field and study the physical properties of selected galaxies. The redshift distribution of selected SFGs peaks at z~2.7, slightly lower than that of Lyman Break Galaxies at z~3. Comparing the observed mid-infrared fluxes of selected galaxies with the prediction of pure stellar emission, we find that our VJL method is effective at selecting massive dusty SFGs that are missed by the Lyman Break Technique. About half of the star formation in massive (M_{star}>10^{10}M_{Sun}) galaxies at 2.3<z<3.5 is contributed by dusty (extinction E(B-V)>0.4) SFGs, which however, only account for ~20% of the number density of massive SFGs. We also use the mid-infrared fluxes to clean our PEG sample, and find that galaxy size can be used as a secondary criterion to effectively eliminate the contamination of dusty SFGs. The redshift distribution of the cleaned PEG sample peaks at z~2.5. We find 6 PEG candidates at z>3 and discuss possible methods to distinguish them from dusty contamination. We conclude that at least part of our candidates are real PEGs at z~3, implying that this type of galaxies began to form their stars at z>5. We measure the integrated stellar mass density of PEGs at z~2.5 and set constraints on it at z>3. We find that the integrated stellar mass density grows by at least about factor of 10 in 1 Gyr at 3<z<5 and by another factor of 10 in next 3.5 Gyr (1<z<3).