A Wide and Deep Exploration of Radio Galaxies with Subaru HSC (WERGS). X. The Massive and Passive Nature of Radio Galaxies at $z \sim 4$

High-$z$ radio galaxies (HzRGs) are considered important objects for understanding the formation and evolution of massive galaxies in the early universe. However, till date, detailed studies of the stellar population of HzRGs such as the star-formation history have been scarce. Therefore, this study conducted a new survey to establish a less-biased sample of HzRGs and consequently investigate their properties. We utilized a sample of $g$-dropout Lyman break galaxies (LBGs) obtained from an optical wide and deep imaging survey made by Subaru Hyper Suprime-Cam (HSC). Based on the cross-matching of this LBG sample with the VLA FIRST radio survey data, we constructed a photometric sample of high-redshift radio galaxies (HzRGs) at $z \sim 4$ for $\sim$560 deg$^2$ survey field. Consequently, we identified 146 HzRG candidates. To analyze the characteristics of these candidates, we focus on objects exhibiting the near-infrared photometry of VIKING or UKIDSS and the mid-infrared photometry of unWISE (28 objects). The results indicate that 7 objects exhibit SEDs consistent with galaxies at $z \sim 4$. The HzRG candidates have very large stellar masses with $\sim 4.2 \times 10^{11} M_{\odot}$ on average. This stellar mass is similar to that of previously discovered USS HzRGs at $z \sim 4$, though our sample is affected by a sample selection bias that selects only HzRGs with $M_{\star} > 10^{11} M_{\odot}$. Further, the SEDs of those HzRG candidates suggest a past fast quenching with a rough timescale of $\sim$0.1 Gyr, as evidenced from the rest-frame UVJ diagram.