Constraints on sub-GeV hidden sector gauge bosons from a search for heavy neutrino decays

Several models of dark matter motivate the concept of hidden sectors consisting of SU(3)_C x SU(2)_L x U(1)_Y singlet fields. The interaction between our and hidden matter could be transmitted by new abelian U'(1) gauge bosons A' mixing with ordinary photons. If such A's with the mass in the sub-GeV range exist, they would be produced through mixing with photons emitted in two photon decays of \eta,\eta' neutral mesons generated by the high energy proton beam in a neutrino target. The A's would then penetrate the downstream shielding and be observed in a neutrino detector via their A'-> e+e- decays. Using bounds from the CHARM neutrino experiment at CERN that searched for an excess of e+e- pairs from heavy neutrino decays, the area excluding the \gamma - A' mixing range 10^{-7} < \epsilon < 10^{-4} for the A' mass region 1 < M_A' <500 MeV is derived. The obtained results are also used to constrain models, where a new gauge boson X interacts with quarks and leptons. New upper limits on the branching ratio as small as Br(\eta -> \gamma X) < 10^{-14} and Br(\eta' -> \gamma X) < 10^{-12} are obtained, which are several orders of magnitude more restrictive than the previous bounds from the Crystal Barrel experiment.