Converting the signal-recycling cavity into an unstable optomechanical filter to enhance the detection bandwidth of gravitational-wave detectors

Current and future interferometeric gravitational-wave detectors are limited predominantly by shot noise at high frequencies. Shot noise is reduced by introducing arm cavities and signal recycling, however, there exists a tradeoff between the peak sensitivity and bandwidth. This comes from the accumulated phase of signal sidebands when propagating inside the arm cavities. One idea is to cancel such a phase by introducing an unstable optomechanical filter. The original design proposed in [Phys.~Rev.~Lett.~{\bf 115},~211104 (2015)] requires an additional optomechanical filter coupled externally to the main interferometer. Here we consider a simplified design that converts the signal-recycling cavity itself into the unstable filter by using one mirror as a high-frequency mechanical oscillator and introducing an additional pump laser. However, the enhancement in bandwidth of this new design is less than the original design given the same set of optical parameters. The peak sensitivity improvement factor depends on the arm length, the signal-recycling cavity length, and the final detector bandwidth. For a 4~km interferometer, if the final detector bandwidth is around 2~kHz, with a 20~m signal-recycling cavity, the shot noise can be reduced by 10 decibels, in addition to the improvement introduced by squeezed light injection. We also find that the thermal noise of the mechanical oscillator is enhanced at low frequencies relative to the vacuum noise, while having a flat spectrum at high frequencies.